Flipkart Big Billion Sale Explore Part 1
Web Scraping data from Flipkart using Python for Big Billion deals
Today, the Internet is flooded with an enormous amount of data relative to what we had a decade ago. The amount of data we produce every day is truly mind-boggling.
Web Scraping is important because, irrespective of the industry, the web contains information that can provide actionable insights for businesses to gain an advantage over competitors.
Web scraping, also known as web data extraction, is the process of retrieving or “scraping” data from a website. This information is collected and then exported into a format that is more useful for the user and it can be a spreadsheet or an API. Although web scraping can be done manually, in most cases, automated tools are preferred when scraping web data as they can be less costly and work at a faster rate.
The simplest way is to check the robots.txt file of the website. You can find this file by appending “/robots.txt” to the URL that you want to scrape. It is usually at the website domain /robots.txt. If all the bots indicated by ‘user-agent: *’ are blocked/disallowed in the robots.txt file, then you’re not allowed to scrape. For this article, I am scraping the Flipkart website. So, to see the “robots.txt” file, the URL is www.flipkart.com/robots.txt
What are the Steps Involved in Web Scraping?
To fetch the data using Web Scraping using Python, we need to go through the following steps:
Find the URL that you want to scrape.
Inspecting the Page and website.
Find the ways to get data; some of them using API or JS etc.
Find the suitable and simple code approach.
Find the data you want to extract.
Run the code & extract the data.
Finally, Store the data in the required format.
Packages used for Web Scraping
Following python packages are used:
Pandas: Pandas is a library used for data manipulation and analysis. It is used to store the data in the desired format.
Requests-HTML: This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible.
Benefits of Data and Analytics as consumer for Flipkart.
1. Price of your products according to regular vs big billion days.
2. What customers select for what products; based on customer demands.
Simple enough, BENEFITS for your purchases regular vs big billion days; save every penny as well.
Whereas client or competitor;
1. You can price your products according to demand and by what customers expect.
2. Manage your inventory better by anticipating customer demands.
3. Customers can select from a combination of features or service that meets their needs.
4. Knowing what products will sell when and where allows manufacturing efforts to anticipate demand and prevent a loss of sales.
5. More competitive prices, reduced costs, and higher sales are results of targeted marketing efforts.
Simple enough, REVENUE is your top line and you’re probably tracking this for your accountant already.
I have selected few categories and will keep update what will benefits after big billion days.
This is purely as an educational purpose; would like to know any clarification on the same. It just 90+ line of code to expelled data as well. I have not shared any code and data.
