Creating a web scraping agent with chrome extension
Data Scraping Studio chrome extension known as "Advanced Web Scraper" is a point and click interface to create scraping agent using CSS selectors in few minutes.
Just install the extension and go to any website and launch it, a sidebar will open on the website right side. Click on the + New button to add a new field and then click on the asterisk(*) button to enable selector for that field. Now click on a page element that would like to extract(it will turn green), then the web scraping app will generate the best CSS selector for that element and will also highlighted(yellow) other matching element by the selector.
You can click on the highlighted(yellow) element to remove from the selector and click on un-highlighted element to add to the selector. Through this process of selection & rejection, web scraping app will help you to find the best selector of item need to be extracted.
text option from drop down to see the instant result preview of text for all the elements matching with selector.
HTML option from drop down to see the instant result preview of HTML for all the element matching with selector.
The ATTR method returns the attributes value of the selected elements, just select the
ATTR option from drop down and write the name of attribute in next input field to see the instant result preview of extracted attribute for all the element matching with selector.
Using the ATTR method you can extract anything from a HTML tag e.g URL from a link, image source(src), content from a meta tag, HTML 5 data attribute and more....
Custom CSS selector
You can also write your custom CSS selector and click on Accept button to save. And web scraping app will show you instant result preview of custom CSS selector as well extract the matching result.
Extract Hidden element
Since hidden element are not visible on a web page - so you cannot point and click. But can write the custom CSS selector for hidden element and extract anything you want from the webpage. For e.g in screenshot below, i've extracted META tag description
<meta name="description" content="Amazon Coupons, Discounts, Coupon Codes & Deals. Get free Amazon discount vouchers to save money on your Amazon purchase. 10 coupons and free shipping." />
Click on the menu icon on top to see the output preview. The current page output can also be downloaded in popular file format such as JSON, CSV, TSV.
Now we are done with setting-up CSS selectors for fields needs to be extracted and our scraping agent is almost ready, we can download the agent to execute with desktop software or hosted app to with advanced web scraping features to run the scraping agent online.
- Click on Done button to save(download) the agent locally.
- The scraping agent will be downloaded in “Downloads” folder in your user account with
- Go to the folder and double click to execute in desktop app.
- And(below screenshot) of Desktop app.
Related Q & A