Hosted Web Scraper Online
July, 1st 2016. We've launched the online version of Data Scraping Studio, named as hosted app with same capabilities and power of cloud with freedom to choose any programming language to mange your account using REST API
Using the application you can..
- Create and host your scraping agents online
- Enter URL manually or with advanced list option to create a list of URLs to mange easily
- Start and schedule your scraping agent on the go
- Get email when extraction completes or configure a webhook to post data on server when ready.
- And many more...
Creating agent for cloud hosted scraping app is same as for desktop app using chrome extension, you can enable the extension in Chrome and click on the HTML elements to generate CSS selectors automatically and then click on done button as explained in this tutorial , the below dialog box will appear.
All scraping agents(*.scraping) file works on both desktop app and the hosted app. If you want you can upload/download the agent file from cloud app after login as well.
Enter the API Id and Admin API Key in text boxes under "Send to Cloud Hosted App" and click on the "Save" button, the agent will be created in your online account. (If you don't have the API id and key, you can get one by logging in and then go to the account section)
The API Id and Key is stored in your chrome local storage when you enter first time to remember in future. If you want to change any time later, just paste again, or the same will be used forever.
Once the agent is created, you can click on the link in success message or go to dashboard to visit the agent page for start, schedule and further configuration.
The hosted app provides flexible scheduling capabilities for your agents to run hourly, daily, weekly or any particular day(s). In order to schedule a agent, go the scraping agent page and click on the "Schedule" button. It will display the below dialog box.
Now go to any tab you want to schedule your agent accordingly to select your choice frequency and save to finish. For example. I've selected "Weekly" scheduler option and checked on "Monday, Thursday and Sunday" with start time 05:03 will fire the agent on these particular days at the mentioned time.
All the time in scheduler and all other places in DataScraping.co are in IST(Indian Standard Time) + 05:30
A particular agent can have only one schedule and each time you click on the "Save" button, the existing schedule will be overwritten if any. If you want to delete the schedule, clicking on the "Reset" button will remove the schedule permanently.
Each online scraper hosted app account has five levels of access available for users: Owner, Admin, Manager and Viewer. Both the Owner and Admin can invite other users to join, and they can assign/change user roles.
- Can only access reports and data.
- Has full access of account except for billing, user management, Re-generate API key.
- Account user with full access.
- Owner : Account user with full access.
The "Owner" role is by default selected by DataScraping.co when a user signup on website and can't be changed using the sub-accounts management feature.