Scheduling Data Crawlers

Written by on July 28, 2013

An updated version of this article is available here.

In this video we will show you how to schedule your data crawlers to run automatically.

– On your project page, click on the Scheduling tab.
– Here you can schedule the start date and time for the crawler to run.
– Choose how often the crawler will run. Depending on your user level, you can run it daily, weekly or monthly.
– You then need to chose a date until when the crawler stays active.

In some cases, you may also want to run the crawler on demand.
You can do this by clicking on the “Run Now” button. Your crawler will then start running in a few minutes.
This feature may not be available to all users. Please contact support to enable this feature.

View more blog posts

Get in touch to start your scraping project
Sign up and create your first project to get started. It takes less than 2 minutes!