Scheduled crawls

Turning a crawl into monitoring

Scheduling is set per project, in the project’s settings. Pick a cadence:

• Hourly
• Daily
• Weekdays (Mon–Fri)
• Weekly
• Custom — pick exactly which days of the week

…then a time of day and a timezone. Scheduled runs fire at the top of the chosen hour.

Scheduling is included on every paid plan (Pro and up). See Plans.

What happens on each run

A scheduled crawl is a normal crawl: it produces a full report in the project’s history and spends credits like any other crawl. That’s the whole point — you get a continuous, comparable record of the site over time, and the Compare tab tells you what changed between any two runs.

Regression alerts

With notifications enabled on the project, we check each scheduled crawl against the previous one and alert you — by email and, if configured, a webhook — when something regressed:

• the Health score dropped meaningfully,
• new error-severity issues appeared, or
• one of your custom rules started failing.

No regression, no email. We don’t send “your crawl finished” noise.

Budgeting credits

A quick rule of thumb: pages per crawl × runs per month = credits needed. A 3,000-page site crawled daily is ~90,000 credits a month — inside Pro’s 100,000. The usage bar in Settings → Plan & billing shows where you stand mid-month.