Run your first crawl

Start a crawl

Click New report in the top bar (it’s available from every page). Pick one of three modes:

Mode	What it does
Whole site	Starts from your URL and follows internal links until it has seen every page (up to your limits).
Single page	Audits exactly one URL — handy for checking a page you just shipped.
URL list	Paste a list of URLs and we audit exactly those, with no link-following.

Enter your URL and hit start. That’s genuinely all a first crawl needs — the defaults are sensible. When you’re ready to tune things, the Advanced options cover max pages, crawl depth, external-link verification, robots.txt, and JavaScript rendering — see Crawl settings.

Sites become projects

When you crawl a whole site, Crawlie either attaches the crawl to the matching project (if you’ve crawled that site before), offers to create one, or runs it as a one-off. Projects are how you get crawl history, comparisons, and scheduled monitoring — worth creating one for any site you care about.

While it runs

Crawls run on our infrastructure, not in your browser tab:

• Close the tab whenever you like — the crawl keeps going and the finished report will be waiting in Reports.
• Running crawls show live in the sidebar and the Reports list, and re-attach if you reload the page.
• Cancel a crawl at any point and we save a report of everything crawled so far.

Limits to know about

• Each page crawled costs one crawl credit. Hobby includes 5,000 credits a month.
• Each plan caps pages per crawl (1,000 on Hobby). If a site is bigger than your cap, we crawl up to the cap and tell you the report was truncated. See Plans.

Next steps

• Reading a report
• Scheduled crawls