Product
Introducing Crawlie Cloud
· by Sean Ryan
Crawlie started as a free, open-source crawler you run on your own machine — no login, no seat licenses, and your data never leaves your laptop. None of that is changing. But some jobs were never a good fit for a laptop, and we kept running into the same ones: the 200,000-page crawl that needs to run overnight, the weekly re-crawl someone has to remember to kick off, the report a client can’t open because they don’t have the app installed.
Today we’re launching Crawlie Cloud — the same audit engine, running on our infrastructure, for the jobs that outgrew your desk.
Crawls that run without you
Start a crawl, close the tab. Cloud crawls run as background jobs on our machines, so a big site keeps crawling whether you’re watching or not. Check progress from your phone, or just come back to a finished report. No install, no laptop pinned open for an hour, no dead battery taking your crawl down at 80%.
Monitoring, not just audits
A one-off audit tells you where you stand today. Crawlie Cloud re-crawls on a schedule — daily, weekly, or monthly — and alerts you by email or webhook the moment something moves: a batch of new broken links, a canonical that flipped, an accessibility regression that shipped on Tuesday. You find out before your visitors do, and long before your client does.
Built for teams
Reports live in a shared workspace instead of on one person’s hard drive. Everyone sees the same history, the same saved reports, the same trend lines. And we price by pages crawled, not seats — inviting your whole team costs nothing extra.
An endpoint for your agents
Crawlie Cloud ships a hosted MCP endpoint at crawlie.app/mcp. One API key gives any MCP-compatible agent the full crawler over the network: crawl_site, audit_url, top_fixes, geo_gaps, diff_reports, and the rest. Agents can kick off a crawl, poll long-running ones with crawl_status, and re-slice saved reports later with get_report — without paying to crawl again. Same tools as the local MCP server, with none of the setup.
Still open source
This is the part we care about most, so let’s be precise about it.
The engine, the CLI, the MCP server, and the desktop app are all MIT-licensed, and they’re staying that way. The code running Crawlie Cloud is the same code sitting in the open repo — pull requests welcome. We keep the hosting and billing plumbing in a separate codebase so the open repo stays clean and easy to contribute to.
Crawlie Cloud isn’t a wall around the engine. Everything that crawls and scores your site is open source — the cloud is just the easiest way to keep it running every day, with uptime, storage, and alerting handled for you. If you’d rather run the crawler yourself and keep everything local, that will always work, and it will always be free.
What’s next
This is the starting line, not the finish. Coming up:
- Own your index. Your site the way search and answer engines actually see it, saved over time — so the history belongs to you, not to a dashboard you rent.
- Smarter alerts. Not just “the score dropped,” but which pages, which checks, and the likely cause, right there in the email.
- More for agents. The MCP endpoint is step one of treating the crawler as infrastructure your agentic tools build on, not just a report you skim.
We’ll keep shipping the way we always have: small releases, often, in the open.
Try it
Create a free account and run your first cloud crawl in a couple of clicks — the free tier includes 5,000 crawl credits a month, no credit card required. It’s the same login as the CLI and desktop app, and the engine is still yours to run anywhere. If you hit something rough, tell us — we genuinely want the bug reports, and we fix things fast.