Scalable Crawling?
ScalableCrawling.com is our sister company, and allows our clients to crawl using enterprise level hardware, all for a set fee and with unlimited data usage and no time contraints.
Where are your servers?
Our servers based in a UK data centre that is patrolled 24/hr’s a day and has temperature management and fire safety. All servers are backed up regularly, and come with 1TB’s worth of cloud storage so you can easily share files with your local machine or a client/team-member.
Why did you build Scalable?
Simply put, we wanted to crawl very large websites, often - so we built the solution for ourselves. Crawling platforms in SEO sit as either a DIY piece of software that lives on your computer or an enterprise grade SAAS that quickly becomes very expensive. Scalable is capabale of crawling millions of pages, exporting the data and delivering the same insights for a fraction of the costs.
How does Scalable support Canonicals?
All of our clients are added to our dedicated server, which is setup to support with both large site crawling and crawling multiple sites concurrently. This means we can have an always-on technical health monitoring solution that costs our clients nothing, finding and highlighting issues before they begin hitting the business’ bottom line.
Can I rent my own box?
Of course, we offer both pre-configured and bespoke boxes over on scalablecrawling.com. Every box is rented on a monthly rolling agreement, and prices are locked for the lifetime of the contract - so no price raises.
We’re also working on a library of tools that all users will gain access to for free. This will empower you to for example run Screaming Frog using a MCP and a webpage, work with the data once exported using AI, and more more cool features.