Oncrawl: the crawler and log analyzer for large-scale websites

Good tool

Oncrawl crawls a website the way Googlebot would, page after page, then compares what it found with the server’s log files, which show what Google’s and AI companies’ bots actually visit. SEOs use it to spot orphan pages, pages buried too deep, slow pages or ignored ones, and can enrich that data with Search Console, Google Analytics or their own sales figures. Built for sites with hundreds of thousands or millions of URLs, the platform is one of the technical SEO tools found in large companies, alongside Botify and Lumar (formerly DeepCrawl).

Visit website

The Oncrawl homepage, with a preview of its crawl analysis dashboards (screenshot from October 2026).

What is Oncrawl?

Oncrawl is online software (SaaS): nothing to install, since crawls run on the company’s servers rather than on the user’s computer. You set up a project, configure a crawl profile (scope, speed, JavaScript rendering, frequency), and the bot collects hundreds of metrics per page: status codes, tags, duplicate content, internal linking, depth, load time, Core Web Vitals. Results are displayed in dashboards you can slice into segments, such as product categories or page templates, and two successive crawls can be compared to catch a regression after a release.

The second pillar is log analysis. Connected to Cloudflare, AWS, Google Cloud, Azure or plain log files, Oncrawl records, without sampling, every visit from Googlebot, Bing and now the bots of OpenAI (GPTBot, OAI-SearchBot), Perplexity, Anthropic (ClaudeBot) or Mistral. Gemini and Google’s AI Overviews, on the other hand, feed on what Googlebot already crawls: no separate visit shows up in the logs, and the Google-Extended token in robots.txt only lets you opt out of that use. By cross-referencing those visits with the crawl, you can measure crawl budget wasted on useless pages and the delay between a page’s publication and its first organic visit. Add-on modules, called “Lenses,” steer the analysis toward visibility in AI search engines or content quality, and an API, an in-house query language and an MCP server let you put the data to work elsewhere.

The tool was launched in 2013 by François Goube and Tanguy Moal, initially to solve the crawling problems of a large French e-commerce site. It is published by Cogniteev, a company based in Mérignac, near Bordeaux, where the team works. In February 2022, BrightEdge, the US specialist in enterprise SEO platforms, bought out the investors; Oncrawl kept its brand, its offering and its team within the group. At the time, it reported nearly 50 employees and more than a thousand customers.

Oncrawl at a glance

Type: Technical SEO SaaS platform
Founded: 2013
Founders: François Goube and Tanguy Moal
Origin: France
Headquarters: Mérignac, near Bordeaux (Cogniteev SAS)
Owner: BrightEdge (United States), since 2022
Free plan: No, and no self-serve trial
Starting price: Custom quote, after a demo

What is Oncrawl used for?

Crawl your site at scale
millions of URLs crawled in the cloud, with JavaScript rendering and no artificial limit on speed or depth.
Read your server logs
every visit from Googlebot and other bots, recorded without sampling, so you know which pages are actually crawled.
Audit technical health
status codes, indexability, duplicate content, internal linking and page depth, broken down by segments of your site.
Measure performance
load time and Core Web Vitals measured page by page during the crawl, in lab conditions: values to compare with the field data in Search Console, which is what Google uses.
Track AI bots
visits from GPTBot, PerplexityBot, ClaudeBot and other AI bots to your pages, and the pages that can be cited in their answers.
Get alerted to regressions
scheduled crawls, crawl-to-crawl comparison and alerts when a metric changes after a release.
Combine your data
Search Console, GA4, Majestic, Piano or your own sales figures, plus an API to send the results back to your BI tools.

Oncrawl review

Good tool

Oncrawl isn’t the tool you open for a quick audit of a small business website. Its reason for being is sites too big or too complex for a desktop crawler: e-commerce catalogs, media sites, classifieds, multilingual platforms. On that ground, it delivers. Crawls of several million pages go through without a hitch, segmentation lets you read a site in large sections rather than page by page, and combining crawl, logs and Search Console data gives you a view few tools offer as easily.

Its strongest suit is still log analysis. Logs aren’t an estimate: they show what Googlebot actually visited, and that is often where the gains hide on a large site (key pages never crawled, crawl budget swallowed by filters or URL parameters). AI bot tracking, added recently, relies on the same raw material: it tells you with certainty which pages GPTBot or ClaudeBot crawled, and when. It doesn’t tell you whether those pages are cited in answers, which is what prompt-based tools such as Otterly.AI or Meteoria measure, within the approximations of their method. The two complement each other. You do, however, need access to the server logs, which often means going through the tech team.

The flip side is a demanding platform. You need to know technical SEO to get anything out of its hundreds of metrics, and getting up to speed takes time, even though onboarding by the team is part of the package. The sales model is an enterprise one: no public pricing, a demo before any quote, and a contract that renews automatically and can be canceled only by registered letter. For an independent consultant or a small business, it is overkill.

Strengths

A cloud crawler that handles sites with several million pages, JavaScript included.
Complete log analysis, without sampling, AI bots included.
Crawl data combined with logs, Search Console, GA4 and your own data.
Flexible segmentation, applied to past analyses too.
An API, exports to BI tools and an MCP server to go further.
Support from a team of technical SEO specialists.

Weaknesses

No public pricing and no self-serve trial: everything goes through a demo.
A tool designed for large sites, overkill for a modest one.
A long learning curve that requires solid technical SEO knowledge.
Log analysis limited to the Core and Scale plans, and dependent on access to the server logs.
A contract that renews automatically, with a possible annual increase of up to 5%.

Is Oncrawl worth it?

Yes, if you manage SEO for a site with several hundred thousand pages, entire sections seem to be ignored by Google, or you are preparing a migration. Having crawl, logs and Search Console data at hand then lets you prioritize work based on facts and show its impact. It is also a good choice for a specialized agency handling large accounts.

Not necessarily, if your site has a few thousand pages. Screaming Frog, desktop software sold as an annual license, or Sitebulb covers most of a technical audit for far less, and Search Console gives you a free overview of how Google crawls your site. Among enterprise platforms, Botify and Lumar play in the same league as Oncrawl: compare the demos and quotes on your own site before committing.

For log analysis alone, Seolyzer, a French tool with a free plan, and Screaming Frog’s Log File Analyser, sold as a license, cover the needs of a mid-sized site. Between those two worlds, JetOctopus combines cloud crawling and logs at a public price well below that of enterprise platforms, and deserves a place in the comparison.

How much does Oncrawl cost?

Oncrawl doesn’t publish any pricing. Its website no longer has a pricing page: the only route is a demo, followed by a quote based on the size of the site, the crawl volume and the modules you choose. Its terms of service describe three plans, all quote-based, whose capacity is counted in monthly “crawl tokens” set in the contract.

Plan	Per month	Included	Who it's for
Basic	Custom pricing	Crawler, core features, Search Console, GA4 and API	Regular technical audits and crawls
Core	Custom pricing	Basic, plus log analysis and the other connectors (Majestic, Piano…)	SEO teams tracking how the site is actually crawled
Scale	Custom pricing	Crawler and log analysis for very large volumes	Very large sites and multi-site groups

Source: official Oncrawl pricing terms (plans described in the terms of service, no prices published), checked on October 31, 2026.

No free plan and no self-serve trial: access goes through a demo and a custom quote.
Paid options: annual crawl tokens for one-off crawls, the Content Lens module for AI-powered content analysis and SEO workshops with the team.
The contract renews automatically for the same term unless canceled by registered letter at least 30 days before it ends, and the price can rise by up to 5% each year.

FAQ

Answers to questions about Oncrawl

How much does Oncrawl cost?

Oncrawl doesn’t publish any prices. Its three plans, Basic, Core and Scale, are sold on a custom quote after a demo, based on the size of the site and the crawl volume you need.

Does Oncrawl have a free plan or a trial?

No. There is no free plan and no self-serve trial: everything starts with a demo request.

Who is behind Oncrawl?

Oncrawl was founded in 2013 by François Goube and Tanguy Moal. It is published by Cogniteev, a company based in Mérignac, near Bordeaux, which has been owned by the US company BrightEdge since 2022.

What is the difference between Oncrawl and Screaming Frog?

Screaming Frog is software installed on your computer, sold as a license and suited to most sites. Oncrawl runs in the cloud, handles sites with several million pages and combines the crawl with server logs and other data sources, for a much higher price.

Oncrawl or Botify?

Both platforms target sites with several hundred thousand pages, combine crawl and logs and are sold on quotes. Oncrawl stands out for its flexible segmentation, its query language and a team based in France. The smart move is to run both demos on your own site before signing.

Information checked on October 31, 2026