Discover how search engines actually crawl your website. Analyze server logs, Screaming Frog exports, and crawl data to identify technical SEO issues, wasted crawl budget, orphan URLs, and indexing opportunities.
How it works
This is a free, in-browser tool for technical SEO. Nothing you upload ever leaves your computer. All the analysis runs locally in your own browser, so it is safe to use with client data. You can load a Screaming Frog crawl, a raw server log, or both together to compare what you want search engines to find against what they are actually crawling.
Get started in three steps
Export your data
From Screaming Frog, use Internal → All and export to CSV. For a server log, download the raw access log from your hosting (Apache or Nginx). Both .csv, .tsv, .log, .txt and gzipped .gz files are accepted.
Drop the file in
Drag your file onto the upload box above, or click Choose files. The tool auto-detects the format. Load a crawl and a log together to compare them, or two logs to merge their date ranges. In a hurry? Click Try sample log to explore with demo data first.
Read the report
Switch between the tabs to see bot verification, crawl frequency, status-code errors, orphan pages, oversized URLs, and a plain-English AI summary. Use the findings to fix waste and indexing gaps.
What can it tell you?
If you upload a Screaming Frog crawl, you will see status codes, redirect chains, indexability problems, orphan pages (pages with no internal links), and large or slow URLs. If you upload a server access log, you will see which bots are really visiting, how often they crawl, whether they are genuine (bot verification), which IPs are most active, and where crawl budget is being wasted. Upload both and the tool compares them, so you can spot pages search engines crawl that you do not care about, and important pages they are ignoring.
Glossary
Events
An “event” is a single line in your server log: one request for one resource. Every time a browser or a bot asks your server for a page, image, or file, that is one event. The total event count tells you how much activity (human and bot) your site is handling over the log’s time period.
URLs
A URL is the web address of a single page or file (for example /blog/seo-guide/). Many separate events can point at the same URL if it is requested repeatedly. Looking at unique URLs shows how many distinct pages were touched, while events show how many times in total.
Response codes
Every request gets a 3-digit status code back from the server. 2xx means success, 3xx is a redirect, 4xx means not found or forbidden (like a 404), and 5xx is a server error. A healthy site is mostly 2xx; lots of 4xx or 5xx codes signal broken links or server problems wasting crawl budget.