Free tools
Small tools that answer one question properly. Each one is built on something measured here rather than wrapping a public API, which is why there are so few of them: a tool worth publishing has to tell you something you could not get from the ten free versions of it that already exist.
-
Robots.txt checker
Whether a crawler you named in robots.txt can now reach paths you block for everyone else. Paste a domain and it resolves the file the way a crawler does.
What it is built on: The 5,000-domain census behind /research/robots-defect. The tool reports your file against that benchmark, so copying it means re-running the study.
No signup, no email, nothing stored. If a tool here cannot answer your question honestly it says so rather than guessing. The research is where the measurements behind these come from, and what I will not sell you sets out the evidence standard all of it is written to.