Guides
Long-form reference material, written in chapters. A guide answers a question completely rather than describing a service, and every factual claim in one is cited to the vendor documentation it came from so you can check it rather than take my word for it.
-
4 chapters
AI crawlers, end to end
Controlling AI crawlers means deciding separately about training and retrieval, then writing robots.txt so the rules reach the bots you named. Most sites get the second part wrong: adding a group for a specific crawler removes the wildcard rules from that crawler entirely, so directives you thought applied everywhere quietly stop applying to the one bot you were trying to control.
Research publishes the studies these guides draw on, with the method and the data attached. The blog carries shorter pieces, and what I will not sell you sets out the positions these guides are written from.