Guides
Help AI find your pages: sitemap, canonical, Link headers
No special sitemap for AI crawlers exists. Use sitemap.xml, self-referencing canonicals, and RFC 8288 Link headers, then verify with a free 0–100 scan.
Jun 12, 2026
AI agent readiness guides
Practical guides on getting your site read, understood, and cited by AI assistants and agents — llms.txt, AI-crawler controls, structured data, and how agent readiness differs from classic SEO.
Guides
Serve Markdown to AI agents: content negotiation done right
One Cloudflare demo page fell from 12,345 HTML tokens to 725 as Markdown. Serve it to AI agents with Accept: text/markdown, the right Content-Type, and Vary.
Jun 7, 2026
Guides
Structured data for AI agents: schema.org assistants use
Schema.org and JSON-LD don't force AI to cite you, but they hand ChatGPT, Perplexity, and Google AI machine-readable facts. Types, errors, free 0–100 scan.
Jun 6, 2026
Guides
Control AI crawlers in robots.txt: GPTBot, ClaudeBot & more
Exact robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot. Copy-paste allow, block, and selective snippets, plus a free 0–100 scan.
Jun 5, 2026
Guides
How to make a JavaScript site readable to ChatGPT and AI
Most AI crawlers fetch raw HTML and don't run JavaScript. Test what bots see, then ship content ChatGPT, Perplexity, and Claude can read. Free 0–100 scan.
Jun 4, 2026
Concepts
What is AI agent readiness, and how does it differ from SEO?
AI agent readiness measures how easily AI assistants and agents can reach, read, and act on your site. See how it differs from SEO and how the 0–100 score works.
Jun 3, 2026
Guides
What is llms.txt, and how do you add one?
llms.txt maps your key pages for AI models, yet only ~10% of 300k domains have one. Learn its format, where to put it, and how it differs from robots.txt.
Jun 3, 2026