Enter any domain and live-fetch their actual llms.txt. Validates format, checks every major AI bot declaration, and flags conflicts with robots.txt — for your own site or any competitor.
llms.txt is an emerging specification, analogous to robots.txt, that gives website owners a structured way to declare their content's availability for AI training, retrieval, and citation. Where robots.txt controls crawl access at the page level, llms.txt operates at the intent level — telling AI systems what the site is, what it covers, and how its content should be used.
The file lives at the root of your domain, is plain text, and follows a simple key-value format. A well-formed llms.txt includes the site name, a description of what the site covers, contact information, and per-bot Allow or Disallow declarations. Engines that support llms.txt — including GPTBot and PerplexityBot — check for the file before indexing content.
Sites with a correctly formatted llms.txt consistently show higher citation rates than comparable sites without one — the signal it sends about technical AEO maturity compounds over time. Generate one free with the llms.txt Generator.
llms.txt says Allow: GPTBot but robots.txt has Disallow: / for all agents.
Fix: add explicit User-agent: GPTBot / Allow: / block in robots.txt above the wildcard rule.
Many SEO professionals assume that allowing AI bots in robots.txt is sufficient. It isn't. llms.txt and robots.txt serve different purposes and both are needed for a complete AI visibility configuration.