SEO Tool

AI Markdown Checker

Test Cloudflare Markdown for Agents & AI SEO

Analyze how AI crawlers interact with your website. Check Markdown for Agents support, AI meta tags (noai/noimageai), robots.txt rules for AI bots, Content-Signal headers, and token usage.

Complete AI Audit

Test Cloudflare Markdown for Agents, meta tags, robots.txt rules, Content-Signal headers, and token usage — all in one scan.

All Major AI Bots

Tests against GPTBot, ChatGPT-User, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Bytespider, Amazonbot, and more.

100% Free & Private

No registration required. We never store your URLs or results. Check as many sites as you want.

Understanding Cloudflare Markdown for Agents & AI SEO

As AI systems like ChatGPT, Claude, Gemini, and Perplexity increasingly crawl the web, a new challenge has emerged: how to serve content efficiently to AI agents while maintaining control over how your content is used. Cloudflare's Markdown for Agents is at the forefront of this evolution.

What is Markdown for Agents?

Markdown for Agents is a feature introduced by Cloudflare in their AI Audit suite (rolled out fully in early 2026). When an AI bot requests a page with an Accept: text/markdown header, Cloudflare automatically converts the heavy HTML into clean, compact Markdown. This reduces token usage by up to 90%, saving significant costs for AI developers while providing cleaner content for AI processing. The conversion happens on-the-fly at Cloudflare's edge network, requiring zero changes to your website's code.

How Does This Checker Work?

Our tool performs a comprehensive analysis of your website's AI compatibility. It sends a request with the Accept: text/markdown header to test for Cloudflare's Markdown for Agents support. It parses your HTML for AI-specific meta directives like 'noai' and 'noimageai'. It fetches your robots.txt to check rules for AI bot user-agents like GPTBot, ClaudeBot, and Google-Extended. It also inspects response headers for Content-Signal directives and the x-markdown-tokens header that reports estimated token usage.

Why Check Your AI Markdown Compatibility?

  • Reduce AI bot bandwidth — Markdown responses use up to 90% fewer tokens than HTML
  • Verify your AI content policies — ensure noai/noimageai meta tags are correctly implemented
  • Check Content-Signal headers — see if your site communicates training permissions (ai-train=yes/no)
  • Audit AI bot access — confirm which AI crawlers can access your content via robots.txt rules
  • Optimize caching — verify the Vary: Accept header ensures proper caching of Markdown vs HTML versions

Privacy First

We only fetch the URL you enter to perform the checks. No data is stored, no cookies are set, and no tracking occurs. Your results are generated in real-time and never saved.

Frequently Asked Questions

What is Cloudflare Markdown for Agents?
Cloudflare Markdown for Agents is a feature that automatically converts HTML pages to compact Markdown when an AI bot requests content with the Accept: text/markdown header. This reduces token usage by up to 90% and provides cleaner content for AI processing. Sites using Cloudflare can enable this feature with a single toggle in their dashboard under the AI Audit section.
What are Content-Signal headers?
Content-Signal headers are HTTP response headers introduced by Cloudflare that communicate what AI bots are allowed to do with your content. For example, 'Content-Signal: ai-train=yes' explicitly permits AI training, while 'ai-train=no' prohibits it. These headers provide a machine-readable way to express content usage policies beyond what robots.txt offers.
What do 'noai' and 'noimageai' meta tags do?
The 'noai' meta tag tells AI crawlers that your content should not be used for training AI models. The 'noimageai' tag specifically prevents images from being used in AI training. These are placed in your HTML head section as: <meta name='robots' content='noai, noimageai'>. These tags are an emerging standard supported by major AI companies.
What is the x-markdown-tokens header?
When Cloudflare serves a Markdown version of a page, it can include the x-markdown-tokens header that reports the estimated number of tokens the page content contains. This helps AI developers understand bandwidth and cost implications. A typical web page might use 5,000-20,000 tokens in HTML but only 500-2,000 tokens in Markdown.
Which AI bots does this tool check for?
We check for all major AI crawlers: GPTBot and ChatGPT-User (OpenAI), Google-Extended (Gemini), ClaudeBot (Anthropic), CCBot (Common Crawl), PerplexityBot, Amazonbot, Bytespider (ByteDance), FacebookBot (Meta), and Applebot-Extended (Apple Intelligence).
Why is the Vary: Accept header important?
The Vary: Accept header tells caches (CDNs, proxies, browsers) that the response varies based on the Accept header. Without it, a cached Markdown version might be served to a regular browser, or vice versa. Cloudflare automatically sets this header when Markdown for Agents is enabled, ensuring proper content negotiation.