Is example.com blocking ChatGPT?

siteiz · check_ai_crawlers

  • OAI-SearchBotallowed
  • ChatGPT-Userallowed
  • GPTBotblocked

No. ChatGPT search can still index and cite the site. Only GPTBot, the training crawler, is blocked.

Illustrative example.

Tools

Four tools, all read-only

check_ai_crawlers

Check AI crawler access

Reads a website's robots.txt and reports, for 11 AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, CCBot, Bytespider), whether each may reach the given page. Separates AI search crawlers (decide if the site appears in ChatGPT, Claude and Perplexity answers) from training crawlers. Also reports llms.txt and declared sitemaps.

view_page_as_ai_crawler

View a page as an AI crawler

Fetches one page the way most AI crawlers do (plain HTTP, no JavaScript) and reports what they actually receive: title, meta description, canonical, H1, heading count, structured data types, visible word count, token reading cost, and a text preview. Use it to check whether content is visible to AI without JavaScript.

explain_ai_crawler

Explain an AI crawler

Explains what an AI crawler user agent does (operator, purpose, and what blocking it in robots.txt changes). Pass a name such as GPTBot or OAI-SearchBot, or omit it to list all crawlers Siteiz tracks.

get_ai_visibility_report

Get a published AI visibility report

Returns a published, dated Siteiz AI visibility report for a well-known company's homepage (score, grade, pillar scores, top issues). Omit the company to list all published reports.

Install

Connect it in a minute

Remote server over Streamable HTTP. Works with any MCP client that supports remote servers.

Claude (claude.ai and desktop)

Settings, Connectors, Add custom connector, then paste the server URL.

https://siteiz.com/mcp

Claude Code

Run once in your terminal.

claude mcp add --transport http siteiz https://siteiz.com/mcp

Cursor

Add to ~/.cursor/mcp.json.

{
  "mcpServers": {
    "siteiz": { "url": "https://siteiz.com/mcp" }
  }
}

VS Code

Add to .vscode/mcp.json.

{
  "servers": {
    "siteiz": { "type": "http", "url": "https://siteiz.com/mcp" }
  }
}

Try asking

Prompts that call Siteiz

  • “Is nytimes.com blocking ChatGPT search or only GPTBot?”
  • “What does an AI crawler actually see on my homepage without JavaScript?”
  • “What is the difference between OAI-SearchBot and GPTBot?”
  • “How does Klarna's homepage score for AI visibility?”

FAQ

MCP server questions

What is the Siteiz MCP server?

A remote Model Context Protocol server that gives AI assistants four read-only tools for AI visibility: check which AI crawlers a site's robots.txt allows, view a page the way an AI crawler receives it, explain any AI crawler, and look up published Siteiz reports.

Is it free? Do I need an account or API key?

It is free, with no account and no API key. Tools that fetch a website have a fair-use rate limit per connection.

Does it change anything on my site?

No. Every tool is read-only. The fetching tools make the same plain HTTP request a crawler would, identified as SiteizBot.

How is this different from the Siteiz scan?

The MCP tools are quick single checks. The full scan on siteiz.com also renders the page in a real browser, compares it with what crawlers receive, and grades it A to F with ranked fixes. Every MCP answer links to it.

Which transport does it use?

Streamable HTTP at https://siteiz.com/mcp, stateless JSON responses, MCP protocol versions 2024-11-05 to 2025-06-18.

The full picture

Want the grade, not just the checks?

The free Siteiz scan renders your page, compares it with what crawlers get, and ranks the fixes by points.

Run the free scanAll free tools