wikipedia
It lets every AI crawler in.
Letting AI in or keeping it out is the site’s choice: we show it, never judge it.
- AI trainingOpenAI, Anthropic, Google, Meta and 4 moreLet in
- AI answers and searchOpenAI, Anthropic, Google, Perplexity and 3 moreLet in
- Text without scripts7,580 characters before scripts runFound
- Markdown copyNot offered: it sends the page’s HTMLNot found
- llms.txtNone foundNot found
- For AI agentsNone of the 8 we look forNot found
Every check, with its evidence
AI companies
Its robots.txt names none of the 24 crawlers of the 10 AI companies we know, so it lets every one in.
None of the 27 other AI crawlers we know is kept out.
What it tells AI crawlers
It gives no Content Signals and no “noai” tag.
Can AI find its pages?
- robots.txtFound, 675 linesFound
- SitemapListed in robots.txtFound
Show evidenceHide evidence
Exhibit 1 · robots.txt, line 424Is this wrong? Report itSitemap: https://en.wikipedia.org/w/rest.php/site/v1/sitemap/0
- Links for AI toolsNone in the home page’s headersNot found
- Search listingAllowed: no noindex ruleAllowed
Can AI read it?
- Text without scripts7,580 characters before scripts runFound
Show evidenceHide evidence
Exhibit 1 · page text, before scripts runMasked for privacy. An email address or phone number can belong to a person, and data protection law, such as the EU’s GDPR, asks us to keep only what we need: that the page lists one, and where.Is this wrong? Report itWikipedia Wikipedia The Free Encyclopedia English 7,254,000+ articles 日本語 1,523,000+ 記事 Deutsch 3.1••.•00+ Artikel Français 2 7•• •00+ articles Русский 2 1•• •00+ статей Español 2.1••.•00+ artículos…
- Markdown copyNot offered: it sends the page’s HTMLNot found
- llms.txtNone foundNot found
- Title and descriptionBothFound
Show evidence (2)Hide evidence
Exhibit 1 · page source, line 5<title>Wikipedia</title>
Exhibit 2 · page source, line 6Is this wrong? Report it<meta name="description" content="Wikipedia is a free online encyclopedia, created and edited by volunteers around the world and hosted by the Wikimedia Foundation.">
- Structured dataNone foundNot found
What it offers AI agents
These matter for sites with an API or a sign-in.
It offers none of them: an API catalog, sign-in details for AI agents, an auth.md, an AI catalog, an MCP server, an agent card, Agent Skills or agents named in DNS.
If this is your site
- Add an llms.txt: a short summary of the site, in Markdown, for AI models.
- Decide what AI may do with the site, and say it in robots.txt. Write the lines
Like 68% of the 1,691 popular sites whose robots.txt we read in October 2026, it keeps no AI training crawler out. How many websites block AI crawlers?
We read the site once, as our named bot: its robots.txt, read as crawlers read it, its home page and the files AI tools ask sites for. What we couldn’t read says so. robots.txt asks and doesn’t enforce: a firewall can treat AI crawlers differently from us.
Automated findings from the site’s public home page and public records, as of 29 Oct 2026. They describe what we found, not the people or business behind the site, and they may be incomplete or out of date. This is information, not advice. Terms · Report an error
What this scan can’t tell you
- Whether the business is real, honest or delivers what it sells.
- Whether a site was made with AI when no traces were left: coding assistants and hand edits leave none.
- How many people visit: we show popularity bands from public lists, not visits.
- Anything beyond the home page as it was at the time of the scan.
Is this your site? Ask for a correction.