Independent domain snapshot · generated 2026-11-08 01:15 UTC · Reused from the 30-day snapshot
Data sources:
Crawl audit + Core Web Vitals (measured)Plain HTTP audit (robots.txt, llms.txt, Markdown negotiation, .well-known)DataForSEO Labs bulk traffic estimation (third-party estimate)DataForSEO Labs ranked_keywords (third-party estimate)DataForSEO Backlinks summary (third-party estimate)Public RDAP registry data
Measured with plain HTTP requests only: no browser, no JavaScript, no LLM. Checks cover robots.txt AI groups, llms.txt, Markdown negotiation, ai-catalog.json, the standard .well-known documents, soft 404s and server-rendered content.
Words in the raw HTML: 27.
Server-render or pre-render the main content: several agent fetchers do not run JavaScript, so a client-side shell reads as an empty page.
robots.txt returned HTTP 404.
No robots.txt means everything is allowed, but no AI-usage preference is declared. Publish one deliberately.
Explicit groups: none.
Every AI agent falls through to the '*' group. Add named groups when your policy differs by purpose (training vs search vs user fetches).
HTTP 404 (text/html; charset=utf-8), 577 bytes.
No /llms.txt. Publish one with an H1, a '>' summary and Markdown links so agents get a compact map of the site.
Content-Type: text/html; charset=utf-8. Vary: Accept not set.
Optional: serve a Markdown version via 'Accept: text/markdown' (with 'Vary: Accept'), a rel=alternate type=text/markdown link, or a .md URL.
Probe returned HTTP 404.
Skipped: no catalog is advertised.
Skipped: no Content-Signal line.
Skipped: no Agentmap line.
Skipped: no discovery document is published.
Score: weighted share of the checks that could be measured (p0 = 3, p1 = 2, p2 = 1; a warning counts half; skipped checks are excluded).
Copy the text or download the file, then upload it to your own site. Auto-publish only works for sites hosted under our domain; for all other sites, download the files and upload them yourself.
Upload to the site root so it is served at https://example.com/llms.txt. It is plain text: no build step and no server change are required.
# example.com > Public site summary for example.com, generated from an independent snapshot on 2026-11-15. This file lists the pages an AI agent can read and states how the content may be used. Every link below points at the live site; nothing here replaces the site. ## Key pages - [Home page](https://example.com/) ## Use of this content by AI systems AI systems may read these pages and cite example.com with attribution. Model-training use is governed by the site's robots.txt, which states the policy per crawler category (model training / AI search and citation / user-triggered fetches). ## Notes - Generated by Global Fact Hub (https://globalfacthub.com) on 2026-11-15. - Keep this file in the site root so it is served at https://example.com/llms.txt.
Upload to https://example.com/.well-known/ai-catalog.json and reference it from a <link rel="ai-catalog" href="/.well-known/ai-catalog.json"> tag or an Agentmap line in robots.txt. The entry's url stays the canonical customer address https://example.com/llms.txt even though we only host a sample copy under https://globalfacthub.com/domain/example.com/. ARD is designed for agent resources (MCP servers, A2A agents); replace this site-summary entry when you publish one.
{
"specVersion": "1.0",
"entries": [
{
"identifier": "urn:air:example.com:llms-txt",
"displayName": "example.com site summary",
"title": "example.com site summary (llms.txt)",
"description": "Public site summary for example.com: key pages, an AI usage statement and the crawler policy that applies to them. Generated from an independent snapshot on 2026-11-15.",
"type": "text/markdown",
"url": "https://example.com/llms.txt",
"representativeQueries": [
"example",
"explam",
"can you show me an example"
]
}
]
}
Append these lines to the existing robots.txt and keep every existing line. Do not replace the file: the block only adds rules and states a default policy you can edit per category.
# ---- AI agent policy (appended by Global Fact Hub on 2026-11-15) ---- # Append this block to the end of the robots.txt of example.com. # No existing rule is removed or rewritten: this block only adds rules below yours. # RFC 9309: a crawler that matches a named group reads only that group, so each # category below repeats its own policy. # 1) Model-training crawlers. Change 'Allow: /' to 'Disallow: /' to opt out of training. User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended Allow: / # 2) AI search and citation crawlers. Keep these allowed so AI answers can cite the site. User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: PerplexityBot Allow: / # 3) User-triggered fetches: a person asked an assistant to open a page. User-agent: ChatGPT-User User-agent: Claude-User User-agent: Perplexity-User Allow: / # ---- end of appended AI agent policy ----
After an owner auto-publish (signed in), the same files are also served from https://globalfacthub.com/domain/example.com/llms.txt and https://globalfacthub.com/domain/example.com/.well-known/ai-catalog.json. We only write to the sample directory we host (globalfacthub.com/domain/example.com/); we never report a third-party site as updated.
Expand the page to at least 300 words of specific, useful copy.
Publish a sitemap and reference it from robots.txt so new pages get discovered.
Add JSON-LD for Organization plus the page type (Product, Article, FAQPage and so on).
Add a rel=canonical link pointing at the preferred URL of each page.
~10,768estimated organic visits / month
Source: DataForSEO Labs · bulk_traffic_estimation, market United States / English. This is a third-party estimate: it models organic search visibility (ETV) and is not measured analytics traffic. It can differ from your own analytics or Search Console numbers. Ranked keywords in the estimate: 470. Estimated paid visits: 0.
| Keyword | Rank | Monthly volume |
|---|---|---|
| example | 2 | 368,000 |
| explam | 18 | 201,000 |
| can you show me an example | 14 | 9,900 |
| what is example | 4 | 9,900 |
| what is example of | 2 | 9,900 |
| example org website | 1 | 5,400 |
| google.ru | 42 | 5,400 |
| example domain | 1 | 3,600 |
| example.com | 1 | 2,900 |
| example 3 | 7 | 1,900 |
Source: DataForSEO Labs · ranked_keywords, market United States / English. Rank and monthly search volume are modelled by the provider, not measured analytics. Showing up to 10 keywords.
Source: DataForSEO Backlinks · summary. Third-party index data, not your own analytics. Rank is the provider's 0-1000 authority score.
| Registrar | RESERVED-Internet Assigned Numbers Authority |
|---|---|
| Created | 1995-08-14 |
| Expires | 2027-08-13 |
| Domain age | 31.2 years |
| Nameservers |
|
Source: RDAP (public registry data).
The free audit tool shows the health score, all five dimension scores and the P0/P1 issues to fix first. No signup required.
| Site health | Crawl audit + Core Web Vitals (measured) Measured |
|---|---|
| AI agent readiness | Plain HTTP audit (robots.txt, llms.txt, Markdown negotiation, .well-known) Measured |
| Traffic estimate | DataForSEO Labs bulk traffic estimation (third-party estimate) Third-party estimate |
| Top keywords | DataForSEO Labs ranked_keywords (third-party estimate) Third-party estimate |
| Backlinks | DataForSEO Backlinks summary (third-party estimate) Third-party estimate |
| Registration (WHOIS) | Public RDAP registry data Registry data |
Scores and estimates in this report come from the sources listed above, measured when this snapshot was generated. Keyword, traffic and backlink figures are third-party estimates and may differ from your own analytics. Missing measurements are shown as no data, never as zero. Snapshot cached for 30 days.