Workflow

Step 1: Implement /llms.txt (recommended but optional)

• Publish a /llms.txt at the site root following the community proposal (https://llmstxt.org/). The file should be plain Markdown, human- and machine-readable, and point to canonical docs, API references, changelogs, and highest-value long-form pages.
• The spec notes a minimal required structure: an H1 with the project/site name; a short blockquote summary is recommended; follow with ordered sections (Priority docs, APIs, changelogs, etc.). See the official spec for exact formatting guidance.
• Optionally publish clean Markdown versions of those pages at the same URL with a .md appended (page.html.md or index.html.md). These make it easier for text-only ingestion and for deterministic context extraction used by retrieval pipelines.
• Practical minimal /llms.txt contents: a one-line site description, a prioritized docs list, and explicit canonical URLs. Treat /llms.txt as additive metadata — it helps discovery but does not replace good structured content.

Example head of /llms.txt

AcmeCorp /llms.txt

AcmeCorp builds secure API tooling for payments and identity verification.

Priority docs

• https://acme.com/docs/getting-started
• https://acme.com/docs/api-reference

(Reference: llms.txt proposal — https://llmstxt.org/)

Step 2: Emit .md artifacts and optional llms-ctx bundles

• If you publish docs from a build pipeline (Sphinx, MkDocs, nbdev, Docusaurus, etc.), add a step to export or mirror clean Markdown (.md) versions of canonical pages.
• Projects following the llms.txt workflow sometimes emit llms-ctx or llms-ctx-full bundles (collections of text ready for ingestion). These are optional but useful when you control the retrieval pipeline for downstream agents.
• Note: independent analyses report mixed short-term impact from /llms.txt alone. Use it as one signal in a broader GEO program and measure empirically.

Step 3: Structure content for deterministic extraction

• Create structured-answer blocks: a one-line direct answer, a 1–2 sentence supporting anchor (numbers, dates, versions), and a primary-source hyperlink.
• Use short, well-labeled sections (H2/H3) and answer-first paragraphs to make retrieval and snippet extraction predictable.

Before (low citability):

We've improved our product's speed. Users like it.

After (high citability):

Acme v3.2 reduced API p99 latency from 340ms to 180ms (47% improvement) by migrating from PostgreSQL full-text search to a Typesense index; measured across 2.1M production queries (Jan 2025). Source: internal benchmark report (link).

Step 4: Score factual density (operational rubric)

Score each paragraph with a simple rubric (example weights):

• Named entity: 2 points
• Statistic with temporal anchor: 3 points
• Explicit comparison (before/after): 2 points
• Primary-source citation/link: 2 points
• Version number or spec reference: 1 point
• Temporal anchor (month/year): 1 point

Operational target: paragraph score >= 8 on priority pages. Paragraphs scoring < 4 are unlikely to be cited.

Step 5: Platform-specific optimization (updated and source-backed)

• OpenAI (Responses API & agent runtimes): OpenAI announced agent-capable Responses API functionality and a container/shell environment for agent runtimes (see OpenAI docs: "Migrate to the Responses API" and related Agents/Tools guides). Operational test: run a grounding flow using the Responses API/Agents SDK and the shell/tooling environment to verify your canonical pages and .md artifacts are reachable from the runtime and included in the agent's context (developers.openai.com guides).

• Google / Gemini: Google introduced Flex and Priority inference tiers (Apr 2, 2026). You can set the service_tier parameter (e.g., "Flex" or "Priority") when calling the Gemini/GenerateContent endpoints and the API response will indicate which tier served the request. Operational guidance: record the service_tier and response metadata for each test query — interactive/priority queries may surface different grounding choices than cost-optimized/background (Flex) runs (Google AI Blog: "Introducing Flex and Priority Inference").

• Citation-first systems (Perplexity and similar): systems that place a premium on explicit source links will prefer pages with multiple short structured answers and clear primary-source hyperlinks. Supply inline links and short evidence snippets to increase probability of attribution.

• Microsoft / Bing agents: keep canonical pages accessible, include OG metadata and FAQPage schema where appropriate, and ensure your .md artifacts are reachable by crawlers and agent runtimes.

Platform behaviors diverge rapidly. Use the platform docs cited above as the authoritative sources for expected behavior and test across model versions and inference tiers.

Step 6: Measure AI citability (concrete)

• Run a controlled set of target queries across AI providers and models (include model name / inference tier when available). Record whether your page is directly cited (3), paraphrased (1), or absent (0). Compute citation rate = total score / (queries × 3).
• For Gemini tests, include the service_tier value and any response metadata indicating tier fallback or downgrade.
• For Responses API/agent tests, record whether the agent runtime successfully fetched .md artifacts or files from your canonical URLs in the shell/container environment.
• Track results over time and annotate tests with provider release notes or Search Central updates to identify correlations.

Examples (concise)

• High-citability pricing block: single-sentence summary + bullets with exact numbers and dates + link to canonical pricing policy.
• API limits: one-line statement of limit, exact number, effective date, and link to changelog or versioned docs.

Decision tree (operational)

• New docs: design structured answers, publish .md artifacts, add pages to /llms.txt
• Low-traffic pages: rewrite first paragraph to meet factual-density target
• Pages ranking organically: add GEO signals but avoid heavy title/headline rewrites that could affect organic click-through
• Versioned technical docs: emit per-version .md artifacts and include clear version labels and dates

Edge cases and gotchas

1. AI crawler blocking — verify robots.txt and allowlists for desired crawlers and agent runtimes. Don't accidentally block common agent user agents.
2. Stale statistics — always attach dates and update benchmark pages regularly. Model caches and retrieval indices can surface older snapshots.
3. Over-optimization — maintain clarity; avoid packing entities where they harm comprehension.
4. /llms.txt adoption is a community proposal and is growing but not universal; independent analyses show mixed short-term impact from llms.txt alone. Use it as high-value metadata but validate with measurement.
5. Attribution stripping — some systems paraphrase without returning links. You cannot force attribution, only increase probability by providing unique, verifiable, and dated material.
6. Platform divergence — model and index updates can change citation behavior. Monitor provider announcements (OpenAI, Google) and independent reporting.

Evaluation checklist

• /llms.txt exists at site root, contains H1 project name, and lists priority docs and canonical links
• Key docs are available as clean .md artifacts or llms-ctx bundles where feasible
• Average factual density per paragraph is >= 6 on priority pages
• Each priority page contains ≥3 structured answer blocks
• robots.txt does not block AI crawlers or agent runtime access needed for your tests
• All statistics include a temporal anchor (month/year)
• JSON-LD and OG tags are present and canonical pages are linked
• Citation rate across AI platforms and models (include inference tier and service_tier where available) is tracked and monitored
• Test logs include service_tier for Gemini calls and agent runtime fetch success for Responses API tests

Operating notes

• Add /llms.txt and .md artifacts to your docs CI pipeline. If feasible, emit llms-ctx bundles for top docs.
• Run a weekly or monthly citability check for the top 25 queries; log provider, model, inference tier/service_tier, and citation outcome.
• Correlate citation drops or lifts with provider release notes (Responses API/Agents docs, Gemini inference-tier announcements, Search Central updates).
• When testing agent grounding flows, ensure the agent runtime can access the canonical URLs and any required files. Test with container-based runtimes and the Responses API shell tool if possible.

Fresh signals to watch (authoritative examples)

• llms.txt — community spec and implementation notes: https://llmstxt.org/
• OpenAI — Responses API, Agents, and runtime/container guidance (see developers.openai.com Agents, Tools, and Migrate to Responses guides)
• Google — Gemini inference tiers (Flex and Priority) and API docs (service_tier parameter and response metadata)
• Google Search Central — AI features and guidance for discoverability

Experiments to run next

1. Emit /llms.txt + .md artifacts for a representative docs site and measure citation rate change over 90 days across ChatGPT (Browse), Perplexity, Gemini (test Flex vs Priority service_tier), and Bing AI.
2. A/B test adding a scored (>=8) structured-answer intro paragraph to 50 low-traffic pages and measure citation lift and organic ranking changes.
3. Run a short experiment to validate agent-grounding: use the Responses API/Agents SDK and the shell/container environment to run a retrieval+answering flow against your docs and measure which pages are chosen as grounding sources and whether canonical .md artifacts are preferred.

Sources and references

• llms.txt proposal and documentation: https://llmstxt.org/
• OpenAI Responses API and Agents documentation (see developers.openai.com guides for Responses, Agents, Tools, and Migrate to Responses)
• Google Gemini API — Flex and Priority inference tiers (Apr 2, 2026): https://blog.google/innovation-and-ai/technology/developers-tools/introducing-flex-and-priority-inference/ and https://ai.google.dev/gemini-api/docs/optimization#inference-tiers
• Google Search Central — AI Features: https://developers.google.com/search/docs/appearance/ai-features
• Independent analysis and industry reporting (varied results): validate for your site rather than assuming consistent lifts