Three overlapping rings labeled Websites, GEO and Codebases
Websites
GEO
Codebases
AI READINESS
Agent Readiness
Making websites and codebases work for AI agents
November 2026
OPENING
Agenda
01
Why agent readiness, and a shared model
OPENING
02
Websites: find, read, control, act
WEBSITES
03
GEO: being chosen and cited by AI answers
GEO
04
Codebases: context, verification, maturity
CODEBASES
05
Other surfaces, roadmap, takeaways
BEYOND
Source: Cloudflare, “Agent readiness” (Apr 2026), framing of browsers → search engines → agents
OPENING
Why now: the web’s third audience
1990s
Humans in browsers
We optimized for rendering
2000s
Search crawlers
We optimized for indexing (SEO)
2020s
AI agents
We optimize for reading, acting, verifying, and agents
Analogy: the “mobile-friendly” moment.  Around 2012, sites that weren’t responsive slowly lost traffic.
What is different this time: agents don’t only read.  They reason over what they read, call tools, fill forms and, increasingly, transact on a user’s behalf.
On the software side, coding agents now write and change production code, so the repo itself has become an interface that a program has to use.
Sources: AgentReady open spec (agentready.org); Factory, “Agent Readiness” (2026), concepts only
OPENING
What “agent readiness” means
How well an environment (a site, API, or repo) lets an agent finish a task correctly, with no human filling the gaps
A property of the environment, not the LLM model
Measurable: binary checks, maturity levels, scores
Built mostly on open standards: RFCs, IETF drafts, open specs
Failure modes: blocked (can’t find), unreadable (can’t read), unactionable (can’t act), unverifiable (can’t check its work/prove who it is)
Find
Read
Agent
Act
Verify
Environment
Things that help humans (semantic structure, plain text, clear docs, accessibility) usually help agents too
2 biggest failure points: hiding the answer behind JavaScript, and blocking access (through robots.txt or bot protection)
OPENING
One model, three surfaces
Stage
Website
GEO
Codebase
Find
robots.txt · sitemap · Link headers
Retrieved for the query
Agent can locate its instructions and the right files (AGENTS.md, repo map)
Read
SSR HTML · Markdown · llms.txt · JSON-LD
Passages easy to extract and cite
Docs · conventions · architecture notes
Act
APIs · MCP · OAuth · WebMCP
n/a: the engine acts by citing you
Build, run
Verify
Content Signals · bot authentication
Accuracy · citation sentiment
Linters · types · tests · CI · guardrails
Sources: AgentReady spec (Find / Read / Act); Cloudflare, “Agent readiness” (2026); Factory, “Agent Readiness” (2026)
AgentReady spec itself defines three stages, Find, Read and Act.  We add Verify because trust and checking work is where most agent failures end up.
SECTION 02 · WEBSITES
Websites
Can an agent find, read, and act on your site, and does it know what it’s allowed to do?
WEBSITES
How agents actually hit your site
PATTERN
EXAMPLE BEHAVIOR
WHAT IT NEEDS
Training crawler
Bulk-collects pages
robots.txt rules, Content Signals
Inference / grounding fetch
Fetches a page to answer a live question
Fast, server-rendered or Markdown content
Browser agent
Clicks and fills forms in a headless browser
Semantic HTML, stable UI, WebMCP
DRAFT / EMERGING
Tool / API agent
Skip UI and calls your API or MCP server
OpenAPI, MCP, OAuth discovery
Sources: Cloudflare, “Agent readiness” (2026); Content Signals (contentsignals.org)
Important trade-off: blocking all “AI bots” also blocks the grounding fetches that drive citations.
WEBSITES
FIND: let agents discover you
robots.txt: explicit rules for AI user agents, not just old search rules
sitemap.xml: a complete list of your canonical URLs
HTTP Link headers (RFC 8288): advertise resources without parsing HTML (can use HEAD request)
Don’t let WAF / bot protection block verified agents MUST
First-party docs that agents can find SHOULD
/robots.txt with explicit AI user agents
User-agent: OAI-SearchBot
User-agent: PerplexityBot
Allow: / # search/answer engines (drive citations)
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
Disallow: / # model training (private data)
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap.xml
Response header
Link: </llms.txt>; rel="describedby"
Sources: Cloudflare, “Agent readiness” (2026); AgentReady spec (AR-FIND-xx practices); RFC 8288, Web Linking
WEBSITES
READ: serve content agents can parse
Answers in the initial HTML, not only client-side JS (SSR) MUST
Correct status codes: 404, 429 + Retry-After, 503 + Retry-After MUST
Markdown negotiation: Accept: text/markdown (ex. mintlify, blume)
llms.txt: a curated Markdown reading list
DRAFT / EMERGING
JSON-LD / schema.org for entities, products, pricing, FAQs
Pricing and key facts as text, not images
Glossy HTML page
markup, scripts, styles
content
Same page as Markdown
content
Up to ~80% fewer tokens in some cases (vendor-reported)
Sources: Cloudflare, “Agent readiness” (Apr 2026), token figure; AgentReady spec; llms.txt proposal (llmstxt.org); RFC 6585 (429)
If the answer isn’t in the first HTML response, many agents never see it
Sources: Cloudflare Radar via “Agent readiness” (Apr 2026, ~200k top domains); contentsignals.org; Squirrelscan rule docs; IETF aipref and webbotauth WGs
WEBSITES
CONTROL: tell agents what they may do
Content Signals in robots.txt: a separate choice per use
search
Search results and citations in answers
ai-input
Use at inference time (grounding, RAG)
ai-train
Training or fine-tuning models
IETF aipref: a standard vocabulary and attachment documents
DRAFT / EMERGING
Web Bot Auth: bots sign requests, sites verify keys
DRAFT / EMERGING
Signals are preferences, not enforcement: pair with verification and access controls
/robots.txt
User-agent: *
Content-Signal: search=yes, ai-input=yes,
ai-train=no
Allow: /
One line in the real file; wrapped here to fit.
~78%
of top sites have robots.txt
~4%
declare Content Signals
Sources: RFCs 9727/8414/9728; MCP SEP-2127; AI Catalog (draft); AgentReady; Cloudflare 2026: <15 of ~200k sites
WEBSITES
ACT: expose capabilities, not just pages
DRAFT / EMERGING
Discovery
/.well-known/api-catalog
/.well-known/ai-catalog.json
Contract
OpenAPI · MCP
Auth
OAuth + PKCE
Payments
emerging, watch only
WebMCP: tools exposed inside the page
DRAFT / EMERGING
OAuth discovery (RFC 8414 authorization server / RFC 9728 protected resource) + PKCE
Agent payments: x402, ACP, UCP (watch-list only)
DRAFT / EMERGING
API Catalog (RFC 9727) at /.well-known/api-catalog
OpenAPI contract, published and public
MCP server (or CLI) + MCP Server Card, listed in an AI Catalog (/.well-known/ai-catalog.json)
WEBSITES
VERIFY: what an agent-ready site returns
curl -s https://example.com/robots.txt | grep -i content-signal
VERIFY
curl -sI https://example.com/ | grep -i '^link:'
FIND
curl -s -H 'Accept: text/markdown' https://example.com/docs | head
READ
curl -s https://example.com/.well-known/api-catalog
ACT
curl -s https://example.com/.well-known/oauth-protected-resource
ACT
curl -s -o /dev/null -w '%{http_code}\n' https://example.com/nope # expect 404
READ
We can verify all of these deterministically.  That's why there's a lot of open-source scanners and spec-based checkers automate these checks, like isitagentready.com, agentchecker.ai, agent-ready.dev
Sources: Cloudflare, “Agent readiness” (2026) check categories; AgentReady spec
WEBSITES
State of the web + your checklist
~78%
have robots.txt
~4%
Content Signals
~3.9%
Markdown negotiation
<15
MCP Cards + API Catalogs
43/100
avg score, 6% grade B+
MUST
Answers in initial HTML Correct status codes Don’t block verified agents MCP server or CLI, if agents should act
SHOULD
Sitemap · AI-aware robots.txt Content Signals · JSON-LD Markdown negotiation · OpenAPI OAuth discovery
MAY (emerging)
llms.txt · API Catalog AI Catalog · MCP Server Card WebMCP · Agent payments
Sources: Cloudflare Radar via “Agent readiness” (Apr 2026, ~200k top domains); AgentReady ORA Scanner (Oct 2026, ~127k sites)
*the MUST / SHOULD / MAY tiers follow the AgentReady spec’s approach (RFC 2119 style)
SECTION 03 · GEO
Generative Engine Optimization
Readiness gets you accessed. GEO gets you chosen and cited.
GEO
From ranking to being cited: SEO vs GEO
SEO
GEO
Output
Ranked list of links
Synthesized answer with a few citations
Unit
Page
Passage / fact
Win condition
Top-10 rank, click
Being quoted and cited
Signals
Links, keywords, UX
Clarity, evidence, authority, freshness
Metric
Rank, CTR
Citation rate, share of voice
Sources: Aggarwal et al., “GEO: Generative Engine Optimization,” KDD 2024; Search Engine Land GEO guide (2026, sponsored)
SEO competes for a spot in a list of links. GEO competes for a sentence inside a generated answer.
GEO
How generative engines pick sources
User query
Sub-queries
Retriever
Passages
LLM
Answer + citations
Passages are judged individually: each section must stand alone
A response typically cites only a handful of domains
Earned, third-party mentions weigh heavily, not just your site
Recency matters: show a visible last-updated date
Sources: Aggarwal et al., KDD 2024; Search Engine Land GEO guide (2026, sponsored; used for practices, not statistics)
The highlighted stage is where you win or lose.  Engines retrieve, chunk, and pick passages.  Each section of your page competes on its own.  If the passage is behind JavaScript or blocked by ai-input=no, it never reaches the highlighter.
GEO
The evidence: what moves visibility
RELATIVE VISIBILITY CHANGE BY METHOD (DIRECTIONAL)
Quotation Addition
+41%
Statistics Addition
≈+31%
Fluency Optimization
≈+28%
Cite Sources
≈+27%
Technical Terms
≈+18%
Easy-to-Understand
≈+14%
Authoritative
≈+10%
Unique Words
≈+6%
Keyword Stuffing
(below baseline)
KDD 2024 study · GEO-bench of 10,000 queries
Best methods: +41% word-count visibility, +28% impression
Quotations, statistics, and cited sources lead
Keyword stuffing: no gain, below baseline
Lower-ranked sites gain most: Cite Sources +115.1% at rank 5
GEO rewards evidence and clarity, not domain size. 
Source: Aggarwal et al., “GEO,” KDD 2024. ≈ values computed from its Table 1 vs. baseline; +41% is stated in the paper.
GEO
GEO practices that hold up
Answer first: direct answer, then context, under a clear H2/H3
Evidence: statistics, quotations, cited primary sources
Entity clarity: consistent names, author pages, Organization JSON-LD
Earned media: third-party coverage, reviews, community mentions
Freshness: update key pages and show the date
Technical base: the Websites section (crawlable, SSR, ai-input=yes)
Extractable, evidenced passages
Entity and authority
Technical readiness
Sources: Aggarwal et al., KDD 2024; Search Engine Land GEO guide (2026, sponsored; practices only)
Write passages that can be extracted, backed by evidence, about a clear entity, and do it on a technically ready site
GEO
Measuring GEO + anti-patterns
You can’t measure GEO with rank trackers.  You have to sample the engines.
Measure
Citation rate
How often you’re cited for target prompts
Share of voice
Your citations compared with competitors
Accuracy and sentiment
Is it describing you correctly?
AI-referred traffic
Referrer analysis on your logs
Method: a fixed prompt set, run repeatedly across engines; track trends
Avoid
Keyword stuffing Hidden text for bots Cloaking Blocking ai-input, then expecting citations JavaScript-only content
Sources: Search Engine Land GEO guide (2026, sponsored; metrics and practices); Aggarwal et al., KDD 2024
Venn diagram: agent readiness (access) and GEO (selection) overlap
Agent readiness (access) crawlable parseable actionable permissioned
GEO (selection) relevant evidenced authoritative quotable
SSR Markdown JSON-LD Signals Freshness
GEO
Access vs. selection
No access means no citation.  Agent readiness decides whether you can be used. GEO decides whether you are used.
Sources: Cloudflare, “Agent readiness” (2026); Aggarwal et al., KDD 2024; Content Signals (contentsignals.org)
SECTION 04 · CODEBASES
Codebases
Can a coding agent build, change, and verify your repo without someone holding its hand?
CODEBASES
Same model, new artifacts
STAGE
ON THE WEB (SECTION 2)
IN THE REPO
Find
robots.txt · sitemap · Link headers
Agent context files (AGENTS.md) and a repo map
Read
SSR HTML · Markdown · JSON-LD
Docs, conventions and architecture notes
Act
APIs · MCP · OAuth
Build, run and test locally with one command
Verify
Content Signals · bot authentication
Linters, types, tests, CI and guardrails
Sources: Factory, “Agent Readiness” (2026); Eno Reyes, “Making Codebases Agent Ready,” AI Engineer (Dec 2025)
CODEBASES
The model isn’t the bottleneck; the repo is
Agents fail in production because of missing context and weak verification, not weak models.
It works in the demo and fails on our repo. Why?
Undocumented env vars and setup steps
Tribal knowledge (“don’t touch that module”)
Slow, flaky, or missing tests: no way to check work
Verification asymmetry: checking is cheaper than producing
Shift: spec-driven, then verification-driven development
If the environment checks work, an agent can iterate
Iceberg: the model is the small visible tip; the environment is the large part below the water
generate
verify
fix
Model
Environment context, tooling, tests
Sources: Eno Reyes, “Making Codebases Agent Ready,” AI Engineer (Dec 2025); Factory, “Agent Readiness” (2026)
The iceberg: the model is the visible tip everyone argues about; the environment underneath decides the outcome.
CODEBASES
The pillars of a ready codebase
1 · Agent context
AGENTS.md, architecture, feature map, CODEOWNERS
2 · Style and validation
Linter, formatter, strict type checking
3 · Build and dev env
Lock files, pinned versions, one-command setup, devcontainer
4 · Testing
Unit test, integration, E2E, coverage, mutation, property-based testing
5 · CI/CD
Simulations, branch protection, deployments, required checks
6 · Code health
Dependency updates, dead-code detection, redundant code, cyclomatic complexity (CRAP)
7 · Observability
Telemetry (logs, traces, metrics)
8 · Security and governance
Sources: Factory, “Agent Readiness” (8 pillars, 60+ criteria); kodustech/agent-readiness (7 pillars, 39 checks); agent-next/agent-ready (9 areas)
CODEBASES
Where to fix it: the inverted pyramid
Inverted pyramid by Lauren Tan: codebase (good API design, architecture, refactoring) at the top, then static analysis (lints, types, compilers, tests, CI), verification (formal, empirical), prose (rules, skills, AGENTS.md), and human review at the narrow bottom
Lauren Tan (@poteto): how to trust your agents’ work more
STRONGEST FIX & EASIEST TO SCALE AT THE TOP
Codebase: make the easy path the right path
Pillars 3 and 6: build, code health
Static analysis: cheap, deterministic, every change
Pillars 2 and 5: lint, types, CI
Verification: prove it works end to end (TLA+, bend)
Pillar 4: generate, verify, fix
Prose: nothing fails if an agent ignores it
Pillar 1: AGENTS.md, rules, skills
Human review: hardest to scale; find gaps
Pillar 1 CODEOWNERS; next slide: branch protection
You don’t fix a missing railing by adding another sign. Throughput comes from trust, not from opening more agents. Make agents trustworthy first, then run them in parallel.
Source and image: Lauren Tan (@poteto), X post (Oct 2026) and pstack skills (github.com/cursor/plugins, pstack)
CODEBASES
Feedback loops and guardrails
An agent is only as good as the feedback loop it runs in.  Make the loop fast, local, deterministic, and blocking.
Fast: lint, typecheck, formatter, rust-based
Deterministic: no flaky tests, pinned toolchains, hermetic builds
Layered: agent hooks, pre-commit, CI, branch protection
Reproducible: devcontainers for identical environments
Readable errors: failure messages that point to the fix
Branch protection + review
CI required checks
Pre-commit
Agent hook
after each edit · fastest
Sources: agent-next/agent-ready (hooks, devcontainer checks); kodustech/agent-readiness; Eno Reyes, AI Engineer (Dec 2025)
Each outer ring catches what the inner ring missed,  the inner rings give the fastest feedback
CODEBASES
Maturity model: Level 1 to 5
L1
Functional / Foundational
Builds and runs; agents need heavy supervision
L2
Documented / Guided
README, setup docs, context file; agents can follow along
L3
TARGET
Standardized
Tests, maintained docs, security scanning, observability; Routine fixes, tests, dependency upgrades
L4
Optimized
Fast feedback, high coverage, rich telemetry; agents can take on larger features
L5
Autonomous
Self-verifying environment; end-to-end autonomous changes
Gating rule: pass ≥80% of a level’s criteria, plus all lower levels
Sources: Factory, “Agent Readiness” (2026; levels, 80% rule, L3 target, concepts only); kodustech/agent-readiness (level names)
Sources: kodustech/agent-readiness and agent-next/agent-ready (MIT; examples, not recommendations); Factory (2026); Eno Reyes (Dec 2025)
CODEBASES
Measure it, gate it, compound it
Measure: run open-source scanners locally or in CI
npx @kodus/agent-readiness . --format json
npx agent-ready check . --json
Gate: fail CI below a minimum level
npx @kodus/agent-readiness . --ci --min-level 3
Stabilize: pin scanner versions; track trends, not single runs
Compound: better environment, more effective agents, more time to improve
Compound
Measure, fix the gaps, agents become more effective, that frees time, and you reinvest the time in the environment.  Each turn makes the next one cheaper.
Flywheel: measure, fix gaps, agents improve, time freed, then measure again
Measure
Fix gaps
Agents improve
Time freed
BEYOND
Other surfaces: APIs, CLIs, docs, knowledge
ACT · VERIFY
APIs
OpenAPI, predictable errors, idempotency keys, rate-limit headers
ACT
CLIs
A --json output mode, non-interactive flags, meaningful exit codes
FIND · READ
Developer docs
Markdown mirrors, copy-pasteable examples, language-tagged code
FIND · READ
Internal knowledge
Wikis and tickets reachable through MCP or search, with access control
ACT · VERIFY
SDKs
Published to registries, typed, versioned
Anything an agent touches has a readiness level.
Same four stages apply.
Sources: AgentReady spec (Act via API, MCP server, CLI); Model Context Protocol (modelcontextprotocol.io)
Q&A
CLOSE
Key takeaways
1
Agents are a new user class. Optimize the environment, not the model.
2
Websites: be findable, readable, permissioned, actionable.
3
GEO: access is necessary. Evidence and clarity get you cited.
4
Codebases: context + fast verification = trustworthy autonomy. Aim for L3.
5
Measure it, gate it in CI, and let the flywheel compound.
APPENDIX
Glossary
Term
Meaning
Agent readiness
How well an environment supports agents completing tasks
GEO
Optimizing content to be selected and cited in AI answers
Content Signals
robots.txt directive: search / ai-input / ai-train
aipref
IETF WG drafting a standard AI-usage vocabulary
Web Bot Auth
IETF draft for cryptographically signed bot requests
llms.txt
Proposed Markdown file listing key content for LLMs
Markdown negotiation
Serving text/markdown on Accept: text/markdown
Term
Meaning
MCP
Model Context Protocol: open protocol linking agents to tools
MCP Server Card
MCP extension: static JSON with a server’s identity and endpoints
WebMCP
W3C community draft: pages expose tools to in-browser agents
API / AI Catalog
RFC 9727 index of APIs; draft ai-catalog.json index of MCP and agent cards
AGENTS.md
Open Markdown format for agent instructions in a repo
Verification asymmetry
Checking is cheaper than producing, so checks make autonomy safe
Share of voice (AI)
Your share of mentions and citations versus competitors