Agent Readiness for Websites: What It Means and How to Improve It
What agent-readiness scores measure, what they miss, and practical checks to make your website easier for AI agents to use.
Agent readiness asks a simple question:
Can an AI agent reliably discover, understand, and interact with your public website?
The answer is not the same as “Does the site have an llms.txt file?” It is also not the same as SEO, accessibility, or API design—although all of those can contribute.
A score is useful only when you know what was tested. Finding a sitemap and completing a checkout are very different things.
A practical agent-readiness ladder
VCL uses measurements from Cloudflare’s Is It Agent Ready. The levels shown in our benchmark are:
| Level | Name | Practical meaning |
|---|---|---|
| L0 | Not Ready | Below the first readiness level. |
| L1 | Basic Web Presence | Foundational discovery signals are present. |
| L2 | Bot-Aware | AI crawler access and usage rules are declared. |
| L3 | Agent-Readable | Important content is available in an agent-friendly format. |
| L4 | Agent-Integrated | Structured tools or interfaces can be discovered. |
| L5 | Agent-Native | Rich agent integrations are available. |
This is one scanner’s framework, not a universal standard or a security certification. Check the individual results before deciding what to fix. A high level does not prove that an agent can finish a task on your site.
What makes a website easier for agents to use?
The following checks cover both discovery and browser interaction. They are not all part of Cloudflare’s score.
1. Crawlable public information
Open the page’s raw HTML. Can you find the main product description and links to important pages? Content that appears only after a click can be missed by systems that do not interact with the page.
2. Semantic structure
Use real headings, labelled buttons, and links. An icon-only control still needs an accessible name. Try navigating with the keyboard to find missing labels and awkward focus order.
3. Consistent product information
Check that page titles, canonical URLs, structured data, and public documentation agree about what the product does and where to find it.
4. Explicit crawler policy
Check your robots rules and any AI crawler policies. Make sure they are reachable and say what you intend. Publishing a policy file does not, by itself, make the site usable.
5. Stable interaction design
Label forms, explain errors, and keep navigation predictable. Test a real task rather than stopping at the homepage.
6. Discoverable integrations
If your product offers an API or an MCP interface for agents, document how to find it, what it does, and which permissions it requires. Keep private workflows protected.
A worked example: check the discovery path on VCL
On 23 October 2026, we fetched four public URLs on VibeCodingList without signing in:
| URL | Observed response | What an agent can learn |
|---|---|---|
| `/robots.txt` | HTTP 200, plain text | Crawler rules, declared content-use signals, and the sitemap location. |
| `/sitemap.xml` | HTTP 200, XML | Canonical public URLs, including blog articles. |
| `/auth.md` | HTTP 200, Markdown | How a user provisions a scoped credential, how it is used, and how to revoke it. |
| `/openapi.json` | HTTP 200, JSON | The Agent API contract and its 11 documented paths at that check date. |
The useful detail is the response body as well as the status. A login page returned with HTTP 200 would not give an agent an API contract. We checked that the Markdown contained the credential instructions and the JSON contained the documented API paths.
Try the same path on your site: find the public documentation, identify a supported task, and check whether it explains the required permissions. If documentation requires a login, provide a public introduction while keeping account data and credentials private. If a discovery URL returns your app shell, serve the intended document at that URL.
These checks establish that discovery information is reachable. They do not assign a Cloudflare level or prove that an agent can complete an authenticated task. For that, test a defined workflow with the right permissions and record where it succeeds, fails, or needs a person's approval. The VCL integration guide is the starting point for our public and account-specific workflows.
Agent readiness is not a replacement for accessibility
People still need to use your website. An agent-readiness score does not establish WCAG conformance, and an accessibility audit does not prove that an agent can safely call your API.
Keep those assessments separate. After making changes, re-scan and test the task you actually want an agent to complete.
Where the VCL benchmark fits
The State of Vibe Coding benchmark shows the measured projects, missing results, source, and cutoff date. The main report is updated through 2026. Earlier editions keep their original figures so citations remain usable.
Use the report methodology to understand those measurements, or the project leaderboard for the latest recorded results. Keep the scanner name and measurement date alongside any score you share.