The OpenAEO rubric, as a specification
This is the scoring model, written so you can disagree with it precisely: eight classes of evidence, four statuses, a stability policy, and a version stamped on every result. Being clear about what this page is not: the 49 individual checks and the code that runs them are ours and are not open source. The MIT package implements the smaller eight-check audit, and every paid result names the evidence class behind each score, so a number can be argued with even where the implementation is closed.
The idea: evidence classes
Every check declares, in advance, the single kind of evidence it may be scored from. It may not be scored any other way. That sounds procedural and is the most important rule here, because the usual failure of a visibility score is quietly inferring one thing from another: reading robots.txt and reporting it as crawler access, or reading a model's prose and reporting it as a citation.
| Class | Checks | Points | What it means |
|---|---|---|---|
page | 23 | 44.5 | the HTML, headers and timing of the page we fetched |
sitemap | 11 | 22.0 | the page inventory declared in sitemap.xml |
asked | 7 | 17.0 | put to a live assistant, read from the URLs it cites, never from its prose |
account | 3 | 6.0 | whether the site's owner measures anything at all |
edge | 1 | 3.0 | how the network in front of the site answers a request carrying a crawler's name |
public | 1 | 3.0 | a public identifier the entity can be disambiguated against |
robots | 2 | 2.5 | the policy stated in robots.txt, which is a claim rather than a behaviour |
linked | 1 | 2.0 | whether declared sameAs links resolve and point back |
Four statuses, and two of them are not failures
- pass / fail
- Measured, and the condition held or did not.
- not-measured
- A property of the run, not the page. We could not look. It leaves the denominator and never scores zero, because a zero you did not earn is a lie that looks like data.
- not-applicable
- A property of the page. The check does not apply here, for a stated reason. A contact page has no price to mark up, and scoring it zero for that would be scoring it for being a contact page.
The score is therefore over what was actually looked at. A site with 12 checks unmeasured is scored out of the remaining 88 points, and the report says so.
Retrieval is a gate, not a score
Retrieval and Index is 11 points of 100, deliberately. Passing it earns almost nothing, because being fetchable is the floor rather than an achievement. Failing it caps everything else, because no amount of schema helps a page an assistant cannot obtain. The dimension weights:
| Dimension | Points | Question |
|---|---|---|
| Retrieval & Index | 11 | can it be fetched and indexed at all |
| Schema | 17 | can a machine tell who and what this is |
| Answer Layer | 26 | is there anything here worth quoting |
| Page Coverage | 14 | do the pages a buyer needs exist |
| Off-Site Consensus | 24 | does anyone else corroborate it |
| Measurement | 8 | would you know if it changed |
The control request
A per-crawler fetch is worth less than it looks. We request the page carrying a crawler's user agent, from our own network. The real crawler arrives from its operator's address range. A firewall keying on address or forward-confirmed reverse DNS can refuse us and admit the real one, or the reverse.
So every run also sends a control: a client that identifies itself honestly as ours, not as a crawler and not as a browser. Naming ourselves is what makes it a control, because then the only variable between the two responses is the name on the request.
- name-based
- Control served, crawler refused. The rule keys on the crawler's name and the reading means something.
- inconclusive
- Control refused too. The site turns away anything that is not a recognised browser, and we cannot tell from outside whether AI crawlers were singled out. We stop rather than claim more.
- clear
- Everything served.
This is why edge evidence is worth 3 points of 100 and is labelled an
indication rather than evidence. Server logs settle it, and where a log contradicts the fetch, the
log wins and the report says which reading was wrong.
Three outcomes for crawler identity
A request claiming to be GPTBot is checked, not believed. But the burden of proof runs one way only, and getting that backwards makes a stale range file look identical to an attack.
- impostor
- Positive evidence of a false claim: a PTR record naming an operator that cannot confirm it.
- outside-range
- Not in the operator's published list. Worth showing, not proof: a published range file lags address space the operator has already started using.
- unverifiable
- We could not check.
Verified rows also record which signal proved them. Google, Microsoft and Apple publish a reverse-DNS convention, so those can be forward-confirmed. OpenAI, Anthropic and Perplexity publish ranges only, which is a weaker claim, and the two are reported separately rather than under one verified flag.
Stability, so scores can be compared
| Change | May it move a score |
|---|---|
| patch | No. Wording, explanations and fix instructions only. |
| minor | Yes, within a dimension. New checks may be added and points redistributed; dimension totals do not move. |
| major | Yes. Weights, evidence classes or the meaning of a status may change. Scores are not comparable across a major. |
Every audit response carries rubricVersion. Two scores may only be
compared when their major versions match.
Conformance
74 of the 100 points are deterministic and can be pinned. The other 26 cannot:
asked queries live models that answer differently between runs, account
depends on who is asking rather than on the site, and edge depends on a network in
front of a site nobody controls. Stating that scope is the difference between a benchmark and a
marketing number.
node tools/verify-conformance.js # check an implementation node tools/verify-conformance.js --record # re-pin, only when a change is intended
Recorded pages and their expected deterministic scores are published. Matching them is what "implements the OpenAEO rubric v2" is allowed to mean.
What this does not measure
Whether a page deserves to be selected. Reachable, quotable and corroborated are
measurable. Which source a model names on a given day is not, and anything claiming otherwise is
selling a drawn curve. The rubric measures whether you can be selected. Whether you
are is a separate and more expensive measurement, which is what the
asked checks do.