ASSmetric by LLM Jesus

The ASS score: methodology

Specification v1 · 2026-09-09 · measured on gpt-5-6 · corpus of 82 recorded answers, 186 fan-out searches

The ASS score is a per-recording measurement of a business’s standing in one ChatGPT answer, decomposed along the three routes by which recommended entities were observed to enter delivered answers: Authority (parametric prior), Sources (retrieval and entity channels), and Specificity (first-party satisfaction of the fan-out’s query intents). It is computed from captured transport, not from a rendered page and not from an API surrogate.

1. Capture

A single buyer-intent prompt is submitted in a fresh conversation on an authenticated consumer session. Two artefacts are recorded: the server-sent event stream of the answer and the persisted conversation record fetched after the stream closes. The persisted record carries the tool-call nodes (recipients web and web.run), the search result groups, and the entity references; the stream carries ordering. Captures are paced at a minimum ninety-second interval, one prompt per conversation, and a conversation with more than one user turn is flagged as contaminated and excluded.

An integrity gate runs before any analysis: stream and persisted record must both be present, agree on conversation identity, and pass the user-turn check. A recording that fails the gate is listed in the report as excluded and contributes nothing to the score.

2. Decoding the fan-out

Before answering, the model emits pipe-delimited tool-call lines. Six call types are decoded by grammar:

Each decoded query is tagged for intent by pattern: year_qualified, pricing, official_lookup, reviews, awards (credential language), availability, near_me, rankings, comparison, site_scoped, and platform_named. Site scopes are split into own-brand and third-party. Words the query added beyond the user’s prompt are recorded as addenda whether or not a tag matched them, so an untagged signal is visible rather than lost.

3. Entry-path attribution

Every brand named in the delivered answer is attributed to one path, resolved in the order events occurred:

1. prior: the brand was written into a fan-out query before any result returned (pre-seeding). Parametric.
2. places: the brand is a record in the entity channel.
3. web: nothing put it there except a retrieved third-party page; the carrying domains are recorded.
4. unattributed: named with no captured source, reported as its own bucket rather than folded into a preferred path.

Across the current corpus the distribution is prior 29.8%, places 37.0%, web 3.5%, unattributed 29.6%. These are the shares quoted beside each pillar on the front page and they are recomputed on every capture.

4. The pillars

Authority (A)

100 if the business is named in the delivered answer. 60 if it is pre-seeded into a fan-out query but not named. 0 otherwise. Name matching is word-bounded and case-insensitive against the answer text with citation markers resolved to their display names.

Sources (S)

Scored only over channels the recording used. When the entity channel was pulled: presence in the records 50, presence in the retrieval pool (title or snippet of an opened page) 30, presence on a named or opened directory or review platform 20. When no entity channel was pulled the 50 is redistributed 60/40 across the remaining two. When nothing was retrieved at all the pillar is not measured.

Specificity (S)

The set of intents asked is the subset of fan-out tags with a first-party check: pricing, year, official, credentials, reviews, hours, service area. The business’s site is read (home page plus up to four internal pages it links to, preferring pages whose path suggests pricing, about, contact, reviews, services, areas or accreditation), and each asked intent is tested against the plain text of those pages. Score is passes over asked, as a percentage. If the fan-out carried no checkable intent, the four most common corpus intents stand in (year, pricing, official, reviews) and the report says so. If the home page cannot be fetched the pillar is not measured.

5. Aggregation and bands

Overall = arithmetic mean of the measured pillars, rounded to an integer. Unmeasured pillars are excluded from the denominator. Bands: 70–100 In the running, 35–69 Known, not chosen, 0–34 Not in the picture. Every score is reported with its recording identifier, model build, locale and timezone, so it can be reproduced or challenged.

6. Known limits

• One recording is one sample. Same-day repeats of the same prompt change 60.4% of retrieved sources; day-to-day repeats change 76.6%. A score is a reading, not a rank.
• All recordings come from one account with a declared locale and timezone. Personalisation and geography are held constant rather than sampled.
• Prompt wording moves the answer. Prompts containing “best” pulled more best-titled pages than neutral prompts in our own corpus, so buyer questions are phrased neutrally.
• Several entity-record fields present on earlier builds are null on gpt-5-6 (is_claimed, popularity_score, provider, ranking_score, review_highlights, reviews). Nothing computed from them is used.
• The first-party read is text-only. Prices in images, content behind forms, and pages not linked from the home page are not seen.

7. Provenance

The capture engine, grammar, analyzers and public corpus are the ones behind llmjesus.com, where the corpus figures are published with their sample sizes and re-measured on every capture. ASSmetric is a Serpply Ltd product and is not affiliated with OpenAI or Google.

← Get your ASS score