A dashboard can count every mention correctly and still answer the wrong business question. If an AI names your brand while recommending a rival, I would not report that as a recommendation win.

That is why I am taking a second look at Manual ChatGPT Checks vs groas in Claude: Which Visibility Number Holds Up?. Tracking your brand’s appearances is not quite the same as tracking whether AI recommends it. Before you use a visibility score to judge progress, this test checks what that number actually means.

My expectation: a mention-based score will overstate recommendations if you read it as a recommendation rate. It puts explicit picks, conditional picks, neutral listings and warnings into one total. This protocol separates them again: 12 buyer prompts, three observations each, one week, and classification rules you write before collecting answers.

No content overhaul yet. First, make the number earn its label.

1. State the question before opening the dashboard

Write this at the top of your audit sheet:

Does this dashboard distinguish answers that recommend my brand from answers that merely name it?

That is the question. Not whether your brand appears often, whether the chart looks healthy or whether the latest answer says something flattering.

A mention names you. A recommendation presents you as a suitable choice. Conductor makes the distinction in its measurement guide: a brand can be mentioned everywhere, recommended nowhere and still report success. This audit checks whether your reporting leaves room for that mistake.

Keep the scope narrow:

  • One named engine, with the model recorded where available.
  • 12 buyer-intent prompts, fixed before collection begins.
  • Three observations per prompt, spread across one stated week.
  • 36 planned observations, with unavailable answers recorded separately.

You are not measuring all of AI search or all buyers. You are checking whether one score keeps its meaning across a small set of answers you care about.

The prompt set is part of the measurement, not packaging around it. Sistrix found that prompt wording caused 2.4 times more variation than day-to-day changes for the same prompt. Lock the wording before you start.

2. Define what counts as a recommendation

Write the rubric before reading the answers. Once you see a flattering paragraph about your brand, close enough becomes a tempting classification rule.

Promptwatch’s definition draws the useful boundary: recommendation rate is stricter than mention rate and different from citation rate. It counts answers that present the brand as a good choice, ranked near the top or tied to the stated need.

Use these categories, with one classification per answer:

  • Explicit recommendation: The answer names your brand as the pick or one of the top picks, with a reason tied to the prompt. “Use Brand X for under-50-employee payroll” counts.
  • Conditional recommendation: The answer recommends your brand only if a stated condition holds. “Good if you need same-day installation, overpriced otherwise.” Keep the condition in your notes.
  • Neutral or incidental mention: The answer lists your brand without endorsing it, or uses it as an example. Being one of six names is not, by itself, a pick.
  • Negative recommendation: The answer names your brand to warn against it, dismiss it or rank it last. Conductor’s recommendation spectrum includes this distinction for a reason: told to avoid is still a mention.
  • No mention: The answer never names your brand.
  • Ambiguous: The answer does not support a clear classification. Park it here rather than forcing it into the positive column.

Keep conditional recommendations separate from explicit picks. Otherwise, “choose this” and “choose this only if” disappear into the same bucket again.

Record citations separately, too. An assistant can cite your page for a statistic while recommending a rival. Onely’s test of 31 best-agency questions found that ChatGPT cited the named brands’ own sites in 119 of 210 answers. A link alone does not settle whether the answer endorsed the brand. That distinction is also the subject of Mentioned vs Cited: The Two Machines Inside AI Search.

3. Build a sample you can inspect

Pick 12 prompts that sound like buyer questions, not requests for a brand biography. Cover your main use cases and the constraints buyers actually add:

  • Budget.
  • Location.
  • Team size.
  • Timeline.
  • The rival they are comparing you against.

For payroll software, best payroll for a 20-person contractor crew is more useful here than tell me about Brand X. The first asks the AI to make a choice. The second practically hands you a mention.

Run each prompt three times across the week, using morning and afternoon observations where practical. Treat the resulting 36 observations as an inspection sample, not a market estimate. The job is to inspect what the score counts, not publish a sweeping share figure.

Keep the controls boring and written down:

  1. Use the same prompt wording every time.
  2. Keep the engine and model consistent where you can, and record what is visible.
  3. Timestamp each observation.
  4. Save the full answer, not just the paragraph containing your brand.
  5. Retain cited URLs separately.
  6. Log unknown settings or metadata as unknown.

If you collect answers manually in ChatGPT, check the memory settings and record the setup you use. Do not assume that a fresh-looking conversation tells you everything about the conditions behind the answer.

Missing metadata is something to disclose, not something to reconstruct from memory.

Check whether the dashboard kept the evidence

Before trusting the headline, ask what sits behind it. For each observation, look for:

  • The exact prompt.
  • The engine and model information available.
  • The timestamp.
  • The full answer text.
  • The cited URLs.

A fresh chat answer cannot replace a missing historical answer. If the dashboard cannot retrieve the records for the week you are auditing, you cannot independently check that week’s score. Mark the limitation and stop that comparison there.

You can still collect a prospective sample. Just do not pretend it reconstructs a past one.

Use groas to inspect records, not improvise a history

Asking ChatGPT whether it recommends you gives you an answer now. On its own, that exchange does not establish a monitoring history or coverage record.

Claude or ChatGPT becomes useful for this audit when it can inspect the tracking data. With groas connected over MCP, your assistant can query the AI-search answers groas collects around the clock, with citations retained for inspection.

I would still make it show its working. Ask for the underlying answers and have it flag unavailable fields rather than fill them in. Use the assistant to inspect evidence, not manufacture continuity. If a record cannot be retrieved, that part of the audit remains unverifiable.

4. Label the answers before reading the headline score

Build one sheet and classify the saved answers before opening the dashboard’s summary. Use these columns:

  • Prompt ID and exact wording.
  • Observation time.
  • Engine and model.
  • Answer reference or link.
  • Classification.
  • A short quotation supporting that classification.
  • Cited URLs.
  • Missing fields or uncertainty.

Label from the full text, not from what you remember the answer felt like. A flattering description can sit immediately before an explicit recommendation for someone else. Read through to the actual choice.

Where possible, ask a colleague to label six to eight answers independently. Compare the classifications. When you disagree, clarify the written rule before finishing the sheet; do not quietly negotiate each awkward answer into the positive column.

The background-mention trap is not just theoretical. The LLM Authority Index found that Life Alert appeared in more than half of pricing answers but won zero recommendations. That is precisely the distinction a mention-only score cannot make for you.

Label first, then look at the headline. If you peek first, you give yourself a number to defend instead of a question to test.

5. Recalculate with visible denominators

Now calculate the rates separately. Business.com’s monitoring guide distinguishes mentions, citations and recommendations. For this audit, split recommendations into explicit and conditional categories as well.

Use successfully observed answers as the denominator:

MetricCalculation
Mention rateUsable answers naming your brand ÷ all usable answers
Explicit-recommendation rateUsable answers classified as explicit recommendations ÷ all usable answers
Conditional-recommendation rateUsable answers classified as conditional recommendations ÷ all usable answers
Citation rateUsable answers linking to your site ÷ all usable answers

Missing answers are not negative answers. Keep failed or unavailable observations outside the usable-answer denominator and show their count beside the rates. Otherwise, collection failures start looking like buyer rejection.

Count neutral and negative appearances as mentions when the answer names your brand. For an ambiguous answer, preserve what you can establish: a visible brand name can count as a mention without becoming a recommendation. Keep the uncertain classification visible.

Then compare like with like. The dashboard comparison must use matching prompts, engines, dates and underlying observations. A manually collected answer and a dashboard answer are not the same record just because the prompt matches.

If the dashboard uses a different definition, document it. That does not automatically make its number wrong. It may make it the wrong number for the claim in your report.

A blended visibility score can stay flat while explicit picks fall, because other components can move in the opposite direction. The practical problem is not that the arithmetic must be broken. One score mixing several outcomes cannot tell you which outcome moved.

6. Read the result without inventing a trend

Keep the interpretation as narrow as the test. Agreement means the dashboard kept its stated meaning for these prompts during this week. It does not establish how every buyer sees your brand.

Use the result to choose the next reporting action:

  • The classifications match the dashboard’s stated definition: Keep using the metric for that purpose, with the definition visible.
  • Neutral or negative mentions support a headline presented as recommendations: Stop calling it a recommendation rate. Keep the tracking; correct the label.
  • The underlying answers are unavailable: State that the score cannot be independently checked from the accessible records.
  • Too many answers remain ambiguous: Tighten the rubric before treating the recommendation split as settled.

Do not turn a small difference between weeks into a performance story. Sistrix showed that a true 25% appearance rate measured with one prompt over 20 days could read anywhere from 0 to 59%. That is why I would use this week as a meaning check, not a performance verdict.

For trend claims, keep the denominator in view. The principle behind Your AI Visibility Score Is a Sample, Not a Ranking applies here: no denominator, no trend claim.

7. Change the reporting contract before changing the content

Keep four core lines on the dashboard: mention rate, explicit-recommendation rate, conditional-recommendation rate and citation rate. Put the definition, usable-answer denominator and missing-observation count beside each. Link to the answers where available.

What the deck calls synergy, I call one number doing three jobs. Separate the jobs.

Do not rewrite pages merely because the split looks less flattering than the headline. Content changes come after the evidence points to a specific failure: a prompt cluster where rivals get the explicit pick and your brand gets the passing mention. The audit tells you where to inspect next. It does not, by itself, diagnose the fix.

To start the inspection, paste this into Claude or ChatGPT with groas connected:

Using only the records you can access, separate explicit recommendations, conditional recommendations and other mentions. Show the denominator and missing records; flag anything you cannot verify.

Read the returned evidence before reading the score. If the assistant cannot answer from stored records, you have found a limit on what you can verify. If it can, you have something more useful than a headline: a checkable split between being named and being picked.

Either outcome changes what you report next month. If explicit recommendations track the headline within this sample, keep the metric under a definition it actually supports. If neutral mentions carry the total, keep the tracking and change the claim.

The dashboard does not need to flatter you. It needs to answer the question you are paying it to answer. Run the check once, and make it earn its name.