Ask an assistant the same question twice and you can get two different answers. That single fact invalidates most of the AI visibility reporting currently being sent to clients.
One run is a possibility, not a position
Because generated answers vary between runs, a screenshot showing your brand named once tells you almost nothing. Serious measurement runs each prompt repeatedly and reports the appearance rate as a range, so real movement can be told apart from noise.
Mentions and citations are different outcomes
Being named inside an answer and having your domain cited as a source carry different value and often move independently. Blending them into one number produces a chart that always looks like progress.
The prompt set must be fixed
Agree the prompts before the work starts and do not change them. Adding a new prompt mid programme, one that happens to favour a page you just published, is the most common way visibility reporting is quietly manufactured.
Competitors belong in the report
If your appearance rate rose four points and every competitor rose six, you lost ground. Share of voice against three named competitors is the only version of this number that means anything.
Check the crawler logs
Before analysing appearance rates, confirm the AI crawlers actually visited. Which bots came, which pages they read, how often. If they never arrived, the rest of the report is speculation.
Report the failures
Every monthly report should contain something that did not work. A report that is all good news is a marketing document, and you paid for a measurement.
These rules cost us the occasional flattering slide. They also mean that when a number moves, both sides can believe it.
See where your own business stands
The free scan runs the technical half of this article against your site automatically.
Run the free scan