Keep app results and API results separate in your reports: a segmentation convention

Keep app results and API results in separate segments in every report, because they are different populations of answers. An API answer is what a model returns when a tool calls it directly; an app answer is what a public chat page shows a visitor. The two can differ for the same prompt, so a number that pools them describes neither. Report each collection channel on its own, add a channel to a trend only with a dated note, and never compare a pooled month with an unpooled one. This post gives you a copyable convention and a migration note template for doing that.

In short

• A collection channel is one provider, platform, surface and collection method. Report each channel as its own segment.
• Pooling channels changes the population behind a number. A trend can move because the mix changed, with no change in how AI engines see you.
• Adding, removing or relabelling a channel is a measurement change. Log it in a dated migration note.
• Compare like with like: the same channel, the same window length, the same prompts.
• Segmentation is a reporting choice you make. It does not tell you which channel your buyers use.

What is the difference between app results and API results?

An API result is an answer returned by an engine's developer interface; an app result is the answer a public chat page or search page shows. Both are observations of an answer, and neither is a ground truth about what any individual buyer saw.

In DiscoveredBy the engines reference draws the line concretely. Four engines (Gemini (API), Perplexity, Claude and Grok) are queried through their own company's API. Four are collected differently: a licensed data provider fetches the answer a person would see. Those four are Google AI Overviews, Google AI Mode, ChatGPT (app) and Gemini (app). The platform records this as the collection method, api for the first four and licensed for the others, and the product shows the second as "Licensed data".

The docs state plainly that API outputs can differ from consumer-app responses. ChatGPT (app) and Gemini (app) are described as what chatgpt.com and gemini.google.com show a session that is not signed in. So do not assume an app channel matches what a signed-in customer would see either. It is a defined, repeatable observation point, and that is its value.

Enlarge image

Which engines have an app channel and an API channel today?

Only Gemini currently has both. The engines reference lists Gemini (API) and Gemini (app) as separate engines, and states that their answers to the same prompt can differ. ChatGPT is collected from the app only: ChatGPT (app) is the only ChatGPT engine.

There is a history behind that. ChatGPT was once also collected through OpenAI's API as a separate engine, ChatGPT (API). It was removed, and its stored answers were deleted. The reference notes that documents kept as they were sent, such as weekly reports, shared snapshots and alerts, may still name it. If you archive old reports, that is the reason a channel name can appear in a document that no longer matches your live project.

Perplexity matters here too. The Sentiment docs note that answers collected through Perplexity's earlier Search API have a different surface from its current Sonar answers, so they form a channel of their own and are not averaged with Sonar. Segmentation is not only an app-versus-API question; it applies to any change in how an engine is collected.

You can read the engine pages for Gemini and ChatGPT, and the glossary entries for ChatGPT (app) and Gemini (app), for the short definitions.

How does pooling change the population?

Pooling adds two populations into one denominator, so the result becomes a weighted blend of two different rates. The blend moves when either rate moves, and it also moves when their weights move.

Take brand visibility, which the Explorer defines as analysed answers naming the brand family divided by analysed answers. If you pool two channels, the numerators and denominators are simply summed, so the pooled figure is a blend that neither channel produced on its own.

That creates two failure modes:

1. Mix shift. A channel joins or leaves the window, and the pooled number moves although neither channel's own rate changed.
2. Hidden divergence. One channel improves while the other declines, and the pooled number stays flat. You conclude nothing is happening.

The Explorer docs address the first case directly. A comparison pooled over every engine includes an engine that started answering during the window (they name ChatGPT (app) or Gemini (app) when it was added) on one side only, and the docs advise breaking the result down by engine, or filtering to one, to compare like for like. The docs also say that two channels of one provider, such as an API call and a browser capture, are two engine values and are never pooled into one row. That keeps them apart as rows. It does not stop pooling in a query: the Explorer docs state that engines are pooled unless engine is a breakdown or a filter, so a query with no engine breakdown gives you a blend, and so does a number copied without its channel.

The channels also do not run the same prompt variants. According to the Explorer and engines docs, Gemini (app) and ChatGPT (app) never run a persona variant or a Chinese variant, and ChatGPT (app) never runs a city target, while the API engines run persona and language variants. Even on identical tracked prompts, then, the two channels can be answering different sets of questions. That is one more reason a gap between channels is a prompt for investigation, not a verdict.

Enlarge image

A segmentation convention you can copy

Write the convention once, put it at the top of your reporting template, and apply it to every chart and table. The point is that no number appears without its channel and its definition.

AI VISIBILITY REPORTING: CHANNEL CONVENTION

1. Segment key
   Every figure is reported per collection channel:
   provider + platform + surface + collection method.
   Example labels: "Gemini (API) · Developer API", "Gemini (app) · Web app · Licensed data".

2. Headline figures
   - One headline per channel. No headline that spans channels unless rule 3 is met.
   - Each figure carries: metric definition, window (dates), prompt set, channel.

3. When a cross-channel figure is allowed
   - The channels are listed beside the figure.
   - The channel mix (answers per channel) is shown beside it.
   - It is never used for a month-to-month comparison unless the channel set
     was identical in both periods.

4. Trends
   - One line per channel, not one pooled line.
   - A channel that starts or stops mid-window is marked on the chart with a dated note.

5. Comparisons
   - Same channel, same window length, same prompt set, same metric definition.
   - If the channel set changed, compare only the channels present in both periods.

6. Small samples
   - Show the answer count beside every rate. Flag any rate built on few answers.

7. Change log
   - Any channel added, removed or relabelled gets a migration note (template below)
     before the next report is sent.

Two details make this practical inside DiscoveredBy. First, the Explorer accepts engine as a breakdown or filter, and you can save the query as a view if you have owner or editor access, so the segmented layout is repeatable. Second, the exports carry the platform, surface, collection method, provider and model as columns on the daily metrics dataset, so a spreadsheet built from the exports can be segmented on the same fields. Read the export docs before you build on them: each daily row counts every attempted response in its slice, including failed ones, while several other columns count completed answers only.

The migration note template

A migration note records a measurement change so that a future reader can tell a real movement from an artefact. Write one every time a channel is added, removed, relabelled or replaced.

MEASUREMENT MIGRATION NOTE

Date of change:            ____
Project / client:          ____
Channel affected:          ____ (provider / platform / surface / collection method)
Change type:               added | removed | replaced | relabelled | split
Old channel(s):            ____
New channel(s):            ____
First day of new data:     ____
Last day of old data:      ____
Stored history:            kept | deleted | partly kept   (say what happened to old answers)

Comparable windows
  Before the change:       ____ to ____   (channels: ____)
  After the change:        ____ to ____   (channels: ____)
  Overlap, if any:         ____ to ____   (both channels running)

Effect on reported figures
  Figures that continue unchanged:   ____
  Figures that break comparability:  ____
  Charts annotated:                  yes | no

How to read the earlier report:
  ____

Who was told, and when:    ____
Sign-off:                  ____

Fill in the overlap line whenever you can. A window where both channels ran is the only place you can look at them side by side without assuming anything about the past.

Worked example: a pooled number that fell with no change in visibility

Illustrative example: Quillstone and its competitors are fictional, and the numbers are made up to show the method.

Quillstone sells document-review software to legal and compliance teams. In March its team reports brand visibility on Gemini (API) only: 40 analysed answers, 26 naming Quillstone, which is 65%.

In April the team adds Gemini (app) to the project. The April data, by channel:

A pooled report would say visibility fell from 65% to 50%. Nothing in the Gemini (API) rate moved: it is 65% in both months. What changed is that a second population, with a lower rate, entered the denominator.

The segmented report says something different and more useful: Gemini (API) is steady at 65%, and Gemini (app) is a new baseline at 35%. That gap is a question to investigate, not a conclusion. It is an observation about two answer populations, and it does not show why they differ. The team can now decide what to read next; see also investigating a disagreement between Gemini app and API answers.

The migration note for April would record: change type "added", first day of new data, no old channel removed, and a line stating that the March-to-April change in any pooled figure is not comparable. A note that short would have prevented the wrong headline.

Common mistakes and what segmentation cannot tell you

Segmentation makes a report more honest; it does not make it complete.

• Treating one channel as "the truth". Neither an API answer nor an app answer is a definitive record of what buyers see. The app channels are described as a session that is not signed in, and the docs do not claim they match a signed-in user's experience.
• Comparing a pooled figure with a segmented one. A quarter-on-quarter chart that switches from pooled to segmented halfway is not a trend.
• Dropping the answer count. A rate for one channel built on a handful of answers can swing widely. Show the count.
• Assuming metrics mean the same thing on every channel. Some metrics are defined only for engines that report the underlying data. For example, the metrics docs say Google AI Overviews, Google AI Mode, ChatGPT (app) and Gemini (app) report only the sources an answer cites, never a list of pages retrieved, so domain coverage (retrieval coverage) has no value for them. Sentiment metrics in the Explorer are read within one channel: a query that spans engines is refused. Check the metrics reference before you put two channels side by side on one measure.
• Reading a channel gap as a cause. A difference between Gemini (API) and Gemini (app) shows the two populations differ. It does not show why the model produced either answer.
• Forgetting old documents. Reports kept as sent may name a channel that no longer exists in the project. Keep the migration notes with the archive.
• Splitting too far. Channel is the right cut for the reasons above. If you also split by country, persona and language in every table, most cells will have too few answers to read. Segment by channel first, then add other cuts only where you need them.

For the wider habit of recording changes, see keeping a measurement change log. To keep the prompt set steady while you do it, see using a fixed prompt cohort for month-to-month comparisons.

Frequently asked questions

Can I ever show one combined AI visibility number?

Yes, if you state exactly which channels it covers and show the answer count for each. Do not use it for a month-to-month comparison unless the channel set was the same in both months. When in doubt, lead with the per-channel figures.

Why does a channel's name include "Licensed data"?

It is the label for the collection method licensed. In the product it marks engines whose answers are fetched by a licensed data provider rather than through the engine's own API. The engines reference lists which engines use it.

Does DiscoveredBy pool app and API answers automatically?

It can, if the query is left unsegmented. The docs say two channels of one provider are two engine values and are never pooled into one row, but they also say engines are pooled in an Explorer query unless engine is a breakdown or a filter. Break the query down by engine, or filter to one, before you copy a number.

What should I do with history from a channel that was removed?

Note it in the migration record, including whether the stored answers were kept or deleted. For ChatGPT (API), the docs state its stored answers were deleted when it was removed, so there is no history to compare against.

How much history do I need before comparing two channels?

There is no fixed rule in the docs. Compare over a window where both ran, show the answer counts, and treat small samples as provisional. The Explorer marks a rate or position built from fewer than 30 observations as provisional for the same reason.

Next step

Open the Explorer, break brand visibility down by engine for your last 28 days, and save that layout as your reporting baseline. Then paste the convention and migration note above into your reporting template. You can start in the app at https://app.discoveredby.ai/auth, and the Explorer docs explain each dimension you will use.