Engines and measurement

The engines we measure

DiscoveredBy measures five engines: ChatGPT, Gemini, Perplexity, Claude and Grok. Every one of them is queried through its official API, never by scraping a browser session. In the platform's own bookkeeping this is recorded as the collection method, and all five engines share the same value: api. PROVIDER_PROVENANCE in api/services/collection.py stamps that value on every one of the five provider rows.

The word the platform uses to track an engine internally is not always the name of the company running it:

Engine you see	Internal slug	Company
ChatGPT	openai	OpenAI
Gemini	gemini	Google
Perplexity	perplexity	Perplexity
Claude	anthropic	Anthropic
Grok	grok	xAI

grok runs on xAI's infrastructure, and gemini runs on Google's: the slug names the API being called, not necessarily the brand a reader might expect from the word alone.

How an answer is labelled

Every prompt execution stamps its own copy of three fields the moment it runs: platform, surface and collection method (PromptExecution.platform, .surface and .collection_method in api/models/prompt_execution.py). These are copied from the provider catalogue at the moment the execution happens. They are not looked up live through the provider record afterward.

That is why an old answer keeps saying how it was actually collected. If the catalogue changes later, a provider renamed, a new surface added, a collection method swapped, none of that can reach back and relabel an execution that already ran: the row already carries its own copy of what happened, not a pointer to a catalogue entry that keeps moving underneath it.

Which engines run for your project

Which of the five engines actually run for your project comes from two things layered together: what your plan includes, and whether each engine is currently available to run at all.

Your project's plan lists which providers it includes (BillingPlan.providers, seeded from DEFAULT_PLANS in api/services/seed.py). Different plans include different subsets of the five engines. What each plan includes is covered on Pricing, not repeated here, so there is one place to keep it correct.

Being on your plan's list is not the same as running today. get_user_provider_ids in api/services/plan.py filters that list twice more: the provider row has to be marked active, and provider_is_available (in api/services/llm.py) has to return true for it, which in turn calls provider_has_api_key to check whether a key for that engine is currently configured on the platform. An engine can be missing from your project for a reason that has nothing to do with your plan: if its API key is not currently configured, that engine will not run for anyone, on any plan, until the key is back in place.

Fan-out

Some engines don't just answer a prompt directly. Internally, they can expand it into several related queries of their own, the way a person might turn one question into a few separate searches, before composing a final answer. DiscoveredBy captures that expansion where a provider exposes it, and shows it alongside the answer.

Fan-out is detail about a single execution, not a separate collected answer. However many queries a prompt expands into, it still counts as one completed execution for that engine on that day: query_executions carries a unique constraint on prompt target, provider and day (uq_qe_target_provider_date in api/models/prompt_execution.py). A prompt target is one prompt tracked in one country, so a prompt tracked in three countries produces three executions per engine per day, not one; fan-out does not change that count either way. Because the metrics on Metrics defined that divide by collected or analysed answers are built from this same execution count, a prompt that fans out widely does not, by itself, move one of those numbers differently from a prompt that does not fan out at all. See that page for which metric divides by which population.

Perplexity is the honest exception. DiscoveredBy calls Perplexity's search endpoint directly, rather than asking a general-purpose model that decides for itself whether and how to search, so there is no model-issued sub-query for Perplexity to report (SURFACES_WITHOUT_FAN_OUT in api/services/collection.py). An empty fan-out on a Perplexity answer means not applicable, not that the engine chose not to search: derive_fan_out_state in api/services/fan_out.py checks for this case first and returns not_applicable before anything else, specifically so a Perplexity run is never misread as an engine that answered without searching.

Where a provider does report sub-queries, DiscoveredBy also has a model read the captured ones, up to a limit on how many one run judges, and file each under a category describing its relationship to the parent prompt, stored in a table of its own rather than as a column on the query record, so a bad classification can never corrupt the record of what the engine actually searched for. See Fan-out for the category list, what happens to a query past that limit, how confident the model has to be before a label is shown, and what a run's fan-out looks like when little or nothing was captured for it.

Related

• Fan-out: the eight categories a captured sub-query can be classified into, and what each capture state means
• Metrics defined: what collected and analysed answers mean, and why fan-out does not change either count
• Pricing: what each plan includes
• Troubleshooting: what to check when an engine looks inactive or a number looks wrong