Audit your prompt list before adding more prompts: a method for finding redundancy

To audit an AI tracking prompt list, judge each prompt by what its answers actually contain, not by how its wording looks. Two prompts are redundant when every source and brand that one of them shows regularly is also seen, at least now and then, by another prompt you keep. Check that over a fixed window, protect any prompt that other work depends on, preview what pausing a candidate would remove, and only then pause. Even a clean audit describes the last few weeks of answers. It cannot promise that a paused prompt would never have surfaced something new.

In short

• Audit before you add. A bigger list is not a better list if several prompts keep returning the same sources and brands.
• Compare prompts by their answers, not their text. Two differently worded questions can overlap completely; two similar-sounding ones can not.
• Separate what a prompt sees regularly from what it sees now and then. Judge redundancy on the regular items and keep the occasional ones in view.
• Protect prompts that something else relies on: fact checks, open optimizations, citation gaps in progress, and the only prompt measuring a country, language or engine.
• Preview before you pause, and read the preview as evidence about the past window, not a guarantee about the future.

Why audit a prompt list before adding more prompts?

A prompt is a question you chose to monitor, run against AI engines on your behalf. Adding prompts feels like progress, but each one uses prompt slots, adds answers to every average, and makes month-to-month comparisons harder to explain.

A prompt slot is one unit of your plan's prompt allowance. In DiscoveredBy, a prompt is tracked as one or more targets, each a combination of location, audience and language, and each target uses a slot. A prompt tracked in 3 countries, as General plus 2 personas, in one language, produces 3 x 3 x 1 = 9 targets. The Prompts docs explain the arithmetic. What counts, and how many you have, depends on your plan; see plans and limits.

So the audit question is practical: which prompts could you pause while still seeing everything you regularly see today? If you are still building the list, start with how to choose AI tracking prompts. This post is about maintaining and trimming a list you already have.

Enlarge image

Should you compare prompts by their wording or by their answers?

Compare them by their answers. Wording is a poor proxy, because differently worded questions can draw on the same sources and brands, while similar-sounding ones may not.

DiscoveredBy's Prompt coverage screen works this way. It does not compare prompt text. It looks at the completed answers from the last 28 days, ending yesterday, and asks which prompts could be paused while everything they regularly see is still seen by a prompt you keep. The numbers are computed when you open the page, from answers already stored, and opening it changes nothing.

Coverage is built from two kinds of item:

• Sources: every domain cited in a counted answer, marked as yours, a tracked competitor's, or other.
• Brands: the brands named in analysed answers, meaning answers whose brand mentions have been extracted. Your own brand is one item, and a tracked competitor is one item for its whole family, so a sub-brand counts as its top-level brand.

A manual first pass still helps as a shortlist, not a verdict: read the list for prompts that ask the same thing in different words, then let the answer overlap confirm or reject each suspicion.

What is the difference between regular and occasional coverage?

A regular item is one a prompt shows often enough to count on, and an occasional item is one it shows only now and then. Real answers carry a long tail of domains and brands that appear once or twice and never again. If a single sighting counted, nearly every prompt would be the only one seeing something, and nothing could ever be paused.

The Prompt coverage screen uses two bars, measured per prompt over the window:

• An item is regular for a prompt when it appears in at least 10% of that prompt's answers (and at least 2 of them), on at least 2 different days.
• An item is seen by a prompt when it appears in at least 2.5% of that prompt's answers (and at least 1).

For sources the percentage is of the prompt's answers; for brands it is of its analysed answers. With 140 answers, regular means at least 14 and seen means at least 4. The bars are fixed for every project, and you cannot change them. Because they sit far apart, two prompts that draw on the same sources and brands cover each other even though each day's sample differs.

The docs are explicit about the trade-off. The recommendation guarantees that every item regular for a paused prompt is still seen by a prompt you keep. It does not guarantee the occasional items, and the preview lists those separately for that reason.

Which prompts should you never pause on redundancy alone?

Any prompt that something else depends on. The Prompt coverage screen labels these Protected and never recommends them, whatever they see. The reasons it documents are:

• Checked for facts: the prompt was chosen on Fact check.
• Measuring an optimization: an optimization names it and is still open, or you marked it applied and its outcome is not settled yet.
• Citation gap in progress: the prompt is linked to a citation gap opportunity that is accepted, in progress, implemented or monitoring.
• The last prompt measuring something: it is the only running prompt for a country tracked country-wide, a city, a persona, a language, or an engine it runs. Pausing it would stop measuring that value entirely.

There is a second reason to be careful. Pausing a prompt never settles a measurement on its absence. If the prompts behind a citation gap or an applied optimization have no answers in the period being measured, the check is recorded as no data and rechecked later. That protects you from a false "it worked" or "it failed", but it also means a pause can leave a measurement pending until fresh answers arrive.

Some prompts are also not judged at all yet. A prompt needs analysed answers on at least 14 different days in the window (half of it) before it can be recommended. A prompt you added last week will read "not enough answers yet", and it keeps running, so what it sees still counts when the page decides about the others.

How do you preview what a pause would remove?

Tick the prompts you are considering on the Prompt coverage screen, and the preview shows the effect without changing anything. The recommended set is selected when the page opens; you can tick or untick any running prompt to preview your own selection. A preview covers 1 to 500 prompts.

For the selection, the preview shows:

• Slots freed and answers removed (and how many are left).
• Lost items: every source or brand that is regular for a selected prompt and that no remaining prompt sees, with the number of answers that named it.
• Also seen only here, occasionally: items the selection sees only now and then, that no remaining prompt sees, with a count and the ten most frequent.
• What no kept prompt would still measure: any country, city, persona, language or engine that would lose its last prompt.
• Answers naming your brand and answers citing your domain, now and after the pause.
• Warnings for selected prompts that are protected, not judged yet, have owned pages linked to them, or are the source of an article topic you selected.

Pausing from the screen asks once, then recomputes the preview and pauses only if it still frees the same slots and shows the same losses. If a daily run or a colleague has changed anything in the meantime, nothing is paused and you are shown the fresh preview. Owners and editors can pause; viewers can read the screen and preview. A paused prompt keeps its history and its variants, and you can resume it later if you have free slots.

Enlarge image

The prompt-review checklist

This is the deliverable: work through it in order, once a quarter or before you add a batch of new prompts. Copy it into your own notes.

PROMPT LIST REVIEW
Project: ______________   Reviewer: ______________   Date: ________
Window used (Prompt coverage uses the last 28 days): ______________

1. INVENTORY
[ ] Running prompts: ____   Paused: ____   Slots in use: ____ of ____
[ ] Prompts added in the last 4 weeks (likely "not enough answers yet"): ____
[ ] Prompts with no classification (intent, buyer stage, theme, branding): ____

2. SUSPECTS (a shortlist, not a verdict)
[ ] Pairs that ask the same buying question in different words
[ ] Prompts nobody can explain the purpose of
[ ] Prompts that overlap only because one is a narrower version of another

3. COVERAGE CHECK (Prompt coverage screen)
For each suspect, record its status:
    Recommended / Only prompt seeing N items regularly /
    Overlaps a recommended prompt / Protected / Not enough answers yet

4. PROTECTION CHECK
For each candidate, is it any of these?
[ ] Chosen for fact check
[ ] Named by an open or unsettled optimization
[ ] Linked to a citation gap in progress
[ ] The last prompt for a country, city, persona, language or engine
[ ] Has owned pages linked to it, or feeds a selected article topic
If yes to any: keep it, or pause it deliberately after reading the warning.

5. PREVIEW (select the candidates, read the preview)
Slots freed: ____   Answers removed: ____   Answers left: ____
Lost items (regular): ____________________________________
Occasional items lost: ____   Values no kept prompt would measure: ____
Own brand rate: now ____% -> after ____%   Own domain cited: now ____ -> after ____

6. DECISION
[ ] Pause   [ ] Keep   [ ] Rewrite (the prompt has a purpose but the wording misses it)
Reason in one sentence: ______________________________________
Comparisons affected (month-to-month, alerts): ______________________

7. AFTER THE PAUSE
[ ] Note the date and the set of paused prompts in your reporting log
[ ] Expect brand visibility trends to shift because the mix changed; annotate them
[ ] Recheck in 4 weeks whether anything you cared about stopped appearing

Two parts of the checklist deserve a note. Step 1 uses the four classifications a prompt can carry (intent, buyer stage, theme, and branded or unbranded), which can be filtered on the prompt list; a prompt with no classification is normal, not an error, and it is not the same as one classified "other". If Search Console is connected, the Google demand column can help you decide between two near-duplicates, but it counts what people typed into Google, not what anyone asked an AI assistant, so treat it as a way to rank your own prompts against each other, not as a measure of AI demand.

A worked example

Illustrative example: Quillstone and its competitors are fictional, and the numbers are made up to show the method.

Quillstone sells document-review software to mid-sized legal and compliance teams. Its team tracks 12 prompts and suspects two of them overlap:

• Prompt A: "Best document review software for legal teams"
• Prompt B: "Top contract review tools for compliance teams"

Each prompt has one General target in one country, run on two engines for 28 days, so each has 2 x 28 = 56 answers. The bars are therefore: regular means at least 6 answers (10% of 56 is 5.6, rounded up), and seen means at least 2 (2.5% of 56 is 1.4, rounded up). Rounding up matches the docs' 140-answer example, where 3.5 becomes 4.

On the Prompt coverage screen, the counts of answers naming each item look like this:

Both prompts look like candidates, so the team previews each in turn. Pausing A loses nothing: every item regular for A is seen by B well above the seen bar of 2. Pausing B is different. Docket North sits between the bars for B (3 answers is above the seen bar of 2 but below the regular bar of 6), and no other running prompt sees it, so it would appear in the preview under items seen only here, occasionally. The bars do not guarantee that item, so the preview favours pausing A, and the team records Docket North as a name to watch on B.

Neither prompt is protected: neither is chosen for fact check, neither belongs to an open optimization or a citation gap in progress, and other running prompts also measure the same country, persona, language and both engines, so neither is the last prompt for any of them. Both have analysed answers on more than 14 days, so both are judged.

Now the effect on the headline rate. Across all running prompts there are 600 analysed answers, 180 of them naming Quillstone. That is 180 / 600 = 30.0%. Prompt A contributed 56 answers, 22 naming Quillstone. The preview's after figure is (180 - 22) / (600 - 56) = 158 / 544 = 29.0%. Nothing about the engines changed; the mix of prompts did. The team writes that into its reporting log so nobody reads a 1.0-point dip as a real movement. Overview and Explorer keep counting a paused prompt's earlier answers inside their window, so the change there arrives gradually and can differ from the preview. The docs also note that alerts comparing a window before a pause with one after it can react to the change of mix, not to anything the engines did.

What can this audit not tell you?

It cannot tell you what a paused prompt would have shown in future, and it cannot look inside the models. The honest limits, all stated in the Prompt coverage docs, are these:

• It describes the past window. The evidence is the last 28 days. A paused prompt's later answer could have named something new.
• The bars are fixed. 10%, 2.5% and 14 days are the same for every project; you cannot tune them.
• Occasional items are not guaranteed. Items under the "seen" bar for a paused prompt are not listed at all.
• Items are pooled. Sources and brands are pooled across engines, locations, personas and languages. A prompt can be recommended because another prompt sees a source anywhere, even if not in the same country or on the same engine. Only the last prompt for each country, city, persona, language and engine is protected.
• Frequency is ignored once the bar is met. An item counts as still covered if a kept prompt sees it at the seen bar, however many of its answers the pause removes.
• Your competitors' numbers move too. The preview shows your own brand's rate only; competitors' visibility and share of voice also shift when the prompt mix changes.
• Unanalysed answers add no brands. They count as collected but not analysed, and contribute sources only.

Observed redundancy describes what answers contained, not why the engines behaved as they did, so treat every pause as reversible.

Common mistakes when auditing a prompt list

1. Merging prompts by wording alone. Look at what the answers cite and name before you decide two prompts are the same.
2. Ignoring the mix effect. A trend that dips after a pause may only reflect a different set of prompts. For like-for-like comparisons, see using a fixed prompt cohort.
3. Pausing a prompt that protects a measurement. If it is protected, there is a reason on screen. Read it.
4. Confusing this with a completeness check. Redundancy asks whether two prompts see the same things. Whether your planned prompts and engines actually ran is a different question, covered in what percentage of your monitoring actually ran.
5. Treating a seasonal prompt as redundant in the off season. A prompt that overlaps in one window may not in another; see separating seasonal buying questions.

Frequently asked questions

How many AI tracking prompts should I have?

There is no universal number, and the docs do not give one. The useful test is whether each prompt regularly sees something no other prompt does, or protects a country, language, persona or engine, or supports work in progress. How many you can run is set by your plan's prompt slots.

Is a prompt redundant if it uses different wording?

Not necessarily. DiscoveredBy compares prompts by what their answers cite and name, not by how similar their text is. Two differently worded prompts can cover the same ground, and two near-identical ones can pull different sources.

What does it mean when a prompt is protected?

It means something else depends on its answers, so the coverage screen will not recommend pausing it. The reason is shown under the label, such as being chosen for fact checks, measuring an open optimization, supporting a citation gap in progress, or being the last prompt measuring a value.

Does pausing a prompt delete its history?

No. A paused prompt stops running and keeps its history and its variants, and its variants stop using your slots. Its earlier answers stop counting as coverage on the Prompt coverage screen, though, because that screen is about what keeps running. Resuming needs free slots.

Why did my visibility number change after I paused prompts?

Brand visibility is a share of analysed answers, so removing some prompts changes the mix behind it. Overview and Explorer keep counting a paused prompt's earlier answers inside their window, so the shift arrives gradually and can differ from the preview's after figure. Annotate the change so it is not read as an engine effect.

Why does a new prompt say "not enough answers yet"?

A prompt is only judged once it has analysed answers on at least 14 different days in the 28-day window. Until then it keeps running and is never recommended for pausing.

Next step: run the audit on your own list

Open Prompt coverage in your project (Prompts, then the Coverage tab), read the recommended set and each prompt's status, and preview your shortlist before you confirm anything. If you do not yet have answers to audit, sign in or create an account and let a few weeks of data build first. For the wider picture of how these numbers are defined, see the metrics reference and the AI visibility feature overview.