AI Visibility Competitor Benchmark: A Hands-On Guide

To build an AI visibility competitor benchmark, test your brand and rivals against the same buyer prompts in ChatGPT, Perplexity, Gemini and Claude. Record who gets mentioned, who is recommended, where each brand appears and which sources support the answer. Repeat the test consistently so you can tell a genuine change from a one-off response.

This tutorial gives you a practical process, from choosing competitors and prompts to calculating share of model and planning the work that follows. If you need a wider view of how AI search visibility fits into your marketing, explore SearchFIT’s AI search resources and our guide to AI search visibility content strategies.

The goal: a benchmark you can repeat

The goal is not to prove that one model is better than another, or to collect a large pile of screenshots. It is to create a fair comparison that answers four questions:

• How often does each engine mention your brand for relevant buying questions?
• How often does it recommend you as a leading choice, rather than simply listing you?
• Which competitors appear when you do not?
• What sources, claims or product details seem to shape the answer?

You will finish with a baseline for each engine, a list of prompts where competitors have an advantage, and a prioritised set of actions. A spreadsheet is enough to run the first benchmark. A monitoring platform such as SearchFIT can help teams manage ongoing AI visibility work and reporting as the programme grows.

Treat this as a structured observation, not a perfect measurement of every AI answer a customer might see. Outputs can vary with the model, browsing mode, location, account settings and the wording of a prompt. The point is to control those factors as much as you reasonably can and document the ones you cannot control.

What you need before you start

Gather these items before running the first prompt:

1. A clear category and use case. For example, do not benchmark merely against project management software. Define the buyer and job, such as project management software for distributed product teams.
2. Your brand and three to five direct competitors. Choose alternatives a buyer could realistically evaluate alongside you. Avoid adding every large company in the market just because it is well known.
3. A prompt set. Start with around 30 to 50 prompts if you have time to evaluate them carefully. The sample structure below uses 50, but you can begin smaller and expand once the process works.
4. A recording sheet. Use one row per prompt, engine and run. Include the prompt, date, engine and mode, brand mentions, recommendation, order, citations, factual issues and notes.
5. A consistent run protocol. Decide who will run the prompts, which interface or API they will use, whether browsing is enabled, and how often you will repeat the test.

Write down any assumptions in the sheet. If you use a consumer chat interface, record whether you were logged in and whether web search or grounding was active. If you use an API, record the model name and settings. Do not compare a browsing-enabled answer from one engine with a non-browsing answer from another as if the conditions were identical.

For teams reporting to clients or leadership, agree on the interpretation before collecting results. SearchFIT’s AI visibility measurement and reporting articles can help you shape the wider reporting routine, while the pricing page is useful if you are deciding whether ongoing monitoring should move beyond a spreadsheet.

Step 1: define the category and competitor set

Start with a one-sentence scope statement. It should name the buyer, the problem and the category being evaluated.

Example: We are benchmarking customer support platforms for mid-market ecommerce teams that need helpdesk, order visibility and integrations with their online store.

This sentence helps stop the benchmark from drifting into prompts that do not represent your market. A brand may be a real competitor in a broad category but irrelevant for a particular buyer or use case.

Next, choose three to five direct competitors. Use a combination of:

• Brands your sales team often sees in deals.
• Products that appear in customer comparisons or review sites.
• Alternatives customers mention when they leave or switch.
• Brands an AI engine already names for your category.

Keep a separate list of indirect competitors if they solve the same problem in a different way. Do not quietly add them to the main score halfway through a reporting period. If your scope changes, note the change and treat the next result as a new baseline or an explicitly revised comparison.

Make a short reference sheet for every brand. Include its official name, common abbreviations, product names, relevant product tiers and known naming variations. For your own brand, add verified facts about features, target customers, pricing model and availability. This makes it easier to distinguish an accurate mention from a name match or an answer that describes the wrong product.

Step 2: build a balanced prompt set

A useful benchmark covers different stages of a buying journey. It should not consist only of prompts that include your brand name. If a prompt already names you, it cannot show whether the engine would surface you without being prompted.

For a 50-prompt set, use four groups:

These counts are a practical starting design, not a claim that every category needs exactly 50 prompts. If you are an agency with several clients, a smaller, carefully reviewed set for each client can be more useful than a large set full of near-duplicates.

Write prompts like real buyers

Keep each prompt clear and self-contained. Avoid leading the engine towards your desired answer. Vary the buyer, situation and constraint without changing the underlying question so much that the results become incomparable.

Examples for a B2B software category:

• What are the best platforms for managing customer support for a growing ecommerce team?
• Which customer support tools work well for a team that needs order data in the helpdesk?
• What are alternatives to [Competitor A] for a mid-market retailer?
• Compare [Your Brand], [Competitor A] and [Competitor B] for a team that needs fast implementation.
• Which platforms offer role-based access and integrations with common ecommerce systems?

Examples for an ecommerce brand:

• Which Australian brands sell refillable skincare for sensitive skin?
• What are good alternatives to [Competitor A] for fragrance-free moisturiser?
• Compare [Your Brand] and [Competitor B] for shoppers looking for a travel-size product.
• Which brands offer a product suitable for [specific, substantiated need] and deliver in Australia?

Use only product claims and constraints that matter to your buyers. If a requirement is not supported by your product, do not include it to make the prompt sound sophisticated. A benchmark should reveal where you fit, not manufacture a reason for the model to select you.

Make prompts distinct. These are likely duplicates: What is the best CRM for a small team? and Which CRM is best for a small team? Keep one, then use the other prompt slot to explore a different buyer, use case or constraint. A varied set gives you more useful diagnostic information.

Freeze and label the prompt set

Give each prompt a stable ID, such as DISC-01 for category discovery or COMP-03 for a competitor alternative. Keep the exact wording in a master sheet and do not edit it during a measurement cycle. If you add or remove prompts, record the change and compare results only on prompts that exist in both periods, or clearly label the new set as a new baseline.

Step 3: set up a fair test across engines

Run the same prompt wording separately in ChatGPT, Perplexity, Gemini and Claude. Choose the closest comparable mode available in each product and note whether the answer used web search, sources or other grounding. Do not assume that a model name alone describes the test: consumer products can switch models or retrieval features over time.

The answer engines do not all work or present sources in the same way. For example, an answer with visible links is not directly comparable to an answer that offers no citations, and a search-grounded response may draw on different material from a response generated without browsing. The overview of how ChatGPT Search, Perplexity and Gemini differ is useful context, but your actual test conditions should come from the interfaces you use on the day.

Use a clean conversation for each prompt. Do not continue a thread where the model has already learned your preferences or brand details. Where practical, use a logged-out or clean profile. If you use an API, keep its settings consistent and record them. A temperature setting of zero may reduce some variation in an API workflow, but it does not make results identical across engines or remove retrieval changes.

For each response, save the full text or a screenshot, the date and time, the engine and model or mode, the prompt ID and any visible citations. Capture the original answer before you classify it. This lets another person review your scoring and prevents an early interpretation from replacing the evidence.

Run the full prompt set once, then repeat it in a second batch after seven days. Treat that interval as a practical check for short-term variation, not a guarantee that the models or search indexes have stayed unchanged. For a monthly baseline, repeat the same set under the same conditions each month. If a model, interface or browsing option changes, record the change rather than pretending the results are perfectly continuous.

Step 4: score mentions, recommendations and position

A raw mention count is not enough. An engine might name your brand as the top choice, mention it neutrally in a long list, or say it is unsuitable. Record those outcomes separately.

Use a spreadsheet with at least these columns:

Set the scoring rules before you inspect the results. A simple recommendation scale works well:

• Endorsed: The answer identifies the brand as a suitable or preferred choice for the stated need.
• Neutral: The brand appears in a list or comparison, without a clear positive or negative judgement.
• Negative or excluded: The answer warns against the brand, says it does not fit, or omits it from a question where it would reasonably qualify.

Do not score a simple appearance as an endorsement. Likewise, do not mark a brand as a negative recommendation just because the answer lists another brand first. Record what the response actually says, then apply your definitions consistently.

For position, note the order in which brands are presented. If the response gives explicit labels such as best overall or best for small teams, record those labels as well as the order. If it mentions several brands in a paragraph without a clear sequence, mark position as unclear instead of inventing a ranking.

A response is eligible for the main category benchmark when it answers a relevant category question and gives the engine a reasonable opportunity to name providers. A prompt asking for instructions, a definition or support for an existing customer may not be a qualified category response. Decide those rules in advance and keep excluded responses in the sheet with a reason, rather than silently deleting them.

Step 5: calculate share of model and recommendation rate

Use a formula that answers a clearly defined question. One practical metric is mention rate: the percentage of eligible responses in which a brand appears.

Mention rate = eligible responses mentioning the brand ÷ total eligible responses × 100

Calculate it separately for each engine. If you have 40 eligible responses for ChatGPT and your brand appears in 12, your mention rate is 30% for that sample. This example shows the arithmetic only; it is not a performance benchmark.

You can calculate a share of model for a fixed set of brands and responses. One useful version is the percentage of eligible response opportunities where a brand appears:

Share of model = eligible responses mentioning the brand ÷ total eligible category responses × 100

This is effectively a mention rate when each prompt and run counts once. State your denominator clearly. If the same response names several competitors, each can earn a mention, so brand percentages can add up to more than 100%. That is expected and does not mean the calculation is wrong.

A different metric is relative citation share: a brand’s citations divided by all tracked competitor citations. That describes how citations are distributed among the selected brands, not how often the brand appears in answers. Keep these definitions separate. Siteimprove’s overview of answer engine optimisation metrics discusses measures such as share of voice, citation rate and sentiment; NetRanks’ guide to AI share of voice is another reference for framing this type of measurement.

Then calculate recommendation rate:

Recommendation rate = eligible responses with an endorsement for the brand ÷ eligible responses × 100

You can also report top-choice share if the answer explicitly identifies one top recommendation:

Top-choice share = responses naming the brand as the top choice ÷ eligible responses × 100

Use a separate measure for recommendation quality if you need to distinguish positive endorsement from neutral inclusion. Always show the count alongside the percentage. For instance, report 12 of 40 responses, not only 30%. A small sample can make a percentage appear more decisive than it is.

Report the results by engine first. An all-engine total can be useful as a summary, but it should not hide differences. Outrank’s AI visibility checker and citation benchmark reports that 79% of cited domains in its benchmark were cited by only one AI engine. That finding is a reason to inspect engine-level patterns rather than assume one combined score tells the whole story. It is not a prediction of how your own results will behave.

Step 6: inspect citations and source gaps

For every response with a visible citation, record the exact URL and the domain. Also note which brand or claim the source appears to support. A citation list without context can be misleading: a source may support a general category explanation, not the specific recommendation you are evaluating.

Group sources into practical types:

• Owned: Your product pages, documentation, comparison pages or company announcements.
• Earned editorial: Independent reviews, news coverage, expert round-ups and category guides.
• Review platforms: Customer review and software comparison sites.
• Community: Discussion threads and user forums.
• Other: Research, directories, retailers or sources that do not fit the groups above.

Then look for patterns in prompts where a competitor wins and your brand is absent or poorly described. Ask:

1. Which sources appear repeatedly across those answers?
2. Are those sources factual and relevant to the buyer’s decision?
3. Does the competitor have a clearer explanation of a feature, use case or price?
4. Is your brand missing from an independent comparison or review source?
5. Is the model using an outdated page or a claim that conflicts with your current information?

Do not assume that a cited URL caused the answer or that publishing a similar page will make your brand appear. Treat citations as evidence about the material visible in that answer, not proof of the model’s complete reasoning. The goal is to find credible gaps you can address, such as missing product information, conflicting claims or a lack of useful independent coverage.

Separate source opportunities by how much control you have. You can update your own pages and documentation directly. You can pitch accurate information to relevant publishers, but you cannot control whether they include you. Community conversations should be approached by being genuinely useful and following each forum’s rules, not by posting promotional material to manufacture mentions.

Step 7: check accuracy and sentiment

A brand mention can still be a problem if the answer gives the wrong price, feature set, audience or limitation. For each brand, mark factual details as correct, incomplete, outdated or unsupported. When possible, verify the statement against a current first-party page and note the date you checked it.

Record sentiment in plain language rather than relying on an unexplained numeric score. For example, write that the answer recommends the product for small teams but warns that it may lack a required integration. This preserves the useful detail and gives the team something specific to investigate.

Prioritise material errors. A minor wording difference is not the same as an incorrect price or a feature claim that could cause a buyer to rule you out. Confirm the correction with the right internal owner before changing site content. A benchmark is a discovery tool, not permission to publish claims that are not true.

Step 8: turn the benchmark into an action plan

Create a short action list from the results. Each action should connect an observed gap to a plausible improvement and an owner. Avoid treating every missing mention as a reason to publish another article.

Prioritise actions by buyer importance, evidence quality and feasibility. A gap on a high-value buying prompt may deserve attention before an isolated mention on an obscure query. An inaccurate product claim may be more urgent than a low mention rate if it is likely to mislead a prospect.

Make the proposed change on the page or channel that best addresses the actual issue. If the missing information belongs in product documentation, improve the documentation. If buyers need a comparison, provide a factual comparison that explains fit and trade-offs. If you lack independent validation, investigate relevant review and editorial opportunities. Do not add unsupported statistics, invented customer claims or competitor comparisons you cannot substantiate.

For more practical guidance on turning visibility measures into a reporting routine, see AEO KPIs for client reporting. Set a review date and use the same prompts and scoring rules to check whether the change corresponds with a measurable shift. A change in results is evidence to investigate, not proof that one edit caused the outcome.

Step 9: report the result without overstating it

A useful benchmark report can fit on one page. Include the date range, tested engines and modes, prompt-set version, number of eligible responses, and any changes in test conditions. Show mention rate and recommendation rate by engine, plus the top citation patterns and the most important accuracy issues.

Include a brief limitations note. For example: results reflect this prompt set and these test conditions; answers may vary by session, model version, location and retrieval state. If an engine did not show citations for a response, report that rather than assuming there were none behind the answer.

Keep a small evidence appendix with the prompt IDs, raw outputs and source URLs. That gives colleagues a way to check the interpretation and makes the next benchmark easier to run. If you are comparing periods, show which prompts and conditions stayed the same. If they changed, identify the change beside the result.

Do not present a single combined score as a definitive measure of AI visibility. A summary number can help with a quick read, but the useful diagnosis is often the difference between engines, prompt types and recommendation categories. The benchmark is most valuable when it tells the team what to investigate next.

A repeatable monthly routine

Use the same operating rhythm each cycle:

1. Confirm that the brand, competitor set and prompt list still reflect the market.
2. Check for model, interface or browsing changes and document them.
3. Run the prompts in clean sessions using the agreed method.
4. Save full answers and citations before scoring.
5. Apply the same eligibility and recommendation definitions.
6. Calculate engine-level rates and compare with the previous comparable run.
7. Review citation gaps, factual errors and important shifts in positioning.
8. Assign a small number of actions, owners and review dates.

Run an additional check after a major product, pricing or positioning change if it could alter how a buyer evaluates you. Keep that check clearly labelled as a targeted test, rather than quietly mixing it into the monthly baseline. Over time, this creates both a consistent trend and a useful record of how individual changes relate to the questions buyers ask.

Your next step

Start with one category, a small set of direct rivals and prompts your buyers genuinely ask. Save the exact wording, run the same test across the four engines, and score mentions separately from endorsements. Then choose one evidence-backed gap to investigate before the next run. For support with ongoing visibility tracking and content planning, explore SearchFIT’s platform and pricing.