Skip to content

Research · Methodology

Answer Tracker methodology: question selection, engines, frequency and scoring

The full method OSKIT Answer Tracker uses to measure citations and mentions in AI answers: question rules, engines, frequency, mention, position and citation scoring, and limits.

By
OSKIT team (Afformation)
Updated
2 min read
On this page
  1. 01What we measure
  2. 021. Question selection
  3. 032. Engines
  4. 043. Frequency
  5. 054. Records
  6. 065. Scoring
  7. 076. Limits
  8. 087. Cost
  9. 098. Where results go

Summary

  • 30 to 100 questions per client are built on 4 axes, approved by a person and fixed per quarter.
  • Each question is asked 3 times a week on 4 API-measurable engines. Copilot and AI Overviews use platform reports.
  • Per-question score = mention 1 + citation 2 + position weight (1st = 1 … 5th or lower = 0.2), averaged over 3 runs.

What we measure

Answer Tracker measures whether a client is mentioned, cited, and at what position in AI answers to questions customers are likely to ask. This page publishes the full method. Client results are not published without consent.

1. Question selection

Count: 30 to 100 per client. Below 30, one question moves the citation rate by more than 3 points. Above 100, review time outweighs the value.

AxisValuesWeighting
Service3–10 main services or productsBy revenue and inquiry share
Country / languageTarget market languagesBy inquiry share
IntentRecommendation, comparison, price, reviews, location≥ 40% recommendation and comparison
Brand nameExcluded / included≥ 70% excluded

Candidates come from Post Engine’s keyword table, Intake conversation logs and Search Console queries. Only questions approved by the account lead and the client are measured. Sets are fixed per quarter with a change log.

2. Engines

EngineMethodAutomated
ChatGPT searchOpenAI Responses API + web_searchYes
GeminiGemini API + google_search groundingYes
PerplexityAgent API + web_search or Search APIYes
ClaudeMessages API + web_searchYes
Microsoft CopilotBing Webmaster AI Performance reportReport upload
Google AI Overviews / AI ModeSearch Console generative AI reportReport upload
Naver AI BriefingManual sampling (10–20 questions)No

3. Frequency

Once a week per engine, same day and time (Monday 07:00 KST), 3 runs per question, spaced within the day.

4. Records

Each response stores question ID and text, language, engine and model version, run time and index, raw answer text, cited URLs in order, and whether the brand was mentioned.

5. Scoring

ComponentPointsCondition
Mention1Official brand name or registered alias in the answer
Citation2A client-domain URL in the citation list
Position weight1 / 0.8 / 0.6 / 0.4 / 0.2Client URL at position 1 / 2 / 3 / 4 / 5+

Maximum per run is 4. A question’s score is the mean of 3 runs. The client score is the mean question score divided by 4, as a percentage. Reports lead with the individual metrics: citation rate, mention rate, stable citation rate (cited in all 3 runs), share of voice and the top 10 competing domains.

6. Limits

  1. 01API answers may differ from consumer apps.
  2. 02Model changes can shift results. Those weeks are marked.
  3. 03Questions outside the set may behave differently.
  4. 04Mention detection is string matching. Aliases are updated quarterly.
  5. 05Naver AI Briefing is sampled manually and excluded from the score.

7. Cost

100 questions × 4 engines × 3 runs = 1,200 calls a week. As of 18 November 2026 web search costs $10 per 1,000 for OpenAI and Claude, $14 per 1,000 for Gemini after 5,000 free a month, and $5 per 1,000 for Perplexity Search API[1][2][3]. That is about $12–17 a week plus tokens.

8. Where results go

The client portal shows raw answers and citations by date. Weekly Review summarises changes every Monday. Post Engine turns uncited questions into post topics. For AI Overviews and AI Mode the first-party data is the Search Console report, never added to regular totals[4][5].

Questions this page answers

Who picks the questions?

Candidates come from Post Engine’s keyword table and inquiry logs. Only questions approved by the account lead and the client are measured.

What happens when a model changes?

Every response stores the model name, and weeks with a model change are marked and compared separately.

Do you publish client results?

Not without client consent. This page publishes the method only.

Sources

  1. [1]OpenAI API — Pricing (web search tool)https://developers.openai.com/api/docs/pricing
  2. [2]Perplexity — Modelshttps://docs.perplexity.ai/getting-started/models
  3. [3]Claude Platform — Pricinghttps://platform.claude.com/docs/en/about-claude/pricing
  4. [4]Google Search Central — AI features and your websitehttps://developers.google.com/search/docs/appearance/ai-features
  5. [5]Google Search Central Blog — Generative AI performance reportshttps://developers.google.com/search/blog/2026/06/gen-ai-performance-reports

Change log

  • First published

Summarize in an AI chat

Only the page URL is sent.

A free check shows your search and AI answer visibility

Enter your website address and see search engine and AI bot access and index status right away.

3 minutes · Free · No contract