Research · Methodology
Answer Tracker methodology: question selection, engines, frequency and scoring
The full method OSKIT Answer Tracker uses to measure citations and mentions in AI answers: question rules, engines, frequency, mention, position and citation scoring, and limits.
- By
- OSKIT team (Afformation)
- Updated
- 2 min read
On this page
Summary
- 30 to 100 questions per client are built on 4 axes, approved by a person and fixed per quarter.
- Each question is asked 3 times a week on 4 API-measurable engines. Copilot and AI Overviews use platform reports.
- Per-question score = mention 1 + citation 2 + position weight (1st = 1 … 5th or lower = 0.2), averaged over 3 runs.
What we measure
Answer Tracker measures whether a client is mentioned, cited, and at what position in AI answers to questions customers are likely to ask. This page publishes the full method. Client results are not published without consent.
1. Question selection
Count: 30 to 100 per client. Below 30, one question moves the citation rate by more than 3 points. Above 100, review time outweighs the value.
| Axis | Values | Weighting |
|---|---|---|
| Service | 3–10 main services or products | By revenue and inquiry share |
| Country / language | Target market languages | By inquiry share |
| Intent | Recommendation, comparison, price, reviews, location | ≥ 40% recommendation and comparison |
| Brand name | Excluded / included | ≥ 70% excluded |
Candidates come from Post Engine’s keyword table, Intake conversation logs and Search Console queries. Only questions approved by the account lead and the client are measured. Sets are fixed per quarter with a change log.
2. Engines
| Engine | Method | Automated |
|---|---|---|
| ChatGPT search | OpenAI Responses API + web_search | Yes |
| Gemini | Gemini API + google_search grounding | Yes |
| Perplexity | Agent API + web_search or Search API | Yes |
| Claude | Messages API + web_search | Yes |
| Microsoft Copilot | Bing Webmaster AI Performance report | Report upload |
| Google AI Overviews / AI Mode | Search Console generative AI report | Report upload |
| Naver AI Briefing | Manual sampling (10–20 questions) | No |
3. Frequency
Once a week per engine, same day and time (Monday 07:00 KST), 3 runs per question, spaced within the day.
4. Records
Each response stores question ID and text, language, engine and model version, run time and index, raw answer text, cited URLs in order, and whether the brand was mentioned.
5. Scoring
| Component | Points | Condition |
|---|---|---|
| Mention | 1 | Official brand name or registered alias in the answer |
| Citation | 2 | A client-domain URL in the citation list |
| Position weight | 1 / 0.8 / 0.6 / 0.4 / 0.2 | Client URL at position 1 / 2 / 3 / 4 / 5+ |
Maximum per run is 4. A question’s score is the mean of 3 runs. The client score is the mean question score divided by 4, as a percentage. Reports lead with the individual metrics: citation rate, mention rate, stable citation rate (cited in all 3 runs), share of voice and the top 10 competing domains.
6. Limits
- 01API answers may differ from consumer apps.
- 02Model changes can shift results. Those weeks are marked.
- 03Questions outside the set may behave differently.
- 04Mention detection is string matching. Aliases are updated quarterly.
- 05Naver AI Briefing is sampled manually and excluded from the score.
7. Cost
100 questions × 4 engines × 3 runs = 1,200 calls a week. As of 18 November 2026 web search costs $10 per 1,000 for OpenAI and Claude, $14 per 1,000 for Gemini after 5,000 free a month, and $5 per 1,000 for Perplexity Search API[1][2][3]. That is about $12–17 a week plus tokens.
8. Where results go
The client portal shows raw answers and citations by date. Weekly Review summarises changes every Monday. Post Engine turns uncited questions into post topics. For AI Overviews and AI Mode the first-party data is the Search Console report, never added to regular totals[4][5].
Questions this page answers
Who picks the questions?
Candidates come from Post Engine’s keyword table and inquiry logs. Only questions approved by the account lead and the client are measured.
What happens when a model changes?
Every response stores the model name, and weeks with a model change are marked and compared separately.
Do you publish client results?
Not without client consent. This page publishes the method only.
Sources
- [1]OpenAI API — Pricing (web search tool)https://developers.openai.com/api/docs/pricing
- [2]Perplexity — Modelshttps://docs.perplexity.ai/getting-started/models
- [3]Claude Platform — Pricinghttps://platform.claude.com/docs/en/about-claude/pricing
- [4]Google Search Central — AI features and your websitehttps://developers.google.com/search/docs/appearance/ai-features
- [5]Google Search Central Blog — Generative AI performance reportshttps://developers.google.com/search/blog/2026/06/gen-ai-performance-reports
Change log
- First published
Summarize in an AI chat
Only the page URL is sent.
A free check shows your search and AI answer visibility
Enter your website address and see search engine and AI bot access and index status right away.
3 minutes · Free · No contract