Blog · Measurement
Measuring AI answer citations through APIs: fields, prices and limits per engine
How to ask the same questions repeatedly through the official ChatGPT, Gemini, Perplexity and Claude APIs, log cited URLs, estimate cost and account for differences from the apps.
- By
- OSKIT team (Afformation)
- Updated
- 2 min read
On this page
Summary
- Consumer AI apps cannot be reproduced exactly through APIs. Measurement uses probes: official APIs with web search tools.
- 100 questions × 4 engines × 3 runs = 1,200 calls a week, about $12–17 a week in search fees.
- For Copilot, the first-party source is Bing Webmaster’s AI Performance report. For AI Overviews, it is Search Console’s generative AI report.
Apps cannot be reproduced through APIs
The ChatGPT app, the Gemini app, Google AI Mode and Naver AI Briefing change their answers by location, chat history and model version. No official API reproduces those screens. AI citation tracking therefore uses probes: ask the same questions repeatedly through each company’s official API with its web search tool and log the cited URLs in the response.
The result is “the answer you get through the API”. Every report says it may differ from the app.
Measurement method and citation field per engine
| Target | Method | Citation field | Price (2026-11-18) |
|---|---|---|---|
| ChatGPT search | OpenAI Responses API + web_search | url_citation, web_search_call.action.sources | $10 per 1,000 searches + tokens[1] |
| Gemini / Google grounding | Gemini API + google_search | url_citation, search queries | 5,000 free per month, then $14 per 1,000 |
| Perplexity | Agent API + web_search, or Search API | search_results, [web:n] in text | $0.0025 per web_search / $5 per 1,000 Search API calls[2] |
| Claude | Messages API + web_search | web_search_result_location | $10 per 1,000 + tokens[3] |
| Copilot / Bing AI | Bing Webmaster AI Performance report | Citations and cited pages | Free[6] |
| AI Overviews / AI Mode | Search Console generative AI report | Impressions per page | Free, no API confirmed[5] |
| Naver AI Briefing | No official API | Manual sampling | – |
Perplexity ended support for Sonar Chat Completions on 27 September 2026, so new tracking uses the Agent API or Search API[2]. Gemini grounding is not the AI Overviews screen. For AI Overviews the first-party data is the Search Console report[4][5].
Building the question set
Use 30 to 100 questions per client. Combine four axes: service or product, country and language (ask Japanese questions in Japanese), intent (recommendation, comparison, price, reviews, location), and whether the brand name is included. Questions are generated from Post Engine’s keyword table, approved by a person and then fixed. Changing questions often breaks time series.
Repetition and metrics
Each question is asked 3 times a week per engine and recorded as “cited in N of 3 runs”.
| Metric | Formula |
|---|---|
| Citation rate | Questions citing your domain ÷ all questions |
| Mention rate | Questions mentioning your brand ÷ all questions |
| Share of voice | Citations of your domain ÷ all citations for those questions |
| Average position | Mean position of your URL in the citation list |
| Competing domains | Other domains cited most often |
Store the raw answer, cited URLs, question, date, engine and model name together.
Cost
100 questions × 4 engines × 3 runs = 1,200 calls a week. At $10–14 per 1,000 search calls that is about $12–17 a week plus tokens[1][3]. Use small models: the goal is to see which URLs the search tool retrieves.
Limits stated in every report
- Results may differ from the apps (model, personalisation, location).
- A model update can change results for the same questions. Model names are logged and changes are marked on charts.
- Questions outside the set may behave differently.
- Naver AI Briefing is outside automated measurement.
In OSKIT
Answer Tracker runs this method. The client portal shows the raw answer and cited URLs per question and date, and Weekly Review summarises changes every Monday.
Questions this page answers
Why ask each question 3 times?
AI answers vary for the same question. Recording how many of 3 runs cite you reduces the effect of a single random result.
Do API results match the ChatGPT app?
They may differ because model, personalisation and location differ. Reports state this.
Is Naver AI Briefing measured automatically?
There is no official API, so it is not collected automatically. It is recorded by manual sampling.
Sources
- [1]OpenAI API — Pricinghttps://developers.openai.com/api/docs/pricing
- [2]Perplexity — Modelshttps://docs.perplexity.ai/getting-started/models
- [3]Claude Platform — Pricinghttps://platform.claude.com/docs/en/about-claude/pricing
- [4]Google Search Central — AI features and your websitehttps://developers.google.com/search/docs/appearance/ai-features
- [5]Google Search Central Blog — Generative AI performance reports (2026-06)https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports
- [6]Search Influence — Bing AI Performance report and Copilot citationshttps://www.searchinfluence.com/blog/bing-ai-performance-report-copilot-citations/
Summarize in an AI chat
Only the page URL is sent.
A free check shows your search and AI answer visibility
Enter your website address and see search engine and AI bot access and index status right away.
3 minutes · Free · No contract