Skip to content

Blog · Measurement

Measuring AI answer citations through APIs: fields, prices and limits per engine

How to ask the same questions repeatedly through the official ChatGPT, Gemini, Perplexity and Claude APIs, log cited URLs, estimate cost and account for differences from the apps.

By
OSKIT team (Afformation)
Updated
2 min read
On this page
  1. 01Apps cannot be reproduced through APIs
  2. 02Measurement method and citation field per engine
  3. 03Building the question set
  4. 04Repetition and metrics
  5. 05Cost
  6. 06Limits stated in every report
  7. 07In OSKIT

Summary

  • Consumer AI apps cannot be reproduced exactly through APIs. Measurement uses probes: official APIs with web search tools.
  • 100 questions × 4 engines × 3 runs = 1,200 calls a week, about $12–17 a week in search fees.
  • For Copilot, the first-party source is Bing Webmaster’s AI Performance report. For AI Overviews, it is Search Console’s generative AI report.

Apps cannot be reproduced through APIs

The ChatGPT app, the Gemini app, Google AI Mode and Naver AI Briefing change their answers by location, chat history and model version. No official API reproduces those screens. AI citation tracking therefore uses probes: ask the same questions repeatedly through each company’s official API with its web search tool and log the cited URLs in the response.

The result is “the answer you get through the API”. Every report says it may differ from the app.

Measurement method and citation field per engine

TargetMethodCitation fieldPrice (2026-11-18)
ChatGPT searchOpenAI Responses API + web_searchurl_citation, web_search_call.action.sources$10 per 1,000 searches + tokens[1]
Gemini / Google groundingGemini API + google_searchurl_citation, search queries5,000 free per month, then $14 per 1,000
PerplexityAgent API + web_search, or Search APIsearch_results, [web:n] in text$0.0025 per web_search / $5 per 1,000 Search API calls[2]
ClaudeMessages API + web_searchweb_search_result_location$10 per 1,000 + tokens[3]
Copilot / Bing AIBing Webmaster AI Performance reportCitations and cited pagesFree[6]
AI Overviews / AI ModeSearch Console generative AI reportImpressions per pageFree, no API confirmed[5]
Naver AI BriefingNo official APIManual sampling–

Perplexity ended support for Sonar Chat Completions on 27 September 2026, so new tracking uses the Agent API or Search API[2]. Gemini grounding is not the AI Overviews screen. For AI Overviews the first-party data is the Search Console report[4][5].

Building the question set

Use 30 to 100 questions per client. Combine four axes: service or product, country and language (ask Japanese questions in Japanese), intent (recommendation, comparison, price, reviews, location), and whether the brand name is included. Questions are generated from Post Engine’s keyword table, approved by a person and then fixed. Changing questions often breaks time series.

Repetition and metrics

Each question is asked 3 times a week per engine and recorded as “cited in N of 3 runs”.

MetricFormula
Citation rateQuestions citing your domain ÷ all questions
Mention rateQuestions mentioning your brand ÷ all questions
Share of voiceCitations of your domain ÷ all citations for those questions
Average positionMean position of your URL in the citation list
Competing domainsOther domains cited most often

Store the raw answer, cited URLs, question, date, engine and model name together.

Cost

100 questions × 4 engines × 3 runs = 1,200 calls a week. At $10–14 per 1,000 search calls that is about $12–17 a week plus tokens[1][3]. Use small models: the goal is to see which URLs the search tool retrieves.

Limits stated in every report

  • Results may differ from the apps (model, personalisation, location).
  • A model update can change results for the same questions. Model names are logged and changes are marked on charts.
  • Questions outside the set may behave differently.
  • Naver AI Briefing is outside automated measurement.

In OSKIT

Answer Tracker runs this method. The client portal shows the raw answer and cited URLs per question and date, and Weekly Review summarises changes every Monday.

Questions this page answers

Why ask each question 3 times?

AI answers vary for the same question. Recording how many of 3 runs cite you reduces the effect of a single random result.

Do API results match the ChatGPT app?

They may differ because model, personalisation and location differ. Reports state this.

Is Naver AI Briefing measured automatically?

There is no official API, so it is not collected automatically. It is recorded by manual sampling.

Sources

  1. [1]OpenAI API — Pricinghttps://developers.openai.com/api/docs/pricing
  2. [2]Perplexity — Modelshttps://docs.perplexity.ai/getting-started/models
  3. [3]Claude Platform — Pricinghttps://platform.claude.com/docs/en/about-claude/pricing
  4. [4]Google Search Central — AI features and your websitehttps://developers.google.com/search/docs/appearance/ai-features
  5. [5]Google Search Central Blog — Generative AI performance reports (2026-06)https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports
  6. [6]Search Influence — Bing AI Performance report and Copilot citationshttps://www.searchinfluence.com/blog/bing-ai-performance-report-copilot-citations/

Summarize in an AI chat

Only the page URL is sent.

A free check shows your search and AI answer visibility

Enter your website address and see search engine and AI bot access and index status right away.

3 minutes · Free · No contract