# API or Interface: That Is the Question

Rodrigo Beckmann · First Answer · Version 1.0 · Published 2026-04-30 · HTML version: https://rodrigobeckmann.com/en/research/api-or-interface/

*Methodological study · FA-MET-02.* Why the same question returns different brands depending on how it is asked.

1,000 runs via API + 1,000 via the ChatGPT interface · [Full study (PDF)](https://rodrigobeckmann.com/pesquisa/FA-MET-02_api-vs-interface_v1.0_2026-04-30_EN.pdf)

## In short

Measuring through the API is not measuring what consumers see. With the same question and the same search tool, the API and the ChatGPT interface only agree on the core: of the 15 most relevant brands, 12 change significantly between environments.

## Key numbers

- **12 of 15**: top brands with a significant difference between API and interface
- **82 pp**: largest presence gap for a single brand between environments
- **64%**: of the 56 brands observed appear in only one environment
- **77.5% × 99.5%**: of runs used web search: API and interface

## From fourth place down, brands swap positions

*Same question, 1,000 runs in each environment.*

| Brand | API | ChatGPT interface |
| --- | --- | --- |
| Brand A | 100.0% | 77.5% |
| Brand B | 100.0% | 99.9% |
| Brand C | 100.0% | 100.0% |
| Brand D | 11.1% | 92.6% |
| Brand E | 7.8% | 90.2% |
| Brand F | 66.5% | 6.4% |
| Brand G | 44.5% | 0.2% |
| Brand H | 37.7% | 6.1% |
| Brand I | 20.3% | 0.4% |
| Brand J | 0.0% | 17.1% |
| Brand K | 15.7% | 0.1% |
| Brand L | 14.3% | 2.3% |
| Brand M | 9.2% | 8.7% |
| Brand N | 8.8% | 2.7% |
| Brand O | 0.0% | 5.4% |

Brands anonymized. Values: API → interface. Source: FA-MET-02, Table 1.

## What we found

1,000 runs in each environment, in parallel, on the same day, with the same question and web search enabled.

1. **Only the core survives the change of environment.** Two brands stay close to 100% in both. Below them there are inversions: one brand goes from 11.1% in the API to 92.6% in the interface; another, from 66.5% to 6.4%.
2. **The brand universes barely overlap.** The API cited 39 brands and the interface 37, but only 20 appear in both. Neither environment contains the other.
3. **Web search explains part of the difference.** The interface searched in 99.5% of runs; the API, in 77.5%. Within the API itself, some brands only appear with search and others only without it.
4. **Forcing search in the API would not fix it.** Comparing only runs with search in both environments, the divergence remains: one brand sits at 93.1% in the interface and 0.3% in the API.
5. **When both search, they search differently.** The API reads 22.6 URLs on average and runs 1.55 queries per answer; the interface, 16.9 URLs in one query.

## 56 brands, little overlap

Distinct brands cited in 1,000 runs per environment.

- 19 API only
- 20 in both
- 17 interface only

## Why it matters

The API is an integration gateway for developers, not where consumers meet the AI.

Anyone who measures brand presence via the API and presents the result as what users see is measuring a different system. Every monitoring program must state the environment it measures, and swapping one for the other is not fixed by a configuration tweak.

## Method, in short

Collection designed to use the same model in both environments; logged out, the interface does not allow the model to be fixed or recorded. Search enabled, default temperature and no custom system instruction. 2,000 runs in parallel, within a window of a few hours, each in a new conversation. Differences assessed with 95% intervals (Wilson and Newcombe).

## Limitations

One category, one model and one date. The mechanism should repeat in other contexts; which brands rise or fall should not. Logged out, we cannot rule out that part of the divergence comes from the interface’s model routing. The study does not test why search differs between environments, only that it does.

## Read the full study

The five findings in detail, with figures, tables, a study sheet and references. [Full study (PDF)](https://rodrigobeckmann.com/pesquisa/FA-MET-02_api-vs-interface_v1.0_2026-04-30_EN.pdf)

## How to cite

Beckmann, R. (2026). *API or Interface: That Is the Question: Why the same question returns different brands depending on how it is asked* (FA-MET-02, version 1.0). First Answer. https://rodrigobeckmann.com/en/research/api-or-interface

## Related study

[FA-MET-01 — How to Measure What an AI Recommends](https://rodrigobeckmann.com/en/research/how-to-measure-what-an-ai-recommends/): How many queries it takes to tell an isolated observation from a real pattern.