Confidentiality and provider trust

Privacy is not the whole question

Removing your name from a document does not remove the value of the idea inside it. Personal-data protection and business confidentiality overlap, but they are not the same promise.

What the Navier–Stokes dispute shows about trusting a provider

In September 2026, researchers released an AI-assisted solution to a long-standing problem in fluid dynamics, the Navier–Stokes equations. Soon after, debate broke out over whether the researchers’ confidential work had indirectly influenced OpenAI’s models, after OpenAI independently claimed to have solved the same problem using its own model.

When OpenAI was asked about the researchers’ Navier–Stokes result, its first account said it could not rule out that de-identified data derived from their use of its products had helped improve its models. A follow-up published on the OpenAI page said their prompts from the preceding two months could not have influenced the system, including through training.

The follow-up is a statement about the company’s own investigation. In the published accounts, both the uncertainty and the reassurance came from OpenAI; the customer was not given the underlying evidence to audit independently. That does not establish misuse. It exposes the central problem: the provider holds the evidence needed to test its account.

And de-identification is not the same as removing value. Take the name off a document and it can still hold an unpublished method, a pricing model, a strategy. The things a competitor would most want are usually not the personal-data points that GDPR protects.

What counts as "Customer Data" is decided by the contract

Contracts distinguish between "Customer Data" and other material the system generates. The line is where value begins to escape. Embeddings, intermediate states, cache contents, evaluation logs and derived statistics are not always classed as Customer Data, and a definition written while a product was new rarely anticipates an AI output with commercial life of its own.

The incentive is not neutral. Training data is a valuable asset for an AI company, and user content can be an abundant source of it. A provider that can treat a representation of your input as non-customer data has found a way to use your material without giving you the rights you thought you had. These contracts are written by the party with the strongest reason to widen the definition.

It is worth asking directly which of these are covered: raw prompts, completed outputs, embeddings that encode the meaning of what you sent, the contents of a context or KV cache that holds your material during a request, and aggregated statistics derived from your usage. If the answer is "raw input only", then the most reusable form of your data may already be outside the promise.

The provider can read what you type when it chooses to

OpenAI has demonstrated, in its own words, that it reads prompt content and acts on it. Its June 2026 threat report describes clusters of ChatGPT accounts that it assessed as covert influence operations and banned, quoting and summarising what those users prompted the model to generate. In other words: the provider examined specific users’ prompt activity and took action based on it. The report presents these actions as responses to covert influence operations.

This published example shows that users’ activities are reviewed and that actions can be taken against their interests. In this particular case, that interference was made known to the public. Users have no independent way to know if and when their own data is reviewed, retained, used or acted on behind the provider’s controls.

None of this has to be sinister to matter. Your inputs travel through someone else’s machines, logs and judgements. The Services Agreement sets out data-use promises in section 4.2; it does not give you direct visibility into how those promises are implemented.

Routing services add hands your data must pass through

Aggregators that sit in front of many model providers, such as OpenRouter, make the vulnerable surface larger. OpenRouter states, in its own documentation, that it does not store prompts by default and offers a setting to avoid routing to providers that may train on your data. That setting exists because some of the providers behind a router do train on what they receive.

The audit question remains even with zero-retention settings: what evidence can each party provide that the agreed policy was applied to your request, including derived data and deletion? Documentation describes the promise; it does not by itself verify performance.

So one request can pass through several companies, each with its own retention and training terms, and you are expected to know and trust all of them. A gateway adds convenience, and it also adds parties: more places where your content exists, more policies to trust, more failures you cannot observe.

Company policy does not stop the leak

Many organisations have policies in place: do not put confidential material into an AI service. The policy is necessary, and on its own it is weak. A proxy or gateway can be configured to catch obvious personal data before it leaves, and GDPR’s fines give an organisation a reason to try. OpenAI’s communications privacy policy is an example of the scope and limits a provider writes for itself. But a filter for personal data is not a filter for secrets.

Personal data often matches a pattern; a trade secret is valuable precisely because it is particular — a formula, a contract position, an unreleased design, a bid. A filter for personal data cannot tell a secret from harmless text; a written policy alone cannot prevent their disclosure. The loss that matters most is also the hardest to defend against and the hardest to quantify. A personal-data breach has a fine attached; a competitor who quietly learns your strategy may have taken something that appears on no breach report, and there may be no practical way to prove it happened or get it back.

Bring your own key solves less than it promises

Consider a hardware security module (HSM). An HSM is a tamper-resistant device that performs encryption and decryption while keeping the protected key inside the device. That is strong protection for a key at rest.

The limit is where the device sits. The HSM lives inside the provider’s environment, and to use it, data and keys travel through the provider’s processes. A symmetric key travelling to an HSM can be observed by whoever runs the machinery — the provider operates the very systems that move the key in flight. Encryption inside a system you do not operate is not encryption from that system’s operator.

Personnel promises have the same shape. An EU-citizen-only hiring policy says something about who is employed; it says nothing about what a US parent company can supply: a key, a maintenance interface, a design change, none of it requiring an EU employee. These are real mitigations. They narrow the attack surface. They do not remove the trust that remains — the trust that a provider will, in practice, decline to do something it is able to do.

What you can actually verify is the question

None of this means providers are hostile. It means confidentiality rests on a mix of what you can inspect, what you must take on assurance, and what remedy exists if the assurance fails. For each layer — a model provider, an aggregator, a key-management feature, a personnel policy — ask which evidence you can inspect rather than counting another policy as another guarantee.

That is the gap a dedicated deployment addresses. With hardware dedicated to one customer, you set the data flows, logging and access arrangements, and your content does not pass through an operator’s shared services. It does not erase software, security or human risk. It moves the boundary you can actually verify to where your content actually is.

Sources and further reading

Discuss your needs