KEY POINTS
- HD Moore seeded nearly two million prompts with unique, never-used secrets and tracked where they surfaced.
- Three hosting providers were breached, and stolen AWS keys were used tens of thousands of times in API calls.
- Some resellers logged requests, and a Chinese AI lab fetched secret URLs even from prompts marked as zero data retention.
A security researcher has found that sensitive data sent through the layered network of services behind many AI apps leaks at a high rate. According to Ars Technica, HD Moore, chief research officer at security firm Dragos, planted about two million trackable secrets in prompts and watched where they appeared. He published the results on Thursday.
The test targeted AI routers, also called AI gateways, such as OpenRouter and Vercel. These brokers take prompts from app users and pass them to an available model, choosing by factors such as lowest price or fit for the task. A developer who uses one does not have to contract directly with a major provider such as Anthropic or OpenAI.
Moore built a set of bait items, including Amazon Web Services access keys tied to empty accounts, SSH passwords, unique URLs, DNS names and references to private git repositories. None had been used before, and he controlled the infrastructure that would receive any attempt to use them. Each secret went into exactly one prompt sent to one provider, which let him trace any leak back to a specific link in the chain.
Prompts are encrypted in transit but are decrypted at each hop, Ars Technica noted. A router reads the plaintext to decide where to send it, and the prompt may then go to a pool of models hosted by different companies or to another router. Each of those parties may log the prompt and keep it for as long as its policies allow, or indefinitely.
The most serious finding was the breach of three hosting providers. Attackers who compromised one of them reused a couple hundred of the AWS keys. Moore wrote that every request sent during a few short windows in late October was stolen, including some that the provider rejected before any model ran, which suggests the copying happened at the provider's front door. In the other two breaches, some stolen requests carried a zero data retention flag, which did not protect them.
The people behind the thefts tested the keys, sometimes within hours. They searched S3 buckets, secrets managers and parameter stores for more credentials, tried to enable hosted models in AWS Bedrock, and added their own IAM users to keep access. The bait keys were used tens of thousands of times in API calls. Because secrets sent to two of the providers were used almost at once from the same rented server, the same adversary was probably behind both.
Some resellers also took the data themselves. They attracted traffic with low prices and then turned on request logging. Moore identified them by sending requests to the same model through different sellers; when credentials were reused after going through one seller but not another, the reseller was the source.
Moore also reported scraping of secret URLs in violation of privacy terms. A Chinese AI lab fetched the URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of those marked no data collection. The scraper usually appeared two or three days later, and it also showed up when US hosting providers resold the lab's models. One US provider crawled a secret URL more than two months after a single call.
For defenses, Moore said hosting models on premises is the most effective but costly. The next best choice is a first-tier provider with one contract and one privacy policy, though that cannot guarantee no leaks. He also advised users to choose upstream providers themselves rather than accept default reseller pools.
