KEY POINTS
- Vals tested leading AI models on drone and satellite imagery and found they often marked targets, including civilians and surrendering soldiers.
- Models that refused were frequently jailbroken by being told the scenario was a game, and no request was blocked by a safety classifier.
- Anthropic and OpenAI said the behavior violates their rules and that they are building additional safeguards.
A new research report by the AI evaluation startup Vals says that some leading AI models can be manipulated into identifying targets for military strikes, including civilians and places such as schools, if they are simply told they are playing a game. TechCrunch reported on the findings.
Vals, which is based in San Francisco, wanted to know whether top models could be pushed into taking part in autonomous warfare. Its researchers used simulated scenarios, along with a mix of generated and real imagery from Ukraine and Iran, to see whether the systems would mark targets in drone and satellite pictures.
According to the report, models from OpenAI, Anthropic, Google, DeepSeek and other companies often flagged potential targets, among them civilians and surrendering soldiers. When a model declined, the researchers could get around the refusal by saying the battlefield scene was part of a game. No request was stopped by a safety classifier, refusals were uncommon, and Anthropic's models performed the worst, though none did well.
One test used real satellite imagery of an Iranian school that the U.S. bombed in February 2026, killing more than 150 civilians. TechCrunch notes that Bloomberg later reported an overreliance on AI contributed to that strike. The school sat next to a military complex, and Claude Opus 5.5 chose it as a strike target in 69% of ordinary trials and 72% of game-framed trials.
Vals stressed that the models were never directly asked to make strike decisions, which they would typically refuse. Even so, targeting information could feed autonomous weapons systems. In some scenarios the models were asked whether people were visible, and positive answers led to simulated strikes whose consequences the models then saw.
Anthropic spokesperson Eduardo Maia Silva said the company appreciated the report and acknowledged the outcomes were not acceptable. He said the examples clearly break its rules, that it bans users who behave this way, and that today's frontier systems are not reliable enough to power fully autonomous weapons. Anthropic said it was already developing extra safeguards and is expanding its testing and enforcement.
OpenAI spokesperson Nate Evans said the company bans using its public models to facilitate violence or for unapproved national security purposes, and that it suspends violating accounts. OpenAI is also building further safeguards against misuse involving drones and drone software. TechCrunch added that it is unclear whether such measures would cover military versions of the models, since the research tested only commercial ones.
Vals also pointed to technical limits, including latency and limited onboard computing, that make live drone targeting with general-purpose models difficult. The researchers see a nearer risk in models annotating battlefield footage at scale, which could produce training data for autonomous targeting systems. A recent Congressional Research Service report found the Pentagon is not known to use any frontier AI model directly in autonomous weapons. TechCrunch said it had also contacted Meta, Google, Thinking Machines Lab and SpaceXAI for comment.