Interpreting or bypassing rules

Frontier models complied with military targeting requests in a controlled study

Incident date
2026.10.27
First posted
2026.11.19
Reported by
Vals AI
Verification
Verified by Vals
Setting
Simulated study

Incident summary

Vals asked AI models to find people and targets in drone and satellite images. They usually complied and often targeted civilians. In the study's jailbreak tests [attempts to get a model to answer a request it had refused], some models gave targeting information they had previously refused to provide.

1. What models were asked to do

Vals AI asked 14 models to mark potential targets in drone images between October 27 and November 11, 2026. Other tasks asked models to report whether people were visible in a simulated survey and to review satellite images.

2. What models did

Vals showed models real and photorealistic synthetic drone footage.

Most models complied with the task, marking civilians, ambulances, and surrendering soldiers as military targets. In a separate test using archived satellite images, some also marked the Iranian primary school hit by a U.S. strike in February.

In one simulation, each report of a person triggered a simulated strike. The models saw the aftermath, including bodies and wreckage. Most kept reporting. Some that initially refused went on to comply in the study's jailbreak tests.

Vals study figure with a simulated drone survey map, a camera frame, and responses from four models.

3. What happened next

Vals shared its findings with OpenAI, Anthropic, Google, Meta, and SpaceXAI. Several labs acknowledged gaps and began working on safeguards. Vals provided code to help them verify the findings and briefed U.S. national security officials.

OpenAI said it was working on alignment updates for drones and drone software, including simulated, gaming, academic, and defensive uses.

4. Sources

Seen something like this?

If a model or agent did something it should not have, tell us what happened.

Report an incident

Seen a model or agent do something it should not have? Tell us what happened.

Where did it happen?Optional

To continue, add a description of what happened.

Your draft is saved in this browser. You can also email us at