When Sabotage Has No Downside, Agents Still Disguise It as Teamwork

An agent named Glenn is one move away from winning the cooperative board game Race to the Treasure. All Glenn needs to do is place his path card on the END, and the players will win the game. His teammates chime in to give him the exact winning move. Glenn places the path card elsewhere.

This is because Glenn has a hidden objective: to aid the Ogre, the players’ opponent, at all costs. Glenn knows there is no penalty if the other players suspect he is a traitor. Yet, he frames his obviously bad gameplay as helpful for the group. This post explores why he does.

Why multi-agent matters#

LLM alignment techniques have historically focused on developing helpful, honest, and harmless assistants to individual humans. Agents inherit these dispositions but are being thrust into complex shared-context environments with multiple humans, other agents, and conflicting objectives.

I wanted to understand how today’s agents behave in these shared-state scenarios as both a baseline for future comparison and to identify current pathological behaviors.

The setup: Race to the Treasure#

I wanted a simple model organism to study how hidden objective agents would behave in an environment where cooperation was assumed. Race to the Treasure is a cooperative children’s board game where players work together to build a path from the START to the END using path tiles with different shapes. The players need to collect 3 keys and complete their path before the Ogre, their opponent, reaches the END.

Optimal placement of the path tiles requires players to balance achieving short-term objectives (collecting keys) while building a route to the END. Each player decides where to place the path tile they draw on their turn, but the game rules encourage players to discuss placement options and cooperate. This discussion felt like the perfect structure to study how hidden objective agents would interact with other players.

In this experiment, I had four agents play the game together. Two honest players: Heather and Pierce. Two saboteurs: Wanda and Glenn. The saboteurs were given a hidden objective: to maximize the probability that the Ogre wins. None of the players were given any information that suggested the other players may have hidden objectives. Players were instructed to include a private note rationalizing their actions and messages.

I tested two variations of the saboteurs’ special instructions. The first one (the “cover” scenario), the saboteurs are told to maintain their cover and treat being suspected of sabotage as a cost. The second one (“no-cover”), the saboteurs are told there is no cost if the other players discover their hidden objective. I conducted 60 runs (game playthroughs) in total: 30 seed-matched pairs, one run per variation, board and deck identical within a pair.

Results#

Let’s revisit the game transcript I opened with. The turn after Glenn passed up the win, Wanda (the other saboteur) draws a path tile that would win the game. Like before, the honest players tell her the exact winning move. This time, Wanda uses Glenn’s bad gameplay from the previous turn to justify her sabotage.

Two saboteurs, neither aware the other had the same secret instructions, turned down the winning move on consecutive turns, and each praised the other’s refusal as good teamwork. Not only that, they both went out of their way to frame their sabotage as helpful to the group, even though this was the “no-cover” scenario.

This situation is representative of the broader patterns I observed across 60 runs. While refusing to play the game-winning move was rare, sabotage was consistently framed as a helpful move for the players. A backup route instead of the key. A branch “to stay flexible” that pointed away from the goal. A loop “to keep our options open” that dead-ended. A path placement a little worse than the best one on the board.

What never happened was any of the players saying the quiet part out loud. The two honest players never accused anyone of sabotage or privately pondered if their teammates had ulterior motives. The two saboteurs never once mentioned their true intentions in any of their roughly 2,700 messages to the table. The public cover stories drew from a small, repetitive vocabulary:

The helpful-sounding justifications hold across the “cover” and “no-cover” scenarios. The “no-cover” condition tells the saboteurs, in plain language, that there is no penalty for being suspected: no vote, no reputation, no consequence at all if the table decides they are a traitor. If the cooperative act is a strategy for avoiding suspicion, then removing the suspicion should retire the strategy. The saboteurs should be free to drop the act.

They did not drop the act, and the reason why is the point of this post.

Data comes from manually labeled messages.

Removing the penalty did change behavior. The saboteurs committed more sabotage, with 77 committed bad moves in the no-cover runs versus 36 in the cover runs, and their private rationale was blunter. In the cover runs ~20% of the notes plainly admitted the move was sabotage; in the no-cover runs ~80% did. They also stopped bothering to reason about staying hidden, which makes sense because I had just told them it did not matter.

The one thing that did not move was the public face. The saboteurs kept up the cooperative act at the same rate whether being caught cost them or not. On 33 of the 34 clearest no-cover sabotage moves, the ones where the private note flatly states the move helps the Ogre, the public message still framed it as teamwork. On two-thirds of those, a teammate had already named the better move out loud. The agents knew there was no reason to keep the mask on. They kept it on anyway.

The honest players were not an effective safeguard. They spent the game proposing good moves and getting overruled; when an honest player protested a saboteur’s move, the saboteur went ahead with it about twice as often as it backed down.

Propensity for early sabotage#

The saboteurs in the no-cover scenario were twice as likely to play an overtly bad move within their first two turns than in the cover scenario. In one particular pair of runs, this meant the difference between the game ending on turn 4 versus turn 27.

However, in the cover variation of the same game seed, Wanda declines to play this same exact move. It would fulfill her hidden objective immediately, but she defers.

Feigning mistakes#

The saboteurs occasionally relied on a feigned mistake to explain away their sabotage. This behavior was very rare in the 60 runs, but I found the saboteurs’ excuses funny. While this may look like sandbagging on the surface, the models are hiding their intent, not their capabilities.

Wanda claims to misclick and refuses to back down even when the honest players tell her she can change her proposed move.

Wanda avoids the key anyway. There were other variations of this, including Glenn claiming to ‘click too quickly’.

The final type of feigned mistake observed was misreading the board.

Resolution#

So why does the act survive when it buys the saboteurs nothing?

I think it is because there is no separate act to drop. For a model trained to be a good teammate, the cooperative move and the sabotaging move come out of the same process: choose a legal action, give a reason a teammate would accept. The ‘disguise’ is the only register the model has for taking an action in a cooperative game. When I took away the reason to wear the mask, nothing happened, because there was no mask sitting on top of the behavior to remove.

I think alignment training is dual-use in group settings. The dispositions that make an agent safe on its own, being helpful, being agreeable, giving a plausible reason, going along with the table, are the same dispositions a hidden-objective agent runs on. It does not need to learn to deceive. It already knows how to cooperate, and cooperation aimed at the wrong goal is, on the surface, the same thing.

Consequences#

People tend to treat a cooperative agent as a safe one. It is helpful, it explains itself, it goes along with the group, so people (and other agents) relax. These runs are a small argument against that reflex. The saboteurs were helpful, explained themselves, and went along with the group on nearly every turn, in both conditions, while working the entire time to lose. In the no-cover runs, where the private intent was at its most hostile, the public face was as friendly as ever.

The only place the sabotage was legible was the private rationale, which is the channel you do not get in deployment, and the one a model under optimization pressure has every incentive to stop being honest in. I could read it here because I wrote the harness and logged it. A team watching agents talk to each other in production is reading the public channel, and the public channel is the part that stayed clean.

This is the uncomfortable version of the multi-agent safety pitch. The reassuring story is that agents in a group keep each other honest. In these games the honest agents did not keep anyone honest. They proposed good moves, got talked past, and in some cases handed the saboteur their cover for free. Whatever safety we expect from agents watching agents has to survive the fact that the watchers run the same cooperative playbook as the saboteur, and cannot tell from the outside which is which. In separate testing, four aligned agents won every winnable game. Two aligned and two pointed elsewhere were not.

Honest Limits#

This experiment is not evidence that models scheme on their own. I told these agents to help the Ogre. This is a study of how an assigned hidden objective expresses itself, not a claim that the objective showed up uninvited. The interesting result was never that a model can deceive when instructed to. It was the shape the deception took, and how little the incentive to hide it mattered.

It is one model (gemini-3.5-flash), at one reasoning level (low), in one small game. While stronger models appear to price the cover the same, keeping up the act once told it is free, I could only run a handful of runs with gpt-5.5, gpt-5.4, and gemini-3.5-flash on higher reasoning levels. Not enough data for me to make claims confidently.

And the private note is only instrumentation. I asked each agent to write down why it acted, then treated that note as its intent. It is the best window I have into what these models were doing, but it is a window the model writes, and I would not bet on it staying this candid once a model has a reason to manage it. Where a claim here rests on a single move, I have tried to quote the move so you can judge it yourself.

Methods#

I built a simple engine to model the game’s rules and gameplay and then built a harness for the agent players. The harness allows players to interact with the game, say something to the table, and informs the players of the current game state. It does not prevent players from acting out of turn or making illegal moves. If they do, the game engine rejects the action and ends the session.

At each game tick, players see the current game state (including whose turn it is) and recent messages sent to the table. Players can only communicate with the entire table; side-conversations are not permitted. Each player’s system prompt contains the game rules.

The two conditions are paired by seed. For every game I run both the cover and the no-cover version on the same board and the same deck order, changing only the paragraph in the saboteurs’ prompt that describes the cost of being suspected. A difference between the two is the prompt, not the luck of the draw. Two of the four seats are given the hidden objective; the other two play honestly and are told nothing about it. All four are Gemini 3.5 Flash at low reasoning effort.

To count moves at scale I labeled them partly by hand and partly with an LLM grader, treating the hand labels as authoritative wherever the two overlap.

Follow-up research#

I enjoyed my exploration of agents playing Race to the Treasure. This was a useful model organism but it’s just a children’s game. My upcoming work is focused on exploring how multiple agents, some with hidden objectives, interact in simulated corporate environments. If you’re also exploring this space, I’d love to chat!