LLMs fail strategic analysis of simplified cooperative poker
a developer tested llms on a simplified cooperative poker game to see if they could exploit logical side-channel communication strategies. despite the trivial nature of the deterministic solution, frontier models failed to derive optimal play, outperforming random chance only marginally.
this experiment probed the logical reasoning and strategy formulation capabilities of llms within a constrained multi-agent environment. the game, 'the gang', requires players to deduce card distributions and coordinate token assignments based solely on the observable sequence of actions, a task easily solved by humans using simple signaling encodings like timing or morse code. the developer bypassed expensive api calls for frontier models after initial tests with local open-source models yielded underwhelming results. even though a deterministic solution exists that completes a four-player game in roughly seven rounds, the models demonstrated an inability to discover these algorithmic shortcuts, relying instead on probabilistic guessing.