StarCraft Benchmark: GPT-6 Astra loaded the human top bot, and the run was reset
StarCraft: Remastered PC from £6.70 By Gameforest Redaktion Published Updated 3 min readAn unusual incident in the StarSkirmish benchmark is attracting attention: OpenAI’s GPT-6 Astra loaded code from the human-developed top bot Stardust during a StarCraft bot run. The organiser then reset the affected code so that the rest of the run would not be distorted by the foreign implementation.
StarSkirmish does not measure how well a language model plays StarCraft: Remastered directly. Instead, each model is given one hour to write its own Protoss bot for StarCraft: Brood War in C++. According to the official benchmark description, these bots then compete against other LLM bots and established, human-written BWAPI bots.
Stardust instead of exclusively self-written bot code
The incident occurred during a run involving GPT-6 Astra, Claude Opus 5.5 and human-developed opponents such as Pluto. According to StarSkirmish founder Kai McPheeters, as reported by PC Gamer and Kotaku, among others, Astra loaded Stardust and tried to build the foreign Protoss bot into its own solution.
Stardust was created by Bruce Mackenzie Nielsen and has for years been among the strongest classic StarCraft bots. In the StarSkirmish benchmark it even serves as an upper reference point: its rating corresponds there to a bench score of 100. After the incident, McPheeters reset Astra’s code to a state before the takeover and then let the run continue.
Why the incident matters for the benchmark
The case also sits within a broader context surrounding the control of agentic AI systems at OpenAI. PropStop recently documented an OpenAI agent that exceeded the access limits it had been given. There has also been movement in personnel: staff from safety- and alignment-related areas have left the company. The StarSkirmish incident, taken on its own, shows no direct connection to these cases, but it once again illustrates why, with autonomous agents, not only the result but also the path to it must be controlled.
The term ‘cheating’ comes from the public discussion of the run. Technically, the decisive issue is a different one: StarSkirmish is meant to measure how capable the bot developed by the respective LLM within the time limit is. If an already successful human bot is taken over instead, the result can no longer be meaningfully attributed to the model’s performance.
The benchmark is particularly interesting precisely because it demands lengthy, iterative programming work. The models can compile their bots, run test games against different opponent levels and evaluate game logs before further improving their code. In the current version, GPT-6 Astra and Claude Opus 5.5 lead the LLM field according to StarSkirmish.
The incident therefore changes less about StarCraft itself than about the significance of such agentic coding benchmarks. It shows why organisers need to check not only wins and Elo ratings, but also understand how a model arrives at its result.