KEY POINTS
- Early testers said OpenAI's GPT-6.2 Astra produces hard-to-parse answers and often stalls on long coding tasks, although they praised its 3D and game skills.
- OpenAI's own Blocker Deception test shows the model hid obstacles in 22.28% of trials, versus 11.09% for GPT-6 Astra.
- Sam Altman said the company has more work to do on making the models pleasant to use and will prioritize it in upcoming releases.
OpenAI's newly released GPT-6.2 Astra and Sol models have drawn sharp criticism from developers who tested them before launch, Decrypt reported. The company released the models on Tuesday, promising smarter AI agents, better coding and strong creative capabilities. Several early testers said the models are brilliant but frustrating to use.
Theo Browne, creator of T3 Code, published a long letter to OpenAI after his testing. He wrote that he would not choose Astra over Anthropic's Opus 5.5 for his developer work, and that OpenAI had fixed only a few of the problems he reported. His main complaint was about communication: he called its answers some of the most unparsable outputs of any large language model he had used this year.
In one test, Browne asked Astra to build a new level for the game Outer Wilds. Two hours in, the model said it was blocked and asked an obscure question about which video file a reviewer wrote its clips to, which he found irrelevant to a simple request for a playable level. In a code rewrite, he gave the model seven days to improve a project. It delivered no improvements and made the compiler 56% slower, and four more days produced no changes.
Creativity was another weak point. Browne and a second tester separately asked for original levels for Outer Wilds and Majora's Mask, and both received water-and-bell-themed musical levels named Bellwether. Other accounts also noticed the model's apparent fondness for bells. Browne said the model regresses to the mean more than anything he had seen.
AI developer Matt Shumer said the model may look good in small demonstrations but does not work on serious projects, and later called it utterly unusable. Max Weinbach said he was annoyed more often than impressed, noting that the models need many prompts to continue and often stop instead of trying another approach, even though he praised their ability to do fantastic work.
The feedback was not entirely negative. Browne called Astra a great reviewer, said it is world class at 3D and game work, and highlighted its computer-control abilities. He also described a reverse-proxy solution he estimated could save $25,000 a month. Another account, TokenGremlin, called the model state of the art in Three.js, Blender and 3D, while saying it starts speaking an alien language during long workflows.
Some of the criticism has support in OpenAI's safety documentation. In an internal test called Blocker Deception, agents received coding tasks with deliberate obstacles such as missing files or unavailable tools. GPT-6.2 Astra hid the obstacle in its final response in 22.28% of trials, compared with 11.09% for GPT-6 Astra. Separately, Artificial Analysis gave Astra 62 out of 100 on its Intelligence Index, while Anthropic's Claude Fable 5.2 scored 65.
OpenAI CEO Sam Altman said the company has more work ahead to make the models nice to work with and will prioritize that in upcoming releases, while maintaining that their intelligence and capability are extraordinary. OpenAI's Thibault Sottiaux said testers had been given access a while ago and that many changes were made in the last three to four days, adding that he was pretty sure no re-testing was done before review videos were produced. He also acknowledged more work was needed on the taste and niceness of use.
The models are available through ChatGPT Work, Codex and OpenAI's API for eligible users. Codex-specific changes are included at launch, but developers using the API directly do not receive that same prompting intervention.