OpenAI’s wet-lab benchmark tests when an AI should say the evidence is insufficient

A bright patch in a microscope image can look like evidence of a dead cell. In one test OpenAI published on November 6, a second dye could also explain the signal. GPT-5.6 Sol called the cells dead. GPT-6 Astra recognized that the evidence was ambiguous. That distinction is part of what BioBasicsBench, a 100-task evaluation developed with Ligo, asks models to make.

Astra earned 66.7% of the available credit at the xhigh reasoning setting. That is a partial-credit average. A single answer can pick up points for reading an image correctly and still lose points for the conclusion it draws.

What went into the score

Scientists built the tasks around their own experimental material, including gel images, microscope recordings and instrument files. Each answer was scored against criteria written for that task. R&D World’s account of the methodology describes three rounds of task review, including one by a scientist outside Ligo. Those reviews helped check the questions and rubrics. The model results came from OpenAI’s evaluation.

OpenAI tested seven of its own models across 37 model-and-effort settings. At each setting, it collected five answers per task. The published 95% confidence intervals resample tasks while keeping their repeated answers together. OpenAI’s results list xhigh as the highest effort setting shared by all seven models.

The table describes performance on these questions at this setting. It gives neither a human baseline nor a comparison with other companies’ models. As Kingy AI’s review of the release notes, the publication does not link the complete tasks, rubrics or raw answers needed to check performance item by item.

A model can read the image and still make the wrong call

In the microscope example, the detection windows overlapped. An overlap means some light could be assigned to the wrong dye, and the example did not measure how much. A microscopy methods paper by Jennifer Waters demonstrates why a single-dye comparison helps establish whether one channel is picking up the other’s signal.

Editorial visualization

In another example, Astra spotted a detail that changed the interpretation. A dish presented as a negative control was empty, but its label showed a different antibiotic from the one used in the experiment. The example reported by R&D World shows why an empty dish cannot answer the intended question when the comparison conditions differ.

Editorial visualization

Astra also made the reverse kind of mistake. In a separate task, it grouped the bands in a gel correctly, then chose the wrong group because it assumed a larger DNA fragment indicated success. The controls pointed the other way. OpenAI’s failure analysis also describes a plate where Astra’s count was about 12% above the reference.

For a team evaluating a lab assistant, those errors suggest something specific to measure alongside the mean score: how often the system makes an actionable recommendation after misreading a control. Reviewers can inspect those calls separately from the parts of an answer that earned credit.