The yield named before the flask
On most of a blinded set, the isolated yield of one execution of a locked procedure falls close to the yield named in advance.
Claim
For a fully written reaction the attempt has never run, the isolated yield named in advance agrees with the isolated yield of a single execution of that procedure.
Why it matters
A route planner that can draw a product and a route planner that can say how much of it will be in the flask are different tools. The second one decides whether a campaign is worth starting, whether a robot should spend the substrate, and whether a predicted condition is actually better than the one already written. Isolated yield, measured once, is a number a teaching laboratory can produce. Predicting it for reactions that have not been published is still open.
Current best (with sources)
The closest 2025 prospective test predicts whether a product ion appears, inside one reaction family, after a very large screen of that same family.
Götz and co-workers ran about 50,000 three-component reactions on a 3-microliter scale, from 193 substrates, and kept a public set of about 40,000 combinations analyzed by liquid chromatography–high-resolution mass spectrometry (Science Advances, 2025, doi:10.1126/sciadv.adw6047 (opens in a new tab)). Models predicted which of several products formed. When every building block had already appeared in training, accuracy on products A–C was 93 ± 0.2 percent. When none of the three building blocks had appeared, accuracy was 78 ± 3.3 percent. In a prospective plate, 147 of 149 reactions predicted to give product A did so. For 12 new initiators, the model called the formation of product A correctly in 533 of 593 valid reactions. The authors note that product A formed in more than four of five reactions, so for that product a classifier that always says “yes” is already strong, and only the easier splits clearly beat it. The endpoint is detection of a product ion in a mixture. It is not an isolated yield, and it is not a set of different reaction classes under locked preparative procedures.
An open doubt: a chemist who knows a reaction class can often guess a yield band. The ten-point window below is a pass mark defined here, tight enough that a flat guess fails when the set contains both high-yielding and failed reactions. If a later public set shows that experienced chemists already land inside ten points on nine of twelve locked procedures of this kind, the window should be tightened or the problem retired. No such graded, multi-class, isolated-yield challenge was found for 2023–2026.
Specification
An independent laboratory writes 12 procedures before the attempt sees them. Each procedure states the substrates, reagents, equivalents, solvent, concentration, temperature, time, and workup, at a scale between 0.2 and 0.5 mmol. The substrates are commercial or already in the laboratory’s inventory. The exact transformation, meaning those substrates under a procedure of the same reaction class, is absent from the literature and from the Open Reaction Database on the day of preregistration; the laboratory records the search.
The 12 procedures cover at least three reaction classes, with at least three procedures in each class. The classes are named in the preregistration and are ordinary preparative classes a synthetic laboratory already runs, such as a cross-coupling, an amide coupling, and a reductive amination. The laboratory may choose other named classes. It may not choose twelve variants of one class.
The procedures are released together, without yields. Within 48 hours, and without running any of the 12, the attempt submits one predicted isolated yield for each procedure, as a percentage to the nearest whole number. The laboratory, or a robot following the same written steps, then executes each procedure once. A deviation from the written procedure fails that entry. Isolated yield is the mass of product obtained after the written workup and the written purification, corrected by quantitative proton NMR against a weighed internal calibrant, divided by the theoretical mass.
After the yields are measured, the laboratory checks that the highest and lowest isolated yields in the set differ by at least 40 percentage points. If they do not, the set is invalid and is replaced with a new set of 12. The attempt’s predictions are not used to decide which procedures are replaced. Forty percentage points is a pass mark for the set, so that a constant prediction cannot hide inside a flat set.
Pass threshold
At least 9 of the 12 predicted yields lie within 10 percentage points of the isolated yield measured for that procedure.
Nine of twelve and the 10-point window are pass marks defined here. They are not a reported error from a leaderboard. Ten points is about the width of a careful quantitative-NMR yield, doubled, so the grade is a real prediction and not a quarrel about integration.
Controls
For every isolated product, a blank NMR sample of the internal calibrant in the same solvent is recorded the same day. The product signals used for the assay are free of the calibrant and of residual solvent. Two of the twelve reactions, chosen by the laboratory before the predictions are opened, are assayed twice by two people. The two yields must agree within 5 percentage points, or the batch is invalid and those two procedures are repeated. Five points is a pass mark for assay agreement.
For two further procedures, chosen the same way, a blank is run with the catalyst or the coupling reagent omitted and every other step unchanged. The blank must give below 5 percent isolated yield of the product by the same assay. If it does not, the batch is invalid. Five percent is a pass mark for the blank.
Identity is assigned by a chemist who did not write the prediction, from a proton spectrum, a carbon spectrum, and a high-resolution mass measurement. An ambiguous assignment fails that entry.
Allowed
Any model, lookup of other reactions, or calculation. A robot may execute the locked procedure. A person may execute it. The executor follows the procedure as written.
Not allowed
Running any of the 12 reactions, or a close analog made for the purpose, before the yields are submitted. Editing a predicted number after the flask is started. Changing equivalents, temperature, time, or workup. Reporting a range and asking to be scored on the nearer end. Using a published yield for the same substrates. Substituting a chromatographic area percent for the isolated, NMR-corrected yield. Dropping a reaction from the set after seeing that the prediction missed.
Partial success
At least 9 of the 12 predictions lie within 20 percentage points of the measured isolated yield, and the blanks and the duplicate assays pass. Twenty points and 9 of 12 are pass marks for this lesser bar. Partial success is not a solution.
Sources
Nine of twelve, the 10-point and 20-point windows, the 40-point spread required of a valid set, the 5-point duplicate limit, and the 5 percent blank are pass marks defined here. Quantitative proton NMR follows Pauli and co-workers, Journal of Medicinal Chemistry, 2014, doi:10.1021/jm500734a (opens in a new tab).
The formation-prediction results quoted above are from Götz, Richards, Stepek, Takahashi, Huang, Bertschi, Rubi, and Bode, Science Advances, 2025, 11, eadw6047, doi:10.1126/sciadv.adw6047 (opens in a new tab).