A series of studies published on arXiv on September 1, 2026, reveals that frontier large language models (LLMs) continue to encounter major bottlenecks when making complex expert decisions in healthcare and the natural sciences, despite achieving high scores on theoretical benchmarks.
High Failure Rates in Oncology Treatment Choices
In a study establishing the Oncology Decision Boundary Benchmark (ODBB), researchers evaluated nine frontier AI models—comprising four proprietary models and five open-weight model families released between June 2025 and April 2026—across 2,005 cancer treatment decision points. The results revealed that in 42.1% of all scenarios, not a single model made the correct choice. Failure rates were heavily concentrated in selecting regimens before performing detailed reasoning:
- 35.7% across 1,586 NCCN guideline regimen items - 66.4% across 419 real-world colorectal cancer cases
Notably, two models fine-tuned for high decisiveness—GPT-5.5 and Gemini 3.1 Pro Preview—made unsafe decisions 3 to 5 times more frequently than the remaining seven models without achieving higher accuracy scores. In 3% to 9% of cases, models correctly identified the next clinical step but refused to commit to execution.
Bottlenecks in Advanced STEM Reasoning
Concurrently, a study introducing the Expert-validated STEM QA dataset—comprising 398 physics, chemistry, biology, and mathematics questions authored by 241 domain experts—recorded an accuracy rate below 25% across cutting-edge models. When applying post-training with a distinct expanded dataset of 2,000 samples, an open-source model achieved a 15% performance improvement on the STEM HLE-verified test set (p=0.045).
Clinical Workflow Optimization and Resource Allocation
Addressing clinical workflow efficiency, the CDPR method combined with the GRPO algorithm was evaluated on the MIMIC-IV and ClinicalBench datasets to model sequential diagnostic procedures, helping reduce unnecessary diagnostic tests and costs. Meanwhile, another research team focused on heart transplant allocation proposed an algorithm that directly learns utility functions from allocation outcomes, elevating the competitive ratio from 0.54 under the existing policy to 0.95.
These empirical findings collectively emphasize that simply expanding training data volume cannot resolve blind spots in multi-step decision-making. Future systems require new AI architectures capable of recognizing their own competency boundaries and deferring judgment to human clinicians and domain experts.