A Shift Toward Real-World Agent Workflows
The September 7, 2026 releases on the arXiv cs.AI repository mark a significant pivot in AI research toward agent evaluation infrastructure, inference reliability, and practical decision-making behavior. Rather than evaluating standalone large language models (LLMs) on static question-answering datasets, recent papers tackle the complexities of multi-step execution paths and specialized runtime environments.
Standardizing Benchmarks and Uncovering Harness Variance
To address fragmentation in evaluation infrastructure, researchers introduced Harbor Adapters alongside the Harbor-Index dataset to standardize agent measurement at scale. The project unified more than 80 benchmarks for consistent cross-model testing. Across 54 benchmarks evaluating 8 model families, Harbor-Index revealed that the top-performing configuration—GPT-5.5 paired with Codex—achieved a pass rate of just 28.0%, with no tested setup surpassing the 30% mark.
In software engineering capabilities, an investigation into Multi-Harness Reinforcement Learning (Multi-Harness RL) based on Qwen3-8B demonstrated that execution environments act as a primary confounding variable:
* Harness Sensitivity: Across 24,000 evaluations on SWE-bench Verified, simply switching the execution harness caused task resolution rates to swing between 2.14% and 9.27% (a 4.3x divergence). * Reward Grouping Impact: Cross-harness Group Relative Policy Optimization (GRPO) yielded a negligible improvement of just +0.25 percentage points on an independent holdout harness.
Quantifying Decision-Making Ethics, Security, and Faithfulness
Safety and behavioral alignment were also quantified through novel simulation frameworks. HarvestBench established a simulated agricultural environment to measure an agent's willingness to incur additional fuel costs to avoid striking wildlife. Across 7,201 pricing decisions across 9 models:
* Harm Rate Disparity: Strike rates ranged from 0.4% in Terra and Sol up to 98.8% in GPT-4o-mini. * Ethical Guidance: Injecting ethical directives into task briefings reduced collision rates below 6% in 5 out of 6 reasoning models.
On the security and interpretability front, additional studies framed indirect prompt injection as a test-time search problem and introduced an un-cited concept ablation technique to boost the faithfulness of LLM explanations without retraining model weights.