Ai2 Releases StateBench to Measure Long-Context State Tracking
· unite.ai · popularity 16/100 · 0 similar stories today · Machine Learning Research (Artificial Intelligence)
Read the original at unite.ai →
Summary
The Allen Institute for AI (Ai2) introduced StateBench on November 3, 2026, releasing a four-level benchmark, dataset and code for testing whether language models can track changing variables across long documents. Ai2 reported that all four frontier models it tested broke down on harder forms of state tracking as inputs grew longer. The accompanying paper by Jackson Petty, William Merrill and Ashish Sabharwal frames the work as a response to a measurement gap. Existing tests have tended to…
Found in: Unite.AI. Rating basis: single outlet; outlet tier 2 (unlisted); 1 feeds; feed position 16.