BabyTheorist: A Benchmark for Learning to Theorize the World from Observation Alone
· arxiv.org · popularity 6/100 · 0 similar stories today · AI & Data Science (Artificial Intelligence)
Read the original at arxiv.org →
Summary
arXiv:2611.08177v1 Announce Type: new Abstract: How would we know whether a machine truly understands the world? We argue that one key capability is \textit{observational theory learning}: acquiring reusable theories from pure observations and applying them to new situations. Current visual reasoning benchmarks focus primarily on the latter, presenting each task as a few-shot puzzle grouped by a common rule. We introduce BabyTheorist, a benchmark for controlled training and evaluation of observational theory learning. BabyTheorist generates continual streams of before-after visual observations from hidden compositional programs; during training, learners see only the resulting transitions, with no program labels, task identities, or support sets. To support theory learning and evaluation, BabyTheorist controls the program language, a curriculum over program complexity, recurring frequent subprograms, and a continual train-test protocol with level skipping. Seven representative baselines plateau well below ceiling, with the gap widening as the curriculum advances---evidence that BabyTheorist isolates a capability current program-learning approaches do not yet address.
Found in: ArXiv CS.AI. Rating basis: single outlet; outlet tier 2 (unlisted); 1 feeds; feed position 55.