Optimized Concept Learning in BEN

Abstract

The Abstraction and Reasoning Corpus (ARC) challenges AI systems to induce general rules from as few as two to four examples. BEN is a program synthesis agent that tackles ARC through a Divide, Align, and Conquer pipeline inspired by analogical reasoning, but its concept learning phase, which synthesizes Boolean rules governing when transformations apply, must frequently choose between candidate rules of equal structural complexity. Because ARC's extreme data scarcity makes such ties highly prevalent, the system's coarse notion of complexity provides insufficient nuance to distinguish between candidates, leaving it vulnerable to overfitting.

We present three optimizations to BEN's concept learner, built on a reimplementation of BEN in Julia. These optimization provide better nuance in complexity at different stages of the pipeline, helping the system more accurately mimic human analogical reasoning. A \emph{Rule Generality} heuristic uses cross-transformation frequency analysis to discount broadly applicable features. A \emph{DNF Model Score} leverages cumulative solver objective values as a semantic tie-breaker between structurally equivalent rules. A \emph{Simple Minimal Transformation Coverage} heuristic extends the greedy set-cover procedure, with secondary criteria based on correspondence ambiguity and transformation complexity. An ablation study on 400 ARC-AGI-1 tasks shows these optimizations increase accuracy from 32 to 36 solved tasks with no regressions and negligible computational overhead. Analysis of rule complexity reveals that generated DNFs consistently require only a single clause, suggesting that the current performance bottleneck lies in the upstream pipeline rather than in concept complexity itself.