An important feature of human learning is the ability to continuously accept new information and unify it with existing knowledge, a process that proceeds largely automatically and without catastrophic side-effects. A generally intelligent machine (AGI) should be able to learn a wide range of tasks in a variety of environments. Knowledge acquisition in partially-known and dynamic task-environments cannot happen all-at-once, and AGI-aspiring systems must thus be capable of cumulative learning: efficiently making use of existing knowledge while learning new things, increasing the scope of ability and knowledge incrementally—without catastrophic forgetting or damaging existing skills. Many aspects of such learning have been addressed in artificial intelligence (AI) research, but relatively few examples of cumulative learning have been demonstrated to date and no generally accepted explicit definition exists of this category of learning. Here we provide a general definition of cumulative learning and describe how it relates to other concepts frequently used in the AI literature.

A generally intelligent machine (AGI) should be able to learn a wide range of tasks. Knowledge acquisition in complex and dynamic task-environments cannot happen all-at-once, and AGI-aspiring systems must thus be capable of cumulative learning: efficiently making use of existing knowledge during learning, supporting increases in the scope of ability and knowledge, incrementally and predictably — without catastrophic forgetting or mangling of existing knowledge. Where relevant expertise is at hand the learning process can be aided by curriculum-based teaching, where a teacher divides a high-level task up into smaller and simpler pieces and presents them in an order that facilitates learning. Creating such a curriculum can benefit from expert knowledge of (a) the task domain, (b) the learning system itself, and (c) general teaching principles. Curriculum design for AI systems has so far been rather ad-hoc and limited to systems incapable of cumulative learning. We present a task analysis methodology that utilizes expert knowledge and is intended to inform the construction of teaching curricula for cumulative learners. Inspired in part by methods from knowledge engineering and functional requirements analysis, our strategy decomposes high-level tasks in three ways based on involved actions, features and functionality. We show how this methodology can be used for a (simplified) arrival control task from the air traffic control domain, where extensive expert knowledge is available and teaching cumulative learners is required to facilitate the safe and trustworthy automation of complex workflows.

Four principal features of autonomous control systems are left both unaddressed and unaddressable by present-day engineering methodologies: (1) The ability to operate effectively in environments that are only partially known at design time; (2) A level of generality that allows a system to re-assess and re-define the fulfillment of its mission in light of unexpected constraints or other unforeseen changes in the environment; (3) The ability to operate effectively in environments of significant complexity; and (4) The ability to degrade gracefully - how it can continue striving to achieve its main goals when resources become scarce, or in light of other expected or unexpected constraining factors that impede its progress. We describe new methodological and engineering principles for addressing these shortcomings, that we have used to design a machine that becomes increasingly better at behaving in underspecified circumstances, in a goal-directed way, on the job, by modeling itself and its environment as experience accumulates. The work provides an architectural blueprint for constructing systems with high levels of operational autonomy in underspecified circumstances, starting from only a small amount of designer-specified code - a seed. Using value-driven dynamic priority scheduling to control the parallel execution of a vast number of lines of reasoning, the system accumulates increasingly useful models of its experience, resulting in recursive self-improvement that can be autonomously sustained after the machine leaves the lab, within the boundaries imposed by its designers. A prototype system named AERA has been implemented and demonstrated to learn a complex real-world task - real-time multimodal dialogue with humans - by on-line observation. Our work presents solutions to several challenges that must be solved for achieving artificial general intelligence.