Model-based testing (MBT) is applied for the verification of simple simulation models used in modeling & simulation for decision-making. It is found that MBT of simulation models is not discussed much in literature, and that existing case studies on this topic do not give guidelines for test model design. It is established that testing simulation models is a difficult problem, as simulation models have many states, a time component, reactiveness, and inherent stochasticity. It is hypothesized that MBT could lead to more efficient and effective testing, and could fit within a modular approach to verification of simulation models. A focus is made on verification that discrete-event simulation models meet their functional requirements, by using automated test packages.

Options for test model design are explored in three case studies where automated test suites are developed for three simple simulation models that form the systems under test (SUT). Options from existing taxonomies are first selected for this, based on their relevance to the properties of simulation models that make dynamic verification difficult. In the process, an existing MBT software tool is selected and evaluated.

The main aspects of MBT that are explored are: nondeterministic test models and SUTs, transition-based test models, online and reactive test execution, specification of oracles at different levels, time advancement of the SUT, the distinction between black-box and white-box approaches, and the problem of test abstraction. Examples are made where the test model, an extended finite state machine (EFSM), is an abstract version of the SUT, that includes its most relevant state transitions. An approach to testing is used, where the test model follows the SUT’s state and selects relevant assertions accordingly. This approach limits the use of stochasticity in the test model, that is often done in MBT.

It is found that MBT can be useful for three types of dynamic verification: of the dynamic behavior, of the (end) results, and of the response to inputs. The first type is explored with a step-by-step approach where oracles are checked after each time advancement in the SUT. The second type is implemented without the use of MBT techniques, by extending the functionality of an existing MBT tool to allow for analysis of multiple simulation runs. The third type is not explored but approaches for it are defined.

The project shows the feasibility of MBT of simulation models and shows different options for test model design. Not discussed are the adequacy of the developed test packages in detecting faults, and the efficiency, effectiveness and quality of MBT for simulation models in general.

Military decision-making involves art and science and aims to translate strategic goals into tactical actions for achieving specific outcomes. It relies on a commander’s capacity to navigate complex and sometimes ambiguous situations to develop detailed plans and actionable orders. At the military tactical level, decisions focus on executing tactical mission tasks that contribute to achieving operational and strategic objectives. Tactical decision-making within the Dutch Army involves a step-by-step rational decision-making process, mainly focused on analytical preparations for battle. An operational analysis or wargame of the developed plans is crucial in military decision-making. Such an analysis offers valuable insights into the most favorable courses of action compared with the presumed plans of other actors. The execution of a wargame empowers a commander to determine the optimal course of action.

Operational analysis, or wargaming, is a formalized military process that technology can support. Modern wargaming originated from the Prussian Army in the 1820s and has since spread to many other armed forces. Wargaming assesses the forces’ positions, strengths/weaknesses, opportunities, and environmental factors using doctrine, experience, and intuition. It is an iterative process highlighting critical tasks and tactical possibilities, often aided by modern information technology to increase efficiency.

Armed forces are using information technology to support decision-making, whereby Modeling and Simulation systems offer valuable tools for better situational understanding and assessing interventions’ effects. These simulations can provide the forces with crucial data to develop and assess possible courses of action. Military simulation spans various domains, from large-scale field exercises to abstract computational models. While simulations can accelerate the development of military knowledge and experience, it is essential to ensure that increased computing power aligns with a comprehensive understanding of warfare.

Within contemporary military simulation, there is a move from comprehensive monolithic systems to component-based simulation systems to facilitate innovation, flexibility, and cost-effectiveness. While reusability and composability are crucial for military simulation, they have been challenging. Reuse offers various technical, business, and economic benefits. Composability is vital for the system of systems that form contemporary military simulation environments, where linkage is challenging due to differing resolutions and levels of abstraction. High-resolution models help military staff to understand phenomena, while low-resolution models aid analytical understanding qualitatively. In these simulation environments, aggregate-level simulations represent military units with multiple entities, while entity-level simulations depict individuals. However, composing both types of simulation within military wargaming to support decision-making remains challenging. Based on this analysis of contemporary military simulation, the following main research question is formulated:

What are design principles for composable elements that can be used in a simulation system for military wargames?

Because military decision-making processes are not intentionally designed to be represented by simulation systems, the wargaming elements derived from relevant perspectives on military wargaming provide insight into which underlying principles may be usable when designing a wargaming simulation. The following principles have been deduced from military wargaming: a representative wargaming experience (principle of recognizability); a user interface that is designed for the users’ intended purpose for the system (principle of utilizability); traceable simulation results that create insightful knowledge (principle of traceability); and adaptable systems that can integrate changes and developments within military doctrine (principle of adaptability).

The challenges arising from Modeling & Simulation research provide rigorous knowledge for the design of military wargaming simulation systems. The following principles are derived from the selected scientific literature: consistency in the models and the attribute states in the wargame simulation system to maintain its validity (principle of consistency); a designer’s awareness and understanding of the implications of design choices for performance (principle of coherence); an ability to combine, recombine, configure, and reconfigure sets of components (principle of composability); and a possibility to represent interactions between different military units and echelons (principle of interoperability).

A system requirements structure links requirements to the above principles to assess the extent to which the identified design principles are usable, ultimately aligning with the defined objectives. Evaluation of the requirements reveals that a wargame simulation system must have a utilizable design that represents real-world wargaming, traceable results that create insightful knowledge, adaptable models to assimilate to evolving military doctrine, combinable, configurable, and interoperable components; it should have a recognizable user interface and consistent model behavior; while a such a simulation system could have a coherent design that incorporates the used Run-Time Infrastructure.

The wargame simulation system prototype within this research is designed to support a specific activity within a military decision-making process, the operational analysis, rather than replacing the entire process. While promising, it is still a prototype with limited functionalities. The prototype is tailored to evaluate design principles by a concise military scenario; however, it can be extended to encompass additional tactical tasks or units, showcasing the flexibility of the flow-based design for creating a Course of Action. While the prototype is intended for evaluating the identified design principles, its current utility for practical military operational analysis remains limited.

This research provides both military practitioners and Modeling & Simulation scientists the opportunity for further field experiments and theoretically explore the identified design principles for composable simulation elements that can be used in a flow-based wargame simulation system. Future scientific research should further improve the envisioned simulation concept into a usable wargame simulation system so military commanders and their staffs can exploit the power of simulation while conducting a wargame.