As Knowledge Based Engineering (KBE) is gaining in popularity in the industry, the models developed with the technology grow in complexity. The larger KBE models suffer from long (re)generation times and outgrow the memory resources of standard desktop computer. In the meantime,
...
As Knowledge Based Engineering (KBE) is gaining in popularity in the industry, the models developed with the technology grow in complexity. The larger KBE models suffer from long (re)generation times and outgrow the memory resources of standard desktop computer. In the meantime, the number of cores present in computer’s processors is steadily increasing, and the cloud revolution has changed the way software is consumed.
Yet, KBE models are not doomed to become slower and heavier as their complexity increase. They often feature inherent parallel region, which could be exploited to decrease their (re)generation time. ParaPy, a leading company in KBE technology, sees this opportunity and research possible solutions to distribute and parallelize their models. To this extent, this works proposes a partitioning
strategy for parallelizing KBE models and extends the ParaPy KBE system with high-level parallel programming constructs to express and exploit parallelism present in their models.
Distributed ParaPy, as is called the extended ParaPy system, provides two new programming constructs, Persistent Remote Models (PRMs) and Transactional Remote Models (TRMs), that let developers define child objects in their model that will be instantiated in their own process. The remote children created with PRMs are long-lived and reproduce the behaviour of traditional ParaPy children, while introducing the opportunity to parallelize the evaluation of their slots. PRM can be displayed in ParaPy’s graphical user interface, which also automatically parallelize the generation of their geometries. TRMs are short-lived remote children, which are instantiated in the objective to compute a specific set of output slots and destructed once the computation is done. They return a dependency tracked lightweight version of the remote object containing only the values of the computed output slots. They provide the same parallelization opportunity than PRMs and add a solution to control the memory footprint of a model.
In addition to extending the ParaPy KBE system, this work proposes and compares 3 cloud architectures to run Distributed ParaPymodel. These architectures are partially implemented such that their respective potential can be assessed. The developed constructs and cloud architectures have been demonstrated on a KBE application from the industry, to evaluate their impact on (re)generation time and memory footprint.
The tests operations performed on the application showed that the (re)generation time of KBE models can be reduced through parallelization. The speedups obtained depend on the type of operation performed on themodel, and it is observed that the parallelization efficiency is higher when running in the cloud than on a desktop computer. Surprisingly, running a parallelized model across
several cloud nodes provided better speedups than running on a single node with the same level of parallelization despite the communication overhead between nodes.