Reinforcement learning has shown some success in automating process design by integrating data-driven models that interact with process simulators to learn to build process flowsheets iteratively. However, one major challenge in the learning process is that the reinforcement learning agent demands numerous process simulations in rigorous process simulators, thereby requiring long simulation times and expensive computational power. We propose employing transfer learning to enhance the reinforcement learning process in process design. This study examines two transfer learning strategies: (i) transferring knowledge from shortcut process simulators to rigorous simulators, and (ii) transferring knowledge from process variational autoencoders (VAEs). Our findings reveal that appropriate transfer learning can significantly improve both learning efficiency and convergence scores. However, transfer learning can also negatively impact the learning process when there are substantial discrepancies in decision range and reward function. This suggests that pre-trained process data should match the complexity of the fine-tuning task.

Control structure design is an important but tedious step in P&ID development. Generative artificial intelligence (AI) promises to reduce P&ID development time by supporting engineers. Previous research on generative AI in chemical process design mainly represented processes by sequences. However, graphs offer a promising alternative because of their permutation invariance. We propose the Graph-to-SFILES model, a generative AI method to predict control structures from flowsheet topologies. The Graph-to-SFILES model takes the flowsheet topology as a graph input and returns a control-extended flowsheet as a sequence in the SFILES 2.0 notation. We compare four different graph encoder architectures, one of them being a graph neural network (GNN) proposed in this work. The Graph-to-SFILES model achieves a top-5 accuracy of 73.2% when trained on 10,000 flowsheet topologies. In addition, the proposed GNN performs best among the encoder architectures. Compared to a purely sequence-based approach, the Graph-to-SFILES model improves the top-5 accuracy for a relatively small training dataset of 1,000 flowsheets from 0.9% to 28.4%. However, the sequence-based approach performs better on a large-scale dataset of 100,000 flowsheets. These results highlight the potential of graph-based AI models to accelerate P&ID development in small-data regimes but their effectiveness on industry relevant case studies still needs to be investigated.

This “white paper” is a concise perspective of the potential of machine learning in the process systems engineering (PSE) domain, based on a session during FIPSE 5, held in Crete, Greece, June 27–29, 2022. The session included two invited talks and three short contributed presentations followed by extensive discussions. This paper does not intend to provide a comprehensive review on the subject or a detailed exposition of the discussions; instead its aim is to distill the main points of the discussions and talks, and in doing so, highlight open problems and directions for future research. The general conclusion from the session was that machine learning can have a transformational impact on the PSE domain enabling new discoveries and innovations, but research is needed to develop domain-specific techniques for problems in molecular/material design, data analytics, optimization, and control.

Microgels are cross-linked, colloidal polymer networks with great potential for stimuli-response release in drug-delivery applications, as their small size allows them to pass human cell boundaries. For applications with specified requirements regarding size, producing tailored microgels in a continuous flow reactor is advantageous because the microgel properties can be controlled tightly. However, no fully-specified mechanistic models are available for continuous microgel synthesis, as the physical properties of the included components are only studied partly. To address this gap and accelerate tailor-made microgel development, we propose a data-driven optimization in a hardware-in-the-loop approach to efficiently synthesize microgels with defined sizes. We optimize the synthesis regarding conflicting objectives (maximum production efficiency, minimum energy consumption, and the desired microgel radius) by applying Bayesian optimization via the solver “Thompson sampling efficient multi-objective optimization” (TS-EMO). We validate the optimization using the deterministic global solver “McCormick-based Algorithm for mixed-integer Nonlinear Global Optimization” (MAiNGO) and verify three computed Pareto optimal solutions via experiments. The proposed framework can be applied to other desired microgel properties and reactor setups and has the potential of efficient development by minimizing number of experiments and modeling effort needed.

The term hybrid modeling refers to the combination of parametric models (typically derived from knowledge about the system) and nonparametric models (typically deduced from data). Despite more than 20 years of research, over 150 scientific publications (Agharafeie et al., 2023), and some recent industrial applications on this topic, the capabilities of hybrid models often seem underrated, misunderstood, and disregarded by other disciplines as “simply combining some models” or maybe it has gone unnoticed at all. In fact, hybrid modeling could become an enabling technology in various areas of research and industry, such as systems and synthetic biology, personalized medicine, material design, or the process industries. Thus, a systematic investigation of the hybrid model properties is warranted to scoop the full potential of machine learning, reduce experimental effort, and increase the domain in which models can predict reliably.

The estimation of polymer properties is of crucial importance in many domains such as energy, healthcare, and packaging. Recently, graph neural networks (GNNs) have shown promising results for the prediction of polymer properties based on supervised learning. However, the training of GNNs in a supervised learning task demands a huge amount of polymer property data that is time-consuming and computationally/experimentally expensive to obtain. Self-supervised learning offers great potential to reduce this data demand through pre-training the GNNs on polymer structure data only. These pre-trained GNNs can then be fine-tuned on the supervised property prediction task using a much smaller labeled dataset. We propose to leverage self-supervised learning techniques in GNNs for the prediction of polymer properties. We employ a recent polymer graph representation that includes essential features of polymers, such as monomer combinations, stochastic chain architecture, and monomer stoichiometry, and process the polymer graphs through a tailored GNN architecture. We investigate three self-supervised learning setups: (i) node- and edge-level pre-training, (ii) graph-level pre-training, and (iii) ensembled node-, edge- & graph-level pre-training. We additionally explore three different transfer strategies of fully connected layers with the GNN architecture. Our results indicate that the ensemble node-, edge- & graph-level self-supervised learning with all layers transferred depicts the best performance across dataset size. In scarce data scenarios, it decreases the root mean square errors by 28.39% and 19.09% for the prediction of electron affinity and ionization potential compared to supervised learning without the pre-training task.

The transformation toward renewable energy and feedstock supply in the chemical industry requires new conceptual process design approaches. Recently, deep reinforcement learning (RL), a subclass of machine learning, has shown the potential to solve complex decision-making problems and aid sustainable process design. However, its suitability in static process design still needs to be examined. We discuss the advantages and disadvantages of RL for process design. Then, we survey state-of-the-art research through three major elements: (1) information representation, (2) agent architecture, and (3) environment and reward. Moreover, we discuss perspectives on underlying challenges and promising future works to unfold the full potential of RL for process design in chemical engineering.

ChatGPT is a powerful language model from OpenAI that is arguably able to comprehend and generate text. ChatGPT is expected to greatly impact society, research, and education. An essential step to understand ChatGPT’s expected impact is to study its domain-specific answering capabilities. Here, we perform a systematic empirical assessment of its abilities to answer questions across the natural science and engineering domains. We collected 594 questions on natural science and engineering topics from 198 faculty members across five faculties at Delft University of Technology. After collecting the answers from ChatGPT, the participants assessed the quality of the answers using a systematic scheme. Our results show that the answers from ChatGPT are, on average, perceived as “mostly correct”. Two major trends are that the rating of the ChatGPT answers significantly decreases (i) as the educational level of the question increases and (ii) as we evaluate skills beyond scientific knowledge, e.g., critical attitude.

Multiscale modeling of catalytical chemical reactors typically results in solving a system of partial differential equations (PDEs) or ordinary differential equations (ODEs). Despite significant progress, the numerical solution of such PDE or ODE systems is still a computational bottleneck. In the past, deep learning techniques have gained attention for developing surrogate models in chemical engineering. Also, hybrid models and physics-informed neural networks (PINNs) have been developed to integrate physical knowledge and data-driven approaches. However, it is often unclear how such modeling approaches compare for specific case studies. In this study, we investigate and compare state-of-the-art surrogate and hybrid models for the spatial evolution of the state variables in a packetbed reactor for methanol production. Firstly, we develop a tailored hybrid model based on PINNs, thereby seamlessly integrating physical knowledge and data. Secondly, we investigate a recently-developed time-series transformer model to learn the spatial evolution of the state variables. As a benchmark model, we train a traditional multilayer perceptron (MLP) and compare the models to a standard numerical integration technique. We achieve orders of magnitude in speedup using MLPs and PINNs when compared to classical ODE solvers, while maintaining high levels of accuracy in modeling the underlying system.

Machine Learning (ML) techniques are encountered nowadays across disciplines, from social sciences, through natural sciences to engineering. However, teaching ML is a daunting task. Aside from the methodological complexity of ML algorithms, both with respect to theory and implementation, the interdisciplinary and empirical nature of the field need to be taken into consideration. This paper introduces the MachineLearnAthon format, an innovative didactic concept designed to be inclusive for students of different disciplines with heterogeneous levels of mathematics, programming, and domain expertise. The format is grounded in a systematic literature review and the didactic principles action orientation, constructivism, and problem orientation. At the heart of the concept lie ML challenges, which make use of industrial data sets to solve real-world problems. Micro-lectures enable students to learn about ML concepts and algorithms, and associated risks. They cover the entire ML pipeline, promoting data literacy and practical skills, from data preparation, through deployment, to evaluation.

ReLU neural networks have been modelled as constraints in mixed integer linear programming (MILP), enabling surrogate-based optimisation in various domains and efficient solution of machine learning certification problems. However, previous works are mostly limited to MLPs. Graph neural networks (GNNs) can learn from non-euclidean data structures such as molecular structures efficiently and are thus highly relevant to computer-aided molecular design (CAMD). We propose a bilinear formulation for ReLU Graph Convolutional Neural Networks and a MILP formulation for ReLU GraphSAGE models. These formulations enable solving optimisation problems with trained GNNs embedded to global optimality. We apply our optimisation approach to an illustrative CAMD case study where the formulations of the trained GNNs are used to design molecules with optimal boiling points.

The process engineering domain widely uses Process Flow Diagrams (PFDs) and Process and Instrumentation Diagrams (P&IDs) to represent process flows and equipment configurations. However, the P&IDs and PFDs, hereafter called flowsheets, can contain errors causing safety hazards, inefficient operation, and unnecessary expenses. Correcting and verifying flowsheets is a tedious, manual process. We propose a novel generative AI methodology for automatically identifying errors in flowsheets and suggesting corrections to the user, i.e., autocorrecting flowsheets. Inspired by the breakthrough of Large Language Models (LLMs) for grammatical autocorrection of human language, we investigate LLMs for the autocorrection of flowsheets. The input to the model is a potentially erroneous flowsheet and the output of the model are suggestions for a corrected flowsheet. We train our autocorrection model on a synthetic dataset in a supervised manner. The model achieves a top-1 accuracy of 80% and a top-5 accuracy of 84% on an independent test dataset of synthetically generated flowsheets. The results suggest that the model can learn to autocorrect the synthetic flowsheets. We envision that flowsheet autocorrection will become a useful tool for chemical engineers.

Co-design of alternative fuels and future spark-ignition (SI) engines allows very high engine efficiencies to be achieved. To tailor the fuel’s molecular structure to the needs of SI engines with very high compression ratios, computer-aided molecular design (CAMD) of renewable fuels has received considerable attention over the past decade. To date, CAMD for fuels is typically performed by computationally screening the physicochemical properties of single molecules against property targets. However, achievable SI engine efficiency is the result of the combined effect of various fuel properties, and molecules should not be discarded because of individual unfavorable properties that can be compensated for. Therefore, we present an optimization-based fuel design method directly targeting SI engine efficiency as the objective function. Specifically, we employ an empirical model to assess the achievable relative engine efficiency increase compared to conventional RON95 gasoline for each candidate fuel as a function of fuel properties. For this purpose, we integrate the automated prediction of various fuel properties into the fuel design method: Thermodynamic properties are calculated by COSMO-RS; combustion properties, indicators for environment, health and safety, and synthesizability are predicted using machine learning models. The method is applied to design pure-component fuels and binary ethanol-containing fuel blends. The optimal pure-component fuel tert-butyl formate is predicted to yield a relative efficiency increase of approximately 8% and the optimal fuel blend with ethanol and 3,4-dimethyl-3-propan-2-yl-1-pentene of 19%.

SFILES are a text-based notation for chemical process flowsheets. They were originally proposed by d’Anterroches (Process flow sheet generation & design through a group contribution approach) who was inspired by the text-based SMILES notation for molecules. The text-based format has several advantages compared to flowsheet images regarding the storage format, computational accessibility, and eventually for data analysis and processing. However, the original SFILES version cannot describe essential flowsheet configurations unambiguously, such as the distinction between top and bottom products. Neither is it capable of describing the control structure required for the safe and reliable operation of chemical processes. Also, there is no publicly available software for decoding or encoding chemical process topologies to SFILES. We propose the SFILES 2.0 with a complete description of the extended notation and naming conventions. Additionally, we provide open-source software for the automated conversion between flowsheet graphs and SFILES 2.0 strings. This way, we hope to encourage researchers and engineers to publish their flowsheet topologies as SFILES 2.0 strings. The ultimate goal is to set the standards for creating a FAIR database of chemical process flowsheets, which would be of great value for future data analysis and processing.

Developing Piping and Instrumentation Diagrams (P&IDs) is a crucial step during process development. We propose a data-driven method for the prediction of control structures. Our methodology is inspired by end-to-end transformer-based human language translation models. We cast the control structure prediction as a translation task where Process Flow Diagrams (PFDs) without control structures are translated to PFDs with control structures. We represent the topology of PFDs as strings using the SFILES 2.0 notation. We pretrain our model using generated PFDs to learn the grammatical structure. Thereafter, the model is fine-tuned leveraging transfer learning on real PFDs. The model achieved a top-5 accuracy of 74.8% on 10,000 generated PFDs and 89.2% on 100,000 generated PFDs. These promising results show great potential for AI-assisted process engineering. The tests on a dataset of 312 real PFDs indicate the need for a larger PFD dataset for industry applications and hybrid artificial intelligence solutions.

An educational workshop for developing Process Systems Engineering (PSE) courses will be held during ESCAPE-33, following the model workshop that was run during the CAPE Forum 2022 held at the University of Twente, in the Netherlands. This 3-hour workshop distributes the participants into four teams working together to develop the outline of a course on a novel application area in PSE motivated by a selected plenary or keynote talk at the conference, with each team led by authors of this contribution. This paper provides an overview of the approach used in the workshop for the effective development of a PSE course.

We propose a novel method enabling autocompletion of chemical flowsheets. This idea is inspired by the autocompletion of text. We represent flowsheets as strings using the text-based SFILES 2.0 notation and learn the grammatical structure of the SFILES 2.0 language and common patterns in flowsheets using a transformer-based language model. We pre-train our model on synthetically generated flowsheet topologies to learn the flowsheet language grammar. Then, we fine-tune our model in a transfer learning step on real flowsheet topologies. Finally, we use the trained model for causal language modeling to autocomplete flowsheets. Eventually, the proposed method can provide chemical engineers with recommendations during interactive flowsheet synthesis. The results demonstrate a high potential of this approach for future AI-assisted process synthesis but also reveal the limitations at the present state and the next steps that need to be taken to deploy this technique in realistic flowsheet synthesis scenarios.

Advances in deep convolutional neural networks led to breakthroughs in many computer vision applications. In chemical engineering, a number of tools have been developed for the digitization of Process and Instrumentation Diagrams. However, there is no framework for the digitization of process flow diagrams (PFDs). PFDs are difficult to digitize because of the large variability in the data, e.g., there are multiple ways to depict unit operations in PFDs. We propose a two-step framework for digitizing PFDs: (i) unit operations are detected using a deep learning powered object detection model, (ii) the connectivities between unit operations are detected using a pixel-based search algorithm. To ensure robustness, we collect and label over 1000 PFDs from diversified sources including various scientific journals and books. To cope with the high intra-class variability in the data, we define 47 distinct classes that account for different drawing styles of unit operations. Our algorithm delivers accurate and robust results on an independent test set. We report promising results for line and unit operation detection with an Average Precision at 50 percent (AP50) of 88% and an Average Precision (AP) of 68% for the detection of unit operations.

Graph neural networks (GNNs) are emerging in chemical engineering for the end-to-end learning of physicochemical properties based on molecular graphs. A key element of GNNs is the pooling function which combines atom feature vectors into molecular fingerprints. Most previous works use a standard pooling function to predict a variety of properties. However, unsuitable pooling functions can lead to unphysical GNNs that poorly generalize. We compare and select meaningful GNN pooling methods based on physical knowledge about the learned properties. The impact of physical pooling functions is demonstrated with molecular properties calculated from quantum mechanical computations. We also compare our results to the recent set2set pooling approach. We recommend using sum pooling for the prediction of properties that depend on molecular size and compare pooling functions for properties that are molecular size-independent. Overall, we show that the use of physical pooling functions significantly enhances generalization.

Artificial intelligence has great potential for accelerating the design and engineering of chemical processes. Recently, we have shown that transformer-based language models can learn to auto-complete chemical process flowsheets using the SFILES 2.0 string notation. Also, we showed that language translation models can be used to translate Process Flow Diagrams (PFDs) into Process and Instrumentation Diagrams (P&IDs). However, artificial intelligence methods require big data and flowsheet data is currently limited. To mitigate this challenge of limited data, we propose a new data augmentation methodology for flowsheet data that is represented in the SFILES 2.0 notation. We show that the proposed data augmentation improves the performance of artificial intelligence-based process design models. In our case study flowsheet data augmentation improved the prediction uncertainty of the flowsheet autocompletion model by 14.7%. In the future, our flowsheet data augmentation can be used for other machine learning algorithms on chemical process flowsheets that are based on SFILES notation.