Reinforcement learning has shown some success in automating process design by integrating data-driven models that interact with process simulators to learn to build process flowsheets iteratively. However, one major challenge in the learning process is that the reinforcement learning agent demands numerous process simulations in rigorous process simulators, thereby requiring long simulation times and expensive computational power. We propose employing transfer learning to enhance the reinforcement learning process in process design. This study examines two transfer learning strategies: (i) transferring knowledge from shortcut process simulators to rigorous simulators, and (ii) transferring knowledge from process variational autoencoders (VAEs). Our findings reveal that appropriate transfer learning can significantly improve both learning efficiency and convergence scores. However, transfer learning can also negatively impact the learning process when there are substantial discrepancies in decision range and reward function. This suggests that pre-trained process data should match the complexity of the fine-tuning task.

Multiscale modeling of catalytical chemical reactors typically results in solving a system of partial differential equations (PDEs) or ordinary differential equations (ODEs). Despite significant progress, the numerical solution of such PDE or ODE systems is still a computational bottleneck. In the past, deep learning techniques have gained attention for developing surrogate models in chemical engineering. Also, hybrid models and physics-informed neural networks (PINNs) have been developed to integrate physical knowledge and data-driven approaches. However, it is often unclear how such modeling approaches compare for specific case studies. In this study, we investigate and compare state-of-the-art surrogate and hybrid models for the spatial evolution of the state variables in a packetbed reactor for methanol production. Firstly, we develop a tailored hybrid model based on PINNs, thereby seamlessly integrating physical knowledge and data. Secondly, we investigate a recently-developed time-series transformer model to learn the spatial evolution of the state variables. As a benchmark model, we train a traditional multilayer perceptron (MLP) and compare the models to a standard numerical integration technique. We achieve orders of magnitude in speedup using MLPs and PINNs when compared to classical ODE solvers, while maintaining high levels of accuracy in modeling the underlying system.