Physics-Informed Neural Networks (PINNs) offer a promising approach to solving partial differential
equations (PDEs). In PINNs, physical laws are incorporated into the loss function, guiding the network to learn a model that adheres to these laws as defined by the PDEs. Training PINNs involves using three types of points: collocation points within the domain of the PDEs, boundary points on the edges of the domain, and initial points at the starting time (t = 0). A common challenge in training PINNs is the imbalance between the number of boundaries and initial points compared to collocation points, which can negatively impact the training process. Typically, the number of each type of point is manually set, and the number of collocation points is usually ten times that of boundary and initial points, leading to an uneven loss distribution over time. Our work introduces a method called Relative Residual Resampling (R3) to address this issue. This method dynamically adjusts the number of each type of training point to ensure a more balanced distribution. Consequently, the loss is more evenly spread across the time domain, enhancing the performance of the PINN. We tested our method on the 1D Heat Equation and 1D Diffusion Equation. The results demonstrate that our approach reduces the overall loss by at least 6%, and balances the loss distribution over time by reducing the loss range for at least 65% compared with the state-of-the-art residual-based resampling strategy. This improvement makes PINNs more accurate and efficient for solving time-dependent PDEs, providing a practical solution for applications where understanding temporal dynamics is crucial. ... Read more

Data-driven approaches are a promising new addition to the list of available strategies for solving Partial Differential Equations (PDEs). One such approach, the Principal Component Analysis-based Neural Network PDE solver, can be used to learn a mapping between two function spaces, corresponding to a PDE. However, the practical limitations of this approach are unclear. This paper seeks to investigate for which types of inputs and outputs this type of solver gives useful results. Using a dataset with inputs sampled from Gaussian Random Fields with different parameters, and outputs for Poisson's equation and the Heat equation, obtained by using a Finite Element solver, neural networks are trained, and their performance is evaluated. The method performs adequately for the chosen inputs, and patterns are found in the resulting error, which differ for each set of input parameters. Thus, for these equations, it seems that this method performs differently for different input distributions, but further research is necessary to investigate if these patterns will hold for other equations. ... Read more

This paper presents a comprehensive exploration of a novel method combining Principal Component Analysis (PCA) and Neural Networks (NN) to efficiently solve Partial Differential Equations (PDEs), a fundamental challenge in modeling a wide range of real-world phenomena. Our research extends the work of Bhattacharya et al. by focusing on PCA for effective dimensionality reduction and utilizing NN for mapping in the reduced dimension. This approach addresses the significant computational challenges and inaccuracies often encountered with classical numerical techniques in solving PDEs.

We specifically investigate the still-water equation, employing our PCA-NN method to learn a reduced order mapping of PDE solutions and evaluate its robustness in diverse noisy environments. Our findings reveal a notable relationship between noise intensity and error, indicating a linear trend for Gaussian and Salt and Pepper noise, and an exponential trend for Uniform noise. Furthermore, this study uncovers a critical weakness of the model in predicting points with a high rate of change.

Overall, our research significantly contributes to understanding the practical applicability and limitations of PCA-NN methods in real-world, noisy settings, offering valuable insights for future applications in this domain. ... Read more

Solving Partial Differential Equations (PDEs) in engineering such as Navier-Stokes is incredibly computationally expensive and complex. Without analytical solutions, numerical solutions can take ages to simulate at great expense. In order to reduce this cost, neural networks may be used to compute approximations of the solution for use during engineering processes. PCA-net is a neural network approach that reduces the dimensionality of the input and output data for PDEs in order to allow mapping from a high-dimensional input and output function with a fully connected neural network through the use of Principal Component Analysis (PCA). In this paper, PCA-net is applied to Navier-Stokes with varying viscosities to test the generalization of PCA-net on viscosity parameters. Training is done on four discrete viscosities, while testing is done on continuous viscosities, extrapolating and interpolating around the training set. Results shows good performance on low viscosities, both with interpolation and extrapolation. Mid-to-high viscosity interpolation shows lesser performance, with high viscosity extrapolation diverging to great error. Omitting high viscosities, performance over varying viscosities is close to that shown by previous research. ... Read more