Data-driven approaches are a promising new addition to the list of available strategies for solving Partial Differential Equations (PDEs). One such approach, the Principal Component Analysis-based Neural Network PDE solver, can be used to learn a mapping between two function spaces, corresponding to a PDE. However, the practical limitations of this approach are unclear. This paper seeks to investigate for which types of inputs and outputs this type of solver gives useful results. Using a dataset with inputs sampled from Gaussian Random Fields with different parameters, and outputs for Poisson's equation and the Heat equation, obtained by using a Finite Element solver, neural networks are trained, and their performance is evaluated. The method performs adequately for the chosen inputs, and patterns are found in the resulting error, which differ for each set of input parameters. Thus, for these equations, it seems that this method performs differently for different input distributions, but further research is necessary to investigate if these patterns will hold for other equations.