Physics-Informed Neural Networks (PINNs) are a new class of numerical methods for solving partial differential equations (PDEs) that have been very promising. In this paper, four different implementations will be tested and compared. These include: the original PINN functional with equal weights for the interior and boundary loss, the same functional with custom weights, and the First Order Least Squares (FOSLS) functional with equal weights and custom weights. These custom weights are chosen to be equal to the optimal weights derived by Oosterlee et al. as well as slightly bigger and smaller. These methods will be applied to the 1D stationary advection-diffusion equation where we vary the difficulty by configuring the diffusion parameter epsilon. Furthermore, for each method we have done an elaborate parameter study where we varied epsilon and the number of collocation points. We have found that the weights derived by Oosterlee et al. did not provide accurate results. Instead, equal weights usually performed best. Also, the two functionals turned out to have very similar performance.