A learning curve can serve as an indicator of the “performance of trained models versus the training set size” [1]. Recent research on learning curve analysis has been carried out within the Learning Curve Database (LCDB) [2] This paper will investigate if there are similarities amongst these curves by clustering those provided by the LCDB. The experiment employs two distinct input parameters: point vectors and statistical vectors. By conducting individual learner analysis, individual dataset analysis, principal component analysis, and other experiments, patterns are isolated for both input sets. Upon further exploration of shapes and distributions, the concluding remark is that the point vector clustering produced one key concrete pattern amongst certain learning techniques. In contrast, the statistical vector findings are more inconclusive and do not exhibit a clear distinction that could be linked to any dominant patterns.

This study explores the extrapolation of learning curves, a crucial aspect in evaluating learner performance with varying dataset sample sizes. We use the Learning Curve Prior Fitted Network (LC-PFN), a transformer pre-trained on synthetic data with proficiency in approximate Bayesian inference, to investigate its predictive accuracy using the Learning Curve Database (LCDB). The assessment involves MSE as an error metric, with 2 baselines from previous studies where we see it outperform the baseline in some cases and keep on par in others. Additionally, we scrutinize instances where the LC-PFN model may exhibit shortcomings to identify trends in curve extrapolation failures, offering insights for potential modifications to the training dataset. We see a pattern in learners where LC-PFN performs consistently poorly on, whereas no significant pattern can be seen for datasets.

This research investigates the impact of data imbalances on the learning curve using the nearest mean model. Learning curves are useful to represent the performance of the model as the training size increases. Imbalanced datasets are often encountered in real-life scenarios and pose challenges to standard classifier models impacting their performance. Thus, the research question is ”How do data imbalances affect the learning curves using the nearest mean model?”. To answer the question, an experiment is conducted using data from a multivariate Gaussian distribution to sample varying levels of imbalances. The imbalance ratio explored is [0.1, 0.2, 0.3, 0.4, 0.5], representing the percentage of the dataset that consists of the minority class. The findings indicated that as the data becomes more imbalanced, the learning curves reach the accuracy plateau at a later rate. The analysis of the curve parameter which follows the logistic function suggests that imbalances have an impact on the maximum achievable accuracy and rightward shift of the curves. However, the maximum achievable accuracy is non-significant and the shape of the curves remains similar. Additionally, false negatives have a significant impact on the learning curves.

Does a convolutional neural network (CNN) always have to be deep to learn a task? This is an important question as deeper networks are generally harder to train. We trained shallow and deep CNNs and evaluated their performance on simple regression tasks, such as computing the mean pixel value of an image. For these simple tasks we show that going deeper does not guarantee an improvement in performance.

With an expectation of 8.3 trillion photos stored in 2021 [1], convolutional neural networks (CNN) are beginning to be preeminent in the field of image recognition. However, with this deep neural network (DNN) still being seen as a black box, it is hard to fully employ its capabilities. A need to tune hyperparameters is required to have a robust CNN that can more accurately do its task. In this study, the batch size, being one of the most important hyperparameters, is our main concern. The batch size is the number of samples that will be propagated through the network before updating the weights. Moreover, we show how the batch affects the performance of Regression CNNs to the following regression tasks: the mean, median, standard deviation (std) and variance of the pixel intensities of a grey-scale MNIST [2] input image. This will be analyzed by how well regression CNNs converge, given different batch sizes and a fixed learning rate. Additionally, we will also be comparing the final mean squared error given by all different batch sizes. At the end of the research, our findings concluded that a higher batch size leads to a higher Mean Squared Error (MSE) and a slower convergence. Additionally, the best performance obtained was for batch sizes of size 8 to 32, with slight differences between the four different regressions tasks.

Learning curves illustrate the relationship between the performance of learning algorithms and the increasing volume of training data [1, 2, 3]. While the concept of learning curves is well-established, clustering these curves based on fitting parameters remains an underexplored area. Our study delves into this domain and leverages the Learning Curve Database (LCDB) to discover potential patterns. We investigate whether different curve models uncover distinct patterns, examine the impact of different datasets on these learners, and explore if various learners display unique characteristics and behaviors or adhere to a common pattern. Curve model analyses conclude that most of the data points are in a single cluster (dominant cluster), indicating a potential commonality. Certain learners, such as QuadraticDiscriminantAnalysis and PassiveAggressiveClassifier, exhibit unique traits and do not conform to this common pattern, regardless of dataset attributes. Moreover, while various learners demonstrate similar characteristics within a single curve model, distinct patterns emerged when comparing across different curve models, indicating internal similarity but external divergence in behavior.

Learning curves are useful to determine the amount of data needed for a certain performance. The conventional belief is that increasing the amount of data improves performance. However, recent work challenges this assumption, and shows nonmonotonic behaviors of certain learners on certain problems. This paper presents a new approach for detecting non-monotonicity in empirical learning curves. This method monitors the degree of monotonicity violation on non-monotonic intervals, using the performance difference. In addition, the accuracy of the algorithm is being assessed through a series of diverse experiments. The proposed algorithm is applied to a subset of the extensive Learning Curve Database (LCDB). The results indicate an experimental accuracy of 95.5% in identifying non-monotonicity within real learning curves. Importantly, the metric demonstrated its ability to distinguish genuine non-monotonic trends from minor fluctuations attributed to measurement errors.

The increasing demand for sustainable energy, results in more wind turbines being built offshore. The blades of wind turbines consist of composites which makes them difficult to design. Because composites consist of a micro- and macro-structure of which their interplay determines the global behavior under loads. Therefore, traditional methods of analysis are often infeasible: full-scale experiments take too long and are too costly, small-scale experiments can not capture the interaction between the micro- and macro-structure, and analytical theories are often only tractable for simple cases. $FE^2$ is one such method that can analyze the behavior of multi-scale materials, such as composites. It consists of a macro finite element model where the constitutive behavior of each integration point is obtained by homogenizing representative volume elements (RVE) that represent the microstructure. Although this method is capable of accurately predicting composites, the computational cost is high, as for each integration point another finite element model must be computed. This process can be sped up by replacing the RVE with a surrogate that is more efficient to compute. Recently, Gaussian Process Regression (GPR), a probabilistic machine learning model, is used as a surrogate for the RVEs. It uses prior knowledge and observations of the RVE to make its predictions. It is based on Gaussian Processes, meaning that the predictions exhibit a multivariate Gaussian distribution which provides an uncertainty measure besides its prediction. This is useful in determining where new observations must be collected from the RVE. To fully capture the RVE a lot of observations are needed from it. This is still a computational bottleneck for the surrogate as they are expensive to compute. The GPR model can be enhanced with observations from a low-fidelity model, for example, linear elastic, which is called GPR with multi-fidelity. The low-fidelity observations, inexpensive and inaccurate, enhance the prediction of the high-fidelity model, which uses high-fidelity observations that are expensive but accurate. This is shown to decrease the number of observations needed for the high-fidelity model. Thus, fewer observations need to be collected from the RVE. However, the correlation between the low- and high-fidelity is often assumed to be constant in these models. This presents a problem as the correlation in the case of surrogate modeling is often non-linear. The literature investigates several extensions of the GPR with multi-fidelity that assume a non-linear correlation, but these models become more complex and lose their simplicity as their predictions are not Gaussian anymore. Instead, this thesis keeps the constant correlation assumption but improves the correlation inference by splitting the model. Splitting means that multiple independent GPR with multi-fidelity models are used in different regions. As splitting results in discontinuous predictions, the thesis also investigates two stitching methods to remove these discontinuities. The first, the constrained boundary (CB) method, constrains the predictive mean and predictive variance of to neighboring models to be equal at their respective boundaries. These constraints to the optimization procedure of the hyperparameters of the local GPR with multi-fidelity models. The second stitching method, the random variable mixture (RVM) uses a weighing procedure based on a mixture of experts approach. However, this method mixes the random variables instead of the probability density distributions. This creates a much simpler and analytically tractable method. For both of these stitching methods, only the high-fidelity uses the stitching procedure while the low-fidelity is split...