Cv
C.C.J. van Engelenburg
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
1 records found
1
Breaking Under Pressure? Scalability and Robustness of Kolmogorov-Arnold Networks
How does the predictive accuracy of Kolmogorov-Arnold Networks (KANs) compare to Multi-Layer Perceptrons (MLPs) as data dimensionality and noise levels increase?
The introduction of Kolmogorov-Arnold Networks (KANs) as an alternative to traditional Multi-Layer Perceptrons (MLPs) has recently gained significant attention, with promising claims about their ability to overcome the curse of dimensionality and achieve superior parameter efficiency compared to MLPs. However, their practical scalability and robustness are not yet fully understood. To bridge this gap, this study provides an empirical analysis of the effects of data dimensionality and noise on the performance of KANs relative to MLP. Our analysis demonstrates that although KANs scale poorly - especially compared to MLPs in simple function-fitting tasks - their empirical data requirements appear to grow sub-exponentially with dimension, which is consistent with resilience to the curse of dimensionality - though we show this result is sensitive to initialization and may reflect a pre-asymptotic regime rather than a genuine architectural property. Furthermore, while both architectures show similar, graceful degradation under increasing Gaussian noise, they struggle significantly with outlier noise in high-dimensional settings. This research provides fellow researchers with insight into the inductive biases and limitations of KANs, highlighting research gaps that need to be addressed to ensure their general applicability.
...
The introduction of Kolmogorov-Arnold Networks (KANs) as an alternative to traditional Multi-Layer Perceptrons (MLPs) has recently gained significant attention, with promising claims about their ability to overcome the curse of dimensionality and achieve superior parameter efficiency compared to MLPs. However, their practical scalability and robustness are not yet fully understood. To bridge this gap, this study provides an empirical analysis of the effects of data dimensionality and noise on the performance of KANs relative to MLP. Our analysis demonstrates that although KANs scale poorly - especially compared to MLPs in simple function-fitting tasks - their empirical data requirements appear to grow sub-exponentially with dimension, which is consistent with resilience to the curse of dimensionality - though we show this result is sensitive to initialization and may reflect a pre-asymptotic regime rather than a genuine architectural property. Furthermore, while both architectures show similar, graceful degradation under increasing Gaussian noise, they struggle significantly with outlier noise in high-dimensional settings. This research provides fellow researchers with insight into the inductive biases and limitations of KANs, highlighting research gaps that need to be addressed to ensure their general applicability.