Solving optimization problems is a recurrent theme across different fields, including large-scale machine learning systems and deep learning. Often in practical applications, we encounter objective functions where the Hessian is ill-conditioned, which precludes us from using optimization algorithms utilizing second-order information. In this paper, we propose to use fractional time series analysis methods that have successfully been used to model neurophysiological processes in order to circumvent this issue. In particular, the long memory property of fractional time series exhibiting non-exponential power-law decay of trajectories seems to model behavior associated with the local curvature of the objective function at a given point. Specifically, we propose a NEuro-inspired Optimization (NEO) method that leverages this behavior, which contrasts with the short memory characteristics of currently used methods (e.g., gradient descent and heavy-ball). We provide evidence of the efficacy of the proposed method on a wide variety of settings implicitly found in practice.

Most optimization problems lack closed-form solutions of the argument that minimizes a given function, and even if these were available it might be prohibitive to compute it. As such, we rely on iterative numerical algorithms to find an approximate solution. In this paper, we propose to leverage fractional calculus in the context of time series analysis methods to devise a new iterative algorithm. Specifically, we propose to leverage autoregressive fractional-order integrative moving average time series, whose coefficients encode a proxy for local spatial information. We provide evidence that our algorithm is efficient and particularly suitable for cases where the Hessian is ill-conditioned.