Neural Network Output Optimization Using Interval Analysis

Neural Network Output Optimization Using Interval Analysis

De Weerdt, E.
Chu, Q.P.
Mulder, J.A.

Aerospace Engineering

Control & Operations

2009-04-03

The problem of output optimization within a specified input space of neural networks (NNs) with fixed weights is discussed in this paper. The problem is (highly) nonlinear when nonlinear activation functions are used. This global optimization problem is encountered in the reinforcement learning (RL) community. Interval analysis is applied to guarantee that all solutions are found to any degree of accuracy with guaranteed bounds. The major drawbacks of interval analysis, i.e., dependency effect and high-computational load, are both present for the problem of NN output optimization. Taylor models (TMs) are introduced to reduce these drawbacks. They have excellent convergence properties for small intervals. However, the dependency effect still remains and is even made worse when evaluating large input domains. As an alternative to TMs, a different form of polynomial inclusion functions, called the polynomial set (PS) method, is introduced. This new method has the property that the bounds on the network output are tighter or at least equal to those obtained through standard interval arithmetic (IA). Experiments show that the PS method outperforms the other methods for the NN output optimization problem.

feedforward neural networks (FFNNs)
global optimization
inclusion function
interval analysis
optimization methods
polynomial set
radial basis function neural networks (RBFNNs)
taylor expansion
taylor model (TM)

http://resolver.tudelft.nl/uuid:b0d2cc9f-c6b9-49b0-91e5-d5ef37e32ecb

IEEE

1045-9227

IEEE Transactions on Neural Networks, 20 (4), 2009

Institutional Repository

journal article

(c) 2009 The Author(s) ; IEEE