Stochastic Control with Complete Observations on a Finite Horizon

Abstract

Optimal stochastic control problems are formulated for a stochastic control system with complete observations on a finite horizon. Dynamic programming yields necessary and sufficient conditions for optimality rather than local optimality conditions as provided by methods based on the calculus of variations or on the maximum principle. Sufficient conditions are formulated for a subset of value functions to be invariant with respect to the dynamic programming operator. Reduction in complexity of a stochastic control system with the controlled output signal is proven using dynamic programming. Examples include: the linear–quadratic–Gaussian optimal control problem, a gambling problem with an exponential value function, and a finite stochastic control system.