To solve the problem of optimal control for nonlinear system, Actor Critic Designs (ACD) can be utilized which use the concept of Reinforcement learning (RL) and function approximators such as Neural networks (NN). Traditional ACD methods require a model NN that needs to be trained offline. Recently, research focus has been shifted to model-free approaches that do not require any model information beforehand and can be applied for online control. This thesis furthers the online methods in ACD by developing Incremental Model based Action Dependent Dual Heuristic Programming (IADDHP). In IADDHP, local system dynamics is identified online which does not require any priori knowledge about the system thus making it essentially ‘model-free’. Experiments are performed using missile model for reference tracking control and the results show that the IADDHP is capable of finding near-optimal control policy for the tasks with noise and system failure. It also outperforms the already existing model-free ADDHP which uses finite difference method (FDM) and has advantage over it in failure detection and adaptation. Being a model-free approach, IADDHP should be applicable for reference tracking control of any system.