Neglecting actuator dynamics in nonlinear control and control allocation can lead to performance degradation, especially when considering fast dynamic systems. This paper provides a novel method to account for actuator dynamics in the nonlinear control allocation solution: dynamic incremental nonlinear control allocation, or D-INCA. The incremental approach allows for the implementation of a first order discrete-time actuator dynamics model in the quadratic programming solver. This model is used to find the optimal command inputs in addition to the desired physical actuator deflections, hereby compensating for actuator dynamics delays. D-INCA does not require feedback of higher order output derivatives than the baseline INCA and can be used with nonlinear non-control affine systems. Furthermore, with adaptive DINCA, or AD-INCA, an actuator dynamics parameter estimator is introduced to adapt the actuator model online, minimizing actuator tracking errors after actuator failures.