We study different possibilities to use adaptive optics (AO) and phase diversity (PD) together in a jointly optimized system. The potential of the joint system is demonstrated through numerical simulations. We find that the most significant benefits are obtained from the improved deconvolution of AO-corrected wavefronts and the additional wavefront sensor (WFS) information that reduces the computational demands of PD algorithms. When applied together, it is seen that the image error can be reduced by 20% compared to traditional PD, working with one focused and one defocused camera image, and the computational load is reduced by a factor of 20 compared to a more reliable PD algorithm requiring more camera images. In addition, we find that the system performance can be optimized by adjusting the magnitude of the applied diversity wavefronts.