This work proposes a control strategy to follow time optimal trajectories planned to visit a given set of waypoints in windy conditions. The aerodynamic effects of quadrotors are investigated, with emphasis on blade flapping, induced and parasitic drag. An extended method to identify all the aerodynamic coefficients is developed, and their influence on the performance is analyzed. A computationally efficient three steps approach is suggested to optimize the trajectory, by minimizing aerodynamic drag and jerk while still guaranteeing near optimal results. The derived smooth trajectory is compared with standard discrete point to point followed by low-pass filtering trajectories, showing energetic improvements in thrust and reductions in Euler angles aggressiveness. By exploiting the non-linear aerodynamic effects and using a priori trajectory information, a thrust vectoring controller is designed and compared with a standard PID controller, showing an increase in performance by reducing the tracking delay and extending the flight envelope.