A novel over-the-air (OTA) measurement setup for simultaneous and near-real-time characterization and mapping of the phased array radiation patterns to the error-vector-magnitude (EVM) is introduced. A cost-effective software-defined radio (SDR) system, including a Pluto SDR for both transmission and reception, is employed. For setup demonstration, the performance of a $4 \times 4$ active phased array antenna (APAA) is studied at 26.5 GHz. Calibration of the transmitter results in an EVM of 0.5%, proving the accuracy of the measurement system. The over-the-air measured EVM of the APAA is 1.2%. The proposed setup and approach provide deep insights into how array radiation characteristics affect signal quality, advancing the evaluation and development of APAAs for next-generation wireless communication systems.