This thesis investigates the performance of distance protection function of actual relay placed on high voltage 400 kV transmission line in a system with high photovoltaic (PV) sources penetration. The benchmark model of system with radial connection of PV plant and conventional power plant was designed in the platform of RSCAD software. By hardware-in-the-loop (HiL) test setup two commercial relays were connected to the Real-Time Digital Simulator (RTDS) and linked to two virtual breakers in transmission line ends modelled in RSCAD. Special logic for correct signal processing of analogical inputs and binary outputs of relays was designed and employed in RSCAD. Several test scenarios were recreated and analysed. Distance protection function was tested with radial connection of synchronous generator to check response of benchmark system and validate correct settings of the relays. Afterwards synchronous generator was substituted by generic PV farm model. Impact of different fault locations, types of fault, power generation level and fault impedance values were studied on a basis of the benchmark model. The goal of this research was to determine the extreme scenarios in the power system when real relay would fail to detect the fault conditions. Failures of distance protection function operation were detected and explained from mathematical side. On the basis of this research the author revealed the cases of relay maloperation and suggested possible ways of solution to this problem by the set of recommendations.