Quantum networks provide numerous potential benefits over classical networks, such as enhanced security and faster computation, making their further development a lucrative prospect. As is the case with any technology, the advancement of quantum networks relies on the development of frameworks to test their quality, and compare different implementations of the technology. One such framework is a benchmarking suite for quantum network systems, that can identify areas for improvement in their implementation, by determining the erroneous properties of the system. This paper examines the viability of using a specific quantum network application as a benchmark for quantum network systems. In order to quantify the application's ability to benchmark, we assess its sensitivity to changes in the properties of the system. These properties include link parameters, quantum gate properties, qubit coherence times, and measurement properties. We use the BB84 protocol as the benchmarking application for this project, which is a Quantum Key Distribution scheme used to establish secure keys between two parties. In particular, we use the qubit error rate and the key generation rate as the performance metrics for the application. For the setup of the experiments, we prepare two system configurations: generic quantum device nodes with a depolarising error channel, and NV device nodes with a heralded link. In order to assess how the application behaves with changes to different system properties, we observe how the performance metrics change while individually varying system parameters and keeping all other parameters constant. We find that the application is sensitive to changes in multiple parameters across both network configurations, such as link parameters, single qubit gate properties, and measurement properties. Contrarily, the application is not affected by changes to parameters such as two qubit gate properties and coherence times. We conclude that the BB84 protocol can be used as an individual localised test for the parameters it is sensitive to, and also in combination with other applications, in a more comprehensive benchmarking suite, that provide coverage for a broader range of parameters.