Many applications, such as video streaming, congestion control, and server selection, can benefit when the data rate of different priority groups between two endpoints is accurately estimated over the end-to-end path. With the introduction of programmable networks, e.g., P4, it is now possible to offload the measurements to the data plane of intermediate devices. Recently, tools have been developed to react to changes in available bandwidth, but a tool to accurately estimate end-to-end per-priority data rates needs to be added. This motivates us to design and implement a new end-to-end and per-priority data rate estimation tool, PrioMeter. PrioMeter can accurately report the data rate per priority group of flows in programmable networks using high-precision timestamps for arbitrary traffic scales. PrioMeter leverages two primitives: quantization and truncation, to achieve its goals. We implement PrioMeter in P4 and test it on BMv2 switches, and our preliminary results using NS3 simulations show that it can accurately estimate the data rate of different priority flows with minimal overhead.

Big data analytics platforms have played a critical role in the unprecedented success of data-driven applications. However, real-time and streaming data applications, and recent legislation, e.g., GDPR in Europe, have posed constraints on exchanging and analyzing data, especially personal data, across geographic regions. To address such constraints data has to be processed and analyzed in-situ and aggregated results have to be exchanged among the different sites for further processing. This introduces additional network delays due to the geographic distribution of the sites and potentially affecting the performance of analytics platforms that are designed to operate in datacenters with low network delays. In this paper, we show that the three most popular big data analytics systems (Apache Storm, Apache Spark, and Apache Flink) fail to tolerate round-trip times more than 30 milliseconds even when the input data rate is low. The execution time of distributed big data analytics tasks degrades substantially after this threshold, and some of the systems are more sensitive than others. A closer examination and understanding of the design of these systems show that there is no winner in all wide-area settings. However, we show that it is possible to improve the performance of all these popular big data analytics systems significantly amid even transcontinental delays (where inter-node delay is more than 30 milliseconds) and achieve performance comparable to this within a datacenter for the same load.