Analysis of high-volume traffic using Complex Event Processing and a Domain Specific Language

Abstract

In the online travel environment it is physically and economically not possible to retrieve the hotel room rates in real-time for every customer request. To overcome this problem the hotel rates are cached, but due to the fact that the suppliers will not send notifications of price changes it is a challenging task to keep the cache up to date. Analysis of the problem of how to improve the cache’s performance, in terms of accuracy and coverage, led to the conclusion that there is no single risk-free overall solution, but that the cache should be enhanced in a step-wise manner minimizing the (financial) impact of a faulty enhancement. The Enhancement Cycle is defined as a number of stages that each step-wise enhancement will go through. The system provides a solution for two of those stages. The system measures the cache performance in a correct manner and predicts the expiration time for a hotel rate. The system uses Complex Event Processing to detect the patterns needed to do the measurements. The system has been generalized and is extended with a Domain Specific Language for a low-effort application to other problems. The system is running in production and processes millions of messages a day. It aggregates and compresses the measurement data without loosing expressivity and its predictions improved the cache’s accuracy.