Learning State Machines in Real-Time on a Small Dedicated Hardware Device

Abstract

The Internet is a technology that was invented in the 1960s and was used only by a few users to do simple communications between computers. Fast forward to 2020, the Internet has become a technology that is being used by billions of users. It allows users to communicate with each other across the world and even allows users to access data without geographic restrictions. The Internet has made the lives of many people easier but it also comes with a price; many malicious users also want to have access to data. Therefore, it is needed to secure our networks to make sure that no attackers can exfiltrate data from a network. One way to do so is to use smart methods to detect anomalies in the network. Recently, a new method has been proposed to learn state machines in real-time from network traffic data. The state machines are then used for anomaly detection. This method was designed to be used on a larger system such as a desktop computer. In this work, we investigated how we can use the newly proposed method to learn state machine in realtime on a smaller device. Smaller devices are cheaper and more mobile than larger systems but these have limited resource compared to the larger systems. Therefore, modifications would need to be made to the method for it to run efficiently on a smaller device. In this work, we propose to use the concepts of Locality Sensitive Hashing to improve the run-time of different parts of the method. We also attempted to reduce its memory footprint. In this work, we show the modifications that we have made and evaluated our modifications with different experiments that used both artificial and real-world data. From our results, it shows that we can use a smaller device to learn state machines in real-time and use these state machines for anomaly detection. Though our modifications have provided an improvement on parts of the method, there are still improvements that can be done.