Implementing and Preforming Randomized Tests on the HotStuff BFT Protocol

Abstract

Although Byzantine Fault Tolerant (BFT) protocols such as HotStuff are nominally resistant to a number of faulty or unreliable participants, implementation or design errors can cause violations in their expected properties. Because of this, it is useful to have reliable automated testing frameworks that can simulate Byzantine behaviour to make bug detection easier. In this paper, we examine the performance of the ByzzFuzz BFT testing tool using our implementation of the HotStuff protocol. We describe the design choices necessary to create a working HotStuff implementation. Then we purposefully introduce implementation flaws to evaluate the behaviour of ByzzFuzz with different parameters and mutation scopes. We compare its performance to that of a baseline random fault injection scheduler. Our results show that it was able to detect the introduced bugs using either process or network faults. ByzzFuzz's partition-based network faults were more effective at detecting bugs than the 'Random' scheduler's network faults. For process faults, we were unable to register significant differences in performance possibly due to HotStuff's simplistic pipelined structure. In our tests, any-scope mutations performed better than their small-scope counterparts for the same configuration. This could be attributed to the nature of the selected faults and HotStuff's pipelined structure.