Finding Bottlenecks at Scale in Consensus Algorithms

Consensus mechanisms for ensuring consistency are some of the most expensive operations in managing large amounts of data. Often, there is a trade off that involves reducing the coordination overhead at the price of accepting possible data loss or inconsistencies. As the demand for more efficient data centers increases, it is important to provide better ways of ensuring consistency without affecting performance.

In recent years, there have been several research proposals around Crash Fault Tolerant (e.g.,[1] and [2]) and Byzantine Fault Tolerant consensus protocols (e.g., [3] and [4]) that, even though offer promising results on a handful of machines, have not been evaluated at large scale. With the increasing importance of Distributed Ledger-based systems, it is important to evaluate the state of the art consensus libraries and identify the bottlenecks they face at scale on 100s of nodes!

Working on this project in the DASYA Lab, you will have the opportunity to perform experiments in the cloud but also build analytical models of the behavior and learn in detail about the different consensus protocols.

[1] [https://dl.acm.org/doi/abs/10.1145/2749246.2749267] [2] [https://www.usenix.org/conference/atc14/technical-sessions/presentation/ongaro] [3] [https://arxiv.org/abs/1906.05552] [4] [https://ieeexplore.ieee.org/abstract/document/6903593]