I had an opportunity to present a research project done on Apache ZooKeeper’s performance. It was the culmination of a semester-long project on distributed computing with a group organized by Professor Benjamin Reed at San Jose State University.
The origin of this project starts with a benchmark suite written by Dr. Reed for the 2010 USENIX ZooKeeper Whitepaper. Those benchmarks still exist on the ZooKeeper GitHub repository but haven’t been run for over a decade. The project goals were to reproduce those benchmarks for every Apache ZooKeeper release to understand regressions and improvements in throughput performance. We were able to produce automated scripts that check out past releases of ZooKeeper, create an ensemble, and run throughput tests on the system.
Although I’ve graduated since this project, our group was able to establish fundamentals that will be built upon in later semesters. Our future goals include developing benchmark hooks for new pull requests of Apache ZooKeeper, introducing latency benchmarks, and hopefully, contributing some of our work into the original Apache ZooKeeper repository for others to use.