Open Source Summit + ELC Europe 2019

The common practice of benchmarking databases and other applications on platforms like Kubernetes by running an application benchmark on an idle platform is not ideal: it does not give insights into how performance will be impacted in production when noisy neighbors are sharing the platform.

In this talk, Manoj Pillai will introduce Controlled Ambient Load Mixing (CALM), a new benchmarking methodology which involves running a benchmark concurrently with a steady background load, and iterating the process with different background load levels. The talk will describe an implementation of CALM aimed at storage workloads and provide results for some popular database benchmarks, demonstrating the insights CALM can provide into application and platform performance characteristics, like application sensitivity to noisy neighbors, while maintaining reproducibility of results.