Open Source Summit + ELC Europe 2019

In containerized and distributed environments, observability tools need to correlate specific information associated with running applications like messages, context, and metadata. Logging is one of the principal's areas and its common workflow is to collect, parse, filter and ship logs to a centralized database, so data analysis can be performed.

This traditional mechanism works and scale, but it also has some penalties associated with performance and timings where a faster feedback loop is desired. How can we improve Logging to get faster "data insights" and take actions like "alerting" based on specific conditions? can we make it better ?.

On this presentation, we will introduce and demonstrate the concept of "Stream Processing on the Edge", which provides the ability to perform data processing while it's still in motion (in-memory). Mixing Logging with Stream Processing brings huge advantages in performance to get faster results and extend to capabilities like alerting and ML.