October 31 - November 1 - Co-Located Events
October 28-30 - Conference
Lyon Convention Centre - Lyon, France
More information for Open Source Summit + Embedded Linux Conference Europe 2019
Back To Schedule
Monday, October 28 • 14:25 - 15:00
Logging with Data Insights: Stream Processing on the Edge - Eduardo Silva, Treasure Data

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
In containerized and distributed environments, observability tools need to correlate specific information associated with running applications like messages, context, and metadata. Logging is one of the principal's areas and its common workflow is to collect, parse, filter and ship logs to a centralized database, so data analysis can be performed.

This traditional mechanism works and scale, but it also has some penalties associated with performance and timings where a faster feedback loop is desired. How can we improve Logging to get faster "data insights" and take actions like "alerting" based on specific conditions? can we make it better ?.

On this presentation, we will introduce and demonstrate the concept of "Stream Processing on the Edge", which provides the ability to perform data processing while it's still in motion (in-memory). Mixing Logging with Stream Processing brings huge advantages in performance to get faster results and extend to capabilities like alerting and ML.

avatar for Eduardo Silva

Eduardo Silva

Principal Engineer, Arm Treasure Data
Eduardo is a Principal Engineer at Arm Treasure Data, he is the author and maintainer of Fluent Bit Log Processor, a CNCF sub-project under the umbrella of Fluentd. He is an international speaker in Open Source conferences, he has participated in Scale California, LinuxConf AU, Linux... Read More →

Monday October 28, 2019 14:25 - 15:00 CET
Bellecour 2