Riding the Jet Streams
Java 8 introduced the Stream API as a modern, functional, and very powerful tool for processing collections of data. One of the main benefits of the Stream API is that it hides the details of iteration over the underlying data set, allowing for parallel processing within a single JVM, using a fork/join framework. I will talk about a Stream API implementation that enables parallel processing across many machines and many JVMs. You will learn how you can use the same API to process massive data sets across large clusters, which you already know how to do in a single JVM. With an explanation of internals of the implementation, I will give an introduction to the general design behind stream processing using DAG (directed acyclic graph) engines and how an implementation provides in-memory performance while still leveraging industry-wide known frameworks as Java Streams API.
Christoph Engelbert is Manager of Developer Relations at Hazelcast. He is a passionate Java developer with a deep commitment for Open Source software. He mostly is interested in Performance Optimizations and understanding the internals of the JVM and the Garbage Collector. He loves to bring software to it's limits by looking into profilers and finding problems inside of the codebase.