Increasingly, businesses are trying to achieve better insights out of large amounts of streamed data by bringing it closer to real time processing power and querying from a live data mart. At the moment we are witnessing a convergence of workflows and technology platforms for real time, analytics, cloud, and in-memory processing. This allows us to effectively address time-sensitive business decisions that involve the volumes of big data while benefiting from the velocities of real time processing. This involves real time analytics capabilities for price optimizations, fraud detection, risk calculations, and operational business intelligence.

In this session, we present a fast data pipeline solution architecture combining in-memory computing and a high performance Spark distribution. We will examine a large-scale transportation use case who implemented this solution for IoT real-time operational intelligence against billions of edge/sensor data points.