“Fast Data” is the next wave in the shake up of enterprise systems. Companies are moving from systems designed for data “at rest” in data warehouses and embracing the value of data “in motion” at the edge–whether from user data, sensors or humans.
Extracting information from incoming data as quickly as possible has become a competitive advantage. Stream processing is the best option and it has sparked new tools, frameworks, and architecture patterns for developers creating applications and systems designed specifically for real-time data.
At the developer level, there is an explosion of frameworks specifically targeting Fast Data and streaming applications. Spark, Kafka, Akka, Akka Streams, Gearpump, Flink – the list goes on and on.
But this can also be confusing; which framework should development teams use for which use cases, which ones have overlapping functionality with the others, and so on.
We’re also seeing some fundamental ways that data streams are forcing rethinking of application and system design, because now your applications are on 24×7, potentially for months or years…
Lightbend’s upcoming Fast Data Platform (FDP) is aiming to create the first complete platform for data teams that are moving from the classic Big Data architectures to the newer Fast Data architectures.
FDP simplifies cluster installation and management for these frameworks built to run in a distributed fashion. This makes it easier to deploy, monitor, and scale applications in production so that development is more productive and production operations are more reliable.
For more about Lightbend Fast Data Platform, visit the full story (including an awesome video!).