Stay Ahead Of The Curve: Key Insights From Forrester On Streaming Data Platforms

Streaming data is the pulse of an enterprise. It represents customers navigating websites to find products, supply chains that are about to break, and pricing that starts to soar or plummet. Streaming data represents the real-time reality of what is happening to the business, internally and externally. That’s why enterprises must leverage streaming data to attain blazing-fast reflexes to conquer intraday business challenges and opportunities. All data starts as streaming data because streaming data is simply data that has just been born in an application. The question is, how can enterprises leverage that data? Streaming data platforms enable enterprises to capture, analyze, and act in real time on streaming data that originates in any application without refactoring its existing architecture.

As a result of these trends, streaming data platform customers should look for providers that offer:

• A breadth of core capabilities. The three core streaming data capabilities are movement, analytics, and processing. Movement is connecting to enterprise data sources in real time and making that data available to any applications that need it. Analytics is gleaning insights from streaming data whether simple event triggers, aggregate queries, or advanced analytics, including machine learning. Processing is performing any number of arbitrary transformations, joins, and enrichments on streaming data in real time. It’s common for enterprises to already have one or more of these core capabilities. For example, many enterprises have adopted open-source Apache Kafka (on-premises or in cloud) to move and deliver streaming data, but lack a robust streaming analytics capability. Thus, enterprise buyers should either look to vendor solutions to augment their existing core streaming capabilities or look to a new, unified streaming data platform to replace one-off implementations.
• Development tools that increase productivity and support fast change. Raw streaming data technology is insufficient for enterprises that wish to implement a steady stream of new use cases. They need tools to design, develop, and deploy additional use cases and manage the platform. Fortunately, tooling has become a major focus of vendor innovation now and on roadmaps. All vendors offer a code-based interface and many now offer tools such as visual streaming pipeline builders, SQL consoles, and other tools aimed at both developer productivity and enabling access to the platform for less technical developers. Enterprise buyers should match their existing developer maturity to the tools that the vendor offers, but also anticipate how additional roles may use the platform, including IT ops and citizen developers.
• Efficient and fault-tolerant scalability. The nature of streaming data is such that it can roar like Niagara Falls and then suddenly, without warning, gurgle like a Zen fountain. Streaming data platforms must be able to handle both of these situations: scale up to handle huge volumes of data, and scale down to avoid wasting compute, storage, and network resources. Enterprise buyers should anticipate and incorporate current and future requirements for scalability in terms of throughput, latency, efficiency (resource utilization), and fault tolerance into their buying criteria.

Evaluation Summary

The Forrester Wave™ evaluation highlights Leaders, Strong Performers, Contenders, and Challengers. It’s an assessment of the top vendors in the market; it doesn’t represent the entire vendor landscape. You’ll find more information about this market in our report The Streaming Data Platforms Landscape, Q3 2023.

Figure 1
Forrester Wave™: Streaming Data Platforms, Q4 2023

Figure 2
Forrester Wave™: Streaming Data Platforms Scorecard, Q4 2023

Vendor Offerings

Forrester evaluated the offerings listed below (see Figure 3).

Figure 3

Vendor Profiles

Our analysis uncovered the following strengths and weaknesses of individual vendors.

Leaders

• Confluent is a streaming force to be reckoned with.Confluent started in 2014 as the preeminent Kafka company started by the founders of the 2011 open-source project that originated in LinkedIn. Fast forward to today, Confluent has grown rapidly and expanded its portfolio to offer a full complement of streaming data platform capabilities including, via a recent acquisition, Apache Flink to gain more stream-processing capabilities. Its superior vision is to transform streaming from an enterprise architecture nice-to-have to a must-have. Confluent’s key strength lies in its offering of cloud-native versions of Kafka that also adds developer tools to make creating streaming applications easier to develop and manage. Confluent’s strategy is to provide a balanced mix of cloud-agnostic, open-source solution and value-added capabilities that make it hard to beat.
  
  The core strengths of Confluent’s product offering are in data movement, data connectors, developer tooling, fault tolerance, and solid capabilities in analytics. Confluent can improve by integrating the Apache Flink engine from its recent acquisition. Reference customers appreciate that Confluent Cloud is portable across cloud service providers and is centered on a significantly improved version of open-source Apache Kafka. Confluent is a good fit for customers who want a solution that is based on Apache Kafka, is cloud portable, and provides ample capabilities for both streaming analytics and processing.
• Microsoft makes world-class streaming data easy to use at cloud scale. Microsoft’s streaming platform uses Azure Event Hubs to receive streaming data from millions of devices. That real-time data in turn can be directed toward numerous other Azure services including Azure Stream Analytics to analyze and take actions on streaming data in real time. Microsoft’s strategic strength lies in hiding the gnarly complexity of cloud-scale streaming data behind no-code development tools or SQL, making world-class streaming data accessible to all levels of developers.
  
  Microsoft has strengths in streaming analytics, developer tools, low latency, and fault tolerance. Microsoft can improve the platform by adding more native data connectors. Reference customers appreciate how quickly developers can learn to develop streaming data applications, the seamless integration with other Azure services, and scalability. Microsoft is a good fit for existing Azure customers as well as new customers who wish to quickly build an enormously scalable IoT application or any other cloud-scale application.
• Google offers a streaming data platform at Google scale. Google’s streaming platform is centered on its Cloud Dataflow service that seamlessly integrates with many other services such as Cloud Pub/Sub, BigQuery, Vertex AI, and Looker. Developers can use Cloud Dataflow to create both streaming and batch applications using the same programming paradigm: SQL or APIs. In addition to horizontal autoscaling, Google also uniquely offers vertical autoscaling to optimize the underlying resources based on the characteristics of the workload. Google consistently invests in Dataflow to seamlessly and more efficiently work with sister cloud services including real-time AI.
  
  Google has strengths in all core streaming data capabilities (movement, analytics, processing), efficiency via autoscaling, fault tolerance, and scalability. Google can improve the platform by enhancing no-code developer tools and adding more data connectors. Reference customers appreciate the integration with other Google Cloud services, autoscaling for throughput, and cloud pricing. Google is a good fit for existing Google Cloud customers and for new customers who want precise autoscaling that minimizes the cost of high volume yet spikey streaming workloads.

Contenders

• IBM offers world-class streaming capabilities but got lost in Cloud Pak. IBM offers IBM Streams in IBM Cloud Pak for Data which includes dozens of various data-related services. IBM Streams shines in the breadth and depth of real-time analytics it offers. However, IBM doesn’t have a strong strategy specifically for the streaming data platform market. Customers scrolling through IBM Cloud Pak for Data services may see IBM Streams as a niche capability rather than an essential platform for the real-time enterprise.
  
  IBM has strengths in data connectors, streaming analytics, developer tools, and low latency. IBM can improve the platform by more tightly integrating data movement and data processing capabilities. IBM is a good fit for enterprises that have a preponderance of data in IBM Cloud Pak for Data services. IBM declined to participate in the full Forrester Wave evaluation process.
• Ververica drives Apache Flink but must expand market awareness. Ververica was originally formed by the founders of the Apache Flink open-source project. Apache Flink has been so successful that many of the other vendors in this evaluation use the open-source version for stream processing. Ververica offers its own supercharged version of Apache Flink named the Ververica Cloud. Ververica is owned by Alibaba and certainly is used in the Alibaba Cloud, but it is also run as a separate company to market Ververica to enterprises. The company’s strategy is to continue to drive the Apache Flink open-source community while also offering the most performant version of Apache Flink available. The company needs to significantly ramp up sales and marketing to be more competitive. Ververica must better articulate its vision for Apache Flink in the enterprise and innovate much more on tooling for developers and other enterprise roles.
  
  Ververica has strengths in stream processing, streaming analytics, and throughput. Ververica can improve the platform by integrating more options for data movement and more developer tooling. Reference customers appreciate the enhanced performance and hyper scalability added on top of Apache Flink. Ververica is a good fit for customers who wish to use an enhanced version of Apache Flink for hyperscale real-time applications.
• Aiven offers a multicloud solution but needs to add developer tooling. Aiven bills itself as “the trusted open source data platform for everyone.” To wit, Aiven offers a bevy of open-source databases and streaming services that run on all major cloud service providers and private clouds. Aiven’s streaming data platform is comprised primarily of Aiven for Apache Kafka and Aiven for Apache Flink. Aiven’s strategy is to offer customers very portable, cost-effective, and managed versions of the most popular open-source data platforms. Aiven needs to articulate a broader real-time vision beyond just offering open-source Kafka and Flink. It could improve its roadmap by adding developer tooling and/or more integration with third-party tools.
  
  Aiven’s strength is in offering a full portfolio of fault-tolerant, open-source data projects, including Aiven for Apache Kafka, Aiven for Apache Flink, and more. Aiven can improve its streaming platform by adding development tools that make it easier for customers to build applications above and beyond coding interfaces. Reference customers appreciate the low cost and flexibility of running key open-source projects on any cloud. Aiven is a good fit for enterprises that have standardized on open source but want to simplify infrastructure management and have the flexibility in deployment to be portable to run on any public cloud or cloud infrastructure.
• Software AG focuses on real-time integration and IoT; ignores broader streaming market. Software AG streaming data platform is comprised of a combination of streaming analytics embedded in an IoT platform and its recent acquisition of StreamSets. When it comes to streaming data platforms, Software AG focuses its capabilities on industrial IoT and continuous integration use cases.
  
  Software AG has strengths in data connections and streaming analytics. Software AG can improve its platform by improving autoscaling and developer tooling. Software AG is a good fit for customers that already use other Software AG integration tools and/or customers that wish to implement an industrial IoT application. Software AG declined to participate in the full Forrester Wave evaluation process.

Challengers

• Red Hat offers open-source streaming capabilities but must add more value. Red Hat offers AMQ Streams, which is based on Apache Kafka running on Red Hat’s OpenShift cloud platform. Having an easier to manage version of Apache Kafka is but an opening wager for a streaming data platform. Enterprises expect more, including streaming analytics and a stream processing engine such as Apache Flink. To be a true player in the market, Red Hat must have a strategy to offer robust streaming analytics and stream-processing capabilities. Apache Flink would be a natural platform for Red Hat to add since it is open source and many of the other vendors base their streaming data platform on it.
  
  Red Hat has strengths in offering streaming data capabilities based on Apache Kafka and other open-source projects. Red Hat can improve its platform by improving additional core capabilities for streaming analytics and stream processing. Reference customers appreciate the ease with which they can deploy and manage open source with their larger OpenShift environment. Red Hat is a good fit for customers that wish to primarily use an Apache Kafka service within Red Hat OpenShift.

Evaluation Overview

We grouped our evaluation criteria into three high-level categories:

• Current offering. Each vendor’s position on the vertical axis of the Forrester Wave graphic indicates the strength of its current offering. Key criteria for these solutions include data, core, development, and scale.
• Strategy. Placement on the horizontal axis indicates the strength of the vendors’ strategies. We evaluated vision, innovation, roadmap, and partner ecosystem.
• Market presence. Represented by the size of the markers on the graphic, our market presence scores reflect each vendor’s revenue and number of customers.

Vendor Inclusion Criteria

Each of the vendors we included in this assessment has:

• Streaming data platforms as identified by Forrester. Vendors included in this evaluation were determined by Forrester to offer a streaming data platform as defined above and as represented in the Landscape report on this market.
• Comprehensive, differentiated streaming data platform. The vendors included in the evaluation offer a platform that provides one or more streaming data core capabilities: movement, analysis, and processing.
• A standalone streaming data platform. We included only solutions that are marketed as streaming data platforms and participate in the market by competing directly with other streaming data platforms. Streaming data platforms that Forrester deemed are technologically embedded into any particular application; business intelligence platform; data prep; extract, transform, load (ETL); databases; or middleware stacks weren’t included in this evaluation.
• Minimum product revenue requirement. Vendors included in this evaluation have greater than $10 million in revenue from the streaming data platform product.

Supplemental Material

Online Resource

The Forrester Wave Methodology

A Forrester Wave is a guide for buyers considering their purchasing options in a technology marketplace. To offer an equitable process for all participants, Forrester follows The Forrester Wave™ Methodology to evaluate participating vendors.

In our review, we conduct primary research to develop a list of vendors to consider for the evaluation. From that initial pool of vendors, we narrow our final list based on the inclusion criteria. We then gather details of product and strategy through a detailed questionnaire, demos/briefings, and customer reference surveys/interviews. We use those inputs, along with the analyst’s experience and expertise in the marketplace, to score vendors, using a relative rating system that compares each vendor against the others in the evaluation.

We include the Forrester Wave publishing date (quarter and year) clearly in the title of each Forrester Wave report. We evaluated the vendors participating in this Forrester Wave using materials they provided to us by September 6, 2023 and did not allow additional information after that point. We encourage readers to evaluate how the market and vendor offerings change over time.

In accordance with our vendor review policy, Forrester asks vendors to review our findings prior to publishing to check for accuracy. Vendors marked as nonparticipating vendors in the Forrester Wave graphic met our defined inclusion criteria but declined to participate in or contributed only partially to the evaluation. We score these vendors in accordance with our vendor participation policy and publish their positioning along with those of the participating vendors.

Integrity Policy

We conduct all our research, including Forrester Wave evaluations, in accordance with the integrity policy posted on our website.