Posted in

My Confluent Chapter: From Apache Kafka Startup to $11 Billion IBM Acquisition

My Confluent Chapter: From Apache Kafka Startup to  Billion IBM Acquisition

Nine years at Confluent: mission accomplished. In May 2017, I received an email from Jay Kreps. He introduced himself as the CEO of Confluent and one of the co-creators of Apache Kafka, and asked whether I might want to discuss this emerging technology. A few weeks later I signed my contract and flew to Silicon Valley to meet a team of roughly 100 people with a bold ambition: to make real-time data streams a foundational element of modern event-driven enterprise architecture.

Confluent is now part of IBM. I will not be following.

This is not a sad post. It is a reflection on one of the most rewarding chapters of my career, on what we built together, and on why this is the right moment to move on.

From Apache Kafka Startup to $11 Billion IBM Acquisition

When I joined Confluent, the company had around 100 employees, was backed by venture capital, and operated out of a Silicon Valley office where the Kafka tree stood at the HQ entrance. Apache Kafka was used by a small number of tech companies. Most enterprises had never heard of it. Those who had typically thought of it as a fast message broker. Nothing more.

The broader picture was almost entirely unknown in enterprise companies at the time. Kafka Connect for data integration, Kafka Streams for stream processing, the persistent event log as the foundation for true decoupling of applications, event replayability, and the architectural backbone of independent microservices and data mesh principles: none of this had found its way into mainstream enterprise thinking yet.

Apache Kafka is now used by more than 150,000 organizations across every industry and geography. It became the de facto standard for data streaming a complete paradigm. Most large enterprises today run Kafka at the heart of their enterprise architecture. It connects operational and analytical systems, enables event-driven applications, and handles both mission-critical transactional workloads and large-scale analytical pipelines in real time.

In 2017, the question was whether Kafka could become more than an ingestion layer for Hadoop. By the mid-2020s, most companies had moved well past that question. Kafka had become infrastructure. The new question is how fast real-time data streaming becomes the foundation for agentic AI, and Confluent invested heavily in positioning the platform for exactly that transition.

Confluent went public on the NASDAQ in June 2021. In December 2025, IBM announced its acquisition at an enterprise value of $11 billion. The deal closed in early 2026. By that point, Confluent had crossed $1.12 billion in revenue, with Confluent Cloud alone contributing $624 million and around 1,500 customers spending $100,000 or more per year.

How Confluent Built the Leading Data Streaming Platform

Confluent started as the company making Apache Kafka enterprise-ready, often called “the Kafka company”. By the time of the IBM acquisition, it had become the provider of a full data streaming platform.

The core of that evolution was Kafka itself. It grew from a distributed log used mainly for data ingestion into a complete infrastructure layer: event streaming, data integration via Kafka Connect using hundreds of connectors, stream processing via Kafka Streams, exactly-once semantics for transactions, tiered storage for separation of compute and storage, diskless Kafka as the next step in decoupling compute from storage entirely, native queue support bringing new consumption models to the platform via Queues for Kafka (QfK).

Beyond Kafka and providing enterprise-grade fully managed cloud services across AWS, Azure, and GCP, Confluent made strategic bets on complementary open source projects. 

Apache Flink became the platform’s stream processing layer, adding stateful computation at scale. Apache Iceberg added an open table format for connecting streaming data directly with lakehouse platforms and AI data pipelines, making Kafka a first-class citizen in the broader analytics and AI ecosystem. 

The pattern across all of these was consistent: identify the open source project defining a layer of the stack, invest in it seriously, and bring it into a managed platform.

A Platform That Runs Everywhere in a Software Category of Its Own

One of the most underappreciated aspects of Confluent’s evolution was its deployment flexibility. The platform runs everywhere: as a fully serverless cloud service, as a Bring Your Own Cloud (BYOC) deployment via WarpStream inside the customer’s own VPC, as a self-managed installation in any data center, and even at the edge in environments like factory floors or retail stores. Few enterprise software companies have solved the hybrid multi-cloud deployment problem across all of those environments. Confluent did.

Research firms eventually recognized what practitioners had known for years. Forrester published its Streaming Data Platforms Wave in late 2023, naming Confluent a leader alongside Microsoft and Google. IDC followed. Data streaming with an event-driven architecture is now an established, recognized software category. For years, analysts and vendors tried to fit Kafka into existing boxes: as a message queue, as an ETL tool, as part of Integration Platform as a Service (iPaaS). That category confusion is finally over. Data streaming stands on its own.

The Field CTO Role: Bridging Executive Leadership and Technology

The Field CTO role does not have a standard definition. Every company that has one defines it differently. I wrote about what the role looked like day to day in a dedicated post on the daily life of a Field CTO.

At Confluent, it sat at the intersection of technology and business, between the product organization and the market, between deep technical knowledge and executive conversation. It was not a pure sales role, not a pure marketing role, and not a pure engineering role. It drew from all three, and the value came from that combination.

In practice, it meant engaging with more than 100 customers, partners, and analysts every year across every major industry and geography. Financial services, manufacturing, retail, telco, healthcare, public sector, logistics: each with its own architecture challenges, regulatory constraints, and data streaming maturity.

The most important distinction in that work was the level of conversation. Presales colleagues focus on product capabilities, features, and implementation. A Field CTO focuses on business value, industry trends, and strategic architecture. 

That means walking into a room with a CTO at one of the world’s most respected luxury automotive companies, a Chief Architect at a large international airline, or a Chief Digital Officer at a leading multinational technology conglomerate, and leading a conversation about why an event-driven architecture is a strategic priority for their business. It means running executive roundtables, presenting at board-level briefings, and helping leadership teams understand how their peers across the industry are solving the same problems.

Part of the job is also telling customers when not to use the product. That credibility is what makes executive conversations work. Executives have no patience for vendors who position their product as the answer to every question. A Field CTO is a trusted advisor first, a product advocate second.

Thought Leadership at Scale

Beyond customer work, the role meant presenting at international conferences, writing public articles that reached thousands of readers per month, publishing landscape analyses and industry books, briefing research analysts at Gartner and Forrester, working with press and journalists, and enabling internal sales and solution engineering teams with the narratives they needed for their own executive conversations.

The many years of engagements, conversations and mission-critical architectures I saw led to a whole book: “The Ultimate Data Streaming Guide“, including several industry editions and a lot of real-life examples from customers such as BMW, Lufthansa, and Siemens. It is still available for free download.

The Ultimate Data Streaming Guide - Book Industry Editions Manufacturing Automotive Financial Services Telecom Media Digital Natives

The core stayed constant across nine years: credibility through content quality, trust through transparency, and impact through education at scale.

Why IBM Changes the Equation

IBM acquiring Confluent makes strategic sense. IBM brings global enterprise reach, deep relationships with the largest organizations in the world, and decades of experience selling into complex, multi-year enterprise deals. 

Confluent brings the leading data streaming platform built around Apache Kafka and Flink, filling a gap in IBM’s portfolio. For Confluent as a product, the acquisition opens doors that an independent company of its size would struggle to reach alone.

IBM’s challenge — and opportunity — is to integrate Confluent with the rest of its product portfolio into a coherent enterprise platform story. The conversation shifts from building a single category to selling a broad portfolio. The culture, the incentives, and the priorities of a nearly $70 billion company are different from those of a focused, founder-led data streaming company. None of that is a criticism. It is the reality of what an acquisition of this scale means in practice.

Confluent is now part of IBM. I will not be making the journey with it. After more than nine years, it is the right time for new challenges and different conversations.

Thank You, Confluent!

Nine years is a long time in any industry. In enterprise software, this is equivalent to several technology generations.

I want to thank Jay Kreps and the founding team for building something worth building, and for the invitation to be part of it from the beginning. I want to thank the colleagues across sales, marketing, product, engineering, and the field, many of whom became friends. And I want to thank the customers, partners, and community members who engaged with the content, challenged the ideas, and helped sharpen the thinking year after year.

What comes next deserves its own post. Stay tuned.

For new landscapes, deep dives, and use cases across data integration, process intelligence, enterprise architecture, and trusted agentic AI, subscribe to my newsletter and follow me on LinkedIn or X.

Leave a Reply

Your email address will not be published. Required fields are marked *