In a conversation with Shaun Clowes, the CPO at Confluent, he delved further into these offerings and the company’s approach to modern AI. He highlighted the shift from relying on batch data for analytical workloads to utilizing real-time data. Apache Kafka, an open-source technology for streaming data feeds, has played a crucial role in enabling this shift. Confluent, led by Kreps, has built commercial products and services around Kafka, including the recent acquisition of Immerok, a leading contributor to the Apache Flink project.
At the Kafka Summit, Confluent launched AI model inference in its cloud-native offering for Apache Flink, simplifying real-time AI and machine learning applications. Previously, teams using Flink had to code and use multiple tools to call AI with streaming data. With AI model inference, Confluent has made this process pluggable and composable, allowing users to make calls to AI engines using simple SQL statements from within the platform.
The plug-and-play approach gives users flexibility in choosing the AI model that best suits their needs. Additionally, the performance of these models evolves over time, allowing users to switch models without changing the underlying data pipeline. Clowes explained that this approach involves two Flink jobs: one generates embeddings from customer data, while the other handles inference requests from customers.
Currently, access to AI model inference is limited to select customers building real-time AI apps with Flink, but Confluent plans to expand access in the coming months. The company also intends to launch more features to make it easier, cheaper, and faster to run AI apps with streaming data. These improvements will include a gen AI assistant to help users with coding and other tasks in their workflows.
In addition to simplifying AI efforts, Confluent introduced Freight Clusters, a new serverless cluster type that offers cost savings. These auto-scaling clusters leverage slower but cheaper replication across data centers, resulting in up to a 90% reduction in cost. While there is some latency, it is still fast enough for many use cases.
Looking ahead, Confluent aims to grow its presence in the APAC region, particularly in India where it plans to increase headcount by 25%. On the product side, the company is investing in capabilities for improving data governance and self-service of data. Clowes emphasized the importance of these elements, which are still immature in the streaming world compared to the data lake world.
Overall, Confluent’s efforts to simplify AI with real-time data and reduce costs through Freight Clusters demonstrate their commitment to empowering businesses with advanced data solutions. As the industry continues to evolve, Confluent believes that more progress will be made in governance and data products for the streaming domain.
