OpenAI, a leading artificial intelligence company, has recently announced its acquisition of Rockset, a vendor known for its real-time analytics database. The deal aims to strengthen OpenAI’s technology stack and enhance its retrieval infrastructure across products. While the terms of the acquisition remain undisclosed, OpenAI has confirmed that Rockset’s entire team will join the company as part of the deal.
This acquisition marks OpenAI’s second major acquisition, following its acquisition of New York City startup Global Illumination, Inc. last year. It comes at a time when the competition in the gen AI domain is intensifying, with one of OpenAI’s biggest competitors, Anthropic, releasing a powerful LLM called Claude 3.5 Sonnet that outperforms OpenAI’s GPT-4o on benchmarks.
Rockset, founded in 2016, offers a cloud-based real-time analytics database that enables developers to build data-intensive applications at scale. The product allows continuous ingestion and indexing of data from various sources, such as Kafka, MongoDB, DynamoDB, and S3, enabling real-time information availability and querying. It can run sub-second SQL queries on semi-structured data without requiring a predefined schema.
The core technology behind Rockset is the open-source RocksDB persistent key-value store created at Meta (formerly Facebook), which acts as an external secondary index for databases, data lakes, and streaming platforms. Throughout 2023, Rockset improved its product with capabilities aimed at powering AI use cases.
With the integration of Rockset’s technology into OpenAI’s products, the retrieval stack of OpenAI will be powered by Rockset’s “world-class” indexing and querying capabilities. This integration will enable OpenAI products to provide customers with the freshest and most relevant information to answer their questions quickly and accurately.
Enterprises and startups using AI tools like OpenAI’s models have been exploring the use of “retrieval augmented generation,” a technique that combines generative AI with external knowledge bases. This technique aims to improve the model’s ability to handle specific queries and reduce the risk of incorrect answers. By leveraging Rockset’s infrastructure, OpenAI can further enhance the capabilities of its models to address user questions with accurate and up-to-date data.
The exact specifics of how this integration will take shape are yet to be shared by the companies. However, Brad Lightcap, COO at OpenAI, expressed excitement about bringing Rockset’s benefits to customers and integrating its foundation into OpenAI products.
Regarding Rockset’s existing customers, it remains unclear how the acquisition will impact their business. Rockset has worked with notable players in the industry, including Klarna, Meta, Whatnot, and Windward. Questions about the future of Rockset’s business sent by VentureBeat remained unanswered at the time of writing.
In conclusion, OpenAI’s acquisition of Rockset signifies a significant move to strengthen its technology stack and improve its retrieval infrastructure. By integrating Rockset’s indexing and querying capabilities, OpenAI aims to provide customers with faster and more accurate answers to their queries. This acquisition aligns with the trend of leveraging external knowledge bases to enhance AI models’ performance and showcases OpenAI’s commitment to staying competitive in the evolving gen AI domain.
