Databricks, led by Ali Ghodsi, has announced new enhancements for its Mosaic AI platform at its annual data and AI conference. These enhancements aim to assist enterprises in deploying large language model (LLM)-powered applications. While Databricks has been providing AI application-building tools for enterprises, the Mosaic AI platform, originating from the company’s acquisition of MosaicML, has greatly accelerated their efforts in the gen AI space.
The latest capabilities of the Mosaic AI platform focus on three key areas: the development of compound AI systems, their evaluation across different metrics, and the governance of the entire pipeline. This comprehensive approach allows enterprises to build reliable gen AI apps from their data.
This move by Databricks not only strengthens their offering but also positions them against Snowflake, which has been moving in a similar direction under the leadership of CEO Sridhar Ramaswamy.
Challenges in Building High-Quality Gen AI Apps
Organizations that are enthusiastic about generative AI are racing to leverage this novel technology by creating applications that utilize their internal data assets with powerful AI models. However, many teams face difficulties in achieving the desired return on investment from large models. These apps often fail to provide high-quality outputs while staying within expected budgets and privacy guardrails.
To address this challenge, organizations have shifted their focus towards building retrieval augmented generation (RAG)-based compound AI systems. These systems leverage multiple components such as small models, retrievers, vector databases, and evaluation, monitoring, security, and governance tools. Databricks has been continuously upgrading the Mosaic AI platform to enable the creation of these systems.
New Features of Databricks Mosaic AI
Databricks has recently announced two new features for the Mosaic AI platform: Mosaic AI Model Training and Agent Framework. The former allows users to fine-tune small, open-source foundation models using the Databricks API or UI. This enables these models to acquire new knowledge and handle specific domains or tasks in a cost-efficient manner.
The latter, integrated with Mosaic AI Vector Search and Model Serving, powers high-quality RAG apps using the fine-tuned models. The Agent Framework includes built-in proprietary AI-assisted evaluation, which can automatically determine the quality of app outputs. It also provides an intuitive tracing UI for feedback from human stakeholders, allowing for rapid iteration on changes. This end-to-end LLMOps workflow enables developers to test every hypothesis and redeploy their application into production.
The Mosaic AI platform also features an AI Tools Catalog that allows organizations to govern, share, and register tools using Databricks Unity Catalog, which is now open source. These tools enhance compound AI systems by equipping them with new capabilities, such as intelligently generating and executing code, searching the web, and calling APIs. Any Python or SQL function registered in the Unity Catalog is supported by the Mosaic AI Tools Catalog, increasing the quality of the final response.
Ensuring Governance and Trust with Mosaic AI Gateway
To further strengthen trust in developed AI apps, Databricks is introducing the Mosaic AI Gateway. This offering provides teams with a unified interface to query, manage, and deploy open-source or proprietary models. It allows for easy switching of LLMs without making complicated changes to the application code.
Crucially, the AI Gateway comes with built-in governance and monitoring capabilities. It supports usage tracking and guardrails, enabling organizations to track who is calling the model and set up rate limits for spending control. Additionally, it provides filters for safety and personally identifiable information, ensuring data privacy and security.
The new offerings of the Mosaic AI platform, except for the AI Tools Catalog, are currently in public preview and expected to become generally available in the coming months. The tools catalog is currently in private preview. Databricks has also announced other notable products at the event, including Databricks AI/BI for gen AI-powered analytics, Databricks LakeFlow for data engineering, and an enterprise-centric image generation model developed in partnership with Shutterstock.
Databricks Data and AI Summit is taking place from June 10 to June 13, 2024.
