In today’s fast-paced world, businesses are increasingly turning to artificial intelligence (AI) models to improve their operations. However, deploying AI models comes with its own set of challenges. One of the most crucial questions that businesses face is when their chosen model is safe to deploy. Testing the model thoroughly on the backend becomes essential to avoid embarrassing mishaps and protect the company’s reputation.
Enter Kolena, a San Francisco-based startup co-founded by a former Amazon senior engineering manager. Kolena aims to address the challenges of testing and validating AI systems with its newly released AI Quality Platform. This web application enables rapid and accurate testing of AI models, ensuring their reliability and safety.
The AI Quality Platform offers a range of features to support software developers and IT personnel in building robust AI systems for real-world use cases. It goes beyond traditional statistical metrics by facilitating the development of detailed test cases from datasets. This allows for a closer scrutiny of AI/ML models in scenarios they will face in the real world.
To use the platform, customers connect their chosen model to Kolena’s API and provide their own dataset along with specific functional requirements for how they want the model to operate. Whether it’s manipulating text, imagery, code, audio, or other content, Kolena’s platform ensures that the model performs as expected.
One unique aspect of Kolena’s platform is its ability to measure attributes such as bias and diversity across various demographics. By running tests simulating hundreds or thousands of interactions, Kolena determines if the model produces undesirable results and under what circumstances. It also re-tests models after updates, training, retraining, fine-tuning, or changes by the provider or customer.
Mohamed Elgendy, Kolena’s co-founder and CEO, emphasizes that the platform takes the guessing out of AI deployment and turns it into a true engineering discipline. It provides valuable insights into where a model may have degraded, enabling businesses to make necessary adjustments and ensure their AI systems perform optimally.
The significance of testing AI systems extends beyond enterprises themselves. Even AI model providers can benefit from Kolena’s platform. For instance, Google’s Gemini, which faced controversy for generating racially confused and inaccurate imagery, could have potentially avoided such issues with thorough testing using Kolena’s AI Quality Platform.
Kolena has spent the last two years refining its platform through a closed beta testing phase. During this time, the company collaborated with Fortune 500 companies, startups, government agencies, and AI standardization institutes to gather feedback and improve the platform. The closed beta customers have already run tens of thousands of tests on AI models using Kolena’s platform.
Looking ahead, Kolena aims to serve customers across three categories: builders of AI foundation models, buyers in tech, and buyers outside of tech. The platform’s pricing follows a software-as-a-service model, with three tiers designed to accommodate a company’s growth with AI.
In conclusion, Kolena’s AI Quality Platform offers a comprehensive solution for testing and validating AI models. By providing detailed test cases, measuring attributes like bias and diversity, and re-testing models after updates, Kolena ensures that businesses can deploy AI systems confidently. With its successful closed beta testing and partnerships with industry leaders, Kolena is poised to revolutionize the way AI models are tested and deployed.
