NIST, the U.S. Commerce Department agency responsible for developing and testing technology, has re-released a tool called Dioptra, designed to measure the impact of malicious attacks on AI systems. Dioptra is an open-source, web-based tool that helps companies and users assess, analyze, and track AI risks. It can be used to benchmark and research AI models, as well as simulate threats in a “red-teaming” environment.
One of the main goals of Dioptra is to test the effects of adversarial attacks on machine learning models. The open-source software is available for free download and can help government agencies and small to medium-sized businesses evaluate the performance of AI systems. Dioptra was launched alongside documents from NIST and the AI Safety Institute, which provide guidance on mitigating the dangers of AI.
Dioptra is part of President Joe Biden’s executive order on AI, which mandates NIST to assist with AI system testing. The order also establishes standards for AI safety and security, requiring companies to notify the federal government and share safety test results before deploying AI models to the public.
However, evaluating AI models is challenging due to the lack of transparency in the most sophisticated models. Companies keep details about infrastructure, training data, and other key aspects of AI models hidden. The Ada Lovelace Institute found that current evaluation methods alone are insufficient to determine the real-world safety of AI models.
While Dioptra cannot completely eliminate risks associated with AI models, it can provide insights into the impact of different types of attacks on performance. It helps quantify the effects of attacks on an AI system’s effectiveness.
One limitation of Dioptra is that it only works with models that can be downloaded and used locally. Models that are accessed through an API, like OpenAI’s GPT-4o, are not compatible with Dioptra at this time.
