In today’s rapidly evolving digital landscape, cybersecurity threats continue to loom large over organizations. One of the most devastating types of attacks is the insider threat, which targets a company’s most strategically important systems and assets. As enterprises embrace artificial intelligence (AI) and deploy internal AI chatbots, they inadvertently create new attack vectors and risks. A recent study has shed light on a nightmare scenario that security teams fear – internal AI chatbots compromised by ASCII art hacking.
Researchers have shown that ASCII art can be used to jailbreak state-of-the-art large language models (LLMs) such as Open AI’s ChatGPT-3.5, GPT-4, Gemini, Claude, and Meta’s Llama2. The attack strategy, known as ArtPrompt, takes advantage of the poor performance of LLMs in recognizing ASCII art to bypass security measures. ArtPrompt requires only black-box access to targeted LLMs and fewer iterations to successfully jailbreak them.
While LLMs excel at semantic interpretation, their ability to interpret complex spatial and visual recognition differences is limited. This is where jailbreak attacks using ASCII art succeed. Researchers conducted a comprehensive benchmark called Vision-in-Text Challenge (VITC) to measure the ASCII art recognition capabilities of different LLMs. The benchmark included two unique data sets – VITC-S and VITC-L – which presented single characters and sequences of characters represented in ASCII art.
ArtPrompt is a two-step attack strategy that involves masking safety words using ASCII text and then replacing them with ASCII art. Researchers found that ASCII text is highly effective at cloaking safety words across different LLMs. This discovery raises concerns about the security of internal AI chatbots and the potential for unauthorized access to critical systems and sensitive information.
The growth of internal AI chatbots has been driven by organizations’ desire for increased productivity, cost savings, and revenue gains. The top-performing enterprises have already deployed generative AI applications at scale, with 44% of them realizing significant value from scaled predictive AI cases. Boston Consulting Group (BCG) found that approximately 50% of enterprises are developing focused Minimum Viable Products (MVPs) to test the value they can gain from generative AI.
However, the rapid adoption of internal chatbots as an attack surface has raised concerns about the security of these systems. The average cost to remediate an attack is a staggering $7.2 million, with the average cost per incident ranging between $679,621 and $701,500. Negligence is the leading cause of insider incidents, accounting for 55% of internal security incidents. The need to design internal chatbots that can recover from negligence and user errors is just as important as hardening them against attacks.
Defending against attacks on LLMs using ASCII art will require an iterative approach. Researchers emphasize the need for multimodal defense strategies that include expression-based filtering support by machine learning models designed to recognize ASCII art. Continuous monitoring and the use of perplexity-based detection, paraphrase and retokenization techniques can also help in mitigating the risks posed by ASCII art attacks.
The cybersecurity industry is evolving its response to threats posed by AI chatbots. Vendors like Cisco, Ericom Security by Cradlepoint’s Generative AI isolation, Menlo Security, Nightfall AI, Wiz, and Zscaler offer solutions to keep confidential data out of chatbot sessions. These solutions need to be extended to trap ASCII text before it is submitted to prevent unauthorized access and compromise of internal AI chatbots.
Zscaler recommends five steps to integrate and secure generative AI tools and apps across an enterprise: defining a minimum set of gen AI and machine learning (ML) applications, selectively approving internal chatbots and apps, creating a private ChatGPT server instance, implementing single sign-on (SSO) with strong multifactor authentication (MFA), and enforcing data loss prevention (DLP) to prevent data leakages.
The complexity of ASCII art and the potential for false positives and negatives necessitate the hardening of chatbots and their supporting LLMs against spatial and visual recognition-based attacks. Multimodal defense strategies are crucial in containing this evolving threat. As organizations continue to embrace AI and deploy internal chatbots, it is imperative to prioritize robust cybersecurity measures to protect critical systems and sensitive information from insider threats.
