OpenAI, a leading artificial intelligence research lab, recently held a press briefing to announce its successful detection and prevention of five covert influence operations within the past three months. These operations, originating from China, Russia, Iran, and Israel, aimed to manipulate public opinion and shape political outcomes while concealing their true identities. OpenAI’s use of AI products allowed them to expose these operations and shed light on the potential impact of AI on upcoming elections.
One of the key findings from OpenAI’s report is that these influence networks leveraged AI tools to generate large volumes of text and images with fewer errors than human-generated content. This tactic was employed to deceive the public more effectively. Ben Nimmo, principal investigator on OpenAI’s Intelligence and Investigations team, emphasized the significance of these findings, stating, “Over the last year and a half there have been a lot of questions around what might happen if influence operations use generative AI. With this report, we really want to start filling in some of the blanks.”
OpenAI categorizes these operations as covert “influence operations,” which differ from disinformation networks that disseminate factually correct information in a deceptive manner. Unlike traditional propaganda networks that rely on social media platforms, these covert operations utilized generative AI tools, marking a novel development in the field. Alongside AI-generated content, these operations also employed more conventional methods such as manually written texts or memes on major social media sites.
The identified operations included groups like “Doppelganger,” a pro-Russian network, “Spamouflage,” a pro-Chinese network, and the International Union of Virtual Media (IUVM), an Iranian operation. OpenAI also flagged previously unknown networks from Russia and Israel. For instance, OpenAI discovered a new Russian group called “Bad Grammar” that utilized AI models and the messaging app Telegram to establish a content-spamming pipeline. This operation involved debugging code to automate posting on Telegram and generating comments across numerous accounts.
Despite the detection of these influence operations, OpenAI reported that they generally failed to gain significant traction. However, Ben Nimmo cautioned against complacency, as history has shown that such operations can unexpectedly gain momentum if undetected. He acknowledged the possibility of other undetected groups utilizing AI tools, stating, “I don’t know how many operations there are still out there. But I know that there are a lot of people looking for them, including our team.”
OpenAI’s ChatGPT is actively sharing threat indicators with industry peers and plans to release more reports in the future to aid in the detection and defense against such influence operations. This collaborative effort between tech companies is crucial in addressing the challenges posed by AI-driven manipulation in the digital age. Meta Platforms Inc., among other major tech companies, has also disclosed similar activities by influence operations, highlighting the ongoing commitment to safeguarding against the misuse of AI technologies.
In conclusion, OpenAI’s identification and thwarting of covert influence operations using AI tools underscore the need for continued vigilance in protecting democratic processes and public opinion. These findings shed light on the potential impact of AI on elections and emphasize the importance of organizations like OpenAI and ChatGPT in detecting and defending against such manipulative tactics. By sharing threat indicators and collaborating with industry peers, these efforts aim to mitigate the risks associated with AI-driven manipulation in today’s digital landscape.
