Generative AI models have a tendency to produce hallucinations, which can pose significant challenges for businesses implementing the technology. These hallucinations occur because the models lack real intelligence and simply predict data based on a private schema. The inaccuracies can lead to false information being generated, as seen in a case where Microsoft’s generative AI invented meeting attendees and implied that conference calls discussed topics that were never actually mentioned.
To address this issue, some generative AI vendors propose a solution called retrieval augmented generation (RAG). Companies like Squirro and SiftHub offer RAG technology, which promises to eliminate hallucinations by ensuring that every generated piece of information can be traced back to a credible source. By incorporating RAG into their systems, businesses can achieve increased transparency, reduced risk, and inspire trust in using AI for their various needs.
RAG was developed by data scientist Patrick Lewis, who coined the term in a 2020 paper. The approach involves retrieving relevant documents using keyword searches and then asking the model to generate answers based on this additional context. By attributing generated content to retrieved documents, RAG allows for fact-checking and prevents copyright infringement. It also enables enterprises in regulated industries like healthcare and law to securely and temporarily utilize their documents without training the model on them.
However, RAG is not a foolproof solution. It cannot completely eliminate hallucinations, and it has certain limitations that many vendors tend to overlook. According to research scientist David Wadden from the Allen Institute’s AI-focused research division, RAG is most effective in scenarios where users have knowledge-intensive information needs. In such cases, keyword-based search queries can easily find documents containing similar keywords to answer the question. However, reasoning-intensive tasks like coding and math pose challenges because it is difficult to specify concepts in a keyword search query or identify relevant documents.
Models can also get distracted by irrelevant content in long documents or ignore retrieved documents altogether, relying solely on their parametric memory. Moreover, implementing RAG at scale requires significant hardware resources and compute power, making it an expensive solution.
Although RAG has its limitations, ongoing efforts are being made to improve its effectiveness. Researchers are exploring models that can decide when to utilize retrieved documents or even choose not to perform retrieval if deemed unnecessary. They are also working on more efficient indexing of massive datasets and better document representations that go beyond keywords, aiming to identify relevant documents for more abstract generation tasks.
In conclusion, while RAG can help reduce hallucinations in generative AI models, it is not a comprehensive solution to all the challenges posed by AI’s hallucinatory problems. Businesses should be cautious of vendors who claim otherwise. Further research and advancements are needed to address the limitations of RAG and improve the overall accuracy and reliability of generative AI models.