Google AI Overview Controversy: Why Dangerous and Incorrect Information is Rising to the Top

Google’s AI Overview feature, previously known as Search Generative Experience, has sparked controversy since becoming the default experience on Google Search for users in the U.S. This feature, which was introduced to compete with OpenAI’s ChatGPT, provides users with an AI-generated snapshot of key information and links to dig deeper into their search queries. However, many users have been sharing examples of wildly incorrect, inflammatory, and even dangerous information that is being displayed in the AI Overview results.

Google has labeled the AI Overview feature as “experimental” and claims to evaluate and improve the quality of its results and products responsibly through user feedback and human reviews. Despite this, some users have called for the removal of the feature, comparing it to Google’s decision to temporarily remove its Gemini AI image generation feature earlier this year due to racially and historically inaccurate images.

Critics argue that Google could be held legally liable for the dangerous results produced by its AI models. Additionally, it has been observed that Google is using its AI models to create summaries of indexed content from its Search index, even if it did not originate the content. This reliance on external sources may explain why the AI Overview feature is more prone to displaying erroneous information from untrustworthy sources or jokes that are treated as serious by the AI models responsible for summarization.

While it is unclear what percentage of searches display this erroneous information, it is crucial for users to be cautious and check alternative sources for accurate information. Rival AI search startup Perplexity has been noted for surfacing correct information more consistently than Google’s AI Overviews. Ultimately, the responsibility lies with users to verify the information provided and make informed decisions to protect their health and safety.