The rise of artificial intelligence has transformed the way we access information, but it also poses significant risks, particularly when it comes to sensitive topics like health. Google's AI Overviews, which appear above search results, are designed to provide users with quick and easy access to information, but they often fail to include vital safety warnings, putting people at risk of harm.
According to Google, its AI Overviews are intended to inform users when it's essential to seek expert advice or verify the information presented. However, an investigation by The Guardian found that the company does not include any such disclaimers when users are first presented with medical advice. Instead, safety labels only appear below additional medical advice assembled using generative AI, and in a smaller, lighter font.
This lack of transparency has raised concerns among AI experts and patient advocates, who argue that disclaimers serve a vital purpose and should appear prominently when users are first provided with medical advice. Pat Pataranutaporn, an assistant professor at the Massachusetts Institute of Technology (MIT), notes that the absence of disclaimers creates several critical dangers, including the risk of misinformation and the prioritization of user satisfaction over accuracy.
Academic Perspectives
Gina Neff, a professor of responsible AI at Queen Mary University of London, believes that the problem with bad AI Overviews is by design. "AI Overviews are designed for speed, not accuracy, and that leads to mistakes in health information, which can be dangerous," she says. Neff argues that Google's investigation into the issue is essential, and that prominent disclaimers are necessary to prevent harm.
Sonali Sharma, a researcher at Stanford University's centre for AI in medicine and imaging (AIMI), agrees that the major issue is that Google AI Overviews appear at the top of the search page, often providing what feels like a complete answer to a user's question. This can create a sense of reassurance that discourages further searching or scrolling through the full summary, where a disclaimer might appear.
Personal Accounts and Expert Analysis
Tom Bishop, head of patient information at Anthony Nolan, a blood cancer charity, calls for urgent action to address the issue. "We know misinformation is a real problem, but when it comes to health misinformation, it's potentially really dangerous," he says. Bishop argues that the disclaimer needs to be more prominent, ideally at the top of the page and in the same size font as the rest of the information.
Google has responded to the criticism, stating that AI Overviews encourage people to seek professional medical advice and frequently mention seeking medical attention directly within the overview itself. However, the company has not denied that its disclaimers fail to appear when users are first served medical advice or that they appear below AI Overviews and in a smaller, lighter font.
Labour Market Data and Broader Economic Framing
The issue of AI-generated medical advice is not just a concern for individuals, but also has broader economic implications. As the use of AI in healthcare continues to grow, it's essential to ensure that the information provided is accurate and reliable. The lack of transparency and accountability in Google's AI Overviews poses a significant risk to public health, and it's crucial that the company takes urgent action to address the issue.
In January, a Guardian investigation revealed that people were being put at risk of harm by false and misleading health information in Google AI Overviews. Following the investigation, Google removed AI Overviews for some but not all medical searches. However, the company needs to do more to ensure that its AI Overviews are safe and reliable.
Conclusion and Future Directions
The hidden dangers of Google's AI-generated medical advice are a stark reminder of the need for transparency and accountability in the use of AI in healthcare. As the technology continues to evolve, it's essential that companies like Google prioritize the safety and well-being of their users. By providing prominent disclaimers and ensuring that AI Overviews are accurate and reliable, Google can help prevent harm and promote a safer and more informed online community.
