Web Search or Conversation with an Artificial Intelligence? Analysis of Misinformation and Relevance in the Case of Radon Gas

Abstract

Health-related information plays a crucial role in public health management, empowering individuals to make informed
decisions and adopt behaviours that mitigate the effects of potential risks. The internet and the emergence of new
technologies, such as conversational models equipped with Artificial Intelligence, present opportunities, and challenges
in this field. This research focuses specifically on the risk of radon, a natural radioactive gas recognised worldwide as
one of the leading causes of lung cancer and a persistent threat over time. The aim of this study is to analyse the
information provided for this specific risk by two key information access tools: web search engines and AI-based
conversational agents (ChatGPT). To carry out this interdisciplinary research (journalism-communication-computer
science) we employ a mixed methodological design (quantitative and qualitative) and apply methods from the areas of
Information Retrieval (IR), Big Data, and Artificial Intelligence (AI). The results of this study show that information on
the internet about the risk of radon often lacks relevance and does not meet the information needs of users. We also
found that some websites provide a significant amount of good quality information but there are often some misleading
contents. ChatGPT proves to be more accurate in providing relevant and good quality information but contains a higher
proportion of misinformation. Consequently, this raises concerns about the integrity of the information provided and
emphasises the need to monitor and improve the accuracy of these computational tools.