Combating food health misinformation and disinformation with advanced RAG-LLM

Abstract

Effective prevention and treatment of cardiovascular disease rely on healthy lifestyle behaviors. However, the expanding and often conflicting nature of scientific literature, along with prevalent food and health misinformation/disinformation, hinders informed decision-making. Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) capabilities offer automated claim verification and summarization. An enhanced RAG-LLM prototype with novel modules is evaluated for effectiveness. Results show that Inclusion-Criteria-based filtering of PubMed articles improved verdict accuracy. Furthermore, employing a PICO-based (Population, Intervention, Comparison, Outcome) approach for summarizing evidence in health claims (e.g., 'Berries reduce blood pressure') increased transparency. Still, the tested RAG-LLM models exhibited biases towards positivity (overemphasizing heart-healthy aspects) and neutrality. Likely causes of these biases and future development challenges are discussed, as well as AI tooling enabling governance on industry influences in nutrition science.