Skip to main content
NUTRITION 2024 Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Education and Teaching (Poster Session)

(P23-011-24) Enhancing Patient Education on the Heart-Healthy Diet With Generative Artificial Intelligence Models

Monday, July 1, 2024
12:45 PM – 1:45 PM CT
Location: Poster Board 69
Disclosure(s):
Som P. Singh, MD: No relevant financial relationship(s) with ineligible companies to disclose.


Objectives: Profile the readability of heart-healthy diet patient education created by generative artificial intelligence (AI) models.

Methods: This study extracted the 20 most frequently asked questions regarding the heart-healthy diet using the Google ® RankBrain algorithm. Questions were extracted and categorized as “Fact”, “Policy”, or “Value” questions using Rothwell’s Classification of Questions. Copilot (Microsoft ®) and Generative-Pre-Trained Transformer-3.5 (ChatGPT) (OpenAI ®) were employed as generative AI models to respond to each question. For the control, the associated educational articles provided with each question were employed. Readability was scored by a healthcare provider (e.g., Flesch Reading Ease, Flesch-Kincaid) for the educational articles and responses by the generative AI models.

Results: The mean Flesch Reading Ease score was highest in the control (60.2 ± 8.7) followed by the Copilot (50.3 ± 8.8) and ChatGPT models (41.8 ± 10.6). One-way analysis of variance (ANOVA) demonstrated a statistically significant difference in Flesch Reading Ease between groups (p= 0.004). The mean Flesch-Kincaid score was lowest in the control (7.9 ± 1.7) followed by Copilot (8.8 ± 1.4) and ChatGPT (10.8 ± 1.9) models and demonstrated statistical significance using ANOVA (p= 0.019). Among Rothwell’s Classification of Questions, a majority of questions were classified as Fact-based (n= 13) followed by Value-based (n= 6) and Policy-based (n= 1).

Conclusions: This study demonstrated there is a statistically significant difference in the readability quality of generative AI models and publicly available educational articles on the heart-healthy diet. ChatGPT performed inferior in both readability metrics. Moreover, the mean Flesch-Kincaid of the ChatGPT model would not meet U.S. grade-reading level recommendations. This could suggest greater difficulty for patients in understanding heart-healthy diet education.

Funding Sources: No funding was used in this study.