Quality, readability, and patient safety of ChatGPT-generated responses to fall-related questions in older adults: a multidisciplinary evaluation
Revista da Associacao Medica Brasileira, vol.72, no.8, 2026 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Volume: 72 Issue: 8
- Publication Date: 2026
- Doi Number: 10.1590/1806-9282.20260388
- Journal Name: Revista da Associacao Medica Brasileira
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, EMBASE, MEDLINE
- Keywords: Accidental falls, Aged, Artificial intelligence, Health literacy, Large language models, Patient safety
- Dokuz Eylül University Affiliated: Yes
Abstract
OBJECTIVE: Older adults increasingly use artificial intelligence-based tools to obtain health information. Although artificial intelligence chatbots such as ChatGPT may enhance access, the quality, readability, and patient safety of fall-prevention information remain uncertain. This study aimed to evaluate the quality, readability, and patient safety implications of ChatGPT-generated responses to common questions about fall risk and home safety in older adults. METHODS: Ten frequently asked fall-related questions were submitted to ChatGPT (version 5.2). Responses were independently assessed by a multidisciplinary panel including physiotherapists, a geriatrician, a physical medicine and rehabilitation physician, an occupational therapist, and an orthopedic specialist. Quality was evaluated using the Mika classification. Readability was measured with the Flesch-Kincaid Grade Level. Interrater reliability was analyzed using a two-way random-effects intraclass correlation coefficient model with absolute agreement (intraclass correlation coefficient [2,k]). RESULTS: Three responses were rated as “excellent,” while seven responses were rated as “satisfactory requiring minimal clarification.” No response received a rating corresponding to “moderately satisfactory” or “unsatisfactory.” The mean Flesch-Kincaid Grade Level was 8.4 (range 4.3–11.9). Five responses exceeded the readability levels commonly recommended for patient education materials. Interrater reliability demonstrated fair agreement (intraclass correlation coefficient [2,k]=0.72; 95%CI 0.64–0.80). CONCLUSION: While ChatGPT provided generally acceptable clinical information, variability in readability and expert ratings raises patient safety concerns. AI-generated health content should be reviewed and tailored to older adults’ health literacy needs before clinical use.