CHATGPT4.O GERIATRICS KNOWLEDGE COMPETENCY AND ITS EVALUATION BY GERIATRICIANS - based on UCLA Geriatrics knowledge test

Sep 28, 2026

Explores whether geriatricians agree with ChatGPT 4.o’s performance on validated UCLA Geriatrics knowledge questions. Eighteen questions were scored with 1 for “correct answer,” -1 for “incorrect answer,” and 0 for “don’t know,” yielding totals from -18 to +18. Test scores compared ChatGPT with trainees using prior published studies. Participant Likert ratings (1–5) assessed ChatGPT4.o responses; ChatGPT scored 18, exceeding all trainees. Six geriatricians rated performance at 4.2, indicating agreement and competency in geriatric knowledge, while concurrent trainee testing continues.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages1

Reading Time3 min

AccessFree to download

Study Aim and Design

Scoring Method and Question Set

Comparative Performance vs Trainees

Geriatrician Likert Ratings

Conclusion and Ongoing Control Testing

350 Innovation in Aging, 2025, Vol. 9, No. S2 exploring whether geriatricians agree with ChatGPT 4.o’s performance. 18 UCLA Geriatrics knowledge questions were answered.“Correct answer”was graded as 1,“incorrect answer”as-1 and “don’t know” as 0. The total score was between-18 to + 18. Test scores were calculated to compare ChatGPT and trainees (medical students, internal medicine residents and Geriatric medicine fellows) from previously published studies. ChatGPT4.o responses were evaluated by participants and graded on a Likert scale of 1-5 (1=strongly disagree, 5=strongly agree) . Score of Geriatric knowledge by ChatGPT 4.0 was 18, higher than all trainees (9.9, 9.5, 13.6, 13,2, 14.7, 14.9. and 17.5 for MS1, MS2, MS3, PGY 1, 2 and 3 IM residents, and Geriatrics fellows respectively). Six geriatricians (four faculty and two fellows) rated its performance at 4.2 on a 1-5 Likert scale. In conclusion, ChatGPT4.o outperformed the trainees in Geriatric knowledge tests. Geriatricians were in agreement with ChatGPT4.o performance. It suggests ChatGPT4.o has competency in Geriatric knowledge. Concurrent control trainees’ testing is underway, as their scores may differ than those of the historical controls’ scores due to the trainees’ own use of artificial intelligence. Abstract citation ID: igaf122.1182 CHATGPT4.O GERIATRICS KNOWLEDGE COMPETENCY AND ITS EVALUATION BY GERIATRICIANS Iriana Hammel1, Natasha Resendes2, Dominique Tosi3, Gokhan Demir4, and Huai Cheng5, 1. Miami VA Medical Center, Miami, Florida, United States, 2. Miami VA GRECC, Miami, Florida, United States, 3. Miami VA Heathcare System, Miami, Florida, United States, 4. University of Miami, Coral Gables, Florida, United States, 5. Minneapolis VA Hospital, Minneapolis, Minnesota, United States ChatGPT has passed USMLE and other medical knowledge exams, demonstrating competence in the medical field. It is less studied in Geriatric Medicine specifically. This study aimed to evaluate the geriatric competency of ChatGPT4.o by examining its performance on the validated UCLA Geriatrics knowledge tests, comparing its performance with that of trainees and

FAQs

How were the UCLA Geriatrics knowledge questions scored in the study?

“Correct answer” was graded as 1, “incorrect answer” as -1, and “don’t know” as 0, with a total possible score from -18 to +18.

What was ChatGPT4.o’s performance compared with trainees?

ChatGPT4.o scored 18 on the geriatrics knowledge test, which was higher than all trainee groups whose scores ranged from 9.5 to 17.5 depending on level.

What did geriatricians think of ChatGPT4.o’s performance?

Six geriatricians rated ChatGPT4.o’s performance at 4.2 on a 1–5 Likert scale, indicating agreement with its performance and supporting competency in geriatric knowledge.

Sep 28, 2026

Uploaded by Olivia Brown