Peeking inside GPT-4 for medical research and practice - Letter to the editor

Sep 29, 2026

Letter to the editor critically evaluates an article titled “ChatGPT surges ahead: GPT-4 has arrived in the arena of medical research.” The correspondence challenges how medical examination performance results are interpreted, noting domain and question-format limitations. It also argues the original discussion of GPT-4’s advantages, publication surge quality, research methodologies, peer review, and ethical considerations is insufficient. It calls for deeper, more balanced analysis to better inform medical research and practice.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages1

Reading Time3 min

AccessFree to download

Letter to the editor

Critique of ChatGPT medical examination performance

Limited comparison and evaluation of GPT-4

Need to assess publication quality, methodology, and ethics

Call for more balanced discussion of AI in medical publishing

J Chin MedAssoc Letter to the editor Peeking inside GPT-4 for medical research and practice Partha Pratim Ray Department of Computer Applications, Sikkim University, Gangtok, Sikkim, India DEAR EDITOR, I am writing to express my critique of the article titled“ChatGPT surges ahead: GPT-4 has arrived in the arena of medical research” by Ying-Mei Wang and Tzeng-Ji Chen, published in the Journal of the Chinese Medical Association.1 While the article provides an overview of the performance and potential applications of ChatGPT and GPT-4 in the medical field, I believe there are certain aspects that warrant further consideration and clarification.2 First, the article emphasizes the remarkable performance of ChatGPT in various medical examinations, citing studies that indicate its ability to approach or surpass passing thresholds without prior training. While these findings are intriguing, it is important to acknowledge the limitations of these studies, such as the specific examination domains and question formats utilized. Medical examinations, particularly those designed for professionals and specialists, often require complex reasoning, critical thinking, and context-specific knowledge.3,4 It is crucial to evaluate the extent to which ChatGPT can handle such nuanced tasks beyond simple question-answering, as highlighted in the study on Taiwan’s family medicine board examination. Furthermore, the article’s discussion of GPT-4’s capabilities and potential impact on medical research and healthcare is rather limited. The article briefly mentions that GPT-4 is more creative, powerful, and reliable but fails to delve into the specific advancements and improvements that GPT-4 offers over ChatGPT. It would have been beneficial to explore the enhanced features of GPT-4, such as improved contextual understanding, advanced reasoning abilities, or domain-specific fine-tuning. Without a comprehensive analysis of GPT-4, the article leaves readers with incomplete information regarding the latest developments in AI language models for medical research.5,6 In addition to the limited discussion on GPT-4, the article briefly mentions the surge in publications related to ChatGPT and GPT-4 in medical journals. However, it does not delve into the quality and reliability of these publications. Given the relatively recent introduction of these models, it is important to scrutinize the methodology, peer review process, and ethical considerations of the studies conducted using these AI systems. A more thorough examination of the robustness and generalizability of the findings presented in these publications would * Address correspondence. Mr. Partha Pratim Ray, Department of Computer Applications, Sikkim University, 6th Mile, PO-Tadong, Gangtok, Sikkim 737102, [India. E-mail address: ppray@cus.ac.in](India. E-mail address: ppray@cus.ac.in) (P. P. Ray). Conflicts of interest: The authors declare that they have no conflicts of interest related to the subject matter or materials discussed in this article. Journal of Chinese Medical Association. (2023) 86: 866. Received July 11, 2023; accepted July 12, 2023. doi: 10. 1097/JCMA.0000000000000961. Copyright © 2023, the Chinese Medical Association. This is an open access article under the CC BY-NC-ND license ([http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)) have enhanced the article’s credibility and provided readers with a better understanding of the current landscape of AI-driven research in the medical field.7 Moreover, the article implies that medical publishing is lagging behind the current development of AI, without providing sufficient evidence or analysis to support this claim. While AI technologies, including language models, have the potential to revolutionize healthcare, it is essential to maintain a cautious and critical approach when integrating them into medical research and practice.8,9 The assertion that medica

FAQs

What is the main purpose of this letter?

To provide a critique of an article about ChatGPT and GPT-4 in medical research, highlighting areas needing further consideration and clarification.

How does the letter assess claims about ChatGPT performance in medical examinations?

It notes that study results may be constrained by specific examination domains and question formats, and that professional exams require nuanced reasoning beyond simple question answering.

What does the letter say is missing from the article’s discussion of GPT-4?

It argues the comparison is too brief, lacking detailed analysis of GPT-4’s specific improvements such as contextual understanding, reasoning, and potential domain-specific refinement.

Sep 29, 2026

Uploaded by Aria Callaghan