GPTZero Performance in Identifying Artificial Intelligence-Generated Medical Texts: A Preliminary Study

被引：8
|
作者：
Habibzadeh, Farrokh [1 ]
机构：
[1] PIHO Polyclin, Res & Dev Unit, Eram Blvd, Shiraz 7143837877, Iran
来源：
JOURNAL OF KOREAN MEDICAL SCIENCE | 2023年 / 38卷 / 38期
关键词：
Scientific Writing; Journalism; Artificial Intelligence; Classification; Sensitivity and Specificity;
D O I：
10.3346/jkms.2023.38.e319
中图分类号：
R5 [内科学];
学科分类号：
1002 ; 100201 ;
摘要：
Background: With emergence of chatbots to help authors with scientific writings, editors should have tools to identify artificial intelligence-generated texts. GPTZero is among the first websites that has sought media attention claiming to differentiate machine-generated from human-written texts.Methods: Using 20 text pieces generated by ChatGPT in response to arbitrary questions on various topics in medicine and 30 pieces chosen from previously published medical articles, the performance of GPTZero was assessed.Results: GPTZero had a sensitivity of 0.65 (95% confidence interval, 0.41-0.85); specificity, 0.90 (0.73-0.98); accuracy, 0.80 (0.66-0.90); and positive and negative likelihood ratios, 6.5 (2.1-19.9) and 0.4 (0.2-0.7), respectively.Conclusion: GPTZero has a low false-positive (classifying a human-written text as machine -generated) and a high false-negative rate (classifying a machine-generated text as human-written).
引用
收藏
页数：12
相关论文
共 50 条
  • [41] Evaluating the Alignment of Artificial Intelligence-Generated Recommendations With Clinical Guidelines Focused on Soft Tissue Tumors
    Matsuoka, Masatake
    Onodera, Tomohiro
    Fukuda, Ryuichi
    Iwasaki, Koji
    Hamasaki, Masanari
    Ebata, Taku
    Hosokawa, Yoshiaki
    Kondo, Eiji
    Iwasaki, Norimasa
    [J]. JOURNAL OF SURGICAL ONCOLOGY, 2024,
  • [42] Quantitative Analysis of Artificial Intelligence-Generated Contours In Comparison With Physician-Generated Contours for Patients with Cancer of The Brain
    Godwin, W.
    McDonald, D.
    Peng, J.
    Maynard, M.
    Owen, G.
    Rapchak, A.
    Roles, S.
    Winiecki, J.
    [J]. MEDICAL PHYSICS, 2022, 49 (06) : E748 - E749
  • [43] Obvious artificial intelligence-generated anomalies in published journal articles: A call for enhanced editorial diligence
    Gulumbe, Bashar Haruna
    [J]. LEARNED PUBLISHING, 2024,
  • [44] Sparks of Generative Pretrained Transformers in Edge Intelligence for the Metaverse: Caching and Inference for Mobile Artificial Intelligence-Generated Content Services
    Xu, Minrui
    Niyato, Dusit
    Zhang, Hongliang
    Kang, Jiawen
    Xiong, Zehui
    Mao, Shiwen
    Han, Zhu
    [J]. IEEE VEHICULAR TECHNOLOGY MAGAZINE, 2023, 18 (04): : 35 - 44
  • [45] Artificial intelligence-generated hip radiological measurements are fast and adequate for reliable assessment of hip dysplasia AN EXTERNAL VALIDATION STUDY
    Archer, H.
    Reine, S.
    Alshaikhsalama, A.
    Wells, J.
    Kohli, A.
    Vazquez, L.
    Hummer, A.
    DiFranco, M. D.
    Ljuhar, R.
    Xi, Y.
    Chhabra, A.
    [J]. BONE & JOINT OPEN, 2022, 3 (11): : 877 - 884
  • [46] Authorship in artificial intelligence-generated works: Exploring originality in text prompts and artificial intelligence outputs through philosophical foundations of copyright and collage protection
    Mazzi, Francesca
    [J]. JOURNAL OF WORLD INTELLECTUAL PROPERTY, 2024,
  • [47] Characterizing the Adoption and Experiences of Users of Artificial Intelligence-Generated Health Information in the United States: Cross-Sectional Questionnaire Study
    Ayo-Ajibola, Oluwatobiloba
    Davis, Ryan J.
    Lin, Matthew E.
    Riddel, Jeffrey
    Kravitz, Richard L.
    [J]. JOURNAL OF MEDICAL INTERNET RESEARCH, 2024, 26
  • [48] Artificial intelligence-generated patient information leaflets: a comparison of contents according to British Association of Dermatologists standards
    Verran, Callum
    [J]. CLINICAL AND EXPERIMENTAL DERMATOLOGY, 2024,
  • [49] Artificial intelligence-generated peripheral blood film images using generative adversarial networks and diffusion models
    Fan, Bingwen Eugene
    Kwok, Ngai Tung Eric
    Chen, David Tao Yi
    Kuperan, Ponnudurai
    Abdul Latiff, Siti Thuraiya Binte
    Wong, Moh Sim
    Ong, Yi Xiong
    Lim, Kian Guan Eric
    Wong, Wei Yong Kevin
    Lim, Shu Ping
    Yong, Bryan Song Jun
    Shanmugam, Hemalatha
    Neo, Yuan Shan
    Ong, Yi Pin Joshua
    Winkler, Stefan
    [J]. AMERICAN JOURNAL OF HEMATOLOGY, 2023, 98 (09) : 1499 - 1502
  • [50] EXTERNAL VALIDATION OF AN ARTIFICIAL INTELLIGENCE-GENERATED ALGORITHM TO DETECT LEFT VENTRICULAR SYSTOLIC DYSFUNCTION WITH ELECTROCARDIOGRAPHY IN THE POPULATION
    Attia, Zachi Itzhak
    Benavente, Ernest Diez
    Tseng, Andrew S.
    Leon, David A.
    Medina-Inojosa, Jose
    Malyutina, Sofia
    Kapa, Suraj
    Schirmer, Henrik
    Kudryavtsev, Alexander
    Noseworthy, Peter
    Ryabikov, Andrey
    Carter, Rickey
    Perel, Pablo
    Friedman, Paul
    Lopez-Jimemez, Francisco
    [J]. JOURNAL OF THE AMERICAN COLLEGE OF CARDIOLOGY, 2020, 75 (11) : 917 - 917
12345