Artificial intelligence large language model scores highly on focused practice designation in metabolic and bariatric surgery board practice questions

BackgroundArtificial intelligence models such as ChatGPT (Open AI) have performed well on the exams of various medical and surgical fields. It is not yet known how ChatGPT performs on similar metabolic and bariatric surgery (MBS) questions.ObjectiveAssess the performance of ChatGPT on Focused Practice Designation in Metabolic and Bariatric Surgery board-style questions.SettingUnited States.MethodsQuestions obtained from the largest commercially available bank of FPD-MBS practice questions were entered into ChatGPT-4, as is, without prior training. We assessed the overall percentage correct as well as the percentage correct within each of the five American Board of Surgery (ABS) question categories. One-way ANOVA was used to determine if the frequency of correct answers differed between categories.ResultsOut of 255 questions, ChatGPT-4 correctly answered 189 (74.1%). Between the five question categories there was no difference between the frequency of correct answers (p = 0.22). It did not matter if questions were entered individually or in groups of up to 10.ConclusionWithout prior training, ChatGPT-4 scored highly when evaluated on the largest practice question bank for the FPD-MBS exam.