900 - Lumbar Disc Herniation with Radiculopathy: A Comparative Analysis of NASS Guidelines and ChatGPT

Objectives: Clinical management of lumbar disc herniation with radiculopathy may warrant utilization of artificial intelligence (AI) models to provide medical information and recommendations. The purpose of this study is to assess and compare ChatGPT responses to The 2012 North American Spine Society (NASS) Clinical Guidelines for the Diagnosis and Treatment of Lumbar Disc Herniation with Radiculopathy.

Design: ChatGPT-3.5 and ChatGPT-4.0 were prompted with 15 questions from The 2012 North American Spine Society (NASS) Clinical Guidelines for the Diagnosis and Treatment of Lumbar Disc Herniation with Radiculopathy. Responses from both versions of the AI were obtained on September 26, 2023 and evaluated for concordance based on operationally-defined parameters of accuracy, over-conclusiveness, supplementary, and incompleteness. A response was deemed accurate if it did not contradict the guideline, over-conclusive if a recommendation was given despite insufficient evidence, supplementary if additional relevant information was provided, and incomplete if it failed to provide relevant information present in the NASS guideline. ChatGPT-3.5 and ChatGPT-4.0 performance was compared via chi-square analyses.

Results: Among the 15 responses produced by ChatGPT-3.5, seven (47%) were accurate, seven (47%) were over-conclusive, 15 (100%) were supplementary, and six (40%) of responses were incomplete. Among ChatGPT-4.0 generated responses, ten (67%) were accurate, five (33%) were over-conclusive, ten (67%) were supplementary, and six (40%) were incomplete. The difference in supplementary information (100% vs. 67%; p=0.014) between ChatGPT-3.5 and ChatGPT-4.0 was statistically significant. However, accuracy (47% vs. 67%; p=0.269), over-conclusiveness (47% vs. 33%; p=0.456), and incompleteness (40% vs. 40%; p=1.000) did not show significant differences between ChatGPT-3.5 and ChatGPT-4.0.

Conclusions: ChatGPT was able to provide recommendations regarding lumbar disc herniation with radiculopathy with reasonable concordance to NASS guidelines. ChatGPT-4.0 was more conservative in its responses compared to ChatGPT-3.5, demonstrating higher accuracy and providing less supplementary information. More validation research is needed to evaluate AI as a clinical resource.