Classification of Diagnostic Certainty in Radiology Reports with Deep Learning

A radiology report is prepared for communicating clinical information about observed abnormal structures and clinically important findings with referring clinicians. However, such observations and findings are often accompanied by ambiguous expressions, which can prevent clinicians from accurately interpreting the content of reports. To systematically assess the degree of diagnostic certainty for each observation and finding in a report, we defined an ordinal scale comprising five classes: definite, likely, may represent, unlikely, and denial. Furthermore, we applied a deep learning classification model to determine its applicability to in-house radiology reports. We trained and evaluated the model using 540 in-house chest computed tomography reports. The deep learning model achieved a micro F1-score of 97.61%, which indicated that our ordinal scale was suitable for measuring the diagnostic certainty of observations and findings in a report.