Interobserver agreement of lung ultrasound findings of COVID-19

Background-Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown. Methods-This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID-19, diagnosed via RT-PCR. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (kappa) were used to calculate IRR. Results-There was substantial IRR on the following items: normal LUS scan (kappa = 0.79 [95% CI: 0.72-0.87]), presence of B-lines (kappa = 0.79 [95% CI: 0.72-0.87]), >= 3 B-lines observed (kappa = 0.72 [95% CI: 0.64-0.79]). Moderate IRR was observed for the presence of any consolidation (kappa = 0.57 [95% CI: 0.50-0.64]), subpleural consolidation (kappa = 0.49 [95% CI: 0.42-0.56]), and presence of effusion (kappa = 0.49 [95% CI: 0.41-0.56]). Fair IRR was observed for pleural thickening (kappa = 0.23 [95% CI: 0.15-0.30]). Discussion-Many LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.