Reliability and Validity of Child Passenger Safety Restraint Observations by Community Observers

Objective: To compare the accuracy of trained community observers for direct observation of child passenger restraint use to certified child passenger safety technicians who are either professional observers or community-based technicians, and to compare these three groups with a gold standard. Methods: This is a cross-sectional study of interobserver agreement and accuracy in which 75 photos of children depicted in different child passenger restraint systems were rated by 9 observers total, with 3 representing professional observers, 3 representing certified child passenger safety technicians, and 3 representing trained community observers. For each photo, observers indicated type of restraint; the appropriateness of the harness, if applicable; and overall appropriateness of the restraint. A gold standard was established by consensus agreement of 2 certified car seat technician instructors. Results: The sensitivity and specificity for trained community observers in identifying broad groupings of restraint types was good (78-100% sensitivity; 93-99% specificity), but they had low agreement with the gold standard for overall appropriateness of the child passenger restraints (kappa = 0.28). The community observer group was 42 percent less likely to code the photo depiction of appropriate restraint use as appropriate. Conclusion: Community trained observers do show good sensitivity and specificity for identifying the type of restraint but have a trend toward poorer judgment when determining harness appropriateness and overall appropriateness. They may be a cost-effective option for limited restraint identification.