Sensor Technology in Assessments of Clinical Skill

To the Editor

Clinical-skills assessments provide motivation and direction for learning.^1,2 Adoption of simulation-based education methods has introduced a wide variety of options for clinical-performance assessment.^3,4 To explore the potential value added by integrating sensor technology with a skills assessment, this study investigated whether sensor data could inform previously accepted observation-based performance recommendations for the clinical breast examination (CBE).⁵ We hypothesized that sensor technology would help to characterize successful and unsuccessful CBE techniques at a level of detail that is not possible with observation alone.

The study protocol was administered in 2013 and 2014 and required practicing physicians to perform simulated CBE under conditions that mimic an office visit for a symptomatic patient. A convenience sample of 553 physicians was recruited at three annual clinical meetings: 136 at the American Society of Breast Surgeons, 236 at the American Academy of Family Physicians, and 181 at the American College of Obstetricians and Gynecologists. Participants completed a one-time demographic survey and then reviewed a clinical scenario, performed a CBE, and documented their findings on a clinical-assessment form for each of four breast models. Station-to-station progress was randomized. Participants were informed that the patient believed she felt a mass on self-examination but was currently unable to demonstrate or locate the lesion. Performance was video recorded simultaneously with sensor-data collection. The goal was to capture CBE technique while clinicians were purposefully seeking a mass. Video 1 (available with the full text of this letter at NEJM.org) shows a snapshot of a variety of CBE techniques used by participants.

We used previously validated, sensor-enabled breast models to measure CBE technique (Fig. S1 in the Supplementary Appendix, available at NEJM.org). Model A had a soft, superficial mass measuring 2 cm by 2 cm. Model B was the same as model A except that the mass was smaller (2 cm by 1 cm) (Fig. S2 in the Supplementary Appendix). Model C had a hard, 2-cm mass located near the chest wall. Model D was the same as model C except that the 2-cm mass was molded from a soft silicone derivative. Participants completed a survey indicating whether they had found a breast lesion.

Final numbers and demographic characteristics of the participants are provided in Table S1 in the Supplementary Appendix. Analysis of the sensor data reveals a significant relationship between the force used during palpation and the accuracy of the assessment of the deep-tissue lesions in models C and D (Fig. 1). Reduced palpation forces (<10 newtons) were noted to place physicians at significant risk for missing deep-tissue lesions near the chest wall. Two CBE techniques are shown in Video 2 (available at NEJM.org), combined with sensor-map recordings, showing successful and unsuccessful CBE techniques. Since variations in force cannot be reliably measured by means of human observation, our findings underscore the potential for sensor technology to add value to existing, observation-based assessments of clinical performance. Integration of sensors into clinical-skills assessments may allow for objective, evidence-based training, assessment, and credentialing.

Figure 1. Relationship between Palpation Force and Accuracy in Clinical Breast Examination.

Shown is a plot of accuracy in correctly identifying a breast lesion as a function of the mean (±SD) force in newtons (N) that the participant applied during palpation in the examination of a simulated breast model. Increasing the amount of palpation pressure improved the probability that the participant would identify a deep-tissue breast lesion (in models C and D). However, the slope begins to plateau between 12 N and 17 N. For super ficial masses (in models A and B), there was no correlation between force and accuracy.