Speech intelligibility test methodology applied to powered air-purifying respirators used in healthcare

Abstract

Powered air-purifying respirators (PAPRs) are worn to protect workers from hazardous respiratory exposures in a wide range of workplaces, including healthcare. However, PAPRs may diminish the ability of wearers to correctly hear words spoken by others, potentially interfering with safe performance of healthcare duties. Accordingly, the impact of PAPRs during healthcare use on speech intelligibility (SI) and consequently on user safety, usability, and patient care is not well studied. The objectives of this study were to (1) determine a listener’s ability to comprehend single-syllable words spoken by a PAPR wearer; (2) determine a PAPR wearer’s ability to intelligibly hear and identify single-syllable words spoken by a PAPR wearer; (3) to assess the variability between speakers, listeners, and PAPR models; (4) to investigate the effects of PAPR design features on SI; and (5) inform a SI requirement for certifying future PAPRs for use in healthcare. This study utilized a Modified Rhyme Test to assess SI for PAPRs. The current National Institute for Occupational Safety and Health (NIOSH) methods for assessing SI are limited to the recently introduced PAPR100 respirator class and the class of respirators claiming chemical, biological, radiological, and nuclear (CBRN) protections. Four NIOSH-approved PAPRs were evaluated using four human subjects. Four experimental conditions were examined:(1) Speaker and Listener with no PAPR; (2) Speaker and Listener both wearing PAPRs; (3) Speaker with a PAPR, Listener without a PAPR; and (4) Speaker without a PAPR, Listener with a PAPR resulted in a total of 144 experiments. Statistical analysis showed that the SI performance ratings were not significantly different among the PAPR models, but experimental conditions had significant impact on SI. The pattern of SI across the conditions of the experiment also showed a significant difference depending on PAPR model. The SI performance rating for all PAPRs could meet the current NIOSH CBRN certification requirement for speech intelligibility.

Introduction

A powered air-purifying respirator (PAPR) is a respiratory protective device equipped with a facepiece, hood or helmet, breathing tube, canister, cartridge, filter, canister with filter, or cartridge with filter, and a blower (Figure 1) (42 C.F.R. Part. 84.2z. 2019). While PAPRs are most often worn by workers in industrial settings, they are also widely used in healthcare and public safety. In 2001, the National Institute for Occupational Safety and Health (NIOSH) conducted a voluntary survey on respirator use within the private sector in collaboration with the U.S. Bureau of Labor Statistics (BLS), which showed that respirators were used in 619,400 private sector workplaces, 42,100 of which reported PAPRs, according to Bureau of Labor Statistics (2002). Findings also indicated that during the same period, approximately 287,100 workers wore PAPRs (BLS 2002). Recently, Wizner et al. (2016) indicated that PAPRs have become more frequently used by healthcare workers when caring for patients.

Figure 1.

Loose-fitting PAPR with helmet. The belt-mounted air purifying unit is worn at the back during operation. Photo credit: NIOSH/NPPTL.

According to the NIOSH/NPPTL’s (National Personal Protective Technology Laboratory) website information, PAPRs use battery-powered blowers to deliver filtered ambient air into either a tight-fitting mask or loose-fitting hood or helmet (Centers for Disease Control and Prevention 2018). Some loose-fitting PAPR hoods are designed to fully cover the wearer’s head and neck to prevent contact with potentially infectious blood and other body fluids. Additionally, loose-fitting PAPRs such as these do not require fit testing when compared to tight-fitting respirators that do require fit testing. Many loose-fitting PAPRs have large transparent lenses, allowing the wearer’s face to remain visible. This may help reduce patient anxiety and improve the quality of communication between healthcare worker and patient because you can see the speaker’s lips moving (ILC Dover 2020; OSHA 2015). PAPRs are reusable with replaceable filters and have a higher assigned protection factor than air-purifying respirators (APRs) and therefore may be more appropriate in high-hazard situations. Table 1 shows two types of PAPRs: tight-fitting PAPRs and loose-fitting PAPRs used in healthcare. Despite these advantages, some disadvantages to using PAPRs have been recognized, such as maintenance requirements, battery usage, higher initial cost, additional weight, and noise when compared to unpowered APRs, particularly Surgical N95 FFRs.

Table 1.

Comparison of classes of respirators used in healthcare.

Respirator type	Surgical N95 Filtering Facepiece Respirator	Tight-fitting PAPR	Loose-fitting PAPR
Assigned Protection Factor	10	50/1000a	25
Fit testing required?	Yes	Yes	No
Notable characteristics	Inexpensive, common, disposable, FDA-cleared medical devices	Reusable, typically with unfiltered front exhalation valves, blown air may improve wearer comfort	No fit test required, blown air may improve wearer comfort

^aTight-fitting half-mask PAPRs have an APF of 50, while full facepiece have an APF of 1000.

Speech intelligibility (SI) is a measure of effective communication and a specific level of hearing effectiveness, i.e., correctly gathering, processing, and interpreting spoken words. Coates et al. (2000) and Radonovich et al. (2010) offer valuable insights into a concern among healthcare workers that loose-fitting PAPRs decrease SI in a multisensory and busy healthcare delivery environment, something that is perhaps most evident in intensive, urgent, or emergency care settings.

Speech intelligibility can degrade when wearing a respirator with a shrouded hood or helmet (Coyne et al. 1998). In particular, the use of a loose-fitting PAPR with a shroud may interfere with hearing effectiveness. The use of the shroud itself may result in varying levels of auditory dampening (rustling of the fabric can occur during movement, and airflow around the ears can vary with the position of the air inlet—over the head, rear of the head, toward the face shield) and orientation of the head (Roberge 2008). Respiratory protection for use by healthcare workers has been shown to decrease effective communication in common healthcare environments (OSHA 1998, 2015). Radonovich and colleagues (2010) found that the use of respirators decreased SI, as measured by the Modified Rhyme Test (MRT), by a range of 1–17%. The average score (i.e., percent correctly hearing spoken words) while wearing a PAPR model was 79% compared to 90% with no respirator (Radonovich et al. 2010), but relatively little further research has been conducted. Stakeholder engagement with the Health and Medicine Division (HMD) of the National Academies (formerly the Institute of Medicine (IOM) 2015), International Safety Equipment Association (ISEA), and respirator manufacturers has consistently identified the need for including SI requirements for NIOSH-approved PAPRs (Palcic 2016).

This study evaluated a selection of PAPRs that may be appropriate for use by healthcare workers using the current NIOSH Standard Test Procedure developed for both powered and non-powered chemical, biological, radiological, and nuclear (CBRN) APRs (Centers for Disease Control and Prevention 2007). The new PAPR100 standard (NIOSH 2020) still uses the same SI method that was used in this study from the new Federal Register (Office of the Federal Register 2020).

Methods

This study utilized the MRT (House et al. 1965) to assess SI for PAPRs. The MRT consisted of 50 six-word lists of monosyllabic English words, most having three sounds in a consonant-vowel-consonant sequence (House et al. 1965). The MRT evaluated a listener’s ability to comprehend single words spoken by the respirator wearer. The MRT test used in this study was based on NIOSH’s (Centers for Disease Control and Prevention 2007) standard test procedures (STP) TEB-CBRN-APR-STP-0313. Four NIOSH-approved PAPRs were studied. Four experimental conditions were used: (1) Speaker and Listener with no PAPR (control condition set as baseline); (2) Speaker and Listener both wearing PAPRs; (3) Speaker with a PAPR, Listener without a PAPR; and (4) Speaker without a PAPR, Listener with a PAPR. When the speaker and listener were both wearing PAPRs, the same PAPR was worn by both participants. Overall, 12 trials of each condition, and a total of 144 individual experiments, were conducted. Four test conditions were randomly assigned. There were 16 total word lists with no repeating words. A unique MRT word list sheet was used for each trial.

PAPR selection

Four PAPR models were studied (Table 2). Respirator models were selected to capture a variety of PAPR styles while substantially representing U.S. market share, availability, and use in industry. Specifically, the models and configurations were chosen to adequately capture and characterize the SI of PAPR models which may reasonably be used in healthcare environments.

Table 2.

PAPR models tested. (Photos courtesy of NIOSH/NPPTL.)

PAPR Model	A	B	C	D
Intended market	General Manufacturing, Transportation	General Manufacturing, Transportation, Healthcare	Public safety	Healthcare
Style	Helmet	Hood	Full facepiece	Shroud
Comments			CBRN
Image

Human subjects test

NIOSH Institutional Review Board approval (HSRB 18-NPPTL-02XP) was obtained prior to test subject recruitment. The NIOSH Standard Test Procedure TEB-CBRN-APR-STP-0313, Determination of Communication Performance Test for Speech Conveyance and Intelligibility of CBRN Full Facepiece APR was used for this study. This STP describes the Communication Performance Tests for Speech Conveyance and Intelligibility of the CBRN APR in sufficient detail to select equipment with the necessary resolution, select the appropriate human subject test panel, conduct the test, and determine whether or not the product passed the test (Centers for Disease Control and Prevention 2007). The STP specified the involvement of eight test subjects: three listeners and five speakers. We deviated from the STP by involving a total of four human subjects who rotated between listening and speaking. The test subjects, recruited from the National Personal Protective Technology Laboratory (NPPTL) test subject pool, did not report hearing impairments, speech defects, or being a non-native English speaker in a pretest questionnaire and did not have facial hair or other conditions that might impair respirator fit. Subjects were trained on the donning and the correct use of the appropriately sized respirator of each model.

Test system setup and equipment

Tests were conducted in a hemi-anechoic chamber, measuring approximately 33 × 55 ft (10 × 16.8 m) at NIOSH/NPPTL (Pittsburgh, PA). The listeners sat in a line spaced 1 foot apart, with the center listener directly in front of the speaker, seated a distance of 10 ft (3 m) away (Figure 2). Nine feet on either side of the midpoint line between the listeners and the speaker, 20–50 Hz pink noise was generated by a Precision Pink Noise Generator (PNG) (GTC Industries, NC MX-Neutrik, Indianapolis, IN). The amplitude of the pink noise was adjusted to 60 dB +/− 2 dBA, measured by two type-2 digital sound meters (Sper Scientific LTD, Model 840029, Scottsdale, AZ). One sound level meter was positioned in front of the speaker and the second sound level meter was positioned at head level 1 foot in front of the center listener. An Acoustical Calibrator (Sper Scientific LTD, model 840031, Scottsdale, AZ) was used to calibrate the sound level meters.

Figure 2.

Diagram of experimental configuration.

The speaker was coached to speak at a level between 75 and 85 dB as measured at the nearer sound level meter. The speaker and listeners donned the respirators according to the randomly selected test schedule. A word list including the 50 test words (there were 16 total word lists with no repeating words), which was randomized, was given to the speaker and multiple-choice answer sheets were given to each listener. The answer sheet contained responses along with five other similar-sounding words for each of the 50 given words. For example, a response sheet for the test word “dent” had the correct response “dent,” as well as the incorrect responses “tent,” “sent,” “bent,” “rent,” and “went.” For each of the test words, the speaker spoke the phrase “the word is” followed by the word. NIOSH personnel verified that the correct words were spoken. Each listener selected the choice of word that best matched the word as heard. The response sheet for each listener in each test was scored and the number correct was recorded.

Data analysis metrics

For each MRT trial, the Adjusted score, Performance rating, and Overall performance rating were calculated. The Adjusted score correcting for the effect of random guesses was calculated according to the equation:

$$\begin{gathered} Adjustedscore=numberofcorrectresponses \\ -\frac{1}{5}(Numberofincorrectresponses) \end{gathered}$$ (1)

The Adjusted score for each listener, with each test involving respirators, was then used to calculate a “Performance rating,” which was normalized by the baseline.

$$Performancerating=\frac{Adjustedscore}{AdjustedscorewithoutPAPR}$$ (2)

The Overall performance rating for each test set was the average of the Performance ratings of the three listeners for each test condition, expressed as a percentage.

$$\begin{gathered} Overallperformancerating(\%) \\ =\frac{PerformanceRating(L1)+PerformanceRating(L2)+PerformanceRating(L3)}{3} \end{gathered}$$ (3)

Using the criterion set in NIOSH STP TEB-CBRN-APR-STP-0313, a PAPR undergoing evaluation for this testing must obtain an overall performance rating greater than or equal to 70% to pass.

Results

A series of experiments was conducted in which SI was assessed as a function of PAPR model and experimental condition (only the speaker wearing the PAPR, only the listener wearing the PAPR, and both the speaker and listener wearing the PAPR). Four PAPR models were evaluated: Helmet, Hood, elastomeric full facepiece, and shroud. Twelve trials were conducted within each experimental condition summing to 144 experiments conducted within a fully crossed experimental model: 3 × 4 × 12. Table 3 depicts the research design, the number of experiments, and the descriptive statistics corresponding to each cell in the design. Given the structure of the data, a 3 × 4 ANOVA was used to assess the main effect of PAPR model, the main effect of experimental condition, and the interaction between the two. The mean percent correct for the baseline condition was 95.20% with a standard deviation of 3.96.

Table 3.

Descriptive statistics by PAPR model and experimental condition.

PAPR Model	Who was wearing the PAPR	Mean Performance rating	Std. deviation
PAPR A (Loose-fitting with Helmet)	Both Speaker and Listener	88.20	13.59
	Speaker	85.99	7.98
	Listener	98.74	6.77
PAPR B (Loose-fitting with Hood)	Both Speaker and Listener	91.02	15.51
	Speaker	87.79	9.74
	Listener	98.22	4.56
PAPR C (Tight-fitting PAPR)	Both Speaker and Listener	95.66	7.08
	Speaker	89.95	12.92
	Listener	94.01	7.11
PAPR D (Shroud)	Both Speaker and Listener	83.84	11.21
	Speaker	97.18	4.12
	Listener	94.07	10.94

The effect of PAPR model on Performance rating was not found to be statistically significant, with F (3,132) = 0.33, p = 0.80. The main effect of experimental condition was significant, F (2,132) = 6.51, p = 0.002, suggesting that SI (as represented by the Overall performance rating) varied significantly as a function of who was wearing the PAPR (Figure 3). When both the speaker and listener were wearing PAPRs, the performance rating averaged across experimental conditions was 89.68, SD = 12.63; when only the speaker was wearing the PAPR, the mean was 90.23, SD = 9.93; and when only the listener was wearing the PAPR, the mean was 96.26, SD = 7.78. Post-hoc, adjusted pairwise contrasts suggested that the condition in which both the speaker and listener were wearing PAPRs was not significantly different from the condition in which only the speaker was wearing the PAPR (p = 0.79). However, SI was significantly higher when the speaker was not wearing a PAPR, as compared with both cases where the speaker wore the PAPR (p = 0.001 with both subjects wearing the PAPR, and p = 0.003 with only the speaker wearing the PAPR). The interaction between experimental condition and PAPR model was also significant, F (6,132) =3.21, p = 0.006, suggesting that the pattern of SI across the conditions of the experiment was different depending on PAPR model, and should be analyzed separately. The means for this significant interaction is reported in Table 3. Post-hoc, adjusted pairwise comparisons of experimental condition for each model were conducted in order to find the pattern of differences. For both PAPR A and PAPR B, the condition in which only the speaker was wearing a PAPR and the condition in which both the speaker and listener were wearing a PAPR were not significantly different (p = 0.59 and p = 0.43, respectively). However, for both PAPR models, the condition in which only the listener was wearing a PAPR was significantly different from the remaining two conditions (p values ranged from 0.002–0.01). For PAPR C, the model with the elastomeric facepiece, none of the experimental conditions had significantly different performance ratings. Finally, for PAPR D, the condition in which both the speaker and listener wore the PAPR resulted in a significantly lower average SI when compared to the conditions in which only the speaker wore the PAPR (p = 0.001) and when only the listener wore the PAPR (p = 0.01). However, the Performance ratings for the condition in which only the speaker and the condition where only the listener wore the PAPR were not found to be significantly different (p = 0.44).

Figure 3.

The mean of overall performance rating at different experimental conditions.

Discussion

Diminished SI has been associated with PAPR use, especially in healthcare settings. Speech may be muffled by the hood or helmet material in front of the mouth; some PAPR models cover the wearer’s ears with fabric that may rustle during head or neck movement, interfering with auditory acuity; and noise produced by the blower motor may be transmitted through the PAPR hose to the ears. In healthcare settings where important clinical information is routinely discussed, diminished SI may pose negative consequences for patient safety. Incorrectly hearing words spoken by a coworker may lead to medical errors. For example, misunderstanding the word “micrograms” as “milligrams” could lead to administration of an inappropriately high dose of medication. A reliable, valid method for evaluating the effect of respirators on speech intelligibility in a healthcare setting would be valuable.

The method used to evaluate the effect of PAPR usage on SI in this study was based on a current standard test procedure used for evaluating the effect of non-powered APRs worn to protect against CBRN hazards, typically worn by workers in the public safety and military sectors. This method had not previously been used in the evaluation of PAPRs. This study indicates that the testing as conducted with the CBRN STP score acceptance limit of 70%, which all tested respirators exceeded, did not identify issues with existing healthcare PAPR models. Radonovich et al.’s (2010) prior study of PAPR SI that was conducted in a simulated noisy intensive care setting did not result in a measured SI score below the 70% limit. Published SI rating methods depending on sentence-length prompts tend to increase word intelligibility, most likely due to the availability of contextual clues, according to Coyne et al. (1998).

Adjusting the testing conditions (e.g., background noise characteristics, movement of speaker and listener) and acceptance parameters (i.e., the MRT score limit) may prove a viable option for creating a more effective evaluation tool. Increasing the level of background noise or adding additional ambient sounds would be expected to better simulate the use case for PAPRs, resulting in more representative measures of SI. Alternatively, the acceptable/unacceptable score criterion could be increased. Either method would require additional study to verify that the procedure results in repeatable, comparable results that accurately reflect the impact of respirators on SI in realistic healthcare situations.

Study limitation and future work

While the authors are aware of informal reports of significantly impaired communication from medical professionals, the selected models passed the requirements developed for elastomeric respirators intended for protection against CBRN agents. A potentially more challenging issue may be that during the MRT, the subjects are remaining relatively still and therefore typical noises arising from flexing of the hood and material movement may not be reflected in the testing. Four different NIOSH-approved PAPR models commonly used in healthcare settings were tested; however, these selected models may not be representative of other PAPR models worn by U.S. healthcare workers. Additional NIOSH-approved PAPR models may require evaluation. Any future testing should include a larger, more representative sample and a healthcare specific SI test using words related to healthcare. Future testing should be conducted in a simulated healthcare environment (e.g., with the sounds produced by ventilators and other equipment in intensive care units and/or emergency departments).

Conclusions

This study adapted NIOSH’s STP for CBRN respirator SI and applied it to four types of PAPRs to assess PAPR SI. This adapted methodology and the requirements established for CBRN-protective respirators are appropriate for PAPRs. This study found no statistically significant differences in SI among different PAPR models. The experimental conditions had significant impact on SI and the interaction between PAPR model and experimental condition was also significant. The SI performance for all PAPRs suggested that the tested existing PAPR models would be capable of meeting the current SI requirements for air-purifying respirators that claim CBRN protections.