A mixed methods study on effectiveness and appropriateness of face shield use as COVID-19 PPE in middle income countries

Abstract

Background

Face shields were widely used in 2020-2021 as facial personal protective equipment (PPE). Laboratory evidence about how protective face shields might be and whether real world user priorities and usage habits conflicted with best practice for maximum possible protection was lacking – especially in limited resource settings.

Methods

Relative protective potential of 13 face shield designs were tested in a controlled laboratory setting. Community and health care workers were surveyed in middle income country cities (Brazil and Nigeria) about their preferences and perspectives on face shields as facial PPE. Priorities about facial PPE held by survey participants were compared with the implications of the laboratory-generated test results.

Results

No face shield tested totally eliminated exposure. Head orientation and design features influenced the level of protection. Over 600 individuals were interviewed in Brazil and Nigeria (including 240 health care workers) in March-April 2021. Respondents commented on what influenced their preferred forms of facial PPE, how they tended to clean face shields, and their priorities in choosing a face cover product. Surveyed health care workers commonly bought personal protection equipment for use at work.

Conclusions

All face shields provided some protection but none gave high levels of protection against external droplet contamination. Respondents wanted facial PPE that considered good communication, secure fixture, good visibility, comfort, fashion, and has validated protectiveness.

Background

Face shields (sometimes called visors) were used extensively during the COVID-19 pandemic, either in conjunction with a face mask or other face covering, or increasingly as an alternative to such face protection. Some policy or public health experts strongly advocated their use as facial personal protective equipment (PPE) during the pandemic, arguing that these products were easier to obtain than medical masks, facilitated communication and were indefinitely reusable.^{1 , 2} Face shields have been particularly popular in low and middle-income countries where governments could not compete for PPE for their own needs amidst global shortages.³ However, in early-mid 2020, reliable data to establish that face shields are protective against respiratory pathogen infection was limited ⁴ and the products had not been tested with SARS-CoV-2 in mind.⁵

Face shields are currently designed and tested as eye protection according to EN166, EN168 and related standards.^{6 , 7} As such, face shields are not regulated as full facial mucosal protection from droplets. Before the Covid pandemic, relatively few studies examined the ability of face shields to protect facial mucosa from droplet contamination, and there is still insufficient understanding of how they are used in real world practice (in a community setting or by health care workers).

It was valuable to generate data on both apparent protective potential of specific face shield products against face shield exposure, and to obtain data on real world perspectives and usage habits. These human factors data can inform the development of safer, more comfortable face shields Hence, we developed protocols to testing how relatively protective face shields were against external droplet exposure, and to undertake such tests to compare different face shield designs. In addition, we collected data from prospective users (who were all respondents, who might or might not have used face shields previously) in middle income countries, to see how well user priorities, understanding or usage habits conformed with possible best practice in using face shields. We also collected data about how often health care workers bought their own personal protection equipment (PPE) for use at their workplace.

Whilst laboratory data is valuable in determining whether a type or PPE or any intervention may work, only real-world data can show whether it does work.⁸ Understanding how face shields may be used or mis-used in the real world is essential to knowing whether or not any value identified in laboratory experiments would be translated into actual benefits through reduced transmission.

Methods

We undertook closely-related projects (denoted as “Tasks”) to assess shield protectiveness supported by understanding of real world user perspectives and practices. A formal protocol was developed as part of securing ethics approval and is available from the authors. Ethics approval for human data collection was secured from institutional review bodies at the World Health Organization (WHO; Ref. CERC.0048), the University of East Anglia (Ref. 2020/21-092), University of Lagos (Ref. NHREC/19/08/2019B) and University of São Paulo (Ref. #4.469.354).

Task 1

We developed and implemented a method to measure the protectiveness of various face shield designs against a droplet challenge from simulated human coughing. Details of the test protocol are given below briefly and otherwise described at greater length in Supplemental file 1. Implementation of this protocol generated data on which face shields offered the most protection against external droplet exposure and helped identify what were the physical features of specific face shield products that offered more or less potential protection.

The test methods were to design and build a cough simulator based on a previously published method,⁹ to mimic the volume, velocity and particle size characteristics of a human cough. This comprised a piston to deliver pressurized air, with an airbrush that introduced an aqueous solution of fluorescein, which fluoresces under Ultra Violet (UV) light, into the airstream. See Figure 1 . The resulting “cough” was directed toward a manikin head wearing a face shield and mounted on a rig based on that from eye protection standard BS EN 168 ⁷ that allowed it to tilt forward and back and side to side. Following standard EN ISO 18526,⁶ 6 headforms are considered to represent 95% of the international adult population; 2 of these headforms were used in the study. The first was the Sheffield Head, as used for respiratory protective device testing (following standard EN 149 ¹⁰) and broadly equivalent to ISO large 1-L (head breadth 157 mm; interpupillary distance 72 mm). The second head was ISO small 1-S (head breadth 135 mm; interpupillary distance 60 mm). For testing purposes, each head was connected to a breathing machine (Inspec International Ltd). The “cough” was timed to coincide with inhalation of the manikin head, thus presenting a worst-case scenario.

Fig 1.

Schematic of manikin head being exposed to simulated cough source.

The face shields were mounted on manikin heads according to manufacturer's specifications, which were described by manufacturers as the fitting most likely to offer maximum coverage.

Deposition of simulated cough droplets on the face were visualized under UV light. This was an adaptation of the method described in EN168 to test for protection against droplets, where an absorbent template was used to capture and visualize deposition.⁷ In this present study, the templates covered the eyes, nose, and mouth regions. Fluorescent droplets deposited on the templates were enumerated as previously done successfully by the authors.¹¹ In this case, the extent of UV fluorescent deposition was classified as undetectable, low, medium, or high, and also respectively given a numerical value 0, 1, 2, or 3. Lower scores meant less deposit, higher scores meant more droplets deposited. One researcher was assigned the categorization task in order to reduce the variability for this analysis and reference templates representing these four categories were used as a point of comparison.

In total, 10 face shields were tested; one each from the United Kingdom National Health Service and the World Health Organization PPE stockpiles, 2 from Nigeria, 5 from Brazil and an additional shield from Tanzania which was unique in that it was constructed from recycled material. Each face shield was tested in triplicate, with seven different head positions relative to the cough source. More detail about the specific face shields that were tested is in supplemental file 1 while the head positions are described in Table 1 . The face shields were ranked from best to worst performing based on the mean overall contamination score (out of a maximum worst performing score of 63) for each face shield.

Table 1.

Positions in which manikin heads were placed

Position 1: Facing forwards with the head face on.

Position 2: Facing forwards and rotated 45 degrees backwards about horizontal axis A (front and looking up).

Position 3: Facing forwards and rotated 45 degrees forwards about horizontal axis A (front and looking down).

Position 4: Rotated 90 degrees to the left about vertical axis B and rotated 45 degrees backwards about horizontal axis A (left and looking up).

Position 5: Rotated 90 degrees to the left about vertical axis B and rotated 45 degrees forwards about horizontal axis A (left and looking down).

Position 6: Rotated 90 degrees to the right about vertical axis B and rotated 45 degrees backwards about horizontal axis A (right and looking up).

Position 7: Rotated 90 degrees to the right about vertical axis B and rotated 45 degrees forwards about horizontal axis A (right and looking down).

Note: Positions are relative to perspective of the manikin head. Eg. “Left” means left as manikin faces forward, but appears as turning to right from perspective of cough source.

Task 2

The second part of our project was to implement a structured survey of people who were prospective users of face shields in urban areas of large metropolitan areas in middle income countries: Diadema, Sao Paulo, Brazil and Lagos, Nigeria. The Brazilian survey was translated to Portuguese using a 3 stage process (English to Portuguese and back to Portuguese followed by verification and refinement). The survey tool in English is available as supplemental file 2, Portuguese language version is available from author MCP. The survey was administered on hand-held Android operating system devices using a data collection platform (Kobo Toolbox) ¹² that could store responses until upload (via WiFi) was feasible. Interviewers received training and helped pilot the questionnaire prior to data collection commencement. Surveyors worked in pairs in pre-selected areas that local community leaders had identified as safe for interviewing members of the public, in a sample of convenience strategy. Interviewers also went to pre-selected primary care clinical environments to administer the same survey to health care workers (HCWs). Target recruitment was 300 responses in each country, with at least one third of responses from HCWs. The survey objective was to acquire data on what features influence shield design preferences and how they are worn in the community and/or clinical environments, to inform how shields can be best designed to minimize transmission of COVID-19 or other influenza-like-illness.

Among other questions, the survey asked for demographic information, preferred form(s) of face cover, prior experiences of using face shields and which factors matter to respondents in their choice of face cover, such as price or comfort. Respondents were asked what cleaning regime (if any) people used for their shields. A list of candidate answers was determined following piloting in the study areas, however, we also solicited also open-ended comments on four specific face shield models shown in picture format; these data will be analyzed thematically and included in future publication reports. The survey was of persons age 18+ only, both health care workers and members of the community, who were interviewed in March-April 2021.

In this report, the data collected in the survey was used to compare user habits, risk perceptions and perspectives with the relative contamination data generated by Task 1.

Results

Task 1: laboratory findings

Table 2 shows overall contamination scores for each face shield tested. More detailed results including for specific head positions are in supplemental file 1. Lower scores indicate least contamination or low breach scores and higher scores represent greater contamination and higher breach scores. On the small head, the best performing face shields were #21, #17, and #18, scoring 2, 3, and 4 respectively. The best performing face shields on the large manikin head were #15 and #18 scoring 8 and 9 respectively.

Table 2.

Overall contamination score for each face shield tested, on small and large manikin heads

Small head			Large head
Ranking	Face shield	Mean overall contamination score (/63)	Ranking	Face shield	Mean overall contamination score (/63)
1	#21	2	1	#15	8
2	#17	3	2	#18	9
3	#18	4	3	#17	18
= average 5	#20	15	4	#23	21
	#15	15	5	#02	22
	#23	15	6	#20	27
= average 7.5	#19	20	7	#22	28
	#02	20	8	#21	29
9	#22	23	9	#16	35
10	#16	27	10	#19	39

The worst performing face shields were #16 (scoring 27) on the small head, and #19 (scoring 39) on the large head. The results demonstrate the protection afforded by each face shield could vary, depending on the head size, for example:

Face shield #17 (Tanzanian recycled material) – When tested on the small head, the results demonstrated that this was one of the more effective visors with a low average score of 3. In comparison, when tested on the large head, the average score was 18 with heavy contamination on the mouth area in position 2 and medium contamination found in the eye and mouth areas in position 4.

Face shield #21 (Brazilian with white head band) – This visor was demonstrated to be the most effective when tested on the small head with an average score of 2. However, when tested on the large head, the average score was 29 with high levels of contamination found when the head was in positions 2 (eyes, nose, mouth); 4 (eyes, nose, mouth); 6 (eyes, mouth); and 7 (nose and mouth).

None of the face shields tested totally eliminated exposure to droplets. There were differences in the level of protection afforded by each face shield model. The orientation of the head influenced the level of protection, and in some instances this could be associated with design features. For example, breaches occurred where face shields were more open at the bottom if the head was tilted back.

Across all 10 face shields, on the small manikin head, position 4 (left looking up) and position 6 (right looking up) gave the highest breach score although mostly as a consequence of low-level breaches, while position 3 (front looking down) resulted in the largest number of high-level breaches.

On the large manikin, head position 4 (left looking up) again gave the highest breach score, while positions 2 (front looking up) and 6 (right looking up) also gave high breach scores. All 3 of these positions resulted in large numbers of high-level breaches.

Survey results

Table 3 Shows demographic information about the persons who were interviewed in Brazil and Nigeria. Demographically, the 2 survey samples were quite similar, for instance about 35% of respondents in both locations had occupations that we could easily denote as professional (= HCWs, educators, or lawyers). The sampling strategy was designed to achieve similar proportions who worked in the health care sector. About 50% of all respondents had previously worn a face shield, while age, and gender demographics were also similar between countries.

Table 3.

Demographic traits of survey respondents

Category	Brazil	Nigeria
Health care workers	130 (43%)	120 (37%)
Community respondents	171 (57%)	207 (63%)
Females	184 (61%)	182 (56%)
Median respondent age band (years)	35-44	35-44
Had previously worn a face shield	135 (45%)	184 (56%)
HCWs who had previously worn a FS	107 (82%)	97 (81%)
Professional occupation	107 (36%)	113 (35%)

Note: FS, Face shield, HCW, health care worker. Only clinically-skilled HCWs, educators or lawyers were categorized as "professional" occupations.

Respondents were asked to indicate which of their priorities influenced their choice of facial PPE. Frequency of responses (where at least 10% of individuals in either survey country chose the item) are shown in Table 4 . This table is ordered in descending order from most to least popular Nigerian responses; the relative descending order for Brazilian considerations is similar if not identical. A heterogenous group of other influences were mentioned by 9.3% of all respondents, the most common of which was ease of use (mentioned by 7 respondents across both countries), reusability/sustainability of the product (n = 5), ability to breathe well (n = 4), aesthetics (n = 4) and the quality or feel of  material used as face covers (n = 4).

Table 4.

Considerations that influenced choice of facial PPE

Consideration	Brazil	Nigeria
Feeling of protection	65%	88%
Comfort	59%	60%
Public health advice	50%	56%
Availability	48%	56%
Legal Regulations	32%	50%
Price	40%	38%
Ability to communicate with others	33%	38%
Social acceptability	18%	34%
Employer mandate about specific product	32%	12%
Other factors	7%	12%

Although the influences in Table 4 were similar and many, we asked a further question to encourage these prospective users to prioritize just 3 considerations with regard to face shields specifically. Figure 2 shows responses that were chosen by a least 10% of respondents in either Brazil or Nigeria as among the “top three” considerations for face shields. Good visibility, feeling of protection and comfort were the 3 most popular considerations in Nigeria. Brazilian respondents chose comfort, not fogging up and good visibility as their top 3 considerations. We cannot say if these or other differences in top priorities reflect differences in culture, local urban environment or specific available product quality in the respective survey locations. Among the most common other considerations mentioned by respondents was personal recommendation (n = 42 across both countries), local or foreign manufacturer (n = 22), ease of breathing (n = 11) and not overheating or avoiding sun exposure (n = 6). Protection from dust or fluids was mentioned by 3 respondents. Some respondents preferred local manufacture some preferred foreign-made products.

Fig 2.

Items chosen as among the 3 most important features to consider in relation to face shields.

It is notable that compared to other features, cost was not the primary consideration in either locality: cost was a top 3 consideration for just 9% in Brazil, 12% in Nigeria. Nevertheless, these were resource-limited settings. A majority of health care workers in Nigeria and a large minority of HCWs in Brazil had purchased PPE items for use at their own workplace (Table 5 ).

Table 5.

PPE items that respondent HCWs had to buy for use at work

Respondents	Hand sanitizer	Goggles	Aprons or coveralls	Face shields	Medical masks	None
Brazil, n = 301	17%	6%	4%	8%	21%	65%
Nigeria, n = 327	53%	1%	8%	28%	54%	28%

Among Brazilians who reported previously wearing face shields, the main cleaning substances used were disinfectants (69.6%), water and soap (20.0%) and water and soap followed by alcohol wiping (6.6%). Among Nigerian users, the majority (87.0%) reused their face shields with about a quarter cleaning it daily before and after use. The cleaning was commonly carried out using soap and water (29.3%), wiping with a dry cloth and/or cotton wool (15.2%) or spraying with disinfectant and/or surgical spirits (9.8%).

Discussion

The laboratory tests demonstrated the range of protectiveness afforded by different face shield designs. It was not within the scope of this project to determine the significance of the breaches, that is, whether the level of breaches made a face shield “safe” or “unsafe,” but to highlight that there is variability in protection quality. For example, by testing in different orientations it was shown that face shields which were more open at the head band led to breaches when the manikin head was tilted forward, while face shields with a shorter visor led to breaches when the manikin head was tilted back. There were more breaches, or breaches at a higher concentration, on a large manikin head compared to a small head, presumably because of less coverage at the periphery of the face. Our laboratory experiment findings are comparable to those in recent similar studies,^{5 ,} 13, 14, 15, 16, 17 in that we observe that the large gaps around many face shield products provide opportunities for droplet ingress.

The laboratory tests indicate that face shields with complete (wrap around) facial coverage tended to be more protective, stable on the head when worn and easier to wear correctly. Users mentioned wanting products to be adjustable, comfortable to wear, and have a good fit for their head size; these concerns align with our findings that face shields were more protective on some (smaller) size heads than large ones, and that the products were likely to be less protective if not worn according to manufacturers’ specifications. Although alternative (incorrect) fittings were not formally tested for protectiveness, it was apparent that incorrect fits were likely to be less protective because the manufacturer's recommended fitting instructions were intended to achieve the maximum coverage that each specific product could provide.

Separate research undertaken by the United Kingdom HSE, to be reported elsewhere, has tested how well face shields retain or divert potential droplets and thus potentially protect others from exposure via a coughing wearer of a face shield. We are aware of only 2 other studies that have done experimental simulations to assess face shields effectiveness as a form of virus source control.^{5 , 15}

In the early phases of the pandemic, some argued that face shields offered a “high degree of protection to the wearer” ² and that face shields were “comfortable” and did not hinder communication.¹ We could not confirm that shields consistently offered high protection to wearers, while our respondents spoke of their experiences that face shields could interfere with communication or be uncomfortable to wear. These findings underscore why it is optimal to obtain both laboratory and real-world-user evidence in assessing the utility of any form of PPE.

Our survey participants very much wanted protection. However, they also highly prioritized good visibility and comfort. High levels of protection may be challenging to achieve without causing some of the diverse problems that users identified when using face shields or other facial PPE, such as fogging up, breathing difficulties, and overheating. Prospective users were more concerned about hindering speech than visual communication; 194 survey participants across both countries said verbal clarity was within the top 3 considerations relative to face shields, while 181 said they would like their face to be seen clearly. A face shield that offers very complete facial coverage might potentially hinder clear speech.

It was reassuring that cost was not a key concern for many respondents; however, the high frequency of product purchases by health care workers, for use in the workplace, suggests that health systems in these locations were under considerable strain to equip workers appropriately. It was useful to document how often HCWs were buying their own PPE, although the results may only reflect the concurrent supply situation and not broadly generalize to other times during the pandemic. It seems evident that if HCWs are choosing and purchasing their own PPE, that they need good quality information about how to make those selections. We also note some products and methods currently used to clean face shields may be damaging or inappropriate, including unvalidated anti-infection or antifogging treatments applied to the shield surface.

Proven protectiveness was a key concern for many respondents. Our research is a key step towards creating a quality assurance framework for face shield products. Part of the project remit was to provide information that the World Health Organization (WHO) could review with a view to informing any future amendments in guidance for face shield use in the WHO Compendium for Medical Devices. One of our wider objectives was also to develop test methods suitable for low resource settings. For instance, our test methods could be adapted by using recycled spray bottles to produce visible dye droplets and using a pump to simulate inhalation.

We believe that our research may be unusual during the Covid pandemic thus far. Most previous research on PPE use in community or clinical settings focused on high income countries; relatively few previous studies have documented how PPE was used by real world users during the Covid pandemic in lower-middle income country settings, or how available PPE seemed to be for health care workers in LMICs.¹⁸

The survey results can help to inform product design, with respect to anti-fogging coatings for instance (and whether they might be robust to cleaning methods and materials used in resource-constrained real world settings). Addressing potential user problems (eg, fogging) while maintaining a high degree of protection from environmental saliva droplets is a key objective for future designers of face shield products. We are aware that manufacturers of other facial protection products (ie, motorcycle helmets) have similar fogging challenges and may have cost-effective strategies that could be employed to reduce or prevent the problem for face shield users. Other research could test exposure to droplets (to the wearer) generated by external coughing from a diverse range of origin distances, and measure how well shields may contain droplets generated when the face shield wearer themselves is speaking or coughing. How well face shields protect against droplet exposure in comparison to medical masks, goggles or cloth face covers is another avenue for future research.¹⁹

Limitations

We only tested the face shields in the scenario of the wearer being exposed to, that is, coughed upon by an external source. In our laboratory research, the face shields were mounted on manikin heads according to manufacturer's specifications and we cannot comment on their protectiveness when worn differently. Only 2 out of the six ISO headforms considered to represent 95% of the international adult population were used in these tests. It was considered that these headforms are representative of international male and female populations at a reasonable resource cost. HCWs were among the persons most likely to have worn face shields in our survey, but our experiments were not optimized with health care procedures in mind. Some research has been done to try to specifically address whether HCWs may be protected by face shields when exposed to virus-laden aerosols from patients undergoing specific procedures.15, 16, 17 We have not considered levels of protection obtained or usage habits when persons wear face shields (or goggles) in combination with medical masks; we acknowledge that this is a common combination of facial PPE, especially for HCWs, and merits its own investigation. Our study was not designed as a prevalence study of PPE habits, but we agree that it could be useful to collect such information formally.

Our survey methods were constrained by local travel restrictions related to the concurrent state of the COVID-19 epidemic in each country: travel to many possible survey sites was not possible. We chose the survey locations based on local knowledge and community partnerships that could facilitate interviewer safety. Therefore, we interviewed in locations of convenience (urban areas of Lagos and São Paulo). This means that the views and experiences of more marginalized socio-demographic groups or rural residents were not solicited and may not be represented.

Although we solicited much information about the practical problems in using face shield products and opinions on specific designs, we have not yet analyzed most of the qualitative data collected; we aspire to undertake and publish such analysis in due course. The qualitative work will endeavor in part to focus on differences observed between Brazilian and Nigerian responses.

Conclusion

Most face shields provide some protection against droplet exposure, but many do not provide a high level of protection against coughs. The protection afforded by each face shield varied with head size: breaches were greater in number and at a higher level of droplet contamination when face shields were on the large manikin head. Persons with smaller heads and specific face shield design features had best chance of being protected by face shields.

Prospective users want facial PPE designs that consider communication, secure fixture, visibility, comfort, fashion, and published efficiency data for protection, ideally available at the point of purchase. The demand for facial PPE has been high, healthcare workers in middle income countries have had to purchase facial PPE out of their own pocket. Improvements in face shield design would ideally provide both fuller face coverage, potential good fit for different head designs and yet minimize possible problems such as fogging, dust, over-heating and muffled speech.

Appendix. SUPPLEMENTARY MATERIALS

List of Supplemental files

Supplemental file 1: Laboratory experiment results (anonymized in peer review version)

Supplemental file 2: Survey instrument