Nonverbal Neurocognitive Assessment during the Coronavirus Disease of 2019 Pandemic: the Effect of Personal Protective Equipment

Abstract

Objective

The coronavirus disease of 2019 pandemic has increased personal protective equipment (PPE) use in medical settings. The current study examined the effect of PPE on a nonverbal measure of neurocognitive functioning.

Methods

The Leiter International Performance Scale, Third Edition (Leiter-3) was administered to 125 children between the ages of three and eight. Fifty-nine children were assessed twice without any PPE and 66 were assessed once without and once with PPE. Group differences on composite scores were evaluated using a repeated measures design, accounting for sex, school attendance, socioeconomic status, and HIV status.

Results

Nonverbal IQ scores increased significantly between test administrations for both groups, but no significant interaction between PPE group and scores on Leiter-3 composites was found.

Conclusions

No main effect of PPE on Leiter-3 outcomes was found. These results suggest clinical and research work using a nonverbal neurocognitive assessment can be completed when PPE is required.

Introduction

The coronavirus disease of 2019 (COVID-19) pandemic makes clinical and research applications of standardized neurocognitive assessment challenging. COVID-19 is highly contagious so infection prevention strategies are essential. Personal protective equipment (PPE) is commonly used to prevent infection (Jara, 2021). Before the pandemic, standard cognitive or neuropsychological testing was completed in-person, without any PPE. In the COVID-19 pandemic, however, one strategy for allowing in-person administration has been to modify the administration procedure by adding PPE to prevent spreading the infection. Commonly used forms of PPE include surgical masks (which obscure the midface) and face shields (large pieces of clear plastic that cover the face and eyes; Koterba et al., 2020). Importantly, neuropsychological tests and measures were not standardized for administration under such conditions. To date, there is limited research on how the use of PPE influences test performance and the reliability of test psychometrics (Loman et al., 2021; Postal et al., 2021).

One previous study of pediatric neuropsychology participants (Lichtenstein et al., 2022b) showed no significant difference in test scores between a group of participants seen just prior to the pandemic (no PPE) and a group of participants seen during the pandemic (with PPE). Importantly, however, that study used a convenience sample and was not designed to assess equivalence. And although some guidelines have been introduced (Postal et al., 2021), no other empirical research has demonstrated whether PPE affects neurocognitive test scores.

Postal and colleagues (2021), writing on behalf of the InterOrganizational Practice Committee, detail various ways in which PPE might impair test performance. The authors venture that masks could be distracting to examinees, make verbalizations harder to understand, and get in the way of nonverbal communications such as facial expressions and movements (Postal et al., 2021). This last point is particularly important for the current study.

This exploratory study used data from an ongoing longitudinal study that started prior to the COVID-19 pandemic. When the pandemic occurred, the study was put on hiatus by the governing IRBs. The study did not work with participants for ~4 months. When the study resumed, all participants wore surgical masks, and examiners wore surgical masks and face shields. This presented an opportunity to understand the effect of PPE on neurocognitive testing. A unique element to this scenario is that our study used an entirely nonverbal test of cognitive functioning (Leiter International Performance Scale, Third Edition [Leiter-3]). In nonverbal assessment, gestures and facial expressions are the only modes of communication between examiner and participant. Without auditory input to rely upon, access to the midface may be particularly important in this form of assessment and so the use of PPE might affect the results. And although PPE may obscure facial expressions (i.e., smiles, quizzical looks, and neutral mouth position) the eyes and hands remain visible, allowing for gestures, hand mannerisms, eyebrow raises, and pointing to continue as tools for the examiner.

To our knowledge, this is the first study to examine the effect of PPE upon standard in-person nonverbal neurocognitive assessment. It is also likely the only study evaluating how PPE affects neurocognitive test scores in non-European-American participants.

Methods

Study Design, Participants, and Procedures

This retrospective cohort was recruited from a longitudinal study of HIV-infected children in Dar es Salaam, Tanzania. The longitudinal study includes ~477 children, some who are children living with HIV (CLWH) and others who are HIV-negative. The research protocol was approved by the Committee for the Protection of Human Subjects of Dartmouth College and the Research Ethics Committee of Muhimbili University of Health and Allied Sciences. A parent or guardian provided consent for all children in the study.

For this exploratory study, we identified two groups of children that differed in PPE on their first two visits. The experimental group (ΔPPE) had their first visit without the use of PPE and a second visit with the use of PPE by the examiner. The control group (øPPE) had both visits without PPE. This selection resulted in 125 total children. Sixty-six ΔPPE participants with an average age of 5.24 years (1.14 SD), and 59 øPPE participants with an average age of 4.98 years (1.01). Socioeconomic status (SES), school attendance, and distribution of sex and HIV-positive status are displayed in Table 1. SES was calculated using principal component analysis (PCA) of relevant variables from a REDCap demographics questionnaire as previously described in Lichtenstein and colleagues (2022a).

Table 1.

Demographic and linear mixed model results

Statistical Analysis

Demographic characteristics of the groups were compared with t-tests on continuous data and chi-squared tests for categorical data (e.g., sex). Data were analyzed using linear mixed-effects models in MATLAB 2020b (Mathworks, Natick, MA). The primary response variables were the Leiter-3 composite scores (Nonverbal IQ, Nonverbal Memory, and Processing Speed). The model fixed effects were time and group, and the random effect was participant variability of repeated measures (i.e., two visits per subject). Including “participant” as a random factor over repeated observations allowed us to estimate fixed effects that replicated over time. The primary hypothesis testing focused on the interaction of time and group. Because both groups started without the use of PPE, we were interested in the trajectory of Leiter-3 composite scores between groups from first to second visit. For this analysis, an effect of PPE would be seen as a worsening of composite scores. CLWH are treated with antiretroviral medications, and HIV-negative control participants are screened in and determined to be healthy. As such, there is not a particular reason for either group of participants to decline across time points (although if decline did occur, it would be more likely in CLWH participants). Thus, if the slope of Leiter-3 scores from first to second visit declined between groups, we would reject the null hypothesis and attribute the difference to a PPE effect. With a total of 135 participants, power was calculated using G*power (Version 3.1.9.6; Faul, Erdfelder, Buchner, & Lang, 2009) for multivariate analysis of variance for two groups over time on three Leiter-3 composite measures. Using a significance level of α = .05, and medium effect size (Cohen’s f repeated measures = 0.3), 0.879 (88%) power was achieved to detect a medium effect between groups over time.

Results

Table 1 shows demographic comparisons and linear mixed effect model results. Demographic factors of age, SES, sex, and school attendance were not significantly different between groups (all p > .05). HIV status was significantly different between groups (p < .001) indicating the ΔPPE group had more HIV-positive children compared to the øPPE group. Linear mixed effect model results showed no significant interaction effects of group x time (all p > .05) for all Leiter composite scores. There was a significant main effect of time found for nonverbal IQ (p < .001) indicating an increase in IQ scores for both groups from first to second visit.

Figure 1 shows the three Leiter-3 composite scores for both groups across the first and second visit. Nonverbal IQ (left panel) shows the increase in scores over time in both groups, which the linear mixed effect model result (time) shows is significant. Figure 1 also shows no divergence in the trajectory of ΔPPE and øPPE groups (red and blue lines respectively) over time. Nonverbal IQ shows a slight difference in slope between groups, but the interaction was not significant (p = .386). Overall, this figure shows no interaction between groups over time and thus no PPE effect was observed.

Fig. 1.

Leiter-3 Composite Scores: Nonverbal IQ (left panel), Nonverbal Memory (center panel), and Processing Speed (right panel) are displayed over time (days). The ΔPPE group is in red and the øPPE is in blue.

Discussion

We found no main effect of PPE upon Leiter-3 performance. Although nonverbal IQ did increase upon second administration for both groups, regardless of PPE use, this might be expected as standardized testing was entirely novel to our participants and so there was likely a learning effect. Familiarity with the testing process at later visits likely resulted in improved performance over time. That said, there was no interaction between group and time. Thus, this finding only further highlights the ability to conduct these assessments with PPE when necessary.

This is the first study, to our knowledge, to examine the effect of PPE upon nonverbal neurocognitive assessment. Despite PPE obscuring the midface, which hinders the examiner’s ability to utilize one of the central methods of nonverbal communication (e.g., facial expressions), test scores did not decline over time. In addition, we demonstrate here that the use of PPE can be easily integrated into testing scenarios in the developing world.

This study had limitations. This study was not designed specifically to test the effect of PPE upon test scores. This was an exploratory study, using data from an ongoing project. Participants were not randomly assigned to PPE groups. Nevertheless, there was no major difference between groups, suggesting that if PPE does have an effect, it is likely small. In addition, there was a difference in the time between test visits for experimental groups. The ΔPPE group had a longer time between their first and second visit (~262 days) compared with the øPPE group (~200 days) due to the temporary shutdown of research operations due to COVID-19. Although this could have affected the trajectory of Leiter-3 results across visits, both groups displayed similar trajectories. Another consideration is that the ΔPPE and øPPE groups were imbalanced with respect to HIV status. The ΔPPE group had more CLWH than the øPPE group. This may not be a limitation. If children were to show worse scores at their second visit, it would be more likely to be CLWH rather than healthy controls as HIV can affect the brains in some children. Instead, we saw either no difference between time points across conditions, or an increase in Nonverbal IQ. Although facial expressions were limited, the examiners still had the ability to use head motions, arm motions, hand gestures, and their eyes to convey the information needed to complete the evaluations. Finally, our sample sizes were not large, but adequately powered to detect a meaningful difference. Future studies on the impact of PPE upon nonverbal neurocognitive assessment should be prospective and enroll larger sample sizes to improve confidence in the generalizability of potential findings.

Clinical programming and research studies using nonverbal neurocognitive assessment can continue during a viral pandemic, such as COVID-19, where PPE is required. These findings build upon prior research indicating that neuropsychological testing can be done with PPE without significantly affecting test scores (Lichtenstein et al., 2022b).

Funding

This study was funded by the National Institutes of Health (NIH), grant number 5R01HD095277 to principal investigator JCB.

Conflict of Interest

None declared.