Conducting Psychoeducational Assessments During the COVID-19 Crisis: the Danger of Good Intentions

Abstract

Decision-makers in school psychology are presently engaged in the process of determining how to, if possible, move forward with conducting mandated psychoeducational evaluations of students in schools during the pandemic. Whereas prominent organizations within the profession (e.g., American Psychological Association, National Association of School Psychologists) have issued guidance and encouraged practitioners to delay testing, it is not clear whether that is a viable option in every jurisdiction. Accordingly, professionals are now considering the potential use of telehealth platforms to conduct assessments, in some form, as we move forward and deal with this crisis. The goal of this brief commentary is to raise some provisional limitations associated with the use of telehealth to conduct psychological assessments that we believe will have to be considered as use of these platforms is debated. Recommendations for professional practice are also provided.

The Coronavirus 2019 (COVID-19) pandemic has led to major operational disruptions in school systems throughout the USA that are likely to persist for the foreseeable future. As of this writing, all 50 states have closed schools for at least a month (Education Week 2020) with the majority of that group (86%) enacting statewide policies that have resulted in the closure of schools for the remainder of the academic year (National Governors Association 2020). In all of the remaining states, where statewide policies have not been adopted at this point, varying time frames of closure have been recommended by policymakers. The speed with which the threat emerged in many jurisdictions necessitated immediate action from federal, state, and local authorities leaving students, families, and educational professionals scrambling to adjust to what is now presently being described as the “new reality” in the scientific literature (Horesh and Brown 2020).

Now that the field is effectively emerging from a period of triage, school psychologists and stakeholders (i.e., educational administrators and policymakers) are beginning to address substantive questions about how to best deliver educational and psychological services to students in the present context. While many local educational agencies (LEAs) have elected to adapt instructional materials to be delivered online, many practicing school psychologists remain unclear as to what, if any, school psychological services they will be expected to deliver during the remaining course of the outbreak. Given this uncertainty and the urgency in which these matters must be addressed, there may be a temptation to engage in or endorse clinical practices that have yet to be fully vetted or may even be ethically or legally questionable (Hiramoto 2020), in particular, some of the options being considered for conducting special education assessments at this time.

Moving Forward with Assessment: Possible Considerations

With school closures and physical distancing guidelines in place, practitioners are faced with managing special education assessments while working from home and potentially with limited resources. From our perspective, there are two viable options regarding special education assessments. Those options are to (a) delay testing until stay-at-home orders are lifted and schools return to some semblance of normal operations, or (b) conduct remote assessments using an online delivery platform. Given the regulatory imbroglio that presently exists in many jurisdictions, each of these approaches carries risks that will need to be carefully considered.

Given its simplicity and the potential limitations associated with remote testing (e.g., Luxton et al. 2014), it is likely that many LEAs will default to delaying testing for the time being. In fact, several state school psychological organizations have explicitly recommended this approach in recent COVID-19 position papers (e.g., Hiramoto 2020). It is important to note that position papers from prominent organizations are persuasive, but they do not carry the force of law and thus cannot be relied upon exclusively to establish a legal basis for the adoption of educational policy (Jacob et al. 2016).

Unfortunately, whether that position is legally permissible has yet to be established given the lack of guidance from state departments of education regarding the status of special education regulations in many jurisdictions and the lack of legal precedent related to the current circumstances. Whereas the United States Department of Education (USDE) has issued an executive order relaxing or, in some cases, suspending some existing regulations, it is not clear what, if any, impact those orders have with regard to the status of local special education regulations and laws. In particular, the impact of such actions on the status of assessment timelines and mandated re-evaluations is unclear. For example, the Office for Civil Rights (OCR) for the USDE has noted in their COVID-19 Fact Sheet that:

If an evaluation of a student with a disability requires a face-to-face assessment or observation, the evaluation would need to be delayed until school reopens. Evaluations and re-evaluations that do not require face-to-face assessments or observations may take place while schools are closed, so long as a student’s parent or legal guardian consents (OCR 2020, p. 3).

Ostensibly, it appears that local jurisdictions are tasked with determining under what circumstances in-person evaluations may be required, a process that we suspect is presently being considered by authorities nationwide.

Decision-makers must ensure that those decisions also comport with existing state regulations which may not offer as much flexibility as the federal guidelines (Yell 2019). Districts that elect to delay testing, absent emerging clarity on these matters, may be at risk of future litigation.1 Accordingly, it is imperative that state departments of education expedite the issuance of specific guidance on what, if any, obligations LEAs have to conduct assessments during this period of disruption so that they can better weigh the options that are actually available to them to determine whether testing should be conducted.

Beyond important regulatory considerations, delaying assessments will necessarily also delay potential access to special education and related services (even though limited, special education services [e.g., speech and language therapy, mental health counseling] are presently being delivered remotely by some service providers).2 School psychologists and stakeholders around the country are considering whether to conduct remote assessments using available platforms to support telehealth. Despite the well intentioned desire to provide immediate access to services at this time, the use of telehealth platforms to conduct remote assessments to determine eligibility for special education and related services has not, to this point, been subjected to thorough empirical, ethical, or legal vetting (Fischer et al. 2018; Wright et al. 2020). To shed insight on these matters, we outline several provisional concerns that we believe bear consideration given the preliminary nature of these debates and the potential implications for students and their families as the field considers how best to deal with the prospect of a long-term disruption to in-person assessment activities.

Training and Access to Resources

The first questions when considering whether to move forward with remote testing are conceptual—do practitioners even have access to the materials required for adopting remote testing at scale? In particular, the administration of conventional norm-referenced cognitive-achievement measures. Virtually all of the established telehealth platforms (e.g., PresenceLearning™) require, at a minimum, that the examiner and examinee have access to a reliable digital platform and a trained proctor present remotely to facilitate the administration of necessary in-person materials for those tests (Taylor 2018). Given the present circumstances, it is likely that parents and/or guardians will be the only proctor candidates available for the foreseeable future. Aside from the question of whether novice individuals can be adequately trained remotely to give restricted assessment materials, the inclusion of parents directly in the testing process may create unforeseen conflict of interest issues which will likely have to be disentangled if either party decides to proceed with litigation involving the assessment that was conducted (Naglieri et al. 2004).

Whereas guidance on administration parameters is presently emerging, publishers of commercial ability measures have offered their own advice specific to the measures under their purview. In this regard, Pearson has established a standalone resource page for administering the WISC-V remotely (Pearson 2020). According to the guidelines on that page, school psychologists are only permitted to administer the WISC-V remotely via Q-Global®, Q-Interactive, or a third-party platform developed by PresenceLearning™. At the least, consideration regarding the cost difference between remote administration and traditional format tests must be considered as many LEAs will likely have to confront significant budget reductions in the coming fiscal year and perhaps even beyond.

Even if school psychologists have access to test materials and the necessary technology to conduct assessments over the internet, they may not have the training and clinical experience to engage in telehealth competently (National Association of School Psychologists 2017). In the realm of applied psychology, the de minimus criteria for one to claim that they are competent to engage in a clinical practice is at least graduate-level training specific in the use of that particular technique and clinical experience and supervision in the deliverance of the said technique (Fouad et al. 2009). It is not presently clear which school psychology training programs (particularly at the specialist level) substantively cover telehealth in their training curriculum. As a result, school-based practitioners will have to determine on a case-by-case basis whether they are sufficiently trained to engage in testing over the internet as baseline procedures needed to deliver telehealth at scale in our profession have yet to be established.

Evidentiary Status of Remote Assessment in School Psychology

In view of the relative infrequency with which telehealth procedures have been utilized in school psychological practice in the schools, it is not surprising that the accompanying evidentiary literature associated with the reliability, validity, and equivalence of the scores obtained from remote testing with conventional ability and achievement measures is presently accumulating (Fischer et al. 2018). First, it should be noted that “the instruments used by school psychologists (cognitive ability/intelligence tests, processing tests, neuropsychological tests, achievement tests, etc.), have not been normed or validated to be used under these conditions” (Hiramoto 2020, p. 1). Thus, any encouragement to move forward with the use of such measures under the present circumstances should be evaluated cautiously.

To date, compelling empirical evidence to suggest that a remote administration of a norm-referenced cognitive or achievement test yields equivalent information to an in-person administration, regardless of the protections employed, has yet to be furnished. To be fair, the previous statement is not to suggest that there is no empirical information that is currently available on the topic. On the contrary, there are numerous studies that have been conducted to date, mostly using brief neuropsychological screening measures to suggest it may be possible to administer select psychological measures reliably and validly through telehealth platforms (e.g., Brearly et al. 2017; Grosch et al. 2015). Further, practitioners have been able to administer and score behavior rating scales and conduct diagnostic interviews remotely for over a decade (Garb 2007; Luxton et al. 2014).

However, most research on telehealth assessment with individually administered norm-referenced cognitive-achievement tests has been conducted in controlled, clinical settings (Cullum et al. 2020). This would not be the case with school-based telehealth assessments, as quality of network connections varies and districts have limited ability to control for unreliable connections. Reliable high-speed connections are needed to ensure consistent performance across remote and on-site testing conditions (Brearly et al. 2017) and thus testing results will tend to be biased for students without access to high-quality network connections.

Moreover, the few studies that have examined the feasibility of administering the measures typically used by school psychologists in applied settings all suffer from substantive methodological flaws. These include the use of questionable sample sizes for the scientific questions being examined (Hodge et al. 2019) and the employ of non-experimental designs featuring non-equivalent comparison groups which may have a substantive impact on the scores that were obtained across the different testing conditions (Wright 2018b).

However, in an unpublished white paper examining the equivalence of scores from face-to-face and remote administered versions of the Reynolds Intellectual Assessment Scales-Second Edition (RIAS-2), featuring groups that were evenly matched demographically, Wright (2018a) found that most of the mean score differences were not statistically significant and effect sizes were mostly trivial between the groups. However, participants in the remote administration condition ultimately scored ~ 7 standard score points lower on the Speeded Processing Index and ~ 3 points lower on the omnibus FSIQ score. Whereas these studies provide some evidence of proof of concept for potentially being able to adapt certain cognitive-achievement tests (i.e., RIAS-2, WJ IV) for remote administration, much is to be known on these matters as it pertains to the reliability and validity of the scores generated from remote testing sessions and we welcome future research to shed much needed insight that best addresses the current pandemic and preparations for ensuring continuity of services in future events.

For obvious reasons, the present circumstances have brought attention to the viability of the digital platforms presently afforded by Pearson to aid the field’s needs at this time (e.g., the use of the WISC-V in remote contexts). Whereas there is preliminary evidence to suggest that the scores obtained from digital administrations of the instrument are equivalent to traditional paper and pencil versions (e.g., Daniel et al. 2014), emerging research suggests that this may not be the case (Gilbert et al. 2020). Similar discrepancies have also been observed for other psychoeducational instruments administered via Q-interactive (Krach et al. 2020). Notably absent from this discussion are any compelling construct validity or invariance studies assessing that the underlying measurement model functions the same across testing conditions. Such studies are necessary before an assessment can be appropriately scored and interpreted (van de Vijver 2016).

Finally, it is also worth considering under what conditions school psychologists can be confident that the behaviors that they observe remotely in a child or adolescent’s home will generalize to conventional school-based settings or the temporal stability of those behaviors given the conditions our students are now confronting. We do know that, as a matter of course, the generalizability of in situ test session behaviors is quite low (McConaughy 2005). Attempting to ascertain whether a child has a legitimate functional need for special education and related services in order to access the curriculum may be difficult, if not impossible, to determine in some circumstances (i.e., referrals for high incidence disabilities with low to moderate levels of observable impairment).

Legal, Ethical, and Scope of Practice Issues

From our perspective, the lack of legal clarity on the potential scope of practice limitations with respect to school-based professionals operating under state-level department of education credentials (i.e., specialist-level practitioners) engaging in telehealth service delivery is most concerning. According to the Guidance for Delivery of School Psychological Telehealth Services (National Association of School Psychologists 2017), “Currently, it is not clear if state department of education certification/licensing regulations specifically address telehealth services in their scope of practice for school psychologists” (p. 1). As such, the legality of providing telehealth services under a school psychologist credential is questionable until these matters are procedurally addressed by representative agencies. A recent review commissioned by the American Psychological Association (2013) found that less than half of states have authored explicit regulations that even permit the practice of telepsychology in any context.3 Conversely, most states impose fines and judicial sanctions if one is found to practice in telepsychology without an appropriate license from a board of psychology. To be fair, some states have so-called “good Samaritan” laws that provide practitioners a form of immunity if they engage in appropriately trained behaviors in emergency situations that may afford some protections in the current context but, for obvious reasons, it should not be assumed that said protections will automatically be conferred in every given situation. There needs to be guidance regarding these protections, but such guidance has been conspicuously absent.

From a well-established legal and regulatory context (see McBride et al. 2011), it is clear that school psychologists must ensure whether they are administering tests remotely or in-person, they are administering and interpreting instruments that yield scores that are (a) reliable, (b) valid, and (c) designed and intended for the populations being assessed. Presently, there are likely very few, if any, instances when practitioners can be confident that these requirements can be fully met when administering a norm referenced cognitive or achievement measure from a telehealth platform.

Conclusion and Recommendations for Practice

We caution school psychologists to be wary of the danger of good intentions during uncertain times. Each of us has a desire to provide for the well-being of children. The provisional limitations enumerated in the present article suggest caution in moving forward, if at all, with remote testing to determine eligibility for special education and related services in response to the unique challenges presented during the COVID-19 pandemic. Whereas it may be possible to employ remote platforms to conduct less restrictive forms of psychoeducational assessment such as the administration of rating scales and diagnostic interviews at the present time, those platforms have yet to be employed at scale in our business. As a consequence, we would encourage decision-makers in school psychology to consider the potential downstream effects that are likely to occur with the premature adoption of unproven technological adaptations in a time of crises (Klein 2007). For example, talkspace—an online therapy service that has received funding since 2012—only recently has been forced to comply with extant state regulations governing behavioral health services and it remains unclear under which jurisdictions such organizations are allowed to operate.

Based on available empirical evidence and legal guidance, we recommend that LEAs embrace the protections that appear to be afforded to them under OCR guidance. While remote administration of norm referenced cognitive-achievement tests may ultimately prove to be a viable option in the future, and there is some evidence to indicate that certain tests can be adapted for use in online platforms, questions remain as to whether it is possible to consistently obtain reliable and valid assessment results that LEAs can use as part of the eligibility determination process for special education using such technologies at scale.4 As noted cogently by Weiner (1989), and as expressed in existing test standards (i.e., American Educational Research Association et al. 2014), assessment professionals should know the limits of their instrumentation and act accordingly.

Biographies

Ryan L. Farmer, Ph.D., BCBA,

is Assistant Professor in the school psychology program at Oklahoma State University.

Ryan J. McGill, Ph.D., BCBA-D, NCSP

is Assistant Professor and Director of the school psychology program at William & Mary.

Stefan C. Dombrowski, Ph.D.,

is Professor and director of the school psychology program at Rider University.

Nicholas F. Benson, Ph.D., NCSP,

is Associate Professor and Director of the school psychology program at Baylor University.

Stephanie Smith-Kellen, M.A.,

is a doctoral candidate in the school psychology program at Oklahoma State.

Adam B. Lockwood, Ph.D., NCSP,

is Assistant Professor in the applied psychology program at Western Kentucky University.

Steven Powell, Ed.S. NCSP,

is a doctoral candidate in the school psychology program at Oklahoma State University.

Christina Pynn, M.S.,

is a doctoral candidate in the school psychology program at Oklahoma State University.

Terry A. Stinnett, Ph.D.,

is Professor and Director of Clinical Training in the school psychology program at Oklahoma State University.

Compliance with Ethical Standards

Disclaimer

This document does not contain legal advice and should not be relied upon to establish the regulatory provision of services by public or privately licensed and/or certifiable practitioners in any jurisdiction.

Conflict of Interest

The authors declare that they have no conflicts of interest.

Data Availability

Not applicable.

Code Availability

Not applicable.