Skip to main content
PMC home page
SAGE - PMC COVID-19 Collection logoLink to SAGE - PMC COVID-19 Collection
. 2020 Oct 3:0734282920962839. doi: 10.1177/0734282920962839

Testing Our Children When the World Shuts Down: Analyzing Recommendations for Adapted Tele-Assessment during COVID-19

Shelley Kathleen Krach 1,✉, Tracy L Paskiewicz 2, Malaya M Monk 1
PMCID: PMC8685593  PMID: 34955591

Abstract

In 2017, the National Association of School Psychologists described tele-assessment as the least researched area of telehealth. This became problematic in 2020 when COVID-19 curtailed the administration of face-to-face assessments. Publishers began to offer computer-adapted tele-assessment methods for tests that had only previously been administered in person. Recommendations for adapted tele-assessment practice had to be developed with little empirical data. The current study analyzed recommendations from entities including professional organizations, test publishers, and governmental offices. The samples for each were small, but the findings were noteworthy. Test publishers were unanimous in recommending the use of their face-to-face assessments through adapted tele-assessment methods (either with or without caution). Governmental agencies were more likely to recommend not using adapted tele-assessment methods or to use these methods with caution. Finally, professional organizations were almost unanimous in their recommendations to use adapted tele-assessment but to do so with caution. In addition to deviations in the types of recommendations provided, entities varied in how the information was distributed. About one-fifth (23.5%) of all entities surveyed provided no recommendations at all. About 45% of the remaining entities provided recommendations on their Web sites. The rest provided information through shared documents, online toolkits, peer-reviewed journals, and emails. Implications for the field of psychology’s future crisis management planning are discussed in response to these findings.

Keywords: tele-assessment, validity, best practices, technology, assessment, COVID-19, crisis, program evaluation

Psychology is considered a “soft science” (Blumenthal-Barby, 2014, p. 531) in part because the field lacks assessment methods that produce psychometrically precise results. Specifically, psychological assessment results are plagued by errors associated with validity (accuracy), reliability (consistency), and appropriate normative (comparisons) information (Sattler, 2018). The more error in the test results, the more inaccurate are decisions based on those results (Matuszak & Piasecki, 2012). Ideally, to reduce error in assessments, the testing experience should be standardized (e.g., same items, same instructions, and similar setting; Sattler, 2018). However, the recent COVID-19 pandemic has forced the use of instruments designed for traditional, standardized face-to-face testing in a tele-assessment environment, thus increasing the risks of testing errors (Krach & Sattler, 2018).

Practitioners considering the use of adapted tele-assessment methods in this situation seek guidance. Adapted tele-assessment methods are defined here as the administration of assessments that were designed to be administered face to face in a computer-mediated manner. Research related to generic tele-assessment is scarce (National Association of School Psychologists, 2017) but further dwindles to only a handful of studies regarding the specific area of adapted tele-assessment (Wright, 2018a, 2018b, 2020).

Given the dearth of available research, guidance regarding the use of adapted tele-assessment had to come from national associations (e.g., American Psychological Association [APA] and National Association of School Psychologists [NASP]), from governmental regulatory groups (e.g., U.S. Department of Education and US Social Security Administration), and from test publishing companies (e.g., Mental Health Systems [MHS] and Pearson). However, these recommendations, when forthcoming, seemed to differ based on the source and the means of recommendation.

The goal of this article was to analyze the professional recommendations related to adapted tele-assessment disseminated during the early days of the COVID-19 crisis. In particular, the goals were to examine: (1) what types of recommendations were provided, (2) differences between recommendations by the type of entity providing them, and (3) dissemination methods of these recommendations to practitioners in the field. The findings from this study may help current psychologists interpret the recommendations that they have received. More importantly, the findings can help future leaders in the field of psychology to make changes to field-wide planning for crisis management.

Tele-Assessment

The term “telehealth” includes distance-based mental health and physical health services in the areas of assessment, prevention, and intervention. Perle and Nierenberg (2013) described the need for telehealth methods to address shortages of services to people who may not otherwise have had access to them in their communities. But with the onset of the COVID-19 pandemic, the sudden increased need for distance services resulted in an exponential surge in telehealth implementation for all clients. Tele-assessment is no longer limited to the few individuals who were receiving telehealth services before the pandemic (Wosik et al., 2020).

“Telehealth” comprises a broad field. For the purposes of this article, “tele-assessment” is defined as “diagnostic, psychological assessment procedures administered to (an) individual(s) who is not in the same room as the examiner through the use of telecommunication technologies.” Telecommunication technologies include, but are not limited to, telephone conferences, online video conferencing (e.g., Zoom video conferencing), and web-based assessments. Furthermore, as documented by Jordan and Shearer (2019), “information transmitted between patients and providers may be occurring synchronously (occurring in real time) or asynchronously (occurring at different time points) and may include text, graphics, video, sounds, or other data” (p. 2).

Some tests have been specifically designed to be administered using a telecommunication device (e.g., the Gibson Test of Cognitive Skills, 2nd Edition [GTCS-2]; Moore & Miller, 2016); therefore, using them as tele-assessments does not break standardization, and such assessments are not the focus of the current article. Instead, this article examines the recommendations regarding the use of adapted tele-assessment. “Adapted tele-assessment” is the administration using telecommunication technology of tests that were standardized for face-to-face administration. Adaptive tele-assessment creates a higher risk of introducing error because it breaks an instrument’s special standardization, testing environment, and norms. Without the consistency and standardized procedures with which the tests were designed, raw scores can no longer be compared to one another with any assurance of psychometric accuracy (Kline, 2015)

Previous Research on Adapted Tele-Assessment

Ideally, any test adapted to a tele-assessment format from a paper-based and/or a face-to-face method should demonstrate psychometric equivalency. The requirements for psychometric equivalency were provided by the American Psychological Association Committee on Professional Standards and Committee on Psychological Tests and Assessment (1986) in their Guidelines for Computer-Based Tests and Interpretations and the joint commission Standards for Educational and Psychological Testing (American Education Research Association [AERA], American Psychological Association [APA], & National Counsel on Measurement in Education [NCME], 2014). These equivalency standards require the following: (a) inter-version equivalency correlations, (b) mean score differences that are not statistically significantly different with small effect sizes, and (c) the score dispersion shapes are not statistically significantly different from one another. Two additional guidelines should also be considered: (d) demographic equivalency between the study sample and the original normative sample (Grosch, Gottlieb, & Cullum, 2011; Hodge et al., 2019; Krach, McCreery, Dennis, Guerard, & Harris, 2020) and (e) sufficient sample size to ensure the statistical power to perform the equivalency analyses (Cohen, 1988; Farmer et al., 2020a). All these criteria must be demonstrated to state definitively that two forms of a test are equivalent.

Unfortunately, the literature is scarce regarding the use of face-to-face, diagnostic, direct assessments through an adapted tele-assessment method. Only the following tests have been evaluated for this purpose: the Woodcock-Johnson Tests of Cognitive and Achievement, Fourth Edition (WJ-IV: COG and WJ-IV: ACH; Wright, 2018b), the Reynolds Intelligence Assessment Scale (RIAS; Wright, 2018a), and the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V; Wright, 2020). None of these studies met all the requirements set forth to definitively support version equivalency.

Although equivalency research examining Pearson’s paper-based and Q-Interactive (Qi) methods (Daniel, 2012a, 2012b, 2013a, 2013b, 2013c; Daniel, Wahlstrom, & Zhang, 2014a; Daniel, Wahlstrom, & Zhou, 2014b) exists, these studies have all been performed using in-person (face-to-face) and not adapted tele-assessment methods (Cayton, Wahlstrom, & Daniel, 2012). In these studies, both the test taker and test administrator were in the same room.

Challenged Beliefs and Practice (COVID-19)

COVID-19 challenged entities and individuals to change their minds and their practices regarding adapted tele-assessment (Farmer et al., 2020a, 2020b). One clear example comes from the NASP. In their Considerations for Delivery of School Psychological Telehealth Services (2017), NASP provided a list of concerns related to tele-assessment, specifically reporting, “most publishers do not currently have well-established or well-vetted systems established for conducting assessments virtually.” Their recommendation at that time was that school psychologists only use “validated assessment tools and methods” (pgs. 4–5, & 25). Given the findings discussed above, none of the adapted tele-assessments described in this article would meet the criteria. In essence, NASP stated that adapted tele-assessment should not be used. They reiterate this when stating that adapted tele-assessment “results may not hold up in a legal proceeding, since test construction and norming samples did not include a sample of those who were administered the assessments remotely” (p. 22).

In 2020, during the height of the COVID-19 crisis, NASP provided an updated set of recommendations for virtual service delivery. In this updated document, they emphasized caution in administering and interpreting assessment results provided through adapted tele-assessment. Specifically, they state that “high-quality evidence” is needed prior to using any assessment adapted for delivery using only “platforms designed for that purpose” (p. 2). They add that any alteration to the standardized form of assessment should be considered carefully when making diagnostic decisions and clearly documented in any resulting assessment report. Therefore, they move from stating not to use traditionally available tests as tele-assessments (National Association of School Psychologists, 2017) to suggesting cautious use.

Farmer and colleagues (2020) go into additional detail as to how these types of policy/recommendation shifts manifest. A shift may have occurred when practitioners were faced with limited choices for conducting direct assessments unless they adapted to tele-assessment methods. Therefore, NASP’s position altered when practitioners were only given two choices: (1) delay Individuals with Disabilities Education Improvement Act [IDEA] (2004) child find requirements or (2) use methods of questionable validity.

How Information Was Disseminated during COVID-19

Initial recommendations regarding adapted tele-assessment became available in mid-March 2020. On March 16, 2020, the Office of Civil Rights posted a white paper to their Web site suspending face-to-face testing for all P-12 schools until schools reopened. Shortly after, on March 30, Pearson provided a webinar that gave step-by-step directions on how to use their Q-Interactive products in an adapted tele-assessment format (Henke, 2020). Pearson’s webinar indicated no restrictions to the use of their products. In addition, on April 3, the American Psychological Association (APA, Wright, Mihura, Pade, & McCord, 2020) published an article on its Web site reiterating that the use of face-to-face tests in adapted tele-assessment was fine as long as caution was used when interpreting the results. On the other hand, the American Psychiatric Association [ApA] (2020) produced a similar white paper indicating no restrictions at all. For practitioners, it must have seemed that adapted tele-assessment recommendations would change depending on what you read, when you read it, who stated it, and how you received it.

These recommendation differences continued as school buildings remained closed and as states extended social distancing recommendations into the summer and fall of 2020, and the methods by which practitioners might access these recommendations varied widely as well. Distribution to practitioners might best be described as haphazard. Some recommendations were provided as announcements on the entities’ Web sites (e.g., MHS; Wheldon, 2020), some were available through multimedia presentations (e.g., podcasts, webinars, and virtual workshops; e.g., Henke, 2020; Sharp, Mcfadden, & Morera, 2020), and still others were shared documents stored in cloud drives (e.g., Wyoming State Board of Education, 2020). Methods of distributing the recommendations included advertisements and sharing on professional listservs, through online NASP communities and through personal and professional email correspondence.

The current authors asked three main research questions related to these recommendations: (1) what information did practitioners receive regarding adapted tele-assessment methods for tests that were traditionally administered as face-to-face, in-person assessments, (2) was there a difference in the recommendation regarding adapted tele-assessment based on what type of entity was providing the information (i.e., test publisher, government agency, or professional organization), and (3) how were recommendations disseminated during the time of the COVID-19 crisis?

Methods

Entities

Three categories of entity recommendations were evaluated. Psychological and mental health–related professional associations (e.g., APA and NASP) were chosen at the national and state levels. Test publishers were chosen if they provide commonly used diagnostic, direct assessments (e.g., Pearson and MHS). Finally, government agencies were chosen if they provide funding or legal guidelines for psychological services (e.g., U.S. Departments of Education and US Social Security Department). All related national professional organizations and test publishers were included.

In addition to the national entities described above, a stratified randomized sample of 15 additional state entities was also included. Given that COVID-19 has been reported as a national pandemic by the Center for Disease Control (2020, https://www.cdc.gov/), a nationally differentiated sample was chosen. In preparation for doing a chi-squared analysis, each group needed a minimum of five entities per cell (McHugh, 2013). Therefore, using US Census data, five state governmental and/or professional agencies were randomly selected by using a random number generator after separating states into one of three potential groups: (1) the 10 states with the highest populations, (2) the 10 states with the lowest populations, and (3) the states whose populations did not fall in either of these two categories.

Procedures

Recommendations were identified through a comprehensive Google search of entity Web sites, announced trainings/webinars through listservs, Facebook links from psychological assessment groups, and email responses directly from the entity. Information was pulled only from sources that were approved agents for disseminating information on behalf of the entity involved. Direct quotes related to adapted tele-assessment were pulled from all the following types of sources: emails, peer-reviewed and non-peer–reviewed journal articles, documents published online, official organizational webpages, blog and news posts, and toolkits specifically dedicated to the dissemination of COVID-19 resources and information. All data were collected between May 1 and July 1 of 2020. Table 1 provides information on the type of information evaluated as well as how the information was disseminated.

Table 1.

Visual Analysis of Recommendations.

Severity Categorization
Entity Do Not Use Caution No Concern No Advice
Professional organizations
 ApA1 X
 ApA2 X
 ApA3 X
 ApA4 X
 APA5 X
 APA6 X
 APHA7 X
 CRS8 X
 NASP9 X
 NRHSP10 X
 NRHSP11 X
 California ASP12 X
 Louisiana SPA13 X
 Montana ASP14 X
 ASP of Pennsylvania15 X
 South Carolina ASP16 X
 Tennessee ASP17 X
 Wisconsin SPA18 X X
Government agencies
 Alaska DEED19 X X
 Alaska DEED20 X
 Alaska DEED21 X
 Alaska DEED22 X
 Idaho SESTA23
 Louisiana DOE24 X X
 Louisiana DOH25 X
 Louisiana state BEP26
 Maryland BEP27 X
 Maryland public schools28 X
 Mississippi DOE29 X
 Montana OPI30 X
 Nebraska DOE31 X
 Public schools of North Carolina32 X X
 North Carolina psychology board33
 Commonwealth of Pennsylvania34 X
 Pennsylvania DOE35 X X
 South Carolina BEP36
 South Carolina DOE37 X
 Tennessee DOE38 X
 U.S. DOE39 X
 U.S. VA40 X
 U.S. VA41 X
 U.S. VA42 X
 U.S. Social security administration43 X
 Centers for medicare and medicaid services44 X X
 Virginia DOE45
 Wyoming SBP46 X X
Test publishers
 PAR47 X
 Pearson/PsychCorp48 X
 Riverside/Harcourt49 X
 MHS50 X
 WPS51 X
Open in a new tab

Note 1. ApA = American Psychiatric Association; APA = American Psychological Association; APHA = American Public Health Association; CRS = Congressional Research Service; NASP = National Association of School Psychologists; NRHSP = National Register of Health Service Psychologists; ASP = Association of School Psychologists; SPA = School Psychological Association; DEED = Department of Education and Early Development; SESTA = Special Education Support and Technical Assistance; DOE = Department of Education; BEP = Board of Examiners of Psychologists; DOH = Department of Health; OPI = Office of Public Instruction; VA = Department of Veteran’s Affairs; SBP = State Board of Psychology.

Note 2. Hilty et al. (2016)1; American Psychological Association [ApA] (2020)2; American Psychological Association [ApA] & American Telehealth Association (2018)3; Hafften (2016)4; Wright et al. (2020)5; Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 20136; American Public Health Association [APHA] (2020)7; Elliott (2020)8; National Association of School Psychologists (2020)9; Alvord, Siegel, and Harris (2019)10; Dalton (2020)11; California Association of School Psychologists [CASP] (2020)12; Louisiana School Psychological Association [SPA] (2020)13; Montana Association of School Psychologists [ASP] (2020)14; Association of School Psychologists [ASP] of Pennsylvania (2020)15; South Carolina Association of School Psychologists [ASP] (2020)16; Tennessee Association of School Psychologists [ASP] (2020)17; Wisconsin School Psychological Association [SPA] (2020)18; Alaska Department of Education and Early Development [DEED] (2020d)19; Alaska Department of Education and Early Development [DEED] (2020a)20; Alaska Department of Education and Early Development [DEED] (2020b)21; Alaska Department of Education and Early Development [DEED] (2020c)22; Idaho Special Education Support and Technical Assistance [SESTA] (2020)23; Louisiana Department of Education [DOE] (2020)24; Louisiana Department of Health [DOH] (2020)25; Louisiana State Board of Examiners of Psychologists [BEP] (2015)26; Maryland Board of Examiners of Psychology [BEP] (2018)27; Maryland Public Schools (2020)28; Mississippi Department of Education [DOE] (2020)29; Montana Office of Public Instruction [OPI] (2020)30; Nebraska Department of Education [DOE] (2020)31; Public Schools of North Carolina (2020)32; North Carolina Psychology Board (2020)33; Commonwealth of Pennsylvania (2010)34; Pennsylvania Department of Education [DOE] (n.d.)35; South Carolina BEP (2020)36; South Carolina Department of Education [DOE] (n.d.)37; Tennessee Department of Education [DOE] (2020)38; United States Department of Education (2020)39; United States Department of Veteran Affairs (2017)40; United States Department of Veterans Affairs (2018)41; Lum and Tambyah (2020)42; Verma (2020)43; Centers for Medicare and Medicaid Services (2020)44; Virginia Department of Education [DOE] (2020)45; Wyoming State Board of Psychology (2020)46; PAR (2020)47; Henke (2020)48; Walker et al. (2020)49; Wheldon (2020)50; WPS, 202051.

Instrumentation

Categorical determination analysis was conducted by a team of four individuals. One of the team members is a professor in the field of school psychology. Two of the others are psychology doctoral students who had completed a minimum of six semester hours of coursework on assessment administration and interpretation. The final rater is a graduate from an Ed.S. degree program in Clinical Mental Health Counseling who previously completed a minimum of three semester hours of coursework and 12 semester hours of internship on assessment administration and interpretation.

A first round of data analysis was completed by the team. In this first round analysis, all four team members examined a subset of about 20 quotes on their own and assigning a descriptor/label to each. These descriptors/labels were compared thematically. Five categories were identified from these themes in this first round analysis. Three of these five categories remained unchanged for the final analysis: (1) “do not use,” (2) “use with no concerns/restrictions,” and (3) “no advice given.” In the first-round analysis, two additional categories listed “mild caution” and “moderate caution” were initially identified. Due to the lack of interrater agreement, these two categories were condensed into a single category of “caution.” Raters then developed an agreed-upon set of rules/definitions for each of the four categories prior to the final analysis.

The final four categories were: (1) do not use, (2) use with no concerns/restrictions, (3) use with caution, and (4) no advice given. The rules/definitions for each of the categories are as follows. For the “no advice given” category, the entity may or may not have provided guidance on telehealth, but they were included in this category if the advice was not specific to direct, psychology-based, adapted tele-assessment. For the “no concerns/no restrictions” category, the entity must have provided guidance on adapted tele-assessment with no caveats as to how the tests should be used. For the “do not use” category, the entity explicitly stated not to use this method of assessment. Finally, the “use with caution” category was chosen if adapted tele-assessment was discussed with caveats about the administration and/or the interpretation of the tests. Provided are two specific examples of categorical decisions.

Example 1 comes from a fact sheet posted on the Web site for the United States Department of Education (2020, pg. 3). “If an evaluation of a student with a disability requires a face-to-face assessment or observation, the evaluation would need to be delayed until school reopens.” In this example, the recommendation was categorized as do not use.

Example 2 comes from the Web site position paper for the APA (Wright et al., 2020, para 2). In this situation, the recommendation was categorized as caution:

However, the situation is more challenging with assessment services that have standardized administration procedures that require in-person contact. In considering these challenges, some psychologists may choose to pause their psychological assessment services during this time; however, there are others who do time-sensitive, high-need, and/or high-stakes assessments that really need to continue. Most current and emerging telehealth guidelines largely focus on psychotherapy, and as such, tele-assessment guidance is necessary.

Each statement was read aloud to the group in a Zoom meeting. The members submitted their votes to the professor once after she had voted. Votes were submitted through the Zoom chat feature with permission set for only the professor to view the responses. For the four categories, first-round, interrater, unanimous agreement was 90% (46 out of 51 cases). The remaining five cases had only one or two of the five members in disagreement. When this happened, the final decision was based on the majority opinion.

Analysis

What recommendations were provided?

To address the first research question, findings were tabulated in a graphical manner for easy reference (Tables 1 and 2).

Table 2.

Frequency Counts for Recommendations by Type of Entity.

Entity Type
Recommendation Professional Organization Government Agency Test Publisher Total
No advice 5 7 0 12
Do not use 5 12 0 17
Caution 7 9 3 19
No concern 1 0 2 3
Total 18 28 5 51
Total removing no advice 13 21 5 39
Open in a new tab

Was there a difference in recommendations across entities?

To address the second research question, a chi-square analysis was attempted with entity type as the grouping variable: professional organizations, government agencies, and test publishers. The categorical variable included: (1) do not use, (2) use with no concerns/restrictions, and (3) use with caution. However, to run a chi-square analysis, two assumptions must first have been met (McHugh, 2013): (1) a frequency of at least one response per cell and (2) 80% or more cells should have a frequency of at least five.

The first chi-square assumption was violated due to issues with the publisher and the government agency grouping variables. None of the test publishers indicated “do not use” in their recommendations, creating the first null set. In addition, none of the government agency entities indicated that they had “no concerns,” resulting in a second null set. The second assumption was also violated in that 55.6% of the cells had counts of less than five per cell. Given the nature of these recommendations, it is possible that collecting additional data would continue to result in at least one null set. So, instead of running the chi-square analysis, a graphical representation of the data was provided in a cross-tabulation in Table 2. Percentages were calculated from all entities that provided advice and reported in the results section.

How were the recommendations disseminated?

Tables 3 and 4 provide information about the manner in which data were provided. From this, percentages were calculated and presented in the results section.

Table 3.

Recommendation Dissemination Method.

Entity COVID-19 Specific? Material Published Dissemination Method
Professional organizations
 ApA1 No Journal article Peer-reviewed journal
 ApA2 Yes Guidelines/recommendations Toolkit
 ApA3 No Guidelines/recommendations Webpage, publication
 ApA4 No Video recording Toolkit
 APA5 Yes Blog/news Webpage
 APA6 No Journal article Peer-reviewed journal
 APHA7 Yes Facts Toolkit
 CRS8 No FAQ Document and congressional report
 NASP9 Yes Guidelines/recommendations Webpage and toolkit
 NRHSP10 No Video recording and blog/news Webpage
 NRHSP11 Yes Blog/news Webpage
 California ASP12 Yes Position paper Webpage and document
 Louisiana SPA13 Yes Position paper Webpage and document
 Montana ASP14 Yes Statement Email
 ASP of Pennsylvania15 Yes Statement Email
 South Carolina ASP16 Yes Statement Email
 Tennessee ASP17 Yes Guidelines/recommendations Document
 Wisconsin SPA18 Yes Statement Webpage and document
Eligibility entities/Service providers
 Alaska DEED19 Yes Guidelines/recommendations Webpage
 Alaska DEED20 No Guidelines/recommendations Document and email
 Alaska DEED21 Yes FAQ Document and email
 Alaska DEED22 Yes Video recording Webpage
 Idaho SESTA23 Yes Guidelines/recommendations Toolkit and Document
 Louisiana DOE24 Yes Guidelines/recommendations Document
 Louisiana DOH25 Yes Guidelines/recommendations Toolkit and document
 Louisiana state BEP26 No Guidelines/recommendations Webpage and document
 Maryland BEP27 No Policy/regulation/ethical code Webpage and document
 Maryland public schools28 Yes FAQ Document
 Mississippi DOE29 Yes Guidelines/recommendations Toolkit
 Montana OPI30 Yes Guidelines/recommendations Toolkit
 Nebraska DOE31 Yes FAQ Document
 Public schools of NC32 Yes FAQ Document and toolkit
 NC psych board33 Yes FAQ Web site
 Commonwealth of PA34 No Policy/regulation/ethical code Document
 Pennsylvania DOE35 Yes FAQ Webpage
 South Carolina BEP36 No Statement Document
 South Carolina DOE37 Yes FAQ Toolkit
 Tennessee DOE38 Yes Guidelines/recommendations Document
 U. S. DOE39 Yes Facts Document
 U. S. VA40 No Facts Document
 U. S. VA41 No Blog/News Webpage
 U.S. VA42 Yes Journal article Peer-reviewed journal
 U. S. SSA43 Yes Blog/news Webpage
 Centers for medicare & medicaid44 Yes Facts and blog/news Document and webpage
 Virginia DOE45 Yes FAQ Webpage
 Wyoming SBP46 Yes Guidelines/recommendations Document and webpage
Test publishers
 PAR47 Yes Guidelines/recommendations Webpage and toolkit
 Pearson/PsychCorp48 Yes Video recording Webpage
 Riverside/Harcourt49 Yes Video recording Webpage
 MHS50 Yes Statement Document and email
 WPS51 Yes Statement Webpage
Open in a new tab

Note 1. Journal article = published, peer-reviewed article; guidelines/recommendations = specific advice given to be used by practitioners and parents; video recordings = webinars, conference presentations, and other recorded video or audio; blog/news = blog and news posts; facts = fact sheets and webpages; FAQ = frequently asked questions; position paper = position of entity regarding tele-assessment; statement = statement by entity on tele-assessment or call to action on tele assessment; policy/regulation/ethical code = legal and ethical guidance on tele-assessment.

Note 2. Email = email communications with stakeholders; peer-reviewed journal = published article that was peer reviewed; publication = article published by an entity that was not in a peer-reviewed journal; document = working documents, PDFs, word documents, etc. published online; webpage = webpages, Web sites, and blogs; toolkits = webpages specifically dedicated to the dissemination of COVID-19 resources and information.

Note 3. ApA = American Psychiatric Association; APA = American Psychological Association; APHA = American Public Health Association; CRS = Congressional Research Service; NASP = National Association of School Psychologists; NRHSP = National Register of Health Service Psychologists; ASP = Association of School Psychologists; SPA = School Psychological Association; DEED = Department of Education and Early Development; SESTA = Special Education Support and Technical Assistance; DOE = Department of Education; BEP = Board of Examiners of Psychologists; DOH = Department of Health; OPI = Office of Public Instruction; VA = Department of Veteran’s Affairs; SBP = State Board of Psychology.

Note 4. Hilty et al. (2016)1; American Psychological Association [ApA] (2020)2; American Psychological Association [ApA] & American Telehealth Association (2018)3; Hafften (2016)4; Wright et al. (2020)5; Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 20136; American Public Health Association [APHA] (2020)7; Elliott (2020)8; National Association of School Psychologists (2020)9; Alvord, Siegel, and Harris (2019)10; Dalton (2020)11; California Association of School Psychologists [CASP] (2020)12; Louisiana School Psychological Association [SPA] (2020)13; Montana Association of School Psychologists [ASP] (2020)14; Association of School Psychologists [ASP] of Pennsylvania (2020)15; South Carolina Association of School Psychologists [ASP] (2020)16; Tennessee Association of School Psychologists [ASP] (2020)17; Wisconsin School Psychological Association [SPA] (2020)18; Alaska Department of Education and Early Development [DEED] (2020d)19; Alaska Department of Education and Early Development [DEED] (2020a)20; Alaska Department of Education and Early Development [DEED] (2020b)21; Alaska Department of Education and Early Development [DEED] (2020c)22; Idaho Special Education Support and Technical Assistance [SESTA] (2020)23; Louisiana Department of Education [DOE] (2020)24; Louisiana Department of Health [DOH] (2020)25; Louisiana State Board of Examiners of Psychologists [BEP] (2015)26; Maryland Board of Examiners of Psychology [BEP] (2018)27; Maryland Public Schools (2020)28; Mississippi Department of Education [DOE] (2020)29; Montana Office of Public Instruction [OPI] (2020)30; Nebraska Department of Education [DOE] (2020)31; Public Schools of North Carolina (2020)32; North Carolina Psychology Board (2020)33; Commonwealth of Pennsylvania (2010)34; Pennsylvania Department of Education [DOE] (n.d.)35; South Carolina BEP (2020)36; South Carolina Department of Education [DOE] (n.d.)37; Tennessee Department of Education [DOE] (2020)38; United States Department of Education (2020)39; United States Department of Veteran Affairs (2017)40; United States Department of Veterans Affairs (2018)41; Lum and Tambyah (2020)42; Verma (2020)43; Centers for Medicare and Medicaid Services (2020)44; Virginia Department of Education [DOE] (2020)45; Wyoming State Board of Psychology (2020)46; PAR (2020)47; Henke (2020)48; Walker et al. (2020)49; Wheldon (2020)50; WPS, 202051.

Table 4.

Information Dissemination Methods.

Dissemination Method
Type Webpage Document Toolkit Journal Email
Professional organization (n = 18) 8 6 4 2 3
Government agencies (n = 28) 11 17 6 1 2
Test publisher (n = 5) 4 1 1 0 1
Total recommendations 23 24 12 3 6
Open in a new tab

Note. Total number of entities = 51.

Results

What Recommendations Were Provided?

Tables 1 and 2 provide information on the types of recommendations that were provided.

Was There a Difference in Recommendations across Entities?

About half (53.8%) of professional organizations suggested caution when using adapted tele-assessments; whereas, over a third (38.5%) recommended that these types of assessments not be used at all. Over half (57.1%) of government agencies recommended that practitioners not use adapted tele-assessments and 42.9% recommended that they be used with caution. The majority (60%) of test publishers suggested caution when using their instruments, with fewer than half (40%) indicating no concerns. Test publishers never advised against the use of their products. Government agencies never advised the use of adapted tele-assessment without concerns.

How Were the Recommendations Disseminated?

Given that some entities provided recommendations through multiple methods, the percentages will not equal 100%. For professional organizations, 44.4% provided information on their Web sites compared to 39.3% of government agencies and 80% of test publishers. Tables 3 and 4 provide more specific information on other methods.

Discussion

At this time, no formal ethical codes exist specific to the use of adapted tele-assessment. The American Psychological Association’s (APA) (2017)Ethical Principles of Psychologists and Code of Conduct, Standard 9.02, provides some guidance related to adapted assessments in general. They emphasize that any adapted instrument should be used in a manner supported by research-based evidence and add the need to disclose any issues related to score reliability, test validity, or other potential limitations. In 2014, Luxton, Pruitt, and Osenback published “Best Practices for Remote Psychology Assessment via Telehealth Technologies.” In this article, the authors mention the need to maintain standardization when adapting existing measures for use in a tele-assessment, recommending to choose a different assessment method if standardization cannot be maintained. They specifically list in-person, Wechsler tests as poor candidates for tele-assessment adaptation.

Teleconferencing options have changed since this publication, as have the reasons for the need for adapted tele-assessment, so practitioners should depend on more current recommendations for how to move forward. To fill this gap, about four-fifths of the entities evaluated in this study offered some official statement on adapted tele-assessment; all of the test publishers had an official statement. In general, these recommendations fell in three categories: “do not use,” “use with caution,” or “use with no restrictions.” Unfortunately, the recommendations were inconsistent across and within entity type (i.e., professional organizations, governmental agencies, and test publishers). Government agencies were unanimous in their recommendation that adapted tele-assessment be conducted with some caution or not at all. Test publishers were unanimous in their recommendation that adapted tele-assessment with their instruments be conducted (either with or without caution).

One hypothesis for these findings is that use decisions are, in some part, made based on the financial stake held by each type of entity (Vitell, Singhapakdi, & Thomas, 2001). For example, test publishers are likely to lose money if no online equivalent to face-to-face testing is available for their products. Therefore, it would make sense that they would recommend adapting instruments for tele-assessment. Second to test publishers in terms of potential financial loss are assessment practitioners. If practitioners cannot provide assessments, they have the potential to lose some or all of their incomes. Possible financial loss may apply more to private practitioners; however, given that the only national, legal requirement for districts to hire school psychologists over other mental health practitioners involves testing (Individuals with Disabilities Education Improvement Act [IDEA], 2004), this could potentially apply to school-based practitioners as well. Although individual practitioners were not evaluated in this study, the professional organizations that represent these practitioners were included. These organizations must balance the financial/practical needs of their members with the ethical needs of their clients. Therefore, it would make sense that they would recommend using adapted tele-assessments but with caution.

Finally, governmental agencies may have more to lose from potential lawsuits resulting in any deviation from established policy and/or regulatory procedures, so it makes sense they would advise against using deviations from standard practice, such as adapted tele-assessment. However, without specific data regarding the rationale behind each recommendation, the reasons for these differences cannot be determined for certainty.

Another, more troubling finding from the evaluations of these recommendations is that there appears to be no systematic method of disseminating recommendations to professionals in a time of crisis. Table 3 shows that although many of the proffered recommendations were specific to the pandemic, many were recycled from previous, more generic policy statements. In addition, often there was no single source that individuals could use to find what they needed. Information came from many different sources, such as Web sites, congressional reports, emails, video recordings, blogs, journal articles, and stored documents. This is problematic, especially when the same entity may offer contradictory recommendations within their own publicly available materials (e.g., Alaska DEED). Given this, entities may need to reevaluate the information dissemination portions of their crisis management plans. If such a plan does not exist, then the current writers suggest that one needs to be forthcoming.

Call for Future Research

In 2017, NASP described tele-assessment as, “the least explored area of service in telehealth” (pg. 7). This still holds true in 2020. There is no current research on how many practitioners are using tele-assessment methods nor is there any into how practitioner and/or clients view adapted tele-assessment techniques. It is clear that additional study is needed, while social distancing restrictions are present.

In addition, if practitioners continue to use adapted tele-assessment methods post-COVID-19 closures, further equivalency studies are a necessity. The field of tele-assessment is relatively new (Brearly et al., 2017), but it does pre-date COVID-19. Barak was evaluating easily available tele-assessment tools as early as 1999 because psychologists have been looking for ways to reach individuals who live a considerable distance from generic and specialized services. Brearly et al. (2017) provided a meta-analysis on methods of providing neuropsychological tele-assessments. Cobb and Sharkey (2007) analyzed different technology modalities to assess individuals with complex motor disabilities. McCreery et al. (2019) investigated game-based technology as a direct method of assessing behaviors that had only previously been tested indirectly. Finally, the possible use of computers as translators (Karpińska, 2017) or as mediation devices for administering translated tests is an enticing concept (Farmer et al., 2020b).

Such studies must examine multiple aspects of equivalency, including: (a) inter-version correlation, (b) mean score differences, (c) score distribution equivalences (American Educational Research Association [AERA] et al., 2014), (d) sample demographic and setting equivalencies (Grosch et al., 2011; Henke, 2020; Hodge et al., 2019; Krach et al., 2020), and include (e) a large enough sample size (Cohen, 1988; Farmer et al., 2020a). Currently, no published studies adequately meet these criteria. In addition to psychometric equivalency, consequential equivalency/validity should also be evaluated. Consequential equivalency would examine if the score variability between administration formats contributes to different diagnostic conclusions (Matuszak & Piasecki, 2012; Ruskin et al., 1998).

Limitations

Unfortunately, there was not enough statistical power available to accurately run a chi-square analysis with this sample, so no statistical significance results are available. However, given the null sets provided by both test publishers and government agencies, it is possible that having a larger sample of groups would still result in violations in chi-square assumptions.

Also problematic is the in-flux nature of adapted tele-assessment recommendations. The current analysis provides a snapshot of how entities responded during the early months of the crisis. It is possible/probable that the recommendations evaluated in this study have already been replaced by new guidelines as of the print date for this article. Problems associated with changing recommendations are compounded by the fact that the field of psychology does not have a singular distribution method for crisis information.

One final concern about this study is that the recommendations here are specifically focused on assessments traditionally conducted by clinical and school psychologists. It is possible that these findings may differ for other telehealth areas (e.g., counseling and consultation) or other tele-assessment methods (e.g., career counseling and neuropsychological assessment).

Conclusion

Given the perceived rarity of the need for tele-assessment services, any push for adapted tele-assessment research was minimal prior to COVID-19. Tele-assessment has been a convenient way to provide assessments at reduced cost to individuals who may not have traditional access to psychological diagnostic options (National Association of School Psychologists, 2017; Luxton, Pruitt, & Osenback, 2014), including rural or homebound clients (Carlbring et al., 2007). Tele-assessment has also been an expedient way to provide services by specialized practitioners (e.g., bilingual psychologists or autism specialists) who are located some distance from clients. These were considered useful but inessential tools for practitioners, until the need naturally exploded when most direct assessments could no longer safely be administered in a face-to-face manner due to the global pandemic. The ensuing research gap has resulted in the widespread use of adapted tele-assessment, but the latter’s lack of standardization potentially renders invalid any results from such assessment.

At the time of this writing, COVID-19 is still considered a public crisis. Decisions by school districts and individual psychologists regarding adapted tele-assessment are ongoing and ever-changing. It is no easy task for practitioners to make decisions regarding tele-assessment, given that there is no singular directive providing a clear map for adapted tele-assessment practice. This lack of clear direction puts both practitioners and the general public at risk.

When practitioners base their practice on not using adapted tele-assessments, clients who need immediate services may not be identified in a timely manner (Individuals with Disabilities Education Improvement Act [IDEA], 2004). On the other hand, practitioners who follow guidance to conduct adapted tele-assessments with no restrictions may risk making inaccurate client diagnoses due to faulty assessment data (National Association of School Psychologists, 2017, 2020). These opposing needs apply undue stress to the mental health and special education systems. In the end, individual practitioners will have to follow their own understanding of professional ethics and best practice, as well as any legal mandates of their governing bodies, while the field awaits future and more consistent recommendations.

Given that entities provided different recommendations across different formats, the main findings from this study are mostly historical in nature. These findings indicate the need for psychologists to speak in a singular voice using an easily identifiable/discoverable platform when crises occur. It is clear that our field was not prepared for this type of crisis; we had no plan in place for what to do. More importantly, we did not have a plan in place to disseminate urgent/emergency information. The fields of psychology and school psychology need to expand their emergency plans to include: (1) unified standards for making decisions regarding adapted tele-assessments and (2) a unified dissemination plan for any crisis-generated recommendations going forward. Hopefully, this study will provide information to future leaders for use when developing such a plan.

Acknowledgments

The authors would like to acknowledge the work done by the Technology Intervention and Assessment in Schools (TIAS) at Florida State University (from redacted) as well as the work of the NASP: Distance Learning in Graduate Education subgroup.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

S. Kathleen Krach https://orcid.org/0000-0002-6853-379X

References

  1. Alaska Department of Education and Early Development [DEED] . (2020. a, February). Guidance for special education personnel [PDF file]. https://education.alaska.gov/Media/Default/static/covid/AK_SPED_Handbook.pdf [Google Scholar]
  2. Alaska Department of Education and Early Development [DEED] . (2020. b, March 18). Special education frequently asked questions [PDF file]. https://education.alaska.gov/Media/Default/static/covid/Virus_Related_FAQs_6.pdf [Google Scholar]
  3. Alaska Department of Education and Early Development [DEED] . (2020. c, March 18). COVID-19 considerations for special education [Webinar]. Webex. https://stateofalaska.webex.com/recordingservice/sites/stateofalaska/recording/play/9518e7e692ea4eee83c62b49e3c8bb57 [Google Scholar]
  4. Alaska Department of Education and Early Development [DEED] . (2020. d, April 3). Alaska special education COVID-19 district guidance [PDF file]. https://education.alaska.gov/news/COVID-19/April03/AK_SPED_Guidance.pdf [Google Scholar]
  5. Alvord M., Siegel A., Harris E. (2019, November 9). A practical guide to providing telepsychology with minimal risks [Seminar]. Washington, DC: National Register of Health Service Psychologists [NRHSP]. https://www.nationalregister.org/npc-telepsych-video/ [Google Scholar]
  6. American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME] . (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. [Google Scholar]
  7. American Psychiatric Association [ApA] . (2020, May 27). Practice guidance for COVID-19 .https://www.psychiatry.org/psychiatrists/covid-19-coronavirus/practice-guidance-for-covid-19 [Google Scholar]
  8. American Psychiatric Association [ApA], & American Telemedicine Association [ATA] . (2018, April). Best practices in videoconferencing-based telemental health [PDF file]. https://www.psychiatry.org/File%20Library/Psychiatrists/Practice/Telepsychiatry/APA-ATA-Best-Practices-in-Videoconferencing-Based-Telemental-Health.pdf [DOI] [PubMed] [Google Scholar]
  9. American Psychological Association . (2017). Ethical principles of psychologists and code of conduct (2002, amended effective June 1, 2010, and January 1, 2017). https://www.apa.org/ethics/code/ [Google Scholar]
  10. American Psychological Association Committee on Professional Standards and Committee on Psychological Tests and Assessment . (1986). Guidelines for computer-based tests and interpretations. Washington, DC: Author. [Google Scholar]
  11. American Public Health Association [APHA] . (2020). COVID-19: What is APHA Doing? https://www.apha.org/covid19 [Google Scholar]
  12. Association of School Psychologists [ASP] of Pennsylvania . (2020, April 7). RE: Special education evaluations & virtual assessment [PDF file]. https://www.aspponline.org/docs/ASPP_COVID_Virtual_testing_letter_to_PDE.pdf [Google Scholar]
  13. Barak A. (1999). Psychological applications on the internet: A discipline on the threshold of a new millennium. Applied and Preventive Psychology, 8(4), 231-245. doi: 10.1016/S0962-1849(05)80038-1 [DOI] [Google Scholar]
  14. Blumenthal-Barby J. S. (2014). Psychiatry's new manual (DSM-5): Ethical and conceptual dimensions: Table 1. Journal of Medical Ethics, 40(8), 531-536. doi: 10.1136/medethics-2013-101468 [DOI] [PubMed] [Google Scholar]
  15. Brearly T. W., Shura R. D., Martindale S. L., Lazowski R. A., Luxton D. D., Shenal B. V., Rowland J. A. (2017). Neuropsychological test administration by videoconference: A systematic review and meta-analysis. Neuropsychology Review, 27(2), 174-186. doi: 10.1007/s11065-017-9349-1 [DOI] [PubMed] [Google Scholar]
  16. California Association of School Psychologists [CASP] . (2020, April 27). Position paper: Mandated special education assessment during the COVID-19 shutdown [PDF file].https://casponline.org/pdfs/position-papers/CASP%20Covid-19%20Assessment%20Position%20Paper.pdf [Google Scholar]
  17. Carlbring P., Brunt S., Bohman S., Austin D., Richards J., Öst L.-G., Andersson G. (2007). Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior, 23(3), 1421-1434. doi: 10.1016/j.chb.2005.05.002 [DOI] [Google Scholar]
  18. Cayton T., Wahlstrom D., Daniel M. (2012). The initial digital adaptation of the WAIS-IV. In Lichtenberger E., Kaufman A. (Eds.), Essentials of WAIS-IV assessment (pp. 389-427). Hoboken, NJ: John Wiley. [Google Scholar]
  19. Centers for Medicare and Medicaid Services . (2020, March 17). Medicare telemedicine health care provider fact sheet: Medicare coverage and payment of virtual services [PDF file].https://www.samhsa.gov/sites/default/files/medicare-telemedicine-health-care-fact-sheet.pdf [Google Scholar]
  20. Cobb S. V. G., Sharkey P. M. (2007). A decade of research and development in disability, virtual reality and associated technologies: Review of ICDVRAT 1996-2006. International Journal of Virtual Reality, 6(2), 51-68. [Google Scholar]
  21. Cohen J. (1988). Statistical power analysis for the behavioral sciences .Hillsdale, NJ: Lawrence Earlbaum Associates. [Google Scholar]
  22. Commonwealth of Pennsylvania . (2010). PA Code chapter 41: State board of psychology. http://www.pacodeandbulletin.gov/secure/pacode/data/049/chapter41/049_0041.pdf [Google Scholar]
  23. Dalton L. (2020, April 20). 50-state emergency teletherapy practice rules survey for counselors, MFTs, psychologists, and clinical social workers. Washington, DC: National Register of Health Service Psychologists (NRHSP). https://personcenteredtech.com/2020/04/20/50-state-emergency-teletherapy-practice-rules-survey-for-counselors-mfts-psychologists-and-clinical-social-workers/?goal=0_e9b2dcace3-8676048392-125900729&mc_cid=8676048392&mc_eid=1e185787fc [Google Scholar]
  24. Daniel M. H. (2012. a). Equivalence of Q-interactive administered cognitive tasks: CVLT®–II and selected D-KEFS® subtests. (Q-interactive technical report 3). Bloomington, MN: Pearson. [Google Scholar]
  25. Daniel M. H. (2012. b). Equivalence of Q-interactive administered cognitive tasks: WAIS–IV. (Q-interactive technical report 1). Bloomington, MN: Pearson. [Google Scholar]
  26. Daniel M. H. (2013. a). Equivalence of Q-interactive and paper scoring of academic tasks: Selected WIAT®–III subtests. (Q-interactive technical report 5). Bloomington, MN: Pearson. [Google Scholar]
  27. Daniel M. H. (2013. b). Equivalence of Q-interactive and paper administrations of cognitive tasks: Selected NEPSY®-II and CMS subtests. (Q-interactive technical report 4).Bloomington, MN: Pearson. [Google Scholar]
  28. Daniel M. H. (2013. c). Equivalence of Q-interactive and paper administration of WMS®-IV cognitive tasks. (Q-interactive technical report 6).Bloomington, MN: Pearson. [Google Scholar]
  29. Daniel M. H., Wahlstrom D., Zhou X. (2014. a). Equivalence of Q-interactive® and paper administrations of language tasks: Selected CELF®–5 tests. (Q-interactive technical report 7). Bloomington, MN: Pearson. [Google Scholar]
  30. Daniel M. H., Wahstrom D, Zang O. (2014. b). Q–interactive equivalence of Q-interactive® and paper administrations of cognitive tasks: WISC®–V. (Q-Interactive Technical Report 8). Bloomington, MN: Pearson. [Google Scholar]
  31. Elliott V. L. (2020). Telehealth and telemedicine: Frequently asked questions (No. R46239) .Washington, DC: Congressional Research Service. https://crsreports.congress.gov/product/pdf/R/R46239 [Google Scholar]
  32. Farmer R. L. McGill R. J. Dombrowski S. C. Benson N. F. Smith-Kellen S. Lockwood A. B., … Stinnett T. A. (2020. a). Conducting psychoeducational assessments during the COVID-19 crisis: The danger of good intentions. Contemporary School Psychology. Advance online publication. doi: 10.1007/s40688-020-00293-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Farmer R. L., McGill R., Dombrowski S. C., McClain M., Harris B., Lockwood A.B., Stinnett T. (2020. b). Teleassessment with children and adolescents during the coronavirus (COVID-19) pandemic and beyond: Practice and policy implications. doi: 10.1037/pro0000349 [DOI] [Google Scholar]
  34. Grosch M. C., Gottlieb M. C., Cullum C. M. (2011). Initial practice recommendations for teleneuropsychology. The Clinical Neuropsychologist, 25(7), 1119-1133. doi: 10.1080/13854046.2011.609840 [DOI] [PubMed] [Google Scholar]
  35. Hafften A.V. (2016, February). Telepsychiatry practice guidelines .Washington, DC: American Psychiatric Association [Video]. Vimeo. https://www.psychiatry.org/psychiatrists/practice/telepsychiatry/toolkit/practice-guidelines [Google Scholar]
  36. Henke J. (2020, March 30). Guidelines for administration via telepractice .Saddle River, NJ: Pearson [Webinar]. [Google Scholar]
  37. Hilty D. M., Shoemaker E. Z., Myers K., Snowdy C. E., Yellowlees P. M., Yager J. (2016). Need for and steps toward a clinical guideline for the telemental healthcare of children and adolescents. Journal of Child and Adolescent Psychopharmacology, 26(3), 283. doi: 10.1089/cap.2015.0129 [DOI] [PubMed] [Google Scholar]
  38. Hodge M. A. Sutherland R. Jeng K. Bale G. Batta P. Cambridge A., …, Silove N.. 2019). Agreement between telehealth and face-to-face assessment of intellectual ability in children with specific learning disorder. Journal of Telemedicine and Telecare, 25(7), 431-437. doi: 10.1177/1357633X18776095 [DOI] [PubMed] [Google Scholar]
  39. Idaho Special Education Support and Technical Assistance [SESTA] . (2020, April 8). COVID-19 virtual supports/teletherapy quick guide [PDF file]. https://idahotc.com/Portals/0/Documents/Virtual%20Supports_Teletherapy%20Quick%20Guide.pdf?ver=2020-04-08-094619-967 [Google Scholar]
  40. Individuals with Disabilities Education Improvement Act [IDEA] . (2004). 20 U.S.C. § 1400.
  41. Joint Task Force for the Development of Telepsychology Guidelines for Psychologists . (2013). Guidelines for the practice of telepsychology. The American psychologist, 68(9), 791-800. doi: 10.1037/a0035001 [DOI] [PubMed] [Google Scholar]
  42. Jordan S. E., Shearer E. M. (2019). An exploration of supervision delivered via clinical video telehealth (CVT). Training and Education in Professional Psychology, 13(4), 323-330. doi: 10.1037/tep0000245 [DOI] [Google Scholar]
  43. Karpińska P. (2017). Computer aided translation – possibilities, limitations and changes in the field of professional translation. The Journal of Education, Culture, and Society, 8(2), 133-142. [Google Scholar]
  44. Kline P. (2015). A handbook of test construction (psychology revivals): Introduction to psychometric design .Abingdon, UK: Routledge. [Google Scholar]
  45. Krach S. K., McCreery M. P., Dennis L., Guerard J., Harris E. L. (2020). Independent evaluation of Q‐Interactive: A paper equivalency comparison using the PPVT‐4 with preschoolers. Psychology in the Schools, 57, 17-30. doi: 10.1002/pits.22325 [DOI] [Google Scholar]
  46. Krach S. K., Sattler J. M. (2018). Computer-based administration, scoring, and report writing. In Sattler J. M. (Eds.), Assessment of children: Cognitive foundations (6th ed., pp. 47-50). San Diego, CA: Jerome M. Sattler, Publisher, Inc. [Google Scholar]
  47. Louisiana Department of Education [DOE] . (2020, April 29). Timelines and documentation during extended school closures for students with disabilities [PDF file]. https://www.louisianabelieves.com/docs/default-source/covid-19-resources/idea-timelines-and-documentation-during-extended-school-closures.pdf?sfvrsn=390c9b1f_4 [Google Scholar]
  48. Louisiana Department of Health [DOH] . (2020, March 25). COVID-19 telemedicine/telehealth facilitation by licensed mental health practitioners [PDF file]. http://ldh.la.gov/assets/docs/BayouHealth/HealthPlanAdvisories/2020/HPA20-8.pdf [Google Scholar]
  49. Louisiana School Psychology Association [SPA] . (2020, April 10). Position statement: School psychological service delivery during COVID-19 pandemic [Google Document]. Retrieved 06/2020 fromhttps://docs.google.com/document/d/1dJU27gcUJ-iOAPzk8in0nYk6BgMqYeltFoZ0mYvJXHw/edit [Google Scholar]
  50. Louisiana State Board of Examiners of Psychologists [BEP] . (2015, January 1). Louisiana telepsychology guidelines [PDF file]. http://www.lsbep.org/wp-content/uploads/Final-Telepsych-Guidelines-1-15-6.0.pdf [Google Scholar]
  51. Lum L., Tambyah P. (2020). Outbreak of COVID-19 - an urgent need for good science to silence our fears? Singapore Medical Journal, 61(2), 55-57. doi: 10.11622/smedj.2020018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Luxton D. D., Pruitt L. D., Osenbach J. E. (2014). Best practices for remote psychological assessment via telehealth technologies. Professional Psychology: Research and Practice, 45(1), 27-35. doi: 10.1037/a0034547 [DOI] [Google Scholar]
  53. McCreery M. P., Krach S. K., Bacos C. A., Laferriere J. R., Head D. L. (2019). Can video games be used as a stealth assessment of aggression? International Journal of Gaming and Computer-Mediated Simulations, 11, 40-49. doi: 10.4018/IJGCMS.20190401 [DOI] [Google Scholar]
  54. McHugh M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143-149. doi: 10.11613/bm.2013.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Maryland Board of Examiners of Psychology [BEP] . (2018, January 19). 10.36.10 Telepsychology. Baltimore, MD: Maryland Department of Health. https://health.maryland.gov/regs/Pages/10-36-10-Telepsychology.aspx [Google Scholar]
  56. Maryland Public Schools . (2020, April). COVID-19 continuity of learning for Maryland Public Schools: Special education FAQ [PDF file]. http://marylandpublicschools.org/newsroom/Documents/COVID/FAQ/Sp_Ed_FAQ.pdf [Google Scholar]
  57. Matuszak J., Piasecki M. (2012). Inter-rater reliability in psychiatric diagnosis: Collateral data improves the reliability of diagnoses. Psychiatric Times, 29(10), 12-13. [Google Scholar]
  58. Mississippi Department of Education [DOE] . (2020). Updates and information in response to COVID-19 (Coronavirus) .Retrieved June 8, 2020, fromhttps://www.mdek12.org/COVID19 [Google Scholar]
  59. Montana Association of School Psychologists [ASP] . (2020, April 9). RE: Virtual service delivery during COVID-19 closures. https://masponline.us/wp-content/uploads/Virtual-Service-Delivery-Letter.docx [Google Scholar]
  60. Montana Office of Public Instruction [OPI] . (2020, April 20). Re-opening Montana's schools 2020 COVID-19 special education information. https://opi.mt.gov/Educators/Teaching-Learning/Special-Education/COVID-19-Special-Education-Information#9384510434-evaluation [Google Scholar]
  61. Moore A. L., Miller T. (2016). Gibson test of cognitive skills – version 2: Digital and interactive test. Technical manual for centers .Colorado Springs, CO: Gibson Institute of Cognitive Research. [Google Scholar]
  62. National Association of School Psychologists . (2017, September). Considerations for delivery of school psychological telehealth services [Brief]. https://www.nasponline.org/x39099.xml [Google Scholar]
  63. National Association of School Psychologists . (2020). Telehealth: Virtual Service Delivery Updated Recommendations [PDF file]. Retrieved 05/2020 fromhttps://www.nasponline.org/resources-and-publications/resources-and-podcasts/covid-19-resource-center/special-education-resources/telehealth-virtual-service-delivery-updated-recommendations [Google Scholar]
  64. Nebraska Department of Education [DOE] . (2020). Q and A for providing special education and early intervention during coronavirus school closure [PDF file]. https://cdn.education.ne.gov/wp-content/uploads/2020/03/NDE-QA-Nebraska-Office-of-Special-Education.pdf [Google Scholar]
  65. North Carolina Psychology Board . (2020). Frequently asked questions regarding the coronavirus (COVID-19) [PDF file]. Retrieved 06/2020 fromhttp://www.ncpsychologyboard.org/covid-19-faq/ [Google Scholar]
  66. PAR . (2020). Partnering with you: Using telehealth for psychological assessment. Retrieved June 5, 2020, fromhttps://www.parinc.com/Using-PAR-digital-assessments-during-the-COVID-19-crisis [Google Scholar]
  67. Pennsylvania Department of Education [DOE] . (n.d.). Special education and preschool early intervention evaluations & virtual assessment guidance .https://www.education.pa.gov/K-12/Special%20Education/FAQContact/Pages/EIEvaluationsFAQ.aspx [Google Scholar]
  68. Perle J. G., Nierenberg B. (2013). How psychological telehealth can alleviate society’s mental health burden: A literature review. Journal of Technology in Human Services, 31(1), 22-41. doi: 10.1080/15228835.2012.760332 [DOI] [Google Scholar]
  69. Public School of North Carolina . (2020, March 15). COVID-19 EC director’s FAQ [PDF file].https://files.nc.gov/dpi/documents/newsroom/covid-19/ec/faq.covid_.update.3.15.20.pdf [Google Scholar]
  70. Ruskin P. E., Reed S., Kumar R., Kling M. A., Siegel E., Rosen M., Hauser P. (1998). Reliability and acceptability of psychiatric diagnosis via telecommunication and audiovisual technology. Psychiatric Services, 49(8), 1086-1088. doi: 10.1176/ps.49.8.1086 [DOI] [PubMed] [Google Scholar]
  71. Sattler J. M. (2018). A primer on statistics and psychometrics. In Sattler J. M. (Ed.), Assessment of children: Cognitive foundations (6th ed., pp. 91-136). San Diego, CA: Jerome M. Sattler, Publisher, Inc. [Google Scholar]
  72. Sharp J., Mcfadden D., Morera C. C. (2020, April 29). Considerations and concerns of remote assessment. Lultz, FL: Par, Inc. [Podcast]. https://hwcdn.libsyn.com/p/d/d/d/ddd8ea32fe1db489/PARinc.mp3?c_id=71445584&cs_id=71445584&expiration=1593024816&hwt=78120cd409d8c13d02eb187388b5be00 [Google Scholar]
  73. South Carolina Association of School Psychologists [ASP] . (2020, May 8). RE: Special education evaluations during COVID-19 [PDF file]. http://scaspweb.org/resources/Documents/SCASP%20Letter%20for%20COVID-19.pdf [Google Scholar]
  74. South Carolina Board of Examiners in Psychology [BEP] . (2020, March 11th). Telehealth statement [PDF file]. https://llr.sc.gov/psych/Forms/PsychologyTelehealthStatement.pdf [Google Scholar]
  75. South Carolina Department of Education [DOE] . (n.d.). Frequently asked questions about COVID-19 closures .Retrieved 06/2020 fromhttps://ed.sc.gov/districts-schools/special-education-services/information-about-covid-19-coronavirus/frequently-asked-questions-about-covid-19-closures/#Initial%20evaluations%20and%20reevaluations [Google Scholar]
  76. Tennessee Association of School Psychologists [ASP] . (2020). School psychologist evaluation checklist during COVID-19 closures .https://tasponline.org/resources/Documents/TASP-School%20Psych%20Eval%20Checklist,%20Covid-19.docx [Google Scholar]
  77. Tennessee Department of Education [DOE] . (2020, March 27). School closures and special education: Guidance on services to students with disabilities [PDF file]. https://www.tn.gov/content/dam/tn/education/health-&-safety/SPED%20Guidance%20COVID_TN%20Final.pdf [Google Scholar]
  78. United States Department of Education . (2020, March). Fact sheet: Addressing the risk of COVID-19 in schools while protecting the civil rights of students [PDF file].https://www2.ed.gov/about/offices/list/ocr/docs/ocr-coronavirus-fact-sheet.pdf [Google Scholar]
  79. United States Department of Veteran Affairs . (2017). VA Telehealth Services: Fact Sheet [PDF file].https://www.va.gov/anywheretoanywhere/docs/Telehealth_Services_factsheet.PDF [Google Scholar]
  80. United States Department of Veterans Affairs . (2018, May 11). VA expands telehealth by allowing health care providers to treat patients across state lines. https://www.va.gov/opa/pressrel/pressrelease.cfm?id=4054 [Google Scholar]
  81. United States Social Security Administration [SSA] . (2020). Social security and Coronavirus disease (COVID-19). Retrieved 05/2020 fromhttps://www.ssa.gov/coronavirus/ [Google Scholar]
  82. Verma S. (2020, April 2). Medicare and coronavirus: What you need to know. Washington, DC: U.S. Social Security Administration. https://blog.ssa.gov/medicare-and-coronavirus-what-you-need-to-know/comment-page-3/ [Google Scholar]
  83. Virginia Department of Education [DOE] . (2020, May 20). Special education and student services (SESS) frequently asked questions .Retrieved June 8, 2020, fromhttps://www.doe.virginia.gov/support/health_medical/office/covid-19-sess-faq.shtml [Google Scholar]
  84. Vitell S. J., Singhapakdi A., Thomas J. (2001). Consumer ethics: An application and empirical testing of the Hunt‐Vitell theory of ethics. Journal of Consumer Marketing, 18(2), 153-178. doi: 10.1108/07363760110386018 [DOI] [Google Scholar]
  85. Walker K. E., Taylor S., Wright J., Johnsrud N. (2020, May 5). Tele-Assessment: What you Need to know [webinar]. Rolling Meadows, IL: Presence Learning in Coordination with Riverside Publishing. https://www.presencelearning.com/sped-ahead-webinar/tele-assessment-what-you-need-to-know/ [Google Scholar]
  86. Wheldon H. (2020). Message from our CEO. https://info.mhs.com/covid-19-update-mhs [Google Scholar]
  87. Wisconsin School Psychologists Association [SPA] . (2020, May 18). WSPA response to extended school closure and evaluation timelines due to COVID-19 .https://www.wspaonline.net/resources/Documents/WSPA%20COVID-19_revised.pdf [Google Scholar]
  88. Wosik J. Fudim M. Cameron B. Gellad Z. F. Cho A. Phinney D., ... Tcheng J. (2020). Telehealth transformation: COVID-19 and the rise of virtual care. Journal of the American Medical Informatics Association, 27(6), 957-962. doi: 10.1093/jamia/ocaa067 [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. WPS . (2020). Statement on tele-assessment .Torrance, CA: Author. https://pages.wpspublish.com/en/wps-tele-assessment-statement [Google Scholar]
  90. Wright A. J. (2018. a). Equivalence of remote, online administration and traditional, face-to-face administration of the reynolds intellectual assessment scales-second edition (Online white paper). San Francisco, CA: Presence Learning. [Google Scholar]
  91. Wright A. J. (2018. b). Equivalence of remote, online administration and traditional, face-to-face administration of Woodcock-Johnson IV cognitive and achievement tests. Archives of Assessment Psychology, 8(1), 23-35. [Google Scholar]
  92. Wright A. J. (2020). Equivalence of remote, digital administration and traditional, in-person administration of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V). Psychological Assessment. Advance online publication. doi: 10.1037/pas0000939 [DOI] [PubMed] [Google Scholar]
  93. Wright A. J., Mihura J. L., Pade H., McCord D. M. (2020, April 3). Guidance on psychological tele-assessment during the COVID-19 crisis. Washington, DC: American Psychological Association. https://www.apaservices.org/practice/reimbursement/health-codes/testing/tele-assessment-covid-19 [Google Scholar]
  94. Wyoming State Board of Psychology . (2020, May 1). Guidance regarding practice of telepsychology by psychologists [PDF file]. https://drive.google.com/file/d/1izlDThvOxTnnBcoUi3v7SgItclZT4RBd/view [Google Scholar]

Articles from Journal of Psychoeducational Assessment are provided here courtesy of SAGE Publications

RESOURCES