Skip to main content
PMC home page
Wiley - PMC COVID-19 Collection logoLink to Wiley - PMC COVID-19 Collection
. 2021 May 24;77(10):2405–2423. doi: 10.1002/jclp.23173

Examining models of psychologists' telepsychology use during the COVID‐19 pandemic: A national cross‐sectional study

Grace B McKee 1,2, Bradford S Pierce 2, Emily K Donovan 2, Paul B Perrin 2,3,✉
PMCID: PMC8242572  PMID: 34028018

Abstract

Objective

The aim of this study is to apply the theory of reasoned action (TRA) and technology acceptance model (TAM) to psychologists' telepsychology use during the COVID‐19 pandemic.

Methods

A sample of 2619 US‐licensed psychologists completed a survey assessing telepsychology use and aspects of both models in May 2020.

Results

Cross‐sectional TRA and TAM path models evidenced excellent fit in explaining psychologists' telepsychology use. The TRA indicated that psychologists' attitudes concerning telepsychology and subjective norms were associated with intentions to use telepsychology, which related to percentage of clinical work performed via telepsychology. The TAM showed that perceived usefulness of telepsychology and perceived ease of use were associated with attitudes toward telepsychology. Perceived usefulness was associated with psychologists' intention to use telepsychology, as was perceived ease of use.

Conclusion

Efforts to facilitate telepsychology provision during the pandemic and broadly may benefit from trainings and campaigns to address attitudes toward telepsychology, subjective norms, and perceived ease of use.

Keywords: technology acceptance model, telehealth, telepsychology, theory of reasoned action

In March 2020, the World Health Organization characterized a novel coronavirus (COVID‐19) as a pandemic (World Health Organization, 2020) following rapid spread worldwide. As a result, many schools, businesses, institutions, and governments in the United States transitioned to online models (Adalja et al., 2020). One of the most notable changes was in healthcare delivery, with elective procedures being postponed and routine patient visits being postponed or conducted virtually as per Centers for Disease Control and Prevention (2020) recommendations. Telehealth service provision rapidly increased across the United States during the first months of the pandemic (Perrin, Pierce, et al., 2020), as did use of telepsychology (Pierce, Perrin, Tyler, et al., 2020). Broadly, telepsychology is defined as the provision of psychological services via technology (American Psychological Association, 2015). An initial study during the pandemic found a 12‐fold increase in use of telepsychology (i.e., real‐time audio [e.g., telephone] and/or video conferencing technology to provide psychological services) in the United States, with psychologists performing 85.5% of their clinical work via telepsychology during the COVID‐19 pandemic, compared with 7.1% previously; 96.5% of psychologists reported using telepsychology in their practice during the pandemic (Pierce, Perrin, Tyler, et al., 2020). This shift to telepsychology has been documented in a variety of contexts, including integrated behavioral health care in primary care clinics (Perrin, Pierce, et al., 2020), parent–child interaction therapy for children with behavior problems (Gurwitch et al., 2020), and psychiatry (Uscher‐Pines et al., 2020). In a May 2020 survey, psychologists reported that they anticipated conducting about 35% of post‐pandemic clinical services via telepsychology, which represents a fivefold increase from pre‐pandemic use (Pierce, Perrin, Tyler, et al., 2020). Although this projection may change over time depending on a number of factors, such as length of the pandemic, continuance of policies supportive of telepsychology, and so on, this estimate does suggest that the shift appears to be built to last. In contrast, before the COVID‐19 pandemic, only about 21% of US psychologists endorsed telepsychology use (Pierce et al., 2020a); however, pre‐pandemic telepsychology use was more prevalent in the Veterans Health Administration (VHA), where telepsychology infrastructure, policies, and training have been in place for years (Godleski et al., 2012).

These drastic changes in telepsychology use during the pandemic can be attributed to a number of factors. For example, to allow for continuity of care while adhering to social distancing guidelines, government agencies and other public and private institutions (e.g., Johns Hopkins [Nitkin, 2020], Mayo Clinic [Stiepan, 2020], and the VHA [2020]), companies, and systems made key policy changes to ameliorate deterrents to adoption of telemedicine and telepsychology in the United States. Such changes included allowing certain applications like Skype and FaceTime to be used for telehealth and telepsychology by loosening security and privacy requirements of the Health Insurance Portability and Accountability Act (U.S. Department of Health and Human Services' Office for Civil Rights, 2020). Other changes included some states allowing psychologists not licensed within their state to provide services under a declaration of emergency (California Board of Psychology, 2020; State of Arizona Board of Psychologist Examiners, 2020; Texas State Board of Examiners of Psychologists, 2020) and telehealth and telepsychology visits being reimbursed at the same rate as in‐person visits (U.S. Centers for Medicare & Medicaid Services, 2020). Training and supervision provisions were also made, including allowance for telepsychology training hours to be counted the same as in‐person hours (Association of Psychology Postdoctoral and Internship Centers, 2020), adoption of telesupervision by psychology training programs (e.g., Perrin, Rybarczyk, et al., 2020), and establishment of training programs and resource banks for both psychologists (e.g., American Psychological Association, 2020; Australian Psychological Society, 2020; European Federation of Psychologists' Associations, 2020; Martin et al., 2020; Telebehavioral Health Institute, 2020) and consumers (Australian Psychological Society, 2020; European Federation of Psychologists' Associations, 2020).

Although these policy changes paved the way for successful implementation of telehealth and telepsychology, a new wave of challenges came with increased traffic on the internet and various video conferencing and streaming services. This increased traffic resulted in greater delays and diminished network speeds (Ookla, 2020) as well as network instability and very slow data transfer rates (Tahir, 2020), all of which made adopting and accessing telehealth and telepsychology platforms more difficult. Prior research has detailed deterrents of telepsychology uptake before the COVID‐19 pandemic, including concerns for psychologists surrounding practice (e.g., insufficient training, handling client safety/crisis, insufficient demand, efficacy) and environmental/policy concerns (e.g., privacy, legality, ethics, reimbursement, prohibitive organizational policies/culture; Pierce et al., 2020c).

Moreover, several theoretical models of behavior change may provide a useful framework for explaining the adoption of telepsychology among psychologists. The theory of reasoned action (TRA) outlines a framework for understanding the relationships among attitudes toward behavior, perceived social pressure (i.e., subjective norms), behavioral intentions, and behavior (Fishbein & Ajzen, 1977). The TRA posits that attitudes and norms precede intentions, which precede behavior (Fishbein & Ajzen, 1977). Derived from the TRA, the technology acceptance model (TAM) was developed to provide a foundation for understanding factors associated with an individual's willingness to adopt computer technologies or why technology has not been accepted in a given population (Davis et al., 1989). Perceived usefulness (i.e., expectation of technology increasing/decreasing performance) and perceived ease of use (i.e., expectation of effort level required and self‐efficacy surrounding use of the technology) are key variables related to computer technology acceptance (Davis et al., 1989). Both theories posit that attitude toward technology influences behavioral intention to use it, but the TAM does not measure subjective norms and adds perceived usefulness as an influence on behavioral intention. In other words, the theories diverge in important ways with regard to which variables might kick off the chain of events leading to attitudes, intention, and actual behavior.

The TRA and TAM previously have been used to examine psychologists' openness to using telepsychology before the COVID‐19 pandemic (Pierce et al., 2020b). Using data from 1791 psychologists currently practicing psychotherapy, Pierce et al. (2020b) developed and analyzed path models derived from the TRA and TAM to assess hypothesized associations among attitudes toward telepsychology, subjective norms, intention to use telepsychology, perceived ease of use, perceived usefulness, and current use of telepsychology. Both the TRA and the TAM models poorly fit the data, and the authors hypothesized that this could be due, in part, to the dichotomous (yes/no) outcome used for use of telepsychology. A third model was developed, which combined the two theoretical models and resulted in adequate to good fit. Within this combined model, variables associated with use of telepsychology by psychologists were perceived attitudes/expectations of others (i.e., subjective norms) that were associated with psychologists' perceptions of telepsychology's usefulness and ease of use. Furthermore, as this study was conducted before the COVID‐19 pandemic, only 20% of participants reported telepsychology use. It is unclear whether these associations would be the same during the COVID‐19 “telemedicine revolution” (Perrin, Pierce, et al., 2020), when many psychologists transitioned some or all of their clinical services to telepsychology. Moreover, given that mental health providers tend to report more positive attitudes toward telepsychology after use (Brooks et al., 2012) and that perceived ease of use also appears to increase (Gibson et al., 2011), it may be that associations among TAM and TRA variables may differ from these previous patterns in the first few months of the pandemic, when telepsychology was being widely implemented.

There is potential for telepsychology to aid psychologists in increasing the reach of mental health services during the COVID‐19 pandemic and beyond, given the profound rates of mental health issues identified on a global scale (Alzueta et al., 2021). Furthermore, telepsychology allows psychologists to maintain the treatment of their clients while following social distancing recommendations. Although telepsychology delivered via telephone or video conferencing technology has been proven effective and non‐inferior to in‐person therapy for a wide range of disorders (Varker et al., 2019), many psychologists did not offer these services before the COVID‐19 pandemic. The purpose of the present study was to test the applicability of the TRA, TAM, and previously combined model (Pierce et al., 2020b) to psychologists' adoption of telepsychology in the context of the COVID‐19 pandemic. While the TAM, TRA, and combined model have been used to examine the uptake of new technology in the context of telehealth broadly (Hu et al., 1999) and telepsychology specifically (Monthuy‐Blanc et al., 2013; Pierce et al., 2020b), the COVID‐19 pandemic may have profoundly impacted the landscape of subjective norms and attitudes contributing to behavior change, thus necessitating a new examination at this unique point in the history of mental healthcare delivery.

Based on previous investigations of the TRA, we expected the following:

  • 1.

    Attitudes toward telepsychology would be positively associated with intention to use telepsychology.

  • 2.

    Subjective norms about the use of telepsychology would be positively associated with intention to use telepsychology.

  • 3.

    Intention to use telepsychology would be positively associated with actual use of telepsychology.

Based on previous studies using the TAM to examine behavior change, the following aspects were hypothesized:

  • 1.

    Perceived usefulness of telepsychology would be positively associated with attitude toward telepsychology.

  • 2.

    Perceived ease of use of telepsychology would be positively associated with attitude toward telepsychology.

  • 3.

    Attitude toward telepsychology would be positively associated with intention to use telepsychology.

  • 4.

    Intention to use telepsychology would be positively associated with actual use of telepsychology.

Based on the previous investigation of a combined model to understand telepsychology (Pierce, Perrin, Tyler, et al., 2020), the following aspects were hypothesized:

  • 1.

    Subjective norms around the use of telepsychology would be positively associated with perceived usefulness of telepsychology.

  • 2.

    Subjective norms would be positively associated with perceived ease of use of telepsychology.

  • 3.

    Perceived usefulness would be positively associated with current use of telepsychology.

  • 4.

    Perceived ease of use would be positively associated with current use of telepsychology.

In addition, it was expected that statistical mediations would operate and that all three models would demonstrate good fit.

1. METHODS

1.1. Participants

The data used in this study are a secondary analysis of participant responses obtained from a larger study of environmental and demographic factors relating to psychologists' use of telepsychology during the COVID‐19 pandemic (Pierce, Perrin, Tyler, et al., 2020). A recruitment list of names and email addresses of potential participants was generated using publicly available information on websites for mental health clinics and counseling centers, professional listservs, and professional organization directories. Criteria for eligibility were that participants must be licensed to practice as a psychologist in the United States at the time of participation, age 18 or older, and actively practicing as a psychologist in the United States at the time of participation. A recruitment flow diagram is shown in Figure 1. Emailed invitations were sent to 27,324 potential participants between May 11, 2020, and May 25, 2020. Among that number, 14.3% (n = 3897) of invitations returned as undeliverable, whereas 15.8% (n = 3694) of delivered emails resulted in individuals at least accessing the study's online information sheet. Among these, 184 individuals chose not to consent to participate in the survey or abandoned the survey after reviewing the information sheet. Another 541 began the survey but failed to complete it. In addition, 198 failed to meet eligibility criteria based on their responses to the initial screener items. Among the remainder, 127 were excluded from analysis after refusing to provide demographic data (e.g., age, years in practice). Finally, data were excluded for 25 participants who indicated they had started practicing as a licensed psychologist before age 18 or failed to provide a meaningful answer to the final validity check question, “In one sentence or less, what do you think the purpose of this study was?” This resulted in an 11.18% (n = 2619) response rate of eligible individuals who fully participated in the study. This aligns with research examining response rates using similar recruitment strategies (Petrovčič et al., 2016).

Figure 1.

Figure 1

Open in a new tab

Study recruitment flow diagram

Women comprised 64.2% of the sample and there was a mean age of 57.29 years, with an average of 24.22 years in practice. Full sample demographics appear in Table 1. These sample characteristics were similar to APA's 2017 membership demographics (American Psychological Association, 2018), although the geographical region was not surveyed and therefore could not be assessed for representativeness. In addition, respondents were presented with 20 theoretical therapeutic approaches and asked to select all approaches that applied to them; cognitive behavioral therapy (70.4%), psychodynamic (38.2%), interpersonal (31.1%), family systems (23.9%), and solution‐focused (23.7%) were the five most commonly identified therapeutic approaches, although all 20 were endorsed by at least some of the sample.

Table 1.

Summary of sample demographics

Characteristics
Age, M, SD 57.29 11.42
Years in Practice, M, SD 24.22 11.03
Gender, N, %
Woman 1681 64.2
Man 928 35.4
Genderqueer 3 0.1
Gender Non‐conforming 3 0.1
Transman 2 0.1
Transwoman 1 0.0
Intersex 1 0.0
Race/Ethnicity, N, %
White/European American 2385 91.1
Latinx/Hispanic 61 2.3
Asian/Asian American (NH/NL) 50 1.9
Black/African American (NH/NL) 50 1.9
Multiracial/Multiethnic 40 1.5
Other 30 1.1
American Indian/Alaska Native/Native American 3 0.1
Primary Practice Setting, N, %
Individual Practice 1424 54.4
Group Practice 397 15.2
Hospital/Medical Center 178 6.8
Academic Medical Center 126 4.8
Other 116 4.4
School/University 113 4.3
Outpatient Treatment Facility 99 3.8
Veterans Affairs Medical Center 80 3.1
Geriatric Facility 23 0.9
Psychiatric Hospital 20 0.8
Correctional Facility 19 0.7
Rehabilitation Center 13 0.5
Residential Treatment Center 11 0.4
Open in a new tab

Abbreviation: NH/NL, Non‐Hispanic/Non‐Latinx.

1.2. Procedure

The current study was approved by the host University's Institutional Review Board. Personalized email invitations were sent to potential participants, requesting them to complete the survey. The survey's purpose was explained as an attempt to understand approaches to treating their patients during the COVID‐19 pandemic, but telepsychology was not mentioned to avoid introducing any bias or assumptions about the study or willingness to participate. To improve participation rates, a reminder invitation was sent one week after the initial invitation. Invited individuals who clicked on the link to the Qualtrics platform were provided an information sheet with further details about the study, including IRB information and contact details for the investigators, before asking them to provide informed consent to participate.

Participants completed a number of demographic questions about themselves and their practice, read the telepsychology definition used in this survey, and then answered the items concerning telepsychology. Participants were given the following description of telepsychology, “For the purpose of this survey, ‘telepsychology’ refers the use of real‐time audio (e.g., telephone) and/or video conferencing technology to provide psychological services.” This description aligns with definitions of telehealth provided by both the Centers for Medicare and Medicaid Services (2019) and the U.S. Drug Enforcement Administration Diversion Control Division (2019). For the current study, participants provided answers based on their experiences “during the COVID‐19 pandemic in the United States.”

1.3. Measures

1.3.1. Demographics

Participants provided data regarding their age, gender, race/ethnicity, years of clinical practice, current primary practice setting, and percentage of clinical work conducted via telepsychology. The level of telepsychology use was assessed by asking participants, “What percentage of your patient treatment is provided using telepsychology?” with answers ranging from 0% to 100%.

1.3.2. Technology acceptance model and theory of reasoned action

Data were collected using an item pool originally developed by Chau and Hu (2002) to assess variables within both the TAM and TRA to understand healthcare professionals' use of technology to deliver health care. These items have demonstrated good validity with healthcare professionals when measuring attitudes, subjective norms, perceived usefulness, perceived ease of use, and behavioral intention (Chau & Hu, 2002). To prevent participant attrition, five items with the highest factor loading for each construct (Chau & Hu, 2002) were selected rather than using the entire 18‐item pool. In these items, the words “telemedicine technology” were replaced with “telepsychology.” For example, the item assessing perceived usefulness read “Using telepsychology can improve my patient care and management.” Participants answered TAM and TRA items using 7‐point Likert‐type scales, where possible answers ranged from 1 “strongly agree,” to 4 “neutral,” to 7 “strongly disagree.”

1.4. Data analysis

Cross‐sectional path models were estimated in IBM AMOS 24.0 using maximum likelihood estimation. Initially, fully saturated versions of each model were run for both the TRA and the TAM by including all theorized variables and all possible paths. Following procedures discussed by Meyers et al. (2017), these saturated models were trimmed by removing the path that would least significantly decrease R 2 and then re‐estimated. Models were trimmed until the only paths that remained were statistically significant and had standardized regression weights of 0.1 or greater, representing at least a small effect size. As an exploratory analysis, we also estimated the cross‐sectional path model combining the TAM and TRA that was previously found to be a better fit than either the TAM or the TRA (Pierce et al., 2020b).

Once final models were determined, they were compared using a number of goodness‐of‐fit criteria including fit cutoffs of 0.90 for the normed fit index (NFI) and incremental fit index (IFI; Bentler & Bonett, 1980), 0.95 for the Tucker–Lewis Index (TLI) and comparative fit index (CFI; Hu & Bentler, 1999), and 0.06 for the root mean square error of approximation (RMSEA; Hu & Bentler, 1999).

2. RESULTS

Table 2 shows the descriptive statistics and bivariate relationships among the study variables. Attitude toward use of telepsychology, subjective norms, perceived usefulness, perceived ease of use, behavioral intention to use telepsychology, and current percentage use of telepsychology were all significantly positively associated. Table 3 shows mean scores of study variables grouped according to reported primary practice setting and number of years in practice. As reported previously (Pierce, Perrin, Tyler, et al., 2020), psychologists reported performing 7.07% (SD = 14.86) of their clinical work using telepsychology before the pandemic, with 45.70% not using telepsychology at all during that time. Psychologists in Veterans Affairs medical centers and individual practices reported the highest percentage use of telepsychology before the pandemic.

Table 2.

Descriptive statistics and bivariate correlations among study variables

Variables 1 2 3 4 5 M (SD) Range
  • 1.
    Attitude toward telepsychology
 6.58 (0.88) 1–7
  • 2.
    Subjective norms
 0.364** 6.05 (1.40) 1–7
  • 3.
    Perceived usefulness
 0.520** 0.421** 6.18 (1.33) 1–7
  • 4.
    Ease of use
 0.303** 0.192** 0.327** 5.42 (1.47) 1–7
  • 5.
    Behavioral intention
 0.582** 0.420** 0.604** 0.345** 6.44 (1.25) 1–7
  • 6.
    Percentage of treatment via telepsychology
 0.347** 0.336** 0.362** 0.232** 0.574** 85.53% (29.24) 0–100
Open in a new tab
**

p < 0.01.

Table 3.

Mean scores by primary practice setting and number of years in practice

Variable Academic medical center Group practice Hospital or medical center Individual practice School or university VA medical center Other practice setting Less than 25 years in practice 25 or more years in practice
Attitude toward telepsychology 6.75 6.59 6.63 6.60 6.47 6.74 6.36 6.62 6.53
Subjective norms 6.63 6.19 6.35 5.89 6.35 6.55 5.93 6.11 5.99
Perceived usefulness 6.54 6.27 6.20 6.19 6.02 6.23 5.86 6.29 6.06
Ease of use 5.46 5.55 5.39 5.46 5.34 5.31 5.10 5.45 5.39
Behavioral intention 6.63 6.54 6.40 6.50 6.42 6.44 5.98 6.51 6.37
Open in a new tab

2.1. Theory of reasoned action

2.1.1. Fully saturated model

The saturated, cross‐sectional TRA model explained 34.1% of the variance in percentage use of telepsychology. Subjective norms were associated with intention to use telepsychology, β = 0.24, p < 0.001, and with current use, β = 0.12, p < 0.001. Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.50, p < 0.001, but not with current use, β = −0.002, p = 0.937. Intention to use telepsychology was associated with current use, β = 0.53, p < 0.001. Fit indices were invalid because the model was saturated.

2.2. Final model

The final model (Figure 2) was constructed by removing the effect of attitude toward telepsychology on current use from the saturated model. This model explained 34.1% of the variance in percentage use of telepsychology and demonstrated excellent fit across all indices (Table 4). Subjective norms were again associated with behavioral intention to use telepsychology, β = 0.24, p < 0.001, and with current use, β = 0.12, p < 0.001. Attitude toward telepsychology was also associated with intention to use telepsychology, β = 0.50, p < 0.001, which was, in turn, associated with current use, β = 0.53, p < 0.001. In addition, there were significant indirect effects (all ps < 0.001) of subjective norms on current use, β = 0.13, and of attitude on current use, β = 0.26, both through intention to use telepsychology.

Figure 2.

Figure 2

Open in a new tab

Final model of the theory of reasoned action [Color figure can be viewed at wileyonlinelibrary.com]

Table 4.

Fit indices of unsaturated and final models

Theory of reasoned action Technology acceptance model
Final model Model 2 Final model Combined model
NFI 1.00 1.00 0.998 0.911
IFI 1.00 1.00 0.999 0.912
TLI 1.002 1.001 0.997 0.468
CFI 1.00 1.00 0.999 0.911
RMSEA 0.00 0.00 0.02 0.213
AIC 18.006 26.906 30.007 137.998
BIC 70.841 103.223 100.454 138.033
Open in a new tab

Abbreviations: AIC, Akaike Information Criterion; BIC, Bayesian Information Criterion; CFI, comparative fit index; IFI, incremental fit index; NFI, normed fit index; RMSEA, root mean square error of approximation; TLI, Tucker–Lewis Index.

2.3. Technology acceptance model

2.3.1. Fully saturated model

The fully saturated, cross‐sectional TAM model explained 33.1% of the variance in percentage use of telepsychology. Within this model, perceived usefulness of telepsychology was associated with attitude toward telepsychology, β = 0.47, p < 0.001, and with intention to use telepsychology, β = 0.39, p < 0.001, but not with percentage use, β = 0.04, p = 0.417. Perceived ease of use was also associated with attitude toward telepsychology, β = 0.15, p < 0.001, intention to use telepsychology, β = 0.11, p < 0.001, and percentage use, β = 0.04, p = 0.036. Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.35, p < 0.001, which was, in turn, associated with percentage use, β = 0.55, p < 0.001. Again, fit indices were invalid because the model was saturated.

2.3.2. Model 2

The first trimmed TAM model was constructed by removing the path from the saturated model that would least decrease R 2, which was the effect of perceived usefulness of telepsychology on current use. This model still explained 33.1% of the variance in percentage use of telepsychology and evidenced excellent fit (Table 4). Although all paths were statistically significant at p < 0.05, the effect of ease of use on current use was below the predetermined standardized regression weight magnitude, β = 0.04, p = 0.024, and was subsequently removed for the final model.

2.3.3. Final model

The final model (Figure 3) explained 33.0% of the variance in percentage use of telepsychology and also evidenced excellent fit. Perceived usefulness of telepsychology was associated with attitude toward telepsychology, β = 0.47, p < 0.001, and intention to use telepsychology, β = 0.39, p < 0.001. Ease of use was associated with attitude toward telepsychology, β = 0.15, p < 0.001, and intention to use telepsychology, β = 0.11, p < 0.001. Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.35, p < 0.001, which was, in turn, associated with percentage use, β = 0.57, p < 0.001. There were also a number of significant indirect effects (all ps < 0.001). Ease of use exerted an indirect effect on intention through attitude, β = 0.05, and current use through both attitude and intention, β = 0.10. Perceived usefulness exerted an indirect effect on intention through attitude, β = 0.16, and on current use through attitude and intention, β = 0.32. And finally, attitude exerted an indirect effect on current use through intention, β = 0.20.

Figure 3.

Figure 3

Open in a new tab

Final model of the technology acceptance model [Color figure can be viewed at wileyonlinelibrary.com]

2.4. Combined model

The cross‐sectional combined model (Figure 4) explained 14.5% of the variance in percentage use of telepsychology. Whereas several of the fit indices demonstrated adequate fit (i.e., NFI, IFI, and CFI), two (TLI and RMSEA) showed less than adequate or poor fit to the data. Within this model, subjective norms were uniquely associated with perceived usefulness, β = 0.42, p < 0.001 and ease of use, β = 0.19, p < 0.001. Both perceived usefulness, β = 0.32, p < 0.001, and perceived ease of use, β = 0.13, p < 0.001, were directly associated with current use of telepsychology. Finally, subjective norms exerted an indirect effect on current use through perceived usefulness and ease of use, β = 0.16, p = 0.012.

Figure 4.

Figure 4

Open in a new tab

Combined model​ [Color figure can be viewed at wileyonlinelibrary.com]

2.5. Model comparison

In examining the fit of the final versions of the TRA and TAM models and Model 2 of the TAM (Table 4), the TRA model demonstrated slightly better fit to the data, as shown by the NFI, IFI, TLI, CFI, and RMSEA indices. However, both final models showed excellent fit across all indices; as neither showed substantially better fit than the other, both were retained, suggesting that both models are useful in explaining use of telepsychology during the pandemic. In contrast, the fit indices for the combined model ranged from poor to adequate fit, which may suggest that it does not describe telepsychology use as well as the TRA or TAM models.

3. DISCUSSION

The current cross‐sectional study is the first known to examine the uptake of telepsychology services among psychologists during the pandemic using theoretical models of behavior change. The TRA and the TAM both demonstrated excellent fit in explaining psychologists' percentage use of telepsychology during the pandemic. All possible indirect effects, mediational and multiple mediational, within each model were statistically significant.

Results from the TRA indicated that psychologists' attitudes concerning telepsychology and subjective norms were both significantly associated with their intentions to use telepsychology. Of these two factors, attitudes showed the strongest association with intention to use telepsychology, and the association between attitude toward telepsychology and current use was fully mediated by intention to use telepsychology. Subjective norms were associated with use of telepsychology, both directly and indirectly through their statistical effect on intention to use telepsychology, reflecting a partial mediation. In turn, intention to use telepsychology was associated with actual percentage use of telepsychology during the pandemic, explaining 34% of the variance in current use. The exploratory trimming procedure used in the current study also demonstrated that subjective norms were directly related to percentage use of telepsychology, an association that is not directly theorized in the TRA.

Several of these associations were also found by Pierce et al. (2020b) in a pre‐pandemic path analysis of psychologists' openness to use telepsychology, although the current model explained more than twice the variance explained by the TRA in that previous study. As in the current study, psychologists' intention to use telepsychology was significantly associated with attitudes toward telepsychology and, to a much lesser degree, by perceived subjective norms; however, in the current study, intention to use telepsychology was less strongly associated with attitude toward telepsychology (β = 0.50 vs. β = 0.67, respectively). Intention to use was also more strongly associated with subjective norms (β = 0.24 vs. β = 0.06, respectively) than in the previous study. In turn, intention to use was associated with actual use, and to a greater degree than in the previous study (β = 0.53 vs. β = 0.41, respectively). However, that model did not include the direct association between subjective norms and current use found in the current model, as it was not theorized in the original TRA, which may partly explain why the model's fit indices ranged from good to poor. Moreover, actual use of telepsychology was dichotomized (yes/no), in contrast to the current study, which used a continuous measure of current use.

Results from the TAM showed that psychologists' perceived usefulness of telepsychology and perceived ease of use both were associated with their attitudes toward telepsychology, although perceived usefulness demonstrated a much stronger statistical effect. Perceived usefulness was also related to psychologists' reported intention to use telepsychology, an association that was posited in the original version of the TAM (Davis, 1986) but was removed in an early revision (Davis et al., 1989). This particular finding is consistent with the author's initial supposition that within an organizational setting, individuals will intend to take up technology that they believe will improve their job performance, and that this belief is distinct from positive or negative attitudes toward technology more generally.

Interestingly, the exploratory trimming procedure revealed perceived ease of use as significantly associated with intention to use telepsychology, even though it was not directly theorized in the TAM. Although it was not as strongly related to intention as was perceived usefulness or attitude toward telepsychology, it did represent a unique effect that was not fully explained by attitude toward telepsychology. Again, perceived usefulness showed a stronger association with intention to use telepsychology than did perceived ease of use, although both were significant. Both of these associations, even though not posited in earlier versions of the TAM, were suggested as potential revisions in a more recent meta‐analysis of associations among the TAM variables in studies of technology uptake (Schepers & Wetzels, 2007); that study also found that perceived usefulness showed a stronger association with behavioral intention than ease of use. Moreover, all mediational and multiple mediational indirect effects within the model were significant, providing internal evidence supporting the overall structure of the model. Taken together, these findings suggest that psychologists' beliefs that telepsychology would positively impact their work and that it would be reasonably easy to implement each impact their attitudes about telepsychology as well as their behavioral intention to use it during the pandemic.

The results from the TAM showed similarities to some pre‐pandemic studies on telepsychology and telehealth uptake among providers. For example, Pierce et al. (2020b; path analytic) also found that psychologists' attitudes toward telepsychology were strongly informed by their perceptions of usefulness and, to a lesser degree, by perceived ease of use. Psychologists' attitudes toward telepsychology and perceived usefulness also showed direct associations with intention to use, which, in turn, was associated with actual use. Again, the model in the current study explained nearly two times as much variance in current use as did the model in the previous study. Associations in the current study between attitudes toward telepsychology and perceived usefulness (β = 0.47 vs. β = 0.66, respectively) and attitudes toward telepsychology and perceived ease of use (β = 0.15 vs. β = 0.21, respectively) were weaker than those found in the previous study. Likewise, the association between attitude toward telepsychology and intention to use was also somewhat weaker than that found previously (β = 0.35 vs. β = 0.44, respectively), although, as with the TRA, the association found in the current study between intention to use and actual use was stronger than that found previously (β = 0.57 vs. β = 0.41, respectively). However, the previous model demonstrated adequate to poor fit to the data, possibly due to omitting significant paths that were not originally theorized in the model, such as the direct path from perceived ease of use to intention to use telepsychology that was significant in the current model. Finally, several of the path coefficients in the current TAM were similar in magnitude to those found in a meta‐analysis of TAM research (Schepers & Wetzels, 2007), in particular the strong association between behavioral intention to use and actual use (β = 0.57 in the current study vs. β = 0.55 in the meta‐analysis). Taking into account results from the previous meta‐analysis and the current study, it may be that the TAM could be expanded to account for these direct associations between behavioral intention to use and perceived usefulness and ease of use, respectively.

In contrast, the current model differed in several ways from that found by Monthuy‐Blanc et al. (2013) in a study of mental health providers' attitudes toward telepsychology in the treatment of First Nations patients. In that study, the primary predictor of intention to use telepsychology was the providers' perceptions of usefulness, rather than attitudes toward telepsychology or perceived ease of use. In fact, attitude toward telepsychology did not significantly predict intention to use at all. Moreover, in a previous study of Hong Kong physicians' receptivity to telehealth, perceived usefulness significantly predicted both attitude and intention but perceived ease of use did not (Hu et al., 1999).

Overall, the findings from both models demonstrate a strong connection between intention to use and actual current use of telepsychology. Attitudes toward telepsychology were highly associated with intention to use telepsychology in both models, suggesting that psychologists' beliefs about telepsychology (e.g., endorsement of myths or stereotypes about telepsychology provision, such as beliefs about inadequacy of crisis intervention options or that serious mental disorders cannot be adequately treated via telepsychology; Fenichel et al., 2002) substantially inform their intention to use it in clinical care. Perceived usefulness and subjective norms were also associated with intention to use telepsychology, as was perceived ease of use, though to a lesser degree.

In contrast, the combined model tested previously (Pierce, Perrin, Tyler, et al., 2020) evidenced less than adequate fit to the data in the current study. As discussed above, these differences may be due, in part, to the previous dichotomization of the outcome variable of current use of telepsychology. However, it may also indicate that the variables included in the path model do not adequately describe use of telepsychology during the pandemic. Absent from this model is intention to use telepsychology, which was strongly associated with actual use of telepsychology, both in the models presented in the current study as well as in previous research (e.g., Schepers & Wetzels, 2007). It could be that psychologists' intention to use telepsychology was a more salient factor associated with actual use due to conditions during the pandemic, in comparison to before the pandemic, in which intention to use telepsychology may not have been sufficient to predict actual use when subjective norms, perceived ease of use, and perceived usefulness were taken into account. For example, before the pandemic, in the absence of systemic and widespread need for alternatives to face‐to‐face clinical care, these other factors may have been more relevant than intention for actual use.

Additional research will be necessary to examine these associations and the applicability of both models for psychologists' use of telepsychology once the pandemic has subsided and it is generally safe to provide in‐person services. It could be, for example, that perceived usefulness and ease of use of telepsychology may become even more salient to psychologists' decisions to provide care via telepsychology when alternative methods of care delivery are more readily available. Alternatively, it could be that usefulness and ease of use could demonstrate greater associations with telepsychology use during the pandemic; for example, psychologists who endorse negative beliefs about telepsychology usefulness or ease of use may have chosen to scale down their clinical practice during the pandemic rather than increase use of telepsychology. Earlier pre‐pandemic research has suggested that providers' attitudes toward telepsychology tend to improve over time with use (e.g., Brooks et al., 2012), which may relate to their increased comfort level with using synchronous video technology (Gibson et al., 2011). However, it is unclear whether those patterns will continue post‐pandemic once providers and patients have the option to safely return to in‐person services.

3.1. Clinical and training implications

Taken together, these findings suggest several important avenues for increasing or improving use of telepsychology among psychologists during the COVID‐19 pandemic and more broadly. First, efforts to increase telepsychology provision may benefit from training or campaigns to address attitudes toward telepsychology, such as by debunking myths about telepsychology or challenging biases against it. Second, training could target subjective norms by increasing providers' awareness of the prevalence of telepsychology; similarly, organizations with established telepsychology infrastructure could promote awareness of provider resources, such as training in telepsychology and availability of supportive policies and procedures, or even incentivize providers' use of telepsychology for appropriate clinical cases. There are a number of resources available through the APA website (American Psychological Association, 2020) that could help address these first two issues, including several articles and internal APA surveys of telepsychology provision during the pandemic. Many of these resources serve to underscore the ubiquity of telepsychology during COVID‐19 and either directly or indirectly address many common misperceptions of telepsychology. Other organizations (e.g., VHA, large hospital systems) also have trainings available for providers interested in learning more.

Third, it would be important to help providers unfamiliar with telepsychology understand more clearly how to use it, such as by providing training on more practical concerns such as connecting to compliant software, troubleshooting connection issues with patients, and documenting telepsychology visits appropriately. A number of practical training resources are available, including best practice guidelines for telepsychology (e.g., Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 2013), as well as tools published by APA (e.g., American Psychological Association, 2020), and other organizations. Given that psychologists across different practice settings tended to report similar mean scores on the study variables, it may be that similar strategies could be used across settings; however, psychologists' scores appeared to differ more for those with more than 25 years of clinical experience, which could suggest that different strategies may be needed to target less versus more experienced psychologists. Empirical research would be needed to confirm whether either of these patterns is associated with differential effectiveness of interventions to increase telepsychology use.

3.2. Research and policy implications

These findings also convey a number of important implications for future telepsychology research and policy. There may be several avenues for policymakers to support the continued use of telepsychology at high rates post‐pandemic. For example, maintaining changes made by the U.S. Centers for Medicare and Medicaid Services (2020) to continue to reimburse telepsychology at the same rate as in‐person visits has the potential to influence providers' perceptions of the usefulness of telepsychology and perhaps their attitudes toward telepsychology more generally. Making permanent some of the temporary changes to the Health Insurance Portability and Accountability Act and to state board requirements for licensure for provision of services across state lines has the potential to ameliorate concerns related to legality and confidentiality reported by mental health providers in a systematic review of attitudes toward telepsychology (Connolly et al., 2020).

As these implications are based on cross‐sectional analyses, we caution that future research should test the effectiveness of these potential interventions or policy changes using experimental or quasi‐experimental methodology that would enable researchers to estimate their causal impact. This reflects an important direction for additional pandemic‐related research but also will be a crucial target for investigation once safety concerns related to the pandemic have been ameliorated. It could be, for example, that these interventions may have more or less impact on telepsychology uptake post‐pandemic when providers and patients have more choices about modality of care. Effects on uptake may also depend in part on whether post‐pandemic healthcare policy continues to support the widespread use of telepsychology.

3.3. Limitations and future directions

There are a number of limitations to the current study that should be taken into account when interpreting the findings. First, participants self‐selected after receiving email invitations to participate in the study, which may bias the sample. Demographic and other data were not available for psychologists who received the study invitation but who chose not to participate, so group comparisons were not feasible. Nonetheless, the current sample's demographics were extremely similar to the overall membership of American Psychological Association (2018).

Second, although the initial variables included in each model were based on the TRA and TAM theoretical models, respectively, the final models were derived using exploratory trimming procedures and do not exactly replicate the theoretical associations among variables as posited by the TRA and TAM. So the trimming procedure could have resulted in a better fit than if a strict adherence to all paths, or lack thereof, in the models had occurred (e.g., Pierce et al., 2020b). Finally, as the data in the current study were cross‐sectional and the path analyses were correlational in nature, causality cannot be proven, and it cannot be assumed that changes in use of telepsychology were directly caused by the other variables. Alternative theoretical models may be operating among the variables. For example, it may be the case that attitudes and beliefs about telepsychology were impacted as psychologists were forced to increase use of telepsychology to comply with social distancing guidelines. Further, although the mediational effects found in the current study were statistically significant, this does not necessarily suggest that these cross‐sectional associations also operate in a temporal or casual fashion. Cross‐sectional mediational analyses have the potential to generate biased estimates (e.g., Maxwell & Cole, 2007), and so it would be important to replicate this study using longitudinal data. Further, future research employing longitudinal designs could use cross‐lagged panel approaches to more closely examine questions of causality as well as mediation.

Although the current study used a definition of telepsychology that was consistent with CMS definitions (“real‐time audio [e.g., telephone] and/or video conferencing technology”), the survey did not assess for psychologists' provision of clinical care via other modalities, such as interactions with a patient via online chat, email, or text message, either as a standalone treatment or as an adjunct to more traditional care; previous research suggests that a sizeable minority of psychologists may use these asynchronous modalities to provide care (e.g., Glueckauf et al., 2018). Moreover, patients may also seek care through mental health apps and web‐based interventions. While research is already under way to explore factors related to healthcare providers' uptake of these m‐health services (see Gagnon et al., 2016, for a review), it would be important to examine whether the results from the current study replicate within a similar sample of US‐licensed psychologists providing care during the pandemic. It could be that demographic factors (such as patient or provider rurality, age, income level, employment status, health insurance status) and TRA or TAM variables may be associated with provision of care through these channels. Furthermore, although testing for differences in study variables based on respondent characteristics was beyond the scope of the current study, the parent study found some differences in telepsychology uptake based on primary practice setting (Pierce, Perrin, Tyler, et al., 2020), and so additional studies may benefit from examining how these theoretical variables may differ as a function of demographic or practice‐related characteristics.

4. CONCLUSION

Despite these limitations, the current study is the only study to date examining psychologists' provision of telepsychology services during the COVID‐19 pandemic using theoretical models of behavior change. Furthermore, it is the largest known survey of licensed psychologists practicing in the United States during the pandemic. The excellent fit of both the TRA and TAM path models suggests extremely ripe targets for training and campaigns to increase and improve psychologists' provision of telepsychology during the pandemic and beyond.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

COPYRIGHT STATEMENT

As Dr. Paul B. Perrin and Dr. Grace B. McKee are employees of the U.S. Government and contributed to the manuscript “Examining Models of Psychologists' Telepsychology Use during the COVID‐19 Pandemic: A National Cross‐Sectional Study” as part of their official duties, the work is not subject to US copyright.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/jclp.23173

ACKNOWLEDGMENTS

Dr. McKee is supported by the Department of Veterans Affairs Office of Academic Affiliations Advanced Fellowship Program in Mental Illness Research and Treatment, the Medical Research Service of the Veterans Affairs Central Virginia Health Care System, and the Department of Veterans Affairs Mid‐Atlantic Mental Illness Research, Education, and Clinical Center (MIRECC). 

McKee, G. B. , Pierce, B. S. , Donovan, E. K. , & Perrin, P. B. (2021). Examining models of psychologists' telepsychology use during the COVID‐19 pandemic: A national cross‐sectional study. J Clin Psychol, 77, 2405–2423. 10.1002/jclp.23173

DATA AVAILABILITY STATEMENT

The study data are available from the corresponding author on request.

REFERENCES

  1. Adalja, A. A. , Toner, E. , & Inglesby, T. V. (2020). Priorities for the US health community responding to COVID‐19. Journal of the American Medical Association, 323(14), 1343–1344. 10.1001/jama.2020.3413 [DOI] [PubMed] [Google Scholar]
  2. Alzueta, E. , Perrin, P. B. , Baker, F. , Caffarra, S. , Ramos‐Usuga, D. , Yuksel, D. , & Arango‐Lasprilla, J. C. (2021). How the COVID‐19 pandemic has changed our lives: A study of psychological correlates across 59 countries. Journal of Clinical Psychology, 77(3), 556–570. 10.1002/jclp.23082 [DOI] [PubMed] [Google Scholar]
  3. American Psychological Association . (2015). Guidelines for the practice of telepsychology. http://apa.org/practice/guidelines/telepsychology.aspx
  4. American Psychological Association . (2018). Demographic characteristics of APA members by membership status, 2017. https://www.apa.org/workforce/publications/17-member-profiles/table-1.pdf
  5. American Psychological Association . (2020). APA COVID‐19 information and resources. https://www.apa.org/topics/covid-19/
  6. Association of Psychology Postdoctoral and Internship Centers . (2020). COVID‐19 information‐APPIC FAQs. https://www.appic.org/NewsArticles/ArtMID/1931/ArticleID/4/COVID-19-Information
  7. Australian Psychological Society . (2020). Psychological services via telehealth: Information for consumers. https://www.psychology.org.au/getmedia/0c1ece8a-7fce-4d0f-ad44-833ed9dba71a/20APS-IS-COVID-19-Public-Telehealth-P2_1.pdf
  8. Bentler, P. M. , & Bonett, D. G. (1980). Significance tests and goodness‐of‐fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. [Google Scholar]
  9. Brooks, E. , Manson, S. M. , Bair, B. , Dailey, N. , & Shore, J. H. (2012). The diffusion of telehealth in rural American Indian communities: A retrospective survey of key stakeholders. Telemedicine and e‐Health, 18(1), 60–66. 10.1089/tmj.2011.0076 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. California Board of Psychology . (2020). Coronavirus disease 2019 (COVID‐19). https://psychology.ca.gov/applicants/covid_19.shtml
  11. Centers for Disease Control and Prevention . (2020). Resources for clinics and healthcare facilities. https://www.cdc.gov/coronavirus/2019-ncov/healthcare-facilities/index.html
  12. Centers for Medicare and Medicaid Services . (2019). Telemedicine. https://www.medicaid.gov/medicaid/benefits/telemedicine/index.html
  13. Chau, P. Y. K. , & Hu, P. J.‐H. (2002). Investigating healthcare professionals' decisions to accept telemedicine technology: An empirical test of competing theories. Information & Management, 39(4), 297–311. 10.1016/S0378-7206(01)00098-2 [DOI] [Google Scholar]
  14. Connolly, S. L. , Miller, C. J. , Lindsay, J. A. , & Bauer, M. S. (2020). A systematic review of providers' attitudes toward telemental health via videoconferencing. Clinical Psychology: Science and Practice, 27(2), e12311. 10.1111/cpsp.12311 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Davis, F. D. (1986). A technology acceptance model for empirically testing new end‐user information systems: theory and results (Doctoral dissertation). Sloan School of Management, Massachusetts Institute of Technology.
  16. Davis, F. D. , Bagozzi, R. P. , & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003. [Google Scholar]
  17. European Federation of Psychologists' Associations . (2020). EFPA psychologists' support hub about COVID‐19 crisis. https://efpa.magzmaker.com/covid_19
  18. Fenichel, M. , Suler, J. , Barak, A. , Zelvin, E. , Jones, G. , Munro, K. , Meunier, V. , & Walker‐Schmucker, W. (2002). Myths and realities of online clinical work. Cyberpsychology & Behavior, 5(5), 481–497. 10.1089/109493102761022904 [DOI] [PubMed] [Google Scholar]
  19. Fishbein, M. , & Ajzen, I. (1977). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison‐Wesley. [Google Scholar]
  20. Gagnon, M.‐P. , Ngangue, P. , Payne‐Gagnon, J. , & Desmartis, M. (2016). m‐Health adoption by healthcare professionals: A systematic review. Journal of the American Medical Informatics Association, 23(1), 212–220. 10.1093/jamia/ocv052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gibson, K. , O'Donnell, S. , Coulson, H. , & Kakepetum‐Scghultz, T. (2011). Mental health professionals' perspectives of telemental health with remote and rural First Nations communities. Journal of Telemedicine and Telecare, 17(5), 263–267. 10.1258/jtt.2011.101011 [DOI] [PubMed] [Google Scholar]
  22. Glueckauf, R. L. , Maheu, M. M. , Drude, K. P. , Wells, B. A. , Wang, Y. , Gustafson, D. J. , & Nelson, E. L. (2018). Survey of psychologists' telebehavioral health practices: Technology use, ethical issues, and training needs. Professional Psychology: Research and Practice, 49(3), 205–219. 10.1037/pro0000188 [DOI] [Google Scholar]
  23. Godleski, L. , Darkins, A. , & Peters, J. (2012). Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006–2010. Psychiatric Services, 63(4), 383–385. 10.1176/appi.ps.201100206 [DOI] [PubMed] [Google Scholar]
  24. Gurwitch, R. H. , Salem, H. , Nelson, M. M. , & Comer, J. S. (2020). Leveraging parent–child interaction therapy and telehealth capacities to address the unique needs of young children during the COVID‐19 public health crisis. Psychological Trauma: Theory, Research, Practice, and Policy, 12(S1), S82–S84. 10.1037/tra0000863 [DOI] [PubMed] [Google Scholar]
  25. Hu, L. , & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. 10.1080/10705519909540118 [DOI] [Google Scholar]
  26. Hu, P. J. , Chau, P. Y. K. , Sheng, O. R. L. , & Tam, K. Y. (1999). Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of Management Information Systems, 16(2), 91–112. 10.1080/07421222.1999.11518247 [DOI] [Google Scholar]
  27. Joint Task Force for the Development of Telepsychology Guidelines for Psychologists . (2013). Guidelines for the practice of telepsychology. American Psychologist, 68(9), 791–800. 10.1037/a0035001 [DOI] [PubMed] [Google Scholar]
  28. Martin, J. N. , Millán, F. , & Campbell, L. F. (2020). Telepsychology practice: Primer and first steps. Practice Innovations, 5, 114–127. 10.1037/pri0000111 [DOI] [Google Scholar]
  29. Maxwell, S. E. , & Cole, D. A. (2007). Bias in cross‐sectional analyses of longitudinal mediation. Psychological Methods, 12(1), 23–44. 10.1037/1082-989X.12.1.23 [DOI] [PubMed] [Google Scholar]
  30. Meyers, L. , Gamst, L. , & Guarino, A. (2017). Applied multivariate research: Design and interpretation (Version 3rd) [Computer software] (3rd ed.). Sage. [Google Scholar]
  31. Monthuy‐Blanc, J. , Bouchard, S. , Maïano, C. , & Séguin, M. (2013). Factors influencing mental health providers' intention to use telepsychotherapy in First Nations communities. Transcultural psychiatry, 50(2), 323–343. 10.1177/1363461513487665 [DOI] [PubMed] [Google Scholar]
  32. Nitkin, K. (2020). In fight against coronavirus, telemedicine ramps up at Johns Hopkins. https://www.hopkinsmedicine.org/coronavirus/telemedicine.html
  33. Ookla . (2020). Tracking COVID‐19's impact on global internet performance. https://www.speedtest.net/insights/blog/tracking-covid-19-impact-global-internet-performance/
  34. Perrin, P. B. , Pierce, B. S. , & Elliott, T. R. (2020). COVID‐19 and telemedicine: A revolution in healthcare delivery is at hand. Health Science Reports, 3, e166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Perrin, P. B. , Rybarczyk, B. D. , Pierce, B. S. , Jones, H. A. , Shaffer, C. , & Islam, L. (2020). Rapid telepsychology deployment during the COVID‐19 pandemic: A special issue commentary and lessons from primary care psychology training. Journal of Clinical Psychology, 76(6), 1173–1185. 10.1002/jclp.22969 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Petrovčič, A. , Petrič, G. , & Lozar Manfreda, K. (2016). The effect of email invitation elements on response rate in a web survey within an online community. Computers in Human Behavior, 56, 320–329. 10.1016/j.chb.2015.11.025 [DOI] [Google Scholar]
  37. Pierce, B. S. , Perrin, P. B. , & McDonald, S. D. (2020a). Demographic, organizational, and clinical practice predictors of U.S. psychologists' use of telepsychology. Professional Psychology: Research and Practice, 51(2), 184–193. 10.1037/pro0000267 [DOI] [Google Scholar]
  38. Pierce, B. S. , Perrin, P. B. , & McDonald, S. D. (2020b). Path analytic modeling of psychologists' openness to performing clinical work with telepsychology: A national study. Journal of Clinical Psychology, 76(6), 1135–1150. 10.1002/jclp.22851 [DOI] [PubMed] [Google Scholar]
  39. Pierce, B. S. , Perrin, P. B. , & McDonald, S. D. (2020c). Pre‐COVID‐19 deterrents to practicing with videoconferencing telepsychology among psychologists who didn't. Psychological Services. 10.1037/ser0000435 [DOI] [PubMed] [Google Scholar]
  40. Pierce, B. S. , Perrin, P. B. , Tyler, C. M. , McKee, G. B. , & Watson, J. D. (2020). The COVID‐19 telepsychology revolution: A national study of pandemic‐based changes in U.S. mental health care delivery. American Psychologist, 76, 14–25. 10.1037/amp0000722 [DOI] [PubMed] [Google Scholar]
  41. Schepers, J. , & Wetzels, M. (2007). A meta‐analysis of the technology acceptance model: Investigating subjective norm and moderation effects. Information & Management, 44(1), 90–103. 10.1016/j.im.2006.10.007 [DOI] [Google Scholar]
  42. State of Arizona Board of Psychologist Examiners . (2020). Summary of board actions—COVID‐19 State of emergency license application waivers/temporary licensure/provisional licensure. https://psychboard.az.gov/sites/default/files/documents/files/COVID-19%20Waivers.pdf
  43. Stiepan, D. (2020). How video appointments are changing the way Mayo Clinic patients receive care. https://newsnetwork.mayoclinic.org/discussion/how-video-appointments-are-changing-the-way-mayo-clinic-patients-receive-care/
  44. Tahir, D. (2020). VA push for telehealth amid coronavirus slowed by balky networks. Politico. https://www.politico.com/news/2020/03/20/va-push-for-telehealth-amid-coronavirus-slowed-by-balky-networks-139542
  45. Telebehavioral Health Institute . (2020). About Telebehavioral Health Institute. https://telehealth.org/about/
  46. Texas State Board of Examiners of Psychologists . (2020). Emergency licensure for practice within a declared disaster area. http://www.tsbep.texas.gov/emergency-temporary-licensure
  47. U.S. Centers for Medicare and Medicaid Services . (2020). Medicare telemedicine health care provider fact sheet. https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet
  48. U.S. Department of Health and Human Services' Office for Civil Rights . (2020). OCR eases HIPAA telehealth enforcement for COVID‐19 emergency. https://www.aha.org/news/headline/2020-03-18-ocr-eases-hipaa-telehealth-enforcement-covid-19-emergency
  49. U.S. Drug Enforcement Administration Diversion Control Division . (2019). Title 21 United States Code (USC) Controlled Substances Act. https://www.deadiversion.usdoj.gov/21cfr/21usc/802.htm
  50. Uscher‐Pines, L. , Sousa, J. , Raja, P. , Mehrotra, A. , Barnett, M. L. , & Huskamp, H. A. (2020). Suddenly becoming a “virtual doctor”: Experiences of psychiatrists transitioning to telemedicine during the COVID‐19 pandemic. Psychiatric Services, 71(11), 1143–1150. 10.1176/appi.ps.202000250 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Varker, T. , Brand, R. M. , Ward, J. , Terhaag, S. , & Phelps, A. (2019). Efficacy of synchronous telepsychology interventions for people with anxiety, depression, posttraumatic stress disorder, and adjustment disorder: A rapid evidence assessment. Psychological services, 16(4), 621–635. 10.1037/ser0000239 [DOI] [PubMed] [Google Scholar]
  52. Veterans Health Administration, Office of Emergency Management . (2020). COVID‐19 response plan. https://www.va.gov.opa/docs/VHA_COVID_19_03232020_vF_1.pdf
  53. World Health Organization . (2020). WHO Director‐General's opening remarks at the media briefing on COVID‐19 –11 March 2020. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The study data are available from the corresponding author on request.

Articles from Journal of Clinical Psychology are provided here courtesy of Wiley

RESOURCES