Skip to main content
PMC home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2021 Feb 9;6(2):327–337. doi: 10.1007/s41347-020-00164-5

A Survey of Residents/Fellows, Program Directors, and Faculty About Telepsychiatry: Clinical Experience, Interest, and Views/Concerns

Cesar Cruz 1, Kali Orchard 2, Erica Z Shoemaker 3, Donald M Hilty 4,5,✉
PMCID: PMC7870779  PMID: 33585672

Abstract

The effectiveness of telepsychiatry video for clinical care is well established, and clinician and psychiatry resident interest in it is growing—particularly in light of the COVID-19 impact. Still, few residency programs in the United States have core curricula, rotations/electives available, and competencies, and many faculty have no experience. A survey was sent via national organization listserves for residents, fellows, faculty, and program directors to complete. The survey queried demographics, clinical experience and interest, and views/concerns, using Likert-like and yes/no questions. Descriptive statistics and other analyses compared the groups to assess impact of clinical experience on interest and views/concerns. Respondents (N = 270) have limited clinical experience with telepsychiatry (46% none), with trainees having less than others, and yet, most were very interested or interested in it (68%). Trainees (N = 123) have concerns about being prepared for future practice. Clinical experience with video in the range of 6–20 h appears to dramatically increase interest and reduce concerns, though less time has a positive effect. Respondents have concerns about connectivity, medico-legal issues, and clinical effectiveness (e.g., diverse populations, psychosis, and emergencies) with telepsychiatry. More research is needed to assess current clinical and curricular experience, interest, and concerns. Additional curricular interventions during residency and fellowship training could build skills and confidence, if this is feasible, and the benefits outweigh the costs.

Keywords: Telepsychiatry, Telehealth, Telebehavioral, Mental, Health, Psychiatry, Graduate, Medical, Education, Survey, Residents, Fellows, Concerns, Perceptions, Views

Introduction

In its sixth decade, telepsychiatry (TP) or telebehavioral health (TBH)—video—is effective for many types of treatments, psychiatric disorders, and populations (Hilty et al. 2013; Nelson and Sharp 2016; Gloff et al. 2015; Hubley et al. 2016). Initial concerns about TBH in terms about engagement, disconnections, and other intangibles have subsided, and it appears good enough in comparison to in-person care (Hilty et al. 2013; Nelson and Sharp 2016; Hilty et al. 2002), in collaboration with primary care (Myers et al. 2015; Hilty et al. 2018c), and for diverse populations (Hilty et al. 2018a) and patients in crisis (Freeman et al. 2020). Recent public health events (i.e., COVID-19) are shifting TP to the forefront of clinical care and training (Torous et al. 2020).

Interest in TBH by patients, students, and psychiatric residents and fellows has significantly increased from the 1990s and 2000s (Glover et al. 2013; Hilty et al. 2020d), but technological experiences can be difficult, and early systems had problems; this affected user experience, and some were hesitant to share their concerns (Hilty et al. 2013; Myers et al. 2015). Residents across medicine have attributed inadequate experience with telemedicine, false beliefs of teachers, and speculation about efficacy as the key limiting factors (Glover et al. 2013; Mucic and Hilty 2018). Patients, clinicians, and others adopt technology on a continuum from slow to fast based on motivation and other factors like generation/age (i.e., digital natives versus immigrants) (Wang et al. 2013). Ironically, as new generations adapt smartphones, social media, and other technologies, they have more personal technology experience than their teachers (Hilty et al. 2020d).

Additional telepsychiatric education has been suggested due to the increasing important of technology in care, growing resident interest, and surveys of programs that show that fewer than half of respondents have an informal clinical experience, and even fewer have a curriculum (Balon et al. 2015; Sunderji et al. 2015; Crawford et al. 2016; Khan and Ramtekkar 2019; Hoffman and Kane (2015). Similarly, there has been a call for an evidence-based pediatric TP curricula for trainees and practicing child and adolescent psychiatrists (Khan and Ramtekkar 2019). Video TP competencies, methods for teaching them, and a plan for evaluation have been put forward, framed in the Accreditation Council for Graduate Medical Education Milestone domains (Hilty et al. 2015, 2017, 2018b). This was followed by social media (Zalpuri et al. 2018), TBH (Maheu et al. 2018; Maheu et al. 2019), mobile health (Hilty et al. 2019c, d, 2020a), and asynchronous (Hilty et al. 2020d) competencies. A key issue though is whether our learners (i.e., residents and fellows) and those teaching them (i.e., clinicians and faculty [Fac]) can keep up with the growing evidence base of technology clinical interventions and outcomes.

This study aimed to assess:

  1. How much clinical experience or exposure residents, fellows, program directors, and Fac have had with TP,

  2. Respondents’ views/concerns and interest in TP, and

  3. If clinical experience affects interest and concerns, and if there is a threshold of clinical experience helps to allay concerns and increase interest in TP.

This may be a valuable pre-COVID snapshot—and could be compared with post-COVID perspectives by investigators with significant resources and a broader sample. We hypothesized that parties’ interest in TBH and concerns may be based on clinical experience, and if it was limited, it may lead to inaccurate views/concerns, which would be negatively correlated with current interest and future practice with TP or TBH.

Methods

Subjects

The inclusion criteria were willingness to complete the survey and being a psychiatric resident, fellow, clinical Fac, or program director. The link to Survey Monkey was sent by e-mail to academic psychiatric organization listserves (e.g., American Association for Directors of Psychiatry Residency Programs and Association for Academic Psychiatry) with a request to forward it to residents (Rs), fellows (Fs), Fac, and program directors (PDs). Reminder emails were sent at 3- and 6-month follow-up. There were no exclusion criteria. There were no incentives or marketing materials.

Information and Survey

The survey queried demographics, clinical experience and interest, and views/concerns about TP. The TP interest and clinical were rated with 5-item Likert like questions (Likert 1932) (e.g., for experience, options were no(ne), 1, 2–5, 6–20, and 20+ h). Views/concerns of TP were asked about with 47 yes/no or true/false questions. The inquiry was in English, took 10–15 min to complete, and was anonymous. The survey used the word TP instead of TBH due to professional scope terminology. Surveys were iteratively developed, piloted, and revised before use. General information included professional demographic information including position in training (i.e., R or F but not year of training) or role in the department (i.e., program or fellowship director or Fac) but not sex, race, ethnicity, or age; specialty and subspecialty, if any; and geographic practice setting.

The 47 questions about views/concerns were put in the negative to flush out concerns, and many were used or adapted from patient and clinician questionnaires: Working Alliance Inventory-Short Form (Horvath and Greenberg 1989); Session Evaluation Questionnaire (Stiles et al. 1994; Distance Communication Comfort Scale (Schneider 1999); Videoconferencing Telepresence Scale (Robillard and Bouchard 2004); and Telemedicine Satisfaction Questionnaire (Yip et al. 2003). These questionnaires focus on goals of services (i.e., evaluation), the interpersonal contact (e.g., subjective impression, degree of comfort, level of engagement) and tasks achieved (e.g., satisfaction, comparison to in-person visits). Internal and external consistency evaluation of the survey was not conducted.

The content of the questions were assorted, but subgrouping was done based on TP themes (Appendix Table 3): clinical care effectiveness/ineffectiveness (general impressions, engagement, satisfaction, 9 questions); specific patients/populations (age, culture, disorders or treatments, 9); communication (successes, limitations, 9); system/service issues, 6; cost/reimbursement, 4; legal, 4; education/training (clinical with TP, 3); and technical, 3. Reframed into GME competencies, the composite would have been: patient care, 8 questions; systems-based practice, 8; professionalism, 2; communication, 7; practice-based learning, 6; knowledge, 11; and miscellaneous (i.e., impressions rather than competency-specific, 5).

Table 3.

Telepsychiatry questionnaire

Clinical
General
  1. One cannot do individual therapy with telepsychiatry.
  2. Adequate follow-up services cannot be provided with telepsychiatry.
  3. There are more adverse outcomes with telepsychiatry than face-to-face psychiatry.
  4. One cannot see more than 1 person at a time using telepsychiatry.
  5. Patients do not like telepsychiatry.
Not effective
  6. Telepsychiatry is not as effective as face-to-face psychiatry.
  7. There is lack of evidence for the efficacy of telepsychiatry.
  8. One cannot use hands-on tools to assess functioning or to provide therapy with telepsychiatry.
  9. One cannot establish therapeutic alliance with telepsychiatry.
  10. One cannot perform a physical exam with telepsychiatry.
Specific patients/populations
  11. One cannot manage emergencies related to safety with telepsychiatry.
  12. One cannot do family/group therapy with telepsychiatry.
  13. One cannot use telepsychiatry to treat certain mental illnesses.
  14. Setting professional boundaries is a problem with telepsychiatry.
  15. Paranoid patients do not like telepsychiatry.
  16. People with physical or mental disabilities cannot use telepsychiatry.
  17. Disruptive behavior cannot be managed with telepsychiatry.
  18. Certain cultures will be less accepting of telepsychiatry.
  19. Telepsychiatry cannot be performed in foreign languages with interpreters.
  20. One cannot properly perform developmental and neurological assessments with telepsychiatry.
Communication
  21. Patients find telepsychiatry to be too impersonal.
  22. Clinicians find telepsychiatry to be too impersonal.
  23. Children are not able to talk to a screen.
  24. One would need at least 1 face-to-face session before applying telepsychiatry.
  25. One cannot express empathy with telepsychiatry.
  26. Nonverbal cues are missed with telepsychiatry.
  27. Eye contact cannot be assessed with telepsychiatry.
  28. One cannot properly assess social interactions with telepsychiatry.
Technical
  29. The technology setup required by clinicians for telepsychiatry is too complicated.
  30. The technology setup required by patients for telepsychiatry is too complicated
  31. Poor internet connection is a roadblock to implementing telepsychiatry.
Cost/econ/reimb
  32. Telepsychiatry offers poor reimbursement for services.
  33. Telepsychiatry favors patients who have means.
  34. The cost of starting a telepsychiatry practice is high.
  35. Coordinating clerical staff to provide billing and scheduling is too difficult with telepsychiatry.
Legal
  36. The liability risks involved in telepsychiatry are unknown.
  37. Privacy is an issue with telepsychiatry.
  38. Malpractice insurance for telepsychiatry is high.
  39. Telepsychiatry is not properly regulated.
Systems/services/roles
  40. The roles and responsibilities of the primary physician versus the consultant are ill-defined.
  41. The primary provider may not follow through on the recommendations made by the telepsychiatrist.
  42. Psychiatrist: telepsychiatry is isolative.
  43. Performing telepsychiatry from home would be uncomfortable.
  44. Coordination of obtaining vital signs and labs with telepsychiatry is not feasible.
Education/training/mentoring
  45. Residency training is insufficient for one to become competent in telepsychiatry.
  46. There are no experts in the field of telepsychiatry to provide mentorship.
  47. Telepsychiatry does not have its own association to join.
Open in a new tab

Data, statistical analysis and comparisons. Data received by the research team was kept in a password protected file. Before completing analyses, questionnaires were examined for completion; for any missing items, the participant’s responses on similar questions from the same category of the questionnaire were substituted (Yip et al. 2003). Descriptive statistics (e.g., averages and percentages) and proportion of responses (e.g., for cohorts or groups) were calculated for each question. Between and intragroup comparisons (e.g., clinical experience for Rs/Fs versus PDs/Fac) were made when possible (e.g., a Pearson chi-square analysis). Linear regression analysis, including multivariate regression analysis, was performed with participants’ clinical experience as primary and interest as secondary factors and then another analysis was conducted in reverse order. For all analyses, a significance level .05 corresponding to a confidence interval of 95% was used to determine statistical significance.

Institutional Review Board (IRB)

An IRB exemption was granted in 2017, as this was an anonymous survey.

Results

All Respondent Data (N = 270)

Subjects totaled 270, with 76 Rs, 47 Fs, 57 PDs, and 90 Fac. Most Rs/Fs responders were from general psychiatry (54%), child and adolescent (33%), and other fellowships (13%; forensic, geriatric, psychosomatic, and substance). The geographic distribution of responders was urban (76%), rural (5%), and both (19%). Rs/Fs reported practicing in urban setting (81%) where academic programs are usually located.

Respondents’ clinical experience varied with TBH, from none (46%), 1 h (11%), 2–5 h (13%)—69% had less than 5 h (Fig. 1a) versus 79% of Rs/Fs (Fig. 1b). In terms of their interest in TBH, overall, 68% of respondents were interested or very interested (34% each) (Fig. 2a); Rs/Fs had less interest, with 58% interested or very interested (21% and 37%, respectively) (Fig. 2b).

Fig. 1.

Fig. 1

Open in a new tab

Clincial exposure or experience to telepsychiatry for all participants versus residents/fellows

Fig. 2.

Fig. 2

Open in a new tab

interest of all respondents versus residents/fellows in telepsychiatry

For all Rs/Fs and PDs/Fac, the 10 most common concerns about TP are (Table 1) that one cannot perform a physical exam (54% overall; 67% Rs/Fs); poor Internet connection is a roadblock to implementing TP (52% overall, 57% Rs/Fs); liability risks involved with TP are unknown (47% overall, 52% Rs/Fs); and certain cultures will be less accepting (39% overall, 52% Rs/Fs). Concerns about privacy, effectiveness, managing emergencies, paranoid patients, and disruptive behavior were rated 25–33%, overall, with Rs/Fs having more concerns about emergencies and paranoid patients. Interestingly, 41% of Rs thought residency is insufficient to become competent in TP. Overall, those in the not interested group had less clinical experience and many more concerns.

Table 1.

Top 10 concerns about telepsychiatry: a comparison of all respondents versus residents/fellows and program directors/faculty

# Item/concern ALL
N = 270		 Rs/Fs
N = 123		 PDs/Fac N = 147
1 One cannot perform a physical exam. 54.47% 63.64%* 47.06%
2 Poor internet connection is a roadblock to implementing TP. 51.63% 54.55% 49.26%
3 Liability risks involved with TP are unknown. 46.75% 50.91% 43.38%
4 Certain cultures will be less accepting. 39.62% 46.36%* 33.09%
5 Nonverbal cues will be missed. 35.77% 44.55%* 28.68%
6 Privacy is an issue. 32.52% 35.25% 30.15%
7 TP is not as effective as to in-person psychiatry. 32.11% 37.27% 27.94%
8 One cannot manage emergencies related to safety with TP. 29.67% 49.09%* 22.06%
9 Residency is insufficient for one to become competent in TP. 29.67% 40.91%* 20.59%
10 Paranoid patients do not like TP. 26.42% 42.67%* 13.24%
Open in a new tab

Abbreviations: Fac, faculty; F, fellow; PD, program director; R, resident; TP, telepsychiatry

*Significant with p < .05 in comparing Rs/Fs versus PDs/Fac

Regression analyses suggest that clinical experience and interest are both negatively associated with concerns, with the former having approximately triple impact. Clinical experience also significantly impacted interest. When interest was run as the primary, it was not predictive of clinical experience but predicted intensity but concerns particularly in the undecided, uninterested, and very uninterested groups.

Impact of Clinical Experience on Interest

Most had little experience of TP (82%), as in none (66%) or only one encounter (16%) They reported being very interested (25%), interested (33%), undecided (25%), uninterested (10%), and very uninterested (5%). A subanalysis showed that compared to those with no clinical experience, one encounter significantly shifted those interested for very interested from 58% to 74%, increasing the very interested group from 23% to 42%; this implies a group moved from undecided to interested and others moved from interested to very interested.

Impact of Clinical Experience on Views

Those with 0–5 h of clinical experience (67%) had many concerns, which were statistically higher than the 6–20 h group (Table 2). The percentages for 0–5 and 6–20 h, respectively, are one cannot perform a physical exam (59–42%); liability risks (54–28%); poor Internet as a road block (54–36%); certain cultures are less accepting (42–31%); and nonverbal cues are missed with TP (41–24%). Perceptions that residency is insufficient and TP as ineffective compared to in-person also decreased.

Table 2.

A comparison of concerns about telepsychiatry between psychiatrists with 0–5 h of experience versus those with multiple encounters (i.e., 6–20+ h)

Item/concern 0- to 5-h experience
N = 123a 		6- to 20-h experience
N = 58a
One cannot perform a physical exam with TP. 59.13%* 41.95%
The liability risks involved in TP are unknown. 54.04%* 28.77%*
Poor Internet connection is a roadblock to TP. 53.71% 36.16%*
Certain cultures will be less accepting. 42.20%* 31.31%*
Nonverbal cues are missed with TP. 41.04%* 23.59%*
Residency training is insufficient for one to become competent with TP. 38.99% 19.54%*
Privacy is an issue. 36.99% 19.90%*
TP is not as effective as face-to-face psychiatry. 36.42% 21.92%*
One cannot manage emergencies related to safety with TP. 33.53% 20.05%*
Clinicians find TP to be too impersonal. 29.48% 13.64%
Open in a new tab

Abbreviation: TP, telepsychiatry

*Significant with p < .05 in comparing no to significant experience (i.e., multiple encounters)

aOverall N = 270; this is a comparison of two subgroups only

Impact of Interest on Views

There were many respondents interested or very interested in TP (66%) regardless of clinical experience: 0–1 (42%), 1–5 (11%), 6–20 (10%), or 20+ (27%) h. Respondents with high interest (40%) had concerns about inability to perform a physical exam (55%), poor Internet connection (54%), liability risk (43%), certain cultures are less accepting (40%), and nonverbal cues being missed (30%).

Respondents undecided in their interest—56% without any exposure and 23% with 1–5 h—had concerns about inability to perform a physical examination (60%), liability risks (54%), poor Internet connection (50%), privacy as an issue (50%), and that effectiveness as in-person psychiatry (44%).

Rs/Fs Data (N = 123)

Overview

Rs/Fs were numerically, but not statistically, less interested in TP than all participants or PDs/Fac, specifically. Respondents typically practice in an urban setting (81%), have clinical experience with TP (54%), and have significant interest (66%; interested or very interested). Rs/Fs clinical experience with TP ranged from none (46%), 1–5 h (23%), 6–20 h (14%), to 20+ h (17%); not significantly different from all respondents or PDs/Fac.

Impact of Clinical Experience on Interest and Views/Concerns

Of the group with 0–5 h of clinical experience, 66% were noted to be interested or very interested in TP, 24% were undecided, and only 8% were uninterested or very uninterested. As clinical experience increased, those very interested or interested increased: none (58%); 1–5 h (61%); 6–20 h (78%); and 20+ h (83%)—the shift from 1–5 to 6–20 h was substantial.

The concerns for all groups were the inability to perform a physical exam (54%), unknown liability related to TP (59%), poor Internet connection, TP residency training being insufficient, and certain cultures being less accepting of TP (37%). Compared to those with no clinical experience, those with one encounter had less concern about liability (59–36%) but did not change in regard to other parameters. Those with more clinical experience (i.e., 6+ h) had similar concerns to those with 0–5 h of TP that poor Internet connection is a road block to good care and that one cannot perform a physical exam. The 6+ h group had statistically less concerns that nonverbal cues are missed (23%) and that paranoid patient do not like TP (19%).

Impact of Interest on Views/Concerns

Those with high interest in TP had variable clinical experience with TP—from no encounters (40%), one-time encounter (13%), 2–5 h (11%), 6–20 h (10%), and 20+ h (27%)—and their concerns did not statistically differ from all Rs or all respondents. Of those uninterested or very uninterested—approximately 80% had no clinical experience—concerns that were statistically significant versus all Rs/Fs were loss of nonverbal cues (67%), ineffectiveness (50%), poor Internet connection (50%), and TP as ineffective (50%). Those undecided, uninterested, and very uninterested did not vary from the other groups in clinical experience or types of concerns but had higher percentages of concerns.

PDs/Fac Data (N = 147)

Overview

PDs/Fac had no (39%), 1–5 (23%), 6–20 (8%), or 20+ h (30%) of clinical experience with TP.

Impact of Clinical Experience on Concerns

Of the 90 respondents with 0–5 h of clinical experience with TP, 57% of these responders reported they are interested or very interested in TP, while 25% were undecided and 16% uninterested or very uninterested. Their concerns did not vary compared to those of all respondents or to Rs/Fs, other than signified specifically in Table 1. Of note, other concerns were about primary care provider follow-up on consultations (29%), poor reimbursement (26%), and management of disruptive behaviors (23%).

While clinical experience more dramatically decreased specific concerns, a closer analysis of moving from 0 to 1 to 2–5 to 6–20 h in this group, respectively, showed trends: ability to perform a physical exam (69% to 12% to 24% to 29%) and Internet connectivity (51% to 15% to 28% to 36%). This suggests two things: All respondent data are skewed by those with 0 h, and there is a learning curve for concerns and skills to mature, with broader perspective and accuracy (Downey and King 1998).

Impact of Interest on Concerns

A high percentage (79%) PDs/Fac were noted to be interested to very interested in TP. Their clinical experience varied: 30% none, 10% a one-time encounter, 13% up to 5 h, 9% from 6 to 20 h, and 38% with more than 20 h of TP. The undecided group varied in clinical experience—53% had no clinical experience—and their concerns did not differ from other groups.

Discussion

There are several preliminary findings of this survey about clinical experience with interest in and concerns about TP or TBH. First, respondents had limited clinical experience with TP, probably due to time, competing interests, and number of available TP options; Rs/Fs have even less experience. Second, respondents’ interest in TP is high—consistent with other surveys (Gloff et al. 2015; Sunderji et al. 2015), though it was very surprising that Rs/Fs have less interest than PDs/Fac, particularly in light of technology adoption data and assumptions (Wang et al. 2013). Clinical experience is not the same as person experience with technology, of course, in terms of skills (Hilty et al. 2018b). While it is premature to conclude that a lack of clinical exposure negatively impacts trainee interest in a causal fashion, interest appears to increase with exposure. This is important in another way, too, as a third finding is that physicians or clinicians who are undecided, uninterested, and very uninterested have higher percentages of concerns. Indeed, during a required rotation, Rs had many valuable tele-experiences: establishing rapport and engaging with patients; working collaboratively with a team; identifying different approaches to use; and becoming aware of how to handle complex cases (Teshima et al. 2016). This enhanced both interest in participating in TP in the future and understanding of providing psychiatric services to underserved communities.

Rs/Fs and PDs/Fac have concerns about effectiveness of clinical care. These include an inability to perform a physical examination, loss of nonverbal cues, managing emergencies, dealing with patients’ paranoia, connectivity, and medico-legal issues (e.g., liability). There is no “gold standard” for to compare to these respondents’ concerns; they may or may not be well-founded. The effectiveness of TP for assessing patients (Hilty et al. 2013; Nelson and Sharp 2016; Gloff et al. 2015; Hubley et al. 2016) and establishing a therapeutic relationship (Hilty et al. 2020c) runs counter to many of these concerns; yet, the TP evidence base needs improvement related to (1) dealing with emergencies (Freeman et al. 2020); (2) doing inpatient and acute care (Hilty et al. 2013); and (3) working with culturally diverse populations (Hilty et al. 2018a). A conceptual framework has been suggested for cultural and video competencies (Hilty et al. 2020b), along with administrative approaches to reduce barriers, implement successfully, and evaluate outcomes (Hilty et al. 2018a, 2020b; Guerra and Kurtz 2017). TP studies related to the treatment of patients with psychosis and delusional frameworks are limited, but few experienced clinicians have concerns; initial assumption that such patients may feel monitored have not been found to be true (Hilty et al. 2013).

Where do clinical care and training go from here? The external mandate to “flatten the curve” of COVID-19 spreading will “accelerate and bend the curve” of digital health care (Torous et al. 2020)—so Fac and trainees are getting more experience. A survey of psychology training programs (Hames et al. 2020) found that some were implementing, planned to implement, or were considering implementing telepsychology services—with trainees and Fac supervisors in the clinics, at home or a combination of both; some academic centers already had TP rotations in place, usually electives, to build upon (Hilty et al. 2019b). A follow-up survey could capture the impact of clinical service changes and this increase in experience; yet, it is unclear if curricula have been significantly changed for rotations (e.g., teaching and supervision on TP) due to the rapid series of events of this difficult time. PDs/Fac cannot depend on trainees’ interest and other positive attitudes alone to develop requisite skills, as they are not a substitute for supervised clinical experience to develop knowledge and skill (Pratt 1998). At a minimum, Fac and trainees who were uncertain about TP gain experience, and they may embrace TP, or they find that they do not like this technology. It is important that they make a more informed decision about this and that Fac help them with the chaos of the pandemic.

PDs/Fac, department chairs, and health system leaders have key decisions to make now and over the next year of the COVID-19 pandemic: (1) services: what % of TP or TBH to shoot for, depending on patient expectations/needs; (2) curricula: return to the past one, largely, with informal changes versus capitalizing to integrate TP or TBH into it; and (3) Fac/professional development: returning to past activities or similarly capitalizing to integrate technology. PDs have been more inclined to design electives rather than required rotations due to competing demands and resources, though interinstitution pooling of resources may help (i.e., curricula, Fac teachers/supervisors, and evaluation processes). Institutional competencies for TP have been suggested for intervening at the Fac, hospital/clinic system, and academic health center level, by consolidating clinical, training, and Fac development missions around competencies (Hilty et al. 2020d; Armstrong et al. 2004; Hilty et al. 2019a, b). Six steps to implement new technologies have been suggested: (1) assess readiness; (2) create/hardwire the culture; (3) write policies and procedures; (4) establish the curriculum and competencies; (5) train learners and Fac; and (6) evaluate/manage change (Hilty et al. 2019b). Two tiers of options for each were provided. For example, to assess readiness, leaders focus on clinical service technology platforms for video, e-mail, and other common technologies and evaluate the workflow and training changes based on the structure/function of groups of their organization. More time and resources, if available, would be needed to tackle the workflow and training changes pertaining to advanced technologies (e.g., mobile health apps, devices, and wearable sensors).

Training program and student directors would select, develop, and adapt regular competencies to include technologically based components at a minimum, and a multiyear curriculum would be better, using a mixture of methods including clinical care, seminar, supervision, research/quality improvement projects, case-based learning core rotations, and advanced-year electives/requirements. This work may be aided by identifying a vice-chair of education (or clinical services) or other mid- or advanced-career Fac member to promote technology and liaison with national psychiatric organizations. Core curricular (i.e., didactic and clinical) interventions appear indicated early in training so that tangible clinical experiences may improve Rs/Fs’ confidence that education/training will be adequate in preparation for future practice. While interest may predict concerns to a degree, and attitude (e.g., interest) is important, it is not a substitute for knowledge or skill (Pratt 1998; Hilty et al. 2015). Training is an opportunity to address Rs and Fac concerns (Hilty et al. 2015; Hames et al. 2020).

Limitations to this report are many. The most important limitation is the concern that the sample size may not be large enough or the sample may not be adequately representative for the results to be reliable. First, there is a potential selection bias (i.e., respondents may have interest in TP). The final sample is small, compared to the total number of psychiatry trainees nationally (i.e., approximately 225 residency programs with 6,700 Rs and 1,200 Fs/year; 51+% of Rs are female with 7% unreported) (American Psychiatric Association 2019) or internationally. Time, high volume email—particularly on the listserves—and other responsibilities were constraints, and there were no incentives to offer. Second, a yes/no survey not as discerning as a Likert-like design and its reliability/validity was not studied. The questions were in the negative to flush out concerns; as in our experience, psychiatrists and Rs do not like to complain or share their concerns. A balance of positive, neutral, and negative is overall better, though. Third, the study did not inquire about the year of residency—this is key since perspectives may change over time; on the other hand, with so few programs doing any curriculum, it may not have changed results. Fourth, the survey did not technically define experience or exposure, so it could have been interpreted as clinical or didactic. Fifth, respondents’ self-identified information was not confirmed in terms of how they defined or rated their clinical experience. The ratings were not temporally proximate to the experience, either; the survey attempted to explore ranges of experience rather than specific quantitative data. Sixth, statistically significant correlations do not indicate causation. Seventh, more specific demographic and training data would be helpful (e.g., year of training and year when exposed to TP). Finally, there is no “gold standard” for norms to compare the respondents’ data to (i.e., normal anxiety about disconnections).

Despite the limitations, the current study provides valuable pre-COVID-19 snapshot on TP or TBH experience, interest, and attitudes that may serve as a foundation for further research and training.

Conclusions

Overall, the effectiveness of TP or TBH is substantial, and interest in it from Rs/Fs is growing, despite limited training opportunities. If they have more supervised experiences, their skills will help to improve access, timeliness, and quality of care. Training, experience, and supervision will also reduce concerns and improve confidence. Institutional competencies for TP are a systematic way to proceed, and sharing resources across health systems may be helpful. Competency implementation and research are needed to determine how exposure is linked with changes in skills, knowledge, and attitudes. Residency and fellowship training appears to be an ideal time for a TP curriculum, if it is feasible, and the benefit outweighs the cost.

Appendix

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Change history

3/4/2021

A Correction to this paper has been published: 10.1007/s41347-021-00201-x

Contributor Information

Cesar Cruz, Email: cesarcruzmd10@gmail.com.

Kali Orchard, Email: kali_orchard@hotmail.com.

Erica Z. Shoemaker, Email: ezshoema@med.usc.edu

Donald M. Hilty, Email: donh032612@gmail.com

References

  1. American Psychiatric Association. (2019). 2018 Resident/ Fellow Census. file:///U:/Telemed/Key%20Docs%20&%20Pubs/APA-Resident-Census-2019.pdf. Accessed 24 July 2020.
  2. Armstrong EG, Mackey M, Spear SJ. Medical education as a process management problem. Academic Medicine. 2004;79(8):721–728. doi: 10.1097/00001888-200408000-00002. [DOI] [PubMed] [Google Scholar]
  3. Balon R, Beresin EV, Coverdale JH, Louie AK, Roberts LW. Strengthening telepsychiatry's role in clinical care and education. Academic Psychiatry. 2015;39(1):6–9. doi: 10.1007/s40596-014-0265-x. [DOI] [PubMed] [Google Scholar]
  4. Crawford A, Sunderji N, López J, Soklaridis S. Defining competencies for the practice of telepsychiatry through an assessment of resident learning needs. BMC Medical Education. 2016;16:28. doi: 10.1186/s12909-016-0529-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Downey RG, King C. Missing data in Likert ratings: A comparison of replacement methods. The Journal of General Psychology. 1998;125(2):175–191. doi: 10.1080/00221309809595542. [DOI] [PubMed] [Google Scholar]
  6. Freeman, R. E., Boggs, K. M., Zachrison, K. S., Freid, R. D., Sullivan, A. F., Espinola, J. A., & Camargo Jr., C. A. (2020). National study of telepsychiatry use in U.S. emergency departments. Psychiatric Services (Washington, D.C.), 71(6), 540-546. 10.1176/appi.ps.201900237. [DOI] [PubMed]
  7. Gloff NE, LeNoue SR, Novins DK, Myers K. Telemental health for children and adolescents. International Review of Psychiatry (Abingdon, England) 2015;27(6):513–524. doi: 10.3109/09540261.2015.1086322. [DOI] [PubMed] [Google Scholar]
  8. Glover JA, Williams E, Hazlett LJ, Campbell N. Connecting to the future: Telepsychiatry in postgraduate medical education. Telemedicine Journal and E-Health. 2013;19(6):474–479. doi: 10.1089/tmj.2012.0182. [DOI] [PubMed] [Google Scholar]
  9. Guerra O, Kurtz D. Building collaboration: A scoping review of cultural competency and safety education and training for healthcare students and professionals in Canada. Teaching and Learning in Medicine. 2017;29(2):129–142. doi: 10.1080/10401334.2016.1234960. [DOI] [PubMed] [Google Scholar]
  10. Hames JL, Bell DJ, Perez-Lima LM, Holm-Denoma JM, Rooney T, Charles NE, Thompson SM, Mehlenbeck RS, Tawfik SH, Fondacaro KM, Simmons KT, Hoersting RC. Navigating uncharted waters: Considerations for training clinics in the rapid transition to telepsychology and telesupervision during COVID-19. Journal of Psychotherapy Integration. 2020;30(2):348–365. doi: 10.1037/int0000224. [DOI] [Google Scholar]
  11. Hilty DM, Nesbitt TS, Marks SM, Callahan EJ. How telepsychiatry affects the doctor-patient relationship: Communication, satisfaction, and additional clinically relevant issues. Primary Psychiatry. 2002;9(9):29–34. [Google Scholar]
  12. Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: A 2013 review. Telemedicine Journal and E-Health. 2013;19(6):444–454. doi: 10.1089/tmj.2013.0075. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hilty DM, Crawford A, Teshima J, Chan S, Sunderji N, Yellowlees PM, et al. A framework for telepsychiatric training and e-health: Competency-based education, evaluation and implications. International Review of Psychiatry. 2015;27(6):569–592. doi: 10.3109/09540261.2015.1091292. [DOI] [PubMed] [Google Scholar]
  14. Hilty DM, Maheu M, Drude K, Hertlein K, Wall K, Long R, et al. Telebehavioral health, telemental health, e-therapy and e-health competencies: The need for an interdisciplinary framework. Journal of Technology in Behavioral Science. 2017;2(3–4):171–189. doi: 10.1007/s41347-017-0036-0. [DOI] [Google Scholar]
  15. Hilty DM, Feliberti J, Evangelatos G, Lu FG, Lim RF. Competent cultural telebehavioral healthcare to diverse populations: Administration, evaluation, and financing. Journal of Technology in Behavioral Science. 2018;4(3):186–200. doi: 10.1007/s41347-018-0076-0. [DOI] [Google Scholar]
  16. Hilty DM, Maheu MM, Drude KP, Hertlein KM. The need to implement and evaluate telehealth competency frameworks to ensure quality care across behavioral health professions. Academic Psychiatry. 2018;42(6):818–824. doi: 10.1007/s40596-018-0992-5. [DOI] [PubMed] [Google Scholar]
  17. Hilty DM, Sunderji N, Suo S, Chan S, McCarron RM. Telepsychiatry and other technologies for integrated care: Evidence base, best practice models and competencies. International Review of Psychiatry (Abingdon, England) 2018;30(6):292–309. doi: 10.1080/09540261.2019.1571483. [DOI] [PubMed] [Google Scholar]
  18. Hilty DM, Liu HY, Stubbe D, Teshima J. Defining professional development in medicine, psychiatry, and allied fields. The Psychiatric Clinics of North America. 2019;42(3):337–356. doi: 10.1016/j.psc.2019.04.001. [DOI] [PubMed] [Google Scholar]
  19. Hilty DM, Unützer J, Ko DG, Luo J, Worley L, Yager J. Approaches for departments, schools, and health systems to better implement technologies used for clinical care and education. Academic Psychiatry. 2019;43(6):611–616. doi: 10.1007/s40596-019-01074-2. [DOI] [PubMed] [Google Scholar]
  20. Hilty DM, Chan S, Torous J, Luo J, Boland R. A telehealth framework for mobile health, smartphones and apps: Competencies, training and faculty development. Journal of Technology in Behavioral Science. 2019;4(2):106–123. doi: 10.1007/s41347-019-00091-0. [DOI] [Google Scholar]
  21. Hilty DM, Chan S, Torous J, Luo J, Boland RJ. A competency-based framework for psych/behavioral health apps for trainees, faculty, programs and health systems. Psychiatric Clinics of North America. 2019;42:513–534. doi: 10.1016/j.psc.2019.05.007. [DOI] [PubMed] [Google Scholar]
  22. Hilty DM, Chan S, Torous J, Luo J, Boland RJ. A framework for competencies for the use of mobile technologies in psychiatry and medicine. Journal of Medical Internet Research Uhealth and Mobile Health. 2020;2020:8(2). doi: 10.2196/12229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hilty DM, Gentry MT, McKean AJ, Cowan KE, Lim RF, Lu FG. Telehealth for rural diverse populations: Telebehavioral and cultural competencies, clinical outcomes and administrative approaches. mHealth. 2020;6:20. doi: 10.21037/mhealth.2019.10.04. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hilty DM, Randhawa K, Maheu MM, McKean AS, Pantera R, Rizzo A. A review of telepresence, virtual reality and augmented reality applied to clinical care. Journal of Technology in Behavioral Science. 2020;5:178–205. doi: 10.1007/s41347-020-00126-x. [DOI] [Google Scholar]
  25. Hilty, D. M., Torous, J., Parish, M., Chan, S., Xiong, G., Scher, L., & Yellowlees, P. M. (2020d). A literature review comparing clinicians' approaches and skills to in-person, synchronous and asynchronous care: Moving toward asynchronous competencies to ensure quality care. Telemedicine Journal and E-Health. 10.1089/tmj.2020.0054. [DOI] [PubMed]
  26. Hoffman P, Kane JM. Telepsychiatry education and curriculum development in residency training. Academic Psychiatry. 2015;39(1):108–109. doi: 10.1007/s40596-013-0006-6. [DOI] [PubMed] [Google Scholar]
  27. Horvath AO, Greenberg LS. The development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36:223–233. doi: 10.1037/0022-0167.36.2.223. [DOI] [Google Scholar]
  28. Hubley S, Lynch SB, Schneck C, Thomas M, Shore J. Review of key telepsychiatry outcomes. World Journal of Psychiatry. 2016;6(2):269–282. doi: 10.5498/wjp.v6.i2.269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Khan S, Ramtekkar U. Child and adolescent telepsychiatry education and training. The Psychiatric Clinics of North America. 2019;42(4):555–562. doi: 10.1016/j.psc.2019.08.010. [DOI] [PubMed] [Google Scholar]
  30. Likert R. A technique for the measurement of attitudes. Archives of Psychology. 1932;22(140):5–55. [Google Scholar]
  31. Maheu M, Drude K, Hertlein KM, Hilty DM. A framework for interprofessional telebehavioral health competencies: Implementation and challenges moving forward. Academic Psychiatry. 2018;42(6):825–833. doi: 10.1007/s40596-018-0988-1. [DOI] [PubMed] [Google Scholar]
  32. Maheu M, Drude K, Hertlein K, Lipschutz R, Wall K, Long R, et al. An interdisciplinary framework for telebehavioral health competencies. Journal of Technology in Behavioral Science. 2019;3(2):108–140. doi: 10.1007/s41347-019-00113. [DOI] [Google Scholar]
  33. Mucic D, Hilty DM. Training in telepsychiatry. In: Pi EH, Hoon TC, MHM H, editors. Mental health and illness worldwide: Education about mental health and illness. New York: Springer Publishing; 2018. pp. 1–31. [Google Scholar]
  34. Myers K, Vander Stoep A, Zhou C, McCarty CA, Katon W. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: A community-based randomized controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry. 2015;54(4):263–274. doi: 10.1016/j.jaac.2015.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Nelson EL, Sharp S. A review of pediatric telemental health. Pediatric Clinics of North America. 2016;63(5):913–931. doi: 10.1016/j.pcl.2016.06.011. [DOI] [PubMed] [Google Scholar]
  36. Pratt DD. Five perspectives on teaching in adult and higher education. Malabar: Krieger Publishing; 1998. [Google Scholar]
  37. Robillard G, Bouchard S. The sense of presence in videoconferencing and emotional engagement. CyberPsychology and Behavior. 2004;7(3):280–281. [Google Scholar]
  38. Schneider, P. L. (1999). Mediators of distance communication technologies psychotherapy: Development of a measure. Paper presented to the American Psychological Association, Boston. https://www.studio5d.com/paul/research/DCCS.html. Accessed 24 July 2020.
  39. Stiles WB, Reynolds S, Hardy GE, Rees A, Barkham M, Shapiro DA. Evaluation and description of psychotherapy sessions by clients using the session evaluation questionnaire and the session impacts scale. Journal of Counseling Psychology. 1994;41(2):175–185. doi: 10.1037/0022-0167.41.2.175. [DOI] [Google Scholar]
  40. Sunderji N, Crawford A, Jovanovic M. Telepsychiatry in graduate medical education: A narrative review. Academic Psychiatry. 2015;39(1):55–62. doi: 10.1007/s40596-014-0176-x. [DOI] [PubMed] [Google Scholar]
  41. Teshima J, Hodgins M, Boydell KM, Pignatiello A. Resident evaluation of a required telepsychiatry clinical experience. Academic psychiatry: The journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry. 2016;40(2):348–352. doi: 10.1007/s40596-015-0373-2. [DOI] [PubMed] [Google Scholar]
  42. Torous J, Jän Myrick K, Rauseo-Ricupero N, Firth J. Digital mental health and covid-19: Using technology today to accelerate the curve on access and quality tomorrow. Journal of Medical Informatics Research Mental Health. 2020;7(3):e18848. doi: 10.2196/18848. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Wang QE, Myers MD, Sundaram D. Digital natives and digital immigrants towards a model of digital fluency. Busin Inform Syst Engin. 2013;5(6):409–419. doi: 10.1007/s11576-013-0390-2. [DOI] [Google Scholar]
  44. Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the telemedicine satisfaction questionnaire to evaluate patient satisfaction with telemedicine: A preliminary study. Journal of Telemedicine and Telecare. 2003;9(1):46–50. doi: 10.1258/135763303321159693. [DOI] [PubMed] [Google Scholar]
  45. Zalpuri I, Liu H, Stubbe D, Wrzosek M, Sadhu J, Hilty D. A competency-based framework for social media for trainees, faculty and others. Academic Psychiatry. 2018;42(6):808–817. doi: 10.1007/s40596-018-0983-6. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Technology in Behavioral Science are provided here courtesy of Nature Publishing Group

RESOURCES