Acceptability of Telepsychiatry in American Indians

Abstract

Telepsychiatry differs from in-person treatment in terms of its delivery mechanism, and this dissimilarity may increase cultural differences between the provider and the patient. Because cultural competence and identification can impact patient satisfaction ratings, we wanted to explore whether cultural differences in our study population influenced telepsychiatric and in-person interviews. Here, we compared the acceptability of conducting psychiatric assessments with rural American Indian veterans by real-time videoconferencing versus in-person administration. The Structured Clinical Interview for DSM-IIIR (SCID) was given to participants both in person and by telehealth. A process measure was created to assess participants' responses to the interview type concerning the usability of the technology, the perceptions of the interviewee/interviewer interaction, the cultural competence of the interview, and satisfaction with the interview and the interview process. The process measure was administered to 53 American Indian Vietnam veterans both in-person and by real-time interactive videoconferencing. Mean responses were compared for each participant. Interviewers were also asked several of the same questions as the participants; answers were compared to the corresponding participant responses. Overall, telepsychiatry was well received and comparable in level of patient comfort, satisfaction, and cultural acceptance to in-person interviews. We also found evidence to suggest that interviewers sometimes interpreted participant satisfaction as significantly less favorable than the participants actually responded. Despite the potential of videoconferencing to increase cultural differences, we found that it is an acceptable means for psychiatric assessment of American Indian veterans and presents an opportunity to provide mental health services to a population that might otherwise not have access.

Introduction

Studies have demonstrated telepsychiatry's utility in bringing direct patient care, consultation, education, and research to previously inaccessible populations, including those who reside in rural areas.¹ Recently published work has shown the potential for telepsychiatry to address health disparities in American Indian communities.^2–4 This is particularly salient for rural American Indian veterans, given their high rates of post-traumatic stress disorder (PTSD)⁵ and often remote locations and dispersion, which act as a significant barrier to care.⁶

Multiple studies have looked at patient satisfaction with telepsychiatry. Overall, these studies found a high degree of satisfaction but were limited by sample size, informal and narrowly focused evaluations, and lack of controls.¹ No well-controlled studies have assessed the reaction of American Indian populations or American Indian veterans with telepsychiatry in the form of live interactive videoconferencing. Understanding patient satisfaction with care is a complex, yet important area of study. Research shows that satisfaction affects overall patient health.^7,8 For example, higher satisfaction rates predict better recall of treatment advice, higher rates of treatment adherence, and greater trust of their provider.^7,8

The role of cultural competence and identification as they relate to patient satisfaction has developed a strong research base.⁹ Studies demonstrate that patients who rate their physicians as more culturally competent express higher satisfaction with their physician's performance.¹⁰ Likewise, patients who identify strongly with their own cultural background may express lower satisfaction for provider care.¹¹ These findings suggest that patient healthcare satisfaction may be influenced by the customs, attitudes, and backgrounds of both the provider and the patient. As a result, cultural competency and awareness is now strongly emphasized for health professionals and researchers working with diverse patient populations.^10,12–15

In the field of telehealth, there are very few studies that explore the interaction between new modes of service delivery and culture. Most of the literature in this area consists of descriptive studies and case reports with minority groups.^4,16,17 To date in telepsychiatry, only one article has specifically addressed the interaction between technology and culture,¹⁵ emphasizing the unique differences that telepsychiatry may face compared to traditional health delivery mechanisms; these differences span many areas such as patient/provider location, communication, trust, and confidentiality issues. Due to the paucity of research in this area, we cannot precisely say how people of diverse backgrounds will react to the use of telehealth technologies. Because telepsychiatry—typically done through videoconferencing—is a different delivery mechanism than that to which most patients are accustomed, cultural differences between the patient and the provider may be more noticeable.

The purpose of this study was to examine, in a prospective controlled fashion, the acceptability to an American Indian population of telepsychiatric assessments conducted over live interactive videoconferencing. The study was designed to focus on the usability of the technology, the interviewee/interviewer interaction, cultural competence, and satisfaction.

Materials and Methods

The study was conducted by the University of Colorado at Denver and Health Science Center's (UCDHSC) American Indian and Alaska Native Programs. It was conducted in conjunction with a study designed to examine the reliability of the Structured Clinical Interview for DSM-III-R (SCID) for psychiatric assessments over a real-time videoconferencing telepsychiatry link (¼T1 or 384 kilobits per second [Kbps] using videoconferencing units with remote capacity) between UCDHSC and a rural American Indian community.

Our study population consisted of 53 American Indian male Vietnam Theater and Era veterans from a Northern Plains tribe. A power analysis indicated that 50 subjects were needed for the purposes of the parent study. The mean and median age was 54, with a range of 46–71. Over 90% (48) of the sample had been married in the past, and 32% (17) were currently married. Approximately half of the sample (28) completed some form of higher education. With respect to ethnic identity, 67% (31) of the subjects reported high Indian identity and 17% (8) reported both high Indian and high White identity. Using a test-retest design in a controlled fashion, the SCID was administered to participants both face-to-face and by real-time interactive videoconferencing on two separate occasions by different interviewers within a 2-week period. For half of the interviews, face-to-face administration preceded the videoconferencing administration; for the other half of the interviews, videoconferencing administration preceded face-to-face administration. UCDHSC Institutional Review Board and tribal approvals were obtained prior to the project's initiation. After a complete description of the study to participants, written informed consent was obtained. Overall, SCID assessment by live interactive videoconferencing did not significantly differ in reliability from face-to-face assessment in this population. The participants received a version of the SCID for DSM-III-R that was modified for the American Indian Vietnam Veteran's Project and further adapted in the American Indian Services Utilization, Psychiatric Epidemiology, Risk and Protective Factors Project¹⁸; these minor modifications made the SCID more culturally relevant for an American Indian population. For this study, two psychiatrists (both with extensive experience working with American Indians and PTSD in cross-cultural settings) were trained to administer the SCID in the same manner as in the American Indian Vietnam Veterans Project and the American Indian Services Utilization, Psychiatric Epidemiology, Risk and Protective Factors Project. Videotapes created specifically for the latter projects were used to assess interrater reliability, with an average kappa >0.80 obtained by both interviewers. Further details of the sample and methods of this study can be found elsewhere.¹⁹

Prior to initiation of the study, a telepsychiatry process measure was developed to assess the domains of the usability of the technology, patient/provider interaction, cultural competence of the interview, and patient satisfaction. A team from UCDHSC^* worked together to develop this process measure (Table 1), which included 26 items, with each item employing a self-report 5-point Likert scale ranging from a low (negative) score of 1 to a high (positive) score of 5. The process measure was designed to be administered after both interviews, and the questions on both were identical except for changes in wording to reflect the interview condition (telehealth versus in person). For example, in the telehealth interview, participants were asked “Was the doctor's image on the television an acceptable size and distance for you” and in the in-person interview, participants were asked “How comfortable were you with the distance the doctor sat from you?” Two additional questions were asked regarding satisfaction with the telepsychiatry intervention. In order to assess interviewers' perceptions of participants' reactions to each interview, the interviewers completed a survey that included items that paralleled specific questions asked of the interviewees (Table 1).

Table 1.

Telepsychiatry Process Measure Domains and Questions

		Intervieweea		Interviewer
Domains		In-Person Mean (SD)	Tele-Psych Mean (SD)	In-Person Mean (SD)	Telepsych Mean (SD)
Usability	To what extent do you feel the doctor was able to hear you over the video system?	4.10 (0.9)	3.96 (0.9)	NA	NA
	To what extent were you able to hear the doctor over the video system?	4.08 (0.9)	3.87 (0.8)	NA	NA
	To what extent were you able to see the doctor over the video system?	4.27 (0.9)	3.96 (0.9)	NA	NA
	Overall I thought the picture quality of the video system was …	NA	4.33 (0.8)	NA	NA
	Before this video interview, how did you feel about using the video system?	3.71 (1.4)	3.59 (1.4)	NA	NA
	How did using the video system make you feel during this interview?	NA	3.72 (1.3)	NA	3.63 (0.9)
	Was the doctor's image on the television an acceptable size and distance for you?	4.25 (1.0)	4.64 (0.7)	NA	NA
Patient/provider interaction	How comfortable did you feel during this video interview?b	3.94 (1.2)	4.17 (1.1)	3.73 (1.4)	3.44 (4.17)
	How did the doctor treat you during this interview?	4.92 (0.4)	4.79 (0.7)	NA	NA
	How sensitive was the doctor to your feelings?	4.53 (1.0)	4.40 (1.0)	NA	NA
	How carefully did the doctor listen to what you had to say?	4.58 (0.7)	4.33 (0.8)	NA	NA
	To what extent did you understand the doctor's questions?b,c	4.65 (0.6)	4.43 (0.3)	4.21 (0.6)	4.06 (0.6)
	To what extent did you feel you could trust the doctor?b,c	4.58 (0.8)	4.55 (0.8)	3.33 (0.8)	2.96 (0.7)
	How comfortable were you with the direct questions that the doctor asked you?	4.29 (1.0)	4.11 (1.0)	NA	NA
	How comfortable were you with the amount of eye contact you had with the doctor?	4.28 (0.9)	4.21 (0.9)	NA	NA
	How comfortable were you with the tone of voice of the doctor?	4.61 (0.8)	4.57 (0.7)	NA	NA
	How comfortable were you with the speed at which the doctor spoke?	4.58 (0.7)	4.47 (0.9)	NA	NA
Cultural competence	To what extent were you asked how race or ethnicity might affect the interview?	3.01 (1.5)	2.85 (1.4)	NA	NA
	To what extent did you feel that your race or ethnicity affected how you were treated during the interview?	2.47 (1.6)	2.56 (1.4)	NA	NA
	In what manner was your culture addressed by the questions that you were asked?	4.36 (0.9)	4.25 (0.9)	NA	NA
	How respectful was the interview of your cultural ways?	4.38 (0.9)	4.29 (0.9)	NA	NA
	How were the questions about your present experience asked?	4.30 (1.0)	4.10 (1.0)	NA	NA
	How were the questions about your past experiences asked?	4.21 (1.0)	4.10 (1.0)	NA	NA
	How were the questions about your future expectations asked?	4.15 (1.0)	4.20 (0.9)	NA	NA
	To what extent did the doctor seem to know about your culture?	3.64 (1.1)	3.57 (1.1)	NA	NA
Satisfaction	Overall, how satisfied were you with this (video) interview?b,c	4.68 (0.8)	4.59 (0.8)	3.81 (0.8)	3.61 (0.9)
	To what extent did this video interview affect your future willingness to use a video system for health care?b	NA	4.28 (0.8)	NA	3.72 (0.6)
	To what extent did this video experience affect your preference to use a video system for healthcare in the future?	NA	3.66 (0.9)	NA	NA

^aThere were no statistically significant differences between mean responses for telepsychiatry versus in-person for any of the items compared among the interviewees based on paired-sample t-tests (p > 0.01).

^bStatistically significant difference between interviewee and interviewer rating for telepsychiatry based on paired-sample t-tests (p < 0.01).

^cStatistically significant difference between interviewee and interviewer rating for in-person based on paired-sample t-tests (p < 0.01).

NA, not available.

Upon the conclusion of both interviews, participants completed an ethnic identity survey consisting of 11 questions that measured the extent to which participants identified with American Indian culture versus White culture. Participants were also administered a face-to-face debriefing interview by a research assistant, composed of nine open-ended questions. This interview was meant to confirm participants' responses on the process measures and convey feedback that participants might have felt more comfortable providing to the research assistant, who was a tribal member and fellow veteran.

Mean responses to each individual item in the process measure were compared between each participant's in-person interview and their live telehealth interview using paired-sample t-tests. For those items with a corresponding item asked of the interviewer, mean responses were compared between the interviewee (participant) and the interviewer, for both in-person interviews and videoconferencing. Due to the large number of comparisons being made, a conservative p value of 0.01 was used to indicate statistical significance.

Results

Participant Satisfaction

We used the process measure to examine differences in patients' satisfaction between the telehealth administration of the SCID versus the in-person administration of the SCID. Table 1 summarizes the process measure items by interview modality for participant and interviewers. Overall, the mean participant satisfaction rates assessed by the process measure were high, ranging from 2.5 to 4.8 for the telehealth interview to 2.5 to 4.9 for the in-person interview. Among this group, there were no statistically significant differences between their responses to the in-person interview survey and the telehealth interview survey.

The debriefing interviews were reviewed for each individual, and general characterizations of responses were made and classified. Overall, 94% (50) of the subjects had a general positive response to the videoconferencing, with 96% (51) indicating that the equipment was easy to use and only a minority finding it distracting 15% (8). Forty-five percent (24) indicated that they preferred the live interview, 20% (11) reported that they preferred the video interview, and 34% (18) did not prefer one over the other. Ninety-two percent (49) would like to use the technology again.

In order to examine the potential influence of culture on the acceptance of telehealth-based interviews, we looked at the results of eight questions in the process measure and responses to the debriefing interview. As noted above, there were no significant differences detected among any of the process measure items. In the debriefing interview, 18% percent (10) of participants felt that there might be specific factors in working with Indian people that could affect the use of telepsychiatry. Of these, 4 participants felt that general cultural factors might inhibit some patients, and 2 indicated that a lack of exposure or comfort to technology might impede telepsychiatric interactions.

Interviewer Satisfaction

The interviewers' ratings of perceived interviewee satisfaction were universally lower than the interviewee ratings, with statistically significant differences for each of the four questions (Table 2). Interviewers indicated lower scores than did the interviewees regarding how comfortable the interviewee was during the interview, interviewees' overall amount of satisfaction with the interview, and whether the interviewee understood the physician's questions and trusted the physician.

Table 2.

Interviewer Interpretation of Interviewee Satisfaction

	In-Person			Telepsychiatry
	Interviwer Mean (SD)	Interviewee Mean (SD)	p-Value	Interviewer Mean (SD)	Interviewee Mean (SD)	p-Value
1. How comfortable did the interviewee appear during the video interview?	3.73 (1.4)	3.94 (1.2)	0.64	3.44 (1.2)	4.17 (1.1)	0.0001
2. To what extent did the interviewee appear to understand your questions?	4.21 (0.6)	4.65 (0.6)	<0.0001	4.06 (0.6)	4.43 (0.3)	0.0006
3. To what extent did the interviewee feel that he could trust you?	3.33 (0.8)	4.58 (0.8)	<0.0001	2.96 (0.7)	4.55 (0.8)	<0.0001
4. Overall, how satisfied do you think the interviewee was with this interview?	3.81 (0.8)	4.68 (0.8)	<0.0001	3.61 (0.9)	4.59 (0.8)	<0.0001
5. How comfortable was the interviewee with using the video system during the interview?	NA	NA		3.63 (1.0)	3.72 (1.3)	0.7826
6. To what extent do you think the interviewee will be willing to use telemedicine in the future?	NA	NA		3.72 (0.6)	3.66 (0.9)	0.5765

NA, not available.

We also compared interviewers' ratings of perceived interviewee satisfaction between the type of interview (in-person versus telehealth). One question, “To what extent did the interviewee feel that he could trust you?”, showed a statistical difference (p = 0.0017), favoring the in-person interview. Interviewees, however, did not rate their level of trust differently between interview type.

Discussion

The primary focus of this research project was to determine whether the administration of a semistructured diagnostic interview would be acceptable to an American Indian population if given over telehealth. Although the participants responded to potential cultural differences through both rated items and open-ended questions, the results consistently demonstrated that telehealth administration of the SCID did not affect participants' satisfaction with the interview. In addition, participant attitudes did not differ significantly between telehealth testing and in-person testing with regard to the usability of the assessment, patient/provider interaction, or overall satisfaction. These findings suggest that telehealth-based semistructured interviews can be effectively used with American Indian participants. This information is important because it suggests that issues pertaining to location, communication, trust, and confidentiality do not differ between videoconferencing and in-person treatment. In the absence of such differences, the use of telehealth technologies is supported for populations that do not have easy access to specialized mental health services.

Of additional interest in this study is the interviewers' lower rating of interviewees' satisfaction and comfort, particularly with the telehealth interviews. There are several confounding factors that could explain the lowered estimate of participant satisfaction, including the limited number of interviewers (n = 2), biases, and misinterpretation of cues from interviewees. Nevertheless, this does raise some intriguing questions that have yet to be addressed in the literature relating to the possible impact of videoconferencing on interpretation of data by providers working in cross-cultural settings. In particular, does telepsychiatry impact clinicians' ability to accurately assess their patients in nonstructured interviews (either diagnostic or ongoing treatment)?

This study has several limitations including the inherent biases of self-rated measures and Likert scales. It is also important to note that we conducted this study with one tribe, and the ethnic homogeneity of the sample may limit the ability to generalize these findings to other populations, including other American Indians and veterans.

Despite its limitations, this study represents an important first step as the only controlled study on acceptability of telepsychiatry to American Indians, as well as provider perception of patient reaction to videoconferencing. Overall, telepsychiatry was well received and comparable in level of patient comfort, satisfaction, and cultural acceptance to in-person interviews. The level of acceptance is notable among this sample with its high level of American Indian identity. This is particularly relevant to ongoing telepsychiatric work with American Indian populations⁴ and other populations that rely on alternative delivery mechanisms for obtaining essential health services.

Acknowledgments

This study was supported by grants 1F32 MH63527-01A1 (National Institute of Mental Health) and P60 MD000507 (American Indian and Alaska Native Health Disparities). The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the Indian Health Service.