Evaluation of Telehealth for Preclinic Assessment and Follow-Up in an Interprofessional Rural and Remote Memory Clinic

Abstract

Using data from a sample of 169 patients, this study evaluates the acceptability and feasibility of telehealth videoconferencing for preclinic assessment and follow-up in an interprofessional memory clinic for rural and remote seniors. Patients and caregivers are seen via telehealth prior to the in-person clinic, and followed at 6 weeks, 12 weeks, 6 months, one year, and yearly. Patients are randomly assigned to in-person (standard care) or telehealth for the first follow-up, then alternating between the two modes of treatment, prior to 1-year follow-up. On average, telehealth appointments reduce participants’ travel by 426 km per round trip. Findings show that telehealth coordinators rated 85% of patients and 92% of caregiversas comfortable or very comfortable during telehealth. Satisfaction scales completed by patient-caregiver dyads show high satisfaction with telehealth. Follow-up questionnaires reveal similar satisfaction with telehealth and in-person appointments, but telehealth is rated as significantly more convenient. Predictors of discontinuing follow-up are greater distance to telehealth, old-age patient, lower telehealth satisfaction, and lower caregiver burden.

Introduction

The development of effective strategies for delivering health care services to seniors in rural communities is a growing concern. Global aging is occurring at an unprecedented pace, with the number of people age 65 and older expected to increase from 500 million world-wide to 1 billion by 2030 (National Institute on Aging, 2007). Aging and migration have changed the composition of rural populations, as the proportion of older people grows. In the United States, for example, 15% of the 50 million people living in rural America are aged more than 65 years (Jones, Kandel, & Parker, 2007). In Canada, 33% of seniors reside in predominantly rural regions (Dandy & Bollman, 2008), with the proportion of seniors increasing with distance from urban centres. “Remote” rural areas have a higher proportion of seniors (16.1%) than cities (13.2%) or rural areas close to cities (13.9%; Statistics Canada, 2007). Rural-urban differences are even more dramatic in the Canadian province of Saskatchewan, where seniors make up 14.7% of the population of cities, compared to 22.4% of towns and 21.7% of villages (Elliot, 2007). The aging of rural populationshas implications for health service delivery for older people, including those with dementia and other age associated illnesses. In a national study of prevalence and risk factors for dementia in Canada (Canadian Study of Health & Aging Working Group, 1994), prevalence increased from 8.0% in those ≥ 65 years, to 34.5% in those ≥ 85 years.

Early diagnosis of dementia is critical for timely and effective management and family support, yet access to specialized services is limitedin rural and remote areas, and previous research suggests that primary care physicians are not confident about diagnosing dementia (Turner et al., 2004). Primary care physicians want specialists to assist them with diagnosis and treatment planning (Williams, 2000), and have identified limited access to consultants and diagnostic services as impediments to diagnosis and management of dementia (Iliffe & Wilcock, 2005; Teel, 2004). This paper reports on a research demonstration project aimed at improving access to diagnosis of people with early stage dementia in rural and remote areas of the Canadian Province of Saskatchewan. The project involved the implementation and evaluation of a one-stop interprofessional Rural and Remote Memory Clinic (RRMC) that uses telehealth videoconferencing for preclinic assessment and follow-up to increase access to early assessment and diagnosis by a specialist team (Morgan et al., 2009). Telehealth, or telemedicine, refers to the use of electronic telecommunications and information technologies to provide health care services when health professionals and patients are separated by distance. In this article we focus on the evaluation of the acceptability and feasibility of the telehealth videoconference component by patients, families, and telehealth coordinators. Results of the evaluation of the one-stop in-person clinic will be reported separately.

A systematic review of patient satisfaction with telehealth (Williams, May, & Esmail, 2001) found a number of limitationsin the design of these studies, including small samples (median = 30), descriptive designs, use of non-standardized scales that were developed for specific studies and described without reliability and validity information, lack of qualitative research, research conducted in “unnatural” settings, and lack of information about satisfaction with telehealth in specific health conditions. Some studies examining the use of telecommunications technology with older patients suggest that geriatric patients are comfortable using telehealth equipment (Bratton & Short, 2001), but others have argued that more research is needed to determine how age-related declines in sensory, motor, and cognitive abilities affect interactions between older patients and health-care providers when technology is used (Stronge, Rogers, & Fisk, 2007). Studies evaluating the use of telehealth videoconferencing suggest that psychometric assessment (Ball & Puffett, 1998, Loh, Donaldson, Flicker, Maher, & Goldswain, 2007; Montani et al., 1996; Saligari et al., 2002; Tyrrell, Couturier, Montani, & Franco, 2001) and neurological examination (Craig, McConville, Patterson, & Wootton, 1999) can be done using this technology. Limitations of these studies include small samples (10 – 23 participants) and the fact that many were conducted in experimental rather than real-life settings. Other than from our own setting (e.g., Crossley, Morgan, Lanting, Dal Bello-Haas, & Kirk, 2008; Morgan et al., 2009), there are few reports of memory clinics specifically for rural residents, or the use of telehealth as a component of service delivery within a functional memory clinic. Although there is evidence that some aspects of cognitive functioning can be assessed via telehealth (McEachern, Kirk, Morgan, Crossley, & Henry, 2008), in-person assessment by an interprofessional team is typical of memory clinics (Lindesay, Marudkar, Diepen, & Wilcock, 2002) because it allows for a full hands-on assessment by all team members. The RRMC was designed to use telehealth for preclinic assessment and follow-up appointments, in conjunction with a coordinated, interprofessional, in-person assessment. Although patients must travel for the full-day clinic, the streamlined one-stop approach means that only one trip is requiredfor patients to receive a complete assessment and diagnosis.

Methods

This research is part of a larger multi-study research program focusing on improving the care of persons with dementia in rural and remote areas. The objectives of the RRMC study are as follows: a) to increase the availability of dementia care in rural and remote settings; b) to determine the acceptability and feasibility of a one-stop clinic and of telehealth vs. conventional follow-up; and c) to develop culturally appropriate assessment protocols for assessment of dementia in non-English speaking older adults, including Aboriginal seniors. The study was approved by the University of Saskatchewan Behavioral Research Ethics Committee. During the first year the team held consultation meetings with health care providers in all 13 rural and remote communities that had a telehealth site operated by the provincial network, and two sites operated by First Nations bands. These meetings highlighted the challenges of delivering specialist care to dispersed populations, especially remote northern communities (McBain & Morgan, 2006).

Setting

The province of Saskatchewan has a population of approximately 1 million people dispersed over an area of 652,000 km² (252,000 mi²). Over 52% of the population lives in “predominantly rural regions” (Bollman & Clemenson, 2008), defined by the Organization for Economic Co-operation and Development (OECD; 1994) as communities having more than 50% of its population living in rural communities with a population density less than 150 persons per square kilometer (km²). Health care, including care delivered by telehealth, is publically-funded and delivered through 13 regional health authorities (RHAs). Population densities across the 13 RHAs range from 0.1 to 9.2 people per km²(median 1.6; Statistics Canada, 2009). The highest densities are found in RHAs that include larger urban centres, but even these densitiesare low due to the inclusion of large surrounding rural areas. The majority of the population lives in the southern third of the province, which is primarily agricultural. The northern two-thirds of the province is covered by parkland, forest, and lakes. The three most northern RHAs cover 40% of province’s area, with only 3.3% of the population, and are described as “remote” for the purposes of this research. The remaining ten RHAs are described as “rural.” Within the three northern RHAs, 86% of the population self-identified as Aboriginal in the 2006 census (Statistics Canada, 2009). Consistent with the population as a whole, the Aboriginal population in Canada is aging. Between 2001 and 2006 the 55–64 age cohort of Aboriginal seniors increased 53.8% and the 65+ age cohort increased 43.0%. In 2006 there were 12,340 Aboriginal people aged more than 55 in Saskatchewan (Rosenberg, Wilson, Abonyi, Wiebe, & Beach, 2008).

The provincially funded and managed Telehealth Saskatchewan network was launched in 1999 and currently has equipment in 63 health facilities in 49 communities. A trained coordinator at each network site provides support for a range of clinical, educational, and research applications. Portable, high performance videoconferencing equipment is used in all locations. Real-time connections are made across a province-wide broadband internet protocol (IP) network dedicated to health, executive government, and education-based connections. The 768 kbps baseline speed, enhanced with Quality of Service (QOS)technology to protect the high quality of video images and sound, provides secure priority queuing for telehealth applications. A general videoconferencing camera is used for the interview portion of the RRMC preclinic and follow-up assessments. A specialized high-quality camera allows for a detailed look at the patient’s writing or drawing during mental status examinations.

Participants

Characteristics of the first 169 patients attending the full clinic day assessment are reported in Table 1. The table also includes descriptions of the sub-samples used for the analyses reported in this article (participants who continued in the study at 6 months, n = 41; participants who had discontinued prior to 6 months, n = 23; participants who alternated between telehealth and in-person follow-up at 6 and 12 weeks, n = 29; and those who alternated between in-person and telehealth follow-up at 6 and 12 weeks (n = 29). Of the 169 participants, approximately 40% were men and the mean age was 72.5 years (SD = 10.9 years). Canada was named as the country of birth for 93% of patients. The majority (69.9%) reported European ancestry, with smaller numbers identifying as First Nations (5.1%) andMétis (2.9%). English was spoken at home by 93.8%. Years of formal education ranged from 3 to 19 (M = 10.8, SD = 3.0). The majority of patients reported marital status as married or common law (72.5%), with widowed (21.3%) as the next largest category. Most patients were living with only their spouse/partner (62.5%), 18.8% were living alone, and most (82.5%) were living in a family-owned home. Alzheimer’s Disease was the most common diagnosis made at the clinic day (35.4%), followed by mild cognitive impairment (15.6%), no dementia (10.6%), dementia related to multiple etiologies (10.0%), and smaller numbers of other dementia subtypes.

Table 1.

Demographic Characteristics of the First 169 Patients

	All Patients (N=169a)	Continued to 6-month follow-up (N=41)	Discontinued by 6-month follow-up (N=23)	Alternated THb to IPc appointment (N=29)	Alternated IP to TH appointment (N=29)
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Age	72.5 (10.9)	73.0 (10.5)	77.2 (9.4)	71.3 (10.3)	72.4 (10.0)
Formal education (years)	10.8 (3.0)c	11.1 (3.2)d	9.3 (3.3)e	10.7 (3.2)f	11.3 (2.8)
Kms to TH (1 way)	37.4 (47.6)	22.0 (33.4)	54.4 (50.8)	42.6 (54.2)	45.6 (47.7)
Kms to clinic (1 way)	250.8 (112.8)	241.7 (112.2)	270.5 (117.1)	218.6 (102.8)	256.4 (108.6)
Kms saved by TH (1 way)	213.4 (114.5)	219.7 (109.8)	216.1 (114.7)	176.0 (73.0)	210.8 (106.8)
	Frequency (%)	Frequency (%)	Frequency (%)	Frequency (%)	Frequency (%)
Sex	N=169	N=41	N=23	N=29	N=29
Male	67 (39.6)	17 (41.5)	12 (52.2)	12 (41.4)	13 (44.8)
Female	102 (60.4)	24 (58.5)	11 (47.8)	17 (58.6)	16 (55.2)
Country of birth	N=159	N=38	N=20	N=28	N=29
Canada	148 (93.1)	34 (89.5)	17 (85.0)	26 (92.9)	28 (96.6)
Other	11 (6.1)	4 (10.5)	3 (15.0)	2 (7.1)	1 (3.4)
Ancestry	N=136	N=33	N=15	N=24	N=25
First Nations	7 (5.1)	2 (6.1)	--	--	--
Métis	4 (2.9)	--	--	--	--
European	95 (69.9)	27 (81.8)	10 (66.7)	23 (95.8)	20 (80.0)
Other	30 (22.1)	4 (12.1)	5 (33.3)	1 (4.2)	5 (20.0)
Language most often spoken at home	N=144	N=36	N=18	N=28	N=26
English	135 (93.8)	32 (88.8)	17 (97.6)	28 (100.0)	26 (100.0)
Cree	5 (3.5)	2 (5.6)	--	--	--
Other	4 (2.7)	2 (5.6)	1 (2.4)	--	--
Marital status	N=160	N=38	N=21	N=28	N=29
Married/common law	116 (72.5)	25 (65.8)	17 (81.0)	20 (71.4)	20 (69.0)
Never married	6 (3.8)	1 (2.6)	--	2 (7.1)	--
Separated/divorced	4 (2.5)	--	--	--	1 (3.4)
Widowed	34 (21.3)	12 (31.6)	4 (19.0)	6 (21.4)	8 (27.6)
Living arrangement	N=160	N=38	N=21	N=28	N=29
With spouse/partner	100 (62.5)	20 (52.5)	17 (81.0)	18 (64.3)	18 (62.1)
Alone	30 (18.8)	8 (21.4)	3 (14.3)	7 (25.0)	8 (27.6)
With spouse & family	25 (15.6)	7 (18.3)	1 (4.7)	3 (10.7)	2 (6.9)
Other	5 (3.1)	3 (7.8)	--	--	1 (3.4)
Current housing	N=160	N=38	N=21	N=28	N=29
Family-owned home	132 (82.5)	28 (73.7)	19 (90.6)	23 (82.1)	26 (89.7)
Rent home/apartment	11 (6.9)	4 (10.5)	1 (4.7)	3 (10.7)	1 (3.4)
Seniors complex	8 (5.0)	2 (5.3)	1 (4.7)	2 (7.1)	1 (3.4)
Other	9 (5.6)	4 (10.5)	--	--	1 (3.4)
Clinic Day Diagnosis	N=160	N=39	N=21	N=27	N=29
Alzheimer’s Disease	56 (35.0)	9 (23.1)	10 (47.6)	8 (29.6)	12 (41.4)
Mild Cog. Impairment	25 (15.6)	5 (12.8)	4 (19.0)	7 (25.9)	5 (17.2)
No dementia (normal)	17 (10.6)	5 (12.8)	1 (4.7)	2 (7.4)	3 (10.3)
Dementia Mult.Etiol.g	16 (10.0)	7 (17.8)	1 (4.7)	3 (11.1)	3 (10.3)
Frontotemporal	12 (7.5).	4 (10.3)	3 (14.2)	1 (3.7)	1 (3.4)
Lewy Body Dementia	10 (6.3)	4 (10.3)	--	3 (11.1)	--
Vascular Cog. Imph	10 (6.3)	1 (2.6)	1 (4.7)	--	2 (6.9)
Dementia Med/Otheri	9 (5.6)	3 (7.7)	1 (4.7)	2 (7.4)	2 (6.9)
Dementia not specified	5 (3.1)	1 (2.6)	--	1 (3.7)	--
	N=33	N=11	N=3	N=2	N=4
Requested change in type of follow-up	33 (19.5)	11 (26.9)	3 (13.0)	2 (7.1)	4 (13.7)
From IP to TH	24 (14.2)	11 (26.9)	3 (13.0)	2 (7.1)	4 (13.7)
From TH to IP	9 (5.3)	--	--	--	--

Note: TH = Telehealth Appointment; IP = In-person appointment. Types of participants involve the following categories: Attending full-day clinic and subsamples of who continued to 6-month Follow-up, discontinued before 6-month follow-up, alternated between telehealth and in-person follow-up, alternated between in-person and telehealth follow-up.

^aSample may be small for some variables due to missing data.

^bMissing data for 18 patients

^cMissing data for 5 patients

^dMissing data for 2 patients

^eDementia due to multiple etiologies (e.g., due to a combination of Alzheimer’s disease, Vascular Dementia, alcohol-induced dementia, Parkinson’s disease, or other medical conditions).

^fVascular cognitive impairment, including Vascular Dementia

^gDementia due to medical conditions or other dementias (e.g., hydrocephalus, dementia associated with Parkinson’s Disease, Huntington’s Disease, alcohol-induced dementia, etc.)

RRMC Intervention and Evaluation Design

The RRMC was designed to provide interprofessional assessment of dementia, and to streamline the assessment process, reduce repeated travel over long distances, and shorten the time to diagnosis. Eligibility for the clinic was limited to non-institutionalized Saskatchewan residents living more than 100 kms (62 miles) from tertiary care (i.e., outside the two major cities of Saskatoon and Regina), and referred to the clinic in Saskatoon because of a concern about a memory disorder. Patients are referred by their family physicians to the clinic neurologist and booked by the clinic nurse. The clinic follows a family-oriented approach. All interested family members and other caregivers are invited to attend all appointments. Details about the development, operation, and evaluation of the clinic are reported elsewhere (Morgan et al., 2009) but are described here briefly to provide a context for the telehealth evaluation. Approximately 8 to 10 patients are seen at the weekly one-day clinic session, including those seen for preclinic assessment, full-day evaluation, and follow-up.

An initial preclinic assessment is carried out via telehealth videoconferencing by the nurse and neuropsychologist. Information provided by the patient and family during the 30-minute session informs planning for the subsequent full-day clinic so that team members can tailor their assessments. A requisition for blood work is faxed so that results are available by clinic day. The telehealth session also provides the opportunity for patient and family to meet team members and learn about what to expect on clinic day. Two new patients are evaluated in Saskatoon on clinic day, where they are seen by a neurologist, neuropsychology team, geriatrician, physical therapist, and neuroradiologist. The clinic day ends with an interprofessional case conference. Referring physicians are invited to participate via telephone conference call.

Figure 1 illustrates the evaluation research design and flow of patients through the clinic. Follow-up assessments are conducted at 6 weeks, 12 weeks, 6 months, 1 year, and thenyearly. To evaluate the feasibility and acceptability of telehealth videoconferencingduring the first four years of operation, patients were randomly assigned to either in-person (standard care) or telehealth for the first follow-up contact at 6 weeks. They then alternated between the two modes until the one year follow-up assessment, except for patients in remote northern communities who were offered telehealth for all follow-up appointments due to the challenges of long-distance travel (average of 938 kms per round trip). This exception for northern patients is reflected in the larger sample for telehealth at the first follow-up as shown in Figure 1. The sample size at each follow-up stage varies due to patterns of missed appointments and requests fora change in type of follow-up (details reported in Results section).

Figure 1.

Evaluation design and flow of patients through the Rural and Remote Memory Clinic

^aSample size at each follow-up varies due to the pattern of missed appointments and requests for change in type of follow-up (see Table 1). ^bTelehealth group larger because remote northern patients are offered telehealth for all follow-up appointments.

This single case design with repeated measures and randomly assigned order was selected based on several factors, including recommendations receivedduring the community consultation visits prior to implementing the clinic. We were advised that a between-group (telehealth vs. in-person) randomized design would jeopardize buy-in by referring physicians because telehealth would not be used in control communities and physicians would not refer if patients had to make repeated trips over long distances. The strengths of the study design include excellent control for individual differences (i.e., patients and caregivers serve as their own control), increased power, increased community and referring physician buy-in, and use of all available telehealth sites. As all participants experience both telehealth and in-person assessments, this design also provides greater ability to assess patient and family satisfaction and acceptability of the two delivery modes, which is the aim of the study. The design was feasible because we were not expecting the telehealth delivery mode to affect clinical outcomessuch as patient cognitive function or behavioral symptoms. The random assignment to telehealth or in-person visits at the beginning of the study (which subsequently alternate) controls for potential differences in satisfaction related to a particular pattern of follow-up.

Prior to one year, the follow -up appointments are conducted by the neurologist, who completes a brief assessment including the Mini Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975). At one year and yearly thereafter, patients return to Saskatoon for a more extensive assessment by the neurologist, physical therapist, and neuropsychology team. Individuals with advanced illness who are unable to complete neuropsychological testingat one year are seen via telehealth, and the Modified Mini-Mental State Exam (3MS; Teng & Chui, 1987) is administered. The alternating telehealth and in-person assessments in the first year of follow-up allowed us to examine whether MMSE scores were comparable when conducted via telehealth versus face-to-face. Analysis of data from the first 71 Rural and Remote Memory Clinic patients with both 6-week and 12-week follow-up data showed that MMSE scores did not differ significantly between the two modes (McEachern, et al., 2008).

Measures

Telehealth satisfaction measures

In addition to a comprehensive set of clinical measures, a number of evaluation instruments were used to collect data on patient/caregiver and telehealth coordinator assessments of telehealth and the one-stop clinic (see Morgan et al., 2009, for detailed information about clinical measures and components of the interprofessional assessment.) In this article we report on data from the four instruments used to assess the telehealth delivery: (1) the telehealth session form, (2) the telehealth coordinator evaluation form, (3) the follow-up evaluation form, and (4) the telehealth satisfaction questionnaire. In Saskatoon, a team member attending the session completed the telehealth session form, which was used to document information about who attended the session, the nature of any technical problems, and appointment length. The rural telehealth coordinators completed and returned the brieftelehealth coordinator evaluation form for each appointment, in which they rated both patient and caregiver’s observed comfort on 5-point Likert scales (1 = very uncomfortable; 5 = very comfortable). They also provided information on their approaches for enhancing the telehealth experience and reported any concerns raised by patient or caregivers. At all follow-up appointments (telehealth and in-person), patient-caregiver dyads completed the follow-up evaluation questionnaire in which they rated their overall satisfaction with the appointment (1 = very dissatisfied, 5 = very satisfied) and its convenience (1 = very inconvenient, 5 = very convenient). Open-ended questions were used to solicit comments on what was liked most and least about the appointment. Immediately following each telehealth appointment, patients and their caregivers completed the 16-item telehealth satisfaction questionnaire, which the telehealth coordinator returned by fax. The instrument has two components, a 12-item embedded scale and four additional questions. It was adapted from a questionnaire used in previous Saskatchewan Telehealth network evaluations (Linassi & Shan, 2005; Miller & Levesque, 2002) and a national First Nations telehealth research project (Health Canada, 2001). Twelve aspects of the telehealth appointment were rated on a 4-point scale (1 = poor, 2 = fair, 3 = good, 4 = excellent) and were summed to create a satisfaction scale. Higher scores indicate greater satisfaction. Internal consistency reliability (coefficient alpha) for the 12-item embedded scale was excellent at all assessment points: .88 for the preclinic assessment, and .91, .92, and .89, for the 6-week, 12-week, and 6-month follow-up assessments, respectively. The remaining four items asked whether the individual would use telehealth again and recommend it to another person, the distance from their home to the telehealth site, and the mode of transportation to the site. Space was provided for responses to open-ended questions.

Caregiver/family member measures

As part of the clinic day assessment, the family member or other accompanying person completes a questionnaire including a number of standardized measures relating to the patient and to the family member, including health, burden, and psychological distress. These caregiver measures were examined as predictors of continuing versus discontinuing follow-up. Physical and mental component scores were computed fromt he 12-item Short-Form Health Survey (SF-12; Ware, Kosinski, & Keller, 1996; Ware, Kosinoski, Turner-Bowker, & Gandeck, 2007) with higher scores indicating better caregiver physical and mental health. The 12-item version of the Zarit Burden Scale (Bédard et al., 2001; O’Rourke & Tukko, 2003) produces scores ranging from 1 to 48, with higher scores indicating greater burden. The 53-item Brief Symptom Inventory (BSI; Derogatis & Melisaratos, 1983) was used to assess caregiver psychological distress. Global Severity Index (GSI) scores for the BSI are standardized T-scores with a mean of 50 and a standard deviation of 10. GSI scores on the BSI range from 35 to 80, with higher scores indicating greater distress.

Analysis

Demographic data, information about who was present at the preclinic and follow-up appointments, occurrence of technical problems, and telehealth coordinator evaluation data were analyzed using descriptive statistics. For the follow-up evaluation questionnaire, a comparison of satisfaction and convenience ratings was conducted on a sub-group of 58 patient-caregiver dyads who completed the evaluation at both the 6- and 12-week follow-up appointments, and who alternated between telehealth and in-person appointments (n = 29), or from in-person to telehealth appointments (n = 29). Dyads with missing data at either time point, or who did not alternate (e.g., northern patients who were offered telehealth for all follow-up) were excluded. A crossover Latin square design was used for this comparison to control for subject variation and the order in which the two types of follow-up were received (Montgomery, 2001). Analyses of variance (ANOVA’s), with an alpha level of .05, were used for all statistical tests.

For the telehealth satisfaction questionnaire, a paired t-test was used to compare mean satisfaction scores at preclinic assessment and 6-month follow-up, for the 41 patients who participated at both time points. Because this analysis excluded those who had discontinued by 6 months, we then compared preclinic telehealth satisfaction scores of patients who had discontinued by 6 months (n = 23) to those who continued (n = 41). Because scores for the groups were not normally distributed, the nonparametric Mann-Whitney test was used to compare the median telehealth satisfaction scores of patients who continued versus discontinued. Univariate logistic regression wasused to further explore predictors of attrition, including preclinic telehealth satisfaction and distance to telehealth, patient variables (e.g., gender, age, stage of dementia), and caregiver variables (e.g., age, SF-12 physical and mental health component scores, burden, and psychological distress). The small samples (n=64, for some variables n=56) limited our ability to do multivariate analysis and interaction checks. Written responses to open-ended questions on three questionnaires (i.e., follow-up evaluation, telehealth satisfaction, and telehealth coordinator) were analyzed thematically.

Results

As described in Table 1, the mean distance from home to the telehealth site in the nearest hospital or health centre was 37.4 kms (SD = 49.1), compared to a mean distance of 250.8 kms (SD = 112.8) to Saskatoon for in-person appointments. On average, the distance saved by attending telehealth vs. in-person appointments was 213.4 kms (SD = 114.5). A request for a change in type of follow-up was made by 33 patients, with 24 patients requesting a change from in-person to telehealth, and 9 patients requesting a change from telehealth to in-person mode. Reasons for requesting a change to telehealth included either travel-related issues (e.g., time, cost, lack of transportation, inability to get time off work, difficulty traveling due to health) (n = 19), or patient-related issues (e.g., patient refused to come to in-person appointment but would accept telehealth, behavioral symptoms) (n = 5). For those requesting a change to in-person appointments, reasons given were: a) patient and/or caregiver were going to be in the cityon the day of the appointment for another reason (n = 2); b) distance to telehealth site was further or equal to the city (n = 4); c) patient preferred in-person (n = 2); and, d) and patient concerned about rural health providers knowing about his/her memory problems (n = 1). For patients who discontinued follow-up, the stages at which this occurred were: a) after full-day clinic (n = 10); b) after 6-week follow-up (n = 16); c) after 12-week follow-up (n = 13); d) after 6-month follow-up (n = 24); and, e) after 1 year (n = 15).

Preclinic and follow-up telehealth sessions

Table 2 describes who was present at the appointments. Family members and the local telehealth coordinator attended most sessions, whereas nurses, translators, and social workers attended infrequently. All technical problems before and during sessions were documented, although most were minor and often resolved prior to the session beginning. Frequencies of problems experienced at each time point are reported in Table 2. Examples of problems establishing communication included poor sound and visual quality, and were more likely to occur when a bridge (link between two telehealth systems) was required to connect to remote northern sites. Problems related to sound quality included occasional echoing or static, and sound breaking up or delayed. Fewer problems were recorded with respect to visual quality, with infrequent freezing of images, blurring, or delay. The mean duration of the preclinic appointments was 28.1 minutes (SD = 10.1), and the average duration of follow-up appointments was 13.7 minutes at 6 weeks, 14.4 minutes at 12 weeks and 12.2 minutes at 6 months.

Table 2.

Preclinic Assessment and Follow-Up Telehealth Appointments: Participants, Technical Problems, and Length of Sessions

	Preclinic assessment (n=162)	6-week Telehealth Appointment (n = 82)	12-week Telehealth Appointment (n = 75)	6-month Telehealth Appointment (n = 75)
	Freqa(%)	Freq (%)	Freq (%)	Freq (%)
Present at Rural/Remote Site
Family members	143 (88.3)	75 (91.5)	65 (86.7)	63 (84.0)
Telehealth Coordinator	158 (97.5)	81 (98.8)	73 (97.3)	72 (96.0)
Nurse	7 (4.3)	1 (1.2)	4 (5.3)	1 (1.3)
Other (translator, social worker)	1 (0.5)	0	1 (1.3)	0
Technical Problems with:
Establishing communication	13 (8.0)	2 (2.4)	7 (9.3)	7 (9.3)
Maintaining communication	1 (.6)	0	1 (1.3)	1 (1.3)
Sound quality	24 (14.8)	10 (12.2)	8 (10.7)	17 (22.7)
Visual quality	3 (1.9)	5 (6.1)	4 (5.3)	3 (4.0)
Length of session (Minutes) Mean (SD)	28.1 (10.1)	13.7 (5.7)	14.4 (5.8)	12.2 (4.1)

^aFrequencies are less than 169 due to missing data

Telehealth Coordinator evaluations

This analysis is based on data for 132 participants due to missing data (16 patients had no preclinic assessment; 12 forms were not returned by the rural telehealth coordinator, the preclinic assessment was conducted in person for 2 patients and via telephone for 7 patients). Although patients and families are informed that they can ask the rural telehealth coordinator to leave during the session, the coordinators indicated that they were usually present for the entire telehealth appointment (95.3% of sessions). The coordinators reported that they often (82.9% of sessions) took measures such as demonstrating equipment, explaining clinic procedures, and providing reassurance, to put the patient and family at ease and that these efforts were usually successful (98.9% of sessions). The rural coordinators evaluated patient comfort during the sessions and reported that most patients were comfortable (55.0%) or very comfortable (29.8%). One patient was rated as very uncomfortable, and a few patients were rated as uncomfortable (6.9%) or neutral (7.6%). The coordinators’ ratings of the family member or other person accompanying the patient indicated somewhat higher comfort levels in these individuals, with 51.2% rated as comfortable and 40.9% rated as very comfortable.

Follow-up evaluation questionnaire

The sample for this analysis was 58 patient/caregiver dyads who alternated between telehealth and in-person appointments (n = 29) or between in-person and telehealth appointments (n = 29) at 6-week and 12-week follow-up. There was no significant difference in mean satisfaction ratings between telehealth (M = 4.66, SD = 0.11) and in-person appointments (M = 4.48, SD = 0.09), F(1,56) = 2.05, p = 0.158. However, convenience ratings were significantly higher (better) for telehealth (M = 4.78, SD = 0.08) compared to in-person appointments (M = 3.76, SD = 0.15), F(1,56) = 37.78, p < 0.0001.

Telehealth satisfaction questionnaire

In addition to the satisfaction scale embedded in this questionnaire, there were four other questions regarding use of telehealth. All respondents indicated they would use telehealth again, and 99% of the participants indicated theywould recommend it to another person. The most common mode of transportation to the telehealth appointment was private vehicle (75%). The fourth question, distance to the telehealth site, was described under Participants (see Table 1). Mean scores for the individual questions on the 12-item satisfaction scale are reported in Table 3. This analysis is based on the full sample of 169 patients, although the sample size for individual scale items variesfrom 124 to 147 due to missing data. Responses are scored on a 4-point Likert scale, with high scoresindica ting higher satisfaction. Item means ranged from 3.05 (wait time for appointment) to 3.65 (how well telehealth staff answered questions about equipment). Frequencies for individual items also are reported in Table 3. Most items were rated as “good” or “excellent.” Summary scores (i.e., total scores) were computed for the scale, with possible scores ranging from 12 to 48. At the initial preclinic telehealth assessment the mean summary score was 42.1 (SD = 4.53).

Table 3.

Family-Client Satisfaction with Telehealth at Initial Preclinic Assessment: Item Means and Frequencies

Scale Items	Individual Scale Itemsa		Frequenciesb
			Good		Excellent
	Item Mean	(SD)	Freq.	(%)	Freq.	(%)
Voice quality (n = 146)	3.45	(0.64)	62	(42.5)	76	(52.1)
Visual quality (n = 146)	3.57	(0.51)	61	(41.8)	84	(57.5)
Personal comfort (n = 145)	3.41	(0.61)	70	(48.3)	68	(46.9)
Wait time for appointment (n = 141)	3.05	(0.91)	62	(44.0)	49	(34.8)
Ease of getting to telehealth department (n = 124)	3.44	(0.62)	57	(46.0)	61	(49.2)
Length of time with Memory Clinic team (n = 145)	3.41	(0.53)	80	(55.2)	62	(42.8)
Explanation of your treatment by the team (n = 145)	3.48	(0.53)	72	(49.7)	71	(49.0)
Thoroughness, skillfulness of team (145)	3.59	(0.51)	58	(40.0)	86	(59.3)
Courtesy, respect, sensitivity of team (n = 147)	3.73	(0.46)	37	(25.2)	109	(74.1)
How well privacy was respected (n = 142)	3.67	(0.47)	47	(33.1)	95	(66.9)
How well staff answered questions re: equipment (n = 144)	3.65	(0.49)	48	(33.3)	95	(66.0)
Overall treatment experience with telehealth (n = 145)	3.59	(0.49)	59	(40.7)	86	(59.3)

^aRange of possible scores for scale items was 1 (poor) to 4 (excellent)

^bFrequencies for Poor and Fair not given due to small cell sizes

A series of three analyses were conducted with a subsample of patients and caregivers who had complete data for preclinic, clinic day, and 6-month follow-up visits (n = 41), and those who had discontinued at any time prior to 6 months and who also had data for the preclinic and clinic day visit (n = 23). Most demographic and other study variables, which were included as predictors in the regression analysis, are collected at the full-day clinic. The demographic characteristics of these two subsamples are reported in Table 1. The only significant difference was distance to telehealth, with longer distance reported by those who discontinued follow-up (p = 0.010). There was a trend toward fewer years of formal education among those who discontinued (p = 0.059). To evaluate whether or not satisfaction with telehealth changed over time, a paired t-test was used to compare summary scores at preclinic assessment and 6-month follow-up for the 41 patients who participated at both time points. Meansatisfaction scores at preclinic assessment (M = 42.27, SD = 4.77) and 6-month follow-up (M = 43.59, SD = 4.47) were not statistically different (p = 0.15). Because this analysis included only patients who were continuing with follow-up, we then compared telehealth satisfaction summary scores at preclinic assessment of patients who continued at 6 months (n = 41) to those who had discontinued by 6-month follow-up (n = 23). Telehealth satisfaction scores were significantly higher for patients who continued, compared to those who did not (median scores 43 vs. 37), p = 0.013.

To further explore predictors of discontinuing follow-up in the clinic, univariate logistic regression analysis was conducted using telehealth satisfaction scores and other study variables presumed to influence whether or not patients continue to attend follow-up appointments (Table 4). Although patients who discontinued had lower preclinic telehealth satisfaction scores compared to those who continued at 6 months ( odds ration [OR]= 0.88, 95% confidence interval [CI]= 0.78, 0.98), distance to telehealth and patient age were also associated with discontinuing. There was an increasing risk of discontinuing when patients had to travel longer distances to the telehealth centre, with 8.7 times higher risk when they needed to travel ≥ 100 kms (OR= 8.75, 95% CI= 1.55, 49.33). There was a trend toward greater risk of discontinuing as the distance saved by using telehealth decreased. Patients who were 80 years and older were 5 times more likely to discontinue compared to those who were aged less than 70 years (OR= 5.12, 95% CI= 1.15, 22.73). Patient gender and stage of dementia did not show any significant relationship to discontinuing. Similarly, caregiver characteristics of age and SF-12 Physical and Mental health scores were not related to discontinuing. Interestingly, caregiver burden scores were significantly lower for patients who discontinued (OR= 0.87, 95% CI= 0.80, 0.95).

Table 4.

Univariate Logistic Regression Results for Patients who Discontinued Follow-up by 6 Months

Variable	Number of observations used in the analysis	P value	Odds ratio (95% Confidence Interval)
Gender (patient)a	64
Male		0.410	1.54 (0.55, 4.30)
Distance to telehealth (one way)b	64
50–99 KM		0.084	3.65 (0.84, 15.84)
≥ 100 KM		0.014	8.75 (1.55, 49.33)
Distance saved by telehealth (one way) c	64
100–199 KM		0.060	0.11 (0.01, 1.09)
≥ 200 KM		0.087	0.13 (0.01, 1.34)
Satisfaction Score	64	0.024	0.88 (0.78, 0.98)
Clinic day SF-12 Physical Component score	60	0.213	0.97 (0.92, 1.02)
Clinic day SF-12 Mental Component score	60	0.140	1.05 (0.98, 1.13)
Stage of Dementiad	56
Mild dementia		0.062	3.49 (0.94, 12.99)
Moderate/ severe dementia		0.911	1.09 (0.25, 4.62)
Patient age groupe	64
70–79 years		0.459	1.78 (0.38, 8.27)
≥ 80 years		0.032	5.12 (1.15, 22.73
Caregiver age groupf	60
50–59 years		0.359	0.46 (0.09, 2.41)
≥ 60 years		0.346	1.87 (0.51, 6.83)
Clinic Day caregiver burdeng	61	0.003	0.87 (0.80, 0.95)
Clinic day caregiver psychological distressh	61	0.143	0.96 (0.92, 1.01)

^aGender (patient): reference is female

^bDistance to telehealth: reference is < 49 KM

^cDistance saved by telehealth^: reference is <99 KM

^dStage of Dementia: reference is very mild dementia

^ePatient age group: reference is <70 years

^fCaregiver age group: reference is <50 years

^gZarit Burden Interview (12-item)

^hBrief Symptom Inventory (Global Severity Index score)

Themes from open-ended questions

Responses from the open-ended comments question on the telehealth satisfaction questionnaire were overwhelmingly positive. Examples include “Excellent idea, brings ease of medical attention to persons in rural areas,” “Excellent program, saves many hours travelling time,” “Great service! We would have had to drive 300 kms one way to [city],” “Very interesting and a comfortable atmosphere,” and “Was nervous coming here, but it was okay. I feel much better now.” The follow-up evaluation questionnaire included open-ended questions asking what patients and caregivers liked most and least about the appointment. The main themes with regard to what they liked most about telehealth were the reduced travel and stress, the convenience, and the quality of care. A number of people commented on the caring, support, and concern shown by the team, as well as the “face-to-face” contact and the “personal interaction” experienced during the sessions. The few responses to the question about what they liked least about the telehealth appointment were related to technical problems or difficulty hearing for those with hearing impairments, and lack of opportunity to talk to the doctor without the patient present. One family member stated “We would prefer not have any more telehealth conferences. Mom feels very intimidated and the meeting feels very impersonal.”

With respect to what patients and families liked most about in-person appointments, a few people noted the “one-on-one” or “face-to-face” discussion. Most comments were general statements about the quality of the service, the ability to ask questions and discuss concerns, and continued support. “We appreciate the fact that there are follow-ups happening on a regular basis. It helps us to deal with this disease.” Responses regarding what they liked least about in-person appointments were the travel, especially in winter, arrangingfor transpo rtation, parking, and time away from work. Several people noted that they had a long drive for a brief appointment. “It seems like a long way to come when the visit is no different than the telehealth. Could we do this over telehealth instead?” Some family members traveled long distances to the patient’s home, before driving to the appointment. “Mom lives four hours from me, so we had to make arrangements to get both of us to the appointment.”

The telehealth coordinator evaluation forms also provided space for comments. Many noted that participants were somewhat anxious before their first telehealth appointment, but became visibly more relaxed during the sessions, and at subsequent appointments. Patient discomfort during the sessions was seldom attributed to the technology, but more often to embarrassment at not being able to answer memory-related questions or emotions related to loss of abilities as a result of the dementia. The coordinators observed that patients with hearing problems sometimes had difficulty following the conversation.

Discussion

The results of this study demonstrate that telehealth videoconferencing is a feasible and acceptable strategy for delivering preclinic assessment and follow-up appointments within a rural and remote memory clinic. Data from patients, family members and other caregivers, and telehealth coordinators, support the use of telehealth for the majority of patients. Team clinicians, who have been providing service in the clinic since its inception, report that the telehealth component works well from their perspective. As a result of these formal evaluation results, and the requests of patients and caregivers to change in-person appointments to telehealth, we now offer telehealth for all follow-up appointments. Although we will see patients face-to-face if they prefer, no patients have requested an in-person assessment since this change to our follow-up protocol. We are looking at other opportunities to use telehealth to expand services to rural and remote patients and family members. For example, the clinic’s neuropsychology team has recently implemented a pilot study to evaluate a telehealth support group for spouses of clinic patients diagnosed with atypical early-onset dementias (e.g., variants of frontotemporal dementia, Huntington’s Disease), and the clinic physical therapist is leading an initiative to study the efficacy of telehealth to deliver a physical fitness program to our patients.

Much of the success of the telehealth component ofthe clinic is due to the excellent infrastructure support provided by the Telehealth Saskatchewan network. The reliability of the telehealth network, including the equipment and personnel, has greatly facilitated the services provided by the memory clinic. The clinic is now integrated into their program and is often highlighted as an example of a successful application of telehealth in the province. To move the clinic from a research demonstration project to apublically funded health care program, we have secured short-tem funding from the provincial health ministry and the local regional health authority, and will be seeking long-term funding to ensure sustainability. Our funding proposal is based on research evidence that demonstrates the feasibilityand acceptability of both the telehealth component reported here and the one-stop interprofessional assessment that will be reported elsewhere.

This study addressed several of the limitations reported in earlier telehealth satisfaction research (Whitten & Love, 2005; Williams et al, 2001). Specifically, we used a larger sample, included qualitative data, used a standardized scale used in previous studiesand with demonstrated reliability, used multiple measures and perspectives, compared telehealth and standard care, and collected data over time in a natural clinical setting. The majority of patients and their caregivers reported high satisfaction with telehealth as assessed with the structured telehealth satisfaction scale. We recognize, however, that such satisfaction measures, when used alone, do not provide adequate evidence of acceptability of services. Measuring satisfaction is complex and there is a lack of consensus on the meaning of satisfaction (Williams, Coyle, & Healy, 1998). Satisfaction research typically finds high satisfaction levels, which may stem from patients’ reluctance to criticize a needed service. Other problems in collecting and analyzing satisfaction data include social desirability response bias and sampling bias that resultswhen drop-outs are excluded (LeVois, Nguyen, & Attkisson, 1981).

Strategies for addressing some of the challenges of satisfaction research include focusing on dissatisfaction data, monitoring satisfaction over time, relating satisfaction to expectations, and triangulating satisfaction measures (Larsen, Attkisson, Hargreaves, & Nguyen, 1979). In this study we have used multiple methods to examine satisfaction and acceptability of the clinic components, including telehealth, and we have monitored satisfaction over time. We have also examined unobtrusive or nonreactive “measures” such as number of patients requesting a change to telehealth from in-person appointment and vice versa (Webb, Campbell, Schwartz, Sechrest, & Grove, 1981). Data from open-ended questions in all questionnaires have been very helpful in interpreting the quantitative data from rating scales and improving our understanding of participants’ needs and experiences.

Comparing data from follow-up evaluation questionnaires completed at telehealth and in-person assessments provided another way of understanding users’ experiences. The finding that participants were equally satisfied with both delivery modes provides further evidence of the acceptability of telehealth. The greater convenience of telehealth is consistent with the themes identified in the qualitative comments andt he travel burden for in-person appointments (approximately 500 km or 5 hours of driving for a round trip). Telehealth appointments saved more than 400 km on average for a round trip, thereby reducing travel time, costs of transportation and accommodation, and stress on patients and caregivers. Telehealth coordinators’ assessments of patient and family comfort during the sessions provided further support for the utility of telehealth in a memory clinic service.

For patients who stayed in the study, satisfaction with telehealth did not change from first exposure at the preclinic assessment to 6-month follow-up. Because patients with low satisfaction might be more likely to discontinue, we compared telehealth satisfaction scores at the preclinic assessment for those who continued versus discontinued by 6 months, and found that satisfaction with telehealth was lower for those who discontinued. However, lower satisfaction with telehealth may not be the only factor, or the most important factor, which contributes to the decision to discontinue. We found that older patients (≥80 years) were 5 times more likely to discontinue than younger patients (≤70 years). Stage of dementia was not a significant predictor variable, so it may be that the physical limitationsassociated with aging or the challenges of arranging transportation may make it more difficult for older patients to attend appointments. Those who had to travel more than 100 km for telehealth appointments were almost 9 times more likely to discontinue than those who had to travel less than 50 km, and there was a trend toward greater risk of discontinuing when the distance saved by using telehealth decreased. The findings highlight the importance of having telehealth sites located in as many communities as possible, and the diminishing benefit of telehealth when it is not easily accessible. Lower caregiver burden was a significant predictor of discontinuing, perhaps reflecting less need for ongoing follow-up by caregivers experiencing lower burden levels. The impact of telehealth on burden could not be assessed because all caregivers received both telehealth and in-person follow-up.

To date the clinic has seen a smaller number of Aboriginal patients than we had hoped, although several studies linked to the RRMC (Andrews, 2008; Cammer, 2006; Lanting, Crossley, Corney, & Cammer, in press) have improved our understanding of the challenges faced by northern Aboriginal patients in attending the clinic. The study by Lanting et al. is ongoing, and focused on the development of culturally appropriate assessment protocols for detecting cognitive impairment in Aboriginal seniors. Several of these modified neuropsychological measures have been in use in the clinic from its inception. Another team member (LM) is examining the availability of long-term care for northern Aboriginal seniors. These studies will guide future adaptations to the clinic aimed at improving accessibility and acceptability for Aboriginal seniors.

A potential limitation of the current study is that patients and family caregivers were assessed as dyads. Given the growing evidence that patients with dementia can competently participate in research (e.g., Ready & Ott, 2008), future research could assess patient and caregiver satisfaction with telehealth independently and as a dyad. From the outset of the current study we have been directly assessing patients’ perspectives on variables such as perceived stress, memory, depression, and quality of life. We have recently initiated in-depth telephone interviews with patients and family members who have been in the program for one year and those who have discontinued, to gather additional information on their experiences with telehealth and the one-stop clinic. Previous patient satisfaction research has shown that people are generally more critical of health servicesin qualitative studies (Williams et al., 1998). Williams et al. recommend using methods that provide patients with a voice in the continuing development of services, and that access patients’ experiences of services, the meaning and value they attach to them, and how they can be improved. Our research team’s goal is to continue to modify the clinic program on the basis ofpatient and family member input, including the use of telehealth delivery methods.