Rapid Electronic Capturing of Patient-Reported Outcome Measures in Older Adults With End-Stage Renal Disease: A Feasibility Study

Abstract

Background:

Patients with end-stage renal disease (ESRD) have a high burden of physical and psychological symptoms. Many remain unrecognized for long periods of time, particularly in older adults. The best strategy to monitor patient-reported outcome measures (PROMs) has not been identified.

Objective:

To assess the feasibility of implementing an iPad-based symptom assessment tool in older adults with ESRD on hemodialysis (HD).

Methods:

We designed an iPad application-delivery system for collecting electronic PROMs (ePROMs). Patient’s ≥60 years of age with ESRD on HD were recruited from a single outpatient dialysis unit. Feasibility was evaluated based on recruitment, retention, and the system usability score (SUS). Assessments were completed at baseline, 3 months, and 6 months after enrollment. ANOVA was used to assess longitudinal symptom variability.

Results:

Twenty-two patients (49% recruitment rate) were enrolled, with an 82% retention at 6 months. Mean age was 69.4 years (SD 6.6), 63.6% were female, and 81.8% were African American. Participants reported minimal difficulty in using the app, with an overall SUS score of 77.6. There were no significant relationships between demographic characteristics (age, race, or education) and SUS. Baseline SF-12 physical score and SF-12 mental score were 40.4 (SD 9.1) and 33.9 (SD 6.7), respectively. No significant changes were seen in longitudinal ePROMs of pain, depression, or anxiety; but was seen in the dialysis symptom index.

Conclusion:

In older patients with ESRD, collection of iPad-based ePROMs is feasible. This process can overcome inefficiencies associated with paper questionnaires and enable systematic monitoring of symptom burden.

Introduction

Older adults (60 years and older) represent the fastest growing segment of patients with end-stage renal disease (ESRD) on hemodialysis (HD). Their experience is marked by a high burden of physical and psychological symptoms, especially pain, anxiety, and depression.^1-3 Patient-reported health related quality of life (HRQOL) and symptom burden in this patient population has been shown to be statistically indistinguishable from those with terminal cancer.⁴ These symptoms are linked with poor outcomes: depression is associated with increased mortality,⁵ while uncontrolled pain and depression have both been associated with decreased dialysis adherence, increased rates of readmissions, and decreased medication adherence.⁶ Symptoms related to pain, anxiety, and depression are strongly associated with impaired HRQOL.^2,3,7 Patients with ESRD and lower HRQOL exhibit lower survival rates and higher rates of hospitalizations.⁸ Thus, given the high symptom burden experienced by patients with ESRD, user-friendly and time-sensitive symptom-focused measures are needed to monitor patients’ symptom burden and allow for prompt intervention.⁹ In addition, improved integration of palliative care into the management of ESRD is needed to help improve overall symptom burden and quality of life in this patient population.^10-13 The Coalition for Supportive Care of Kidney Patients has called the current state of palliative care for ESRD suboptimal¹⁴ and the National Priorities Partnership has identified palliative care as 1 of 6 national priorities.¹⁵

Information collected directly from patients’ related to their self-perception of living with a medical condition and its impact on quality of life, physical, functional, and psychological consequences, along with other health-related constructs are known as patient reported outcome measures (PROMs).¹⁶ Although PROMs should be routinely collected according to the ESRD Quality Incentive Program (ESRD QIP), the implementation of serial PROMs screening into clinical practice has been limited.⁹ In addition, the traditional method of data capture (paper questionnaires) requires several steps and is fraught with the risk of information loss or alteration.⁹ For example, the most commonly used tool to assess HRQOL is the Kidney Disease Quality of Life (KDQOL)-36 survey, an instrument limited by its length (up to 30 minutes to complete) and pen and paper delivery.¹⁷ In the electronic era, technological solutions to improve monitoring of PROMs in patients with ESRD ought to be explored. Electronic PROMs (ePROMs) offer patients the ability to “electronically” enter data via a smartphone, tablet device, or computer and grants clinicians the ability to review these data in real-time.¹⁸ Previous studies have indicated that implementation of ePROMs is feasible.^19-22 For example, Schick-Makaoff et al. demonstrated that using tablet computers to assess the KDQOL-36 along with Edmonton Symptom Assessment Scale was feasible among home dialysis patients.²³ However, additional work is needed to determine feasibility of using ePROMs specifically in older patients with ESRD on HD.²⁴ Here we describe our experience of designing and assessing the feasibility of an iPad-based ePROM application called “K-Pal” in older patients with ESRD receiving HD at an outpatient dialysis center.

Methods

Study Participants

Patients aged 60 and older with prevalent ESRD, on long-term HD for 3 months or longer, were recruited from a single outpatient dialysis unit affiliated with a large academic tertiary medical center in North Carolina. A research assistant approached all consecutive dialysis patients who met the inclusion criteria between June-August of 2017. Patients with a diagnosis of moderate to severe dementia, those who could not read or write English, patients followed by a non-Wake Forest nephrologist (due to the need to send messages through the electronic health record to primary nephrologist), and those who had severe vision impairment were excluded. Participants were compensated $25 for their time upon the successful completion of all measures. Implementation occurred between September 2017 and March 2018. The study was approved by the Institutional Review Board at Wake Forest School of Medicine. Written informed consent was obtained from all potential participants before enrollment.

Study Design

An iPad-based ePROM application called “K-Pal” was designed using validated screening PROMs for pain, depression, anxiety, and overall symptom burden. Feasibility was evaluated based on recruitment, retention, and usability. We considered success as ≥40% recruitment rate, system usability score (SUS) of 68 or higher, and ≥80% retention of study participants at 6 months (final assessment).^19,25-28 With the exception of HRQOL and SUS, all measures were assessed at baseline, 3, and 6 months after enrollment (Figure 2). Due to concerns of questionnaire burden, HRQOL was only measured at baseline. In addition, prognostic awareness, patient-reported sociodemographic data, and rates of advance care planning were measured at baseline.

Figure 2.

Consort flow diagram.

The K-Pal app took about 6 months to develop. The primary investigator developed the app with assistance from the Wake Forest Informatics Team. Validation testing was undertaken using 10 patient simulations with a variety of responses to the questionnaires. These responses were keyed into the K-Pal app and data was checked for accuracy. This was repeated 3 times until 100% accuracy was achieved. Font size was 18 to accommodate presbyopia, common in an older population. Security was included in the machines that automatically connected the device to the secure wireless encryption protocol-enabled network. No information was stored directly on the iPad. The devices were locked so that patients had no ability to navigate off the page of questionnaire. Hard stops were put into place so that questions could not be skipped by participants (Figure 1).

Figure 1.

Screenshots of the K-Pal App interface.

Assessments were conducted while the participants were undergoing their HD treatments in which a research assistant would hand them an iPad loaded with the K-Pal app to complete. The questionnaires were automatically scored and sent to the patient’s primary nephrologist. In the event of a positive screen for suicidal ideation, a large stop sign would appear on the tablet with instructions to alert the nurse before continuing to answer any additional questions; the nurse would then contact the nephrologist within the dialysis center so that rapid intervention could occur. The patient would then be scheduled for another day to complete the rest of the survey questions.

Measures

System Usability Scale (SUS).

Usability was assessed using the validated SUS, a 10-item questionnaire. Items are rated on a 5 point scale and responses to all items are summed and multiplied by 2.5. Scores range from 0 to 100, with scores of 68 or higher indicating above-average usability.²⁹

Short-Form McGill Pain Questionnaire 2 (SF-MPQ-2).

Pain was assessed using the Short-Form McGill Pain Questionnaire (SF-MPQ-2).³⁰ With this instrument, patient’s rate 22 different descriptors of pain on a 0-10 scale, with 0 equating to no pain and 10 suggesting the worst pain experienced during the week prior to the assessment.³⁰ The questionnaire is divided in 4 subscales including Continuous (throbbing pain, cramping pain, gnawing pain, aching pain, heavy pain, tender), Intermittent (shooting pain, stabbing pain, sharp pain, splitting pain, electric-shock pain, piercing), Neuropathic (hot-burning pain, cold-freezing pain, pain caused by light touch, itching, tingling or “pins and needles,” numbness), and Affective (tiring-exhausting, sickening, fearful, punishing-cruel) pain.

Patient Health Questionnaire-9 (PHQ-9).

Depressive symptoms were assessed using the PHQ-9 questionnaire which evaluates the frequency of symptoms related to depression during the 2 weeks prior to the assessment.³¹ Items are rated on a 4-point Likert-type scale, ranging from 0 (not at all) to 3 (nearly every day). Items were summed to create a total score that represents no depression symptoms (0–4), mild (5–9), moderate (10–14), moderately severe (15–19), or severe (20–27).

Generalized Anxiety Disorder 7 Item Survey (GAD-7).

Anxiety was assessed using the GAD-7, a 7-item self-report instrument used to assess symptoms of generalized anxiety disorder experienced during the prior 2 weeks.³² Each item is scored on a 4-point Likert scale indicating symptom frequency, ranging from 0 (not at all) to 3 (nearly every day). The GAD-7 total score can range from 0 to 21, further categorized as no anxiety (0–5), mild (6–10), moderate (11–15), or severe (16–21) anxiety.

Dialysis Symptom Index (DSI).

Overall symptom burden was assessed using the Dialysis Symptom Index (DSI).³³ The DSI consists of 30 items. Questions related to physical symptoms and emotion symptoms are assessed using a 5-point Likert scale, ranging from “not at all bothersome” to “very bothersome.” Responses are summed and total scores range from 0 to 150.

Kidney Disease Quality of Life (KDQOL-36).

HRQOL was assessed using the KDQOL-36 survey.^34,35 The KDQOL-36 consists of 36 items, and includes the SF-12 Physical and Mental Components. Raw scores were converted to t-scores, with a mean of 50 and a standard deviation of 10.³⁶ It also assesses burden of kidney disease (4 items), symptoms and problems of kidney disease (12 items), and effects of kidney disease (8 items). The total score ranges from 0-100, with higher scores indicating better quality of life.³⁷

Information Abstracted from the Medical Record

Missed dialysis days, hospitalization rates, emergency room visits, mortality, presence of advance directive within the electronic health record and Charlson Comorbidity Index were collected from the electronic health record. Charlson Comorbidity Index (CCI) was calculated based on hospitalization encounter diagnoses, and patient problem list.

Statistical Analysis

Feasibility, recruitment and retention rates were calculated and reported as percentages. We considered success as ≥40% recruitment rate, system usability score of 68 or higher, and ≥80% retention of study participants at 6 months (final assessment).^19,25-28 For descriptive analyses, means and medians were used to describe continuous variables and percentages and frequencies were used to describe categorical variables. Logistic regression was used to assess the relationship between KDQOL-36 scores and rates of hospitalization and missed dialysis days. Hospitalization was coded as a binary variable that measured whether a patient was hospitalized over the 6 months following enrollment. The presence or absence of one or more missed dialysis days was recorded as a binary indicator between baseline and the 6-month follow-up assessment. Repeated measures ANOVA (RM ANOVA) with a Huynh-Feldt correction were used to identify any significant statistical difference between depression, anxiety, pain, and overall symptom burden at baseline, and 3 and 6 months after study enrollment.³⁸ To reduce the probability of a Type I error, significant omnibus tests were followed up with pairwise comparisons using a Bonferroni correction. Spearman correlation was used to assess the correlation between system usability testing with age and education, where education was an ordinal variable representing different levels of education. A set of independent sample t-tests were used to determine SUS between group differences for race and gender. SAS (version 9.4, Cary, NC, USA) was used for all analyses; p < 0.05 was assumed to be significant.

Results

Baseline Demographics

Forty-five patients who met the inclusion criteria were identified, of which 22 enrolled in the study, representing a 49% recruitment rate. Of these, 1 patient expired after the baseline assessment and 3 patients withdrew from the study, representing an 82% retention rate (Figure 2). Twenty patients completed the 3-month follow-up and 18 patients completed 6-month follow-up. Participants had a mean (standard deviation) age of 69.4 (6.6) years; 63% were female and 82% were African American (Table 1). Most (63%) were of lower socioeconomic status (household annual income <$20,000) and the mean Charlson Comorbidity Index was 8.45 (2.28).

Table 1.

Baseline Demographics and Clinical Characteristics.

Variable	N (%)
Sex
Male	8 (36.4)
Female	14 (63.6)
Age, years
60-64	6 (27.3)
65-70	7 (31.8)
71-75	6 (27.3)
81-85	2 (9.1)
Race/ethnicity
White	3 (13.6)
Black/African American	18 (81.8)
Asian	1 (4.6)
Hispanic/Latino	0 (0.0)
Marital Status
Single, never married	1 (4.6)
Married	7 (31.8)
Divorced	7 (31.8)
Widowed	6 (27.3)
Separated	1 (4.6)
Education
< High School Graduate	6 (27.3)
High School Graduate or equivalent	5 (22.7)
Some college or Tech/Vocational	4 (18.2)
Bachelor’s Degree	4 (18.2)
Master’s Degree	2 (9.1)
Professional Degree	1 (4.6)
Household Annual Income, $
<20,000	14 (63.6)
20,00-40,000	3 (l3.6)
>40,000-75,000	5 (22.7)
>75,000	0 (0.00)
Time on Dialysis
3-6 months	2 (9.1)
7-12 months	2 (9.1)
1-2 years	5 (22.7)
3-4 years	4 (18.2)
5-6 years	2 (9.1)
7-8	1 (4.6)
9-10	2 (9.1)
>10 years	4 (18.2)
Charlson Comorbidity Index (CCI)
1-3 Comorbidities	0 (0.0)
4-5 Comorbidities	2 (9.1)
6-9 Comorbidities	13 (59.1)
≥ 10 Comorbidities	7 (31.8)
Number of missed hemodialysis sessions
0-1	16
2-4	3
>4	3
Number of hospitalizations
0-1	17
2-4	5
>4	0

Usability, Prognostic Awareness, and Advance Care Planning

Most participants indicated the K-Pal application was easy to use (88.9% rated “strongly agree” or “agree”). The mean SUS score was 77.6 (5.8; 95% CI: 69.8-85.4), indicating a grade of B or “Good.”²⁹ No significant correlation was found between the SUS and age (r_s = −0.09, p = 0.72) or education (r_s = −0.38, p = 0.12). No significant difference was found on SUS between gender (Male = 73.8 (11.3), Female = 76.0 (14.2), p = 0.72)) or race ((White = 77.5 (17.5), African American = 74.5 (12.2), p = 0.72)). Participants were asked to estimate their prognosis and 50% (n = 11) believed their remaining life expectancy was ≥10 years. Half of the cohort (n = 11) reported having an advance directive, yet only 36% (n = 8) had a documented copy within the electronic medical record. Thirteen percent (n = 3) lacked an understanding of advanced directives. There was no significant relationship between self-perceived life expectancy and the rate of advance directive completion (Fisher’s Exact p = 0.79) or highest-achieved level of education (r_s = −0.11, p = 0.61).

Quality of Life

Table 2 shows the distribution of KDQOL-36 subscale scores at study enrollment. The patients reported poorer than average physical and mental function, along with significant burden of kidney disease, as indicated by mean subscale scores below 50 for all measures. Using the KDQOL scores as a continuous variable, we found no association between quality of life components and rates of hospitalization or noncompliance with HD; though this study was not powered to detect a differences in clinical events (Table 2).

Table 2.

Baseline Electronic KDQOL-36 Scores and Associations with Rates of Hospitalization and Missed Hemodialysis Treatments.

Baseline electronic KDQOL-36 scores
Variable	Mean	SD	Median	IQR*
SF-12 Physical	40.4	9.1	41.2	34.2-45.4
SF-12 Mental	33.8	6.7	34.4	26.6-37.9
Symptoms and Problems	82.9	14.6	85.4	70.8-95.8
Effects of Kidney Disease	81.3	16.5	84.4	71.9-96.9
Burden of Kidney Disease	36.9	15.3	43.8	25.0-50.0
Associations between KDQOL-36 scores with hospitalization and HD non-compliance
	Hospitalization		Missed Dialysis
Variable	Odds Ratio	P-value	Odds Ratio	P-value
SF12 Physical	0.99 (0.90-1.09)	0.86	0.98 (0.88-1.09)	0.66
SF12 Mental	1.01 (0.88-1.15)	0.93	1.16 (0.98-1.38)	0.09
Symptom Problem	1.00 (0.94-1.06)	0.93	1.01 (0.94-1.08)	0.82
Effects of Kidney Disease	0.99 (0.94-1.04)	0.66	1.03 (0.96-1.09)	0.41
Burden of Kidney Disease	1.02 (0.96-1.09)	0.50	0.93 (0.87-1.01)	0.07

^*Interquartile range (IQR); Standard Deviation (SD); 12-Item Short Form Health Survey (SF-12); HD = Hemodialysis; KDQOL = Kidney Disease Quality of Life;

Symptom Assessment

Using the PHQ-9, 36% (n = 8) screened positive for depression at baseline, 35% (n = 7) at 3 months, and 33% (n = 6) at 6 months, with no significant difference seen in the burden of depressive symptoms between the 3 points of assessment (Table 3). No patients screened positive for suicidal thoughts during our intervention. Based on the GAD-7 questionnaire, 13.6% of the patients (n = 3) at baseline, 15% (n = 3) at 3 months, and 22% (n = 4) at 6 months screened positive for anxiety. As with depressive symptoms, no significant changes were detected in the burden of anxiety symptoms over time. Similarly, there were no differences in pain ratings over time, [50% (n = 11) of patient reported pain scores of ≥ 5 in 2 or more questions at baseline, 45% (n = 9) at 3 months, and 66% (n = 12) at 6 months].

Table 3.

EPROMS: Depression, Pain, Anxiety and Overall Symptom Burden.

Variable	Baseline (N = 22)		Month 3 (N = 20)		Month 6 (N = 18)		RM ANOVA F; p-value
	Mean	SD	Mean	SD	Mean	SD
PHQ-9	3.6	3.31	4.1	4.1	4.2	3.4	1.16; p = 0.35
GAD-7	1.4	1.9	2.1	3.3	2.9	3.5	1.15; p = 0.33
McGill Pain Total Score	28.7	37.9	30.5	37.5	30.4	25.7	0.14; p = 0.83
McGill Continuous	16.2	19.39	15.8	17.9	16.5	12.1	0.03; p = 0.97
McGill Intermittent	13.5	19.6	13.5	18.8	16.5	12.1	0.51; p = 0.61
McGill Neuropathic	14.5	17.8	14.8	17.6	16.8	10.9	0.04; p = 0.96
McGill Affective	6.4	11.4	8.4	12.1	9.2	10.0	0.37; p = 0.69
DSI Scores	12.0	12.3	20.6	18.9	24.2	13.6	5.41; p = .01

Electronic Patient-Reported Outcome Measures (ePROMs); Dialysis Symptom Index (DSI); General Anxiety Disorder 7 item scale (GAD-7); Patient Health Questionnaire (PHQ-9); Standard Deviation (SD)

Based on the DSI assessments, the mean number of symptoms were 12.0 at baseline, 20.6 at 3 months, and 24.2 at 6 months. The most common symptoms experienced by patients were bone/joint pain, cramping, dry skin, itching, fatigue, and numbness in feet (Figure 3). After correcting for multiple comparisons, DSI scores were significantly higher at the 6 month follow up compared to baseline (difference = 10.8, p = 0.001).

Figure 3.

Dialysis symptom index.

Discussion

In this study, we showed that using an iPad-based application (K-Pal) for capturing ePROMs in an outpatient dialysis center was technically and practically feasible in older adults with ESRD on HD. Most patients receiving standard in-center dialysis spend 12 hours per week at the dialysis center, making the collection of ePROMs via an iPad-based application logistically feasible. In addition, most dialysis centers already have embedded social workers in their workflow to do symptom assessments related to quality of life (KDQOL), pain, and depression; thus implementation of the K-Pal app into clinical workflow is very feasible. The majority of older adults felt the use of the K-Pal application was easy to use and prior studies have shown that older adults are increasingly using technological devices like tablets and smartphones, indicating the feasibility of using tablet-like interventions in this population.³⁹ To our knowledge, this is the first study to assess the feasibility of collecting ePROMs through a tablet-based application in older adults with ESRD on HD. Electronic methods may offer greater advantages than traditional pencil-and-paper methods due to assurances of privacy and complete data collection by requiring 1 response per item.⁴⁰

Furthermore, as ePROMs can be directly incorporated into the electronic health record, their use can decrease administrative burden, avoid secondary data entry errors, offer potential cost savings, and allow for providers to review this data in real-time.^19,41 Recent studies suggest that ePROMs may be sensitive enough to detect clinically relevant changes in patients with chronic kidney disease and provide early warning prompts for intervention⁴²; thereby enhancing patient engagement in their treatment.⁴³ Greater recognition with consequent better management could lead to improved quality of life, reduced hospitalizations, and longer survival. Our preliminary findings suggest that implementation of ePROMs within a dialysis center may be an efficient method to monitor overall symptom burden and quality of life metrics. Further studies are needed to assess their longitudinal impact on clinical care.

Although the majority of evaluations were positive, a small number of potential improvements were identified for future versions of the app. Some participants were somewhat overwhelmed with the amount of questionnaires, especially during their initial use of the K-Pal app. Thus it was recommend to space out different ePROMs over time instead of testing all at once. Another challenge we did encounter with implementation was related to patient fatigue from their HD treatment; thus we did have to reschedule some patients to complete the K-Pal app during an upcoming HD treatments; further emphasizing the importance of spacing out symptom assessments. Another participant felt it was sometimes challenging moving between survey questionnaires and recommended numbering them in order to help facilitate flow. There were also suggestions that an introductory tutorial video could be added for first-time users that could be extremely beneficial and would help decrease amount of time needed for clinical support.

A strength of this investigation was the use of standardized and validated survey instruments. In addition, the majority of the patients were African American, which is an understudied population in geriatrics research. A potential study limitation was that the patient population was drawn from a convenience sample in relation to an outpatient dialysis clinic location, but we believe that these patients are representative of older population with ESRD on HD. Another potential limitation is the relatively small sample size lacking a case-matched group for comparison between the electronic and the paper questionnaires. Further prospective studies with a larger sample size and within a more diverse patient population are needed.

In conclusion, in older patients with ESRD on HD, collection of ePROMs by use of tablet technology is feasible. It has the potential to significantly improve the management of patients with ESRD by streamlining the integration of survey responses into direct clinical care; and assist in identifying a potential target population for implementing a palliative medicine co-management model.