Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Oct 9.
Published in final edited form as: J Health Commun. 2015 Oct 9;20(0):83–91. doi: 10.1080/10810730.2015.1058442

Effect of Health Literacy on Research Follow-up

Cardella Leak 1, Kathryn Goggins 2,3,4, Jonathan S Schildcrout 5,6, Cecelia Theobald 7, Katharine M Donato 8, Susan P Bell 9,10, John Schnelle 7,10,11, Sunil Kripalani 2,3,4,7
PMCID: PMC4706551  NIHMSID: NIHMS747017  PMID: 26513035

Abstract

Previous research has not examined the effect of health literacy on research subjects' completion of scheduled research follow-up. This article evaluates patient factors associated with incomplete research follow-up at three time points after enrollment in a large, hospital-based prospective cohort study. Predictor variables included health literacy, age, race, gender, education, employment status, difficulty paying bills, hospital diagnosis, length of stay, self-reported global health status, depression, perceived health competence, medication adherence and healthcare system distrust. In a sample of 2042 patients, multivariable models demonstrated that lower health literacy and younger age were significantly associated with a lower likelihood of completing research follow-up interviews at 2-3 days, 30 days and 90 days after hospital discharge. Additionally, patients who had less education, were currently employed, and had moderate financial stress were less likely to complete 90-day follow-up. This study is the first to demonstrate that lower health literacy is a significant predictor of incomplete research follow-up.

Keywords: Health literacy, incomplete follow-up, attrition

I. Introduction

Attrition has traditionally been described as “permanently dropping out of a study;” however, attrition can also be described as a process that varies over time and includes respondents who may drop out for a time point but may return to complete other parts of the study (Lugtig, 2014).Much like attrition, incomplete follow-up is an important limitation associated with many prospective research studies because it threatens internal validity, affects sample size, and compromises the accuracy of longitudinal outcome measurements (Cotter, Burke, Stouthamer-Loeber, & Loeber, 2005; Kristman, Manno, & Cote, 2005; Psaty et al., 1994). In 1996, a systematic review of 87 studies found that attrition had not been given adequate attention by researchers (Goodman & Blum, 1996). Since then, researchers have charged colleagues to be more transparent about incomplete follow-up and attrition (i.e. presenting reasons for attrition and proposing a standard for reporting missing responses) (Barry, 2005; Dumville, Torgerson, & Hewitt, 2006; Fewtrell et al., 2008).

Internal validity can be jeopardized when participants with incomplete follow-up data differ from participants with complete follow-up data (Barry, 2005; Boys et al., 2003; Cotter et al., 2005; Psaty et al., 1994)and outcomes or exposures of interest to the study might be missing for a particular group of participants(Green, Navratil, Loeber, & Lahey, 1994; Psaty et al., 1994).Previous research has shown that those with complete data and those without complete data might be similar at baseline, but may be systematically different at the time of follow-up (Gustavson, von Soest, Karevold, & Roysamb, 2012; Sullivan, Rumptz, Campbell, Eby, & Davidson, 1996; Woolard et al., 2004). This can limit the representativeness of the sample and can ultimately lead to potentially invalid conclusions about the cohort(Kapteyn, Michaud, Smith, & van Soest, 2006).

Although many attributes appear to be related to incomplete follow-up, there is no consensus in prior studies about which attributes are most important, especially in the context of other relevant factors. Patient characteristics associated with study follow-up including age, sex, race, education, employment status, psychological distress, clinical illness, or inconsistent health care utilization have been referenced in previous research (Cotter et al., 2005; Davis, Broome, & Cox, 2002; Psaty et al., 1994). One study of cardiovascular patients identified black race, male gender, younger age, lack of health insurance, obesity, and depression as being related to study attrition (Bambs et al., 2013). However, overall, there remain inconsistencies in the literature, and many studies have failed to analyze characteristics together to determine their independent contributions to completing follow-up.

Additionally, other patient factors that could potentially be related to incomplete follow-up have been investigated less, including medication adherence behaviors, health care system distrust, and patient-reported health status measures. Health literacy is a predictor of particular interest, and is related to comprehension of informed consent, but has not been examined in the context of study attrition(Donovan-Kicken et al., 2012; Kripalani, Bengtzen, Henderson, & Jacobson, 2008; Lindau, Basu, & Leitsch, 2006; Sudore et al., 2006; Waldrop-Valverde, Guo, Ownby, Rodriguez, & Jones, 2014). Furthermore, little literature has examined incomplete follow-up over multiple periods of time after enrollment, especially in the inpatient population where factors such as acute health status may affect follow-up differently than in an outpatient or general population.

We sought to examine the independent association of health literacy and other psychosocial and demographic factor son incomplete study follow-up in an inpatient population at three distinct follow-up times (2-3 days, 30 days, and 90 days after hospital discharge) and overall (number of follow-ups missed). We hypothesized that lower health literacy, younger age, minority status, and lower self-reported health status would be associated with reduced likelihood of completing follow-up.

II. Methods

Study design and setting

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective cohort study of patients with cardiovascular disease admitted to Vanderbilt University Hospital (VUH) in Nashville, Tennessee and Williamson Medical Center in Franklin, Tennessee. The goal of this prospective cohort study is to examine how a broad set of social, behavioral, and clinical determinants of health affect the quality of hospital discharge transitions, as well as post-discharge outcomes including Emergency Department visits, hospital readmissions, functional status, and mortality. Details of VICS, including a conceptual framework and rationale for the selection of measures, are described elsewhere(Meyers et al., 2014). The study was approved by the Vanderbilt University Institutional Review Board (IRB).All data were collected using Research Electronic Data Capture, REDCap (Harris, 2009).

Participants

Study participants were enrolled and interviewed in the hospital and scheduled for three follow-up calls after hospital discharge. We recruited patients over the age of 18 hospitalized with a diagnosis of acute coronary syndrome (ACS) and/or acute decompensated congestive heart failure (ADHF).Eligible participants presented initially to VUH or were transferred to VUH within 7 days of initial presentation to another hospital and met clinical criteria for ADHF or intermediate to high likelihood of ACS as defined by clinical signs and symptoms as determined by medical record review conducted by a physician. Exclusion criteria were: unable to communicate in English, under hospice care, unstable psychiatric condition, inability to consent or participate due to medical condition or treatment (significant dementia, sedated), uncooperative or in police custody, visual or hearing impairment precluding participation, and already enrolled in VICS or a conflicting study. Patients who were delirious or too ill to participate early during admission were re-assessed for up to 7 days for potential eligibility. Patients who provided informed consent between October 2011 and October 2014 were included in this analysis.

Follow-up procedures

After patients were discharged from the hospital, trained research assistants (RAs) adhered to a standard follow-up procedure described below. Patients were contacted by telephone for the follow-up interviews approximately 2-3 days (range 1-7), 30 days (range 25-35), and 90 days (range 85-95) after discharge with each call requiring approximately 20, 15, and 10 minutes of time to complete, respectively (Meyers et al., 2014). Patients were told that during follow-up calls, RAs would ask them questions that were similar to the baseline interview, as well as other questions regarding their hospital stay and medications. Patients were contacted using the phone numbers that they gave RAs during the baseline interview.

RAs attempted to reach patients using a variety of methods. Patients were called based on their preferences of day and time as indicated during the baseline interview. When patients could not be reached during their preferred time, they were called at various times throughout the day and evening. If RAs reached the voicemail of a patient, they left a voicemail with a toll-free number for the patient to call back at his/her earliest convenience. If a patient provided a phone number during the baseline interview that was incorrect or disconnected, the electronic medical record (EMR) was used to locate alternative numbers to reach the patients. As also attempted to contact patients via email if phone calls were unsuccessful and if patients had provided an email address. RAs attempted to reach patients until their call window expired (after seven days for the first call or ten days for the second and third follow-up calls) or after twenty-one attempts.

Definition of incomplete follow-up

Follow-up status was considered to be incomplete if the participant refused the telephone interviewor did not answer any call attempts. Patients who died during the follow-up time period were excluded from the analysis. Patients who withdrew from the study could have been considered to have incomplete follow-up, but they were not included in this analysis per IRB policies prohibiting use of their data.

Baseline measures

In this analysis, we included measures from the parent study that had been examined previously in the context of attrition, as well as additional items that the research team hypothesized might be related to attrition. The domains that were assessed include demographics, clinical measures, socioeconomic status, psychosocial factors, health literacy, and behavioral factors. Most were collected during the baseline interview administered by the RA in the hospital, except age, gender, diagnosis, and hospital length of stay were collected from the EMR.

Demographics included age, gender, and race. Race was self-reported using the categories specified by the National Institutes of Health(i.e., White, Black or African American, Asian, Native Hawaiian or other Pacific Islander, American Indian or Alaska Native, or Other).

Diagnosis, length of hospital stay and perceived health status represent the clinical measures that were evaluated. Self-reported health status was measured using the first five questions from the Patient-Reported Outcomes Measurement Information System (PROMIS) global health status scale that assesses overall, physical, mental, and social health status as well as quality of life using a 5-item response scale(Hays, Bjorner, Revicki, Spritzer, & Cella, 2009).

Socioeconomic status was evaluated using education level, employment status and financial strain. Financial strain was measured by asking the patient to report difficulty paying monthly bills using four response categories ranging from “not at all difficult” to “very difficult.”

The psychosocial domain was characterized based on self-report perceived health self-efficacy, depression, and healthcare system distrust. Two items from the Perceived Health Competence Scale (PHCS)were administered. These items were summed and scores ranged from 2 to 10, with higher scores indicating higher health competence (Smith, Wallston, & Smith, 1995).We assessed the patient's depressive symptoms during the two weeks prior to hospitalization using the 8-item version of the Patient Health Questionnaire (PHQ-8), where higher scores indicate more depressive symptoms(Kroenke, 2009). Healthcare system distrust was measured using the 9-item revised Healthcare System Distrust Scale, which asks patients to assess the values and competence of hospitals, community clinics, labs, insurance companies, and drug companies with higher scores indicating more distrust (Shea et al., 2008). This measure was of interest as previous research has shown that trust can affect how a person interacts with the medical system, and possibly also with research studies(Gupta et al., 2014).

Health literacy was evaluated using the short form of the Test of Functional Health Literacy in Adults (s-TOFHLA), with higher scores representing higher health literacy(Baker, Williams, Parker, Gazmararian, & Nurss, 1999). The shortened 7-item version of the Adherence to Refills and Medications Scale (ARMS-7) was used to measure a patient's adherence to their pre-admission medication regimen(Kripalani, Risser, Gatti, & Jacobson, 2009).We report the medication adherence score as a continuous value with higher scores indicating lower adherence to medication refills and administration instructions. The ARMS-7 was not administered to patients who were not taking any medications prior to admission.

Statistical Analysis

To describe the VICS sample included in the analysis, we used averages and standard deviations for continuous variables (age, education, length of stay, health literacy, perceived health competence, self-reported global health status, depression, medication adherence, and healthcare system distrust) and counts and percentages for categorical variables (gender, diagnosis, race, financial strain, and employment status). Based on the distribution of the independent variable data, we conducted unadjusted tests for associations with the binary outcome variables (phone call completed at 2-3 days, 30 days, and 90 days) using the Mann-Whitney U test for quantitative variables and the Pearson Chi-square tests for categorical variables.

To examine patient factors that were independently associated with incomplete follow-up, we ran three multiple logistic regression models (one for each follow-up call stage, e.g. 2-3, 30, and 90 days). We included the a priori determined set of patient factors listed above in all models, and we report odds ratios and 95% confidence intervals to characterize the strength of the associations. To examine the patient factors associated with increased number of incomplete follow-up calls among the three that were attempted, we fit a proportional odds model. The proportional odds model, an extension of logistic regression for ordinal outcomes ,examines the log odds of being in the higher of the two categories for any dichotomization of the cumulative response (e.g., 3 versus 2, 1, or 0 incomplete calls; 3 or 2 versus 1, or 0 incomplete calls; 3, 2, or 1 versus 0 incomplete calls). For our purposes, the odds ratios are interpreted as the odds of not completing incrementally more follow-up calls per unit change in an independent variable while fixing all other independent variables.

Across all patients and variables, we observed a 1.2% missingness rate, and we report results based on using multiple imputation analyses with five imputation replicates using fully conditional specification. All analyses were conducted using SPSS version 22.

III. Results

With a sample of 2042 participants, the mean age was 60 years, 58% were male, and 83% were white (Table 1). Average educational attainment was about 14 yearsand63% of patients were hospitalized with ACS, with 30% and 7%of patients being hospitalized ADHF and for both ACS and ADHF, respectively. Approximately 34% of participants were employed(which included full-time, part-time, or self-employed), 35% reported no difficulty in paying bills and 17% reported it was very difficult to pay their monthly bills.

Table 1. Patient characteristics.

Patient characteristics M ± SD* or n (%)
N=2042
Age 60.2 ± 12.6
Education 13.6 ± 2.9
Race, White 1697 (83.1)
Gender, Male 1191 (58.3)
Diagnosis
 ACS 1293 (63.3)
 ADHF 611 (29.9)
 Both (ACS and ADHF) 138 (6.8)
Employed (full time, part time or self) 694 (34.0)
Difficulty paying bills
 Not at all difficult 706 (34.6)
 Not very difficult 385 (18.9)
 Somewhat difficult 593 (29.0)
 Very difficult 340 (16.7)
Length of stay 4.6 ± 4.8
Health literacy (sTOFHLA) (range 0-36) 29.3 ± 7.9
Self-reported global health status (PROMIS) (range 1-5) 3.0 ± 0.9
Depression (PHQ) (range 0-24) 7.8 ± 5.4
Perceived health competence (PHCS) (range 2-10) 7.6 ± 2.2
Medication adherence (ARMS) (range 0-21) 18.7 ± 2.5
Healthcare system distrust (HCSD) (range 9-45) 25.0 ± 6.2
Open in a new tab
*

Mean ± Standard Deviation

Missing values: Education=2, Race=5, Employment status=1, Difficulty paying bills=18, sTOFHLA=97, PROMIS=24, PHQ=31, PHCS=31, ARMS=104, HCSD=38

A total of 112 patients were excluded from the analysis due to death during the follow-up period (n=98) and study withdrawal (n=14). At each of the three patient contact times, 11% to 12% of participants missed follow-up interviews. Patients refused follow-up at the rate of 2.6% for the 2-3 day call, 2.2% for the 30-day call, and 1.9% for the 90-day call and 9.3%, 8.6% and 9.6% of participants were unreachable by phone to complete follow-up for the 2-3 day call, 30-day call, and 90-day call, respectively. Overall, 12% of patients missed one follow-up interview, 4% missed two interviews, and 5% missed all three interviews. Table 2 displays the results of the unadjusted, bivariate analyses at each stage of follow-up. At the 2-3 day phone call, younger age, fewer years of education, longer hospital length of stay, and lower self-reported health status were significantly associated with incomplete follow-up status. At the 30-day phone call, younger age, fewer years of education, diagnosis of ADHF, lower health literacy, lower self-reported health status, and lower perceived health competence were significantly associated with incomplete follow-up status. At the 90-dayphone call, younger age, fewer years of education, being employed, lower health literacy, lower self-reported health status, and increased difficulty paying bills were significantly associated with incomplete follow-up status.

Table 2. Unadjusted associations between patient characteristics and incomplete follow-up at 2-3, 30, and 90 days.

2-3 day 30 day 90 day
Patient characteristics Complete follow-up
(n=1808)		 Incomplete follow-up
(n=234)		 p-value Complete follow-up
(n=1823)		 Incomplete follow-up
(n=219)		 p-value Complete follow-up
(n=1799)		 Incomplete follow-up
(n=243)		 p-value
Age† 60.6 ± 12.4 56.9 ± 13.7 0.000** 60.5 ± 12.3 57.0 ± 14.7 0.000** 60.6 ± 12.5 56.9 ± 13.0 0.000**
Education† 13.6 ± 2.9 13.2 ± 3.0 0.014* 13.7 ± 2.9 12.9 ± 2.9 0.001** 13.7 ± 2.9 12.9 ± 2.9 0.000**
Race, White^ 1504 (83.4) 193 (82.8) 0.836 1526 (83.8) 171 (78.8) 0.060 1501 (83.7) 196 (80.7) 0.238
Gender, Male^ 1043 (57.7) 148 (63.2) 0.105 1057 (58.0) 134 (61.2) 0.363 1045 (58.1) 146 (60.1) 0.554
Diagnosis^
 ACS 1156 (63.9) 137 (58.5) 0.097 1173 (64.3) 120 (54.8) 0.020* 1152 (64.0) 141 (58.0) 0.184
 ADHF 527 (29.1) 84 (35.9) 529 (29.0) 82 (37.4) 527 (29.3) 84 (34.6)
 Both (ACS and ADHF) 125 (6.9) 13 (5.6) 121 (6.6) 17 (7.8) 120 (6.7) 18 (7.4)
Employment status, employed^ 612 (33.8) 82 (35.2) 0.684 614 (33.7) 80 (36.7) 0.374 598 (33.2) 96 (39.7) 0.047*
Difficulty paying bills^
 Not at all difficult 637 (35.4) 69 (30.4) 0.380 643 (35.5) 63 (29.3) 0.186 627 (35.1) 79 (33.2) 0.005**
 Not very difficult 342 (19.0) 43 (18.9) 346 (19.1) 39 (18.1) 344 (19.3) 41 (17.2)
 Somewhat difficult 523 (29.1) 70 (30.8) 524 (29.0) 69 (32.1) 534 (29.9) 59 (24.8)
 Very difficult 295 (16.4) 45 (19.8) 296 (16.4) 44 (20.5) 281 (15.7) 59 (24.8)
Length of stay† 4.5 ± 4.7 5.3 ± 5.7 0.035* 4.6 ± 4.7 5.2 ± 5.9 0.267 4.6 ± 4.7 5.0 ± 5.4 0.407
Health literacy (sTOFHLA) (range 0-36)† 29.4 ± 7.8 28.3 ± 8.6 0.184 29.5 ± 7.8 28.0 ± 8.4 0.039* 29.6 ± 7.7 27.4 ± 8.8 0.001**
Self-reported global health status (PROMIS) (range 1-5)† 3.0 ± 0.8 2.8 ± 0.9 0.009** 3.0 ± 0.9 2.8 ± 0.9 0.001** 3.0 ± 0.9 2.9 ± 0.9 0.023*
Depression (PHQ) (range 0-24)† 7.8 ± 5.3 8.2 ± 5.8 0.609 7.7 ± 5.3 8.5 ± 5.8 0.169 7.7 ± 5.3 8.3 ± 5.5 0.217
Perceived health competence (PHCS) (range 2-10)† 7.6 ± 2.2 7.6 ± 2.3 0.726 7.7 ± 2.2 7.3 ± 2.3 0.049* 7.6 ± 2.2 7.4 ± 2.3 0.196
Medication adherence (ARMS) (range 0-21)† 18.7 ± 2.4 18.3 ± 2.9 0.265 18.7 ± 2.5 18.4 ± 2.6 0.113 18.7 ± 2.4 18.3 ± 2.9 0.208
Healthcare system distrust (HCSD) (range 9-45)† 25.0 ± 6.1 25.2 ± 7.3 0.974 24.9 ± 2.5 25.5 ± 6.7 0.271 25.0 ± 6.1 25.0 ± 7.1 0.545
Open in a new tab
^

n (%), statistical test= Chi-square

†

M ± SD, statistical test= Mann-Whitney U

*

p-value<0.05,

**

p-value<0.01

In the multiple logistic regression analyses, health literacy and age were most consistently associated with a higher likelihood of incomplete follow-up(Table 3). Patients with lower health literacy were more likely to miss the 2-3 day phone call (odds ratio (OR)=0.75 per 9 point change in the s-TOFHLA, 95% confidence interval (CI)=0.63-0.90) as well as younger patients (OR=0.65 per 15 year change in age, 95% CI=0.54-0.80). For the most part, results were similar for the 30-day phone call model and the 90-day phone call model. At 90-days, we also observed some evidence of associations with education, employment status, and ease of paying bills with incomplete follow-up. The proportional odds model suggested that patients with lower health literacy, younger age, and a diagnosis of ADHF (OR=1.36; 95% CI=1.05 to 1.75)had higher odds of missing more follow-up calls .Other patient characteristics did not exhibit evidence for associations with completion of follow-up at these time points.

Table 3. Adjusted associations from logistic regression models(for 2-3 day, 30 day, and 90 day calls) and proportional odds regression (for greater number of incomplete calls).

Patient characteristic 2-3 day
OR (95% CI)^ 		30 day
OR (95% CI)^ 		90 day
OR (95% CI)^ 		Proportional odds model
OR (95% CI)^
Age (per 15 year change) 0.65 (0.54-0.80)*** 0.69 (0.57-0.85)*** 0.64 (0.53-0.78)*** 0.69 (0.59, 0.81)***
Education (per 4 year change) 0.92 (0.74-1.15) 0.81 (0.65-1.02) 0.80 (0.64-0.99)* 0.85 (0.72, 1.01)
Race (White=ref) 0.73 (0.50-1.09) 1.01 (0.69-1.49) 0.82 (0.56-1.20) 0.91 (0.68, 1.23)
Gender (Male=ref) 0.81 (0.60-1.08) 0.88 (0.65-1.18) 1.00 (0.75-1.33) 0.87 (0.69, 1.09)
Diagnosis (ACS=ref)
 ADHF 1.20 (0.87-1.66) 1.34 (0.97-1.87) 1.22 (0.89-1.69) 1.36 (1.05, 1.75)*
 Both (ACS and ADHF) 0.80 (0.44-1.49) 1.28 (0.73-2.25) 1.20 (0.69-2.09) 1.13 (0.72, 1.76)
Employment status (employed=ref) 0.95 (0.68-1.32) 0.76 (0.54-1.07) 0.65 (0.47-0.90)** 0.77 (0.59, 1.01)
Difficulty paying bills
 Not very difficult 0.94 (0.61-1.44) 0.94 (0.61-1.45) 0.74 (0.49-1.13) 0.88 (0.63, 1.23)
 Somewhat difficult 0.88 (0.60-1.29) 0.94 (0.64-1.40) 0.60 (0.41-0.89)* 0.79 (0.59, 1.08)
 Very difficult 0.91 (0.58-1.43) 0.94 (0.59-1.49) 1.04 (0.69-1.59) 1.03 (0.72, 1.45)
Length of stay 1.02 (0.99-1.04) 1.01 (0.98-1.04) 1.00 (0.98-1.03) 1.01 (0.99, 1.03)
Health literacy (sTOFHLA) (per 9 point change) 0.75 (0.63-0.90)** 0.76 (0.63-0.91)** 0.67 (0.56-0.80)*** 0.71 (0.62, 0.82)***
Perceived health status (PROMIS) (per 1 point change) 0.83 (0.65-1.06) 0.88 (0.69-1.13) 0.92 (0.72-1.16) 0.87 (0.72, 1.04)
Depression (PHQ) (per 7 point change) 0.95 (0.76-1.20) 1.01 (0.79-1.29) 1.00 (0.79-1.26) 0.99 (0.82, 1.19)
Health competence (PHCS) (per 4 point change) 1.17 (0.83-1.63) 0.94 (0.67-1.32) 1.01 (0.73-1.39) 1.02 (0.79, 1.31)
Medication adherence (ARMS) (per 4 point change) 0.91 (0.71-1.16) 1.03 (0.80-1.32) 0.93 (0.74-1.17) 0.93 (0.77, 1.12)
Healthcare system distrust (HCSD) (per 9 point change) 1.08 (0.86-1.37) 1.13 (0.88-1.45) 1.00 (0.80-1.23) 1.08 (0.90, 1.29)
Open in a new tab
^

Odds ratio (95% Confidence interval)

*

p<0.05,

**

p<0.01,

***

p<0.001

IV. Discussion

This study is the first, to our knowledge, to determine that lower health literacy is independently associated with lower rates of follow-up among research subjects. Furthermore, it is one of a few studies that simultaneously examined multiple predictors of incomplete follow-up status in a robust multivariate model. We found that at all time points, lower health literacy and younger age were consistently associated with increased likelihood of incomplete study follow-up that persisted after controlling for several other patient factors. Both lower health literacy and younger age were also associated with a higher likelihood of missing incrementally more follow-up phone calls. At longer follow-up intervals, additional social factors including lower education attainment, current employment, and moderate financial stress were also independently associated with greater loss to follow-up. However, some factors such as depression, health competence, medication adherence, healthcare system distrust, length of stay, nor diagnosis were not associated with a greater loss to follow-up, regardless of the follow-up interval.

There are several possible reasons why low health literacy may be associated with study attrition. First, health literacy is known to be associated with comprehension of informed consent (Kripalani et al., 2008). Although we used teach-back to verify patients' understanding of informed consent elements (including the procedures for study follow-up), it is possible that participants with low health literacy understood follow-up processes to a lesser degree and were less likely to engage as a result. Second, patients with low health literacy may be more avoidant of health care contact, due to prior difficulties with navigating the health system. We do not believe that this is related to trust in the healthcare system, per se, as we adjusted for that in the analysis. Third, patients with low health literacy may have worse health status after hospital discharge, which could make them less able or interested in participating in follow-up telephone calls. In this analysis, we intentionally adjusted only for baseline factors, rather than those measured during the follow-up period, so were unable to determine if post-discharge health status mediated the relationship; however, we plan to pursue this in the future. Of note, it is interesting that health literacy did not appear to be associated with completion of 2-3 day follow-up in the bivariate analysis but was highly associated with call completion in the adjusted model (p<0.01). This reinforces the concept that variable selection for models including a number of independent variables should not be based on the results of unadjusted analyses, as such an approach can lead to invalid findings.

The association between younger age and follow-up rates has at least two possible explanations. First, we observed that older patients were more likely to be at home when called. Younger study participants were more likely to be employed, and they may have also had more scheduled activities during the day, making them less available to take phone calls. We did have procedures for calling patients at their preferred time, including during the early evening and on Saturday, but this may still have been insufficient. Second, younger age has been associated with less adherence in other contexts, such as medication adherence(Kripalani, Risser, Gatti, & Jacobson, 2009), and this general tendency may have a carry-over effect to other adherence behaviors, such as study follow-up(Kripalani et al., 2009).

Prior research has also found that age and education (though not health literacy, specifically) were associated with incomplete follow-up (Ettinger, Qian, Xie, & Watkins, 2004; Gill et al., 2012; Ndiaye et al., 2009; Stewart et al., 2012).It is important to note that this previous research was conducted with different populations, in different countries, and with diverse follow-up methods (telephone, home visit, and clinic visit), yet similar results were reported. Conversely, others have found sex and race to be associated with attrition(Allman et al., 2011; Badawi, Eaton, Myllyluoma, Weimer, & Gallo, 1999; Fair et al., 2008; Janson, Alioto, Boushey, & Asthma Clinical Trials, 2001; Menchine et al., 2013; Ndiaye et al., 2009; Stewart et al., 2012), though they were not significantly associated in our results. This could be due to the fact that the effects of race and/or gender are no longer significant when the model controls for a diverse set of predictors (for instance, health literacy, financial difficulty, and other behavioral factors). Another possibility is that, due to small numbers in certain racial groups (e.g., Asian), we collapsed race to compare Whites versus non-Whites. This may have limited our ability to detect more subtle differences in follow-up likelihood between races.

Many of the other patient factors assessed here have also been under-studied in the context of research follow-up, including patient adherence behaviors, self-rated global health status, and health care system trust. We examined all of these factors in the context of a largestudy with diverse patient enrollment that included robust, well-validated measures such as the s-TOFHLA, PROMIS, PHCS, PHQ-8, and ARMS-7 to determine the factors that contribute the most to follow-up completion.

While another study with a similar population found mild depression to be associated with attrition (Bambs et al., 2013), our sample did not deliver the same results. During no follow-up time point did depression appear to be even closely associated with greater loss to follow-up. While the populations are similar, this difference may be attributed to the type of research being conducted: prevention versus observational.

The observed associations should be interpreted in the context of the study's response rates and follow-up duration. Response rates were relatively high for a prospective cohort study, running approximately 89% at each follow-up time point. This may have been the case because of the fairly short time period for follow-up (i.e., months rather than years).Other studies with a longer follow-up period are more likely to experience attrition, particularly as participants move, change contact information, and are generally less interested in remaining involved (Elliott & Shepherd, 2006).Furthermore, the high completion rate could be likely due to rigorous study procedures including multiple attempts at follow-up, use of the EMR to find additional contact information, and use of a toll-free call-back number enabling patients to actively reach out to research staff. That being said, perhaps those methods are not effective in the populations of patients who are younger or with lower health literacy and other modes of contact are needed.

The extensive use of the EMR was another critical strength of this study. This resource was also used to determine whether or not patients were currently hospitalized at VUH. If RAs saw that a patient was hospitalized during his/her follow-up time, RAsattempted to approach the patient in the hospital to complete follow-up. Similarly, if RAs saw that a patient was in an inpatient rehab facility or another hospital, the RAs attempted to contact the patient via the facility. The EMR was also used to search for an indication that the patient was deceased. This ultimately enabled the research team to better determine if patients should be coded as deceased, rather than as having incomplete follow-up.

This study only included patients with ACSor ADHF, which may limit generalizability. The inclusion of a very specific population may help explain the negative results which have been typically been seen as associations in other studies. However, the inclusion of ACS and ADHF patients could potentially inform large studies involving patients with cardiovascular disease. We found that patients with ADHF were more likely to miss more calls than patients with ACS, as shown in the proportional odds model. Our observation was that these patients were more likely to feel too unwell to participate in calls when attempted. Special considerations should be made to improve follow-up rates for sicker patients, particularly during the period after hospital discharge, when they are often still recovering from the acute exacerbation.

Finally, we noted that different patient factors were associated with greater loss to follow-up at different time intervals, suggesting that these patient factors may not be consistent for longer follow-up beyond ninety days. Additional research is warranted to look at how a diverse group of patient factors influence follow-up over a longer time period (i.e. six months, one year, or multiple years).

Many of the factors we observed to be associated with lower follow-up rates do not appear to be easily reversible (i.e. age and difficulty paying bills), but there are several possible ways to compensate for these patient factors. For example, stratifying patients into higher risk groups (by age or health literacy) and giving patients alternative options to respond is one possibility. Some younger people may be more responsive to electronic messaging or completion of surveys via mobile devices, rather than telephonic follow-up, and those that work may be more willing to respond to mail or electronic mail surveys. In any case, if health literacy and age are important variables associated with a study's hypotheses, this paper suggests that incomplete follow-up response data is more likely to occur for these groups and hence will be more problematic.

V. Conclusion

Patients who had lower health literacy and were younger were more likely to have incomplete follow-up over three stages of research data collection. Being aware of these association sin the planning stages of a study could allow for the researchers to target patients with more aggressive means to ensure that patients more equally complete research follow-up.

Acknowledgments

We acknowledge the following additional members of the VICS research team who contributed to the study design or conduct: Courtney Cawthon, MPH; Catherine Couey; Olivia Dozier; Catherine Evans; Vanessa Fuentes; Frank E. Harrell, PhD; Blake Hendrickson; Daniel Lewis; Abby G. Meyers, MD; Heidi Poppendeck; Monika Rizk; Hannah Rosenberg; Russell L. Rothman, MD, MPP; Amanda H. Salanitro, MD, MS, MSPH; Eduard E. Vasilevskis, MD, MPH; Ken Wallston, PhD; Kelly H.S. Wright, MA.

This research was supported by the National Heart Lung and Blood Institute (R01 HL109388), and in part by the National Center for Research Resources (UL1 RR024975-01), now at the National Center for Advancing Translational Sciences (UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health. The authors' funding sources did not participate in the planning, collection, analysis or interpretation of data or in the decision to submit for publication.

References

  1. Allman RM, Sawyer P, Crowther M, Strothers HS, 3rd, Turner T, Fouad MN. Predictors of 4-year retention among African American and white community-dwelling participants in the UAB study of aging. Gerontologist. 2011;51(Suppl 1):S46–58. doi: 10.1093/geront/gnr024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Badawi MA, Eaton WW, Myllyluoma J, Weimer LG, Gallo J. Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981-1996. Soc Psychiatry Psychiatr Epidemiol. 1999;34(2):91–98. doi: 10.1007/s001270050117. [DOI] [PubMed] [Google Scholar]
  3. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38(1):33–42. doi: 10.1016/s0738-3991(98)00116-5. [DOI] [PubMed] [Google Scholar]
  4. Bambs CE, Kip KE, Mulukutla SR, Aiyer AN, Johnson C, McDowell LA, Reis SE. Sociodemographic, clinical, and psychological factors associated with attrition in a prospective study of cardiovascular prevention: the Heart Strategies Concentrating on Risk Evaluation study. Ann Epidemiol. 2013;23(6):328–333. doi: 10.1016/j.annepidem.2013.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Barry AE. How attrition impacts the internal and external validity of longitudinal research. J Sch Health. 2005;75(7):267–270. doi: 10.1111/j.1746-1561.2005.00035.x. [DOI] [PubMed] [Google Scholar]
  6. Boys A, Marsden J, Stillwell G, Hatchings K, Griffiths P, Farrell M. Minimizing respondent attrition in longitudinal research: practical implications from a cohort study of adolescent drinking. J Adolesc. 2003;26(3):363–373. doi: 10.1016/s0140-1971(03)00011-3. [DOI] [PubMed] [Google Scholar]
  7. Cotter Robert B, Burke Jeffrey D, Stouthamer-Loeber Magda, Loeber Rolf. Contacting participants for follow-up: how much effort is required to retain participants in longitudinal studies? Evaluation and Program Planning. 2005;28(1):15–21. doi:http://dx.doi.org/10.1016/j.evalprogplan.2004.10.002. [Google Scholar]
  8. Davis LL, Broome ME, Cox RP. Maximizing retention in community-based clinical trials. J Nurs Scholarsh. 2002;34(1):47–53. doi: 10.1111/j.1547-5069.2002.00047.x. [DOI] [PubMed] [Google Scholar]
  9. Donovan-Kicken E, Mackert M, Guinn TD, Tollison AC, Breckinridge B, Pont SJ. Health literacy, self-efficacy, and patients' assessment of medical disclosure and consent documentation. Health Commun. 2012;27(6):581–590. doi: 10.1080/10410236.2011.618434. [DOI] [PubMed] [Google Scholar]
  10. Dumville JC, Torgerson DJ, Hewitt CE. Reporting attrition in randomised controlled trials. BMJ. 2006;332(7547):969–971. doi: 10.1136/bmj.332.7547.969. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Elliott J, Shepherd P. Cohort profile: 1970 British Birth Cohort (BCS70) Int J Epidemiol. 2006;35(4):836–843. doi: 10.1093/ije/dyl174. [DOI] [PubMed] [Google Scholar]
  12. Ettinger RL, Qian F, Xie XJ, Watkins CA. Evaluation and characteristics of “dropouts” in a longitudinal clinical study. Clin Oral Investig. 2004;8(1):18–24. doi: 10.1007/s00784-003-0238-z. [DOI] [PubMed] [Google Scholar]
  13. Fair AM, Wujcik D, Lin JM, Egan KM, Grau AM, Zheng W. Timing is everything: methodologic issues locating and recruiting medically underserved women for abnormal mammography follow-up research. Contemp Clin Trials. 2008;29(4):537–546. doi: 10.1016/j.cct.2008.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fewtrell MS, Kennedy K, Singhal A, Martin RM, Ness A, Hadders-Algra M, Lucas A. How much loss to follow-up is acceptable in long-term randomised trials and prospective studies? Arch Dis Child. 2008;93(6):458–461. doi: 10.1136/adc.2007.127316. [DOI] [PubMed] [Google Scholar]
  15. Gill RS, Karmali S, Hadi G, Al-Adra DP, Shi X, Birch DW. Predictors of attrition in a multidisciplinary adult weight management clinic. Can J Surg. 2012;55(4):239–243. doi: 10.1503/cjs.035710. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Goodman Jodi S, Blum Terry C. Assessing the non-random sampling effects of subject attrition in longitudinal research. Journal of Management. 1996;22(4):627–652. [Google Scholar]
  17. Green StephanieM, Navratil JudithL, Loeber Rolf, Lahey BenjaminB. Potential dropouts in a longitudinal study: Prevalence, stability, and associated characteristics. Journal of Child and Family Studies. 1994;3(1):69–87. doi: 10.1007/BF02233912. [DOI] [Google Scholar]
  18. Gupta C, Bell SP, Schildcrout JS, Fletcher S, Goggins KM, Kripalani S, Vanderbilt Inpatient Cohort, Study Predictors of health care system and physician distrust in hospitalized cardiac patients. J Health Commun. 2014;19(Suppl 2):44–60. doi: 10.1080/10810730.2014.934936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gustavson K, von Soest T, Karevold E, Roysamb E. Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health. 2012;12:918. doi: 10.1186/1471-2458-12-918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)- A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hays RD, Bjorner JB, Revicki DA, Spritzer KL, Cella D. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res. 2009;18(7):873–880. doi: 10.1007/s11136-009-9496-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Janson SL, Alioto ME, Boushey HA, Asthma Clinical Trials, Network Attrition and retention of ethnically diverse subjects in a multicenter randomized controlled research trial. Control Clin Trials. 2001;22(6 Suppl):236S–243S. doi: 10.1016/s0197-2456(01)00171-4. [DOI] [PubMed] [Google Scholar]
  23. Kapteyn A, Michaud PC, Smith JP, van Soest A. Effects of Attrition and Non-response in the Health and Retirement Study. 2006 Working paper. Retrieved from http://www.econstor.eu/bitstream/10419/33824/1/515865575.pdf.
  24. Kripalani S, Bengtzen R, Henderson LE, Jacobson TA. Clinical research in low-literacy populations: using teach-back to assess comprehension of informed consent and privacy information. IRB. 2008;30(2):13–19. [PubMed] [Google Scholar]
  25. Kripalani S, Risser J, Gatti ME, Jacobson TA. Development and evaluation of the Adherence to Refills and Medications Scale (ARMS) among low-literacy patients with chronic disease. Value Health. 2009;12(1):118–123. doi: 10.1111/j.1524-4733.2008.00400.x. [DOI] [PubMed] [Google Scholar]
  26. Kristman VL, Manno M, Cote P. Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol. 2005;20(8):657–662. doi: 10.1007/s10654-005-7919-7. [DOI] [PubMed] [Google Scholar]
  27. Kroenke K, et al. The PHQ-8 as a measure of current depression in the general population. Jounral of Affective Disorders. 2009;114:163–173. doi: 10.1016/j.jad.2008.06.026. [DOI] [PubMed] [Google Scholar]
  28. Lindau ST, Basu A, Leitsch SA. Health literacy as a predictor of follow-up after an abnormal Pap smear: a prospective study. J Gen Intern Med. 2006;21(8):829–834. doi: 10.1111/j.1525-1497.2006.00534.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lugtig Peter. Panel Attrition: Separating Stayers, Fast Attriters, Gradual Attriters, and Lurkers. Sociological Methods & Research. 2014 doi: 10.1177/0049124113520305. [DOI] [Google Scholar]
  30. Menchine M, Oberfoell S, Schriger D, Walker C, Riddell J, Arora S. Improving telephone follow-up for patients discharged from the emergency department: results of a randomized controlled trial. Acad Emerg Med. 2013;20(5):456–462. doi: 10.1111/acem.12128. [DOI] [PubMed] [Google Scholar]
  31. Meyers AG, Salanitro A, Wallston KA, Cawthon C, Vasilevskis EE, Goggins KM, Kripalani S. Determinants of health after hospital discharge: rationale and design of the Vanderbilt Inpatient Cohort Study (VICS) BMC Health Serv Res. 2014;14:10. doi: 10.1186/1472-6963-14-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ndiaye B, Ould-Kaci K, Salleron J, Bataille P, Bonnevie F, Choisy P, Yazdanpanah Y. Incidence rate and risk factors for loss to follow-up in HIV-infected patients from five French clinical centres in Northern France - January 1997 to December 2006. Antivir Ther. 2009;14(4):567–575. [PubMed] [Google Scholar]
  33. Psaty BM, Cheadle A, Koepsell TD, Diehr P, Wickizer T, Curry S, Wagner EH. Race- and ethnicity-specific characteristics of participants lost to follow-up in a telephone cohort. Am J Epidemiol. 1994;140(2):161–171. doi: 10.1093/oxfordjournals.aje.a117226. [DOI] [PubMed] [Google Scholar]
  34. Shea JA, Micco E, Dean LT, McMurphy S, Schwartz JS, Armstrong K. Development of a revised Health Care System Distrust scale. J Gen Intern Med. 2008;23(6):727–732. doi: 10.1007/s11606-008-0575-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Smith MS, Wallston KA, Smith CA. The development and validation of the Perceived Health Competence Scale. Health Educ Res. 1995;10(1):51–64. doi: 10.1093/her/10.1.51. [DOI] [PubMed] [Google Scholar]
  36. Stewart JI, Moyle S, Criner GJ, Wilson C, Tanner R, Bowler RP, For The Copdgene, Investigators Automated telecommunication to obtain longitudinal follow-up in a multicenter cross-sectional COPD study. COPD. 2012;9(5):466–472. doi: 10.3109/15412555.2012.690010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Sudore RL, Landefeld CS, Williams BA, Barnes DE, Lindquist K, Schillinger D. Use of a modified informed consent process among vulnerable patients: a descriptive study. J Gen Intern Med. 2006;21(8):867–873. doi: 10.1111/j.1525-1497.2006.00535.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Sullivan Cris M, Rumptz Maureen H, Campbell Rebecca, Eby Kimberly K, Davidson William S. Retaning Participants in Longitudinal Community Research: A Comprehensive Protocol. The Journal of Applied Behavioral Science. 1996;32(3):262–276. doi: 10.1177/0021886396323002. [DOI] [Google Scholar]
  39. Waldrop-Valverde D, Guo Y, Ownby RL, Rodriguez A, Jones DL. Risk and protective factors for retention in HIV care. AIDS Behav. 2014;18(8):1483–1491. doi: 10.1007/s10461-013-0633-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Woolard RH, Carty K, Wirtz P, Longabaugh R, Nirenberg TD, Minugh PA, Clifford PR. Research fundamentals: follow-up of subjects in clinical trials: addressing subject attrition. Acad Emerg Med. 2004;11(8):859–866. doi: 10.1111/j.1553-2712.2004.tb00769.x. [DOI] [PubMed] [Google Scholar]

RESOURCES