Factors Associated with Attrition of Adult Participants in a Longitudinal Database: A National Institute on Disability, Independent Living, and Rehabilitation Research Burn Model System Study

Abstract

Participant attrition in longitudinal studies can lead to substantial bias in study results, especially when attrition is nonrandom. A previous study of the Burn Model System (BMS) database prior to 2002 identified participant and study-related factors related to attrition. The purpose of the current study was to examine changes in attrition rates in the BMS longitudinal database since 2002 and to revisit factors associated with attrition. Individuals 18 years and older enrolled in the BMS database between 2002 and 2018 were included in this study. Stepwise logistic regression models identified factors significantly associated with attrition at 6, 12, and 24 months postburn injury. The percentage of individuals lost to follow-up was 26% at 6 months, 33% at 12 months, and 42% at 24 months. Factors associated with increased risk of loss to follow-up across two or more time points include male sex, lower TBSA burn size, being unemployed at the time of burn, shorter duration of acute hospital stay, younger age, not having private health insurance or workers’ compensation, and a history of drug abuse. Retention levels in the BMS have improved by at least 10% at all time points since 2002. The BMS and other longitudinal burn research projects can use these results to identify individuals at high risk for attrition who may require additional retention efforts. Results also indicate potential sources of bias in research projects utilizing the BMS database.

INTRODUCTION

Longitudinal studies have important advantages over cross-sectional studies, including the ability to examine causal relationships and evaluate patterns over time. However, they also present many significant challenges, including participant attrition. It is widely recognized that attrition in longitudinal studies can reduce both statistical power and cause bias. Specifically, attrition bias occurs when dropout of participants is not random, and variables associated with attrition are correlated with outcome variables under evaluation. Past research suggests that attrition of up to 20% may be acceptable and limit bias.¹ However, studies have found when attrition is between 20 and 40%, there can be significant bias in study results, especially when attrition is nonrandom.² In studies of individuals that last for years or decades, achieving follow-up rates of up to 80% are nearly impossible, even when significant study resources are available. When study resources are limited, this task becomes even more difficult and statistical methods that attempt to estimate the impact of attrition on study bias are the only available options.

Of key importance to both minimizing attrition and evaluating the effect of attrition on study outcomes is determining factors associated with attrition within and among longitudinal studies. By evaluating and identifying the characteristics of persons at highest risk of loss to follow-up, researchers can prioritize the use of resources and use recruitment and retention strategies that are tailored toward these high-risk individuals. In addition, by understanding which factors impact attrition within a study, researchers can better understand how attrition might bias-specific research outcomes and results and attempt to address those using statistical approaches.

Research indicates that a variety of participant factors can influence attrition. Several factors that have been consistently associated with dropout across populations include male sex, being single, low socioeconomic status, smoking, low education, minority ethnic group, substance abuse, and depression.^3–7 Other factors, including age and poor health or co-morbidities, have been inconsistently associated with attrition and are more population dependent.^3,4,8–10 For example, in persons with spinal cord injury or traumatic brain injury, participants with more medical conditions and higher injury severity are more likely to have follow-up data^10,11; alternatively, a longitudinal study of women’s health found that those with poorer health had higher loss to follow-up³.

In addition to participant characteristics, many study-specific factors have also been found to be associated with attrition. These can vary widely and include things such as sensitive nature of the study topic, inadequate study resources for locating and following participants, inconvenience of study location or procedures, transportation barriers, amount and type of study compensation, and failure to achieve good rapport with study participants.^12–14 Thus, improving retention must focus on both improving study-related factors where possible and identifying participants at highest risk of loss to follow-up so that efforts can be optimized for maximizing the number of retained study participants.

In this study, we sought to examine factors specifically associated with loss to follow-up in persons with burn injuries. To date, only three studies have examined attrition in persons with burn injuries.^15–17 Findings based on data collected by the Burn Model System (BMS) National Longitudinal Database up until 2002 found that younger age, lower education, un-employment, prior drug abuse, and no health insurance were associated with lower rates of follow-up, while longer hospital stay was associated with increased likelihood of retention.¹⁶ Recruitment site was also significantly associated with attrition rates, signaling that study-related factors may also be of importance as data collection sites utilized different recruitment and retention methods. The findings of the previous BMS study¹⁶ resulted in the adoption of additional strategies to improve recruitment and retention, as well as standardization of procedures across BMS sites where possible.¹⁸ Some of these changes included implementation of locator services by all BMS centers in 2007, thank you letters sent upon completion of follow-up surveys, and in 2006 changes in recruitment criteria to require surgery for wound closure in 2006, leading to focused recruitment and retention of persons with more severe burns.¹⁹ The specific purpose of this study was to (i) examine changes in retention rates in the BMS longitudinal database since 2002; (ii) revisit factors associated with attrition found by a previous study of the BMS database¹⁶ in order to examine sources of bias in analyses done using BMS data and to identify participant subgroups that are most at risk for dropout and could benefit from additional retention efforts.

METHODS

Sample

The dataset used for the analyses in this paper includes records from the Burn Injury Model System National Longitudinal Database (BMS NDB). The BMS was established in 1994 and was funded by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR). Beginning in 1994, funded BMS grantees began collecting information on survivors of burn injury, including demographics, clinical characteristics (such as cause of injury and days of hospital stay), and health outcomes using both medical record review and self-report survey. The goals and ideology behind the BMS program have been previously described²⁰ and an overview of the type of data available has been published.^18,20,21 A previously published study looking at loss to follow-up in the BMS database¹⁶ utilized records for individuals with a burn injury between 1994 and October 1, 2002. The current study sought to explore loss to follow-up between October 1, 2002 and September 30, 2018, and to examine predictors of loss to follow-up during this time period. All available data from four sites including the UW Medicine Regional Burn Center at Harborview Medical Center in Seattle, Washington (UW), the University of Texas Southwestern Medical Center at Dallas (UTSW), Texas, Boston-Harvard Burn Injury Model System (consisting of burn centers at Massachusetts General Hospital, Shriners Hospital for Children Boston, and Brigham and Women’s Hospital), and Johns Hopkins Regional Burn Center in Baltimore, Maryland (JH) were included. Excluded were data from the University of Texas Medical Branch/Shriners Hospital BMS center (site 5, Figure 1), as it almost exclusively recruits pediatric participants. The current study focuses on adult participants; reasons for loss to follow-up are potentially different in juvenile populations.^22,23 All participants met the BMS inclusion criteria that were current at the time of their recruitment. The requirements and changes made to the recruitment criteria over time are detailed in Table 1. In addition, the principles outlined in the Declaration of Helsinki were followed and informed consent was obtained from all participants.

Figure 1.

Flowchart of participant participation at each follow-up time point and reasons for loss to follow-up. Site 5 refers to the University of Texas Medical Branch/Shriners Hospital center and was excluded for reasons described in the text. JH refers to individuals from the Johns Hopkins Regional Burn Center with a follow-up window post September 30, 2012 (see text).

Table 1.

Summary of enrollment eligibility criteria for the Burn Model System (BMS) by year

Prior to 2005	2006–2008	2009–2018
Deep second or third degree burns TBSA burned >10% in patients over 50 years old Deep second or third degree burns >20% TBSA in all other age groups Deep second or third degree burns with serious threat of functional or cosmetic threat that involve face, hands, feet, genitalia, perineum, or major joints Third degree burns >5% TBSA in any age group Deep electrical burns including lightning injury; inhalation injury with burn injury Circumferential burns of the extremity or chest	2005 criteria with the below changes: Burn surgery for wound closure required TBSA requirements by age group were changed to >10% for those 65 years or older and 20% for those 18 to 64 years Participants were also required to receive primary treatment at a burn model systems center with reasonable expectation of follow-up treatment at a BMS center	2006 criteria with the below changes: Exclusion of frostbite, toxic epidermal necrolysis, abrasions, necrotizing fasciitis, meningococcemia, and other skin disease

Participants’ data were included in this study if they were 18 years or older at the time of injury, received care from one of the four listed burn centers, and consented to participate in the BMS data collection process. Baseline data were collected from consenting participants within 30 days of their acute hospital discharge following their burn injury. Follow-up data were collected during specific follow-up windows as follows: 6 (±2)-month post injury, 12 (±3) months postinjury, and 24 (±6) months postinjury. Consistent with the previous study in this population,¹⁶ participants were considered lost to follow-up when phone calls, mailings, and clinic visits failed to result in data collection from the individual during the follow-up window. If a participant’s follow-up window had not yet closed on September 30, 2018, the data point was treated as missing and the participant was not included in the follow-up calculation rate. A participant was defined as not lost to follow-up if any part of the assessment was completed either in person, by mail, or over the phone, regardless of completeness of the assessment. In addition, participants were not considered lost to follow-up if at least some data were collected using medical record review for specific variables collected during the follow-up window. Consented individuals were still considered eligible for subsequent follow-up regardless of their participation at previous follow-up intervals unless they specifically requested to be withdrawn from the study. Individuals who died after admission for their initial injury were excluded from all calculations. Those who refused consent at initial contact were excluded from the study because they were not tracked and no effort was made to follow these individuals. Participants who died after consent were excluded from analyses for any subsequent follow-up visits, as the intention of this study was to examine other factors aside from mortality associated with loss to follow-up.

Measures

Since its inception in 1994, the information and instruments used to collect data in the BMS database have evolved, resulting in some inconsistency in outcome measures collected at specific time points or sites.¹⁸ Only variables available across participants, regardless of when they were enrolled, were utilized in this study. Information is collected using both medical record review and self-report surveys. Demographic variables utilized in this study included age in years, sex, employment status at time of burn, race/ethnicity, location of residence postacute discharge, and primary sponsor of care (ie, insurance type) at discharge. Employment status was recoded as a yes/no variable by recoding the initial response options as follows (1 = working; 2 = not working, homemaker/caregiver, volunteer, or retired). Ethnicity and race were combined to generate the categories of white non-Hispanic, Hispanic (white or nonwhite), Black non-Hispanic, and any other race (“other”). The insurance variable was collected with the following sponsor options: Medicare, Medicaid, Private Insurance, Workers’ Compensation/Labor and Industry, Health Maintenance Organization (HMO)/Preferred Provider Organization (PPO)/Prepaid/Managed care, Champus, Self-pay, Indigent, Public Support, Veterans Affairs (VA), Other, Philanthropy/Private Support, and Unknown. For all analyses, these insurance categories were collapsed based on similarity: category 1 includes “private insurance, HMO, PPO, VA, and Champus,” category 2 includes “Medicaid, self-pay, indigent, and philanthropy,” category 3 includes “Medicare,” and category 4, “Workers’ compensation.” Because marital status was not consistently recorded over time by the BMS, the variable which assessed that the participant was living with at the time of postacute injury release was used as a proxy. For living situation, participants who listed themselves as living with a spouse or partner were coded as “living with partner” and those living alone or in other situations were coded as “other.” Burn and injury-related variables considered in the analyses included site of recruitment location (ie, burn center site), burn size (percent TBSA burned), length of hospital stay in days, and if the injury was job-related or intentional. Additional variables considered in the analyses included concomitant medical problems and a past history of alcohol or drug abuse.

Analyses

Overall follow-up rates for each assessment period were calculated by comparing the number of individuals with or without follow-up out of those eligible for each assessment period and is consistent with the methodology used by Holavanahalli et al.¹⁶ Univariate analyses were completed for each variable considered for inclusion in the regression using chi-square tests for categorical variables and Wilcoxon rank-sum tests for continuous variables regardless of statistical significance in subsequent regression analyses. Logistic regression analyses with robust standard errors were completed by beginning with a full model that included all variables listed in Table 2 with the exception of participation in prior study assessment periods. Prior participation at 6 or 12 months was not included in the models as the purpose of the regression analyses was to determine whether demographic, medical, or burn-related variables were associated with loss to follow-up. In addition, individuals recruited by the JH site were dropped from the regression models if their follow-up window closed after September 30, 2012, because JH was no longer funded after this date and no follow-ups were attempted for individuals from that site after this date. Table 2 includes prior study participation in order to examine the likelihood of participation once a participant did not respond or participate at a previous time point and differentiate the rate of follow-up among those who had already participated in one or more prior assessment periods. The dependent variable, follow-up status at each assessment period, was coded as 1 for those lost to follow-up and 0 for those not lost to follow-up. Stepwise backward regression was completed for each of the three assessment periods (6, 12, and 24 months) and only variables statistically significant at the α level of 0.05 were kept in the final model. All models included the variables of sex and age regardless of their significance. Site of recruitment was included as a covariate in all models in order to adjust for operational disparities that might exist between the four included study sites. Individuals with complete data on all variables in the full model were included in the regression analyses. For all three regression models, unusual and influential data points as well as the regression assumption of linearity were investigated. High leverage individuals were identified through visual inspection of DFBETA statistics, which are measures of standardized differences between regression coefficients when a given observation is included or excluded. Model adequacy and goodness of fit were evaluated using the model specification linktest²⁴and the Hosmer–Lemeshow goodness-of-fit test.²⁵ A sensitivity analyses was completed to compare those excluded from the regression analyses (ie, those with complete vs partial missing data) on age, race/ethnicity, insurance, drug use, employment prior to burn, percent TBSA burned, and length of hospital stay using chi-squared or Student’s t-tests.

Table 2.

Distributions and statistical tests of association of characteristics for Burn Model System participants discharged alive, consented and 18-year-old or older participants lost to follow-up

	6 Months				12 Months				24 Months
	Followed	Lost	% Lost	P-value	Followed	Lost	% Lost	P-value	Followed	Lost	% Lost	P-value
Number of eligible participants	2040				1954				1784
Overall sample follow-up status	1508	532	26%		1302	652	33%		1029	755	42%
Participants lost to follow-up by site
Site 1	521	164	24%	<.001	448	209	32%	<.001	353	241	41%	<.001
Site 2	632	121	16%		549	176	24%		420	251	37%
Site 3	184	197	52%		164	213	57%		156	219	58%
Site 4	171	50	23%		141	54	28%		100	44	31%
Ethnicity
White	1011	323	24%	.001	871	410	32%	.023	696	476	41%	.112
Black	198	103	34%		174	117	40%		138	130	49%
Hispanic	208	56	21%		177	72	29%		135	90	40%
Other	80	31	28%		72	32	31%		54	41	43%
Sex
Male	1068	391	27%	.24	916	489	35%	.031	722	562	44%	.047
Female	440	141	24%		386	163	30%		307	193	39%
Employed at time of burn
No	531	198	27%	.032	444	256	37%	.002	340	301	47%	<.001
Yes	962	285	23%		841	355	30%		676	415	38%
History alcohol abuse
No	1274	379	23%	.004	1107	478	30%	.001	882	574	39%	.001
Yes	188	84	31%		156	105	40%		114	119	51%
History drug abuse
No	1309	381	23%	<.001	1139	480	30%	<.001	916	566	38%	<.001
Yes	142	77	35%		116	97	46%		77	116	60%
Non-intentional injury
No	90	40	31%	.104	80	43	35%	.502	64	50	44%	.571
Yes	1394	450	24%		1201	566	32%		945	661	41%
Job-related injury
No	1133	399	26%	.019	968	493	34%	.009	747	576	44%	.001
Yes	351	91	21%		313	116	27%		262	135	34%
Where living at release
Other	435	163	27%	.138	366	201	35%	.07	298	228	43%	.374
Personal residence	1045	332	24%		911	413	31%		705	491	41%
Who living with at release
Alone	156	60	28%	.005	125	78	38%	<.001	102	79	44%	<.001
Spouse/partner	630	164	21%		587	194	25%		480	261	35%
Other	625	231	27%		509	295	37%		380	330	46%
Primary sponsor of care (i.e., insurance)
Private, HMO, PPO, VA, and Champus	487	114	19%	<.001	420	162	28%	<.001	336	190	36%	<.001
Medicaid, self-pay, indigent, philanthropy	498	230	32%		412	283	41%		315	317	50%
Medicare	220	70	24%		197	72	27%		145	94	39%
Workers compensation	248	50	17%		227	59	21%		187	81	30%
Followed at 6 months
No					113	388	77%	<.001	102	362	78%	<.001
Yes					1189	264	18%		927	393	30%
Followed at 12 months
No									109	481	82%	<.001
Yes									920	274	23%
Mean TBSA burned (%)	19.2	16.3		<.001	20.3	15.1		<.001	20.3	16.6		<.001
Mean age at follow-up (years)	44.5	42.4		.003	45.6	41.8		<.001	46.6	43.0		<.001
Length of acute hospital stay (days)	30.8	29.9		<.001	33.5	25.1		<.001	34.0	27.0		<.001

HMO: Health Maintenance Organization; PPO: Preferred Provider Organization; VA: Veterans Administration.

RESULTS

The final dataset included the following number of eligible individuals with closed follow-up windows for each of the follow-up time periods (Figure 1): 2040 at 6 months, 1954 at 12 months, and 1784 at 24 months. Of the 2040 individuals eligible for follow-up at 6 months 532 (26%) were lost to follow-up. This number increased to 652 of 1954 (33%) at 12 months and 755 of 1784 (42%) at 24 months.

Univariate Analyses

The results of the univariate analyses indicated that recruitment site location, previous participation at 6 or 12 months, no history of drug or alcohol abuse, living with a spouse/partner after discharge, workers compensation or private insurance, being employed at time of burn, having a job-related injury, larger % TBSA burn size, older age, and longer length of hospital stay at acute injury were all consistently associated with lower loss to follow-up at all time points at the α level of 0.05 (Table 2). There was no association between follow-up status and nonintentional injury or residence at discharge from acute hospital stay at all three time points. Ethnicity and sex were inconsistently associated with loss to follow-up across the three time points.

Multivariate Analyses:

Linearity, model adequacy. and goodness of fit were acceptable for all three models. Two high leverage individuals were identified for the 12-month model and were dropped from the analyses. Further examination of these individuals indicated that they had extreme values of TBSA burn size (>90%).²⁶

The sensitivity analysis indicated that individuals with partial missing data excluded from the regression models were not statistically different with respect to sex, age, insurance payer, or duration of hospital stay. However, differences were significant with respect to race/ethnicity, TBSA burned, and preburn employment at all three time points and prior drug use at 6 and 12 months. Individuals dropped due to partial missing data were less likely (all P ≤ .003) to be Hispanic (6% vs 15% at 6 months), less likely (all P ≤ .03) to be employed prior to burn (43% vs 36% at 6 months), report lower average TBSA burned (19% vs 15% at 6 months) (all P ≤ .003), and more likely (both P = .01) to report prior drug use (16% vs 11% at 6 months).

Six-Month Follow-Up

Age, sex, and site were included in the model regardless of their statistical significance in univariate analyses. Male sex, recruitment site, race/ethnicity, primary insurance, and prior history of drug abuse were the only variables statistically significantly associated with probability of loss to follow-up at 6 months. Specifically, individuals in the Medicaid/Self-Pay/Indigent/Philanthropy or Medicare groups had odds of loss to follow-up that were nearly twice that of individuals who had private insurance/HMO/PPO/VA/Champus (odds ratio (OR):2.13 and 1.92, respectively; Table 3). Conversely, individuals who listed workers’ compensation as their insurance had lower odds of loss to follow-up (OR: 0.82) than the private insurance reference group. Those with “other” race or ethnicity had odds of loss to follow-up twice that of non-Hispanic White individuals (OR: 2.13). For the drug abuse variable, individuals who reported prior drug abuse had higher odds of loss to follow-up (OR: 1.62).

Table 3.

Logistic regression results examining variables significantly associated with loss to follow-up at 6, 12, and 24 months postburn injury in the BMS database

	6 Months				12 Months				24 Months
	N = 1680, pseudo R2 = 0.098				N = 1596, pseudo R2 = 0.112				N = 1442, pseudo R2 = 0.086
	OR	z	P-value	(95% CI)	OR	z	P-value	(95% CI)	OR	z	P-value	(95% CI)
Male Sex	1.34	2.09	0.037	(1.02, 1.79)	1.71	3.75	<0.001	(1.29, 2.27)	1.57	3.34	0.001	(1.21, 2.06)
Age (10-year increment)	0.94	−1.28	0.199	(0.85, 1.03)	0.85	−3.63	<0.001	(0.77, 0.93)	0.83	−4.09	<0.001	(0.76, 0.91)
Site 2*	0.50	−4.25	<0.001	(0.37, 0.69)	0.55	−4.18	<0.001	(0.41, 0.73)	0.73	−2.31	0.021	(0.56, 0.95)
Site 3*	3.69	7.13	<0.001	(2.58, 5.28)	1.85	3.35	0.001	(1.29, 2.65)	1.53	2.32	0.021	(1.07, 2.19)
Site 4*	0.61	−1.95	0.052	(0.37, 1.00)	0.53	−2.55	0.011	(0.33, 0.87)	0.54	−2.43	0.015	(0.33, 0.89)
Blackα	1.08	0.43	0.664	(0.76, 1.53)
Hispanicα	0.85	−0.83	0.407	(0.57, 1.26)
Other race/ethnicityα	2.16	3.04	0.002	(1.31, 3.55)
Medicaid, self-pay, indigent, philanthropyß	2.13	4.65	<0.001	(1.55, 2.94)	1.50	2.64	0.008	(1.11, 2.02)	1.51	2.79	0.005	(1.13, 2.01)
Medicareß	1.92	2.86	0.004	(1.23, 3.00)	1.26	0.99	0.320	(0.80, 1.98)	1.41	1.56	0.119	(0.91, 2.18)
Workers compensationß	0.81	−1	0.316	(0.54, 1.22)	0.55	−2.93	0.003	(0.37, 0.82)	0.59	−2.79	0.005	(0.41, 0.86)
Prior drug abuse	1.62	2.46	0.014	(1.10, 2.39)	1.54	2.18	0.029	(1.04, 2.27)	2.31	4.44	<0.001	(1.60, 3.35)
Prior alcohol abuse					1.60	2.52	0.012	(1.11, 2.30)
Unemployed at time of burn					1.36	2.14	0.033	(1.03, 1.81)	1.48	2.74	0.006	(1.12, 1.95)
Not living with partner					1.55	3.42	0.001	(1.20, 1.98)
Length of acute hospital stay					0.90	−2.48	0.013	(0.83, 0.98)	0.93	−2.84	0.005	(0.89, 0.98)
(10-day increment)
TBSA burned (10% increment)					0.88	−2.47	0.014	(0.79, 0.97)	0.88	−2.88	0.004	(0.81, 0.96)

OR: odds ratio; CI: confidence interval.

^*Reference group is site 1;

^αReference group is White non-Hispanic;

^ßReferences group is private, HMO, PPO, VA, or Champus.

Twelve-Month Follow-Up

At the 12-month follow-up, age, sex, and site were all statistically significant predictors of loss to follow-up with male sex and younger age associated with higher odds of loss to follow-up (Table 3). As with 6 months, insurance and prior drug abuse history were both associated with probability of loss to follow-up. However, in addition, length of acute hospital stay in days, TBSA burn, prior alcohol abuse history, employment status at time of burn, and living situation were also significantly associated with follow-up status. More specifically, those with smaller burns, shorter hospital stays, a history of prior alcohol abuse, living without a partner, and being unemployed at the time of burn were more likely to be lost to follow-up (Table 3).

Twenty-Four-Month Follow-Up

Age, sex, and recruitment site were all significantly associated with the probability of loss to follow-up, with younger age and male sex having higher odds of loss to follow-up, similar to the 12-month model. Prior drug abuse, having shorter length of acute hospital stay, smaller TBSA burn, being unemployed at the time of burn, and having Medicaid/Self-pay/Indigent/Philanthropy or Medicare listed as insurance were all associated with increased odds of loss to follow-up (see Table 3).

DISCUSSION

The results of this study indicate that loss to follow-up rates in the BMS database have improved by at least 10% since 2002 at all time points, ranging from 26% (at 6 months) to 42% (at 24 months) between 2002 and 2018. The improved lost-to-follow-up rates suggest that strategies aimed at minimizing attrition have been effective. The results of the analyses suggest that those at highest risk of loss to follow-up across two or more time points include males, persons of younger age, shorter duration of acute hospital stay, individuals without private health insurance or workers’ compensation, persons with a history of drug abuse, being unemployed at the time of burn, and lower TBSA burn size. Factors less consistently associated with being lost to follow-up (ie, only one time points) were ethnicity/race (6 months), prior alcohol abuse (12 and 24 months), and not living with a partner or spouse postinjury (12 months).

The findings of this study support the prior study looking at pre-2002 data collected by the BMS¹⁶ in that all of the factors associated at one or more time points in this study were also associated with loss to follow-up at one or more time points in the previous study, with the exception of sex. Results of both studies can be used to develop targeted recruitment efforts to burn survivors with characteristics associated with greater risk of dropping out. In addition, BMS centers could create a nomogram, such as the one created by the SCI model system,²⁷ which would allow study staff to assess a new participant’s risk of loss to follow-up at the time of enrollment and individualize recruitment efforts. Study site also continues to play a significant role in retention of participants and exploring site differences in patient populations and recruitment strategies may be informative.

Our findings also indicate that once participants miss one data collection they are unlikely to respond to subsequent requests. Of those who did not respond or participate at 6 months, only 21 to 22% had follow-up data at future time points while 83 and 68% of those followed at 6 months had follow-up data at 12 or 24 months, respectively. Similarly, of those followed at 12 months, 76% had follow-up data 1 year later. This finding is to be expected as the reasons for loss to follow-up at 6 months are similar if not the same as reasons for loss to follow-up at later time points. For example, if a participant has moved and cannot be located by study staff at 6 months, it is unlikely that they would be located after additional time has passed. In addition, if the participant no longer needs or seeks burn care at a burn center at 6 or 12 months, it is unlikely they would need or seek burn care at a future time point. Thus, medical record abstraction of follow-up data would be impossible for these individuals and contact with study staff further limited because participants are no longer receiving clinical care. These findings imply that the earlier time points are important for retention efforts, and that the BMS may be better served by focusing limited resources on locating participants and collecting data at earlier time points.

The recommendations for follow-up rates for cohort studies range from 50 to 80%,^1,2 and in the literature cohort studies in other populations report response rates of anywhere between 46 and 92% at 1-year,¹⁴ though survey rates have been shown to be declining significantly worldwide in recent years.²⁸ The current BMS follow-up rates fall between the rates of follow-up in the Traumatic Brain Injury (TBI) and Spinal Cord Injury (SCI) model systems databases, which have different eligibility criteria^29,30 but are similarly managed, have multiple sites contributing data to the database, and report 41 and 23% attrition rates at 1-year follow-up, respectively.^{10, 27} This study also indicates that there are many overlapping factors that are associated with loss to follow-up across the three model systems databases. These factors include age, marital status, alcohol use/abuse, race/ethnicity, employment, severity of injury, and insurance payer.^{10, 27} Retention efforts should focus on groups that may exhibit a higher attrition rate, and researchers who include these traumatic injury populations in longitudinal studies should design their tracking and follow-up procedures accordingly. In addition, researchers in the BMS could consider additional tools and resources for further reducing attrition where feasible. For example, monetary or nonmonetary incentives, prenotification of survey, offering survey results, reducing the length of the survey to reduce participant burden or consideration of a core subset of measures to be administered to some or all participants may increase participation rates.³¹ The use of online survey administration and/or Computer Adaptive Testing also has the potential to reduce burden and increase participation, both of which recently have been or are currently being implemented by the BMS.

The same factors that put participants at higher risk of loss to follow-up in this study have also been shown to be associated with poorer health outcomes in persons with burn injuries.¹⁸ For example, lower satisfaction with life postburn injury is associated with prior alcohol abuse, marital status, and days in intensive care,³² all three of which are associated with loss to follow-up in this study. In addition, Esselman et al found that self-reported community integration is associated with sex, living situation, age, and preinjury employment.³³ Selective attrition due to for instance, substance abuse, may distort the relationship between predictive factors and outcomes, leading to the under or over-estimate of the impact of a particular factor on a specific outcome or limiting the generalizability of the results to the larger population of individuals with a burn injury. Thus, nonrandom attrition of individuals with these characteristics may have biased estimates of rates of community integration and levels of satisfaction with life postburn injury. The results of this study are of value to researchers who might wish to study health outcomes using the BMS data that may also be related to loss to follow-up. Researchers may wish to use statistical approaches and/or data imputation methods to handle missing data due to nonrandom loss to follow-up. Multiple methods for imputing missing data or for evaluating the impact of missing data on risk estimates exist,³⁴ though there is currently no recommended robust method³⁵ and incorrect application of statistical approaches may introduce additional bias.^{36, 37} However, at a minimum, researchers who use the BMS data can evaluate the potential for bias in their study based on these loss to follow-up results and provide adequate discussion and warning to those who wish to use or generalize their findings. Our data indicate that researchers presenting longitudinal patient-reported outcome data must address issues related to the characteristics of the groups who were lost to follow-up, whether the attrition is likely to have affected the generalizability of their results and provide appropriate sensitivity analyses if possible.¹

Study Limitations

This study has several limitations. First, analyses did not include participants who were lost to follow-up due to death, and therefore the results of these analyses are not as informative for studies looking at outcomes related to mortality. Second, a subset of individuals were included in the regression analyses because of missing data and therefore the regression results are not representative of the entire BMS sample. Third, types of attrition are not analyzed in depth. Loss to follow-up due to refusal vs inability to locate vs nonresponse are potentially different and warrant additional research.

CONCLUSIONS

The current study highlights factors associated with loss to follow-up in the BMS national database and provides BMS researchers with information for identifying participants who may require additional retention efforts. Furthermore, this study indicates that attrition has decreased over time, suggesting improvements can be made. However, reaching loss to follow-up rates <20% over an extended follow-up period is an extremely challenging task and success will depend on numerous factors, many of which are outside the control of the BMS researchers, including the clinical population in focus. In addition, some have suggested that trying to achieve a fixed level of “acceptable” follow-up is unhelpful and unnecessary as it could curtail research in the field.¹ Future efforts should focus on innovative follow-up methods, including use of social media and tracking databases, when approved by institutional review boards. Statistical techniques to minimize the impact of missing data should also be considered.