The older the better: are elderly study participants more non-representative? A cross-sectional analysis of clinical trial and observational study samples

Abstract

Objective

Study participants can differ from the target population they are taken to represent. We sought to investigate whether older age magnifies such differences, examining age-trends, among study participants, in self-rated level of activity compared to others of the same age.

Design

Cross-sectional examination of the relation of participant age to reported ‘relative activity’ (ie, compared to others of the same age), a bidirectionally correlated proxy for relative vitality, in exemplars of randomised and observational studies.

Setting

University of California, San Diego (UCSD)

Participants

2404 adults aged 40–79 including employees of UCSD, and their partners (San Diego Population Study, observational study). 1016 adults (aged 20-85) not on lipid medications and without known heart disease, diabetes, cancer or HIV (UCSD Statin Study, randomised trial).

Measurements

Self-rated activity relative to others’ age, 5-point Likert Scale, was evaluated by age decade, and related via correlation and regression to a suite of health-relevant subjective and objective outcomes.

Results

Successively older participants reported successively greater activity relative to others of their age (greater departure from the norm for their age), p<0.001 in both studies. Relative activity significantly predicted (in regression adjusted for age) actual activity (times/week exercised), and numerous self-rated and objective health-predictors. These included general self-rated health, CES-D (depression score), sleep, tiredness, energy; body mass index, waist circumference, serum glucose, high-density lipoprotein-cholesterol, triglycerides and white cell count. Indeed, some health-predictor associations with age in participants were ‘paradoxical,’ consistent with greater apparent health in older age—for study participants.

Conclusions

Study participants may not be representative of the population they are intended to reflect. Our results suggest that departures from representativeness may be amplified with increasing age. Consequently, the older the age, the greater the disparity may be between what is recommended based on ‘evidence, ’ and what is best for the patient.

Trial Registration

UCSD Statin Study—Clinicaltrials.gov # NCT00330980 (http://ClinicalTrials.gov)

Article summary.

Article focus

• Persons who are less vigorous or less healthy may be less able and apt to participate in research studies, and as age advances, more participants may be less vigorous and less healthy.

• We therefore hypothesised that progressively older adults who do participate in studies, may be progressively less representative of the group they are intended to reflect—eg, more non-representatively vigorous and robust.

• We examined this, in  an observational study sample and in a clinical trial sample, using a question that enquired how active participants deemed themselves to be relative to others their age—a measure that was ‘validated’ against assessments of actual exercise and objective tests with a known relation to exercise—that are also known to predict health outcomes.

Key messages

• Successively older study participants departed progressively more (favourably) from others their age.

• Since healthier people have a better risk–benefit balance with many drugs and procedures, these findings dictate that with increasing age of patients and participants, studies may convey progressively more strongly (typically favourably) distorted estimates of risk–benefit balance for many treatments.

• Often few elderly are included in studies, and it is recognised that findings in younger participants may distort risk–benefit balance for older ones. These findings suggest that even if studies focus upon elderly (or older elderly), a misleading picture of risk–benefit balance for that age-group may be conveyed.

• An unsettling implication is that ‘evidence-based’ medicine is generally not meaningfully evidence-based, where the elderly are concerned.

Strengths and limitations of this study

• Self-report was used to ascertain activity relative to others of the same age (because objective evaluation comparing to non-participants cannot be done, as non-participants have not consented to participate). However, this self-report measure is likely to be a good index, as it showed strongly significant relationships to other measures known to relate to activity including exercise measures and objective health predictors, within the study population.

• Strengths of the study include assessment in a randomised and observational sample (with concordant findings); inclusion of a broad age range; and validation of the measure against multiple other health- and activity-relevant variables.

Introduction

Relevance of data from human research studies to the general population depends on the similarity of study participants to those they are taken to represent, that is, the ‘target’ population. It is recognised that study samples may differ from the target population.^{1 2} Often the study sample directly or disproportionately excludes the elderly^3–5 who have worse health and higher expected mortality⁶ and who may differ from younger participants in treatment effects.

Although there has been increasing emphasis (at least in principle) on inclusion of the elderly in studies,⁷ there are reasons for concern that elderly study participants may be less representative of their age group than younger participants.

Self-selection by participants themselves of a relatively healthier and more functional study population may occur in all ages,^8–11 since even morbidity not requiring exclusion may nonetheless inhibit participation.¹ But since health problems and functional limitations that lead to self-exclusion may increasingly affect those older in age, we theorised that older age participants might be progressively less representative in indices relevant to function and vitality. Direct comparison of consenting participants to non-participants is problematic, since inherently the researcher has access only to the former group. Participants’ ratings of themselves relative to others their age provides a tentative approach to evaluate whether departures rise with age, if such relative measures can be validated against direct measures.

We validated ‘relative-activity’, that is, self-rated activity-level compared to other individuals of the same age, against an activity metric that is absolute (vs relative), and assessed its relation to health-relevant outcomes. We examined reported relative-activity, compared to other individuals one's age, from available exemplars of two types of medical studies (observational and randomised controlled trial) to evaluate whether reported departure from normative function rises with increasing participant age.

Methods

Randomised controlled trial participants:

A total of 1016 male and female participants aged 20–85 from the San Diego area were enrolled in the UCSD Statin Study, a double-blind, randomised, placebo-controlled trial assessing the effects of statin cholesterol-lowering drugs on a relatively broadly sampled group of adults (a primary prevention sample). There was no imposed upper age limit. Participants were men over age 20 and surgically or chronologically postmenopausal women not on lipid medications and without extremes of LDL-cholesterol (high or low), diagnosed cardiovascular disease, diabetes or HIV. More information on study population and design is available elsewhere.¹²

Observational study participants

In total, 2404 selected men and women aged 40–79 were enrolled in the San Diego Population Study, a population-based observational study identifying prevalence of arterial and venous disease. Participants were drawn from current and former employees of the University of California, San Diego (UCSD), as well as their spouses/significant others—inclusion of which modestly extended the age range of participants in both directions.¹³ In addition, a small number of non-UCSD volunteers were included. Participants represented a spectrum of socioeconomic status, including unemployed and retired as well as working persons. A full description of the study population is available elsewhere.¹³

Both studies were approved by the UCSD Human Research Protections Program, and all participants gave informed consent to participate.

Relative activity variable

Participants in both studies were asked to rate their level of physical activity ‘Compared to other persons your age’ on a 5-point Likert scale (1=much less active, 2=somewhat less active, 3=about as active, 4=somewhat more active, 5=much more active). We refer to this activity rating as ‘relative activity.’ Single-item self-rated assessments have shown strong predictive validity.^14–16

Validation variables

Other measures used: From the randomised trial, several other variables were chosen against which to validate the relative activity variable. All variables were assessed at baseline (prior to study treatment).

Absolute activity: We validated the relation of this relative activity measure to self-reported actual exercise frequency (number of episodes of vigorous activity for at least 20 min over a week). Direct measurement of activity was not performed, but self-reported exercise-frequency related significantly to objective measures known to be affected by exercise (eg, body mass index (BMI), triglycerides, high-density lipoprotein (HDL)-cholesterol, each p<0.001) in age–sex-adjusted analysis.

Health predictor variables: Self-rated and questionnaire variables known to predict mortality and health outcomes that were considered against relative activity included depressed mood (Center for Epidemiological Studies—Depression Scale aka CES-D), and single-item self-ratings of energy, tiredness, muscle weakness, fatigue with exertion, overall health, and satisfaction with health. Objective measures included platelet count (acute phase reactant), white cell count, serum glucose, HDL-cholesterol, triglycerides, BMI, and waist circumference.

Analyses

Self-rated relative activity was tabulated by age decade. For each study, significance of self-rated relative activity change with age was assessed across the full age range. Activity associations and health implications of the relative activity measure were examined in older study participants (age >50) from the randomised trial sample (in which these health variables were assessed), using correlation and also regression analysis. (Both by expectation and empirically in this sample, people in their 30s and 40s were comparatively similar in their self-rated relative activity, consistent with the expectation that age-related health conditions are not yet strongly present, leading to the emphasis on those over age 50.) In the latter, age-relative activity was the independent variable, and assessments were adjusted for actual age.

For both study samples, we conducted bivariate analyses examining the reported relative activity level as a function of age decade. This was followed by multivariable regression using ordinal logit with robust standard errors (aka White standard errors)¹⁷ controlling for sex, ethnicity (categorical variable) and education (scaled from 1=grade school or less to 9=doctoral degree).

All analyses were conducted using Stata™ V.8.0; StataCorp, College Station, Texas, USA. Two-sided p values less than 0.05 were designated statistically significant.

Results

Self-reported activity relative to others’ age related strongly to actual activity: (unadjusted) correlation 0.42, p<0.0001; (adjusted) regression beta (SE) 1.2 (0.092), p<0.001.

Self-rated activity relative to others’ age also related strongly to multiple measures known to predict health, healthcare utilisation and mortality, such as general self-rated health, energy, tiredness, depression (CES-D), sleep, muscular weakness, fatigue with exertion and metabolic syndrome factors of HDL, triglycerides, BMI, waist circumference and serum glucose (table 1).

Table 1.

Self-rated ‘relative activity’* relates to health predictors (age>50)

Variable	Correlation coefficient	p Value	Regression coefficient (SE), age adjusted	p Value	Age relation, sign and p value
Times/week exercise at least 20 mins	0.42	<0.0001	1.2 (0.092)	<0.001	(−) 0.024
CES-D (0–52)	−0.21	<0.0001	−1.3 (0.23)	<0.001	NS
Depressed (0–10)	−0.13	0.0083	−0.21 (0.086)	0.017	NS
Energy (0–10)	0.21	<0.0001	0.34 (0.064)	<0.001	(+) 0.031
Sleep problems (0–10)	−0.084	0.028	−0.21 (0.095)	0.024	NS
Sleep quality (0–30)	0.078	0.036	0.35 (0.21)	0.092	(+) 0.018
Tired (0–10)	−0.29	<0.0001	−0.72 (0.13)	<0.001	(+) 0.001
Muscle weakness (0–10)	−0.14	<0.0002	−0.29 (0.070)	<0.001	(+) 0.005
Fatigue with exertion (0–10)	−0.26	<0.0001	−0.61 (0.12)	<0.001	(+) 0.002
Health (0–10)	0.20	<0.0001	0.31 (0.061)	<0.001	(+) 0.071
Satisfaction with health (0–100)	0.30	<0.0001	5.6 (0.69)	<0.001	NS
Glucose (mg/dl) ‡	−0.073	0.049	−0.73 (0.31)	0.019	(+) 0.014
HDL (mg/dl)	0.10	0.0063	1.2 (0.53)	0.028	(+) 0.001
Triglycerides (mg/dl)	−0.17	<0.0001	−10 (2.3)	<0.001	NS
Body mass index	−0.26	<0.0001	−0.97 (0.15)	<0.001	(−) 0.002
Waist (cm)	−0.23	<0.0001	−3.9 (0.63)	<0.001	NS
Platelets	−0.073	0.051	−2.7 (1.7)	0.11	(−) 0.043
White cell count	−0.083	0.027	−0.13 (0.050)	0.012	(+) 0.058

*Level of activity ‘compared to other persons your age’ measured on a 5-point Likert scale: 1=much less active, 2=somewhat less active, 3=about as active, 4=somewhat more active, 5=much more active.

†Regression examines relative activity level relation to health predictor, among those age >50, with age as an adjusted covariate in the regressions.

(The column to the far right gives the sign of the coefficient for the age variable, and its significance.)

‡Note: patients with diabetes or measured glucose over 142 were excluded. This finding is thus despite range restriction.

Note that in these study participants, there is a ‘paradoxically’ favourable age association for some variables that generally worsen with rising age, including energy, sleep quality, health and HDL-cholesterol.

CES-D, Center for Epidemiological Studies-Depression scale; HDL, high density lipoprotein cholesterol; NS, non-significant.

Self-rated relative physical activity showed a graded positive relation to age on unadjusted analysis (p<0.001) (table 2). This was true in the clinical trial sample and the observational study sample. Findings were monotonic in the observational sample, and nearly so in the clinical trial sample for participants from their 40s to 80s.

Table 2.

‘Relative-activity’ ratings*, by age

	Clinical trial sample		Observational sample
Age decade	N	Relative activity* Mean (SD)	N	Relative activity* Mean (SD)
30s	80	3.35 (1.02)	34	3.26 (1.24)
40s	180	3.30 (1.20)	565	3.27 (1.23)
50s	308	3.49 (1.20)	650	3.68 (1.15)
60s	261	3.92 (1.07)	569	3.94 (1.05)
70s	151	3.89 (1.01)	512	3.97 (1.04)
80s	20	4.10 (1.07)	24	4.17 (1.31)
Significance of change by age decade		p<0.001		p<0.001

*Level of physical activity ‘compared to other persons your age’ measured on a 5-point Likert scale: 1=much less active, 2=somewhat less active, 3=about as active, 4=somewhat more active, 5=much more active.

N, number.

Multivariable regression (table 3) affirmed that a significant relation of age to reported relative activity was retained with adjustment for variables (sex, ethnicity and education level) that could relate to age and activity of participants (p<0.001).

Table 3.

Older participant age associated with greater self-reported ‘relative-activity’*, ordinal logit analyses

	Clinical trial sample				Observational sample
	Coefficient for age decade	Standard error	p Value	95% CI	Coefficient for age decade	Standard error	p Value	95% CI
Unadjusted	0.29	0.045	<0.001	0.21 to 0.38	0.37	0.033	<0.001	0.31 to 0.44
Multivariable adjusted†	0.35	0.053	<0.001	0.25 to 0.45	0.37	0.035	<0.001	0.31 to 0.44

*Level of activity ‘compared to other persons your age’ measured on a 5-point Likert scale: 1=much less active, 2=somewhat less active, 3=about as active, 4=somewhat more active, 5=much more active.

†Ordinal logit adjusted for age, gender, ethnicity, and education level.

Discussion

To our knowledge this is the first explicit demonstration that progressively older study participants may depart successively more from parity with those they are taken to represent, in observational and clinical trial settings. This was found in exemplars of observational studies and clinical trials. Adults in their 30s and 40s reported being only modestly more active than others their age (closest to ‘about as active’ as others). By the oldest decades, participants had surpassed the ‘somewhat more active’ mark, even on average, and were partway, on average, towards the maximum rating of ‘much more active’ compared to others their age. These differences by age were strongly significant.

This finding is concordant with expectations that might be generated from previous observations linking study participation with higher health and vitality. All participants who self-select for study participation may differ in systematic ways from the target population or population as a whole.^8–11 Prior studies have noted that clinical trial participants are generally younger and healthier than referred and registry patients.⁴ Our results further show that successively older participants who do participate in research studies may be successively less typical of their age cohort in a metric with an expected—and indirectly observed—relation to health. For instance, it related to general self-rated health, which has been found to strongly predict physical function/disability, healthcare utilisation and mortality.^14–16 Relative activity also related in expected directions to other assessed factors known to predict health and mortality in elderly, such as fasting glucose¹⁸ white cell count,^19–21 HDL-cholesterol,²² sleep problems^{23 24} and depression.^25–31

Our evidence accords with and extends recent evidence from survey studies. Participants who indicated (on a survey) they would volunteer for an exercise study reported less physical function decline, more physical activity and less chronic pain than those who would not, as well as better self-reported health;³² however, these reflect hypothetical intentions rather than participation, and the manner in which participants were shown to be differential focused largely on domains that may affect comfort and performance for that study's assessments. A survey study of Finns aged 52–76 found that ‘Favorable health was generally more frequent among respondents than non-respondents,’ gauging health status by medicine reimbursements (ascertained by linking to register data).³³ Whether disparities progressed successively as age advanced was not ascertained.

Selective participation by healthier elderly has potential to influence trial outcomes. This is particularly true for outcomes for which vitality, function, activity or any of the range of health-relevant correlates of relative activity may serve as effect modifiers. (Such health correlates include those elucidated here, and presumably many others that were not examined.) The study also has relevance for outcomes for which differences in participants’ activity and/or function, through their relation to expected health, may modify study power. For example, a doubling or halving of mortality by an intervention (or with a risk factor), even in the absence of effect modification, will have lower statistical power in a sample with lower baseline risk of mortality outcomes (as a healthier sample portends). Healthier elderly may reduce power for the risk-side of the equation, which can shift the apparent risk–benefit balance.

Limitations of the present analysis are several. Activity relative to others of the same age was assessed by self-report. Objective evaluation of non-participants, to permit direct comparison, is inherently problematic (as they have not consented to participate). This limitation is mitigated by the demonstration of strongly significant relationships of relative activity to health predictors within the study population. (A relation to hard outcomes like mortality could not be assessed: the observational study was not longitudinal, and the trial sample enrolled generally healthy participants with only 6-month follow-up.)

It is possible that participants may over-represent their functional state relative to others, but this would not produce an expected age association. In principle, older participants may differ from younger participants in the manner of such amplification, but there is little reason to believe this is the case, and the age-adjusted association of our relative activity measure to an exercise frequency measure further diminishes this concern. There is reason to predict that as limiting comorbidities and disabilities accrue with rising age, and as function and the ability to sustain activity declines progressively with age, more elderly individuals will more often find participation too burdensome—yielding a successively more rarefied sample that is progressively more non-representatively robust and healthy, compatible with the findings shown. Indeed, better health has been reported to influence self-selection for participation in studies in general,¹ an observation that might be predicted to drive the finding observed, since health problems increase in prevalence with increasing age.

Factors driving self-selection for participation may vary depending on the character of the study. Although theoretical considerations suggest our findings may generalise broadly, other studies should evaluate how these findings are moderated based on the type of study and condition being examined.

One unsettling implication is that clinical guidelines lack a meaningful evidence basis, when applied to those of older age. Concerns have previously been expressed that when ‘evidence based’ study findings based on younger individuals are implemented in elderly patients with comorbidities, via clinical practice guidelines reinforced by performance pay, this may result in perverse incentives that may diminish rather than enhance quality of care for elderly,³⁴ by promoting promiscuous polypharmacy. Our findings suggest that such concerns pertain even when recommendations derive from data actually procured in elderly participants. (Analogous concerns may apply, irrespective of age, for patients with multiple comorbidities, polypharmacy, dementia, disability, limited life expectancy and/or past adverse responses to the recommended treatment—groups that, like elderly, often bear less favourable risk–benefit prospects.)

For older elderly, some have urged a more individualised ‘less is more’ approach placing a greater emphasis on clinical judgment, quality of life and in-depth consultation with the patient and family.^34–36 This seems rational, given (1) the absence of applicable evidence that medication benefits similarly apply, (2) increased medication burden, as age-related morbidities accrue, (3) amplified risk of drug-adverse events, drug interactions and medication-taking errors in elderly with implications to quality of life and function, (4) magnified impact of added functional compromise in the elderly, coupled with (5) evidence, albeit non-randomised, suggesting striking subjective and objective benefits among elderly when a systematic discontinuation of medications is undertaken.^{35 36}

In conclusion, as age advances, those who participate in clinical trials and observational studies may depart increasingly from those they are taken to represent. That is, real patients may depart increasingly from an ever more rarefied, non-representative, healthiest subsegment of the elderly population who volunteers to participate in clinical studies, rendering study findings of increasingly doubtful applicability. This magnifies concerns that, as the elderly swell as a fraction of the population, the chasm may grow, between what is recommended based on ‘evidence’, and what is best for the patient.