Using Case–Control Studies to Assess the Prevention of Inflicted Traumatic Brain Injury

Abstract

Because the incidence of inflicted traumatic brain injury (inflicted TBI) is low, even in populations at increased risk, very large samples are necessary to have adequate statistical power to conduct a randomized clinical trial of the effectiveness of a potential intervention to prevent inflicted TBI. This requirement for large samples, in addition to the logistic demands of prospective clinical trials, makes it prohibitively expensive to conduct such studies. Case–control studies provide a statistically efficient and logistically and economically feasible alternative approach to evaluating such interventions. However, because these are observational studies, they are susceptible to bias. Approaches are presented to conducting and analyzing case–control studies to evaluate interventions to prevent inflicted TBI while assessing and minimizing possible bias.

Introduction

Case–control studies have been instrumental in allowing investigators to identify causal associations between risk factors and certain adverse outcomes, such as the associations between cigarette smoke and lung cancer, oral contraceptives and thromboembolism, and stilbesterol and vaginal cancer.^1,2 In addition to providing important clues to such associations, case–control studies have also been used to assess the effectiveness of a variety of interventions, particularly vaccines.^3,4 Case–control studies are especially useful when the outcome being assessed is relatively rare. This article briefly reviews this technique, its advantages, some of its potential pitfalls, and how it might be applied to assess the effectiveness of interventions to prevent inflicted traumatic brain injury (inflicted TBI).

In contrast to longitudinal studies (including randomized clinical trials) in which subjects are selected based on their exposure either to a potential risk factor or to an intervention and are then followed forward in time to assess one or more outcomes (Figure 1), the subjects in case–control studies are selected based on their outcomes. That is, a group of subjects that have the outcome of interest (e.g., cancer, an infection, or inflicted TBI) is identified and they become the cases. Another group of subjects that does not have the outcome of interest is selected and they become the controls. The investigator then looks back in time to determine the proportion of each group that previously was exposed to the factor of interest (e.g., a risk factor, such as cigarette smoke, or an intervention, such as a drug, a vaccine, or an educational program) (Figure 2). The measure of association for case–control studies, the odds ratio (OR), is calculated from this information as shown in Table 1. For rare diseases, the OR closely approximates the relative risk (risk ratio [RR]) that would be calculated in a longitudinal study.⁵

Figure 1.

Basic architecture of a randomized (R) clinical trial to assess the protective efficacy of Program X in preventing inflicted traumatic brain injury (inflicted TBI) in infants.

Figure 2.

Basic architecture of a case–control study to assess the effectiveness of Program X in preventing inflicted TBI in infants.

Table 1.

Contingency table for a case–control study

	Cases	Controls
Exposed	a	b
Unexposed	c	d

Odds ratio = $\frac{a}{c}÷\frac{b}{d}=\frac{ad}{bc}$

Methods for Conducting Studies

Determining whether Program X, which consists of distributing a pamphlet that explains the dangers of shaking an infant followed by verbal reinforcement by a social worker, is effective in preventing inflicted TBI is of importance. The ideal way of answering this question would be to conduct an experimental study, a randomized clinical trial, in which subjects are assigned randomly to receive either the intervention (Program X) or a placebo (perhaps, Program Y, a pamphlet that explains the consequences of overfeeding an infant followed by verbal reinforcement by a nutritionist). All subjects would be followed forward in time. The proportions of infants with inflicted TBI in the two groups would be determined, from which the protective efficacy of Program X would be determined.

Although randomized trials are the gold standard for assessing therapeutic efficacy, their disadvantages often make it impossible to conduct such studies.^6,7 If the outcome of interest is rare, the statistical power of randomized trials generally is poor. Consequently, very large samples may be needed to conduct a statistically meaningful study. This, coupled with the need for longitudinal follow-up of the subjects, makes such studies very expensive. In addition, it may be necessary to have both individual patients and their physicians agree to allow subjects to be allocated randomly to receive an ineffective placebo, which may be ethically unacceptable. Thus, logistic, financial, and ethical obstacles often make clinical trials a practical impossibility.

Case–control studies provide a statistically powerful, logistically manageable, and ethically acceptable alternative.^1–3 They do not require longitudinal follow-up, and because they are observational (the investigator does not influence who receives the intervention or who is exposed to a risk factor), ethical issues usually are not a problem. In a case–control study of the effectiveness of Program X, children with inflicted TBI would be identified and enrolled as cases. Children without inflicted TBI would be enrolled as controls. The proportions of each group whose parent had received Program X would then be determined and the OR calculated. The effectiveness of the intervention is estimated as 1–OR. A theoretic example of what might be found is shown in Table 2. In this example, 25 of 100 cases and 50 of 100 controls received Program X. The OR is 0.33. This difference is statistically significant by the chi-square test. In other words, Program X worked and was 67% effective (1–OR) in preventing inflicted TBI.

Table 2.

Effectiveness of program X in preventing inflicted traumatic brain injury (inflicted TBI)

	Cases	Controls
Received Program X	25 (25%)	50 (50%)
Did not receive Program X	75	50
Total	100	100

OR=(25 × 50) / (50× 75) = 0.33

Effectiveness=1− OR= 1−0.33 = 0.67 or 67%

χ² = 13.33; p<0.001

Of course, to be able to conduct such a study, there must be adequate variation in the exposure of the population to Program X. Depending on the frequency of the outcome of interest, it could be possible to conduct a case–control study of a relatively rare exposure if the magnitude of the effect of the exposure is large and/or the size of the sample is large. However, it would be virtually impossible to conduct a case–control study if either almost all or almost none of the subjects were exposed (i.e., received the intervention), because the size of the sample that would be needed in such circumstances would be prohibitively large. Examples of the number of cases (and the size of the population from which the cases emerge) that would be needed to assess the effectiveness of an intervention to prevent inflicted TBI, given certain assumptions about the magnitude of the effect of the intervention and the incidence of inflicted TBI as well as the proportions of controls that are exposed are shown in Table 3. Note that increasing the number of controls per case can reduce the size of the sample that is needed, although the incremental reduction is small once the number of controls per case exceeds two or three. Matching controls to cases also will increase statistical power, but it makes statistical analyses more complex.

Table 3.

Approximate size of population and number of cases needed for 80% statistical power to be able to detect effectiveness of Program X^a

# of controls/cases	50%	60%	75%
If 70% of controls received Program X
1	520,000 (156)	287,000 (86)	133,000 (40)
2	384,000 (115)	217,000 (65)	100,000 (30)
3	340,000 (102)	193,000 (58)	90,000 (27)
4	317,000 (95)	180,000 (54)	83,000 (25)
5	303,000 (91)	173,000 (52)	80,000 (24)
If 50% of controls received Program X
1	473,000 (142)	310,000 (93)	150,000 (45)
2	353,000 (106)	230,000 (69)	110,000 (33)
3	310,000 (93)	203,000 (61)	93,000 (28)
4	290,000 (87)	187,000 (56)	87,000 (26)
5	277,000 (83)	180,000 (54)	83,000 (25)

^aAssumes the annual incidence of inflicted traumatic brain injury in the population is 30/100,000. Numbers in parentheses are the number of cases needed

Analysis of Results

Many issues must be considered when conducting and analyzing case–control studies. For example, suppose it was suspected or known that a particular factor, such as substance abuse, was an important determinant of the risk of inflicted TBI. Awareness of this association might also lead physicians to be more likely to refer or to recommend that a parent who is known to have a problem with substance abuse participate in a program to prevent inflicted TBI. In this instance, patients with a problem with substance abuse might have both a different (higher) risk of developing the outcome (inflicted TBI) and a different likelihood of receiving the intervention (Program X) than other patients. Factors associated with differences in both the risk of developing the outcome and the likelihood of receiving the intervention (or of being exposed to a risk factor) are called confounders. If the proportion of substance abusers among cases differs from the proportion of substance abusers among controls, it could lead to biased results. It is important to identify potential confounders and to take steps to control for the effects of these factors in either the design of the study or in the statistical analyses. Appropriate strategies include matching, stratification, and multivariate analyses.^1,2

In a matched study, the potential confounder is used as a matching variable when controls are selected. This assures that the same proportion of cases and of controls will have the potential confounder (e.g., substance abuse). Consequently, the matched factor should not affect the results. In addition, either matching on or controlling for the severity of the substance abuse and/or the length of time the subject has been an abuser also might be important. Although matching protects against bias, it means that the effect of the factor cannot be measured, which is one disadvantage of this strategy. Another disadvantage is that the greater the number of variables used for matching (e.g., age of the parent or socioeconomic status), the more difficult it is logistically to identify appropriate controls.

Stratification involves classification of the subjects into groups (strata) based on a confounding factor. Because all of the cases and the controls within each stratum are homogeneous with regard to the confounder (i.e., the confounder is either present or absent in all subjects within the stratum), the relationship between the outcome and exposure is not affected by the confounder. Weighted estimates of the ORs from each stratum may be combined to get a summary estimate of the OR, adjusted for the affect of the stratified variable.⁸ However, the ability to stratify is limited by the sample size. The creation of multiple strata in a small sample leaves individual strata with too few subjects to yield meaningful estimates.

Finally, a multivariate statistical model, such as logistic regression, could be used to control for confounding. This technique utilizes a mathematical model in which the log odds of the outcome are expressed as a function of the intervention or risk factor of interest plus the other factors that are entered in the equation. This method has the advantage that the simultaneous affects of multiple, different confounders can be assessed and controlled. One potential disadvantage is that statistical power may be diminished with the addition of each variable to the equation. In addition, it is not intuitively clear for many people just how this technique works.

Bias

Another potential pitfall of case–control studies is that because the patients are not allocated randomly to the different groups, like most observational studies, case–control studies are susceptible to biases.^1,2,8–10 These include detection bias (people identified as cases may not be representative of all people in the population with the outcome); selection bias (people selected as controls may not be representative of all people in the population without the outcome); and information bias (in which the accuracy of ascertainment of exposure differs between cases and controls). Both detection bias and selection bias are forms of sample distortion bias. However, it is important to recognize that bias will occur only if the samples are not representative with respect to the joint distribution of both exposure and outcome.

In fact, detection bias is likely to be a potential problem in a case–control study of interventions to prevent inflicted TBI in infants. For example, given certain symptoms, infants from lower socioeconomic strata might be more likely to undergo tests that would detect inflicted TBI (e.g., they may be more likely to be evaluated in an emergency department where it may be more likely that imaging of the brain would be done than in a child from a higher socioeconomic stratum who might see a private pediatrician in an office setting who might be less likely to order imaging). The same bias might occur based on the socioeconomic status of the parents alone, even if both children were seen in an emergency department. Parents in lower socioeconomic strata might also have a different likelihood of being exposed to the intervention than parents in higher socioeconomic strata. Detection bias might be less likely to occur among children with more severe degrees of injury; thus, as one way to assess the occurrence of this potential bias, the consistency of estimates of the effect of an intervention in different strata of severity of injury might be examined. Another strategy that has been used in cancer studies to try to avoid detection bias is the utilization of a diagnostic registry to select both cases and controls. For example, cases might be subjects who had a routine screening mammogram and were found to have breast cancer, while controls would be subjects who had a screening mammogram that was found to be normal. If studying inflicted TBI, controls might be children with unintentional injuries on whom a computerized tomographic scan of the brain was performed.

Selection bias might occur in a study of inflicted TBI if cases were identified through hospital-based surveillance while controls were selected from rosters of patients in private practices. Such a strategy likely would result in substantial bias, since confounders likely would be distributed unevenly among the groups. Strategies to try to avoid selection bias include selecting controls from population-based lists (e.g., birth certificates), selecting controls randomly from a list of potential controls, or selecting controls from the same diagnostic registry from which the cases emerge.

Information bias, another important potential bias to consider, might occur in a case–control study of the effectiveness of an intervention for inflicted TBI if ascertainment of exposure to the intervention is not equivalent in the two groups. This might be particularly problematic for simple interventions, such as distributing written material or attending a brief, postpartum discussion. Parents whose child has the outcome of interest (i.e., inflicted TBI) may be more (or perhaps less) likely, 1 or 2 years later, either to recall or to report having participated in the intervention than the parent of an unaffected child. For this reason, ascertainment of participation in or receipt of the exposure that was documented in writing at the time is likely to be a more-reliable and less-biased method of ascertaining exposure than depending on the recall and reporting of a parent. Extensive discussion of the many potential biases that may affect case–control studies can be found elsewhere.^{1, 9–12}

Another powerful strategy to help assess the validity of a case–control study is to use “sham” outcomes and/or “sham” exposures and compare these results to the results when true outcomes and exposures are used.¹² An example is a case–control study that showed that sigmoidoscopy was effective in reducing mortality from cancer of the distal colon (which is visualized by sigmoidoscopy) but not of the proximal colon (which is not visualized by sigmoidoscopy).¹³ In a case–control study of inflicted TBI, a potential sham outcome might be unintentional injuries.

Use of sham exposures can perform a similar function. A sham exposure that might be used in a study of inflicted TBI might be another intervention that was likely to occur at about the same time as Program X is offered—for example, receipt of hepatitis B vaccine in the immediate postpartum period.

Conclusion

The case–control method is a type of observational study that can be extremely useful to investigators who want to assess the effectiveness of an intervention. Because it is a statistically powerful technique, the effectiveness of interventions to prevent inflicted TBI can be assessed with much smaller samples than would be necessary for a randomized clinical trial of the intervention. However, because case–control studies are observational rather than experimental, especially careful attention to and assessment of potential biases are crucial to ensure the validity of such studies.