In this issue, we answer three questions with respect to loss to follow-up in a clinical trial:
How important is loss to follow-up?
How is loss to follow-up calculated?
How many patients can be lost to follow-up without mistrusting the results?
1. How important is loss to follow-up?
The simple answer to this question is “very important” because loss to follow-up can severely compromise a study's validity. Incomplete follow-up biases the results when either:
The dropout rates are different between study groups; or
The patients who drop out are different from those who do not drop out.
Why do these situations make a difference? Because in each situation, those lost to follow-up often have a different prognosis than those who complete the study. For example, patients who receive treatment for cervical myelopathy may not return for follow-up because they became asymptomatic and felt no need to return to see the surgeon. Conversely, some patients may not return because they had a particularly bad outcome (worse pain or function) or complication, or because they died. In either case, bias can affect the validity of the inferences drawn from the study.
2. How is loss to follow-up calculated?
There is much confusion about how to determine the proportion of patients lost to follow-up. In order to correctly calculate the follow-up rate, one needs to know the denominator. In a randomized controlled trial (RCT), the denominator for each group is the number of patients who were randomized, not the number who received the treatment. For example, suppose we have an RCT comparing two treatment groups, Group A and Group B. The investigators evaluate 178 patients and randomize 120; 61 to Group A and 59 to Group B (Fig 1). Following the figure, we note that 49 patients received treatment A and 52 received treatment B. At the final follow-up 40 were analyzed in Group A and 41 in Group B. How many were considered lost to follow-up? Many would consider the loss to follow-up rate to be 9 (18%) of 49 in treatment A and 11 (21%) of 52 in treatment B using as the denominator only those that were treated. However, the real proportion lost to follow-up must consider those who were randomly assigned, even if they did not receive treatment. In the present example, this is calculated as 21 (34%) of 61 for treatment A and 18 (31%) of 59 for treatment B.
Fig 1.
Hypothetical example of patients lost to follow-up in a randomized controlled trial.
When calculating loss to follow-up in a retrospective cohort study, all individuals receiving treatment during the study period should be used as the denominator, not just those with complete data. For example, let's say you want to compare decompression plus lumbar fusion with decompression alone in disc herniation and the data available are all patients receiving either treatment in the last 5 years (N = 275). However, the database from which the data are obtained is incomplete and only 190 have the necessary data available. Since the investigators stated as part of the inclusion criteria that only those patients with complete data are included, they consider the follow-up to be 100% (190/190). This conclusion is wrong. The denominator should include all patients who underwent the surgery irrespective of completeness of data. The follow-up rate for this example is 69% (190/275).
3. How many patients can be lost to follow-up without mistrusting the results?
Some have suggested that <5% loss leads to little bias, while >20% poses serious threats to validity 1. This may be a good rule of thumb, but keep in mind that even small proportions of patients lost to follow-up can cause significant bias 2. One way to determine if loss to follow-up can seriously affect results is to assume a worst-case scenario with the missing data and look to see if the results would change. Here is an example:
Let's assume a multicenter study enrolled 500 patients into each arm of a study comparing artificial disc replacement (ADR) with fusion, and the end point is adjacent segment disease (ASD). The trial numbers are found in Fig 2.
Fig 2.
Hypothetical example of the effect of loss to follow-up considering a worst-case scenario. ADR indicates artificial disc replacement; ASD, adjacent segment disease.
The proportion of patients with ASD in the ADR group is half as much versus the fusion group, 25% (100/400) compared with 50% (200/400). If we assume that the 100 lost to follow-up in the ADR group had ASD and the 100 lost to follow-up in the fusion group did not, then the rate of ASD in each group would be 40% (200/500). In this case, adopting the worst-case scenario for the intervention group with respect to those lost to follow-up causes the results to change significantly from half the rate of ASD with ADR to the same rate. When this happens, loss to follow-up can threaten the internal validity of the trial. Only when the worst case does not change the inferences derived from the results is lost to follow-up not a problem.
Conclusions
Loss to follow-up is very important in determining a study's validity because patients lost to follow-up often have a different prognosis than those who complete the study.
Properly calculating the loss to follow-up can only be done by determining the right denominator. That includes all those randomly assigned in an RCT, and all who had the procedure during a pre-specified time in a retrospective cohort study.
A good rule of thumb is that <5% loss leads to little bias, while >20% poses serious threats to validity. However, even less than 20% loss to follow-up can be a problem. Considering a worst-case scenario can help determine whether loss to follow-up poses a potential threat to validity.
References
- 1.Sacket D L, Richardson W S, Rosenberg W, New York: Churchill Livingstone; 1997. Evidence-Based Medicine: How to Practice and Teach EBM. [Google Scholar]
- 2.Bhandari M, Guyatt G H, Swiontkowski M F. User's guide to the orthopaedic literature: how to use an article about a surgical therapy. J Bone Joint Surg Am. 2001;83(6):916–926. doi: 10.2106/00004623-200106000-00015. [DOI] [PubMed] [Google Scholar]