Reply to Hughes and Beganovic, “Introduction of Selection Biases Due to Loss to Follow-Up in Infectious Disease Retrospective Outcomes Studies”

REPLY

Hughes and Beganovic wish to make the point that the inclusion of retrospective studies with patients who were lost to follow-up introduces selection bias in a meta-analysis (1). In reference to our analysis (2), they commented on 3 specific studies (3–5). Notably, the study in reference 4 was not included in our meta-analysis (details in reference 2). Regarding the study by Giannella et al. (3), it reported unavailable clinical data in 53 patients of 856 patients included and analyzed, and as noted in reference 2, the authors used a propensity score matching for receiving short-course treatment to minimize bias arising from antibiotic treatment selection (3).Regarding the study in reference 5, for our analysis, we communicated with the authors and obtained detailed information on the patients that were lost to follow-up. As detailed in Table 1 of our study (2), we presented the number of patients who were lost to follow-up from each treatment group: 26 patients from the short-course group and 95 from the long-course group. The study excluded patients in whom antibiotic treatment was dictated by outcome or clinical response (5), and since the short- and long-course groups included 117 and 294 patients, respectively, the loss to follow-up corresponded to 18.2% and 24.4% of each group.

It is self-evident that a meta-analysis of multiple, large, randomized controlled trials is ideal and that pooling of nonrandomized studies bears a greater potential for bias (6). However, data from multiple, randomized controlled trials are not always available, and authors, such as the authors of the studies that were discussed in the previous paragraph, can address the potential of bias arising (i.e., propensity score matching and exclusion of specific patients from the analysis). Moreover, there are well-established tools that allow the assessment of quality. For example, in reference 2, we used the Newcastle-Ottawa scale, and the studies were graded high with 8 (3) and 7 (5) stars.

In conclusion, as we noted in reference 2, meta-analyses carry inherent limitations and should be interpreted with caution. Synthesizing nonrandomized studies to generate pooled estimates and answer important clinical questions can be acceptable when randomized data are unavailable or scarce. Well-conducted observational studies can yield effects similar to those obtained from randomized studies when the risk of bias which arises exclusively from observational studies is small (7).