Pirfenidone in idiopathic pulmonary fibrosis

To the Editors

We read with interest the article of Taniguchi et al. [1] entitled “Pirfenidone in idiopathic pulmonary fibrosis”. These authors have spearheaded the investigation of this novel antifibrotic agent for idiopathic pulmonary fibrosis, and their efforts are commendable. In an editorial that accompanied their manuscript, Collard [2] pointed out a number of shortcomings of the study that raise concerns about the robustness of the results. We agree with Collard [2] (and the authors' comments in the Discussion section) that these limitations (and others not mentioned) detract from the validity of the study's results, the clinical significance of which continue to be questioned by the research community, and underscore the need for further investigation of pirfenidone.

We would like to use the article of Taniguchi et al. [1] to highlight some points regarding longitudinal studies in general, with particular emphasis on the use of statistical analyses and reporting of results. First, we believe the research community has advanced beyond the point where the use of mixed-effects (also called linear-mixed, random-effects, random-coefficient or hierarchical) models is a novelty; indeed, the capability to analyse data with these models has been available in almost every statistical package for several years. The merits of mixed-effects models include the following: 1) all available data can be used (imputation is not necessary, thus avoiding case-wise deletion due to missing data); 2) the model structure is extremely flexible, and, in contrast to ANCOVA, allows for time-varying covariates; and 3) it relaxes some of the assumptions about missing data inherent in analyses that do not use all of the data. These beneficial aspects permit these models to yield information regarding how groups change over time, how interventions effect that change and how patient characteristics might modify treatment effects [3]. Instead of using mixed-effects models to analyse data for their primary end-point, Taniguchi et al. [1] used ANCOVA, with imputation of missing data by last observation carried forward (LOCF), methods that fall short of what should, we believe, be the minimum standard for analysing longitudinal data.

In addition to using mixed-effects models to analyse longitudinal data, we urge investigators to move forward from the blanket use of LOCF for imputation of missing data; it may be appropriate to use LOCF in certain studies, but it may not. There are certain methods that authors can use to better convince readers that LOCF is right for their studies (a brief discussion follows), and sensitivity analyses are always indicated when LOCF is used.

Taniguchi et al. [1] used a mixed model approach as a sensitivity analysis, instead of using it as the method for the primary analysis, but they failed to mention the covariates or the covariance structure used in their approach. Further, they state that the sensitivity analysis “showed significant or marginally significant treatment effects and supported the LOCF,” but they do not reveal these results. Reporting these results, which could easily be made available in the online supplementary material, would further convince readers that the use of LOCF was reasonable.

Additional information not reported by Taniguchi et al. [1] that would help readers better understand the data includes full reporting of the numbers of subjects with missing data, which could be accomplished via a plot for each treatment arm stratified by last assessment (fig. 1). In figure 1, although LOCF would seem appropriate for group 1, it would not be for groups 2 or 3, because the outcome is on an upward trajectory for group 2 and a downward trajectory for group 3. Thus, for groups 2 and 3, LOCF artificially deflates or inflates outcome values for time-points after the last study visit.

Figure 1.

Mean value of outcome of interest for subjects whose last study visit occurred at 16 (group 1), 8 (group 2) or 4 weeks (group 3). BL: baseline.

Taniguchi et al. [1] write “We were under the impression that LOCF would not tip the balance in favour of either of the treatment groups, if there were no substantial differences in the rate of drop-outs.” However, drop-out rate only tells part of the story. A figure that includes plots for each arm stratified by reason for drop-out/missing data (e.g. lack of efficacy/disease progression, side-effects of treatment and other) would shed light on whether the trajectories prior to drop-out are similar for patients with different reasons for drop-out, and whether or not missing data are missing completely at random. This has implications regarding what methods should be used for imputing missing values. Whatever the case, readers need to know that authors are inadvertently making the arm with greater drop-out look better with their method for dealing with missing data.

In their study, Taniguchi et al. [1] were confronted with hurdles commonly faced by researchers who run longitudinal studies; these include drop-out and missing data. Mixed-effects models provide a powerful and versatile statistical method for analysing longitudinal data and should be used wherever possible. Efforts by authors, including a full disclosure of the number of missing data and the reason for missingness (accomplished with plots for each arm stratified on the reason why data are missing), will help readers to better interpret, understand and apply results.