The Analysis of Proteomics by Machine Learning in Separating Idiopathic Pulmonary Fibrosis from Connective Tissue Disease–Interstitial Lung Disease

Machine learning (ML) is a branch of artificial intelligence in which a model or set of rules is derived on the basis of an initial training set. This model is then used to evaluate a new dataset (1). ML has made important contributions to the analysis of high-resolution computed tomography scans in predicting progression of idiopathic pulmonary fibrosis (IPF) (2, 3). The work by Huang and colleagues (pp. 444–454) reported in this issue of the Journal has now extended the role of ML to proteomic analysis in pulmonary fibrosis, to enhance diagnostic ability and increase our understanding of the disease process of interstitial lung disease (ILD) (4). The objective of the present study was to identify proteins that separate and classify patients with connective tissue disease (CTD)–associated ILD from those with IPF.

The study cohort was drawn from four registries—the Pulmonary Fibrosis Foundation Patient Registry at the University of Virginia, at the University of Chicago, and at the University of California, Davis (5), and the United Kingdom RECITAL (Rituximab versus Cyclophosphamide in Connective Tissue Disease–ILD) clinical trial (6)—providing both patients with IPF (n = 1,247) and those with CTD–ILD (n = 352), with matched proteomic and clinical data. Olink (Proteomics), an unsupervised proteomics platform, had an output 2,912 proteins between CTD–ILD and IPF. The model was derived from the Pulmonary Fibrosis Foundation as the training cohort. For the appropriate downstream analysis, patient numbers and gender for included diseases such as IPF, rheumatoid arthritis (RA)–associated ILD, and scleroderma-associated ILD had to be balanced against each other. This was done using a technique called random subsampling, in which subjects from the cohorts are randomly allocated but balanced for diagnosis and gender.

Once balancing was complete, the authors developed their classifier, their model of the proteomic signature that differentiates CTD–ILD from IPF. They used recursive feature elimination (RFE). In this selection method, the weakest features are progressively removed in an iterative process until a specific number of strong(er) features is reached (7). In addition, RFE removes multicollinearity when two presumed independent variables (proteins) correlate with each other. RFE ranked 37 proteins as a single classifier between CTD–ILD and IPF.

The 37-protein classifier was then subjected to several ML techniques, including support vector machine, which helps solves binary problems, in this case the proteomic separation of CTD–ILD versus IPF; the least absolute shrinkage selection operator, which is a regression analysis technique that was focused on specific variables (proteins) to establish the diagnosis; and random forest (RF), in which predictions from different decision trees are merged. The “imbalanced” RF is used when the cohorts have different numbers. These ML modalities in the University of Virginia and Chicago classification cohorts and the RECITAL/University of California, Davis, prediction cohorts had accuracy values of 83% and 77% with the support vector machine, 80% and 83% with the least absolute shrinkage selection operator, 83.1% and 82.5% with RF, and 77.8% and 80.7% with imbalance RF, respectively, in separating CTD–ILD from IPF (Figure 1). The authors concluded that “multiple machine learning models trained with large cohort proteomic datasets consistently distinguished CTD-ILD from IPF.”

Figure 1.

(A) The schematic of the study. The PFF cohort constituted the training set, which underwent random subsampling and recursive feature elimination to obtain a 37-protein signature. These 37 proteins were subjected to ML analysis from the UVA/Chicago classification cohorts and the RECITAL (Rituximab versus Cyclophosphamide in Connective Tissue Disease–ILD)/UC-Davis prediction cohorts. (B) Proportion of patients with different composite diagnosis scores (CDSs). Patients with CDSs of 0 or 1 were classified as CTD–ILD, those with CDSs of 2 were unclassified, and those with CDSs of 3 or 4 were classified as IPF. CTD = connective tissue disease; ILD = interstitial lung disease; IPF = idiopathic pulmonary fibrosis; LASSO = least absolute shrinkage selection operator; PFF = Pulmonary Fibrosis Foundation; RF = random forest; UC-Davis = University of California, Davis; UVA = University of Virginia.

The diagnosis of CTD is based on guidelines from the American College of Rheumatology for different autoimmune conditions on the basis of clinical features, organ involvement, and serology (8). The pooled prevalence of ILD was 11% for RA, 47% for systemic sclerosis, 41% and 56% for mixed CTD (39–72%), and 6% for systemic lupus erythematosus (3–10%). Usual interstitial pneumonia was the most prevalent ILD pattern for RA (pooled prevalence of 46%), while nonspecific interstitial pneumonia was the most common ILD pattern for all other CTD subtypes (pooled prevalence range, 27–76%) (9).

In contrast, the diagnosis of IPF is based on American Thoracic Society/European Respiratory Society/Japanese Respiratory Society guidelines, with subpleural, bibasal honeycombing comprising the radiological (computed tomography) pulmonary fibrosis pattern of usual interstitial pneumonia (10). The significant difference in the radiology and histology between CTD–ILD and IPF is reflected in the proteomics of respective blood samples.

Although the authors did not specifically aim to find a diagnostic proteomic test to classify CTD–ILD and IPF, the clinical need for a diagnostic proteomic test is limited. Indeed, the lack of a confident diagnosis in 30% of ILD cases and the reclassification in 10% of ILD cases in multidisciplinary discussion is unlikely to pertain to CTD–ILD versus IPF, as the clinical, serological, and radiological features are different between the two conditions. However, there is a fibrosing phenotype within CTD–ILD that may have clinical and radiological overlap with IPF. The prevalence of the fibrosing phenotype of ILD in RA lung disease is 40%, in systemic sclerosis 32%, and in mixed CTD–ILD 24% (11–13). The authors do not describe whether the proteomics study was performed in a fibrosing CTD–ILD cohort. This would be informative, as the proteomic analysis would have greater relevance for separating fibrosing CTD–ILD from IPF. This is helpful for diagnosis when fibrosing CTD–ILD occurs before the onset of autoantibody positivity and so may be (mis)classified as IPF. This scenario can lead the multidisciplinary discussion participants to reclassify IPF as fibrosing CTD–ILD when the full clinical and serology picture unveils itself. Furthermore, proteomic comparisons between fibrosing CTD–ILD and IPF might also delineate important differences in pathogenic pathways in the respective conditions.

The proteomic patterns contrasting CTD–ILD and IPF demonstrate a predominantly inflammatory profile in CTD–ILD, in contrast to proteins associated with epithelial injury, fibrosis, and extracellular matrix turnover in IPF. This result is in line with the pathogenesis of both CTD–ILD and IPF, respectively. There are also “counterintuitive” proteins and pathways that have not been previously associated with these conditions (14, 15). Such an example in this paper is the pathway associated with response to silica dioxide. Further work is required to determine if these proteins have been “falsely” ranked by the proteomic, statistical, or ML methods or whether these common novel pathways worth pursuing. Conundrums such as this will continue to pose a challenge when using ML and bioinformatics, so increasingly sophisticated approaches will be required to gain further granularity about protein ranking.

A further consideration for this study is the acceptable threshold for proteomic separation between CTD–ILD and IPF. The authors propose “PC37,” the number of proteins used in their training set to separate CTD–ILD from IPF. How do we address the separation if <37 proteins are detected in a particular patient? Would we then reject a diagnosis of CTD–ILD versus IPF? Perhaps this could be addressed in practice by analyzing proteomic outputs in the setting of clinical features to inform decisions.

Given that omics platforms will be used increasingly with ML, a major hurdle will be the cost-effectiveness of using these expensive assays and sophisticated ML to make a diagnosis. Hopefully, this will not turn out to be a major concern as these technologies become more mainstream and cheaper. Proteomics and ML might find new relationships, pathogenic/diagnostic pathways, and clusters that will change the diagnosis and management of patients in the future. This brave new world should be embraced.