Extensions of Heterogeneity in Integration and Prediction (HIP) With R Shiny Application

ABSTRACT

Multiple data views measured on the same set of participants are becoming more common and have the potential to deepen our understanding of many complex diseases by analyzing these different views simultaneously. Equally important, many of these complex diseases show evidence of subgroup heterogeneity (e.g., by sex or race). HIP (Heterogeneity in Integration and Prediction) is among the first methods proposed to integrate multiple data views while also accounting for subgroup heterogeneity to identify common and subgroup‐specific markers of a particular disease. However, HIP is applicable to continuous outcomes and requires programming expertise by the user. Here we propose extensions to HIP that accommodate multi‐class, Poisson, and Zero‐Inflated Poisson outcomes while retaining the benefits of HIP. Additionally, we introduce an R Shiny application, accessible on shinyapps.io at https://multi‐viewlearn.shinyapps.io/HIP_ShinyApp/, that provides an interface with the Python implementation of HIP to allow more researchers to use the method anywhere and on any device. We applied HIP to identify genes and proteins common and specific to males and females that are associated with exacerbation frequency. Although some of the identified genes and proteins show evidence of a relationship with chronic obstructive pulmonary disease (COPD) in existing literature, others may be candidates for future research investigating their relationship with COPD. We demonstrate the use of the Shiny application with publicly available data. An R‐package for HIP is available at https://github.com/lasandrall/HIP.

Abbreviations

COPD

Chronic Obstructive Pulmonary Disease

HIP

Heterogeneity in Integration and Prediction

1. Introduction

Chronic obstructive pulmonary disease (COPD) is a chronic disease affecting the lungs and airways in almost 4% of the global population in 2017 [1]. Cigarette smoking is a known risk factor for COPD, but fewer than 50% of heavy smokers develop COPD [2]. There are also many genetic and environmental factors influencing risk [3, 4, 5, 6]. COPD research is further complicated by the subgroup heterogeneity that exists between males and females. In a meta‐analysis, female smokers had a faster annual decline in forced expiratory volume in 1 s (FEV) even if they smoked fewer cigarettes [7]. Another study found that women smokers generally had higher airway wall thickness (AWT) compared to male smokers [8]. Additionally, researchers found that women experienced an increased risk of hospitalization for COPD compared to males even when controlling for smoking [9].

The Genetic Epidemiology of COPD (COPDGene) Study [10] was designed to understand genetic factors related to the development of COPD. The study collected genetic and proteomic data on a subset of participants at the Phase 2 (P2) study visit. Given the availability of multi‐view data (e.g., genomic and proteomic data) and the known subgroup heterogeneity between males and females, the application of an integrative analysis method accounting for subgroup heterogeneity to the COPDGene Study data offers the opportunity to gain new insights into COPD. Butts et al. [11] recently proposed a method called HIP, short for Heterogeneity in Integration and Prediction, for integrating data from multiple sources and simultaneously predicting a continuous outcome while accounting for subgroup heterogeneity. HIP allows to identification of common and subgroup‐specific variables contributing most to the overall association among the views and the variation in the outcome. HIP was used to investigate airway wall thickness (AWT) as a proxy for COPD severity and the authors demonstrated that HIP was capable of identifying genes and proteins common and specific to males and females that were predictive of AWT. However, AWT is not the only way to characterize the effects of COPD, and one may be interested in different types of outcomes that are not continuous.

One such outcome of interest is the number of COPD exacerbations, generally defined as an acute worsening of symptoms that require a change in treatment; these symptoms can include cough, wheezing, dyspnea, chest tightness, and decreased exercise tolerance. While exacerbations vary in severity, severe cases may need to be hospitalized and put on a ventilator; patients who require ICU treatment have a 43%–46% risk of death within a year of the hospitalization [12]. As such the number of exacerbations experienced by patients is a clinically meaningful outcome. Additionally, the TORCH (Towards a Revolution in COPD Health) study found that females had a rate of exacerbations that was 25% higher than males during their 3‐year follow‐up [13]. This emphasizes not only the importance of looking at exacerbation frequency but also accounting for sex differences. Because exacerbation frequency is a Poisson or rate variable, it is not compatible with the originally proposed HIP [11].

Our goal is to deepen our investigation into the molecular underpinnings of sex differences in COPD mechanisms using data from the COPDGene Study [10] by identifying genes and proteins common and specific to males and females related to exacerbation frequency. For this analysis, we included COPDGene Study [10] participants with COPD (defined as GOLD stage $\ge 1$) at P2 and with genomic, proteomic, and selected clinical covariates (age, BMI, race, pack‐years, FEV%, AWT, and % emphysema) available. Figure 1 shows the distributions of many exacerbations in the past year for males and females in this subsample. First, we note a statistically significant difference in the number of events per person‐year between males and females. Second, both males and females had several participants with zero exacerbations suggesting that a zero‐inflated Poisson (ZIP) outcome may better fit this data.

FIGURE 1.

Distribution of exacerbation frequency for males and females at COPDGene Study visit P2. The figure includes data from the P2 visit of the COPDGene Study for the subset of subjects included in our analysis. There is a statistically significant difference in the number of exacerbations experienced per person‐year between males and females.

To address our research goal, the simplest approach we could consider would be to use a penalized regression method such as the Elastic Net [14] or Lasso [15]. The glmnet package [16] can implement the Lasso and Elastic Net on a binary or Poisson outcome, but these methods neither perform integrative analysis nor account for subgroup heterogeneity; they also cannot accommodate a ZIP outcome. These approaches would require concatenating data views as they can only accept a single data view; they would also require running separate analyses for each subgroup to allow for subgroup heterogeneity. These approaches are easily implemented but are not ideal for our research goal where we want to integrate data from multiple sources and associate these data with a ZIP outcome while accounting for subgroup heterogeneity.

There are several integrative analysis methods we could consider for this analysis, but to the best of our knowledge, none of them account for subgroup heterogeneity, and few can accommodate a ZIP outcome. For example, Canonical Variate Regression (CVR) [17] is a one‐step method for simultaneously associating data from multiple sources and predicting a clinical outcome. CVR can accommodate continuous, binary or Poisson outcomes but not a ZIP outcome. Sparse Integrative Discriminant Analysis (SIDA) [18] is a one‐step method for joint association and classification of data from multiple sources, but it is only applicable to classification problems (i.e., binary or multi‐class outcome). The aforementioned methods are one‐step in that the problem of associating multiple views is coupled with the problem of predicting an outcome. We could use a two‐step method to first model the associations between views and then model the Poisson or ZIP response using information from the first step. For example, we could perform canonical correlation analysis (CCA) using SELP (Sparse Estimation through Linear Programming) [19] and then use the canonical variates as predictors in a Poisson or ZIP regression model. However, these all still fail to account for subgroup heterogeneity. To use any of these integrative analysis methods, we would have to either (a) concatenate the subgroups which ignores any potential subgroup heterogeneity or (b) run a separate analysis for each subgroup which limits power, especially in a high‐dimensional data setting where the sample size is typically less than the number of variables.

Alternatively, we could consider methods that account for subgroup heterogeneity, but to the best of our knowledge, none of them perform integrative analysis. One example is the Joint Lasso [20], but this method is unable to accommodate a binary, Poisson, or ZIP outcome. More generally, methods that account for subgroup heterogeneity but do not perform integrative analysis would require either (a) concatenating data views within each subgroup which fails to model the associations between data views or (b) considering each view separately which fails to fully utilize the multi‐view data and requires combining results post hoc.

With all of these existing methods, the only way to get subgroup‐specific information is to fit the model separately on each subgroup, but this can greatly reduce the sample size and thus power to detect effects. Alternatively, if these methods are run on the combined subgroup data, the presence of any potential subgroup heterogeneity will be ignored. Existing methods do not fully utilize the available COPDGene Study [10] data to address our interest in identifying genes and proteins common and specific to males and females related to exacerbation frequency (modeled as a ZIP outcome). Thus, we propose extensions of HIP that can accommodate multi‐class, Poisson, and ZIP outcomes while preserving the existing benefits of HIP (joint association and prediction method for integrative analysis accounting for subgroup heterogeneity and feature ranking) to identify common and subgroup‐specific features predictive of an outcome. Further, since HIP requires Python programming expertise which is limiting, we developed a web application using the R Shiny framework and hosted it on shinyapps.io, allowing HIP to be used anywhere, on any device, and by users with limited programming expertise. We believe that this will increase the widespread adoption of HIP by biomedical researchers interested in integrative analysis methods that also account for subgroup differences.

The remainder of this paper is structured as follows. In Section 2, we present the proposed methods to extend HIP. In Section 3, we describe the algorithmic implementation of this extension. In Section 4, we describe the design and results of simulations assessing the performance of HIP in comparison with existing methods. In Section 5, we apply HIP to data from the COPDGene Study [10] to examine the relationship of genes and proteins with exacerbation frequency. Section 6 introduces an R Shiny [21] application that provides a graphical interface to the underlying Python implementation of HIP. We conclude with a discussion of limitations and future work in Section 7.

2. Methods

2.1. Notation and Problem

Consider the scenario where we have $D$ data views (e.g., genomics, proteomics, clinical) measured on the same set of $N$ subjects. Each view has $p_d$ variables and $S$ subgroups (known a priori). Each subgroup has sample size $n_s$. For our application, $S=2$ for biological sex (males and females). The total number of samples is $N={\sum }_{s=1}^S{n}_s$. The data matrix for subgroup $s$ is denoted by ${\mathit{X}}^{d,s}\in {ℛ}^{n_s\times p_d}$ where rows are for samples and columns are for variables. We define the outcome for each subgroup as ${\mathit{Y}}^s$. For a multi‐class outcome, ${\mathit{Y}}^s\in {ℛ}^{n_s\times m}$ with $m$ being the number of classes, is an indicator matrix where each row has a one in the column corresponding to the class of the observation and a 0 in all other columns. For a Poisson or ZIP outcome, ${\mathit{Y}}^s\in {ℛ}^{n_s\times 2}$ where the first column is the observed counts and the second column is an offset. If no offset is provided, we add a column of ones as the offset which does not affect estimation.

2.2. Existing HIP Framework

Since we extend the original HIP to accommodate different types of outcomes beyond continuous outcome(s), we briefly introduce HIP for completeness' sake. HIP simultaneously associates data from multiple views, predicts an outcome, and ranks or identifies common and subgroup‐specific variables contributing to the overall dependency structure and the variation in an outcome by minimizing an objective function that is a sum of three terms: association term, hierarchical penalty term, and prediction term. Consider the association term. HIP assumes that each view ${\mathit{X}}^{d,s}$ can be approximated by a product of low‐rank view‐independent matrix, ${\mathit{Z}}^s\in {ℛ}^{n_s\times K}$ and view‐specific loadings ${\mathit{B}}^{d,s}\in {ℛ}^{p_d\times K}$. That is, HIP assumes that ${\mathit{X}}^{d,s}={\mathit{Z}}^s{{\mathit{B}}^{d,s}}^T+{\mathit{E}}^{\mathit{d}\mathit{,}\mathit{s}}$ where ${\mathit{Z}}^s$ are latent scores describing association across views and is shared by all views for subgroup $s$, ${\mathit{B}}^{d,s}\in {ℛ}^{p_d\times K}$ are view and subgroup‐specific variable loadings, and ${\mathit{E}}^{d,s}$ are errors as a result of approximating ${\mathit{X}}^{d,s}$ with ${\mathit{Z}}^s{{\mathit{B}}^{d,s}}^T$. Here, $K$ is the number of latent components, which is typically $K\ll p_d$, resulting in data dimensionality reduction. The factor decomposition model is a popular approach to capture dependencies across variables (e.g., Fan et al. [22], Bai J [23], and Chekouo and Safo [24]). It is well known that this model decomposition is only identifiable up to orthonormal rotations. We note that when using latent models for prediction, rotational invariance is irrelevant. In other words, since our focus is not to interpret the regression coefficients of ${\mathit{Z}}^s$ and we do not explicitly order the latent components ${\mathit{Z}}^s$ and the factor loadings ${\mathit{B}}^{d,s}$, we are not concerned with non‐identifiability of this model. Given this latent decomposition, ${\mathit{B}}^{d,s}$ and ${\mathit{Z}}^s$ are estimated to minimize the difference between the observed and estimated data via the loss function:

$$F({\mathit{X}}^{d,s},{\mathit{Z}}^s,{\mathit{B}}^{d,s})=\Vert {\mathit{X}}^{d,s}-{\mathit{Z}}^s{{\mathit{B}}^{d,s}}^T{\Vert }_F^2$$ (1)

where $\Vert ·{\Vert }_F$ is the Frobenius norm. We now consider the hierarchical penalty term which allows for identifying common and subgroup‐specific variables. The hierarchical penalty implemented in HIP is a modified version of the penalty proposed in the Meta Lasso [25]. For common and subgroup‐specific variable selection, HIP decomposes ${\mathit{B}}^{d,s}$ into the element‐wise product of ${\mathit{G}}^d$ and ${\mathbf{\Xi }}^{d,s}$, i.e., ${\mathit{B}}^{d,s}={\mathit{G}}^d·{\mathbf{\Xi }}^{d,s}$. Here, ${\mathit{G}}^d$ allows us to model common effects across subgroups while ${\mathbf{\Xi }}^{d,s}$ allows us to model heterogeneity between subgroups. Since ${\mathit{G}}^d$ is common to all subgroups, its estimation allows us to borrow strength from all subgroups and improves power in estimation, especially in high‐dimensional settings where the number of samples is smaller than the number of variables. Of note, in this reparameterization, exact values of ${\mathit{G}}^d$ and ${\mathbf{\Xi }}^{d,s}$ are not identifiable, but also are not directly needed for variable ranking since variable ranking is based on ${\mathit{B}}^{d,s}$. HIP imposes a block $L_{2,1}$ penalty on both ${\mathit{G}}^d$ and ${\mathbf{\Xi }}^{d,s}$ which encourages the selection of a variable in all $K$ components or none of the components. Specifically, the hierarchical penalty term proposed in HIP is:

$$\sum _{d=1}^D\sum _{s=1}^S𝒥({\mathit{B}}^{d,s})={\lambda }_G\sum _{d=1}^D{\gamma }_d\sum _{l=1}^{p_d}\Vert {\mathit{g}}_l^d{\Vert }_2+{\lambda }_{\xi }\sum _{d=1}^D{\gamma }_d\sum _{s=1}^S\sum _{l=1}^{p_d}\Vert {\mathit{\xi }}_l^{d,s}{\Vert }_2$$ (2)

where ${\mathit{g}}_l^d$ is a vector of length $K$ in the $l$th row of ${\mathit{G}}^d$ and ${\mathit{\xi }}_l^{d,s}$ is a vector of length $K$ in the $l$th row of ${\mathbf{\Xi }}^{d,s}$. The strength of the penalty on ${\mathit{G}}^d$ and ${\mathbf{\Xi }}^{d,s}$ is controlled by hyperparameters ${\lambda }_G>0$ and ${\lambda }_{\xi }>0$ respectively. The ${\gamma }_d$ is a user‐specified indicator of whether to penalize view $d$, i.e., whether to select variables in view $d$. This term is useful to force the inclusion of covariates believed to affect the outcome in the integrative analysis model.

In our previous work, we used the factor regression model approach to relate the outcome with the factors and hence the association term. In particular, we considered the loss function: $F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)=\Vert {\mathit{Y}}^s-{\mathbf{𝒥}}_{n_s}{\beta }_0^T-{\mathit{Z}}^s{\mathbf{\Theta }}^T{\Vert }_F^2$, where $\mathbf{\Theta }\in {ℛ}^{q\times K}$ is a matrix of regression coefficients, ${\mathbf{𝒥}}_{n_s}$ is a vector of ones of size $n_s$ and ${\beta }_0^T$ is a vector of length $q$ for the intercept. $\mathbf{\Theta }$ is common and not subgroup‐dependent. This allows the outcome data for all subgroups to be used in estimating the parameters and can improve the overall prediction of the outcome. Of note, for single continuous outcome, $q=1$, and for multiple continuous outcome, $q>1$. Of note, the constraint ${\mathit{Z}}^{s^T}{\mathit{Z}}^s=\mathit{I}$ could be imposed to allow the columns of ${\mathit{Z}}^s$ to be uncorrelated so that each of the $K$ components provides unique information. However, since our goal is not to interpret the coefficients $\mathbf{\Theta }$ as in regression analysis but to use $\mathit{Y}$ to guide the estimation of the ${\mathit{B}}^{d,s}$ and to rank the variables corresponding to those coefficients, we do not impose this constraint. The presence of ${\mathit{Z}}^s$ in both the prediction and association terms is what allows HIP to be a one‐step method. In other words, HIP simultaneously models the association between multiple views and predicts an outcome, compared to two‐step methods that separate the association and prediction problems which may result in the ${\mathit{Z}}^s$ not being clinically meaningful. Since the two steps are combined in HIP, the estimation of the low‐dimensional view‐independent components ${\mathit{Z}}^s$ is guided by an outcome, and therefore ${\mathit{Z}}^s$ is naturally endowed with prediction capabilities. Put together, HIP solves the following optimization problem to estimate the subgroup‐specific view‐independent components ${\mathit{Z}}^s$, view‐specific common variables ${\mathit{G}}^d$, view‐ and subgroup‐specific variables ${\mathbf{\Xi }}^{d,s}$ and regression coefficients $\mathbf{\Theta }$ that associate multiple views, predicts a continuous outcome(s) and identifies common‐ and subgroup‐specific variables:

$$\begin{array}{ll}\hfill ({\widehat{\mathit{B}}}^{d,s},{\widehat{\mathit{Z}}}^s,\widehat{\mathbf{\Theta }},{\widehat{\beta }}_0)& =\underset{{\mathit{B}}^{d,s},{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0}{\min }\sum _{s=1}^{S}F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)+\sum _{d=1}^D\sum _{s=1}^{S}F({\mathit{X}}^{d,s},{\mathit{Z}}^s,{\mathit{B}}^{d,s})\\ \hfill & +\sum _{d=1}^D\sum _{s=1}^S𝒥({\mathit{B}}^{d,s})\end{array}$$ (3)

Because the prediction term relies on the type of outcome, in the current work, we redefine $F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)$ based on the type of outcome. We define $F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)$ for multi‐class, Poisson, and ZIP outcomes below.

Remark 1

We note that the factor analysis model, which decomposes high‐dimensional data $\mathit{X}$ into $\mathit{X}=\mathit{Z}{\mathit{B}}^T+\mathit{E}$, where $\mathit{Z}$ is a matrix of latent factors (we suppress the subscripts $s$), and $\mathit{E}$ are errors uncorrelated with $\mathit{Z}$, is a popular approach to capture dependencies across variables (e.g., Fan et al. [22], Bai J [23], and Chekouo and Safo [24]). The factor regression model $\mathit{Y}=\mathit{Z}\mathit{\theta }+{\mathit{e}}_y$ is also commonly used in the literature to relate latent factors with a continuous outcome. More recently, Fan et al. (2023) proposed a factor regression model that depended on both the latent factors $\mathit{Z}$ and error term $\mathit{E}$: $\mathit{Y}=\mathit{Z}\mathit{\theta }+\mathit{E}\mathit{\beta }+e_y$. The factor regression model we use is a special case of that proposed by Fan et al. [22]. with $\mathit{\beta }=\mathbf{0}$. The addition of errors in the factor regression model allows the use of additional information beyond the linear space spanned by the predictors (i.e., beyond the latent components).

2.3. Beyond Gaussian Outcome(S): Modification of HIP Prediction Term

2.3.1. Multi‐Class Outcome

Similar to the continuous outcome, we relate the shared component ${\mathit{Z}}^s$ with the multi‐class outcome. For this purpose, we use the cross‐entropy loss function: $F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)=-{\sum }_{i=1}^{n_s}{\sum }_{j=1}^m{y}_{ij}^s\log (a_{ij}^s)$, where $a_{ij}^s=\frac{\exp \{w_{ij}^s\}}{{\sum }_{j=1}^m\exp \{w_{ij}^s\}}$ is the softmax function generalizing logistic regression from binary to multi‐class problems. Here, ${\mathit{W}}^s={\mathbf{𝒥}}_{n_s}{\beta }_0+{\mathit{Z}}^s\mathbf{\Theta }$ represents scores, ${\mathbf{𝒥}}_{n_s}$ is an $n_s\times 1$ matrix of ones, $\mathbf{\Theta }\in {ℛ}^{K\times m}$, and $w_{ij}^s$ is the $ij$th entry in ${\mathit{W}}^s$. The softmax function forces the sum of each row in ${\mathit{W}}^s$ to be 1 so that each entry in the row represents a probability of case $i$ belonging to class $j$.

2.3.2. Poisson Outcome

For a Poisson outcome, we define the prediction term: $F({\mathit{y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)={\sum }_{i=1}^{n_s}-y_i^s[\log (t_i^s)+{\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta }]+t_i^s\exp ({\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta })+\log (y_i^s!)$, which is based on the negative log‐likelihood for a Poisson regression with an offset using a log link. Here, $y_i^s$ is the outcome for the $i$th subject in subgroup $s$, $t_i^s$ is the offset for the $i$th subject in subgroup $s$, and ${\mathit{Z}}_i^s$ is the $i$th row of the ${\mathit{Z}}^s$ matrix. Here $\mathbf{\Theta }\in {ℛ}^{K\times 1}$ and ${\beta }_0\in {ℛ}^{1\times 1}$.

2.3.3. Zero‐Inflated Poisson Outcome

As mentioned in Section 1, the motivating data set has a ZIP outcome rather than a true Poisson outcome. To accommodate this, we add a loss function to the code based on Lambert [26]. In this model, the observed outcome is from the zero state with probability $\tau$ and a Poisson random variable with probability $1-\tau$. We assume that covariates are only related to the Poisson mean ($\lambda$) and not $\tau$. Further, we assume that there is no relationship between $\lambda$ and $\tau$. Thus, $\log ({\lambda }_i)=\log (t_i)+{\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta }$. This assumption is a simplifying assumption, so while we could miss modeling a relationship that is there, it does not enforce or require any specific constraints. Using this distribution, we use the negative log‐likelihood to define the loss function as:

$$\begin{array}{ll}F({\mathit{Y}}^s,{\mathit{Z}}^s,\mathbf{\Theta },{\beta }_0)=& \sum _{y_i^s=0}-\log (\exp [\tau ]+\exp [-t_i^s\exp ({\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta })])\\ \hfill & -\sum _{y_i^s>0}(y_i^s[\log (t_i^s)+{\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta }]+t_i^s\exp [{\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta }])\\ \hfill & +\sum _{i=1}^{n_s}(\log (y_i^s!)+\log [1+\exp (\tau )])\end{array}$$ (4)

This does require estimation of the additional parameter $\tau$ which is described in Section 3. Again $\mathbf{\Theta }\in {ℛ}^{K\times 1}$ and ${\beta }_0\in {ℛ}^{1\times 1}$.

2.4. Prediction

Suppose we have a test data, ${\mathit{X}}_{test}^{d,s}$, $s=1,\dots ,S$, $d=1,\dots ,D$. Our goal in this section is to use the estimated optimization parameters $\widehat{{\mathit{Z}}^s}$, $\widehat{\mathbf{\Theta }}$, and ${\widehat{\beta }}_0$ and the test data ${\mathit{X}}_{test}^{d,s}$ to predict the test outcome ${\widehat{\mathit{Y}}}^s$ for subgroup $s$. We first estimate the test shared component for subgroup $s$, ${\widehat{\mathit{Z}}}_{pred}^s$, by solving the optimization problem:

$$\begin{array}{ll}\hfill {\widehat{\mathit{Z}}}_{pred}^s& =\underset{{\mathit{Z}}^s}{\min }\sum _{d=1}^D\sum _{s=1}^{S}F({\mathit{X}}_{test}^{d,s},{\mathit{Z}}^s,{\widehat{\mathit{B}}}^{d,s})\\ \hfill & =\underset{{\mathit{Z}}^s}{\min }\sum _{d=1}^D\sum _{s=1}^S\Vert {\mathit{X}}_{test}^{d,s}-{\mathit{Z}}^s{\widehat{\mathit{B}}}^{{d,s}^T}{\Vert }_F^2\end{array}$$ (5)

Let ${\mathit{X}}_{cat}^s$ be an $n_s\times \{p_1+\cdots +p_d\}$ matrix that concatenates all $D$ views for subgroup $s$, i.e., $X_{cat}^s=[{\mathit{X}}_{test}^{1,s},\dots ,{\mathit{X}}_{test}^{D,s}]$. Similarly, let ${\widehat{\mathit{B}}}_{cat}=[{\widehat{\mathit{B}}}^{1,s},\dots ,{\widehat{\mathit{B}}}^{D,s}]$ be a $\{p_1+\cdots +p_D\}\times K$ matrix of variable coefficients. Then the solution to the optimization problem (5) is given as: ${\widehat{\mathit{Z}}}_{pred}^s={\mathit{X}}_{cat}^s{\widehat{\mathit{B}}}_{cat}^s{({\widehat{\mathit{B}}}_{cat}^{s^T}{\widehat{\mathit{B}}}_{cat}^s)}^{-1}$ for $s=1,\dots ,S$. Given the predicted ${\widehat{\mathit{Z}}}_{pred}^s$, we predict the test outcome as follows:

$${\widehat{y}}_i^s=\left\{\begin{array}{ll}{\text{arg max}}_j\frac{\exp \{{\widehat{w}}_{ij}^s\}}{{\sum }_{j=1}^m\exp \{{\widehat{w}}_{ij}^s\}}& \text{Multi‐class}\\ t_i^s\exp ({\widehat{\beta }}_0+{\widehat{\mathit{Z}}}_{{pred}_i}^s\widehat{\mathbf{\Theta }})& \text{Poisson}\\ (1-\widehat{\tau })t_i^s\exp ({\widehat{\beta }}_0+{\widehat{\mathit{Z}}}_{{pred}_i}^s\widehat{\mathbf{\Theta }})& \text{ZIP}\end{array}\right.$$

where ${\widehat{\mathit{Z}}}_{{pred}_i}^s$ is the $i$th row of ${\widehat{\mathit{Z}}}_{pred}^s$, ${\widehat{w}}_{ij}^s$ is the $ij$th element of ${\widehat{\mathit{W}}}^s={\mathbf{𝒥}}_{n_s}{\widehat{\beta }}_0+{\widehat{\mathit{Z}}}_{pred}^s\widehat{\mathbf{\Theta }}$.

3. Algorithm

3.1. Optimizing Parameters and Variable Ranking

We use an alternating minimization algorithm to solve the optimization problem (3). We initialize the entries of ${{\mathit{Z}}^s}^{(0)}$ by randomly sampling from a $U(0.9,1.1)$ distribution. We initialize the entries of ${\mathit{G}}^{d^{(0)}}$ for $d=1,\dots ,D$, ${\mathbf{\Theta }}^{(0)}$, and ${\beta }_0^{(0)}$ with ones. We initialize ${\mathbf{\Xi }}^{{d,s}^{(0)}}$ as ${\mathbf{\Xi }}^{{d,s}^{(0)}}={[{({{\mathit{Z}}^{s^{(0)}}}^T{\mathit{Z}}^{s^{(0)}})}^{-1}{{\mathit{Z}}^{s^{(0)}}}^T{\mathit{X}}_{train}^{d,s}]}^T$.

We estimate ${\widehat{{\mathit{Z}}^s}}^{(t)}$ at iteration $t$ by optimizing the following problem using gradient descent with gradients calculated using PyTorch [27]:

$$\begin{array}{ll}\hfill {\widehat{{\mathit{Z}}^s}}^{(t)}& =\underset{{\mathit{Z}}^s}{\min }\sum _{s=1}^{S}F({\mathit{Y}}^s,{\mathit{Z}}^s,{\mathbf{\Theta }}^{(t-1)},{\beta }_0^{(t-1)})+\sum _{d=1}^D\sum _{s=1}^S\Vert {\mathit{X}}^{d,s}-{\mathit{Z}}^s{\mathit{B}}^{{{d,s}^{(t-1)}}^T}{\Vert }_F^2\end{array}$$ (6)

where $F({\mathit{Y}}^s,{\mathit{Z}}^s,{\mathbf{\Theta }}^{(t-1)},{\beta }_0^{(t-1)})$ depends on the type of outcome. We use FISTA (fast iterative shrinkage‐thresholding algorithm) with backtracking [28] to speed up convergence and select an appropriate step size. For a ZIP outcome, further consideration has to be given to the additional parameter, $\tau$. The parameter $\tau$ is the probability that a given observation is in the zero state. Note that this differs from $P(y_i^s=0)=\tau +(1-\tau )e^{-{\lambda }_i}$ as $P(y_i^s=0)$ includes the probability that is in the zero state plus the probability of a zero from a Poisson distribution with mean ${\lambda }_i$. We initialize $\tau$ using the observed proportion of excess 0s beyond the proportion predicted by the Poisson model across all observations [26] shown in Equation (7). Specifically, the first term is the observed proportion of zeros, and the second term is the $P(y_i^s=0|{\lambda }_i={\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta })$ averaged across all observations. In the sub‐optimizations for ${\mathit{Z}}^s$ and $\mathbf{\Theta }$/${\beta }_0$, $\tau$ is treated as a fixed value. Once all other model estimates have been updated, $\tau$ is recalculated with the current model estimates of ${\mathit{Z}}^s$, $\mathbf{\Theta }$, and ${\beta }_0$ as:

$${\widehat{p}}_0=\frac{{\sum }_{s=1}^S{\sum }_{i=1}^{n_s}I(y_i^s=0)-{\sum }_{s=1}^S{\sum }_{i=1}^{n_s}\exp [-\exp ({\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta })]}{{\sum }_{s=1}^S{n}_s}$$ (7)

To estimate ${\mathit{B}}^{{d,s}^{(t)}}$, we first estimate ${\mathit{G}}^{d^{(t)}}$ for each of the $D$ data views. For a fixed ${\mathbf{\Xi }}^{{d,s}^{(t-1)}}$, we solve the following optimization problem using the Adagrad [29] optimizer in PyTorch [27]:

$$\begin{array}{ll}\hfill {\widehat{{\mathit{G}}^d}}^{(t)}& =\underset{{\mathit{G}}^d\in R^{p_d\times K}}{\min }\sum _{s=1}^S\Vert ({\mathit{X}}^{d,s}-{\mathit{Z}}^{s^{(t)}}){({{\mathit{G}}^d·{\mathbf{\Xi }}^{d,s}}^{(t)})}^T{\Vert }_F^2+{\lambda }_G{\gamma }_d\sum _{l=1}^{p_d}\Vert {\mathit{g}}_l^d{\Vert }_2\end{array}$$ (8)

Convergence is defined as the relative change in Equation (8) evaluated at ${\widehat{\mathit{G}}}^{d^{(t)}}$ and ${\widehat{\mathit{G}}}^{d^{(t-1)}}$. We then use these updated estimates for ${\mathit{Z}}^s$ and ${\mathit{G}}^d$ to estimate ${{\mathbf{\Xi }}^{d,s}}^{(t)}$ by solving the optimization problem:

$$\begin{array}{ll}\hfill {\widehat{\mathbf{\Xi }}}^{{d,s}^{(t+1)}}& =\underset{{\mathbf{\Xi }}^{d,s}\in R^{p_d\times K}}{\min }\Vert ({\mathit{X}}^{d,s}-{\mathit{Z}}^{s^{(t)}}{({{\mathit{G}}^{\mathit{d}}}^{(t)}·{\mathbf{\Xi }}^{d,s})}^T){\Vert }_F^2+{\lambda }_{\xi }{\gamma }_d\sum _{l=1}^{p_d}\Vert {\mathit{\xi }}_l^{d,s}{\Vert }_2\end{array}$$ (9)

We use the same technique described for the optimization of ${\mathit{G}}^d$ with an analogous convergence criterion defined as the relative change in Equation (9) evaluated at ${\widehat{\mathbf{\Xi }}}^{{d,s}^{(t)}}$ and ${\widehat{\mathbf{\Xi }}}^{{d,s}^{(t-1)}}$.

We note that because we use an automatic differentiation algorithm, the $L_{2,1}$ (or block $l_2/l_1$) penalty does not result in zero coefficients. However, the magnitude of the coefficients in ${\widehat{\mathit{B}}}^{d,s}$ for the noise variables are clearly smaller than the coefficients of the signal variables. Thus, we rank and identify relevant variables based on the magnitude of the $L_2$ norm of the rows in ${\widehat{\mathit{B}}}^{d,s}$. In implementing the ranking procedure, the user specifies the number of variables (denoted as $N_{top}$) they wish to keep, which could vary across data views. We run Algorithm 1 on the full training data and select the $N_{top}$ variables for each view and subgroup based on the estimated ${\mathit{B}}^{d,s}$. Then, we run Algorithm 1 a second time but with the selected variables. The estimated parameters based on this “subset” of data are used in the prediction procedure described in Section 2.4.

ALGORITHM 1. Overview of optimization algorithm.

We estimate $\mathbf{\Theta }$ and ${\beta }_0$ using ${\widehat{{\mathit{Z}}^s}}^{(t)}$ to optimize the equation problem:

$$({\widehat{\mathbf{\Theta }}}^{(t)},{\widehat{\beta }}_0^{(t)})=\underset{\mathbf{\Theta },{\beta }_0}{\min }\sum _{s=1}^{S}F({\mathit{Y}}^s,{\mathit{Z}}^{s^{(t)}},\mathbf{\Theta },{\beta }_0)$$

We use ISTA (iterative shrinkage‐thresholding algorithm) with backtracking [28] to select an appropriate step size. The convergence criterion is the relative change in Equation (3.1) evaluated at ${\widehat{\mathbf{\Theta }}}^{(t)},{\widehat{\beta }}_0^{(t)}$ and ${\widehat{\mathbf{\Theta }}}^{(t-1)},{\widehat{\beta }}_0^{(t-1)}$.

3.2. Tuning Parameters

The optimization problem depends on $\lambda =({\lambda }_G,{\lambda }_{\xi })$ and $K$. We use grid and random [30] searches for selecting $\lambda$, and we follow the automatic or scree plot approaches to selecting $K$ described in Butts et al. [11]. However, we include eBIC (extended Bayesian Information Criterion) [31] as an additional model selection criterion. This criterion modifies the priors used in the traditional BIC (Bayesian Information Criterion) to prevent larger probabilities from being assigned to models with more covariates. If we consider the set of models with $q$ covariates ${ℳ}_q$, then for the $j^{th}$ model ${ℳ}_{qj}\in {ℳ}_q$ with maximum likelihood estimated parameters $\widehat{\theta }({ℳ}_{qj})$, eBIC is defined as

$$\begin{array}{ll}\hfill eBI{C}_{\delta }({ℳ}_{qj})& =-2\log {ℒ}_n(\widehat{\theta }({ℳ}_{qj}))+\nu ({ℳ}_{qj})\log (n)+2\delta \log (\kappa ({ℳ}_q))\end{array}$$

for $0\le \delta \le 1$ where $\nu ({ℳ}_{qj})$ is the number of estimated parameters, $\kappa ({ℳ}_q)$ is the number of models in ${ℳ}_q$, and $n$ is the number of observations. When $\delta =0$, eBIC is equivalent to the standard BIC, and $\delta =1$ ensures consistency when the number of covariates is very large.

The eBIC criterion for HIP is defined in Equation (10). We use $F({\mathit{Y}}^s,{\widehat{\mathit{Z}}}^s,\widehat{\mathbf{\Theta }},\widehat{{\beta }_0})$ in the first term as it is proportional to the log‐likelihood. For the number of parameters $\nu$, we count the number of variables that will be included in the subset model fit for HIP. A variable is included in the subset fit if it is one of the $N_{top}$ variables selected in at least one subgroup, where $N_{top}$ is user‐specified (e.g., top 10% of variables or top 50 variables) and can vary by data view. Thus, we define $\nu (\widehat{\mathit{B}})={\sum }_{d=1}^D{\sum }_{l=1}^{p_d}I({\widehat{\mathit{B}}}_l^{d,s}\in N_{top}\text{for any}s=1,\dots ,S)$; note this value is the same for all subgroups. For the number of models in ${ℳ}_q$, we note that in a given view, we could include as few as $N_{top}$ variables in the case where all subgroups select the same set of variables and as many as $S\ast N_{top}$ variables when all subgroups select distinct sets of variables. Thus, to count the size of ${ℳ}_q$ where $q=N_{top}$, we sum the combinations of choosing each possibility between $N_{top}$ and $S\ast N_{top}$ from the $p_d$ variables in the view. The code will return values for $eBI{C}_0$, $eBI{C}_{0.5}$, and $eBI{C}_1$:

$$\begin{array}{ll}\hfill eBI{C}_{\delta }& =2\sum _{s=1}^{S}F({\mathit{Y}}^s,{\widehat{\mathit{Z}}}^s,\widehat{\mathbf{\Theta }},\widehat{{\beta }_0})+\sum _{s=1}^S\log (n_s)\nu (\widehat{\mathit{B}})+2\delta \sum _{d=1}^D\log \left[\sum _{w=N_{top}}^{S\ast N_{top}}\left(\genfrac{}{}{0ex}{}{p_d}{w}\right)\right]\end{array}$$ (10)

4. Simulations

4.1. Set‐Up

Simulations were run for binary, Poisson, and ZIP outcomes. For all outcomes, there were two views and two subgroups, i.e., $D=S=2$, and there were $n_1=250$ subjects belonging to the first subgroup and $n_2=260$ subjects belonging to the second subgroup. We considered two different scenarios for the degree of subgroup heterogeneity: Full Overlap and Partial Overlap. In both scenarios, there are 50 variables important to each subgroup for each view. In the Full Overlap scenario, the 50 important variables are the same for both subgroups; in the Partial Overlap scenario, 25 of the important variables are common to both subgroups and 25 are unique to each subgroup. For each of the outcomes and scenarios, there were two different sets of variable dimensions in the data views. In the low dimensional setting, the first data set had $p_1=300$ variables, and the second had $p_2=350$. In the high‐dimensional setting, the first data set had $p_1=2000$ variables, and the second had $p_2=3000$ variables. We also considered simulation settings where (i) each variable for each view was binary (i.e., non‐Gaussian) and (ii) the sample sizes for the two groups were unbalanced, to investigate the robustness of the proposed method. Please refer to the Supporting Information for simulation results for these scenarios.

4.2. Data Generation

The data were generated following a process based on Luo et al. [17]. Entries in rows of ${\mathit{B}}^{d,s}$ corresponding to a signal variable were drawn from a $U(-1,-0.5)\cup U(0.5,1)$; otherwise, the entry was set to 0. The columns of each ${\mathit{B}}^{d,s}$ were then orthonormalized using a QR decomposition. The entries of ${\mathit{Z}}^s$ are drawn from $N(\mu =25.0,\sigma =3.0)$ and entries of ${\mathit{E}}^{d,s}$ from $N(\mu =0.0,\sigma =1.0)$. Then the covariate matrices ${\mathit{X}}^{d,s}$ are formed as ${\mathit{Z}}^s{\mathit{B}}^{{d,s}^T}+{\mathit{E}}^{d,s}$.

At this point, the processes diverge somewhat for the different outcomes. For the binary outcome, we apply the softmax function to ${\mathit{W}}^s={𝒥}_{n_s}{\beta }_0+{\mathit{Z}}^s\mathbf{\Theta }+{\mathit{E}}_y^s$ where ${𝒥}_{n_s}$ is an $n_s\times 1$ matrix of ones, $E_y^s$ is an $n_s\times 2$ matrix of standard normal errors and assign the class with the largest probability for each observation. Here ${\beta }_0=\left[\begin{array}{ll}0.5& 0.5\end{array}\right]$ and $\mathbf{\Theta }=\left[\begin{array}{ll}1.0& 0.5\\ 0.2& 0.8\end{array}\right]$. For the Poisson and ZIP outcomes, we first standardize the columns of ${\mathit{Z}}^s$ to have mean 0 and variance 1. The observations are then generated from the Pytorch [27] Poisson random variable generator with mean $\exp ({𝒥}_{n_s}{\beta }_0+{\mathit{Z}}^s\mathbf{\Theta })$, i.e., observation $y_i^s\sim Poisson(\exp ({\beta }_0+{\mathit{Z}}_i^s\mathbf{\Theta }))$. Here ${\beta }_0=2.0$ and $\mathbf{\Theta }={\left[\begin{array}{ll}0.7& 0.2\end{array}\right]}^T$. For the ZIP outcome, each observation is then multiplied by a draw from a Bernoulli distribution that is 0 with probability $\tau =0.25$.

4.3. Comparison Methods

For all outcomes under consideration (binary, Poisson, and ZIP), we are unaware of any other methods that perform integrative analysis while also accounting for subgroup heterogeneity. For the binary outcome, we compare HIP to two integrative analysis methods: CVR [17] as implemented in the R package CVR [32] and SIDA [18] as implemented in the R package mvlearnR [33]. Because neither of these methods accounts for subgroup heterogeneity, we implement each method in two ways: (1) all subgroups concatenated within each view (Concatenated) and (2) a separate model for each subgroup (Subgroup). We also compare HIP to the Lasso [15] and the Elastic Net [14] as implemented in the R package glmnet [16]. Neither of these two methods performs integrative analysis, so the two views are concatenated for these methods. We again implement two ways: (1) concatenating the subgroups (Concatenated) and (2) separate models for each subgroup (Subgroup).

For the Poisson outcome, SIDA is no longer applicable, so we instead compare HIP to the two‐step integrative analysis method SELPCCA [19] as implemented in the R package mvlearnR [33]. Since SELPCCA does not account for subgroup heterogeneity, we again fit concatenated and subgroup models (Concatenated SELPCCA and Subgroup SELPCCA, respectively).

For the ZIP outcome, CVR, Lasso, and Elastic Net cannot explicitly account for a ZIP outcome, but we still fit these models using a Poisson family. We also fit HIP specifying a Poisson outcome [HIP (Grid)‐Poisson and HIP (Random)‐Poisson] to demonstrate the importance of accounting for the zero‐inflated nature of the data. Finally, because SELPCCA is a two‐step method, we use the canonical variables from SELPCCA in a ZIP regression model fit with the zeroinfl function [34] in R package pscl [35]. We again fit a model on the concatenated subgroups (Concatenated SELPCCA‐ZIP) and separate models for each subgroup (Subgroup SELPCCA‐ZIP).

Tuning parameters for the comparison methods were selected using 10‐fold cross‐validation. For the Elastic Net, we set $\alpha =0.5$. Tuning parameters for HIP were selected using $eBI{C}_1$; we considered a range of $(0,2]$ for ${\lambda }_{\xi }$ and ${\lambda }_G$ with 8 steps for each. The true values for $K$ (i.e., 2) and $N_{top}$ (i.e., 50) were used for all simulations.

4.4. Evaluation Measures

We compare HIP to the existing methods in terms of variable selection and prediction ability for new data. For variable selection, we will estimate the true positive rate ($\text{TPR}=\frac{\text{True Positives}}{\text{True Positives + False Negatives}}$), false positive rate ($\text{FPR}=\frac{\text{False Positives}}{\text{True Negatives + False Positives}}$), and F1 score ($\text{F1}=\frac{\text{True Positives}}{\text{True Positives}+\frac{1}{2}\text{(False Positives + False Negatives)}}$) which are all constrained to be between 0 and 1. Ideally, TPR and F1 are 1 and FPR is 0.

To compare predictive ability for the binary outcome, we look at classification accuracy, and for both Poisson and ZIP outcomes, we look at the fraction of deviance explained, $D^2=\frac{D_{null}-D_{opt}}{D_{null}}$, where $D_{null}$ is the deviance of the null model and $D_{opt}$ is the deviance of the model with optimal tuning parameters [36]. Results are averaged over 20 Monte Carlo data sets.

4.5. Results

We focus on the results for the ZIP outcome here as the motivating data have a ZIP outcome (Figures 2 and 3); results for the ZIP outcome with Full Overlap scenario as well as binary and Poisson outcomes are presented in the Supporting Information: Figures S1–S8.

FIGURE 2.

Results for ZIP outcome, full overlap scenario. The first row corresponds to the low dimension ($p_1$ = 300, $p_2$ = 350) and the second to the high dimension ($p_1$ = 2 000, $p_2$ = 3 000). For all settings, $n_1=250$ and $n_2=260$. The right column is the fraction of deviance explained ($D^2$), so a higher value indicates better performance. Results are mean $\pm$ one standard deviation summarized across 20 generated data sets.

FIGURE 3.

Performance results for ZIP outcome, partial overlap scenario. The first row corresponds to the low dimension ($p_1$ = 300, $p_2$ = 350) and the second to the high dimension ($p_1$ = 2 000, $p_2$ = 3 000). For all settings, $n_1=250$ and $n_2=260$. The right column is the fraction of deviance explained ($D^2$), so a higher value indicates better performance. Results are mean $\pm$ one standard deviation summarized across 20 generated data sets.

First, we note the performance of HIP (Grid) and HIP (Random) are very similar in both the low‐ and high‐dimensional settings for both the Full and Partial Overlap scenarios (Figures 2 and 3, respectively), so we recommend the use of HIP (Random) as it is computationally faster. In terms of variable selection, HIP (Grid) and HIP (Random) show the highest TPR and F1 values; all methods show low FPRs. The ZIP model only improves variable selection slightly over the HIP Poisson fits. The other two integrative methods, CVR and SELPCCA‐ZIP, show similar variable selection performance to each other and show a benefit over the non‐integrative methods (Elastic Net and Lasso), but they do not perform nearly as well as HIP. All of the comparison methods seem to be missing many of the true signal variables as evidenced by lower TPRs.

In terms of predictive ability in the Full Overlap Scenario, we note that HIP and the SELPCCA‐ZIP models have a much higher fraction of deviance explained than the methods that do not account for the zero‐inflated nature of the data, highlighting the importance of doing so. In the Partial Overlap Scenario, Concatenated SELPCCA‐ZIP has a much lower $D^2$ than in the Full Overlap Scenario suggesting that this method has more difficulty when subgroup heterogeneity exists in the data. HIP has the highest fraction of deviance explained out of all methods applied for both the low‐ and high‐dimensional settings in both the Full and Partial Overlap Scenarios indicating HIP is still favorable even if subgroup heterogeneity does not exist in the data.

Computation times for all methods are summarized for the Full and Partial Overlap Scenarios in Supporting Information: Figures S9 and S10 respectively. The Lasso and Elastic Net are the fastest in all scenarios, but because these methods do not perform integrative analysis or account for subgroup heterogeneity, the variable selection and predictive ability suffer. Of the methods that perform integrative analysis, HIP (Random) is consistently the fastest with larger computational advantages in the high‐dimensional setting particularly when compared to CVR. Concatenated SELPCCA‐ZIP has similar computation times as HIP (Random) in the Partial Overlap setting, but this is the setting where the performance of Concatenated SELPCCA‐ZIP was reduced dramatically.

5. Application to Exacerbation Frequency

5.1. Goals

In this section, our goal is to use the genetic and proteomic data from the COPDGene Study [10] in combination with clinical data to gain new insights into the molecular architecture of COPD in males and females. To be included in the analyses, participants had to have COPD at P2 (defined as GOLD stage $\ge 1$) and have proteomic, genomic, and selected clinical covariates (age, BMI, race, pack‐years, FEV%, AWT, and % emphysema) available. There were $N=1374$ participants meeting these criteria with $n_1=780$ males and $n_2=594$ females; demographic characteristics of this sample are in Table 1. Continuous variables were compared between males and females using t‐tests and categorical variables using ${\chi }^2$ tests. Participants were predominantly non‐Hispanic white; there were no statistically significant sex differences in age, BMI, BODE index, percentage of current smokers, or percentage with diabetes. Males and females differed in their exacerbation frequency (p $<0.001$) but did not have significantly different lung function as measured by mean FEV% predicted. Given the available data and the sex differences in exacerbation frequency, we first identify genes and proteins common and specific to males and females associated with exacerbation frequency. Using those identified genes and proteins, we then explore pathways enriched for males and females.

TABLE 1.

COPDGene participant characteristics.

Variable	Males	Females	p
	N = 782	N = 594
Age	68.28 (8.35)	68.03 (8.36)	0.581
BMI	28.03 (5.62)	27.69 (6.59)	0.317
FEV1% Predicted	61.94 (22.97)	62.91 (22.59)	0.431
BODE index	2.45 (2.45)	2.63 (2.38)	0.176
% Emphysema	11.30 (11.86)	9.39 (11.45)	0.003
Pack years	53.05 (26.63)	47.57 (24.99)	$<$ 0.001
Airway wall thickness	1.17 (0.23)	1.00 (0.21)	$<$ 0.001
Non‐Hispanic White (%)	82	78	0.084
Current Smoker (%)	66	65	0.510
Diabetes (%)	17	14	0.204

Note: The measurements presented were collected at the Year 5 study visit to align with the collection of proteomic and genomic data collection.

Abbreviations: BMI = Body Mass Index; BODE = Body mass index, airflow Obstruction, Dyspnea, and Exercise capacity; COPD = Chronic Obstructive Pulmonary Disease; ${\text{FEV}}_1$ = Forced Expiratory Volume in 1 s.

5.2. Applying HIP and Existing Methods

The COPDGene data includes 19263 genes and 4979 proteins. We first performed unsupervised filtering to select the 5 000 genes and 2 000 proteins with the largest variances. To preserve as much generalizability as possible, we then randomly split the data 50 times into train (75%) and test (25%) data sets keeping the proportions of males and females the same. Genes and proteins that were consistently selected across these splits were considered “stable” as these would be most likely to show consistent findings. Within each split, we performed supervised filtering using the training data by regressing exacerbation frequency on each of the genes and proteins retained after the unsupervised filtering. We used the zeroinfl function [34] where the current gene or protein was the only predictor in the count model and only an intercept for the zero model. Genes and proteins with an uncorrected p‐value $<0.10$ were included in the models. Because this supervised filtering was repeated for each split, a different set of variables could enter the models for each split of the data.

For existing methods, we fit the subgroup implementations described in Section 4.3 for CVR (using a Poisson family), SELPCCA‐ZIP, Lasso (using a Poisson family), and Elastic Net (using a Poisson family). For HIP, we applied HIP (Grid) and HIP (Random) using the ZIP family and the Poisson family [HIP (Grid)‐Poisson and HIP (Random)‐Poisson]. Additionally, we fit HIP using the ZIP family with an additional clinical data view that was not penalized [HIP (Grid)‐ZIP+Clinical and HIP (Random)‐ZIP+Clinical]. We used the training data to select tuning parameters and calculate training $D^2$ and then used the test data to calculate a test $D^2$.

We defined the “stable” genes and proteins by ranking them based on the product of (a) the number of splits in which the variable was included in the $N_{top}$ variables and (b) the number of splits in which the variable was included in the $N_{top}$ variables divided by the number of splits in which the variable was entered into the models. The top 1% of genes and proteins for males and females based on this ranking were identified as the “stable” genes and proteins for each method.

To select tuning parameters, CVR, SELPCCA, Lasso, and Elastic Net used 10‐fold cross‐validation. For HIP, when we used the $\lambda$ range $(0,2]$ used in simulations, the upper bound was consistently being selected, so we increased the range until this was not happening resulting in a range of $(0,15]$ for ${\lambda }_G$ and ${\lambda }_{\xi }$. The best model was selected using $eBI{C}_1$. We also needed to specify $K$, the number of latent components for HIP, and an equivalent parameter for CVR. The automatic approach specified in Butts et al. [11] with a threshold of 0.20 on the concatenated data suggested $K=3$. On the separate ${\mathit{X}}^{d,s}$, it suggested $K=3$ for the genes and $K=1$ for the proteins. Scree plots for the concatenated and separate data are in Supporting Information: Figure S24. We thus selected $K=3$ components for HIP and CVR. For HIP, we set $N_{top}=125$ genes and 50 proteins.

5.3. Results

5.3.1. Fraction of Deviance Explained and Computation Times

Figure 4 shows violin plots of the train and test fraction of deviance explained for the 50 Train/Test data splits. As expected, HIP (using the ZIP model) and SELPCCA‐ZIP, the two methods that account for the zero‐inflated nature of the data, have the best predictive ability. The number of variables selected in the 50 Train/Test splits are summarized in Supporting Information: Table S1. Similarly, Supporting Information: Figure S25 shows a violin plot of the run times for each method. Of the methods performing integrative analysis, HIP (Random) regardless of family tended to have the fastest computation times, although SELPCCA‐ZIP often had similar run times.

FIGURE 4.

Fraction of deviance explained across 50 data splits of the COPDGene data. Violin plots display the distribution of the fraction of deviance explained across the 50 data splits. For each of CVR, SELPCCA‐ZIP, Lasso, and Elastic Net, there is a violin plot for each subgroup as comparator methods require separate models for each subgroup to allow for possible subgroup heterogeneity.

5.3.2. Selected Genes and Proteins

Supporting Information: Table S2 shows the number of “stable” genes and proteins common and specific to males and females identified by each method. Supporting Information: Figure S26 shows the overlap of “stable” genes and proteins selected for males and females by each method. There are few overlaps in the “stable” genes and proteins, but the overlaps that do occur tend to be in the methods that perform integrative analysis (i.e., HIP, CVR, and SELPCCA) suggesting integrative methods may result in more reproducible findings.

HIP (Random) and HIP (Grid) showed strong agreement in the “stable” genes and proteins (Supporting Information: Table S3) which also supports the use of the random search instead of the grid search. The “stable” genes selected by HIP (Random) for males and females and their average estimated weights are in Supporting Information: Tables S4 and S5 respectively. Analogous results for proteins are presented in Supporting Information: Table S6. The weights in these tables are averages of the estimated weights ($L_2$ norm of rows in ${\widehat{\mathit{B}}}^{d,s}$) from the subset fit in the splits where the variable was included in the subset fit (i.e., the variable was in $N_{top}$ for at least one subgroup). Supporting Information: Tables S7–S9 list the “stable” gene IDs and proteins selected by the competing methods.

The top gene for males was activating signal cointegrator 1 complex subunit (ASCC2). Wilson et al. [37] looked for genes that were differentially expressed in COPD patients with and without cachexia (a loss of weight and muscle) and identified ASCC2 in a sample of 400 COPDGene Study participants and replicated the finding in a sample of 114 participants from the ECLIPSE Study; cachexia occurs more frequently in those with more advanced COPD. The top gene for females was dematin actin binding protein (DMTN). Lee et al. [38] measured DNA methylation on 100 participants (60 with and 40 without COPD) from a Korean COPD Cohort and identified DMTN (also known as EPB49) as a differentially methylated region when comparing current smokers to never smokers; the authors also note this gene has been identified in previous epigenome‐wide association studies of smoking. Thus this gene may be a candidate for future research to investigate the relationship with COPD specifically.

The top protein for males was SHC adaptor protein 1 (SHC1) (ranked fifth for females). Li et al. [39] ranked candidate genes based on gene risk scores where a higher rank indicated a stronger relationship to COPD. The SHC1 gene was ranked third out of 200 candidate genes and had lower expression levels in patients with COPD compared to healthy controls. The top protein for females was amyloid beta precursor protein (APP) (ranked second for males). Almansa et al. [40] compared gene expression levels of 12 patients with COPD requiring treatment in the ICU compared to 16 patients with COPD who were admitted to the hospital for treatment but did not require the ICU. They found that the patients admitted to the ICU showed higher expression levels of APP compared to patients who were not admitted to the ICU.

5.3.3. Pathway Analysis

We tested for overrepresentation of pathways in our “stable” proteins and genes for males and females using Ingenuity Pathway Analysis [41]. Table 2 shows the top 10 canonical pathways for genes and proteins for males and females.

TABLE 2.

Top canonical pathways from IPA enrichment analysis.

View	Subgroup	Canonical pathway	Molecules	Unadjusted p
Genes	Males	Iron homeostasis signaling pathway	CDC34, FECH, SLC25A37	0.003
		Methylglyoxal Degradation I	HAGH	0.006
		Heme Biosynthesis from Uroporphyrinogen‐III I	FECH	0.008
		Pentose Phosphate Pathway (Non‐oxidative Branch)	RPIA	0.013
		Heme Biosynthesis II	FECH	0.019
		Pentose Phosphate Pathway	RPIA	0.023
		Erythropoietin Signaling Pathway	BCL2L1, GATA1	0.054
		ID1 Signaling Pathway	BCL2L1, GSPT1	0.068
		Sertoli Cell‐Sertoli Cell Junction Signaling	SPTB, YBX3	0.069
		Autophagy	GABARAPL2, SLC1A5	0.076
	Females	Heme Biosynthesis II	ALAS2, FECH	$<$ 0.001
		Iron homeostasis signaling pathway	ALAS2, CDC34, FECH, SLC25A37	$<$ 0.001
		Methylglyoxal Degradation I	HAGH	0.006
		Heme Biosynthesis from Uroporphyrinogen‐III I	FECH	0.008
		Tetrapyrrole Biosynthesis II	ALAS2	0.010
		Hypoxia Signaling in the Cardiovascular System	CDC34, UBE2H	0.011
		Pentose Phosphate Pathway (Non‐oxidative Branch)	RPIA	0.013
		Pentose Phosphate Pathway	RPIA	0.023
		Erythropoietin Signaling Pathway	BCL2L1, GATA1	0.054
		ID1 Signaling Pathway	BCL2L1, GSPT1	0.068
Proteins	Males	Role of JAK2 in Hormone‐like Cytokine Signaling	EPO, LEP, PTPN6	$<$ 0.001
		White Adipose Tissue Browning Pathway	BDNF, LEP, NPPB	$<$ 0.001
		Erythropoietin Signaling Pathway	EPO, LEP, PTPN6	$<$ 0.001
		Serotonin Receptor Signaling	ADIPOQ, BDNF, LEP, NPPB	$<$ 0.001
		AMPK Signaling	ADIPOQ, INS, LEP	0.001
		Leptin Signaling in Obesity	INS, LEP	0.002
		IL‐3 Signaling	PPP3R1, PTPN6	0.002
		Maturity Onset Diabetes of Young (MODY) Signaling	ADIPOQ, INS	0.002
		Thyroid Cancer Signaling	BDNF, INS	0.002
		ABRA Signaling Pathway	NPPB, PPP3R1	0.003
	Females	Granulocyte Adhesion and Diapedesis	PF4, PPBP, TNFRSF1A	$<$ 0.001
		Agranulocyte Adhesion and Diapedesis	PF4, PPBP, TNFRSF1A	$<$ 0.001
		Wound Healing Signaling Pathway	EGF, PF4, TNFRSF1A	$<$ 0.001
		Huntington's Disease Signaling	BDNF, CPLX2, EGF	0.001
		Pathogen Induced Cytokine Storm Signaling Pathway	PF4, PPBP, TNFRSF1A	0.002
		Glioma Signaling	CDKN2D, EGF	0.004
		Type II Diabetes Mellitus Signaling	ADIPOQ, TNFRSF1A	0.005
		Axonal Guidance Signaling	BDNF, EGF, PAPPA	0.005
		Tumor Microenvironment Pathway	EGF, TNFRSF1A	0.007
		Regulation of the Epithelial‐Mesenchymal Transition by Growth Factors Pathway	EGF, TNFRSF1A	0.008

There were both common and subgroup‐specific gene pathways for males and females. The top gene pathway for males was the CLEAR Signaling Pathway, and the top gene pathway for females was the STAT3 pathway. The STAT3 pathway is known to be involved in inflammatory responses to many diseases including COPD [42]. The Iron homeostasis signaling pathway that was found in the AWT analysis in Butts et al. [11] for both males and females was again present in the top gene pathways for males and females.

There was complete overlap in the top 10 protein pathways for males and females; this makes some sense because of the 20 stable proteins identified for males and females, 17 of them overlapped. For both males and females, the top pathway was the wound healing signaling pathway followed by PDGF signaling. The PDGF family includes PDGFA (platelet‐derived growth factor subunit A) and PDGFB (platelet‐derived growth factor subunit B) and is associated with wound healing; when PDGFs are found outside the context of wound healing, it seems to contribute to many diseases [43]. Though these authors focus on asthma rather than COPD, they also state that PDGF is expressed in airway epithelial cells and PDGFB is expressed in inflamed airway tissue.

6. R Shiny Application

While HIP and its extension to additional outcomes has many research areas to which it could be applied (within and outside of human health), it is implemented in Python which may be a barrier to some researchers without a coding background. We introduce an R Shiny [21] application that provides a graphical interface to apply HIP to data that is either uploaded, simulated, or available within the application. In this section, we provide a brief overview and highlights of the application. A detailed example of using the application is in the Supporting Information. The application is accessible on shinyapps.io at https://multi‐viewlearn.shinyapps.io/HIP_ShinyApp/.

The Shiny application has three tabs. The first tab, “About”, provides a brief overview of the method and related links (Figure 5a). The second tab, “HIP”, is where the user will set‐up the data and parameters for the method. We provide options for the user to upload their data, simulate data (based on our simulation examples), and implement HIP on an example of COVID‐19 data (Figure 5b). The third tab, “Results”, is where the user will submit the analysis and see results. This tab produces outputs of the model including prediction information and variable importance to help the user understand the results.

FIGURE 5.

HIP Shiny application screens.

Once the “Run Analysis” button has been clicked, a progress notification (Supporting Information: Figure S32) will appear in the lower right corner of the screen so the user knows the analysis has started. The user will know the analysis is complete when the progress notification disappears and the run time is displayed to the right of the “Run Analysis” button. After the analysis is complete, the left column in the “Result Summary” section will provide a button to download results, and the right column will display some basic information about the results for the user including the convergence status, the $\lambda$ values used, the $eBI{C}_0$ or other selection criterion value, the run time, and the applicable training prediction metric (Supporting Information: Figure S33). We also show prediction metrics for the training data and the test data if they are available. If there are no test data available, a message will be printed for the user stating so. The prediction metric is the mean squared error for Gaussian outcomes, classification accuracy for multi‐class outcomes, and fraction of deviance explained for Poisson and ZIP outcomes. The appropriate metric and formula are displayed in a gold box in the left column for users.

The weights from the estimated ${\mathit{B}}^{d,s}$ matrices are displayed in two formats. The first is a variable importance plot that includes all variables that were included in the subset model fit (Supporting Information: Figure S35). The variables are ranked based on the weights within each view and subgroup. The second output is an interactive table with columns for the rank (within view and subgroup), variable, weight, view, and subgroup (Supporting Information: Figure S36). This table can be sorted by or filtered on any column so that the user can explore the results. Some possible options that could be of interest include selecting the top‐ranked variable(s) for each view and subgroup, selecting a specific variable to compare between subgroups, or selecting all results for a single view and subgroup (Supporting Information: Figures S37–S39).

In general, user inputs are in gray boxes, generated outputs have a white background, and information for the user is in gold boxes. Please refer to the Supporting Information for detailed descriptions of HIP and an illustration of HIP on publicly available data on COVID‐19.

7. Conclusion

In this paper, we have extended HIP as proposed in Butts et al. [11] to accommodate multi‐class, Poisson, and zero‐inflated Poisson (ZIP) outcomes allowing researchers to investigate additional clinically relevant outcomes in an integrative analysis framework that also accounts for subgroup heterogeneity. We retain the benefits of a joint association and prediction method for data integration to select clinically meaningful subgroup‐specific and common features and the ability to include clinical covariates. In simulations, HIP demonstrated improved variable selection abilities for binary, Poisson, and ZIP outcomes compared to existing methods. While all methods showed similar classification accuracy in the binary outcome simulations, HIP showed small improvements in $D^2$ in the Poisson outcome simulations and substantial improvements in $D^2$ in the ZIP outcome simulations. When applied to data from the COPDGene Study, we were able to identify common and subgroup‐specific genes and proteins associated with exacerbation frequency; previous literature has identified at least some of these as being related to COPD.

The R Shiny Application developed here provides a user‐friendly interface to apply HIP to any data with multiple data views, measured on pre‐specified subgroups, being used to predict an outcome. In the first tab, the application introduces HIP and describes why HIP may be a desirable method to consider for analyzing a multi‐view data set with potential subgroup heterogeneity. In the second tab, the application guides the user through inputting data, selecting an appropriate $K$, and setting parameters for running HIP. Finally, the third tab produces outputs of the model including prediction information and variable importance to help the user understand the results.

There are still some limitations to HIP requiring further research. As in the originally proposed HIP [11], $N_{top}$, the number of variables to keep for each view, must be specified. In simulations, we know the true value of $N_{top}$, but we cannot know the true value in applications to real data. Sensitivity analyses could help investigate the performance when values other than the true value are specified. We investigated using simulations the performance of different $N_{top}$ set below and above the truth. There was a decline in the performance of the variable selection compared to when $N_{top}$ was set to the truth, but the impact of specifying $N_{top}$ less than the true number was smaller than when it was greater than the true number, particularly in the high‐dimensional scenarios. In general, $N_{top}$ had less impact on variable selection in the low‐dimensional scenario when it was not set to the true value. HIP was still competitive, producing better or comparable variable selection and prediction estimates. We recommend that users investigate different cutoff points for their specific applications, while considering the balance between prediction estimates and parsimonious models. The inclusion of clinical covariates in the COPDGene application did not improve the predictive performance of the test data. The reason for this is unclear and could be for multiple reasons. One possibility is the variables in the clinical data do not explain additional variation in the outcome beyond the genes and proteins. Another possibility is that the large difference in the number of variables in this view compared to the other views affects the estimation. As another limitation, we focused on identifying joint or shared latent components among views for each subgroup. However, there might exist view‐ and subgroup‐specific variations in some applications that may not be captured by our method. Future work could consider extensions of HIP that model both joint and view‐specific latent components for each subgroup. The factor regression model proposed by Fan et al. (2023) [22] that uses additional information beyond the linear space spanned by the predictors (i.e., beyond the latent components) is appealing, but its benefits in our model are uncertain. Future studies could explore this further. We have assumed that each of the predictors comes from a Gaussian distribution. However, this assumption might not hold in practice, as real‐world data can have different underlying distributions. As a result, future work could explore alternative loss functions beyond the Frobenius norm to better accommodate various data distributions, potentially improving model robustness and predictive accuracy. Despite these limitations, our extension to HIP allows researchers to explore a wide variety of new questions by considering multi‐class, Poisson, or ZIP outcomes that are clinically meaningful.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting information.

Data Availability Statement

Access to the clinical and genomic data can be requested through dbGaP (IDs: phs000951.v4.p4 and phs000179.v6.p2). The proteomic data can be requested from the COPDGene Study Group (http://www.copdgene.org/). The Python source code and R‐package for implementing the methods and generating simulated data along with README files is available on GitHub at https://github.com/lasandrall/HIP. A Shiny Application of HIP for users with limited programming expertise can be found at https://multi‐viewlearn.shinyapps.io/HIP_ShinyApp/.