Three-part random effect models for longitudinal skewed survey data with “not applicable” responses

Abstract

In survey data some questions are asked only of a subset of applicable participants. This frequently occurs together with floor effects of the provided responses. For example, in the longitudinal Population Assessment of Tobacco and Health (PATH) survey, nicotine dependence is assessed only for a sub-sample of individuals at each occasion and when assessed often has value at the lower end of the scale. To capture trends over time in an unbiased and efficient way, it is important to jointly model the probabilities of being asked the questions of interest, of giving a response at the lower end of the scale and of the mean response when above the lower end of the scale. We propose a three-part model for such data which consists of two logistic sub-models and a truncated normal model. Correlations among repeated observations on the same individual are induced by random effects. Maximum likelihood estimation and inference is performed in SAS PROC NLMIXED. The PATH data on young adults are used for illustration. A simulation study investigates bias and efficiency of the three-part model compared to simpler models. The three-part model has much lower bias and better coverage probabilities for the regression coefficients than simpler models.

1. Introduction

In survey data, some questions are asked only of a subset of participants, that is, participants for whom the questions are considered to be applicable. This frequently occurs together with floor effects of the provided responses; that is, there is a lower limit (floor) on the possible values for the survey question and many participants score at this lower limit of the scale. For example, in the longitudinal Population Assessment of Tobacco and Health (PATH) survey, a nationally representative survey of tobacco use in U.S. ([1]), nicotine dependence to tobacco is assessed only for a subset of individuals at each timepoint (certain type of tobacco users) and it is measured on a scale from 1 to 5, with higher values meaning more dependence. When assessed, nicotine dependence often has value at the lower end of the scale (that is, 1=no dependence) because there is a subset of participants who didn’t develop dependence. That means that the data can be viewed as arising from three parts. The first part is a binary part indicating whether the questions of interest were applicable, the second part (also binary) indicates whether the provided response was at the lower end of the scale (did the participant have any dependence?), and the third part (continuous) determines the level of the response (nicotine dependence) for those with any dependence. Our motivating example is a common situation in surveys assessing any substance use which is a widespread behavioral issue with serious and long-ranging implications. Additionally, situations with missing responses due to inapplicability also occur in educational contexts. As an example, students may be asked in a survey to rate their satisfaction with the career services offered at their school, or, similarly, teachers may be asked to rate their satisfaction with the professional development services provided by the school on a scale of 1 to 7, with higher values representing higher satisfaction (e.g. [2]). This question may only be considered applicable/asked of subjects who used the services recently, and, as in other surveys measuring subject satisfaction ([3]), the distribution of the satisfaction variables may be skewed with many values at the highest (or lowest) end of the scale. If the values are inflated at the upper end of the scale, the scale can be reversed.

There is an extensive literature on analyzing cross-sectional and longitudinal semi-continuous data that has a spike at zero and a continuous distribution for the non-zero (positive) values as this kind of data is encountered in many areas of research (e.g. medical costs, alcohol consumption, biomarkers, microbiome). Papers [4], [5], and [6] provide reviews of methods for analyzing cross-sectional and longitudinal semicontinuous data. The models usually consist of a logistic regression for the first part (zero vs. non-zero model) and a normal (or other continuous distribution) model for the non-zero values or some transformation of these values ([7, 8, 9, 10, 11]). Correlations among repeated observations on the same individual and between the two parts are induced by random effects. Simulation studies have found that assuming independence between the 2 parts (that is, modeling them separately) can result in biased regression coefficient and variance component estimates in the continuous part [12].

There is scarcer literature specifically devoted to dealing with “not applicable” responses. The study in [13] examines ways of dealing with “not applicable” responses to items in the AMC Linear Disability Score project item bank measuring functional status. The items measure subjects’ abilities to perform activities and have three possible categories: “I could carry out the activity”, “I could not carry out the activity”, and “not applicable” if patients never experienced the activity. Four methods of dealing with “not applicable” responses were considered: cold deck imputation, hot deck imputation, ignoring them, and using a model which accounts for the tendency to respond to items. The authors recommend one of the last three methods, depending on the purpose of the analysis. To deal with “not applicable” responses to ordinal items in scale measurement, the study in [14] proposes using item response theory (IRT) models for semi-ordered data. Compared to models that ignore the “not applicable” responses, the models accommodating “not applicable” responses reduced standard error and nonresponse bias of trait estimates for participants with “not applicable” responses, and provided information about the relationship between “not applicable” responses and the trait. Thus, due to increased information, using the “not applicable” responses increased the reliability of estimates for persons with “not applicable” responses for one or more items on the scale.

In the context of our nicotine dependence data, two-part models could be applied only if we ignored the part indicating who was asked the question of interest. However, this part could provide useful information and it may be advantageous to jointly model this part (Part 1: the probability of being asked the questions of interest) with the two parts corresponding to the continuous response (Part 2: probability of giving a response at the lower end of the scale, and Part 3: the mean response when the response is above the lower end of the scale). Subjects who are asked the questions may differ from people who are not asked the questions in meaningful ways, just as people who give responses on the lower range may differ from people who provide responses ab ove the lower range. We propose a three-part model for such data which consists of two logistic regressions sub-models and a truncated normal model. The model is able to handle repeatedly measured data; correlations within a subject and between model parts are induced by random effects in each one of the three parts and assuming a random distribution for those. Maximum likelihood estimation and inference is performed by using adaptive quadrature approximation available in standard statistical software. We use a simulation study to investigate bias and efficiency of the three-part model compared to using simpler models. We illustrate the model by applying it to data on nicotine dependence to tobacco measured repeatedly over time in the Population Assessment of Tobacco and Health survey, but the model can be applied to the analysis of longitudinal skewed survey data with “not applicable” responses in other fields of study such as education and behavior research.

The paper is organized as follows: Section 2 describes the motivating data, Section 3 presents related models in previous studies and introduces our three-part model for longitudinal skewed survey data, Section 4 applies the model to the PATH nicotine dependence data, Section 5 presents the simulation study, while Section 6 discusses conclusions and possible future work.

2. Motivating Data Description

The PATH study is a national longitudinal survey study of tobacco use and health ([1]). Participants were interviewed over four waves with approximately a year in between consecutive waves. Wave 1 was collected between September 12, 2013 and December 14, 2014, Wave 2 between October 23, 2014 and October 30, 2015, Wave 3 between October 19, 2015 and October 23, 2016, and Wave 4 between December 1, 2016 and January 3, 2018. The PATH survey study was conducted by Westat, a contract research organization, and approved by the Westat Institutional Review Board (IRB) (Westat IRB number 0000695; PATH project number 8954). Participants provided informed consent to Westat before participating in the study. Interviewing procedures, questionnaires, and sampling methods can be found at https://doi.org/10.3886/Series606. For this analysis, we focused on young adult (18–34 year old) current established tobacco smokers as this is the group with largest expected change in nicotine dependence (N = 6699). Of these, 15% (N = 1018) had no follow-up data at Waves 2-4 and were excluded (N = 5681 left). The data from Wave 1 was not included in the longitudinal model because the nicotine dependence questions were applicable for everyone in our sample at Wave 1 (as they were current established smokers).

Nicotine dependence to tobacco was assessed at each wave by using a scale consisting of 16 items (e.g. “I frequently crave tobacco products”, “tobacco products control me”) ([15]). The items were asked of participants in random order and were on a 1 to 5 scale (1= Not true of me at all, 5= Extremely true of me), with higher scores indicating more dependence. The scale was computed by averaging the available items for each subject as [15] showed that averaging these items provide a psychometrically valid scale for measuring nicotine dependence. Cronbach alpha values at each one of the Waves 2-4 were all 0.96 indicating good reliability.

The nicotine dependence items were only asked of respondents for whom the questions were applicable. PATH used a complex definition for determining the sub-sample that was asked these questions. For example, at Wave 2 the questions were asked of adult respondents who were current established or new former established tobacco users, or adult respondents who were current experimental tobacco users and used more than one product. This skip logic may inadvertently omit individuals who are at risk for nicotine dependence, but do not meet the criteria for membership in the user groups defined by PATH. More specifically, researchers interested in examining risk for nicotine dependence are likely to miss some individuals who are using or have used tobacco products. Hence, ignoring the skip logic might introduce some bias in the results.

We modeled together the nicotine dependence scale and whether the scale was applicable to a particular subject at a particular wave. Because the nicotine dependence scale had a skewed distribution with many 1 values, a two-part model was necessary to model its distribution. Note that the cutoff of 1 is conceptually meaningful as it indicates the absence of nicotine dependence symptoms. Therefore, a three-part model was used with the following outcomes measured at each wave $j=2,3,4$ for each subject $i=1,\dots ,N$ (Figure 1):

Figure 1:

Diagram of three-part model

• a binary variable $Y_{1ij}$ indicating whether the nicotine dependence questions were applicable to subject $i$ at wave $j$ ($Y_{1ij}=1$ if questions applicable, 0 otherwise)

• a binary variable indicating whether the subject had any nicotine dependence at wave $j$ (that is, whether the nicotine dependence score $Y_{2ij}$ was greater than 1 given that $Y_{1ij}=1$)

• model for continuous nicotine dependence level at wave $j$, $Y_{2ij}$, for subjects with nicotine dependence $(Y_{2ij}\mid Y_{2ij}>1,Y_{1ij}=1)$.

3. Methods

3.1. Related Models in Previous Studies

Previous studies ([4], [5] and [6]) provide descriptions of joint two-part random effects models for longitudinal semicontinuous data that has a spike at the lowest value (typically zero) and a continuous distribution for the other values. The models consist of a logistic regression for the first part (zero vs. non-zero model) and a normal (or other continuous distribution) regression model for the raw or transformed positive values ([7, 8, 9, 10, 11]). However, these models don’t have a part to account for whether the questions of interest were applicable which we introduce in our model in the next section.

3.2. Joint Three-Part Random Effects Model

The model for nicotine dependence data consists of the following three parts:

• Part 1: The probability that the nicotine dependence questions are applicable to subject $i=1,\dots ,N$ at wave $j=2,3,4$, ${\pi }_{ij}=P(Y_{1ij}=1\mid {\mathit{u}}_{\mathit{i}0})$, is modeled by

  $$\mathrm{logit}({\pi }_{ij})={\mathit{x}}_{\mathit{ij}}^{\mathit{T}}\alpha +{\mathit{z}}_{\mathit{ij}}^{\mathit{T}}{\mathit{u}}_{\mathit{i}0},$$

  where $\alpha$ represents the regression coefficient vector for fixed effects (including intercept), ${\mathit{u}}_{\mathit{i}0}$ represents the Part 1 random effects vector for subject $i$, ${\mathit{x}}_{\mathit{ij}}$ is the covariate vector for fixed effects, and ${\mathit{z}}_{\mathit{ij}}$ is the covariate vector for random effects.
• Part 2: The probability that the subject $i$ had any nicotine dependence at wave $j$, $p_{ij}=P(Y_{2ij}>1\mid {\mathit{u}}_{\mathit{i}1})$, is modeled by

  $$\mathrm{logit}(p_{ij})={\mathit{x}}_{\mathit{ij}}^{\mathit{T}}\beta +{\mathit{z}}_{\mathit{ij}}^{\mathit{T}}{\mathit{u}}_{\mathit{i}1},$$

  where $\beta$ represents the regression coefficient vector for fixed effects, ${\mathit{u}}_{\mathit{i}1}$ the Part 2 random effects vector (for subject $i$), and ${\mathit{x}}_{\mathit{ij}}$ and ${\mathit{z}}_{\mathit{ij}}$ are covariate vectors corresponding to this part of the model.
• Part 3: The probability distribution of the transformed nicotine dependence scores given any nicotine dependence, $Y_{3ij}\mid (Y_{2ij}>1,{\mathit{u}}_{\mathit{i}2})\sim TN({\mu }_{ij},{\sigma }^2)$, where $Y_{3ij}=10\sqrt{Y_{2ij}},Y_{2ij}$ is the original dependence score, and TN denotes a truncated normal distribution. These scores were square-root transformed because of skewness and then multiplied by 10 to help model convergence. The transformed scores were modeled as truncated normal, that is, a normal distribution with possible values between 10 and $10\sqrt{5}$. This distribution is more suitable here than the normal distribution because the normal distribution assumes that all values are possible whereas the truncated normal allows for a bounded support ([16, 17]) The mean of this distribution was modeled by

  $${\mu }_{ij}={\mathit{x}}_{\mathit{ij}}^{\mathit{T}}\gamma +{\mathit{z}}_{\mathit{ij}}^{\mathit{T}}{\mathit{u}}_{\mathit{i}2},$$

  where $\gamma$ represents the regression coefficient vector for fixed effects, ${\mathit{u}}_{\mathit{i}2}$ the Part 3 random effects vector (for subject $i$), and ${\mathit{x}}_{\mathit{ij}}$ and ${\mathit{z}}_{\mathit{ij}}$ are covariate vectors corresponding to Part 3 of the model.

The random effects for subject $i$ across the three parts, ${\mathit{u}}_{\mathit{i}}={\left({\mathit{u}}_{\mathit{i}0}^{\mathit{T}},{\mathit{u}}_{\mathit{i}1}^{\mathit{T}},{\mathit{u}}_{\mathit{i}2}^{\mathit{T}}\right)}^T$, are assumed to be normally distributed with mean $0$ and variance-covariance matrix $\mathit{D}$. The elements of $\mathit{D}$ represent the association among the random effects corresponding to the three parts of the model. In the model equations above, the same set of covariates ${\mathit{x}}_{\mathit{ij}}$ and ${\mathit{z}}_{\mathit{ij}}$ were used in all three parts although this is not required for model fitting. For longitudinal studies, the random effects ${\mathit{u}}_{\mathit{i}}$ could be random intercepts or random intercepts together with random slopes. Random quadratic terms or higher order terms could also be used as needed but adding many random effects could be computationally demanding. Random intercepts allow for the starting point of the outcome variables to vary across subjects while random slopes allow for slopes of change over time to vary across subjects.

The likelihood of the joint model is $L={\prod }_{i=1}^N{L}_i$, where $L_i$, the likelihood for the $i\text{th}$ subject, equals:

$$\int \left[\prod _{j=2}^{t_i}{(1-{\pi }_{ij})}^{1-y_{1ij}}{\left({\pi }_{ij}{(1-p_{ij})}^{I(y_{2ij}=1)}{p}_{ij}^{I(y_{2ij}>1)}{f}_{TN}{(y_{2ij}\mid u_i)}^{I(y_{2ij}>1)}\right)}^{y_{1ij}}\right]f_U({\mathit{u}}_{\mathit{i}})d{\mathit{u}}_{\mathit{i}},$$ (4)

with $f_{TN}$ representing the truncated normal density, $f_U$ the random effect distribution (multivariate normal density with $0$ mean), and $j$ is the wave counter running from 2 to last available wave for subject $i$, $t_i$. The counter skips over the missing waves.

Parameters were estimated using maximum likelihood estimation assuming data is missing-at-random (MAR) (i.e. missingness depends only on the observed data). While MAR is often reasonable, it’s an assumption that cannot be tested. We cannot distinguish between MAR and missing-not-at-random (MNAR) as we cannot verify that missingness does not depend on unobserved data, but sensitivity analyses can be conducted to investigate how results might change under an MNAR assumption (e.g. pattern mixture models or selection models [18]). Because the likelihood function is an integral with no closed form expression, we used adaptive Gaussian quadrature in Proc NLMIXED in SAS to approximate it using a fixed number of quadrature points ([19]). We used 10 quadrature points per dimension for the three- and two-part models (as these are computationally expensive), and 100 quadrature points for the separate (one-part) models.

4. Real Data Analysis

As mentioned in the Motivating Data Description section (Section 2), the outcome of interest, nicotine dependence to tobacco, was measured at each wave of the Population Assessment of Tobacco and Health (PATH) study using a continuous 1 to 5 scale.

We analyze the trends in the nicotine dependence data, with the following fixed effects in the model (both categorical variables): wave (with the most recent wave, Wave 4, as reference) and gender (with female as reference). We include a random intercept in all three parts of the model:

• Part 1: $\mathrm{logit}({\pi }_{ij})={\alpha }_0+{\alpha }_1\ast I(wave=2)+{\alpha }_2\ast I(wave=3)+{\alpha }_3\ast I(gender=male)+u_{i0}$

• Part 2: $\mathrm{logit}(p_{ij})={\beta }_0+{\beta }_1\ast I(wave=2)+{\beta }_2\ast I(wave=3)+{\beta }_3\ast I(gender=male)+u_{i1}$

• Part 3: $u_{ij}={\gamma }_0+{\gamma }_1\ast I(wave=2)+{\gamma }_2\ast I(wave=3)+{\gamma }_3\ast I(gender=male)+u_{i2}$,

where $I$ denotes the indicator function that equals 1 when the condition in parentheses is satisfied, and $u_{i0}$, $u_{i1}$ and $u_{i2}$ are random intercepts corresponding to the three parts. The $\alpha$’s, $\beta$’s, and $\gamma$’s are the regression coefficients corresponding to fixed effects for Part 1, 2, and 3, respectively. The variance-covariance matrix $\mathit{D}$ of the three random intercepts is given by

$$\mathit{D}=\left(\begin{array}{ccc}\hfill {\sigma }_{u_1}^2\hfill & \hfill {\rho }_{12}{\sigma }_{u_1}{\sigma }_{u_2}\hfill & \hfill {\rho }_{13}{\sigma }_{u_1}{\sigma }_{u_3}\hfill \\ \hfill {\rho }_{12}{\sigma }_{u_1}{\sigma }_{u_2}\hfill & \hfill {\sigma }_{u_2}^2\hfill & \hfill {\rho }_{23}{\sigma }_{u_2}{\sigma }_{u_3}\hfill \\ \hfill {\rho }_{13}{\sigma }_{u_1}{\sigma }_{u_3}\hfill & \hfill {\rho }_{23}{\sigma }_{u_2}{\sigma }_{u_3}\hfill & \hfill {\sigma }_{u_3}^2\hfill \end{array}\right),$$ (5)

with ${\sigma }_u$’s denoting the standard deviation of the random intercepts in the three parts and $\rho$’s the correlations between the random intercepts. The size of the variance components (${\sigma }_{u_1}^2$, ${\sigma }_{u_2}^2$, and ${\sigma }_{u_3}^2$) indicates the amount of variability among subjects in the log-odds of the questions being applicable, log-odds of having any nicotine dependence, and nicotine dependence score (when score>1), respectively. A larger value of the variance component means more variability among subjects.

We fit three model types: 1) joint model, 2) three separate models (i.e. same as assuming independent random effects across the three parts), and 3) fitting a Part 1 model and a model for the last two parts together (i.e. same as assuming random effects in first part independent of the random effects in the last two parts). We compare models using AIC/BIC fit criteria. For the purpose of obtaining AIC/BIC values that are based on the same sample and comparable for all the models, AIC/BICs for the “Three separate models” and “Part 1 + two-part joint model” were estimated from fitting three-part joint models with appropriate correlations between parts set to zero. Because the random effects across the three parts are jointly normal, zero correlation between any two parts implies independence between the two parts. We also fit a three-part joint model using a normal distribution in Part 3 rather than truncated normal and the truncated normal model was a better fit in terms of AIC/BIC (Supplementary Table S17).

Most subjects (94%, N = 5343 out of 5681) had outcome data available at Wave 2, 87% (N = 4964) at Wave 3, and 80% (N = 4531) at Wave 4. The proportion of males was 57%. Gender was available for everyone in the sample as we used the imputed version of this variable provided by PATH (two missing values were logically imputed by PATH from the household screener data as described in the PATH Study Public-Use Files User Guide at https://www.icpsr.umich.edu/files/NAHDAP/documentation/ug36498-all.pdf).

Table 1 contains descriptive statistics related to the nicotine dependence questions. The nicotine dependence questions were applicable to almost everyone at Wave 2 but that proportion changed in the following waves: questions were not applicable to 0.3% of subjects in Wave 2, 10.4% in Wave 3, and 13.6% in Wave 4. Overall across the waves, 14.8% of the nicotine dependence scores were at the lower end of the nicotine dependence scale meaning the individuals had no nicotine dependence. This percentage didn’t change much across waves (15.2% at Wave 2, 14.8% at Wave 3, 14.3% at Wave 4). Figure 2 displays the distribution of the nicotine dependence scores by wave.

Table 1:

Descriptive statistics of nicotine dependence questions by wave

Variable	N(%) or Mean (SD)	N
Are nicotine dependence questions applicable at Wave 2?		5343
No	14 (0.26%)
Yes	5329 (99.7%)
Are nicotine dependence questions applicable at Wave 3?		4964
No	516 (10.4%)
Yes	4448 (89.6%)
Are nicotine dependence questions applicable at Wave 4?		4531
No	616 (13.6%)
Yes	3915 (86.4%)
Nicotine dependence score equals 1 (=no nicotine dependence) at Wave 2		5300
No	4497 (84.8%)
Yes	803 (15.2%)
Nicotine dependence score equals 1 (=no nicotine dependence) at Wave 3		3960
No	3372 (85.2%)
Yes	588 (14.8%)
Nicotine dependence score equals 1 (=no nicotine dependence) at Wave 4		3866
No	3314 (85.7%)
Yes	552 (14.3%)
Mean (SD) of nicotine dependence scores greater than 1 at Wave 2	2.55 (1.05)	4497
Mean (SD) of nicotine dependence scores greater than 1 at Wave 3	2.62 (1.08)	3372
Mean (SD) of nicotine dependence scores greater than 1 at Wave 4	2.65 (1.06)	3314

Figure 2:

Histograms of nicotine dependence to tobacco by wave

The model results are in Table 2, and plots of the results are in the Supplementary File Figure S1. The results have a subject-specific interpretation (conditional on random effects). For the joint model, estimates from the Part 1 of the model ($\alpha$ coefficients) indicate that, for a given subject, the nicotine dependence questions were more likely to be applicable in earlier waves compared to the last wave (Wave 4) (p < .0001). Among those for whom the questions were applicable (Part 2, $\beta$ coefficients) and who had the same random effect, males were less likely than females to report any nicotine dependence (log OR = −0.77, 95% CI (−1.00, −0.55)), and participants were more likely to report any nicotine dependence at Wave 2 than at Wave 4 (log OR = 0.70, 95% CI 0.52 to 0.87). Among those with any nicotine dependence (Part 3, $\gamma$ coefficients), males had lower dependence scores than females with same random effects, and participants had higher scores at Wave 2 than at Wave 4. The model estimated strong positive correlations among the random effects in the three parts of the model: 0.78 for the correlation between the random intercepts for parts 1 and 2, 0.68 between parts 1 and 3, and 0.81 between parts 2 and 3. The size of the SD of the random intercepts (${\sigma }_{u_1}$, ${\sigma }_{u_2}$, ${\sigma }_{u_3}$) indicated large variability among subjects in the log-odds of the questions being applicable, log-odds of having any nicotine dependence, and nicotine dependence score (when score>1), respectively.

Table 2:

Results from application

	Three-part joint model		Three separate models		Part 1 + Two-part joint model
Parameter	Estimate	95% CI	Estimate	95% CI	Estimate	95% CI
Part 1
α0 (Intercept)	4.76	(4.30, 5.23)	4.80	(4.31, 5.3)	4.80	(4.31, 5.3)
α1 (wave 2 vs 4)	7.64	(6.71, 8.56)	7.70	(6.75, 8.66)	7.70	(6.75, 8.66)
α2 (wave 3 vs 4)	0.74	(0.53, 0.94)	0.73	(0.52, 0.94)	0.73	(0.52, 0.94)
α3 (male)	0.21	(−0.14, 0.55)	0.29	(−0.07, 0.65)	0.29	(−0.07, 0.65)
Part 2
β0 (Intercept)	3.28	(3.02, 3.54)	3.52	(3.25, 3.78)	3.35	(3.10, 3.60)
β1 (wave 2 vs 4)	0.70	(0.52, 0.87)	0.28	(0.11, 0.45)	0.33	(0.16, 0.49)
β2 (wave 3 vs 4)	0.15	(−0.02, 0.32)	0.09	(−0.08, 0.26)	0.09	(−0.08, 0.26)
β3 (male)	−0.77	(−1.00, −0.55)	−0.75	(−0.97, −0.52)	−0.74	(−0.95, −0.53)
Part 3
γ0 (Intercept)	15.06	(14.85, 15.28)	15.82	(15.62, 16.02)	15.21	(15, 15.42)
γ1 (wave 2 vs 4)	0.14	(0.02, 0.25)	0.02	(−0.10, 0.14)	0.06	(−0.06, 0.18)
γ2 (wave 3 vs 4)	−0.03	(−0.15, 0.10)	−0.05	(−0.18, 0.08)	−0.04	(−0.16, 0.09)
γ3 (male)	−1.10	(−1.36, −0.84)	−0.99	(−1.23, −0.75)	−1.10	(−1.36, −0.85)
SD/correlations
ρ12 (correlation Parts 1-2)	0.78	(0.73, 0.82)
ρ13 (correlation Parts 1-3)	0.68	(0.64, 0.72)
ρ23 (correlation Parts 2-3)	0.81	(0.78, 0.85)
σe (SD residual Part 3)	2.14	(2.09, 2.18)	2.16	(2.11, 2.21)	2.14	(2.09, 2.19)
σu1 (SD intercept Part 1)	4.05	(3.66, 4.45)	4.13	(3.71, 4.56)	4.13	(3.71, 4.56)
σu2 (SD intercept Part 2)	2.91	(2.69, 3.12)	2.74	(2.52, 2.96)	2.58	(2.37, 2.79)
σu3 (SD intercept Part 3)	4.29	(4.15, 4.43)	3.87	(3.76, 3.99)	4.21	(4.08, 4.35)
Model fit
AIC	66096		67745		67050
BIC	66222		67851		67163

Note: The scores in Part 3 were sqrt-transformed to reduce skewness and then multiplied by 10 to help convergence. For ”Three separate models” and ”Part 1 + Two-part joint model”, AIC/BIC were estimated from fitting three-part joint models with appropriate correlations between parts set to zero.

For Parts 1 and 2, the conclusions based on the simpler models were the same as the conclusions based on the joint model, although the coefficients were somewhat different. Notably, we saw an attenuation of the effect of wave 2 vs. wave 4 parameter in Part 2 in the simpler models. We also saw this attenuation of the effect of wave 2 vs. wave 4 parameter in Part 3 in the simpler models, with the effect becoming statistically non-significant in the simpler models. The table also presents model fit criteria (AIC/BICs) which indicate the three-part joint model provides the best fit.

5. Simulation Study

5.1. Simulation Description

We used simulations to evaluate bias and efficiency of the proposed three-part model compared to simpler models. We considered four simulation conditions, and for each condition we randomly generated 100 datasets with N = 1000 subjects in each dataset measured at three waves.

The fixed parameter estimates for all simulations conditions were α₂=0.74, α₃=0.21, β₀=3.28, β₁=0.70, β₂=0.15, β₃=−0.77, γ₀=15.06, γ₁=0.14, γ₂=−0.03, γ₃=−1.10, σ_e=2.14, σ_u1=2.14, σ_u2=4.05, σ_u3=2.91, and σ_u4=4.29 as obtained from the motivating data. We decreased α₁ from 7.64 to 2.1 in order to increase the proportion of subjects for whom the questions were not applicable at Wave 2 (from 0.3% in real data to an average of 6% in simulated datasets).

5.1.1. Condition 1

The correlations between the random effects (ρ′s=0.68, 0.78, 0.81) and the intercept of Part 1, α₀ = 4.76, were as observed in the motivating data. Each dataset had 3000 observations (3 waves per subjects times 1000 subjects). In each dataset, 570 participants were male and the rest (430) were female in order to obtain a sample proportion of males (57%) similar to the one in the motivating dataset. For each participant we generated first a binary value (0/1) indicating whether nicotine dependence questions were applicable according to the Bernoulli distribution in Part 1 of the model. For those for whom the questions were applicable, we selected which subjects had any nicotine dependence (score > 1) according to the Bernoulli distribution in Part 2 of the model, and then for those subjects we generated a transformed nicotine dependence score between 10 and $10\sqrt{5}$ from the truncated normal distribution in Part 3 of the model.

5.1.2. Condition 2

The correlations between the random effects (ρ′s) were all set to 0.4 and as in the motivating data α₀ = 4.76. The correlations between random effects estimated from the data and used in Condition 1 were in the moderate-high range (0.68, 0.78, 0.81). In order to investigate what happens when the three parts are less correlated, we ran simulations under a condition that set all the correlations (ρ′s) at 0.4.

5.1.3. Condition 3

The correlations between the random effects were as in the motivating data (ρ′s=0.68, 0.78, 0.81) and the intercept of Part 1, α₀, was set to 2. Because the sample consists of currently established tobacco users, the proportion of observations (person-wave combinations) for which the questions are applicable is high (92% in the observed data). In order to evaluate what happens under a lower proportion, we replaced α₀ (4.76) by 2, which gives an average of 75% proportion of observations for which the questions are applicable across the simulated datasets.

5.1.4. Condition 4

The correlations between the random effects (ρ′s) were all set to 0.4 and the intercept of Part 1, α₀, was set to 2. This condition replaces α₀ (4.76 in the data example) by 2 to decrease the proportion to whom questions are applicable (as in Condition 3) and at the same time reduce the correlations between random effects to 0.4 (as in Condition 2).

For each condition, we considered three model types: 1) joint model, 2) three separate models (i.e. same as assuming independent random effects), and 3) fitting a model for Part 1 and another model for last 2 parts together (i.e. same as random effect in first part independent of the random effects in the last two parts). For each condition and model type, we present the mean estimate for each parameter, the bias of these estimates, the square-root of mean squared error (RMSE), and the coverage probabilities of the 95% confidence intervals (percentage of the intervals that contain the true value) across the 100 simulated datasets. The mean estimate for a parameter (true value θ) is $S^{-1}{\sum }_{s=1}^S{\widehat{\theta }}_s$, where s runs over the simulation replicates (S=100 in total) and ${\widehat{\theta }}^s$ is the estimate obtained from simulation replicate s. Bias for a parameter is defined as the average difference between the estimate and the true value across the simulation replicates: $S^{-1}{\sum }_{s=1}^S({\widehat{\theta }}_s-{\theta }_s)$. RMSE is defined by RMSE = ${(S^{-1}{\sum }_{s=1}^S{({\widehat{\theta }}_s-{\theta }_s)}^2)}^{1∕2}$, that is, the average squared difference between the estimate and the true value across the simulation replicates. Coverage proportion was defined as the proportion of confidence intervals across the simulations that included the true value: $S^{-1}{\sum }_{s=1}^{S}I({\widehat{\theta }}_s$ < upper limit of Wald CI for θ_s and ${\widehat{\theta }}_s$ > lower limit of Wald CI for θ_s). In the Supplementary file, we are also reporting the SD of parameter estimates and the average of the standard error across simulation replicates to see whether the maximum likelihood estimation captures the appropriate size of uncertainty of parameter estimates (Supplementary Tables S9-S16).

We also ran a similar simulation study with a lower sample size of N=500 subjects.

5.2. Simulation Results

The results from the four simulation conditions for N=1000 subjects are described below. The simulation results with N=500 subjects led to the same substantive conclusions (Supplementary Tables S5-S8).

5.2.1. Condition 1

For the joint model, there was little to no bias in estimating the parameters and the coverage probabilities were close to 0.95 except for the coverage probability of the intercept of Part 1, α₀ (Figure 3 and Supplementary Table S1). For the three separate models, significant bias and low 95% CI coverage probabilities were seen for estimating the intercept and standard deviation of the random effect in Part 3 (γ₀ and σ_u3 ), but there was also a noticeable bias in estimating other coefficients (e.g. β₀, β₁, standard deviation of random effect of Part 2, σ_u2). On average, fitting the last two parts together (“Part 1+Two-part joint model”) gave better results than the separate models but there was still considerable bias in estimating certain parameters such as γ₀, β₀, β₁ , and σ_u2.

Figure 3:

Simulation results Condition 1

Note. RMSE = root mean squared error; Coverage = 95% CI coverage proportion; α₀, β₀, γ₀ - intercepts for the 3 parts; α₁, β₁, γ₁ - wave 2 vs 4 coefficients for the 3 parts; α₂, β₂, γ₂ - wave 3 vs 4 coefficients for the 3 parts; ρ_ij - correlation Parts i-j; σ_e - SD residual Part 3; σ_ui - SD intercept Part i

5.2.2. Condition 2

With lower correlations among the three parts, there was higher bias than with higher correlation in estimating the intercept (α₀) and the standard deviation of the random effect (σ_u1) for Part 1 of the joint model (Figure 4 and Supplementary Table S2). The separate models performed better than in Condition 1 in terms of bias although the bias for γ₀ remains significant. Modeling the last two parts together had an overall good performance except for bias in the two intercepts (β₀, γ₀) and the standard deviation of the random effect (σ_u2) for Part 2.

Figure 4:

Simulation results Condition 2

Note. RMSE = root mean squared error; Coverage = 95% CI coverage proportion; α₀, β₀, γ₀ - intercepts for the 3 parts; α₁, β₁, γ₁ - wave 2 vs 4 coefficients for the 3 parts; α₂, β₂, γ₂ - wave 3 vs 4 coefficients for the 3 parts; ρ_ij - correlation Parts i-j; σ_e - SD residual Part 3; σ_ui - SD intercept Part i

5.2.3. Condition 3

When we replaced α₀ (4.54) by 2 in order to reduce the proportion of observations for which the nicotine dependence questions are applicable, for the joint model the bias in estimating the intercept (α₀) was greatly reduced (−0.03) (Figure 5 and Supplementary Table S3). All the other biases were relatively low (less than 0.05 in absolute value).

Figure 5:

Simulation results Condition 3

Note. RMSE = root mean squared error; Coverage = 95% CI coverage proportion; α₀, β₀, γ₀ - intercepts for the 3 parts; α₁, β₁, γ₁ - wave 2 vs 4 coefficients for the 3 parts; α₂, β₂, γ₂ - wave 3 vs 4 coefficients for the 3 parts; ρ_ij - correlation Parts i-j; σ_e - SD residual Part 3; σ_ui - SD intercept Part i

For the three separate models, as before, there was significant bias in estimating the intercept and standard deviation of the random effect in Part 3 (γ₀ and σ_u3) and β₀, β₁, σ_u2. On average, fitting the last two parts together gave better results than the separate models but there was still considerable bias in estimating certain parameters such as the intercepts (γ₀, β₀), β₁ , σ_u2 , and σ_u3.

Of note, the biases observed for the non-three-part-joint models were larger in magnitude than in all other conditions (e.g. as high as 1.13 for γ₀ when fitting three separate models and as high as 0.84 for γ₀ when fitting last two parts together).

5.2.4. Condition 4

With lower correlations among the three parts and lower α₀, there were negligible biases for all the coefficients of the joint model (Figure 6 and Supplementary Table S3). As the three parts were more loosely related, the separate models performed better than in Condition 3 in terms of bias although there was still very high bias for estimating γ₀, β₀ and σ_u3 . Modeling the last two parts together showed high bias in estimating some parameters such as γ₀ and β₀.

Figure 6:

Simulation results Condition 4

Note. RMSE = root mean squared error; Coverage = 95% CI coverage proportion; α₀, β₀, γ₀ - intercepts for the 3 parts; α₁, β₁, γ₁ - wave 2 vs 4 coefficients for the 3 parts; α₂, β₂, γ₂ - wave 3 vs 4 coefficients for the 3 parts; ρ_ij - correlation Parts i-j; σ_e - SD residual Part 3; σ_ui - SD intercept Part i

6. Discussion

In this paper, we developed a three-part random effect model for longitudinal skewed survey response questions. The three parts are needed because only a subset of participants are being asked the question of interest (the participants to whom the questions are applicable) and the response has an excess of values at the lower end of the scale. The subset may differ in meaningful ways from the subset not being asked the question and that information needs to be modeled. Also, excess values at the lower end of the scale are known to introduce bias in models when not accounted for. The three parts of the proposed joint model are: 1) model for the probabilities of being asked the question of interest (logistic model), 2) model for the probabilities of giving a response at the lower end of the scale (logistic model) and 3) model for the mean response when above the lower end of the scale (truncated normal model). We correlate these three parts and the longitudinal measurements within a subject by using random effects. The model can be fit using standard software procedures (e.g. PROC NLMIXED in SAS). We applied the model to data on nicotine dependence to tobacco collected by the PATH national survey. Our interpretation is specifically for the group of current established young adult smokers for which we evaluated change in nicotine dependence after a period of consistent nicotine product use. Among current established young adult smokers, males (compared to females with the same levels of the random effects) had lower odds of any nicotine dependence to tobacco and lower mean score when dependent to tobacco. For a given subject, there was evidence of an effect of wave on the magnitude of the dependence score (when dependent to tobacco) in the joint three-part model but not in the simpler models that we used for comparison.

The simulation study used to investigate the characteristics of the three-part model versus simpler models showed that, overall, the three-part model had lower biases and better coverage probabilities for the regression coefficients than simpler models. The biases in the simpler models became worse as we reduced the proportion of observations for which the nicotine dependence questions are applicable indicating that modeling the probabilities of being asked the questions of interest is more important when fewer subjects are being asked those questions. Also, simpler models performed much worse when the three parts were more highly correlated suggesting that when the three parts are highly correlated modeling them together preserves more information from the data. The higher biases for the simpler models were observed for the intercept of the second and third parts, but there were also biases for other coefficients. This is similar to the simulation results from other studies that examined biases when separating the parts of two- and three-part models ([12, 20]). For example, [12] found that, when the two parts of a two-part model for semicontinuous data were modeled separately, the highest bias occurred in the intercept of the second part but biases were also seen in other coefficients.

Future work for this model could consider non-normal distributions for the random effects ([21]) or for the response ([22]). In our application, since we only had three time-points, we did not use a random slope to model the data as the time trend is more freely modeled by a categorical variable for time. For the situations when time is continuous, a random slope or some higher order random terms may be needed for each one of the three parts of the model. Introducing more random effects in each one of the three parts may significantly increase the time it takes to run the models or cause convergence failure. The runtime also limits the number of quadrature points used for the models. Future work could consider decreasing the computational time for fitting the model so that more random effects can be considered and more comprehensive simulations studies can be performed. For example, one could consider decreasing the number of quadrature points (e.g. to 5) as some studies have shown this can produce reasonable parameter estimates ([23, 24]). Also, a Bayesian approach could be considered as a way to deal with an increased number of random effects ([11, 25]) when the adaptive Gaussian quadrature fails, although Bayesian methods can be computationally demanding too. For illustration of the method we presented a simple analysis with only two covariates in the models — however, the analysis could be extended to consider additional baseline and time-dependent covariates in the model at the expense of increasing the computational time (for example, potential covariates could be age at enrollment, age of initiating smoking, tobacco product used, frequency of use, other demographic variables). The model could also be extended to use survey weights in the analysis.

More complicated within-individual covariance structures (e.g. unstructured, AR(1)) could be used to represent correlations that are not accounted for by the random effects. The paper by Harring and Blozis [26] shows a method for implementing them in SAS PROC NLMIXED. We reported the Wald confidence intervals for all the parameters, including correlations and standard deviations, as produced by PROC NLMIXED. Although based on normal approximations, these intervals are reasonable because our study has a large sample size and the estimates are not close to the boundaries (−1 and 1 for correlations, and 0 for standard deviations). In case of small sample sizes or estimates close to the boundaries, confidence intervals could be computed by bootstrapping or the profile likelihood method.

The three-part model we proposed has coefficient interpretations that are conditional on the random effects. For the simple mixed logistic regression model, there is a relationship approximation between the population average coefficients and the subject-specific ones where the subject specific coefficient is multiplied by a scaling factor to get the population level coefficient ([27]). However, for the two-part and three-part models it would be necessary to integrate out the random effects to get the population level estimates. As an alternative, a marginal model could be used if inferences are needed only for the effect of covariates on the marginal mean in the entire population ([10]).

The three-part model proposed provides a method for modeling longitudinal skewed response data obtained from survey questions that are only applicable to a subset of participants, and estimating in a more unbiased way the effect of the predictors on this type of data.

Data availability statement

The data that support the findings of this study are openly available in the Population Assessment of Tobacco and Health (PATH) Study [United States] Public-Use Files (ICPSR 36498) at https://doi.org/10.3886/ICPSR36498.v10.