Data analysis methods for assessing palliative care interventions in one-group pre–post studies

Abstract

Objectives:

Studies of palliative care are often performed using single-arm pre–post study designs that lack causal inference. Thus, in this study, we propose a novel data analysis approach that incorporates risk factors from single-arm studies instead of using paired t-tests to assess intervention effects.

Methods:

Physical, psychological and social evaluations of eligible cancer inpatients were conducted by a hospital-based palliative care team. Quality of life was assessed at baseline and after 7 days of symptomatic treatment using the European Organization for Research and Treatment of Cancer QLQ-C15-PAL. Among 35 patients, 9 were discharged within 1 week and 26 were included in analyses. Structural equation models with observed measurements were applied to estimate direct and indirect intervention effects and simultaneously consider risk factors.

Results:

Parameters were estimated using full models that included associations among covariates and reduced models that excluded covariates with small effects. The total effect was calculated as the sum of intervention and covariate effects and was equal to the mean of the difference (0.513) between pre- and post-intervention quality of life (reduced model intervention effect, 14.749; 95% confidence intervals, −4.407 and 33.905; p = 0.131; covariate effect, −14.236; 95% confidence interval, −33.708 and 5.236; p = 0.152).

Conclusion:

Using the present analytical method for single-arm pre–post study designs, factors that modulate effects of interventions were modelled, and intervention and covariate effects were distinguished based on structural equation model.

Introduction

Palliative care is defined by the World Health Organization as

an approach that improves the quality of life of patients and their families facing the problems associated with life-threatening illness, through the prevention and relief of suffering by means of early identification and comprehensive assessment and treatment of pain and other problems, physical, psychosocial and spiritual.¹

Palliative care is provided through the combined expertise, knowledge and skills of various healthcare professionals and plays an important role in improving quality of life (QOL) by alleviating pain in patients and their family members.²

The demand for evidence-based palliative care has been growing in recent years, and randomized controlled trials (RCTs) are indispensable for scientific evaluations of the effectiveness of palliative care.³ However, because patients requiring palliative care are physically or mentally vulnerable and have diverse cultural and religious backgrounds, ethical considerations often preclude rigorous study designs. Accordingly, randomization of palliative care patients to treatment and reference arms is often difficult,^4–8 and most studies in this field still rely on patient and epidemiological surveys.^9–11

Although attempts to evaluate the effects of interventions using only a single treatment arm fail to reveal causal relationships due to the absence of rigorous controls, observations from single-arm studies are often critical and may be used to inform planning for subsequent study phases. Paired t-tests are predominantly used to examine intervention effects. However, we propose a novel data analysis approach that can be utilized with all available data from single-arm intervention studies. Specifically, we employ structural equation models (SEMs) with only observed measurements to evaluate intervention effects and simultaneously investigate modelling associations between intervention outcomes and risk factors and among risk factors. Subsequently, we applied the proposed method to a single-arm hospital-based palliative care team (HPCT) intervention study.

Methods

Patients and interventions

Eligible hospitalized patients with malignant tumours were enrolled between 1 November 2009 and 30 March 2010. Inclusion criteria were as follows: (1) age 18 years and over, (2) pathological diagnosis with a malignant tumour, (3) ability to respond to the questionnaire and (4) written consent. Exclusion criteria were as follows: (1) inability to respond to the questions because of disturbed consciousness or cognitive disorder and (2) inappropriateness for the study as judged by the physician in charge.

Decisions for interventions were reached by the HPCT using the screening sheets and criteria described by Morita et al.¹² and Akizuki et al.,¹³ with minor modifications. Patients requiring intervention were asked to confirm their intention to receive HPCT physical, psychological and social interventions in accordance with pain levels and individual needs. Before and 1 week after interventions, QOL was assessed using the European Organization for Research and Treatment of Cancer QLQ-C15-PAL Questionnaire.¹⁴ Because 9 of 35 patients who satisfied the eligibility criteria for HPCT intervention were discharged from the hospital within 1 week of intervention, the present analysis included only 26 patients for whom QOL assessments were completed at 1 week after intervention. Background variables of the patients are presented in Table 1, and written informed consent was obtained from all participants. The study was approved by the Ethics Committee of Kobe University Hospital and was performed in accordance with the Declaration of Helsinki and the Ethical Guidelines for Clinical Research in Japan.

Table 1.

Demographic and clinical characteristics of subjects.

	No.	Percentage
Sex
Male	19	73.1
Female	7	26.9
Age (years)
Median	58.5
Range	22-81
Performance status
0	2	7.7
1	11	42.3
2	6	23.1
3	7	26.9
4	0	0

Statistical models for one-group pre–post design

In pre–post RCTs, endpoints are measured before and after the intervention, and although two-sample t-tests are often employed, analysis of covariance (ANCOVA) models using pre-data as covariates is considered the optimal statistical design.^15–18

Because no data from a control arm are available, this study design is referred to as ‘one-group pre–post testing’, and the following statistical model is analogous to ANCOVA for RCTs, although the group effect was omitted as follows

$$D_i=\alpha +\beta X_i+{\epsilon }_i$$ (1)

where D_i denotes the difference in efficacy between pre- and post-intervention measurements in each patient, α indicates the effects of the intervention, β refers to the influence of the pre-intervention value $X_i$ and ${\epsilon }_i$ represents measurement error. From equation (1), the mean pre–post difference ($\overline{D}$) can be defined as $(\overline{D}=\widehat{\alpha }+\widehat{\beta }\overline{X})$, which comprises $\widehat{\alpha }$ (without covariate influences) and $\widehat{\beta }\overline{X}$ (with influences of pre-intervention values). Although equation (1) incorporates only the influence of the pre-intervention value into the model, even models that accommodate the influence of numbers (p) of covariates ($D_i=\alpha +{\beta }_1{X}_{i1}+\cdots +{\beta }_p{X}_{ip}+{\epsilon }_i$) usually allow the mean pre–post differences $\overline{D}$ to be divided into intervention effects ($\widehat{\alpha }$) and covariance effects $(\widehat{{\beta }_1}\overline{X_1}+\cdots +{\widehat{\beta }}_p\overline{X_p})$ during model application.

Intervention effect models with covariates

The objective of the one-group pre–post design is often limited to hypothesis generation with exploratory data analysis rather than hypothesis testing with pre-specified statistical models. Moreover, the significance of covariates in multiple regression models is strongly influenced by the magnitude of associations between them. Thus, because multiple covariates are often strongly associated with each other, clinical interpretations of covariate effects are often difficult during comparisons of results from several regression models. To resolve this problem, hypothesized models that allow simple clinical interpretations can be created using SEM, which enables simultaneous modelling of associations among covariates and their influences on efficacy indicators. To illustrate the proposed data analysis approach, the path diagram shown in Figure 1(a) is modelled using pre–post outcome change scores as endpoints and age, sex, performance status (PS) and pre-intervention outcome values as covariates. The depicted associations among measurements are only one of many possibly clinically interpretable model structures, and structural equations corresponding to the Figure 1(a) are defined in equation (2)

Figure 1.

Path diagrams of (a) the full model and (b) the reduced model.

$$\begin{array}{l}{Y}_1={\alpha }_1+{\gamma }_{11}{X}_1+{\gamma }_{12}{X}_2+{\zeta }_1\\ Y_2={\alpha }_2+{\beta }_{21}{Y}_1+{\gamma }_{21}{X}_1+{\gamma }_{22}{X}_2+{\zeta }_2\\ Y_3={\alpha }_3+{\beta }_{31}{Y}_1+{\beta }_{32}{Y}_2+{\gamma }_{31}{X}_1+{\gamma }_{32}{X}_2+{\zeta }_3\\ X_1={\kappa }_1+{\epsilon }_1\\ X_2={\kappa }_2+{\epsilon }_2\end{array}$$ (2)

where Dif (the pre–post difference) in Figure 1(a) corresponds to $Y_3$ in equation (2), and ${\alpha }_3$ is a parameter that expresses the intervention effect (similar to $\alpha$ in equation (1)). Equation (2) describes a model with the assumption that a given covariate affects the dependent (objective) variable directly or indirectly via other covariate(s), and each covariate effect can be expressed as the product of a path coefficient. For example, the influence of PS on Dif can be expressed as the sum of ${\alpha }_1{\beta }_{31}$, which is an effect that originates in the path from PS to Dif, and ${\alpha }_1{\beta }_{21}{\beta }_{32}$, which is an effect that originates in the path mediated by the pre-value. Similar to the model in equation (1), ${\overline{Y}}_3$ can be determined as the sum of ${\widehat{\alpha }}_3$ independently of covariates and as a part that is dependent on covariates $({\widehat{\beta }}_{31}{\widehat{Y}}_1+{\widehat{\beta }}_{32}{\widehat{Y}}_2+{\widehat{\gamma }}_{31}{\widehat{X}}_1+{\widehat{\gamma }}_{32}{\widehat{X}}_2)$This equation is also consistent with the results of the generalization $(\overline{D}=\widehat{\alpha }+\widehat{\beta }\overline{X})$, as shown in the preceding section. Because the standard error of the influence of covariates can be approximated using the multivariate delta method,¹⁹ it is possible to calculate confidence interval (CI) for each estimate of the effect independently of covariates and their influences.

Square box indicates observed measurements, and Greek letter indicates model parameter. Single-headed arrow depicts direction of association between two measurements.

Statistics

Statistical analyses were performed using STATA version 12.0 (Stata Corporation, College Station, TX). All p-values were generated from two-sided tests and were considered significant when p < 0.05. All CIs had a two-sided probability coverage of 95%.

Results

SEMs

The results of analyses using the present method were generated using physical function data. The full model shown in the path diagram (Figure 1(a)) was applied, and estimates, standard errors and CI were calculated for each parameter (Table 2). The total effect comprised the sum of intervention and covariate effects and was consistent with the mean pre–post difference $(\overline{D}=0.513)$. The HPCT intervention effect (${\alpha }_3$) tended to improve physical function, although the change was not significant (intervention effect: 24.57; CI: −16.791 to 65.932; p = 0.24). In contrast, the analysed covariates tended to lower physical function, although this was not significant (covariate effect: −24.06; CI: −65.687 to 17.571; p = 0.26).

Table 2.

Intervention effect models with covariates (full model).

Effect	Parameter	Estimate	SE	95% CI
Intervention effect	${\alpha }_3$	24.571	21.103	−16.791 to 65.932
Covariate effect	Covariate total	−24.058	21.239	−65.687 to 17.571
Pre	${\alpha }_2{\beta }_{32}$	−54.121	18.323	−90.034 to −18.209
PS (⩾2) → Pre	${\alpha }_1{\beta }_{21}{\beta }_{32}$	1.341	6.734	−11.858 to 14.540
Age → PS (⩾2) → Pre	${\kappa }_1{\gamma }_{11}{\beta }_{21}{\beta }_{32}$	7.727	6.903	−5.802 to 21.257
Age → Pre	${\kappa }_1{\gamma }_{21}{\beta }_{32}$	14.776	11.039	−6.861 to 36.413
Sex (male) → Pre	${\gamma }_{22}{\beta }_{32}$	−7.434	4.621	−16.491 to 1.622
Sex (male) → PS (⩾2) → Pre	${\gamma }_{12}{\beta }_{21}{\beta }_{32}$	−1.159	2.522	−6.102 to 3.783
PS (⩾2)	${\alpha }_1{\beta }_{31}$	0.877	4.443	−7.832 to 9.585
Age → PS (⩾2)	${\kappa }_1{\gamma }_{11}{\beta }_{31}$	5.053	5.656	−6.032 to 16.138
Sex (male) → PS (⩾2)	${\kappa }_2{\gamma }_{12}{\beta }_{31}$	−0.758	1.726	−4.142 to 2.626
Age	${\kappa }_1{\gamma }_{31}$	−18.283	14.750	−47.193 to 10.626
Sex (male)	${\gamma }_{32}$	27.924	6.824	14.549 to 41.300
Total effect (intervention + covariate)		0.513	5.679	−10.618 to 11.644
Paired t-test		0.513	5.472	−10.212 to 11.238

SE: standard error; CI: confidence interval; PS: performance status.

Age and PS were affected less by covariates and were excluded from the full model to generate the reduced model. To obtain a parsimonious construct and retain potentially significant covariates in the reduced model, p-values of covariates in the full model were set to 0.15. Accordingly, the path diagram (Figure 1(b)) shows that the reduced model encompasses the effect of gender mediated by a pre-value and the direct effect of the pre-value and gender on Dif. Table 3 shows the intervention effect on physical function and the estimated values of covariates. As in the full model, ${\alpha }_3$ tended to improve physical function, although this was not statistically significant (intervention effect: 14.749; CI: −4.407 to 33.905; p = 0.131). Together, the analysed covariates tended to lower physical function, although the change was not statistically significant (covariate effect: −14.236; CI: −33.708 to 5.236; p = 0.152).

Table 3.

Intervention effect models with covariates (reduced model).

Effect	Parameter	Estimate	SE	95% CI
Intervention effect	${\alpha }_3$	14.749	9.773	−4.407 to 33.905
Covariate effect	Covariate total	−14.236	9.935	−33.708 to 5.236
Pre	${\alpha }_2{\beta }_{32}$	−32.844	9.439	−51.346 to −14.343
Sex (male)	${\kappa }_2{\gamma }_{32}$	27.812	7.034	16.211 to 39.412
Sex (male) → pre	${\kappa }_2{\gamma }_{22}{\beta }_{32}$	−9.204	5.919	−20.804 to 2.397
Total effect (intervention + covariate)		0.513	5.365	−10.003 to 11.029
Paired t-test		0.513	5.472	−10.212 to 11.238

SE: standard error; CI: confidence interval.

Impact of small sample size on SEM

Only small samples were available to fit full and intervention effect models. However, all models converged after few iterative calculations. Because the normal likelihood method was used to estimate standard errors of parameters, the stability of these estimates was examined by evaluating the variance of the estimator using bootstrapping. Estimates of asymptotic standard errors and bootstrap standard errors, and of bias-corrected CI, are shown in Table 4. Results of bootstrap analyses indicated that SEM with observed measurements was applicable to relatively small sample sizes.

Table 4.

Estimates of standard error using the bootstrap method.

Effect	Parameter	Normal likelihood		Bootstrap
		Estimate	SE	SE	95% CI
Dif (intervention effect)	${\alpha }_3$	14.749	9.773	9.261	−1.798 to 36.444
Pre	${\beta }_{32}$	46.667	9.113	11.518	22.222 to 67.500
Sex	${\kappa }_2$	0.731	0.087	0.086	0.577 to 0.923
Pre → Dif	${\beta }_{32}$	−0.704	0.148	0.140	−0.986 to −0.444
Sex → Dif	${\gamma }_{32}$	38.058	8.493	10.971	17.875 to 61.804
Sex → pre	${\gamma }_{22}$	17.895	10.661	12.550	−6.316 to 42.667
Variance (Pre)		581.377	161.245	115.406	406.566 to 904.872
Variance (Sex)		0.197	0.055	0.039	0.130 to 0.250
Variance (Dif)		332.904	92.331	71.068	232.649 to 505.513

SE: standard error; CI: confidence interval.

Discussion

Studies of palliative care often adopt single-arm study designs to accommodate patient conditions that preclude randomization. However, single-arm studies offer limited estimates of causal relationships between interventions and effects. Hence, study designs that meet the requirement of evidence-based palliative care have recently been proposed.^20,21 In this study, we propose a method for effective use of data from single-arm studies by modelling relationships among covariates.

In one-group pre–post studies, intervention effects are usually evaluated using paired-t tests.^22,23 In contrast, the present analytical method uses ANCOVA models that exclude group effects and enables clinical interpretation using SEM to evaluate the effects of palliative care interventions on QOL. Critically, this model accommodates associations among covariates and permits extraction of factors that directly or indirectly influence efficacy indicators, thus enabling consideration of randomization designs in which these factors serve as allocation factors. However, future studies are required to assess the advantages and shortcomings of this approach in comparison with conventional ad hoc allocation factor selection methods.

During modelling of associations among covariates using SEM, trial calculations of bias-free sample sizes are possible as in analyses using ANCOVA models, which are used to estimate intervention effects (not affected by covariates) and their dispersions. However, the present method enables estimation of intervention effects by adjusting the influences of covariates on efficacy indicators. Moreover, HPCT interventions and covariate effects were estimated even during modelling of simultaneous influences of multiple covariates. Thus, the proposed method may serve as a novel analytical tool for one-group pre–post studies that are performed under study-limiting conditions. Data analyses using SEM usually require large sample sizes.²⁴ Thus, application of SEM to small samples may lead to unstable estimates of model parameters. Thus, unusually large parameter estimates require cautious interpretation.