Skip to main content
BMJ Open logoLink to BMJ Open
. 2021 Mar 16;11(3):e042268. doi: 10.1136/bmjopen-2020-042268

Occurrence and timely management of problems requiring prompt intervention among Indigenous compared with non-Indigenous Australian palliative care patients: a multijurisdictional cohort study

John A Woods 1,, Judith M Katzenellenbogen 2,3, Kevin Murray 2, Claire E Johnson 4,5,6,7, Sandra C Thompson 1
PMCID: PMC7970279  PMID: 33727263

Abstract

Objectives

Anticipation and prompt relief of symptoms among patients with a life-limiting illness is a core element of palliative care. Indigenous Australians commonly encounter cultural barriers in healthcare that may impair outcomes. The Palliative Care Outcomes Collaboration collects patient care data for the purposes of continuous quality improvement and benchmarking, with each recorded care episode divided into phases that reflect a patient’s condition. We aimed to investigate differences between Indigenous and non-Indigenous patients in the occurrence and duration of ‘unstable’ phases (which indicate unanticipated deterioration in a patient’s condition or circumstances), and determine attainment of the relevant benchmark (resolution of unstable phases in ≤3 days in 90% of cases) for both groups.

Design

Cohort study.

Setting

Australia-wide hospital-based and community-based specialist palliative care (1 January 2010 to 30 June 2015).

Participants

139 556 (1502 Indigenous and 138 054 non-Indigenous) adult patients.

Outcome measures

Indigenous and non-Indigenous patients were compared on (1) the risk of a phase being categorised as unstable, (2) the duration of unstable phases, and (3) the risk of unstable phases being prolonged (>3 days). Crude and adjusted estimates were produced from three-level robust Poisson regression and complementary log-log discrete time survival models.

Results

Unstable phases occurred with similar frequency overall among Indigenous and non-Indigenous patients (adjusted relative risks 1.06; 95% CI 1.00 to 1.11; not significant after correction for multiple comparisons). The duration and risk of prolongation of unstable phases were similar in both patient groups, with no significant differences evident among subgroups. The benchmark was not met for either Indigenous or non-Indigenous patients (unstable phase duration >3 days in 24.3% vs 25.5%; p=0.398).

Conclusions

Despite well-documented shortcomings of healthcare for Indigenous Australians, there is no clear evidence of greater occurrence or prolongation of unanticipated problems among Indigenous patients accessing specialist palliative care services in hospital or the community.

Keywords: adult palliative care, quality in health care, public health


Strengths and limitations of this study.

  • The study is based on a large multijurisdictional data set that provides detailed information on patients with a broad range of life-limiting illnesses and who have been cared for in diverse settings.

  • The analysis incorporated multiple regression methods accounting for the three-level structure of the data and the discrete basis for time measurement.

  • The data are not entirely representative of specialist palliative care services across Australia.

  • There is the potential for bias in the timeliness with which the onset of unanticipated problems has been ascertained.

  • Further, there is the potential for residual confounding by the type and stage of disease, although not in a manner likely to cast doubt on the conclusions.

Introduction

A principal aim of palliative care is the prevention and alleviation of suffering among persons who have a life-limiting illness.1 Accordingly, a core task of palliative care clinicians is anticipation of new, unanticipated problems such as ‘breakthrough’ symptoms in order to reduce their occurrence when possible, along with early detection and prompt intervention to curtail and minimise the impact of such problems when they occur. The course of a life-limiting illness, including the occurrence of unpredicted events, is potentially influenced by the quality of care as well as by the nature of the underlying disease, comorbidities and attributes of the patient in her or his social context.2–4 In turn, the effectiveness of care is dependent on timely access to appropriate services and the skills of clinicians. The quality of communication between clinicians and patients along with their carers is of particular importance.5

In Australia, Aboriginal and Torres Strait Islander (hereafter respectfully referred to as Indigenous) people experience life-limiting illnesses at younger average ages than other Australians. Furthermore, Indigenous people more often encounter logistical and cultural barriers to high-quality healthcare.6–8 Ineffective communications between health service providers and Indigenous patients may impair healthcare intervention and health outcomes.9 10 Importantly, personal or collective historical adverse healthcare experiences may engender reluctance among Indigenous patients to disclose symptoms such as pain.11 Although there is a growing literature on the particular needs and experiences of Indigenous Australians with life-limiting illnesses,12 13 there have been no large-scale studies of healthcare system performance in addressing unanticipated problems in the palliative care context.

The Australian Palliative Care Outcomes Collaboration (PCOC) is a national continuous quality improvement initiative funded by the Commonwealth Department of Health.14 The Collaboration collects standardised patient care data from participating palliative care services for the purposes of analysis and regular reporting, benchmarking and research.14 15 Established in 2005, PCOC collected data from services accounting for about 80% of specialist palliative care provision nationwide when data for the current study were collected. (The purview has since expanded to encompass non-specialist services providing palliative care.) The PCOC data set is structured hierarchically, comprising the records of one or more ‘episodes’ of care provided to each patient by a participating service, with each episode made up of one or more ‘phases’ that are categorised to delineate a patient’s changing condition and care needs.15 16 An ‘unstable’ phase is defined by the recognition of either an unanticipated problem or an unanticipated worsening of a patient’s condition or circumstances (eg, family/carer issues impacting on care) that requires an urgent intervention or change in the management plan. The resolution of an unstable phase is defined by the institution of a care plan to deal with the unanticipated problem or when death is likely within days.15 One of the PCOC quality-of-care benchmarks addresses timeliness of intervention in such circumstances, stipulating that ‘90% of unstable phases must last for 3 days or less’.15

The current study was conducted as part of a research project using the multijurisdictional PCOC data set to investigate the quality of care provided to Indigenous Australians with life-limiting illnesses, with a particular focus on the PCOC benchmarks.15 We hypothesised that inequity in the care of Indigenous compared with non-Indigenous patients may be reflected in differences between the two groups in the occurrence and resolution of unstable phases. Our specific objectives were to compare Indigenous with non-Indigenous patients with respect to (1) the proportion of phases that were unstable, (2) the duration of unstable phases, and (3) the proportion of unstable phases that were prolonged (ie, >3 days), reflecting comparative attainment of the benchmark.

Methods

Study design and patient population

This was a retrospective cohort study using the longitudinal, hierarchically structured PCOC data set (details available at https://ahsri.uow.edu.au/pcoc/4researchers/dataset/index.html). Palliative care phase was the unit of observation and the Indigenous identifier of the patient was the principal explanatory variable of interest. The study comprised the phase records (n=448 799) from all episodes of care provided to adult patients (>18 years at entry to care) during the period 1 January 2010 to 30 June 2015.

Definition of palliative care phase as the unit of observation

In PCOC data, palliative care phases experienced by patients are categorised non-sequentially as stable, unstable, deteriorating or terminal.15 For the present study, phase types were recoded in binary form as ‘unstable’ or ‘other’, following exclusion of the 2.6% of records representing bereavement care for the family or carers after a patient’s death and the 0.01% of records with a missing value for phase type. The PCOC data set includes start and end dates for each phase, allowing phase length calculation in days but not hours.

Study measures

The three primary outcome measures examined were the occurrence of unstable phases (as a proportion of all phases), unstable phase duration and the proportion of unstable phases that were prolonged. The relevant PCOC benchmark, based on ‘time in the unstable phase’,15 was operationalised by defining a prolonged unstable phase as one lasting >3 days.

The main exposure variable investigated was the Indigenous identifier of patients, categorised in binary form as follows. Any patient identified as Aboriginal, Torres Strait Islander or both at entry to care was categorised as Indigenous. All other patients with a value recorded for this variable were categorised as non-Indigenous. The remaining patients (n=4980; 3.5%) with a missing value for the Indigenous identifier were excluded from the analyses.

In order to rule out biases arising from differences between Indigenous and non-Indigenous patients in episode structure (type and order of phases), we investigated the proportion of phase types in both patient groups, both overall and immediately preceding and following unstable phases.

Other covariates from PCOC data included in the analysis were sex, age in years at episode start, principal diagnosis, remoteness of residence, care setting, year of occurrence (recorded at phase onset) and selected clinical status indicators. PCOC categorises patient diagnoses according to 29 organ/system-specific codes,15 which are grouped into the binary categories ‘cancer’ or ‘other’. This binary grouping of principal diagnosis was adopted for the analysis because protocols for high-quality palliative care for patients with non-neoplastic life-limiting illnesses are less firmly established than for those with cancer.17 Remoteness of a patient’s residence at entry to care is categorised according to the five-tiered Australian Statistical Geography Standard.18 For the current study the five categories were collapsed into three: (1) major cities; (2) inner regional; and (3) outer regional, remote or very remote. The setting in which an episode of care is provided is categorised as (1) overnight hospital/palliative care service admission, (2) hospital day admission/outpatient attendance, or (3) community. Validated clinical status indicators considered for the analyses were (1) the Australia-modified Karnofsky Performance Status (AKPS) scale, a clinician-rated assessment of a patient’s physical abilities,19 and (2) the Symptom Assessment Scale (SAS), which comprises symptom-related distress scores from seven domains (breathing, bowel problems, appetite problems, pain, insomnia, nausea and fatigue) rated by the patient (or by the carer if the patient is unable).20 These clinical status indicators, recorded at the start of each phase, were modelled as numeric variables with each of the seven SAS items modelled as a separate covariate. Finally, the study period (during which substantial improvement in benchmark attainment was evident from preliminary inspection) was divided into two equal intervals: first half (1 January 2010 to 30 September 2012) and second half (1 October 2012 to 30 June 2015).

Statistical analysis

Approach to missing data

Comprehensive data quality assessment and cleaning for the research project instituted prior to the study incorporated the recoding of a small number of implausible values as missing.21 Analytical results are based on records with non-missing values across all modelled covariates. Among included records (with a non-missing patient Indigenous identifier), the proportion of records with at least one missing value was ~14%, falling to ~3% when clinical status indicator scores were disregarded (online supplemental table 1). The main analyses therefore excluded the clinical status indicators, with sensitivity analyses that included these indicators undertaken for selected models to evaluate the potential impact of their exclusion.

Supplementary data

bmjopen-2020-042268supp001.pdf (229.1KB, pdf)

Characteristics of patients and their care

Selected characteristics of Indigenous and non-Indigenous patients and their care were compared.

Also investigated was the potential for bias arising from differences between the two groups in the structure of patients’ ‘journeys of care’ (number of care episodes experienced, number of phases, mix of phase types overall and immediately preceding and following an unstable phase).

Analyses of unstable phase occurrence

Unstable phase occurrence among Indigenous compared with non-Indigenous patients was investigated in three-level (patient, episode and phase) random intercept models, with relative risks (RR) and 95% CIs estimated using Poisson regression with robust variance structure.22 23 Given that occurrence of an unexpected event (resulting in recording of an unstable phase) at the outset of a care episode may influence the setting of care, we undertook separate analyses for first and subsequent phases. Crude and adjusted estimates of RR were calculated. Adjusted model 1a incorporated the following covariates: age (modelled as a continuous variable), sex, principal diagnosis (modelled in binary form; ie, cancer vs other), remoteness and calendar year. Model 1b (subsequent phases only) comprised all covariates listed for model 1a plus setting. Finally, subgroup analyses were undertaken to estimate adjusted RRs for the Indigenous identifier across each stratum of the other covariates.

Analyses of phase duration

Time-to-event analyses were performed in order to investigate overall differences between Indigenous and non-Indigenous patients in the duration of each unstable phase (ie, the time from the phase start date until the phase end date (indicating institution of the care plan)). Software constraints precluded extension of the Cox proportional hazards model to three-level hierarchical data nesting,24 and the hazard function assumptions of multilevel parametric (eg, Weibull distribution) survival models could not be fitted to the data.25 Further, considering that unstable phase duration in PCOC data is ascertainable only from discretised start and end dates, it was not appropriate to model time as a continuous variable.26 Accordingly, the time-to-event analyses were based on multilevel discrete time survival models incorporating a complementary log-log link to estimate the log hazard.26 Unstable phases ending in death (3.6%: Indigenous 2.7%, non-Indigenous 3.6%) were right censored.27 A further small minority of unstable phases of extreme duration (>30 days: 1.2% of phases from both Indigenous and non-Indigenous patient groups) prevented model convergence and were excluded. Violation of the proportional hazards assumption by covariates considered for multivariable regression modelling was investigated by preliminary modelling of interactions between the time variable and each covariate. Covariates demonstrated to have duration-dependent effects were stratified in subgroup analyses rather than incorporated into the regression models.

Analyses of unstable phase prolongation

Unstable phases that ended in death (n=3543 (3.6% of total; 2.7% of those of Indigenous patients and 3.6% of non-Indigenous)) were excluded from analyses of prolongation. In order to investigate the achievement of the PCOC benchmark, raw percentages of unstable phases that were prolonged (>3 days) were calculated for the total sample as well as Indigenous and non-Indigenous patients separately. Further, as with the analysis of unstable phase occurrence and duration (above), RRs of prolongation were estimated in three-level robust Poisson models for each independent variable: Indigenous identifier, age group (dichotomised as <65 years vs ≥65 years), sex, principal diagnosis, remoteness, setting and calendar year. For the Indigenous identifier, adjusted RRs were also estimated. The principal adjusted model incorporated the following covariates: age (modelled as a continuous variable), sex, principal diagnosis (modelled in binary form; ie, cancer vs other), remoteness, calendar year and setting. A sensitivity analysis comprised all covariates listed for the principal model plus the clinical indicators (ie, AKPS and the seven SAS domains). Finally, subgroup analyses were undertaken to estimate adjusted RRs for the Indigenous identifier for each stratum of the other model 1 covariates.

Additionally, crude differences in occurrence and prolongation of unstable phases were examined in relation to the setting of care and calendar time, in order to contextualise the comparisons between the phases of Indigenous and non-Indigenous patients.

P values were corrected for false discovery rate using the Benjamini-Hochberg method28 applied across all 46 analyses in the study, with a corrected value p<0.0022 considered statistically significant. All analyses were conducted using Stata V.15.1 (StataCorp, College Station, Texas, USA).

Patient and public involvement

This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Results

Study population

The sample comprised 139 556 patients, 1502 Indigenous and 138 054 non-Indigenous (table 1), with a further 4980 patients lacking an Indigenous identifier excluded from the study. Indigenous patients were a decade younger on average at entry to palliative care than non-Indigenous patients (63.3 vs 73.0 years), and a higher proportion were female (51.5% vs 46.4%). Similar proportions of patients in both groups had a cancer as the principal diagnosis (78.2% vs 78.9%). Indigenous patients more often resided outside major cities at entry to care (46.1% vs 23.0%), and more often received their care as inpatients admitted overnight (63.0% vs 57.8%). The proportion of Indigenous patients entering care over the study period did not change significantly while the proportion of patients with a missing identifier diminished progressively from 4.9% (2010) to 2.0% (2015). The proportions of phase categories among Indigenous and non-Indigenous patients were similar (table 1).

Table 1.

Characteristics of patients and their episodes and phases, by Indigenous identifier

Patients Indigenous Non-Indigenous
n=1502 n=138 054
Baseline characteristics
Age (years) at entry to care, mean (SD) 63.3 (14.3) 73.0 (13.6)
Age group at entry, n (%)
 <65 years 801 (53.3) 34 520 (25.0)
 ≥65 years 698 (46.5) 103 452 (74.9)
Sex, n (%)
 Female 773 (51.5) 64 038 (46.4)
 Male 726 (48.3) 73 955 (53.6)
Principal diagnosis, n (%)
 Cancer 1174 (78.2) 108 921 (78.9)
 Other 304 (20.2) 27 424 (19.9)
Remoteness of residence at entry, n (%)
 Major cities 809 (53.9) 106 361 (77.0)
 Inner regional 337 (22.4) 21 965 (15.9)
 Outer regional 242 (16.1) 6963 (5.0)
 Remote/very remote 88 (5.9) 654 (0.5)
Episodes Indigenous Non-Indigenous
n=2258 n=191 784
Setting of care, n (% by setting)
 Inpatient overnight admission 1423 (63.0) 110 930 (57.8)
 Hospital ambulatory (day admission, OP) 67 (3.0) 2916 (1.5)
 Community 768 (34.0) 77 938 (40.6)
Phases Indigenous Non-Indigenous
n=4878 n=443 921
Phase type, n (% by type)
 Stable 1509 (30.9) 136 454 (30.7)
 Unstable 1156 (23.7) 96 776 (21.8)
 Deteriorating 1568 (32.1) 148 802 (33.5)
 Terminal 645 (13.2) 61 889 (13.9)
Study period of phase start date, n (% within half)
 Earlier half (1 January 2010 to 30 September 2012) 2015 (41.3) 176 234 (39.7)
 Later half (1 October 2012 to 30 June 2015) 2863 (58.7) 267 687 (60.3)

Percentage totals may not add up to 100% due to missing data.

OP, outpatient.

The number of care episodes, phases overall and of each type of phase was similar among Indigenous and non-Indigenous patients (online supplemental table 2a). Further, the mix of phase types was similar in the two patient groups, both immediately preceding and immediately following unstable phases (online supplemental table 2b, c).

Outcomes among Indigenous compared with non-Indigenous patients

Occurrence of unstable phases

Overall, phases were less often categorised as unstable when care was provided in hospital day admission/outpatient settings (crude RR 0.82; 95% CI 0.78 to 0.86) and community settings (crude RR 0.52; 95% CI 0.51 to 0.52) compared with inpatient settings. Differences between settings in unstable phase occurrence varied with the phase sequence within an episode of care. First phases were less often categorised as unstable when care was provided in hospital day admission/outpatient settings or community settings compared with inpatient settings, with crude RRs respectively 0.47 (95% CI 0.44 to 0.50) and 0.20 (95% CI 0.19 to 0.20). The associations were reversed among subsequent phases, which were more often categorised as unstable when care was provided in hospital day admission/outpatient settings (crude RR 1.85; 95% CI 1.69 to 2.01) and community settings (crude RR 1.34; 95% CI 1.31 to 1.37) compared with inpatient settings. The occurrence of unstable phases diminished during the study period; phases were less often categorised as unstable during the second half of the study period (1 October 2012 to 30 June 2015) compared with the first half (1 January 2010 to 30 September 2012) (crude RR 0.79; 95% CI 0.78 to 0.80).

The crude RR of a phase being unstable versus all other types of phase was significantly higher among Indigenous compared with non-Indigenous patients even after correction for multiple comparisons (RR 1.09, p=0.002, 95% CI 1.03 to 1.15) (table 2). However, there were no significant crude differences between phases of the two patient groups when phases were stratified by order within an episode as either first or subsequent. After adjustment and correction for multiple comparisons, there were no significant differences across the total sample in RRs of a phase being unstable versus all other types of phase among Indigenous compared with non-Indigenous patients. In the subgroup analyses, the adjusted RR of a phase being unstable among Indigenous compared with non-Indigenous patients was significantly different (after correction) only in patients living in major cities (RR 1.13, p<0.001, 95% CI 1.05 to 1.20).

Table 2.

Relative risks for unstable phase occurrence according to Indigenous identifier, overall and by sequence within an episode of care and subgroups

RR P value 95% CI
Crude estimates
 All phases 1.09 0.002* 1.03 to 1.15
 First phases 1.08 0.031 1.01 to 1.16
 Subsequent phases 1.04 0.404 0.95 to 1.14
Adjusted estimates: total sample
 Model 1a† all phases 1.06 0.038 1.00 to 1.11
 Model 1a† first phases 1.08 0.038 1.00 to 1.15
 Model 1a† subsequent phases 1.00 0.971 0.92 to 1.10
 Model 1b‡ (subsequent phases only) 1.02 0.732 0.93 to 1.12
Adjusted§ estimates: selected subgroups
Age group
 <65 years at episode start 1.04 0.269 0.97 to 1.11
 ≥65 years at episode start 1.11 0.008 1.03 to 1.20
Principal diagnosis category
 Cancer 1.08 0.005 1.02 to 1.14
 Other 0.83 0.091 0.67 to 1.03
Remoteness
 Major cities 1.13 <0.001* 1.05 to 1.20
 Inner regional 1.01 0.794 0.92 to 1.12
 Outer regional/remote/very remote 0.87 0.052 0.75 to 1.00
Care setting¶
 Inpatient 1.06 0.020 1.01 to 1.11
 Community 0.92 0.175 0.81 to 1.04
Study period
 First half (1 January 2010 to 30 September 2012) 1.06 0.075 0.99 to 1.14
 Second half (1 October 2012 to 30 June 2015) 1.05 0.211 0.97 to 1.13

*Significant after correction for multiple comparisons (Benjamini–Hochberg method).28

†Model 1a: adjusted for age (years: continuous/quadratic), sex, broad diagnosis, remoteness (three categories), calendar date (phase start: continuous/linear).

‡Model 1b (subsequent phases only): adjusted for setting in addition to the covariates listed for model 1a.

§Each subgroup analysis incorporated covariates from model 1a above other than the variable of stratification.

¶Hospital day admission/outpatient setting was excluded from stratification because small numbers interfered with model convergence.

RR, relative risk.

Duration and prolongation of unstable phases

Unstable phases more often had a duration of greater than 3 days when care was provided in hospital day admission/outpatient settings (crude RR 2.11; 95% CI 1.99 to 2.15) and community settings (crude RR 1.57; 95% CI 1.53 to 1.61) compared with inpatient settings. Just as occurrence of unstable phases diminished across the study period, so did prolongation of unstable phases (>3 days); phases were less often prolonged during the second half of the study period compared with the first half (crude RR 0.55; 95% CI 0.548 to 0.56).

There were no significant differences between the unstable phases of Indigenous and non-Indigenous in either the HR for duration or the RR of prolongation, in either crude estimates (figure 1) or adjusted estimates for the total sample or selected subgroups (table 3), after correction for multiple comparisons.

Figure 1.

Figure 1

Crude time to completion of unstable phases among Indigenous compared with non-Indigenous patients, stratified by halves of study period. (Only unstable phases completed by institution of a care plan within 30 days are included. Phases ending in death were censored.)

Table 3.

Comparisons by Indigenous identifier of unstable phase duration and prolongation (>3 days), overall and by subgroups

HRs for conclusion of unstable phase through institution of care plan
HR P value* 95% CI
Crude estimates
 Indigenous identifier 1.03 0.532 0.94 to 1.13
Adjusted estimates: Total sample (principal model)
1.04 0.435 0.94 to 1.14
Adjusted estimates: Total sample (principal model+additional adjustment for baseline clinical status)
1.02 0.646 0.93 to 1.13
Adjusted estimates: Selected subgroups§
Age group
 <65 years at episode start 1.03 0.686 0.90 to 1.17
 ≥65 years at episode start 1.05 0.495 0.91 to 1.21
Principal diagnosis category
 Cancer 1.03 0.589 0.93 to 1.13
 Other 1.18 0.359 0.83 to 1.70
Remoteness
 Major cities 1.07 0.302 0.94 to 1.21
 Inner regional 0.85 0.071 0.70 to 1.01
 Outer regional/remote/very remote 1.12 0.351 0.88 to 1.42
Care setting¶
 Inpatient 1.12 0.024 1.02 to 1.25
 Community 0.85 0.167 0.68 to 1.07
Study period
 First half (1 January 2010 to 30 September 2012) 1.09 0.236 0.95 to 1.24
 Second half (1 October 2012 to 30 June 2015) 1.00 0.944 0.89 to 1.14
Relative risks for an unstable phase being prolonged >3 days
RR P value* 95% CI
Crude estimates
 Indigenous identifier 1.03 0.562 0.94 to 1.13
Adjusted estimates: Total sample (principal model)
1.02 0.716 0.92 to 1.13
Adjusted estimates: Total sample (principal model+additional adjustment for baseline clinical status)
1.01 0.903 0.90 to 1.13
Adjusted estimates: (Selected subgroups)§
Age group
 <65 years at episode start 1.04 0.583 0.91 to 1.19
 ≥65 years at episode start 0.93 0.328 0.79 to 1.08
Principal diagnosis category
 Cancer 0.98 0.722 0.88 to 1.09
 Other 1.07 0.739 0.71 to 1.61
Remoteness
 Major cities 1.00 0.952 0.89 to 1.13
 Inner regional 1.09 0.368 0.91 to 1.31
 Outer regional/remote/very remote 0.75 0.187 0.50 to 1.15
Care setting¶
 Inpatient 0.94 0.334 0.84 to 1.06
 Community 1.12 0.237 0.93 to 1.34
Study period
 First half (1 January 2010 to 30 September 2012) 1.00 0.989 0.88 to 1.13
 Second half (1 October 2012 to 30 June 2015) 1.04 0.650 0.88 to 1.22

*No p values were significant after correction for multiple comparisons (Benjamini–Hochberg method).28

†Principal model covariates: age (continuous, quadratic), sex, principal diagnosis category, setting, remoteness (three categories) and calendar date (phase start: continuous/linear).

‡Principal model plus additional covariates: start-of-phase Symptom Assessment Scale (seven domains; continuous); start-of-phase Australia-modified Karnofsky Performance Status scale (continuous).

§Each subgroup analysis incorporated covariates from model 1a above other than the variable of stratification. The following covariates violated the proportional hazards assumption: remoteness, care setting and study period. For correspondence with other results tabulated in the paper, the HR data were also subgrouped by age and principal diagnosis category.

¶Hospital day admission/outpatient setting was excluded from stratification because small numbers interfered with model convergence.

HR, hazard ratio; RR, relative risk.

The benchmark of resolution of unstable phases (excluding those ending in death) in ≤3 days in 90% of instances was not achieved for either Indigenous (70.8% of unstable phases ≤3 days) or non-Indigenous patients (69.8%) (p=0.498). Despite some improvement over time, completion within 3 days during the second half of the study period (1 October 2012 to 30 June 2015) was still attained only in 78.2% and 78.0% of unstable phases experienced by Indigenous and non-Indigenous groups, respectively (p=0.945) (figure 1).

Discussion

In this longitudinal, multijurisdictional study of unanticipated problems experienced by Indigenous compared with non-Indigenous Australian patients during specialist palliative care, we found no overall differences in the proportions of unstable palliative care phases or the proportion of unstable phases that were prolonged between the two groups, after appropriate adjustment. These generally null findings were robust to sensitivity analyses that compared exclusion with inclusion in regression models of the clinical status covariates, which had a substantial proportion of missing values. After correction for multiple comparisons, a significantly higher risk of a phase being unstable among Indigenous patients was evident among those living in major cities but in no other subgroup. However, this association was not predicted a priori, was modest in magnitude and may have been influenced by unmeasured confounding. The pertinent PCOC benchmark stipulating that resolution of an unstable phase be attained within 3 days in ≥90% instances was not met for either patient group during the study period. However, there was progressive decrease across the study period in the occurrence of unstable phases and the proportion of these phases that were longer than 3 days.

Uncertainty contributes to distress and diminishes quality of life among people with life-limiting illnesses.29 However, there has been little research specifically on the occurrence and timeliness of interventions to ameliorate unpredicted problems of palliative care patients or exploring these outcomes among patients from disadvantaged populations. PCOC data provide opportunity to investigate this topic on a large scale. Indigenous Australians experience disparities in health service provision and outcomes across their lives, and this is likely to occur within their journey with a life-limiting illness.30 31 In the case of cancer (the principal diagnosis of the majority of patients receiving specialist palliative care), Indigenous Australians have lower uptake of screening,32 and they commonly experience substantial delays in diagnosis.33 34 Additionally, they have more limited access to definitive treatments,32 35 have lower uptake of hospitalisations36 and frequently have unmet supportive care needs.37 A far lower proportion of Indigenous compared with non-Indigenous patients receive specialist palliative care at the end of life.38 In contrast to these ‘upstream’ care disparities, the findings of the current study provide reassurance in relation to one important aspect of the care provided to Indigenous patients who access specialist palliative care services.

Given the acknowledged importance of cultural competence of clinicians in dealing with patient symptoms39 and findings from previous qualitative research that communication difficulties may disadvantage Indigenous patients in the context of life-limiting illness,9 40 the near equivalence between the two groups across the measured outcomes is encouraging. It suggests that at least for those Indigenous patients accessing care in services that report data to PCOC, challenges in communication between them (or their carers) and service providers do not impact substantially on prevention of and intervention for unpredictable acute problems. These findings are particularly reassuring in consideration of the 10-year differential in average patient age (concordant with the well-known life expectancy gap), given that younger patients may present distinctive challenges for services geared predominantly to dealing with older patients.41

It is a testament to PCOC as a large-scale continuous quality initiative that a consistent and substantial reduction occurred across the 5.5-year study period in both the occurrence of unstable phases (both at the outset and later during episodes of care) and their undue prolongation. Based on the summary data available in the PCOC’s regular public domain reporting,42 the trend has continued since the data for the present study were provided, reflecting ongoing efforts to address this important concern. This marked temporal trend in outcomes did not influence the similar outcomes among Indigenous and non-Indigenous patients, suggesting that both groups have benefited to a comparable extent from the progressive improvement.

The lower proportion of unstable phases in outpatient and community settings compared with inpatient setting as the reference was accentuated in first phases and reversed in subsequent phases (table 2), vindicating our stratification of occurrence by phase order. From the unadjusted models, care provided in inpatient hospital settings was associated with a much higher risk of unstable phases at the commencement of a care episode, with second and subsequent unstable phases more common in outpatient and community settings. These findings are unsurprising as identification of unanticipated or increasingly complex problems (corresponding with recognition of an unstable phase) in community settings frequently results in an inpatient admission. Conversely, clinical monitoring of patients is likely to be reduced in outpatient and community settings, where it may rely more on proactive communication from patients and their families.

Strengths and limitations

PCOC has assembled an information-rich longitudinal data set that allows for large-scale investigation of health service quality in Australian palliative care, including studies of care equity. For the small subgroup of Indigenous patients receiving care, we used several complementary modelling strategies to deal with the limitations of available information.

The timing of onset of unstable phases will likely be influenced by clinicians’ recognition of ‘unanticipated problems’ and clinician–patient (or clinician–carer) communication. Therefore, there is a potential for a varying ‘lag’ period in recognition of an unanticipated problem, which might plausibly differ on average among Indigenous compared with non-Indigenous patients. Such a lag is not inherently ascertainable from the recorded start-of-phase date. However, we reasoned that delays in detection may be reflected in heightened problem severity at phase start as perceived by the patient and/or by greater deterioration in functional status, and thereby may be captured indirectly in start-of-phase responses to the SAS or AKPS questionnaires, respectively. In this regard, our unstable phase duration models incorporated modelling with and without these items as covariates. However, we recognise that patients’ responses to the SAS items may be culturally modulated and could find no evidence that the SAS has been specifically validated for use by Indigenous Australians, raising the possibility of differential under-reporting of symptom severity between the two patient groups.

Despite adjustment in multiple regression models to account for the diverse natural histories of underlying principal diagnoses, some degree of residual confounding by disease process may have occurred, particularly as the data did not include stage of disease at entry to care. Indigenous compared with non-Indigenous Australians experiencing a life-limiting illness tend (particularly in the case of cancers) to be diagnosed at a later stage, and with a disease type having an inherently poorer prognosis,43 and are also more likely to have comorbidities.44 Consequently, any residual confounding by disease stage or aggressiveness would be expected to increase the occurrence and duration of unanticipated problems among Indigenous patients. Therefore, confounding on this basis is unlikely to account for unstable phase occurrence and duration being no greater than among non-Indigenous patients. Although most specialist palliative care services nationwide provided data to PCOC throughout the study period, differential participation inevitably means that results are not representative of all services, with some having better or poorer performance than those that participated. No services from the Northern Territory provided PCOC data during our study time frame38 (although several have since become involved).45 This jurisdiction is home to an estimated 9% of Indigenous Australians,46 a high proportion of whom live in remote or very remote areas, and many preferentially speak an Indigenous language rather than English. Consequently, the study excluded a subgroup of Indigenous patients at risk of especially heightened barriers to high-quality healthcare. Further, linkage of data from individuals cared for by more than one participating service or with other health data sets was not available so it was not possible to identify patients who received care from more than one service, or to incorporate external information such as cancer registry data on date and stage of diagnosis.47 Finally, the data did not permit direct investigation of cultural safety (and how this may have influenced the outcomes) or of care appraisal from an Indigenous worldviews perspective, both of which are critical to improving health service performance for Indigenous people.48 49 Incorporation of a cultural appraisal tool50 into PCOC data collection for Indigenous patients would be an innovation worthy of consideration.

Conclusion

Australian Indigenous patients receiving specialist palliative care do not experience unanticipated problems or undue prolongation of such problems substantially more often than do non-Indigenous patients. While these findings are encouraging, caution is required in their interpretation. Future refinement of design and analytical methods through data linkage may offer opportunities for further systematically exploring palliative care outcomes and provide greater insights into the experiences of Indigenous patients receiving palliative care.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

PCOC is funded under the National Palliative Care Program by the Australian Government Department of Health. PCOC Statistician/Data Manager Mr Samuel Allingham and PCOC National Director Ms Karen Quinsey provided helpful advice for the researchers’ investigation of data quality and data cleaning. Mr Allingham prepared the data set and, upon request, provided refined episode sequence data based on lookback to the data set’s inception in 2005. PCOC National Director Dr Barbara Daveson and National Quality and Education Manager Ms Sabina Clapham provided helpful comments on the text. Dr Kou Kou of Cancer Council Queensland provided advice on Stata coding for the multilevel discrete time survival models.

Footnotes

Contributors: JAW designed the study, performed the analysis and drafted the manuscript. JMK and KM advised on data analysis and interpretation. CEJ co-conceived the research project and assisted in data acquisition and data interpretation. SCT co-conceived the research project and contributed to data interpretation. All authors critically revised the manuscript for important intellectual content and approved the final version.

Funding: This work was supported by the Australian National Health and Medical Research Council grant number 1133793 (Postgraduate Research Scholarship, JAW), a University of Western Australia Athelstan and Amy Saw Top-Up Scholarship (JAW) and a University Postgraduate Award and Australian Government Research Training Program Scholarship at The University of Western Australia (JAW).

Disclaimer: The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: CEJ is a PCOC chief investigator and receives funding to support the collection of data from palliative care services that participate in PCOC. CEJ is a member of the PCOC Executive Directors Group that approves release of data to researchers, but was excluded from the approval process in this instance in view of the potential for competing interest.

Patient consent for publication: Not required.

Ethics approval: Ethics approval for the study was provided by the Western Australian Aboriginal Health Ethics Committee (reference 616) and The University of Western Australia Human Research Ethics Committee (reference RA/4/1/7441). The bodies that granted permission determined that individual consent of subjects was not required; data for analysis were anonymous and only aggregated data are presented.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data availability statement: Data may be obtained from a third party and are not publicly available. Restrictions apply to the availability of the data that support the findings of this study, which were used under licence, and so are not publicly available. Under the User Agreement for Release of PCOC Data, the authors have undertaken (21 June 2015) ‘not to disclose the Confidential Information [ie, de-identified individual patient records] in any released output’. Non-identifiable extracts of data from the PCOC longitudinal database can be made available for use in research. Researchers must apply in writing and provide detailed project descriptions for approval by the PCOC Executive Directors Group: https://ahsri.uow.edu.au/pcoc/4researchers/accessingdata/index.html. The authors of the current study accessed the data by this means. Details of variables included in the PCOC data set are available online in the PCOC Data Dictionary & Technical Guidelines: http://tinyurl.com/PCOC-V3-DataDictionary.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

  • 1.World Health Organization . Palliative care, 2020. Available: https://www.who.int/health-topics/palliative-care [Accessed January 18, 2021].
  • 2.Portz JD, Kutner JS, Blatchford PJ, et al. High symptom burden and low functional status in the setting of multimorbidity. J Am Geriatr Soc 2017;65:2285–9. 10.1111/jgs.15045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Santos Salas A, Watanabe SM, Tarumi Y, et al. Social disparities and symptom burden in populations with advanced cancer: specialist palliative care providers' perspectives. Support Care Cancer 2019;27:4733–44. 10.1007/s00520-019-04726-z [DOI] [PubMed] [Google Scholar]
  • 4.Noble S, Noble M. Emergencies in palliative care. Medicine 2015;43:722–5. 10.1016/j.mpmed.2015.09.010 [DOI] [Google Scholar]
  • 5.Seccareccia D, Wentlandt K, Kevork N, et al. Communication and quality of care on palliative care units: a qualitative study. J Palliat Med 2015;18:758–64. 10.1089/jpm.2014.0408 [DOI] [PubMed] [Google Scholar]
  • 6.Kelly J, Dwyer J, Willis E, et al. Travelling to the City for hospital care: access factors in country Aboriginal patient journeys. Aust J Rural Health 2014;22:109–13. 10.1111/ajr.12094 [DOI] [PubMed] [Google Scholar]
  • 7.Shahid S, Finn LD, Thompson SC. Barriers to participation of Aboriginal people in cancer care: communication in the hospital setting. Med J Aust 2009;190:574–9. 10.5694/j.1326-5377.2009.tb02569.x [DOI] [PubMed] [Google Scholar]
  • 8.Whop LJ, Garvey G, Lokuge K, et al. Cancer support services--are they appropriate and accessible for Indigenous cancer patients in Queensland, Australia? Rural Remote Health 2012;12:2018. [PubMed] [Google Scholar]
  • 9.Shahid S, Durey A, Bessarab D, et al. Identifying barriers and improving communication between cancer service providers and Aboriginal patients and their families: the perspective of service providers. BMC Health Serv Res 2013;13:460. 10.1186/1472-6963-13-460 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jennings W, Bond C, Hill PS. The power of talk and power in talk: a systematic review of Indigenous narratives of culturally safe healthcare communication. Aust J Prim Health 2018;24:109–15. 10.1071/PY17082 [DOI] [PubMed] [Google Scholar]
  • 11.Strong J, Nielsen M, Williams M, et al. Quiet about pain: experiences of Aboriginal people in two rural communities. Aust J Rural Health 2015;23:181–4. 10.1111/ajr.12185 [DOI] [PubMed] [Google Scholar]
  • 12.McGrath PD, Phillips EL. Insights from the Northern Territory on factors that facilitate effective palliative care for Aboriginal peoples. Aust Health Rev 2009;33:636–44. 10.1071/ah090636 [DOI] [PubMed] [Google Scholar]
  • 13.O'Brien AP, Bloomer MJ, McGrath P, et al. Considering Aboriginal palliative care models: the challenges for mainstream services. Rural Remote Health 2013;13:2339. [PubMed] [Google Scholar]
  • 14.Eagar K, Watters P, Currow DC, et al. The Australian Palliative Care Outcomes Collaboration (PCOC)--measuring the quality and outcomes of palliative care on a routine basis. Aust Health Rev 2010;34:186–92. 10.1071/AH08718 [DOI] [PubMed] [Google Scholar]
  • 15.Clapham S & Holloway A for the Palliative Care Outcomes Collaboration . Palliative care outcomes Collaboration–Clinical manual August 2017. Wollongong, NSW: Australian Health Services Research Institute (AHSRI), University of Wollongong, 2017. [Google Scholar]
  • 16.Masso M, Allingham SF, Banfield M, et al. Palliative care phase: inter-rater reliability and acceptability in a national study. Palliat Med 2015;29:22–30. 10.1177/0269216314551814 [DOI] [PubMed] [Google Scholar]
  • 17.Hofstede JM, Raijmakers NJ, van der Hoek LS, et al. Differences in palliative care quality between patients with cancer, patients with organ failure and frail patients: a study based on measurements with the consumer quality index palliative care for bereaved relatives. Palliat Med 2016;30:780–8. 10.1177/0269216315627123 [DOI] [PubMed] [Google Scholar]
  • 18.Pink B. Australian statistical geography standard (ASGS): volume 5–Remoteness structure, Australia July 2011 (ABS catalogue No. 1270.0.55.005). Canberra: Australian Bureau of Statistics, 2013. [Google Scholar]
  • 19.Abernethy AP, Shelby-James T, Fazekas BS, et al. The Australia-modified Karnofsky Performance Status (AKPS) scale: a revised scale for contemporary palliative care clinical practice [ISRCTN81117481]. BMC Palliat Care 2005;4:7. 10.1186/1472-684X-4-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Aoun SM, Monterosso L, Kristjanson LJ, et al. Measuring symptom distress in palliative care: psychometric properties of the symptom assessment scale (SAS). J Palliat Med 2011;14:315–21. 10.1089/jpm.2010.0412 [DOI] [PubMed] [Google Scholar]
  • 21.Woods JA, Johnson CE, Allingham SF, et al. Collaborative data familiarisation and quality assessment: reflections from use of a national dataset to investigate palliative care for Indigenous Australians. Health Inf Manag 2021;50:64–75. 10.1177/1833358320908957 [DOI] [PubMed] [Google Scholar]
  • 22.Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004;159:702–6. 10.1093/aje/kwh090 [DOI] [PubMed] [Google Scholar]
  • 23.Yelland LN, Salter AB, Ryan P. Performance of the modified poisson regression approach for estimating relative risks from clustered prospective data. Am J Epidemiol 2011;174:984–92. 10.1093/aje/kwr183 [DOI] [PubMed] [Google Scholar]
  • 24.Austin PC. A tutorial on multilevel survival analysis: methods, models and applications. Int Stat Rev 2017;85:185–203. 10.1111/insr.12214 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Kou K, Baade PD, Gatton M, et al. Individual- and Area-Level socioeconomic inequalities in esophageal cancer survival in Shandong Province, China: a multilevel analysis. Cancer Epidemiol Biomarkers Prev 2019;28:1427–34. 10.1158/1055-9965.EPI-19-0203 [DOI] [PubMed] [Google Scholar]
  • 26.Singer JD, Willett JB. Applied longitudinal data analysis: modeling change and event occurrence. New York: Oxford University Press, 2003. [Google Scholar]
  • 27.Lau B, Cole SR, Gange SJ. Competing risk regression models for epidemiologic data. Am J Epidemiol 2009;170:244–56. 10.1093/aje/kwp107 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 1995;57:289–300. 10.1111/j.2517-6161.1995.tb02031.x [DOI] [Google Scholar]
  • 29.Zhang Y. Uncertainty in illness: theory review, application, and extension. Oncol Nurs Forum 2017;44:645–9. 10.1188/17.ONF.645-649 [DOI] [PubMed] [Google Scholar]
  • 30.Shahid S, Bessarab D, van Schaik KD, et al. Improving palliative care outcomes for Aboriginal Australians: service providers' perspectives. BMC Palliat Care 2013;12:26. 10.1186/1472-684X-12-26 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.McGrath P, Holewa H, Kail-Buckley S. "They should come out here. research findings on lack of local palliative care services for Australian aboriginal people. Am J Hosp Palliat Care 2007;24:105–13. [DOI] [PubMed] [Google Scholar]
  • 32.Banham D, Roder D, Keefe D, et al. Disparities in breast screening, stage at diagnosis, cancer treatment and the subsequent risk of cancer death: a retrospective, matched cohort of Aboriginal and non-Aboriginal women with breast cancer. BMC Health Serv Res 2019;19:387. 10.1186/s12913-019-4147-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Banham D, Roder D, Keefe D, et al. Disparities in cancer stage at diagnosis and survival of Aboriginal and non-Aboriginal South Australians. Cancer Epidemiol 2017;48:131–9. 10.1016/j.canep.2017.04.013 [DOI] [PubMed] [Google Scholar]
  • 34.Shahid S, Teng T-HK, Bessarab D, et al. Factors contributing to delayed diagnosis of cancer among Aboriginal people in Australia: a qualitative study. BMJ Open 2016;6:e010909. 10.1136/bmjopen-2015-010909 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Banham D, Roder D, Eckert M, et al. Cancer treatment and the risk of cancer death among Aboriginal and non-Aboriginal South Australians: analysis of a matched cohort study. BMC Health Serv Res 2019;19:771. 10.1186/s12913-019-4534-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Callander E, Bates N, Lindsay D, et al. The patient co-payment and opportunity costs of accessing healthcare for Indigenous Australians with cancer: a whole of population data linkage study. Asia Pac J Clin Oncol 2019;15:309–15. 10.1111/ajco.13180 [DOI] [PubMed] [Google Scholar]
  • 37.Bernardes CM, Beesley V, Martin J, et al. Unmet supportive care needs among people with cancer: a cross-cultural comparison between Indigenous and non-Indigenous Australians. Eur J Cancer Care 2019;28:e13080. 10.1111/ecc.13080 [DOI] [PubMed] [Google Scholar]
  • 38.Woods JA, Newton JC, Thompson SC, et al. Indigenous compared with non-Indigenous Australian patients at entry to specialist palliative care: cross-sectional findings from a multi-jurisdictional dataset. PLoS One 2019;14:e0215403. 10.1371/journal.pone.0215403 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Martin EM, Barkley TW. Improving cultural competence in end-of-life pain management. Nursing 2016;46:32–41. 10.1097/01.NURSE.0000475480.75266.9a [DOI] [PubMed] [Google Scholar]
  • 40.McGrath P, Ogilvie KF, Rayner RD, et al. The "right story" to the "right person": communication issues in end-of-life care for Indigenous people. Aust Health Rev 2005;29:306–16. 10.1071/AH050306 [DOI] [PubMed] [Google Scholar]
  • 41.Krok JL, Baker TA, McMillan SC. Age differences in the presence of pain and psychological distress in younger and older cancer patients. J Hosp Palliat Nurs 2013;15:107–13. 10.1097/NJH.0b013e31826bfb63 [DOI] [Google Scholar]
  • 42.Connolly A, Burns S, Allingham S. Patient outcomes in palliative care in Australia: national report January–June 2018. Wollongong, NSW: palliative care outcomes collaboration, Australian health services research Institute, University of Wollongong 2018.
  • 43.Cunningham J, Rumbold AR, Zhang X, et al. Incidence, aetiology, and outcomes of cancer in Indigenous peoples in Australia. Lancet Oncol 2008;9:585–95. 10.1016/S1470-2045(08)70150-5 [DOI] [PubMed] [Google Scholar]
  • 44.Randall DA, Lujic S, Havard A, et al. Multimorbidity among Aboriginal people in New South Wales contributes significantly to their higher mortality. Med J Aust 2018;209:19–23. 10.5694/mja17.00878 [DOI] [PubMed] [Google Scholar]
  • 45.Palliative Care Outcomes Collaboration . PCOC national report on patient outcomes in palliative care in Australia: list of services, January to June 2020. Wollongong: Australian Health Services Research Institute, The University of Wollongong, 2020. [Google Scholar]
  • 46.Australian Bureau of Statistics . 2071.0 Census of Population and Housing: Reflecting Australia–Stories from the Census (2016)–Aboriginal and Torres Strait Islander Population. Canberra: ABS, 2017. [Google Scholar]
  • 47.Roder DM, Fong KM, Brown MP, et al. Realising opportunities for evidence-based cancer service delivery and research: linking cancer registry and administrative data in Australia. Eur J Cancer Care 2014;23:721–7. 10.1111/ecc.12242 [DOI] [PubMed] [Google Scholar]
  • 48.Laverty M, McDermott DR, Calma T. Embedding cultural safety in Australia's main health care standards. Med J Aust 2017;207:15–16. 10.5694/mja17.00328 [DOI] [PubMed] [Google Scholar]
  • 49.Kendall E, Marshall CA, Barlow L. Stories rather than surveys: a journey of discovery and Emancipation. Int J Qual Methods 2013;12:258–71. 10.1177/160940691301200112 [DOI] [Google Scholar]
  • 50.Elvidge E, Paradies Y, Aldrich R, et al. Cultural safety in hospitals: validating an empirical measurement tool to capture the Aboriginal patient experience. Aust Health Rev 2020;44:205–11. 10.1071/AH19227 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2020-042268supp001.pdf (229.1KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES