Abstract
Introduction
Relieving obstructive jaundice in inoperable pancreato-biliary cancers improves quality of life and permits chemotherapy. Percutaneous transhepatic cholangiography with drainage and/or stenting relieves jaundice but can be associated with significant morbidity and mortality. Percutaneous transhepatic biliary drainage (PTBD) in malignant biliary obstruction was therefore examined in a national cohort to establish risk factors for poor outcomes.
Methods
Retrospective study of adult patients undergoing PTBD for palliation of pancreato-biliary cancer in England between 2001 and 2014 identified from Hospital Episode Statistics. Multivariate logistic regression analysis was used to examine associations with mortality and the need for a repeat PTBD within 2 months.
Results
16 822 patients analysed (median age 72 (range 19–104) years, 50.3% men). 58% pancreatic and 30% biliary tract cancer. In-hospital and 30-day mortality were 15.3% (95% CI 14.7% to 15.9%) and 23.1% (22.4%–23.8%), respectively. 20.2% suffered a coded complication within 3 months. Factors associated with 30-day mortality: age (≥81 years OR 2.68 (95% CI 2.37 to 3.03), p<0.001), increasing comorbidity (Charlson score 20+, 3.10 (2.64–3.65), p<0.001), pre-existing renal dysfunction (2.37 (2.12–2.65), p<0.001) and non-pancreatic cancer (unspecified biliary tract 1.28 (1.08–1.52), p=0.004). Women had lower mortality (0.91 (0.84–0.98), p=0.011), as did patients undergoing PTBD in a ‘higher volume’ provider (84–180 PTBDs per year 0.68 (0.58–0.79), p<0.001).
Conclusions
In patients undergoing PTBD for the palliation of malignant biliary obstruction, 30-day mortality was high at 23.1%. Mortality was higher in older patients, men, those with increasing comorbidity, a cancer site other than pancreas and at ‘lower-volume’ PTBD providers.
Keywords: hepatobiliary tumours, adult palliative care, interventional radiology
Strengths and limitations of this study.
This is the largest study to date examining the outcomes of patients undergoing percutaneous transhepatic biliary drainage (PTBD) for unresectable biliary tract obstruction.
Use of Hospital Episode Statistics (HES) database ensures high-quality diagnostic, procedural and mortality data at a national level.
Diagnostic and procedural coding relevant to study locally validated, confirming a high level of accuracy.
Accuracy of HES database dependent on quality of medical records and the staff coding the records.
Information relevant to use of PTBD not recorded in HES includes pathology results, use of antibiotics, technical details of PTBD and seniority/experience of radiologist performing procedure.
Introduction
Jaundice may arise from biliary obstruction by cholangiocarcinoma, pancreatic, duodenal, gallbladder or primary or secondary liver malignancies. The majority of such patients present at a late stage and are unsuitable for curative surgery. Biliary obstruction may impair quality of life and result in pruritus, cholangitis and liver failure.1 2 In patients who are unsuitable for curative resection, relief of obstructive jaundice improves quality of life.3 4 Furthermore, biliary drainage can be a bridge to palliative chemotherapy, improving survival in locally advanced and metastatic pancreatic cancer,5 and in advanced biliary tract cancer.6 7
Biliary drainage can be achieved surgically, via endoscopic retrograde cholangiopancreatography (ERCP) or via percutaneous transhepatic cholangiography (PTC). The approach employed will depend on whether the cancer is operable and its location. PTC facilitates external and internal biliary tree drainage and is the primary method of relieving biliary obstruction for malignant lesions above the level of the common hepatic duct,8 or when ERCP has failed to relieve more distal obstruction. However, percutaneous transhepatic biliary drainage (PTBD) is associated with significant mortality and morbidity, with complications reported to be as high as 30%, including cholangitis, sepsis, haemorrhage and stent blockage.2
Case series and randomised controlled trials examining outcomes of PTBD in malignancy have typically involved less than 100 patients. National data were therefore used to examine unselected outcomes for PTBD in a very large cohort of inoperable pancreato-biliary cancer to establish factors associated with poor outcomes, such as early mortality.
Methods
Data source
Hospital Episode Statistics (HES) is an administrative database that records all elective and emergency care episodes in English National Health Service hospitals. A unique identifier allows individuals to be followed through their hospital admissions and outpatient attendances. Each recorded episode contains diagnostic, procedural, demographic, administrative and geographical information. Diagnostic data are coded using the International Classification of Diseases V.10 (ICD-10) and procedures coded using the Office of Population Censuses and Surveys Classification of Interventions and Procedures fourth revision (OPCS-4). HES is linked to the Office of National Statistics mortality data, providing information on date and cause of death.9
Study population
Inclusion criteria
All patients aged over 18 years undergoing PTBD between 2001 and 2014 with a diagnosis of cancer of the pancreas, gallbladder, liver, intrahepatic bile ducts, small bowel or unspecified bile duct cancer were included. PTBD and diagnosis data were identified by OPCS-4 and ICD-10 coding, respectively. All analyses relate to the first PTBD each patient underwent.
Exclusion criteria
Patients were excluded if they were diagnosed with cancer more than 2 years prior to their first PTBD or more than 6 months after their first procedure in order to exclude patients with potentially incorrectly coded diagnoses and to allow for delays in coding of a cancer diagnosis. Patients who went on to have a surgical resection of their malignancy after PTBD were identified by OPCS-4 coding and were excluded, as were those with incomplete demographic data.
Patient and public involvement
This research was carried out without patient or public involvement.
Validation of PTBD population
To assess the validity of the PTBD population, the number of PTBDs undertaken at University Hospital Birmingham (UHB) between April 2009 and April 2014 that met the study criteria were extracted from the UHB radiology database and compared with the number of PTBDs coded in HES at UHB for the same period.
Study variables and data extraction
Demographics
Demographic data including age, gender and ethnicity were extracted from coding at the time of PTBD. Age was divided into quintiles. Ethnicity was classified into White, Asian or Asian British, Black or Black British, Mixed, any other ethnic group and unknown.
Comorbidity
The Charlson comorbidity index was calculated using ICD-10 codes for secondary diagnoses, excluding any form of cancer, and divided into five categories by number of comorbidities. The derivation of Charlson score from ICD-10 coding in HES has previously been assessed and found to be valid for assessing comorbidity in patients undergoing surgery for urological cancer. There was an 83% agreement between Charlson scores derived from ICD-10 coding in HES and those derived from ICD-9 comorbidity codes.10
Socioeconomic status
Deprivation was assessed using the index of multiple deprivations 2007, which is calculated from an aggregate score for each English lower layer super output area (LSOA), based on income, employment, health, education, training and skills, barriers to housing and services, crime and living environment.11 Patients’ LSOAs are recorded in HES based on postcode of residence and deprivation was analysed in quintiles, with 1 being the most and 5 the least deprived.
Healthcare provider
Healthcare providers were stratified by their PTBD volume per year into quintiles.
Endoscopic retrograde cholangiopancreatography
Patients who had undergone an ERCP prior to their PTBD were identified by OPCS-4 coding.
Outcomes
Outcomes were calculated from the date of the first PTBD and included in-hospital, 7 and 30 day mortality and median survival. Emergency readmissions into any hospital within 30 days of discharge post-PTBD were also identified. The proportion of patients that suffered complications related to PTBD was identified using ICD-10 coding. Patients undergoing chemotherapy after their PTBD were identified by ICD-10 (Z080, Z511, Z542, Z926) and OPCS 4 codes (X70, X71, X72, X73, X352, X384).
Ethics
HES includes only pseudonymised data and therefore ethical approval is not necessary. It is available under a data sharing agreement for the purposes of service evaluation.
Statistical analysis
All statistical analyses were carried out using STATA SE V.14 (StataCorp LP). Univariate analyses were performed to compare characteristics of included and excluded patients, as well as factors affecting the rates of chemotherapy (tables 1 and 2 respectively), using χ2 tests for categorical variables. Bonferroni correction was applied to these analyses and results were considered statistically significant if p values were <0.0045. A multivariate model was produced to examine associations with mortality following adjustment for the variables identified on univariate analyses. A further multivariate model was produced to examine associations with needing a further PTBD procedure, adjusting for the variables identified on univariate analysis. In the multivariate analyses p-values of <0.05 were considered statistically significant. Odds ratios, 95% CI and p values were generated from the multivariate model. Unadjusted Kaplan Meier analysis was undertaken for 7-day and 30-day mortality, split by age quintile.
Table 1.
Included patients | Excluded patients | P value | |
Sex | |||
Male | 8465 (50.3%) | 527 (52.4%) | 0.203 |
Female | 8357 (49.7%) | 479 (47.6%) | |
Age | |||
<61 | 3267 (19.4%) | 237 (23.6%) | 0.005 |
62 to 68 | 3082 (18.3%) | 173 (17.2%) | |
69 to 74 | 3253 (19.3%) | 179 (17.8%) | |
75 to 80 | 3412 (20.3%) | 219 (21.8%) | |
≥81 | 3808 (22.6%) | 198 (19.7%) | |
Deprivation | |||
1 | 3258 (19.4%) | 183 (18.2%) | 0.08 |
2 | 3284 (19.7%) | 199 (19.8%) | |
3 | 3356 (19.5%) | 189 (18.8%) | |
4 | 3453 (20.5%) | 201 (20.0%) | |
5 | 3343 (19.9%) | 225 (22.4%) | |
Unknown | 71 (0.4%) | 9 (0.9%) | |
Ethnic group | |||
White | 13 190 (78.4%) | 848 (84.3%) | <0.001 |
Asian or Asian British | 348 (2.1%) | 22 (2.2%) | |
Black or Black British | 271 (1.6%) | 11 (1.1%) | |
Mixed | 32 (0.2%) | * | |
Any other ethnic group | 184 (1.1%) | 11 (1.1%) | |
Unknown | 2797 (16.6%) | 111 (11.0%) | |
Comorbidities | |||
<5 | 9456 (56.2%) | 660 (65.6%) | <0.001 |
5 to 10 | 1953 (11.6%) | 104 (10.3%) | |
10 to 15 | 3519 (20.9%) | 153 (15.2%) | |
15 to 20 | 1128 (6.7%) | 54 (5.4%) | |
>20 | 766 (4.6%) | 35 (3.5%) | |
Type of cancer | |||
C17—malignant neoplasm of small intestine | 526 (3.1%) | 41 (4.1%) | <0.001 |
C22—malignant neoplasm of liver and Intrahepatic bile ducts | 5069 (30.1%) | 332 (33.0%) | |
C23— malignant neoplasm of gallbladder | 715 (4.3%) | 45 (4.5%) | |
C24—malignant neoplasm of other and unspecified parts of biliary tract | 762 (4.5%) | 74 (7.4%) | |
C25—malignant neoplasm of pancreas | 9750 (58%) | 514 (51.1%) | |
Pre-existing renal dysfunction | 1747 (10.4%) | 128 (12.7%) | |
Previous ERCP | 10 384 (61.8%) | 631 (62.7%) |
ERCP, endoscopic retrograde cholangiopancreatography.
Table 2.
Type of cancer | Number of patients who had chemotherapy pre-PTBD | Number of patients who had chemotherapy post-PTBD | Pvalue |
C17—malignant neoplasm of small intestine | 73 (13.9%) | 100 (19%) | <0.001 |
C22—malignant neoplasm of liver and intrahepatic bile ducts | 251 (5%) | 1011 (19.9%) | |
C23—malignant neoplasm of gallbladder | 62 (8.7%) | 122 (17.1%) | |
C24—malignant neoplasm of other and unspecified parts of biliary tract | 24 (3.1%) | 76 (10%) | |
C25—malignant neoplasm of pancreas | 855 (8.8%) | 2219 (22.8%) | |
Age group | |||
<61 | 462 (14.1%) | 1263 (38.7%) | <0.001 |
62–68 | 352 (11.4%) | 937 (30.4%) | |
69–74 | 279 (8.6%) | 771 (23.7%) | |
75–80 | 127 (3.7%) | 461 (13.5%) | |
≥81 | 45 (1.2%) | 96 (2.5%) | |
Year of PTBD | |||
2001/2002 | 18 (2.0%) | 115 (13.1%) | <0.001 |
2002/2003 | 48 (4.9%) | 137 (13.9%) | |
2003/2004 | 44 (4.8%) | 142 (15.4%) | |
2004/2005 | 68 (6.6%) | 173 (16.8%) | |
2005/2006 | 53 (4.7%) | 206 (18.4%) | |
2006/2007 | 73 (6.0%) | 247 (20.5%) | |
2007/2008 | 118 (8.9%) | 286 (21.6%) | |
2008/2009 | 98 (7.1%) | 305 (22.0%) | |
2009/2010 | 144 (9.2%) | 345 (22.0%) | |
2010/2011 | 152 (9.6%) | 362 (22.8%) | |
2011/2012 | 145 (8.9%) | 384 (23.7%) | |
2012/2013 | 149 (9.4%) | 387 (24.5%) | |
2013/2014 | 155 (9.6%) | 439 (27.3%) |
PTBD, percutaneous transhepatic biliary drainage.
Results
Study population
Between 2001 and 2014, 19 525 patients underwent PTBD for one of the study malignancies. 1006 patients were diagnosed with cancer more than 2 years before or more than 6 months after their PTBD and were excluded. A further 1438 patients who underwent potentially curative resection after their PTBD and 259 patients with incomplete demographic data were also excluded, giving a final study population of 16 822 (figure 1).
Patient characteristics
The characteristics of the study population and excluded patients are shown in table 1. 50.3% of patients were men and the median age was 72 (range 19–104) years. 58% of patients had pancreatic cancer, with malignant neoplasm of the liver and intrahepatic bile ducts accounting for 30.1% of patients, of whom 90.4% had a diagnosis of cholangiocarcinoma. 86.3% of patients underwent only one procedure, with a range from 1 to 15 and 61.8% had undergone a previous ERCP. 57.9% of patients undergoing prior ERCP had their PTBD carried out during the same admission episode, indicating that the PTBD was likely a salvage procedure.
Validation
The number of PTBDs meeting the study criteria at UHB between April 2009 and April 2014 was 321 and the number of PTBDs coded in HES for UHB in the same time period was 305 (95%), suggesting misclassification in HES is unlikely. Univariate analysis comparing excluded and included patients is in table 1. There was no difference in gender between excluded and included patients. Excluded patients were younger (under 61 years of age (23.6% vs 19.4%, p=0.005)), more likely to be white (84.3% vs 78.4%, p<0.001), have less comorbidities (<5 65.6% vs 56.2%, p<0.001) and were less likely to have pancreatic cancer (51.1% vs 58%, p<0.001). Only 259 (1.3%) out of 19 525 patients were excluded for incomplete demographic data.
Crude mortality and emergency readmission rates
5.2% (95% CI 4.9% to 5.6%) of patients died within 7 days of PTBD, 15.3% (95% CI 14.7% to 15.9%) died in hospital and 23.1% (95% CI 22.4% to 23.8%) died within 30 days of their first PTBD. Median survival was 92 (IQR 33–242) days and the median length of stay after PTBD was 9 (IQR 4–16) days. The emergency readmission rate within 30 days was 20.8% (95% CI 20.1% to 21.5%).
Complications
5.9% of patients suffered a complication within 7 days of their PTBD, and 20% within 3 months. Infection was the most common complication with 2.4% of patients experiencing this within a week, and 9% within a month (cholangitis 3.9%, sepsis 3.9%, bacterial infection of unspecified site 0.8%, cholecystitis 0.4%). 2.9% of patients had a code for stent displacement or blockage (mean time to stent blockage or displacement 6.3 (SD 8.6) months) and 2.1% for gastrointestinal haemorrhage. The rate of post-PTBD acute kidney injury was 0.9% within 7 days and 2.4% within 3 months.
Chemotherapy
38.7% of patients under the age of 61 were coded as undergoing chemotherapy after their PTBD (table 2). This was less common in older patients: 62–68 years (30.4%), 69–74 (23.7%), 75–80 (13.5%) and ≥81 (2.5%) (p<0.001). Patients with pancreatic cancer were the most likely to receive chemotherapy at 22.8%. Over the time period studied, more patients received chemotherapy after PTBD (2001/2002 13.1%, 2013/2014 27.3% (p<0.001).
Univariate regression analysis
Age, gender, comorbidity, deprivation, pre-existing renal failure, type of cancer, year of procedure and provider PTBD volume were all found to be associated with mortality and adjusted for in the subsequent multivariate analyses.
Multivariate regression analysis
Demographic factors and mortality
The results of the multivariate regression analysis for demographic factors associated with mortality are shown in table 3. Age was strongly associated with mortality, with the ≥81 age group having the highest 7-day (2.87 (95% CI 2.23 to 3.69) p<0.001), in-hospital (3.47 (95% CI 2.97 to 4.05), p<0.001) and 30-day mortality (2.68 (95% CI 2.37 to 3.03), p<0.001). Women had a better outlook, with lower 7-day (0.82 (95% CI 0.71 to 0.95), p=0.007) and 30-day mortality (0.91 (95% CI 0.84 to 0.98), p=0.011).
Table 3.
In-hospital mortality | 7-day mortality | 30-day mortality | |||||
OR | P value | OR | P value | OR | P value | ||
Age group | <61 | 1 (baseline) | 1 (baseline group) | 1 (baseline) | |||
62–68 | 1.49 (1.25–1.77) | <0.001 | 1.28 (0.96–1.71) | 0.095 | 1.33 (1.16–1.52) | <0.001 | |
69–74 | 1.66 (1.40–1.96) | <0.001 | 1.58 (1.20–2.08) | 0.001 | 1.43 (1.25–1.63) | <0.001 | |
75–80 | 2.36 (2.01–2.77) | <0.001 | 2.20 (1.70–2.85) | <0.001 | 1.89 (1.66–2.15) | <0.001 | |
≥81 | 3.47 (2.97–4.05) | <0.001 | 2.87 (2.23–3.69) | <0.001 | 2.68 (2.37–3.03) | <0.001 | |
Sex | Male | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
Female | 0.98 (0.89–1.07) | 0.62 | 0.82 (0.71–0.95) | 0.007 | 0.91 (0.84–0.98) | 0.011 | |
Comorbidity score | <5 | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
5–10 | 1.39 (1.21–1.61) | <0.001 | 1.46 (1.16–1.82) | 0.001 | 1.38 (1.23–1.56) | <0.001 | |
10–15 | 1.65 (1.47–1.85) | <0.001 | 1.72 (1.44–2.06) | <0.001 | 1.96 (1.78–2.16) | <0.001 | |
15–20 | 2.15 (1.82–2.54) | <0.001 | 2.12 (1.65–2.72) | <0.001 | 2.29 (1.99–2.65) | <0.001 | |
20+ | 2.96 (2.48–3.53) | <0.001 | 2.76 (2.15–3.55) | <0.001 | 3.10 (2.64–3.65) | <0.001 | |
Deprivation Score | 1 (most deprived) | 1.49 (1.29–1.72) | <0.001 | 1.31 (1.05–1.64) | 0.017 | 1.28 (1.13–1.44) | <0.001 |
2 | 1.31 (1.13–1.51) | <0.001 | 1.12 (0.89–1.41) | 0.32 | 1.21 (1.07–1.37) | 0.002 | |
3 | 1.08 (0.94–1.36) | 0.273 | 1.09 (0.87–1.36) | 0.472 | 1.01 (0.90–1.29) | 0.863 | |
4 | 1.18 (1.02–1.36) | 0.026 | 1.06 (0.85–1.33) | 0.601 | 1.14 (1.01–1.29) | 0.03 | |
5 (least deprived) | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | ||||
Pre-existing renal dysfunction | 3.48 (3.09–3.92) | <0.001 | 2.94 (2.48–3.50) | <0.001 | 2.37 (2.12–2.65) | <0.001 | |
Type of cancer | C17—malignant neoplasm of small intestine | 1.36 (1.07–1.73) | 0.013 | 1.52 (1.07–2.15) | 0.019 | 1.16 (0.94–1.44) | 0.161 |
C22—malignant neoplasm of liver and intrahepatic bile ducts | 1.26 (1.14–1.39) | <0.001 | 1.07 (0.91–1.26) | 0.403 | 1.14 (1.04–1.24) | 0.004 | |
C23—malignant neoplasm of gallbladder | 1.30 (1.05–1.61) | 0.018 | 1.09 (0.76–1.56) | 0.639 | 1.18 (0.98–1.42) | 0.077 | |
C24—malignant neoplasm of other and unspecified parts of biliary tract | 1.33 (1.09–1.61) | 0.005 | 1.16 (0.86–1.57) | 0.336 | 1.28 (1.08–1.52) | 0.004 | |
C25—malignant neoplasm of pancreas | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | ||||
Previous ERCP | 0.90 (0.82–0.99) | 0.026 | 0.93 (0.80–1.08) | 0.332 | 0.90 (0.83–0.97) | 0.007 |
ERCP, endoscopic retrograde cholangiopancreatography.
30-day mortality was associated with deprivation (1.28 (95% CI 1.13 to 1.44), p<0.001), increased comorbidity (Charlson index 20+, 3.10 (95% CI 2.64 to 3.65), p<0.001) and pre-existing renal dysfunction (2.37 (95% CI 2.12 to 2.65), p<0.001), with 7-day mortality following a similar pattern.
Patients with unspecified bile duct cancer (1.28 (95% CI 1.08 to 1.52), p=0.004) or liver and intrahepatic bile duct cancer (1.04 (95% CI 1.14 to 1.24), p=0.004) had a higher 30-day mortality than those with pancreatic cancer. Patients undergoing ERCP prior to PTBD also had lower 30-day mortality (0.90 (95% CI 0.83 to 0.97), p=0.007).
Procedural factors and mortality
The results of the multivariate regression analysis of procedural factors associated with mortality are shown in table 4. There was a five-fold variation in 30-day mortality between providers, ranging from 9.1% to 50%. Compared with providers undertaking 1–15 PTBDs per year, there was a significantly decreased in-hospital and 30-day mortality in providers performing 28–43 PTBDs per year (0.72 (95% CI 0.62 to 0.83), p<0.001 and 0.79 (95% CI 0.69 to 0.89), p<0.001, respectively). In providers performing more than 44 PTBDs per year, there was an even larger decrease in in-hospital (0.68 (95% CI 0.57 to 0.82), p<0.001), 7-day (0.54 (95% CI 0.40 to 0.74), p<0.001) and 30-day mortality (0.63 (95% CI 0.54 to 0.74), p<0.001). The effect of provider volume on 30-day mortality can be seen in figure 2.
Table 4.
In-hospital mortality | 7-day mortality | 30-day mortality | |||||
OR | P value | OR | P value | OR | P value | ||
PTBD volume per year | ≤15 | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
16–27 | 0.79 (0.69–0.90) | 0.001 | 0.92 (0.76–1.13) | 0.44 | 0.93 (0.83–1.04) | 0.218 | |
28–43 | 0.72 (0.62–0.83) | <0.001 | 0.81 (0.65–1.01) | 0.063 | 0.79 (0.69–0.89) | <0.001 | |
44–83 | 0.68 (0.57–0.82) | <0.001 | 0.54 (0.40–0.74) | <0.001 | 0.63 (0.54–0.74) | <0.001 | |
84–180 | 0.68 (0.57–0.82) | <0.001 | 0.71 (0.52–0.95) | 0.023 | 0.68 (0.58–0.79) | <0.001 | |
Pancreaticoduodenectomy volume | ≤1 | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
2–7 | 0.96 (0.84–1.11) | 0.603 | 0.91 (0.73–1.12) | 0.364 | 0.90 (0.80–1.01) | 0.073 | |
8–23 | 1.08 (0.94–1.24) | 0.278 | 0.97 (0.79–1.20) | 0.79 | 0.99 (0.88–1.11) | 0.814 | |
24–30 | 1.12 (0.93–1.35) | 0.226 | 0.88 (0.66–1.18) | 0.397 | 0.93 (0.80–1.09) | 0.377 | |
30–86 | 1.01 (0.83–1.22) | 0.96 | 0.87 (0.64–1.18) | 0.371 | 0.95 (0.81–1.12) | 0.549 | |
Day of procedure | Monday to Thursday | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
Friday | 1.09 (0.99–1.20) | 0.089 | 0.93 (0.79–1.09) | 0.346 | 0.99 (0.91–1.08) | 0.794 | |
Saturday | 1.16 (0.78–1.71) | 0.464 | 1.18 (0.64–2.16) | 0.596 | 1.05 (0.75–1.48) | 0.776 | |
Sunday | 2.54 (1.66–3.91) | <0.001 | 3.69 (2.16–6.30) | <0.001 | 2.20 (1.47–3.28) | <0.001 | |
Bank Holiday or Bank Holiday Weekend | 1.26 (0.98–1.61) | 0.075 | 0.99 (0.65–1.51) | 0.956 | 0.99 (0.79–1.25) | 0.959 | |
Year of PTBD | 2001/2002 | 1 (baseline group) | 1 (baseline group) | 1 (baseline group) | |||
2002/2003 | 1.11 (0.86–1.44) | 0.426 | 1.15 (0.75–1.76) | 0.526 | 1.05 (0.84–1.31) | 0.669 | |
2003/2004 | 1.11 (0.85–1.45) | 0.424 | 1.45 (0.96–2.19) | 0.081 | 1.05 (0.84–1.31) | 0.687 | |
2004/2005 | 1.07 (0.82–1.38) | 0.635 | 1.02 (0.66–1.58) | 0.913 | 1.03 (0.83–1.29) | 0.775 | |
2005/2006 | 1.14 (0.88–1.47) | 0.319 | 1.24 (0.83–1.87) | 0.297 | 1.08 (0.87–1.34) | 0.468 | |
2006/2007 | 1.03 (0.80–1.33) | 0.803 | 1.04 (0.69–1.58) | 0.852 | 1.03 (0.83–1.27) | 0.82 | |
2007/2008 | 0.90 (0.70–1.16) | 0.411 | 0.88 (0.58–1.33) | 0.539 | 0.78 (0.63–0.97) | 0.023 | |
2008/2009 | 0.93 (0.73–1.19) | 0.571 | 1.20 (0.81–1.78) | 0.366 | 0.96 (0.78–1.19) | 0.728 | |
2009/2010 | 0.84 (0.66–1.08) | 0.173 | 0.99 (0.67–1.47) | 0.967 | 0.84 (0.68–1.03) | 0.091 | |
2010/2011 | 0.70 (0.55–0.90) | 0.005 | 0.71 (0.47–1.07) | 0.104 | 0.78 (0.63–0.96) | 0.019 | |
2011/2012 | 0.73 (0.57–0.94) | 0.013 | 0.89 (0.60–1.33) | 0.579 | 0.86 (0.70–1.05) | 0.136 | |
2012/2013 | 0.71 (0.56–0.91) | 0.006 | 0.80 (0.54–1.19) | 0.281 | 0.92 (0.75–1.13) | 0.428 | |
2013/2014 | 0.58 (0.45–0.74) | <0.001 | 0.71 (0.48–1.06) | 0.094 | 0.65 (0.53–0.81) | <0.001 |
PTBD, percutaneous transhepatic biliary drainage.
The day of procedure had little influence on mortality with only Sunday having a higher in-hospital (2.54 (95% CI 1.66 to 3.91), p<0.001) and 30-day mortality (2.20 (95% CI 1.47 to 3.28), p<0.001) compared with Monday to Thursday. However, the number of patients who had a PTBD on a Sunday was very small at 114 (0.7%) and therefore these results should be treated with caution. In-hospital mortality post-PTBD on multivariate analysis fell over the period studied, with persistently lower in-hospital mortality between 2013 and 2014 (0.58 (95% CI 0.45 to 0.74), p<0.001) compared with 2001/2002.
Repeat PTBD
The results of the multivariate analyses of factors associated with needing a further PTBD are shown in table 5. Patients undergoing an additional PTBD procedure within 2 months of their initial PTBD were younger (81+years, 0.21 (95% CI 0.16 to 0.27), p<0.001), less likely to have comorbidities (20+, 0.45 (95% CI 0.29 to 0.70), p<0.001) and were more likely to have cholangiocarcinoma (2.05 (95% CI 1.77 to 2.37), p<0.001). Patients undergoing their procedure in a high-volume centre performing between 84 and 180 procedures per year were much less likely to require a second procedure within 2 months (0.47 (95% CI 0.36 to 0.62), p<0.001). The majority of patients underwent a repeat PTBD in the same centre, with only 222 patients (22.1%) being referred to another provider. Repeat PTBD procedures were usually undertaken during emergency admissions (62.4%) rather than elective episodes (37.6%). 1923 patients (11.4%) underwent an ERCP within 2 months of their initial PTBD.
Table 5.
OR | P value | ||
Age group | <61 | 1 (baseline group) | |
62–68 | 0.81 (0.68–0.97) | 0.026 | |
69–74 | 0.67 (0.56–0.81) | <0.001 | |
75–80 | 0.50 (0.41–0.61) | <0.001 | |
81+ | 0.21 (0.16–0.27) | <0.001 | |
Sex | Male | 1 (baseline group) | |
Female | 0.84 (0.74–0.96) | 0.01 | |
Comorbidity score | <5 | 1 (baseline group) | |
5–10 | 0.86 (0.70–1.06) | 0.161 | |
10–15 | 0.70 (0.59–0.83) | <0.001 | |
15–20 | 0.30 (0.20–0.45) | <0.001 | |
20+ | 0.45 (0.29–0.70) | <0.001 | |
Year of PTBD | 2001/2002 | 1 (baseline group) | |
2002/2003 | 1.07 (0.69–1.67) | 0.757 | |
2003/2004 | 1.00 (0.63–1.57) | 0.997 | |
2004/2005 | 0.94 (0.60–1.48) | 0.803 | |
2005/2006 | 0.91 (0.58–1.42) | 0.664 | |
2006/2007 | 1.32 (0.88–1.99) | 0.186 | |
2007/2008 | 1.24 (0.82–1.86) | 0.308 | |
2008/2009 | 1.52 (1.03–2.26) | 0.035 | |
2009/2010 | 1.78 (1.22–2.60) | 0.003 | |
2010/2011 | 1.57 (1.07–2.31) | 0.022 | |
2011/2012 | 1.99 (1.37–2.90) | <0.001 | |
2012/2013 | 2.02 (1.39–2.94) | <0.001 | |
2013/2014 | 1.91 (1.31–2.79) | 0.001 | |
Deprivation score | 1 (most deprived) | 0.78 (0.63–0.97) | 0.023 |
2 | 0.97 (0.79–1.19) | 0.792 | |
3 | 0.94 (0.77–1.15) | 0.543 | |
4 | 0.99 (0.81–1.21) | 0.918 | |
5 (least deprived) | 1 (baseline group) | ||
PTBD volume per year | <=15 | 1 (baseline group) | |
16–27 | 0.72 (0.60–0.87) | 0.001 | |
28–43 | 0.40 (0.32–0.50) | <0.001 | |
44–83 | 0.55 (0.42–0.72) | <0.001 | |
84–180 | 0.47 (0.36–0.62) | <0.001 | |
Type of cancer | C17—malignant neoplasm of small intestine | 1.47 (1.02–2.11) | 0.038 |
C22—Malignant neoplasm of liver and intraheptic bile ducts | 2.05 (1.77–2.37) | <0.001 | |
C23—Malignant neoplasm of gallbladder | 1.68 (1.23–2.31) | 0.001 | |
C24—Malignant neoplasm of other and unspecified parts of biliary tract | 1.12 (0.77–1.63) | 0.55 | |
C25—Malignant neoplasm of pancreas | 1 (baseline group) |
PTBD, percutaneous transhepatic biliary drainage.
Discussion
This is the largest study to date examining the outcomes of patients undergoing PTBD for unresectable malignant biliary tract obstruction. 7-day and 30-day mortality was 5.2% and 23.1%, respectively, with a median survival of 92 days. Reported 30-day mortality rates following PTBD vary considerably in the literature. In a 2008 review examining PTBD case series, 30-day mortality for patients with distal biliary obstruction varied from 2% to 20% and for hilar lesions from 9% to 20%.2 However, 30-day mortality has been reported to be as high as 39% in other case series.3 12–14 In 2012, the results of a large UK audit of biliary drainage and stenting procedures were published. This showed an in-hospital mortality of 19.8% in those with malignant biliary obstruction.15 Finally, a 2018 randomised controlled trial examining outcomes in patients with perihilar cholangiocarcinoma was stopped prematurely because of a higher mortality in the PTBD group compared with the endoscopic group (41 vs 11%).16
Surgical or endoscopic drainage of the biliary system was not examined in this study. Surgical drainage is associated with higher postoperative mortality and morbidity and an increased length of stay compared with non-surgical intervention.17–20 The choice between endoscopic or percutaneous biliary drainage is less clear, as few randomised trials exist.21 The decision often depends on the level of biliary obstruction. In England, PTBD is often the preferred technique for lesions above the common hepatic duct and when ERCP has failed.8 22 Endoscopic drainage, if technically possible, is probably perceived to be safer due to a small, very old, prospective randomised trial from 1987 in hilar and distal bile duct obstruction comparing the two techniques. The endoscopic approach had a higher success rate (81% vs 61%, p=0.017) and a lower 30-day mortality (15% vs 33%, p=0.016).23 However, only plastic stents were used in this study, which does not reflect current practice. There is only one randomised trial comparing PTBD with ERCP in hilar biliary obstruction due to gallbladder cancer.3 This demonstrated that PTBD had a higher success rate than ERCP (89% vs 41%, p<0.001) and a lower rate of early cholangitis (11% vs 48%, p=0.02). In a recent meta-analysis comparing PTBD with ERCP for the relief of malignant jaundice, patients undergoing PTBD were less likely to develop cholangitis compared with ERCP (0.55 (95% CI 0.36 to 0.84), p=0.006). However, there was no difference in success rate or 30-day mortality between the two approaches.24
The regression analysis in this study identified patients with an increased risk of death post PTBD. Older men, with increasing deprivation and comorbidity (especially pre-existing renal dysfunction), and those with a cancer type other than pancreatic had a worse prognosis. Over 60% of patients in our cohort underwent a prior ERCP and this was expected, given that 62.5% of the patients had pancreatic or extra-hepatic cholangiocarcinoma. Undergoing an ERCP prior to PTBD was associated with lower mortality. This was likely to be due to a high proportion of those patients undergoing ERCP having pancreatic cancer, which carried a better prognosis. Patient fitness to undergo a second biliary drainage procedure was also a likely contributing factor. A reduced mortality in providers performing a higher volume of PTBDs each year was identified. A number of factors may contribute to this difference including variability in peri-procedural care such as antibiotics and post-procedure management of complications such as sepsis and renal failure. Higher volume centres may also have a more rigorous approach to patient selection, with a greater emphasis on careful multi-disciplinary team discussion of management prior to PTBD. We recognise that it is not realistic to expect all patients to be transferred to high volume centres for PTBD but the authors would recommend that PTBD outcomes are audited regularly and peri-procedural practices from high volume centres with good outcomes adopted in low volume centres.
Biliary obstruction in malignancy may result in pruritus, pain, cholangitis and liver failure.1 2 The goal of biliary drainage in inoperable patients is to improve quality of life and, in those with a good performance status, allow palliative chemotherapy. Two studies have shown that biliary drainage can improve quality of life,3 4 and it has been shown that gemcitabine-based combinations improve progression free survival in pancreatic cancer (HR 0.78 (95%CI 0.70-0.88)).5 A combination of gemcitabine and cisplatin can improve median survival in advanced biliary tract cancer from 8.1 to 11.7 months (HR 0.64 (95% CI 0.52 to 0.80), p<0.001).6 7 However, in our study, it is important to recognise that when considering PTBD, most patients did not receive chemotherapy after their procedure, with the rates decreasing significantly with age (aged <40 years 40.7% chemotherapy, aged 70 to 80 17.4%, aged over 80 2.5%).
The rate of coded post-procedural complications in our study was high. 5.9% of patients were coded as experiencing a serious complication within 7 days, and 20.2% within 3 months. However, it was not possible to clarify the relationship between these later complications and the procedure or the underlying malignancy. Complication rates after PTBD vary between case series from 7% to 30%,2 but cholangitis was common with rates of between 9% and 11% reported.3 12–14 A UK audit reported a minor complication rate of 26% and a major complication rate of 7.9%, including a 3.5% rate of sepsis.16 It has therefore been recommended that all patients undergoing percutaneous drainage receive prophylactic antibiotics prior to their procedure.25–28 However, there are no national or international guidelines to date on this issue. Rates of cholangitis in these studies were higher in patients with a low serum albumin or raised C reactive protein, those with proximal or multiple points of intrahepatic biliary obstruction, neoplastic invasion or compression of the duodenum and if Staphylococcus aureus was present at the site of skin puncture. With these factors in mind, patients should be monitored closely post-PTBD for early signs of sepsis, and infection treated aggressively with intravenous antibiotics and fluids. Stent occlusion due to the deposition of a bacterial biofilm and biliary sludge or tumour overgrowth is an important late complication.29 In this study, stent blockage or displacement was coded in 6.2% of patients and rates in other series have been reported at between 5% and 27%.2 17 30 13.5% of patients in this study underwent more than one PTBD procedure. A 2008 review of PTBD series reported recurrence of obstructive jaundice in between 5% and 25% of patients, with the majority undergoing a repeat PTBD.2
Every healthcare provider in England is required to submit diagnostic and procedural data to HES. However, the accuracy of the coding data submitted is a potential concern as it depends on the quality of the medical records and on the staff coding the records. HES produces a yearly report on the quality of the data received. In the 2012/2013 report, 99.3% of primary diagnoses and 99.9% of primary procedure codes were accurate.31 In order to validate the accuracy of diagnostic and procedural coding relevant to this study, the number of PTBDs meeting the study criteria at UHB between 2009 and 2014 was compared with the number submitted to HES. 321 patients were recorded in HES as undergoing PTBD at UHB and 305 patients were identified from examining local radiology data, giving an accuracy of 95%. HES data are unfortunately not linked to cancer registry data, due to restrictions under which the data are held. Patients were therefore excluded who had very long periods following an apparent diagnosis of malignancy and PTBD and those with long delays in cancer diagnosis following PTBD. There are some important aspects of the patient’s care that are not recorded in HES. Information regarding the exact location of the lesion, the precise technique used (such as use of external drainage or type of stent placed) and whether the procedure was performed with ultrasound guidance, by a supervised trainee, or by an experienced interventional radiologist was not available. Important data such as whether any technical difficulties were encountered, performance status, bilirubin and albumin levels, clotting profile or inflammatory markers were also not available. In particular, prescription data regarding antibiotic use are not recorded, which limits our ability to investigate further the high frequency of septic complications. Recording of chemotherapy in HES may also not be entirely complete.31
In conclusion, 30-day mortality in patients undergoing PTBD for relief of unresectable malignant biliary obstruction was high at 23.1%. Older men and those with increasing comorbidity (especially pre-existing renal dysfunction) and deprivation have a poorer prognosis. Patients undergoing a PTBD in a provider that performs more than 28 procedures per year have a significantly lower risk of death and there is a large variation in outcomes between providers. In light of the high mortality found in this study, the authors strongly recommend that patients undergo careful multidisciplinary discussion prior to PTBD in order to identify risk factors for a poor outcome, to treat renal dysfunction and sepsis early, and to confirm that the patient is likely to benefit from PTBD, in terms of either symptom relief or as a bridge to chemotherapy.
Supplementary Material
Acknowledgments
The authors would like to acknowledge the contribution of Mr Keith Roberts, consultant hepatobiliary surgeon at University Hospitals Birmingham, with identifying relevant surgical procedures.
Footnotes
Contributors: JGR: first author, study concept and design, data interpretation, drafting of manuscript; JM: acquisition of data, data analysis. FE: acquisition of data, data analysis. KSM: study concept and design, critical revision of manuscript for important intellectual content. PP: study concept and design, data interpretation, critical revision of manuscript for important intellectual content. NJT: senior author, study concept and design, data interpretation, drafting of manuscript, study supervision.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: KSM is the inventor of a biliary biopsy forceps kit that was first licensed in 2013. The rights have been transferred to Cook Medical and he receives minimal royalties for sales of this device.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: All data relevant to the study are included in the article or uploaded as supplementary information.
References
- 1. Chu D, Adler DG. Malignant biliary tract obstruction: evaluation and therapy. J Natl Compr Canc Netw 2010;8:1033–44. 10.6004/jnccn.2010.0075 [DOI] [PubMed] [Google Scholar]
- 2. van Delden OM, Laméris JS. Percutaneous drainage and stenting for palliation of malignant bile duct obstruction. Eur Radiol 2008;18:448–56. 10.1007/s00330-007-0796-6 [DOI] [PubMed] [Google Scholar]
- 3. Saluja SS, Gulati M, Garg PK, et al. . Endoscopic or percutaneous biliary drainage for gallbladder cancer: a randomized trial and quality of life assessment. Clin Gastroenterol Hepatol 2008;6:944–50. 10.1016/j.cgh.2008.03.028 [DOI] [PubMed] [Google Scholar]
- 4. Barkay O, Mosler P, Schmitt CM, et al. . Effect of endoscopic stenting of malignant bile duct obstruction on quality of life. J Clin Gastroenterol 2013;47:526–31. 10.1097/MCG.0b013e318272440e [DOI] [PubMed] [Google Scholar]
- 5. Sultana A, Smith CT, Cunningham D, et al. . Meta-Analyses of chemotherapy for locally advanced and metastatic pancreatic cancer. J Clin Oncol 2007;25:2607–15. 10.1200/JCO.2006.09.2551 [DOI] [PubMed] [Google Scholar]
- 6. Valle J, Wasan H, Palmer DH, et al. . Cisplatin plus gemcitabine versus gemcitabine for biliary tract cancer. N Engl J Med 2010;362:1273–81. 10.1056/NEJMoa0908721 [DOI] [PubMed] [Google Scholar]
- 7. Valle JW, Wasan H, Johnson P, et al. . Gemcitabine alone or in combination with cisplatin in patients with advanced or metastatic cholangiocarcinomas or other biliary tract tumours: a multicentre randomised phase II study - The UK ABC-01 Study. Br J Cancer 2009;101:621–7. 10.1038/sj.bjc.6605211 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Silva MA, Tekin K, Aytekin F, et al. . Surgery for hilar cholangiocarcinoma; a 10 year experience of a tertiary referral centre in the UK. Eur J Surg Oncol 2005;31:533–9. 10.1016/j.ejso.2005.02.021 [DOI] [PubMed] [Google Scholar]
- 9. Health and Social Care Information Centre Hospital episode statistics. Available: www.hscic.gov.uk/hes [Accessed June 2018].
- 10. Nuttall M, van der Meulen J, Emberton M. Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery. J Clin Epidemiol 2006;59:265–73. 10.1016/j.jclinepi.2005.07.015 [DOI] [PubMed] [Google Scholar]
- 11. Office for National Statistics Index of multiple deprivation (IMD), 2007. Available: https://data.gov.uk/dataset/index_of_multiple_deprivation_imd_2007 [Accessed June 2018].
- 12. Stoker J, Laméris JS, van Blankenstein M. Percutaneous metallic self-expandable endoprostheses in malignant hilar biliary obstruction. Gastrointest Endosc 1993;39:43–9. 10.1016/S0016-5107(93)70009-7 [DOI] [PubMed] [Google Scholar]
- 13. Inal M, Akgül E, Aksungur E, et al. . Percutaneous placement of biliary metallic stents in patients with malignant hilar obstruction: unilobar versus bilobar drainage. J Vasc Interv Radiol 2003;14:1409–16. 10.1097/01.RVI.0000096762.74047.A6 [DOI] [PubMed] [Google Scholar]
- 14. Schima W, Prokesch R, Österreicher C, et al. . Biliary wallstent endoprosthesis in malignant hilar obstruction: long-term results with regard to the type of obstruction. Clin Radiol 1997;52:213–9. 10.1016/S0009-9260(97)80275-9 [DOI] [PubMed] [Google Scholar]
- 15. Uberoi R, Das N, Moss J, et al. . British Society of interventional radiology: biliary drainage and stenting registry (BDSR). Cardiovasc Intervent Radiol 2012;35:127–38. 10.1007/s00270-011-0103-4 [DOI] [PubMed] [Google Scholar]
- 16. Coelen RJS, Roos E, Wiggers JK, et al. . Endoscopic versus percutaneous biliary drainage in patients with resectable perihilar cholangiocarcinoma: a multicentre, randomised controlled trial. Lancet Gastroenterol Hepatol 2018;3:681–90. 10.1016/S2468-1253(18)30234-6 [DOI] [PubMed] [Google Scholar]
- 17. Smith AC, Dowsett JF, Russell RC, et al. . Randomised trial of endoscopic stenting versus surgical bypass in malignant low bileduct obstruction. Lancet 1994;344:1655–60. 10.1016/S0140-6736(94)90455-3 [DOI] [PubMed] [Google Scholar]
- 18. Andersen JR, Sørensen SM, Kruse A, et al. . Randomised trial of endoscopic endoprosthesis versus operative bypass in malignant obstructive jaundice. Gut 1989;30:1132–5. 10.1136/gut.30.8.1132 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Hyöty MK, Nordback IH. Biliary stent or surgical bypass in unresectable pancreatic cancer with obstructive jaundice. Acta Chir Scand 1990;156:391–6. [PubMed] [Google Scholar]
- 20. Shepherd HA, Royle G, Ross AP, et al. . Endoscopic biliary endoprosthesis in the palliation of malignant obstruction of the distal common bile duct: a randomized trial. Br J Surg 1988;75:1166–8. 10.1002/bjs.1800751207 [DOI] [PubMed] [Google Scholar]
- 21. Ho CS, Warkentin AE. Evidence-based decompression in malignant biliary obstruction. Korean J Radiol 2012;13 Suppl 1:S56–61. 10.3348/kjr.2012.13.S1.S56 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Hatzidakis A, Adam A. The interventional radiological management of cholangiocarcinoma. Clin Radiol 2003;58:91–6. 10.1053/crad.2002.1139 [DOI] [PubMed] [Google Scholar]
- 23. Speer A, Christopher R, Russell G, et al. . Randomised trial of endoscopic versus percutaneous stent insertion in malignant obstructive jaundice. Lancet 1987;330:57–62. 10.1016/S0140-6736(87)92733-4 [DOI] [PubMed] [Google Scholar]
- 24. Zhao X-qian, Dong J-hong, Jiang K, et al. . Comparison of percutaneous transhepatic biliary drainage and endoscopic biliary drainage in the management of malignant biliary tract obstruction: a meta-analysis. Dig Endosc 2015;27:137–45. 10.1111/den.12320 [DOI] [PubMed] [Google Scholar]
- 25. Saad WEA, Wallace MJ, Wojak JC, et al. . Quality improvement guidelines for percutaneous transhepatic cholangiography, biliary drainage, and percutaneous cholecystostomy. J Vasc Interv Radiol 2010;21:789–95. 10.1016/j.jvir.2010.01.012 [DOI] [PubMed] [Google Scholar]
- 26. Audisio RA, Bozzetti F, Severini A, et al. . The occurrence of cholangitis after percutaneous biliary drainage: evaluation of some risk factors. Surgery 1988;103:507–12. [PubMed] [Google Scholar]
- 27. Nomura T, Shirai Y, Hatakeyama K. Bacteribilia and cholangitis after percutaneous transhepatic biliary drainage for malignant biliary obstruction. Dig Dis Sci 1999;44:542–6. 10.1023/A:1026653306735 [DOI] [PubMed] [Google Scholar]
- 28. Ahn S, Lee Y-S, Lim KS, et al. . Malignant biliary obstructions: can we predict immediate postprocedural cholangitis after percutaneous biliary drainage? Support Care Cancer 2013;21:2321–6. 10.1007/s00520-013-1796-5 [DOI] [PubMed] [Google Scholar]
- 29. Leung JWC, Ling TKW, Kung JLS, et al. . The role of bacteria in the blockage of biliary stents. Gastrointest Endosc 1988;34:19–22. 10.1016/S0016-5107(88)71223-7 [DOI] [PubMed] [Google Scholar]
- 30. Stoker J, Laméris JS, Jeekel J. Percutaneously placed Wallstent endoprosthesis in patients with malignant distal biliary obstruction. Br J Surg 1993;80:1185–7. 10.1002/bjs.1800800941 [DOI] [PubMed] [Google Scholar]
- 31. Health and Social Care Information Centre The quality of nationally submitted health and social care data England, annual report, 2013, experimental statistics. Available: https://files.digital.nhs.uk/publicationimport/pub11xxx/pub11530/second-annu-data-qual-rep-2013.pdf [Accessed July 2019].
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.