Is the QCI framework suited for monitoring outcomes and costs in a teaching hospital using value-based healthcare principles? A retrospective cohort study

Abstract

Objectives

The objective is to develop a pragmatic framework, based on value-based healthcare principles, to monitor health outcomes per unit costs on an institutional level. Subsequently, we investigated the association between health outcomes and healthcare utilisation costs.

Design

This is a retrospective cohort study.

Setting

A teaching hospital in Rotterdam, The Netherlands.

Participants

The study was performed in two use cases. The bariatric population contained 856 patients of which 639 were diagnosed with morbid obesity body mass index (BMI) <45 and 217 were diagnosed with morbid obesity BMI ≥45. The breast cancer population contained 663 patients of which 455 received a lumpectomy and 208 a mastectomy.

Primary and secondary outcome measures

The quality cost indicator (QCI) was the primary measures and was defined as

QCI = (resulting outcome * 100)/average total costs (per thousand Euros)

where average total costs entail all healthcare utilisation costs with regard to the treatment of the primary diagnosis and follow-up care. Resulting outcome is the number of patients achieving textbook outcome (passing all health outcome indicators) divided by the total number of patients included in the care path.

Results

The breast cancer and bariatric population had the highest resulting outcome values in 2020 Q4, 0.93 and 0.73, respectively. The average total costs of the bariatric population remained stable (avg, €8833.55, min €8494.32, max €9164.26). The breast cancer population showed higher variance in costs (avg, €12 735.31 min €12 188.83, max €13 695.58). QCI values of both populations showed similar variance (0.3 and 0.8). Failing health outcome indicators was significantly related to higher hospital-based costs of care in both populations (p <0.01).

Conclusions

The QCI framework is effective for monitoring changes in average total costs and relevant health outcomes on an institutional level. Health outcomes are associated with hospital-based costs of care.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study included multiple populations with a large number of patients followed over a prolonged period.

• In the study, multiple clinical indicators and patient-reported outcomes were defined to calculate textbook and resulting outcome.

• To create a pragmatic framework, a binary outcome measure was defined. On a patient–physician level, this can lead to an overestimation or underestimation of value.

Introduction

Medical costs have risen rapidly in recent decades, particularly in developed countries. This increase even exceeded the growth of the gross domestic product (GDP),¹ implying that an increasing percentage of GDP has been spent on healthcare. Simultaneously, the gain in life expectancy was marginal and perceived health status remained approximately stable.² Also changing population demographics, such as ageing² and an increasing prevalence of multimorbidity³ and healthcare innovations⁴ contributed to the rise in healthcare costs. Therefore an improved health-related effectiveness evaluation model is required to achieve affordable, high-quality care in the future.

Cost-effectiveness analysis (CEA) is the leading evaluation model to aid priority setting in healthcare budgeting⁵ on the macro, national, level.⁶ CEA quantifies differences in costs and outcomes for each new medical intervention compared with care as usual. CEA value is usually defined as health-related effectiveness, and operationalised using the Quality Adjusted Life Years (QALYs) metric, while costs include healthcare and societal expenditure.⁷

CEA is less suitable for priority setting on a meso, institutional, level.⁶ In CEA, the QALY model includes patients’ life span adjusted for health-related quality of life (HRQoL), which is based on patients’ generic health status.⁷ In daily practice however, these data are not available, nor easily obtained at the institutional level. Also, data regarding social expenditure are neither present nor manageable on an institutional level. Furthermore, results in CEA can be hard to interpret for clinicians and healthcare managers. This makes it difficult to manage healthcare costs and health outcomes using the CEA framework.

In addition, because CEA analyses are based on the patients lifespan, either the remaining lifespan should be estimated, or the analysis can only be performed after the patient is deceased. Combining quality of life and quantity of life in a single metric can then be challenging. Both are different units, one qualitative and the other qualitative, and so are not easily summarised in a single statistic.

In 2006, Porter and Teisberg proposed the concept of value-based healthcare (VBHC).⁸ VBHC describes value as patient-centred health outcomes per unit costs. CEA and VBHC both agree that decision-making in healthcare should be based on a trade-off between health outcomes and healthcare costs using a outcomes/costs ratio.⁹ However, in CEA, health outcomes are based on generic HRQoL measures whereas in VBHC relevant outcomes have been defined as disease or care path-specific indicators. The latter approach makes VBHC more feasible at the institutional level,⁶ because care paths are well defined and disease-specific clinical outcome measures are registered in the electronic medical records (EMR).

For benchmarking between institutions and measuring outcomes within an institution over time, health outcomes need to be standardised.¹⁰ The International Consortium for Health Outcomes Measurement (ICHOM) has therefore published several sets of ‘patient-centred outcome measures’.^11–14

Because health outcomes are anyhow disease specific, they are inherently multidimensional and can vary between patients and over time. Several VBHC studies applied or proposed a multicriteria analysis to summarise multiple outcome indicators into one metric.^{9 15–17} Yet, there is no standardised operational VBHC ‘common metric system’ that aggregates care path-specific outcomes over time. Also for most indicators, there is no clear criterion when such a relevant outcome is either fully achieved, failed or partially achieved. However, if a single VBHC metric produces understandable results, it could support managerial decision-making on an institutional level.

Because current frameworks have a high complexity and limited applicability for priority setting on an institutional⁶ level, we propose a more pragmatic framework that includes costs and outcomes of care paths over time. Such a framework can be used for monitoring care paths, identification of suboptimalities within care paths, and serve as reference for quality improvement programmes and quality improvement reports at the institutional level. Furthermore, we will investigate if the framework can be used for clinical and managerial decision-making and/or quality-of-care assessments in daily clinical practice. Finally, we will analyse which (combination of) health outcomes are associated with increases in healthcare costs.

Methods

Patient population

This proof-of-concept study was a retrospective, real-world cohort study, performed in two use cases, namely: bariatric surgery and breast cancer surgery in Franciscus Gasthuis & Vlietland Hospital, a large medical teaching hospital in Rotterdam, The Netherlands.

Bariatric population

For the bariatric population, all aged ≥18-year older patients diagnosed with morbid obesity (body mass index (BMI) ≥40), treated with gastric bypass or a gastric sleeve resection surgery in 2019 or 2020, were included. Patients who received bariatric surgery in previous years were excluded because it was unclear if/ when a new primary treatment started and the previous treatment stopped.

Breast cancer population

For the breast cancer population, all patients aged ≥18 years older diagnosed with malignant mammary neoplasm and treated with a mastectomy, wide local excision and possibly a breast reconstruction in 2019 or 2020 were included. Patients with stage IV breast cancer at the start of the care path were excluded, because they received palliative care. Also, patients with any breast cancer diagnosis prior to the study were excluded because it was unclear if/ when the new primary treatment started and the previous treatment stopped.

Quality cost indicator (QCI) model

In collaboration with physicians, patient representatives and healthcare managers, we developed a model to support managerial decision-making based on VBHC principles: the QCI model. This model was built on five concepts: textbook outcome (TO) resulting outcome (RO), average total costs (ATC), QCI date and QCI period. Each of these concepts is described below.

Textbook outcome

TO¹⁸ is accomplished when patients meet all health outcome indicators, as defined for a specific care path. For example, survival should and HRQoL can be a part of TO.

Resulting outcome

The RO rate refers to the number of patients who achieved TO divided by the total number of patients included in the care path. RO varies between 0 and 1.

Average total costs

Total costs (TC) are calculated as the sum of the healthcare utilisation costs incurred at the healthcare provider. These costs include the costs of the primary treatment plus any costs following the treatment of symptoms, adverse events or comorbidities of the evaluated patients. The (ATC) equals the TC divided by the total number of patients included in the care path.

QCI date

The TO, RO and ATC parameters are attributed to a QCI date. This is a specific date for each patient, such as the surgery date or date of diagnosis depending on the intervention that will be evaluated.

QCI period

The QCI period is a follow-up period in which the outcomes (TO and RO) and costs (ATC) should be determined. All costs, RO and TO should be considered from the start of the care path (which can occur before the QCI date) until the end of the QCI period. The length of the QCI period can vary according to the goal of the analysis. For example, short-term cost analysis and managerial decisions often require a short QCI period while treatment effectiveness from the patients’ and/ or physicians’ perspective can require a longer QCI period.

With these five concepts in mind, QCI values can be calculated as follows:

$${\displaystyle \begin{array}{l}\mathrm{Q}\mathrm{C}\mathrm{I}=\left(\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{e}\ast 100\right)/\\ \mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{c}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{s}\left(\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{h}\mathrm{o}\mathrm{u}\mathrm{s}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{E}\mathrm{u}\mathrm{r}\mathrm{o}\mathrm{s}\right)\end{array}}$$

Outcome indicators

The following outcome indicators were defined to calculate RO over time. Data regarding outcome indicators were extracted from the EMR.

Bariatric outcomes

Because there is no VBHC standard outcome indicator set for bariatric surgery, outcome indicators were defined by physicians in consultation with patient representatives. This was achieved via flow tables in which patients provided feedback to physicians on what they found high-quality care. Table 1 summarises the outcome indicators for the bariatric population (online supplemental appendix 1 shows the full definitions).

Table 1.

Description of the clinical outcome indicators per population

Patient population	Clinical outcome indicator	Threshold value
Bariatric	Reoperation	If a surgery related to the bariatric treatment was performed within 30 days following the primary surgery, the treatment failed to meet the clinical outcome indicator.
	Deficiency	The mineral and vitamin blood level measure after 9 months and before 21 months (local protocol) closest to 1-year mark post surgery was used as the measure to decide if the patient was deficient. If in this measure any of the blood levels were below the norm level, the patient was classified as deficient and therefore failed the clinical outcome indicator.
	Readmission	If there was an additional unplanned admission related to the primary diagnosis (not as a result of an additional surgery), the treatment failed to meet the clinical outcome indicator.
	Admission time	If admission time of the admission related to the primary surgery exceeded 72 hours, the treatment failed to meet the clinical outcome indicator.
	Emergency department visit (ED)	If there was an ED visit related to the bariatric treatment within 30 days post surgery, the treatment failed to meet the clinical outcome indicator.
	Total weight loss (TWL)	If TWL exceeded 20% within 455 days (local protocol) following the surgery, the clinical outcome indicator was considered successfully passed.
	Disease-specific survival	If a patient passed away during the QCI period due to the primary diagnosis, the treatment failed to meet the clinical outcome indicator.
	HRQoL	HRQoL was measured for the bariatric population using the RAND-36 scale of physical health.30 When the physical health scale 1 year post surgery was improved or a least equal to the physical health scale presurgery, the HRQoL indicator was considered to be successfully passed.
Breast cancer	Reoperation	If a patient received surgery due to infections or bleeding as the result of the primary surgery, the treatment failed to meet the clinical outcome indicator.
	Surgical margins	If a patient received a relumpectomy due to positive surgical margins, the treatment failed to meet the clinical outcome indicator.
	Recurrence	If a patient received a lumpectomy or mastectomy to treat a recurrence, the treatment failed to meet the clinical outcome indicator.
	Disease-specific survival	If a patient passed away during the QCI period due to the primary diagnosis, the treatment failed to meet the clinical outcome indicator.

HRQoL, health-related quality of life; QCI, quality cost indicator.

QCI values for the bariatric population were calculated twice, once including and once excluding the HRQoL indicator. QCI values including HRQoL were only calculated for surgery dates in 2019 due to data availability.

Breast cancer outcomes

Table 1 also shows the clinical outcome indicators for the breast cancer population, which were based on the ICHOM set¹¹ (again online supplemental appendix 1 shows the full definitions). Because patient–reported outcome measures (PROMs) have only been incorporated in breast cancer care since 2021, we were unable to include the HRQoL indicator.

Healthcare utilisation costs

For each patient, TC were calculated as number of activities of care * costs per unit of each activity. For example, the number of MRI scans * costs per MRI scan. Activities of care are specified according to the nationwide Dutch cost price model¹⁹ which covers all hospital-based costs (not reimbursement fees). The Dutch cost price model relates cost per units of specific activities to the diagnosis for which the activity is performed. At one point in time, one activity can be used to treat one diagnosis. Therefore, the primary and follow-up diagnosis needed to be included for all populations. All data regarding healthcare utilisation costs were extracted from the financial module of the EMR.

Bariatric utilisation costs

For the bariatric population, all healthcare utilisation costs based on a morbid obesity diagnosis in the surgery department and on an adiposity/obesity diagnosis in the internal medicine department were included. Moreover, costs related to readmissions or ED visits within the bariatric population were included when one of the diagnoses in online supplemental appendix 1 was present.

Breast cancer utilisation costs

For the breast cancer population, all healthcare utilisation costs based on a malignant mammary neoplasm diagnosis in the surgery department and based on a mammary malignancy diagnosis in the internal medicine department were used.

Reference prices from 2019 were used to calculate costs for both populations throughout the entire QCI period. Finally, for calculating additional costs of expensive medication, average billing prices per medication were calculated and multiplied by the number of times these were administered per patient. These costs were then added to the utilisation costs of the patient to complete the full hospital-based costs of care.

QCI date and period

For all populations, the QCI date was selected as the surgery date. The QCI follow-up period was set at 1 year. To include all costs of the treatment for all patients, all costs from 1 year prior to the QCI date (2019 or 2020) until the end of the QCI period (2020 or 2021) were included.

Outcome categorisation

For analysing the association between outcome indicators and costs, outcome indicators need to contain a minimum number of patients per outcome status (achieved/failed). Because some indicators were failed only a few times, indicators were categorised. The categories were defined such that they contained at least five patients.

Bariatric outcome categories

The bariatric population contained the following outcome indicator categories:

• Admission time after the primary surgery failed.

• Deficiency failed.

• ED visit failed.

• Total weight loss (TWL) failed.

• Other failed.

• TO.

Each category except Other failed category contained patients who only failed the respective indicator, or passed all indicators (TO). The Other failed category contained patients who either failed the reoperation or readmission indicator or a combination of indicators.

Due to differences in outcomes (achieved/failed outcome indicators) between the bariatric population including and excluding HRQoL, the admission time failed category was replaced by the HRQoL failed category, containing patients who only failed the HRQoL indicator, in the population including HRQoL. The Other failed category therefore contained patients who either failed the reoperation, admission time or readmission indicator or a combination of indicators. All other outcome categories remained the same.

Breast outcome categories

For the breast cancer population, the outcome categories were as follows:

• Positive Margins failed.

• Reoperation failed.

• Other failed.

• TO.

Again all categories except the Other failed category contained patients who only failed the respective indicator, or passed all indicators (TO). Patients in the Other failed category either failed the recurrence or survival indicators or failed multiple indicators.

Analysis

Stratification

To increase comparability of QCI values, we adjusted ATC and RO values for patient characteristics using stratification.

Bariatric population

The bariatric population was stratified according to:

• Gender.

• Age (</≥40 years).

• BMI at the start of the treatment (BMI ≥45, or BMI <45).

National guidelines indicated that gender, BMI and age impact the costs and outcomes of the treatment.²⁰ Data of the bariatric population also showed that patients over 40 had relatively higher costs than patients under 40.

Breast cancer population

The breast cancer population was stratified for:

• Age (</≥70 years).

• Tumour, nodule, metastasis score/ductal carcinoma in situ.

• Oestrogen receptor status.

• Human epidermal growth factor receptor 2 status.

As per national guidelines, these patient and disease-specific characteristics indicate whether new (expensive) medication will be administered. These characteristics therefore have a large effect on costs of care.²¹

Case mix adjusted values

For each stratum, the expected RO is the average RO of that stratum over all quarters. For each quarter, the expected RO is the weighted average of the stratum-specific expected RO values in that quarter. The observed RO per quarter is the average observed RO of all the patients in that quarter. The average observed RO is the average RO over all the quarters. Therefore, case-mix adjusted RO values are calculated as follows:²²

$${\displaystyle \begin{array}{ll}\mathrm{A}\mathrm{d}\mathrm{j}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{R}\mathrm{O}=& \mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{d}\mathrm{R}\mathrm{O}\\ & \left(\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{d}\mathrm{R}\mathrm{O}/\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{R}\mathrm{O}\right)\end{array}}$$

The ATC are calculated as follows:

$${\displaystyle \begin{array}{l}\mathrm{A}\mathrm{d}\mathrm{j}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{s}=\mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{d}\mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{s}\ast \\ \left(\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{d}\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{s}/\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{C}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{s}\right)\end{array}}$$

Missing data

Missing indicator data could indicate an increased likelihood of failing or succeeding the indicator. With this in mind, we could not estimate TO for patients with missing indicator data. Therefore, all patients missing one or more clinical indicators were excluded from all analyses.

Supplementary analysis

Because extreme cost outliers may influence QCI values, results were calculated with and without outliers. The cut-off value was selected at P95.

Statistical testing

The Kruskal-Wallis test with an alpha value of 0.05 (two sided) was used to determine whether costs were significantly different across outcome categories within populations.

Patient and public involvement

As previously mentioned, patients were involved in the development of the outcome measures via flow tables. Patients were not involved in the design or conduct of this registry-based study.

Results

Patient population

Bariatric population

In total, 1172 patients had bariatric surgery between 2019 and 2020. Due to missing clinical data, 316 patients (27%) were excluded. Of the 856 remaining patients, 639 had a BMI lower than 45 at the start of the treatment (table 2). Regarding the bariatric HRQoL population, 270 patients were included of which 203 were diagnosed with a BMI lower than 45 at the start of the treatment.

Table 2.

Characteristics of the breast cancer population and bariatric population with and without health-related quality of life (HRQoL) (number of patients and percentages of total)

Bariatric (n=856)
Age, median (IQR)	46 y (34–53) y
Diagnosis	Morbid obesity BMI <45	639	75%
	Morbid obesity BMI ≥45	217	25%
Gender	M	177	21%
	F	679	79%
Surgery type	Bypass	422	49%
	Sleeve	434	51%
Bariatric including HRQoL (n=270)
Age, median (IQR)	46 y (34−52, 75 y)
BMI	Morbid obesity BMI <45	203	75%
	Morbid obesity BMI ≥45	67	25%
Gender	M	57	21%
	F	213	79%
Surgery type	Bypass	136	50%
	Sleeve	134	50%
Breast cancer (n=663)
Age, median (IQR)	63 y (52–72 y)
Cancer stage	0	102	15%
	I	280	42%
	II	236	36%
	III	45	7%
Oestrogen receptor status	Pos	406	61%
	Neg	87	13%
	Unknown	170	26%
Human epidermal growth factor receptor 2 status	Pos	56	8%
	Neg	437	66%
	Unknown	170	26%
Surgery type	Mastectomy	208	31%
	Lumpectomy	455	69%
Neoadjuvant chemotherapy	Yes	122	18%
	No	541	82%
Adjuvant chemotherapy	Yes	148	22%
	No	515	78%

*No large imbalances between patients having and not having missing data for all baseline variables except age were found. Lower age was significantly associated with the likelihood of having missing data (p=0.02).

BMI, body mass index.

Breast cancer population

In 2019–2020, a total of 671 patients underwent breast cancer surgery. Eight patients had cancer stage IV at the start of the treatment and were excluded. Of the remaining 663 patients, 208 received a mastectomy and 455 received a lumpectomy.

Outcome indicators over time

Table 3 describes the crude percentages of patients achieving the health outcome indicator and RO per year quarter and in total within the different populations.

Table 3.

Number (successful/not successful) and percentage of patients achieving the respective health outcome indicator per year, quarter within the different populations

Population	Outcome indicator	Q1 2019	Q2 2019	Q3 2019	Q4 2019	Q1 2020	Q2 2020	Q3 2020	Q4 2020	Total
Baratric care (n=856)	Reoperation	105/2	129/2	90/2	130/3	85/4	54/0	171/2	77/0	841/15
		98%	98%	98%	98%	96%	100%	99%	100%	98%
	Deficiency	83/24	96/34	64/28	89/44	59/30	39/15	127/45	61/16	618/236
		78%	74%	70%	67%	66%	72%	74%	79%	72%
	Readmission	105/2	129/2	90/2	128/5	88/1	54/0	167/6	76/1	837/19
		98%	98%	98%	96%	99%	100%	97%	99%	98%
	Admission time	102/5	125/6	87/5	130/3	86/3	52/2	162/11	74/3	818/38
		95%	95%	95%	98%	97%	96%	94%	96%	96%
	Emergency Department (ED) visit	101/6	121/10	88/4	122/11	81/8	52/2	160/13	74/3	799/57
		94%	92%	96%	92%	91%	96%	92%	96%	93%
	TWL (total weight loss)	104/3	123/8	80/12	122/11	84/5	52/2	165/7	77/0	807/48
		97%	94%	87%	92%	94%	96%	96%	100%	94%
	Survival	107/0	130/1	92/0	133/0	89/0	54/0	172/1	77/0	854/2
		100%	99%	100%	100%	100%	100%	99%	100%	100%
	Resulting outcome * 100%	72/35	81/50	47/45	73/60	50/39	34/20	107/66	56/21	520/336
		67%	62%	51%	55%	56%	63%	62%	73%	61%
Bariatric including HRQoL care (n=270)	Reoperation	41/2	82/0	56/1	86/2	- /-	- /-	- /-	- /-	265/5
		95%	100%	98%	98%	–	–	–	–	98%
	Deficiency	32/11	64/18	41/16	60/28	- /-	- /-	- /-	- /-	197/73
		74%	78%	72%	68%	–	–	–	–	73%
	Readmission	42/1	82/0	55/2	86/2	- /-	- /-	- /-	- /-	265/5
		98%	100%	96%	93%	–	–	–	–	98%
	Admission time	40/3	81/1	56/1	86/2	- /-	- /-	- /-	- /-	263/7
		93%	99%	98%	98%	–	–	–	–	97%
	ER visit	42/1	76/6	53/4	82/6	- /-	- /-	- /-	- /-	253/17
		98%	93%	93%	93%	–	–	–	–	94%
	TWL (total weight loss)	43/0	80/2	52/5	80/8	- /-	- /-	- /-	- /-	255/15
		100%	98%	91%	91%	–	–	–	–	94%
	HRQoL	42/1	80/2	53/4	82/6	- /-	- /-	- /-	- /-	257/13
		98%	98%	93%	93%	–	–	–	–	95%
	Survival	43/0	82/0	57/0	88/0	- /-	- /-	- /-	- /-	270/0
		100%	100%	100%	100%	–	–	–	–	100%
	Resulting outcome * 100%	29/14	56/26	31/26	48/40	- /-	- /-	- /-	- /-	164/106
		67%	68%	54%	55%	–	–	–	–	61%
Breast cancer care (n=663)	Reoperation	79/3	55/6	77/3	75/6	88/7	86/5	85/3	83/2	628/35
		96%	90%	96%	93%	93%	95%	97%	98%	95%
	Survival	82/0	60/1	80/0	81/0	94/1	90/1	87/1	85/0	659/4
		100%	98%	100%	100%	99%	99%	99%	100%	99%
	Surgical margins	75/7	57/4	74/6	74/7	85/10	87/4	76/12	81/4	609/54
		91%	93%	93%	91%	89%	96%	86%	95%	92%
	Recurrance	82/0	61/0	80/0	81/0	95/0	91/0	87/1	85/0	662/1
		100%	100%	100%	100%	100%	100%	99%	100%	100%
	Resulting outcome * 100%	73/9	51/10	72/8	70/11	78/17	81/10	72/16	79/6	576/87
		89%	84%	90%	86%	82%	89%	82%	93%	87%

HRQoL, health-related quality of life.

Bariatric outcomes

In the bariatric population, all clinical indicators scored above 90% except the deficiency indicator (73%). RO was 0.61 (range 0.51–0.73). For the bariatric population including HRQoL RO was 0.61, range (0.54–0.68). During the same period, RO was 0.59 in the bariatric population. The higher RO values in the bariatric population including HRQoL were mainly based on a higher percentage of patients achieving the TWL and deficiency indicators.

Breast cancer outcomes

In the breast cancer population, RO was 0.87, range (0.82–0.93). The surgical margins indicator was least successful, with 92% of patients achieving it. The recurrence indicator was the most successful.

QCI values

Figure 1 displays case-mix adjusted RO (panels 1A, 1B), ATC (panels 2A, 2B) and QCI values (3A and 3B). Supplementary results comparing RO, ATC and QCI values including and excluding cost outliers are attached in online supplemental file 1 (breast cancer) and 2 (bariatric population).

Figure 1.

Case mix adjusted resulting outcome (panels 1A, 1B), average total costs (ATC) (panels 2A, 2B) and quality cost indicator (QCI) values (panels 3A, 3B) over time for both populations. HRQoL, health-related quality of life.

Bariatric RO, ATC and QCI values

For the bariatric population, RO, ATC and QCI values are presented twice, once including the HRQoL indicator and once for the total bariatric population. As presented in panel 1A, RO values of the bariatric population including HRQoL are higher compared with the total bariatric population. Because ATC of both groups are similar (panel 2A), QCI values of the bariatric population including HRQoL are higher in 2019 Q2 and Q3. Overall, because ATC of the bariatric population has a low variance (SD = €230.42), deviations in QCI values are mostly due to changes in RO values. Excluding outliers had little impact on RO and QCI (online supplemental file 1) values for both groups. ATC values were slightly lower (AVG €8388.26 instead of €8833.55) and less variable (a decrease in IQR of €253.31).

Breast cancer RO, ATC and QCI values

The results of the breast cancer population show that QCI (panel 3B) values can be impacted by a combination of both ATC (panel 2B) and RO (panel 1B). For the breast cancer population, this is especially true in 2019 Q3 and 2020 Q1. In 2019, Q3 QCI values were highest due to a combination of low costs and high RO values. The opposite is true for 2020 Q3 where a combination of high ATC and low RO values resulted in the lowest QCI value. Excluding outliers impacted the ATC of the breast cancer population predominantly in 2020 (online supplemental file 2), producing higher average QCI values (8.08 instead of 6.70) in that period.

Costs per population per outcome category

Table 4 describes the characteristics of the cost distributions per outcome category for each population.

Table 4.

Descriptive measures of the total costs per outcome category for each population

Population	Outcome category	Number of patients	Average costs (€)	SD costs (€)	IQR of costs (€)	P value
Bariatric care	Admission time failed	21	11 854.96	4535.65	3465.92	<0.01
	Deficiency failed	192	8452.58	1457.45	1603.95
	ER visit failed	25	8867.37	1778.55	1697.46
	Other failed	66	12 871.99	10 290.17	3915.20
	Textbook outcome	520	8470.70	1450.37	1599.11
	TWL failed	32	8603.64	1448.56	1685.98
Bariatric care including HRQoL	HRQoL failed	6	9267.24	3587.81	1754.82	0.02
	Deficiency failed	54	8404.37	1173.08	1236.08
	ER visit failed	7	8661.00	888.35	1192.50
	Other failed	30	12 189.78	10887.56	2235.78
	Textbook outcome	164	8483.20	1263.51	1536.37
	TWL failed	9	8165.08	1029.86	1733.97
Breast cancer care	Other failed	11	18 231.83	8878.90	12 516.42	<0.01
	Positive margins failed	47	14 772.80	10 234.77	7093.75
	Reoperation failed	29	18 792.08	12 317.22	10 084.12
	Textbook outcome	576	12 277.29	10 273.12	6666.06

P values refer to the overall difference in total costs across outcome categories within populations.

HRQoL, health-related quality of life; TWL, total weight loss.

Costs of care per outcome category for the bariatric population

For the bariatric population, patients in the Other failed and Admission time failed categories had high average costs of care, compared with the other categories. The Other failed category also had the highest SD and IQR. For the bariatric population including HRQoL, patients in the Other failed category had the highest average costs of care whereas patients in the TWL failed category had the lowest average costs. TC by outcome category differed significantly for both populations.

Costs of care per outcome category for breast cancer population

For patients with breast cancer, the average costs of the Reoperation failed and Other failed outcome categories were comparable. Patients in the TO category had the lowest average costs of care. Again TC by outcome category differed significantly.

Discussion

In this proof-of-concept study, we developed the QCI model in collaboration with physicians, patient representatives and healthcare managers based on VBHC principles. The QCI model suits the need for monitoring healthcare costs and health outcomes, and thereby the evaluation and quality assessment of healthcare interventions or quality improvements on an institutional level. The framework provides management information on (patient perceived) health outcomes and costs in a single metric and underlying components, for clinicians and healthcare management.

The results show that the QCI framework is sensitive to changes in ATC and RO. A strength of the QCI framework is that routinely collected cost and outcome data from EMR and PROMs software are directly available for QCI analysis. Furthermore, the QCI framework has a high level of flexibility because health gains can be based on a large variety of outcomes, preferably those outcomes that cover all main health effects associated with the care path. This flexibility in outcomes also makes the QCI framework applicable for a variety of medical conditions. These findings indicate that the framework is suitable for monitoring the performance of care in terms of outcomes and costs in clinical practice, on an institutional level. Using the framework in a plan-do-act-check cycle, with or without using the underlying indicator and cost data, should help evaluate results and guide continued improvement processes.²³

Not all outcome categories equally affected hospital-based costs of care. Failure of outcome indicators with regard to surgery was associated with higher costs. These results can be explained by the direct impact of surgical procedures on hospital-based costs.²⁴ However, no direct impact of the deficiency and TWL indicators on hospital-based costs of care was visible. On the other hand, deficiencies can lead to the development of metabolic bone diseases in the long term²⁵ and obesity is a known risk for coronary heart disease.²⁶ The follow-up period of 1 year might not have been sufficient to investigate the expected increase in cost due to these complications. Investing in adherence to follow-up could result in a higher percentage of patients passing the TWL and deficiency indicators thereby possibly preventing these long-term costs.²⁷ Investing in adherence to follow-up could therefore, in short-term QCI analysis, lead to higher costs and a higher RO. This indicates that short-term QCI values can either improve or decrease depending on whether the gain in RO outweighs the gain in costs. In a long-term QCI analysis, QCI values could improve as additional costs to treat the aforementioned complications could be avoided and RO increases. Different follow-up periods could therefore result in different QCI values.

VBHC, CEA and QCI all agree that decision-making in healthcare should be based on a trade-off between health outcomes and healthcare costs using a outcomes/costs ratio.⁹ However, both VBHC and CEA require complex calculations to summarise outcomes.7 9 15–17 In contrast, in QCI, the RO parameter is an understandable statistic summarising outcomes. Also, in the basis, VBHC and CEA include social as well as direct healthcare costs. QCI, in this analysis, is limited to costs incurred at the healthcare provider. This matches the clinician and managerial perspective. Alternatively, the model could also include out-of-hospital and non-medical costs if this is relevant for hospital management. Finally, both CEA and VBHC recommend analysis of the full cycle of care. In QCI, different follow-up periods are available. Short-run or medium-term analysis can provide valuable information for managerial purposes.

The definition of TO as ‘all clinical outcome indicators are met’ gives healthcare providers a pragmatic method to summarise outcomes. However, a limitation of this binary scoring model is that value is only created when all health indicators are met (1), and no value is created ‘when at least one of the indicators is failed’ (0). In the first case, when all indicators are passed, the patient can still perceive the treatment as less then optimal. In the latter case, the patient can view the treatment (at least partially) as a success. The binary scoring rule could therefore result in an overestimation of value, but is more likely to lead to an underestimation of value. An alternative approach could be to define an outcome measure with a continuous score that varies between completely suboptimal outcome (0) and completely optimal outcome (100), with or without assigning different weights to different outcome indicators. Another limitation is missing data. When data from one indicator is missing and the other indicators are passed, QCI values are not defined. Because the QCI framework can give an underestimation as well overestimation (or no estimation in the case of missing data) of value on the microlevel (physician–patient encounter),⁶ the framework is not applicable at this level. The QCI model can give valuable information on an institutional level as it summarises group outcomes and costs in an understandable fashion.

Areas of future research include defining upper and lower margins for outcome, costs and QCI values to guide continued improvement processes. Because as time passes chances of failing or succeeding an indicator can vary,²⁸ future research should investigate RO and TO values over multiple follow-up periods. Comparing hospital performance using the QCI framework could be helpful for optimising QCI values and underlying costs and outcome results.²⁹ Finally, the QCI framework should be applied to more medical conditions, in chronic and acute settings to study the generalisability of the properties of the QCI model.

Conclusions

This proof-of-concept study showed that the QCI framework is effective for monitoring the performance of care paths in terms of costs and health outcomes on an institutional level. An overall impact of health outcome indicators on hospital-based costs of care is found. Also, some indicators, or combination of indicators, impact costs more than others.

Data availability statement

Data are available upon reasonable request. The datasets used in this study are not publicly available under the Dutch privacy legislation. Anonymised datasets can be made available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Because the study evaluated care as usual a METC review is not obliged under Dutch Legislation. The Institutional Review Board of the Franciscus Gasthuis & Vlietland hospital, named Advies Commissie Wetenschap (ACW) reviewed and waived the study protocol (ref. 2023-002). The study was conducted in accordance with the Declaration of Helsinki exempted this study. The ACW stated that retrospective collection of informed consent was not required for this registry-based study. The effort to do so would be disproportionate due to the number of patients in the study and the fact that the treatment occurred some time ago.