Administrative Model for Profiling Hospital Performance on Coronary Artery Bypass Graft Surgery: Based on the Chinese Hospital Quality Monitoring System

Abstract

Background

We aimed to develop an administrative model to profile the performance on the outcomes of coronary artery bypass grafting across hospitals in China.

Methods and Results

This retrospective study was based on the Chinese Hospital Quality Monitoring System (HQMS) from 2016 to 2020. The coronary artery bypass grafting cases were identified by procedure code, and those of 2016 to 2017 were randomly divided into modeling and validation cohorts, while those in other years were used to ensure the model stability across years. The outcome was discharge status as “death or withdrawal,” and that withdrawal referred to discharge without medical advice when patients were in the terminal stage but reluctant to die in the hospital. Candidate covariates were mainly identified by diagnoses or procedures codes. Patient‐level logistic models and hospital‐level hierarchical models were established. A total of 203 010 coronary artery bypass grafts in 699 hospitals were included, with 60 704 and 20 233 cases in the modeling and validation cohorts and 40 423, 42 698, and 38 952 in the years 2018, 2019, and 2020, respectively. The death or withdrawal rate was 3.4%. The areas under the curve were 0.746 and 0.729 in the patient‐level models of modeling and validation cohorts, respectively, with good calibration and stability across years. Hospital‐specific risk‐standardized death or withdrawal rates were 2.61% (interquartile range, 1.87%–3.99%) and 2.63% (interquartile range, 1.97%–3.44%) in the modeling and validation cohorts, which were highly correlated (correlation coefficient, 0.96; P<0.001). Between‐hospital variations were distinguished among hospitals of different volumes and across years.

Conclusions

The administrative model based on Hospital Quality Monitoring System could profile hospital performance on coronary artery bypass grafting in China.

Nonstandard Abbreviations and Acronyms

CCSR

Chinese cardiac surgery registry

DWR

death or withdrawal rate

HGLM

hierarchical generalized linear model

HQMS

Hospital Quality Monitoring System

RSDWR

risk‐standardized death or withdrawal rate

Research Perspective.

What Is New?

• We established the methodology using administrative health data (the Chinese Hospital Quality Monitoring System data) to profile the outcome performance of coronary artery bypass grafting across hospitals in China, and the core is to develop the hospital‐level hierarchical model to obtain hospital‐specific risk‐standardized death or withdrawal rates of coronary artery bypass grafting.

• The model we developed can distinguish the between‐hospital variation well, even for hospitals with small and medium volumes, and it is stable across years.

What Question Should Be Addressed Next?

• The methodology we established could be a template for evaluating quality variation on other cardiovascular procedures or diseases on the basis of administrative data, and such promotion is crucial and urgent to improve the outcome of patients with cardiovascular diseases.

• Quality indicators including more dimensions of the medical quality should be explored and evaluated, referring the methodology established by this study, thus making the quality evaluation based on administrative data more comprehensive in the future.

Coronary artery bypass graft (CABG), an important myocardial revascularization procedure, has been the mainstay of treatment for severe coronary artery disease. ¹ As one of the most common and complex cardiac surgeries, CABG remains a significant part of hospital performance evaluation. ² Previous studies have reported a remarkable improvement in CABG performance in China; nonetheless, most evidence was based on large teaching hospitals. ³ , ⁴ , ⁵ , ⁶ It remains unclear on the quality of the majority of hospitals with small or medium volumes, which is important for achieving an overall quality improvement across China. ⁷

Lack of data is a major challenge for comprehensive hospital quality evaluation across China, and the disparity in resources allocation adds difficulties to establish a national research‐based hospital collaborative network. ⁸ Administrative health care data, which are primarily collected for administrative or billing purposes, have wide coverage and large sample size to enable multiple analyses at either hospital or administrative division level. Nowadays, administrative data play an important role not only in health system management but in health services research ⁹ , ¹⁰ ; for example, they are widely used to profile hospitals regarding multiple disease status, including cardiovascular diseases. ¹¹ , ¹² , ¹³ To monitor the performance of public hospitals, the Chinese National Health Commission established the Hospital Quality Monitoring System (HQMS) in 2011; all the public tertiary hospitals are required to transmit the front page data of inpatient medical records regularly. ¹⁴ The HQMS has gradually perfected and covered nearly all the tertiary hospitals, which makes it possible to conduct detailed quality evaluation on specific disease status across China without much extra work. However, because the HQMS data are different from clinical research data with regards to population, contents, collection methods, and the like, the analysis strategies need to be developed and validated.

In this study, we aimed to take CABG as a typical example to explore appropriate methodologies using HQMS, especially the risk adjustment model for evaluating the hospital performance across China. First, we selected patient‐level risk factors of adverse CABG outcome through multivariable logistic regression. Second, we further established hierarchical regression models to calculate the hospital risk‐standardized outcomes accounting for the patient case mix and, thus, to profile the hospital performance and identify between‐hospital variation.

Methods

The HQMS data are the national health care administrative data, and as the government policy stipulates, it is not permissible to make the raw data publicly available at this time.

Ethics

The institutional review board of our hospital approved this study (No. 2023–2003) and granted a waiver of informed consent for using deidentified administrative database.

Data Resource

This retrospective study used the data of a national administrative database HQMS, which has been described in some previous studies. ¹⁴ , ¹⁵ Briefly, the HQMS was an administrative data set that covered nearly all tertiary hospitals capable of cardiac surgeries in mainland China. Details of the establishment, data collection process, and data quality control of HQMS are provided in Data S1. The HQMS collects accurate, structural information from the front page of inpatient medical records (Data S2), which includes key care processes and outcomes during a hospitalization, including patient demographics, diagnoses, procedures, discharge status, and so on.

Study Population and Cohort Definition

In this study, 699 hospitals with at least 1 CABG surgery between 2016 and 2020 were included. The distribution of those hospitals in each province is shown in Figure S1. We included patients aged ≥18 years who underwent CABG during the study years (defined by admission date). The patients undergoing CABG in HQMS were identified by a procedure code of “36.1” in the primary or 40 secondary procedure codes following the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) procedure code.

To form the modeling cohort and the validation cohort, the CABG cases between the years 2016 and 2017 were randomly divided by 3:1 accordingly. Cases in the years 2018, 2019, and 2020 were then used separately to confirm the model stability across years.

Outcome Measurement

The outcome was in‐hospital death or withdrawal from treatment (hereafter called “death or withdrawal”), as the discharge status recorded in the “Discharge Status” section was “death” or “discharge against medical advice” (Data S2). The “discharge against medical advice” refers to the situation in which the patient is supposed to continue hospitalization according to their disease condition, but the patient or their family requests to be withdrawn from hospital treatment and be discharged for personal reasons regardless of the medical advice. As such, discharge is not decided by the medical staff on the basis of the patient's condition; it is classified as “discharge against medical advice.” ¹⁶ In China, withdrawal from treatment is common because many patients in terminal status are reluctant to die in the hospital or pay for extra treatment, and Chinese governments often take death or withdrawal rate (DWR) as a hospital quality measure. ¹⁷

Candidate Covariates

Although the HQMS was able to record a primary code and up to 40 secondary codes of diagnoses or procedures, but <1% of hospitalizations had more than 15 records, we used only the first 15 secondary diagnosis/procedure codes to identify the comorbidities and surgical conditions that were most reflective of the patient's overall condition in our study, and they were regarded as candidate predictors for modeling. These predictors were selected on the basis of clinical significance reported in other cardiac registry studies such as the CCSR (Chinese Cardiac Surgery Registry) ¹⁸ and previous CABG predictive models, such as SinoSCORE. ¹⁹ Additional variables deemed clinically significant or mentioned in literature reviews were also considered. Consequently, the candidate predictors included (1) basic information: age, sex, marital status, occupation, and ethnicity; (2) comorbidities: diabetes, hypertension, atherosclerosis, angina pectoris, atrial fibrillation, myocardial infarction, heart failure, cerebrovascular disease, peripheral vascular disease, renal failure, chronic obstructive pulmonary disease, pneumonia, mild liver disease, rheumatic disease, cancer, and dementia; (3) previous history: previous percutaneous coronary intervention and previous cardiac surgeries; (4) cardiac status: New York Heart Association class and perioperative critical state (ie, cardiogenic shock, cardiopulmonary resuscitation, ventricular fibrillation/flutter, or intra‐aortic balloon pump implantation before or during hospitalization); and (5) surgical factors: nonelective surgery, combined with other cardiac surgeries. The “nonelective surgery” was defined as the duration between surgical day and admission day was <1 day; “combined with other cardiac surgeries” was defined as having any other major cardiac surgeries within the same day of CABG surgery, including valve surgery, congenital heart surgery, aortic surgery, structural heart disease interventions, and other major cardiac surgeries. The detailed definitions and corresponding diagnosis or procedure codes were listed in Data S3 and S4.

Logistic Model Development and Evaluation

Patient‐level logistic models were established to determine risk factors associated with CABG outcomes, which were further used to adjust for patient case mix when evaluating the performance among hospitals. We first selected the variables in the modeling cohort during the years 2016 and 2017. The dependent variable was death or withdrawal, and the independent variables included all the candidate predictors mentioned above. Backward elimination selection was taken with an exit criterion of 0.05, and the clinical significance was also considered for the determination of final model covariates. We then validated the model in the validation cohort during the years 2016 and 2017, and also the cohorts of years 2018, 2019, and 2020 to confirm the model stability.

The models were evaluated through the following aspects: (1) discrimination: the area under curve (AUC) and the predictive ability defined as the observed outcomes in the lowest and highest deciles determined by estimated model ²⁰ ; (2) calibration: the slope and intercept of a linear regression with predicted DWR as independent variable and the observed DWR as dependent variable, and the calibration plot made by 10 ordered pairs of the mean predicted and observed DWR (and 95% CIs) divided by the deciles of the predicted probability, and furthermore, the Hosmer–Lemeshow test was conducted. Additionally, we compared our newly developed model with SinoSCORE II, ²¹ a risk prediction model for postoperative outcomes of CABG in the Chinese population. We refitted the model with HQMS data and compared the model discrimination and calibration.

Hierarchical Model Development and Risk‐Standardized Outcomes

Considering the patient case mix and clustering effect, a hierarchical generalized linear model (HGLM) with hospitals as random effects was developed to obtain hospital risk‐standardized outcomes. This analytic method can account for both within‐hospital correlation and between‐hospital variation. ¹² , ¹³ In the modeling cohort, the dependent variable was death or withdrawal, and the independent variables were the covariates finally included in the patient‐level logistic model described above. The variable coefficients in the hospital‐level model were compared with the patient‐level model to ensure stable estimates.

Hospital‐specific risk‐standardized death or withdrawal rate (RSDWR) was used as an indicator to evaluate hospital performance in our study. It was calculated as the ratio of predicted to expected DWR of specific hospital through the HGLM, multiplied by the unadjusted average DWR of all hospitals. ²² We obtained the hospital‐specific RSDWRs in both modeling and validation cohorts between 2016 and 2017, and explored the correlation of RSDWRs between these 2 cohorts. The distribution of RSDWRs in the cohorts of year 2018, 2019, and 2020 were also described to demonstrate the temporal trend of hospital quality across years.

We further assessed the quality variation among hospitals of different annual volumes between 2016 and 2020. The hospitals were classified into 4 groups as <5 cases, 5 to 100 cases, 101 to 500 cases, and >500 cases, corresponding to 170, 299, 148, and 82 hospitals, respectively, and the distribution of RSDWRs in each volume group was compared.

Sensitivity Analysis

Because the survival outcomes of patients who were discharged against medical advice were not recorded after hospital discharge, we used clinically recorded in‐hospital death as the outcome measure and repeated the corresponding analyses. Given that it is difficult for any analytical methods to accurately estimate the outcomes of extremely low‐volume hospitals, we reevaluated the hospital quality and between‐hospital variation after excluding hospitals with <5 CABG cases. Moreover, there was a higher‐level division unit as the province, we further established a 3‐level (patient‐hospital‐province) HGLM to estimate the risk‐standardized outcomes and compared with the 2‐level (patient‐hospital) HGLM.

Statistical Analysis

The continuous variables were expressed as mean (SD) or median (interquartile range [IQR]), and the categorical variables were reported as frequency (percentage). The proportions of missing key variables were 0.01% to 0.70% (Table S1); thus, subjects with any missing variables were excluded when developing specific models. Multivariable logistic regression and HGLM were used to establish patient‐ and hospital‐level models (and the patient‐hospital‐province model) described above, respectively. We also calculated required sample size for predictive models according to the methods proposed by Riley et al, ²³ with the outcome rate of 3.4%, and the sample size was 2292. The subjects in the HQMS data set were enough for model establishment. Our study followed the guidelines for studies based on administrative health data described in the Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement. All statistical inferences were performed by 2‐tailed test, and P<0.05 was considered statistically significant. The analyses were conducted with SAS version 9.4 (SAS Institute Inc, Cary, NC).

Results

Study Population and Characteristics

From 2016 to 2020, 203 010 CABG cases in 699 hospitals were included. The mean age of this population was 62.5 (SD, 8.8) years, with 26.5% women and 24.4% had combined surgeries. The overall death rate and DWR were 1.9% and 3.4%, respectively. Detailed patient characteristics of the overall population and different cohorts are listed in Table 1 and Table S2.

Table 1.

Patient Characteristics of Overall CABG Cases in the HQMS Between 2016 and 2020

Variables	Total (N=203 010)
Demographics
Age, mean (SD)	62.5 (8.8)
Female sex, n (%)	53 778 (26.5)
Married, n (%)	193 305 (95.2)
Occupation type, n (%)
Worker	8733 (4.3)
Farmer	46 739 (23.0)
Retired	42 355 (20.9)
Others	105 183 (51.8)
Han ethnicity, n (%)	194 203 (95.7)
Comorbidities, n (%)
Diabetes	60 251 (29.7)
Hypertension	111 867 (55.1)
Atherosclerosis	165 536 (81.5)
Angina pectoris	90 899 (44.8)
Atrial fibrillation	10 015 (4.9)
Myocardial infarction	47 961 (23.6)
Heart failure	91 332 (45.0)
CVD	31 819 (15.7)
PVD	14 491 (7.1)
Renal failure	4329 (2.1)
COPD	8623 (4.2)
Pneumonia	3401 (1.7)
Mild liver disease	9231 (4.5)
Rheumatic disease	574 (0.3)
Cancer	1170 (0.6)
Dementia	202 (0.1)
Previous history, n (%)
Previous PCI	16 795 (8.3)
Previous cardiac surgeries	3671 (1.8)
Cardiac status, n (%)
NYHA III/IV	31 424 (15.5)
Perioperative critical state	10 185 (5.0)
Surgical factors, n (%)
Nonelective surgery	8262 (4.1)
Combined with other cardiac surgeries	49 594 (24.4)
Outcomes, n (%)
Death	3943 (1.9)
Death or withdrawal	6891 (3.4)

CABG indicates coronary artery bypass graft; COPD, chronic obstructive pulmonary disease; CVD, cerebrovascular disease; HQMS, Hospital Quality Monitoring System; NYHA, New York Heart Association; PCI, percutaneous coronary intervention; and PVD, peripheral vascular disease.

Logistic Model Development and Evaluation

Remaining variables in the model after selection included age, sex, marital status, occupation type, atherosclerosis, angina pectoris, atrial fibrillation, myocardial infarction, renal failure, pneumonia, previous cardiac surgeries, NHYA III/IV, perioperative critical state, nonelective surgery, and combined with other cardiac surgeries. The coefficients of the covariates are listed in Table S3.

The AUCs of the logistic model derived from the 2016 to 2017 modeling and validation cohorts were 0.75 (95% CI, 0.73–0.76) and 0.73 (95% CI, 0.71–0.75), and the slopes and intercepts of the linear regression between the predicted DWR and the observed DWR were 0.94 (0.002) and 0.92 (0.002), respectively. The models across the years from 2018 to 2020 also showed good discrimination and calibration (Table 2), as well as stable covariate coefficients (Table S4). The calibration plots for each cohort are listed in Figure 1.

Table 2.

Performance of Patient‐Level Logistic Models in Different Study Cohorts

Study cohort	AUC (95% CI)	Predictive ability* (mean rate of lowest/highest decile)	Calibration indices (slope, intercept)	P value of Hosmer–Lemeshow test
Modeling cohort (2016–2017)	0.746 (0.733–0.758)	0.48, 12.21	0.936, 0.0019	0.0446
Validation cohort (2016–2017)	0.729 (0.706–0.751)	1.09, 12.46	0.922, 0.0023	0.3985
2018	0.764 (0.750–0.778)	0.69, 15.68	0.929, 0.0024	0.0897
2019	0.796 (0.784–0.809)	0.75, 19.04	0.932, 0.0025	0.0549
2020	0.818 (0.806–0.830)	0.72, 21.36	0.916, 0.0033	0.0089

AUC indicates area under the curve.

^*Observed rates in the lowest and highest deciles of predicted rates determined by estimated model. The larger interval indicates the relative better discrimination.

Figure 1. Calibration plots in different study cohorts.

The figure shows the calibration plots in modeling, validation, and different year cohorts. The horizontal axis indicates the deciles of the predicted death or withdrawal rate, and the vertical axis indicates the deciles of the observed death or withdrawal rate. CABG indicates coronary artery bypass graft.

The AUCs of the model with covariates in SinoSCORE II in the modeling and validation cohort were 0.73 (95% CI, 0.72–0.74) and 0.71 (95% CI, 0.69–0.74), respectively, which were all lower than the model newly developed in this study. Similar results were seen in cohorts of other years, indicating our model had better performance than existed models and was more suitable for the HQMS data set (Table S5).

Hierarchical Model Development and Risk‐Standardized Outcomes

The hospital‐level variances of HGLM models in the 2016 to 2017 modeling and validation cohorts were 1.69 (SD, 0.17) and 1.71 (SD, 0.22), respectively. The coefficients of the covariates in the logistic model and HGLM models derived from the modeling and validation cohorts were similar (Tables S3 and S6). The median observed DWR in the modeling cohort was 1.37% (IQR, 0%–5.88%) and 0% (IQR, 0%–4.17%) for the validation cohort. After risk standardization, the median RSDWR were 2.61% (IQR, 1.87%–3.99%) and 2.63% (IQR, 1.97%–3.44%) for the modeling and validation cohorts, respectively. The hospital‐specific RSDWRs in the modeling and validation cohorts were highly correlated with a correlation coefficient of 0.96 (P<0.001). The distributions of observed and risk‐standardized DWRs across different years, stratified by hospital volume groups and in different study cohorts, are displayed in Figure 2, Figure 3, and Table S7, respectively.

Figure 2. Distributions of observed and risk‐standardized hospital‐level death or withdrawal rates in different study cohorts.

The figure shows the hospital‐level observed (blue bins) and risk‐standardized (red bins) death or withdrawal rates in different year cohorts and the entire cohort between 2016 and 2020.

Figure 3. Distributions of observed and hospital‐level risk‐standardized death or withdrawal rates, group by hospital volume.

The figure shows the observed (blue bins) and risk‐standardized (red bins) rates of death or withdrawal in the hospitals of <5 (top left, 170 hospitals), 5 to 100 (top right, 299 hospitals), 101 to 500 (bottom left, 148 hospitals) and >500 (bottom right, 82 hospitals) cases.

Sensitivity Analysis

For models with death as an outcome measure, the AUCs of the patient‐level logistic model in 2016 to 2017 modeling and validation cohorts were 0.80 (95% CI, 0.79–0.82) and 0.76 (95% CI, 0.73–0.79), respectively (Table S8). After risk standardization, the median risk‐standardized death rates were 1.62% (IQR, 1.22%–2.40%) and 1.53% (IQR, 1.28%–1.63%) for the modeling and validation cohorts (Table S7), with a correlation coefficient of hospital‐specific risk‐standardized death rates in these 2 cohorts of 0.51 (P<0.001).

Excluding low‐volume hospitals (<5 cases) increased the discrimination of hospitals (Figure 4, left panel). The median observed and risk‐standardized rates of death or withdrawal of all hospitals across the study years were 3.03% (IQR, 0.00%–8.00%, P₁–P₉₉, 0.00%–100%) and 3.20% (IQR, 2.47–4.68%; P₁–P₉₉, 0.73%–16.92%), respectively. After excluding low‐volume hospitals, the median RSDWR was 3.10% (IQR, 2.21%–4.98%; P₁–P₉₉, 0.69%–17.49%).

Figure 4. Distributions of hospital‐level risk‐standardized death or withdrawal rates calculated by different hierarchical models in different hospital groups.

The figure shows the risk‐standardized death or withdrawal rates calculated by patient‐hospital (left) and patient‐hospital‐province (right) hierarchical models. Blue bins refer to all hospitals, red bins are hospitals of ≥5 cases.

The 3‐level HGLM with hospitals and provinces random effects also had better  hospital classification (Figure 4, right panel). The median RSDWR of all hospitals was 3.25% (IQR, 2.29%–4.77%; P₁–P₉₉, 0.68%–16.35%). After excluding low‐volume hospitals, the median RSDWR was 3.15% (IQR, 2.10%–4.90%; P₁–P₉₉, 0.67%–17.06%).

Discussion

In this study, we developed and validated an administrative data–based model to profile outcome performance and between‐hospital variation on CABG surgeries among hospitals using the Chinese HQMS database. This model took death or withdrawal as the major quality measure on outcome and accounted for the patient case mix and clustering effect by taking the hierarchical analysis method and adjusting for multiple patient factors. This is the first model for hospital profiling of CABG performance on the basis of a national administrative data set in China, and it suggested that the HQMS could act as a reasonable substitute for quality evaluation in certain clinical scenarios.

We conducted multiple analyses to assess the performance of the model. The AUCs of HQMS patient‐level models ranged from 0.73 to 0.82 across different study cohorts. Compared with previous models based on large administrative or registry data, including a CABG 90‐day mortality prediction model based on Medicare claims data (AUC, 0.766–0.772) from the Centers for Medicare and Medicaid Services, ¹¹ and CABG 30‐day mortality prediction model based on the Society of Thoracic Surgeons Adult Cardiac Surgery Database (AUC, 0.79–0.80), ²⁴ the HQMS model demonstrated a comparable performance, indicating the robustness of our current methodology. In addition, we compared the performance of the HQMS patient‐level model with other widely used models, including SinoSCORE II, specialized for postoperative mortality prediction. Results showed that the variables we chose for the HQMS model were preferable given the higher AUC, which further demonstrated that the model we developed with the HQMS was more suitable for adjusting the patient case mix when profiling hospital performance.

Simple and sustainable strategies are needed for persistent medical quality monitoring, evaluation, and feedback. The wide coverage and good data quality without much extra effort in data collection makes the HQMS a unique and ideal source for hospital performance profiling. Similar administrative data such as the Centers for Medicare and Medicaid Services and National Inpatient Sample databases have been used and provide important data support for continuous quality improvement in health care. ²⁵ , ²⁶ However, the target population of the Centers for Medicare and Medicaid Services are patients aged >65 years, and the National Inpatient Sample is a sample population. Instead, the HQMS data set covers all age groups and all tertiary hospitals in China thanks to the government's vigorous promotion, especially including nearly all the hospitals that can carry out complex operations such as CABG. With a heterogeneous population, the HQMS model can still achieve a satisfying performance, indicating a good extrapolation of the model in other clinical scenarios and better application in the future.

There were also problems when using the HQMS for hospital profiling, and we took several strategies to overcome the difficulties. First, clinical information (including patient comorbidities, complications, and treatments) was not recorded in detail as compared with disease‐specific registries or clinical studies using complete electronic health record data. We addressed this by using diagnosis and procedure codes to identify comorbidities from complications. Professional physicians and researchers were involved in the determination of selection rules and ICD codes. Second, in Chinese culture, patients in the terminal stage may be reluctant to die in the hospital or pay for additional treatment and therefore choose to withdraw from care. Considering that we were not able to obtain those patients' life status after their discharge at the current stage, and they were most likely to die in a short time, we consider death or withdrawal from treatment as the outcome measure. This measure may be different from some other research in developed countries, but were more reflective of the situation in China and some Asian countries. Third, CABG volumes varied significantly among hospitals in China, which added difficulties in overall quality evaluation. Therefore, we established the hierarchical model with hospital random effect, which could take the nested data structure and different hospital volume (and corresponding contribution) into account when estimating the overall effects. We further excluded the hospitals with extreme small volumes and added the province‐level random effect in the sensitivity analyses to reevaluate the national CABG performance. These attempts can reduce the effect of tending to the midpoint to some extent. For the overall national evaluation, the results were similar, which suggests that the HQMS model can be a proper and simple way to profile the hospitals, but it also indicates that we can extend the analysis method to other specific situations.

Our findings also suggested that considerable variation existed on CABG performance across hospitals in China, which provided data‐based evidence for the Chinese health care administrators. Nationwide health care evaluation has long been a difficult task, especially in developing countries. Our study shed light on hospital quality profiling using the nationwide administrative database HQMS in China, as well as other developing countries, of which the national electronic health records were not completed. With proper statistical methods and cautious interpretation, administrative data can provide us with new perspectives on the quality disparities and shortcomings to overcome. In this study, we took CABG as a specific clinical scenario, while the HQMS can also be used to evaluate quality of other disease treatments that people commonly received in tertiary hospitals.

This study had some limitations. First, we obtained model covariates from diagnosis and procedure codes recorded by physicians; their experiences and coding habits may cause variations among hospitals. However, we chose the disease status with a relatively clear diagnosis, and the National Health Commission carried out the standardization on the front‐page filling and coding rules; the consistency is improving persistently. Second, the acquisition of previous disease history was based only on the index hospitalization record, which may underestimate the severity of patient conditions and may degrade the performance of corresponding hospitals. However, patients with severe conditions are prone to seeking care in well‐known hospitals, and we are still likely to identify hospitals of poor quality but with lower‐risk patients. Third, although RSDWR or DWR is the most important measurement to evaluate hospital quality, there are other dimensions that we did not include, such as length of stay and cost. These important aspects to reflect medical quality should also be evaluated in future studies by referring to our study methods. Finally, we selected the covariates and outcome measures suitable for CABG; as for the other clinical settings, we suggested referring to the methodology rather than using the model directly.

In conclusion, the HQMS model, with good discrimination, calibration, and stability, was capable of evaluating the hospital performance on CABG and profiling the hospitals across China. In the future, enriched outcome measures such as length of stay and major complications can be adopted with a similar analytic framework, and thus the quality evaluation can be more comprehensive and of greater value for care improvements and policy making.

Sources of Funding

The work was supported by the National High Level Hospital Clinical Research Funding (No. 2022‐GSP‐GG‐28), the Chinese Academy of Medical Sciences Innovation Fund for Medical Sciences (2021‐I2M‐1‐063), and the Capital's Funds for Health Improvement and Research (CFH; no. 2022‐1‐4031).

Disclosures

None.

Supporting information

Data S1–S4

Tables S1–S8

Figure S1