Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators

Abstract

Objective

The aim of the study was to investigate the feasibility of using administrative data to screen adverse events in Korea.

Methods

We used a diagnosis-related groups claims data set and the information of the checklist of healthcare quality improvement (a part of the value incentive program) to verify adverse events in fiscal year 2018. Adverse events were identified using patient safety indicator (PSI) clusters and a present on admission indicator (POA). The PSIs consisted of 19 clusters representing subcategories of adverse events, such as hospital-acquired infection. Among the adverse events identified using PSI clusters, “POA = N,” which means not present at the time of admission, was only deemed as the case in the final stage. We compared the agreement on the occurrence of adverse events from claims data with a reference standard data set (i.e., checklist of healthcare quality improvement) and presented them by PSI cluster and institution.

Results

The cases of global PSI for any adverse event numbered 27,320 (2.32%) among all diagnostic codes in 2018. In terms of institutional distribution, considerable variation was observed throughout the clusters. For example, only 13.2% of institutions (n = 387) reported any global PSI for any adverse event throughout the whole year. The agreement between the reference standard and the claims data was poor, in the range of 2.2% to 10.8%, in 3 types of adverse events. The current claims data system (i.e., diagnostic codes coupled to POA indicators) failed to capture a large majority of adverse events identified using the reference standard.

Conclusions

Our results imply that the coding status of International Classification of Diseases, Tenth Revision, codes and POA indicators should be refined before using them as quality indicators.

The World Health Organization publicly announced global actions on patient safety at the 72nd World Health Assembly in 2019.¹ Although an environment that puts patient safety first has been fostered globally, there is still a way to go in terms of relevant figures on patient safety.² To achieve fundamental changes in patient safety, the first step involves identifying the current status and continuously monitoring it.³ Despite the importance of patient safety, systems for error detection and monitoring have rarely been established in the healthcare field.^4,5 Where patient safety culture is immature, healthcare providers tend to hide patient safety incidents without disclosing them.⁶ Therefore, healthcare systems have adopted a nonpunitive, confidential, and voluntary patient safety reporting system, to collect information on patient safety incidents including near-miss events.⁷ However, the information obtained from the voluntary reporting system is insufficient to understand the actual situation regarding patient safety because incidents are usually underreported.⁷

Medical record reviews have been used as a measure of the incidence of adverse events.⁸ Efforts have been made to use medical records to identify adverse events in various countries, since the Harvard Medical Practice Study estimated the incidence of adverse events using a 2-stage retrospective record review.⁸ Although a review of medical records is regarded as the criterion standard for assessing adverse events, such a review is associated with several concerns as follows: (1) reliability of record review, (2) variations in adverse events between institutions, (3) money and time spent on the review process, and (4) variations in the measurement of adverse events depending on the quality of medical record review.^9–11 Therefore, various complementary methods are needed to identify adverse events. Administrative data are one potential alternative source of data to capture adverse events. Diagnostic codes in administrative data have been used to monitor adverse drug reactions and other patient safety incidents or to estimate the burden of disease attributed to patient safety incidents.^12,13

When administrative data are used to assess adverse events, the detection of cases depends on the accuracy of diagnostic codes. Therefore, in previous studies, the validity of diagnostic codes was assessed by different methodologies. Ackroyd-Stolarz et al¹⁴ suggested the high validity of administrative data for detecting adverse events, although contradictory evidence was also presented.^15,16 Information on the timing of diagnosis, which is essential to differentiate between adverse events and comorbidity, is not collected in conventional administrative data.¹⁷ Accordingly, present on admission (POA), a diagnosis-timing flag, was introduced to indicate whether the diagnosis was POA.¹⁸ However, few studies evaluating patient safety status using diagnostic codes and POA indicators have been performed.^19,20 The validity of administrative data including diagnostic codes and POA indicators needs to be further investigated before using them as a source of data to measure adverse events.

In the Republic of Korea (hereinafter Korea), efforts have been made to assess adverse events under the prospective payment system (PPS) using the diagnosis-related groups (DRGs) for seven disease groups (i.e., lens operation, tonsillectomy/adenoidectomy, appendectomy, inguinal/femoral hernia operation, hemorrhoidectomy, uterine/adnexa operation, and cesarean delivery).^21,22 The Korean DRG payment for 7 disease groups was mandated and expanded to all healthcare providers since 2013, after introducing a voluntary model in 2002. In 2018, the DRG payment comprised 7.1% of total inpatient claims and 5.7% in total medical expenses.²³ The Healthcare Insurance Review and Assessment Service (HIRA) has collected information about adverse events, such as physical accidents, healthcare-associated infections, and postoperative complications, during admission using the individual checklist of healthcare quality improvement (CHQI) for patients in the system of the value incentive program (VIP).²⁴ The accuracy of the CHQI is evaluated by comparing it with medical records in identifying adverse events.²⁴ In the current VIP scheme of DRG/PPS, there are no incentives or disincentives regarding identified adverse events from the CHQI. In addition, the HIRA has required that healthcare providers report POA indicators for the DRG/PPS payment since 2012.²⁵

We investigated whether diagnostic codes combined with POA indicators in the DRG/PPS claims data could be used to detect adverse events.

METHODS

Data Sources

We used health insurance claims data for inpatients collected through the DRG/PPS by the HIRA in fiscal year 2018. The diagnostic codes in the HIRA database are based on the Korean version of the International Statistical Classification of Diseases, Tenth Revision (ICD-10). In addition to the diagnostic codes, the POA indicators are required to differentiate whether the conditions existed before admission within the DRG/PPS. We used the information extracted from the CHQI in the DRG/PPS as a reference standard to identify adverse events.

Capturing Potential Adverse Events Using Diagnostic Codes From Claims Data

We captured candidate codes for adverse events by screening diagnostic codes with the patient safety indicators (PSIs) devised by Southern et al,¹⁹ multiple code clusters identifying potential complications associated with quality of care, from the claims data set.²⁰ The PSIs have 19 clusters representing adverse events, and each cluster consists of relevant diagnostic codes based on the International Statistical Classification of Disease, Tenth Revision (ICD-10-CA, Canada). The PSI clusters are as follows: (1) global PSI for any adverse event, (2) hospital-acquired infection, (3) decubitus ulcer, (4) endocrine and metabolic complications, (5) venous thromboembolic events, (6) cardiac complications, (7) respiratory complications, (8) hemorrhagic events, (9) drug-related adverse events, (10) adverse events related to fluid management, (11) obstetrical complications affecting mother, (12) obstetrical complications affecting fetus, (13) complications directly related to surgery, (14) traumatic injuries arising in hospital, (15) anesthesia-related complications, (16) delirium, (17) central nervous system complications, (18) gastrointestinal, and (19) severe events proximally threatening to life or to major vital organs. Taking into account the international transferability of ICD-10 codes, we only used the first 4 digits of the diagnostic codes.²⁶

Confirming Adverse Events Using POA Indicators From Claims Data

In addition to diagnostic codes, we used a POA indicator to differentiate between complications occurring during hospital admission and conditions existing before admission.¹⁸ The POA indicator was coded as follows: (N) not present at the time of admission, (Y) present at the time of admission, (U) insufficient information, and (W) clinically undetermined.²⁵ Among the diagnostic codes screened with PSIs, the conditions flagged as “POA = N” are considered potential adverse events. On the other hand, the codes flagged as “POA = Y” represent preexisting comorbidities, which are unlikely to be adverse events.

Estimating Incidence of Adverse Events for Each PSI Cluster and Its Institutional Distribution

The number of confirmed codes for adverse events, using diagnostic codes combined with POA indicators, was transformed to a claims unit (annual incident cases) depending on PSI clusters. Then, the number of cases (numerator) was divided by the number of claims at risk (denominator). The denominators were calculated taking into account the characteristics of each PSI cluster. While the PSI cluster of obstetrical complications affecting mother was subjected to claims of females for the denominator, infants younger than 1 year were counted for the PSI cluster of obstetrical complications affecting fetus. For the other PSI clusters, the total number of claims in fiscal year 2018 was used for the denominator to calculate the annual incidence probability (%). We also showed the institutional distribution of incidence by the frequency of adverse events with graphs. The count of adverse events in a claims unit was calculated by institution. Then, the incident case numbers of individual institutions were plotted as a bar graph. In terms of the global PSI for any adverse event, we described overall proportions in addition to individual proportions according to the types of institutions.

Selection of Adverse Events for Reference Standard From the CHQI

The CHQI consists of 3 sections (preoperative, during admission, postoperative) and 14 individual items.²⁴ Three of the 14 items of the CHQI were chosen as a reference standard considering comparability with the PSI clusters: (1) all physical accidents occurring during hospitalization, irrespective of the patient’s status (e.g., falls); (2) anesthesia-related complications increasing morbidity or mortality, except for adverse events attributed to surgical procedures; and (3) healthcare-associated infections neither present nor incubating at the time of admission. The healthcare-associated infections include surgical site infections and the other infections except for surgical site infections. The criteria for nonsurgical site infections are as follows: fever (temperature >38.3°C), purulent discharge; pyuria, or positive bacterial culture test from blood, urine, or discharge samples.²⁴ The following 3 PSI clusters corresponded to the selected CHQI items: traumatic injuries arising in hospital, anesthesia-related complications, and hospital-acquired infections.

Agreement Between Reference Standard and Claims Data

The agreement on the occurrence of adverse events was evaluated by comparing reports from the CHQI and the claims data using ICD-10 diagnostic codes combined with POA indicators (Fig. 1). First, an adverse event of interest (physical accidents, anesthesia-related complications, and healthcare-associated infections) was identified from the CHQI. Second, we matched the cases of adverse events identified from the CHQI to the claims data. Then, we investigated whether the cases of adverse events have the diagnostic codes with the corresponding PSI clusters. For example, a case of healthcare-associated infection has true agreement when the case has a code designated in the PSI hospital-acquired infections, such as A080 (rotaviral enteritis). In addition, the codes flagged “POA = N” are accepted as proper code practice for adverse events in the claims data. The agreement between the CHQI and claims data is presented as frequency and proportion. This research was approved by the institutional review board of Asan Medical Center (2021–0507).

FIGURE 1.

Flow diagram identifying code validity from the CHQI and claims data.

RESULTS

Estimated Adverse Event Incidence in DRG Payment System

In 2018, the total numbers of insurance claims and disease codes were 1,175,155 and 2,179,284 in the DRG payment system, respectively. In total, 2936 institutions submitted claims. The distribution of institutional types is as follows: tertiary hospitals, 42 (1.43%); general hospitals, 278 (9.47%); hospitals, 395 (13.45%); the other types including clinics, 2221 (75.65%). Among 19 PSI clusters, 5 categories (global PSI for any adverse event, hospital-acquired infections, hemorrhagic events, obstetrical complications affecting mother, complications directly related to surgery) showed relatively high frequencies (Table 1). The cases of global PSI for any adverse event numbered 27,320 (2.32%) among all collected diagnostic codes in 2018. In 8 categories, such as cardiac complications, a moderate level of frequency was reported of between 10 and 100. The other 6 categories including decubitus ulcer showed very low incidences. In particular, there were no reports in the clusters on the following: decubitus ulcer, endocrine and metabolic complications, and obstetrical complications affecting fetus.

TABLE 1.

Proportions of PSI Clusters

Cluster	Case, n	Proportion, %*
Global PSI for any adverse event	27,320	2.32
Hospital-acquired infections	214	0.02
Decubitus ulcer	0	0.00
Endocrine and metabolic complications (electrolyte abnormalities, diabetes, etc.)	0	0.00
Venous thromboembolic events	8	0.00
Cardiac complications	30	0.00
Respiratory complications	35	0.00
Hemorrhagic events	26,322	2.24
Drug-related adverse events	43	0.00
Adverse events related to fluid management	1	0.00
Obstetrical complications affecting mother (for females only)	10,094	1.41
Obstetrical complications affecting fetus (for age <1 y only)	0	0.00
Complications directly related to surgery	677	0.06
Traumatic injuries (nonprocedural) arising in hospital	86	0.01
Anesthesia-related complications	90	0.01
Delirium	6	0.00
Central nervous system complications	43	0.00
Gastrointestinal	71	0.01
Severe events proximally threatening to life or to major vital organs	32	0.00

*The denominator is the total number of DRGs claims in 2018 (N = 1,175,155) except 2 clusters (obstetrical complications affecting mother, n = 715,709; obstetrical complications affecting fetus, n = 1977).

Frequency of Adverse Events by Institution From the Claims Data

The distribution of the incidence of global PSI for any adverse event by institution is described in Figure 2. Among 2936 institutions, no global PSI for any adverse event was reported in a majority of them. The proportion of institutions reporting any global PSI for any adverse event was only 13.2% (n = 387). A total of 86.8% of the institutions (n = 2549) did not report any global PSI for any adverse event throughout the whole year. Figure 2 also shows the variation between institutional types (tertiary hospital, general hospital, hospital, and the others including clinics). In particular, the proportions of institutions reporting no incidence were lower in upper types of institutions such as tertiary hospitals.

FIGURE 2.

Distribution of global PSIs for any adverse event by institution.

Large variations between institutions were also observed in the identification of other adverse events from the claims data using PSIs and POA indicators (Supplementary Fig. S1A–1O, http://links.lww.com/JPS/A440). The distributions of individual PSI categories were skewed to the right, which means that most institutions reported no incidence. In particular, adverse events were identified in only a few institutions in the PSI clusters as follows: venous thromboembolic events (n = 8) and adverse events related to fluid management (n = 1).

Agreement Between Reference Standard and Claims Data

The count of cases of traumatic injuries arising in hospital from the CHQI was 95, but only 7.4% of cases (n = 7) gained validity with proper diagnostic codes from the claims data (Table 2). The distributions of the POA indicators were as follows: N, 42.9% (n = 3); Y, 57.1% (n = 4). In terms of anesthesia-related complications, 45 cases were identified in the CHQI, whereas only 2.2% of cases (n = 1) had associated diagnostic codes from the PSIs. The POA indicator of this 1 case was N, indicating occurrence after admission. In total, 688 cases were identified as hospital-acquired infections from the CHQI. Among them, 10.8% of cases (n = 74) were coded with diagnostic codes listed in the category of hospital-acquired infections. Whereas 41.9% of diagnostic codes (n = 31) were flagged as Y, 58.1% of them (n = 43) were obtained after admission (POA = N).

TABLE 2.

Agreement Between the CHQI and Claims Data

Type of Adverse Event	CHQI*	Code†	POA indicator
			N‡	Y	Others
Traumatic injuries arising in hospital	95	7 (7.4)	3 (42.9)	4 (57.1)	0 (0.0)
Anesthesia-related complications	45	1 (2.2)	1 (100.0)	0 (0.0)	0 (0.0)
Hospital-acquired infections	688	74 (10.8)	43 (58.1)	31 (41.9)	0 (0.0)

Values presented as n or n (%).

*Adverse events identified from the CHQI.

^†Cross-checking between the CHQI and claims data (ICD-10 codes) using PSIs.

^‡Cross-checking between the CHQI and claims data (ICD-10 codes coupled to POA indicators).

N, flagged N; Y, flagged Y.

DISCUSSION

We evaluated the feasibility of capturing adverse events from insurance claims data using ICD-10 codes coupled to POA indicators. Our results described the current status of adverse events captured via the claims data of DRG/PPS in Korea. Several important results were obtained in terms of the use of administrative data for identifying adverse events in the current claims system.

First, the incidence of global PSI for any adverse event (2.32%) was lower than in previous studies using record reviews in other countries, which ranged from 3.76% to 20.82%.²⁷ The incidence of overall adverse events from our study was still lower even when compared with a study using discharge data in Canada (5.20%).¹⁹ A recent study using record reviews reported an incidence of adverse events of 10.70% in Korea.¹¹ On the other hand, a previous study identifying adverse events using specific ICD-10 codes (Y codes) reported lower incidence (0.20%) than in our results.²⁸ Although the incidence of adverse events from this study was higher than the figure reported using Y codes, our results seem to be an underestimate compared with a study using medical record review, the criterion standard for evaluating adverse events.

Notably, there was significant variation between PSI clusters, as shown in Table 1. In several PSI clusters including endocrine and metabolic complications, zero incidence was observed. The limited coverage of DRG/PPS, usually applying to 7 minor surgeries (lens operation, tonsillectomy/adenoidectomy, appendectomy, inguinal/femoral hernia operation, hemorrhoidectomy, uterine/adnexa operation, and cesarean delivery), might explain the markedly low incidence in most clusters.²² On the other hand, comparatively substantial numbers in several clusters (obstetrical complications affecting mother and hemorrhagic events) might be attributed to the limited coverage of DRG/PPS. Considering that inpatient newborn care is covered by the fee-for-service instead of the DRG/PPS, zero incidence seems to be due to the limited data source covering only a part of claims data. Nevertheless, our results, no incidence for some clusters in a year, raise doubts about the validity of codes for claims data to monitor adverse events. Considering the possibility of unstable glucose control after general surgery, no incidence in the cluster of endocrine and metabolic complications does not seem to reflect good quality of care.²⁹ Furthermore, variation in incidence between institutions was observed throughout the PSI clusters (Fig. 2, Supplementary Figure S1, http://links.lww.com/JPS/A440). The variations between institutional types imply that small-scale hospitals need more support to improve the validity of the claims data. Overall low incidence in PSIs might reflect the absence of claim coding validation systems. Patient safety indicators obtained from claims data are used for evaluating the quality of healthcare in the United States.³⁰ Considering the validity of ICD-10 codes and POA indicators, close attention should be paid to the introduction of PSIs for evaluating healthcare quality in Korea. Before the utilization of PSIs, the validity of codes should be ensured.

This study has significant features in terms of the methodology and practical use of PSIs. We attempted to improve the validity of PSIs by using POA indicators to identify the time of diagnosis, in addition to the ICD-10 code. The value of POA was stressed to rule out false-positive cases from potentially preventable complications captured using PSIs from the administrative data.³¹ In addition, we used the CHQI as a reference standard, which currently provides the best available data to identify adverse events. Healthcare providers are obligated to report the CHQI in the process of claims, and the validity of the CHQI is assessed via medical record reviews in the VIP scheme of DRG/PPS.²⁴ Therefore, we assumed that the CHQI is an optimal reference standard due to monitoring the quality of the CHQI reporting.

Finally, we described the distribution of PSIs depending on the institution. Considering that PSIs are used in the value-based payment such as hospital value-based purchasing program by the Centers for Medicare and Medicaid Services, institutional variations from our results imply that the current claims data of Korean DRG/PPS are imperfect to capture adverse events and inappropriate to use in value-based payment models.

This study has limitations based on the research methods. First, we used DRG/PPS claims data covering only 7 diseases because of the availability of POA indicators and accessibility to the data of the CHQI. Therefore, we cannot generalize our results to the current status of adverse events or coding practice in Korea because of the limitation of the data source. Nevertheless, the zero incidence in several PSI clusters including endocrine and metabolic complications suggested the need to improve the quality of coding practice of diagnostic codes and POA indicators. In further studies, adverse events should be identified using an expanded data set, ensuring generalization that the findings can be extended to the whole population of Korea. Second, we calculated the incidences of PSIs using claim counts instead of episode counts because of the lack of information about personal identity, which is not accessible because of measures to ensure the anonymity of subjects. Therefore, the incidence might be slightly overestimated. Thirdly, we used the PSIs developed based on ICD-10-CA because there are no indigenous PSIs in Korea. Therefore, we should consider the discordance of ICD-10 codes between countries when interpreting the results, although we only used the first 4 digits of codes in light of the international transferability of ICD-10. Because of the difference of healthcare environment between countries, we expect that a set of PSIs based on the Korean healthcare system will be developed. Finally, we only used claims data to capture adverse events to focus on the feasibility of capturing adverse events from claims data. However, other data sources such as surveillance (e.g., Korean National Healthcare–associated Infections Surveillance System, KONIS) or electronic medical records also can be used for patient safety evaluation according to specific purposes. The KONIS, a nationwide surveillance network, is optimal source to screen national level healthcare-associated infection.³² On the other hand, the electronic medical records data have more clinical information, although the data usage is limited to the institutions because of different structures of the data between them.

CONCLUSIONS

We estimated the incidence of adverse events using ICD-10 codes coupled to POA indicators in the claims data set of DRG/PPS and evaluated the code validity using the reference standard. Our results imply that the validity of the method of ICD-10 coupled with POA indicators should be prioritized before using claims data to identify adverse events. Efforts to improve the validity of the claims data should now be made, such as via training and monitoring of ICD-10 codes and POA indicators.