Abstract
The original pediatric complex chronic conditions classification system developed in 2000/2001 is the gold standard in classifying children with life-limiting illnesses. It was significantly modified in 2014; yet the two systems have not been evaluated. The objective of this study was to evaluate the agreement and validity of the original versus modified complex chronic conditions classification systems. HCUP KID data from 2012 was used with a sample of infant decedents less than 1 years. The agreement (i.e. Cohen’s Kappa Statistic) and validity (i.e. sensitivity, specificity, and positive predictive value) statistics were calculated. Among the 10,175 infants, the modified system performed well in identifying infants who had a complex chronic condition and captured infants that the original classification did not. The modified system represents an improvement over the original, but additional testing is warranted.
Keywords: complex chronic conditions, infants, HCUP KID, Sensitivity, Specificity, positive predictive value
Until recently, the pediatric complex chronic conditions (CCC) classification system has been the accepted method of classifying children with life-limiting illnesses. The system was originally developed by Feudtner and colleagues in 2000/2001 (Feudtner, Christakis, & Connell, 2000; Feudtner et al., 2001). CCCs are medical conditions that would reasonably be expected to last at least 12 months and involve either several organ systems or 1 organ system requiring specialty pediatric care (Feudtner et al., 2000). Feudtner et al. (2000, 2001) created the classification system of diseases based on cause of death information, using 1980 to 1997 Washington State data from pediatric decedents under 19 years. To create the classification, International Classification of Disease, Ninth Revision (ICD-9) codes most commonly associated with pediatric mortality were grouped into 9 categories with 30 corresponding subcategories (Table 1). Specific ICD-9 codes for each subcategory were provided.
Table 1.
Original and Modified CCC Categories and Subcategories
| Categories | Original CCC Subcategories | Modified CCC Subcategories |
|---|---|---|
| Neurologic & Neuromuscular | Brain/spinal cord malformation, mental retardation disability, CNS disease, infantile cerebral palsy, epilepsy, muscular dystrophy/myopathies | Brain/spinal cord malformation, mental retardation disability, CNS disease, infantile cerebral palsy, epilepsy, other CNS disorders, occlusion of cerebral arteries, muscular dystrophy/myopathies, movement diseases, devices |
| Cardiovascular | Heart/great vessel malformations, cardiomyopathies, conduction disorders/dysrhythmias | Heart/great vessel malformations, endocardium diseases, cardiomyopathies, conduction disorders, dysrhythmias, other, devices, transplantation |
| Respiratory | Respiratory malformations, chronic respiratory disease, cystic fibrosis | Respiratory malformations, chronic respiratory disease, cystic fibrosis, other, devices, transplantation |
| Renal & Urologic | Congenital abnormalities, chronic renal failure | Congenital abnormalities, chronic renal failure, other, chronic bladder disease, devices, transplantation |
| Gastrointestinal | Congenital anomalies, chronic liver disease/cirrhosis, inflammatory bowel disease | Congenital anomalies, chronic liver disease/cirrhosis, inflammatory bowel disease, other, devices, transplantation |
| Hematologic or Immunologic | Sickle cell disease, hereditary anemias, hereditary immunodeficiency, HIV | Hereditary anemias, aplastic anemias, hereditary immunodeficiency, coagulation/hemorrhagic, leukopenia, hemophagocytic syndromes, sarcoidosis, HIV, polyarteritis nodosa/related conditions, diffuse disease of connective tissue, other, transplantation |
| Metabolic | Amino acid metabolism, carbohydrate metabolism, lipid metabolism, storage disorders, other metabolic disorders | Amino acid metabolism, carbohydrate metabolism, lipid metabolism, storage disorders, other metabolic disorders, endocrine disorders, devices |
| Other Congenital or Genetic Defect | Chromosomal abnormalities, bone/joint abnormalities, diaphragm/abdominal abnormalities, other congenital abnormalities | Chromosomal abnormalities, bone/joint abnormalities, diaphragm/abdominal abnormalities, other congenital abnormalities |
| Malignancy | Neoplasms | Neoplasms, transplantation |
| Premature & Neonatal | Fetal malnutrition, extreme immaturity, cerebral hemorrhage at birth, spinal cord injury at birth, birth asphyxia, respiratory diseases, hypoxic-ischemic encephalopathy, other | |
| Miscellaneous | Devices, transplantation |
Note: CNS = central nervous system; HIV= Human Immunodeficiency Virus
Classification Systems of Pediatric Complex Chronic Conditions
Over the past two decades, the original classification system has been frequently utilized in pediatric nursing research. It has been employed to identify a study sample, especially in end-of-life research because CCCs are derived from cause of death information (Ananth, Melvin, Feudtner, Wolfe, & Berry, 2015; Feinstein, Feudtner, & Kempe, 2014; Hudson et al., 2014; Golden & Nageswaran, 2012;). The classification system has also been used to support the development of study measures: illness severity and multimorbidity (Keele, Keenan, Sheetz, & Bratton, 2013; Lindley, 2017; Lindley & Keim-Malpass, 2017). Finally, the system has been utilized in the development of pediatric indices (daFonseca & Ferreira, 2014). Thus, the original CCC classification system is considered the gold standard in identifying children with life-limiting illness.
The pediatric CCC classification underwent a modification in 2014 (Table 1; Feudtner, Feinstein, Zhong, Hall, & Dai, 2014). Feudtner and colleagues revised the original CCC system because ICD-10 had replaced the ICD-9 codes, the original system lacked categories for conditions originating in the neonatal period, and the original did not include information about medical technology utilization or organ transplantation, which would indicate CCC status. The modified CCC classification system added a 10th category and revised/expanded original subcategories. For example, the Premature and Neonatal category was added and codes for medical devices and transplantation were included in most categories. The authors provided conversions tables from ICD-9-CM to ICD-10-CM, along with programming language in SAS and STATA to assist in creating the modified CCC categories and subcategories. The ICD conversation information has been widely utilized; however, research utilization of the new modified system is limited (Khan et al., 2018).
Despite the availability of two pediatric CCC classification systems, there has been a lack of rigorous, head-to-head comparison between the performance of the original and modified systems. In creating the modified CCC classification, Feudtner and colleagues (2014) conducted a basic assessment of the two systems. They compared the proportion of modified CCC categories to the original categories using the 2009 Healthcare Cost and Utilization Project – Kids’ Inpatient Database (HCUP- KID) and 2010 Nationwide Emergency Department Sample (NEDS) data files. The results indicated that the modified system classified more children as having a CCC than the original system with exceptions in the malignancy and cardiovascular CCC categories. The largest increase was in gastrointestinal CCC, followed by neurologic and neuromuscular. In contrast, they found that congenital CCC had no change between the original and modified CCC classification systems. From a recent investigation using 2006 to 2014 New York City hospital data, researchers evaluated the performance of the original pediatric CCC system versus the Charlson/Deyo comorbidity score to predict mortality and length of stay (Hessels, Liu, Cohen, Shang, & Larson, 2018). These researchers found that the original system performed better at predicting both mortality and length of stay, compared to the Charlson/Deyo. Although these findings are sparse, they suggest that the two CCC systems may be different. The lack of consistent and generalizable evidence about the classification systems is a significant gap in knowledge, especially at end of life.
Understanding the performance of the pediatric CCC classification systems is important and timely in advancing the nursing science of pediatric end-of-life care (Berry, Hall, Cohen, O’Neill, & Feudtner, 2015). Given the increasing health complexity of children at end of life, improving knowledge about classifying children is critical to advancing the science. Although the inclusion of different and/or additional diagnoses in the modified CCC system may suggest that it is an improved classification system, a systematic evaluation with rigorous analytical methods will guide researchers in understanding why one system might be better than another and in what manner. Because the pediatric CCC classification system is commonly used in research sample definition and measurement of health, generating evidence about the best approach to classify children will ensure that the nursing science is robust. This work will ultimately enable nurse researchers to select the most appropriate classification to identify children, and research using these classifications will improve the quality of pediatric end-of-life care delivered.
Purpose
The purpose of the study was to assess the agreement and validity of the original and modified CCC classification systems. Based on prior research experience with the CCC classifications systems, it was hypothesized that the systems would be significantly different from each other in identifying infants with a CCC because of the addition of categories and sub-categories to the modified version. Therefore, the study aim was to evaluate the agreement (i.e. Cohen’s Kappa Statistic) and validity (i.e. sensitivity, specificity, and positive predictive value) of the modified version of the pediatric CCC classification systems as compared to the original in a national sample.
Methods
Design and Data Source
This non-experimental study used inpatient records for pediatric decedents with the 2012 Kids’ Inpatient Database (KID), Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality (AHRQ). The HCUP KID files are the largest multistate, nationally representative database of US hospitalizations for children and include data from 4179 acute care hospitals in 44 states. KID data are provided at the hospital discharge level and identifiable data was removed from the discharge summaries by HCUP. Each record includes up to 25 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes, demographic data, procedure codes, and payment information. ICD-9-CM codes were used for this study because the 2012 HCUP KID was created prior to the ICD-10-CM conversion. To produce national estimates of inpatients hospitalizations for children, the data include a weight variable for each observation. This study was approved by the Institutional Review Board of the University of Tennessee, Knoxville.
Participants
Given that the modified CCC classification system focused on infant-specific revisions, the sampling frame was limited to infants less than 1 year (Lindley & Fortney, 2019). The sample was restricted to those infants, who had a discharge disposition of death in the hospital occurring from January 1 to December 21, 2012. An International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) that indicated external cause of injury or poisoning were excluded. Observations with any missing or invalid ICD-9-CM codes from the analysis were also excluded. The optimal sample size was estimated based on a calculation of measure of agreement, which suggested 123 infants were needed for the study (Watson & Petrie, 2010). Thus, the final sample size was sufficient to conduct these analyses at 10,175 infants.
Measures
Measures of the original and the modified CCC classification system were created for this study (Table 1). The original CCC classification includes nine categories: neurologic/neuromuscular, cardiovascular, respiratory, renal, gastrointestinal, hematologic/immunologic, metabolic, other congenital or genetic defect (chromosomal abnormalities, bone/joint abnormalities, diaphragm/abdominal abnormalities, other abnormalities), and malignancy (Feudtner et al., 2000, 2001). An overall measure of the original CCC classification was created, which was defined as whether an infant was identified as having an original CCC, and separate binary measures were created for each category.
The modified CCC classification was expanded to include infant diagnoses, transplantation codes, and devices (Feudtner et al., 2014). The eleven categories of the modified CCC classification are: neurologic/neuromuscular, cardiovascular, respiratory, renal, gastrointestinal, hematologic/immunologic, metabolic, other congenital or genetic defect (chromosomal abnormalities, bone/joint abnormalities, diaphragm/abdominal abnormalities, other abnormalities), malignancy, premature/neonatal, and miscellaneous. A overall measure of the modified CCC classification was created, which was whether an infant was identified as having a modified CCC, along with separate binary measures for each modified category.
A group of infant demographic variables was included in the study: gender (male, female), race/ethnicity (Caucasian, African American, Hispanic, Other), household income (<$39,000, $39,000‱$47,999, $48,000‱$62,999, or >$63,000), insurance type (Medicaid, commercial, other), region of residence (South, Midwest, West, Northeast), urban/non-urban, and comorbidities (≥2 conditions).
Analysis Plan
Descriptive statistics were calculated for the characteristics of the infants. For this study, the agreement and validity of the original versus the modified CCC classification systems were evaluated. The analysis examined the performance of the overall CCC classifications within the infant sample. In addition, separate analyses with each CCC category were conducted. Due to the diagnostic nature of this study, sample weights were not applied to analyses. Statistical significance was p<0.5. Data analyses were performed using Stata version 11 (StataCorp LP, College Station, TX).
Agreement.
The Cohen’s Kappa Statistic was calculated to assess whether the modified CCC classification was a reasonable alternative to the original CCC classification in identifying infants with a CCC (Watson & Petrie, 2010). The Kappa Statistics provided the level of agreement between the original and modified CCC classifications. Perfect agreement was equal to 1; whereas a value of zero suggested that the agreement was no better than chance. Although there is no standard interpretation of Kappa Statistic, the following range was used for this analysis (Landis & Koch, 1977):
Poor = below 0.00
Slight = 0.00–0.20
Fair = 0.21–0.40
Moderate = 0.41–0.60
Substantial = 0.61–0.80
Almost perfect = 0.81–1.00
Validity.
A two-by-two table was constructed using the modified and original CCC classification. Sensitivity, specificity, and positive predictive value (PPV) were calculated to assess validity, using the original CCC classification as the gold standard (Lalken & McClusky, 2008; Parikh, Mathai, Parikh, Sekhar, & Thomas, 2008). Sensitivity captures accuracy of the modified CCC classification to identify infants with an original CCC classification (Watson & Petrie, 2010). For interpretation, if the modified CCC classification has a 100% sensitivity, it identifies all the infants with an original CCC. In instances where sensitivity is low, it means the modified CCC is not capturing infants that the original did. Specificity reports how accurately the modified CCC classification performs in identifying infants without an original CCC classification (Watson & Petrie, 2010). Perfect, or 100%, specificity means the modified CCC identifies all the infants without an original CCC. PPV indicates the probability the infant was identified in the original CCC (Watson & Petrie, 2010). If the PPV parameter is sufficiently high, it indicates that the modified captures all of the infants in the original. Where PPV is low, there are many instances where the modified identifies an infant that the original did not (Re et al., 2011). The focus in this study was on both sensitivity and PPV, because the intent was to demonstrate where the similarities and differences occur between the modified and originally CCC classifications. More specifically, we want to understand where the modified CCC may differ from the original in the identification of infants because of the addition of categories and sub-categories.
Results
The characteristics of the infants in the study are shown in Table 2. The overall sample was characterized by the majority of infants being male (55.54%). The most common race/ethnicity reflected was Caucasian (36.71%), followed by African American (23.19%), and Hispanic (16.26%). Over 60% of the overall sample had a median household income that was below $48,000, and the majority were covered by Medicaid (51.13%). Most of the overall sample was from the South (39.46) and located in urban areas (86.00%). Slightly, less than one third of the infants had comorbidities (31.38%). The characteristics of the infants identified by the original CCC system was similar in nature to that of the original sample except for race/ethnicity and comorbidities. The original CCC system had a slightly different distribution of race/ethnicity consisting of 40.20 % Caucasians, 17.59% African Americans, 17.76% Hispanics, and 24.45% other. Additionally, the original system had 63.63% of the infants with comorbidities. The other characteristics across all of the modified CCC infants were similar to the original CCC group.
Table 2.
Characteristics of Study Participants
| Variables | Overall % (N=10,175) |
Original CCC Classification % (n=4,196) |
Modified CCC Classification % (n=8,807) |
|---|---|---|---|
| Female | 44.46% | 46.12% | 44.83% |
| Race/Ethnicity | |||
| Caucasian | 36.71% | 40.20% | 36.26% |
| African American | 23.19% | 17.59% | 23.30% |
| Hispanic | 16.26% | 17.76% | 16.60% |
| Other | 23.84% | 24.45% | 23.84% |
| Household Income (median) | |||
| <$39,000 | 35.76% | 35.13% | 35.84% |
| $39,000–$47,999 | 25.43% | 26.24% | 25.20% |
| $48,000–$62,999 | 22.40% | 22.16% | 22.43% |
| >$63,000 | 16.41% | 16.47% | 16.54% |
| Insurance Type | |||
| Medicaid | 51.13% | 54.53% | 50.93% |
| Commercial | 35.81% | 35.44% | 36.40% |
| Other | 13.06% | 10.03% | 12.67% |
| Region of Residence | |||
| South | 39.46% | 40.44% | 39.29% |
| Midwest | 23.11% | 22.28% | 22.80% |
| West | 21.35% | 22.47% | 21.19% |
| Northeast | 16.09% | 14.80% | 16.73% |
| Non -Urban | 14.00% | 15.49% | 13.64% |
| Comorbidities | 31.38% | 63.63% | 31.38% |
Agreement
The results of the Cohen’s Kappa Statistic analyses to assess the degree of agreement between classifications are displayed in Table 3. Overall, there was only slight agreement (k=0.125) between the original and modified classifications in identifying infants with a CCC, which included all the categories in the analysis. Among the CCC categories, there was almost perfect agreement between the neurologic and neuromuscular (k=0.909), respiratory (k=0.816), renal and urologic (k=0.998), and other congenital or genetic defect (k=0.992). Substantial agreement was present in the gastrointestinal (k=0.755) and metabolic (k=0.766) categories. Cardiovascular (k=0.417) and malignancies (k=0.595) showed moderate agreement. Fair agreement was found in the hematologic or immunologic (k=0.317) category, and there was no agreement on premature and neonatal (k=0.000) and miscellaneous (k=0.000) because these categories were not included in the original classification.
Table 3.
Agreement Analysis Between Modified and Original CCC Classifications (N=10,175)
| Variable | Kappa Statistic (SE) | Rating |
|---|---|---|
| Overall | 0.125 (0.01) | Slight |
| CCC Categories | ||
| Neurologic & Neuromuscular | 0.909 (0.01) | Almost Perfect |
| Cardiovascular | 0.417 (0.01) | Moderate |
| Respiratory | 0.816 (0.01) | Almost Perfect |
| Renal & Urologic | 0.998 (0.01) | Almost Perfect |
| Gastrointestinal | 0.755 (0.01) | Substantial |
| Hematologic or Immunologic | 0.317 (0.01) | Fair |
| Metabolic | 0.766 (0.01) | Substantial |
| Other Congenital or Genetic Defect | 0.992 (0.01) | Almost Perfect |
| Malignancy | 0.595 (0.01) | Moderate |
| Premature & Neonatal | 0.000 | None |
| Miscellaneous | 0.000 | None |
Note: SE = standard error
Validity
The distribution of original vs. modified CCC classification is noted in Table 4. Among the sample, 3,991 infants had a CCC classification in both systems (i.e., true positive) and conversely 1,163 infants did not have a CCC classification in the original and modified systems (i.e., true negative). Two hundred and five infants had a CCC classification in the orginial system, yet did not have a modified CCC classification (i.e., false negative), while 4,816 infants had a modified CCC classification but did not have an original CCC classification (i.e., false positive). The difference between the 4,196 infants identified in the original versus the 8,807 in the modified suggests that the modified identified more infants.
Table 4.
Contingency Table Using the Modified and Original CCC Classifications
| Original CCC Classification (Gold Standard) | ||||
|---|---|---|---|---|
| Positive | Negative | Total | ||
| Modified CCC Classification | Positive | 3,991 | 4,816 | 8,807 |
| Negative | 205 | 1,163 | 1,368 | |
| Total | 4,196 | 5,979 | 10,175 | |
The validity measures of sensitivity and specificity for the classifications are displayed in Table 5. Among the 4,196 infants with an original CCC classification, a CCC was confirmed in 3,991 infants by the modified CCC classification (Table 4). This means that the overall sensitivity was 95.1% (Table 5). This result suggests that the modified classification performs almost as well as the original classification in identifying infants with a CCC. Looking across the CCC categories, low sensitivity was found in the cardiovascular (33.6%) and malignancy (42.6%) categories. The respiratory category (77.8%) had moderate sensitivity. For specificity, the modified CCC identified 19.5% of all the infants who did not have an original CCC. The low specificity results from the 4,816 infants who were identified by the modified CCC and not the original. These additional infants are from the addition of categories such as premature and neonatal. Performance in specificity across the categories ranged from 98.4%for neurologic and neuromuscular to 100% in renal & urologic and malignancy.
Table 5.
Validity Measures of Modified versus Original CCC Classifications
| Variable | Sensitivity | Specificity | PPV |
|---|---|---|---|
| Overall | 95.1% | 19.5% | 45.3% |
| CCC Categories | |||
| Neurologic & Neuromuscular | 98.6% | 98.4% | 85.9% |
| Cardiovascular | 33.6% | 99.2% | 92.8% |
| Respiratory | 77.8% | 99.0% | 89.6% |
| Renal & Urologic | 100% | 100% | 99.7% |
| Gastrointestinal | 100% | 98.6% | 61.4% |
| Hematologic or Immunologic | 100% | 98.6% | 19.1% |
| Metabolic | 100% | 98.5% | 62.9% |
| Other Congenital or Genetic Defect | 100% | 99.8% | 98.6% |
| Malignancy | 42.6% | 100% | 100% |
| Premature & Neonatal | n/a | n/a | n/a |
| Miscellaneous | n/a | n/a | n/a |
Table 5 also shows the results of the PPV analysis. Overall, the modified CCC had a PPV of 45.3%. This low percentage is being driven by a larger denominator for the PPV, which comes from the additional 4,816 infants that are identified by the modified that are not identified in the original (Table 4). The low PPV occurred within the hematologic or immunologic (19.1%), gastrointestinal (61.4%), and metabolic (62.9%) categories. In these categories, many subcategories were added, which suggests the modified CCC system might be identifying more infants than the original. For example, within the hematologic or immunologic categories there were additional subcategories of polyarteritis nodosa/related conditions, diffuse disease of connective tissue, transplantation, and other. Therefore, the additional subcategories resulted in additional modified CCC infants being identified with a CCC, which subsequently lowered the PPV value.
Discussion
The overall goal of the study was to evaluate the performance of the new, modified CCC classification system against the original, gold standard CCC classification system. From the evaluation of agreement, the findings revealed that the modified CCC system was sufficiently different that the original CCC system. The Kappa Statistic (k=0.125) suggested only a slight agreement between the original and modified systems. Furthermore, when the specific CCC categories were examined, almost half of the categories ranked moderate or less on agreement. Although the inclusion of new categories, procedure codes, and diagnoses significantly changed the structure of the modified system, it is also possible that the difference between original and modified systems reflect a conceptual change. The original system measured the construct of life-limiting illness, while the modified system appears to operationalize medical complexity. Cohen et al. (2011) defines medical complexity as multisystem diseases with a need for significant health care services, along with a marked functional impairment and/or technology dependence for activities of daily living. This definition of medical complexity is consistent with the inclusion of medical technology and transplant codes in the modified CCC system. However, more research at a conceptual level is warranted to understand why differences in identification of infants and children exist between the modified and original classifications. Additional studies might explore the theoretical constructs of these two systems and provide the theoretical framework for the systems. Thus, future nursing research on the classification systems might offer researchers a theoretical foundation and additional evidence to guide pediatric research.
The validity testing provided additional information on the performance of the original versus modified CCC classification systems. The results showed the modified system identified 95% of the actual CCC infants under the original system and among infants not classified under the original system 19% were also not classified under the modified system. These findings suggest that the modified CCC system identified more CCC infants than the original, which was reinforced with the relatively low overall PPV value. There were also significant differences in the performance of the cardiovascular, malignancy, hematologic or immunologic, gastrointestinal, and metabolic categories across the sensitivity and PPV measures. The modified CCC classification system was intended to capture the information contained with the original CCC and include more information consistent with clinical practice. The study results found that the modified CCC classification identified almost all of the infants that the original CCC did. However, exceptions occurred within the cardiovascular and malignancy categories that demonstrated low sensitivity. These exceptions were consistent with the findings of Feudtner and colleagues (2014) in their initial comparisons of the classifications among children. Perhaps, the additional information surrounding devices and transplantation fundamentally changed the infants identified within these categories, because transplantation and devices are life-enhancing, and only a very small subset of infants would experience mortality as a result of complications from them. Additional testing is needed to evaluate the performance of the two systems to predict end-of-life outcomes important to infants and their families. The science of measurement validation is well-established and includes a variety of sohpisticated analytical approaches that strive to ensure measures are valid in order to minimize bias in research studies. This study provided important baseline information about the differences in the two systems, specifically testing how they differed. Additional research is needed to understand whether the modified actually performs better than the orginial system. Nurse researchers might compare the modified and original CCC classification system’s ability to predict outcomes such as palliative care consultation, recipient of intensive care, and expenditures.
The addition of new subcategories, including premature/neonatal and miscellaneous, to the modified CCC classification system contributed to an overall change in classification performance. The overall low specificity and PPV results signaled that the modified CCC classification system may be capturing additional information and classifying more infants as compared to the original. Given that infants are often excluded from pediatric research studies because of their medical conditions and differences from other child populations, the modified CCC classification system can be used to more adequately identify and include appropriate infants in studies, as well as make them easier to compare to older children. It is important to note that both the original and modified CCC classification systems were used to identify infants retrospectively in this study. Further research is warranted to investigate how these classification systems perform in prospective recruitment of samples of infants where the admission criteria is often much different than the infant’s eventual cause of death. It may be that these criteria are a good starting point for screening and recruitment of potentially eligible infants. It may also be possible that these criteria could serve as triggers for palliative care consultation as soon as an infant is able to be categorized.
There are several notable limitations for this study. First, the study sample was US infants admitted to children’s hospitals. This limits generalizability of the study’s findings to other pediatric decedents such as toddler or adolescents. There is an opportunity to conduct future research that tests these classification systems might among older children. However, as a national study of infants, the current research examined the largest group of pediatric decedents in the United States (Murphy, Mathews, Martin, Minkovitz, & Strobino, 2017). The findings from the study may not be generalizable to other health care institutions. Furthermore, this study was conducted with data from US hospitals which also reduces generalizability to non-US infant populations because US children’s hospital system is often different from those in other countries. Second, only discharge diagnoses are included in the HCUP KID data files. Admission diagnoses were not identified, which may be different than the diagnosis upon discharge and limit the ability to determine CCC diagnoses among infants. Third, infants’ deaths outside the hospital were not included in the HCUP KID files. This population is most likely very small because most seriously ill infants often remain in the hospital after birth until their death. Notwithstanding these limitations, HCUP KID is the largest publicly-available all-payer pediatric inpatient database in the United States. Hence, the results may provide valuable insight into the performance of two classifications systems for pediatric research. Although the study findings suggest the modified CCC classification system is an improvement upon the original classification, additional research is need to understand the magnitude of that difference.
To the best of our knowledge, there are no studies that have conducted a comparison of the original and modified CCC classification systems by assessing agreement and validity in a sample of infants. In this evaluation, the findings revealed only slight agreement between the original and modified classifications in identifying infants with a CCC; therefore, the two classifications might be capturing different constructs. The validity testing identified that the differences between systems were based on changes in the subcategories and the additional of new categories. Thus, the modifications have improved performance of the CCC classification system to identify infants with CCC.
Acknowledgments
Funding Source: This publication was made possible by Grant Number R01NR017848 from the National Institute of Nursing Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institute of Nursing Research or National Institutes of Health.
Footnotes
Financial Disclosure: The authors have no financial relationships relevant to this article to disclose.
Conflict of Interest: Dr. Lindley declares she has no conflict of interest. Dr. Fortney declares she has no conflict of interest. Dr. Cozad declares she has no conflict of interest.
Contributor Information
Lisa C. Lindley, College of Nursing, University of Tennessee, Knoxville, Knoxville, Tennessee 37996.
Melanie J. Cozad, Department of Health Services Policy and Management, Center for Effectiveness Research in Orthopedics, University of South Carolina.
Christine A. Fortney, Martha S. Pitzer Center for Women, Children Youth, College of Nursing, The Ohio State University, Columbus, Ohio 43210.
References
- Anath P, Melvin P, Feudtner C, Wolfe J, & Berry JG (2015). Hospital use in the last year of life for children with life-threatening complex chronic conditions. Pediatrics, 136(5), 938–946. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berry JG, Hall M, Cohen E, O’Neill M, & Feudtner C (2015). Ways to identify children with medical complexity and the importance of why. Journal of Pediatrics, 167(2), 229–237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen E, Kuo DZ, Agrawai R, Berry JG, Bhagat SK, Simon TD, & Srivastava R (2011). Children with medical complexity: An emerging population for clinical and research initiatives. Pediatrics, 127(3), 529–538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- daFonseca JG, & Ferreira AR (2014). Application of the pediatric index of mortality 2 in pediatric patients with complex chronic conditions. Journal de Pediatria, 90(5), 506–511. [DOI] [PubMed] [Google Scholar]
- Feinstein JA, Feudtner C, & Kempe A (2014). Adverse drug event-related emergency department visits associated with complex chronic conditions. Pediatrics, 133(6), e1575–e1585. [DOI] [PubMed] [Google Scholar]
- Feudtner C, Christakis DA, & Connell FA (2000). Pediatric death attributable to complex chronic conditions: A population-based study of Washington state, 1980–1997. Pediatrics, 106, 205–209. [PubMed] [Google Scholar]
- Feudtner C, Hays R, Haynes G, Geyer J, Neff J, & Koepsell T (2001). Deaths attributed to pediatric complex chronic conditions: National trends and implications for supportive care services. Pediatrics, 107(6), e99–e103. [DOI] [PubMed] [Google Scholar]
- Feudtner C, Feinstein JA, Zhong W, Hall M, & Dai D (2014). Pediatric complex chronic conditions classification system version 2: Updated for ICD-10 and complex technology dependence and transplantation. BMC Pediatrics, 14, 199–206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Golden SL, & Nageswaran S (2012). Caregivers voices: Coordinating care for children with complex chronic conditions. Clinical Pediatrics, 51(8), 723–729. [DOI] [PubMed] [Google Scholar]
- Hessels AJ, Liu J, Cohen B, Shang J, & Larson EL (2018). Severity of illness measures for pediatric inpatients. Journal of Healthcare Quality, 40(5), e77–e89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hudson SM, Mueller M, Hester WH, Magwood GS, Newman SD, & Laken MA (2014). At-risk characteristics for hospital admissions and ED visits. Journal for Specialists in Pediatric Nursing, 19, 183–193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keele L, Keenan HT, Sheetz J, & Bratton SL (2013). Differences in characteristics of dying children who receive and do not receive palliative care. Pediatrics,132, 72–78. [DOI] [PubMed] [Google Scholar]
- Khan A, Spector ND, Baird JD, Ashland M, Starmer AJ, Rosenbluth G, … Landrigan CP (2018). Patient safety after implementation of a coproduced family centered communication programme: Multicenter before and after intervention study. BMJ, 363, k4764 10.1136/bmj.k4764 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lalkhen AG, & McClusky A (2008). Clinical tests: Sensitivity and specificity. Continuing Education in Anaesthia and Critical Care Pain, 8(6), 221–223. [Google Scholar]
- Landis JR, & Koch GG (1977). Measurement of observer agreement for categorical data. Biometrics, 33, 159–174. [PubMed] [Google Scholar]
- Lindley LC (2017). Multiple complex chronic conditions and pediatric hospice utilization among California Medicaid beneficiaries, 2007–2010, Journal of Palliative Medicine, 20(3), 241–246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lindley LC, & Fortney CA (2019). Pediatric complex chronic conditions: Does the classification system work for infants? American Journal of Hospice and Palliative Medicine, 1049909119838985. [DOI] [PMC free article] [PubMed]
- Lindley LC, & Keim-Malpass J (2017). Quality of pediatric hospice care for children with and without multiple complex chronic conditions. International Journal of Palliative Nursing, 23(5), 112–119. [DOI] [PubMed] [Google Scholar]
- Murphy SL, Mathews TJ, Martin JA, Minkovitz CS, & Strobino DM (2017). Annual Summary of Vital Statistics: 2013–2014. Pediatrics, 139(6), 1–12. [DOI] [PubMed] [Google Scholar]
- Parikh R, Mathai A, Parikh S, Sekhar GC, & Thomas R (2008). Understanding and using sensitivity, specificity, and predictive values. Indian Journal of Opthalmology, 56, 45–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Re VL, Lim JK, Goetz MB, Tate J, Bathulapalli H, Klein MB, … Justice AC (2011). Validity of diagnostic codes and liver-related laboratory abnormalities to identify hepatic decomposition events in the Veterans Aging Cohort Study. Pharmacoepidemiology and Drug Safety, 20, 689–699. 10.1002/pds.2148 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watson PF, & Petrie A (2010). Method agreement analysis: A review of correct methodology. Therio, 73, 1167–1179. [DOI] [PubMed] [Google Scholar]