Abstract
Objective
The ability to identify children with special health care needs (CSHCN) is crucial to evaluate disparities in the quality of health care for children in Medicaid Managed Care. We developed and assessed the accuracy of a new method to classify CSHCN.
Data Sources
Secondary data analysis was conducted using NYS Medicaid administrative data and the Children with Chronic Conditions Screener (CCC Screener).
Study Design
This study included 5,907 NYS Medicaid beneficiaries (17 years old or younger) whose parents completed the CCC Screener in 2014. Medicaid administrative data were used to create a risk score to assess the risk of special needs, and a cut point was identified to differentiate between children with versus without special needs. Diagnostic accuracy of the method was assessed using sensitivity and specificity analyses.
Principal Findings
Applying the CCC Screener as the “gold standard,” the risk score correctly classified the majority of CSHCN as positive (sensitivity = 75 percent) and the majority of the children without special needs as negative (specificity = 79 percent). This method demonstrated decent diagnostic ability (AUC = 0.77).
Conclusions
Our method can identify CSHCN in the NYS Medicaid Managed Care population and will help the State monitor the quality of care for this vulnerable population.
Keywords: Chronic disease, child and adolescent health, Medicaid, administrative data uses, disability, mental health
Children with special health care needs (i.e., CSHCN) have been defined by the Maternal and Child Health Bureau as, “children who have or at risk for a chronic physical, developmental, behavioral, or emotional condition, and who also require health and related services of a type or amount beyond that required by children generally (McPherson et al. 1998).” A few commonly identified health care conditions among CSHCN include ADD/ADHD, asthma, and learning disabilities (Bramlett et al. 2009; C.A.H.M.I. 2012). However, many CSHCN have comorbid conditions (Newacheck and Halfon 1998; Neff et al. 2002). Commonly reported problems that CSHCN experience include difficulty learning, understanding or paying attention, respiratory problems, anxiety, and depression (C.A.H.M.I. 2012). CSHCN constitute a vulnerable population in that they need and utilize more mental and physical health care services than children without special needs (Aday 1992; Szilagyi et al. 2003; Cohen et al. 2012).
CSHCN not only have worse health outcomes than children without special needs (Newacheck and Halfon 1998; Newacheck et al. 1998), they are also vulnerable because they have more difficulty accessing the health care that they need. Specifically, CSHCN have more unmet needs in terms of prescription medication, specialists, and urgent care than children without special needs (Mayer, Skinner, and Slifkin 2004; Houtrow, Kim, and Newacheck 2008). There is also evidence that children with medical complexity An (2016) and more severely limited CSHCN are more likely to have unmet health care needs (Dusing, Skinner, and Mayer 2004).
Recently, The Center for Medicare and Medicaid Services released “The Improving the Quality of Care for Medicaid Beneficiaries Final Rule,” which states that to support key quality goals, mechanisms should be implemented by states to identify individuals who have special health care needs (C.M.S. 2016). Examining care for CSHCN within the Medicaid population is also important because their care is restricted to specific providers and waiver programs. The literature suggests that access to care is affected by payer, such that CSHCN with Medicaid report more unmet dental needs, more difficulty getting medical care, and lower overall health than CSHCN with private or employer sponsored insurance (Sarkar et al. 2017). Given CSHCNs' high level of service need, utilization, and unmet health care needs, it is important to identify members of this population and to ensure that Medicaid provides them with the quantity and quality of care that they need. However, a systematic, cost‐effective way to identify Medicaid beneficiaries with special needs has yet to be identified.
Several different assessments have been used to identify CSHCN. However, the use of surveys to identify CSHCN imposes major limitations. First, relying on surveys limits states' ability to regularly monitor the quality of care for CSHCN, and the ability to drill down into subpopulations with different types of health care needs. Additionally, surveys are expensive to administer and it is not possible to conduct such assessments across the entire population of interest. Hence, many analyses are limited to drawing conclusions about CSHCN based on the small sample whose parent or guardian has completed such an assessment. This limits the generalizability of CSHCN research and may introduce self‐selection bias, if there are systematic differences between the parents who do or do not complete such a survey.
Instead of using surveys to identify CSHCN, other researchers have attempted to use administrative data to identify CSHCN. Some researchers have utilized one variable as a proxy for special needs status, such as clinical risk group (Neff et al. 2002, 2010). However, limited accuracy was observed when this method was applied to the current study. Another method, the Pediatric Medical Complexity Algorithm, uses a set of diagnosis codes to identify children with noncomplex and complex chronic conditions (Simon et al. 2010). This method accurately identified children with complex chronic conditions and children without chronic conditions, but was limited in the ability to identify children with noncomplex chronic conditions. Therefore, the current study seeks to identify a systematic method using statewide, administrative health data to provide a comprehensive assessment of which Medicaid beneficiaries have special needs.
The Current Study
The current research involves using variables within the NYS Medicaid data, which is available for all Medicaid beneficiaries, to create a risk score to identify CSHCN. The diagnostic accuracy of the risk score was assessed by comparing it with data from the Children with Chronic Conditions Screener (CCC Screener) using sensitivity and specificity analyses.
The CCC Screener is one of the most commonly used CSHCN surveys and has been integrated into The Consumer Assessment of Health Providers and Systems Survey (CAHPS), the Questionnaire for Identifying Children with Chronic Conditions, and the National Health Interview Survey. It was designed to identify, characterize, and monitor the health and health care quality for populations of children in health plans and national and state geographic contexts (Bethell et al. 2015). Rather than specifying certain conditions that indicate that a child has special needs, the CCC Screener categorizes children as CSHCN if they have one or more health‐related consequence that is due to a health condition which has lasted or is expected to last at least 12 months. The health consequences include functional limitations, medication use, specialized therapies, greater use of services, and treatment/counseling. The CCC Screener is an inclusive measure that classifies children as CSHCN both if their parent or guardian states that the child currently uses health services due to an ongoing condition and if the child needs services they are not receiving for an ongoing condition (Bethell et al. 2002a). Although there is no official gold standard for measuring CSHCN, the CCC Screener is extensively used and is highly regarded because it operationalizes the federal Maternal and Child Health Bureau definition of CSHCN. This measure also has high internal and external validity (Carle, Blumberg, and Poblenz 2011; Bethell et al. 2015). For these reasons, it was used as the gold standard in this study.
Methods
This study was approved by the New York State Department of Health Institutional Review Board. Data from children whose parents completed the CCC Screener were matched with the children's Medicaid data. As recommended by The Child and Adolescent Health Measurement Initiative, CSHCN were identified using a combination of diagnosis and service‐use information available in the NYS DOH Medicaid claims and encounter data (C.A.H.M.I. 2002). This method was chosen over a categorical method, which classifies individuals as having special health needs only if they have specific diagnoses, because the categorical method may exclude less prevalent diagnoses, the administrative data may miss emerging conditions among children, and diagnoses may be applied inconsistently by providers (Perrin et al. 1993; Stein et al. 1993). Medicaid variables were selected to identify the five different subgroups identified on the CCC Screener (i.e., children with functional limitations, medication use, specialized therapies, greater use of services, and treatment/counseling). The selected Medicaid variables were ranked based on how much they were expected to predict special needs status and were combined into a continuous, composite risk score for each child.1
Risk Score: Assessing Diagnostic Ability and Assigning a Cut Point
The receiver operating characteristic (ROC) curve was used to assess the overall diagnostic ability of the risk score by plotting the sensitivity versus the false‐positive rate (Shapiro 1999). When the area under the ROC curve falls between 0.70 and 0.90, this indicates that the measure is moderately accurate (Swets 1988; Fischer, Bachmann, and Jaeschke 2003). Since the risk score is continuous, a cut point was identified to differentiate children without special needs from CSHCN. Youden's Index, which indicates the maximum potential effectiveness of a biomarker, was used to determine a cut point at which children should be considered CSHCN (Perkins and Schisterman 2006).
The validity of the risk score was assessed by comparing it with the CCC Screener using sensitivity and specificity analyses. High sensitivity and high specificity between the two methods for classifying children as special needs provide evidence that the risk score accurately differentiates between children with and without special needs. Sensitivity provides us with information about the extent to which variables correctly classify children who have special needs as “positive”, whereas specificity informs the extent to which variables correctly classify children who do not have special needs as “negative”.
Equations 1 and 2 (below) were utilized to assess the sensitivity and specificity of the different methods for categorizing children's special need status:
(1) |
(2) |
Results
Study Population
The sample was composed of 5,907 NYS Medicaid‐enrolled children whose parent or guardian completed the CCC Screener in the CAHPS Survey in 2014. There were 15,310 children excluded from this analysis because there was no response via phone or mail (73.66 percent), or their parent/guardian refused (9.15 percent), had an incomplete or incorrect address or phone number (8.22 percent), did not complete the CCC Screener (3.88 percent), had a language barrier (2.98 percent), did not meet the eligible population criteria (2.08 percent), or had died (0.03 percent).
Demographic information is available in Table 1 on the children, and parents or guardians in the current sample for children classified as CSHCN by the risk score, and the CCC Screener. As previously mentioned, the CCC Screener classified children into one or more of the five relevant special need subgroups: 4,609 children (78 percent) as not having any special needs, 360 children (6 percent) as CSHCN based on medication use, 114 children (2 percent) as CSHCN based on treatment/counseling, 31 children (0.5 percent) as CSHCN based on greater use of services, 28 children (0.5 percent) as CSHCN based on functional limitations, 42 children (1 percent) as CSHCN based on specialized therapies, and 723 children (12 percent) as CSHCN based on multiple special need categories.
Table 1.
Children and Parent Demographic Information for Children Classified as CSHCN by Risk Score and Traditionally Scored CCC Screener
Demographic Characteristics | Risk Score from Medicaid Variables | Traditionally Scored CCC Screener | ||
---|---|---|---|---|
(N = 1,926, 33%) | (N = 1,298, 22%) | |||
N | % | N | % | |
Children | ||||
Gender | ||||
Male | 1109 | 57.58 | 757 | 58.32 |
Female | 799 | 41.48 | 535 | 41.22 |
Missing | 18 | 0.93 | 6 | 0.46 |
Age in years | ||||
0–4 | 439 | 22.79 | 209 | 16.10 |
5–9 | 611 | 31.72 | 450 | 34.67 |
10–14 | 590 | 30.63 | 457 | 35.21 |
15–17 | 265 | 13.76 | 176 | 13.56 |
Missing | 21 | 1.09 | 6 | 0.46 |
Race | ||||
White | 797 | 41.38 | 588 | 45.30 |
Black | 328 | 17.03 | 251 | 19.34 |
Asian | 107 | 5.56 | 46 | 3.54 |
Other | 299 | 15.52 | 157 | 12.10 |
Multiracial | 204 | 10.59 | 171 | 13.17 |
Missing | 191 | 9.92 | 85 | 6.55 |
Ethnicity | ||||
Latino | 762 | 39.57 | 418 | 32.20 |
Non‐Latino | 1126 | 58.46 | 855 | 65.87 |
Missing | 38 | 1.97 | 25 | 1.93 |
Region | ||||
Central | 158 | 8.20 | 158 | 12.17 |
Hudson Valley | 251 | 13.03 | 150 | 11.56 |
Long Island | 122 | 6.33 | 60 | 4.62 |
NYC | 698 | 36.24 | 347 | 26.73 |
Northeast | 131 | 6.80 | 114 | 8.78 |
Western | 566 | 29.39 | 469 | 36.13 |
Parents or guardians | ||||
Gender | ||||
Male | 227 | 11.79 | 136 | 10.48 |
Female | 1670 | 86.71 | 1153 | 88.83 |
Missing | 29 | 1.51 | 9 | 0.69 |
Age in years | ||||
>18 | 140 | 7.27 | 110 | 8.47 |
18–34 | 716 | 37.18 | 468 | 36.06 |
35–54 | 910 | 47.25 | 610 | 47.00 |
55–75+ | 112 | 5.82 | 91 | 7.01 |
Missing | 48 | 2.49 | 19 | 1.46 |
Relationship with child | ||||
Mother or father | 1720 | 89.30 | 1141 | 87.90 |
Grandparent | 92 | 4.78 | 73 | 5.62 |
Aunt or uncle | 13 | 0.67 | 14 | 1.08 |
Older sibling | 4 | 0.21 | 3 | 0.23 |
Other relative | 3 | 0.16 | 3 | 0.23 |
Legal guardian | 36 | 1.87 | 29 | 2.23 |
Missing | 58 | 3.01 | 35 | 2.70 |
Language spoken at home | ||||
English | 1019 | 52.91 | 804 | 61.94 |
Non‐English | 850 | 44.13 | 479 | 36.90 |
Missing | 57 | 2.96 | 15 | 1.16 |
Highest level of education | ||||
8th grade or less | 195 | 10.12 | 98 | 7.55 |
Some high school, did not graduate | 296 | 15.37 | 189 | 14.56 |
High school graduate or GED | 599 | 31.10 | 380 | 29.28 |
Some college or 2‐year degree | 558 | 28.97 | 452 | 34.82 |
4‐year college graduate | 127 | 6.59 | 94 | 7.24 |
More than 4‐year college degree | 88 | 4.57 | 61 | 4.70 |
Missing | 63 | 3.27 | 24 | 1.85 |
Risk Score: Designing a Method to Identify CSHCN
Diagnosis and service‐use variables from the Medicaid data were examined with respect to their ability to differentiate between children with and without special health care needs. We used the items from the CCC Screener to identify important variables in the Medicaid data. Additionally, we weighted the Medicaid variables on a scale from 0 to 3 based on how much we expected them to predict special needs status and used these weighted variables to create a composite risk score.
Mental Health Treatment
The CCC Screener examined whether children needed or used mental health treatment for a condition that lasted or was expected to last at least 12 months long. To identify children who received mental health treatment, we used two methods. First, we used the 3M Health Information System which analyzes children's service information to determine whether children had a major diagnostic category (MDC) of mental diseases and disorder. Children who had an MDC of mental diseases or disorder received two points that went toward their composite risk score.
Second, we classified children as having a serious emotional disturbance (SED) if they had a MDC of mental diseases and disorder, and they had at least one service in the past 12 months with a diagnosis of schizophrenia, other psychotic disorders, major depression, bipolar disorders, cyclothymic disorder, schizotypal, chronic hypomanic, borderline personality disorders, post‐traumatic stress disorder, attention deficit disorder, obsessive–compulsive disorder, eating disorder, conduct disorder, or development disorder. Children with a SED received three points that went toward their composite risk score. Children that did not have an MDC of mental disorder or SED did not receive any points toward their composite risk score.
Functional Limitations
The CCC Screener examined whether children were limited or prevented from doing things most children of the same age can do due to a condition that lasted or was expected to last at least 12 months long. To identify children within the Medicaid system who had a functional disability, we used information about whether the children received supplemental security income (SSI), which is federal program that provides stipends to low‐income people who are either aged 65 or older, blind, or disabled. Children who received SSI received three points that went toward their composite risk score. Children who did not receive SSI did not receive any points toward their composite risk score.
Special Therapy
The CCC Screener examined whether children needed or used special therapy (i.e., physical, occupational, or speech therapy) due to a condition that lasted or was expected to last at least 12 months long. To identify children within the Medicaid system who received therapies, we used service information to indicate which children used physical therapy, occupational therapy, speech therapy, and preschool or school supportive health care service programs. Children who did receive specialized therapies received three points that went toward their composite risk score. Children who did not receive these therapies did not receive any points toward their composite risk score.
Prescription Medications
The CCC Screener examined whether children needed or used prescription medication for a condition that lasted or was expected to last at least 12 months long. To identify children within the Medicaid system who frequently used prescription medication, we used claims data to determine the length of time that children were prescribed medication during the year (i.e., 0–30, 31–90, 91–182 days, and >183 days). Children prescribed medication for 183 days or more received three points; children prescribed medication between 91 and 182 days received two points; children prescribed medication between 31 and 90 days received one point; and children prescribed medicine for 30 days or less did not receive any points toward their composite risk score.
More Medical, Mental Health, or Educational Services
The CCC Screener examined whether children needed or used more medical, mental health, or educational services than most children of the same age due to a condition that lasted or was expected to last at least 12 months long. Since this is the broadest classification in the CCC Screener, we used two different types of information to try to identify this group of CSHCN. First, to identify children within the Medicaid system who used greater educational services, we identified children with intellectual or developmental disabilities (IDD) using diagnosis codes that previous researchers have used to identify individuals with intellectual and developmental disabilities (McDermott et al. 2018). Some diagnoses utilized include unspecified intellectual disability, autism or pervasive developmental disorder, and cerebral palsy. Children who had an IDD received three points toward their composite risk score. Children who did not have an IDD did receive any points toward their composite risk score.
Additionally, since the CCC Screener item includes children with greater medical service usage than usual, we identified children within the Medicaid system with chronic conditions using the 3M Health Information System's episodic disease category (EDC). This variable identified whether children had a chronic manifestation, moderate chronic condition, or dominant chronic condition (Averill et al. 1999). Dominant chronic EDCs are defined as, “serious chronic diseases which often result in the progressive deterioration of an individual's health and often lead to or significantly contribute to an individual's debility, death and future need for medical care.” Children with dominant chronic conditions received three points toward their composite risk score. Moderate chronic EDCs are “serious chronic diseases which, usually do not result in the progressive deterioration of an individual's health but can significantly contribute to an individual's debility, death and future need for medical care.” Children with moderate chronic conditions received two points toward their composite risk score. A chronic manifestation EDC is “the manifestation or acute exacerbation (i.e., neuropathy) and indicates the presence of the underlying chronic disease (i.e., diabetes).” Children with chronic manifestations received one point toward their composite risk score. Children who did not have a dominant or moderate chronic condition or a chronic manifestation did not receive any points toward their composite risk score.
The previously indicated Medicaid variables were combined into a continuous, composite risk score for each child. Table 2 displays a complete crosswalk of the Medicaid variables used to identify children in the CCC Screener subgroups and the number of children within each category of the Medicaid variables.
Table 2.
Crosswalk of Medicaid Variables Used to Identify CCC Screener Subgroups
CCC Screener Subgroup | Medicaid Variables | Levels of Medicaid Variable | N |
---|---|---|---|
Child currently needs/uses prescription medication | Days of prescription medication within last year | 0 = 0 days–30 days (no days–1 month) | 3,232 |
1 = 31 day – 90 days (31 days–<3 months) | 1,503 | ||
2 = 91 days – 182 days (3 months–<6 months) | 622 | ||
3 = 183 days + (6 months or longer) | 550 | ||
Child needs/uses mental health treatment | Major diagnostic category of mental disease (MDC MH) | 0 = No MDC MH | 5,022 |
2 = MDC MH and No SED | 237 | ||
3 = SED | 648 | ||
Serious emotional disturbance (SED) | |||
Child needs/uses more medical, mental health, or educational services than usual | Intellectually or developmentally delayed (IDD) | 0 = No IDD | 5,377 |
3 = Has IDD | 530 | ||
Child is limited in the ability to do things | Receives SSI (indication of disability) | 0 = Does not receive SSI | 5,682 |
3 = Receives SSI | 225 | ||
Child needs/uses special therapy | Utilized special therapy (e.g., occupational, physical therapy) | 0 = Did not use special therapies | 5,578 |
3 = Used special therapies | 329 | ||
Any CCC Screener subgroups | Chronic Episodic Disease Category (EDC) | 0 = No chronic EDC | 4,372 |
1 = Chronic manifestation | 7 | ||
2 = Moderate chronic condition | 1,218 | ||
3 = Dominant chronic condition | 310 | ||
Risk score | Sum of all indicators above |
Risk Score: Assessing Diagnostic Ability and Assigning a Cut Point
To assess the diagnostic ability of the risk score, receiver operating curve (ROC) analysis was utilized. Analyses were conducted to determine a cut point along the continuous risk score that would best differentiate CSHCN from children without special needs.2 Youden's Index indicated that a cut point of 3 was the optimum value to differentiate between children with versus without special needs (see Figure 1). The area under the curve is 0.77, which indicates decent diagnostic ability. Children with scores below 3 were classified as not having special needs; whereas, children with a risk score of 3 or higher were classified as having special needs. Using this cut point, the risk score classified 3,981 children (67 percent) as not having special needs, and 1,926 children (33 percent) were classified as potentially having special needs.
Figure 1.
- Note. The Y next to the risk score of 3 is the cut point indicated by Youden's Index.
When comparing the risk score with the CCC Screener to assess the validity of the method, there were 970 true positives, 328 false negatives, 956 false positives, and 3,653 true negatives. Overall, there was relatively high sensitivity (75 percent) and specificity (79 percent) between these two measures of CSHCN.
Discussion
The risk score has allowed us to classify the majority of children who were designated as CSHCN on the CCC Screener as “special needs” (i.e., sensitivity) and the majority of the children without special needs on the CCC Screener as “not special needs” (i.e., specificity) when compared with the results from the CCC Screener. The risk score estimates that 33 percent of the children in the current sample have special needs. Epidemiological studies show that between 3.8 percent −32 percent of children can be classified as having special needs, depending on the diagnostic criteria used (Newacheck, McManus, and Fox 1991; Beers et al. 2003). The prevalence of CSHCN tends to be higher among children enrolled in Medicaid Managed Care (MMC) than among children within the general population. Among Medicaid recipients, 21–30 percent of children were classified as having special needs (Bethell et al. 2002a; Stewart et al. 2014), whereas, in the general population, between 12 and 19 percent of children were identified as having special needs depending upon the method used (Bethell et al. 2002b, 2008).
The estimated prevalence of CSHCN is higher using the risk score than the CCC Screener. This may be accounted for by the fact that the current methodology is based on diagnostic and service‐use information among members who had access to a wide range of health care services through the NYS Medicaid Program. Additionally, the fact that the estimated prevalence of CSHCN is higher using this method may be beneficial from a quality improvement standpoint. Since this method is more inclusive, and it will enable us to monitor and help children with less severe chronic health care needs, as well as children with severe chronic health care needs.
Implications
This study provides information that can guide system‐ and policy‐level decision making. New York State will utilize this information to obtain yearly estimates on the prevalence of CSHCN with the Medicaid system. Additionally, we will utilize this methodology to differentiate CSHCN from those without special needs, in order to meet The Centers for Medicare and Medicaid Service's “Improving the Quality of Care for Medicaid Beneficiaries Final Rule” by examining whether there are disparities in unmet health care needs, and the quality of care that CSHCN receive (C.M.S. 2016). Particularly, research suggests that it is especially important to examine whether there are unmet health care needs among CSHCN who have a disability (Houtrow et al. 2011). If disparities are identified, NYS will be able to work toward improving the quality of care for CSHCN by implementing programs and improving services geared toward children with specific types of health care needs.
This methodology could be utilized to produce an estimate of the prevalence of CSHCN within the Medicaid population in other states and could be used to compare across states. Although a cut point of 3 was identified by Youden's Index as the optimum value to differentiate between children with versus without special needs in the 2014 NYS Medicaid data, this cut point may not be optimal when used to classify data from other states or different time periods. Each state should examine their own data to determine which cut point along the risk score is optimal. Using this risk score to identify CSHCN could lead to innovations in the ability to detect unmet health care needs among CSHCN, develop the evidence that could inform large‐scale policy change, and motivate public health professionals and policy makers to work toward ameliorating the health disparities experienced by this population.
Strengths
Since CSHCN have a combination of different physical, emotional, developmental, or psychological issues, creating a continuous risk score attempts to utilize many different indicators to capture the broad range of issues experienced by this population. A multifacetted risk score should allow us to correctly identify more children across the spectrum of special health care needs than a diagnosis‐based approach which may miss certain segments of the population (Stein, Westbrook, and Bauman 1997; Beers et al. 2003; Bethell et al. 2015).
In addition, comorbidity is common among special needs populations (Szilagyi et al. 2003; Simon et al. 2010; Cohen et al. 2012). The risk score is especially likely to classify children who meet multiple criteria for special needs status. Utilizing a multifacetted, continuous risk score to assess special needs status increases the likelihood that we will classify children with comorbid physical and/or mental health problems as having special needs.
Limitations
There are limitations that affect our ability to develop a method that utilizes Medicaid data to differentiate between children who were classified as special needs from those who were not classified as having special needs on the CCC Screener. The Medicaid variables that we included in our risk score do not perfectly capture all the children in the sample with special needs. Some of our indicators identify CSHCN more effectively than others, but our data suggest that they can identify the majority of CSHCN when used as a composite measure. Additionally, the physical, emotional, developmental, and psychological problems that CSHCN experience present at differing levels of severity, thereby leading to stark differences in the diagnoses, treatment, and outcomes experienced by these children (C.A.H.M.I. 2002; Quach et al. 2015). Therefore, the presence or absence of special health care needs is not a clear‐cut designation and likely impacts the accuracy of our method.
Although the Medicaid data provide a wealth of information, it is not an all‐inclusive repository for health data which may affect the ability of the risk score to identify CSHCN. Certain children's services may not be captured within Medicaid administrative data. Children who receive specialized therapies at school or through the foster care system will not have service information available in the Medicaid data. Similarly, children who are dual‐eligible (i.e., who have Medicare and Medicaid) may receive services through Medicare that are not identified in the Medicaid data. Likewise, children with unmet health care needs would not have service information in the Medicaid administrative data. The literature suggests that children who are poor, African American, who have behavioral health problems, and who have unstable health care needs are more likely to have unmet health care needs (Dusing, Skinner, and Mayer 2004; Mayer, Skinner, and Slifkin 2004; Warfield and Gulley 2006; Hill et al. 2008). This lack of information could mean that children within the aforementioned groups may be more likely to be classified as false negatives.
To address this, we examined whether differences in demographic and service information predicted which children were false negatives (i.e., the children classified as not having special needs by our method who were identified as CSHCN by the CCC Screener) and false positives (i.e., children classified as CSHCN by our method who were identified as not having special needs by the CCC Screener). Although racial and ethnic identity did not significantly predict false negatives, children with a major diagnostic category (MDC) of mental illness and children with a moderate chronic condition were more likely to be false negative than children who did not have an MDC of mental illness or who did not have a chronic condition. This finding is driven by the fact that these children have some service use, but they may not yet have received diagnoses or have service information that indicates that they are CSHCN within the current method. The results regarding false positives showed that Asian children and children of unknown race were more likely to be false positive than were White children. This may be because the parents or guardians of Asian children or children of unknown race may be less likely to report their children's mental and physical health problems on the CCC Screener. Additionally, children with high medication use, specialized therapy use, mental illness, chronic conditions, SSI, or IDD were less likely to be false positives, which make sense due to the way the risk score was constructed.
The CCC Screener was used as the gold standard for measuring CSHCN, as it is one of the most commonly used, well‐validated tools to delineate which children have special health care needs. However, to be classified as having special needs on the CCC Screener, the child's parent or guardian must respond positively to three sequential questions: (1) they must report that their child meets the criteria for at least one of the special need categories; (2) they must indicate that their child meets the category because of a medical, behavioral, or other health condition; and (3) they must note that their child's condition has lasted or is expected to last for at least 12 months. Given the subjective nature of these questions, parents' and guardians' answers may incorrectly categorize their child.
This method is not perfect. It has a sensitivity of 75 percent and a specificity of 79 percent, which means that 25 percent of CSHCN were classified as not having special needs and 21 percent of children without special needs were classified as CSHCN. Therefore, this method should not be used to label individual children as CSHCN nor should it be used to determine program eligibility for CSHCN. This method was developed to estimate the prevalence of CSHCN and create a program‐level indicator of special needs status to inform quality improvement efforts within NYS Medicaid Managed Care.
Future Studies
Although a stratified random sample of 1,500 children was drawn per health plan when NYS Medicaid members were contacted to complete the CCC Screener, the sample of parents and guardians who completed the screener differed from the sample of parents and guardians who did not. Parents and guardians of children who did not use specialized therapies and whose children had fewer days of prescribed medication were less likely to complete the survey than parents and guardians of children who used therapies and who were prescribed with more than 3 months of medication. This may inflate the percentage of the population labeled as CSHCN in this study because parents and guardians of healthier children were less inclined to complete the Screener. Additionally, parents and guardians of Black and multiracial children, parents and guardians of younger children, and parents and guardians in Central, Northeastern, and Western NY were less likely to complete the survey than were the parents and guardians of White children, parents and guardians of older children, and parents and guardians in NYC. It is important that future research examines the generalizability of the current method within different populations.
Many researchers have endorsed the idea that quality of care needs to be examined among children with and without special needs (Szilagyi et al. 2003; Simon et al. 2010; Cohen et al. 2011). Researchers suggest that it is especially important to examine quality of care within domains that CSHCN report high levels of unmet health care needs (Szilagyi et al. 2003). Future research should examine (1) whether there are disparities in unmet health care needs between children with and without special health care needs and (2) whether there are disparities in the quality of care that CSHCN receive.
Supporting information
Appendix SA1: Author Matrix.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: This research was conducted by and financially sponsored by the Office of Quality and Patient Safety within the New York State Department of Health. The authors state that they have no conflict of interest and no other disclosures.
Disclosures: None.
Disclaimers: None.
Notes
To be congruent with the Maternal and Child Health Bureau definition of CSHCN which states that children with special health care needs have certain types of conditions and have greater than average service usage, we examined whether including service utilization (i.e., inpatient, outpatient, and emergency department), as well as conditions would improve the diagnostic ability of our model. Since including service utilization did not improve the predictive ability of our model, it was not included in the current analysis.
The cut points with the highest area under the curve values (AUC) were 3 and 4. A cut point of 3 had a sensitivity of 0.75, specificity of 0.79, and AUC of 0.77. A cut point of 4 had a sensitivity of 0.66, specificity of 0.88, and AUC of 0.77.
References
- Aday, L. A. 1992. “Health Insurance and Utilization of Medical Care for Chronically Ill Children with Special Needs. Health of our Nation's Children, United States, 1988.” Advance Data 215: 1–8. [PubMed] [Google Scholar]
- An, R. 2016. “Unmet Mental Health Care Needs in U.S. Children with Medical Complexity, 2005‐2010.” Journal of Psychosomatic Research 82: 1–3. [DOI] [PubMed] [Google Scholar]
- Averill, R. F. , Goldfield N. I., Eisenhandler J., Muldoon J. H., Hughes J. S., Neff J. M., Gay J. C., Gregg L. W., Gannon D. E., Shafir B. V., Bagadia F. A., and Steinbeck B. A.. 1999. Development and Evaluation of Clinical Risk Groups (CRGs). Wallingford, Conn: 3M Health Information Systems. [Google Scholar]
- Beers, N. S. , Kemeny A., Sherritt L., and Palfrey J. S.. 2003. “Variations in State‐Level Definitions: Children with Special Health Care Needs.” Public Health Reports 118 (5): 434–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bethell, C. D. , Read D., Stein R. E. K., Blumberg S. J., Wells N., and Newacheck P. W.. 2002a. “Identifying Children with Special Health Care Needs: Development and Evaluation of a Short Screening Instrument.” Ambulatory Pediatrics 2 (1): 38–48. [DOI] [PubMed] [Google Scholar]
- Bethell, C. D. , Read D., Neff J., Blumberg S. J., Stein R. E., Sharp V., and Newacheck P. W.. 2002b. “Comparison of the Children with Special Health Care Needs Screener to the Questionnaire for Identifying Children with Chronic Conditions–Revised.” Ambulatory Pediatrics 2 (1): 49–57. [DOI] [PubMed] [Google Scholar]
- Bethell, C. D. , Read D., Blumberg S. J., and Newacheck P. W.. 2008. “What is the Prevalence of Children with Special Health Care Needs? Toward an Understanding of Variations in Findings and Methods across Three National Surveys.” Maternal and Child Health Journal 12 (1): 1–14. [DOI] [PubMed] [Google Scholar]
- Bethell, C. D. , Blumberg S. J., Stein R. E. K., Strickland B., Robertson J., and Newacheck P. W.. 2015. “Taking Stock of the CSHCN Screener: A Review of Common Questions and Current Reflections.” Academic Pediatrics 15 (2): 165–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bramlett, M. D. , Read D., Bethell C., and Blumberg S. J.. 2009. “Differentiating Subgroups of Children with Special Health Care Needs by Health Status and Complexity of Health Care Needs.” Maternal and Child Health Journal 13 (2): 151–63. [DOI] [PubMed] [Google Scholar]
- C.A.H.M.I. 2002. “Approaches to Identifying Children and Adults with Special Health Care Needs: A Resource Manual for State Medicaid Agencies” [accessed on May 7, 2017]. Available at http://www.cahmi.org/wp-content/uploads/2014/06/CSHCNS-CMS-manual.pdf
- C.A.H.M.I. 2012. “Who are Children with Special Health Care Needs (CSHCN).” Data Resource Center, supported by Cooperative Agreement 1‐U59‐MC06980‐01 from the U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB) [accessed on April 2, 2012]. Available at http://www.childhealthdata.org
- Carle, A. C. , Blumberg S. J., and Poblenz C.. 2011. “Internal Psychometric Properties of the Children with Special Health Care Needs Screener.” Academic Pediatrics 11 (2): 128–35. [DOI] [PubMed] [Google Scholar]
- C.M.S. 2016. “Medicaid and CHIP Managed Care Final Rule: Improving Quality of Care for Medicaid Beneficiaries” [accessed on June 1, 2017]. Available at https://www.medicaid.gov/medicaid/managed-care/downloads/improving-the-quality-of-care-for-medicaid-beneficiaries-fact-sheet.pdf
- Cohen, E. , Kuo D. Z., Agrawal R., Berry J. G., Bhagat S. K., Simon T. D., and Srivastava R.. 2011. “Children with Medical Complexity: An Emerging Population for Clinical and Research Initiatives.” Pediatrics 127 (3): 529–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen, E. , Berry J. G., Camacho X., Anderson G., Wodchis W., and Guttmann A.. 2012. “Patterns and Costs of Health Care Use of Children with Medical Complexity.” Pediatrics 130 (6): e1463–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dusing, S. C. , Skinner A. C., and Mayer M. L.. 2004. “Unmet Need for Therapy Services, Assistive Devices, and Related Services: Data from the National Survey of Children with Special Health Care Needs.” Ambulatory Pediatrics 4 (5): 448–54. [DOI] [PubMed] [Google Scholar]
- Fischer, J. E. , Bachmann L. M., and Jaeschke R.. 2003. “A Readers' Guide to the Interpretation of Diagnostic Test Properties: Clinical Example of Sepsis.” Intensive Care Medicine 29 (7): 1043–51. [DOI] [PubMed] [Google Scholar]
- Hill, K. S. , Freeman L. C., Yucel R. M., and Kuhlthau K. A.. 2008. “Unmet Need among Children with Special Health Care Needs in Massachusetts.” Maternal and Child Health Journal 12 (5): 650–61. [DOI] [PubMed] [Google Scholar]
- Houtrow, A. J. , Kim S. E., and Newacheck P. W.. 2008. “Health Care Utilization, Access, and Expenditures for Infants and Young Children with Special Health Care Needs.” Infants & Young Children 21 (2): 149–59. [Google Scholar]
- Houtrow, A. J. , Okumura M. J., Hilton J. F., and Rehm R. S.. 2011. “Profiling Health and Health Related Services for Children with Special Health Care Needs with and without Disabilities.” Academic Pediatrics 11 (6): 508–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mayer, M. L. , Skinner A. C., and Slifkin R. T.. 2004. “Unmet Need for Routine and Specialty Care: Data from the National Survey of Children with Special Health Care Needs.” Pediatrics 113 (2): e109–15. [DOI] [PubMed] [Google Scholar]
- McDermott, S. , Royer J., Cope T., Lindgren S., Momany E., Lee J. C., McDuffie M. J., Lauer E., Kurtz S., and Armour B. S.. 2018. “Using Medicaid Data to Characterize Persons With Intellectual and Developmental Disabilities in Five U.S. States.” American Journal on Intellectual and Developmental Disabilities 123 (4): 371‐81. [DOI] [PubMed] [Google Scholar]
- McPherson, M. , Arango P., Fox H., Lauver C., McManus M., Newacheck P. W., Perrin J. M., Shonkoff J. P., and Strickland B.. 1998. “A New Definition of Children with Special Health Care Needs.” Pediatrics 102 (1 Pt 1): 137–40. [DOI] [PubMed] [Google Scholar]
- Neff, J. M. , Sharp V. L., Muldoon J., Graham J., Popalisky J., and Gay J. C.. 2002. “Identifying and Classifying Children with Chronic Conditions Using Administrative Data with the Clinical Risk Group Classification System.” Ambulatory Pediatrics 2 (1): 71–9. [DOI] [PubMed] [Google Scholar]
- Neff, J. M. , Clifton H., Park K. J., Goldenberg C., Popalisky J., Stout J. W., and Danielson B. S.. 2010. “Identifying Children with Lifelong Chronic Conditions for Care Coordination by Using Hospital Discharge Data.” Academic Pediatrics 10 (6): 417–23. [DOI] [PubMed] [Google Scholar]
- Newacheck, P. W. , and Halfon N.. 1998. “Prevalence and Impact of Disabling Chronic Conditions in Childhood.” American Journal of Public Health 88 (4): 610–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Newacheck, P. W. , McManus M. A., and Fox H. B.. 1991. Prevalence and Impact of Chronic Illness among Adolescents. American Journal of Diseases of Children (1960) 145 (12): 1367–73. [DOI] [PubMed] [Google Scholar]
- Newacheck, P. W. , Strickland B., Shonkoff J. P., Perrin J. M., McPherson M., McManus M., Lauver C., Fox H., and Arango P.. 1998. “An Epidemiologic Profile of Children with Special Health Care Needs.” Pediatrics 102 (1): 117–23. [DOI] [PubMed] [Google Scholar]
- Perkins, N. J. , and Schisterman E. F.. 2006. “The Inconsistency of “Optimal” Cut‐points Using Two ROC Based Criteria.” American Journal of Epidemiology 163 (7): 670–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perrin, E. C. , Newacheck P., Pless I. B., Drotar D., Gortmaker S. L., Leventhal J., Perrin J. M., Stein R. E., Walker D. K., and Weitzman M.. 1993. “Issues Involved in the Definition and Classification of Chronic Health Conditions.” Pediatrics 91 (4): 787–93. [PubMed] [Google Scholar]
- Quach, J. , Jansen P. W., Mensah F. K., and Wake M.. 2015. “Trajectories and Outcomes among Children with Special Health Care Needs.” Pediatrics 135 (4): e842–50. [DOI] [PubMed] [Google Scholar]
- Sarkar, M. , Earley E. R., Asti L., and Chisolm D. J.. 2017. “Differences in Health Care Needs, Health Care Utilization, and Health Care Outcomes among Children with Special Health Care Needs in Ohio : A Comparative Analysis between Medicaid and Private Insurance.” Journal of Public Health Management and Practice 23 (1): e1. [DOI] [PubMed] [Google Scholar]
- Shapiro, D. E. 1999. “The Interpretation of Diagnostic Tests.” Statistical Methods in Medical Research 8 (2): 113–34. [DOI] [PubMed] [Google Scholar]
- Simon, T. D. , Berry J., Feudtner C., Stone B. L., Sheng X., Bratton S. L., Dean J. M., and Srivastava R.. 2010. “Children with Complex Chronic Conditions in Inpatient Hospital Settings in the United States.” Pediatrics 126 (4): 647–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stein, R. E. , Westbrook L. E., and Bauman L. J.. 1997. “The Questionnaire for Identifying Children with Chronic Conditions: A Measure Based on a Noncategorical Approach.” Pediatrics 99 (4): 513–21. [DOI] [PubMed] [Google Scholar]
- Stein, R. E. , Bauman L. J., Westbrook L. E., Coupey S. M., and Ireys H. T.. 1993. “Framework for Identifying Children Who Have Chronic Conditions: The Case for a New Definition.” Journal of Pediatrics 122 (3): 342–7. [DOI] [PubMed] [Google Scholar]
- Stewart, K. A. , Zickafoose J. S., Wu B., Brown R. S., and Ireys H. T.. 2014. “Association between NCQA Patient‐Centered Medical Home Recognition for Primary Care Practices and Quality of Care for Children with Disabilities and Special Health Care Needs” [accessed on May 30, 2017]. Available at https://aspe.hhs.gov/pdf-report/association-between-ncqa-patient-centered-medical-home-recognition-primary-care-practices-and-quality-care-children-disabilities-and-special-10health-care-needs:MathematicaPolicyResearch.
- Swets, J. 1988. “Measuring the Accuracy of Diagnostic Systems.” Science 240 (4857): 1285–93. [DOI] [PubMed] [Google Scholar]
- Szilagyi, P. G. , Shenkman E., Brach C., LaClair B. J., Swigonski N., Dick A., Shone L. P., Schaffer V. A., Col J. F., Eckert G., Klein J. D., and Lewit E. M.. 2003. “Children with Special Health Care Needs Enrolled in the State Children's Health Insurance Program (SCHIP): Patient Characteristics and Health Care Needs.” Pediatrics 112 (6): e508–e20. [PubMed] [Google Scholar]
- Warfield, M. E. , and Gulley S.. 2006. “Unmet Need and Problems Accessing Specialty Medical and Related Services among Children with Special Health Care Needs.” Maternal and Child Health Journal 10 (2): 201–16. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix SA1: Author Matrix.