Identifying reproductive-aged women with physical and sensory disabilities in administrative health data: A systematic review

Abstract

Background:

Women with disabilities experience significant health disparities. A barrier to progress in addressing these disparities is the lack of population-based data on their health outcomes, which are needed to plan health care delivery systems. Administrative health data are increasingly being used to measure the health of entire populations, but these data may only capture impairment and not activity and participation restrictions.

Objective/Hypothesis:

We conducted a systematic review to identify and appraise existing literature on the development and validation of algorithms to identify reproductive-aged women with physical and sensory disabilities in administrative health data.

Methods:

We searched Medline, EMBASE, CINAHL, PsycINFO, and Scopus from inception to April 2019 for studies of the development and/or validation of algorithms using diagnostic, procedural, or prescription codes to identify physical and sensory disabilities in administrative health data. Study and algorithm characteristics were extracted and quality was assessed using standardized instruments.

Results:

Of 14,073 articles initially identified, we reviewed 6 articles representing 2 unique algorithms. One algorithm aimed to correlate diagnoses, procedure codes, and prescriptions with ability to access routine care as an indicator of functional limitation. The other algorithm used diagnostic and procedure codes to identify use of mobility-assistive devices to measure functional limitation. Only one algorithm was validated against self-reported disability.

Conclusions:

Our findings underscore the need to strengthen current methods to identify disability in administrative health data, including linkage with other sources of information on functional limitations, so that population-based data can be used to optimize health care for women with disabilities.

INTRODUCTION

The International Classification of Functioning, Disability, and Health (ICF) defines disability as “impairments, activity limitations, and participation restrictions” that result from environments that do not accommodate differing abilities (1). Disabilities vary by cause, timing, and impact. They can be classified, based on common limitations (2), as physical disabilities that impact mobility (e.g., cerebral palsy), sensory disabilities that impact hearing or vision, psychiatric disabilities that impact emotions or behaviours (e.g., schizophrenia), and intellectual and developmental disabilities that impact cognitive and adaptive abilities (e.g., Down syndrome, autism spectrum disorder). Approximately one in four North Americans has a disability (3). Disabilities significantly impact reproductive-aged women; 12% of women aged 15 to 44 years in the United States and Canada have a disability (3, 4).

Women with disabilities experience multiple health disparities, including high rates of chronic medical conditions and mental illness, low rates of preventive care access, and increased risk for negative reproductive and perinatal outcomes (5–10). Yet, they report that their medical care is usually focused on their disability rather than on their broader health needs (11). An obstacle to progress in the development of comprehensive health care programs is the lack of population-based data on the health of reproductive-aged women with disabilities. Quantitative studies in this area have typically relied on surveys (5, 6), clinical data from selected hospitals (9), or insurance claims (10). While some surveys provide data relevant to the ICF definition of disability (e.g., activity limitations), samples are restricted to individuals who consent and are able to participate in research, thereby creating important selection bias by excluding those with severe disabilities or challenging social circumstances. One solution to this selection bias involves use of de-identified clinical and health insurance data, which may cover whole population subgroups and facilitate analysis without participant effort. However, reliance on clinical data from specialized hospitals skews samples toward high-risk groups, and individuals who qualify for certain types of health insurance (e.g., Medicaid) may differ from the broader population in terms of characteristics related to education, income, and employment (14).

Administrative health data are increasingly used to measure the health of entire populations. Diagnostic, procedure, and prescription codes from administrative health data facilitate construction of large population-based cohorts capable of estimating rare outcomes and producing generalizable results. While identification of disability using such codes focuses on impairment (15), some algorithms relate codes to the probability of activity limitations and participation restrictions. For example, the Access Risk Classification System identifies individuals with disabilities according to their ability to access routine care (16). Disability algorithms have two limitations when applied to the identification of women with disabilities. First, due to the frequent inability to link health records over long spans of time, algorithms have been applied to health care encounters over short periods of time (e.g., 1 year) (16), thereby missing disabilities diagnosed in childhood and not documented thereafter. Second, disability algorithms typically ignore specific categories of disability (e.g., physical, sensory, psychiatric, intellectual and developmental) for comparisons within and across groups, despite the heterogeneity of health and health care challenges faced by individuals with different types of disabilities. These limitations to existing disability algorithms must be addressed to maximize the utility of administrative health data in studying health outcomes among women with disabilities.

We conducted a systematic review to identify and appraise literature on the development and validation of algorithms to identify reproductive-aged women with physical disabilities and sensory disabilities in administrative health data. We focused on these two disabilities because prior studies have already considered algorithms for adults with intellectual and developmental disabilities (18) and psychiatric disorders (19, 20) using administrative health data.

METHODS

Data sources and search strategy

Our systematic review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (21). In consultation with a research librarian, we used a published search strategy for physical disabilities and sensory disabilities (22) along with MeSH terms and keywords for algorithms and administrative data to search Medline, EMBASE, CINAHL, PsycINFO, and SCOPUS from inception to April 10, 2019 for studies on the development and/or validation of algorithms to identify physical and sensory disabilities in administrative health data (Appendix A). To identify articles missed in the database search, backward citation searching was conducted by hand-searching the bibliographies of all articles chosen for full-text review. Forward citation searching was conducting by examining Google Scholar to determine if relevant articles had referenced the articles chosen for full-text review.

Eligibility criteria

To be included in our investigation, published studies were required to: (a) report on the development and/or validation of algorithms using diagnostic, procedure, and/or prescription codes to identify disability in administrative health data; (b) be peer-reviewed; and (c) be written in English. Since our preliminary literature review revealed that many algorithms capture disability in general and not specific categories of disabilities (16), we included all studies of disability in general and those of physical and sensory disabilities specifically. We excluded studies that (a) described algorithm use but not development or validation; (b) were age-restricted to children or elderly populations; (c) examined psychiatric or intellectual and developmental disabilities only (18–20); (d) identified chronic disease, but not disability specifically (e.g., Charlson Comorbidity Index; Johns Hopkins Adjusted Clinical Groups); and (e) exclusively used the US Social Security Administration’s definition of disability, since this agency’s disability determinations require participants to have substantial financial needs and be incapable of substantial gainful activity (i.e., work) (23). Two authors independently reviewed titles and abstracts over a 2-month period to determine whether studies met our inclusion criteria.

Data extraction

We developed a database a priori to enter salient study data. From each study, two authors independently extracted data on study descriptors (i.e., location, study period, study design, data sources, inclusion criteria, sample size, type(s) of disability investigated); algorithm development (i.e., conceptual definition of disability, data used to identify disability, categories of disability identified, modifications relevant to reproductive-aged women); and algorithm validation (i.e., data sources, gold standard used for validation, results). For validation results, we examined accuracy (i.e., the area under the Receiver-Operator Curve, which graphically represents the degree to which an algorithm is able to distinguish between individuals with and without a disability), sensitivity (i.e., the ability of an algorithm to correctly classify individuals as having a disability), specificity (i.e., the ability of an algorithm to correctly classify individuals as not having a disability), positive predictive value (i.e., the probability that individuals classified by an algorithm as having a disability actually have a disability), and negative predictive value (i.e., the probability that individuals classified by an algorithm as not having a disability do not actually have a disability). Values > 0.70 are considered acceptable (24). Two co-authors discussed discrepancies in data extraction until reaching consensus.

Quality assessment

Two authors independently assessed article quality using an adapted version of the Cochrane Public Health quality assessment tool (25). The focus of the quality assessment was on components of the study design relevant to the development and/or validation of the algorithms. Studies were rated on selection bias (response rate, representativeness), attrition bias, and missing data for the development sample and validation sample (where applicable). Several identified studies reported on the association between disability status and a specific health outcome (e.g., incidence of chronic disease or pregnancy complications) in addition to describing their algorithm. However, because these associations were not the focus of our review, we did not evaluate their risk of confounding. Risk of bias was rated as being present, absent, or unclear. We discussed discrepancies in assigned quality ratings until we reached consensus.

Data synthesis

We described the algorithms along with their development and validation using a narrative synthesis.

RESULTS

Search results

Figure 1 shows the article selection process. The database search revealed 14,073 articles. After removal of duplicates, 13,559 articles were screened based on title and abstract, and 13,532 irrelevant articles were excluded. Twenty-seven potentially relevant full-texts were evaluated. We excluded 21 of these because they (a) described algorithm use but provided no data on algorithm development or validation (n=4); (b) were restricted to either pediatric or elderly populations (n=4); (c) examined only psychiatric disabilities or intellectual and developmental disabilities (n=5); (d) examined chronic disease, not disability (n=3); (e) used the US Social Security Administration definition of disability (n=3); or (f) were not written in English (n=2). We included 6 articles that represented 2 unique algorithms. Two articles described algorithm development and validation (16, 26), three studies reported further development (e.g., additions) relevant to reproductive-aged women (17, 27, 28), and one report described validation only (29).

Figure 1.

Article selection process for the systematic review.

Characteristics of included studies

Study characteristics are described in Table 1. All studies were retrospective cohort studies conducted in the United States. All used population-based data to develop and/or modify their algorithms; validation was conducted with population-based data (n=1) (26) or surveys (n=2) (16, 29). Participants included adults (n=2) (16, 29), adult women (n=1) (26), and women with an obstetrical delivery (n=3) (17, 27, 28). Three studies included the entire population (17, 27, 28), and three were restricted to Medicare or Medicaid beneficiaries (16, 26, 29). Studies ranged from 12,415 to 6,280,443 participants. Most studies examined physical (n=4) (16, 26–28) or sensory disabilities separately (n=3) (16, 27, 28), while two examined disability in general (including, but not distinguishing between disability types) (17, 29). Disabilities in adult women and adults in the general population were measured in brief time periods of 1 (16), 2 (29), and 4.5 years (26). Disabilities in women with an obstetrical delivery were measured using the delivery record and one year before (17) or “near” (27, 28) the delivery hospitalization.

Table 1.

Study characteristics.

First Author, Year	Study location, study period	Study design	Data sources	Inclusion criteria	Sample size	Disability investigated
Ben-Shalom, 2016 (29)	United States, 2003–2006	Retrospective cohort study	Medicare claims files	Community-dwelling Medicare beneficiaries aged 18 years and older	N=12,415	Any disability, measured in 2 years of claims data
Clements, 2016 (17)	Massachusetts, United States, 2006–2009	Retrospective cohort study	Massachusetts Pregnancy to Early Life Longitudinal Data System	Female residents of any age with an obstetrical delivery	N=221,867	Any disability, measured one year before and during delivery hospitalization
Darney, 2017 (27)	California, United States, 2000–2010	Retrospective cohort study	Linked birth and death certificates and hospital discharge data	Women of any age with an obstetrical delivery, excluding those with prior cesarean section, multiple gestations, or breech presentation	N=4,610,955	Physical, sensory, and intellectual and developmental disabilities separately, measured at or near delivery hospitalization
Horner-Johnson, 2017 (28)	California, United States, 2000–2010	Retrospective cohort study	Linked birth and death certificates and hospital discharge data	Women of any age with an obstetrical delivery, excluding those with multiple gestations, breech presentation, and pre-viable gestational ages	N=5,432,533	Physical, sensory, and intellectual and developmental disabilities separately, measured at or near delivery hospitalization
Khoury, 2013 (26)	Florida, United States, 2001–2005	Retrospective cohort study	Medicaid eligibility and claims files	Female Medicaid beneficiaries, aged 18–64 years, with continuous enrolment and ≥1 outpatient visits, excluding those with dual Medicaid / Medicare eligibility or limited benefits	N=74,851	Physical disabilities, measured in 4.5 years of claims data
Palsbo, 2008 (16)	California, United States, 2003	Retrospective cohort study	Inland Empire Health Plan data	Medicaid beneficiaries aged 18 years and older, with continuous enrolment for the 12-month period and with English as the preferred language	N=29,695	Physical, sensory, and intellectual and developmental disabilities separately, measured in 12 months of claims data

Study quality

Table 2 reports the evaluation of study quality. Assessments of selection bias, attrition bias, and missing data were conducted in relation to both the population-based samples used to develop and/or modify the algorithms and the samples used to evaluate their validity. In relation to the samples used to develop or modify the algorithms, none of the included studies had risk of selection bias because all used population-based data including linked vital statistics and hospital discharge (17, 27, 28) and Medicare or Medicaid data (16, 26, 29), although the latter may only be generalizable to populations eligible for these programs. Due to good linkage of data for these retrospective cohort studies, risk of attrition bias was low. However, several studies had unclear or high risk of bias due to missing data (n=3) (16, 17, 26). One study excluded individuals with missing data from the analyses, but did not report the rate of missingness or how individuals with missing data differed from participants who were included in the study (17); others did not report on missingness at all (n=2) (16, 26). The two studies that performed validation analyses did so with surveys in which respondents were asked to self-report disability-related variables. Ben-Shalom et al. (29) did not report a response rate, while Palsbo et al. (16) noted there was only a 13% response rate in their validation survey. Risk of attrition bias in validation samples was low, but the risk of bias from missing data was high (16, 29): Ben-Shalom et al. (29) noted younger individuals in their Medicare sample who were dually eligible for Medicaid and Medicare were more likely to be missing data on services due to coverage by Medicaid rather than Medicare. Individuals who are dually eligible for Medicaid and Medicare have high health care costs, which are typically attributed to excessive prevalence of chronic and disabling conditions (30), raising concerns about selection bias. Palsbo et al. (16) reported up to 10% missingness for disability diagnostic and prescription codes for validation survey respondents.

Table 2.

Risk of bias in the included studies.

First author, year	Algorithm development/modification				Algorithm validation
	Selection bias	Attrition bias	Missing data	Selection bias	Attrition bias	Missing data
Ben-Shalom, 2016 (29)	n/a	n/a	n/a	Unclear	n/a	Present
Clements, 2016 (17)	Absent	Absent	Unclear	n/a	n/a	n/a
Darney, 2017 (27)	Absent	Absent	Absent	n/a	n/a	n/a
Horner-Johnson, 2017 (28)	Absent	Absent	Absent	n/a	n/a	n/a
Khoury, 2013 (26)	Absent	n/a	Unclear	n/a	n/a	n/a
Palsbo, 2008 (16)	Absent	Absent	Unclear	Present	Absent	Present

n/a = not applicable

Disability algorithms

Access Risk Classification System (ARCS)

The Access Risk Classification System (ARCS) used health insurance claims data to identify disability according to the need for assistance or accommodation while accessing routine health care (16). The ARCS attempted to incorporate the ICF definition of disability by recognizing that disability is jointly produced by an individual’s impairment and his or her environment. In this context, “access risk” reflected the “gap between the environment and an individual’s ability to navigate that environment” (16). The ARCS algorithm used ICD-9 diagnostic codes, Common Procedure Terminology (CPT) codes, and prescriptions to identify likelihood of functional limitations. Individuals were classified as level A (i.e., no risk, with acute or emergent medical needs), level B (i.e., low risk, with ≥ 1 chronic conditions associated with functional limitations), level C (i.e., medium risk, with ≥ 1 chronic conditions associated with major functional limitations), and level D (i.e., high risk, with multiple chronic conditions associated with severe functional limitations, resulting in complex medical needs) (16, 17). While distinctions were not made between categories of disability (i.e., physical, sensory) in the original study, diagnoses could be identified and subsequently aggregated by body system (16).

Palsbo et al. (16) used self-reported disability to validate the ARCS algorithm’s ability to identify functional limitations. Classification in the most complex needs group, Level D, had a positive predictive value of 59% and negative predictive value of 57%; classification in levels C or D had positive and negative predictive values of 72% and 46%; and classification in levels B, C, or D had positive and negative predictive values of 89% and 29%, respectively. Overall, 26% of individuals were correctly classified for level A, 10% for level B, 19% for level C, and 70% for level D. Levels A, B, and C tended to over-identify disability, while level D tended to under-identify disability. Level D is often used as the cut-off for defining disability (29), since it represents the most complex needs, which are most likely to affect functioning. However, Palsbo et al. (16) showed that in adults 18 years and older, classifying disability at levels C or D had 88% sensitivity and 30% specificity compared to self-report.

Ben-Shalom et al. (29) validated the ARCS along with 5 other algorithms (none of which met our criteria) among 18 to 64-year-olds. Compared to 44% of those who self-reported ≥ 1 limitations in activities of daily living, and 70% who reported ≥ 1 limitations in activities of daily living or instrumental activities of daily living, 71% of individuals were rated by the ARCS as being in levels C or D. After adjusting for sex, age, and Medicaid and Medicare dual eligibility, classification in ARCS levels C or D was a significant predictor of having ≥ 1 self-reported limitations to activities of daily living (OR=1.41, p<0.05). However, combining the ARCS with 5 other algorithms to predict self-reported disability, the area under the Receiver-Operator Curve was 0.62. Individuals who self-reported disability but were classified by the algorithm as not having a disability were more likely than those who were correctly classified to be dually eligible for Medicaid and Medicare, suggesting data on long-term supports could improve predictions.

In a population-based cohort of women with an obstetrical delivery, Clements et al. (17) subsequently modified the ARCS to exclude ICD-9 codes for non-malignant neoplasms, since these were deemed unlikely to cause functional limitations. The authors did not conduct a formal validation of this revised algorithm.

Khoury et al. (2013) (algorithm not named)

Khoury et al. (26) developed their algorithm by listing 41 conditions associated with mobility limitations using ICD-9 codes for physically disabling conditions and CPT codes for use of mobility-assistive devices such as wheelchairs, walkers, and canes. Although the authors acknowledged their reliance on diagnostic codes made their algorithm consistent with a medical model of disability, the inclusion of mobility-assistive device use in their analyses provided more nuanced information about functional limitation than diagnostic codes alone. Diagnostic codes were classified as congenital anomalies, diseases of the musculoskeletal and connective tissue, disorders of the peripheral nervous system, hereditary and degenerative diseases of the central nervous system, injuries, and other diseases of the central nervous system. The authors used an ordinal scale to identify physical disability: no physical disability or assistive device, physically disabling condition but no assistive device, and physically disabling condition and assistive device utilization. This categorization was meant to reflect the idea that disability is not “an all or nothing phenomenon”, but rather is defined by varying levels of functional limitation.

The authors stated their algorithm was validated via a review of the diagnostic and procedural codes by a disability epidemiologist and a physician, who examined these codes for face validity. They did not report quantitative characteristics of their validation procedure, but noted the two reviewers were highly consistent in their code assessments.

This algorithm was subsequently adapted by Darney et al. (27) and Horner-Johnson et al. (28) in a population-based cohort of women with obstetrical deliveries. In consultation with clinicians and disability researchers, the authors added diagnostic codes for several conditions (i.e., cystic fibrosis, late effects of polio, limb amputation, and spinal muscular atrophy) that may be associated with physical disability but not necessarily with mobility limitations. These authors also removed diagnostic codes for acute injuries that may not have a lasting effect (e.g., spinal fracture without spinal cord injury). Neither of these studies conducted a formal validation of this revised algorithm.

DISCUSSION

Summary of findings

We identified six studies describing the development and/or validation of two unique algorithms to identify women with physical and sensory disabilities using administrative health data. These algorithms largely relied on ICD-9 diagnostic codes and procedural codes, and one also used prescriptions. In the ARCS, Palsbo et al. (16) argued that by examining correlations between disability diagnoses and subsequent need for assistance accessing routine health care, their algorithm captured activity and participation restrictions relevant to the ICF definition of disability. Khoury et al. (26) explicitly stated that their algorithm could only address the medical model of disability but used procedure codes to assess severity of functional limitations. Only Palsbo et al. (16) examined the validity of their algorithm in relation to self-reported disability, with modest results. Our findings underscore the need for further research to explore whether and how disability can be identified using administrative health data.

Findings in the context of previous research

To our knowledge, this is the first systematic review to identify and appraise studies on the development and validation of algorithms to identify physical and sensory disabilities using administrative health data. Several of our findings align with Iezzoni’s 2002 commentary on the use of administrative health data to study disability (15). Iezzoni emphasized that administrative health data do not provide information on limitations in the performance of daily activities, participation restrictions, or the physical or social environment, which are central to the ICF definition of disability. The studies included in our review differed in how they situated their algorithms in the context of the ICF definition. Palsbo et al. (16) attempted to correlate diagnoses included in the ARCS with functional limitations through their associations with need for assistance accessing routine health care. Broader measures of morbidity, such as the Charlson Comorbidity Index, also use health care utilization patterns to reflect morbidity severity. However, we argue that health care patterns do not clearly or consistently reflect disability specifically. Khoury et al. acknowledged that their algorithm could only follow a medical model of disability (26). It is noteworthy, however, that they included use of assistive devices as a measure of functional limitations in their algorithm. This study’s framing is more consistent with Iezzoni’s concerns about the ability of administrative health data to measure disability.

Recognizing that disability and health are related but distinct constructs has important implications. Conflation of disability and illness falsely presupposes that people with disabilities cannot be healthy and prevents public health attention to the health disparities among people with vs. without disabilities (31). This is particularly important in the context of health care needs that are specific to reproductive-aged women with disabilities. Such health care needs may include preconception health, when the management of chronic conditions such as diabetes and hypertension are paramount to ensuring optimal perinatal outcomes (32). They may also include broader preventive health care needs, including cancer screening (33). Algorithms that do not distinguish between disability and health hinder the investigation of disparities in these areas.

The challenges of using administrative health data to identify disability described in our review are similar to those reported by the authors of algorithms used to identify disability in different age groups or different types of disabilities. For example, Chien et al. (34) compiled a list of 669 ICD codes classified by pediatricians as having ≥ 75% probability of being associated with the ICF definition of disability. Their algorithm had 75% sensitivity and 86% specificity compared to parent report of long-term functional impairments and 98% sensitivity and 50% specificity compared to physician assessment of disability from patient charts (34). Similar to our review, the authors noted the challenges of using administrative health data, including differences between perceived disability and diagnosed disability (34). Lin et al. developed an algorithm to identify intellectual and developmental disabilities in administrative health data using diagnoses recorded in physician visits, emergency department visits, and hospitalizations (35). However, when they applied this algorithm to both administrative health data and social services data on disability income support recipients, they found that administrative health data alone only identified 66.4% of adults with intellectual and developmental disabilities (36). As we describe below, administrative health data alone may be inadequate to identify disability.

Limitations

Our findings must be considered in the context of the limitations of the included studies as well as our review’s methodological limitations. Not all included studies were thorough in their conduct or reporting of algorithm validation. While the authors of the ARCS validated their algorithm against self-reported disability (17), Khoury et al. (26) only made general statements about reviewers supporting the face validity of their diagnostic and procedural codes. For the ARCS, classification in levels C or D was significantly associated with self-reported limitations of activities of daily living (29). However, disability status was misclassified for a large proportion of the population at every level of the classification system, with positive and negative predictive values often falling well below the standard of 0.70. Misclassification even occurred in level D (the most complex needs), for which 30% of individuals were not correctly classified (16). Future research should validate algorithms against formal assessments of functioning, including difficulties with activities of daily living, self-report of disability, or both.

Diagnostic codes carry other potential issues when applied to the measurement of disability. First, while some diagnoses (e.g., spinal cord injuries) allow for inferences about functional limitations (e.g., mobility), in many cases, the diagnosis reveals little about functional characteristics of the individual and nothing about the environment. Second, chronic conditions tend to be under-reported in administrative health data because the focus of most health care encounters is acute health problems (15). This phenomenon is particularly problematic for disabilities that are not the focus of active treatment, such as sensory disabilities diagnosed in childhood (15). Such disabilities may not be coded in later records. This lack of fit is a significant issue for cross-sectional studies completed in adulthood and retrospective cohort studies with short lookback periods. All but two of the six studies included in our review examined disability diagnoses within the previous year (16, 17, 27, 28). However, other studies have shown that even in conditions as severe as quadriplegia and multiple sclerosis, only 57% and 58% of people with these conditions, respectively, had codes for these conditions recorded again in the year after their diagnosis (37). Finally, diagnostic codes may be limited in their accuracy and completeness. None of the authors indicated the accuracy of their codes.

The studies we identified were conducted in the United States, and most used Medicaid or Medicare data. These algorithms may have limited utility in other datasets that differ from these (i.e., lack procedure or prescription codes, or do not employ ICD-9-CM codes). We explicitly excluded studies which only measured general morbidity (e.g., the Charlson Comorbidity Index, Johns Hopkins Adjusted Clinical Groups) as well as those that exclusively used the US Social Security Administration’s definition of disability, since this agency’s disability determination procedures require participants to have substantial financial needs and to be incapable of employment (23, 38). The latter resulted in the exclusion of three studies, one of which purportedly measured disability despite not distinguishing between chronic disease and disability in the context of determining how to allocate public health insurance payments in states’ administration of managed health care (37). Our findings therefore do not reflect broader work used for disability insurance payments or income transfer amounts.

Finally, our search may have missed studies on the development or validation of relevant algorithms, despite our efforts to avoid this problem with the application of a published disability search strategy, consultation with a research librarian to develop the other components of the database search, and the use of two independent reviewers to assess all titles and abstracts. For example, studies that used administrative health data in their measurement of disability may have been missed if the authors did not use the terms related to administrative data that were included in our search strategy in their title or abstract. Further, the published search strategy for disability that we used (22) is not without its own limitations. For example, a recent article testing this search strategy (39) found that there was little overlap between articles returned using the general disability search terms of the algorithm and articles returned using condition-specific terms. This may also have resulted in omitting some articles that may have been eligible for our study. However, we expect that our efforts to identify articles missed in the initial database search, such as the use of backward and forward citation searches, mitigate some of these concerns.

Implications

Population-based data are critical for health care planning because they provide information on entire populations with statistical power to examine rare but serious health problems. Findings from this review highlight the need to improve administrative health data if these data are to be used to study disability. The optimal approach would be systematic inclusion of measures of functional limitations (not just diagnoses) in administrative health data. The ICF can be used to characterize several dimensions of disability related to functioning and contextual factors (40). Functioning and disability includes body structures and functions (i.e., physiological and anatomical changes and ability to complete daily activities) and activity and participation (i.e., performance of physical and mental activities). Contextual factors include environmental factors (i.e., physical and social environment) and personal factors (e.g., age, gender, social background). There are four levels of coding for body structures, body functions, activities and participation, and environmental factors, and each can be qualified in relation to the level of health or severity of the problem. The ICF system is being used in routine reporting in home care and rehabilitation services in Australia, Canada, India, Italy, Japan, and Mexico. However, uptake in general medical settings is limited (41, 42). Because the medical model of disability is still predominant in these settings (43), many health care providers do not assess functioning, or they sporadically document such assessments (44). Further, the ICF coding system is complex to administer and impractical in resource- and time-limited clinical settings (43). There may be opportunities to create abbreviated versions of the ICF (45). For example, a “Mini-ICF” has been developed for use with individuals with mental disorders (46). Similar assessments could be developed for other types of disabilities to improve the feasibility of implementation.

Until that time, there are several ways in which disability ascertainment methods can be strengthened using existing data sources. Most importantly, administrative health databases must be linked across time so diagnoses reflecting disability can be identified through and across multiple years; otherwise, diagnoses are likely to be underestimated. In perinatal populations in particular, working cross-sectionally means there is significant under-recognition of maternal disability status if only delivery records (or even maternal records in only the year before delivery) are used. This problem is particularly important for disabilities diagnosed during childhood that are not the direct subject of a recent health care visit, or conditions the provider does not consider to be related to the health care visit (e.g., hearing impairments). A life course perspective, which includes records of permanent disabilities recorded in childhood, is needed.

Researchers should also explore linkage of administrative health data with data sources that include functional assessments for income support programs or other services. While these datasets are limited in their scope by only being able to identify a subset of the population with disabilities based on other characteristics (i.e., inability to work, residence in a long-term care facility), together with administrative health data they may provide a more complete picture of disability status. Embedding questions about disability in other population-based data sources may also be useful. The Washington City Group created a set of six questions related to the most commonly occurring functional difficulties and some basic activities of daily living and instrumental activities of daily living (47). These questions have been used in the Censuses of approximately 30 countries and, if linked at the individual level to administrative health data, have enormous potential for the identification of disability in population-based data (48).

Finally, machine learning methods have been applied to structured data supplemented by clinical notes to provide better classification of psychiatric disabilities and specific physical disabilities such as rheumatoid arthritis (49, 50). Where administrative data are linked to electronic health records, such methods could prove valuable for optimizing and validating disability ascertainment algorithms.

CONCLUSIONS

Population-based data hold potential for providing evidence to support the optimization of health care for persons with disabilities. Understanding the physical and mental health, preventive health care needs, and reproductive and perinatal outcomes of women with disabilities across the life course is vital, so health care services can be tailored to meet their needs. We identified two unique algorithms to identify women with physical and sensory disabilities using administrative health data. Our findings demonstrate a clear need to strengthen current disability identification methods in administrative health data. Doing so will provide a necessary foundation to improve population-based research on reproductive-aged women with disabilities, and women’s health research more broadly.

Table 3.

Description of algorithms used to identify physical and sensory disabilities.

Algorithm name	Algorithm development				Modifications relevant to reproductive-aged women
	First author, year	Conceptual definition of disability	Data used to identify disability	Categories of disability identified
Access Risk Classification System	Palsbo, 2008 (16)	Disability defined according to the need for assistance or accommodation while accessing routine health care, where “access risk” represents a gap between the environment and an individual’s ability to navigate that environment	ICD-9 diagnostic codes, procedure codes, and prescriptions applied to claims data	Disability is identified for individuals with Levels C/D access risk, where Level A: no risk, Level B: low risk, Level C: medium risk, Level D: high risk;	Clements et al. (2016) (17) excluded non-malignant neoplasms, which are unlikely to be associated with functional limitations
Khoury et al. algorithm	Khoury, 2013 (26)	Physical disability is defined as the presence of one or more mobility limitations	ICD-9 diagnostic codes and procedure codes	Disability is defined on an ordinal scale as physical disabling condition and assistive device, physically disabling condition and no assistive device, or no physically disabling condition or assistive device	Darney et al. (2017) (27) and Horner-Johnson et al. (2017) (28) added several conditions associated with physical disability but not mobility limitations and removed acute injuries

Table 4.

Description of algorithm validation.

Algorithm name	First author, year	Data sources used for validation	Definition of disability used for validation	Validation results
				Overall accuracy	Sensitivity, specificity, PPV, NPV	Other
Access Risk Classification System	Ben-Shalom, 2016 (29)	Medicare Current Beneficiary Survey (panel survey of nationally representative sample of Medicare beneficiaries)	Self-reported disability on the basis of difficulties with activities of daily living (ADLs) or instrumental activities of daily living (IADLs)	Compared to 44.0% of individuals who self-reported ≥1 limitations to ADLs, and 70.3% who reported ≥1 limitations to ADLs or IADLs, 70.5% of individuals were rated as being ARCS levels C/D	Sensitivity = 54%, specificity = 67% (for ARCS and 5 other disability algorithms together)	After adjusting for sex, age, and dual eligibility, ARCS level C/D was a significant predictor of having ≥1 self-reported limitations to ADLs (OR=1.41, p<0.05)
	Palsbo, 2008 (16)	Survey mailed to a stratified random sample of Inland Empire Health Plan members entitled to Medicaid because of disability	Self-reported disability on the basis of a list of health problems and difficulties with activities of daily living	26% of individuals were correctly classified for level A, 10% for level B, 19% for level C, and 70% for level D	Level D: PPV = 59%, NPV = 57% Levels C/D: PPV = 72%, NPV = 46%, sensitivity 88%, specificity 30% Levels B/C/D: PPV = 89%, NPV = 29%	Levels A, B, and C tended to over-identify disability, while level D tended to under-identify disability
Khoury et al. algorithm	Khoury, 2013 (26)	Disability epidemiologist and physician reviewed the diagnostic and procedural codes for face validity	n/a	n/a	n/a	Reviewers were ‘highly consistent’ in their assessments of the codes