Abstract
Background
Despite expansions in public health insurance, many children remain uninsured or experience gaps in coverage. Community health centers (CHCs) provide primary care to many children at risk for uninsurance and are well-positioned to help families obtain and retain children’s coverage. Recent advances in health information technology (HIT) capabilities provide the means to create tools that could enhance CHCs’ insurance outreach efforts.
Objective
To present the study design, baseline patient characteristics, variables, and statistical methods for the Innovative Methods for Parents And Clinics to Create Tools for Kids’ Care (IMPACCT Kids’ Care) study.
Methods/design
In this mixed methods study, we will design, test and refine health insurance outreach HIT tools through a user-centered process. We will then implement the tools in four CHCs and evaluate their effectiveness and barriers and facilitators to their implementation. To measure effectiveness, we will quantitatively assess health insurance coverage continuity and utilization of healthcare services for pediatric patients in intervention CHCs compared to matched control sites using electronic health record (EHR) and Oregon Medicaid administrative data over 18 months pre- and 18 months post-implementation (n = 34,867 children). We will also qualitatively assess the implementation process to understand how the tools fit into the clinics’ workflows and the CHC staff experiences with the tools.
Conclusions
This study creates, implements, and evaluates health insurance outreach HIT tools. The use of such tools will likely improve care delivery and health outcomes, reduce healthcare disparities for vulnerable populations, and enhance overall healthcare system performance.
Keywords: Children, Health insurance, Community health centers, Access to healthcare
1. Introduction
Expansions in the US Children’s Health Insurance Program (CHIP) and the Affordable Care Act (ACA) have improved coverage for children (and adults) [1,2]; specifically, CHIP reduced the number of uninsured children from 14% in 1997 to 7% in 2012 [3]. Yet rates of ‘churning’ on and off insurance remain high, creating preventable coverage gaps [4,5]. Health insurance gaps of only a few months are associated with unmet healthcare needs, diminished access to essential care, and poor health outcomes [6–10]. Lack of coverage also negatively impacts the healthcare system: patients’ insurance status impacts clinicians’ ability to provide quality care [11] and patients without insurance have higher emergency department utilization, hospitalization for ambulatory-sensitive conditions, and overall system costs [12,13].
Previous research identified barriers parents face when accessing and maintaining public health insurance for their children, including confusion about eligibility and enrollment requirements, and uncertainty about coverage status [14–16]. As community health centers (CHCs) provide primary care services for many children at risk for uninsurance, they are well-positioned to help families obtain coverage and prevent coverage gaps. While most CHCs assist patients with obtaining health insurance, little is known about the usefulness of health information technology (HIT) tools in facilitating CHC-based health insurance enrollment and re-enrollment efforts. The recent adoption of electronic health records (EHRs) by CHCs and other primary care delivery sites presents an unprecedented opportunity to develop and assess the effectiveness of health insurance outreach HIT tools [17]. Early studies suggest that technological and multi-strategy children’s health insurance outreach approaches show promise [18,19]; however, most current efforts to enroll and retain eligible children in public coverage were implemented outside of the healthcare system and do not utilize EHRs or other HIT tools.
The objective of the Innovative Methods for Parents And Clinics to Create Tools for Kids’ Care (IMPACCT Kids’ Care) study is to design, iteratively test and refine HIT tools that serve this purpose. These ‘IMPACCT tools’ will be implemented in four intervention CHCs and compared to four matched control sites. Using EHR and Oregon Medicaid administrative data, we will assess the IMPACCT tools: 1) quantitatively for effectiveness by comparing children’s insurance coverage continuity and utilization of healthcare services, and 2) qualitatively for barriers and facilitators to implementation. The purpose of this paper is to present the IMPACCT Kids’ Care study design, baseline patient characteristics, variables, and statistical methodology.
2. Materials and methods
2.1. Study overview
The aim of this mixed methods study is to design, test and refine new health insurance outreach HIT tools. We will use observational and modified user-centered design techniques to develop the IMPACCT tools and iteratively test and refine them based on stakeholder feedback. We will then implement the tools in four CHCs and evaluate their effectiveness and barriers and facilitators to their implementation. To measure effectiveness, we will quantitatively assess children’s health insurance coverage continuity and utilization of healthcare services. We will also qualitatively assess the implementation process to understand how the tools fit into the clinics’ workflows and the CHC staff experiences with the tools.
2.2. IMPACCT tool development
To develop the IMPACCT tools, we will start with several EHR tools which have been shown to support clinical decision-making and population health (e.g., panel management reports, best practice alerts) [20–29]. With these tools as a conceptual foundation, we will engage stakeholders through interviews and observation to adapt the tools for use in supporting health insurance enrollment efforts and to identify points in the workflow at which tools could be utilized. The proposed IMPACCT tools will incorporate EHR data already being collected by CHCs, augmented by additional data collection that will be supported by electronic data entry points built through this project. The IMPACCT tools will be finalized in partnership with stakeholders through an iterative process of testing and refining. Methods to design the tools and engage stakeholders are described in detail elsewhere [30,31]. Once sufficiently refined, the tools will be released into production, beta-tested, and then implemented in the intervention clinics. One month after the tools are first implemented (the ‘go-live’ date) in the intervention clinics, they will be assessed for usability and additional modifications will be made, if necessary. The tools will be fully implemented after any additional needed modifications are completed.
2.3. Study setting and data
2.3.1. OCHIN EHR data
OCHIN, Inc. is a 501(3)(c) created to facilitate the implementation of EHRs and other HIT systems in CHCs. OCHIN (previously known as the Oregon Community Health Information Network) now has member clinics in Oregon and 16 other states with >350 primary care clinics and >5000 providers caring for >2,000,000 all-time patients. All member clinics share a single instance of EpicCare© EHR, hosted by OCHIN, and are part of the OCHIN practice-based research network (PBRN) [32]. Since the intervention clinics share an EHR, the tools will be built on the same HIT platform and will be implemented by OCHIN. CHCs in the OCHIN PBRN serve low-income, ethnically diverse populations and have a high percentage of publicly insured and uninsured pediatric patients. OCHIN EHR data from the study clinics will be used to assess coverage continuity, utilization of healthcare services, and use of the IMPACCT tools.
2.3.2. Oregon Medicaid administrative data
The Oregon Medicaid administrative data have a unique identifier and include individuals’ coverage start and end dates. These data will be linked to the OCHIN EHR data using the Medicaid unique identifier and may be used to confirm coverage status of the study population.
2.4. Study population
Four clinics that are members of the OCHIN PBRN volunteered to be the intervention sites. We then selected four matched-control sites from a pool of 38 non-intervention clinics with characteristics similar to the study clinics’. As clinic context may influence children’s coverage needs, insurance outreach efforts, and how services are delivered to children, we used a propensity score matching technique to select the four control sites most closely matched to the four intervention sites. The propensity scores used to match control clinics to intervention clinics were generated from a logistic regression of intervention status on demographic and clinic variables from the four designated intervention clinics and 38 potential control clinics in the OCHIN PBRN. The predictor variables used in the propensity model were: total number of clinic patients, ratio of child to adult patients, less than 20% Hispanic ethnicity, and total months of clinic experience using the EHR. Predicted probability of intervention status from this analysis represents the probability of a clinic being in the intervention group based on these variables. Clinics were matched using nearest neighbor approach: the potential clinic with the closest propensity score was selected as the matched control for each intervention clinic.
For the effectiveness component of the study, we used data from OCHIN’s EHR to identify baseline pediatric patients, encounter counts, and demographics from the eight study clinics (four intervention and four control sites). This baseline data, shown in Table 1, includes all pediatric patients aged 0–20 at first encounter (n = 34,867) and ‘established’ pediatric patients with ≥2 billed encounters at a study clinic between 11/1/2012–10/31/2014 (n = 24,877).
3. Statistical considerations
3.1. Outcome measures
The primary independent variable for the intervention effectiveness analysis is whether or not a clinic was provided the IMPACCT tools. The primary dependent variables are changes in pediatric health insurance coverage status at visits, continuity of coverage at visits over time, and utilization of healthcare services. Specifically, we will use OCHIN EHR data to collect health insurance coverage status at each visit during the study period; insurance continuity will be measured by assessing changes in coverage among visits. For patients with more than one visit, discontinuous coverage will be defined as having different insurance coverage at visits; for example, if a patient has an uninsured visit followed by a Medicaid-insured visit. Utilization of healthcare services will include primary care visit rates and quality of care that can be feasibly measured in EHR data (e.g., well-child and preventive dental visits) [33]. We will compare monthly changes in these measures in the 18 months before versus 18 months after the HIT tools go-live date. All variables in our analyses can be obtained from OCHIN’s EHR data. Insurance status may be confirmed using Oregon Medicaid administrative data (Table 2).
Table 2.
Example variables for analyses.
| Predisposing factors |
| Child age |
| Child gender |
| Child race |
| Child ethnicity |
| Household income |
| Primary language of family |
| Geographic residence |
| Contextual clinic and state level factors |
| Enabling/hindering resources |
| Use of HIT tools |
| Child’s Medicaid/CHIP insurance status/continuity |
| Child’s continuity of care (provider- and clinic-level variables) |
| Need for services |
| Special health care needs [34,35] |
| Frequency of visits |
| Outcomes |
| Child’s insurance status and continuity [1] |
| Utilization of healthcare services [33,36] |
3.2. Effectiveness analysis
To assess the IMPACCT tools’ effectiveness, we will use a clinic-based matched cohort controlled study design comparing four intervention CHCs to four propensity score matched control CHCs. The HIT tools’ effectiveness compared to the control group will be evaluated monthly at the clinic and individual levels by comparing changes in insurance coverage status and continuity, and utilization of healthcare services provided 18 months pre- and 18 months post-go-live date.
We will test the hypothesis that children receiving care in CHCs using the HIT tools will have better insurance coverage rates, fewer gaps in coverage, and increased utilization of recommended healthcare services compared to children in CHCs without such tools. We will also qualitatively assess the implementation process to better understand how the HIT tools fit into the clinics’ workflows and to assess the CHC staff experiences with the tools.
We will conduct longitudinal analyses utilizing a comparative interrupted time series design for clinic level analyses and a difference-in-differences approach for patient-level analyses [37–40]. Initially we will summarize baseline study variables for each study clinic using descriptive statistics and data visualization methods (e.g., histograms, box plots, and scatter plots). We will assess baseline differences between intervention and matched control clinics, and if imbalances in covariate distributions are present, we will include those covariates in our final models to control for residual confounding.
For clinic level analysis, we will use monthly data (e.g., % of children covered by insurance and % of children who utilized healthcare services) and methods that take into account the matching sample to estimate pre- vs. post-implementation change for each clinic. We will use a time-series analysis approach to adjust for serial correlation and potential clinic level confounders and to estimate pre- vs. post-implementation change in the outcome variables, standard errors, and 95% confidence intervals. We will also compare pre- vs. post-changes in intervention versus matched control sites.
For individual level analyses, we will use generalized linear/non-linear mixed models [41]. We will evaluate pre- vs. post-implementation change at the clinic level, and the relative independent contribution to child’s insurance stability and utilization of healthcare services related to which IMPACCT tools were used, and the extent to which these tools were used.
3.3. Power calculations
For clinic level analyses, we should have sufficient power to detect a meaningful difference between study groups, as power estimation for interrupted time series (ITS) models is driven by the number of time points. Here, we will evaluate 36 monthly time points (18 pre and 18 post) which exceed the minimum number needed to do an ITS model [40] and exceed the number used in similar analyses to detect even modest effects [39,42,43]. For patient level analyses, our study population of four intervention and four control sites with a total of 34,867 pediatric patients will provide 80% power at a 5% significance level to detect a 4.5% absolute change in children’s insurance rates before and after the IMPACCT tools go-live date compared to control group (e.g., pre- and post-implementation change of 3% increase in children’s insurance rates in the control sites vs. 7.5% increase in the intervention sites) assuming an intra-cluster correlation coefficient (ICC) of 0.01. Setting ICC = 0.01 provides a conservative estimate as most studies show an ICC <.001 [44]. If we assume an ICC of 0.001, we will have 80% power to detect a 1.6% change.
3.4. Implementation analysis
Data on the implementation process will be collected through site visits and interviews asking clinic staff about their perceptions, acceptance, and use of the IMPACCT tools, and through EHR data tracking tool use. Site visits will be conducted in the intervention CHCs (and possibly in some control CHCs, if outlier sites are identified in quantitative analyses). These site visits will focus on understanding how the IMPACCT tools are being used by practices, and the facilitators and barriers to implementing and using the tools. In addition to observing the use of the IMPACCT tools, we will interview CHC staff to assess acceptance of the tools. We will analyze the qualitative data obtained during site visits to identify themes in how CHC staff users perceived the tools, facilitators and barriers to using the tools, and which aspects they found most helpful. We will investigate how the clinics redesigned workflows to incorporate IMPACCT tool use, whether CHC staff users who reported high use of the tools actually used the tools frequently (as determined by EHR data), and what the barriers to acceptance were and how they were overcome. This data will help identify multilevel factors associated with IMPACCT tool use (Table 3).
Table 3.
Qualitative methods used to analyze implementation.
| Topic | Post-implementation data sources | Measures |
|---|---|---|
User perceptions of the IMPACCT tools
|
CHC site visits; staff interviews | Perceived ease of use, usefulness, social influences [45]. |
Acceptance of the IMPACCT tools
|
CHC site visits; staff interviews | Behavioral intention to use IT tools; satisfaction and acceptance of tools. |
Use of the IMPACCT tools
|
CHC site visits; staff interviews | Observation of tasks, key usage areas, and workflow assessment. Information from users regarding facilitators/barriers to use. |
4. Discussion
The novel health insurance outreach HIT tools we will develop, implement, and test in this study have the potential to significantly improve children’s health insurance coverage rates and increase utilization of recommended healthcare services. Since having health insurance is associated with increased access to healthcare and receipt of recommend services, and decreased preventable hospitalizations and mortality [6–13,46–49], this study allows an opportunity for real-time impacts on improving care delivery and health outcomes and reducing healthcare disparities for vulnerable populations. Additionally, this study could enhance healthcare system performance. When children churn on and off public insurance programs, it increases administrative burden and expense for healthcare systems and public programs [4]; interventions that effectively reduce this problem can achieve efficiencies. Beyond these likely improvements, we will focus our investigation on evaluating the tools’ effectiveness at improving children’s access to health insurance and recommended healthcare services. Such capacities could, in turn, reduce unnecessary, costly utilization of hospital services [12,13,50]. Once built, the IMPACCT tools could be adapted to meet changing practice and policy demands.
The major strengths of this study are our purposively selected sample of four intervention sites and four matched control sites and our use of mixed methods analyses. To minimize bias, we will use propensity scoring to select control sites and conduct the analyses both at the clinic and individual levels. As clinic level analyses are typically performed in cluster-randomized trials, this study is strengthened by the additional individual level analyses, which are typically performed in observational studies in order to account for socio-demographic differences among clinics. For the individual level analysis, we will use flexible regression modeling to accommodate different sources of correlations. Another strength of this study is that we will assess factors arising from healthcare reform in Oregon that may have an impact on our study outcomes including outreach by the state to support the health insurance exchange and Medicaid expansion. As this is a real-time study of the implementation of HIT tools designed to improve insurance coverage rates, any changes in state or national health insurance policies could lead to a need to update our analyses. While this study will take place in selected CHC sites, these sites are representative of OCHIN’s larger network. If proven effective, OCHIN’s linked EHR and other network-wide HIT capabilities will allow for rapid dissemination of the study findings and intervention to more than 500,000 children.
5. Conclusion
This timely study can inform and greatly impact how clinics can utilize HIT tools to conduct health insurance outreach. The use of such tools will likely improve health insurance stability, healthcare delivery and health outcomes, reduce healthcare disparities for vulnerable populations, and enhance overall healthcare system performance.
Table 1.
Baseline pediatric patients (aged 0–20), encounter counts, and demographics by study site.
| Intervention sites
|
Control sites
|
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Clinic 1N (%) | Clinic 2N (%) | Clinic 3N (%) | Clinic 4N (%) | Intervention sites, totalN (%) |
Matched 1N (%) | Matched 2N (%) |
Matched 3N (%) |
Matched 4N(%) |
Control sites, total N (%) |
|
| Total patients, 11/1/2012–10/31/2014 | 6766 | 3659 | 5210 | 3714 | 18,510 | 8162 | 3427 | 2242 | 2674 | 16,357 |
| Total encounters, 11/1/2012–10/31/2014 | 26,947 | 15,362 | 21,899 | 14,118 | 78,326 | 34,465 | 14,208 | 8700 | 7472 | 64,845 |
| Established patientsa | 4819 | 2771 | 3421 | 2514 | 13,525 | 6016 | 2448 | 1474 | 1414 | 11,352 |
| Established patient encounters, 11/1/2012–10/31/2014 | 24,747 | 14,247 | 19,777 | 12,342 | 71,113 | 31,131 | 12,787 | 7579 | 5906 | 57,403 |
| Female sex | 2474 (51.3) | 1430 (51.6) | 1799 (52.6) | 1334 (53.1) | 7037 (52.0) | 3091 (51.4) | 1261 (51.5) | 733 (49.7) | 769 (54.4) | 5854 (51.6) |
| ≥ 138% FPL | 4639 (96.3) | 2632 (95.0) | 3179 (92.9) | 2341 (93.1) | 12,791 (94.6) | 5257 (87.4) | 2240 (91.5) | 1295 (87.9) | 1287 (91.0) | 10,079 (88.8) |
| Age group | ||||||||||
| 0 | 650 (13.5) | 360 (13.0) | 589 (17.2) | 254 (10.1) | 1853 (13.7) | 1099 (18.3) | 342 (14.0) | 233 (15.8) | 178 (12.6) | 1852 (16.3) |
| 1–4 | 1108 (23.0) | 544 (19.6) | 802 (23.4) | 489 (19.5) | 2943 (21.8) | 1552 (25.8) | 556 (22.7) | 337 (22.9) | 254 (18.0) | 2699 (23.8) |
| 5–9 | 1214 (25.2) | 653 (23.6) | 804 (23.5) | 564 (22.4) | 3235 (23.9) | 1422 (23.6) | 601 (24.6) | 282 (19.1) | 258 (18.3) | 2563 (22.6) |
| 10–14 | 951 (19.7) | 668 (24.1) | 569 (16.6) | 586 (23.3) | 2774 (20.5) | 1030 (17.1) | 475 (19.4) | 272 (18.5) | 314 (22.2) | 2091 (18.4) |
| 15–20 | 896 (18.6) | 546 (19.7) | 657 (19.2) | 621 (24.7) | 2720 (20.1) | 913 (15.2) | 474 (19.4) | 350 (23.7) | 410 (29.0) | 2147 (18.9) |
| Race/ethnicity | ||||||||||
| Hispanic | 3113 (64.6) | 2504 (90.4) | 2730 (79.8) | 1235 (49.1) | 9582 (70.9) | 4346 (72.2) | 1248 (51.0) | 477 (32.4) | 696 (49.2) | 6767 (59.6) |
| Non-Hispanic white | 1102 (22.9) | 246 (8.9) | 515 (15.1) | 1144 (45.5) | 3007 (22.2) | 1213 (20.2) | 300 (12.3) | 498 (33.8) | 663 (46.9) | 2674 (23.6) |
| Non-Hispanic other | 440 (9.1) | 18 (0.7) | 104 (3.0) | 78 (3.1) | 640 (4.7) | 444 (7.4) | 842 (34.4) | 466 (31.6) | 49 (3.5) | 1801 (15.9) |
| Unknown | 164 (3.4) | 3 (0.1) | 72 (2.1) | 57 (2.3) | 296 (2.2) | 13 (0.2) | 58 (2.4) | 33 (2.2) | 6 (0.4) | 110 (1.0) |
| Primary language | ||||||||||
| Spanish | 2424 (50.3) | 2210 (79.8) | 2422 (70.85) | 935 (37.2) | 7991 (59.1) | 3593 (59.7) | 1066 (43.6) | 327 (22.2) | 532 (37.6) | 5518 (48.6) |
| English | 1937 (40.2) | 553 (20.0) | 941 (27.5) | 1560 (62.1) | 4991 (36.9) | 2049 (34.1) | 1050 (42.9) | 752 (51.0) | 857 (60.6) | 4708 (41.5) |
| Other | 457 (9.5) | 8 (0.3) | 54 (1.6) | 7 (0.3) | 526 (3.9) | 257 (4.3) | 294 (12.0) | 340 (23.1) | 23 (1.6) | 914 (8.1) |
| Unknown | 1 (0) | 0 (0) | 4 (0.1) | 12 (0.5) | 17 (0.1) | 117 (1.9) | 38 (1.6) | 55 (3.7) | 2 (0.1) | 212 (1.9) |
| Coverage at first encounter | ||||||||||
| Uninsured | 617 (12.8) | 332 (12.0) | 575 (16.8) | 357 (14.2) | 1881 (13.9) | 153 (2.5) | 141 (5.8) | 61 (4.1) | 201 (14.2) | 556 (4.9) |
| Medicaid | 4047 (84.0) | 2354 (85.0) | 2738 (80.0) | 2029 (80.7) | 11,168 (82.6) | 5827 (96.9) | 2273 (92.9) | 1398 (94.8) | 1160 (82.0) | 10,658 (93.9) |
| Private | 149 (3.1) | 68 (2.5) | 97 (2.8) | 101 (4.0) | 415 (3.1) | 26 (0.4) | 29 (1.2) | 11 (0.8) | 32 (2.3) | 98 (0.9) |
| Other | 6 (0.1) | 17 (0.6) | 11 (0.3) | 27 (1.1) | 61 (0.5) | 10 (0.2) | 5 (0.2) | 4 (0.3) | 21 (1.5) | 40 (0.4) |
≥ 2 billed encounters at a study clinic between 11/1/2012–10/31/2014, ages 0–20 at first encounter in a 2 year period.
Acknowledgments
This study was funded by the Patient-Centered Outcomes Research Institute [(PCORI) Health Systems Cycle I (2012)]. The funding source was not involved in the study design, collection, analysis and interpretation of data, or writing of the report, and submitting it for publication.
Footnotes
ClinicalTrials.gov Identifier: NCT02298361
References
- 1.The Henry J. Kaiser Family Foundation; Kaiser Commission on Medicaid the Uninsured, editor. Children’s health coverage: Medicaid, CHIP and the ACA. Washington, D.C: 2014. Issue Brief. [Google Scholar]
- 2.Henry J. Kaiser Family Foundation; Kaiser Commission on Medicaid and the Uninsured, editor. How is the ACA Impacting Medicaid Enrollment? Washington, D.C: 2014. [Google Scholar]
- 3.Henry J. Kaiser Family Foundation; Kaiser Commission on Medicaid the Uninsured, editor. The Impact of the Children’s Health Insurance Program (CHIP): What Does the Research Tell Us? Washington, D.C: 2014. [Google Scholar]
- 4.Fairbrother G, et al. Churning in Medicaid managed care and its effect on accountability. J Health Care Poor Underserved. 2004;15(1):30–41. doi: 10.1353/hpu.2004.0003. [DOI] [PubMed] [Google Scholar]
- 5.Short PF, Graefe DR, Schoen C The Commonwealth Fund, editor. Churn, Churn, Churn: How Instability of Health Insurance Shapes America’s Uninsured Problem. 2003 [PubMed]
- 6.Olson LM, Tang SF, Newacheck PW. Children in the United States with discontinuous health insurance coverage. N Engl J Med. 2005;353(4):382–391. doi: 10.1056/NEJMsa043878. [DOI] [PubMed] [Google Scholar]
- 7.Weissman JS, Epstein AM. The insurance gap: does it make a difference? Annu Rev Public Health. 1993;14:243–270. doi: 10.1146/annurev.pu.14.050193.001331. [DOI] [PubMed] [Google Scholar]
- 8.Sudano JJ, Baker DW. Intermittent lack of health insurance coverage and use of preventive services. Am J Public Health. 2003;93(1):130–137. doi: 10.2105/ajph.93.1.130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.DeVoe JE, et al. “Mind the gap” in children’s health insurance coverage: does the length of a child’s coverage gap matter? Ambul Pediatr. 2008;8(2):129–134. doi: 10.1016/j.ambp.2007.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cassedy A, Fairbrother G, Newacheck PW. The impact of insurance instability on children’s access, utilization, and satisfaction with health care. Ambul Pediatr. 2008;8(5):321–328. doi: 10.1016/j.ambp.2008.04.007. [DOI] [PubMed] [Google Scholar]
- 11.Meyers DS, et al. Primary care physicians’ perceptions of the effect of insurance status on clinical decision making. Ann Fam Med. 2006;4:399–402. doi: 10.1370/afm.574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Peterson TH, et al. Insurance-associated disparities in hospitalization outcomes of Michigan children. J Pediatr. 2011;158(2):313–318. e1–2. doi: 10.1016/j.jpeds.2010.08.002. [DOI] [PubMed] [Google Scholar]
- 13.Bindman AB, et al. Preventable hospitalizations and access to health care. JAMA. 1995;274(4):305–311. [PubMed] [Google Scholar]
- 14.DeVoe JE, Ray M, Graham A. Public health insurance in Oregon: underenrollment of eligible children and parental confusion about children’s enrollment status. Am J Public Health. 2011;101(5):891–898. doi: 10.2105/AJPH.2010.196345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.DeVoe JE, et al. Why do some eligible families forego public insurance for their children? A qualitative analysis. Fam Med. 2012;44(1):39–46. [PMC free article] [PubMed] [Google Scholar]
- 16.Hatch BA, et al. Citizenship documentation requirement for Medicaid eligibility: effects on Oregon children. Fam Med. 2014;46(4):267–275. [PubMed] [Google Scholar]
- 17.DeVoe JE, et al. Health information technology: an untapped resource to help keep patients insured. Ann Fam Med. 2014;12(6):568–572. doi: 10.1370/afm.1721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cousineau MR, Stevens GD, Farias A. Measuring the impact of outreach and enrollment strategies for public health insurance in California. Health Serv Res. 2011;46(1 Pt 2):319–335. doi: 10.1111/j.1475-6773.2010.01202.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Meng Q, et al. Outreach strategies for expanding health insurance coverage in children [Review] Cochrane Database Syst Rev. 2010;8:CD008194. doi: 10.1002/14651858.CD008194.pub2. [DOI] [PubMed] [Google Scholar]
- 20.Sitting DF, et al. Improving clinical quality indicators through electronic health records: it takes more than just a reminder. Pediatrics. 2009;124:375–377. doi: 10.1542/peds.2009-0339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bright TJ, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450. [DOI] [PubMed] [Google Scholar]
- 22.Bates DW, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10:523–530. doi: 10.1197/jamia.M1370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bryan C, Boran SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16:79–91. doi: 10.14236/jhi.v16i2.679. [DOI] [PubMed] [Google Scholar]
- 24.Bates DW, Bitton A. The future of health information technology in the patient-centered medical home. Health Aff (Millwood) 2010;29(4):614–621. doi: 10.1377/hlthaff.2010.0007. [DOI] [PubMed] [Google Scholar]
- 25.Zhou YY, et al. Improving population care with an integrated electronic panel support tool. Popul Health Manag. 2011;14(1):3–9. doi: 10.1089/pop.2010.0001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Dexheimer JW, et al. Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2008;15:311–320. doi: 10.1197/jamia.M2555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Baron RC, et al. Intervention to increase recommendation and delivery of screening for breast, cervical, and colorectal cancers by healthcare providers a systematic review of provider reminders. Am J Prev Med. 2010;38(1):110–117. doi: 10.1016/j.amepre.2009.09.031. [DOI] [PubMed] [Google Scholar]
- 28.Shojania KG, et al. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;3:CD001096. doi: 10.1002/14651858.CD001096.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Zai AH, et al. Lessons from implementing a combined workflow-informatics system for diabetes management. J Am Med Inform Assoc. 2008;15(4):524–533. doi: 10.1197/jamia.M2598. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.DeVoe J, et al. Use of qualitative methods and user-centered design to develop customized health information technology tools within federally qualified health centers to keep children insured. J Ambul Care Manage. 2014;37(2):148–154. doi: 10.1097/JAC.0000000000000016. [DOI] [PubMed] [Google Scholar]
- 31.Likumahuwa-Ackman S, et al. IMPACCT Kids’ Care: a real-world example of stakeholder involvement in comparative effectiveness research. J Comp Eff Res. 2015;4(4):351–357. doi: 10.2217/cer.15.24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.DeVoe JE, et al. Developing a network of community health centers with a common electronic health record: description of the Safety Net West Practice-based Research Network (SNW-PBRN) J Am Board Fam Med. 2011;24(5):597–604. doi: 10.3122/jabfm.2011.05.110052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Gold R, et al. Feasibility of evaluating the CHIPRA care quality measures in electronic health record data. Pediatrics. 2012;130(1):139–149. doi: 10.1542/peds.2011-3705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Neff JM, et al. Identifying and classifying children with chronic conditions using administrative data with the clinical risk group classification system. Ambul Pediatr. 2002;2(1):71–79. doi: 10.1367/1539-4409(2002)002<0071:iaccwc>2.0.co;2. [DOI] [PubMed] [Google Scholar]
- 35.Neff JM, et al. Using medical billing data to evaluate chronically ill children over time. J Ambul Care Manage. 2006;29(4):283–290. doi: 10.1097/00004479-200610000-00004. [DOI] [PubMed] [Google Scholar]
- 36.Agency for Healthcare Research and Quality. Initial Core Set of Children’s Healthcare Quality Measures. 2009 [cited 2009 October 9]; Available from: http//www.ahrq.gov/chip/listtable.htm.
- 37.Bertrand M, Duflo E, Mullainathan S. How much should we trust differences-indifferences estimates? Q J Econ. 2004;119(1):249–275. [Google Scholar]
- 38.Donald S, Lang K. Inference with difference-in-differences and other panel data. Rev Econ Stat. 2007;89(2):221–233. [Google Scholar]
- 39.Kastner M, et al. A knowledge translation tool improved osteoporosis disease management in primary care: an interrupted time series analysis. Implement Sci. 2014;9:109. doi: 10.1186/s13012-014-0109-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr. 2013;13(6):S38–S44. doi: 10.1016/j.acap.2013.08.002. [DOI] [PubMed] [Google Scholar]
- 41.Dean C, Nielsen J. Generalized linear mixed models: a review and some extensions. Lifetime Data Anal. 2007;13(4):497–512. doi: 10.1007/s10985-007-9065-x. [DOI] [PubMed] [Google Scholar]
- 42.Wagner AK, et al. Segmented regression analysis of interrupted time series in medication use research. J Clin Pharm Ther. 2002;27(4):299–309. doi: 10.1046/j.1365-2710.2002.00430.x. [DOI] [PubMed] [Google Scholar]
- 43.Simon SR, et al. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc. 2006;54(6):963–968. doi: 10.1111/j.1532-5415.2006.00734.x. [DOI] [PubMed] [Google Scholar]
- 44.Hayes RJ, et al. Design and analysis issues in cluster-randomized trials of interventions against infectious diseases. Stat Methods Med Res. 2000;9:95–116. doi: 10.1177/096228020000900203. [DOI] [PubMed] [Google Scholar]
- 45.Venkatesh V, Speier C, Morris M. User acceptance enablers in individual decision making about technology: toward an integrated model. Decis Sci. 2002;33(2):297–316. [Google Scholar]
- 46.Hadley J. Sicker and poorer: the consequences of being uninsured. Med Care Res Rev. 2003;60(2 Suppl):3S–75S. doi: 10.1177/1077558703254101. [DOI] [PubMed] [Google Scholar]
- 47.Dorn S Urban Institute, editor. Uninsured and Dying Because of It: Updating the Institute of Medicine Analysis on the Impact of Uninsurance on Mortality. Washington, DC: 2008. [Google Scholar]
- 48.Greene WR, et al. Insurance status is a potent predictor of outcomes in both blunt and penetrating trauma. Am J Surg. 2010;199(4):554–557. doi: 10.1016/j.amjsurg.2009.11.005. [DOI] [PubMed] [Google Scholar]
- 49.Asplin BR, et al. Insurance status and access to urgent ambulatory care follow-up appointments. JAMA. 2005;294(10):1248–1254. doi: 10.1001/jama.294.10.1248. [DOI] [PubMed] [Google Scholar]
- 50.Smulowitz P, et al. Emergency department use by the uninsured after health care reform in Massachusetts. Intern Emerg Med. 2009;4(6):501–506. doi: 10.1007/s11739-009-0313-9. [DOI] [PubMed] [Google Scholar]