Abstract
Objectives:
In 2011, Wisconsin introduced the 2003 Revision of the US Standard Certificate of Live Birth, which includes a variable for principal payer. This variable could help in estimating Medicaid coverage for delivery services, but its accuracy in most states is not known. Our objective was to validate Medicaid payer classification on Wisconsin birth records.
Methods:
We linked 128 141 Wisconsin birth records (2011-2012 calendar years) to 54 600 Medicaid claims. Using claims as the gold standard, we measured the payer variable’s validity (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV]) overall and by maternal age, race/ethnicity, education, facility delivery volume, and the Medicaid proportion of facility delivery volume. Multivariable log-binomial regression tested the association between each characteristic and payer misclassification among Medicaid-paid births.
Results:
Of 128 141 birth records, 50 652 (39.5%) indicated Medicaid as the principal payer and 54 600 (42.6%) linked to a Medicaid claim. The birth record misclassified 10 007 of 54 600 (18.3%) Medicaid-paid births as non-Medicaid and 6059 of 73 541 (8.2%) non-Medicaid births as Medicaid-paid. The payer variable was less sensitive (81.7%) than specific (91.8%), and PPV and NPV were similar (∼88%). Sensitivity was highest among mothers who were Hispanic, had no high school diploma, or delivered in Medicaid-majority delivery facilities. Maternal age ≥40, maternal education >high school, and delivering in a non–Medicaid-majority delivery facility were positively associated with payer misclassification among Medicaid-paid births.
Conclusion:
Differential misclassification of principal payer in the birth record may bias risk surveillance of Medicaid deliveries.
Keywords: maternal and child health, Medicaid/Medicare, perinatal, pregnancy, record linkage
In 2003, the Centers for Disease Control and Prevention’s National Center for Health Statistics (NCHS) adopted a revised birth certificate, which introduced a variable for reporting the source of the payment for delivery services.1 Because birth records register 99% of deliveries in the United States,2 and Medicaid-paid deliveries have greater risks of adverse outcomes than do privately insured deliveries,3 the updated birth record could be a vital and convenient tool for surveilling Medicaid-paid delivery outcomes. Leveraging this benefit requires validating the payer variable.
Although numerous studies have validated data from birth records on maternal characteristics4-7 and clinical risk factors5,7,8 with data from medical claims, only recent literature has expanded these analyses to validate payer information. An analysis of Iowa birth records in calendar years 2007-2009 found that the payer variable was valid overall, but its validity was worst among mothers who were aged <20, were non-white, were Hispanic, or did not complete high school.9 Subsequent research confirmed the variable’s overall validity in New York City and Vermont in 2009, but it did not assess validity in demographic groups.10 Finally, a study of Oregon birth records found moderate but diminishing validity for the payer variable during 2008-2014.11 All 3 studies found that birth records underreported Medicaid coverage of deliveries.9-11 These studies proposed that facility staff training,9,10 the inability to indicate multiple payers in a single birth record,9 and Medicaid policy changes11 may affect the validity of payer information.
No current study has validated the classification of payers in birth records by birth facility characteristics. Large facilities may have a greater capacity than small facilities to support the administrative processes of obstetric care,12,13 and such support could improve the accuracy of birth records. In addition, the composition of payers for services at a birth facility (ie, the proportion of deliveries paid by Medicaid vs other payers, such as private insurance or self-pay) could contribute to underreporting or overreporting Medicaid payment on birth records. Facilities that primarily serve Medicaid beneficiaries may be less likely than facilities that primarily serve non-Medicaid beneficiaries to misclassify Medicaid births.
In 2011, Wisconsin introduced the 2003 Revision of the US Standard Certificate of Live Birth. The objective of our study was to validate Medicaid payer classification on Wisconsin birth records for deliveries during 2011-2012. We validated the measure overall and in subgroups on the basis of maternal and birth facility characteristics.
Methods
We used data from the Big Data for Little Kids project, which merged more than 400 000 birth records of live in-state deliveries among Wisconsin residents (2007-2012) to multiple administrative sources, including paid Medicaid fee-for-service claims and managed care encounters (hereinafter, “claims”) related to prenatal care, delivery, and postnatal care. We matched birth records and Medicaid claims through a Medicaid demographic base file.14 Using deterministic matching, we linked birth records to the base file on mother’s full name (first, middle, and last name) and birth date, and we linked claims to the base file with Medicaid-specific personal identifying numbers. Thirteen Current Procedural Terminology billing codes indicated Medicaid-paid live delivery: 59400, 59409, 59410, 59414, 59510, 59514, 59515, 59610, 59612, 59614, 59618, 59620, and 59622.15
We identified 131 442 birth records for calendar years 2011-2012, Wisconsin’s first 2 years of using the 2003 Revision of the US Standard Certificate of Live Birth, and 57 182 Medicaid delivery claims during the same period, including emergency delivery claims, which facilities can submit within 60 days after delivery.16 Of these claims, before identifying duplicates and applying inclusion criteria, 55 099 (96.4%) linked to birth records. We identified 375 (<0.03%) birth records in which the mother’s name matched to multiple unique beneficiary records on the Medicaid demographic base file, so we eliminated those birth records from our sample. We then excluded 2926 birth records that reported home deliveries (n = 2505) or reported a facility with <10 deliveries in the birth year (n = 421). The exclusion process yielded 128 141 birth records (97.5% of all unique records) in our final sample and 54 600 (42.6%) birth records linked to a Medicaid claim.
Variables
The birth record has 7 principal payer options: Medicaid (or a comparable state program), private insurance, self-pay, CHAMPUS/TRICARE (Civilian Health and Medical Program of the Uniformed Services, now TRICARE), Indian Health Service, other government source (federal, state, or local), or other/unknown source. The NCHS advises that facility staff members use the patient’s face sheet (ie, a form with the patient’s basic demographic, medical, and insurance information) to determine the patient’s insurance information. If a sheet lists multiple payers, the NCHS recommends that the staff member select the payer that will cover most of the delivery costs.17 We created a dichotomous variable for Medicaid payment (ie, record indicates Medicaid as the principal payer or record indicates a non-Medicaid principal payer).
Using Medicaid claims as the gold standard for defining principal payer, we categorized birth records by their reported principal payer: (1) true Medicaid, (2) true non-Medicaid, (3) false Medicaid, and (4) misclassified Medicaid. True Medicaid and true non-Medicaid indicated agreement between birth records and Medicaid claims. False Medicaid indicated that a birth record reported Medicaid as payer but did not link to a paid Medicaid claim. Finally, misclassified Medicaid indicated that the birth record reported non-Medicaid payment but linked to a paid Medicaid claim.
We abstracted data for several maternal characteristics from the birth records: age at delivery (<18, 18-19, 20-24, 25-34, 35-39, ≥40), race/ethnicity (non-Hispanic white, non-Hispanic black, Hispanic, non-Hispanic Asian/Pacific Islander, non-Hispanic Native American, and non-Hispanic other race including multiple), and education at delivery (no high school diploma, high school diploma or equivalent, >high school diploma). Other covariates included plurality (singleton birth or plural birth) and quality of prenatal care, determined by using the Kotelchuck Index (adequate, intermediate, inadequate, none).18 Infant outcomes included birth weight in grams and gestational age in completed weeks, from which we generated variables for low birth weight (<2500 g) and preterm birth (<37 weeks), respectively.
We developed 2 variables for facility characteristics: facility delivery volume and Medicaid proportion of facility delivery volume. We defined facility delivery volume as the number of live deliveries that occurred in the birth record–reported facility in the year that the child was born. We calculated the Medicaid proportion of delivery volume by dividing the facility’s number of birth-year–specific Medicaid-paid births by its birth-year facility delivery volume. We grouped facility delivery volume into 5 categories (10-249 births, 250-499 births, 500-999 births, 1000-2499 births, ≥2500 births), and we grouped the Medicaid proportion of facility delivery volume into 3 categories (10.0%-24.9%, 25.0%-49.9%, ≥50.0%). We defined facilities as “Medicaid-majority” when the Medicaid proportion was ≥50.0%. No facilities had <10% of births that were Medicaid paid.
Statistical Analysis
We tabulated birth record–reported payer by Medicaid payment, and we calculated the distribution of baseline characteristics for all births across 4 categories of payer based on the birth record and linkage to Medicaid claims. Two-sample t tests and Pearson χ2 tests of proportions compared the differences between Medicaid births and non-Medicaid births and between true Medicaid births and misclassified Medicaid births, with P < .05 considered significant.
We estimated validity of Medicaid classification on the birth record against linked claims by calculating its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity is the percentage of Medicaid-paid births that reported Medicaid as the principal payer on the birth records, whereas specificity is the percentage of non-Medicaid–paid births that reported a non-Medicaid principal payer on the birth record. PPV is the percentage of true Medicaid-paid births among all birth records that reported Medicaid as the principal payer, and NPV is the percentage of true non-Medicaid–paid births that reported a non-Medicaid principal payer. We measured validity for the entire sample and in subgroups by maternal age, maternal race/ethnicity, maternal education, facility delivery volume, and Medicaid proportion of facility delivery volume. We computed all point estimates with 95% confidence intervals (CIs). As a sensitivity analysis, we repeated the full validity analysis after removing plural births to account for multiple counting of Medicaid classification for deliveries with plural births.
Last, multivariable log-binomial regression tested factors’ relative risks (RRs) of payer misclassification among Medicaid-paid births.19 We adjusted for all previously listed covariates and clustered standard errors at the level of facility and mother. Our threshold for significance was P < .05.
We conducted analyses with SAS version 9.4,20 and the University of Wisconsin–Madison Institutional Review Board approved our project.
Results
Of 128 141 birth records, 50 652 (39.5%) reported Medicaid as the payer, 54 600 (42.6%) linked to a Medicaid claim, and 112 075 (87.5%) correctly classified Medicaid status (Table 1). Overall, 44 593 (34.8%) birth records were true Medicaid births, 67 482 (52.7%) were true non-Medicaid births, 10 007 (7.8%) were misclassified Medicaid births, and 6059 (4.7%) were false Medicaid births. The prevalence of payer misclassification on the birth record was 18.3% (10 007 of 54 600) among all Medicaid-paid births, 20.0% (2627 of 13 155) among Medicaid-paid births that linked to an individual claim for delivery, and 17.4% (7232 of 41 445) among Medicaid-paid births that linked to an encounter for bundled delivery services. About 88% (44 593 of 50 652) of birth records that indicated Medicaid as principal payer were Medicaid paid, and we found considerable variation in birth record–reported payer based on Medicaid payment status (Table 2). Medicaid claims matched to 42.7% (714 of 1673) of birth records that reported self-payment, 43.7% (76 of 174) of records that reported Indian Health Service payment, 52.3% (264 of 505) of records that reported another government payment source, and 38.3% (1858 of 4845) of records that reported “other” or unknown payment source. Only 10.2% (7071 of 69 664) of records that reported private coverage and 3.8% (24 of 628) of records that reported CHAMPUS/TRICARE coverage linked to Medicaid claims. Of the 54 600 births that were linked to a Medicaid claim, 7071 (13.0%) reported a private insurer. Approximately 5.4% (2936 of 54 600) of Medicaid births and 6.6% (4889 of 73 541) of non-Medicaid births reported self-payment or coverage that was not Medicaid or private insurance.
Table 1.
Medicaid classification cross-tabulation of Wisconsin birth records for live in-state resident deliveries, 2011-2012a
| Principal Payment Reported on Birth Record | No. (%) | ||
|---|---|---|---|
| Medicaid Claim Linked | Medicaid Claim Not Linked | Medicaid Claim Linked + Medicaid Claim Not Linked | |
| Medicaid | 44 593 (34.8)b | 6059 (4.7)c | 50 652 (39.5) |
| Non-Medicaid | 10 007 (7.8)d | 67 482 (52.7)e | 77 489 (60.5) |
| Total | 54 600 (42.6) | 73 541 (57.4) | 128 141 (100.0) |
aData come from Big Data for Little Kids, a data system that merged Wisconsin birth records for live in-state deliveries among resident mothers (2007-2012) to multiple administrative sources, including paid Medicaid claims.14 Birth records reported Medicaid payment or 1 of 6 non-Medicaid payers. Birth records could not report multiple payers.
bTrue Medicaid.
cFalse Medicaid.
dMisclassified Medicaid.
eTrue non-Medicaid.
Table 2.
Medicaid coverage of deliveries in facilities, by primary payer indicated on birth record (n = 128 141), for live in-state resident deliveries in Wisconsin, 2011-2012a
| Payer Reported on Birth Record | Full Sample of Birth Records, No. | Birth Record Linked to a Medicaid Claim, No. (%b) | Birth Record Not Linked to a Medicaid Claim, No. (%b) |
|---|---|---|---|
| Medicaid | 50 652 | 44 593 (88.0) | 6059 (12.0) |
| Private insurance | 69 664 | 7071 (10.2) | 62 593 (89.8) |
| Self-pay | 1673 | 714 (42.7) | 959 (57.3) |
| Indian Health Service | 174 | 76 (43.7) | 98 (56.3) |
| CHAMPUS/TRICARE | 628 | 24 (3.8) | 604 (96.2) |
| Other government | 505 | 264 (52.3) | 241 (47.7) |
| Other/unknown | 4845 | 1858 (38.3) | 2987 (61.7) |
| Total | 128 141 | 54 600 (42.6) | 73 541 (57.4) |
Abbreviations: CHAMPUS, Civilian Health and Medicaid Program of the Uniformed Services (now known as TRICARE).
aData source: Big Data for Little Kids, a data system that merged Wisconsin birth records for live in-state deliveries among resident mothers (2007-2012) to multiple administrative sources, including paid Medicaid claims.14 Birth records reported Medicaid payment or 1 of 6 non-Medicaid payers. Birth records could not report multiple payers.
bPercentages calculated as row percentage.
Deliveries differed significantly across all characteristics by Medicaid payment status (Table 3). Medicaid births tended to occur among mothers who were younger, were non-Hispanic white, did not have a high school diploma, or were unmarried. Moreover, preterm birth, low birth weight, and lower Kotelchuck Index scores were more common among Medicaid births than non-Medicaid births. Approximately 54% of Medicaid-paid deliveries occurred in facilities with ≥1000 deliveries annually, and Medicaid-paid deliveries were more likely than non-Medicaid deliveries to occur in Medicaid-majority facilities. Relative to women with correctly classified Medicaid-paid births, women with misclassified Medicaid births were more likely to be non-Hispanic white, be older, be more educated, be married, have adequate or no prenatal care, or deliver low-birth-weight or preterm infants.
Table 3.
Characteristics of mothers, infants, and birth facilities as recorded in Wisconsin birth records for live in-state resident deliveries overall, by 4 categories of Medicaid classification, 2011-2012a
| Characteristicsb | Total Sample (N = 128 141) | All Births | All Medicaid-Paid Births | ||
|---|---|---|---|---|---|
| Non–Medicaid-Paid Birthsc (n = 73 541) | All Medicaid-Paid Birthsd (n = 54 600) | True Medicaid Birthse (n = 44 593) | Misclassified Medicaid Birthsf (n = 10 007) | ||
| Maternal age, mean (SD), y | 28.0 (5.6) | 29.7 (5.0) | 25.7 (5.5) | 25.6 (5.5) | 26.2 (5.6) |
| Maternal age, y | |||||
| <18 | 2368 (1.8) | 631 (0.9) | 1737 (3.2) | 1454 (3.3) | 283 (2.8) |
| 18-19 | 6170 (4.8) | 1431 (2.0) | 4739 (8.7) | 3971 (8.9) | 768 (7.7) |
| 20-24 | 27 206 (21.2) | 8342 (11.3) | 18 864 (34.6) | 15 712 (35.2) | 3152 (31.5) |
| 25-34 | 75 994 (59.3) | 50 865 (69.2) | 25 129 (46.0) | 20 149 (45.2) | 4980 (49.8) |
| 35-39 | 13 518 (10.6) | 10 183 (13.8) | 3335 (6.1) | 2684 (6.0) | 651 (6.5) |
| ≥40 | 2885 (2.2) | 2089 (2.8) | 796 (1.5) | 623 (1.4) | 173 (1.7) |
| Maternal race/ethnicity | |||||
| Non-Hispanic white | 92 626 (72.3) | 61 782 (84.0) | 30 844 (56.5) | 24 244 (54.4) | 6660 (66.0) |
| Non-Hispanic black | 12 321 (9.6) | 2593 (3.5) | 9728 (17.8) | 8273 (18.6) | 1455 (14.5) |
| Hispanic | 12 969 (10.1) | 4331 (5.9) | 8638 (15.8) | 7719 (17.3) | 919 (9.2) |
| Asian/Pacific Islanderg | 5939 (4.6) | 3341 (4.5) | 2598 (4.8) | 2056 (4.6) | 542 (5.4) |
| Native Americang | 1449 (1.1) | 397 (0.5) | 1052 (1.9) | 893 (2.0) | 159 (1.6) |
| Non-Hispanic otherh | 2771 (2.2) | 1062 (1.4) | 1709 (3.1) | 1385 (3.1) | 324 (3.2) |
| Missing data | 66 (0.1) | 35 (0.1) | 31 (0.1) | 23 (0.1) | 8 (0.1) |
| Maternal education at delivery | |||||
| No high school diploma | 15 420 (12.0) | 3508 (4.8) | 11 912 (21.8) | 10 554 (23.7) | 1358 (13.6) |
| High school diploma or equivalent | 32 512 (25.4) | 10 933 (14.9) | 21 579 (39.5) | 18 147 (40.7) | 3432 (34.3) |
| >High school diploma | 79 771 (62.2) | 58 913 (80.1) | 20 858 (38.2) | 15 716 (35.2) | 5142 (51.4) |
| Missing data | 438 (0.3) | 187 (0.2) | 251 (0.5) | 176 (0.4) | 75 (0.8) |
| Maternal marital status | |||||
| Married | 79 406 (62.0) | 60 896 (82.8) | 18 510 (33.9) | 14 165 (31.8) | 4345 (43.4) |
| Unmarried | 48 640 (38.0) | 12 550 (17.1) | 36 090 (66.1) | 30 428 (68.2) | 5662 (56.6) |
| Missing data | 95 (0.1) | 95 (0.1) | 0 | 0 | 0 |
| Kotelchuck Indexi | |||||
| Adequate | 106 303 (83.0) | 64 860 (88.2) | 41 443 (75.9) | 33 545 (75.2) | 7898 (78.9) |
| Intermediate | 3915 (3.1) | 1692 (2.3) | 2223 (4.1) | 1962 (4.4) | 261 (2.6) |
| Inadequate | 12 466 (9.7) | 4227 (5.8) | 8239 (15.1) | 7029 (15.8) | 1210 (12.1) |
| None | 703 (0.6) | 331 (0.4) | 372 (0.7) | 255 (0.6) | 117 (1.2) |
| Missing data | 4754 (3.7) | 2431 (3.3) | 2323 (4.2) | 1802 (4.0) | 521 (5.2) |
| Plurality | |||||
| Singleton birth | 123 739 (96.6) | 70 601 (96.0) | 53 138 (97.3) | 43 455 (97.4) | 9683 (96.8) |
| Plural birth | 4402 (3.4) | 2940 (4.0) | 1462 (2.7) | 1138 (2.6) | 324 (3.2) |
| Birth weight, mean (SD), g | 3319.3 (591.5) | 3366.7 (586.2) | 3255.5 (592.5) | 3253.3 (585.3) | 3264.8 (623.5) |
| Low birth weight (<2500 g) | |||||
| Yes | 9915 (7.2) | 4693 (6.4) | 4502 (8.2) | 3621 (8.1) | 881 (8.8) |
| No | 118 921 (92.8) | 68 832 (93.6) | 50 089 (91.7) | 40 966 (91.9) | 9123 (91.2) |
| Missing data | 25 (0.02) | 16 (0.02) | 9 (0.02) | 6 (0.01) | 3 (0.03) |
| Gestational age, mean (SD), wk | 38.5 (2.1) | 38.6 (2.1) | 38.5 (2.1) | 38.5 (2.1) | 38.4 (2.3) |
| Preterm birth (gestation <37 weeks) | |||||
| Yes | 11 964 (9.3) | 6610 (9.0) | 5354 (9.8) | 4299 (9.6) | 1055 (10.5) |
| No | 116 030 (90.6) | 66 858 (90.9) | 49 172 (90.1) | 40 240 (90.2) | 8932 (89.3) |
| Missing data | 147 (0.1) | 73 (0.1) | 74 (0.1) | 54 (0.1) | 20 (0.2) |
| Facility delivery volume, no. of birthsj | |||||
| 10-249 | 8993 (7.0) | 4337 (5.9) | 4656 (8.5) | 3951 (8.9) | 705 (7.0) |
| 250-499 | 17 535 (13.7) | 9714 (13.2) | 7821 (14.3) | 6431 (14.4) | 1390 (13.9) |
| 500-999 | 28 825 (22.3) | 16 547 (22.5) | 12 278 (22.5) | 10 148 (22.8) | 130 (21.3) |
| 1000-2499 | 36 701 (28.6) | 21 272 (28.9) | 15 429 (28.3) | 11 754 (26.4) | 3675 (36.7) |
| ≥2500 | 36 087 (28.2) | 21 671 (29.5) | 14 416 (26.4) | 12 309 (27.6) | 2107 (21.1) |
| Medicaid proportion of facility delivery volumek | |||||
| 10.0%-24.9% | 16 264 (12.7) | 12 761 (17.4) | 3503 (6.4) | 2856 (6.4) | 647 (6.5) |
| 25.0%-49.9% | 70 729 (55.2) | 44 626 (60.7) | 26 103 (47.8) | 19 464 (43.6) | 6639 (66.3) |
| ≥50.0% | 41 148 (32.1) | 16 154 (22.0) | 24 994 (45.8) | 22 273 (50.0) | 2721 (27.2) |
aData source: Big Data for Little Kids, a data system that merged Wisconsin birth records for live in-state deliveries among resident mothers (2007-2012) to multiple administrative sources, including paid Medicaid claims.14 Birth records reported Medicaid payment or 1 of 6 non-Medicaid payers. Birth records could not report multiple payers. All values are number (percentage) unless otherwise indicated.
bContinuous variables compared by using 2-sample t test and categorical variables compared by using Pearson χ2 test of proportions. The difference between all Medicaid and all non-Medicaid was significant at P < .05 for all categories. The difference between true Medicaid and misclassified Medicaid was significant at P < .05 for all categories, except birth weight.
cBirth record did not link to a paid Medicaid claim for live delivery.
dBirth record linked to a paid Medicaid claim for live delivery.
eBirth record reported Medicaid payment on the delivery payer variable and linked to a Medicaid claim for live delivery.
fBirth record reported non-Medicaid payment but linked to a paid Medicaid claim.
gNon-Hispanic.
hThe category was specified as “non-Hispanic other race including multiple.”
iThe Kotelchuck Index is a measure of prenatal care use that depends on the date of prenatal care initiation and the frequency of prenatal care visits during the pregnancy.18
jThe number of live deliveries that occurred in the birth record–reported facility during the year that the child was born.
kCalculated by dividing the facility’s number of birth-year–specific Medicaid-paid births by its birth-year facility delivery volume.
The Medicaid payment indicator’s sensitivity (81.7%; 95% CI, 81.4%-82.0%) was lower than its specificity (91.8%; 95% CI, 91.6%-92.0%) in the total sample (Table 4). Across age categories, sensitivity was somewhat consistent (range, 78.3%-83.8%), although specificity was worse among younger mothers. For example, specificity was <70% among mothers aged <20, but it was >90% among mothers aged ≥25. We found some differences in sensitivity and specificity among racial/ethnic groups. Sensitivity (<80%) was lower than specificity (>89%) among non-Hispanic white and non-Hispanic Asian/Pacific Islander mothers. In contrast, sensitivity (>84%) was higher than specificity (<72%) among mothers who were non-Hispanic black, non-Hispanic Native American, or Hispanic. Sensitivity was lowest (78.6%) among non-Hispanic white mothers and highest among Hispanic mothers (89.4%), and specificity was highest among non-Hispanic white mothers (95.6%) and lowest among Hispanic mothers (61.1%). As education levels increased, sensitivity decreased (range, 75.4%-88.6%) but specificity increased (range, 49.1%-96.6%). Sensitivity was similar across categories of facility delivery volume (range, 82.2%-85.4%) except for facilities with 1000-2499 births per year (76.2%), and specificity was consistently high across delivery volume categories (range, 89.6%-93.0%). Sensitivity and specificity varied by Medicaid proportion of facility delivery volume. Among Medicaid-majority facilities, sensitivity was 89.1%, and specificity was 82.1%. Among facilities in which <50.0% of deliveries were Medicaid paid, specificity was higher than sensitivity, particularly for facilities with 25.0%-49.9% of deliveries paid by Medicaid (74.6% sensitivity and 93.9% specificity).
Table 4.
Validity of Medicaid payer classification on Wisconsin birth records for live in-state resident deliveries when compared with Medicaid claims, 2011-2012a
| Characteristic | Sensitivity,b % (95% CI) | Specificity,c % (95% CI) | Positive Predictive Value,d % (95% CI) | Negative Predictive Value,e % (95% CI) |
|---|---|---|---|---|
| Overall | 81.7 (81.4-82.0) | 91.8 (91.6-92.0) | 88.0 (87.8-88.3) | 87.1 (86.9-87.3) |
| Maternal age, y | ||||
| <18 | 83.7 (82.0-85.4) | 66.6 (62.9-70.2) | 87.3 (85.7-88.9) | 59.7 (56.1-63.4) |
| 18-19 | 83.8 (82.7-84.8) | 59.3 (56.8-61.9) | 87.2 (86.3-88.2) | 52.5 (50.1-54.9) |
| 20-24 | 83.3 (82.8-83.8) | 76.8 (75.9-77.7) | 89.0 (88.6-89.5) | 67.0 (66.1-68.0) |
| 25-34 | 80.2 (79.7-80.7) | 94.7 (94.5-94.9) | 88.1 (87.7-88.6) | 90.6 (90.4-90.9) |
| 35-39 | 80.5 (79.1-81.8) | 95.2 (94.8-95.6) | 84.6 (83.3-85.9) | 93.7 (93.2-94.2) |
| ≥40 | 78.3 (75.4-81.1) | 93.9 (92.9-95.0) | 83.1 (80.4-85.8) | 91.9 (90.7-93.1) |
| Maternal race/ethnicity | ||||
| Non-Hispanic white | 78.6 (78.1-79.1) | 95.6 (95.4-95.7) | 89.9 (89.5-90.2) | 90.0 (89.7-90.2) |
| Non-Hispanic black | 85.0 (84.3-85.8) | 62.5 (60.6-64.3) | 89.5 (88.9-90.1) | 52.7 (50.9-54.5) |
| Hispanic | 89.4 (88.7-90.0) | 61.1 (59.7-62.6) | 82.1 (81.3-82.9) | 74.2 (72.8-75.7) |
| Asian/Pacific Islanderf | 79.1 (77.6-80.7) | 89.5 (88.5-90.5) | 85.4 (84.0-86.8) | 84.7 (83.5-85.8) |
| Native Americanf | 84.9 (82.7-87.1) | 71.0 (66.6-75.5) | 88.6 (86.6-90.6) | 64.0 (59.5-68.4) |
| Non-Hispanic otherg | 81.0 (79.2-82.9) | 81.9 (79.6-84.2) | 87.8 (86.2-89.4) | 72.9 (70.3-75.4) |
| Maternal education at delivery | ||||
| No high school diploma | 88.6 (88.0-89.2) | 49.1 (47.4-50.7) | 85.5 (84.9-86.2) | 55.9 (54.2-57.7) |
| High school diploma or equivalent | 84.1 (83.6-84.6) | 79.6 (78.8-80.4) | 89.1 (88.6-89.5) | 71.7 (70.9-72.5) |
| >High school diploma | 75.4 (74.8-75.9) | 96.6 (96.5-96.7) | 88.7 (88.2-89.1) | 91.7 (91.5-91.9) |
| Facility delivery volumeh | ||||
| 10-249 | 84.9 (83.8-85.9) | 89.6 (88.7-90.5) | 89.7 (88.8-90.6) | 84.6 (83.6-85.7) |
| 250-499 | 82.2 (81.4-83.1) | 93.0 (92.5-93.5) | 90.4 (89.8-91.2) | 86.7 (86.0-87.3) |
| 500-999 | 82.7 (82.0-83.3) | 92.5 (92.1-92.9) | 89.1 (88.6-89.7) | 87.8 (87.3-88.3) |
| 1000-2499 | 76.2 (75.5-76.9) | 92.6 (92.2-92.9) | 88.1 (87.6-88.7) | 84.3 (83.8-84.7) |
| ≥2500 | 85.4 (84.8-86.0) | 90.3 (89.9-90.7) | 85.4 (84.8-86.0) | 90.3 (89.9-90.7) |
| Medicaid proportion of facility delivery volumei | ||||
| 10.0%-24.9% | 81.5 (80.3-82.8) | 96.5 (96.2-96.8) | 86.5 (85.3-87.7) | 95.0 (94.6-95.4) |
| 25.0%-49.9% | 74.6 (74.0-75.1) | 93.9 (93.7-94.1) | 87.7 (87.3-88.2) | 86.3 (86.0-86.6) |
| ≥50.0% | 89.1 (88.7-89.5) | 82.1 (81.5-82.7) | 88.5 (88.1-88.9) | 83.0 (82.4-83.6) |
aData source: Big Data for Little Kids, a data system that merged Wisconsin birth records for live in-state deliveries among resident mothers (2007-2012) to multiple administrative sources, including paid Medicaid claims.14 Birth records reported Medicaid payment or 1 of 6 non-Medicaid payers. Birth records could not report multiple payers.
bPercentage of Medicaid-paid births that reported Medicaid as the principal payer on the birth records.
cPercentage of non-Medicaid–paid births that reported a non-Medicaid principal payer on the birth record.
dPercentage of true Medicaid-paid births among all birth records that reported Medicaid as the principal payer.
ePercentage of true non-Medicaid–paid births that reported a non-Medicaid principal payer.
fNon-Hispanic.
gThe category was specified as “non-Hispanic other race including multiple.”
hThe number of live deliveries that occurred in the birth record–reported facility during the year that the child was born.
iCalculated by dividing the facility’s number of birth-year–specific Medicaid-paid births by its birth-year facility delivery volume. There were no facilities in which <10% of deliveries were paid by Medicaid.
The indicator’s overall PPV (88.0%; 95% CI, 87.8%-88.3%) and NPV (87.1%; 95% CI, 86.9%-87.3%) were similar. PPV was consistent across all maternal characteristics, ranging from 82.1% among Hispanic mothers to 89.9% among non-Hispanic white mothers. However, NPV varied across most demographic groups. NPV was low among mothers aged <20 (<60.0%) and 20-24 (67.0%), but it was >90% among mothers aged ≥25. The greatest NPVs among racial/ethnic groups were among non-Hispanic white (90.9%) and non-Hispanic Asian/Pacific Islander (84.7%) mothers, and the lowest NPV was among non-Hispanic black mothers (52.7%). NPV was 91.7% among mothers who had more than a high school diploma, and it was ≤71.7% for less educated mothers. PPV and NPV were 83.0%-90.3% across groups of facility delivery volume and Medicaid proportion of delivery volume, although NPV was 95.0% for facilities with 25.0%-49.9% of deliveries paid by Medicaid.
Our sensitivity analysis recalculated all validity measures after excluding plural births. Results were nearly identical, so multiple Medicaid classification counts from deliveries with plural births did not affect the validity of payer classification.
Among Medicaid-paid births, regression estimates showed that certain maternal, obstetric, and facility characteristics were associated with the risk of misclassifying the principal payer on the birth record (Figure). Having no prenatal care relative to adequate care according to the Kotelchuck Index was the strongest maternal risk factor (RR = 1.18; 95% CI, 1.12-1.24). Other significant maternal characteristics that were positively associated with misclassification included being aged <18 or ≥40 (reference, age 25-34) and having >high school diploma (reference, no high school diploma). In contrast, being non-Hispanic black (reference, non-Hispanic white), having a high school diploma (reference, no high school diploma), and having intermediate or inadequate prenatal care (reference, adequate prenatal care) were associated with slightly lower risks of payer misclassification. Delivering in a facility with an annual delivery volume of 250-499 births, 500-999 births, or 1000-2499 births was associated with a 2%-5% increased risk of payer misclassification (reference, ≥2500 births), and delivering in a facility with a non-majority proportion of Medicaid births was associated with a 13% increased risk of misclassification (reference, ≥50.0% Medicaid births). Three sensitivity models—one without low birth weight, one without preterm birth, and one without the Kotelchuck Index—tested robustness to correlated birth outcomes or bias from missing data. All results were consistent with the results of the main model.
Figure.
Relative risks and 95% confidence intervals (CIs) of payer misclassification on the birth record among Medicaid-paid deliveries in Wisconsin, 2011-2012 (n = 54 600). Data come from Big Data for Little Kids, a data system that merged Wisconsin birth records for live in-state deliveries among resident mothers (2007-2012) to multiple administrative sources, including paid Medicaid claims.14 Birth records reported only 1 of 7 payer options, including Medicaid payment. A log-binomial regression generated relative risks and 95% CIs, and the regression adjusted for all listed covariates and clustered standard errors at the maternal and facility levels. The outcome was reported non-Medicaid payment on the birth record. There were no birth records from facilities in which <10% of deliveries were paid for by Medicaid.
Discussion
During Wisconsin’s first 2 years of using the 2003 Revision of the US Standard Certificate of Live Birth, the Medicaid payment indicator demonstrated substantial validity relative to Medicaid claims. Specificity and NPV were worst among mothers who were aged <20, were Hispanic or non-Hispanic black, or had no high school diploma. In contrast, sensitivity was worst among mothers who were non-Hispanic white or had education beyond high school. We also identified facility-level factors associated with payer misclassification among Medicaid births.
Our findings generally confirmed the results of studies in other states. Overall accuracy of Medicaid classification on Wisconsin birth records was consistent with those reported in Iowa,9 Oregon,11 Vermont, and New York City.10 In addition, maternal demographic variation in the payer variable’s validity was similar to variation observed in Iowa9 and Oregon.11 However, Medicaid classification sensitivity was lower among Hispanic mothers than among non-Hispanic white mothers in Iowa,9 in contrast to our findings.
Medicaid proportion of facility delivery volume may be a stronger predictor than facility delivery volume of birth record Medicaid misclassification. Medicaid-majority facilities were less likely than Medicaid-minority facilities to underreport Medicaid payment and may have the institutional knowledge and resources to efficiently identify and enroll eligible Medicaid beneficiaries. For example, federally qualified hospitals that serve a disproportionately large portion of a state’s Medicaid population may have staff members who can enroll eligible women during their prenatal visits or delivery admissions.21 Standardized on-site Medicaid enrollment means that staff members can detect and enroll eligible women quickly, and their birth records can accurately reflect Medicaid payment. Facilities that primarily serve non-Medicaid patients may lack such services, and Medicaid reimbursement for eligible but unenrolled women could occur after the hospital finalizes the birth record.
Three reasons may explain the misclassification of principal payer in birth records. First, as Kane and Sappenfield9 suggest, misclassification may result from hospital staff members being permitted to select only one payer even if multiple sources cover the delivery. The NCHS birth record guide instructs staff members to choose the expected primary payer if multiple payers are named on the hospital or admitting office face sheet.17 This problem is particularly relevant for mothers with a primary government insurer and supplemental Medicaid coverage. Indeed, 33% of Medicaid beneficiaries and 60% of military health care (such as CHAMPUS/TRICARE) beneficiaries had multiple coverage types in 2015.22 Second, birth records may misclassify Medicaid-eligible mothers who were not enrolled during the pregnancy but for whom hospitals later receive Medicaid payment for labor and delivery. Third, birth records may not capture information on emergency Medicaid coverage for delivery. Emergency Medicaid services will pay for eligible nonbeneficiary deliveries up to 60 days after the delivery date,16 but no mechanism for birth records exists to update payment information after submission to the state.
Our study has research and public health implications. We found differential misclassification of Medicaid payment by maternal characteristics, which can bias risk estimates of adverse birth outcomes.23 Misclassified Medicaid births had characteristics associated with lower-risk deliveries, such as higher levels of maternal education or better prenatal care quality. Misclassifying these potentially healthier deliveries as non-Medicaid could artificially strengthen associations between Medicaid coverage and adverse outcomes if one uses birth records to determine delivery coverage. Researchers who use Medicaid payment as a risk factor in analyzing birth records should consider this possible differential classification bias. Likewise, policies and interventions for improving maternal and child health rely on active birth record surveillance. Stakeholders may use these data to identify pregnant Medicaid beneficiaries and tailor public health efforts for that population. If data are misclassified and at-risk populations are overlooked, resulting interventions may not target these populations despite their potential need.
Birth records provide a relatively inexpensive source of delivery payment information to investigators, so there is a strong incentive to improve their validity. Health services may provide a solution. Recently, state-based perinatal quality collaborative organizations developed multi-institutional analytic networks to identify and enhance processes in obstetric care, including the process of completing the birth record accurately.24 Existing quality improvement efforts can modify the process of birth record completion to better verify payer classification. Such initiatives can increase the cost-efficiency of estimating Medicaid delivery coverage and conducting research on Medicaid deliveries without the resource-intensive process of linking birth records to claims data.
Limitations
This study had some limitations. We analyzed birth records of live in-state deliveries among resident mothers only, so our findings do not apply to residents who delivered outside Wisconsin. In addition, our results may have limited generalizability. Compared with other states, Wisconsin has considerable racial/ethnic segregation and geographic variation in health care infrastructure,25 substantial racial/ethnic disparities in prenatal and infant health,26 and a low teenage birth rate.27 Medicaid misclassification varied by maternal and facility characteristics in Wisconsin, so states with different demographic characteristics or health care infrastructure may have similarly different validity of Medicaid classification on birth records. Demographic patterns in payer misclassification are consistent with past research,9,11 but the literature is nonetheless sparse, and no contemporary study validated the payer classification variable by facility characteristics. Verifying these patterns on a national scale necessitates birth record validation in other states.
Conclusion
Most Wisconsin birth records correctly classified Medicaid payment (87.5%), and the validity of Medicaid payer classification on Wisconsin birth records was similar to validity in other states, but misclassification differed by demographic and facility characteristics. Researchers who use birth records for delivery surveillance, particularly for Medicaid-paid births, should consider the possibility and consequences of differential payer misclassification. Quality improvement efforts in the collection of birth facility data may help enhance the validity of payer information on birth records.
Acknowledgments
The authors of this article are solely responsible for the content therein. The authors thank the Wisconsin Department of Children and Families and Department of Health Services for the use of data for this analysis, but these agencies do not certify the accuracy of the analyses presented.
Footnotes
Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support was provided by the University of Wisconsin (UW)–Madison Clinical and Translational Science Award program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences, grant UL1TR000427; the UW School of Medicine and Public Health’s Wisconsin Partnership Program (WPP); and the UW–Madison Institute for Research on Poverty (IRP). The content of this article does not necessarily represent the official views of NIH, WPP, or IRP.
ORCID iD: David C. Mallinson, MS 
https://orcid.org/0000-0003-1069-6040
Deborah B. Ehrenthal, MD, MPH
https://orcid.org/0000-0002-2441-0712
References
- 1. Centers for Disease Control and Prevention. Revisions of the US Standard Certificates and Reports. Updated August 30, 2017 https://www.cdc.gov/nchs/nvss/revisions-of-the-us-standard-certificates-and-reports.htm. Accessed January 11, 2018.
- 2. Martin JA, Hamilton BE, Osterman MJ, Curtin SC, Mathews TJ. Births: final data for 2013. Natl Vital Stat Rep. 2015;64(1):1–65. [PubMed] [Google Scholar]
- 3. D’Angelo D, Williams L, Morrow B, et al. Preconception and interconception health status of women who recently gave birth to a live-born infant—Pregnancy Risk Assessment Monitoring System (PRAMS), United States, 26 reporting areas, 2004 MMWR Surveill Summ. 2007;56(10):1–35. [PubMed] [Google Scholar]
- 4. DiGiuseppe DL, Aron DC, Ranbom L, Harper DL, Rosenthal GE. Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern Child Health J. 2002;6(3):169–179. doi:10.1023/A:1019726112597 [DOI] [PubMed] [Google Scholar]
- 5. Northam S, Knapp TR. The reliability and validity of birth certificates. J Obstet Gynecol Neonatal Nurs. 2006;35(1):3–12. doi:10.1111/j.1552-6909.2006.00016.x [DOI] [PubMed] [Google Scholar]
- 6. Vinikoor LC, Messer LC, Laraia BA, Kaufman JS. Reliability of variables on the North Carolina birth certificate: a comparison with directly queried values from a cohort study. Paediatr Perinat Epidemiol. 2010;24(1):102–112. doi:10.1111/j.1365-3016.2009.01087.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Zollinger TW, Przybylski MJ, Gamache RE. Reliability of Indiana birth certificate data compared to medical records. Ann Epidemiol. 2006;16(1):1–10. doi:10.1016/j.annepidem.2005.03.005 [DOI] [PubMed] [Google Scholar]
- 8. DiGiuseppe DL, Aron DC, Payne SM, Snow RJ, Dierker L, Rosenthal GE. Risk adjusting cesarean delivery rates: a comparison of hospital profiles based on medical record and birth certificate data. Health Serv Res. 2001;36(5):959–977. [PMC free article] [PubMed] [Google Scholar]
- 9. Kane DJ, Sappenfield WM. Ascertainment of Medicaid payment for delivery on the Iowa birth certificate: is accuracy sufficient for timely policy and program relevant analysis? Matern Child Health J. 2014;18(4):970–977. doi:10.1007/s10995-013-1325-7 [DOI] [PubMed] [Google Scholar]
- 10. Dietz P, Bombard J, Mulready-Ward C, et al. Validation of selected items on the 2003 U.S. Standard Certificate of Live Birth: New York City and Vermont. Public Health Rep. 2015;130(1):60–70. doi:10.1177/003335491513000108 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Evans MA, Siu V, Markwardt K, Hargand S. Oregon Medicaid paid births: validity and reliability of birth certificate reported payer and Medicaid claims data 2008-2014. Am J Public Health Res. 2017;5(2):36–42. doi:10.12691/AJPHR-5-2-2 [Google Scholar]
- 12. D’Alton ME, Bonanno CA, Berkowitz RL, et al. Putting the “m” back in maternal–fetal medicine. Am J Obstet Gynecol. 2013;208(6):442–448. doi:10.1016/J.AJOG.2012.11.041 [DOI] [PubMed] [Google Scholar]
- 13. Friedman AM, Ananth CV, Huang Y, D’Alton ME, Wright JD. Hospital delivery volume, severe obstetrical morbidity, and failure to rescue. Am J Obstet Gynecol. 2016;215(6):795.e1-795.e14. doi:10.1016/j.ajog.2016.07.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Larson A, Berger LM, Mallinson DC, Grodsky E, Ehrenthal DB. Variable uptake of Medicaid-covered Prenatal Care Coordination: the relevance of treatment level and service context. J Community Health. 2019;44(1):32–43. doi:10.1007/s10900-018-0550-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. American Medical Association. CPT (Current Procedural Terminology). https://www.ama-assn.org/amaone/cpt-current-procedural-terminology. Accessed May 16, 2019.
- 16. Wisconsin Department of Health Services. Wisconsin Medicaid and BadgerCare Plus—emergency services. Updated February 26, 2015 https://www.dhs.wisconsin.gov/forwardhealth/p-10072.htm. Accessed August 16, 2018.
- 17. Centers for Disease Control and Prevention. Guide to completing the facility worksheets for the Certificate of Live Birth and Report of Fetal Death (2003 revision). Updated May 2016 https://www.cdc.gov/nchs/data/dvs/GuidetoCompleteFacilityWks.pdf. Accessed August 16, 2018.
- 18. Kotelchuck M. An evaluation of the Kessner Adequacy of Prenatal Care Index and a proposed Adequacy of Prenatal Care Utilization Index. Am J Public Health. 1994;84(9):1414–1420. doi:10.2105/AJPH.84.9.1414 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. McNutt LA, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol. 2003;157(10):940–943. doi:10.1093/aje/kwg074 [DOI] [PubMed] [Google Scholar]
- 20. SAS Institute, Inc. SAS: Version 9.4. Cary, NC: SAS Institute, Inc; 2013. [Google Scholar]
- 21. Markus AR, Rosenbaum S. The role of Medicaid in promoting access to high-quality, high-value maternity care. Womens Health Issues. 2010;20(suppl 1):S67–S78. [DOI] [PubMed] [Google Scholar]
- 22. Barnett JC, Vornovitsky MS. Health Insurance Coverage in the United States: 2015. Washington, DC: US Government Printing Office; 2016. https://www.census.gov/content/dam/Census/library/publications/2016/demo/p60-257.pdf. Accessed August 16, 2018. [Google Scholar]
- 23. Delgado-Rodríguez M, Llorca J. Bias. J Epidemiol Community Health. 2004;58(8):635–641. doi:10.1136/jech.2003.008466 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Henderson ZT, Suchdev DB, Abe K, Johnston EO, Callaghan WM. Perinatal quality collaboratives: improving care for mothers and infants. J Womens Health (Larchmt). 2014;23(5):368–372. doi:10.1089/jwh.2014.4744 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. US Department of Health and Human Services. 2016 National Healthcare Quality and Disparities Report. Rockville, MD: Agency for Healthcare Research and Quality; 2017. https://www.ahrq.gov/sites/default/files/wysiwyg/research/findings/nhqrdr/nhqdr16/final2016qdr-cx.pdf. Accessed August 18, 2018. [Google Scholar]
- 26. Mathews TJ, MacDorman MF, Thoma ME. Infant mortality statistics from the 2013 period linked birth/infant death data set. Natl Vital Stat Rep. 2015;64(9):1–29. [PubMed] [Google Scholar]
- 27. Martin JA, Hamilton BE, Osterman MJK, Driscoll AK, Drake P. Births: final data for 2016. Natl Vital Stat Rep. 2018;67(1):1–55. [PubMed] [Google Scholar]