Updating a Perinatal Risk Scoring System to Predict Infant Mortality

Abstract

Objective

The Birth Score Project (Project WATCH) began in the rural state of West Virginia (WV) in the United States in 1984. The project is intended to identify newborns with a greater risk of infant mortality. The primary objective of this study was to update the current Birth Score based on current literature and rigorous statistical methodology.

Study Design

The study merged data from the Birth Score, Birth Certificate (birth years 2008–2013), and Infant Mortality Data (N = 121,640). The merged data were randomly divided into developmental (N = 85,148) and validation (N = 36,492) datasets. Risk scoring system was developed using the weighted multivariate risk score functions and consisted of infant and maternal factors.

Results

The updated score ranged from 0 to 86. Infants with a score of ≥17 were categorized into the high score group (n = 15,387; 18.1%). The odds of infant mortality were 5.6 times higher (95% confidence interval: 4.4, 7.1) among those who had a high score versus low score.

Conclusion

The updated score is a better predictor of infant mortality than the current Birth Score. This score has practical relevance for physicians in WV to identify newborns at the greatest risk of infant mortality and refer the infants to primary pediatric services and case management for close follow-up.

Infant mortality continues to be a major public health issue in the United States. Although infant mortality rates have declined to an all-time low in 2014 to 5.82 deaths per 1,000 births, comparisons to other industrialized nations reveal some sobering statistics.¹ The United States has a higher infant mortality rate than other industrialized nations,² even after accounting for factors that could artificially deflate the infant mortality in other countries, such as not counting extreme preterm births toward the live birth count.³ Furthermore, within the United States, there is considerable state-to-state variation in infant mortality statistics. In West Virginia (WV), an Appalachian state in the United States, the infant mortality rate has always remained significantly above the national average.⁴

Historical Perspective: The West Virginia Birth Score

The high infant mortality rate in WV, relative to national U.S. averages, in the late 1970s and early 1980s prompted the state to initiate an intervention to decrease this number. The chosen intervention was a risk-based scoring tool to identify infants born in WV who are likely to die of sudden infant death syndrome (SIDS). The risk identification scoring tool created by Myerberg et al⁵ was based on an English scoring tool for the Sheffield Intervention Program.⁶ The scoring tool was titled the WV Birth Score and has been used statewide since its inception in 1984. The scoring tool used 13 variables categorized into maternal characteristics (such as age, education, marital status, and race), pregnancy-related characteristics (such as parity, few prenatal care [PNC] visits, complications, and multiple births), and infant characteristics (such as male sex, low Apgar, and birth defects) that were individually weighted and added together to create the “score.” Infants were designated as having a “high score” if they were in the top 15% of scores. Infants in the “high score” group were deemed to be at an increased risk of preventable death in the first year of life, and a schedule of more frequent visits to a physician was arranged for this group.

From 1986 to 1989, there was a consistent reduction in postneonatal mortality in WV, perhaps due in part to increased identification of the most high-risk infants and linking them to medical services. A nationwide renewed focus on reducing death from SIDS with the advent of the Back to Sleep campaign⁷ may have also contributed to modest decreases in postneonatal mortality in WV in the 1990s, although nationwide effects had also plateaued by 2011.⁸ Given the decrease in infant mortality with the inception of the statewide Birth Score, completion of the score on every infant became a statewide mandate in 1998 with House Bill 2388.⁹

In the 2000s, the Birth Score continued to be used to identify infants at a high risk of mortality in the state of WV. High-risk infants and their families were referred to a variety of improved state-funded medical and social support services, such as Right From The Start, Birth To Three, and HealthCheck, in addition to a notification to the primary care physician. Starting in 2005 and implemented statewide in 2007,¹⁰ an updated analysis was conducted to reassess the most important variables contributing to infant mortality. Revisions included an automatic “high score” for infants born with a very low birth weight (LBW; <1,500 g), congenital abnormalities, and low Apgar score (<3) at 5 minutes, and the addition of new variables such as nicotine use during pregnancy. This revised scoring system has been in place up to the present day and can be viewed in its entirety in Mullett et al’ 2010 publication.¹⁰

Rationale for an Updated Infant Mortality Risk Prediction Score

Despite improvements since the 1980s, the infant mortality rate in WV remains significantly higher than the national average (7 per 1,000 births in 2014, as compared with 5.8 in the United States).¹¹ Thus, the authors wished to reassess the validity of the current Birth Score in its ability to consistently identify infants at the highest risk of mortality in the first year of life. State epidemiologists note that WV infant mortality continues to be highest among infants born to mothers with less than high school education, those who are unmarried, those who smoked or drank during pregnancy, and those who did not seek PNC visits.¹² Thus, an updated look at variables predictive of infant mortality found in the United States and uniquely relevant to WV was warranted.

Objective

The primary objective of this study was to update the Birth Score using rigorous scientific methodology. The goal of a sensitive and specific assessment of infant mortality using the revised evidence-based tool would continue to be an immediate referral of high-risk families to federal-, state-, and hospital-level initiatives and services to address and prevent infant mortality occurrence. Thereby, results could help ensure that state-level services serve the infants at the highest risk to ensure optimization of state and federal resources and ultimately continue to decrease infant mortality in the state of WV.

Methods

Community and Expert Input

To assist with risk score revision and development, a clinical and research partnership group was established. The Medical Advisory Group (MAG) included two general pediatricians, a neonatologist, an obstetrician, a pediatric cardiologist, a geneticist, and four pediatric researchers including a perinatal epidemiologist, a biostatistician, and state public health officials. This group worked closely with the research team on all aspects of the project.

Data Sources and Study Population

The study used merged data from the Birth Score and Birth Certificate from the years 2008 to 2013 with the Infant Mortality Data (Vital Statistics) of WV (N = 121,859). The infants who died within the first week of life, termed as early neonatal death (< 7 days; n = 219) and were excluded, with a final sample size of N = 121,640. This exclusion criterion was suggested by the MAG to exclude infants who likely died while in the hospital with no further referral to services.

Factor Inclusion

The first step of the score development was to examine the literature on maternal and infant risk factors associated with infant mortality. A total of 38 risk factors were identified. Our study had information about 23 of these risk factors. Based on the availability of risk factors and input from the MAG, as well as examining significant (p < 0.05) bivariate associations related to infant mortality, 16 risk factors were selected for the initial stage of model development. The literature review table and the list of the 38, 23, and 16 risk factors are provided in Supplementary Material (available in the online version).

The dataset (N = 121,640) was then randomly divided into a 70/30 split to create developmental (N = 85,148) and validation (N = 36,492) datasets, respectively. Randomization was checked for key variables (Supplementary Material, available in the online version). The development dataset was used to develop the final logistic regression model that included the 16 risk factors. Variables with the highest nonsignificant p-value were deleted from the model one at a time. All variables with p-values < 0.15 were included in the final model.¹³ The risk factors at birth included in the final model were as follows: (1) birth weight,(2) gestational age, (3) congenital abnormality, (4) Apgar score, (5) maternal age, (6) maternal education, (7) maternal substance use during pregnancy, (8) maternal smoking during pregnancy, (9) previous pregnancies, and (10) PNC visits. Table 1 includes the descriptive information of these variables as well as the unadjusted odds ratios (ORs).

Table 1.

Descriptive characteristics of significant perinatal risk factors and unadjusted ORs of infant death, with 95% CI, for the full dataset excluding deaths within the first week (N = 121,640)

Perinatal risk factor	N	%	OR [95% CI]
Apgar score (5 min)	121,400
<8	2,736	2.25	10.93 [8.43, 14.16]
≥8 (ref.)	118,664	97.75
Birth weight (grams)	121,636
<1,500	1,303	1.07	56.09 [47.86, 65.72]
1,501–2,000	1,959	1.61	36.65 [32.72, 41.05]
2,001–2,500	7,034	5.78	14.33 [13.59, 15.11]
2,501–3,000	25,015	20.57	2.4 [2.31, 2.49]
>3,000 (ref.)	86,327	70.97
Congenital abnormalities	121,640
Yes	433	0.36	8.7 [4.74, 15.97]
No (ref.)	121,207	99.64
Gestational age (weeks)	121,537
<28	497	0.41	59.1 [43.61, 80.08]
28 to <32	1,027	0.85	7.22 [4.34, 12.03]
32 to <37	11,205	9.22	2.54 [1.92, 3.36]
≥37 (ref.)	108,808	89.53
Maternal age (years)	120,581
<24	45,809	37.99	1.63 [1.33, 2]
≥24 (ref.)	74,771	62.01
Maternal education (years)	119,947
<10	6,353	5.3	3.16 [2.22, 4.51]
10–11	15,076	12.57	1.87 [1.36, 2.56]
12	44,202	36.85	1.64 [1.29, 2.09]
>12 (ref.)	54,316	45.28
Maternal smoking during pregnancy	121,630
Yes	36,635	30.12	2.8 [2.28, 3.43]
No (ref.)	84,995	69.88
Maternal substance and alcohol use during pregnancy	102,393
Yes	3,368	3.29	3.55 [2.48, 5.07]
No (ref.)	99,025	96.71
Prenatal care visits (no.)	119,825
<10	28,767	24.01	3.49 [2.84, 4.29]
≥10 (ref.)	91,058	75.99
Previous pregnancy	121,636
Yes	79,473	65.34	1.34 [1.07, 1.68]
No (ref.)	42,163	34.66

Abbreviations: CI, confidence interval; OR, odds ratio.

Factors Used in Risk Score Development

The risk score was developed based on the presentation of multivariate data for clinical use tutorial by Sullivan et al.¹⁴ The final model included seven binary and three categorical risk factors, as described in the following.

Multiple Categorical Factors

Birth weight:

The conventional birth weight categories include LBW (<2,500 g), normal birth weight (2,500–4,000 g), and high birth weight (HBW; >4,000 g).¹⁵ The HBW category was not significantly associated with infant mortality and thus was not analyzed as a separate category. LBW, which was significant, was further categorized as <1,500, 1,501 to 2,000, 2,001 to 2,500, and 2,501 to 3,000g. Birth weight of >3,000 g was used as the referent/base category. The reference values for the other groups were calculated as the midpoints for each category. To determine the reference values for the first category (< 1,500 g) and last category (>3,000 g), we used the 1st percentile (1,418 g) and 99th percentile (4,486 g) to minimize the influence of extreme values.

Gestational age:

The gestational age were categorized as extremely preterm (<28 weeks), very preterm (28 to <32 weeks), moderate-to-late preterm (32 to <37 weeks), and term birth (≥37 week).¹⁶ Gestational age of ≥37 weeks was used as the base category. The reference values for the other groups were calculated as the midpoints for each category. To determine the reference values for the first category (< 28 weeks), instead of using the 1st percentile we used the viable gestational age in the United States (24 weeks),¹⁷ and for the last category (≥37 term), we used the 99th percentile (41 weeks) to minimize the influence of extreme values.

Maternal education:

The maternal education in years ranged from 1 to 17 and was categorized into four categories including <10, 10 to 11, 12, and >12. The referent/base category was higher than 12th grade (high school) education.

Binary Factors

Apgar score:

A low Apgar score at 5 minutes is strongly associated with a risk of infant mortality.^18,19 Although some studies have reported a cutoff of <7,¹⁹ the 5-minute Apgar score was categorized as <8 and ≥8 based on receiver operating characteristic (ROC) analysis results for this population.

Congenital abnormality:

Birth Score collects information on broad categories of congenital abnormalities and was analyzed as a binary (yes/no) risk factor.

Maternal age:

Literature suggests that maternal age and the risk of infant death have a U-shaped relationship.^20,21 We first assessed for this U-shaped relationship using maternal age as 5 categories (< 20, 20–29 (referent), 30–34, 35–39, and ≥40), but none of the categories were significantly associated with infant mortality. Therefore, maternal age was dichotomized as <24 and ≥24 years based on the ROC analysis.

Maternal smoking during pregnancy:

Maternal smoking during pregnancy was self-reported and included as a binary variable (yes/no).

Maternal substance use and alcohol use during pregnancy:

Maternal substance use and alcohol use during pregnancy were self-reported and combined to make a single binary variable (yes/no).

PNC visits:

The current recommendation for PNC visit schedule for uncomplicated pregnancies by the American Congress of Obstetrics and Gynecology (ACOG) consists of a visit every 4 weeks until 28 weeks, every 2 to 3 weeks until 36 weeks, and weekly after 36 weeks until delivery. Based on this, the optimal number of PNC visits for an uncomplicated pregnancy is 10 or 11. The median number of PNC visits in the United States is 11,²³ but ROC analysis for this study showed that 10 visits optimized the sensitivity and specificity of PNC visits and infant mortality. The bivariate association was stronger for <10 versus ≥10 compared with <11 versus ≥11;²⁴ therefore, we analyzed the <10 visits compared with ≥10 PNC visits.

Previous pregnancy:

Several studies have shown a higher risk of infant mortality in multiparous women compared with nulliparous.^25,26 This factor was categorized as a binary variable (i.e., 0 and ≥1 number of previous pregnancies).

Establishing Weights of the Factor Categories

After the risk factor variables were selected and the base categories were assigned for each variable, the next step was to determine how far each category was from the base category in regression units. For the variables with more than two categories, we subtracted the value of each category (W_ij) from the base value (W_{i_ref}) and then multiplied it by the regression coefficient (βi). For the binary categorical variables, the distance between a variable and its base category in regression coefficient units was equal to the size of the coefficient. The constant of the scoring system was defined as the increase in risk in regression units associated with every 500-g decrease in birth weight. For our data, the constant (B) was −500 g multiplied by (−0.00022) and equaled 0.11. The points associated with each category of each risk factor were computed by its distance from the base category in regression coefficient units divided by the constant $\left[{\text{Point}}_{ij}=\frac{\beta i\left(Wij-W{i}_{\text{ref }}\right)}{B}\right]$. The points were then rounded to the nearest integer to get its point value. The base category for each risk factor was assigned a 0 in the scoring system.

Predicted Probability of Risk for Each Factor

The next step was to determine the predicted probability or the risk associated with each point total using the multiple logistic regression equation:

$$p^{\wedge }=\frac{1}{1+\text{exp}\left(-{\sum }_{i=0}^p\mathrm{\beta }\text{iXi}\right)}$$

The point total, when multiplied by the constant (B = 0.11), approximates ${\sum }_{i=0}^p\mathrm{\beta }\text{iXi}$, where ${\sum }_{i=0}^p\mathrm{\beta }\text{iXi}\approx {\beta }_{\text{o}}+{\beta }_1$ (birth weight) + β₂ (gestational age) + β₃ (maternal age) + B(Point total).¹⁴

Establishing a Clinical Cutoff of High Risk for the Score

Once the weighted points were established for each risk factor, the total score was calculated by adding up the points for all risk factors using complete case analysis. ROC analysis was then performed to establish a cutoff of the total score that would maximize the sensitivity and specificity. The number needed to treat (NNT) was also calculated using the cutoff established by the ROC analysis.

Validation

To validate the model derived from the developmental data, we applied this scoring method to the validation dataset (N = 36,492). The model fit statistics of the current Birth Score, updated risk score established using the developmental dataset, and updated risk score applied to the validation dataset were then compared. Model comparisons were made based on the most commonly used measures of calibration and discrimination (such as Hosmer–Lemeshow’s goodness-of-fit test and c-statistics).^27,28 In addition, measures of global fit were also examined; these included the Akaike Information Criteria and the Bayesian Information Criteria.²⁸

Results

A total of 592 infant deaths occurred in 2008 to 2013, of which 219 deaths occurred within the first week and were excluded from analysis. The final dataset of 121,640 observations was divided into developmental and validation data-sets that included 267 and 106 infant mortality cases, respectively. Table 1 presents the population characteristics as well as the bivariate associations of these characteristics with infant mortality.

Development of the Perinatal Risk Score (N = 85,148)

Logistic regression using the developmental data with the three continuous risk factors (birth weight, gestational age, and maternal education) and the seven binary risk factors provides the ORs in Table 2. As the birth weight, maternal education, and gestational age increased, the odds of infant mortality decreased. Strong associations were observed for maternal smoking, congenital abnormality, Apgar score, and infant mortality. The odds of infant mortality were twice among mothers who smoked versus did not smoke during pregnancy, five times among children born with congenital abnormalities versus no congenital abnormalities, and nearly four times among infants with a <8 Apgar score at 5 minutes versus ≥8 Apgar score.

Table 2.

Logistic regression and point scores for risk factors from the development dataset (N = 85,148)

Perinatal risk factor	Parameter estimate (B)	Standard error	Wald-chi-square	p-Value	OR [95% CI]	Ref. values	Bx (ref. values − ref.)	Bx (ref. values − ref.)/constant	Rounded values
Apgar score (5 min)
<8	1.3403	0.2204	36.9859	<0.0001	3.82 [2.48, 5.88]	1	1.3403	12.1845	12
≥8 (ref.)						0	0
Birth weight (grams)	−0.00022	0.000129	2.8421	0.0918	1 [1. 1]
< 1,500						1,418	0.67496	6.1360	6
1,501–2,000						1,750	0.60192	5.4720	5
2,001–2,500						2,250	0.49192	4.4720	4
2,501–3,000						2,750	0.38192	3.4720	3
>3,000 (ref.)						4,486	0
Congenital abnormalities
Yes	1.6378	0.4111	15.8707	<0.0001	5.14 [2.30, 11.52]	1	1.6378	14.8891	15
No (ref.)						0	0
Gestational age (weeks)	−0.1631	0.0282	33.3599	<0.0001	0.85 [0.80, 0.90]
<28						24	2.7727	25.2064	25
28 to <32						29.5	1.87565	17.0514	17
32 to <37						34	1.1417	10.3791	10
≥37 (ref.)						41	0
Maternal age (years)
<24	0.2682	0.1506	3.1726	0.0749	1.31 [0.97, 1.76]	1	0.2682	2.4382	2
≥24 (ref.)						0	0
Maternal education (years)	−0.0984	0.0362	7.4059	0.0065	0.91 [0.84, 0.97]
<10						8	0.8856	8.0509	8
10–11						10.5	0.6396	5.8145	6
12						12	0.492	4.4727	4
>12 (ref.)						17	0
Maternal smoking during pregnancy
Yes	0.7466	0.1479	25.4666	<0.0001	2.11 [1.58, 2.82]	1	0.7466	6.7873	7
No (ref.)						0	0
Maternal substance and alcohol use during pregnancy
Yes	0.3703	0.2573	2.0701	0.1502	1.45 [0.87, 2.40]	1	0.3703	3.3664	3
No (ref.)						0	0
Prenatal care visits (no.)
<10	0.4225	0.1502	7.9142	0.0049	1.53 [1.14, 2.05]	1	0.4225	3.8409	4
≥10 (ref.)						0	0
Previous pregnancy
Yes	0.4521	0.1633	7.661	0.0056	1.57 [1.14, 2.16]	1	0.4521	4.1100	4
No (ref.)						0	0

Abbreviations: CI, confidence interval; OR, odds ratio.

Note: Parameter estimates are from the initial logistic regression using continuous variables for nonbinary factors. The constant was 0.11.

Table 2 also shows the calculations in developing the point scores for all risk factors included in the final logistic regression model. The point scores system ranged from 0 (infant born with none of the 10 risk factors) to 86 (infant scored within the highest risk categories on all 10 risk factors). The mean score was 12.23 (standard deviation ± 8.28). The predicted probabilities (risk) of infant mortality corresponding to the scores ranged from 0.03 to 78.64% (Table 3).

Table 3.

Predicted risk of infant mortality

Total risk score	Predicted risk (%)	Total risk score	Predicted risk (%)	Total risk score	Predicted risk (%)	Total risk score	Predicted risk (%)
0	0.03	22	0.32	44	3.50	66	28.97
1	0.03	23	0.36	45	3.89	67	31.29
2	0.04	24	0.40	46	4.32	68	33.70
3	0.04	25	0.45	47	4.80	69	36.20
4	0.04	26	0.50	48	5.33	70	38.78
5	0.05	27	0.56	49	5.91	71	41.42
6	0.06	28	0.62	50	6.56	72	44.11
7	0.06	29	0.69	51	7.26	73	46.84
8	0.07	30	0.77	52	8.04	74	49.58
9	0.08	31	0.86	53	8.89	75	52.33
10	0.09	32	0.96	54	9.83	76	55.06
11	0.10	33	1.07	55	10.84	77	57.77
12	0.11	34	1.19	56	11.95	78	60.43
13	0.12	35	1.33	57	13.16	79	63.02
14	0.13	36	1.48	58	14.47	80	65.55
15	0.15	37	1.65	59	15.89	81	67.99
16	0.17	38	1.84	60	17.41	82	70.33
17	0.19	39	2.05	61	19.05	83	72.58
18	0.21	40	2.28	62	20.80	84	74.71
19	0.23	41	2.54	63	22.68	85	76.73
20	0.26	42	2.83	64	24.66	86	78.64
21	0.29	43	3.15	65	26.76

Establishing a Clinical Cutoff of High Risk for the Score

The results from the ROC analysis (Fig. 1) showed that the area under the curve (AUC) was 0.78 (95% confidence interval [CI]: 0.75, 0.81), which is significantly better than a chance of 0.50 (difference score = 0.28, chi-square (1) = 263.96; p < 0.0001). The Hosmer–Lemeshow goodness-of-fit test (chi-square (8) =12.13; p = 0.15) and the c-statistics = 0.77 showed that the model fit the data well in terms of discrimination and calibration²⁷ (Table 4). There is a significant relationship between higher score and increased odds of infant mortality in the state of WV (OR = 1.1 [95% CI: 1.09, 1.11]).

Fig. 1.

Receiver operating characteristic (ROC) curve and corresponding area under the curve (AUC) statistics for the total risk score of predicting infant mortality in the developmental dataset (N = 85,148). The ROC graphs the fraction of true-positive results (sensitivity) against the false-positive rate (100% specificity) for a series of cutoff points. The AUC was 0.78 (95% confidence interval: 0.75, 0.81). The cutoff of 17 maximized the sensitivity and specificity of the risk score in predicting infant mortality.

Table 4.

Validation comparisons between the developmental dataset (N = 70,299), validation dataset (N = 30,156), and the current Birth Score used with entire dataset excluding perinatal deaths (N = 121,638)

	Developmental	Validation	Birth Score
Total frequency	70,299	30,156	121,638
Log likelihood	−1,331.3	−532.6	−2,472.3
Error rate	0.00316	0.00292	0.00307
AIC	2,666.67	1,069.261	4,948.537
AICC	2,666.67	1,069.261	4,948.537
BIC	2,684.991	1,085.889	4,967.955
SC	2,684.991	1,085.889	4,967.955
R2	0.00462	0.00456	0.00097
Max-rescaled R2	0.111	0.117	0.024
AUC [95% CI]	0.7796 [0.7459, 0.8133]	0.762928 [0.7055, 0.8204]	0.649199 [0.6201, 0.6783]
Brier score	0.003089	0.002856	0.003053
Wald (DF) Pr > Wald)	454.60 (1) <0.0001	197.49 (1) <0.0001	128.70 (1) <0.0001
Odds ratio [95% CI]	1.1 [1.09, 1.109]	1.1 [1.085, 1.114]	1.012 [1.010, 1.015]
Hosmer-Lemeshow goodness-of-fit (X2 (DF) Pr > X2)	12.13 (8) 0.1453	10.04 (8) 0.2626	41.51 (8) <0.0001
Contrast reference = chance (X2 (DF) Pr > X2)	263.96 (1) <0.0001	80.54 (1) <0.0001	100.70 (1) <0.0001
Percent concordant	67.1	62.2	46.8
Percent discordant	14	14.3	20.2
Percent tied	18.9	23.6	33
Pairs	15,487,238	2,645,984	45,231,845
Somers’ D	0.531	0.479	0.266
Gamma	0.655	0.627	0.397
Tau-a	0.003	0.003	0.002
c	0.765	0.74	0.633
Binary	Developmental	Validation	Birth Score
High score cutoff	>17	>17	>99
High score	15,387 (18.07)	6,510 (17.84)	20,828(17.12)
DF, estimate (standard error) Wald X2 Pr > X2	(1) 1.7224 (0.1234) 194.96 <0.0001	(1) 1.5716 (0.1948) 65.10 <0.0001	(1) 1.0949 (0.1069) 104.89 <0.0001
Odds ratio [95% CI]	5.598 [4.396, 7.129]	4.814 [3.287, 7.052]	2.989 [2.424, 3.686]
AIC	3,613.617	1,452.071	5,064.126
SC	3,622.969	1,460.575	5,073.835
−2 Log L	3,611.617	1,450.071	5,062.126
Percent concordant	45.2	41.9	31.6
Percent discordant	8.1	8.7	10.6
Percent tied	46.8	49.4	57.9
Pairs	22,663,227	3,856,91 6	45,231,845
Somers’ D	0.371	0.332	0.21
Gamma	0.697	0.656	0.499
Tau-a	0.002	0.002	0.001
c	0.686	0.666	0.605
Diagnostic	Developmental	Validation	Birth Score
Sensitivity	55.06 [48.87, 61.12]	50.94 [41.05, 60.78]	38.07 [33.12, 43.21]
Specificity	82.05 [81.79, 82.30]	82.26 [81.86, 82.65]	82.94 [82.73, 83.15]
Positive likelihood ratio	3.07 [2.75, 3.42]	2.87 [2.38, 3.47]	2.23 [1.96, 2.54]
Negative likelihood ratio	0.55 [0.48, 0.63]	0.60 [0.49, 0.72]	0.75 [0.69, 0.81]
Disease prevalence	0.31 [0.28, 0.35]	0.29 [0.24, 0.35]	0.31 [0.28, 0.34]
Positive predictive value	0.96 [0.86, 1.06]	0.83 [0.69, 1]	0.68 [0.60, 0.78]

Abbreviations: AIC, akaike information criterion; AICC, akaike information criterion corrected for small sample size; AUC, area under the curve; BIC, Bayesian information criterion; CI, confidence interval; DF, degrees of freedom; SC, Schwarz criterion.

ROC analysis results demonstrated that a risk score of 17 would maximize the sensitivity and specificity of this updated score in predicting infant mortality. Thus, infants with a score of ≥17 were categorized as infants with a high score (N = 15; 387 (18.07%). The odds of infant mortality was 5.6 times (OR = 5.6 [95% CI: 4.4, 7.1]) among those who had a high score versus low score. Two categories of gestational age had a score of ≥17, giving this infant an automatic high score if the infant was born before 32 weeks of gestation (Table 2). For NNT, the results show that 128 infants need to be identified as having a high score using this updated score to prevent one death.

Validation

Table 4 compares model fit statistics of the updated risk score established using the developmental dataset, and the updated risk score applied to the validation dataset as well as the current Birth Score. Notably, the Hosmer-Lemeshow goodness-of-fit test was not significant for the developmental data (chi-square (8) =12.13; p = 0.15) and the validation data (chi-square (8) = 10.04; p = 0.26), but it was significant for the current Birth Score (chi-square (8) = 41.51; p < 0.0001), indicating a good fit for the updated risk score but not for the current Birth Score. The AUC and the c-statistics were also significantly different from the chance for all datasets but were highest for the developmental data (AUC = 0.78) compared with the validation data (AUC = 0.76) and the current Birth Score (AUC = 0.65).

Discussion

This study derived, and internally validated, a perinatal risk score model for predicting the risk of infant mortality based on 121,640 births in the rural state of WV from the years 2008 to 2013. Based on the results, the updated score is a better predictor of infant mortality than the currently used score. This score has clinical relevance for pediatricians in terms of better targeting those newborns who are potentially at the greatest risk of infant mortality and arranging for closer follow-up. This follow-up could be achieved by seeing children in the clinic more often than normal well-child visits or initiating more intensive counseling during the normal well-child visits. Additionally, this would likely include linking these high-risk infants to social workers, case managers, and other pediatric statewide initiatives. According to the new score, 3,600 infants will be, on average, identified as having a high score each year (18% of 20,000 average births per year in WV). With the NNT being 128, approximately 28 infant mortality cases can be potentially prevented annually through the use of this risk score.

Comparison

Compared with the current Birth Score, the new score has better predictive capabilities and model fit in terms of discrimination and calibration. The current Birth Score comprises seven risk factors, five of which were also included in the updated score (e.g., birth weight, maternal age, maternal education, number of previous pregnancies, and maternal smoking during pregnancy). Congenital abnormalities and high Apgar score at 5 minutes were given an automatic high score in the current Birth Score. These two factors were not an automatic high score in the updated score. Two factors that are included in the current Birth Score (infant’s sex and mother’s feeding intention) were excluded from the updated score. The updated score includes three additional risk factors (i.e., number of PNC visits, alcohol and substance use during pregnancy, and gestational age).

Strengths

The updated score was derived and internally validated on a large cohort of all WV births over a large span of time using traditional, appropriate statistical techniques. Moreover, this population-based registry collects data on numerous factors that were identified as important risk factors for infant mortality. The factors included in the logistic regression model were based on a thorough literature review. The model had good predictive capability and fits the data well, as observed by the model fit statistics.

Limitations

One of the main limitations of this risk score is its lack of generalizability to other populations due to the unique demographics of the rural, Appalachian state in the United States. As with any risk scoring system, this model provides a risk estimate for the population as a whole and not a specific individual. Clinicians should treat this as a useful tool in making informed decisions about an infant’s health care management.

Future Directions

Although the infant mortality rate in the state of WV is higher than the national U.S. average, the total number of infant deaths compared with total births is fairly low. Therefore, future studies using the updated scoring system to potentially predict other outcomes that are more commonplace, such as developmental delay, would be useful. Additionally, future studies may wish to consider the development of bioinformatics methodology in validating the risk score. Finally, validating the risk score in populations outside of this rural Appalachian state in the United States will be important.

Conclusion

In conclusion, the risk score that was developed and validated in this study is a better predictor of infant mortality than the currently used score in the rural Appalachian state of WV in the United States. More generally, these research findings may provide a useful foundation for those interested in addressing the surprisingly high incidence of infant mortality in the United States. From a clinical perspective, these findings provide a useful tool for WV clinicians for predicting the risk of infant mortality. These findings have broader public health and health policy implications and could help guide policymakers for allocating health care resources to families of infants who have a high-risk score. Targeted interventions along with the allocation of health care resources to infants with a high score have a potential to lower the incidence of infant mortality in this state.