The Effectiveness of Food Insecurity Screening in Pediatric Primary Care

Abstract

Background

Food insecure children are at increased risk for medical and developmental problems. Effective screening and intervention are needed.

Methods

Our purpose was to (1) evaluate the validity and stability of a single item food insecurity (FI) screen. (2) Assess whether use may lead to decreased FI. Part of a larger cluster randomized controlled trial, pediatric residents were assigned to SEEK or control groups. A single FI question (part of a larger questionnaire) was used on SEEK days. SEEK residents learned to screen, assess, and address FI. A subset of SEEK and control clinic parents was recruited for the evaluation. Parents completed the USDA Food Security Scale (“gold standard”), upon recruitment and 6-months later. Validity, positive and negative predictive values (PPV, NPV) was calculated. The proportion of screened families with initial and subsequent FI was measured. Screening effectiveness was evaluated by comparing SEEK and control screening rates and receipt of Supplemental Nutrition Assistance Program (SNAP) benefits between initial and 6-month assessments.

Results

FI screen stability indicated substantial agreement (Cohen’s kappa =0.69). Sensitivity and specificity was 59% and 87%, respectively. The PPV was 70%; NPV was 81%. SEEK families had a larger increase in screening rates than control families (24% vs. 4.1%, p<0.01). SEEK families were more likely to maintain SNAP enrollment (97% vs. 81%, p=0.05). FI rates remained stable at approximately 30% for both groups.

Conclusions

A single question screen can identify many families with FI, and may help maintain food program enrollment. Screening may not be adequate to alleviate FI.

BACKGROUND

Food security is defined as having access, at all times, to provide enough food for an active, healthy life [1]. Food insecurity (FI) occurs when nutritionally adequate food is unavailable, or its availability is limited or uncertain. Approximately 21% of U.S. households with children were food insecure in 2011 [2].

FI impairs children’s health and well-being. Children who are food insecure have a higher frequency of acute and chronic illnesses and hospitalizations compared to food secure children [3–5]. They are also at higher risk for developmental delay, learning difficulties, attachment problems, depression, and both internalizing and externalizing behavioral problems [6–10]. Pregnant women who are food insecure are more likely to have infants with birth defects and low birth weight [11,12].

Because of their frequent contact with young children, pediatricians are uniquely positioned to identify and help address FI. However, time with children and families is limited. Without brief, effective screening tools, pediatricians may be reluctant to screen families for FI. The objective of this study was to evaluate the validity and stability of a single question on FI in the SEEK Parent Screening Questionnaire (PSQ), and to assess whether its use reduced the rate of FI.

METHODS

Context and Setting

The evaluation of this single item FI screen was part of the SEEK I study, a randomized controlled trial conducted in a pediatric resident continuity clinic serving urban, low income children [13]. The objectives of SEEK were to teach pediatric residents to screen for child maltreatment risk factors using a standardized screening tool (the Parent Screening Questionnaire, or PSQ); to briefly assess positive screens; and to provide brief interventions such as information on community resources, social work referral, and/or motivational interviewing for behavior change.

Resident clinic days were randomized by coin toss to be either SEEK (Intervention) days or usual care (Control) days. Residents assigned to clinics on SEEK days were trained to screen for, assess, and provide initial management of FI. FI interventions included providing information on how to apply for Supplemental Nutrition Assistance Program (SNAP, formerly food stamps) benefits and other federal food assistance programs, and providing information about local food resources such as food pantries. Screening on SEEK days was standardized using the Parent Screening Questionnaire (PSQ), discussed below. Families with pediatricians in the control group received routine well-child care.

Sample

We recruited a subset of parents from each group to evaluate the model. Parents who brought their children under age 6 to the resident continuity clinic for a checkup were eligible to participate. Parents were excluded if they had another child in the study or if the child was in foster care. Because the clinic population was nearly all English speaking, the PSQ was not translated into other languages, and we excluded non-English speakers. A research assistant approached eligible parents assigned to both SEEK and control clinic days to explain the study. If interested, informed consent was obtained following procedures approved by the University’s Human Subjects Review Committee. Parent recruitment took place over approximately 18 months. This meant that many parents had been exposed to the SEEK model before being recruited, and the initial data therefore represent early outcomes rather than true baseline.

SEEK Evaluation

All parents of children under 6 years of age assigned to SEEK clinic days were asked to complete the PSQ while waiting for their child’s checkup. Parents recruited for the evaluation from both SEEK and control clinics were scheduled to return within two weeks to complete a self-administered computerized interview in our laboratory. The interview included computerized versions of: (1) the PSQ (hereafter referred to as the Lab PSQ) to determine questionnaire stability; and (2) the Food Security Scale (FSS), [14–17] a comprehensive, standardized FI measure, to examine validity of the screening question.

Six months after their initial laboratory visit, recruited parents returned to the laboratory and completed the Lab PSQ and FSS to evaluate the effectiveness of SEEK. Medical record reviews were performed by a trained medical student who reviewed records of children in recruited families for evidence of FI screening. FI screening was considered to have occurred if a completed SEEK PSQ was in the chart or there were notations about FI screening in the clinic visit note. Because the PSQ was only present in the records of SEEK group children, the student could not be blinded to study group assignment. Bias was minimized through the development and use of a standard data abstraction tool with a set of decision rules.

Screening Questionnaire

The SEEK Parent Screening Questionnaire (PSQ) is a 1-page paper and pencil questionnaire developed to screen parents for child maltreatment risk factors, including FI, during routine well child care. A brief introduction framed the questions in a supportive manner. All questions had yes/no responses. The SEEK project and PSQ have been described in detail elsewhere [13, 18–23]. The full SEEK questionnaire in English, Spanish, and Chinese is available at: http://theinstitute.umaryland.edu/seek/seek_psq.cfm

The PSQ FI question was developed through a review of validated screening measures [24,25]. After pilot testing, the single item selected was “In the last year, did you worry that your food would run out before you got money or food stamps to buy more?” This question came directly from the U.S. Department of Agriculture Food Security Scale [14].

Validation of the PSQ

Validation Measure

The U.S. Department of Agriculture’s Food Security Scale (FSS) is an 18-item measure designed to assess FI and hunger in the U.S [14]. We used the Guide to Implementing the Core Food Security Module, Revised to measure FI [15]. Respondents indicated whether they experienced any of 18 situations in the past year. According to the USDA, a family is food insecure if there are three or more affirmative responses. The reliability and validity of the FSS have been demonstrated [16,17].

Data Analysis

Sample sizes were calculated for the primary study outcome: reduction in incidence of child maltreatment. They were not conducted for secondary outcomes such as retention of SNAP benefits or reduction in FI. All analyses were conducted using SAS version 8.1 (Cary, NC). All study procedures were reviewed and approved by our institutional IRB.

Only data from SEEK parents were used to assess PSQ stability and validity, because control parents did not complete the PSQ in the pediatric clinic. These analyses were limited to the 205 parents who returned to complete the Lab PSQ and FSS within two weeks of completion of the Clinic PSQ (Figure 1).

Figure 1.

Algorithm of Families Recruited.

Stability

Because the PSQ completion method changed from paper (in the clinic) to computer (in the lab), we were unable to determine test-retest reliability. Instead, we examined the measure’s stability by comparing the FI question on the clinic PSQ to the Lab PSQ using Cohen’s Kappa [26]. Kappa values range from −1 to 1, with values of < 0.21 implying no to slight agreement, 0.21 – 0.6 implying fair to moderate agreement, and values > 0.6 implying substantial to perfect agreement.

Validity

To determine the validity of the clinic FI screen we calculated the sensitivity, specificity, positive (PPV) and negative predictive values (NPV), and likelihood ratios by comparing the Clinic PSQ question to the FSS.

Screening Effectiveness

Effectiveness of screening was examined in three ways:

(1) The rates of screening, assessment (additional questions to clarify), and action (recommendations and referrals) taken for parents in SEEK clinics were compared to those assigned to control clinics. To determine whether the screening rates for SEEK parents increased more than for control parents, mean rates of screening during well child visits for SEEK and control parents were calculated, and the changes in these rates were compared. (2) The rates of FI by Lab PSQ and by FSS were calculated initially and at 6-month follow-up. P-values for the rate difference between SEEK and control groups at each time period were calculated using chi-square analyses. The analyses were limited to parents with data both initially and 6 months later (N=419 for Lab PSQ; N=384 for FSS; Figure 1). (3) Among families with FI initially, rates of Supplemental Nutrition Assistance Program (SNAP, formerly called food stamps) receipt for SEEK and control groups were calculated. Rates at initial and 6-month assessments were compared by chi-square analysis to assess changes in SNAP receipt between SEEK and control parents. Data for this analysis were limited to the 63 parents with FI at baseline who completed the FSS and questions about SNAP receipt at both time points.

RESULTS

Participant Demographics

More than 90% of caregiver participants were mothers. Caregivers were predominantly African-American, and unmarried. Approximately one-third were employed. There were few statistically significant differences in demographics between the SEEK and control groups (Table 1). The median age of SEEK children was slightly younger than control children (6 vs. 8 months, p=0.03), and SEEK families had fewer children living in the home (mean 2.2 children for SEEK vs. 2.5 for control, p=0.04).

Table 1.

Demographic Characteristics of the SEEK and Control Group Families at the Time of Entry (N=558)

	SEEK Families (N=308)	Control Families (N=250)	P
Child
Age (months), median (IQR§)*	6.0 (13)	8.0 (17)	.03
African-American, N (%)	285 (93)	234 (94)	.74
Female, N (%)	143 (46)	127 (51)	.30
Parent
Age (years), mean (SD)	25.3 (6.8)	25.3 (7.3)	.94
Caregiver relationship, N (%)			.47
Mother	287 (93)	231 (92)
Father	13 (4)	15 (6)
Other	8 (3)	4 (2)
Marital status, N (%)			.98
Single	268 (87)	216 (86)
Married	26 (8)	22 (9)
Separated/Divorced/Widowed	14 (5)	12 (5)
Education, N (%)			.11
< High school	112 (36)	104 (42)
HS or GED	111 (36)	96 (38)
At least some college	85 (28)	50 (20)
Employed, N (%)	95 (32)	86 (35)	.47
Family
# of children in home, mean (SD)	2.2 (1.4)	2.5 (1.4)	.04
# of adults in home, mean (SD)	2.2 (1.2)	2.2 (1.1)	.65
Medical Assistance,¶ N (%)	270 (93)	224 (92)	.78

^*p<0.05.

Screen Stability and Validity

The stability of the FI screening question, as measured with Cohen’s kappa was 0.69, indicating substantial agreement between the Clinic PSQ and the Lab PSQ. The sensitivity and specificity of the screening question were 59% and 87%, respectively (Table 2). The positive predictive value of the screen was 70%, indicating that families with a positive screen had a 70% likelihood of having FI. The negative predictive value was 81%, indicating that families with a negative screen had an 81% likelihood of not having FI (i.e. they were food secure). Positive and negative likelihood ratios were 4.5 and 0.47, respectively. This indicates that families with FI were 4.5 times more likely to have a positive screen than those without FI, and those without FI were about half as likely to screen positive than those with FI.

Table 2.

Sensitivity, Specificity, Predictive Values and Likelihood Ratios for the Clinic PSQ Food Insecurity Question Compared with the USDA Food Security Scale (FSS) (N = 205)^a

Food Insecurity Question	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	LR+	LR−
‘In the last year, did you worry that your food would run out before you got money or food stamps to buy more?’	59	87	70	81	4.5	0.47

^aFamilies were classified as food Insecure if the number of affirmative responses to the 18 items on the FSS > 2. The PSQ was completed by SEEK parents in the pediatric clinic. The FSS was completed by 205 SEEK parents in the laboratory within 2 weeks of completing the clinic PSQ.

LR− = Negative Likelihood Ratio; LR + = Positive Likelihood Ratio; NPV = Negative Pedictive Value; PPV = Positive Predictive Value.

Screen Effectiveness

Rates of Screening, Assessment, and Action

Before SEEK was implemented, the proportion of families screened for FI was quite low – 14.5% in the SEEK group and 4.5% in the control group (Table 3). After the implementation of SEEK, screening rates went up in both groups – to 87.5% in the SEEK group but only to 11.9% in the control group (p<0.01). Among a subset of families with at least 2 well child visits prior to SEEK, the average rate of screening increased more in the SEEK group relative to the control group (24% vs. 4% increase, p<0.01) (Data not shown).

Table 3.

Rate of Food Insecurity Screening and Follow-Up Action by SEEK and Control Residents Before and During Implementation of the SEEK Model^a

Period	Group	Screened		Screened Positive		Assessed		Problem Found and Action Taken
		N	%	N	% out of totalb	N	% out of totalb	N	% out of totalb
Before	SEEK N=69	10	14.5	4	5.8	3	4.3	3	4.3
	Control N=67	3	4.5	2	3.0	2	3.0	2	3.0
After	SEEK N=296	259	87.5	83	28.0	53	17.9	50	16.9
	Control N=243	29	11.9*	11	4.5*	7	2.9*	7	2.9*

^aData Obtained by Chart Review; All analyses based on original group assignment.

^bPercentages of families where screening, assessment, or action was performed, among all participants whose child’s medical chart was available.

^*P<.01 by chi square test for the difference between SEEK and control groups.

When a screen was positive, the problem was usually assessed, and action was taken (Table 3). Following implementation of SEEK, SEEK parents were significantly more likely to have FI identified and addressed compared to parents in the control group (16.9% vs. 2.9% of all participating families; p<0.01).

Change in Rates of FI Over Time

There was no significant difference in FI rates – by the Lab PSQ or the FSS – between the SEEK and control families either before or after implementing SEEK (Table 4). Of the 223 SEEK parents providing initial and follow-up data, 73 (32.7%) met criteria for FI on the FSS initially, and 66 (29.6%) met criteria 6 months later. Among the 161 control parents, 50 (31.1%) met FI criteria initially, and 48 (29.8%) did so 6 months later.

Table 4.

Rates of Food Insecurity Identified on SEEK PSQ, and on FSS at Initial and Follow-up Assessments^a

	Timing of Assessment	SEEK N (%)	Control N (%)	p-valueb
PSQ N=419c	Initial	88/241 (36.5%)	76/178 (42.7%)	0.2
	6-month follow-up	75/241 (31.1%)	50/178 (28.1%)	0.5
FSS N=384d	Initial	73/223 (32.7%)	50/161 (31.1%)	0.7
	6-month follow-up	66/223 (29.6%)	48/161 (29.8%)	0.9

^aData includes only participants who completed PSQs initially and at the 6-month follow-up; All analyses based on original group assignment.

^bAll p-values are comparisons of SEEK to control group.

^cN=241 for the SEEK group and N=178 for the control group.

^dN=223 for the SEEK group and N=161 for the control group.

FSS = USDA Food Security Scale; PSQ = Parent Screening Questionnaire.

Change in Rate of SNAP Benefits Receipt

123 Families (73 SEEK and 50 control) met criteria for FI at time 1, and 63 (32 SEEK and 31 control) of these families provided information about SNAP benefits receipt at both time periods. There was no significant difference in the percentage receiving SNAP benefits at time 1 (96.9% of SEEK vs. 100% of control). However, while almost all SEEK families maintained their SNAP enrollment, 20 percent of control families were no longer enrolled at 6-month follow-up [31/32 (96.9%) vs. 25/31 (80.6%); p=0.05] (Table 5).

Table 5.

SNAP Benefit Receipt at 6-month Follow-up Among SEEK and Control Families Who Met Criteria for Food Insecurity by FSS at Initial Assessment^a

SNAP Benefit Receipt at 6-month Follow-up	SEEK N (column %)	Control N (column %)
Not Receiving SNAP	1 (3.1%)	6 (19.4%)
Receiving SNAP	31 (96.9%)	25 (80.6%)

^a123 families met criteria for food insecurity at initial assessment. Data on SNAP benefit receipt was provided by 63 parents. P-value by 2-sided Fisher’s Exact test = 0.05.

FSS = USDA Food Security Scale; SNAP = Supplemental Nutrition Assistance Program.

DISCUSSION

Food insecurity is, unfortunately, a common problem, affecting about 1 in 5 children in the U.S. Pediatricians and other child health providers will encounter children and families with food insecurity. However, because most children with FI have normal growth parameters [27,28], they won’t be identified by child health providers unless families are specifically asked about FI.

Our data demonstrate that using a single question screen in a primary care setting can effectively identify families with FI. In addition, the SEEK model, including the PSQ, increased both the rate of screening and the rate of FI intervention when compared to routine care without standardized screening. Our study is not the first study to demonstrate that brief screens can identify FI [24,25]. However, ours is the first to show that screening may help families maintain enrollment in food assistance programs. Unfortunately, this improvement was not enough to lessen FI.

Other authors have developed FI screening questionnaires for the pediatric primary care setting. Kleinman, et al. used a single item FI screen: “In the past month, was there any day when you or anyone in your family went hungry because you did not have enough money for food?” [24]. This question had 83% sensitivity and 80% specificity for identifying food insecure families. Hager and colleagues assessed the validity of a 2-item screen, using the first two questions from the FSS [25]. Respondents completed the screening and gold standard questions in a pediatric clinic. However, the responses were not shared with the primary care provider. An affirmative response to either one of these questions had a sensitivity of 97% and a specificity of 83%. Limiting the screening to the same FSS question that was included in the PSQ provided a sensitivity of 93% and a specificity of 85%. It is possible that the lower sensitivity of our screen was due to our method of validation; we used the actual questionnaire from the clinical setting, where some families may have been reluctant or embarrassed to disclose FI to their child’s doctor. Our positive predictive value of 70% was slightly higher than the 62% positive predictive value in the Hager study, reflecting the lower prevalence of FI in their sample (23% vs. 34%).

Regardless of how well screening tests perform, a vital factor in determining whether to recommend routine screening is whether screening leads to positive outcomes for children and families. Our data demonstrate that systematic screening increases the likelihood that families with FI will be referred for services. The data also suggest that systematic screening can increase the likelihood that families will maintain enrollment in the SNAP benefits program.

Unfortunately, while systematic screening was associated with maintenance of SNAP enrollment, it was not associated with increased food security among participating families. These findings suggest that SNAP benefits may not be adequate to eliminate FI. While a number of studies have shown that government assistance programs can reduce FI and improve child health, [29–32] studies have also indicated that SNAP benefits generally are not adequate to support an entire family’s nutritional needs, [33–35] and may not fully eliminate FI in some families.

An advantage of the SEEK PSQ over other brief FI screens is that it also addresses other prevalent family risk factors, including depression and intimate partner violence [13,18,20]. Because children living in families with intimate partner violence or maternal depression are less likely to receive nutrition assistance, [36] screening for all of these family risk factors may increase the likelihood that food insecure families are able to access services.

There were a number of limitations to our study. First, our FI screen was validated in a clinic with a very high prevalence of both poverty and FI. Therefore, the utility of implementing screening in practices serving primarily middle income families is unclear. This validation study was limited to families with young children; the validity in families with older children has not been established. Because the SEEK families were recruited over an extended period of time, we do not have baseline data to determine whether the two groups had similar rates of food security prior to implementation of the SEEK model.

Although we compared the rates of SNAP benefit receipt in the SEEK and control groups at the end of the SEEK study, it is possible that the maintenance of SNAP enrollment in the SEEK group was not due to SEEK involvement but due to intervention by other service agencies. While 81% of families completed follow-up assessments, and the demographic characteristics of families with and without follow-up were similar, unmeasured biases in retention or questionnaire completion may have led to finding a difference in SNAP enrollment at follow-up when none existed. For example, families with poor organizational skills or multiple stressors would be less likely to enroll in SNAP and also less likely to complete the follow-up assessment.

Despite these limitations, our single item FI screen is a promising addition to well child care, particularly for practices in low-income communities. With more than 12 million children living in food insecure households, [2] FI places a tremendous burden on both individual and societal health and productivity. Furthermore, the lack of progress toward the Healthy People 2020 goal of 6% of households with FI makes trying new approaches all the more important [37]. While most health care providers do not currently screen for FI, both family physicians and pediatricians express willingness to do so [38]. Large scale implementation of the PSQ with an effective single FI screening question could benefit many children and families in the United States.

CONCLUSIONS

Given the prevalence and negative consequences of FI and the ease and potential benefits of screening, we recommend that screening should be a routine part of well child care, particularly for practices serving many low-income families.