Validation of a Parent Report Questionnaire

Abstract

Objective. To evaluate the reliability and validity of the Infant Gastrointestinal Symptom Questionnaire (IGSQ), a tool to assess feeding tolerance in infants. Methods. Qualitative methods were used to develop IGSQ content across 5 symptom clusters, yielding a 13-item index of parent-reported infant digestion and elimination behaviors over the prior 7 days. Classical psychometric methods evaluated factor structure, interrater and retest reliability, and validity in 4 prospective studies of 836 infants. Results. Interrater and retest reliability were acceptable to good. IGSQ Index score was highly correlated (r = 0.89) with daily parent reports. IGSQ scores were significantly different between infants whose parents planned to switch formulas because of perceived feeding problems and those without parental concerns. Conclusions. The IGSQ is a practical, reliable, and valid method for assessment of infant gastrointestinal-related behaviors. Its use in clinical studies can provide empirical evidence to advance parent education regarding both normal and clinically meaningful feeding-related behaviors.

Introduction

Newborn and young infants exhibit many signs of gastrointestinal (GI) functioning that are observed by parents and other caregivers but are difficult to interpret and communicate effectively to pediatricians. Parents often decide that their very young infant is not tolerating feedings, whether human milk or formula, based on signs such as spitting up or stool consistency that may be normal for the immature digestive system. Although the United States has achieved the 2010 Healthy People goal of breast-feeding initiation among 75% of newborns,¹ the majority of breast-fed infants are switched to formula prior to 6 months of age^1,2 usually because of parental concerns about common infant GI behaviors.^3-5 Feeding switches that result from normal GI functioning may pose unnecessary nutritional risk during this influential stage of development.⁶ Although it is important to educate new parents regarding normal infant digestive and elimination-related behaviors,⁷ there is almost no longitudinal data available to guide them. A simple, reliable, and meaningful parental assessment of infants’ GI signs could support such research, potentially informing pediatric care.

To address this need, the Infant Gastrointestinal Symptom Questionnaire (IGSQ) was developed. Its reliability and validity across 4 studies conducted in 3 countries are reported. The final form of the IGSQ is a 13-item interviewer-administered questionnaire that allows parents to describe the frequency and intensity of their infant’s GI signs and symptoms of digestion and elimination for the previous 7 days.

Methods

Development of the IGSQ

Literature Review

Experts in infant GI functioning and disorders, infant nutrition, and instrument development undertook a multiple stage process of instrument development, beginning with a literature review. Several longitudinal studies were identified that were based on physician-supplied data;^4,7,8 no validated feeding tolerance questionnaires were located. Clinical tools for parents to monitor symptoms of gastroesophageal reflux disease (GERD) were identified, including the GERD Symptom Questionnaire,⁹ the Infant Gastroesophageal Reflux Questionnaire,¹⁰ and the Infant Gastroesophageal Reflux Questionnaire–Shortened Form.¹¹ These questionnaires are not useful for characterizing typical GI functioning.

Item Development and Content Validity

The literature review and expert input guided the preliminary structure and possible content of the IGSQ, which included frequency, amount, and consistency of stools, as well as spitting up, vomiting, burping, crying, body tenseness indicating pain, refusal to eat/suck, irritability, fussiness, and flatulence. Parental input was also obtained at this early stage through a series of interviews with 10 English-speaking mothers of very young infants.

Based on these sources of information, more than 30 items were developed, each with 5 response options and a 1-week recall period. Five clusters of signs and symptoms were defined, including stooling, spitting up/vomiting, flatulence/gassiness, crying, and fussiness. Input from pediatric clinical scientists (mainly registered dietitians) was used to modify and eliminate items.

In cognitive debriefing interviews, 5 English-speaking mothers of young infants described what each question meant and whether the questions fully and clearly addressed the range of infant feeding–related behaviors. Parents’ explanations and comments generally supported the meaningfulness and clarity of the items, although several items required wording clarification. A trial version of the IGSQ was developed and translations were made for 2 languages spoken where the studies were conducted (Tagalog for studies 1 and 4 in the Philippines; Mandarin Chinese for study 2 in Shanghai, China). The translations further refined the item wording. For example, “spitting up” and “vomiting” were replaced with the clearer description of “milk coming out of your baby’s mouth.”

The trial version of the IGSQ was administered in 4 studies to a total of 836 parents. Psychometric analyses were conducted as described below to determine which items best discriminated between groups and which best contributed to the total score. These analyses identified the most efficient and discriminating items.

Validation of the IGSQ: Studies 1 to 4

Design, Setting, and Sample

Table 1 summarizes the demographic characteristics of each sample population, which were collected via parent interviews at baseline. Studies 1 and 4 involved a comparison of formula-fed and human milk–fed infants.

Table 1.

Demographic/Study Variables for Studies 1 to 4.

	Study 1	Study 2	Study 3	Study 4
Sample size, N	336	64	61	375
Infant age at baseline, days, mean (SD)	9.6 (2.9)	55.2 (12.3)	75.1 (49.1)	10.9 (2.1)
Gestational age, weeks, mean (SD)	38.5 (0.9)	39.4 (1.0)	38.9 (1.3)	38.7 (1.0)
Child sex, n (%)
Male	168 (50.0)	29 (45.3)	36 (59.0)	186 (49.5)
Female	168 (50.0)	35 (54.7)	25 (41.0)	189 (50.5)
Study location	Philippines	China	USA	Philippines
Infant’s race, n (%)
Asian	336 (100.0)	64 (100.0)	19 (31.2)	375 (100.0)
Black/African American			2 (3.3)
White			35 (57.4)
Mixed race/other			5 (8.2)
Maternal education level, n (%)a
Less than high school	23 (6.9)	14 (21.9)	9 (14.8)	59 (15.7)
Completed high school/vocational school	173 (52.0)	8 (12.5)	16 (26.2)	173 (46.1)
Some postsecondary/college/training	70 (21.0)	15 (23.4)	9 (14.8)	83 (22.1)
Completed college or advanced training	64 (19.2)	27 (42.2)	27 (44.2)	60 (16.0)

^aEducation data missing for 6 mothers in Study 1.

Study 1 involved a convenience sample of mothers with newborn infants recruited from pediatric practices in the Philippines for a 16-week study.¹² Infants were an average of 9.6 days old at baseline IGSQ, with subsequent administrations at 30, 60, 90, and 120 days later.

Study 2 involved a convenience sample of 64 mothers and their healthy full-term infants (40-82 days old at baseline) recruited from pediatric practices in Shanghai, China. This study compared IGSQ scores, which are based on parents’ recall of infant GI signs over the previous week, with a summary of GI signs submitted to researchers on each of the previous 7 days.

Study 3, conducted in the United States, involved 61 parents (87% mothers, 13% fathers) and their infants in pediatric practices in Kentucky or Virginia. Parents of 31 infants were considering switching their infants’ formula because of concerns that their otherwise healthy infants were not tolerating feedings. A comparable 30 parent-infant dyads without feeding concerns were recruited from the same pediatric settings. Family and infant characteristics did not differ between groups.

Study 4, conducted in pediatric practices in the Philippines, involved 300 mothers and their formula-fed infants, with a reference group of 75 exclusively human milk–fed infants. Infants were an average of 10.9 days old when the baseline IGSQ was administered; subsequent administrations were 28 and 56 days later.

IGSQ Index Score

The 13-item IGSQ index score reported here assesses infants’ GI-related signs and symptoms observed by parents over the previous week in 5 domains: stooling, spitting up/vomiting, flatulence, crying, and fussiness. Parents indicated their response after each item was read to them by a trained clinical interviewer. Items were scored on a scale of 1 to 5, with higher values indicating greater GI distress. The total IGSQ score was calculated by summing item responses. Thus, the possible range in scores was 13 to 65, where a score of 13 indicated no GI distress at all and a score of 65 represented extreme GI distress.

Procedures

Standard Data Collection

For each study, the protocol, informed consent form, IGSQ, and planned subject compensation were reviewed and approved by the appropriate internal review board and/or national ethics committee, and appropriate government agencies, in accordance with Good Clinical Practice guidelines. Written informed consent was obtained from the parent/legal guardian of each infant. Research staff trained and oversaw the clinic staff who carried out the study, including the principles and procedures of human subjects’ protection. Parents were interviewed with their infants in a private clinical space for baseline and follow-up visits. Responses were later entered into an electronic database using a highly reliable methodology to ensure accuracy.

IGSQ Validation

Study 1

Mothers were recruited at their infants’ first pediatric visit. Those who agreed to participate were administered the IGSQ and other assessments at baseline and 4 subsequent clinic visits over the 16-week study period. The first 15 mothers were administered the IGSQ a second time by a clinical research coordinator who was not present during the first administration, providing a test of interrater reliability. As this was the first use of the IGSQ in clinical practice, the clinic staff interviewers were debriefed to elicit their perceptions of how well mothers understood the items, how relevant the items were, and whether the efficiency of administration procedures could be improved.

Study 2

Parents were administered the IGSQ in the pediatric office at 2 time points: on day 1 (baseline) and day 9. On days 2 through 8, the parents completed a daily record of their infants’ stools and GI distress on an electronic handheld device in a format that paralleled the IGSQ. The daily records were electronically communicated each evening to the central study site. Parents did not retain a copy of their daily reports. Researchers computed a total score for the week based on daily records.

Study 3

Parents in both pediatric practices were administered the IGSQ once by trained clinic staff.

Study 4

Trained clinic staff administered the IGSQ to mothers at baseline and at 2 subsequent 4-week follow-up assessments.

Data Management and Statistical Analysis

Data management for all studies was performed by independent clinical research organizations. Data collection, data entry, query process, data review, and database lock were performed according to standard procedures for producing highly reliable data in electronic form. Data analysis was conducted using SAS software version 9.1.3 (SAS Institute, Cary, NC, USA). A standard criterion of P < .05 was used to define statistically significant differences.

Descriptive Statistics

For each study, IGSQ total score means and standard deviations (SDs), as well as the percentage of respondents who obtained the lowest possible IGSQ total score (floor) were calculated for salient sub-groups at each time point.

Structure of the IGSQ

Exploratory factor analyses were conducted to determine whether the IGSQ performed better as a single factor, multiple factors (subscales), or as an index (ie, a combination of distinct signs and symptoms that are only modestly correlated). Factor analyses were conducted on the baseline data from each study separately and for all 4 studies combined. Bartlett’s chi-square criteria indicated that between 1 and 5 common factors could be identified, and 1, 2, 3, 4, and 5 factors were extracted and rotated according to varimax (orthogonal) criteria. Each factor solution was evaluated using multiple criteria (eg, satisfaction of scree test, retention of ≥3 items with salient loadings per factor, adequate percent variance explained by factors).

Interrater Reliability

Measures of reliability indicate the extent to which an assessment tool is able to produce a consistent score when no change in the measured characteristic has occurred. Interrater reliability was assessed in study 1 by comparing the IGSQ item responses of 15 mothers who were administered the instrument twice on the same day by independent interviewers.

Retest Reliability

Retest reliability was evaluated in study 2 by correlating IGSQ scores obtained on days 1 and 9. This was a conservative measure of reliability for 2 reasons. First, it is possible that infants’ GI functioning may have matured somewhat between days 1 and 9, thereby reducing day 9 GI distress. Second, parents kept daily records of these same signs between days 2 and 8, thus potentially influencing their second completion of the IGSQ.

Accuracy of Parents’/Caregivers’ Recall

The accuracy of the IGSQ rests on parents’ ability to recall their infants’ GI signs over the past week. In Study 2, parents kept a daily record of the same aspects of GI functioning that are assessed on the IGSQ. Daily scores were electronically sent to researchers and summed for the 7-day period (methodology available from authors). The Pearson correlation between the summary score for the week and the day 9 IGSQ score was computed. The between-instrument correlation was determined to assess the validity of the IGSQ compared with the daily record; correlations between 0.6 and 0.8 are indicative of a strong relationship between 2 measures.¹³

Validity

An instrument’s validity reflects the extent to which it provides a true, accurate assessment of the construct being measured. Validity of the IGSQ was evaluated by comparing scores of infants whose parents thought they were not tolerating their feedings with infants without such parental concerns in study 3. Significant differences in IGSQ scores between these 2 groups would support the instrument’s construct validity.

Results

Descriptive Statistics

Table 2 summarizes the IGSQ 13-item total score means and SDs, as well as the percentage of infants who received the lowest possible score, indicating the absence of GI distress, for relevant subgroups in each study. Although a moderate proportion of the healthy infants had the best possible score (ie, the lowest score reflecting the instrument’s floor effect), this is not problematic as it simply reflects a high proportion of infants at each age without GI distress. No ceiling effects were observed (ie, no infant obtained the highest possible score), which would be problematic for the IGSQ, as such a score would not permit detection of a deteriorating course.

Table 2.

Infant Gastrointestinal Symptom Questionnaire Index Scores by Study, Feeding Group, and Time Point for Data Collection.

	n	Mean	SD	% Floor
Study 1	336
Baseline
Formula-fed	220	17.3	3.3	4.6
Human milk–fed	112	16.3	2.5	7.1
30 days
Formula-fed	215	16.9	3.3	11.6
Human milk–fed	110	15.4	2.1	16.4
60 days
Formula-fed	211	15.3	1.9	26.5
Human milk–fed	110	15.2	1.8	20.9
90 days
Formula-fed	211	15.2	1.9	27.0
Human milk–fed	110	14.4	1.6	46.4
120 days
Formula-fed	222	15.2	2.4	28.8
Human milk–fed	110	14.5	1.5	40.9
Study 2	64
Day 1
Formula-fed	60	23.1	5.3	0.0
Day 9
Formula-fed	61	23.0	6.2	1.6
Study 3	61
Formula-fed intolerant	31	38.3	8.2	3.5
Formula-fed tolerant (comparison group)	30	20.9	5.3	0.0
Study 4	375
Baseline
Formula-fed	318	19.3	3.5	0.0
Human milk–fed only	57	17.7	3.0	1.8
4 weeks
Formula-fed	315	18.7	2.6	0.0
Human milk–fed only	55	18.5	2.1	0.0
8 weeks
Formula-fed	317	18.0	2.0	0.0
Human milk–fed only	55	17.8	1.7	0.0

Structure of the IGSQ

The IGSQ was shown to be a composite index of overall GI distress, such that a higher score indicated a greater burden of GI distress. The factor analyses showed that no overall factor or set of subfactors existed. Internal consistency, as measured by the Cronbach’s alpha coefficient, was 0.72. Multiple dimensions were measured by the IGSQ, and having a high score on one set of items (eg, spitting up) did not mean that an infant had a high level of distress on another.

Reliability

Table 3 provides the reliability summaries, showing the percentage of times there was exact agreement between the 2 assessments.

Table 3.

Interrater and Retest Reliability of Individual Infant Gastrointestinal Symptom Questionnaire (IGSQ) Items^*.

	Interrater Reliabilitya (Study 1; n = 15)	Retest Reliabilityb (Study 2; n = 60)
Times baby passed hard stool in past week	86.7	82.6
Times baby had difficulty passing bowel movement in past week	93.3	71.0
Times milk came out of baby’s mouth on a usual day in past week	26.7	54.8
Amount of milk that usually came out when baby spat up in past week	73.3	59.7
How often did baby seem uncomfortable/ fussy when spitting up in past week	93.3	71.0
How many times baby arched back in pain when spitting up/feeding in past week	93.3	77.4
Amount of time baby usually cried in a day in past week	93.3	50.0
Number of times unable to soothe baby to stop crying in past week	100.0	62.9
Number of times baby cried during or right after feeding in past week	86.7	58.1
Number of days baby was fussy in past week	93.3	40.3
Number of times unable to soothe baby when he/she was fussy in past week	100.0	74.2
Number of times baby passed a lot of gas/was gassy on a usual day in past week	60.0	46.8
Number of times gas seemed to make baby uncomfortable/fussy in past week	100.0	80.7

^aPercentage of responses with exact agreement across 2 same-day administrations of the IGSQ by 2 different interviewers in study 1.

^bPercentage of responses with exact agreement on days 1 and 9 in study 2.

^*Abbreviated item wording.

Interrater Reliability

The percentage of exact item-level agreement between the 2 interviewers who administered the IGSQ to the 15 mothers twice on the same day in study 1 provided an estimate of the interrater reliability. When interviewed by 2 different interviewers, parents gave the same response at least 85% of the time on 10 of the 13 items. This agreement method was used rather than correlational analysis because the low rate of GI distress resulted in low item-level variation, such that the correlations could be spuriously low.¹⁴

Retest Reliability

Table 3 shows exact item agreement in study 2 between parents assessed on days 1 and 9. The Pearson correlation between the day 1 and day 9 IGSQ Index scores was r = 0.69. Moreover, the 2 mean IGSQ Index scores were not significantly different (mean [SD] scores 22.44 [5.56] and 22.62 [6.60], respectively). However, as shown in Table 3, parents’ answers on days 1 and 9 were not always identical. Given that the infants were maturing and the parents were monitoring GI signs daily, this was a conservative test of retest reliability and indicates reasonable overall stability over time.

Accuracy of Parents’/Caregivers’ Recall on the IGSQ Compared With Daily Monitoring Reports

Also in study 2, a very strong correlation, r = 0.89 (P < .01), was observed between the day 9 IGSQ Index score and the summary score computed by the researchers from the daily monitoring records submitted by parents at the end of each of the 7 previous days. This indicated that parents’ overall assessment of the prior week on the IGSQ accurately summarized infants’ GI distress over the prior 7 days (based on the daily records). The item-level Pearson correlations between the daily monitoring summary and the day 9 IGSQ item score ranged from 0.46 to 0.88 (all were statistically significant, P < .001).

Validity

Known Group Comparisons

As predicted, there was a large and statistically significant difference in study 3 IGSQ Index scores for infants whose parents reported formula intolerance and the comparison group (mean [SD] of 38.3 [8.2] and 20.9 [5.3], respectively; P < .0001). Comparison of the item scores for the 13 signs of GI functioning and distress demonstrated that as expected, the “case” group of infants experienced significantly worse scores for all items in 4 areas: gassiness, fussiness, spitting up, and crying. There were no differences in the number of stools per day and number of hard stools, but “difficulty passing a stool” occurred significantly more often among formula-intolerant infants.

Differences between healthy formula-fed and human milk–fed infants were reflected in studies 1 and 2. The items “number of hard stools” and “difficulty passing a stool” indicated that these signs of GI distress occurred about 1.5 times more often for formula-fed infants at baseline, a difference that was maintained at 30 days. However, by 60 days, these differences no longer existed. The other notable difference was that formula-fed infants initially had significantly more gassiness, but by the 30-day assessment this difference was no longer present.

Meaningful Difference in IGSQ Index Scores

Estimating a clinically meaningful score or “cutpoint” is an important but complex determination that cannot be definitively calculated from these studies. However, the likely range can be suggested, using the parent-reported formula-intolerant infant group, whose mean IGSQ Index score was 38. It is likely that even lower scores suggest digestive distress. One “rule of thumb” approach is to use the size of the pooled SD difference between such a clinical “case” group and a healthy group. In study 3, the pooled SD was 6.8 points. Adding this to the highest group score observed for healthy infants (23 points) suggests that a group scoring greater than 30 may have clinically meaningful digestive distress. Applying this logic to the differences between the human milk–fed and formula-fed groups, none would be clinically significant.

Discussion

The IGSQ, a brief parent report of infant GI functioning and signs of distress over the previous week, was shown to be useful in clinical research in 4 studies conducted in 3 countries (the Philippines, the United States, and China). The reliability and validity of the IGSQ was supported, and it was shown to be sensitive to relatively subtle differences between human milk–fed and formula-fed infants, and most important, to differences between infants with and without parent-reported feeding problems. Its effectiveness reflects the extensive formative research that was conducted during its development, which involved pediatric GI experts, clinicians, and parents. Items are single concepts, written as simply as possible, allowing effective translation even into languages syntactically very distinct from English. The Flesch-Kincaid reading level is grade 8.1, indicating that parents are likely to be able to complete the IGSQ on their own, although in these studies it was administered to parents by clinicians. The tool was validated in 3 languages, representing diverse regions of the world, allowing the possibility of its use across different cultures and languages, thereby increasing its potential usage in multicenter trials.

The IGSQ effectively identified infants whose parents were ready to switch formulas because of infant GI distress, as seen in study 3. This group of US infants without known medical problems, whose average age of 75 days suggests that they should no longer be experiencing frequent GI distress, had significantly higher IGSQ Index scores and higher item scores on almost all IGSQ items compared with infants with similar feeding histories and family background, but no parental concerns regarding GI tolerance to feedings. Even in healthy infants with low levels of GI distress, the IGSQ detected an average 2-point difference between healthy human milk–fed and formula-fed infants. This is attributable to slightly higher frequency of hard stools, fussiness, and spitting up in formula-fed infants, which were reduced by the 1-month assessment without any change in formula.

Although the level of digestive distress is generally low in infants at a well-child visit, one of the most important contributions of the IGSQ to routine pediatric practice is likely to be in characterizing the time frame in which common signs of GI distress resolve in healthy infants. Frequent gassiness was identified most commonly, affecting about two-thirds of babies throughout the first 4 months of life. Spitting up and being cranky or fussy were also common, but these persisted only for the first 4 to 5 weeks of life. The IGSQ items can be clinically discriminating. For example, despite the frequency of spitting up, only a very small proportion of infants were also reported to be uncomfortable or fussy when spitting up.

Infant GI distress is one of the most common problems seen in pediatric practice, one that parents often seek to solve by changing their infants’ feedings. Parents report that such changes resolve their infants’ digestive problems,^3,5 yet no controlled research has evaluated such feeding changes. It is equally possible that the change occurred around the time that the infants’ GI systems became sufficiently mature so that digestive trouble was minimal. Major changes in feedings require adjustments of the parents and infant, potentially disrupting nutritional intake. These IGSQ data suggest that by 2 months of age signs of GI distress resolve with maturation, without switching feedings. If parents of otherwise healthy infants knew that their infants’ signs of GI distress are typical and will resolve in a short period of time, they may be less likely to disrupt their infants’ feeding regimens.

Here, the IGSQ has been shown to be an effective tool for assessing infant GI distress in groups of infants in distinctly different cultures. It was also effective in identifying a clinically meaningful difference between groups, and in characterizing changes in the average level of GI problems within a group over time. As this tool was developed and evaluated in patient populations, further research is required to determine its utility in making clinical decisions for individual patients. Nevertheless, the IGSQ may be a useful tool for parents to monitor their infants’ GI signs and may serve as a means to communicate symptomatology with the health care provider. Furthermore, if pediatricians ask concerned parents to use the IGSQ for a brief period of time, they are likely to find it a useful method for demonstrating normal infant GI functioning, as well as the process and timing of problem resolution associated with maturation. Experience with the IGSQ may help in advising new parents about the natural course over which GI distress resolves and help reduce the frequency of switching formulas or from human milk to formula.

As with all research, care should be taken in interpreting these results. Although these studies were carried out in diverse settings with mothers from very distinct cultures, clinical staff were involved in all studies and all studies involved infants younger than 1 year. Use of the IGSQ in clinical research and longitudinal studies is needed to fully characterize “typical” or average scores in healthy infants of specific ages.

In summary, the IGSQ is a scientifically designed and highly credible assessment tool for obtaining parents’ perspectives on the frequency and severity of infants’ GI signs and symptoms. The IGSQ was developed as a research tool with an extensive period of expert, clinical, and parental input into the content, format, and wording of the items. The primary application of the IGSQ is for infant research on tolerance of feedings, identification of infants with persistent and nonnormative patterns of GI distress and problems, and monitoring of group-level interventions designed to reduce GI distress in infants with significant digestive symptoms and distress. Use of the IGSQ in clinical studies can provide valuable empirical evidence that may be useful for educating parents about normal and clinically meaningful feeding-related behaviors.

Author Contributions

AWR contributed to conception and design; analysis and interpretation; drafted manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy. JT contributed to conception; interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy. MY contributed to conception; acquisition and interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy. KBB contributed to design; analysis and interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy. PAD contributed to conception and interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.