The Parent Supervision Attributes Profile Questionnaire: a measure of supervision relevant to children's risk of unintentional injury

Abstract

Objective

To further establish the psychometric properties of the Parent Supervision Attributes Profile Questionnaire (PSAPQ), a questionnaire measure of parent supervision that is relevant to understanding risk of unintentional injury among children 2 through 5 years of age.

Methods

To assess test‐retest reliability, parents completed the PSAPQ twice, with a one month interval. Internal consistency estimates for the PSAPQ were also computed. Confirmatory factor analyses were applied to the data to assess the four factor structure of the instrument by assessing the convergent and divergent validity of the subscales and their respective items.

Results

Test‐retest reliability and internal consistency scores were good, exceeding 0.70 for all subscales. Factor analyses confirmed the hypothesized model—namely that the 29 item questionnaire comprised four unique factors: protectiveness, supervision beliefs, risk tolerance, and fate influences on child safety.

Conclusions

Previous tests comparing the PSAPQ with indices of actual supervision and children's injury history scores revealed good criterion validity. The present assessment of the PSAPQ revealed good reliability (test‐retest reliability, internal consistency) and established the convergent and divergent validity of the four factors. Thus, the PSAPQ has proven to have strong psychometric properties, making it a unique and useful measure for researchers interested in studying links between supervision and young children's risks of unintentional injury.

Unintentional injury poses a serious threat to the health and safety of children. In industrialized nations, unintentional injury is the leading cause of death and hospitalization for children over 1 year of age.¹,² In the United States, for example, children are more likely to die of unintentional injury than of the next nine leading causes combined.³ For toddlers, the majority of these injuries occur in or around the home when their safety is the responsibility of a parent or other caregiver.⁴ There has been considerable speculation that inadequate supervision may be an important contributing factor for understanding childhood injuries.⁵,⁶,⁷,⁸,⁹ In fact, a recent study examining the characteristics of injury deaths among children aged 0–6 years of age in Alaska and Louisiana concluded that inadequate supervision was specifically the most common preventable factor that led to death, accounting for 43% of deaths.¹⁰ Evidence linking supervision directly to child injury risk, however, has proven difficult to obtain, largely because of the methodological challenges in measuring supervision.¹¹,¹²

To date, the methods used to study supervision have included naturalistic observations,¹³,¹⁴ self reports about supervision,¹⁵,¹⁶,¹⁷ and participant event monitoring methods in which caregivers track ongoing supervision in diary records.¹⁸,¹⁹,²⁰,²¹,²² These methods have provided the first direct evidence that inadequate supervision is associated with increased risk of injuries to young children. However, these methods are extremely time consuming and resource draining, making them impractical for wide range application in research on childhood injury. To address the need for an efficient and valid measure that assesses injury risk due to inadequate supervision, we developed a questionnaire, the Parent Supervision Attributes Profile Questionnaire (PSAPQ).²³ Questionnaires have proven to be both reliable and valid measures of many parenting behaviors, attitudes, and beliefs,²⁴,²⁵,²⁶,²⁷ making this type of instrument a promising choice for an efficient measure of supervision.

In developing the PSAPQ, a broad conceptual approach to supervision was adopted in which behaviors as well as attitudes and beliefs relevant to supervision were considered. This type of conceptual approach redirects the measurement focus from situation specific behaviors to more general patterns of supervisory styles and the underlying attributes that give rise to such styles of reacting (for example, attitudes, beliefs, values²⁸,²⁹,³⁰,³¹) and which act to direct and constrain supervision, as well as contribute to cross situational and temporal stability in styles of supervision. Thus, the PSAPQ differs from other self report measures because it samples the underlying attributes giving rise to supervisory behavior, rather than asking respondents to report on supervisory behavior per se; there are numerous examples of the successful application of this measurement approach in diverse areas of psychology including personality³² and parenting.³³,³⁴

To develop items for the PSAPQ, literature on the topic of supervision was surveyed, supervisory behaviors as well as parenting attributes were identified, and questionnaire items were then developed to tap into these various factors. Tests were conducted both with parents, to ensure item comprehensibility, and with other professionals in the field to establish content validity (that is, adequacy of content sampled). To assess the questionnaire's criterion validity, scores on the PSAPQ were examined in relation to naturalistic observations of parent supervision, children's risk taking behaviors, and children's injury history scores.²³ The PSAPQ exhibited good criterion validity: subscales significantly related to observed parental supervision and/or children's injury histories. Thus, the results from this initial study demonstrated that it was possible for parent attributes relevant to supervision and children's risk of injury to be measured reliably using a questionnaire.

Building on these initial findings, the aim of the present study was to further establish the psychometric properties of the instrument by assessing two important aspects of reliability: test‐retest reliability (that is, stability in responding over a specified time interval) and internal consistency reliability (that is, extent to which the items within a subscale measure something in common). In addition, we also sought to confirm the factor structure of the measure by assessing the construct validity for the subscales by examining if the four subscales showed adequate convergent validity (that is, that the items adequately related to their specific subscale) and divergent validity (that is, that each subscale contributed uniquely to the measurement of supervision). Confirmatory factor analyses (CFA) procedures were used to assess convergent and divergent validity³⁵ and determine if the model we specified a priori was supported by the data based on examination of the covariance structure.³⁶

Methods

Participants

The sample consisted of 192 parents of children aged 2 through 5 years; children were developing normally, as reported by parents, with no known or obvious mental or physical disabilities. There was a good distribution of child ages represented in the sample, including 122 parents having a boy or girl 24–47 months of age (mean 33.5 (SD 7.4) and 34.6 (7.1) months, respectively) and 70 parents of a boy or girl 48–59 months of age (mean 54.3 (SD 7.1) and 56.5 (SD 7.9) months, respectively). The highest level of education of parents included: 10% completed some or graduated high school, 71% completed some or graduated college or university, and 19% completed some graduate training or obtained a graduate degree. The annual gross family income for participants included: 3% reporting less than $20,000; 13% reporting $20,000 to $39,000; 30% reporting $40,000 to $59,000; 25% reporting $60,000 to $79,000; and 28% reporting above $80,000. The remaining 2% of participants did not report their income. Nearly all parents were white and all spoke English as their first language. The study was reviewed and approved by the university research ethics committee and all parents granted consent for their data to be included.

Materials

Initial development and establishment of content validity of the PSAPQ included a thorough literature review to identify dimensions of parenting that seemed relevant to supervision (for example, protectiveness, risk tolerance), feedback from child development specialists about subscales and item content that tapped into these various subscales,³⁵ and preliminary testing with parents to confirm comprehensibility of items and the response format.²³ An initial test of criterion validity involved comparing PSAPQ scores with actual measures of supervision and with children's injury history scores.²³ Based on these results, the PSAPQ was reduced to four factors that yielded adequate internal consistency (>0.70) and related to actual supervision and children's injury history scores. These four factors comprise 29 items (listed in table 1) tapping protectiveness (nine items), supervision beliefs (nine items), tolerance for children's risk taking (eight items), and extent of belief in fate as the primary determinant of children's safety (three items). These items were randomly ordered and presented to parents in the present study. A five point scale (1 = strongly disagree to 5 = strongly agree) was used in judging each statement.

Table 1 PSAPQ factor and parcel structure by item including factor scores.

Factor	Parcel	Factor score
Protectiveness
	Parcel 1	0.75
	I feel very protective of my child
	I think of all the dangerous things that could happen
	I keep my child from playing rough games or doing things where he/she might get hurt
	Parcel 2	0.71
	I make him/her keep away from anything that could be dangerous
	I feel fearful that something might happen to my child
	I warn him/her about things that could be dangerous
	Parcel 3	0.67
	I keep an eye on my child's face to see how he/she is doing
	I feel a strong sense of responsibility
	I try things with my child before leaving him/her to do them on his/her own
Supervision
	Parcel 1	0.83
	I have my child within arm's reach at all times
	I know exactly what my child is doing
	I can trust my child to play by himself/herself without constant supervision
	Parcel 2	0.84
	I stay within reach of my child when he/she is playing on the equipment
	I keep a close watch on my child
	I say to myself that I can trust him/her to play safely
	Parcel 3	0.64
	I stay close enough to my child that I can get to him/her quickly
	I hover next to my child
	I make sure I know where my child is and what he/she is doing
Risk tolerance
	Parcel 1	0.70
	I encourage my child to try new things
	I let him/her learn from his/her own mishaps
	Parcel 2	0.87
	I let my child take some chances in what he/she does
	I let my child do things for him/herself
	I let my child experience minor mishaps if what he is doing is lots of fun
	Parcel 3	0.74
	I let my child make decisions for himself/herself
	I encourage my child to take risks if it means having fun during play
	I wait to see if he/she can do things on his/her own before I get involved
Fate
	Items
	When my child gets injured it is due to bad luck	0.61
	Whether or not my child gets injured is largely a matter of fate	0.71
	Good fortune plays a big part in determining whether or not my child gets injured	0.92

Procedure

Using the same procedure as outlined previously,²³ parents with young children were approached in public parks and asked to complete the PSAPQ; other questionnaire measures were also completed but will not be reported herein. For approximately 36% of the sample (n = 70), parents repeated completion of these measures one month later via the mail to assess reliability of responding (mean (SD 3.57) 3.98 weeks).

Results

Reliability

Test‐retest reliability

Test‐retest reliability was assessed using Pearson correlations and was found to be acceptable for each subscale (supervision, r(72) = 0.76, p<0.001; protectiveness, r(72) = 0.72, p<0.001; risk tolerance, r(72) = 0.76, p<0.001, and fate, r(72) = 0.80, p<0.001) over the one month interval.

Internal consistency

Internal consistency was assessed using Cronbach's alpha and was good for all four subscales: supervision (α = 0.77), protectiveness (α = 0.78), fate (α = 0.78), and risk tolerance (α = 0.79).

Testing for construct validity preliminary procedures

Before applying CFA procedures to assess for construct validity, individual items from protectiveness, supervision, and risk tolerance were combined into 2–3 item “parcels”; the creation of item parcels is a recommended practice in preparing for CFA.³⁸,³⁹,⁴⁰ Item parcels are calculated by summing responses to small groupings of items within each subscale, with each item assigned to only one parcel. These aggregates, rather than individual items, function as indicator variables in the CFA model. This practice has been shown to offer a number of advantages over the use of individual items.³⁵,³⁶ For example, item parcels improve the sample size to parameters ratio by reducing the number of observed variables in the model. They also tend to minimize multivariate normality violations and reduce the influence of idiosyncratic response tendencies to individual items.³⁵,³⁷ The statistical approach recommended by Russell et al³⁸ was implemented in the construction of item parcels. Specifically, all items from one subscale were entered into an exploratory factor analysis in which only one factor was extracted. Items were then ranked according to their loadings on this factor from highest to lowest. Groups of two or three items were summed according to their factor loadings so that the mean difference between parcels was minimized. For example one high loading item was combined with one medium and one lower loading item for each parcel. This helps to ensure that latent variables have structurally equivalent indicators (see table 1 for how specific items were parcelled). Because the fate subscale has only three items, parcelling would not have been appropriate and individual items were therefore used as indicators.

Confirmatory factor analysis

Confirmatory factor analysis was conducted using AMOS 4 to test the construct validity of the PSAPQ. Of particular interest was the convergent and discriminant validity of the subscales. In interpreting the results, cutoff values of >0.90 for goodness of fit index (GFI), >0.95 for the comparative fit index (CFI), and <0.08 for the standardized root mean square error of approximation (RMSEA) were employed, as recommended.³⁶,⁴¹,⁴² Results of the CFA, using maximum likelihood estimation, revealed a good model fit by all indicators. The GFI was 0.93, the CFI was 0.96, and the RMSEA was 0.06. In addition, although the χ² was significant (48, n = 192) = 85.78, p<0.005, this statistic has been shown to be overly sensitive in studies having large sample sizes, such as in the present case. When large samples are used, therefore, the ratio of χ² to degrees of freedom, which reduces sensitivity to sample size effects, is recommended as a more appropriate indicator of model fit.³⁶,⁴¹,⁴² This value was 1.79; values less than 3 are considered acceptable.³⁶,⁴¹,⁴² Thus, by all indicators, the data showed a good fit to our proposed four factor model.

Table 1 shows the factor scores for each indicator variable. Note that all factor scores were reasonably high and significant at p<0.001. The fact that all observed variables load heavily on their factors is indicative of convergent validity. Moreover, as shown in figure 1, correlations among factors were within the acceptable range,³⁶ showing that the factors are significantly distinct and therefore representative of different underlying constructs. Thus, the factors display a reasonably high level of discriminant validity. The highest correlation between factors was between protectiveness and supervision r(192) = 0.62, p<0.001 which was expected given that both constructs tap into a parental motivation toward harm reduction. Risk tolerance was negatively correlated with both supervision, r(192) = −0.55, p<0.001, and protectiveness r(192) = −0.37, p<0.001. Supervision was also negatively correlated with fate r(192) = −0.21, p<0.05.

Figure 1 Confirmatory factor analyses results testing the four factor model of the PSAPQ, including protectiveness, supervision beliefs, risk tolerance, and fate beliefs.

Discussion

Previous research comparing PSAPQ scores with indices of actual supervision and children's injury history scores provided evidence of both construct and predictive validity.²³ The present study provides further evidence of the psychometric soundness of this new instrument, yielding information that confirms additional aspects of reliability and validity for this new instrument. First, both test‐retest reliability and internal consistency reliability were good (>0.70). Second, results confirmed that the factor structure (that is, item groupings) of the PSAPQ exhibited both convergent and discriminant validity, providing support for the four factor structure of this measure. Evidence of convergent validity for the subscales is essential and lends strong support to the notion that theoretical constructs related to supervision and injury risk can be effectively measured via a self report questionnaire instrument. Also of critical importance, evidence of divergent validity substantiates that the subscales measure independent constructs that uniquely index supervision and contribute to explain child injury risk. Thus, the hypothesized model of core attributes that contribute to caregiver supervision and relate to child injury risk was confirmed in the present test of the PSAPQ.

The pattern of correlations between the four factors of the PSAPQ confirms the unique contribution of each caregiver attribute to understanding the relation between injury risk and supervision, but also provides insights into how these attributes interrelate. The most pronounced relation is the positive one between protectiveness and supervision attributes, which is perhaps not surprising given studies have linked both higher levels of supervision¹⁷,¹⁹ and higher levels of protectiveness¹⁸,²⁰ to lower risk of child injury. Although no studies have directly linked how caregivers supervise to measures of protectiveness, the positive relation between these attributes obtained in the present study suggest that such links likely exist. The negative correlations between the fate construct and the protectiveness and supervision constructs are consistent with previous research. To date, two studies have shown that parents who believe that their children's health and safety was predominantly a matter of luck or fate had children with a history of more frequent injuries than parents who believed that they could exercise greater control over their children's health and safety.¹⁸,⁴³ Clearly, whether or not parents view themselves as having control over their child's safety has implications for their child's risk of injury. The present findings extend this model, however, by revealing that parents who score highly on the fate construct score lower on the protectiveness and supervision constructs on the PSAPQ. Thus, the effect of high fate beliefs on child injury risk may be realized via decreased supervision.

Key points

• Caregiver supervision contributes to children's risk of injury but is difficult to measure in reliable, valid, and efficient ways.

• The Parent Supervision Attributes Profile Questionnaire, a measure of supervision that comprises four factors, shows good reliability and validity.

• This measure may help to identify those children at risk of injury resulting from inadequate supervision by caregivers.

The negative correlations between risk tolerance and both protectiveness and supervision constructs is also congruent with existing research. A number of studies have shown that children who engage in more risk taking behaviors tend to experience more injuries.⁴³,⁴⁴,⁴⁵ It makes sense that parents who are tolerant of such risk taking would also be less likely to supervise closely or be very protective of their children. Thus, the profile of correlations between attributes on the PSAPQ are quite consistent with existing literature, lending further support to the four attribute model tested in the present study.

In summary, the findings of the present study add to the accumulating evidence that the PSAPQ is a psychometrically sound measure of caregiver supervision that has relevance for child injury risk. The pattern of associations between constructs of the PSAPQ are congruent with data from a number of sources, which lends strong support to the PSAPQ being a reliable measure of supervision related beliefs and behaviors relevant to childhood injury risk. Thus, the evidence to date indicates that this measure provides a valid, reliable, and efficient means of assessing caregiver supervision. In ongoing research, we are seeking to develop norms and cut off scores on the PSAPQ that can aid in the identification of children at risk of injury as a result of parental supervision practices, and to develop a version of the measure that is applicable to older children 6–12 years of age.

Abbreviations

CFA - confirmatory factor analyses

CFI - comparative fit index

GFI - goodness of fit index

PSAPQ - Parent Supervision Attributes Profile Questionnaire

RMSEA - root mean square error of approximation