Exploring the Psychometric Properties of the Parent Daily Report – Toddler Version (PDR-T)

Abstract

The present study examined the psychometric properties of a brief parent-report daily checklist of toddler behavior (Parent Daily Report – Toddler Version; PDR-T). Data were collected from three groups of 18-36 month-olds who were followed longitudinally for approximately 1 year: 1) internationally adopted children (n = 156), 2) children placed in foster care due to child maltreatment (n = 79), and 3) community comparison children raised by their biological families (n = 80). An exploratory factor analysis of this measure resulted in three factors, measuring aggressive/noncompliant, positive, and distress behaviors. While there were estimation issues with the positive and distress factors, the aggressive/noncompliant factor exhibited invariance across time and groups, and partial invariance between genders. Significant correlations were observed between this factor and measures of externalizing behavior and inhibitory control (r = .26-.56), but not shyness, fearfulness, or negative affect. This provides support for both convergent and discriminant validity. Reliability of this factor was adequate to good across time and group. Results provide preliminary support for the utility, reliability, and consistency of one factor of the PDR-T as an easy parent-report tool to assess daily patterns and changes in child aggressive/noncompliant behavior over time.

1. Introduction

1.1. The Importance of Assessing Dimensions of Child Behavior

Child problem behavior has long been a focus of psychological research. Efforts to categorize specific domains of problem behavior have proven important for guiding clinical decisions and research, and the assessment of problem behavior is critical for detecting the need for early intervention and documenting trajectories of change (Caspi et al., 2008; Forgatch & DeGarmo, 1999). Over the last several decades, a variety of assessment methods have been developed. In a recent review, Achenbach et al. (2016) reported that 49 different measures have been used to measure problem behavior. Yet, just six of these instruments were determined to be methodologically sound—measuring a broad spectrum of problem behavior and demonstrating reliability and validity with large samples of children. In addition, issues about reporting bias and intensity of resources needed to complete behavioral assessment measures (e.g., time spent administering and coding observations and clinical interviews) have been cited (Chamberlain and Reid, 1987). The availability of assessments such as the Strength and Difficulties Questionnaire (SDQ) and the Brief Infant Toddler Socioemotional Assessment (BITSEA), which provide positive and negative scale scores, highlight the importance of and interest in capturing not only the presence of negative behavior, but also delays in competence and/or positive adjustment (Briggs-Gowan, Carter, Irwin, Wachtel & Cicchetti, 2004; Goodman, 1997). The ability to capture the presence or absence of positive behavior may be critical to studies examining protective and risk factors related to resiliency as well as studies exploring the impact of prevention and intervention programs on child outcomes. Thus, the present investigation provides a psychometric analysis of a toddler version of a parent-report daily checklist of child behavior. Variations of this checklist have been used in numerous studies with older children (Chamberlain & Reid, 1987; 1994; Chamberlain, Price, Reid, Landsverk, Fisher, & Stoolmiller, 2006; Patterson, Chamberlain, & Reid, 1982); however, there is a dearth of psychometric information available for the toddler version. Furthermore, there is a general need for reliable, brief measures of behavior at this stage of development.

1.2. History and Development of the Parent Daily Report - Toddler Version (PDR-T)

The PDR-T (Kavanagh, 1986), the focus of the present investigation, is an adaption of the Parent Daily Report (PDR) and which was originally developed at the Oregon Social Learning Center as an assessment of behavioral disturbances in school-aged children (Patterson, 1964). Parents were asked to indicate whether or not their children exhibited certain behaviors (e.g., arguing, defiance, noisiness) to an extent that warrant changing over the last 24 hours. Endorsed items were then summed to yield a ‘total behavior’ score for that day (Reid and Chamberlain, 1987). The assessment was often completed on multiple, subsequent days to better represent behavioral patterns. The PDR has proven to be particularly beneficial due to its sensitivity to change over time, as reported in several intervention studies (e.g., Chamberlain & Reid, 1994; Patterson, Chamberlain, & Reid, 1982; Price, Roesch, Walsh, and Landsverk, 2015; Schindler, Fisher, and Shonkoff, 2017; Schoorl, van Rijn, de Wied, van Goozen, and Swaab, 2017; Urtesky, Lee, Greeno, and Barth, 2017).

Additionally, the PDR addresses the issue of recall bias common with other parent-report measures of child behavior that require parents to recall behavior over long periods of time. The PDR is less time and resource intensive than observational measures and may detect behaviors otherwise unseen during observational sessions (Chamberlain and Reid, 1987; Chamberlain et al., 2006). For these reasons, the PDR has been used in numerous studies with diverse populations, including children referred for behavioral concerns and youth in foster care (Chamberlain et al., 2006; Fisher et al., 2011; Martin, Kim, and Fisher, 2016; Leve et al., 2005; Price et al., 2015; Schoorl et al., 2017).

Similarly, a toddler parent daily report (PDR-T) was created to gather information on child behavior for children younger than 3 years of age. Fagot and Kavanagh (1990) asked parents of 2-year-olds to indicate the three most appropriate/positive behaviors and three most inappropriate/negative behaviors exhibited by their children. The 14 most commonly reported positive behaviors (e.g., play, good manners, expression of affection) and 14 most frequently indicated negative behaviors (e.g., aggression, noncompliance, withdrawal/depressive behaviors) formed the PDR-T (see Supplemental Materials for the full measure) which is typically completed independently by parents as a brief same-day paper checklist. Despite its strengths, the factor structure of the PDR-T has yet to be rigorously tested and its psychometric properties have not been examined. While the items are described as “positive” or “negative”, it is possible that items cluster in more specific categories, which could alter the dimensionality of child behavior in studies using the PDR-T. For example, rather than having one “negative behavior” factor, there might be separable “withdrawal” and “aggression” factors. It is also possible that, for example, the withdrawal items would negatively load onto a “social engagement” factor, along with positive items. Furthermore, it is important to demonstrate invariance across time, gender, and different populations to ensure that this checklist can be used to model change over time and compare problem behaviors across different groups. For example, behaviors exhibited by typically-developing children may be drastically different than those of children who experienced psychosocial deprivation in the form of institutional care (Merz & McCall, 2010), which may affect the psychometric properties of measures assessing child behavior. Similarly, some studies have used the PDR as an indicator of change over time following intervention (e.g., Schoorl et al., 2017; Urtesky et al., 2017). The PDR-T is designed to measure behaviors in toddlerhood, which is a time of dramatic developmental change. Therefore, it will be important to demonstrate that the PDR-T operates similarly across this developmental period (approximately 18-48 months) to ensure that it can be used consistently in longitudinal and intervention studies that span this age range.

To date the PDR-T has been used infrequently (exceptions include: Eddy, Leve, & Fagot, 2001; Fagot & Pears, 1996) relative to the PDR for older children.The present investigation sought to demonstrate the psychometric properties of the PDR-T with the hope to promote its increased use in future studies. Considering the importance of assessing child behavior in the psychological literature, as well as the aforementioned advantages of the PDR compared with currently established measures, the PDR-T may provide a feasible, psychometrically sound alternative to clinical interviews, lengthy questionnaires, and observational measures. It was hypothesized that both positive and negative factors of child behavior will emerge consistent with the 14 positive and 14 negative items that are included in the measure. Furthermore, it was hypothesized that such factors will be associated with similar measures of child behavior, including the BITSEA, Child Behavior Checklist (CBCL), and Early Childhood Behavior Questionnaire (ECBQ)/Childhood Behavior Questionnaire (CBQ). Notably, greater understanding of the psychometric properties of the PDR-T may allow for future investigations of problem behaviors across different developmental phases in conjunction with the more-researched PDR utilized with older children.

2. Material and Methods

2.1. Participants

2.1.1. Purpose of the two studies

The three groups of children included in the present investigation were combined from two parallel studies supported by an NIMH Center grant designed to elucidate the effects of early environmental adversity on important child developmental outcomes and investigate the potential to ameliorate these effects after being placed in more supportive environments. Specifically, these studies explored the impact of early adversity on the development of 1) children adopted internationally from institutions or foster care into American families and 2) American children recently placed in foster care due to child maltreatment. Both internationally adopted children and children placed in foster care tend to show more problem behaviors (Koss et al., 2016; Merz & McCall, 2010; Leve et al., 2012) and lower social competence (Pears, Fisher, Bruce, Kim, & Yoerger, 2010; Pitula et al., 2017). Thus, it is important to be able to accurately and repeatedly assess behavioral development in these populations over time using a method like the PDR-T that is not time- or resource-intensive. Toddlers in the present study were expected to provide a broad range of behaviors allowing for a thorough analysis of the psychometric properties of the PDR-T.

2.1.2. Internationally adopted, foster care, and community comparison groups

The first group consisted of children who were internationally adopted out of institutions or foster care into well-resourced American families between 16-36 months (IA; M age at adoption = 20.1, SD = 8.8; n = 156). The second group consisted of children placed in foster care due to maltreatment between 17-36 months (FC; n = 79). The third group included community comparison children raised in their biological families, with a wide range in family income level (CC, n = 80). This group was recruited to be similar in age to the IA and FC groups (M age = 26.8 months, SD = 5.7). Demographic information for all three groups can be found in Table 1.

Table 1.

Demographic information for internationally adopted, foster care, and community comparison groups.

	Internationally Adopted	Foster Care	Community Comparison
Sample size n (% female)	156 (48.7% F)	79 (41.8% F)	80 (56.3% F)
Age at first assessment M(SD)	28.9 mos (5.9)	27.1 mos (5.3)	26.8 mos (5.7)
Income n (%)
< $50,000	6 (4.0%)	46 (59.7%)	34 (43.0%)
$50,000-100,000	58 (38.4%)	27 (35.1%)	22 (27.8%)
$100,000+	87 (57.6%)	4 (5.2%)	23 (29.1%)
Race %
Asian	78 (50.0%)	—	2 (2.5%)
African/Black	33 (21.2%)	5 (6.3%)	4 (5.0%)
Caucasian	22 (14.1%)	45 (57.0%)	58 (72.5%)
Latin American/American Indian	14 (9.0%)	14 (17.7%)	6 (7.5%)
Mixed Race/Other/Unknown	9 (5.7%)	15 (19.0%)	10 (12.5%)

2.2. Procedure

Each group was assessed three times over approximately 1 year. The first assessment occurred at 18-36 months of age (1-2 months post-adoption/placement for IA and FC groups). The second and third assessments occurred at 6-8 month intervals after the first assessment. At all three assessments, the PDR-T was completed by the parent in the home for three consecutive days, resulting in 9 total possible observations for each child. All other questionnaires were collected in a laboratory session by parent report as close as possible to the home assessment, generally within a couple of weeks (Median = 12-16 days). All parent-report data were provided by the primary caregiver, the vast majority of which were female. For a summary of which measures were collected at which assessments for each group, see the footnote below the regression results in Table 5.

Table 5.

Regression results predicting PDR-T factor from concurrent BITSEA, ECBQ/CBQ, and CBCL scales.

	Assessment
	1	2	3
Brief Infant-Toddler Social Emotional Assessment
Problems (n1 = 277, n2 = 281, n3 = 173)	0.38*** [0.37,0.39]	0.33*** [0.33,0.34]	0.31*** [0.30,0.32]
Competence (n1 = 280, n2 = 286, n3 = 181)	−0.15** [−0.16,−0.14]	−0.18** [−0.19,−0.17]	0.03 [0.01,0.05]
(Early) Childhood Behavior Questionnaire
Fearfulness (n1 = 271, n2 = 278, n3 = 255)	0.03 [−0.02,0.07]	0.02 [−0.02,0.06]	0.09 [0.05,0.12]
Shyness (n1 = 269, n2 = 277, n3 = 260)	0.11* [0.07,0.14]	0.02 [−0.01,0.05]	0.06 [0.03,0.10]
Inhibitory Control+ (n1 = 98, n2 = 86, n3 = 77)	−0.36*** [−0.41,−0.31]	−0.24* [−0.30,−0.18]	−0.30** [−0.38,−0.23]
Cuddliness+ (n1 = 98, n2 = 81, n3 = 79)	0.00 [−0.08,0.08]	−0.06 [−0.12,−0.00]	−0.10 [−0.19,−0.00]
Effortful Control+ (n1 = 101, n2 = 87, n3 = 76)	−0.26** [−0.35,−0.17]	−0.14 [−0.22,−0.05]	−0.22 [−0.32,−0.12]
Negative Affectivity+ (n1 = 96, n2 = 85, n3 = 66)	0.06 [−0.03,0.15]	0.16 [0.06,0.27]	0.25* [0.15,0.36]
Child Behavior Checklist
Externalizing+ (n1 = 103, n2 = 90, n3 = 77)	0.39*** [0.39,0.40]	0.44*** [0.44,0.45]	0.54*** [0.53,0.54]
Internalizing+ (n1 = 101, n2 = 88, n3 = 75)	0.17* [0.16,0.17]	0.31** [0.30,0.32]	0.45*** [0.44,0.46]

Note.

^*p < .05,

^**p < .01,

^***p < 0.001.

⁺Only collected for the foster care group and a portion of the community comparison group. Results are standardized β estimates with 95% confidence intervals in brackets. Data at the third assessment for the foster care group and a portion of the community comparison group include the corresponding scales of the Childhood Behavior Questionnaire in place of the Early Childhood Behavior Questionnaire. Sample sizes for each regression are indicated in each row for Assessments 1-3 (n₁, n₂, n₃, respectively).

2.3. Measures

2.3.1. Parent Daily Report – Toddler Version (PDR-T)

The Parent Daily Report – Toddler Version (PDR-T) is a checklist completed independently by the primary caregiver about behaviors the child exhibited on that same day. The PDR-T is comprised of 28 yes/no questions asking about specific positive or negative behaviors (14 items each; e.g., “Did your child share or get along with others?”, “Was your child angry/threw temper tantrums?”). The IA group and a portion of the CC group also included a “too young” option in case a parent felt the child was too young to express a particular behavior (e.g., 16-month-olds at their first assessment might be too young to show good manners – item #22). For the purposes of the current analyses, this option was recoded as “no” to be consistent with the data from the rest of the sample. The PDR-T also gave parents an opportunity to list other positive or negative behaviors that their children displayed that day. Additionally, parents were asked about strategies they used in response to the behaviors listed and the perceived effectiveness of those strategies. However, the present paper focused on the 28 items regarding child behavior.

2.3.2. Brief Infant/Toddler Social Emotional Assessment (BITSEA)

The BITSEA (Briggs-Gowan et al., 2004) was created to screen for socioemotional and behavioral problems and measure socioemotional competence in very young children. The two global scales of problems (26 items) and competence (9 items) were used to assess convergent validity of the factors extracted from the PDR-T in the present analyses. Answers were given on a 3-point scale (0 = not true/rarely, 1 = somewhat true/sometimes, 2 = very true/often). The BITSEA problems scale showed adequate to good reliability across time (α = .66-,85). Similarly, the BITSEA competence scale showed adequate to good reliability across time (α = .60-,79).

2.3.3. Early Childhood Behavior Questionnaire (ECBQ) and Childhood Behavior Questionnaire (CBQ)

The ECBQ was created as a downward extension of the CBQ (Putnam, Gartstein, & Rothbart, 2006; Rothbart, Ahadi, Hershey, & Fisher, 2001) – both of which measure multiple aspects of child temperament and include 18 subscales and three broad dimensions of temperament: Extraversion/Surgency, Negative Affectivity, and Effortful Control. The broad dimensions of Negative Affectivity (39 items) and Effortful Control (26 items), as well as the subscales of fearfulness (7 items), shyness (4 items), inhibitory control (5 items), and cuddliness (5 items) were used to assess convergent and discriminant validity of the factors extracted from the PDR-T in the present analyses. Answers were given on a 7-point scale (1 = never, 4 = about half the time, 7 = always) regarding behavior over the last 2 weeks. All of the above scales were collected for the FC group and a portion of the CC group; only the subscales of fearfulness and shyness were collected for the IA group and the rest of the CC group. Additionally, for the FC group and a portion of the CC group, the CBQ was used at the third assessment to reflect the fact that most of the children had moved into the target age range of the CBQ instead of the ECBQ.

The shyness scale showed good reliability across time (α = .69-,78) and the fearfulness scale showed poor to good reliability across time (α = .47-,75). The inhibitory control and cuddliness scales showed adequate to good reliability across time (inhibitory control: α = .55-.79, cuddliness: α = .59-,85). The broad domains of Effortful Control and Negative Affectivity showed adequate to excellent reliability (Effortful Control: α = .70-,91, Negative Affectivity: α = .58-,86).

2.3.4. Child Behavior Checklist – Ages 1½ – 5 years (CBCL)

The CBCL (Achenbach & Ruffle, 2000) was created as a standardized checklist for parents to report their children’s behavioral and emotional problems. The CBCL was only collected with the FC group and a portion of the CC group, and the broad dimensions of externalizing and internalizing problems (24 and 36 items, respectively) were used to assess convergent and discriminant validity of the factors extracted from the PDR-T in the present analyses. Answers were given on a 3-point scale (0 = not true/rarely, 1 = somewhat true/sometimes, 2 = very true/often). These dimensions demonstrated good to excellent reliability across time (externalizing: α = .91-,93, internalizing: α = .86-,88).

2.4. Data analysis plan

Initially, 11 items of the PDR-T were excluded from analyses due to unclear wording (2) or lack of variance (9; < 5% or > 95% endorsement of the item). Of the remaining items, several showed reduced variability across the three days of reporting such that caregivers typically endorsed an item on either all three days or none of the days, with a small number endorsing the item on one or two out of the three days. Item-level data across the three days at each assessment time were moderately correlated, with 92% of the tetrachoric correlation coefficients ranging from .35 – .89 (the majority between .60-.80). Thus, item-level data for each assessment time were collapsed across the three days to create a new dichotomous variable for each item where 0 = the behavior was never displayed at this assessment time and 1 = behavior was displayed on at least one day of reporting at this assessment time. We used exploratory factor analysis (EFA) to assess the dimensionality and extract scales from the PDR-T because, while several studies have been published using the PDR (e.g., Chamberlain et al., 2006; Fisher et al., 2011; Leve et al., 2005), very few studies have been published using the PDR-T (Eddy et al., 2001; Fagot & Pears, 1996; Kavanagh, 1986) and none of them included a rigorous testing of the measure’s factor structure and dimensionality. Therefore, there was no predetermined a priori factor structure to test and the measure was not created with an explicit factor structure in mind. The EFA was performed on the combined three groups with data from the second assessment to ensure that the IA and FC children had been in their families for several months but avoid a reduction in sample size due to attrition at the third assessment. This categorical EFA was performed in R (Version 3.3.1; R Core Team, 2017) using the “fa” function in the ‘psych’ package (Version 1.6.6; Revelle, 2017) with a tetrachoric correlation matrix. Items with loadings equal to or greater than 0.40 were considered for inclusion in a factor, with the goal of achieving a simple structure that did not show frequent cross-loading of items. Coefficient alphas were calculated for all factors extracted.

We then examined measurement invariance (Liu et al., 2017) of the resulting factors across time, group, and gender using the ‘lavaan’ package (Version 0.5-23.109; Rosseel, 2012). Two multiple-group longitudinal models were fit as described in Little (2013): one to simultaneously assess invariance across time and group and another to assess invariance across time and gender. The configural model was fit according to guidelines described in Millsap & Yun-Tein (2004) for model identification for dichotomous data and the null model for calculating fit statistics was estimated according to Liu et al. (2017). All models shown in the tables below reflect comparisons against this null model. With dichotomous data, the invariance model constraining thresholds to be equal is equivalent to the configural model because there is only one threshold per item (Wu & Estabrook, 2016) and therefore these models could not be compared. Thus, invariance was examined systematically using the ‘semTools’ package (Version 0.5-1, Jorgensen et al., 2018) by constraining loadings, intercepts, and then residuals to be equivalent and comparing each of these models to the previous model. If these models have adequate fit (TLI and CFI > .90, RMSEA < .08) and do not show a significant decrease in fit compared to the configural model, measurement invariance was concluded.

We also assessed convergent and discriminant validity using scales from other questionnaires administered during these two studies: the BITSEA, ECBQ/CBQ, and CBCL described above. The latent factor was regressed on each questionnaire scale at all available assessment times. To conclude that the PDR-T is useful it must have a logical, theoretically justifiable factor structure and show measurement invariance across time and gender. The investigation of invariance across groups (IA, FC, CC) was considered to be exploratory.

3. Results

3.1. Exploratory Factor Analysis

We examined one- to four-factor solutions using a geomin rotation (an oblique rotation) based on scree plot results showing a notable drop off in eigenvalues after the third factor. The EFA conducted on the second assessment for all three groups showed that the three-factor solution fit the data best (see Table 2). The three-factor solution explained 60% of the item-level variance and corroborated the scree plot. The items loading on the first factor include breaking household rules, hitting others, and throwing tantrums, suggesting that this factor measures aggressive and noncompliant behavior (9 items, see Table 2, Factor 1). The items loading on the second factor, referred to as the positive factor, include exploration, social behaviors, and cheerfulness (5 items, see Table 2, Factor 2). The items loading on the third factor (3 items) suggest that it is measuring distress or “fussiness” (see Table 2, Factor 3). The coefficient alphas were 0.70, 0.94, and 0.43 for the aggressive/noncompliant, positive, and distress factors respectively.

Table 2.

Exploratory Factor Analysis of the PDR-T at the second assessment across all groups.

	Factor 1 (Aggression/Noncompliance)	Factor 2 (Positivity)	Factor 3 (Distress)
1. Touch things they shouldn’t	0.67	0.16	0.06
4. Break household rules	0.73	−0.15	0.01
10. Selfish/not share with others	0.72	0.08	0.04
11. Noisy/yell	0.46	0.01	0.35
15. Dangerous behavior	0.59	0.26	−0.07
19. Angry/threw temper tantrum	0.58	0.11	0.24
23. Hit others	0.58	0.23	0.01
27. Throw things	0.68	0.00	−0.02
28. Rough play	0.67	−0.13	−0.29
7. Curious/explore	0.03	0.95	−0.02
9. Cheerful/content	0.04	0.97	0.01
17. Show self-confidence	−0.04	0.98	−0.01
20. Socialize with others	−0.13	0.96	0.11
26. Express love/affection to others	0.07	0.96	−0.15
5a. Moody	0.23	−0.07	0.75
18. Clingy/whiny	0.04	−0.12	0.74
25. Handle change/difficult situation	0.22	−0.13	−0.59
% variance explained	27.1	22.6	9.9

Note. N = 266. Eleven items were excluded from analyses due to lack of variance (< 5% or > 95% endorsing the item) or lack of association with other items. Geomin rotation was used. Loadings > 0.40 (in bold) were considered as significantly loading onto a factor.

3.2. Measurement Invariance Across Time and Group

Next, measurement invariance was assessed simultaneously across all three assessment times and all three groups. See Figure 1 for a path diagram of the longitudinal model. There were estimation issues with the positive and distress factors. For example, the positive items were so highly correlated over time (approaching r = 1) that there was a non-positive definite covariance matrix. As shown in Table 2 and Figure 1, all item loadings on this factor were also virtually 1. We tried to diagnose the problem by eliminating certain items; however, this did not improve model fit. Therefore, we decided not to pursue these factors any further.

Figure 1.

N = 315. A path diagram representing the longitudinal model. Estimates are standardized loadings and factor correlations in the model constraining loadings, thresholds, and intercepts to be equal. Because they are constrained to be equal over time and group, factor loadings are only shown for Assessment 1 for clarity. In the multi-group longitudinal invariance model, paths shown above are systematically held constant across all groups. Item covariances were modeled but estimates were not shown here for clarity. Refer to Table 2 for the descriptions of the manifest items included above.

For the aggression/noncompliance factor, we found that the configural model did not have significantly better fit than the loadings invariance model, and the loadings model did not have significantly better fit than the intercepts invariance model. Finally, the intercepts model did not have significantly better fit than the model where all thresholds, factor loadings, intercepts, and residual variances were equal across time and groups (see Table 3). Therefore, we had evidence that supports that the thresholds, loadings, intercepts, and residual variances were invariant across time and groups. In other words, there was evidence to suggest that this factor functions similarly across time and across IA, FC, and CC children. Coefficient alphas for this factor across the three assessments ranged from 0.70 – 0.72.

Table 3.

Measurement invariance statistics and model comparisons.

Model	df/Δdf	χ2 / Δχ2	RMSEA [95% CI]	TLI / ΔTLI	CFI / ΔCFI
Invariance across time and group
Thresholds (configural)	882	759.65	[0.021, 0.034]	0.942	0.952
Thresholds, loadings	56	58.16	[0.021, 0.034]	0.002	−0.001
Thresholds, loadings, intercepts	8	7.25	[0.020, 0.033]	0.003	0.002
Thresholds, loadings, intercepts, residuals	72	72.57	[0.017, 0.030]	0.010	0.006
Invariance across time and gender
Thresholds (configural)	588	499.74	[0.012, 0.031]	0.967	0.972
Thresholds, loadings	35	53.64*	[0.020, 0.035]	−0.017	−0.016
Thresholds, loadings, intercepts	5	5.00	[0.019, 0.034]	0.001	0.000
Thresholds, loadings, intercepts, residuals	45	51.76	[0.020, 0.035]	−0.002	−0.005

Note. N = 315. Fit indices were derived based on the null model as described in Liu et al. (2017). Estimates for the configural models are provided, and then each successive estimate represents the change (Δ) from the previous model.

^*p < .05

The finding of measurement invariance across time for the aggressive/noncompliant factor (see Table 3) provides evidence that this instrument is measuring behavior in the same way over time and allows us to then compare levels of this factor over time. As a post-hoc exploration, we examined whether mean scores on this factor across all three groups combined would significantly decrease over time, consistent with the general increase in self-regulation skills typically seen across development (Blair & Raver, 2012). While scores on this factor did decrease across time (M = 0.95, 0.81, 0.72; SE = 0.14, 0.12, 0.12, respectively), it was not a significant difference, as holding all factor means constant across time did not result in a significant decrease in model fit (Δχ² = 6.19, p = 0.13). Similarly, child age was not significantly associated with factor scores at any assessment (p’s > .26). Further, because internationally adopted children and children placed in foster care tend to show more problem behaviors than children raised in their biological families (Koss et al., 2016; Merz & McCall, 2010; Leve et al., 2012), we also expected that participants in the IA and FC groups would have significantly higher scores on this factor compared to those in the CC group. However, this was determined not to be the case, as holding all factor means constant across groups and time did not result in a significant decrease in model fit (Δχ² = 1.92, p = 0.34).

3.3. Measurement Invariance Across Time and Gender

We next examined measurement invariance simultaneously across time and gender. The configural model for the aggression/noncompliance factor fit significantly better than the loadings invariance model (see Table 3). We then investigated partial invariance and found that, when loadings for item #15 were allowed to vary, the configural model did not fit significantly better than the partial loadings invariance model (Δχ² = 22.95, p = 0.07). The partial loadings model did not fit significantly better than the intercepts invariance model, nor did the intercepts model fit better than the model constraining loadings, intercepts, and residuals to be equivalent (see Table 3), implying partial invariance across time and between genders. These results provide preliminary support that this aggressive/noncompliant factor behaves similarly across time and gender.

Because of this, we conducted a post-hoc analysis to examine whether males would have higher scores than females on the aggressive/noncompliant factor. This would be consistent with reports using other measures that boys tend to display higher rates of overt aggression than girls (Baillargeon et al., 2007). This was not supported, as evidenced by a nonsignificant decrease in model fit when holding factor means constant across gender and time (Δχ² = 3.73, p = 0.22).

3.4. Post-hoc Assessment of Convergent and Discriminant Validity

To assess the convergent and discriminant validity of the aggressive/noncompliant factor (see Table 4 for the descriptive statistics), the PDR-T mean scores were regressed on the scores on other previously validated questionnaire scales. Given that this factor seems to be measuring aggressive/noncompliant behavior, we expected that the factor would be negatively associated with BITSEA competence and ECBQ/CBQ inhibitory control and Effortful Control and positively associated with BITSEA problems and CBCL externalizing problems. We also expected that this factor would not be significantly associated with CBCL internalizing problems or ECBQ/CBQ fearfulness, shyness, cuddliness, or Negative Affectivity.

Table 4.

Percentage of groups endorsing each item on at least one day and proportion of items endorsed for the aggressive/noncompliant factor for each group.

	Internationally adopted (IA) (N = 156)			Foster care (FC) (N = 79)			Community comparison (CC) (N = 80)
Assessment	1	2	3	1	2	3	1	2	3
1. Touch things they shouldn’t (%)	76.3	67.4	67.6	79.4	75.9	76.1	75.0	71.8	63.1
4. Break household rules (%)	73.7	77.4	75.9	77.9	70.7	73.9	59.2	67.6	64.6
10. Selfish/not share (%)	35.1	43.8	44.4	67.6	51.7	45.7	49.3	46.5	52.3
15. Dangerous behavior (%)	14.0	9.5	9.3	29.4	20.7	23.9	11.8	9.9	12.3
19. Angry/temper tantrum (%)	64.0	59.1	47.2	56.7	62.1	54.3	59.2	50.7	47.7
23. Hit others (%)	38.6	31.4	24.1	47.1	43.1	54.3	39.5	31.0	27.7
27. Throw things (%)	63.2	53.3	50.0	58.8	55.2	47.8	65.8	62/0	46.2
28. Rough play (%)	43.9	43.1	49.1	42.6	41.4	47.8	55.3	52.1	55.4
Mean (SD)	0.51	0.49	0.48	0.58	0.53	0.53	0.52	0.50	0.48
Range (all groups) = 0 - 1	(0.22)	(0.23)	(0.23)	(0.29)	(0.29)	(0.28)	(0.26)	(0.24)	(0.25)

We found that the PDR-T factor was positively correlated with the BITSEA problems score across all three assessments and negatively correlated with the competence score for the first and second assessments (see Table 5). We also found a negative association with the ECBQ/CBQ inhibitory scale across all three assessments such that children scoring higher on inhibitory control had lower daily reports of aggressive and noncompliant behavior on the PDR-T. The aggressive/noncompliant factor was significantly associated with the ECBQ/CBQ Effortful Control scale, but only at the first assessment. We also found significant positive associations between the aggressive/noncompliant factor and the CBCL externalizing and internalizing scales (see Table 5). Notably, correlations between the aggressive/noncompliant factor and the CBCL externalizing scale were generally larger in magnitude than those with the internalizing scale, and the CBCL internalizing scale no longer significantly predicted the aggressive/noncompliant factor when controlling for the CBCL externalizing scale. Also shown in Table 5, we found that the aggressive/noncompliant factor was not associated with fearfulness, shyness (with one exception), or cuddliness as reported on the ECBQ/CBQ, which supports the idea that this factor is distinct from a more general negative emotionality. Interestingly, however, the aggressive/noncompliant factor was associated with the ECBQ/CBQ Negative Affectivity scale at the third assessment.

4. Discussion

The present study is the first to rigorously examine the psychometric properties of the PDR-T, a brief parent-report daily checklist of toddler behaviors. This measure was developed to provide a quick, easy way for parents to report on their toddler’s daily behaviors, which may be more ecologically valid than a measure that requires retrospective reporting over longer periods of time or structured laboratory observational measures. While there are clear benefits to using measures that inquire about longer periods of time (e.g., 6 months, 1 year), this type of retrospective reporting is still weighted towards more recent events (Raphael, Cloitre, & Dohrenwend, 1991). Additionally, toddlerhood is a developmental period of marked changes over a short time scale and the PDR-T can better examine rapid change over a highly transformative developmental period. Further, the PDR for older children has previously shown sensitivity to change over time in intervention studies (Chamberlain & Reid, 1994; Patterson, Chamberlain, & Reid, 1982; Price et al., 2015; Schindler et al., 2017; Schoorl et al., 2017; Urtesky et al., 2017). We found an aggressive/noncompliant behavior factor that was invariant across time and groups, was partially invariant between genders, showed good reliability, and displayed evidence of convergent and discriminant validity. It is important to note that these results pertain to an aggregate across three daily reports of the PDR-T, not a single day, which may be critical to understanding the utility and limitations of the PDR-T. Still, by using a rigorous multiple-group longitudinal model that acknowledges the categorical nature of the data, we can be more certain that the results of our analyses accurately represent the relationships between the items on the PDR-T.

The EFA analyzing data collected at the second assessment resulted in a three-factor model that fit the data best. One factor that included positive behaviors had items that were too highly correlated, which caused estimation issues. Another factor that seemed to assess distress behavior would not allow the measurement invariance model to converge. However, the third factor, with items measuring aggression and noncompliant behavior, had good reliability and had invariance of thresholds, loadings, intercepts, and residuals across time, group, and gender. This is important given 1) this measure’s utility in measuring change over time and/or following intervention and 2) the consistent gender and group differences typically found in aggressive/noncompliant behavior. Overall, these results provide preliminary evidence that this factor of the PDR-T effectively measures day-to-day aggressive/noncompliant behavior in toddlers that can be compared across time, gender, and different populations who had experienced early adversity. Still, future research should investigate invariance with larger samples to ensure that other factors can also be reliably measured and that this checklist can be used to compare populations that have experienced different types of early adversity and levels of SES.

In an attempt to understand the construct(s) captured by the PDR-T factor that we have termed aggression/noncompliance, we examined associations with other established measures of child behavior. The PDR-T aggressive/noncompliant factor was positively correlated with BITSEA problems and CBCL externalizing behavior, supporting the idea that this factor measures aggressive and noncompliant behavior. Furthermore, the factor was negatively correlated with BITSEA competence and ECBQ/CBQ inhibitory control; however, it was only correlated with the ECBQ/CBQ Effortful Control at the first assessment. Prior research has shown that children who exhibit more aggressive or noncompliant behavior tend to have deficits in effortful control (Friedman-Weieneth, Harvey, Youngwirth, & Goldstein, 2007; Olson et al., 2011), so the lack of association in this study warrants further investigation. This factor was also positively correlated with the CBCL internalizing scale, which was not hypothesized, but is consistent with the comorbidity of internalizing and externalizing behavior problems in early childhood (Gjone & Stevenson, 1997) and this association was no longer significant when controlling for CBCL externalizing scores.

Aggressive/noncompliant behavior (or lack thereof) are sometimes conflated with negative affectivity more generally, shyness, or fear (Chan, 2010; Serbin, Schwartzman, Moskowitz, & Ledingham, 2013); however, the aggressive/noncompliant factor of the PDR-T was not associated with ECBQ/CBQ Negative Affectivity, shyness, or fear. There was an unexpected positive association between the PDR-T factor and Negative Affectivity at the third assessment. The unexpected lack of association with Effortful Control at the third assessment, coupled with the unexpected association with Negative Affectivity at this assessment, suggests that this factor may be associated with different developmental constructs at different ages. It is also possible that this difference is driven by adoptive and foster families’ changing perspectives after longer periods of time spent with their adoptive/foster child. Regardless, this pattern of results is important to note for researchers using the PDR-T with toddlers and young children. Overall, these findings corroborate the hypothesis that the PDR-T aggressive/noncompliant factor is indeed measuring aggression, noncompliance, and externalizing symptoms and not other unrelated constructs, but future research should continue to explore this PDR-T factor and its associations with other developmental constructs.

We expected that there would be changes in levels of the aggression/noncompliance factor over time, and we also expected group and gender differences. However, we found no evidence of significant differences in the factor over time, group, or gender. As evidenced in Table 4, average scores were virtually identical over time and across groups. This finding was unexpected, and it may be a product of the significant variation found within each group. Other questionnaires ask parents to report on their child’s behavior across several months; however, the PDR-T represents an aggregation of three reports over three 24-hour periods. This shorter time scale may result in a measure that better captures short-term within-individual variation rather than the more global variation that arises over longer time scales. If so, this would be a particular strength of this measure when assessing change in behavior over short time scales, such as before and immediately after an intervention. Still, a stronger approach to investigate the lack of group-level change over time might be to control for the heterogeneity within the groups using propensity score matching. However, the present sample was collected from two parallel studies with a limited number of overlapping measures from which we could create propensity scores. Future research should investigate a propensity score matching approach with larger samples to more systematically test invariance and group differences across specific characteristics. In particular, because we only found partial evidence of invariance between genders, this finding should be further investigated with larger samples.

A portion of the sample included a “too young” option for each item that was recoded as “no” for the present analyses. Future investigations should assess whether including a “too young” option changes the factor structure or dimensionality of the PDR-T. Given that there are different versions of the PDR for different ages, it makes the most sense to administer the appropriate version based on the age of the children, eliminating the need for a “too young” option. Future studies should also assess the psychometric properties of other versions of the PDR to assess whether its psychometric properties remain consistent and a developmentally appropriate scale of aggressive/noncompliant behavior is present in all versions of the PDR. If this is the case, multiple versions could be administered within the same study depending on the age range of the children. Similarly, it is important to note that age was confounded with assessment time in this study, which may impact our conclusions about invariance across age. Thus, it may be more accurate to say that we have found invariance across time, but not necessarily developmental age. Additionally, the present analyses use a dichotomous variable for each item due to the high frequency of caregivers endorsing a given behavior on all (three) or none of the reporting days per assessment, with few caregivers endorsing an item for only one or two days. Future researchers using the PDR-T in its current form will likely encounter the same issue – for which we recommend dichotomizing each item as we have done above. However, this is also a potential avenue for improvement of the instrument. Future iterations of the PDR-T could seek to identify items that have better sensitivity to variation in reporting.

The PDR-T was created to assess positive and negative behaviors; however, the positive items had little to no variance or were too highly correlated with each other, which resulted in estimation issues. One item (#13 – Did your child play?) had zero variance at all assessments. We suspect that this may partially be due to socially desirable reporting biases, such as a parent’s unwillingness to report a lack of positive behavior in their toddler. Problem behavior, on the other hand, may be common enough in toddlers that social desirability does not influence reporting to a similar degree. Conversely, the positive items may simply have higher base rates of occurrence; thus, it is extremely unlikely that the children would not display the behavior at some point throughout the day. Nonetheless, it is also problematic that the positive items had correlations approaching r = 1. Future iterations of this checklist should consider altering the positive items to better capture variance in daily positive behavior and minimize the potential influence of socially desirable reporting.

Several items of the PDR-T were included to measure behavior related to internalizing and withdrawal symptoms; however, most of these items did not load together in the EFA. The three items that did load onto a factor labeled distress is the closest approximation of this category of behavior. Unfortunately, when we attempted to assess this factor’s psychometric properties, the invariance model did not converge for reasons that could not be determined. This may be indicative of a broader limitation related to parent-reported child internalizing behavior, particularly in early childhood. Parent-reported internalizing symptoms tend to be unreliable and/or not strongly associated with child-reported internalizing symptoms (Treutler & Epkins, 2003). Additionally, externalizing behaviors may be more salient or require more of a parent’s attention in toddlers, allowing for more reliable parent report. Given the lack of clear internalizing phenotypes in toddlerhood, it may be particularly difficult to recognize internalizing behaviors on a short time scale like 24 hours, as on the PDR-T.

The present analyses provide preliminary evidence of the psychometric properties of an aggressive/noncompliant factor of the PDR-T, a checklist designed for parents to quickly and easily report on their toddlers’ daily behaviors. This measure can be used to assess changes in problem behavior over time and in different populations. The above analyses have also identified items that can be adjusted or replaced to better reflect positive and internalizing behaviors. Future research should determine whether the above-mentioned factors show predictive validity for other developmental outcomes, as well as whether individual items or a total behavior score (all 28 items) would be similarly informative. The PDR-T is easy, quick to administer, and avoids parent recall bias by only asking about their children’s behavior over the last 24 hours, which makes this checklist extremely useful to detect short-term within-individual variations in behavior or record trajectories of children’s development over time, following an intervention, or after a major change in their environment.