Evaluating Caregiver Sensitivity to Infants: Measures Matter

Abstract

The significance of caregiver sensitivity for child development has been debated among scholars, not least due to sensitivity’s inconsistent predictive value over time and across contexts. A lack of uniformity in the definition of sensitivity contributes to this debate, but shortfalls of inter-tool concordance and construct validity in the instruments used to assess sensitivity may also be at issue. This study examines correspondences among four established standardized measures of caregiver sensitivity in independent classifications of the same sample of mothers of infants. 50 European American mother- infant dyads of diverse SES were independently assessed with three observational caregiver sensitivity measures: the Emotional Availability Scales (EAS; Biringen, 2008), the Parent Child Interaction – Nursing Child Assessment Feeding Scale (PCI-NCAFS; Oxford & Findlay, 2015), and the Maternal Behaviour Q-Sort (MBQS; Moran, Pederson & Bento, 2009). Ratings were juxtaposed with classifications of the same sample based on the original Ainsworth Maternal Sensitivity Scales (AMSS; Ainsworth, 1969). The EAS, NCAFS, and MBQS related to the AMSS, but large proportions of variance were unshared. Researchers and clinicians should be cautious when assuming that popular observational assessment instruments, commonly believed to measure a generic construct of caregiver sensitivity, are interchangeable, as these measures may evaluate different features of sensitivity to infants.

Sensitivity to infants ranks among the most central yet somewhat equivocal constructs in the field of parenting. Noticing an infant’s signals, “interpreting them accurately and…responding to them appropriately” (Ainsworth, Blehar, Waters, & Wall, 1978, p. 40) have long been considered cornerstones of the type of good parenting that leads to a child’s secure attachment and optimal physical and mental health (Bigelow et al., 2010; Bornstein, 2015; Bowlby, 1969; Breatherton, 2013; McElwain & Booth-LaForce, 2006; van den Boom, 1997; World Health Organization, 2004). Yet many questions persist about the strength of associations between sensitivity and child development generally (Atkinson et al., 2000; Bigelow et al., 2010; De Wolff & van IJzendoorn, 1997; Goldsmith & Alansky, 1987; Paavola et al., 2006; Page, Wilhelm, Gamble & Card, 2010; Pillhofer et al., 2015; Tamis-LeMonda, Briggs, McClowry, & Snow, 2009) and the predictive value of sensitivity in diverse contexts (Ispa et al., 2004; Jin, Jacobvitz, Hazen, & Jung, 2012).

The putative significance of sensitivity is based on studies that have linked it to important psychosocial, cognitive, and physiological developmental outcomes (Bakermans- Kranenburg, van IJzendoorn, & Juffer, 2003; Bernier, Carlson, & Whipple, 2010; Hirsh-Pasek & Burchinal, 2006; Feldman, Eidelman, & Rotenberg, 2004; Lemelin, Tarabulsy, & Provost, 2006; Mesman, van IJzendoorn, & Bakermans-Kranenburg, 2012). Research has further uncovered associations of caregiver sensitivity with specific aspects of child language and intellectual and academic development (Feldman et al., 2004; Hirsh-Pasek & Burchinal, 2006; Lemelin et al., 2006; Nozadi et al., 2013; Paavola, Kemppinen, Kumpulainen, Moilanen, & Ebeling, 2006; Roger Mills-Koonce et al., 2015; Vallotton, Mastergeorge, Foster, Decker, & Ayoub, 2017), associations that may not be quite so robust or unique as those reported for other aspects of child development (Page et al., 2010).

Not surprisingly, caregiver sensitivity has been most consistently linked to child social and emotional development, largely because of sensitivity’s established ties to the quality of caregiver-infant attachment, which in turn generally predicts positive mental health outcomes (Bigelow et al., 2010; Bretherton, 2013; McElwain & Booth-LaForce, 2006; Moran, Pederson, & Tarabulsy, 2011; Stams, Juffer, & van IJzendoorn, 2002). Stams and colleagues (2002), for example, asserted that higher quality maternal sensitive responding in infancy (birth to 3 years), measured with the Ainsworth Scales (1974), uniquely predicted better social development at age 7 (r = .22, p < .01). However, even the sensitivity–mental health association has been challenged by studies that have failed to confirm the universal superiority of sensitivity over other variables (e.g., availability of play materials; parental supervision, orderly household; verbal stimulation) when forecasting social and affective developmental pathways (Diener, Nievar & Wright, 2003; Longo, McPherran Lombardi & Dearing, 2017; Page et al., 2010).

It is also fair to say that “caregiver sensitivity” remains poorly defined. Still overwhelmingly referred to as maternal sensitivity, caregiver sensitivity is habitually used interchangeably with maternal responsiveness, caregiver-infant interaction quality, or mother- infant congruence, etc., and lacks grounding in universally accepted, clearly delineated, and specific behaviors (Bigelow et al., 2010; Meins, Fernyhough, Fradley, & Tuckey, 2001; Posada et al., 2016; Shin, Park, Ryu, & Seomun, 2008). Scholars largely agree that sensitivity should be considered a statement about the interaction between caregiver and infant, a highly complex phenomenon, and important aspect of parenting (van den Boom, 1997) but its operationalization is far from standardized. Several theorists have sought to analyze and synthesize confusing data pertaining to sensitivity. De Wolff and van IJzendoorn (1997) and Meins et al. (2001) for example have offered cohesive and accessible definitions of the construct. However, insights generated by studies that critically examine the sensitivity paradigm, and query its consistency across contexts and populations, do not always reach clinical researchers and practitioners in a timely manner. Such delays in knowledge mobilization are problematic given the universal importance that assessment of sensitivity assumes in research and clinical settings, where it is typically used to classify and to make decisions about quality of caregiving, parental psychopathology, and parenting capacity.

Caregiver sensitivity and its ramifications have been examined in countless studies using a growing array of standardized measures (Tryphonopoulos, Letourneau, & DiTommaso, 2016). Importantly however, little is known about the validity and interchangeability of instruments that are purported to measure sensitivity. In other words, do we really know what we are assessing?

Sensitivity is typically assessed through one of two approaches: recording specific behaviors by observing caregivers and children during structured or naturalistic interactions, yielding “microanalytic” codes, or more holistic, global assessments that result in “macroanalytic” descriptions (Bornstein, Hahn, Suwalsky, & Haynes, 2011). Developmentalists have long debated the unique contributions provided by each approach (Bornstein et al., 2011; Cairns & Green, 1979; Clarke-Stewart & Hevey, 1981; Maccoby & Martin, 1983; Rothbaum & Crockenberg, 1995; Waters, 1978), showing that single behavior-event related vs. holistic assessments yield complementary but distinct data about parent–child relationships. Microanalytic coding captures situation-specific events and may vary from context to context for the same parent-child dyad, whereas macroanalytic assessments highlight more stable, global, enduring interaction styles that are not as dependent on contextual variability (Bornstein et al., 2011; Maccoby & Martin, 1983; Rothbaum & Crockenberg, 1995). Perhaps global analyses more fully capture the essence of established relationships and core characteristics (vs. discrete behaviors) of the two members of the interacting dyad. Moment-to-moment event recording provides detailed information about highly specific caregiver and/or child behaviors and deconstructs dyadic events, allowing for the examination of relationships (Isabella, Belsky, & von Eye, 1989). Micro-level coding is considered more straightforward or objective, less open to interpretation, and thus easier to learn and become reliable on. Macro-level coding is preferred by some researchers and clinicians because “macro-level constructs tend to be more conceptually attractive and …less labor-intensive…and … might more accurately abstract a global picture of detailed and complicated data” (Bornstein et al., 2011, p. 87).

In this study, we examine the concordance of four commonly used standardized measures of caregiver sensitivity in classifying the same sample of mothers of infants. The focus of the current study is not to differentiate the relative merits of micro- vs. macroanalytic assessments of caregiver sensitivity; however, while none of the measures of sensitivity studied here use micro- level coding per se, the four instruments used here are situated on a continuum from prototypically holistic to clearly specific-behavior-event related.

We examined the Emotional Availability Scales (EAS; Biringen, Robinson, & Emde, 1998; Biringen, 2008), the Parent Child Interaction – Nursing Child Assessment Satellite Training Feeding Scale (NCAFS; Barnard 1978; Oxford & Findlay, 2015), and the Maternal Behaviour Q Sort (MBQS; Moran et al., 2009), and juxtaposed their respective ratings with classifications of the same sample based on the original Ainsworth Maternal Sensitivity Scales (AMSS; Ainsworth, 1969). We chose these measures as among the most well known and accessible in research and clinical practice, as documented in reviews of commonly used observational assessment tools for evaluating the quality of caregiver-infant interaction (see Massouda, Davis, & Logsdon, 2011; Tryphonopoulous et al., 2016). We also conducted a PsycInfo search using the search terms “maternal sensitivity,” “parental sensitivity,” “caregiver sensitivity,” and randomly selected 200 relevant articles: 148 (74%) studies used at least 1 of the 4 measures we compare here (21.5% used the EAS; 9.0% the NCAST/NCAFS, 27.5% the MBQS, and 49% the Ainsworth Scales). The instruments included here all require formal training, a rigorous certification process, and evidence of reliability from researchers and/or clinicians who wish to use these measures in their work. All are used internationally in addition to the United States and Canada.

The Ainsworth Maternal Sensitivity Scales (AMSS; Ainsworth, 1969) constitute a longstanding and widely accepted prototypical naturalistic observation measure. Ainsworth’s scales provided the foundation for research in mother-infant sensitivity and its link to attachment style. Ainsworth pioneered the construct of maternal sensitivity and is often referenced when defining maternal sensitivity in the attachment literature (e.g., EAS, MBQS). Rated maternal behaviour with the AMSS when their infants were 9 to 12 months old is strongly related to attachment security in the Strange Situation (concurrent validity; r = .78; Ainsworth, Bell, & Stayton, 1971; Ainsworth et al., 1978; Pederson, Bailey, Tarabulsy, Bento, & Moran, 2014). These observations led Ainsworth and her colleagues to conclude that maternal sensitivity plays a central role in attachment theory. Results from meta-analytic reviews of maternal sensitivity and attachment security support Ainsworth’s maternal sensitivity hypothesis, but with smaller effect sizes (r = .24 − .32; Goldsmith & Alasnky, 1987; DeWolff, & van IJzendoorn, 1997). A reason for this notable difference in effect sizes could be that the durations of observation periods in replication studies were much shorter than the observation periods in the original study (Pederson et al., 2014).

The Emotional Availability Scales (EAS; Biringen et al., 1998; Biringen, 2008) are also a well-known measure of caregiver sensitivity that is widely used in research. The maternal sensitivity dimension of Emotional Availability is inspired by Ainsworth’s sensitivity conceptualization but is broader than the original maternal sensitivity concept (Mesman & Emmen, 2013). The EAS differ from attachment theory-based maternal sensitivity assessments in that the EAS integrates attachment theory and an emotional availability perspective (Emde, 1980; Mesman & Emmen, 2013; Tryphonopoulos et al., 2016). The EAS emphasize the emotional features of mother-infant interactions, including the mother’s emotional signaling and understanding of her infant’s signaling, in its conceptualization of sensitivity.

The Parent Child Interaction – Nursing Child Assessment Satellite Training Feeding Scale (NCAFS; Barnard 1978; Oxford & Findlay, 2015), a commonly used maternal sensitivity measure, is one of the most valid and user-friendly measures of mother-infant interactions (Byrne & Keefe, 2003; Tryphonopoulos et al., 2016). This assessment tool was developed to measure the quality of mother-infant interaction and how it influences later child cognitive development (Sumner & Spietz, 1994). A high total caregiver score suggests a high degree of maternal sensitivity. The NCAFS has also been used to predict positive child development, behavior, and attachment quality. The NCAFS demonstrates predictive validity with the Bayley II Scales of Infant Development (r = .72), the Preschool Behavior Questionnaire (r = .79), and the Home Observation for Measurement of the Environment (HOME; r = .76; Badr, Bookheimer, Purdy, & Deeb, 2009). The 3-month NCAFS score was a significant, although weak, predictor of security of attachment in the Ainsworth Strange Situation at 1 year (r = .19; Britton, Britton, & Gronwaldt, 2006).

The Maternal Behaviour Q-Sort (MBQS; Moran et al., 2009; Pederson et al., 1990) is a maternal sensitivity-focused, naturalistic observational assessment tool that is also used often in the mother-infant relationship and attachment literatures. Theoretically, the MBQS is rooted in Ainsworth’s maternal behaviour descriptions and her Maternal Sensitivity Scales, which describe a mother’s sensitivity as well as acceptance, accessibility, and cooperation, providing a conceptual framework for measuring maternal sensitivity. A high MBQS global maternal sensitivity score suggests a high degree of maternal sensitivity. There are multiple versions of MBQS; for this study, the Mini-MBQS-VR was used because it is more suitable for coding filmed interactions, compared to earlier versions of the MBQS (Tarabulsy et al., 2009). According to Tarabulsy et al. (2009), the mini-MBQS-VR is moderately associated with the Original MBQS-90 (r = .35) and with the Attachment Q-Sort index of attachment security (r =.34). A systematic review by Mesman and Emmen (2013) reported that the MBQS maternal sensitivity score is related to maternal attachment state of mind (Bailey, Moran, Pederson, & Bento, 2007; Lindhiem, Bernard, & Dozier, 2011; Whipple, Bernier, & Mageau, 2011), associated with infant attachment security (Atkinson et al., 2000), and sensitive to improvements in parenting quality post intervention (Moss et al., 2011).

In summary, sensitivity is commonly considered a strong predictor of healthy infant and child development, and is thus a critical element of caregiving in research and clinical circles alike, but the construct is beset by a number of inconsistencies and unanswered questions. We propose that, to address the incongruities that characterize sensitivity, it is necessary first to ask whether the tools that have been used to assess sensitivity across diverse studies – studies that are routinely pooled in meta-analyses to yield the empirical basis for theory and practice – all tap into a reliable, recognizable core concept, and could thus be considered (relatively) interchangeable.

Methods

Participants

The sample consisted of 50 European American mother-infant dyads (50% mothers-daughters). Mothers were recruited via mailing lists of births in the Washington DC metropolitan area, including suburbs of Virginia, Maryland, and West Virginia, with a letter describing the study and an invitation to contact the researchers if mothers were interested in learning more about the study and/or participating. Infants averaged 5.38 months (SD = .23) at the observation and 3467.45 g (SD = 444.66) at birth. 98.0% of the infants were term; non-term infants were healthy and not outliers on any sensitivity measure and were therefore retained in the sample. Mothers averaged 27.48 years of age (SD = 6.92) and varied in educational achievement (18% had not completed high school, 16% completed high school, 20% partially completed college, 26% completed college or university, and 20% completed university graduate programs); families varied in socioeconomic status (SES; Hollingshead, 1975; M = 48.39, SD = 13.93) across a range from 19–66. The sample is socioeconomically diverse, but ethnically homogenous, to enable the examination of cross-measure reliability eschewing ethnicity as a confounding variable (Bornstein, Jager, & Putnick, 2013; Jager, Putnick, & Bornstein, 2017).

The present study was conducted according to guidelines laid down in the Declaration of Helsinki, with written informed consent obtained from a parent or guardian for each child before any assessment or data collection. All procedures involving human subjects in this study were approved by the Institutional Review Board at the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Procedures

In the 2 weeks prior to each home visit, mothers completed a demographic questionnaire asking for background information about the infant, mother, and family. Each mother-infant dyad was visited in the home by a single observer to film a 1-hour video of naturalistic mother and infant behavior. The caregivers decided how to spend that hour. Before filming began, mothers reviewed and signed informed consent forms. A female filmer stated that she was interested in the infant’s usual activities and asked the mothers to carry on as they normally would. The filmer refrained from making eye contact with or interacting with the mother and the infant.

Coding

Mother-infant interactions were coded from videorecordings. All coders were research reliable as per requirements by the instruments’ authors. Coders were specific to the instruments except for MBQS and AMSS which were coded by the same group of three coders as per MBQS coding guidelines. MBQS coding guidelines indicate that MBQS and AMSS should be coded in tandem. For the EAS, 10 researchers coded the interactions because several other samples amounting to hundreds of interactions were coded at the same time. For the NCAFS, 2 coders coded the interactions. All EAS and NCAFS coders were trained to reliability by an author of the scales and maintained reliability with one another through double-coding. All coders were blind to hypotheses and purposes of the study and to additional information about the dyads. All scales were coded independently, without mutual consultation or guidance. In addition to instrument-specific research reliability, coders also obtained high intercoder reliability with other independent coders, assessed by using average absolute agreement intra-class correlation coefficients (ICC) in a two-way random effects model (McGraw & Wong, 1996; Shrout & Fleiss, 1979). ICC among independent coders is provided below for each of the measures. Details for each measure follow.

Measures

AMSS (Ainsworth, 1969)

The AMSS consist of four scales. The Sensitivity to the baby’s signals scale assesses the caregiver’s capacity to be aware of the infant’s signals, to interpret, and to respond appropriately and promptly. Scores range from 1 (highly insensitive) to 9 (highly sensitive). The Cooperation with baby’s ongoing behavior scale assesses the caregiver’s degree and frequency of physical cooperation (and lack of interference) with the infant’s activity. Scores range from 1 (highly interfering) to 9 (conspicuously cooperative). The physical and psychological Availability scale assesses the caregiver’s accessibility in terms of responsiveness to the infant. Scores range from 1 (highly inaccessible, ignoring or neglecting) to 9 (highly accessible). The Acceptance of the baby’s needs scale assesses the caregiver’s balance of positive and negative feelings about the infant. Scores range from 1 (highly rejecting) to 9 (highly accepting). The first 20 min of the interactions were coded with the AMSS. The ICCs, computed on 20% of the coded caregiver-infant interactions, were .93 for Sensitivity, .92 for Cooperation, .94 for Availability, and .93 for Acceptance. The four Ainsworth subscales were highly correlated and were therefore averaged to create a total scale, α = .95, which were used in the analyses.

EAS (3^rd edition; Biringen et al., 1998)

The EAS capture six dimensions of caregiver – child interaction divided into parent EAS (4 scales) and child EAS (2 scales which were not used in this study). The Sensitivity scale assesses the caregiver’s responsiveness to the infant’s communications, affect, regulation, and creativity in play from 1 (highly insensitive) to 9 (highly sensitive); the Structuring scale assesses the caregiver’s ability to appropriately facilitate, scaffold, or organize the infant’s play, exploration, or routine by providing rules without compromising the infant’s autonomy from 1 (non optimal) to 5 (optimal); the Nonintrusiveness scale assesses the caregiver’s ability to support the infant’s play, exploration, or routine by appropriately initiating interactions, without interrupting by being overly directive, overstimulating, overprotective, or interfering from 1 (intrusive) to 5 (nonintrusive); the Nonhostility scale assesses the caregiver’s ability to interact with the infant by being patient, pleasant, and harmonious and not rejecting, abrasive, impatient, or antagonistic from 1 (markedly hostile) to 5 (non hostile). All EAS were coded in ½ points. All four EAS scales of interest were coded to provide scores for each 15-min interval of the filmed interaction and for the total hour of the interaction. The correlations between comparable ratings for the first 15 min and the full hour ranged from .70 for Nonhostility to .88 for Sensitivity. Hence, we chose to use the scores for the first 15 min so that they would be comparable to the 10- to 20-min segments coded for other scales. The ICCs, computed on 22% of the coded caregiver-infant interactions, were .84 for Sensitivity, .68 for Structuring, .70 for Nonintrusiveness, and .73 for Nonhostility. The ICCs for Structuring, Nonintrusiveness, and Nonhostility were a result of low variance in the reliability sample; agreements within 1 point were 91%, 100%, and 82%, respectively. The four EAS scales were highly correlated and were therefore standardized and averaged to create a total scale, α = .83.

NCAFS Feeding Scale (Barnard 1978; Oxford & Findlay, 2015)

The NCAFS includes 76 binary (yes/no) items that are descriptive of the caregiver-infant dyadic relationship and organized into six subscales. Four subscales focus on caregiver behavior (Barnard, 1978). The Sensitivity to Cues scale assesses the caregiver’s ability to recognize and respond to the infant’s cues (range = 1–16). The Response to Distress scale assesses the caregiver’s ability to soothe or quiet a distressed child (range = 1–11). The Social-Emotional Growth Fostering scale assesses the caregiver’s affect and ability to communicate a positive feeling tone (range = 1–14). The Cognitive Growth Fostering scale assesses the caregiver’s ability to make learning experiences available to the infant (range = 1–9). (Two additional scales describe the infant’s contribution to the interaction; they were not considered in the present study.) Four caregiver NCAFS subscales were coded on 10-min feeding episodes in the interactions for each dyad. Feeding episodes occurred spontaneously in all the naturalistic observations. Seventy-six percent of the feeding episodes occurred in the first 20 min of the interaction and therefore overlapped with the segments for the other coding systems. Over half of the sample was breast or bottle fed (66%), 18% were fed solid food, 14% were fed a combination of bottle and solid food, and 2% were fed with a dropper. When controlling for maternal age, the feeding method was unrelated to NCAFS scores, F(3, 45) = 1.85, p = .152. The ICCs, computed on 24% of the coded caregiver-infant interactions, were .74 for Sensitivity to Cues, .75 for Response to Distress, .60 for Cognitive Growth Fostering, and .77 for Caregiver Total scales. The four caregiver subscales were summed to provide a total caregiver score, α = .76.

Mini MBQS-VR (Moran et al., 2009; Tarabulsy et al., 2009)

The Mini MBQS-VR is a shorter form of the original 90-item MBQS card set, consisting of 25 items (Tarabulsy et al., 2009). The Mini MBQS-VR focuses on specific sensitive caregiver behaviors in relation to the infant, including “Monitors baby’s activities during visit”, “Speaks to baby directly”, and “Praises baby”. The items are sorted into five groups, with five items per group. Items are designated as most like (+2), like (+1), neutral (0), unlike (−1), or most unlike (−2) the behaviors observed in the caregiver. The total score obtained for a given caregiver is then correlated with the developers’ criterion sort for the prototypically sensitive caregiver, generating a global sensitivity score. Scores vary from −1.0 (least like the prototypically sensitive caregiver) to 1.0 (most like the prototypically sensitive caregiver). The global sensitivity score can be obtained through an unforced or a forced-choice sort. The difference between an unforced and a forced sort is that for an unforced sort the coder is permitted to assign the quality of maternal behavior to the groups, without being restricted by the maximum number of items that are allowed per group. By contrast, when obtaining a forced sort, the coder is restricted by the maximum number of items that are allowed per group. As the unforced and forced scores were correlated, r(48) = .97, p < .001, we used the forced-choice sort in these analyses. The first 20 min of the interactions were coded with Mini MBQS-VR to ensure that the MBQS would be able to capture at least 10 min of direct caregiver-infant interaction, which is required for arriving at a global sensitivity score (Tarabulsy et al., 2009). The ICC, computed on 22% of the coded caregiver- infant interactions, was .98.

Analytic Plan

Prior to formal analysis, univariate distributions of the total scores for the AMSS, EAS, NCAFS, and MBQS, and their subscales were examined for normality, outliers, and influential cases (e.g., single cases that change the interpretation of the results), and standard transformations were applied to resolve problems of non-normality (Tabachnick & Fidell, 2012). Transformed variables were used in all analyses, but for ease of interpretation descriptive statistics are presented in the variables’ original metrics. EAS Nonintrusiveness and Nonhostility were skewed (ps < .05), and no transformation would normalize them; they were therefore analyzed with nonparametric statistics. The skewed distributions of the EAS Nonintrusiveness and Nonhostility scales are not surprising; these two scales were designed to capture specific types of negative behaviors that are uncommon in low-risk samples.

To assess relations among the AMSS, EAS, NCAFS, and MBQS, zero-order correlations were computed between the scales and between their subscales. Partial correlations were then computed, controlling for covariates, to assess whether relations among scales were inflated by sociodemographic factors. Further, bivariate correlation comparisons, using Fisher’s r-to-z transformation, were computed to evaluate whether reported correlation coefficients differed significantly from one another.

Four sociodemographic variables were considered as covariates: infant birth weight and age, and maternal age in years and education at the time of the home visit. Infant birth weight and age were considered as covariates because infant maturity is associated with maternal sensitivity (Muller-Nix et al., 2004). Maternal age was considered as a covariate due to existing empirical evidence that adult mothers demonstrate higher sensitivity than adolescent mothers (Lounds, Borkowksi, Whitman, Maxwell, & Weed, 2005; Secco & Moffatt, 2003). Additionally, lower maternal education is related to lower caregiver sensitivity (Tamis-LeMonda et al., 2009). As maternal age and education were highly correlated, r(48) = .74, p < .001, we assessed unique relations of each with the total scales, finding that maternal age but not maternal education, had unique relations to all scales. Hence, only maternal age was controlled in analyses. Infant birth weight and age were unrelated to AMSS, EAS, and MBQS scales, but dyads with older infants scored lower on the NCAFS, rs(48) = −.31 to −.45, ps = .029 – .001.

Statistical Power

A post-hoc power analysis was computed prior to data analysis to determine whether the sample size provided sufficient power to detect a medium-sized effect in a correlational design (Faul, Erdfelder, Lang, & Buchner, 2007). Given that we expected positive correlations among scales, we computed one-tailed tests. Power to detect a correlation of .30 was .81 with α = .10 (i.e., one-tailed correlations) and N = 50, indicating adequate power to detect medium and large correlations.

Results

Descriptive statistics of the AMSS, EAS, NCAFS, and MBQS total scores and subscales are presented in Table 1. These statistics indicate that there was adequate variance in all scores.

Table 1.

Descriptive statistics of the maternal sensitivity scores and subscales

	M	SD	range
AMSS Total	4.29	2.33	1.0 – 8.5
Sensitivity	3.82	2.35	1.0 – 8.0
Cooperation	4.14	2.47	1.0 – 9.0
Availability	4.58	2.45	1.0 – 9.0
Acceptance	4.60	2.74	1.0 – 9.0
EAS Total	.00	.81	−2.39 – 1.01
Sensitivity	5.68	1.64	2.0 – 8.5
Structuring	3.89	.94	1.5 – 5.0
Nonintrusiveness	4.71	.52	3.0 – 5.0
Nonhostility	4.56	.75	1.5 – 5.0
NCAFS Total	39.66	5.62	24.0 – 48.0
Sensitivity to Cues	13.60	1.95	8.0 – 16.0
Responsiveness to Distress	9.88	1.35	6.0 – 11.0
Emotional Growth Fostering	10.78	2.00	6.0 – 14.0
Cognitive Growth Fostering	5.40	1.98	1.0 – 9.0
MBQS scale	−.22	.60	−.90 – .80

Correlations among Total Scales

Table 2 displays correlations among the total scores for the AMSS, EAS, NCAFS, and MBQS (below the diagonal). Correlations with the NCAFS were medium-sized, and correlations among other scores were large in size. Despite the strength of these relations, 10–91% of the variance between scales was unshared (above the diagonal in Table 2). Perhaps because they were coded by the same coders on the same video segments, the correlation between the AMSS and MBQS scores was very large, and larger than the correlations of the AMSS and MBQS with other scores (see Table 2 notes for z-scores comparing the correlations). Furthermore, the EAS total score was more highly correlated with the AMSS score than the NCAFS score. When maternal age was controlled, relations of the NCAFS total score with other scores attenuated to nonsignificance. Relations between the AMSS, EAS, and MBQS remained large and statistically significant.

Table 2.

Correlations among total sensitivity scores and their unshared variances in percent

	1	2	3	4
1. AMSS total	--	56	83	10
2. EAS total	.66***12/.59***	--	91	70
3. NCAFS total	.41**3/.14	.30*1/.04	--	85
4. MBQS	.95***2345/.93***	.55***4/.46***	.39**5/.12	--

Note. Pearson zero-order 1-tailed correlations are below the diagonal before the slash and partial correlations controlling for maternal age (and infant age for correlations with the NCAFS total scale) follow the slash. The percentages of unshared variance based on zero-order correlations are above the diagonal. Zero-order correlations with the same superscript were significantly different from one another.

¹z = 2.84, p = .005.

²z = −5.90, p < .001.

³z = −7.54, p < .001.

⁴z = −8.09, p < .001.

⁵z = −7.84, p < .001.

^*p ≤ .05.

^**p ≤ .01.

^***p ≤ .001.

Correlations among Subscales

Table 3 presents zero-order correlations and partial correlations controlling for maternal age (and infant age for NCAFS correlations) for the subscales of each instrument. Zero-order correlations among the sensitivity subscales (shaded cells in Table 3) for the AMSS, EAS, NCAFS (sensitivity to cues), and MBQS were medium to large (rs = .39–.91). Two of the four NCAFS subscales, emotional and cognitive growth fostering, were largely unrelated to the subscales of other measures. NCAFS responsiveness to distress was also largely unrelated to the EAS subscales. Partial correlations, controlling for maternal age, reduced the magnitude of relations slightly, but most remained medium to large in size.

Table 3.

Correlations among EAS, NCAFS, AMSS, and MBQS subscales

		EAS				NCAFS				MBQS
		Sensitivity	Structuring	Nonintrusiveness	Nonhostility	Sensitivity to Cues	Responsiveness to Distress	Emotional Growth Fostering	Cognitive Growth Fostering	Scale
AMSS
Sensitivity	r	.63***	.68***	.40**	.51***	.46***	.48***	.33**	.18	.91***
	rp	.54***	.65***	.26*	.47***	.28*	.32*	.02	−.07	.89***
Cooperation	r	.48***	.56***	.42***	.42***	.42***	.45***	.31*	.22	.83***
	rp	.37**	.51***	.30*	.37**	.26*	.32*	.04	.04	.79***
Availability	r	.60***	.60***	.16	.41**	.40**	.42***	.29*	.18	.82***
	rp	.53***	.56***	.03	.36**	.23	.28*	.04	.04	.79***
Acceptance	r	.57***	.64***	.45***	.52***	.39**	.47***	.23	.06	.80***
	rp	.49***	.60***	.34**	.48***	.21	.33*	−.08	−.08	.76***
EAS
Sensitivity	r	--	--	--	--	.41***	.36**	.24*	.14	.51***
	rp	--	--	--	--	.27*	.23*	−.02	−.05	.40**
Structuring	r	--	--	--	--	.39**	.26*	.16	−.01	.55***
	rp	--	--	--	--	.27*	.12	−.06	−.21	.49***
Nonintrusivenessa	ρ	--	--	--	--	.38**	.24*	.07	.08	.31*
	ρp	--	--	--	--	.22	.09	−.27*	−.13	.17
Nonhostilitya	ρ	--	--	--	--	.44***	.23	.06	−.02	.44***
	ρp	--	--	--	--	.33*	.09	−.18	−.24	.38**
MBQS scale	r	--	--	--	--	.39**	.44**	.29*	.18	--
	rp	--	--	--	--	.21	.28	−.01	−.05	--

Note. Pearson zero-order correlations (r) or Spearman’s rho (ρ) and partial correlations (r_p) or partial rho (ρ_p) controlling for maternal age (and infant age for correlations with the NCAFS subscales). Shaded cells contain correlations between sensitivity subscales.

^*p ≤ .05.

^**p ≤ .01.

^***p ≤ .001.

Discussion

Ainsworth’s seminal AMSS, once the “gold standard” for measuring caregiver sensitivity, has given way to an array of more accessible, teachable, and less time-consuming tools that were all designed to capture the quality of “sensitivity” in caregiver–infant interaction as originally conceived by Ainsworth. Our data suggest that the most popular observational assessment tools for caregiver sensitivity today, administered by reliable coders, may not be transposable in assessing the quality of dyadic relationships. The three assessment tools examined here do indeed conceptually relate to sensitivity as originally conceived by Ainsworth (AMSS), as shown by several medium-to-high associations between measures. However, what is surprising is that large fractions of the variance between these scales are unshared (see Table 2). The current results support De Wolff and van IJzendoorn’s (1997) concerns about a large unexplained variance in the observational sensitivity measures they studied in the context of their meta-analysis that examined the relation between sensitivity and attachment. It is thus important to acknowledge that these popular tools are not similar enough to be used interchangeably in either research or clinical practice.

Our findings offer a partial explanation for inconsistent reports on relations between sensitivity and developmental outcomes in children (Paavola et al., 2006; Pillhofer et al., 2015). The data raise broad questions about the examination of caregiver sensitivity in research and practice, the repercussions for its predictive value in child development, and, not least, the value of using sensitivity assessment results in the design of parenting interventions (Nozadi et al., 2013; Vallotton et al., 2017; Vandell, Belsky, Burchinal, Steinberg, & Vandergrift, 2010). Given the common assumption that available caregiver sensitivity measures all capture a common construct, child development researchers typically may prefer to use measures that require less administration time, less training, and simpler reliability procedures, for example the NCAFS (Tryphonopoulos et al., 2016). In addition, developmental scientists with limited budgets may gravitate to brief behavior-based event coding systems that appear to be inherently more objective as well as being more economical to administer (for example, tools such as the NCAFS). In this study however, of the three instruments examined, the NCAFS proved to have the weakest association with the AMSS, calling into question its suitability as a good substitution for a tool that assesses sensitivity as proposed by Ainsworth. In comparing these measures, it is worthwhile to note that NCAFS is structured differently than EAS, MBQS, and AMSS. NCAFS scales require yes/no decisions about behaviors that, if occurring even once are credited to the caregiver, whereas EAS, MBQS, and AMSS are continuous, global scales that more holistically assess the quality of a dyadic relationship. The differences between these measures’ methods of assessing caregivers’ behavior, some focusing more on quality and others more on quantity, should be taken into consideration when interpreting the results, as should the limitations inherent in each type of assessment. For example, when minimal discrete caregiver behaviors are counted and recorded, lack of genuineness or warmth in the relationship may be missed. As well, with some scales (for example the NCAFS) caregivers who show a codable behavior once during the observation obtain the same score for that behavior as caregivers who show the behavior repeatedly. Other measures make use of scales that allow for consideration of the quantity and/or quality of the caregiving behavior. In contrast, global rating systems, while capturing the subtleties and “flavor” of interactions when used by experienced clinicians, are by definition more subjective, and possibly less reliable when novice coders are involved. However, relatively stringent reliability requirements and mandatory certification increasingly address these concerns. What seems to be clear is that researchers need to carefully select measures to match specific study questions. The current data also suggest that researchers should be cautious when assembling meta-analyses and reviews of studies based on potentially divergent methods of assessing a possibly multifocal global concept such as sensitivity.

Infant mental health and parenting clinicians by necessity tend to select measures that are user-friendly and cost effective, again assuming that existing tools are interchangeable when it comes to assessing dyadic interactions and planning for caregiving interventions. Based on our findings, clinicians would be well advised to consider a specific measure’s idiosyncrasies when interpreting caregiver sensitivity scores in clinical settings, as one instrument might assess a caregiver as more sensitive than another. Clinicians should be aware that currently no one instrument provides a definitive assessment of sensitivity and should be wary about drawing broad conclusions from the results of any single evaluation of parenting, even when provided by a certified assessor. Clinicians should also be encouraged to select the assessment tool that best matches their specific purpose in assessing or treatment planning. For example, researchers or clinicians who want to identify particular behaviors as foci for intervention may elect a behaviorally focused measure like the NCAFS, whereas researchers or clinicians who are interested in the global sensitivity climate created by the mother might choose the AMSS or MBQS, and researchers or clinicians interested in a global dyadic measure of sensitivity might elect the EAS.

Limitations

Our study sample consisted of participants with a wide range of educational achievement, SES, and mothers’ age, but its ethnic homogeneity may limit the generalizability of the findings while eliminating variability associated with ethnic variation in or interpretation of sensitivity. Another potential limitation of this study pertains to the number of correlational analyses that were conducted simultaneously, with an increased chance that some significant results may be spurious. That eventuality would, however, favor correlation inflation which would in turn only reduce the striking unshared variance we uncovered. It is also important to be mindful of the global natures of the EAS, NCAFS, MBQS, and AMSS, as well as differences in the criteria required by each measure for coding dyadic interactions as optimal. Most of these measures require making a global judgment of the dyad’s functioning, which may be subject to inherent biases (such caregiver appearance). However, we carefully monitored coder reliability to ensure consistency across coders. Moreover, the AMSS, EAS, and MBQS call for naturalistic observations of caregiver-infant interactions, whilst the NCAFS requires a feeding episode, and the MBQS is deemed most suitable for coding play interactions. The AMSS and MBQS are designed to focus on caregiver behavior, whereas the EAS and NCAFS are designed to assess dyadic behavior. Another consideration is that, based on MBQS protocol, the AMSS and MBQS were scored by the same coders. Finally, for the purpose of this study, we excluded the subscales and total scales that assessed infant behavior in order to contrast only scales of the AMSS, EAS, NCAFS, and MBQS that are relevant to caregiver behavior, thus potentially forfeiting additional helpful information about the quality of dyadic interaction.

Areas for Future Study

Next steps should involve an examination of common core concepts in a larger sample of caregivers, and perhaps a critical discussion of what exactly is measured when “sensitivity” is assessed from different vantages. More comprehensive, longitudinal studies of the predictive validity of commonly used caregiver sensitivity scales (such as the scales examined here), in both ethnically homogeneous and culturally diverse samples, are especially recommended.

On the basis of the results of the present study, investigators should be guarded about drawing conclusions about the caregiving skills of parents based on investigations that are limited to the use of a single measure of sensitivity. Maternal sensitivity does not take the same form across diverse cultural groups (Bornstein, 2012; Ekmekci et al., 2016; Emmen, Malda, Mesman, Ekmekci, & van IJzendoorn, 2012; Kelley & Tseng, 1992), and questions remain about its value when predicting healthy child development in cultures that vary in their customs from standard Western middle-class caregiver practices (Chan, Penner, Mah & Johnston, 2010; Ispa et al., 2004; Jin, et al., 2012; Kelley & Tseng, 1992; Liu, Chen, Bohr, Wang & Tronick, in press; Keng-Ling & Li-Jeung, 2010; Lieber, Fung, & Leung, 2006; Lin & Fu, 1990; Wu et al., 2002). Identifying measures of sensitivity that are culturally appropriate, capture culturally specific aspects of caregiver sensitivity, and are predictive of healthy child development should be a priority.

Conclusion

Caregiver sensitivity is widely considered to be an important index of parenting and to contribute to positive child development. Nevertheless, questions remain about what it means for caregivers to be “sensitive” and how “sensitivity” is operationalized. By showing that four popular assessment tools used with the same mother-infant dyads share relatively little common variance when it comes to capturing caregiver sensitivity, the current study confirms that there is much work left to be done in standardizing and harmonizing the definition, operationalization, and, by extension, assessment of sensitivity. It would appear that different measures of sensitivity are not interchangeable. Furthermore, findings from studies that used diverse instruments should not be pooled in meta-analyses that examine the correlates and outcomes of caregiver sensitivity.