Abstract
The Scale to Assess the Therapeutic Relationship in Community Mental Health Care (STAR) is a frequently-administered tool for measuring therapeutic relationships between clinicians and patients. This manuscript tested the STAR’s psychometric properties within a community health worker (CHW)-led intervention study involving pregnant and postpartum women. Women (n = 141) enrolled in the study completed the 12-item participant STAR survey (STAR-P) at two time points over the course of pregnancy and at two time points after delivery. The factor structure of the STAR-P proved to be unstable with this population. However, a revised 9-item STAR-P revealed a two-factor model of positive and negative interactions, and demonstrated strong internal consistency at postpartum time points. The revised STAR-P shows strong psychometric properties, and is suitable for use to evaluate the relationship developed between CHWs and pregnant and postpartum women in an intervention program.
Keywords: STAR, Community health worker, Pregnancy, Psychometric scale adaptation, Exploratory factor analysis
Introduction
Prenatal and postpartum women face a series of physical and mental health challenges [16, 22]. Women in low-income populations, particularly racial and ethnic minorities, experience increased stressors, have fewer resources to navigate those stressors, and more limited health care options, all interacting to produce a population at greater risk of negative health outcomes particularly during and after pregnancy [13, 21]. Social relationships in a pregnant woman’s life may buffer the effect of negative factors by shaping a mother’s perception of the availability of interpersonal support [2]. These social relationships likely also interact with health navigation and subsequent health-seeking behavior [5]. Presumably through a combination of these factors, women who both received more prenatal social support and who were more satisfied with that support reported fewer health problems during pregnancy, and experienced improved birth outcomes [3, 6].
Many states provide expanded health services for women and children during the perinatal period, which creates a public health opportunity to reach at-risk populations to educate, inform behavior, and provide additional social support [18]. Community health workers (CHWs) often act as frontline health navigators in these programs by providing additional services to pregnant and postpartum women [1, 19, 29]. They also are embedded in more robust interventions to improve participant health by educating expectant and postpartum mothers on appropriate health-seeking behavior, and fostering improved social relationships and social support [14, 19].
CHWs are lay health workers recruited from the local community, who often share a common language, ethnicity, or race with their target populations [27]. Due to this common background, it is presumed that CHWs are well-equipped to offer culturally-competent health information, and cultivate a trusting relationship expected to result in lasting changes in the mother’s behavior and knowledge [27]. Despite the importance of building relationships in the CHW model, no scale has been developed to assess the quality of this relationship and instead data on CHW effectiveness has been limited to client satisfaction surveys [12]. This research gap hampers CHW utilization by limiting the understanding of what makes the CHW model effective, particularly for at-risk pregnant and postpartum women [12].
A measurement tool [Scale to Assess the Therapeutic Relationship in Community Mental Health Care (STAR)] was developed and validated to measure the therapeutic relationship between clinicians and patients in community psychiatry settings [11]. As with CHWs, the development of an effective therapeutic relationship represents a vital component for the delivery of interventions, treatment, and health information among clinicians. The STAR, collected from both the perspective of the clinician and the patient, has previously been used to examine the quality of the therapeutic relationship, as a mediator and outcome, in clinical settings [7, 8, 10, 17, 25]. The STAR has been a valuable tool to measure therapeutic relationship, and a more thorough analysis of its psychometric properties when used outside of community psychiatry is warranted.
This manuscript presents findings regarding the application of the participant version of the STAR survey (STAR-P) to evaluate how pregnant and postpartum women perceive their relationship with a CHW during a text-based and in-person intervention study. To do this we first explored whether the factor structure of the original 12-item STAR-P was stable for use in this setting. Conducting an exploratory factor analysis (EFA), and using a set of criteria for evaluating items and factors, we determined that the STAR-P’s validity would be improved through a modification to the original scale, and as a result, we recommended a 9-item version of the STAR-P. We next applied the revised 9-item STAR-P factor structure at data collection time points ranging from a participant’s 2nd trimester to 6 months postpartum to examine if (a) the stability of the factor structure across prenatal assessments and (b) the stability of the overall measure across the intervention.
Methdology
Study Design and Recruitment
The Transdisciplinary Research Consortium for Gulf Resilience on Women’s Health (GROWH) implemented a community-based, participatory research project targeting low-income pregnant women living in Southeastern Louisiana [9]. GROWH was designed to respond to community health concerns following the 2010 Gulf Oil Spill regarding the safety of the seafood, regional air and water, and the health and resilience of pregnant women. An intervention study within GROWH (Building Community Resilience through Disaster Mobile Health) aimed to train CHWs to serve as disaster interventionists and patient navigators in order to evaluate the effect of disaster mobile health technologies and delivery systems on community resilience. CHWs, equipped with mobile health technology, promoted education and access to support and health services beyond the health care system, while providing one-on-one support to expectant women (Fig. 1).
Fig. 1.
GROWH project study design
Central to this GROWH project’s study design was the use of CHWs. Five CHWs were recruited from the study area (Southeast Louisiana parishes: Orleans, Jefferson, Plaquemines, St. Bernard, Terrebonne, and Lafourche), and educated on a competency-based curriculum tailored to the role and function of health navigation, and certified internally on core public health disciplines including subject-specific modules on disaster preparedness, reproductive health, and environmental health. As study partners, CHWs were also responsible for the recruitment of study participants. Mobile health technology was also used to facilitate data collection and provide one-on-one support to the participants during and after their pregnancy.
First-time pregnant women were recruited in the first trimester of pregnancy. Women were recruited from Women, Infants, and Children (WIC) clinics and were required to meet the clinic’s eligibility criteria [31]. The cohort was recruited between July 2013 and January 2014 and subsequently followed from baseline (1st trimester of pregnancy) to study completion (6-months postpartum) by their assigned CHW. CHWs interacted with participants by text messaging and phone calls throughout the study period. Five in-person data collection points were scheduled to collect psychosocial surveys. Each time point represented a specific time period during pregnancy: first trimester (1–13 weeks gestation), second trimester (14–27 weeks gestation), third trimester (28–40 weeks gestation), 6-weeks postpartum, and 6-months postpartum.
Measures
The STAR survey [11] is comprised of two sets of a 12-item scale that measures perception of relationship. One set is completed by the clinician (STAR-C) and the other is completed by the patient (STAR-P). The 12 items on both sets are rated on a scale from 0 (never) to 4 (always). The psychometric properties of STAR have been analyzed to support its internal consistency, test–retest reliability, and validity in the field of community psychiatry [19]. The STAR-P is scored by taking the total STAR-P score and three subscale scores: Positive Collaboration (PCO: a good rapport and shared understanding of goals and the experience of mutual openness and trust), Positive Clinician Input (PCI: extent to which the clinician encourages, regards, supports, listens to and understands the patient) and Non-supportive Clinician Input (NCI: degree to which the patient perceives the clinician cannot empathize with them) [19]. Participants and CHWs completed their section of the STAR at the 2nd–5th time point. The language of the scale was modified to fit the study population by changing “clinician” to “health worker”.
Ethics Statement
This study was approved by the Tulane University Human Research Protection Office. Written consent was obtained at the start of all study data collection.
Statistical Analysis
Exploratory and Confirmatory factor analyses were used to assess the psychometric properties of the STAR-P survey at each separate study time point, independent of each other. The EFA was conducted on the 6-months postpartum time point. This time point was chosen to allow for the longest duration of interaction between the CHW and the participant. The EFA used the maximum likelihood (ML) factor extraction technique and orthogonal rotation method. Indicators of items and factors in the EFA models were evaluated based on four criteria to determine which items and factors would be retained:
Items were evaluated based on their factor loadings. Factor loadings in excess of 0.40 were categorized as fair, above 0.55 were categorized as good, and those in excess of 0.71 were categorized as excellent or high [4].
Based on the criterion that subscales have a simple structure, retained items were required to have meaningful high loadings on a single factor. Items with cross-loadings above 0.30 were excluded.
Items without a meaningful loading on any factor were excluded from the analysis.
Factors with less than 3 items were not considered for further analysis, as recommended by Suhr [24] and Truxillo [26]. Factor loading of the final solution was used to evaluate the interpretability of the factors.
Maximum-likelihood (ML) was used to estimate the parameters in the confirmatory factor analysis (CFA). Model fit of the CFA model was assessed using several tests including the Chi square, root mean square error of approximation (RMSEA), non-normed Fit Index (NNFI), and Bentler’s Comparative Fit Index (CFI). Internal consistency reliability of the adapted items in both instruments was examined using Cronbach’s alpha. The cut off reliability was set at 0.70 [15]. Convergent and divergent validity of the measure was assessed through the two-sided Pearson correlation coefficient. All study hypotheses were tested at the 5 % significance and all analyses was conducted using the Statistical Analysis System (SAS 9.3).
To create a more complete picture of the potential construct validity of the STAR measure within this population, the Pearson correlation coefficient was used to estimate the correlations between the STAR-P score and psychosocial health of the study participants. Anxiety was measured using the State-Trait Anxiety Inventory (STAI), which consists of two separate 20-item self-report scales that measure transient state anxiety and dispositional trait anxiety. Participants are asked to indicate on a four-point scale how they feel right now, at this moment. Higher scores represent more intense state anxiety [23]. The emotional state of the women were evaluated using the positive and negative affect schedule profile of mood states (PANAS-X), which is a survey that assesses how the respondent is currently feeling using different feelings and emotions. Two scales are created, “negative affect” and “positive affect” [28]. Both surveys were administered and collected at each study time point, and the correlation was analyzed with the STAR-P score at the same time point.
Results
The GROWH cohort (n = 141) was predominately young, black, and low-income (Table 1). Table 2 shows the results of statistical analyses for the original 12 items of the STAR-P collected at the final time point of the study (6-months postpartum).
Table 1.
Study demographics (N = 141)
| Characteristics | Study baseline N (%) |
|---|---|
| Age | |
| <25 years old | 92 (65.2) |
| 25–29 | 31 (22.0) |
| 30–34 | 10 (7.1) |
| 35–39 | 8 (5.7) |
| Race | |
| African American | 93 (68.9) |
| White | 35 (25.9) |
| Other | 7 (4.9) |
| Education | |
| No high school diploma | 21 (14.9) |
| High school diploma but no college | 89 (63.1) |
| College | 31 (22.0) |
| Marital status | |
| Married/living with partner | 56 (43) |
| Single/divorced/separated | 78 (57) |
| Health Insurance | |
| Uninsured | 28 (20.3) |
| Private or employer-based | 22 (15.9) |
| Medicaid/bayou health plan | 84 (60.9) |
| Annual family income | |
| <$20,000 | 96 (73.3) |
| $20,000–$29,000 | 14 (10.7) |
| >30,000 | 21 (14.8) |
Table 2.
Statistical analysis results of the original 12-item STAR-P survey at 6 months postpartum
| Label | Item | Mean ± SD |
|---|---|---|
| PCI1 | My health worker speaks with me about my personal goals and goals of my pregnancy | 3.48 ± 0.78 |
| PCO1 | My health worker and I are open with one another | 3.40 ± 0.82 |
| PCO2 | My health worker and I share a trusting relationship | 3.48 ± 0.79 |
| NCI1a | I believe my health worker withholds the truth from me | 3.26 ± 1.44 |
| PCO3 | My health worker and I share an honest relationship | 3.55 ± 0.70 |
| PCO4 | My health worker and I work towards mutually agreed upon goals | 3.38 ± 0.92 |
| NCI2a | My health worker is stern with me when I speak about things that are important to me and my situation | 2.29 ± 1.74 |
| PCO5 | My health worker and I established an understanding of the kind of changes that would be good for me | 3.40 ± 0.90 |
| NCI3a | My health worker is impatient with me | 3.65 ± 1.03 |
| PCI2 | My health worker seems to like me regardless of what I do or say | 3.39 ± 1.15 |
| PCO6 | We agree on what is important for me to work on | 3.46 ± 0.80 |
| PCI3 | I believe my health worker has an understanding of what my experiences have meant to me | 3.53 ± 0.78 |
CFA model summary for the 12-item participant scale
Chi square: 131.15 (p < 0.001); RMSEA Parsimony fit index: 0.12; Bentler-Bonnett NNFI: 0.89; Bentler CFI: 0.91
PCI Positive Clinician Input, PCO Positive Collaboration, NCI Non-supportive Clinician Input
Items were reverse-coded
Exploratory Factor Analysis and Evaluation of the 12-Item STAR-P at 6-Months Postpartum
Based on the fit indices and the residuals (Table 2), CFA of the adapted STAR-P 12-item model did not fit the participant factor structure at 6-months postpartum. Therefore, an EFA examined the underlying factor structure that would explain a better and more efficient fit for the scale (Table 3).
Table 3.
Two exploratory factor analyses (EFA) and one confirmatory factor analysis (CFA) fit results on STAR-P survey data at time point 5 (6 months postpartum) (N = 103)
| Label | First EFA factor loadings |
Second EFA factor loadings |
CFA model summary for the second EFA factor loading | ||||||
|---|---|---|---|---|---|---|---|---|---|
| F1 | F2 | F3 | F1 | F2 | Chi square | RMSEA Parsimony fit index | Bentler-Bonnett NNFI | Bentler CFI | |
| PCO2 | 0.89 | 0.22 | −0.01 | 0.90 | 0.01 | 42.248 p = .023 |
0.078 | 0.97 | 0.98 |
| PCO3 | 0.85 | 0.33 | 0.00 | 0.92 | 0.02 | ||||
| PCI1 | 0.84 | 0.22 | −0.09 | 0.87 | −0.06 | ||||
| PCO1 | 0.80 | 0.30 | 0.01 | 0.86 | 0.03 | ||||
| PCO4 | 0.76 | 0.44 | 0.02 | 0.88 | 0.04 | ||||
| PCO5 | 0.68 | 0.53 | 0.03 | 0.82 | 0.04 | ||||
| PCI3 | 0.58 | 0.74 | 0.01 | Removed | |||||
| PCO6 | 0.53 | 0.67 | 0.05 | Removed | |||||
| NCI1 | −0.02 | −0.07 | 0.87 | −0.06 | 0.86 | ||||
| NCI3 | −0.10 | −0.03 | 0.58 | −0.12 | 0.60 | ||||
| NCI2 | 0.21 | 0.12 | 0.51 | 0.22 | 0.50 | ||||
| PCI2 | 0.24 | 0.36 | −0.36 | Removed | |||||
Significance (H0: No common factors; χ2 = 650.37; p < 0.001. HA: two factors are sufficient; χ2 = 29.02; p = 0.0656)
Bold values indicate the items that ‘load’ on the factor
PCI Positive Clinician Input, PCO Positive Collaboration, NCI Non-supportive Clinician Input, F factor
Six iterations of the EFA model were needed to reach an interpretable factor solution. Three factors emerged in the first EFA model (Table 3). Factor 1 had high loadings on 5 items from the Positive Collaboration scale (PCO1–PCO5) and one item from Positive Clinician Input (PCI1). Factor 2 had high loadings on one item from Positive Collaboration (PCO6) and one item from Positive Clinician Input (PCI3). Factor 3 had high factor loadings on 3 items from the Non-supportive Clinician Input subscale (NCI-NCI3). Item PCO5 had high cross loadings with Factor 2 (r = 0.53) and item PCI3 and PCO6 had high cross loadings with Factor 1 (r = 0.58 and 0.53, respectively). Item PCI2 loaded equally on all three factors (r = 0.24, 0.36, and −0.36, respectively). Items PCI2, PCI3, and PCO6 were removed in the second EFA model. In the second EFA model (Table 3), the factor loadings included 9 items with meaningful and high factor loadings and specified a two-factor solution based on the first model’s findings. The reduction in the number of items led to significant improvements in fit as the Chi square tests for model fit were 650.37, p < 0.0001 and 29.02, p = 0.07.
Confirmatory Factor Analysis on the Abridged 9-Item STAR-P at 6-Months Postpartum
Confirmatory factor analysis was done to assess whether the factor structures from the second EFA adequately explained the relationships among the observed variables characterizing the participant relationship (Table 3). The Chi square test (42.25, p < 0.02) indicated an inadequate model fit. RMSEA of the Parsimony fit index exceeded the cutoff of 0.07 (0.078). The results of the Incremental Index showed good fit with all indices above 0.95 (Bentler and Bonnett’s NNFI = 0.97; Bentler’s CFI = 0.98). Inter-correlations between the two factors in the second EFA were −0.02. Internal consistency of the adapted 9 items using Cronbach’s alpha was 0.84. For the subscales, the reliability was 0.95 for Factor 1 and 0.66 for Factor 2.
Exploratory and Confirmatory Factor Analysis of the Abridged STAR-P at Other Time Points
The next step was to determine if the factor structure of the STAR-P scale at 6 months postpartum was identical with the STAR-P scale at the other time points. In the second trimester, the correlation matrix for the second trimester on the 9 items showed almost perfect correlations (Table 4) and therefore no factor analysis was attempted on the measurements taken during the second trimester.
Table 4.
Inter-correlations between the 9-item STAR-P survey items at second trimester (N = 110)
| PCO1 | PCO2 | PCO3 | PCO4 | PCO5 | PCI1 | NCI1 | NCI2 | NCI3 | |
|---|---|---|---|---|---|---|---|---|---|
| PCO1 | 1 | ||||||||
| PCO2 | 1 | 1 | |||||||
| PCO3 | 1 | 1 | 1 | ||||||
| PCO4 | 1 | 1 | 1 | 1 | |||||
| PCO5 | 0.99999 | 0.99999 | 0.99999 | 0.99999 | 1 | ||||
| PCI1 | 1 | 0.99999 | 0.99999 | 0.99999 | 0.99999 | 1 | |||
| NCI1 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 1 | ||
| NCI2 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 0.99998 | 1 | |
| NCI3 | 0.99999 | 0.99999 | 0.99999 | 0.99999 | 0.99999 | 0.99999 | 0.99998 | 0.99998 | 1 |
PCI Positive Clinician Input, PCO Positive Collaboration, NCI Non-supportive Clinician Input
For third trimester, optimization for the CFA on the 9-item scale could not be completed even after 10 thousand iterations indicating that a fit summary table might not be appropriate, therefore an EFA was conducted (Table 5). Factor 1 had high loadings on 1 item from the Non-Supportive Clinician Input subscale (NCI2). Factor 2 had high factor loadings on the remaining Positive Collaboration scale (PCO1–PCO5) and one item from Positive Clinician Input (PCI1). Factor 3 had high loadings on 2 items from Positive Collaboration scale (PCI3 and PCO3). Items NCI1 and NCI2 did not load on any of the 3 factors. NCI2 was dropped since it was the only item with a high loading on Factor 1. PCO3 and PCO5 had high cross loadings (r = 0.38, r = 0.29) with Factor 3 and PCO5 were also dropped for the second EFA model at third trimester. The second EFA model (Table 5) resulted in a one-factor solution. Chi square tests for model fit were 260.52, p < 0.0001 and 1.7423, p = 0.419. The CFA for the 4 item, one-factor scale at the third trimester (Table 5) showed good Absolute fit index (Chi square = 1.42, p = 0.41), Parsimony fit index (RMSEA = 0.001), and Absolute fit index (Bentler and Bonnett’s NNFI = 1.0; Bentler’s CFI = 1.0).
Table 5.
EFA and CFA fit results on 9-item STAR-P survey at time point 3 and 4
| Label | Time point 3 (3rd trimester) (n = 99) | Time point 4 (6-weeks postpartum) (n = 103) | ||||
|---|---|---|---|---|---|---|
| EFA factor loadings | Revised EFA factor loadings | EFA factor loadings | ||||
| F1 | F2 | F3 | F1 | F1 | F2 | |
| NCI1 | 0.15 | −0.06 | 0.02 | Removed | −0.13 | 0.86 |
| NCI2 | 1 | 0 | 0 | Removed | 0.26 | 0.38 |
| NCI3 | 0.12 | −0.17 | 0.07 | Removed | −0.34 | 0.57 |
| PCI1 | 0.11 | 0.71 | 0.25 | 0.72 | 0.76 | −0.1 |
| PCO1 | 0.2 | 0.92 | −0.1 | 0.94 | 0.86 | −0.1 |
| PCO2 | 0.15 | 0.87 | −0.07 | 0.88 | 0.86 | 0 |
| PCO3 | 0.08 | 0.77 | −0.38 | Removed | 0.92 | −0.06 |
| PCO4 | 0.21 | 0.8 | 0.26 | 0.82 | 0.92 | −0.07 |
| PCO5 | 0.16 | 0.82 | 0.29 | Removed | 0.84 | −0.06 |
“My health worker withholds the truth from me” from time point 4 to time point 5
Bold values indicate the items that ‘load’ on the factor
For the 4th data collection point (6-weeks postpartum), EFA on the 9-item STAR-P scale (Table 5) showed a two-factor solution similar to the factor solution at 6 months postpartum. Factor 1 had high loadings on five items from the Positive Collaboration scale (PCO1–PCO5) and one item from Positive Clinician Input (PCI1). Factor 2 had high loadings on the three NCI items (NCI1–NCI3). There were no cross loadings greater than 0.30 and loadings on the factors ranged from 0.30 to 0.91. Chi square tests for model fit were 604.78 p < 0.0001 and 27.89, p = 0.09. Inter-correlation between the two factors in the EFA was −0.23. Model fit indices of CFA of the two-factor model at 6-weeks postpartum do not show as good as the fit indices at 6 months postpartum with Chi square test statistic at 48.44 (p < 0.005), RMSEA at 0.09, Bentler and Bonnett’s NNFI at 0.95 and Bentler’s CFI at 0.89.
Measures of the 9-item STAR-P collected at the two post-partum time points that demonstrated a consistent factor structure (6-weeks postpartum and 6-month postpartum) were compared (Table 6). While average scores for the positive interaction factor (time point 4 Factor 1 mean 20.95 ± 4.29; time point 5 Factor 1 mean 20.68 ± 4.38) and reverse-coded negative interaction factor (time point 4 Factor 2 mean 9.50 ± 2.8; time point 5 Factor 3 mean 9.20 ± 3.27) decreased, there were no significant differences in the total average scores by factor. However, significant differences were found on the average scores for two specific items (compared using an analysis of variance). Participants on average tended to rate their relationships worse on the specific questions, “My health worker and I are open with one another”, and “I believe”.
Table 6.
Average scores at time point 4 and 5 on 9-item STAR-P survey
| 9-Item abridged STAR-P survey with question wording | Time point 4 (6-weeks postpartum) |
Time point 5 (6-months postpartum) |
||
|---|---|---|---|---|
| Mean | SD | Mean | SD | |
| Factor 1-1. My health worker and I are open with one another* | 3.58 | 0.70 | 3.40 | 0.82 |
| Factor 1-2. My health worker and I share a trusting relationship | 3.51 | 0.83 | 3.48 | 0.79 |
| Factor 1-3. My health worker and I share an honest relationship | 3.54 | 0.73 | 3.55 | 0.7 |
| Factor 1-4. My health worker and I work towards mutually agreed upon goals | 3.39 | 0.92 | 3.38 | 0.92 |
| Factor 1-5. My health worker and I established an understanding of the kind of changes that would be good for me | 3.44 | 0.89 | 3.4 | 0.9 |
| Factor 1-6. My health worker speaks with me about my personal goals and goals of my pregnancy | 3.49 | 0.77 | 3.48 | 0.78 |
| Factor 2-1. I believe my health worker withholds the truth from mea* | 3.55 | 1.14 | 3.26 | 1.44 |
| Factor 2-2. My health worker is stern with me when I speak about things that are important to me and my situationa | 2.26 | 1.77 | 2.29 | 1.74 |
| Factor 2-3. My health worker is impatient with mea | 3.68 | 0.87 | 3.65 | 1.03 |
| Factor 1 (positive interaction) Total | 20.95 | 4.29 | 20.68 | 4.38 |
| Factor 2 (negative interaction) Total | 9.50 | 2.80 | 9.20 | 3.27 |
p value significant differences between time points (p < 0.05)
Items on the Factor 2 were reverse-coded
Finally, correlations of anxiety (S-STAI) and positive emotional state (positive affect) and negative emotional state (negative affect) with the STAR-P score were calculated for the validated time point at 6-months postpartum (Table 7). Both the positive interaction factor and the total STAR-P score were significantly, negatively correlated with scores of negative affect, while only the positive interaction factor was significantly, positively correlated with scores of positive affect. Meanwhile, both the positive interaction factor, and the total STAR-P score were significantly, negatively correlated with state anxiety.
Table 7.
Pearson correlations between STAR-P and psychosocial measures at 6-months postpartum
| STAR-P positive interaction factor | STAR-P negative interaction factor | STAR-P total scorea | |
|---|---|---|---|
| Negative affect (PANAS-X) | −0.29* | 0.05 | −0.21* |
| Positive affect (PANAS-X) | 0.27* | −0.11 | 0.15 |
| State anxiety (STAI) | −0.37* | 0.04 | −0.28* |
Significant p < 0.05
Total score was calculated by adding reverse-coded negative interaction factor with the positive interaction factor
Discussion
The aim of the analyses presented in this manuscript was to determine if an instrument intended to measure therapeutic relationship in a community psychiatry setting has applicability in a CHW-led intervention for pregnant and postpartum women. While a 12-item STAR-P scale has been validated in community psychiatry setting this structure proved to be unstable in our population. Using a systematic set of criteria for evaluating the EFA models, two stable factors emerged (rather than the three found in the original STAR-P, [11]), which we labeled as positive interactions and negative interactions. Using only items contained on those two factors, an abridged 9-item STAR-P was constructed.
Analyzing the abridged 9-item STAR-P on earlier study data time points indicated that the ability of participants to evaluate the relationship with their assigned CHW evolved over a multi-month intervention. Results of first STAR-P data collection point (collected at 2nd trimester) showed almost perfect correlation with each other, as participants gave almost no distinction in their evaluation of positive or negative aspects of their relationship. This suggests that even after several weeks of interaction, the study cohort offered no consistency in their perception of their relationship with their assigned CHW. By the third trimester a one-factor model was found, indicating participants were reliably able to indicate positive attributes of their CHW relationship, but were unable to reliably evaluate negative attributes. Finally, by the next STAR-P data collection point after delivery (6-weeks postpartum), a two-factor model emerged, as participants consistently evaluated both positive and negative aspects of their CHW relationship. This same factor model was found at the final data collection point at 6-months postpartum.
The effect of time on STAR’s reliability illustrates the development of a therapeutic relationship between subjects and their CHWs. Several months of interaction between CHW and participants were needed until the participants reliably evaluated the therapeutic relationship. This is contrary to a previous study of the STAR-P in community psychiatry where the factor structure of the survey response did not change over a 12-month intervention period [20]. However, the study population of this longitudinal intervention had a relationship with their care team prior to data collection [20].
The findings presented in this manuscript indicate systematic differences in how CHWs and clinicians interact (in terms of both frequency and intensity) with their respective clients. In the context of community psychiatry, patients may see their clinician at regular set intervals and are exposed to a more structured interaction format that occurs at specified locations and times. On the other hand, CHWs in the GROWH project only formally met with participants for five data collection time points, with the majority of communication occurring through text-messaging and phone calls. As a result, we are unable to differentiate whether the (a) number of study interactions or (b) the duration of the study itself was driving the development of STAR’s reliability and validity. Future CHW intervention studies using this validated measurement can now compare whether a range of intensity of interactions and differences between in person and text-based/social media based interactions result in different patterns of stability of the STAR-P and the therapeutic relationship. Future research could also test the factor structure of STAR-P in studies with a longer duration than the GROWH project to analyze whether the therapeutic relationship construct further evolves at later points into a more sophisticated 3- or 4-factor model. Alternatively it is feasible that the relationship grows more nuanced, the participant is better able to differentiate different aspects of the relationship with the CHW. More intensive or in-person CHW interaction might also result in greater stability of the therapeutic relationship, and not demonstrate any evolution, similar to findings in community psychiatry [30]. Lastly, the quality of the relationship between a pregnant woman and a CHW may also reflect a range of different factors related to the pregnancy itself including the uniqueness of pregnancy as its own developmental time period, whether or not the pregnancy was wanted or planned, the mother’s own stress level and previous exposure to health navigators.
Finally, the construct validity should be further explored. While our findings show that women with more positive psychosocial states (less anxiety and more positive affect) tended to rate their CHW relationships better, future studies could better assess the predictive validity of this new scale. It was originally created, and shown to be associated with factors such as treatment adherence, admissions, and symptom severity [11]. However, no comparable measure was found in this present study.
In summary, the current investigation found that the factor structure of the original 12-item STAR-P was unstable, but an abridged version of the instrument had excellent psychometric properties in the postpartum period of a CHW cohort intervention study. In the abridged version, however, there was no stability in psychometric properties in the prenatal period, suggesting participants had not yet formulated a stable opinion of their relationship with their CHW. Despite this limitation, the STAR-P tool offers significant potential in evaluating how relationships develop between CHWs and their long-term clients. Additional research examining factors that influence the development of therapeutic relationship represent important next steps. Most importantly, future research can explore whether CHW’s relationship influences project/intervention success, and explore the mechanisms behind successful relationship building. Finally, the STAR-P can be strengthened by validating it for specific subpopulations other than pregnant/postpartum women.
One of the central benefits of embedding CHWs in a study design is the development of a therapeutic relationship which, if meaningful, could help reduce health risks and increase access and healthcare utilization. There are limited tools to measure the therapeutic relationship in a non-clinical setting such as the interaction between CHWs and pregnant participants. Therefore, by validating the STAR-P instrument for our specific participants, this study is contributing an essential tool for future public health research that can accurately measure the complex interaction between CHWs and pregnant/postpartum participants.
Acknowledgments
This research was supported by the National Institutes of Health under Grant Number 5U19ES020677. The authors thanks to the CHWs Starleen Maharaj-Lewis, Christine Dennis, Onita Harris, Courtney Schultheis, Patricia Davis, and Kimisha Sawyer for their commitment to the study subjects and aiding in data collection. Special thanks to Farah Arosemena and Dr. Hannah Covert, for their guidance and expertise during the design and implementation of the study.
Funding This study was funded by the National Institutes of Health (5U19ES020677).
Footnotes
Compliance with Ethical Standards
Conflict of interest The authors declare that they have no conflict of interest.
References
- 1.Braun R, Catalani C, Wimbush J, Israelski D. Community health workers and mobile technology: A systematic review of the literature. PloS One. 2013;8(6):e65772. doi: 10.1371/journal.pone.0065772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychological Bulletin. 1985;98(2):310. [PubMed] [Google Scholar]
- 3.Collins NL, Dunkel-Schetter C, Lobel M, Scrimshaw SC. Social support in pregnancy: Psychosocial correlates of birth outcomes and postpartum depression. Journal of Personality and Social Psychology. 1993;65(6):1243. doi: 10.1037//0022-3514.65.6.1243. [DOI] [PubMed] [Google Scholar]
- 4.Comrey AL, Lee HB. A first course in factor analysis. Psychology Press; 2013. [Google Scholar]
- 5.Dunkel-Schetter C, Sagrestano LM, Feldman PA, Killingsworth C. Handbook of social support and the family. Springer US: 1996. Social support and pregnancy; pp. 375–412. [Google Scholar]
- 6.Feldman PJ, Dunkel-Schetter C, Sandman CA, Wadhwa PD. Maternal social support predicts birth weight and fetal growth in human pregnancy. Psychosomatic Medicine. 2000;62(5):715–725. doi: 10.1097/00006842-200009000-00016. [DOI] [PubMed] [Google Scholar]
- 7.Gairing SK, Jäger M, Ketteler D, Rössler W, Theodoridou A. Evaluation of the German version of the STAR scale. Psychiatrische Praxis. 2011;38(4):178–184. doi: 10.1055/s-0030-1265979. [DOI] [PubMed] [Google Scholar]
- 8.Kvrgic S, Beck E-M, Cavelti M, Kossowsky J, Stieglitz R-D, Vauth R. Focusing on the adult attachment style in schizophrenia in community mental health centres: Validation of the Psychosis Attachment Measure (PAM) in a German-speaking sample. International Journal of Social Psychiatry. 2012;58(4):362–373. doi: 10.1177/0020764011399004. [DOI] [PubMed] [Google Scholar]
- 9.Lichtveld MY, Arosemena FA. Resilience in the Aftermath of the Gulf of Mexico Oil Spill: An Academic-Community Partnership to Improve Health Education, Social Support, Access to Care, and Disaster Preparedness; Paper presented at the International Oil Spill Conference Proceedings.2014. [Google Scholar]
- 10.Loos S, Kilian R, Becker T, Janssen B, Freyberger H, Spiessl H, et al. Psychometric properties of the German version of the scale to assess the therapeutic relationship in community mental health care (D-STAR) European Journal of Psychological Assessment. 2015;28(4):255–261. [Google Scholar]
- 11.Mcguire-Snieckus R, McCABE R, Catty J, Hansson L, Priebe S. A new scale to assess the therapeutic relationship in community mental health care: STAR. Psychological Medicine. 2007;37(01):85–95. doi: 10.1017/S0033291706009299. [DOI] [PubMed] [Google Scholar]
- 12.Nemcek MA, Sabatier R. State of evaluation: community health workers. Public Health Nursing. 2003;20(4):260–270. doi: 10.1046/j.1525-1446.2003.20403.x. [DOI] [PubMed] [Google Scholar]
- 13.Norbeck JS, Anderson NJ. Psychosocial predictors of pregnancy outcomes in low-income black, Hispanic, and white women. Nursing Research. 1989;38(4):204–209. [PubMed] [Google Scholar]
- 14.Norris SL, Chowdhury FM, Van Le K, Horsley T, Brownstein JN, Zhang X, Satterfield DW. Effectiveness of community health workers in the care of persons with diabetes. Diabetic Medicine. 2006;23(5):544–556. doi: 10.1111/j.1464-5491.2006.01845.x. [DOI] [PubMed] [Google Scholar]
- 15.Nunnally JC, Bernstein IH, Berge JMT. Psychometric theory. Vol. 226. New York: McGraw-Hill; 1967. [Google Scholar]
- 16.O’hara MW, Swain AM. Rates and risk of postpartum depression-a meta-analysis. International Review of Psychiatry. 1996;8(1):37–54. [Google Scholar]
- 17.Park SG, Derman M, Dixon LB, Brown CH, Klingaman EA, Fang LJ, et al. Factors associated with shared decision-making preferences among veterans with serious mental illness. Psychiatric Services. 2014;65(12):1409–1413. doi: 10.1176/appi.ps.201400131. [DOI] [PubMed] [Google Scholar]
- 18.Roman LA, Gardiner JC, Lindsay JK, Moore JS, Luo Z, Baer LJ, Fitzgerald HE. Alleviating perinatal depressive symptoms and stress: A nurse-community health worker randomized trial. Archives of Women’s Mental Health. 2009;12(6):379–391. doi: 10.1007/s00737-009-0083-4. [DOI] [PubMed] [Google Scholar]
- 19.Roman LA, Raffo JE, Meghea CI. Maternal perceptions of help from home visits by Nurse-Community Health Worker Teams. American Journal of Public Health. 2012;102(4):643–645. doi: 10.2105/AJPH.2011.300455. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rugkåsa J, Molodynski A, Yeeles K, Vazquez Montes M, Visser C, Burns T. Community treatment orders: Clinical and social outcomes, and a subgroup analysis from the OCTET RCT. Acta Psychiatrica Scandinavica. 2015;131(5):321–329. doi: 10.1111/acps.12373. [DOI] [PubMed] [Google Scholar]
- 21.Scheppers E, Van Dongen E, Dekker J, Geertzen J, Dekker J. Potential barriers to the use of health services among ethnic minorities: A review. Family Practice. 2006;23(3):325–348. doi: 10.1093/fampra/cmi113. [DOI] [PubMed] [Google Scholar]
- 22.Slusser W. Breastfeeding and maternal and infant health outcomes in developed countries. AAP Grand Rounds. 2007;18(2):15–16. [Google Scholar]
- 23.Spielberger CD, Gorsuch RL. State-trait anxiety inventory for adults: Manual, instrument, and scoring guide. Menlo Park: Mind Garden, Incorporated; 1983. [Google Scholar]
- 24.Suhr DD. Principal component analysis vs. exploratory factor analysis; Paper presented at the SUGI 30 Proceedings; 2005. http://www2.sas.com/proceedings/sugi30/203-30.pdf. [Google Scholar]
- 25.Theodoridou A, Schlatter F, Ajdacic V, Rössler W, Jäger M. Therapeutic relationship in the context of perceived coercion in a psychiatric population. Psychiatry Research. 2012;200(2):939–944. doi: 10.1016/j.psychres.2012.04.012. [DOI] [PubMed] [Google Scholar]
- 26.Truxillo C. Multivariate statistical methods: Practical research applications. Cary, NC: SAS Institute Inc; 2008. [Google Scholar]
- 27.Viswanathan M, Kraschnewski JL, Nishikawa B, Morgan LC, Honeycutt AA, Thieda P, Jonas DE. Outcomes and costs of community health worker interventions: A systematic review. Medical Care. 2010;48(9):792–808. doi: 10.1097/MLR.0b013e3181e35b51. [DOI] [PubMed] [Google Scholar]
- 28.Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988;54:1063–1070. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
- 29.Whitley EM, Everhart RM, Wright RA. Measuring return on investment of outreach by community health workers. Journal of Health Care for the Poor and Underserved. 2006;17(1):6–15. doi: 10.1353/hpu.2006.0015. [DOI] [PubMed] [Google Scholar]
- 30.Wittorf A, Jakobi UE, Bannert KK, Bechdolf A, Müller BW, Sartory G, Herrlich J. Does the cognitive dispute of psychotic symptoms do harm to the therapeutic alliance? The Journal of Nervous and Mental Disease. 2010;198(7):478–485. doi: 10.1097/NMD.0b013e3181e4f526. [DOI] [PubMed] [Google Scholar]
- 31.Women, Infants and Children: WIC Eligibility Requirements. [Retrieved April 29, 2015];2015 from http://www.fns.usda.gov/wic/wic-eligibility-requirements.