Development of the Practice Readiness to Evaluate and address Perinatal Depression (PREPD) Assessment

Abstract

Objective:

Perinatal depression is a common pregnancy complication and universal screening is recommended. The Practice Readiness to Evaluate and address Perinatal Depression (PREPD) was developed to measure obstetric practice readiness to integrate depression care into workflows. Objectives were to describe: (1) the PREPD; (2) associated characteristics by readiness level; and (3) use of the assessment to measure change.

Method:

The PREPD has four components, each scored to a 16-point maximum: (1) Environmental Scan (10% of PREPD); (2) Depression Detection, Assessment, and Treatment Questionnaire (30%); (3) Depression-related Policies Questionnaire (10%); and (4) Chart Abstraction (50%). Components were weighted and summed for an overall score. Summary and component scores were calculated by patient, practice, and provider.

Results:

Average overall PREPD score was 7.3/16 (range: 4.8-9.9); scores varied between practices. The Environmental Scan averaged 2.0/16 (range: 0-5.2); Detection, Assessment, and Treatment averaged 8.3/16 (range: 3.0-11.5); Chart Abstraction averaged 7.2/16 (range: 5.1-9.6); and Depression-related Policies averaged 10.4/16 (range: 7.5-15).

Conclusion:

We found wide variation in obstetric practices’ readiness to implement interventions for depression; most were minimally prepared. These data may be used to tailor practice intervention goals and as benchmarks with which to measure changes in integration of depression care over time.

1. INTRODUCTION

Perinatal depression is the most common complication of pregnancy, negatively affecting birth, infant, and child outcomes (Britton et al., 2001; Deave et al., 2008; Grote et al., 2010). Upwards of one in seven women are affected by perinatal depression, with most not receiving treatment (Gavin et al., 2005; Rowan et al., 2012). Although universal screening for depression has been recommended by the American College of Obstetricians and Gynecologists (ACOG) (2018) and other professional societies (Earls et al., 2010; Siu et al., 2016), screening alone is inadequate to improve treatment rates or outcomes (Byatt et al., 2015) and needs to be done in the context of systems of care, ensuring that women receive necessary treatment (Kendig et al., 2017). In 2016, the Maternal Mental Health Safety Bundle was created to help better identify and address perinatal mood and anxiety disorders in obstetric settings (Kendig et al., 2017). Concurrently, there have been efforts to create programs to help obstetric practices integrate depression screening, assessment, and treatment into perinatal care (Byatt et al., 2016a; Byatt et al., 2015; Byatt et al., 2017; Grote et al., 2010). This includes an ongoing randomized controlled trial (RCT), entitled the PRogram In Support of Moms (PRISM) (Moore Simas et al., 2019), developed by our team (Byatt et al., 2017; Byatt et al., 2016b).

Because health care organizations are complex, evolving systems in which one-size-fits-none (Chillers, 1998; Maguire et al., 2006; McDaniel and Driebe, 2001; Plsek, 2017; Stroebel et al., 2005), it is important for practices to understand their readiness for change to tailor their implementation processes (Holt et al., 2007; Scott et al., 2017; Shea CM, 2014). This includes identifying current approaches to patient care and policies and their strengths and weaknesses. Practice readiness is defined in organizational change literature as, “the extent to which organizational members are psychologically and behaviorally prepared to implement organizational change” (Shea CM, 2014; Weiner BJ, 2008). Examining practice readiness helps practices understand their unique culture and their specific barriers to progress.

Tools designed to understand and quantify practice readiness are used in primary care settings (Scott et al., 2017; Substance Abuse and Mental Health Services Administration), yet such assessments have not been adapted for obstetric practices. There is a need to develop such tools to help ensure the success of programs designed to integrate perinatal depression care into obstetric settings, like PRISM. These tools can also measure practices’ changes over time for quality improvement.

To ensure the success of PRISM implementation and create a resource with which to evaluate its ongoing progress, we developed the Practice Readiness to Evaluate and address Perinatal Depression (PREPD) assessment. The PREPD assessment was created to align with the recommendations of the Council’s Maternal Mental Health Safety Bundle (Kendig et al., 2017). Study objectives were to: (1) describe the PREPD assessment tool and the process of using the tool; (2) describe provider, practice, and patient characteristics that associate with obstetric practice readiness for addressing depression; and, (3) demonstrate how the PREPD assessment can be used by practices to examine change over time.

2. METHODS

The PREPD assessment was developed and completed as part of the PRISM study, a cluster randomized controlled trial (RCT) to assess the extent to which two interventions may differentially improve perinatal depression outcomes. The PRISM study randomized obstetric practices from across Massachusetts into an intervention arm and enhanced usual care arm as detailed elsewhere (Moore Simas et al., 2019). In brief, practices randomized to the enhanced usual care arm were exposed to Massachusetts Child Psychiatry Access Program (MCPAP) for Moms, a statewide Perinatal Psychiatry Access Program that assists front-line providers in caring for perinatal mental health conditions in their patients through consultation, training, and resource and referrals (Byatt et al., 2016a; Byatt et al., 2018). Practices randomized to the intervention (or PRISM) arm were exposed to the PRISM intervention which builds on MCPAP for Moms to help obstetric practices integrate depression care into their workflow through proactive implementation assistance. Study results will be forthcoming in other manuscripts.

The Practice Readiness to Evaluate and address Perinatal Depression (PREPD) assessment was developed as a tool to facilitate the implementation assistance in the intervention/PRISM arm. It aims to: (1) evaluate how obstetric practices identify, assess, and treat perinatal depression; (2) identify strengths and weaknesses within a practice, including existing approaches to perinatal depression care; and (3) track the progress of practices towards achieving depression care goals and PRISM study outcomes. The full PREPD assessment tool is available in Supplementary Figure 1.

The data we present here were collected as part of the PRISM RCT. Data were collected from 12 practices, 351 patients served by those practices, and 248 providers from those practices from September 2016 - June 2017. Demographic details can be found in Supplementary Table 1 and an overview of how each population specifically contributed to the PREPD data collection can be found in Supplementary Table 2. The study was approved by University of Massachusetts Medical School Institutional Review Board.

2.1. Overview

In developing the PREPD assessment, we used our prior experience with the 10-step “Addressing Problems Through Organizational Change” (APTOC) model to guide us (Guydish et al., 2012). This model has established success in providing change management support to organizations that aim to integrate behavioral modifications into their practices (e.g., smoking cessation initiatives). Individual items in the PREPD assessment were adapted directly from this program’s tools and questionnaires for use in addressing depression in obstetric settings.

The PREPD is structured to capture quantitative data that may then be used by practices to customize changes in their efforts to address perinatal depression in obstetric settings. As noted, individual questions were developed based on our prior work with APTOC-derived models and meant to capture practice and provider workflows relating to detecting, assessing, treating, and following depression in perinatal patients. They were split into four components, based on theorized content domains and data collection methodologies (Supplementary Table 2): (1) the Environmental Scan; (2) the Detection, Assessment and Treatment; (3) the Depression-related Policies; and (4) the Patient Chart Abstraction (Table 1). The Chart Abstraction is further broken down into six sub-components, providing a more descriptive picture of the practice’s activities: (a) Depression Detection; (b) Bipolar Detection; (c) Depression Assessment; (d) Depression Treatment; (e) Depression Follow-up and Monitoring; and (f) Transfer of Care.

Table 1 -. Practice Readiness to Evaluate and address Perinatal Depression (PREPD) components and their contribution to the overall Practice Readiness Index (PRI) score.

Details each PREPD subcomponent, their procedures and scoring methodology, and its contribution (weight) to the overall PRI score.

Component	Description	Data collection procedures	Scoring	Weight
I. Environmental Scan	How practices use physical environment (e.g., web & media presence, patient communication) to increase awareness of depression and of avenues for help-seeking.	Study staff review: 1) physical practice locations for print media in patient or staff view, 2) other patient-facing media used by the practice.	Based on the percentage of available areas that contain media, variety in media content, and use of media	10%
II. Detection, Assessment, and Treatment	Assesses activities occurring in the practice related to depression care.	Practice leader(s) complete a questionnaire about: 1) patient materials, 2) screening tools and procedures, 3) practices for documentation, 4) treatment and monitoring practices.	Dichotomous and Likert-scale questions yield a summary score	30%
III. Depression-Related Policies	Assesses established policies and procedures	Practice leader(s) complete a questionnaire about express practice policies for: 1) depression screening, 2) assessment, 3) patient engagement, 4) monitoring, 5) treatment.	Dichotomous and Likert-scale questions yield a summary score	10%
IV. Chart Abstraction	Assesses documentation of depression care patient charts	Study staff abstract chart data about: 1) depression screening time-points (sub-component IV-a), 2) bipolar screening (sub-component IV-b), 3) assessment & diagnosis (sub-component IV-c), 4) treatment (sub-component IV-d), 5) follow-up & monitoring (sub-component IV-e), 6) maintenance or transfer of mental health care postpartum (sub-component IV-f)	Chart abstraction form calculates a ratio of actual to available points for a summary score*	50%
Supplementary components
The Provider Survey	Assesses behaviors and attitudes towards perinatal depression care.	Individual providers complete a questionnaire about: 1) demographics, 2) depression screening practices, 3) acceptability of screening, 4) perceived gaps in referral supports, 5) treatment approaches, 6) attitudes, 7) practice culture and leadership.	N/A	N/A

^*See Supplementary Table 1 for details

The PREPD was developed by our expert team of perinatal psychiatrists, obstetrician/gynecologists, epidemiologists, and health services researchers with prior experience with APTOC-derived tools and studies to help practices and organizations implement new procedures to achieve their behavioral change goals.

2.2. Component I: Environmental Scan

The Environmental Scan examines how practices use their physical environment, web and media presence, and patient communication to increase awareness of depression and of avenues for help-seeking. It includes a review of physical practice locations that contain any print media in patient or staff view. Other patient-facing media used by the practice, including programming, websites, or social media, are reviewed. Scoring is based on the number of areas that contain media, the variety in media content, and use of media to educate patients about depression. Scoring methodologies account for variations in the amount of physical space.

To complete the environmental scan, study staff walked through each practice, inspected all designated areas for the presence of media, asked practice personnel about the use of electronic media and, when possible, reviewed this media for the presence of depression-related information.

2.3. Component II: Detection, Assessment, and Treatment

This questionnaire is designed to assess the activities occurring in the practice related to depression care. It was completed by a practice leader that knows the practice workflow. The questionnaire includes dichotomous and Likert-type questions about: 1) materials given routinely to all patients, 2) procedures for screening with validated questionnaires for depression and bipolar disorder (e.g., the Edinburgh Postnatal Depression Scale [EPDS], the Mood Disorder Questionnaire [MDQ]), 3) practices for documentation of screening and treatment, and 4) treatment and monitoring practices. Response items were weighted based on input from our expert team.

2.4. Component III: Depression-related Policies

Completed by practice leadership, this questionnaire is designed to assess the practice’s established policies and procedures for depression screening, assessment, patient engagement, monitoring, and treatment. Implementation and scoring are similar to Component II.

2.5. Component IV: Chart Abstraction

The Chart Abstraction assesses the extent to which depression care is documented in patient medical records. An overall score as well as six sub-component scores are calculated (Supplementary Table 3). To complete, the medical charts of 30 patients per practice were reviewed and data abstracted using a standardized form. Charts selected for abstraction included only those women who began prenatal care with the practice before 20 weeks gestational age. A HIPAA waiver was approved to complete Component IV.

Study staff abstracted data regarding: 1) depression screening time-points, 2) bipolar screening, 3) assessment, diagnosis, treatment, follow-up, and 4) maintenance or transfer of mental health care postpartum. Charts were reviewed from the initiation of prenatal care through up to 13 months postpartum for the index pregnancy.

Several factors unrelated to depression care quality may impact scoring, including that patients may enter or leave the practice at different points in pregnancy and vary in their experience of depression. Therefore, scoring is reported as a ratio of the number of points earned relative to the number of potential points.

2.6. Summary scoring: The Practice Readiness Index

Components I-IV of the PREPD assessment generate a summary score called the Practice Readiness Index (PRI). The PRI indicates the overall extent to which an individual practice has integrated depression care and is a benchmark against which to measure change prospectively. Supplementary Figure 2 illustrates the procedures used to calculate the PRI. Components I-III each have potential point values of 16. Component IV sums the scores from individual charts into a single ratio, which is multiplied by 16 to scale the score to a comparable potential point value of the other sections. More information on how the overall ratio for Component IV is calculated can be found in Supplementary Table 1.

The PREPD sections are weighted as follows: 10% for the Environmental Scan (Component I); 30% for Detection, Assessment and Treatment Questionnaire (II); 10% for Depression-related Policies Questionnaire (III); and 50% for Chart Abstraction (IV) (Supplementary Figure 2). The relative weights were established based on input from our expert team and practice-based intervention extant literature, which emphasizes chart reviews as the most reliable and objective indicator of patient care and practice policy (Duffy et al., 2008; Ragazzi et al., 2011). Further, the Chart Abstraction component is also the only section that directly incorporates patient results. Therefore, it was weighted most heavily in our design. Following weighting, resultant points are summed to derive the PRI.

While the four components are used to create the PRI, each can also be used individually to provide insight into different parts of depression care.

2.7. Descriptive Scoring: Interpretation of the PRI scores for practices

Based on iterative feedback from all study practices during the development process (conducted via in-person meetings between our study team and the obstetric practices, during which practices would discuss their experiences and suggestions), we created an approach for translating the quantitative PREPD results into practical, descriptive, and qualitative feedback. To help practices develop action plans for change and have a benchmark for ongoing self-assessment after intervention implementation, numerical scores were translated into a color-coded visual management system (Figure 1).

Figure 1 -. Example of the PREPD Assessment as a visual management system, used to give practice qualitative and quantitative feedback.

After 2 or more administrations of the PREPD assessment, practices can use their data to see how their readiness level has changed over time. This can help them tailor further efforts in the integration of depression care.

For this purpose, the points total for each component and sub-component were translated into categorical readiness ratings, or qualitative/verbal feedback. Components I-III were divided into four categories (Supplementary Table 4a), for a simple way to provide feedback into manageable categories for practices. The categories were as follows: 0-4 points (0-25%) is Not Prepared, 5-8 points (25.1-50%) is Minimally Prepared, 9-12 points (50.1-75%) is Moderately Prepared, and 13-16 points (75.1-100%) is Highly Prepared. Additionally, the six sub-components of the Chart Abstraction (Component IV) were divided into similar categories. All-subject ratios were converted to percentages for a level of readiness (Supplementary Table 4b): 0%–25% is Not Prepared, 26%–50% is Minimally Prepared, 51%–75% is Moderately Prepared, and 76%–100% is Highly Prepared.

Using the color-coded qualitative feedback tool depicted in Figure 1, a score of 100% on any individual component/sub-component is translated into a qualitative “score” that reads as “Highly Prepared” and is shaded green. Conversely, a score of 40% is translated into a “Minimally Prepared” score and shaded red to indicate providers that they should “stop” and consider potential further interventions. These visual and verbal cues, as well as the ability to easily see changes over time, allow for practices to follow their progress and easily identify areas for further improvement. An example practice from the PRISM study has been shown in Figure 1; this practice was selected for demonstration as it shows a range of changes from baseline measures and, thus, the use of the color-coded visual management system to illustrate change.

2.8. The Provider Survey

To further help tailor implementation procedure, we collected data from the providers. The Provider Survey is a self-administered questionnaire with questions assessing demographics, behaviors, attitudes, and perceptions towards perinatal depression care. The survey contains specific questions regarding depression screening practices, acceptability of screening, perceived gaps in referral supports, treatment approaches, provider attitudes, and practice culture and leadership. The Provider Survey was completed by obstetric providers in the PRISM study (physicians, nurse practitioners, nurse midwives, and physician assistants).

All data for the PREPD assessment were entered into a REDCap (Harris et al., 2019; Harris et al., 2009) database by study staff or providers responding to the questionnaire.

2.9. Data Collection and Analyses

Development of new assessments for health research are multi-phasic, iterative processes (Boateng et al., 2018). In this study, we evaluated the PREPD assessment’s initial ability to perform as its design intends, to behave in a predictable manner, and, thus, to show evidence of construct validity (Person et al., 2015). This was done by comparing results of the Provider Survey directly with the PREPD assessment scores. The responses collected from providers captured their individual readiness to evaluate and address perinatal depression. Items in the Provider Survey that were more likely to be predictors of high practice readiness for PRISM implementation, such as provider knowledge of when to screen for depression or administer evidenced-based treatment, should be associated with higher PREPD scores.

Therefore, we examined associations between practice and provider characteristics, provider attitudes, and provider self-efficacy, and the PREPD assessment scores. This was done by first dividing practices up into tertiles by PRI score (4 practices per tertile), categorizing into three groups of ascending readiness: tier 1 (average PRI = 5.5), tier 2 (PRI = 7.4), and tier 3 (PRI = 9.0). Then, practice, provider, and patient summary statistics were calculated and reported by PRI tier. Finally, associations between Provider Survey items and PRI were examined and reported by tier. Scores were evaluated using correlation coefficients between continuous variables and one-way ANOVAs for categorical variables. Ordered categories, such as patient education, were evaluated against PRI scores using trend tests to examine statistical significance. Some items, including questions with the Likert-type scales, were made dichotomous for ease of comparison. Statistical significance was assessed using a threshold of alpha = 0.05. Analyses were conducted using STATA-14.2.

3. RESULTS

The PREPD was first completed as a baseline assessment for the PRISM randomized controlled trial; study sample demographic information, including practice, provider, and patient characteristics, are listed in Supplementary Table 1 (Clinical Trials #: NCT02760004) (Moore Simas et al., 2019).

3.1. PREPD Practice Readiness Index, Component, and Sub-Component scores and associations with practice characteristics

Practices had an average Practice Readiness Index of 7.3 out of 16 points, or 45.6% (Supplementary Figure 3). Baseline scores ranged from 4.8 to 9.9, indicating variability across practices in preparedness to implement practice change. There was wide variation in component scores, both between practices and within categories (Supplementary Figures 4a and 4b). Component I (Environment Scan) had low scores across practices, whereas Component III (Depression-related Policies) had higher scores (Supplementary Fig 4a). Component IV (Chart Abstraction) varied by sub-component (Supplementary Fig 4b). Sub-Component C (Depression Assessment) had consistent scores across practices, averaging at 76.1% preparedness (Highly Prepared). The other sub-components had wide ranges of preparedness, with Depression Detection ranging from 14.6-96.7% (average = 40.7%, Minimally Prepared), Depression Treatment from 6.7-88.0% (average = 51.2%, Moderately Prepared), Depression Follow-up and Monitoring from 0-50.0% (average = 17.9%, Not Prepared), and Depression Transfer of Care from 0-100% (average = 40.7%, Minimally Prepared). Bipolar Detection had a unanimous preparedness of 0%, (Not Prepared).

3.2. Practice Readiness Index and associations with patient characteristics

Table 2 demonstrates associations between PRI and patient demographics. Greater proportions of younger patients (<25 years) were seen at practices with higher PRIs (tier 3), while greater proportions of older patients (>35 years) were seen at practices with lower PRIs (tier 1). Greater proportions of patients with higher education received care from practices with lower PRIs. Although race was not associated with differences in PRI scores, Hispanic ethnicity was more prevalent among patients in tier 3 practices. Greater proportions of patients with public insurance, versus private, attended tier 3 practices.

Table 2 -. Associated patient characteristics with Practice Readiness to Evaluate and address Perinatal Depression (PREPD) scores.

The Practice Readiness Index is the summative score of the entire PREPD assessment, with a maximum possible score of 16. Practices were divided into tertiles (n = 4 in each), based on their total PREPD scores (Practice Readiness Index, PRI), categorizing them into 3 groups of readiness: tier 1 (average PRI = 5.5), tier 2 (PRI = 7.4), and tier 3 (PRI =9.0). Patient characteristics were then associated, and percentages are reported by tier.

Patient characteristics	Patients in each tier of PREPD assessment score that the characteristic applies (%)			P-Value
	Tier 1 (PRI = 5.5)	Tier 2 (PRI = 7.4)	Tier 3 (PRI = 9.0)
Patient age (years)				0.001
<25	5.0	7.6	17.9
25-35	63.3	65.5	63.4
>35	31.7	26.9	18.8
Patient Education				<0.001
Grade school/Some HS	1.9	2.4	9.2
HS/GED	12.5	13.3	26.4
Some college	12.5	21.7	26.4
Associate Degree	2.9	6.0	6.9
Bachelor’s Degree	27.9	21.7	17.2
Master’s Degree	29.8	24.1	11.5
Doctoral degree or equivalent	12.5	10.8	2.3
Hispanic ethnicity	6.7	15.7	28.7	<0.001
Non-white race	28.0	36.7	26.6	0.361
Patients with public health insurance	21.8	31.7	62.8	<0.001

PRI: practice readiness index; bolded p-values are <0.05; total percentages may not equal 100% due to rounding.

3.3. Practice Readiness Index and associations with provider characteristics (Provider Survey)

Table 3 lists characteristics of providers and practices and their associations with PRI based on responses to the Provider Survey. Generally, characteristics that were consistent with appropriate treatment of depression were found at higher rates in practices that also scored higher on the PREPD assessment (tier 3) (Table 3a). Compared to those providers in tier 1 or 2, greater proportions of providers in tier 3 practices reported that they screen for depression in the 2^nd and 3^rd trimesters of pregnancy, feel that they could effectively treat depression, and monitor and adjust depression medications for their obstetric patients (Table 3b). Providers in tiers 2 & 3 had greater proportions of prescribing antidepressants and depression severity monitoring compared with tier 1 providers.

Table 3a -.

Associated practice characteristics with Practice Readiness to Evaluate and address Perinatal Depression (PREPD) scores

Practice characteristics	Practices or providers in each tier of PREPD assessment score that the characteristic applies (%)			P-Value
	Tier 1 (PRI = 5.5)	Tier 2 (PRI = 7.4)	Tier 3 (PRI = 9.0)
Practice tracks patients who screen positive for depression*	35.0	52.5	74.4	0.002
Practice has standard processes for directing patients to appropriate mental health resources in the community*	79.0	83.6	94.5	0.026
Support staff at practice feel that they have the knowledge and skills they need to detect and address depression*	72.6	83.3	90.1	0.028
Practice has resident physicians	0	41.7	45.2	<0.001

^*Characteristics that are starred were asked as Likert-style questions, but divided into dichotomous variables for the purpose of this analysis (1 = strongly agree or agree, 0 = strongly disagree or disagree)

PRI: practice readiness index; bolded p-values are <0.05; total percentages may not equal 100% due to rounding.

Table 3b -. Associated provider characteristics with Practice Readiness to Evaluate and address Perinatal Depression (PREPD) scores.

The Practice Readiness Index is the summative score of the entire PREPD assessment, with a maximum possible score of 16. Practices were divided into tertiles (n = 4 in each), based on their total PREPD scores (Practice Readiness Index, PRI), categorizing them into 3 groups of readiness: tier 1 (average PRI = 5.5), tier 2 (PRI = 7.4), and tier 3 (PRI =9.0). Provider and practice characteristics were then associated, and percentages are reported by tier.

Provider characteristics	Practices or providers in each tier of PREPD assessment score that the characteristic applies (%)			P-Value
	Tier 1 (PRI = 5.5)	Tier 2 (PRI = 7.4)	Tier 3 (PRI = 9.0)
Provider reports that they screen women for depression in the…
1st trimester	31.7	35.4	31.0	0.814
2nd trimester	1.2	10.1	13.8	0.011
3rd trimester	2.4	19.0	23.0	<0.001
Provider feels they can effectively treat depressed obstetric patients*	28.9	41.9	68.3	0.001
Provider starts obstetric patients on antidepressant medications when indicated*	73.8	84.1	92.9	0.062
When a patient is prescribed an antidepressant, provider follows up by monitoring their depression severity*	70.7	100	97.6	<0.001
Provider monitors obstetric patients’ depression and adjust medications as indicated*	46.2	71.1	87.2	<0.001

^*Characteristics that are starred were asked as Likert-style questions, but divided into dichotomous variables for the purpose of this analysis (1 = strongly agree or agree, 0 = strongly disagree or disagree)

PRI: practice readiness index; bolded p-values are <0.05; total percentages may not equal 100% due to rounding.

3.4. PREPD Assessment Practice Feedback

Figure 1 illustrates a real-world example of how the PREPD assessment can be used to provide practice level feedback over time. These data came from an individual practice in the PRISM study and shows how that practice has changed from baseline in implementing depression policies into their clinical care delivery. By comparing baseline and mid-assessment PREPD scores, the assessment allowed the practice to see that their scores across all components have gone up, with most in the “Highly Prepared” category now, demonstrating their successes in implementing changes. This feedback also allowed the practice to identify areas that remain “Minimally Prepared” and, thus, to focus efforts more on these areas going forward.

4. DISCUSSION

We present a new tool for evaluating obstetric practices’ readiness to address perinatal depression: the Practice Readiness to Evaluate and address Perinatal Depression (PREPD) assessment and its summary Practice Readiness Index (PRI). The organizational change literature highlights the importance of evaluating practice policies, procedures, and services at baseline to assist in implementation of new interventions, and then repeating longitudinally for comparison. Just as developing and implementing protocols to address perinatal depression are important, helping practices identify their current care processes, including their strengths and opportunities for improvement, is critical to tailoring implementation efforts. Our data suggest that the PREPD assessment can help inform how to best move forward in implementing and integrating behavioral health interventions aimed at addressing depression in the obstetric setting.

This study presents the first steps in the development and utilization of a novel practice readiness evaluation assessment. The development of robust and useful instruments is an ongoing process that includes retesting items with new data and populations, and subsequent refinement. As is suggested in the literature by experts in instrumentation development (Streiner et al., 2015), after item and scale domain development and initial testing, revision of items to make clearer and more parsimonious as well as further validity and reliability testing should be conducted as next steps. As the field grows and pivots to address a wider breadth of maternal mental health conditions in the obstetric setting (Byatt et al., 2020), adaptations to the PREPD assessment will be considered to measure practice readiness in addressing conditions beyond depression, including anxiety disorders, trauma, bipolar disorder, and substance use disorders. In the interim, we have found the PREPD assessment to be useful and to be helpful in facilitating the integration of depression care into obstetric practices as part of our ongoing randomized controlled trial (Moore Simas et al., 2019).

The PREPD can also be useful for longitudinal quantitative evaluation of practice progression in intervention implementation, by monitoring score changes over time, and can qualitatively inform practices about the initiatives that have been implemented, using the descriptive scoring. Similarly to how the APTOC-derived Environmental Scan (Guydish et al., 2012) is used in facilitating and quantifying smoking cessation implementation efforts, we plan to continue to use the PREPD assessment to examine practice success in implementing protocols to screen for and address perinatal depression in our PRISM randomized controlled trial. The PREPD also helps to identify patterns of care that exist within practices or systems which may need to improve further.

Our current study has many strengths. Our assessment was created using the knowledge of a multidisciplinary team with many years of experience in the field of perinatal depression, behavioral health interventions, and implementation science. The corresponding Provider Survey allowed us to evaluate for preliminary construct validity of our tool in its ability to predict practice readiness, on which we will build with data from on ongoing RCT. For example, as shown in Table 3, we found evidence to suggest convergent findings from two separate instruments. Generally, provider characteristics (from the Provider Survey) that were consistent with appropriate treatment of depression were found at higher rates in practices that also scored higher on the PREPD assessment.

Our study also has limitations and ways in which the PREPD can be further refined and improved upon. For example, the PREPD needs to be administered outside of the PRISM RCT and in multiple populations. We consider this paper the first step in establishing evidence for the validity of this tool and acknowledge that replication and its use in other settings will be important in evaluating its effectiveness. To address this, our team is currently administering a revised version of the PREPD in another cluster RCT with a national sample of obstetric practices (Grant #1R41MH113381-01). Additionally, in this study we found substantial variability in: 1) the readiness (PRI) by practice with regard to the characteristics of patients served, and 2) sub-component scores across practices. After further validity testing, it will be important to explore this variation as it relates to the practices, providers, and patients survey (e.g., the importance of the range of the sub-component scores across practices in PRISM, meaningful associations of patient characteristics and PRI scores).

Another important point is that the PREPD has yet to be examined against patient outcomes. This is critical to establishing the effectiveness of the tool; our team will examine the PREPD against patient outcomes as the PRISM study comes to a close and final analyses can be conducted. At the end of the PRISM study, we will also be able to conduct reliability testing and more comprehensive validity testing. Finally, we currently have only evaluated the PREPD assessment’s use in one state, in an area of the country with progressive healthcare initiatives. This may limit generalizability; thus, the ongoing, nationwide RCT will help to assess this limitation. Data from other states will also allow us to explore some of the associations found in our data, such as that practices with greater readiness to address perinatal depression prior to intervention tended to treat more patients who are disproportionately affected by depression (e.g., groups that are marginalized, with less education, and barriers to care). It will be important to explore whether these associations continue and if they indicate the progression of practice procedures based on patient need.

4.1. CONCLUSIONS

As obstetric practices and providers increasingly incorporate perinatal depression care into their workflow, tools are needed to quantify the best ways to tailor their changes accordingly and to measure successful integration over time. The Practice Readiness to Evaluate and address Perinatal Depression (PREPD) assessment may be a valuable tool for these purposes in the future.