Abstract
The Royal Canadian Mounted Police (RCMP), like all public safety personnel (PSP), are frequently exposed to potentially psychologically traumatic events that contribute to posttraumatic stress injuries (PTSI). Addressing PTSI is impeded by the limited available research. In this protocol paper, we describe the RCMP Study, part of the concerted efforts by the RCMP to reduce PTSI by improving access to evidence-based assessments, treatments and training as well as participant recruitment and RCMP Study developments to date.
The RCMP Study has been designed to (1) develop, deploy and assess the impact of a system for ongoing annual, monthly and daily evidence-based assessments; (2) evaluate associations between demographic variables and PTSI; (3) longitudinally assess individual differences associated with PTSI; (4) augment the RCMP Cadet Training Program with skills to proactively mitigate PTSI; and (5)Âassess the impact of the augmented training condition (ATC) versus the standard training condition (STC). Participants in the STC (nÂ=Â480) and ATC (nÂ=Â480) are assessed before and after training and annually for 5 years on their deployment date; they also complete brief monthly and daily surveys.
The RCMP Study results are expected to benefit the mental health of all participants, RCMP and PSP by reducing PTSI among all who serve.
Keywords: PTSD, posttraumatic stress injuries, longitudinal, risk, resilience, transdiagnostic, Unified Protocol, police, public safety personnel
Highlights
Research is limited on how to mitigate posttraumatic stress injuries (PTSIs) among Royal Canadian Mounted Police (RCMP) who are exposed to potentially psychologically traumatic events.
The RCMP Study has been designed to develop, deploy and assess the impact of skills taught to proactively mitigate PTSI.
RCMP cadets recruited into the study to receive the augmented training are assessed before and after training and annually for 5 years on their deployment date.
The RCMP Study results are expected to benefit the mental health of study participants, RCMP and other public safety personnel by mitigating PTSI among all who serve.
Introduction
Public safety personnel (PSP), including border services personnel, correctional workers, firefighters, operational and intelligence personnel, paramedics, police officers, public safety communicators, and search and rescue personnel, “ensure the safety and security of Canadians.”1 Of all Canadian PSP, Royal Canadian Mounted Police (RCMP) report the highest average number of exposures to potentially psychologically traumatic events (PPTEs), often more than 11 exposures to each type of PPTE.2
Using self-reported symptoms (based on validated measures, but not structured clinical interviews) of surveyed Canadian PSP, Carleton et al.3 determined that a very high percentage of RCMP officers screened positive for posttraumatic stress disorder (PTSD) or other posttraumatic stress injuries (PTSI; e.g. major depressive disorder, panic disorder); half (50.2%) screened positive for one or more mental health conditions.3 Many have reported suicidal behaviours during the past year (i.e. ideation [9.9%], planning [4.1%], attempts [0.2%]) or during their lifetimes (i.e. ideation [25.7%], planning [11.2%], attempts [4.2%]).4
Despite commitments to support PSP mental health and widespread efforts involving implementations of different interventions, evidence for effective programs designed to support PSP mental health remains extremely limited.5-11
Contemporary programs designed to support PSP mental health focus on increasing knowledge, reducing stigma and increasing help-seeking behaviours.12 Most studies of PTSI among PSP use cross-sectional data with short follow-up periods and assess very small subsets of variables posited as important.6,13 The limited research suggests the extant programs produce small, time-limited, highly variable benefits,7,8,10,14,15 likely due to low fidelity of delivery16 and limited specification of the mechanisms of action for mitigating PTSI.6,10 A particularly large, robustly designed trial with serving PSP compared psychoeducation to resilience training focussed on stress reduction and mindfulness,17 but found no statistically significant differences between the treatment groups. The researchers recommended future programs target specific modifiable individual differences.17
Individual differences that have been posited as resilience factors for psychopathology include some personality traits (i.e. extroversion, conscientiousness),18 hope,19 distress tolerance,20 optimism,21 interpersonal supports22 and positive life activities (e.g. exercise).23 Environmental factors, individual differences in psychological variables and individual differences in physiological variables have also been posited as risk factors for psychopathology.24 Environmental risk factors for psychopathology include PPTE and stressors (e.g. adverse childhood experiences, difficult socioeconomic status),25 family history of psychopathology,26 pre-existing psychopathology27 and peritraumatic experiences28,29. Individual psychological difference risk factors for psychopathology include some personality traits (e.g. neuroticism, world view),29 anxiety sensitivity,30 fear of negative evaluation,31 illness/injury sensitivity,32 pain-related anxiety/fear,33 intolerance of uncertainty,34 rumination,27 maladaptive self-appraisal,35 dissociation36 and anger.37 Individual physiological difference risk factors for psychopathology include autonomic nervous system dysregulation.38-41,*
Aversive avoidant reactions to emotions are particularly critical risk factors for developing PTSI.30,34 Greater acceptance of emotions reduces reliance on avoidant coping strategies (e.g. alcohol use, avoiding reminders of the event) that exacerbate PTSI symptoms and, paradoxically, lead to more frequent negative emotions.42-44 The Unified Protocol43,44 is an evidence-based cognitive behaviour intervention designed to help individuals cultivate an approach-oriented stance towards emotions. The Unified Protocol was designed to reduce symptoms of diverse anxiety- and mood-related disorders.42 The Unified Protocol is supported by considerable evidence demonstrating transdiagnostic effectiveness across several delivery formats (e.g. individual, group, self).45-59
There is preliminary support for the Unified Protocol as a proactive intervention to mitigate PTSI based on a randomized trial assessing participants with elevated nonclinical symptoms of depression and anxiety.60 Participants found the proactive Unified Protocol training to be highly acceptable and satisfying; at 1-month follow-up, they reported using the new skills “some” to “most” of the time, and statistically significant improvements were observed from baseline to 1-month follow-up. The Unified Protocol appears to have potential as a proactive intervention that can be efficiently and effectively delivered to PSP to use to protect their mental health13,47,61-65 and enhance job satisfaction.66
The current paper describes the RCMP Study† (i.e. design, measures, materials, hypotheses, planned analyses, expected implications, limitations). The RCMP Study is part of the concerted efforts the RCMP is making to address PTSI by improving access to evidence-based assessments, treatments and training. The overarching objectives for the RCMP Study are as follows:
(1) to develop, deploy and assess the impact of a system for ongoing (i.e. annual, monthly, daily) evidence-based assessments of environmental factors and individual differences (i.e. measurements of biometrics, mental health, social experiences);
(2) to evaluate associations between demographic variables and symptoms of PTSD and other PTSI;
(3) to longitudinally assess environmental factors and individual differences associated with PTSI;
(4) to integrate the adapted Unified Protocol training into the RCMP Depot Division (“Depot”) Cadet Training Program,63,67,68 to create the augmented training condition (ATC); and
(5) to assess for differences in participants receiving the ATC relative to those receiving the standard training condition (STC) (e.g. ATC participants should report better mental health).
The current paper details the RCMP Study design, protocols, measures, materials and hypotheses, planned analyses, expected implications and limitations; and describes participant recruitment and study progress to date.
Additional details about individual differences of interest for the RCMP Study are available in the supplemental materials (see http://hdl.handle.net/10294/14680).
On 28 June 2017, the RCMP posted the public request for proposals, “Longitudinal Study of Operational Stress Injuries / Ãtude longitudinale sur les traumatismes liÃs au stress opÃrationnel” (Solicitation Number M7594-171491/C). On 30 November 2017, the research team led by R. Nicholas Carleton was notified that they were the successful applicant. The project is now referred to by stakeholders as the “RCMP Study.”
Methods
Study design
The RCMP Study necessarily uses a longitudinal prospective sequential experimental cohort design to create a clustered randomized trial69-71 that engages individual participants for 5.5 years. The structure of the Cadet Training Program does not allow for randomizing individual participants or individual groups of participants; nevertheless, meta-analytic evidence suggests that results from studies using this design and results from true randomized controlled trials do not typically differ meaningfully or statistically significantly, and both methods produce comparable groups at baseline.72,73
The RCMP have provided continuous coordinated feedback, through co-authors GPK and KSH, on the study design and measurement tools.
RCMP cadets are recruited into training cohorts called troops, each of which typically includes 32 cadets. Between 30 and 50 troops are trained per year (i.e. 800–1200 cadets).
Ethics approvals
The University of Regina Ethics Board provided initial approval on 10 April 2019 (File #2019-055), and the RCMP Research Ethics Board followed with approval on 12 April 2019 (File #SKM_C30818021312580). The study was also approved through a Privacy Impact Assessment as part of the overall National Administrative Records Management System approval (201611123286) and Public Services and Procurement Canada approval (201701491/M7594174191).
The project is bound by the Privacy Act, R.S., 1985, c. P-21 and the Personal Information Protection and Electronic Documents Act, SC. 2000, c.5 and approved by Public Services and Procurement Canada (PSPC) M7594-171491/001/SS.
All interested people were provided with printed and electronic copies of the study information at several points, and all participants were required to explicitly indicate consent before proceeding. Consent was explicitly reaffirmed at several points during the data collections.
Participant information
Potential participants include RCMP cadets starting the Cadet Training Program—Canadian citizens or permanent residents, aged 19 to 57 years, who read, write and speak either English or French fluently.74 Cadets must meet several recruiting requirements, including security clearances, medical examinations, a polygraph test and minimum physical standards. There are no conditions that would exclude any cadet from participating in the RCMP Study. There is no reason to expect that the demographic representation of participants will differ from the demographic representation of cadets overall.
All RCMP constables are trained at Depot. The training program is a rigorous and highly structured 26-week program during which cadets study and practise well beyond 8 hours of scheduled classroom time per day.75,‡ The average age of recruits is 29 years and about 25% self-identify as female. All cadets attending the Cadet Training Program after the ATC launches will receive the augmented training; RCMP Study participation remains voluntary.
Sample size requirements and power analyses
We performed power analyses for a cluster randomized trial using SAS version 9.3 (SAS Institute Inc., Cary, NC, US) and RMASS web application (Center for Health Statistics, University of Chicago, IL, US). Based on these analyses, a sample of 480 participants per condition (i.e. STC and ATC) at T2 (pre-deployment, at about 24 weeks after recruitment) will provide adequate power (i.e. >80%).76,77 Power analyses assumptions were (1) clustering of participants into RCMP troops; (2)Ârepeated measures among participants over time (i.e. clustering within participants); and (3)Âattrition (i.e. loss to follow-up) ranging from 5% to 20%. Our goal in conducting the power analyses was to assess the sample size requirements under several different scenarios and to assess the multiple outcomes of interest.
Analyses
Continuous (e.g. symptoms) and cross-sectional (single time point) analyses
We assumed power of 80%, a significance level (α) of 0.05 and standardized effect size estimates from 0.2 to 0.5. Intraclass correlation coefficients for clustering of participants into troops were assessed from weak (0.01) to strong (0.20), with troop cluster size set at 24 to account for non-graduating cadets.
Binary (e.g. diagnostic status) and cross-sectional (single time point) analyses
We assumed power of 80%, a significance level (α) ofÂ0.05 and cumulative incidence rates of 10% to 20% for the ATC and 25% to 35% for the STC. Intraclass correlation coefficients for clustering of participants into troops were assessed from weak (0.01) to strong (0.20), with troop cluster size set at 24 to account for non-graduating cadets.
Longitudinal continuous analyses
We assumed power of 90% and 95%, with 40 troop clusters, balanced condition allocation and a significance level (α) of 0.05. The attrition rate across six measurement occasions was set to 0% (at T2), 5%, 5%, 10%, 15% and 20%. The intraclass correlation coefficient between study participants (i.e. across repeated measurements) was set as 0.2 and within troops as 0.1. We assumed a linear trend across measurement occasions with a first-order autoregressive within-participant correlation structure with an estimated coefficient of 0.40. The study will be adequately powered if the effect size is greater than 0.25.
Participant recruitment and retention
The website describing the RCMP Study in detail is publicly available (www.rcmpstudy.ca) and is actively used by stakeholders, but that accessibility makes it impossible to know when potential participants first learn about the study. Recruitment for the STC concluded on March 2022 and recruitment for the ATC began on June 6, 2022.
The research team introduces the RCMP Study to potential participants with an email sent from the RCMP National Recruiting Program§ before cadets arrive at Depot. RCMP Study advertisements (e.g. posters, tent cards, pop-up banners) are also distributed throughout Depot.
After potential participants arrive at Depot, each troop attends a recruitment session delivered by members of the research team. The session includes video content from serving RCMP members (~10 minutes), introductions to the research team, a didactic lecture with a slide show presentation (~35 minutes long) and an opportunity for potential participants to ask questions (~15 minutes). The presentation outlines the RCMP Study rationale, design, requirements, expected outcomes and the potential benefits to the RCMP, the broader PSP community and all Canadians.
The presentation also explains potential benefits to individual participants (e.g. monitoring individual mental health, potentiating earlier access to mental health support). Potential participants are given a paper copy of the full participant information sheet and consent form and invited to consider participating. The formal participation decision occurs a few days later during a dedicated on-boarding session.
The on-boarding session begins with a brief video testimonial from an RCMP officer with lived experience (~10 minutes). The research team then provides a highly detailed and interactive tutorial (~45Âminutes) explaining, for example, the technology set-up and using the study software and hardware. The on-boarding session includes an opportunity to ask questions (~10 minutes).
Although participants are encouraged to complete all assessments, because of the size, scale and length of the RCMP Study very few participants manage to complete all assessments and missing data are expected. Cadets who choose not to participate are invited to provide anonymous feedback about their decision. Participants who leave the research project are also invited to provide anonymous feedback about their decision. Participants who leave the RCMP during the research project will be invited to complete the assessment phase following their departure.
Cumulative attrition from longitudinal police studies without the employer paying for participation time typically ranges from 3% at 1 year to 43% at 3 years.78-82 A similar research design with military participants has reported cumulative attrition rates of less than 10% and overall rates of missing data of less than 30%, with anecdotal reports that participants appreciate being able to track their own activities and symptoms.83,84 All RCMP Study participants can participate during paid time and the RCMP have demonstrated a high affinity for research participation (G.ÂKrÃtzig, personal communication, 12 February 2019). That affinity, coupled with plans for recruitment and retention, and intrinsic motivation (i.e. the prospect of improved mental health), suggests attrition rates consistent with the military research.83,84 Based on previous research,27,83,84 anticipated participation recruitment was expected to take about 24 months.
Gender and sex
Gender structures implicitly and explicitly influence how PSP experience occupational stressors and interact with mental health. Policing culture emphasizes hegemonically masculine traits,85 creating specific difficulties for women including stereotyping, discrimination and sexual harassment on duty.30,86 Masculinized work patterns87 create challenges for women accessing parental leave and balancing work with care obligations.30,87 Perhaps due to such stressors, women police officers more frequently report mental health challenges than men police officers,3 but with substantial variability associated with potential causes.30,34 The RCMP Study will support sex- and gender-disaggregated analyses across various topics (see the supplemental tables at http://hdl.handle.net/10294/14680). Data are primarily quantitative and self-reported; still, participants can provide open-ended responses. Gender and sex will be treated as variables for all analyses. Qualitative analyses will include open-ended responses where possible. The results will provide insight into gendered structures affecting PSP mental health.
Data collection timeframe
Participants will be assessed for at least 66Âmonths, via full assessments conducted at T1 and T2, and annually thereafter (i.e. T3, T4, T5, T6, T7), as well as monthly assessments, daily assessments and biometric assessments (see the section “ Assessments, surveys and interviews” for details), to allow for sufficient time to potentially develop PTSI symptoms after deployment. The data collection time-period uses seven broad milestones (see Table 1 for a summary and the supplemental tables at http://hdl.handle.net/10294/14680 for details): pre-training (T1); pre-deployment (T2; ~24 weeks after recruitment); and on or about each of five deployment anniversaries (T3 to T7), the first (T3) being about 12 months after deployment. Each milestone involves a full assessment (FA1 to FA7). Recruitment will continue until 480 ATC participants have completed FA2. Unless extended by the RCMP, FA7 concludes data collection from each participant.
Table 1. RCMP Study recruitment and data collection timeframe overview for STC and ATC.
Participants complete their first monthly assessment (i.e. MA1) about 4 weeks after completing FA1 and do not complete a monthly assessment concordant with completion of a full assessment (i.e. maximum number of monthly assessments per participant is 65). Participants can complete their first daily assessment (i.e. DA1) on the same day as FA1 (i.e. maximum number of daily assessments per participant ~2008). Cadets cannot be enrolled into the ATC until all STC participants have deployed, creating, by necessity, a 26-week gap that will be used to prepare to transition the Cadet Training Program to the ATC (see supplemental tables at http://hdl.handle.net/10294/14680).
Unified Protocol adaptation and training details
A 13-week protocol based on the Unified Protocol and called Emotional Resilience Skills Training (ERST) was developed for seamless integration with the Cadet Training Program. The ERST includes an instructor guide, didactic PowerPoint slides and a cadet workbook, all conforming to the extant Cadet Training Program formats.
The ERST leader for the research team (SSZ, a co-developer of the Unified Protocol) will train and certify several Cadet Training Program instructors as master trainers to provide the ERST to other training program instructors, RCMP officers and ATC cadets. The RCMP selects which instructors (nÂ=Â6–8) are to receive the week-long interactive workshop and then provide ongoing quality control for the ATC implementation.
Master trainers will each train two to three trainers (instructors selected to provide the ERST to ATC cadets; nÂ=Â18) by presenting the ERST material as if the trainers were cadets. Master trainer training fidelity will be assessed and supported by having their training sessions audiorecorded and rated by a member of the research team (SSZ). The trainers will then provide the ERST to all 200 Depot team members, including other instructors. The training sessions for the 200 Depot team members will also be audiorecorded for review by the master trainers to support fidelity.
The master trainers will work with all Cadet Training Program instructors and the research team to integrate the ERST—and testing of ERST skill use—into every other aspect of the program, therein creating a high-fidelity ATC with didactic training supplemented by substantial experiential practice. Cadets should have sufficient practice that the ERST skills become as automatic as any other skill set, facilitating ongoing use after deployment and protecting their mental health. Participants will also have ongoing access to ERST to support skill retention after deployment.
The integration should overcome challenges of previous mental health training program deployments related to fidelity16 and skill development.6,7,14,15
Communication tools – Moodle and Qualtrics
All communications between the research team and participants, administration of surveys, feedback for participants (including their clinical assessment reports) and distribution of the ERST materials will be coordinated through a tailored and dedicated instance of the online learning platform Moodle (i.e. the Portal) paired with a software application (i.e. the App) downloadable onto compliant smart phones. Surveys are administered in English or French through a secured Qualtrics account. Clinical interviews are supported by electronic administration of the Mini-International Neuropsychiatric Interview (MINI).88-90
Assessments, surveys and interviews
Full assessments – full surveys
Details of the full surveys (e.g. questionnaire titles, details, psychometric information, references) are in the “Supplemental Psychometrics and References for Self-Report Measures (Alphabetically)” (see http://hdl.handle.net/10294/14680).
The initial full survey assesses stable demographics (i.e. sex, date of birth, height, ethnicity, gender, sexual orientation, adverse childhood experiences) and reference characteristics (i.e. pre-recruitment education, employment, language(s) spoken, religion, work history, living location and mental health history). All remaining full surveys assess demographics expected to change more (i.e. physical health conditions, body mass, work and living locations, socioeconomic status, marital status, rank, work hours, education, household composition).
Average completion time for each full survey is 72 minutes. Participants can access their full survey results in a dedicated report containing context and academic references through the Portal. All full surveys also assess for the following:
(1) symptoms of PTSI including generalized anxiety disorder, major depressive disorder, panic disorder, PTSD and social anxiety disorder as well as PTSI correlates (e.g. substance use, chronic pain, insomnia, relationship dissatisfaction);
(2) environmental factors and individual differences positively or negatively associated with PTSI:
PPTE exposures;
personality;
anxiety sensitivity, fear of negative evaluation, illness/injury sensitivity, intolerance of uncertainty, pain-related anxiety, resilience, anger, beliefs about emotions, experiential avoidance, emotion regulation, and mindfulness;
mental health care knowledge, access and use;
occupational stressors, work fulfilment, institutional betrayal, stigma, family stressors, posttraumatic growth, social support and self-care; and
(3) ERST retention and use for ATC participants.
Responses consistent with one or more mental disorders are flagged for participants and include recommendations for accessing additional mental health supports.
Full assessments – clinical interviews
Each full assessment includes a semistructured MINI clinical interview88-90 conducted by a registered clinical psychologist or by supervised trainees. The MINI provides a standardized diagnostic approach consistent with DSM-5 criteria.91 The interviewer reviews the full survey results before MINI administration in either English or French, whichever the participant prefers.
The published MINI interrater reliability exceeds 75%.88,89 Current interrater reliability will be assessed with the κ (kappa) statistic and “observed agreement” (i.e. a second interviewer observes 15% of all clinical interviews). Average completion for each clinical interview is 45 minutes.
Participants receive verbal (from the interviewer) and written (through the Portal) summaries of their full assessment and are advised of responses consistent with one or more mental disorders and referred for mental health support as indicated (e.g. to registered psychological services available, through the RCMP and independent of the RCMP); however, a diagnosis is not provided.
Monthly assessments
Details of the monthly assessments (e.g. questionnaire titles, details, psychometric information, references) are in the “Supplemental Psychometrics and References for Self-Report Measures” (see http://hdl.handle.net/10294/14680). The maximum number of monthly assessment questions is 271, determined by participant responses to header questions. Average completion time for each monthly survey is 15 minutes. All monthly surveys assess for the following:
(1) symptoms of PTSI including generalized anxiety disorder, major depressive disorder, panic disorder, PTSD and social anxiety disorder as well as PTSI correlates (e.g. substance use, chronic pain, insomnia);
(2) individual differences associated with PTSI: PPTE exposures; resilience; mental health care knowledge, access and use; occupational stressors, social support and self-care; and
(3) ERST retention and use for ATC participants.
Participants receive written (through the Portal) summaries of their monthly assessment including timeline charts for participants to monitor fluctuations, facilitating participation healthy habits.92-94 Responses consistent with one or more mental disorders are flagged for participants and include recommendations for accessing additional mental health supports.
Daily assessments
The daily assessments are very brief self-report questionnaires that allow participants to reflect and report on their mood, attitude and performance; physical wellness; emotional state; work hours; sleep hours; sleep quality; eating; physical activity; social activity; substance use and gambling; and ERST use (ATC participants only).
Mood, attitude and performance and physical wellness are rated on 100-point visual analog scales with anchors of ill (0–25), injured (26–50), reacting (51–75) and healthy (76–100). A 24-point rating scale is used for emotional state, work hours and sleep hours. Sleep quality, eating, physical activity, social activity and substance use are reported dichotomously (i.e. yes/no responses), with discretionary options for participants to record details.
The daily assessments also allow participants to log PPTE or other significant emotional events, creating a record of exposures to stressors. Average completion time for each Daily Survey is about 1Âminute. Completing the daily assessments supports regular self-reflection and provides participants with graphical feedback to encourage healthy habits.92-94
Biometric assessments
The original RCMP Study design used electrocardiography to measure heart rate variability through beat-to-beat intervals in consecutive QRS complexes (i.e. R–R intervals) of sinus origin.95-97 The process provides a relative assessment of health based on information about the autonomic nervous system97 using the method of choice for valid assessments of heart rate variability.98,99 R–R intervals from electrocardiography are robustly related to other collection methods;99,100 however, heart rate variability analyses for autonomic nervous system changes reflecting psychopathology require knowing the sinus, supraventricular or ventricular origin,101 inspecting all heartbeats and using complete physiological recordings,97 instead of mathematically mediated estimations.98,100,102-104
Hexoskin wearable biosensor garments (Carrà Technologies Inc., MontrÃal, QC) were modified for policing operational requirements and used for heart rate variability measurements. Initial plans required participants to wear the garment upon waking for a 5-minute resting period to establish baseline metrics.97 The garment was worn during training and work shifts, producing 3 to 5 days of recordings per week. Participants were also equipped with an Apple Watch (series 4 and then series 5; Apple Inc., Cupertino, CA), to supplement the biometric data collection with sleep and metabolic data, supported by a dedicated Apple iPhone (Apple Inc., Cupertino, CA). Participants use the iPhones with Wi-Fi at no cost and could use the iPhones as personal phones, but they were not provided with voice or data plans. Biometric recordings were downloaded for offline processing and analyses.
Participants found using the Hexoskin garments challenging, and in 2021, data collection transitioned to the Recordis cardiac sensor device (LLA Technologies Inc., Vancouver, BC) to collect M-mode echocardiography timing events (i.e. systole, diastole, isovolumic contraction and relaxation periods, rapid ejection) as well as twist forces of the ventricle (a surrogate for contractility), heart rate variability and Myocardial Performance Index variant.105
The Recordis is applied to the sternum base (~1 cm above the xiphoid process) with a single electrocardiography electrode or heart belt strap, and uses a smartphone application providing immediate and ongoing user feedback. The current protocol requires a daily 1-minute recording upon waking, with data downloaded for offline processing and analyses. This equipment is expected to better support participant compliance without compromising data collections to address hypotheses.
Data management and confidentiality
Data transfers from participant devices to secured research servers in Canada are protected using Transport Layer Security. The RCMP Study also employs a PKI Class 3 SSL Certificate, with a 2048-bit digital signature and 256-bit encryption. Data stored on the servers are automatically encrypted using server-side Advanced Encryption Standard–256 before being saved to disk and decrypted prior to downloading. The data are also “salted” (i.e. they include false participants) to further protect participant privacy. The data are stored separately from the data dictionary and the codes necessary for interpretation.
Participants log into the Portal using a unique randomly generated Participant Identification Code (PIC) and a password of their choosing. A PIC is created for each RCMP cadet, irrespective of their decision to participate in the study. The PIC is used for storing participant data and linking responses over time because the research team does not have access to participant names. The RCMP stores the list pairing cadet names with PICs in a secured file at Depot that is accessible by only two employees; the organization never has access to individual participant RCMP Study data. The file is only opened under two circumstances: to add new participants; and if a participant discloses an imminent intent to die by suicide, a registered clinical psychologist from the research team may contact the RCMP to provide the PIC—but no other information—to try to save the participant’s life.
Hypotheses
The RCMP Study hypotheses were pre-registered.* * Hypotheses specific to individual difference variables are provided in supplemental tables (see http://hdl.handle.net/10294/14680; i.e. “Posttraumatic Stress Injury Symptom Measures”; “Primary Differences Associated with Posttraumatic Stress Injuries”; and “Secondary Individual Differences Associated with Posttraumatic Stress Injuries”). Overarching RCMP Study hypotheses are shown in Table 2.
Table 2. RCMP Study hypotheses.
Planned analyses
We will initially describe study data using frequencies, means and standard deviations. We will test differences in baseline demographic characteristics of STC and ATC participants using a χ2 (chi-square) test for independence. Missing data will be described within and across times when data are measured. Multiple imputation will be adopted assuming a missing-at-random mechanism111 and completed based on the clustering of individuals into troops and on a thorough examination of variables associated with missing values and response distributions.112 We will adopt strategies to control the familywise error rate (i.e. the probability of at least one Type I error in a family of tests),113 while accounting for Type II errors.114
Analyses will include mixed-effects multiple linear and non-linear regression models including covariates (i.e. sex, age, marital status, education, province of residence) and trend analyses as needed. To test for differences between troops in the cross-sectional analyses, we will use a mixed-effects model that accounts for clustering within troops, while to test for changes over time and to test for differences between troops in the longitudinal analyses, we will use a mixed-effects model that accounts for clustering within troops as well as within individuals (i.e. repeated measurements for each individual). We will use recursive Bayesian algorithm and ecological momentary analyses to analyze the daily assessments and biometric data. Sex and gender analyses will be conducted. Open-ended data will be coded using conventional qualitative content approaches before analyses.115
Knowledge translation
We plan to share main results with the research community via publication in peer-reviewed journals. Results will be of interest to RCMP leadership and members and their families, other PSP stakeholders, clinicians and policymakers. Knowledge translation will be tailored for the different audiences. Technical reports and lay summaries will be available through the study website (www.rcmpstudy.ca), in English and French, and provided to senior RCMP officials and the Department of Public Safety and Emergency Preparedness. Results will also be communicated to PSP stakeholders through the Canadian Institute for Public Safety Research and Treatment (www.cipsrt-icrtsp.ca). The University of Regina Communications Department will issue press releases as appropriate.
Current study status and impacts of COVID-19
STC recruitment and data collection began on 22 April 2019 and continued for 27Âtroops until 9 December 2019. There were 25 on-boarding sessions from 25ÂApril 2019 to 12 December 2019. Depot closed in response to COVID-19 containment restrictions on 19 March 2020, and all participants still in the Cadet Training Program were discharged from the RCMP Study. Many participants had completed the full assessment at T1 (nÂ=Â496) and T2 (nÂ=Â167). STC recruitment restarted on 16 November 2020 and continued until more than 480 participants had completed their full assessments at T2, which occurred on 21 March 2022. The ATC recruitment began on 6 June 2022.
The COVID-19 pandemic inconvenienced RCMP Study participants, delayed the study timeline, increased study costs and led to the creation of pre- and post-COVID-19 groups within the STC, adding a new covariate for analyses.
The data collection timeframe was on track until Depot closed on 19 March 2020. Depot did not reopen to new cadets until 2 November 2020. Cadets attending the training program after 2 November 2020 started with 2 weeks in mandatory isolation to offset COVID-19 risks. The closure and reopening resulted in: all RCMP Study participants at Depot at the time of shutdown (nÂ=Â130) being removed from the RCMP Study; substantially increased recruitment time for the STC due to removal of existing participants; the ad hoc creation of pre- and post-COVID-19 STC participant groups (i.e. STC-PREC19 and STC-POSTC19); and the need to assess for and control for the impact of COVID-19 on the STC by statistically comparing the STC-PREC19 and STC-POSTC19, and by adding the COVID-19 STC group status in subsequent analyses.
The Cadet Training Program has evolved considerably since 1885. A problem-based learning model is currently used to acquire police driving, firearms and self-defence skills; emphasis is placed on communication skills for de-escalation and crisis management.
The RCMP does not release email addresses of new recruits to any external group.
Pre-registration with aspredicted.org for the RCMP Study and associated hypotheses occurred on 7 November 2019 with the name, “Risk and resiliency factors in the RCMP: A prospective investigation” (#30654).
Discussion
The RCMP Study was designed to develop, deploy and assess the impact of a system for ongoing (i.e. annual, monthly, daily) evidence-based assessments of environmental factors, individual differences in psychological variables and individual differences in physiological variables (i.e. measurements of biometrics, mental health, social experiences). The implemented system is now referred to as the RCMP Study Protocol. The RCMP Study was also designed to prospectively assess for interactions between demographic variables, environmental factors, individual differences and PTSI symptoms. Data collection is well under way and initial results will be reported in peer-reviewed publications. The RCMP Study has developed and will integrate and test the impact of integrating ERST into the Cadet Training Program. ATC participants are expected to report significant and substantive mental health benefits relative to STC participants.
Expected implications for clinical practice, policy and research
The RCMP Study was designed to benefit all participants: through evidence-based assessments that encourage self-monitoring and earlier access to care92-94,116-118; through reductions in mental health stigma through increased discourse119-122 and social supports22,119,123; with tangible evidence of organizational commitment to improve evidence-based mental health supports119,124,125; by creating independent electronic mental health records; and through shared altruistic engagement in improving mental health for all PSP126,127.
Participants in the ATC are expected to receive substantial additional benefits from receiving the ERST, a set of skills that may also help RCMP to support civilians who are experiencing distress. Integrating the ERST to create the ATC appears to be unique among contemporary efforts to proactively support PSP mental health. The ATC includes specific and regular fidelity checks for the ERST, assessments of ERST engagement and use, and ongoing ERST access. In effect, the “dose” of the intervention will be larger, more rigorously applied and more rigorously assessed than previous efforts,6,128 and will provide critical information about the potentially positive proactive impact of any PSP mental health training program.
The RCMP Study was also designed to benefit the RCMP as an organization, by providing evidence-based information for ongoing enhancements to training and assessments and by deploying and testing a set of tools to support RCMP mental health. The RCMP Study results and the RCMP Study Protocol are expected to also inform assessments, treatments and programming for diverse PSP, military, veterans and other people at risk for PPTE exposures (e.g. nurses).129,130 For example, a project funded by the Canadian Institutes of Health Research is now testing an adaptation of the RCMP Study Protocol with samples of firefighters, paramedics, municipal police and public safety communicators.
Strengths and limitations
The RCMP Study has several strengths: longitudinal design elements that can inform causal relationships; prospective design elements that can inform predictive and proactive discussions associated with PPTE; large STC and ATC sample sizes at milestone T2; ongoing multimodal assessments of environmental factors and individual differences, including PPTE exposures, occupational stressors and social stressors; and interrater reliability assessments for clinical interviews. The sequential experimental cohort design elements required creating an exceptionally tailored and structured training program (i.e. the ATC) from the ERST adaptation of the rigorously well-supported Unified Protocol.42,43 The ATC is an inherent study strength, and the ERST is a tangible deliverable for the RCMP.
The RCMP Study also has several necessary limitations. The sequential experimental cohort design was necessary to accommodate Cadet Training Program ecological realities. Cadets interact within and across troops while in the training program and, as such, cannot be truly randomized to the STC or ATC.
Relatedly, Depot trainers could not feasibly provide two separate training conditions, simultaneously or in parallel, prohibiting a truly randomized design. The study design strengths coupled with founding the ATC on the Unified Protocol should sufficiently offset the randomization design limitation.72,73 There is no obfuscation of participant condition and no true “sham” training condition, creating unknowable influence from expectancy effects. Direct research on such impacts remains relatively nascent,131 but the available results suggest little or no difference between open and closed label designs where participants are provided with a sufficient rationale for the study design (e.g. Locher et al.132).
The assessments are uncommonly detailed, which may produce intermittent response fatigue among participants; however, participation occurs on paid time and the RCMP actively support participant morale and engagement with the RCMP Study.
Despite the detailed assessments, there may be important unassessed variables. If the interim analyses or participant feedback evidence are missing variables, the research study team will work to accommodate additional data collection.
The detailed assessments also increase Type I error risks from spurious correlations.133 The pre-registration of hypotheses, as well as the a priori provision of expected results in the current protocol paper, are expected to mitigate Type I error risks and protect against equally problematic Type II error risks.133 The voluntary nature of cadet participation in the RCMP Study creates an unknowable influence from self-selection biases. The sample size and analytic plan are designed to offset attrition, but protecting participant privacy prohibits definitively assessing for systematic biases based on attrition. The lengthy time spans associated with data collection and results increase attrition risk, so the research team is working to regularly disseminate meaningful interim results.
Conclusion
The RCMP Study has been designed as an applied longitudinal prospective sequential experimental cohort research project. Participants, the RCMP as an organization, past, present and future RCMP and all PSP should benefit directly and indirectly from the RCMP Study. The benefits should occur irrespective of any specific RCMP Study component (i.e. risk variables, resilience variables, biological variables, the relative impact of the ATC). Through the RCMP Study, the RCMP have become global leaders in the international efforts to better support the mental health of all PSP. The current protocol paper provides details to inform and support similar efforts by other researchers.
Study funding, support and acknowledgements
R. N. Carleton is supported by a Medavie Foundation Project Grant. L. M. Lix is supported by a Tier I Canada Research Chair in Methods for Electronic Health Data Quality. T. O. Afifi is supported by a Tier I Canada Research Chair in Childhood Adversity and Resilience. S. H. Stewart is supported by a Tier 1 Canada Research Chair in Addictions and Mental Health.
The current study was supported by the RCMP, the Government of Canada and the Ministry of Public Safety and Emergency Preparedness. Special thanks to the following people (alphabetically by last name) who have provided tremendous support for the current study in several different ways since inception:
RCMP and Government Leaders – William Sterling Blair, Jasmin Breton, Sylvie ChÃteauvert, Daniel Dubeau, Ralph Edward Goodale, Louise Lafrance, Brenda Lucki, Sylvie Bourassa Muise, Robert Paulson, Stephen White;
Academics – Kelly J. Abrams, Murray P. Abrams, Billea Ahlgrim, Katie Andrews, Myles Ferguson, Jennifer Gordon, Chet Hembroff, Bridget Klest, Jolan Nisbet, Laurie Sykes-Tottenham, Kristi Wright;
University of Regina executive, administrative and technical team members– Olabisi Adesina, Seerat Bassi, Chris Beckett, Brad Berezowski, Jonathan Burry, Murray Daku, Krysten Forbes, Jolene Goulden, Sally Gray, Kadie Hozempa, Xiaoqian Huang, Maria Kamil, Anita Kohl, Donna King, Jordan MacNeil, David Malloy, Akiff Maredia, Kathy McNutt, Megan Milani, Sara Moradizadeh, Sajid Naseem, Obimma Onuegbu, Abimbola Ogunkoyode, Steve Palmer, Cynthia Sanders, Mikhail Shchukin, Shubham Sharma, Vianne Timmons, Preeti Tyagi, Abinyah Walker, Keyur Variya, Christopher Yost, Zhe Zhang;
Clinical staff and supervised clinicians– Andreanne Angehrn, Michael Edmunds, Amelie Fournier, Lis Hansen, Stephanie Korol, Caeleigh Landry, Katherine Mazenc, Michelle Paluszek, Vanessa Peynenburg, Lloyd Robertson, Robyn Shields, Joelle Soucy, Emilie Thomas, Vivian Tran.
Authors’ contributions and statement
All authors made substantial contributions consistent with the International Committee of Medical Journal Editors.
Conceptualization: RNC, GJGA, AJF, DG, NAJ, GPK, JPN, SSZ
Methodology: RNC, TOA, GJGA, KD, AJF, DG, KSH, OH, TMK, GPK, LML, RM, MM, JPN, SSZ, JS, SHS
Validation: RNC, LJ, GPK, RM, JPN, KQM, TAT
Formal analysis: RNC, AJF, LML, JPN, KQM, TAT
Investigation: RNC, JPN, AB, GPK, JPN, SSZ
Resources: RNC, AB, GPK, SSZ
Data curation: RNC, GJGA, LJ, JPN, KQM, TAT
Writing – Original draft preparation: RNC, GJGA, AJF, LJ, GPK, LML, JPN, KQM, SSZ, TAT
Writing – Review & editing: All authors
Supervision: RNC, JPN, SSZ
Project administration: RNC, GPK, JPN, KQM, SSZ
Funding acquisition: RNC
All authors viewed and approved the submitted version of the manuscript.
The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.
References
- CIPSRT. Regina(SK): 2019. Glossary of terms: a shared understanding of the common terms used to describe psychological trauma. [Google Scholar]
- Carleton RN, Afifi TO, Taillieu T, et al, et al. Exposures to potentially traumatic events among public safety personnel in Canada. Can J Behav Sci. 2019:37–52. [Google Scholar]
- Carleton RN, Afifi TO, Turner S, et al, et al. Mental disorder symptoms among public safety personnel. Can J Psychiatry. 2018;63((1)):54–64. doi: 10.1177/0706743717723825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carleton RN, Afifi TO, Turner S, et al, et al. Suicidal ideation, plans, and attempts among public safety personnel in Canada. Can Psychol. 2018;59((3)):220–31. [Google Scholar]
- Oliphant RC, et al. House of Commons. Ottawa(ON): 2016. Healthy minds, safe communities: supporting our public safety officers through a national strategy for operational stress injuries: report of the Standing Committee on Public Safety and National Security. [Google Scholar]
- Anderson GS, Nota PM, Groll D, Carleton RN, et al. Peer support and crisis-focused psychological interventions designed to mitigate post-traumatic stress injuries among public safety and frontline healthcare personnel: a systematic review. Int J Environ Res Public Health. 2020;17((20)):7645–31. doi: 10.3390/ijerph17207645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leppin AL, Gionfriddo MR, Sood A, et al, et al. The efficacy of resilience training programs: a systematic review protocol. Syst Rev. 2014;3((1)):20–31. doi: 10.1186/2046-4053-3-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robertson IT, Cooper CL, Sarkar M, Curran T, et al. Resilience training in the workplace from 2003 to 2014: a systematic review. J Occup Organ Psychol. 2015;88((3)):a systematic review–31. [Google Scholar]
- Wild J, Greenberg N, Moulds ML, et al, to do, et al. Pre-incident training to build resilience in first responders: recommendations on what to and what not to do. Psychiatry. 2020;83((2)):128–42. doi: 10.1080/00332747.2020.1750215. [DOI] [PubMed] [Google Scholar]
- Wild J, El-Salahi S, Esposti MD, et al. The effectiveness of interventions aimed at improving well-being and resilience to stress in first responders. Eur Psychol. 2020;25((4)):252–71. [Google Scholar]
- Carleton RN, et al. Collaborating to support the mental health of public safety personnel: the Canadian Institute for Public Safety Research and Treatment (CIPSRT) Can Psychol. 2021;62((2)):167–73. [Google Scholar]
- Papazoglou K, Andersen JP, et al. A guide to utilizing police training as a tool to promote resilience and improve health outcomes among police officers. Traumatology. 2014;20((2)):103–11. [Google Scholar]
- Gallagher MW, Thompson-Hollands J, Bourgeois ML, Bentley KH, et al. Cognitive behavioral treatments for adult posttraumatic stress disorder: current status and future directions. J Contemp Psychother. 2015;45((4)):235–43. [Google Scholar]
- Carleton RN, Korol S, Mason JE, et al, et al. A longitudinal assessment of the road to mental readiness training among municipal police. Cogn Behav Ther. 2018;47((6)):508–28. doi: 10.1080/16506073.2018.1475504. [DOI] [PubMed] [Google Scholar]
- Szeto A, Dobson KS, Knaak S, et al. The road to mental readiness for first responders: a meta-analysis of program outcomes. Can J Psychiatry. 2019 doi: 10.1177/0706743719842562. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fikretoglu D, Liu A, Nazarov A, Blackler K, et al. A group randomized control trial to test the efficacy of the Road to Mental Readiness (R2MR) program among Canadian military recruits. BMC Psychiatry. 2019;19((1)):326–28. doi: 10.1186/s12888-019-2287-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wild J, El-Salahi S, Esposti M, Thew GR, et al. Evaluating the effectiveness of a group-based resilience intervention versus psychoeducation for emergency responders in England: a randomised controlled trial. PLoS One. 2020;15((11)):e0241704–28. doi: 10.1371/journal.pone.0241704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell-Sills L, Cohan SL, Stein MB, et al. Relationship of resilience to personality, coping, and psychiatric symptoms in young adults. Behav Res Ther. 2006;44((4)):585–99. doi: 10.1016/j.brat.2005.05.001. [DOI] [PubMed] [Google Scholar]
- Glass K, Flory K, Hankin BL, Kloos B, Turecki G, et al. Are coping strategies, social support, and hope associated with psychological distress among Hurricane Katrina survivors. J Soc Clin Psychol. 2009;28((6)):779–95. [Google Scholar]
- Marshall-Berenz EC, Vujanovic AA, Bonn-Miller MO, Bernstein A, Zvolensky MJ, et al. Multimethod study of distress tolerance and PTSD symptom severity in a trauma-exposed community sample. J Trauma Stress. 2010;23((5)):623–30. doi: 10.1002/jts.20568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prati G, Pietrantoni L, et al. Optimism, social support, and coping strategies as factors contributing to posttraumatic growth: a meta-analysis. J Loss Trauma. 2009;14((5)):364–88. [Google Scholar]
- Prati G, Pietrantoni L, et al. The relation of perceived and received social support to mental health among first responders: a meta-analytic review. J Community Psychol. 2010;38((3)):403–17. [Google Scholar]
- LeardMann CA, Kelton ML, Smith B, et al, et al. Prospectively assessed posttraumatic stress disorder and associated physical activity. Public Health Rep. 2011;126((3)):371–83. doi: 10.1177/003335491112600311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brewin CR, Andrews B, Valentine JD, et al. Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults. J Consult Clin Psychol. 2000;68((5)):748–66. doi: 10.1037//0022-006x.68.5.748. [DOI] [PubMed] [Google Scholar]
- Koenen KC, Moffitt TE, Poulton R, Martin J, Caspi A, et al. Early childhood factors associated with the development of post-traumatic stress disorder: results from a longitudinal birth cohort. Psychol Med. 2007;37((2)):181–92. doi: 10.1017/S0033291706009019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ozer EJ, Best SR, Lipsey TL, Weiss DS, et al. Predictors of posttraumatic stress disorder and symptoms in adults: A meta-analysis. Psychol Bull. 2003:52–73. doi: 10.1037/0033-2909.129.1.52. [DOI] [PubMed] [Google Scholar]
- Wild J, Smith KV, Thompson E, ar F, Lommen MJ, Ehlers A, et al. A prospective study of pre-trauma risk factors for post-traumatic stress disorder and depression. Psychol Med. 2016;46((12)):2571–82. doi: 10.1017/S0033291716000532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saumier D, Brunet A, et al. Peritraumatic distress and the course of posttraumatic stress disorder symptoms: a meta-analysis. Can J Psychiatry. 2012;57((2)):122–9. doi: 10.1177/070674371205700209. [DOI] [PubMed] [Google Scholar]
- Kyron MJ, Rees CS, Lawrence D, Carleton RN, McEvoy PM, et al. Prospective risk and protective factors for psychopathology and wellbeing in civilian emergency services personnel: a systematic review. J Affect Disord. 2021:517–32. doi: 10.1016/j.jad.2020.12.021. [DOI] [PubMed] [Google Scholar]
- Angehrn A, Krakauer RL, Carleton RN, et al. The impact of intolerance of uncertainty and anxiety sensitivity on mental health among public safety personnel: when the uncertain is unavoidable. Cog Ther Res. 2020:919–30. doi: 10.1007/s10608-020-10107-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Collimore KC, Carleton RN, Hofmann SG, et al. Posttraumatic stress and social anxiety: the interaction of traumatic events and interpersonal fears. Depress Anxiety. 2010:1017–26. doi: 10.1002/da.20728. [DOI] [PubMed] [Google Scholar]
- Andreski P, Chilcoat H, Breslau N, et al. Post-traumatic stress disorder and somatization symptoms: a prospective study. Psychiatry Res. 1998;79((2)):131–8. doi: 10.1016/s0165-1781(98)00026-2. [DOI] [PubMed] [Google Scholar]
- Asmundson GJ, Katz J, et al. Understanding the co-occurrence of anxiety disorders and chronic pain: state-of-the-art. Depress Anxiety. 2009;26((10)):888–901. doi: 10.1002/da.20600. [DOI] [PubMed] [Google Scholar]
- Korol S, Vig KD, Sapach MJ, Asmundson GJ, Carleton RN, Police J, et al. Demographic and cognitive risk factors for police mental disorder symptoms. Police J. :94(1):40–57. doi: 10.1177/0032258X19894619. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bryant RA, Guthrie RM, et al. Maladaptive appraisals as a risk factor for posttraumatic stress: a study of trainee firefighters. Psychol Sci. 2005;16((10)):749–52. doi: 10.1111/j.1467-9280.2005.01608.x. [DOI] [PubMed] [Google Scholar]
- McCaslin SE, Inslicht SS, Metzler TJ, et al, et al. Trait dissociation predicts posttraumatic stress disorder symptoms in a prospective study of urban police officers. J Nerv Ment Dis. 2008:912–8. doi: 10.1097/NMD.0b013e31818ec95d. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meffert SM, Metzler TJ, Henn-Haase C, et al, et al. A prospective study of trait anger and PTSD symptoms in police. J Trauma Stress. 2008;21((4)):410–6. doi: 10.1002/jts.20350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pole N, et al. The psychophysiology of posttraumatic stress disorder: a meta-analysis. Psychol Bull. 2007;133((5)):725–46. doi: 10.1037/0033-2909.133.5.725. [DOI] [PubMed] [Google Scholar]
- Pineles SL, Rasmusson AM, Yehuda R, et al, et al. Predicting emotional responses to potentially traumatic events from pre-exposure waking cortisol levels: a longitudinal study of police and firefighters. Anxiety Stress Coping. 2013;26((3)):241–53. doi: 10.1080/10615806.2012.672976. [DOI] [PubMed] [Google Scholar]
- Blechert J, Michael T, Grossman P, Lajtman M, Wilhelm FH, et al. Autonomic and respiratory characteristics of posttraumatic stress disorder and panic disorder. Psychosom Med. 2007;69((9)):935–43. doi: 10.1097/PSY.0b013e31815a8f6b. [DOI] [PubMed] [Google Scholar]
- Gutner CA, Pineles SL, Griffin MG, Bauer MR, Weierich MR, Resick PA, et al. Physiological predictors of posttraumatic stress disorder. J Trauma Stress. 2010;23((6)):775–84. doi: 10.1002/jts.20582. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellard KK, Fairholme CP, Boisseau CL, Farchione TJ, Barlow DH, et al. Unified protocol for the transdiagnostic treatment of emotional disorders: protocol development and initial outcome data. Cogn Behav Pract. 2010;17((1)):88–101. doi: 10.1016/j.cbpra.2009.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barlow DH, Farchione TJ, Zavala S, et al, et al. Unified protocol for transdiagnostic treatment of emotional disorders: therapist guide. Oxford University Press. 2018 [Google Scholar]
- Barlow DH, Sauer-Zavala S, Carl JR, Bullis JR, Ellard KK, et al. The nature, diagnosis, and treatment of neuroticism: back to the future. Clin Psychol Sci. 2014;2((3)):344–65. [Google Scholar]
- Cassiello-Robbins C, Southward MW, Tirpak JW, Sauer-Zavala S, et al. A systematic review of unified protocol applications with adult populations: facilitating widespread dissemination via adaptability. Clin Psychol Rev. 2020:101852–65. doi: 10.1016/j.cpr.2020.101852. [DOI] [PubMed] [Google Scholar]
- Sakiris N, Berle D, et al. A systematic review and meta-analysis of the unified protocol as a transdiagnostic emotion regulation based intervention. Clin Psychol Rev. 2019:101751–65. doi: 10.1016/j.cpr.2019.101751. [DOI] [PubMed] [Google Scholar]
- Pearl SB, Norton PJ, et al. Transdiagnostic versus diagnosis specific cognitive behavioural therapies for anxiety: a meta-analysis. J Anxiety Disord. 2017:11–24. doi: 10.1016/j.janxdis.2016.07.004. [DOI] [PubMed] [Google Scholar]
- Newby JM, McKinnon A, Kuyken W, Gilbody S, Dalgleish T, et al. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clin Psychol Rev. 2015:91–110. doi: 10.1016/j.cpr.2015.06.002. [DOI] [PubMed] [Google Scholar]
- Reinholt N, Krogh J, et al. Efficacy of transdiagnostic cognitive behaviour therapy for anxiety disorders: a systematic review and meta-analysis of published outcome studies. Cogn Behav Ther. 2014;43((3)):171–84. doi: 10.1080/16506073.2014.897367. [DOI] [PubMed] [Google Scholar]
- Bullis JR, Sauer-Zavala S, Bentley KH, Thompson-Hollands J, Carl JR, Barlow DH, et al. The unified protocol for transdiagnostic treatment of emotional disorders: preliminary exploration of effectiveness for group delivery. Behav Modif. 2015;39((2)):295–321. doi: 10.1177/0145445514553094. [DOI] [PubMed] [Google Scholar]
- Bullis JR, Fortune MR, Farchione TJ, Barlow DH, et al. A preliminary investigation of the long-term outcome of the unified protocol for transdiagnostic treatment of emotional disorders. Compr Psychiatry. 2014;55((8)):1920–7. doi: 10.1016/j.comppsych.2014.07.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bullis JR, Barlow DH, et al. The unified protocol for transdiagnostic treatment of emotion disorders: a progress report. Clin Psychol. 2015;68((3)):4–15. [Google Scholar]
- Farchione TJ, Bullis JR, et al. Addressing the global burden of mental illness: why transdiagnostic and common elements approaches to evidence-based practice might be our best bet. Cognit Behav Pract. 2014;21((2)):124–6. [Google Scholar]
- Sauer-Zavala S, Boswell JF, Gallagher MW, Bentley KH, Ametaj A, Barlow DH, et al. The role of negative affectivity and negative reactivity to emotions in predicting outcomes in the unified protocol for the transdiagnostic treatment of emotional disorders. Behav Res Ther. 2012;50((9)):551–7. doi: 10.1016/j.brat.2012.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson-Brenner H, Brooks GE, Boswell JF, et al, et al. Evidence-based implementation practices applied to the intensive treatment of eating disorders: summary of research and illustration of principles using a case example. Clin Psychol Sci Pract. 2018:e12221–7. [Google Scholar]
- Thompson-Brenner H, Singh S, Gardner T, et al, et al. The Renfrew Unified Treatment for Eating Disorders and Comorbidity: long-term effects of an evidence-based practice implementation in residential treatment. Front Psychiatry. 2021:641601–7. doi: 10.3389/fpsyt.2021.641601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson-Brenner H, Boswell JF, Espel-Huynh H, Brooks G, Lowe MR, et al. Implementation of transdiagnostic treatment for emotional disorders in residential eating disorder programs: a preliminary pre-post evaluation. Psychother Res. 2019;29((8)):1045–61. doi: 10.1080/10503307.2018.1446563. [DOI] [PubMed] [Google Scholar]
- Youn S, Sauer-Zavala S, Patrick K, et al, et al. Barriers and facilitators to implementing a transdiagnostic treatment for homeless problems. J Nerv Ment Dis. 2019:585–94. doi: 10.1097/NMD.0000000000001010. [DOI] [PubMed] [Google Scholar]
- Sauer-Zavala S, Ametaj AA, Wilner JG, et al, et al. Evaluating transdiagnostic, evidence-based mental health care in a safety-net setting serving homeless individuals. Evaluating transdiagnostic, evidence-based mental health care in a safety-net setting serving homeless individuals. Psychotherapy (Chic) 2019;56((1)):100–14. doi: 10.1037/pst0000187. [DOI] [PubMed] [Google Scholar]
- Bentley KH, Boettcher H, Bullis JR, et al, et al. Development of a single-session, transdiagnostic preventive intervention for young adults at risk for emotional disorders. Behav Modif. 2018:781–805. doi: 10.1177/0145445517734354. [DOI] [PubMed] [Google Scholar]
- Gallagher MW, et al. Transdiagnostic mechanisms of change and cognitive-behavioral treatments for PTSD. Curr Opin Psychol. 2017:90–5. doi: 10.1016/j.copsyc.2016.12.002. [DOI] [PubMed] [Google Scholar]
- Maia AC, Nardi AE, Cardoso A, et al. The utilization of unified protocols in behavioral cognitive therapy in transdiagnostic group subjects: a clinical trial. J Affect Disord. 2015:179–83. doi: 10.1016/j.jad.2014.09.023. [DOI] [PubMed] [Google Scholar]
- Blumberg DM, Schlosser MD, Papazoglou K, Creighton S, Kaye CC, et al. New directions in police academy training: a call to action. Int J Environ Res Public Health. 2019;16((24)):E4941–83. doi: 10.3390/ijerph16244941. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berking M, Meier C, Wupperman P, et al. Enhancing emotion-regulation skills in police officers: results of a pilot controlled study. Behav Ther. 2010:329–39. doi: 10.1016/j.beth.2009.08.001. [DOI] [PubMed] [Google Scholar]
- Romosiou V, Brouzos A, Vassilopoulos SP, et al. An integrative group intervention for the enhancement of emotional intelligence, empathy, resilience and stress management among police officers. Police Pract Res. 2019;20((5)):460–78. [Google Scholar]
- Chitra T, Karunanidhi S, et al. The impact of resilience training on occupational stress, resilience, job satisfaction, and psychological well-being of female police officers. J Police Crim Psychol. 2021;36((1)):8–23. [Google Scholar]
- tzig G, Hudy C, Haberfeld MR, Clarke CA, Sheehan DL, et al. From theory to practice: simulation technology as a training tool in law enforcement. Springer. 2012:65–79. [Google Scholar]
- Dochy F, Segers M, Bossche P, Gijbels D, et al. Effects of problem-based learning: a meta-analysis. Learning Instr. 2003;13((5)):533–68. [Google Scholar]
- Wang X, Kattan MW, et al. Cohort studies design, analysis, and reporting. Chest. 2020;158((1)):S72–S8. doi: 10.1016/j.chest.2020.03.014. [DOI] [PubMed] [Google Scholar]
- Louie CE, D’Agostino E, Woods A, Ryken T, et al. Study design in neurosurgical research: considerations for observational and experimental cohort studies. Neurosurgery. 2020;86((1)):14–8. doi: 10.1093/neuros/nyz386. [DOI] [PubMed] [Google Scholar]
- George Washington University. Washington(DC): Study design 101: cohort study [Internet] pp. cohort study [Internet]–8. Available from: https://himmelfarb.gwu.edu/tutorials/studydesign101/cohorts.cfm. [Google Scholar]
- Anglemyer A, Horvath HT, Bero L, et al. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. :MR000034–8. doi: 10.1002/14651858.MR000034.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Concato J, Shah N, Horwitz RI, et al. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000:1887–92. doi: 10.1056/NEJM200006223422507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Royal Canadian Mounted Police. Ottawa(ON): Qualifications and standards to become an RCMP officer [Internet] Available from: https://www.rcmp-grc.gc.ca/en/qualifications-and-requirements. [Google Scholar]
- Royal Canadian Mounted Police. Ottawa(ON): Cadet training [Internet] Available from: https://www.rcmp-grc.gc.ca/en/qualifications-and-requirements. [Google Scholar]
- Roy A, Bhaumik DK, Aryal S, Gibbons RD, et al. Sample size determination for hierarchical longitudinal designs with differential attrition rates. Biometrics. 2007;63((3)):699–707. doi: 10.1111/j.1541-0420.2007.00769.x. [DOI] [PubMed] [Google Scholar]
- Donner A, Klar N, et al. Arnold. London(UK): 2000. Design and analysis of cluster randomization trials in health research. [Google Scholar]
- Komarovskaya I, Maguen S, McCaslin SE, et al, et al. The impact of killing and injuring others on mental health symptoms among police officers. J Psychiatr Res. 2011;45((10)):1332–6. doi: 10.1016/j.jpsychires.2011.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yuan C, Wang Z, Inslicht SS, et al, et al. Protective factors for posttraumatic stress disorder symptoms in a prospective study of police officers. Psychiatry Res. 2011;188((1)):45–50. doi: 10.1016/j.psychres.2010.10.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bowler RM, Harris M, Li J, et al, et al. Longitudinal mental health impact among police responders to the 9/11 terrorist attack. Am J Ind Med. 2012:297–312. doi: 10.1002/ajim.22000. [DOI] [PubMed] [Google Scholar]
- Obst PL, Davey JD, Sheehan MC, et al. Does joining the police service drive you to drink. Drugs Educ Prev Policy. 2001;8((4)):347–57. [Google Scholar]
- Wortley RK, Homel RJ, et al. Police prejudice as a function of training and outgroup contact: a longitudinal investigation. Law Hum Behav. 1995:305–17. [Google Scholar]
- Lee HJ, Goudarzi K, Baldwin B, Rosenfield D, Telch MJ, et al. The Combat Experience Log: a web-based system for the in theater assessment of war zone stress. J Anxiety Disord. 2011:794–800. doi: 10.1016/j.janxdis.2011.03.018. [DOI] [PubMed] [Google Scholar]
- Telch MJ, et al. Predicting PTSD, anxiety, and depressive symptoms in the war zone: findings from the Texas Combat PTSD Risk Project. Telch MJ, editor [Google Scholar]
- Workman-Stark A, et al. From exclusion to inclusion: a proposed approach to addressing the culture of masculinity within policing. Equal Divers Incl. 2015;34((8)):764–75. [Google Scholar]
- Ashlock JM, et al. Gender attitudes of police officers: selection and socialization mechanisms in the life course. Soc Sci Res. 2019:71–84. doi: 10.1016/j.ssresearch.2018.12.008. [DOI] [PubMed] [Google Scholar]
- Silvestri M, et al. Police culture and gender: revisiting the ‘cult of masculinity’. Policing. 2017;11((3)):289–300. [Google Scholar]
- Sheehan D, Lecrubier Y, Sheehan K, et al, et al. The validity of the Mini International Neuropsychiatric Interview (M.I.N.I.) according to the SCID-P and its reliability. Eur Psychiat. 1997;12((5)):232–41. [Google Scholar]
- Sheehan DV, Lecrubier Y, Sheehan K, et al, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998:22-33–quiz 34-57. [PubMed] [Google Scholar]
- Lecrubier Y, Sheehan D, Weiller E, et al, et al. The Mini International Neuropsychiatric Interview (M.I.N.I.): a short diagnostic structured interview: reliability and validity according to the CIDI. Eur Psychiat. 1997;12((5)):224–31. [Google Scholar]
- th ed, et al. Washington(DC): 2013. Diagnostic and Statistical Manual of Mental Disorders. [Google Scholar]
- Thompson-Hollands J, Bentley KH, Gallagher MW, Boswell JF, Barlow DH, et al. Credibility and outcome expectancy in the unified protocol: relationship to outcomes. J Exp Psychopathol. 2014;5((1)):72–82. [Google Scholar]
- Burke LE, Wang J, Sevick MA, et al. Self-monitoring in weight loss: a systematic review of the literature. J Am Diet Assoc. 2011;111((1)):92–102. doi: 10.1016/j.jada.2010.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steinberg DM, Bennett GG, Askew S, Tate DF, et al. Weighing every day matters: daily weighing improves weight loss and adoption of weight control behaviors. J Acad Nutr Diet. 2015;115((4)):511–8. doi: 10.1016/j.jand.2014.12.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- n MB, Hasson D, Theorell T, Canlon B, Osika W, et al. Possibilities and limitations of the Polar RS800 in measuring heart rate variability at rest. Eur J Appl Physiol. 2012;112((3)):1153–65. doi: 10.1007/s00421-011-2079-9. [DOI] [PubMed] [Google Scholar]
- Sassi R, Cerutti S, Lombardi F, et al, et al. Advances in heart rate variability signal analysis: joint position statement by the e-Cardiology ESC Working Group and the European Heart Rhythm Association co-endorsed by the Asia Pacific Heart Rhythm Society. Europace. 2015;17((9)):1341–53. doi: 10.1093/europace/euv015. [DOI] [PubMed] [Google Scholar]
- Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation. 1996;93((5)):1043–65. [PubMed] [Google Scholar]
- Sacha J, Sobon J, Sacha K, Barabach S, et al. Heart rate impact on the reproducibility of heart rate variability analysis. Int J Cardiol. 2013;168((4)):4257–9. doi: 10.1016/j.ijcard.2013.04.160. [DOI] [PubMed] [Google Scholar]
- Barbosa TC, Vianna LC, Hashimoto T, et al, et al. Carotid baroreflex function at the onset of cycling in men. Am J Physiol Regul Integr Comp Physiol. 2016;311((5)):R870–8. doi: 10.1152/ajpregu.00173.2016. [DOI] [PubMed] [Google Scholar]
- Seiler S, Haugen O, Kuffel E, et al. Autonomic recovery after exercise in trained athletes: intensity and duration effects. Med Sci Sports Exerc. 2007;39((8)):1366–73. doi: 10.1249/mss.0b013e318060f17d. [DOI] [PubMed] [Google Scholar]
- Appelhans BM, Luecken LJ, et al. Heart rate variability as an index of regulated emotional responding. Rev Gen Psychol. 2006;10((3)):229–40. [Google Scholar]
- Parati G, Saul JP, Rienzo M, Mancia G, et al. Spectral analysis of blood pressure and heart rate variability in evaluating cardiovascular regulation. Hypertension. 1995;25((6)):1276–86. doi: 10.1161/01.hyp.25.6.1276. [DOI] [PubMed] [Google Scholar]
- Taylor JA, Studinger P, et al. Counterpoint: cardiovascular variability is not an index of autonomic control of the circulation. J Appl Physiol (1985) 2006:678-81–discussion 81. doi: 10.1152/japplphysiol.00446.2006. [DOI] [PubMed] [Google Scholar]
- Parati G, Mancia G, Rienzo M, Castiglioni P, et al. Point: cardiovascular variability is/is not an index of autonomic control of circulation. J Appl Physiol (1985) 2006;101((2)):676-8–discussion 81-2. doi: 10.1152/japplphysiol.00446.2006. [DOI] [PubMed] [Google Scholar]
- rensen T, Mogelvang R, Jensen JS, et al. Prognostic value of cardiac time intervals measured by tissue Doppler imaging M-mode in the general population. Heart. 2015;101((12)):954–60. doi: 10.1136/heartjnl-2014-307137. [DOI] [PubMed] [Google Scholar]
- Government of Canada. Ottawa(ON): Table 1: Rates of selected mental or substance use disorders, lifetime and 12 month, Canada, household population 15 and older, 2012. Available from: https://www150.statcan.gc.ca/n1/pub/82-624-x/2013001/article/tbl/tbl1-eng.htm. [Google Scholar]
- Carleton RN, et al. Into the unknown: a review and synthesis of contemporary models involving uncertainty. J Anxiety Disord. 2016:30–43. doi: 10.1016/j.janxdis.2016.02.007. [DOI] [PubMed] [Google Scholar]
- Jacquart J, Dutcher CD, Freeman SZ, et al, et al. The effects of exercise on transdiagnostic treatment targets: A meta-analytic review. Behav Res Ther. 2019:19–37. doi: 10.1016/j.brat.2018.11.007. [DOI] [PubMed] [Google Scholar]
- Powers MB, Asmundson GJ, Smits JA, et al. Exercise for mood and anxiety disorders: the state-of-the science. Cogn Behav Ther. 2015;44((4)):237–9. doi: 10.1080/16506073.2015.1047286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sommer JL, El-Gabalawy R, Taillieu T, Afifi TO, Carleton RN, et al. Associations between trauma exposure and physical health conditions among public safety personnel. Can J Psychiatry. 2020;65((8)):548–58. doi: 10.1177/0706743720919278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Graham JW, et al. Missing data: analysis and design. Springer. 2012 [Google Scholar]
- Caille A, Leyrat C, Giraudeau B, et al. A comparison of imputation strategies in cluster randomized trials with missing binary outcomes. Stat Methods Med Res. 2016;25((6)):2650–69. doi: 10.1177/0962280214530030. [DOI] [PubMed] [Google Scholar]
- Farcomeni A, et al. A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion. Stat Methods Med Res. 2008;17((4)):347–88. doi: 10.1177/0962280206079046. [DOI] [PubMed] [Google Scholar]
- Nakagawa S, et al. A farewell to Bonferroni: The problems of low statistical power and publication bias. Behav Ecol. 2004;15((6)):1044–5. [Google Scholar]
- Hsieh HF, Shannon SE, et al. Three approaches to qualitative content analysis. Qual Health Res. 2005;15((9)):1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- Murnane EL, Cosley D, Chang P, et al, et al. Self-monitoring practices, attitudes, and needs of individuals with bipolar disorder: implications for the design of technologies to manage mental health. J Am Med Inform Assoc. 2016;23((3)):477–84. doi: 10.1093/jamia/ocv165. [DOI] [PubMed] [Google Scholar]
- Gatto A, Miyazaki Y, Cooper LD, et al. Help me help myself: examining an electronic mental health self-monitoring system in college students. High Educ. 2020;83((2)):1–20. [Google Scholar]
- Ehlers A, Clark DM, Hackmann A, et al, et al. A randomized controlled trial of cognitive therapy, a self-help booklet, and repeated assessments as early interventions for posttraumatic stress disorder. Arch Gen Psychiatry. 2003:1024–32. doi: 10.1001/archpsyc.60.10.1024. [DOI] [PubMed] [Google Scholar]
- Carleton RN, Afifi TO, Turner S, et al, et al. Mental health training, attitudes towards support, and screening positive for mental disorders. Cogn Behav Ther. 2020;49((1)):55–73. doi: 10.1080/16506073.2019.1575900. [DOI] [PubMed] [Google Scholar]
- Corrigan PW, Druss BG, Perlick DA, et al. The impact of mental illness stigma on seeking and participating in mental health care. Psychol Sci Publ Int. 2014;15((2)):37–70. doi: 10.1177/1529100614531398. [DOI] [PubMed] [Google Scholar]
- Ricciardelli R, Carleton RN, Mooney T, Cramm H, et al. “Playing the system”: structural factors potentiating mental health stigma, challenging awareness, and creating barriers to care for Canadian public safety personnel. Health (London) 2020;24((3)):259–78. doi: 10.1177/1363459318800167. [DOI] [PubMed] [Google Scholar]
- Krakauer RL, Stelnicki AM, Carleton RN, et al. Examining mental health knowledge, stigma, and service use intentions among public safety personnel. Front Psychol. 2020:949–78. doi: 10.3389/fpsyg.2020.00949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vig KD, Mason JE, Carleton RN, Asmundson GJ, Anderson GS, Groll D, et al. Mental health and social support among public safety personnel. Mental health and social support among public safety personnel. Occup Med (Lond) 2020;70((6)):427–33. doi: 10.1093/occmed/kqaa129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carleton RN, Afifi TO, Taillieu T, et al, et al. Assessing the relative impact of diverse stressors among public safety personnel. Int J Environ Res Public Health. 2020;17((4)):1234–33. doi: 10.3390/ijerph17041234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ogbonnaya C, Tillman CJ, Gonzalez K, et al. Perceived organizational support in health care: the importance of teamwork and training for employee well-being and patient satisfaction. Group Organ Manage. 2018;43((3)):475–503. [Google Scholar]
- Curry OS, Rowland LA, Lissa CJ, Zlotowitz S, McAlaney J, Whitehouse H, et al. Happy to help. J Exp Soc Psychol. 2018:320–9. [Google Scholar]
- Hui BP, Ng JC, Berzaghi E, Cunningham-Amos LA, Kogan A, et al. Rewards of kindness. Psychol Bull. 2020:1084–116. doi: 10.1037/bul0000298. [DOI] [PubMed] [Google Scholar]
- Nota PM, Bahji A, Groll D, Carleton RN, Anderson GS, et al. Proactive psychological programs designed to mitigate posttraumatic stress injuries among at-risk workers: a systematic review and meta-analysis. Syst Rev. 2021;10((1)):126–116. doi: 10.1186/s13643-021-01677-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stelnicki AM, Jamshidi L, Ricciardelli R, Carleton RN, et al. Exposures to potentially psychologically traumatic events among nurses in Canada. Can J Nurs Res. 2021;53((3)):277–91. doi: 10.1177/0844562120961988. [DOI] [PubMed] [Google Scholar]
- Stelnicki AM, Carleton RN, et al. Mental disorder symptoms among nurses in Canada. Can J Nurs Res. 2021;53((3)):264–76. doi: 10.1177/0844562120961894. [DOI] [PubMed] [Google Scholar]
- Petkovic G, Charlesworth JE, Kelley J, Miller F, Roberts N, Howick J, et al. Effects of placebos without deception compared with no treatment: protocol for a systematic review and meta-analysis. BMJ Open. 2015;5((11)):e009428–76. doi: 10.1136/bmjopen-2015-009428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Locher C, Nascimento A, Kirsch I, Kossowsky J, Meyer A, Gaab J, et al. Is the rationale more important than deception. Pain. 2017;158((12)):2320–8. doi: 10.1097/j.pain.0000000000001012. [DOI] [PubMed] [Google Scholar]
- Osborne JW, et al. Best practices in quantitative methods. Sage Publications Inc. 2008 [Google Scholar]