Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 1.
Published in final edited form as: Appl Nurs Res. 2019 Nov 6;51:151199. doi: 10.1016/j.apnr.2019.151199

Methodological challenges in conducting instrumentation research in non-communicative palliative care patients

Karen Snow Kaiser a,*,1, Deborah B McGuire b,2,3, Timothy J Keay c,3, Mary Ellen Haisfield-Wolfe d,3
PMCID: PMC7200296  NIHMSID: NIHMS1544318  PMID: 31759841

Abstract

Well-designed, rigorously implemented instrumentation studies are essential to develop valid, reliable pain assessment tools in non-communicative (non-self-reporting) palliative care patients. When conducting a pain instrumentation study, a research team identified methodologic challenges surrounding informed consent, eligibility criteria, acute pain operational definitions, patient recruitment, missing data, and study-related training during a run-in phase at the beginning of the project and during the conduct of the study. The team dealt with these challenges through identifying root causes, implementing remedial measures, and collecting data to demonstrate improvement or resolution. Effective strategies included obtaining Institutional Review Board (IRB) approval for a waiver of informed consent, modifying eligibility criteria, ensuring that operational definitions and study procedures were consistent with clinical practice, decreasing time from screening to data collection to improve recruitment, increasing study nurse staffing by re-budgeting grant funds, focusing time and resources on high accruing clinical units, revising processes to minimize missing data, and developing detailed training for users of the instrument. With these multi-pronged solutions, the team exceeded the patient accrual target by 25% within the funding period and reduced missing data. While pain instrumentation studies in non-communicative patients have similar challenges to other palliative care studies, some of the solutions may be unique and several are applicable to other palliative care studies, particularly instrumentation research. The team’s experience may also be useful for funders and IRBs.

Keywords: Pain assessment instrument, Palliative care, Non-communicative patients, Informed consent waiver, Methodologic challenges

Uncontrolled pain is a major issue in many adult palliative care patients (Coyne, Mulvenon, & Paice, 2018), with non-communicative patients (i.e., those who cannot self-report their pain) at higher risk for adverse outcomes (Coyne et al., 2018; Herr, Coyne, McCaffery, Manworren, & Merkel, 2011). Adequate pain management is a measure of palliative care effectiveness (National Consensus Project for Quality Palliative Care, 2018), but there is no “gold standard” for assessing pain in these often frail patients who have a variety of painful conditions and are cared for in multiple settings (Herr et al., 2011). Robust and effective pain assessment and management strategies in palliative care patients have focused primarily on those who can self-report, neglecting the many non-communicative palliative care patients who are critically ill and/or at the end of life (Puntillo et al., 2014). In addition pain assessment tools developed for non-communicative patients often use patient self-report for comparison, which limits their validity (McGuire, Kaiser, Haisfield-Wolfe, & Iyamu, 2016; Barzanji, Zarelyan, Nezamzadeh, & Marzhari, 2019), thus additional guidance is necessary regarding instrument development research in non-communicative palliative care subjects.

Researchers conducting randomized controlled trials (RCTs) in palliative care patients have identified numerous difficulties (Bouca-Machado et al., 2017; Hanson et al., 2014; Hussain et al., 2016; Snowden & Young, 2017), but published information is scarce on the challenges associated with conducting clinical pain research in non-communicative patients, particularly instrument development studies. Moreover, it is unclear whether the challenges that have been described by other palliative care and pain researchers apply to the non-communicative population. The purposes of this paper are two-fold: 1) to describe the challenges encountered by the authors while conducting a pain instrument development study in non-communicative adult palliative care patients in an acute care setting, and 2) to discuss the research team’s solutions and outcomes. The information presented herein has important implications for researchers who are studying this vulnerable population, as well as for funding agencies and Institutional Review Boards.

1. Background

Palliative care is an approach that improves patients’ and families’ quality of life associated with life threatening illness through early identification, prevention, and relief of pain and suffering (World Health Organization, 2010). Many patients cared for in acute care hospital settings meet this definition and may also be non-communicative for at least part of their hospitalization due to various illnesses and injuries, such as post-surgical status, medical sedation, or brain injury. These patients may experience pain caused by illness or injury, medical procedures, and nursing activities (such as turning or repositioning that can be painful), but this pain is often unrecognized or undertreated (Puntillo et al., 2014). In clinical settings, a valid and reliable pain assessment tool for non-communicative patients could help determine the need for and effectiveness of pain interventions as well as monitor and evaluate quality improvement interventions (Barr et al., 2013). Ultimately it should minimize the morbidity and mortality associated with pain (Azevedo-Santos & DeSantana, 2018; Coyne et al., 2018; Herr et al., 2011). Similarly, in research settings, such a tool could be used to evaluate pain management interventions. However, without a standardized tool, it can be difficult to make comparisons across studies. Since non-communicative adult palliative patients are cared for in a variety of settings, the development of a valid and reliable pain assessment tool that can be used across settings is imperative. This paper is based on a large, federally funded study that addressed this urgent clinical and research need.

Two commonly reported challenges in palliative care research are recruitment and retention of subjects (Hanson et al., 2014; Hui, Glitza, Chisholm, Yennu, & Bruera, 2013). Suboptimal accrual can affect study power, compromising the ability to detect significant differences and to generalize findings. Gate-keeping, which may be conscious or unconscious (Snowden & Young, 2017), has been reported as a reason for the failure of many palliative care studies to accrue adequate samples (Kars et al., 2016; Snowden & Young, 2017). It occurs in various types of studies, with the most frequent reason being the perceived or actual fear of burdening vulnerable patients, or conversely, the desire to protect them (Hanson et al., 2014; Kars et al., 2016). Once enrolled, retention of palliative care patients is also a challenge, with studies experiencing attrition for a variety of reasons, that are expected in this population such as disease progression, fatigue, frailty, illness acuity, non-communication, and death (Aoun & Nekolaichuk, 2014; Hanson et al., 2014; Hui et al., 2013; Hussain et al., 2016). The principal concern with attrition in palliative care research is the potential for not achieving sufficient statistical power and for missing data that can interfere with analysis and interpretation of results.

The literature about challenges in palliative care research focuses on observational and randomized controlled trials. The authors did not know the degree to which these identified or additional challenges might affect the rigor and completion of their instrumentation study (McGuire, Kaiser, Soeken, Reifsnyder, & Keay, 2011a), but anticipated that the recruitment and retention issues described in other palliative care studies could be even more prevalent in a sample comprised solely of non-communicative palliative care patients and that additional problems might arise. Thus, they conducted a run-in phase at the start of the study prior to enrollment of patient subjects. During the run-in phase and once patient enrollment commenced, the authors carefully monitored and addressed challenges that arose. Following a brief description of the study, the authors describe the challenges they encountered, approaches they used to improve the rigor of the research, and the relevant outcomes.

2. Description of the study

Based on the paucity of valid, reliable pain assessment tools for non-communicative patients, particularly the critically ill (Azevedo-Santos & DeSantana, 2018), and on preliminary studies of a pain assessment measure using behavioral and physiologic pain indicators in cognitively impaired or non-responsive hospice patients (McGuire, Reifsnyder, Soeken, Kaiser, & Yeager, 2011b), the authors initiated a full-scale psychometric evaluation of the measure, the Multidimensional Objective Pain Assessment Tool (MOPAT). The multi-site study was conducted in tertiary acute care hospital and inpatient hospice settings. The aims of the National Institute of Nursing Research-funded study were to: 1) test the reliability and validity of the MOPAT for assessing acute pain in non-communicative palliative care patients in an inpatient tertiary acute care hospital (McGuire, Kaiser, Soeken, Reifsnyder, & Keay, 2011a) and in the inpatient units of a community-based hospice (McGuire, Harrold, Kaiser, & Bortle, 2013), as well as 2) appraise the clinical utility of the MOPAT when used by nurses to assess acute pain in non-communicative patients in these settings. This paper only focuses on challenges encountered in the inpatient tertiary acute care hospital as the challenges encountered in the hospice setting were different and had less potential impact on study rigor. To provide context for the discussion of methodological challenges, Table 1 provides a comprehensive overview of the study’s methods in the setting.

Table 1.

Overview of study methods in the tertiary acute care hospital.

Study design element Details Comments
Design Cross-sectional instrumentation design in which pairs of nurse raters assessed patients before and after a pain-relieving intervention Rater pairs were either two study nurses (SNs) or one SN and one volunteer staff nurse (VSN)
Setting University-affiliated tertiary acute care hospital in a major urban center 22 inpatient units including intensive care, trauma, medicine, and surgery
Sample Patients: 1) met the WHO definition for receiving palliative care; 2) were receiving palliative care in the participating settings; 3) had at least one diagnosis, condition, or factor known to cause acute pain; and 4) were unable to communicate their pain by self-report or rating because of any one of several defined non-communicative states. N = 163 (target was 130)
Nurses: 1) were staff nurses, 2) cared for palliative care patients on participating inpatient units, and 3) consented to participate in the study N = 47 (target was 20) (these nurses were called VSNs)
Instruments and measures MOPAT (before/after pain-relieving intervention)
Richmond Agitation-Sedation Scale (RASS) (Sessler et al., 2002) (at study entry and before a pain rating)
Palliative Performance Scale v2 (PPSv2) (Palliative Performance Scale v.2, 2006) (at study entry and before a pain rating)
Patient Data Form (PDF) (at study entry)
Nurse Data Form (NDF) (at study entry)
Clinical Utility Questionnaire (CUQ) (monthly)		 MOPAT completed by SNs and VSNs
RASS, PPSv2, and PDF completed by SNs
NDF and CUQ completed by VSNs
Procedures Patients meeting eligibility criteria were enrolled, and the RASS, PPSv2, and PDF completed. VSNs completed the NDF, were trained in use of MOPAT, and completed CUQ monthly. To rate patients’ pain, rater pairs verified eligibility, then rated patients’ pain using the MOPAT before and after a pharmacologic or non-pharmacologic pain-relieving intervention. Provided data for a cross-sectional analysis of reliability and validity, and a summary analysis of clinical utility over time
Analysis Reliability: Cronbach’s alpha coefficient, inter-rater reliability appraisal, and generalizability coefficients of raters and items Valid analyses require achievement of minimum target sample sizes noted above and minimal missing data
Validity: Sensitivity analysis comparing changes in pain from before to after a pain intervention
Clinical utility: Descriptive analysis of nurses’ perceptions of using the MOPAT
Open in a new tab

3. Methodological challenges

The authors conducted an approximately two week-long run-in phase before initiation. The purpose of the run-in phase was to carry out study procedures in various participating inpatient units (e.g., critical care, step-down, intermediate care, acute care, trauma). The explicit intent was to identify potential issues and resolve them to facilitate study implementation and completion. During the run-in phase and after study implementation, the team met routinely to review study progress, identify methodological challenges relating to the conduct of the research, and develop solutions. Specific examples of these challenges accompanied by discussion of strategies used by the team to address them, and the resulting outcomes are described below.

3.1. Informed consent

A major challenge in any clinical research involving non-communicative patients is obtaining informed consent. This study originally required written informed consent from each patient’s legally authorized representative (LAR), but this approach rapidly emerged as a major issue when only 16 patients were enrolled in six months. Analysis of screening and enrollment data for a four-month period revealed that numerous patients were eligible, but three critical factors interfered with enrollment: 1) the window of opportunity for enrollment was narrow due to patients’ rapidly changing medical conditions, 2) LARS who needed to provide consent were frequently not available during this window, and 3) study staff members’ attempts to contact LARs were often unsuccessful. Thus, the study was losing more than two-thirds of potential subjects and was at significant risk of not achieving its aims within the funding period. Because the IRB did not sanction fax or verbal telephone consents, the research team determined after consulting with IRB staff that a waiver of informed consent coupled with a full privacy waiver to enable collection of demographic and clinical information was the most practical solution. The authors formally requested a waiver of written informed consent from the IRB based on the following justification: 1) risk level would remain minimal even if consent was waived, 2) a waiver would not adversely affect rights and welfare of subjects, 3) data collection procedures for obtaining pain assessments would essentially be no different from standard clinical nursing practice when assessing pain in non-communicative patients, 4) study nurses (SNs) would not touch or move patients in any way and would not administer pain interventions, 5) only de-identified data would be collected, and 6) the practicality of obtaining written informed consent was not feasible. Following IRB approval of the waiver, enrollment of patient subjects increased to 44 subjects in three and one-half months with an eventual overall accrual of 163 patients, exceeding the minimum target sample size of 130 required to perform sufficiently powered statistical analyses. There was no negative feedback from clinicians or LARs after this change was made.

3.2. Eligibility criteria

An issue that arose during the study’s run-in phase was a mismatch between the inclusion/exclusion criteria and the eligibility screening tool (the Richmond Agitation-Sedation Scale [RASS]) in these non-communicative patients. The RASS required patients to be able to see, hear, and move with or without stimulation, and similarly, the MOPAT required observation of extremity movement. Therefore, study eligibility criteria were modified to exclude patients with hearing loss, severely impaired visual acuity, or quadriplegia. In addition, a clear definition of acute pain was important since the MOPAT measured behavioral and physiologic indicators of pain which may be diminished in chronic pain. Therefore, during the grant proposal development phase the team added an eligibility criterion that stipulated patients have at least one diagnosis, condition, or factor known to cause acute pain according to common definitions (Foley, 2005; International Association for the Study of Pain Task Force on Acute Pain, 1992). These revised criteria helped ensure study integrity and more precise generalization of findings.

3.3. Adequate operational definitions of pain

The study design called for two MOPAT ratings, one before and one after a pain-relieving intervention. Early in the study, the research team discovered that clinical nursing staff did not usually provide additional pain medication to patients prior to nursing activities and procedures causing episodic or procedural pain. For example, patients receiving non-pharmacologic interventions for pain (such as repositioning) that caused brief episodic pain were not pre-medicated. This scenario likely occurred because some were seriously ill medical, surgical, or trauma patients receiving continuous intravenous analgesia and sedation and had well-controlled pain with the maximum RASS level of sedation allowed (−4). In addition, the ultimate goal of the non-pharmacologic pain intervention was pain reduction. Consistent with the existing operational definitions of pain (see Table 2), the team modified study procedures. The modification stipulated that when no additional analgesic modality was used for the activity or procedure, the MOPAT “before” rating would be obtained during the painful intervention or procedure, while the “after” rating would be obtained at a suitable interval following the painful intervention or procedure. The SNs then worked closely with bedside staff nurses to coordinate being present at the time of brief painful activities such as turning, getting a patient into or out of bed, or during physical therapy, even when patients did not receive a pain-relieving pharmacologic or non-pharmacologic intervention. This strategy increased patient enrollment, particularly for procedural pain, and allowed generalization of the findings to procedural and episodic pain as intended in the study.

Table 2.

Operational definitions of pain.

Category Operational definition
Episodic pain Pain that occurred briefly during functional activities such as bathing, turning, getting in or out of bed, or during physical or occupational therapy.
Procedural pain Pain that occurred as a result of a procedure such as peripheral blood draws, suctioning, dressing change, debridement, or nasogastric tube placement.
Uncontrolled pain (escalating persistent pain) Pain characterized by increasing intensity that appeared to be due to underlying disease, new trauma, other medical conditions, or inadequate analgesia.
Open in a new tab

3.4. Patient recruitment

Another methodological challenge encountered was recruitment of a sufficient number of eligible patients to achieve the target sample size (separate from the issue of informed consent). Palliative care patients in the tertiary acute care hospital setting were sometimes medically unstable, moving quickly between self-reporting and not communicating, with fluctuating levels of sedation. If there was a delay between eligibility screening and enrollment, they often became ineligible for MOPAT pain ratings. Complicating this situation was the fact that the study had 22 participating inpatient nursing units, thus it was difficult for SNs to cover all units in their recruiting efforts. To maximize recruitment and accrual, SNs made more frequent rounds on the high-accruing inpatient units to screen for and enroll eligible patients, and to obtain paired pain ratings as soon as possible to avoid “losing” subjects. In addition, the team re-budgeted funds from other categories to increase the number of SN positions (with sponsor approval). Finally, although the study was IRB-approved as minimal risk, SNs occasionally encountered individual gate-keeping behavior among selected clinical staff that deemed patients too unstable to participate. In these instances, and depending on circumstances, SNs returned later to re-assess patients and enroll them if possible. Collectively, these strategies resulted in the research team exceeding its target accrual goal by 33 patients within the funding period.

3.5. Missing data

Missing data were minimized in patient subjects since SNs collected much of the data directly from the medical record and performed many of the MOPAT pain ratings with volunteer staff nurses (VSNs). They also reviewed all data collection forms for completeness while still at the data collection site. Despite these procedures, an unexpected source of missing patient data emerged early in the study. One of the MOPAT’s subscales, the Behavioral subscale, consisted of four behaviors rated from 0 = none/normal to 3 = most severe. The fourth item on the Behavioral subscale was for rating patient sounds (e.g., moaning). In rating patients who were unable to vocalize because they were intubated, SNs questioned what to record since a ‘0’ (‘no vocalization’) would not be accurate. To preserve the item and enable scoring the instrument as designed, the research team added a code of ‘4’ to the response options to indicate ‘unable to vocalize’ and made appropriate corrections to the small number of previously collected vocalization items. Ratings of ‘4’ were then subjected to an imputation formula during the psychometric analysis so that all items could be used (McGuire, Kaiser, Soeken, Reifsnyder, & Keay, 2011a; McGuire, Harrold, Kaiser, & Bortle, 2013; McGuire, Kaiser, Haisfield-Wolfe, & Iyamu, 2016) and subgroup analysis based on ability to vocalize could be performed. Missing VSN data was another concern. The Clinical Utility Questionnaires (CUQ), which were to be completed monthly by the VSNs, were not always completed. The team developed an approach that included individual email reminders from the project manager, and if needed, in-person follow-up visits to meet with VSNs who had not returned the CUQ. This strategy both reduced missing CUQ data and helped to identify reasons for non-return of the CUQs (e.g., vacation, extended leave, ineligibility due to transfer).

3.6. Design and implementation of training

Another challenge was the design and implementation of training for VSNs on how to use the MOPAT. To reduce the potential for subjective discrepancies in ratings of pain behaviors, the research team developed a set (demonstrating behaviors before and after a pain intervention) of three video vignettes using volunteers as patients. These vignettes portrayed some of the possible ratings on each of the MOPAT Behavioral subscale pain indicators (e.g., frowning/grimacing). Prior to study implementation, the research team and a group of pain/palliative care experts developed consensus ratings for each MOPAT item for each video vignette. New SNs and VSNs were trained on the MOPAT in groups or individually as follows. They received a didactic introduction to pain assessment in the non-communicative patient as well as the MOPAT and its subscales (including the none, mild, moderate and severe ratings of each item on the Behavioral subscale). Then trainees viewed the video vignettes (which were showed only some of the possible behaviors) and completed their own “before” and “after” MOPAT ratings. Their ratings were discussed with the research staff vis-à-vis the expert panel’s consensus ratings for three simulated patients, with an emphasis on reaching overall consensus. The vignettes were used by the SNs throughout the study to assess for drift and retrain. The training process was well-received by VSNs and seemed robust at the time. However, the team’s subsequent reliability analysis of the MOPAT suggested that more comprehensive and systematic training was needed (McGuire, Kaiser, Soeken, Reifsnyder, & Keay, 2011a). This challenge could not be solved during the study, but provided direction for the research team’s future research.

4. Discussion

The palliative care literature describes frequently encountered methodological challenges and solutions (Abernathy et al., 2014; Aoun & Nekolaichuk, 2014), although most reviews are based on clinical trials (Boland et al., 2015; Bouca-Machado et al., 2017; Hanson et al., 2014; Hickman, Cartwright, Nelson, & Knafl, 2012; Hui et al., 2013; Hussain et al., 2016; Kars et al., 2016; Polito et al., 2014; Rebers, Aaronson, van Leeuwen, & Schmidt, 2016) or longitudinal research (Teague et al., 2018), not instrument studies. Similarly, descriptions of how researchers identified challenges and developed solutions were absent in the literature. In their routine interdisciplinary research team meetings, the authors of this paper routinely tracked and evaluated study processes, which helped identify methodological challenges. They discussed and developed potential solutions, and monitored outcomes. Some of these challenges and solutions have been described by others and some appear to be previously unreported. Nonetheless, not all of the team’s challenges and solutions will be relevant to other palliative care studies.

4.1. Informed consent

Low recruitment has been identified as one of the common weaknesses of palliative care trials (Evans, Harding, Higginson, & MORECare, 2013). This issue was reported in an observational study about recruitment where the sample consisted of all critically ill patients eligible for one of 23 studies of various designs during a one-month period across Canada (Burns et al., 2013). Approximately 25% of the lost recruitment opportunities were due to a lack of, or difficulties in contacting, an authorized decision maker, which was substantially less than the two-thirds lost in the authors’ instrument study. The team took a primary approach different from those suggested by Burns et al. (2013) by working with the IRB to obtain approval for a waiver of informed consent as recommended by Abernathy et al. (2014). Waivers are controversial (Rebers et al., 2016), may not be available in certain countries (Regulation EU No 536/2014, 2014), and may not be approved for studies which are more than minimal risk. Since IRBs at different sites may differ in their rulings for the same study conducted at multiple sites, it is possible that other IRBs would not approve a waiver (Polito et al., 2014) or would require other processes such as deferred consent, community consultation, or independent safety monitoring (Rincon & Lee, 2015).

4.2. Eligibility criteria

The number of available subjects was also affected by the RASS, which was used to determine eligibility status. While physical capabilities (e.g., cognitive issues and near to death) can limit palliative care study participation (Hanratty et al., 2012), the patient’s abilities to see, hear, and move have not been identified as an important screening component for psychometric research when using behavioral-based tools. Patients who cannot mount behavioral responses (e.g., due to paralysis or deep sedation) on a behavioral-based pain measure may be incorrectly classified as having no pain or less pain than they are experiencing, thus introducing systematic bias. The authors found that rapid changes in mentation affected eligibility status, thus timely reevaluation of the patient’s communication ability prior to data collection using any behavioral, non-self-report pain assessment tool is required. Palliative care researchers need to carefully consider all physical abilities of potential non-communicative subjects since these can affect study eligibility, appropriateness of items on a measure, the potential for missing data, and validity of findings, including generalizability.

4.3. Adequate operational definitions of pain

Integrating palliative care studies into clinical setting routines is a well-known general strategy for addressing challenges in palliative care research (Bullen, Maher, Rosenberg, & Smith, 2014; Ersek, Polissar, Du Pen, Herr, & Neradilek, 2012). In contrast to these referenced studies, the research team modified study procedures so they more adequately represented what was occurring in the clinical setting while maintaining consistent operational definitions of pain throughout the study. This approach also had the effect of broadening the eligibility criteria, a strategy recommended by LeBlanc, Lodato, Currow, and Abernethy (2013), to increase the number of eligible participants.

4.4. Patient recruitment

This research team used multiple strategies to meet patient accrual targets. We closely monitored recruitment, increased the number of SN, focused on high accruing units to save time and resources, and were expedient (e.g., we focused on reducing the time between screening and enrollment). Other researchers have reported similar strategies (Demiris, Oliver, Capurro, & Wittenberg-Lyles, 2013; Fischer, Burgener, Kavanaugh, Ryan, & Keenan, 2012; Hickman et al., 2012; LeBlanc et al., 2013), but these were not reported in the one systematic review of recruitment strategies (Boland et al., 2015). Gatekeeping has also been identified as persistent problem responsible for many palliative care study failures (Kars et al., 2016). The small but real problem of clinician gate-keeping encountered in the authors’ minimal risk study was resolved with flexible SN schedules (e.g., multiple visits per unit per day, multiple visits per patient per day, or scheduled visits based on planned patient participant activities), a strategy to address this issue that has not been reported by other palliative care studies.

4.5. Missing data

Attrition in palliative care studies is high, ranging from a median of 22% (Bouca-Machado et al., 2017) to 44% (Hui et al., 2013) in systematic reviews of palliative care randomized trials. Missing data, largely caused by attrition, are a reason for poor quality palliative care research (Hussain et al., 2016; Hussain et al., 2017). The approaches that the team employed to meet this challenge (i.e., closely monitored attrition rates, decreasing time between patient enrollment and data collection, and using flexible data collection periods) have been employed by others to decrease attrition in palliative care clinical trials (Demiris et al., 2013; Fischer et al., 2012; Hickman et al., 2012). The design of this instrument study meant patient participant burden was reduced and only a short patient data collection period was required, which are also strategies reported to reduce missing data rates (Bullen et al., 2014; Hui et al., 2013). Missing data from the VSNs was potentially a bigger issue since data were collected longitudinally for the duration of the study. The team’s strategies of monitoring data completion rates and providing reminders and feedback were successful in decreasing VSN missing data, and were noted to increase completion in survey research (Dillman, Smyth, & Christian, 2014) and longitudinal cohort palliative care research (Teague et al., 2018).

4.6. Design and implementation of training

Training has not been well described in other pain instrument studies in non-communicative palliative care patients (McGuire, Kaiser, Haisfield-Wolfe, & Iyamu, 2016). A well-developed, comprehensive training program for data collectors is imperative in instrument development studies using behavioral based tools since there is a potential for unintended discrepancies when nurses rate somewhat subjective pain behaviors (even if they are considered “objective”) (Waltz, Strickland, & Lenz, 2017). Based on the outcomes of this study the research team used standardized patients to develop video clips that demonstrate the entire range of each behavior and new vignettes to improve reliability analyses in subsequent research. The new video based training has been used in subsequent research and consists of: a) a brief overview of the importance of pain assessment in non-communicative patients and the MOPAT, b) video clips of the entire range of each individual behavior on the Behavioral subscale, c) a review of the Physiologic scale, d) practice patient vignettes with an explanation of the pre-determined consensus ratings, e) optional practice vignettes, f) and a test vignette (with additional test vignettes if competency is not achieved). The research team has found the new video based training allows for independent training at any time of the day or night and refresher training for nurses who might use a measure infrequently without substantially increasing training time. This particular challenge and strategy may only be applicable to instrument research studies.

Although palliative care research challenges and solutions have been reported to be unique to the patient population or the setting (Fischer et al., 2012), many of this study’s methodological challenges and solutions were identified and used by other palliative care researchers in non-instrument studies. This research team did identify some new challenges and successfully applied some broad recommendations from the literature differently than in other palliative care studies. In addition, the research team also effectively used strategies used from non-palliative care research and applied multiple strategies to address complex challenges.

5. Implications

This report has implications beyond palliative care pain researchers. Other researchers who study non-communicative patients are likely to encounter similar issues when performing much-needed symptom assessment research. Internal Review Boards (Abernathy et al., 2014) and funding agencies need to be sensitive to and support the extra recruiting efforts (e.g., time and personnel) required to conduct research in this vulnerable population. Activities that are instrumental to the conduct of rigorous palliative care instrument development research in non-communicative patients are also applicable to the clinical setting. For example, appropriate eligibility criteria (e.g., screening for the ability to self-report before using a behavioral based pain tool), operational definitions and study procedures consistent with clinical practice, as well as comprehensive training in use of tools, and imperative to both research and clinical care (Bouca-Machado et al., 2017; Hussain et al., 2016; Hussain et al., 2017).

6. Future research

Palliative care researchers are encouraged to formally disseminate how their teams identified challenges and developed solutions to achieve positive outcomes. This is particularly important for developers of instruments to be used with non-communicative subjects, as the literature is lacking. Future research could also examine the most efficient and effective strategies for any given methodological challenge, so that scarce resources are used appropriately.

7. Conclusion

Palliative care research often employs tools that have not been validated in palliative care settings (Aoun & Nekolaichuk, 2014; Evans, Bernalia, et al., 2013). Since adequate pain management is a measure of palliative care effectiveness, pain measurement tools validated in that population should be used in palliative care studies (Agar & Luckett, 2012). Well-designed and rigorously implemented studies are essential to address the paucity of tools for assessing and managing pain in non-communicative hospitalized palliative care patients (Azevedo-Santos, et al., 2018; McGuire, Kaiser, Haisfield-Wolfe, & Iyamu, 2016). This paper demonstrates that it is possible to conduct scientifically rigorous instrument development research in a non-communicative palliative care population by closely monitoring the various components of the research process then identifying and addressing methodological challenges in a timely manner using a systematic collaborative research team approach. The methods described in this report may help other researchers studying this vulnerable population, thereby contributing to well-designed, rigorously implemented, and ultimately clinically useful studies in the critically important area of assessing and managing pain in those who cannot self-report.

Acknowledgements

The authors acknowledge JoAnne Reifsnyder, PhD, RN, for her input into the design and conduct of the study, Karen Soeken, PhD for her input into the design and conduct of the study as well as statistical expertise, as well as the study nurses who participated in this project.

Role of the funding source

The study sponsor did not participate in study design; in collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.

Funding source and declaration of interest

This work was supported by a grant from the National Institute of Nursing Research, National Institutes of Health, Bethesda, MD, USA (5R01NR0009684, Deborah B. McGuire, Principal Investigator).

Abbreviations

CUQ

Clinical Utility Questionnaire

EAPC

European Association of Palliative Care

IASP

International Association for the Study of Pain

IPAL-ICU

Improving Palliative Care in the ICU

LAR

legally authorized representative

MOPAT

Multidimensional Objective Pain Assessment Tool

NDF

Nurse Data Form

PPSv2

Palliative Performance Scale version 2

PDF

Patient Data Form

RASS

Richmond Agitation-Sedation Scale

SN

staff nurse

VSN

volunteer staff nurse

References

  1. Abernathy AP, Capell WH, Aziz NM, Ritchie C, Prince-Paul M, Bennett RE, & Kutner JS (2014). Ethical conduct of palliative care research: Enhancing communication between investigators and institutional review boards. Journal of Pain and Symptom Management, 48(6), 1211–1221. 10.1016/j.jpainsymman.2014.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Agar M, & Luckett T (2012). Outcome measures for palliative care research. Current Opinions in Supportive Palliative Care, 6(4), 500–507. 10.1097/SPC.0b013e32835a66ca. [DOI] [PubMed] [Google Scholar]
  3. Aoun SM, & Nekolaichuk CJ (2014). Improving the evidence base in palliative care to inform practice and policy: Thinking outside the box. Journal of Pain and Symptom Management, 48(6), 1222–1235. 10.1016/j.jpainsymman.2014.01.007. [DOI] [PubMed] [Google Scholar]
  4. McGuire DB, Kaiser KS, Soeken K, & Reifsnyder J (2011a). Measuring pain in non-communicative patients in the acute care setting: Psychometric evaluation of the Multidimensional Objective Pain Assessment Tool (MOPAT). Journal of Pain and Symptom Management, 41, 299–300. 10.1016/j.jpainsymman.2010.10.224. [DOI] [Google Scholar]
  5. McGuire DB, Reifsnyder J, Soeken K, Kaiser KS, & Yeager KA (2011b). Assessing pain in non-]responsive hospice patients: Development and preliminary testing of the Multidimensional Objective Pain Assessment Tool (MOPAT). Journal of Palliative Medicine, 14(3), 287–292. 10.1089/jpm.2010.0302. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. McGuire DB, Harrold J, Kaiser KS, & Bortle D (2013). Measuring pain in non-communicative patients in the inpatient hospice setting: Psychometric evaluation of the Multidimensional Objective Pain Assessment Tool (MOPAT). Journal of Pain and Symptom Management, 45(2), 403–404. 10.1016/j.jpainsymman.2012.10.261. Abstract retrieved from https://www.jpsmjournal.com. [DOI] [Google Scholar]
  7. McGuire DB, Kaiser KS, Haisfield-Wolfe ME, & Iyamu F (2016). Pain assessment in non-communicative adult palliative care patients. Nursing Clinics of North America, 51(3), 397–431. 10.1016/j.cnur.2016.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Azevedo-Santos IF, & DeSantana JM (2018). Pain measurement techniques: Spotlight on mechanically ventilated patients. Journal of Pain Research, 11, 2969–2980. 10.2147/JPR.S151169 (eCollection 2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Barr J, Fraser GL, Puntillo K, Ely EW, Gelinas C, Dasta JF, … Jaeschke R (2013). Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Critical Care Medicine, 41(1), 263–306. 10.1097/CCM.0b013e3182783b72. [DOI] [PubMed] [Google Scholar]
  10. Barzanji A, Zarelyan A, Nezamzadeh M, & Marzhari MS (2019). Evaluation of observational and behavioural pain assessment tools in nonverbal intubated critically adult patients after open - heart surgery: A systematic review. Open Access Marcedonian Journal of Medical Sciences, 7(3), 446–457. 10.3889/oamjms.2019.10. from https://www.id-press.eu/mjms/article/view/oamjms.2019.103/2943, Accessed date: 5 May 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Boland J, Currow CC, Wilcock A, Tieman J, Hussain JA, Pitsillides C, … Johnson JM (2015). A systematic review of strategies used to increase recruitment of people with cancer or organ failure into clinical trials: Implications for palliative care research. Journal of Pain and Symptom Management, 49(4), 762–772e5. [DOI] [PubMed] [Google Scholar]
  12. Bouca-Machado R, Rosário M, Alarcão J, Correia-Guedes L, Abreu D, & Ferreira JJ (2017). Clinical trials in palliative care: A systematic review of their methodological characteristics and of the quality of their reporting. BMC Palliative Care, 16(1), 10 10.1186/s12904-016-0181-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bullen T, Maher KM, Rosenberg JR, & Smith B (2014). Establishing research in a palliative care clinical setting: Perceived barriers and implemented strategies. Applied Nursing Research, 27(1), 78–83. [DOI] [PubMed] [Google Scholar]
  14. Burns KE, Zubrinich C, Tan W, Raptis S, Xiong W, Smith O, … Cook DJ (2013). Research recruitment practices and critically ill patients. A multicenter, cross-sectional study (the Consent Study). American Journal of Respiratory and Critical Care Medicine, 187(11), 1212–1218. 10.1164/rccm.201208-1537OC. [DOI] [PubMed] [Google Scholar]
  15. Coyne P, Mulvenon C, & Paice JA (2018). American Society for Pain Management Nursing and Hospice and Palliative Nurses Association position statement: Pain management at the end of life. Pain Management Nursing, 19(1), 3–7. 10.1016/j.pmn.2017.10.019. [DOI] [PubMed] [Google Scholar]
  16. Demiris G, Oliver D, Capurro D, & Wittenberg-Lyles E (2013). Implementation science: Implications for intervention research in hospice and palliative care. The Gerontologist, 54(2), 163–171. 10.1093/geront/gnt022 (Epub 2013 Apr 4). [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dillman DA, Smyth JD, & Christian LM (2014). Internet, phone, mail, and mixed-mode surveys the tailored design method. New Jersey: John Wiley& Sons. [Google Scholar]
  18. Ersek M, Polissar N, Du Pen A, Herr K, & Neradilek MB (2012). Addressing methodological challenges in implementing the nursing home pain management algorithm randomized controlled trial. Clinical Trials, 9(5), 634–644. 10.1177/1740774512454243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Evans CJ, Benalia H, Preston NJ, Grande G, Gysels M, Short V, … MORECare (2013). The selection and use of outcome measures in palliative and end-of-life care research: The MORECare International Consensus Workshop. Journal of Pain and Symptom Management, 46(6), 925–937. 10.1016/j.jpainsymman.2013.01.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Evans CJ, Harding R, Higginson IJ, & MORECare (2013). ‘Best practice’ in developing and evaluating palliative and end-of-life care services: A meta-synthesis of research methods for the MORECare project. Journal of Palliative Medicine, 27(10), 885–898. 10.1177/0269216312467489. [DOI] [PubMed] [Google Scholar]
  21. Fischer DJ, Burgener SC, Kavanaugh K, Ryan C, & Keenan G (2012). Conducting research with end-of-life populations: Overcoming recruitment challenges when working with clinical agencies. Applied Nursing Research, 25(4), 258–263. 10.1016/j.apnr.2011.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Foley K (2005). Acute and chronic cancer pain syndromes In Doyle D, Hanks G, Cherney NI, & Calman K (Eds.). Oxford textbook of palliative medicine (pp. 298–316). (3rd ed.). New York, NY: Oxford University Press. [Google Scholar]
  23. Hanratty B, Lowson E, Holmes L, Addington-Hall J, Arthur A, Grande G, … Seymour J (2012). A comparison of strategies to recruit older patients and carers to end of life research in primary care. BMC Health Services Research, 12, 342 10.1186/1472-6963-12-342. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hanson LC, Bull J, Wessell K, Massie L, Bennett RE, Kutner JS, … Abernethy A (2014). Strategies to support recruitment of patients with life-limiting illness for research: The Palliative Care Research Cooperative Group. Journal of Pain and Symptom Management, 48(6), 1021–1130. 10.1016/j.jpainsymman.2014.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Herr K, Coyne PJ, McCaffery M, Manworren R, & Merkel S (2011). Pain assessment in the nonverbal patient: Position statement with clinical practice recommendations. Pain Management Nursing, 12(4), 230–250. 10.1016/j.pmn.2011.10.002. [DOI] [PubMed] [Google Scholar]
  26. Hickman SE, Cartwright JC, Nelson CA, & Knafl K (2012). Compassion and vigilance: Investigators’ strategies to manage ethical concerns in palliative and end-of-life research. Journal of Palliative Medicine, 15(8), 880–889. 10.1089/jpm.2011.0515. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hui D, Glitza I, Chisholm G, Yennu S, & Bruera E (2013). Attrition rates, reasons, and predictive factors in supportive care and palliative oncology clinical trials. Cancer, 119(5), 1098–1105. 10.1002/cncr.27854. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hussain JA, Bland M, Langan D, Johnson MJ, Currow DC, & White IR (2017). Quality of missing data reporting and handling in palliative care trials demonstrates that further development of the CONSORT statement is required: A systematic review. Journal of Clinical Epidemiology, 88, 81–91. 10.1016/j.jclinepi.2017.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hussain JA, White IR, Langan D, Johnson MJ, Currow DC, Torgerson DJ, … Bland M (2016). Missing data in randomized controlled trials testing palliative interventions pose a significant risk of bias and loss of power: A systematic review and meta-analyses. Journal of Clinical Epidemiology, 74, 57–65. 10.1016/j.jclinepi.2015.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. International Association for the Study of Pain (IASP) Acute Pain Task Force. (1992). Management of acute pain: A practical guide Vol. 1 In Ready LB L. B., & Edwards WT (Eds.), Seattle, Washington: IASP Publications. [Google Scholar]
  31. Kars MC, van Thiel GJ, van der Graaf R, Moors M, de Graeff A, & van Delden JJ (2016). A systematic review of reasons for gatekeeping in palliative care research. Palliative Medicine, 30(6), 533–548. 10.1177/0269216315616759. [DOI] [PubMed] [Google Scholar]
  32. LeBlanc TW, Lodato JE, Currow DC, & Abernethy AP (2013). Overcoming recruitment challenges in palliative care clinical trials. Journal of Oncology Practice, 9, 277–282. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Mercadante S, Radbruch L, Caraceni A, Cherny N, Kaasa S, Nauck F, … The Steering Committee of the European Association for Palliative Care (EAPC) Research Network (2002). Episodic (breakthrough) pain: Consensus conference of an expert working group of the European Association for Palliative Care. Cancer, 94(3), 832–839. 10.1002/cncr.10249. [DOI] [PubMed] [Google Scholar]
  34. National Consensus Project for Quality Palliative Care. (2018). Clinical practice guidelines for quality palliative care, 4th Ed. Richmond, VA: National Coalition for Hospice and Palliative Care; https://www.nationalcoalitionhpc.org/ncp (web archive link, 10 February 2019). Accessed 10 February 2019. [Google Scholar]
  35. Palliative Performance Scale (PPSv2) version 2 (2006). Medical care of the dying (4th ed.). Copyright Victoria Hospice Society; http://www.victoriahospice.org/sites/default/files/pps_english.pdf, Accessed date: 2 May 2016. [Google Scholar]
  36. Polito CC, Cribbs SK, Martin GS, O’Keeffe T, Herr D, Rice TW, & Sevransky JE (2014). Navigating the institutional review board approval process in a multi-center observational critical care study. Critical Care Medicine, 42(5), 1105–1109. 10.1097/CCM.0000000000000133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Puntillo K, Nelson JE, Weissman D, Curtis R, Weiss S, Frontera J, … Mulkerin C (2014). Palliative care in the ICU: Relief of pain, dyspnea, and thirst—A report from the IPAL-ICU Advisory Board. Intensive Care Medicine, 40(2), 235–248. 10.1007/s00134-013-3153-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Puntillo KA, Max A, Timsit JF, Vignoud L, Chanques G, Robleda G, … Azoulay E (2014). Determinants of procedural pain intensity in the intensive care unit. The Europain® study. American Journal of Respiratory and Critical Care Medicine, 189(1), 39–47. 10.1164/rccm.201306-1174OC. [DOI] [PubMed] [Google Scholar]
  39. Rebers S, Aaronson NK, van Leeuwen FE, & Schmidt MK (2016). Exceptions to the rule of informed consent for research with an intervention. BMC Medical Ethics, 17, 9 10.1186/s12910-016-0092-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Regulation EU No 536/2014, 2014 (2014). Official Journal of European Union. 27.5/2014: L 158/1-L158/26. https://ec.europa.eu/health/sites/health/files/files/eudralex/vol-1/reg_2014_536/reg_2014_536_en.pdf.
  41. Rincon F, & Lee K (2015). Ethical considerations in consenting critically ill patients for bedside clinical care and research. Journal of Intensive Care Medicine, 30(3), 141–150. 10.1177/0885066613503279. [DOI] [PubMed] [Google Scholar]
  42. Sessler CN, Gosnell M, Grap MJ, Brophy GM, O’Neal PV, Keane KA, … Elswick RK (2002). The Richmond Agitation-Sedation Scale: Validity and reliability in adult intensive care patients. American Journal of Respiratory and Critical Care Medicine, 166(10), 1138–1344. 10.1164/rccm.2107138. [DOI] [PubMed] [Google Scholar]
  43. Snowden A, & Young J (2017). A screening tool for predicting gatekeeping behaviour.Nursing Open, 4(4), 187–199. 10.1002/nop2.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Teague S, Youssef GJ, Macdonald JA, Sciberras E, Shatte A, Fuller-Tyszkiewicz M, … Hutchinson D (2018). Retention strategies in longitudinal cohort studies: A systematic review and meta-analysis. BMC Medical Research Methodology, 18(1), 151 10.1186/s12874-018-0586-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Waltz CF, Strickland OL, & Lenz ER (2017). Measurement in nursing research (5th ed.). Philadelphia: F.A: Davis. [Google Scholar]
  46. World Health Organization (WHO). (2010). WHO definition of palliative care. http://www.who.int/cancer/palliative/definition/en/ Accessed 24 September 2010.

RESOURCES