Developing a Pain Intensity Measure for Persons with Dementia: Initial Construction and Testing

Abstract

Objective.

The goal of this study was to identify a limited set of pain indicators that were most predicive of physical pain. We began with 140 items culled from existing pain observation tools and used a modified Delphi approach followed by statistical analyses to reduce the item pool.

Methods.

Through the Delphi Method, we created a candidate item set of behavioral indicators. Next, trained staff observed nursing home residents and rated the items on scales of behavior intensity and frequency. We evaluated associations among the items and expert clinicians’ assessment of pain intensity.

Setting.

Four government-owned nursing homes and 12 community nursing homes in Alabama and Southeastern Pennsylvania.

Participants.

Ninety-five residents (mean age = 84.9 years) with moderate to severe cognitive impairment.

Results.

Using the least absolute shrinkage and selection operator model, we identified seven items that best predicted clinicians’ evaluations of pain intensity. These items were rigid/stiff body or body parts, bracing, complaining, expressive eyes, grimacing, frowning, and sighing. We also found that a model based on ratings of frequency of behaviors did not have better predictive ability than a model based on ratings of intensity of behaviors.

Conclusions.

We used two complementary approaches—expert opinion and statistical analysis—to reduce a large pool of behavioral indicators to a parsimonious set of items to predict pain intensity in persons with dementia. Future studies are needed to examine the psychometric properties of this scale, which is called the Pain Intensity Measure for Persons with Dementia.

Introduction

Worldwide, an estimated 46.8 million people were living with dementia in 2015. This number is projected to grow to more than 131.5 million by 2050 [1]. People with dementia frequently have coexisting medical problems such as cancer, diabetes, and osteoarthritis that result in pain. Estimates of pain prevalence in persons with dementia vary, but recent estimates place the proportion at about 50% [2,3]. Studies also show that many people with cognitive impairment, including those with dementia, are at high risk for underidentification and undertreatment of pain [4].

The human and economic costs of our failure to identify and treat pain in people with dementia are staggering. Untreated pain is associated with depression, impaired social functioning, sleep disturbances, physical disability, and agitation, as well as other behavioral problems [5]. Behavioral disturbances, which may be caused by unrelieved pain, significantly increase caregiver and nursing care burden [6], as well as health care costs [7]. Pain-related behaviors may be treated inappropriately with antipsychotic medications, restraints, and other therapies that have serious adverse effects. Thus, unrelieved pain causes avoidable suffering, increased caregiver burden, and enormous health care and long-term care costs, which will only increase with the anticipated burgeoning of the aging population.

By definition, pain is an experience, and thus the gold standard for measuring it is self-report. However, many persons with dementia are unable to provide consistently reliable pain reports. Without self-report, clinicians must rely on other pain indicators, which include observable behaviors that reflect a complex internal state such as pain [8]. Many observational tools have been developed, and most have undergone at least preliminary psychometric testing. Some tools screen for the presence of pain, while others are designed to yield an assessment of pain intensity [9]. Despite the plethora of tools, however, it is unclear which pain behaviors are most effective in predicting pain intensity [9,10]. Furthermore, Corbett et al. argued that “there remains an urgent need to draw on the currently available resources and to develop an easy to use assessment tool which has utility in both research and clinical settings and robust validation data to support its implementation” [10].

Our objective was to address the need for an accurate, clinically useful pain intensity measure by identifying a parsimonious set of items from existing measures that best predict pain intensity in people with dementia. First, we conducted a modified Delphi study to identify a range of potential items that would be specific to physical pain experience in this population. Second, we reduced the candidate item by identifying a subset of items that were most predictive of an expert clinician’s evaluation of pain intensity. We called the reduced item set the Pain Intensity Measure for Persons with Dementia (PIMD).

Methods

Phase 1: Construction of the Candidate Items for the PIMD Using a Modified Delphi Technique

We used a modified Delphi technique to obtain consensus on the items to be included in the first-phase version of the PIMD [11]. In this iterative method, a researcher initiates a survey on the topic of interest with specific queries, and a panel of experts provides their responses. These responses are summarized, and feedback is provided to individual members so they can consider their answers in the context of other expert panel members’ responses. The panel members reevaluate their judgments and are allowed to change their responses in light of the feedback. The process continues until a predetermined level of consensus is achieved [12]. This approach often includes structured discussions of the ratings to provide insight and clarification of members’ ratings and guide the formulation of the final recommendations [11].

Delphi Panel Members

We recruited five clinicians who were not study investigators to serve on the Delphi panel. We used purposive sampling to identify authorities in the field with at least five years of clinical and/or research experience in geriatric psychiatry, dementia care, and/or geriatric pain. The panel included one board-certified geropsychiatrist and palliative care physician with research experience, three geriatric advance practice nurses, including one certified in palliative care, and one clinical geropsychologist. As a thank you, members of the Delphi panel received a $50 electronic gift card for completing all phases of the survey.

Procedure

A set of 140 candidate items was harvested from 12 published observational pain tools that had undergone at least some psychometric evaluation. Only tools published and validated before 2013 (the year we conducted the Delphi panel) were included [13–20]. For this phase, we included exact items, words, and phrases used in published measures and avoided collapsing similar concepts together. For example, a particular behavior was called “elopement” in one instrument and “exiting, trying to leave” in another instrument; we therefore included both in the candidate item set. We then developed a survey in which items were presented in alphabetical order. Items were not grouped by content categories (e.g., vocalizations, facial expressions) to avoid biasing panel members’ perceptions of items. The candidate item set was comprised of 140 items (Table A1).

In round one of the Delphi process, we sent a survey to panel members asking them to consider a specific clinical scenario and rate each behavior on the list as not useful at all, somewhat useful, or very useful in evaluating pain intensity (Supplementary Data). Before sending the survey to the Delphi panel, we pilot-tested the survey with one geriatric medicine physician and one pain clinician who were not otherwise involved in the study. They evaluated the clarity, precision, and efficiency of the instructions and items; we refined the survey based on their feedback [12].

Values of 1, 2, and 3 were assigned to responses of not useful at all, somewhat useful, or very useful, respectively, resulting in a range of 5 to 15, where higher scores indicate more useful items. Delphi panel members completed their evaluations independently without input from other members or the research team. A trained research coordinator compiled the scores after rankings were completed.

In round two, panel members received a listing of all the original items, along with initial ratings across panel members. The ratings were categorized as highly endorsed (≥14), moderately endorsed (10–13), or low endorsement (≤9). Panel members reviewed the list and rescored items based on the results of round one. As with the initial survey, items were presented alphabetically. In addition to re-rating the items, we asked panel members to group the remaining items into categories that made sense to them. We did this to better understand the different heuristic approaches that clinicians used in observing for pain; in addition, we wanted to establish a categorization scheme for the remaining items using consensus rather than predetermined groupings. Finally, we asked panel members to identify items that they found redundant or thought could be combined or reworded to reduce further the total number of candidate items.

After compiling the results from round two, Delphi panel members received a summary of the responses including a list of candidate items, the categorization schemes for each Delphi panel member, and a list of potentially redundant items. The panel members then participated in a two-hour phone conference call with investigators that was facilitated by a member of the scientific team (KC) who is a psychometrician with extensive experience developing and testing pain measures [21–23]. The investigator team developed a list of topics to guide the discussion (Supplementary Data). The entire phone call was transcribed and reviewed by a research coordinator (FN) and the lead author (ME). Following the conference call with Delphi panel members, the study investigators (ME, KH, KC, ALS, NP, MN) met by conference call to confer and adjudicate the panel’s recommendations.

Phase 1 Results (Delphi Panel)

In the initial ranking exercise, 59 of the collected 140 items had ratings of 14 or 15. Collectively, these items encompassed all the American Geriatrics Society–recommended categories of behaviors, as well as behaviors such as rapid breathing and inconsolability that did not easily fit into these categories. Following round two, 53 items had overall ratings of 14 or 15. These 53 items were discussed during the conference call with the study investigators.

The results from the categorization exercise yielded wide variation in the ways in which the Delphi panel members thought about these behaviors. Although some of the inductively derived categories matched groupings in existing instruments (e.g., facial expressions, vocalizations), others reflected members’ efforts to discern which behaviors were specific/nonspecific to pain. One Delphi panel member also attempted to differentiate between behavioral, physiologic, and psychological components. These differences prompted an in-depth discussion in which Delphi panel members decided that using “specificity for pain” was not a useful categorization, because panel members had already highly rated the items they believed were specific to pain. Furthermore, grouping items as behavioral/physiologic/psychological pain indicators was not useful because the psychological or emotional component of pain could only be inferred in people with dementia by examining behavioral indicators. By the end of the discussion, the Delphi panel members and study investigators reached consensus on eight categories of behavior: agitation, physically resisting care or movement, movement and protection, vocalizations, facial expressions, autonomic nervous system activation, consolability, and changes in usual patterns of behavior.

During the Delphi panel phone conference with study investigators, we also reviewed each panel member’s list of potentially redundant items that remained in the 53-item set. The panel reached consensus about combining several items to decrease redundancy. For example, the panel members agreed that two items, “move slowly” and “move stiffly” could be combined into a single item. The discussion also resulted in debate among the panel members regarding two items that focused on eye movements/expressions. The item “opening eyes wide” was part of the final 53-item set, but “narrowed or closed eyes” was dropped after the Delphi round one. Two members of the panel felt strongly that narrowed or closed eyes was a strong pain indicator, including forcibly shutting the eyes. After discussion, the Delphi panel agreed to add the latter item back into the final item set (adding “squeezed shut”) and decided that including two items describing eye movements/expressions was unnecessary. They suggested the term “expressive eyes,” which incorporated the following descriptors: eyes are open wide or bulging, eyebrows are lifted high, eyes are narrowed or squeezed shut. They also agreed that this item was different from those for other facial expressions, such as grimacing and frowning, which also are critical in evaluating pain [24–27].

At the completion of all steps of the Delphi panel (see the Supplementary Data for an outline of the Delphi steps), the study investigators established a preliminary 39-item PIMD (Supplementary Data), grouped into the eight aforementioned categories. Thirty-three of the items could be scored using direct observation; six items required observation of changes in a person’s usual patterns of behavior over a period of a week. Items with clarifying descriptions are listed in the Supplementary Data. Each item was rated with respect to the intensity of the behavior (absent, mild, moderate, severe; scored 0, 1, 2, or 3, respectively) and its frequency (never, occasionally, often, and constantly; scored 0, 1, 2, or 3, respectively) as assessed over the five- to 10-minute observation period.

Phase 2: Item Reduction

Sample

In the second phase, PIMD candidate items were used to rate the pain of a sample of 95 nursing home residents who resided at four government–owned and operated nursing homes and 12 community nursing homes in Alabama and southeastern Pennsylvania. We reviewed unit rosters with nurse managers to identify all long-term care residents of the participating nursing homes who were age 50 years or older with a documented diagnosis of dementia and moderate to severe cognitive impairment, defined as a score of <10 on the Brief Inventory of Mental Status (BIMS) [28,29]. Additionally, we enrolled residents only if they had moderate to severe pain, as judged by licensed nursing staff who were familiar with the resident. We attempted to contact the legally authorized surrogates for all residents who met the above criteria and sought written proxy consent to participate from all of them.

We obtained written proxy consent from the legally authorized representative or next of kin for all participants. We also inferred assent from residents if they did not object (either verbally or through behaviors such as increased combativeness) to any part of the expert clinician evaluation or the presence of the research coordinators while they completed research instruments that required observation. There were no instances in which research activities were halted as a result of residents’ responses to research staff. All research procedures were approved by the Central IRB of the Department of Veterans Affairs.

Measurements and Procedures

For the above-described residents, we only used the 33 items that could be evaluated by observers who had no previous interaction with the person with dementia. We chose this approach because we wanted the final item set to be useful in a variety of settings, including acute care, where caregivers are unlikely to have had previous interaction or be familiar with patients.

PIMD items were administered by trained research coordinators from two sites (Tuscaloosa, AL, USA, and Philadelphia, PA, USA). For training purposes, two study investigators (ALS, ME) and all research coordinators viewed 10 video clips (60–90 seconds each) of nursing home residents with dementia and completed the PIMD. The videos were produced as part of an earlier pain observation study conducted by one of the investigators (ALS). Videotapes covered a range of activities, from lying quietly in bed to transferring, ambulating, and turning in bed, as well as engaging in personal care (e.g., combing their hair or having it combed for them, brushing their teeth, taking a shower or receiving a bed bath). Video clips were selected to represent the full range of types of behaviors (e.g., guarding, bracing, screaming) and a range of behavior intensities. The two investigators and the research coordinators then met by videoconference or in person to review discrepancies in ratings, refine and clarify the definitions and descriptions of each item, and reach consensus on the rating for each item for each clip. Following the training, each research coordinator independently rated 24 additional video clips and met the criterion of 90% or greater agreement with the study investigators’ ratings on each item. One refresher training, after approximately six months of data collection, was also required of all research coordinators.

Because reliable self-report is not available for many persons with advanced dementia, we used an expert clinician pain intensity rating (ECPIR) as the gold standard against which to evaluate ratings. To ensure the quality of the ECPIR, a clinical evaluation protocol was developed by a panel of geriatric/pain clinicians and investigators from multiple disciplines (i.e., psychology, nursing, medicine). The ECPIR was completed either by the first author and investigator (ME, a doctorally prepared nurse with extensive experience in pain assessment and measurement in persons with dementia) or by one of four geriatric nurse practitioners, each of whom had a minimum of five years’ clinical experience working with people who have dementia. The first author (ME) trained all nurse practitioners in the evaluation protocol. The protocol involved a comprehensive pain assessment consisting of the following components: 1) a medical record review to identify pain-related diagnoses, pain evaluations and assessments, and pharmacologic and nonpharmacologic pain therapies that were ordered and administered; 2) a targeted physical examination; 3) consultation with available licensed nurses, nursing assistants, families, physical therapists, occupational therapists, primary care providers, and others to obtain surrogate reports of pain; 4) resident interview including self-report of pain, if possible; and 5) observation during rest and activity. Following data collection from these five components, the expert clinician completed a form that included assessment of average, worst, least, and current pain using a 1–10 Likert scale where 0 was “no pain” and 10 was “the worst pain imaginable” [30].

The expert clinician also evaluated pain intensity concurrently with a trained research coordinator who completed the PIMD, following five- to 10-minute periods of in-person observation while the rated resident was at rest and during physical movement (two separate evaluations), using the 1–10 Likert scale. We used a concurrent ECPIR “at movement” score (per resident) as the gold standard to reduce the PIMD to items that were predictive of the gold standard and to develop an overall PIMD scale. We chose an at movement score because the ECPIR and PIMD “at rest” ratings showed little variability both within and across subjects; for example, only three of the 33 PIMD items had ≥15% nonzero values when the PIMD was completed while the resident was sitting or lying still and not engaged in any activity. At movement activities varied across observations and included one of the following categories: moving in bed, transferring with or without assistance, ambulating, or participating in activities that required movement of the upper or lower extremities while seated. Thus, our analyses included observing activities that involved a wide range of gross and fine motor movements and varying levels of exertion, which captured low to high intensities of movement.

Statistical Analysis

The goal of the statistical analysis was to identify PIMD items that were predictive of the ECPIR and to assist in the development of an overall PIMD score. Initially, means, standard deviations, ranges, and percentages of missing and not applicable (N/A) responses were calculated for each PIMD item (see the Supplementary Data for items with N/A options). Spearman correlations were used to quantify the strength of associations among PIMD items and of associations of individual PIMD item scores with the ECPIR (in a univariate analysis). Principal components analysis was used to evaluate the ability to reduce all PIMD items into fewer (derived) items.

To derive the final set of items for the PIMD, we used the least absolute shrinkage and selection operator (LASSO) model [31], a modern variable selection technique that addresses overfitting by introducing a penalty on regression coefficients. The predictive ability of the LASSO model was evaluated using residual standard deviation and the R² statistic, both calculated using 10-fold cross-validation repeated 50 times. Cross-validation takes into account the potential overfitting of the models to this specific sample of 95 subjects. Model overfitting can occur in these various analyses due to the limited sample size and the numerous potential models that can be created from the available PIMD items. All calculations were carried out in R, version 3.3.1 (Vienna, Austria) [32].

Phase 2 Results (LASSO Analysis)

Table 1 presents descriptive statistics for the participant sample. About two-thirds of the sample were women, and 74% of the sample was white. Forty percent of the sample had a diagnosis of arthritis, and, on average, participants had one to two potentially painful diagnoses (mean = 1.7, range = 0–6).

Table 1.

Participant characteristics (N = 95)

Age, mean (SD), y	84.9 (9.9)
	N	%
Female	64	67.4
Education
High school or less	34	35.8
Postsecondary education	28	29.5
Missing	33	34.7
Race
White	70	73.7
Black or African American	25	26.3
Other	0	0
Missing	0	0
Ethnicity
Hispanic	2	2.1
Non-Hispanic	93	97.9
Missing	0	0
Potentially painful conditions
Arthritis	38	40.0
Miscellaneous pain (muscle spasms, diagnosis of chronic or generalized pain not otherwise accounted for)	31	32.6
Constipation	24	25.3
Peripheral vascular disease	17	17.9
Pressure ulcer or painful skin condition	13	13.7
Fracture or contracture or osteomyelitis	12	12.6
Neuropathic pain: diabetic neuropathy, sciatica, phantom limb pain, postherpetic neuralgia, spinal cord compression	4	4.2
Cancer	4	4.2
Lower GI pain: irritable bowel syndrome, hemorrhoids, Crohn’s disease	3	3.2
Gout	2	2.1
Headache	2	2.1
Total No. of painful conditions, mean (SD)	1.7 (1.6)
Range	0–6
Expert Clinician Pain Intensity Rating, mean (SD)	3.2 (2.7)
Range	0–10

GI = gastrointestinal.

Table 2 presents descriptive statistics for the individual items. The mean intensity score for items ranged from a low of 0.02 for hitting/scratching/biting, sweaty/clammy skin, and rapid breathing to a high of 1.43 for rigid or stiff body or body part. The mean frequency ranged from 0.01 for rapid breathing to 1.89 for rigid/stiff body or body part. Several items had very few nonzero values; rapid breathing and sweating/clammy skin had one nonzero subject each, and hitting and scratching had two nonzero subjects. Two items, limping and pacing, could not be evaluated in many participants because the participants were not observed while walking or were not mobile. Only 41 and 52 cases (out of 95) could be scored on the limping and pacing items, respectively.

Table 2.

Descriptive statistics for PIMD items^*

			Intensity†	Frequency‡
PIMD Item	N§	N > 0	Mean ± SD (Range)	Mean ± SD (Range)
Agitated	95	19	0.36±0.78 (0–3)	0.35±0.77 (0–3)
Restlessness	95	13	0.22±0.60 (0–3)	0.29±0.80 (0–3)
Hitting, scratching, biting	95	2	0.02±0.14 (0–1)	0.03±0.23 (0–2)
Irritable, cranky	95	9	0.15±0.50 (0–3)	0.14±0.48 (0–3)
Thrashing	95	2	0.03±0.23 (0–2)	0.04±0.29 (0–2)
Pacing	52	6	0.21±0.67 (0–3)	0.21±0.67 (0–3)
Resisting movement	95	6	0.08±0.35 (0–2)	0.08±0.35 (0–2)
Pulling away	95	5	0.08±0.40 (0–3)	0.06±0.28 (0–2)
Resisting care	95	5	0.07±0.33 (0–2)	0.07±0.33 (0–2)
Rubbing or massaging	95	10	0.16±0.49 (0–2)	0.20±0.61 (0–3)
Guarding	95	7	0.11±0.40 (0–2)	0.12±0.46 (0–3)
Bracing	94	16	0.33±0.77 (0–3)	0.36±0.87 (0–3)
Limping	41	7	0.27±0.67 (0–3)	0.32±0.79 (0–3)
Rigid/stiff	95	72	1.43±1.02 (0–3)	1.89±1.26 (0–3)
Fetal position	95	4	0.08±0.40 (0–2)	0.11±0.52 (0–3)
Move slowly or stiffly	95	66	1.08±0.91 (0–3)	1.35±1.14 (0–3)
Groaning/moaning/grunting	95	22	0.34±0.68 (0–3)	0.32±0.66 (0–3)
Sighing	95	10	0.16±0.49 (0–2)	0.12±0.35 (0–2)
Repeated calling out	95	4	0.06±0.32 (0–2)	0.06±0.32 (0–2)
Protesting	95	19	0.37±0.81 (0–3)	0.29±0.67 (0–3)
Whining	95	4	0.08±0.40 (0–2)	0.06±0.32 (0–2)
Complaining	95	34	0.52±0.78 (0–3)	0.51±0.78 (0–3)
Screaming	94	4	0.07±0.37 (0–2)	0.06±0.32 (0–2)
Rapid breathing	95	1	0.02±0.21 (0–2)	0.01±0.10 (0–1)
Labored breathing	95	7	0.12±0.43 (0–2)	0.11±0.40 (0–2)
Sweating, clammy	95	1	0.02±0.21 (0–2)	0.02±0.21 (0–2)
Flinching	95	17	0.31±0.72 (0–3)	0.22±0.51 (0–2)
Grimacing	95	22	0.45±0.87 (0–3)	0.34±0.71 (0–3)
Frowning	95	15	0.31±0.76 (0–3)	0.25±0.65 (0–3)
Clenched jaw or teeth	95	3	0.03±0.18 (0–1)	0.03±0.18 (0–1)
Crying, tearfulness	95	4	0.06±0.32 (0–2)	0.07±0.36 (0–2)
Expressive eyes	95	43	0.80±1.00 (0–3)	0.74±0.94 (0–3)
Consolability¶	90	51	0.67±0.65 (0–2)

PIMD = Pain Intensity Measure for Persons with Dementia.

^*Based on at movement PIMD ratings.

^†Intensity ratings: 0 = absent, 1 = mild, 2 = moderate, 3 = severe.

^‡Frequency ratings: 0 = never, 1 = occasionally, 2 = often, 3 = constantly.

^§N excludes items scored as not applicable.

^¶Item rated by intensity only: 0 = no need to console, 1 = able to be consoled or distracted by touch or words; 2 = cannot be consoled by touch or words; N/A = no need to console.

Spearman correlations between PIMD items and the experts’ evaluation of pain intensity appear in Table 3. The items most correlated with the experts’ evaluation of pain intensity were expressive eyes (r = 0.63), rigid/stiff body or body part (r = 0.50), inability to be consoled (r = 0.45), grimacing (r = 0.42), and groaning/moaning/grunting (r = 0.40).

Table 3.

Univariate Spearman correlations between PIMD items and expert clinician pain intensity rating (N = 88, intensity during movement, such as transferring and ambulating)^*

Item #	Brief Description	r	Item #	Brief Description	r	Item #	Brief Description	r
32	Expressive eyes	0.63	18	Sighing	0.25	15	Fetal position	0.08
14	Rigid/ stiff	0.50	27	Flinching	0.24	08	Pulling away	0.08
33	Consolability	0.45	31	Crying, tearful	0.23	19	Repeated calling out	0.07
28	Grimacing	0.42	01	Agitated, upset	0.20	07	Resisting movement	0.00
12	Bracing	0.41	11	Guarding	0.18	02	Restlessness	0.00
17	Groaning/moaning/ grunting	0.40	23	Screaming	0.17	09	Resisting care	−0.01
29	Frowning	0.39	21	Whining	0.15	30	Clenched jaw	−0.02
22	Complaining	0.38	16	Moving slowly or stiffly	0.14	10	Rubbing	−0.10
20	Protesting	0.28	04	Irritable, cranky	0.08

PIMD = Pain Intensity Measure for Persons with Dementia.

^*Limited to 88 subjects with complete cases for all PIMD items.

Interitem correlations among the intensity ratings (during movement) of the 33 items generally were low. The median of the absolute values of all pairwise Spearman correlations among the 33 intensity items was 0.12 (detailed results not shown). Only three pairs of items had Spearman correlation >0.50: protesting and complaining (r = 0.56), flinching and expressive eyes (r = 0.52), and expressive eyes and inconsolability (r = 0.58). This finding was further corroborated by principal components analysis, which showed that the first five components explained 65% of the variance and the first 10 principal components explained 84% of the variance in PIMD scores (results not shown). The relatively large number of principal components needed to explain a large fraction of the variance indicates that the 33 intensity items were not highly redundant, and thus collinearity did not pose a serious challenge to the variable selection.

After reviewing the item statistics, the study investigators decided to drop several items from further analyses. These included rapid breathing, sweating/clammy skin, thrashing, and hitting/scratching/biting (few nonzero values), and limping and pacing (because of the high number of “unable to assess” responses and very few nonzero values). We also eliminated labored breathing for two reasons. First, the item reflects an autonomic nervous system response that is often dampened in people with dementia [33]. Second, the research coordinators reported that they often scored this item when people were exerting themselves, such as during a transfer, and were unable to determine whether the change in breathing reflected pain or the effort of movement. Finally, we excluded the frequency ratings for the items because the mean and range of these ratings were similar to those of the intensity ratings and, as shown later, use of the frequency ratings instead of the intensity ratings did not improve prediction of the ECPIR.

Table 4 shows the multivariate LASSO model to predict ECPIR. Bracing, rigid/stiff, sighing, complaining, grimacing, frowning, and expressive eyes were selected by the LASSO model as the set most predictive of ECPIR. The LASSO model gave the most weight to expressive eyes (0.86), complaining (0.68), bracing (0.62), and rigid/stiff (0.37).

Table 4.

LASSO model to predict expert clinician pain intensity ratings using PIMD intensity items (N = 88)

Item	Coefficient
1. Bracing	0.62
2. Rigid/stiff	0.37
3. Sighing	0.01
4. Complaining	0.68
5. Grimacing	0.04
6. Frowning	0.15
7. Expressive eyes	0.86
Ability to predict ECPIR*
Residual SD	1.84
R2	0.51

ECPIR = expert clinician pain intensity ratings; LASSO = least absolute shrinkage and selection operator; PIMD = Pain Intensity Measure for Persons with Dementia.

^*The residual SD and R² statistic were calculated by 10-fold cross-validation, repeated 50 times, of the LASSO model.

We also conducted sensitivity analyses to understand the impact of adding the item labored breathing in the model-building (with the entire model rebuilt from the beginning) and, in a separate analysis, the impact of using frequency ratings instead of intensity ratings. We used the same LASSO regression model-building process as was described earlier for building the models using the intensity ratings. We found no appreciable impact of either optional strategy (i.e., using frequency ratings or adding the labored breathing item) on the predictive ability of the overall PIMD score. We found similar or worse predictive statistics (R² and residual SD) for the sensitivity analysis models compared with the final LASSO model in Table 4.

Based on the final LASSO model, the study investigators concluded that the final PIMD item set would consist of seven items. The tool is scored by summing the intensity score on each item (i.e., absent = 0, mild = 1, moderate = 2, severe = 3), yielding a possible range of 0–21, with higher scores indicating higher pain intensity.

Discussion

This study identified a parsimonious set of behavioral pain indicators predictive of pain intensity in persons with dementia. We began with a large sample of items derived from existing tools and applied a combination of expert consensus (Delphi panel) and statistical techniques to arrive at a final set of items. Our initial approach was similar to the multinational European effort described by Corbett et al. [10], which combined a rigorous literature review of all existing pain measures for persons with dementia and the Delphi panel consensus to identify a large item pool consisting of items in three domains and 12 subcategories. Our study expanded the Corbett methodology by further reducing the item pool using LASSO to identify a smaller set of items to predict clinician pain intensity ratings. As such, it went a step further than the European collaboration and may provide guidance for ongoing efforts by other investigators.

The seven items in the final item set appear in other behavioral pain measures (Table A1) but represent a parsimonious and unique combination of items. Three of the seven focus on facial expressions, which are universal components, in varying degrees, of every behavioral pain measure. There is substantial evidence that facial expressions are among the most sensitive and reliable indicators of pain in persons with dementia [26,27,34,35]. Of the three facial expressions, the item expressive eyes had the highest regression coefficients. This finding may have been the result of defining the item using a broad but specific set of descriptors to guide raters (i.e., eyes are open wide or bulging, eyebrows are lifted high, eyes are narrowed or squeezed shut). We were careful to define grimacing and frowning objectively and as precisely as possible, as recommended in the literature, thereby aligning these items with the facial action coding system [26,34,35].

We found little variability in scores on observed pain behavior items and expert clinical ratings of pain when participants were at rest (results not shown). As a result, our statistical analyses used data only from measurements taken when participants were engaged in some type of movement. Although this calls into question the value of observing pain behaviors when people are resting, Oosterman et al. [26] point out that observations of individuals during rest may provide a useful baseline from which to evaluate behaviors during movement. It is yet unclear whether it would be useful from a clinical or measurement perspective to complete a pain behaviors tool under both conditions and compare the results. On the other hand, focusing on pain-related behaviors during movement is consistent with recommended best practices [4,13].

Items omitted from our statistical analyses were those in which pain was evaluated by recent changes in behavioral patterns such as sleep patterns, engagement in social activities, and eating. Three of the six categories of “Common Pain Behaviors,” identified by the American Geriatrics Society [36], involve altered behavioral patterns (i.e., changes in interpersonal interactions, cognition, and activity patterns or routines). Moreover, our Delphi panel chose six items that reflected changes in behaviors: sleep patterns, appetite, cognition, social engagement, and ability to perform activities of daily living (Supplementary Data). The decision to omit these items was informed by our focus on the association between concurrent (in the moment) ratings on the PIMD items and the gold standard, ECPIR. We also wanted to develop a tool that did not rely on a rater’s prior knowledge of or familiarity with the person with dementia. Although there is anecdotal support to consider changes in behavioral patterns as pain indicators, there is little empirical evidence for the inclusion of these items in pain observation tools. Some of most rigorous psychometric evaluation studies published to date used methods that focused on acute, short-term pain and/or used raters who were unfamiliar with research participants, thereby rendering “change in behavior” items uninformative [37,38]. Further research should be designed to evaluate the sensitivity, specificity, and clinical utility of examining changes in usual patterns of behavior to measure pain intensity.

The LASSO method enabled us to evaluate many different combinations of items to identify a set that best predicted an expert’s evaluation of pain intensity. To our knowledge, this approach has not previously been used to construct or refine pain observation tools. Our seven items represent a best estimate for predicting pain intensity. However, these seven items explain only 51% of the variance in the ECPIR. Although this accuracy is very good, and similar to the associations between physician pain ratings and the Mobilization-Observation-Behaviour-Intensity-Dementia Pain Scale, another validated pain tool [39], there is room for improvement. Future studies should assess whether the accuracy of the tool could be enhanced by incorporating additional data elements to measure pain [8,40], including identifying painful conditions and using surrogate pain reports. However, efforts to improve the prediction of pain intensity using multiple inputs are limited, and results have been disappointing [41].

Our study had several limitations. First, our Delphi panel was small, consisting only of five participants. However, all panel members had extensive expertise in geriatric pain assessment, particularly in persons with dementia. Moreover, this small size allowed for a robust conversation about how clinicians arrived at gestalt assessments of a person’s pain and the heuristics they employed to arrive at that evaluation. Second, the sample for the statistical analyses was limited to 95 participants in whom pain intensity levels were quite low, despite our focused efforts to recruit residents with moderate to severe pain. The mean ECPIR (SD) was 3.2 (2.7) on a scale of 0–10; thus, the effectiveness of these seven items in accurately reflecting moderate to severe pain intensity needs to be tested in future studies. These studies may need to be conducted in settings where persons with dementia are more likely to experience higher pain levels (e.g., acute care settings or post–acute care units). Third, the final seven-item set may not fully differentiate pain-related behavior from other behaviors associated with dementia. However, the expert clinicians’ ratings that we used as the standard to evaluate items in our LASSO model were specific to pain intensity. Moreover, the final seven-item set for the PIMD did not include behaviors that are commonly associated with dementia, such as agitation, repeated calling out, irritability, and hitting [42,43]. Finally, we did not specifically test whether a different scale would have been more effective in predicting pain. We used a four-point Likert scale ranging from 0 (behavior not observed) to 3 (severe); although this approach is consistent with existing measures of pain and other symptoms in persons with dementia [10,42,44], comparison of this scaling compared with other measurement approaches is warranted.

Despite the limitations, we believe that this is the first study to examine pain indicators across many tools and to systematically reduce the item pool using a Delphi panel and advanced statistical methods. Further psychometric testing is necessary to examine the validity and reliability of the PIMD when used as a single tool to measure pain intensity. In addition, we need to examine its performance in a larger sample with higher pain levels and to evaluate its ability to detect changes in pain intensity in response to therapies. The present study, then, is an initial but important step in systematically identifying the best indicators and scaling for an effective, clinically useful pain intensity tool for vulnerable populations of older adults with dementia. This research also contributes to the field by presenting an approach for choosing and evaluating items on other pain observation tools. Finally, our findings provide further support for the importance of including facial expressions, vocalizations, and psychomotor indicators in pain observation tools.

Appendix

Table A1.

Listing of all items presented to the Delphi panel

Behavioral, Physiologic Indicator	Instrument
	MOBID [14,39,45]	PAINE [46]	PROMIS-SF [49,50]	MDS 3.0	ADD [19]	Keefe & Smith [48]	PAINA D [52]	CNPI [18,50]	NO-PAIN [51]	PACSLAC [52]	Dolo-plus [53]	Abbey [54]
Abusing others sexually				x
Agitation, agitated, upset					x					x
Aggressive behavior					x
Angry, mad										x
Anxious										x
Apathy, apathetic		x
Biting							x
Blank look											x
Blinking eyes					x
Blood pressure/pulse outside normal limits												x
Bracing		x				x		x
Breathing, Cheyne-Stokes							x
Breathing, labored							x
Breathing, noisy					X		x
Breathing rapid, hyperventilation							x
Changes in appetite, increased appetite										x
Change in appetite, refusing food					x					x		x
Changes in cognition, increased confusion										x
Changes in gait					x
Changes in mobility, decreased movement					x						x
Changes in behavior: sudden change in patterns, increased demands for attention											x
Changes in function, decreased activity, impaired ADLs					x					x	x
Changes in participation in activities					x						x
Changes in rest, increased rest periods
Changes in sleep, decreased sleep, difficulty falling asleep, insomnia, frequent waking					x					x	x
Changes in sleep, increased sleep during day										x
Alteration of usual pattern												x
Changes in wandering: increased wandering
Changes in vital signs		x
Clenched fists					x		x			x
Clenched jaw, teeth							x	x		x
Closing eyes, narrowed eyes	x				x		x	x
Cold and clammy										x
Combative behavior
Confusion					x							x
Consolability, unable to console, distract comfort							x
Contractures												x
Crouching	x
Crying, tearfulness, weeping		x			x	x	x	x	x			x
Decreased social interactions											x
Dirty look										x
Disrobing in public				x
Disruptive behavior					x
Disruptive sounds				x
Distorted face		x					x	x
Elopement					x
Exiting, trying to leave					x					x
Fidgeting							x			x		x
Flinching										x
Flushed face										x		x
Freezing	x
Frequent shifting, inability to stay still									x
Frightened face					x		x					x
Frowning	x				x		x			x		x
Frustrated										x
Furrowed brow, creased forehead							x	x	x	x
Grabbing				x
Gasping	x	x						x	x
Grim face										x
Grimacing	x	x	x		x	x	x		x	x		x
Grumbling							x
Grunting		x			x			x	x	x
Groaning	x	x					x	x	x	x		x
Guarding, protection of sore areas	x	x			x	x			x	x	x	x
Hitting others, striking out, punching				x			x			x
Hitting self				x
Holding body part					x				x	x
Hyperactiviity					x
Hypo-activity					x
Irritability, cranky		x	x		x					x
Isolate self			x
Kicking										x
Knees pulled up, fetal position							x			x
Limping										x
Moaning		x			x		x	x	x	x
Moodiness		x
Move slowly			x							x
Move stiffly, stiffness			x
Mumbling					x		x			x
Muttering							x
Not wanting to be touched										x
Not wanting people near										x
Open mouth										x
Opening eyes wide					x		x
Pacing				x			x			x
Pain expression										x
Pale face										x
Protective body postures											x
Posturing, weird movements		x
Pulling away							x			x
Pulling you toward self (resident)		x
Pushing	x			x			x			x
Refusing medications										x
Repetitive movements, repetitive behavior		x								x	x
Resisting care, uncooperative					x					x
Resisting movement, refusing to move										x
Restlessness		x			x			x		x
Restricted movement
Rigid or stiff body or body part		x					x		x	x
Rocking								x	x			x
Rubbing body part, massaging					x	x	x	x	x
Rummaging				x
Sad look					x		x			x
Scratching others				x						x
Scratching self				x
Screaming, yelling	x	x		x	x					x
Screwing up nose										x
Sexual acts (public)				x
Shaking, trembling										x
Sighing		x				x		x	x
Socially inappropriate behavior					x
Squinting										x
Squirming							x
Sweating, perspiring										x		x
Temperature												x
Tense body					x
Thrashing			x							x
Throwing or smearing bodily wastes				x
Throwing or smearing food or other things				x						x
Tightened eyes
Tightening mouth, lips	x							x
Tight face										x
Looking tense												x
Vocalizations: calling out, repeated calling out					x		x
Vocalization: chanting					x
Vocalizations: complaints											x
Vocalization: cursing, swearing							x	x	x
Vocalization: “help me,” asking for help			x							x
Vocalizations: “ouch,” “that hurts”	x							x	x	x
Vocalization: “stop that,” “that’s enough”								x	x
Vocalization: verbally abusive, aggressive					x					x
Vocalization: a specific sound or vocalization										x
Wandering					x					x
Whining		x			x		x
Whimpering		x										x
Winces									x	x
Withdrawn, withdrawal, decreased communication					x						x	x
Worried look					x
Wringing hands					x
Wrinkled forehead

ADD = Assessment of Discomfort in Dementia protocol; CNPI = Checklist of Nonverbal Pain Indicators; MDS = Minimum Data Set, version 3.0, Item J8: Indicators of pain or possible pain; MOBID = Mobilization-Observation-Behaviour-Intensity-Dementia Pain Scale; NO-PAIN = Non-Communicative Patient’s Pain Assessment Instrument; PACSLAC = Pain Assessment Checklist for Seniors with Limited Ability to Communicate; PAINAD = Pain Assessment in Advanced Dementia scale; PAINE = Pain Assessment in Noncommunicative Elderly; PROMIS = Patient-Reported Outcomes Measurement Information System, version 1.0, Item Bank–Pain Behavior–Short Form.