II. Indices of pain intensity derived from ecological momentary assessments and their relationships with patient functioning: an individual patient data meta-analysis

Abstract

Pain intensity is a complex and dynamic experience. A focus on assessing patients’ average pain levels may miss important aspects of pain that impact functioning in daily life. In this second of three articles investigating alternative indices of pain intensity derived from Ecological Momentary Assessments (EMA), we examine the indices’ associations with physical and psychosocial functioning. EMA data from 10 studies (2,660 patients) were reanalyzed to construct indices of Average Pain, Maximum Pain, Minimum Pain, Pain Variability, Time in High Pain, Time in Low Pain, Pain after Wake-up. Three sets of individual patient data meta-analyses examined (1) the test-retest reliability of the pain indices, (2) their convergent validity in relation to physical functioning, fatigue, depression, mental health, and social functioning, and (3) the incremental validity of alternative indices above Average Pain. Reliabilities approaching or exceeding a level of 0.7 were observed for all indices, and most correlated significantly with all functioning domains, with small to medium effect sizes. Controlling for Average Pain, Maximum Pain and Pain Variability uniquely predicted all functioning measures, and Time in High Pain predicted physical and social functioning. We suggest that alternative pain indices can provide new perspectives for understanding functioning in chronic pain.

Perspective

Alternative summary measures of pain intensity derived from EMA have the potential to help better understand patients’ pain experience. Utilizing EMA for the assessment of Maximum Pain, Pain Variability, and Time in High Pain may provide an enhanced window into the relationships between pain and patients’ physical and psychosocial functioning.

Introduction

Pain intensity is one of the core outcome measures in chronic pain research and practice. Even though the label “chronic pain” carries a notion of constancy, pain experiences in everyday life are not static, but rather dynamically changing within days and across days, even in patients with chronic pain [9,31,52]. Often the primary outcome variable in clinical settings has been the average of an individual’s pain intensity over a specified period of time (e.g., the average pain over the course of one week). However, such an emphasis on a patient’s average pain level misses patterns and dynamics of the everyday pain experience that may uniquely contribute to or may be even more important than average pain for understanding patient functioning and adjustment to chronic pain.

This article is the second in a series of three articles to investigate alternative indices that characterize different aspects of a patient’s pain experience based on ecological momentary assessments (EMA). Using EMA, patients rate their current pain in real time multiple times each day in their natural environment, which provides the opportunity to capture the ebb and flow of pain intensity and to construct new outcome measures [4,55,63]. Our goal is to evaluate a pool of potential pain indices to identify those with the greatest potential for chronic pain research. Whereas the first article examined stakeholder preferences for different pain indices based on qualitative interviews [61], the present research provides a large-scale empirical examination of the relationships between alternative pain indices and measures of patient physical, emotional, and social functioning.

There are many candidates for alternative indices (see [61,63]). One pain characteristic derived from EMA is the amount of variability in a person’s pain levels; in fact, some studies suggest that having more pain variability is associated with poorer psychosocial outcomes such as depression [1,52,77]. Two others are the maximum (“worst”) or minimum (“least”) pain levels that a patient tends to experience over a period of time, which have been recognized as theoretically meaningful pain measures [17,30], even though little is known about their unique relationships with patient functioning. The amount of time patients spend either at low or at high pain levels considers the frequency and duration of certain pain states; evidence suggests that the frequency of pain is distinct from average pain intensity [34,54,75], and our prior research found that prolonged episodes of high pain states may be uniquely predictive of physical functioning limitations [53]. In addition, in view of diurnal variations in pain intensity [6,62] and evidence that morning pain flares predict subsequent poorer adjustment to pain [27,69], we investigate the importance of pain levels after wake-up in this study.

It should be noted that several of these pain indices (most prominently, average, worst, and least pain) have been commonly assessed using retrospective (e.g., 24-hour or 7-day recall) measures [17,30]. A comparison of EMA-based and recall-based measures is beyond the purview of this study. Despite some recognition of the potential usefulness of alternative EMA-derived metrics for characterizing patients’ pain intensity, investigations of their validity have been scant and typically restricted to small studies. To overcome this limitation and to enhance the generalizability of results, the present research uses secondary analyses of a number of EMA studies to construct and compare alternative pain indices. Results are combined across studies using meta-analysis methods. We first examine the reliability of alternative pain indices derived from EMA because reliability sets an upper limit for validity. Next, we evaluate the convergent validity of the pain indices by examining their relationships with several patient functioning domains. The domains are physical functioning, fatigue, depression, general mental health, and social functioning, all of which are considered key aspects of patients’ health and wellbeing [14]. Finally, we evaluate incremental validity by examining whether alternative pain indices uniquely contribute to understanding patient functioning above an EMA index of “average pain”.

Methods

Study identification

The sample of studies for the current project was drawn from a larger systematic review of EMA methodology in chronic pain research [37]. This larger review covered the existing body of studies that presented empirical results based on EMA measures of pain intensity in adult chronic (non-cancer) pain samples. Potentially eligible datasets were identified through a systematic literature search conducted in October 2016 using PubMed and Web of Science databases with the following search terms: [("Ecological Momentary Assessment" or "Experience Sampling" or "Electronic Diary" or "Electronic Diaries" or "Electronic Interview" or "Electronic Interviews" or "Interactive Voice Response" or "Intensive Diaries" or "Ambulatory Monitoring" or "Ambulatory Assessment") and "Pain"].

To be included in the present project, studies needed to consist of more than 30 adult patients (studies investigating pediatric patients were excluded) and had to administer a minimum of 3 EMA pain intensity prompts per day for at least 7 days with a fixed or random assessment schedule. This was required to allow for sufficient data for the computation of the targeted indices capturing different aspects of pain intensity over the course of a week from the EMA ratings. Studies using daily diaries (which consist of a single assessment per day, typically covering a 24-hr reporting period) were not included. Another requirement was that EMA data needed to be assessed via electronic diaries, smart phones, or interactive voice responses; studies using paper diaries were excluded because responses are not electronically time-stamped, which can undermine the validity of these data (e.g., back-filling, forward-filling) [68]. Observational studies and clinical trials were included; data from clinical trials were limited to no-intervention/baseline assessment periods because relationships between pain and functioning might change as a result of treatment. EMA pain assessments needed to focus on the monitoring of momentary (i.e., current) pain intensity; excluded were studies utilizing EMA exclusively as an intervention trigger (e.g., just-in-time adaptive interventions).

Full-text versions of all articles included in the larger systematic review [37] were assessed for eligibility in the present analyses. Authors of studies that were deemed eligible were contacted via email to ask whether they would be willing to share their dataset for the purposes of the current project. Authors who gave permission were asked to share electronic copies of the de-identified primary data files and annotated codebooks (if available) for the present secondary data analyses. Datasets needed to include time-stamped EMA ratings of current pain intensity (raw data) for each participant, as well as demographic characteristics (age, gender, medical diagnosis) and baseline questionnaire data (item or scale scores of physical, emotional, and/or social functioning). Codebooks on item scoring were developed by the research team for those datasets that did not include annotated codebooks (ambiguities in the datasets were resolved through communication with the original investigators). The University of Southern California Institutional Review Board approved the secondary analysis project.

Analysis strategy

Individual patient data (IPD) meta-analysis techniques were used to analyze the data and to combine results across the different studies [50]. In contrast to conventional meta-analyses, which rely on summary statistics reported in published (or unpublished) articles, IPD meta-analyses use raw data as the basis for analysis, which allowed us to generate pain indices from EMA data that were not considered in the original studies. Data analyses proceeded in three broad steps. In the first step, the different pain indices under investigation were generated from the raw EMA data in each study. The second step consisted in calculating effect size statistics (and inverse variance weights) examining the reliability, convergent validity, and incremental validity of the different pain indices in each study. In the third analysis step, the effect sizes of each index were combined across studies using random effects meta-analysis. Details on each of the 3 steps are described below.

Calculating pain indices

Table 1 provides a description of the pain indices examined, along with their interpretation. A measure of Average Pain was created by taking the arithmetic mean of pain ratings over a given week. The Average Pain index served as the standard of comparison for the reliability and validity of the other indices. Indices of Maximum Pain and Minimum Pain were defined as the highest and lowest momentary pain rating in a week, respectively, following prior research [64] and in accordance with conceptualizations of patients’ worst and least pain levels [59]. The standard deviation of each patients’ pain ratings over the week was calculated as measure of Pain Variability. Out of various available metrics to quantify variability, the intraindividual standard deviation was selected because it is arguably the most frequently utilized variability metric [16,40], its statistical properties are well understood, and it has proved useful in pain research [18,24,66]. Unlike indices of time-structured variability that take the ordering of observations into account, the standard deviation is a measure of “net variability” that is not affected by different time lags and that can be readily calculated from unequally spaced EMA ratings [45]. Indices estimating how much time a patient spent in high pain or in low pain were created as the percentage of pain ratings above or below predefined cut-offs. Studies included in the analyses either used a 0-100 visual analogue scale (VAS) or a 0-6 scale, and we defined the Amount of Time in High Pain as the percent of ratings ≥75 (on the VAS) or the percent of ratings at 5-6 (on the 0-6 scale), and the Amount of Time in Low Pain as the percent ratings ≤34 (on the VAS) or the percent ratings at 0-1 (on the 0-6 scale); the VAS cut-offs were guided by previous work providing thresholds for severe and mild pain from VAS ratings [7]. Finally, a measure of the Average Pain After Wake-up was constructed by averaging the first pain rating of each day (if it was taken before noon) over a given week.

Table 1:

Definition of pain indices derived from EMA

Pain index	Description	Meaning
Average Pain Level	Arithmetic mean of all pain intensity ratings over the course of a week	Higher values indicate worse pain intensity on average across the week
Maximum Pain Level	Highest momentary pain level over the week	Worst pain level of the week
Minimum Pain Level	Lowest momentary pain level over the week	Least pain level of the week
Pain variability	Intraindividual standard deviation of all momentary pain ratings over the week	Higher values indicate that pain levels show more fluctuation with a greater amplitude
Amount of Time in High Pain	Percentage of momentary pain ratings at a level of ≥75 points on a 0-100 scale (or ratings of 5-6 on a 0-6 scale)	Higher values indicate that the patient spends more time in “severe” pain.
Amount of Time in Low Pain	Percentage of momentary pain ratings at a level of ≤34 points on a 0-100 scale (or ratings of 0-1 on a 0-6 scale)	Higher values indicate that the patient spends more time experiencing low (e.g., “tolerable”) pain.
Average Pain After Wake-up	The first momentary pain rating of each day (if provided before noon), averaged across the 7 days of the week	Indicates a patient’s average (or “typical”) pain level early in the day.

In calculating the different pain indices, we had to define the period of time over which they were to be constructed. We decided to use 7-day (i.e., 1-week) periods, because a week is a common reporting period for patient-reported outcomes and arguably constitutes a sufficiently long window of time to capture different aspects of the pain experience. Using 7-day intervals also aligned well with the study designs of all included datasets, in that the original studies employed EMA sampling for time periods of either 7, 14, or 28 days (i.e., 1, 2, or 4 weeks, considering only the baseline assessment periods in clinical trials). Pragmatically, this had the advantage that we could utilize all relevant EMA data from all of the studies without needing to arbitrarily dismiss data that would fall outside of the time-frame selected to create the indices. In studies that collected EMA data for more than 1 week, the pain indices were computed for each week of EMA assessment (allowing for test-retest reliability analyses, described below).

The measurement of pain indices can be impacted by trends in the data. For example, adequate measurement of pain variability assumes that the EMA data are (weakly) stationary, such that the mean and variance of a person’s ratings do not change over time [74]. We tested this assumption using the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test [35], separately for each person and week in all datasets. The KPSS test was not significant for the vast majority (91.2%) of participants and weeks, suggesting that the data were sufficiently stationary.

An additional consideration was that too few momentary pain ratings during a week would fail to adequately capture the different pain indices and to discriminate different aspects of the pain experience. Consequently, we required that at least 14 EMA pain assessments needed to be available during a given week for the pain indices to be computed, and we excluded weeks with less than 14 EMA assessments (e.g., due to missed prompts or device malfunction) from the analyses (if a patient had multiple weeks of EMA data and met the criterion for one week, but not for other weeks, only the former week was retained).

Effect size calculation

For each pain index and in each study, we computed standardized effect size statistics to examine the (1) test-retest reliability, (2) convergent validity, and (3) incremental validity of the indices.

Reliability was assessed in studies that provided EMA data for more than 1 week, using the intraclass correlation coefficient (ICC) as the average reproducibility of scores across weeks [56]. As recommended by Raykov and Markoulides [47], the ICCs were logit transformed to normalize the sampling variances before combining the ICCs across studies.

Convergent validity was assessed using product moment correlation coefficients. For each index, we calculated its correlation with measures of physical, emotional, and social functioning available in each study. The correlations were Fisher z-transformed for use in meta-analysis.

To assess the incremental validity of alternative pain indices above the Average Pain index, we examined their unique relationship with measures of patient functioning after partialling out the effect of Average Pain. Semipartial correlations were calculated as effect size measure for this purpose (Fisher z-transformed for meta-analysis). The semipartial correlation is a recommended effect size statistic to be used to represent results of regression analyses with multiple continuous predictors [3]. It represents the partial association between the dependent variable (here a measure of patient functioning) and a focal predictor variable (here an alternative pain index) after statistically controlling for the effects of a covariate (here the Average Pain index). The square of the semipartial correlation is the incremental variance explained by the focal predictor.

Standard errors (used to compute inverse variance weights for the meta-analysis) of the effect size measures (ICCs, correlations, and semipartial correlations) were estimated using heteroscedasticity-consistent (i.e., robust) standard errors, which are robust to potential deviations from normality. In studies that provided EMA data for multiple weeks, we calculated effect sizes addressing convergent and incremental validity for each week and obtained the pooled mean effect size across all weeks. To derive unbiased standard errors in these studies, we needed to consider that the correlations were not independent because the indices themselves were correlated across the weeks. Therefore, we estimated the full covariance matrix between the pain index computed for each of the weeks and a given functioning measure, and derived the standard errors of the pooled mean effect size by means of the multivariate delta method [46]. All effect sizes and associated standard errors were calculated using maximum likelihood parameter estimation in Mplus version 8.2 [41].

Meta-analysis models

Random effects meta-analyses were conducted to combine the effect sizes (ICCs, correlations, and semipartial correlations) across studies, separately for each pain index and functioning measure. The choice to use random effects models was made a priori. Whereas fixed effects meta-analysis models assume that the studies and samples were drawn from one single homogeneous population, the studies in our analyses were conducted in a variety of different environments and involving different pain conditions, making random effects models likely more suitable.

Standard errors of random effects coefficients in the meta-analysis models were obtained using restricted maximum likelihood parameter (REML) estimation [71]. A statistical consideration was that the meta-analyses were conducted with a limited number of studies and based on unbalanced sample sizes (with modest sample sizes in some studies and large sample sizes in other studies, see Results section). For this reason, we adjusted the random effects variances (and confidence intervals) using the Knapp and Hartung method [33]. This method has been shown to yield more accurate confidence intervals compared to unadjusted methods especially when a small number of studies with unbalanced sample sizes is combined in random-effects meta-analysis [29,51]; rather than assuming a standard normal distribution, individual coefficients and confidence intervals are based on a t-distribution (with degrees of freedom equaling the number of studies minus one), which generally yields more conservative p-values.

In addition, the presence of outliers may affect the validity of meta-analytic results, especially when the number of studies is modest. Following Viechtbauer and Cheung [73], we examined the studentized deleted (also called externally studentized) residuals in each meta-analysis model using formulas developed for random effects meta-analysis. We formally considered a study an outlier if its externally studentized residual was greater than 3 in absolute value. Where one or more outlier studies were detected in a meta-analysis model, we conducted sensitivity analyses in which the original results were compared with results from a model where outlier studies were removed.

The Q-statistic was used to perform significance tests of effect size heterogeneity between studies, and the I² statistic was calculated to estimate the percentage of variation in the observed effect sizes between studies that was due to true heterogeneity in effects rather than chance [25]. Age, gender (proportion of female patients in the sample), chronic pain diagnosis (proportion of patients with fibromyalgia in the sample, which was the most prevalent condition across datasets), and EMA sampling density (the number of EMA ratings collected per week) were explored as effect size moderators to explain potential effect size heterogeneity in mixed effects (meta-regression) meta-analysis models.

P-values smaller than .05 were considered statistically significant. Given that a series of meta-analyses were performed, the p-values of overall effect sizes (correlations and semipartial correlations combined across studies) were adjusted for multiple testing using adaptive step-up Bonferroni correction [26]; 95% confidence intervals of the effect sizes were adjusted accordingly. All meta-analysis models were estimated with R software version 3.4.3 using the metaphor package [72].

Results

Data acquisition

The initial literature search identified 699 unique articles (see Figure 1). Of those, 539 records were excluded based on a review of article title and abstract, and 52 were excluded after full-text article review (e.g., no EMA data, no chronic pain sample, no adult sample). The remaining 108 articles were based on 64 unique databases and these were assessed for eligibility in the current analyses. Of those, 43 databases (63 articles) were eliminated from consideration (e.g., less than 3 EMA prompts per day, EMA data collection for less than 7 days, paper diaries), and 21 databases (reported in 44 articles) were eligible. After contacting the authors of eligible studies, EMA data were received for 13 of the 21 databases; 8 datasets were not received because authors did not respond to the request, declined to provide the data, or because the data were no longer available. Three datasets were not included in the analyses because they provided only partial data or did not include questionnaire data on functioning, and 10 datasets were analyzed.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram

Descriptive study and patient characteristics

The analyzed studies were conducted in the USA (7 studies), the Netherlands (2 studies), and Belgium (1 study). All studies used a random EMA prompting schedule, and 9 of the 10 studies used palm pilots as the data entry device (one study [36] used interactive voice response technology via cell phones). Most (7 studies) used an observational design and 3 were randomized controlled trials from which only data collected during the baseline assessment periods were used for the present purposes. Of 5549 assessment weeks (from 2718 patients) combined across datasets, 194 weeks were excluded from the analyses because they had less than 14 EMA pain ratings; 58 patients were fully excluded because they had no week with 14 or more EMA ratings or were missing data on the functioning questionnaires. Thus, 2660 patients were included in the analyses, together providing a total of 178,267 momentary pain assessments over 5355 assessment weeks.

Descriptive sample characteristics are shown in Table 2. The median number of patients per study was 67 (interquartile range = 65 to 114), with sample sizes ranging from 31 to 1196 patients across studies. EMA pain ratings were collected for 1 week in 3 studies, for 2 weeks in 5 studies, and for 4 weeks in 2 studies. Across datasets, the median number of EMA pain ratings completed per week was 34 (interquartile range = 29 to 38, mean = 33.8, SD = 8.1).

Table 2:

Descriptive characteristics of studies and patients included in the meta-analysis

	Broderick et al. [8] †	Clauw et al. [11]	Huijnen et al. [28]	Litt et al. [36]	Mease et al. [38]	Peters et al. [43]	Smyth et al. [60]	Stone et al. (a) [65] †	Stone et al. (b) [9] †	Viane et al. [70]
Sample size	106	888	65	65	1196	65	31	68	116	60
Modal number of EMA weeks	4	2 ‡	2	1 ‡	2 ‡	4	1	2	1	2
EMA prompts/week, mean (SD)	37.7 (6.3)	33.0 (5.7)	39.9 (10.3)	20.11 (4.3)	32.4 (6.1)	23.1 (3.2)	30.6 (7.2)	40.4 (21.0)	47.4 (8.1)	42.7 (7.3)
Age, mean (SD)	55.29 (10.63)	49.45 (10.69)	48.43 (10.01)	39.10 (11.80)	50.23 (10.61)	40.13 (9.66)	50.03 (13.08)	50.94 (10.50)	57.37 (13.12)	46.22 (9.51)
Female (%)	85.9%	95.6%	43.1%	81.5%	96.2%	78.5%	74.2%	85.3%	84.5%	81.7%
Average pain, mean (SD)	4.2 (2.1)	6.6 (1.3)	4.8 (1.7)	3.2 (1.9)	6.2 (1.4)	4.5 (2.1)	4.2 (2.1)	4.4 (2.2)	3.8 (1.9)	5.7 (1.5)
Diagnosis (%)
Fibromyalgia	10.5%	100%	--	--	100%	--	--	66.2%	36.2%	--
Osteoarthritis	48.6%	--	--	--	--	--	--	41.2%	84.5%	--
Rheumatoid arthritis	28.6%	--	--	--	--	--	100%	7.4%	20.7%	--
Low back pain	--	--	100%	--	--	--	--	--	--	--
Temporomandibular disorder	2.9%%	--	--	100%	--	--	--	--	--	--
Other/not specified	31.1%	--	--	--	--	100%	--	7.4%	--	100%
Functioning measures
Physical functioning	SF-36, PF	SF-36, PF	SF-36, PF	--	SF-36, PF	SF-36, PF	--	SF-36, PF	SF-36, PF	--
Fatigue	SF-36, VT	SF-36, VT	SF-36, VT	--	SF-36, VT	SF-36, VT	--	SF-36, VT	SF-36, VT	--
Depression	BDI	BDI	BDI	CES-D	BDI	--	CES-D	BDI	BDI	HADS-D
General mental health	SF-36, MH	SF-36, MH	SF-36, MH	MPI-AD	SF-36, MH	SF-36, MH	--	SF-36, MH	SF-36, MH	--
Social functioning	SF-36, SoF	SF-36, SoF	SF-36, SoF	--	SF-36, SoF	--	--	SF-36, SoF	SF-36, SoF	--

Note: Sample sizes and patient characteristics refer to those included in the present analyses, not to those collected in the studies.

^†Diagnoses do not sum up to 100% in these studies because individual patients could have multiple diagnoses.

^‡Clincial trials; only data collected during the baseline assessment weeks were used from these studies. Average pain levels were derived from EMA pain ratings; because the number of scale points for EMA ratings varied across studies, the scores were translated onto a common 0-10 scale using the following formula: transformed score = 10 * (original score + 0.5) / number of scale points [42]. PF = SF-36 Physical Functioning subscale; VT = SF-36 Vitality scale; BDI = Beck Depression Inventory; CES-D = Center for Epidemiologic Studies Depression scale; HADS-D = Hospital Anxiety and Depression scale, depression subscale; MH = SF-36 Mental Health subscale; MPI-AD = Multidimensional Pain Inventory Affective Distress subscale; SoF = SF-36 Social Functioning subscale.

In terms of patient characteristics, the pooled average patient age was 49.85 (SD = 10.76) years, and 91.45% of the patients were women. Five studies focused on patients with one specific diagnosis (fibromyalgia, rheumatoid arthritis, low back pain, temporomandibular disorder), 3 enrolled patients with mixed medical diagnoses, and 2 studies examined patients with multiple pain sites categorized according to the location of their pain.

Most studies provided questionnaire data on patient functioning in multiple domains: Physical functioning was assessed in 7 studies, all of which used the SF-36 Physical Functioning subscale [76]. Fatigue was assessed in the same 7 studies with the SF-36 Vitality subscale. Depression was measured in 9 studies using the Beck Depression Inventory [5] (6 studies), the Center for Epidemiologic Studies Depression (CES-D) scale [44] (2 studies), and the Depression subscale of the Hospital Anxiety and Depression Scale [79] (1 study). General mental health was assessed in 8 studies using the SF-36 Mental Health subscale (7 studies) and the Multidimensional Pain Inventory Affective Distress subscale [32] (1 study). Social functioning was assessed in 6 studies using the SF-36 Social Functioning subscale.

Test-retest reliabilities

Figure 2 shows the overall test-retest reliabilities of each pain index from meta-analyses across the 7 studies that provided EMA data for more than 1 week. ICCs of .80 or above were observed for measures of Average Pain, the percent Time in High Pain, and the percent Time in Low Pain. Indices of Minimum Pain and the Average Pain after Wakeup showed ICCs ranging between .75 and .80. Maximum Pain and Pain Variability were captured somewhat less reliably, with ICCs between .65 and .70. Overall, these results suggest moderate to high temporal stability of all pain indices.

Figure 2.

Results from random effects meta-analyses examining test-retest reliability of each pain index. Effect sizes are intraclass correlation coefficients. ICC = intraclass correlation coefficient; CI = confidence interval. All intraclass correlations are significant at p < .001.

Analyses of effect size heterogeneity (Q and I² statistics in Figure 2) showed no significant between-study variation of the ICCs for 5 of 7 indices; significant variation was only evident for ICCs of Average Pain and Time in Low Pain, resulting in a wide confidence interval especially for the latter index. None of the examined effect size moderators (age, gender, diagnosis, number of EMA ratings) significantly accounted for between-study differences in ICCs. While no study was flagged as an outlier based on inspection of externally studentized residuals, we conducted sensitivity analyses estimating the overall ICCs when any one of the studies was removed from the meta-analyses of these indices. The impact of leaving out any one study was modest, resulting in ICCs ranging between .85 to .88 for Average pain and ICCs ranging between .79 and .86 for Time in Low Pain.

Associations between pain indices and patient functioning measures

The usefulness of alternative pain indices would be severely undermined if some of them were near-perfectly correlated with each other. To check this possibility, we inspected the correlations among the indices before investigating the relationships with other functioning measures. Pooled across studies, the Average Pain index was significantly positively correlated with Maximum Pain (r = .74), Minimum Pain (r = .85), the percent of Time in High Pain (r = .85), and Pain After Wakeup (r = .87), and it was negatively correlated with Pain Variability (r = −.37) and the percent of Time in Low Pain (r = −.76; this negative correlation is expected because higher scores on this index mean that a patient spent more time at low pain levels). The highest correlation among the alternative indices was found for Pain After Wakeup and Minimum Pain (r = .76). Thus, although several indices showed a sizeable degree of communality (the median shared variance was 47%), they were sufficiently distinct to justify analyses of their convergent and incremental validity.

Convergent validity results

Figure 3 presents the overall effect sizes (i.e., correlations) for the relationships between the pain indices and functioning measures in each domain. For presentation, all functioning measures were scored in the direction indicated by the label of the domain. Higher Average Pain levels were significantly associated with worse physical functioning (p = .001), more fatigue (p = .014), more depression (p = .002), worse mental health (p = .022), and worse social functioning (p = .002; all p-values are Bonferroni-corrected). The magnitude of the correlations for Average Pain ranged between .22 (for mental health) and .31 (for physical functioning), indicating small to medium effect sizes following Cohen’s conventions. With the exception of Pain Variability, all of the alternative pain indices were also significantly associated with all domains of functioning in expected directions (ps < .05 after Bonferroni correction). The correlations between the alternative pain indices and the functioning measures were numerically weaker than those for Average Pain in all instances, even though the differences in correlations were for the most part very small (see Figure 3).

Figure 3.

Results from random effects meta-analyses examining convergent validity of each pain index in relation to the functioning domains. Effect sizes are zero-order correlations. Mean ES = overall effect size across studies; CI = confidence interval. *p < .05; **p < .01; ***p < .001; p-values and 95% confidence intervals are corrected for multiple testing (35 tests) using adaptive step-up Bonferroni correction.

Pain Variability showed no significant zero-order correlations with any functioning domain. Sensitivity analyses were conducted to examine whether this finding was attributable to statistical properties of the index. The observed standard deviation of a participant’s ratings can be affected by the boundaries of the rating scale in that the maximum possible variability is lower the closer the mean score is located at one of the extreme ends of the scale. To examine whether this impacted the results, we repeated the analyses using a recently proposed index of “relative variability”, which is computed by dividing the standard deviation by the maximum possible standard deviation given the observed mean [39]. The results were near identical across both variants of the variability index, with effect sizes (overall correlations from meta-analyses) for the “relative variability” index of r = .027 (physical functioning), r = −.041 (fatigue), r = .048 (depression), r = −.032 (mental health), and r = −.030 (social functioning, all ps>.05).

Significant between-study effect size heterogeneity was found in 21 of the 35 (i.e., 7 pain indices x 5 functioning domains) meta-analyses. Only a few effect size moderators that were tested to account for this heterogeneity were significant. A greater proportion of patients with fibromyalgia in the sample was associated with lower correlations between depression and several pain indices: Average Pain (t[7] = −3.38, p = .011), Minimum Pain (t[7] = −2.54, p = .039), Time in Low Pain (t[7] = 4.53, p = .003), and Pain after Wakeup (t[7] = −3.51, p = .010). In addition, female gender was associated with a less strongly negative correlation between Average Pain and physical functioning (t[7] = 2.63, p = .047). For analyses of general mental health as the criterion variable, one study was flagged as an outlier in 6 of 7 meta-analyses (except for analyses involving Time in High Pain) with studentized residuals consistently exceeding an absolute value of 3.0 (mental health was assessed with a subscale of the Multidimensional Pain Inventory in this study [36], whereas all other studies used an SF-36 subscale). Removing this study from the meta-analyses for general mental health did not change the pattern of results, even though the magnitude of all correlations was reduced to r = −.14 (Average Pain), r = −.13 (Maximum Pain), r = −.09 (Minimum Pain), r = .01 (Pain Variability), r = −.13 (Time in High Pain), r = .12 (Time in Low Pain), and r = −.13 (Pain after Wakeup).

Incremental validity results

The final set of analyses examined whether the alternative pain indices uniquely explained differences in patient functioning above the effects of Average Pain. Results of these meta-analyses addressing incremental validity are shown in Figure 4. No significant mean semipartial correlations were evident for the Time in Low Pain and for Pain after Wakeup in these analyses. However, both the Maximum Pain and the Pain Variability index showed consistently significant (after Bonferroni correction) mean semipartial correlations with each of the functioning domains. After statistically controlling for Average Pain, higher Maximum Pain levels were uniquely associated with lower physical functioning (p = .017), higher fatigue (p = .012), higher depression (p = .004), lower mental health (p = .030), and lower social functioning (p = .007). Similarly, greater Pain Variability was uniquely associated with lower physical functioning (p = .024), higher fatigue (p = .044), higher depression (p = .013), lower mental health (p = .043), and lower social functioning (p < .001; using the “relative variability” index as an alternative Pain Variability measure produced highly similar results). In addition, a higher amount of Time in High Pain was uniquely associated with worse physical functioning (p = .005) and worse social functioning (p = .004; ps > .05 for other domains). Significant semipartial correlations were also evident for Minimum Pain, even though the direction of these relationships was unexpected: after controlling for Average Pain, higher Minimum Pain levels were uniquely associated with lower depression (p = .027) and better social functioning (p = .013).

Figure 4.

Results from random effects meta-analyses examining incremental validity of each pain index (above the contribution of average pain levels) in relation to the functioning domains. Effect sizes are semipartial correlations controlling for the effect of average pain. Mean ES = overall effect size across studies; CI = confidence interval. *p < .05; **p < .01; ***p < .001; p-values and 95% confidence intervals are corrected for multiple testing (30 tests) using adaptive step-up Bonferroni correction.

Only 1 of the 30 (i.e., 6 pain indices x 5 functioning domains) meta-analyses showed significant between-study variation in the semipartial correlations, indicating that the effect sizes were quite homogeneous across studies in these analyses. The effects were not significantly moderated by age, gender, diagnosis, or the number of EMA ratings. Inspection of studentized residuals did not evidence outlier studies in any of the models.

Discussion

Precisely determining the associations between pain intensity and physical, emotional, and social functioning domains may facilitate the detection of underlying mechanisms that link pain with impaired quality of life, enhance understanding of comorbidities between chronic pain and other conditions, and ultimately inform improved pain assessment and treatment. In this study, we examined whether alternative summary indices capturing different aspects of pain intensity could enhance understanding of the relationships between real-time pain assessments and patient functioning. We believe our results provide considerable support that this is the case, and several results are noteworthy.

In our first set of meta-analyses investigating reliability, all of the examined indices evidenced levels of test-retest stability approaching or exceeding a value of 0.7. This result is encouraging and indicates that different aspects of the pain experience can be reliably measured from momentary pain intensity reports, at least when ratings are collected several times per day over one week. We did not find that the number of EMA assessments moderated the reliability coefficients, but caution should nevertheless be exercised when computing pain indices from fewer assessments, in that simulation studies have shown that indices based on within-person patterns are unreliably measured when the number of assessments per person is low [13,16].

The second set of meta-analyses showed that patients’ Average Pain levels derived from EMA were significantly associated with each of the functioning domains, with small to medium effect sizes ranging from .22 (mental health) to .31 (physical functioning). Our findings are in line with prior research using retrospective (e.g., 7-day recall) measures of average pain; this research has evidenced that while there are consistent linkages of average pain with emotional distress and functioning limitations, these links are not overwhelmingly strong [14,19,75]. Importantly, all alternative pain indices except for Pain Variability showed significant zero-order correlations with all functioning domains in magnitudes similar to Average Pain, suggesting that they capture relevant aspects of the pain experience that might enhance understanding of the linkages between pain and functioning.

Results from our third set of meta-analyses examining the incremental validity of alternative pain indices confirmed this hypothesis for several indices. Specifically, indices of Maximum Pain and Pain Variability had the most consistent effects and they explained additional variance in all of the functioning domains above the effects of Average Pain, suggesting that they might have a wide-ranging impact on patients’ quality of life. The amount of Time in High Pain also evidenced incremental validity but these effects were constrained to physical and social functioning domains, suggesting potential specificity in the detrimental impact of spending more time in high pain.

Given that assessments of Maximum Pain (or “pain at its worst”) have previously been recommended as outcome measures for chronic pain [14], it is perhaps surprising that only few studies have examined the role of Maximum Pain in relation to patient functioning. Some existing studies have compared whether average or worst pain correlates more strongly with functional interference, with some reporting higher correlations for worst pain [23,57] and others reporting equivalent or higher correlations for average pain [10,78]. However, these studies did not evaluate whether worst pain levels uniquely contributed to functioning above average pain and they relied on daily or weekly recalled pain ratings, which have been shown to be only moderately associated with EMA-derived indices [8,64]. The present results suggest that the level of pain maxima or peaks (e.g., due to short-term symptom flares) plays an important debilitating role in the lives of patients with chronic pain and may impede several areas of functioning. Interestingly, the results observed for Minimum Pain – where lower levels were associated with higher depression and worse social functioning after controlling for Average pain – lend additional support to the idea that short-term pain exacerbations are uniquely important, in that high Maximum Pain coupled with low Minimum Pain produces the most pronounced spikes in symptom experience. In clinical research, recording the magnitude and timing of momentary pain maxima and minima may be useful in developing individually tailored treatment recommendations for improving patient functioning. For example, EMA assessments could help patients recognize maximum and minimum pain levels, which in turn could facilitate construction of personalized behavioral treatments, including applying coping techniques and thoughtful scheduling of activities during periods when pain is less severe [2].

The index of Pain Variability had non-significant zero-order correlations with functioning; however, more Pain Variability predicted greater functioning impairments across multiple domains after partialling out Average Pain. This finding adds to the growing body of research suggesting that Pain Variability holds great promise in several areas of clinical pain research: measuring Pain Variability assessed with EMA or daily diaries may contribute to understanding of patients’ adjustment to chronic pain [1,52,77]; facilitate the prediction of individual differences in treatment response [18] to advance precision pain medicine[15]; elucidate errors and biases in retrospective judgments of pain [53,67]; and, augment accurate classification of chronic pain conditions [20].

The finding that spending more Time in High Pain predicted worse physical and social functioning replicates the results of previous analyses conducted by our group, though with a different statistical approach (a “regime-switching” time series model was used previously)[53]. In those analyses, we found that the dominance of time spent in high relative to low pain was uniquely associated with limitations in physical functioning and pain interference, but not with emotional distress [53]. The experience of prolonged elevated pain with relatively brief periods of pain relief has been viewed as an indicator of biological dysregulation in descending pain modulation (such as non-habituation and abnormal sensitization) [22,58]. As such, EMA-based assessments of Time in High Pain may be useful in research developing phenotypical markers of disease activity that are associated with physical functioning limitations.

Considering the magnitude of the effect sizes, the unique effects of alternative pain measures in relation to patient functioning are small by common conventions. The largest semipartial correlations of .10 explain 1% additional variance in patient functioning above average pain, which by itself explained between 5% and 10% of the variance in the patient functioning measures. Nevertheless, we believe that our results are robust and unlikely to capitalize on chance. Our analysis strategy aimed to provide conservative estimates by (a) calculating effect sizes with robust standard errors, (b) conducting random effects instead of fixed-effects meta-analyses with confidence interval adjustments recommended for unbalanced sample sizes, and (c) Bonferroni-correction of p-values. Interestingly, our results are also corroborated by patients’ own views and clinicians’ views about the importance of different pain metrics. In qualitative interviews, we found that patients and clinicians judged Maximum Pain levels and the amount of Time in High Pain as representing the most important pain indices, in part due to their substantial influences on everyday functioning and their relevance for guiding treatment decisions [61].

Study limitations

Several important study limitations need to be considered. Although we combined data across multiple pre-existing studies on the basis of a systematic literature search, we were not able to obtain several of the identified datasets, so the sample of studies is not completely representative of all pain intensity studies conducted using EMA. The samples were predominantly female and to a large extent comprised of patients with fibromyalgia, which limits generalizability. The fact that we did not find consistent effect size moderators in the meta-analyses may be due to this lack of diversity in demographic and medical characteristics. An additional limitation is that while the measurement of the pain indices was based on real-time momentary ratings, all of the functioning domains were assessed via traditional (i.e., recall) questionnaires, which can be subject to memory bias and may reflect patients’ beliefs about their functioning rather than their actual functioning in daily life. Furthermore, the observational study design does not allow for conclusions about the directionality of relationships and whether different pain measures are causally implicated in patient functioning.

Even though we tested several different pain intensity measures that can be readily calculated from EMA, we note that the list of pain indices considered in this study is not exhaustive. For instance, several additional measures of pain variability could be derived from EMA, including the autocorrelation of pain ratings, the mean of squared successive differences, and the probability of acute changes, all of which have quite different nuances (for review, see [40]). Other indices that are potentially relevant could be derived by coupling pain ratings with information from other sources (e.g., pain levels depending on activities or location). For example, in qualitative interviews, we evaluated stakeholders’ views of the importance of measuring the unpredictability of pain fluctuations [61]; we did not test this index in this study because it arguably cannot be captured from real-time pain intensity ratings alone and involves the coupling of pain with assessment of the individual’s expectations or predictions of changes in pain [21]. Finally, while the present project focused on pain intensity, the presented summary indices can be derived for other momentary experiences as well, and the presented techniques are potentially equally relevant for many other types of symptoms (e.g., momentary fatigue, pain qualities) or cognitions (e.g., catastrophizing).

Finally, it is important to note that the present study was exclusively focused on comparisons among different EMA-based pain intensity measures, and our results are not meant to suggest that EMA-based measures are equivalent or generally superior to traditional retrospective pain intensity measures. As has been emphasized previously [12], momentary and recall measures tap into different types of information that are functionally and neuroanatomically different from each other, and it would likely be counterproductive to use one as a simple replacement of the other. Specifically, it has been suggested that measures derived from EMA are more closely linked with physiological parameters such as cardiovascular or immune system functions [12], or with other immediate processes, such as coping behaviors or stress responses, which speak directly to pain phenomenology [37,40]. In contrast, information from recall-based measures may be more closely tied to the overall state of the patient and to longer-term planning and decision making [12,48,49]. Examining the extent to which EMA-based and recall-based pain intensity measures provide overlapping or possibly complementary types of information, both of which may be uniquely relevant for patient functioning, is an important avenue for future research.

HIGHLIGHTS.

• Multiple indices characterizing patterns of pain intensity were derived from EMA

• How the EMA pain indices relate to patient functioning was examined in 10 studies

• Several indices uniquely contributed to functioning outcomes above average pain

• Alternative EMA pain indices can enhance understanding of patient functioning