Ecological Momentary Assessment methodology in chronic pain research: A systematic review

Abstract

Self-reported pain intensity assessments are central to chronic pain research. Ecological Momentary Assessment (EMA) methodologies are uniquely positioned to collect these data, and are indeed being utilized in the field. However, EMA protocols are complex, and many decisions are necessary in the design of EMA research studies. A systematic literature review identified 105 articles drawing from 62 quantitative EMA research projects examining pain intensity in adult chronic pain patients. Study characteristics were tabulated in order to summarize and describe the use of EMA, with an emphasis placed on various dimensions of decision-making involved in executing EMA methodologies. Most identified studies considered within-person relationships between pain and other variables, and a few examined interventions on chronic pain. There was a trend toward the use of smartphones as EMA data collection devices more recently, and completion rates were not reported in nearly one-third of studies. Pain intensity items varied widely with respect to number of scale points, anchor labels, and length of reporting period; most used numeric rating scales. Recommendations are provided for reporting to improve reproducibility, comparability, and interpretation of results, and for opportunities to clarify the importance of design decisions.

Introduction

Pain intensity represents the primary outcome in most clinical trials of pain disorders and is nearly universally assessed in chronic pain research^{32, 79}. Chronic pain affects over 11% of the population of the United States⁹⁰, and there is an undisputed need for the accurate and reliable assessment of pain. Although alternatives to self-reported pain intensity (e.g., observation of pain behaviors⁶⁴) have previously been considered, self-reports presently constitute the gold standard of pain assessment because they are able to reflect the subjectivity inherent in the pain experience³².

Within the family of self-report methodologies, Ecological Momentary Assessment (EMA) is uniquely positioned to assess a patient’s pain experience with high precision. EMA involves momentary data collection in participants’ natural environments at multiple points in time, and its advantages follow from these three central aspects¹¹². First, momentary measurement reduces recall biases by capturing present pain experiences rather than pain beliefs or summary ratings based on memory. Second, EMA occurs in patients’ natural environments and social contexts, thus increasing the ecological validity of the assessment. Third, multiple repeated assessments occur over time, providing potentially fine-grained information about pain experiences. Whereas pain research is often based on cross-sectional snapshots, EMA methodologies provide rich data that facilitate the examination of short-term shifts, temporal dynamics, and the effects of specific contexts on the pain experience. In addition, Ecological Momentary Interventions, also known as Just-in-Time Adaptive Interventions, become possible with EMA^{57, 91}.

The importance ascribed to the advantages of EMA by organizations that drive protocol design for chronic pain research [e.g., Food and Drug Administration (FDA) guidelines¹⁰⁸, Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) recommendations³²], along with the increased availability of associated technologies, has naturally translated into an increase in the use of EMA in pain research. Nevertheless, it remains a specialized approach, and knowledge of its implementation is still relatively fragmented.

It is a central tenet of research design that the way in which data are collected influences the type of data collected. In a study comprised of a single assessment, the collection of pain intensity data upon waking from one group and in the middle of the day from another group would certainly be relevant to the interpretation of results. EMA methodology can similarly introduce bias, and, because of its complexity, can harbor many potential means of doing so. For example, contextual factors (e.g. location, social environment) may be associated with pain, which makes it important to consider the timing and frequency of EMA sampling¹²⁸. If sampling times are too sparse, the design may brush over symptom exacerbations or contextual influences may be overlooked, whereas too frequent sampling may be burdensome and may negatively impact data quality. Similarly, reporting decisions, such as whether compliance with momentary pain assessments is reported before or after exclusion of dropouts, can impact the appearance of study results. The importance of the various design and reporting decisions that are implicated in EMA studies necessitates detailed and comprehensive documentation.

This systematic review aims to summarize and describe the use of EMA in chronic pain research while emphasizing the various domains of decision-making involved in EMA methodologies. Since there is large variation among studies, this review is primarily descriptive. The main purpose was to examine characteristics pertaining to study populations and sampling procedures, the rationale for using EMA, data input modalities, pain assessment instrumentation, EMA completion rates, and statistical reporting. We recommend thorough reporting with respect to these domains to improve reproducibility, understanding of comparability across studies, and accuracy of interpretation of study results; we also note opportunities for future research to clarify the importance of design decisions.

Methods

Search strategy

A systematic literature search was conducted through PubMed and Web of Science databases with the following search terms: [("Ecological Momentary Assessment" or "Experience Sampling" or "Electronic Diary" or "Electronic Diaries" or "Electronic Interview" or "Electronic Interviews" or "Interactive Voice Response" or "Intensive Diaries" or "Ambulatory Monitoring" or "Ambulatory Assessment") and "Pain"]. The goal of the search was to include studies that specifically reference EMA methodologies, and we recognize the possibility that articles presenting dynamic data without specifically referring to the methodologies used to obtain the data were unintentionally excluded from the review. The search was conducted in October 2016 and therefore includes only articles published prior to that point; no other restrictions were placed on publication date.

This review focuses on quantitative EMA studies of pain intensity in adult chronic pain patients. As such, any studies that (1) did not present empirical data, (2) did not measure pain intensity with EMA, (3) did not consider a chronic (non-cancer) pain sample, or (4) did not consider a sample of adults, as well as any (5) case or qualitative studies, were excluded from the review. EMA involves the contemporary assessment of variables in participants’ natural environments; therefore, studies asking participants to recall pain experienced across the last day, as is typical in daily diary studies, as well as laboratory studies taking place outside of participants’ natural environments (e.g., studies of procedural pain), were excluded. Both paper and electronic methods of EMA data collection were acceptable for inclusion. Pain items were required to be specific to pain intensity (excluding, e.g., pain quality) and dichotomous pain variables describing the presence or absence of pain were not in and of themselves considered to be pain intensity items. Further, patient populations for whom the pain history, mechanisms, or treatment strategies would take on different forms (e.g., acute pain, cancer pain, children)^{50, 143} were also excluded. There were no other disease or design restrictions, and methodological, observational, feasibility, and intervention studies were all deemed relevant and therefore included.

The search identified 685 unique articles, and 11 articles were additionally identified through other sources (e.g., by consulting the reference lists of articles identified by the database search). Articles were considered for inclusion in a two-step process (see Figure 1): first, the abstract of each article was reviewed, and any that met the exclusion criteria were removed from the sample; second, full-text versions of all remaining publications were examined, and articles were again excluded on the basis of the aforementioned exclusion criteria. A total of 105 articles drawing from 62 unique research projects were included in the present review.

Fig. 1.

PRISMA flowchart describing the identification of articles

Extraction of study characteristics

We extracted and tabulated study characteristics from each of the articles (see Table 1). If an article referenced another article for additional protocol details, data were extracted from both articles, even if the second article was not part of the review. Review characteristics were grouped according to the following general themes: sample and design characteristics; rationale for using EMA methodologies; EMA sampling approach; data input modality; pain intensity item; EMA completion rates; statistical reporting. Two of the authors (MM and MO) created a coding manual for the review categories, which were drawn from existing guidelines for the reporting of EMA data^{74, 111, 131}. Each article was coded twice, and discrepancies between the two coders were resolved after a joint review and discussion.

Table 1.

EMA studies of chronic pain

First Author	Year	Project	Use of EMA	Design	Chronic Pain n	Diagnosis	Data input	Duration	Intensity	% Completion	Pain Item
Alschuler10	2011	1	Btwn	O	20	B/NSP	Watch	5.0	4	NR	NRS; 11-pt; M
Alschuler11	2011
Bruehl19	2012	2	Wthn	O	48	B/NSP	Palm	7.0	4	99.0	VAS; 101-pt; M
Burns20	2015	3	Wthn	O	105	B/NSP	Palm	14.0	5	83.6	NRS; 9-pt; C
Burns21	2013
Dhingra31	2014	4	Wthn	O	36	CP	Palm	7.0	6	NR	NRS; 11-pt; M
Fischer35	2016	5	Wthn	O	32	R	S-Phone	14.0	6	85.0	VAS; 101-pt; M
Focht36	2002	6	Wthn	O	32	R	Paper	6.0	5.5	91.3	NRS; 11-pt; M
Focht37	2004
Garcia-Palacios40	2014	7	Met	E	47	R	S-Phone	14.0	3	69.8	NRS; 11-pt; M
Geisser41	1995	8	Wthn	O	21	B/NSP	Paper	3.0	NR	NR	VAS; 11-pt; M
Graham-Engeland52	2016	9	Wthn	O	31	R	Palm	7.0	5	NR	NRS; 7-pt; M
Russell100	2016
Smyth116	2014
Hallman53	2014	10	Btwn	O	29	B/NSP	S-Phone	3.0	8	NR	NRS; 11-pt; M
Hamilton55	2008	11	Btwn, Wthn	O	89	R	Palm	30.0	3	98.9	NRS; 7-pt; C
Tennen136	2006
Affleck4	2000
Affleck7	1996
Affleck5	1998
Affleck6	2001
Zautra147 (Study 2)	2001
Hamilton54	2007	12	Wthn	O	49	R	Paper	2.0	7	NR	NRS; 7-pt; NR
Honkoop59	1999	13	Wthn	O	56	H/M	Palm	70.0	6	80.0	NR
Kinne68	1999	14	Btwn	O	30	Other	Palm	1.0	5	84.0	VAS; 20-pt; M
Linnemann75	2015	15	Wthn	O	30	R	S-Phone	14.0	5	89.7	VAS; 101-pt; M
Liszka-Hackzell76	2004	16	Wthn	O	18	B/NSP	Palm	21.0	9	NR	NRS; 11-pt; M
Liszka-Hackzell77	2005
Lousberg83	1997	17	Wthn, Met	O	57	CP	Paper	6.0	10	88.3	NRS; 7-pt; NR
Vendrig139	1997
Mujagic86	2015	18	Met	O	26	IB	S-Phone	7.0	10	76.8	NRS; 7-pt; NR
Glaros43	2005	19	Btwn, Wthn	O	171	TMD	Paper	7.0	NR	NR	NRS; 11-pt; M
Glaros47	2005
Glaros48	2008
Glaros49	2005
Glaros44	2016
Glaros45	2007	20	Intv	E (CT)	14	TMD	Paper	14.0	NR	NR	NRS; 11-pt; NR
Glaros42	2014	21	Intv, Btwn	E (CT)	23	H/M	Paper	7.0	NR	NR	NRS; 11-pt; NR
Glaros46	2007
Odawara92	2015	22	Intv	E (CT)	27	H/M	Palm	56.0	5	82.4	VAS; 101-pt; M
Okifuji93	2011	23	Wthn	O	81	R	Palm	30.0	4	80.2	NRS; 7-pt; M
Robinson97	2012	24	Btwn	O	30	CP	Palm	7.0	7	49.6	NRS; 11-pt; M
Weinland141	2011	25	Btwn	O	58	IB	Palm	14.0	7	74.0	OTH; 11-pt; M
Murphy87	2012	26	Wthn	O	44	R	Watch	14.0	5	NR	NRS; 11-pt; M
Viane140 (Study 2)	2004	27	Btwn, Wthn	O	62	CP	Palm	14.0	8	88.0	NRS; 7-pt; M
Crombez28	2013
Roelofs99	2006	28	Btwn, Wthn, Intv	Q-E	79	B/NSP	Palm	14.0	8	72.7	NRS; 7-pt; M
Roelofs98	2004
Huijnen62	2009
Huijnen61	2011
Smith112	2016	29	Wthn	O	120	R	Call	7.0	4	80.8	NRS; 5-pt; NR
Sorbi118	2007	30	Intv	O	5	H/M	Palm	8.5	8	82.7	VAS; NR; M
Peters94	2000	31	Wthn, Met	O	80	CP	Palm	28.0	4	82.9	NRS; 7-pt; M
Sorbi119	2006
Sorbi120	2006
Kop69	2005	32	Wthn	O	38	R	Watch	5.0	5	NR	NRS; 10-pt; NR
Lewis73	1995	33	Met, Intv	O	36	R	Palm	70.0	4	NR	NRS; 7-pt; C
Lewis72	1994
Litt82	2009	34	Intv	E (CT)	54	TMD	Call	21.0	4	71.5	NRS; 7-pt; M
Litt81	2004	35	Wthn	O	30	TMD	Palm	7.0	4	81.0	NRS; 11-pt; M
Murphy88	2010	36	Btwn, Wthn	O	40	R	Watch	5.0	6	NR	NRS; 5-pt; NR
Murphy89	2008
Alsaadi9	2014	37	Wthn	O	80	B/NSP	Paper	7.0	1	NR	NRS; 11-pt; M
Christian24	2015	38	Wthn	O	90	CP	S-Phone	15.0	1	79.3	NRS; 6-pt; M
Houtveen60	2013	39	Wthn	O	87	H/M	S-Phone	42.0	4	89.5	NRS; 3-pt; M
Kikuchi66	2006	40	Wthn, Met	O	40	H/M	Watch	7.0	6	97.0	VAS; 101-pt; M
Kikuchi67	2007
Kikuchi65	2015
Aaron1	2004	41	Wthn, Met, Intv	E (CT)	158	TMD	Palm	56.0	3	98.0	VAS; 11-pt; C
Aaron2	2005
Aaron3	2005
Turner137	2004
Wig143	2004
Turner138	2005
Allen8	2009	42	Wthn	O	157	R	Palm	2.0	7	NR	VAS; 101-pt; NR
Clauw25	2008	43	Intv	E (CT)	1196	R	Palm	28.0	5	NR	VAS; 101-pt; M
Harris56	2005	44	Intv	E (CT)	125	R	Palm	28.0	3.4	NR	OTH; 124-pt; M
Smith114	1993	45	Met	E	30	CP	Palm	7.0	6.1	N/A	VAS; 11-pt; NR
Stone129	2003	46	Wthn, Met	E	68	R	Palm	14.0	12	94.5	VAS; 101-pt; M
Stone130	2004
Litcher-Kelly80	2004
Rasell 96	2007
Stone131	2005
Stone127	1997	47	Wthn, Met	O	35	R	Paper	7.0	7	94.2	NRS; 7-pt; M
Cruise29	1996
Broderick18	2008	48	Wthn, Met	O	106	R	Palm	30.0	7	96.0	VAS; 101-pt; M
Broderick17	2008
Broderick15	2009
Schneider103	2011
Stone128 (Study 1)	2012
Stone126	2010	49	Met	O	128	R	Palm	14.0	9	85.0	VAS; 101-pt; M
Stone134	2003	50	Met	E	80	CP	Palm	21.0	3	52.3	NR
Stone133	2002
Litcher-Kelly78	2007	51	Met	O	16	IB	Palm	21.0	12	79.0	NR
Broderick16	2003	52	Met	Q-E	27	CP	Paper	24.0	3	29.1	NR
Williams145	2004	53	Met	O	14	R	Palm	84.0	6	85.0	NR
Wolf146	2015	54	Wthn	O	220	R	S-Phone	21.0	1	83.0	NRS; 101-pt; C
Zia148	2016	55	Met	O	11	IB	S-Phone	14.0	NR	78.0	VAS; 101-pt; NR
Sterling125	2010	56	Wthn	O	32	Other	Palm	2.0	4	85.0	VRS; 12-pt; M
Lacy71	2013	57	Intv	E (CT)	175	Other	Palm	2.0	10	NR	NRS; 11-pt; M
Evans34	2007	58	Intv	E (CT)	229	Other	Palm	NR	6	90.9	OTH; 13-pt; M
Jamison63	2006	59	Met	O	21	B/NSP	Palm	371.0	1	NR	VAS; 11-pt; M
Eich33	1985	60	Met	O	57	H/M	Paper	NR	24	NR	NRS; 11-pt; M
McLean84	2005	61	Wthn	O	28	R	Call	2.0	5	98.1	NRS; 101-pt; C
Sheftell109	2005	62	Intv	E (CT)	2696	H/M	Palm	NR	4	NR	VRS; 4-pt; NR

Note: NR = Not Reported.

^aBtwn = Between-person study of pain, Wthn = Within-person study of pain, Met = Methodological study of pain measurement, Intv = Intervention to reduce pain.

^bO = Observational, E = Experimental, CT = Clinical Trial, Q-E = Quasi-Experimental.

^cB/NSP = Back/Neck-Shoulder Pain, CP = Chronic Pain (unspecified), R = Rheumatological disorder(s), H/M = Headache/Migraine, TMD = Temporomandibular Disorder, IB = Irritable Bowel Disease/Syndrome.

^dPalm = Palmtop Computer, S-Phone = Smartphone, Call = Telephone Call.

^eNRS = Numeric Rating Scale, VAS = Visual Analogue Scale, VRS = Verbal Rating Scale, OTH = Other, M = Momentary, C = Coverage.

A single data collection project sometimes yields multiple research articles, and this complicates the summary of information in systematic reviews. Careful consideration of this fact is essential in order to avoid biasing results towards datasets that yield a high number of publications^{58, 85}. Information from all publications drawing from the same dataset was therefore aggregated into a single project-based entry (see Table 1), although we recognize the possibility that additional publications stemming from these projects were not identified by the present review. The manner of aggregation employed for each variable is described in greater detail below, and cases in which it was deemed advantageous to consider publications belonging to the same project independently from one another (e.g., in consideration of the reporting of completion rates) are specifically noted.

Sample and design characteristics

Data about project sample sizes were drawn from the abstract or methods section of each publication, and were intended to reflect the number of individuals who engaged in the protocol, rather than the number of individuals with whose data the final analyses were conducted. Tabulated sample size values reflect only patient groups, although studies were additionally classified into those that considered only individuals with chronic pain, those that included individuals with chronic pain and a healthy comparison group, and those that included individuals with chronic pain and other specialized comparison groups.

Information about the specific chronic pain conditions included in the sample was taken from the methods section and subdivided into rheumatological diseases, headache or migraine, back or neck-shoulder pain, temporomandibular disorder, irritable bowel disease or syndrome, other specific chronic pain, and general chronic pain. Based on protocol characteristics, the study design was classified as observational, experimental (clinical trial), experimental (non-clinical trial), or quasi-experimental.

In order to consider research projects rather than individual publications, the highest sample size and the greatest number of groups described across relevant publications were taken to represent the project sample. When both experimental and non-experimental publications stemmed from the same project, the project was considered to be experimental in nature.

Rationale for using EMA methodologies

Research articles were described according to the main reason for their use of EMA methodologies. EMA is often a single component of a multifaceted study, and the rationale for the use of EMA is therefore not necessarily synonymous with the study design presented in a particular article. Articles drawing from the same project were coded separately since the purpose of EMA varies by research question. Four categories emerged: (1) within-person studies of pain, (2) between-person studies of pain, (3) methodological studies of pain measurement, and (4) interventions to reduce pain. Although it was possible for an article to pursue more than one research question, articles were reviewed for their primary purpose and assigned to only one of the four themes. Within each theme, articles were further categorized in terms of the content or constructs examined, into subcategories that varied across themes.

EMA sampling approach

The completion of repeated daily assessments places a burden on study participants, and careful consideration of the specific EMA sampling approach is, therefore, essential. EMA protocols can be broadly described as either event-based or time-based, with the former referring to assessments of concrete episodes or behaviors and the latter including assessments that are scheduled randomly or in regularly spaced or fixed intervals¹¹¹. Random schedules can be administered completely at random or randomized within specific blocks of the day, while fixed schedules occur at pre-defined time points throughout the day. Event-based approaches often rely on the participant to make an entry after the occurrence of a specific event. Each project was coded for (1) the type of sampling schedule, (2) the intensity of assessments (i.e., the number of scheduled or targeted prompts per day), and (3) the total duration of the EMA protocol (i.e., the number of days). Where multiple articles from the same project differed with respect to information regarding sampling approaches, the greatest intensity and longest duration of the EMA protocol as described across publications were taken to represent the corresponding project.

Data input modality

Since the advent of EMA, devices and technologies utilized for data input have changed dramatically. The projects identified by this review were categorized into five groups that roughly trace the evolution of EMA data input modalities: (1) paper booklets, often coupled with signaling devices (e.g., pagers), (2) manually- or digitally-initiated telephone calls, (3) watches or wrist-worn data input devices, (4) palmtop computers, and (5) smartphones. Paper diaries were the earliest modality of EMA data collection; this particular methodology, however, is vulnerable to issues like backfilling, which compromise data quality¹³². To ensure the accurate recording of response times, technologies with automated time-stamping of assessment entries are now being utilized. Although the fourth and fifth groups of input devices share similar functionalities, smartphones possess greater flexibility than palmtop computers and allow for the integration of EMA with passive, unobtrusive sensor technology¹³.

Projects were additionally categorized based on the flexibility to engage with the EMA protocol that was afforded by a particular data input technology. Whether or not convenience features like delaying or suspending prompts were available was noted. In this case and with respect to data input devices, the most complete information from any particular article was taken to represent the corresponding project.

Pain intensity item

Although an early adaptation of EMA in pain research traces back to the 1980s³³, there are no established rules or standards for the construction of an effective EMA pain item. To summarize the field’s current practices, both item characteristics and response scale characteristics were considered.

Although EMA generally uses very short reporting periods, researchers have been taking different approaches to the characterization of “momentary” pain experiences by varying the definition of the reporting period. These approaches were broadly categorized into two models of EMA item design: (1) a momentary model and (2) a coverage model²⁷. The momentary model intends to capture immediate experience by asking about one’s status “right now” or “right before the prompt”. The coverage model intends to capture a brief yet extended period of one’s experience, ranging from “in the last half hour” to “since the last prompt”; by extending the reporting period, this model covers a fuller range of within-day momentary experiences.

Items utilized in the identified studies also vary with respect to the specification of location(s) of pain sensation. Patients’ pain experiences vary depending on chronic pain diagnosis, with some diagnoses involving pain sensation in one location and others involving pain sensation in multiple locations. Depending on the scope of the study, specifying pain location(s) within an item may be essential to avoid criterion contamination in pain ratings (i.e., consideration of pain that is irrelevant to the diagnosis of concern).

The final item characteristic considered was whether researchers assessed pain intensity with a single rating or took a two-stage approach to isolate pain frequency and pain intensity¹⁰². In the latter approach, researchers first ask participants about the presence or absence of their pain, and then, if pain is present, follow up by asking about the intensity of pain. Although “no pain” is often included as a response option in one-stage approach items, Schneider and Stone¹⁰³ have demonstrated the utility of examining frequency and intensity separately.

Projects were additionally classified according to the type of response scale, the number of response scale points, and the wording of response scale anchors. Response scales encountered in this review included numeric rating scales (NRS), verbal rating scales (VRS), visual analogue scales (VAS), as well as visual ordinal and Gracely⁵¹ scales. A standard approach to recall-based pain items commonly administered in clinics is the use of an 11-point scale that ranges from “no pain” to “worst possible pain”³², though whether this practice is common within the EMA pain literature was unclear prior to this review. A related feature with respect to response scales is how each or some of the scale points are verbally anchored. Due to a broad consensus regarding the lowest level of pain (i.e., “no pain”), anchoring descriptors were examined only for the maximum response option. Descriptors often convey both the domain (i.e., pain) and the subjective degree of momentary experiences; this is in line with general recommendations that the sensation to be observed should be defined as part of the response scale ³⁹. Once again, the most complete information from any particular article was taken to represent the corresponding project with respect to the pain intensity item.

EMA completion rates

The conceptualization and presentation of completion rates in research publications varies widely. Completion was defined as the percentage of non-self-initiated, momentary pain intensity item prompts that participants responded to. In some instances, completion rates for pain items could not be disentangled from completion rates for non-pain items, or only completion rates for non-pain items (e.g., fatigue items) were reported. In other instances, completion rates for momentary prompts could not be separated from completion rates for nonmomentary prompts (e.g., end of day diaries). In all of these as well as in related cases, any available data regarding completion rates were taken to reflect the completion rates of non-self-initiated, momentary pain intensity items. If multiple completion rates were presented for different aspects of the protocol (e.g., paper diary completion vs. electronic diary completion), the average of these values was taken.

Publications that reported completion rate data were classified according to five additional, non-mutually-exclusive categories. In line with recommendations from Stone and Shiffman¹³¹, the presentation of completion rates, ranges of completion rates, and/or distributions of completion rates (e.g., by time of day) was noted. Whether completion data were presented only for participants who exceeded a set completion rate threshold and whether information about the handling of missing data (e.g., due to suspend functions or technological issues) was presented was also indicated.

Whether and how completion rates were reported, as described above, was considered on the basis of articles. Actual completion rates of EMA protocols were considered at the project level. If completion rates differed across articles belonging to a project, which was common due to variations in the sample considered, the highest reported completion rate was extracted. To calculate an average completion rate across projects, the completion rates from each project were arcsine transformed and weighted by sample size, and the average of the transformed, weighted values was back-transformed into a percentage value.

Statistical reporting

Data obtained from EMA studies are multilevel in nature¹⁰⁵. When assessing pain intensity on a momentary basis, variance components can be calculated at both within- and between-person levels. These components can be used to compute an intraclass correlation coefficient (ICC), which describes the proportion of the variance that is due to between-person differences, with the remainder of the proportion due to within-person differences and error. Reporting ICCs or multi-level variance estimates is important when examining within-person relationships, as it provides information about the origin of variance. It is also important when aggregating pain scores to examine between-person relationships, in which case it can be considered as a measure of reliability. For each article, whether the ICC or between- or within-person variance estimates were presented was noted. Because statistical reporting was considered on the basis of articles, data were not aggregated by projects.

Results

Sample and design characteristics

A total of 105 articles met the inclusion criteria for the present review. The articles draw from 62 individual research projects. Of the 62 projects, 42 resulted in a single publication, 11 resulted in two publications, and the remaining 9 resulted in three, four, five, six, or seven publications. Few articles were published before the year 2000, and the rate of publication increased sharply in subsequent years (see Figure 2). Forty-six (74.2%) projects were observational studies, 10 (16.1%) were experimental (clinical trials), 4 (6.5%) were experimental (non-clinical trials), and 2 (3.2%) were quasi-experimental studies.

Fig. 2.

Number of studies (n = 105) by publication year

Fifty-two projects (83.9%) considered only chronic pain patients, 7 projects (11.3%) considered chronic pain patients and a healthy comparison group, and 3 projects (4.8%) considered chronic pain patients and an additional specialized comparison group (spouses, participants with asthma, participants with acute pain). Rheumatological diseases were the most common type of chronic pain studied (n = 23, 37.10%), and they included fibromyalgia (n = 11), osteoarthritis (n = 6), rheumatoid arthritis (n = 2), and mixed rheumatological diseases (n = 4). Back or neck-shoulder pain samples were included in 9 projects (14.5%), headache or migraine samples in 8 projects (12.9%), temporomandibular disorder samples in 5 projects (8.1%), irritable bowel disease or syndrome samples in 4 projects (6.5%), and other specific types of chronic pain in 4 projects [sensory neuropathy (1.6%), whiplash-associated disorder (1.6%), coronary heart disease (1.6%), abdominal pain (1.6%)]. Unspecified chronic pain samples were considered in 9 projects (14.5%).

Patient sample sizes ranged from 5 to 2,696 (mean = 123.5, SD = 365.4, median = 45.5). Two of the projects were pharmaceutical clinical trials with sample sizes over 1000. When these were removed, the average sample size was 62.7 (SD = 51.4, range: 5–229, median = 42.0).

Rationale for using EMA methodologies

Articles were categorized according to the main purpose of their use of EMA methodologies. These categorizations are presented in Table 2.

Table 2.

Use of EMA methodology in chronic pain research

	Totala		Subcategoriesb
Purpose	n	%	n	%
Within-person studies of pain	50	47.6%
And emotion			16	15.2%
And coping			8	7.6%
And fatigue/bodily symptoms			7	6.7%
And stress/cortisol			10	9.5%
And physical activity			9	8.6%
And sleep			5	4.8%
And health behavior			3	2.9%
And pain related functioning			6	5.7%
Temporal characteristics of pain			7	6.7%
Between-person studies of pain	15	14.3%
Comparisons with healthy control groups			6	5.7%
Relationships with psychosocial and physical functioning			9	8.6%
Methodological studies of pain measurement	27	25.7%
Acceptability of EMA / completion rates			8	7.6%
Reactivity to EMA			5	4.8%
Reliability of EMA			2	1.9%
EMA used to examine recall bias			16	15.2%
Interventions to reduce pain	13	12.4%
EMA as outcome measure			11	10.5%
Ecological momentary intervention			2	1.9%

Note:

^aAssignments of articles to major categories are mutually exclusive and sum to the total number of articles.

^bAssignments to subcategories are not mutually exclusive.

Within-person studies of pain

Almost half of the articles (n = 50, 47.6%) examined concurrent or sequential (i.e., lagged) within-person relationships between pain intensity and other momentary constructs to gain insight into contexts, predictors, or consequences of momentary changes and fluctuations in pain. A variety of momentary constructs related to pain were examined (see Table 2): emotional states (e.g., anger, depressive mood, positive affect), health behaviors (e.g., medication use, smoking), coping (e.g., state catastrophizing, momentary attention to pain), fatigue, stress, physical activity, sleep (e.g., duration, quality), and pain-related functioning (e.g., activity limitations). Few studies utilized other forms of ambulatory monitoring in addition to EMA, such as physiological (n = 3, 2.8%; cortisol as an indicator of stress) or behavioral/activity-based (n = 6, 5.6%; accelerometry, electromyography) ambulatory assessments. In addition, several studies examined temporal patterns of pain, most notably cyclic changes of pain, throughout the day.

Between-person studies of pain

In about one-seventh (n = 15, 14.3%) of the articles, aggregates of multiple EMA ratings over time were used to examine between-person differences in average pain levels and relationships with other measures of disability. Out of these, the majority of articles examined relationships between patients’ pain and psychosocial (e.g., affective distress, social support) and physical functioning. Other articles reported comparisons of pain intensity and disability levels between specific chronic pain conditions and healthy comparison groups.

Methodological studies of pain measurement

Roughly a quarter (n = 27, 25.7%) of the articles were methodological in nature, and these can be further subdivided into two groups. One group of studies addressed the feasibility and quality of EMA data collection in patients with chronic pain; primary foci of the articles were EMA completion rates (e.g., comparing paper diaries and electronic diaries), reactive arrangements due to repeated momentary sampling, and the reliability of momentary assessments as outcome measures. The second group of studies employed EMA methodology to examine the accuracy of traditional questionnaire assessments and to evaluate biases in recall measures.

Interventions to reduce pain

Intervention studies were relatively infrequent in this review of the literature (n = 13, 12.4%). In the identified intervention studies, EMA was primarily used as an outcome measure after the completion of an intervention. Only 2 studies capitalized on the unique features of EMA (densely repeated measures) to test the effects of short-term momentary intervention strategies (i.e., ecological momentary interventions).

EMA sampling approach

Projects were categorized according to sampling schedule as well as intensity (i.e., number of prompts per day) and duration (i.e., number of days) of assessments. The majority of projects (n = 51, 82.3%) employed a time-based sampling approach consisting of either fixed (n = 23, 37.1%) or random/stratified random (n = 28, 45.2%) assessment schedules. A single project relied exclusively on event-based sampling (n = 1, 1.6%), and the remaining projects utilized a combination approach including both random and event-based assessments (n = 10, 16.0%). Across 58 projects, the median number of EMA assessments per day was 5.0 (mean = 5.9, SD = 3.5, range: 1–24); four projects did not report on the scheduled or targeted intensity of EMA assessments. Across 59 projects, the median duration of EMA sampling was 14.0 days (mean = 23.4 days, SD = 49.4, range: 1- 371), with durations of one week (22.0%) and two weeks (20.3%) being most common; 3 projects did not specify the duration of the EMA protocol.

Data input modality

Over half (n = 34, 54.8%) of the projects described in this review used palmtop computers to capture momentary pain. The remaining projects utilized paper booklets (n = 11, 17.7%; either alone or in combination with a rating in a paper booklet), smartphones (n = 9, 14.5%), watches (n = 5, 8.1%), or phone calls (n = 3, 4.8%). Paper booklets and palmtop computers were utilized approximately equally prior to the year 2000 (see Figure 3). Between 2000 and 2009, the proportion of palmtop computers increased while the proportion of paper booklets decreased. The use of smartphones only appears after 2010, following the emergence of mobile-specific operating systems like Android in 2008 and iOS in 2007; it partially replaces the use of palmtop computers for EMA data collection, although there are unique issues that accompany respondents’ use of their own smartphones for EMA protocols. The use of paper booklets has continued to decrease since 2010; paper-based assessments were infrequently utilized in recent studies.

Fig. 3.

Distribution of data input modalities by decade of publication

Fewer than half of the projects reported on device features that enhance participants’ flexibility to engage in momentary assessments (n = 24, 38.7%). These included nap, suspend, or delay features that participants could utilize during times of the day inconvenient for prompting (n = 17, 27.4%) as well as extended completion timeframes potentially accompanied by reminder prompts (n = 7, 11.3%). It is possible that some of the studies not reporting these features did include them, but simply did not mention them in the article.

Pain intensity item

Characteristics of the items used to assess pain intensity are summarized in Table 3. Of the 62 EMA projects, 15 (24.2%) did not report information regarding pain intensity item reporting period. The majority of the projects that did report this information utilized a momentary (n = 41, 66.1%) rather than a coverage (n = 6, 9.7%) approach. Reporting periods for coverage models ranged from 30 to approximately 180 minutes. About 65% (n = 40) of the projects did not specify pain location, whereas 25.8% (n = 16) asked about pain in specific locations (e.g., jaw, head, abdominal). A couple (n = 2, 3.2%) of the projects combined these two approaches by, for instance, averaging multiple location-specific pain ratings to form overall scores^{29, 126}. Lastly, most projects (n = 53, 85.5%) used a single pain intensity item, and only 4 (6.5%) used a two-stage approach, separating questions about the presence and intensity of pain.

Table 3.

Reporting recommendations for EMA studies of pain intensity

Introduction	Report the following:
Rationale	Rationale for the use of EMA (e.g., to examine within-patient relationships, to ensure ecological validity)
Methods	Report the following:
Data input	Data input device type (e.g., paper booklet, palmtop computer, smartphone) or combination of device types Device model, operating system, and software (if applicable) Data input device features (e.g., delay/suspense function)
Sampling approach	Number of waves of EMA data collection Number of days per wave Targeted number of prompts per day Prompting strategy (event-based, interval-based) Type of schedule (fixed, random, or combination)
Pain intensity item	Precise wording of item/s (including whether a specific bodily area was to be rated) Type of response scale (e.g., NRS, VAS) with anchor descriptors Number of response options Reporting period (e.g., right now, before prompt, past 2 hours)
Results	Report the following:
Completion rates	Attrition rates throughout the study Reasons for missed prompts (e.g., technical failure, noncompliance) Completion rate distributions (mean, median, SD, range) for full sample and analysis sample (if different) Systematic differences in completion rates (e.g., by time of day, by demographic variables)
Statistical reporting	Between-and within-patient variance components or reliability estimates (e.g., intraclass correlations)

With respect to response scales, more than half (n = 34, 54.8%) of the projects utilized an NRS and approximately one quarter (n = 17, 27.4%) used a VAS. The pain rating scales varied widely in their response option labels and in the descriptors used to anchor the highest pain intensity level (see Table 3). Only about half (n = 28, 45.2%) of the projects utilized response scales that included the word “pain” in the anchoring descriptors. Finally, there was substantial variation in the number of response options, which ranged from 3 to 124 points (see Table 3). Most commonly used were 11-point scales (n = 19, 30.6%), 7-point scales (n = 13, 21.0%), and 101-point scales (n = 12, 19.4%).

EMA completion rates

One of the 105 publications described an entirely event-based (i.e., self-initiated) protocol, for which the reporting of completion rates as defined by the authors (i.e., the percentage of non-self-initiated, momentary pain intensity item prompts that participants responded to) is inapplicable. All values noted in this section therefore reflect a full sample of 104 articles. Out of 104 publications, 33 (31.7%) did not report any completion data. The remaining 71 publications were classified according to five non-mutually-exclusive categories. The majority of articles (n = 66, 63.5%) provided mean completion rates (that is, provided either a mean completion rate value, a range of completion rate values for particular variables or combinations of variables, or information that allowed for the calculation of a mean completion rate). Data on completion distributions, e.g., completion rates by time of day or the number of participants who completed a certain number of prompts, were included in 15 articles (14.4%). Information on types of missing data (describing participants’ use of convenience features like suspend or delay functions, the amount of data missing due to technological issues, or how data missing due to the convenience features or technological issues were accounted for in calculation of the completion rate) was included in 8 publications (7.7%). Completion data were presented only for those participants who exceeded a certain completion-related threshold (e.g., those who completed >50% of prompts for >75% of weeks in the protocol⁸) in 7 publications (6.7%). The range (i.e., minimum and maximum) of completion rates was included in 5 articles (4.8%).

Completion data were available for 39 projects (62.9%), and the distribution of completion rates by sample size is shown in Figure 5. The average completion rate across projects was 86.0%; however, there was substantial variation between studies. Mean completion rates of individual projects ranged from 29.1% to 99.0%, and the lowest completion rate was drawn from a project that used an electronic signaling procedure for paper diary data collection in order to verify self-reported completion rates¹⁶. The average of the completion rate values drawn from the other projects utilizing paper diaries was 91.3%. Additionally, as is shown in the figure, completion rates were high when thresholds for minimally acceptable completion were applied to calculate completion rates. Given that the use of paper diaries and the aforementioned thresholds poses a potential risk for biased EMA completion estimates, we examined the observed mean completion rate across projects when these studies were excluded from the analysis. This strategy yielded an average completion rate of 84.1%.

Fig. 5.

Funnel plot of mean EMA completion rates Studies applying a threshold for minimally acceptable completion before calculation of completion rates are shown as squares, paper diaries are shown as diamonds

Statistical reporting

There were 10 articles (9.5%) that reported ICCs or multilevel variance estimates for the pain variable. This may be a conservative estimate of this type of statistical reporting, however, because some articles were not intended to demonstrate relationships between variables (e.g., feasibility studies) or had different outcomes of interest (e.g., fatigue). In these cases, reporting ICCs or multilevel variance estimates for pain intensity may not be essential.

Discussion

The volume of studies identified by the present review suggests that EMA methodologies have been accepted as a sound approach to data collection in chronic pain research. The heterogeneity of this type of research is also evident, and the studies considered here vary widely in various aspects of their design. EMA methodologies are complex, and a multitude of decisions is required for their implementation. We highlight some essential decisions involved in conducting EMA studies and focus on the importance of reporting them (see Table 4 for a summary of recommended reporting practices for EMA study decisions in chronic pain research). Ambiguities regarding the consequences of decision-making that would benefit from further investigation are also noted.

Rationale for using EMA methodologies

EMA methodologies are uniquely suited to examine temporal relationships between pain and affective, cognitive, and behavioral factors. The reviewed studies reflect this notion in that nearly half of the articles examined concurrent or lagged associations between pain and other momentary constructs. While this indicates that EMA is well accepted in observational research, EMA has been very infrequently used to construct outcome measures in chronic pain intervention research, with less than one-eighth of reviewed articles falling into this category. This is surprising in light of the potential of EMA to provide ecologically valid estimates of treatment effects and assay sensitivity in clinical trials. We recommend the use of EMA for outcome measures in clinical trials to measure changes in actual pain experiences that are not confounded by changes in beliefs about pain and symptom recollections. EMA also provides the ability to construct new pain intensity metrics, including pain variability, the proportion of time spent in high or low pain, and the frequency of acute shifts in pain that may uniquely enhance understanding of clinical trial outcomes¹²⁷. In addition, ecological momentary interventions, which afford real-time treatment in patients’ natural contexts^{38, 57, 70}, offer immense potential for optimizing treatment benefits⁵⁷. Future research should continue to explore these directions in the assessment and treatment of chronic pain.

EMA sampling approach

The intensive nature of momentary sampling, which offers a fine-grained and nuanced perspective on the assessed constructs, places a burden on study participants. EMA research has often utilized a schedule of three to five assessments per day to minimize burden, and the present review is in line with this; the median number of EMA prompts across projects was 5.0 per day and the median duration of EMA sampling was 14 days. The particular sampling density typically depends on the research question^{111, 116}; while relatively few assessments may suffice when the goal is to examine group differences in average pain levels, many assessments per day are typically required to adequately capture dynamic within-person processes. We expect that future research will continue to balance the duration and intensity of EMA sampling with regard to participant burden and saturation of the constructs under study, but an accepted calculus for achieving this balance does not yet exist.

Data input modality

This review observed a shift in the data input mode of EMA sampling over time: from paper diaries to palmtop computers to smartphones. Notably, completion rates in our sample were over 7% higher in studies that did not use electronic timestamping than in those that did use electronic timestamping and did not employ thresholds for calculating completion. We call for a moratorium of the use of paper diaries given the well-known problems (e.g., back-filling, forward-filling) that can undermine the validity of assessments. Some research protocols utilizing smartphones allow participants the option of bringing their own device into the study, but it is unclear whether this feature is associated with increased or decreased participant engagement. Less than half of the reviewed projects reported on convenience features that allowed for flexibility in the completion of EMA prompts, including nap and delay options. While these features may be advantageous for keeping participants engaged in repeated assessments, they also complicate interpretations of the intended approach to data capture (particularly for random sampling approaches) and the calculations of participant completion of the study protocol⁷⁸. New technologies involving passive sensors (e.g., GPS) could allay these concerns and allow for the optimization of EMA assessment time points^{122, 123}.

Pain intensity item

Limited attention has been paid to the psychometric properties of EMA pain ratings and the characteristics of EMA pain rating scales. In fact, there was substantial variation between identified studies with respect to the type of rating scale (NRS, VRS, VAS, Other), the number of scale points (ranging from 3 to 124), the labels used to anchor the response scale, the joint assessment of pain frequency and intensity (one- versus two-stage approach), and the length of the reporting period (momentary versus coverage models), with little or no rationale provided for the specific choices made for each study. The need for standardized and normed instruments is increasingly acknowledged in many areas of patient-reported outcomes research^{12, 23, 32}, including in the measurement of pain in clinical trials^{79, 114}, and these issues are similarly important in EMA research. We recommend that EMA pain intensity items clearly indicate the reporting period to the respondent (e.g., current pain, pain before the prompt), specify any pain location of interest, and that the item be clearly worded to ensure that participants exactly understand what is being rated. Additionally, scales with less than five response options should be avoided in that they likely yield poor psychometric properties⁹⁵.

Even ostensibly minor differences in the decisions regarding EMA items are likely to have non-trivial consequences. For example, varying the wording of pain scale anchors and the number of scale points has been shown to affect response distributions and participants’ interpretations of the rating scale^{30, 95, 106, 107, 109}. In addition, asking participants to rate their pain in the present moment or over the past several hours potentially evokes different mental processes; while momentary ratings presumably involve direct introspection, the use of even short recall periods is likely to capture pain experiences from episodic memory, thereby potentially introducing episodic memory biases such as peak-and-end effects^{26, 104}. However, it could also be argued that self-report covering several hours provides a more reliable depiction of pain states compared to the “spot-checks” involved in momentary pain reports¹³⁴. In either case, it is important to note that the appropriateness of a particular recall period is also driven by the clinical and theoretical needs of the study in question.

Given that precise and accurate assessment is foundational to EMA methodology, we believe that it is important for future research to identify exactly how EMA items should be designed in order to best reflect the reality of patients’ pain experiences. This includes empirical examination and identification of item characteristics that impact sensitivity to the detection of momentary fluctuations and longer-term changes in pain; convergent and predictive validity of the items with respect to other relevant phenomena should also be established. Further, potential adaptations for older and cognitively impaired patients might be considered²². At present, empirical data on which to base further recommendations for EMA item design and administration are scarce. Lacking these data, a consensus among researchers with respect to standardization would serve as a foundation that would increase comparability across studies

EMA completion rates

The success of EMA protocols hinges on high participant completion of repeated momentary assessments. Because data are usually missing systematically (e.g., response rates might be lower on weekends), low completion rates affect the representativeness of the data and limit the validity and generalizability of study findings^{118, 141}. Recognizing the importance of participant completion in EMA designs, Stone and Shiffman¹³¹ emphasized that completion data should be reported in detail. However, in nearly one-third of the studies considered in this review, completion rates were not reported at all. Even when completion data were reported, publications rarely went into detail about the range and distributions of completion rates.

The average completion rate across projects that reported completion data was 86.0%, indicating that participants are willing to take on the burden of frequent assessment necessitated by EMA. Excluding studies with paper diaries and those that implemented a threshold for minimally acceptable completion slightly reduced the completion rate to 84.1%. These values are comparable to the 83% completion rate documented in other reviews of the chronic pain literature⁸⁵ and in other research areas^{74, 141}. However, it is important to note the possibility that these reviews as well as the present undertaking were impacted by publication bias, and that completion rates may therefore be generally overestimated. Since a large proportion of non-significant findings generally remain in a file drawer¹⁰⁰, unpublished, it is well possible that studies with low completion rates also remain unpublished.

In addition to under-reporting and publication bias, thresholds are further consideration likely to influence completion rates. Although thresholds were indicated in only 6.7% of articles considered in this review, it is possible that other studies used thresholds without explicitly stating this in the article. We recommend that researchers wishing to implement thresholds in their studies calculate and present completion rates for the full study sample and those participants that were included in the analyses.

Statistical reporting

Multilevel variance estimates, or ICCs, provide essential information about the reliability of EMA pain ratings as well as about statistical power¹⁴. Without considering these values, for example, it is possible to create an elaborate model to explain momentary pain without recognizing that the within-person variance is minimal, and that a larger portion of the variance actually stems from between-person differences. This would compromise the utility of the tested model. Nevertheless, multilevel variance estimates were infrequently presented in the studies identified by this review. The reporting of multilevel variance estimates should be customary in EMA research to document the adequacy of statistical model assumptions and to facilitate power analyses for the design of future studies.

Conclusion

The present endeavor constitutes the first systematic review to describe EMA methodologies in chronic pain research. It is constrained by limitations common to systematic reviews, namely that it is possible that articles relevant to the review were not identified and therefore excluded, and that other relevant studies were never published and therefore also excluded. Despite these limitations, this paper provides a thorough overview of the current state of chronic pain literature with respect to EMA methodologies, illustrating the importance of reporting protocol design decisions.

The field of EMA in chronic pain research is still young and relatively disaggregated, and the choices made by investigators when designing a particular study are not always clear. Thorough descriptions of these decisions and the rationale that underlies them would inform reproducibility, comparability across studies, and the interpretation of study results, and guide future researchers in similar decision-making situations. Consideration of variability in these design decisions naturally begs the question of whether or not the decisions, some of them seemingly trivial, actually come to bear on study results. In and of itself, this provides vast opportunities for future research to determine which design aspects are critical and whether studies that employed different strategies can nevertheless be considered comparable.

Fig. 4.

Characteristics of momentary response scales NR=Not reported

Highlights.

• ◦105 articles from 62 research projects were identified and included
• ◦Protocol design features of reviewed studies varied widely and were not consistently reported
• ◦Careful selection of design features and thorough reporting are essential to EMA

Perspective.

Studies that utilize Ecological Momentary Assessment methodologies to assess pain intensity are heterogeneous. Aspects of protocol design, including data input modality and pain item construction, have the potential to influence the data collected. Thorough reporting on design features and completion rates therefore facilitates reproducibility, comparability, and interpretation of study results.