Abstract
Objective:
To evaluate SLE patients who persistently frequent the ED to identify opportunities to improve outpatient care.
Methods:
We conducted a retrospective study of SLE patients who frequented the ED for ≥3 visits in a calendar year from 2013–2016. Persistent users met criteria for frequent use for at least 2 out of the 4 years, and limited users for 1 out of the 4 years. Each ED encounter was categorized as; SLE-, infection-, pain- related, or “other”. We compared ED use between persistent and limited users, and analyzed factors associated with pain-related encounters among persistent users through multivariate logistic regression.
Results:
We identified 77 participants having 1143 encounters as persistent users, and 52 participants having 335 encounters as limited users. Persistent users accounted for 77% of ED use by SLE patients who frequented the ED. Pain-related ED visits were more common among persistent users (32%) than limited users (18%). Among persistent users, most pain-related encounters were discharged from the ED (69%) or within 48 hours of admission (20%). Persistent users with pain-related encounters accounting for >10% of ED use, were more likely to be obese, have fewer comorbid conditions, and be on LTOT.
Conclusion:
Pain is major cause of ED use. SLE patients persistently utilizing the ED for pain are likely to be non-critically ill, as evidenced by frequent discharges from the ED and short stay admissions. SLE patients who persistently frequent the ED for pain represent a viable target for interventions to improve outpatient quality of care.
Systemic lupus erythematosus (SLE) is a chronic inflammatory autoimmune disorder associated with substantial socioeconomic burden and healthcare resource utilization. Inpatient care accounts for the largest component of direct costs (1–4). However, SLE patients have more ED visits than hospital admissions, with 40–70% of SLE patients having at least one ED visit in a year (1–6). In addition, hospitalizations are increasingly originating from the ED (7), and approximately 20% of admissions of SLE patients are avoidable (8). For these reasons, understanding ED utilization among persons with SLE may provide insight into drivers of both healthcare resource utilization and poor quality of care for SLE in the outpatient setting.
As in the general population, SLE patients with low socio-demographic status, lower education level, and poor adherence more frequently utilize the ED, and account for the majority of all ED visits (9). The definition of frequent ED use is variable but, frequent ED users generally account for 4.5 to 8% of all ED patients and 21 to 28% of all ED visits (10). Frequent ED use has been generally thought to arise from difficulty in access to primary or specialty care (11, 12). However, studies show that most frequent ED users have insurance coverage and are more likely to utilize all existing forms of healthcare resources including outpatient care (9, 10, 13–15). In addition, there is evidence to suggest that the use of ED, for most people, is an affirmative choice over other sources of healthcare rather than a last resort (16).
It is increasingly recognized that frequent ED users are not a homogenous population (10, 17, 18). In the general population, studies have demonstrated that most individuals cease to qualify as frequent ED users within a year (17, 19). This brief period of frequent ED use may be due to an acute event requiring multiple ED visits, pregnancy related complications, or flare of a chronic disease. In contrast, a small but consistent percentage persistently frequent the ED over years (17, 19). Causes, and therefore interventions, for this subgroup of patients are likely to be different than for those with a brief period of frequent ED use. Understanding the factors underlying persistently frequent ED use may help inform interventions to improve chronic disease management and care co-ordination in the outpatient setting.
In this study, we sought to identify SLE patients who persistently frequented the ED over four years. We examined the characteristics and patterns of ED utilization at the individual patient- and encounter-level. Our research aimed to answer the following questions: What are the demographic and disease characteristics of SLE patients who persistently frequent the ED? How do persistently frequent users compare to those with limited frequent ED use? Is persistently frequent ED use associated with certain co-morbidities, in particular, chronic pain?
Methods
Subjects
We performed an electronic health record (EHR) based query in EPIC for a cohort where International Classification of Diseases (ICD), 10th edition code of M32 for SLE was entered at least once in either the problem list, encounter diagnosis or as a billing code during the study period between 1/1/2013 and 12/1/2016, and met criteria for persistently frequent ED use. Persistently frequent ED use was defined as having more than three ED visits during the 12 months in a calendar year, similar to previous studies on ED utilization among persons with SLE (9), for at least two out of the four years during the study period, consecutive or non-consecutive, between 2013 and 2016, at a large urban tertiary medical center.
We then verified the diagnosis of SLE through in-depth EHR review. Only those who met American College of Rheumatology criteria for SLE (20), or had SLE documented by a rheumatologist, nephrologist, or dermatologist, or were on active immunosuppressive therapy treatment for no other medical condition, were determined to have a verified diagnosis of SLE and included in the study.
For those diagnosed with SLE during the study period, we reviewed and censored ED encounters preceding diagnosis unless diagnosis of SLE was probable at the time of visit based on physician documentation and/or serologic work-up. We then re-evaluated the number of ED encounters for these newly diagnosed SLE patients to ensure they still met criteria for persistently frequent ED use after removal of censored visits. In instances of patient death prior to close of the study period, we reviewed the number of ED visits from study inception to time of death to ensure fulfillment of criteria for persistently frequent ED use.
To understand the comparative magnitude and pattern of ED utilization among persistently frequent users, we performed a second EHR-based query and applied the same criteria to verify diagnosis of SLE and number of ED encounters to identify SLE patients who had limited frequent ED use. Limited use was defined as meeting criteria for frequent ED use for 1 out of the 4 years during the study period.
The study protocol was approved by the Human Research Protection Program at our Institution.
Data
We collected patient- and encounter-level data through retrospective in-depth physician review of the EHR using a standardized data abstraction template.
Patient-Level Measures
We collected demographic information including age, gender, and race/ethnicity. Zip code information was collected to calculate the Area of Deprivation Index (ADI) (21). The ADI is a geographic area-based measure of socioeconomic deprivation (22). It combines 17 different indicators of SES, including level of education, income, employment, value of assets, and poverty level derived from decennial census data. Higher ADI values represent greater deprivation. We also queried the EHR for primary insurance coverage at time of enrollment, and categorized type of insurance as Medicaid, Medicare, or private/commercial.
We collected information on SLE history, including manifestations, disease duration, and organ involvement prior to the index encounter through in-depth retrospective EHR review. For those with lupus nephritis (LN), we reviewed treatment history, and/or active renal replacement therapy through either hemodialysis or peritoneal dialysis, and/or transplant status. Information related to lupus disease activity at time of ED encounter was not consistently available in the EHR. We also collected medication history, including exposure to glucocorticoids (GC), hydroxychloroquine (HCQ), and/or additional disease modifying anti-rheumatic drugs (DMARDs) such as azathioprine, methotrexate, and mycophenolate mofetil. We categorized LTOT as having prescription for daily or near-daily use of opioids for at least 90 days, or total days of opioid supply for more than 120 days (23, 24). We also collected information on relevant medical comorbidities including depression.
Encounter-Level Measures
We classified disposition of each encounter as discharged from the ED or admitted to the hospital. For encounters resulting in ED-initiated admission, we obtained information on initial admission floor status (i.e. observation, medical/surgical floor, step down unit (SDU), intensive care unit (ICU)) and length of stay in the hospital (number of days). We categorized ED-initiated admissions without a claims code for ED critical care, not admitted to the SDU/ICU, and discharged within 48 hours as potentially avoidable short stay (PASS) admissions.
We categorized each ED encounter as: 1) SLE-related, 2) infection-related, 3) pain-related or 4) “other”. This categorization was applied after discharge, either from the ED or after ED-initiated admission. We classified encounters into one of these four groups based on the principle discharge diagnosis supported by physician documentation and diagnostic evaluation results. An encounter was classified as being SLE-related, if a patient presented with a SLE flare or SLE-related disease activity, and/or was prescribed GC, HCQ or other DMARD during the encounter by a rheumatologist, nephrologist or dermatologist. An encounter was classified as infection-related, if a patient had positive culture, or imaging diagnostic of infection, and received antibiotics in either the ED or on discharge. An encounter was classified as pain-related if the primary discharge diagnosis was for pain not attributable to SLE, trauma, or without a specific etiology or organic cause based on unremarkable diagnostic evaluation (e.g. no changes in electrocardiogram, no elevation in troponin, no abnormal imaging), and without indication for invasive or surgical intervention. By study definition, categories of SLE- and pain-related encounters were mutually exclusive. However, an encounter could be infection-related and SLE- or pain-related. For those few cases (n= 8), the encounter was classified according to the principle discharge diagnosis. Encounters that were neither SLE-, infection-, or pain-related were classified as “other” (described in greater detail in Appendix 1, 2).
Analyses
Demographic, and disease characteristics were described using means, standard deviations (SDs), and proportions, as appropriate. We compared the distribution of encounters by category group at discharge from either the ED or after ED-initiated admission. In addition, for ED encounters that led to admission, we analyzed the length of stay and initial admission floor status to identify PASS admissions.
We also compared sociodemographic and disease characteristics between SLE patients who persistently frequented to ED during the study period to those who had limited frequent ED use using t-test for continuous measures and Chi-square or Fisher’s exact for categorical measures. Variables with p-value <0.1 or with clinical significance were then included in a multivariate logistic regression model. The same analytic approach was conducted to assess factors related with higher propensity to utilize the ED for pain-related encounters among SLE patients who persistently frequent the ED. We compared a group of persistent users who had pain-related encounters accounting for >10% of their total ED use to those for whom pain-related encounters constituted ≤10% of ED use. Data were analyzed using Stata/IC 14.2 (Stata Corp).
Results
Subject Characteristics
We initially identified 187 participants with possible SLE who met criteria for persistently frequent ED use and 132 that met criteria for limited frequent ED use from 2013–2016 through EHR query. After in-depth retrospective EHR review to verify diagnosis of SLE and censor ED encounters for date of SLE diagnosis and death, 77 and 52 SLE participants met all inclusion criteria for persistently and limited frequent ED use, respectively, during the study period (described in greater detail in Appendix 3).
Overall (N=129), most of the participants were young African American female (n=77, 59.7%) with mean age 41.5 (SD 15.6). All had some form of insurance with most having Medicaid or Medicare as their primary coverage (n=106, 82.2%). ADI was higher compared to the region (mean 87.3, SD 26.7), reflecting higher neighborhood socioeconomic deprivation. Most were on GC (74.4%) and/or some form of DMARD (89.1%) during the study period.
Characteristics for persistent and limited users are presented in Table 1. Approximately one in three persistent users (31.2%) and one in five limited users (19.2%) had diagnosis of depression. LTOT was nearly three times more prevalent among persistent users (37.7%) than limited users (13.5%). More persistent users had renal involvement on dialyses (19.5%) compared to limited users (5.8%).
Table 1.
Demographics and disease characteristics of SLE patients who were limited and persistently frequent ED users between 2013 and 2016
| Variables | Persistently Frequent ED use (n=77) | Limited Frequent ED use (n=52) | p-value |
|---|---|---|---|
| Demographics | |||
| Age, mean years (SD) | 42.3 (15.4) | 40.3 (15.9) | 0.482 |
| Female, n (%) | 70 (90.9) | 46 (88.5) | 0.205 |
| Race, n (%) | 0.026 | ||
| Caucasian | 11 (14.3) | 16 (30.8) | |
| African American | 53 (68.8) | 24 (46.1) | |
| Hispanic/Latino | 13 (16.9) | 12 (23.1) | |
| Insurance, n (%) | <0.01 | ||
| Medicaid | 38 (49.3) | 37 (71.1) | |
| Medicare | 28 (36.4) | 3 (5.8) | |
| Private/Commercial | 11 (14.3) | 12 (23.1) | |
| Area of Deprivation Index, mean (SD)† | 105.6 (11.7) | 102.6 (10.6) | 0.143 |
| Co-morbidities | |||
| Psychiatric diagnosis, n (%) | 27 (35.1) | 17 (32.7) | 0.077 |
| Depression, n (%) | 24 (88.9) | 10 (58.8) | 0.131 |
| Hypertension, n (%) | 48 (62.3) | 30 (57.7) | 0.597 |
| Hyperlipidemia, n (%) | 14 (18.2) | 18 (34.6) | 0.034 |
| Diabetes, n (%) | 18 (23.4) | 10 (19.2) | 0.575 |
| Coronary Artery Disease, n (%) | 8 (10.4) | 5 (9.6) | 0.886 |
| Cardiovascular Accident, n (%) | 7 (9.1) | 7 (13.5) | 0.434 |
| Congestive Heart Failure, n (%) | 11 (14.3) | 5 (9.6) | 0.430 |
| Asthma, n (%) | 15 (19.5) | 9 (17.3) | 0.756 |
| Chronic Obstructive Pulmonary Disease, n (%) | 2 (2.6) | 3 (5.8) | 0.360 |
| No. of co-morbidities, mean (SD) | 1.6 (1.5) | 1.7 (1.3) | 0.614 |
| Body Mass Index, mean (SD) | 30.4 (9.8) | 29.5 (7.4) | 0.584 |
| Long-term opioid therapy, n (%) | 29 (37.7) | 7 (13.5) | 0.003 |
| SLE Disease Characteristics, n (%) | |||
| Disease duration ≥ 10 years | 29 (38.7) | 18 (34.6) | 0.642 |
| Renal Involvement | 34 (44.2) | 18 (34.6) | 0.279 |
| Lupus Nephritis on Dialyses | 15 (44.1) | 3 (16.7) | 0.027 |
| Lupus Nephritis with Transplant | 7 (20.5) | 3 (16.7) | 0.489 |
| Lung Involvement | 15 (19.5) | 16 (30.8) | 0.141 |
| Pericarditis | 14 (18.2) | 13 (25.0) | 0.350 |
| Medication Use | |||
| None, n (%) | 6 (7.8) | 1 (1.9) | 0.149 |
| Glucocorticoids, n (%) | 55 (71.4) | 41 (78.8) | 0.344 |
| Hydroxychloroquine, n (%) | 59 (76.6) | 45 (86.5) | 0.162 |
| Other DMARD*, n (%) | 39 (50.6) | 34 (65.4) | 0.098 |
| Azathioprine | 15 (38.5) | 16 (47.0) | 0.141 |
| Methotrexat | 8 (20.5) | 10 (29.4) | 0.155 |
| Myocophenolate mofetil | 24 (61.5) | 15 (44.1) | 0.778 |
sum of number of participants on Azathioprine, Methotrexate and MMF exceed number of participants on other DMARD, as some were concomitantly on more than one DMARD; SD=standard deviation, No. of co-morbidities=number of co-morbidities, Other DMARD=disease modifying antirheumatic drug (exclude HCQ)
In multivariate analysis, SLE patients who persistently frequented the ED were more likely to be African American, have Medicare as their primary insurance coverage, be on dialysis, and be on LTOT, compared to those with limited frequent ED use (Table 2).
Table 2.
Factors associated with persistently frequent ED use compared to limited frequent ED use
| Variable | OR (95% CI) | p-value |
|---|---|---|
| Age | 0.99 (0.96–1.03) | 0.966 |
| Female | 1.67 (0.40–7.03) | 0.482 |
| Race | ||
| Caucasian | Reference | - |
| African American | 5.24.(1.63–16.84) | 0.005† |
| Hispanic/Latino | 2.12 (0.52–8.68) | 0.295 |
| Insurance | ||
| Medicaid | Reference | - |
| Medicare | 15.77 (3.8–73.65) | <0.001† |
| Private/Commercial | 1.71 (0.57–5.15) | 0.342 |
| Number of co-morbidities | 0.70 (0.48–1.01) | 0.061 |
| Depression | 1.97 (0.66–5.82) | 0.222 |
| Long-term opioid therapy | 3.09 (1.02–9.38) | 0.046† |
| Renal Involvement on Dialysis | 5.03 (1.06–23.84) | 0.042† |
| Other DMARD | 0.44 (0.18–1.08) | 0.075 |
Other DMARD=disease modifying antirheumatic drug (exclude HCQ)
statistically significant with p-value<0.05
ED Encounters in Persistent vs Limited Frequent Users
The 77 SLE patients who persistently frequented the ED had 1143 ED encounters and the 52 patients with limited frequent ED use had 335 ED encounters. Persistent users had more than twice the average number of ED encounters (mean 14.8, SD 8.8) compared to limited users (mean 6.4, SD 2.0) during the study period (p<0.001). SLE patients who persistently frequented the ED had more encounters that led to ED-initiated admission (48.6%) than limited users (39.7%) (p=0.004). More encounters were pain-related (32.4%) among persistent users compared to limited users (18.2%) (p<0.001). On average, persistent users had 4.8 pain-related encounters (SD 6.1) and limited users had 1.2 pain–related encounters (SD 1.4) during the study period (p<0.001). One in four persistent users (26%) had more than five pain-related encounters, whereas one single limited user (1.9%) had more than five pain-related encounters from 2013–2016 (p<0.001). Infection-related (12.9%) and SLE-related (6.7%) encounters were less common among persistently frequent users compared to limited users (15.5% and 10.5%, respectively). “Other” encounters accounted for the majority of ED use for persistent (48.0%) and limited (56%) users.
ED Utilization among SLE patients who Persistently Frequent the ED
The 77 persistently frequent ED users accounted for 77% of all ED use by SLE patients who had 3 or more ED visits in a calendar year between 2013–2016. Of the 1143 encounters incurred by SLE patients who persistently frequented the ED, 588 (51.4%) resulted in discharge from the ED and 555 (48.6%) led to ED-initiated admissions. A substantial portion of encounters resulting in discharge from the ED were pain-related (43.7%), some were infection-related (10.4%) and few were SLE-related (1.4%) (Figure 1). The eight encounters categorized as SLE-related on discharge from the ED involved evaluation either by a rheumatologist or a nephrologist during the ED course, and had documentation to support findings of SLE-related activity/complications in the EHR. Among encounters resulting in ED-initiated admission, 20.4% were pain-related, 15.5% were SLE-related and 12.4% were infection-related.
Figure 1.
Proportion of ED encounters in each encounter category group at discharge from either the ED or after ED-initiated admission among SLE patients who persistently frequented the ED from 2013–2016
Among encounters that led to ED-initiated admission, the majority (65.5%) of pain-related encounters resulted in admission with discharge within 48 hours and were significantly more likely than any other encounter category group to meet criteria for PASS admissions (p<0.001). Infection-related encounters were least likely to lead to admission with discharge within 48 hours (19.8%), and were more often initially admitted to the SDU/ICU (12.8%). In comparison, 43.5% of SLE-related encounters resulted in admissions with discharge within 48 hours. Among the 56.5% of SLE-related encounters resulting in ED-initiated admissions with a length of stay longer than 48 hours, 10.3% were initially to the SDU/ICU.
The number of participants having at least one ED visit related to each encounter category group varied. Thirty-two (41.6%) participants had one or more SLE-related encounters, and 55 (71.4%) and 61 (79.2%) participants had at least one infection- and pain-related encounter, respectively. All patients had at least one ED encounter classified as “other”.
Patient Characteristics Associated with Pain-related Encounters among SLE patients who Persistently Frequent the ED
We observed a high burden of pain among SLE patients who persistently frequented the ED, with 50.7% of encounters coding pain as the chief complaint at initiation of the ED encounter. Pain was the presenting symptom for 51 (66.2%) of SLE-related, 38 (25.8%) of infection-related and 171 (31.1%) of “other” encounters. Of the 580 encounters with pain symptoms reported at presentation, 320 (55.2%) were categorized as pain-related encounters upon discharge. These pain-related encounters, as aforementioned, accounted for a third of ED use by SLE patients who persistently frequented the ED, representing 61 (79.2%) participants. We observed a wide range in the frequency of pain-related encounters among participants with at least one pain-related encounter. One participant had a single pain-related encounter, whereas another had 31 pain-related encounters during the study period.
To understand factors associated with higher propensity to utilize the ED for pain, we compared characteristics of participants who had pain-related encounters accounting for >10% of their total ED use to those with pain-related encounters accounting for ≤10% of their total ED use (Table 3). Participants with higher propensity to persistently frequent the ED for pain-related encounters were younger (p=0.028), more likely to be African American (p=0.001), and come from more socioeconomically deprived neighborhoods (p=0.016). No difference in the prevalence of depression was observed, although, LTOT was more common in this group (p=0.040). In addition, participants with >10% pain-related encounters had fewer comorbid conditions (p=0.019) and were more likely to use DMARDs other than HCQ (p=0.041). In multivariate analysis, African Americans, fewer comorbid conditions, LTOT, and higher BMI were associated with higher propensity to utilize the ED for pain (Table 4).
Table 3.
Comparison of patient characteristics with various degrees of pain-related encounters among SLE patients who persistently frequented the ED
| Variables | Percentage of Pain-related ED Encounters |
||
|---|---|---|---|
| ≤10% | >10% | p-value | |
| Number of participants | 24 | 53 | |
| Percentage of participants, % | 31.2 | 68.8 | |
| Demographics | |||
| Age, mean years (SD) | 48.0 (18.4) | 39.7 (13.2) | 0.028 |
| Female, n (%) | 22 (91.7) | 48 (90.6) | 0.623 |
| Race, n (%) | 0.001 | ||
| Caucasian | 5 (20.8) | 6 (11.3) | |
| African American | 10 (41.7) | 43 (81.1) | |
| Hispanic/Latino | 9 (37.5) | 4 (7.6) | |
| Insurance, n (%) | 0.892 | ||
| Medicaid | 12 (50.0) | 26 (49.1) | |
| Medicare | 8 (33.3) | 20 (37.7) | |
| Private/Commercial | 4 (16.7) | 7 (13.2) | |
| Area of Deprivation Index, mean (SD) | 100.8 (10.7) | 107.7 (11.6) | 0.016 |
| Co-morbidities | |||
| Hypertension, n (%) | 17 (70.8) | 31 (58.5) | 0.218 |
| Hyperlipidemia, n (%) | 8 (33.3) | 6 (11.3) | 0.028 |
| Diabetes, n (%) | 9 (37.5) | 9 (17.0) | 0.049 |
| Coronary Artery Disease, n (%) | 4 (16.7) | 4 (7.5) | 0.205 |
| Congestive Heart Failure, n (%) | 8 (33.3) | 3 (5.7) | 0.003 |
| Cerebrovascular Accident, n (%) | 4 (16.7) | 3 (5.7) | 0.131 |
| Chronic Obstructive Pulmonary Disease, n (%) | 2 (8.3) | 0 (0.0) | 0.094 |
| Asthma, n (%) | 4 (16.7) | 11 (20.7) | 0.467 |
| No. of co-morbidities, mean (SD) | 2.3 (2.0) | 1.3 (1.1) | 0.019 |
| Depression, n (%) | 10 (41.7) | 14 (26.4) | 0.181 |
| Long-term opioid therapy, n (%) | 5 (20.8) | 24 (45.3) | 0.040 |
| Body Mass Index, mean (SD) | 27.3 (6.9) | 31.8 (10.6) | 0.058 |
| SLE Disease Characteristics, n (%) | |||
| Disease Duration ≥ 10years | 7 (50.0) | 23 (54.8) | 0.757 |
| Renal Involvement | 13 (54.2) | 21 (39.6) | 0.234 |
| Lupus Nephritis on Dialyses | 9 (69.2) | 6 (28.6) | 0.024 |
| Lupus Nephritis with Transplant | 1 (7.7) | 5 (23.8) | 0.237 |
| Medication Use, n (%) | |||
| Glucocorticoids | 16 (66.7) | 39 (73.6) | 0.534 |
| Hydroxychloroquine | 16 (66.7) | 43 (81.1) | 0.165 |
| Other DMARD | 8 (33.3) | 31 (58.5) | 0.041 |
SD=standard deviation, No. of co-morbidities=number of co-morbidities, Other DMARD=disease modifying antirheumatic drug (other than HCQ)
Table 4.
Patient characteristics associated with higher propensity to utilize the ED for pain among SLE patients who persistently frequented the ED
| Variables | OR (95% CI) | p-value |
|---|---|---|
| Age | 1.00 (0.95–1.06) | 0.870 |
| Race | ||
| African American | Reference | - |
| Caucasian | 0.25 (0.02–3.09) | 0.283 |
| Hispanic | 0.02 (0.00–0.17) | <0.001 |
| Area of Deprivation Index | 1.05 (0.98–1.12) | 0.201 |
| No. of co-morbidities | 0.54 (0.33–0.89) | 0.015 |
| Long-term opioid therapy | 7.50 (1.19–47.43) | 0.032 |
| Body Mass Index Other DMARD use |
1.12 (1.01–1.24) 2.55 (0.50–12.97) |
0.034 0.258 |
OR-=odds ratio, 95% CI=95% confidence interval, No. of co-morbidities=number of co-morbidities, Other DMARD=disease modifying antirheumatic drug (other than HCQ)
Characteristics of SLE patients who Persistently Frequent the ED with Pain-Related PASS Admissions
One in five hospitalized encounters were pain-related upon discharge from ED-initiated admission among SLE patients who persistently frequented the ED, of which 74 (65.6%) encounters met criteria for PASS admissions. Table 5 describes the 25 (32.5%) participants who accounted for the 74 pain-related PASS admissions. All except one were female. The mean age was 38.4 years (SD 13.8); 18 (72%) were African American, 5 (20%) were Caucasian, and 2 (8%) were Hispanic. All had some form of insurance; 24 (96%) had public insurance, either Medicaid or Medicare, and only one subject (4%) had private/commercial insurance as their primary insurance. Of the 25 persistent users with pain-related PASS admissions, 13 (52.0%) were on LTOT. Even within this subgroup of participants, heterogeneity in the frequency of pain-related PASS admissions was observed. Fourteen (56%) participants had two or fewer pain related PASS admissions, whereas one participant accounted for 10 (13.5%) of these encounters. Overall, the 25 persistently frequent ED users with pain-related PASS admissions constituted a third of the study participants, and accounted for 43.8% of all ED encounters.
Table 5.
Characteristics of 25 persistently frequent ED user with pain-related PASS admissions and their pattern of ED utilization during the study period
| Patient ID |
Age | Sex | Race/ Ethnicity |
Insurance | ADI | LTOT | No. Pain PASS |
Total ED visit |
No. SLE- related |
No. Infection- related |
No. Pain- related |
No. “other” |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| P01 | 22 | Female | AA | Medicaid | 108.73 | Yes | 1 | 31 | 11 | 1 | 5 | 16 |
| P02 | 21 | Female | AA | Medicaid | 107.47 | Yes | 9 | 41 | 1 | 1 | 26 | 13 |
| P03 | 46 | Female | AA | Medicaid | 115.60 | Yes | 1 | 17 | 2 | 2 | 12 | 1 |
| P08 | 33 | Female | AA | Medicare | 114.64 | Yes | 2 | 11 | 1 | 0 | 3 | 7 |
| P09 | 36 | Male | AA | Medicare | 115.60 | Yes | 5 | 31 | 7 | 0 | 23 | 1 |
| P10 | 28 | Female | AA | Medicaid | 109.07 | Yes | 4 | 37 | 2 | 1 | 15 | 19 |
| P15 | 51 | Female | AA | Medicaid | 108.24 | No | 5 | 12 | 0 | 1 | 9 | 2 |
| P16 | 22 | Female | AA | Medicare | 126.32 | No | 1 | 18 | 1 | 4 | 3 | 10 |
| P21 | 36 | Female | AA | Medicaid | 111.65 | No | 3 | 11 | 2 | 1 | 5 | 3 |
| P23 | 44 | Female | Hispanic | Medicaid | 96.64 | No | 1 | 10 | 1 | 2 | 1 | 6 |
| P26 | 36 | Female | Caucasian | Medicaid | 89.39 | Yes | 10 | 42 | 0 | 1 | 32 | 9 |
| P27 | 22 | Female | AA | Medicaid | 103.25 | No | 1 | 13 | 2 | 1 | 9 | 1 |
| P29 | 27 | Female | AA | Medicaid | 86.08 | No | 3 | 21 | 2 | 6 | 12 | 1 |
| P32 | 39 | Female | AA | Medicaid | 116.35 | No | 1 | 10 | 0 | 4 | 3 | 3 |
| P34 | 67 | Female | AA | Medicare | 102.61 | No | 2 | 11 | 0 | 0 | 9 | 2 |
| P36 | 37 | Female | AA | Medicare | 126.82 | No | 2 | 10 | 0 | 3 | 5 | 2 |
| P43 | 48 | Female | Caucasian | Medicaid | 108.86 | Yes | 4 | 19 | 2 | 3 | 11 | 3 |
| P44 | 39 | Female | AA | Medicare | 107.10 | Yes | 6 | 22 | 2 | 1 | 14 | 5 |
| P49 | 60 | Female | Caucasian | Medicare | 81.76 | Yes | 1 | 11 | 0 | 3 | 4 | 4 |
| P54 | 72 | Female | AA | Medicare | 116.71 | Yes | 1 | 7 | 0 | 0 | 4 | 3 |
| P58 | 47 | Female | AA | Private/Commercial | 97.76 | No | 1 | 10 | 0 | 1 | 3 | 6 |
| P62 | 32 | Female | AA | Medicaid | 112.22 | Yes | 1 | 19 | 2 | 0 | 10 | 7 |
| P65 | 30 | Female | Hispanic | Medicaid | 115.60 | No | 1 | 9 | 0 | 0 | 6 | 3 |
| P69 | 24 | Female | Caucasian | Medicare | 109.09 | Yes | 4 | 49 | 0 | 6 | 8 | 35 |
| P73 | 40 | Female | Caucasian | Medicare | 101.70 | No | 4 | 29 | 5 | 3 | 17 | 4 |
AA=African American, ADI=Area of Deprivation Index, LTOT=long-term opioid therapy, No. Pain PASS=number of pain related potentially avoidable short stay admissions, No. SLE-related= number of SLE-related encounters, No. Infection-related= number of infection-related encounters, No. Pain-related= number of pain-related encounters, No. “other”=number of “other” encounters
Discussion
To our knowledge, this is the first study to characterize persistently frequent ED use among SLE patients. In this study, SLE patients who frequented the ED were mostly young African American females all of whom had some form of insurance. Persistent users were more likely to have Medicare as their primary insurance and be on LTOT compared to limited users. Medicare was associated with persistent use when adjusted for age, and may be confounded by dialysis status and other factors unaccounted for in this study that relate to permanent disability or disability benefit status which are eligibility criteria for Medicare coverage. LTOT and depression were each observed in one in three SLE patients who persistently frequented the ED.
In this study, persistent users disproportionately utilized the ED compared to limited users, and mostly for non-lupus related pain reasons. Chronic pain, a symptom frequently experienced by patients with SLE (25, 26), was a major cause of ED utilization and ED-initiated admissions among SLE patients who persistently frequented the ED. SLE patients persistently utilizing the ED for pain were more likely to be non-critically ill, as evidenced by frequent discharge from the ED and PASS admissions. And so, SLE patients who persistently frequent the ED for chronic pain represent a viable and high impact target for early intervention and education to improve chronic care management and coordination.
Lessons on how to improve the delivery of care to SLE patients may be learned from other chronic diseases, such as sickle cell anemia. Sickle cell and SLE share certain characteristics in that both are complex chronic diseases, with periods of exacerbation, which disproportionately affect young African Americans, and are frequently associated with chronic pain. Outpatient pain has been shown to be predictive of ED utilization among patients with sickle cell disease and intensive ambulatory management with frequent outpatient visits has been successful in reducing healthcare resource utilization (27–29). However, despite projected therapeutic efficacy and cost-effectiveness of ambulatory chronic pain management, compliance with and sustained improvement of healthcare resource utilization through nonpharmacological pain management may be challenging. Studies have identified poor social support and communication with providers, limitation of financial and transportation resources, reliance on opioids, lack of belief in, and inadequacy of pain control as barriers to multimodality pain management (30–33). For these reasons, and findings that regardless of access to care, some patients continue to preferentially utilize the ED for ambulatory care sensitive conditions such as chronic pain (16, 34, 35), ED-based interventions for chronic pain management, such as case management, use of chronic pain protocols, and pain specialist consultation in the ED, should be developed to complement outpatient services. Some studies have explored the use of individualized home pain management programs and community health workers who provide social support, navigation of health systems and resources, and counseling, for the management of chronic pain in sickle cell disease (36, 37). Web-based non-pharmacologic interventions may also be a viable option for chronic pain management in young SLE patients, with ready access to and familiarity with technology, but often with limited access to outpatient specialty pain clinics (38–41).
This study has several limitations. Findings are based on a small number of participants at a single tertiary medical center. The cohort of SLE patients who persistently frequented the ED, however, is expected to be small as frequent ED users typically consist of 4.5–8% of all ED patients, and persistently frequent ED users are a smaller subgroup of this population (10, 17). In addition, the criteria to confirm diagnosis of SLE was designed to have high specificity for this study, further limiting the size of the cohort. Utilization of validated EHR based search algorithms with high positive predictive value to identify SLE patients in future studies would increase both generalizability and reproducibility. Although based on a small cohort, this study included a comparison cohort, incorporated a substantial number of unique ED encounters, and detailed information on patient and encounter level variables for each visit that were obtained through retrospective in-depth physician chart review. However, because data on lupus-related disease activity either through validated or laboratory measures at time of each ED encounter were not consistently available, we were unable to assess the relationship between SLE-disease activity, pain and ED utilization. Findings from this study would be strengthened by conducting key informant qualitative interviews. SLE patients who persistently frequent the ED can be engaged to elicit their perception of, and barriers to, ambulatory care coordination and, as relevant, chronic pain management. Clinical impression at time of care transition from the ED and factors influencing physician decision for admissions can inform understanding of ED-initiated admissions. In this study, ED encounters were categorized using a priori criteria based on the principle discharge diagnosis. Further delineation of “other” encounters, particularly those that led to ED-initiated admission and were more likely to have greater complexity and discharge diagnosis codes, may provide further insight into the burden of pain not attributable to lupus and persistently frequent ED utilization. In addition, information on healthcare resource utilization during admission, especially during the first 48 hours, would allow for factors associated with PASS admissions to be ascertained, and should be included in future studies to inform opportunities to reduce ED-initiated admission of non-critically ill SLE patients and improve outpatient chronic disease management.
In conclusion, SLE patients who persistently frequented the ED were young African American females, living in more economically deprived areas, with a high burden of depression and LTOT. Pain was a major cause of both ED utilization and ED-initiated admissions, most of which were PASS admissions. SLE patients who persistently frequent the ED, particularly for pain, would benefit from targeted early interventions, in both the ED and outpatient settings, to improve chronic disease management and care co-ordination.
Significance and Innovation.
-
●
SLE is associated with substantial socioeconomic burden and healthcare resource utilization. SLE patients with low socioeconomic status, irrespective of their access to care, frequent the ED. This pattern of ED use suggests a gap in the care of SLE.
-
●
Increasingly it is recognized that frequent ED use is not a stable phenomenon. Most high-utilizers only experience a brief period of frequent ED use (<12months), however, a subgroup continue to frequent the ED over years.
-
●
Understanding persistently frequent ED use in SLE can help provide insight into opportunities to reduce health care resource utilization and improve quality of care.
Acknowledgments
Financial Interest: Lee, J, None; Lin, J, None; Suter, L works under contract with the Centers for Medicare & Medicaid Services to develop and maintain performance measures; Fraenkel, L, None. Research reported in this publication was also supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases, part of the National Institutes of Health, under Award Number AR060231 (Fraenkel).
Appendix 1. Criteria for categorization of “other” visits into different medical specialties according to the principle discharge diagnosis of the encounter
| Medical Specialty | Encounter Categorization Criteria |
|---|---|
| Allergy | Encounters due to allergic reactions such as hives, urticaria or angioedema |
| Cardiology | Encounters related to cardiac disorders which were not attributed to SLE such as arrhythmia, myocardial infarction, or congestive heart failure |
| Dermatology | Encounters primarily due to burns or skin lesions which were not attributed to SLE. |
| Endocrinology | Encounters related to glycemic control, or other hormonal dysfunction such as hypothyroidism/hyperthyroidism |
| Gastroenterology | Encounters related to gastrointestinal disorders such as gastritis, ulcers, pancreatitis, obstruction, hepatitis etc., not attributable to SLE |
| General Internal Medicine | Encounters due to hyper- or hypotension and other types of encounters that could not be clearly categorized into one of the thirteen other medical specialty groups, for example encounters for medication refill, abnormal labs, and dental issues |
| Hematology | Encounters related to hematopoietic dysfunction or thromboembolic event not attributable to SLE or antiphospholipid syndrome (APS) |
| Nephrology | Encounters due to acute or chronic kidney disease, unrelated to LN, with documentation from nephrology or pathology results |
| Neurology | Encounters of disorder of the nervous system such as seizures, CVA, or altered mental status not attributable to SLE |
| Obstetrics/Gynecology | Encounters due to pregnancy complications or other gynecologic issues not attributable to APS |
| Psychiatry | Encounters due to suicidal or homicidal ideation, alcohol intoxication, substance abuse or mood disorders not attributable to SLE |
| Pulmonology | Encounters due to respiratory disorders such as asthma, COPD, respiratory failure, not attributable to SLE |
| Surgery | Encounters that primarily required surgical intervention except for transplant due to LN |
| Trauma | Encounters due to trauma, fall, motor vehicle accident or other mechanical injury |
Appendix 2. Categorization of other ED encounters based on primary discharge diagnosis
| Medical Specialty | Other ED encounter by Persistently Frequent ED users, n (%) (n=549) |
Other ED encounter by Limited Frequent ED users, n (%) (n=190) |
|---|---|---|
| Allergy | 14 (2.6) | 7 (3.7) |
| Cardiology | 30 (5.5) | 5 (2.6) |
| Dermatology | 28 (5.1) | 7 (3.7) |
| Endocrinology | 9 (1.6) | 5 (2.6) |
| Gastroenterology | 65 (11.8) | 19 (10.0) |
| General Internal Medicine | 91 (16.6) | 53 (27.9) |
| Hematology | 30 (5.5) | 8 (4.2) |
| Nephrology | 24 (4.4) | 8 (4.2) |
| Neurology | 50 (9.1) | 18 (9.5) |
| Obstetrics/Gynecology | 72 (13.1) | 14 (7.4) |
| Psychiatry | 27 (4.9) | 1 (0.5) |
| Pulmonology | 64 (11.7) | 19 (10.0) |
| Surgery | 13 (2.4) | 1 (0.5) |
| Trauma | 32 (5.8) | 25 (13.2) |
Appendix 3. Characteristics of SLE patients who had limited frequent ED use during 2013–2016
Selection of SLE patients who persistently frequented the ED
We initially identified 187 participants with possible SLE who met criteria for persistently frequent ED use and 132 that met criteria for limited ED use from 2013–2016 through EHR query. After in-depth retrospective EHR review, 83 had a diagnosis of SLE based on one of the four pre-defined criteria. Ten participants were diagnosed with SLE during the study period and, after censoring for ED encounters where diagnoses of SLE was not probable, six were excluded. Four patients died during the study period, but all remained in the study, as each met the definition of persistently frequent ED use prior to death. The final cohort of SLE patients who persistently frequented the ED from 2013–2016 consisted of 77 participants. Diagnosis of SLE was verified based on documentation of SLE by a rheumatologist for 72 participants, and biopsy proven LN by a nephrologist for four participants. One participant, who met diagnosis of SLE based on ACR criteria had thrombosis in the setting of a positive lupus anticoagulant.
Selection of SLE patients who had limited frequent ED use
We initially identified 132 participants with possible SLE who met criteria for limited frequent ED use from 2013–2016 through EHR query. After in-depth retrospective EHR review, 59 had a diagnosis of SLE based on one of the four pre-defined criteria. Seven participants were diagnosed with SLE during the study period and, after censoring for ED encounters where diagnoses of SLE was not probable, all were excluded. No patients died during the study period. The final cohort of SLE patients who had limited frequent ED use from 2013–2016 consisted of 52 participants. Diagnosis of SLE was verified based on documentation of SLE by a rheumatologist for 48 participants, and biopsy proven LN by a nephrologist for four participants.
References
- 1.Nichol MB, Shi S, Knight TK, Wallace DJ, Weisman MH. Eligibility, utilization, and costs in a California Medicaid lupus population. Arthritis Rheum 2004;51:996–1003. [DOI] [PubMed] [Google Scholar]
- 2.Garris C, Jhingran P, Bass D, Engel-Nitz NM, Riedel A, Dennis G. Healthcare utilization and cost of systemic lupus erythematosus in a US managed care health plan. J Med Econ 2013;16:667–77. [DOI] [PubMed] [Google Scholar]
- 3.Garris C, Shah M, Farrelly E. The prevalence and burden of systemic lupus erythematosus in a medicare population: retrospective analysis of medicare claims. Cost Eff Resour Alloc 2015;13:9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Furst DE, Clarke A, Fernandes AW, Bancroft T, Gajria K, Greth W, et al. Resource utilization and direct medical costs in adult systemic lupus erythematosus patients from a commercially insured population. Lupus 2013;22:268–78. [DOI] [PubMed] [Google Scholar]
- 5.Zhu TY, Tam LS, Li EK. Cost-of-illness studies in systemic lupus erythematosus: A systematic review. Arthritis Care Res (Hoboken) 2011;63:751–60. [DOI] [PubMed] [Google Scholar]
- 6.Li T, Carls GS, Panopalis P, Wang S, Gibson TB, Goetzel RZ. Long-term medical costs and resource utilization in systemic lupus erythematosus and lupus nephritis: a five-year analysis of a large medicaid population. Arthritis Rheum 2009;61:755–63. [DOI] [PubMed] [Google Scholar]
- 7.Schuur JD, Venkatesh AK. The growing role of emergency departments in hospital admissions. NEJM 2012;367:391–3. [DOI] [PubMed] [Google Scholar]
- 8.Ward MM. Avoidable hospitalizations in patients with systemic lupus erythematosus. Arthritis Rheum 2008;59:162–8. [DOI] [PubMed] [Google Scholar]
- 9.Panopalis P, Gillis JZ, Yazdany J, Trupin L, Hersh A, Julian L, et al. Frequent use of the emergency department among persons with systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2010;62:401–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.LaCalle E, Rabin E. Frequent users of emergency departments: the myths, the data, and the policy implications. Ann Emerg Med 2010;56:42–8. [DOI] [PubMed] [Google Scholar]
- 11.Richardson LD, Hwang U. Access to care: a review of the emergency medicine literature. Academic emergency medicine : official journal of the Society for Acad Emerg Med. 2001;8:1030–6. [DOI] [PubMed] [Google Scholar]
- 12.Milbrett P, Halm M. Characteristics and predictors of frequent utilization of emergency services. J Emerg Nurs 2009;35:191–8. [DOI] [PubMed] [Google Scholar]
- 13.Hunt KA, Weber EJ, Showstack JA, Colby DC, Callaham ML. Characteristics of frequent users of emergency departments. Ann Emerg Med 2006;48:1–8. [DOI] [PubMed] [Google Scholar]
- 14.Zuckerman S, Shen Y-C. Characteristics of occasional and frequent emergency department users: do insurance coverage and access to care matter? Medical Care 2004;42(2):176–82. [DOI] [PubMed] [Google Scholar]
- 15.Block L, Ma S, Emerson M, Langley A, Torre Dde L, Noronha G. Does access to comprehensive outpatient care alter patterns of emergency department utilization among uninsured patients in East Baltimore? J Prim Care Community Health 2013;4:143–7. [DOI] [PubMed] [Google Scholar]
- 16.Ragin DF, Hwang U, Cydulka RK, Holson D, Haley LL, Jr., Richards CF, et al. Reasons for using the emergency department: results of the EMPATH Study. Acad Emerg Med 2005;12:1158–66. [DOI] [PubMed] [Google Scholar]
- 17.Johnson TL, Rinehart DJ, Durfee J, Brewer D, Batal H, Blum J, et al. For many patients who use large amounts of health care services, the need is intense yet temporary. Health Aff (Millwood) 2015;34:1312–9. [DOI] [PubMed] [Google Scholar]
- 18.Harris LJ, Graetz I, Podila PS, Wan J, Waters TM, Bailey JE. Characteristics of hospital and emergency care super-utilizers with multiple chronic conditions. J Emerg Med 2016;50:e203–14. [DOI] [PubMed] [Google Scholar]
- 19.Mandelberg JH, Kuhn RE, Kohn MA. Epidemiologic analysis of an urban, public emergency department’s frequent users. Acad Emerg Med 2000;7:637–46. [DOI] [PubMed] [Google Scholar]
- 20.Tan EM, Cohen AS, Fries JF, Masi AT, McShane DJ, Rothfield NF, et al. The 1982 revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum 1982;25:1271–7. [DOI] [PubMed] [Google Scholar]
- 21.Area of Deprivation Index [https://www.hipxchange.org/ADI]
- 22.Singh GK. Area deprivation and widening inequalities in US mortality, 1969–1998. Am J Public Health 2003;93:1137–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Chou R, Fanciullo GJ, Fine PG, Adler JA, Ballantyne JC, Davies P, et al. Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain. J Pain 2009;10:113–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Von Korff M, Saunders K, Thomas Ray G, Boudreau D, Campbell C, Merrill J, et al. De facto long-term opioid therapy for noncancer pain. The Clinical journal of pain 2008;24(6):521–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Atzeni F, Cazzola M, Benucci M, Di Franco M, Salaffi F, Sarzi-Puttini P. Chronic widespread pain in the spectrum of rheumatological diseases. Best Pract Res Clin Rheumatol 2011;25:165–71. [DOI] [PubMed] [Google Scholar]
- 26.Di Franco M, Bazzichi L, Casale R, Sarzi-Puttini P, Atzeni F. Pain in systemic connective tissue diseases. Best Pract Res Clin Rheumatol 2015;29:53–62. [DOI] [PubMed] [Google Scholar]
- 27.Koch KL, Karafin MS, Simpson P, Field JJ. Intensive management of high-utilizing adults with sickle cell disease lowers admissions. Am J Hematol Oncol 2015;90:215–9. [DOI] [PubMed] [Google Scholar]
- 28.Jonassaint CR, Beach MC, Haythornthwaite JA, Bediako SM, Diener-West M, Strouse JJ, et al. The Association between educational attainment and patterns of emergency department utilization among adults with sickle cell disease. Int J Behav Med 2016;23:300–9. [DOI] [PubMed] [Google Scholar]
- 29.Ezenwa MO, Molokie RE, Wang ZJ, Yao Y, Suarez ML, Angulo V, et al. Outpatient pain predicts subsequent one-year acute health care utilization among adults with sickle cell disease. J Pain Symptom Manage 2014;48:65–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Bair MJ, Matthias MS, Nyland KA, Huffman MA, Stubbs DL, Kroenke K, et al. Barriers and facilitators to chronic pain self-management: a qualitative study of primary care patients with comorbid musculoskeletal pain and depression. Pain Med 2009;10:1280–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Park J, Hirz CE, Manotas K, Hooyman N. Nonpharmacological pain management by ethnically diverse older adults with chronic pain: barriers and facilitators. J Gerontol Soc Work 2013;56:487–508. [DOI] [PubMed] [Google Scholar]
- 32.Jerant AF, von Friederichs-Fitzwater MM, Moore M. Patients’ perceived barriers to active self-management of chronic conditions. Patient Educ Couns 2005;57:300–7. [DOI] [PubMed] [Google Scholar]
- 33.Simmonds MJ, Finley EP, Vale S, Pugh MJ, Turner BJ. A qualitative study of veterans on long-term opioid analgesics: barriers and facilitators to multimodality pain management. Pain Med 2015;16:726–32. [DOI] [PubMed] [Google Scholar]
- 34.Brown LE, Burton R, Hixon B, Kakade M, Bhagalia P, Vick C, et al. Factors influencing emergency department preference for access to healthcare. West J Emerg Med 2012;13:410–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.DeLia D, Cantor JC, Brownlee S, Nova J, Gaboda D. Patient preference for emergency care: can and should it be changed? Med Care Res Rev 2012;69:277–93. [DOI] [PubMed] [Google Scholar]
- 36.Hsu LL, Green NS, Donnell Ivy E, Neunert CE, Smaldone A, Johnson S, et al. Community health workers as support for sickle cell care. Am J Prev Med 2016;51(1 Suppl 1):S87–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Crosby LE, Simmons K, Kaiser P, Davis B, Boyd P, Eichhorn T, et al. Using quality improvement methods to implement an electronic medical record (EMR) supported individualized home pain management plan for children with sickle cell disease. J Clin Outcomes Manag 2014;21:210–7. [PMC free article] [PubMed] [Google Scholar]
- 38.Nevedal DC, Wang C, Oberleitner L, Schwartz S, Williams AM. Effects of an individually tailored web-based chronic pain management program on pain severity, psychological health, and functioning. J Med Internet Res 2013;15:e201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Macea DD, Gajos K, Daglia Calil YA, Fregni F. The efficacy of web-based cognitive behavioral interventions for chronic pain: a systematic review and meta-analysis. J Pain 2010;11:917–29. [DOI] [PubMed] [Google Scholar]
- 40.Stinson J, White M, Isaac L, Campbell F, Brown S, Ruskin D, et al. Understanding the information and service needs of young adults with chronic pain: perspectives of young adults and their providers. Clin J Pain 2013;29:600–12. [DOI] [PubMed] [Google Scholar]
- 41.Williams DA. Web-based behavioral interventions for the management of chronic pain. Curr Rheumatol Rep 2011;13:543–9. [DOI] [PubMed] [Google Scholar]