Using electronic health records data to identify patients with chronic pain in a primary care setting

Abstract

Objective

To develop and validate an accurate method to identify patients with chronic pain using electronic health records (EHR) data at a multisite community health center.

Materials and methods

We identified patients with chronic pain in our EHR system using readily available data elements pertaining to pain: diagnostic codes (International Classification of Disease, revision 9; ICD-9), patient-reported pain scores, and opioid prescription medications. Medical chart reviews were used to evaluate the accuracy of these data elements in all of their combinations. We developed an algorithm to identify chronic pain patients more accurately based on these evaluations. The algorithm's results were validated for accuracy by comparing them with the documentation of chronic pain by the patient's treating clinician in 381 random patient charts.

Results

The new algorithm, which combines pain scores, prescription medications, and ICD-9 codes, has a sensitivity and specificity of 84.8% and 97.7%, respectively. The algorithm was more accurate (95.0%) than pain scores (88.7%) or ICD-9 codes (93.2%) alone. The receiver operating characteristic was 0.981.

Discussion

A straightforward method for identifying chronic pain patients solely using structured electronic data does not exist because individual data elements, such as pain scores or ICD-9 codes, are not sufficiently accurate. We developed and validated an algorithm that uses a combination of elements to identify chronic pain patients accurately.

Conclusions

We derived a useful method that combines readily available elements from an EHR to identify chronic pain with high accuracy. This method should prove useful to those interested in identifying chronic pain patients in large datasets for research, evaluation or quality improvement purposes.

Background and significance

Chronic pain is an extremely prevalent and costly condition that affects an estimated 100 million Americans, with an annual cost of up to US$635 billion in medical treatment and lost productivity.¹ It is estimated that more than half of the patients with chronic pain will receive their care in a primary care setting.^{2 3} However, primary care providers are not optimally prepared to manage chronic pain.^4–7 Strategies are clearly needed to improve the quality of pain care in primary care.

The increasing use of electronic health records (EHR) provides an opportunity to employ new strategies to improve the care of patients with chronic pain. Evidence suggests that data from registries and clinical dashboards can improve the quality of care for certain chronic illnesses managed in primary care.^8–10 Chronic pain management could benefit from a similar approach.

However, identifying patients with chronic pain in such datasets poses a challenge. Chronic pain lacks a simple means of identification. Data elements captured in most EHR, such as diagnostic codes, pain scores and prescription data, each have drawbacks: chronic pain lacks a unique set of diagnostic codes;¹¹ the widely used patient self-reported numeric pain rating scale¹² is only modestly accurate in identifying patients with pain;¹³ and medications used to treat pain often have multiple other uses. Currently, there is no straightforward and reliable method to identify chronic pain patients. The last attempt to develop such a method was more than 10 years ago.¹⁴ The accuracy, sensitivity and specificity of this method are unknown. It is technically complex, costly and unfeasible for routine use in a primary care setting.

As part of a large-scale quality improvement initiative focused on improving chronic pain management in a state-wide, multisite community health center, we analyzed EHR data to determine an accurate and reliable method for identifying patients with chronic pain. Identifying these patients is a critical first step for creating patient registries and developing decision-support tools for primary care clinicians, which would enable them to deliver the best possible care to patients with chronic pain.

This study was reviewed and approved by the institutional review board of the Community Health Center, Inc. (CHCI).

Materials and methods

Setting

This research study was carried out at CHCI, a multisite federally qualified health center in Connecticut. CHCI provides comprehensive primary care services in 13 primary care health centers across the state. Additional sites of care include school-based clinics, homeless shelters, and mobile dental sites. CHCI cares for over 130 000 medically underserved patients, 60% of whom are from racial/ethnic minorities. Over 90% are below the 200% federal poverty level, 60% have state-funded insurance, and 22% are uninsured.

Data preparation and processing

All care at CHCI, including medical, dental and behavioral health, is documented in an integrated EHR system called eClinicalWorks (ECW). All data from ECW are housed in a customized clinical data warehouse. The warehouse is a secure, subject-oriented data warehouse built to meet or exceed all International Organization for Standardization standards. Data were extracted using structured query language programing. All data from ECW were retrieved, de-identified, and analyzed by the study team. Data included patients’ demographics, clinic utilization, patient-reported pain scores, opioid prescription records, diagnostic codes, laboratory results, and referrals.

Patient-reported pain scores

All patients have an intake conducted by a trained medical assistant at the outset of their medical visit that includes an assessment of the chief complaint, recording of vital signs, and their current degree of pain using the 11-point numeric pain rating scale.¹² The pain score is recorded as structured data in the EHR as a ‘vital sign’.

Diagnostic codes

Visit diagnoses are coded by the primary care provider using the International Classification of Disease, revision 9 (ICD-9) coding system. The list of ICD-9 codes for potential chronic painful conditions was composed by the study team after extensive literature review of relevant chronic pain conditions and associated ICD-9 codes.^{11 15–19}

Opioid prescription medications

Medications are ordered electronically with the EHR through a Multim medication database that houses all medication names and dosages. All data on opioid prescribing were collected from the electronic prescribing record within the EHR. All medications belonging to the opioid family were included in the analysis except buprenophrine, which is not used for pain management at CHCI. Chronic opioid use was defined as the use of one or more opioid medications for 90 days or more during the measurement year. For prescriptions without an identified duration, such as ‘14 days’ or ‘1 month’, duration was calculated by dividing the total number of pills dispensed by the daily frequency ordered. For prescriptions written as ‘PRN’ (as needed), the maximum frequency indicated was used in this calculation. For example, an opioid prescription with instructions to take 2 pills BID (twice a day), dispense 60 pills would account for 15 days of opioid use. A prescription with instructions to take one pill BID PRN, dispense 40 would account for 20 days of opioid use.

Study design

The study included five steps: (1) identification of patients with potential chronic pain using the following data elements: diagnostic visit codes related to chronic pain, pain scores, and opioid medications; (2) determination of each data element's positive predictive values (PPV) at distinguishing chronic pain through chart reviews; (3) development of an algorithm combining the different data elements; (4) validation of the newly developed algorithm; and (5) application of the algorithm.

1. Identification of patients with chronic pain: We used a consensus definition for chronic pain: non-cancer pain that is continuous and persistent and lasting for more than 90 days.^20–23 All patients aged 18 years and older who had at least one visit with a medical provider at CHCI during the study period, 1 March 2011 to 29 February 2012 were included in the analysis. The following data elements pertaining to pain were used to identify patients with potential chronic pain: patient pain scores recorded at each medical visit, opioid prescription medications, and ICD-9 codes from visit codes and problem list entries. Using the three data elements individually, we generated three separate, overlapping datasets from our EHR system: (1) patients with two or more pain scores that were greater than or equal to 4, reported at visits separated by 90 days or more; (2) patients who were prescribed 90 days or more of opioid analgesic medications; and (3) patients who had at least one visit with an ICD-9 code for a potential chronic painful condition during the study period.

2. Determination of accuracy of data elements at distinguishing chronic pain: We conducted random chart reviews to evaluate the accuracy of using pain scores, opioid prescription medications, and ICD-9 codes in all possible combinations to identify chronic pain. The presence of chronic pain was considered definitive if: (1) there was any documentation in the chart by the treating clinician of a pain complaint as being chronic in nature; OR (2) if the patient had two or more encounters separated by at least 90 days in which the same or similar pain complaint was indicated as being present and continuous. Documentation appeared in the chart's ‘history of present illness’, or ‘treatment’ sections. We looked for terms that describe the pain as constant or recurrent. For example, if the clinician's note stated that a patient had ‘chronic’, ‘persistent’, ‘recurrent’, or ‘ongoing’ pain in the same body area for more than 90 days or if the same pain was consistently documented in visit notes for more than 90 days, then the patient was considered to have chronic pain.

3. Development of the chronic pain identification algorithm: By reviewing charts and exploring the PPV of the three data elements in all possible combinations, and by further refining the list of ICD-9 codes for painful conditions, we developed an identification algorithm that optimized the predictive values of each data element and most accurately identified chronic pain.

4. Validation of the chronic pain identification algorithm: This algorithm was validated by reviewing randomly selected patient charts using the same chart evaluation process. The sensitivity, specificity, PPV, negative predictive value (NPV), accuracy, likelihood ratio, and receiver operating characteristic (ROC) were calculated for the final chronic pain identification algorithm.

5. Application of the chronic pain identification algorithm: We applied the algorithm to the CHCI's patient database and identified a cohort of patients with chronic pain. Descriptive statistics were used to analyze the data. We compared the clinical and demographic characteristics and utilization patterns of the two mutually exclusive groups of patients—the chronic pain cohort and the cohort of patients without chronic pain. For comparisons, we used the χ² test or Fisher’s exact test, when appropriate, to compare proportions, and the Student's t test to compare means. All tests were two-sided and considered significant at p<0.05.

Results

Identification of patients with chronic pain

A total of 38 520 patients, aged 18 years and older, had at least one medical visit between 1 March 2011 and 29 February 2012 and were included in the evaluation. Queries using the three data elements individually: pain scores, ICD-9 codes, and opioid prescription medications generated three initial datasets of patients. There were 12 186 patients with an ICD-9 coded visit or problem list entry for a potential chronic pain condition. There were 7805 patients with two or more pain scores greater than or equal to 4 separated by 90 days, and 1308 patients who received 90 days or more of an opioid analgesic medication during the measurement year.

Determination of accuracy of data elements at distinguishing chronic pain

The data elements identified patients with pain with different degrees of accuracy. While most of the patients receiving chronic opioids were also identified by either the ICD-9 codes or pain score queries, there was less overlap between the patients captured using ICD-9 codes and pain scores. We calculated the PPV of using the data elements to identify patients individually, and the PPV of using various combinations of the data elements to identify patients (table 1).

Table 1.

Chart review results to determine accuracy of data elements

Data elements used	Total no of patients	No of patients reviewed	Confirmed chronic pain	PPV (%)
Pain score	7805	100	72	72
Opioid prescription	1308	100	98	98
ICD-9 code (unrefined)	12186	100	81	81
Pain score+ICD-9	5405	50	41	82
Pain score+opioid	1166	50	48	96
ICD-9+opioid	1144	50	49	98
Pain score+opioid+ICD-9	1306	50	49	98

PPV, positive predictive value.

Development of the chronic pain algorithm

To improve the accuracy of using ICD-9 codes to identify chronic pain, as opposed to acute pain, we conducted additional chart reviews and distinguished two refined ICD-9 code sets (see supplementary appendix 1, available online only). One consisted of codes that were ‘highly likely’ to identify chronic pain and one consisted of codes that were ‘likely’ to identify chronic pain. ICD-9 codes in the ‘highly likely’ set all explicitly mention chronic pain (eg, 338.2 chronic pain, 338.4 chronic pain syndrome, etc.), whereas ICD-9 codes in the ‘likely’ set only described some type of pain (eg, 719.41 joint pain, shoulder, 722.52 degeneration of lumbar intervertebral disc, 724.5 back pain, 729.5 pain in limb, etc.). The former was able to identify chronic pain even if the codes were used only in one occurrence in the study period, while the latter was more indicative of chronic pain if the codes were used in at least two occurrences separated by at least 30 days in the study period.

Using a combination of criteria, including the refined ICD-9 code sets, pain scores and opioid prescription medications, we developed an algorithm and identified patients with confirmed chronic pain more accurately than using any of the criteria individually. The new algorithm uses the following set of criteria for identifying patients with chronic pain:

All patients, aged 18 years and older, with at least one medical visit in the past year with one or more of the following criteria:

1. A single occurrence of an ICD-9 code in the ‘highly likely to represent chronic pain’ code set in the study period

   OR

2. Two or more occurrences of ICD-9 codes in the ‘likely to represent chronic pain’ code set, separated by at least 30 days in the study period

   OR

3. Receipt of at least 90 days of opioid medication in the study period

   OR

4. One occurrence of an ICD-9 code likely to represent chronic pain AND two or more pain scores greater than or equal to 4 in the study period.

Validation of the chronic pain algorithm

We calculated the need for a sample size of 381 patients to validate the algorithm with a 5% margin of error and 95% confidence level. From a list of all adult patients seen at least once during the study timeframe, we selected 381 patient charts to review at random. The randomly selected charts included 74 patients identified by the algorithm as having chronic pain (19.4%). The algorithm had a sensitivity of 84.4%, a specificity of 97.7%, a PPV of 90.5% and a NPV of 96.1%. These results are summarized in table 2. The algorithm demonstrated a high degree of accuracy (95.0%). A patient who was identified by the algorithm for chronic pain was 36.6 times more likely to have chronic pain compared to someone who was not identified. The ROC was 0.981 (SD=0.014) (figure 1).

Table 2.

Validation of the chronic pain identification algorithm—chart review results

		Using ‘gold standard’ medical chart evaluation process		Total
		Chronic pain present	Chronic pain absent
Using chronic pain algorithm	Patient identified	67	7	74	PPV=90.5%
	Patient not identified	12	295	307	NPV=96.1%
	Total	79	302	381
		Sensitivity=84.8%	Specificity=97.7%

NPV, negative predictive value; PPV, positive predictive value.

Figure 1.

ROC curve for the chronic pain algorithm.

Using this method was more accurate and sensitive than using the data elements individually. Table 3 provides a summary of the sensitivity, specificity, PPV, NPV, accuracy, likelihood ratio, and ROC of the algorithm, the initial pain scores and chronic opioids usage datasets and the two refined ICD-9 code sets.

Table 3.

Summary of chart reviews

	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Accuracy	Likelihood ratio	Fitted ROC	Empirical ROC
Algorithm	84.8	97.7	90.5	96.1	95.0	36.6	0.981	0.918
Pain scores	68.4	94.0	75.0	91.9	88.7	11.5	0.932	0.816
Chronic opioids usage	20.3	100.0	100.0	82.7	83.5	N/A	N/A	0.616
ICD-9 codes likely to represent chronic pain	70.9	99.0	94.9	92.9	93.2	71.4	0.909	0.860
ICD-9 codes highly likely to represent chronic pain	20.3	99.3	88.9	82.6	82.9	30.6	0.531	0.613

NPV, negative predictive value; ROC, receiver operating characteristic.

Application of the chronic pain identification algorithm to measure patient characteristics

Using this new algorithm, we identified a cohort of health center patients with chronic pain. Chronic pain was highly prevalent in the adult CHCI patient population. Of the 38 518 adult patients seen in the measurement year, 7491 (19%) had chronic pain as indicated by our algorithm. Table 4 shows demographic and healthcare utilization characteristics for these patients as compared to patients without chronic pain.

Table 4.

Comparison of patients with and without chronic pain at CHCI

	No of patients	All CHCI patients	Chronic pain patients	Non-chronic pain patients	p Values
		38 518	7491	31 027
Sex	Female	22 953 (60%)	4752 (63%)	18 201 (59%)	<0.001
Age (years)	18–29	9506 (25%)	832 (11%)	8674 (28%)	<0.001
	30–39	8231 (21%)	1360 (18%)	6871 (22%)	<0.001
	40–49	8676 (23%)	2131 (28%)	6545 (21%)	<0.001
	50–59	7209 (19%)	2049 (27%)	5160 (17%)	<0.001
	60–69	3463 (9%)	821 (11%)	2642 (9%)	<0.001
	69+	1433 (3.7%)	298 (4.0%)	1135 (3.7%)	0.189
Race	Caucasian	15 615(41%)	3399 (45%)	12 216(39%)	<0.001
	Black	4599 (12%)	895 (12%)	3704 (12%)	0.981
	Hispanic	14 784 (38%)	2823 (38%)	11 961(39%)	0.167
	Other	3520 (9%)	374 (5%)	3146 (10%)	<0.001
Visits/year	1–5 Medical visits	30 705 (80%)	3200 (43%)	27 505 (89%)	<0.001
	6–10 Medical visits	6118 (16%)	3015 (40%)	3103 (10%)	<0.001
	11–15 Medical visits	1304 (3%)	969 (13%)	335 (1%)	<0.001
	16–20 Medical visits	298 (1%)	235 (3%)	63 (0%)	<0.001
	20+ Medical visits	93 (0%)	72 (1%)	21 (0%)	<0.001
	Average no of medical visits/year	3.7	7.0	2.9	<0.001
Pain scores	At least two pain scores ≥4	7805 (20%)	5677 (76%)	2128 (7%)	<0.001
	At least one pain score ≥8	12 349 (32%)	5568 (74%)	6781 (22%)	<0.001
Opioid usage	Any opioid medication prescribed at least once	5272 (14%)	3231 (43%)	2041 (7%)	<0.001
	≥90 Days of opioids prescribed	1308 (3%)	1308 (17%)	0 (0%)	<0.001
Behavioral health	Patients with behavioral health visits	4878 (13%)	1734 (23%)	3144 (10%)	<0.001
	Referrals to any behavioral health	4630 (12%)	1391 (19%)	3239 (10%)	<0.001
Pain-related referrals	Chiropractic	55 (0%)	45 (1%)	10 (0%)	<0.001
	Acupuncture	6 (0%)	2 (0%)	4 (0%)	0.331
	Physical therapy	1903 (5%)	1181 (16%)	722 (2%)	<0.001
	Pain management	626 (2%)	542 (7%)	84 (0%)	<0.001
	Physiotherapy	18 (0%)	10 (0%)	8 (0%)	0.001
	Physical medicine and rehabilitation	1558 (4%)	1040 (14%)	518 (2%)	<0.001
	Neurological surgery	305 (1%)	221 (3%)	84 (0%)	<0.001
	Orthopedic surgery	2151 (6%)	1358 (18%)	793 (3%)	<0.001
	Rheumatology	392 (1%)	229 (3%)	163 (1%)	<0.001
	No of patients with at least one referral	5671 (15%)	3482 (46%)	2189 (7%)	<0.001
Insurance status	Medicaid	22 080 (57%)	5005 (67%)	17 075 (55%)	<0.001
	Medicare	4157 (11%)	1248 (17%)	2909 (9%)	<0.001
	Private insurance	4347 (11%)	560 (7%)	3787 (13%)	<0.001
	Uninsured	7464 (19%)	660 (9%)	6804 (22%)	<0.001

CHCI, Community Health Center, Inc.

Patients with chronic pain utilized primary care services at a much higher rate, with an average of seven visits per year as compared to 2.9 per year for non-chronic pain patients and 3.7 visits per year for all CHCI adult patients. Patients with chronic pain were predominantly women (63%), between the ages of 30 and 59 years (73%), Caucasian (45%), and covered by Medicaid insurance (67%). Twenty-three per cent had a behavioral health visit during the measurement year. Opioid medications were commonly prescribed to patients with pain, with 43% of patients having received an opioid medication prescription at least once in the measurement year, and 17% received 90 days or more of opioid analgesic medications. Patients with chronic pain were significantly more likely to be women, covered by Medicaid insurance, and have a behavioral health visit than patients without chronic pain. There were significant differences in the racial/ethnic composition between patients with and without chronic pain (see table 4). Referral rates to other pain-related specialties were low for patients with pain and for the overall CHCI patient population. Patients with chronic pain had a higher likelihood of being referred to orthopedics than patients without chronic pain. Referrals to other specialists were uncommon. Referrals to complementary and alternative medicine providers were essentially non-existent in both populations, and referrals to pain management specialists were also extremely low.

Discussion

By exploring the accuracy of different data elements from an EHR system, we demonstrated the limitations of each element individually to identify chronic pain. By combining various elements, all of which are readily available in many EHR systems such as ours, we were able to identify patients with chronic pain more accurately. The new algorithm, combining opioid prescribing, ICD-9 codes and pain scores, proved to be substantially more accurate than the usage of individual data elements. In particular, our results showed that neither ICD-9 codes alone nor pain scores alone are accurate at identifying patients with chronic pain. Chronic opioid data are highly specific but not sensitive. Combining the data elements, however, improved sensitivity and specificity.

By applying this algorithm to our patient population, we were able to gain a better understanding of the extent of chronic pain and how it is managed in our health centers. Consistent with other studies,^{3 5 24–30} we found chronic pain to be extremely prevalent, and patients with pain to account for much higher primary care utilization than those without pain. Patients with pain accounted for seven visits per year, more than double the rate of patients without chronic pain (2.9 visits per year). These numbers are similar to those observed in a study by Group Health Cooperative in which patients with back pain were seen for an average of eight visits per year.³¹ Patients with chronic pain had a much higher rate of behavioral health utilization compared to the overall clinic population. This finding is consistent with previously published work.³² A significant number of these patients received opioid medications, but almost none were referred to pain specialists, despite current guidelines recommending such consultation for patients with coexisting mental health and/or substance abuse problems before opioids are prescribed. This probably reflects the paucity of pain specialists available for consultation.

One of the strengths of our study was the use of detailed medical chart review to determine the accuracy of the data elements for identifying pain in all possible combinations. The resulting analysis should prove useful for researchers, health systems, or other entities interested in identifying study cohorts and determining the prevalence of chronic pain. Previous studies using large datasets to detect and analyze characteristics of patients with various types of pain have relied exclusively on billing and coding information.^33–37 Using opioid prescribing and pain score data and combining them with our two-tiered ICD-9 coding set enhanced our ability to detect patients with chronic pain who would otherwise have been missed. Small errors in the detection of patients can result in a large number of incorrect entries and calculations.³⁸ Our validation process allowed us to minimize errors and provided us with a good estimation of the accuracy of our data when analyzing patient characteristics.

Our study has several limitations. We utilized the patient self-reported 0–10 numerical rating scale for assessment of pain. Studies have shown that the accuracy of this tool to assess pain intensity accurately is moderate at best.¹³ Furthermore, it is possible for a patient to have two completely different pain complaints with pain scores greater than or equal to 4 and separated by 90 days. Five of our seven false positive cases were attributable to this issue. In addition, although we used medication information to improve the detection of pain, we relied exclusively on opioid medications and did not collect data on other classes of pain medication, given the fact that most have multiple uses for conditions other than pain. However, future analyses may benefit from including other classes such as non-steroidal anti-inflammatory agents or muscle relaxants. Finally, the chart review validation method to determine true chronic pain has limits. The validation relied exclusively on the accuracy of documentation of chronic pain in the charts by the primary care provider as the ‘gold standard’ to confirm the diagnosis. However, free-text chart reviews are one of the few tools that have the ability to validate coded diagnosis and detect false negatives.³⁹ CHCI providers receive training during orientation on the proper use of custom folders in the ‘history of present illness’ section of the EHR to document pain issues. In addition, they receive ongoing training on pain care documentation standards. Providers also receive training at orientation on proper coding for medical visits. However, adherence to documentation and care standards is variable. During the chart review to validate the algorithm, we found 12 missed cases of chronic pain. Five cases were missed because of poor documentation by the primary care provider. Provider training on appropriate EHR use can mitigate the variability of EHR documentation.⁴⁰ Natural language processing tools may supplement this algorithm and further improve its sensitivity and specificity.

Conclusion

Our chronic pain identification algorithm has the potential to provide practical benefit to organizations such as ours interested in studying and improving the care of patients with chronic pain. Quality improvement strategies are needed to improve the quality and safety of chronic pain management. Studies show both patients and providers are dissatisfied with pain management.^{5 41–43} In addition, opioids are being prescribed at rapidly increasing rates for the management of pain.^{44 45} This increase in opioid prescribing is directly contributing to the marked increase in morbidity and mortality from opioid analgesic medications.^{36 44–51} Primary care providers need access to patient registries and other information on their pain management practice that can be used to drive practice improvement. By developing an algorithm using a variety of readily available data elements from an EHR we have refined our ability to identify chronic pain and taken an important first step in addressing this critical public health challenge.