Impact of an Electronic Pain and Opioid Risk Assessment Program: Are There Improvements in Patient Encounters and Clinic Notes?

Abstract

Objective. A comprehensive electronic self-report assessment, called PainCAS^® (Clinical Assessment System), was developed and implemented in three clinics. PainCAS captures demographic information, pain assessment, quality-of-life variables, and contains validated, electronic versions of screeners for risk of aberrant opioid-related behaviors (the SOAPP and COMM). This investigation sought to determine the impact of PainCAS on documentation of pain and opioid risk evaluations. Exploratory hypotheses examined changes in the content of the patient-provider interaction and any impact on outcome.

Methods. In study 1, chart reviews were conducted between pain patients who completed the electronic program (N = 89) and controls who represented standard of care (N = 120). In study 2, two groups of chronic pain patients (treatment-as-usual Control condition = 75, PainCAS Experimental condition = 72) were interviewed after completing their index clinic visit and completed mailed questionnaires 3 months later.

Results. Results revealed significantly more key, pain-relevant chart elements documented in charts of patients who completed the PainCAS than those using a traditional paper questionnaire (Study 1; <0.001). In Study 2, the Experimental group reported more discussion about legal issues, substance use history, and medication safety compared with the Control group (p < 0.05). Satisfaction questionnaire responses supported provider and patient perceived benefit from using PainCAS. However, as expected, no differences were found between conditions on outcome measures of pain, mood, and function.

Conclusions. Results indicate that use of the PainCAS electronic pain assessment improves documentation of chart elements in clinic notes and is associated with increased discussion of key, pain-relevant topics during the clinical visit.

Introduction

There has been recent interest in implementing electronic pain assessment programs in clinic settings to assess and monitor persons with chronic pain [1,2]. Evidence exists for the benefits of such programs to assess opioid risk, reduce personnel time, and document change along the continuum of pain care [3]. Preliminary studies suggest that a web-based secure electronic assessment program outweighs the benefits of a paper questionnaire [4]. However, barriers and challenges exist and further evidence is needed to justify the implementation of an electronic pain assessment program to determine its benefits over and above traditional paper-based assessments [5,6]. Despite the interest electronic assessment programs have generated, there is a critical need for evaluation of these technology-based monitoring programs and clinical decision-making tools for chronic pain management in order to understand whether there is evidence of improvement in the clinical care of individuals with chronic pain along the course of treatment. Although the Institute of Medicine’s report on chronic pain recommended promoting and enabling the use of technology to improve standardized assessment of pain [7] they indicated that there is little evidence of the benefit of these programs to support clinical decision-making in pain. This can be especially important in settings where electronic health records use applications intended for patient self-management and provider decision-making [8]. Nevertheless, use of electronic formats to replace pencil-and-paper questionnaires that must be scanned into the medical record is likely inevitable, and at face value, well-designed software should be more efficient, save time and money, and be a step toward standardization of care and improved outcomes. However, studies are needed to establish the usability of software tools for providers, patients, and administrative staff and to determine if such electronic assessments result in a more standardize approach to pain assessment and, ultimately, whether they indeed improve patient self-management and provider decision making.

The PainCAS^® (Clinical Assessment System) program is a systematic computer-administered, patient self-report assessment for chronic pain patients [3] (see Figure 1). It is an electronic assessment and tracking program designed to provide a comprehensive evaluation of chronic pain patients that includes [1] demographic information (e.g., age, gender, medical history), [2] pain assessment and quality of life evaluation (e.g., pain intensity, activity interference, mood, medication use, side effects), and [3] opioid risk assessments using electronic versions of the Screener and Opioid Assessment for Patients with Pain (SOAPP^®), including the revised version (SOAPP^®-R), which is a validated 24-item self-reported assessment tool to help identify risk potential for aberrant medication-related behaviors among chronic pain patients prescribed long-term opioid therapy [9–12], and the Current Opioid Misuse Measure (COMM)^®, which is a validated 17-item assessment of current opioid misuse, abuse or other aberrant behaviors [13–15]. The PainCAS has two separate assessments that are selected by the clinician; a detailed initial or intake version that collects a wealth of historical information, and a much briefer follow-up version. The initial assessment focuses on presenting pain complaint, medical and family history, and current status, while the follow-up assessment shifts focus to changes in pain, function, and risk along the continuum of care. The PainCAS is not intended to be administered by the health care provider. Rather, the patient can complete the assessment online prior to the visit, or any Internet-connected device (e.g., tablet computer provided by a clinic staff person to be completed in the waiting room immediately prior to the visit). As this is a patient self-reported assessment, impact on the provider or clinical visit-time should be minimal. The PainCAS program offers the provider an “at-a-glance,” summary of patient information with a pain diagram and risk assessment scores presented in a dashboard-like manner that can be downloaded as a PDF report and easily attached to any electronic medical record (EMR) system (see Figure 2). A focus of the program is to summarize and document the results of the assessment and to continually evaluate and track the risk of aberrant drug-related behavior for those considered for opioid analgesic therapy. It was especially designed to offer clinicians a way to rapidly identify, track, and appropriately treat chronic pain patients at risk for aberrant drug-related behavior. Patients can complete PainCAS assessments from home, or at the clinic, before their appointment, and the program generates a report for both the clinician and patient once the assessment is completed. The PainCAS clinician report is immediately available to be downloaded and attached to a patient’s medical record by a clinic staff member.

Figure 1.

Example screen of the PainCAS assessment.

Figure 2.

Example provider PainCAS summary output of patient information with pain diagram, ratings of pain, functioning, and risk assessment scores over time.

The purpose of this study was to assess the impact of using an electronic pain assessment program designed to replace existing paper-and-pencil pain assessments at participating treatment facilities. We intended to implement the web-based PainCAS electronic assessment software program for chronic pain patients in separate clinical settings and study the impact of the program on assessment documentation and patient experience with their providers. The primary objective was to test the impact on pain-related documentation between a treatment-as-usual condition and a PainCAS condition in which selected patients and providers would be invited to use PainCAS. It was expected that important chart elements related to a comprehensive pain evaluation would be documented significantly more often in the PainCAS condition as compared with the treatment-as-usual condition. A secondary objective was to understand how the PainCAS program influenced discussion among providers and patients during a typical clinic visit. While no individual visit requires complete coverage of all topics, we hypothesized that the patient having completed the PainCAS program and the provider having access to PainCAS reports would tend to be engaged in more clinically relevant topics covered during any given visit as a result of completing the program. We also wanted to examine differences between the treatment-as-usual and PainCAS conditions on pain and functioning outcomes. The PainCAS is intended to be an assessment tool and not an intervention; therefore, we did not hypothesize that the PainCAS would reliably be associated with pain or function-related outcomes.

Development of the PainCAS

The PainCAS was conceptualized as a clinically useful self-report, electronic pain assessment that is practical in the clinical setting and could fit seamlessly into the office and clinical flow of a practice. The vision was to create an electronic assessment and tracking program that would replace paper questionnaires that require scanning into the electronic medical record (EMR) and require the provider to page through patients’ responses to locate positive findings. The PainCAS would therefore need to be easy for patients to complete unaided, relieve administrative staff burdens of scanning completed paper questionnaires, and present providers with organized reports that highlight important positive findings and evaluate and track pain, function, and opioid risk over time. PainCAS also provides a patient-focused report that organizes their own responses in a way that may help facilitate communication with their provider.

To achieve this functionality, careful attention was paid to 1) content validity of the intake and follow-up assessment questions, 2) content and structure of the clinician and patient report output, 3) cognitive interviewing of items and reports for providers and patients, and 4) usability testing of providers and patients. Briefly, a seven-member expert advisory panel of nationally recognized pain treatment experts assisted the research team with a review of the literature to identify domains and information considered important for documentation in the assessment of chronic pain patients [16,17]. Candidate items for the initial and follow-up assessment questions were further evaluated by 36 pain treatment professionals from a variety of clinical professions (physicians 31%, registered nurses 33%, nurse practitioners 8%, psychologists 14%, and other pain professionals 4%) using concept mapping, a technique for deriving consensus that utilizes multidimensional scaling and cluster analysis [18]. Concept mapping has been used to establish content validity of other clinical assessments [9]. Finally, a series of patient and provider cognitive interviews and usability testing sessions ensured comprehensibility of the items presented to patients along with patient and provider output reports and usability of the PainCAS software.

Study 1

Methods

The first study was conducted at three independent sites, which represented three hospital-based treatment centers; a pain center at a large teaching hospital (Center 1), and a pain center at an affiliate community hospital (Center 2), both in Massachusetts, and an outpatient family practice clinic in Mississippi (Center 3). Data were collected between January 2014 and April 2015. The study was approved by their respective institutional review boards (IRBs). The study involved a chart review of documentation among medical records chosen at random of patients at the three sites who either completed a paper-and-pencil comprehensive pain assessment (standard-of-treatment control) or matched patients who were invited to complete the PainCAS program (Experimental). Onsite research assistants (RAs), who were trained in reviewing medical records and blinded to the study hypotheses, reviewed the charts (electronic medical records) for those elements judged to be important for documentation in the chart of a patient with a chronic pain condition using a checklist of the presence or absence of separate chart elements. Documentation was assessed among charts chosen at random of Control patients representing treatment-as-usual and charts chosen at random of Experimental patients who had been invited to complete the PainCAS before their clinic visit. Charts in the treatment-as-usual condition were included in the study if the subjects 1) were able to read and write in English, 2) were being treated for chronic pain, 3) had at least one clinical visit in the past two months, 4) were on or had been considered for prescription opioid therapy, and 5) were patients of participating clinicians. Inclusion criteria for the PainCAS condition were the same, except that the patients 1) had at least one clinical visit during the PainCAS implementation in the past 2 months and 2) had been emailed the PainCAS invitation to complete the assessment.

Statistical Analysis

The primary hypothesis of Study 1 was that the chart review in the Experimental condition (i.e., patients who completed the PainCAS assessment) would reveal a higher frequency of documentation of chart elements when compared with a treatment-as-usual Control condition (i.e., no PainCAS). Analyses for the chart-review outcomes testing for differences between the Control condition (no PainCAS) and PainCAS condition involved Chi-Square Tests of Independence of proportions or Fisher’s Exact Test if the expected counts were too small. The level of significance was set at α = .05. All analyses were performed using SAS 9.4 (SAS Institute, Inc., Cary, NC, USA). We were also interested in testing whether the proportion of medical records that included an opioid risk assessment (e.g., SOAPP-R or COMM) would substantially increase after patients had been invited to complete the electronic pain assessment program in place of the paper-pencil versions of the assessments.

Results

Two hundred and nine (N = 209, PainCAS = 89; Control = 120) patient charts were included in the analyses; 100 were selected and reviewed at Center 1 (PainCAS = 50; Control = 50), 48 were reviewed at Center 2 (PainCAS = 23, Control = 25), and 61 charts were reviewed at Center 3 (PainCAS = 16; Control = 45). Patient demographic differences between centers are presented in Table 1. Patients in Center 3 tended to be older, female, and more racially diverse compared with Centers 1 and 2. Of the 209 patients in this study, 43% (N = 90) were initial visits and 57% (N = 119) were follow-up patients. A total of 23 clinicians consented to participate in Study 1: 16 physicians, six nurse practitioners and one psychologist. Of these, 10 (43%) were primary care providers and the remaining 57% were pain specialists.

Table 1.

Demographic differences among the three centers (N = 243): Brigham and Women’s Hospital (BWH), Newton-Wellesley Hospital (NWH), and University of Mississippi Medical Center (UMS)

	BWH (2) (N = 100)	NWH (1) (N = 48)	UMS (N = 95) N (%)	P-value
Age (mean ± SD)	45.34 ± 13.07*	47.19 ± 15.99†	55.23 ± 14.12*,†	<.0001
Gender
Male	47 (47.00%)	14 (29.17%)‡	68 (71.58%)‡	0.014
Race
White/Caucasian	82 (82.00%)§	42 (87.50%)¶	28 (29.47%)§,¶	<.0001

*^{,†,‡,§,¶}Unique superscript indicates significant pairwise post hoc Tukey test (P < 0.05).

^∥Race was categorized as White/Caucasian versus Other.

Chi-square tests of Independence were conducted to test for significant differences between PainCAS and Control (no-PainCAS) conditions for a number of present chart elements. Significantly greater levels of chart documentation were observed for the PainCAS condition for a number of pain-related individual chart elements; pain description (p = 0.003), pain frequency and duration (p = 0.03), and pain severity, impact of pain, and current workers compensation and/or disability status (p < 0.001)), compared with the no-PainCAS charts (Table 2). Charts in the PainCAS condition had also a higher percentage of documentation of use of herbal supplements, non-pharmacological approaches, interventional treatments, complementary and alternative treatment, seeing other health care providers, self-treatment, history of adverse effects with pain medications, having specific, desired treatment outcomes, and pain-related litigation (all differences significant at P < 0.001). Participants in the PainCAS group had a greater percentage than the no-PainCAS condition documentation relating to psychological symptoms such as depression, anxiety, irritability, concentration problems, social isolation, fatigue, and forgetfulness. Significant findings were also observed for documentation related to the presence/absence of formal mental health treatment/counseling, support system evaluation, and overall opinion of personal health (P < 0.001). There were significantly more charts in the PainCAS condition that included reports on past or present history of alcohol abuse, smoking, prescription drug abuse, family past or present history of substance use litigation compared with the no-PainCAS condition (P < 0.001). Finally, significant differences (P < 0.001) were found between conditions in the percent found with any risk assessment (SOAPP, COMM, or other formal risk assessment), opioid risk stratification, and presence of a risk monitoring plan regarding opioid use, with those in the PainCAS condition revealing greater levels of such risk documentation than the treatment-as-usual condition.

Table 2.

Pain description, treatment history, and risk assessment variables found to be present in patient records (Control = 120; Experimental = 89)

	Control % Present n = 120	Experimental % Present n = 89	${\mathit{\chi }}^2$	P
Pain description chart elements
Pain location	97.50	100.00	2.257	0.263
Pain description	78.33	93.26	8.779	0.003
Pain frequency and duration	81.67	92.13	4.686	0.030
Pain severity	58.33	87.64	21.236	<0.001
Impact of pain	46.67	86.52	35.068	<0.001
Current workers compensation and/or disability status	28.33	52.81	12.897	<0.001
Treatment history chart elements
Current medications	97.50	100.00	2.257	0.263
Herbal supplements	7.50	34.83	24.666	<0.001
Non-pharmacological approaches	45.00	79.78	25.708	<0.001
Interventional treatments	52.50	80.90	18.020	<0.001
Complementary and alternative treatments	24.17	69.66	43.027	<0.001
No prior treatment (Initial patients only)	18.06	14.71	0.184	0.668
Other health care providers seen	42.50	85.39	39.435	<0.001
Self-treatment	24.17	53.93	19.458	<0.001
History of adverse effects with pain medications	30.00	64.04	23.982	<0.001
Co-existing medical conditions	80.00	85.39	1.020	0.313
Desired outcomes of treatment (Initial patients only)	18.06	100.00	62.836	<0.001
Pain-related related litigation (Initial patients only)	0.00	100.00	106.00	<0.001
Psychosocial evaluation chart elements
Indications of depression, anxiety, irritability, concentration problems, social isolation, fatigue, forgetfulness	55.83	85.39	20.665	<0.001
Formal mental health treatment/counseling	20.00	70.79	54.242	<0.001
Support system	39.17	77.53	30.449	<0.001
Patient’s overall opinion of personal health (Initial patients only)	12.50	100.00	73.337	<0.001
Substance use history chart elements
Past or present detailed history of illegal drugs	49.17	59.55	2.215	0.137
Past or present history of alcohol use	49.17	80.90	21.996	<0.001
Past or present history of smoking	53.33	84.27	21.955	<0.001
Past or present history of prescription drug abuse	45.00	74.16	17.768	<0.001
Family history of substance use (Initial patients only)	34.72	97.06	36.219	<0.001
Aberrant medication related behaviors	11.67	70.79	76.757	<0.001
Substance use/abuse litigation (Initial patients only)	0.00	100.00	106.00	<0.001
Risk assessment chart elements
Any opioid risk assessment (SOAPP, COMM, or Other)	5.83	82.02	125.552	<0.001
Opioid risk stratification	5.00	53.93	63.854	<0.001
Monitoring plan regarding extent of opioid use	10.00	51.7	44.287	<0.001

^¥Chi-square values, tests for difference between Control and PainCAS intervention groups.

^†Screener and Opioid Assessment for Pain Patients – Revised.

^§Current Opioid Misuse Measure.

^£Initial patient visits only: Control n = 72, Experimental n = 34.

Study 2

Methods

This study was conducted at Centers 1 and 2 only. Patients over 18 with greater than 6 months of chronic noncancer pain and an email address were recruited and consented for this study and randomized to Experimental or Control groups. All patient participants met the same inclusion criteria as in Study 1. The Control subjects received treatment as usual while the Experimental subjects were asked to complete the PainCAS program prior to their next provider visit. The consented patients were asked to complete a baseline battery of self-report questionnaires and received $50 for their participation. All patients were asked to complete a post-visit interview with the onsite RA after their next clinic visit and to endorse topics covered during the clinical encounter they had just experienced. The patients were asked to remember whether a variety of topics were discussed during their clinic visit with their physician among four categories: 1) pain-related issues, 2) psychosocial factors, 3) substance use history, and 4) medication safety/treatment monitoring. If the clinician covered a particular topic, the patient would respond with a ‘yes’ (code = 1). If the topic was not discussed, then the patient would respond with a ‘no’ (code = 0). The percentages of individuals reporting a particular topic were calculated and compared across conditions. Question items within each topic area were summed, and a mean value for each of the four topics compared across conditions. Three months later patients received an email message with an attached follow-up assessment and received $75 for participation.

Clinicians in each clinic were recruited to participate in this study and signed an informed consent. Following the intervention period, participating physicians were interviewed to determine the impact of the electronic pain assessment program on their clinical workflow and most received $100 for participating in the interview.

Patient Measures

Background Questionnaire [19]

This self-report questionnaire completed online was adapted for this study that gathered demographic information on each participant including age, gender, race, marital status, education, diagnosis, length of time he or she had a chronic pain condition, and length of time he or she had been seeing a doctor for a chronic pain condition.

Chronic Pain Treatment Satisfaction Scale (CPTSS) [20]

The CPTSS includes 39 items grouped in five dimensions related to pain treatment satisfaction: information (5 items); medical care (8 items); impact of current pain medication (8 items); satisfaction with pain medication, which included two subscales of medication characteristics (3 items) and medication efficacy (3 items); and side effects (12 items). It is a valid, comprehensive instrument to assess satisfaction with treatment of pain based on independent modules that has demonstrated satisfactory psychometric performance. The CPTSS has shown adequate internal reliability coefficients (from 0.83 to 0.92) and test-retest reliability coefficients (from 0.67 to 0.81). All dimensions except medical care discriminated well according to pain severity. The satisfaction with efficacy dimension, hypothesized to change in the acute pain population, indicated good preliminary responsiveness properties (effect size 0.37; P < 0.001).

The Brief Pain Inventory (BPI) [21]

This self-report questionnaire, formerly the Wisconsin Brief Pain Questionnaire [22], is a well-known measure of clinical pain. The questionnaire provides information about pain history, intensity, and location as well as the degree to which the pain interferes with daily activities, mood, and enjoyment of life. Scales (rated from 0 to 10) indicate the intensity of pain in general, at its worst, at its least, and pain “right now.” A figure representing the body is provided for the patient to shade the area corresponding to the location(s) of pain. Test-retest reliability for the BPI reveals correlations of .93 for worst pain, .78 for usual pain, and .59 for pain now. Research suggests the BPI has adequate validity [20].

Profile of Mood States (POMS) [23]

The original POMS contains 65 self-report items using the 5-point Likert Scale. Participants can choose from 0 (not at all) to 4 (extremely). Internal consistency for the Profile of Mood States was reported at 0.63 to 0.96 Cronbach alpha ratings. For the brief version, POMS-SF, the internal consistency ratings were 0.76 to 0.95. The correlation between the sub-scales and the total score in POMS and POMS-SF was calculated as 0.84. In addition, the POMS was correlated with scores on the Functional Assessment of Cancer Therapy scale and the Psychological Well-Being scale.

Patient Global Impression of Change (PGIC) [24]

The PGIC is a single-item, self-report question used and validated in multiple studies. It is a self-rated, 7-point, evaluative instrument for assessment of overall treatment experience and efficacy of chronic pain treatments.

Satisfaction questionnaires

All Experimental patients completed an online satisfaction questionnaires created for this study at the time of the 3-month follow-up assessment to assess their experience in using the electronic pain assessment program. Satisfaction ratings were obtained on the usability of the program, appropriateness of and satisfaction with content of the patient reports, and other features of the program. Providers completed a similar satisfaction questionnaire at the end of the study with most items rated on 5-point rating scale from 0 = “not at all” to 5 = “extremely.”

Statistical Analyses

The analysis examined differences in the discussion of pain-relevant topics during the visit based on patient interviews immediately following their clinical visit. It was expected that patients in the PainCAS condition would report a greater level of discussion with their provider of relevant topics than patients in the Control-non-PainCAS condition based on post-visit interviews with the RAs. Although the PainCAS assessment was not expected to impact pain and functioning outcomes, exploratory analyses examined differences between conditions on several measures of clinical outcomes including the POMS total and subscale scores, BPI severity and interference subscales, and CPTSS-level of treatment satisfaction from baseline to the 3-month follow-up assessment. Mean differences on the COMM and PGIC between conditions at the 3-month follow-up were also examined. Finally, for patients and providers exposed to PainCAS, level of patient and provider satisfaction with the PainCAS program was examined. Chi-Square Tests of Independence were conducted to examine group differences in the amount of discussion of relevant topics during the visit. Linear mixed models (LMMs) were used to test for differences between conditions on pain and functioning outcomes. Models for these secondary analyses included the following fixed effects: condition (PainCAS vs. Control), time (baseline and 3 months post-baseline), and condition-by-time interaction. Post-hoc tests were conducted to determine the specific differences between conditions. Satisfaction results were assessed based on percent satisfied. All data collected through the semi-structured interviews were summarized using standard descriptive statistical techniques, including calculating averages and percentages of participant responses.

Results

One hundred forty seven (N = 147) patients consented to participate in this study. We were not able to track the number of patients approached who declined to participate. Of the consented patients, 57% were initial patients and 43% were follow-up patients. Fifty nine percent were female and 84% were Caucasian. Of the 24 non-Caucasian participants, eight reported their race as African American, seven identified as Hispanic/Latino, one identified as Asian American, and eight identified as other or mixed race. Fifty one percent reported having more than one pain site. The mean duration of pain was 8.5 years (SD = 8.9). No significant differences in demographic information between the Control and the PainCAS groups and between the two centers were found.

Eighteen of the 23 clinicians who participated in Study 1 also participated in Study 2 (one clinic only participated in Study 1). The consented patients of those clinicians were interviewed in this study. Six clinicians were female, 12 were physicians, 5 were nurse practitioners, and one was a psychologist; 67% described themselves as pain medicine specialists, while the other 33% described themselves as ‘primary care’ or ‘other.’ Fifty-six percent of the providers described their practice as a hospital-based clinical setting, 39% practiced in an office-based clinical setting, and 5% reported their practice setting as ‘other.’ Sixty seven percent practiced in an academic clinical setting, 28% practiced in a community-based clinical setting, and six percent practiced in a research clinical setting. They averaged 20 years of clinical experience; the average number of years involved in treating patients with chronic pain was 12 years; and they believed that the majority of their patients (average 70%) had chronic pain.

During the post-visit interview, patients in the PainCAS condition reporting discussing five of 29 possible topics (17%) significantly more often compared with patients in the treatment-as-usual Control condition (Table 3). These topics included 1) worker’s compensation and/or disability, 2) involvement with the legal system, 3) family history of substance abuse, 4) taking pain medications differently than prescribed, and 5) past and present legal problems related to alcohol or illegal drug use. Counts of specific items within topic areas yielded no statistically significant differences between treatment-as-usual and PainCAS subjects for pain-related, psychosocial, and medication composite scores at the post-visit interview, although individuals in the PainCAS condition were significantly more likely to report having discussed substance use issues with their provider than patients in the treatment-as-usual condition (P < 0.05; Table 4). These results suggest that providers generally cover important pain-related topics during their visits, the addition of the PainCAS appears to increase the likelihood that more potentially difficult-to-discuss topics, like substance use, may be covered.

Table 3.

Post-visit interview discussion topics for Control and Experimental subjects

Baseline post-visit interview pain indicators totals for control and experimental subjects	Control % Present n = 75	Experimental % Present n = 72	${\mathit{\chi }}^2$	P
Pain related
Where in your body you feel pain	96.00	94.44	0.196	0.715
What your pain feels like	93.33	91.67	0.147	0.701
What you think is causing the pain	67.57	74.65	0.883	0.347
What makes the pain better and/or worse	81.33	80.56	0.014	0.904
How long you have had your pain	80.00	79.17	0.016	0.900
How often you feel pain	84.00	84.51	0.007	0.933
Rate your pain	81.33	88.89	1.648	0.199
Pain affects your ability to do daily activities	65.33	69.44	0.282	0.595
Being on workers compensation and/or disability	16.00	31.94	5.148	0.023
Treatment you have had for your pain in the past or steps you have taken to manage your pain	86.67	84.72	0.113	0.736
Side effects from pain medications	44.00	55.56	1.962	0.161
Other problems you are experiencing that are related to your pain	72.00	68.06	0.273	0.602
What your pain treatment goals are or what you expect from treatment	71.62	74.29	0.129	0.719
Involvement you have with legal system/law suits related to your pain	6.76	18.31	4.448	0.035
Psychosocial
Mental and emotional health	44.59	59.42	3.143	0.076
Substance use history
Illegal drugs you use now or used in the past	40.54	45.07	0.304	0.582
How much alcohol you drink now or used to drink in the past	37.84	46.48	1.110	0.292
Whether or not you smoke now or ever did in the past	53.42	59.15	0.480	0.488
Any use of prescription pain medications now or in the past without the direction of a health care provider	31.08	40.85	1.502	0.220
Any past or current substance abuse in your family	12.16	30.99	7.638	0.006
Ever take pain medications differently from how they are prescribed to you	12.00	38.89	14.101	<0.001
Legal problems you may have related to past or current alcohol or illegal drug use	2.67	13.89	6.171	0.013
Discuss the reason for giving a urine sample or results of a urine sample	17.33	27.78	2.302	0.129
Ask you to give a urine sample	32.00	33.33	0.030	0.863
Medication safety/treatment monitoring
Discuss medication safety practices with you	17.33	22.22	0.554	0.457
Recommend additional places to look for advice/information on medication safety	12.00	15.28	0.336	0.562
Recommend additional places to look for advice on dealing with pain	34.67	37.50	0.128	0.721
Discuss what you can expect at a follow up visit	82.67	73.61	1.769	0.184
Discuss medication safety practices with you	17.33	22.22	0.554	0.457

^¥Chi-square values, tests for difference between Control and PainCAS intervention groups. Significant values are bolded.

^£1 individual did not answer this question.

^Ł2 individuals did not answer this question.

^€3 individuals did not answer this question.

^§4 individuals did not answer this question.

Table 4.

Post-visit interview summary areas of discussion for Control and Experimental subjects

Areas of discussion in the clinic visit	Control N = 75 Mean ± SD	Experimental N = 72 Mean ± SD	t-Statistic	P-value
Pain-related	9.44 ± 2.83	9.92 ± 3.30	−0.94	0.348
Psychosocial	0.45 ± 0.50	0.59 ± 0.50	−1.78	0.077
Substance use history and medication misuse	2.36 ± 2.18	3.33 ± 2.97	-2.26	0.026
Medication safety and monitoring	1.47 ± 0.98	1.15 ± 0.14	−0.11	0.912

^†t-test differences—significant difference bolded.

^Ł4 participants did not specify psychosocial indicators.

Table 5 shows comparisons between the two conditions at baseline and 3-month follow-up on the POMS, BPI, CPTSS, PGIC, and COMM. No statistically significant differences were found on these measures between participants who used the PainCAS and those who did not.

Table 5.

Least squares means and standard errors for all outcome measures at baseline and 3-month follow-up

Variable	Control baseline (N = 75)	Control follow-up (N = 63)	PainCAS baseline (N = 72)	PainCAS follow-up (N = 65)
POMS Total	33.7±4.85	33.4± 5.08	30.9±4.92	34.5±5.07
BPI Pain Severity (mean)	5.1±0.24	4.5±0.25	5.7± 0.24	5.2±0.25
BPI Activity Interference (mean)	5.8±0.33	5.7±0.35	6.2± 0.33	5.9±0.35
CPTSS Information	18.1±0.45	17.2±0.48	17.5±0.46	17.6±0.48)
CPTSS Provider understood	20.5±0.50	19.1±0.52	19.6±0.50	19.4±0.52
CPTSS Provider attention	10.6±0.27	10.1 ±0.29	10.3±0.27	9.9±0.28
CPYSS Satisfaction	13.1±0.33	12.4±0.35	12.3±0.34	11.8±0.35
CPTSS Quality	19.6±0.56	19.0±0.60	19.0±0.57	17.9±0.59
CPTSS Agree	12.8±0.34	11.9±0.36	12.2±0.34	11.4±0.35
PGIC	–	3.3±1.38	–	3.6±1.43
COMM	–	9.0 ± 7.18	–	7.5 ± 5.43

^†All differences were nonsignificant.

^‡These measures were only completed at 3-month follow-up.

Patient Satisfaction

The majority of subjects (70.5%) indicated that the pain questionnaire report (PainCAS) was very easy or easy to understand. The majority also indicated that the information (76.5%) and educational material (64.7%) in the pain questionnaire report was helpful (either extremely, a lot, or somewhat). On average, using a scale of 1-5, with 1 being “Not at all” and 5 being “Extremely,” participants found the electronic pain questionnaire only marginally helpful in 1) improving communication with their health care provider (M = 2.3), 2) feeling involved in decisions about their treatment (M = 2.2), 3) understanding their treatment progress (M = 2.2), or 4) understanding their treatment plan (M = 2.1).

Clinician Satisfaction

Clinicians were also administered a satisfaction questionnaire after participating in this study. A majority of clinicians indicated that PainCAS provided information in an intuitive manner (88.9%; either extremely, a lot, or somewhat), increased documentation of patient pain and opioid risk assessments (77.8%), improved documentation of patient pain and opioid risk assessments (72.2%), facilitated timely access to assessment results (72.2%), increased clinical value of assessment reports (66.7%), allowed for more streamlined review of assessment (61.1%), impacted clinical decision making (61.1%), and affected opioid risk-related monitoring (55.6%). Interestingly, perceptions regarding the impact of the electronic pain assessment program on the patient-clinician interactions were less pronounced: 50% indicated that the PainCAS improved patient-clinician communication or enhanced shared decision making, while 45% reported the program as having facilitated tracking patient change over time, and 39% indicated the program helped to initiate discussion with the patient that might not have taken place otherwise or improved patient comprehension of treatment planning.

The automated risk-assessments offered in the PainCAS were appreciated by nearly three-quarters of the 18 clinicians, who felt that the automatically generated SOAPP/COMM scores were important to have in the PainCAS assessment reports (72.2%; either extremely or a lot). The majority also found the automatically generated monitoring recommendations (55.6%) and the educational materials (55.6%) in the PainCAS assessment reports to be helpful (somewhat to extremely). Sixty one percent felt that their practice was benefited by PainCAS, 77.8% indicated that they probably or definitely would continue to use the PainCAS system in their practice, and most (83.3%) probably or definitely would recommend PainCAS to their colleagues. Given the documented reluctance of clinicians to modify their workflow [3], half (50.0%) of clinicians in this study indicated that the PainCAS program was easier or much easier to use compared to their current assessment system.

Discussion

This study explored the benefits of a patient self-report, electronic pain assessment and tracking system called PainCAS. In the first study, preliminary chart-review findings suggest that posted reports on the patient medical record from the electronic assessment increases the presence of key pain assessment information that would not necessarily be found when incorporating notes from a traditional paper-and-pencil questionnaire. In particular, information on past treatments, adverse effects, psychological symptoms of depression, anxiety, and irritability, mental health treatment, past history of smoking, litigation, and substance abuse were documented more frequently in the PainCAS condition. The second study found evidence of increased discussion between patients and providers among those who completed the electronic pain assessment particularly around issues of substance abuse history and medication misuse. No differences were found between groups on the frequency of other topics of discussion. In general, patients assigned to complete the electronic pain assessment program and their providers found the program to be useful with positive clinical value. As expected, the electronic assessment and tracking program, which is not intended as an intervention, was not found to be directly beneficial in improving pain, mood, or function.

With the advent of heightened scrutiny in tracking aberrant drug-related behavior among individuals prescribed opioids for pain, there appears to be significant clinical value in including opioid risk assessment (in this case, SOAPP or COMM) among chronic pain patients prescribed or considered for opioids for pain [19,25]. Further, opioid risk assessment and monitoring is included in recommended guidelines for chronic opioid therapy [26]. Compared with practice-as-usual (no-PainCAS condition), highly significant differences in favor of an electronic pain assessment were observed for the presence of documentation of opioid risk assessment for baseline and follow-up assessment visits. Past studies have demonstrated an almost 80% increase in this documentation when using an electronic risk assessment [3]. Finally, documentation of a detailed chronic pain and opioid risk assessment demonstrating adherence to opioid treatment guidelines may go some distance toward reducing medico-legal exposure or regulatory scrutiny by identifying patients who are at high risk of opioid diversion, misuse, and/or abuse [27,28].

Data from the post-clinic interviews helped us to examine the effect that an electronic assessment program like PainCAS might have on face-to-face patient-clinician meetings. It was hypothesized that the presence of a comprehensive pain assessment report would have a positive effect on clinically-relevant topics of discussion during the clinic visit. It was not surprising to find that most topics related to the chief pain complaint, medications, and psychosocial concerns were observed in both treatment-as-usual and PainCAS condition. We did observe increases in some areas of discussion when patients and providers were exposed to the PainCAS program, particularly around topics of disability, substance abuse history, opioid misuse, and legal involvement. Unfortunately, we were unable to determine the accuracy of patient memory in recalling the meeting they had with their physician. We also know that differences exist in the amount of time each clinician expects to spend with their patient. Past studies that have examined the content of patient and practitioner interviews suggest that many of the areas of information found on comprehensive pain evaluation questionnaires are not raised by the patients and some providers failed to even look at patient reports [5,6]. Future studies may wish to record and monitor the interviews to help validate the information shared by the patient.

It is important to consider why substance abuse and legal problems were discussed more often among patients and physicians who had access to the PainCAS report, while no differences on the incidence of discussion of other topics were found between groups. Quite possibly, substance use issues were more likely to be raised because the SOAPP and COMM scores were very prominently displayed on the summary report page, and providers may have been made more aware of opioid risk assessment issues. Other issues, when present, such as pain-related litigation, are also prominently presented on the summary report page, which may have likewise raised attention to these issues for both provider and patient. In the treatment-as-usual condition, such topics may be more difficult to raise de-novo, whereas having the electronic program raise these issues may make it easier for providers to segue to discussion of these topics. The potential for positive interplay between formal risk assessment and in-person clinical discussions has been noted by others [29,30]. The other topics of discussion that were not different between groups are what would commonly be covered in a typical clinical interview (e.g., pain description, pain intensity). Again, future objective taping and outside ratings of doctor-patient interactions might shed light onto this issue.

There were a number of patient-reported perceived benefits of the electronic pain assessment program. They reported increased ease of use and helpfulness of the information supplied by the program report. However, they did not feel that completion of the assessment program improved communication with their provider. Nor did they believe that it improved their participation in the decision-making process. Although it was not anticipated that the pain assessment program would significantly improve pain-related outcomes, there is some evidence that a comprehensive assessment and frequent monitoring of pain can have a positive effect on coping [4,31]. During the short follow-up period (3 months), no impact on pain, mood, or function was found among patients who completed the electronic assessment compared with those who did not. On the other hand, one might expect that, over time, increased standardization of assessment on a large scale (e.g., in a large clinic or health care system) might lead to detectable improvements in quality of pain care.

Clinician participants perceived a number of benefits of the program including more streamlined assessment, timeliness of the assessment, help with their decision-making, improved documentation, and benefits of opioid risk assessment and educational material. The program was perceived to be less beneficial in improving patient-clinician communication, tracking patients over time, and providing patients with a comprehensive treatment plan. PainCAS program was perceived as most valuable for helping to assess patients who might be considered for opioid therapy or who are taking opioids, but not as beneficial for patients who are not taking opioids for pain. When asked, providers felt that their traditional assessment approach was as easy to use as the electronic pain assessment. This finding is consistent with other studies indicating that any change in an organizational process is likely to be met with a certain amount of resistance and highlights the slow process of adapting to change [3,6]. Prior research has demonstrated the difficulties associated with successful integration of any new innovation into the clinical process. In a study on dissemination of guidelines for the treatment of chronic pain [5], prescribers expressed resentment that treatment guidelines implied they could not manage their patients successfully, or they felt a disease management program was a “cookie-cutter” approach to treatment with no allowance for decisions based on subjective impressions of the patients. Even when an innovation, such as electronic diaries with summary data is perceived as useful, this perception does not necessarily result in changes in treatment practice or outcome [6]. A separate study on investigating issues associated with integrating the PainCAS into the clinical workflow [3] found that perceived benefits of the PainCAS were primarily related to ease of use, automation, and incorporation of information in an electronic format into an EMR. However, challenges and barriers to implementation included resistance of office staff and prescribers to changes in their regular workflow, and concerns about patient compliance and capabilities. Such barriers are likely to have accounted for only half of providers rating that PainCAS as “easier or much easier” than the workflow with which they are already accustomed. Despite such barriers, the results of this study suggest that the value and benefits of this added information would likely outweigh the problems and challenges that implementation of the program may entail [32,33]. Successful integration of any new innovation into the clinical process requires a commitment to change by providers, staff and administrators, who must be convinced of the value of the change and its impact the work load of personnel within a busy clinic practice.

It should be highlighted that the PainCAS is designed for initial evaluation and repeated follow-up assessments. The initial PainCAS report, which in this study took a median time for patients to complete of 34 minutes, gathers a significant amount of information that is not included in the follow-up assessment, which took about 13 minutes to complete. The frequency with which follow-up reports are requested from patients would be determined by different clinical and administrative standards of each provider and/or clinic. Although each provider and clinic may employ different standards for administration of the PainCAS tool, it is thought that the follow-up assessments would not need to be administered any more frequently than once a month, and, among low risk patients, could be administered a minimum of every 12 months.

There are a number of other limitations of this study that need to be highlighted. First, we recruited a select number of patients in each center and a limited number of providers. We know that there are individual differences in how each provider engages with their patients and, unfortunately, with the limited number of participants, we could not identify any demographic variables that might have been significantly associated with more favorable or unfavorable responses. A follow-up study with more providers in different types of centers and in various regions of the country would be helpful. Second, the post-visit interviews relied on patient recall and memory of the session they had with their physician. Even though the interviews were conducted right after the clinic visit, memory of what was discussed was based on self-report and may not have always been accurate. We also did not determine whether the physician looked at the PainCAS report before or during the patient visit or whether the patient brought up the assessment with the physician at the time of the visit. Future studies would benefit from including more clinicians and obtaining objective recorded information about the amount of time spent with the patients and what specific information was discussed. Third, all the subjects had to have an email account and a home computer. Although we did not find any demographic differences between those who participated in this study and those who did not, these selection criteria could have served as a selection bias by excluding some types of patients in the study. We did not track socioeconomic status of patients (e.g., income, education, occupation, or place of residence) which might be relevant to patients’ use of an electronic assessment, although it should be emphasized that patients without access to a computer can complete the assessment in the clinic. Another limitation was that the electronic assessment program was not routinely used for repeated follow-up assessments. Thus, some patients were not consistently tracked over time, and this may account for some lower satisfaction ratings of clinicians in using the PainCAS program.

Finally, as an IRB-approved clinical trial, study patients were volunteers who provided informed consent to be in the study. We were unable to track the number of patients who declined to participate in Study 2, which is a limitation of the study. Patients who volunteered may not represent all chronic pain patients for whom the PainCAS was developed. It is possible that patients at high risk may have refused to participate, although risk assessment was not specifically mentioned in the consent form. Patients were told they would be asked to complete “a computer-administered assessment for chronic pain patients.” It remains possible that some patients may have refused to complete the PainCAS or the risk assessments when risk assessments are selected by the clinician. Clinically, when administered not as part of study, refusal to comply would most likely be handled like any other such refusal, as for instance, refusing a urine drug test (UDT). Such refusal may be indicative of the patient having “something to hide” [34,35]. Further, it should be noted that the PainCAS is currently being used in clinics across the country for everyday clinical purposes. As of December 2015, 19 clinics have administered the initial or follow-up PainCAS assessment to 5,966 patients in ongoing practice (i.e., not as part of a study). In our experience, when providers and clinic staff present the PainCAS as an expected part of the course of treatment, patients are generally compliant.

Despite these limitations, this study suggests that significant differences in favor of an electronic pain assessment (i.e., the PainCAS) condition were found, particularly around documentation of opioid risk assessment and patient-doctor discussion of substance abuse issues. These preliminary results suggest that an electronic pain assessment program has the benefit of increased documentation of key elements of pain patient information in the medical record and increased discussion of substance abuse issues in the patient-physician encounter. Although use in this study of the PainCAS was not associated with patient outcome in this brief follow-up, the potential for standardization of assessments leading in the long-term to improved quality of care remains.