Development of an electronic database for Acute Pain Service outcomes

From mitigating complications during hospital stay to reducing the incidence of chronic pain, improving pain management positively impacts short- and long-term outcomes of treatment. Quality assurance has become the focus of many health care programs in an effort to confront the challenges presented by pain and its management. The analysis of treatment strategies and patient responses form the basis of a reflective, evidence-based practice, which can raise awareness of about the importance of adequate pain management. The capabilities of an electronic database can be exploited to organize large amounts of data, which can then be used to characterize symptoms and analyze treatment responses. However, the types of data to collect, the burden of data collection on workers, and costs must be considered before implementing an electronic database for research and/or everyday practice. This study tested the relevance and usability of an electronic database in an acute pain service by assessing several variables important to database development and evaluation.

Abstract

BACKGROUND:

Quality assurance is increasingly important in the current health care climate. An electronic database can be used for tracking patient information and as a research tool to provide quality assurance for patient care.

OBJECTIVE:

An electronic database was developed for the Acute Pain Service, University of Alberta Hospital (Edmonton, Alberta) to record patient characteristics, identify at-risk populations, compare treatment efficacies and guide practice decisions.

METHOD:

Steps in the database development involved identifying the goals for use, relevant variables to include, and a plan for data collection, entry and analysis. Protocols were also created for data cleaning quality control. The database was evaluated with a pilot test using existing data to assess data collection burden, accuracy and functionality of the database.

RESULTS:

A literature review resulted in an evidence-based list of demographic, clinical and pain management outcome variables to include. Time to assess patients and collect the data was 20 min to 30 min per patient. Limitations were primarily software related, although initial data collection completion was only 65% and accuracy of data entry was 96%.

CONCLUSIONS:

The electronic database was found to be relevant and functional for the identified goals of data storage and research.

Increasingly, quality assurance projects are becoming a focus in health care. Quality assurance is achieved through reflective and evidence-based practice, and includes the analysis of both treatment patterns and patient responses (1). In an attempt to improve patient outcomes and address the barriers associated with pain management, the American Pain Society’s Quality of Care Committee (2005) defined the following five recommendations for quality improvement:

• Early recognition and treatment of pain

• Involving patients and their families in pain management plans

• Improving treatment patterns among health care professionals

• Frequent reassessment and adjustment of treatment plans

• The ongoing monitoring of processes and outcomes in pain management (1).

The complexity of pain and its management presents many challenges. Improving pain management for patients will positively impact short- and long-term health outcomes such as complications experienced during hospital stay and the incidence of chronic pain (2). By analyzing current treatment strategies and establishing explicit policies and protocols for the regular assessment, treatment and documentation of pain, an acute pain service can raise awareness about the importance of adequate pain management (1).

One tool that enables quality assurance is an electronic database containing patient information. A database can be used to log relevant patient characteristics, as well as treatment and outcome data. Large quantities of organized data can then be used to analyze patient populations, characterize symptoms and evaluate responses to treatments as part of observational studies, or as part of the planning and justification of prospective investigations (3).

Constructing a database for the purposes of quality assurance involves careful consideration of information to include. Variables should be selected based on their validity and reliability, the burden on data collectors and the costs associated with the collection (4). In general, data collected can be categorized as demographic (such as age and sex), clinical (such as admission date or surgical procedure) and outcomes (such as adverse events). Other variables relevant to an acute pain service include the assessment of pain, treatments used and measures of effectiveness (such as satisfaction with pain management and the occurrence of side effects).

A pilot test of the database can identify the burden of data collection, the accuracy and completeness of the data and its overall usability (4). Assessing the strengths and weaknesses in the design and function will assist in improving the electronic database as a tool for quality assurance and research. This is an important step before incorporating the electronic database into everyday practice.

PURPOSE OF THE PROJECT

The goal of the present project was to create an electronic database and test its relevance and usability. The database will be used to identify strengths and limitations of current pain management strategies employed by the Acute Pain Service (APS), University of Alberta Hospital (Edmonton, Alberta) to aid in the development of new protocols to meet standards of care and support research.

The project addressed the following questions:

For database development:

• What are relevant patient demographic variables in the management of acute pain?

• What are relevant clinical variables in the management of acute pain?

• What are relevant variables for the assessment, treatment, and outcomes in the management of acute pain?

• What is a functional software and format for the electronic database?

For database evaluation:

• Is the electronic database relevant and useable for the APS?

• How did the electronic database perform as an information storage system and research tool for quality assurance?

• What are the strengths and limitations of the electronic database?

METHODS

Database development

A review of the current literature (2000 to 2009) was conducted to determine variables to include in the database. Electronic databases searched include Medline In-Process & Other Non-Indexed Citations, MEDLINE (1950 to present), CINAHL, and Health and Psychosocial Instruments. Results were limited to published articles involving adults and written in English. Manual searches of the reference lists of relevant articles were also conducted. Articles were selected to support the inclusion of variables relevant to the assessment and management of acute pain. The variable list was submitted to the members of the APS (eight anesthetists, one nurse practitioner and two registered nurses [RNs]) for feedback and endorsement.

Once feedback was obtained, the variable list was used to guide the creation of a data collection form. The data collection form was trialled for two weeks by the primary data collectors – the two RNs with the APS. They were then asked about the time required to collect patient’s data to determine the burden on daily workload. The nurses were also interviewed for their opinions on the importance of data collection relative to other daily tasks including direct patient care, patient and staff education, and administrative responsibilities. The feedback was used to develop a protocol to assist with the implementation of the data collection form.

A survey of current data management practices of acute pain services across Canada (n=8) was concurrently conducted via an electronic special interest group concerned with issues in pain management. Members were asked whether they collected patient information, what information was collected, how the information was managed and how it was used. Responses were voluntary and collected over a period of two weeks. These responses were considered when investigating the software programs for the APS’s electronic database.

Several electronic database software programs were investigated. A software program was needed for tracking overall trends in patient demographics and health outcomes, not individual hospital stays per se. Multiple programs, some pain specific, are available to store and analyze data, but one software suite that included programs to create data collection forms, enter data and perform statistical analysis was found. PASW Data Collection Author (creates data collection form and the structure of the data entry and storage processes), PASW Interviewer (data entry program), and PASW Statistics (data analysis program) were eventually selected as the software programs for the APS’s electronic database based on its functionality, cost and the operator’s familiarity with the program. (PASW, formerly SPSS, is now IBM SPSS [IBM Corporation, USA].)

The creation of the database began with defining an electronic data file. Considerable time was required to learn how to create an electronic data file that would effectively capture all of the chosen variables in a user-friendly format. Several trial versions were created before finalizing the format for deployment to the data collection program. The final electronic data file included the demographic, clinical and outcome variables selected for the APS’s database. Due to the nature of the software programs, a patient’s entire case with the APS can be entered, but not in ‘real time’. Each case is entered once the patient is discharged from hospital because all of the information must be entered at one time. Point of care entry would assist with ease of data entry but is not essential because the data are used primarily for quality assurance as opposed to an electronic medical record. A flowchart outlining the steps in database development is shown in Figure 1. A second flowchart illustrating the basic design of the database is shown in Figure 2.

Figure 1).

Flowchart for the development of an electronic database. Based on the Plan Do Study Act model from the Institute for Healthcare Improvement (16)

Figure 2).

Basic design of the electronic database. APS Acute Pain Service; PCA Patient controlled analgesia

Database evaluation

For the initial evaluation of the electronic database as an information storage tool, the data entry process was trialled in a pilot study. Past information collected by the APS’s RNs from June 2008 to July 2009 was available for entry. Using data already collected allowed for a reflection on data collection practices to help guide development of the new database. Previously, the nurses collected information on patient demographics, clinical variables and health outcomes, similar to what was currently proposed. The variables collected were based on information believed to be useful for monitoring the daily progress of patients; no formal analysis had been conducted historically.

Data entry was completed over several days using the Interviewer program. The previously collected information (June 2008 to July 2009) was entered into the electronic database using the Interviewer program. Once the data were entered, the file could be exported into Statistics for analysis. Completeness of data collection and accuracy of data entry were then examined by randomly assessing five cases at (approximately 10% of the cases entered). Smaller, more frequent audits resulted in higher-quality data (5). Missing data were assessed as a blank field; completeness was calculated as the number of fields with data divided by the number of fields that could have been collected. Accuracy was calculated as the number of correctly entered fields divided by the total fields entered. To improve accuracy and provide guidance for additional data entry personnel, protocols were created for standardized data collection and entry.

To evaluate the database as a research tool, the retrospective data were used for the pilot test to examine analgesic efficacy for patients involved with the APS who underwent a nephrectomy. The information in the database was used to assess the severity of pain, to identify analgesic modalities employed and to compare the efficacy of an epidural versus a transversus abdominis plane (TAP) nerve block versus neither for managing pain following surgery.

Statistical analysis was used to describe patient characteristics and to compare the effectiveness of three analgesic modalities: epidural catheters, TAP nerve block catheters and opioid-only therapies. The three groups were compared for differences in distribution of age, sex and history of chronic pain. The patient outcomes compared included pain severity at rest and with activity, sedation scores, the incidence of side effects during the time with the APS and length of hospital stay. χ² or independent t tests were used, as appropriate, for group comparisons.

RESULTS

Database development

One of the strengths of the database was selecting variables that were supported by current literature and with a specific purpose in mind. The APS wanted an electronic database to track pertinent patient characteristics as well as monitor certain health outcomes reflective of treatment response. This goal resulted in the list of demographic, clinical and pain management outcome variables (Table 1).

TABLE 1.

Variables of the Acute Pain Service electronic database

Variables	Measure
Demographic variables
Age	Years
Sex	Male/female
Height/weight/BMI	cm, kg, kg/m2
Marital status	Single/married/divorced/widowed
Race	Caucasian/Aboriginal/other
Born in Canada?	Yes/no
Clinical variables	Measure
Admission date	Day/month/year
Discharge date	Day/month/year
Procedure/surgery date	Day/month/year
Procedure/surgery	Describe
Surgeon	Name
Date of APS consult	Day/month/year
APS anesthetist	Name
APS nurse	Name/s
History of chronic pain	Yes/no (defined as persistent pain >3 months)
Chronic pain	Describe chronic pain condition
Regular opioid use	Yes/no
Comorbidities	Diabetes, heart disease (CAD, PVD, stroke, CHF, MI, HTN, congenital, etc), depression, other chronic disease
Pain management outcomes
Modality	Epidural, nerve block, PCA, other
Site	Treatment site
Drug	Medication used
Dose/regimen	Descriptive dose in mg, mL
Adjuncts given	Acetaminophen, ibuprofen, ketorolac, gabapentin, pregabalin, nabilone, other
Site assessment	Intact, erythema, pain, drainage, hematoma, infection
Catheter assessment	Intact, occlusion, leak, disconnect, migration in, migration out
Motor block	0–5 muscle strength scale
Sensory block	Yes/no
Pain severity – rest	0–10 NRS
Pain severity – activity	0–10 NRS
Impact of pain on sleep	0–10 NRS (How much has pain affected your sleep?)
Impact of pain on mood	0–10 NRS (How much has pain affected your mood?)
Impact of pain on appetite	0–10 NRS (How much has pain affected your appetite?)
Impact of pain on mobility	0–10 NRS (How much has pain affected your mobility?)
Sedation scale	0, 1, 2, 3, S
Adverse effects	Nausea, vomiting, pruritus, constipation, urinary retention, delirium/confusion, myoclonus, respiratory depression, LAT, hypotension
Satisfaction with pain management (overall)	0–10 NRS

APS Acute Pain Service; BMI Body mass index; CAD Coronary artery disease; CHF Congestive heart failure; HTN Hypertension; LAT Local anesthetic toxicity; MI Myocardial infarction; NRS Numerical rating scale; PCA Patient controlled analgesia; PVD Peripheral vascular disease; S Sleeping

Demographic variables, such as age, have been demonstrated to influence the perception and reporting of pain severity. Older adults are typically more ‘stoic’ and under-report their pain as opposed to younger adult patients who will rate pain as more severe (6–10). Similar findings have been reported when comparing male and female patients. Women often report more severe pain and are at a higher risk of developing complications related to chronic pain (1,6,11,12). The same can also be observed when considering race as differing expectations influence pain perception and coping skills (1,13).

Other patient characteristics are more predictive of pain syndromes and related disorders. For example, a high body mass index has a correlation with increased knee pain and the development of osteoarthritis (14). This may help identify patients who are at risk for more severe pain postoperatively and, thus, at risk for pain related complications.

Tracking various clinical variables helps to organize information in the database. Categorizing the surgery or the procedure the patient has undergone allows for isolation of specific patient populations in the Statistics file for analysis and treatment comparison. Recording admission and discharge dates provides the patient’s length of hospital stay and can also be used to assess workload of the APS throughout the year. Assigning the patients to an anesthetist and nurse will help with performance reviews for members of the APS. For example, the statistics for an APS anesthetist can be examined to assess the number of nerve blocks performed, the incidence of adverse effects and patient satisfaction ratings.

Logging information about a patient’s medical history is useful in identifying at-risk populations as well as ensuring these patients are properly treated once identified. A history of chronic pain is a predictor of moderate to severe postoperative pain (6,7,11). It is important to identify these patients to ensure their postoperative pain regimen includes consideration of their preoperative medications. Other comorbidities such as diabetes, heart disease and depression have chronic pain conditions associated with them and may influence treatment decisions (7,9).

Another goal described by the APS for the database is the ability to track the use of pain management treatments and compare patient outcomes. Because of this, the analgesic modality, the nerve block or epidural catheter site (as applicable), the medications prescribed and the dose administered were included in the data collection form. To be able to track any effect of adjunctive medications (such as acetaminophen) on patient outcomes, only adjuncts administered, as opposed to prescribed, are logged.

Patient outcomes and response to treatments are captured through assessments of pain severity at rest and with activity, as well as the impact pain has on patient life. Patients are asked to rate their pain on an 11-point numeric rating scale (NRS), with zero representing no pain and 10 the worst pain imaginable. The NRS is both sensitive and specific and is easily administered to a diverse population as compared with other rating scales (15). Other responses such as motor and sensory block help to assess treatment efficacy by indicating whether or not the desired effect is achieved. Catheter and site assessments along with sedation scores and side effects track the incidence of adverse events. Patient satisfaction, although notoriously positively skewed, provides feedback on how the patient perceived his or her care. Patients are asked on a scale of 0 to 5 how satisfied they are with overall pain management at the end of the APS involvement.

The primary data collectors, the two RNs with the APS, were asked to complete a questionnaire regarding the data collection form. On average, the nurses were spending 20 min to 30 min per patient per day on data collection. This included the patient assessment needed to collect the data; assessment and data collection were undertaken simultaneously. The overall usability of the data collection form was rated at 8/10. The impacts of pain on sleep, mood, mobility and appetite were described as the least practical to collect because some patients had difficulty understanding the question. When asked if anything should be changed about data collection, the nurses commented on the amount of time required. They did, however, recognize the importance of the data collected and that the time constraints might be more directly related to increasing workload in general.

Limitations with the electronic database were found to lie primarily with the software itself. First, considerable time was lost in attempting to load the originally purchased software package onto a laptop with an incompatible operating system. The software was originally labelled as compatible with the current operating system on the laptop; the software company remedied this problem by providing an updated software suite. The next issues were identified once the electronic data file was deployed to the Interviewer program to allow for data entry. The electronic data collection form could only be deployed in the ‘Live Interview’ format, generally reserved for conducting interviews over the phone. The ‘Data Entry’ format, which would normally be used, would not allow for the entry of the variables collected daily due to a programming error that the company is aware of and hopes to resolve in the future. Unfortunately, data entered through the Live Interview format cannot be edited, deleted, or even viewed. Any data entered must be viewed or edited once converted to a statistics file. This remains problematic and will hopefully be remedied by the software company.

A second limitation with the database is the time required to enter and audit data. Regular, daily data entry will be required to maintain the database. Also, some patients’ treatments and outcomes are difficult to enter into the database. For example, patients who are non-English speaking are assessed using tools other than the NRS. The database currently accommodates data using the NRS only; data fields for these patients are entered as ‘Not Available’, an inaccuracy. The assessments and database do, however, capture the majority of patient’s data in a meaningful and accurate way.

Database evaluation

Information collected previously (June 2008 to July 2009) by the APS’s RNs was entered into the electronic database to facilitate its evaluation. The collection of this retrospective data was sporadic due to the workload of the nurses and the low priority assigned to the task. The anesthetists with the APS, who assess and monitor the patients, had never formally collected data on the weekends. Because of this, a large proportion of missing data was found. In addition, inter-rater reliability could not be assumed because no standard protocol for data collection existed. Buy-in from members of the APS was obtained during a team meeting and data collection protocols were created to address missing data.

To assess the degree of missing data and accuracy of data entry, five random cases were chosen and evaluated. Completeness of data collection was calculated as 65%. This was anticipated due to the issues described with the retrospective data. A new protocol was thus created to standardize data collection. In addition, the APS anesthetists (n=8) also agreed to begin data collection on weekends.

Data entry accuracy was also evaluated using an audit of five random cases. Accuracy, in this case, was calculated as 96%. Variance is, in part, explained by the iterative process required for learning data entry. Subsequently, a data entry protocol was defined. This protocol will standardize data entry in the case of multiple data entry personnel and eliminate subjective interpretation in the transcription process. Incorporating an audit process for ‘data cleaning’ has assisted in both the evaluation of the database as well as establishing a plan for ongoing assessments.

The electronic database was further evaluated by using the retrospective data to examine its capacity for research. The overall effectiveness of the analgesic modalities used for patients following a nephrectomy – epidural and TAP nerve block – were compared. The sample comprised 58 patients identified as having undergone a nephrectomy at the University of Alberta Hospital between June 2008 and July 2009 and were under the care of the APS. Ten patients had their pain managed with an epidural catheter, 44 received a TAP nerve block and four were prescribed opioids via intravenous or subcutaneous injections. For the purposes of comparison, only the epidural and the nerve block patients were included in the comparison because the opioid-only patients were a small and heterogeneous group.

When comparing the two groups for differences in demographic and clinical variables, the two groups were similar in age and history of chronic pain (Tables 2 and 3). History of chronic pain, however, should be interpreted with caution because the epidural group had 70% of the data missing. The proportion of males in the nerve block group was significantly higher than the proportion of males in the epidural group (χ²=4.4; P=0.04), which could account for some variance in the pain management outcomes (Table 4).

TABLE 2.

Age (years) of nephrectomy patients

Group	n	Mean±SD	Minimum	Maximum	t* (P)
Epidural	10	53.6±17.5	18	79	−0.32 (0.74)
Nerve block	44	55.0±11.0	26	74	–

^*Student’s t test

TABLE 3.

History of chronic pain

Group	n (missing)	Valid, %	χ2 (P)
Epidural	1 (7)	33.3	0.02 (0.88)
Nerve block	12 (12)	37.5

TABLE 4.

Sex of nephrectomy patients

Group	n		%		χ2 (P)
	Male	Female	Male	Female
Epidural	2	8	20.0	80.0	4.4 (0.04)*
Nerve block	25	19	56.8	43.2

^*P<0.05, two-tailed

Pain scores at rest and with activity on postoperative day 1 were significantly lower for the epidural group (t= −2.1; P=0.04 and t= −2.6; P=0.01). No difference was observed for day 2 pain scores at rest and with activity or for length of hospital stay (Table 5). The two groups could not be compared for the absence of side effects or the incidence of moderate to severe sedation because the missing data resulted in frequencies too small for comparison (Table 6). These results, although not strong enough to indicate practice change directly, do support the need for further study. A significant difference in pain scores could warrant a randomized comparison of the two treatments to explore further the risks and benefits of epidurals and nerve blocks for this patient population. This is precisely one of the goals for the use of the electronic database.

TABLE 5.

Comparison of pain scores and length of stay

Variable	Group	n (missing)	Mean±SD	Min	Max	t* (P)
NRS: Pain at rest day 1	Epidural	9 (1)	1.3±2.2	0	6	−2.1 (0.04)†
	Nerve block	39 (5)	3.5±2.9	0	9
NRS: Pain at rest day 2	Epidural	7 (3)	1.6±1.9	0	5	−1.3 (0.22)
	Nerve block	28 (16)	2.6±2	0	6
NRS: Pain with activity day 1	Epidural	5 (5)	3.0±1.6	1	5	−2.6 (0.01)†
	Nerve block	27 (17)	6.4±2.8	1	10
NRS: Pain with activity day 2	Epidural	5 (5)	3.8±1.5	2	6	−2.0 (0.06)
	Nerve block	19 (25)	5.8±2.1	2	10
Length of stay, days	Epidural	10 (0)	5.5±1.6	4	9	−1.5 (0.14)
	Nerve block	44 (0)	6.6±2.2	4	15

^*Student’s t test;

^†P<0.05, two-tailed; Max Maximum; Min Minimum; NRS Numerical rating scale

TABLE 6.

Comparison of side effects and sedation scores

Variable	Group	n (missing)	Valid, %
No side effects day 1	Epidural	3 (5)	30
	Nerve block	18 (18)	40.9
No side effects day 2	Epidural	2 (6)	25
	Nerve block	16 (21)	40
Sedation score ≥2 day 1	Epidural	0 (5)	0
	Nerve block	1 (23)	4.8
Sedation score ≥2 day 2	Epidural	0 (9)	0
	Nerve block	2 (32)	16.7

From the beginning to the end of the evaluation, the database was easy to use. Data entry was fluid and required approximately 5 min to enter an entire case. Once entered, the data was automatically coded and transferred into the statistics file. Isolating the patient population required the user to know the surgeries of interest, and obtaining descriptive statistics was a matter of selecting the appropriate variables within the program’s analysis functions. Comparing the two treatment groups did require some familiarity with how the variables were coded but was manageable with training. Based on this evaluation, the electronic database will meet the APS’s goal of developing research questions and identifying at-risk patient populations.

DISCUSSION

Having stakeholder buy-in and extensive planning at the outset for the intended use of the database was key to the success of this project. The APS required a way to track patient outcomes and engage in reflective, evidence-based practice to ensure quality patient care. Designing and implementing an electronic database was a practical way to meet these goals.

The electronic database is an effective data storage tool. Its design allows for the continued growth of both the number of patients and the variables included. The data entered are highly organized and individual cases can be identified as needed. Subgroups of patients can be isolated for analysis based on a wide variety of variables such as sex, race or surgery type. The data can also be evaluated to identify at-risk populations, to compare treatment efficacy or to support potential prospective research studies. This facilitates both academic endeavours, such as randomized trials, and helps justify the existence of the APS.

Limitations of the database that were identified included the burden of data collection on nursing and physician workloads and missing data. Thus, efforts were made to include input from members of the APS and to create a data collection form that was both reflective of the desired variables and intuitive to use. Despite this, data collection may continue to be incomplete and the APS will need to reevaluate the priority of different variables in the future. In addition, available resources may also affect the completeness of data collection.

As well, statistical analysis using the database does require some familiarity and skill with the software. The user must first have an understanding of general statistical concepts. Time must then be devoted to learning the functions of the software and how to manipulate the data to perform the desired calculations. PASW Statistics is a broadly used statistical software program, but does require a considerable time investment.

Finally, there were problems encountered with the software programs. The programming errors affected the usability of the electronic database to some extent; however, alternative solutions were found to compensate for this. As software updates become available, this may be resolved in the future. Given the current limitations of the software, long-term sustainability of the database may be affected.

Suggestions for further development of the database include modifying the data collected by adding variables pertinent to quality assurance (complications on insertion of nerve blocks/epidurals, use of ultrasound, and the level of sensory change on assessment) and removing the less practical (patient born in Canada). Expanding the assessments portion to include tools other than the NRS would accommodate non-English speaking patients or older patients unable to score their pain. Including the number of doses ordered for adjuvant medications would enable the calculation of the percentage of medication administered. The effects of pain on mood and function remain important data to collect, but difficult to assess/report in a database. Other suggestions would be to modify the database or change to a different software program to allow for point of care data entry. Suggestion for a further study of the database would be to have a ‘before’ and ‘after’ comparison to determine whether the data collection and entry protocols affect missing data and data entry accuracy.

CONCLUSION

The data storage and analysis functions of this electronic database allows the APS to evaluate patient care practices. Establishing standards for data collection, entry and evaluation was an important step in justifying the existence of the APS and its continued role in patient care. The electronic database will play a critical role in ensuring evidence-based practice. With this, the APS can demonstrate a focus on quality assurance and patient-centred care.