Comparison of Measures of Pain Intensity during Sickle Cell Disease Vaso-Occlusive Episodes

Abstract

We aimed to determine the minimal clinically important difference (MCID) in pain severity and agreement between the visual analog scale (VAS) and the verbal numeric rating scale (NRS) in people with sickle cell disease experiencing an acute vaso-occlusive episode in the emergency department (ED). In the COMPARE-VOE trial (NCT03933397), participants were administered the VAS (0-100), NRS (0-100), and descriptor scale (a lot better, a little better, same, a little worse, much worse) every 30 minutes while in the ED. We analyzed data from 100 participants (mean age 30.2 years; 61% female). We calculated the mean differences and 95% confidence intervals (CI) between current and preceding scores when the participant reported a little worse or a little better pain for each scale (255 VAS and 150 NRS observations) to assess the MCID for the VAS & NRS. Pearson correlation and the Bland Altmann method were used to assess the agreement among 411 paired VAS & NRS observations. Our results indicated that the MCID for the VAS was 8.77 mm (95% CI: 7.43 mm, 10.83 mm) and the NRS was 8.29 (95% CI: 6.47, 11.60). The VAS & NRS scales had a correlation of 0.88 (p < 0.001). The Bland Altmann indicated a mean difference of −4.6 ±1.96 and the 95% limits of agreement ranged from 20 to −29. Despite high correlation, there was considerable variability of agreement between the VAS and NRS scales, indicating that these scales are not interchangeable to assess pain during a vaso-occlusive event.

Introduction

Pain is an impactful and detrimental symptom of sickle cell disease (SCD) and the leading cause of emergency department visits for people living with this inherited red blood cell disorder. In the United States, SCD affects approximately 100,000 people, predominantly of African, Hispanic, or Latino descent.¹ This monogenic disease is caused by a variant on the hemoglobin subunit beta gene that leads to an altered type of hemoglobin, changing the shape of red blood cells, causing them to become both stickier and more fragile, leading to increased hemolysis, anemia, and initiating an inflammatory cascade within blood vessels.² These alterations lead to clumping of the red blood cells with resultant vaso-occlusion of the vasculature, referred to as vaso-occlusive episodes (VOEs).³ People experiencing VOEs generally present at the emergency department with severe acute pain and complications (e.g., acute chest syndrome, stroke, infection) that require rapid evaluation and treatment.⁴

Clinical guidelines for treatment of VOEs include frequent re-assessment of pain and re-administration of pain medications until pain is relieved.^{5, 6} The use of objective and valid measures to determine improvement or worsening of pain as well as treatment efficacy is important. Although measures of pain intensity are unidimensional, they are quick and simple tools that require minimal training to utilize in the fast-paced ED environment. Additionally, these tools are useful for tracking changes in pain intensity after clinical and research interventions.

The verbal numeric rating scale (NRS) and visual analog scale (VAS) are among the tools most commonly used to measure pain intensity. The NRS is often used clinically, and the VAS is frequently used in research trials. Responses on these scales are often highly correlated; however, prior research has shown that ratings can differ depending on the scale used.^{7, 8} Additionally, interpretation of the minimal clinically important difference (MCID) in pain intensity provides important insights on the size of treatment effects but is context-specific.⁹ Previous research involving adults with SCD experiencing a VOE in the ED has demonstrated a MCID to be 13.5 mm on the VAS.¹⁰ Similarly, a MCID of 13 mm on VAS was identified in a sample of adult patients with acute pain from trauma who were being treated in the ED.¹¹ However, limitations of this research include small samples (N = 48-74) with only 80-91 observations used to determine MCID.^{10, 11} Notably, no MCID for the NRS has been established in adults with SCD.

In the context of a larger randomized controlled trial, we had the opportunity to administer different pain scales (VAS, NRS, and descriptor scale) every 30 minutes during ED visits by adults with SCD for the treatment of VOE. The current report aimed to (a) determine a MCID in pain severity for both the VAS and NRS, and (b) assess agreement between the VAS and verbal NRS among adults with SCD experiencing an acute VOE in the ED.

Methods

Parent Trial Design, Setting and Participants

We conducted a randomized controlled trial including a total of 104 adults with SCD, methods have been previously reported but are summarized here.^{12, 13} Individuals who consented were enrolled in the parent study, COMPARE-VOE, which compared changes in pain scores for individuals randomized to receive either a weight-based or patient-specific protocol to treat VOE pain in the ED.¹³ The project was conducted in six EDs across the United States. While the parent trial was based on years of research identifying the needs of improving ED pain management, there was no direct involvement of patients or the public in conceptualizing or designing the study.

Inclusion criteria for the current study were as follows: 1) adults ages ≥18 with SCD and the following hemoglobin genotypes: SS, SB⁰, and SB⁺; 2) consented, randomized and pre-enrolled in an outpatient setting prior to an ED visit; 3) after randomization in the outpatient setting, a weight-based or patient-specific protocol was written by the patient’s SCD outpatient provider and placed in the electronic health record (EHR); 4) patient had a subsequent ED visit for a VOE after the weight-based or patient-specific protocols were available in the EHR. Exclusion criteria were as follows: 1) the patient-specific protocol did not include parenteral opioids; 2) patient had an outpatient prescription for methadone or buprenorphine to treat chronic pain (as it is difficult to determine an equianalgesic dose of morphine sulfate or hydromorphone); 2) patient presented to ED for complaint other than a VOE (e.g., stroke, acute chest pain); and 4) patient highly suspicious for COVID-19 at the time of ED presentation (as COVID-19 may affect pain scores).

A total of 344 patients were screened in the outpatient setting, 328 of whom consented to participate; of these, 104 had a subsequent ED visit after having been enrolled and had a weight-based or patient-specific protocol written and entered into the EHR. Among the 104 enrolled patients, 3 were confirmed to be positive for COVID-19 during their ED visit, and 1 withdrew informed consent; therefore, these 4 patients were excluded, and the final sample consisted of 100 patients. All participants provided either written or verbal informed consent prior to any data collection. Patients were compensated $35.00 if they had an ED visit and contributed to data collection. The study was approved by the Duke University Institutional Review Board as well at each participating site. The parent study was registered with Clintrials.gov (NCT03933397).

Procedures

Patients meeting inclusion criteria provided informed consent and were randomized in the outpatient setting for participation in the trial should they have a subsequent ED visit for treatment of a VOE. After consent and randomization, the SCD team wrote either a patient-specific or weight-based opioid protocol to treat VOE. Protocols were based on National Heart, Lung, and Blood recommendations for the treatment of VOE⁵; these include use of a patient-specific opioid protocol if available, or a “standard” opioid protocol if unavailable. For the COMPARE-VOE trial, we used a weight-based opioid protocol as the standard protocol. The study team developed a standardized procedure for determining a specific opioid dose for ED management of VOE. The procedure and calculator to develop patient-specific and weight-based doses are widely available at no cost.¹⁴ The patient-specific protocol took into account any daily opioid use for chronic pain when applicable, pain medications typically taken when the patient experienced severe pain, and previously used opioid doses used in the ED setting. Each completed protocol was entered into the EHR for use by ED providers in the event of a subsequent ED visit by the participant which met study inclusion criterion.

Research staff at each of the six study sites screened for enrolled and consented patients as they presented to the ED. Research staff placed them in an ED treatment space and obtained pain scores at that point and every 30 minutes thereafter until the study endpoint, which occurred 6 hours from ED treatment space placement, or when a decision was made to discharge the patient from the ED or admit them to the hospital. The participant’s pain was assessed by research staff using three different pain assessment tools. All pain scores were obtained directly from the patient rather than from the EHR. A 10-minute time frame was used to determine whether a pain score was “missing” (e.g., if the patient was taken to radiology and not returned to the ED after 40 minutes, the 30-minute pain assessment could not be obtained and was considered missing).

Measures

Three different pain intensity ratings were obtained every 30 minutes. At the beginning of the trial (November 2019), pain intensity scores were measured using the VAS. Participants were provided with a 0-100 mm line with anchors of no pain and worst pain ever and asked to indicate where they currently ranked their pain right now; they were not shown their previous pain score. The line was included on a separate data collection form that was pre-printed and carefully measured to equal 100 mm. Patients always placed a mark on the scale. There were no other units included with the 0 and 100 anchors. Each site received these instructions and were instructed not to photocopy to ensure no distortion of the line. During the COVID-19 pandemic, one of the study sites ceased using paper; as a result, the study team added the use of the verbal Numeric Rating Scale (NRS, 0-100) which is administered verbally, was added to the data collection process for all sites as of August 2020. Research staff at each site were instructed to use the following language when assessing pain with this scale: “Please rate your pain right now with 0 being no pain, and 100 being the worst pain ever.” No other major units were suggested. Research staff were asked to administer both the VAS and NRS whenever possible, and both were used as primary outcome for the study, defined as the change in pain intensity score from placement in an ED treatment space to disposition decision or a maximum treatment duration of 6 hours. Finally, qualitative descriptors were included in the 30-minute pain intensity assessments from study inception. Subjects were asked, “Compared to the last time you marked your pain, tell us how much your pain has changed.” Response choices included a lot worse, a little worse, same, a little better, a lot better, and not done. Similar qualitative measures have been used previously to determine a clinically important difference.^{10, 15}

Data Analyses

All 100 participants in the final sample were included in the analyses, and described by age, ethnicity, race, gender, and SCD genotype. Mean (standard deviation [SD]) and median (inter quartile range) were used to describe age. Frequency and percent were used to describe the other categorical variables.

Before estimating the MCID, we provided descriptive statistics of the change in VAS and NRS by the qualitative descriptor in each of the five categories (i.e., a lot better, a little better, same, a little worse, a lot worse). Although we reported the mean difference and range in the VAS or NRS scores from the prior assessment timepoint by the qualitative descriptor, we also provided the 95% confidence interval (CI) of the change by using a linear mixed model to account for the correlation of repeated differences from the same participant.

Similar to Lopez et al.,¹⁰ we used the pair observations associated with a description of a change in pain of a little better or a little worse to determine the minimal clinically important difference. Specifically, the measured difference in the categories of a little better and a little worse were combined by multiplying the signs of the difference scores of a little worse by −1 to make the MCIDs reported in the results positive and thus able to be compared to previously reported values in the literature. The MCID was estimated by the mean of the differences in this combined group. Because each patient could possibly contribute multiple such paired differences, we used a linear mixed model to account for correlation among repeated measures for individual patients, where the difference was the dependent variable with only intercept in the model and patient as a random effect. The 95% confidence interval (CI) for the intercept term is the 95% CI for the estimated MCID. These analyses were carried out for VAS and NRS, separately.

To describe the agreement between VAS and NRS, we plotted the VAS against NRS scores measured at the same time with 45-degree line. The Pearson correlation coefficient was calculated to get a sense of association between VAS and NRS. Limits of agreement with and Bland and Altmann plot were used to assess agreement between VAS and NRS. We plotted the average of VAS and NRS scores versus the difference of VAS and NRS in the Bland and Altman plot along with the horizontal lines at the values of the limits of agreement. The limits of agreement were calculated as mean ± SD, where SD was calculated as the standard deviation of a random selected difference of VAS and NRS. Because each patient could contribute multiple differences of VAS and NRS due to measurements over time, we used a linear mixed model to estimate such SD by using the estimated variance components due to participant and random error. All analyses were carried out in SAS version 9.4.

Results

One hundred participants enrolled in the COMPARE-VOE trial were included in this ancillary study. Table 1 provides an overview of the sample characteristics. In summary, participants had a mean age of 30.2 years; a large majority self-reported being female (61%) and Black or African American (97%); and 75% had one of the more severe SCD genotypes (SS or SB⁰).

Table 1.

Characteristics of Sample (N = 100)

Characteristic
Age
Mean Age (SD)	30.2 (8.4)
Median Age (IQR)	28 (24-35)
Range	19-65
	n (%)
Ethnicity
Hispanic or Latino	2 (2.0%)
Race
Black or African American	97 (97.0%)
Other	3 (3.0%)
Gender
Male	38 (38.0%)
Female	61 (61.0%)
SCD Genotype
SC, SB+	25 (25.0%)
SS, SB0	75 (75.0%)

Minimal Clinically Important Difference (MCID)

Table 2 provides a summary of the change in the VAS and NRS pain ratings by the paired qualitative descriptor. There were an average of 7 VAS and 4 NRS observations per participant. Among the 574 VAS differences from the prior assessment timepoint, 46% (n = 264) of the observations reflected the same pain when compared to prior pain assessment, whereas 40.4% of observations reflected an improvement in pain (i.e., a little better [n = 192] or a lot better [n = 40]), and 13.6% reported a worsening of pain (i.e., a little worse [n = 63] or a lot worse [n = 15]). Among the 342 NRS observations, 47.4% (n = 162) of the observations reflected the same pain when compared to prior pain assessment, whereas 39.2% of observations reflected an improvement in pain (i.e., a little better [n = 113] or a lot better [n = 21]), and 13.5% reported a worsening of pain (i.e., a little worse [n = 37] or a lot worse [n = 9]).

Table 2.

Summary of the Change in Visual Analogue Scale & Verbal Numeric Rating Scale by the Qualitative Descriptor

	Visual Analogue Scale (VAS)			Verbal Numeric Rating Scale (NRS)
Qualitative Descriptor	n*	Mean (95% CI)^	Range	n*	Mean (95% CI)^	Range
A Lot Better	40	−16.58 (−23.52, −9.63)	−94 to 7	21	−17.43 (−28.16, −6.88)	−70 to 24
A Little Better	192	−10.05 (−11.95, −8.51)	−49 to 27	113	−8.44 (−11.21, −6.35)	−40 to 40
Same	264	−0.17 (−1.32, 0.79)	−41 to 29	162	−0.52 (−1.45, 0.27)	−30 to 26
A Little Worse	63	4.87 (1.61, 9.20)	−17 to 40	37	7.84 (3.47, 15.36)	−10 to 42
A Lot Worse	15	8.40 (0.22, 18.82)	−8 to 45	9	7.0 (−0.91, 15.46)	0 to 30

^*n = number of observations;

^{^}mean is the average change in the VAS or NRS scores from the prior assessment timepoint

There were a total of 256 VAS differences linked with the qualitative descriptors from the previous pain score (reported 30 minutes earlier) of a little better (n = 192) or a little worse (n = 63); the VAS MCID was identified as a mean difference of 8.77 (95% CI: 7.43, 10.83).

There were a total of 150 differences linked with the qualitative descriptors of a little better (n = 113) or a little worse (n = 37); the NRS MCID was identified as a mean difference of 8.29 (95% CI: 6.41, 11.60).

Level of Agreement between VAS & NRS Scales

There were a total of 411 paired VAS and NRS observations. Figure 1 shows the number of observations in the context of the assessment timepoints, with approximately 51% occurring within 90 minutes of the baseline assessment. There was a significant positive correlation between the paired reports of pain severity on each of the VAS and NRS scales (r[411] = 0.88, p < .0001; Figure 2). Furthermore, the level of agreement between the VAS and NRS scales using the Bland Altman Plot indicated a mean difference of −4.6±1.96 and limits of agreement ranging from 20 to −29, indicating the pain intensity ratings from the VAS is, on average, about 5 points less than the pain score from the NRS (Figure 3). Test for normality showed that the differences of VAS and NRS are not normality distributed with p < 0.001, indicating that the normality assumption in calculating the limits of agreement is violated. We performed further analysis without using the normality assumption by the quantile regression at 95% on the absolute differences to obtain an estimated total deviation index of 29.16 This finding implies that 95% of the differences of VAS and NRS lie between −29 to 29, yielding the same conclusion as the limits of agreement.

Figure 1. Number of Observations with a Paired VAS and NRS Report by Assessment Time Points.

This figure depicts the number of observations where a participant completed both the visual and verbal pain scales at each of the assessment time points (in increments of 30 minutes).

Figure 2. Correlation between the Visual Analogue Scale & Verbal Numeric Rating Scale.

This figure depicts observations (n = 411) when a participant completed both the Visual Analogue Scale & Verbal Numeric Rating Scale (range of 0-100 for both scales). Pearson correlation = 0.88 (p < .0001).

Figure 3. Bland-Altman Plot of Visual Analogue Scale (VAS) & Verbal Numeric Rating Scale (NRS).

This figure depicts the mean difference and limits of agreement of 411 pain reports on the VAS & NRS at the same time point.

Discussion

In this study, we report findings from an ancillary investigation of the COMPARE-VOE parent trial that sought to determine the MCID on the VAS and NRS pain intensity scales as well as the level of agreement between the scales. This study advances the existing evidence base as it includes the largest number of pain intensity observations to date to determine MCID for the VAS scale and is the first to identify MCID for the verbal NRS scale among adults with SCD experiencing a severe VOE presenting to the ED for treatment. Moreover, our study is also among the first to report on the level of agreement between using the VAS and NRS scales to measure pain intensity in a population of people living with SCD.

Our findings indicate that the MCID for the VAS scale is lower compared to previously published data (8.77 mm versus 13.5 mm, respectively).¹⁰ Our findings for the lower VAS score could be attributed to the larger number of paired observations included in our analysis (n = 256) compared to a prior study of this population (n = 91).¹⁰ Additionally, the MCID of 8.29 for the verbal NRS scale was slightly smaller than the measurement on the VAS scale; however, the difference between the MCID for the two scales in this study is likely not clinically significant. These findings have pertinence for clinicians because smaller improvements using these scales are recognized as important by patients with SCD.

The Bland and Altman plot allowed our team to quantify agreement between two commonly used pain intensity scales. Although scores for each of the scales were highly correlated, indicating high linear association, the Bland Altman analyses indicated considerable variability in agreement between patient-reported observations on the two scales. This finding is consistent with a recent study that found poor agreement between the VAS and NRS scale in studies conducted with children and adolescents with SCD⁷ and with older adults.⁸ It is important to note that these findings cannot be used to determine whether one scale is superior to the other; however, the data suggest that the two pain intensity scales are not interchangeable. This finding has implications for clinicians and research teams as it suggests that determination of the most appropriate scale must be based on the context of the clinical or research situation and environment. For example, pain reassessments should consistently be conducted using the same scale as prior assessments.

The findings from this study should be considered to have a few limitations. Additionally, the pain intensity scales we used measure only one dimension of pain. Although we also used qualitative descriptors to determine the MCID, the final value is reflective of pain intensity. Clinicians and researchers must be mindful that interpretations of MCIDs provide an average estimation and should not supersede individual patient reports of pain or effects of treatment; clinical judgment and patient reports should always take precedence. To remain consistent with the use of the 0-100 VAS and allow for comparison with the NRS, we chose to administer the NRS using a 0-100 vs. 0-10 response range. Previous clinical trials have used 0-10. In clinical settings, patients typically report their pain using a 0-10 scale, hence they are more familiar with this scale. We cannot estimate in what direction this may impact our findings. Furthermore, although this study included the largest number of pain assessment observations to date, the sample was relatively small as it comprised only people seeking ED care for a VOE, and their pain ratings were generally high upon baseline assessment. Our findings are likely context dependent and may not be generalizable to non-ED settings or to individuals with SCD presenting to the ED with other complaints. Future research is recommended to validate our findings within a broader context.

Conclusion

In this ancillary study of the COMPARE-VOE parent trial, we found that the average MCID values for the VAS and NRS are slightly lower than previously found. Although both scales were highly correlated, the two scales are not interchangeable. When measuring pain, it is best to determine which pain scale is more user-friendly for the setting, and repeated assessments should use the same scale. Future studies are needed to determine the context when each tool is most useful in measuring VOE pain in adults with SCD presenting to the ED for care and measuring response to treatments provided.

Perspective:

The average MCID in pain severity for individuals with sickle cell disease experiencing a VOE using the VAS (8.77 mm) is lower than previously reported, and the MCID for NRS was (8.29). We assessed the agreement between the VAS and NRS and determined that the scales cannot be used interchangeably to measure SCD pain intensity.

Highlights.

• The average VAS MCID was 8.77 mm; lower than previously reported.

• We are the first to report the MCID for NRS (8.29) for this population.

• The VAS and NRS cannot be used interchangeably to measure SCD pain intensity.

Data Availability:

The deidentified data will be submitted to the NIH dbGaP data repository in the near future and will be publicly available at that time.