Validity and reliability of a pressure ulcer monitoring tool for persons with spinal cord impairment

Abstract

Objective

The purpose was to provide support for validity and reliability of the spinal cord impairment pressure ulcer monitoring tool (SCI-PUMT) to assess pressure ulcer (PrU) healing.

Design

Expert panels developed a 30-item pool, including new items and items from two established PrU healing tools, to represent potential variables for monitoring PrU healing. Subjects were prospectively assessed weekly for each variable over a 12-week period.

Setting

Data collection was conducted on a cohort of inpatients and outpatients in one Spinal Cord Injury/Disorders Center in the Veterans’ Health Administration.

Subjects

A convenience sample of Veterans (n = 66) with spinal cord impairment (SCI) was recruited. Eligible subjects had at least one PrU (n = 167) and a history of SCI for longer than 1 year.

Interventions

Not applicable.

Outcome Measure

A change in PrU volume was calculated using VeV Measurement Documentation software and a digital imaging camera.

Results

Content validity was established for a pool of items designed to gauge PrU healing. Exploratory factor analysis (construct validity) identified a parsimonious set of seven items for inclusion in the SCI-PUMT to assess PrU healing. The SCI-PUMT was found to explain 59% of the variance of the volume across the study. Inter-rater reliability was 0.79 and intra-rater reliability ranged from 0.81 to 0.99 among research assistants. Similar levels of reliability were subsequently established among registered nurses, who used the SCI-PUMT in the clinical setting.

Conclusions

The final version of the SCI-PUMT was determined to be valid, reliable, and sensitive in detecting PrU healing over time in Veterans with SCI.

Introduction

Individuals with spinal cord impairment (SCI) are at extreme risk for developing pressure ulcers (PrU) due to immobility, lack of sensation, moisture, and other risk factors.¹ The prevalence of PrU in persons with SCI ranges from 14 to 32%.² The Veteran's Health Administration (VHA) SCI Quality Enhancement Research Initiative identified PrUs as a top priority for SCI research based upon the “staggering costs and human suffering” that this complication exacts.³

The Consortium for Spinal Cord Medicine (CSCM)¹ developed an evidence-based clinical practice guideline (CPG) addressing PrU risk, prevention, and management in persons with SCI. This CPG recommended that the treatment plan be modified if the PrU showed no evidence of healing within a 2–4-week period. To determine whether healing occurred over time, the CPG further recommended the use of a valid and reliable clinical instrument. Guidelines of the National Pressure Ulcer Advisory Panel (NPUAP) and European Pressure Ulcer Advisory Panel also recommended using a validated measurement tool to assess healing progression.⁴

Previously, our study team conducted focus groups with experienced SCI care providers regarding acceptance and clinical application of the CSCM CPG recommendations; several barriers to implementing these CPG recommendations were identified.⁵ Specifically, SCI providers were not convinced that existing PrU monitoring tools were generalizable to persons with SCI.⁵ Persons with SCI are typically younger than most populations with PrU and are characterized by paralysis, lack of sensation, spasms, autonomic dysfunction, collagen degradation, and other factors below the level of injury that result in impaired healing compared with persons without SCI.¹ Specifically, the following limitations were identified in existing tools to monitor PrU healing: (1) lack of established validity, reliability, and sensitivity for persons with SCI; (2) designed predominantly for use with older adult populations; (3) applicable to acute rather than chronic wounds; (4) long length of assessment limiting clinical utility; and (5) omission of important clinical characteristics unique to PrU in persons with SCI.

The objective of this measurement study was to evaluate the psychometric properties of a new clinical assessment tool, the SCI Pressure Ulcer Monitoring Tool (SCI-PUMT). Research questions were: (1) Is there evidence to support the validity of the SCI-PUMT as a measure of PrU healing? (2) Is there evidence to support the reliability of the SCI-PUMT as a measure of PrU healing in the clinical setting? (3) How does the prediction of PrU healing by the SCI-PUMT compare with that of currently used clinical tools over time?

Evaluating PrU healing in the SCI population has been challenging, making the quantitative effectiveness of treatment strategies that result in PrU changes over time difficult to measure. Practice to monitor PrU healing among VHA SCI centers ranged from using tools with established reliability and validity for non-SCI populations, using locally developed tools lacking psychometric testing, and using non-PrU wound assessments. Although determining the success of topical treatments, support surfaces, nutrition, and other interventions are based on the trajectory of PrU healing, methods to measure this have been inconsistent.⁶

Two widely used tools to assess PrU healing are the Pressure Ulcer Scale for Healing (PUSH)⁷ and the Bates-Jensen Wound Assessment Tool (BWAT),⁸ formerly known as the Pressure Sore Status Tool (PSST).⁹ The PUSH and BWAT provided a starting point for the development of SCI-PUMT and will be briefly discussed.

Pressure Ulcer Scale for Healing (PUSH)

The PUSH, developed by the NPUAP,⁴ is a commonly used tool in the VHA. It is comprised of three sub-scores: length × width, exudate amount, and tissue type. Values calculated for each of these sub-scales are summed with the maximum score of 17, indicating that the ulcer is severe. The PUSH is relatively simple to use and addresses key aspects of PrU assessment.⁷ Characteristics of the PUSH tool include: (a) the surface area constitutes 58% of the total score when the measurements are >24 cm², which is the maximum surface area on the scale, (b) the PrU receives the highest tissue-type score if the PrU has any slough or eschar, regardless of the amount, (c) it omits undermining and tunneling that are frequently seen in patients with SCI,¹⁰ and (d) it omits depth which is necessary in calculating a PrU volume.

The PUSH tool was developed using a research database of 37 subjects (mean age 74 years) with mostly Stages III/IV PrUs; data from 10 additional patients were used to evaluate predictive validity.¹¹ Principal components analysis determined that surface area, exudate amount, and surface appearance offered the best healing model. Stotts et al.¹² reported on two additional retrospective studies providing evidence for the PUSH's validity and sensitivity to PrU healing. The first study (N = 103) represented 10 sites in 8 states with complete data for at least 10 weeks. Subject mean age was 74.6 (SD = 15.0); approximately half of the subjects were women (50.7%). Principal components analysis confirmed that PUSH variables of surface area, exudate amount, and surface appearance provided the best set of items to measure healing, explaining 58–74% of the variance. Pairwise comparisons of mean PUSH scores over time were conducted to provide evidence that healing was being measured. Statistically significant differences were found between scores in Weeks 1–5 but not Weeks 6–10, reflecting early changes indicative of healing. Based on the results of this study, minor modifications to the PUSH scoring algorithm were implemented. A secondary analysis of data was conducted from data recorded in the National Pressure Ulcer Long-Term Care Study (NPLUS). The NPLUS cohort included data from 2490 residents in 111 long-term care facilities; 70% were women and the mean age was 80 years (ranged 18–102 years). The analysis was restricted to 137 PrUs that healed and 132 that did not. Principal components analysis confirmed that the PUSH variables represented the best model of healing and accounted for between 39 and 57% of variance over time. Pairwise comparisons of mean scores over time were conducted to determine how well the PUSH reflected healing. The authors suggested that the pattern of statistically significant results of the PUSH might be more sensitive to change in larger ulcers than smaller ones. Gunes¹³ conducted a prospective study in which he collected PUSH scores for 86 ulcers in a convenience sample of 72 patients, followed over an 8-week period, that were admitted to a university hospital. The mean age of the patients was 66.9 (SD 12.8). Total PUSH scores decreased significantly in repeated measures and differentiated between unhealed and healed ulcers over time. The researcher suggested that an ulcer size and depth sub-scale would improve the tool's utility in the clinical setting.

Bates-Jensen Wound Assessment Tool (BWAT)

The PSST was revised in 2001 by Barbara Bates-Jensen to become the BWAT. Additions, deletions, and reformatting (e.g. scaling) differentiated the BWAT from the PSST.

The BWAT has 13 items: size (i.e. length × width), depth, edges, undermining, necrotic tissue type and amount, exudate type and amount, skin color surrounding wound, peripheral tissue edema and induration, granulation tissue, and epithelialization.⁸ The maximum score is 65, indicating that the PrU is severe.¹⁴ The BWAT is longer than the PUSH and variable assessment is more specific. Characteristics of the BWAT include: (a) surface area constitutes 8% of the score when the measurements are >80 cm², which is the maximum surface area on the scale, (b) more time is required to assess 13 variables compared with the 3 variables in the PUSH, (c) a score of “5” (worse) is given if there is any tunneling or sinus tract formation, regardless of length, (d) undermining is captured in the variable related to tunneling, and (e) epithelialization is partially scored based upon the percent of the wound that is filled or covered, which would be difficult to determine if the clinician had not observed the PrU at baseline.

Harris et al.¹⁵ conducted a pictorial guide content validation project involving the BWAT to develop a learning aid for a variety of different wounds (e.g., PrU and venous ulcers). Researchers selected photographs depicting 11 of the BWAT characteristics. Fifteen wound care nurses then used the BWAT characteristics to validate 73% of 75 photographs of wounds, 22 of which were PrU. Nine wound care nurses used BWAT characteristics to validate 100% of 53 photographs of wounds, 9 of which were PrUs.¹⁵

Bates-Jensen described the process of developing the PSST with 20 PrU experts using a modified Delphi technique.¹⁶ Nine expert panel members evaluated the content validity of the PSST (CVI = 0.91).¹⁷ Two enterostomal therapy nurses independently used the PSST to assess 20 PrU on 10 medical-surgical patients with inter-rater reliability (IRR) of 0.91–0.92 and intra-rater reliability (IrRR) of 0.96–0.99. In a long-term setting, reliability was found acceptable for seven licensed practical nurses, two physical therapists, and six registered nurses (RNs) with no wound care training (IRR 0.78; IrRR 0.89).¹⁶ Using the wound intelligence system, a database of 13 facilities using the PSST, the presenting or initial PSST total score correlated significantly with time-to-healing for 113 ulcers; variables of exudate type and amount, undermining, epithelialization, and induration correlated with time-to-healing.¹⁴ In a comprehensive review of the psychometric properties of 10 tools to measure wound healing, Pillen et al.¹⁸ did not find information regarding predictive validity or sensitivity of the PSST, and the BWAT was not assessed.

Methods

A prospective, longitudinal study was used to develop a parsimonious, valid, reliable, clinical tool (i.e. SCI-PUMT) for monitoring PrU healing in persons with SCI. This study had four phases: (1) development of an item pool, (2) data collection, (3) development of the SCI-PUMT, and (4) assessment of SCI-PUMT reliability in a clinical setting.

Phase 1: Development of an item pool

Support for the content validity of the items included in the tool was based on the input of field experts. Two expert panels consisting of nationally recognized PrU and SCI clinicians from both the VHA and private sector were convened to assist in the development of an item pool. Panel members were composed of physicians, occupational therapists, clinical nurse specialists, physical therapist, psychometrician, certified wound specialists, advance RN practitioner, staff nurses, and a registered dietitian. Expert panel members were from clinical and academic settings and included several persons, who participated on CSCM and NPUAP panels or who were reviewers to develop CPGs on PrU management.

The first expert panel, composed of 11 experts, convened for a 1 day on-site meeting to identify variables that they thought were important to PrU healing in persons with SCI. The brainstorming session identified a wide array of possible variables. Panel members then discussed the viability of each variable and its contribution to the study's objectives. A list of potential items was mailed to a second expert panel (n = 11) for review. Panelists were asked to agree or disagree with inclusion of each item and provide suggestions for alternate wording, terminology, and format. These new items were combined with all items in the PUSH and BWAT to create the 30-item pool for testing. The final item pool consisted of variables from the expert panel, PUSH, and BWAT (Table 1).

Table 1.

Study item pool for pressure ulcer assessment

Assessment variable	Study addition (expert panel)	PUSH	BWAT
Length		X	X
Width		X	X
Surface area		X	X
Depth	X		X
Shape			X
Edges	X		X
Undermining	X		X
Tunneling	X		X
Necrotic tissue type			X
Necrotic tissue amount	X		X
Exudate type	X		X
Exudate amount	X	X	X
Tissue type		X	X
Surrounding skin	X		X
Edema			X
Induration			X
Granulation	X	X	X
Epithelialization		X	X

Phase 2: Data collection

We recruited Veterans with SCI who had Stages II–IV and unstageable PrU (i.e. PrUs obscured by necrotic tissue). Exclusion criteria included immunosuppression or history of acquired immunodeficiency syndrome, current diagnosis of cancer excluding basal or squamous cell carcinoma, severely mentally ill, cognitively impaired, or a terminal illness with a life expectancy of less than 6 months as determined by the attending physician. Persons with SCI of less than 12 months were also excluded because neurologically impaired skin undergoes metabolic changes initially after injury (e.g. rapid increase in collagen catabolism and decrease in an enzyme of collagen biosynthesis below the level of injury) that may alter PrU characteristics.¹⁹ The unit of analysis was the PrU; multiple PrUs on a single patient were included in the sample.

Four RN research assistants were trained regarding the study data collection protocols. The role of these research assistants was to recruit subjects, obtain informed consent, collect data, take photographs using a 3.0 megapixel digital camera and VeV Measurement Documentation (VeV MD) software at baseline and weekly for 12 weeks, document research notes in the computerized record system, and participate in discussions with the research team.

Participants were recruited from inpatient, outpatient, and home care settings. Outpatients had to reside within 40 miles (67 km) of the medical center to allow for follow-up assessments in their homes. Women and minorities were actively solicited.

This study was approved by the Institutional Review Board and the hospital's Research and Development Committee.

The data collection protocol included explicit training of the research assistants concerning how to measure PrUs, including identification of “optimal” ulcer manipulation, positioning of the participant in the bed, and other factors. The use of an assistant (e.g. caregiver) was also specified for the positioning of the participants and for safety issues since PrU assessment also occurred in the patient's home environment.

Intra-class correlation (ICC) was used to assess the IRR and IrRR of the research assistants as part of their training prior to data collection.²⁰ To calculate the ICCs, repeated measures analysis of variance was conducted with the scores on the assessment tools acting as the dependent variable, time acting as a random effect, and the number of repeated measures by raters acting as a fixed effect.²¹ For the IRR, the number of repeated measures was four, based on the four RN Research Assistants’ assessment of each ulcer; for IrRR, the number of repeated measures was two, based on the two measurements on each PrU by each RN Research Assistant. Since the SCI-PUMT was not defined at the time of data collection, the BWAT and PUSH total scores were used to calculate IRR and IrRR values. Our study protocol followed the recommendation of Nunnally and Bernstein and sought ICCs of 0.70 or greater by our research assistants on a sample test.²² Following our initial training, however, the IRR and some IrRR values fell below 0.70. As a result, additional training, coaching, and testing were provided. Immediately following the training session, each of the research assistants assessed 12 PrU using a convenience sample from the inpatient care setting. Each research assistant also evaluated a subset of these PrUs twice within 1.5 hours to establish the IrRR. After the training, both the overall IRR for all of the nurses and the IrRR for individual nurses exceeded the 0.70 target level and data collection began.

Once the training for the research assistants was complete, data were collected for 30 items in the item pool at 13 time points: baseline and 12 weekly assessments. Additional baseline data were collected including age, gender, education, pain levels, spasticity levels, PrU history, co-morbidities, location, and stage of current ulcer(s). Data collection was concluded after 12 weeks from enrollment, when complete healing had occurred, the patient withdrew from the study, the patient was discharged from the hospital and lived more than 40 miles from the hospital, or the patient had a plastic surgery coverage procedure for the study PrU (e.g. myocutaneous flap, skin graft).

Phase 3: Development of SCI-PUMT

Support for construct validity was provided through exploratory factor analysis (EFA).

Exploratory factor analysis was used to select a parsimonious set of items from the larger pool and identify latent constructs represented by the 30-item pool. Principal factor extraction with Varimax rotation was employed. Factors with eigenvalues greater than 1 were considered with final decisions also using information from the scree plot to select the number of factors to be retained. Items in Table 2 were retained in the analysis if these had factor loadings with absolute values greater than 0.30 (15% of shared variance) or communalities of 0.20 or higher (20% of variance in the item). The EFA was conducted as part of an iterative process. At each step, items with low factor loadings or communalities were eliminated. The one with the higher factor loading was retained if two items measuring the same construct (e.g., from two sources) had factor loadings above the 0.30 minimum criteria on the same factor. Items loading on a factor were considered components of summative scales, thus providing a parsimonious measure of PrU healing in persons with SCI.

Table 2.

Results of preliminary exploratory factor analysis*

Variable	Source	Geometric factor	Substance factor
Depth	SCI-PUMT	0.82	NA
Tunneling	SCI-PUMT	0.77	NA
Edges	BWAT	0.55	NA
Undermining	SCI-PUMT	0.48	NA
Surface area	SCI-PUMT	0.35	0.51
Necrotic amount	BWAT	NA	0.52
Exudate type	SCI-PUMT	NA	0.40

*Factor loading <|0.30| have been replaced with “NA” for ease of reading.

SCI-PUMT scoring

Once the final seven items of the SCI-PUMT were identified, a scoring algorithm was developed. To ensure that the final SCI-PUMT would be easy to use in the clinical setting, items were formatted to facilitate a simple hand calculation. Variables measured on continuous scales were transformed to ordinal scales. For example, surface area was converted to a 1–10 score. To help establish cut points for this transformation, distributions of responses to the items from the data at baseline were reviewed with more response options included for variables with a wider range of continuous scores. Scores were weighted as follows: (a) surface area accounted for 38%, (b) depth accounted for 15%, (c) tunneling and undermining each contributed 12%, and (d) edges, exudate type, and necrotic tissue amount individually contributed to 8% of the total score. The option of “zero” was included for items where the absence of the attribute was possible.

Support for criterion validity was based on comparing mean change in SCI-PUMT scores with a criterion measure of healing. We defined PrU healing as the reduction in volume of the PrU, and complete healing as re-epithelialization (i.e. resurfacing) on two consecutive observations. By direct measurement, volume was determined by multiplying the surface area of the PrU by the manually measured depth. When measuring depth was not possible (i.e. ulcer too shallow), the median value of the lowest value recorded in each week of the study (0.25 cm) was assigned for a depth to allow calculation of volume. This was the median value of the lowest value recorded in each week of the study. Reduction in PrU volume, and therefore healing, was verified using a 3.0 megapixel digital camera and VeV MD software (Fig. 1).^23–25 Competency of the research assistants in use of the camera was verified. Surface area (i.e. length × width) was obtained by tracing the wound margins and identifying length and width vectors. The volume was an estimate, obtained by multiplying the calculated surface area by the depth, which was measured using a cotton-tipped applicator at the deepest part of the PrU.⁴ Calibrated measurements, corrected for camera angle and distance, were obtained by using a target plate placed in the plane of the PrU as a size and angle reference.

Figure 1.

The VeV MD software and digital imaging camera.

The IRR was 0.99 in a study of 66 raters in three wound models of different shapes.²⁶

To describe how well changes in the SCI-PUMT scores reflected changes in PrU volume across the study period, mean values of each were plotted for PrUs that healed and those that did not (excluding PrUs of patients who withdrew, PrUs that merged, and one very large ulcer). If it is a valid measurement of healing, the slope of the mean score for the SCI-PUMT would be expected to be similar to that of the volume on separate measurement scales. We also calculated an estimated R² summary statistic of the association between the two scores across time. We employed the strategy based on proportional reduction in mean-squared prediction error suggested by Snijders and Bosker²⁷ as described by Recchia²⁸ to calculate this estimate R².

Values of PrU volume were found to be highly positively skewed; therefore, we conducted a log transformation to facilitate regression analyses used to calculate R².

Since the original item pool contained all of the items in the PUSH and BWAT tools, we were able to estimate the amount of variance (R²) that each scale explained in the PrU volume using the same methods as described above. To directly compare the three measures, SCI-PUMT scores were used to predict the PUSH and BWAT scores over time. The resultant R² described the amount of variance explained by the SCI-PUMT, PUSH, and BWAT scores.

Phase 4: Assessment of SCI-PUMT reliability in a clinical setting

To provide evidence for the reliability of the SCI-PUMT in clinical practice, the IRR and IrRR were calculated as described in Phase 1. To determine how well the SCI-PUMT would work in the clinical setting, all of the 26 RNs who worked on a day shift on an SCI unit were trained to use the new tool. The training included a review of the assessment of all of the PrU variables, and other information included on the tool (e.g. positioning), but was not conducted in as great a depth as with the RN research assistants. We evaluated the IRR and IrRR after 2 months. Six of the RNs were asked to evaluate 16 PrUs twice with an interval of 1.5 hours between assessments. The 1.5 hour interval between assessments to assess IrRR had previously been used in another study testing the reliability of a tool in patients with PrUs.¹⁷

Results

Sample

Sixty-six Veterans with 167 PrU were enrolled in the study. Five participants with 11 PrU elected to withdraw from the study. Others were withdrawn by protocol including: 2 subjects with 3 ulcers who were discharged more than 40 miles from the hospital before the 12-week study duration; 9 subjects with 11 ulcers for whom a decision was made to perform a flap, skin graft, or amputation; and 2 subjects with 4 PrU expired. Two ulcers from the same subject merged with an adjacent PrU during the study period. Of the remaining PrUs, 31 healed and 105 completed 12 weeks of the study without healing. Mean time-to-healing was 5.9 weeks with a range of 1–12 weeks. One very large PrU with a mean volume greater than 350 cm³, and the two ulcers that merged during the study, were eliminated from the statistical analyses.

The study sample (n = 66) had a mean age of 59.8 years (SD = 9.8), 98% (n = 65) were male, and 92.3% (n = 60) were high school graduates or higher. Subjects were divided between paraplegics (48.5%) (n = 32) and tetraplegics (46.9%) (n = 31); and three participants had missing data. The mean SCI duration was 23 years (SD = 16.2). Participants had 1–9 PrUs (mean = 2.3, SD = 1.67), 39.4% (n = 26) had previously undergone surgery for the PrU (e.g. myocutaneous flap), and 61% (n = 101) had experienced a PrU in the same site. The two most common sites of the PrU were the ischia (42.8%, n = 71) and sacrum or coccyx (26%, n = 43).

One-third of the subjects (33%, n = 22) had chronic osteomyelitis in their PrU, and approximately one-fifth (18%, n = 12) of patients experienced PrU pain. Overall, spasticity and pain were not major problems for this group of subjects. Fifteen (22.7%) of the patients had used nicotine the preceding week, and five had a history of substance abuse. Approximately 1 of 10 subjects (10.6%, n = 7) had diagnoses of depression or anxiety and lacked social support (9.1%, n = 6). Anemia was a documented diagnosis in almost one-quarter of the subjects (24.2%, n = 16).

Construct validity

Data from individual PrUs at baseline were included in the EFA (Table 2). All of the items (n = 30) in the item pool were entered into the initial analysis. A two-factor solution including seven items explained 100% of the common variance in the analysis. The first factor included items related to PrU surface area (length × width), depth, edges, tunneling, and undermining. The second factor included items relating to exudate type and necrotic tissue amount. The variable of surface area was found to be complex, loading on both factors.

Final SCI-PUMT

The final version of the SCI-PUMT contained seven items (Fig. 2). The first group of items was named “Geometric Factor” because these items characterized the shape of the ulcer. These items included surface area (length × width), depth, edges, tunneling, and undermining. The second group was named “Substance Factor” because these items reflected ulcer contents (i.e., exudate type and necrotic tissue amount). The SCI-PUMT total score is the sum of the 7-item scores with the maximum score of 26. Higher scores indicate more severe PrUs.

Figure 2.

Spinal Cord Impairment Pressure Ulcer Monitoring Tool (SCI-PUMT).

Criterion validity

Fig. 3 depicts the mean SCI-PUMT score and PrU volume over time for PrUs that did (n = 30) and did not (n = 105) heal. While measured on a different scale of measurement (2–26), the SCI-PUMT scores reflect the slope of change in the PrU volume. The mean of the PrU volume increased in Week 9 of the study due to a temporary increase in size of several PrUs. The resultant peak in the graph was not clearly evident in the SCI-PUMT score. Another minor difference is that the PrU volume appeared relatively flat after Week 5, while the SCI-PUMT score continued to heal on a more gradual slope. Results of the R² analysis found that the SCI-PUMT accounted for 59% of the variance in the PrU volume over the study period. Models using the PUSH and BWAT scores to predict PrU volume explained 57 and 24% of the variance, respectively. In the subsequent R² analyses directly comparing the clinical measures, the SCI-PUMT score was found to explain 64% of the variance in the PUSH score and 71% of the variance in the BWAT score.

Figure 3.

Healing over time SCI-PUMT vs. PrU volume.

Reliability of the SCI-PUMT

Phase 4 of the study concentrated on establishing IRR and IrRR in the clinical setting. IRR was 0.79 while the IrRR ranged from 0.81 to 0.99.

Discussion

In this study, a sample of 167 PrUs in Veterans with SCI was followed prospectively for 12 weeks to determine whether a parsimonious set of items could be identified that reflected healing. It represents one of the largest prospective studies among persons with SCI and resulted in the SCI-PUMT. By recruiting patients from both inpatient and outpatient settings, we were able to assess the utility of the tool to detect changes in ulcers at all stages of healing, including patients with new, short-term, and long-standing ulcers. We leveraged PrU monitoring tools developed in non-SCI elderly populations by including items from the PUSH and BWAT in our initial item pool. These items were reviewed and supplemental items were developed by an expert panel with the goal of focusing on aspects of PrU healing specifically for Veterans with SCI.

Several methods have been used in the past to provide criterion validity for PrU monitoring tools. Developers of the PUSH described statistically significant differences of mean scores over time and results of regression models in which a dichotomous classification of healed or not healed was used to predict PUSH scores.¹² In a study of the PSST, change in total score, correlation with time-to-healing, and comparison of score with the stage of PrU were provided as support for criterion validity.¹⁴

In our study, we provide support for criterion validity based on how well the SCI-PUMT mirrored and predicted changes in PrU volume over time as measured by the VeV MD software and digital imaging camera.²³ The major advantage of this strategy was that PrU volume provided a measure of healing over time rather than simple classification of the PrU as being healed or not healed. We believe the R² statistic, calculated from repeated measures data, provided a useful summary statistic that supported the ability of the SCI-PUMT to reflect healing as measured by PrU volume. The disadvantage of the strategy is that the PrU volume probably does not adequately capture information about geometric items (e.g., tunneling and undermining) or substance items (i.e., exudate type and necrotic tissue amount) included in the SCI-PUMT.

The physical size of a PrU is the predominate attribute, which reflects the healing process if monitored over time. In our study, we employed a measure of PrU volume to reflect the changes in physical size (healing) across 12 weeks of measurement. The SCI-PUMT explained slightly more variance in PrU volume (59%) than did the PUSH for surface area (57%). The variance explained is directly related to the amount of the total score in each assessment assigned to the anatomical size of the PrU (38% of the SCI-PUMT score and 58% of the PUSH score to surface area). We believe the SCI-PUMT to be an improved measure of healing in Veterans with SCI because it has the advantage of monitoring anatomical size while including clinical factors such as undermining, tunneling, and necrotic tissue amount that were endorsed by our clinical experts as being important to the healing process.

The BWAT score only accounted for 24% of the variance in PrU volume but these results are affected by the smaller amount of weight (8%) assigned to the maximum anatomical size of PrU in the BWAT scoring directions. Emphasis in the scoring of the BWAT is placed on other attributes related to PrU healing including undermining, tunneling, necrotic tissue, skin color around the wound, and granulation tissue, several of which are also included in the SCI-PUMT. Our study found that the SCI-PUMT score accounted for 71% of the variance in the BWAT scores over time in Veterans with SCI. These results suggest that the SCI-PUMT measures similar attributes of PrU healing as the BWAT with approximately one-half the number of items (7 vs. 13).

It should be noted that the RN research assistants encountered many assessment challenges during the study (Table 3).²⁹ To achieve acceptable levels of IRR and IrRR, research assistants initially assessed subjects in pairs, reviewed numerous digital photographs of PrUs in a group, reviewed the BWAT instructional DVD multiple times, and engaged in discussions regarding PrU assessment with study team members. Lessons learned during this process informed the training procedures employed for the cohort of nurses that participated in the subsequent tests of reliability of the final SCI-PUMT in the clinical setting. The difficulty in establishing the initial IRR and IrRR with the RN research assistants, who were highly experienced in performing PrU monitoring, calls into question the utility of established tools that are often employed in clinical practice with little or no training. The clinical investigators believed that a detailed protocol for using the SCI-PUMT was essential to maximize reliability among those who would ultimately be using the tool.

Table 3.

Challenges with obtaining IRR and IrRR among RN research assistants

Issue	Challenge	Examples
Measurements	Length and width depend upon measurement orientation	Length may extend from head-to-toe or greatest length; width may be perpendicular to length or the greatest width
	Patient positioning and gravity affects the PrU's surface area	Measurements are typically greater in the ischial and sacrococcygeal areas if the hip/leg are flexed
	Surface area is influenced by the degree of manually “spreading” the PrU if in a compressed area	Moderately “spreading” a PrU in the sacrococcygeal area (and other compressed areas, e.g., ischial) can enhance visualization and provide more accurate measurements
	Non-contiguous areas within an open PrU lead to variations in surface area	Length may not be measured over “islands” of intact skin
	Depth measurements require a vertical orientation to the PrU	In partially closed PrUs, depth was deemed vertical whereas a tunnel was oblique
	A horizontal frame of reference is required to determine the PrU depth	The intersection of two applicators that are perpendicular to each other provides more precise measurement than a vertical applicator alone
Exudate	Exudate amount depends upon the type of dressing	Hydroactive dressings may have a moderate or large amount of drainage that may appear to be purulent
	Exudate amount is affected by the length of time between dressing changes	Dressings that are changed frequently may have a lesser amount of exudate
Undermining tunneling	Undermining and tunneling must be differentiated	A cotton-tipped applicator was used to determine if it could or could not be palpated or visualized on the skin surface when inserted into a tunnel or undermined area
Necrotic tissue edges	Strict interpretation of the BWAT and PUSH is required	Even <5% ulcer slough constitutes necrotic tissue, which results in the worst score on the PUSH; any hyperkeratosis warrants the worst score on the BWAT
Digital imaging	Digital imaging competence must be verified	Image blurring may occur if a target plate, or other method of verifying focus, is not used

Limitations

Study limitations included the following: (a) Stage was not an assessed variable in this study, as weekly descriptions of the skin and tissue provided more accurate assessments. Although Stage II PrUs have a different physiological method of healing than Stages III or IV, the intent was to make a single tool that would be applicable to Stages II–IV and unstageable PrU. Furthermore, according to the NPUAP guidelines, PrUs may not be “reverse staged”, so the stage would remain constant⁴; (b) the healing process could be altered by tissue type and the depth of the PrU (e.g. 16% were Stage II PrU that healed without granulation; 39.4% had undergone PrU surgery in the same location as the study PrU); (c) the sample included persons with SCI from one SCI center; (d) the study design did not allow for controlling certain factors known to contribute to healing. Topical, primary, and secondary dressings were at the discretion of the provider and team managing the subject's PrU care. We did not control for the types and frequency of dressings or other interventions (e.g. negative pressure therapy). Bacterial colonization could also potentially be a factor that predicts PrU healing. However, we do not believe that routine assessment of bacterial burden using tissue biopsy, which is recommended in CPGs, is part of routine clinical care. The purpose of our study was to develop an instrument for visual assessment of PrU healing, and; (e) one large PrU was an outlier in our sample and was excluded from the statistical analyses. Similarly, two PrUs that merged were excluded. While it may be that the SCI-PUMT could be used to measure PrUs with these characteristics, the current study did not provide evidence in support of this contention.

Implications

Clinicians in healthcare settings currently use tools such as PUSH, BWAT, and “home-grown” methods that may include combinations of tools. To date, however, no study involving a representative SCI population has examined whether one of these instruments has sufficient validity, reliability, and sensitivity to predict PrU outcomes. An evidence-based outcome tool (i.e. the SCI-PUMT) will improve communication among SCI healthcare providers. The SCI-PUMT could form a basis for outcomes monitoring of PrU healing in persons with SCI and assist the clinician in critical decision-making affecting the management of the PrU. A standardized monitoring system could also facilitate comparisons of healing rates within facilities and across sites for quality improvement purposes. These data may contribute to local and national performance measures. Data will enable relative quantification of success in treating PrUs in persons with SCI across healthcare facilities.

This study will serve as a foundation for future studies to evaluate PrU management. For example, the SCI-PUMT could be used to compare types of topical treatment options (e.g., dressings, medications) or support surfaces (e.g., air mattress replacement vs. air-fluidized therapy for Stage IV PrU). Research in PrU healing using the SCI-PUMT may translate to more expeditious healing, improved quality of life, decreased morbidity and mortality, and more cost-effective clinical care.

Conclusions

This study found that the SCI-PUMT was a reliable, valid, and sensitive instrument for measuring PrU healing in persons with SCI in a 100-bed VHA SCI center. This tool can help to improve communication among SCI healthcare providers, form the basis for outcomes monitoring of PrU healing in persons with SCI, and assist clinicians in critical decisions affecting overall PrU management.

Future directions

The VHA SCI Strategic Health Group seeks to standardize PrU monitoring throughout all SCI Centers in the VHA. This project was the first step towards achieving this goal. An educational toolkit was developed to facilitate SCI-PUMT implementation including an instructional DVD that demonstrates assessment of each variable of the SCI-PUMT and scoring of the tool. A study funded by the VHA will examine SCI-PUMT implementation in six SCI Centers in the VHA and a quality improvement project will explore how the SCI-PUMT is used in decision-making regarding PrU treatment in one VHA SCI Center.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Presentations

Twenty-first Annual Symposium on Advanced Wound Care (Poster)	San Diego, CA	24–27 April 2008
40th Annual Conference – Wound, Ostomy Continence Nurses Society (Poster)	Orlando, FL	21–25 June 2008
National SCI Outcomes Nurse Training (Paper)	Tampa, FL	15 July 2008
SCI Pressure Ulcer Conference (Paper)	Orlando, FL	10 August 2008
25th Annual Conference – American Association of Spinal Cord Injury Nurses (Paper)	Kissimmee, FL	13 August 2008
James A. Haley Veterans’ Hospital – Research Day (Poster)	Tampa, FL	13 May 2009
Pressure Ulcer Prevention and Treatment Following Spinal Cord Injury – Department of Veterans’ Affairs (entire 1½ day Conference)	Minneapolis, MN	4–5 May 2011

Grant and financial support

This work was supported by the Department of Veteran Affairs, Veterans Health Administration, Health Services Research and Development Service, Nursing Research Initiative (NRI 03-245-4) (IRB#104145).