The development and testing of a skin tear risk assessment tool

Abstract

The aim of the present study is to develop a reliable and valid skin tear risk assessment tool. The six characteristics identified in a previous case control study as constituting the best risk model for skin tear development were used to construct a risk assessment tool. The ability of the tool to predict skin tear development was then tested in a prospective study. Between August 2012 and September 2013, 1466 tertiary hospital patients were assessed at admission and followed up for 10 days to see if they developed a skin tear. The predictive validity of the tool was assessed using receiver operating characteristic (ROC) analysis. When the tool was found not to have performed as well as hoped, secondary analyses were performed to determine whether a potentially better performing risk model could be identified. The tool was found to have high sensitivity but low specificity and therefore have inadequate predictive validity. Secondary analysis of the combined data from this and the previous case control study identified an alternative better performing risk model. The tool developed and tested in this study was found to have inadequate predictive validity. The predictive validity of an alternative, more parsimonious model now needs to be tested.

Introduction

Skin tears are reported to be a common wound among older adult, public hospital inpatients and residents of aged care facilities 1. In Western Australia, skin tears were found to be the third most prevalent wound in the acute care sector with prevalence rates of between 8% and 11% in all four WoundsWest prevalence surveys 2. Their prevalence has been found to be even higher in the residential aged care population, ranging from 9·8% to 20% in recent Australian studies 3, 4. Skin tears are not only common but can be emotionally and physically costly to the individual and their family as well as financially costly to both them and the treating facility 5, 6, 7.

Despite the prevalence and cost of skin tears, there has been little investigation into what causes people to be the most at risk of developing a skin tear and how their level of risk can be assessed.

The research to address the first of these evidence gaps was recently completed in Western Australia. A case control study was conducted in a tertiary hospital to identify the characteristics of older patients highly associated with having developed a skin tear. Four skin characteristics, ecchymosis (bruising), senile purpura, haematoma and evidence of a previously healed skin tear, together with oedema and difficulty repositioning onesel, were identified as constituting the optimum explanatory model for skin tear development in that study 8.

Building on that previous work, the present study was designed to examine whether these characteristics could be developed into a reliable risk assessment tool able to accurately predict the likelihood of an older person developing a skin tear within 10 days of hospital admission.

Study aim

Our aim was to develop a reliable skin tear risk assessment tool that can accurately identify who is the most at risk of developing a skin tear in hospital.

Materials and methods

Study design

The study was designed to have three stages:

1. Tool development

2. Ensuring tool reliability

3. Validation of the tool

The tool was developed using a modified nominal group technique, a similar methodology to that used in the development of the Skin Tear Audit Research (STAR) Skin Tear Classification System 7.

Two groups, a state development group and national expert panel, from each Australian state were invited to attend workshops and tele‐conferences, respectively. Members were recruited to these groups by placing an advertisement for expressions of interest to participate in the group or panel in ‘Wound Practice and Research’, the journal of the Australian Wound Management Association (AWMA), as well by approaching contacts from the AWMA (Western Australia) Committee directly.

State Development Group

Eight wound specialists from different clinical settings in Western Australia (WA) participated in two 4‐hour workshops held in August and September 2011. The aim of these workshops was to reach consensus on the best way to assess the characteristics identified by the case control study as constituting the best risk model.

Suggested definitions for each of the characteristics of interest were forwarded by the participants to the workshop convenor prior to the workshops. These were then ranked and discussed at the workshops until consensus was achieved. Definitions were edited on several occasions during the workshops and then voted on until the group agreed. The key decisions were that grading (degree that a characteristic was present) would not be required but rather each characteristic would simply be assessed as present or absent; high quality photographs and a list of considerations per characteristic would be needed as would definitions of other characteristics that might be confused with those of interest, for example, lentigines (age/sun spots).

National Expert Panel

Once consensus on a draft tool was achieved by the state group, it was referred to the 16‐strong national expert panel for content validation. This panel included members from each state and from a range of clinical settings. Many of the experts had participated in the earlier STAR classification study 7 and were therefore familiar with the process. This familiarity meant that it took only two 1‐hour teleconferences to achieve consensus on the minor changes needed to be made to the tool and to agree with the state group that high quality photographs would need to be added.

The content of the tool and the additions required having been agreed by the national panel, photographs were added, and definitions and ‘other considerations’ were written and added to produce a second draft of the tool. Small modifications were then made to this version following input from two dermatologists as well as the national panel. These last ‘tweaks’ were needed to facilitate differentiation between the skin characteristics and to ensure the tool's practical utility. The final version was then signed off by both the state group and national expert panel as the tool to be tested.

In all situations, consensus was considered as having been achieved when everyone in the group was in agreement.

Tool reliability

Prior to testing the ability of the tool to predict who was most likely to develop a skin tear when in hospital, it was essential to ensure that it could be used reliably by the research nurses. Reliability is considered a pre‐requisite for any validity testing 9. Both the inter‐rater (IRR) and test–retest reliability of the tool were therefore tested.

The testing was done in a residential aged care facility rather than using photographs as in earlier studies as it was found that photographs can have variable quality based on photographer education and environmental factors such as lighting and distance of camera from the wound 7, 10.

Following familiarisation with the tool (i.e. the opportunity to read through and discuss the tool and the reference sheet describing and providing illustrations of the six characteristics with the researchers), the research nurses independently assessed the same residents on two separate occasions. An independent assessment by a wound care consultant was also completed on the first occasion. The levels of agreement between each of the research nurses' and the consultant's assessments and the two nurses' assessments of the same residents and between the same nurses' assessments of the same resident on different days were then determined.

Thirty‐seven residents had their skin assessed. The research nurses' assessments agreed with the consultant's 97% and 92% of the time, respectively, with each other's 89% and 92%, respectively, and with their own earlier assessment 95% and 92% of the time, respectively. Both inter‐rater and test‐retest reliability were considered to be adequate for validity testing. This same testing approach was used when one of the research nurses resigned and had to be replaced. On this occasion, the new research nurse's agreement with the consultant's assessment was 100%, with the other research nurse 93% and with her own, on two separate occasions, 100%.

Tool validation

Setting and population

The third stage, a prospective cohort study, was conducted in a 500 bed metropolitan tertiary hospital in Western Australia between August 2012 and September 2013. The study was conducted in all in‐patient areas except the short stay and psychiatric wards as these areas were considered to be low risk for skin tears.

Sample size

A sample size of 1400 was estimated as being required for the study given that the data would be analysed using receiver operating characteristic (ROC) curve analysis; the prevalence of skin tears could be expected to be 7% (the most recent WoundsWest skin tear prevalence rate for this site) 2; this would result in approximately 101 cases, and a sample of 101 positive cases and 101 negative cases would achieve 80% power to detect a difference of 0·1 between the Area under the Curve (AUC) under the null hypothesis of 0·8 and an AUC of 0·7, using a two‐sided z‐test at a significance level of 0·05.

Patient eligibility and enrollment

Eligibility was defined as being 50 years or over and having been recently admitted to one of the study wards at the participating hospital. Eligibility was also determined by an expectation that patients were anticipated to stay as an inpatient for at least 2 days or more and were able to give informed consent.

Eligibility was restricted to individuals 50 years of age or older as it has been well established that there is an association between advancing age and skin tear development 2, 11, 12, 13, 14, 15, 16, and the case control study 8 on which this study is based also used this criterion.

Patients who were newly admitted to the hospital during the study period and who met the eligibility criteria were approached by the research nurse and asked to participate in the study. If they were interested, they were given a short information sheet and the opportunity to ask any questions before being asked to sign the consent form.

Data collection and measures

Once consented, the research nurse examined the patient's skin and completed a data collection form that incorporated an illustration of the body so that any existing skin tears could be noted; the risk assessment tool; questions that could be answered from information in patient notes on characteristics known, or hypothesised to be, associated with increased vulnerability to skin tears, for example, age, medical history, current physical condition and treatment, cognitive status; and the data needed to calculate comorbidity‐adjusted life expectancy, that is, the Charlson Comorbidity Index 17.

The research nurse visited the wards frequently to ascertain whether any of the study participants had developed a skin tear and to interview those who had. Patients were followed up for 10 days after recruitment to determine whether they had developed a new skin tear since being in hospital. Those who had not developed a skin tear and were discharged from hospital before the 10 days were followed up by telephone.

Data management and primary analysis

The study data were double entered into a study database, cleaned and analysed using Stata v13 18. The predictive capacity of the risk assessment tool was assessed by calculating the AUC of an ROC. This is an accepted technique for determining the diagnostic accuracy of medical tests in which the area under the calculated curve is used to estimate the probability that a positive case will be rated/ranked as more likely to be at risk on the test than a negative case 19. In addition to the AUC, this technique calculates sensitivity and specificity – the two indices most commonly cited when assessing the diagnostic accuracy of tests. Sensitivity indicates how good the test is at identifying true positives and specificity how good it is at excluding true negatives 20.

Secondary data analyses

Unfortunately, the primary data analysis showed the performance of the tool in assessing risk to be relatively poor. It was therefore decided that secondary data analyses be conducted to determine if there was an alternative model that performed better in predicting risk in both the current and previous cohorts that might form the basis of future tool development. For these analyses, the data from the present study were combined with the data from the previous case control study. The data set was then randomly split into two and a revised risk model generated using one half of the data set. The revised risk model was generated using stepwise logistic regression and the predictive ability of the resultant model assessed, first using the same half of the data set and then using the second half, again using ROC analysis. All analyses are presented with their 95% confidence intervals.

Ethics approval

Approval for the study was obtained from the human research ethics committees of the university that auspiced the research (HR46/2012) as well as the hospital where the research was undertaken (EC2012/061).

Results

Of the 1466 eligible patients admitted to the hospital and assessed on admission, 108 developed a skin tear within the follow up period, just over half (52%) having developed in hospital. Patient demographics and median length of time before a skin tear was developed are presented in Table 1.

Table 1.

Patient demographics and median time to develop a skin tear

Variable	Whole sample*	Individuals who developed skin tears	P value
Sex
Male	50·7% (743)	46·3% (50)	0·310
Female	49·3% (723)	53·7% (58)
Age (years)
50–54	6·9% (101)	0·9% (1)	0·003
55–64	22·0% (322)	13·0% (14)
65–74	27·6% (404)	26·9% (29)
75–84	28·8% (422)	38·9% (42)
85–94	13·6% (199)	17·6% (19)
95+	1·2% (18)	2·8% (3)
Admission type
Emergency	89·4% (1283)	88·0% (95)	0·612
Elective	10·6% (152)	12·0% (13)
Smoking status:
Non‐smoker	87·4% (1187)	86·1% (93)	0·682
Smoker	12·6% (185)	13·9% (15)
Cognitive impairment
No	99·8% (1453)	100% (107)	0·625
Yes	0·2% (3)	0%
Mean days (SD) from recruitment to skin tear	NA	5·41 (2·13)

^*Numbers do not always add up to 1466 due to missing data.

When using the risk tool to predict skin tear development, ROC analysis found the AUC to be 0·5342 and the tool to have only correctly classified 40%, the specificity being particularly low (Table 2).

Table 2.

Results of ROC analysis of original model, 0·5 cut off

Measure	Formula	Value	95% Confidence interval
Prevalence	Pr(A)	7·4%	6·1%	8·83%
Sensitivity	Pr(+|A)	87%	79·2%	92·7%
Specificity	Pr(−|N)	36·1%	33·5%	38·7%
ROC area	(sensitivity + specificity)/2	0·616	0·581	0·65

ROC, receiver operating characteristic.

As described earlier in the Methods section, an attempt was then made to find a better risk model by combining the present study data with the earlier case control study data. This combined dataset was split randomly in half and a series of backward stepwise logistic regressions performed on the first half. The optimal explanatory model identified by this process (Table 3) included just four of the characteristics from the original model,plus age.

Table 3.

Logistic regression results for new four marker model plus age

Characteristics	Coefficient	Standard error	z	P > |z|	95% Confidence interval
Senile Purpura	0·8413	0·2664	3·16	0·002	0·3191	1·3635
Haematoma	1·2839	0·2064	6·22	0·000	0·8794	1·6885
Previous skin tear	0·4502	0·2225	2·02	0·043	0·0141	0·8864
Unable to reposition	0·5150	0·2275	2·26	0·024	0·0691	0·9609
Age	0·0256	0·0098	2·62	0·009	0·0064	0·0447
Constant	−5·1437	0·7313	−7·03	0·000	−6·5770	−3·7105

ROC analysis of this half of the data set, using this model, found the AUC to be 0·78. However, while the specificity was 99%, the sensitivity was only 5·38% if a cutoff value of 0·5 was used. A range of cutoff values were therefore explored, and the sensitivity and specificity of this model were found to be maximised to 71·5% and 70·4%, respectively, when a cutoff value of 0·13 was adopted (Figure 1).

Figure 1.

Sensitivity and specificity of new risk model with different cut off values.

The validity of the four marker plus age model was then tested by applying ROC analysis to the second half of the combined data set (the validation data set) with a cut off value of 0·13. The model was found to perform well. With the probability of developing a skin tear set to 13%, it correctly classified 72·4% of the patients, had an AUC of 0·765, 65·4% sensitivity and 73·5% specificity (Table 4).

Table 4.

Results of ROC analysis of four marker with age model and 0·13 cut off

Measure	Formula	Value	95% Confidence interval
Prevalence	Pr(A)	13%	11%	15·3%
Sensitivity	Pr(+|A)	65·4%	56·5%	73·5%
Specificity	Pr(−|N)	73·5%	70·4%	76·4%
ROC area	(sensitivity + specificity)/2	0·694	0·651	0·738

Discussion

This prospective study tested whether a tool designed to measure the presence of the six characteristics identified in an earlier case control study as associated with an increased likelihood of developing a skin tear 8 successfully predicted who developed a skin tear during a 10‐day follow‐up period. Unfortunately, it did not. Although the sensitivity of the instrument was high, the specificity was low, and only 40% of the patients was correctly classified in regard to their risk of developing a skin tear.

Given this poorer than expected result, further analyses of a dataset combining data from both the case control and prospective studies were conducted. These analyses resulted in the identification of a risk model that performed well using ROC analysis when the probability of developing a skin tear was changed from 0·5 to 0·13. This new model included four of the characteristics in the first model, senile purpura, haematoma, evidence of previously healed skin tear and the ability to reposition oneself, plus age. One of the two characteristics no longer included in the risk model was ecchymosis, which could reflect the difficulty identified by the state development group and national expert panel in differentiating between senile purpura and ecchymosis. In fact, the distinction between senile purpura and ecchymosis in the literature is not always clear. The two terms have been used interchangeably 13, and authors have even used the descriptor ‘senile ecchymosis’ 21. In other literature, senile purpura is described as an ecchymosis that appears under the skin caused by the increase in the fragility of blood vessels and decreased collagenous support as a result of the ageing process and sun damage 12, 22. To address this ambiguity, which was identified during the tool development, the experts agreed that for the purpose of this study, if there was a bruise‐like appearance on older, wrinkly, dry, sun‐damaged or pigmented (i.e. haemosiderin staining) skin, the appropriate description was senile purpura, while ecchymosis was defined as a bruise on ‘good’ skin, that is, without the aforementioned characteristics. Additionally, to facilitate the differentiation of a haematoma from ecchymosis, the panel agreed that to be identified as a haematoma, a bruise needed to be palpable. This tightening up of the definitions of the skin descriptors between the case control and prospective studies may well have contributed to the change in their ability to contribute to the prediction of skin tear development.

The other characteristic not found to be significantly associated with skin tear development in our secondary analyses was oedema. A possible explanation for this could be that although oedema physically stretches the skin and makes it more susceptible to damage, it also occurs secondary to trauma. Once skin is damaged, inflammation occurs as part of the wound‐healing process, and this often causes oedemas, resulting in worsening of the epidermal damage 23. It is therefore quite possible that the oedema identified in the earlier case control study 8, rather than being present before the skin tear occurred, was a result of the inflammatory response to the skin tear.

Age was the only variable found to be significantly associated with skin tears in our secondary analyses that was not identified in the case control study. However, the pathophysiological changes to the skin that occur with ageing are frequently associated with skin tears in the literature 2, 11, 12, 13, 14, 15.

A tool constructed using the new risk model would be expected to correctly identify 65% of patients who would go on to develop a skin tear (i.e. sensitivity). The question then is whether this level of prediction is sufficiently good to justify the additional time for nurses' to undertake another risk assessment on admission. The most commonly studied risk assessment tool, used in many hospitals in Australia, is the Braden Scale for measuring pressure injury risk 9. According to Brown, studies that have examined the predictive validity of the Braden Scale with different populations have found its sensitivity to range between 38% and 88% 24.

Based on these findings, there is quite appropriately some debate concerning the value of risk assessment tools such as the Braden Scale. While studies have found 25, 26 that clinical judgement is as effective as formal tools for predicting the risk of pressure injury, others see value in such scales being part of a comprehensive pressure injury prevention program to compliment nurses' clinical judgement 27. A risk assessment tool like the Braden Scale is seen as providing a standardised way of assessing and documenting risk, enabling agencies to formalise preventative tasks and setting a minimum level of assessment and expectation of practice that can be an improvement on nurses' judgement 24.

Pressure injuries, their potentially disastrous outcomes and how to successfully prevent them are, however, well understood 28, 29, 30, 31. The evidence regarding skin tear prevention, prognosis and treatment, in contrast, is scant 6. While it is known that they can become chronic leg ulcers 32, a common wound among older people that can be extremely debilitating, long‐lasting and costly to the health system; the frequency with which this happens is not yet known. This lack of outcome data, together with our lack of knowledge concerning effective prevention strategies, makes it hard to build a convincing case for routine assessment of skin tear risk in hospitals. The case in residential aged care may be considered stronger given the higher incidence of skin tears among this population and the limited availability of nursing staff to tend to any wounds that do occur. On the other hand, a more cost‐effective alternative to risk assessment and individualised prevention strategies could be the introduction of universal prevention approaches with demonstrated efficacy, such as regular skin moisturising 1.

Limitations

As part of the development and testing of a new diagnostic instrument, its accuracy is usually measured and compared to the equivalent ‘gold standard’ test to review its validity. A risk assessment tool can, however, only be validated prospectively, and the outcome under consideration is influenced by the environment and what is happening to the individual (e.g. their treatment). This has previously been identified as problematic with pressure injury risk tool development as the equipment, quality of care and prevention strategies available at the facility where a tool is tested influence development of the wound 24. This may have been the case for this study as at the hospital where the study was conducted, general skin tear prevention strategies were introduced in 2007. However, no reduction in the prevalence of skin tears was evident in WoundsWest surveys in subsequent years nor any documentation of individually applied prevention strategies found during this study. Recall bias could also be a limitation although efforts were made to reduce this bias by setting up a process for nurses to track participants closely; many patients had been discharged before 10 days, and final follow‐up was via telephone. As a consequence, skin tears were not always observed, and additional data regarding cause and place of development was sometimes not collected.

A further limitation may exist with regards to the generalisability of the model to other care settings. While the new model potentially predicts risk of developing skin tears in hospital populations of older people, it may not be as good at predicting risk in other settings.

Further research

The predictive validity of the new risk model needs to be tested. Given the higher prevalence of skin tears in residential aged care; the greater potential benefit of a risk tool in that setting; and the need for the study results to be generalizable, a prospective cohort study is needed in a cross section of residential aged care agencies. Such a study could, in addition to testing the predictive validity of the tool, if large enough and with a long enough timeframe, look at the subsequent course of the skin tears, the frequency with which they become a leg ulcer, their cost and the final outcome. A study such as this would be considerably easier to conduct if there was a National Wound Data Repository with a reliable mechanism for linking data from different sources about a particular wound. The Australian Wound Management Innovation Cooperative Research Centre is in the process of developing such a repository.

Conclusion

The tool developed using the six characteristics identified in a previous study as best explaining the risk of developing a skin tear was found not to be a good predictor of skin tear development. A new, even more parsimonious model of risk was developed using the data from both studies. The predictive validity of this new model shows some promise but now needs to be tested and, if shown to be an accurate predictor of who develops a skin tear, used to develop an easy‐to‐use clinical tool.