Validation of the Malignant Wound Assessment Tool – Research (MWAT-R) using cognitive interviewing

Abstract

Malignant wounds as a result of cancer are under-recognized for the physical and emotional distress they cause patients and their families. Unfortunately, there is a lack of valid and reliable screening and assessment tools to aid in the management of malignant wounds. This study aims to validate a patient-reported outcome measurement tool, Malignant Wound Assessment Tool - Research (MWAT-R). Eight patients were recruited and interviewed using the cognitive interviewing methodology to validate this tool. Patients’ understanding and overall impression of the MWAT-R were explored. Our findings showed that the wording and response options posed challenges for patients in completing the tool. Overall, participants felt that questions captured the key issues related to dealing with a malignant wound and accounted for the patients’ perspective. Establishing the content and face validity of the MWAT-R from the patients’ perspectives using cognitive interviews has provided further evidence to the validity of this tool.

Malignant wounds as a result of cancer are often under-recognized for the physical, social, and emotional distress they may cause to the patient and their family (Naylor, 2005). Malignant wounds develop in a number of ways: as a lesion, nodule or ulcer, as a result of primary skin cancers (e.g., melanoma, basal cell or squamous cell); as an eruption through the skin, as a result of an underlying cancerous tumour or as a result of distant metastatic spread of a cancer presenting as a lesion, nodule or ulcer erupting through the skin layers (Naylor, 2002a). Patients’ quality of life may be significantly impacted due to the unpleasant, often difficult-to-manage symptoms of a malignant wound. In some circumstances, malignant wounds signify progressive and life-threatening disease adding to the emotional distress of cancer.

An accurate assessment of a patient’s symptoms and distress related to the symptoms reported by the patient is the foundation of effective symptom management (Naylor, 2002b). Some of the difficulties in assessing and managing malignant wounds is that there has not been a valid and reliable screening or assessment tool specifically designed to measure or describe the occurrence, severity, or distress from the perspective of the patient.

LITERATURE REVIEW

A literature review was performed to identify reliable and valid assessment tools that would assist the health care provider to identify the patients’ perspectives about symptoms and distress of symptoms associated with living with a malignant wound. Databases of Medline, CINHAL and PsycINFO were searched. Some key words used were “malignant wound” or “fungating wound” or “ulceration”, “palliative”, “symptom distress” or “quality of life”, “patient perspective”. Filters were used for English-only articles, with no limits set for publication year (up until the time of preparing this manuscript, July 2017). Only four articles were identified and all four were qualitative studies. They all focused on summarizing patients’ subjective experience of living with a malignant wound (Lo, Hu, Hayter, Chang, Hsu, & Wu, 2008; Lund-Nielsen, Muller, & Adamson, 2005(a); Lund-Nielsen, Muller, & Adamson, 2005(b); Piggin & Jones, 2007). Four tools were found that assessed such conditions: a) The Wound Symptoms Self-Assessment Chart (WoSSAC) (Naylor, 2002b); b) The Toronto Symptom Assessment System for Wounds (TSAS-W) (Maida, Ennis, & Kuziemsky, 2009); c) Malignant Wound Assessment Tool – Clinical (MWAT-C) (Schulz, Kozell, Biondo, Stiles, Tonkin, & Hagen, 2009); and d) Malignant Wound Assessment Tool - Research (MWAT-R) (Schulz et al., 2009). The WoSSAC and TSAS-W have not been tested for reliability or validity.

Of the four tools, the Malignant Wound Assessment Tool – Clinical (MWAT-C) and Malignant Wound Assessment Tool - Research (MWAT-R) are the only assessment tools that have been validated clinically by researchers for patients with malignant wounds (Schulz et al., 2009). Both MWAT-C and MWAT-R have similar components, but MWAT-R collects more patient-reported information in one of the three sections (Part B). For instance, psychosocial symptoms are assessed with greater depth in the MWAT-R (see Methods for further description of the MWAT-R).

In sum, all four wound assessment tools are focused on capturing the physical symptoms associated with wounds and the severity of these symptoms, as perceived by the patient. However, only the MWAT tools have had clinician and researcher validation and have an assessment component that measures distress associated with wound symptoms and the complex physical, psychosocial and emotional disruption patients may experience when living with a malignant wound. A standardized assessment tool that includes the multidimensional aspects of wound experience would facilitate optimal communication of patients’ needs and clinicians’ ability to develop appropriate symptom management strategies. This study aimed to further examine the face and content validity of the MWAT-R because it is the most specific to patients with malignant wounds and has had the most validity testing.

METHODS

Research Design

This study is a descriptive, instrument validation study using a cognitive interviewing methodology to assess patients’ understanding and overall impression of the questions within the MWAT-R using a combination of verbal probing and think aloud methods. Cognitive interviewing is increasingly used to improve instrument design (Knafl et al., 2007). This technique focuses on the wording of items and response options in the tool, paying particular attention to the mental processes used by the respondents to answer questions; therefore, discovering covert, as well as overt problems with the items (Jobe, 2003).

A cognitive interviewing guide with scripted questions (probes) was developed using a modified version of the original Question Appraisal System (QAS) checklist, with probes appropriate to investigate common problems in survey questions (Willis, 2005). The study focused on four out of five categories identified in the QAS that are used to evaluate questions. The categories of potential problems are: 1. Clarity: problems related to communicating the intent or meaning of the question; 2. Sensitivity/Bias: questions that may be sensitive in nature or wording that is bias; 3. Response categories: adequacy and range of responses available to the patient. An additional four questions were added at the end to capture the overall impression of the tool.

Setting and Sample

The study was conducted at a university-affiliated cancer centre in Toronto, Canada. Patients were recruited from both the ambulatory oncology clinics and from inpatient cancer care areas. A purposeful sampling strategy was used to select participants with a malignant wound and achieve maximal variation on gender and diverse types of cancer, at any point in the illness trajectory with a malignant wound. A total of eight participants was recruited using a purposeful sampling strategy. The decision of sample size was based on saturation of data (i.e., recruitment stopped when no new information arose from the interviews) (Willis, 2005).

For the purpose of this research, we have defined malignant wound as a cancerous tumour infiltrating through the skin presenting as a lesion, nodule, eruption, ulcer or fungation located on any part of the body as a result of a primary or metastatic cancer. The inclusion criteria for eligibility were as follows: English-speaking, 18 years of age or older, and aware of the diagnosis of the malignant wound. Patients identified as cognitively impaired were excluded from the study.

Study Procedure

Following approval from research ethics boards, participants were recruited at inpatient units and outpatient clinics between December 2009 and July 2010. The MWAT-R consists of two sections. Section 1 includes patient demographics, which was not assessed as part of this study. Section 2, referred to as “Patient History”, includes a total of 38 questions that assess physical wound symptoms (e.g., pain, odour, drainage, bleeding), patients’ activities, appetite, peri-wound symptoms, mood, coping, self-image, social interaction and what bothers the patient most about having the wound. Respondents were asked to answer “yes” or “no” to certain questions (e.g., “Do you notice any odour?”); or to provide a numerical rating from a range of “0” to “10” with “10” being the most severe symptom (e.g., pain).

At the beginning of each interview, a copy of the MWAT – R Section 2 was given to the participant. The research assistant (RA) explored patient responses to the items using a technique of concurrent probing (Willis, 1999). This technique is characterized by: a) the RA asking the question, b) the participant answering the question, c) the RA asking a probe question, d) the participant answering the probe question, and e) possibly, further cycles of this same process (Willis, 1999). To establish face and content validity upon completion of the tool, the research assistant asked the patients their overall impression of the tool, if there were any additional questions needed regarding a symptom, and if there were questions the participants felt could be omitted. Basic demographic information was collected from patients at the end of the interview. Each interview was limited to one hour or less.

Data Analysis

The interviews were audio recorded on a digital recorder and transcribed. The principal investigators independently reviewed the transcripts after each interview and made notes of problems that emerged related to meaning and comprehension or structure. The principal investigators then compared their findings and came to agreement on phases that were worded ambiguously, were unclear in their intended meaning, or terms that were omitted and thought to be important for inclusion by the patient participants. Following every third interview (Willis, 2005) the principal investigators determined if new wording or additional symptoms that had been suggested by a participant( s) should be included with subsequent interviews. The QAS checklist categories were used to code the results in a chart format in order to organize the data into meaningful categories. This data reduction technique involved filtering the written material into the common themes (Willis, 2005). The principal investigators completed a final analysis identifying noteworthy problems associated with the questions (Willis, 2005).

RESULTS

Participant characteristics are illustrated in Table 1. The questions from MWAT-R are divided into three major categories: physical symptoms, quality of life, and psychosocial. Issues identified by participants were categorized into clarity, sensitivity/bias, response category, and other problems.

Table 1.

Participant Demographics (N=8)

Demographic variable	Frequency (%)
Outpatient	4 (50.0%)
Inpatient	4 (50.0%)
Gender
Female	7 (87.5%)
Male	1 (12.5%)
Mean age (range)	60 years (26–87 years)
Ethnicity
Caucasian	6 (75%)
Asian	1 (12.5%)
Other	1 (12.5%)
Education
University/College	5 (62.5%)
Some College	1 (12.5%)
High school	2 (25%)
Employment	*note categories below are not mutually exclusive
Full time	1 (12.5%)
Part time	1 (12.5%)
Disability	1 (12.5%)
Unemployed	2 (25%)
Retired	3 (37.5%)
Not recorded	1 (12.5%)
Cancer Type
Breast	3 (37.5%)
Skin	3 (37.5%)
Ovarian	1 (12.5%)
Brain	1 (12.5%)
Location
Chest	4 (50%)
Head/Face	2 (25%)
Groin	1 (12.5%)
Head/Chest/Back/Arm	1 (12.5%)
Length of time of wound (range)	3 months. – 3 years
Treatment
Chemotherapy	4 (50%)
Radiation	6 (75%)
Other Treatments	3 (37.5%)
Dressings	8 (100%)
Change once/day	4 (50%)
Twice/day	4 (50)

Out of a total of 30 physical questions, 18 questions were identified by participants as unclear or challenging to respond (Table 2). The majority of the issues identified with the physical questions were due to the items being unclear in meaning. For example, participants (n=4) were confused about the timing and rating of their pain (Question 1a–b, Table 2) because many took pain medications. They were unsure if the self-administration of pain medications should be taken into considerations when rating the timing or severity of pain. Also, participants (n=5) had difficulty giving a numerical rating to most questions (e.g., Question 3a: “How much drainage is there from the wound?”) and many felt the need to clarify their answer because the perception of severity (such as amount of drainage) was subjective. One question had response category problems (Question 8c: “How much does the skin around the wound bother you?”). Participants felt that the response should not be limited to a numerical scale (i.e., 0 to 10). For instance, participants could not indicate whether the bother was emotional (e.g., making them feel sad) or physical (e.g., there is itching around wound).

Table 2.

QAS Checklist Category Per Assessment Item

Question in MWAT-R	Screening Question	Assessment Question	Unclear in intended meaning	Mismatched question to answer category	Bias/Sensitivity
1a Presence of pain	X		X
1b (i- v) Rating of pain		X	X
1c Pain relief		X
1d Worsening of pain		X
1e Pain interfering with life, past week		X
1f Pain location		X
1g Feeling of pain		X
2a Presence of wound odour	X
2b – a Rating wound odour 1		X	X
2b – b Rating wound odour 2		X	X
2c- a Odour bother 1		X
2c-b Odour bother 2		X
3a Rating of drainage		X	X
3b Drainage bother		X	X
3c Presence of leakage	X		X
4a Presence of bleeding	X		X
4b Timing of bleeding		X	X
4c Rating of bleeding		X	X
4d Bleeding bother		X	X
5a Rating of swelling		X	X
5b Swelling bother		X
5c Rating of swelling elsewhere		X	X
5d Swelling elsewhere bother		X
6a Difficulty with activities	X			X
6b Identify activities from 6a	X			X
7a Wound affecting appetite	X
7b How does wound affect appetite		X
8a Surrounding skin change	X
8b How does surrounding skin change		X
8c Surrounding skin bother		X		X
9 Feeling of living with wound		X			X
10 How one sees self	X				X
11 Coping with wound	X		X
12a Family’s interaction with you	X		X	X
12b Friends’ interaction with you	X		X	X
12c Your interaction with family	X		X	X
12d Your interaction with friends	X		X	X
13 Overall bother		X

All four questions on quality of life were challenging to answer. For example, Question 6 had a response category problem because participants felt that the response options listed in 6B (e.g., breathing, seeing, hearing) were incongruent with Question 6A, which asked about difficulty with activities (e.g., showering, shopping, taking stairs). The response options were seen as “physical functions” instead. Participants found Question 7 (“Loss of appetite related to the wound”) confusing because they attributed the loss of appetite to their cancer and treatments, rather than the wound itself.

Various issues were identified with questions in the psychosocial category. Participants felt that Question 9 (“How does living with wound make you feel?”) was a very hard question to answer because it was more related to living with cancer rather than the wound. For Question 11 (“difficulty with coping”), participants had trouble with the numerical rating. This issue came up again in Question 12a–d, but this might have been caused by the anchors (e.g., “completely withdrawn from me”), which were identified by the participants as ill-fitting to the question. Many participants had to pause to consider their response. Another factor that led to this difficulty was the challenge in discerning the impact from the wound and the impact from cancer. One patient responded saying, “It’s really since I have had cancer. We don’t separate cancer from the wound. We don’t ever call it the wound.”

For questions regarding the overall structure and content of this tool, participants’ responses are summarized in Table 3. Participants thought that the tool should collect information on how the malignant wound was managed. Two participants suggested asking about collaboration with health care providers (e.g., physicians, nurses) in managing wound. Two participants suggested asking about wound care products used and any issues associated with these products. With regards to the order of questions, one participant endorsed pain being assessed as the first question because it was thought to be “most important” to assess.

Table 3.

Summary of feedback about the overall structure and content of MWAT-R

Question	Findings
What is your overall assessment of this wound assessment tool?	Participants used the descriptors of “good”, “important”, “necessary” to describe the tool.
Are there any additional questions that you would ask in this assessment tool?	Four participants said no addition. Two participants alluded to the need to ask about satisfaction of care and working with providers. - “I think there should be a little question about nursing and what effect that has because I now have an infection...” Two participants suggested to ask about specific product evaluation associated with their wound - “am I getting the right kind of bandages, creams, product to take care of my wound, have I been shown enough (pause), demonstrated enough how the product works, so that I can use it comfortably and it is working for me. I think that is it”.
Are there any questions that you think should not be asked in this wound assessment tool?	Six participants responded “no”, one said “I don’t know”. One participant said question 12 because it is not about wound management. However, two participants commented on the utility of this question probing the emotional aspect of wound management.
Would you change the order of the questions in this wound assessment tool?	Seven participants said “no”. One participant suggested pain assessment should remain at the beginning “because I think that [pain] is most important”.

DISCUSSION

The current study aimed to examine the face and content validity of the MWAT-R. Our findings showed that the wording and response options posed challenges for patients in completing the tool. However, overall, participants felt that all questions were relevant, captured the key issues related to dealing with a malignant wound, and accounted for the patient’s perspective. Further recommendations are discussed in this section that may assist with the usability of the tool from a patient perspective and improve accuracy of the information gathered.

From participants’ response to Questions 7 and 9, it was apparent that the challenges of living with a malignant wound may be difficult to assess separately from those of living with cancer. This finding supported the need for comprehensive assessment because issues related to the malignant wound, cancer and its treatment may overlap (e.g., loss of appetite). It is not possible to manage challenges associated with each in silos.

Standardized instruments serve to enhance clinicians’ understanding of the extent and nature of challenges associated with a condition. This understanding begins during the screening and assessment process, which helps match the patient with appropriate treatment. The purpose of screening is to determine whether an assessment is needed (Center for Substance Abuse Treatment, 2009); while the purpose of assessment is to gather detailed information needed for a treatment plan that meets individual needs. Although the intent of MWAT-R was to “systematically assess and document malignant wounds” (p. 267, Schulz et al., 2009), our findings suggested some items did not meet the aforementioned definition of assessment because these items did not explore the issues further (e.g., pain).

Another challenge in responding to instrument items concerned response options and the format of measurement. In particular, participants had difficulty responding to numeric rating scales. There are well-documented advantages and disadvantages associated with numeric rating scales (Mohan, Ryan, Whelan, & Wakai, 2010; Kahl & Cleland, 2005). While numerous response formats for questions exist, the format of measurement is one of the earliest considerations in instrument development and this should occur simultaneously with the generation of items (DeVellis, 2016). Recommendations for changing wordings and adding a free-text option to some items were proposed (see next section). However, changing the format of measurement is beyond the scope of the current study.

Specific Recommendations Based on Study Findings

Numerical ratings and screening versus assessment questions were key issues. Most participants had trouble and felt the need to explain their response, so this would give the patients better opportunity to respond in-depth to the question. However, numerical ratings could be helpful if used for certain assessments when combined with further probing. One recommendation is to add a free-text option in addition to numerical ratings for the majority of the questions and, in particular, the screening items (e.g., Question 3c: “Is there any leakage from the dressing?”).

There were also a number of questions in the MWAT-R that were identified as screening questions that were not followed up by any assessment questions, or assessment questions without preceding screening questions (Table 2 lists the screening and assessment questions). It is important to distinguish the two types of questions, so that information can be collected, explored and interpreted properly and usefully.

Finally, since the final question, “What bothers you most about having the wound?” (Question 13), was said to be extremely useful in gathering information from the participants, investigators recommended that this question be moved to be the first question. This initial screening question is a patient-centred approach and would facilitate the clinician’s understanding of the patient’s concerns before going into the structured questionnaire.

Significance of Study

In recent years, patient-reported outcomes have played a central role in the development of self-management interventions. The validation of the MWAT-R in this study ensured appropriate assessment of issues experienced by patients with malignant wounds. Clinicians may utilize this validated tool to explore patient’s concerns, insight to the patient’s quality of life, as well as monitor efficiency and side effects of treatment (Naylor 2002(b); Newell, Sanson-Fisher, Girgis & Bonaventura, 1998).

Limitations

The use of a qualitative methodology limits replicability. Sampling from one single cancer centre limits transferability of findings. Our study did not examine the clinical assessment portion of the tool, which is meant to be used in conjunction with the patient-reported outcome instruments. This omission may have limited the considerations that these items may complement one another.

Future Research

It is imperative that further qualitative studies be conducted to elicit patient input and ongoing feedback from clinicians over time to advance clinical care of the patients with a malignant wound. These inquiries may explore factors associated with the various dimensions of living with a malignant wound, or explore the utility of assessment and screening tools for this condition. Quantitative psychometrical analysis would further confirm construct validity.

CONCLUSION

Establishing the content and face validity of the MWAT-R from the patients’ perspectives using cognitive interviews has provided further evidence for the validity of the tool. Findings from this study suggest the need and provide directions for further development or modification of this important tool, which will strengthen the patient-centred approach to care. The authors recommend future research using the MWAT-R tool, as it will inform best practices in malignant wound assessment and contribute to the understanding of malignant wound management.