Abstract
BACKGROUND
Patient-reported outcomes are critical to research directed at maximizing patient benefit. The outcomes patients consider most relevant in the treatment of skin cancer have not been directly investigated.
OBJECTIVE
To develop a ranked list of outcomes deemed most important by patients with skin cancer through a proctored Delphi process.
METHODS
Twenty-one patients with a history of skin cancer volunteered to participate in the patient summit. The patient members participated in a guided discussion to generate a long list of potentially relevant outcomes. The list was then condensed and ranked through 2 rounds of a proctored Delphi process.
RESULTS
Patients were diverse in their skin cancer histories and complexities, ages, and states of residence. Twelve themes were rated as highly important by 70% or more of participants. Most of these themes related to patient education and the collaborative nature of the physician–patient relationship. Fear of recurrence and cosmetic outcome were also highly rated. Limitations include a preponderance of older patients and patients from the midwest and northeast.
CONCLUSION
Patients with skin cancer overwhelmingly prioritize a shared decision-making process, in which they are actively engaged and value detailed education regarding their disease. This should inform future research directed at skin cancer treatment and current physician–patient interactions.
Patient input increasingly guides the metrics of health care quality.1 Although the opinions of many types of stakeholders are considered in selecting outcomes to measure, patient–stakeholder opinions and values have been recognized as particularly important in the outcome development process.2,3 Specifically, information about what patients truly value is used to inform the identification and measurement of so-called patient-reported outcomes.4,5 Physician-developed instruments to measure such patient-reported outcomes are currently being refined.6–10 The Patient-Centered Outcome Research Institute (PCORI), a federally funded effort, was formed to improve health care delivery and the patient centeredness of care through comparative effectiveness research.11,12
The Delphi process is a formal consensus method that is frequently used to help groups of individuals reach agreement regarding the importance of particular outcomes.13 Unlike traditional discussion-based consensus processes, where the strenuous advocacy of a few individuals may disproportionately sway group decision-making, the Delphi process is an iterative process in which participants are able to anonymously rate the importance of items. After an initial rating session, participants view the anonymous ratings (means with measures of dispersion) of the group, which may be paired with individual rater-provided reasons for specific choices and then modify their responses as they deem appropriate based on the feedback of the group. Several rounds of Delphi may be used to gradually move toward consensus. This process has been used effectively in a number of medical contexts to develop core outcome sets. A core outcome set is a minimum set of outcomes which multiple stakeholders have agreed should be assessed in all trials addressing a given disease or condition. When researchers studying a particular condition or disease use core outcome measures in their investigations, the data that are collected are more easily pooled and compared across trials. Core outcome sets for many diseases and conditions have been developed with the inclusion of patient input. The ongoing Outcome Measures in Rheumatology initiative13 has been a trailblazer in this area.
Skin cancer and its associated treatments have not been well-studied regarding patient preference. Much of the available literature reports traditional metrics such as disease-free and overall survival. Effects on cosmesis or function are typically evaluated by physician-reported outcomes.14–18 There is a paucity of information regarding the outcomes most valued by patients with skin cancer. To better understand what treatment-related outcomes and themes are most important to patients, the American Society for Dermatologic Surgery organized a Patient Outcomes Summit to bring together a representative panel of patients with skin cancer to participate in a Delphi consensus process.
Methods
Patient participants were recruited through email newsletters from patient organizations (i.e., the Cancer Community, the Partnership to Improve Patient Care, and the American Cancer Society); emails to the physician membership of the American Society for Dermatologic Surgery; and from the clinical practices of the members of steering committee (I.A.M., T.V.C., C.L.B. A.R., M.A., N.L., and J.S.B.). Any patient with a personal history of skin cancer was deemed eligible. Twenty-one patients were able to attend the consensus conference in Arlington, VA, in August 2016. To avoid contamination of patient opinions by physician input, 2 statisticians with expertise in the Delphi methodology and structured consensus processes (E.S.A. and A.B.) were recruited to act as facilitators for the Delphi process and perform subsequent data analysis.
Delphi Process
The patient participants initially received a brief introduction to the Delphi process and existing patient-reported outcomes that had been developed and used in studies of skin cancer (e.g., the Skin Cancer Index, Skin Cancer Quality of Life Impact Tool, and FACE-Q). They then participated in a 2-hour guided discussion managed by the designated facilitators to elicit from the patient cohort the outcome- and treatment-related themes they considered most important. All suggested items were recorded by the facilitators and the physician observers. The collected items were then reviewed by the facilitators and physician observers, such that identical or similar items were consolidated before the first Delphi rating round. Subsequently, patient participants rated each item using an electronic audience response system allowing for anonymous rating. Ratings of 7 to 9 conveyed that a particular rater found a specific item to be “highly important,” 4 to 6 denoted “intermediate importance,” and 1 to 3 denoted “low importance.” Mean ratings were computed, and each item that was not rated by at least 80% of respondents in the same major category (i.e., highly, intermediate, or low) was defined as not meeting the threshold of consensus. Such non-consensus items were discussed in an open forum with the assistance of the facilitator. The facilitator offered clarification when there were points of confusion among participants but strove to avoid influencing the ratings. This proctored discussion was followed by a second anonymous Delphi rating round. The final patient scores were then reviewed, and similar items within the 3 levels were collapsed into major themes. The themes were further grouped under categories named to reflect their common attributes.
The consensus conference was supported by an institutional research contract from PCORI. Patient participants were reimbursed for reasonable travel expenses. No other compensation was provided. The study protocol was approved by the Saint Louis University Institutional Review Board.
Results
All 21 patients who had consented to attending the inperson conference, and who had each previously received surgical treatment for skin cancer, participated in the consensus process. The group demographics were heterogeneous in terms of patients’ age, sex, and primary tumor (Table 1). Overall, 56 treatment-related outcomes were identified by the facilitators and observers. These were then rated through 2 rounds of a Delphi process. The results categorized by rating class are shown in Tables 2–5.
TABLE 1.
Demographic Data for 21 Patient Delphi Participants
| Sex | Males = 6 | Females = 15 |
|---|---|---|
| Tumor type | Melanoma only = 7 | NMSC only = 10 |
| Melanoma and NMSC = 4 | ||
| Age at participation in years | <30 = 1 | 31–50 = 2 |
| 51–70 = 13 | >70 = 5 | |
| States of residence | IL = 6 | NY = 1 |
| PA = 2 | MN = 1 | |
| MI = 2 | MO = 1 | |
| NJ = 1 | DC = 1 | |
| KS = 1 | GA = 1 | |
| GA = 1 |
TABLE 2.
Raw Data of Item Rating as Highly Important by Patients (>80% Respondents Rating 7–9) and Proposed Consolidation of Similar Concepts Into Themes
| Low (%) | Mod (%) | High (%) | Major Themes | |
|---|---|---|---|---|
| Empowered to ask questions | 0.0 | 0.0 | 100.0 | 1. Empowered to ask questions |
| Access to reliable information resources, specific to your condition | 0.0 | 0.0 | 100.0 | 2. Physician listens |
| Open communication with my physician | 0.0 | 0.0 | 100.0 | 3. Fully informed about condition and treatment options |
| Able to return to normal activities | 0.0 | 4.8 | 95.2 | 4. Able to return to normal activities |
| Confidence in physician’s ability | 5.0 | 0.0 | 95.0 | 5. Support in navigating health care system |
| Being informed about treatment options | 0.0 | 9.5 | 90.5 | 6. Confidence in physician’s ability |
| Feeling fully educated about your condition | 5.0 | 5.0 | 90.0 | 7. Proactive outreach from physician’s office |
| Ease of navigating the health system to obtain care | 9.5 | 4.8 | 85.7 | 8. Fear of recurrence risk |
| Being heard by my physician | 10.0 | 5.0 | 85.0 | |
| Gratitude for outcome | 0.0 | 15.0 | 85.0 | |
| Feeling empowered to make my own choices (for treatment) | 9.5 | 9.5 | 81.0 | |
| Proactive outreach from physician’s office | 4.8 | 14.3 | 81.0 | |
| Fear of unknown future risk of recurrence | 14.3 | 4.8 | 81.0 |
TABLE 5.
Items Rated as Low Importance by Patients (>30% Rated 1–3)
| Low (%) | Mod (%) | High (%) | Major Themes | |
|---|---|---|---|---|
| Ability to be a spouse or parent | 38.1 | 19.0 | 42.9 | 24. Affect relationship with spouse, close relative (parent and child), or close friend |
| How you’re affected by other people’s reaction to way you looka | 35.0 | 25.0 | 40.0 | 25. Feeling alone or depressed |
| Depression | 38.9 | 22.2 | 38.9 | |
| Feeling alone | 40.0 | 25.0 | 35.0 | |
| Feeling bad about not seeking treatment sooner | 61.9 | 4.8 | 33.3 | |
| Impact on relationships with friends and family | 42.9 | 28.6 | 28.6 | |
| Social withdrawal | 52.4 | 23.8 | 23.8 | |
| Strained or lost friendships | 55.0 | 25.0 | 20.0 | |
| Feeling guilty about having survived | 71.4 | 9.5 | 19.0 | |
| Fear of judgment | 61.9 | 19.0 | 19.0 | |
| Sad or depressed | 52.4 | 28.6 | 19.0 | |
| Connected to others spiritually | 66.7 | 14.3 | 19.0 | |
| Denial about ongoing, chronic nature of condition | 47.6 | 33.3 | 19.0 | |
| Anger | 70.0 | 20.0 | 10.0 |
After additional analysis guided by the patient discussion, the facilitators condensed the outcomes obtained into 25 major themes by eliminating very low scoring items and consolidating similar concepts. These themes are shown in the last column of each table and are grouped further into 8 major categories (Figure 1). Twelve consensus themes were rated as highly important by 70% or more of participants. Most of these themes related to patient education about the disease and shared decision-making. Concern over recurrence risk and “severity of the scar” was also among the high ranking outcomes.
Figure 1.
Categorization of themes: 23 themes rated by the patient panel of intermediate or higher importance organized into 8 major categories.
Another 11 themes were assigned a rating of intermediate importance (Table 4). These themes were dominated by more immediate perioperative issues (e.g., pain, anxiety, recovery time, and financial costs) and concerns over cosmesis. They also frequently exhibited a dichotomous scoring pattern, with a cluster of participants rating them very highly and a second cluster dismissing them as unimportant.
TABLE 4.
Items Rated as Intermediate Importance by Patients (40%–69% Respondents Rating 7–9 and <30% Rating 1–3)
| Low (%) | Mod (%) | High (%) | Major Themes | |
|---|---|---|---|---|
| Ignorance of what treatment may cost me | 14.3 | 19.0 | 66.7 | 13. See self as ugly or disfigured |
| See self as disfigured | 30.0 | 5.0 | 65.0 | 14. Others view me as ugly or disfigured |
| Bumps and bleeding of the affected skin area | 33.3 | 4.8 | 61.9 | 15. Linkage to resources for psychological support |
| Amount of time to full recovery | 9.5 | 28.6 | 61.9 | 16. Experience wide swings in emotions |
| Coordination of care with other health care providers (primary care, general derm, etc.) | 15.0 | 25.0 | 60.0 | 17. Speed of recovery |
| Others see me as disfigured | 35.0 | 5.0 | 60.0 | 18. Anxiety about procedure |
| Fear of incidential finding | 20.0 | 20.0 | 60.0 | 19. Uncertainty about cost of services |
| Change in lifestyle | 10.5 | 31.6 | 57.9 | 20. Ability to pay for treatment |
| Linkage to resources for psychosocial support | 23.8 | 19.0 | 57.1 | 21. Coordination of care with other health care providers |
| Anxiety about the procedure | 27.8 | 16.7 | 55.6 | 22. Control of pain |
| Anxiety about uncertainty of life (after procedure) | 20.0 | 25.0 | 55.0 | 23. Encourage others to get skin checked |
| Giving up or changing important activities to me (sports) | 20.0 | 25.0 | 55.0 | |
| Amount of time away from usual activities required for treatment | 19.0 | 28.6 | 52.4 | |
| Emotional roller coaster (or wide swing in emotions) | 28.6 | 19.0 | 52.4 | |
| Feeling “used up” or exhausted from repeated treatment | 28.6 | 19.0 | 52.4 | |
| Accepting the “new normal” | 25.0 | 25.0 | 50.0 | |
| Ability to pay for treatment | 25.0 | 25.0 | 50.0 | |
| Pain | 4.8 | 47.6 | 47.6 | |
| Responsible for encouraging others to get skin checked | 27.8 | 27.8 | 44.4 | |
| Pass along genes or skin cancer risk to my children | 28.6 | 28.6 | 42.9 | |
| Complications of procedure (infection, allergic reaction,and bleeding) | 23.8 | 33.3 | 42.9 | |
| Longer time to heal than expected | 28.6 | 28.6 | 42.9 |
The final grouping consisted of 2 themes generally rated low by most patients. These pertained to the psychosocial impacts of surgery and of a cancer diagnosis (Table 5).
Discussion
To augment the limited information regarding the disease-specific outcomes of greatest importance to patients with skin cancer, a patient summit on skin cancer was convened. The patient participants generated a long list of outcomes potentially relevant to skin cancer management and then participated in a 2-step Delphi process, guided by a nonphysician proctor, to rate these candidate outcomes by relative importance. As shown in Table 2, good physician–patient communication was a point of strong emphasis in the list of most critical outcomes. Patient education throughout the treatment process was another important theme. Concern over the appearance of the scar was also highly rated by participants, although, interestingly, a sizeable minority of patients (22%) believed this was relatively unimportant. Perioperative issues, such as pain control and speed of recovery, that have traditionally been the focus of physicians were rated of intermediate importance only.
The panel of patients was diverse regarding time elapsed from last skin cancer diagnosis, with several panelists being long-term melanoma survivors while others had recently undergone surgery for a basal cell carcinoma or cutaneous squamous cell carcinoma. This may have led to the relatively low importance ascribed by several patients to immediate perioperative concerns. Future panels directed at skin cancer patients with a more proximate history of treatment could be valuable in clarifying the impact of timing.
Patient-Centered Outcome Research Institute was established as part of the US Patient Protection and Affordable Care Act of 2010 to fund patient-centered comparative clinical effectiveness research. One of PCORI’s primary missions is to focus on research questions of key relevance to patients. Patient-Centered Outcome Research Institute was founded on the belief that patients possess unique perspectives that can inform and enhance research. Patient-Centered Outcome Research Institute is the largest public funding source for comparative clinical effectiveness research and emphasizes the import of engaging patients in the entire research process from development of the hypothesis to implementation of the results. Patient-Centered Outcome Research Institute funds research that is likely to change practice and improve patient outcomes. It also aims to influence how clinical research is performed by others, so that a greater proportion of all clinical research is useful to patients and other health care decision makers.
Historically, clinical research in skin cancer, as in many other diseases, studied traditional cancer metrics or physician-determined outcome measures. Active involvement of patients in developing outcomes has been rare19 in investigations of skin cancer. The authors sought to engage patients with skin cancer formally in the research process through the solicitation of their perspective on treatment-associated outcomes. This process generated some informative results, which the authors hope will guide future research in skin cancer.
Specifically, additional studies on skin cancer treatment should strive to include outcomes that the authors identified as particularly important to the patient experience, namely those pertaining to physician–patient trust, communication, and patient education. A physician–patient interaction wherein the physician serves as a “facilitator” rather than a “fixer” was the model preferred by the patient panel and should also be a subject of subsequent research. Although this study identified outcomes of importance to patients, an essential next step will be to develop and validate measures to assess these nontraditional treatment outcomes.
The insights gleaned from this patient summit can help improve individual physician–patient relationships by enhancing physician understanding of the import patients place on being fully involved in their care and of having a thorough understanding of available treatment options. Patients with skin cancer find the health care system overwhelming and look to their physicians for guidance and “proactive outreach.”
The results of this study seem generalizable given the diversity of patient representation in terms of demographics, skin cancer type, and stage. A number of domains achieved near consensus despite the heterogeneous experiences of the panelists. The independent facilitator was able to generate lively participation and elicit patient input without influencing the results.
Limitations included the preponderance of female patient panelists and the relatively high mean age of the panel. Although the average age at diagnosis of melanoma is 63 years, melanoma is also one of the most common cancers in young adults,20 and it would be helpful to know whether the treatment-related outcomes considered most important by those younger than 50 years are the same as those rated highly by older patients.
A number of domains had dichotomous ratings with some patients considering them highly important, and others rating them as unimportant. The anonymous Delphi format does not permit subgroup analysis, but it was the unanimous perception of the physician observers that these bimodal distributions in ratings reflected different priorities between patients with melanoma and those with other forms of skin cancer. Anonymity also precludes measuring the concordance or discordance of thematic rankings, so any correlations among the major thematic categories cannot be assessed.
In conclusion, patient-centered outcomes in skin cancer research remain underused. Few studies of skin cancer treatment have specifically studied outcomes of importance to patients despite a national move in this direction. Part of the problem is a paucity of knowledge regarding which outcomes patients with skin cancer value most.
To address these shortcomings, a skin cancer patient summit was held. Through facilitated development of a comprehensive catalog of options, followed by a proctored Delphi process, a ranked list of outcomes of importance to patients was generated. This showed that patients consider especially crucial their involvement in the decision-making process and the education they receive regarding their disease. Consensus on the most important outcomes transcended the diversity of the patient group in terms of skin cancer type and stage. However, for other outcomes, differences emerged in the rankings between patients with melanoma versus nonmelanoma skin cancers. Future studies that subdivide patients with skin cancer into cohorts with melanoma versus nonmelanoma cancer, and those with one skin cancer versus multiple tumors, may clarify subgroup differences regarding the salience of treatment-associated outcomes. This, in turn, may improve the capacity of physicians to meet their patients’ needs.
TABLE 3.
Items Rated as Important by Patients (70%–80% Respondents Rating 7–9)
| Low (%) | Mod (%) | High (%) | Major Themes | |
|---|---|---|---|---|
| Treatment not going according to plan | 14.3 | 9.5 | 76.2 | 9. Treatment going according to plan |
| Acceptance of my health status | 4.8 | 19.0 | 76.2 | 10. Acceptance of change in health status and activities |
| Anxiety about next tumor (in different location) – “never really a survivor” | 14.3 | 9.5 | 76.2 | 11. Feel respected |
| Feeling respected | 15.8 | 10.5 | 73.7 | 12. Severity of the scar |
| Severity of the scar | 22.2 | 5.6 | 72.2 | |
| Fear of recurrence | 9.5 | 19.0 | 71.4 | |
| Loss of capabilities to do my work (or job required change) | 19.0 | 9.5 | 71.4 |
Footnotes
Supported in part by research contracts from the Patient-Centered Outcome Research Institute. The authors have indicated no significant interest with commercial supporters.
References
- 1.Arevalo JD. Perspectives in value-based insurance design for patients with diabetes: assessment and application. Am Health Drug Benefits 2011;4:27–33. [PMC free article] [PubMed] [Google Scholar]
- 2.Boers M, Kirwan JR, Gossec L, Conaghan PG, et al. How to choose core outcome measurement sets for clinical trials: OMERACT 11 approves filter 2.0. J Rheumatol 2014;41:1025–30. [DOI] [PubMed] [Google Scholar]
- 3.Gottlieb AB, Levin AA, Armstrong AW, Abernethy A, et al. The International Dermatology Outcome Measures Group: formation of patient-centered outcome measures in dermatology. J Am Acad Dermatol 2015;72:345–8. [DOI] [PubMed] [Google Scholar]
- 4.Christensen R, Maxwell LJ, Jüuni P, Tovey D, et al. Consensus on the need for a hierarchical list of patient-reported pain outcomes for metaanalyses of knee osteoarthritis trials: an OMERACT objective. J Rheumatol 2015;42:1971–5. [DOI] [PubMed] [Google Scholar]
- 5.Bates AS, Davis CR, Takwale A, Knepil GJ. Patient-reported outcome measures in nonmelanoma skin cancer of the face: a systematic review. Br J Dermatol 2013;168:1187–94. [DOI] [PubMed] [Google Scholar]
- 6.Lee EH, Klassen AF, Nehal KS, Cano SJ, et al. A systematic review of patient-reported outcome instruments of nonmelanoma skin cancer in the dermatologic population. J Am Acad Dermatol 2013;69:e59–67. [DOI] [PubMed] [Google Scholar]
- 7.Chen T, Bertenthal D, Sahay A, Sen S, et al. Predictors of skin-related quality of life after treatment of cutaneous basal cell carcinoma and squamous cell carcinoma. Arch Dermatol 2007;143:1386–92. [DOI] [PubMed] [Google Scholar]
- 8.Gibbons E, Casanas i Comabella C, Fitzpatrick R. A structured review of patient-reported outcome measures for patients with skin cancer, 2013. Br J Dermatol 2013;168:1176–86. [DOI] [PubMed] [Google Scholar]
- 9.Rhee JS, Loberiza FR, Matthews BA, Neuburg M, et al. Quality of life assessment in nonmelanoma cervicofacial skin cancer. Laryngoscope 2003;113:215–20. [DOI] [PubMed] [Google Scholar]
- 10.Rhee JS, Matthews BA, Neuburg M, Smith TL, et al. Skin cancer and quality of life: assessment with the Dermatology Life Quality Index. Dermatol Surg 2004;30:525–9. [DOI] [PubMed] [Google Scholar]
- 11.Anise A, Hasnain-Wynia R. Patient-centered outcomes research to improve asthma outcomes. J Allergy Clin Immunol 2016;138:1503–10. [DOI] [PubMed] [Google Scholar]
- 12.Jensen RE, Snyder CF, Basch E, Frank L, et al. All together now: findings from a PCORI workshop to align patient-reported outcomes in the electronic health record. J Comp Eff Res 2016;5:561–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Tugwell P, Boers M, Brooks P, Simon L, et al. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials 2007;8:38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Morton DL, Thompson JF, Cochran AJ, Mozzillo N, et al. Final trial report of sentinel-node biopsy versus nodal observation in melanoma. N Engl J Med 2014;370:599–609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Faries MB, Cochran AJ, Elashoff RM, Thompson JF, et al. Multicenter selective lymphadenectomy trial-I confirms the central role of sentinel node biopsy in contemporary melanoma management: response to “’No survival benefit for patients with melanoma undergoing sentinel lymph node biopsy: critical appraisal of the Multicenter selective lymphadenectomy trial-I final report”. Br J Dermatol 2015;172:571–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Council ML, Alam M, Gloster HM, Bordeaux JS, et al. Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry. J Am Acad Dermatol 2016;74:739–45. [DOI] [PubMed] [Google Scholar]
- 17.Honaker JS, Bordeaux JS, Tuttle MS. Retrospective review of risk factors predictive of complications in patients after below the knee Mohs micrographic surgery. Dermatol Surg 2016;42:568–70. [DOI] [PubMed] [Google Scholar]
- 18.Boyer JD, Zitelli JA, Brodland DG, D’Angelo G. Local control of primary Merkel cell carcinoma: review of 45 cases treated with Mohs micrographic surgery with and without adjuvant radiation. J Am Acad Dermatol 2002;47:885–92. [DOI] [PubMed] [Google Scholar]
- 19.Frank L, Basch E, Selby JV; Patient-Centered Outcomes Research Institute. The PCORI perspective on patient-centered outcomes research. JAMA 2014;312:1513–4. [DOI] [PubMed] [Google Scholar]
- 20.American Cancer Society Cancer Statistics Center. Available from: http://cancerstatisticscenter.cancer.org/. Accessed January 27, 2017.