Patient reported outcomes measures in gynecologic oncology: A primer for clinical use, Part I

I. Introduction

Health-related quality of life (HRQoL) has been increasingly recognized as a critical metric to ensure delivery of high quality, patient-centered cancer care. HRQoL can be measured in several ways, but often involves collecting patient reported outcomes (PROs). PROs collected on validated questionnaires are known as patient reported outcome measures (PROMs).

PROs may be one of the meaningful measures of value delivered in healthcare, demonstrating when a patient’s symptoms are improving or worsening in association with interactions with his or her care team. PROMs can address any potential discordance between the physician and the patient as to the magnitude of their symptoms, (1–3) and their use has led to improved overall survival in the setting of metastatic cancer. For these reasons, payors and regulators are also interested in the collection and use of PROs to assess the value of care delivered by a particular provider or health care system.

Oncology has a long and rich tradition of valuing PROs. The administration of PROMs plays an important role in identifying treatment toxicities, improving symptom management and end of life care and increasingly serves as a prognostic factor for survival in select clinical trial settings. It is important to note, however, that the knowledge base around PROMs has developed almost exclusively from the clinical trial setting. Though a growing body of literature suggests that collection of PROMs is feasible in the routine clinical setting and perhaps represents best practice, comparatively little is known about how to export these tools from clinical trials into routine clinical practice and determine what will be acceptable to patients and providers.

In early 2019, a survey was sent to 1146 SGO members in the US regarding their use of PROMs in clinical practice. While results are likely to be biased by the low response rate (10%), they are nevertheless informative. Among the 121 responders, tremendous variability was reported in the PRO instruments used, with 23% using (presumably nonvalidated) instruments developed within their institution, with the remainder equally divided between FACT, EORTC, and PRO-CTCAE, demonstrated an opportunity for standardization. While 57% of responders reported PROMs to be integrated into their electronic health record, only 30% felt they were easy to use and 70% received no tabulated feedback on results. Time and support services were identified as the most common barriers to implementation. Other obstacles to routinely collecting PROs in clinical practice, discussed below, include: selection of the most appropriate PROM for a given symptom or disease site, building an infrastructure that can accommodate the administration of PROMs, training physician and nursing personnel on how to interpret and act upon the PROM, and data warehousing in order to analyze aggregate results (1).

Rather than duplicate exhaustive reviews of PROMs in the past (4), the intent of this two-part manuscript is the following:

1. To affirm that the Society of Gynecologic Oncology (SGO) believes the collection and use of PROM data is an essential component of comprehensive, patient centered care.

2. To provide an overview of the current landscape for clinical collection of PROMs.

3. To discuss practical considerations for collecting PROMs in routine gynecologic oncology practice, including challenges to implementation and opportunities to standardize collection.

4. To make pragmatic, disease-specific recommendations around specific PROMs that can be helpful in assessing women with gynecologic malignancies.

In December 2018, the Policy, Quality, and Outcomes Taskforce of the SGO convened a daylong seminar including experts in the field of PROs to further SGO’s vision of integrating PROMs into routine clinical practice as a component of patient-centric care. Recognizing our role in managing cancer patients across the entire continuum of surgical and medical care, we believe gynecologic oncologists are uniquely positioned to make important contributions to this field and influence the delivery of care in the wider discipline of oncology.

Part I includes a primer on PROMs in oncology, a review of available instruments, and a discussion of challenges to consider prior to clinical implementation. Also included is a discussion of social determinants of health in the context of patient reported outcomes assessment. Part II includes recommendations for specific PROMs for clinical use in gynecologic oncology by disease site.

II. Background: A Primer on PROMS in Oncology

PROMs have evolved since the early 1990s, having been originally developed as self-reported symptoms for secondary aims in clinical research (5). The most commonly used tools include the Functional Assessment of Cancer Therapy (FACT) and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ C30) (5, 6). Typical PRO surveys consist of four broad domains of well-being: physical, functional, social, and emotional, posing multiple questions within each domain. Later, other specific domains of interest were added (e.g. disease-specific assessments and predominant symptoms associated with each disease; see section IV). In addition, new tools became available to measure outcomes at discrete time intervals during a cancer patient’s journey, such as during treatment, surveillance, and palliation (7).

Initially, PROMs were mostly limited to clinical research, focusing on the effects of treatments on health related quality of life (e.g. whether patients had a higher quality of life on chemotherapy versus best supportive care). PROMs have also been investigated for post-surgical monitoring and in the evaluation of non-traditional side effects associated with novel therapeutics as compared to conventional chemotherapy (8). With the renewed focus on patient-centered care, the next phase of growth will be to routinely apply PROMs in clinical practice to facilitate communication between the patient and health care professional, to improve monitoring of patient response to treatment, and to ultimately improve patient satisfaction and outcomes.

Integration of PROMs into healthcare delivery has also correlated with an improved ability to detect important clinical outcomes. An investigation of more than 13,000 patients enrolled in oncology trials showed that PROs were often better predictors of survival than patient performance status (9). The authors of this study hypothesized that this finding may be attributable to one or more of the following: 1) PROs are a better reflection of survival-related patient functioning and well-being than traditional prognostic indicators, 2) PROMs identify patients with worse prognostic factors and thus exclude them from more aggressive therapy, 3) higher PROM scores are linked with more positive behaviors, such as adherence to medical regimens and healthy lifestyles that affect survival, and 4) PROM scores reflect individual characteristics that have biologic significance associated with tumor progression or survival(10).

Currently, there are numerous validated PROM instruments appropriate for consideration in the oncology setting. Many of these questionnaires are lengthy and therefore better suited for research than routine clinical care. There is no defined set of PROMs that are all-encompassing and represent a gold standard for clinical use; all will contain various advantages and disadvantages. The most important characteristics of any PROM considered for clinical use are that it directly addresses a symptom domain that is relevant to the patient or disease state, that it is easy to administer, and that it provides valuable information to the patient and/or provider (e.g. alerting providers to an actionable symptom, affording prognostic information, or helping a patient assess themselves compared to other similar patients).

III. Available PROM instruments

The instruments in Table 1 are a partial, but representative list of available measures ranging from general QoL questionnaires to cancer-specific questionnaires, symptom measures, and measures of sexuality, stress, and emotional health. The range and scope of all these options can be overwhelming, and many measures include questions that are not relevant for routine clinical purposes. Excessive survey length may discourage patients from participating. One strategy to render PROMs more clinically useful is to use short questionnaires that focus only on questions of clinical relevance, and frame questions in such a way that the patient responses are clinically actionable. This has the benefit of decreasing respondent burden while maintaining the ability to faithfully compare results with the original scales. For example, the 7-item FACT-G7, an abbreviated version of the 27-item FACT-G, focuses on the most relevant patient issues and may be more easily applied to cancer patients in the clinic setting (11). The validation study of the FACT-G7 revealed that fatigue and ability to enjoy life were ranked as the greatest challenges for patients, followed by worry of worsening condition, nausea, contentment with present QoL, sleep, and pain. As such, the FACT G7 focuses on the most troublesome symptoms and can be easily implemented in the clinic due to its brevity. Questions can have a clinical threshold set, typically the two most extreme negative response options (e.g., “quite a bit” and “very much” fatigued, nauseated, worried, etc.) would trigger an alert to the care team that a conversation with the patient is necessary along with a clinical action plan to address the issue. While the FACT-G7 does not query all of the symptoms in the original FACT-G, it does provide a quick, high-level overview of the symptoms that matter most to oncology patients. This is a choice often seen in PROMs: deciding between a comprehensive, lengthy survey versus a brief but less complete survey of most important items.

Table 1.

A partial list of available patient reported outcomes measures (provided references are mainly limited to endometrial cancer for brevity)

Domain		Instrument Abbreviation	Full Instrument Name	Items	Studies
Health-Related Quality of Life HRQOL	General QOL (physical, social/family, emotional, and functional)	FACT-G	Functional Assessment of Cancer Therapy	27	Kornblith (41)
		SF-36	Short-Form Health Survey	36	Oldenburg (42)
		EORTC-QLQ	European Organization for Research and Treatment of Cancer Quality of Life Questionnaire	30	Nout (43)
		SF-12	Short-Form Health Survey	12	Arms (44)
Cancer Specific HRQOL	Endometrial specific	FACT-EN	FACT – Endometrial Cancer Subscale	43	Rowlands (45)
	Endometrial specific	EORTC-QLQ-EN24	EORTC Quality of Life Questionnaire-Endometrial Module	24	Nicolaije (46)
	Cervix specific	EORTC-QLQ-CX24	EORTC Quality of Life Questionnaire-Cervix Module	24	Ferrandina (47)
	Adult cancer survivors	QLACS	Quality of Life in Adults Cancer Survivors	47	Song (48)
	HRQOL and post-op	EQ-5D	Euroquol 5-D	5	Ferguson (49)
Sexuality	Sexual function	FSFI	Female Sexual Function Index	19	Ferguson (49)
	Vaginal changes	SVQ	Sexual Function Vaginal Changes Questionnaire	27	Rowlands (45)
	Sexual activity	SAQ	Sexual Activity Questionnaire	10	Wenzel (50)
	Sexual function	PROMIS-Sxf	Patient Reported Outcomes Measurement Information System Sexual Function	81	Flynn (51)
	Sexual function(interest, activity, enjoyable, dryness)	EORTC_ QLQ-C30 -OV-28	EORTC Quality of Life Questionnaire-Ovarian Module	28	Nout (43)
	Sexual function	SSFS	Short Sexual Function Scale	3	Aerts (52)
	Sexual distress	SSPQ	Specific Sexual Problems Questionnaire	31	Aerts (52)
	Vaginal and Vulvar Tissue Quality	VAS and VuAS	Vaginal Assessment Scale and Vulvar Assessment Scale	Total 8	Easton (53)
Symptoms Assessment	Fatigue	FACIT – F: FACT G and FACIT-Fatigue	Functional Assessment of Chronic Illness Therapy - Fatigue	Total 40	Donnelly (54)
	Fatigue	BFI	Brief Fatigue Inventory	9	Basen-Engquist (55)
	Fatigue Assessment Scale	FAS	Fatigue Assessment Scale	10	Oldenburg (42)
	Pain	BPI	Brief Pain Inventory	11	Basen-Engquist (55)
	Severity of symptoms interfering with daily functioning	MDASI	MD Anderson Symptom Inventory	19	Arms RG (44)
	Sleep	PSQI	Pittsburgh Sleep Quality Index	19	Clevenger (56)
	Weight Loss	WELSQ	Weight Efficacy Life Style Questionnaire	20	McCarroll (57)
	Exercise Motivation	EMI-2	Exercise Motivation Inventory	51	Nock (58)
	Anemia	FACIT-An	Functional Assessment of Chronic Illness - Anemia	47	Dangsuwan (59)
	Bowel and Bladder (Urinary and bowel symptoms)	EORTC-QLQ-C30-PR-25	EORTC Quality of Life Questionnaire-Prostate Module	12	Nout (43)
	Lymphedema	GCLQ	Gynecologic Cancer Lymphedema Questionnaire	20	Carter (60)
	Neuropathy	FACT/GOG-Ntx	FACT Gynecologic Oncology Group - Neurotoxicity	11	Cella (61)
	Menopausal Symptoms	FACT-ES	FACT – Endocrine Symptoms	46	Li CC (62)
	Menopausal Symptoms	MSCL	Menopausal Symptom Checklist	36	Carter (60)
	Menopausal Symptoms	MRS	Menopause Rating Scale	11	Onujiogu (63)
Emotional Well-being	Depression	CESD	Center Epidemiology Scale Depression	20	Onujiogu (63)
	General well-being	WHO-5	World Health Organization 5 Well-being Scale	5	Aerts (52)
	Psychological Distress	BSI	Brief Symptom Inventory	53	Song (48)
	Anxiety / Depression	HADS	Hospital Anxiety and Depression Scale	14	Kornblith (41)
	Beck Depression Inventory	BDI	Beck Depression Inventory	21	Armbruster (64)
	Body Image	SABIS-G	Sexual Adjustment and Body Image Scale – Gynecology Cancer	9	Ferguson (49)
	Life Experiences	M-LES	Modified version of Life Experiences Survey	Original LES: 57	Telepak (65)
	Emotional Coping Scale	Brief COPE	Brief COPE	28	Telepak (65)
	Anxiety and Depression	SIGH-AD	Structured Interview Guide for the Hamilton Anxiety/ Depression Scale	22	Telepak (65)
	Anxiety and Depression	IDAS	Inventory of Anxiety and Depression Symptoms		Honerlaw (66)
	Stress	PSS	Perceived Stress Scale	10	Song (48)
Cancer Distress	Cancer Distress	IES	Impact of Events Scale	15	Honerlaw (66)
Relationship	Dyadic Assessment	DAS	Dyadic Adjustment Scale	32	Aerts (52)
	Social Support	SSSA	Sources of Social Support Scale	50	Telepak (65)
Decisional Measures	Decision Process	SWD	Satisfaction with Decision Scale	6	Arms (44)

Newer options, such as an item library or item bank, allow a “shopping cart” approach to measurement, even at the item (i.e. individual question) level. Both approaches allow the end-user to select only those questions that are important to clinical assessment and avoid lengthy preexisting scales. An item library is a collection of questions that measure various concepts, ranging from symptoms to functional abilities and perspectives on one’s health. An example of such an approach is FACT/FACIT, which maintains a well-known, large 800 item library. With an item library, one can shop the entire set of items and construct a questionnaire comprised only of questions deemed directly relevant to a given clinical setting. One disadvantage of this approach is that a customized set of questions does not have associated summary scores in the way that a disease site questionnaire such as the FACT-Ovary does, prohibiting a high level assessment summary based on the entire instrument. FACIT offers an item library and custom survey builder available through the FACIT website < www.facit.org >

Like an item library, an item bank also consists of a large pool of questions, or items. However, each item in a bank is statistically calibrated to a common latent (i.e., not directly observable) variable, such as pain, depression, fatigue, or physical function. Therefore, not only can one select only those questions relevant to the clinical context, but one can also score patients, or groups of patients, on that same concept being measured. This produces a very flexible assessment base. PROMIS has over 100 item banks available (www.healthmeasures.net.)

IV. Obstacles to collecting PROMs in clinical practice

Previous sections have discussed challenges in PROM selection; this section will discuss challenges in PROM program implementation. Implementing a high functioning PROM program into clinical practice requires a robust platform for PROM collection, engagement of patients and providers, and an informatics infrastructure that is capable not only of collecting PROM data, but making it actionable at the patient level and useful for quality improvement efforts at a system level. Challenges in each of the areas below are significant, but surmountable:

1. Infrastructure and Reporting

PROMs can be collected via phone or on paper, but collection via an electronic platform provides the greatest advantages in clinical care. Studies demonstrate that electronic patient reported outcomes (ePROs) are preferred by both patients and providers and result in fewer unanswered questions (12–16). Electronic capture facilitates visualization of a patient’s longitudinal change and allows for transfer of PROM data to three key locations: a patient’s medical record (where it can be accessed by the care team to improve quality and safety of care), an institution’s data warehouse (for outcome monitoring and research), and if required, to payors to demonstrate value. ePROs accommodate the branching logic required by many instruments and improves patient compliance by enabling advances such as computerized adaptive testing (CAT) (17, 18). Finally, institutions have demonstrated remarkable improvements in quality of care when patients are able to submit PROs from home, a feat greatly facilitated by electronic platforms (19). Yet despite the benefits of ePROs and the widespread use of electronic health records, collection of ePROs remains a complex endeavor with many challenges.

The first challenge in the collection of ePROs lies in the choice of platform. In recent years the number of third party vendors has grown exponentially; interested clinics can choose from a variety of options. These solutions often have a well-styled, patient-friendly interface that offers simple export functions to easily visualize data. However, these platforms tend to be expensive and reside outside of a clinic’s EHR software. Despite vendor claims of easy integration with EHR software, implementation may be complicated by obstacles related to information security policies, difficulty translating the PROM data into actionable features in the EHR such as best practice alerts or safety warnings, and the complexity of maintaining data integrity. Although the three largest EHR systems (Cerner, Allscripts, EPIC) all have PRO solutions, they are only available via patient portal or clinic collection, have rigid assignment criteria, and have limited provider and patient visualization (20).

Once a practice has selected a platform for ePRO administration, attention must be paid to three domains: input, interface, and architecture. Input of PROs refers to the patients’ experience. Patients should have the ability to answer PRO surveys at home (via text, portal or app) and in the clinic (tablet or kiosk) to guarantee equitable collection from patients of different ages, socioeconomic status, and technologic ability. Careful attention should be paid to clinic workflow; interruption of workflow or delay caused by faulty Wi-Fi or a malfunctioning platform almost invariably leads to failure of the program. Interface refers to the interaction of providers and patients with the platform. The platform should be simple, visually pleasing, and provide alerts for critical symptoms. Underpinning the entire enterprise is the architecture of the system. Gynecologic oncology patients have distinct symptom burdens based on their specific disease, treatment modality, and phase of care, necessitating context specific PRO measurements. The architecture of an ePRO system should direct patients to PROMs that are relevant to their specific clinical condition, assign surveys at reasonable intervals (preventing survey fatigue), and link PROM data with relevant clinical data. Survey validity time periods must be set in advance, and ideally coordinate with other disciplines that care for gynecologic cancer patients to avoid survey fatigue. Finally, the system must accommodate the large amount of data necessary to be linked to anchoring events such as surgery, chemotherapy, or radiation. These data must be exportable together with its relevant event to an institution’s data warehouse in order to guarantee that aggregate data is available to assess performance.

2. Provider Engagement

In 2019, Partners Healthcare conducted a study examining factors predictive of success or failure of a clinical PRO program throughout their health care system. With over 4 million PROMs completed in 213 clinics spanning 56 specialties, the single greatest predictor for success of a clinical PRO program was provider engagement, which was defined as more than 50% of the providers in a clinic trained in PRO’s and actively using them in their clinical practice (21). Other factors associated with success were administrative engagement, monitoring of front desk staff, and use of pre-existing paper collection of PROMs, indicating that clinics that were already facile with PROM administration were better poised for success on an electronic platform. PROM implementation failed in less than 1% of clinics with strong clinician engagement.

How do we ensure that our clinicians are engaged and convince them that PROMs are worthwhile? Little research has been performed in this area, but existing investigations suggest two key factors. First, physicians must believe that PROMs improve clinical care. In a pilot study examining PROM collection in patients receiving chemotherapy for a gynecologic malignancy, 24 items from the PRO-CTCAE that were thought by clinicians to be relevant were administered to patients at their first and second clinic visit (22). 90% of patients felt the questionnaire was easy to use and 78% felt that it facilitated discussion with their physician. 97% of physicians felt the questionnaire asked about important symptoms, 95% felt it helped with communication, and 97% wanted to continue to use this questionnaire. These results can be contrasted with the findings of a study of an ePRO platform in patients undergoing surgery for gynecologic malignancy. Postoperative patients used a web-based portal to respond weekly to patient versions of the EORTC QLQ C30 and NCI CTCAE 3.0 (23). Nursing alerts were generated if a patient scored out of range. While 84% of patients found the platform useful, 75% of nurses felt the assessments of pain and quality of life were not accurate, none felt it improved their ability to detect important patient symptoms, and all felt it increased their workload (24). A key difference between the two studies that may explain the disparate outcome is that in the former, clinicians were empowered to choose the specific questions to be asked, whereas in the second patients were administered general questionnaires that were not specific to the patient’s recent surgery. In addition, in the first study PROMs were administered as part of a routine clinic visit, whereas in the second PROMS were administered outside of the usual clinical workflow. The discordant results on provider acceptance illustrate the importance of engaging clinicians on the front lines of patient care to provide input on the selection and administration of PROMs.

The second factor necessary for successful implementation is to ensure that PRO responses are understandable and actionable. In a literature review of over 43 publications examining PROs in oncology, clinicians indicated a preference for visual representation of PRO data through histograms, bar charts and line graphs. They also indicated a desire for education on how to interpret and use PRO data (25). When possible, efforts to collect PROMs should be paired with evidence-based approaches that improve outcomes among patients undergoing treatment for gynecologic cancer. However, the existing literature offers little guidance on when and how to intervene in patients with poor PROs. In the absence of evidence-based intervention, clinics should feel empowered to develop their own algorithms for concerning results, disseminate them amongst their providers, and publish their outcomes. As collection of PROMs becomes more widespread, published reports of successes and failures will help to guide appropriate clinical responses to out of range answers.

3. Patient Engagement

Multiple qualitative studies, prospective observational studies, and randomized clinical trials show that patients feel PROs allow their clinician to be more responsive to their needs and to monitor their symptoms more closely (25). A qualitative study of multiple cancer patients in focus groups showed an overwhelmingly positive attitude to PROMs (26). One patient stated, “I felt that people were still interested in me. People were still wanting to know about me and my symptoms. I wasn’t written off altogether.”

Patients have made it clear that considering their preferences and comfort are critical when choosing PROMs. Studies examining PROMs in oncology patients demonstrate a preference for questionnaires to be less than 20 questions and 10 minutes in length, to be on an electronic device rather than face to face, to have a private and comfortable space to complete questionnaires, and to have questions tailored to their needs and cancer type (27–29). Finally, patients must feel that their responses are being used (25). If a patient completes a PROM questionnaire prior to a clinical encounter and their provider does not make use of their responses, they may be less willing to complete such a questionnaire in the future.

V. Social Determinants of Health

A range of social, economic, and environmental factors directly and indirectly drive health and healthcare outcomes for all people across the world (30). These are referred to as social determinants of health and include income, wealth, education, environment, and exposure to discrimination. The physical, emotional, functional, and social wellbeing domains of PROMs are all influenced by social determinants of health. For example, Medicaid insurance is associated with greater odds of post-operative complications (31, 32) and emergency department visits (33). These adverse post-operative outcomes, in turn, negatively impact both short and long-term PROMs (34). Conversely, higher income can be associated with significantly higher odds of experiencing a clinically meaningful quality of life improvement after surgery (35). Therefore, if PROMs are used to evaluate care delivery, those caring for populations with favorable social standing may unfairly appear to deliver better care than those caring for vulnerable populations. In addition to physical and functional outcomes, social determinants of health also impact critical outcome domains such as patient satisfaction. In a gynecologic oncology population, white race, higher education level, and less time travelled for care were all associated with high satisfaction scores, independent of clinical characteristics (36). In other studies, factors such as age, city dwelling, and English proficiency were associated with satisfaction (37–39).

The majority of PROMs tools were validated in populations that may not mirror a typical American clinic population. For example, the initial validation study of the general quality of life instrument EORTC QLQ-C30 was performed in three language/cultural groups: Northern Europe, Southern Europe, and English speaking countries (6). While this is certainly a large international validation, one could argue that it does not necessarily represent the racially and culturally diverse populations present in many US clinics. A later validation study of the EORTC QLQ C-30 in breast, ovarian and lung cancer patients does not mention race (40). Finally, it is important to note that even in a validation study that includes racial and cultural heterogeneity, validation of a PRO must include disease specific validation. Many cancer types affect unique demographics and thus validation in one cancer type does not necessarily reflect validation for use in patients with gynecologic malignancy. Further study is required to minimize the potential for implicit bias and to ensure that PROM tools adequately represent all patients.

Relevant social determinants for gynecologic oncology patients include race, ethnicity (including foreign vs native born), payer status, English proficiency, annual income and wealth, education level, neighborhood, housing status, and access to transportation. Collecting social needs data will help identify groups at high risk for poor outcomes. This approach is also important to avoid exacerbating existing disparities by developing interventions that are not accessible to all patients. In addition, capturing and incorporating social determinants of health into PROMs can more accurately benchmark performance across sites and aid in making informed choices as to how to adapt interventions for different populations.

VI. Summary

In Part I of this review, we discuss the importance of measuring PROs in clinical practice and identify obstacles to implementation. There is little published evidence available to guide appropriate clinical use of PROMs. Nevertheless, due to their importance, we encourage clinicians and clinical research to focus on the clinical use of PROMs. Challenges to implementation are significant and many solutions remain elusive. In particular, obstacles created by diverse and flawed EHR platforms are currently the rate-limiting step to implementation. Providers and patients alike must continue to demand more from vendors to accommodate collection of PROMs. Only after years of effort is the promise of genetics and targeted therapy coming to fruition, and similar efforts may be necessary in order to realize the full potential of PROMs for our patients.

HIGHLIGHTS.

Patient reported outcomes measurements (PROMs) are an essential component of comprehensive, patient centered care.

Practical considerations and challenges to collecting PROMs in routine gynecologic oncology practice are discussed.

Pragmatic, disease-specific recommendations for the use of PROMs are provided for women with gynecologic malignancies.