Patient-reported outcome measures suitable to assessment of patient navigation

In the 1990s, Harold P. Freeman, a surgical oncologist at Harlem Hospital, observed that women at his center – largely African American and socioeconomically disadvantaged – died from breast cancer at rates much higher than would be expected from national data (1). Dr Freeman observed that these women often received their cancer diagnoses late, after their disease had spread. To reduce these disparities in cancer outcomes, Freeman designed an innovative approach he coined “patient navigation”(1). Dr Freeman trained members of the community to navigate or guide direct patients to ensure proper cancer screening and adherence to their treatment recommendations once the patient’s diagnosis was made. Freeman’s reports on the success of the Harlem program, which suggested a deep decline in the percentage of patients diagnosed with late-stage cancers and a corresponding rise in survival, initiated national and international enthusiasm for the patient navigator concept (2;3). Today, patient navigation services have been promoted widely for patients with all types of cancer (4). The model is also being used for other conditions such as stroke(5) and cardiac events (6).

Although a growing number of cancer patients now have access to patient navigators, and emerging data suggest benefit, particularly for breast cancer screening (7), there is need for a stronger evidence-base for patient navigation in terms of impact on outcomes and costs (8). To date, the field of patient navigation has lacked a common set of outcomes needed to guide research. To address this need, the American Cancer Society convened the National Leadership Summit on Patient Navigation in March 2010 to derive a set of common measures that could be used in research on patient navigator outcomes. Working groups were formed and tasked to address different aspects of patient navigation. Our group focused on patient reported outcomes (PROs).

PROs are designed to capture outcomes that are clinically relevant, and most importantly, meaningful to patients(9;10) including those related to cancer (11). Common examples of PROs include quality of life and health status measures (12), patient satisfaction and experience (13), psychological distress, and pain (14). The common element of PROs is that they are based on the patient’s own perspective and typically obtained through questionnaires often administered by mail, phone or point of service.

There are a number of challenges to standardizing a core set of PROs for research in patient navigation. First, hundreds, if not thousands, of PRO assessment tools have been developed for patients with medical conditions. One review of 75 health-related quality of life studies identified 159 different quality of life instruments used (15). To be sure, relatively few of the myriad of measures that have been published have been well-validated or are widely used, but the sheer number of measures may partly reflect the fact that the number of PRO domains a researcher may wish to examine is conceivably endless. No standardization effort can satisfactorily cover all domains of interest. Conversely, no well-validated measure has been created for a number of domains that are of particular interest to patient navigation outcomes. Last, there are few data indicating which measures are most sensitive to patient navigation. Context (e.g. patient characteristics and needs, local resources, and navigator training) likely impact outcomes and measure sensitivity.

In this paper, we present the findings of PRO working group of the ACS Task Force on Patient Navigation. We summarize our methods, our findings and discuss their implications in terms of current application, limitations, and future research.

METHODS

The PRO Working Group

The PRO Working Group (PROWG) consisted of individuals from a wide variety of perspectives, both in terms of expertise and in terms of patient populations of interest. Members included full-time researchers, medical physicians, nurses, full-time patient navigators, psychologists, and others interested in patient navigation issues. Clinical settings where group members worked, included highly urban as well as isolated rural locations, and patient populations that working group members associated with tended to be diverse. Co-chaired by a health services research physician (KF) and a research-trained psychologist (SR), the group met in a face-to-face format over the course of two days to develop broad outline for patient navigation. The group proceeded by: 1) agreeing on a definition of navigation 2) determining the scope of work 3) agreeing on a conceptual model 5) identifying key domains for PROs relevant to navigation 4) establishing criteria for selecting existing measures from these domains.

Definition of Patient Navigation

We adopted the C-Change definition: “Patient navigation in cancer care refers to individualized assistance offered to patients, families, and caregivers to help overcome barriers and facilitate timely access to quality medical and psychosocial care from pre-diagnosis through all phases of the cancer experience.”

Scope of Task

The group began by establishing consensus on the scope of the recommendations as well as on a number of domains that were found most important in conceptualizing research that utilizes PROs. The working group acknowledged that the primary focus would be on cancer navigation rather than attempting to be broadly applicable to all conditions for which patient navigation may be used, although it was assumed that many of the recommendations made would be generalizable for navigation research across different target populations and disease models. The group also aimed to identify PRO measures that would be applicable at various points in the cancer time continuum, including screening and follow-up on abnormalities, treatment, survivorship, and end of life. Last, the group recognized that PRO measures could prove useful not only to patient navigation researchers, but also for program evaluators. Many community-based programs are interested in assessing outcomes among patients in their own programs for staff and program evaluation and quality improvement.

Criteria for PRO measures

The group agreed that the major determination regarding the recommendation of a measure within a relevant domain would be its demonstrated reliability and validity in relevant populations. Other considerations included readability for low-literacy patients, whether or not the measure had multiple language versions, the existence of national norms, and respondent burden.

Conceptual Model for Patient Navigation

Selecting relevant PRO measures for patient navigation requires a conceptual model that links patient navigation to outcomes through specific pathways. The group recognized that navigator provided services in two broad domains 1) instrumental/logistical reflecting technical competence and 2) interpersonal/educational – reflecting the relational alliance. The pathways leading from these to potential PROs are reflected in figure 1.

Figure 1.

Existing Measures

In addition to drawing from the broad PRO literature in cancer, the working group recognized that three federally funded projects would likely to provide a strong foundation for recommendations regarding PRO measures. These include the Patient-Reported Outcomes Measurement Information System (PROMIS) (16), a project aimed at standardizing health-related PROs across disease categories, the Consumer Assessment of Healthcare Providers and Systems (CAHPS) (17), a satisfaction-oriented set of surveys that asks patients about their experiences in the health care setting, and the measurement development efforts of the Patient Navigation Research Program (PNRP) (8). PNRP consists of 9 institutions that have been funded to provide rigorous testing of patient navigation initiatives under the auspices of the National Cancer Institute. As one of the first large-scale projects specifically aimed at patient navigation research, researchers involved with the PNRP were tasked to develop instruments to test patient navigation effectiveness. Such instruments, once validated, may prove useful to other cancer navigation investigators.

Selection of Domains for PROs

Finally, the working group reached consensus on a variety of domains that were considered to be the most important across the cancer continuum when considering PRO domains for research purposes. These included, in relative order of importance:

• Treatment adherence

• Barriers to treatment

• Satisfaction with care

• Satisfaction with patient navigation services

• Working alliance with patient navigator

• Knowledge regarding cancer and cancer treatment

• Global quality of life

• Specific quality of life symptoms (including depression and anxiety)

• Cultural competency (language, shared decision making, respect, discrimination)

Given the early state of the science of patient navigation, well-validated measures are absent in some domains. In other instances, particularly patient functional assessment, symptoms and experience of care, there is a rich supply of well-validated measures. Promising measurement research is currently progressing in others.

RESULTS

The results of the PROWG’ review is summarized in Table 1. Each domain is addressed below.

Table 1.

Summary of Potential Patient Reported Outcome Measures by Phase of Cancer Continuum

PHASE OF CANCER CONTINUUM	DOMAIN	METRIC	STRENGTHS OF MEASURE	LIMITATIONS OF MEASURE
Screening
	Adherence	NHIS questions (Have you ever had…?)	Cognitively tested	Less accurate than medical records, reporting bias
	Attitudes and beliefs about screening
	Barriers	MEPS question	Cognitively tested and well validated Available in Sp	Not specifically tested for cancer screening
	Cultural competency	CAHPS, cultural competency supplement	Cognitively tested and well validated Available in Sp	Under development
Follow-up on screening
	Self efficacy
	Adherence	NHIS question (Have you ever had at a test where the results NOT normal? Because of the results, did you have additional tests? CAHPS	Cognitively tested	Less accurate than medical records,
	Attitudes/fears	?
	Barriers	MEPS question (Which of these best describes the main reason you were unable to get medical care, tests, or treatments?	Cognitively tested	Not specific tested for cancer screening
	Patient satisfaction with cancer related care	CAPHS question PSC	Focuses on follow-up for cancer	Under development
	Patient satisfaction with navigation	PSN	Focuses specifically on Navigation	Under development
	Psychological distress	PROMIS	Cognitively tested, well validated, computer assisted available online and in Sp
	Cultural competency	CAHPS, cultural competency supplement Perceived Similarity to Navigator	Cognitively tested and well validated Available in Sp Short, specific to navigation	Under development Under development
Primary Treatment
	Adherence	NHIS question
		CAHPS
		MOS adherence scale
	Self-efficacy	CASE-Cancer PAM	Focuses on cancer Strong psychometrically	Only used one publication Not cancer specific
	Patient Beliefs about Medication	BMQ, BMS	Addresses attitudes towards medication	Not widely used in cancer
	Barriers	MEPS question	Cognitively tested Available in SP	Not specific tested for cancer screening
	Patient satisfaction with cancer related care	CAPHS, cancer supplement PSC	Focuses on follow-up for cancer Available in Sp	Under development
	Patient satisfaction with navigation	PSN-L	Focuses specifically on Navigation Available in Sp	Under development
	Family satisfaction	FamCare
	Functional Health Status	FACIT/FACT	Strong psychometrically Available in Sp	? Sensitivity to navigation
	Functional health status and symptoms burden	PROMIS	Cognitively tested, well validated, computer assisted available online and in Sp
	Symptoms	FACIT/symptoms	Strong psychometrically Available in Sp
	Working Alliance	Satisfaction with Navigation – Interpersonal scale	Specific to navigation Available in Sp	Under development
	Cultural competency	CAHPS, cultural competency supplement Perceived Similarity to Navigator	Cognitively tested and well validated Available in Sp Specific to navigation	Under development Under development
Survivorship
	Treatment	MOS adherence scale Morisky MARS Cost-related non-adherence	Global measure Brief, widely used Brief Brief, validated, Available in Sp	Not widely used in cancer Relevant to prescription medications, not widely used in cancer, Relevant to prescription medications, not widely in cancer
	Self-efficacy	CASE-Cancer PAM	Focuses on cancer Strong psychometrically	Only one publication Not cancer specific
	Patient Beliefs about Medication	BMQ, BMS	Addresses attitudes towards medication	Not widely used in cancer
	Barriers	MEPS question	Cognitively tested	Not specific tested for cancer screening
	Coordination of care	CAHPS
	Patient satisfaction with cancer related care	CAHPS Cancer	Cognitively tested and well validated Available in Sp	Under development
	Patient satisfaction with navigation		None
	Functional Health Status and symptom burden	FACIT/FACT	Strongly psychometrically, Cancer type specific Available in multiple language	? Sensitivity to navigation
Strong psychometrically Available in Sp	Symptoms	FACIT	Strongly psychometrically Cancer type specific Available in multiple language
	Cultural competency	CAHPS, cultural competency supplement	Cognitively tested and well validated Available in Sp	Under development
End of Life
	Barriers to care
	Functional health status	FACT/FACIT
	Symptom burden	Symptoms/FACIT
	Functional health status and symptoms burden	PROMIS	Cognitively tested, well validated, computer assisted available online and in Sp
	Caregiver burden
	Patient[proxy] satisfaction with EOL care		??VOICES
	Patient satisfaction with navigation		None
	Family satisfaction	FamCare-2
	Cultural competency	CAHPS, cultural competency supplement	Cognitively tested and well validated Available in Sp	Under development

Treatment adherence

Patient navigation was designed to facilitate timely cancer diagnosis and treatment. Thus, timeliness is a core metric. However, timeliness is optimally measured using objective data such as medical record or claims data. A related construct treatment adherence was identified by the PROWG as a key outcome for patient navigation research. Although objective measures of treatment adherence may be derived from patient medical records or in the case of medications from pharmacy claims, patient reported adherence may be important in contexts where such data are unavailable or when patient perceptions are the more significant research question.

Surveys sponsored by the National Center for Health Statistics (NCHS) of the Centers for Disease Control and Prevention (e.g. National Health Interview survey and Behavioral Risk Factor Surveillance System)(18), the Agency for Healthcare Research and Quality (AHRQ) (e.g. Medical Expenditure Panel Survey) (19), use questions that have been cognitively tested among relevant populations. Typically, surveys are available in English and Spanish. The National Health Interview Survey sponsored by the NCHS asks about receipt of cancer screening (breast, cervical, colorectal, and prostate) including when screening was last obtained, whether results were abnormal, and whether the abnormal results were followed up and treated. AHRQ sponsors the CAHPS family of measures also available in English and Spanish (20). CAHPS addresses patient experience throughout the health care system in a variety of domains. Items are developed through focus groups, undergo extensive cognitive and field testing, and are subjected to psychometric validation. One item asks whether the provider followed up to provide the patient with the tests or screening results. Surveys can be down loaded without charge from the AHRQ website along with benchmarking data. Other CAHPS surveys are discussed in other sections.

A global measure of adherence was developed and validated as part of the large Medical Outcomes Study. This five-item scale assessed how difficult it was for the patient to adherence to physicians recommendations (21) as promising because it addresses general adherence rather than simply medication adherence, but to our knowledge it has not been validated among cancer patients. In addition, there are validated scales that assess medication adherence (22;23), but none have been well-validated among cancer patients.

In addition to being affected by barriers (see next section), adherence is affected by factors such as self efficacy or patient activation. The Perceived Health Competence Scale has been used in cancer patients, but has been not well validated in subsequent samples(24). The Patient Activation Measure (PAM)(25), also validated in short form(26), is a unidimensional survey developed through Rasch analysis and classical test theory psychometric methods. It measures an underlying construct (i.e. being in charge of one’s own health) that is highly relevant to patient navigation.

Perceived Barriers to Treatment

The development of the patient navigation was inspired by the need to address the myriad of barriers to treatment that patients often face when interfacing with a health care system that can be complex and forbidding, particularly in a disease such as cancer where multiple medical teams (e.g., medical oncology, radiation oncology, surgery, etc.) may be involved and patient costs may be prohibitive. Thus, the PROWG deemed it important to assess the barriers that patients face in order to assess whether these differ between navigated and un-navigated patients. The Medical Expenditure Panel survey, a national survey on health care use, produced by AHRQ includes questions on access (27). These include a list of barriers related to why the respondent was unable to get medical care, tests, or treatments you or a doctor believed necessary. Examples of barriers include affordability, insurance hassles, lack of physician participation in plan, language, travel/transportation, work conflicts, child care, or time constraints.

Cost is one of the most common barriers faced by cancer patients and potentially affected by navigation. The Medicare Current Beneficiary Survey (MCBS) sponsored by the Centers for Medicare and Medicaid Services includes a validated measure of cost-related non-adherence for medications (28;29). However, it has not been specifically validated for cancer-related drugs.

Another common barrier is patient beliefs about medications. The Beliefs about Medication Questionnaire (30) has been validated but has been used less commonly with cancer patients.

The Satisfaction with Patient Navigation scale discussed under Satisfaction with Navigation also addresses whether patients experienced a particular barrier and whether they were helped by the navigator in addressing it.

Patient Satisfaction with cancer care

The Jean-Pierre and other PNRP investigators recently completed a validation of the Patient Satisfaction with Cancer-Related Care (PSCC) (31). This 18-item scale showed good construct, convergent and divergent validity and can be administered in English and Spanish. In addition, AHRQ and the National Cancer Institute are currently sponsoring the development of a “Patient Experience Survey for Cancer.” This will likely include items related to care coordination, shared decision making, and assistance with management of symptoms and side-effects (32).

Satisfaction with patient navigation services

PNRP investigators have validated a scale for Satisfaction with Patient Navigation-Interpersonal component for peer review (33). This scale assesses patient satisfaction within the interpersonal/educational (e.g. easy to talk to, respectful, caring, etc). A Satisfaction with Patient Navigation-Logistical is currently being validated.

Working alliance with patient navigator

Although the relationship between the navigator and the patient was considered to be an important factor that may be of use for research, no measure is known to specifically assess the working alliance in patient navigation activities. Alliance has been studied for several decades in the psychotherapy literature and, despite ongoing academic debate about the nature of the construct, there is consensus that a warm, trusting relationship yields benefits that are unattainable without one (34). Numerous therapeutic alliance measures have been developed in psychotherapy, some have been translated into various languages, and these appear to present a possible foundation for the development of a working alliance measure in patient navigation, however, psychotherapy working alliance measures do not readily translate into patient navigation services because of the highly directive nature of patient navigation versus the more collaborative and personal growth orientation that is more typical of psychotherapy. Although researchers who seek to develop a measure on working alliance (35) in patient navigation would likely find that the psychotherapy literature provides a good starting point, no measure at this time is recommended for use in patient navigation research.

The patient satisfaction with navigation-Interpersonal subscale, discussed above, captures many of the elements in a working alliance with the navigator. For example, it includes items related to patient perceptions of navigator caring, listening, ability to help patient figure out problems among other relational items.

Knowledge regarding cancer and cancer treatment

Patient navigators often serve as primary educators to patients regarding their treatment and disease and patient knowledge about their disease and its treatment was rated as an important PRO in patient navigation research by the working group, despite the difficulties that such a construct poses to health researchers. Knowledge assessments are used occasionally in the current cancer literature but such assessments are often idiosyncratically based on the researchers’ specific research questions and lack standardization or psychometric testing of reliability or validity. Researchers who wish to use develop a knowledge assessments would be well served to rely on expertise developed among educational researchers to develop assessments with the highest possible validity. Despite its importance and relevance to patient navigation research, no well-developed cancer knowledge assessment is known to exist. The literature includes examples of assessment of misinformation questions include Stein et al (36) and cancer heath literacy (37) although these have not been subjected to psychometric testing.

Functional Assessment and Symptom Burden

The functional assessment of Cancer Therapy (FACT) measurement system is a family of validated quality of life survey addressing cancer-related health status for prevalent types of cancer (38). Questionnaires address treatment issues, cancer specific symptoms, and detailed symptom burden. FACT has expanded to include a family of instruments designed to assess the management of chronic illness, referred to as functional assessment of chronic illness treat treatment (FACIT) Questionnaires (including translated versions) are available at the FACIT website (http://www.facit.org).

The Patient-Reported Outcomes Measurement Information System (PROMIS) is a network of NIH-funded research sites and coordinating centers that are collaborating to develop a family of reliable and valid PRO measures typically based on item response theory (http://www.nihpromis.org) (16). Measures are available through an online item bank address and different aspects of health-related quality of life (overall function, pain, fatigue, anxiety, depression, social functioning, physical functioning, quality of sleep, etc.). Use of computer adaptive testing helps minimize respondent burden and results can be scored online and compared to norms. PROMIS includes many measures and items developed through FACIT.

Cultural Competency

AHRQ is sponsoring development of a cultural competency supplement to CAHPS (39). This survey will include items related to patient-provider communication, alternative medication, shared decision making, discrimination, trust, and language access. In addition, investigators at the Rochester PNRP have developed a scale designed to capture the extent to which patients view the navigator as similar to themselves. This 10-item scale called the Perceived Similarity to Navigator Scale was adapted from the Perceived Similarity to Physician scale. It assesses perceived racial, ethnic, cultural, and communication similarities. Preliminary analysis show high reliability and divergent and convergent validity (40).

DISCUSSION

Our working group set out to identify relevant PROs for patient navigation. This entailed use of a conceptual model that outlines various potential pathways between navigation and outcomes. It also involved review of existing measures, including those that have been well-validated in multiple populations, those that are widely available and for which benchmarks exist.

Although the working group gave priority to proximal measures that are likely to be more directly affected by navigation (e.g. adherence, unmet needs/barriers, satisfaction/experience with care), there appears to be a dearth well-validated scales across the cancer continuum. The Patient Satisfaction with Cancer-Related Care and the CAHPS Patient Experience with Cancer Survey show promise for assessing experience with cancer related care. Further work is needed to validate existing adherence measures and patient activation measures in relevant populations.

While we were unable to identify a working alliance with patient navigators, two scales under development show promise. These are the Patient Satisfaction with Navigation-Interpersonal scale (9 items) and the 10-item Perceived Navigator Similarity scale may also capture important dimensions of patient-navigator partnership.

Patient navigation was originally designed to address patient barriers to treatment. Thus, a key measure of its success is whether barriers have been successfully alleviated. To date, there isn’t such a validated measure available. However, items from MEPS and other federally sponsored surveys could be used. Furthermore, PNRP investigators are currently in the process of validating such scale that assesses this among patients who are navigated. This scale could be useful for programmatic evaluation, but would require modification to compare navigated and un-navigated patients.

Assessment of quality of life across multiple domains is arguably among the most meaningful of PRO measures and there are many well-validated measures available through PROMIS and FACIT. PROMIS offers the advantage of online administration, minimization of respondent burden, and scoring. The primary downside to these measures is whether they will prove sufficiently sensitive to the effects of navigation because they are affected by so many other factors. To date evidence is mixed.