Evaluation of Treatment- and Disease-Related Symptoms in Advanced Head and Neck Cancer: Validation of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22 (NFHNSI-22)

Abstract

Context

The Functional Assessment of Cancer Therapy-Head and Neck is a well-validated assessment of quality of life used with patients diagnosed with head and neck cancers (HCNs). The present study is an attempt to evaluate and modify this instrument as necessary in light of the recent regulatory guidelines from the Food and Drug Administration on the use of patient-reported outcomes in clinical trials.

Objectives

Overall, the goal was to identify patients’ highest priority cancer symptoms, compare these symptoms with those suggested by oncology experts, and construct a brief symptom index to assess these symptoms and categorize them as treatment-related, disease-related, or related to general function and well-being.

Methods

Patients (N = 49) with advanced (Stages III and IV) HCNs were recruited from participating National Comprehensive Cancer Network institutions and community cancer support organizations in the Chicago area. Patients completed open-ended interviews and symptom checklists. Participating oncology physician experts also rated symptoms. Content validity was obtained by evaluating results alongside items in the Functional Assessment of Chronic Illness Therapy system. Eleven oncologists categorized symptoms in terms of importance and also whether the symptoms were primarily related to disease, treatment, or functional well-being.

Results

HCN-related symptoms endorsed as high priority by both patients and oncology experts were selected for the new National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22. The final version includes 22 items, which are broken down into disease-related symptoms, treatment side effects, or general function and well-being. The new scale has acceptable internal consistency (Cronbach’s coefficient alpha = 0.86), content validity for use in chemotherapy trials of patients with advanced disease, and concurrent validity as demonstrated by moderate-to-strong correlations with the existing Functional Assessment of Chronic Illness Therapy measure.

Conclusion

The National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22 adequately reflects symptom and side effect concerns of advanced HCN patients as well as oncology physicians. This instrument can be used to evaluate the most important disease-related symptoms, treatment side effects, and function/well-being in patients with advanced HCNs in clinical practice and research.

Introduction

Approximately 49,260 new cases of head and neck cancer (HNC) (excluding eye, orbit, and thyroid cancers) were diagnosed in the U.S. in 2010. Incidence has been declining since 1980, although recent studies have shown increases in those cancers related to human papillomavirus infection. Death rates have decreased in recent years; nonetheless, an estimated 7880 deaths from oral cavity and pharynx cancer were expected in 2010. In addition, survival rates are poor for late stage HNC, with five-year survival rates of 54% and 32% for Stage III and Stage IV disease, respectively.¹

Although some have suggested that psychological factors, such as coping style or personality variables, may contribute more to quality of life (QOL) in HNC patients compared with disease- or treatment-related variables,² it is generally accepted that psychological variables are highly correlated with treatment- and disease-related variables. In fact, certain QOL domains may be independent predictors of important outcomes, such as survival time.³ Also, emotional symptoms affecting QOL, for example, depression, appear to modulate functional abilities, such as swallowing, in patients with HNC.⁴

A recent review of randomized controlled trials published from 1994 to 2004 found that only 22% of these studies defined QOL or symptom control as a primary endpoint.⁵ This study further stated that “current standards for analyzing QOL and symptom control in randomized controlled trials are poor” and urged further refinement of QOL measurement in cancer clinical trials. Research has shown that validated and commonly used health-related QOL instruments, even among disease-specific instruments, are not interchangeable, and it is important to consider the specific patient population in determining the instrument that is most appropriate for measuring QOL.⁶

QOL differs greatly based on the stage of disease, with higher T-stage (T3 and T4) and higher overall stage (III and IV) having lower mean QOL scores.⁷ Some QOL instruments and disease-specific modules (such as the European Organization for Research and Treatment of Cancer QOL questionnaire) have demonstrated adequate sensitivity to differentiate between T3 and T4 staging,⁸ but more refinement is being undertaken.⁹ The goal of this current work is to better assess QOL in the late stage HNC using more specifically targeted instruments.

The U.S. Food and Drug Administration (FDA) Oncology Study Endpoints and Label Development group has advanced the position that claims about health-related QOL cannot be made based on only one or two domain measurements. They suggest that assessment of symptoms should be a starting point to re-fine current QOL assessment tools.¹⁰ Furthermore, despite the existence of a number of well-validated disease-specific QOL measures, some have noted that these instruments may be of limited use in detecting meaningful treatment changes and symptoms, given their multidimensional nature.¹¹ Patient ratings of symptom importance may differ from those of oncology professionals, such as nurses and physicians.^12–14 Therefore, to meet FDA specifications, it is important to have feedback and guidance from both oncology professionals and patients. The present study is an attempt to further the scientific measurement of QOL based on these criteria. The purpose of the study is to create a new symptom checklist for advanced HNC, with the aim of including the symptoms rated as most important by cancer patients and obtaining expert oncology professional ratings of whether these same symptoms/concerns are predominantly disease-related, treatment-related, or more generally related to patient function or well-being. The study also measures reliability and validity of the new index, in the interest of presenting a symptom measurement tool that is ready immediately for use in research and clinical settings.

Methods

Design

The primary aim of the study was to combine the ratings of both patients and oncology experts to develop a modified symptom index that reflects the most important symptoms in advanced HNC. Initial evidence for validity and reliability also was obtained. This study is an extension of two previous studies. In the first study, a symptom list was created from the existing Functional Assessment of Cancer Therapy-General¹⁵ and HNC-specific subscale items.¹⁶ Oncology physicians and nurses were surveyed at National Comprehensive Cancer Network (NCCN) institutions and asked about the symptoms they viewed as priorities in advanced HNC.¹⁷ In the second study, experienced providers in treating advanced HNC were asked to identify five symptoms that were most important. The overall results of these efforts across disease sites were described in an earlier article.¹⁸ In this study, items endorsed most frequently by both patients and medical experts were retained on the disease-specific symptom indexes, and physician input was obtained on the origin of symptoms.

More specifically, patients with advanced HNC were administered the updated symptom list. The surveys began with open-ended interview questions allowing for spontaneous generation of ideas, concerns, and issues faced by those with advanced HNC. Patients were asked to identify and then rank the importance (on a 1–10 scale) of at least 10 of the priority symptoms, issues, or concerns identified. Next, patients filled out a checklist on which they selected their top symptoms and concerns from among the original items on the Functional Assessment of Cancer Therapy-Head and Neck (FACT-HN) or chosen by expert clinicians in HNC. Four versions of the checklist were created to control for order effects (response bias). Participants selected the 10 most important symptoms or concerns from this list. Then, from the 10 items selected, they were asked to identify the five “very most important” items. An open response field also was included where patients could add additional symptoms or concerns not listed.

An additional sample of physicians from NCCN member institutions were asked to rate the most probable origin of the symptoms on the checklist. The symptoms were identified as “exclusively disease-related,” “predominantly disease-related,” “exclusively treatment-related,” “predominantly treatment-related,” or “too close to determine.” In addition, physicians were given the option of choosing whether a given item was a symptom or a reflection of function or well-being.

Participants

We enrolled 40 men and nine women (N = 49) with advanced (Stages III and IV) HNC into the present study. These individuals had prior experience with chemotherapy for at least two cycles (or one month for noncyclical chemotherapy). They were recruited from three NCCN institutions (Robert H. Lurie Comprehensive Cancer Center of Northwestern University, Fred Hutchinson Comprehensive Cancer Center, and Duke University Comprehensive Cancer Center) and from participating members of the Cancer Health Alliance of Metropolitan Chicago (Cancer Wellness Center, Wellness Place, Evanston, and Glenbrook), an organization of community cancer organizations in the greater Chicago area. Exclusion criteria included other primary malignancy diagnosed within the previous five years except nonmelanoma skin cancer, inability to understand English and provide informed consent, insufficient cognitive abilities to complete questionnaires without difficulty, and younger than 18 years.

Patients rated themselves on the Eastern Cooperative Oncology Group (ECOG) Performance Status scale. Participants classified themselves into one of the following: fully ambulatory without symptoms (0), fully ambulatory with symptoms (1), requiring less than 50% of awake time to rest (2), requiring more than 50% of awake time to rest (3), and bedridden (4). All participants provided informed consent in accordance with institutional review board and The Health Insurance Portability and Accountability Act of 1996 guidelines and completed self-report questionnaires at one time point. Participating patients were paid $50 for their participation.

Eleven oncology physicians and nurses were recruited from NCCN institutions and provided evidence of at least three years’ experience in treating at least 100 HNC patients with advanced disease. Most professionals (54.6%) had more than 16 years of experience treating advanced cancer and had treated more than 2000 patients with advanced cancer (54.5%). All oncology professionals provided informed consent and were paid $100 for their participation.

Data Analysis

We calculated the frequency with which patients selected a particular symptom/concern. To determine which items would be included in the final checklist, the probability of chance endorsement as one of the top five symptoms was calculated by dividing five (the total number of “very most important symptoms”) by 28 (the total number of items on the symptom checklist) and multiplying this number by 49 (the total number of participants). Items that met or exceeded this cutoff value were judged unlikely to have been endorsed by chance and, therefore, were included in the final index. If a symptom/concern was endorsed by patients significantly more than others but did not meet or exceed the cutoff value, it was reviewed for inclusion in the index. When possible, items to be added were extracted from the Functional Assessment of Chronic Illness Therapy measurement system because of available translations and to enable efficient extraction of the symptom index from the available and valid pool of Functional Assessment of Chronic Illness Therapy items.

Write-in items were reviewed for the existence of new concepts that had not previously appeared on the checklists. Items were considered for inclusion if greater than 10% of patients in a disease category reported a particular new symptom/concern that was not judged to be treatment-related or greater than 20% of patients if the new symptom/concern was judged to be primarily treatment-related.

In terms of the physician ratings, if greater than 50% of the physicians indicated an item was exclusively/predominantly treatment-related, these items were categorized in the treatment side effect (TSE) subscale. The remaining items were judged to be either exclusively/predominantly disease-related or descriptive of general functioning. Those judged to be disease-related were included on the disease-related subscale (DRS) and those judged to be general functioning-related were included on the functional well-being (FWB) subscale.

Results

Demographic Characteristics

A total of 40 men and nine women (mean ± SD age = 55.9 ± 9.2 years) with advanced HNC participated in the study. Table 1 lists the sociodemographic characteristics of individuals in the study. Patients in the study were mainly Caucasian (77.6%), highly educated (69.4% more than high school), and largely unemployed, retired, or on disability, with only 34.7% working part- or full-time. Self-reported ECOG status of 1 (some symptoms but does not require bed rest during the waking day) was reported by 59.2% of patients, and this was similar to the physicians’ rankings (58.7% of patients were judged to have an ECOG status of 1).

Table 1.

Sociodemographic Characteristics of Participants (N = 49)

Characteristics	n (%)
Race/ethnicity
Caucasian	38 (77.6)
African American	7 (14.3)
Asian	2 (4.1)
Other	2 (4.1)
Highest education
Less than high school	9 (18.4)
High school graduate or GED	6 (12.2)
Vocational college or some college	11 (22.5)
College degree	13 (26.5)
Professional or graduate degree	10 (20.4)
Current occupational status
Homemaker	3 (6.1)
Unemployed	1 (2.0)
Retired	8 (16.3)
On disability	14 (28.6)
On leave of absence	6 (12.2)
Full-time employed	15 (30.6)
Part-time employed	2 (4.0)
ECOG performance status (MD rated)
0	9 (19.6)
1	27 (58.7)
2	7 (15.2)
3	3 (6.5)

ECOG = Eastern Cooperative Oncology Group.

Patient Ratings of Symptom Importance

Of the 28 symptoms preselected and presented on the Symptom Checklist (as described in the study by Cella et al.¹⁸), 12 items were endorsed as important with a probability greater than chance. These items are listed in Table 2.

Table 2.

Patient Rankings of Top Five Symptoms/ Concerns

Symptom/Concern	Overall Rank	% Endorsed Top Five
Pain in mouth, throat, and neck	1	39
Able to swallow	1	39
Lack of energy (fatigue)	3	35
Nausea	4	31
Able to eat foods I like	5	22
Able to enjoy life	5	22
Able to sleep well	5	22
Feeling ill	5	22
Trouble meeting needs of family	9	20
Able to eat solid foods	10	18
Able to work	10	18
Weight loss	10	18

Reliability and Validity of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22

Appropriate/similar Functional Assessment of Cancer Therapy (FACT) items were identified for the majority of priority symptoms, including those generated by patients. Expert head and neck oncology professionals classified the 22 items on the final FHNSI index into those that were primarily disease-related (DRS), primarily treatment-related (TSE), or related to functional well-being (FWB). Of the 22 symptom/concerns on the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22 (NFHNSI-22), an appropriate FACT item corresponded to all items except for five. The 22-item NFHNSI is presented in Table 3. The internal consistency reliability for the full NFHNSI-22 was good (18 items with data, Cronbach’s alpha = 0.86). The DRS (alpha = 0.83) demonstrated equally good internal consistency reliability; however, the TSE (alpha = 0.62) and FWB (alpha = 0.68) subscales demonstrated less adequate reliability.

Table 3.

NFHNSI-22 Items and Subscales

Subscales	Item	Item on the FACT-HN
DRS	I have pain	Yes
	I am losing weight	No
	I have trouble breathing	Yes
	I feel ill	Yes
	I have pain in my mouth, throat, or neck	Yes
	Because of my physical condition, I have trouble meeting the needs of my family	Yes
	I worry that my condition will get worse	Yes
	I can swallow naturally and easily	Yes
	I am able to eat the foods that I like	Yes
	I am able to communicate with others	Yes
	I can eat solid foods	Yes
	I am sleeping well	Yes
	I have a lack of energy	Yes
	I have nausea	Yes
TSE	I have had a change in the way food tastes	No
	I have mouth sores	No
	I am bothered by hair loss	No
	I am bothered by the side effects of treatment	Yes
	I have a good appetite	No
	I am able to work (includes work at home)	Yes
FWB	I am able to enjoy life	Yes
	I am content with my quality of life right now	Yes

NFHNSI-22 = National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Head and Neck Cancer Symptom Index-22; FACT-HN = Functional Assessment of Cancer Therapy-Head and Neck; DRS = disease-related subscale; TSE = treatment side effect subscale; FWB = functional well-being subscale.

The NFHNSI-22 was highly correlated with the HNC-specific subscale of the FACT (r = 0.68, P < 0.0001) and the overall Functional Assessment of Cancer Therapy-General (r = 0.83, P < 0.0001). Analyses of variance revealed that participants’ scores on the NFHNSI-22 differed significantly by performance status (P < 0.02) for the overall score, such that patients with worse performance status reported lower NFHNSI-22 scores. Follow-up subscale comparisons revealed non-significant trends in the same direction (DRS, P = 0.07; TSE, P = 0.06; and FWB, P = 0.11).

Discussion

Head and neck cancers are a “symptom-heavy” cluster of diagnoses, with significant impairments in QOL occurring through a number of mechanisms, including disease-and treatment-related mechanisms. As a result, the NFHNSI-22 is a valuable addition to the patient-reported outcomes literature in that it measures not only symptom burden and relative importance from a patient perspective but also takes into account the rankings of importance as well as causality of symptoms from the perspective of expert oncology clinicians. The NFHNSI-22 is the first symptom index developed through both qualitative and quantitative methodologies to assess advanced HNC symptomatology perceived as high priority by both patients and clinicians. The present study also measured internal consistency and relationships with outside validity indicators. The full index demonstrated good internal consistency and was associated with validity criteria, such as performance status and HNC-specific QOL measures, in the expected direction. In other words, as symptom burden increases, performance status decreases, and QOL ratings on this index are reflective of these changes.

Patients rated their top concerns as pain in the head, neck, and throat area, and ability to swallow. This is not surprising, given that both the disease and treatment can cause pain and difficulty in swallowing in patients with advanced HNCs. These impairments greatly decrease QOL, and in the event that eating becomes problematic, can lead to cachexia and fatigue, further decreasing functional capacity. Patients also rated several psychosocial concerns among their top priorities, including the ability to enjoy life, sleep well, provide for the needs of family, and work. This shows that psychosocial symptoms, in addition to physical symptoms, are important to patients with advanced HNCs and should inform provider referrals for psychosocial support and interventions.

Despite the high concordance between the FACT HNC-specific subscale and the NFHNSI-22, scores on the NFHNSI-22 cannot be directly calculated from the FACT-HN because of the inclusion of several items that are not part of the FACT-HN. Investigators wishing to calculate comparable scores using FACT-HN data may prorate the scores upward to make them comparable with the 22-item NFHNSI. Scores should be calculated using the following formula: 22 × [(sum of NFHNSI item responses)/(number of NFHNSI items completed)] as long as more than 50% of the 22 items are completed.

Several limitations of the present study warrant mention. First, the patient sample was largely Caucasian (more than 77%). Larger and more diverse samples of patients with advanced HNC should be sought for further validation. Second, the patients were predominantly well educated and may not be representative of the wide range of educational attainment in patients with advanced HNC. Third, the cross-sectional design precludes measuring test-retest reliability or responsiveness to change. Fourth, the initial data for this study were collected in 2007, and since that time, treatment paradigms have changed somewhat in the HNC area. In particular, the use of cetuximab has been found to be beneficial after five years of investigation.¹⁹ The impact of this is that other symptoms may emerge as important over time (e.g., rash in patients treated with this agent), and these changes are beyond the scope of the present study.

Several minor limitations to the data collected also are noted. First, data on tumor size and location were not collected. Second, data regarding nonchemotherapeutic treatments were not collected. Finally, there were not enough female subjects (n = 9) for a gender-based comparison of QOL reports. It is unlikely, however, that tumor location, tumor size, and patient sex would have had a notable impact on QOL concerns, as most studies have not found these variables to be impactful.²⁰

Despite these limitations, the present study represents the first effort to use both qualitative and quantitative methods to measure the importance of specific disease- and treatment-related symptoms, as endorsed by both patients and head and neck oncologists. Although there was high concordance between the FACT-HN scale and this instrument, comparison to other HNC-specific QOL measures is beyond the scope of the present study. However, future work could explore this area. This measure represents a valid brief symptom index that can easily be used with patients diagnosed with advanced HNC to measure symptom burden over time. This index will hopefully be used to monitor changes in QOL over time as chemotherapeutic and radiotherapeutic treatment progresses. Finally, this instrument is appropriate for use in both clinical and research settings as suggested by the recent FDA guidelines.