Concordance Between Patient Reports of Cancer-Related Symptoms and Medical Records Documentation

Abstract

Context

Two sources of symptom data, patient report and medical records documentation, have been used in studies focusing on chronic conditions. The concordance of patient reported cancer-related symptoms and clinician reports as documented in the medical records needs to be evaluated.

Objectives

To compare patient reports with medical record documentation of 12 disease and treatment-related symptoms for women with advanced breast cancer undergoing chemotherapy or hormonal therapy for cancer control.

Methods

Women (n=384) were recruited from 13 oncology clinics in the midwestern U.S. They completed telephone interviews at intake, 5 and 11 weeks, where they reported the presence of 12 symptoms using a checklist. Medical records were abstracted when women completed the study. The concordance between patient reports and medical record documentation was assessed using percent agreement, kappa statistics, and McNemar’s tests. Administration of medication for symptoms and patient characteristics were investigated in relation to the agreement of the two sources of data.

Results

Poor to slight agreement was found, and disagreement was significant for all 12 symptoms. The concordance between symptom presence in the medical record and administration of medication for the management of those symptoms was moderate. Patient characteristics were not associated with agreement, except for age. The agreement was higher for older women for the symptom of mouth sores.

Conclusion

Medical records may not provide adequate documentation of symptoms, and collection of patient-reported symptom data from women with advanced breast cancer is critical to quality clinical management.

Introduction

By definition, symptoms are patients’ perceptions of abnormal states (1,2); thus, the patient’s perspective in symptom reporting cannot be over-emphasized. The concordance between patient reports of symptoms and medical records documentation has been questioned, beginning with the study of Strömgren and colleagues (3). These investigators found poor agreement between advanced cancer patients’ responses to symptom questionnaires and medical records in the palliative care setting. Similar findings of low concordance have been reported for patients with chronic conditions other than cancer (4,5). Recently, this issue has been raised in the context of oncology clinical trials by Basch and colleagues (6–8), who described the results of studies where patient-clinician concordance on adverse events (Common Terminology Criteria of Adverse Events [CTCAE] and Patient Reported Outcomes [PRO-CTCAE]) was generally good, with relatively higher severity ratings obtained from patients compared with clinicians (8,9). The findings of good agreement (6–8,10,11) may lead to greater use of CTCAE and PRO-CTCAE in research and clinical practice. However, in the studies of Basch and colleagues, the design was such that both patient and clinician were specifically asked to report symptoms, and did so using the same form. Such findings cannot be translated to the natural setting where chart reviews are used as the clinicians’ data source. By contrast, the present study is unique in that it reports on data that were collected from chart reviews in a natural clinical setting.

This study begins to fill the gap in the oncology literature by evaluating patient reports and medical record documentation of 12 common cancer and treatment-related symptoms that were collected during chart abstraction. To our knowledge, this is the first large multisite study where these two sources of symptom data were compared for women with advanced breast cancer undergoing chemotherapy and/or hormonal therapy for cancer control. These data were obtained in the course of conducting a randomized clinical trial of reflexology that evaluated the effect of this complementary therapy on symptoms and physical functioning during cancer treatment. Women who participated in this trial were not receiving new investigational drugs, thus clinician symptom recording in the chart was not required as part of the assessment of the adverse drug-related events.

The research questions addressed in this paper include: 1) What is the concordance between patient reports and medical record documentation of 12 cancer and cancer treatment-related symptoms? 2) What is the concordance between pharmaceutical prescribing and documentation of symptoms in the medical record by clinicians? and 3) What patient characteristics are related to agreement or disagreement between symptom data reported by patients and documented in the medical record by clinicians?

Methods

The Sample

Women were eligible to participate in the study if they: 1) had advanced breast cancer (stage III or IV or earlier diagnosis in the medical record of stage I or II, with later recurrence or metastasis); 2) were 21 years or older; 3) were able to perform basic activities of daily living; 4) were free of diagnosis of mental illness in the medical chart; 5) were able to speak and understand English; 6) had access to a telephone; 7) were undergoing chemotherapy and/or hormonal therapy for breast cancer at intake to the study; 8) had a Palliative Prognostic Score of 11 or lower (i.e., a 30-day survival probability of ≥ 70%) (12); and 9) were oriented to time, place and person as determined by the nurse recruiter. Exclusion criteria included: 1) receiving an investigational chemotherapy drug; 2) receiving hospice care at intake; 3) living in an extended care facility; 4) bedridden; 5) undergoing bone marrow transplant; and 6) regularly using either reflexology or foot massage or pedicure with foot massage.

Patients (n=385) were recruited from 13 community-based cancer clinics throughout the midwestern U.S. Nurse recruiters from each site identified and approached eligible patients. The study was approved by the Institutional Review Boards of the investigators’ university and of each clinical site.

Data Collection

Following recruitment and consent, patients were interviewed at baseline via the telephone, and then randomized into one of three groups: reflexology, foot manipulation performed by a lay person, or standard care control. Women in the active groups (i.e., reflexology and lay foot manipulation) were blinded to their group assignment. The details about the trial are published elsewhere (13–16). Post-intervention data were collected by telephone interview at study weeks 5 and 11. Medical records were abstracted after the 11-week interview, or after women dropped out of the study. Of the women who completed an intake interview, medical record data were abstracted for all but one; therefore, this report is based on a sample size of n=384. All data collectors/abstractors (i.e., recruiters, interviewers and medical record reviewers) and oncology clinic staff were blinded to patients’ group assignments.

Measures

For this report, 12 symptoms were chosen from the structured medical record abstraction form that matched symptoms that were included in the patient interviews: dyspnea, diarrhea, insomnia, fatigue, pain, nausea, vomiting, poor appetite, constipation, mouth sores, dry mouth, and cough. In the patient interviews, two of these symptoms, pain and fatigue, were assessed using the Brief Fatigue Inventory (17) and the Brief Pain Inventory (18,19), respectively. Women were asked if they had these symptoms in the past seven days, how severe they were on a scale from 0 = no symptom to 10 = as bad as you can imagine, and how much they interfered with daily activities on a scale from 0 = does not interfere to 10 = completely interferes. The remaining 10 of 12 symptoms were part of the checklist (20). Women were asked if they had the symptom in the past week, and if yes, they were asked to identify how severe this symptom was on a scale from 1 = mild to 5 = worst possible, and to what extent this symptom disrupted regular daily activities on a scale from 1 = small extent to 5 = greatest possible extent. The symptom instruments were administered at baseline, and study weeks 5 and 11. The wording of the symptom questions in the interview maximized ease of understanding by patients (e.g., difficulty breathing or shortness of breath instead of dyspnea). For the purposes of this analysis, a symptom was considered present in self-report if women answered “yes” to the question about symptom presence in any of the three interviews.

The medical record audit covered the same 11 weeks that women were in the study, or a shorter period if women dropped out. In either case, the interviews and medical record audit covered the same period of time. A structured form was used for medical record abstraction. The form included symptoms, signs, and specific conditions. For each symptom, the chart abstractor noted on the form if a record of the symptom was present in the chart between the date of the woman’s consent and the date of the last study contact. The date of symptom documentation in the medical record also was collected. In addition to the presence of these symptoms in the medical record, data also were abstracted on medications that were prescribed for each symptom during the study period, and the dates of prescriptions. Each of the 13 participating medical oncology clinics followed their routine procedures for medical record documentation, and it was unknown whether the documentation was complete. For example, when there was no record of a symptom in the medical record, it was unknown whether the symptom was never assessed, or whether it was assessed but not recorded. In either case, no record was regarded as absence of a symptom in the medical record documentation.

Demographic and clinical characteristics were obtained during recruitment, and were confirmed at the baseline interview and through medical record audit after the woman’s last study contact. The medical record audit also was the data source for comorbid conditions (i.e., cardiac, endocrine, pulmonary, gastrointestinal, neurodegenerative, and ophthalmic). For analysis, these comorbid conditions were summarized as presence or absence, and the total number of conditions per patient.

Data Analysis

Two binary variables were analyzed. The first variable reflected the presence of a symptom in any of the three telephone interviews. The second variable reflected symptom documentation in the medical record at any time during the woman’s participation in the study (yes or no). Three methods were employed to assess concordance between the two variables. Kappa statistic was used to quantify the level of agreement according to the classification system proposed by Landis and Koch (21): <0 – no agreement, 0–0.20 – slight agreement, 0.21–0.40 – fair agreement, 0.41–0.60 – moderate agreement, 0.61–0.80 – substantial agreement, 0.81–1.00 – almost perfect agreement.

Second, because of the documented drawbacks of kappa (22–24), percent agreement was also computed for each symptom. This was done by using the number of women for whom the symptom was either present or absent in both self-report and medical record out of the total number of women, that is, the overall percent agreement reflects both agreement on presence and absence of the symptom. Further, since no single numerical summary fully describes the agreement or disagreement (24), the percentages for positive and negative agreement also were computed. Positive agreement proportion was calculated as the number of cases where a symptom was present in both self-report and medical record documentation divided by the average number of positive cases in two sources, that is, the average of the number of cases where a symptom was present in the self-report and the number of cases where a symptom was present in the medical record. This proportion was expressed as a percentage. Similarly, negative agreement proportion was computed as the ratio of the number of cases where a symptom was absent in both the self-report and medical record documentation and the average number of negative cases in both sources.

Third, in addition to descriptive measures of agreement, McNemar’s test was performed to determine if disagreement between the two sources of symptom data was significant. In order to control for the probability of type I error, a Bonferroni adjustment was planned for, such that the overall significance level remained below 5%, (i.e., significance level for each of the 12 comparisons was kept at 0.004.

To investigate the factors associated with each symptom’s documentation in the medical record, the agreement was assessed between symptom presence in the medical record and pharmaceutical prescribing for that symptom using the three methods described above. Further, polytomous logistic regression modeling was employed to examine if concordance between patient reports and medical record documentation was associated with patient characteristics such as age, education, recurrence, metastasis, or comorbid conditions. The dependent variable had three levels: 1) agreement in two sources, 2) presence of symptom in patient report, but not in medical record, and 3) presence of symptom in medical record, but not in patient report. The referent category was agreement of the information from the two sources, and the demographic and clinical characteristics listed above were used as explanatory variables for agreement versus disagreement. All analyses were conducted using SAS 9.2 (SAS Institute, Inc., Cary, NC).

Results

Figure 1 presents the flow of patients who participated in the study. Demographic and clinical characteristics of the sample are summarized in Table 1. Over a third of the women had recurrent breast cancer, and over 75% had distant metastasis.

Fig. 1.

Flow of the participants throughout the trial.

Table 1.

Demographic and Clinical Characteristics of Study Participants

	Entire Sample n=385		Those With Audit Completed n=384a
	n	%	n	%
Racea
Caucasian/White	321	83.38	320	83.33
Other	53	13.77	53	13.80
Employmenta
Employed	134	34.81	134	34.90
Not employed	249	64.68	248	64.58
Educationa
High School or Less	103	26.75	103	26.82
Some College or More	279	72.74	278	72.40
Marital Statusa
Married or living with a partner	246	63.90	245	63.80
Not married	135	35.06	135	35.16
Stage of cancera
I	20	5.19	20	5.21
II	53	13.77	53	13.80
III	126	32.73	126	32.81
IV	183	47.53	182	47.40
Recurrent Disease
Yes	132	34.29	131	34.11
No	253	65.71	253	65.89
Metastasis
Yes	298	77.40	297	77.34
No	87	22.60	87	22.66
	Mean	SD	Mean	SD
Age	55.72	11.05	55.76	11.04
Number of symptoms reported during baseline interview	8.66	4.91	8.64	4.90

^aSome data are missing.

Table 2 presents the prevalence of each of the 12 symptoms in patient reports, medical records, and the summary statistics for the agreement of the information from the two sources. Based on the values of the kappa statistic, the agreement was slight for all symptoms except nausea (kappa=0.26), mouth sores (kappa=0.22), and constipation (kappa=0.21), where the agreement was fair.

Table 2.

Prevalence of Symptoms in Self-Report and Medical Record Documentation, and Summary of Concordance for the Sample of n=384 Women

Symptom	Prevalence in Self-Report %	Prevalence in Medical Record Audit %	Kappa	Agreement %	Positive Agreement %	Negative Agreement %	McNemar’s Test P-value a
Dyspnea	44.53	14.84	0.20	63	38	74	<0.0001
Diarrhea	42.71	10.16	0.17	63	31	75	<0.0001
Insomnia	79.43	9.90	0.04	29	21	36	<0.0001
Fatigue	98.44	41.15	0.01	42	59	4	<0.0001
Pain	92.45	36.46	0.04	41	54	17	<0.0001
Vomiting	20.57	6.51	0.19	80	27	89	<0.0001
Nausea	54.17	20.05	0.26	61	48	69	<0.0001
Poor appetite	57.29	7.55	0.09	49	21	62	<0.0001
Constipation	50.78	15.10	0.21	60	40	70	<0.0001
Mouth Sores	31.25	5.73	0.22	74	30	84	<0.0001
Dry mouth	60.16	1.30	0.01	41	3	57	<0.0001
Cough	42.71	7.29	0.10	60	21	74	<0.0001

^aDisagreement was significant for all symptoms based on Bonferroni adjusted alpha of 0.004.

The percent agreement ranged from 29% for insomnia to 80% for vomiting. The agreement of two sources of data was largely driven by the agreement on the absence of symptoms. The exceptions are fatigue and pain, two symptoms with very high prevalence in self- report. Overall, the prevalence of patient-reported symptoms was much higher compared to the medical record documentation. In particular, the prevalence of pain and fatigue in the self-report was over twice that in the medical record. Because the National Comprehensive Cancer Network guidelines (25) suggest management of fatigue with a severity of 4 or higher on a 0–10 scale, the agreement between the presence of moderate or severe fatigue (at 4 or higher) in the patient report versus medical record documentation also was examined. The agreement between medical records and patient-reported fatigue in the moderate to severe range was better compared to any fatigue, but not substantially: percent agreement increased to 49%, from 41%, and kappa increased to 0.08 from 0.01. This little change in agreement is not surprising because 88% of women in this population with advanced breast cancer reported moderate or severe fatigue during at least one of the interviews.

The prevalence of patient-reported insomnia was eight times that found in the medical record documentation. The discordance was statistically significant for all 12 symptoms, and the P-values <0.0001 would indicate statistical significance after Bonferroni adjustment for multiple tests: P-values <0.004 (Bonferroni-adjusted level of significance=0.05/12).

The second research question addressed the concordance of pharmaceutical prescribing and symptom documentation in the medical record by clinicians. In other words, medication prescribed for a symptom was assessed as one of the reasons for documentation of a symptom in the medical record. The agreement was moderate for most symptoms (Table 3). It was substantial for nausea, constipation and pain, the symptoms for which pharmacologic treatments are readily available, and slight for dyspnea and fatigue, which can be harder to manage using medications. The percentage agreement was high and driven by the fact that for the medication to be documented in the medical record as the one used for treatment of a specific symptom, a symptom had to be recorded. Therefore, the negative agreement percentages are very high. Positive agreement percentage was highest for nausea and pain, the symptoms for which pharmaceutical means of symptom management are frequently used. Since antiemetics can be prescribed for either nausea or vomiting, or both when they are present or as a prophylactic measure, the combination of nausea and vomiting also was explored. When two symptoms were combined, kappa was 0.75, and percentage agreement was 92% (data not in tables).

Table 3.

Prevalence of Pharmaceutical Prescribing for Symptoms, and Summary of Concordance of Pharmaceutical Prescribing and Symptom Documentation in the Medical Record for the Sample of n=384 Women

Symptom	Prevalence of Pharmaceutical Prescribing %	Rate of Pharmaceutical Prescribing When Symptom Was Documented	Kappa for Agreement Between Symptom Documentation and Pharmaceutical Prescribing	% Agreement Between Symptom Documentation and Pharmaceutical Prescribing	Positive Agreement %	Negative Agreement %	McNemar’s Test P-value a
Dyspnea	1.82	10.53	0.16	86	19	93	<0.0001
Diarrhea	3.39	33.33	0.47	93	50	96	<0.0001
Insomnia	3.39	34.21	0.48	93	51	97	<0.0001
Fatigue	4.17	10.13	0.12	63	18	76	<0.0001
Pain	23.44	62.14	0.66	85	76	90	<0.0001
Vomiting	2.34	36.00	0.51	96	53	98	<0.0001
Nausea	15.10	71.43	0.78	93	81	96	<0.0001
Poor appetite	2.34	27.59	0.40	94	42	97	<0.0001
Constipation	8.07	50.00	0.61	92	65	95	<0.0001
Mouth Sores	2.86	45.45	0.59	97	61	98	<0.0001
Dry mouth	0.26	20.00	0.33	99	33	99	0.0455
Cough	2.34	32.14	0.47	95	49	97	<0.0001

^aDisagreement was significant for all symptoms except dry mouth based on Bonferroni adjusted alpha of 0.004.

Finally, the third research question examined patient characteristics in relation to agreement or disagreement between symptom data reported by patients and documented in the medical record by clinicians. Patient characteristics such as level of education, comorbid conditions, or disease characteristics including recurrence or metastasis were not associated with agreement. Only age showed significant associations in the polytomous logistic regression models that included all these explanatory variables (data not in tables). The strength of these associations over and above comorbid conditions was the same regardless of whether comorbidities were entered as a count or the presence or absence of specific conditions. The agreement was higher for older women for the symptoms of vomiting (P=0.04) and dry mouth (P=0.007), and lower for mouth sores (P=0.0013). However, after applying Bonferroni corrections, only the associations for mouth sores remained statistically significant: the odds ratio for the presence in the patient report, but not in the medical record versus the agreement of two sources was 1.03 for each year of age (95% confidence interval 1.01, 1.06), data not in tables).

Discussion

Cancer symptoms present a major burden to the patient as well as a substantial public health concern (26). Patient-reported symptoms are an example of patient-reported outcomes (PROs), and much work has been performed in developing standards and measuring PROs in cancer and other chronic conditions (2,27–29). The collection of PROs has been proposed as a regulatory requirement for reporting of adverse events in cancer clinical trials (30) and has been facilitated by the Food and Drug Administration guidance on the use of PROs in medical product development to support labeling claims (27). When a symptom corresponds to a high-grade adverse event, it is required to be reported under the National Cancer Institute reporting system (31).

With the strong impetus to measure and incorporate PROs in oncology (29), it is critical to evaluate patient symptom reporting in relation to the documentation by clinicians as was done in the present study. Few similar reports exist among cancer patients, and their conclusions vary. Some note that patients are willing to report symptoms to their oncologists (6,7,32), whereas others present discrepancies between patient-reported and provider-reported symptom data, including lower numbers of symptoms and decreased severity in provider reports (3,33–35). In the study by Strömgren et al. (3), which comprised 58 cancer patients in palliative care, good concordance was found only for pain. Our study was done with patients in medical oncology settings and included only advanced breast cancer, and similar poor agreement was found for multiple symptoms as well as pain. In contrast to other studies with an experimental design (6, 10, 11), the present study addressed real world clinical situations that occur outside of trials mandating symptom reporting for drug labeling or other purposes.

The results support the need to collect symptom data from patients. The findings point out that except for mouth sores, vomiting and constipation, symptoms with relatively low prevalence compared to other symptoms, medical records may not provide adequate documentation of symptoms experienced by women with advanced breast cancer. The discrepancy in symptom documentation may be patient and/or health care provider driven. It could be the result of the use of symptom instruments (or checklists) when collecting data from the patients, whereas medical record documentation may be used to reflect clinician observations. Related to patient-reported symptoms, Homsi et al. (36) found that the median number of symptoms reported by patients in palliative care using a systematic assessment was 10 times higher than the median number of symptoms that patients voluntarily reported without prompting. When patients do not volunteer their symptoms to clinicians during office visits, these symptoms may or may not be noted by clinician and documented in the medical record. One potential explanation for this could be that patients are hesitant to report high symptom levels such as fatigue, for fear their treatment might be altered to a less aggressive protocol.

A potential provider-driven explanation for the lack of concordance in oncology is that the clinician is required to document only signs and symptoms relevant to the patient’s cancer diagnosis. This practice may result in a lower rate of symptom presence in the oncology medical record compared to patient reports, especially in cases where the symptom could be attributed to a comorbid condition other than cancer. In this study, comorbidity was considered as an explanatory variable for concordance, but the finding was negative.

Pharmacologic management of symptoms is a good predictor for the presence of symptom documentation in the medical record. Symptoms for which no medication is available, demonstrate poor agreement between patient reports and medical records. It is possible that clinicians ask the patient primarily about symptoms for which a prescription medication is available, and this could explain a somewhat better concordance.

Another source of motivation for greater concordance between patient reports and medical record chart-documented symptoms may come from the American Society of Clinical Oncology’s Quality Oncology Practice Initiative (QOPI), which stresses documentation of all symptoms as part of symptom management (37). The QOPI could increase the rates of symptom documentation in medical records, and research on symptom reporting could inform and facilitate its implementation. The modules of QOPI relevant to the population of women with advanced breast cancer include a symptom toxicity/management module and end-of-life module. One item in the symptom toxicity module addresses the administration of chemotherapy with moderate or high emetic risk. A second example is provided by the end-of-life QOPI module that includes a pain assessment item, which could improve chart documentation of pain, and raise the level of agreement between patient reports and the medical record. According to data from the present study, prevalence of symptoms according to chart documentation represented less than half of the prevalence in patient report. Although recent QOPI data have shown high quality of care (37), there is still variation from one practice to another and room for improvement. In addition, these specific QOPI modules could facilitate the documentation of symptoms that are listed in the modules, but would not identify other symptoms that patients may be experiencing and not reporting to the clinicians. Thus, a more comprehensive assessment of cancer symptoms may be needed in clinical practice, and this report could inform clinicians as to potential directions for improvements in symptom documentation.

The limitations of this research include a relatively short list of symptoms. Longer lists were used in the actual interviews and in the medical record audit, but only symptoms with a very close conceptual match in wording were analyzed. The symptom lists that could be used by clinicians during patient visits in the various oncology settings may differ across practices, and not match the list on the structured form that was used to perform medical record abstraction in this study. The findings of this study apply to advanced breast cancer patients in the medical oncology setting, and may not be generalizable to other sites of cancer and clinical settings.

Despite these limitations, the consistency of findings across multiple symptoms indicates that often symptoms are not adequately documented in the medical records, and the collection of symptom data from women with advanced breast cancer could be one of the steps to better symptom management and improving quality of life during cancer treatment.