Validity of self-reports of breast cancer treatment in low-income, medically underserved women with breast cancer

Abstract

Few studies have assessed the agreement between subjects’ self-report and medical records among patients with breast cancer (BC), and none has addressed this issue in low-income women with BC. We assessed the level of agreement between self-report and medical records data for key BC treatment and prognostic characteristics using correct proportion and the Kappa statistic, among 726 low-income BC patients. Unconditional regression was used to investigate the association between accuracy of self-report and potential explanatory factors. Overall agreement between self-report and medical records was 95.3–99.6% for BC treatments including surgery, chemotherapy, radiotherapy and hormone therapy (Kappa = 0.79–0.99). Specific agreement was 87–89.5% for surgery type (Kappa = 0.51–0.96); 86.3% for chemotherapy completion (Kappa = 0.46) and 98.7% for radiotherapy completion (Kappa = 0.43); 95.2% for medical oncologist consultation (Kappa = 0.59) and 96% for radiation oncologist consultation; 97.3% for metastasis (Kappa = 0.56); and 93.6% for recurrence (Kappa = 0.30). When accepting answers within 15 days of the medical record date, 78.2% of women correctly reported surgery date, yet only around 55% of women correctly reported the start and/or end date of radiotherapy. Older age, less education, BC recurrence and poor patient–physician communication were associated with the lesser accuracy of patients’ self-report compared to medical records (P < 0.05). The results of this study suggest that self-reporting of key treatment and prognostic information is relatively accurate among low-income women with BC. Self-report seems to be a reliable source for accurate information when medical record review is unavailable or unfeasible. Interventions to enhance patient–physician communication may facilitate more accurate information reporting among vulnerable populations.

Introduction

Breast cancer (BC) is the most common malignancy among US women [1]. Diagnosing for BC is devastating under any circumstances, but especially so for those who are medically uninsured [2]. The vulnerable population of poorer and less educated women is more likely to present with late-stage BC and less likely to survive 5 years [3, 4]. Medical records are considered to be the most accurate data resource for assessing quality of care. However, most studies rely on self-report to collect data rather than medical records because of the feasibility and cost of accessing medical records. Yet, self-report data may be susceptible to biases (i.e., recall, social desirability, and health knowledge and awareness) [5, 6]. Validation between patients’ self-report and medical records has been assessed in other settings and for other outcomes such as utilization of health care services and cancer-screening examinations. The level of agreement varies from low to high depending on the study setting and type of data collected [7–10].

Treatment information and prognostic characteristics are key measurements of quality of care for BC. Despite a rigorous medical literature review, only two studies have assessed the accuracy of patients’ self-report of BC treatments, and these were performed in Canada and Australia [11, 12]. Although both studies reported high agreement between self-report and medical records for BC treatment, their usefulness is limited. It can be very challenging to obtain medical records for low-income population because of their limited access to specialty care and frequent use of multiple sources of care. Assessing the accuracy of the self-report information for this population of previously uninsured low-income women can provide further scientific support for the reliability and validity of self-report data in studies of vulnerable populations.

Furthermore, identifying those factors that contribute to higher or lower levels of accuracy may provide potential interventions to enhance the data quality from patients’ self-report. Efficient patient–physician communication and interaction are associated with increased patient knowledge about BC [13], which can lead to greater accuracy of self-reports [14]. Most of the prior validation studies focused on the relationship between the accuracy of self-report and patient demographics [15–18], but no study has focused on the relationship between patient–physician communication and the accuracy of patients’ self-report.

In a diverse, statewide sample of low-income women receiving treatment for BC through the California Breast and Cervical Cancer Treatment Program (BCCTP), we assessed (1) the agreement between patients’ self-reported data and medical record data focusing on BC treatment, metastasis at the time of presentation and recurrence up to 18 months after diagnosis and (2) the association between accuracy of self-report and potential explanatory factors including patient–physician communication.

Methods

Study sample

A low-income population of women aged 18 years and older, newly diagnosed with BC and continuously enrolled in the California BCCTP between February 2003 and September 2005 was recruited for this study. The study was approved by the UCLA Human Subjects Protection Committee. The BCCTP is funded in part through Medicaid and through the state of California to fund the treatment of breast and cervical cancer for un- and underinsured, low-income women (≤ 200% Federal Poverty Level).

Eligible women were interviewed by telephone at 6 and 18 months after their diagnosis of BC. Participants also agreed to release their medical records up to 18 months post diagnosis for the performance of a structured chart abstraction. Women who did not speak English or Spanish, had a previous history of BC, or were receiving treatment for another cancer were excluded from the study. A total of 921 women were initially recruited for the baseline interview at 6 months, with a 61% response rate. Compared with survey responders, nonresponders were older (52 vs. 50 years, P < 0.05), more likely to be Asian and African American and less likely to be Latina (9, 8, 46 vs 4, 6, 56%, respectively, P < 0.05). Of those who completed the baseline interview, 123 did not finish the 18-month follow-up interview due to death or refusal or because they were lost to contact. Medical records were obtained for 91% of the 798 women who finished both interviews, for a final sample size of 726 subjects in this study. There were no significant differences in self-reported age, race, educational background, income and health status between participants in this validation study and those who were excluded from the study because of the unavailability of medical record. Further details of the parent study can be found in a previously published article [13].

Self-reported information

The questionnaires for this study were initially developed to assess the impact of patient–physician interaction on receipt of indicated BC care and health-related quality of life. Data collected included detailed information on patient-level characteristics, health care system–level characteristics, interpersonal characteristics and measures to assess quality of care in BC.

Women were interviewed by telephone 6 and 18 months after their diagnosis of BC, interviews were 30–45 min in duration, and all interviews were conducted by trained bilingual interviewers. In this validation study, we used responses from both interviews to questions related to BC treatments, metastases and recurrences to ensure comparability to the period of time covered by the medical records.

Medical abstraction information

Medical records from each cancer provider seen by the participant were obtained to include clinical records over the 18-month period from the time of BC diagnosis. This duration was chosen because based on treatment guide-lines, the California BCCTP only covered BC treatment for up to 18 months from the time of diagnosis. The medical records were obtained by both external and internal means and after which a structured medical record abstraction was implemented and conducted by trained nurse abstractors supervised by one of the physician co-investigators.

Measures

Specific to the validation study, key information from the self-report questionnaires and medical abstraction included type and dates of breast surgery, details of adjuvant therapy (chemotherapy, radiotherapy, hormone therapy; dates of treatment, treatment completion and treatment consultation) and information on metastasis and recurrence. For each treatment, the woman was first asked whether the treatment was received (e.g., “Did you have surgery to remove your breast cancer?”). For those who reported receiving specific treatment, more detailed information of the treatment was then asked (e.g., “Do you remember when you had surgery to remove the breast cancer?”, “What type of surgery or surgeries did you have?”). In addition to information on key treatment and prognostic characteristics, sociodemographic information (age, educational background, marital status, race/ethnicity), personal health characteristics (perceived health status, comorbidities) and patient–physician communication were also used to investigate the association of accuracy of self-report with potential explanatory factors. Patient–physician communication was measured in the following ways: (1) interactive information-giving by physicians, (2) patient-perceived self-efficacy in patient–physician communication and (3) physicians’ encouragement of patient’s participation in decision-making. Information-giving was measured by a previously published index [19], which asked patients how many of 15 BC-related topics their physicians had discussed with them. Self-efficacy was measured using the validated Perceived Efficacy in Patient–Physician Interactions (PEPPI) questionnaire [20]. PEPPI measures patients’ perceived ability to obtain needed medical information and attention to their chief medial concerns from physicians. The PEPPI sum scale ranged from 0 to 50 with a 0.96 Cronbach’s alpha. Physicians’ encouragement of patient’s participation was measured by “How much did your breast cancer doctors ask you for your input or opinion about which treatment you preferred?” The response set was a 4-point Likert scale ranging from “not at all” to “a great deal.”

Language can serve as a significant barrier to optimal communication; therefore, among Latinas, language use and preference was determined by the five-item Marin Acculturation Scale [21]. The internal consistency reliability was 0.99 for this scale in the studied sample. “More acculturated” Latina was defined as being equally or more comfortable or conversant with English than Spanish; the ‘less acculturated” Latina was defined as being less comfortable or conversant with English than Spanish.

Data analysis

Summary statistics, including means or percentages, were calculated to describe participants’ demographic characteristics. Self-reported information was compared to medical record data to evaluate agreement between these data-collection sources on BC key treatments and prognosis characteristics. Considering medical record abstraction as the gold standard for data collection, the consistency of self-report with data from medical records was evaluated by both concordance and Kappa statistic. “Concordance” is defined as the proportion correct between self-report and medical record abstraction. Kappa coefficient was also used to adjust for agreement that may occur by chance [22]. Values of Kappa can range from −1 to +1. Negative values indicate agreement worse than chance, 0 is agreement by chance alone, and positive values signify agreement better than chance. A value less than 0.21 is considered to be slight agreement; 0.21–0.40 fair agreement; 0.41–0.60 moderate agreement; 0.61–0.80 substantial agreement; and values over 0.80 perfect agreement. Based on Landis and Koch’s suggestion [23], patients who responded as “unknown” were treated as “no” for data validation evaluation. For questions that were asked in both interviews, patients who responded with “yes” in either of the questionnaires were considered as “yes” for data validation evaluation. Patients who answered within 15 days of the dates documented in the medical abstraction were considered to have responded correctly.

We also conducted unconditional logistic regression analyses to examine the association between the accuracy of self-report and potential explanatory factors, including age, educational background, marital status, race/ethnicity, perceived health status at the time of baseline questionnaire administration and patient–physician communication, coding accuracy of self-report as “yes” or “no” based on concordance with the medical record.

All statistical analyses were conducted using SAS, version 9.1; two-sided alpha levels of p values less than 0.05 were considered to be statistically significant.

Results

Table 1 shows the sociodemographic and health characteristics of the participants. The average age was 51 years, and the respondents were mainly Latina (52.5%) and white (33.2%). Approximately half were married or partnered, and slightly more than half had graduated from high school. Most of the participants were unemployed (81.7%), and almost two-thirds had an annual household income less than $20,000. Approximately 30% of the participants reported one or more comorbidities; and one-third reported fair or poor health status. According to data from the medical records, most of the participants (96.6%) received surgery treatment for their BC. Among these women, 44.5% underwent mastectomy, and 52.1% underwent lumpectomy. Overall, 93.9% women received at least one type of adjuvant treatment (radiotherapy, chemotherapy, hormone therapy), 40.6% received two types and 34.8% received all three types. Five percent of women had a documented recurrence within 18 months.

Table 1.

Descriptive statistics of the study sample (N = 726)

	Value
Age (years)
Mean (SD)	51.0(9.5)
Range	25.0−85.0
Ethnicity, N (%)
White	241(33.2)
Latina	381(52.5)
African American	41(5.7)
Asian/Pacific Islander	52(7.2)
Other	11(1.1)
Education, N (%)
<High School	299(41.2)
≥High School	427(58.2)
Married/Partnered, N (%)
No	369(50.1)
Yes	357(49.2)
Employment, N (%)
No	593(81.7)
Yes	133(18.3)
Income, N (%)
<$20,000	458(63.6)
>=$20,000	262(36.4)
Comorbidity (N, %)
None	511(70.4)
Any	215(29.6)
Health Status (N, %)
Fair/Poor	241(33.2)
Good/Excellent	485(66.8)

Agreement between self-report and medical records regarding the proportion of concordant responses and the Kappa statistics is summarized in Table 2. A very high proportion of women (98.8%) were able to correctly identify whether they had undergone surgery for BC (Kappa = 0.79). Agreement for surgery type was 87.9% for mastectomy (Kappa = 0.96), 87% for lumpectomy (Kappa = 0.72) and 89.5% for lymph node dissection (Kappa = 0.51). Most of the women (78.2%) reported the surgery dates within 15 days of the dates documented in the medical records.

Table 2.

Agreement between self-reports and medical records for low-income women with breast cancer

	Proportion correct (%)	Kappa
Breast cancer surgery (Yes/No)	98.8	0.79
Type of surgery
Mastectomy (Yes/No)	87.9	0.96
Lumpectomy (Yes/No)	87.0	0.72
Lymph nodes (Yes/No)	89.5	0.51
Date of surgery (±15 days)	78.2	N/A
Chemotherapy (Yes/No)	99.6	0.99
Completion of chemotherapy (Yes/No)	86.3	0.46
Consultation with medical oncologist (Yes/No)	95.2	0.59
Radiotherapy (Yes/No)	97.7	0.94
Completion of radiotherapy(Yes/No)	98.7	0.43
Date of radiotherapy (±15 days)
Start date	56.4	N/A
End date	58.6	N/A
Consultation with radiation oncologist (Yes/No)	96.0	0.90
Hormone therapy (Yes/No)	95.3	0.90
Metastasis (Yes/No)	97.3	0.56
Recurrence (Yes/No)	93.6	0.30

N/A not applicable

In unconditional regression analyses, no factors were found to be associated with the accuracy for self-reports on type of surgery performed. However, women with at least one comorbidity and those who received more information from their physicians were more likely to correctly report whether or not they had undergone a lymph node dissection (OR = 2.13, P = 0.01; OR = 1.08, P = 0.04, respectively), while both less acculturated Latinas and Asian/ Pacific Islanders were less likely than whites to report correctly on this question (OR = 0.54, P = 0.04; OR = 0.28, P = 0.004; respectively).

Agreement between patients’ self-report and the medical record was excellent for all three adjuvant treatments. Over 95% of the participants correctly reported receiving chemotherapy, radiotherapy or hormone therapy, with Kappa values all above 0.90. No factors were significantly associated with the accuracy of chemotherapy reporting. However, women who had a BC recurrence were less likely to correctly report whether they had received radiotherapy (OR = 0.24, P = 0.03). Subjects with fair or poor heath status (OR = 0.43, P = 0.003) or lower self-efficacy (OR = 0.44, P = 0.04) were associated with less accuracy of self-response for the hormone therapy question.

Over 95% the women were able to correctly recall whether they had consulted a medical oncologist for chemotherapy (Kappa = 0.59) or a radiation oncologist for radiotherapy (Kappa = 0.90). Younger age, being married or having a significant other and receiving more information from physicians were positively associated with the accuracy of self-response for oncologist consultation. Women with better self-efficacy (OR = 2.6, P = 0.03) and those who received more BC treatment-related information (OR = 1.12, P = 0.01) were more likely to correctly report having seen a radiation oncologist. Also, most women correctly reported that they had completed chemotherapy (86.3%, Kappa = 0.46) or radiotherapy (98.7%, Kappa = 0.43). Greater participation in treatment decision-making was positively associated with accurately reporting the completion of chemotherapy (OR = 1.25, P = 0.03), while women who had a BC recurrence were less likely to correctly report completing radiotherapy (OR = 0.22, P = 0.001). Only about half of the women correctly reported their start and end dates of radiotherapy (56.4, 58.6%, respectively). Women with higher educational background were more likely to report correct dates of radiotherapy (OR = 1.43, P = 0.04), while African Americans were less likely than whites (OR = 0.41, P = 0.03).

The proportions of women who correctly reported their metastasis and recurrence status were also high. Over 97% (Kappa = 0.56) of subjects accurately reported whether their BC had metastasized at the time of diagnosis, and at least 93% (Kappa = 0.30) of the subjects reported their recurrence status correctly. Younger patients (OR = 2.08, P = 0.02) were more likely to report metastasis status correctly, and African Americans had greater accuracy on reporting recurrence status (OR = 2.32, P = 0.005).

Discussion

This study of agreement between patients’ self-report and medical records regarding BC treatments and prognostic characteristics for accuracy is the first such study performed in a low-income population of women with BC, as well as being the first to examine the impact of patient–physician communication on recall of disease events. Overall, findings from this study indicate high levels of agreement between information reported by women and that obtained via a systematic structured medical records abstraction for key BC treatment and prognostic characteristics. Dimensions of physician communication of BC information-giving, patient involvement in decision-making and patient empowerment in interacting with physicians appeared to guide patients to correctly report events of disease treatment and prognosis. These results suggest that self-reported information from low-income women with BC is reliable substitute for obtaining key medical data. In addition, improving patient–physician communication would be a potential intervention to enhance the quality of data collected via from self-report.

Over 95% of the women correctly reported the broad questions about whether they received BC surgery or adjuvant therapy, yielding almost excellent chance-corrected agreements (Kappa = 0.79–0.99) for these questions. For the subquestion about details of the treatment, agreement was also high for type of surgery, consultation and completion of chemotherapy and radiotherapy, and in all instances the proportion correct was at least 86%. These findings are consistent with those from previous studies, thereby supporting the supposition that events related to cancer treatments are easy to recall [12, 24]. It has been noted that people who have undergone a sudden and life-threatening health crisis manifest very clear recall of the details surrounding the event [25]; patients with BC, for example, can easily recall the precise time when they first noticed their symptoms [26] and information on treatment because of the rareness and high personal/economic/social cost [27].

The ability to recall disease events is also influenced by patient’s cognitive competency [28]. Patient characteristics such as age, education and health status might impact this competency of retrieving data from memory and have been found to be associated with self-report accuracy [15, 17, 18, 29, 30]. Consistent with these previous studies, our findings showed that women who were older, were in poor/fair health status, had lower educational background or had BC recurrence were less likely to report correctly to certain questions. Our findings also showed that patient–physician communication can affect recall. Worse patient–physician communication seemed to predict less accuracy on self-reports. Women who had lower self-efficacy capabilities were less likely to report correctly for the hormone therapy question, and women who received more BC-treatment-related information had greater accuracy on reporting chemotherapy/radiotherapy consultation. The explanation may be that greater self-efficacy in interacting with physicians and physician information-giving have been found as powerful predictors of BC knowledge, [13] and enough clinical knowledge was essential to get accurate patient reports [14]. Our findings highlight the importance of patient–physician communication on improving self-reports accuracy.

To our knowledge, only two studies examined the accuracy of self-report for BC treatment; however, these studies were limited by small sample size and non-US settings [11, 12]. Consistent with the high agreement between self-reports and medical records found in these two studies, our study adds to the literature about the value of reliability of self-report for low-income women with BC. Medical records for low-income populations may be difficult to obtain because they are disproportionately uninsured, lack access to specialty care and frequently use multiple sources to fulfill their medical needs. Moreover, the use of the medical record as a golden standard is sometimes questionable for certain types of data. For example, assessment of health behaviors, education and counseling, follow-up about behavior change and compliance with recommendations might be underreported in medical records due to the time pressure on physicians to record them [31, 32].

The results from this analysis should be evaluated in the context of its sample characteristics that may limit the generalizability of the findings. Because the study was conducted in a sample of low-income, medically underserved women in a specific Medicaid BC treatment program in California, external generalizability of the findings to other low-income populations may be limited. This study is also limited by the number of care and treatment measures that were tested.

In conclusion, this study demonstrates that low-income, medically underserved women with BC are able to correctly report key BC treatment and prognostic information. Self-report seems to be a more cost-effective method of collecting accurate information in this population, because of the complications and difficulties of accessing medical records. In addition, interventions to enhance patient–physician communication may facilitate more accurate information reported in this vulnerable population. However, studies similar to our study need to be replicated in a wider population of women with BC to evaluate whether our findings may be more applicable to women in general.