Abstract
Among high-risk breast cancer (BC) survivors, genetic counseling (GC) and genetic testing (GT) may inform cascade testing and risk management. Compared to non-Hispanic White BC survivors, Spanish-preferring Latina BC survivors are less likely to report discussing GC with a healthcare provider. However, few studies have examined Latinas’ experiences with GC/GT, particularly outside of the mainland USA. This study aimed to compare frequency of provider discussion of GC between Spanish-preferring Latina BC survivors living in Florida (FL) and Puerto Rico (PR). We conducted secondary data analysis of baseline assessments from a randomized pilot of an educational intervention for Spanish-preferring Latina BC survivors. Participants (N = 52) were GC/GT-naive, but met clinical criteria for GC/GT referral. Participants self-reported sociodemographic, clinical, and cultural variables, including previous provider discussion of GC. Descriptive statistics characterized frequency of GC discussion. Logistic regression examined the relationships between sociodemographic, clinical, and cultural characteristics and GC discussion. Only 31% of participants reported previous GC discussion. More participants from PR reported having GC discussions (43% vs. 21% in the mainland USA). In multivariable analyses, greater likelihood of GC discussion was associated with PR (vs. mainland USA) residence (odds ratio [OR] = 6.00, p = .03), older age at baseline (OR = 1.19, p = .04), and younger age at BC diagnosis (OR = 0.80, p = .03). Few high-risk Spanish-preferring Latina BC survivors in the mainland USA and PR had discussed GC with their providers. These results highlight a gap in the implementation of evidence-based genetics guidelines. Provider-directed interventions may be needed to increase uptake of GC/GT among Latina BC survivors.
Keywords: Breast cancer, Genetic counseling, Genetic testing, Healthcare provider, Hispanic, Latina
Although they were all eligible for genetic counseling, few Spanish-preferring Latina breast cancer survivors had previously discussed genetic counseling with a healthcare provider.
Graphical Abstract
Implications.
Practice: High-risk Spanish-preferring Latina breast cancer survivors are not receiving adequate information about genetic counseling from their healthcare providers. It is imperative that providers identify those at high-risk and promote genetic counseling and testing in this population.
Policy: Healthcare policies must provide coverage for genetic counseling and testing for high-risk breast cancer survivors and address ethnic/racial inequities in preventive care. For Spanish-preferring survivors, policies that increase access to Spanish-speaking providers are critical.
Research: Future research should continue to evaluate barriers to genetic counseling and testing faced by Spanish-preferring Latinas in and outside the mainland USA. The development and testing of culturally sensitive interventions is needed to improve equitable access to genetic counseling and testing.
Introduction
Breast cancer (BC) is the leading cause of cancer incidence and mortality among Latinas and Hispanic women (hereafter referred to as “Latinas”) in the USA [1]. Compared to non-Hispanic White (NHW) women, Latinas in the USA are diagnosed with BC at a younger age [2], at more advanced stages [2, 3], and with BC subtypes that are more aggressive (e.g., triple-negative and HER2-overexpressing tumors) [2]. Latinas are also less likely to receive guideline-concordant treatment for BC [3], and have a greater risk of mortality due to BC [2].
Inherited genetic variants also contribute to cancer disparities among Latinas [4, 5]. Pathogenic variants in the BRCA1 and BRCA2 (BRCA1/2) mutations account for the majority of hereditary BCs [6, 7]. Pathogenic variants in the BRCA1/2 genes are associated with lifetime risks of up to 72% for BC [8–10], 83% for contralateral BC [11], and 59% for ovarian cancer [8–11]. Once identified, women with BRCA1/2 mutations have additional options for cancer treatment and risk management for secondary primary cancers [7, 12]. In BRCA1/2 negative families, identification of pathogenic variants in high- and moderate-penetrance genes (e.g., ATM, CDH1, PALB2, etc.) may also impact screening and treatment recommendations [12]. Identification of pathogenic variant carriers can also facilitate targeted cascade testing and primary prevention for at-risk family members [13]. For these reasons, the National Comprehensive Cancer Network (NCCN) recommends genetic counseling (GC) and genetic testing (GT) for individuals at high risk for genetic variants [12], including women who are diagnosed with early-onset BC (i.e., diagnosed age ≤45 years), have known blood relatives with pathogenic variants in a cancer susceptibility gene, have multiple blood relatives with BC, and are of Ashkenazi Jewish ancestry.
Although the prevalence of pathogenic BRCA1/2 variants among Latinas is estimated to be similar to other U.S. racial/ethnic groups (excluding Ashkenazi Jewish women) [14, 15], they are less likely than NHW women to be tested for BRCA1/2 variants after a BC diagnosis [16]. Additional disparities exist based on language preference; Spanish-speaking Latina BC survivors are less likely to be referred for GT than English-speaking Latina survivors [17, 18].
Because Latinas report lower awareness of GC/GT than other racial/ethnic groups [19, 20], provider recommendation is crucial for addressing disparities in GC/GT uptake. Prior research has demonstrated that provider discussion of GC/GT is more common among patients who are younger, have higher income, are not employed outside the home, have a family history of breast or ovarian cancer, and are at higher risk of carrying a genetic mutation [18, 21]. In addition, Latinas—particularly those who are Spanish-preferring—are less likely to have discussed GC/GT with a healthcare provider [18, 22]. Latino cultural values, such as religiosity and familismo (strong identification with and attachment to one’s family [23]), may contribute to GC/GT discussion. Previous research has shown that familismo motivates Latina women to participate in GC/GT, while spirituality may hinder participation in GC/GT [24–27].
However, these studies were conducted in the mainland USA; less is known about the experiences of Latinas living outside mainland USA, who may encounter different structural barriers related to GC/GT. Unique barriers for Spanish-preferring Latina survivors in the mainland USA may include language barriers (particularly regarding access to Spanish-speaking genetic counselors), obtaining accurate family histories, fear of discrimination, medical mistrust, lack of cultural sensitivity when receiving healthcare, and immigration status [24, 28, 29]. In Puerto Rico (PR), unique barriers to GC/GT may include scarcity of services, distance to care (most clinics offering clinical cancer GC/GT are in the largest cities), and limitations in Medicaid/Medicare reimbursement system (due to PR’s status as a U.S. territory) [30–32]. Compared to individuals in the mainland USA, PR residents report more difficulties in getting needed care and getting care quickly, but better doctor communication experiences [33]. Given these unique barriers, rates of GC discussions may differ between Spanish-preferring Latina BC survivors residing in the mainland USA and PR.
To fill this gap in knowledge, we surveyed Spanish-speaking Latina BC survivors living in the mainland USA (Tampa, FL) and PR (Ponce, PR) about prior discussion of GC with their healthcare providers. We examined differences in prior GC discussion between survivors in the mainland USA and PR, and explored the relationships between sociodemographic, clinical, and cultural characteristics, and prior GC discussion.
Methods
Participants and procedures
This is a secondary analysis of baseline data from a multisite pilot randomized clinical trial (RCT) testing a culturally targeted educational intervention to increase uptake of GC/GT among Latina BC survivors [34]. All procedures were approved by the Moffitt Cancer Center (IRB #18601) and Ponce Health Sciences University-Ponce Research Institute (IRB #160607-EC) Institutional Review Boards. This study followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines [35], and the CONSORT diagram has been previously published [34].
Briefly, eligible participants were (i) female BC patients, (ii) self-identifying as Latina, (iii) age ≥25, (iv) Spanish-preferring, (v) GC/GT-naive, and (vi) eligible for GC/GT referral based on 2017 guidelines from the NCCN [36]. We recruited participants from January to June 2017. Based on our prior work [37, 38], we primarily posted/distributed study flyers through institutional and community-based clinics and registries, community oncology clinics, and cancer support groups. We also attended a few in-person events (e.g., cancer education events, health fairs) hosted by our community partners. Finally, we advertised the study through Spanish language media channels and social media sites. Recruitment utilized our robust community outreach infrastructure, which includes community advisory boards in both Tampa and PR representing BC survivors, advocacy groups, and community organizations serving the Latino community [39, 40]. Individuals interested in participation called a toll-free telephone number to learn more about the study. A bilingual bicultural research assistant screened callers for eligibility and mailed eligible individuals two copies of the informed consent and a business reply mail envelope. During in-person recruitment (e.g., at community events, clinic), a bilingual bicultural research assistant immediately screened and consented potential participants face-to-face.
All RCT participants who consented and completed the baseline assessment (N = 52) were included in this secondary data analysis. Participants did not consent separately into these analyses; the informed consent for study participation included consent for their data to be used for secondary data analysis. Participants did not receive any additional compensation for this secondary data analysis.
Measures
Spanish-language baseline surveys were completed in person, via mail, or by telephone (depending on participant preference).
Sociodemographic and clinical characteristics
Participants self-reported age, race, partner status, education, employment status, income, insurance status, age at first BC diagnosis, and stage of most recent BC diagnosis. Residence (mainland USA vs. PR) was documented by research staff as part of study enrollment.
Provider discussion of GC
Past experiences with provider discussion of GC were assessed using a single face-valid item: “Prior to participating in this study, did you discuss genetic counseling with a healthcare provider?” Participants could respond “yes” (1), “no” (2), or “I don’t know” (3). Participants also indicated which type(s) of provider was involved in the discussion (surgeon, medical oncologist, radiation oncologist, nurse, primary care provider, or other) and who initiated the discussion (the healthcare provider, a family member, me, other, or don’t remember).
Cultural characteristics
We assessed two cultural constructs important to Latina BC survivors [41]: religious coping and familismo. Based on the prior literature [24–27], we hypothesized that religious coping would be negatively associated with GC discussion and familismo would be positively associated with GC discussion.
Religious coping.
The Brief RCOPE [42, 43] includes two 7-item subscales assessing: (i) positive religious coping (e.g., “I looked for a stronger connection with God”) and (ii) negative religious coping (e.g., “Wondered whether God had abandoned me”). Items are rated on a four-point Likert scale from 1 (“not at all”) to 4 (“a great deal”) and summed to create subscale scores ranging from 7 to 28. Higher scores indicate greater use of coping strategies. For the present study, instructions were modified to specify coping with one’s cancer diagnosis. Internal consistency was α = .91 for positive religious coping and α = .94 for negative religious coping.
Familismo.
Familismo is a core value of the Latino culture [44], and is characterized by strong identification with and attachment to one’s family [23]. Lugo Steidel and Contreras’ Attitudinal Familism Scale [45] assesses how much people endorse familismo as a core value. Items are rated on a five-point Likert scale from 1 (“strongly disagree”) to 5 (“strongly agree”) and averaged to create a total score ranging from 1 to 5. Higher scores indicate a stronger sense of connectedness to family members. Internal consistency was α = .74.
Analytic strategy
First, chi-squared tests (for categorical variables) or t tests (for continuous variables) examined whether sociodemographic, clinical, or cultural characteristics differed by residence (mainland USA vs. PR). Second, descriptive statistics characterized frequency of GC discussion overall and by residence (mainland USA vs. PR). Finally, multivariable logistic regression models explored the relationship between sociodemographic, clinical, and cultural characteristics and prior GC discussion. Participants who responded “I don’t know” to the provider discussion item (n = 2; 4%) were excluded, allowing for comparison of a binary (yes/no) outcome. We used a backward stepwise procedure for selection of predictor variables. Given the lack of data on this topic, we set p <.2 as the criterion for staying in the model. Model goodness-of-fit was evaluated by Hosmer and Lemeshow’s test [46], with a nonsignificant χ2 (p > .05) indicating adequate goodness-of-fit. All analyses were conducted using SPSS (version 27, IBM). Cases with missing data were deleted listwise. All tests were two-tailed and significance was specified as α <.05.
Results
Preliminary and descriptive statistics
The flow of participants through the study is presented in Fig. 1. Of 82 BC patients assessed for eligibility, 66 (80%) met inclusion criteria, and 52 (79%) consented to participate and completed the baseline survey. Thus, the final analytic sample included 52 participants: 28 (54%) from the mainland USA and 24 (46%) from PR.
For a complete description of the sample, see Table 1. Participants’ mean age was 54 years (SD = 9 years) and 64% of participants were partnered. The majority identified as White (69%), and had at least a high school education (87%) and medical insurance (89%). The majority (69%) had been diagnosed with locoregional (Stage 0–III) BC and were an average of 6.7 years post-diagnosis (SD = 4.5 years). Regarding cultural characteristics, participants reported greater use of positive (M = 22.7) than negative religious coping (M = 9.5). The familismo mean score was 3.9 out of 5, indicating participants reported overall high levels of attitudinal familism.
Table 1.
Variables | Total (N = 52) | Mainland USA (n = 28) | PR (n = 24) | p |
---|---|---|---|---|
Sociodemographic characteristics | ||||
Age, M (SD) | 54.2 (8.8) | 54.9 (7.0) | 53.5 (10.7) | .57 |
Race, n (%) | .29 | |||
White | 36 (69) | 20 (71) | 16 (67) | |
Black or African American | 3 (6) | 0 (0) | 3 (13) | |
Multiple | 9 (17) | 5 (18) | 4 (17) | |
Other | 3 (6) | 2 (7) | 1 (4) | |
Partnered, n (%) | 33 (64) | 20 (71) | 13 (54) | .20 |
Education, n (%) | .53 | |||
<High school/GED | 7 (13) | 3 (11) | 4 (17) | |
≥High school/GED | 45 (87) | 25 (89) | 20 (83) | |
Employment status, n (%) | .93 | |||
Employed full-time | 16 (31) | 8 (29) | 8 (33) | |
Employed part-time | 9 (17) | 5 (18) | 4 (17) | |
Not employed | 27 (52) | 15 (54) | 12 (50) | |
Household income, n (%) | .17 | |||
<$35,000/year | 36 (69) | 18 (64) | 18 (75) | |
≥$35,000/year | 14 (27) | 10 (36) | 4 (17) | |
Insured, n (%) | 46 (89) | 23 (82) | 23 (96) | .12 |
Clinical characteristics | ||||
Years since BC diagnosis, M (SD) | 6.7 (4.5) | 7.8 (4.7) | 5.4 (4.1) | .06 |
Age at BC diagnosis, M (SD) | 46.3 (8.1) | 46.2 (5.9) | 46.3 (10.1) | .11 |
BC stage at diagnosis, n (%) | .25 | |||
Locoregional (0–III) | 36 (69) | 20 (71) | 16 (67) | |
Distant (IV) | 5 (10) | 4 (14) | 1 (4) | |
Unknown | 11 (21) | 4 (14) | 7 (29) | |
Cultural characteristics | ||||
Positive religious coping, M (SD) | 22.7 (6.0) | 23.0 (5.9) | 22.5 (6.1) | .99 |
Negative religious coping, M (SD) | 9.5 (4.9) | 10.1 (5.4) | 8.8 (4.3) | .25 |
Familismo, M (SD) | 3.9 (0.7) | 4.0 (0.8) | 3.8 (0.5) | .16 |
BC breast cancer; GED General Education Development; PR Puerto Rico.
There were no significant differences between mainland USA and PR participants in sociodemographic, clinical, or cultural characteristics (all p’s ≥ .06).
Frequency of GC discussion
The majority of participants (n = 33, 64%) had not discussed GC with a provider. More participants from PR reported provider discussion of GC (42% vs. 21% in the mainland USA). Of the 16 (31%) participants who discussed GC with a provider, most reported discussing GC with their medical oncologist (n = 10, 19%), followed by surgeon (n = 6, 12%), radiation oncologist (n = 2, 4%), primary care provider (n = 2, 4%), and nurse (n = 1, 2%). The provider typically initiated the discussion (n = 12, 75%) with only two participants (13%) reporting that they had initiated the discussion of GC themselves.
Factors associated with GC discussion
Results of the multivariable logistic regression model are presented in Table 2. The initial model included 13 predictor variables: residence (mainland USA vs. PR), age at baseline, race, partner status, education, employment status, income, insurance status, age at first BC diagnosis, stage of most recent BC diagnosis, positive religious coping, negative religious coping, and familismo. Of these, six predictor variables were selected in stepwise logistic regression. Two variables were positively associated with likelihood of GC discussion: PR (vs. mainland USA) residence (odds ratio [OR] = 6.00, 95% confidence interval [CI] = 1.15–31.37) and age at baseline (OR = 1.19, 95% CI = 1.00–1.41). Age at BC diagnosis (OR = 7.26, 95% CI = 3.48–15.15) was negatively associated with likelihood of GC discussion (OR = 0.80, 95% CI = 0.65–0.98). Three additional variables selected into the model had suggestive, but nonstatistically significant associations with likelihood of GC discussion: non-White race (OR = 0.21, 95% CI = 0.03–1.34), household income ≥$35,000/year (OR = 5.85, 95% CI = 0.98–34.88), and having medical insurance (OR = 13.71, 95% CI = 0.33–568.10). Partner status, education, employment status, positive religious coping, negative religious coping, and familismo were not significantly associated with likelihood of GC discussion (p > .2) and were not included in the final model. The model demonstrated adequate goodness-of-fit (χ2 = 7.20, p = .41).
Table 2.
Predictor variable | # reporting discussion of GC | Multivariable-adjusted OR | 95% CI | p |
---|---|---|---|---|
Residence | ||||
Mainland USA | 6 | (Ref.) | ||
PR | 10 | 6.00 | 1.15, 31.37 | .03* |
Age | – | 1.19 | 1.00, 1.41 | .04* |
Race | ||||
White | 12 | (Ref.) | ||
Non-White | 4 | 0.21 | 0.03, 1.34 | .10 |
Household income | ||||
<$35,000/year | 11 | (Ref.) | ||
≥$35,000/year | 5 | 5.85 | 0.98, 34.88 | .05 |
Insurance status | ||||
Uninsured | 1 | (Ref.) | ||
Insured | 15 | 13.71 | 0.33, 568.10 | .17 |
Age at BC diagnosis | – | 0.80 | 0.65, 0.98 | .03* |
BC breast cancer; CI confidence interval; GC genetic counseling; OR odds ratio; PR Puerto Rico.
*p < .05.
Discussion
Identification of pathogenic BRCA1/2 variants has significant implications for cancer prevention and control. GC/GT for high-risk BC survivors may inform risk management for secondary primary cancers, as well as cascade testing among at-risk relatives. Despite national guidelines that specify criteria for GC/GT referral for BC survivors [12], few Latina BC survivors receive GC/GT [16–18]. Healthcare provider recommendation is thought to play a critical role in uptake of GC/GT among Latina BC survivors [47]. Thus, identifying factors associated with discussion of GC with healthcare providers among high-risk Latina BC survivors has the potential to increase guideline-concordant care. The present study addressed this question by surveying high-risk Latina BC survivors in Tampa, FL and Ponce, PR. The multisite design enables us to compare the experiences of Latina BC survivors living inside and outside the mainland USA. Given PR’s status as a U.S. Territory and the related healthcare system challenges [31, 32], this study provides unique insight into system-level effects on GC discussion.
Despite eligibility for GC/GT, few survivors in this study had discussed GC with their healthcare providers. Given that study participants were an average of 6.7 years post-BC diagnosis, they should already have been recommended/referred to GC at the time of this study [36]. However, these data are in line with prior studies, in which 24%–70% of Spanish-speaking Latina BC survivors reported discussing GC with a healthcare provider [17, 18]. The rate of provider discussion was higher in PR (42%) than the mainland USA (21%). There are several possible explanations for this finding. First, Spanish is the predominant language in PR and virtually all healthcare providers in PR are Spanish-speaking. In comparison, a minority of physicians and genetic counselors in the mainland USA report fluency in Spanish [28, 48]. In language-discordant encounters, providers report more discomfort with the medical encounter [49] and Spanish-speaking patients report lower satisfaction with patient–provider communication [50]. Given that our sample consisted entirely of Spanish-preferring Latinas, our findings may reflect differences in availability of bilingual providers. Second, in this study, BC survivors from PR were diagnosed more recently than survivors from the mainland USA (M = 5.4 vs. 7.8 years). Given the increasing public awareness of genetics in recent years [51], provider discussion of GC may be becoming more common.
In multivariable analyses, two other patient sociodemographic characteristics were significantly associated with previous discussion of GC with a healthcare provider: age at baseline, and age at the time of BC diagnosis. First, patients who were older at the time of the baseline survey were more likely to have discussed GC with a provider. This was surprising, as previous research demonstrated a negative association between age and awareness/knowledge of GC/GT [52, 53]. In addition, one recent population-based survey (N = 2,029) found that older participants were less likely to discuss GC/GT with a healthcare provider [54]. Yet, this finding could be attributed to the inclusion criterion of GC/GT-naive BC survivors. It is possible that older Spanish-preferring Latina BC survivors are less likely to participate in GC after discussing it with their provider than younger survivors. This could have resulted in a study sample with a greater proportion of older BC survivors with previous provider discussion of GC.
Second, patients who were younger at the time of BC diagnosis were more likely to have discussed GC with a provider. These results are consistent with the prior literature [55]. Younger age of onset has been called a “hallmark” of hereditary cancer syndromes, and guidelines for GC/GT among BC survivors specifically state that early-onset BC (i.e., a diagnosis before the age of 45) indicates eligibility for GC/GT [12]. While healthcare providers demonstrate inconsistencies in identifying women at high risk for hereditary breast and ovarian cancer [56–58], an atypically early BC diagnosis may alert providers to the need for genetics services and prompt GC discussion. Unfortunately, this may mean that providers are overlooking women who may be eligible for GC/GT by virtue of other criteria, besides age at BC diagnosis. Educational interventions for healthcare providers and/or collaboration with genetics professionals may be needed to improve the identification of BC survivors who are eligible for GC/GT [59, 60].
Interestingly, the Latino cultural values examined in this study—religious coping and familismo—were not related to prior discussion of GC with a healthcare provider. Based on the prior literature highlighting the important roles of religious beliefs and familismo in decision-making about GC/GT [24–27], we expected that these cultural values might affect prior discussion of GC with a healthcare provider by motivating (or demotivating) patients to initiate such discussions. The lack of a relationship between these variables may reflect an actual lack of relationship between cultural values and previous discussion of GC with a healthcare provider, or may be a statistical artifact due to small sample size or measurement error. First, this study was powered for the primary outcomes of the RCT [34], and based on sample size recommendations for pilot studies [61–63]. Given that this study was not designed to examine cultural factors associated with GC discussion, we may be underpowered to detect such effects. Second, the relatively high level of familismo and positive religious coping reported by this sample may have resulted in a “ceiling effect” and limited our ability to observe relationships between cultural characteristics and our outcome of interest. Thus, additional studies with larger sample sizes, specifically designed to investigate the relationship between cultural variables and provider discussion of GC, are needed to confirm the findings presented here.
Clinical implications
Our results highlight a gap in the implementation of evidence-based genetics guidelines. Interventions are needed to close this gap and increase GC/GT among high-risk Latina BC survivors. Interventions that aim to increase GC/GT among high-risk Latina BC survivors have three possible targets: patients, providers, and systems.
At the patient level, interventions could provide education about genetics and BC risk and prepare patients for patient–provider communication about genetics [64]. In our sample, the vast majority of patients who had discussed GC with a healthcare provider reported that the provider initiated the discussion. Although Latina BC survivors report an interest in and desire for GC/GT [18, 47], they may lack the awareness and knowledge to initiate such discussions [19, 20, 22]. Psychoeducational interventions show promise in increasing awareness and knowledge of GC/GT in this population. For example, a narrative video tailored for Latinas increased knowledge, reduced negative attitudes, and increased intentions to participate in GC/GT [64]. Widespread dissemination of such interventions in the Latina community may lead to increased discussion of GC/GT between high-risk Latina women and their healthcare providers.
At the provider level, interventions could improve knowledge of GC/GT, address concerns about the appropriateness of GC/GT, and promote open and effective patient–provider communication about GC/GT [65, 66]. Providers may be reluctant to raise the topic of GC if they are unsure of the genetics resources subsequently available to their patients (particularly those who are Spanish-preferring). Provider education may be needed to reinforce the idea that some GT companies also include GC services (including Spanish-language GC services) and that GT for high-risk BC survivors is typically covered by medical insurance.
In the present study, we found that nonmodifiable factors (i.e., age at baseline, age at BC diagnosis) were associated with likelihood of GC discussion. Implementation of systems-level interventions may address these nonmodifiable factors. Specifically, universal assessment of hereditary BC risk factors would reduce provider reliance on “hallmark features” (e.g., early age of BC diagnosis) and ultimately improve the identification of women who meet national guideline recommendations for GC/GT [12].
Strengths and limitations
This study has several strengths. First, this study focuses on Latina BC survivors, who are historically underrepresented in the cancer survivorship literature [67]. Second, the multisite design enables us to compare the experiences of Latina BC survivors living inside and outside the mainland USA. Given PR’s status as a U.S. Territory and the related healthcare system challenges [31, 32], this study provides unique insight into system-level effects on GC discussion. Lastly, our findings have the potential to impact cancer care delivery by identifying gaps in the translation of evidence-based genetics guidelines into clinical practice. Results identify specific subpopulations of BC survivors at risk for substandard care, enabling the future development of targeted interventions to reduce disparities in GC/GT uptake.
Nonetheless, limitations must be acknowledged. First, simulation studies have demonstrated that multivariable models become unstable when there is a low ratio of events to predictors; as such, 5–15 events per variable are typically recommended to ensure the stability of logistic regression [68, 69]. Given the exploratory nature of this study, we tested a model with a low ratio of events to predictors (16:6). However, the low ratio of events to predictors may result in an unstable final model. Thus, the final model presented here needs to be replicated in future studies, with a higher ratio of events to predictors. Second, our sample size (N = 52) was based on sample size recommendations for pilot studies [61–63]. While this design enables researchers to assess whether an intervention produces a clinically significant signal (vs. a control condition), the small comparison groups (28 vs. 24) limit the generalizability of our results. Replication studies with larger sample sizes are needed to support the interpretation of the results presented here. Third, generalizability of study findings are also limited due to the sample characteristics (all Latina BC survivors, limited representation of Afro-Latinas) and the study’s inclusion criteria (i.e., GC/GT-naive participants). Almost all participants in this study identified as Colombian, Cuban, and Puerto Rican; this cohort may not represent the differing cultural and historical perspectives of other Latino groups. Similarly, participants in this study do not represent Spanish-preferring Latina BC survivors who have received GC/GT. Fourth, our convenience sample may be subject to selection bias; it is possible that participants who have previously discussed GC with a healthcare provider may be more likely to enroll in a study of GC/GT. Thus, our sample may not be representative of the broader population of high-risk Latina BC survivors. Fifth, assessments were conducted in person, via mail, or by telephone. Assessment modality was based on patient preference, and may have introduced bias into the study. We did not systematically collect assessment modality for each participant, and thus are unable to examine differences in our outcome of interest by assessment modality (e.g., in-person vs. via telephone). Sixth, data on provider discussion of GC were collected via self-report, and may be subject to patient recall bias. It is possible that providers discussed GC with patients, but patients do not remember the discussion. Future studies might incorporate more “objective” measures of GC discussion that reduce the risk for recall bias, such as EMR (electronic medical record) documentation, coding of audio-recorded clinical encounters, and/or self-report questionnaires administered immediately following clinical encounters. Seventh, in this study, we only assessed prior discussion of GC with a provider; we did not collect data on referral for GC/GT. It is possible that healthcare providers refer for GC/GT without a formal discussion with the patient. Referral may be necessary, but not sufficient, for ultimate uptake of GC/GT. Additional research is needed to identify the unique effects of provider discussion of GC, recommendation for GC, and referral for GC on patient uptake of GC/GT. Finally, we aimed to understand and highlight the sociodemographic, clinical, and cultural factors associated with GC discussion. However, there may be additional predictors of GC discussion (e.g., partner/family influence, care facility, quality of/satisfaction with care) not assessed in this study.
Conclusions
This study addresses key gaps in knowledge regarding the experiences of Latina BC survivors with GC/GT. These results highlight a gap in the implementation of evidence-based genetics guidelines, and have scientific, clinical, and policy implications. Our results suggest some unique characteristics that might impact discussion of GC with healthcare providers among Latina BC survivors; future research with larger samples is needed to validate the effects identified here. Future work should prospectively identify factors that will further enhance our current understanding, ultimately improving quality of care for Spanish-preferring Latina BC survivors. Ultimately, finding ways to engage high-risk BC survivors and their providers in GC discussions should occur in the context of a healthcare ecosystem where GC resources and high-risk cancer management clinics are broadly available.
Acknowledgments
The authors would like to thank Lindsay Fuzzell, PhD, for her careful review of this manuscript. This work was supported in part by the Biostatistics Core Facility at the H. Lee Moffitt Cancer Center & Research Institute, an National Cancer Institute (NCI)-designated Comprehensive Cancer Center (P30CA076292; PI: John Cleveland) and by grants from the NCI (U54CA163071, PIs: Jaime Matta and J. Dutil; U54CA163068, PIs: Kenneth Wright and Alvaro Monteiro; T32CA090314, PIs: Thomas Brandon and S.T. Vadaparampil).
Contributor Information
Claire C Conley, Georgetown Lombardi Comprehensive Cancer Center, Georgetown University, Washington, DC 20007, USA.
Jessica N Rivera Rivera, Moffitt Cancer Center, Tampa, FL 33612, USA.
Eida M Castro-Figueroa, Ponce Health Sciences University, Ponce 00716, Puerto Rico.
Laura Moreno, Moffitt Cancer Center, Tampa, FL 33612, USA.
Julie Dutil, Ponce Health Sciences University, Ponce 00716, Puerto Rico.
Jennifer D García, Moffitt Cancer Center, Tampa, FL 33612, USA.
Charité Ricker, University of Southern California Norris Comprehensive Cancer Center, Los Angeles, CA 90033, USA.
Gwendolyn P Quinn, Grossman School of Medicine, New York University, New York, NY 10016, USA.
Hatem Soliman, Moffitt Cancer Center, Tampa, FL 33612, USA.
Susan T Vadaparampil, Moffitt Cancer Center, Tampa, FL 33612, USA.
Compliance with Ethical Standards
Conflict of Interest: The authors declare that they have no conflicts of interest.
Authors’ Contributions: Claire Conley: conceptualization, data curation, formal analysis, visualization, writing–original draft preparation. Jessica Rivera Rivera: conceptualization, writing–original draft preparation. Eida Castro-Figueroa: conceptualization, funding acquisition, supervision, methodology, writing–review and editing. Laura Moreno: project administration, investigation, writing–review and editing. Julie Dutil: methodology, investigation, data curation, writing–review and editing. Jennifer García: project administration, investigation, writing–review and editing. Charité Ricker: conceptualization, resources, writing–review and editing. Gwendolyn Quinn: conceptualization, methodology, writing–review and editing. Hatem Soliman: conceptualization, resources, writing–review and editing. Susan Vadaparampil: conceptualization, funding acquisition, supervision, methodology, writing–review and editing.
Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All procedures were approved by the Moffitt Cancer Center (Institutional Review Board [IRB] #18601) and Ponce Health Sciences University–Ponce Research Institute (IRB #160607-EC) IRBs.
Informed Consent: Informed consent was obtained from all individual participants included in the study.
Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.
Data Transparency:
Study registration: This study was not formally registered.
Analytic plan registration: The analysis plan was not formally preregistered.
Availability of data: De-identified data from this study are not available in an a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.
Availability of analytic code: Analytic codes used to conduct the analyses presented in this study are not available in a public archive. They will be made available by emailing the corresponding author.
Availability of materials: Materials used to conduct the study are not publically available. They will be made available by emailing the corresponding author.
References
- 1. U.S. Cancer Statistics Working Group. Cancer statistics at a glance. 2021. Available at www.cdc.gov/cancer/dataviz. Accessibility verified August 30, 2021.
- 2. Martínez ME, Gomez SL, Tao L, et al. . Contribution of clinical and socioeconomic factors to differences in breast cancer subtype and mortality between Hispanic and non-Hispanic white women. Breast Cancer Res Treat. 2017;166(1):185–193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Chen L, Li CI.. Racial disparities in breast cancer diagnosis and treatment by hormone receptor and HER2 status. Cancer Epidemiol Biomarkers. 2015;24(11):1666–1672. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Serrano-Gomez SJ, Sanabria-Salas MC, Fejerman L.. Breast cancer health disparities in Hispanics/Latinas. Curr Breast Cancer Rep. 2020;12(3):175–184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Lynce F, Graves KD, Jandorf L, et al. . Genomic disparities in breast cancer among Latinas. Cancer Control. 2016;23(4):359–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Euhus DM, Robinson L.. Genetic predisposition syndromes and their management. Surg Clin. 2013;93(2):341–362. [DOI] [PubMed] [Google Scholar]
- 7. Ellsworth RE, Decewicz DJ, Shriver CD, Ellsworth DL.. Breast cancer in the personal genomics era. Curr Genomics. 2010;11(3):146–161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Kuchenbaecker KB, Hopper JL, Barnes DR, et al. . Risks of breast, ovarian, and contralateral breast cancer for BRCA1 and BRCA2 mutation carriers. J Am Med Assoc. 2017;317(23):2402–2416. [DOI] [PubMed] [Google Scholar]
- 9. Antoniou A, Pharoah PD, Narod S, et al. . Average risks of breast and ovarian cancer associated with BRCA1 or BRCA2 mutations detected in case series unselected for family history: a combined analysis of 22 studies. Am J Hum Genet. 2003;72(5):1117–1130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Chen S, Parmigiani G.. Meta-analysis of BRCA1 and BRCA2 penetrance. J Clin Oncol. 2007;25(11):1329–1333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Mavaddat, N, Peock, S, Frost, D, et al. . Cancer risks for BRCA1 and BRCA2 mutation carriers: results from prospective analysis of EMBRACE. J Natl Cancer Inst. 2013;105(11):812–822. [DOI] [PubMed] [Google Scholar]
- 12. Daly MB, Pilarski R, Yurgelun MB, et al. . NCCN guidelines insights: genetic/familial high-risk assessment: breast, ovarian, and pancreatic, version 1.2020: featured updates to the NCCN guidelines. J Natl Compr Canc Netw. 2020;18(4):380–391. [DOI] [PubMed] [Google Scholar]
- 13. Tuffaha HW, Mitchell A, Ward RL, et al. . Cost-effectiveness analysis of germ-line BRCA testing in women with breast cancer and cascade testing in family members of mutation carriers. Genet Med. 2018;20(9):985–994. [DOI] [PubMed] [Google Scholar]
- 14. Chapman-Davis E, Zhou ZN, Fields JC, et al. . Racial and ethnic disparities in genetic testing at a hereditary breast and ovarian cancer center. J Gen Intern Med. 2021;36(1):35–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. John EM, Miron A, Gong G, et al. . Prevalence of pathogenic BRCA1 mutation carriers in 5 US racial/ethnic groups. J Am Med Assoc. 2007;298(24):2869–2876. [DOI] [PubMed] [Google Scholar]
- 16. Levy DE, Byfield SD, Comstock CB, et al. . Underutilization of BRCA1/2 testing to guide breast cancer treatment: black and Hispanic women particularly at risk. Genet Med. 2011;13(4):349–355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Cragun D, Weidner A, Lewis C, et al. . Racial disparities in BRCA testing and cancer risk management across a population-based sample of young breast cancer survivors. Cancer. 2017;123(13):2497–2505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Jagsi R, Griffith KA, Kurian AW, et al. . Concerns about cancer risk and experiences with genetic testing in a diverse population of patients with breast cancer. J Clin Oncol. 2015;33(14):1584–1591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Wideroff L, Vadaparampil ST, Breen N, Croyle RT, Freedman AN.. Awareness of genetic testing for increased cancer risk in the year 2000 National Health Interview Survey. Public Health Genomics. 2003;6(3):147–156. [DOI] [PubMed] [Google Scholar]
- 20. Mai PL, Vadaparampil ST, Breen N, McNeel TS, Wideroff L, Graubard BI.. Awareness of cancer susceptibility genetic testing: the 2000, 2005, and 2010 National Health Interview Surveys. Am J Prev Med. 2014;46(5):440–448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. McCarthy AM, Bristol M, Fredricks T, et al. . Are physician recommendations for BRCA1/2 testing in patients with breast cancer appropriate? A population-based study. Cancer. 2013;119(20):3596–3603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Vadaparampil ST, McIntyre J, Quinn GP.. Awareness, perceptions, and provider recommendation related to genetic testing for hereditary breast cancer risk among at-risk Hispanic women: similarities and variations by sub-ethnicity. J Genet Couns. 2010;19(6):618–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Marin G, Marin BV.. Research With Hispanic Populations. Thousand Oaks, CA: Sage Publications, Inc.; 1991. [Google Scholar]
- 24. Hurtado-de-Mendoza A, Graves K, Gómez-Trillos S, et al. . Provider’s perceptions of barriers and facilitators for Latinas to participate in genetic cancer risk assessment for hereditary breast and ovarian cancer. Healthcare. 2018;6(3):116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Sussner KM, Edwards T, Villagra C, et al. . BRCA genetic counseling among at-risk Latinas in New York City: new beliefs shape new generation. J Genet Couns. 2015;24(1):134–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Rajpal N, Muñoz J, Peshkin BN, Graves KD.. Insights into BRCA1/2 genetic counseling from ethnically diverse Latina breast cancer survivors. J Genet Couns. 2017;26(6):1221–1237. [DOI] [PubMed] [Google Scholar]
- 27. Chalela P, Pagán JA, Su D, Muñoz E, Ramirez AG.. Breast cancer genetic testing awareness, attitudes and intentions of Latinas living along the US–Mexico border: a qualitative study. J Community Med Health Educ. 2012;2. Article no.: 152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Augusto B, Kasting ML, Couch FJ, Lindor NM, Vadaparampil ST.. Current approaches to cancer genetic counseling services for Spanish-speaking patients. J Immigr Minor Health. 2019;21(2):434–437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Gómez-Trillos S, Sheppard VB, Graves KD, et al. Latinas’ knowledge of and experiences with genetic cancer risk assessment: barriers and facilitators. J Genet Couns. 2020;29(4):505–517. [DOI] [PubMed] [Google Scholar]
- 30. Cruz-Correa M, Perez-Mayoral J, Dutil J, Echenique M, Mosquera R, Rivera-Roman K; Puerto Rico Clinical Cancer Genetics Consortia. Clinical cancer genetics disparities among Latinos. J Genet Couns. 2017;26(3):379–386. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Portela M, Sommers BD.. On the outskirts of national health reform: a comparative assessment of health insurance and access to care in Puerto Rico and the United States. Milbank Q. 2015;93(3):584–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Roman J. The Puerto Rico healthcare crisis. Ann Am Thorac Soc. 2015;12(12):1760–1763. [DOI] [PubMed] [Google Scholar]
- 33. Elliott MN, Haviland AM, Dembosky JW, Hambarsoomian K, Weech-Maldonado R.. Are there differences in the Medicare experiences of beneficiaries in Puerto Rico compared with those in the US mainland? Med Care. 2012;50(3):243–248. [DOI] [PubMed] [Google Scholar]
- 34. Conley CC, Castro-Figueroa EM, Moreno L, et al. . A pilot randomized trial of an educational intervention to increase genetic counseling and genetic testing among Latina breast cancer survivors. J Genet Couns. 2021;30(2):394–405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152(11):726–732. [DOI] [PubMed] [Google Scholar]
- 36. Daly MB, Pilarski R, Berry M, et al. NCCN guidelines insights: genetic/familial high-risk assessment: breast and ovarian, version 2.2017. J Natl Compr Canc Netw. 2017;15(1):9–20. [DOI] [PubMed] [Google Scholar]
- 37. Vadaparampil ST, Quinn GP, Small BJ, et al. . A pilot study of hereditary breast and ovarian knowledge among a multiethnic group of Hispanic women with a personal or family history of cancer. Genet Test Mol Biomarkers. 2010;14(1):99–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. August EM, Quinn GP, Perales R, et al. . Important considerations for recruiting women to cancer genetics studies in Puerto Rico. J Cancer Educ. 2012;27(1):105–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Gwede CK, Castro E, Brandon TH, et al. . Developing strategies for reducing cancer disparities via cross-institutional collaboration: outreach efforts for the partnership between the Ponce School of Medicine and the Moffitt Cancer Center. Health Promot Pract. 2012;13(6):807–815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Jiménez J, Ramos A, Ramos-Rivera FE, et al. . Community engagement for identifying cancer education needs in Puerto Rico. J Cancer Educ. 2018;33(1):12–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Barden SM, Gutierrez D, Gonzalez J, Ali S.. Healing faith: a qualitative exploration of Latina breast cancer survivors. Counsel Values. 2016;61(2):142–158. [Google Scholar]
- 42. Pargament KI, Smith BW, Koenig HG, Perez L.. Patterns of positive and negative religious coping with major life stressors. J Sci Study Relig. 1998;37(4):710–724. [Google Scholar]
- 43. Pargament K, Feuille M, Burdzy D.. The Brief RCOPE: current psychometric status of a short measure of religious coping. Religions. 2011;2(1):51–76. [Google Scholar]
- 44. Zinn MB. Familism among Chicanos: a theoretical review. Humboldt J Soc Relat. 1982;10(1):224–238. [Google Scholar]
- 45. Steidel AGL, Contreras JM.. A new familism scale for use with Latino populations. Hisp J Behav Sci. 2003;25(3):312–330. [Google Scholar]
- 46. Hosmer DW, Lemesbow S.. Goodness of fit tests for the multiple logistic regression model. Commun Stat-Theory Methods. 1980;9(10):1043–1069. [Google Scholar]
- 47. Vadaparampil ST, Quinn GP, Dutil J, et al. . A pilot study of knowledge and interest of genetic counseling and testing for hereditary breast and ovarian cancer syndrome among Puerto Rican women. J Community Genet. 2011;2(4):211–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Moreno G, Walker KO, Grumbach K.. Self-reported fluency in non-English languages among physicians practicing in California. Fam Med. 2010;42(6):414. [PMC free article] [PubMed] [Google Scholar]
- 49. Parsons JA, Baker NA, Smith-Gorvie T, Hudak PL.. To “get by” or “get help”? A qualitative study of physicians’ challenges and dilemmas when patients have limited English proficiency. BMJ Open. 2014;4(6):e004613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Flower KB, Skinner AC, Yin HS, et al. . Satisfaction with communication in primary care for Spanish-speaking and English-speaking parents. Acad Pediatr. 2017;17(4):416–423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. American Society of Human Genetics. Public attitudes toward genetics & genomics research literature and polling review report. 2020. Available at https://www.ashg.org/wp-content/uploads/2020/01/2020-Public-Views-Genetics-Literature-Review.pdf. Accessibility verified August 30, 2021.
- 52. Krakow M, Ratcliff CL, Hesse BW, Greenberg-Worisek AJ.. Assessing genetic literacy awareness and knowledge gaps in the US population: results from the Health Information National Trends Survey. Public Health Genomics. 2017;20(6):343–348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Ashida S, Goodman M, Pandya C, et al. Age differences in genetic knowledge, health literacy and causal beliefs for health conditions. Public Health Genomics. 2011;14(4–5):307–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Makhnoon S, Yu R, Cunningham SA, Peterson SK, Shete S.. Factors influencing discussion of cancer genetic testing with health-care providers in a population-based survey. Public Health Genomics. 2021;24(3–4): 160–170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Katz SJ, Ward KC, Hamilton AS, et al. . Gaps in receipt of clinically indicated genetic counseling after diagnosis of breast cancer. J Clin Oncol. 2018;36(12):1218–1224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Pal T, Cragun D, Lewis C, et al. . A statewide survey of practitioners to assess knowledge and clinical practices regarding hereditary breast and ovarian cancer. Genet Test Mol Biomarkers. 2013;17(5):367–375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Nair N, Bellcross C, Haddad L, et al. . Georgia primary care providers’ knowledge of hereditary breast and ovarian cancer syndrome. J Cancer Educ. 2017;32(1):119–124. [DOI] [PubMed] [Google Scholar]
- 58. Wideroff L, Vadaparampil ST, Greene MH, Taplin S, Olson L, Freedman AN.. Hereditary breast/ovarian and colorectal cancer genetics knowledge in a national sample of US physicians. J Med Genet. 2005;42(10):749–755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Eichmeyer JN, Burnham C, Sproat P, Tivis R, Beck TM.. The value of a genetic counselor: improving identification of cancer genetic counseling patients with chart review. J Genet Couns. 2014;23(3):323–329. [DOI] [PubMed] [Google Scholar]
- 60. Reid S, Cragun D, Tezak A, et al. . Disparities in BRCA counseling across providers in a diverse population of young breast cancer survivors. Genet Med. 2020;22(6):1088–1093. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 2008;31(2):180–191. [DOI] [PubMed] [Google Scholar]
- 62. Schoenfeld D. Statistical considerations for pilot studies. Int J Radiat Oncol. 1980;6(3):371–374. [DOI] [PubMed] [Google Scholar]
- 63. Cocks K, Torgerson DJ.. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201. [DOI] [PubMed] [Google Scholar]
- 64. Hurtado-de-Mendoza A, Graves KD, Gómez-Trillos S, et al. . Culturally targeted video improves psychosocial outcomes in Latina women at risk of hereditary breast and ovarian cancer. Int J Environ Res Public Health. 2019;16(23):4793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65. Mallen AR, Conley CC, Fuzzell L, et al. . “I think that a brief conversation from their provider can go a very long way”: patient and provider perspectives on barriers and facilitators of genetic testing after ovarian cancer. Support Care Cancer. 2021;29(5):2663–2677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. Hamilton JG, Abdiwahab E, Edwards HM, Fang M-L, Jdayani A, Breslau ES.. Primary care providers’ cancer genetic testing-related knowledge, attitudes, and communication behaviors: a systematic review and research agenda. J Gen Intern Med. 2017;32(3):315–324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67. Nápoles AM, Ortiz C, Santoyo-Olsson J, et al. . Post-treatment survivorship care needs of Spanish-speaking Latinas with breast cancer. J Community Support Oncol. 2017;15(1):20–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR.. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–1379. [DOI] [PubMed] [Google Scholar]
- 69. Vittinghoff E, McCulloch CE.. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165(6):710–718. [DOI] [PubMed] [Google Scholar]