Abstract
Resilience is highly relevant in the context of cancer, and understanding how survivors adapt and potentially thrive following their diagnosis and treatment may provide insights into better supports and interventions to promote healthier survivorship. In this paper, we characterize two different ways to conceptualize and study resilience in cancer survivorship, as a trait and as a process. We focus specifically on the transition from active treatment to post-treatment survivorship. We present data from 225 cancer patients transitioning from active treatment (baseline assessment) to early survivorship (6-month follow-up). Results demonstrate that resilience assessed as a trait at baseline was unrelated to changes in survivors’ mental or physical wellbeing at follow-up, but did predict a decline in social satisfaction and spiritual wellbeing over time. However, when resilience is conceptualized as a dynamic process, the sample showed substantial resilience on multiple aspects of wellbeing. We suggest that different ways of conceptualizing resilience--as a trait versus as a dynamic process--may lead to very different conclusions and discuss future research directions for cancer survivors and for science of resilience.
Keywords: Social Functioning, Transitions, Spiritual Wellbeing, Quality of Life
Resilience in the Context of Cancer
The notion that resilience can be defined in diverse ways was well-captured by Zautra, Hall, and Murray (2010): “Is resilience best categorized as a process, an individual trait, a dynamic developmental process, an outcome, or all of the above?” (p. 4). Indeed, resilience has been conceptually defined in myriad ways across many different populations who have experienced a wide range of adverse events (Masten et al., 2021). Within the context of cancer, resilience is highly relevant; the diagnosis, treatment, and life disruption cancer brings has been likened to trauma (Arnaboldi et al., 2017; Eicher et al., 2015). In the context of adult cancer, resilience has been understudied but is important in understanding cancer patients’ and survivors’ physical and psychological well-being (Eicher et al., 2015). Resilience may relate to symptom appraisals, psychological wellbeing, and overall cancer experiences (Tian & Hong, 2014). While the vast majority of psycho-oncology research has focused on the costs and adverse consequences of cancer, a focus on resilience may be useful in promoting more positive psychosocial outcomes during both active treatment and post-treatment survivorship (Molina et al., 2014).
Studying Resilience in Adults Diagnosed with Cancer
Both cancer diagnosis and treatment can constitute a tremendous set of stressors for patients and their loved ones. Being diagnosed with cancer and going through treatment at any age is associated with distress that can lead to long-term negative psychological consequences (Seiler & Jenewein, 2019). At diagnosis, cancer patients are presented with the challenge of dealing with a specific traumatic life change to which they must adapt throughout their treatment. When primary treatment ends, they must adapt once again, to survivorship, a status they maintain throughout the rest of their lives (Seiler & Jenewein, 2019; Molina et al., 2014). Patients may have additional challenges when confronting their diagnosis, including competing life demands of family and career. Distress is a significant concern in the context of cancer as it is associated with reduced treatment compliance, increased mortality, and lower reported quality of life (QOL; Brandão et al., 2016; Conley et al., 2016; Pinquart et al., 2010). Cancer patients vary greatly in the degree of psychological distress they experience (Wang et al., 2012).
The timing of a cancer diagnosis can have significant meaning for individuals and can influence their levels of distress. Those diagnosed with cancer at younger ages may have more psychological difficulties, given the relative infrequency of cancer earlier in life. Findings from one study indicate that older age is strongly associated with less psychological distress in cancer patients (Matzka et al., 2016); similarly, a study of individuals with colorectal cancer found that older cancer patients experienced less psychological distress and adapted better than did younger patients (Cohen et al., 2014). Additionally, a systematic review of distress in female breast cancer survivors reported that being younger at diagnosis increased the risk of distress (Syrowatka et al., 2017) compared with older female survivors. However, some studies suggest that resilience in cancer survivors weakens with age due to the accumulation of physical and cognitive decline as well as reduced resources such as social support (MacLeod et al., 2016), while other studies suggest resilience increases with older age (Cohen et al., 2014; Matzka et al., 2016). Overall, the data on the relationship between older age and resilience remains conflicting (Seiler & Jenewein, 2019), and many studies have been cross-sectional in design, limiting conclusions that can be drawn (Cohen et al., 2014; Matzka et al., 2016). Therefore, additional research is needed to assess the relationship between age and resilience over time in survivors of cancer.
Resilience has been described as a characteristic that protects individuals from adverse psychosocial sequelae (Deshields et al., 2016). Perhaps most often, resilience is described as trait-like (e.g., “the ability to ‘bounce back’”; DeShields et al., 2016) or “an individual’s stress-coping ability” in response to adversity (Seiler & Jenewein, 2019). Accordingly, resilience is often assessed with trait-like measures. The most commonly used resilience measure is the Connor–Davidson Resilience Scale (CD-RISC; Connor, 2006). The original scale comprises 25 items rated on a 5-point Likert scale, with higher scores indicating higher levels of resilience. Sample items are, “I am able to adapt when changes occur,” “I can deal with whatever comes my way,” and “Having to cope with stress can make me stronger.” A more recent addition to trait-like resilience measures is the Response to Stressful Experiences Scale (RSES; Johnson et al., 2011), a self-report instrument created to complement existing measures of resilience by focusing on how individuals characteristically respond during and immediately after highly stressful events. More specifically, the RSES evaluates individual differences in cognitive, emotional, and behavioral responses to stressful life events. Sample items include, “take action to fix things,” “find a way to do what’s necessary to carry on,” and “find opportunity for growth.”
Other researchers disagree with conceptualizing resilience as trait-like, instead defining it as a process of responding to a specific stressful encounter. In this way of thinking, resilience refers to maintaining (or quickly returning to) a relatively stable level of healthy functioning following exposure to a specific stressor (e.g., Masten & Cicchetti, 2016). Some researchers have argued that resilience refers to “demonstrating stable healthy levels of wellbeing and the absence of negative outcomes during or following potentially harmful circumstances” as opposed to the ability to “bounce back” from adversity (Bonanno, 2005: see Infurna & Luthar, 2018). Others have argued that this definition of resilience (essentially no distress or decrement in functioning following a stressful encounter) may be too stringent and appears to lose the notion of “bouncing back” entirely (see Masten & Cicchetti, 2016). Researchers defining resilience as a process typically assess participants multiple times--ideally measuring functioning or well-being prior to the stressor and then multiple times afterwards (Infurna, 2020). In the present study, we aimed to compare the results of conceptualizing resilience as a trait and as a process that takes place over time.
In the context of dealing with cancer, much of the research has relied on trait-like measures and cross-sectional study designs (see Seiler and Jenewein, 2019, for a review). Numerous studies have demonstrated positive cross-sectional associations between cancer survivors’ self-reported trait resilience (e.g., as measured by the CD-RISC) and subjective aspects of wellbeing across a variety of cancer types (e.g., MacDonald et al., 2020; Markovitz et al., 2015; Matzka et al., 2016). Previous studies have found that cancer patients who rate themselves higher on resilience (using the CD-RISC) report lower psychological distress and higher levels of adjustment to their cancer (Dong et al., 2017; Markovitz et al., 2015). Additionally, higher self-reported resilience has been shown to be negatively associated with self-reported anxiety, depression, and emotional distress (Ye et al., 2017). Taken together, self-reported resilience seems to be consistently associated with concurrent psychological wellbeing and lower distress in cancer patients and survivors. However, none of the aforementioned studies followed up with patients to examine whether self-reported resilience actually predicted subsequent wellbeing over time. Thus, while cross-sectional studies provide preliminary support for the importance of studying resilience as a potential buffer of distress in cancer survivors, these study designs limit our ability to understand the dynamic process of resilience and its role in predicting wellbeing over time.
As in the general trauma literature, longitudinal studies of resilience in cancer survivors are needed to adequately examine whether self-reported resilience predicts individuals’ ability to bounce back following cancer diagnosis/treatment across dimensions of wellbeing (Infurna, 2020). Based on this conceptualization of resilience, individuals who are resilient will either demonstrate relatively high levels of wellbeing over time or experience recovery of wellbeing (i.e., bounce back) over time. While other studies have examined resilience with longitudinal study designs across a wide range of stress- and trauma-exposed populations (Cosco et al., 2017; Friedberg & Malefakis, 2018), studies using this method among cancer survivors are rare. There is some evidence that resilience in the first few months after a cancer diagnosis is similar to long-term resilience, as those who show low distress in the first few months after a cancer diagnosis are likely to maintain low distress levels six years later (Lam et al., 2012).
Transition from Active Treatment to Post-treatment Survivorship
Active/primary treatment can last several years and brings with it challenges that persist well beyond the point at which treatment concludes (Costanzo, Ryff, & Singer, 2009; Elliott et al., 2011; Santin, Mills, Treanor & Donnelly, 2012). Indeed, results of several large-scale studies indicate high levels of clinically significant symptoms of depression and anxiety following treatment (Champagne et al., 2016; Deimling, Kahana, Bowman, & Schaefer, 2002), with most survivors reporting being significantly concerned about recurrence (Baker, Denniston, Smith, & West, 2005). Further, a large, nationally representative study indicated that cancer survivors, relative to matched non-cancer controls, reported significantly lower levels of physical and mental health as well as greater role limitations and social functioning (Santin et al., 2012). Another large nationally-representative sample drawn from the MIDUS Study examined pre- to post-cancer-diagnosis changes in survivors and found that although cancer survivors largely experienced reductions in mental health following diagnosis, many demonstrated resilience in social well-being and personal growth, with levels comparable to matched controls (Costanzo et al., 2009).
In addition to these studies of people in survivorship, there is some evidence that the transition from active treatment to survivorship is a highly stressful and potentially critical juncture in determining their long term post-active treatment health and quality of life. During treatment, which can last many months, patients adjust to the frequent contact with their support team and often focus on their prognosis and survival. At the conclusion of primary treatment, patients’ frequent interactions with, and support from, their medical team abruptly ceases. It is not uncommon for cancer patients to experience decreased social and emotional support, face fears of cancer recurrence, and experience lingering or emerging adverse physical and psychological effects of diagnosis and related treatment (Stanton et al., 2015).
Survivors may also strive to make sense of their cancer diagnosis and its ultimate impact on their lives (Stanton, 2012). Yet despite the widespread dissemination of a seminal report from the Institute of Medicine, From Cancer Patient to Cancer Survivor: Lost in Transition (2006), attention to this transition has been surprisingly lacking. Following this publication, researchers and clinicians increased attention to survivorship care plans, but to date few observational studies have been conducted of the actual experiences of people in active treatment as they make this transition, sometimes termed “re-entry” to signify the dramatic shift back to their “normal” lives (Stanton et al., 2015). Indeed, although survivorship research has expanded exponentially, little of this work actually focuses on the point of the transition from active treatment to early survivorship.
Studies examining long-term wellbeing from the time of active treatment to the post-treatment survivorship period are limited. One study of breast cancer survivors in active treatment at baseline found significant decreases in depression and sleep problems and increases in meaning in life and improved cortisol cycles 14 months later (Hsiao et al., 2013). These findings suggest that survivors may improve on numerous clinical outcomes over this potentially high-stress transitional period.
Current Study: Comparing Methods of Testing Resilience in Transition to Survivorship
Although a great deal of research has studied resilience in cancer survivors using different methods, we were unable to find any literature directly comparing these methods in studying resilience to the transition to survivorship. Because both trait-like measures and repeated measures of wellbeing over time are commonly used to study resilience (Fletcher & Sarkar, 2013; Masten et al., 2021), determining how they compare in understanding the survivorship experience is important. Thus, we aimed to compare these two methods using data from a large, ongoing prospective study of resilience trajectories in individuals as they transition from cancer patient to cancer survivor.
The prospective design of this study provides an opportunity to examine whether trait resilience, captured by the RSES, described earlier, is related to resilience conceptualized as positive adaptation (i.e., improvement in well-being and functioning over time). Because there is no clear consensus in the existing body of literature about the conceptualization and measurement of resilience, the primary aim of the current study was to investigate the extent to which these two definitions of resilience and their corresponding methodological approaches produce similar findings. To do so, the current study collected self-reported measures of resilience and well-being from breast, prostate and colorectal cancer survivors while they were still receiving active treatment (baseline) and after they had moved into early cancer survivorship (6-month follow-up). To comprehensively capture wellbeing, we included measures of physical, mental, social and spiritual functioning. Our research questions were as follows:
(1) Does self-reported trait resilience improve from baseline to 6-month follow-up?
(2) How resilient were participants when resilience is defined as improving in QOL from baseline to follow-up?
(3) Does self-reported trait resilience predict improvements in QOL from baseline to follow-up?
(4) Is age related to resilience as defined by either of these conceptualizations?
Method
Participants
Individuals newly diagnosed with breast, prostate or colorectal cancer were identified by the Yale Cancer Center Rapid Case Ascertainment program and recruited from three Yale New Haven Health-affiliated hospitals. Following confirmation of eligibility, consent was obtained first from the attending physician and then from the patient. The current study presents data from an ongoing study consisting of five data collection time points over a 12-month period. At each timepoint, participants are asked to complete a self-report questionnaire containing measures of psychosocial resources, coping, and multiple domains of well-being. Due to the nature of the ongoing project, analyses currently presented include data only from the baseline and 6-month follow-up timepoints for 225 cancer survivors. Participants from the overall sample who were in watchful waiting for prostate cancer (N = 17) were also excluded from analyses due to the present study’s focus specifically on resilience as cancer patients transition from active treatment to early survivorship.
Procedures
Participants in the present study were recruited as part of a larger NIH-funded study of cancer survivors. Cancer patients were eligible for study participation if they had been to one of three Yale New Haven Health-affiliated hospitals for cancer care, had no history of cancer prior to this diagnosis, and were within six months of primary treatment conclusion. To document physician acknowledgement and support of the study, an email was sent to the physician introducing the study and notifying them of the physician permission procedures. We also requested an email acknowledgment response. After eligible patients were identified, a letter was sent to their physician asking permission to contact his/her patient(s). If we did not hear a refusal of participation from the physician within 14 days, the study team sent a letter to eligible patients explaining the study.
This letter was followed by a telephone call from a study coordinator. At the time of the telephone call, the study coordinator described the study and procedures and inquired about patient interest. Patients who verbally agreed to participate were asked to provide an email address to which their questionnaires could be sent. If patients did not provide an email address, they were sent the questionnaires via US Postal Service. Participants were required to electronically sign the informed consent form on the first page before moving onto the electronic survey. Study data were collected and managed using Research Electronic Data Capture (REDCap) electronic data capture tools hosted at University of Connecticut (Harris et al., 2009; Harris et al., 2019). REDCap is a secure, web-based software platform designed to support data capture for research studies (Harris et al., 2009; Harris et al., 2019).
Participants in the larger study completed assessments at five timepoints at three-month intervals. As data is still being collected for the larger study, all participants who had baseline (Time 1; during active treatment) and 6-month follow-up (Time 3) data available, and who were not in watchful waiting for prostate cancer, were included in this study (N = 225). Participants received a $50 gift card upon completion of each assessment.
Measures
Trait Resilience.
The Brief Response to Stressful Experiences Scale (RSES) was used to measure trait resilience (e.g., “During and after life’s most stressful events, I tend to…. find a way to do what’s necessary to carry on”). The brief RSES is a 4-item scale with response options ranging from 0 (not at all like me) to 4 (exactly like me). Previous research has shown the 4-item RSES to have strong convergent, discriminant, and concurrent validity in addition to strong internal consistency (De La Rosa, 2016). Cronbach’s alpha in the present study was .87 at Time 1 and .89 at Time 3.
Mental and physical quality of life.
Mental and physical quality of life were assessed via the Short Form Health Survey (SF-12; Ware, Kosinski, & Keller, 1996). Participants responded to twelve items querying about both physical (e.g., “During the past 4 weeks have you had any of the following problems with your work or other regular activities as a result of your physical health…accomplished less than you would like?”) and mental (e.g., “How much time during the past 4 weeks… have you felt downhearted and blue?”) quality of life with varying Likert response ranges. Items were summed, utilizing a weighting algorithm, to produce a physical and mental component score (PCS and MCS, respectively).
Satisfaction with Social Roles.
The PROMIS-Satisfaction with Social Roles and Activities (PROMIS-SSRA; 8-item short form) assesses the degree of satisfaction with performing one’s usual social roles and activities (e.g., “I am satisfied with my ability to participate in family activities”). Items are rated on a 5-point scale ranging from 1 (not at all) to 5 (very much). The scale yields a total sum score (ranging from 8 to 40), with higher scores indicating higher social satisfaction. Cronbach’s alpha in the current study was .97 at Time 1 and Time 3.
Spiritual Wellbeing.
The FACIT-Spiritual Questionnaire (FACIT-Sp; Peterman et al., 2002) is a 12-item scale that measures spiritual wellbeing as derived from faith, meaning and peace (e.g., “I find strength in my faith or spiritual beliefs;” “I feel a sense of purpose in my life;” “I feel peaceful”). Item response options range from 0 (not at all) to 4 (very much). A total score is generated by summing item responses, and higher scores indicate greater spiritual well-being. The FACIT-Sp has been shown to have good internal consistency and convergent validity (Peterman et al., 2002). In the present study, Cronbach’s alpha was .86 at Time 1 and .87 at Time 3.
Data Analysis
All analyses were conducted using SPSS Version 27. A p-value of .05 was selected a priori to establish statistical significance for all tests. Descriptive statistics for all study variables were examined. Of those who had completed baseline and had reached their 6-month follow-up timepoint, N = 27 (7.8%) of the total eligible sample dropped out prior to completing their 6-month follow-up. Remaining data for the variables of interest in the analytic sample were missing at 1% – 7%.
Change scores were generated for each of the four wellbeing variables (SF-12 MCS, SF-12 PCS, PROMIS-SSRA, and FACIT-Sp) to account for changes in constructs from baseline (T1) to 6-month follow-up (T3). Next, paired-sample t-tests were performed to assess changes in each variable from T1 to T3. A series of sequential ordinary least squares (OLS) regressions were run with age and gender entered in block 1 and change scores for the wellbeing measures entered in block 2, to examine baseline trait resilience as a predictor of change in aspects of wellbeing. Positive change scores indicate increases from T1 to T3. Finally, bivariate correlations were conducted to characterize the relationships between age and wellbeing variables at T1 and T3 and change scores.
Results
Participants (n= 225) were mostly female (71.1%) and averaged 58.9 years of age (SD = 11.5). Cancer types were breast (65.3%), colorectal (11.1%), or prostate (23.6%). Overall, participants were highly educated; two-thirds of the sample held at least a Bachelor’s degree. They also tended to be partnered (68.5%) and were working either full-time (42.2%) or part-time (11.1%) or retired (26.7%) at baseline. Participants were mostly White (88.4%, n = 199), with few Black (4.4%, n = 10), Asian (1.8%, n = 4) or mixed race (1.3%, n = 3) (see Table 1). We compared all baseline study variables for those who completed T3 versus those who had dropped out of the study prior to T3 (N = 27); no significant differences were noted between the groups in terms of gender or age, though they differed significantly by race (p < .001) as none of the dropped participants were White. The two groups did not demonstrate differences in mental health, physical health or satisfaction with social roles, but participants who dropped out prematurely reported lower spiritual wellbeing than did individuals included in the analytic sample (p = .024).
Table 1.
Demographic data by clinical cancer typology (N=225)
| Breast (n=147, 65.3%) | Prostate (n=53, 23.6%) | Colorectal (n=25, 11.1%) | Total Sample (N=225) | |
|---|---|---|---|---|
|
| ||||
| Gender | ||||
|
| ||||
| Female | 147 (100%) | 0 (0.0%) | 13 (52.0%) | 160 (71.1%) |
| Male | 0 (0.0%) | 53 (100%) | 12 (48.0%) | 66 (28.9%) |
|
| ||||
| Education level | ||||
|
| ||||
| No formal education | 1 (0.7%) | 0 (0.0%) | 0 (0.0%) | 1 (0.4%) |
| High school graduate | 13 (8.8%) | 5 (9.4%) | 8 (32.0%) | 26 (11.6%) |
| Some college or associates degree | 35 (23.8%) | 7 (13.2%) | 7 (28.0%) | 49 (21.8%) |
| Bachelor’s degree | 45 (30.6%) | 21 (39.6%) | 7 (28.0%) | 73 (32.4%) |
| Graduate or professional degree | 2 (1.4%) | 5 (9.4%) | 0 (0.0%) | 7 (3.1%) |
| Master’s degree | 40 (27.2%) | 9 (17.0%) | 3 (12.0%) | 52 (23.1%) |
| Doctoral degree or professional degree | 8 (5.4%) | 4 (7.5%) | 0 (0.0%) | 12 (5.3%) |
| Missing or prefer not to answer | 3 (2.0%) | 2 (3.8%) | 0 (0.0%) | 5 (2.2%) |
|
| ||||
| Ethnicity | ||||
|
| ||||
| Non-Hispanic/Latino | 119 (81.0%) | 44 (83.0%) | 17 (68.0%) | 180 (80.0%) |
| Hispanic/Latino | 12 (8.2%) | 1 (1.9%) | 2 (8.0%) | 15 (6.7%) |
| Missing/Not reported/Unknown | 16 (10.8%) | 8 (15.0%) | 6 (24.0%) | 30 (13.3%) |
|
| ||||
| Race | ||||
|
| ||||
| White | 127 (86.4%) | 51 (96.2%) | 21 (84.0%) | 199 (88.4%) |
| Black or African American | 8 (5.4%) | 0 (0.0%) | 2 (8.0%) | 10 (4.4%) |
| American Indian or Alaska Native | 1 (0.7%) | 0 (0.0%) | 0 (0.0%) | 1 (0.4%) |
| Asian | 2 (1.4%) | 2 (3.8%) | 0 (0.0%) | 4 (1.8%) |
| Native Hawaiian or other Pacific Islander | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
| Mixed race | 3 (3.1%) | 0 (0.0%) | 0 (0.0%) | 3 (1.3%) |
| Missing/Not reported/Unknown | 6 (4.1%) | 0 (0.0%) | 2 (8.0%) | 8 (3.6%) |
|
| ||||
| Marital status | ||||
|
| ||||
| Divorced | 21 (14.3%) | 4 (7.5%) | 4 (16.0%) | 29 (12.9%) |
| Domestic partnership | 5 (3.4%) | 0 (0.0%) | 1 (4.0%) | 6 (2.7%) |
| Married | 88 (59.9%) | 44 (83.0%) | 16 (64.0%) | 148 (65.8%) |
| Never married | 18 (12.2%) | 2 (3.8%) | 2 (8.0%) | 22 (9.8%) |
| Separated | 2 (1.4%) | 0 (0.0%) | 1 (4.0%) | 3 (1.3%) |
| Widowed | 10 (6.8%) | 2 (3.8%) | 0 (0.0%) | 12 (5.3%) |
| Missing/Not reported | 3 (3.0%) | 1 (1.9%) | 1 (4.0%) | 5 (2.2%) |
|
| ||||
| Employment status | ||||
|
| ||||
| Unemployed | 10 (6.8%) | 0 (0.0%) | 1 (4.0%) | 11 (4.9%) |
| Student | 3 (2.0%) | 0 (0.0%) | 0 (0.0%) | 3 (1.3%) |
| On medical leave | 11 (7.5%) | 0 (0.0%) | 2 (8.0%) | 13 (5.8%) |
| Retired | 34 (23.1%) | 24 (45.3%) | 2 (8.0%) | 60 (26.7%) |
| Full-time homemaker or family caregiver | 10 (6.8%) | 0 (0.0%) | 1 (4.0%) | 11 (4.9%) |
| Working part-time | 20 (13.6%) | 2 (3.8%) | 3 (12.0%) | 25 (11.1%) |
| Working full-time | 55 (37.4%) | 26 (49.1%) | 14 (56.0%) | 95 (42.2%) |
| Other | 2 (1.4%) | 1 (1.9%) | 2 (8.0%) | 5 (2.2%) |
| Missing/Not reported | 2 (1.4%) | 0 (0.0%) | 0 (0.0%) | 2 (0.9%) |
|
| ||||
| M (SD) Range | M (SD) Range | M (SD) Range | M (SD) Range | |
|
| ||||
| Age | 56.5 (12.2) 24 – 79 | 66.5 (6.7) 50 – 79 | 56.8 (7.9) 38 – 73 | 58.9 (11.5) 24 – 79 |
A paired-sample t-test on trait resilience at T1 and T3 addressed our first research question: Does self-reported trait resilience increase from baseline to 6-month follow-up? Trait resilience at T1 (M = 10.8, SD = 3.0) was significantly lower than trait resilience at T3 (M = 11.9, SD = 3.2), (t(206) = −5.45, p < .001) (see Table 2).
TABLE 2.
Paired samples t-tests comparing scores from Time 1 (pre) and Time 3 (post) in total sample
| Pre-Test | Post-Test | Paired Samples Differences | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N | Mean | Mean | Mean Difference | SD | t-Statistic | p-value | r | Effect Size 95% CI Lower | Effect Size 95% CI Upper | |
| FACIT-Sp | ||||||||||
| Spiritual Wellbeing | 206 | 33.74 | 33.66 | 0.09 | 5.63 | 0.22 | 0.82 | 0.81 | −0.12 | 0.15 |
| SF-12 | ||||||||||
| Mental Composite Score | 197 | 40.93 | 42.01 | −1.07 | 10.35 | −1.45 | 0.15 | 0.62 | −0.24 | 0.04 |
| Physical Composite Score | 197 | 43.42 | 43.42 | −1.56 | 5.90 | −3.71 | 0.00 | 0.50 | −0.41 | −0.12 |
| PROMIS-SSRA | 212 | 30.44 | 32.00 | −1.55 | 6.92 | −3.71 | 0.00 | 0.63 | −0.36 | −0.09 |
| Resilience | 207 | 10.78 | 11.93 | −1.14 | 3.02 | −5.45 | 0.00 | 0.52 | −0.52 | −0.24 |
Our second question was, how resilient were participants, defined as improving in wellbeing, from baseline to follow-up? Results of paired-sample t-tests on physical and mental QOL, satisfaction with social roles and activities, and spiritual wellbeing (Table 2) revealed significant improvements in physical QOL (t(196) = −3.71, p < .001) and satisfaction with social roles and activities (t(211) = −3.71, p = .001) from T1 to T3. However, no significant change was observed from T1 to T3 in mental QOL (t(196) = −1.45, p = .15) nor spiritual wellbeing (t(205) = .22, p = .82).
To answer our third research question, does self-reported resilience correspond with change in wellbeing from baseline to follow-up, trait resilience at T1 was regressed on each of the four wellbeing variable change scores separately to assess whether trait resilience at T1 predicted change in any of the wellbeing domains after controlling for age and gender. Trait resilience did not predict change in physical QOL (β = .02, p = .78) or mental QOL (β = −.10, p = .18). However, trait resilience significantly predicted satisfaction with social roles (β = −.17, p = .02) such that higher T1 resilience predicted a decrease in satisfaction with social roles from T1 to T3. In addition, trait resilience significantly predicted change in spiritual wellbeing, (β = −.20, p = .008), such that higher resilience was associated with a decrease in spiritual wellbeing from T1 to T3
Bivariate correlations addressed the final research question, whether age was associated with either of the resilience conceptualizations. Although age was positively associated with mental QOL at both time points (r = .31 at T1, p < .001; r = .27 at T3, p < .001), spiritual wellbeing at T1 only (r = .19, p < .01), and greater satisfaction with social roles and activity scores at both T1 and T3 (r = .15, p = .03 and r = .14, p = .04, respectively), age was not correlated with any of the change scores nor trait resilience at either timepoint.
Discussion
Research on resilience across disciplines has led to differing definitions in how the term is used and measured. It is not surprising that multiple definitions of resilience have been proposed, leading to debates regarding the way to operationally define resilience. Our study is the first, to our knowledge, to examine two different definitions of resilience in a sample of newly diagnosed adult cancer survivors transitioning from treatment to early survivorship. Our findings suggest the manner in which resilience is defined and measured should be carefully considered when interpreting study results. We found when examining resilience as a trait-like characteristic of the individual in the context of transitioning off cancer treatment, the construct was unrelated to how survivors actually recovered following the transition from active treatment to early survivorship on two of the four wellbeing measures, and, in fact, was related to declines in satisfaction with social roles and activities and spiritual wellbeing over the transition from active treatment to early survivorship. These findings may suggest that self-reported trait-like measures of resilience in the context of cancer (and perhaps in other areas as well) might be tapping into something very different from resilience, defined as the ability to bounce back. Trait-like definitions of resilience might be measuring constructs such as self-perceptions of strength or positive self-views rather than the ability to adapt and restore mental and physical health.
Very few studies have captured resilience data prior to and following a stressful period such as we have done here with cancer treatment. We do not have information regarding participants’ wellbeing prior to their diagnosis and are unable to address the pre- to post-diagnosis changes they may have experienced. We focus on a second stressful transition in the cancer continuum, the transition from active treatment to survivorship. This transition is also a time of elevated stress and anxiety (Champagne et al., 2016), but participants in our study improved on average in physical and social wellbeing. Given the challenges of cancer treatment and recovery, resilience may be a key aspect of optimizing wellbeing in cancer survivorship, and understanding factors that promote resilience is important in improving survivors’ physical and mental health. A necessary precursor to understanding facilitators and barriers to resilience is to accurately operationalize the construct. Moreover, considering resilience to be trait-like precludes abilities to learn resiliency; rather, it is a quality that people either have or do not have. However, if resilience is conceptualized as improvement in well-being, or bouncing back, following a stressor, then mutable targets of intervention are possible.
In designing our study, we featured resilience as reflected in their QOL, given that QOL has become a major focus in cancer care (Sosnowski et al., 2017). Our results indicate physical QOL and social functioning improved from baseline, during active treatment, to follow-up. However, no significant changes from baseline to follow-up were evident in mental QOL or spiritual wellbeing. The findings for mental QOL are not surprising as many longitudinal studies of mental health in cancer survivors find persistently good mental health over time, particularly in older adults, those married or partnered, and those with greater optimism and self-efficacy (Naughton & Weaver, 2014). However, changes in spiritual wellbeing are frequently reported following cancer treatment (e.g., Casellas‐Grau, Ochoa, & Ruini, 2017; Costanzo et al., 2009); we anticipated seeing improvements here as well. In fact, our participants were substantially lower in spiritual wellbeing than is typically found in samples of cancer patients and survivors (Munoz et al., 2015), a factor warranting additional investigation. In this study, the transition from active treatment to survivorship was a time of elevated physical and social stress, and while on average, our participants did fairly well across this transition, there were many who struggled. Understanding characteristics of these individuals who did not fare well can help target interventions to those who may be struggling.
The present study also aimed to examine if age was associated with resilience in cancer survivorship. Our findings indicate that age was not associated with trait resilience at either time point. Research on the relationship between resilience and age are mixed, as previously discussed, with some studies suggesting resilience decreases with age (MacLeod et al., 2016), while other studies suggest increases in resilience with age (Cohen et al., 2014). More research is needed to examine if age is associated with resilience in cancer survivors over time. The findings of the present study indicate that age was associated with higher mental quality of life at both time points. A systematic review on quality of life in cancer patients found that much of the research that has been conducted has not been age specific and has not considered the influence of age on perceptions towards quality of life (Shrestha et al., 2019). Quality of life may have a different meaning for cancer survivors older in age (Scotté et al., 2018), and previous research suggests that older cancer patients have reported higher quality of life compared to younger patients (Harden et al., 2009). Taking a life-span approach, gains and losses an individual experiences and appraises may be different depending on developmental life stage (Harden et al., 2009). As such, assessment of psychosocial needs and interventions to improve resilience in cancer survivors should consider patient characteristics such as age and developmental life stage.
These results have important implications for future research and interventions on cancer survivors and also for the larger field of resilience research. First, with regards to survivorship research, a growing body of evidence links resilience to improved physical and mental health outcomes, healthy lifestyle behaviors, and better adherence to survivorship follow-up care and surveillance (Seiler & Jenewein, 2019). Once we have a full complement of data, we will conduct additional analyses to identify distinct trajectories and underlying mechanisms of resilience/recovery (Bonnano, 2004) that have been identified in other stressful experiences (e.g., bereavement, natural disasters). This approach has seldom been applied in the context of cancer and has not yet been applied specifically to the active-treatment-to-post-treatment period. This often highly-stressful transition is a crucial point for determining long-term adjustment and great heterogeneity exists: some individuals achieve healthy post-cancer adjustment fairly readily while others experience high levels of distress and still others thrive over time. Currently, no comprehensive research explaining the underlying mechanisms and outcomes of resilience in this cancer transition has been conducted. Resilience trajectories provide a useful way of modeling individual variation in survivors’ post-treatment adjustment and may provide the foundation for targeted interventions.
In the larger field of resilience research, we need to determine what these self-report measures are assessing as well as to better understand the strengths and weaknesses of differing approaches. Most trait-like measures of resilience focus on the resilient personality and do not address the complexity of resilience. Many resilience theories suggest that resilience can be measured and understood at multiple levels that incorporate intra- and inter-personal factors, the environment, the lifecourse, and myriad biological influences. Thus, definitions of resilience need to be clearly articulated and operationally defined. Future research would clearly benefit from a multidisciplinary, multi-level, dynamic approach to studying resilience (Masten et al., 2021).
In conclusion, this study provides both a peek into resilience during a crucial time in survivorship (Institute of Medicine, 2006) and a view of how the operational definition of resilience can influence the conclusions of a given study. This work thus has relevance to both future research and clinical applications. We hope that these perspectives usefully inform subsequent studies of resilience and clinical interventions to promote wellbeing in survivorship.
Acknowledgments
Funded by NCI: GRANT UH3CA220642
References
- Arnaboldi P, Riva S, Crico C, & Pravettoni G. (2017). A systematic literature review exploring the prevalence of post-traumatic stress disorder and the role played by stress and traumatic stress in breast cancer diagnosis and trajectory. Breast Cancer: Targets and Therapy, 9, 473. 10.2147/bctt.s111101 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baker F, Denniston M, Smith T. & West MM (2005). Adult cancer survivors: How are they faring? Cancer, 104, 2565–2576. 10.1002/cncr.21488 [DOI] [PubMed] [Google Scholar]
- Bonanno GA (2005). Resilience in the face of potential trauma. Current Directions in Psychological Science, 14(3), 135–138. 10.1111/j.0963-7214.2005.00347.x [DOI] [Google Scholar]
- Brandão T, Tavares R, Schulz MS, & Matos PM (2016). Measuring emotion regulation and emotional expression in breast cancer patients: A systematic review. Clinical psychology review, 43, 114–127. 10.1016/j.cpr.2015.10.002 [DOI] [PubMed] [Google Scholar]
- Burgess C, Cornelius V, Love S, Graham J, Richards M, & Ramirez A. (2005). Depression andanxiety in women with early breast cancer: Five year observational cohort study. BMJ, 330:702. 10.1136/bmj.38343.670868.D3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Champagne AL, Brunault P, Huguet G, Suzanne I, Senon JL, Body G, ... & Camus V. (2016). Personality disorders, but not cancer severity or treatment type, are risk factors for later generalized anxiety disorder and major depressive disorder in non metastatic breast cancer patients. Psychiatry Research, 236, 64–70. 10.1016/j.psychres.2015.12.032 [DOI] [PubMed] [Google Scholar]
- Cohen M, Baziliansky S, & Beny A. (2014). The association of resilience and age in individuals with colorectal cancer: an exploratory cross-sectional study. Journal of Geriatric Oncology, 5(1), 33–39. 10.1016/j.jgo.2013.07.009 [DOI] [PubMed] [Google Scholar]
- Connor KM (2006). Assessment of resilience in the aftermath of trauma. Journal of Clinical Psychiatry, 67(2), 46–49. [PubMed] [Google Scholar]
- Conley CC, Bishop BT, & Andersen BL (2016). Emotions and emotion regulation in breast cancer survivorship. Healthcare, 4(3), 56. 10.3390/healthcare4030056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Connor KM, & Davidson JRT (2003). Development of a new resilience scale: The Connor–Davidson Resilience Scale (CD–RISC). Depression and Anxiety, 18(2), 76–82. 10.1002/da.10113 [DOI] [PubMed] [Google Scholar]
- Cosco TD, Kaushal A, Hardy R, Richards M, Kuh D, & Stafford M. (2017). Operationalising resilience in longitudinal studies: a systematic review of methodological approaches. Journal of Epidemiology and Community Health, 71(1), 98–104. 10.1136/jech-2015-206980 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costanzo ES, Ryff CD, & Singer BH (2009). Psychosocial adjustment among cancer survivors: Findings from a national survey of health and well-being. Health Psychology, 28, 147–156. 10.1037/a0013221 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Deimling G, Kahana B, Bowman K, & Schaefer M. (2002). Cancer survivorship and psychological distress in later life. Psycho-Oncology, 11, 479–494. 10.1002/pon.614 [DOI] [PubMed] [Google Scholar]
- Deshields TL, Heiland MF, Kracen AC, & Dua P. (2016). Resilience in adults with cancer: Development of a conceptual model. Psycho‐Oncology, 25(1), 11–18. 10.1002/pon.3800 [DOI] [PubMed] [Google Scholar]
- Dong X, Li G, Liu C, Kong L, Fang Y, Kang X, & Li P. (2017). The mediating role of resilience in the relationship between social support and posttraumatic growth among colorectal cancer survivors with permanent intestinal ostomies: A structural equation model analysis. European Journal of Oncology Nursing, 29, 47–52. 10.1016/j.ejon.2017.04.007 [DOI] [PubMed] [Google Scholar]
- Dubey C, De Maria J, Hoeppli C, Betticher DC, & Eicher M. (2015). Resilience and unmet supportive care needs in patients with cancer during early treatment: A descriptive study. European Journal of Oncology Nursing, 19(5), 582–588. 10.1016/j.ejon.2015.03.004 [DOI] [PubMed] [Google Scholar]
- Eicher M, Matzka M, Dubey C, & White K. (2015). Resilience in Adult Cancer Care: An Integrative Literature Review. Oncology Nursing Forum, 42(1), E3–E16. 10.1188/15.ONF.E3-E16 [DOI] [PubMed] [Google Scholar]
- Elliott J, Fallows A, Staetsky L, Smith PWF, Foster CL, Maher E, & Corner J. (2011). The health and well-being of cancer survivors in the UK: Findings from a population-based survey. British Journal of Cancer, 105, 11–20. 10.1038/bjc.2011.418 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Friedberg A, & Malefakis D. (2018). Resilience, trauma, and coping. Psychodynamic Psychiatry, 46(1), 81–113. 10.1521/pdps.2018.46.1.81 [DOI] [PubMed] [Google Scholar]
- Fletcher D, & Sarkar M. (2013). Psychological resilience. European Psychologist, 18, 12–23. 10.1027/1016-9040/a000124 [DOI] [Google Scholar]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, McLeod L, Delacqua F. Kirby J, & Duda SN (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208. 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hsiao F-H, Chang K-J, Kuo W-H, Huang C-S, Liu Y-F, Lai Y-M, Jow G-M, Ho RTH, Ng S-M, & Chan CLW (2013). A longitudinal study of cortisol responses, sleep problems, and psychological well-being as the predictors of changes in depressive symptoms among breast cancer survivors. Psychoneuroendocrinology, 38(3), 356–366. 10.1016/j.psyneuen.2012.06.010 [DOI] [PubMed] [Google Scholar]
- Infurna FJ (2020). What does resilience signify? An evaluation of concepts and directions for future research. Gerontology, 66 (4), 323–331. 10.1159/000507365 [DOI] [PubMed] [Google Scholar]
- Infurna FJ, & Luthar SS (2018). Re-evaluating the notion that resilience is commonplace: A review and distillation of directions for future research, practice, and policy. Clinical Psychology Review, 65, 43–56. 10.1016/j.cpr.2018.07.003 [DOI] [PubMed] [Google Scholar]
- Infurna FJ, & Luthar SS (2016). Resilience to major life stressors is not as common as thought. Perspectives on Psychological Science, 11(2), 175–194. 10.1177/1745691615621271 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Institute of Medicine (2006). From cancer patient to cancer survivor: Lost in transition. Washington, DC: National Academies Press. [Google Scholar]
- Johnson DC, Polusny MA, Erbes CR, King D, King L, Litz BT, Schnurr PP, Friedman M, Pietrzak RH, & Southwick SM (2011). Development and initial validation of the Response to Stressful Experiences Scale. Military Medicine, 176(2), 161–169. 10.7205/MILMED-D-10-00258 [DOI] [PubMed] [Google Scholar]
- Lam WW, Shing YT, Bonanno GA, Mancini AD, & Fielding R. (2012). Distress trajectories at the first year diagnosis of breast cancer in relation to 6 years survivorship. Psycho‐Oncology, 21(1), 90–99. 10.1002/pon.1876 [DOI] [PubMed] [Google Scholar]
- MacDonald C, Theurer JA, Fung K, Yoo J, & Doyle PC (2020). Resilience: An essential element in head and neck cancer survivorship and quality of life. Supportive Care in Cancer, 1–9. 10.1007/s00520-020-05873-4 [DOI] [PubMed] [Google Scholar]
- MacLeod S, Musich S, Hawkins K, Alsgaard K, & Wicker ER (2016). The impact of resilience among older adults. Geriatric Nursing, 37(4), 266–272. 10.1016/j.gerinurse.2016.02.014 [DOI] [PubMed] [Google Scholar]
- Markovitz SE, Schrooten W, Arntz A, & Peters ML (2015). Resilience as a predictor for emotional response to the diagnosis and surgery in breast cancer patients. Psycho-Oncology, 24(12), 1639–1645. 10.1002/pon.3834 [DOI] [PubMed] [Google Scholar]
- Masten AS, Lucke CM, Nelson KM, & Stallworthy IC (2021). Resilience in development and psychopathology: Multisystem perspectives. Annual Review of Clinical Psychology, 17, 521–549. 10.1146/annurev-clinpsy-081219-120307 [DOI] [PubMed] [Google Scholar]
- Masten AS, & Cicchetti D. (2016). Resilience in development: Progress and transformation. In Cicchetti D. (Ed.), Developmental psychopathology (3rd ed., Vol. 4, pp. 271–333). New York, NY: Wiley. 10.1002/9781119125556.devpsy406 [DOI] [Google Scholar]
- Matzka M, Mayer H, Köck-Hódi S, Moses-Passini C, Dubey C, Jahn P, Schneeweiss S, & Eicher M. (2016). Relationship between resilience, psychological distress and physical activity in cancer patients: A cross-sectional observation study. PLoS One. 11(4):e0154496. 10.1371/journal.pone.0154496 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Molina Y, Yi JC, Martinez-Gutierrez J, Reding KW, Yi-Frazier JP, & Rosenberg AR (2014). Resilience among patients across the cancer continuum: diverse perspectives. Clinical Journal of Oncology Nursing, 18(1), 93–101. 10.1188/14.CJON.93-101 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Munoz AR, Salsman JM, Stein KD, & Cella D. (2015). Reference values of the Functional Assessment of Chronic Illness Therapy‐Spiritual Well‐Being: A report from the American Cancer Society’s studies of cancer survivors. Cancer, 121(11), 1838–1844. 10.1002/cncr.29286 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Naughton MJ, & Weaver KE (2014). Physical and mental health among cancer survivors: Considerations for long-term care and quality of life. North Carolina Medical Journal, 75(4), 283–286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Panjwani AA, Marín-Chollom AM, Pervil IZ, Erblich J, Rubin LR, Schuster MW, & Revenson TA (2019). Illness uncertainties tied to developmental tasks among young adult survivors of hematologic cancers. Journal of Adolescent and Young Adult Oncology, 8(2), 149–156. 10.1089/jayao.2018.0024 [DOI] [PubMed] [Google Scholar]
- Peterman AH, Fitchett G, Brady MJ, Hernandez L, & Cella D. (2002). Measuring spiritual well-being in people with cancer: The functional assessment of chronic illness therapy—Spiritual Well-being Scale (FACIT-Sp). Annals of Behavioral Medicine, 24(1), 49–58. 10.1207/S15324796ABM2401_06 [DOI] [PubMed] [Google Scholar]
- Pinquart M, & Duberstein PR (2010). Depression and cancer mortality: A meta-analysis. Psychological Medicine, 40(11), 1797–1810. 10.1017/S0033291709992285 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Santin O, Mills M, Treanor C, & Donnelly M. (2012). A comparative analysis of the health and wellbeing of cancer survivors to the general population. Supportive Care in Cancer, 20, 2545–2552. 10.1007/s00520-011-1372-9 [DOI] [PubMed] [Google Scholar]
- Scotté F, Bossi P, Carola E, Cudennec T, Dielenseger P, Gomes F, Knox S, & Strasser F. (2018). Addressing the quality of life needs of older patients with cancer: a SIOG consensus paper and practical guide. Annals of Oncology : Official Journal of the European Society for Medical Oncology, 29(8), 1718–1726. 10.1093/annonc/mdy228 [DOI] [PubMed] [Google Scholar]
- Shrestha A, Martin C, Burton M, Walters S, Collins K, & Wyld L. (2019). Quality of life versus length of life considerations in cancer patients: A systematic literature review. Psycho- oncology, 28(7), 1367–1380. 10.1002/pon.5054 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seiler A, & Jenewein J. (2019). Resilience in cancer patients. Frontiers in Psychiatry, 10, 208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stanton AL, Ganz PA, Rowland JH, Meyerowitz BE, Krupnick JL, & Sears SR (2005). Promoting adjustment after treatment for cancer. Cancer: Interdisciplinary International Journal of the American Cancer Society, 104(S11), 2608–2613. 10.1002/cncr.21246 [DOI] [PubMed] [Google Scholar]
- Stanton AL (2012). What happens now? Psychosocial care for cancer survivors after medical treatment completion. Journal of Clinical Oncology, 30, 1215–1220. 10.1200/jco.2011.39.7406 [DOI] [PubMed] [Google Scholar]
- Stanton AL, Rowland JH, & Ganz PA (2015). Life after diagnosis and treatment of cancer inadulthood: Contributions from psychosocial oncology research. American Psychologist, 70,159–174. 10.1037/a0037875 [DOI] [PubMed] [Google Scholar]
- Sosnowski R, Kulpa M, Ziętalewicz U, Wolski JK, Nowakowski R, Bakuła R, & Demkow T. (2017). Basic issues concerning health-related quality of life. Central European Journal of Urology, 70(2), 206–211. 10.5173/ceju.2017.923 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tian J, & Hong JS (2014). Assessment of the relationship between resilience and quality of life in patients with digestive cancer. World Journal of Gastroenterology, 20(48), 18439–18444. 10.3748/wjg.v20.i48.18439 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang X, Wang SS, Peng RJ, Qin T, Shi YX, Teng XY, Liu DG, Chen WQ, & Yuan ZY (2012). Interaction of coping styles and psychological stress on anxious and depressive symptoms in Chinese breast cancer patients. Asian Pacific Journal of Cancer Prevention, 13(4), 1645–1649. 10.7314/apjcp.2012.13.4.1645 [DOI] [PubMed] [Google Scholar]
- Ye ZJ, Qiu HZ, Li PF, Liang MZ, Zhu YF, Zeng Z, Hu GY, Wang SN, & Quan XM (2017). Predicting changes in quality of life and emotional distress in Chinese patients with lung, gastric, and colon-rectal cancer diagnoses: the role of psychological resilience. Psycho-Oncology. 26(6), 829–835. 10.1002/pon.4237 [DOI] [PubMed] [Google Scholar]
- Zautra AJ, Hall JS, & Murray KE (2010). Resilience: a new definition of health for people and communities. In Reich JW, Zautra AJ, & Hall JS (Eds.), Handbook of Adult Resilience: Concepts, Methods, and Applications (pp. 3–29). John Wiley. [Google Scholar]
