Factors Predicting Bereaved Caregiver Perception of Quality of Care in the Final Week of Life: Implications for Health Care Providers

Abstract

Background: Cancer caregivers are key stakeholders in the final weeks of life and in bereavement. Research has highlighted end-of-life (EOL) factors important to caregivers, as well as factors contributing to caregiver mental health and bereavement outcomes. There has been limited data on factors predicting caregiver perceptions of quality of EOL care.

Objective: This study's purpose was to identify modifiable predictors of caregivers' Caregiver Evaluation of Quality of End of Life Care (CEQUEL) scores, with the broader aim of informing clinical interventions to improve caregiver impressions of care and subsequent bereavement adjustment.

Methods: Study data came from Coping with Cancer I (CwC1). CwC1 investigators interviewed advanced cancer patients and caregivers prior to the patient's death (Wave 1) and reinterviewed caregivers following the death (Wave 2) (N=275 dyads). The authors identified potential Wave 1 predictors of CEQUEL scores and performed a series of linear regression analyses to identify a parsimonious predictive model using corrected Akaike's Information Criterion (AICc) values.

Results: In adjusted analyses, caregivers rated quality of care as poorer when patients died in a hospital (B=−1.40, SE=0.40, p=0.001) (B, unstandardized regression coefficient; SE, standard error) or had less than one week of inpatient hospice care (B=−1.98, SE=−0.70, p=0.006). Whole-person physician care and caregiver religiosity were associated with perceived higher quality of care in unadjusted, but not adjusted, analyses.

Conclusions: Findings suggest that place of death and hospice length of stay best predict bereaved caregiver evaluations of quality of EOL care. These findings equip health care providers with modifiable targets to improve caregivers' experience of EOL care and subsequent bereavement.

Introduction

With the evolution of patient-focused, family-centered care at the end of life, caregivers—defined here as those family members or other loved ones providing the majority of a patient's unpaid, informal care—are recognized as playing a central role in the experience of dying patients.¹ Caregivers impact the health care team's ability to care for dying patients, and their mental health is also impacted by their perception of that care.^2–6 Many meet criteria for anxiety, depression, and other psychiatric disorders, but forego mental health support.^7–11 Caregivers in acute and critical care settings, particularly those opting for more aggressive end-of-life (EOL) measures, may be at particular risk for poor bereavement outcomes.^6,12–14

In order to improve caregiver well-being, it is critical to identify modifiable predictors of how they perceive quality of EOL care. Caregiver priorities for high-quality EOL care typically include shared decision making, communication about patient wishes, awareness of prognosis, preparation for death, and avoiding prolonged death or suffering.^1,15–18 The few existing instruments that measure quality of EOL care typically elicit the patient's experience rather than that of the caregiver.^19–23 Those that do assess caregivers, including the After-Death Bereaved Family Member Interview from the Toolkit of Instruments to Measure End-of-Life Care (TIME),²⁴ have omitted key factors of patient suffering and prolongation of death.^1,25 This omission is important, as memories of prolonged death or suffering may foster post-loss symptoms of depression and regret and impede successful grief resolution.^2,26,27

The present study expands on two of the authors' previously published reports in which they used Coping with Cancer I (CwC1) data from 275 patient/caregiver dyads to develop and validate the 13-item Caregiver Evaluation of Quality of End of Life Care (CEQUEL) scale. CEQUEL is a more comprehensive measure of caregiver-perceived quality of care in the final week of life that builds on the TIME instrument by including perceived suffering and prolonged dying.²⁸ The purpose of the present study was to identify modifiable predictors of caregiver CEQUEL scores, with the goal of developing clinical interventions to improve caregiver experiences at the end of life and in bereavement. Figure 1 illustrates the authors' conceptual model hypothesizing that caregiver-perceived quality of care (indicated by CEQUEL scores and including perceived suffering, prolonged dying, preparation, and shared decision making) is predicted by a combination of patient-, caregiver-, and ‘procedurally’ derived aspects of EOL care, while also acknowledging that unavoidable circumstances (e.g., refractory pain) may also contribute to perceptions of care.

FIG. 1.

Conceptual model of caregiver-perceived quality of care at the end of life (EOL, end of life; CG, caregiver).

Methods

The present study employed frequency and descriptive statistics, means-difference testing, correlational analyses, and uni- and multivariate regression analyses, all performed using SPSS 19.0 (IBM, Armonk, NY). Data were derived from CwC1, a federally funded, longitudinal, multisite study of psychosocial adjustment and EOL care preferences in advanced cancer patients and their caregivers. CwC1 recruited patients between 2002 and 2008 from eight participating sites in the Northeast United States (US) and Texas. Patients and caregivers were interviewed at baseline (Wave 1), and caregivers were interviewed again following patients' death (Wave 2). Caregivers completed interview questionnaires on their own (mean time=45 minutes). Completion rate for Wave 2 interviews was over 94%, with few reports of interview burden. Additional information was obtained via postmortem interviews with caregivers or providers caring for patients at the time of death. Of the 726 patients completing Wave 1 interviews, 414 had died at the time of data analysis, and 315 of their caregivers completed Wave 2 interviews. The present study used data for the 275 patient/caregiver dyads with complete caregiver CEQUEL data. Mean time from Wave 1 to death was 3.7 months, from death to Wave 2 was 6.3 months, and from death to postmortem was 1.5 months.

Outcome variable

The study's outcome variable was caregiver CEQUEL scores. Methods used for initial construction and validation of CEQUEL are detailed elsewhere.²⁹ Thirteen dichotomous items measuring caregiver evaluations of four factors (Prolongation of Death, Perceived Suffering, Shared Decision Making, and Preparation for the Death) were summed as a total CEQUEL score, with a possible range of 13 to 26 and with higher scores indicating perceived better quality of care (see Fig. 2). Internal consistency and reliability analyses were evaluated using Cronbach's α (α=0.69). CEQUEL demonstrated strong convergent validity, with higher CEQUEL scores correlating positively with hospice enrollment (z=−2.09; p≤0.05) and patient-physician therapeutic alliance (ρ=0.13; p≤0.05) and negatively with bereaved caregiver regret (ρ=−0.36, p≤.001) and trauma symptoms (z=−2.06; p≤0.05).

FIG. 2.

Caregiver Evaluation of Quality of End-of-Life Care (CEQUEL) scale (PT, point).

Predictor variables

The authors followed a theory-based analytical protocol established by Garrido and Prigerson²⁹ to identify the most parsimonious model of factors predicting CEQUEL scores. Potential predictors included items from Wave 1 patient and caregiver interviews and from postmortem interviews. Research specific to factors impacting perceived quality of care is scarce; the authors selected predictor variables based on the existing literature and/or their own clinical experience regarding factors impacting caregiver quality of life, mental health, bereavement, and experience of EOL care.^18,30,31 Because CEQUEL is derived from Wave 2 interviews, other Wave 2 variables were not considered in order to make a stronger statement of predictive value. The authors selected for regression analyses only those items potentially modifiable in the clinical context, dividing them into conceptual categories detailed in Figure 3.

FIG. 3.

Conceptual categories of potentially modifiable factors predicting CEQUEL scores. (Regression analyses were performed using variables first within and then across conceptual categories to identify the most parsimonious explanatory set of predictors fitting the authors' conceptual model.)

Patient predictor variables

Quality of life

Caregivers answered questions about their perception of patient QOL via the McGill Quality of Life questionnaire, a measure of physical, psychological, and existential factors widely used and validated within cancer populations (scale 0–10; 0=desirable and 10=undesirable).^32–34

Mental health

Patients completed a Structured Clinical Interview for DSM Disorders (SCID) Axis I modules for Major Depressive Disorder (MDD), Generalized Anxiety Disorder (GAD), Panic Disorder (PD), and Posttraumatic Stress Disorder (PTSD),³⁵ which were summed to indicate the presence of any mental health diagnosis. Patients also reported how often they felt deep inner peace or harmony.³⁶

Mental health service use

Patients answered whether or not they had discussed mental health concerns with a health care professional or accessed any mental health intervention since cancer diagnosis (both yes/no).

Acknowledgment of terminal illness

Patients described their health status as terminally ill or not terminally ill. Caregivers of patients acknowledging terminal illness are less likely to experience post-loss PTSD.^6,14,37

Religiosity, religious support, and religious coping

Patients reported frequency of religious service attendance (scale 1–6; 1=never, 6=more than once a week); time spent in private religious activities (scale 1–6; 1=never, 6=more than once a day); importance of religion (scale 1–3; 1=very important, 3=not important); use of religious coping (scale 1–5; 1=not at all, 5=most important); and whether religious views affected recent medical decisions; and these were summed as a measure of religiosity. Patients reported support of their religious beliefs by their religious community and the medical system (scale 1–5; 1=not at all, 5=completely); receipt of pastoral care in the hospital/clinic; and visits with outside clergy (both yes/no), which were summed as a measure of religious support. These measures have been associated with better patient QOL, increased hospice enrollment, and lower likelihood of ICU death.^38–39 Patient use of positive and/or negative religious coping was also assessed via the Brief RCOPE.^40,41

Therapeutic alliance

Therapeutic alliance between patients and their oncologists has been associated with dying patients' greater acceptance of terminal illness and decreased ICU time.⁴² Patient responses regarding how much they trusted and respected their physicians, felt respected and seen as a whole person by their physicians, and felt comfortable asking their physicians questions (scale 1–4; 1=not at all, 4=a great deal) were summed as a measure of therapeutic alliance. Comfort and whole-person care were also analyzed as individual predictors.

Advance care planning

Patients reported if they had completed a health care proxy and/or living will, if they had discussed EOL wishes with their doctor, and if they had completed a DNR order.

Caregiver predictor variables

Physical and emotional well-being

Caregivers completed the Medical Outcomes Study Short Form 36 Health Survey.⁴³

Health-promoting behavior

Caregiver concerns about staying in shape (yes/no); nutrition (scale 1–4; 1=not concerned, 4=very concerned) and calorie intake (scale 1–3; 1=very little, 3=very careful); and reports on exercise, taking vitamins, annual checkups, and getting enough rest (all yes/no) were summed as a measure of health-promoting behavior.

Physical health service use

Caregivers reported if they had overnight treatment in a hospital, treatment in an emergency room, or if they had visited a medical office or dentist in the last three months.

Mental health

Caregiver responses to SCID Axis I modules for MDD, GAD, PD, and PTSD were summed to capture any mental health diagnosis.

Mental health service use

Caregivers reported if they had discussed mental health concerns with a health care professional or accessed any mental health intervention since the patient's diagnosis (both yes/no).

Religiosity, religious support, and religious coping

Summary measures of caregiver religiosity and religious support were constructed parallel to those of patients. Caregiver religious coping was assessed via the Brief RCOPE. Negative religious coping has been associated with greater caregiver burden, poorer QOL, decreased satisfaction, and increased depression and anxiety.⁴⁴

Support system and family relationships

Caregivers completed the Interpersonal Support Evaluation List, a validated 16-item measure of perceived social support (e.g., “When I feel lonely, there are several people I can talk to”) (scale 1–4; 1=definitely true, 4=definitely false),⁴⁵ and the 12-item Family Relationships Index (e.g., “Family members really help and support one another”) (scale 1–4; 1=definitely true, 4=definitely false).⁴⁶

Caregiver burden

Caregivers completed the Caregiver Burden Scale, in which they were asked about time spent (“demand”) on six tasks (e.g., assistance with activities of daily living, medications, household tasks) as well as difficulty associated with each task (scale 1–4; 1=little or no, 4=a great deal). Task scores were summed to create separate measures of caregiver demand and difficulty. Caregivers also answered binary questions about whether caregiving felt positive and/or enriching and how well they were performing as caregivers.⁴⁷

Tobacco and alcohol use

Caregivers reported amount and frequency of cigarette and alcohol use.

End-of-life care predictor variables

Aggressive care in the final week of life

Caregiver or provider yes/no reports of ventilator use, resuscitation attempts, ICU stays, feeding tubes, chemotherapy, and antibiotics in the final week of life were summed as a measure of any aggressive care received.

Place of death

Place of death was recorded as acute care hospital (including ICU), home, nursing home, or inpatient hospice. Caregivers or providers reported if patients died where they wanted to die.

Hospice involvement

Inpatient and outpatient hospice involvement were analyzed both individually and together as a measure of any hospice involvement. Length of inpatient or outpatient hospice was also reported.

Consciousness in final week

Patient level of consciousness in the final week of life was reported.

Sociodemographic variables

Caregivers answered questions about their gender, age, race/ethnicity, marital status, income, education, religion, relationship to the patient, and length of time in the caregiving role. Caregiver recruitment site was also recorded.

While previous studies have set out, a priori, to test the predictive validity of specific variables while controlling for confounds (e.g., EOL discussions, hospice use), this study's analytical protocol allowed the authors to analyze a range of variables in order to derive the most parsimonious explanatory set fitting their conceptual model. In order to do so, the authors first constructed a correlation matrix within each conceptual category to identify highly correlated variables (i.e., correlations >0.40), then conducted a series of linear regression analyses first within and then across conceptual categories, using individual items and combinations of items with low correlations.

Models with the lowest corrected Akaike's Information Criterion (AICc) values were retained. Items were dropped from analysis based on high AICc values and not because of high inter-item correlations (e.g., caregiver religiosity and use of positive religious coping), though high correlations prohibited models combining these items.

After identifying a semifinal model, the authors reentered into analysis all variables that had been statistically significant but dropped from preliminary models, in order to maximize the chances of capturing the best predictive model. These included Wave 1 patient QOL, inner peace, religiosity, religious support, and DNR status; Wave 1 caregiver emotional well-being, religious support, social support, physical and mental health service use, and alcohol use; and postmortem reports of patient consciousness in the final week of life and whether patients died where they wanted to. The authors selected a final unadjusted model with the lowest AICc value and least number of predictors, and then adjusted this model for caregiver sociodemographics.

Results

Seventy-six percent of caregivers in this sample were female, 70% were white, 53% were the patient's spouse or partner, and 39% were Catholic. Caregivers ranged in age from 20 to 83 years (mean=51.9) (see Table 1). In bivariate analyses of sociodemographic characteristics, CEQUEL scores varied only by religious affiliation, with Catholic caregivers scoring lower than non-Catholics, and those without religious affiliation scoring lower than those with an affiliation (data not shown).

Table 1.

Sociodemographic Characteristics of Sample Caregivers (N=275^a)

Characteristic	Caregivers (N)	Years	Percentage
Sex
Male	64		24.0
Female	201		76.0
Age
Mean		51.9
SD		13.6
Race/ethnicity
White	185		70.0
Black	37		14.0
Asian-American, Pacific Islander, Indian	5		2.0
Hispanic	33		12.5
Other	4		1.5
Marital status
Married	172		68.0
Income
<$31,000	62		25.0
≥$31,000	123		50.0
Don't know	45		18.0
Declined	14		6.0
Education
Mean		13.5
SD		3.6
Religion
Catholic	102		39.0
Protestant	47		18.0
Baptist	36		14.0
Pentecostal	11		4.0
Jewish	13		5.0
Other	37		14.0
None	18		7.0
Relationship to patient
Spouse/partner	120		53.0
Son/daughter	57		25.0
Sibling	15		7.0
Other relative	17		7.0
Friend	6		2.0
Parent	11		5.0
Other	2		1.0
Recruitment site
Yale University Cancer Center	65		24.0
Veterans' Affairs Connecticut Comprehensive Cancer Clinics	13		5.0
Memorial Sloan-Kettering Cancer Center	18		6.5
Simmons Comprehensive Cancer Center	21		7.5
Parkland Hospital	89		33.0
Dana-Farber Cancer Institute	8		3.0
Massachusetts General Hospital	1		0.5
New Hampshire Oncology Hematology	56		20.5

SD, standard deviation.

^aAvailable Ns for each characteristic (due to missing data) were as follows: sex, age: N=265; race/ethnicity, education, religion: N=264; marital status: N=253; income: N=244; relationship to patient: N=228; recruitment site: N=27.

Regression analyses resulted in a final unadjusted model containing four predictor variables (see Table 2). CEQUEL scores were significantly higher when caregivers had greater religiosity (B=0.06, SE=0.02, p=0.004) and when the patient felt seen as a whole person by their physician (B=0.99, SE=0.48, p=0.04), and significantly lower when patients died in a hospital (versus all other locations) (B=−0.87, SE=0.33, p=0.01) or had an inpatient hospice length of stay (LOS) <1 week (versus one or more weeks, and not including patients with no inpatient hospice stay) (B=−2.12, SE=0.58, p=<0.001). After adjusting for caregiver sociodemographics and recruitment site, hospital death (B=−1.40, SE=0.40, p=0.001) and inpatient hospice LOS <1 week (B=−1.98, SE=−0.70, p=0.006) remained statistically significant (see Table 3).

Table 2.

Final Unadjusted Model (N=193)

Variable	B	SE B	β	p
Pt feels physician sees him/her as a whole person	0.99	0.48	0.14	0.040
Hospital (non-ICU) death	−0.87	0.33	−0.18	0.010
Inpatient hospice LOS<1 week	−2.12	0.58	−0.25	<0.001
Caregiver religiosity	0.06	0.02	0.20	0.004

B, unstandardized regression coefficient; β, standardized regression coefficient; ICU, intensive care unit; LOS, length of stay; p, significance level; pt, patient; R², coefficient of determination (proportion of variation explained by regression; SE, standard error.

Adjusted R²=0.12.

Table 3.

Final Adjusted Model Including Caregiver Sociodemographic Characteristics (N=135)

Variable	B	SE B	β	p
Pt feels physician sees him/her as a whole person	0.84	0.67	0.11	0.213
Hospital (non-ICU) death	−1.40	0.40	−0.29	0.001
Inpatient hospice LOS<1 week	−1.98	0.70	−0.23	0.006
Caregiver religiosity	0.05	0.03	0.15	0.114
Northeast recruitment site	−0.46	0.51	−0.11	0.374
Male	0.82	0.42	0.17	0.055
Income <$31,000 (compared to ≥$31,000)	0.52	0.43	0.12	0.234
Married	0.45	0.45	0.10	0.321
Patient has health care insurance	−0.18	0.55	−0.04	0.744
Race (reference=White)
Black	−0.42	0.58	−0.07	0.469
Asian or other	−1.18	1.22	−0.08	0.333
Hispanic	−0.92	0.69	−0.14	0.180
Religious affiliation (reference=Catholic)
Protestant	0.64	0.55	0.10	0.252
Jewish	−0.65	2.02	−0.03	0.747
Other religion	0.54	0.58	0.09	0.354
No religion	−1.70	0.78	−0.22	0.030
Pentecostal	1.86	0.92	0.18	0.047
Baptist	0.70	0.62	0.12	0.262
Length of time as primary caregiver	0.001	0.003	0.020	0.823
Relationship to patient: spouse (1=spouse, 0=all others)	−0.25	0.46	−0.06	0.584
Relationship to patient: child (1=child, 0=all others)	−0.66	0.56	−0.13	0.240
Age	0.01	0.02	−0.04	0.711
Education (years)	−0.001	0.060	−0.001	0.991

B, unstandardized regression coefficient; β, standardized regression coefficient; ICU, intensive care unit; LOS, length of stay; p, significance level; pt, patient; R², coefficient of determination (proportion of variation explained by regression; SE, standard error.

Adjusted R²=0.23

Discussion

This study's finding of hospital death as a predictor of perceived poor quality of care (which, in turn, is associated with poor bereavement outcomes) is consistent with previous research suggesting that hospital death, compared to home death, is associated with increased physical and emotional distress and worse QOL for patients, increased risk of prolonged grief disorder in caregivers, and worse caregiver-rated quality of death and dying.^14,48 A number of studies have further highlighted the detrimental impact of EOL decision making in the hospital on caregiver mental health.^5,49–52 Our findings related to hospice LOS are also consistent with existing research suggesting that shorter hospice LOS is associated with worse bereaved caregiver outcomes, and that ‘late-stage’ hospice admissions often feel chaotic, emotional, and like a crisis for cancer caregivers.^6,53–55

These findings are particularly relevant to health care professionals working in acute care settings. Recent U.S. data highlight a decrease in overall hospital deaths and an increase in hospice enrollment in the last month of life, tempered by an increase in ICU admission in the final month of life and hospice admission in the final three days of life.^56–57 The likelihood that more cancer patients are dying or at least spending time in ICUs is cause for concern, given that ICU care typically entails the most aggressive and costly medical interventions including prolonged life-sustaining treatments—care that has been shown to have a negative impact on caregivers at the end of life and in bereavement.⁶

Health care providers should think proactively about the best plan of care for dying patients and their caregivers, including measures to prevent unnecessary or unwanted hospital deaths and to foster earlier hospice enrollment for patients who desire it. Such measures might include standardized team-family meetings to clarify goals of care, development of triggers for palliative care consultation, adherence to established professional EOL care guidelines, proactive consultation of clinical social workers and chaplains, and promotion of generalist palliative care competencies across all disciplines.^58–62 For patients who cannot leave the hospital, teams should be identifying ways to make in-hospital care feel more peaceful and dignified (e.g., 24-hour family access, private hospital rooms) and developing policies to ensure high-quality EOL care in the acute care setting (e.g., protocols for ventilator withdrawal and palliative sedation).

Caregiver religiosity and patients feeling seen as a whole person, though excluded from the final adjusted model, merit further exploration for clinical intervention. Religion and spirituality are important to most advanced cancer patients.⁶³ A recurrent theme in our findings, religion may provide caregivers with a moral and theological framework, as well as a sense of community with God and fellow church members, from which to approach a loved one's death. Nurses, social workers, and physicians should not only continue to collaborate with chaplain colleagues, but also enhance their own ability to assess religious and spiritual needs. Patient-physician therapeutic alliance, including the feeling of being seen as a whole person, appears to play an important role for caregivers as well. Many caregivers will have been present for clinic visits, hospitalizations, and treatment discussions over the course of illness, and their observations of patient-physician dynamics during these moments will likely influence their perception of care. Social workers, chaplains, and bedside nurses may have a unique vantage point in identifying deficits in patient- or caregiver-physician relationships, and can coach patients and caregivers in communicating their needs to physicians.

This study's findings highlight several limitations as well as directions for future research. While CwC1 included a Veterans Affairs hospital and two community-based institutions in its recruitment sites, the study also included several academic medical centers that may have more inclination or resources to pursue aggressive interventions with advanced cancer patients, including trial participation.

Generalizability of findings may be further limited by culture variations between one U.S. region and the next, and between the United States and countries with different resources or health care policies. CwC1 data come from patients and caregivers active in the medical system and may miss factors unique to those who choose to forego or are otherwise unable to access traditional medical care.

CEQUEL's reliability and validity need to be confirmed in noncancer patient and caregiver samples. While the six-month period between CwC1 patient deaths and Wave 2 interviews was respectful of caregivers' immediate grieving period, future research will need to compare the reliability of bereaved caregiver reports taken closer to the time of death. While caregiver reporting of whether a patient died where they wanted to die was not a significant predictor of CEQUEL scores, it is important to note that these reports may not reflect patients' true wishes. Asking patients directly about their preferences would enhance future studies, both to assess for patient-caregiver agreement and to identify predictive influence on CEQUEL scores.

It is also important to acknowledge the conceptual overlap between patient and/or caregiver religiosity, religious support, and religious coping, as well as the potential for conflation between religiosity and spirituality.⁶⁴ Our measure of religious support does not capture broader spiritual support that patients may have received in the community or in the hospital, which also may influence CEQUEL scores. Finally, while the adjusted model explains almost a quarter of the variance in CEQUEL scores, a large proportion remains unexplained. Future studies should explore additional factors that may help to predict caregiver perceptions of EOL care.

Conclusion

The results of this study suggest that place of death, including dying in a hospital and short hospice LOS, best predicts bereaved caregiver evaluation of quality of EOL care. These factors, as well as whole-person care and caregiver religiosity, provide health care providers with key, modifiable targets to improve the caregiver's experience of care at the end of life and in bereavement.

Acknowledgments

Dr. Higgins would like to thank Dr. Kathleen McInnis-Dittrich, PhD; Dr. Barbara Berkman, PhD; and Dr. Melissa Kelley, PhD, for their contributions in conceptualization and analysis of this study.

Author Disclosure Statement

This research was supported in part by the following grants and other funding sources: Dr. Higgins is supported by DSW1009403SW from the American Cancer Society. Dr. Garrido is supported by CDA 11-201/CDP 12-255 from the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development Service. Dr. Prigerson was supported by MH63892 from the National Institute of Mental Health and CA106370 and CA156732 from the National Cancer Institute. These grants paid for the Coping with Cancer study on which these analyses are based.

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. government. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.