Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: Palliat Support Care. 2016 Aug 12;15(1):57–66. doi: 10.1017/S1478951516000626

The Development of the Bereavement Risk Inventory and Screening Questionnaire (BRISQ): Item Generation and Expert Panel Feedback

Kailey Roberts 1,2, Jimmie Holland 1,3, Holly G Prigerson 3, Corinne Sweeney 1, Geoffrey Corner 4, William Breitbart 1,3, Wendy G Lichtenthal 1,3
PMCID: PMC5296244  NIHMSID: NIHMS829877  PMID: 27516152

Abstract

Objectives

Following the loss of a loved one to cancer, a significant subset of bereaved family members are at heightened risk for mental and physical health problems; however, often these family members “fall through the cracks” in the healthcare system. A brief, clinically useful self-report bereavement risk screening tool could facilitate more effective identification of family members in need of psychosocial support before and after a cancer loss. Thus, the purpose of this study was to develop and refine the Bereavement Risk Inventory and Screening Questionnaire (BRISQ), a self-report bereavement screening tool, and to assess its utility using feedback from bereavement experts.

Methods

Quantitative and qualitative feedback from a panel of 15 clinical and research experts in bereavement was obtained through an online survey to identify the most clinically useful items and understand expert opinion on bereavement screening.

Results

The qualitative and quantitative feedback was synthesized, resulting in a 22% reduction of the item pool. While there was a general consensus between experts on the most clinically useful risk factors for bereavement-related mental health challenges and on the utility of screening, they also offered feedback on language and formatting that guided substantial revisions to the BRISQ.

Significance of Results

These findings were used to refine the BRISQ in preparation for a second study to obtain family member feedback on the measure. By incorporating both expert and family member feedback, the intention is to create a screening tool that represents top clinical and research knowledge in bereavement in a way that effectively addresses barriers to care.

Keywords: bereavement, grief, high-risk, screening, family-centered care

Attention to end-of-life care for both patients diagnosed with cancer and their families has gained an increasingly important place in medical care policy recommendations (Qaseem et al., 2008; Institute of Medicine, 2014). Yet, addressing the psychosocial needs and potential mental health struggles of family members at end-of-life and throughout bereavement can prove challenging. For example, psychosocial resources for family members may be limited and follow-up may be offered inconsistently (Downar et al., 2014; Prigerson, 2001). A brief, clinically useful risk self-report assessment tool would enable clinicians from a variety of disciplines to better identify individuals at risk and not at risk for challenges in bereavement in order to facilitate access to psychosocial support.

The present study was the first phase of a 3-phase project to address this need by developing a clinically useful self-report risk screening tool, the Bereavement Risk Inventory and Screening Questionnaire (BRISQ), to identify high-risk family members before or after losing a loved one to cancer. Using patient-reported outcome (PRO) methods, this first phase involved item generation, the second study phase will involve item improvement through cognitive interviewing with family members, and the third phase will involve clinical validation of the tool (Rothrock, 2011).

While the intensity of psychological symptoms tends to decrease with time in bereavement (Bonanno & Kaltman, 2001), a relatively small but important subset of bereaved family members (Lannen, 2008) go on to develop more severe and persistent bereavement-related mental health challenges (Kersting et al., 2011; Kristensen, 2014; Prigerson et al., 2009; Aoun, 2015), such as Major Depressive Disorder, Post-traumatic Stress Disorder (PTSD), Prolonged Grief Disorder (PGD), and substance abuse. Screening for risk factors may not only provide the opportunity for prophylactic mental health support, but also for prevention of the numerous emotional and physical health challenges associated with intense, impairing psychological distress in bereaved individuals, including psychiatric hospitalization (Li et al., 2005), suicidality (Latham & Prigerson, 2004), increased rates of cancer, heart problems, gastrointestinal problems, sleep disturbances, unhealthy behaviors (Prigerson et al., 1997; Stroebe et al., 2007; Ott, 2003), stress-related physiological vulnerabilities (Kiecolt-Glaser et al., 2003), and even mortality (Schulz & Beach, 1999; Cohen-Mansfield et al., 2013).

Risk Factors

Literature reviews (Burke & Neimeyer, 2012; Lobb et al., 2010) have identified a number of risk factors associated with mental health challenges. These are captured by three main categories: background, illness/death-related, and bereavement-related factors (Burke & Neimeyer, 2012). We further describe these categories and the supporting literature below.

Background Risk Factors

Background risk factors for bereavement-related mental health challenges are varied and include demographic factors such as younger age (Tomarken et al., 2008), female gender (Stroebe et al., 2001; Chiu et al., 2010), minority race or ethnicity (Goldsmith et al., 2008; Youngblut et al., 2013), and lower socioeconomic status (van der Houwen et al., 2010). Additionally, prior loss or trauma experiences and/or a previous psychiatric diagnosis can put individuals at risk when they face bereavement (Kelly et al., 1999; Ellifritt et al., 2003). Studies also suggest that an avoidant or a ruminative style of coping (Galatzer-Levy & Bonanno, 2012; Kapari et al., 2010) and insecure attachment style (Field & Filanosky, 2010; Johnson, 2007) have been identified as risk factors. Particularly in spouses, if the relationship with the deceased was characterized by a high degree of dependency, the surviving family member may also be at higher risk (Thomas et al., 2014; Lichtenthal & Sweeney, 2014; Johnson, 2007). Finally, kinship to the deceased has been associated with mental health challenges, with parents of young children and elderly spouses being at the greatest risk (Bonnano et al., 2004; Kersting et al., 2011; van Doorn, 1998).

Illness/Death-Related Factors

Greater cancer severity toward end-of-life (Zisook et al., 1998), challenges with advanced care directives (Kristensen et al., 2012), and poor patient quality of life at the end-of-life (Garrido & Prigerson, 2014) have been linked with greater psychological distress in bereavement. Additionally, when the circumstances of a cancer death are perceived as traumatic or unexpected by family members, such as medical interventions that result in disfigurement, uncontrolled pain, seizing or bleeding at the time of death, family members are at greater risk for PTSD specifically (Currier et al., 2006; Zisook et al., 2013; Sanderson et al., 2013). Caregivers who experience high levels of emotional distress and/or stressful events during the patient’s cancer may be particularly vulnerable to intense emotional distress during bereavement (Kapari et al., 2010; Tomarken et al., 2008). Finally, lack of emotional preparedness for the death (Barry et al., 2002; Lobb et al., 2010) and low social support (Kurtz et al., 1997; Thomas et al., 2014) can place caregivers at greater risk.

Bereavement-related Factors

While the type and circumstances of death can have a significant impact on bereavement outcomes, studies have also found that the way individuals process the death, particularly the degree to which an individual experiences feelings of regret, guilt, shame or self-blame (Akiyama et al., 2010; Rosenzweig et al., 1997; Kelly et al., 1999) can also place them at risk. Those who significantly struggle with maintaining a sense of meaning in life or purpose and/or making sense of the death may be at increased risk for mental and physical health problems (Lichtenthal et al., 2010; Neimeyer & Sands, 2011; Lichtenthal et al., 2013).

Protective Factors

Spirituality and religion have been found to be both protective and risk factors for intense grief reactions (Bonanno et al., 2002). Insufficient and/or negative social support has been identified as a significant risk factor, while positive, high quality social support can be a significant protective factor present at any point in the bereavement process, prior to, or following the death (van der Houwen et al., 2010; Ott, 2003).

Need for Bereavement Screening

Despite the increased risk for protracted distress that bereaved family members of cancer patients may face (Vanderwerker et al., 2005; Allen et al., 2013), they also tend to underutilize mental health services (Cherlin et al., 2007; Lichtenthal, 2015). As they become disconnected from medical providers after the death or if the treating medical institution has limited resources to initiate and maintain bereavement care, surviving family members often “fall through the cracks” (Azoulay et al., 2005; Sealey, 2015b). A recent survey of Intensive Care Unit (ICU) medical providers and family members of ICU patients post-loss revealed that 68% of family members expressed a desire for bereavement support but only 31% reported receiving professional support for emotional concerns (Downar et al., 2014). Further, over half of the family members believed routine bereavement screening should be a standard of care and 98% of medical providers reported they would support and/or participate in bereavement screening. Additionally, in their interviews with palliative care providers, Sealey et al. (2015b) found that, though providers supported the idea of bereavement screening, they viewed current methods of assessing bereavement risk as insufficient and limited by staff intuition.

Despite the compelling need for a brief self-report bereavement risk screening tool, measures to assess transdiagnostic psychosocial risk factors for the development of bereavement-related mental health challenges have a number of limitations, contributing to challenges in the implementation of systematic screening (Sealey, 2015a). Clinician-administered screening tools have not proven reliable (Rose et al., 2011; Kristjanson, 2006; Sealey, 2015b) and self-report bereavement risk assessments (Jordan et al., 2005; Guldin et al., 2011) have not been widely used or validated due to their burdensome length and limited clinical utility (Agnew et al., 2010). In addition, such measures have not been developed using systematically obtained expert feedback or respondent input. This step in measure development is not only quickly becoming a standard in psychometric methodology in health care settings (Sofaer, 2002; Ahmed et al., 2009) but is also crucially important in the case of bereavement, as the topic is subject to controversy (Lamb et al., 2010), emotionally provocative, and the target respondents can be considered emotionally vulnerable. Finally, though screening tools exist for bereavement-related disorders (Ito, 2012), their purpose is to assess for symptoms associated with a specific disorder, rather than transdiagnostic risk factors, potentially excluding family members who do not meet the criterion assesssed.

The purpose of identifying family members at risk for developing bereavement-related mental health challenges pre- or post-loss is two-fold: 1) to make a connection with the at-risk family members so that they might feel more comfortable seeking assistance from the institution in the future should they need psychosocial services and 2) to track family members identified as at-risk in order to allocate limited bereavement follow-up resources. Importantly, the BRISQ may reduce the impact of informal gatekeeping and staff bias noted by Sealey et al. (2015b) as a barrier to psychosocial care by offering a brief method of universal assessment of family members pre- or post-loss that is not dependent on staff recognition of clinically significant symptoms. Many cancer settings do not have the resources to provide consistent, in-depth, specialized psychosocial care to every family member. Similarly, hospitals may lack the tools to track family members once patients die (Downar et al., 2014) and when tools are available, they have not been systematically implemented, suggesting limitations to current bereavement screening practices (Sealey, 2015a). A brief, self-report tool administered in medical care settings could be used as a basic way to initiate a conversation about psychosocial care for each family member pre-loss in order to provide them with psychosocial treatment options and referrals before the death occurs, regardless of risk (Lichtenthal et al., 2011).

This study represents the first steps in developing the BRISQ to assess bereaved family members and ultimately to facilitate appropriate and effective psychosocial support. To determine the clinical utility of the BRISQ, this study used qualitative and quantitative methods to seek expert (clinicians and academics specializing in bereavement) input on an initial item pool and bereavement screening overall.

Methods

Participants

Consistent with methods utilized in previous studies to recruit expert panels (Butler et al., 2008), the selection of experts was purposive. Several prior studies have found that 8 to 20 is an acceptible number of experts to provide valuable information in order to validate the content of a measure (Grant & Davis, 1997; Polit et al., 2007). Using personal recommendations from other experts in the field including members of the study team (W.L., H.P., J.H.), 20 experts in the field of bereavement were invited to complete an online survey to provide quantitative and qualitative feedback on the preliminary BRISQ.

Of the 20 invited, 15 bereavement experts participated. The remaining 5 individuals were unresponsive to the invitation to participate. The majority of participants identified as both a researcher and mental health clinician (n = 10), with the remaining 5 identifying as either a clinician (n = 3) or a researcher (n = 2). Of those who reported engaging in clinical work, most were psychologists (n = 9) and the other 4 clinicians identified as a grief counselor (n = 1), psychiatrists (n = 2), social worker (n = 1), or another specialty (n = 2). Participants had been in the bereavement field for an average of 26 years, with a range of 7 to 41 years. While the experts all identified as non-Hispanic and Caucasian, there was an approximately equal distribution of gender, with 8 male and 7 female experts. Additionally, the expert panel represented United States (n = 11) and international (n = 4) perspectives. The average age of the participants was 58 years and ranged from 36 years to 75 years. Participants were informed about all components and goals of this project and were offered $50 compensation for their time.

Measures

Bereavement Risk Inventory and Questionnaire (BRISQ)

The overall design of the BRISQ was formatted to assess various time points and to include a multitude of risk factors. Additionally, the BRISQ was designed in checklist format, with a “yes” or “no” response format for each item, making the screening tool potentially easier to understand and faster to complete and score. However, we also sought feedback to determine the best response scale. For a number of constructs (e.g., attachment style, sense of meaning), several items were proposed in order to obtain expert feedback on the most appropriate method of assessing for these factors. Finally, the BRISQ was designed to ultimately have two different versions to be administered either pre- or post-loss with language appropriate to the respective time frame.

The preliminary items for the BRISQ were developed based on a thorough review of the bereavement risk factor literature (as described above), using reviews by Lobb et al. (2010) and Burke and Neimeyer (2012) as guides. Additionally, items verified by the literature as prominent risk factors were selected from pre-existing bereavement screening tools, including the GEM (Jordan et al., 2005) and those developed by Rose et al. (2011) and Guldin et al. (2011). Items were identified as being within three main categories of risk described above: background factors, illness/death-related factors, and bereavement-related factors (Burke & Neimeyer, 2012; Lobb et al., 2010).

Expert Survey

Experts provided ratings and input on all candidate items compiled through an anonymous web-based survey on a secure data collection platform (Qualtrics, 2005). The Content Validity Index (CVI), a scale utilized in medical research scale development, was used to obtain expert input on item relevance and to establish that the BRISQ has appropriate selection of items sufficient to measure the construct being studied (Polit et al., 2007). The CVI is a 4-point Likert-type scale, with 1 being not relevant and 4 being highly relevant, that has been demonstrated as effective in assessing for content validity in other studies that used expert ratings (Polit et al., 2007). Relevance was considered synonymous with clinical utility in assessing expert feedback using the CVI. For certain items that could be relevant to pre- or post-loss experiences (e.g., emotional preparedness for death), experts were asked to indicate the time point at which the item would be appropriate to ask. Free response boxes were also included to enter any qualitative feedback experts had about a given item. Finally, they were asked to indicate their overall opinion on the value of risk screening.

Procedures

Sixty-five risk factors were selected from systematic reviews of the bereavement risk literature and previous bereavement risk screening tools for the initial item pool. These items underwent first-level refinement using the expert opinion of the authors (K.R., W.L., J.H.), resulting in a preliminary pool of 49 items presented in the expert survey.

Based on modified methods proposed in previous studies (Butler et al., 2008; Polit et al., 2007), quantitative (CVI) and qualitative results from the expert survey were used to refine the preliminary BRISQ and develop pre- and post-loss versions. The CVI results were analyzed using descriptive statistics to determine which items had the highest mean ratings of clinical relevance and the distribution of ratings. Items were determined to have sufficient content validity by calculating inter-rater agreement using methods described by Polit et al. (2007; i.e., the proportion of experts that gave an item a moderate to high relevance rating [3 or 4] on the CVI). Items that less than 80% of the experts rated as relevant were eliminated, except in cases where the qualitative feedback suggested the item should be retained with changes.

The qualitative feedback provided for individual items and the overall measure was coded for shared themes using a thematic analysis approach (Braun & Clarke, 2006). Two coders (K.R. & C.S.) developed a coding system and independently coded the responses for feedback related to content, language, additions, eliminations, and utility. Coders reached consensus on the themes and these findings were discussed by W.L. and K.R. to determine refinements to the BRISQ. In cases where expert feedback significantly differed from evidence in the literature or suggested changing it to such a degree that it did not represent the risk factor, the item or its language was retained. These pre- and post-loss versions of the BRISQ were further reviewed by H.P. and W.L. to refine the language and ensure all potentially significant risk factors were included following the elimination of items.

Results

The original item pool of the BRISQ was reduced by 22%, with retained items reflecting those that at least 80% of the experts rated as quite or highly relevant. These items corresponded with the experts’ qualitative feedback and items that the primary authors (W.L. & K.R.) determined as representative of significant risk factors within the literature. In most cases, items that were rated as relevant by the least number of experts also had weak support for their unique predictive validity in the literature (e.g., female gender, level of education and African American race). The expert ratings of whether the items should be asked pre- and post-loss or both was also considered for each item and when the experts indicated an item should be used both for pre- and post-loss, the item language was changed accordingly in each version.

The qualitative feedback provided by the experts was generally positive regarding the included items, though suggestions for changing item language were offered to enhance the items’ clinical utility. These language suggestions ranged from purely grammatical to increasing the emotional or cultural sensitivity of an item. Additionally, some experts suggested that certain items (e.g., trauma history) should include clear indicators of the time frame being assessed (e.g., “prior to your adulthood”) or should eliminate specific descriptors (e.g., “feeling constantly angry” to “feeling angry”). Though there was consensus on the utility of including items assessing attachment style and personality traits, experts were conflicted on how to best assess these concepts, offering various suggestions. Experts suggested additional items, such as one assessing the family member’s perception of the physical pain experienced by the patient.

Specific items that were eliminated based on the expert relevance ratings and qualitative feedback on the items’ clinical utility included time since death (given its particular use in diagnosis of PGD rather than assessing for overall risk), experiencing stress during caregiving, experiencing other stressors in daily life, feeling anxious, lacking a sense of peace, and finding comfort in spirituality or religion. Experts who rated these items as irrelevant provided reasons such as lack of specificity of the item (e.g., other stressors eliminated), ubiquity of the experience (e.g., stress during caregiving eliminated) or overlap with an item that captured the concept better (e.g., one attachment item retained; spirituality eliminated in favor of meaning-making). The feedback led to a number of changes to the item language including minor rephrasing, the combination of items (e.g. those assessing attachment) and clarifying key terms.

Experts also provided qualitative input on the design of the BRISQ. With regard to the response options, some experts suggested using Likert-type scales, rather than a binary yes/no option. Accordingly, items were revised to be in question, rather than statement format. Four-point Likert-type scales were assigned to each item with the response options varying based on whether the question was meant to assess frequency or degree of agreement. One important piece of feedback was to alter references to the deceased individual, originally referred to as the “loved one,” to a term without emotional valence, particularly since having a conflicted relationship with the deceased puts one at risk. Thus, references to “loved one” were changed to the more neutral “family member.” Finally, all expert participants agreed that there would be value to screening, with some noting a self-report tool would be useful for their clinical practice or institution.

Combined, these findings guided the refinement of the screening tool language, format and content, resulting in a final set of 34 items in the pre-loss version (BRISQ-P) and 38 items in the post-loss version (BRISQ-B). Overall, most of the risk factor categories captured in the original item pool were retained, but the items were refined and the number was reduced to increase the BRISQ’s utility and clarity. Table 1 displays the risk factors categories included in the revised BRISQ, the specific risk factors assessed, and examples of revised items.

Table 1.

Displays the broader risk categories, factors assessed, and examples of items in the revised Bereavement Risk Inventory and Screening Questionnaire.

Risk Category Risk Factors Example Item
Background Family member age
Relationship to patient or deceased		 What is your relationship to your family member who is ill?
Support Presence of social support
Satisfaction with social support
Desire for professional support		 Overall, do you feel satisfied with the ways in which others have supported you during this difficult time?
Relationship Style Anxious personality or relational style
Avoidant personality or relational style		 In general, do you often have concerns about getting too close to people or depending too much on them?
Prior Stressors and Coping Style Past difficulty coping with loss or separation
History of mental health problems
Trauma history
Prior suicidal ideation
Prior suicide attempt		 Prior to your adulthood, did you ever experience any major challenges (for example, loss, separation, abuse) that you consider traumatic?
Relationship with Family Member Conflictual relationship with patient or deceased)
Dependence on patient or deceased		 Would you characterize your relationship with your ill family member as difficult, challenging, or conflicted?
Circumstances of Illness (or Death) Perception that patient experiencing uncontrolled pain/distress (BRISQ-P)
Perception of death as painful or violent (BRISQ-B)
Perception of illness or death as traumatic
Emotional preparation for death
Witnessing aggressive medical procedures
Witnessing death (BRISQ-B)
Satisfaction with death notification (BRISQ-B)
Satisfaction with quality of death (BRISQ-B)		 From your perspective, has your family member been experiencing uncontrolled physical pain or emotional distress?
Caregiving Experiences Being active caregiver
Conflict with family members		 Have you played an active role in caring for your family member?
Illness (or Bereavement) Experiences Difficulty functioning
Isolating self or feeling isolated
Experiencing flashbacks or disturbing images
Financial problems
Anger about illness or death
Current suicidal ideation
Emotional numbness
Avoidance of reminders (BRISQ-B)
Feelings of guilt associated with illness or death
Difficulty with acceptance		 Have you found it difficult to engage and function in important areas in your life since your family member became ill (or since the death of your family member)?
Meaning, Identity and Purpose Difficulty finding meaning in illness or death
Disconnection from meaning or purpose in life
Disconnection from sense of identity		 Do you feel you have lost your sense of meaning or purpose since your family member became ill (or since the death of your family member)?
Open in a new tab

Discussion

This study obtained expert feedback on the clinical utility and wording of items being considered for inclusion in a brief, self-report, clinically useful bereavement risk screening tool that could be used to inform referrals for professional psychosocial support. In general, participating bereavement experts confirmed Sealey et al.’s (2015b) findings on the need for a clinically useful self-report bereavement screening tool and verified the utility and thoroughness of the preliminary item pool. Results reflected a general consensus between experts on risk factors for bereavement-related mental health challenges, with the majority of the feedback suggesting modifications to the phrasing of items and the response scale.

Although the experts agreed that the proposed content is important to include, their feedback resulted in substantial revisions in the length, presentation and language used in the BRISQ. A significant change was the adaptation from checklist format (with items phrased as statements and responses as yes/no options) to question format (with Likert-type scale response options). Items that were eliminated because of their lower clinical relevance ratings by the experts or qualitative feedback were noted to be inconsistently and/or not uniquely linked to bereavement-related mental health challenges in the literature. Most notably, the items on race and gender were eliminated because the evidence of risk associated with female gender or African American race (Burke et al., 2010; Stroebe et al., 2001) did not point to these being risk factors on their own but rather representative of the cumulation of other risk factors (e.g., lower income, lack of provider cultural competency, and systemic oppression). Because the intent of the BRISQ is to facilitate referrals for individuals in distress, the consensus among the experts and the authors was that these demographics would be insufficient indicators of risk for prolonged distress. These and others were eliminated in order to reduce the item pool and burden of completion. We focused on retaining the most potentially powerful predictors as suggested by the literature and the experts.

The results of this study demonstrate the importance of researcher and clinician input on the specifics of item development, such as language and phrasing, rather than solely relying on the research literature, as much of the revised language on the BRISQ was facilitated by the experts’ breadth and depth of knowledge. Despite this strength, the study had several limitations, most notably the lack of racial/ethnic diversity and diversity of professional disciplines represented by the sample. This limitation is largely due to the use of a convenience sample of experts known to the authors, a method common to psychometrics but not ideal for gathering a wide range of perspectives. A related challenge is the limited utility of expert panels on the whole. Though expert feedback has been found to be useful in assessing content validity of items (Patrick et al., 2011), care was taken to not rely solely on expert opinion to eliminate items because of the potential to miss an important predictor. The evidence for items in the literature should be considered and items should be retained until the measure’s predictive validity can be assessed in a longitudinal study.

In order to further refine the BRISQ and improve its utility as a self-report tool, a second phase of the study is underway that involves conducting cognitive interviews with family members of cancer patients and bereaved family members to obtain information about their comprehension of and feedback on the BRISQ-P and BRISQ-B respectively. In part due to the limitations of expert panels, this is an essential step in PRO methodology used in healthcare settings (Patrick et al., 2011). Approaching the development and validation of this screening tool using PRO methodology increases its utility and emotional sensitivity, and promotes person-centered care within palliative, hospice and bereavement care. The BRISQ will be further refined based on family member feedback and refined versions will be used in a longitudinal study of the psychometrics to test and finalize the BRISQ. By incorporating both expert and family member feedback, the intention is to create a screening tool that represents top clinical and research knowledge in bereavement in a way that addresses barriers to care (Sealey, 2015b; Lichtenthal, 2015) and is comprehensible and sensitive to family members who may be struggling during bereavement or caregiving.

Acknowledgments

This research was generously supported by National Cancer Institute grants T32 CA009461 26, K07 CA172216, P30 CA008748, F31 CA192447 and by the Seth Sprague Foundation.

The authors would like to thank Robert Neimeyer, Ph.D., Laurie Burke, Ph.D., Sherry Schachter, Ph.D., and Stacy Stabler, M.D. for their contributions to the project and Lisa Rubin, Ph.D., the first author’s (KR) graduate school advisor at the New School for Social Research, for her invaluable mentorship. We also thank the MSK Behavioral Research Methods Core for their support.

Footnotes

Ethics approval: This study was approved by the Memorial Sloan Kettering Cancer Center Institutional Review Board on April 16, 2014. The protocol was approved as exempt research as per 45 CFR 46.101 (b)(c). HIPAA Authorization and informed consent were waved as per 45 CFR 164.512(i)(2)(ii) and 45 CFR 46.116(d.).

Competing Interests: The authors declare that they have no competing interests.

Authors’ Contributions: KR: Designed the study, conducted data collection and analysis, and drafted and edited the manuscript. JH: Assisted with screening tool item development, editing the manuscript, and final approval. HP: Assisted with screening tool item development, editing the manuscript, and final approval. CS: Assisted with data analysis, editing the manuscript, and final approval. GC: Assisted with data analysis, editing the manuscript, and final approval. WB: Assisted with editing the manuscript and final approval. WL: Designed the study, conducted data analysis, and assisted with drafting and editing the manuscript. All authors read and approved the final manuscript.

References

  1. Agnew A, Manktelow R, Taylor BJ, Jones L. Bereavement needs assessment in specialist palliative care: A review of the literature. Palliative Medicine. 2010;24(1):46–59. doi: 10.1177/0269216309107013. [DOI] [PubMed] [Google Scholar]
  2. Ahmed N, Bestall JC, Payne SA, Noble B, Ahmedzai SH. The use of cognitive interviewing methodology in the design and testing of a screening tool for supportive and palliative care needs. Support Care Cancer. 2009;17:665–673. doi: 10.1007/s00520-008-0521-2. [DOI] [PubMed] [Google Scholar]
  3. Akiyama A, Numata K, Mikami H. Importance of end-of-life support to minimize caregiver’s regret during bereavement of the elderly for better subsequent adaptation to bereavement. Archives of Gerontology and Geriatrics. 2010;50(2):175–178. doi: 10.1016/j.archger.2009.03.006. [DOI] [PubMed] [Google Scholar]
  4. Allen JY, Haley WE, Small BJ, Schonwetter RS, McMillan SC. Bereavement among hospice caregivers of cancer patients one year following loss: Predictors of grief, complicated grief, and symptoms of depression. Journal of Palliative Medicine. 2013;16:745–751. doi: 10.1089/jpm.2012.0450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Aoun S, Breen LJ, Howting D, Rumbold B, McNamara B, Hegney D. Who needs bereavement support? A population based survey of bereavement risk and support need. PLOS One. 2015;10(3) doi: 10.1371/journal.pone.0121101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Azoulay E, Pochard F, Kentish-Barnes N, Chevret S, Aboab J, Adrie C, Schlemmer B. Risk of post-traumatic stress symptoms in family members of intensive care unit patients. American Journal of Respiratory Critical Care Medicine. 2005;171:987–994. doi: 10.1164/rccm.200409-1295OC. [DOI] [PubMed] [Google Scholar]
  7. Barry L, Kasl S, Prigerson HG. Psychiatric disorders among bereaved persons: The role of perceived circumstances of death and preparedness for death. The American Journal of Geriatric Psychiatry. 2002;10:447–457. [PubMed] [Google Scholar]
  8. Bonanno GA, Kaltman S. The varieties of grief experience. Clinical Psychology Review. 2001;21:705–734. doi: 10.1016/s0272-7358(00)00062-3. [DOI] [PubMed] [Google Scholar]
  9. Bonanno GA, Wortman CB, Lehman DR, Tweed R, Haring M, Sonnega J, Carr D, Nesse RM. Resilience to loss and chronic grief: A prospective study from pre-loss to 18-months post-loss. Journal of Personality and Social Psychology. 2002;83:1150–1164. doi: 10.1037//0022-3514.83.5.1150. [DOI] [PubMed] [Google Scholar]
  10. Bonnano GA, Wortman CB, Nesse RM. Prospective patterns of resilience and maladjustment during widowhood. Psychology and Aging. 2004;19:260–271. doi: 10.1037/0882-7974.19.2.260. [DOI] [PubMed] [Google Scholar]
  11. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101. [Google Scholar]
  12. Burke LA, Neimeyer RA, editors. Prospective risk factors for complicated grief. New York: Routledge; 2012. [Google Scholar]
  13. Burke LA, Neimeyer RA, McDevitt-Murphy M. African American homicide bereavement: Aspects of social support that predict complicated grief, PTSD, and depression. Omega. 2010;61(1):1–24. doi: 10.2190/OM.61.1.a. [DOI] [PubMed] [Google Scholar]
  14. Butler SF, Fernandez K, Benoit C, Budman SH, Jamison RN. Validation of the revised screener and opioid assessment for patients with pain. Journal of Pain. 2008;9:360–372. doi: 10.1016/j.jpain.2007.11.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cherlin EJ, Barry CL, Prigerson HG, Green DS, Johnson-Hrzeler R, Kasi SV, Bradley EH. Bereavement services for family caregivers: How often used, why, and why not. Journal of Palliative Medicine. 2007;10:148–158. doi: 10.1089/jpm.2006.0108. [DOI] [PubMed] [Google Scholar]
  16. Chiu YW, Huang C, Yin S, Huang Y, Chien C, Chuang H. Determinants of complicated grief in caregivers who cared for terminal cancer patients. Supportive Care Cancer. 2010:1321–1327. doi: 10.1007/s00520-009-0756-6. [DOI] [PubMed] [Google Scholar]
  17. Cohen-Mansfield J, Shmotkin D, Malkinson R, Bartur L, Hazan H. Parental bereavement increases mortality in older persons. Psychological Trauma: Theory, Research, Practice, and Policy. 2013;5(1):84–92. [Google Scholar]
  18. Currier JM, Holland JM, Neimeyer RA. Sense-making, grief, and the experience of violent loss: toward a mediational model. Death Studies. 2006;30(5):403–428. doi: 10.1080/07481180600614351. [DOI] [PubMed] [Google Scholar]
  19. Downar J, Barua R, Sinuff T. The desirability of an Intensive Care Unit (ICU) clinician-led bereavement screening and support program for family members of ICU decedents (ICU Bereave) Journal of Critical Care. 2014;29(2):311.e319–311.e316. doi: 10.1016/j.jcrc.2013.11.024. [DOI] [PubMed] [Google Scholar]
  20. Ellifritt J, Nelson KA, Walsh D. Complicated bereavement: A national survey of potential risk factors. American Journal of Hospice and Palliative Medicine. 2003;20:114–120. doi: 10.1177/104990910302000209. [DOI] [PubMed] [Google Scholar]
  21. Field N, Filanosky C. Continuing bonds, risk factors for complicated grief and adjustment to bereavement. Death Studies. 2010;34:1–29. doi: 10.1080/07481180903372269. [DOI] [PubMed] [Google Scholar]
  22. Galatzer-Levy IR, Bonanno GA. Beyond normality in the study of bereavement: heterogeneity in depression outcomes following loss in older adults. Social Science and Medicine. 2012;74(12):1987–1994. doi: 10.1016/j.socscimed.2012.02.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Garrido MM, Prigerson HG. The end-of-life experience: modifiable predictors of caregivers’ bereavement adjustment. Cancer. 2014;120(6):918–925. doi: 10.1002/cncr.28495. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Goldsmith B, Morrison RS, Vanderwerker LC, Prigerson HG. Elevated rates of prolonged grief disorder in African Americans. Death Studies. 2008;32(4):352–365. doi: 10.1080/07481180801929012. [DOI] [PubMed] [Google Scholar]
  25. Grant JS, Davis LL. Selection and use of content experts for instrument development. Research in Nursing Health. 1997;20(3):269–274. doi: 10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g. [DOI] [PubMed] [Google Scholar]
  26. Guldin MB, O’Connor M, Sokolowski L, Jensen AB, Vedsted P. Identifying bereaved subjects at risk of complicated grief: Predictive value of questionnaire items in a cohort study. BioMedCentral Palliative Care. 2011;10:1–7. doi: 10.1186/1472-684X-10-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Institute of Medicine. Committee on care at the end of life, Division of Health Care Services. In: Field MJ, Cassel CK, editors. Dying in America: Improving quality and honoring individual preferences near the end of life. Washington D.C: National Academy Press; 2014. [Google Scholar]
  28. Ito M, Nakajima S, Fujisawa D, Miyashita M, Kim Y, Shear M, Ghesquiere A, Wall M. Brief measure for screening complicated grief: Reliability and discriminant validity. PLOS One. 2012;7(2) doi: 10.1371/journal.pone.0031209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Johnson JG, Zhang B, Greer JA, Prigerson HG. Parental control, partner dependency, and complicated grief among widowed adults in the community. Journal of Nervous and Mental Disease. 2007;195(1):26–30. doi: 10.1097/01.nmd.0000252009.45915.b2. [DOI] [PubMed] [Google Scholar]
  30. Jordan JR, Baker J, Matteis M, Rosenthal S, Ware ES. The grief evaluation measure (GEM): an initial validation study. Death Studies. 2005;29(4):301–332. doi: 10.1080/07481180590923706. [DOI] [PubMed] [Google Scholar]
  31. Kapari M, Addington J, Hotopf M. Risk factors for common mental disorder in caregiving and bereavement. Journal of Pain and Symptom Management. 2010;40(6):844–856. doi: 10.1016/j.jpainsymman.2010.03.014. [DOI] [PubMed] [Google Scholar]
  32. Kelly B, Edwards P, Synott R, Neil C, Baillie R, Battistutta D. Predictors of bereavement outcome for family carers of cancer patients. Psychooncology. 1999;8(3):237–249. doi: 10.1002/(SICI)1099-1611(199905/06)8:3<237::AID-PON375>3.0.CO;2-C. [DOI] [PubMed] [Google Scholar]
  33. Kersting A, Brahler E, Glaesmer H, Wagner B. Prevalence of complicated grief in a representative population-based sample. Journal of Affective Disorders. 2011;131(1–3):339–343. doi: 10.1016/j.jad.2010.11.032. [DOI] [PubMed] [Google Scholar]
  34. Kiecolt-Glaser JK, Preacher KJ, MacCallum RC, Atikinson C, Malarkey WB, Glaser R. Chronic stress and age-related increases in the pro inflammatory cytokine IL-6. Proceedings of the National Academy of Sciences. 2003;100:9090–9095. doi: 10.1073/pnas.1531903100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kristensen TE, Elklit A, Karstoft K. Posttraumatic stress disorder after bereavement: Early psychological sequelae of losing a close relative due to terminal cancer. Journal of Loss and Trauma. 2012;17:508–521. [Google Scholar]
  36. Kristensen TE, Elklit A, Karstoft KI, Palic S. Predicting chronic posttraumatic stress disorder in bereaved relatives: A 6-month follow-up study. American Journal of Hospice and Palliative Medicine. 2014;31(4):396–405. doi: 10.1177/1049909113490066. [DOI] [PubMed] [Google Scholar]
  37. Kristjanson L. Evaluation of the Bereavement Risk Index (BRI): A community hospice care protocol. International Journal of Palliative Nursing. 2006;11(12):610–618. doi: 10.12968/ijpn.2005.11.12.20226. [DOI] [PubMed] [Google Scholar]
  38. Kurtz ME, Kurtz JC, Given CW, Given B. Predictors of post bereavement depressive symptomatology among family caregivers of cancer patients. Supportive Care in Cancer. 1997;5:53–60. doi: 10.1007/BF01681962. [DOI] [PubMed] [Google Scholar]
  39. Lamb K, Pies R, Zisook S. The bereavement exclusion for the diagnosis of major depression: To be or not to be. Psychiatry. 2010;7:19–25. [PMC free article] [PubMed] [Google Scholar]
  40. Lannen PK, Wolfe J, Prigerson HG, Onelov E, Kreicbergs UC. Unresolved grief in a national sample of bereaved parents: Impaire mental and physical health 4 to 9 years later. Journal of Clinical Oncology. 2008;26(36):5870–5876. doi: 10.1200/JCO.2007.14.6738. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Latham AE, Prigerson HG. Suicidality and bereavement: Complicated grief as psychiatric disorder presenting greatest risk for suicidality. Suicide and Life Threatening Behaviors. 2004;34:350–362. doi: 10.1521/suli.34.4.350.53737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Li J, Laursen TM, Precht DH, Olsen J, Mortensen PB. Hospitalization for mental illness among parents after the death of a child. New England Journal of Medicine. 2005;352(12):1190–1196. doi: 10.1056/NEJMoa033160. [DOI] [PubMed] [Google Scholar]
  43. Lichtenthal WG, Corner GW, Sweeney CR, Wiener L, Roberts KE, Baser RE, Li Y, Breitbart W, Kissane DW, Prigerson HG. Mental health services for parents who lost a child to cancer: If we build them, will they come? Journal of Clinical Oncology. 2015;33:1–10. doi: 10.1200/JCO.2014.59.0406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Lichtenthal WG, Currier JM, Neimeyer R, Keesee NJ. Sense and significance: A mixed methods examination of meaning making after the loss of one’s child. Journal of Clinical Psychology. 2010;66(7):791–812. doi: 10.1002/jclp.20700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Lichtenthal WG, Neimeyer RA, Currier JM, Roberts K, Jordan N. Cause of death and the quest for meaning after the loss of a child. Death Studies. 2013;37(4):311–342. doi: 10.1080/07481187.2012.673533. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Lichtenthal WG, Nilsson M, Kissane DW, Kacel E, Jones EC, Prigerson HG. Underutilization of mental health services among bereaved caregivers with prolonged grief disorder. Psychiatric Services. 2011;62:1225–1229. doi: 10.1176/appi.ps.62.10.1225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Lichtenthal WG, Sweeney C. Families ‘at risk’ of complicated bereavement. In: Kissane D, editor. Bereavement Care for Families. New York: Routledge; 2014. [Google Scholar]
  48. Lobb EA, Kristjanson LJ, Aoun SM, Monterosso L, Halkett GKB, Davies A. Predictors of complicated grief: a systematic review of empirical studies. Death Studies. 2010;34(8):673–698. doi: 10.1080/07481187.2010.496686. [DOI] [PubMed] [Google Scholar]
  49. Neimeyer RA, Sands DC. Meaning reconstruction in bereavement: From principles to practice. In: Neimeyer RA, Harris DL, Winokuer HR, Thornton GF, editors. Grief and Bereavement in Contemporary Society: Bridging Research and Practice. New York: Routledge; 2011. pp. 9–22. [Google Scholar]
  50. Ott CH. The impact of complicated grief on mental and physical health at various points in the bereavement. Death Studies. 2003;27(3):249–272. doi: 10.1080/07481180302887. [DOI] [PubMed] [Google Scholar]
  51. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health. 2011;14(8):978–988. doi: 10.1016/j.jval.2011.06.013. [DOI] [PubMed] [Google Scholar]
  52. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity?: Appraisal and recommendations. Research in Nursing and Health. 2007;30:459–467. doi: 10.1002/nur.20199. [DOI] [PubMed] [Google Scholar]
  53. Prigerson H, Jacobs SC. Perspectives on care at the close of life. Caring for bereaved patients: “all the doctors just suddenly go”. Journal of American Medical Association. 2001;286(11):1369–1376. doi: 10.1001/jama.286.11.1369. [DOI] [PubMed] [Google Scholar]
  54. Prigerson HG, Bierhals AJ, Kasl SV, Reynolds CF, Shear MK, Day N, Beery LC, Newsom JT, Jacobs S. Traumatic grief as a risk factor for mental and physical morbidity. American Journal of Psychiatry. 1997;154(5):616–623. doi: 10.1176/ajp.154.5.616. [DOI] [PubMed] [Google Scholar]
  55. Prigerson HG, Horowitz MJ, Jacobs SC, Parkes CM, Aslan M, Goodkin K, Raphael B, Marwit SJ, Wortman C, Neimeyer RA, Bonanno GA, Bonanno G, Block SD, Kissane D, Boelen P, Maercker A, Litz BT, Johnson JG, First MB, Maciejewski PK. Prolonged grief disorder: Psychometric validation of criteria proposed for DSM-V and ICD-11. PLoS Med. 2009;6(8) doi: 10.1371/journal.pmed.1000121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Qaseem A, Snow V, Shekelle P, Casey DE, Cross T, Owens DK. Evidence-based interventions to improve the palliative care of pain, dyspnea, and depression at the end of life: A clinical practice guidelines from the American College of Physicians. Annals of Internal Medicine. 2008;148:141–146. doi: 10.7326/0003-4819-148-2-200801150-00009. [DOI] [PubMed] [Google Scholar]
  57. Qualtrics. Qualitrics. Provo, Utah: Qualtrics; 2005. [Google Scholar]
  58. Rose C, Wainwright W, Downing M, Lesperance M. Inter-rater reliability of the bereavement risk assessment tool. Palliative and Supportive Care. 2011;9:153–164. doi: 10.1017/S1478951511000022. [DOI] [PubMed] [Google Scholar]
  59. Rosenzweig A, Prigerson H, Miller M, Reynolds CF. Bereavement and late life depression: Grief and its complications in the elderly. Annual Review of Medicine. 1997;48:421–428. doi: 10.1146/annurev.med.48.1.421. [DOI] [PubMed] [Google Scholar]
  60. Rothrock N, Kaiser KA, Cella D. Developing a valid patient-reported outcome measure. Clinical Pharmacology Therapeutics. 2011;90(5):737–742. doi: 10.1038/clpt.2011.195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Sanderson C, Lobb EA, Mowll J, Butow PN, McGowan N, Price MA. Signs of post-traumatic stress disorder in caregivers following an expected death: a qualitative study. Palliative Medicine. 2013;27(7):625–631. doi: 10.1177/0269216313483663. [DOI] [PubMed] [Google Scholar]
  62. Schulz R, Beach S. Caregiving as a risk factor for mortality: The caregiving health effects study. Journal of the American Medical Association. 1999;282(23):2215–2219. doi: 10.1001/jama.282.23.2215. [DOI] [PubMed] [Google Scholar]
  63. Sealey M, Breen LJ, O’Connor M, Aoun SM. A scoping review of bereavement risk assessment measures: Implications for palliative care. Palliative Medicine. 2015a;29(7):577–589. doi: 10.1177/0269216315576262. [DOI] [PubMed] [Google Scholar]
  64. Sealey M, O’Connor M, Aoun SM, Breen LJ. Exploring barriers to assessment of bereavement risk in palliative care: Perspectives of key stakeholders. BMC Palliative Care. 2015b;14(49) doi: 10.1186/s12904-015-0046-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Sofaer S. Methodology matters: Qualitative research methods. International Journal for Quality in Health Care. 2002;14(4):329–336. doi: 10.1093/intqhc/14.4.329. [DOI] [PubMed] [Google Scholar]
  66. Stroebe M, Schut H, Stroebe W. Health outcomes of bereavement. Lancet. 2007;370(9603):1960–1973. doi: 10.1016/S0140-6736(07)61816-9. [DOI] [PubMed] [Google Scholar]
  67. Stroebe M, Stroebe W, Schut H. Gender differences in adjustment to bereavement: An empirical and theoretical review. Review of General Psychology. 2001;5:62–83. [Google Scholar]
  68. Thomas K, Hudson P, Trauer T, Remedios C, Clarke D. Risk factors for developing prolonged grief during bereavement in family carers of cancer patients in palliative care: a longitudinal study. Journal of Pain and Symptom Management. 2014;47(3):531–541. doi: 10.1016/j.jpainsymman.2013.05.022. [DOI] [PubMed] [Google Scholar]
  69. Tomarken A, Holland J, Schachter S, Vanderwerker L, Zuckerman E, Nelson C, Coups E, Ramirez PM, Prigerson H. Factors of complicated grief pre-death in caregivers of cancer patients. Psychooncology. 2008;17(2):105–111. doi: 10.1002/pon.1188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. van der Houwen K, Stroebe M, Stoebe W, Schut H, van den Bout J, Wijngaards-de Meij L. Risk factors for bereavement outcome: A multivariate approach. Death Studies. 2010;34:195–220. doi: 10.1080/07481180903559196. [DOI] [PubMed] [Google Scholar]
  71. van Doorn C, Kasl SV, Beery LC, Jacobs SC, Prigeson HG. The influence of marital quality and attachment styles on traumatic grief and depressive symptoms. Journal of Nervous and Mental Disease. 1998;186(9):566–573. doi: 10.1097/00005053-199809000-00008. [DOI] [PubMed] [Google Scholar]
  72. Vanderwerker LC, Laff RE, Kadan-Lottick NS, McColl S, Prigerson HG. Psychiatric disorders and mental health service use among caregivers of advanced cancer patients. Journal of Clinical Oncology. 2005;23(28):6899–6907. doi: 10.1200/JCO.2005.01.370. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Youngblut JM, Brooten D, Cantwell P, del Moral T, Totapally B. Parent health and functioning 13 months after infant or child NICU/PICU death. Pediatrics. 2013;132(5):1295–1301. doi: 10.1542/peds.2013-1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Zisook S, Chentsova-Dutton Y, Shuchter SR. PTSD following bereavement. Annals of Clinical Psychiatry. 1998;10(4):157–168. doi: 10.1023/a:1022342028750. [DOI] [PubMed] [Google Scholar]
  75. Zisook S, Pies R, Iglewicz A. Grief, depression and the DSM-5. Journal of Psychiatric Practice. 2013;19(5):386–396. doi: 10.1097/01.pra.0000435037.91049.2f. [DOI] [PubMed] [Google Scholar]

RESOURCES