Instruments for assessing health professionals’ skills in breaking bad news: protocol for a systematic review of measurement properties

Abstract

Introduction

Health professionals are often involved in the process of breaking bad news (BBN), which remains a difficult challenge, as it requires not only theoretical knowledge, but also the development of humanistic, emotional and communication skills. Therefore, optimal BBN assessment is essential. In this regard, sound measurement instruments are needed to evaluate BBN properly in research, teaching and clinical settings. Several instruments have been designed and validated to assess BBN. In this context, choosing the most appropriate instrument for assessing health professionals’ skills in BBN is essential. The aims of this systematic review are to: (1) identify all the instruments used for assessing health professionals’ skills in BBN; and (2) critically appraise their measurement properties.

Methods

A systematic review will be undertaken according to the most up-to-date COnsensus-based Standards for the selection of health status Measurement INstruments’ (COSMIN) methodology. The protocol of this systematic review was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The search strategy will be performed following the Peer Review of Electronic Search Strategies. The search strategy will be conducted in CINAHL, MEDLINE, Embase, PsycINFO, SciELO and Open Grey. Two review authors will independently appraise the full-text articles according to the COSMIN Risk of Bias checklist. Quality ratings and evidence synthesis will be performed using a modified Grading of Recommendations Assessment, Development and Evaluation approach.

Ethics and dissemination

Ethical approval is not necessary for systematic review protocols. The results will be disseminated by publication in a peer-reviewed journal and presented at a relevant conference.

PROSPERO registration number

CRD42020207586.

Strengths and limitations of this study.

• This will be the most up-to-date systematic review identifying and critically appraising the measurement properties of instruments assessing health professionals’ skills in breaking bad news (BBN) according to the COnsensus-based Standards for the selection of health status Measurement INstruments’ methodology.

• This systematic review protocol was developed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

• A limitation of this systematic review is the inclusion of articles published only in English and Spanish, which may lead to language bias. To reduce potential language bias, studies using translated versions of instruments assessing BBN will be included.

Introduction

Bad news is any information that changes a person’s view of the future in a negative way.^{1 2} Although breaking bad news (BBN) is frequently viewed as a process that is important for physicians working in end-of-life care, most health professionals break bad news on different levels with different types of patient populations.^{3 4} In addition, health professionals are often involved in the process of BBN, but it remains a difficult challenge for them.^{5 6} In this context, different elements from health professions such as a lack of ability in dealing with own feelings as well as the listener’s reaction, or a lack of communication skills make the BBN process difficult.^{7 8} Accordingly, the complexity of communicating bad news lies in numerous factors, such as the health professional’s response to the emotional reactions of patients and their caregivers, who may feel fearful or unable to cope with the situation, which can result in emotional reactions that include shock, anger, sadness, denial, relief and acceptance; as well as managing ones’ own emotional reactions to bad news.⁹ Therefore, delivering BBN requires not only theoretical knowledge, but also the development of a series of humanistic, emotional and communication skills.^{3 7 9} In this regard, focused training in communication skills and techniques to facilitate BBN has been demonstrated to improve patient satisfaction and health professionals comfort.^{10 11}

The assessment of the BBN process is influenced by psychological and emotional factors from both sides of the communication: the patient and family, and the healthcare professional.^{2 3 12} In this regard, BBN is difficult to evaluate and one of the most feared processes for professionals and patients. Although different measures have been developed for evaluating this construct, there is little evidence of assessment of the BBN process, and therefore instruments to measure BBN are needed, as assessment of the BBN process is increasingly used in research, teaching and clinical settings.^{7 12–16} In this context, choosing the most appropriate instrument for assessing BBN is challenging, and the choice should be dependent on the measurement properties and feasibility of the instruments.¹⁷ In this regard, previous systematic reviews on BBN assessment have been performed.^{16 18} In this context, Singh and Agarwal carried out a systematic review which identified and described different protocols for BBN and Johnson and Panagioti conducted a systematic review and meta-analysis on interventions to improve BBN by physicians and medical students. However, although previous systematic reviews on BBN have been performed, no methodologically sound review has been published that contrasts the measurement properties of all the instruments assessing the BBN process according to the most up-to-date COnsensus-based Standards for the selection of health status Measurement INstruments’ (COSMIN) methodology.¹⁹

Therefore, the aim of this systematic review is twofold: (1) to identify all the instruments that have been used to evaluate the BBN process; and (2) to critically appraise the measurement properties of all these instruments.

Methods

Objectives

This systematic review aims to: (1) identify the measures assessing BBN in the healthcare setting; and (2) critically appraise the methodological quality of studies on measurement properties.

Design

The protocol of this study was developed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) and was registered in the International Prospective Register of Systematic Reviews (PROSPERO).²⁰

The systematic review will be carried out according to the most up-to-date COSMIN methodology.¹⁹ In this context, the COSMIN initiative aims to improve the selection of health measurement instruments in research and clinical practice by developing tools and guidelines to select the most appropriate instrument for a given purpose.

Search strategy

Searches will be conducted in MEDLINE, PsycINFO, CINAHL, Cochrane Library, Web of Science, Scopus, COSMIN database of systematic reviews and Open Grey following the Peer Review of Electronic Search Strategies.²¹

The search strategy will include a combination of the following aspects:

• Construct search.

• Population search.

• Instruments search.

• Filter for psychometric properties.

• Exclusion filter.

The exclusion filter will be mainly limited to publication types and subject groups. A search strategy has been proposed (table 1). Moreover, other relevant papers may be identified by checking the reference lists of included articles.

Table 1.

Search strategy

1	“Communicating bad news” [tiab] OR “Breaking bad news” [tiab] OR “Delivering bad news” [tiab] OR “Difficult news” [tiab] OR “Truth disclosure” [MeSH] AND “Communication skills” [tiab] OR “Communication abilities” [tiab]
2	“Health professional” [tiab] OR “Healthcare professional” [tiab] OR “Health workers” [tiab] “Healthcare workers” [MeSH] OR “Health staff” [tiab] OR “Healthcare staff” [tiab] OR “Health personnel” [MeSH] “Healthcare providers” [MeSH]
3	(Instrument[tiab] OR instruments[tiab] OR measure [tiab] OR measures [tiab] OR questionnaire[tiab] OR questionnaires[tiab] OR scale[tiab] OR scales[tiab] OR tool[tiab] OR tools[tiab] OR survey [tiab] OR test [tiab]
4	Instrumentation[sh] OR methods[sh] OR “Validation Studies”[pt] OR “Comparative Study”[pt] OR “psychometrics”[MeSH] OR psychometr*[tiab] OR clinimetr*[tw] OR clinometr*[tw] OR “outcome assessment (health care)”[MeSH] OR “outcome assessment”[tiab] OR “outcome measure*”[tw] OR “observer variation”[MeSH] OR “observer variation”[tiab] OR “Health Status Indicators”[Mesh] OR “reproducibility of results”[MeSH] OR reproducib*[tiab] OR “discriminant analysis”[MeSH] OR reliab*[tiab] OR unreliab*[tiab] OR valid*[tiab] OR coefficient[tiab] OR homogeneity[tiab] OR homogeneous[tiab] OR “internal consistency”[tiab] OR (cronbach*[tiab] AND (alpha[tiab] OR alphas[tiab])) OR (item[tiab] AND (correlation*[tiab] OR selection*[tiab] OR reduction*[tiab])) OR agreement[tiab] OR precision[tiab] OR imprecision[tiab] OR “precise values”[tiab] OR test-retest[tiab] OR (test[tiab] AND retest[tiab]) OR (reliab* [tiab] AND (test[tiab] OR retest[tiab])) OR stability[tiab] OR interrater[tiab] OR inter-rater[tiab] OR intrarater[tiab] OR intra-rater[tiab] OR intertester[tiab] OR inter-tester[tiab] OR intratester[tiab] OR intra-tester[tiab] OR interobserver[tiab] OR inter-observer[tiab] OR intraobserver[tiab] OR intraobserver[tiab] OR intertechnician[tiab] OR inter-technician[tiab] OR intratechnician[tiab] OR intra-technician[tiab] OR interexaminer[tiab] OR inter-examiner[tiab] OR intraexaminer[tiab] OR intra-examiner[tiab] OR interassay[tiab] OR inter-assay[tiab] OR intraassay[tiab] OR intra-assay[tiab] OR interindividual[tiab] OR inter-individual[tiab] OR intraindividual[tiab] OR intra-individual[tiab] OR interparticipant[tiab] OR inter-participant[tiab] OR intraparticipant[tiab] OR intra-participant[tiab] OR kappa[tiab] OR kappa’s[tiab] OR kappas[tiab] OR repeatab*[tiab] OR ((replicab*[tiab] OR repeated[tiab]) AND (measure[tiab] OR measures[tiab] OR findings[tiab] OR result[tiab] OR results[tiab] OR test[tiab] OR tests[tiab])) OR generaliza*[tiab] OR generalisa*[tiab] OR concordance[tiab] OR (intraclass[tiab] AND correlation*[tiab]) OR discriminative[tiab] OR “known group”[tiab] OR “factor analysis”[tiab] OR “factor analyses”[tiab] OR dimension*[tiab] OR subscale*[tiab] OR (multitrait[tiab] AND scaling[tiab] AND (analysis[tiab] OR analyses[tiab])) OR “item discriminant”[tiab] OR “interscale correlation*”[tiab] OR error[tiab] OR errors[tiab] OR “individual variability”[tiab] OR (variability[tiab] AND (analysis[tiab] OR values[tiab])) OR (uncertainty[tiab] AND (measurement[tiab] OR measuring[tiab])) OR “standard error of measurement”[tiab] OR sensitiv*[tiab] OR responsive*[tiab] OR ((minimal[tiab] OR minimally[tiab] OR clinical[tiab] OR clinically[tiab]) AND (important[tiab] OR significant[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR (small*[tiab] AND (real[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR “meaningful change” [tiab] OR “ceiling effect”[tiab] OR “floor effect”[tiab] OR “Item response model”[tiab] OR IRT[tiab] OR Rasch[tiab] OR “Differential item functioning”[tiab] OR DIF[tiab] OR “computer adaptive testing”[tiab] OR “item bank”[tiab] OR “cross-cultural equivalence”[tiab]
5	#1 AND #2 AND #3 AND #4
6	“Protocol” [Publication Type] OR “addresses”[Publication Type] OR “biography”[Publication Type] OR “case reports”[Publication Type] OR “comment”[Publication Type] OR “editorial”[Publication Type] OR “congresses” [Publication Type] OR “consensus development conference”[Publication Type] OR “consensus development conference”[Publication Type] OR “practice guideline”[Publication Type]) OR “suffering from”[tiab] OR “animals”[MeSH]
7	#5 NOT #6
8	FILTER: Language (English and Spanish)

The search strategy will be limited to articles published in English and Spanish.

Inclusion and exclusion criteria

Eligible studies will be those that satisfy the following inclusion criteria: (a) all study designs which used instruments assessing BBN in a healthcare setting; (b) assessing at least one measurement property of an instrument for measuring BBN; (c) published in English or Spanish; (d) all publication years.

Biographies, addresses, editorials, case reports, comments and congresses, conferences, practice guidelines and articles not published in the English or Spanish languages will be excluded.

Outcomes

All measurement properties of the instruments assessing BBN will be considered as outcomes. The COSMIN taxonomy will be adopted for the measurement properties of the instruments.

Study screening and selection

Titles and abstracts will be screened independently by two review authors. All identified references will be entered into Mendeley bibliographic software and recorded duplicates removed. The full article will be retrieved when decisions cannot be made based on the abstract and title alone. Any disagreements between the two reviewers will be resolved by a third reviewer, as necessary.

A PRISMA flow diagram for this systematic review will be provided.

Quality appraisal

To evaluate the quality of each study as a whole, a specific reporting guideline will be used, depending on the study design. The methodological quality of each study for each measurement property will be assessed according to the COSMIN Risk of Bias checklist.

Two review authors will independently appraise the full-text articles (DGS and JPLS) according to the most up-to-date COSMIN Risk of Bias checklist with a four-point scale, and disagreements will be resolved by a third reviewer.²² This tool considers 10 measurement properties. A methodological quality score is provided for the following measurement properties: instrument development; content validity; structural validity; internal consistency; cross-cultural validity/measurement invariance; reliability; measurement error; criterion validity; hypotheses testing for construct validity; and responsiveness. Each measurement property contains 3–35 items and each item has four response options: ‘very good’; ‘adequate’; ‘doubtful’; or ‘inadequate’. The overall methodological quality score of each measurement property is obtained based on the worst score counts’ method.²²

Data extraction

Two researchers will independently participate in the data extraction process, and any disagreements will be resolved by a third reviewer. The following information will be extracted: study author and date of publication; country and language; study population; sample size; setting; instrument description; measurement properties; COSMIN score; and measurement values.

Data synthesis

Data synthesis will comprise three steps. First, the methodological quality of each study that meets the inclusion criteria will be rated using the COSMIN Risk of Bias checklist with quality ratings as either very ‘good’, ‘adequate’, ‘doubtful’ or ‘inadequate’.¹⁹ Second, the result of each study will be rated against the updated criteria for good measurement properties as either ‘sufficient’ (+), ‘insufficient’ (−) or ‘indeterminate’ (?).^{19 23} Third, the evidence for each measurement property will be summarised and the quality of the evidence for each measurement property will be graded as ‘high’, ‘moderate’, ‘low’ or ‘very low’, using the Grading of Recommendations Assessment, Development and Evaluation approach.²⁴ Two researchers will independently participate in the data extraction process, and any disagreements will be resolved by a third reviewer. The results from different studies on measurement properties will be statistically pooled in a meta-analysis. Mean values of statistical analysis on each measurement property (eg, Cronbach’s alpha and intraclass correlation coefficient) will be calculated from pooled data from methodologically similar studies and the results statistically summarised via meta-analysis into forest plots with estimates of heterogeneity (option A). The results of studies will be qualitatively summarised to achieve a result (option B).

Correlation between the two researcher’s assessments will be calculated using Cohen’s kappa. Inter-rater agreement for screening, study selection and data extraction will be calculated with the statistical program SPSS (V.24.0), using Cohen’s kappa coefficient.

Patient and public involvement

Patients and the public were not involved in the design of this protocol.

Ethics and dissemination

No individual data are involved in this study. The results will be disseminated by publication in a peer-reviewed journal and presented at a relevant conference.

Discussion

Although health professionals are often involved in the process of BBN, it remains a significant challenge.^{5 6} The assessment of health professionals’ BBN is increasingly used in research, teaching and clinical settings.^12–16

BBN is an important clinical skill, and its evaluation with validated instruments is crucial.^{13 14} In this regard, the assessment of BBN will allow us to facilitate and improve the process in clinical, teaching and research settings.¹⁶

Several validated questionnaires are now available for assessing the BBN process, and choosing the most appropriate instrument for assessing BBN is essential.^{13 14 17} In this context, the main aim of this systematic review is to identify and appraise all the instruments assessing BBN and their measurement properties.

This systematic review aims to provide a comprehensive assessment of current evidence of instruments assessing the BBN process. In addition, assessing health professionals’ skills in BBN may result in the improvement of the BBN process, increased patient satisfaction and in health professionals feeling more comfortable with treatment decisions.

Strengths and limitations

To the best of our knowledge, this will be the most up-to-date systematic review of instruments assessing BBN in the healthcare setting. The results of this study will inform clinicians and researchers on the quality and the appropriateness for different populations of the instruments assessing BBN.

This review will be carried out following the COSMIN statement. In this context, COSMIN is one of the most widespread and comprehensive tools to critically appraise the measurement properties of existing outcome measures’ instruments, and for selecting the most suitable instrument.^{19 22}

Although a meticulous search was conducted in several databases, it is possible that not all the instruments available for the measurement of BBN will be identified. However, the sensitive search strategy that was specifically designed to identify instruments for assessing BBN, and the double independent review process (ie, study screening and selection of eligible studies) will minimise this problem.

Ethics statements

Patient consent for publication

Not required.