The importance and complexity of regret in the measurement of ‘good’ decisions: a systematic review and a content analysis of existing assessment instruments

Abstract

Background or context  Regret is a common consequence of decisions, including those decisions related to individuals’ health. Several assessment instruments have been developed that attempt to measure decision regret. However, recent research has highlighted the complexity of regret. Given its relevance to shared decision making, it is important to understand its conceptualization and the instruments used to measure it.

Objectives  To review current conceptions of regret. To systematically identify instruments used to measure decision regret and assess whether they capture recent conceptualizations of regret.

Search strategy  Five electronic databases were searched in 2008. Search strategies used a combination of MeSH terms (or database equivalent) and free text searching under the following key headings: ‘Decision’ and ‘regret’ and ‘measurement’. Follow‐up manual searches were also performed.

Inclusion criteria  Articles were included if they reported the development and psychometric testing of an instrument designed to measure decision regret, or the use of a previously developed and tested instrument.

Main results  Thirty‐two articles were included: 10 report the development and validation of an instrument that measures decision regret and 22 report the use of a previously developed and tested instrument. Content analysis found that existing instruments for the measurement of regret do not capture current conceptualizations of regret and they do not enable the construct of regret to be measured comprehensively.

Conclusions  Existing instrumentation requires further development. There is also a need to clarify the purpose for using regret assessment instruments as this will, and should, focus their future application.

Introduction

As the philosopher Amiel said, ‘Accept life, and you must accept regret’ (Henri‐Frédéric Amiel, 1821–1881). This quote conveys the omnipresent and inevitable nature of regret. It further casts doubt on whether we can truly achieve the idealization that Edith Piaf famously sang about: living a life with ‘no regrets’. Svenson’s Differentiation and Consolidation theory of decision making suggests that two common goals of all decisions are to limit cognitive dissonance and the potential for regret, achieved through the cognitive process of differentiating decision options from each other. ¹ , ² Regret’s inextricable link with decision making, ³ thus its relevance to shared decision making, has prompted the current review. If one of the key goals of decision making is to reduce regret, it is important to understand its conceptualization, what might be done to limit it, and the instruments used to measure it.

Whether large or small, decisions pervade our daily lives. Decisions are made in varying contexts, ranging from the workplace to the home. People decide whether to continue in education, undergo a risky surgical procedure, what Government policy to implement, or simply what breakfast to eat. In doing so, they typically choose between alternative courses of action, or between action and inaction.

An increasing emphasis on patient participation in decision making has meant that people are increasingly required to make, or at least be involved in, decisions about their individual health care. Decision support interventions, ⁴ , ⁵ or decision aids, have been designed to facilitate this involvement, and their use has proven beneficial in terms of increasing knowledge, satisfaction, confidence in making the right choices, and reducing anxiety. ⁶ However, despite the efficacy of these interventions, the fact remains that some decisions can lead to regret. Cancer related decisions, such as treatment for prostate cancer or early breast cancer, are often made in a context of uncertainty and ‘equipoise’, ⁷ especially when there is no clearly preferable clinical option: a context for which decision aids have been developed. ⁶ As a result, several studies have shown that regret is a common consequence of ‘preference sensitive’ cancer‐related decisions. ⁸ , ⁹ , ¹⁰ , ¹¹

Patient involvement in decision making has also raised issues about ‘decision quality’; ¹² , ¹³ specifically, the means by which we assess whether someone has made a ‘good’ decision. Trials have used several measurement instruments, including regret, in an attempt to determine the effect of decision support interventions on the quality of decision making. More recently, however, concern has been expressed over the use of decision quality measures that focus on post‐decisional outcomes. ¹⁴ If researchers are to use decision regret as a proxy measure for ‘good’ decision making, it is important we understand how the concept is measured, and assess whether current instruments are suitable for this purpose.

Regret has attracted attention from researchers across several fields of investigation, and it is not restricted to research on patient decision making in health and medicine. ¹⁵ , ¹⁶ Regret research spans several contexts including business and economics, ¹⁷ law, ¹⁸ and neuroscience. ¹⁹ Research has also led to the development of several instruments that measure regret. ⁹ , ¹⁵ , ²⁰ As we discuss below, however, the conceptualization of regret is far from simple, and existing measures fail to capture the multiplicity and complexity of the concept highlighted by current research. ³ As such, there are two distinct components to this review. Section A provides an overview of current conceptualizations of regret. This section concludes with a model of regret, representing the aspects of regret that we consider as relevant and important to its measurement. Section B describes the systematic review and the evaluation of each instrument, in relation to the recent conceptualizations of regret. The review concludes with a synthesis and a discussion of whether the instruments are sufficiently robust for continuing research about regret.

This paper has three aims:

• 1 To review current conceptions of regret.
• 2 To conduct a systematic review to identify instruments that have been developed to measure regret.
• 3 To evaluate the instruments to assess whether they capture current conceptions of regret.

Section A – conceptual overview of regret

Conceptualizations of regret

Generally, regret is defined as an aversive emotion that we experience when we realize or imagine that our current situation would be more favourable if we had chosen differently. ³ Counterfactuals are mental representations of alternative versions of the past. ²¹ As such, regret is defined as a counterfactual emotion due to the accompanying cognitive processes of comparing current outcomes to what ‘might have’, ‘should have’, or ‘could have’ been. ³ , ²² Regret is therefore a backward looking, affectively unpleasant, self‐focused emotion which tends to direct comparisons upward rather than downward. ²³ This means that individuals tend to focus on how the past could have been better than it was, rather than how it could have been worse. Regret also has an element of responsibility, or self‐blame, and this feature differentiates it from other negative emotions, such as disappointment. ³

In the period following a decision, internal subjective processes continue to review the choice made. ² Research has highlighted this natural tendency to focus on the foregone as opposed to the acquired, ²⁴ and has even linked regret to neurological functioning. ¹⁹ This tendency, combined with the innumerable decisions an individual makes on a daily basis, provides many opportunities for regret to be experienced, or indeed anticipated. Regret is the second most frequently cited emotion, ²⁵ after anxiety, which suggests that regret is a common consequence of decision making. In this sense, regret is conceptualized as a prototypical decision‐related emotion, ³ due to the inextricable link between decisions and regret: regret only occurs in the context of decision making.

A theory of regret

Two key observations that arise from the regret literature are that the research spans many fields of investigation and its conceptual boundaries are far from clear. The lack of consensus regarding its conceptualization, combined with the vast number of research studies from different fields, has prompted a recent attempt by Zeelenberg and Pieters to summarize the large number of findings and integrate them into their ‘Theory of Regret Regulation 1.0’. ³

The theory puts forward ten distinct propositions on regret that capture up‐to‐date research, most of which are relevant to its measurement. The theory acknowledges the functional nature of regret, believing a pragmatic approach is essential to understand the experience of regret. According to the theory, regret can be experienced about past (retrospective) and future (prospective) decisions. Regret can also result from both action and inaction, and in this sense, regret is a temporally bound emotion. Gilovich and Medvec ²⁶ proposed a temporal pattern to the experience of regret, whereby commissions, or actions, lead to greater regret in the short term and omissions, or inactions, lead to greater regret in the long term. Kahneman ²⁷ disagreed, proposing that long‐term regrets regarding inaction are wistful, and thus they are not troublesome. Recent accounts, however, acknowledge the wistful nature of inaction regrets, whilst maintaining that regrets of inaction can sometimes be troublesome. ²⁸

Whilst the theory and previous research highlights the temporal pattern of regrets concerning action and inaction, we propose that the temporal dimension of experienced regret may have been overlooked. We believe that there is no reason to presume that experienced regret will remain static from the point of initial experience and it is likely to follow a temporal pattern. What an individual experiences 1 month after making a decision probably does not reflect how they feel 12 months after making a decision.

For instance, if a cancer patient has made the decision to undergo a specific form of treatment, and the treatment follows an unpleasant course, the patient may regret both the option and the outcome of their decision. However, if 12 months after making the decision, there are no signs of the cancer returning, the patient may no longer regret the option and the outcome of the chosen decision alternative. We therefore propose that the concept of experienced regret should be divided into immediate and delayed regret. Longitudinal and prospective research is needed to examine the emergence of regret and its subsequent course.

Multiplicity of regret types

Conceptualizations of regret are further complicated by the multiplicity of regret types. Although previous work tended to treat regret as a single construct, Zeelenberg and Pieters ³ , ²⁹ point out that regret is not a unified construct, but rather it can be understood at multiple levels. Specifically, regret can be experienced about decision processes and decision outcomes. Connolly and Reb ³⁰ have discussed the multiplicity of regret in further detail by highlighting three types of regret in the decision‐making context which can be distinguished by their targets: process, option and outcome regret. Process regret involves feelings of self‐blame for an unjustified decision process, for example, failing to seek information on all available options before making a decision. Option regret simply involves regret about the decision alternative chosen, and outcome regret involves a comparative evaluation, whereby one regrets that the outcome is poorer than the counterfactual outcome (see Box 1 for an example of how the various types of regret can be experienced). Significantly, they propose that the various forms of regret can be experienced alongside each other or independently. Importantly, they can also be anticipated (prospective) or experienced (retrospective).

Table Box 1.

 The various types of regret

To exemplify the various types of regret, imagine a woman faced with the decision to have a lumpectomy or mastectomy for the treatment of early stage breast cancer. She may experience process regret if she does not make an informed decision, makes a hasty decision, and she may experience role regret if she has adopted a passive role with which she is not comfortable or satisfied. However, if she decides to have lumpectomy over mastectomy, and is subsequently happy with the aesthetic results, it is unlikely that she will experience option regret, regardless of the process leading up to the decision. However, if she experiences recurrence of the cancer, she may subsequently experience outcome regret.

Other research has highlighted dissociation between decision regret and role regret. ³¹ In the latter case, individuals might regret the role they played in the decision making process, without actually regretting the decision alternative chosen. It seems reasonable to suggest that role regret and process regret are linearly related. If an individual adopts a passive role, which he or she later comes to regret, this might impact on the processes leading to the decision. For instance, adopting a passive role is likely to mean that an individual does not seek to inform themselves of the options available, thereby creating a deficit in the decision process. Likewise, if another individual heavily influences the decision process, this is likely to mean that the individual adopts a more passive role. Both scenarios could subsequently give rise to role and process regret.

Positive and negative outcomes

Whilst regret is primarily viewed as a negative emotion that can result in emotional distress, it may also be positive, or functional, and this further complicates its conceptualization. Zeelenberg and Pieters ¹ point out that regret is not only an affective reaction, but also a powerful force in motivating future behaviour and shaping future decisions. Research has shown that both anticipated ³² and experienced ³³ regret can influence future decision making. The study by Zeelenberg and Beattie ³³ indicated that those who anticipate or experience regret subsequently engage in emotion or regret management. As such, individuals will behave in such a way that the regret they have experienced or anticipated will disappear. Additionally, future decisions will be determined by those options that minimize the experience of regret. This suggests that whilst the experience of regret is generally aversive, its ability to help us to make better decisions and to learn from past mistakes also makes it beneficial. The positive value of regret is supported by research into people’s perceptions of regret. ³⁴ Saffrey et al. found that people value regret significantly more than other negative emotions, and people make positive evaluations of regret. Therefore, the experience of regret can be classified as either positive or negative.

A model of regret

Overall, the Theory of Regret Regulation 1.0 ³ highlights the complexity of regret and provides a sophisticated conceptualization of a multifaceted emotion. In the light of recent conceptualizations of regret, and the theory ³ in particular, we produced a model (see Fig. 1) that incorporates those aspects of regret that we feel are relevant to its measurement. First, the model acknowledges that regret can result from action or inaction, and can be experienced or anticipated. Second, the model distinguishes between the different targets of regret (process, option and outcome), which can be anticipated or experienced either independently or in conjunction with each other. Third, the model acknowledges that both positive and negative outcomes can result from the different targets of regret. Fourth, the model is temporal: this is intended to represent the temporal nature of the stages of regret, whether that is anticipated regret, followed by the decision, followed by experienced regret, or the shift from process, to option, to outcome regret.

Figure 1.

 A model representing the aspects of regret important to its measurement.

These elements reflect ideas put forward by the Theory of Regret Regulation 1.0 ³ and Connoly and Reb. ³⁵ In addition, we have incorporated immediate and delayed regret; the idea that experienced regret does not remain static at the initial point of experience, and follows a temporal pattern that is dynamic in nature. This idea has not been addressed in previous models (cf. Zeelenberg and Pieters ³ ). The elements of the model provide a framework in which research instruments are used to evaluate regret. Some elements of the model are specific to context: whether regret is measured after action or inaction, whether anticipated or experienced regret is measured, and whether the instrument is used in cross‐sectional or longitudinal research. Other elements relate to the instruments ability to measure both positive and negative outcomes and the different target independently or, where relevant, in combination.

Section B – systematic review and evaluation of instruments

Various instruments have been developed to measure both anticipated ³⁶ , ³⁷ and experienced regret. ¹⁵ , ²⁰ , ³⁶ , ³⁸ Researchers in health care have measured regret in several contexts, including men who have been treated for localized prostate cancer, ³⁹ women following breast reconstruction, ⁴⁰ and women following sterilization. ¹⁶ Although research incorporating current instruments has made important contributions to the understanding of regret, the results may be confounded by problems at the conceptual level. As mentioned, regret is a complex emotion that has many elements. It can be understood at multiple levels, it is temporally bound, and it can result in positive (functional) and negative outcomes.

In a brief analysis, it appears that current instruments may fail to capture these elements of regret, and thus they may not be sufficiently robust to reflect recent conceptualizations of regret. Some authors have already highlighted discrepancies between the conceptual definition of decision regret and the outcome measure used to assess this construct. ⁴¹ If constructs and terminology are unclear, this will hinder the interpretation of empirical results. Even if statistically significant associations are found, one cannot be sure that the constructs measured are actually the same and interpretation of results will remain unclear at the conceptual level. ⁴¹ An understanding of the relationship between specific variables and decision regret is needed, not only for improved conceptualizations, but also in order to target interventions to reduce the negative outcomes of regret. An investigation of this relationship, however, is dependent on the availability of valid and reliable measures. As a review of this nature has not been carried out previously, we aimed to search literature from several fields of investigation to ensure that we have a broad understanding of the concept and the instruments that are used to measure it.

Methods

Search strategy

To address our second aim of identifying existing instruments that measure decision regret, we conducted a systematic review. The search strategies were devised in collaboration with an information specialist to identify studies that reported the development or use of validated instruments that measure decision regret. Five electronic databases were searched, with no language restrictions: MEDLINE (1960–2008), EMBASE (1980–2008), PsycINFO (1960–2008), ASSIA (1960–2008) and Web of Science (1981–2008). The latter two databases were suitable to cover humanities, business, economic, and social science literature. Search strategies used a combination of MeSH terms (or database equivalent) and free text searching. Full details of search strategies are available from the first author on request (see Appendix 1 for Medline search strategy). Search outputs from each electronic database were downloaded and merged into Endnote (version V.02), and duplicates were removed. Follow‐up searches included manual searches of the most frequently cited publications and examination of reference lists of included papers.

Inclusion criteria

Articles were included if they reported the development and psychometric testing of an instrument designed to measure decision regret, or the use of a previously developed and tested instrument. Multi‐item instruments that quantitatively measure regret were included. Instruments were included if they reported at least two of the agreed psychometric evaluations for confirming the validity and reliability of measurement instruments. ⁴² Articles that lacked data regarding the development and testing of the instrument were excluded, as were articles reporting instruments that were unavailable for scrutiny (after contacting the authors). As study designs vary widely across disciplines (e.g. health, economics, psychology) we did not exclude articles based on study design. Articles relating to multi‐dimensional instruments (e.g. regret and disappointment) were considered, provided separate psychometric data were provided for the regret sub‐scale. Articles that report use of a modified version of a previously developed and tested scale were included, provided further psychometric data or justifications were available.

Data extraction

Relevant information for extraction was agreed by NJ‐W, GE and AE and data extraction forms were piloted. Data were extracted under the following headings for articles reporting the initial development and validation of an instrument (development studies) or reporting the use of a previously developed instrument (evaluation studies): instrument and study characteristics (name, first author, year, publication details, country, language, context, participants, design, whether original or modified instrument was used, and description of instrument); instrument development process; theoretical base/conceptual framework; reported reliability; reported validity, including face, content, construct (convergent and discriminant) and criterion.

Content evaluation of instruments

Our third aim was to assess whether included instruments capture recent conceptualizations of regret and are valid for use in research that seeks to improve decision making and decisions, and minimize regret. Articles that reported the initial development and validation of an instrument were included for the content evaluation. Data extracted to assess the content validity of the instrument were based on Fig. 1. We assessed: (i) whether the instrument addressed process/role regret, option regret or outcome regret, (ii) whether the instrument was able to measure different targets of regret separately, and (iii) whether the instrument was able to measure if regret resulted in positive and/or negative outcomes. In addition to this, we noted whether the instrument was measuring anticipated or experienced regret, or both, and whether the instrument had been applied cross‐sectionally and/or longitudinally.

Results

Thirty‐two articles were included which described 10 instruments. Figure 2 describes the progress through the systematic review, including details of database search outputs, the stages at which articles identified were excluded or underwent data extraction, and the articles included. Of the 32 articles included, 10 report the development and validation of an instrument that measures decision regret and 22 report the use of a previously developed and tested instrument.

Figure 2.

 A flowchart of progress through the systematic review.

The instruments

The included instruments were: the Anticipated Regret Questionnaire; ³⁷ the Anticipated Regret Scale; ³⁶ the Decision Regret Scale; ¹⁵ the Experienced Regret Scale; ⁴³ the Regret and Disappointment Scale; ⁴⁴ the Regret Experience Measure; ²⁰ two Regret scales; ⁸ , ³⁸ the Regret and Maximization Scale; ⁴⁵ the Regret Measurement. ⁴⁶ , Table 1 summarizes instrument details, study details, instrument development, and psychometric properties of the included instruments.

Table 1.

 Instrument details, study details, and instrument characteristics and psychometric data

Instrument and study details				Instrument development and psychometric properties
Instrument name	First author, reference (year, country)	Version of scale used/context/temporal application	Description of instrument	Instrument development and item selection	Validity	Reliability
Anticipated Regret Questionnaire 37	Development: Godin 37 (2005, Canada) Evaluation: McMahon 47 (2008, Ireland)	Original (Godin) 37 /Health and Medicine/Longitudinal 37 Cross‐sectional 47	Original: 3‐item 5‐point rating scale (1 = very unlikely to 5 = very likely). Rate agreement with statement ‘if I did not give blood’: Item 1: I will regret it; Item 2: it would bother me; Item 3: I would be disappointed.	Regret definition: no specific definition provided. Used ‘Etic‐Emic’ approach: item to measure theoretical construct (etic) based on information obtained from population under study (emic). 1st version piloted and submitted for validity and reliability testing.	No data available	Internal consistency: Cronbach’s α = 0.87 2 week test‐retest reliability = 0.58 37 Cronbach’s α = 0.91 47
Anticipated Regret Scale 36	Development: Sheeran 36 (1999, UK)	Original (Sheeran) 36 /Business and Economics/Cross‐sectional	Original: 2‐item 5‐point rating scale (1 = strongly agree to 5 = strongly disagree). Rate agreement with statements using scale. Item1: If I missed playing the national lottery for 1 week I would regret it; Item 2: Not playing the national lottery for 1 week would upset me. Introduced different time periods in items and action version of instrument.	Regret definition: based on economic theories 48 , 49 . People compare outcomes of decisions. A person will experience (or anticipate) regret when the outcome of alternative is (or anticipated to be) more favourable than the outcome of the option chosen. Regret items part of larger questionnaire measuring constructs of the Theory of Planned Behavior 50 . No further information about development of the instrument or selection of items.	Discriminant and convergent validity demonstrated in 3 studies. Regret items load high on single factor with no cross‐loadings.	Internal consistency: Cronbach’s α = 0.85 (Study 1); α = 0.96 (Study 2); α = 0.83 (Study 3‐inaction scenario) and α = 0.94 (action scenario)
	Evaluation: Abraham 51 (2003, UK) Sheeran 52 (2003, UK)	Original (Sheeran) 36 /Psychology and Health/Cross‐sectional 51 Longitudinal 52	Same items as Original with different anchors (1 = definitely yes to 7 = definitely no)	N/A	Discriminant and convergent validity. Factor analysis revealed 2 factors. Regret items load high on a single factor with no cross‐loadings. 51	Internal consistency: Cronbach’s α = 0.82 (study 1); α = 0.63 (study 2) 51 Cronbach’s α = 0.68 52
	Rapaport 53 (2000, UK)	Modified (Rapaport) 53 /Psychology/Cross‐sectional	Modified: Additional item added to original instrument. 3‐item 7‐point rating scale. Item 1: 1 = strongly agree to 7 = strongly disagree. Item 2: 1 = regret to 7 = no regret. Item 3: 1 = upset to 7 = not upset).	Authors do not provide information or justification for the inclusion of another item.	Discriminant and convergent validity. Regret items load high on single factor with no cross‐loadings.	Internal consistency: Cronbach’s α = 0.82
Decision Regret Scale 15	Development: Brehaut 15 (2003, Canada) Evaluation: Davison 54 (2003, Canada); Davison 39 (2007, Canada); Feldman‐Stewart 55 (2004, Canada); Goel 56 (2001, Canada); Sheehan 40 (2007, Australia); Wakefield 57 (2008, Australia)	Original (Brehaut) 15 /Health & Medicine/Cross‐sectional	Original: 5‐item 5‐point scale (1 = strongly agree to 5 = strongly disagree). User reflects on relevant healthcare decision and rates statements using scale. Item 1: It was the right decision; Item 2: I regret the choice that was made; Item 3: I would go for the same choice if I had to do it over again; Item 4: The choice did me a lot of harm; Item 5: The decision was a wise one.	Regret definition: ‘Remorse of regret over a (health care) decision’ [15, p. 283]; ‘aspect of regret stemming from the knowledge that the choice made was non‐optimal…’[15, p. 282]; regret concerning a poor outcome. Agreed the definition of regret and developed pilot version. 3 items consistent with previous instruments and 2 new items. Iterative process of editing items and informal pilot testing. Several items dropped (no further information).	Assessed in 4 contexts. Construct validity – regret over poor outcome construct of interest. Found significant correlations between increased regret and poorer physical/psychological outcomes and quality of life. Convergent validity – found significant correlation between increased regret and decreased satisfaction with decision/doctor’s visit and increased decisional conflict 15	Internal consistency in 4 contexts 15 : (i) Hormone replacement therapy decision, Cronbach’s α = 0.92; (ii) Breast cancer therapy decision, α = 0.84; (iii) Breast cancer surgery decision, α = 0.86; (iv) Prostate cancer treatment decision, α = 0.81 Cronbach’s α = 0.83 54 Cronbach’s α = 0.91 39 Cronbach’s α = 0.90 40
	Diefenbach 58 (2007, USA)	Modified (Diefenbach) 58 /Health and Medicine/Longitudinal	Modified version 1: Selected 2 items from original instrument. Used different anchors. 2‐item 5 point rating scale (1 = not at all to 5 = very much).	Removed 3 items to reduce subjects’ burden. No further information.	Convergent validity – found significant correlation between regret and satisfaction with treatment decision and with decision conflict.	Internal consistency: Cronbach’s α = 0.47 (6 months); Cronbach’s α = 0.46 (12 months). Test‐retest reliability (6 months and 12 months): r = 0.53
	Diefenbach 59 (2008, USA)	Modified (Diefenbach) 59 /Health and Medicine/Longitudinal	Modified version 2: Selected 3 items from original instrument. Used different anchors. 3‐item 5 point rating scale (1 = not at all to 5 = very much).	Removed 2 items to reduce subjects’ burden. No further information.	PCA confirmed unidimensionality of 3 items.	Internal consistency: Cronbach’s α = 0.58
Experienced Regret Scale 43	Development: Keaveney 43 (2007, USA)	Original (Keaveney) 43 /Marketing/Cross‐sectional	Original: 3‐item 7‐point rating scale (1 = do not regret at all to 7 = regret very much). User reflects on specific decision and rates agreement with statements using scale. Item 1: How much happier would you have been, if you had made a different decision?; Item 2: How annoyed do you feel that you made a wrong decision?; Item 3: How much would you regret your decision to buy this vehicle?	Regret definition: regret is experienced when we would change an action we took (failed to take) as we can imagine that the outcome of the alternative would be preferable 60 Regret items part of larger questionnaire measuring pre‐purchase and post‐purchase stages of buyer decision‐making process. No further information about development of the instrument or selection of items. State they use items used in previous research.	Construct validity – Confirmatory factor analysis showed 21 items (3 regret) loaded highly on their corresponding factors. Convergent validity –t values on factor loadings ranged 15.80–16.28. Also examined with factor reliability. Average variance extracted = 0.64.	Internal consistency: Cronbach’s α = 0.95
Regret & Disappointment Scale 44	Development: Marcatto 44 (2008, Italy)	Original (Marcatto) 44 /Psychology/Cross‐sectional	Original: 2‐item 7‐point rating scale (1 = not at all/statement not pertinent to 7 = totally agree). Administered with regret scenario. User rates agreement with statements using scale. Item 1: I wish I had made a different choice; Item 2: I feel responsible for what happened to me. Also given 3rd item asking user to choose between 1 of 2 counterfactuals (regret or disappointment).	Regret definition: regret results from our ability to compare ‘what is’ with ‘what might have been’ & (unlike disappointment) has strong association with a feeling of responsibility 19 Regret items form part of a larger 7‐item instrument that also measures disappointment. Regret items selected from main cognitive antecedents of regret based on Camille’s definition 19	Construct validity – Study 1 – PCA confirmed 2‐factor structure (regret & disappointment). Significant interaction between regret scenario used and regret items in instrument. Study 2 – PCA confirmed 2‐factor structure from study 1. Binary logistic regression found Increased regret score was associated with choice of regret counterfactual. Study 3 – confirmed significant interaction from study 1 and binary logistic regression in study 2. Convergent validity – Significant correlation with the ‘Regret Scale’ 45	Internal consistency: Cronbach’s α = 0.64 (study 1)
Regret Experience Measure 20	Development: Creyer 20 (1999, Australia) Evaluation: Lin 62 (2006, Taiwan)	Original (Creyer) 20 /Marketing/Cross‐sectional	Original: 8‐item 7‐point rating scale (1 = disagree completely to 7 =  agree completely). Item 1: I regret my choice; Item 2: I think I made an error in judgment; Item 3: Before I received outcome feedback, I knew I had made an excellent decision; Item 4: I am confident I made the best choice based on the information I had available; Item 5: Before I should have chosen differently; Item 6: I knew that I should have chosen differently; Item 7: I feel really good about my choice.	Regret definition: general feeling that we would change our decision, if given the opportunity, a belief an error was made at the time of the decision, & sense of self‐recrimination. 49 , 61 Pool of 9 items generated that addressed 3 aspects of regret. Factor analysis revealed 1 factor structure. 1 item with low item to total correlation dropped. Factor analysis of 8 items revealed high positive correlations with the first factor.	Construct validity – Significant correlation between increased regret and increasingly negative feedback about outcomes (study 1 & 2). Factor analysis revealed high positive correlations with 1 factor (study 1 & 2). 20	Internal consistency: Cronbach’s α = 0.85 (study 1); α = 0.85 (study 2) 20 Unclear if evaluation study 62 reports new psychometric data or data from original study.
Regret Scale 38	Development: Clark 38 (1997, USA)	Original (Clark) 38 /Health and Medicine/Cross‐sectional	Original: 3‐item rating scale. Item 1: 6 point rating scale (1 = all of the time to 6 = none of the time). Have you wished that you could change your mind about the kind of treatment you chose for you prostate cancer? Item 2 & 3: 5 point rating scale (1 = definitely true to 5 = definitely false). Item 2: feel that I would be better off if I had chosen the other treatment for prostate cancer. Item 3: It bothers me that other men with prostate cancer get treatment that Is very different from what I have received.	Regret definition: uncertainty over choice made, a wish to reverse the decision, & feeling that one would be better off if the other option was chosen. Regret items form part of a larger 9‐scale instrument that addresses indicators of quality of life of men with metastatic prostate cancer. Focus group transcripts analysed to identify themes of patients’ experiences after treatment. 65 candidate items developed piloted. PCA performed on each theme. Items with low communality & those that cross‐loaded were deleted. PCA identified 9 dimensions of quality of life, including regret.	Construct validity – PCA confirmed regret items load on 1 factor. Regret scale able to discriminate between 2 groups. Regret scores lower (worse outcome) in patients who reported many symptoms. Discriminant validity – Significant correlation between regret scale and the other sub‐scales suggest it measure a different quality of life.	Internal consistency: Cronbach’s α = 0.80
	Evaluation: Clark 9 (2001, USA) Hu 10 (2003, USA)	Modified (Clark) 9 /Health and Medicine/Cross‐sectional	Modified: Selected 2 items from Original instrument. Used Item 1 & 2. Used same scaling and anchors.	Item 3 that defined the internally consistent original instrument added little information and was removed.	No data available	No data available
Regret Scale 8	Development: Clark 8 (2003, USA) Evaluation: Clark 63 (2003, USA)	Original (Clark) 8 /Health and Medicine/Cross‐sectional	Original: 5‐item instrument. No information about item scaling or anchors. Users rates statements using scale. Item 1: I wonder if I would have been better off with a different treatment; Item 2: I sometimes wonder whether it was really worthwhile being treated at all; Item 3: I sometimes feel that the treatment I had was the wrong one for me; Item 4: If I had to do it over, I would choose some other treatment; Item 5: I sometimes wish I could change my mind about the kind of treatment I chose for my prostate cancer.	Regret definition: a feeling of having made the wrong treatment choice, persistent doubt, & the wish to change the decision. Regret items form part of a larger 11‐scale instrument that addresses indicators of quality of life of men with early stage prostate cancer. Focus group transcripts analysed to identify themes of patients’ experiences after treatment. 84 candidate items developed piloted. PCA performed on each theme. Items with low communality & those that cross‐loaded were deleted. Psychometric analyses defined 11 scales, including regret.	Discriminant validity – Correlations between the 11 scales were consistent with pattern of item‐scale discrimination. 8	Internal consistency: Cronbach’s α = 0.81 8
Regret and Maximization Scale 45	Development: Schwartz 45 (2002, USA) Evaluation: Parker 64 (2007, USA) Saffrey 34 (2008, Canada)	Original (Schwartz) 45 /Psychology/Cross‐sectional	Original: 5‐item 7‐point rating scale (1 = completely disagree to 7 = completely agree). Users rate agreements with statements using scale. Item 1: Whenever I make a choice, I’m curious about what would have happened if I had chosen differently. Item 2: Whenever I make a choice, I try to get information about how the alternatives turned out. Item 3: If I make a choice and it turns out well, I still feel like something of a failure if I find out that another choice would have turned out better; Item 4: When I think about how I am doing in life, I often assess opportunities I have passed up; Item 5: Once I make a decision, I don’t look back (reversed).	Regret definition: no specific definition provided but state regret may result from non‐optimal choice. Regret items from part of a larger 18‐item instrument that also measures maximization. Preliminary 42‐item instrument (9 items measuring regret) created. On basis of reliability and face validity, instrument reduced to 22 items (5 regret items). Presented to 11 judges. 4 regret items judged by 10/11 to be about regret and 1 item judged by 9/11 to be about regret. 22 items (5 regret) submitted to a PCA.	Construct validity – Regret items loaded highly on single factor. Convergent validity – Significant positive correlation between regret and maximization. 45 Convergent validity – Significant positive correlation between regret and maximization 64	Internal consistency: Cronbach’s α = 0.67 45 Cronbach’s α = 0.76 64 Cronbach’s α = 0.67 (when using scale on self); α = 0.77 (when using scale for other). 34
Regret Measurement 46	Development: Tsiros 46 (1998, USA) Evaluation: Huang 65 (2007, China) Hung 66 (2007, China)	Original (Tsiros) 46 /Business and Economics/Cross‐sectional	Original: 2‐item 7‐point rating scale (−3 = strongly disagree to 3 = strongly agree). Administered with hypothetical scenario. User rates agreement with statements using scale. Item 1: I feel sorry for having chosen company ALPHA; Item 2: I feel regretful for having chosen company ALPHA.	Regret definition: regret is the unpleasant feeling of finding out that the alternative option would have led to better outcomes 48 , 49 Regret items form part of a larger questionnaire that also measures satisfaction. Limited information about item selection. State 7‐point rating scales were used on the basis of reported literature. Subjected to reliability and validity testing to determine final instrument.	Construct validity – 2 regret items loaded highly on a single factor. Discriminant and Convergent validity – demonstrated using a correlation matrix of individual items 46 Content validity confirmed by authors. Convergent and Discriminant validity – regret items load high on different factor to satisfaction. 66	Internal consistency: Cronbach’s α = 0.82 46 Cronbach’s α = 0.86 65 Cronbach’s α = 0.85 (study 1); α = 0.74 (study 2) 66
	Pierro 67 (2008, Italy)	Modified (Pierro) 67 /Marketing/Cross‐sectional	Modified version 1: Selected 2 items from previously modified instrument 64 and added additional item. 3‐item 7‐point rating scale. Items 1 & 2: 1 = not at all to 7 = very much. Item 3: 1 = not at all – 7 = definitely.	Removed item 3 from previously modified scale. 68 No information about removal and selection of new item.	Construct validity – Found significant difference between difference groups (study 1 & 2)	Internal consistency: Cronbach’s α = 0.85 (study 1); α = 0.74 (study 2)
	Tsiros 68 (2000, USA)	Modified (Tsiros) 68 /Marketing / Cross‐sectional	Modified version 2: Additional item added to original instrument. 3‐item 7‐point rating scale (1 = strongly agree to 7 = strongly disagree).	Regret items form part of a larger questionnaire that also measures satisfaction. Used 2 items from original instrument. Used 1 additional item from another un‐validated instrument. Selected item as it addressed regret. Other items excluded as they addressed cognitive dissonance and re‐purchase intentions.	Construct validity – Factor analysis revealed 2 factors. Regret items loaded high on regret factor and low on satisfaction factor, and vice versus. Discriminant validity – Examined dimensionality of 6 items (3 regret, 3 satisfaction). Confirmatory factor analysis revealed 2‐factor solution better than 1.	Internal consistency: Cronbach’s α = 0.82

The earliest included measure was published in 1997, ³⁸ indicating the relatively recent interest in measurement, and the most recent in 2008. ⁴⁴ Eight of the 10 instruments were designed to measure experienced regret ⁸ , ¹⁵ , ²⁰ , ³⁸ , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ and two were designed to measure anticipated regret. ³⁶ , ³⁷ All instruments have been used in cross‐sectional studies and two instruments have been used longitudinally. ¹⁵ , ³⁷ Four instruments were developed and tested in a health research context, ⁸ , ¹⁵ , ³⁷ , ³⁸ two in a psychology context, ⁴⁴ , ⁴⁵ and four in a marketing, business or economics context. ²⁰ , ³⁶ , ⁴³ , ⁴⁶ Two instruments contained items that were specific only to the measurement of regret, ¹⁵ , ²⁰ and eight instruments contained items relevant to the measurement of regret within a larger questionnaire. ⁸ , ³⁶ , ³⁷ , ³⁸ , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ Instruments were developed in Australia, ²⁰ Canada, ¹⁵ , ³⁷ Italy, ⁴⁴ UK, ³⁶ and USA. ⁸ , ³⁸ , ⁴³ , ⁴⁵ , ⁴⁶

Evaluation of instrument content

The content evaluation will now be described (see Table 2 for summary and Table 1 for details of instrument item/scaling).

Table 2.

 Summary of content evaluation

Targets of regret

Overall, none of the instruments were designed with the intention to measure a specific target of regret. The target(s) of regret we have identified as being measured by each instrument is derived from the reviewers’ assessment, which is based on the definition of regret provided by the instrument’s authors and the items included in the instrument (See Table 1).

Based on the definition of regret, none of the instruments’ addressed process regret. Four of the 10 instruments addressed option regret, ⁸ , ²⁰ , ⁴⁵ , ⁴⁶ three addressed outcome regret, ³⁶ , ⁴³ , ⁴⁴ and two addressed both option and outcome regret. ¹⁵ , ³⁸ A clear definition of regret was not provided for one of the instruments ³⁷ so we were unable to determine the target of regret being measured. Based on the items included in the instrument, only one of the 10 instruments assessed process regret. ⁴⁴ This instrument also assessed option and outcome regret. Four of the 10 instruments assessed option regret, ²⁰ , ³⁷ , ⁴³ , ⁴⁶ two assessed outcome regret, ³⁶ , ⁴⁵ and three assessed both option and outcome regret. ⁸ , ³⁸ For three of the instruments there was an adequate match between the target of regret addressed in the definition and the target of regret measured by the items included in the instrument. ²⁰ , ³⁶ , ⁴⁶ For four instruments, we found there were issues of content validity, whereby the target of regret used in the definition was not sufficiently addressed by items in the instrument. ⁸ , ⁴³ , ⁴⁴ , ⁴⁵ For two instruments we found a partial match between targets of regret addressed in the definition provided and the items used to measure regret. ¹⁵ , ³⁸ For both of these instruments, the authors provide definitions of regret that could address both option and outcome regret. They also include items in the instrument that could measure option and outcome regret. However, this appears to be due to inconsistencies in the definitions of regret and the items used, and raises issues of content validity: although multiple targets of regret are referred to, the authors do not explicitly acknowledge them and they do not attempt to measure them separately.

Overall, several instruments allude to multiple targets of regret by means of the items used in the instrument, ⁸ , ⁴⁴ or both the definition and the items. ¹⁵ , ³⁸ However, none of the instruments explicitly refer to the multiple targets of regret and are therefore unable to measure different targets separately.

Positive and negative outcomes

All instruments captured negative aspects of regret and none captured positive aspects. The focus on negative aspects of regret is reflected by the terminology used in the response items: ‘bother’, ‘disappointed’, ³⁷ ‘harm’, ¹⁵ ‘upset’, ³⁶ ‘annoyed’, ⁴³ ‘feel really good’ (reverse coded item). ²⁰

Discussion

Principal findings

Existing instruments for the measurement of regret do not fully capture current conceptualizations of regret and they do not enable the construct of regret to be measured comprehensively. Specifically, the instruments identified are unable to measure the different targets of regret separately, and they cannot determine whether the outcomes of regret experienced were positive or negative. The instruments were rarely used in longitudinal research, which does not allow us to examine a potential temporal pattern to experienced regret. Most instruments included in the review were developed prior to recent conceptualizations of regret, and it is not surprising that they do not sufficiently measure this concept. Nonetheless, these important findings have implications for the future development of conceptualizations of regret and the instruments that should be designed to measure it.

The 10 instruments included were not able to capture the multiplicity of regret targets; process, option and outcome regret. If, as other research suggests, there are multiple targets of regret that can be experienced independently or in combination, ³ , ²⁹ , ³⁰ , ³⁵ these instruments would not be adequate to measure these. However, it is important to note that the notion of different regret targets was not completely overlooked by the instruments’ developers. For instance, the definition of regret used in the Regret Scale ³⁸ addresses both option and outcome regret. Additionally, the Decision Regret Scale ¹⁵ uses more than one definition of regret; one focuses on option regret and another focuses on outcome regret. However, in these cases, the use of different targets in the definitions is due to inconsistencies in defining the construct, rather than an explicit acknowledgement of different targets of regret.

Different targets of regret are alluded to via the items included in the various instruments. For instance, most of the Decision Regret Scale ¹⁵ items appear to focus on option regret (e.g. ‘I regret the choice that was made’). However, item four (‘the choice did me a lot of harm’) suggests that outcome regret is the target of interest. The authors do not explicitly acknowledge the different targets, nor attempt to measure them separately, and the use of the item appears to reflect an inconsistency in the definition of regret used and the items used to measure it. Similarly, other instruments ³⁶ , ⁴³ , ⁴⁴ reflect this inconsistency by referring to outcome regret in their definition and using items that focus on option regret.

The 10 instruments do not assess whether the experience, or indeed anticipation, of regret results in positive as well as negative outcomes. Based on the general terminology used, in the definitions of regret and the response items, the 10 instruments appear to focus on negative outcomes. This includes the use of terms such as ‘disappointed’, ‘harm’, ‘bother’, ‘upset’, ‘distress’, ‘annoyed’, and ‘remorse’. Regret can indeed be an unpleasant emotion with negative ramifications. However, if regret can also have positive outcomes, to the extent that it is functional and allows us to make ‘better’ decisions in future, it is important to be able to assess this as well.

Strengths and limitations

This review has examined for the first time the extent to which regret measurement instruments capture current conceptualizations of regret. We searched several relevant fields of investigation to gain a better understanding of how the concept is measured. The coverage of various fields however, may also pose a disadvantage to the current research. We found that publications in this area are spread over many journals across different fields, and it is possible that some articles were overlooked through variable indexing and use of subject headings. Additionally, the model of regret (Fig. 1) is not based on direct empirical research. Instead, it has mainly relied on the regret literature to date and the synthesis of research by other authors. ³

Implications for future research

We propose current instruments require further development to be capable of measuring the different targets of regret and assessing whether regret results in positive as well as negative outcomes. Clear consistent definitions of the constructs are needed for rigorous item development according to psychometric principles. If recent conceptualizations of regret incorporate the different targets of regret, and recognize the positive, or functional role, it can play in future decisions, an important step forward in the measurement of this concept would be the development of an instrument with sound construct validity.

Whilst we believe that different targets of regret can be experienced independently from one another, we also appreciate that the quality of the decision process and the valence of the decision outcomes often co‐vary: bad decision processes lead to bad outcomes, and vice versus. As a result, people will be more used to experiencing the collective emotion of regret, which means they are unlikely to make automatic attempts to differentiate the source of their experienced regret. However, as research has shown, ²⁹ people can regret a bad decision process regardless of the quality of the decision outcomes, which indicates the potential for people to differentiate between the sources. Therefore, whilst many people may not be able to distinguish the sources of their regret intuitively, a well designed instrument that explicitly guides the user to think about the source would facilitate this.

Instruments that can highlight the targets of regret experienced, or indeed anticipated, would allow us to design interventions that ensure negative outcomes associated with regret at each stage of the decision are minimized and positive outcomes maximized. For example, if a measure of anticipated regret could determine that a person anticipates process regret (e.g. if they fail to seek relevant information about treatment options), interventions could be put in place before the decision (e.g. decision aids) to reduce experienced process regret post‐decision. On the other hand, if a person experienced outcome regret, regret reduction interventions that help an individual ‘live’ with their decision, such as psychological repair work or re‐appraisal of the alternative, ³ may be suitable. These interventions would be particularly relevant to situations where the decision is irreversible and an individual does not have the opportunity to go through another decision process to amend the choice, such as choosing mastectomy over lumpectomy for early‐stage breast cancer treatment.

An instrument capable of differentiating between positive and negative outcomes of regret would be useful as the two types may have different effects. It is important to identify whether a person experiences negative outcomes as these could be emotionally damaging, and efforts should focus on reducing these effects. However, it is equally important to determine whether a person interprets experienced regret as positive. An instrument that could identify the target of experienced regret, and establish whether the regret was negative or positive, would allow people to think about what went wrong with that decision and what they have learnt for future decisions.

It is important to note that whilst a key goal of decision making is to reduce the potential for the negative experienced emotion, ¹ the same goal does not necessarily apply to its meta‐cognition. As discussed, thinking about potential experienced regret ahead of a decision can encourage the user to act in ways that will minimize experienced regret. Similarly, thinking about experienced regret after a decision is made can lead to positive outcomes, in so far that it allows us to learn from past mistakes and it could be an enlightening experience about ones actual decision‐making capabilities. If we did aim to reduce the meta‐cognition of experienced regret, we might inhibit the positive outcomes and the functional role it can play in future decision making. Clearly, all of these issues have implications for the use of current instruments in evaluations of decision support interventions: increased levels of experienced regret may coincide with increased thinking about regret, which could be beneficial.

Process regret to assess ‘good decisions’?

We have proposed that current instruments require further development. However, significant challenges remain for those instruments that can measure the different targets of regret, namely the question of which target is the most appropriate measure of decision quality. Recent work concerning definitions and evaluations of a ‘good’ decision has highlighted the lack of distinction between deliberation and determination stages of decision making; ¹⁴ the former relating to the process of arriving at a decision and the latter referring to the actual decision and consequential events. The authors also express concern over post hoc decision quality measures that focus on post‐decisional outcomes. As such, even if instruments could measure the different targets of regret, process regret might be the most appropriate proxy measure of decision quality.

Conclusions

An instrument that can differentiate between and measure the different targets of regret would afford us a greater understanding of the concept. Prospective and retrospective application of this instrument would allow us to have a better understanding of the emergence of regret and its subsequent course for individuals as they experience it. These together would enable us to develop and implement pre‐ or post‐decision interventions that focus on preventing or reducing regret respectively.

However, whilst we have argued that existing instrumentation needs to be improved, we also emphasize the need for clarity on why we measure decision regret. If the purpose is to use regret as a proxy measure for defining good decision making, we must acknowledge the issues highlighted in the discussion section of this paper. The concern over post hoc evaluations of decisions ¹⁴ suggests that a measure of process regret would be a valid measure of a good decision. This will, and should, limit what regret instruments are used for in future.

Source of funding

None.

Conflict of interest

The authors declare that they have no conflicts of interest.

Appendix 1: Medline search strategy

#	Searches
1	exp Decision Making/
2	exp Choice Behavior/
3	exp Judgment/
4	decision$.mp.
5	choice$.mp.
6	preference$.mp.
7	judgment$.mp.
8	1 or 2 or 3 or 4 or 5 or 6 or 7
9	regret$.mp.
10	disappoint$.mp.
11	dismay.mp.
12	dissatis$.mp. [mp=title, original title, abstract, name of substance word, subject heading word]
13	9 or 10 or 11 or 12
14	exp Research Design/
15	exp “Reproducibility of Results”/
16	exp Questionnaires/
17	exp Psychometrics/
18	exp Evaluation Studies as Topic/
19	(measur$ or assessment$ or scal$ or scor$ or instrument$ or analys$ or evaluat$ or valid$ or reliab$).mp.
20	14 or 15 or 16 or 17 or 18 or 19
21	8 and 13 and 20
22	limit 21 to yr=“1960 ‐ 2008”
23	limit 22 to humans