Decisional Conflict Regarding Disease-Modifying Treatment Choices Among Patients with Mid-Stage Relapsing-Remitting Multiple Sclerosis

Abstract

Purpose

Shared decision-making is critical in multiple sclerosis (MS) due to the uncertainty of the disease trajectory over time and the large number of treatment options with differing efficacy, safety and administration characteristics. The aim of this study was to assess patients’ decisional conflict regarding the choice of a disease-modifying therapy and its associated factors in patients with mid-stage relapsing-remitting multiple sclerosis (RRMS).

Methods

A multicenter, non-interventional study was conducted. Adult patients with a diagnosis of RRMS (2017 revised McDonald criteria) and disease duration of 3 to 8 years were included. The level of uncertainty experienced by a patient when faced with making a treatment choice was assessed using the 4-item Decisional Conflict Scale. A battery of patient-reported and clinician-rated measures was administered to obtain information on symptom severity, illness perception, illness-related uncertainty, regret, MS knowledge, risk taking behavior, preferred role in the decision-making process, cognition, and self-management. Patients were recruited during routine follow-up visits and completed all questionnaires online using electronic tablets at the hospital. A multivariate logistic regression analysis was conducted.

Results

A total of 201 patients were studied. Mean age (Standard deviation) was 38.7 (8.4) years and 74.1% were female. Median disease duration (Interquartile range) was 6.0 (4.0–7.0) years. Median EDSS score was 1.0 (0–2.0). Sixty-seven (33.3%) patients reported a decisional conflict. These patients had lower MS knowledge and more illness uncertainty, anxiety, depressive symptoms, fatigue, subjective symptom severity, a threatening illness perception, and poorer quality of life than their counterparts. Lack of decisional conflict was associated with MS knowledge (Odds ratio [OR]=1.195, 95% CI 1.045, 1.383, p=0.013), self-management (OR=1.049, 95% CI 1.013, 1.093, p=0.018), and regret after a healthcare decision (OR=0.860, 95% CI 0.756, 0.973, p=0.018) in the multivariate analysis.

Conclusion

Decisional conflict regarding the selection of a disease-modifying therapy was a common phenomenon in patients with mid-stage RRMS. Identifying factors associated with decisional conflict may be useful to implement preventive strategies that help patients better understand their condition and strengthen their self-management resources.

Introduction

Multiple sclerosis (MS) is a chronic autoimmune disease of the central nervous system that causes disability, poor quality of life, and productivity loss even at early stages.^1–3 The MS therapeutic landscape has changed considerably in recent years with different treatment options capable of controlling the relapse rate and reducing the risk of disability progression.^4,5 However, many patients still suffer from a variety of disabling symptoms such as fatigue, sphincter disturbances, cognitive problems or pain.^2,6 Furthermore, despite our increasing understanding of numerous prognostic factors, the disease’s natural course varies, often making it challenging for neurologists to reliably predict their patients’ long-term prognosis.⁷

Selecting an appropriate disease-modifying treatment (DMTs) for MS presents numerous challenges.^1,5 The expanding repertoire of available therapies, each characterized by its unique efficacy, safety considerations, and method of administration, complicates the decision-making process.^1,8 In addition, patient preferences play a crucial role in decision-making, as MS patients often have unique experiences and priorities that can significantly influence their treatment choices.⁹ Factors such as lifestyle, personal goals, the impact of treatment side effects, and desired level of involvement in managing the disease vary from patient to patient over time and are influenced by emotional, social, and experiential factors.^9–11

Decisional conflict is defined as personal uncertainty about what to do when the choice between different options involves risk, regret or a challenge to personal life values.^12,13 In a complex MS treatment landscape context, patients can doubt whether their treatment decision was right, leading to decisional conflict.^14–16 In a recent study with 254 MS patients in the United Kingdom, 53% of participants reported decisional conflict, especially in those who were unsatisfied with their treatment.¹⁴ The aim of this study was to assess the decisional conflict related to the choice of a disease-modifying therapy and its associated factors in patients with mid-stage relapsing-remitting multiple sclerosis (RRMS).

Materials and Methods

This non-interventional, cross-sectional study was conducted at 19 hospital-based neuro-immunology care units in Spain (FACE-MS study). Key inclusion criteria included a diagnosis of RRMS according to 2017 revised McDonald criteria, a disease duration of 3 to 8 years, and being on disease-modifying therapy.¹⁷ Focusing on patients in a mid-term stage of MS provides a more nuanced perspective on the disease’s impact and treatment outcomes while minimizing the risk of bias associated with early disease stages.^18–20 The decision to study patients with a disease duration of 3 to 8 years was intended to capture individuals with a more comprehensive understanding of the disease trajectory. These patients have already passed the initial phase of processing the diagnosis and have experienced some degree of disease progression and response to treatment.²⁰

Patients were invited to participate in the study by their treating neurologists in the context of their regular follow-up visits. The study was approved by the research ethics board of the Hospital de Galdakao-Usansolo (Galdakao, Spain) and performed in accordance with the 1964 Helsinki Declaration and its later amendments. All participants provided a written informed consent and were recruited from January to November 2022.

The level of comfort with a treatment decision was evaluated using the 4-item Decisional Conflict Scale (DCS SURE).²¹ Patients were asked to recall when they initiated their disease-modifying treatment or changed their most recent treatment. The scores of the four items were summed, with a score of less than 4 indicating the presence of decisional conflict.²¹ The DCS SURE has shown adequate internal consistency, moderate correlation to the full 16-item DCS, and adequate sensitivity and specificity of its cutoff score for identifying clinically significant decisional conflict.^13,22 The validated Spanish version of the instrument was used in this study.²³

In addition, other outcome measures were included based on previously identified associations in the literature with the decision-making process.^24–27 These measures included the SymptoMScreen (SyMS), Brief-Illness Perception Questionnaire (B-IPQ), 5-item Modified Fatigue Impact Scale (MFIS-5), 54-item MS Quality of Life (MSQOL-54), Hospital Anxiety and Depression Scale (HADS), Mishel Uncertainty of Illness Scale (MUIS), Decision Regret Scale (DRS), MS Self-Management Scale-Revised (MSSM-R), MS Knowledge Assessment Scale (MSKAS), General Risk Propensity Scale (GRiPS), Control Preference Scale (CPS), and Symbol Digital Modalities Test (SDMT).^28–39 All questionnaires were administered through an electronic tablet and completed online at the hospital.

The SyMS assesses MS symptom severity across 12 neurologic domains: mobility, hand function, spasticity and stiffness, pain, sensory symptoms, bladder control, fatigue, vision, dizziness, cognition, depression, and anxiety.²⁸ Each item is assessed on a 7-point Likert scale from 0 (not at all affected) to 6 (total limitation). Higher scores indicate more severe symptom endorsement. The B-IPQ assesses cognitive and emotional illness representations.²⁹ It consists of eight items rated on a scale from 0 (minimum) to 10 (maximum). Higher scores indicate a threatening illness perception. The MFIS-5 assesses physical, cognitive, and psychosocial components of fatigue.³⁰ Each item scores on a 5-point Likert scale from 0 (never) to 4 (almost always). Higher scores indicate more severe fatigue. This 54-item MSQOL-54 measures quality of life combining both generic and MS-specific items.³¹ It generates a mental health and a physical health composite scores, and two additional single-item measures: satisfaction with sexual function and change in health. Higher scores indicate better quality of life. The HADS is a 14-item, self-assessment scale to measure symptoms of anxiety and depression.³² Each item is scored on a 4-point Likert scale from 0 to 3. A total subscale score >8 indicates a probable case of anxiety or depression, respectively. The MUIS is a 17-item scale to measure uncertainty about the disease trajectory and the treatments.³³ Each item is assessed on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). Higher scores indicate greater uncertainty. The DRS is a 5-item scale to measure distress or remorse after a healthcare decision.³⁴ Each item is assessed on a 5-point Likert scale from 1 (strongly agree) to 5 (strongly disagree). Higher scores indicate a greater regret. The MSSM-R is a 23-item scale to measure patient´s skills to continue an active life and a satisfying psychological status in the face of MS.³⁵ Each item is assessed on a 5-point Likert scale from 1 (completely disagree) to 5 (completely agree). Higher scores indicate a higher level of self-management. The MSKAS is a 22-item scale to measure MS knowledge.³⁶ Respondents were provided with three answer options for each statement: “true”, “false”, and “I don’t know”. These responses were subsequently coded as either correct (1) or incorrect (0). “I don’t know” responses were also coded as incorrect. Higher scores indicate greater knowledge. The GRiPS is an 8-item scale to measure risk-taken behavior.³⁷ Each item is assessed on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). Higher scores indicate a higher propensity to take risks. The CPS consists of five cards with a cartoon describing different roles in the decision-making process.³⁸ Patients who prefer to make decisions on their own, or jointly with the physician, or who prefer the physician to make the decision are classified as active, collaborative, or passive, respectively. The SDMT measures patient attention and information processing speed.³⁹ A cut-off of ≤49 correct substitutions is used to identify patients with cognitive impairment.

Statistical Analysis

Descriptive statistics were presented for all variables. Continuous variables were analyzed with measures of central tendency (mean and median), variability/dispersion (standard deviation [SD] and interquartile ranges [IQR]), and categorical variables with absolute and relative frequency distributions (percentages of groups). We chose to employ a binary approach of the DCS SURE based on the recommendations by its creators, which underscores the utility and validity of this binary transformation for identifying clinically significant decisional conflict.⁴⁰ A multivariate logistic regression analysis was conducted to determine the association between patients’ characteristics and lack of decisional conflict (decisional certainty). This analysis was performed using three different models. The first model was a multivariate logistic regression model that included all the covariates that were statistically significant or near significant (p<0.2) in the bivariate analysis plus other clinically significant covariates. The second model was also a multivariate logistic regression model but with a stepwise selection procedure. Finally, the third model was a least absolute shrinkage and selection operator (LASSO) multivariate logistic regression model.⁴¹ LASSO is a penalized regression approach in which the estimates of the regression coefficients are sparse, which means that many components are exactly zero and thus this method automatically deletes unnecessary covariates. This approach relies on the selection of a parameter λ that modulates the accuracy and complexity of the model. The logistic LASSO regression was implemented using the R package glmnet and λ was selected as the value that gives the most accurate and simplest model.

Results

Of the 209 patients who agreed to participate, eight were ineligible for analysis due to not meeting the inclusion criteria. A total of 201 patients were studied, and there were no withdrawals from the study. Mean age (SD) was 38.7 (8.4) years and 74.1% were female. Median (IQR) disease duration and EDSS score were 6.0 (4.0–7.0) years and 1.0 (0–2.0), respectively. Almost half of the patients changed DMTs at least once since diagnosis (n=94, 46.5%). The main cause of the change was self-reported lack of efficacy (63.8%), followed by tolerability or safety problems (38.3%). The main characteristics of the study population are shown in Table 1.

Table 1.

Sociodemographic, Clinical, and Trait Characteristics of Participants

	Total N=201	Decisional Conflict N=67	No Decisional Conflict N=134	p-value
Age, years, mean (SD)	38.7 (8.4)	39.8 (8.8)	38.1 (8.2)	0.171
Sex, female, n (%)	149 (74.1)	47 (70.1)	102 (76.1)	0.362
Disease duration, years, median (IQR)	6.0 (4.0, 7.0)	6.0 (4.0, 7.0)	6.0 (5.0, 7.0)	0.284
Number of relapses, median (IQR)	2.0 (1.0, 3.0)	2.0 (1.0, 4.0)	2.0 (1.0, 3.0)	0.472
EDSS score, median (IQR)	1.0 (0, 2.0)	1.0 (0, 2.0)	1.0 (0, 2.0)	0.497
SyMS score, median (IQR)	10.0 (5.0, 20.0)	16.0 (7.0, 27.0)	9.0 (5.0, 17.0)	0.016
MFIS-5 score, mean (SD)	7.7 (5.1)	9.0 (5.6)	7.0 (4.8)	0.011
SDMT, ≤49 correct answers, n (%)	77 (38.3)	29.6 (6.2)	30.6 (6.6)	0.473
HADS
Anxiety score, mean (SD)	7.7 (4.6)	9.0 (4.9)	7.1 (4.2)	0.009
Depression score, mean (SD)	4.6 (4.1)	5.8 (4.7)	4.1 (3.7)	0.022
B-IPQ score, mean (SD)	35.7 (10.8)	40.4 (11.1)	33.3 (9.9)	<0.001
MSQOL-54
Physical score, mean (SD)	65.2 (12.4)	61.5 (13.9)	67.0 (11.2)	0.011
Psychological score, mean (SD)	59.8 (9.2)	57.7 (8.4)	60.8 (9.4)	0.013
MUIS score, mean (SD)	38.2 (10.8)	43.3 (11.3)	35.6 (9.6)	<0.001
GRiPS score, mean (SD)	18.7 (8.2)	18.1 (8.5)	19.0 (8.0)	0.301
MSKAS score, mean (SD)	11.1 (2.8)	10.0 (2.7)	11.6 (8.0)	<0.001
MSSM-R score, mean (SD)	83.2 (12.1)	77.7 (13.8)	85.9 (10.1)	<0.001
CPS, n (%)				0.344
Collaborative	92 (46.5)	27 (40.9)	65 (49.2)
Passive	59 (29.8)	24 (36.4)	35 (26.5)
Active	47 (23.7)	15 (22.7)	32 (24.2)
DRS, mean (SD)	7.8 (0.6)	9.1 (3.7)	7.1 (2.5)	<0.001

Abbreviations: B-IPQ, Brief Illness Perception Questionnaire; CPS, Control Preference Scale; DRS, Decision Regret Scale; EDSS, Expanded Disability Status Scale; GRiPS, General Risk Propensity Scale; HADS, Hospital Anxiety and Depression Scale; IQR, Interquartile range; MFIS-5, 5-item Modified Fatigue Impact Scale; MSKAS, Multiple Sclerosis Knowledge Scale; MSQOL-54, Multiple Sclerosis Quality of Life; MSSM-R, Multiple Sclerosis Self-Management Scale Revised; MUIS, Mishel Uncertainty Illness Scale; SD, Standard deviation; SDMT, Symbol Digit Modalities Test; SyMS, SymptoMScreen.

Sixty-seven (33.3%) patients reported a decisional conflict. Those patients had a poorer quality of life, a threatening illness perception, lower MS knowledge, and more illness uncertainty, anxiety, depressive symptoms, fatigue, and subjective symptom severity than their counterparts (Table 1).

Lack of decisional conflict was associated with MS knowledge (Odds ratio [OR]=1.195, 95% CI 1.045, 1.383, p=0.013), self-management (OR=1.049, 95% CI 1.013, 1.093, p=0.018), and regret after a healthcare decision (OR=0.860, 95% CI 0.756, 0.973, p=0.018) in the multivariate analysis (Table 2).

Table 2.

Lack of Decisional Conflict: Multivariate Logistic Regression Analysis

	Full Model		Stepwise Logistic		LASSO
	OR (95% CI)	p-value	OR (95% CI)	p-value	OR
Age, years	0.981 (0.935–1.027)	0.410	-	-	-
Sex, female	1.721 (0.692–4.264)	0.239	-	-	-
Number of relapses	0.849 (0.662–1.088)	0.190	-	-	-
Disease duration	1.020 (0.784–1.330)	0.882	-	-	-
EDSS score	1.212 (0.840–1.790)	0.316	-	-	-
SyMS score	1.003 (0.943–1.067)	0.920	-	-	-
MFIS-5 score	0.964 (0.857–1.083)	0.538	-	-	-
SDMT, ≤49 correct answers	0.980 (0.946–1.014)	0.243	-	-	-
B-IPQ score	0.962 (0.904–1.021)	0.202	0.939 (0.896–0.981)	0.006	0.997
HADS anxiety score	1.008 (0.877–1.161)	0.913	-	-	-
HADS depression score	1.144 (0.965–1.369)	0.131	1.104 (0.984–1.248)	0.101	-
MSQOL-54 physical score	1.017 (0.951–1.088)	0.616	-	-	-
MSQOL-54 psychological score	1.004 (0.955–1.055)	0.886	-	-	-
MUIS score	0.967 (0.919–1.015)	0.178	-	-	0.984
GRiPS score	1.000 (0.951–1.053)	0.994	-	-	-
MSKAS score	1.168 (1.012–1.362)	0.040	1.195 (1.045–1.383)	0.013	1.049
MSSM-R score	1.040 (1.003–1.084)	0.039	1.049 (1.013–1.093)	0.018	1.010
DRS score	0.856 (0.738–0.986)	0.034	0.860 (0.756–0.973)	0.018	0.964

Abbreviations: B-IPQ, Brief Illness Perception Questionnaire; CI, Confidence interval; DRS, Decision Regret Scale; EDSS, Expanded Disability Status Scale; GRiPS, General Risk Propensity Scale; HADS, Hospital Anxiety and Depression Scale; LASSO, Least absolute shrinkage and selection operator; MFIS-5, 5-item Modified Fatigue Impact Scale; MSKAS, Multiple Sclerosis Knowledge Scale; MSQOL-54, Multiple Sclerosis Quality of Life; MSSM-R, Multiple Sclerosis Self-Management Scale Revised; MUIS, Mishel Uncertainty Illness Scale; OR, Odds ratio; SDMT, Symbol Digit Modalities Test; SyMS, SymptoMScreen.

Discussion

The uncertainty experienced by a patient when faced with making a difficult decision is a phenomenon described in different diseases.⁴² RRMS is a paradigmatic example for investigating the decisional conflict due to the unpredictable trajectory of the disease and the numerous treatment options available, each differing in efficacy, safety, and administration characteristics.^4,7,8 Decisional conflict arises when patients feel unsure about which treatment option to choose, have conflicting values or preferences, or lack adequate information to make a well-informed decision, fear of making the wrong choice, feelings of regret, and the tendency to blame physicians for unfavorable outcomes.^9,10,12

In our study, decisional conflict related to the choice of a DMTs was a common phenomenon (33,3%) in a sample of mid-stage RRMS patients. Two recent studies in MS utilizing standardized scales such as the DCS revealed a significant proportion of patients experiencing conflict regarding their treatment decisions.^14,43 In a study conducted in the United Kingdom, 53% of MS patients reported experiencing decisional conflict.¹⁴ This figure rose to 75% among patients who were initiating treatment and was 44% among those already undergoing treatment. Regret after a decision was higher in those patients who were unsatisfied with their DMTs and those who were receiving low-efficacy agents.¹⁴ In a separate study in Canada, researchers found that 36% of a sample predominantly consisting of treatment-naive patients with RRMS experienced some level of decision-making conflict.⁴³

Patients’ beliefs and expectations about a disease influence their emotional reactions and coping resources.¹¹ In our study, the lack of decisional conflict was significantly associated with illness-related knowledge, self-management, and low regret. An adequate knowledge of the disease has been shown to facilitate a positive behavior change and improve decision-making, treatment adherence and satisfaction with care among patients with MS.^27,44 Self-management in MS embodies the proactive adoption of strategies and skills by individuals to manage their condition, symptoms, and overall well-being.⁴⁵ This approach integrates a comprehensive understanding of pathophysiology and disease progression, recognition and management of symptoms, awareness of treatment options, and the impact of lifestyle factors. By actively monitoring symptoms and changes in their condition, individuals can optimize health outcomes, improve quality of life, and minimize the daily impact of MS. Effective self-management empowers individuals with MS to make informed decisions about their care, enhancing their ability to navigate the complexities of the disease and take charge of their health.⁴⁵ In the current MS therapeutic landscape, characterized by numerous efficacy alternatives, decision regret is influenced by several crucial factors.¹⁴ Regret can stem not only from obvious errors in decision-making, but also from the presence of readily available counterfactual alternatives.^46,47 Decisional conflict can increase the likelihood of decisional regret by fostering uncertainty, emotional distress, and a sense of missed opportunity if the outcome falls short of expectations.

Educational strategies play a crucial role in empowering MS patients to make well-informed decisions regarding DMTs.⁴⁸ These strategies encompass a spectrum of approaches, ranging from simple, practical decision aids that help patients prepare for discussions with healthcare professionals, to more comprehensive educational interventions aimed at reducing decisional conflict during clinical consultations.^43,49,50

Our study has some limitations. First, a selection bias may have influenced the prevalence of decisional conflict, as the study participants may have been skewed towards more motivated or cooperative individuals who chose to participate. This selection bias might not accurately represent the broader population of patients with RRMS, potentially impacting the generalizability of the study’s findings. Second, the reliance on patients’ ability to accurately recall details about their treatment decisions, such as the initiation or change of DMTs. Patients may not recall the specifics of their decision-making process accurately, especially if some time has passed since the decision was made. This could introduce inaccuracies or inconsistencies in the reported levels of decisional conflict, potentially affecting the study’s findings. Third, outcome bias could influence the study results. Patients’ perceptions of their treatment outcomes might influence their retrospective assessment of decisional conflict. For instance, patients who perceive their treatment as effective or ineffective may retrospectively justify their decision-making process, impacting the reported levels of decisional conflict. Third, the perspectives of patients not currently receiving DMTs were not represented in our sample. Finally, the cross-sectional study design limits the ability to establish causal relationships between the factors assessed and decisional conflict, emphasizing the need for future longitudinal studies.

Conclusions

Decisional conflict among mid-stage RRMS patients when choosing DMTs was a common phenomenon. Decisional conflict gives insight into the patient’s perspective on the quality of the decision-making process and decisions being made. Empowering patients with comprehensive disease education and enhancing self-management skills could potentially reduce decisional conflict and improve treatment satisfaction and outcomes in RRMS care. Future studies should consider mitigating these biases through more objective measures or prospective assessments to enhance the validity of their results.

Funding Statement

This study was funded by Roche Farma Medical Department, Spain (ML43469). The funding source had no role in the design, analysis and interpretation of the data, review or approval of the manuscript, or in the decision to submit for publication.

Author Contributions

All authors made a significant contribution to the work reported, whether in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all of these areas; all took part in drafting, revising or critically reviewing the article; all gave final approval of the version to be published; all have agreed on the journal to which the article has been submitted; and all agree to be accountable for all aspects of the work.

Disclosure

Julia Sabin received honoraria for lecturing, consulting or travel expenses from Biogen, Merck, Teva, Novartis, BMS, Janssen, Almirall, Roche and Sanofi. Jesús Martín-Martínez served on scientific advisory boards and/or received speaking honoraria, research funding and support to attend scientific meetings from Biogen, Merck, Novartis, Roche and Teva. Francisco Javier Barrero received compensation for consulting services and speaking honoraria from Almirall, Biogen, Genzyme, Merck, Novartis, Roche, Sanofi and Teva. Ana Alonso received compensation for consulting services from Biogen, BMS, Sanofi, Roche, Janssen and Novartis; and speaking honoraria from Biogen, BMS, Sanofi, Roche, Janssen, Merck, Almirall and Novartis. José Luis Sánchez-Menoyo received support to attend scientific meetings from Novartis, Merck, and Biogen; speaking honoraria from Biogen, Novartis, Sanofi, Merck, Almirall, Bayer and Teva; and participated in clinical trials from Biogen, Merck and Roche. Laura Borrega received compensation for consulting services, speaking honoraria and support to attend scientific meetings from Bayer, Celgene, Biogen, Sanofi, Merck, Novartis, Roche, Almirall and Teva. Monserrat Gómez-Gutiérrez received honoraria as speaker from Biogen, BMS, Janssen, Merck, Mylan, Novartis, Roche, Sanofi and Bial. Sara Eichau received consulting fees and lecture honoraria from Biogen, Novartis, Sanofi, Merck, Roche and Almirall. Carmen Calles received compensation for consulting services, speaking honoraria and support to attend scientific meetings and courses from Merck, Teva, Sanofi, Novartis, Biogen, Roche, and BMS. Eva Fernández-Díaz received honoraria and travel expenses for participation in scientific meetings and advisory boards from Almirall, Biogen, Merck, Roche and Sanofi. Olga Carmona received honoraria for speaker services and advisory boards from Sanofi, Roche, Janssen and Merck. Aida Orvíz received research grants, travel support, advisory activities or honoraria for speaking engagements from Almirall, Biogen, BMS, Merck, Mylan, Novartis, Roche, Sanofi, and Teva. Ana López-Real received speaker and consultation fees, and congress travel support from Biogen, Janssen, Merck, Novartis, Roche and Sanofi. Amelia Mendoza received research grants, travel support or honoraria for speaking engagements from Biogen, Merck, Novartis, Roche, Sanofi, and Teva. Eduardo Agüera received speaking honoraria from Roche, Novartis, Merck, Sanofi and Biogen. Elisa Salas and Jorge Maurino are employees of Roche Farma Spain. The authors report no other conflicts of interest in this work.