Skip to main content
PLOS One logoLink to PLOS One
. 2022 Dec 16;17(12):e0278526. doi: 10.1371/journal.pone.0278526

Trade-offs during the COVID-19 pandemic: A discrete choice experiment about policy preferences in Portugal

Luís Filipe 1,*,#, Sara Valente de Almeida 2,#, Eduardo Costa 3,#, Joana Gomes da Costa 4,#, Francisca Vargas Lopes 5,6,#, João Vasco Santos 7,8,9,#
Editor: Karyn Morrissey10
PMCID: PMC9757580  PMID: 36525428

Abstract

The need to control the sanitary situation during the COVID-19 pandemic has led governments to implement several restrictions with substantial social and economic impacts. We explored people’s trade-offs in terms of their income, life restrictions, education, and poverty in the society, compared to their willingness to avoid deaths. We applied a web-based discrete choice experiment to elicit preferences of the Portuguese citizens for these attributes and computed the marginal rate of substitution in terms of avoided deaths. We recorded 2,191 responses that faced the possibility of having 250 COVID-19 related deaths per day as the worst possible outcome from the choice levels presented. Estimates suggested that individuals would be willing to sacrifice 30% instead of 10% of their income to avoid approximately 47 deaths per day during the first six months of 2021. For the same period, they would also accept 30% of the students’ population to become educationally impaired, instead of 10%, to avoid approximately 25 deaths; a strict lockdown, instead of mild life restrictions, to avoid approximately 24 deaths; and 45% of the population to be in risk of poverty, instead of 25%, to avoid approximately 101 deaths. Our paper shows that avoiding deaths was strongly preferred to the remaining societal impacts; and that being a female, as well as working on site, led individuals to be more averse to such health hazards. Furthermore, we show how a DCE can be used to assess the societal support to decision-making during times of crisis.

1. Introduction

The management of the COVID-19 pandemic was characterised by difficult choices. In several phases of the pandemic, most governments were compelled to implement strict lockdowns. These lockdowns made people stay at home, often only being allowed to leave for essential reasons. It also implied the temporary closure of schools, retail, and many other activities. All these measures had a clear and negative impact on families’ income, education, and social lives [1,2]. These were significant policy choices that implied difficult trade-offs. It is not clear whether, when making these decisions, governments were able to understand and reflect society preferences and follow the scientific evidence, at the same time.

Understanding individuals’ preferences in this context is extremely important as the efficacy of contingency measures depended heavily on individuals’ behaviours. Literature has shown that it is challenging to capture these preferences [3,4]. To overcome this, researchers have been increasingly using Discrete Choice Experiments (DCE) to understand preferences and inform policy decisions, which is especially relevant in healthcare studies [5].

DCEs allow the study of choices with a high degree of flexibility. This tool is implemented by creating hypothetical games where individuals choose between options, implicitly revealing their preferences. Such knowledge of preferences, inherently linked to beliefs and behaviours, is a key piece of information to guide government decisions. It is particularly relevant in the context of a crisis, when unprecedented measures must be adopted in short time frames [6,7]. DCE studies have been used in the context of COVID-19, not only to measure how the citizens value the costs and benefits of lockdown policies [812] but also to understand potential barriers and preferences towards the uptake of COVID-19 measures, such as testing, contact-tracing and vaccination [1316].

The objective of this study was to use a DCE to assess individuals’ preferences regarding the consequences of COVID-19 related policies in Portugal and to explore which of their characteristics might be associated with heterogeneous preferences among the population.

We evaluated preferences in terms of five attributes: number of daily deaths, loss of household income, life restrictions (limitations to individual freedom), educational impairment, and levels of poverty in the society. Our study provides new information on how Portuguese citizens considered these policy trade-offs, by estimating their willingness to avoid deaths relative to the other attributes. We examined heterogeneous preferences based on an extensive set of individual characteristics collected in our survey through i) an unconditional subgroup analysis, and ii) a latent class model. Furthermore, we were able to capture preferences at a particularly challenging moment of the pandemic management. Our survey was applied during the third and most deadly COVID-19 wave in Portugal. The period that led to the third wave was marked by stronger public debate around restrictions. While conducting the survey, COVID-19 numbers were growing fast, and the Portuguese National Health Service (NHS) displayed signals of being overwhelmed. Thus, the survey captured preferences of individuals at a crucial phase of decision-making by the government.

2. The COVID-19 pandemic in Portugal

On March 18th, 2020, following the rapid increase of COVID-19 cases in European countries, Portugal declared the beginning of the “State of Emergency”, allowing a set of exceptional legal provisions to be implemented in fighting the pandemic. Among other measures, over a six-week period, only citizens from the essential services could work outside their homes. All other citizens including children had to be confined and were only allowed to leave their homes in exceptional circumstances.

In mid-April, at the worst stage of the first wave, Portugal reached 1,516 daily new cases of COVID-19 and 37 daily deaths (Portugal has approximately 10 million inhabitants). In May, the numbers started to decrease significantly. Most measures adopted in the first stage of the pandemic were gradually scaled down in the following months. The summer was quiet, with the number of new daily cases stabilising around 300 and the number of daily deaths in values below five.

The number of daily cases started to slowly rise again in September 2020, with the growth hitting an exponential trend in October. On November 9th, the country went back to a “State of Emergency”. In this second wave, the maximum number of new daily cases increased to 6,994 (November 19th) and the highest number of daily deaths was 98 (December 13th).

After Christmas and New Year’s Eve celebrations, when restrictions were partially relaxed and gatherings were allowed, the country saw a sudden rise in the number of cases that would lead to the third and worst wave of the pandemic. At this time, Portugal was among the first western countries to be hit with the third wave and recorded the highest levels of cases and deaths per capita at a global scale. On January 15th, the government reinstated a nationwide lockdown due to the surge in new daily cases. The highest daily number of new cases and deaths occurred on January 28th with 16,432 new cases and 303 deaths. Portugal remained in lockdown for the next few months. The “State of Emergency” was lifted on May 1st, 2021. From then onwards, the restrictive measures were gradually dropped.

Our survey was distributed between January 12th and March 14th, 2021, capturing the most precarious period of the pandemic in Portugal. During the survey collection period, vaccination was at a very early stage—in the peak of the third wave, only about 2.5% of the population had received a first dose of the vaccine.

3. Methodology

3.1. DCE

We used a DCE to evaluate the preferences of the Portuguese citizens regarding the possible effects of different COVID-19 policies. This is a recognized methodology to elicit latent preferences and has been used when presenting the respondents with difficult moral trade-offs [17,18].

3.2. Attributes and levels

The study used five attributes: number of COVID-19 related deaths, loss of household income, educational impairment, life restrictions, and poverty levels in the society (Table 1). Each attribute was divided into three levels of low, medium, and high intensity. Based on international literature, other DCEs were used to inform this study [8,12]. Their attributes were adapted to the Portuguese context based on the findings of a previous COVID-19 survey [19] and authors’ perception of policy relevance of the outcomes. For each attribute we defined the high (worst) level based on a critical evaluation of the data and the policies implemented at the time of the survey [20]. For the quantitative attributes, medium and low levels were constructed by adopting linear decrements of a third to a fourth of the highest level. In terms of the duration of the effects, we defined they would last for a period of six months (January to June 2021).

Table 1. Attributes and levels of the survey for the period between January and June 2021.

Attributes Units Low Medium Maximum
COVID-19 related deaths Daily deaths attributed to the pandemic either directly (i.e., COVID-19 infection) or indirectly (i.e., limited response by the health care system) 50 150 250
Household income lost % decrease in respondent’s household income motivated by the pandemic 10 20 30
Educational impairment % of students permanently affected by the pandemic in the school population 10 20 30
Life restrictions Type of restrictions in place for all the citizens Large-events restrictions and no bars/ clubs Night curfew and max 5 people gatherings Full lockdown
Poverty level % of the population in risk of poverty 25 35 45

In Portugal, the number of COVID-19 deaths was in the spotlight every day since the beginning of the pandemic. This led us to include COVID-19 related deaths as the baseline attribute in our survey, i.e., the marginal rate of substitution (MRS) was defined as the number of deaths to be traded for other attributes. The highest level for this attribute was defined as Portugal’s linear equivalent to the highest daily number of deaths observed in Spain in 2020, during that country’s severe first wave.

With the consecutive lockdowns, the economy slowed down considerably (GDP in 2020 decreased by 8.4%). This led to a considerable loss of household income for many people, either because they owned a business that was affected or because they were laid off [2123]. The second attribute was the percentage decrease in the respondent’s household income related with the pandemic. The levels for this attribute were 10%, 20% and 30%. The choice of maximum loss of 30% considered that laid off employees or companies that had to shut down would still have access to 70% of their income, through social security schemes.

Schools belong to the wide range of institutions that closed during the first times of the pandemic. Portuguese high schools were closed for 97 days, between January 2020 and May 2021 [2]. As a third attribute, we included the percentage of the school population that would become permanently affected by the pandemic. To some extent, this percentage reflects the inequalities associated with schools’ closure, distance learning and other mitigation strategies. The levels for this attribute were 10%, 20% and 30%. The maximum level was chosen in accordance with a report from the World Bank [24] estimating the number of students who may suffer from losses because of the school closures.

Strict restriction policies impacted the regular day-to-day life of the population. Being deprived of social interactions can be burdensome [25]. The fourth attribute was the level of life restrictions imposed by the government. The levels of the attributes were chosen to reflect different scenarios that occurred during the first year of the pandemic, as described in section 2. The low intensity level contemplated only large events’ restrictions and bars and discos closed at night; the medium level was characterised by a mandatory curfew from 23h00 do 5h00 and gatherings restricted to five persons; the high level was a full lockdown, with people working from home, circulation restrictions and schools shut down.

The economic collapse driven by life restriction measures has led to an inevitable loss of global wealth. The potential measures to fight the pandemic ended up pushing people into poverty. The fifth attribute was the percentage of the population in risk of poverty. The levels for this attribute are 25%, 35% and 45%. The maximum level corresponded to the approximate percentage of the population at risk of poverty before any social transfer, observed in the past 10 years [26]. This would correspond to a catastrophic scenario where social transfers would be unavailable.

3.3. Survey

The project was approved by the NOVA School of Business and Economics Ethics Committee on the 28th of December 2020. When opting to answer the survey, the respondents have consented to the publication of potential findings.

The data was collected through an online survey between January 12th and March 14th, 2021. Most observations were collected during January. Any adult living in Portugal in this period was eligible to answer. The survey was distributed using social media and mailing lists of the University of Porto and Nova School of Business and Economics (in Lisbon). The survey was created in Qualtrics and divided in two parts. In the first part, respondents were asked about their socio-demographic characteristics. The second part implemented a DCE with 16 choice sets, separated in two blocks. Participants were randomly allocated to respond to one of the two blocks of 8 choice sets. The alternatives for each set were generated by a D-Optimal design [27,28]. We conducted a pilot study between January 2nd and January 6th with 73 respondents. The pilot was used to pre-test our survey and adjust the D-Optimal design parameters. Since no changes were made to the survey, the pilot is also included in the analysis.

Table 2 shows a choice set translated to English (an original example and instructions in Portuguese can be found in S1 and S2 Figs in S1 Appendix). Before starting the DCE, respondents were informed about the scope of the survey. It was clearly stated that alternatives were hypothetical, and that respondents should disregard the potential relation between attributes. They were instructed to consider that effects would last for six months, between January and June of 2021. Respondents were also given a brief instruction on the measures included in each life restriction attribute level and on how to interpret the probability values in the other three attributes. In the first choice set respondents were given the alternative to quit.

Table 2. Example of a choice set.

For the period between January and June 2021 Option A Option B
Daily deaths attributed to COVID-19, directly or indirectly (e.g., delay on healthcare provision) 150 50
% decrease on household income motivated by the pandemic 20 10
% of students permanently affected by the pandemic 10 20
Level of restrictions to day-to-day life* High Low
% of the population in risk of poverty (living with < €6/day) 25 35

*Low: Large-events restrictions and no bars/clubs; Medium: Night curfew and maximum 5 people gatherings; High: Full lockdown.

3.4. Population in the survey

The survey had a total of 2,191 respondents. Key descriptive statistics are displayed in Table 3. Respondents’ age ranged from18 years old up to over and 63% of our sample was represented by individuals below 35 years old. Most of the respondents were women (approximately 63%) and had a bachelor’s degree (approximately 32%), or a master’s degree or higher (approximately 40%). Individuals in the sample displayed a median monthly equivalised income of around €1,250. The poorer 25% (excluding missing answers) received an equivalent household income of less than €750 per individual; the 25%-50% equivalent household income quartile corresponded to €751-€1,250 per individual; the 50%-75% quartile was between €1,251-€1,750 per individual; and the 75%-100% quartile was above €1,750.

Table 3. Key descriptive statistics for the sample and comparison with the Portuguese population.

SAMPLE PORTUGAL
N (%) N (%)
Age
19–25 years 646 (29.48) 557,119 (5.41)
26–35 years 739 (33.73) 1,114,060 (10.82)
36–45 years 426 (19.44) 1,435,773 (13.94)
46–55 years 219 (10.00) 803,727 (7.81)
56–65 years 135 (6.16) 1,423,032 (13.82)
66–75 years 21 (0.96) 1,178,309 (11.44)
76–85 years 4 (0.18) 788,663 (7.66)
> 85 years 1 (0.05) 328,066 (3.19)
No answer - -
Gender
Female 1,379 (62.94) 5,430,098 (52.48)
Male 807 (36.83) 4,917,794 (47.52)
No answer 5 (0.23) -
Education attainment level
No college Degree1 579 (26.43) -78.8
Bachelor’s degree2 711 (32.45) (21.2)3
Master’s Degree or Higher4 894 (40.80)
No answer 7 (0.32)
Monthly equivalent household income
< €750 497 (22.68) N/A5
€751-€1,250 487 (22.23)
€1,251-€1,750 515 (23.51)
> €1,751 518 (23.64)
No answer 174 (7.94)
Area of residence
Porto 917 (41.85) 1,729,390 (16.80)
Lisbon 512 (23.37) 2,270,980 (22.05)
Other 762 (34.78) 6,296,711 (61.15)
Number of persons in household
Mean 3.01 2,5
Number of children in household
Younger than 6 285 (13.01) N/A5
Both older and younger than 6 101 (4.61)
Older than 6 446 (20.36)
None 1,287 (58.74)
No answer 72 (3.29)
Occupation
Unemployed 108 (4.93) N/A5
Student 620 (28.30)
Student-Worker 21 (0.96)
Researcher 34 (1.55)
Public servant 457 (20.86)
Retired 33 (1.51)
Large enterprise employee 326 (14.88)
SME employee 375 (17.12)
Self-employed 194 (8.85)
Other 9 (0.41)
No answer 14 (0.64)
Home Office
No 1,070 (48.84) N/A5
Yes 1,121 (51.16)
No answer -

1 - Levels 0, 1, 2, and 3 (ISCED).

2 - Level 6 (ISCED).

3 - Information on University Education was provided as an aggregate measure.

4 - Level 7 and 8 (ISCED).

5 - Non-Available Information.

Nearly 42% of the individuals in the sample were living in Porto in comparison with approximately 23% of the respondents living in Lisbon and the remaining elsewhere in the country. Households presented a mean of three individuals, and approximately 60% of all households had no children. Our sample was mainly constituted by students (approximately 28%) and public servants (approximately 21%). Half of the sample (approximately 51%) was working from home at the time of the survey.

3.5. Analysis

We performed a fourfold analysis. First, we ran a conditional logit model [29], with all attribute levels coded as binary. Second, we re-estimated the model with number of daily deaths coded as a continuous variable, to compute the MRS using deaths as a numeraire. Third, we performed unconditional subgroup analysis, through the estimation of a conditional logit for the relevant characteristics of the population collected in the survey. Fourth, we conducted a latent class analysis [30,31] to account for class preferences’ heterogeneity. We tested models from 2 up to 10 latent classes. Three classes were selected to be presented in the manuscript based on the Consistent Akaike information criterion (CAIC) and the Bayesian information criterion (BIC).

Data analysis and the creation of the D-optimal design were made with STATA 17 software.

4. Results

4.1. Main results

Fig 1 shows the results of the conditional logit model and respective MRS. As expected, coefficients for high and medium levels of the attributes are negative and statistically significantly different from the low level, which serves as the baseline level. However, despite the difference relative to the low level, coefficients for medium and high levels for the education and life restrictions attributes have similar magnitudes.

Fig 1. Coefficients from the main conditional logit model and the respective MRS.

Fig 1

Note: S2 Table in S1 Appendix, displays the numbers supporting these graphs.

From the relative size of coefficients for different attributes we can conclude that 250 daily deaths due to COVID-19 was the attribute level that affected preferences the most (relative to 50 deaths due to COVID-19). Considering only the maximum levels of each attribute, next comes 45% of the population at the risk of poverty (relative to 25%), followed by 30% loss of respondent’s household income (relative to 10%), education impairment in 30% of the school population (relative to 10%), and the highest level of life restrictions–a full lockdown (relative to mild life restrictions).

Based on the analysis of the MRS, the respondents in this survey were willing to sacrifice 30% of their income (instead of 10%) for six months to avoid approximately 47 daily deaths during the same period. They would also accept 30% of the school population to become educationally impaired (instead of 10%) to avoid approximately 25 daily deaths; going from a situation in which there were only restrictions to large-events, bars, and clubs to a full lockdown to avoid approximately 24 daily deaths; and 45% of the population to be at the risk of poverty (instead of 25%) to avoid roughly 101 daily deaths, all for six months.

4.2. Subgroup analysis

Fig 2 shows the MRS for a range of subgroups. While estimates of MRS (coefficients available in S3 Table in S1 Appendix) differed in magnitude for several of the characteristic examined, we were reluctant to interpret those without a statistically significant difference. There were, however, two characteristics for which we find statistically significant differences in preferences: gender and working location during the pandemic.

Fig 2. Marginal rate of substitution (MRS) by subgroups.

Fig 2

In Fig 2, we see that women had lower MRS than men for most attributes, significantly different in terms of life restrictions and higher level of educational impairment. For example, women were willing to accept a stricter lockdown, instead of milder restrictions, to avoid approximately eight daily deaths, whereas men were willing to accept a stricter lockdown, instead of milder restrictions, to avoid approximately 53 daily deaths.

The same pattern was observed in the workplace subgroup (first graph of the second column), where people working on site have statistically significantly lower MRS for the maximum levels of education impairment and life restrictions than those working from home. For example, people working on site were willing to accept a stricter lockdown, instead of milder restrictions, to avoid approximately 15 daily deaths, whereas people working from home were willing to accept a stricter lockdown, instead of milder restrictions, to avoid approximately 32 daily deaths.

4.3. Latent class model

Three latent classes of patient preferences were identified. Results are shown in Table 4. People with Class 1 preferences, who make 28% of the sample, gave the lowest importance to COVID-19 deaths. According to Fig 3, they gave higher relative value to education, life restrictions, and risk of poverty than the other two classes. People with Class 1 preferences were more likely to be male, work remotely, and to be part of a household with only children older than six years old.

Table 4. Coefficients from the latent class model with 3 classes.

Class1 Class2 Class3
Attribute Levels Coeff 95% CI Coeff 95% CI Coeff 95% CI
Deaths—150 -0.47 -0.62 -0.32 -1.31 -1.52 -1.11 -4.15 -4.94 -3.35
Deaths—250 -0.55 -0.81 -0.29 -2.64 -3.04 -2.23 -12.47 -15.09 -9.86
Household income lost—20% -0.63 -0.84 -0.42 -0.16 -0.28 -0.04 0.60 -0.62 1.83
Household income lost—30% -1.28 -1.55 -1.01 -0.60 -0.73 -0.47 -3.10 -4.21 -1.99
Compromised education—20% -0.42 -0.60 -0.25 -0.41 -0.53 -0.28 -1.13 -1.89 -0.37
Compromised education—30% -0.90 -1.13 -0.68 -0.54 -0.67 -0.41 3.91 2.71 5.10
Life restrictions—medium -0.32 -0.49 -0.14 0.00 -0.11 0.10 0.55 0.11 0.99
Life restrictions—high -0.07 -0.23 0.09 -0.18 -0.31 -0.06 0.60 -0.14 1.33
Risk of poverty—35% -1.21 -1.49 -0.93 -0.56 -0.68 -0.45 4.36 3.03 5.69
Risk of poverty—45% -1.60 -1.85 -1.36 -1.51 -1.68 -1.34 -4.02 -5.38 -2.67
Class membership model
Gender: Female (reference) Reference Class
Gender: Male 0.49 0.20 0.78 0.04 -0.25 0.33
Age: 18–25 (reference)
Age: 26–45 -0.24 -0.72 0.25 -0.40 -0.89 0.09
Age: >45 -0.58 -1.14 -0.03 -1.01 -1.59 -0.43
Working type: On site (reference)
Working type: Remote 0.31 0.03 0.59 -0.09 -0.36 0.19
Region: Other (reference)
Region: Porto -0.13 -0.44 0.18 -0.33 -0.64 -0.02
Region: Lisboa 0.05 -0.31 0.42 -0.05 -0.42 0.33
Children in HH: <6
Children in HH: <6 & >6 0.09 -0.60 0.77 -0.03 -0.73 0.66
Children in HH: Only >6 0.63 0.13 1.13 0.40 -0.11 0.90
Children in HH: None 0.38 -0.05 0.81 0.31 -0.11 0.74
Education: No college degree (reference)
Education: Bachelors 0.19 -0.19 0.57 -0.17 -0.54 0.19
Education: Masters+ 0.04 -0.37 0.46 -0.36 -0.77 0.05
HH eq. income: < €750
HH eq. income: €751 – €1,250 -0.10 -0.47 0.27 0.00 -0.38 0.37
HH eq. income: €1,251-€2,000 0.03 -0.34 0.41 0.25 -0.14 0.64
HH eq. income: >€2,001 0.13 -0.26 0.52 0.34 -0.07 0.75
Student: No (reference)
Student: Yes -0.09 -0.57 0.39 -0.09 -0.58 0.39
Constant -0.61 -1.42 0.20 0.69 0.03 1.34
Class share 0.28 0.41 0.31

Numbers in bold stand for statistically significant coefficients at 95% confidence level.

Number of observations: 33,808; Log likelihood = -8242.7859; AIC: 16609.61 CAIC: 17024.52; BIC: 16962.52.

Fig 3. Relative importance of attributes per class.

Fig 3

Note: The relative importance is computed by taking the absolute value of the maximum level of each attribute and dividing them by the sum of the absolute value of all the maximum levels of each attribute, per class [32]. The downward bars for “Education” and “Life” attributes in Class 3 are purely illustrative, to convey that the valuation given to those attributes’ levels are contrary to rationally consistent preferences.

People with Class 2 preferences, representing 41% of the sample, were characterized by a higher valuation of COVID-19 deaths, when compared to people with Class 1 preferences. They also had a lower valuation for household income losses, when compared with people from Class 1 and Class 3. Fig 3 shows that they gave a relatively lower importance to education, life restrictions and poverty, then people with Class 1 preferences. People with Class 2 preferences were less likely to be aged 45 years or more and to be from Porto.

Individuals with Class 3 preferences, representing 31% of the sample, had the strongest valuation toward all levels of the deaths’ attribute. However, they displayed rationally inconsistent results for the education impairment and life restriction attributes. Fig 3 shows that people in this class gave the most relative importance to deaths. People from Class 3 were more likely to be older than 45 years than people with Class 1 or 2 preferences.

5. Discussion

We found a high number of deaths due to COVID-19 to be the most impactful attribute on individual’s preferences. Respondents were willing to trade-off considerable impacts on their lives in terms of income loss and limitations to individual freedom, as well as impacts for the society in terms of educational impairment and poverty, to avoid going from 50 to 250 daily deaths related to COVID-19. Levels of poverty were the second attribute in priority for our respondents, with individuals’ household income losses coming in third. While the individual household income loss reported a more selfish view of the economic impacts of the pandemic, the levels of poverty captured a more altruistic perspective. The high importance given to poverty levels suggests that the respondents were very concerned with the pandemic effects on the economical capacity of their countrymen. Subgroup results suggest that women presented lower MRS than men for the education and life restriction attributes. S4 Table in S1 Appendix, shows that these results are driven by both a stronger negative effect of deaths and a weaker negative effect of the two attributes on women’s preferences, when compared to men. Several studies conducted have found general evidence that women reported to agree more and be more compliant with the restrictions imposed in different countries [3335]. Women reported greater fear, more negative expectations about health-related consequences and were more likely to consider COVID-19 infection a serious health problem. Moreover, during the pandemic, women reported higher levels of stress and concern, as well as higher levels of overwork and informal care, compared to their male counterparts [3638]. Our results also support this evidence since being more worried naturally increases one’s MRS to stop the spreading of the virus and avoid deaths. The gender differences found in our results might also be reflecting the sample selection bias induced in our data. There is, for example, a larger proportion of women than men working on site in our sample (67.38% vs. 32.62%), as shown in S1 Table in S1 Appendix. Our findings also suggest that the MRS of those working remotely was twice as high as the ones working on site, for the maximum level of the life restrictions’ attribute. These results are compatible with potential saturation felt by individuals in remote working; but they might also result from lower exposition to risk. At the same time, this can also be explained by the largest share of female respondents working on-site. Further investigation would be needed to disentangle these two potential effects.

The latent class analysis supported, to some extent, the results found in the unconditional sub-group analysis. People with Class 1 preferences were more likely to be males and working from home. While the differences were now in different attributes, it remains that men and people working from home tended to have different preferences from women and people working on site. Also, the latent class model identified a class where people were more likely to give a major importance to the deaths’ attribute, to a point where only the higher levels of the household income loss and population in risk of poverty had rationally consistent coefficients. As this is not observed in other studies, such as in Chorus (2020) [8], we believe it may be a consequence of our analysis encompassing a period of higher emergency. The class in question had a larger share of people over 45 years old which may indicate that older people developed a bigger sense of urgency than younger people, on average.

Our results are broadly in line with other DCE studies focusing on a similar question in the Netherlands [8], United States of America [12], Australia [10] and Germany [11], even though these studies collected responses at an earlier phase of the pandemic: Chorus (2020) [8] in April 2020, Reed (2020) [12] in May 2020, Manipis (2020) [10] between July and August 2020, Li (2021) [9] end of August 2020 and Mühlbacher (2022) [11] from October to November 2020). Although there are some differences between the domains studied, attributes related to COVID-19 excess mortality were consistently those with a larger effect on individual’s preferences. An exception was the German sample studied by Mühlbacher (2022) [11], in which the attribute related to a decrease of individual income was the most important. Most studies also pointed to considerable heterogeneity depending on individuals’ characteristics. Chorus et al. (2020) [8] found older and more highly educated Dutch citizens to be less willing to sacrifice domains as mental health, educational disadvantage, and income loss to avoid fatalities; while Li et al. (2021) [9] identified differences by age, unemployment status and ability to work from home. In line with our results, the authors found respondents who can work from home to be less willing to sacrifice some attributes, namely, to have lower willingness to stay at home to reduce unemployment and number of COVID-19 daily cases.

An important limitation of this study is the survey’s lack of representativeness, which resulted from the fact that it was collected online with two universities as key channels of dissemination. Therefore, the survey ended up reaching a larger share of individuals with higher education than that of the general population. This bias may also imply a higher proportion of high-income individuals in the sample.

Preferences displayed by individuals are likely to be affected by the timing of the study. This survey was distributed during the worst wave of the pandemic in Portugal, at a time where cases and deaths were growing fast, and the Portuguese NHS was displaying strong signals of being disrupted. While having information about preferences in the peak of the pandemic gives us important insight about decision-making under extreme situations of crisis, our results may be capturing some sense of urgency, potentially not appropriate for predicting preferences in other periods.

Moreover, the COVID-19 pandemic was characterised by substantial uncertainty. The DCE setting implies using fixed levels for each attribute, throughout the experiment. These levels are defined based on the literature and on existing expectations. Some of these levels may become less optimal at a given moment, considering ongoing changes in the pandemic situation. For instance, the highest level for deaths considered in our DCE, was below the maximum level observed afterwards. Combined with the required time to design and implement, DCEs do not present substantial flexibility in volatile contexts. Additionally, during the survey collection period, the vaccination program was already being rolled out. Even though the vaccination program could potentially affect one’s preferences, we do not expect this to have substantially impacted the results. In fact, in the peak of the third wave, only 2.5% of the population had received a first dose.

In the specific case of the third COVID-19 wave in Portugal our results suggest our sample supported the decisions adopted by the Portuguese government, which prioritised lives over the economy and lifestyle, by implementing a second full lockdown in the country. This is an interesting finding, particularly considering that just before the onset of the third wave there were some signs of behavioural fatigue in the Portuguese population. In this context our findings might suggest that individuals adapt their preferences quickly when facing serious health hazards. Depending on how many deaths we assume that a lockdown could avoid, we may use our MRS results for some back-of-the-envelope calculations on the quantitative level of support of our respondents. For example, according to Flaxman et al. (2020) [39] the lockdown during the first wave of the pandemic saved around 3.1 million lives, including 470,000 in the UK, 690,000 in France and 630,000 in Italy. Even though they do not contemplate Portugal, rescaling those numbers would yield approximately 1,000 daily deaths avoided in Portugal for the same period, which is much higher than the combined MRS for all the maximum levels of the other four attributes.

Last, and while our study is a product of a convenience survey with limited generalizability, it is a first step in gathering information about the public preferences for pandemic outcomes and support for government decision-making at a time of crisis. Additionally, no feedback regarding the completion of the DCE was collected from respondents. Overall, and despite these limitations, we show that DCE are instruments available to help guide policy decision making and we defend that they are a flexible and efficient tool to systematically assess implicit preferences of the population.

6. Conclusion

Our results show that, during the peak of the pandemic in Portugal, individuals in this sample were willing to sacrifice substantial amounts of their income, and everyday life freedom, as well as educational outcomes and levels of poverty in the society, to avoid the daily toll of COVID-19 deaths happening at that time. Additionally, findings from the heterogeneity analysis suggest that preferences varied with characteristics such as gender and place of work. These results provide some insights to inform future research on the topic and confirm that DCEs are a useful and flexible tool to incorporate public preferences and behaviours on the design of new policies, particularly, in the context of a crisis.

Supporting information

S1 Appendix

(PDF)

Acknowledgments

We are very grateful to Samare Huls (Erasmus University Rotterdam), Sara Machado (London School of Economics), Diogo Nogueira Leite (Faculty of Medicine, University of Porto), Nova Health Economics and Management Knowledge Centre, the Portuguese Association of Health Economics, the Health Economists’ Study Group, and the Erasmus Choice Modelling Centre.

Data Availability

The dataset is anonymous and does not allow the identification of individual respondents. However, based on the informed consent signed by respondents and on the authorization granted by NOVA School of Business and Economics Ethics Committee, data cannot be shared in a public repository to respect data protection regulation. A minimal data set for replication of the study findings is available upon request to the NOVA School of Business and Economics Ethics Committee (research.office@novasbe.pt) and the corresponding author.

Funding Statement

This report is independent research funded by the National Institute for Health Research (NIHR) Applied Research Collaboration North West Coast (ARC NWC). The views expressed in this publication are those of the authors and not necessarily those of the NHS, NIHR, or the Department of Health and Social Care. There are neither financial nor non-financial conflicts. No ethical considerations apply.

References

  • 1.Fernandes N., “Economic Effects of Coronavirus Outbreak (COVID-19) on the World Economy.” Rochester, NY, Mar. 22, 2020. doi: 10.2139/ssrn.3557504 [DOI] [Google Scholar]
  • 2.OECD, “The state of education during the COVID pandemic—OECD,” OECD Publ. Paris, 2021, Accessed: Aug. 29, 2022. [Online]. https://www.oecd.org/education/state-of-school-education-one-year-into-covid.htm. [Google Scholar]
  • 3.Craig B. M., Lancsar E., Mühlbacher A. C., Brown D. S., and Ostermann J., “Health preference research: an overview,” Patient—Patient-Centered Outcomes Res., vol. 10, no. 4, pp. 507–510, Aug. 2017, doi: 10.1007/s40271-017-0253-9 [DOI] [PubMed] [Google Scholar]
  • 4.Jacobs M., “Accounting for Changing Tastes: Approaches to Explaining Unstable Individual Preferences,” Rev. Econ., vol. 67, no. 2, pp. 121–183, Aug. 2016, doi: 10.1515/roe-2015-1007 [DOI] [Google Scholar]
  • 5.Wang Y., Wang Z., Wang Z., Li X., Pang X., and Wang S., “Application of Discrete Choice Experiment in Health Care: A Bibliometric Analysis,” Front. Public Health, vol. 9, 2021, Accessed: Aug. 29, 2022. [Online]. https://www.frontiersin.org/articles/10.3389/fpubh.2021.673698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Rosenthal U. and Hart P., “Experts and Decision Makers in Crisis Situations,” Knowledge, vol. 12, no. 4, pp. 350–372, Jun. 1991, doi: 10.1177/107554709101200402 [DOI] [Google Scholar]
  • 7.Sotiropoulos D. A., “Southern European Governments and Public Bureaucracies in the Context of Economic Crisis,” Eur. J. Soc. Secur., vol. 17, no. 2, pp. 226–245, Jun. 2015, doi: 10.1177/138826271501700205 [DOI] [Google Scholar]
  • 8.Chorus C., Sandorf E. D., and Mouter N., “Diabolical dilemmas of COVID-19: An empirical study into Dutch society’s trade-offs between health impacts and other effects of the lockdown,” PLOS ONE, vol. 15, no. 9, p. e0238683, Sep. 2020, doi: 10.1371/journal.pone.0238683 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Li L., Long D., Rad M. R., and Sloggy M. R., “Stay-at-home orders and the willingness to stay home during the COVID-19 pandemic: A stated-preference discrete choice experiment,” PLOS ONE, vol. 16, no. 7, p. e0253910, Jul. 2021, doi: 10.1371/journal.pone.0253910 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Manipis K., Street D., Cronin P., Viney R., and Goodall S., “Exploring the Trade-Off Between Economic and Health Outcomes During a Pandemic: A Discrete Choice Experiment of Lockdown Policies in Australia,” Patient—Patient-Centered Outcomes Res., vol. 14, no. 3, pp. 359–371, May 2021, doi: 10.1007/s40271-021-00503-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mühlbacher A. C., Sadler A., and Jordan Y., “Population preferences for non-pharmaceutical interventions to control the SARS-CoV-2 pandemic: trade-offs among public health, individual rights, and economics,” Eur. J. Health Econ., Feb. 2022, doi: 10.1007/s10198-022-01438-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Reed S., Gonzalez J. M., and Johnson F. R., “Willingness to Accept Trade-Offs Among COVID-19 Cases, Social-Distancing Restrictions, and Economic Impact: A Nationwide US Study,” Value Health, vol. 23, no. 11, pp. 1438–1443, Nov. 2020, doi: 10.1016/j.jval.2020.07.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Dong D. et al., “Public preference for COVID-19 vaccines in China: A discrete choice experiment,” Health Expect., vol. 23, no. 6, pp. 1543–1578, Dec. 2020, doi: 10.1111/hex.13140 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Jonker M., de Bekker-Grob E., Veldwijk J., Goossens L., Bour S., and Rutten-Van Mölken M., “COVID-19 Contact Tracing Apps: Predicted Uptake in the Netherlands Based on a Discrete Choice Experiment,” JMIR MHealth UHealth, vol. 8, no. 10, p. e20741, Oct. 2020, doi: 10.2196/20741 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Luyten J., Tubeuf S., and Kessels R., “Rationing of a scarce life-saving resource: Public preferences for prioritizing COVID-19 vaccination,” Health Econ., vol. 31, no. 2, pp. 342–362, Feb. 2022, doi: 10.1002/hec.4450 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Zimba R. et al., “Patterns of SARS-CoV-2 Testing Preferences in a National Cohort in the United States: Latent Class Analysis of a Discrete Choice Experiment,” JMIR Public Health Surveill., vol. 7, no. 12, p. e32846, Dec. 2021, doi: 10.2196/32846 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.de Blaeij A., Florax R. J. G. M., Rietveld P., and Verhoef E., “The value of statistical life in road safety: a meta-analysis,” Accid. Anal. Prev., vol. 35, no. 6, pp. 973–986, Nov. 2003, doi: 10.1016/s0001-4575(02)00105-7 [DOI] [PubMed] [Google Scholar]
  • 18.Mouter N., van Cranenburgh S., and van Wee B., “Do individuals have different preferences as consumer and citizen? The trade-off between travel time and safety,” Transp. Res. Part Policy Pract., vol. 106, pp. 333–349, Dec. 2017, doi: 10.1016/j.tra.2017.10.003 [DOI] [Google Scholar]
  • 19.Valente de Almeida S., Costa E., Lopes F. V., Santos J. V., and Pita Barros P., “Concerns and adjustments: How the Portuguese population met COVID-19,” PLOS ONE, vol. 15, no. 10, p. e0240500, Oct. 2020, doi: 10.1371/journal.pone.0240500 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.European Centre for Disease Prevention and Control, “Data on country response measures to COVID-19,” European Centre for Disease Prevention and Control, Aug. 25, 2022. https://www.ecdc.europa.eu/en/publications-data/download-data-response-measures-covid-19 (accessed Aug. 29, 2022). [Google Scholar]
  • 21.Almeida V., Barrios S., Christl M., De Poli S., Tumino A., and van der Wielen W., “The impact of COVID-19 on households’ income in the EU,” J. Econ. Inequal., vol. 19, no. 3, pp. 413–431, Sep. 2021, doi: 10.1007/s10888-021-09485-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Cantó O. et al., “Welfare Resilience at the Onset of the COVID-19 Pandemic in a Selection of European Countries: Impact on Public Finance and Household Incomes,” Rev. Income Wealth, vol. 68, no. 2, pp. 293–322, Jun. 2022, doi: 10.1111/roiw.12530 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Gama A. et al., “Double jeopardy from the COVID-19 pandemic: risk of exposure and income loss in Portugal,” Int. J. Equity Health, vol. 20, no. 1, p. 231, Oct. 2021, doi: 10.1186/s12939-021-01569-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Azevedo J. P., Amer H., Goldemberg D., Koen G., and Iqbal S. A., “OUP accepted manuscript,” World Bank Res. Obs., vol. 36, no. 1, pp. 1–40, 2021. [Google Scholar]
  • 25.Pancani L., Marinucci M., Aureli N., and Riva P., “Forced Social Isolation and Mental Health: A Study on 1,006 Italians Under COVID-19 Lockdown,” Front. Psychol., vol. 12, p. 663799, May 2021, doi: 10.3389/fpsyg.2021.663799 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Pordata, “Taxa de risco de pobreza: antes e após transferências sociais,” Base de dados Portugal contemporâneo, 2022. https://www.pordata.pt/Portugal/Taxa+de+risco+de+pobreza+antes+e+ap%C3%B3s+transfer%C3%AAncias+sociais-2399 (accessed Aug. 29, 2022).
  • 27.Carlsson F. and Martinsson P., “Design techniques for stated preference methods in health economics,” Health Econ., vol. 12, no. 4, pp. 281–294, Apr. 2003, doi: 10.1002/hec.729 [DOI] [PubMed] [Google Scholar]
  • 28.Cook R. D. and Nachtrheim C. J., “A comparison of algorithms for constructing exact d-optimal designs,” Technometrics, vol. 22, no. 3, pp. 315–324, Aug. 1980, doi: 10.1080/00401706.1980.10486162 [DOI] [Google Scholar]
  • 29.D. McFadden, “Conditional logit analysis of qualitative choice behavior,” 1973.
  • 30.J. A. Hagenaars and A. L. McCutcheon, “Applied Latent Class Analysis,” Cambridge Core, Jun. 2002. https://www.cambridge.org/core/books/applied-latent-class-analysis/30C364913C52083262DD7CE5A2E05685 (accessed Sep. 04, 2022).
  • 31.Hole A. R., “Modelling heterogeneity in patients’ preferences for the attributes of a general practitioner appointment,” J. Health Econ., vol. 27, no. 4, pp. 1078–1094, Jul. 2008, doi: 10.1016/j.jhealeco.2007.11.006 [DOI] [PubMed] [Google Scholar]
  • 32.Arslan I. G. et al., “Patients’, healthcare providers’, and insurance company employees’ preferences for knee and hip osteoarthritis care: a discrete choice experiment,” Osteoarthritis Cartilage, vol. 28, no. 10, pp. 1316–1324, Oct. 2020, doi: 10.1016/j.joca.2020.07.002 [DOI] [PubMed] [Google Scholar]
  • 33.Alsharawy A., Spoon R., Smith A., and Ball S., “Gender Differences in Fear and Risk Perception During the COVID-19 Pandemic,” Front. Psychol., vol. 12, p. 689467, Aug. 2021, doi: 10.3389/fpsyg.2021.689467 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Galasso V., Pons V., Profeta P., Becher M., Brouard S., and Foucault M., “Gender differences in COVID-19 attitudes and behavior: Panel evidence from eight countries,” Proc. Natl. Acad. Sci., vol. 117, no. 44, pp. 27285–27291, Nov. 2020, doi: 10.1073/pnas.2012520117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.New York University, “Women more likely to embrace behaviors aimed at preventing the spread of COVID-19,” ScienceDaily, 2022. https://www.sciencedaily.com/releases/2020/10/201005092343.htm (accessed Aug. 29, 2022).
  • 36.Office for National Statistics, “Coronavirus (COVID-19) and the different effects on men and women in the UK, March 2020 to February 2021—Office for National Statistics,” 2022. https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/articles/coronaviruscovid19andthedifferenteffectsonmenandwomenintheukmarch2020tofebruary2021/2021-03-10 (accessed Aug. 29, 2022).
  • 37.Raiber K. and Verbakel E., “Are the gender gaps in informal caregiving intensity and burden closing due to the COVID-19 pandemic? Evidence from the Netherlands,” Gend. Work Organ., vol. 28, no. 5, pp. 1926–1936, Sep. 2021, doi: 10.1111/gwao.12725 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Smyth L., “The burden of informal care on women: how has it increased during COVID-19, and what are the implications for women’s working lives?,” Nov. 2021. [Google Scholar]
  • 39.Flaxman S. et al., “Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe,” Nature, vol. 584, no. 7820, Art. no. 7820, Aug. 2020, doi: 10.1038/s41586-020-2405-7 [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Karyn Morrissey

28 Jul 2022

PONE-D-22-17412Opening to deaths: A discrete choice experiment about Covid-19’s policy preferences in PortugalPLOS ONE

Dear Dr. Filipe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 11 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Karyn Morrissey

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"We are grateful to Nova Health Economics and Management Knowledge Centre, the Portuguese

Association of Health Economics and the Erasmus Choice Modelling Centre. This report is independent research funded by the National Institute for Health Research (NIHR) Applied Research Collaboration North West Coast (ARC NWC). The views expressed in this publication are those of the authors and not necessarily those of the NHS, NIHR, or the Department of Health and Social Care. There are neither financial nor non-financial conflicts. No ethical considerations apply."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"This report is independent research funded by the National Institute for Health Research (NIHR) Applied Research Collaboration North West Coast (ARC NWC). The views expressed in this publication are those of the authors and not necessarily those of the NHS, NIHR, or the Department of Health and Social Care. There are neither financial nor non-financial conflicts. No ethical considerations apply."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical.

4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. 

5. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Comments for the Author

Thank you for the opportunity to review this manuscript. This study used DCE methods to examine the preferences pertaining to consequences of COVID-19 policies in Portugal. There have been other studies using DCEs (in other countries) that have examined consequences from COVID-19 policies, however the manuscript has referenced these studies appropriately. This study offers a different perspective from other the DCEs investigating COVID-19 policies in that the study is unique for the experience in Portugal, but more interestingly, they have conducted the study at a later timepoint where the population have already experienced prior lockdowns/COVID-19 restrictions, thereby capturing sentiment in subsequent waves. There are comments, which should be addressed by the authors, and changes that need clarification.

General comments

• The term ‘willingness to pay (WTP)’ is used to describe the marginal rate of substitution (MRS) for attribute levels, using death (as a continuous variable) as the numeraire. Using the term WTP is confusing as a cost attribute were not used in the DCE. The authors should change the wording throughout the manuscript to reflect that the MRS were measured rather than WTP.

• Data were collected when vaccination were available. Did the survey collect data about vaccination? Is this a factor that may potentially affect the results given that this study reports on the preferences of subsequent waves? This could be included in the discussion section.

Section comments

Introduction

• Line 63: “We find that avoiding deaths ……” is one of the results found. Please delete this line from the introduction.

• Lines 64-65: see comment below regarding the data analysis (under methodology) pertaining to method used to assess heterogeneity.

Methodology

• Data analysis: The major limitation in this manuscript pertains to the analysis used, which was conditional logit modelling, followed by subgroup analyses to explore heterogeneity. Although conditional logit models are typically used to initially analyse DCE data, and there are better methods that could be used when exploring heterogeneity. The authors should consider using other methods e.g., mixed logit modelling or latent class analysis, to explore heterogeneity within the sample. I’ve noted some references the authors may wish to review1-4. Including these analyses would require a substantial change to the manuscript; however, this would improve the manuscript and interpretation pertaining to heterogeneity.

• Under statistical analysis, please state the software used to analyse the data; and under survey (line 171-172) include the software used to create the D-optimal design.

• Section 3.2:

o Line 117-118: “The attributes were chosen based on international literature (Chorus 2020 and Reed 2020)”; as these are DCEs it would be better to state that other DCEs were used to inform this study.

o Line 120: Change “attributed” to “attribute”.

o Line 120-122: Is there a reference that could be included referencing the policies at the time of the survey?

• Survey, line 173: Were any changes made after the pilot study? If yes, please note the changes that were made. Was the data from the pilot used in the final analysis of the DCE?

• Table S1: move the example of the choice set into the main text rather than the appendix. Noting that the choice question is likely in Portuguese, could you translate and include the question respondents were asked to answer in their choice task? This is good for a reader to get an overview of what was explicitly asked to do in the DCE.

• Line 186-187: The sentence “We divided our analysis of the respondents characteristics into three groups: individual, household, and work-related aspects” is unclear. Can you clarify what this refers to and how these groupings were used? I’m assuming each term is an umbrella term for other more specific terms, which should be clarified in the text.

• Lines 193-195: Euros are denoted after the numeric figure and should be placed before the number.

• Did you collect any information from the respondents about their experience in completing this DCE? Was there any feedback they had given about any of the survey? If not, add as a note in the limitations section.

• Lines 215-217: The results for the conditional logit indicate that the levels for death were treated categorically. Can you please clarify this in the text? This is clarified for the MRS analysis, but not the conditional logit analysis.

• Section 3.4: the model equations available in the appendix are perhaps not needed. It may be better to reference a textbook which readers can refer to. Alternatively, specify the equations to be specific for this DCE if they wish to retain.

Results

• The main results section may need to be rewritten. The difference in the levels should be explained relative to the base level, which is not clear in the text, especially if readers are not familiar with DCEs. For example, relative to 50 deaths, respondents do not prefer 250 deaths, or something to that effect.

• line 226: Reword text “which serve as baseline” to the “which serves as the baseline level”.

• Line 226-227: “education and life restrictions have statistically indistinguishable coefficients for levels medium and high.” This is unclear, can you please clarify this text.

• Table 3: Both the conditional logit (using death as a categorical variable) and the MRS (using death as a continuous variable) are reported in this table. I suggest merging Table 3 and Table S3 in the appendix to form one table e.g., models across the column headings and attributes and levels as rows, then move this table to the appendix. For the results, graph in a forest plot, as this is better visually for a reader. Please report model statistics models for the results (e.g., number of observations, AIC, BIC, loglikelihood) in the table.

• Figure 1: Willingness-to-pay (WTP) by subgroups: It is difficult to read the bars for each group, can you redo the bars so that teach of the groups are not overlapping? Alternatively present the more interesting one in the main body (gender and workplace), and make these larger, and move other categories to the appendix.

• Line 248-249: Two characteristics are noted as having statistically significant dfferences in preference. Under analysis, can you include what tests were run to determine statistical signifcance and report the results in the text in this section.

• Line 253: as you have used a continuous variable, please rephrase the sentence such that the “lower number of deaths avoided” is quantified as per the MRS.

Discussion

• Line 262 and line 290: Change the term “utility” to “preferences” as this was the outcome being estimated. Please change all instances throughout manuscript.

• Line 302: A comparison with the general population is made with reference to income. However, the baseline characteristics table does not report the comparative income for the general population. I can understand this may be the case, but I think this should be rephrased somewhat or a reference included noting what the mean or median income is for Portugal.

• The higher risk of poverty is an interesting finding pertaining to the MRS results. The 45% level being described as a catastrophic scenario and the implications, could be described in more detail in the discussion. If space is an issue, the discussion pertaining to women could be shortened.

• Line 272-274: Rephrase sentence “During the pandemic ….”

• Another limitations to include in the discussion is that the highest level for deaths in the DCE was below the maximum level.

1 Lancsar E, Fiebig DG, Hole AR. Discrete Choice Experiments: A Guide to Model Specification, Estimation and Software. PharmacoEconomics 2017; 35: 697-716.

2 Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health--a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health 2011; 14: 403-413.

3 Hauber AB, González JM, Groothuis-Oudshoorn CG, et al. Statistical Methods for the Analysis of Discrete Choice Experiments: A Report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value Health 2016; 19: 300-315.

4 Reed Johnson F, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health 2013; 16: 3-13.

Reviewer #2: Comments for authors

This is an interesting paper on public preferences for COVID controls using DCEs. I have some major and some minor comments. My major concerns are around the design and implementation of the survey and the way you present some of the results.

Major comments

1. I’m not sure your title “Opening to deaths” really encapsulates the breadth of analysis in your paper.

2. In terms of the duration covered, you used “January to June 2021”, it would be good to see the choice card/full questionnaire in Portuguese to be able to see what the respondents were looking at. I have to say that this is a little bit vague to me and could be interpreted as either being 5 or 6 months depending on how you read it. Was there a reason you chose 6 months? I also find it a bit strange that you are asking about the past as well to some – as this was conducted until March 2021.

3. I have some concerns over the designs and levels. First, can you comment on having both a poverty level and income level in the same choice experiment. These are both income levels at the end of the day – I wonder if you can reflect on that (and if e.g. employment might be better to use if this is to be repeated?).

4. In terms of the educational impairment – why was a % of students used rather than a level of education days lost? The latter seems to be more of a realistic measure. Surely most are affected in some way by a lockdown.

5. In Table 2 you present the descriptive statistics for income as being “household income” but in the text this is presented as being “per individual” – this is a bit confusing. Is there not also a potential issue that this income level has been impacted by COVID - did you consider asking the “pre-COVID” level? This might affect how you interpret the results in terms of the household income loss being presented as percentages… and this is also I imagine the reason you are not able to report national level percentage rates (though you could use the latest Census?).

6. It is good to see that the authors obtained ethical clearance from an ethics board in Portugal. I just want the authors to confirm that there was approval for data collection with those under 18 (as 28.79% of the sample in Table 2 is being reported as being <18). If not explicit, then the authors may need to clarify that they have such approval from the Nova Business School ethics committee – I realise open surveys of this type are hard to control. You might want to rephrase line 347 – there clearly are ethical considerations in this kind of survey!

7. I must say I find it a bit surprising that given the age distribution of the sample that only 28% have no college degree – presumably the majority of under 18s and about half of the next age group would not be of the age to have a college degree.

8. I find it strange that you use “Willingness to Pay” and use a deaths metric as the basis for it (rather than a euro value). Maybe you could rephrase this? “Death acceptance”? (though admittedly there is a difference between “willingness to accept” and “willingness to pay” – I have a feeling here that acceptance may be a better phrase!). When you talk of things like “women have a lower WTP than men” in my mind that suggests they value life less, when in fact this is the opposite!

9. I am wondering if it would be worth thinking about what your results would imply if you translated this (via the income change) to a Value of a Prevented Fatality – a rough calculation I have done suggests the numbers you are getting are pretty low (which might reflect the perception that older and those more likely to die sooner are those most at risk).

10. Given that this was very much a time of much change in terms of COVID regulation and information, I am wondering if you have considered including examining whether these preferences had any time element – e.g. if there was any systematic difference between the responses in March compared to those in January.

11. I think there is a question you may want to think about in terms of uncertainty of a pandemic spread on health. In your experiment you use fixed numbers – when the epidemiological modelling of risked deaths would probably be much higher (given the deaths on a day when things were controlled was higher than your 250 number). It might be worth reflecting on this and the implications this has on the use of DCEs in this context. DCEs do take some time to design and implement/analyse – so compared to focus groups and other forms of surveys this may restrict their applicability. Sampling, as you note, is also important (given your sample is not representative).

Minor comments:

1. Line 27 and Line 326 – I don’t think “auscultate” is quite the right word – my understanding is that this is more a medical term (I will have to admit I had to look up what it meant). Perhaps consider using “assess” instead – which I think conveys what you mean.

2. Line 305 – “the NHS” – not clear what this is (I’m presuming you mean the SNS in Portugal but would be better to specify – as you also use the NHS on line 346 to refer to UK NHS).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Dec 16;17(12):e0278526. doi: 10.1371/journal.pone.0278526.r002

Author response to Decision Letter 0


12 Sep 2022

September 11, 2022

“Trade-offs during the COVID-19 pandemic: a discrete choice experiment about policy preferences in Portugal”

Previously titled as:

“Opening to deaths: A discrete choice experiment about Covid-19’s policy preferences in Portugal”

Dear Editors,

We thank the editors and reviewers for providing us with valuable and constructive feedback, as well as the opportunity to submit a revised version of our manuscript.

We believe that the paper is now clearer and stronger. The main changes in the revised version include:

1. The inclusion of a latent class model analysis to account for class preference heterogeneity

2. Changes in graphs and table to enhance comprehensibility

3. Changes in terminology that improve the technical consistency of the paper

Please find below the detailed answers to the comments raised by each of the reviewers. Note that references to pages and lines are according to the clean version of the manuscript. Thank you for your consideration of our manuscript.

Reviewer #1

General comments

1. The term ‘willingness to pay (WTP)’ is used to describe the marginal rate of substitution (MRS) for attribute levels, using death (as a continuous variable) as the numeraire. Using the term WTP is confusing as a cost attribute were not used in the DCE. The authors should change the wording throughout the manuscript to reflect that the MRS were measured rather than WTP.

Response: Thank you for the excellent comment. We understand that using WTP without a traditional cost attribute may be confusing. As such, we follow the recommendation of the reviewer and substitute WTP by MRS across the entire text.

2. Data were collected when vaccination were available. Did the survey collect data about vaccination? Is this a factor that may potentially affect the results given that this study reports on the preferences of subsequent waves? This could be included in the discussion section.

Response: Thank you for the comment. During the survey collection period, vaccination was at a very early stage - in the peak of the third wave, only about 2.5% of the population had received a first dose of the vaccine. For this reason, no information was collected regarding vaccination status. We do not believe that vaccination affected results substantially for two main reasons. During the study period, the coverage rate of vaccines was very low (and focused only on elderly groups of the population). Moreover, there was no clear consensus regarding the efficacy of the vaccine - which was only verified some months after the survey was completed. We included this in the discussion section (page 22, line 386).

Section comments

Introduction

4. Line 63: “We find that avoiding deaths ……” is one of the results found. Please delete this line from the introduction.

Response: Thank you for noticing, we deleted the sentence.

5. Lines 64-65: see comment below regarding the data analysis (under methodology) pertaining to method used to assess heterogeneity.

Response: Thank you for the excellent comment. Given the reviewer’s following comments we decided to add a latent class model (with 3 classes) to the analysis. We rewrote the introduction section accordingly (page 4, lines 78), and also adjusted the methodology (page 12, line 235) and discussion sections (page 20, line 341).

Methodology

6. Data analysis: The major limitation in this manuscript pertains to the analysis used, which was conditional logit modelling, followed by subgroup analyses to explore heterogeneity. Although conditional logit models are typically used to initially analyse DCE data, and there are better methods that could be used when exploring heterogeneity. The authors should consider using other methods e.g., mixed logit modelling or latent class analysis, to explore heterogeneity within the sample. I’ve noted some references the authors may wish to review1-4. Including these analyses would require a substantial change to the manuscript; however, this would improve the manuscript and interpretation pertaining to heterogeneity.

Response: Thank you for noticing and making such a relevant suggestion. The reviewer is right and a subgroup analysis is not enough to account for preference heterogeneity. We therefore decided to include an extra section with a latent class model with 3 classes (based on CAIC and BIC) (page 17, line 286). Our paper is now in the style of Chorus (2020), with a conditional logit, an unconditional subgroup analysis and a latent class model. We decided in favour of a latent class model in detriment of a mixed logit because we believe that accounting for class preferences’ heterogeneity better fits the problem we are trying to address. We also tested a latent class model with 2 classes. The results are similar to the 3-class model but without the “garbage class” (class 3). Therefore, we decided to refer only to the 3-class model in the text.

7. Under statistical analysis, please state the software used to analyse the data; and under survey (line 171-172) include the software used to create the D-optimal design.

Response: Thank you for your comment. The statistical analysis was conducted using STATA 16 software. This information is now included in the manuscript (in the methodology section).

8. Section 3.2: Line 117-118: “The attributes were chosen based on international literature (Chorus 2020 and Reed 2020)”; as these are DCEs it would be better to state that other DCEs were used to inform this study.

Response: Thank you for noticing, the text was revised based on your suggestion (page 6, line 133).

Line 120: Change “attributed” to “attribute”.

Response: Thank you for your comment. This typo was corrected in the manuscript.

o Line 120-122: Is there a reference that could be included referencing the policies at the time of the survey?

Response: Thank you for your comment. We have included a reference in the manuscript to describe the policies implemented by the Portuguese government during the pandemic.

9. Survey, line 173: Were any changes made after the pilot study? If yes, please note the changes that were made. Was the data from the pilot used in the final analysis of the DCE?

Response: No, no changes were made after the pilot. The pilot was used to inform the priors of the D-Optimal design and was included in the analysis. We added a sentence (page 9, line 196) clarifying that we use the pilot as part of the overall dataset in analysis,

10. Table S1: move the example of the choice set into the main text rather than the appendix. Noting that the choice question is likely in Portuguese, could you translate and include the question respondents were asked to answer in their choice task? This is good for a reader to get an overview of what was explicitly asked to do in the DCE.

Response: Thank you for the excellent comment. The choice set was moved to the main text - in the methodology section. This choice set is translated to English from its original version (in Portuguese). The original version (in Portuguese) was also added to the appendix. Figures S1 and S2 now display the instructions to the DCE and the one choice set with the opt-out possibility.

11. Line 186-187: The sentence “We divided our analysis of the respondents characteristics into three groups: individual, household, and work-related aspects” is unclear. Can you clarify what this refers to and how these groupings were used? I’m assuming each term is an umbrella term for other more specific terms, which should be clarified in the text.

Response: Thank you for your comment. The text was revised and that particular sentence was deleted.

12. Lines 193-195: Euros are denoted after the numeric figure and should be placed before the number.

Response: Thank you for your suggestion. The text was revised accordingly.

13. Did you collect any information from the respondents about their experience in completing this DCE? Was there any feedback they had given about any of the survey? If not, add as a note in the limitations section.

Response: Thank you for your comment. We did not collect feedback regarding the survey completion. This limitation was included in the manuscript (page 23, line 405).

14. Lines 215-217: The results for the conditional logit indicate that the levels for death were treated categorically. Can you please clarify this in the text? This is clarified for the MRS analysis, but not the conditional logit analysis.

Response: Thank you for your comment. The description of the analysis was revised, and included in the methodology section. It now explicitly mentions that the conditional logit model, used for the first analysis, was estimated with all attribute levels coded as binary (page 12, line 236).

15. Section 3.4: the model equations available in the appendix are perhaps not needed. It may be better to reference a textbook which readers can refer to. Alternatively, specify the equations to be specific for this DCE if they wish to retain.

Response: Thank you for your comment. The model equations were removed from the appendix, and references were included in the main text.

Results

16. The main results section may need to be rewritten. The difference in the levels should be explained relative to the base level, which is not clear in the text, especially if readers are not familiar with DCEs. For example, relative to 50 deaths, respondents do not prefer 250 deaths, or something to that effect.

Response: Thank you for your comment. We agree that not including the reference levels in the text could be misleading. The entire section has been altered to always refer to the baseline levels for each attribute.

17. line 226: Reword text “which serve as baseline” to the “which serves as the baseline level”.

Response: Thank you for the suggestion, which we have implemented.

18. Line 226-227: “education and life restrictions have statistically indistinguishable coefficients for levels medium and high.” This is unclear, can you please clarify this text.

Response: Thank you for your comment. The text was revised to clarify this issue.

19. Table 3: Both the conditional logit (using death as a categorical variable) and the MRS (using death as a continuous variable) are reported in this table. I suggest merging Table 3 and Table S3 in the appendix to form one table e.g., models across the column headings and attributes and levels as rows, then move this table to the appendix. For the results, graph in a forest plot, as this is better visually for a reader. Please report model statistics models for the results (e.g., number of observations, AIC, BIC, loglikelihood) in the table.

Response: Thank you for your relevant comment. As suggested, we have moved table 3 to the appendix. Now the table has three columns, showing: first, the coefficients of the model that considers deaths as a categorical variable; second, the coefficients of the model with deaths as a continuous variable; third, the MRS computed from the latter, using deaths as numeraire. In the main text we now present the forest plots for these three columns.

20. Figure 1: Willingness-to-pay (WTP) by subgroups: It is difficult to read the bars for each group, can you redo the bars so that teach of the groups are not overlapping? Alternatively present the more interesting one in the main body (gender and workplace), and make these larger, and move other categories to the appendix.

Response: Thank you, we have updated the figures according to the reviewer’s suggestions.

21. Line 248-249: Two characteristics are noted as having statistically significant differences in preference. Under analysis, can you include what tests were run to determine statistical significance and report the results in the text in this section.

Response: Thank you for your comments. The conclusions were taken visually by looking at the 95% confidence intervals. In order to fit the page, we do not include the confidence intervals in table S3, in appendix. But we believe that with the changes the reviewer suggested they should now be easier to assess in the graphs.

22. Line 253: as you have used a continuous variable, please rephrase the sentence such that the “lower number of deaths avoided” is quantified as per the MRS.

Response: Thank you for your comment. We have rewritten the sentence, to clarify it.

Discussion

23. Line 262 and line 290: Change the term “utility” to “preferences” as this was the outcome being estimated. Please change all instances throughout manuscript.

Response: Thank you for your comment. We have revised the manuscript accordingly.

24. Line 302: A comparison with the general population is made with reference to income. However, the baseline characteristics table does not report the comparative income for the general population. I can understand this may be the case, but I think this should be rephrased somewhat or a reference included noting what the mean or median income is for Portugal.

Response: Thank you for your comment. While we have evidence of the high proportion of high-educated individuals, we are only hypothesizing its link to income. Therefore, we have rewritten the sentence, accordingly.

25. The higher risk of poverty is an interesting finding pertaining to the MRS results. The 45% level being described as a catastrophic scenario and the implications, could be described in more detail in the discussion. If space is an issue, the discussion pertaining to women could be shortened.

Response: Thank you for your comment. We added the discussion about the poverty attribute in the discussion, as suggested (page 19, line 315)

26. Line 272-274: Rephrase sentence “During the pandemic ….”

Response: Thank you for your comment. We have rewritten the sentence.

27. Another limitation to include in the discussion is that the highest level for deaths in the DCE was below the maximum level.

Response: Thank you for your comment. In fact, the COVID-19 pandemic was characterized by substantial uncertainty. The DCE setting implies using fixed levels for each attribute, throughout the experiment. These levels are defined based on the literature and on existing expectations. Some of these levels may become obsolete at a given moment, considering ongoing changes in the pandemic situation. This limitation was now included in the discussion section (page 22, line 379).

Reviewer #2: Comments for authors

Major comments

28. I’m not sure your title “Opening to deaths” really encapsulates the breadth of analysis in your paper.

Response: Thank you for your comment. We adjusted the title of the paper to be more precise.

29. In terms of the duration covered, you used “January to June 2021”, it would be good to see the choice card/full questionnaire in Portuguese to be able to see what the respondents were looking at. I have to say that this is a little bit vague to me and could be interpreted as either being 5 or 6 months depending on how you read it. Was there a reason you chose 6 months? I also find it a bit strange that you are asking about the past as well to some – as this was conducted until March 2021.

Response: Thank you for your comment. The instructions given to the respondents and a choice set (in Portuguese) were added in appendix (Figure S1 and S2). We make it explicit in the instruction that we mean 6 months. The reason to choose 6 months came from our perception of what would be a realistic timeframe for the Covid-19 associated policies, while minimising potential recall bias. In 2020, the restrictions imposed by the governments started to ease during the summer. We assumed they would follow the same trend in 2021.

The reviewer concern with the survey being distributed during the hypothetical period of the experience is a valid one. While a DCE is hypothetical (and we make that clear in the instructions), the respondents could have been influenced by the reality they were living when answering the survey. For example, the reviewer pointed out, in another comment, that the levels of daily deaths surpassed our maximum level at a certain point during the period of the survey. We have extended the discussion to include these topics.

30. I have some concerns over the designs and levels. First, can you comment on having both a poverty level and income level in the same choice experiment. These are both income levels at the end of the day – I wonder if you can reflect on that (and if e.g. employment might be better to use if this is to be repeated?).

Response: Thank you for your comment. When designing the study, we decided to include both attributes (poverty level and income level) since they are expected to capture two different dimensions - which are not necessarily related. While the income level refers to the respondent’s individual income, the poverty level refers to the average society poverty level. While the income attribute captures a more “selfish” perspective, the poverty attribute captures a more “altruistic” dimension.

31. In terms of the educational impairment – why was a % of students used rather than a level of education days lost? The latter seems to be more of a realistic measure. Surely most are affected in some way by a lockdown.

Response: Thank you for your excellent comment. We agree that the number of education days lost is a more realistic measure. However, when designing the study, we were afraid that the average respondent would have difficulties in understanding the concept. For this reason, we opted for using the % of students impaired, which should be seen as a proxy for the number of education days lost. We believe that this variable is easier to grasp by the respondents.

32. In Table 2 you present the descriptive statistics for income as being “household income” but in the text this is presented as being “per individual” – this is a bit confusing. Is there not also a potential issue that this income level has been impacted by COVID - did you consider asking the “pre-COVID” level? This might affect how you interpret the results in terms of the household income loss being presented as percentages… and this is also I imagine the reason you are not able to report national level percentage rates (though you could use the latest Census?).

Response: Thank you for your comment. This is a very interesting point. However, to avoid an overly long survey, we restricted the number of questions asked to the most relevant characteristics. For this reason, no information regarding the pre-Covid income levels was collected. The definition of household income was not clearly explained in the first version of the manuscript. In the survey, we collected information on each respondent’s household income, as well as on household characteristics. We then estimated, based on the size and composition of the household, the equivalised income of each individual. We have rewritten the descriptive statistics accordingly to make this distinction clearer.

33. It is good to see that the authors obtained ethical clearance from an ethics board in Portugal. I just want the authors to confirm that there was approval for data collection with those under 18 (as 28.79% of the sample in Table 2 is being reported as being <18). If not explicit, then the authors may need to clarify that they have such approval from the Nova Business School ethics committee – I realise open surveys of this type are hard to control. You might want to rephrase line 347 – there clearly are ethical considerations in this kind of survey!

Response: Thank you for your comment. In fact, open surveys such as this one are able to collect responses from potentially any individual. The previous version of the manuscript had a typo: the sample included very few answers from individuals under 18. Nonetheless, we eliminated all answers from individuals below 18 years old. These records were destroyed from the original database. The new analysis has been conducted only for individuals with at least 18 years old, and the results have been revised accordingly - although no substantial changes are identified.

34. I must say I find it a bit surprising that given the age distribution of the sample that only 28% have no college degree – presumably the majority of under 18s and about half of the next age group would not be of the age to have a college degree.

Response: Thank you for noticing. As mentioned before, there was a typo in the manuscript that we regret. We have corrected this and the new version includes a revised table and description.

35. I find it strange that you use “Willingness to Pay” and use a deaths metric as the basis for it (rather than a euro value). Maybe you could rephrase this? “Death acceptance”? (though admittedly there is a difference between “willingness to accept” and “willingness to pay” – I have a feeling here that acceptance may be a better phrase!). When you talk of things like “women have a lower WTP than men” in my mind that suggests they value life less, when in fact this is the opposite!

Response: Thank you for the comment. The term ‘willingness to pay (WTP)’ was used to describe the marginal rate of substitution (MRS) for attribute levels, using death as the numeraire. We understand that using WTP, without a traditional cost attribute may be misleading. As such, we follow the recommendation of both reviewers and substitute WTP by MRS across the entire text - which is a more precise measure.

36. I am wondering if it would be worth thinking about what your results would imply if you translated this (via the income change) to a Value of a Prevented Fatality – a rough calculation I have done suggests the numbers you are getting are pretty low (which might reflect the perception that older and those more likely to die sooner are those most at risk).

Response: Thank you for your question. We agree this is a very valid point and something we also thought about. However, the story of the paper is about to what extent the preferences of the respondents were in line with the policies adopted during the Covid-19 pandemic. While we agree that measuring the statistical value of life would be interesting, we also think it would require a longer discussion and a longer set of references. Also, it would require certainty of generalizability. For space concerns and objectivity of our message we decided not to include a discussion about the statistical value of life.

Still, as the reviewer points out, (a very rough) back of the envelope calculation would yield approximately 1.8 million euros per life, which would be close to the lower intervals of the values generally found for developed countries. We believe this could have been caused by a “normalisation” of deaths during the period in analysis, as many may have seen some deaths as “inevitable” during the pandemic. Also, as the reviewer points out, the average age of the people who were dying was high, which may have impacted these numbers.

37. Given that this was very much a time of much change in terms of COVID regulation and information, I am wondering if you have considered including examining whether these preferences had any time element – e.g. if there was any systematic difference between the responses in March compared to those in January.

Response: Thank you for your comment. This is an excellent point - and it was aligned with our initial idea (to analyse how responses changes over time, and whether we could identify any pattern). However, given the data collection mechanism, such analysis was not possible to perform. As with many online surveys, we had a peak of answers in the first weeks after launching the survey, followed by a very low number of answers in later weeks. For this reason, we do not have a significant number of observations to perform a statistical analysis with a time dimension.

38. I think there is a question you may want to think about in terms of uncertainty of a pandemic spread on health. In your experiment you use fixed numbers – when the epidemiological modelling of risked deaths would probably be much higher (given the deaths on a day when things were controlled was higher than your 250 number). It might be worth reflecting on this and the implications this has on the use of DCEs in this context. DCEs do take some time to design and implement/analyse – so compared to focus groups and other forms of surveys this may restrict their applicability. Sampling, as you note, is also important (given your sample is not representative).

Response: Thank you for your comment. You raise very relevant and important points. We have extended the discussion section to include these topics (page22, line 379).

Minor comments:

39. Line 27 and Line 326 – I don’t think “auscultate” is quite the right word – my understanding is that this is more a medical term (I will have to admit I had to look up what it meant). Perhaps consider using “assess” instead – which I think conveys what you mean.

Response: Thank you. We agree with the suggestion, the text was revised accordingly.

40. Line 305 – “the NHS” – not clear what this is (I’m presuming you mean the SNS in Portugal but would be better to specify – as you also use the NHS on line 346 to refer to UK NHS).

Response: Thank you for the comment. The text was revised accordingly, we now refer to the “Portuguese NHS” instead of to the “NHS”.

Attachment

Submitted filename: Trade-offs during the COVID-19 pandemic - RebuttalLetter.docx

Decision Letter 1

Karyn Morrissey

1 Nov 2022

PONE-D-22-17412R1Trade-offs during the COVID-19 pandemic: a discrete choice experiment about policy preferences in PortugalPLOS ONE

Dear Dr. Filipe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Dec 16 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Karyn Morrissey

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

********** 

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

********** 

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

********** 

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

********** 

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

********** 

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Comments for the Author

Thank you for making changes to your initial manuscript. The change in title is better than the original title proposed. The authors have addressed some of the main issues, but I think the paper needs another revision before it is ready for publication.

The main points I made in the initial review pertain to the analyses that were conducted. Although the authors didn’t include a mixed logit analysis, they have revised the analysis to include a latent class analysis which improves the manuscript. The conditional logit (CL) analysis and corresponding subgroup analyses extending from the CL models have been retained in the manuscript but are not sufficient to explain the inferences made concerning heterogeneity, which the authors do note in their response.

There are other issues I’ve noted with respect to the MRS results (which may be typos) that need to be reconciled. For example, the abstract notes “Estimates suggest that individuals would be willing to sacrifice 20% of their income to avoid 47 deaths per day….”; however, results in the table denote this should probably be for the level of household income being 30% (Table S2). The authors should check all the results in the tables correspond with the text. There are other instances in the abstract and the results (on page 14 paragraph 1 lines 261-267) where this is inconsistent and should be updated.

On page 17 (lines 281-282, and 284-285) the wording with respect to the comparison between the subgroups such as men and women is difficult to understand - “This means that women, …… and everyday life restrictions”. Giving an interpretation for both groups within the subgroup, similar to what you have when you are discussing the main results e.g., (assuming 2 corresponds to the mid-level and 3 corresponds to the worst level perceived) would improve readability e.g., the women were willing to sacrifice 20% of their income to avoid seven daily deaths, whereas the men were willing to sacrifice 20% of their income to avoid 17 daily deaths.

There are still quite a few typos and grammar issues, for example, consistency in the tense used and numeric formatting, and I have noted a few of these below.

Other comments

Line 95-96: “during three fortnights” is odd phrasing; perhaps change to “over a six week period”.

Where numeric figures are below 10, please write out value in full e.g., line 104 – “5” to “five”.

The numeric formatting should be consistent throughout the manuscript e.g., line 99 “1.516”, line 215 “1,250 euro”, line 219 “€1750”, lines 220-224 where decimal places in the text pertaining to percentages are written as either no decimal places or to one or two decimal places; remove the decimal from all numeric figures that are in the thousands throughout. e.g., “1.516”, “6.994”. etc.

Line 141-142: “ the attribute of percentage of impaired students is used as a proxy for “number of education days lost…”. This is not a major issue, but maybe add something about the framing of this attribute, as I don’t think it does refer to the number of education days lost. The basis of the attribute appears to be ‘person-based and does not quite reflect “hours lost”. I’d probably add this as a limitation in the discussion if you want to discuss it further.

Overall phrasing under Section 3.4 p10: I think some of the wording used in the manuscript should be changed to make it easier to read. For example:

Line 211: “Respondents’ age ranged from 18 up to over 85 years old” to “Respondents aged 18 years and over”;

Line 211: Delete the words “Concentration is high in young ages”

Line 214-215: “Considering on the differences in household’s size and composition, individuals display a median monthly equivalised income of around 1,250 euros”. It should be made clear whether it is the study sample or the population.

Results and attributes and levels: “social restrictions” and “life restrictions” are used interchangeably. Are these the same? What was given to the respondents? That probably should be used consistently in the manuscript.

Line 283: “observed” would be better to use than “verified”. Another instance in the discussion.

Latent class analysis:

In your response I can see you tested models with 2 or 3 classes. Did you test whether model fit using CAIC and BIC beyond three latent classes? You should add a statement in your manuscript confirming whether the three class model was the best model fit based on CAIC and BIC parameters and the range of model with different classes tested.

The summary of results requires more careful review. For example in line 292, it states that Class 1 is “more affected by household income losses and poverty losses”; which is not quite the case. A stronger aversion to household income reducing by 30% is observed in Class 3 (reference class) compared to Class 1.

In line 293-294, the text noting “It (Class 1) has a higher prevalence of males… “ should be reworded. For example “people with Class 1 preferences a more likely to be male and work remotely”. In lines 343-345 in the discussion i.e., “higher share of males …..”. Please rephrase as noted above. People with Class X preferences are more likely to have characteristics such as ……

Is there a reference you used for calculating the relative importance of the attributes as shown in Figure 3?

********** 

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Dec 16;17(12):e0278526. doi: 10.1371/journal.pone.0278526.r004

Author response to Decision Letter 1


16 Nov 2022

Reviewer #1

Thank you for making changes to your initial manuscript. The change in title is better than the original title proposed. The authors have addressed some of the main issues, but I think the paper needs another revision before it is ready for publication.

We thank the reviewer for the comment! The reviewers’ contributions have been fundamental to create a better version of this manuscript.

The main points I made in the initial review pertain to the analyses that were conducted. Although the authors didn’t include a mixed logit analysis, they have revised the analysis to include a latent class analysis which improves the manuscript. The conditional logit (CL) analysis and corresponding subgroup analyses extending from the CL models have been retained in the manuscript but are not sufficient to explain the inferences made concerning heterogeneity, which the authors do note in their response.

We thank the reviewer for the comment! We included a latent class model because we believed that class heterogeneity would better accommodate our research question. The rational has to do with the way society has split into groups of people, in which for some, avoiding deaths was prioritized while others advocated those negative economic impacts should be kept to a minimum. We follow the same methodology as Chorus 2020, published in this journal.

There are other issues I’ve noted with respect to the MRS results (which may be typos) that need to be reconciled. For example, the abstract notes “Estimates suggest that individuals would be willing to sacrifice 20% of their income to avoid 47 deaths per day….”; however, results in the table denote this should probably be for the level of household income being 30% (Table S2). The authors should check all the results in the tables correspond with the text. There are other instances in the abstract and the results (on page 14 paragraph 1 lines 261-267) where this is inconsistent and should be updated.

We thank the reviewer for the comment! There are no typos, but we agree that the sentences are not clear. Therefore, we made clarifications both in the abstract and the results’ section. For example, the sentence in the abstract now reads as: “Estimates suggest that individuals would be willing to sacrifice 30% instead of 10% of their income to avoid approximately 47 deaths per day during the first 6 months of 2021” Note that the text is now also clear in stating that the numbers are approximations (to avoid decimal cases).

On page 17 (lines 281-282, and 284-285) the wording with respect to the comparison between the subgroups such as men and women is difficult to understand - “This means that women, …… and everyday life restrictions”. Giving an interpretation for both groups within the subgroup, similar to what you have when you are discussing the main results e.g., (assuming 2 corresponds to the mid-level and 3 corresponds to the worst level perceived) would improve readability e.g., the women were willing to sacrifice 20% of their income to avoid seven daily deaths, whereas the men were willing to sacrifice 20% of their income to avoid 17 daily deaths.

We thank the reviewer for the recommendation! We have revised the text accordingly.

There are still quite a few typos and grammar issues, for example, consistency in the tense used and numeric formatting, and I have noted a few of these below.

Other comments

Line 95-96: “during three fortnights” is odd phrasing; perhaps change to “over a six week period”.

We thank the reviewer for the correction! The text was revised accordingly.

Where numeric figures are below 10, please write out value in full e.g., line 104 – “5” to “five”.

We thank the reviewer for the correction! The text was revised accordingly.

The numeric formatting should be consistent throughout the manuscript e.g., line 99 “1.516”, line 215 “1,250 euro”, line 219 “€1750”, lines 220-224 where decimal places in the text pertaining to percentages are written as either no decimal places or to one or two decimal places; remove the decimal from all numeric figures that are in the thousands throughout. e.g., “1.516”, “6.994”. etc.

We thank the reviewer for identifying those text inconsistencies! The text was revised accordingly.

Line 141-142: “ the attribute of percentage of impaired students is used as a proxy for “number of education days lost…”. This is not a major issue, but maybe add something about the framing of this attribute, as I don’t think it does refer to the number of education days lost. The basis of the attribute appears to be ‘person-based and does not quite reflect “hours lost”. I’d probably add this as a limitation in the discussion if you want to discuss it further.

We thank the reviewer for the comment! We agree that saying this attribute is a proxy for days lost may come as a little bit of a stretch. Therefore, we deleted the sentence in Line 141-142. We have rephrased it to “To some extent, this percentage reflects the inequalities associated with schools’ closure, distance learning and other mitigation strategies.”, which is the correct reason for this attribute being included. This sentence is now in Page 8, Line 161-162.

Overall phrasing under Section 3.4 p10: I think some of the wording used in the manuscript should be changed to make it easier to read. For example: Line 211: “Respondents’ age ranged from 18 up to over 85 years old” to “Respondents aged 18 years and over”;

We thank the reviewer for the comment! The text has been revised accordingly.

Line 211: Delete the words “Concentration is high in young ages”

We thank the reviewer for the suggestion! The text has been revised accordingly.

Line 214-215: “Considering on the differences in household’s size and composition, individuals display a median monthly equivalised income of around 1,250 euros”. It should be made clear whether it is the study sample or the population.

We thank the reviewer for the suggestion! A clarification has been made.

Results and attributes and levels: “social restrictions” and “life restrictions” are used interchangeably. Are these the same? What was given to the respondents? That probably should be used consistently in the manuscript.

We thank the reviewer for the correction! The reviewer is correct. In the Portuguese survey the question asked about life restrictions. To make the text consistent with the question asked to the respondents, we have updated the text to contain only the term “life restrictions”.

Line 283: “observed” would be better to use than “verified”. Another instance in the discussion.

We thank the reviewer for the suggestion The text has been revised accordingly.

Latent class analysis:

In your response I can see you tested models with 2 or 3 classes. Did you test whether model fit using CAIC and BIC beyond three latent classes? You should add a statement in your manuscript confirming whether the three class model was the best model fit based on CAIC and BIC parameters and the range of model with different classes tested.

We thank the reviewer for the comment! We tested the model from 2 up to 10 latent classes. A clarification if made in the analysis section (Page 13, line 241)

The summary of results requires more careful review. For example in line 292, it states that Class 1 is “more affected by household income losses and poverty losses”; which is not quite the case. A stronger aversion to household income reducing by 30% is observed in Class 3 (reference class) compared to Class 1.

We thank the reviewer for the comment. Indeed, that sentence is not correct and is therefore misleading. The entire section was revised to be more clear about whether the inference is relative to another class or is of absolute value.

In line 293-294, the text noting “It (Class 1) has a higher prevalence of males… “ should be reworded. For example “people with Class 1 preferences a more likely to be male and work remotely”.

We thank the reviewer for the suggestion! The entire section has been revised to accommodate for the type of wording suggested by the reviewer.

In lines 343-345 in the discussion i.e., “higher share of males …..”. Please rephrase as noted above. People with Class X preferences are more likely to have characteristics such as ……

We thank the reviewer for the suggestion! The text has been revised accordingly.

Is there a reference you used for calculating the relative importance of the attributes as shown in Figure 3?

We thank the reviewer for the question! Figure 3 values are computed by taking the absolute value of the maximum level of each attribute and dividing them by the sum of the absolute value of all the maximum levels of each attribute, per class. This type of graph is common in other DCE papers, since it is very simple way to illustrate relative preferences. One example would be Arslan et al. 2020, that we added in our references.

Regarding figure 3, however, we noted there was a computing mistake regarding the denominator of class 3. The mistake has been corrected and the figure and respective descriptions are now updated. We also made some minor quality of life improvements that should make the figure easier to interpret.

Decision Letter 2

Karyn Morrissey

18 Nov 2022

Trade-offs during the COVID-19 pandemic: a discrete choice experiment about policy preferences in Portugal

PONE-D-22-17412R2

Dear Dr. Filipe,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Karyn Morrissey

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Karyn Morrissey

9 Dec 2022

PONE-D-22-17412R2

Trade-offs during the COVID-19 pandemic: a discrete choice experiment about policy preferences in Portugal

Dear Dr. Filipe:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Karyn Morrissey

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Appendix

    (PDF)

    Attachment

    Submitted filename: Trade-offs during the COVID-19 pandemic - RebuttalLetter.docx

    Data Availability Statement

    The dataset is anonymous and does not allow the identification of individual respondents. However, based on the informed consent signed by respondents and on the authorization granted by NOVA School of Business and Economics Ethics Committee, data cannot be shared in a public repository to respect data protection regulation. A minimal data set for replication of the study findings is available upon request to the NOVA School of Business and Economics Ethics Committee (research.office@novasbe.pt) and the corresponding author.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES