Do You Prefer Safety to Social Participation? Finnish Population-Based Preference Weights for the Adult Social Care Outcomes Toolkit (ASCOT) for Service Users

Abstract

Introduction. The Adult Social Care Outcomes Toolkit (ASCOT) was developed in England to measure people’s social care–related quality of life (SCRQoL). Objectives. The aim of this article is to estimate preference weights for the Finnish ASCOT for service users (ASCOT). In addition, we tested for learning and fatigue effects in the choice experiment used to elicit the preference weights. Methods. The analysis data (n = 1000 individuals) were obtained from an online survey sample of the Finnish adult general population using gender, age, and region as quotas. The questionnaire included a best-worst scaling (BWS) experiment using ASCOT. Each respondent sequentially selected four alternatives (best, worst; second-best, second-worst) for eight BWS tasks (n = 32,000 choice observations). A scale multinomial logit model was used to estimate the preference parameters and to test for fatigue and learning. Results. The most and least preferred attribute-levels were “I have as much control over my daily life as I want” and “I have no control over my daily life.” The preference weights were not on a cardinal scale. The ordering effect was related to the second-best choices. Learning effect was in the last four tasks. Conclusions. This study has developed a set of preference weights for the ASCOT instrument in Finland, which can be used for investigating outcomes of social care interventions on adult populations. The learning effect calls for the development of study designs that reduce possible bias relating to preference uncertainty at the beginning of sequential BWS tasks. It also supports the adaptation of a modelling strategy in which the sequence of tasks is explicitly modelled as a scale factor.

Introduction

The rising demand for long-term care (LTC) due to the ageing of the population raises the question of how public sector decision makers can effectively allocate limited resources within LTC systems to provide support for individuals with LTC needs and their informal carers.^1,2 Social care interventions support people in daily activities by enabling them to compensate for losses in functional ability that are caused by a physical, mental, or emotional impairment or old age, with the final aim of enhancing well-being and quality of life (QoL). ³ Hence, the outcomes of social care interventions are broader than improvements in health. ⁴ Several systematic reviews have indicated that measurement of health alone, such as the quality-adjusted life year (QALY), ⁵ is insufficient to measure the effects of social care interventions.^6,7

Population-based preferences have been used to assist priority-setting in many areas of public policy.^8,9 As a response to the need to have an outcome measure for social care interventions, Adult Social Care Outcome Toolkit (ASCOT) was developed in England.^10–12 The ASCOT draws on Sen’s^13,14 capability approach to describe social care-related quality of life (SCRQoL) and is applicable to a range of different population groups and multiple care and support settings. ³ We focus on the ASCOT instrument for service users (hereafter ASCOT)—the four response-level interview and self-completion versions.

Internationally, there has been considerable interest in using ASCOT, for example, in Australia, ¹⁵ Austria,^16,17 Denmark, ¹⁸ Finland, ¹⁹ Italy, ²⁰ Japan, ²¹ and the Netherlands. ²² At this time, the official Finnish translations of the ASCOT are available. ²³ The Finnish ASCOT measure has been validated, ²⁴ but its preference weights have not been estimated. Evidence indicates that country-specific differences in sociocultural values, demographic backgrounds, and political and economic systems influence valuations of well-being and QoL.^25,26 It has become a common practice in the field of health-related QoL measurement to establish country-specific preference weights to reflect country-specific values and perceptions concerning different health states.^27–29 A previous study showed that when comparing Spanish and UK time trade-off (TTO) values for EQ-5D health states, Spanish and UK values were similar for mild health states, but for health states worse than death, the Spanish weights generated lower utility scores than the UK weights. ³⁰ Therefore, to enable the use of the Finnish ASCOT so that the ASCOT-QoL states of the Finnish population are correctly assessed, it is necessary to develop Finnish preference weights for the Finnish ASCOT measure.

Evidence from sequential choice experiments suggests that respondents’ behavior may be influenced by fatigue (or boredom)* and learning.^31–33 Where respondents become fatigued, their engagement with the survey declines over time. This can be observed through greater inconsistency in the way they tend to respond to choice tasks toward the end of the choice experiment. Where respondents learn over the course of the choice tasks such that they become better at understanding the choice tasks, there is greater consistency in their responses toward the end of the choice experiment.^31–33 In practice, the model scale, capturing the variance of the error term, can allow for the effect of these behavioral mechanisms on estimated preference parameters. ³⁴

The primary aim of this study is to establish Finnish preference weights for the ASCOT, using data collected from a general population survey including a best-worst scaling (BWS) choice experiment.^35,36 As this experiment included a large number of sequential choices enabling the study of learning and fatigue, our secondary aim is to examine the existence of learning and fatigue effects in the BWS experiment. We will report here the conducted preference study and the estimation results, including evidence of the learning effect in the BWS experiment.

Methods

The ASCOT and Best-Worst Scaling

The ASCOT measures SCRQoL across eight attributes (domains): 1) control over daily life (CONT), 2) personal cleanliness and comfort (PERC), 3) food and drink (FOOD), 4) accommodation cleanliness and comfort (HOME), 5) personal safety (SAFE), 6) social participation and involvement (SOCI), 7) occupation (OCCU), and 8) dignity (DIGN). The basic domains are PERC, FOOD, HOME, and SAFE, and the rest are the higher order domains (Supplemental Table S1). ³ The dignity domain aims to capture the effects of the care process on the service user’s self-esteem. ³⁷ Attribute-levels indicating the intensity of needs are the following: ideal state (top level, level_1), no needs (level_2), some needs (level_3), and high needs (bottom level, level_4).

To collect preference data, we used the BWS method (profile case), ³⁸ following Netten et al. ³ In a BWS choice experiment, respondents are asked to choose the most (best) and least preferred (worst) alternatives within each choice profile. In a traditional discrete choice experiment (DCE), respondents choose the most preferred alternative from at least two choice profiles at a time. In each BWS choice profile in this study, the same attributes were shown but their attribute-levels varied: respondents compared the displayed attribute-levels with each other to identify the best and worst, and the second-best and second-worst attribute-levels.^39,40 The BWS method is argued to provide easier tasks and to be less cognitively burdensome on the respondents compared to the DCE method.^35,39 It was also shown that using both DCE and BWS methods resulted in similar estimation results in the context of the ASCOT. ⁴¹

Design of Experiment and Survey, and Sampling

Different BWS scenarios were developed using an orthogonal main effects plan. ⁴² A fractional-factorial design was applied because a full-factorial design (4⁸ possible profiles) would lead to far too many states for presentation.^43–45 We used a design matrix of 32 profiles which were divided into four 8-profile blocks (i.e., hypothetical scenarios describing the ASCOT attributes). A randomly chosen 8-profile block was shown to each respondent. The respondents were asked to do eight choice tasks by sequentially selecting the best, worst, second-best, and second-worst attribute-levels from each of eight BWS profiles (Supplemental Figure S1). A foldover design was applied to reduce the appearance of extreme values from each scenario (i.e., to eliminate easy or simple choices) and to ensure that both the occurrence of each attribute-level and the co-occurrences of the attribute-levels were equal. ⁴⁶ The ordering of the attributes was also randomized between respondents (not within respondents) to prevent ordering bias and control for position effects on the selecting of attribute-levels.^47,48

An online survey (managed by Research Now) was conducted between July and August 2016, using an internet panel as the sampling frame. To obtain a representative sample of the general adult population in Finland, a quota sampling approach was used with quotas for gender; age groups 18 to 24, 25 to 34, 35 to 44, 45 to 54, and ≥55 years old; and region. Regions in mainland Finland were included, and they were defined using the NUTS-2 classification (i.e., four large areas of Finland: West, Helsinki-Uusimaa, South, and East & North; Table 2). The questionnaire included questions about the respondents’ demographic and socioeconomic background, well-being (self-assessed health [SAH] and overall QoL) and other information (the respondents’ experience of caring and need for social care). The BWS section of the questionnaire also included questions about how well respondents understood the BWS tasks to provide insights into the validity and reliability of responses. To enhance the reliability of the results, respondents who spent less than 4.5 minutes completing the BWS tasks—an implausibly fast time based on piloting of the tasks—were excluded during the data collection phase. Excluding those with no information on their education (n = 8), the analysis sample had 1000 respondents and the long-format panel data contained 32,000 choice observations (i.e., 32 choices per respondent).

Table 2.

Descriptive Characteristics of the Sample Population and the General Population

	Sample (n = 1000)		General Population		Source
Variable	Mean	SD	Mean	N
Gender			1.000	4,431,392	Statistics Finland 49
Male	0.481	0.500	0.488	2,163,845
Female	0.519	0.500	0.512	2,267,547
Age			1.000	4,431,392	Statistics Finland 49
18–24 years	0.078	0.268	0.103	455,977
25–34 years	0.155	0.362	0.159	704,402
35–44 years	0.160	0.367	0.151	671,350
45–54 years	0.185	0.388	0.161	712,553
55–64 years	0.272	0.445	0.166	737,135
65–79 years	0.145	0.352	0.194	861,876
80 years or older	0.005	0.071	0.065	288,099
Marital status			1.000	4,431,392	Statistics Finland 49
Married	0.417	0.493	0.451	1,998,678
Divorced	0.188	0.391	0.128	568,184
Widowed	0.027	0.162	0.064	282,794
Single	0.346	0.476	0.357	1,581,736
Not reported	0.022	0.147
Employment status			1.000	4,431,392	Statistics Finland 50
Employed	0.429	0.495	0.514	2,275,679
Student	0.064	0.245	0.053	236,335
Pensioner	0.296	0.456	0.314	1,389,266
Unemployed	0.143	0.350	0.080	355,364
Other	0.068	0.252	0.039	174,748
Education			1.000	4,591,285	Statistics Finland51,a
Lower secondary school	0.092	0.289	0.293	1,345,561
Upper secondary school	0.484	0.500	0.407	1,867,828
Lowest level tertiary school	0.126	0.332	0.097	447,112
Lower level tertiary school	0.175	0.380	0.105	484,271
Higher level tertiary school	0.114	0.318	0.088	403,731
Doctorate level	0.009	0.094	0.009	42,782
Housing tenure			1.000	5,363,637	Statistics Finland52,a
Own house/apartment	0.546	0.498	0.711	3,813,335
Rent	0.447	0.497	0.270	1,446,729
Other	0.007	0.083	0.019	103,573
Religion			1.000	4,609,119	Statistics Finland53,a
Any religion	0.620	0.485	0.733	3,376,789
No religion	0.380	0.485	0.267	1,232,330
Regions			1.000	4,407,913	Statistics Finland49,a
Helsinki and Uusimaa	0.291	0.454	0.297	1,311,203
Southern Finland	0.219	0.414	0.215	948,790
Western Finland	0.251	0.434	0.252	1,110,490
North-Eastern Finland	0.239	0.426	0.235	1,037,430

^aReligion (Statistics Finland) ⁴⁹ and education (Statistics Finland) ⁵¹ refer to the population aged 15 or older. Housing tenure (Statistics Finland) ⁵² refers to the whole housing population and regions (Statistics Finland) ⁴⁹ to the population aged 18 or older.

Modelling Strategy

Our choice data were collected using the BWS method, ⁴² which is a stated-preference method based in random utility theory (RUT).^36,54 Using RUT, one is able to elicit preferences for complex multidimensional commodities. RUT was extended to the case of discrete choices from multiple alternatives, ⁵⁵ and further works have been developed (see Louviere et al.). ⁵⁶ Similar to revealed preference approaches that measure and estimate preferences based on observations of individual choices, the BWS stated-preference method also assumes that respondents’ choices reveal their utilities (preferences) and respondents choose [avoid] the alternative from which they will derive the highest utility [the lowest utility]. ⁵⁶ Researchers do not directly observe utility (i.e., utility is latent) but can observe choices made by respondents. ⁵⁶ The expected utilities from choosing different alternatives were modelled in terms of attribute-levels rather than attributes of the individuals.^54,55 The preference parameters to be estimated are a function of choice frequencies, and the choice of a particular attribute-level describes the importance of that attribute-level relative to other attribute-levels. ⁴²

We applied models that described the choice process when the axioms of RUT do not fully hold. As a starting point, we used a traditional multinomial logit (MNL) model.^54,55 The basic MNL model is based on three assumptions: the independence of the errors; each error term follows a Gumbel distribution; and the errors are identically distributed^54,55 (see Appendix 1). Since the scale of the idiosyncratic error that captures the variance of the error term in the MNL model is usually normalized to a constant (generally to unity) to ensure identification of the model parameters, scale heterogeneity is often ignored. ⁵⁷ To use the traditional MNL model, the assumption of independence of irrelevant alternatives should be valid, but proportional substitution across alternatives is likely to happen in the actual data. ⁵⁸

Scale heterogeneity can distort preference parameters. ⁵⁹ Hence, we considered the scale multinomial logit (S-MNL) model to be more suitable than the basic MNL model to derive preference parameters. ⁵⁷ To investigate which factors scale heterogeneity in the sample was associated with, we focused on the heterogeneity in error variance that was not accounted for by taste heterogeneity related to observed characteristics of respondents. Therefore, before selecting the final S-MNL model for the estimation of preference parameters, we first ran the mixed logit model using observed characteristics of respondents (e.g., age, gender, and education) to examine taste heterogeneity only, and then ran the generalized MNL (G-MNL) model to examine scale heterogeneity, simultaneously controlling for taste heterogeneity. ⁵⁷ The G-MNL model enabled us to find appropriate scale factors. These models are defined in more detail in Appendix 1.^{55,57,58,60–62}

Our five-step modelling process is described in Table 1. For simplicity’s sake, we have named the five specifications of the G-MNL as follows: 1) basic (Model I), 2) taste (Model II), 3) taste-and-scale (Model III), 4) scale (Model IV), and 5) taste-adjusted scale (Model V) models (Table 1). The estimated attribute-level coefficients from Model V were then adjusted for significant taste differences between the sample population and the general population using modified post-stratification to become final preference weights (Model V+). ⁶³

Table 1.

Model Developing Process and Specifications

Step	Model	Specificationa,b	Result
1. Basic model	MNL	Attribute position variables (separately for both the best or second-best and the worst or second-worst choices) as explanatory variables.	Model I
2. Taste model c	Mixed logit	Including in the basic model (step 1): 1) the attribute-specific constants (ASCs) for the worst or second-worst choices, and 2) the interactions between the individual characteristics (e.g., age, gender, education) and the attribute-levels or single attributes to measure taste heterogeneity. We aimed to control for taste heterogeneity and to minimize unexplained variations.	Model II: Not reported
3. Taste-and-scale model	G-MNL	Including in the taste model (step 2): different sets of 4 to 5 variables at a time to test whether these variables could capture scale heterogeneity.	Model III: Supplemental Table S2
4. Scale model	S-MNL	Keeping the scale variables obtained from step 3 and the position variables. The ASCs for the worst or second-worst choices and the variables capturing taste heterogeneity were excluded.	Model IV
5. Taste-adjusted scale model	S-MNL with taste variables	Including in the scale model (step 4) several taste variables. These were interaction terms between attribute-levels and observed characteristics of respondents that were not representative of the Finnish general adult population.	Model V: Supplemental Table S3
		Final preference weights	Model V+

G-MNL, generalized multinomial logit; MNL, multinomial logit; S-MNL, scale multinomial logit.

^aEach model always included attribute-level variables that we were interested in.

^bThe specified variables were included in the model as explanatory variables.

^cThe interaction terms (the taste variables) measured the impacts that individual characteristics had on the preferences for a particular attribute-level or attribute.

In all models, in addition to attribute-levels, the position variables for the best or second-best choices and those for the worst or second-worst choices were included to account for the overall effects of attribute ordering associated with the experimental choice task design.^64,65 Furthermore, we included the attribute-specific constants (ASCs) for the worst or second-worst choices in the taste and taste-and-scale models (Supplemental Table S2) to capture differences in the likelihoods of selecting attributes as the least or second-least preferred alternatives. The ASC is assumed to account for the average effect on the utility of all factors that are not included in the model. ⁶⁶

The bottom level of the control attribute, cont4, was used as the reference level and was set to zero. To ensure model identification, the position variable of one attribute (for each set of best or worst choices) were set at zero, and the ASC of one attribute in the scenario for the worst or second-worst choices in the taste and taste-and-scale MNL models was fixed at zero. Due to the repeated choice data, sandwich estimators were used to get robust standard errors. ⁶⁶ The variables capturing taste heterogeneity were obtained using the “Apply Run” procedure in ALOGIT. ⁶⁷ The models were estimated by the maximum likelihood using the BIOGEME software. ⁶⁸

Scale Heterogeneity and Behavioral Mechanisms

We tested whether scale factors were associated with age, gender, education, residential area, housing tenure, SAH, overall QoL, experience of care, time to complete the BWS tasks, and the best and worst choices. To select scale variables, we undertook a series of scale heterogeneity analyses—each time we estimated a taste-and-scale model by including in it four to five covariates above with different subgroups of each covariate.

Fatigue and learning may arise from the repeated and sequential choice tasks in choice experiments, which can influence the respondents’ preferences.^31–33 Hence, we expected that a scale factor may be associated with the sequence of the BWS choice tasks. Following Carlsson et al., ³² we defined two identical sequences of four tasks in the BWS experiment. We tested for the presence of learning [or fatigue] in the first sequence of four tasks relative to the second sequence of four tasks. Using this specification, learning [fatigue] suggests that the error variance is higher [lower] in the first four-task sequence than in the second four-task sequence.^32,34

Final Preference Estimates

The established preference weights should be representative of the general population, being the averages of the preference estimates for all individuals. ⁴² Several subgroups of covariates in the sample were significantly under- or overrepresented when comparing them to the general population (>10 percentage points of P < 0.05). This was the case for those aged 65 or older, those aged 54 to 64, and those without any religion (Table 2). For these subgroups, applying a post-stratification method, ⁶³ we used group-specific population weights to adjust the preference estimates from the taste-adjusted S-MNL model* (Model V; Supplemental Table S3).^3,69,70 The final preference estimates can then reflect the general population’s values (Model V+; Table 4). We also computed the standard errors of the adjusted preference estimates, using population weights (Table 2) and the estimated variance-covariance matrix of the parameters provided by BIOGEME. ⁶⁸

Table 4.

Estimated Finnish Preference Weights for the ASCOT for Service Users (n = 32,000)

Variable	Model I			Model IV			Model V+a,b
	Estimated coeff.	Robust t-value	Normalized coeff.	Estimated coeff.	Robust t-value	Normalized coeff.	Estimated coeff.	Robust t-value	Normalized coeff.	Rescaled coeff.
Attribute-level
cont1	5.077	41.96	1.000	7.086	23.27	1.000	6.903	23.44	1.000	0.156
cont2	4.986	41.48	0.982	6.917	23.24	0.976	6.731	23.39	0.975	0.152
cont3	2.524	30.68	0.497	3.436	20.34	0.485	3.262	20.45	0.473	0.063
cont4	0.000	—	0.000	0.000	—	0.000	0.000	—	0.000	−0.020
perc1	3.734	38.35	0.735	5.198	22.62	0.734	5.017	22.73	0.727	0.108
perc2	3.589	36.68	0.707	5.003	22.13	0.706	4.823	22.26	0.699	0.103
perc3	1.239	20.13	0.244	1.673	16.29	0.236	1.510	16.21	0.219	0.018
perc4	1.195	20.70	0.235	1.584	16.89	0.224	1.420	16.75	0.206	0.016
food1	4.353	41.18	0.857	6.048	23.07	0.854	5.845	23.11	0.847	0.129
food2	4.386	41.31	0.864	6.091	23.66	0.860	5.888	23.74	0.853	0.130
food3	0.627	11.47	0.124	0.845	10.52	0.119	0.607	6.94	0.088	−0.005
food4	0.243	4.19	0.048	0.311	3.91	0.044	0.055	0.63	0.008	−0.019
home1	3.378	37.55	0.665	4.679	22.10	0.660	4.478	22.30	0.649	0.094
home2	3.168	34.35	0.624	4.418	21.60	0.623	4.239	21.72	0.614	0.088
home3	2.097	30.68	0.413	2.835	21.36	0.400	2.670	21.70	0.387	0.048
home4	1.694	28.89	0.334	2.271	20.31	0.320	2.109	20.74	0.306	0.034
safe1	3.923	37.65	0.773	5.497	22.31	0.776	5.325	22.34	0.771	0.116
safe2	2.284	31.53	0.450	3.128	21.06	0.441	2.893	20.95	0.419	0.054
safe3	1.066	17.83	0.210	1.397	14.70	0.197	1.095	13.11	0.159	0.008
safe4	0.607	10.91	0.120	0.787	9.77	0.111	0.669	6.87	0.097	−0.003
soci1	4.078	37.68	0.803	5.716	21.93	0.807	5.535	22.03	0.802	0.121
soci2	3.656	37.62	0.720	5.090	22.26	0.718	4.908	22.42	0.711	0.105
soci3	2.334	32.53	0.460	3.213	21.30	0.453	3.047	21.56	0.441	0.058
soci4	0.912	16.30	0.180	1.186	13.98	0.167	1.031	13.27	0.149	0.006
occu1	4.859	41.25	0.957	6.763	23.09	0.954	6.544	23.16	0.948	0.147
occu2	4.785	41.26	0.942	6.644	23.32	0.938	6.459	23.49	0.936	0.145
occu3	2.549	32.73	0.502	3.490	21.46	0.493	3.315	21.74	0.480	0.065
occu4	0.634	12.26	0.125	0.814	10.68	0.115	0.653	9.23	0.095	−0.003
dign1	4.301	38.65	0.847	5.975	22.57	0.843	5.792	22.67	0.839	0.128
dign2	3.212	31.72	0.633	4.475	20.87	0.632	4.297	21.03	0.622	0.090
dign3	1.225	20.21	0.241	1.606	16.29	0.227	1.444	15.89	0.209	0.017
dign4	0.406	7.21	0.080	0.516	6.75	0.073	0.356	5.00	0.052	−0.011
Position for best/second-best (most/second most preferred) alternatives
pos2_B	−0.110	−2.28		−0.148	−2.25		−0.147	−2.23
pos3_B	−0.192	−3.96		−0.255	−3.74		−0.255	−3.75
pos4_B	−0.204	−4.15		−0.245	−3.62		−0.246	−3.63
pos5_B	−0.308	−5.84		−0.404	−5.55		−0.401	−5.52
pos6_B	−0.319	−6.24		−0.405	−5.70		−0.407	−5.74
pos7_B	−0.395	−7.64		−0.501	−6.99		−0.503	−7.04
pos8_B	−0.323	−6.17		−0.401	−5.59		−0.402	−5.62
Position for worst/second-worst (least/second least preferred) alternatives
pos2_W	0.027	0.55		0.040	0.60		0.044	0.66
pos3_W	0.037	0.74		0.047	0.68		0.052	0.75
pos4_W	0.015	0.31		0.023	0.35		0.027	0.41
pos5_W	0.073	1.44		0.090	1.31		0.097	1.43
pos6_W	0.036	0.74		0.051	0.77		0.058	0.87
pos7_W	0.057	1.13		0.070	1.00		0.076	1.09
pos8_W	0.056	1.09		0.070	0.99		0.066	0.94
Scale parameter
λlearning				0.893	6.46		0.889	6.71
λsah				0.811	5.34		0.812	5.29
λedu				0.861	3.56		0.864	3.46
λtime				0.748	6.01		0.748	6.04
Observations		32,000		32,000			32,000
df		45		49			54
Log likelihood value		−42056.3		−41711.9			−41698.4
AIC		84202.6		83521.8			83504.8
BIC		84315.3		83644.6			83640.1
Rho2 (0)		0.292		0.298			0.298

AIC, Akaike information criterion; BIC, Bayesian information criterion.

^aThe coefficients of the attribute-levels in Model V+ were adjusted for the observed taste differences between the sample and general populations. Corrections were manually made for 10 attribute-levels (occu1, home1, and all four levels of both FOOD and SAFE attributes).

^bFive interaction terms capturing taste heterogeneity were included in the taste-adjusted S-MNL model (df = 54) (Supplemental Table S3).

We normalized the attribute-level coefficients, using the largest coefficient from each estimated model as the common denominator. To better understand the quantified changes in different SCRQoL states, we linearly rescaled the final attribute-level coefficients to an index by applying a conversion method.^44,69,70 We anchored this index at a value of zero for the set of states presented by the eight lowest attribute-level coefficients (each per domain) and at a value of one for the set of states presented by the eight highest attribute-level coefficients (each per domain), keeping the relative differences between the attribute-level coefficients unchanged. Thus, the ASCOT index measuring SCRQoL ranges between zero and one, where zero indicates the worst SCRQoL represented by the eight worst ASCOT-QoL states and one indicates the best SCRQoL represented by the eight best ASCOT-QoL states.

Results

Sample Characteristics

The sample proportions of many covariates were close to those of the Finnish general adult population. We found larger differences between the sample and the general population for age, education, religion, and housing tenure than for the other variables (Table 2). Compared to the general population, the sample had more people aged 55 to 64 years and fewer those aged 80 or older, fewer people with the lowest level of education and belonging to some religion, as well as fewer house owners.

Regarding how often respondents were able to put themselves in the imaginary situations in the BWS exercises, 67% of them were able to do so all of the time and 29.6% some of the time. Nearly everyone reported that they had understood the situations in the best-worst exercises all or some of the time (99.7%; Table 3).

Table 3.

Descriptive Statistics of Variables Used in the Analysis (n = 32,000) ^a

Description b	Descriptive Value
	Name	Mean
		All	Best/Second-Best Choice	Worst/Second-Worst Choice
Attributes or attribute-levels
Control over daily life	CONT	0.173	0.217	0.129
1 I have as much control over my daily life as I want	cont1	0.048	0.091	0.004
2 I have adequate control over my daily life	cont2	0.048	0.091	0.005
3 I have some control over my daily life, but not enough	cont3	0.030	0.031	0.028
4 I have no control over my daily life	cont4	0.048	0.004	0.091
Personal cleanliness and comfort	PERC	0.103	0.091	0.116
1 I feel clean and am able to present myself the way I like	perc1	0.024	0.044	0.005
2 I feel adequately clean and presentable	perc2	0.022	0.040	0.005
3 I feel less than adequately clean or presentable	perc3	0.030	0.004	0.056
4 I don’t feel at all clean or presentable	perc4	0.027	0.003	0.051
Food and drink	FOOD	0.159	0.145	0.173
1 I get all the food and drink I like when I want	food1	0.035	0.066	0.004
2 I get adequate food and drink at OK times	food2	0.037	0.068	0.005
3 I don’t always get adequate or timely food and drink	food3	0.041	0.006	0.075
4 I don’t always get adequate or timely food and drink, and I think there is a risk to my health	food4	0.047	0.005	0.088
Accommodation cleanliness and comfort	HOME	0.061	0.063	0.059
1 My home is as clean and comfortable as I want	home1	0.020	0.035	0.006
2 My home is adequately clean and comfortable	home2	0.013	0.022	0.005
3 My home is not quite clean or comfortable enough	home3	0.011	0.003	0.019
4 My home is not at all clean or comfortable	home4	0.017	0.003	0.030
Personal safety	SAFE	0.120	0.067	0.173
1 I feel as safe as I want	safe1	0.026	0.047	0.004
2 Generally I feel adequately safe, but not as safe as I would like	safe2	0.021	0.014	0.028
3 I feel less than adequately safe	safe3	0.033	0.003	0.062
4 I don’t feel at all safe	safe4	0.041	0.003	0.078
Social participation and involvement	SOCI	0.098	0.109	0.086
1 I have as much social contact as I want with people I like	soci1	0.031	0.057	0.005
2 I have adequate social contact with people	soci2	0.022	0.041	0.004
3 I have some social contact with people, but not enough	soci3	0.013	0.009	0.017
4 I have little social contact with people and feel socially isolated	soci4	0.032	0.003	0.061
Occupation	OCCU	0.157	0.207	0.107
1 I’m able to spend my time as I want, doing things I value or enjoy	occu1	0.044	0.086	0.003
2 I’m able do enough of the things I value or enjoy with my time	occu2	0.045	0.085	0.004
3 I do some of the things I value or enjoy with my time, but not enough	occu3	0.030	0.032	0.028
4 I don’t do anything I value or enjoy with my time	occu4	0.038	0.004	0.072
Dignity	DIGN	0.128	0.100	0.156
1 The way I’m helped and treated makes me think and feel better about myself	dign1	0.034	0.062	0.006
2 The way I’m helped and treated does not affect the way I think or feel about myself	dign2	0.019	0.028	0.010
3 The way I’m helped and treated sometimes undermines the way I think and feel about myself	dign3	0.031	0.007	0.056
4 The way I’m helped and treated completely undermines the way I think and feel about myself	dign4	0.044	0.004	0.084

^aThe first column introduces 8 attributes, 32 attribute-levels (4 per attribute), 16 attribute position variables (8 for best/second-best choices; 8 for worst/second-worst choices), and four variables capturing scale heterogeneity. The second column indicates empirical names for the variables used in the models. For the attributes and attribute-levels, the next three columns describe the proportions of the attributes or attribute-levels that respondents chose totally and by choice type. For the attribute position variables, the third and fourth columns describe the sample mean and standard deviation of each attribute position variable. For two questions that explained how respondents understood the BWS tasks in the experiment, the proportions of multiple response items are reported.

^bThe ASCOT measure is disclosed in full herein but ordinarily should not be used for any purposes without the appropriate permissions of the ASCOT team and the copyright holder—the University of Kent. Please visit www.pssru.ac.uk/ascot or email finascot@thl.fi to enquire about permissions.

^cThe reference group 1) for learning: the last four BWS tasks; 2) for SAH: very good or good SAH; 3) for education: lowest or lower or higher level tertiary school or doctorate level; 4) for short completion time: longer completion time (= the second or third or fourth quartile of the distribution of time used to complete the BWS tasks).

The three attributes mostly preferred were the control (CONT), food (FOOD), and occupation (OCCU) attributes (Table 2). The cont1, cont2, occu1, and occu2 attribute-levels were most preferred: they were mostly selected as the best or second-best choices. The cont4, food4, dign4, and safe4 attribute-levels were least preferred: they were mostly chosen as the worst or second-worst choices. In particular, food2 was preferred to food1; it was selected more often than food1 in total and across choices. For the best or second-best choices, until the seventh position, the further away from the first attribute on the choice list, the less likely it was that an attribute was chosen. For the worst or second-worst choices, the position of the attribute did not matter a great deal: the probability of choosing an attribute remained quite stable from the first to the eight position.

Preference Estimates

The final preference estimates (Model V+; Table 4) were derived using results from the taste-adjusted scale MNL (Model V; Supplemental Table S3).* Concerning the goodness-of-fit, Model IV was significantly better than Model I, and Model V+ was significantly better than Model IV. ^† Because a pseudo-R² with values between the range of 0.3 and 0.4 can be regarded as an R² with values between the range of 0.6 and 0.8 for the equivalent linear regression, ⁷¹ the pseudo-R² of 0.298 (Models IV and V⁺) suggests a decent fit. Below, we discuss the estimation results from Model V+ if not otherwise specified.

The cont4 attribute-level, “I have no control over my daily life,” the lowest-valued state, was followed by the attribute-levels food4, dign4, food3, and occu4 in an ascending order (Table 4; Figure 1). Since the coefficients of the other attribute-levels were greater than zero, the other ASCOT-QoL states were valued more than the ASCOT-QoL state represented by cont4. The cont1 attribute-level, the mostly valued state, was followed by cont2 and the top two levels of the OCCU attribute in a descending order.

Figure 1.

Attribute-level coefficients and their 95% confidence intervals for the Finnish ASCOT for service users (n = 32,000).

The ordering of the attribute-level coefficients by each attribute followed the original ordering of the attribute-levels except for the FOOD attribute. The difference between attribute-levels 1–2 was not statistically significant for attributes: CONT (P = 0.555), PERC (P = 0.375), FOOD (P = 0.864, with the difference in switched attribute-levels 2–1), HOME (P = 0.228), and OCCU (P = 0.760). The ASCOT-QoL states indicated by attribute-levels 1–2 were valued more than those indicated by attribute-levels 3–4. The change in ASCOT-QoL due to moving from level_3 to level_2 was valued more than that due to the moving from level_2 to level_1. However, for the SAFE and SOCI attributes, the mostly valued change in ASCOT-QoL was associated with the movement from safe2 [no needs] to safe1 [ideal state] and that from soci4 [high needs] to soci3 [some needs] (Table 4; Figure 1).

For the best or second-best choices, the coefficients of the position variables were statistically significant (Table 4). As implied by the negative sign of the coefficients, respondents were less likely to choose an item on the choice list that appeared following the first item. Furthermore, the attributes were increasingly less likely to be chosen the further down the list they appeared, until reaching the seventh position (−0.503), which was less likely to be chosen than the eight position (−0.402). However, respondents were quite indifferent to the items appearing on the fifth to sixth rows. The coefficients of the position variables for the worst or second-worst choices were not statistically significant.

The value of each scale parameter was smaller than unity. ^‡ Respondents who had a high level of education or a better SAH, or took a longer time doing the BWS tasks were more consistent in their choices than those who had a lower level of education or worse SAH, or took less time doing the BWS tasks (Table 4). Moreover, the estimate of the scale parameter for learning (0.889) indicates that the error variance was higher in the first four-task sequence than in the second four-task sequence. This evidence shows that learning occurred in the later four tasks of the sequential BWS choice experiment.

Final Preference Weights

Table 4 also reports normalized and rescaled values of the attribute-level coefficients. Due to disparities between each attribute-level and the average value of all lowest rated attribute-levels,^44,70 their rescaled values were also negative (Tables 4 and 5). Regarding the FOOD attribute, food1 had a smaller coefficient (5.845) than food2 (5.888), although their difference was not statistically significant (Table 4). To preserve the rank of the attribute-levels and better quantify changes in SCRQoL in practice, we switched these estimated coefficients. Therefore, the normalized (rescaled) preference weight of food1 was 0.853 (0.130) and that of food2 was 0.847 (0.129) (Table 5).

Table 5.

Final Preference Weights for the Finnish ASCOT for Service Users (n = 32,000) ^a

Preference Weight	Level	Control Over Daily Life	Personal Cleanliness	Food and Drink	Accommodation Cleanliness	Personal Safety	Social Participation	Occupation	Dignity
Panel 1. Normalized values	1. Ideal state	1.000	0.727	0.853	0.649	0.771	0.802	0.948	0.839
	2. No needs	0.975	0.699	0.847	0.614	0.419	0.711	0.936	0.622
	3. Some needs	0.473	0.219	0.088	0.387	0.159	0.441	0.480	0.209
	4. High needs	0.000	0.206	0.008	0.306	0.097	0.149	0.095	0.052
Panel 2. Rescaled values	1. Ideal state	0.156	0.108	0.130	0.094	0.116	0.121	0.147	0.128
	2. No needs	0.152	0.103	0.129	0.088	0.054	0.105	0.145	0.090
	3. Some needs	0.063	0.018	−0.005	0.048	0.008	0.058	0.065	0.017
	4. High needs	−0.020	0.016	−0.019	0.034	−0.003	0.006	−0.003	−0.011

^aFor the food and drink attribute, the current preference weight of level_1 was the originally estimated preference weight of level_2 and the current preference weight of level_2 was the originally estimated preference weight of level_1.

If we do not need preference weights for four separate attribute-levels, we can combine the parameters of food1 and food2. Using the restriction that the parameters of food1 and food2 are same, we ran a new taste-adjusted S-MNL model (log-likelihood value −41698.55; df = 53; Rho² = 0.298). Regarding the goodness-of-fit, this model was supported by the LR test (statistic 0.30; df = 1) compared to Model V+ (Table 4). The estimated coefficient of the combined attribute-level (called food12) was exactly the average of the original coefficients of food1 and food2, and the rest of the estimated parameters were very similar to those in Model V+. Nevertheless, to preserve the order of the attribute-levels indicating the intensity of need for each ASCOT attribute, we switched the estimated preference weights of the top two levels of FOOD (Table 5).

ASCOT index values can be used to illustrate changes in SCRQoL associated with different ASCOT-QoL states. As individuals’ SCRQoL is an additive combination of eight attribute-levels, an improvement in SCRQoL, for example, from a poor state of 34424343 ^* to an improved state of 12313212 would be a SCRQoL gain of 0.481 ^† (Supplemental Figure S2). Those who would like to utilize the Finnish ASCOT can use the normalized or rescaled values of the preference weights displayed in Table 5.

Discussion

In this study, we derived the Finnish population-based preference weights for the Finnish ASCOT and provided evidence on the learning effect in the BWS choice experiment. Although population-based preferences cannot fully capture population-specific expectations and responses, ⁷² they are regarded as appropriate for evaluating the effects of social care interventions for adult populations. ³

The Finnish respondents placed both highest and lowest values to attribute-levels of the higher-order control and occupation attributes. A comparison of the Finnish preference weights for the ASCOT instrument to the English, ³ Austrian, ¹⁷ and Japanese weights ⁷³ also suggests that the control and occupation attributes were mostly valued (Supplemental Tables S4). Although the estimated models had different scale factors, ⁷⁴ we can study the relative size of the differences in SCRQoL by using one of the attribute-levels as a common denominator. ³ The most valued attribute-level was the cont1 state in Finland and England, while it was the occu1 state in Austria and Japan. The least valued attribute-level was the cont4 state in Finland and England, but it was the dign4 state in Austria and Japan. In fact, cont4 and dign4 were equally the lowest valued states in Japan. The Finnish and English preference weights were very highly positively correlated—the Pearson correlation coefficient was 0.97 (P < 0.0001)—indicating strong consistency between the weights.

We found evidence that the probability of an item being selected depended on its position in the profile but only for the best or second-best choices. This finding confirmed that attributes in the profiles in a BWS design survey should be rotated to mitigate position bias that may influence the respondents’ choice behavior and decisions, leading to invalid coefficient estimates. ⁶⁴ In addition to the randomization of the attributes during the study design stage, researchers can control for the potential existence of position effects in the analysis data by including position-specific constants in the empirical model. Not accounting for the position effect can result in biased estimates for preference weights and may affect their validity if the estimates are used to provide policy recommendations. ⁶⁴

Two scale factors were influenced by education and health, which are recognized as being associated with cognitive functioning.^70,75 A short completion time of the BWS task as a significant scale factor could suggest that the respondents used a heuristic method to make choices quickly ⁷⁶ or that they made a reduced effort while engaging in the choice tasks and considering the available alternatives properly. The learning effect has not been explored earlier in BWS studies, except in Nguyen et al. ²⁴ In addition to two four-task sequences, we tested other sequential divisions of the choice tasks (e.g., the first two tasks vs. the later six tasks), but they were not statistically significant.

Similar to the position effects discussed above, the detected learning effect has more general implications for study designs and methods. When data are used to obtain utility estimates and to inform decision makers, researchers should pay attention to the ordering of the profiles in experiments in which choices are sequentially made. ³² In practice, to take into account the effects that respondent learning (or fatigue) might have on preference estimates, researchers can explicitly model learning (or fatigue) as a scale parameter, for example, by using the sequences of BWS tasks, as we have done in this study.

The scale factors were explicitly modelled to account for differences in random component variances between different groups of respondents or situations. ⁵⁹ This also calls for approaches, by which researchers can separate scale heterogeneity from taste heterogeneity to derive accurate preference estimates (e.g., Flynn et al. ⁴⁰ ). In this respect, our modelling approach could be useful. We first studied taste heterogeneity (using the mixed logit) and then both taste and scale heterogeneity (using the G-MNL) assuming the values of coefficients are different to the worst and best choices. Having identified the scale factors, we used the taste-adjusted S-MNL model to derive the population-based preference estimates.

We have established a set of preference weights for the Finnish ASCOT for service users measure that are prerequisites for calculating social care QALYs (SC-QALYs) in Finland. Based on the relationship between BWS weights and time-trade off (TTO), Netten et al. ³ developed a formula for an English SC-QALY, with “0” equivalent to “being dead” and “1” being the “ideal” SCRQoL state. Possible SC-QALY scores in England range from −0.171 to +1. Service users reported a significantly lower SCRQoL (score 0.73) than participants in the general population (score 0.86). Similar TTO study is out of scope of this article, but it may call for a new study in the future to derive a similar formula for a Finnish SC-QALY based on the association between Finnish TTO and BWS weights.

This study used preference data collected online. Evidence has indicated that modes of survey administration, such as internet-based surveys, might cause stronger fatigue effects and weaker learning effects, ³¹ but no notable comparable differences in the estimates from a model using online BWS data compared to those from a model using face-to-face interview data were observed. ⁷⁷ Nonetheless, the learning effect was found in this study. The effect that the modes of survey administration may have on respondents’ learning and fatigue presents an interesting area of future research in preference elicitation studies using the BWS method.

Our study has some limitations. First, since the respondents were recruited online, despite the quotas, the structure of the panel was not fully representative. This limitation also exists in other studies.^3,17 Nevertheless, we adjusted the preference weights as in the English and Austrian preference studies,^3,17 and computed the standard errors of the adjusted preference weights. Second, regardless of the exclusion of those with short completion times before the end of the data collection, the used survey administration method did not allow us to observe internal and external incentives or impetuses during the experiment, such as respondent behavior, burden and engagement, or changes in the task environment. However, we conducted face-to-face pretests to learn more about the participants’ response behavior, which was also done in the Austrian study. ¹⁷

To conclude, we have successfully established the preference weights for the Finnish ASCOT for service users instrument. Our contribution enlarges the number of valid measures that can be used to evaluate the capability-based QoL in a general population to consider the impact of social care interventions. The found learning effect calls for the development of study designs that reduce possible bias relating to preference uncertainty at the beginning of the test battery of BWS tasks. The finding also supports using a modelling strategy in which the sequence of tasks is explicitly modelled as a scale factor in an S-MNL model. Similarly, the attribute ordering effect calls for randomizing the items appearing in the choice list. The preference weights serve as a means to promote outcomes research in Finland and support Finnish policy makers in making evidence-based decisions regarding the use of resources for LTC services.