A systematic review of questionnaires about patient’s values and preferences in clinical practice guidelines

Abstract

Objective

We conducted a systematic review to evaluate questionnaires about patient’s values and preferences to provide information on the most appropriate questionnaires to be used when developing clinical practice guidelines.

Methods

A systematic literature search of the Cochrane Library, MEDLINE, Embase, Web of Science, Chinese Biomedical Database, China National Knowledge Infrastructure, and the Wanfang Database was performed to identify studies on questionnaires evaluating patient’s values and preferences. The articles that used fully structured questionnaires or scales with standardized questions and answer options were included. We assessed the questionnaires’ construction and content with a psychometric methodology and summarized the domains and items about patient’s preferences and values.

Results

A total of 7,008 records were retrieved by the search strategy and scanned, and 20 articles were finally included. Of these, 10 (50%) articles described the process of item generation and only four questionnaires (20%, 4/20) mentioned the pilot testing. Regarding “validity”, seven questionnaires (35%, 7/20) assessed validity and only one (5%, 1/20) questionnaire assessed internal consistency, with Cornbrash’s α values of 0.74–0.87. For “acceptability”, the time to complete the questionnaires ranged from 10 to 30 minutes and only nine studies (45%, 9/20) reported the response rates. In addition, the results of domains and items about patient’s preferences and values showed that the “effectiveness” domain was the most considered item in the patient’s value questionnaire followed by “safety”, “prognosis”, and others, whereas the least considered domain was “physician’s experience”.

Conclusion

Only a few studies have developed questionnaires with rigorous psychometric methods to measure patient’s preferences and values. Currently, still there is no valid or reliable questionnaire for patient’s preferences and values for use when developing clinical practice guidelines. Further study should be conducted to develop standardized instruments to measure patient’s preferences and values. This study provides the domains and items that may be used in formulating questionnaires about patient’s preferences and values.

Introduction

Over the last century, there have been several medical innovations offering multiple viable treatment options for most diseases. In addition, increasing patient’s awareness and autonomy have made it necessary to consider patient’s preferences and values in the treatment decision-making process.¹ Patient’s preference refers to a patient’s perspective, expectations, and goals for health, as well as the processes involved in evaluating the potential benefits, harms, and costs of each management option offered to the patient.² Patient’s value refers to the benefit that a patient assigns to a given treatment option.³ Health care policymakers recommend that the patient’s preferences should be considered through active involvement of patient in the treatment decision-making process.^4–7 The National Academy of Medicine (formerly the Institute of Medicine),⁸ the Guidelines International Network,⁹ and Appraisal of Guidelines Research and Evaluation II (AGREE II)¹⁰ all recommend incorporating patient’s values in the development of clinical practice guidelines. A clinical practice guideline that does not consider patient’s preferences may provide recommendations that are not optimal or consistent with patient’s preferences or values.¹¹ For example, the US Preventive Service Task Force clinical practice guideline recommends that the prostate-specific antigen screening for prostate cancer should be patient specific, providing room for the patient’s values and preferences to be incorporated in the clinical decision-making process.¹²

There are several methods to determine patient’s preferences, including questionnaires, interviews, and discrete choice experiments.¹³ Herein, we examined the use of questionnaires to obtain patient’s preferences and values. Questionnaires are an important tool to survey attitudes, knowledge, and practice;^14,15 to generate or refine research questions; and to evaluate the impact of clinical research in practice. Questionnaires can use a descriptive qualitative method¹⁶ to report factual data or an explanatory method¹⁷ to draw inferences and relationships between constructs or concepts. The development of patient’s preference questionnaires should be performed ideally through a psychometric approach, ensuring item generation, pretest and pilot testing,¹⁸ validity,¹⁹ reliability testing,²⁰ and acceptability²¹ in measuring patient’s preferences and values. However, there are many factors that influence the formulation and implementation of questionnaires, including its contents, mode of administration, number of questions, time to complete, the setting of the questions, and ease of understanding.

To contribute to the development of optimal methods to design and conduct questionnaires in primary studies on patient’s values and preferences, we conducted a systematic review to summarize characteristics of studies and assessed the quality of the development of questionnaires used to assess patient’s preferences. Our findings will help researchers to identify the most appropriate questionnaires to use when retrieving information on patient’s preferences and values in different clinical scenarios.

Methods

Data sources and search strategy

An electronic literature search of the MEDLINE, Embase, Web of Science, the Cochrane Library, Chinese Biomedical Database (which is considered as an equivalent of MEDLINE in China and only contains studies published in Chinese), China National Knowledge Infrastructure, and Wan Fang databases was performed, from their inception to January 2018. In addition, hand searches were accomplished in Google from the reference lists of included articles. The search was run using free-text terms and medical subject headings; the search strategy is available as supplementary material.

Eligibility criteria

Studies were included if they 1) focused on patient’s preferences and/or values and 2) included fully structured questionnaires or scales with standardized questions and answer options that were patient (self-) reported. Studies were excluded if 1) they contained incomplete questionnaires of diseases focused on patient’s preferences and/or values; 2) the questionnaires reported a rating by an interviewer; or 3) they were published in a language other than Chinese or English.

Study selection

The study selection process was piloted by two independent reviewers at the beginning of the review. All titles, abstracts, and full articles were independently reviewed by more than two reviewers, and the full-text articles of relevant studies were obtained. Disagreements between the reviewers were initially resolved by consensus and when necessary by a third reviewer.

Data extraction

We developed and piloted a data extraction form to extract the data. The following information was extracted: 1) characteristics of the studies, such as the disease domain, research type, study duration, setting, patient characteristics, and the method used to calculate sample size; 2) characteristics of the questionnaires (ie, mode of administration, item generation, pilot testing, time to complete, response rates, the number of items, the setting of questions, response options, and incentive offered to respondents); and 3) the data related to conduct questionnaires (item and dimension generation, pretest and pilot testing, validity, reliability testing, and acceptability).

We abstracted the content relevant to patient’s preferences into the data extraction form. After discussing and sorting discrepancies related to wording or interpretations, we summarized the domains and items about patient’s preferences. Two authors (JL and FB) independently extracted the data from the questionnaires. Disagreements were initially resolved by consensus and when necessary with the help of a third reviewer (K-HY).

Data analysis and assessment

Two independent reviewers psychometrically assessed the identified questionnaires, determining how items and dimensions were generated and if pretests and pilot testing of the questionnaires were performed for reliability, validity, and acceptability.^22–25 These items were judged as follows: “yes” (when the criterion was explicitly met), “no” (when the criterion was explicitly not met), “uncertain” (when the item was relevant but not described completely), and “not applicable”. Table 1 shows the details of the psychometric properties assessed. Apart from assessing the quality of the questionnaires, we used descriptive statistics to analyze the extracted data and calculated absolute frequencies and proportions for all items.

Table 1.

Items of the psychometric methods

1. Item and dimension generation
The item generation phase should include all important elements of patient’s preferences by reviewing the existing questionnaires, literature, and opinions from experts and patient focus groups. The authors should describe the generation of initial items, including literature review,26 the Delphi survey,27 and consensus meetings,28 etc.	⎕ Yes
	⎕ No
	⎕ Uncertain
	⎕ Not applicable
2. Pretest and pilot testing
The authors should state that they pretested the items of questionnaires. The process of pretest and pilot testing is for revision of the questionnaire into the final validated version by using the response from the protest group. Items with ambiguous meanings can be eliminated to maximize the reliability and validity of the questionnaire.	⎕ Yes
	⎕ No
	⎕ Uncertain
	⎕ Not applicable
3. Validity testing
Content validity. It is usually reported in questionnaires judged by the panel after literature reviews and focus group interviews. During the testing phase, patients are asked to review the items in the questionnaire for content validity on patient’s values and preferences before it is administered to study participants.	⎕ Yes
	⎕ No
	⎕ Uncertain
Construct validity. Because there is no standard for measuring patient’s satisfaction, researchers usually compare their questionnaire with other validated instruments or other related questions for this correlation. Discriminant validity requires that the construct should not show correlation with dissimilar variables.	⎕ Not applicable
4. Reliability testing
The authors should state that they tested the reliability.	⎕ Yes
Internal consistency. Cronbach’s α should be reported, the value should be 0.7–0.9, as a value above 0.9 may indicate that the questionnaire is too narrow in scope.	⎕ No
	⎕ Uncertain
Test-retest reliability. The minimum value of the correlation coefficient should be 0.7.	⎕ Not applicable
Intrarater and interrater agreements. Intrarater agreement is the agreement between observations made by the same rater on two different occasions.
5. Feasibility/acceptability
Time to complete. The authors should provide the time to complete the questionnaires.	⎕ Yes
	⎕ No
Response rates. The authors should provide the response rates of the questionnaires. A response rate of 50% is considered adequate for analysis.29	⎕ Uncertain
	⎕ Not applicable

Results

Study selection

Figure 1 shows a flow diagram of the study selection and identification process. The electronic database search produced 6,956 references, and 52 additional references were identified through other sources. A total of 7,008 records were identified by removing duplicates after the title and abstract screening, excluding 4,974 records, and resulting in 187 articles for full-text assessment. Of these, 21 articles were excluded because they were not specific to patient’s preferences and 106 articles were excluded because they had no questionnaires. In addition, seven articles were excluded because the questionnaires were incomplete. Finally, 20 articles that had complete questionnaires focusing on patient’s preferences were included.

Figure 1.

Trial selection flow chart.

Characteristics of included studies

The final sample included 20 articles in English, published between 1999 and 2017, with study duration ranging from 2 to 20 months, including a total patient sample size of 6,300 (Table 2). Table 2 shows the included study characteristics. Six (30%) of these studies were related to cancer diseases,^{30,33,37,43,46,47} and 15 (75%) were conducted in the outpatient department. Furthermore, nine (45%) studies did not report the study design,^{30,34,38,40,41,43,44,46} 12 (60%) studies provided training to the study participants,^{32–38,40,41,45,47,49} while four (20%) studies did not report information about patient training.^42–44,48 In addition, only five (25%) studies provided the method used to calculate the sample size^{35–39,39–42,42–47} and only six (30%) studies investigated patient’s family members.^{31,34–37,49} Finally, 17 (85%) studies provided information assessing patient knowledge and beliefs.30,32–35,37–45,47–49

Table 2.

Characteristics of included studies

Author (year)	Disease domain	Study design	Study duration (months)	Setting	No of patients	Provided training to patients	Calculated sample size	Involved family members	Examined knowledge about diseases
Bo et al (2014)30	Lung cancer	NR	19	Inpatient	135	NR	NR	No	Yes
Welsh and Tiffin (2014)31	Psychosis	Brief self-report survey	3	NR	75	NR	NR	Yes	No
Vu et al (2015)32	Colonoscopy	Single-center, prospective survey	NR	Outpatient	500	Yes	NR	No	Yes
Tong et al (2016)33	Lung cancer	Pilot study	NR	NR	225	Yes	NR	No	Yes
Sekimoto et al (2004)34	T2DM	NR	3	Outpatient	133	Yes	NR	Yes	Yes
Rid et al (2015)35	Incapacity	Self-administered quantitative survey	20	Inpatient Outpatient	1,116	Yes	Yes	No	Yes
Noble et al (2015)36	Thrombosis	Conjoint analysis surveys	NR	Inpatient	100	Yes	Yes	No	No
Mazur et al (1999)37	Prostate cancer	Cross-sectional study	NR	Outpatient	228	Yes	NR	No	Yes
Matti et al (2010)38	Optic neuritis	NR	5	Outpatient	27	Yes	NR	No	Yes
Maciver et al (2016)39	Heart failure	Qualitative study	12	Outpatient	25	NR	Yes	No	Yes
Sanford et al (2014)40	Myeloid leukemia	NR	3	Outpatient	56	Yes	NR	Yes	Yes
Koh et al (2010)41	ACL reconstruction	NR	22	Outpatient	129	Yes	NR	No	Yes
Ha and Mcdonald (2017)42	Preterm birth	Cross-sectional survey	2	Outpatient	311	No	Yes	Yes	Yes
Gareen et al (2015)43	Colorectal cancer	NR	12	Outpatient	2,310	No	NR	No	Yes
Fiks et al (2013)44	ADHD	NR	6	Outpatient	148	No	NR	Yes	Yes
Eckman et al (2015)45	Thrombus	Cross-sectional study	NR	Outpatient	123	Yes	NR	No	Yes
Choudhry et al (2015)46	Melanoma	NR	12	Outpatient	301	NR	NR	No	No
Calderwood et al (2011)47	Colorectal cancer	Cross-sectional survey	14	Outpatient	100	Yes	Yes	No	Yes
Bolge et al (2016)48	RA	Self-administered online survey	NR	Outpatient	243	No	NR	No	Yes
Hofman et al (2011)49	T1DM	Open-label, randomized, crossover usability test	NR	Inpatient	15	Yes	NR	Yes	Yes

Abbreviations: ACL, anterior cruciate ligament reconstruction; ADHD, attention deficit/hyperactivity disorder; NR, not reported; RA, rheumatoid arthritis; T1DM, type 1 diabetes mellitus; T2DM, type 2 diabetes mellitus.

Characteristics of questionnaires in the included studies

The characteristics of questionnaires in the included studies are listed in Table 3.

Table 3.

Characteristics of included questionnaires

Questionnaire from studies	Mode of administration	Format						Incentives	Ethics statement
		No of items	No of pages	The setting of the question options for the questionnaires	No of sections	Response options (No of items)	Range of scores		Anonymous	Informed consent
Bo et al (2014)30	NR	9	1	SCQ	2	–	–	NR	Yes	Yes
Welsh and Tiffin (2014)31	Interview	10	3	CT	4	4-point scale (10)	Very likely, likely, unlikely, very unlikely	NR	NR	Yes
Vu et al (2015)32	Interview	12	2	SCQ+CT+OeQ	2	4-point scale (3)	Very important to not important at all	NR	Yes	NR
Tong et al (2016)33	Interview	50	17	SCQ+CT	4	5-point scale (3)	Strongly prefer left, somewhat prefer left, indifferent, somewhat prefer right, strongly prefer right	Yes	Yes	Yes
Sekimoto et al (2004)34	Interview	7	2	SCQ+CT	2	–	–	NR	Yes	Yes
Rid et al (2015)35	Interview	46	6	SCQ+CT+OeQ	4	4-point scale (23)	Extremely important, moderately important, just a little important, not important at all	NR	Yes	NR
Noble et al (2015)36	Interview	10	7	SCQ+CT+OeQ	2	5-point scale (10)	Do not agree at all to fully agree	NR	Yes	Yes
Mazur et al (1999)37	Interview	2	1	SCQ+OeQ	1	–	–	NR	NR	Yes
Matti et al (2010)38	Interview	19	3	MCQ+SCQ	3	–	–	NR	Yes	Yes
Maciver et al (2016)39	Interview	6	3	SCQ+OeQ	2	–	–	NR	Yes	Yes
Sanford et al (2014)40	Interview	18	4	SCQ+CT+OeQ	5	5-point scale (11)	Nil, minor, tolerable, tolerable but I adjust my activities or take medications for it, intolerable	NR	Yes	Yes
Koh et al (2010)41	NR	7	1	MCQ+SCQ	1	–	–	NR	Yes	Yes
Ha and Mcdonald (2017)42	NR	17	6	SCQ+CT	6	5-point scale (3)	Extremely important, slightly important, neutral, slightly not important, not at all important	NR	Yes	NR
Gareen et al (2015)43	Interview	3	9	SCQ+OeQ	6	–	–	NR	NR	NR
Fiks et al (2013)44	Interview	9	1	CT	3	6-point scale (9)	Not at all, a little, somewhat, very much, completely	NR	Yes	Yes
Eckman et al (2015)45	Interview	2	1	SCQ	2	–	–	NR	Yes	Yes
Choudhry et al (2015)46	Interview	11	4	SCQ+CT+OeQ	3	–	–	NR	No	Yes
Calderwood et al (2011)47	Interview	20	12	SCQ+CT	3	5-point scale (13)	Strongly agree, agree, neither agree nor disagree, disagree, strongly disagree	Yes	Yes	Yes
Bolge et al (2016)48	online survey	11	3	SCQ+CT	2	5-point scale (2) 4-point scale (1)	Not at all open, not very open, somewhat open, very open, extremely open; strongly prefer self-injection to strongly prefer IV infusion	NR	Yes	Yes
Hofman et al (2011)49	Interview	13	2	CT+OeQ	2	6-point scale (13)	Very difficult–very easy; not at all likely–very likely	NR	Yes	Yes

Abbreviations: CT, choice task; IV, intravenous; MCQ, multiple-choice question; NR, not reported; OeQ, open-ended question; SCQ, single-choice question.

Most questionnaires were found to be easy to administer; 16 questionnaires (80%) were conducted through an interview survey,^{31–40,43–47,49} whereas only one questionnaire (5%) was conducted via an online survey;⁴⁸ three questionnaires (15%) did not report the method used for survey administration.^30,41,42 The mean number of pages per questionnaire was 4.4 (range, 1–17), the mean number of items per questionnaire was 14 (range, 2–50), and 16 (80%) questionnaires were filled anonymously.30,32–42,44,45,47–49 Further, 16 studies (80%) obtained informed consent from the patients before the study,^{30,31,33,34,44–49} whereas four studies (20%) did not report if informed consent had been obtained.^32,35,42,43 Most questionnaires used 4- to 7-point scales of measurement. The single-choice question format with open-ended questions and a choice task was the most common type of survey setting (25%), followed by single-choice questions with a choice task (33%). In addition, two studies^33,47 (10%) reported that they used a patient incentive during the survey, of which one study³³ reported that respondents were provided with a bar-code to initiate payment of $1.00 and another study⁴⁷ reported compensation of respondents with a $15 gift card.

Questionnaire content evaluation

The evaluation of contents of questionnaire is summarized in Table 4.

Table 4.

The evaluation of components of questionnaires

	Item generation	Pilot testing and redesign	Validity testing	Reliability testing (Cronbach’s α)	Acceptability
					Time to complete	Response rates
Bo et al (2014)30	No	No	No	No	No	No
Welsh and Tiffin (2014)31	No	No	No	No	No	Yes
Vu et al (2015)32	No	No	No	No	No	No
Tong et al (2016)33	Yes	Yes	No	No	Yes	Yes
Sekimoto et al (2004)34	No	No	Yes	No	No	Yes
Rid et al (2015)35	Yes	Yes	Yes	No	Yes	Yes
Noble et al (2015)36	Yes	No	Yes	No	Yes	No
Mazur et al (1999)37	No	No	No	No	No	No
Matti et al (2010)38	Yes	No	No	No	Yes	Yes
Maciver et al (2016)39	Yes	Yes	No	No	No	No
Sanford et al (2014)40	No	No	No	No	Yes	Yes
Koh et al (2010)41	Yes	No	Yes	No	No	No
Ha and Mcdonald (2017)42	Yes	No	Yes	No	Yes	Yes
Gareen et al (2015)43	No	No	Yes	No	Yes	Yes
Fiks et al (2013)44	Yes	Yes	Yes	Yes	No	Yes
Eckman et al (2015)45	Yes	No	No	No	No	No
Choudhry et al (2015)46	Yes	No	No	No	No	Yes
Calderwood et al (2011)47	No	No	No	No	Yes	No
Bolge et al (2016)48	No	No	No	No	Yes	No
Hofman et al (2011)49	No	No	No	No	No	No

From the included studies, 10 (50%) described the process of item generation.^{33,35,36,38,39,41,42,44–46} As for pretest and pilot testing, only four questionnaires (20%) mentioned that the survey had undergone cognitive pretesting or pilot testing.^33,35,39,44 Regarding validity, seven questionnaires (35%) assessed validity by asking patients if there were other aspects of care that were not mentioned in the questionnaires,^{34–36,42–44} whereas only one questionnaire (5%) assessed internal consistency using Cronbach’s α with values of 0.74–0.87.⁴³ For acceptability, the time to complete the questionnaires ranged from 10 to 30 minutes, with nine studies (45%) reporting such data.^{35,36,38,40,42,43,47,48} Furthermore, response rates of the questionnaires exceeded 50% as reported by 10 studies.^{31,33–35,38,40,42–44,46} In addition, only two studies showed the highest questionnaire content quality following psychometric analysis;^35,44 four studies did not follow the psychometric methods to use the questionnaire measuring patient’s preferences and values since neither did they describe item generation and pilot testing nor did they report the related information of validity, reliability, and acceptability.^30,32,37,49

Items and domains identified for measuring patient’s preferences

Table 5 presents a list of domains and examples of items about patient’s preferences and values in the questionnaires reviewed. We identified the items that measured patient’s values and preferences and grouped them into four domains, namely effectiveness, safety, prognosis, and others. The “others” domain included information on cost, physician’s experience, physician’s recommendation, and initiation of the decision-making process. Some domains and items were more frequently reported than others. For example, the “effectiveness” domain was the most considered in the patient’s value questionnaire (36.4%, 12/33), while the least considered domain was “physician’s experience” (3.03%, 1/33).

Table 5.

Summary of domains and items for reviewing the questionnaires addressing values and preferences

Domain	Study ID, author (year)	Examples	Scale	Type of disease
Effectiveness
The convenience of treatment	Bolge et al (2016)48	Would you prefer to take your biologic medication by self-injection at home or have the medication given to you by a health care professional through intravenous infusion? 1=strongly prefer self-injection, 2=somewhat prefer self-injection, 3=no preference between self-injection and intravenous infusion, 4=somewhat prefer intravenous infusion, 5=strongly prefer the intravenous infusion	5-point scale	RA
	Hofman et al (2011)49	How easy or difficult it was to detach/remove the needle cap? 1=very difficult, 6=very easy	6-point scale	Adolescents with diabetes
The frequency required for testing	Calderwood et al (2011)47	Which of the following attributes most influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	SCQ	Colorectal cancer
The types of drug preparations and therapy	Welsh and Tiffin (2014)31	How likely would you be to accept this kind of support? Usually given as tablets Tablet or liquid format Talking therapy	4-point scale	Psychosis
The benefit of the interventions	Ha and Mcdonald (2017)42	Did the medication work very well? (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Ha and Mcdonald (2017)42	When considering cerclage treatment, how important or unimportant is each of the following Whether the medication works well (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Sekimoto et al (2004)34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
Treatment time	Tong et al (2016)33	If these procedures were identical in all other ways, which would you prefer? Open surgery: 3 hours Minimally invasive surgery: 3 hours Focused radiation: 1-hour outpatient visit	SCQ	Lung cancer
	Tong et al (2016)33	What bothers you the most about focus radiation? Need to be still for 1 hour for each treatment The tumor is not removed, only killed Difficult traveling for outpatient treatments	SCQ	Lung cancer
The frequency of drug administration	Noble et al (2015)36	Which frequency of administration do you prefer? Twice daily (duration of action over 24 hours) Once daily (duration of action over 12 hours)	SCQ	Cancer-associated thrombosis
	Sanford et al (2014)40	How likely are you to take your medications every day? Very unlikely, unlikely, neither unlikely nor likely, likely, very likely	5-point scale	CML
Hospital stay time	Tong et al (2016)33	What bothers you the most about open surgery? 5-night hospital stay 6- to 8-week recovery time at home	SCQ	Lung cancer
Safety
The risk of adverse effects	Bo et al (2014)30	Reasons for treatment decision making (choose the three answers with the most consideration): Fewest side effects Physician’s recommendation Least painful	SCQ	Lung cancer
The risk of complications	Ha and Mcdonald (2017)42	When considering cerclage treatment, how important or unimportant is each of the following: Concerns about potential harm to the baby (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Calderwood et al (2011)47	Which of the following attributes mostly influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	5-point scale	Colorectal cancer
Communicating the risk of the interventions	Mazur et al (1999)37	Which do you prefer when your physician communicates to you the risk of the intervention? Words Numbers Other	SCQ	Lung cancer
	Sekimoto et al (2004)34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
The risk of disease transmission	Koh et al (2010)41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
Prognosis
The risk of relapse	Sanford et al (2014)40	In the following questions, if the risk of relapse is different, how likely are you to stop medications for your CML? If the risk of relapse were 20%, 40%, 60%?, I would absolutely stop, I would likely stop, I would be neutral, I would likely not stop, I would absolutely not stop	SCQ	CML
	Sekimoto et al (2004)34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
	Matti et al (2010)38	What is your view about having an action plan in the event of a relapse? Not interested, may be useful but unsure, will consider, will definitely have an action plan	SCQ	Optic neuritis
Better healing after treatment/recover time	Koh et al (2010)41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL
Others
The cost patients have to pay	Ha and Mcdonald (2017)42	When considering cerclage treatment, how important or unimportant is each of the following Whether there would be a cost for you (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Calderwood et al (2011)47	Which of the following attributes mostly influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	SCQ	Colorectal cancer
	Koh et al (2010)41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
Physician’s experience	Tong et al (2016)33	How important is your physician’s experience to you? Unimportant compared with other factors, somewhat important, very important, extremely important	4-point scale	Lung cancer
Physician’s recommendation	Bolge et al (2016)48	When deciding on a treatment for rheumatoid arthritis, which of the following best applies? 1=I make the final treatment decision, 2=I make the final treatment decision after considering my rheumatologist’s recommendations, 3=I share responsibility with my rheumatologist when deciding on a treatment, 4=my rheumatologist makes the final treatment decision, but seriously considers my opinion, 5=I leave all treatment decisions to my rheumatologist	5-point scale	RA
	Bolge et al (2016)48	How open are you to having your RA medication administered at home by self-injection if your rheumatologist suggested it? 1=not at all open, 2=not very open, 3=somewhat open, 4=very open, 5=extremely open	5-point scale	RA
	Sekimoto et al (2004)34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
	Bolge et al (2016)48	When deciding on a treatment for rheumatoid arthritis, which of the following best applies? 1=I make the final treatment decision, 2=I make the final treatment decision after considering my rheumatologist’s recommendations,3=I share responsibility with my rheumatologist when deciding on a treatment, 4=my rheumatologist makes the final treatment decision, but seriously considers my opinion, 5=I leave all treatment decisions to my rheumatologist	5-point scale	RA
	Koh et al (2010)41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
	Bo et al (2014)30	Reasons for treatment decision-making: (choose the three answers with the most consideration) Fewest side effects Physician’s recommendation Least painful	SCQ	Lung cancer
Initiation of the decision process	Maciver et al (2016)39	Who do you think should start these discussions? Would you want the physician to wait until you asked about it or would you want the physician to start the discussion?	OeQ	ICD treatment

Abbreviations: ACL, anterior cruciate ligament; CML, chronic myeloid leukemia; ICD, implantable cardioverter-defibrillators; MCQ, multiple-choice question; OeQ, subjective questions; RA, rheumatoid arthritis; SCQ, single-choice questions; T2DM, type 2 diabetes mellitus.

Discussion

The present study reviews the content and construction of questionnaires to provide information on the most appropriate questionnaires to assess patient’s preferences and values for the development of clinical practice guidelines. Our survey of the literature showed that few questionnaires met the psychometric methodology, resulting in a low quality of content and construction of questionnaires, and that even fewer studies reported the method used to calculate the sample size. These results emphasize the importance of having a standard method available to design and conduct questionnaires to survey patient’s preferences and values. Future research should follow the psychometric methodology to develop questionnaires measuring patient’s preferences and values, including the main steps of item generation, pretesting or pilot testing of the initial questionnaire, and testing of the final version of the questionnaire for content validity, reliability, and acceptability. In addition, ethics statements⁵⁰ of questionnaires (referring to informed consent) should be described as “written informed consent was obtained from each patient prior to study participation” or “written consent was obtained prior to initiating the survey”.

Another issue identified in our systematic review was that domains and items regarding patient’s preferences and values should consider effectiveness, safety, prognosis, and other factors into the formulation of patient’s value and preference questionnaires.

Effectiveness

Treatment effectiveness is the most considered aspect by patients. The effectiveness domain included questions on the convenience of the treatment option, the frequency of testing, the types of drug preparations, the benefit of the intervention, treatment time, and frequency of drug administration. The benefit of the intervention was the most important factor for the majority of patients in the domain of effectiveness, followed by the convenience of the treatment option, hospital length of stay, and frequency of testing. Medication route, types of drug preparations, frequency of drug administration, and duration of treatment are important considerations when evaluating patient’s preferences. These factors should be considered when designing items for patient’s preference and value questionnaires.

Safety

The safety domain includes adverse effects, complications from interventions, and the risk of disease transmission. The risk of complications and communicating the risk of the interventions were the most important factors for the majority of patients in the domain of safety, followed by the risk of adverse effects and disease transmission. The majority of treatment options are accompanied by adverse effects, including infection, bleeding, pain, and, at the extreme, death.^50–53 Therefore, when developing questionnaires to measure patient’s values and preferences, it is important to include questions on treatment safety, so that patients can make an informed decision regarding their preferred treatment. For example, Rid et al³⁵ measured safety by asking “when your physician communicates with you about medical risk (that is, the chance or come will occur related to a probability of adverse effects and complications with specific medical intervention you are considering), which treatment would you prefer?”

Prognosis

Prognosis includes the risk of relapse and the likelihood of a return to normal health following treatment. Our study recommends a detailed display of information on prognosis when formulating such questionnaires. For example, statements such as “we will ask you to think about how you would feel about these different treatments, you will be given detailed information about three treatment options that differ in terms of the risk of relapse, recovery time, and the chance of returning to normal health after treatment, which treatment would you prefer?” should be presented. In summary, all diseases and their paths vary, and patients have varying emotions, values, and preferences; therefore, prognosis factors should be considered to make more valid questionnaires on patient’s values and preferences.⁵⁴

Others

The “others” domain included cost, physician’s experience, physician’s recommendation, and initiation of the decision-making process. The results showed that the financial cost or financial burden of treatment accounts for a large proportion of the questionnaire items affecting patient’s values and preferences when considering treatment choice. Questionnaire developers could display the medical expenses for alternative treatments in detail to fully capture the factors influencing patient’s preferences with regards to cost. In addition, the physician’s recommendation and experience should be considered in the development of questionnaires on patient’s preferences.

Strengths and limitations

This is the first study to conduct a systematic review evaluating questionnaires measuring patient’s values and preferences. The results herein will provide valuable guidance to researchers and policymakers during the development of clinical practice guidelines and in the shared clinical decision-making process. The strengths of our study include explicit eligibility criteria, a comprehensive literature search, duplicate assessment of eligibility with a high agreement, and a detailed iterative process for the identification of items suggested in the reviews and categorization of items into domains.

Nevertheless, it also has several limitations. First, only studies published in English were included. Second, although we finally identified 20 complete questionnaires, it is possible that we failed to identify other eligible questionnaires due to varying terminology and suboptimal indexing of patient’s preferences.

Conclusion

Only a few studies have developed questionnaires with rigorous psychometric methods to measure patient’s preferences and values, and there are still no valid or reliable questionnaires to do this in the process of treatment decision making and for the development of clinical practice guidelines.

Availability of data and material

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Supplementary material

• Search strategies.

• Search strategies (take PubMed as an example):

• #1 Search “Guidelines as Topic” [Mesh]

• #2 Search “Practice Guidelines as Topic” [Mesh]

• #3 Search “Guideline” [Publication Type]

• #4 Search (((((guideline* [Title/Abstract]) OR consensus [Title/Abstract]) OR instruction[Title/Abstract]) OR routine [Title/Abstract]) OR “clinical practice” [Title/Abstract]) OR “recommendation*” [Title/Abstract])))))

• #5 Search #1 OR #2 OR #3 OR #4

• #6 Search “Patient Preference” [Mesh]

• #7 Search patient preference* [Title/Abstract]

• #8 Search #6 OR #7

• #9 Search “Patient Satisfaction” [Mesh]

• #10 Search Patient* Satisfaction [Title/Abstract]

• #11 Search #9 OR #10

• #12 Search “Attitude to Health” [Mesh]

• #13 Search (attitude to health [Title/Abstract]) OR health attitude* [Title/Abstract]

• #14 Search #12 OR #13

• #15 Search “Treatment Adherence and Compliance” [Mesh]

• #16 Search (adherence [Title/Abstract] AND compliance [Title/Abstract])

• #17 Search #15 OR #16

• #18 Search patient decision [Title/Abstract]

• #19 Search patient acceptance [Title/Abstract]

• #20 Search “Patient Acceptance of Health Care” [Mesh]

• #21 Search #19 OR #20

• #22 Search patient perspective [Title/Abstract]

• #23 Search health state utilit* [Title/Abstract]

• #24 Search #8 OR #11 OR #14 OR #17 OR #18 OR #21 OR #22 OR #23

• #25 Search #5 AND #24

Abbreviations

AGREE II

Appraisal of Guidelines Research and Evaluation II

NR

not reported

ACL

anterior cruciate ligament reconstruction

ADHD

attention deficit/hyperactivity disorder

RA

rheumatoid arthritis

T2DM

type 2 diabetes mellitus

T1DM

type 1 diabetes mellitus

ICD

implantable cardioverter-defibrillators

CML

chronic myeloid leukemia

SCQ

single-choice question

OeQ

subjective question

MCQ

multiple-choice question