Development of a patient experience questionnaire to improve lifestyle services in primary care

Abstract

Aim

We developed a self-report tool to assess patients’ experiences with lifestyle services in team-based primary care and pilot tested the questionnaire as part of a lifestyle intervention study to reverse metabolic syndrome.

Background

Older client satisfaction questionnaires have been generally inadequate for quality improvement purposes, as they have been focused mainly on interpersonal skills of providers and/or in the context of one disease. New approaches to assessing the patient experience of lifestyle programs in primary care are needed and could inform quality improvement efforts over time.

Methods

The first phase in developing the questionnaire involved a group-administered questionnaire distributed to 38 healthcare providers in five groups to prioritize variables to include in the survey. Concepts were taken from a previous review of available questionnaires assessing primary care services. The draft questionnaire was reviewed by 11 participants from a lifestyle program using think-aloud cognitive interviewing techniques. The modified self-administered questionnaire (paper and online versions) was then pilot tested with 164 recipients of a nutrition and physical activity intervention program.

Findings

Providers ranked the top variables to include in the questionnaire as: ‘trust,’ ‘general communication,’ ‘first-contact accessibility,’ ‘whole-person care,’ and ‘respectfulness.’ After cognitive interviewing and revisions, 21 multiple choice and two open-ended questions were used for pilot testing. Pilot testing identified additional minor wording changes that were needed for clarity, a decreased number of questions for redundant concepts, and decreased options for ceiling effects, resulting in 20 multiple choice and one open-ended question.

Conclusions

The modified self-administered patient experience questionnaire to assess lifestyle services in primary care has undergone rigorous development. Further validation is needed. The assessment of patient experience of lifestyle programs can be used to supplement other data to assess the overall effectiveness of such programs.

Background

Lifestyle programs to improve diet and physical activity behaviors are increasingly being offered in team-based primary care (PC) in Canada, largely related to efforts to curb the obesity epidemic, prevent type 2 diabetes, and decrease cardiovascular risk. Such programs are costly to organize and deliver, are currently taking many forms, and vary in length, format, and content. Anecdotally, such programs often suffer from high attrition rates, although this is not well documented in the healthcare system (Miller and Brennan, 2015). What is known is that attrition in weight loss programs varies from 10 to 80% and is associated with poor long-term adherence to lifestyle change (Moroshko et al., 2011; Desroches et al., 2013). Research to date has not been able to identify optimal formats and organization of programs, although it is generally thought that longer (weeks to months), more behaviorally oriented programs are more successful (Lin et al., 2014). Lifestyle change is inherently challenging for most people, and it would be expected that the organization of program delivery may have an impact. Assessment of the patient experience at the program level could add valuable information to inform quality improvement efforts over time. Older client satisfaction questionnaires have generally been inadequate for quality improvement purposes, as they have focused mainly on interpersonal skills of one provider and/or in the context of one disease (Salisbury et al., 2010). Choice options or scaling have also often been inadequate to detect more minor problems, with patients being mainly satisfied with their experiences.

Lifestyle services are only one possible service within PC and it would be best if any new questionnaire could be positioned within overall development of indicators of the patient experience in PC more generally. In Canada, the Canadian Institute for Health Information (CIHI) undertook a program of work to develop a comprehensive national framework for assessing the quality of PC services (CIHI, 2006; 2012). Some indicators are best assessed by patients and considerable work was done to develop draft questionnaires relevant to the patient experience (CIHI, 2013). Wong and Haggerty reviewed previous publicly available questionnaires and identified 22 dimensions and sub-dimensions of PC services (Wong and Haggerty, 2013). This approach was taken as these dimensions represented a ‘reduced’ list of commonly agreed upon aspects of PC (Haggerty et al., 2007).

Building on this work, we used the Wong and Haggerty framework to develop a set of questions addressing patient experience of lifestyle programs, which could be used on their own or with other questions developed by CIHI or others. The intent was to develop a series of questions for self-report that could be adapted to different PC practice contexts, were in line with previous work on PC quality indicators, aligned with key concerns for program improvement, and were broadly aimed at both diet and exercise. Core principles of survey design were addressed (Aday and Cornelius, 2006), and we report here initial development and pilot testing of the questions based on the input of the providers delivering the programs as to the most important dimensions to assess.

The specific objectives were to: (i) develop a self-report tool to assess patients’ experiences with lifestyle services (nutrition and physical activity) offered in interdisciplinary PC settings and (ii) pilot test the tool as part of an evaluation of a lifestyle intervention study to reverse metabolic syndrome.

Methods

The development of the questionnaire consisted of three phases: determination of the high-priority concepts/variables to include, generating the first draft of the questionnaire and modifying it based on feedback, and pilot testing the modified questionnaire in a PC nutrition and physical activity intervention program. Ethics approval for the first two phases was obtained from the University of Guelph Research Ethics Board and for the third phase from the Research Ethics Boards of the pilot testing sites.

Phase 1: Determining questionnaire topics

Study design

The first task was to reduce and prioritize which variables should be considered in a self-administered questionnaire. A group-administered cross-sectional questionnaire study design was chosen, based on prior experience of the strengths and limitations of inter-professional focus groups in eliciting diverse perspectives on lifestyle services in PC (Bonilla et al., 2016; Bourgeois et al., 2016), the complexity of the framework which required some explanation (see Figure 1), and the need to document rankings.

Figure 1.

Dimensions of patients’ experiences in primary healthcare (Wong and Haggerty, 2013). Rectangle boxes represent dimensions and spheres in the same color represent sub-dimensions. Black box indicates top-ranked dimensions, and check marks indicate next ranked dimensions.

Development of group-administered questionnaire

The plan for the group session was developed with a professional moderator who recommended use of a graphic to support review of the dimensions of the framework (see Figure 1), followed by general group discussion and review, before individual completion of a ranking exercise using a paper-based questionnaire.

Population/sampling

As the questionnaire would be mainly used by PC providers offering health education/coaching/counseling services for diet and physical activity, it was considered relevant to solicit the views of providers on the most critical dimensions to include. Family Health Teams (FHTs) were chosen to recruit providers as they include a variety of healthcare providers in addition to family physicians, and many offer lifestyle group programs [Ontario Ministry of Health and Long Term Care (MOHLTC), 2016]. All FHTs within a 2-h travel radius of the University of Guelph who had allowed the MOHLTC to post their contact information were invited by email to participate.

Data collection

Participants provided written informed consent before beginning the group session, which was conducted over the lunch period. Each session was moderated by the professional moderator, who guided participants through a discussion of the Wong and Haggerty dimensions and sub-dimensions (Wong and Haggerty, 2013) and elicited participants’ ideas of aspects most important to include in a patient experience questionnaire on lifestyle services. Participants then individually rated the importance of each of the dimensions/sub-dimensions, and ranked their top five dimensions/sub-dimensions. Each session lasted about 60 min; sessions were audio-recorded and a note-taker took notes on the main concepts discussed. After analysis, results of ranking were sent back to participants by email as a form of member checking.

Analysis

Using the ranking exercise responses, each dimension/sub-dimension was given a total importance score to inform prioritization, with higher scores indicating higher prioritization for inclusion in the questionnaire. Dimensions/sub-dimensions were then given a priority rank by discussion group score and an overall priority rank by total score. We further analyzed the discussion group audiofiles using directed qualitative content analysis to identify themes that added context to ranking results (Hsieh and Shannon, 2005).

Phase 2: Creating draft for pilot testing

A sequential process of question development and revision was undertaken. Possible questions were adapted from previous questionnaires. New items were generated if suitable questions were not found. Only one to two questions were chosen to address each concept.

A convenience sample of participants in lifestyle programs reviewed the questions using cognitive interviewing. Participants were recruited from a group of 25 patients attending a lifestyle program at one FHT. Of the 25 patients, 18 expressed interest in participating in the interview and 11 were available for cognitive interviewing. A sample size of 10 is generally sufficient to identify major issues (Blair and Conrad, 2011). Interviews were scheduled for 60 min and digitally audio-recorded. The interviews were conducted using a combination of ‘think aloud’ techniques with scripted, spontaneous, concurrent, and retrospective probing (Willis, 1999; Beatty and Willis, 2007). Participant comments and problems identified were aggregated by question and grouped by problem type using Willis’ coding classifications (Buers et al., 2013). Revisions were made to the questionnaire after each set of five or six interviews. After revision, the 23 questions were ready for pilot testing (Table 1).

Table 1.

Concepts considered for the first questionnaire draft, by Wong and Haggerty dimension/sub-dimension

Dimension	Sub-dimension	Discussion concepts considered for first draft	Questions used for pilot testing	Revised questions after pilot testing
Access	First-contact accessibility	Marketing of services Wait times Provider of choice Timely access	(1) How did you find out about this service? (2) How did you access this service? (4) How long did you wait from the time you were referred to the time you had your first appointment? (9) Were you able to contact a team member about your nutrition and/or physical activity when you wanted to?	(1) How did you find out about this service? (2) How did you access this service? (4) How long did you wait from the time you were referred to the time you had your first appointment? (9) Were you able to contact a team member about your nutrition and/or physical activity when you wanted to outside of scheduled appointments?
	Accommodation	Specific and different issues in each FHT
	Economic accessibility	Barriers to attending appointments Barriers to making changes	(8) Were there times when you found it difficult to get to nutrition and/or physical activity appointments because of the additional costs or time it involved (eg, childcare, parking, transportation, work schedule, other commitments)?	(8) Were there times when you found it difficult to get to nutrition and/or physical activity appointments because of the additional costs or time it involved (eg, childcare, parking, transportation, work schedule, other commitments)?
Interpersonal communication	General communication	Understandable communication Enough time with providers Clear expectations of services provided	(6) Did you know what to expect from the nutrition and/or physical activity services in terms of how many appointments you would have, what you would learn, and the amount of support you would be offered? (10) How often did the team members explain things to you in a way that you clearly understood?	(6) Before the program started, did you know what to expect from the nutrition and/or physical activity services in terms of how many appointments you would have, what you would learn, and the amount of support you would be offered? (10) How often did the team members explain things to you in a way that you clearly understood?
	Respectfulness	Feeling heard Feeling respected	(12) How often did the team members treat you with courtesy and respect?	(12) How often did the team members treat you with courtesy and respect? Changed to 2 options: ‘Always’/‘not always’
	Shared decision making	Patients goals Joint discussion	(17) Were you involved in setting nutrition and/or physical activity goals to address the health issues that matter to you?	(17) Were you involved in setting nutrition and/or physical activity goals to address the health issues that matter to you?
	Whole-person care	Take whole person into consideration	(13) Did the team members consider your personal situation (lifestyle, income, traditions, and culture) when making nutrition and/or physical activity recommendations for you?	(13) Did the team members consider your personal situation (lifestyle, income, traditions, and culture) when making nutrition and/or physical activity recommendations for you?
Continuity and coordination	Relational continuity
	Informational continuity	Patient information that was needed was available to all healthcare providers
	Coordination
	Team functioning	Consistent messaging	(15) How often were you told different things (that did not make sense together) about nutrition and/or physical activity?	(15) How often were you told different things by different team members about nutrition and/or physical activity?
Comprehensiveness of services	Services provided	Meet needs Suggest improvements Importance of service to patient	(3) How would you rate the importance of this service to your health? (7) How would you describe the program length to support your lifestyle changes? (16) Have the team members provided the information and support you need to make changes to your nutrition and/or physical activity? (23) Are there services that you think should be added, removed, or changed?	(3) How would you rate the importance of this service to your health? (7) How would you describe the program length in total in supporting your lifestyle changes? Question #16 deleted and replaced with original Q22 below: (22) Has working with the team members on nutrition and/or physical activity helped you better manage your health? (20) Are there services that you think should be added, removed, or changed?
	Health promotion and primary prevention
Trust		Comfortable sharing personal information Trust provider Trust information	(11) Were you comfortable sharing personal information with the team members? (14) Did you have confidence in the information about nutrition and/or physical activity that the team members gave you?	(11) Were you comfortable sharing relevant personal information with the team members? (14) Did you have confidence in the information about nutrition and/or physical activity that the team members gave you?
Patient-reported impacts of care	Patient activation Patient safety Confidence in the PHC system	Action on goals	(18) How confident are you that you can maintain the changes in your nutrition? (19) How confident are you that you can maintain the changes in your physical activity? (20) Were there times when you did not try the suggested nutrition changes because something got in the way? (21) Were there times when you did not try the suggested physical activity changes because something got in the way? (22) How has working with the team members on nutrition and/or physical activity helped you better manage your health?	(18) How confident are you that you can maintain the changes in your nutrition? (19) How confident are you that you can maintain the changes in your physical activity? (20) Were there times when you were unable to make the suggested nutrition and physical activity changes because something got in the way? Question #21 repetitive – combined with original question #20 Overlap with original question #16 Question #22 moved to new question #16
Additional concepts		Attrition	(5) Did you stop attending before the program finished?	(5) Did you stop attending before the program finished?

Phase 3: Pilot testing questionnaire in PC

A 12-month nutrition and physical activity before-after feasibility study for metabolic syndrome (n=293) conducted in three diverse PC sites across Canada, provided an ideal opportunity to pilot test the questionnaire (Jeejeebhoy et al., 2017). The objective of this phase was to assess multi-modal delivery feasibility, clarity, missing/redundant concepts, and floor/ceiling effects.

Adaptation of the generic questionnaire consisted of writing an introduction and having the research group (including several family physicians) review the questionnaire. No further changes were made to the questions. The questionnaire was translated into both French and Russian to meet the language needs of the participating PC sites. Participants could complete either a paper-based version or an online version (French and English) available using Qualtrics online survey software (qualtrics.com 2015 Provo, Utah). The Russian version was offered only in paper format. Questionnaire data collection was managed by research coordinators (RCs) from each of the PC sites. Each RC received the questionnaires in the languages appropriate for their site along with pre-stamped return envelopes. The paper questionnaire included a web address to the online questionnaire so that participants could complete the questionnaire in either paper or online modalities. RCs were also provided with three different reminder modes: paper postcards, a telephone script template, and an email template. RCs were instructed to remind patients to complete the questionnaire using one of the three modes, 1 to 2 weeks after the questionnaire was mailed out to the participants but this process was not monitored.

Participants were asked to complete the questionnaire after completion of the 12-month study or at the point where they had decided to withdraw from the study. There was no financial remuneration offered for completing the questionnaire. For the analysis, all paper questionnaires were transcribed into Qualtrics and all data was then downloaded and exported into Microsoft Excel (2010) for analysis. Analysis included questionnaire response rates and individual question response rate. Comments written on the questionnaires were thematically analyzed to identify issues of question redundancy or clarity.

Results

Provider ranking and discussion of dimensions/sub-dimensions

A total of 38 providers attended a discussion at one of five FHTs. These included one rural FHT, three urban non-academic FHTs, and one urban academic FHT. Most participants identified as female (92%), the average age was 40, and there were multiple provider categories in attendance including nurses/nurse practitioners (n=10), dietitians (n=7), mental health counselors (n=5), administrative staff (n=7; eg, executive directors, reception staff), and other (n=9; not disclosed but included other health professionals not listed). There were no family physicians in the discussion groups despite requests to recruit physicians as part of the mixed group of providers. On average, participants had been in their profession for 14 years and had worked in their FHT for 3.8 years. The top five provider ranked dimensions/sub-dimensions were: ‘trust,’ ‘general communication,’ ‘first-contact accessibility,’ ‘whole-person care,’ and ‘respectfulness’ (Figure 1 also shows the next five ranked dimensions/sub-dimensions). This ranking did not change when the number of top five votes was tallied for each dimension and sub-dimension. Member checking confirmed the top five ranked dimensions.

As noted in Table 1, column 3, providers understood the concepts from the framework well, but would have combined some categories, added a category or felt some concepts overlapped. Within comprehensiveness of services, some participants did not see the need to separate out the two sub-dimensions, ‘services provided’ and ‘health promotion and primary prevention’:

I would see services provided and health – I would see both of these – I’m having a hard time separating them. I’m seeing them as one… so the services I’m giving are more health promotion or preventing further complications down the line rather than kind of that immediate tertiary care…

(DG2 Participant)

The concept of equity was not obvious in the framework and was identified in one group as an important aspect to capture:

What about equity? So all patients whether they have a home or are homeless or handicapped have equitable service, the same as your typical patient. Equity you know whether it be a child or have a horse and buggy…how can we meet those needs.

(DG4 Participant)

For some participants, there were dimensions of the framework that overlapped substantially such as access and comprehensiveness of services, trust with all other dimensions, whole-person care with team functioning, and accommodation with patient safety.

So you could, you could attribute trust to every single – It ties in globally… I think trust ties into every single of these categories. …Every single one.

(DG1 Participant)

Certain dimensions were not highly ranked for a patient questionnaire, but were discussed by provider groups in detail. For example, ‘accommodation’ issues came up in each group but the specific issues differed across organizations. ‘Team functioning’ was a sub-dimension that many felt would be difficult for patients to comment on, as sometimes patients are not aware of who is part of the team and that a well functioning team might not be ‘noticed.’ Several groups discussed the need to find out if services provided were meeting patients’ needs, however thought it would be difficult for patients to know and compare different possible services to know what might meet their needs better. Comments from member checking confirmed the final ranking, with recognition that all dimensions are important to quality care.

Developing questionnaire

We used the ranking and discussion concepts that emerged from the review of transcripts as a starting place for selecting and adapting questions from previous work, especially analyses done in the development of the CIHI patient experience questionnaire (Table 1) (Haggerty et al., 2011a; 2011b). Response options for Likert questions were similar to past questions. For example, within the sub-dimension of ‘first-contact accessibility,’ providers mentioned that patients may have a poor awareness of the available lifestyle services and might not find out about them for many years. ‘Services provided’ had not been ranked highly, but given that this would be foundational to patient experience of lifestyle services, core questions were added. A question on attrition was also added, given the evidence of high attrition in lifestyle programs (Moroshko et al., 2011).

Participants for cognitive interviewing were 11 adults (10 women, 1 man) who ranged in age from 32 to 78 with an average of 57 years. After two rounds of revision, the questions for pilot testing consisted of 23 questions: 21 multiple choice, with five that included space to add extra comments and two open-ended questions (Table 1 – Questions used for pilot testing).

Pilot testing

A total of 164 questionnaires were received (57 French, 107 English, and 0 Russian) of 293 potential recipients (56% response rate); 76% as paper questionnaires and 24% online responses. RCs used differing approaches; some did mail the questionnaires, while others completed the questions with participants as an interviewer-administered questionnaire. None of the Russian questionnaires were completed as the RC at this site translated and completed English questionnaires with participants. Question responses for closed-ended questions (Q1–Q21) ranged from 93% to 100%; responses to the two open-ended questions were 84% (Q22) and 68% (Q23). The <100% response rate for the first few questions may have been because participants missed these questions due to double-sided copying of the paper questionnaire. Comments reflecting a misunderstanding of clarity or meaning of questions were received for Q6, 7, 9, 11, 15 and 20 and revised wording is highlighted in Table 1. With respect to possible floor and ceiling effects, several questions had four or five ordinal response options, (such as Never to Very Often) where extreme values were rarely used, but only Q12 was changed at this point in development, as 100% of respondents felt they were always treated with courtesy and respect. Response options for this question were changed to two options: ‘always’/‘not always.’ Two questions were deleted or combined with another question as the responses and comments suggested redundancy (Q16 and 21). The revised questionnaire reflecting these comments consists of 20 multiple choice questions and one open-ended question which can be completed in about 5 min (Supplementary Material File). The actual results of the pilot test will be reported elsewhere along with results of a focus group study.

Discussion

This initial work to develop and pilot test a new self-administered patient experience questionnaire for lifestyle programs in team-based PC was successful in generating a set of questions based on the perceptions and insights of providers and patients as to the critical issues to be assessed for quality improvement purposes in PC. It is intended to supplement other data to assess the overall effectiveness of such programs. Lifestyle programs are increasingly being offered for diabetes prevention, cardiovascular disease prevention and/or weight management within PC. The work builds on the strong framework and key indicators already developed by CIHI over the past several years (CIHI, 2013). Thus, it is hoped these questions could be used in conjunction with other questions and methods from the framework for assessing PC services.

Review and prioritization by providers suggested some overlap in concepts from the providers’ perspectives, but there was general consensus that all concepts in the Wong and Haggerty framework were important. Their framework cataloged questions developed by previous groups on the patient experience of PC (Wong and Haggerty, 2013). As the questionnaire was intended from the beginning to be used in routine practice, issues of feasibility, relevance, and acceptability to providers and patients were paramount in initial development. A self-administered questionnaire would be most feasible, but it was clear from the outset that it would be impossible to address all aspects of the framework in such a questionnaire. After discussion, the same five dimensions/sub-dimensions emerged from all discussion groups. These were: trust, first-contact accessibility, and three under interpersonal communication – general communication, whole-person care, and respectfulness.

Accessibility and interpersonal communication have been enduring foci in previous questionnaires and our results confirm their importance based on the large number of questions included in these concepts. The pilot testing revealed ceiling effects for respectfulness, with 100% reporting they had been treated with respect. Trust stands out as it has been contentious in some studies (Brauer et al., 2012) and less commonly measured in older general PC questionnaires. Safran et al. (1998) were among the first to include trust as a dimension in the Primary Care Assessment Survey (Safran et al., 1998), related to trust in the physician (see Hall et al., 2002 for review of the older literature) (Hall et al., 2002). There is renewed interest in improved measurement of trust as it has been suggested that trust is associated with intervention effectiveness (Ozawa and Sripad, 2013). Given that trust is a multi-dimensional concept that has been mainly directed to trust in physicians, additional psychometric studies will be needed to develop a brief trust scale targeted to lifestyle programs offered by teams (Muller et al., 2014). Only two aspects of trust were included in the draft questionnaire; trust in the provider and trust in the information.

The discussion with providers revealed a nuanced understanding of the strengths and limitation of patients’ insights on their experience. For example, it was noted that patients may not be able to comment on the appropriateness of offered lifestyle services for their health and personal situation. This is particularly relevant in lifestyle services as programs can vary widely in their focus. For example, the area of obesity management remains highly controversial within healthcare and in the general community ranging from acceptance of ‘health at every size’ to a strong weight loss focus (Tylka et al., 2014). It is beyond the ability of the patient experience questionnaire to assess the appropriateness of lifestyle services offered, which is better assessed in other ways (eg, clinical outcome measures).

Patient experience questionnaires for lifestyle services are frequently used in PC practice, but rarely have undergone rigorous development and validation. A strength of our approach was to build on a framework for main dimensions assessed by previous researchers, while incorporating the priorities of the front-line providers involved in delivering services in PC. Established methods for initial assessment were used, including cognitive interviewing with participants from the target population and pilot testing in both English and French.

Due to feasibility issues, the providers who participated were all working in one model of team-based PC in southern Ontario and it is not known if priorities would differ in other models or locations. Pilot testing in three diverse PC settings across Canada as part of a study of lifestyle treatment of metabolic syndrome (CHANGE program) helped to address generalizability of the tool (Jeejeebhoy et al., 2017). Physicians as a group may or may not have somewhat different priorities than other providers with respect to the most important dimensions of lifestyle programs, although the dimensions of the Wong and Haggerty framework were identified from review of questionnaires directed mainly to physician practice (Wong and Haggerty, 2013). Lack of physician input in the initial discussion was partially mitigated in the pilot testing phase, as several family physicians on the research team reviewed the questionnaire and are co-authors on another paper of patient experience of the CHANGE program (Klein et al., 2017).

Some might also argue that patients should have been involved in the initial priority setting exercise. As the terminology for describing the dimensions would be unfamiliar to most patients, meaningful patient engagement on setting priorities was beyond the scope of the current project. A different and targeted methodology to obtain meaningful input would be needed, based on our experience with patient engagement strategies for new obesity services (Royall et al., 2017) and in practice guideline development (Buckland et al., 2017). Instead, we sought patient involvement in review of the actual questionnaire. Interestingly, no comments advocating for new concepts emerged in that process. As with physician views, somewhat different priorities may emerge in future studies comparing patient and provider priorities.

Pilot testing suggested that multi-modal availability (paper and online versions) is needed to ensure the majority of participants have an opportunity to comment on their experiences, especially among middle-aged and older groups. While a self-report questionnaire was planned, in some cases the RC completed the questionnaire in-person with participants at a clinic visit. This may have undermined assessment of the clarity of the questions and for missing/redundant concepts, as RCs could have explained the concepts. This could also have led to social desirability bias and greater floor/ceiling effects, as participants may have felt constrained in expressing any problems with the program. Thus, only one question was revised due to ceiling effects.

Conclusions/next steps

A set of questions were drafted and pilot tested on the patient experience of lifestyle services in team-based PC organizations. These questions are intended to be self-administered and to identify key aspects of programs that may require improvement. Local providers from one model of team-based PC prioritized the dimensions to be assessed. Accessibility and sub-dimensions of interpersonal communication were highly ranked and their inclusion was consistent with previous patient satisfaction surveys. Trust was also highly ranked, but has been less commonly assessed in previous questionnaires (Wong and Haggerty, 2013). Given the multi-dimensional nature of trust, further development of questions to assess trust within lifestyle programs is needed. Pilot testing in French and English suggested additional minor wording changes. Further development and validation of the questionnaire is needed, including assessment of test-retest reliability, item scaling and convergent and divergent validity. These questions provide a starting point for further studies of the patient experience of lifestyle programs.