Development and Pretesting of a Questionnaire to Assess Patient Experiences and Satisfaction with Medications (PESaM Questionnaire)

Abstract

Background

The aim of this study was to develop, together with the Lung Foundation Netherlands and Dutch Kidney Patients Association, patients and clinicians, a measure to evaluate patient experiences with the orphan drugs pirfenidone (for idiopathic pulmonary fibrosis [IPF]) and eculizumab (for atypical haemolytic uraemic syndrome [aHUS]), as well as a generic measure of patient experiences and satisfaction with medications.

Methods

Development of the Patient Experiences and Satisfaction with Medications (PESaM) questionnaire consisted of four phases: literature review (phase I); focus groups and individual patient interviews (phase II); item generation (phase III); and face and content validity testing (phase IV). Literature review aimed to identify existing disease-specific and generic patient experience measures to provide guidance on the domains of medication use relevant to patients, the number of items and type of response categories, and to generate an initial pool of items. Subsequent focus groups and patient interviews were conducted to gain insight into the perceived effectiveness of the therapies, the burden of side effects, and how the medication impacted on a patient’s daily life. Focus groups and interviews were recorded and transcribed verbatim. Coding was carried out by highlighting passages in the text and assigning each passage a code representing the following predefined categories: (1) perceived effectiveness; (2) side effects; (3) ease of use; and (4) impact of medication. Using data from phase I and II, a panel of experts selected items relevant for inclusion in the questionnaire. Individual patient interviews with IPF and aHUS patients (n = 18), using a retrospective verbal probing technique, were conducted to assess face validity, time needed to fill out the questionnaire, and content validity.

Results

The PESaM questionnaire that was developed consisted of two disease-specific modules that assessed patient experiences with pirfenidone for the treatment of IPF, and eculizumab for the treatment of aHUS, a generic module, applicable to any medication, and a module to assess patient expectations. Review of the literature identified multiple disease- or medication-specific questionnaires and two generic patient satisfaction questionnaires. Common domains across most questionnaires were effectiveness, side effects, ease of use and overall satisfaction. Patient interviews revealed the social impact (e.g. unable to go outside) of side effects such as photosensitivity associated with pirfenidone and the risk of infection associated with eculizumab. Each PESaM module focuses on patients’ perceived effectiveness of the medication, side effects, and ease of use, and the impact these aspects have on physical and emotional health and daily life. The generic module additionally includes items related to satisfaction with the medication. Individual interviews with patients in phase IV confirmed, in general, that questions and response options of the modules were clear and content validity was good. The mean time to complete the modules ranged from 6 min for the disease-specific (aHUS) module to 9 min for the generic module.

Conclusions

We developed the PESaM questionnaire to quantitatively assess patient experiences and satisfaction with medications. A validation study is currently underway to examine the psychometric properties of the PESaM questionnaire.

Electronic supplementary material

The online version of this article (doi:10.1007/s40271-017-0234-z) contains supplementary material, which is available to authorized users.

Key Points for Decision Makers

Data on patient experiences with novel drug therapies provide insight into how patients feel and function, and how the treatment impacts their lives.

A measure to quantitatively and systematically collect patient experiences is currently lacking.

The Patient Experiences and Satisfaction with Medications (PESaM) questionnaire was developed together with patients, clinicians and patient organisations.

The PESaM questionnaire may be employed to facilitate communication between patients and care providers, and guide treatment choices or to better incorporate the patient perspective in the reimbursement decision-making process.

Background

The importance of patient experiences when assessing the value of novel drug therapies and in optimising care is increasingly recognised [1–7]. Clinical outcomes and health-related quality-of-life measures alone may not be able to include all relevant benefits, harms and characteristics of a therapy to the patient. Patient experiences can refer to patients’ reports of structure and process aspects of care such as accessibility, information, waiting times and choice of provider [8–11]. These patient experiences are regularly used as health care quality indicators [12–15]. However, patient experiences can also refer to the way the patient evaluates health care outcomes, either in general or related to a specific treatment [11, 16–19]. These ‘subjective experiences’ incorporate an evaluation or value judgement regarding aspects and outcomes of health care provision and treatments. For example, if a specific drug therapy is administered through weekly intravenous injections in hospital, the term ‘patient experience’ may refer to the patient’s objective report of this fact (i.e. the actual administration mode and frequency) [20]. On the other hand, a subjective patient experience refers to how the patient is affected by this fact, e.g. in terms of inconvenience and perceived impact on daily life. Patients’ subjective experiences are unlikely to be fully represented in traditional clinical outcomes. Furthermore, health-related quality-of-life instruments usually miss out on process aspects of treatment considered important to patients, yet they can directly influence outcomes such as satisfaction, adherence to medication, and, ultimately, treatment effectiveness [16, 21–24].

Evaluation of subjective patient experiences with medications can be useful in several decision-making contexts. In a clinical setting, it can provide a more complete understanding of the impact of a therapy on the patient’s life and aid treatment choices [2, 7]. Quantitative and systematic assessment of patient experiences could also provide scientific evidence for guideline development that incorporates the patient’s perspective [6, 25]. Alternatively, assessment of patient experiences may broaden the perspective of formal assessments regarding the utility of new drugs aimed to inform reimbursement decision making [4, 25]. To date, the patients’ views and experiences with medications are mostly considered through active patient participation, i.e. consultation rounds with patients or patient representatives, patient memberships in committees, or, indirectly, through personal anecdotes from patients [25, 26]. Alternatively, the patient perspective can be considered by systematically measuring patients’ experiences with medications, providing a more scientific foundation for the incorporation of the patient perspective in decision making [25]. To our knowledge, a measure to evaluate such patient experiences is currently lacking.

In 2015, two patient organisations in The Netherlands—the Lung Foundation Netherlands and The Dutch Kidney Patients Association—together with patients, health scientists, government representatives and physicians, launched a project to develop a measure to better capture patient experiences with the new drug pirfenidone (for idiopathic pulmonary fibrosis [IPF]), and eculizumab (for atypical haemolytic uraemic syndrome [aHUS]). These orphan drugs had been granted conditional approval for reimbursement by the Dutch government, requiring a national registry with physiological outcomes as well as evaluation of patient-reported outcomes such as health-related quality-of-life and patient experiences as input for a re-evaluation after 4 years [27]. In addition, health care providers expressed a need to obtain more insight into the patients’ perspectives and experiences during disease course and treatment, to monitor and improve care delivery, and promote tailored use. In IPF and aHUS, there is a paucity of well-validated patient-reported outcome measures (PROMs) reflecting how patients feel and function while receiving treatment, and how treatment impacts their lives [7]. Hence, the aim of this study was to jointly develop a measure to evaluate patient experiences with pirfenidone (for IPF) and eculizumab (for aHUS), as well as a generic measure of patient experiences with medications. In this paper we describe the development and pretesting of the ‘Patient Experiences and Satisfaction with Medications’ (PESaM) questionnaire.

Conceptual Model and Context

Conceptual Model

Strasser’s holistic model for patient satisfaction distinguishes patient experiences, expectations and satisfaction, and was used as input for the conceptual framework for the PESaM questionnaire [18]. The conceptual framework is centred around the patient’s subjective experience of a stimulus (situation, event or outcome) related to medication use, for example a side effect (Fig. 1). The subjective experience is the result of the patient’s internal process in which he/she identifies, interprets, values and/or evaluates the stimulus. How the patient experiences the stimulus depends on his/her individual characteristics, such as sociodemographics, health status, beliefs, values and expectations. Satisfaction, in turn, is the attitudinal response to the patient’s subjective experience. This attitudinal response subsequently predicts a behavioural reaction (e.g. adherence) [18]. Expectations are central to the interpretation of satisfaction; a patient’s experience may be negative (e.g. ‘the side effects negatively impact on my ability to work’), but the patient may nevertheless report high levels of satisfaction with treatment if this negative experience met prior expectations [28].

Fig. 1.

Conceptual framework PESaM questionnaire (adapted from Strasser et al. [18]). Patient expectations regarding drug therapy and outcomes (block A) are evaluated in the expectations module of the PESaM questionnaire. The disease-specific modules assess stimuli (block B) and subjective experiences (block C) related to a specific therapy and patient group. The generic module focuses on subjective experiences (block C) and satisfaction (block D) with any drug therapy. PESaM Patient Experiences and Satisfaction with Medications

Since patient experiences, satisfaction, and expectations are interrelated, the measure that was developed to evaluate patient experiences encompasses all three concepts, i.e. the PESaM questionnaire focuses on patients’ subjective experiences regarding a medication’s effectiveness (perceived efficacy and impact), side effects (bothersomeness and impact) and ease of use (inconvenience and impact), expectations, and (dis)satisfaction.

Context

The PESaM questionnaire was developed and pretested in IPF and aHUS patients. IPF is a chronic and progressive lung disease that is characterised by irreversible loss of lung function [29]. Prognosis is poor, with an average survival of 3–5 years after diagnosis, and treatment options are limited [30]. At the start of the project, the antifibrotic agent pirfenidone was recommended for treatment. During the course of the project (in November 2015) a second antifibrotic agent, nintedanib, became available for IPF patients [31, 32]. Pirfenidone and nintedanib both reduce the rate of disease progression but are not equally effective in all patients. As a consequence, potential side effects of these drugs need to be balanced with the treatment effect [33, 34]. Evaluation of patient experiences can thus play a significant role when considering treatment approaches [35, 36]. In The Netherlands, it is estimated that between 800 and 1600 people are diagnosed with IPF each year [37]. Pirfenidone and nintedanib are currently conditionally reimbursed in The Netherlands for patients with IPF with a certain range of pulmonary function impairment (forced vital capacity [FVC] ≤80% predicted or a demonstrated decline in disease, and an FVC ≥50% predicted). As a result, while epidemiological data are still limited, clinical experts estimated that approximately 250 patients across the country were receiving pirfenidone or nintedanib in 2015. Atypical HUS is an extremely rare and life-threatening disease characterised by sudden abnormal breakdown of red blood cells, low platelet counts, and acute renal failure [38]. Atypical HUS can occur in children (60%) and adults (40%). People with aHUS are at constant risk of sudden and progressive damage to and failure of vital organs, particularly the kidneys. The prognosis for people with aHUS is poor, with approximately 2–10% of people with the disease dying in the initial, acute phase. In 50% of aHUS cases, end-stage kidney failure develops, requiring dialysis [38], and, later on, renal transplantation, with increased risks of developing aHUS (20–100%) in the renal transplant [38, 39]. National epidemiological data of aHUS are limited but it is estimated that only 15–20 people are diagnosed with aHUS each year, of whom three to five are children. With the implementation of the new drug eculizumab, outcome perspective improved significantly; however, the costs of the drug are tremendously high, i.e. up to €500,000 per patient per year for only the drug itself [40, 41].

Methods

The development of the PESaM questionnaire consisted of four phases: literature review (I); focus groups and individual interviews (II); item generation and questionnaire development (III); and face and content validity test (IV).

Literature Review

A literature review was conducted to (i) identify whether there were existing patient experience measures suitable for our patient population, and (ii) compile an overview of existing measures focused on experiences and satisfaction with medications. Existing measures were used to identify domains of medication use relevant to patients, provide guidance on the number of items and type of response categories, and to generate an initial series of items. A search of MEDLINE (from inception to 12 February 2015) was conducted using the following terms in the title or abstract: ‘measure’ or ‘questionnaire’ or ‘survey’ or ‘tool’ or ‘instrument’ AND ‘medication’ or ‘drugs’ or ‘pharmacotherapy’ AND ‘experience’ or ‘satisfaction’ or ‘beliefs’ or ‘views’ or ‘preferences’ AND ‘validity’ or ‘validation’ or ‘psychometric properties’ or ‘evaluation’ or ‘assessment’ or ‘reliability’. A paper was included if (i) it reported on a measure assessing patient experience or satisfaction with medication; (ii) psychometric properties were evaluated; and (iii) a copy of the measure (or item description) was available. Questionnaires that focused on medication adherence and the broader concept of treatment burden were excluded. Treatment burden takes into account everything patients do to take care of their (multiple) chronic illness, including issues outside the scope of the PESaM project, such as economic burden, medical tests, lifestyle changes, and impact of the illness on family [42].

Focus Groups and Individual Interviews

Focus groups and individual interviews were held to gain a deeper understanding and obtain information directly from patients about their experiences with eculizumab and pirfenidone. The interviews assisted to refine the content pool for the PESaM items. An interview guide containing a number of broad and open-ended questions, exploring three targeted domains (efficacy, side effects and ease of use), supported the collection of data (Online Appendices A and B). The questions were designed to encourage patients to describe their experiences using their own language. The following topics were discussed: patients’ perceptions regarding effectiveness of the medication, experienced side effects and their bothersomeness, ease of use, advantages and disadvantages of the medication, impact on everyday life, and overall satisfaction with the medication. Participants were probed for final thoughts, including what, if any, additional themes could be added that might impact satisfaction or dissatisfaction with the medication.

Focus groups took place between June and September 2015. Two medical specialists, based in two distinct hospitals in The Netherlands, were asked to invite outpatients with IPF and using pirfenidone to participate in a focus group. One medical specialist (MW) was part of the project team and the other (author RM) was approached by the team based on his expertise with IPF patients and experience with pirfenidone treatment. MW is based in the Erasmus Medical Centre, a university hospital in the city of Rotterdam and expert centre for interstitial lung diseases. RM is based in the Zuyderland Medical Centre, a peripheral hospital in the south of The Netherlands. Due to the small IPF patient population, convenience sampling was used aimed at including participants who reflected a range of the patient population in terms of age, sex and time on medication. The medical specialist initially informed eligible patients about the focus group and handed out the patient information sheet. If the patient agreed, contact details were provided to the researcher (MK or AR), who would then contact the patient to further explain the study purpose and procedures, and ask whether the patient was willing to participate. Written consent was collected before the start of the focus groups. The focus groups were held in the hospitals of the participating medical specialists. It was anticipated that a focus group would last between 1.5 and 2 h (including a short coffee break).

For aHUS patients, individual face-to-face interviews were conducted (either in hospital or at the patient’s home) since this disease is extremely rare and patients lived scattered around the country. One hour was scheduled for the individual interviews. Interviews were facilitated by one of two researchers (MK or AR) in the team with experience in conducting qualitative interviews and with no prior relationship with the participants. Field notes were made during the interviews and focus groups. In addition, all interviews were digitally recorded. Interviews were transcribed verbatim, with all identifiable information removed to protect the anonymity of the patients. The data were subjected to manual qualitative analysis by two data coders (MK and AR). Coding was carried out by highlighting passages in the text and assigning each passage a code representing the following predefined categories: (1) perceived effectiveness; (2) side effects; (3) ease of use; and (4) impact of medication. The categories were based on the topics from the interview guide. If a category was identified that did not fit one of the codes, it was assigned the code ‘other’. The passages in category 4 (impact of the medication) where then grouped into subcategories, again predetermined by the interview guide (i.e. physical, emotional, social) or newly emerging (i.e. ‘other’) [43].

Item Generation and Questionnaire Development

The PESaM questionnaire was developed by a panel of experts involved in all steps of the process. The panel was made up of two experts in health outcomes research, one health sciences researcher, four medical specialists, two policy advisors of patient organisations (of aHUS and IPF, respectively), and one policy advisor of the National Health Care Institute (ZiNL). The contents and themes derived from existing satisfaction measures, focus groups and individual interviews, and discussions with the panel of experts were used to define domains, potential categories within the domains, and a first pool of items relevant for inclusion in the PESaM questionnaire. This initial pool of items was discussed in a face-to-face meeting with the panel of experts and an independent board of advisors (four experts in the field of expensive medications, implementation of guidelines, health care decision making, and pharmaceutical care). During this meeting, the initial items, as well as different types of response options, the appropriate recall period, minimum age, and questionnaire instructions were discussed. Following this meeting a second, shorter version of the questionnaire was developed. Any deletion of items or adjustments were based on majority agreement. After several rounds of drafting, evaluation and revision among the panel of experts, the questionnaire was reviewed by a language consultant specialised in low literacy. She reviewed the readability to ensure broad applicability among patients. Last adjustments based on her advice led to the final version ready for the face and content validity test.

Face and Content Validity Test

Individual patient interviews (n = 18) were conducted to assess the face validity (comprehensibility of the instructions, questions and response options), the time needed to fill out the questionnaire, and the content validity (item relevance and missing domains or items). Interviews were conducted, if possible, with different patients than those used for the development of the questionnaire. However, new eligible aHUS patients, willing to participate, could not be identified between the individual interviews (June to September 2015) and the face validity tests (January 2016). Interviews employed a retrospective verbal probing technique. In this technique, a participant completes a paper and pencil version of the questionnaire. After completion of the questionnaire, the interviewer ‘probes’ further into the basis for the response [44]. Interviewer notes were complemented with the help of audio recordings.

The study protocol for the development and pretesting of the measure was reviewed and approved by the Medical Ethics Committee of Erasmus Medical Centre (MEC-2015-265). The study is registered in The Netherlands National Trial Register (code 5860).

Results

Identification of Existing Measures

The literature review led to 1238 references. Review of titles and abstracts revealed 18 relevant papers describing 2 generic and 14 medication-specific measures that fulfilled our inclusion criteria. Measures (either disease-specific or generic) that focused exclusively on subjective experiences with medications were not identified. An overview of measures and their domains is presented in Online Appendix C.

Fourteen disease-specific measures relating to a wide range of illnesses and conditions were identified that could serve as examples for our PESaM questionnaire: intraocular pressure [45, 46], osteoporosis [47], schizophrenia [48], diabetes [49], migraine [50], asthma [51, 52], osteoarthritis [53], pain [54, 55], Crohn’s disease [56], overactive bladder [57] and cancer [58]. Some measures focused on a specific type of treatment (e.g. drops, inhaled asthma treatment), while others focused on ‘your medication’ or ‘your treatment’ more generally. All measures focused on the concept of satisfaction. Most measures had multiple items in one or more of the following domains: effectiveness, side effects, ease of use, impact on everyday life, quality of life, functional benefit, and overall perception/global satisfaction. The number of items in the measures ranged from 1 to 32, with most measures (64%) having between 15 and 25 items. Most item responses were recorded on a 5-, 6- or 7-point rating Likert-type scale (very dissatisfied to very satisfied, disagree to agree, always to never, etc.).

The ‘Treatment Satisfaction Questionnaire for Medication’ (TSQM) [59, 60] and the ‘Treatment Satisfaction with Medicines Questionnaire’ (SATMED-Q) [61] were two valid multidimensional questionnaires measuring satisfaction with treatment with medicines. The TSQM consists of 14 questions and provides scores on four domains: side effects, effectiveness, convenience and global satisfaction [59]. Items were scaled using either a 5- or 7-point scale. Five-point scales were used for unidimensional continua (e.g. ‘extremely’ to ‘not at all’), while 7-point scales were used for bipolar continua (e.g. ‘extremely positive’ to ‘extremely negative’). The SATMED-Q consists of 17 questions and has six domains: treatment effectiveness, convenience of use, impact on daily living/activities, medical care, undesirable side effects, and global satisfaction. Response items were scaled using a 5-point scale ranging from ‘not at all’ to ‘very much’ [61].

Focus Groups and Individual Interviews

Two focus groups with six and seven IPF patients using pirfenidone, respectively, were conducted. As planned, the focus groups lasted between 1.5 and 2 h. The mean age of IPF patients was 69 years (range 55–77 years) and most patients (85%) were male. The mean time on treatment was 15 months (range 2–36).

Four face-to-face individual interviews with aHUS patients using eculizumab were conducted. Patients were all female and mean age was 37 years (range 24–46 years). The mean time since diagnosis of aHUS was 3 years. All patients were currently using eculizumab and had been using the medication on and off for the past 6 months to 3 years. Interviews lasted between 45 and 60 min. Ideally, one or two extra interviews with aHUS patients receiving eculizumab were conducted, but due to the extreme rarity of the disease, no other patients were eligible and willing to participate.

Idiopathic Pulmonary Fibrosis

Table 1 provides an overview of issues related to pirfenidone use that were extracted from the focus groups. Most, but not all, respondents had the feeling that the medicine stabilised their (decrease in) health and lung function. Furthermore, these respondents experienced a decrease in coughing, fatigue, and dyspnoea. As a result, they felt that the medicine had a positive effect on daily life and would increase life expectancy.

Table 1.

Summary of responses extracted from the idiopathic pulmonary fibrosis focus groups (n = 13)

Category (domain)	Responses
Effectiveness	Stable health
	Stable long capacity
	Stable disease progression
	Longer life expectancy
	Reduced coughing
Side effects	Coughing
	Dry mouth
	Loss of appetite
	Nausea
	Diarrhoea
	Photosensitivity (skin burn)
	Sweating
	Dry skin
	Rash
	Muscle pain
	Tiredness
	Low energy levels
	Sleepiness
Ease of use	Inconvenient packaging
	Restricted alcohol use
	Need to plan meals
	Frequent intake
Impact on everyday life	Physical
	Prevent lung transplantation
	Feeling better
	Being able to do things
	Emotional
	More positive outlook
	Social
	Not able to go on holiday and enjoy the sun (due to photosensitivity)
	Not able to go outside on a sunny day (due to photosensitivity)
	Other
	Continuity of life through stable health
Other	Exhaustion (symptom of the disease)
	Shortness of breath when being active (symptom of the disease)
	Difficult to be active (symptom of the disease)
	Limited oxygen leading to headache (symptom of the disease)
	Good service from the pharmacist [home delivery] (process aspect of care)

“I used to cough 24 hours a day, it makes you exhausted. I am very satisfied because I don’t cough anymore, it must be the pirfenidone” (male, 71 years).

Respondents expressed they found it difficult to discern the effects of pirfenidone from the effect of other medications and treatments they were receiving (e.g. physical rehabilitation or medications to treat side effects and comorbidities). Some respondents who had recently started the medication did not yet feel the medication helped them and they did experience side effects. Often, together with their doctors, they decided to (temporarily) reduce the daily doses of the medication. Side effects that respondents experienced were coughing, dry mouth, appetite loss, nausea, diarrhoea, stomach complaints, photosensitivity, sweating, rash, fatigue, sleepiness and muscular pain. They felt that side effects negatively impacted their quality of life. For example, three patients expressed that photosensitivity (i.e. sunburn) was so severe that they could not go outside on a clear day or felt uncomfortable taking a break to a sunny destination.

“The only thing I worry about is the sun. Because you have to use factor 35 or 50 and that’s awful cause I got family in Portugal but I just don’t want to go on a holiday, I mean what’s the fun of sitting in the shade and not even being able to walk down the street without a hat and gloves” (male, 65 years).

Nevertheless, participants with advanced stages of IPF expressed they feel they ‘have no choice’ due to the severity of their disease and limited treatment options, therefore side effects were taken for granted. Regarding ease of use, respondents mainly complained about the packaging of the medicines and the frequency of taking the medicines (two to three times a day). Furthermore, the need to take the medicines together with food also required strict planning when going out for (part of) the day.

Atypical Haemolytic Uremic Syndrome

Table 2 provides an overview of issues related to eculizumab use that were extracted from the interviews. Respondents experienced, as an effect of eculizumab, that the signs of disease diminished and health improved. They expressed they felt better, less sick and less tired. Patients experienced that they had more energy to participate in society, enjoy social activities, and take care of their family.

Table 2.

Summary of responses extracted from the atypical haemolytic uraemic syndrome interviews (n = 4)

Category	Responses
Effectiveness	Recovery of the body
	Stable health
	Staying alive
	Prevent recurrence of disease
	More energy
Side effects	Hair loss
	Bruising
	Vomiting
	Nausea
	Tremor
	Muscle pain
	Pain in the legs
	Fatigue
	Blurry vision/vision impairment
	Lack of energy
	Moody
	Risk of meningitis
	Risk of infections
Ease of use	Frequency of hospital visits
	Intravenous insertion
	Administration not possible at home
Impact on everyday life	Physical
	Better fitness levels
	Feeling better/less sick
	Return to ‘old’ life before illness
	Emotional
	Life changing/avoiding death
	Unknown long-term harms of treatment (worry)
	Feeling protected against the disease (reassurance)
	Social
	Participate in family life
	Participation in society
	Avoiding busy public spaces
	Active social life
	Preventing dialysis or kidney transplant
	Other
	Need for immediate access to antibiotics
	Avoiding treatments with increased risk of infection
	Change diet (avoid certain foods)

“I don’t have enough energy to work full-time, but I can care for my young son again and do some voluntary work” (female, 37 years).

Side effects that respondents experienced were nausea, vomiting, bruises, hair loss, joint pain, tremor, pain in the legs, fatigue, a moody feeling and loss of sight. Fatigue was most debilitating in the first days after receiving the therapy. Respondents did find it difficult to separate side effects from actual symptoms of aHUS. Furthermore, patients worried about possible long-term sequelae and their increased risk of infection. Because of their fear of infections, some respondents avoided busy public places and food they felt would impose a higher risk of infections.

“I avoid airplanes and air-conditioning, I even don’t kiss (greet) my best friend anymore… . These infections spread via air, personal contact” (female, 46 years).

Regarding ease of use, some respondents were bothered by the frequency of administration, administration in hospital, and the intravenous administration. However, they much preferred receiving eculizumab than the dialysis they had received or would otherwise need. While side effects and administration mode were important to respondents, they were extremely grateful and satisfied with the therapy as they felt they could not have survived or participate in everyday life without the medication.

“Without eculizumab I would be on dialysis 3 times a week, with all associated consequences. I wouldn’t be able to participate in society, and I am so young” (female, 24 years).

Item Generation and Questionnaire Development

Existing measures identified in the literature review, discussion with experts, and the focus groups and individual interviews with patients resulted in the PESaM questionnaire, which consisted of four modules: two disease-specific modules for the treatment of IPF (10 items) and aHUS (12 items), respectively; a generic module (16 items), applicable to any medication; and a patient expectations module (11 items). The modules can be used in combination or separately. Online Appendix D provides a summary table of the content of the four modules. All modules start with a short instruction. Patients are instructed what drug, and for which disease/diagnosis, the items in the module refer to. It is emphasized that the items are about their personal experiences with the drugs, and thus there are no right or wrong answers, and that they should tick the response option that best describes their personal perception or feeling. The recall period for the items in the disease-specific and generic modules is ‘in the past 4 weeks’.

In general, all modules follow a similar structure and focus on patients’ experiences (or expectations in the expectations module) related to three domains: effectiveness, side effects, and ease of use of the medication. Items within these domains relate to the perceived impact of the medication on aspects of physical health, emotional health, and everyday life (social and work). A 5-point (Likert-type) scale with the following anchor levels was chosen as the response format for most items: ‘not at all’, ‘a little’, ‘reasonable’, ‘a lot’ and ‘very much’. It was advised by the language consultant to describe the response categories in greater detail to facilitate comprehension. For example, when an item asks to what extent a respondent was bothered by side effects of the medication, response categories were ‘not at all bothered’, ‘a little bothered’, etc., rather than only ‘not at all’ and ‘a little’. We also assumed that respondents experienced a positive influence of the effectiveness of a medication (if any) and a negative influence of side effects (if any).

The disease-specific modules for aHUS and IPF were developed in close collaboration with clinicians and with input from patients through the focus groups and individual interviews. The modules evaluate patients’ experiences of the medication regarding specific disease symptoms and provide a checklist of potentially experienced side effects (see Online Appendix D). Identified issues (e.g. side effects) from the focus groups and interviews were generally operationalised as items in the domains of the disease-specific modules. For example, the module for aHUS asks about the influence of eculizumab on energy levels and the ability to participate in society, fear of infection (meningitis), and disease recurrence. For each experienced side effect included in the checklist, such as tremor, fatigue and muscle pain, respondents are asked to rate how bothered they are by that side effect. A category ‘other side effect’ was available in case a patient experienced a side effect not present in the checklist. The module for IPF focuses on its perceived ability to slow down disease progression (rather than cure), feeling tired and out of breath, and whether respondents experienced side effects such as photosensitivity and diarrhoea (see Online Appendix D). The items in ease of use focus on the potential inconvenience of mode of administration and whether patients have skipped medication.

The focus groups and individual interviews revealed that patients have difficulties distinguishing symptoms of the disease with side effects, or effects of one medication from the effects of another, and thus a ‘don’t know’ response category was available for items, if applicable.

Besides two disease-specific modules, the PESaM questionnaire includes two generic modules that can be used for any medication: one that focuses on patient experiences, and one that focuses on patient expectations. The generic module for patient experiences focuses on perceived effectiveness, side effects and ease of use, and their impact on physical health, emotional health and everyday life, and also includes an item regarding satisfaction for each domain, one item about overall (dis)satisfaction with the medication, and a final item to assess the relative importance of the three domains (Fig. 2). Satisfaction items are scored on a numeric rating scale ranging from −5 (not at all satisfied) to +5 (very satisfied). The generic modules, in contrast to the disease-specific modules, do not include any disease- or medication-specific items, such as specific symptoms that are targeted by the therapy or specific side effects experienced.

Fig. 2.

Items of the generic experiences module of the PESaM questionnaire. PESaM Patient Experiences and Satisfaction with Medication

The structure and items of the module focused on expectations are similar to the items of the generic experiences module; however, questions relate to expectations rather than experiences (see Appendix D). For example, where item 1 of the generic module is “How effective has the medication been over the past four weeks?”, the item in the expectations module is “How effective do you expect the medication to be?”. The expectations module is meant for administration before the start of a new drug and can serve as a baseline measurement.

Face and Content Validity

Thirteen respondents completed the generic and disease-specific modules, in two rounds of interviews. Four respondents had aHUS and were currently using eculizumab, five respondents had IPF and were currently using pirfenidone, and four respondents had IPF and were currently using nintedanib. The expectations module was completed by five respondents (all IPF patients; three commencing treatment with pirfenidone and two commencing treatment with nintedanib) who either started their medication up to 3 days before or were due to start in the next few weeks.

The first round of interviews (n = 6) identified some problems that required adaptation of the modules. Two respondents overlooked ‘positive influence’ in the first items of the generic modules and therefore this expression was emphasized using bold text. Furthermore, the instruction to skip remaining items in the domain when a respondent did not experience any side effects (in the generic experiences module) was often overlooked. This was also emphasized using bold text. The overall satisfaction question was shortened as respondents had difficulty comprehending the question because of its length. Finally, regarding the item about skipping medication because of side effects or ease of use, respondents missed an answer category to indicate they skipped the medication for reasons other than side effects or inconvenience, such as simply forgetting or having the flu. Hence, the answer categories were adapted to reflect this potential response.

The second round of interviews (n = 12) tested the adapted modules. Overall, all items in these versions were well understood by patients and were reported to be relevant. One respondent felt the questions relating to side effects and ease of use were irrelevant as the efficacy of the medication was most important since this resulted in her still being alive. On the other hand, another respondent felt that negative aspects (side effects) should receive more attention. The majority of respondents felt the questions appropriately covered most relevant aspects of their experiences with, or expectations of, the medications. Missing items or side effects were not reported by respondents. Hence, face and content validity were considered good and no more changes were needed.

On average, the generic module was completed in 9 min (range 5–17 min), the disease-specific module for IPF was also completed in 9 min (range 5–16 min), the module for aHUS was completed in 6 min (range 5–8 min), and the expectations module was completed in 6 min (range 5–8 min). The instructions were clear to all respondents and there were no missing responses.

Discussion

The aim of this study was to develop, together with clinicians, patients and patient representatives, a measure to evaluate patient experiences with the orphan drugs pirfenidone (for IPF) and eculizumab (for aHUS), as well as a generic measure of patient experiences and satisfaction with medications. The PESaM questionnaire, consisting of four modules, which evaluates patient experiences, satisfaction and expectations regarding the use of medications, was developed. Specific attention was given to conceptual basis, universality, self-administration, number of items, response options and accessibility of the questionnaire [62, 63]. The generic modules are applicable to multiple therapies (medications). The layout, language and uniform response options promote accessibility for respondents with low (health) literacy levels. Face validity tests with 18 patients confirmed that questions and response options of the different PESaM modules were clear and content validity was good. While the disease-specific module was originally developed for use in IPF patients using pirfenidone, it is also suitable for the new antifibrotic agent nintedanib. Treatment efficacy and side effects are comparable between the two therapies, although the prevalence and severity of the side effects differ. Diarrhoea is the most commonly reported side effect of nintedanib [33].

The generic module includes questions on (dis)satisfaction with each domain, and an overall satisfaction item. This is in line with Strasser’s suggestion that patient satisfaction is both a multidimensional construct and a summary construct (a unique summary judgement) since patients form both types of judgements [18]. However, due to the addition of satisfaction items for each domain, the generic module has some overlap with other generic patient satisfaction measures such as the TSQM and SATMED-Q, which also focus on (among other domains) effectiveness, side effects and ease of use of medications [59, 61]. Nevertheless, the generic module distinguishes itself from these measures by evaluating experiences and satisfaction separately; items on satisfaction are distinct from subjective experiences and their impact on health. Hence, experiences may be negative (e.g. a respondent reports that the side effects of the medication negatively impacted on daily life), but patients may still be satisfied; a strong relationship between experiences and satisfaction is not a prerequisite for the generic module of the PESaM to be valid. Especially with medications for rare (and often severe) diseases, the presence of side effects and inconvenience of administration may not lead to overall dissatisfaction. Therefore, rather than serving as a satisfaction measure alone, the generic module evaluates both experiences and satisfaction.

A recently published opinion piece by IPF experts illustrates the potential value of the PESaM questionnaire [7]. In IPF, there is a paucity of specific well-validated patient-reported outcome and experience measures, while it is increasingly acknowledged that understanding what matters to patients is essential for patient-centred care and research. PROMs, like the PESaM, focusing on the impact of therapy on a patient’s well-being and daily life, have the potential to facilitate communication between patient and care provider [64], detect unrecognised problems and guide treatment choice [7, 65, 66]. The disease-specific module provides detailed insight into the impact of the antifibrotic agents on specific disease symptoms such as cough and fatigue, as well as how bothered patients are with experiencing side effects. The generic module of the PESaM questionnaire could be included as secondary or exploratory endpoints in trials or registries. Systematic collection of patient experiences with pirfenidone and nintedanib, for example, enables better understanding of variable treatment response and identification of subgroups of patients that might benefit from a certain treatment [2]. What is more, with new (orphan) drugs continuously being developed and entering the market, regulatory bodies will require proof of value for money. In The Netherlands, the ZiNL has the responsibility of advising the Minister of Health, Welfare and Sport about what should be included in the basic health care package. The advisory process of the ZiNL consists of four phases: scoping, assessment, appraisal and final advice formalising. In the assessment phase, all relevant information regarding the four criteria of necessity, effectiveness, cost effectiveness and feasibility are collected. An appraisal phase is included if the evidence in the assessment phase was not convincing enough or if important societal implications are expected. A committee assigned by the Minister then reassesses the information from the different package criteria, giving special consideration to the societal implications. In both phases, external parties, including patient organisations and clinicians, are asked for additional evidence or their opinion. Since assessment of value for money for orphan drugs can be challenged by the rarity of the disease compromising the quality of orphan drug evidence, as well as unfavourable cost-effectiveness ratios due to their high costs [67], there is opportunity for the patient perspective, and thus patient experience measures, to play a more significant role when assessing and appraising the value of orphan drugs. However, to date, the patients’ views and experiences with medications are often expressed through personal anecdotes or opinions [25]. Data collected by the PESaM questionnaire provides much more systematic, quantitative and robust evidence on how these (orphan) drugs impact on the lives of patients. The generic module of the PESaM specifically focuses on the impact of the medication on physical, emotional and social health, and allows for comparison with other drugs.

This study has a number of limitations. First, focus groups and individual interviews (phase II), as well as the face validity test (phase III), were conducted in a relatively small population (n = 13, n = 4 and n = 18, respectively). In addition, aHUS patients who participated in the face-validity testing were the same patients who were interviewed in phase II. However, aHUS and IPF are rare diseases and not all patients are eligible for the medications. Hence, with a set time frame to develop the questionnaire, a larger pool of eligible patients could not be identified. Nonetheless, for its next step, psychometric validation of the questionnaire, the PESaM questionnaire is currently sent out to IPF patients in The Netherlands who (start to) use pirfenidone or nintedanib in one of ten participating hospitals across the country. Hence, a much larger study population is expected for the psychometric validation. Similarly, in order to receive eculizumab, aHUS patients are required to take part in a national monitoring study in which patient experiences and satisfaction, as assessed by the PESaM questionnaire, are secondary outcome measures. Finally, to promote generalisability of the generic module, psychometric validation is also conducted in a third patient group not involved in the development of the questionnaire. The generic experiences module of the PESaM is completed (three times, including a test–retest) by adult patients who have had a kidney or liver transplant and are using the drug advagraf to prevent rejection. Second, patients involved in the development and pretesting of the questionnaire were all still receiving the therapy. This could have introduced bias towards the generally more satisfied patients, generating a risk that an important issue or domain causing patients to discontinue the medication was overlooked. Nonetheless, several participants of the focus groups reported severe side effects, which had forced them to temporarily discontinue therapy or reduce the daily dosage. Moreover, the face validity interviews included respondents who had stopped using pirfenidone and had switched to nintedanib. They did not report missing any side effects or other important issues related to their medication use in the PESaM questionnaire. Third, the decision to develop an expectations module was made after conclusion of phase II of the project, and did therefore not undergo a rigorous development process. Nevertheless, the module was well received by respondents in the face validity test. It is expected that responses on the PESaM expectations module can assist in the psychometric testing of the generic module: to what extent are experiences and satisfaction related to prior expectations? Fourth, the PESaM questionnaire was originally developed for use in adolescent and adult populations; however, the youngest participant in the development and pretesting phase was 24 years of age. Hence, it is unknown whether the PESaM is suitable for use in an adolescent population. The age range among participants of the validity phase will most likely be wider, potentially providing evidence for its use in this population. Still, the majority of participants in the validity study are IPF patients who are generally older than 20 years of age [37]. Finally, at this point, we are unable to define cut-off scores for the PESaM, i.e. a score of x means that a patient has had a positive experience and is satisfied. Further research is planned to quantitatively evaluate the psychometric properties and interpretability of the scores, as well as to assess the impact of the PESaM on clinical and reimbursement decision making. Once the psychometric properties are established, the questionnaire will be translated into English.

Conclusions

The PESaM questionnaire was developed to quantitatively and systematically evaluate patient experiences and satisfaction with medications. Data collected using the PESaM questionnaire aim to better promote understanding of the impact of a therapy on a patient’s daily life. It can potentially serve as a tool to assist shared decision making in clinical practice, as well as provide more scientific evidence towards the patient’s perspective in reimbursement decision making.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with Ethical Standards

Conflict of interest

Merel Kimman, Adrienne Rotteveel, Marlies Wijsenbeek, Rémy Mostard, Nelleke Tak, Xana van Jaarsveld, Marjolein Storm, Kioa Wijnsma, Marielle Gelens, Nicole van de Kar, Jack Wetzels and Carmen Dirksen have no conflicts of interest, including nonfinancial, that are directly relevant to the content of this article.

Funding

This project was funded by The Federation of Patients and Consumer Organisations in The Netherlands (NPCF), Lung Foundation Netherlands, and The Netherlands Organisation for Health Research and Development (ZonMw). We thank Ms Jeanine van der Giessen (M.Sc.), health literacy specialist, University Medical Center Utrecht, for reviewing the questionnaires. We also gratefully acknowledge the support of Ms. Mirjam van Manen (M.Sc.), Ph.D. candidate at the Department of Respiratory Medicine of the Erasmus Medical Center in Rotterdam.