Measuring treatment burden related to general practice in patients with multimorbidity: development and validation of a PROM

Abstract

Introduction

This study aimed to either identify or develop and validate a patient-reported outcome measure (PROM) to assess treatment burden related to general practice for patients with multimorbidity, which can be used alongside the MultiMorbidity Questionnaire part 1 (MMQ1) without overwhelming the target population with redundant items.

Methods

We conducted a systematic literature review to identify all existing PROMs measuring treatment burden. If no suitable PROM was found, our plan was to: (1) develop a draft PROM using items from existing instruments, (2) carry out group and individual interviews with patients with multimorbidity to ensure the PROM’s understandability, clarity, completeness and relevance and (3) undertake psychometric validation with a diverse sample of primary care patients with chronic conditions.

Results

We did not identify an eligible PROM in the literature review. The draft PROM consisted of 30 items divided into six domains; Information about treatment, Challenges with medication, Medical appointments, Self-monitoring, Health behaviour and Challenges in the contact to the health system. In the psychometric validation, neither these domains nor any other theoretical constellation of items had adequate psychometric properties. Individual items had good criterion validity and sensitivity to change.

Conclusions

In this study, we developed a 30-item PROM with high content validity where various individual items showed adequate criterion validity and sensitivity to change, making these items useful as a supplemental measure to the MMQ1.

Trial registration number

NCT05676541 Registration Date: 16 December 2022.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Treatment burden is a pivotal concept to measure along with health-related quality of life in patients with chronic disease and multimorbidity in general practice.

• Patient-reported outcome measure (PROMs) for both concepts exist, but these are often not designed to supplement each other without overwhelming the target population with redundant questions.

WHAT THIS STUDY ADDS

• We developed and validated a new PROM to assess treatment burden related to general practice to supplement the MultiMorbidity Questionnaire part 1 (MMQ1).

• The PROM had high content validity, but the hypothesised scales in the included domains had insufficient fit to the confirmatory factor analysis model and internal consistency.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• Some of the individual items showed adequate criterion validity and sensitivity to changes, making these items useful as a supplemental measure to the MMQ1 and for further exploration of the concept of treatment burden related to general practice.

Background

Patient-experienced treatment burden is often reported as a negative result of living with a chronic disease and multimorbidity and is reported as a core outcome in studies involving patients with multimorbidity.¹,³

In the development of a randomised clinical trial concerning patients with complex multimorbidity managed in general practice, we aimed at measuring need-based quality of life using the part 1 of the MultiMorbidity Questionnaire part 1 (MMQ1) plus other aspects of treatment burden related to general practice not already covered by the MMQ1.⁴ The MMQ1 is a patient-reported outcome measure (PROM) developed and validated to measure the need-based quality of life in patients with multimorbidity.^{5 6} Although not designed for measuring treatment burden, it includes several domains often reported in qualitative studies and PROMs concerning treatment burden, for example, the impact of multimorbidity on the economy and social life.⁷,⁹

The Patient Experience with Treatment and Self-management (PETS) instrument has been identified in a systematic review to have higher validity than most PROMs with the aim of measuring treatment burden.^{10 11} However, we found that the MMQ1 and PETS instrument encompassed several content-wise redundant questions, which could give the target population an unnecessary large response burden.^{6 11} In addition, a cross-cultural validation of PETS into Norwegian also identified several measurement problems in a Scandinavian context.¹² Although the PETS instrument may not be completely compatible with the MMQ1 instrument, items from the PETS instrument as well as other validated measures, could serve as building blocks in the construction of a new, compatible PROM.

Thus, the primary aim of this study was to identify one or more PROMs that could measure treatment burden in patients with multimorbidity, based on the framework from the PETS instrument,^{8 9} and with adequate measurement properties for unaltered use in addition to the MMQ1 without resulting in redundant questions. If no such PROM was found, we proceeded to our secondary aim, which was to develop and validate a new PROM specifically designed to measure treatment burden in patients with multimorbidity, which could be applied in addition to the MMQ1 instrument.

Methods

The validation process involved five steps inspired by the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN checklist).¹³ (1) First, we selected a framework to define treatment burden related to general practice in patients with multimorbidity. (2) Next, a literature search was conducted to identify PROMs that assess treatment burden in patients with multimorbidity, followed by a quality assessment of these PROMs. The primary goal of this search was to find one or more PROMs suitable for use without modification. (3) If no suitable PROM was found, we planned to proceed to steps 3 to 5. Step 3 involved extracting relevant domains or items from the identified PROMs, provided they measured aspects of treatment burden in patients with multimorbidity as defined by the selected framework. The item extraction process serves the purpose of building on previous work while leaving room for larger alterations. After the extraction of items, we constructed a draft PROM with high face validity. (4) In step 4, this drafted PROM was refined through group and individual interviews with the target population to ensure high content validity. (5) Finally, once the PROM achieved high content validity, we advanced to step 5: the psychometric validation. The development and validation steps for the PROM are depicted in figure 1.

Figure 1. The steps in the development and validation of the PROM. PROM, patient-reported outcome measure.

Step 1: choice of framework

To decide on a framework, we searched the literature, scrutinised commonly used PROMs used to measure treatment burden in patients with multimorbidity and their references^{11 14} and consulted experts within our network. We preferred to use a framework which (1) had a strong theoretical and/or conceptual foundation, (2) was further refined in qualitative interviews with the target populations(s) and (3) had proven consistency across cultures and health systems. We could not find a framework fully living up to these standards. We decided to use the framework from the PETS instrument to explain the concept of treatment burden.^{8 9} This framework was developed in the USA and based on qualitative interviews among 32 patients with chronic illness and a complex self-care regimen and was later refined in additional interviews with 18 patients from a primary care clinic serving a population without insurance. The framework consisted of three domains: (1) the work patients must do to care for their health, (2) the challenges and stressors that exacerbate the perceived burden and (3) the impacts of the perceived burden. The first domain relates to the different tasks patients engage in to care for their health such as taking medications and keeping medical appointments. The second domain includes the obstacles the patients may face, trying to carry out these tasks, for example, financial or social problems. The third domain is the impact of the two first domains on the person’s well-being such as exhaustion and limitations in their daily life. Later in the PETS development process, the domains were subdivided into 15 subdomains.⁹ The framework was mainly constructed based on the data from qualitative interviews and lacked a strong theoretical foundation. The framework had been translated and culturally adapted in the European setting, although not in Denmark.¹² However, the use of interviews to inform the domains was still a stronger foundation to build on than other available frameworks. Although the PETS instrument was developed in primary care, it aims to measure aspects of treatment burden in both primary and secondary care,^{8 9} which would likely require some adaptation to suit our purpose.

Step 2: literature search

Search strategy and inclusion of studies

We included PROM development and validation studies conducted in primary care or a comparable setting. The PROMs should measure the concept of treatment burden in a population of adult patients with multimorbidity.

We searched PubMed using the search string: (patient-reported outcome measure[TiAb] OR PROM[TiAb] OR instrument[TiAb] OR validation[TiAb]) AND (multimorbid* OR (chronic AND (illness OR disease))) AND (treatment burden). We included studies published in Danish, Swedish, Norwegian or English before 1 October 2022. Titles and abstracts were screened, and full-text articles were retrieved on potential relevant manuscripts. Full-text articles were reviewed for eligibility.

Eligibility criteria for quality assessment and potential item extraction

An identified PROM could be included for risk of bias assessment and item extraction if they: (1) measured the concept of treatment burden in primary care setting and (2) was validated among patients with at least two unselected chronic conditions.

Quality assessment

The COSMIN Risk of Bias checklist box 1–4 (PROM development, content validity, structural validity and internal consistency) was used for risk of bias assessment.¹⁵ The risk of bias was assessed by two authors (AH and SFB). In case of disagreement, a third author (KHB) was consulted. If the quality of the PROM had already been assessed in a previous systematic review performed by our unit,¹⁰ we did not repeat the quality assessment.

Eligibility criteria for use of unaltered PROM

To be eligible for the use of the full PROM, the PROM had to fulfil the following additional criteria: (1) have at least adequate quality in domains 1–4 in the COSMIN checklist and (2) be compatible with the MMQ1 instrument without any changes.

Step 3: content validation; item extraction, construction of a draft PROM and face validity

The content validation process involved two main steps: (1) extracting items and developing a draft PROM and (2) conducting focus group and individual interviews with patients with multimorbidity. If no PROM met both the item extraction criteria and achieved at least a ‘good’ rating per the COSMIN checklist (boxes 1–4) for direct use in the study, we planned to extract items from PROMs that met item extraction criteria and aligned with dimensions of the selected framework, then use these to draft a new PROM.

We manually extracted all items from all eligible PROMs and grouped these items into the three domains (1) the work patients must do to care for their health, (2) the challenges and stressors that exacerbate perceived burden and (3) the impacts of the perceived burden and their corresponding 15 subdomains as depicted in table 1. Items were extracted and organised into the 15 domains.

Table 1. Number of items in each domain at each step in the content validation process.

Name in new PROM/original name of domain(s)	Items extracted	Items after the removal of redundant items and domains	Items after the expert panel for the first group interview	Items after the first focus group interview	Items after the second focus group interview	Items after the third focus group interview	Items after 12 individual interviews
Information about treatment/ 1a Learn about conditions and care+2d Confusion about medical information:	9	8	4	4	4	4	4
Challenges with medication/ 1b Medications 2a Challenges with taking medication:	13	13	10	11	11	11	11
Medical appointments/1c Medical appointments:	9	9	2	2	2	2	2
Self-monitoring/ 1d Monitoring health status	4	4	4	4	3	3	3
Health behaviour/ 1e Health behaviours	5	5	4	4	4	8	8
1f Medical equipment/devices	0	0
2b Interpersonal challenges	10	0
2 c Financial challenges	6	0
2e Barriers to self-care	5	0
Challenges in the contact to the health system/ 2f Healthcare provider obstacles—individual provider+2g Healthcare provider obstacles—system issues	9	9	2	2	2	2	2
3a Role and social activity limitations	11	0
3b Physical and mental exhaustion of self-care	10	0
Total	91	48	24	27	26	30	30

PROM, patient-reported outcome measure.

Four of the authors (AH, ABL, and JBB with input from KHB) ensured high face validity and modified the PROM after each group interview and after the final individual interview by adjusting items and domains based on clinical and research experience. JBB and AH are both clinically active general practitioners and JBB, AH and KHB have experience with PROM development. All five health regions in Denmark were represented in the author group, and each was consulted to clarify regional perspectives when needed.

Step 4: content validation; group and individual interviews

Focus group interviews

The three focus group interviews in different regions of Denmark were conducted with patients with multimorbidity to assess the drafted PROM’s content relevance, coverage, understandability and functionality. Focus groups allowed for open discussion on the PROM’s content and layout as well as the generation of new items. Patients were recruited via their GP through purposive sampling, with five patients per focus group to balance group dynamics and allow for sharing personal experiences. The GPs were asked to recruit patients with at least two chronic conditions who, according to the GP’s judgement, experienced problems in their daily lives due to their multimorbidity. The GPs were contacted through the authors’ network. Sessions lasted around 2 hours and followed a semistructured format. We did not collect data on individual participants’ characteristics.

The PROM was to be developed and validated for use in a trial evaluating extended consultations for patients with multimorbidity, defined in this context as experiencing significant daily life challenges due to multiple chronic conditions (25). To ensure high content validity and identify patients likely to encounter care fragmentation and limited patient centredness, we applied an extensive multimorbidity definition, previously validated for identifying patients facing daily life and health inequalities (26); aged over 18 years, had two or more chronic conditions and experienced significant problems concerning their life and health due to their multimorbidity. To clarify the concept of multimorbidity, we specified that patients’ problems could be either biomedical or psychosocial: (1) having several chronic conditions from different organ systems, (2) having one or more of the conditions newly diagnosed or poorly regulated, (3) lacking social network, (4) being anxious or nervous and (5) feeling limited physically and socially by their conditions or (6) having problem in relation to the healthcare system.

JBB, an experienced moderator, led the first two groups, assisted by ABL, with roles swapped in the third session. The interviews were audio-recorded and transcribed and both moderators took notes for later analysis. The transcriptions served to identify new meaningful items. Patients completed the draft PROM and the MMQ1, noting any challenges. The focus groups lasted approximately 2 hours.

JBB, ABL and AH conducted the content analysis of the interviews. In cases where consensus could not be reached, the transcripts were revisited, or the audio recordings and participants’ responses were reviewed again until agreement was achieved.

Individual interviews

Individual interviews were conducted to ensure proper functionality with ease of completion and identify potential minor errors that had not been found in the group interviews. Participants were recruited in the same way as for group interviews. They were asked to complete the PROM and ‘think aloud’ while completing it. This ‘think aloud’ technique was applied also to test the participant’s understanding of the PROM, the structure and comprehensiveness of the response categories and the questionnaire layout.¹⁶ If the participants had difficulties understanding or completing an item or had critical comments, the interviewer probed the participant to elaborate on the nature of the problem.

The individual interviews lasted between 30 min and 60 min, took place in the general practitioners’ offices and were conducted by ABL. The interviews were audio recorded, and notes were taken. The audio recordings, the completed participant’s replies to the PROM, and ABL’s notes were discussed by JBB, ABL and AH. Items were modified based on these discussions and tested in subsequent interviews. The individual interviews were repeated until no new information or problems emerged.

Step 5: psychometric validation

The data for the psychometrical validation were collected in a survey distributed in connection with a cluster randomised trial (the MM600 trial) in which 250 general practices in Denmark participated.⁴ The draft PROM was sent to all chronic care patients, 18 years or older listed at participating practices in early 2023 and again in early 2024.

Confirmatory factor analysis (CFA) model fit was assessed with the goodness of fit index >0.95; root mean square error of approximation<0.06; standardised root mean square residual <0.06; and the Comparative Fit Index >0.95.^{17 18}

The domains were tested for internal consistency using Cronbach’s Alpha and reliability using the total reliability coefficient.¹⁹ Ceiling and floor effects were assessed by the number and percentage of participants scoring minimum resp. maximum on the corresponding scale.

We planned to test the theoretical domains; Information about treatment, Challenges with medication, Medical appointments, Challenges in the contact to the health system, Self-monitoring and Health behaviour. In case, these did not fit the CFA model, we planned to test alternative configurations of items based on the selected framework.

In addition, we assessed the individual items with regards to ceiling and floor effects and response rate (percentage not replying ‘not relevant’). Also, each item was assessed for criterion validity using Pearson’s correlation coefficient of each item with the following criteria: (1) the number of chronic conditions, (2) the number of prescription medicines, (3) the MMQ1 scale; limitations in everyday life⁶ as well as (4) two scales from the newly developed patient centredness in consultations (PCC) PROM; coordination of care and therapeutic alliance.²⁰ Furthermore, sensitivity to change for each item was assessed by the calculation of the effect size of the change in the criteria mentioned above, except the number of chronic conditions, which was only reported at baseline, on the change in item score between the baseline questionnaire and the 1-year follow-up, that is, the mean difference in item score for a one-point increase in the criterion divided by the criterion SD at baseline. Statistical analyses were performed in SAS, R and DIGRAM.²¹

All data management processes followed established regulations, including General Data Protection Regulation (GDPR) compliance. Data collection, storage and analysis were conducted in accordance with institutional guidelines and applicable legal requirements to maintain data integrity and confidentiality. Only authorised personnel had access to the data. The data were securely stored on encrypted drives, with regular backups performed to safeguard against data loss.

Patient and public involvement

Patients and the public were not involved in the design, conduct or dissemination of this study.

Results

Step 2: literature search

The initial search in PubMed yielded 561 results. After the screening of titles and abstracts, nine studies from the search and two additional studies from two systematic reviews identified in the search^{22 23} were selected for full-text reading. One study did not concern a PROM²⁴ and six studies investigated either a different patient group or a different concept than stated in our inclusion criteria.²⁵,³⁰ Two studies fulfilled the inclusion criteria^{11 14} and two additional studies were also included for quality assessment and data extraction, although they also included patients with only one chronic condition.^{31 32} See online supplemental appendix 1 for the PRISMA flowchart and online supplemental appendix 2 for reasons for exclusion.

The four included PROMs are presented in table 2. The Multimorbidity Treatment Burden Questionnaire (MTBQ), the PETS and the Treatment Burden Questionnaire (TBQ) had all been quality assessed previously.¹⁰ The Long Term Conditions Questionnaire (LTCQ) instrument was new, not previously quality assessed. The validation process of the LTCQ was described in five articles.³¹³³,³⁶

Table 2. Quality assessment of identified PROMs. COSMIN: COnsensus-based Standards for the selection of health status Measurement INstruments.

	Domains	Items	Quality in COSMIN domains
			1	2	3	4
LTCQ3133,36	1	20	Inadequate	Implicitly inadequate	Implicitly inadequate	Implicitly inadequate
MTBQ14	1	13	Inadequate	Implicitly inadequate	Implicitly inadequate	Implicitly inadequate
PETS8 9 11	10	48	Adequate	Adequate	Adequate	Very good
TBQ32	4	13	Doubtful	Implicitly doubtful	Implicitly doubtful	Implicitly doubtful

LTCQ, Long Term Conditions Questionnaire; MTBQ, Multimorbidity Treatment Burden Questionnaire; PETS, The Patient Experience with Treatment and Self-management; PROMs, patient-reported outcome measures; TBQ, Treatment Burden Questionnaire.

Box 1 in the COSMIN checklist, PROM development of LTCQ, was rated as ‘inadequate’ because the construct to be measured was poorly described and because interviews with patients were not used to generate items. Box 2, content validity ended up being rated as inadequate since it was not stated whether experienced interviewers were used. Otherwise, the rating would have been ‘very good’. Box 3, structural validity was rated as ‘adequate’. Box 4, internal consistency was rated ‘very good’. However, since box 1 was rated inadequate, boxes 2–4 were implicitly inadequate.

The PETS instrument had adequate validity, but, unfortunately, had several content-wise redundant items with the MMQ1 and would require substantial alterations, which would require a new content and psychometric validation. The LTCQ, MTBQ and the TBQ did not have adequate quality. Thus, none of the PROMs was suitable for unaltered use.

Step 3: content validation; item extraction, construction of a draft PROM and face validity

The four identified PROMs were used for item extraction. We extracted each item in the original language and grouped the items according to the 15 domains used in the PETS framework.^{10 11} In total, 91 items were extracted. We merged some of the domains as depicted in table 1. The domains 2b Interpersonal challenges, 2e Barriers to self-care, 3a Role and social activity limitations and 3a Role and social activity limitations and 3b Physical and mental exhaustion of self-care were removed because these aspects of treatment burden were already included in the MMQ1. After the removal of redundant domains and items, the draft PROM consisted of 6 domains and 48 items, which were ad hoc translated into Danish by a research assistant fluent in Danish and with good English skills. Since the items were to undergo further refinement or possibly deletion in the interviews, we did not use more extensive translation procedures.

The draft version of the PROM was reviewed for face validity by an expert panel consisting of AH (some experience in content validation and general practitioner with 15 years of clinical experience), ABL (some experience in qualitative research), JBB (extensive experience in content validation and GP with more than 30 years of clinical experience), and KB (the first author on the development of the MMQ1 instrument, thorough experience in content validation). The items were regrouped under new headlines presumed more relevant and understandable to the patients based on the panel’s experience. Content-wise redundant items were removed. The domain 1f Medical equipment/devices had no items, and we decided to remove it and explore further in the interviews if there were issues that could not be covered in the remaining domains. The items in the domain ‘Health behaviour’ did not seem suitable in their current form as further explained in a later section. Therefore, they were removed and replaced with four items relating to the health advices from the Danish Health Authorities concerning smoking, diet, alcohol and physical activity. At this point, we rephrased the questions on system obstacles to relate to obstacles encountered in general practice.

The resulting PROM consisted of six domains; Information about my treatments (1a+2d, 4 items), Challenges with medicines (1b+2a, 10 items), Medical appointments (1c, 2 items), Challenges in the contact to the health system (2f+2g, 2 items), Self-monitoring (1d, 4 items) and Health behaviour (1e, 4 items). The inclusion and removal of items are seen in table 1. Five response categories were designed to comply with the response options of the MMQ1: 0 (no, not at all), 1 (yes, a little), 2 (yes, some), 3 (yes, a lot) and 4 (not applicable/ do not know).

Step 4: content validation; group and individual interviews

Three focus group interviews were conducted: one in Region Zealand (four women and one man), one in the Capital Region (three women and two men) and one in the Central Region (three women and two men). In the first group interview, 1 new item on challenges with medicine was generated. In the second, no new items were generated, but one item was discarded about self-monitoring. In the third focus group interview, four new items in health behaviour were generated. 12 individual interviews with eight women and four men aged 53–81 years were conducted. The results of these individual interviews were minor corrections to phrasing and layout. No new items were generated in these individual interviews. Thus, we concluded that data saturation had been achieved. Following all interviews, the expert panel iteratively updated the questionnaire based on the findings in the interviews.

All participants, in both the focus group interviews and the individual interviews, were asked, ‘Are the instructions, questions and response categories understandable?’, thus probing the understandability of the questionnaire. Some participants expressed challenges with understanding one of the items in the domain related to medical appointments; ‘It is difficult for me to get an appointment with my GP within the timeframe recommended by my GP or other healthcare professionals.’ Although they eventually grasped the question after reading it several times, they found it initially difficult to comprehend. The participants were prompted to suggest alternative phrases to enhance clarity, but neither the participants nor the expert group could find a way to do so without altering the item’s meaning. Consequently, it was decided to retain the item as it was. Originally, in the draft, the domain ‘Health behaviour’ consisted of four items concerning health behaviour, with specific advice placed below the items. However, during the first focus group interview, some participants found it challenging to answer questions about specific health behaviours without having the corresponding advice adjacent to the question. Therefore, the decision was made to relocate the specific advice directly beneath the item it pertained to.

During the second focus group interview, some participants expressed that the domain concerning challenges with medication was not comprehensive. While they acknowledged the domain’s importance, they felt it did not address their primary concerns regarding medicine. They highlighted a lack of items related to the practical aspects of managing medication on a daily basis (such as opening packages, pushing pills out of the packaging and unscrewing lids) as well as opposite-oriented restrictions related to medicine (such as whether it should be taken with a meal, should not be taken simultaneously with other kinds of medication or should be taken on an empty stomach). In response to this feedback, several items were added to the medical challenges domain.

During the third focus group interview, some participants expressed difficulties regarding the relevance of the health behaviour domain. In Denmark, health behaviour is categorised by the Danish Health Authority into advice regarding smoking, diet, alcohol and physical activity. Some participants felt annoyed by being repeatedly asked about these factors, especially when they interacted with the Danish health system. They perceived it as a waste of time, particularly those who struggled with aspects such as smoking, as health professionals would extensively discuss challenges related to their health or any side effects, they experienced due to these factors. In collaboration with the participants, we added the questions to address the item, ‘It is burdensome for me to constantly be reined of the Danish Health Authority’s advice regarding smoking/diet/alcohol/physical exercise’.

The results of the qualitative evaluation were six domains containing 30 questions regarding treatment burden: 4 items regarding information about treatment, 11 items regarding challenges with medicine, 2 items regarding medical appointments, 2 items regarding challenges in the contact with the healthcare system, 3 items regarding self-monitoring and 8 items regarding health behaviour (table 1).

Step 5: psychometric properties

The PROM was distributed to 160 584 people (159 619 digitally and 965 by post) as part of an ongoing trial.⁴ 35 977 (22%) responded to the survey. 28 724 (80% of participants) agreed to reply to the treatment burden PROM, which was optional, and 28 476 (99%) completed. For our analysis, we excluded patients who did not take any medicines or did not report having any chronic conditions (n=1947) leaving 26 529 unique patients to be included in the analysis. 18 324 (69%) completed the treatment burden PROM again in 2024. These data were used for psychometric evaluation only. More details of the survey and the trial results will be described separately in future publications.

The characteristics of the respondents to the PROM compared with all the respondents and the full population are presented in table 3 and the distribution of covariates among the respondents is shown in online supplemental appendix 4.

Table 3. The characteristics of the respondents to the treatment burden PROM compared with all the respondents to the questionnaire and the full population, who received the questionnaire.

Variable n (%)	Respondents to the treatment burden PROM 2023, n=26 529	Respondents to the treatment burden PROM 2024, n=18 324	Respondents to the full questionnaire, n=35 977	Full population
Female	13 820 (52.1%)	9326 (50.9%)	18 665 (51.9%)	84 178 (52.4%)
Age
<40 years	1018 (3.8%)	439 (2.4%)	1595 (4.4%)	12 722 (7.9%)
40–49 years	1670 (6.3%)	838 (4.6%)	2263 (6.3%)	14 260 (8.9%)
50–59 years	4489 (16.9%)	2679 (14.6%)	6045 (16.8%)	30 262 (18.8%)
60–69 years	8023 (30.2%)	5621 (30.7%)	10 499 (29.2%)	42 366 (26.4%)
70 or older	11 329 (42.7%)	8747 (47.7%)	15 575 (43.3%)	60 974 (38.0%)
Education
Basic schooling	1669 (6.3%)	980 (5.4%)	2289 (6.9%)	–
High school or courses	11 916 (45.3%)	8127 (44.6%)	15 166 (45.9%)	–
Higher education 2 years	2694 (10.2%)	1913 (10.5%)	3361 (10.2%)	–
Higher education 3–4 years	7171 (27.3%)	5157 (28.3%)	8737 (26.4%)	–
Higher education 5–6 years	2863 (10.9%)	2061 (11.3%)	3515 (10.6%)	–
Chronic conditions
1 chronic condition	3816 (14.4%)	2546 (13.9%)	7844 (22.1%)	–
2–3 chronic conditions	12 248 (46.2%)	8488 (46.3%)	15 662 (44.2%)	–
4–5 chronic conditions	7867 (29.7%)	5494 (30.0%)	9054 (25.6%)	–
6 or more chronic conditions	2589 (9.8%)	1796 (9.8%)	2876 (8.1%)	–

PROM, patient-reported outcome measure.

None of the domains; Information about treatment, Challenges with medication, Medical appointments, Challenges in the contact to the health system, Self-monitoring and Health behaviour had sufficient fit to the CFA model and internal consistency (online supplemental appendix 3). We tried a number of alternative configurations of items based on the original framework. None of the alternative domains had sufficient psychometric properties (online supplemental appendix 3).

For the individual items, sufficient criterion validity represented by correlation coefficients above 0.3 or below −0.3 was mainly seen in the four items in the domain; ‘Information about treatment, the item ‘taking too much medicine’ in the ‘challenges with medication’ domain, and in the items regarding getting a medical appointment, seeing multiple doctors and experiencing bad communication between different doctors and for the item; ‘Living up to exercise advice’ and ‘being reminded about exercise advice’ (figure 2). The MMQ1 domain ‘limitations’ showed the highest degree of correlation to the items in this PROM although the items regarding getting a medical appointment, seeing multiple doctors and experiencing bad communication between different doctors also correlated with the PCC domains, especially ‘coordinated care’ (figure 2). Sensitivity to change was best in the item ‘taking too much medicine’ (effect size=0.31), with regards to changes in medication, and in the item ‘Lack of information about conditions’ (effect size=−0.22) with regards to self-reported health. Otherwise, sensitivity to change was low in this 1-year follow-up. All items had a substantial floor effect in this population. For most items, less than 10% of the respondents used the ‘don’t know’ category (for details about the item statistics, see online supplemental appendix 5).

Figure 2. Criterion validity, sensitivity to change and floor and ceiling effects for all the individual items. MMQ, MultiMorbidity Questionnaire; PCC, patient centredness in consultations.

Discussion

Summary of results

Through item extraction of existing questionnaires and construction of new items, we developed a PROM consisting of six domains encompassing 30 items regarding treatment burden: 4 items regarding information about treatment, 11 items regarding challenges with medicine, 2 items regarding medical appointments, 2 items regarding challenges in the contact with the health system, 3 items regarding self-monitoring and 8 items regarding health behaviour. The PROM has high content validity. The psychometric validation failed to show adequate measurement properties for any of the domains. Analyses of the individual items showed moderate criterion validity and sensitivity to changes for 10 items distributed among all the individual domains except self-monitoring.

Strengths and weaknesses

This study followed a rigorous, multistep development and validation process that included a literature review, item extraction, focus group interviews, individual interviews and psychometric testing within a broad sample of patients from primary care. However, the study had several limitations.

The systematic review excluded non-PubMed databases such as PsychInfo and Embase. This decision was based on a prior systematic review with a related focus that did not identify additional PROMs using these databases.¹⁰ The omission of other databases may have led to unidentified PROM development studies. We identified several potential PROMS, but none of them could be used unaltered, and we proceeded to develop and validate a new PROM, which could be used in combination with MMQ1. In this process, some domains described in the original framework were discarded, because they were already covered by MMQ1. While the initial focus group interviews were conducted by an experienced interviewer in content validation, the subsequent interviews were led by a qualitative researcher with limited experience in this area. We conducted focus groups in three geographical regions, but the remaining two regions in Denmark were only covered by individual interviews. However, our findings in the interviews were quite consistent across geographical regions contrary to what we expected. Moreover, the study did not include interviews with clinical experts concerning face validity, as recommended by the COSMIN risk of bias checklist. However, several of the group members were experienced general practitioners, partly making up for this limitation. The cohort for the psychometric validation consisted of a broad sample of patients with chronic illness, but not necessarily experiencing problems due to their illness. While this does not compromise the internal validity of our findings, we did observe substantial floor effects in most items. Thus, when evaluating the usefulness of items for other populations, our sample distribution should be taken into account. The primary limitation of this study was the fact that none of psychometric analyses provided evidence of adequate measurement properties of the domains. We did not use other statistical methods to identify alternative data-driven domains since, without a solid theory, these would not be reliable to capture the concept fully. The most likely explanations for the lack of fit to the model are that the original framework was developed in a completely different setting and lacked a strong theoretical foundation justifying the use of CFA. If that is the case, the development of a PROM to measure treatment burden in a Danish context would require rethinking a theoretical framework that could conceptualise the phenomenon called treatment burden in a setting with a free healthcare, short distances and a high degree of technological health solutions. A substantial floor effect was observed for all the items. This can be explained by the relatively low impairment in our sample of chronic care patients used for the psychometric validation, which differed from patients with more complex multimorbidity who participated in the qualitative development phase.

Relation to the literature

Several PROMs have already been developed and validated to measure treatment burden in patients with multimorbidity. The MTBQ is commonly used in clinical trials due to its briefness and ease of use.¹⁴ The MTBQ was also based on the framework by Eton et al⁹ but did not report a formal content validation procedure. They did not report any tests of unidimensionality but provided correlations between the total MTBQ score and several other measures. In addition, they reported linear regression coefficients to the EQ-5D-5L and PACIC scores at 1-year follow-up of −0.14 and −0.17, respectively, but not the responsiveness to changes in the number of medications or other measures of morbidity. Thus, our results are difficult to compare to the MTBQ. The PETS obtained the best COSMIN score in our quality assessment. In addition, the authors have recently developed the PETS into a clinical tool to monitor the treatment burden of individual patients.³⁷ The PETS had a solid content validation procedure. They used the psychometric analysis to modify the framework and exclude items, which may have limited the content validity to increase the psychometric properties, and they tested all 46 items in a single, two-factor model with nine domains, which had a good fit in the CFA analysis. As in our study, the floor effects were substantial. The authors have developed a shorter version of PETS (published after our literature review), which found a more moderate internal consistency comparable to the one found in most of our domains.³⁸

Conclusions

We developed and validated a PROM to measure treatment burden in patients with a chronic disease and multimorbidity in general practice, which could be used as a supplement in studies using measuring need-based quality of life with the MMQ1. The PROM had high content validity, but our psychometric analyses did not provide evidence of unidimensionality within the hypothesised scales. Ten individual items showed adequate criterion validity, and some also had sensitivity to changes. Thus, the PROM can primarily be used in its current form using the individual items. However, we plan to further refine the PROM in qualitative interviews using those single items with the most adequate measurement properties to explore further understandings of the concept of treatment burden in the context of Danish general practice.

Data availability statement

Data are available upon reasonable request.