Identifying a core outcome set for pulmonary sarcoidosis research – the Foundation for Sarcoidosis Research – Sarcoidosis Clinical OUtcomes Taskforce (SCOUT)

Abstract

Background:

Pulmonary sarcoidosis is a rare granulomatous disease of unknown aetiology. Heterogeneity in the outcomes measured in trials of treatment for pulmonary sarcoidosis has impacted on the ability to systematically compare findings, contributing to research inefficiency. The FSR-SCOUT study has aimed to address this heterogeneity by developing a core outcome set that represents a patient and health professional consensus on the most important outcomes to measure in future research for the treatment of pulmonary sarcoidosis.

Research design and methods:

systematic review of trial registries, narrative synthesis of published qualitative literature on the patient experience and results of a patient survey contributed to the development of a comprehensive list of outcomes that were rated in a two round online Delphi survey. The Delphi survey was completed by patients/carers and health professionals and the results discussed and ratified at an online consensus meeting.

Results:

259 patients/carers and 51 health professionals completed both rounds of the Delphi survey. A pre-agreed definition of consensus was applied and the results discussed at an online consensus meeting attended by 17 patients and 7 health professionals). Fifteen outcomes, across five domains (physiological/clinical, treatment, resource use, quality of life, and death), reached the definition of consensus and were included in the core outcome set.

Conclusions:

The core outcome set represents a patient and health professional consensus on the most important outcomes for pulmonary sarcoidosis research. The use of the core outcome set in future trials, and efforts to validate its components, will enhance the relevance of trials to stakeholders and will increase the opportunity for the research to contribute to evidence synthesis.

Appendix

Supplementary file 1. Written questionnaire capturing patient voice

FSR Endpoints Initiative

The members of the Endpoints Steering Committee for the Endpoints Initiative are committed to ensure that both the physicians and the patients have a voice in determining which Core Outcomes are important based on life experience alongside data. The purpose of this questionnaire is to hear from you, as a pulmonary sarcoidosis patient, and learn how the disease impacts your life. This will help to identify Core Outcomes that are important to pulmonary sarcoidosis patients.

What is a Core Outcome? A Core Outcome is something that physicians and researchers use to measure how well a treatment or an assessment is working. Research studies testing treatments often measure different outcomes. For example, one pulmonary sarcoidosis trial might use a CT scan to track improvements, while another might use a 6-minute walk test. For patients, outcomes related to pain and fatigue might be of great importance, but can be hard to measure. If researchers measure different things, it makes it difficult to compare and combine the results. But if all future research studies measure the same important outcomes, then the results will be combined and new treatments that work will be available for people with pulmonary sarcoidosis more quickly. The information you provide in this questionnaire will help to decide what outcomes are the most important and should be measured in all future pulmonary sarcoidosis research. Your opinion is very important.

Written Questionnaire

Topic 1: Symptoms That Matter Most to Patients

Of all the symptoms you have experienced with pulmonary sarcoidosis, which do you consider to have the most significant impact on your daily life?

How often do these symptoms affect you, using scale - never, sometimes, often, always?

Rank the top three aspects/symptoms of the disease that impact your daily life.

How has your condition and its symptoms changed over time?

How much of your day is spent thinking and planning activities because of shortness of breath? (025%, 25 - 50%, 50 - 75%, or >75%) Please explain.

Things to consider when answering: Planning steps and routes in order to get form one place to the next. Avoiding stairs or inclines, and looking for elevators or escalators. Planning meals around times you know you have to walk.

Topic 2: Daily Impacts That Matter Most to Patients

Are there specific activities that are important to you, but you can’t do at all or as fully as you would like because of your condition?

How do your symptoms and their negative impacts affect your daily life on the best days? On the worst days?

Things to consider when answering: How do your symptoms impact you at work? How do they impact you at home? How do they impact you with your relationships or social activities?

How has your condition and its symptoms impacted you emotionally?

How often do you avoid interactions with people or feel embarrassed due to the breathlessness brought on by sarcoidosis? Use the scale - never, sometimes, often, always. Please explain.

Things to consider when answering: Trouble walking at a "normal" pace with others, trouble walking and talking at the same time, and inability to participate in activities.

How fearful has sarcoidosis caused you to become? Use the scale - not at all fearful, somewhat fearful, often fearful, very fearful. Please explain.

Things to consider when answering: Fear of contracting infections, impact on your financial future, declining health, and dying.

How often has sarcoidosis made you feel guilty? Use the scale - never, sometimes, often, always. Please explain.

Things to consider when answering: Not being able to fulfill responsibilities at work or home, and the perception of being a burden to family and friends.

In addition to the care you receive from your doctors, what else or who else has helped you manage the disease? How important are they to you?

Things to consider when answering: support groups, family, friends, spirituality, and faith.

Topic 3: Outcomes That Matter Most to Patients

In your own words, describe a core outcome?

What outcomes do you think are the most critical to measure during a pulmonary sarcoidosis clinical trial?

Suplementary file: Feedback questionnaire

The questions below was provided to participants using Google Forms and were answered anonymously.

	Response options
Which best describes how you voted at the meeting?	As a patient
	As a health professional
Please tell us how much you agree with each of the following statements
1 strongly disagree, 2 - disagree, 3 neither agree nor disagree, 4 - agree, 5 - strongly agree	Response options
The information that the organisers provided me with in advance of the meeting was helpful.	1-5 Likert scale
I was satisfied with the amount of background to the study presented on the day	1-5 Likert scale
I was satisfied with the information, presented at the meeting, about how the day was going to work	1-5 Likert scale
I was satisfied with the process used to discuss and vote on outcomes	1-5 Likert scale
I was satisfied with the way the meeting was facilitated	1-5 Likert scale
I felt able to contribute to the meeting	1-5 Likert scale
I felt comfortable communicating my views	1-5 Likert scale
The workshop produced a fair result	1-5 Likert scale
The meeting length was just right	1-5 Likert scale
I was satisfied with the number of comfort breaks	1-5 Likert scale
I was satisfied with the use of Zoom to conduct the meeting	1-5 Likert scale
Was there anything that could have been done to improve the workshop	free text
Is there anything else that you would like to tell us	free text

Supplementary file 5. Attrition bias analysis

• 166 (32.9%) of 504 participants with data in round 1 did not provide any scores in round 2

  • In round 1, participants provided a mean (SD) of 45 (12.2) outcome ratings (median 49, IQR = 49,49) (range = 1,51).
  • In round 2, participants provided a mean (SD) of 32 (24.4) outcome ratings (median 51, IQR = 0,51) (range = 0,51).

Histograms are shown below for all participants combined, and for patients (including carers);

Foundation for Sarcoidosis Research - Sarcoidosis Clinical Outcomes Taskforce (FSR-SCOUT) - Consensus meeting report

Meeting date and time: 16^th October 2020

Location: Online

Contents

1 Summary	1
2 Consensus meeting participants	1
3 Outcomes	2
3.1 Physiological/clinical outcomes	2
3.1.1 Outcome Discussion and rating	3
Cough	3
Recurrence	3
Fatigue	Errore. Il segnalibro non è definito.
3.2 Health and Quality of Life Outcomes	3
Overall quality of life/General health/Perceived health status	4
3.3 Life impact outcomes	4
Activities of daily living/ Ability to work or study/ Ability to undertake usual role/responsibilities/Ability to take part in usual family life/activities/Cognitive Function	4
3.4 Treatment outcomes	5
Satisfaction with treatment	5
4 Discussion	5

1 Summary

Following a two round online Delphi survey an online consensus meeting was held on the 16^th October 2020 to discuss outcomes where, according to the pre-agreed definition of consensus, consensus for inclusion in, or exclusion from, the core outcome set had not been reached. This report summarises these discussions and the resulting core outcome set.

2 Consensus meeting participants

Twenty-five participants, who had completed both rounds of the online Delphi survey, attended the online meeting (7 health professionals, 17 patients) (Table 1). In addition, nine pharmaceutical company representatives attended the meeting and were invited to contribute to discussions but were not eligible to rate the inclusion of outcomes in the core outcome set.

3 The average R2 ratings for participants of the consensus meeting compared to all participants is shown in table 2. Outcomes

At the end of round 2 of the Delphi survey the definition of consensus was applied to the responses for each stakeholder group (Supplementary file 1 - consensus matrix). Fifteen outcomes met the definition for "consensus in" to be included in the core outcome set and six met the definition of "consensus out" and were excluded from the core outcome set, the remaining outcomes had no consensus. Seventeen outcomes were prioritised for discussion at the consensus meeting as either 70% or more of participants in one stakeholder group, or 50-69% of participants in both stakeholder groups, had rated the outcome 7-9.

For the purpose of the consensus meeting the 17 outcomes, prioritised for discussion, were grouped into four domains, physiological/clinical (8 outcomes), health and quality of life (3 outcomes), life impact (5 outcomes) and treatment (1 outcome). All outcomes in a particular domain were presented, alongside those outcomes already included in the COS, and participants of the meeting invited to provide comments for inclusion of outcomes followed by comments against. After discussion of outcomes in that domain participants rated each outcome, that had been discussed, using the 1-9 scale (1 not that important - 9 critically important). Patients and health professionals voted separately, for an outcome to be included in the core outcome set 70% or more of participants in both groups were required to give a rating of 7-9. The results of consensus meeting ratings are provided in Table 3.

3.1 Physiological/clinical outcomes

Eight outcomes were prioritised for discussion in the physiological/clinical domain. These were described and presented alongside outcomes in the same domain that had reached "consensus in" in the Delphi surveys.

No participant offered reasons for inclusion of the outcomes "systemic inflammation", "pain", "chest pain", "mobility" or "infection" as such these outcomes were not discussed or rated.

3.1.1 Outcome Discussion and rating

Cough - the impact of cough on quality of life was discussed, with patients explaining that this was one of the worst, most debilitating symptoms. A different view held was that cough is not necessarily specific to pulmonary sarcoidosis and should only be measured as an outcome if it is attributed to sarcoidosis. However, it was also raised that the cause is irrelevant and what matters to patients is the cough and how debilitating it is. It was noted that it would be important to measure whether a cough gets better but also if a cough gets worse.

Result of voting: Not included in the core outcome set

Fatigue - the prevalence of fatigue was discussed together with the impact of fatigue on quality of life and on other aspects like the ability to work. It was raised that fatigue can be present when there is no granulomatous inflammation and so whilst it is important it might not be a core outcome for every trial. It was also discussed that fatigue might be a side effect of some treatments and that side effects of treatment is already an included outcome domain.

Result of voting: Not included in the core outcome set

Recurrence - pulmonary sarcoidosis is characterised by remission and relapse. Patients agreed that it is important to measure "recurrence of sarcoidosis" to know whether a treatment is effective long term. However, patients also commented that it makes the assumption that the sarcoidosis goes away at some point but this may not be the same for everyone. It may also take a number of months/years for patients to go into remission and this outcome could only be measured in studies with a long follow up period. It was noted that if there is a positive effect of treatment then it is important to know if this effect is maintained long term.

Result of voting: Not included in the core outcome set

3.2 Health and quality of life outcomes

All three outcomes in the health and quality of life outcomes domain were discussed together. There was a comment that health professionals may have rated these outcomes less important as they felt that the concepts would already be covered by "health related quality of life". The meeting facilitator reminded participants that this meeting would recommend the overall core outcome set but future research would be needed to decide how to measure each outcome. Participants were asked to consider whether specific additional outcomes should be included alongside "health related quality of life" as this would then contribute to the outcome measurement tool chosen. After discussion participants were asked to rate the outcomes based on whether it was critical to include these as part of the "health related quality of life" assessment.

Overall quality of life/General health/Perceived health status - Some patients felt that specific symptoms affected their quality of life and so measuring those specific symptoms was more important. Some felt that they considered "health related quality of life" to be the most important whilst others felt that an overall quality of life measure, which assessed usual day to day activities and mental health issues, was important to consider but that this was also correlated to health. It was noted that it was important for this to be assessed from the patient perspective rather than the health professionals’ opinion.

Result of voting: Not included in the core outcome set

3.3 Life impact outcomes

All outcomes in the life impact domain were discussed. Clarification was sought on whether the "health related quality of life" outcome would include assessment of these life impact outcomes. The meeting facilitator clarified that it should not be assumed that these are included and if these individual outcomes are critically important to measure then they should be voted in individually.

Activities of daily living/ Ability to work or study/ Ability to undertake usual role/responsibilities/Ability to take part in usual family life/activities/Cognitive Function

For some patients these outcomes were considered to be secondary to symptoms i.e. specific symptoms have a life impact and so it is those that are critically important to measure. Discussion also took place around the impact on the ability to work/study and that this can have a major impact on a patient’s life. There was some discussion that patients would want their doctor to ask these questions but that they may not be critical to measure in a trial.

Outcomes were discussed together but voted on individually.

Result of voting: Not included in the core outcome set

3.4 Treatment outcomes

Participants were asked if "satisfaction with treatment" was critical to include alongside the other treatment outcomes that were already in the core outcome set.

Satisfaction with treatment - clarification was sought that this outcome was for clinical trials and that these might be for new or established treatments. Some felt that this was important as it gave the perspective of the treatment in terms of side effects of treatment and tolerability which, although included, might not capture the patient perspective. Others felt that this outcome did not add anything to the outcomes already included.

Result of voting: Not included in the core outcome set 4

4 Discussion

Fifteen outcomes were included, in the core outcome set, after the two round online Delphi survey and no further outcomes were added at the consensus meeting - the core outcome set is reported in Table 4.

Review and discussion of the outcomes with "no consensus" highlighted the different stakeholder viewpoints in relation to each outcome. Patients and health professionals agreed that five of the 17 outcomes did not need further discussion at the consensus meeting and would not be included in the core outcome set. Of the 12 outcomes that were discussed patients and health professionals both agreed that six of these should not be included in the core outcome set with 50% or less of participants in both groups rating the outcome 7-9. However, for the remaining outcomes there were differences of opinion between the stakeholder groups. There were four outcomes (recurrence of sarcoidosis, cognitive function, satisfaction with treatment and activities of daily living) that did not reach the definition of consensus "in" for either group but were rated more highly by patients (rated7-9 by 50-67%) than health professionals (rate 7-9 by 0-29% ).

Two outcomes "cough" and "fatigue" were rated critically important by patients (70% and 80% of patients rating the outcome 7-9 respectively) but not by health professionals (57% and 14% of health professionals rating 7-9). Discussions and comments from patients highlighted the impact that cough and fatigue have on everyday life, with comments being made that these were the most important symptoms of pulmonary sarcoidosis. Existing patient reported outcome measures for "health related quality of life" include questions relating to these items. Future work to identify a suitable health related quality of life measurement instrument (the ’how’ to measure) may also want to consider whether items relating to "cough" and "fatigue" are included.

Table 1.

Consensus meeting participants

	N (%)
Healthcare professionals	7 (100%)
Role
Sarcoidosis specialist	3 (43%)
Researcher in the field	1 (14%)
Industry representative	3 (43%)
Country of residence
United States	5 (71%)
India	1 (14%)
The Netherlands	1 (14%
Patients with pulmonary sarcoidosis	17 (100%)
Country of residence
United States	14 (82%)
UK	3 (18%)

Table 2.

Delphi round 2 ratings for participants of the consensus meeting versus all participants.

	Average R2 rating of all participants	Average R2 rating of those attending the consensus meeting	Average R2 rating for Cough All participants	Average R2 rating for cough - Consensus meeting participants	Average R2 rating for Fatigue - all participants	Average R2 rating for fatigue - consensus meeting participants
Health professionals	6.5	5.8	6.6	6.6	6.1	5.4
Patients	7.3	7.4	7.1	6.9	6.9	8.5

Table 3.

Summary of outcome discussed and rated during the consensus meeting

Domain	Outcome		% patients rating 7-9 in online Delphi	% HCPS voting 7-9 in online Delphi	% Patients voting 7-9 in consensus meeting	% HCPs voting 7-9 in consensus meeting	Result
Physiological/clinical	Cough	Cough	70	59	70%	57%	Not included in the COS
	Fatigue	An overwhelming, sustained sense of extreme tiredness or lethargy resulting in physical and/or mental weariness	88	61	80%	14%	Not included in the COS
	Recurrence of sarcoidosis	Sarcoidosis coming back after a period of remission	86	69	50%	29%	Not included in the COS
	Systemic inflammation	An immune response resulting in inflammation in the whole body, including the lungs	92	54	Not discussed or rated	Not discussed or rated	Not included in the COS
	Pain	A feeling of noticeable discomfort or an unpleasant physical sensation, in general, experienced anywhere in the body.	73	29	Not discussed or rated	Not discussed or rated	Not included in the COS
	Chest pain	Pain specifically in the chest	78	27	Not discussed or rated	Not discussed or rated	Not included in the COS
	Mobility	Includes ability to walk, climb, run, stand, sit without difficulty, taking into account stiffness.	78	45	Not discussed or rated	Not discussed or rated	Not included in the COS
	Infection	The presence of an unexpected and unwanted virus, bacteria, fungus or mycobacteria anywhere in the body	79	26	Not discussed or rated	Not discussed or rated	Not included in the COS
Quality of life/ general health	Overall quality of life	A state of health, happiness, comfort and well-being	88	67	47	0	Not included in the COS
	General Health	Someone’s general health	74	45	44	0	Not included in the COS
	Perceived health status	How someone thinks their overall health is	63	55	25	0	Not included in the COS
Life impact outcomes	Activities of daily living	Being able to take care of oneself and complete usual daily activities/tasks, for example, eating, bathing and dressing oneself	83	69	58	17	Not included in the COS
	Ability to work or study	Someone’s ability to work or study, ability to gain or keep employment, influence of pulmonary sarcoidosis on the type of job or study undertaken, time off from work or study, impact on career.	78	63	17	17	Not included in the COS
	Ability to undertake usual role/responsibili ties	Being able to manage personal role and responsibilities	75	49	33	17	Not included in the COS
	Ability to take part in usual family life/activities	Being able to take part in family life and activities	70	43	42	17	Not included in the COS
	Cognitive Function	Mental abilities/mental processes, including memory, concentration, language and thinking.	80	45	67	0	Not included in the COS
Treatment outcomes	Satisfaction with treatment	How satisfied someone is with the treatment/s they are receiving including satisfaction with its effectiveness (how well it’s working) and the time spent on treatment.	85	49	50	0	Not included in the COS

Table 4.

Outcomes included in the Core Outcome Set

Domain	Outcome	Outcome description
Physiological/Clinical	Disease activity	A measure of current, active, inflammation indicating active sarcoidosis.
Physiological/Clinical	Extra pulmonary organ involvement	Having sarcoidosis in other organs as well as the lungs
Physiological/Clinical	Extra pulmonary organ impairment	When sarcoidosis causes problems in other organs meaning that they don’t function properly and/or may worsen over time.
Physiological/Clinical	Dyspnoea	Shortness of breath/being unable to catch breath
Physiological/Clinical	Pulmonary function	How well someone’s lungs are working
Physiological/Clinical	Oxygenation	How well oxygen is being sent to parts of the body
Physiological/Clinical	Functional exercise capacity	Includes what day to day activities someone is able to do including the ability to do complete physical activity and exercise. This includes the ability to walk (including, for example, walking up an incline, walking a long distance and walking whilst talking)
Quality of Life	Health related quality of life	An overall measure of how a person’s health affects their general wellbeing; perceived physical, mental and social health over time
Treatment	Adherence to treatment	The degree to which someone follows medical advice or guidance from their doctor, for example, taking their prescribed medications.
Treatment	Tolerability of treatment	How tolerable the treatment is, for example, burden of treatment, side effects etc.
Treatment	Treatment failure	When the current treatment is no longer working to control pulmonary sarcoidosis symptoms
Treatment	Side effects of treatment	When the treatment given causes unwanted/unintended effects
Resource Use	Need for hospitalisation because of pulmonary sarcoidosis	How often someone is admitted to hospital because of pulmonary sarcoidosis
Death	Death - any cause	Death from any cause
Death	Death - pulmonary sarcoidosis	Death as a result of having pulmonary sarcoidosis

Background

Sarcoidosis is a systemic granulomatous disease of unknown aetiology. Sarcoidosis can affect any organ but most commonly affects the lungs with lung involvement observed in more than 90% of sarcoidosis patients [1-3]. Pulmonary sarcoidosis may cause significant pulmonary symptoms, pulmonary dysfunction, and life-threatening complications such as pulmonary hypertension and end-stage pulmonary disease. The management of pulmonary sarcoidosis is aimed at preventing/controlling organ damage, relieving symptoms, and improving the patient’s quality of life.

A systematic review of outcomes measured in clinical trials evaluating treatments for pulmonary sarcoidosis has identified heterogeneity in the outcomes measured [4] . This review also noted differences in the outcomes measured and reported in phase2/3/4 clinical trials and studies that report the patient experience suggesting that outcomes that are most important/relevant to patients may not have always be considered in clinical trials. This heterogeneity in the choice of outcomes and the methods of assessment has impacted on the ability to combine evidence in meta-analyses [5, 6]. One way to address this heterogeneity and subsequent research waste is to use a core outcome set (COS), defined as “the minimum [set of outcomes] that should be measured and reported in all clinical trials of a specific condition” [7].

Whilst some work to harmonise outcomes in the field of pulmonary sarcoidosis has already been undertaken [8, 9] there have been limitations to the methodology used, for example, a limited range of stakeholders or the rating of a short list of pre-selected outcomes only. Consequently, a need to develop a set of core outcomes, developed in-line with the COS-STAD guidelines[10], that reflected the opinions of health professionals, patients and researchers was identified.

The aim of the Sarcoidosis Core Outcomes Taskforce (SCOUT) study was to develop a COS for use in clinical trials of any intervention for the treatment of pulmonary sarcoidosis that includes input from health professionals, patients and researchers in the field. Recognizing that many of the outcomes that have been used are not validated, the long-term goal of this project is to prioritize outcomes that can be subjected to future research for validation.

Methods

The development of the COS involved three stages: the generation of a long list of outcomes for use in an online Delphi survey, a two round online Delphi survey with key stakeholders and an online consensus meeting to discuss the results of the survey and agree the COS (figure 1).

Figure 1.

Overview of the development of the Core Outcome Set

The methods for each step are described briefly below, a study protocol and systematic review describing methods have been published elsewhere [11, 12].

Outcome list generation

The outcome list for use in the online Delphi survey was generated using three sources: registered clinical trials for interventions to treat pulmonary sarcoidosis, published qualitative literature relating to the patient experience and a written patient questionnaire completed by a patient advisory group. The search strategies used for registered trials and qualitative literature have been published elsewhere [12]. The patient questionnaire is provided in supplementary file 1. Outcomes were extracted verbatim from each source and then grouped using a standardised outcome name. Outcomes were also categorised using the taxonomy of Dodd et al [13]. Outcomes relating to a diagnostic procedure, specific to pulmonary hypertension or considered by the SSC to be unrelated to pulmonary sarcoidosis were not included. The resulting list of outcomes was reviewed by the SSC and plain language descriptions developed for each outcome.

Delphi survey

The final list of outcomes was used to populate an online Delphi survey delivered using the DelphiManager platform [14]. Delphi participants were invited from three key stakeholder groups: health professionals with experience of treating sarcoidosis, researchers in the field, and patients with pulmonary sarcoidosis and their carers. No restrictions were placed on patients in terms of time with pulmonary sarcoidosis, current or previous treatment or co-morbidities. However, patients with co-morbidities were advised to consider only their pulmonary sarcoidosis when responding to the Delphi survey. Invitations to take part were distributed via the Foundation for Sarcoidosis Research using established mailing lists of patients and health professionals, the study invitation was also distributed to the WASOG (World Association of Sarcoidosis and other Granulomatous Disorders), AASOG (Americas Association of Sarcoidosis and Other Granulomatous Disorders), and St. Antonius international network of expertise sarcoidosis centre. As part of the registration for the online Delphi, patient participants self-selected as having experience of living with pulmonary sarcoidosis, no further detail was collected on the nature or duration of symptoms, or the presence of co-morbidities.

The Delphi process comprised two rounds, round 1 (R1) and round 2 (R2). In each round the list of outcomes was presented and asked participants to rate each outcome, on how important it was to include it in the COS, using a nine point Likert scale presented in the format 1 to 9, with 1 to 3 labelled ‘not important’, 4 to 6 labelled ‘important but not critical’ and 7 to 9 labelled ‘critically important’[15]. At the end of R1 participants were able to add any additional outcomes that they felt were missing from the list. Outcomes added in R1 were reviewed by the SSC and any suggestions representing a new outcome were added to the list to be rated in R2. Outcomes were not removed from the list between R1 and R2.

During R2, participants were shown their rating from R1 along with a histogram of the distribution of scores for each stakeholder group for each outcome. Participants were asked to consider this information before rating the outcome again using the same 1-9 Likert scale.

For the purpose of the histograms two stakeholder groups were shown “health professionals” and “patients” with “researchers” included in the “health professionals group”.

Consensus meeting

An online consensus meeting was held using the Zoom platform. The meeting was structured using the consensus matrix of round 2 results (supplementary file 2) to identify outcomes that had met the pre-defined definition of consensus “in” or consensus “out” (table 1) and outcomes where there was disagreement between stakeholder groups. The criteria for the inclusion of an outcome was 70% or more in each stakeholder group rating an outcome 7-9 and less than 15%, in each group, rating 1-3. This criteria was chosen based on cut off values used in previous core outcome sets. 70% represents a balance between a less stringent cut off e.g. 50% that could result in COS with an unwieldy number of outcomes, and a more stringent cut off e.g. 90% that may exclude some important outcomes. Participants who had completed both R1 and R2 of the Delphi survey were invited to attend the consensus meeting and, if interested in attending, were asked to confirm this at the end of R2.We anticipated a maximum of 30-40 consensus meeting participants, if the number of people expressing an interest in attending exceeded this, places would be offered to ensure representation of each stakeholder group and the roles within these i.e. patient or carer, clinical role, research experience etc. Prior to the meeting participants received a summary of what to expect on the day, a summary of outcomes that would be discussed at the meeting that included the Delphi ratings of each stakeholder group, and a copy of their own ratings from the online Delphi. The meeting was chaired by an independent non-clinical researcher with expertise in COS development. Outcomes that had reached the definition of “consensus in” or “consensus out” were sent to participants prior to the meeting and not discussed. Outcomes that had had been rated 7-9 by 70% or more of participants in one stakeholder group were discussed at the meeting. Outcomes where neither group rated the outcome as “consensus in” were not discussed. For the purpose of the consensus meeting, outcomes prioritised for discussion were grouped into four domains; physiological/clinical, health and quality of life, life impact and treatment. All outcomes for discussion in a particular domain were presented, alongside outcomes in the same domain that had met the definition of “consensus in” and would be included in the COS. Meeting participants were invited to provide comments for inclusion of outcomes followed by comments against. After discussion of outcomes in that domain participants rated each outcome, that had been discussed, on how important it was to include it in the COS using the 1-9 scale (1 not that important – 9 critically important). Patients and health professionals voted separately and anonymously, using separate polls delivered using the Zoom platform. For an outcome to be included in the core outcome set 70% or more of participants in both groups were required to give a rating of 7-9.

Table 1.

Definition of consensus

Consensus Classification	Description	Definition
Consensus in	Consensus that outcome should be included in the core outcome set	70% or more participants in each stakeholder group scoring as 7-9 AND <15% participants in each stakeholder group scoring as 1-3
Consensus out	Consensus that outcome should not be included in the core outcomes set	50% or fewer participants scoring 7-9 in each stakeholder group.
No consensus	Uncertainty about importance of outcome	Anything else

Other analyses

Attrition bias between R1 and R2 of the online Delphi was assessed by comparing the distribution of mean R1 scores for participants completing R1 only and participants completing both R1 and R2. Satisfaction with the consensus meeting process, organisation and outcome was assessed using an online questionnaire sent to consensus meeting participants by email (supplementary file 3).

Ethical approval, study registration and study oversight

The FSR-SCOUT study was prospectively registered with the COMET Initiative (Core Outcome Measures in Effectiveness Trials) (ref 1156). Ethical approval was obtained from the University of Liverpool Research Ethics Committee prior to undertaking the consensus methods (online Delphi and consensus meeting) ref:5211. The FSR-SCOUT study is reported in line with the Core Outcome Set – Standards for Reporting (COS-STAR) reporting guidance [16]. Study oversight was provided by a Steering Committee comprised of five sarcoidosis experts and healthcare professionals, a psychometrician, a patient, two pharmaceutical representatives with sarcoidosis research experience, 2 regulatory experts with FDA experience, a representative from the Foundation for Sarcoidosis Research and three members from the COMET Initiative.

Results

An overview of the COS development process and final COS is shown in figure 1. The final COS includes 15 outcomes across five domains (table 2).

Table 2.

Outcomes included in the Core Outcome Set

Domain	Outcome	Outcome description
Physiological/Clinical	Disease activity	A measure of current, active, inflammation indicating active sarcoidosis.
Physiological/Clinical	Extra pulmonary organ involvement	Having sarcoidosis in other organs as well as the lungs
Physiological/Clinical	Extra pulmonary organ impairment	When sarcoidosis causes problems in other organs meaning that they don’t function properly and/or may worsen over time.
Physiological/Clinical	Dyspnoea	Shortness of breath/being unable to catch breath
Physiological/Clinical	Pulmonary function	How well someone’s lungs are working
Physiological/Clinical	Oxygenation	How well oxygen is being sent to parts of the body
Physiological/Clinical	Functional exercise capacity	Includes what day to day activities someone is able to do including the ability to do physical activity and exercise. This includes the ability to walk (including, for example, walking up an incline, walking a long distance and walking whilst talking)
Quality of Life	Health related quality of life	An overall measure of how a person’s health affects their general wellbeing; perceived physical, mental and social health over time
Treatment	Adherence to treatment	The degree to which someone follows medical advice or guidance from their doctor, for example, taking their prescribed medications.
Treatment	Tolerability of treatment	How tolerable the treatment is, for example, burden of treatment, side effects etc.
Treatment	Treatment failure	When the current treatment is no longer working to control pulmonary sarcoidosis symptoms
Treatment	Side effects of treatment	When the treatment given causes unwanted/unintended effects
Resource Use	Need for hospitalisation because of pulmonary sarcoidosis	How often someone is admitted to hospital because of pulmonary sarcoidosis
Death	Death - any cause	Death from any cause
Death	Death - pulmonary sarcoidosis	Death as a result of having pulmonary sarcoidosis

Development of the long list of outcomes

The systematic review of clinical trials and qualitative literature has been presented in detail elsewhere[12].

The review of registered clinical trials identified 36 trials, eligible for inclusion that reported a total of 364 individual outcomes, representing 56 unique outcomes. Six qualitative reports were included reporting 179 individual and 82 unique outcomes. Three patient questionnaires were completed and the verbatim free text responses identified 54 individual outcomes representing 26 unique outcomes in that data set. The taxonomy of Dodd et al [13] was applied to all outcomes. The unique outcomes were pooled from all sources and grouped by taxonomy domain, outcomes in each domain were then reviewed by the SSC. Outcomes were further grouped where appropriate, for example, outcomes relating to extra-pulmonary organ involvement such as outcomes relating to the eyes or heart were grouped into the outcome “extra pulmonary organ involvement” and the outcome “pulmonary inflammation” was grouped with “disease activity”. Outcomes considered to be related to a diagnosis rather than treatments were not included. In the interests of achieving a manageable outcomes list outcomes that were reported in a single study only were reviewed by the SSC and were not taken forward to the R1 outcomes list. The final list of outcomes (supplementary file 4) that was rated in R1 of the Delphi survey included 49 outcomes grouped under five domains (mortality n=2, life impact n = 27, physiological/clinical n=17,resource use n= 2 and adverse events n=1)[17]. The list of outcomes was randomised by domain in the online Delphi process.

Online Delphi process

Three hundred and ten participants completed both R1 and R2 of the online Delphi survey. Participants comprised 378 patients/carers and 53 health professionals (Table 3).

Table 3.

Round 2 completion rates

	Number of participants (%)
Total number of participants invited to R2	433
Patients and carers invited to R2	378
Health professionals invited to R2	53
Total completing R2	310 (71)
Total patients and carers completing R2	259 (68)
Total Health Professionals	51 (96)

At the end of R1 13 outcomes had reached the definition of “consensus in” with 70% or more of participants in both stakeholder groups rating the outcome 7-9. One hundred and eighteen responses were received to the free text question that asked participants if there were any outcomes they thought were missing from the list and should be added. The free text relating to additional outcomes was reviewed by the SSC. Sixteen free text additional outcome responses and one feedback comment related to “extra pulmonary organ impairment” and this was included as an outcome in R2. The remaining 102 responses were excluded as they either did not represent an outcome i.e. were related to “how” an outcome should be measured (n=37), were not related to pulmonary sarcoidosis (n=6), or were already included in an existing outcome (n=59).

Feedback provided by participants was also reviewed for potential outcomes. The feedback included one comment related to relapse “at present I am in remission but worry about relapse” and the SSC agreed that this should be included as an additional outcome in R2 “Relapse: sarcoidosis coming back after a period of remission”.

At the end of R2 the definition of consensus was applied to the responses for each stakeholder group (Supplementary file 2 – consensus matrix). Fifteen outcomes met the definition for “consensus in” and are in COS. This included the 13 outcomes that had reached “consensus in” in R1 plus “death from any cause” and “extra pulmonary organ impairment” the latter of which was only rated in R2. Six outcomes met the definition of “consensus out” and were excluded from the core outcome set, the remaining outcomes had no consensus. The overall attrition rate between rounds was 28%, the rate of attrition was higher for patients (42%) compared to health professionals (4%) (Table 4).

Table 4.

Attrition between R1 and R2

Stakeholder	Number registered (% of total registrations)	Completed R1 n (% of registrations)	Number of participants invited to R2	Completed R2 n (% of completed R1 and invited to R2)
Patients with pulmonary sarcoidosis or their carers	479	391 (82)	380	259 (68)
Healthcare professionals	61	54 (89)	53	51 (96)
Total	540	445 (82)	433*	310 (72)

*This figure takes into account participants who could not be reached because of mail delivery failures (n=7) or who based on the comments that they provided in R1 were not eligible to take part because they did not have pulmonary sarcoidosis (n=3)

The impact of attrition between rounds was assessed by comparing the average R1 scores of those who did not complete R2 against the distribution of scores for those completing both R1 and R2. Overall the average scores of participants completing R1 only were contained within the average scores of those completing both R1 and R2 (supplementary file 5).

Consensus meeting

All those who expressed an interest in attending the consensus meeting and had completed both R1 and R2 (n=39) were given further meeting information including the date of the meeting. Twenty five participants (7 health professionals, 17 patients), confirmed they were able to attend. Based on the number who responded, no restrictions were put in place on attendance. However, to try an increase the number of health professionals attending, an additional email invitation was sent to all health professionals completing R1 and R2. (Table 5).

Table 5.

Consensus meeting participants

	N (%)
Healthcare professionals	7 (100%)
Role
Sarcoidosis specialist	3 (43%)
Researcher in the field	1 (14%)
Industry representative	3 (43%)
Country of residence
United States	5 (71%)
India	1 (14%)
The Netherlands	1 (14%
Patients with pulmonary sarcoidosis	17 (100%)
Country of residence
United States	14 (82%)
UK	3 (18%)

Seventeen outcomes, that had not reached consensus, were prioritised for discussion at the consensus meeting as either 70% or more of participants in one stakeholder group, or 50-69% of participants in both stakeholder groups, had rated the outcome 7-9.

For the purpose of the consensus meeting the 17 outcomes, prioritised for discussion, were grouped into four domains, physiological/clinical (8 outcomes), health and quality of life (3 outcomes), life impact (5 outcomes) and treatment (1 outcome). All outcomes in a particular domain were presented, alongside those outcomes already included in the COS, and participants of the meeting invited to provide comments for inclusion of outcomes followed by comments against. After discussion of outcomes in that domain participants rated each outcome, that had been discussed, using the 1-9 scale (1 not that important – 9 critically important). Patients and health professionals voted separately, for an outcome to be included in the core outcome set 70% or more of participants in both groups were required to give a rating of 7-9. The results of consensus meeting ratings are provided in Table 6 and a full meeting report is available in supplementary file 6.

Table 6.

Summary of outcome discussed and rated during the consensus meeting

Domain	Outcome		% patients rating 7-9 in online Delphi	% HCPs voting 7-9 in online Delphi	% Patients voting 7-9 in consensus meeting	% HCPs voting 7-9 in consensus meeting	Result
Physiological/clinical	Cough	Cough	70	59	70%	57%	Not included in the COS
	Fatigue	An overwhelming, sustained sense of extreme tiredness or lethargy resulting in physical and/or mental weariness	88	61	80%	14%	Not included in the COS
	Recurrence of sarcoidosis	Sarcoidosis coming back after a period of remission	86	69	50%	29%	Not included in the COS
	Systemic inflammation	An immune response resulting in inflammation in the whole body, including the lungs	92	54	Not discussed or rated	Not discussed or rated	Not included in the COS
	Pain	A feeling of noticeable discomfort or an unpleasant physical sensation, in general, experienced anywhere in the body.	73	29	Not discussed or rated	Not discussed or rated	Not included in the COS
	Chest pain	Pain specifically in the chest	78	27	Not discussed or rated	Not discussed or rated	Not included in the COS
	Mobility	Includes ability to walk, climb, run, stand, sit without difficulty, taking into account stiffness.	78	45	Not discussed or rated	Not discussed or rated	Not included in the COS
	Infection	The presence of an unexpected and unwanted virus, bacteria, fungus or mycobacteria anywhere in the body	79	26	Not discussed or rated	Not discussed or rated	Not included in the COS
Quality of life/general health	Overall quality of life	A state of health, happiness, comfort and well-being	88	67	47	0	Not included in the COS
	General Health	Someone’s general health	74	45	44	0	Not included in the COS
	Perceived health status	How someone thinks their overall health is	63	55	25	0	Not included in the COS
Life impact outcomes	Activities of daily living	Being able to take care of oneself and complete usual daily activities/tasks, for example, eating, bathing and dressing oneself	83	69	58	17	Not included in the COS
	Ability to work or study	Someone’s ability to work or study, ability to gain or keep employment, influence of pulmonary sarcoidosis on the type of job or study undertaken, time off from work or study, impact on career.	78	63	17	17	Not included in the COS
	Ability to undertake usual role/responsibilities	Being able to manage personal role and responsibilities	75	49	33	17	Not included in the COS
	Ability to take part in usual family life/activities	Being able to take part in family life and activities	70	43	42	17	Not included in the COS
	Cognitive Function	Mental abilities/mental processes, including memory, concentration, language and thinking.	80	45	67	0	Not included in the COS
Treatment outcomes	Satisfaction with treatment	How satisfied someone is with the treatment/s they are receiving including satisfaction with its effectiveness (how well it’s working) and the time spent on treatment.	85	49	50	0	Not included in the COS

Feedback forms from the meeting were completed by 4 (57%) health professionals and 15 (88%) patients. Overall meeting participants were satisfied with the information provided before and during the meeting, with the meeting facilitation and opportunities to contribute to the meeting, the meeting length and format, and that the meeting produced a fair result. Free text feedback included the desire for a greater number of health professional participants in the meeting and a wider geographical range of participants. The meeting was conducted using the Zoom platform and although participants overall were satisfied with the use of Zoom the free text feedback was mixed about the desire to have as short as meeting as possible whilst also having more time to allow for a longer discussion and the challenges of having a longer online meeting. One participant also commented on the challenge of following the chat discussion alongside the verbal discussion.

Discussion

The FSR-SCOUT study has developed a COS for pulmonary sarcoidosis with consensus from both patients and health professionals. Although the opinions of a large number of patients has contributed to the consensus process these, like the health professionals, were predominantly from the United States. The geographical location of participants is largely due to the areas covered by the patient and health professional organisations who distributed the invitations to take part. Although some of these had an international reach, particularly for health professionals, global uptake of the invitation to the Delphi survey was low. Ratification of the COS and further engagement with international patient and health professional organisations may be helpful to confirm the importance of the outcomes to stakeholders with differing cultures and experiences of healthcare.

Previous work to identify important outcomes for pulmonary sarcoidosis research has involved a single stakeholder group and in one case considered only a small specific set of outcomes. Nevertheless there is overlap in the outcomes in the current COS with the majority of outcomes recommended by Baughman et al and Kampstra et al[18, 9, 8] (supplementary file 7). The exception being “imaging” and imaging components of “clinical outcome status” (chest X-ray scanning and High-resolution computed tomography (HRCT) score[18]). In the current study these outcomes were grouped and rated in the Delphi survey as “radiographic outcomes” but, after R2, this outcome did not meet the criteria for “consensus in” in either of the stakeholder groups. Judson et al have also proposed overarching endpoints for trials of treatment for acute pulmonary sarcoidosis, chronically treated pulmonary sarcoidosis and fibrotic pulmonary sarcoidosis. These include improvement/resolution of granulomatous inflammation, improvement/worsening in pulmonary physiology/function, improvement/worsening in function status or QOL or both, and reduction in side effects of treatment which are included within the outcomes within the proposed COS.

Despite the large overlap in outcomes identified by the different initiatives, the inclusion of both patients and health professionals in the current study has identified additional, critically important, outcomes relating to treatment, disease progression and symptoms that have not previously been prioritised, highlighting the importance of integrating stakeholder opinions in the development of the COS.

The study included a consensus meeting to discuss outcomes that had not reached consensus during the Delphi process. An opportunistic sample of participants expressing an interest in attending formed the basis of the consensus meeting and resulted in twenty-four consensus meeting participants of which only seven were health professionals. This small number of health professionals may mean that a smaller range of views and experiences were represented. However, the average R2 scores of health professionals attending the consensus meeting were similar to the group average for all outcomes (5.8 and 6.5 respectively) and also similar for individual outcomes of cough (6.6 consensus meeting participants and 6.6 all participants) and fatigue (5.4 consensus meeting participants and 6.1 all participants). This was also true for patients (supplementary file 6).

Consensus has been reached on the inclusion of 15 outcomes, yet there were a number of other outcomes, that met the definition of “no consensus” and that patients rated as “consensus in” and health professionals did not These outcomes were discussed and voted on at the consensus meeting. Two of the outcomes, “fatigue” and “cough” continued to meet the definition of no consensus (table 1). Both fatigue and cough are frequently reported symptoms of pulmonary sarcoidosis. In the literature, exploring patient experiences of sarcoidosis, fatigue was reported by 90% of respondents [19] and in one study it was reported as the most disabling symptom by 40% of sarcoidosis patients [20]. Likewise cough is often frequently reported and may affect up to 53% of patients [21, 22] and both fatigue and cough have been reported to impact on the quality of life of patients with sarcoidosis [23, 24] . Existing pulmonary specific HRQL measures such as the St George’s Respiratory Questionnaire, Sarcoidosis Health Questionnaire[25] and the Kings Sarcoidosis Questionnaire [26] include some items relating to cough, and tiredness/fatigue. Consensus is now needed on how each of the outcomes in the core outcome set should be measured and we recommend that the outcomes “fatigue” and “cough” be taken into consideration when agreeing how to measure health related quality of life.

Conclusions

The COS developed in the FSR-SCOUT study has identified outcomes considered to be the most important, and critical to measure in future research for pulmonary sarcoidosis, by both patients and health professionals. The uptake of the COS will increase the relevance of research outcomes to key stakeholders and the potential for comparisons to take place across trials, thereby reducing waste in research.

Funding:

This study was funded by the Foundation for Sarcoidosis Research (https://www.stopsarcoidosis.org/). The Funder was an active member of the Study Management Group; they facilitated organization of meetings with the Study Steering Committee, contributed to the study design and reviewed the manuscript.

Conflicts of Interest:

NLH, SLG, PRW, RPB, MAJ, NAK, DEV, JCG, DAC have no competing interests. ESB is an employee of Janssen Research and Development, LLC, who has sponsored research studies in Sarcoidosis. EJS is an employee of Insmed Incorporated. MW employee of Janssen Research & Development. HJ is a sarcoidosis patient. TA-H, HN and NS were employed by the funder (The Foundation for Sarcoidosis Research) during the research.