Proposed categories for reporting of service organization in rehabilitation in clinical trials: a discussion paper

Abstract

BACKGROUND

The lack of standardized reporting for crucial organizational factors in rehabilitation poses a significant barrier to understanding their impact on patient outcomes in clinical trials and meta-analyses.

AIM

Based on the categories in the International Classification of Service Organization in Rehabilitation (ICSO-R 2.0), we aimed to develop reporting standards specifically for organizational factors in clinical trials.

METHODS

A comprehensive two-step process was conducted. In Step 1, important categories were identified. The identification was based on previous results from a Delphi survey with international stakeholder participation, two systematic literature reviews and results from focus groups with users in Germany, Indonesia and Norway. Step 2 involved the necessary reduction of categories and the proposal of reporting specifications, achieved through two voting rounds among key researchers, stakeholders and users.

RESULTS

The suggested minimum reporting set comprises Context and Setting as well as Quality assurance and management. The Context and Setting is proposed to include whether the intervention is delivered by Hospital, Community or Other service providers. The Mode of delivery is proposed to be specified as Inpatient, Outpatient, In-home, or Tele-rehabilitation. Furthermore, the Level of specialization (Primary/Secondary) and the Phase of service delivery Acute, Subacute or Long-term rehabilitation services should be reported. The Quality assurance and management should be reported as Yes or No, with the option Yes requiring description of the quality assurance applied in the methods section.

CONCLUSIONS

This study proposed a compulsory and standardized reporting of organizational factors in clinical trials to facilitate the generation of scientific evidence regarding effective service provision and delivery in rehabilitation medicine. Authors are encouraged to consider the proposed reporting set to testing, criticism, and modification to enhance its applicability and robustness.

The increasing number of people living with disabilities after diseases and injuries calls for capacity building, as well as evidence based and effective rehabilitation services.^{1, 2} The number of clinical trials evaluating rehabilitation programs with improved methodological approaches has strengthened the scientific basis for rehabilitation over the last decades.³ Guidelines for methodological approaches and reporting have been important for this development. Examples are the Template for Intervention Description and Replication (TIDieR)⁴ and Consort guidelines⁵ along with the Cochrane guidelines initiative adapted to rehabilitation.⁶ Hence, the participants and target groups, their demographic characteristics, design and statistical approaches are generally well described and can be aggregated and compared across studies in reviews and meta-analyses.

This development in rehabilitation research is also facilitated by the universal acceptance of the International Classification of Function, Disability and Health (ICF).⁷ The application of ICF has contributed to a better conceptual understanding of functioning and disability and contributed to a standardized description of the target groups and their challenges in functioning. The importance of contextual factors has also been highlighted in using ICF. However, the ICF mainly describes the individual, i.e., the micro level.⁸ Contextual factors may equally well represent the meso level, i.e., organization of the rehabilitation services.⁹ Systematic descriptions of service provision and delivery aspects are often lacking in clinical trials, and the terminology varies.¹⁰ Hence, these aspects of service provision and delivery are seldom systematically included in the analyses of the clinical trial, nor included in meta-analyses, even though such factors could significantly impact the outcome of the interventions.^{11, 12} Reviews and meta-analyses failing to recognize the confounding of organizational factors and their impact on outcomes across different rehabilitation interventions limit the validity of knowledge summaries and represent a barrier to the successful implementation of rehabilitation interventions.¹³

Based on the lack of a framework to systematically describe the organization of rehabilitation services, the International Classification of Service Organization in Rehabilitation (ICSO-R 2.0) was developed. It consists of two dimensions, the Provider and Delivery dimensions, comprising 23 categories.¹⁴ ICSO-R 2.0 was developed to provide a tool for describing rehabilitation service organization across different countries and health systems. This framework can be used for various purposes, such as reporting contextual influences in clinical trials, comparing rehabilitation services in health systems and supporting quality management of services. Yet, it is too extensive to be applied as standardized information in clinical trials and the categories are too broad for analytical purposes. Consistent reporting of predefined value sets for limited number of categories in the publications is a necessity for application in reviews and meta-analyses.

Hence, the aim of the present study was to identify the most relevant categories of the ICSO-R 2.0 for a minimum reporting data set including existing evidence, stakeholders and users’s perspectives. Furthermore, we aimed to propose value sets for reporting in clinical trials and meta-analyses.

Materials and methods

A two-step approach was taken with identification of the most important categories from ICSO-R 2.0 in Step 1 and reduction of categories and elaboration of reporting specifications in Step 2.

Step 1 comprised identification of the most important categories in ICSO-R 2.0. The identification was based on a previously conducted Delphi survey among international clinicians and stakeholders.¹⁵ In the Delphi survey, participants ranked the six most important categories in the Provider dimension and the eight most important categories in the Delivery dimension.¹⁵ The identification was also based on two systematic literature reviews (one topic review¹⁶ and one review of randomized controlled trials¹⁷). Additionally, we undertook focus groups with users in Germany, Indonesia and Norway nominating the most important categories of service provision and delivery from the perspective of the end users (Table I).

Table I. —Categories in the service provider and delivery dimensions.

Code	Category	Topic review	RCT review	Delphi Survey ranking	Focus groups
Provider dimension
1.1	Context	x	x	2	x
1.2	Ownership			6	x
1.3	Location of provider	x		5
1.4	Governance/leadership				x
1.5	Quality assurance and management			4	x
1.6	Human resources	x	x	1	x
1.7	Technical resources	x		3	x
1.8	Funding of provider		x		x
1.9	Other
Delivery dimension
2.1	Health strategies
2.2	Service goals	x			x
2.3	Target groups	x	x	1	x
2.4	Mode of referral	x		6	x
2.5	Location of service delivery	x	x	5	x
2.6	Facility	x	x	7	x
2.7	Setting (level, mode and phase)	x	x	4	x
2.8	Integration of care	x			x
2.9	Patient centeredness	x			x
2.10	Aspects of time and intensity	x	x	3	x
2.11	Rehabilitation team	x	x	2	x
2.12	Reporting and documentation	x	x	8	x
2.13	Funding of service delivery	x			x
2.14	Other

Categories of importance emerging from the reviews and focus groups are marked with x. In the Delphi survey, the six most important categories in the Provider and the eight most important categories in the Delivery dimension were ranked from the most to the least important category (i.1. 1 to 6 in the Provider and 1 to 8 in the Delivery dimension).

No ranking of categories was performed in the focus groups or reviews. The ranking of the categories from the survey, along with the identified important categories from the literature reviews and the focus groups were summarized in a table and used as a basis for Step 2.

Step 2 included two voting rounds and an iterative process defining the value sets based on the descriptive options in ICSO-R 2.0. The first round involved 17 participants representing researchers, rehabilitation professionals, stakeholders (N.=15) (including medical doctors with specialty in physical and rehabilitation medicine, and nurse, physiotherapist, occupational therapist, psychologist, all with background in rehabilitation), and user representatives from patient organizations (N.=2) from four countries. The participants were recruited through the contact network of users, researchers and health professionals of the Research Centre for Habilitation and Rehabilitation Models and Services (CHARM). The second voting and iterative process included the same participants except for the two user representatives. The reason for this was that this process involved firsthand knowledge of other guidelines for reporting of clinical research as well as skills in English language. However, the two users were included in the final elaboration of the manuscript.

In the first voting, the participants were presented with ICSO-R 2.0. Given the need for reduction of categories, they were instructed to nominate eight key categories (three from the Provider and five from the Service Delivery dimension). Based on the experience from step 1 with a high number of retained categories when not restricting nomination, the maximum number of categories in each dimension was determined in order to obtain a reduction of 2/3 of the categories. The reduction of 2/3 was based on the experience from the minimum ICF Core set development¹⁸ and the goal to implement the reporting in the limited available space in manuscripts.

Value sets based on the category description by Gutenbrunner et al.¹⁴ were elaborated for the three categories in the Provider and the five categories in the Delivery dimension, receiving the highest number of votes. Value sets that could be reported with a minimum number of distinct and mutually independent values meaningful for statistical analysis were proposed. Before the second voting process the participants were presented with a written description of the categories, a proposal for the value sets and an instruction to cross check the categories regarding overlap with the CONSORT and Tidier reporting specifications.^{4, 19}

In the second voting, the participants were instructed to vote for retaining or removing categories, along with specification of their value sets. All participants were also encouraged to give comments to the value sets during the voting. Finally, a minimum reporting set was elaborated based on the categories where two thirds or more of the participants agreed on their meaningfulness and reporting options.

Results

Step 1

The ranking of the categories from the Delphi survey, and the categories indicated to be of importance according to the reviews and focus groups, are summarized in a table (Table I) and presented to the participants before voting in Step 2.

Step 2

The categories included in first the voting round rand the percentage of votes they received are presented in Table II.

Table II. —Categories included in the Step 2 voting process, and percentages of votes received.

Category		Votes	Category		Votes
Provider dimension			Delivery dimension
1.1	Context*	77%	2.2	Service goals	29%
1.2	Ownership	6%	2.3	Target groups*	35%
1.3	Location of provider	18%	2.4	Mode of referral	18%
1.4	Governance/leadership	47%	2.5	Location of service delivery	29%
1.5	Quality assurance and management*	53%	2.6	Facility	12%
1.6	Human resources*	59%	2.7	Setting*	82%
1.7	Technical resources	12%	2.8	Integration of care*	71%
1.8	Funding of provider	29%	2.9	Patient centeredness*	71%
			2.10	Aspects of time and intensity	24%
			2.11	Rehabilitation team*	77%
			2.12	Reporting and documentation	0%
			2.13	Funding of service delivery	18%

*The three categories in the provider dimension and the five categories in the delivery dimension receiving the highest number of votes.

The three categories from the Provider dimension and the five categories from the Delivery dimension with highest percentages of votes were included in the second voting process (Table II) and were presented with proposed value sets for the second voting.

Second voting

Provider dimension

Context: according to ICSO-R 2.0, Context describes whether the provider is independent or embedded in a parent or larger organization, and how the context is organized. Inclusions: Independent unit, hospital, university, community, network of organizations, or another umbrella organization.

Proposed value sets: “Hospital,” “Community” and “Other.” Hospitals would include rehabilitation hospitals/centers at all levels and somatic hospitals including rehabilitation units. “Community” would refer to the services provided by the local administrative health unit, and “Other” will include private or public providers of health services for example private or public organizations.

Results from the second voting: 100% of the votes allocated for inclusion in the minimum reporting set.

The value “Other” was suggested to be removed by some of the participants. “In home” was discussed as a potential additional value. As in home cannot characterize the Provider and is included in the Mode of Service Delivery, this suggestion was not implemented.

Human resources: ICSO-R 2.0 defines Human Resources as the Spectrum of staff/personnel (different types of health professionals, administrative staff, technical staff, researcher, and other personnel) within the provider.

Inclusions: full-time-equivalents of staff, affiliated and supportive staff, regular volunteers.

Propose value sets: no value set proposed.

Results from the second voting: 100% of votes allocated for rejection. Arguments for rejection were that authors are already obliged to report this in accordance with existing guidelines and that statistically meaningful value sets are difficult to define.

Quality assurance and management: within ICSO-R 2.0, this category refers to Activities and programs, promoted by the owner or provider, intended to assure or improve the quality of service delivery. Inclusions: Assessment or evaluation of the quality of service delivery, identification of problems or shortcomings in service delivery, designing activities to overcome these deficiencies, and follow-up monitoring to ensure effectiveness of corrective steps. Any systematic way to pursue quality assurance activities (internal and external), including accreditation/certification or audit; appointed quality manager; single interventions with the explicit aim to improve structure/process/outcome quality.

Proposed value set: “Yes” and “No.”

Results from the second voting: 73% of the votes advocated for inclusion in the minimum reporting set with the proposed value sets. It was emphasized that an instruction for authors needs to be elaborated regarding the Minimum reporting set. In this instruction, it should be clearly stated that a description of the quality assurance and management would need to be described in the methods section in order to qualify for a “Yes.”

Delivery dimension

Setting refers in ICSO-R 2.0 to three subcategories: Levels of care, Mode of service delivery, and Phase of healthcare under which rehabilitation interventions take place.

Subcategory Levels of care refers to the degree of specialization of care provided by rehabilitation health professionals. Including primary, secondary, and tertiary levels of specialization.

Proposed value sets: “Primary level of specialization” and “Secondary level of specialization (including tertiary level of specialization if existing).”

Results from the second voting: 87% voted for inclusion and supported the value set.

Subcategory Mode of service delivery refers to the way services are delivered to the users including inpatients, outpatients day hospital/service, home and community, tele-rehabilitation and any other setting for service delivery.

Proposed value sets: “Inpatient,” “Outpatient,” “In-home” and “Tele-rehabilitation.”

Results from the second voting: 93% voted for inclusion and supported the value set.

Subcategory Phase of health care refers to refers to the types of rehabilitation services responding to patients’ needs in different phases of their health conditions. Inclusions: habilitation, pre-habilitation, acute rehabilitation care, sub-acute rehabilitation care, post-acute rehabilitation care and long-term/chronic rehabilitation care.

Proposed value sets: “Acute,” “Subacute and post-acute” and “Long-term” rehabilitation services.

Results from the second voting: 93% voted for inclusion.

The value set was supported. The participant who voted against argued that this category overlapped with existing requirements for reporting of the intervention.

Rehabilitation team: defined as professions and competencies of rehabilitation team members, team structure and methods of team communication. It includes health and health-related professionals delivering services to users (patients), peer counsellors, and others, multi-professional team composition, interdisciplinary way of working, etc., patients as part of the rehabilitation team and team supervision, counselling, etc.

Proposed value sets: no proposal.

Results from the second voting: 87% voted for rejection due to lack of uniform and meaningful reporting specifications.

Those not rejecting the category provided no clearly defined value sets for reporting different from the already existing guidelines for reporting clinical trials.

Patient-centeredness: defined as rehabilitation tailored on the person’s needs and provided in partnership with them, their families and communities. Inclusions: shared decision-making, individual rehabilitation plan, patient, family or other caregiver’s education and empowerment, patient family or other caregiver’s integration in the rehabilitation process, involvement of peer counsellors, involvement of patients as prosumers.

Proposed value sets: “Yes” or “No.”

Results from the second voting: 53% voted for inclusion.

Those voting for inclusion argued for letting Yes represent interventions describing clearly defined collaborative rehabilitation plans.

Integration of care: defined as the management of rehabilitation services in conjunction with other health services so that people receive timely, comprehensive and well-coordinated care, according to their needs and across different levels (vertical integration) and along the continuum of care (horizontal integration). These include continuum of care, admission and discharge planning, collaboration of health professionals, and shared electronic patients’ records.

Proposed value sets: “Yes” or “No” with emphasis on including integration of care aspects in the description of the interventions in the trials.

Results from the second voting: 60% voted for rejection. The reasons for rejection were difficulties to provide universal criteria relevant across different rehabilitation interventions defining the “Yes” value.

Target groups: groups of people with rehabilitation needs and their caregivers for which the service is delivered. These include patients with any or specific health condition(s), persons with any or specific impairment, activity limitations and/or participation restrictions, and other target group(s), such as age-related groups, formal or informal caregivers.

Proposed value sets: health conditions.

Results from the second voting: 87% voted for rejection. The arguments were that this category already is included in existing reporting guidelines.

Those voting yes did not provide value sets distinctive from existing reporting guidelines.

The final proposal for compulsory reporting of organizational factors in clinical trials, which includes two categories from the ICSO-R 2.0 Provider and one category from the Delivery dimension, is provided in Table III.

Table III. —Proposal for compulsory reporting of organizational factors in clinical trials.

Categories	Value sets
Context and setting	Hospital/community/other
	Inpatient/outpatient/in-home/tele-rehabilitation
	Primary level of specialization/secondary level of specialization
	Acute/subacute/long-term rehabilitation services
Quality assurance and management	Yes/no

Discussion

This paper outlines the development of a test-version of organizational factors that we recommend to report in clinical trials in the field of rehabilitation medicine. The development of this minimum reporting set was informed by a comprehensive approach, involving previous literature reviews, Delphi rounds, focus groups with users. In the current study, voting processes and consensus among experts were also undertaken. The final set includes the Context and Setting of services and Quality assurance and management.

The present study was based on a multimethod approach aligned with the methodological approaches applied in the ICF Core Set development.²⁰ ICSO-R 2.0 is a comprehensive classification of service organization in rehabilitation, encompassing 21 specified and two optional free text categories. In the previously conducted Delphi survey, the health professionals ranked these categories regarding their relevance to study outcomes, distinctiveness across different rehabilitation settings and feasibility to report in clinical trials.¹⁵ While slightly different categories emerged from the two literature reviews, it is crucial to recognize that patient perspective lies at the core of rehabilitation.²¹ This perspective is now implemented into the definition of rehabilitation for research purposes.²² User involvement in the rehabilitation services framework is a necessity to ensure the relevance and validity as well as to promote its use in practice.²³ Hence, supplementary focus groups were conducted to provide foundation for the development of a minimum reporting set in this study.

ICSO-R 2.0 as well as the minimum reporting sets aims to be applicable across different countries, continents and health systems. The Delphi survey covered an international sample and we included users from three countries with very different health systems. The Norwegian system represents the Scandinavian model with public access to rehabilitation services independent of work status or insurances.²⁴ In the German system, most people are covered under statutory health insurance. The insurance systems and funders of rehabilitation are the Health Insurance (medical treatment, rehabilitation and sick pay), the Accident Insurance (work related medical treatment and rehabilitation and occupational disability pension), Pension Insurance (pension, disability pension and rehabilitation), unemployment insurance (unemployment and employment promotion) and Care Insurance (nursing care costs). Indonesia has a national health system including rehabilitation, but with much lower coverage of specialized rehabilitation in the hospitals compared to the systems in Germany and Norway.²⁵ With this inclusive process, the majority of the categories in ICSO-R 2.0 were considered important. Hence, a process reducing the categories was clearly needed to define a minimum reporting set that could be included in clinical trials. Looking to the brief Core sets in ICF,²⁰ we aimed to reduce the categories to one third, constituting eight categories. We deemed eight categories feasible to include as a compulsory part of the methods section of clinical trials. Furthermore, overlap with existing reporting standards should be avoided due to the burden for authors as well as clarity of the publications. Finally, a minimum reporting set will be most useful for analytical purposes²² if the categories have predefined and mutually independent value sets.²⁶ Given the broad description of the categories in ICSO-R 2.0, there was a clear need for further specifications of the reporting options.

As can be seen from the voting processes in Step 2, the Human resources category was voted as one of the three most important categories. However, we deemed that this category partially overlaps with existing reporting guidelines, which emphasize reporting of personnel involved in the interventions.⁴ Furthermore, meaningful value sets for this category would be difficult to determine. Consequently, we opted to exclude the Human Resource category. Similarly, the rehabilitation team in the Delivery dimension was excluded. In this dimension, Target group was also excluded due to overlap with Consort reporting guidelines.⁵ Patient centeredness along with Integration of care were voted to be among the five most important categories, yet the group had diverging opinions regarding the possibility to develop meaningful value sets for these categories. The Patient centeredness may also be deemed as an inherent part of the rehabilitation definition, and thus overlap with the intervention description.²² For the Integration of care, it may be difficult to create a small number of universal value sets being meaningful across every clinical trial. Yet, we emphasize the need to report this category when relevant for the intervention, as indicated by our literature review, which suggests its potential to influence on patient outcome.¹⁷ Furthermore, location of the services along with financial system for the rehabilitation services may well influence the patient selection, content as well as outcome of the services.²⁷ The funding scheme of rehabilitation services is very different across countries with a highly variable a mix of private, public, insurance and out of pocket payment at both the Provider and Delivery levels. We believe that there is a need to define consistent and clear categories for the funding of the services applicable in clinical trials, which warrants further emphasis and elaboration internationally.

The ICSO-R 2.0 is developed to capture important organizational factors at the meso level of rehabilitation services.¹⁰ However, for the minimum reporting set we recommend that the reporting of value sets for the categories should relate to the actual intervention(s) delivered. Authors should also be required to specify differences between the intervention arms regarding value sets of the categories. For example, a clinical trial evaluating early supported discharge after stroke delivered by health professionals from a Department of physical and rehabilitation medicine in a regional hospital within the patients’ home environment, the Provider Context is Hospital. The Setting and Mode of Service delivery would be In Home and Levels of care would be Secondary level of specialization. If the control intervention comprises services provided to the patient by their general practitioners (GPs), and the patients visit the GPs and community physiotherapists, the control arm should be reported with Community, Outpatient, Primary level of specialization.

Authors should be instructed to include a section termed Context, Setting and Quality assurance at the beginning of their methods description and choose one option for each of the categories above when reporting. The health condition of the target group should also be reported if not already integrated in the population description. Furthermore, authors should be encouraged to describe the services in more detail and apply standardized terminology for replication or implementation purposes.

Strengths and limitations of the study

Potential strengths of the study lie in its integrated approaches, including expert opinions through the Delphi survey, insights from systematic literature reviews, and user perspectives from focus groups, therefore providing a comprehensive view of the categories deemed important in the context (Table I). Step 2 further applied a structured approach, involving two rounds of voting and meticulous evaluation criteria, ensuring a thorough and consensus-driven process in selecting the most relevant and meaningful categories (Table II). Incorporating international perspectives and experts with diverse professional backgrounds add richness to the selection process. However, several limitations should be acknowledged. First, the recently published ICSO-R 2.0 has not been fully implemented, and potentially impacts the study’s findings. Moreover, the majority of testing for ICSO-R 2.0 has been undertaken in European countries, possibly limiting the generalizability of the results. Although seeking to include a specter of researchers and health professionals, both social workers and speech therapists were lacking. Finally, a larger scale consensus process of the category reduction and value assessment has not been undertaken. It is our contention that refining the minimum reporting set necessitates in-depth knowledge of clinical research and analyses, as well as familiarity with the organization of rehabilitation services and ICSO-R. Furthermore, the decision on one value set without overlapping categories, may have flawed the selection. For example, Integration of care could have been described with several different value sets i.e., presence of (admission) discharge process planning or not, communication and information transfer between the care settings procedures or not, and patient and family/caregiver involvement in care strategies or not. Such factors have shown to impact quality of care,^{28, 29} but would have expanded the minimum reporting set. This decision may be questioned, and it is imperative to underscore that this set is a test-set and encourage constructive criticism to successfully refine categories and value sets and possibly also decide on a brief and a more comprehensive version. We also urge the research society to discuss and refine the terminology in the field of rehabilitation sciences and include the users in these discussions. For example the use of patients versus persons or users’ needs to be discussed along with the importance of financing of the rehabilitation services. We also allocate for including more precise description of the services in future scientific publications.

Conclusions

The authors of this paper advocate for compulsory and standardized reporting of organizational factors that may impact on the effectiveness of rehabilitation interventions in clinical trials. The proposed reporting set is introduced as a test-set, with an invitation for rehabilitation researchers to scrutinize, criticize and contribute to its refinement, aiming to facilitate the generation of scientific evidence regarding effective service provision and delivery in rehabilitation medicine.