Development of the WHO STARS: A Tool for the Systematic Assessment of Rehabilitation Situation

Abstract

Objective

To present the development process of the World Health Organization (WHO) Systematic Assessment of Rehabilitation Situation (STARS).

Design

A conceptualization phase to establish its objective and identify the content for comprehensively describing rehabilitation in countries, a second phase to draft, and a third phase to refine the tool. Reviews of existing health system assessment (HSA) tools used in other areas of health as well as expert consultations occurred.

Setting

The WHO initiated the development of STARS because there is currently no comprehensive HSA tool for rehabilitation that supports stakeholders describing their country situation and identifying priority actions.

Participants

The WHO rehabilitation team, experts representing rehabilitation professions and from all WHO regions, and government and rehabilitation consumer groups.

Interventions

Conceptualizing, drafting, consulting, and reviewing of the WHO STARS.

Main Outcomes Measures

Development of a tool that is comprehensive and able to support countries to describe rehabilitation and identify priorities.

Results

STARS was developed in a participatory process, uses a logic model to structure the 50 components of rehabilitation selected for assessment, and assesses both capacity and performance.

Conclusions

STARS is the first HSA shaped to rehabilitation, has been developed by the WHO in a participatory process based on several expert consultations, and has the potential to meaningfully support governments to better understand the status of rehabilitation in their countries, define priorities for action to strengthen rehabilitation, and facilitate the monitoring of system level changes. Moreover, STARS information can be used in research to support evidence-informed policy and programs.

Globally, the population experiencing declines in functioning is rapidly increasing, mainly because of aging populations and the prevalence of noncommunicable diseases. The prevalence of health conditions associated with severe limitations in functioning increased by nearly 183 million between 2005 and 2017, and by 2050 over 22% of the world population will be older than 60 years.^1,2 Strengthening rehabilitation—the health strategy to optimize functioning—is therefore essential.³ Current estimates show that 2.4 billion persons have a health condition that benefits from rehabilitation⁴; the unmet need for rehabilitation is profound.⁵

Scaling up rehabilitation, irrespective of how weak current services are, should be pursued. For this, a structured and standardized overview of existing services is essential so that effective prioritization and strategic planning can take place.⁶ For example, in some countries rehabilitation is scarce because there is limited workforce and assistive products are not available. Also, frequently the leadership for rehabilitation is lacking and few government resources are allocated into rehabilitation.⁷ In other cases, there is a lack of information about rehabilitation needs; in some countries all the above apply. A health system assessment (HSA) is required for identifying priority actions.^8,9

Currently there is no HSA tool for rehabilitation. The World Health Organization (WHO) therefore initiated the development of the Systematic Assessment of Rehabilitation Situation (STARS) within the Rehabilitation 2030 initiative.¹⁰ Our objective is to present the development process of the WHO STARS.

Methods

The conceptualization phase had 2 purposes: (1) to establish the objective of STARS and (2) to identify the content for comprehensively describing rehabilitation in a country. In a second phase, STARS was drafted and then refined in a third phase after consultations and a preliminary field test in 1 country. The fourth phase of field testing is the subject of a forthcoming paper. An outline of these 4 phases, key steps, and developments and modifications to STARS is included in table 1. Although the phases are presented as discrete, in reality there was some overlap.

Table 1.

Phase, timeline, features, and modifications of STARS

Phase	Date	Key Steps	Features Developed or Addressed	Modifications Made as a Result of Consultation and Field Testing
1. Conceptualizing	July-December 2016	Search and content analysis of existing HSAs	Concept, objectives, and key features of STARS	Scope of rehabilitation assessed in countries was defined by the WHO definitions of rehabilitation and health systems. Inclusion of both capacity and performance measures was sought.
			Information collection template	Initial organizing of information under health system building blocks and drafting of template for data collection
	September 2016	1st consultation meeting (12 experts)	Objective of STARS	Objective was initially to undertake comprehensive assessment
			Rehabilitation Logic Model	Moved the system attributes from outcomes box to one that sits under the output and outcomes of logic model
				The output was initially “services across settings and population groups” and “services practices” altered to “services available” and “service quality”
				Identified different categories of “drivers” of change and surrounded the logic model in a larger ecological framework
			Information collection template	Added private sector in workforce data collection
2. Drafting	February 2017	2nd consultation meeting (34 experts)	Objective of STARS	Objective was sharpened to undertake comprehensive assessment that can inform rehabilitation planning processes
			Method for undertaking STARS	Confirmed that STARS process led by government and a rehabilitation technical working group should be created in countries
				Included flexibility and tailoring of methods and tool in countries
3. Consultation and preliminary field test	May 2017	Preliminary field test in Botswana (8 experts from country level)	Rehabilitation Logic Model	Terms were defined, ecological framework removed because of complexity and difficulty in separating out the drivers
			Method for undertaking STARS	Described method as a 4-step process
				Adapted a country rating exercise into a clearly defined grading system and created the Rehabilitation Maturity Model
				Altered the STARS country report format to include a final section that summarized strengths, priority areas for action, and recommendations.
	February 2018	3rd consultation: online email review (10 experts)	Rehabilitation Maturity Model	Revisions and definitions that clarified the descriptive phrases and terminology used in the maturity model
				Concerns regarding multiple aspects within components were raised; subcomponents were therefore created at this time, and the total number of components increased from 35 to 56
	June-July 2018	4th consultation: during face-to-face training and online review before and immediately after training (18 experts)	Rehabilitation Logic Model	Clarified that access (an output) was a result of availability, acceptability, and affordability and that accessibility leads to coverage (an outcome). Also, that because data for coverage is mostly unavailable, the descriptions regarding access are used to cautiously inform conclusions about coverage.
			Information collection template	Structure of information collected regarding services was streamlined with less focus on impairment categories
			Rehabilitation Maturity Model	Further revision of definitions and descriptive phrases and terminology used in the maturity model
				Expanded the definitions of the 4 grade levels
4. Field testing in 7 countries	August 2018-March 2020	Field tests in Myanmar, Sri Lanka, Solomon Islands, Jordan, Guyana, Haiti, Laos	Rehabilitation Logic Model	Removed the term “with financial protection” that was being used at the impact level, already addressed under accessibility
			Information collection template	Aligned data collection about services to the levels of service (eg, tertiary, secondary, primary) and delivery platforms (eg, inpatient specialized rehabilitation beds, community settings)
				Changed name of Rehabilitation Capacity Questionnaire to Template for Rehabilitation Information Collection because of confusion in countries that led to multiple people completing
			Rehabilitation Maturity Model	RMM was streamlined, reduced number of components from 56 to 50, and removed subcomponents because of feedback about its complexity
			Methods for undertaking STARS	Created substeps under each step and expanded the preparatory step with 6 substeps

Phase 1. Conceptualization

To define the content of STARS, an analysis of existing HSA tools was conducted. Existing tools were searched for in MEDLINE and Embase as well as in Google and Google Scholar. For MEDLINE and Embase the key terms “health care,” “health systems,” and “public health” were combined with “planning,” “assessment,” “analyses,” “tool,” and “review.” For Google and Google Scholar, the terms “health system assessment,” “situation assessment,” “situational analysis,” and “tool” were combined with terms reflecting major key health programmatic areas, such as maternal health. Additionally, the web pages of the World Bank, WHO, United Nations Children's Fund, and Global Fund were searched using similar terms.

Tools were considered if they met the following inclusion criteria: developed after 2005, used for whole health system assessment, used for assessment of specific health programmatic areas (“deep dive” assessment¹¹), used for assessments across multiple WHO health system building blocks,¹² and used for international use as a comprehensive assessment tool (neither short guidance nor capacity survey tool).

At the outset of the content analysis, input from WHO experts in HSA and the emerging guidance regarding HSAs¹³ indicated that a desirable feature was that it assesses a health system's performance in addition to describing its capacity. As the content analysis commenced, the inclusion of performance was confirmed because many HSAs assessed this. The considered tools were analyzed regarding the overall structure, components of health system building blocks including measures of capacity, criteria and measures used to assess performance, and guidance provided for the assessment method. Capacity was defined as resources available in the system, such as human, financial, or institutional, that enable action by responsible health system authorities; these are often quantifiable elements that inform status descriptions and enable comparisons across countries.¹⁴ Performance was defined as how well the systems components perform their functions and achieve their goals; for example, provide stewardship and achieve equitable health outcomes.¹⁵

On completion of the HSA content analysis, commonalities of items across tools were identified; for example, policies, workforce numbers, and these headings were organized to sit under the 6 building blocks or a performance category. Under the performance category a list of subcategories emerged, such as health outcomes, equity, and sustainability. These subcategories were identified through the HSA's assessment items and cross-checked with categories identified in other WHO frameworks that conceptualize health system performance, for example, the framework developed by the WHO Western Pacific that features the “key attributes of a high performing health system.”¹⁶ The WHO rehabilitation team then selected from these using the following 3 criteria: relevant to rehabilitation, adaptable to rehabilitation, and important for a well-functioning mix of rehabilitation services.

Phase 2. Drafting

Drafting of the STARS occurred by using the items that had been selected by the WHO rehabilitation team and adapting them to rehabilitation. In the beginning these items were placed into an information collection template that used the health system building blocks structure. The template focused primarily on the quantifiable information and data related to human, financial, and institutional resources, characterized as mostly capacity assessment items. A summary of these adapted items was categorized by building blocks and types of performance, and this was placed into an input, output, outcome, and impact logic model structure. The adoption of the logic model structure for the rehabilitation assessment was done to support the assessment of performance as it prompts the consideration of the outcomes and impact of rehabilitation at population level. The logic model incorporated the health systems building blocks at the input and output level, as well as types of performance across outcome and impact level (eg, health outcomes, equity). The method for the assessment process was also drafted.

Phase 3. Consultation and preliminary field test

Experts from all 4 consultation rounds were (1) rehabilitation professionals from different backgrounds, including rehabilitation medicine, physiotherapy, occupational therapy, speech and language therapy, prosthetics and orthotics, psychology, and in 2 rounds also users of rehabilitation services; (2) representatives from government, nongovernment agencies, and individual consultants; and (3) from high-, middle-, and low-income countries.

The objective, logic model, information collection template, and guidance for assessment method were revised with 2 rounds of feedback. After those revisions, it was tested in a first situation assessment in Botswana in May 2017. During this assessment it was recognized that facilitating government engagement in the assessment process would optimize their learning. This led to the exploration of HSA grading (scoring) systems in which governments participate and reflect on “where they are” within a context. These grading mechanisms also enable visual presentation of findings and can guide situation descriptions. The rehabilitation components identified for assessment were subsequently described across 4 levels on a maturity continuum, a system reflecting the “stages of growth,” and became the Rehabilitation Maturity Model (RMM).

The logic model, the information collection template, the maturity model, and the guidance for assessment method were evaluated in a third consultation (via email) with participants of the previous rounds. The fourth consultation (via email and face-to-face) included 18 WHO staff and individual consultants (rehabilitation professionals) who were trained for STARS field testing. Both rounds of consultation sought feedback through edits and comments on the documents (a rating exercise was not used at this stage). This feedback was consolidated and integrated into the revision of the tool by the WHO rehabilitation team. Based on this process a version for field testing was finalized.

Results

Phase 1. Conceptualization

The objective of STARS was to inform rehabilitation planning processes through a comprehensive assessment.

Seventeen HSA tools were identified and are listed in table 2; 14 were identified through Google and Google Scholar. Two tools aimed to assess the whole health system (tools 5, 11), 1 targeted national health plans (tool 8), 1 could be adapted to areas using a diagnostic approach (tool 6), and 13 were related to a specific programmatic area. Six tools were structured by key features of the health program/area (tools 4, 10, 14, 15, 16, 17); another 10 reflected the features of the health programmatic/area across the health system building blocks. All tools assessed capacity either explicitly or implicitly, and most considered components of performance with 4 doing so more comprehensively (tools 2, 5, 6, 11). Six tools incorporated methods to grade the situation assessment (tools 7, 9, 12, 15, 16, 17), scoring the extent of implementation, the system maturity, or the capacity to deliver. All tools included guidance for the assessment method of varying comprehensiveness, ranging from 1 page (tools 3, 14) to 30+ pages (tools 6, 10). Many tools provided definitions, frequently asked questions, and practical resources in annexes.

Table 2.

Health system assessments tools

No.	Tool Name	Agency	Year	Area of Health	Overview of Tool andLength of Guidance Provided for the Process	Structure to Assessment ToolKey Methods for Assessment of Capacity and Performance
1.	WHO Assessment Instrument for Mental Health Systems	WHO # Tool designed for global use	2005	Mental health services	Primarily a tool to guide data collection with additional information, it includes extensive definition of items and indicators with defined numerators and denominators. Includes a template for report writing. Guidance for the assessment process is 5 pages, including data collection & 3-page FAQ.	Structure is based on the health system building blocks.
						The capacity of the mental health system is explicitly assessed through the key indicators that are defined with numerators and denominators resulting in quantitative assessment as well as prompting descriptive analysis.
						Assessment of performance is implied through the indicators which reflect functions and outcomes. However, there is limited guidance for descriptive analysis or interpretation of indicator results; it is not characterized as a comprehensive performance assessment.
2.	A tool kit for rapid assessment of health systems and tuberculosis control; Systemic Rapid Assessment Toolkit	WHO # Tool designed for global use	2007	TB	Provides the guidance for assessment through questions to be answered. It builds on current TB program assessment practices, contextualizes them within health systems, and facilitates assessment of a program theory. Guidance for the assessment process is 17 pages, including annex 1 and 2.	Structure is somewhat based on health system building blocks.
						Assessment of capacity is embedded into building blocks with prompts for specific data collection and questions specifically exploring capacity to deliver on the program.
						Performance assessment occurs through the descriptive evaluation of the situation, with guiding questions prompting assessment of what is achieved (“in your view”) as well as questions regarding health service effectiveness and outcomes. Most elements of expected performance are reflected.
3.	Tool for assessing the performance of the health system in improving maternal newborn, child, and adolescent health	WHO EURO # Tool designed for use in EURO	2009	Maternal, newborn, child, and adolescent health	A tool for assessing the extent to which key functions and corresponding standards of the program area are implemented. The “standards” (essentially expected tasks/actions to be achieved/in place) are rated by a group based on their “implementation.” Guidance for the assessment process is 1 page.	The tool is structured by the functions of health system with the building blocks somewhat represented within these; the including services are integrated into this.
						The capacity within the system is not explicitly assessed but standards are provided (such as criteria to be met), and the assessment is based on the extent of implementation.
						The performance is assessed through the assumption that the level of implementation of the standards reflects how well the system functions. There is not a focus on the health outcomes; therefore, overall it is not characterized as comprehensive assessment of performance.
4.	WHO-IAEA National cancer control programs core capacity self-assessment tool	WHO # Tool designed for global use	2011	Cancer	A questionnaire designed to be completed by national teams through rating of items that predominantly reflect the system's capacity to deliver on expected roles/functions. Limited guidance for process and analysis of results. Guidance for the assessment process is 2 pages.	Structure is based on the program area.
						Assessment of capacity is embedded into the self-grading exercises through questions assessing the ability (or capacity) to deliver programs.
						Performance is not the focus of this assessment; a small number of grading exercises include broader health and system outcomes, such as questions about equitable access and efficient use of resources.
5.	Health System Assessment Approach: A How-To Manual version 2.	USAID # Tool designed for global use	2012	Whole health system	A tool that extensively guides the process of situation assessment, including definitions, indicators to assess against, information to support interpretation, and analysis. Guidance for the situation assessment process is 19 pages.	Structure is based on the health system building blocks.
						Assessment of the health system's capacity occurs through prompts to report specific data and describe capacity within context of each building block.
						Assessment of system performance is integrated into each building block by a series of defined indicators relevant to the building block capacity and functions. Health system performance also assessed against the following criteria: access, coverage, efficiency, equity, quality, safety, and sustainability. Finally, the impact is considered in terms of, responsiveness, risk protection, and improved health. A results chain is provided.
6.	Health System Rapid Diagnostic Tool	FHI 360 # Tool designed for global use	2012	Whole health system with a focus area	An adaptable tool that guides an assessment process that is focused on a selected health system function. The assessment process identifies strengths and weaknesses and guides a process of root cause analysis. The tool provides indicators (metrics) for assessment against the health system function, including guiding questions. Guidance for the assessment process occurs through a description of 17 steps over 5 phases, across 29 pages. It is intertwined with assessment items.	Structure is based on the health system building blocks and also includes a component on community.
						Assessment of the health system's capacity occurs through prompts to report specific data and describe capacity within context of building block functions.
						Assessment of performance occurs through describing the extent to which the health system achieves its stated functions as well as performance indicators that reflect core functions and health outcomes that are selected during the assessment process.
7.	Tool kit for assessing health system capacity for crisis management. User manual and tool kit.	WHO EURO # Tool designed for use in EURO	2012	Emergency	Primarily a capacity assessment questionnaire that includes a 3-level self-grading with prompts to justify grade and summarize the key findings. Guidance for the assessment process is 4 pages.	The tool is structured across the health system building blocks.
						Assessment of the capacity is focus of the tool; under each building blocks are key components and attributes for self-grading.
						Performance is not explicitly assessed; the assessment focuses primarily on the capacity to respond to crisis.
8.	JANS. Combined Joint Assessment Tool and Guidelines version 3. How to conduct a JANS based on country experience version 2.	IHP+ WHO World Bank # Tool designed for global use	2013	Whole health system, shaped by strategic plan itself	A tool that guides a review of a national health strategic plan. It does this through assessing against 5 essential ingredients or attributes of a sound national strategy. These 5 are: strategy based on clear situation analysis and identification of priorities; appropriate process through which plan is developed and endorsed; sound costs and budgeting of the strategy; sound implementation and management arrangements; and appropriate monitoring, evaluation, and review processes. Guidance for the assessment process is 20 pages; it is a separate document.	Structure based on 5 defined attributes of successful strategic plans.
						The assessment of capacity is implicit and integrated into overall review of the strategic plan.
						The review process analyses the strengths and weaknesses of a national health strategic plan against the 5 attributes and desired characteristics. In this way some elements of its performance are embedded, but this is not explicit nor characterized as a comprehensive assessment of performance.
9.	Tool to support countries to identify bottlenecks and solutions to scale up newborn care.	WHO # Tool designed for global use	2013	Newborn care	A questionnaire to be completed by national working groups whereby situation is described, bottlenecks are identified, and the effect of these on scaling up care is assessed. Incudes a summarizing self-grading question at the end of each sections. Guidance for the assessment process is 2 pages.	Structure of tool influenced by the 6 health system building blocks, including seventh area on community ownership and partnership. First part of tool is structured by building blocks and second part uses 9 key interventions of the newborn care, and under each the building blocks are crosscut.
						Assessment of the program area capacity is implicit in the questions asked, most of which request descriptions and explanations that may be backed up with data.
						Performance of program area is considered throughout and specifically in the self-grading questions that are based on how much action is required to achieve expected functions, but the focus is on bottleneck identification.
10.	UNAIDS sex assessment tool	UNAIDS # Tool designed for global use	2014	HIV	A tool that guides the assessment of the sex dimensions of both the HIV epidemic and context in a country as well as the national response. It facilitates the assessment though providing questions to be answered on matters by a national working group related to national context and response. Guidance for the assessment process occurs over 4 stages; it is intertwined with assessment items/questions prompts across 30 pages. The annex includes additional process guidance.	The tool is structured around the specific matters related to this program area.
						Assessment of capacity is not explicit with few requests for quantitative data; it is implicit in the questions used to guide the assessment items.
						The assessment of performance is embedded into the questions that guide analysis of both HIV and sex areas; it includes question related to population health outcomes, but it is not characterized as a comprehensive performance assessment.
11.	Health Systems in Transition. Template for Authors. As adapted for use in the Asia Pacific Region.	WHO/Asia Pacific Observatory # Tool designed for Asia Pacific	2014	Whole health system	Primarily a template for writers to comprehensively describe the situation of a national health system. It includes content instructions for the report and explanatory text, notes, definitions, and descriptions. Guidance for the situation assessment process is 2 pages.	Structure is based on the health system building blocks; also includes section to assess recent health reforms.
						Capacity of the health system is assessed through reporting on recommended data within context of building blocks.
						Assessment of how well the health system is performing occurs within descriptions of how well the building blocks achieve expected functions; it occurs through evaluation against its current stated objectives (eg, within its national health strategic plan) and by considering the extent to which the health system achieves financial protection, equity, population health outcomes, efficiency, transparency, and accountability.
12.	Ear and Hearing Care. Situation Analysis Tool	WHO # Tool designed for global use	2015	Ear and hearing care	Primarily shaped as a questionnaire to guide data and information collection. Includes definitions of key terms and self-grading question sets that summarize each section. Guidance for the assessment process is 6 pages, including annex 1.	Structure is based on the health system building blocks.
						Assessment of the capacity of the program area occurs through prompts to report specific data and describe capacity within context of each building block.
						The performance of ear and hearing care sector is assessed through the self-grading questions that reflect both overall status and some aspects of performance, but not comprehensively.
13.	Eye care service assessment tool	WHO # Tool designed for Global use	2015	Eye care service provision.	Primarily shaped as a questionnaire to guide data and information collection. Includes definitions of key terms and potential sources of information. Includes basic guidance for the process and suggestions for report writing. Guidance for the assessment process is 3 pages.	Structured based on the health system building blocks. It also incorporates components of the WHO Eye Health Global Action Plan, eg, includes reporting on multisectoral engagement and partnerships.
						Assessment of the program area capacity occurs through prompts to report specific data and describe capacity within context of each building block.
						Assessment of program area performance includes reporting on eye health outcomes at population level and coverage of interventions, but other elements of performance are not included; it is not characterized as comprehensive.
14.	Public-private mix for drug-resistant TB: a situation assessment tool to engage relevant care providers in drug-resistant TB management.	WHO # Tool designed for global use	2015	TB	Primarily shaped as a questionnaire to guide data and information collection. Includes definitions and basic guidance for the process. Contains prompts to identify strengths, weaknesses, challenges, and bottlenecks. Guidance for the assessment process is 1 page.	Structure of tool is based on the program area and the specific matters related to a public-private mix of services.
						Assessment of the program area capacity occurs through prompts to report specific data and describe capacity within context of each building block.
						Performance is assessed through consideration of population-level health and the functions of a TB program but somewhat limited because focus is linked to identification of bottlenecks (reasons for underperformance).
15.	Joint External Evaluation Tool for IHR	WHO # Tool designed for global use	2016	IHR	This tool guides an external assessment process that involves countries completing a survey, the results of which are given to joint external evaluation team that then facilitate a country visit, in-depth discussion, and grading across a capacity scale. Guidance for the assessment process is 2 pages.	This tool is structured by the components of the IHR.
						Assessment of capacity is built into the grading system because the 5-level scale is about the capacity to undertake the public health tasks or functions of the IHR.
						The tool primarily assesses the capacity to perform core IHR tasks or functions; the tools does not measure outcomes or effect, and in this way it is not comprehensive.
16.	WHO South East Asia Regional Office tool to conduct system wide analysis of AMR containment programs	WHO # Tool designed for use in SEARO	2017	AMR	A tool used to guide assessments undertaken specifically between WHO and national stakeholders using a participatory process. The assessment is shaped by the components of a mature AMR national program. A scale for self-grading the extent of implementation is included. Guidance for the assessment process is 2 pages.	The tool is structured by the 7 core focus areas of a national program, with indicators under each of these.
						Assessment of capacity is embedded into building blocks with prompts for specific data collection, as well as through the self-grade scale because capacity is implicit in some of these.
						The performance is assessed through the self-graded scale that reflects extent of implementation that is more focused on capacity; it does not incorporate health outcomes, and in this way, it is not characterized as comprehensive.
17.	Report on Immunization in Africa region - utilizing the Maturity Model	WHO AFRO Business case # Tool designed for use in AFRO	2018	Immunization	A maturity grid was developed for this report to support the grading of immunization programs across the Africa region. The tool was structured on the 6 key elements of a country's immunization system with components under each of these and a 4-level grid of maturity.	The tool is structured by the elements of an immunization program.
						The capacity is implicitly assessed by the items that are evaluated using the 4-level maturity grid.
						Performance is integrated into the assessment through the grading along the 4-level maturity grid. The population health outcomes are not graded in this way, only system components, and in this way, it is not characterized as comprehensive performance assessment.

Abbreviations: AFRO, WHO Africa Regional Office; AMR, antimicrobial resistance; EURO, WHO European Regional Office: FAQ, frequently asked questions; FHI, Family Health International; IAEA, International Atomic Energy Agency; IHP, International Health Partnership; IHR, International Health Regulations; JANS, Joint Assessment of National Health Strategies and Plans; SEARO, WHO South East Asia Regional Office; TB, tuberculosis; UNAIDS, United Nations Programme on HIV/AIDS; USAID, United States Agency for International Development.

Phase 2. Drafting

The items identified and adapted to assess rehabilitation were initially described and situated within a guidance manual; they were sequenced based on their position across the logic model (fig 1). Inputs included 4 health system building blocks: governance, financing, workforce, and information systems. Outputs included services, both their accessibility and quality. Medicines and assistive technology sit across inputs, outputs, and outcomes, being integrated across governance, financing, service accessibility, and coverage. Outcomes include the coverage of rehabilitation for those who need it and functioning of those who received it. Impact refers to the overall health and functioning of the population. Additionally, under output and outcome the performance attributes of equity, efficiency, accountability, and sustainability were included; definitions (table 3) of these were based on the WHO Health System Strengthening Glossary.¹⁸

Fig 1.

Rehabilitation Logic Model.

Table 3.

Rehabilitation Logic Model: definitions

Log Frame Area	Term	Definition
Input	Governance	The exercise of political, economic, and administrative authority in the management of matters related to rehabilitation, at all levels, comprising the complex mechanisms, processes, relationships, and institutions.
	Financing	The function of a health system concerned with the mobilization, accumulation, and allocation of money to cover the rehabilitation needs of the people, individually and collectively, in the health system.
	Human resources	The human resources for health that are engaged in actions whose primary intent is to provide rehabilitation within health care; this includes the regulation, management, and training of health workers.
	Information	The aspects of the health information system that collects, analyzes, and disseminates the data and information relevant to rehabilitation; ensures its overall quality, relevance, and timeliness; and enables use of information for decision making.
Output	Services accessibility17	Accessibility of rehabilitation is the people's ability or opportunity to obtain the rehabilitation services they need and to do so while benefiting from financial risk protection. Access to services has 3 dimensions: availability, affordability, and acceptability. Availability is the physical presence of services within reasonable reach of those in need. Affordability is the ability to pay without experiencing financial hardship. Acceptability reflects people's willingness to seek services. Accessibility of services will lead to coverage of rehabilitation intervention in the population groups that need them.
	Services quality	The extent to which the rehabilitation provided to individuals and patient populations increases the likelihood of desired health outcomes.
Outcome	Coverage of rehabilitation interventions	Refers to the level at which people in the population who need rehabilitation have accessed it.
	Functioning outcomes	The outcome of rehabilitation is the functioning gain that occurs in people during the period they receive rehabilitation.
System attributes	Equity	Equity is defined as the absence of systematic or potentially remediable differences in access to health care across population groups based on common social, economic, demographic, or geographic stratifiers.
	Efficiency	Efficiency is considered in terms of both allocative and technical efficiency. Allocative efficiency is considered in terms of the overall structure, organization, and distribution of rehabilitation services within the health system. Technical efficiency is focused at the level of delivery of rehabilitation services; it refers to how efficiently individual client outcomes are achieved in services.
	Accountability	Accountability is the extent to which agencies take responsibility for what they are supposed to do and demonstrate transparency in relation to rehabilitation.
	Sustainability	Sustainability was defined as the availability of financial (capital) and institutional (human capital) resources in the context of future requirements to deliver rehabilitation, as well as through resilience (ability of rehabilitation to respond, adapt, and recover to crisis).
Impact	Better population health and functioning	This describes functioning at the population level; it takes into account the bodily functions as well as human activities and participation in everyday life. It reflects health and demographic trends, outcomes of health and rehabilitation interventions, and interventions that address the environment.

The information collection template was drafted following the structure of the building blocks (table 4). It also included a section on rehabilitation needs and emergency preparedness. Emergency preparedness is a priority area for many countries, particularly for low- and middle-income countries (LMICs) that commonly experience a higher incidence of emergencies and disasters and lower capacity within government systems to respond.

Table 4.

Structure of the Template for Rehabilitation Information Collection

Categories of Information	Types of Relevant Rehabilitation Information Sought
General country information	Population, socioeconomic profile, health strategy, disability strategy, or plan.
Rehabilitation needs	Information to inform the description of rehabilitation needs, including national injury data, communicable and noncommunicable disease specific data, vision, hearing, mental health data, country age structure, relevant health condition registries.
Leadership and governance	Rehabilitation-relevant policy, legislation, planning, coordination, regulation. Government (central administrative agency) capacity (personnel) for rehabilitation, user engagement, intersections with other health, disability, early childhood intervention strategies, etc.
Financing for rehabilitation	Overall expenditure for health and rehabilitation, health financing mechanisms, inclusion in funding packages, out-of-pocket costs for health, arrangements for contracting of rehabilitation providers.
Human resources for rehabilitation	Rehabilitation professions, number of personnel, training, workforce planning practices, remuneration.
Rehabilitation service delivery	Structured across levels of health care, specialized rehabilitation for complex needs, number of rehabilitation beds, and specialized programs; rehabilitation in tertiary, secondary, and primary care; rehabilitation delivered in the community. Rehabilitation across the phases of care, rehabilitation for children with developmental difficulties and disabilities, quality and safety features of rehabilitation.
Assistive technology	Assistive product governance and regulation, planning, financing mechanisms, expenditure, procurement, taxes and duties, availability, providers, service standards, and quality.
Rehabilitation infrastructure	Rehabilitation infrastructure, medical equipment, relevant rehabilitation medicines available.
Rehabilitation information	Rehabilitation data collected at different levels of health information systems, rehabilitation research.
Emergency preparedness	Services in high-risk areas and assistive product stockpiles, emergency preparedness actions undertaken.

The method for the process to undertake a situation assessment was drafted drawing on HSAs, and it has 4 steps. Not all HSA tools emphasized government leadership during the assessment process; however, this was identified as an important feature because of the limited rehabilitation leadership that some governments display.¹⁹ This approach was agreed on at the second consultation meeting where WHO shared plans to link STARS with creation of a new WHO guidance for developing rehabilitation strategic plans and their monitoring frameworks.²⁰

Phase 3. Consultation and preliminary field test

The feedback from the first 2 consultations did not suggest major structural changes, and modifications are described in table 1. Feedback during February 2018 suggested avoiding the circumstance whereby multiple assessment aspects sit under 1 RMM component; this resulted in version 1 of the RMM expanding from 35 to 56 components with a subcomponent structure. Version 1 of STARS was then ready for field testing, phase 4 of its development. The final version (after phase 4) of the RMM's 50 components that are assessed are listed in table 5. See supplemental table S1 (available online only at http://www.archives-pmr.org/) for the composition of experts engaged in consultation and supplemental appendix S1 (available online only at http://www.archives-pmr.org/) for weblinks to WHO STARS documents.

Table 5.

Rehabilitation Maturity Model components and definitions for maturity continuum

Inputs		Components
Rehabilitation governance	1.	Rehabilitation legislation, policies, and plans
	2.	Leadership, coordination, and coalition building for rehabilitation
	3.	Capacity levers for rehabilitation policy and plan implementation
	4.	Accountability, reporting, and transparency of rehabilitation
	5.	Regulation of rehabilitation and assistive technology
	6.	Assistive technology policies, plans, and leadership
	7.	Assistive technology procurement processes
Rehabilitation financing	8.	Rehabilitation financing and coverage of the population
	9.	Scope of rehabilitation included in financing
	10.	Financing of rehabilitation and out-of-pocket costs
Rehabilitation human resources and infrastructure	11.	Rehabilitation workforce availability
	12.	Rehabilitation workforce training and competencies
	13.	Rehabilitation workforce management and planning
	14.	Rehabilitation workforce mobility, motivation, and support
	15.	Rehabilitation infrastructure and equipment
Rehabilitation information	16.	Information on rehabilitation needs, including population functioning and disability
	17.	Information on rehabilitation availability and utilization
	18.	Information on rehabilitation quality and outcomes
	19.	Rehabilitation information used during decision making
Outputs
Rehabilitation accessibility (availability, affordability, acceptability)	20.	Availability of specialized, high-intensity, longer-stay rehabilitation
	22.	Availability of community-delivered rehabilitation
	22.	Availability of rehabilitation in tertiary health care
	23.	Availability of rehabilitation in secondary health care
	24.	Availability of rehabilitation in primary health care
	25.	Occurrence of informal, self-directed care
	26.	Availability of rehabilitation across the acute, subacute, and long-term phases of care
	27.	Availability of rehabilitation across mental health, vision, and hearing programs
	28.	Availability of rehabilitation for target population groups based on country need
	29.	Early identification and referral to appropriate health and rehabilitation for children with developmental difficulties and disabilities
	30.	Availability of rehabilitation in hospital, clinical, and community settings for children with developmental difficulties and disabilities
	31.	Availability of assistive products, including for mobility, environment, vision, hearing, communication, and cognition
	32.	Service delivery of assistive products
	33.	Affordability of rehabilitation
	34.	Acceptability of rehabilitation
Rehabilitation quality	35.	Extent to which evidence-based rehabilitation interventions are used
	36.	Extent to which rehabilitation interventions are of sufficient specialization and intensity
	37.	Extent to which rehabilitation interventions empower, educate, and motivate people
	38.	Extent to which rehabilitation interventions are underpinned by appropriate assessment, treatment planning, outcome measurement, and note-taking practices
	39.	Extent to which rehabilitation is timely and delivered along a continuum, with effective referral practices
	40.	Extent to which rehabilitation is person-centered, flexible, and engages users, family, and caregivers in decision making
	41.	Extent to which health personnel and community members are aware, knowledgeable, and seek rehabilitation
	42.	Extent to which rehabilitation is safe
Outcomes, impact, and attributes of rehabilitation
	43.	Coverage of rehabilitation interventions for population groups that need rehabilitation
	44.	Functioning outcomes of rehabilitation for those who receive rehabilitation
	45.	Equity of rehabilitation coverage across disadvantaged population groups
	46.	Allocative and technical efficiency of rehabilitation
	47.	Multilevel accountability for rehabilitation performance
	48.	Financial and institutional sustainability of rehabilitation
	49.	Resilience of rehabilitation for crisis and disaster
	50.	The functioning of the population
Maturity continuum definitions
Needs no immediate action		This implies the component is at a high level of maturity and correspondingly performs well. Although there may be small concerns that need monitoring and addressing over time, currently no action is needed.
Needs minor strengthening		This implies the component is at a moderate level of maturity and correspondingly performs moderately well; the component is well established, but there are a few areas for improvement.
Needs major strengthening		This implies the component is at a low level of maturity and correspondingly performs weakly; the component is established, but there are many areas that need improvement.
Needs establishing		This implies the component is at a very low level of maturity and is either not established or just emerging; correspondingly it performs very weakly.

Discussion

STARS is a comprehensive HSA tool developed by WHO to support countries identifying priority actions to strengthen rehabilitation. This article presents its participatory development process, which lasted several years and involved a broad range of rehabilitation experts from government and nongovernment agencies and from low-, middle-, and high-income countries. Results of STARS field testing during 2018-2019 in 7 countries is in preparation.

STARS was developed to generate a standardized and structured situation assessment of rehabilitation to inform decision makers, particularly government stakeholders in LMICs. Designing a HSA tool to meet government needs is in line with the preferred demand-driven approaches promoted for an HSA.²¹ The STARS structure and guidance supports the generation of accessible and informative reports relevant to the country context. The tool is designed to support governments to identify strengths, weaknesses, and priorities to strengthen rehabilitation. Although some information collected during STARS enables intercountry comparisons, creating a tool that supports this was not the primary purpose.

HSA tools that are valued have clear definitions, methods that reveal the interaction between assessment components,²² and approaches that reduce complexity.²³ The decision to integrate the health system building blocks and performance components into the logic model was made to reveal the interaction between assessment components and to provide an overarching coherence to STARS. The logic model helps to reveal the interaction between components through its 1-direction sequential structure; for example, the inputs of workforce and financing contribute to the output of accessible services that contributes to the coverage of rehabilitation in the population. Valid critiques of the logic model highlight the nonlinearity and feedback loops that commonly exist, but using this structure does not limit the description of these during the assessment writing.

Understanding not only the capacity but also the performance of a health system is important.^24,25 Performance was prioritized in STARS for 3 reasons. First, without considering performance the assessment only describes what exists and not how effectively, efficiently, or sustainably it performs its expected functions. Second, assessing performance through consideration of the “coverage of rehabilitation in population groups that need it” addresses the important question of “who is getting rehabilitation and who is not,” which is essential for understanding equity. Third, because assessing performance includes consideration of population outcomes and impact, such as population functioning, it highlights the potential health gain by strengthening rehabilitation in a country's health system. Indeed, population functioning is proposed as a third category of health indicators, in addition to mortality and morbidity, and rehabilitation is a key health strategy that addresses this.²⁶ The analysis of existing HSA tools revealed that performance was commonly considered through the population health outcomes (eg, mortality, morbidity); hence, including population functioning is in line with these approaches.

An HSA requires measurement, analysis, and description of the situation,⁶ and measurement typically informs both the analysis and description, so it is a mainstay. However, measurement has challenges, and for rehabilitation, very often the data and information are unavailable, not comparable, inaccurate, or incomplete. In many countries all those challenges exist, a key reason being the limited integration of rehabilitation into health systems, in particular the health information systems. To address this, the STARS guidance includes definitions for measurement items, allocation of an appropriate amount of time for data collection, and creation of a multistakeholder rehabilitation technical working group that may validate measures and be used for consensus. Additionally, consultants, when writing a STARS report, can describe the situation based on what they observe or is reported to them during their time in the country, then combine this with measures (albeit often incomplete) to make cautious conclusions. This approach is typically necessary for the final section of the STARS report when conclusions are cautiously made about the “likely” or “probable” population coverage, equity, efficiency, and sustainability of rehabilitation. Over time, it is anticipated that government efforts to integrate rehabilitation into health systems, including health information systems, will significantly increase its measurability.

Given the relatively weak governance of rehabilitation in several LMICs,²⁷ the WHO has repeatedly called on countries to strengthen their leadership for rehabilitation.²⁸ The WHO therefore consciously emphasized government leadership in STARS. Involving government leaders in the situation assessment encourages government ownership, promotes uptake of findings, and more likely leads to commitment to implement recommendations. In a sector such as rehabilitation that has been frequently underprioritized by LMIC health ministries and includes unsustainable services provided by nongovernmental organizations, this is important.

STARS data can be used for health policy and systems research. There is a notorious lack of good-quality research in rehabilitation, especially in LMICs, an important motivation to ensure that STARS data can be used in research. Indeed, studies using HSA data exist, for instance, from the WHO Assessment Instrument for Mental Health Systems²⁹ tool.

Study limitations

This article must be understood in the light of its limitations. First, the search for HSA tools and rehabilitation frameworks was not systematic, albeit sufficient to inform STARS. Second, although STARS is comprehensive, it is not exhaustive. Areas such as rehabilitation delivered in the context of work reintegration or substance abuse are not yet explicitly addressed and could be considered in future amendments or add on modules.

Conclusions

STARS is the first HSA shaped to rehabilitation and has been developed by WHO in a participatory process based on several expert consultations. STARS meaningfully supports governments to better understand the status of rehabilitation in their countries, define priorities for action to strengthen rehabilitation, and facilitate the monitoring of system level changes. Moreover, STARS data can be used by health policy and systems research to support evidence-informed policy. In this sense, WHO STARS may become a key element in strengthening rehabilitation in health systems, as requested in the Rehabilitation 2030 call for action.