Community Rehabilitation for Rural and Remote Australia: Measuring What Matters Based on the International Classification of Functioning, Disability and Health (ICF): A Scoping Review

ABSTRACT

Objective

To review the quantitative outcome measures that have been used to evaluate community rehabilitation services delivered across rural and remote Australia.

Design

A scoping review was completed and reported using the Preferred Reporting Items for Systematic reviews and Meta‐Analyses (PRISMA) extension for Scoping Reviews.

Methods

Peer‐reviewed, original research published between January 2010–September 2023 was searched using MEDLINE, PubMed, EMBASE, CINAHL, Scopus, Cochrane database, Joanna Briggs Library, PsychINFO, Web of Science and Google Scholar. Studies were selected if they involved allied health outpatient, subacute or nonacute services for Australian rural or remote dwelling populations. Participants were required to have an underlying impairment. Face‐to‐face or telehealth delivery in a rural and remote location was included. Measures were identified and then mapped to the ICF domains of activity and participation, as well as quality of life.

Results

A total of 27 studies were included that yielded 40 different outcome measures of activity, participation and/or quality of life. Few measures, however, were used consistently across studies, and even fewer demonstrated a significant change across more than one study. Most studies evaluated single interventions, and few studies evaluated the service model as a whole.

Conclusion

To ensure robust evaluation of community rehabilitation services in rural and remote Australia, a core data set and common framework for evaluation of community rehabilitation services is required. The evaluation framework must ensure consistency in measurement that reflects rural and remote service models and takes into account the environment in which services are delivered.

Summary.

• What is already known on this subject?

  • ○
    Evaluation of allied health services in rural and remote communities is very limited.
  • ○
    An evidence base to support emerging community rehabilitation service models is required.
  • ○
    Quantitative data are generally used in the allocation of resources for health and disability services.
  • ○
    Quantitative outcome measures are required to enable evidence‐informed investment in community rehabilitation services.
• What does this paper add?

  • ○
    This paper provides a comprehensive overview of quantitative outcome measures used to evaluate a diverse range of services offered in rural and remote settings.
  • ○
    Quantitative outcome measures have not been used consistently across studies, and few have demonstrated a significant change.
  • ○
    To provide a foundation for evidence‐informed investment, an evaluation framework that includes a core data set is required.
  • ○
    Any such core data set must account for heterogeneity and enable the pooling of data for comparison between interventions, service models and national standards.

1. Introduction

Evaluation of community rehabilitation services in rural and remote Australia occurs infrequently and inconsistently, if at all. As a result, an evidence base to support existing or emerging community rehabilitation service models is lacking. Instead, the rural and remote service landscape is littered with design, development or pilot projects that have not progressed or been sustained due to a lack of information about outcomes [1]. Repeatedly, service delivery defaults to long‐held mainstream models that continue to be funded despite a lack of evidence of the effectiveness or ineffectiveness of these models [2]. It is clearly time for robust evaluation of community rehabilitation services for rural and remote Australia.

The disabling consequences of disease, injury and ageing place a heavy and ever‐increasing burden of care on families, communities and services in rural and remote Australia. In First Nations communities, rates of disease and injury are disproportionately higher than in non‐Indigenous communities, and consequently, rates of disability are similarly higher [3]. Moreover, service provision is difficult largely due to remoteness, cultural diversity and a disparately located population. Services that exist are often siloed, fragmented and under‐resourced. Allied health professionals, who play a crucial role in community rehabilitation services, are often in short supply and difficult to recruit and retain [4]. More recently, the National Disability Insurance Scheme (NDIS) roll‐out across Australia provided the opportunity to address this shortfall and to improve access and quality of community rehabilitation services for people living in rural and remote areas. As a result, a proliferation of new models of service delivery has emerged [5]. Examining the benefits of each new model is imperative; however, it is difficult to compare the benefits of one model to the next and to long‐held mainstream models.

How to evaluate community rehabilitation services in rural and remote Australia is a challenging question. Although quantitative data are generally used in the allocation of resources for health and disability services, there is a heavy reliance on qualitative methodologies by rural health researchers [6]. While often seen as the most appropriate methodology for demonstrating benefits for communities with small and diverse service user groups, reliance on qualitative methodologies may also reflect the limitations of quantitative measurement tools in the rural and remote context [7]. Such limitations include the need for sample sizes that are large enough to demonstrate benefit, equivalence and/or superiority of one service model over another. To aggregate data to achieve the necessary sample size, consistency in outcome measurement is required, regardless of the diversity in service users. Also, the scientific imperative to ensure the evaluation data are sufficient is often contrary to answering research questions that respond to context and place [6]. Measurement tools that measure what matters to people living a rural and remote lifestyle, and that are sufficiently sensitive to demonstrate change, appear to be lacking [8]. Recognising that what gets measured gets managed, the risk is that the needs and aspirations of the most disadvantaged communities in rural and remote Australia will be undervalued and underfunded.

The first step in developing robust evaluations of community rehabilitation services in rural and remote Australia is to review the current scientific literature on outcome measures that have been administered in this setting. In so doing, measures need to be considered that incorporate diversity: across health conditions; interventions; lifestages; contexts in which people live and priorities in terms of outcomes. The International Classification of Functioning, Disability and Health (ICF) offers a comprehensive and standardised language and categorisation system for data comparability that includes all aspects of a person's health and well‐being [9]. The ICF framework, which is underpinned by a person‐centred approach, considers not only the person's intrinsic health capacity, but also their lived experience of health [10]. In other words, the ICF takes into account what the person actually does, as a result of the interaction between their health capacity and contextual factors. For this reason, the ICF offers a suitable framework to categorise outcome measures that are relevant and meaningful to individuals, families and communities in rural and remote locations.

The purpose of community rehabilitation services is to improve an individual's ability to perform daily activities and participate in meaningful roles within the community, and in turn, enhance their quality of life. Accordingly, the ICF domains of activity and participation offer a suitable reference system for comparative analysis and standardised reporting of community rehabilitation services across a wide range of conditions, different settings and different priorities. The evidence can then be used to inform decision making regarding the allocation of resources and the development of policies and programs aimed at improving activity, participation and quality of life.

The aim of this study was to review quantitative outcome measures used to evaluate community rehabilitation services delivered in rural and remote Australia. The objectives were to: (1) Identify measures aligned with the ICF domains of activity and participation and their impact on quality of life; and (2) map each measure to the ICF domains of activity, participation and quality of life. The information will be used in the development of an evaluation framework for rural and remote community rehabilitation services in Australia.

2. Methods

2.1. Study Design

A scoping review, based on the approach proposed by Arksey and O'Malley [11] and subsequent scoping methodology reviews [12, 13], was completed. A scoping review offers a process for knowledge synthesis using a systematic approach [11] in fields with emerging evidence or complex, heterogeneous service delivery where systematic review of randomised controlled trials is challenging or not possible [12, 14]. The review was reported using the Preferred Reporting Items for Systematic reviews and Meta‐Analyses (PRISMA) extension for Scoping Reviews [15].

2.2. Search Strategy

A comprehensive search was completed of peer‐reviewed original research published in English, between January 201–September 2023. Electronic databases used included: MEDLINE, PubMed, EMBASE, CINAHL, Scopus, Cochrane database, Joanna Briggs Library, PsychINFO, Web of Science and Google Scholar. In consultation with a university librarian, search terms were identified following iterative preliminary searches of CINAHL and MEDLINE titles, abstracts and index terms used to describe articles relevant to the review [12]. Database searches were all completed using all identified keywords and index terms. Key search terms included: (rehabilitation* or stroke rehabilitation or neurological rehabilitation or physical and rehabilitation medicine or sub‐acute care or human development or behavioural and mental disorders or health services for persons with disabilities or health services for the aged or specialities, allied health or occupational therapy or physical therapy* or speech‐language pathology or disab*) and (rural or remote or rural health* or rural populations or rural areas or hospitals, rural or rural health centres or rural health personnel) and (Australia). See Data S1 for the full search string and results for each database. Eligibility criteria for inclusion were developed using the PICOT structure and are outlined in Table 1.

TABLE 1.

Inclusion and exclusion criteria.

Criteria	Inclusion	Exclusion
Population	Australian rural or remote dwelling children, young people, adults and older adults with an underlying impairment or their carers/family. Rural and remote dwelling was defined by the authors of each study.	Less than 10% of the participant sample were rural or remote dwelling.
Intervention	Service delivered face to face or via telehealth in a rural and remote location in the community, outpatient subacute or nonacute setting by, or with oversight from, allied health professionals.	Health promotion services in the absence of underlying impairment.
Outcome	Quantitative outcome measures used to determine change in activity, participation or quality of life.
Comparison	Not required.
Time	January 2010–September 2023

2.3. Study Selection

Database searches were conducted by one reviewer (Author 1) and exported into EndNote [16]. Screening of titles and abstracts was conducted independently by two reviewers (Author 1, Author 2) for a minimum of 50% of articles and by one reviewer (Author 1) for the remainder. All full texts were independently screened for inclusion by both reviewers. Any discrepancies between the two reviewers or uncertainties for the one reviewer were resolved through discussion between the two reviewers. Additional studies for inclusion were identified through manual searching of reference lists and citation tracking of literature reviews and studies that met the inclusion criteria. Two articles were considered as one study if they reported on the same participant group receiving the same intervention but different outcome measures. Protocol papers were excluded.

2.4. Data Extraction and Analysis

Study characteristics were extracted using a customised Excel spreadsheet and included: authors; publication year; location by state; rurality; service delivered; participants; and study design. Outcome measures were selected that related to activity and/or participation (defined by AC and RB) or that used a quality‐of‐life tool (defined by individual study authors); and whether a statistically significant within‐group change for rural and remote participants was demonstrated. Activities were considered actions or tasks executed by an individual, and participation was considered involvement in life situations as defined by the ICF [17].

Data were then mapped to the nine ICF domains of activity and participation and to quality of life. The nine domains are: Learning and applying knowledge (d1); general tasks and demands (d2); communication (d3); Mobility (d4) Self‐care (d5); Domestic life (d6); Interpersonal interactions (d7); Major life areas (d8); and Community, social and civic life areas (d9). Mapping was done by the two reviewers, each with over 20 years of clinical experience from two different allied health professions (occupational therapy and physiotherapy) working in a range of community and rural practices.

2.5. Quality Assessment

Methodological quality, conceptual quality and reporting quality of the quantitative component of included studies was assessed using the Mixed Methods Appraisal Tool (MMAT) [18], with 50% of studies assessed by the two reviewers (Author 1, Author 2) with any difference of opinion resolved by discussion to reach a consensus. The remainder of the studies were assessed by one reviewer (Author 1). Studies were not excluded on the basis of methodological quality.

3. Results

A total of 6732 articles were identified from the database searches, with 2844 duplicates removed. Title screening led to the removal of 3231 articles, leaving 657 articles for abstract screening, and then 187 articles for full‐text review. Searching of references and citations identified five more studies for full‐text review. Finally, 28 articles reporting 27 studies were included in the scoping review. The study selection flow chart is contained in Figure 1.

FIGURE 1.

Study selection PRISMA diagram.

Characteristics of the 27 included studies are summarised in Table 2. Included studies were published between January 2010 and September 2023. There was a substantial increase in studies over the study period, with only eight included studies published between 2010 and 2015 compared to 22 studies published between 2016 to 2023. Quantitative study designs were predominantly non‐randomised (n = 15, 56%), to a lesser extent randomised controlled (n = 11, 41%) with one quantitative descriptive study. Methodological quality of included studies is detailed in Table 2. Overall, the quality was generally low and ranged from one study that met 100% of quality criteria (represented as •••••) [19], five studies scored •••• [20, 23, 24, 27, 34, 42], nine studies scored ••• [21, 29, 30, 31, 33, 38, 39, 40, 45], seven studies scored •• [26, 28, 32, 35, 36, 43, 46], and five studies scored • [22, 25, 37, 41, 44].

TABLE 2.

Summary of included study characteristics.

Author, Year; State of Australia; Rurality index	Participants (n=); Age (central tendency) % rurality % First nations if stated	Service; Service recipients; Service setting; Single discipline (SD) or Multi‐discipline (MD)	Quantitative study design according to MMAT categories; Measurement time points	Activity, Participation and quality of life outcome measures b ; Change for rural cohort; *< 0.01; **< 0.001	Quality; MMAT total	1 a	2 a	3 a	4 a	5 a
Almeida et al. 2021; [19]; WA; ASGC 2–5	307 adults; x¯ 71 years; 65 > 80; 100% rural	8‐week telebehavioural activation programme + usual care vs. usual care; People with depression; Setting not stated; SD; Psychology	Randomised controlled. Pre‐post, 6, 12 months	Self‐report; –SF‐36 Mental subscale**	•••••	✓	✓	✓	✓	✓
Baggio & Buckley, 2016; [20]; NSW	83 adults; x¯= 59 years; 22–87; 100% rural	116 rehabilitation programmes; Neurological conditions; Ambulatory setting; MD	Nonrandomised Pre‐post.	Self‐report – Lawton IADL Scale** – GAS**	•••	✓	✓	✓	╳	╳
Conlan, Thompson, & Fary, 2016; [21]; WA	6 women; x¯= 35 years; 100% rural	6‐week tele‐stress urinary incontinence telehealth; Home setting; SD; Physiotherapy	Quantitative descriptive Pre‐post	Self‐report –ICIQ‐LUTSqol (CS)	•••	✓	╳	✓	╳	✓
Courtney‐Pratt, 2012; [22]; Tasmania	8 adults; 52–87; 100% rural	7‐week cardiac rehabilitation; People at risk; Hospital or home setting; MD	Nonrandomised. Pre‐post and 6‐month	Self‐report –HeiQ	•	╳	╳	╳	╳	✓
Cox et al. 2022; [23]; Victoria	142 adults; x¯= 68 years; 29% rural	8‐week pulmonary rehabilitation; Dx chronic respiratory disease; Centre v home setting; MD	Randomised controlled. Pre, post, 12 months.	Self‐report – CRQ‐D (CS) – SF‐36 v2 Therapist‐administered – 6MWT	••••	✓	✓	✓	╳	✓
Dadds et al. 2019; [24]; NSW	133 families; 133 children; x¯= 7 years; 50% rural & remote	6–10 week online parenting programme (AccessEI); Families with children 3–9 years with disruptive behaviour; Telehealth in home (rural arm); SD, Psychology	Randomised controlled. Pre, post, 3 months.	Parent report –SDQ** –CPRS‐R**	••••	✓	✓	✓	╳	✓
Dent et al. 2017; [25]; SA	145 adults; x¯= 66 years; SD 66; 100% rural	3‐month musculoskeletal self‐management programme v usual care; People with pain score > 3/10; Community setting; SD, Physiotherapy	Randomised controlled. Pre, 6 months	Self‐report – EQOL–5D‐5L	•	╳	╳	✓	╳	╳
Dowell et al. 2021; [26]; State NS	74 youth; x¯=13 years [12, 13, 14, 15]; 44% rural; 19% First Nations	5‐month mental health and well‐being programme for junior rugby league players (RISE); Sports setting; MD	Nonrandomised; Pre, post	Self‐report –SDQ‐Prosocial scale*	••	╳	✓	╳	╳	✓
Eakin et al. 2012; [27]; QLD; ASGC 2–5	143 women; x¯= 52 years; 100% non‐urban	8‐month exercise programme; People who have undergone breast cancer surgery vs. usual care; Online in home setting; SD, Exercise Physiology	Randomised controlled; Pre, 6 and 12 months	Self‐report –FACIT‐F** –FACT‐B + 4**	••••	✓	✓	✓	╳	✓
Farmer & Rupert, 2013; [28]; Victoria	98 parents; 100% rural	6‐week autism education programme; Parents of children 2‐6 years with Autism; Rural early intervention centre; MD	Nonrandomised; Pre, post	Self‐report –UASD questionnaire – not validated.**	••	╳	╳	✓	╳	✓
Frensham, Parfitt & Dollman, 2018; [29]; SA	91 adults; x¯= 65 years; 47% rural	12‐week walking intervention; (STRIDE); Cancer survivors; Hybrid setting—community and online in home; SD, Exercise Physiology	Randomised controlled; Pre‐post, 3 months.	Self‐report – SF‐36v2** Therapist‐administered – 6MWT – Step counts (pedometer)**	•••	╳	✓	✓	╳	✓
Ghahari & Parker, 2012; [30]; WA	115 adults; x¯= ~50 years; 40% rural	7‐week fatigue self‐management; Adults with neurological conditions; Online v face‐to‐face; SD, Occupational therapy	Randomised controlled	Self‐report –GSE –PWI –ACS‐Aus –DSSI	•••	✓	✓	╳	╳	✓
Langbecker et al. 2019; [31]; QLD	98 children; x¯ ~ 7 years [5, 6, 7, 8, 9, 10, 11, 12]; 100% rural	12–48 week student‐led therapy telehealth service; Children requiring support; School setting and online; MD	Nonrandomised; Pre, post	Teacher report –T‐SLEOP** (unvalidated)	•••	╳	╳	✓	✓	✓
Lim et al. 2021; [32]; All states	867 adults; x¯= 49.9 years; 66% non‐urban	16‐week online pain management programme (Reboot Online); People with persistent pain; Home setting; MD	Nonrandomised; Pre, mid, post	Self‐report –PDI**	••	x	✓	x	✓	x
Marsden et al. 2010; [33]; NSW	25 adults; x¯= 70 years, SD9; 17 carers; x¯= 66 years, SD10; 100% rural	7‐week CLASSSiC stroke rehabilitation programme; People with stroke and their carers; Local public hospital setting; MD	Randomised controlled; Pre, post, 3 months	Self‐report – SIS – HIS Therapist‐administered – 6MWT – TUG – mRS	•••	╳	✓	✓	╳	✓
Mesa‐Castrillon et al. [34]; All states; MM 2–5	156 adults; x¯ = 63 years, SD11.2; 87% rural	3‐month eHealth physical activity programme for low back pain/knee osteoarthritis; Setting unknown; SD, Physiotherapy	Randomised controlled; Pre, post, 6 months	Self‐report –PSFS* –AQoL‐8D* –IPAQ‐SF –RMDQ* –WOMAC*	••••	✓	✓	✓	╳	✓
Missen et al. 2021; [35]; Victoria	17 adults; 24–56 years; 100% rural	6‐week therapeutic gardening People recovering from substance abuse; Day rehabilitation setting; MD	Nonrandomised; Pre‐post and 1 month	Self‐report –WHOQOL‐BREF*	••	╳	✓	╳	╳	✓
Monroe et al. 2023; [36]; NSW	6 older people; Median = 82 years (55–81); 100% rural	6‐week aged care choir (MuSiCON); People with cognitive impairment; Residential aged care facility; SD, Speech Pathology	Nonrandomised; Pre, 4 weeks, post	Self‐report – DEMQoL‐4 proxy	••	╳	╳	✓	╳	✓
McLeod et al. 2023; [37]; NSW, Vic, QLD	45 children; x¯= 6.3 years [4, 5, 6, 7, 8, 9, 10, 11, 12]; 100% rural	4‐week online speech exercises in game format (SayBananas!); Children with speech sound disorders; Home setting; SD, Speech Pathology	Nonrandomised; Pre‐post	Child report – SPAA‐C Parent report – ICS	•	╳	✓	╳	╳	╳
Newby, Mewton & Andrews, 2017; [38]; NSW	2109 adults; x¯= 40 years; 25.2% rural	6 session online transdiagnostic internet cognitive behaviour therapy; Adults with depression and anxiety Primary care setting; MD	Nonrandomised; Pre‐post	Self‐report –WHODAS 2.0	•••	✓	✓	╳	✓	╳
Raghavendra et al. 2015; [39]; SA	8 youth; x¯= 15 years; 100% rural	5‐month social media education Youth with disabilities and communication difficulties; Telehealth in home setting; MD	Nonrandomised; Pre‐post	Self‐report –GAS –COPM* –CCP*	•••	╳	✓	✓	╳	✓
Raghavendra et al. 2018; [40]; SA	9 youth; x¯=17 years; 100% rural	4‐month social media education Youth with disabilities; Telehealth in home setting; MD	Nonrandomised; Pre‐post	Self‐report –GAS –COPM** –CCP*	•••	╳	✓	✓	╳	✓
Riley, Smith & Oakes, 2011; [41]; SA	551 adults; x¯=45 years; SD 11.72; 10% rural	12‐week exposure therapy for people with gambling problems; Hybrid—clinic, home; F2F, Tele; MD	Nonrandomised; Pre‐post	Self‐report –WSAS*	•	╳	✓	╳	╳	╳
Sangster et al. 2015; [20, 42]; NSW	313 adults; x¯= 64 years; 44.8% rural	8‐week telehealth weight and physical activity programme for people with cardiac disease; Home setting; MD	Randomised controlled; Pre, mid, post	Self‐report –Physical activity (accelerometer) –AQoL4D	••••	✓	✓	✓	╳	✓
Scriven, Doherty & Ward, 2019; [43]; QLD	21 adults; x¯= 59 years; 29–83; 29% rural; 71% remote	4‐week telehealth pain management program; Adults with persistent pain; Health facility; MD	Nonrandomised; Pre‐post	Self‐report –PROMIS Global –EQ‐5D c	••	✓	✓	╳	╳	╳
Stone & Packer, 2010; [44]; WA	14 adults; x¯= 66 years; SD11.86; 100% rural	Stanford Chronic Disease Self‐Management Program; Adults with chronic disease; Setting not stated; MD	Nonrandomised; Pre‐post.	Self‐report –ACS‐Aus –HeiQ—RETRO*	•	╳	✓	╳	╳	╳
Wagner et al. 2020; [43]; WA	271 children; x¯= 8.7 years; SD 1.7; 100% remote; 90% First Nations	8‐week Alert Program; At risk school children; School setting; MD	Randomised controlled; Pre‐post	Teacher report – SESBI Parent report – ECBI**	••	✓	╳	✓	╳	╳

Note: Outcome measure detected within group statistically significant change at *p > 0.05 **p > 0.001.

Abbreviations: %, First Nation and participants reported where known; ASGC, Australian Standard Geographic Classification; F2F, face to face delivery; hybrid relates to services delivered via a mix of face to face and telehealth; For studies including service delivery in metro and regional locations the % of rural/remote participants where the service was delivered rurally is reported where known; MM, Modified Monash Model; NS, not stated or unknown/unclear; NSW, New South Wales; NT, Northern Territory; QLD, Queensland; SA, South Australia; WA, Western Australia; Workforce or practice models: SD, single discipline; MD, multidisciplinary.

^a MMAT total based on the quantitative category of each study [30] ✓ = yes, ╳ = no or cannot tell for each MMAT criteria 1–5.

^b See Supporting Information for glossary of measures and reference for each measure.

^c EQ‐5D was calculated using the PROMIS scores.

Across the 27 included studies, there were 5865 participants in total, with sample sizes ranging from 6 to 2109 with a median sample size of 98. There were 2024 participants who received services in rural and remote locations; 25 in regional; and 1021 in either regional, rural or remote; and 46 in metropolitan, regional, rural or remote. The remaining 2750 participants were in metropolitan locations, largely due to one study with a metro and nonmetro arm [38]. Rurality was formally classified using the Modified Monash in one study, the ASGC in two studies, and not classified in the remaining studies. Participants were from across the lifespan; however, the majority were adults and older people (n = 5227) and most studies evaluated services that targeted the health of adults and older people (n = 19) [19, 20, 21, 22, 23, 25, 27, 29, 30, 32, 33, 34, 35, 36, 38, 41, 42, 43, 44, 45]. A total of 258 First Nations participants were identified in two studies [26, 46], with one of these studies evaluating an intervention exclusively delivered in remote Aboriginal communities [46].

Face‐to‐face services were received by more than a quarter (26.7%) of all participants [22, 23, 24, 25, 26, 28, 29, 30, 33, 35, 36, 39, 40, 41, 44, 45, 46], while the remainder received services via telehealth. Service settings included health facilities (n = 8), the home (n = 13), school (n = 2), a sports setting (n = 1) or not stated (n = 6). Allied health workforce models were predominately multidisciplinary (n = 17 studies); the remaining reported single‐discipline interventions from five single allied health disciplines [19, 21, 24, 25, 27, 29, 30, 34, 36, 37].

Services evaluated were predominantly single interventions or programmes (e.g., continence, mobility, cognitive behaviour therapy) for specific diagnostic groups (e.g., cardiac, neurological, mental health) or for a particular life stage (e.g., children, adults and older people). Only one evaluation included a realistic evaluation of a whole service that included all programmes delivered and all participants of the service [45]. Comparative service model evaluations that involved rural participants in all arms of the studies were present in eight studies, of which five studies compared telehealth delivery to ‘usual care’.

3.1. Quantitative Outcome Measures of Activity, Participation, and Quality of Life

Across the 27 included studies, 40 different outcome measures were identified (Table 3; see Supporting Information for references for the included outcome measures). Self‐report measures were used in 24 studies; carer or parent report in three studies [24, 37, 46] and teacher report in two studies [31, 46]. The remaining three studies reported therapist‐reported or administered outcomes [23, 29, 33]. A total of 31 measures were reported in one study each, seven measures in two studies each, ACS‐Aus [30, 44], AQoL [34, 42], CCP and COPM [39, 40], EQ‐5D/EQ‐5D‐5L [25, 43], HeiQ [22, 44] SDQ [24, 26] and three measures used in three studies each, GAS [39, 40, 45], 6MWT [23, 29, 33], SF36 [19, 23, 29]. A statistically significant change for rural and remote participants was demonstrated in 15 studies.

TABLE 3.

Glossary for included outcome measures in alphabetical order.

Abbreviation	Outcome measure
ACS‐Aus	Activity card sort—Australian version
AQoL 4D/8D	Assessment of quality of life 4D/8D
CCP	Circles of communication partners
COPM	Canadian occupational performance measure
CPRS‐R	Conners' parent rating scale—Revised
CRQ‐D	Chronic respiratory disease questionnaire dyspnoea
DEMQOL‐4	Dementia quality of life questionnaire (version 4)
DSSI	Duke social support index
ECBI	Eyberg child behaviour inventory – parent rated
EQ‐5D/−5L a	EQ‐5D‐5 levels
FACT‐B + 4	Functional assessment of cancer therapy‐ breast
FACIT‐F	Functional assessment of chronic illness therapy—fatigue scale
GAS	Goal attainment scaling
GSE	Generalised self‐efficacy scale
HeiQ/‐RETRO	The health education impact questionnaire/Version 2
HIS	Health impact scale (carers)
ICS	Intelligibility in context scale
ICIQ‐LUTSqol	International consultation on incontinence questionnaire—lower urinary tract symptoms quality of life
IPAQ‐SF	International physical activity questionnaire short form
Lawton IADL	Lawton instrumental activities of daily living scale
6MWT	6 min walk test
mRS	Modified rankin scale
PA	Physical activity (Accelerometer and step counts/Pedometer)
PDI	Pain disability index
PROMIS	Patient reported outcome measurement information system—global
PSFS	Patient‐specific functional scale
PWI	Personal wellbeing index
RMDQ	Ronald‐Morris disability questionnaire
SDQ	Strengths and difficulties questionnaire (parent and teacher version)
SESBI	Sutter Eyberg student behaviour inventory‐revised—teacher rated
SF‐12/36/v2	Short form health survey, 12/36 item
SIS	Stroke impact scale
SPAA‐C	Speech participation and activity assessment of children
TUG	Timed up and go
T‐SLEOP b	Teacher‐rating child's speech and language abilities, educational outcomes and participation in class
UASD b	Understanding autism and understanding my child with autism
WOMAC	Western Ontario and McMaster Osteoarthritis index
WSAS	Work and social adjustment scale
WHODAS 2.0	World health organisation disability assessment schedule
WHOQOL‐BREF	World health organisation quality of life brief

^a EQ‐5D is not an abbreviation.

^b Unvalidated tool.

3.2. Mapping of Outcome Measures to the ICF Domains of Activity, Participation and Quality of Life

Outcome measures are mapped to the ICF categories of activity, participation and quality of life in Table 4. All ICF domains were reflected by two outcome measures that involved participants identifying their own goals (GAS and PSFS). The Interpersonal Interactions and Relationships domain was most commonly represented by 25 measures. The Mobility domain was the next most commonly represented, with 21 measures. Quality of life was represented by 13 outcome measures.

TABLE 4.

Content mapping of outcome measures based on the International Classification of functioning, Disability and Health (ICF).

Measure	ICF domain									QoL	Purpose
	Learning and Applying Knowledge	General tasks and demands	Communication	Mobility	Self‐care	Domestic life	Interpersonal Interactions Relationships	Major Life Areas	Community, Social and Civic Life		Condition‐specific or generic
	d1	d2	d3	d4	d5	d6	d7	d8	d9
ACS‐Aus									✓		Generic
AQoL		✓	✓	✓	✓	✓	✓		✓	✓	Generic
CCP							✓				Generic
CHAMPS				✓		✓	✓	✓	✓		Generic
COPM					✓	✓		✓	✓		Generic
CPRS‐R	✓	✓	✓				✓	✓			Generic
CRQ‐D		✓		✓	✓	✓			✓	✓	Respiratory
DEMQOL			✓		✓		✓			✓	Dementia
DSSI							✓		✓		Generic
ECBI		✓					✓				Generic
EQ‐5D		✓		✓	✓	✓		✓		✓	Generic
FACT‐B + 4							✓	✓		✓	Cancer
FACIT‐F		✓									Chronic
GAS a	✓	✓	✓	✓	✓	✓	✓	✓	✓		Generic
GSE	✓	✓									Generic
HeiQ	✓	✓	✓				✓				Chronic
HIS	✓		✓	✓	✓	✓	✓	✓	✓	✓	Generic
ICS			✓								Generic
ICIQ‐LUTS		✓				✓	✓	✓		✓	Continence
IPAQ‐SF				✓							Generic
Lawton	✓	✓		✓	✓	✓					Generic
6MWT				✓							Generic
mRS		✓		✓	✓	✓					Generic
PA				✓							Generic
PDI					✓	✓	✓	✓	✓		Pain
PROMIS				✓			✓		✓	✓	Generic
PSFS a	✓	✓	✓	✓	✓	✓	✓	✓	✓		Generic
PWI							✓		✓	✓	Generic
RMDQ		✓		✓	✓	✓					Back pain
SDQ	✓						✓				Generic
SESBI		✓					✓				Generic
SF‐12/36		✓		✓	✓	✓	✓	✓	✓	✓	Generic
SGRQ			✓	✓	✓	✓	✓	✓	✓	✓	Respiratory
SIS	✓		✓	✓	✓	✓	✓	✓	✓	✓	Stroke
SPAA‐C			✓				✓				Generic
TUG				✓							Generic
T‐SLEOP	✓		✓					✓			Generic
UASD b	✓	✓									Autism
WOMAC				✓	✓	✓					Arthritis
WSAS						✓	✓	✓	✓		Generic
WHODAS 2.0	✓	✓	✓	✓	✓	✓	✓	✓	✓		Generic
WHOQOL‐BREF		✓		✓	✓		✓	✓	✓	✓	Generic

Note: Purpose—condition‐specific refers to an outcome measures that address a specific disease or condition, for example, pain; ICF, International Classification of Functioning; QoL, Quality of Life.

^a ICF area is determined by the person's goals.

^b Questionnaire tests parents understanding of Autism.

4. Discussion

The purpose of this scoping review was to identify outcome measures for activity, participation and quality of life used to evaluate community rehabilitation services in rural and remote Australia. A comprehensive overview of measures was generated from 27 studies that yielded 40 different outcome measures to evaluate a diverse range of services across rural and remote Australia. Few measures, however, were used consistently across studies, and few measures were consistently associated with a significant change. Thus, it would not be possible to draw any reliable conclusions about the benefits of any single intervention or service, to compare the benefits of one service over another, or to benchmark against national standards. Therefore, if informed decisions are to be made about the best services to deliver, the findings of this study highlight the need for attention to quantitative outcome measurement to evaluate community rehabilitation services for rural and remote Australia.

Measures of activity and participation and quality of life are important in any service evaluation because they demonstrate the impact of services on the individual's day‐to‐day life. Outcome measures and testing protocols used, however, need to be consistent across services, to allow for comparison within and between services and to allow pooling of data from similar services and benchmarking against national standards. This could be achieved by creating a core data set of quantitative outcome measures to be used consistently and uniformly across services. To create a high‐quality core data set, specifically for the rural and remote context, a consensus‐based approach could be used that includes the engagement of key stakeholders to ensure relevance and to accommodate diversity. A proposed core data set should draw on existing mainstream frameworks to facilitate benchmarking and incorporate flexibility to accommodate heterogeneity [47]. A core data set will provide a solid foundation for continuous quality improvement and evidence‐informed decision‐making [48].

Most measures in this review were used to evaluate single interventions (e.g., continence management) for a single condition (e.g., multiple sclerosis) or lifestage (e.g., older people) or to compare one mode of delivery to another (e.g., telehealth to in‐person). While such evaluations play an important role in continuous quality improvement, they reflect little of the generalist nature of services delivered in the rural and remote context, that is, services delivered to people with a range of conditions, clinical needs, service types and across the lifespan. Evaluation of service models is particularly important, as the success of an intervention is often dependent on the extent to which the service model matches the needs and aspirations of a rural or remote community. For example, the success of a continence management intervention provided for First Nations men and women is likely to depend on who provides the service, how it is provided, where it is provided, and how well the model matches the community's cultural norms [49]. The success of a service model may also arise from innovative uses of scarce resources available. For example, services for older people provided by allied health students alongside local community partners amid a scarcity of allied health professionals [50]. Furthermore, it is important to demonstrate the benefits of new models that are emerging to ensure that evidence‐informed investment can replace historically based investment in traditional models of service delivery that lack an evidence base [51].

The environment in which the population lives, and in which services are delivered, is the most notable difference between rural, remote and urban settings. The premise on which the ICF is based is that an individual's level of function or disability, is determined by the dynamic interaction between their health condition and their personal and environmental factors [10]. Hence, if increasing function and reducing disability includes eliminating environmental barriers and strengthening environmental facilitators, environmental factors need to be monitored and evaluated so that strategies can be designed accordingly [52]. The Measure of the Quality of the Environment (MQE) offers a promising tool because it provides a mechanism for monitoring the impact of environmental factors on the individual's performance [53]. The MQE measures an individual's perception of their physical and social environment and factors that facilitate or impede their performance of activities of daily living and social roles across six domains: social network; income; governmental and public services; physical environment and accessibility; technology; and political orientation. With its clear links to the ICF domains, this measure could inform strategies to address the complex physical, social and attitudinal environmental conditions that underpin the overrepresentation of disease and disability in rural and remote communities [53].

Services for First Nations people were rarely considered in this review, nor their conceptual understandings or cultural and historical experiences of disability [54], although with one notable exception [46]. This may be due to a lack of community rehabilitation services delivered in remote First Nations communities, a lack of service evaluations or a lack of outcome measures that have been validated for use with First Nations peoples [55]. With the focus of this review on individual‐level service delivery, the findings may not reflect the broad needs of First Nations communities who have consistently requested services that support well‐being for the whole community and build community capacity and their local workforce [50, 56]. The outcomes of community‐level work are not routinely measured in the evaluation of community rehabilitation services, despite the importance of this work to First Nations people and perhaps more generally [57]. Importantly, cultural considerations and the broader social and cultural determinants of health and their impact on health and disability for First Nations people [54] need to be taken into account during the development and implementation of data sets and evaluation frameworks.

4.1. Limitations

In this review, it is probable that studies have been missed, given the broad definition of community rehabilitation services in the context of rural and remote services. Furthermore, the search strategy was limited to articles published in peer‐reviewed journals. Without including the grey literature, it is possible that evaluations conducted in‐house or by external consultants have been missed.

5. Future Research

Community allied health services in rural and remote communities require more consistent and robust evaluation. The use of quantitative outcome measures to evaluate rural and remote community rehabilitation services is essential to support continuous quality improvement and build evidence to support interventions and service models that are responsive to community needs and aspirations. To ensure robust evaluation is possible, an evaluation framework that is designed to measure what matters for rural and remote communities is required. The framework must include a consensus‐based core data set that accounts for heterogeneity and enables pooling of data for comparison between interventions, service models and national standards. Stakeholders who could guide the development of such a data set must include consumer and community representation. Such an approach will provide a sound foundation for evidence‐informed investment.

Quantitative outcome measures of activity, participation and quality of life, including existing and emerging measures, need to lie at the heart of any evaluation framework to measure the impact on those individuals for whom the service is designed. To recognise the importance of innovation to achieve those outcomes, the framework must also include information about the environment in which people live and in which services are delivered. For instance, inclusion of measures of the social determinants of health that include service access that reflects actual rather than intended workforce availability at a given point in time [58]. Notwithstanding, measures must be easy to apply by the workforce on the ground, who will also often be experiencing workforce constraints. Hence, all measures must be quick and easy to administer, record, analyse and report.

6. Conclusion

A comprehensive review of quantitative outcome measures for evaluating community rehabilitation services was completed. Findings indicate that a core data set, with a common framework for evaluating community rehabilitation services, is required to ensure consistency in measurement, that reflects rural and remote service models, and takes into account the environment in which services are delivered.

Author Contributions

Alice Cairns: conceptualization, investigation, funding acquisition, writing – original draft, writing – review and editing, visualization, validation, methodology, formal analysis, software, project administration, resources, data curation, supervision. Ruth Barker: resources, supervision, data curation, project administration, formal analysis, writing – review and editing, visualization, validation, methodology, writing – original draft, funding acquisition, investigation, conceptualization, software.

Disclosure

This research is supported through the Tropical Australian Academic Health Centre (TAAHC), seed funding grant. This work has not previously been published.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Data S1.

Data S2. Glossary of outcome measures included in the review with original references for the measures.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.