A review of instruments assessing participation in individuals with spinal cord injury

Abstract

Study design

A systematic search of the literature.

Objectives

To critically review instruments which assess participation in persons with spinal cord injury (SCI).

Setting

Vancouver, British Columbia

Methods

Four electronic databases (MEDLINE/PubMed, CINAHL, EMBASE, PsychInfo) were searched for studies published between 1980 and March 2008. Instruments were included if information was published in English in at least one peer reviewed journal on its measurement properties (reliability, validity and/or responsiveness) in a sample which included adults with SCI. Instruments were evaluated using criteria proposed for disability outcome measures.

Results

Six instruments were included: Craig Handicap Assessment and Reporting Technique (CHART); Impact on Participation and Autonomy Questionnaire (IPA); Assessment of Life Habits Scale (Life-H); Occupational Performance History Interview (OPHI); Physical Activity Recall Assessment for People with Spinal Cord Injury (PARA-SCI); and Reintegration to Normal Living Index (RNL). Evidence supporting the reliability of the instruments was reported for 4 of the 6 instruments and was adequate except poor proxy reliability was noted in two CHART domains. Validity was assessed in all the instruments. Only the Life-H and CHART have been compared to each other. No evidence was available on the responsiveness of the instruments.

Conclusion

The CHART instrument has the most evidence supporting its measurement properties. More recently participation has been operationalized to consider the person’s perspective and in the future more evidence will be available for instruments such as the IPA in persons with SCI.

Sponsorship

Rick Hansen Foundation, Ontario Neurotrauma Fund

Introduction

Disability has a tremendous impact on both the individual and society. It is expected that disability rates will continue to rise as the population ages, as survival from injuries and diseases increases with medical advances and as the surveillance of disability improves.¹ With the realization that disability is not just a medical issue there has been considerable work to try and understand and measure the behavioral and social impacts of disability.

Several conceptual models have been developed to assess disability dating back to Nagi’s model of the disability process in 1965² and more recently the International Classification of Functioning, Disability and Health (ICF)³ published by the WHO in 2001. Although the terminology in these models may differ, the idea of measuring a person’s participation in his or her life activities is common to all models and helps to understand the impact of disability.

Over time there have been some significant changes in how the concept of participation is operationalized. In the International Classification of Impairments, Disabilities and Handicaps (ICIDH)⁴, the predecessor to the ICF, the term handicap is used instead of participation. Handicap is defined as “the disadvantage of a given individual resulting from an impairment or a disability, that limits or prevents the fulfillment of a role that is normal (depending on age, sex and social and cultural factors) for that individual” (page 182)⁴. Instruments which assess handicap based on the ICIDH use objective or quantitative information and a comparison is made with societal norms.⁵ Handicap is measured by determining how much a person deviates from roles fulfilled by an able-bodied member of society.⁶ Not all instruments assessing objective aspects of participation assess handicap. For example, the frequency with which a person engages in recreation activities could be assessed but not necessarily compared to societal norms.

In the ICF the term handicap was replaced with the term participation due to its negative connotations.⁵ Participation is defined as “the involvement in a life situation” and a participation restriction is defined as “a problem an individual may experience in involvement in life situations” (page 10).³ In this definition participation refers more to the personal fulfillment of roles rather than fulfilling roles deemed important by society, which is an important shift.⁷ In 1994 Carr and Thompson⁸ suggested that a handicap instrument should include the individual’s experiences and preferences to understand the needs and problems experienced and this is referred to as assessing subjective aspects of participation.⁵ Since the ICIDH terminology is no longer used the term participation will be used in this paper. The concept handicap is equivalent to measuring objective aspects of participation and using societal norms to determine how much a person with SCI is disadvantaged.

Participation is cited as being important to a person’s quality of life and well-being and is an important outcome in rehabilitation.^9,10 It is therefore critical that instruments are available to measure participation. Since the measurement properties of an instrument are not an intrinsic property of an instrument but rather provide evidence as to the meaning of scores for a particular health condition^11,12 it is necessary that participation instruments are tested with persons with SCI. The purpose of this paper is to provide an overview of participation instruments assessed in persons with SCI and to critically evaluate their measurement properties.

Materials and Methods

Search Strategy

A review of instruments assessing participation in individuals with SCI was conducted as a part of the Spinal Cord Injury Rehabilitation Evidence (SCIRE) project. Instruments were identified using a key word search of multiple electronic databases (MEDLINE/PubMed, CINAHL, EMBASE, PsychInfo), reviewing the references in the articles identified from the search, and from hand searches of SCI journals from 1980 to March 2008. The search phrase ‘spinal cord injury’ was used in each database, while the following terms varied depending on the database searched: validation studies; instrument validation; external validity; internal validity; criterion-related validity; concurrent validity; discriminant validity; content validity; face validity; predictive validity; reliability; inter-rater reliability; intra-rater reliability; test-retest reliability; reproducibility; responsiveness; sensitivity to change; evidence-based medicine; outcome measures; clinical assessment tools; scales; and measures.

Inclusion Criteria

A review team consisting of researchers and clinicians with expertise in SCI reviewed all the articles identified from the systematic searches. Instruments were included if there was information published on its measurement properties (reliability, validity and/or responsiveness) in adults with SCI and published in at least one peer reviewed journal. Articles were only included if they were published in English. Instruments were considered to assess participation if the domains covered 2 or more of the following ICF chapter headings: domestic life (chapter 6); interpersonal interactions and relationships (chapter 7); major life areas (chapter 8); and community, social and civic life (chapter 9). Instruments primarily assessing mobility (chapter 4) and self-care (chapter 5) were considered to assess the ICF concept activity.¹³

Data Abstraction

Data were abstracted based on the eleven criteria proposed by Andresen¹⁴ for assessing instruments used in people with disability. The definitions for five criteria (reliability, validity, responsiveness, item/instrument bias, measurement model) were applied to instruments tested in persons with SCI. Reliability and validity each have three types and so a total of nine measurement properties were assessed and then assigned a grade (see Table 1) based on the evidence available for persons with SCI. Data pertaining to reliability, validity, responsiveness and the measurement model of the instrument were abstracted by two team members. Item/instrument bias was assessed by reviewing the original articles describing the instrument as well as articles included in this review to determine if individuals with SCI examined the instrument content. When the grades varied across studies (e.g. Grade A in one study and Grade B in another study) the results are presented as a range. Information on the remaining six criteria [conceptual model; norms available; respondent burden (i.e. time to complete); administrative burden (i.e. scoring/interpretation); alternate forms (i.e. modes of administration); language adaptations] were obtained but not graded since the information is generic. Study specific sample characteristics were also included.

Table 1.

Criteria for Evaluating the Instruments

Criteria	Description	Evaluation
Reliability	The degree to which an instrument is consistent or free from random error.	Test re-test repeatability (ICC and kappa) A = ≥0.75 B = >0.40, <0.75 C = ≤0.40
	Types include: test-retest repeatability, internal consistency, and comparison with proxy responses	Internal consistency (Coefficient Alpha) A = ≥0.80 B = <0.80, >0.70 C = ≤0.70
		Proxy responses (ICC) A = ≥0.75 B = >0.40, <0.75 C = ≤0.40
Validity	The degree to which an instrument measures what it intends to measure. Types include: factorial structure (assess dimensionality); convergent correlations (comparisons with other measures); and discriminant (differentiate based on known groups).	Factorial structure (exploratory or confirmatory factor analysis, Rasch analysis) A = confirmed, Rasch analysis is good B = factorial analysis is good or Rasch has some problems C = inadequate statistical analysis
		Convergent correlations A = ≥0.60 B = >0.30, <0.60 C = ≤0.30
		Discriminant (differences by means or %) A = strong, in expected direction B = moderate or conflicting evidence C = weak
Responsiveness	The ability of an instrument to measure important changes following intervention(s).	Clinical criteria for change A = strong, in expected direction B = moderate or conflicting evidence C = weak or based only on statistical methods
Item/Instrument bias	Assesses in practical terms if individual questions or summary scores are biased for individuals with SCI	A = persons with SCI reviewed the instrument and acceptability is published B = there is adequate face-validity to support low bias C = bias is evident or tested
Measurement model	Examines if there are problems with floor effects (lowest level of ability) or ceiling effects (highest level of ability).	The instrument has scales or measures where 20% of persons with SCI are grouped at scoring extremes. Also can consider the score distribution (mean and standard deviation). A = no problems B = few or marginal problems C = substantial skewing of scales/measures

Abbreviation: ICC, intraclass correlation coefficient

Results

The systematic search identified eight instruments: Athletic Identity Measurement Scale (AIMS); Craig Handicap Assessment and Reporting Technique (CHART)⁶; Employment Questionnaire (EQ); Impact on Participation and Autonomy Questionnaire (IPA)⁷; Assessment of Life Habits Scale (Life-H)¹⁵; Occupational Performance History Interview (OPHI)¹⁶; Physical Activity Recall Assessment for People with Spinal Cord Injury (PARA-SCI) ¹⁷; and Reintegration to Normal Living Index (RNL) ¹⁸. The AIMS and EQ were excluded based on the definition of participation used in this review since these instruments primarily assess athletic/recreation activities (ICF Chapter 9) and employment (Chapter 8) respectively.

Instrument Description

An overview of the six instruments included in the systematic search is described in Table 2. Each of the instruments were developed using different conceptual models. The CHART was designed to measure level of handicap as defined in the ICIDH⁶ and the IPA was based on beta version of the ICF called the ICIDH-2¹⁹. A conceptual model very similar to the ICF, called the Disability Creation Process Model (DCP) was used to develop the Life-H.²⁰ The OPHI was developed to be used with the Model of Human Occupation and the RNL was designed to measure the concept of re-integration to normal living²¹. The PARA-SCI measures physical activity in persons with SCI¹⁷, making it the only condition-specific instrument. In terms of the content, Table 1 provides an overview of the domains covered. All of the instruments assess aspects of mobility (ICF chapter 4), daily activities (ICF chapters 5/6), recreation/leisure (ICF chapter 9) and work/education (ICF chapter 8). Inter-personal relationships (ICF chapter 7) are not specifically mentioned in the PARA-SCI or the OPHI.

Table 2.

Description of Participation Instruments

Instrument	Conceptual model	Questions	Domains	Mode of administration	Time to complete	Other languages	Scoring
CHART	ICIDH	27	physical independence; mobility; occupation; social integration; economic self-sufficiency	interview, self-report	15 min	yes	scoring procedure (original) is available and compute a score from 0 to 100 for each domain; total maximum score is 500
CHART	ICIDH	32	also includes cognitive independence	interview, self-report	see above	see above	add another score (revised) from 0 to 100 for this domain
IPA	ICIDH-2	39a	autonomy outdoors; autonomy indoors; family role; social relations; paid work/education.	self-report	30 min ±15min	yes	5 participation domain scores and 5 problem experience scores are produced by summing questions; 2 overall scores (participation and problem experience)
Life-H (Short Form)	DCP	77	nutrition; fitness; personal care; communication; housing; mobility; responsibilities; inter-personal relationships; community life; education; employment; recreation	interview, self-report	20–30 min	yes	ratings for difficulty and assistance are used to derive a 9- point accomplishment scale; a score for each domain and an overall score is reported; can also report satisfaction for each question
Life-H (Long Form)	see above	240	see above	interview, self-report	20–120 min be reported	see above	see above
OPHI (version 1.0) rating scale	model of human occupation	20	organization of daily living routines; life roles; interests, values and goals; perception of ability and responsibility; environmental influences.	interview	21 min	NR	report past and present adaptive status separately; can derive domain and overall score
PARA-SCI	physical activity	based on activity level	cumulative activity; leisure-time physical activity; lifestyle activity	interview	20–30 min	NR	report average number of minutes per day (mild, moderate, heavy, total) for leisure-time and lifestyle activity; combine 2 domains to obtain cumulative activity
RNL	concept of reintegration	11	mobility; self-care; daily activity; recreational activity; family roles	self-admin	10 min	yes	each question is scored from 0 to 10; 11 questions are summed and converted to a score out of 100

Note:

^a– recently two additional questions were added to the IPA and the new total number of questions is 41⁴⁷

Abbreviations: CHART, Craig Handicap Assessment and Reporting Technique Short Form; DCP, Disability Creation Process; ICIDH, International Classification of Impairments, Disabilities and Handicaps; ICIDH-2, a draft version of the International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy Questionnaire; Life-H, Assessment of Life Habits Scale; NR, not reported; OPHI, Occupational Performance History Interview; PARA-SCI, Physical Activity Recall Assessment for People with Spinal Cord Injury; RNL, Reintegration to Normal Living Index

There are differences in the types of information asked in six instruments. The CHART and the PARA-SCI include questions measuring objective or quantifiable data, for example “on a typical day, how many hours are you out of bed” (CHART) or number of minutes engaged in moderate intensity leisure-time physical activity (PARA-SCI). In the CHART quantifiable information is used to determine the amount the respondent deviates from roles generally fulfilled by able-bodied members of society. The PARA-SCI quantifies activity intensity within leisure, lifestyle and cumulative activity to measure physical fitness. Since these two instruments use quantifiable information they are considered to assess objective aspects of participation.

The remaining four instruments primarily assess subjective aspects of participation. Questions in these instruments require judgement from the respondent, for example asking about difficulty in “maintaining relationships” (Life-H), “my chances of fulfilling my role at home as I would like are” (IPA) and “I am able to take trips out of town as I feel are necessary” (RNL). The OPHI rating scale is the only instrument in which the assessment is provided by a clinician following an interview with the individual, which assesses his or her behaviour. All four instruments include the environment as either part of the question (IPA, RNL) or separately (Life-H, OPHI). However in all four instruments the environment questions are included in the participation score. The Life-H also includes additional questions which ask about level of satisfaction for each aspect of participation.

There is considerable variation in the number of questions, ranging from 11 in the RNL to 240 in the Life-H (Long Form) requiring anywhere from 10 to 120 minutes to complete. Two instruments must be administered by interview (PARA-SCI, OPHI) and the remaining four instruments can be self-administered. Four of the instruments have been translated into other languages (CHART, Life-H, IPA, RNL). Scoring for the instruments include domain as well as summary scores with the exception of the RNL which only reports an overall score. It has been recommended that the CHART is reported using the domain scores since the total score can provide a misleading assessment.²² Normative data is available for the CHART.^22,23

Measurement Properties

Evidence pertaining to the reliability of the instruments is provided in Table 4. Only two instruments had information on the internal consistency (IPA, Life-H). Internal consistency for the IPA participation experience was 0.94 and 0.82 for the problem experience and these estimates were derived from the person separation index in a Rasch analysis which is similar to internal consistency²⁴. Test re-test reliability was assessed in CHART, Life-H and PARA-SCI. The CHART had the most evidence with three studies assessing test re-test reliability.^6,25,26 The test re-test reliability was higher in the Life-H (Short Form) (ICC=0.83) compared to the Life-H (Long Form) (ICC=0.70).¹⁵ In the PARA-SCI the leisure-time physical activity domain (ICC=0.72) had a lower test retest reliability compared with the lifestyle activity (ICC=0.78) and cumulative activity (ICC=0.79).¹⁷ Only the CHART assessed proxy inter-rater reliability and it was consistently lower compared with the test re-test reliability as would be expected. Within the CHART mobility had the highest proxy inter-rater reliability (r=0.84, ICC=0.86) and the domains with the lowest proxy inter-rater reliability included social integration (r=0.29) and cognitive independence (ICC=0.34).^27,28

Table 4.

Reliability of the Participation Instruments

Instrument	Internal consistency	Test Re-test time period	coefficients	Inter-rater time period	coefficients
CHART	NR	1 week	overall r=0.936; physical r=0.926; mobility r=0.956; occupation r=0.896; economic self-sufficiency r=0.806; social integration r=0.816	1 week (proxy)	overall r=0.836; physical r=0.806; mobility r=0.846; occupation r=0.816; economic self-sufficiency r=0.696; social integration r=0.296
		2 weeks	overall ICC=0.8726; physical ICC=0.7126; cognitive ICC=0.7026; mobility ICC=0.8926; occupation ICC=0.7226; economic self-sufficiency ICC=0.8126; social integration ICC=0.7326	2 weeks (proxy)	overall ICC=0.8427; physical ICC=0.6927; cognitive ICC=0.3427; mobility ICC=0.8627; occupation ICC=0.6027; economic self-sufficiency ICC=0.5927; social integration ICC=0.5727
		3 weeks	overall r=0.7825; physical r=0.5325; mobility r=0.9625; occupation r=0.8625; economic self-sufficiency r=1.0025; social integration r=0.7825
IPA	participation experience =0.94a,b29 problem experience=0.82 a,b29	NR		NR
Life-H	short form=0.91a20 short form>0.8248 long form>0.9048	2 weeks	short form ICC=0.8315; long form ICC=0.7015	NR
OPHI	NR	NR		NR
PARA-SCI	NA	1 week	cumulative activity ICC=0.7917; leisure-time physical activity ICC=0.7217; lifestyle activity ICC=0.7817	NR
RNL	NR	NR		NR

Notes:

^a– person separation reliability obtained from Rasch analysis

^b– estimates are prior to removal of misfitting questions

Abbreviations: ICC, intraclass correlation coefficient; NA, not applicable, NR, not reported

Validity was assessed in all of the instruments (see Table 5). Rasch analysis was used in testing the CHART, IPA, Life-H which provided evidence for the factorial structure.^6,20,29 Convergent validity was assessed in all of the instruments with the exception of the IPA. Two instruments included in this review (CHART, Life-H) were compared by mapping all of the Life-H (Short Form) questions into the corresponding domains of the CHART.³⁰ The Life-H and CHART physical independence domains had the highest correlation (ρ=0.76) and social integration domains had the lowest (ρ=0.14).³⁰ Discriminant validity was assessed in 4 of the 6 instruments. There were differences among the 4 instruments. For example the CHART²² and PARA-SCI leisure time physical activity¹⁷ were able to discriminate among subjects of various ages but the Life-H (Short Form)³¹ was generally not able to. Only one instrument (PARA-SCI)¹⁷ involved persons with SCI in the development of the content however, persons with SCI were part of the initial testing for the CHART⁶, IPA¹⁹ and Life-H¹⁵. Rehabilitation clinicians were asked to review the content in all six instruments. Floor effects were noted in PARA-SCI for high intensity for lifestyle activity¹⁷ and ceiling effects were observed in the domains of the Life-H¹⁵ and CHART²². There were also some floor effects in the IPA problem experience domain and in the IPA participation experience and problem experience primarily in persons with less impairment, but were less than 10%.²⁹ None of the instruments assessed responsiveness.

Table 5.

Validity and Related Measurement Properties of the Participation Instruments

Instrument	Validity - Factorial structure	Validity - Convergent correlations	Validity - Discriminant groups	Item bias	Measurement model
CHART	Rasch analysis of questions within subscales indicate good fit, item t-values <1.2 and person-fit statistics mean=0 and standard deviation=1.26	see Life-H	age, race/ethnicity education level, and work/school status predicted all 5 subscales and CHART total score22; clinicians’ ratings of subjects’ handicap (low versus high) correctly classified subjects’ based on CHART scores except economic self-sufficiency6; able to discriminate subjects employed versus unemployed in all domains except social integration25; able to discriminate across health conditions and disability status (physical and cognitive based on FIM)26	persons with SCI were included in the initial testing6	ceiling effects ranged from 23%–63% of sample for social integration and economic self-sufficiency domains for all types of SCI, ceiling effects noted in all domains for individuals with a motor score >5022
IPA	Rasch analysis supports 2 unidimensional participation and problem experience scales after removal of 4 and 2 questions respectively29	NR	NR	content developed by various disciplines and included persons with disabilities19; persons with SCI were included in the initial testing19	ceiling effects noted in 4% sample for participation experience scale; floor effects and ceiling effects noted in 7%and 9% of sample respectively for the problem experience scale29
Life-H	PCA 7 factors 1st factor 30.3% of variance (eigenvalue 17.3), in Rasch analysis multidimensionality noted in PCA of residuals20	questions mapped to CHART domains physical ρ=0.76 mobility ρ=0.33 occupation ρ=0.36 social integration ρ=0.1430; agreement between clinicians’ rating of difficulty and empirical data r=0.8920	able to discriminate between injury severity (complete/incomplete tetraplegia/paraplegia)31; age and years post injury only negatively affect a few life habits31	reviewed by rehabilitation clinicians who work with persons with SCI15,20	ceiling effects noted in 5 domains based on mean and SD values for accomplishment scores15
OPHI	NR	CES-D r=−0.1916; MPI for 6 pain scales r=0.28–0.5516	able to discriminate between past and present OPHI scores16	reviewed by occupation therapists therapists and field tested with various health conditions49	NR
PARA-SCI	NR	leisure time physical activity and cumulative activity were positively correlated with measures of aerobic fitness (r=0.26–0.35) and muscle strength (r=0.22–0.36)50; lifestyle activity were not associated with aerobic or muscle strength50; PARA-SCI intensities (moderate, heavy, total) and indirect calorimetry measures ranged r=0.63–0.88, mild intensity cumulative activity was r=0.27but not significant17	leisure time physical activity able to discriminate groups based on age, sex, gym/sports membership and frequency of participation; lifestyle and cumulative activity not able to discriminate50	developed in consultation with persons with SCI, rehabilitation clinicians, attendants and family members17	floor effects noted lifestyle activity at intensity with 64% subjects reporting high intensity life activity17
RNL	NR	QLI (r=−0.65)38; ISNCSCI MS (r=−0.20)38a; RSES (r=−0.48)38; RIE (r=−0.04)38	NR	content reviewed by rehabilitation clinicians, persons with disabilities (i.e. other than SCI) family members, healthy lay person21	mean RNL score=23.05+13.54 suggesting no floor or ceiling effects38

Notes:

^a– not significant

Abbreviations: 1RM, maximal load that can be lifted in one repetition; CES-D, Centre for Epidemiologic Studies Depression Scale; FIM, Functional Independence Measure; ISNCSCI MS, International Standards for Neurological Classification of Spinal Cord Injury Motor Score; MPI, Multidimensional Pain Inventory; NA, not applicable; NR, not reported; PCA, principal components analysis; QLI, Quality of Life Index; RIE, Rotter’s Internal-External scale; RSES, Rosenberg Self-Esteem Scale; SD, standard deviation

A summary of the grades assigned for each of the eight measurement properties is included in Table 6. Responsiveness is not included since no information was available. The CHART had the most evidence supporting its measurement properties and grades were assigned for seven of the eight measurement properties achieving a good to moderate rating in all categories. The OPHI and RNL had the least amount of evidence, with only three measurement properties assessed in each instrument.

Table 6.

Summary of the Measurement Properties

Instrument	Reliability - Internal consistency	Reliability - Test re-test repeatability	Reliability - Proxy responses	Validity- Factorial structure	Validity - Convergent correlations	Validity - Discriminant groups	Item bias	Measurement model
CHART	--	A-B	A-C	A	A-B	A	B	C
IPA	A	--	--	A	--	--	B	A
Life-H	A	A-B	--	B	A-B	A	B	B
OPHI	--	--	--	--	B	B	B	--
PARA-SCI	--	A-B	--	--	A-C	B	A	B
RNL	--	--	--	--	A-B	--	B	A

Note:

-- indicates that no information is available for individuals with spinal cord injury

Discussion

Even though participation is an important goal of the rehabilitation process, it is not often measured.²² This paper included six participation instruments that have been assessed in persons with SCI and each instrument was reviewed considering critical measurement criteria. Information from this review may assist clinicians in selecting an appropriate measure, identifying areas of future research and hopefully will increase awareness about the importance of this rehabilitation outcome.

In order to be considered as a measure of participation, instruments were required to have domains that cover a minimum of two ICF chapters from 6 to 9. This operational definition of participation is just one way to differentiate activity from participation. In the ICF, the user is provided with a list of activity and participation domains (ICF chapter headings) and can allow for partial or no overlap within either the domains or categories included within each domain.³ It has been suggested that activity and participation are distinct concepts that must be differentiated conceptually and operationally.³² Whiteneck³² stated that activity is at an individual level and performed alone, where as participation is at a societal level and performed with others. Whereas Nordenfelt³³ recommended not distinguishing between activity and participation and instead suggested combining them and refer to them as ‘actions’ based on philosophical action theory.³⁴ Future work must continue to clarify if activity does differ from participation and then if these are two distinct concepts, how they differ.

Although the ICF was used to classify the participation instruments only the IPA was based on a draft version of the ICF, referred to the ICIDH-2. Since the conceptual models used in developing the instruments differ it is difficult to compare the content of the instruments using the ICF chapter headings. Based on the domains within the instruments it appears that they all assess aspects of mobility, daily activities, recreation/leisure and work/education. Since the PARA-SCI was developed to assess physical activity and the OPHI was developed to be used with the model of human occupation, it is not too surprising that inter-personal relationships are not specifically covered in these instruments. In order to obtain a better understanding of the content it would be useful to link the individual questions to the ICF to determine what ICF categories each question covers. The CHART and the IPA have been linked to the ICF by Perenboom and Chorus⁹ and findings from this study highlighted the differences in how activity was distinguished from participation. More recently Cieza et al.^35,36 published methodology on how to link content within instruments to the ICF and future studies should use these methods to link the content of the participation instruments included in this review.

In terms of how participation was operationalized, two instruments assess objective aspects of participation and four assess subjective aspects of participation. Quantifiable information obtained from participation instruments is helpful to describe participation restrictions in epidemiological studies and enables comparisons to be made between groups or even societies. Since the PARA-SCI was developed specifically for persons with SCI, the comparisons using scores derived from this instrument will provide meaningful information that can be used in intervention studies as well as to create public health guidelines for persons with SCI. It is important to consider the source and relevance of the data when using a participation instrument that utilizes normative data, such as the CHART. For example, the CHART uses normative data from able-bodied individuals in the US population and this must be reviewed to ensure it is relevant in other countries.

The four instruments that assess subjective aspects of participation differ in the wording used to measure areas of participation. The Life-H includes questions asking about difficulty and type of assistance as well as satisfaction for each life habit. In the RNL the respondent rates how he or she has re-integrated back to the various roles in their life. The IPA assesses perceived participation and autonomy as well as problems experienced when participating. The problem experience domain in the IPA allows rehabilitation professionals to set goals since not all restrictions are deemed equivalent by the respondent.⁷ The OPHI rating scale is different from the three other instruments since the interviewer rates how well the person has adapted and is able to function following an interview. Although this instrument uses an external person to rate adaptive status it was considered to be measuring subjective aspects of participation since the interviewer elicits subjective information from the interviewee and uses this information to determine the rating. Future research should compare these four instruments to each other using qualitative and quantitative studies to determine if the subjective information obtained from these instruments differs and which instrument is superior.

The amount of evidence available on the instruments’ measurement properties varied, with the CHART having the most evidence and the OPHI and RNL having the least evidence. In terms of reliability, internal consistency has only been assessed in the IPA and Life-H and the results suggest these are very good. Test re-test reliability in the CHART, Life-H and PARA-SCI varied between the studies, however, overall the results suggest these instruments provide stable information. Proxy reliability has only been assessed in the CHART and was higher at one week compared to two weeks following the initial assessments.^6,27 The domains social integration and cognitive independence appear to be the least reliable assessment when provided by proxy ratings and it is because these domains may vary with time or not be as easy to observe behaviour.²⁷ It is therefore questionable whether instruments assessing subjective aspects of participation instruments could be completed by proxy since subjective information is not considered to be as reliable as objective information and in order to be valid must be reported by the person.³⁷

Further advances in the conceptualization and the use of modern measurement methods such as Rasch and other item response theory models will enhance how participation is operationalized and measured. Of the instruments which have been tested using Rasch analysis, the IPA and Life-H both contained questions which did not fit the Rasch model. By removing misfitting items in the IPA the participation experience and problem experience scales were each found to be unidimensional.²⁹ If instruments such as the Life-H are multidimensional it supports the use of domain scores as opposed to overall scores for participation.²⁰

Convergent validity has been assessed in five of the six instruments and was graded in this review by considering how the constructs operationalized in the instruments would be expected to differ. The RNL was compared to the Quality of Life Index and a correlation of r=−0.65 was reported which is not surprising given the similarities between the questions used to measure participation and quality of life.³⁸ Only the CHART and Life-H have been compared to each other and the low correlation (ρ=0.14) demonstrates the differences in objective versus subjective aspects of participation.³⁰ Various types of known groups have been assessed. The CHART and the PARA-SCI were able to differentiate individuals based on the age but this was not evident in the Life-H. Objective aspects of participation such as combined income are included in the CHART which is more likely affected by age however these questions are not included in the Life-H.

The criterion item bias assesses the qualitative aspects of how the instrument was developed in this paper. The PARA-SCI was the only instrument which involved persons with SCI in the developing the content. However, the CHART, IPA and Life-H both included persons with SCI in the initial testing which support their suitability for persons with SCI. The involvement of persons with SCI in the development of new instruments is critical. A study by Gray et al.³⁹, which included persons with SCI, described how the findings from qualitative focus groups regarding what participation means as well as what are the barriers and supports to participation formed the basis for developing the Participation Survey/Mobility (PARTS/M). Developers of new instruments will gain tremendous insight and produce more meaningful measures by engaging individuals who live with conditions such as SCI.

Finally, the measurement model of the participation instruments was assessed considering floor and ceiling effects. If information was not reported on the percentage of the sample with extreme scores then the mean and standard deviation was considered. Floor effects were noted in the PARA-SCI and ceiling effects were noted in the domain scores for the CHART and Life-H. If participation instruments are going to be used to assess rehabilitation interventions or changes over time due to aging then floor and ceiling effects may prevent the these changes from being captured. To date, none of the instruments have reported evidence assessing responsiveness.

Recommendations

A comprehensive instrument that contains objective and subjective information and has minimal floor and ceiling effects would be ideal. Moreover because participation is considered one of the most coveted rehabilitation outcomes existing and new instruments require evidence on responsiveness so that clinicians and researchers can assess if their interventions are making a difference.

Currently, the CHART has been tested the most in persons with SCI and includes questions covering content from chapters 6 to 9 of the ICF. However, the CHART only assesses objective aspects of participation and uses normative data to quantify the participation restriction. Instruments such as the Life-H, IPA and RNL also include questions covering content from chapters 6 to 9 of the ICF and assess subjective aspects of participation. To date, there is more evidence supporting the measurement properties of the Life-H compared to the IPA or RNL. No information on the measurement properties of the Life-H satisfaction questions was reported and this should be tested in future studies. In addition, there has only been one study which compared the Life-H and the CHART and future research should continue to concurrently assess instruments included in this review to determine how they compare. Since the ICF was published in 2001 a number of new participation instruments have been developed using the ICF^39–46. Some of these instruments have been assessed in persons with SCI^{39,41–43,45} but no data on the measurement properties were specified for this group and so they were not included in this review. Future studies reporting the measurement properties of these instruments in persons with SCI will provide the evidence necessary to use these instruments in research and clinical practice.

Conclusions

The instruments included in this review vary in content as well as how participation is operationalized. It is therefore important to first determine what information is required about the concept participation before selecting an instrument. Future work establishing the measurement properties of these six instruments as well as newly developed participation instruments will be essential to ensure the measures are reliable, valid and responsive for assessing participation in individuals with SCI. This will enable clinicians and researchers to select appropriate measures and obtain meaningful information regarding this important rehabilitation outcome.

Table 3.

Description of Study Subjects

Instrument	Country	Number of subjects	Age (mean, range yrs)	Gender (% male)	Type of SCI	Time since injury (mean, range yrs)
CHART	USA	135	mean: 33 yrs	84%	30% complete tetraplegia	7 yrs
(27-questions)6			range: 16–74 yrs		28 % incomplete tetraplegia	2–35 yrs
					31% complete paraplegia
					10% incomplete paraplegia
(32-items)26,27	USA	224 SCI27	41% ≤29 yrs	75%	NR	17% < 1 yr
		236 SCI26	54% 30–64 yrs			42% 2–4 yrs
			5% ≥65 yrs			41% ≥5 yrs
		100 proxies	NR	NR	NA	NA
		n=29 spouse
		n=35 parent
		n=8 other family
		n=28 other
(27-item)22	USA	1998	57% <31 yrs	82%	9% C1-C4 complete tetraplegia	25% 1 yr
			21% 31–40 yrs		9% C5-C8 complete tetraplegia	22% 2 yrs
			11% 41–50 yrs		9% C1-C4 incomplete tetraplegia	20% 5 yrs
			6% 51–60 yrs		19% C5–8 incomplete tetraplegia	15% 10 yrs
			4% 61–70 yrs		32% complete paraplegia	12% 15 yrs
			2% >70 yrs		17% incomplete paraplegia	7% 20 yrs
(27-item)25	Japan	n=293	38.3±11.9 yrs	84%	54% cervical	8.7±6.6 yrs
			range: 18–60 yrs		42% thoracic and below	range: 12–58 yrs
					4% missing data
IPA29	Sweden	n=161	52±18.2 yrs	63%	38% tetraplegia	NR
			range: 17–84 yrs		62% paraplegia
Life-H	Canada	n=25	42.5±13.1 yrs	88%	40% tetraplegia	12.2±8.2 yrs
(58 & 248 items)15,48					60% paraplegia
(57 item)20,30,31	Canada	n=482	42.4±12 yrs	81%	24% complete tetraplegia	15±7 yrs
					19% incomplete tetraplegia
					38% complete paraplegia
					18% incomplete paraplegia
OPHI16	USA	n=143	39.3 yrs	78%	NR	9.3 yrs
			range: 3–29yrs			range: 3–29 yrs
Para-SCI17a	Canada	n=102	36.9+10.2 yrs	79%	51% complete tetraplegia	11.2±8.5 yrs
					49% incomplete tetraplegia
			41.1±12.2 yrs	64%	63% complete paragplegia	12.5±11.2 yrs
					37% incomplete paraplegia
RNL38	Canada	n=98	45.2 yrs	78%	NR	15.5 yrs
			range: 21–81 yrs			range: 1–78 yrs

Notes:

^a– if the same individuals were used in multiple studies then the largest study sample is reported