Inter-rater agreement of a modified and extended Swedish version of the Frenchay Activities Index (FAI)

Abstract

The point of departure for this study was the need for an instrument capturing social activities updated to the living circumstances of the 21st century. Starting out from the Frenchay Activities Index (FAI), the objective was to investigate the inter-rater agreement of the scoring of a modified and extended Swedish version, capturing activity performance and participation. Thirty-one older people, living in the community post stroke, were interviewed in their homes by two raters using the Swedish FAI version, extended with items on use of mode of transport and use of telephone, and modified with additional scales. Besides the original frequency scale, the new scales captured changes in frequency, reasons for change, and performance satisfaction; inter-rater agreement was analyzed with kappa statistics. Overall, the inter-rater agreement was high or very high, with weighted  = 0.924 for the frequency scale and = 0.784–0.940 for the new scales. As a conclusion, while further validity and reliability testing is necessary, when out-of-home activities are of interest the scoring of the modified and extended Swedish FAI version can be administered with high inter-rater agreement.

Introduction

Maintaining an active life with preserved societal participation is important along the life course, also for people experiencing disabling events in later life. Consequently, the ultimate goal of rehabilitation is the client-centered reintegration into a desired selection of activities, and achievement of this goal is also naturally of major importance for society itself, to support active and healthy aging. To efficiently support clients in planning and evaluating progress, assessment instruments that are clinically relevant and easy to administer are needed. Validity and reliability should also be established (Streiner and Norman 2008). However, some of the instruments currently in use were developed decades ago and might not capture aspects of high relevance for today’s society, and there is a constant need for updating (Wade 1992).

Activities performed in the recent past are usually targeted in setting realistic individual goals for rehabilitation. Based on professional judgment, most often personal care followed by household activities are targeted, while out-of-home activities often are overlooked. “Social outcome measures” are used in assessments of changes in social functioning of people with disabilities that result directly or indirectly from impairments or functional limitations (Dijkers et al. 2000). The concept of social activity has a broader meaning, thereby constituting an important aspect of participation, i.e., involvement in life situations (Dijkers et al. 2000; Schepers et al. 2007; Trigg and Wood 2003; World Health Organization 2001). Most traditional instruments that target activities of daily living (ADL) do, however, lack items representing activities relevant to the living circumstances of the 21st century. For instance, even if mobility in society has seen a marked increase during recent decades and use of varying modes of transport is of crucial importance for participating in activities outside home (Carlsson 2002; Iwarsson et al. 2003; Primeau 1996), such aspects are seldom included in sufficient detail in existing assessment instruments used within rehabilitation (Schepers et al. 2007).

Stroke is among the most common causes of disability in industrialized societies (O’Brien et al. 2003), in particular in later life. The Frenchay Activity Index (FAI) (Holbrook and Skilbeck 1983) (Table 1) is a frequently used instrument in stroke rehabilitation, and is one out of few existing instruments that targets a broader repertoire of activities also representing social and civic life, i.e., participation (Schepers et al. 2007). Therefore, the FAI was our starting point and the focus of this study. During the 1980’s the FAI was developed for use with stroke patients, to briefly measure lifestyle. That is, the aim of the instrument is to reflect the self- or spouse-reported pre-morbid everyday activities of normal living during the previous three and six months. The 15 activities (e.g., personal care, preparing main meals, driving a car/bus travel, gardening) included in the original version of the instrument (Holbrook and Skilbeck 1983) (Table 1) represent three main factors; domestic chores, leisure/work, and outdoor activities. All the items included require some decision-making and organization, and a higher level of independence. The four-level rating scale was based on frequency, with a higher score denoting more participation. The utility of the FAI has been confirmed, and high construct validity and test–retest reliability have been reported (Dijkers et al. 2000; Salter et al. 2005; Turnbull et al. 2000).

Table 1.

Original FAI version

Question: In the last 3 months how often have you undertaken?
1. Preparing main meals	1 = Never
2. Washing up	2 = Under than once a week
	3 = 1–2 times a week
	4 = Most days
3. Washing clothes	1 = Never
4. Light housework	2 = 1–2 times in 3 months
5. Heavy housework	3 = 3–12 times in 3 months
6. Local shopping	4 = At least weekly
7. Social outings
8 Walking outdoors over 15 min
9. Pursuing active interest in hobby
10. Driving a car/travel on bus
Question: In the last 6 months how often have you undertaken?
11. Outings/car rides	1 = Never
	2 = 1–2 times in 6 months
	3 = 3–12 times in 6 months
	4 = At least weekly
12. Gardening	1 = None
13. Household and/or car maintenance	2 = Light
	3 = Moderate
	4 = All necessary
14. Reading books	0 = None
	1 = 1 in 6 months
	2 = Less than 1 a fortnight
	3 = Over 1 a fortnight
15. Gainful work	0 = None
	1 = Up to 10 h/week
	2 = 10–30 h/week
	3 = Over 30 h/week
Total____, Factor 1____, Factor 2____, Factor 3____

According to Holbrook and Skilbeck (1983)

Among the few studies of inter-rater agreement reported, Post and de Witte (2003) found the reliability of local shopping, social outings, and pursuing active interest in hobby to be moderate but still insufficient, suggesting an improvement of the scoring instructions. In another study, Wade et al. (1985) reported poor agreement for social outings and pursuing active interest in hobby (Piercy et al. 2000), and moderate agreement for heavy housework, local shopping, walking outdoors >15 min, and outings/car rides. Based on these results, these authors concluded that there was a need for clearer instructions. Subsequently, for example Wade et al. (1985) contributed with revisions to the wording of the instrument and the development of guidelines.

Even if the FAI emphasizes social activity and participation (Dijkers et al. 2000; Holbrook and Skilbeck 1983), and for a long time has been advocated for use with stroke survivors, it is not well suited to today’s living circumstances (Appelros 2007). In particular, in today’s society out-of-home activities constitute a prerequisite for societal participation. Even if outdoor activities constitute one of the main factors of the original FAI, such activities are not targeted in sufficient detail. That is, to be able to value present activity performance, the assessment must be put in a broader everyday life context. Due to the disparities among individual activity repertoires, relying on norm values (Turnbull et al. 2000) or defined thresholds for different levels of activity is not considered appropriate (Dijkers et al. 2000; Schepers et al. 2005), not least since several studies have indicated that more activities need to be included (Appelros 2007; Turnbull et al. 2000). In addition, it would be more valid to compare the situation before and after a disabling event (Dijkers et al. 2000). However, assessments in late stroke phases, with the aim to evaluate change over time, could be problematic unless a former FAI assessment had been accomplished (Appelros 2007). In addition, a change may be caused by many different reasons, and not all of them would be relevant to act upon in rehabilitation. In order to collect information useful for the identification of needs for relevant interventions, questions capturing self-reported cause for change in frequency could be helpful. Moreover, the item definitions denote that the scoring concerns independent activity. Since previous studies suggest that there are floor but no ceiling effects with stroke patients (Appelros 2007; Dijkers et al. 2000), this most likely indicates that independent activity is a challenge to many of these individuals. Consequently, questions have also been raised on the accuracy of scoring based solely of independent performance (Dijkers et al. 2000). Finally, the FAI does not cover any qualitative aspects such as performance satisfaction, while such client perceptions are considered to be of major importance for successful rehabilitation. Consequently, we identified needs for further instrument optimization.

While assessment instruments must be tested for many psychometric properties, inter-rater agreement is a basic quality that needs attention (Streiner and Norman 2008). Therefore, the aim of this study was to investigate the inter-rater agreement of the scoring of a modified and extended version of the FAI, capturing detailed use of mode of transport in terms of frequency, change in frequency, self-reported cause for change in frequency, and performance satisfaction.

Methods

This study was based on data that were part of a larger survey aiming at studying the long-term situation of individuals living in the community, with a clinical diagnosis of stroke during the previous 18-36 months and assessed as having affected cognitive functioning in the acute phase. Results based on other parts of the survey have been presented elsewhere (Wendel et al. 2008).

Sample

The participants were recruited from a national quality assessment register of stroke incidents available at the Department of Neurology Malmö, Skåne University Hospital, Sweden. The following inclusion criteria were applied:

(1) Stroke onset during a defined 18-month period; (2) cognitive functional limitation in at least one domain documented post stroke; (3) walking, independent of another person at least indoors 3 months after stroke; and (4) living in ordinary housing 3 months after stroke. Those with ongoing contra-indicative medical conditions (e.g., cancer treatment) were excluded, as were those not able to communicate due to aphasia or difficulty with the Swedish language. Further details on the sampling procedure have been presented elsewhere (Wendel et al. 2008).

In order to determine the inclusion criterion “presence of cognitive functional limitations,” the Cognistat instrument was used (Kiernan et al. 1987; Mueller et al. 2001), administered by an occupational therapist with adequate training for such assessments. The Cognistat comprises three general areas (consciousness, orientation, and attention), and another five major areas (language, visual constructive skills, memory, calculation, and reasoning) of which two are divided into sub-areas (language and reasoning). Each area/sub-area is scored as average, or mild/moderate/severe impairment.

The first 31 consecutively included participants constituted the sample for the present study; 21 men and ten women having had a stroke >18 months earlier (median = 27 months), with a mean education length of 10 years. Thirteen of our participants (43 %) had a Cognistat composite score indicating a marked cognitive functioning (score <63) (Drane et al. 2003; Ruchinskas et al. 2001). Additional participant characteristics are shown in Table 2.

Table 2.

Study sample characteristics, N = 31

Characteristic			Characteristic
Agea	Median	75 years	First strokea	Yes/no	58 %/42 %
	Min–Max	54–94 years
			Time post strokea	Mean	27 months
Stroke diagnosisa	Infarction	89 %	Living conditionb	Alone	55 %
	Hemorrhage	11 %		Spouse/friend	45 %
Hospital carea	Mean	10 days	Getting outdoorsb	Independently	94 %
				With help	6 %
Side of injurya,c	Right	36 %	Cognistatb	Marked cognitive functiond	38 %
	Left	48 %
	Bilateral	7 %		“Dementia syndrome”	14 %
	Unknown	10 %
				Years of education (M)	10, 4

^aFrom the Riks-stroke register or medical records

^bData collection within the study

^cData provided by a senior neurologist (co-author)

^dCognistat composite score <63 (Kiernan et al. 1987; Mueller et al. 2001)

Instruments

The FAI version used as a starting point for the present study was a Swedish version of the original FAI in widespread national use, but of unknown origin. After close inspection, this version was considered to be linguistically suboptimal, and, therefore, the English original (Holbrook and Skilbeck 1983) was again translated and reviewed by a linguistic expert, rendering suggestions for improvements. Two senior researchers with experience of instrument development within the health sciences and public transport planning, respectively, then independently reviewed the corrections suggested. After further discussions concerning some details, consensus was reached on the optimized Swedish instrument version constituting the starting point for the current study.

In a next step, the modification and extension of the item pool was accomplished. That is, one of the items was modified; the original item merging driving a car/travel on bus was split up into two new items: driving a car/motorbike and going by bus or train. To arrive at a more detailed assessment of out-of-home activities and mode of transport, additional items were developed; wheelchair outdoors >15 min, going by bicycle/moped, powered wheelchair, passenger private car/taxi, and Special Transport Service (STS; the so-called door-to-door service). Finally, using the telephone was added, since this had been recognized as a missing item in an earlier study (Dijkers et al. 2000), and also of relevance for the modified FAI version since using a telephone is of importance for using taxi or STS. After this extension, the new FAI version comprised 22 items. In accordance with the original FAI (Holbrook and Skilbeck 1983), each item was scored on a four-level frequency scale from 0 to 3, with a maximum score of 66; higher scores denote more activity.

Thereafter, the instrument was extended by the addition of three new scales on other aspects of performance, all to be scored for each single item. These scales targeted:

• Frequency changes compared to the pre-stroke situation (decreased frequency unchanged frequency, increased frequency)

• Self-reported cause for change (physical/cognitive functioning, economical resources, desire/need to perform the activity, worries from proxies/friends, social network changes)

• Satisfaction with activity performance (5-graded Likert scale)

In order to attain face and content validity, experienced practitioners as well as an expert group of senior scientists were involved in the development process outlined above, including two seminars to discuss and reach consensus on the extension and additions. Thereafter, a pilot version of the new FAI was tested on three women and two men (stroke patients, not part of the study sample). This pilot resulted in further revisions, ending up with the instrument version tested in the current study (Table 3).

Table 3.

Overview of the modification and extension of the Swedish Frenchay Activities Index version tested in this study

Original item	New item						Original frequency scale
8. Walking outside >15 min	8 A. Walking outdoors >15 min	8 B. Wheelchair outdoors >15 min					0 = Never
							1 = 1–2 times in three months
10. Driving a car/travel on bus	10 A. Driving a car/motorbike	10 B. Going by bus or train	10 C. Going by bicycle/moped	10 D. Powered wheelchair	10 E. Passenger private car/taxi	10 F. Special transport service	2 = 3–12 times in three months
							3 = At least weekly (used for items 8 and 10 in the original instrument)
——	16. Using the telephone						0 = Never
							1 = < Once weekly
							2 = 1–2 times weekly
							3 = Most days (used for items 1 and 2 in the original instrument)

New scales, used for all items
Frequency changesa	Self-reported cause for changeb	Satisfaction with activity performancec
Decreased frequency	1 = Physical/cognitive functioning	1 = Very dissatisfied
Unchanged frequency	2 = Economical resources	2 = Dissatisfied
Increased frequency	3 = Desire/need to perform the activity	3 = Neither satisfied, nor dissatisfied
	4 = Worries from proxies/friends	4 = Satisfied
	5 = Social network changes	5 = Very satisfied

^aAlways rated

^bNot rated for activities where the frequency is unchanged

^cNot rated for activities where the frequency is 0 and unchanged

Procedure

Applying the principle of informed consent, the potential participants were contacted by telephone. If no exclusion criteria were found and they agreed to have information about the study sent to them, they were again contacted a week later and asked to participate. Data were collected at home visits accomplished by two experienced occupational therapists (of whom one is the first author). Besides the modified and extended FAI, the questionnaire administered included participant characteristics (Table 2) and other stroke-related instruments (data not used for the present study).

The three additional scales of the modified and extended FAI were administered in a pre-set order, after the original frequency scale. The simultaneous scoring accomplished by the two raters was based on an assessment interview with each participant. The interview was conducted by one of the raters, according to written instructions with all items and scales well defined. The participant responses were independently interpreted by the raters, sometimes after asking subsequent questions in case of a first answer that was difficult to interpret. Large-letter printed versions were available to facilitate for the participants to keep the response alternatives in mind when answering. The raters remained unaware of each other’s scores throughout the study. The mean time for the administration was 40 min.

The regional Ethical Board in Lund approved the study.

Data analysis design and statistics

The kappa statistic , a strict measure of agreement that corrects for chance agreement (Altman 1991; Bartko and Carpenter 1976), was used to investigate agreement between the two raters for each FAI item. For nominal scales, Cohen’s kappa (Cohen 1960) was calculated, while weighted kappa was calculated for the ordinal scales. It should be noted that it is not possible to calculate kappa values for items scored 0 in all cases; kappa will, therefore, remain undefined for some items. Besides item-specific kappa values, mean kappa values () were calculated for the four scales. The κ values were interpreted following Altman’s guidelines (Altman 1991): <0.20 = poor agreement, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = high agreement, and 0.81–1.00 = very high agreement. All analyses were carried out using SPSS 11.0.1.

Results

The mean weighted κ for the 22 items using the frequency scale was 0.924. For one of the new single items (using the telephone), weighted κ was 0.717, while for all other items weighted κ > 0.8 (Table 4). For the three new scales, capturing other aspects of activity than frequency, the mean values indicated high to very high agreement; frequency changes, weighted κ = 0.940; self-reported cause for change, weighted κ = 0.784; satisfaction with activity performance, weighted κ = 0.874. As to the agreement within each additional scale (Table 5), some single items stood out. Regarding frequency changes compared to the pre-stroke situation, the lowest weighted kappa value was found for using the telephone, but still at a level indicating high agreement. Turning to the scale targeting self-reported cause for change, the agreement was high (weighted κ > 0.8) for the great majority of the items. For a few items a poor to moderate agreement was indicated; washing clothes, passenger in private car/taxi, STS, and using the telephone (Table 5). The satisfaction scale, weighted κ > 0.8 for all but one item (STS), still demonstrated high agreement.

Table 4.

Distribution of FAI frequency scores and agreement of two raters, N = 31

Item	0a	1	2	3	Weighted kappa
	A/Bb	A/B	A/B	A/B
Preparing main meals	16/17	3/2	5/5	7/7	0.976
Washing up	5/5	3/1	2/3	21/22	0.908
Washing clothes	15/15	0/0	11/11	5/5	1.000
Light housework	12/12	0/0	5/3	14/16	0.956
Heavy housework	12/12	0/0	4/3	15/16	0.844
Local shopping	6/6	0/1	4/2	21/22	0.819
Social outings	8/8	3/4	9/8	11/11	0.975
Walking outdoors >15 min	9/9	0/0	2/2	20/20	1.000
Wheelchair outdoors >15 min	29/29	0/0	1/0	1/2	0.903
Pursuing active interest in hobby	10/11	0/0	3/3	18/17	0.930
Driving a car/motorbike	22/22	0/0	0/0	9/9	1.000
Going by bus or train	15/15	3/2	7/7	6/7	0.951
Going by bicycle/moped	25/25	2/1	1/2	3/3	0.956
Powered wheelchair	31/31	0/0	0/0	0/0	Undefined
Passenger private car/taxi	5/4	10/9	10/11	6/7	0.880
Special transport service	21/21	2/2	3/3	5/5	1.000
Outings/car rides	16/15	8/8	7/8	0/0	0.851
Gardening	21/22	2/2	3/2	5/5	0.939
Household and/or car maintenance	25/24	0/0	3/3	3/3	0.923
Reading books	16/15	1/2	10/7	4/6	0.873
Gainful work	29/28	0/0	2/2	0/0	1.000
Using the telephone	2/1	6/4	6/5	17/20	0.717

^aHigher figure denotes more frequent activity performance

^bRaters A and B

Note items added/modified in bold. For items with 0 frequency, kappa is undefined

Table 5.

Inter-rater agreement of the scoring of modified and extended Swedish FAI, additional scales, N = 31

Item	Agreement in scoring
	Frequency changesa	n	Self-reported cause for changeb	n	Satisfaction with activity performancea	n
Preparing main meals	1.000	31	1.000	14	1.000	21
Washing up	1.000	31	1.000	8	0.956	27
Washing clothes	0.914	31	0.500	6	0.913	18
Light housework	1.000	31	1.000	6	0.810	20
Heavy housework	0.957	31	0.879	17	0.961	27
Local shopping	0.954	31	1.000	13	0.953	29
Social outings	1.000	31	1.000	16	0.964	29
Walking outdoors >15 min	0.953	31	1.000	15	0.916	24
Wheelchair outdoors >15 min	1.000	31	0.000	1	0.000	2
Pursuing active interest in hobby	0.825	31	1.000	11	1.000	23
Driving a car/motorbike	1.000	31	1.000	10	0.924	18
Going by bus or train	1.000	31	1.000	19	0.940	20
Going by bicycle/moped	0.968	31	1.000	7	1.000	10
Passenger private car/taxi	0.935	31	0.261	13	0.929	25
Special transportation service	1.000	31	0.158	8	0.640	9
Outings/car rides	0.935	31	0.803	14	0.946	22
Gardening	0.968	31	1.000	7	0.917	13
Household/car maintenance	0.968	31	1.000	7	1.000	12
Reading books	0.968	31	0.609	9	0.957	20
Gainful work	1.000	31	undefined	3	0.815	5
Using the telephone	0.719	31	0.462	7	0.821	30

Note powered wheelchair not listed since not in use; undefined kappa. Varying n since analyses only included participants with a changed activity frequency recorded by both raters. For the satisfaction scale n varies (not scored if no and unchanged performance)

^aWeighted kappa

^bKappa

Discussion

The results of this study suggest that the Swedish modified and extended FAI version tested can be administered with high to very high agreement of the scoring between raters. In order to target participation in the 21st century, the modification and extension of the item pool, and an extension with three additional scales are appropriate and in accordance with earlier research (Bond et al. 1995; Dijkers et al. 2000). That is, adding using the telephone, use of wheelchair outdoors, and an extension of the categorization of modes of transport, along with additional scales to capture more dimensions of participation, i.e., satisfaction (Gray et al. 2006), improves the clinical validity of the instrument. Overall, the results show higher agreement for single original FAI items compared to previous studies (Piercy et al. 2000; Post and de Witte 2003; Wade et al. 1985), and modified or added items demonstrated as high agreement as the original items. However, based on a single, first study with a small sample of stroke survivors in Sweden, we do not know if the inter-rater agreement of the scoring of the added items would be high in other settings, and with other sub-groups. As the current study was solely concentrated on inter-rater agreement of the scoring of the original frequency scale and three additional scales, it should be kept in mind that more research efforts are warranted before we can recommend the modified and extended version of the FAI for widespread use.

The majority of new items reflect different modes of transport, resulting in an instrument version somewhat dominated by out-of-home activities. The extension accomplished rests on previous knowledge from transport research in an inter-disciplinary collaboration process, combined with longstanding experience from stroke rehabilitation and health sciences research. With the ambition to improve the content validity of the instrument in relation to a well-accepted conceptual framework provided, the out-of-home activity items added were linked to the mobility domain in the ICF (World Health Organization 2001); the categories “walking and moving”(d450–d469) and “moving around using transportation”(d470–d489). The additional items were well accepted by the participants, but the validity of this modification needs more research.

It would also be of interest to test the modified and extended version of the FAI in other European countries. In a European perspective, access to as well as use of mode of transport is often studied based on a categorization of modes of transport as used in this study, specifically when studying older people (Mollenkopf et al. 2004; Risser et al. 2010). Thus, previous research shows that access to one single mode of transport might impact on out-of-home activity while others might not, and, moreover, that an accessible transport system is a major component of quality of life. Knowledge concerning the impact on out-of-home activities and different modes of transport are of high relevance in societal planning, not the least for the prioritization of measures in order to increase the accessibility and usability of out-of-home environments for older people and people with disabilities (Risser et al. 2010; Ståhl et al. 2008; Wennberg et al. 2009; Wennberg et al. 2010).

If warranted, it is still possible to compare results from the instrument version tested in the present study (Table 4) with those of previous studies using the original FAI (Table 1), by merging the items of driving a car/motorbike and going by bus or train, and excluding the rest of the added items. Inclusion of more items has been proposed before; sport, physical exercise, and caring for children, to make the FAI suitable for a wider age range of stroke survivors (Turnbull et al. 2000), and watching TV and listening to the radio, as common among older people (Appelros 2007; Bond et al. 1995). These suggestions for further modifications seem appropriate and in accordance with others (Schepers et al. 2007; Turnbull et al. 2000), during the present study we also considered adding economic transactions as an item. However, since the time needed for administration of an instrument is of great importance in clinical settings as well as in research, the ambition to catch detailed information must be counterbalanced with feasibility. While in the current study, the administration of the Swedish modified and extended FAI version (Tables 3 and 4) was experienced as easy and time efficient, an increase of the 40 min administration time might not be perceived as positive.

The additional scales on frequency changes, self-reported cause for change, and satisfaction (with the actual performance of an activity) in the FAI version tested in the present study are important for descriptive and evaluative purposes, and demonstrated promising agreement. The scale on frequency change was developed to capture a before and after scenario, and we considered our three levels of change as more trustworthy than a retrospective estimation by use of the same scoring system as for present frequency. Thus, in contrast to the original version (Table 1), several aspects of participation can be assessed at one data collection occasion. The repeatedly administered present FAI version might offer an improved responsiveness due to the recording of satisfaction, obviously of great concern (Salter et al. 2005), but more research is needed to establish this.

Studying the agreement results in more detail, agreement was poor to fair for several items in the self-reported reasons for change scale (Table 4). Since the analyses included only those few participants where both raters recorded a change of activity frequency, the small sample size most likely affected this facet of the results. However, the reason for low kappa values might be a systematic difference in the raters’ interpretations of responses to this scale. Obviously, two of the response alternatives were not sufficiently differentiated, namely physical/cognitive functioning and desire/need to perform the activity.

Regarding the low kappa value for changed use of telephone, reflecting in hindsight upon the actual interview situation the reason might be that the primary cause for change in some cases were mixed with secondary effects of that situation. That is, the raters did not sufficiently separate the response alternatives from each other during the interview. This is an example of an experience that should result in further revision and optimization of the instrument guidelines. Moreover, the numbers are small and the 95 % CIs around the kappa values is wide. These limitations clearly indicate that further optimization of the instrument, followed by studies based on larger samples, are needed. Most importantly, the instrument’s sensitivity to change should be established.

We do acknowledge that our study approach is unusual, since it might be thought of as unnecessary to test the inter-rater agreement of the scoring of a highly structured interview format. However, considering the number of participants in the study sample where a marked limitation in cognitive functioning was indicated (Table 2), even if this is only a first step of reliability testing, the facet of agreement we studied is a basic requirement for the collection of valid data. With people with cognitive functional limitations constituting the target group, we do argue that more attention is needed to challenges hitherto not recognized concerning reliability, potentially inherent in the interview situation. That is, even with a highly structured interview format, scoring based on the rater’s interpretation of the respondent’s answers to the questions posed could be difficult. Moreover, the two raters made their assessments at the same point in time since both internal and external circumstances easily could interfere with the interpretation of the information given by the participant, constituting the basis for the scoring made by the rater. In order to standardize the way the questions were posed, the actual questioning was always done by the same rater, using rater instructions with a detailed wording to be used during administration.

In conclusion, the scoring of the modified and extended FAI version tested in this study can be administered with high to very high agreement. The instrument is more comprehensive than the original as it is more relevant for community-living persons and has the potential to provide researchers and clinicians with more comprehensive information. Considering obvious study limitations, the present study is only a first step of an ongoing process of psychometric testing; the modified and extended version now tested in a small sample of stroke survivors in Sweden should be tested in other contexts.