Impact on Participation and Autonomy: Test of Validity and Reliability for Older Persons

Abstract

In research and healthcare it is important to measure older persons’ self-determination in order to improve their possibilities to decide for themselves in daily life. The questionnaire Impact on Participation and Autonomy (IPA) assesses self-determination, but is not constructed for older persons. The aim of this study was to examine the validity and reliability of the IPA-S questionnaire for persons aged 70 years and older. The study was performed in two steps; first a validity test of the Swedish version of the questionnaire, IPA-S, followed by a reliability test-retest of an adjusted version. The validity was tested with focus groups and individual interviews on persons aged 77-88 years, and the reliability on persons aged 70-99 years. The validity test result showed that IPA-S is valid for older persons but it was too extensive and the phrasing of the items needed adjustments. The reliability test-retest on the adjusted questionnaire, IPA- Older persons (IPA-O), showed that 15 of 22 items had high agreement. IPA-O can be used to measure older persons’ self-determination in their care and rehabilitation.

Introduction

According to both the United Nations and the European Union declarations, healthcare targeting older persons should be built on respect for the individual’s self-determination.^1,2 Older persons’ with multiple diseases and/or affected ability to perform activities of daily living (ADL), often meet an abundance of health care professionals.³ To exercise self-determination and to be in control in life are important for health and well-being in old age.⁴ It is important that older persons are allowed to make and influence decisions over their own daily life for their health and well-being,^4,5 especially for those persons who are dependent on others.⁴ Older persons right to self-determination is known by professionals.⁴ In spite of this, recent studies indicate that older persons’ self-determination is not fully respected within the healthcare sector.^6-9

Self-determination can be described as the ability to think, choose, decide and act independently.¹⁰ The concepts of autonomy and self-determination are often used synonymously.¹¹ When the concept is defined in a context of persons with affected ability to perform ADL independently, the concept according to Collopy ought to be divided into decisional autonomy and executional autonomy.¹² Decisional autonomy is the capacity to make personal choices, irrespectively of the person’s ability to perform their own choices, while executional autonomy is the performance of the choices.¹² Due to Cardol et al.¹³ self-determination is equated to decisional autonomy. Henceforth, in this article the concept of self-determination will be used as a synonymous to decisional autonomy.

One questionnaire that assesses self-determination is the generic self-report questionnaire Impact on Participation and Autonomy (IPA), but it is not constructed for the older population. The first version of the IPA was developed in the Netherlands and has been psychometrically tested in terms of homogeneity and construct validity for persons with various disabling conditions with the mean age about 48 years (range 23-79 years).¹⁴ The original IPA items have shown good construct, divergent and convergent validity, and have also shown good ability to sense changes over time, homogeneity and test-retest reliability.^14-16 The Swedish version IPA-S, equal to the original version of IPA,¹⁷ has been tested for validity and reliability for persons mean age 52 years (range 17-84 years) with spinal cord injury, showing promising psychometric results.¹⁷ Since the original IPA based its psychometric properties on persons <79 and the IPA-S on a single chronic condition, further psychometric studies for older persons are needed. To our knowledge, there is no other instrument to assess older persons’ self-determination.

The aim of this study was to examine the validity and reliability of the IPA-S questionnaire for persons aged 70 years and older.

Materials and Methods

Research design

The present study was performed in two steps; a validity test and a reliability test-retest, (Figure 1). First the face- and content validity of the IPA-S was tested using focus group discussions and individual interviews. An expert panel (n=4) analyzed and discussed the results,¹⁸ which resulted in an adjustment of the IPA-S called IPA-O (Older persons). The expert panel had knowledge in geriatrics (n=1), occupational therapy (n=3), instrument development (n=2), in interviewing older persons (n=4) and knowledge in focus groups methodology (n=3). The expert panel were selected persons from the research group covering a broad relevant expertise area of frail older persons. The reliability of the IPA-O was tested with a test-retest with Svensson’s method.^19,20

Figure 1.

Research design.

Questionnaire

The IPA-S was used in the validity test. The original IPA addresses the following five dimensions: autonomy indoors (seven items), family role (seven items), autonomy outdoors (five items), social relationship (six items) and work and education (six items), in total 31 items. The questionnaire has five answer alternatives: very good, good, fair, poor or very poor and the respondents score the items by themselves. At the end of each dimension a question regarding how perceived health problem affects perceived self-determination is answered with: no problems, minor problems or severe problems. Higher scores represent severe problems.¹⁵

Participants

Inclusion criteria were community living persons aged 70 years and older, dependent on another person in at least one ADL-activity according to the ADL Staircase.²¹ Before asked to participate in the study, they were assessed by healthcare professionals to be cognitively intact. The healthcare professionals also invited them to participate in the study. In the validity test, eligible persons were recruited from a community based service centre, a convenient way to find dependent, frail older persons living at home. In total, 13 of the 15 persons who were asked to participate actually participated. Nine persons accepted to participate in focus group discussions forming two groups (4+5), four persons accepted to participate in individual interviews. All lived in ordinary housing and home care service varied from having some assistance with heavy shopping by siblings, to four times daily formal home care service. All had at least one chronic condition such as heart failure, arthritis, vertigo or had residues after stroke or cancer (Table 1).

Table 1.

Characteristics of participants in the validity step.

Participants (n=13)	N.	%
Age, range (years)	70-88	-
Gender, M/F	3/10	23/77
Marital status, living alone	13	100
Housing, ordinary	13	100
Dependence
I-ADL	13	100
I-ADL and P-ADL	6	46

I-ADL, Instrumental Activities of Daily Living; P-ADL, Personal Activities of Daily Living.

In the reliability test-retest, eligible community living older persons were recruited from the Gothenburg region. Older persons that had finalized geriatric rehabilitation or a continuum of care intervention were asked to participate.²² In total, 39 of the 45 invited persons participated. The persons who declined to participate had either poor health or had no interest in participating (Table 2).

Table 2.

Characteristics of participants in the test-retest step.

Participants (n=39)	N.	%
Age, range (years)	70-99	-
Gender, M/F	13/26	33/67
Marital status
Married/cohabiting	9	23
Living alone	30	77
Housing
Ordinary housing	35	90
Nursing home	4	10
Dependence
I-ADL	39	100
I-ADL and P-ADL	19	49

I-ADL, Instrumental Activities of Daily Living; P-ADL, Personal Activities of Daily Living.

Validity of the IPA-S

Procedure and analysis

All questions from the IPA-S were read out loud in both the focus groups and in the individual interviews. Both focus groups and the individual interviews focused on whether each item was comprehensible and relevant. Also, the participants were asked about their overall impression of the questionnaire. The sessions were recorded and transcribed verbatim. The focus group discussions were analysed based on the Krueger’s method.²³ Content analysis was used for analyzing the individual interviews.²⁴ Relevant sections in the focus groups discussions were identified and sorted according to the aim of the study and the analysis was founded on how the participants discussed matters and the meaning that emerged from that discussion. For the individual interviews, the content analysis was performed with manifest content by looking for meaning units. Thereafter, meaning units from both the focus group discussions and the individual interviews were summarized and analysed into categories by the expert panel. The validity test including the interviews performed at the service centre was performed by the last author and the findings were analyzed and discussed by the expert panel. The IPA-S was then adjusted by the expert panel resulting in IPA for Older persons (IPA-O).

Reliability of the IPA-O

Procedure

The participants were contacted by phone and received information about the study. After the participants verbally agreed to participate and the written information about the study was presented, the first interview was conducted as a face-to-face interview in the participants’ home. At the first interview the participants had an enlarged copy of the answering categories in front of them and the questions were asked in given order. About fourteen days after the first test, the participants were contacted by telephone and the retest was performed. In total, 37 interviews were conducted by telephone and two by face-to-face interviews for the participants who had hearing impairment. Before the retest was performed, the interviewer ensured that nothing had changed in terms of participants’ health, which could have affected their self-determination. No participant was excluded due to this. Again, the participants had the enlarged copy of answering categories in front of them and the interviewer read the questions in a given order. The test and retest were therefore conducted in a similar way. The first, second and last author did the first test. The retests were done by the first and last author.

Statistica anaysis

The data were analyzed in accordance with Svensson’s method,^19,20 which is a rank-based statistical method developed for treatment of paired assessments of ordinal data. The method made it possible to identify and separately assess occasional and systematic disagreement.^19,20 The software for Svensson’s method version 1.1.2 was used. The test-retest evaluations comprised of the following measures: i) the percentage agreement (PA) is a basic value for categorical pairs of data between two assessments.25 PA of ≤59 % was considered as low percentage agreement, PA of 60-69% as moderate and PA of ≥70 % as high percentage agreement. ii) The relative rank position (RP) is a measurement of systematic shift in categorical levels between two assessments ranging between −1 and 1. The higher value on RP, the higher systematic group change. RP close to zero indicates negligible change over time.^19,20 A 95% CI on RP indicates an significant change on group level.20 iii) The relative rank variance (RV) is a measure of the observed individual variability.20 RV range from 0 to 1. Higher values are a signal of presence of individual variability.20 RV ≥0.1 was considered as a cut-off for the evidence of systematic disagreement regardless of the CI value. A 95% CI on RV indicates an significant change on individual level.20

The values of RP and RV were calculated in all 22 items. If PA had high percentage agreement (PA of >70%) the RP and RV were not considered relevant.

Ethical considerations

The study followed the ethical principles for medical research.26 In both the validity test and reliability test-retest, all participants were informed of their right to withdraw from the study at any time, that all data would be handled confidentially, and that no individuals could be identified. All participants gave both oral and written consent.

Results

Validity of the IPA-S and adjustments

The participants overall impression of the questionnaire were that the questions were relevant and of importance, but too extensive. Three categories emerged: Valued that their voice could be heard, The questionnaire was extensive and Focus on the execution. The participants Valued that their voice could be heard about their possibility of deciding for themselves. They gave numerous examples of friends and acquaintances that were questioned or even opposed in their self-determination either by relatives or by routines in home health care. The individual habits in daily life were particularly hampered when participants were assisted by others. Both focus groups discussed the point in having the questionnaire, but concluded that it could pinpoint this shortage of self-determination.

Participant 2: I think there are many that hear; don’t do that or don’t do that. I think many have others that decide for them. But I do what I want.

Participant 3: Yes, but you can still do it yourself

Participant 2: Yes, but if I’m disabled I can’t. There are those who have others that decide for them, they don’t have courage to decide for themselves.

Interviewer: So, do you think it’s good to have such a question?

Participant 3: Yes, yes.

Participant 1: Yes, yes.

(Focus group)

The participants expressed The questionnaire was extensive, and recommended some items to be removed. The dimension Work and Education was considered irrelevant. They argued that the dimension about family role had too many items; My chances of contributing to looking after my home the way I want to, My chances of fulfilling my role at home as I would like to, could be removed. Also, there were too many items in the domain Social relations. The quality of my relationship with people who are close to me and My chances of having an intimate relationship were recommended to be omitted. The item about intimate relationship was discussed thoroughly, even if it did not have to imply a sexual relationship; the participants recommended its removal. Participants also emphasized one item that should not be omitted since it was found extremely valuable; My chances of helping or supporting people in any way.

Interviewer: If we should cut down on the amount of questions, which ones should we erase?

Participants’: Those about education and work. But the question about helping others is important. You feel satisfied; you’ve done a good thing.

(Individual interview)

The participant’s Focused on the execution in the items and not on the decisional part. Both in the focus group discussions and in the individual interviews the participants had to be reminded that it was not their ability to perform the different items they assessed, but their chances to have them done the way they wanted or when they wanted.

Interviewer: My chances of getting washed and dressed when I want to are?

Participant: No I can’t. Pulling down my pants, it’s very difficult. And then, in the bathroom, getting them up again, it’s very difficult. They pull them up (home care service personnel). (Individual interview)

IPA-S was adjusted in accordance with the participants’ recommendations and the expert panels’ analysis resulting in 22 items (Table 3). The items were adjusted from questions to statements and the response options were adjusted to; totally agree, partly agree, neither agree nor disagree, disagree, and totally disagree. The items about perceived health problems were seen as important and relevant, both for the participants and the expert panel and therefore valid for the study group. No adjustments were made.

Table 3.

The values of percentage agreement (PA), relative position (RP) and relative variance (RV) in the questionnaire Impact on Participation and Autonomy (IPA-O).

Dimensions and items	N.	PA%	RP	RP 95%CI	RV	RV 95%CI
Mobility (4 items)
My chances to decide where to get around in my house are good	39	97	–0.03	–0.08 to 0.03	0.00	0.00 to 0.03
My chances to decide when I want to get around in my house are good	39	90	0.03	–0.02 to 0.08	0.00	0.00 to 0.03
My chances to decide when to visit relatives and friends are good	38	71	0.07	0.02 to 0.12	0.03	0.00 to 0.06
My chances to decide to go on the sort of trips and holidays 1 want to are good	37	54	–0.08	–0.13 to –0.03	0.10	0.07 to 0.12
Self care (5 items)
My chances to decide to get washed and dressed the way 1 want are good	38	92	0.03	–0.03 to 0.08	0.00	0.00 to 0.03
My chances to decide when 1 get washed and dressed are good	39	79	–0.02	–0.08 to 0.02	0.01	0.00 to 0.04
My chances to decide when 1 want to go to bed or get up are good	39	95	–0.05	–0.10 to 0.00	0.00	0.00 to 0.03
My chances to decide when 1 want to go to the toilet and when 1 need to are good	39	88	–0.03	–0.08 to 0.03	0.00	0.00 to 0.03
My chances to decide when 1 want to eat and drink are good	39	87	0.02	–0.03 to 0.08	0.00	0.00 to 0.03
Activities in and around the house (4 items)
My chances to get light tasks done around the house, either by myself or by others the way 1 want are good	38	82	0.08	0.03 to 0.13	0.01	0.00 to 0.03
My chances to get heavier tasks done around the house, either by myself or by others the way 1 want are good	38	61	0.13	0.08 to 0.19	0.03	0.01 to 0.06
My chances to get housework done, either by myself or by others when 1 want are good	38	63	0.11	0.06 to 0.16	0.09	0.06 to 0.12
My chances to get minor repairs and maintenance work done in my house and garden either by myself or by others the way 1 want are good	26	69	0.15	0.11 to 0.19	0.02	0.00 to 0.04
Financial situation (1 item)
My chances to choose how 1 spend my own money are good	38	100	0.00	–0.05 to 0.05	0.00	0.00 to 0.03
Use of time (1 item)
My chances to use leisure time the way 1 want are good	39	67	–0.14	–0.20 to –0.09	0.04	0.01 to 0.06
Social relationship (5 items)
My chances to talk to people close to me on equal terms are good	37	97	0.00	–0.05 to 0.05	0.00	0.00 to 0.03
The respect 1 receive from people who are close to me are good	37	95	0.05	0.00 to 0.10	0.00	0.00 to 0.03
My chances to talk to acquaintances on equal terms are good	38	76	–0.00	–0.05 to 0.05	0.01	0.00 to 0.04
The respect 1 receive from acquaintances are good	38	89	–0.03	–0.08 to 0.02	0.00	0.00 to 0.03
My chances to see people as often as 1 want are good	36	72	0.02	–0.03 to 0.07	0.05	0.02 to 0.07
Help and support others (1 item)
My chances to help or support people in any way are good	38	63	0.03	–0.02 to 0.08	0.04	0.01 to 0.06
Summary (1 item)
My chances to live the way 1 want are good	39	64	0.11	0.06 to 0.17	0.03	0.01 to 0.06

Reliability of the IPA-O

The range of PA between the test and retest assessments of all 22 items differed from 54 to 100% (Table 3). In three of the seven dimensions all items showed high agreement (PA ≥70%).

Fifteen items indicated high agreement (PA ≥70%). No evidence of systematic disagreement was found for these items. Six items showed moderate agreement PA range from 61 to 69%, item 11-13, 15, 21-22. The RP in item 11 indicated a change between the two tests at group level and the RV was close to zero demonstrating negligible presence of individual variability. The RP in item 12 indicated a decline of self-determination and the RV did not cover the zero value of the measures. In item 13, only 26 of the 39 participants answered. In item 11-13, both RP and PV had positive values with positive 95% CI values, indicating that the study group had scored higher perceived self-determination in the first test, than in the retest. In item 15, a systematic disagreement at group level was shown, towards a higher level of self-determination at the retest and the RV indicated presence of reduced self-determination on individual level. In item 21 both the RP (0.03) and RV (0.04) were negligible small, indicating a reduction of self-determination on both group and individual level. In item 22, RP indicated a change at group level, with non-zero values of 95% CI and the RV (0.03) demonstrated negligible presence of individual variability, with a reduced self-determination of the total perception after the retest.

One item showed low agreement with PA of 54% (item 4). The RP indicated an increase of self-determination and the RV was on the cut-off limit (RV >0.10) with a 95% CI 0.07 to 0.12 indicating a significant presence of individual variability.

Discussion

In the validity part, the participants overall impression of the 1PA-S questionnaire was that the questions were relevant and they emphasized being asked about their decision making in daily activities, but the questionnaire was too extensive. The participants' answers revealed that they focused on the exe-cutional and not on the decisional part of the activities in the questionnaire. Due to this, IPA-S was adjusted to a version called IPA-O, with fewer items with the decisional part of self-determination emphasized. The test-retest results indicated that IPA-O is a reliable questionnaire in 15 of the 22 items. Six items had moderate agreement and only one indicated low agreement. In item 11-13, dimension Activities in and around the house, both the RP and RV showed positive values which indicated that the participants’ self-determination decreased between the two tests. The conclusion may be drawn that the participants after the first test had reconsidered their self-determination to be more reduced in the decision making process than before, in heavier tasks, housework or repairs. In the original version of 1PA, items 11 and 13 had moderate agreement,^14-16 which means that the present result is consistent with earlier.

Only the item 4, dimension Mobility, was considered to have low agreement (PA of ≤59%). The values of RP indicated that the participants’ self-determination had increased after the first test on group level and the RV indicated a reduction in perceived self-determination after the first test on individual level. The low agreement and the contradiction between the values can be explained by the complexity in some of the items in IPA-O, which deal with a wide area of daily activities. The complexity vary from less complex; My chances to decide when I want to go to bed or get up are good, to more complex; My chances to get housework done, either by myself or by others when I want are good. The items with moderate PA values are of the latter category and thus probably explain these lower values. More precise items could increase the PA values but would at the same time narrow down the area of relevance and possibly hamper the validity of the questionnaire. Consequently, the items with moderate PA values are recommended to be kept without change. The item with low PA; My chances to decide to go on the sort of trips and holidays I want to are good is even more complex item since it includes both trips and holidays, these activities can involve ways of travelling each time. Still, even this item is kept based on the importance of the item as perceived by the target group but it is recommended to be interpreted with caution. There is a need for an instrument that assesses older persons’ own perspectives and possibilities to be self-determined even when getting dependent on others, both clinically in rehabilitation and for research purposes.¹⁷ One limitation with the IPA-S, was that the participants answered the questionnaire emphasizing the executional part of daily activities and not both the decisional and executional part as intended by Cardol.¹³ Thus, it was imperative to rephrase the questionnaire but still observing the intent of IPA. The intent has been kept in spite of rephrasing from questions to statements. In addition, spontaneous feed-back by the participants in the test-retest part of the study reflects that the decisional part is now more clearly noticeable.

The adjustments of the IPA-S resulted in fewer items based on the participants’ comments and expert panels’ analysis. There are several advantages by using fewer questions for this study population as expanded questionnaires can be exhausting to answer. There is a risk by decrease the amount of questions, but the older persons commented that some of the questions were not of relevance for them. Consequently, the benefits are greater than the risks.

The participants were few, which is a major limitation. Hence, the results must be interpreted with caution. Nevertheless, the characteristics of the participants indicate that they reflect the target group of older persons with affected ability to perform ADL independently. The participants’ ability varied from having support several times a day to more occasional. All participants were assessed as being cognitively intact; no formal assessment was performed but the healthcare professionals performing the clinical assessment knew them well. Thus, further psychometric studies are needed to examine the usefulness of IPA-O for cognitively affected older persons. A strength with the validity test was that two data collection methods were used. In addition, a multi-professional expert panel was used to analyse the focus groups’ discussions and the individual interviews, as recommended in face and content validity tests.²⁷ Thus, a variety of perspectives was obtained strengthening the triangulation.

The composition of the expert panel, could have influenced the design of IPA-O, but as their analysis is based on the participants’ comments of the questionnaire and the expert panel are covering a broad relevant expertise area of frail older persons it is not considered as a major limitation. To test the reliability, a rank-invariant method by Svensson was used.^19,20 Another method often used for ordinal data is Cohen’s Kappa.²⁸ A comparison of the Kappa statistics and the rank-invariant method, showed that the rank-invariant method was superior when identifying and separately measuring systematic and random disagreement between test and retest.²⁹ The time interval between the test and retest was 14 days, which is considered appropriate since the time is long enough for the persons to have forgotten the questions.³⁰ Moreover, the time interval was appropriate in order to reduce the risk of further deterioration of the participants’ health due to their frailty. The study group in the reliability test-retest consisted of older persons that were dependent on another person in at least one daily activity. Older persons with multiple diseases and decreased ability in daily activities are a complex group and their health situation can change radically. To ensure that this was not the case the participants were asked about whether their health had changed between the test and retest in as far as it could have affected their self-determination. No one was excluded on this account, but it still cannot be dismissed that their own perception of their self-determination can have been affected during this time interval.

Conclusions

The validity test showed that the questions in IPA-S were important, but some could be erased. To capture the decisional part of autonomy, the items were rephrased to make it clearer. IPA-S was revised into IPA-O.

The reliability test-retest of IPA-O showed that the 15 items that had high agreement are reliable. The additional seven items are important aspects when assessing older persons’ self-determination. Therefore, we recommend their inclusion but they must be interpreted with caution.

IPA-O can be used for older dependent persons when planning their healthcare and rehabilitation to ensure their right to self-determination and participation. The questionnaire is recommended for use in all levels of rehabilitation, health care and in social care for the age group and is appropriate in the scientific research area when the purpose is to examine and investigate the older populations’ perceptions of their self-determination.

Funding Statement

Funding: this study was supported by grants from the Research and Development Council of the County of Gothenburg and South of Bohuslän and Vårdalinstitutet, the Swedish Institute for Health Sciences.