Validation of the Italian version of Mini-ICF-APP, a short instrument for rating activity and participation restrictions in psychiatric disorders

Abstract

Aims.

The assessment of limitations in social capacities can be done with the Mini-ICF-APP, a rating scale built in reference to the International Classification of Functioning, Disability and Health (ICF). The aim of this study was to assess the reliability and the convergent validity of the Italian version of this scale.

Methods.

We recruited 120 consecutive patients diagnosed with schizophrenia, major depression, bipolar I disorder and anxiety disorders. Included measures were the Brief Psychiatric Rating Scale (BPRS), the Clinical Global Impression Scale (CGI-S), the Personal and Social Performance Scale (PSP) and the Social and Occupational Functioning Assessment Scale (SOFAS).

Results.

The median CGI-S and BPRS scores were 5 and 16.5. Mean Mini-ICF-APP total score was 18.1. Schizophrenics' Mini-ICF-APP score was higher, while that of anxious patients was lower than in the other diagnoses. Intra-class correlations (ICC) revealed a significant inter-rater agreement for total score (ICC 0.987) and for each item of the Mini-ICF-APP. The test–retest agreement was also highly significant (ICC 0.993). The total score of the Mini-ICF-APP obtained good negative correlations with PSP (r_s = −0.767) and with SOFAS scores (r_s = −0.790). The distribution items of the Mini-ICF-APP showed some skewness, indicating that self-care (item 12) and mobility (item 13) were amply preserved in most patients. The Mini-ICF-APP total score was significantly correlated with both CGI-S (r_s = 0.777) and BPRS (r_s = 0.729).

Conclusions.

As a short instrument, the Mini-ICF-APP scale seems to be well suited to everyday psychiatric practice as a means of monitoring changes in psychosocial functioning, in particular in schizophrenic patients.

Introduction

Mental disorders are often associated with problems in social or occupational adjustment and functioning (Hensing et al. 2000; Savikko et al. 2001; Linden & Weidner, 2005; Burns & Patrick, 2007). According to the bio-psycho-social model of the International Classification of Functioning, Disability and Health, ICF (World Health Organization, 2001) illness must not only be described on the level of ‘impairment of functions’ (i.e., symptoms of illness like reduction in drive) but also in terms of ‘limitations in activities or capacities’ (e.g. inability to organize a task) and ‘restrictions in participation’ (e.g. reduced productivity at work). The ICF is grounded on the concept that the relations between functions, capacities and participation are not linear but interactive.

Since the publication of the ICF, research on the measurement of social and occupational consequences of illnesses has increased (Boldt et al. 2005; Cieza & Stucki, 2005; Stier-Jarmer et al. 2005; Burns & Patrick, 2007).

In the field of mental disorders, the relation between dysfunctions, dyscapacities and problems with participation poses special problems (Baron & Linden, 2008, 2009). Important domains of capacities which are impaired especially in the presence of mental illness are adherence to regulations, planning and structuring of tasks, flexibility, endurance, assertiveness, self-maintenance, mobility or competence to make judgements or decisions.

A question in need of further study is whether different mental disorders can be characterized not only by different impairment of functions but also by different limitations in capacities. The answer to this question would allow to say which capacities are still intact or not in certain mental disorders. This could help to look for specific compensatory treatments, or to find living conditions or jobs tailored to the affected individual.

The assessment of limitations in capacities can be done with the Mini-ICF-APP (Mini instrument for the observer rating according to ICF of Activities and Participation in Psychological disorders), a rating scale that was originally developed in a German version (Linden et al. 2009).

The main aim of this study was to assess the inter-rater and test–retest reliability and the validity of the Italian version of the Mini-ICF-APP. Secondary aims were to analyse the item performance of the Mini-ICF-APP in order to explore the sensitivity of this instrument, and to evaluate the relationship between social functioning and psychopathology.

Methods

We recruited 120 consecutive patients attending the Community Mental Health Centre (CMHC) of North-Udine (Italy). These patients had been diagnosed with four disorders: 30 patients with schizophrenia, 30 with major depression, 30 with bipolar I disorder and 30 with anxiety disorders. The diagnoses were assigned on a clinical ground by the psychiatrists working in the CHMC, using the DSM-IV nosography. All patients signed a written consent to participate. The study was approved by the Local Ethics Committee.

The author of the original German version of the Mini-ICF-APP supervised the validation study of the Italian version of the instrument and is accredited as one of the author of this paper (ML).

Instruments

For the patients' assessment, the included measures were a socio-demographic form, the Mini-ICF-APP scale, the BPRS scale, the CGI scale, the PSP scale and the SOFAS scale.

The Mini-ICF-APP (Linden & Baron, 2005; Baron & Linden, 2008, 2009; Linden et al. 2009) is a clinician-rated measure for limitations of capacities in the context of mental disorders. It has been developed following the structure and dimensions of ICF (World Health Organization, 2001), with additional reference to definitions of the Groningen Social Disabilities Schedule II (GSDS II; Wiersma et al. 1988). Activity/capacity limitations are rated under the aspect of competence – means activity competence limitations. Thirteen domains are assessed: (1) adherence to regulations, (2) planning and structuring of tasks, (3) flexibility, (4) competency, (5) endurance, (6) assertiveness, (7) contact with others, (8) public exposure, (9) intimacy, (10) non-work activities, (11) self-maintenance, (12), mobility and (13) competence to judge and decide. Each dimension is rated on a five-point Likert-scale (0: no disability; 1: mild disability; 2: moderate disability; 3: severe disability; 4: total disability).

The Mini-ICF-APP rater must get information on the ‘uniform standard environment’ or social reference group which applies to the case – e.g. from what social or ethnic group does the person come, is it a man or a woman, is it a labourer or an academic, what degree of professional training does he or she have, which role expectations apply to this person in his/her private or occupational life, etc. Personal factors according to ICF, such as constitution or education-related performance limits, are not taken into account in respective judgments. The appraisal is based on available information (self-reports, information from the family, colleagues, friends, caregivers and health staff involved in the case, from clinical observations and from medical exams or standardized tests) about the person and his/her life situation. Observations are in general of greater significance than reports by the patient. The assessment with the Mini-ICF-APP requires at least a sufficient knowledge of the proband. This is the usual situation of people in care with community mental health services. When a sufficient knowledge of the patient has been acquired, the compilation of the Mini-ICF-APP needs about 20 min. In a rating manual, available in German, English and Italian language, anchor definitions and suggestions for interview questions for each item have been given (Linden et al. 2009).

The Brief Psychiatric Rating Scale (BPRS) is a clinician-rated tool designed to assess change in severity of psychopathology. The BPRS was initially designed to measure symptom change in patients with psychotic illness. Thus, the items on the BPRS focus on symptoms that are common in patients with psychotic disorders, including schizophrenia and other psychotic disorders, as well as those found in patients with mood disorders. It has been extensively used in research in many countries as a measure for the overall symptom severity and the change over time (Overall & Gorham, 1962). The 24 items of the BPRS were scored from 0 (no pathology) to 6 (extreme severity).

The Social and Occupational Functioning Assessment Scale (SOFAS) has been derived from the Global Assessment of Functioning Scale (GAF) which is part of the DSM-IV (APA, 1994; Morosini et al. 2000). It allows operationalizing symptomatology, work, social contacts and function. It differs from the GAF in that it focuses exclusively on the individual's level of social and occupational functioning and is not directly influenced by the overall severity of the individual's psychological symptoms. Also in contrast to the GAF Scale, any impairment in social and occupational functioning that is due to general medical conditions is considered in making the SOFAS rating. The SOFAS considers social and occupational functioning on a continuum from excellent functioning to grossly impaired functioning. Include impairments in functioning due to physical limitations, as well as those due to mental impairments. To be counted, impairment must be a direct consequence of mental and physical health problems; the effects of lack of opportunity and other environmental limitations are not to be considered. The score range is from 1 to 100, with lower scores indicating more severe impairment.

The Personal and Social Performance Scale (PSP) (Morosini et al. 2000; Juckel et al. 2008; Patrick et al. 2009) is a clinician-rated tool designed for completion by trained clinical staff. PSP has been developed through focus groups and reliability studies on the basis of the social functioning component of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS). Patient functioning is assessed in four main areas: ‘socially useful activities’, ‘personal and social relationships’, ‘self-care’ and ‘disturbing and aggressive behaviours’. Difficulty in each area is rated on a single item using a six-point scale, where lower ratings indicate better functioning. A global item is rated by the interviewer, ranging from 1 to 100 in ten-point intervals, where lower scores indicate poorer functioning. The PSP global score incorporates ratings for the four main areas, as well as levels of functioning in other areas, to adjust the precision of the rating within the ten-point intervals. In this study, we considered only PSP global score for comparison with Mini-ICF-APP.

The Clinical Global Impression Scale (CGI-S) is a 3-item observer-rated scale that measures illness severity, global improvement or change (CGI-C) and therapeutic response (Guy, 1976). The illness severity and improvement sections of the instrument are used more frequently than the therapeutic response section in both clinical and research settings. In this study, only the CGI-S has been used.

Raters and procedures

In the inter-rater reliability assessment patients with the four selected diagnoses have been interviewed by two expert psychiatrists who have attended a national training course to use the ICF system and have participated in RCT and research mental health projects. One psychiatrist interviewed the patient, while the other participated as an observer. All ratings were carried out by both interviewers independently.

The approach of test–retest agreement assumes that there is no substantial change in the construct being measured between the two occasions. However, functioning is changeable over time and should indeed vary in the course of therapeutic interventions. To solve this problem, patients were interviewed with the PSP and the CGI-S in a period ranging from 1 to 2 weeks following the first interview by one psychiatrist blind to the scores. Only patients whose PSP and CGI scores were the same at both assessments were included in the analysis and re-interviewed with the Mini-ICF-APP.

The validity assessment was carried out by correlation analysis between Mini-ICF-APP and PSP measures of all patients. The PSP global score was the gold standard measure. In addition, a correlation analysis between Mini-ICF-APP and SOFAS was carried out, since in the literature the SOFAS is also used to assess the social capacities.

To evaluate the sensitivity of the Mini-ICF-APP for very mild or severe forms of social dysfunction, we calculated the percent of the sample scoring the lowest (floor effect) and the highest (ceiling effect) for each item. These effects were evaluated for the sample as a whole and for the four diagnoses separately, to determine whether the range of limitations of capacities was restricted for one diagnosis more than for another. Since there are no absolute guidelines to determine how much of a ceiling or floor effect is too much, we used 20% as the value at which ceiling or floor effects were considered substantial, in analogy with Holmes et al. (1996) and as suggested by Heinemann (2010). To calculate the skewness, we used the rule of thumb suggested by Bulmer in his classic text of statistics (1979):

• •less than −1 or greater than +1: distribution highly skewed
• •between −1 and −0.5 or between +0.5 and +1: distribution moderately skewed
• •between −0.5 and +0.5: distribution approximately symmetric.

Sample size

The sample size calculation for the inter-rater agreement was based on the following assumptions: investigators were hoping for ICC of at least 0.80 (excellent agreement) and had determined that ICC of 0.70 (good agreement) would be the minimally acceptable level of agreement (Juckel et al. 2008). Based on these assumptions, with two interviewers, α = 0.05 and β = 0.20, 118 evaluable patients were required (Walter et al. 1998).

The sample size calculation for the test–retest agreement was based on the following assumptions: investigators were hoping for ICC of at least 0.80 (excellent agreement) and had determined that ICC of 0.5 (moderate agreement) would be the minimally acceptable level of agreement. Based on these assumptions, with two interviewers, α = 0.05 and β = 0.20, 22 evaluable patients were required (Walter et al. 1998).

The sample size calculation for the convergent validity was based on the following assumptions: investigators were hoping for correlations of at least 0.85 (strong correlation), and had determined that r of 0.75 (good correlation) would be the minimally acceptable level of correlation (Juckel et al. 2008). Based on these assumptions, with α = 0.05 and β = 0.20, 101 evaluable patients were required.

In order to achieve the above described goals, we planned to recruit 120 subjects: 30 patients suffering from schizophrenia, 30 from anxiety disorders, 30 from depression and 30 from bipolar I disorders.

Statistical analysis

Quantitative data were summarized by appropriate descriptive statistics (mean and standard deviation or median and range). The Shapiro–Francia and Levene's tests, respectively, were used to assess for normality and homogeneity of variances of the distributions. Qualitative data were summarized by means of contingency tables.

For quantitative data, comparisons among groups were performed with a general linear model (GLM), if the assumptions of normality and homogeneity of variances were verified. GLM was used even in the presence of moderate violations in the assumption of homoscedasticity because the sample sizes were equal in all groups (Box & Andersen, 1955). Otherwise, we used the Kruskal–Wallis test. T-test or Mann–Whitney test was used to compare continuous data between groups, as appropriate. Bonferroni's correction was performed for multiple comparisons. Chi-square tests were used to analyse categorical values; when assumptions for chi-square test were not verified, Fisher's exact test was used.

The inter-rater reliability was assessed by calculating the intraclass correlation coefficient (ICC) (Shrout & Fleiss, 1979), because the items of Mini-IFC-APP are rated on ordinal scale. The ICC was even used to compare the Mini-ICF-APP scores in the two observations made by each psychiatrist (test–retest).

Pearson's or Spearman's correlation analyses, as appropriate, were used to estimate the association between Mini-ICF-APP and PSP or SOFAS scales. The same analyses were used to explore the association between Mini-ICF-APP and BPRS or CGI-S scores.

For correlation analyses between BPRS and Mini-ICF-APP scores, we accepted a level of significance of p ≤ 0.001.

The analyses were computed with the statistical package STATA 12 or the SPSS version 18.

Results

During the period considered for the enrolment, 120 patients were included (42.8% males and 57.8% females), selected on the basis of the consecutive presentation to the CMHC. The main characteristics of the sample are described in Table 1. Most of the patients were living independently and/or with relatives, could rely on some type of social support and were regularly employed. More than two-thirds had previous psychiatric contacts and were using psychotropic drugs. In particular, a psychotropic drug treatment was present in 60% of schizophrenic patients, 93.3% of bipolar patients, 70% of depressed patients and 86.7% of anxious patients.

Table 1.

Characteristics of the sample (No. 120 patients)

Characteristics	Statistics	%
Mean age (SD)	51.8 (14.9)
Diagnosis
Schizophrenia	30	25.0
Bipolar	30	25.0
Depression	30	25.0
Anxiety	30	25.0
Marital status
Single	58	48.3
Married	38	31.7
Separated/divorced	24	20.0
Living situation
Self-sufficient/with relatives	112	93.3
Clinic/residential facility	8	6.7
Occupation
Employed	74	61.7
Temporary job	18	15.0
Housewife/student	7	5.8
Retired	21	17.5
Education
Primary school	24	20.0
Secondary school	40	33.3
High school diploma	42	35.0
University degree	14	11.7
Previous psychiatric contacts
No	39	32.5
Yes	81	67.5
Actual use of psychotropic drugs
No	27	22.5
Yes	93	77.5
Social support
Absent	13	10.8
Family/friends	70	58.3
Public/co-op services	37	30.8

The median CGI-S score was 5 (range 3–7), with median scores ranging from 4 (i.e., moderately ill) in anxious patients to 6 (i.e., severely ill) in schizophrenics. BPRS median value for the whole sample was 46 (range 28–113), for schizophrenic patients was 75 (range 31–113), for bipolar patients was 42 (range 31–74), for depressed patients was 39 (range 28–101) and for anxious patients was 40 (range 35–65).

The mean SOFAS score in the whole sample was 58.03 ± 16.40 (range: 15–100) and the mean PSP global score was 59.61 ± 17.19 (range: 16–90).

Mean Mini-ICF-APP total score was 18.08 ± 11.60, with a moderate variance in the level of disability (range: 0–45). Schizophrenic patients showed a mean Mini-ICF-APP score of 30.33 ± 8.74, bipolar patients of 18.87 ± 7.86, depressed patients of 15.77 ± 10.25 and anxious patients of 7.37 ± 5.37. The Shapiro–Francia test results were p = 0.203 for schizophrenic group, p = 0.132 for bipolar group, p = 0.112 for depressed group and p = 0.310 for anxiety group; the homogeneity of variance was tested using the Levene's test (p = 0.038). The GLM showed that schizophrenics' Mini-ICF-APP total score was significantly higher than in all other diagnoses (p < 0.0001), while the Mini-ICF-APP total score of anxious patients was significantly lower than in all other diagnoses (p < 0.0001 v. schizophrenic and bipolar patients; p = 0.001 v. depressed patients). No differences were present between bipolar and depressed patients.

Reliability and validity

Intra-class correlations revealed significant inter-rater agreement for the ratings of the Mini-ICF-APP total score (ICC 0.987, IC_95% 0.981–0.990). The rater agreement was shown to be excellent for all items of the Mini-ICF-APP, with ICC ranging from 0.882 to 0.968 (Table 2).

Table 2.

Intra-class correlations of Mini-ICF-APP scores for the two raters

Item	ICC	IC95%
1. Adherence to regulations	0.968	0.954–0.978
2. Structuring of tasks	0.942	0.917–0.960
3. Flexibility	0.907	0.867–0.935
4. Competency	0.956	0.936–0.969
5. Judgement	0.963	0.947–0.974
6. Endurance	0.953	0.933–0.967
7. Assertiveness	0.926	0.894–0.948
8. Contact with others	0.917	0.881–0.942
9. Group integration	0.925	0.893–0.948
10. Intimate relations	0.938	0.911–0.957
11. Spontaneous activities	0.932	0.903–0.953
12. Self-care	0.959	0.942–0.972
13. Mobility	0.882	0.831–0.918

Highly significant positive correlation could be found for the Mini-ICF-APP total score at the first assessment in relation to the scores at the following assessment (ICC 0.993, IC_95% 0.984–0.997). The test–retest agreement was excellent for all items apart from item 13, which is anyway at the limits of the excellence (Table 3).

Table 3.

Test–retest correlations of Mini-ICF-APP scores

Item	ICC	IC95%
1. Adherence to regulations	0.969	0.925–0.987
2. Structuring of tasks	0.931	0.834–0.971
3. Flexibility	0.848	0.635–0.937
4. Competency	0.967	0.921–0.986
5. Judgement	0.946	0.869–0.977
6. Endurance	0.964	0.913–0.985
7. Assertiveness	0.982	0.957–0.993
8. Contact with others	0.931	0.835–0.971
9. Group integration	0.929	0.911–0.950
10. Intimate relations	0.949	0.878–0.979
11. Spontaneous activities	0.977	0.945–0.990
12. Self-care	0.982	0.956–0.992
13. Mobility	0.792	0.499–0.914

As for the convergent validity, Fig. 1 shows that the total score of the Mini-ICF-P obtained good negative correlations with PSP total score (r_s = −0.767; p < 0.0001) and with SOFAS total score (r_s = −0.790; p < 0.0001).

Fig. 1.

Correlations between Mini-ICF-APP total score and the values of PSP, SOFAS, CGI-S and BPRS total scores.

Item performance

As shown in Table 4, the item analysis on the performance of the 120 respondents showed that a floor effect was present in 8 out of 13 items, while no ceiling effect was detected. To note, mobility (item 13) showed a marked floor effect (85.8%), indicating that the vast majority of the patients did not show any problem to move around and also self-care (item 12) was sufficiently preserved in the majority of patients. These two items were also highly skewed.

Table 4.

Item performance of Mini-ICF-APP

Items	1	2	3	4	5	6	7	8	9	10	11	12	13
	Adherence to regulations	Structuring of tasks	Flexibility	Competency	Judgement	Endurance	Assertiveness	Contact with others	Group integration	Intimate relations	Spontaneous activities	Self-care	Mobility
Schizophrenia
Mean (SD)	2.1 (1.0)	2.4 (1.1)	2.8 (0.9)	2.5 (1.0)	2.8 (0.9)	2.5 (1.0)	2.7 (0.8)	2.8 (0.8)	2.8 (0.9)	2.9 (1.0)	2.6 (1.1)	1.3 (1.0)	0.2 (0.3)
Floor	3.3%	6.7%	0.0%	3.3%	3.3%	3.3%	0.0%	0.0%	0.0%	0.0%	3.3%	20.0%	86.7%
Ceiling	10.0%	13.3%	30.0%	13.3%	23.3%	10.0%	16.7%	16.7%	20.0%	36.7%	20.0%	3.3%	0.0%
Skewness	0.2	−0.5	−0.2	−0.5	−0.9	−0.6	−0.1	−0.3	−0.6	−0.4	−0.4	0.7	2.9
Bipolar
Mean (SD)	1.5 (1.0)	1.6 (0.8)	1.8 (0.8)	1.4 (0.8)	1.3 (0.9)	1.6 (0.8)	1.6 (0.7)	1.9 (0.7)	1.9 (0.8)	1.7 (0.8)	1.7 (1.0)	0.7 (0.8)	0.1 (0.3)
Floor	23.3%	13.3%	3.3%	13.3%	23.3%	10.0%	6.7%	0.0%	3.3%	10.0%	13.3%	50.0%	90.0%
Ceiling	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%
Skewness	−0.5	−0.6	−0.0	−0.4	−0.9	−0.5	−0.7	0.1	−0.2	−0.6	−0.2	1.0	2.8
Depression
Mean (SD)	0.7 (1.0)	1.2 (1.1)	1.4 (1.0)	1.0 (1.1)	1.3 (1.2)	1.1 (1.1)	1.3 (1.2)	1.9 (0.9)	2.0 (0.9)	1.7 (0.9)	1.3 (1.1)	0.6 (1.0)	0.3 (0.7)
Floor	60.0%	33.3%	20.0%	43.3%	36.7%	36.7%	33.3%	3.3%	3.3%	6.7%	30.0%	66.7%	80.0%
Ceiling	0.0%	0.0%	0.0%	0.0%	3.3%	3.3%	3.3%	0.0%	3.3%	0.0%	3.3%	0.0%	0.0%
Skewness	1.3	0.5	0.2	0.6	0.5	0.9	0.5	−0.0	0.1	0.1	0.4	1.6	1.9
Anxiety
Mean (SD)	0.5 (0.8)	0.6 (0.7)	1.0 (1.0)	0.3 (0.5)	0.4 (0.7)	0.3 (0.6)	8.8 (0.7)	0.9 (0.7)	0.7 (0.7)	0.9 (0.9)	0.4 (0.6)	0.2 (0.4)	0.2 (0.5)
Floor	70.0%	56.7%	30.0%	73.3%	66.7%	76.7%	36.7%	26.7%	43.3%	43.3%	63.3%	76.7%	86.7%
Ceiling	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%	0.0%
Skewness	1.7	0.9	0.0	1.6	1.3	1.8	0.3	0.1	0.5	0.5	1.2	1.3	2.7
Whole sample
Mean (SD)	1.2 (1.1)	1.4 (1.1)	1.7 (1.1)	1.3 (1.2)	1.5 (1.3)	1.4 (1.2)	1.6 (1.1)	1.9 (1.0)	1.8 (1.1)	1.8 (1.2)	1.5 (1.2)	0.7 (0.9)	0.2 (0.5)
Floor	39.2%	27.5%	13.3%	33.3%	32.5%	31.7%	19.2%	7.5%	12.5%	15.0%	27.5%	53.3%	85.8%
Ceiling	2.5%	3.3%	7.5%	3.3%	6.7%	3.3%	5.0%	4.2%	5.8%	9.2%	5.8%	0.8%	0.0%
Skewness	0.5	0.3	0.2	0.4	0.3	0.3	0.2	0.1	0.0	0.1	0.3	1.3	2.6

However, a close inspection into the item performance within each disorder showed striking differences. A floor effect was always present in anxious patients and in nine items (69.2%) of depressed patients. On the contrary, schizophrenic patients showed only ceiling effects, in particular, on dimensions of flexibility, judgments and intimate relations.

Social functioning and psychopathology

As shown in Fig. 1, a good correlation was present between the Mini-ICF-APP total score and the CGI-S score (r_s = 0.777; p < 0.0001), and this was confirmed at the diagnostic subgroup level, for schizophrenia (r_s = 0.570; p = 0.001), bipolar (r_s = 0.536; p < 0.002), depressive (r_s = 0.664; p < 0.0001) and anxiety disorders (r_s = 0.478; p = 0.008).

Significant correlations were found between BPRS and Mini-ICF-APP total scores in the whole sample (r_s = 0.729; see also Fig. 1), and both in schizophrenia and in depressive disorders (Table 5). Moreover, several BPRS items were correlated with Mini-ICF-APP scores: more than 70% in the whole sample, around 40% in schizophrenics and depressed patients, while only one BPRS item each in bipolar and anxious patients.

Table 5.

Correlations (Spearman's rho) of Mini-ICF-APP total scores for the whole sample and by diagnostic groups with single items and total scores of BPRS. Correlations significant at the 0.001 level (two-tailed) were highlighted with grey cells

BPRS	ICF total score all patients No. 120	ICF total score schizophrenia No. 30	ICF total score bipolar No. 30	ICF total score depression No. 30	ICF total score anxiety No. 30
1. Somatic concern	0.106	−0.079	0.273	0.324	0.056
Sign.	0.251	0.679	0.145	0.080	0.770
2. Anxiety	0.013	0.102	0.329	0.622	−0.167
Sign.	0.884	0.593	0.076	0.000	0.378
3. Depression	0.015	−0.118	0.048	0.538	0.381
Sign.	0.871	0.536	0.803	0.002	0.038
4. Suicidality	0.009	−0.496	0.161	0.565	0.105
Sign.	0.925	0.005	0.395	0.001	0.582
5. Guilt	0.611	0.555	0.306	0.520	0.374
Sign.	0.000	0.001	0.101	0.003	0.041
6. Hostility	0.654	0.704	0.530	0.484	0.437
Sign.	0.000	0.000	0.003	0.007	0.016
7. Elated mood	0.695	0.833	0.346	0.613	0.276
Sign.	0.000	0.000	0.061	0.000	0.140
8. Grandiosity	0.567	0.625	0.158	0.311	0.429
Sign.	0.000	0.000	0.403	0.094	0.018
9. Suspiciousness	0.504	0.617	n.v.	n.v.	n.v.
Sign.	0.000	0.000
10. Hallucinations	0.304	0.207	0.129	0.456	0.054
Sign.	0.001	0.272	0.497	0.011	0.778
11. Unusual thought content	0.655	0.667	0.139	0.784	0.601
Sign.	0.000	0.000	0.465	0.000	0.000
12. Bizarre behaviour	0.470	0.371	0.198	0.268	0.351
Sign.	0.000	0.044	0.294	0.151	0.057
13. Self-neglect	0.187	0.037	0.073	0.597	0.282
Sign.	0.041	0.848	0.700	0.000	0.131
14. Disorientation	0.646	0.268	0.453	0.785	0.401
Sign.	0.000	0.152	0.012	0.000	0.028
15. Conceptual disorganization	0.184	0.443	−0.090	0.684	−0.251
Sign.	0.044	0.014	0.636	0.000	0.181
16. Blunted affect	0.345	0.401	−0.044	0.418	0.365
Sign.	0.000	0.028	0.817	0.021	0.047
17. Emotional withdrawal	0.659	0.761	0.677	0.517	0.466
Sign.	0.000	0.000	0.000	0.003	0.009
18. Motor retardation	0.567	0.508	0.425	0.535	0.270
Sign.	0.000	0.004	0.019	0.002	0.149
19. Tension	0.183	−0.172	−0.212	0.677	0.159
Sign.	0.045	0.364	0.262	0.000	0.400
20. Un-cooperativeness	0.548	0.558	0.448	0.458	0.477
Sign.	0.000	0.001	0.013	0.011	0.008
21. Excitement	0.668	0.732	0.414	0.612	0.341
Sign.	0.000	0.000	0.023	0.000	0.066
22. Distractibility	0.474	0.312	0.163	0.268	n.v.
Sign.	0.000	0.093	0.388	0.151
23. Motor hyperactivity	0.487	0.471	0.367	0.268	n.v.
Sign.	0.000	0.009	0.046	0.151
24. Mannerisms/posturisms	0.459	0.342	0.181	0.601	0.056
Sign.	0.000	0.064	0.339	0.000	0.770
Total	0.729	0.777	0.479	0.804	0.479
Sign.	0.000	0.000	0.007	0.000	0.007

n.v.: not valuable because BPRS items are constant.

Discussion

Our study aimed at validating a new instrument to assess disabilities and/or impairments and limitations in social functioning/activities/participation in mentally ill subjects. Our sample of patients was composed by a group of people representative of subjects attending a community psychiatric centre in Italy. The fact that it was a relatively well functioning group of people (nearly everybody was working) with high level of education (47% had high school or more) may confirm that community psychiatric services in our reality are only relatively burdened from stigma attitudes of the population, since they are accessible to people with any level of education and employment. In our opinion, there is no a priori reason to think that the characteristics of the sample may have influenced the reliability and validity analyses.

A preliminary question is about the need of a new tool in the panorama of instruments already available to analyse the social integration of mental health patients. To answer this question, we must consider that this instrument has been developed following the structure and dimensions of ICF (World Health Organization, 2001). The ICF has chapters under the titles of ‘body functions’, ‘body structures’, ‘environmental factors’ and ‘activities and participation’. ‘Activities’ describe the behaviour of a person and are further subdivided into ‘performance’ (what a person does) and ‘capacities’ (what a person could do). ‘Participation’ describes role fulfilment and inclusion. In the chapter ‘activities and participation’ there are items that narrowly describe activities (be it performance or capacity) like ‘d4300 lifting’ (rising up an object in order to move it from a lower to a higher level), while others describe rather complex participation such as ‘d760 family relationships’ (creating and maintaining kinship relationships). The ICF acknowledges that the lack of differentiation between activities and participation is a problem, and that additional instruments are needed to represent and better differentiate the underlying dimensions of the ICF. For clinical purposes, it is important to describe capacities (instead of performance) as targets of assessment and treatment. The ICF suggests that the qualification of capacity limitations should be done in reference to a ‘standard environment’ which is valid for the individual person. The Mini-ICF-APP follows this concept by describing capacities in reference to the environment which gives the standard for a special individual and by judging the degree of problems in role fulfilment. In order to reduce the larger number of items of the ICF, the Mini-ICF-APP proposes only 13 summarizing dimensions which encompass such capacities which are preferably impaired in the presence of mental disorders. For example, the dimension ‘contact to others’ encompasses ICF categories such as relating with others, formal relationships, starting a conversation, sustaining a conversation and discussion. It was developed for clinical use in order to allow a differentiated, economical and reliable description of disabilities within the context of mental disorders.

There are many other instruments for the assessment of limitations of capacities and participation restrictions in general, as well as in respect to mental disorders. A well known and established example is the ADL scale (activities of daily living; Katz et al. 1963). It is used in dementia, but does not catch problems of other mental disorders. WHO itself has published the WHODAS 2.0 (Üstün et al. 2010), a 12- or 36-item self- and interviewer-administered instrument, which covers six domains: cognition, mobility, self-care, getting along, life activities and participation. Similar to the ICF, the WHODAS 2.0 does not discriminate between capacities and participation. Most items refer to participation (e.g. ‘making new friends’ and ‘doing household tasks’), while only few and very specific capacities are mentioned (‘remembering to do important things’ and ‘standing up from sitting’), which furthermore are not very relevant for mental disorders. The PSP (Morosini et al. 2000; Juckel et al. 2008; Patrick et al. 2009) measures participation only, without specification of capacities. The SOFAS (APA, 1994; Morosini et al. 2000) does mix functions and participation, but does not describe capacities at all. The Mini-ICF-APP therefore fills a gap in addition to these existing instruments for the measurement of disability.

In conclusion, the Mini-ICF-APP seems to be a useful tool with specific characteristics not present in other instruments. It is sufficiently detailed and shaped on the needs of mental health disorders and allows an important link to the classification of the World Health Organization. Moreover, it takes into consideration the views of the family and of close informants, which are often involved in the case (Ruggeri et al. 2008; Prytys et al. 2011). This is not to say that the members of the patient's family or the caregivers are always reliable and valid, but that their personal views should be confronted with that of the clinician and of the rest of the mental health staff. Under this respect, a few studies analysed the variance across different sources of history, concluding that taking careful stock of all sources of information is required to fully understand the psychotic symptoms as well as the functional abilities and outcomes of the patient in general (Becchi et al. 2004; Ho et al. 2004; Harvey, 2011; Sabbag et al. 2011).

Psychometric properties

We found excellent psychometric properties, which present the Mini-ICF-APP scale as a reliable and valid instrument for assessing capacities of patients with different mental health diagnoses. We obtained a rater agreement of ICC = 0.987, with ICC ranging from 0.882 to 0.968 across the 13 items. Even the test–retest was equally excellent for all items, with only item 13 (mobility) showing an agreement slightly less than excellent.

These results can be compared with those obtained using the original version of Mini-ICF-APP in a sample of 213 patients attending the rehabilitation centre of the German Pension Insurance Association (Linden et al. 2009). In this sample, 61% of the subjects suffered from disorders in section F4 of the ICD-10 (neurotic, stress and somatoform disorders), 29% from F3 (affective disorders) and 10% from F6 (personality disorders). Highly significant correlations were found between the independent trained raters (0.92; range 0.71–0.99). No test–retest analysis was performed.

Our analyses showed also a good correlation between Mini-ICF-APP and both PSP total score (r_s = −0.767) and SOFAS total score (r_s = −0.790). This confirms the results obtained by Schaub et al. (2011) on a sample of 103 outpatients with schizophrenia or with a schizoaffective disorder. These authors found that the total score of PSP was significantly correlated with the Mini-ICF-APP (r = −0.90) and with the SOFAS score (r = 0.94).

Item performance

We used customary analyses for a Likert scale, such as floor and ceiling effects, to measure the distribution of the sample at the lowest and highest scores of the Mini-ICF-APP. By definition, floor and ceiling effects occur when a measure's lowest and highest scores, respectively, are unable to assess a patient's level of ability. Our results show that the Mini-ICF-APP, which graduates the levels of disability from none to complete, measures different dimensions than psychopathology scales. Some types of severe psychopathology do and others do not result in capacity limitations. Therefore, many disorders do not come along with substantial disability (floor effect). Social consequences of illness are different than medical parameters.

Correlation with psychopathology

We found a number of correlations between the levels of social functioning and psychopathology, as measured with the BPRS. These results were particularly evident for schizophrenia and for depression. Unlike other studies that focused only on schizophrenia (Patrick et al. 2009; Schaub et al. 2011), our assessment explored the activity and participation restrictions also in potentially less severe disorders. The level of social restrictions was different across diagnostic groups. The schizophrenic patients were the most disabled and the anxious patients the least, following the distribution of the severity of illness as measured with CGI-S.

The social restrictions were related to several symptoms, and this varied in relation with the diagnosis. In particular, a few negative and positive symptoms of schizophrenia had a strong impact on social functioning. On the other hand, the quite complete lack of correlation of BPRS items with Mini-ICF-APP in bipolar patients may be explained by the fact that most of these patients were on treatment when entering into this study. Yet, from another viewpoint this explanation might be insufficient, since bipolar patients were markedly ill on the CGI-S (median score of 5).

In conclusion, we were able to demonstrate that the Mini-ICF-APP scale is a reliable and valid instrument to assess activity and participation restrictions according to the ICF in patients with different psychiatric disorders. As a short instrument, the Mini-ICF-APP scale seems to be well suited to everyday psychiatric practice as a means of monitoring changes in psychosocial functioning in particular in schizophrenic patients. Our aims are to further use this instrument for the analysis of efficacy of community mental health treatments, following the recommendations put forward by the ICF.

Declaration of interests

No fees, grants or private/public funding were involved in this study. None of the authors received any economic support for the preparation of this paper. No potential conflict of interest related to the present paper is present.