Abstract
Objective
The WHODAS-II was substantially modified for use in the World Mental Health Surveys. This paper considers the modified WHODAS-II’s psychometric properties and implications of filter items employed to reduce respondent burden.
Study design and setting
Seventeen surveys in 16 countries administered a modified WHODAS-II to population samples (N=38,934 adults). Modifications included introducing filter questions for four sub-scales and substituting questions on the number of days activity was limited for the Life Activities domain. We evaluated distributional properties, reliability, and validity of the modified WHODAS-II.
Results
Most respondents (77%–99%) had zero scores on filtered subscales. Lower bound estimates of internal consistency (alpha) for the filtered subscales were typically in the 0.70’s, but were higher for the Global scale. Loadings of subscale scores on a Global Disability factor were moderate-to-high. Correlations with the Sheehan Disability Scale were modest but consistently positive, while correlations with SF-12 Physical Component Summary were considerably higher. Cross-national variability in disability scores was observed, but was not readily explainable.
Conclusions
Internal consistency and validity of the modified WHODAS-II was generally supported, but use of filter questions impaired measurement properties. Group differences in modified WHODAS-II disability scores may be compared within, but not necessarily across, countries.
Keywords: Disability, Measurement, Reliability, Validity, Survey, Epidemiology
The International Classification of Functioning, Disability and Health (ICF) [1] organizes the consequences of disease into four dimensions: body functions and structure (symptoms and impairment); activities; participation, and environmental factors [2]. The World Health Organization Disability Assessment Schedule (WHODAS-II) is a disability instrument designed based on the ICF framework, assessing six domains of functioning in daily life. The WHODAS-II is a standardized measurement of disability for use in diverse cultural settings, translated into 16 languages to date [3,4]. The instrument allows group comparisons of global disability and domain-specific disability for physical and mental disorders. The WHODAS-II domains include: Understanding and Communicating; Getting Around; Self Care; Getting Along With Others; Life Activities; and Participation in Society [3,4]. Initial psychometric analyses carried out by the WHO indicated that the WHODAS-II domains yielded a two level hierarchical structure in which individual items load onto one of six domains, which in turn load onto a Global Disability latent variable. The WHODAS-II website reports that all six domains have factor loadings of at least 0.7 on a Global Disability latent variable [3].
There is a growing literature evaluating psychometric properties of the WHODAS-II [5–13]. Among primary care patients with depression or back pain, Chwastiak and Von Korff [5] reported high internal consistency for all domains except Self-Care (whose alphas were in the vicinity of 0.65), and good responsiveness to change. They also found good convergent validity in relation to the SF-36, the Work Limitations Questionnaire and condition-specific severity scales. Perini et al. [6] found that the WHODAS-II was associated with symptom change among persons with anxiety disorders. Chisolm et al [7] reported moderate to high correlations with SF-36 scales, adequate test-retest reliability, and good internal consistency for all domains except Getting Along With Others. Overall, initial evaluation of the WHODAS-II suggests acceptable internal consistency, test-retest reliability, convergent validity and responsiveness to change.
This paper assesses the psychometric performance of the WHODAS-II as modified for the World Mental Health (WMH) Surveys [14]. The WMH Survey Initiative is a series of population surveys concerning mental disorders conducted in participating countries on a global scale. Substantial changes to the WHODAS-II were made for the WMH surveys. While these changes were deemed necessary to achieve the overall goals of the WMH surveys, they have the potential to alter the psychometric properties of the WHODAS-II. The purpose of this paper is to assess the psychometric properties of the modified WHODAS-II including: 1) internal consistency of the five retained domains; 2) convergent validity with the Sheehan Disability Scale and SF-12 Physical and Mental Component Summary scores; and 3) assessing with confirmatory factor analysis whether the five retained domain scores have high loadings on a global disability latent variable. We also present population normative data for a Global Disability Scale based on the five domain scores. Since the WMH version of the WHODAS-II will be used to assess functional disability in reporting WMH survey results, understanding its strengths and limitations is important for interpretation of disability studies reported by the WMH surveys. The use of filter questions is an increasingly common method for reducing respondent burden in multi-purpose surveys. This paper considers how well the filter questions introduced for the WHODAS-II subscales performed.
METHODS
Samples
The analyses reported here are based on seventeen surveys carried out in 16 countries in the Americas (Colombia, Mexico, United States), Europe (Belgium, France, Germany, Italy, Netherlands, Spain, Ukraine), the Middle East/Africa (Israel, Lebanon, Nigeria), Asia (Japan, separate surveys in Beijing and Shanghai in the People’s Republic of China), and the South Pacific (New Zealand). All surveys were based on multi-stage, clustered area probability household samples in 2001 to 2004. All interviews were carried out in person by trained lay interviewers. The six Western European surveys were carried out jointly in 2001–2003.
In all surveys except Israel, internal sub-sampling was used to reduce respondent burden by dividing the interview into two parts. Part 1 included a diagnostic assessment of mental disorders. Part 2 included additional information relevant to a wide range of survey aims, including assessment of functional disability and the modified WHODAS-II. All respondents completed Part 1. All Part 1 respondents who met criteria for any mental disorder and a random sample of all other respondents were administered Part 2. Results from Part 2 respondents can be weighted by the inverse of their probability of selection for Part 2 to adjust for differential sampling, in addition to sample design weights. The psychometric and correlational analyses reported in this paper are based on unweighted data, whereas population estimates of means, standard deviations and order statistics are based on weighted data. The samples showed substantial cross-national differences in age structure (younger in less-developed countries) and educational status (lower in less-developed countries), but not in gender distribution.
Training and Field Procedures
The central World Mental Health staff trained bilingual supervisors in each country. Consistent interviewer training documents and procedures were used across surveys. The WHO translation protocol was used to translate instruments and training materials. Standardized descriptions of the goals and procedures of the study, data usage, and protection and rights of respondents were provided in both written and verbal form to all potentially eligible respondents before obtaining verbal informed consent for participation in the survey. Quality control protocols, described in more detail elsewhere [15], were standardized across countries to check on interviewer accuracy and to specify data cleaning and coding procedures. The institutional review board of the organization that coordinated the survey in each country approved and monitored compliance with procedures for obtaining informed consent and protecting human subjects.
Disability measures
The modified version of the WHODAS-II (see Appendix I) assessed five domains of functioning: Understanding and Communicating; Self-Care, Getting Around, Getting Along with Others, and Life Activities. In the first four domains, there was a filter question, a series of specific items with severity ratings, and a question about days in the last month that these kinds of interference occurred. The Life Activities domain did not use a filter question.
Appendix.
The World Mental Health Survey Initiatve version of the WHODAS-II
| Life Activities (Domain 5) | NUMBER OF DAYS (0–30) | DK (8) | RF (9) |
|---|---|---|---|
| *FD4. Beginning yesterday and going back 30 days, how many days out of the past 30 were you totally unable to work or carry out your normal activities? | ________ | 998 | 999 |
| DAYS | |||
|
| |||
| *FD7. How many days out of the past 30 were you able to work and carry out your normal activities, but had to cut down on what you did or not get as much done as usual? | ________ | 998 | 999 |
| DAYS | |||
|
| |||
| *FD8. How many days out of the past 30 did you cut back on the quality of your work or how carefully you worked? | ________ | 998 | 999 |
| DAYS | |||
|
| |||
| *FD9. How many days out of the past 30 did it take an extreme effort to perform up to your usual level at work or at your other normal daily activities because of problems with either your physical health, your mental health, or your use of alcohol or drugs? | ________ | 998 | 999 |
| DAYS | |||
Modifications to the WHODAS-II
The rationale for modifying the WHODAS-II for the WMH Surveys was to reduce respondent burden, to substitute a validated method of assessing work disability for WHODAS-II items specifically assessing disability in occupational role, and to obtain estimates of activity limitation days. Modifications were made to the 36 item version of the WHODAS-II that asks about activity limitation days in the prior 30 days (without grouping the number of days) [3]. The key changes were as follows: 1) For the Understanding and Communicating, Getting Around, Self-Care, and Getting Along With Others domains, a filter question was used to determine whether additional disability questions were asked for each domain. 2) For these four domains, a question asking about days with activity interference in the past month referred to the activities mentioned in the filter question rather than the activities mentioned in the detailed questions. 3) A series of questions about activity limitations days was substituted for the WHODAS-II Life Activities domain questions. This change was made to accommodate use of the WHO Health and Work Performance Questionnaire (HPQ) [16], which was asked only of persons working for pay. 4) Items from the Participation domain concerning stigma, discrimination, and family burden were only asked of respondents with prior responses indicating significant functional disability, so the participation items were not used in scoring the WMH version of the WHODAS-II. These changes to the original WHODAS-II are substantial. Respondents who answered “no” to the filter questions for each of the first four domains were not asked any questions from the original WHODAS-II.
The Sheehan Disability Scale [17] and the SF-12 Physical and Mental Function scales [18] are employed in this paper to assess the convergent validity of the modified WHODAS-II with another disability measure. The Sheehan Disability Scale items were asked in relation to one chronic condition reported by respondents with one or more chronic physical conditions, with random selection of one chronic condition if more than one was reported. Common chronic physical conditions (e.g. diabetes, asthma, arthritis, back pain, headache) were reported using a standard set of questions. Respondents were asked to rate interference with activities in the month when the condition was at its worst in the prior year, whereas the modified WHODAS-II asked about disability in the month prior to the interview. In the six Western European surveys, the SF-12 was administered to the Part 2 respondents.
Scoring the modified WHODAS-II
The WHODAS-II is scored on a 0–100 scale with higher scores indicating greater disability (although a standard scoring method has not been defined by WHO). Typically, a score of 100 indicates the maximum possible score, and intermediate scores indicate the percent of the maximum possible score. The WMH surveys followed this approach. For the Understanding and Communicating, Getting Around, Self-Care, and Getting Along With Others domains, the modified WHODAS-II was scored by estimating (on a 0–100 scale) the percent of the maximum possible score observed for the sum of the severity items, and the percent of the maximum number of days of activity limitation in the prior month. These two scores were multiplied and then divided by 100, so that the resulting score ranged from 0 to 100. For these subscales, we compared multiplicative scoring, additive scoring and scores based on Item Response methods. We found that the different scoring methods gave highly correlated results. Since they yielded nearly identical results, multiplicative scoring was employed because the European Survey coordinating group had previously decided to use this method.
For the Life Activities domain, a weighted sum of activity limitation days in the prior month was estimated. The following terms were added together: (1) The number of days totally unable to carry out normal activities in the prior month (item FD4); (2) One-half the number of days of reduced activities (item FD7); (3) One-half the number of days of reduced quality or care in work activities (item FD8); and (4) One-quarter the number of days requiring extreme effort to perform at one’s usual level (item FD9). If this sum exceeded 30, it was recoded to equal 30 so that the sum had a range from 0 to 30. The sum was then divided by 30 and multiplied by 100 so that the resulting Life Activities score also ranged from 0 to 100. A Global Disability Scale score was estimated by averaging the scores of the Understanding and Communicating, Getting Around, Self-Care, Getting Along With Others and Life Activities domain scores.
Analyses
In order to evaluate internal consistency, Chronbach’s Alpha [19] was estimated for each of the five sub-scale domains and for the Global Disability scale for all 17 surveys individually based on unweighted data. Due to the filter items, we initially assigned zero scores for persons who skipped out of the detailed questions for a particular domain. We also estimated Alpha coefficients for each subscale with the skipped items set to missing. The latter approach excludes from analysis for a domain anyone giving a negative response to the filter question for that domain. Alpha estimates with skipped item scores set to zero likely overestimate true internal consistency, whereas setting those items to missing is likely to underestimate true internal consistency.
M-Plus [20] was used to carry out a confirmatory factor analysis that assessed whether the five domain scores had high loadings on a uni-dimensional latent Global Disability variable. We report the factor loadings for the four surveys (the United States, New Zealand, the Ukraine, and Israel) with sufficient sample size to support a reliable confirmatory factor analysis. Since notable differences in WHODAS-II distributions were observed across countries, we did not believe it was appropriate to combine data from the countries with smaller samples to carry out a pooled analysis. Convergent validity of the modified WHODAS-II was examined by estimating correlations of the modified WHODAS-II Global Disability score with items from the Sheehan Disability Scale and with the SF-12 Physical and Mental Summary Component scores for the Western European surveys. For the SF-12 Physical and Mental Summary Component scores, a higher score indicates lower levels of disability, whereas for the WHODAS-II and Sheehan, a higher score indicates greater levels of disability.
RESULTS
Table 1 provides the (weighted) percent of the population for each survey that had a non-zero score for each of the five domain subscales, and for the Global Disability Scale. Persons giving a negative answer to the filter question for the first four domains would always have a zero score. The results in Table 1 show that the large majority of respondents had zero scores for each of the domains, particularly for the four domains with filter questions. Self-Care and Getting Along With Others tended to have the lowest percentages with a non-zero score, whereas the unfiltered Life Activities domain consistently had the largest percentage with a non-zero score. There was considerable cross-national variability in the percent with non-zero scores for each of the five domains.
Table 1.
Percent of the population (weighted estimate) with non-zero scores for each of the five domains and for Global Disability, World Mental Health WHODAS.
| Understanding & Communicating | Getting Around | Self care | Getting Along With Others | Life Activities | Global | Part II Sample size | Response Rate | |
|---|---|---|---|---|---|---|---|---|
| Colombia | 4.3 % | 4.6 % | 1.2 % | 1.2 % | 18.0 % | 20.7 % | 2381 | 87.7 % |
| Mexico | 4.6 % | 3.4 % | 0.7 % | 0.9 % | 11.6 % | 14.7 % | 2362 | 76.6 % |
| United States | 10.2 % | 17.0 % | 4.5 % | 5.9 % | 33.4 % | 37.9 % | 5692 | 70.9 % |
| Japan | 7.4 % | 8.0 % | 4.5 % | 3.7 % | 18.0 % | 20.4 % | 887 | 56.4 % |
| Beijing | 4.5 % | 5.0 % | 2.8 % | 3.2 % | 14.8 % | 18.2 % | 914 | 74.8 % |
| Shanghai | 5.3 % | 6.5 % | 2.1 % | 1.5 % | 13.2 % | 17.3 % | 714 | 74.6 % |
| New Zealand | 12.3 % | 15.2 % | 3.4 % | 6.4 % | 31.1 % | 36.3 % | 7435 | 73.3 % |
| Belgium | 5.0 % | 13.5 % | 2.5 % | 3.0 % | 23.5 % | 29.1 % | 1034 | 50.6 % |
| France | 7.8 % | 12.0 % | 5.1 % | 3.2 % | 24.0 % | 29.0 % | 1436 | 45.9 % |
| Germany | 4.0 % | 15.5 % | 3.1 % | 2.9 % | 15.9 % | 23.6 % | 1323 | 57.8 % |
| Italy | 3.8 % | 8.9 % | 2.3 % | 2.4 % | 15.3 % | 19.2 % | 1779 | 71.3 % |
| Netherlands | 8.1 % | 15.6 % | 1.1 % | 5.5 % | 30.1 % | 33.9 % | 1094 | 56.4 % |
| Spain | 3.1 % | 7.9 % | 1.6 % | 1.7 % | 12.9 % | 15.7 % | 2121 | 78.6 % |
| Ukraine | 7.5 % | 20.1 % | 7.8 % | 4.1 % | 41.3 % | 44.6 % | 1720 | 78.3 % |
| Lebanon | 4.6 % | 10.4 % | 2.9 % | 2.9 % | 21.3 % | 26.5 % | 1031 | 70.0 % |
| Nigeria | 2.2 % | 3.2 % | 1.5 % | 1.1 % | 17.4 % | 18.6 % | 2143 | 79.3 % |
| Israel | 12.5 % | 22.7 % | 6.8 % | 8.5 % | 40.1 % | 45.9 % | 4859 | 72.6 % |
Total sample size = 80,737
Part II sample size = 38,934
Combined response rate = 70.1 %
The internal consistency (alpha) coefficients for the Understanding and Communicating, Getting Around, Self-Care, and Getting Along With Others domains were high across all of the populations surveyed when skipped item scores were assigned a zero score (Table 2). When skipped items were set to missing, the alpha coefficients were consistently lower. When skipped items were set to missing, out of the 68 alpha coefficients estimated for the first four domains with filter questions, 46 (68%) had alpha coefficients that were 0.70 or greater. In contrast, the Life Activities domain had higher alpha coefficients, with all of the estimates exceeding 0.85.. The alpha coefficients for Global Disability score based on all five sub-scales were higher, with alphas exceeding 0.80 for all but two of 15 surveys. It was not possible to estimate Global Disability alpha coefficients for two of the 17 surveys when persons skipped out were set to missing because all respondents had at least one item set to missing (i.e. listwise deletion was employed in estimating alpha coefficients).
Table 2.
Cronbach’s alpha for the 5 domains and for Global Disability with/without imputing zero scores for persons skipped out of each domain by the filter question (except Life Activities).
| Understanding & Communicating | Getting Around | Self care | Getting Along With Others | Life Activities | Global | |
|---|---|---|---|---|---|---|
| Colombia | .91/.69 | .93/.76 | .91/.72 | .93/.84 | .94 | .91/.58 |
| Mexico | .91/.72 | .95/.82 | .93/.79 | .95/.85 | .94 | .92/.85 |
| United States | .89/.71 | .93/.70 | .91/.70 | .91/.78 | .93 | .92/.89 |
| Japan | .89/72 | .89/.58 | .88/.64 | .90/.68 | .88 | .93/.94 |
| Beijing | .79/.60 | .89/.67 | .91/.81 | .86/.74 | .94 | .90/.82 |
| Shanghai | .82/.69 | .86/.66 | .91/.81 | .86/.78 | .98 | .91/NC |
| New Zealand | .85/.64 | .92/.68 | .91/.74 | .89/.73 | .90 | .92/.86 |
| Belgium | .89/.74 | .94/.80 | .94/.72 | .91/.76 | .87 | .90/.84 |
| France | .85/.61 | .93/.70 | .89/.61 | .90/.66 | .85 | .92/.86 |
| Germany | .90/.70 | .93/.73 | .92/.67 | .91/.82 | .93 | .94/.93 |
| Italy | .92/.77 | .95/.74 | .96/.82 | .92/.76 | .91 | .93/.90 |
| Netherlands | .90/.65 | .94/.56 | .87/−.71 | .91/.69 | .89 | .91/.54 |
| Spain | .92/.77 | .96/.74 | .95/.75 | .94/.83 | .94 | .94/.88 |
| Ukraine | .89/.71 | .94/.75 | .95/.80 | .94/.75 | .87 | .93/.92 |
| Lebanon | .88/.44 | .90/.54 | .91/.56 | .91/.62 | .90 | .90/NC |
| Nigeria | .91/.74 | .93/.76 | .92/.75 | .94/.83 | .95 | .92/.94 |
| Israel | .91/.72 | .94/.77 | .95/.75 | .93/.76 | .94 | .94/.89 |
For the four surveys with adequate sample size to support reliable confirmatory factor analyses (the United States, Israel, New Zealand and the Ukraine), we assessed whether the five domain scores replicated the factor structure reported for the original WHODAS-II. The loadings of the five subscale scores on the Global Disability factor were moderate to high, but these factor loadings were consistently lower than those reported for the original instrument on the WHODAS-II website. Whether the five domains can be considered to form a unidimensional latent variable can be evaluated by goodness of fit statistics. The Confirmatory Fit Index (CFI) and the Tucker-Lewis Index (TLI) indicate good fit if their values exceed 0.95 [21]. The Root Mean Square Error of Approximation (RMSEA) suggests adequate fit if it is less than 0.08, while a value less than 0.05 indicates good fit. The fit statistics in Table 3 do not support the hypothesis that the five modified WHODAS domains form a unidimensional latent variable of Global Disability. However, since the factor loadings exceeded 0.4, the factor analyses do support the utility of a Global Disability summary variable. The fit statistics indicate that there is unique residual variance in disability explained by the individual domain subscale scores. Across the four surveys included in the confirmatory factor analyses, the Getting Around domain and the Life Activities domain tended to show the highest loadings on the Global Disability latent variable.
Table 3.
Confirmatory factor analysis assessing whether the five domains of the WMH WHODAS measure an underlying, uni-dimensional latent variable of global disability. Only surveys with a sample size of persons with non-zero scores sufficient to support confirmatory factor analysis were included.
| Factor loadings on a Global Disability latent variable assuming a one-factor solution | ||||
|---|---|---|---|---|
| US (n=5692) | Israel (n=4859) | New Zealand (n=7312) | Ukraine (n=1719) | |
| Understanding & Communicating | 0.485 | 0.508 | 0.564 | 0.458 |
| Getting Around | 0.745 | 0.824 | 0.65 | 0.814 |
| Self-Care | 0.514 | 0.711 | 0.545 | 0.739 |
| Getting Along With Others | 0.469 | 0.527 | 0.533 | 0.414 |
| Life Activities | 0.721 | 0.726 | 0.672 | 0.638 |
| Goodness of fit statistics | ||||
| CFI | 0.81 | 0.91 | 0.78 | 0.92 |
| TLI | 0.62 | 0.83 | 0.56 | 0.84 |
| RMSEA | 0.21 | 0.17 | 0.22 | 0.14 |
CFI: Confirmatory Factor Index
TLI: Tucker-Lewis Index
RMSEA: Root Mean Square Error Approximation
The correlation coefficients for the modified WHODAS-II Global Disability Scale with Sheehan Disability Scale items provide modest support for its validity (Table 4). Among respondents with chronic physical conditions, modified WHODAS-II Global Disability showed positive correlations with Sheehan Disability ratings (a mean of 0.30 for home management, 0.31 for work, 0.23 for close relationships, and 0.28 for social life). The correlation of the Global Disability Scale with the SF-12 Mental Summary Component scores was similar (average r=−0.30). In contrast, the correlations for the association of the modified WHODAS-II Global Disability Scale with the SF-12 Physical Summary Component scores was substantially higher (average r=−0.56). The negative correlations are due to the reverse scoring of the SF-12 (i.e. higher scores indicate lower disability).
Table 4.
Correlations of the modified Global WHODAS-II score with items from the Sheehan Disability Scale and from the SF-12 Physical and Mental Copmponent Summary Scores among Part 2 respondents reporting one or more chronic conditions.
| Pearson Correlations | ||||||
|---|---|---|---|---|---|---|
| Sheehan-Home Management | Sheehan – Work | Sheehan – Close Relationships | Sheehan – Social Life | SF-12 Physical | SF-12 Mental | |
| Colombia | 0.17 | 0.16 | 0.09 | 0.12 | NA | NA |
| Mexico | 0.20 | 0.23 | 0.21 | 0.22 | NA | NA |
| US | 0.37 | 0.40 | 0.31 | 0.38 | NA | NA |
| Japan | 0.34 | 0.31 | 0.18 | 0.36 | NA | NA |
| Beijing | 0.27 | 0.33 | 0.24 | 0.24 | NA | NA |
| Shanghai | 0.14 | 0.26 | 0.10 | 0.07 | NA | NA |
| New Zealand | 0.33 | 0.28 | 0.24 | 0.26 | NA | NA |
| Belgium | 0.30 | 0.27 | 0.11 | 0.24 | −0.56 | −0.32 |
| France | 0.30 | 0.30 | 0.14 | 0.31 | −0.51 | −0.28 |
| Germany | 0.42 | 0.42 | 0.32 | 0.38 | −0.59 | −0.18 |
| Italy | 0.34 | 0.22 | 0.33 | 0.36 | −0.51 | −0.33 |
| Netherlands | 0.29 | 0.35 | 0.19 | 0.22 | −0.62 | −0.33 |
| Spain | 0.36 | 0.41 | 0.28 | 0.28 | −0.58 | −0.37 |
| Ukraine | 0.31 | 0.30 | 0.20 | 0.32 | NA | NA |
| Lebanon | 0.29 | 0.29 | 0.30 | 0.25 | NA | NA |
| Nigeria | 0.31 | 0.29 | 0.28 | 0.32 | NA | NA |
| Israel | 0.39 | 0.40 | 0.40 | 0.41 | NA | NA |
| Mean correlation | 0.30 | 0.31 | 0.23 | 0.28 | −0.56 | −0.30 |
Note: The Sheehan Disability Scale items asked about disability related to a particular chronic condition at its worst in the prior year.
NA: Not available. The SF-12 was administered only in the Western European surveys.
Table 5 provides descriptive statistics for the modified WHODAS-II Global Disability Score for each of the surveys based on weighted data, including population means, standard deviations, the percent with zero scores, and the disability score at the 90th percentile of the population. As might be expected, there was considerable variation in distributional characteristics across the surveys. Since there were marked differences in age distribution (not accounted for in Table 5), some of the differences in modified WHODAS-II score distributions may be attributable to age variation.
Table 5.
Population normative data (weighted estimates) for the five domain Global Disability Scale
| Mean | SD | % > 0 | 90th Percentile Score | |
|---|---|---|---|---|
| Colombia | 1.17 | 4.22 | 20.7 % | 3.33 |
| Mexico | 0.87 | 4.29 | 14.7 % | 0.67 |
| United States | 3.77 | 9.00 | 37.9 % | 14.33 |
| Japan | 1.09 | 4.78 | 20.4 % | 2.67 |
| Beijing | 1.03 | 4.15 | 18.2 % | 1.83 |
| Shanghai | 0.90 | 4.25 | 17.3 % | 1.19 |
| New Zealand | 2.87 | 7.60 | 36.3 % | 10.00 |
| Belgium | 3.29 | 8.46 | 29.1 % | 12.33 |
| France | 3.38 | 8.60 | 29.0 % | 10.00 |
| Germany | 2.63 | 7.96 | 23.6 % | 8.33 |
| Italy | 2.09 | 6.91 | 19.2 % | 7.00 |
| Netherlands | 4.28 | 8.94 | 33.9 % | 18.33 |
| Spain | 2.25 | 7.68 | 15.7 % | 5.28 |
| Ukraine | 4.81 | 9.73 | 44.6 % | 16.33 |
| Lebanon | 1.86 | 5.78 | 26.5 % | 4.28 |
| Nigeria | 1.13 | 4.09 | 18.6 % | 2.33 |
| Israel | 5.52 | 12.03 | 45.9 % | 20.00 |
DISCUSSION
The modifications to the WHODAS-II implemented in the World Mental Health surveys produced a shorter test thereby reducing respondent burden. Reduced respondent burden appeared to have come at some cost in terms of reduced internal consistency along with a large proportion of respondents having zero scores. Validity coefficients were also not as high as indicated by prior research, except for the SF-12 Physical Component Summary score which showed a high correlation with the modified WHODAS-II Global Disability Scale. The filter questions employed to increase the efficiency of administration may have inadvertently reduced the sensitivity of the modified WHODAS-II to measurement of disability in the mild to moderate range, particularly for persons experiencing disability in social relationships.
Reliability
This population survey was able to assess internal consistency, not test-retest or inter-rater reliability. For the four domains with filter questions, when zero scores were assigned for those who did not endorse the filter question, internal consistency was high. However, there is circumstantial evidence that assigning zero scores may over-estimate internal consistency due to imperfect performance of the filter questions. The proportion with non-zero scores on the Life Activities domain was consistently higher than the proportion with non-zero scores on the filtered domains. When we exclude individuals who did not endorse the filter question – rather than assigning zeros to these individuals –the median alpha estimate for the four domains with filter questions was in the 0.70’s, reflecting moderate internal consistency. However, setting all items skipped to missing excludes all persons with low disability levels, so these estimates of internal consistency underestimate true internal consistency. The internal consistency estimates for the Life Activities domain were considerably higher.
Validity
The correlations of modified WHODAS-II scale scores with Sheehan Disability scores and with the SF-12 Mental Component Summary score were lower than those reported in prior comparisons in clinical samples of the WHODAS-II with other disability measures [5,7]. In contrast, the correlations with the SF-12 Physical Component Summary score were comparable to those reported in prior validity studies. The prevalence of disability is likely to be lower in a general population sample than in a clinical sample, which can reduce item intercorrelation. The Sheehan ratings asked about disability related to particular chronic conditions at their worst in the past year, whereas the modified WHODAS-II ratings concerned disability in the prior month. For these reasons, the lower validity coefficient with the Sheehan items employed in this study may reflect, in part, methodological differences. However, it is likely that the modifications to the WHODAS-II introduced for the WMH surveys were an important factor affecting reliability and validity estimates. The higher correlation of the modified WHODAS-II Global Disability score with SF-12 Physical Component Summary Score may reflect the greater contribution of disability in self-care and mobility (more severe forms of disability) to the association of these two measures.
Analysis and interpretation
Table 5 provides descriptive information on the distributional characteristics, but does not evaluate or explain differences in observed scores across surveys. Given large differences in mean disability scores and the percent with non-zero scores across countries, cross-national methods differences likely influenced WHODAS-II scores. The analyses reported here cannot assess or explain cross-national differences. The use of filter questions complicates cross-national comparison of item performance, as scores on filtered items reflect both the performance of the item and the performance of the filter. The need for caution in making cross-national comparisons of disability from self-report measures is not unique to the modified WHODAS-II [22].
Modified WHODAS-II scores had highly skewed distributions across populations, with low mean scores and large proportions of the populations with zero scores. Given these distributional properties, and the variability in modified WHODAS-II scores across countries, for some purposes we recommend dichotomizing the modified WHODAS-II Global Disability Score at the 90th percentile for each country, analyzing disability as a binary variable (i.e. significant disability present or absent). Given the large proportion of the study populations with zero scores, the use of a mean disability score is difficult to interpret as it reflects a combination of the percent with any disability and the mean level of disability among those with non-zero scores. In analyses restricted to persons with a higher prevalence of non-zero scores, the use of a continuous Global Disability Score could be useful. Dichotomizing WHODAS-II Global Disability scores at the 90th percentile for each survey masks cross-national differences in the prevalence of elevated WHODAS-II scores, since by definition 10% of each survey will be defined as having disability. Modified WHODAS-II scores may also be used without standardization to country or survey-specific norms when pooling data across countries. In that case, introducing indicator variables for country to adjust for country-specific measurement differences is recommended, with cautious interpretation of results.
Modifications to the WHODAS-II
The filter questions used for the first four domains may have been too stringent, particularly for the Getting Along With Others domain. This limitation may reduce the ability of the modified WHODAS-II to differentiate among persons with less severe functional disability. An alternative strategy to reduce respondent burden is to use a sub-set of items selected from each of the WHODAS-II domains. For example, the original WHODAS-II has a 12 item brief version. By avoiding filter questions, the frequency of zero scores is likely to be reduced. In cross-national surveys, where the performance of filter questions may differ across countries in ways that are difficult to predict or explain, the use of a standard set of questions administered to most or all respondents could reduce methods variance and increase opportunities for methods research comparing item performance across countries. When feasible, the use of computerized adaptive tests to avoid floor effects induced by filter questions is an alternative approach to address this problem.
Global Disability latent variable
The domain subscale scores showed moderate to high loadings on a Global Disability latent variable, while the domains retained unique variance not explained by Global Disability. However, the confirmatory factor analysis did not replicate earlier factor analysis results reported by the WHO in which the sub-scale scores had very high loadings on a Global Disability latent variable. This is not necessarily a problem, as it suggests that the domain scores may be useful in understanding differences in disability profile. The purpose of the confirmatory factor analysis was not to assess dimensionality, but rather to evaluate how closely the modified WHODAS-II replicated the factor structure shown on the WHO website. In analyses applying a bi-factor model to U.S. survey data (not presented here), we found that the modified WHODAS-II was sufficiently uni-dimensional to use the total score as a uniform construct.
Conclusions
The modified WHODAS-II used in the World Mental Health Surveys was found to have acceptable levels of internal consistency and validity, albeit lower than observed for the original WHODAS-II in clinical samples. The use of filter items appeared to be a significant factor affecting the internal consistency and validity of the modified WHODAS-II. While the modified WHODAS-II can be used to compare group differences in disability prevalence within countries, we recommend caution in interpretation of cross-national differences in disability levels.
What is new?
Disability items loaded on a Global Disability factor, in line with prior research.
Use of filter items for subscales impaired measurement properties.
Cross-national differences in disability scores were observed, but were not readily explainable.
In future cross-national studies where questionnaire length is an issue, use of a short-form disability scale with all items administered should be preferred to use of sub-scale filter items.
Acknowledgments
The surveys included in this report were carried out in conjunction with the World Health Organization World Mental Health (WMH) Survey Initiative. We thank the WMH staff for assistance with instrumentation, fieldwork, and data analysis. These activities were supported by the United States National Institute of Mental Health (R01MH070884), the John D. and Catherine T. MacArthur Foundation, the Pfizer Foundation, the US Public Health Service (R13-MH066849, R01-MH069864, and R01 DA016558), the Fogarty International Center (FIRCA R01-TW006481), the Pan American Health Organization, Eli Lilly and Company, Ortho-McNeil Pharmaceutical, Inc., GlaxoSmithKline, and Bristol-Myers Squibb. A complete list of WMH publications can be found at http://www.hcp.med.harvard.edu/wmh/. The Mexican National Comorbidity Survey (MNCS) is supported by The National Institute of Psychiatry Ramon de la Fuente (INPRFMDIES 4280) and by the National Council on Science and Technology (CONACyT-G30544-H), with supplemental support from the PanAmerican Health Organization (PAHO). The Lebanese survey is supported by the Lebanese Ministry of Public Health, the WHO (Lebanon) and unrestricted grants from Janssen Cilag, Eli Lilly, GlaxoSmithKline, Roche, Novartis and anonymous donations. The ESEMeD project was funded by the European Commission (Contracts QLG5-1999-01042; SANCO 2004123), the Piedmont Region (Italy), Fondo de Investigación Sanitaria, Instituto de Salud Carlos III, Spain (FIS 00/0028), Ministerio de Ciencia y Tecnología, Spain (SAF 2000-158-CE), Departament de Salut, Generalitat de Catalunya, Spain, and other local agencies and by an unrestricted educational grant from GlaxoSmithKline. The Chinese World Mental Health Survey Initiative is supported by the Pfizer Foundation. The Colombian National Study of Mental Health (NSMH) is supported by the Ministry of Social Protection, with supplemental support from the Saldarriaga Concha Foundation. The Israel National Health Survey is funded by the Ministry of Health with support from the Israel National Institute for Health Policy and Health Services Research and the National Insurance Institute of Israel. The World Mental Health Japan (WMHJ) Survey is supported by the Grant for Research on Psychiatric and Neurological Diseases and Mental Health (H13-SHOGAI-023, H14-TOKUBETSU-026, H16-KOKORO-013) from the Japan Ministry of Health, Labour and Welfare. The New Zealand Mental Health Survey (NZMHS) is supported by the New Zealand Ministry of Health, Alcohol Advisory Council, and the Health Research Council. The Nigerian Survey of Mental Health and Wellbeing (NSMHW) is supported by the WHO (Geneva), the WHO (Nigeria), and the Federal Ministry of Health, Abuja, Nigeria. The South Africa Stress and Health Study (SASH) is supported by the US National Institute of Mental Health (R01-MH059575) and National Institute of Drug Abuse with supplemental funding from the South African Department of Health and the University of Michigan. The Ukraine Comorbid Mental Disorders during Periods of Social Disruption (CMDPSD) study is funded by the US National Institute of Mental Health (RO1-MH61905). The US National Comorbidity Survey Replication (NCS-R) is supported by the National Institute of Mental Health (NIMH; U01-MH60220) with supplemental support from the National Institute of Drug Abuse (NIDA), the Substance Abuse and Mental Health Services Administration (SAMHSA), the Robert Wood Johnson Foundation (RWJF; Grant 044708), and the John W. Alden Trust.
Understanding and Communicating (Domain 1)
*FD10. Was there ever a time in the past 30 days when health-related problems caused difficulties with either (READ SLOWLY) your concentration, memory, understanding, or ability to think clearly?
| YES...........................1 | |
| NO............................5 | GO TO *FD12 |
| DON’T KNOW........8 | GO TO *FD12 |
| REFUSED.................9 | GO TO *FD12 |
*FD10a. How many days did you have these difficulties during the past 30 days?
| ________ DAYS |
| DON’T KNOW....................998 |
| REFUSED............................999 |
*FD11. (RB, PG26) (Look at page 26 in your booklet.) During (that/those NUMBER FROM *FD10a) day(s), how much difficulty did you have in each of the following areas:
| (IF NEC: None, mild, moderate, or severe difficulty?) | NONE (1) |
MILD (2) |
MOD (3) |
SEV (4) |
(IF VOL) CANNOT DO (5) |
DK (8) |
RF (9) |
|---|---|---|---|---|---|---|---|
| *FD11a. Concentrating on doing something for ten minutes – none, mild, moderate, or severe difficulty? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD11b. Understanding what was going on around you? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD11c. Remembering to do important things? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD11d. Learning a new task – for example, learning how to get to a new place? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
Getting Around (Domain 2)
*FD12. Was there ever a time in the past 30 days when health-related problems caused you difficulties with mobility, such as standing for long periods, moving around inside your home, or getting out of your home?
| YES...................................................1 | |
| NO....................................................5 | GO TO *FD14 |
| DON’T KNOW................................8 | GO TO *FD14 |
| REFUSED.........................................9 | GO TO *FD14 |
*FD12a. How many days did you have these difficulties during the past 30 days?
| ________ DAYS |
| DON’T KNOW....................998 |
| REFUSED............................999 |
*FD13. (RB, PG26) (Look at page 26 in your booklet.) During (that/those NUMBER FROM *FD12a) day(s), how much difficulty did you have in each of the following areas:
| (IF NEC: none, mild, moderate, or severe difficulty?) | NONE (1) |
MILD (2) |
MOD (3) |
SEV (4) |
(IF VOL) CANNOT DO (5) |
DK (8) |
RF (9) |
|---|---|---|---|---|---|---|---|
| *FD13a. Standing for long periods, such as 30 minutes? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD13b. Moving around inside your home? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD13c. Walking a long distance such as (a kilometer/half a mile)? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
Self-Care (Domain 3)
*FD14. Was there ever a time in the past 30 days when health-related problems caused you difficulties with self care, such as washing your whole body, getting dressed, or feeding yourself?
| YES..................................................1 | |
| NO...................................................5 | GO TO *FD16 |
| DON’T KNOW...............................8 | GO TO *FD16 |
| REFUSED........................................9 | GO TO *FD16 |
*FD14a. How many days did you have these difficulties during the past 30 days?
| ________ DAYS |
| DON’T KNOW..................998 |
| REFUSED...........................999 |
*FD15. (RB, PG26) (Look at page 26 in your booklet.) During (that/those NUMBER FROM *FD14a) day(s), how much difficulty did you have in each of the following areas:
| (IF NEC: none, mild, moderate, or severe difficulty?) | NONE (1) |
MILD (2) |
MOD (3) |
SEV (4) |
(IF VOL) CANNOT DO (5) |
DK (8) |
RF (9) |
|---|---|---|---|---|---|---|---|
| *FD15a. Washing your whole body? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD15b. Getting dressed? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD15c. Staying by yourself for a few days? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
Getting Along With Others (Domain 4)
*FD16. Was there ever a time in the past 30 days when health-related problems caused you difficulties either getting along with people, maintaining a normal social life, or participating in social activities?
| YES....................................................1 | |
| NO.....................................................5 | GO TO *FD18 |
| DON’T KNOW.................................8 | GO TO *FD18 |
| REFUSED..........................................9 | GO TO *FD18 |
*FD16a. How many days did you have these difficulties during the past 30 days?
| ________ DAYS |
| DON’T KNOW..................998 |
| REFUSED...........................999 |
*FD17. (RB, PG26) (Look at page 26 in your booklet.) During (that/those NUMBER FROM *FD16a) day(s), how much difficulty did you have in each of the following areas:
| (IF NEC: none, mild, moderate, or severe difficulty?) | NONE (1) |
MILD (2) |
MOD (3) |
SEV (4) |
(IF VOL) CANNOT DO (5) |
DK (8) |
RF (9) |
|---|---|---|---|---|---|---|---|
| *FD17a. Starting and maintaining a conversation? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD17b. Dealing with people you did not know well? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD17c. Maintaining friendships? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD17d. Making new friends? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
| *FD17e. Controlling your emotions when you were around people? | 1 | 2 | 3 | 4 | 5 | 8 | 9 |
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.World Health Organization. International classification of functioning, disability and health (ICF) Geneva, Switzerland: World Health Organization; 2001. [Google Scholar]
- 2.Bickenbach JE, Chatterji S, Badley EM, Ustun TB. Models of disablement, universalism, and the international classification of impairments, disabilities and handicaps. Soc Sci Med. 1999;48:1173–1187. doi: 10.1016/s0277-9536(98)00441-9. [DOI] [PubMed] [Google Scholar]
- 3.See WHO WHODAS II. website: http://www.who.int/icidh/whodas/index.html.
- 4.Vazquez-Barquero JL, Vazquez Bourgon E, Herrera Castanedo S, et al. Spanish version of the new World Health Organization Disability Assessment Schedule II (WHO DAS II): initial phase of development and pilot study. Actas Esp Psiquiatr. 2000;28:77–87. [PubMed] [Google Scholar]
- 5.Chwastiak LA, Von Korff M. Disability in depression and back pain: Evaluation of the World Health Organization Disability Assessment Schedule (WHO DAS II) in a primary care setting. J Clin Epidemiol. 2003 Jun;56(6):507–14. doi: 10.1016/s0895-4356(03)00051-9. [DOI] [PubMed] [Google Scholar]
- 6.Perini SJ, Slade T, Andrews G. Generic effectiveness measures: sensitivity to symptom change in anxiety disorders. J Affect Disord. 2006 Feb;90:123–30. doi: 10.1016/j.jad.2005.10.011. [DOI] [PubMed] [Google Scholar]
- 7.Chisolm TH, Abrams HB, McArdle R, Wilson RH, Doyle PJ. The WHODAS II: psychometric properties in the measurement of functional health status in adults with acquired hearing loss. Trends Amplif. 2005;9(3):111–26. doi: 10.1177/108471380500900303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Annicchiarico R, Gibert K, Cortes U, Campana F, Caltagirone C. Qualitative profiles of disability. J Rehabil Res Dev. 2004 Nov-Dec;41(6):835–46. doi: 10.1682/jrrd.2004.02.0016. [DOI] [PubMed] [Google Scholar]
- 9.Kim JM, Stewart R, Glozier N, Prince M, Kim SW, Yang SJ, Shin IS, Yoon JS. Physical health, depression and cognitive function as correlates of disability in an older Korean population. Int J Geriatr Psychiatry. 2005;20:160–7. doi: 10.1002/gps.1266. [DOI] [PubMed] [Google Scholar]
- 10.Chopra PK, Couper JW, Herrman H. The assessment of patients with long-term psychotic disorders: application of the WHO Disability Assessment Schedule II. Aust N Z J Psychiatry. 2004 Sep;38(9):753–9. doi: 10.1080/j.1440-1614.2004.01448.x. [DOI] [PubMed] [Google Scholar]
- 11.Gallagher P, Mulvany F. Levels of ability and functioning: using the WHODAS II in an Irish context. Disabil Rehabil. 2004 May 6;26(9):506–17. doi: 10.1080/0963828042000202257. [DOI] [PubMed] [Google Scholar]
- 12.McKibbin C, Patterson TL, Jeste DV. Assessing disability in older patients with schizophrenia: results from the WHODAS-II. J Nerv Ment Dis. 2004 Jun;192(6):405–13. doi: 10.1097/01.nmd.0000130133.32276.83. [DOI] [PubMed] [Google Scholar]
- 13.van Tubergen A, Landewe R, Heuft-Dorenbosch L, Spoorenberg A, van der Heijde D, van der Tempel H, van der Linden S. Assessment of disability with the World Health Organisation Disability Assessment Schedule II in patients with ankylosing spondylitis. Ann Rheum Dis. 2003;62:140–5. doi: 10.1136/ard.62.2.140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Demyttenaere K, Bruffaerts R, Posada-Villa J, Gasquet I, Kovess V, Lepine JP, Angermeyer MC, Bernert S, de Girolamo G, Morosini P, Polidori G, Kikkawa T, Kawakami N, Ono Y, Takeshima T, Uda H, Karam EG, Fayyad JA, Karam AN, Mneimneh ZN, Medina-Mora ME, Borges G, Lara C, de Graaf R, Ormel J, Gureje O, Shen Y, Huang Y, Zhang M, Alonso J, Haro JM, Vilagut G, Bromet EJ, Gluzman S, Webb C, Kessler RC, Merikangas KR, Anthony JC, Von Korff MR, Wang PS, Brugha TS, Aguilar-Gaxiola S, Lee S, Heeringa S, Pennell BE, Zaslavsky AM, Ustun TB, Chatterji S WHO World Mental Health Survey Consortium. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA. 2004;291:2581–90. doi: 10.1001/jama.291.21.2581. [DOI] [PubMed] [Google Scholar]
- 15.Kessler RC, Ames M, Hymel PA, Loeppke R, McKenas DK, Richling DE, Stang PE, Ustun TB. Using the World Health Organization Health and Work Performance Questionnaire (HPQ) to evaluate the indirect workplace costs of illness. J Occup Environ Med. 2004;46(Suppl):S23–37. doi: 10.1097/01.jom.0000126683.75201.c5. [DOI] [PubMed] [Google Scholar]
- 16.Kessler RC, Berglund P, Chiu WT, Demler O, Heeringa S, Hiripi E, et al. The US National Comorbidity Survey Replication (NCS-R): design and field procedures. Int J Methods Psychiatr Res. 2004a;13:69–92. doi: 10.1002/mpr.167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Leon AC, Olfson M, Portera L, Farber L, Sheehan DV. Assessing psychiatric impairment in primary care with the Sheehan Disability Scale. Int J Psychiatry Med. 1997;27(2):93–105. doi: 10.2190/T8EM-C8YH-373N-1UWD. [DOI] [PubMed] [Google Scholar]
- 18.Ware J, Kosinkski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Medical Care. 1996;34:220–233. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
- 19.Nunnally JC. Psychometric theory. McGraw Hill; New York: 1967. pp. 210–211. [Google Scholar]
- 20.Muthen LK, Muthen BO. Mplus: Statistical Analaysis with Latent Variables, User’s Guide. 2. Muthen and Muthen; Los Angeles, CA: [Google Scholar]
- 21.Hu LT, Bentler PM, Kano Y. Can fit statistics in covariance structure analysis be trusted. Psychological Bulletin. 1992;112:351–362. doi: 10.1037/0033-2909.112.2.351. [DOI] [PubMed] [Google Scholar]
- 22.Crane PK. Commentary on comparing translations of the EORTC QLQ-C30 using differential item functioning analyses. Qual Life Res. 2006;15:1117–1118. doi: 10.1007/s11136-006-0040-x. [DOI] [PubMed] [Google Scholar]