Abstract
This paper describes the development of a standard survey instrument designed to identify children with functional difficulties through censuses and surveys. Using the World Health Organization’s (WHO) International Classification of Functioning, Disability, and Health (ICF) as a conceptual framework the Washington Group on Disability Statistics (WG) and UNICEF, in collaboration with experts, developed a survey module on child functioning for 2–4 and 5–17 year olds, which was piloted in different countries. To-date, the module has been used in four population surveys, including the Children and Women National Survey in Mexico (ENIM 2015). The availability of a standard method for identifying children with disabilities may help to orient and develop better practices aimed at improving the life conditions and well-being of children.
This paper is based on the working documents of the UNICEF/WG Child Functioning workgroup and work that has been previously published in International Measurement of Disability: Purpose, Method and Application – The work of the Washington Group on Disability Statistics. B. Altman (Ed). (Crialesi et al, 2016).
Background
The Convention on the Rights of the Child, adopted in 1989 (UNICEF, 1989), included the first explicit provision relating to the rights of children with disabilities. It included a prohibition against discrimination on the grounds of disability (art. 2), and obligations to provide services for children with disabilities, in order to enable them to achieve the fullest possible social integration (art. 23).
The more recent Convention on the Rights of Persons with Disabilities (UN, 2006), adopted in 2006, further strengthened the rights of children with disabilities with a dedicated article on children (art. 7). This article outlined the obligation on States to ensure the realization of all rights for children with disabilities on an equal basis with other children, to promote their best interests, and to ensure their right to be heard and taken seriously. It incorporates, within its general principles (art. 3), respect for the evolving capacities of children with disabilities and their right to preserve their identities, and introduces a general obligation (art. 4) to consult with children, through their representative organizations, when developing relevant legislation and policies.
These Conventions focus on the challenges faced by children with disabilities and call for improvements in their access to services, and in their participation in all aspects of life. In order to achieve these goals, there is a need for improved data collection internationally. The current lack of accurate data impedes the development, implementation and evaluation of policies and programmes that would improve the lives of children with disabilities.
The UN Convention on the Rights of Persons with Disabilities encourages States to collect appropriate information, including statistical and research data, to enable them to formulate and implement policies to give effect to the Convention (art. 31).
The limitations of currently available data and the importance of improving statistical information on disability in order to develop internationally comparable indicators for policy purposes has also been stressed by the UN General Assembly 2011 - special section on “Status of the Convention on Rights of the Child” and in the World Report on Disability 2011 (WHO, 2011).
The Washington Group on Disability Statistics (WG) is a United Nations (UN) sponsored City Group commissioned in 2001 to improve the quality and international comparability of disability measures. The main purpose of the WG is the promotion and co-ordination of international co-operation in the area of health statistics focusing on disability measures suitable for censuses and national surveys. The major objective is to provide basic necessary information on disability which is comparable throughout the world. To that end the WG has developed a short set of disability measures, suitable for use in censuses, sample-based national surveys, or other statistical formats, for the primary purpose of informing policy on equalization of opportunities for people with disabilities. The short set includes six core functional domains: seeing, hearing, walking, cognition, self-care and communication. The WG has also developed an extended set of survey items on functioning to be used as components of population surveys or as supplements to specialty surveys. This extended set of survey items expands on the short set of disability measures, including additional domains of functioning (upper body functioning, affect, pain, and fatigue) and more information per domain (for example, functioning with and without assistance).
The WG chose to use the World Health Organization’s (WHO) International Classification of Functioning, Disability, and Health (ICF) (WHO, 2001) as the conceptual framework for the development of question modules. Furthermore, the WG selected questions that reflect a functional approach to operationalizing disability. The functional approach was selected for a variety of reasons that are only summarized in this paper but can be found in detail in Madans et al (2004). In short, the WG approach to disability measurement identifies six basic, universal activity or functioning domains (seeing, hearing, walking, remembering and concentrating, communicating and self-care) where respondents may experience difficulty functioning in their environments.
The WG short set was adopted in 2006 and the extended set on functioning was adopted in 2010. Both the short and extended sets of questions on functioning have undergone several rounds of cognitive and field testing in several countries, with training and technical assistance provided by the WG. Detailed information on the testing is available in Miller et al (2011).
The short set of questions developed by the WG was initially intended for use in population censuses, and though certain questions may be suitable for some child/youth sub-populations, the set was not developed with this particular group in mind. The WG acknowledged that the use of this survey module on child and youth populations would underestimate the prevalence of disability in those age cohorts by omitting several functional domains that are important in child development. The WG recognized the need for a set of questions on functioning and disability that would produce internationally comparable data for children and youth. At the 10th WG meeting (2009) a sub-group was established that would focus on the development of a set of questions intended to measure child (and youth) disability in surveys.
Over the years UNICEF has supported the collection of data on this topic through the Multiple Indicator Cluster Survey (MICS). MICS is a household survey programme developed by UNICEF to assist countries in filling data gaps for monitoring the situation of children and women through statistically sound, internationally comparable estimates of socioeconomic and health indicators. Since the initiation of the MICS in 1995, more than 240 surveys have been implemented in more than 100 low- and middle-income countries (defined based on per capita gross national income)2. Data on child disability were first collected during the second round of MICS in 2000. Since then, more than 50 surveys have gathered information on disability, making MICS the largest source of internationally comparable data on children with disabilities for low- and middle-income countries. In 2011 UNICEF decided to revise the disability module used in the MICS surveys and collaborate with the WG for the development of a joint module.
Guiding Principles for the Development of a Survey Module on Child Functioning
The workgroup on Child Functioning, composed mainly of representatives from National Statistical Offices from both developed and developing countries, initially agreed to certain principles that have guided the production of the questions set. The UNICEF/WG Module on Child Functioning was then developed by considering the following aspects / criteria:
The primary purpose of the questions is to identify children with functional difficulties that may place children at risk of experiencing limited participation in an unaccommodating environment.
The questions will provide cross-nationally comparable data. The module is designed to identify children with similar types of functional difficulties in basic, universal activities, regardless of nationality or culture; and the questions reflect basic functional actions that are applicable to children in different countries and life situations.
Disability is understood as a complex process that “denotes the negative aspects of the interaction between an individual (with a health condition) and that individual’s contextual (environmental and personal) factors” (WHO, 2001).
The International Classification of Functioning, Disability and Health for Children and Youth (ICF-CY), (WHO, 2007) is chosen as the conceptual framework and for the selection of relevant domains for the development of current, relevant, and sustainable questions on child functioning.
The development of the child functioning questions builds upon the work of the WG short set and extended set of questions for adults. Findings from several studies and national and international surveys are also taken into account.
Consultation with experts, including survey statisticians, paediatricians, developmental psychologists, speech therapists, etc. will be sought to support the work of the WG/UNICEF collaboration.
The population reference age for the child functioning questions is 0–17 years, as per ICF-CY recommendations. However, capturing disabilities among children under 2 years of age through population surveys is challenging. Due to the transitional nature of the development process for young children, a developmental delay at this age is not necessarily indicative of functional limitations. Therefore, trying to assess difficulties in functioning may yield misleading results for this age group.
Questions on child functioning should be asked of parents or primary caregivers. Although the use of proxy may introduce some bias as children and parents may have different perceptions of the child’s ability, parents/primary caregivers can actually facilitate the assessment of children over a wide age range. Indeed questions for measuring disability addressed directly to children are rare and information provided by children usually supplements information provided by parents or caregivers.
For reference, and to focus the respondent on the functioning of their own child in comparison to that child’s cohort, where appropriate, questions are prefaced with the statement: “Compared with children of the same age…”.
Disability is not a yes/no dichotomy, but can be conceptualized on a continuum from minor difficulties in functioning to severe difficulties that may have a major impact on a person’s life. Therefore graded answer categories are designed to reflect this continuum. With a few exceptions, these are: no difficulty, some difficulty, a lot of difficulty or cannot do at all.
The set of proposed questions will be validated through cognitive and field testing, following established WG procedures.
The range and the types of functional difficulties – as an expression of disability – are not the same for children and adults. While adults (especially in advancing years) have more difficulty in mobility, sensory, and personal care domains; children more often have difficulties related to intellectual functioning, affect and behaviour.
The set of child functioning questions is designed for use as a component of national population surveys or as a supplement to surveys on specific topics; for example, a health or education survey.
Milestones in the development of the UNICEF/WG Module on Child Functioning
1. Selection of appropriate and feasible domains of functioning and question design.
Questions were developed according to a range of domains identified through the ICF-CY and survey questionnaire already in use in several countries. Based on this assessment, an initial set of domains was selected: seeing, hearing, mobility, communication/comprehension, learning, relationships, and playing. These were the most common domains found in existing survey questionnaire, but because child disability comprises a wide range of domains, reflecting child development, additional domains of functioning were developed and included: emotions, behaviour, focusing attention/concentrating and coping with change.
It was important that question design avoided a medical approach (that focuses on impairments or conditions), and rather operationalized the biopsychosocial model. Questions focused on a particular aspect of the ICF (difficulties doing basic, universal activities) that would identify children at risk of restricted participation in a non-accommodating environment. The workgroup felt it would be beneficial to use, when appropriate, the questions already tested and adopted by the WG; and to conform to established WG question/response design in order to both harmonize the child functioning questions with existing WG products and to capture the continuum of difficulty.
2. Development of age cohorts and proxy respondents
While, recognizing the importance of early detection of children with functional difficulties, it is extremely challenging to capture children under 2 years of age through surveys designed for research purposes. Among infants and children in this age range, the development process is very subjective and culturally influenced, and a developmental delay is not necessarily a sign of functional limitation. The inclusion of children under 2 years of age may lead to large proportions of false positive cases due to the nature of the development process for children of this age. Therefore it was agreed that the population age reference for the set of child functioning questions is 2–17 years.
Furthermore, questions were developed in a way that would be appropriate for two specific age cohorts: pre-schoolers age 2–4 years and school-aged children 5–17 years.
As mentioned earlier, it was determined that questions would be directed to a proxy respondent, the child’s mother or primary caregiver. In order to standardize the expectations of the proxy-respondents to focus their attention on the functioning of their own child in relation to the child’s age, the questions, where appropriate, are prefaced with the clause: “Compared with children of the same age…”.
3. Validating the module
Question design is an iterative process of drafting, test, revision, retest etc. Question evaluation through cognitive testing was used to evaluate the cross-cultural equivalence of the Module, verifying that the questions were understood according to their intent, and applicable to the widest range of respondent’s life contexts. Following previously established WG validation procedures, the Child Functioning Module underwent initial cognitive testing in 2012 and 2013 in India, Belize, Oman, Montenegro, and the USA (Massey et al, 2014).
The results of cognitive testing were presented in Amman, Jordan at the 13th WG meeting in October 2013, and a revised version of the Child Functioning Module was prepared based on these findings as well as on comments from the participants of the 13th WG meeting. The revised version of the module was cognitively tested in January 2014 in the USA, and findings were presented at the 14th WG meeting in Buenos Aires, Argentina in October 2014.
A separate cognitive test was conducted in the USA in 2015, specifically to evaluate the differences in how the same questions are interpreted/answered by the teens and by their parent-proxies. The results showed that while teens often have greater insight into the specifics of their difficulties, there were high levels of agreement indicating that parents provide accurate information on their teens’ difficulties and in some domains the parent’s understanding of the meaning of the question was more in line with the intent (Massey et al, 2015).
Field testing on the module began in Samoa in July 2014 by the Samoa Bureau of Statistics.3 The Child Functioning module was included in the Samoa Demographic and Health Survey 2014. Preliminary findings, presented at the 15th WG meeting in Copenhagen, Denmark highlighted the need for further adjustments to the questions. The revised module was cognitively tested for the third time in the USA in August 2014, with revisions finalized in preparation for further field testing.
The Directorate of Statistics and Censuses in El Salvador field tested the module in October 2015, as part of a National Survey on Disability, and the Statistical Office of the Republic of Serbia field tested it in February 2016. Findings from the two field tests resulted in further, minor, changes being made to some questions, with the revised version of the module undergoing further cognitive testing in the USA in February 2016, India in March 2016, and Jamaica in April 2016.
4. Establishment of analytic properties of the module
Disability may be a complex ‘process’ involving body functions and structure, activity limitations and participation restrictions, personal characteristics – and not least aspects of environmental barriers and facilitators. The determination of disability status (based on difficulties doing basic activities), on the other hand, need not be complex. The graded responses to the questions in the UNICEF/WG Child Functioning Module allow for the generation of several thresholds or cut-offs for determining disability in a population. It is possible to determine the proportion of those who have mild difficulties (at least some difficulty on one or more domain of functioning), or moderate levels of difficulty (those who respond at least a lot of difficulty) or those with severe difficulties (those who respond cannot do at all). Since disability is not a simple, single yes/no dichotomy, guidance is provided on producing several disability indicators based on the above cut-offs. In addition it is the recommendation of the UNICEF/WG collaboration that the cut-off at the level of a lot of difficulty be operationalized for reporting data internationally, and for the disaggregation of outcome indicators (like school attendance) by disability status.
Based on the analysis of field test data, computer syntax was generated (in SPSS) that provides the information necessary to compute these disability indicators.
In-depth data analyses were also conducted to explore the psychometric properties of the module.
Supporting documentation
Similar to the other disability measures recommended by the WG, the module on child functioning is accompanied by interviewer instructions. A description of its technical properties, as well as methodological guidance for its implementation will follow in early 2017. To that end, a manual is being prepared that will provide guidance on all aspects of the implementation of the module, as well as in data analysis and the dissemination of results. It will include the following main topics:
How the module was developed, and the rationale of each question
Results of the cognitive and field tests
How this module differs from the WG short set and the WG extended set for adults.
How and where this module can best be implemented (new or ongoing surveys, etc.)
Instructions on translations
Sampling issues
Data processing and tabulation plans
Template for reporting and guidelines for data analysis and presentation
Instruction of interviewers and related training material.
Conclusion
The work of the UNICEF/WG collaboration provides a standard way to identify, at the population level (censuses and surveys), the prevalence of functional difficulties among children aged 2 to 17 years, and to use this information to monitor participation (for example, access to education) in accordance to the Convention on the Rights of the Child and the UN Convention on the Rights of Persons with Disabilities. These data may help countries and governments plan and develop better practices to improve the living conditions and well-being of children with disabilities around the world.
The finalized UNICEF/WG Module on Child Functioning is available on the WG website here: http://www.washingtongroup-disability.com/washington-group-question-sets/child-disability/ and on the UNICEF website here: https://data.unicef.org/topic/child-disability/child-functioning-module/
Official translations are currently available in Spanish and Vietnamese. Translations into the other official UN languages (Arabic, Chinese, French, and Russian) are pending.
Footnotes
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the National Center for Health Statistics
Ad hoc independent fields testing on a subset of draft questions was carried out in India and Cameroon by the London School of Hygiene and Tropical Medicine in 2013/2014, and in Italy in 2012/2013 by the Italian Institute of Statistics.
References
- Crialesi R, De Palma E, Battisti A, and Children’s Workgroup. Building a Module on Child Functioning and Disability In: International Measurement of Disability: Purpose, Method and Application – The work of the Washington Group on Disability Statistics. Altman B (Ed). Springer, Social Indicators Research Series, Volume 61:151–165, 2016 [Google Scholar]
- Madans JH, Altman BM, Rasch EK, Mbogoni M, Synneborn M, Banda J, et al. Washington Group Position Paper: Proposed Purpose of an Internationally Comparable General Disability Measure. 2004. Available online at: http://www.cdc.gov/nchs/data/washington_group/WG_purpose_paper.pdf
- Massey M, Chepp V, Zablotsky B, Creamer C, “Analysis of Cognitive Interview Testing of Child Disability Questions in Five Countries”, NCHS, available at http://wwwn.cdc.gov/qbank/NewReports.aspx, 2014. [Google Scholar]
- Massey M, Scanlon P, Lessem S, Cortes L, Villarroel M, Salvaggio M, “Analysis of Cognitive Testing of Child Disability Questions: Parent-Proxy vs. Teen Self-Report”, NCHS, available at http://wwwn.cdc.gov/qbank/NewReports.aspx, 2015. [Google Scholar]
- Miller K, Mont D, Maitland A, Altman B, Madans J. Results of a cross-national structured cognitive interviewing protocol to test measures of disability. Quality and Quantity 2011; 45(4):801–815. [Google Scholar]
- UNICEF. Convention on the Rights of the Child, Available online at: http://www.unicef.org/crc/, 1989.
- UN (United Nations). Convention on the Rights of Persons with Disabilities, UN, Geneva, 2006. [Google Scholar]
- WHO (World Health Organization). World Report on Disability. World Health Organization, Geneva, 2011. Available online at: http://whqlibdoc.who.int/publications/2011/9789240685215_eng.pdf [Google Scholar]
- WHO (World Health Organization). International Classification of Functioning, Disability and Health: ICF. World Health Organization, Geneva, 2001. [Google Scholar]
- WHO (World Health Organization). The International Classification of Functioning, Disability and Health for Children and Youth: ICF–CY. World Health Organization, Geneva, 2007. [Google Scholar]