Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 1.
Published in final edited form as: Phys Occup Ther Pediatr. 2015 Dec 7;36(4):422–439. doi: 10.3109/01942638.2015.1101044

Establishing the cultural equivalence of the Young Children’s Participation and Environment Measure (YC-PEM) for use in Singapore

Chun Yi Lim 1, Mary Law 2, Mary Khetani 3, Nancy Pollock 4, Peter Rosenbaum 5
PMCID: PMC5176101  NIHMSID: NIHMS820633  PMID: 26642891

Abstract

Aims

Establishing the cultural equivalence of clinical assessments is critical to ensuring culturally competent care. Developed in North America, the Young Children’s Participation and Environment Measure (YC-PEM) is a caregiver questionnaire of participation in home, preschool, and community activities and perceptions of environmental impact on participation. The purpose of this study is to establish the cultural equivalence of YC-PEM content for use in Singapore.

Methods

We conducted semi-structured interviews with 10 early childhood and healthcare providers and cognitive interviews with 10 parents in Singapore to examine the relevance of original YC-PEM content (activities, environmental factors, item wording). Interviews were transcribed, summarized and categorized according to item, semantic, conceptual, and operational dimensions of the Applied Cultural Framework that served as a decision-making guide to keep, modify or delete the items.

Results

Providers and parents agreed on conceptual, item and semantic changes but less on operational changes. Significant changes were made to improve comprehension of the YC-PEM.

Conclusions

Despite using the same language, the majority of the YC-PEM items needed modification to be relevant to the Singapore culture. Cultural adaptation of the YC-PEM is important because there are cultural differences in children’s participation, their environments as well as the way people understand words.

Keywords: Participation, environment, children, culture, adaptation, assessment

Introduction

The incidence of disability among children younger than seven years in Singapore was estimated at about 7,000 (or 3.2%) in 2010 (Enabling Masterplan [EM] 2012-2016 Steering Committee, 2012). Between 2004 and 2010, the majority of young children who accessed early intervention services from the national Child Development Program were diagnosed with speech and language disorders (24-37%) and autism spectrum disorders (21-29%). The Enabling Masterplan for the Disability Sector 2012-2016 was recently developed to guide policy and program development for people with disabilities in Singapore, with the vision of empowering people with disabilities to achieve full participation as an indicator of societal inclusion (EM 2012-2016 Steering Committee, 2012). Toward this end, a framework has been proposed to improve the effectiveness of early intervention services in achieving full participation of young children with disabilities (EM 2012-2016 Steering Committee, 2012). However, little is known about the participation of young children with disabilities in Singapore and the effectiveness of early intervention services in helping them to achieve full participation. This may be due to the lack of participation measures in Singapore. There is a need for a participation measure that is relevant to the Singapore context to inform the participation patterns of young children with disabilities in Singapore. It could also be used as one of the outcome measures to evaluate the effectiveness of early intervention services in Singapore.

Although English is the primary language in Singapore and no language translation is required, it is important for the measure content to be culturally adapted because participation occurs within a context that is heavily influenced by environmental factors (King, 2013). To our knowledge, there are no children’s participation measures that have been developed or adapted for use in a Singaporean context. The Young Children’s Participation and Environment Measure (YC-PEM) is the first caregiver questionnaire to combine the assessment of both participation and environmental supports and barriers to participation for children 0 to 5 years with and without disabilities (Khetani et al., 2013b). The YC-PEM is a newly validated caregiver-report questionnaire for use in large-scale research in North America (Khetani et al., 2014a). Prior studies informed the design of the YC-PEM in terms of content (i.e., activities, environmental factors) and scaling (Khetani et al., 2012; Khetani et al., 2013; Khetani et al., 2011).

Psychometric evaluation of the YC-PEM revealed internal consistency of .68 to .96 for participation scales and .92 to .96 for environment scales for children with and without developmental delay (Khetani et al., 2014a). Test-retest reliability ranged from .31 to .93 for participation scales and .91 to .94 for environment scales within a 2-4 week period (Khetani et al., 2014a). Majority of the participation and environment scales discriminated between disability groups for all three settings and the participation scales demonstrated small to moderate correlations with functional performance scores (Khetani et al., 2014a). The YC-PEM can be filled out independently by caregivers, and captures a broad range of relevant settings (i.e., home, daycare/preschool, community) and environmental factors (e.g., physical layout, sensory qualities, time, money) that impact participation. The measurement approach of the YC-PEM is similar to the Participation and Environment Measure for Children and Youth (PEM-CY) (Coster et al., 2011) in that participation and environment are assessed together in the same questionnaire, to afford for greater specificity in examining environmental impact on participation in the home, preschool and community settings (Khetani et al., 2014a). The YC-PEM is a relatively new measure compared to the PEM-CY, and the PEM-CY has been found to be useful in practice (Khetani et al., 2014b), research (Law et al., 2013; Coster et al., 2013; Bedall et al., 2013) and policy making (Khetani et al., 2014c).

The Singaporean YC-PEM was the first to be culturally adapted from the original North American version. While the YC-PEM was developed to consider the viewpoints of parents of children of diverse ages, disabilities, and background, the content and scaling of the YC-PEM were based on the perspectives of North American parents (Khetani et al., 2011) and may not be relevant for Singaporean parents with young children. In a systematic review of cross-cultural equivalence testing of participation measures for adults, Stevelink & Brackel (2013) found that it is important to understand variability in the conceptualization of participation across cultures and that cultural validation of participation measures is essential before they are used in a new culture. They used an adapted version of the Applied Cultural Equivalence Framework (refer to Table 1) to guide the process of cultural equivalence testing of several participation measures for adults. This study used the same adapted version of the Applied Cultural Equivalence Framework proposed by Stevelink & Brackel (2013) to guide the process of establishing the cultural equivalence of the Singapore YC-PEM. This framework includes five categories of equivalence (conceptual, item, semantic, operational and measurement) and can be used to determine how equally suitable an instrument is for use in two or more cultures. This framework was chosen because (1) it elaborates on categories of equivalence based on various sources (Herdman et al., 1998; Bowden & Fox-Rushby, 2003; Terwee et al., 2007), (2) it provides clear criteria of assessing how suitable an instrument is in a new culture, and (3) it was found to be useful in testing the cultural equivalence of participation instruments (Stevelink & Brackel, 2013). This study focused on establishing conceptual, item, semantic, and operational equivalence. Future studies are underway to establish measurement equivalence.

Table 1.

Adapted version of the Applied Cultural Equivalence Framework (Stevelink & Brackel, 2013)

Equivalence Categories Definition of Equivalence
Conceptual Relevancy of the underlying concept, domains and focus of the questionnaire to the new culture
Item Acceptability of the items to the culture that the measure is being adapted in
Semantic Transfer of meaning across languages
Operational Suitability of the instructions, format and response options in the culture that the measure is being adapted in
Measurement Equivalence of the psychometric properties of the adapted version of the participation measure

Studies have also shown that culture shapes parental beliefs that influence the types of activities that children participate in at home and in the community (Ullenhag et al., 2012; Harkness et al., 2011; Gallimore et al., 1993). Several children’s participation and activity measures have been culturally adapted to ensure the relevance of the measure to a specific culture (Costa, 2014; Bult et al., 2010; Ullenghag et al., 2012a; Ullenghag et al., 2012b; Stofel & Berg, 2008; Nordtorp et al., 2013). Cultural adaptation improves consistency in the face and content validity between the original and culturally adapted versions of a measure (Guillemin et al., 1993). Adapting the YC-PEM to the Singapore cultural context may help to ensure that it accurately reflects the participation and environment of young children in Singapore. The objective of this study is therefore to develop a Singaporean version of the YC-PEM by establishing its cultural equivalence. This study is the first phase of a larger study that includes psychometric testing of the adapted version and exploring its utility.

Methods

Study Design

To establish the cultural equivalence of the YC-PEM, qualitative interviews were conducted with the original version of the YC-PEM to elicit provider and parent perspectives about the clarity and comprehensiveness of items. The purpose of the interviews was to improve on the cultural relevance of the questionnaire for use in clinical practice, so we engaged with providers and parents in Singapore who would likely be the primary users of the YC-PEM during baseline assessment in clinical practice. Face-to-face semi-structured interviews were conducted with providers in Singapore to gather item-by-item feedback. Face-to-face cognitive interviews were also conducted with caregivers of young children with developmental delays. Cognitive interviews assess respondents’ interpretation and understanding of the items in self-report questionnaires, improving its content validity and reliability (Irwin et al., 2009; Knafl et al., 2007). The purpose of the cognitive interviews with the parents was to ensure that the content of the YC-PEM retained its equivalence to the original version from the perspectives of the study population.

Participants

Purposive and snowball sampling techniques were used to sample providers with diverse experience in terms of practice settings (e.g. acute, community), discipline (e.g. occupational therapist, psychologist, social worker, etc.) and professional background (e.g. educator, researcher, practitioner, etc.) We first approached providers from Singapore’s leading referral centre for children with developmental needs, the Department of Child Development (the DCD) at KK Women’s and Children’s Hospital. Recruited providers then referred us to other providers with whom they work, from community agencies, universities and preschools. The inclusion criteria for the providers were at least 5 years of clinical, research or teaching experience in the field of early childhood. Table 2 describes the characteristics of the providers sampled.

Table 2.

Characteristics of Service Providers (N=10)

Characteristic n
Professional background Early Childhood Educator 3
Occupational Therapist 3
Psychologist 2
Social Worker 1
Pediatrician 1
Setting Education 4
Community 2
Acute Hospital 4
Years of working experience 5-10 3
>10 7
Parent of a child without disability Yes 9
No 1

Willis, D. B. (1999) stated that cognitive interviews to improve on survey questions should focus on the diversity of individuals and not quantity. Parents were recruited through a series of sampling methods. First we used maximum variation sampling to accomplish diversity in the sample in terms of socioeconomic status (low, middle and high income) and child’s age (0-2, 2-4, 4-6 years old). We then used snowball sampling to expand the recruitment. For this study, children with a presence of disability are children with developmental delay enrolled in early intervention programs. For parents of typically developing children, we approached providers from the DCD who knew of parents that fit our inclusion criteria. For parents of children with developmental delay, we distributed recruitment flyers at parent support groups, early intervention centers and the waiting areas of the DCD. Recruited parents then referred us to additional parents who they thought would be willing to participate in the study. Finally, we used quota sampling by recruiting specific characteristics of participants whom socioeconomic status and age have not been represented for. The inclusion criteria for the parents were (i) they identified themselves as a parent or legal guardian of the child who was the focus of the survey; and (ii) they were able to read and write English; (iii) their child was between 0 and 7 years old at the time of the study. The YC-PEM was developed for children aged 0 to 5 in North America (Khetani et al., 2013b). However, in Singapore, young children who access early intervention services range from 0 to 7 years old. Therefore, our sample included children 0 to 7 years old. Table 3 lists the demographic characteristics of the parents.

Table 3.

Characteristics of Parents and Children (N=10)

Characteristic n
Age 20-29 2
30-39 8
Gender Female 10
Male 0
Race Chinese 8
Malay 1
Indian 1
Annual income (SGD) >90,000 5
50,000-69,999 3
30,000-49,999 2
Education level Primary 1
Secondary 3
Polytechnic 1
University 1
Post-graduate 4
Main activity Working full-time 2
Working part-time and caring for family 7
Caring for family 1
Child’s age 0-2 years 3
2-4 years 1
4-6 years 6
Child’s gender Male 6
Female 4
Child’s diagnosis Autism spectrum disorder 3
Speech and language delay 2
No diagnosis 5
Child’s education settings* Kindergarten 5
Childcare 3
Early intervention program 5
No school 2
Child’s main caregivers* Mother 4
Grandparents 4
Domestic helpers 5
Childcare 2
*

Some children attend 2 types of education settings and have 2 caregivers

Most parents identified two main caregivers of the child who was the focus of the survey. The parents who participated in this study were one of the main caregivers of the child. In Singapore, caregiving is often shared by the extended family (e.g. grandparents) and domestic helpers, who live together with the family.

Measure

Providers completed a demographic questionnaire and parents completed both the demographic questionnaire and the original version of the YC-PEM. The YC-PEM assessed caregiver perceptions of their young child’s participation in broad types of activities that take place in the home (13 items, e.g., mealtime, cleaning up, indoor play and games, celebrations at home), daycare/preschool (3 items, e.g., group learning, socializing with friends, field trips and events), and community (12 items, e.g., dining out, classes and lessons, community attractions, overnight visits or trips) settings. The caregiver was provided with examples of each type of activity. For each type of activity, the caregiver assessed three dimensions of their child’s participation: 1) frequency (8-point scale, from never [0] to once or more each day [7]); 2) level of involvement (5-point scale, from not very involved [1] to very involved [5]; participants skipped this step if they selected “never” for frequency); and 3) their desire for change in the child’s participation (yes [1] vs. no [0]). If yes, the caregiver clarified if change is desired in terms of frequency (i.e., more often or less often), level of involvement (i.e., more interactive and/or more helpful), and/or participation in a broader variety of activities of that type.

After completing participation items for a setting, caregivers evaluated the impact of types of environmental features (e.g., physical layout, sensory qualities, activity demands, social relationships, attitudes, safety, weather, policies) and resources (e.g., transportation, equipment and supplies, information, time, money) on the child’s participation in that setting (13 items for home, 16 items for daycare/preschool, and 17 items for community). The caregiver was provided with examples of environmental features and resources. Perceived impact of environmental features on participation was assessed on a 3-point scale (no impact/usually helps [3] to usually makes harder [1]). Perceived impact of environmental resources on participation was assessed on a 3-point scale (not needed/usually yes [3] to usually no [1]).

YC-PEM summary scores are calculated for each setting: (1) Frequency is calculated as the average of all ratings within a setting (range = 0-7); (2) Level of Involvement is calculated as the average of all ratings within a setting (range = 1-5); (3) Desire for change was calculated by summing the number of items scored as 'yes, change desired', divided by the total number of items, and multiplied by 100 (range = 0-100); and (4) Environment support was calculated by summing ratings across all environmental features and resources items and dividing by the maximum possible score, and multiplied by 100 (range = 0-100).

Procedures

Institutional Review Board approval was obtained prior to participant recruitment and data collection. Interested participants contacted the investigator who ensured that they fit the inclusion criteria before they signed the informed consent form to participate in the study. The participants decided on the date, time and venue of the interview. The providers’ interviews were conducted first. Two weeks before the interview, the providers were emailed the interview questions, and the YC-PEM. They were encouraged to prepare before the interview by reading the questions and the YC-PEM. Interviews were conducted at the respective workplaces of seven providers and at the respective homes of three providers. After all the providers’ interviews were conducted, cognitive interviews with parents were conducted with the original version of the YC-PEM. Two weeks before the interview, parents were emailed the interview questions, and the YC-PEM. On the day of the interview, parents completed the family demographic questionnaire, followed by the YC-PEM. After completing each page of the YC-PEM, the principal investigator utilized the verbal probing technique of cognitive interview (Knalf et al., 2007). Parents were probed to verbalize their interpretation of items and to paraphrase/comment on the wording of items to identify unclear or irrelevant questions. Interviews were conducted at the respective homes of eight parents, the workplace of one parent and one parent was interviewed at the investigator’s office. The principal investigator, who was trained in the use of cognitive interviews, conducted all the interviews. Both providers and parents interviews were audiotaped and each interview took about 60-90 minutes. All parents were given a $10 shopping voucher after the interview. Table 4 describes the interview guide for providers and parents.

Table 4.

Interview guide for providers and parents

Questions related to: Providers Parents
YC-PEM in general
  1. Can Singaporean parents understand the instructions of the YC-PEM? If no, how do you suggest we change them?

  2. Potential of YC-PEM in measuring participation in Singapore?

  3. To what extent do you think YC-PEM is clinically useful?

  4. Any other comments or suggestions regarding YC-PEM and its use in Singapore?

  1. How do you feel about the instructions? Are you able to understand them easily? If not, how do you suggest we should change it?

  2. Do you think the information gathered from the YC-PEM will be useful to you in parenting your child? If so, please describe. If not, why not.

  3. Any other comments or suggestions regarding YC-PEM and its use in Singapore?

Activity items
  1. Do you think Singaporean parents can understand and relate to this item? If no, how do you suggestion we should change it?

  2. Do you think Singaporean parents can relate to the examples given in this item? If no, how do you suggest we should change it?

  3. Do you think the response options are appropriate for Singaporean parents? If no, how do you suggest we should change it?

  4. Do think YC-PEM has covered a sufficient range of activities that typical Singaporean young children engage in? Are there any other activities that we should consider?

  1. How would you interpret this activity? Is it relevant to you?

  2. How would you interpret the given examples of the activity and are they relevant to you?

  3. Please “paraphrase” the question or give suggestions to change it if you feel that it is not relevant to you.

  4. Do you think the response options are relevant to you? If no, how do you suggest we should change it?

  5. Do you feel that the YC-PEM has covered a sufficient range of activities that your child typically engages in? Are there any other activities that we should consider?

Environment items
  1. Do you think Singaporean parents can understand and relate to this item? If no, how do you suggest we should change it?

  2. Do you think Singaporean parents can relate to the examples given in this item? If no, how do you suggest we should change it?

  3. Do you think the response options are appropriate for Singaporean parents? If no, how do you suggest we should change it?

  4. Do think YC-PEM has covered a sufficient range of environmental supports and barriers associated to the participation of young children? Are there any other environmental factors that we should consider?

  1. How would you interpret this item? Is it relevant to you?

  2. How would you interpret the given examples of the item and are they relevant to you?

  3. Please “paraphrase” the question or give suggestions to change it if you feel that it is not relevant to you.

  4. Do you think the response options are relevant to you? If no, how do you suggest we should change it?

  5. Do think YC-PEM has covered a sufficient range of environmental supports and barriers associated to the participation of your child? Are there any other environmental factors that we should consider?

Data analysis

The guideline proposed by Knafl et al. (2007) was used to analyze the interviews and it included transcription of interview data, summarization of participants’ interpretation of items and identification of types of problems, analysis of item summaries, and decision-making about items. All interviews were audiotaped and transcribed verbatim with the individual YC-PEM item as a unit of analysis. For each item, data was summarized across providers and parents with similar problems coded then mapped to the four dimensions of the Applied Cultural Equivalence Framework: Conceptual, Item, Semantic and Operational (Stevelink & Brackel, 2013). Summarized data from provider and parents for each item was systematically compared according to 1) areas of overlap between provider and parent perspectives; 2) item where one but not both, raised concern and; 3) items without any concern. The final analysis of each item provided an overview of range of interpretations, type of equivalence and supported the decision-making process to keep, delete or modify the items. The investigators, which included the author of the YC-PEM, reviewed and discussed the analysis of each item until consensus was reached as to whether the item should be retained, deleted, or modified. Table 5 illustrates how an item was analysed.

Table 5.

Example of analysis of an item

Item Providers’ comments Parents’ comments Equivalence type Decision

Indoor play and games (e.g. puzzles, workbooks, stuffed animals, cars, blocks, water and sand play, pretend play and dress-up, peek-a-boo, hide-and-seek and board games) Children in Singapore do not play with sand at home due to physical constraints. ‘Workbooks’ should not come under indoor play and games, as they are academic-related. Item Removed ‘sand-play’ and ‘workbooks’
Children in Singapore do workbooks but for academic reasons and not as play and games. These workbooks are usually homework from school or enrichment classes that they attend. Added an additional item ‘Structured learning time (e.g. school or enrichment classes homework, workbooks, assessment books)’ because it is relevant to Singaporean children but missing from the original YC-PEM

Results

Results are organized according to the Applied Cultural Framework, namely conceptual, item, semantic and operational. In general, the participants understood the concepts of participation and environment. They also understood the main domains and categorizations of the YC-PEM.

Most content revisions were to establish item equivalence. Thirteen out of 28 participation items and thirteen out of 36 environmental items were modified. Most of the modifications were made to the examples of the activities that were perceived to be irrelevant to our context.

Despite using the same English language, 8 words and 4 phrases in the YC-PEM had to be modified to achieve sematic equivalence.

For operational equivalence, nine out of 10 providers commented that the YC-PEM’s format and instructions were too lengthy and may be difficult for parents to understand. However, all parents could understand the instructions so there were no modifications made. For format, all parents felt that the presentation of some items was confusing and with their suggestions, we changed the sequence of some items to enhance their understanding. For example, instead of starting the questionnaire with the activity ‘getting rest’ which parents were confused because it was difficult to quantify, we started with the activity ‘mealtime’ which has a frequency that is easier to quantify. For response options, specific to type of change desired, five out of 10 parents felt that they could relate more easily to ‘more cooperative’ than to ‘more helpful’ and we modified the response option as suggested. Other operational modifications made were increasing the font size and making some words bolder.

Discussion

Importance of cultural adaption of measures

Our study found that it is important to establish cultural equivalence of participation measures before they are used in another culture even without translation. Despite using the same language, most parents in this study had difficulty understanding some parts of the original YC-PEM. Evidently, some words and phrases were understood differently in different contexts. Singaporean parents also required more explicit examples to facilitate their understanding of items. Costa (2014) found that although there are many cultural commonalities in Western societies such as Canada, Australia and Europe, regarding cultural relevant activities for children, there were still a number of cultural adaptations required from the original version of the measure. Our study demonstrated that the types and emphasis of participation varied between Singapore and North America. For example, Singaporean parents in this study would like their children to be more ‘cooperative’ instead of more ‘helpful’. This is aligned with Eastern cultures’ emphasis on self-control and obedience (Lim, Rodger & Brown, 2013; Wong, 2008). The addition of the item ‘structured learning time’ is aligned with Singapore’s culture of introduction of learning tasks at an early age (Lim, Rodger & Brown, 2013). If the YC-PEM was not culturally adapted, essential information on Singaporean children’s participation in structured learning time would not be captured, when it is such an integral part of their daily routine. The YC-PEM will thus be an invalid measure and the potential of the YC-PEM as a tool in assessing young children’s participation and environment effectively will be lost.

To our knowledge, this is the first study that culturally adapted a participation and environment measure without language translation and documented its cultural adaptation process. Cultural adaptation studies often present psychometric properties of the translated version of the questionnaire but not results of the translation or how the items were culturally adapted (Ullenhag et al., 2012a; Nordtorp et al., 2013, Bult et al., 2010; Stofel & Berg, 2008). An exception is a study by Costa (2014) that translated and culturally adapted the Austrian-German Perceived Efficacy and Goal Setting System (PEGS) by interviewing 23 occupational therapists and a 14 year-old boy. Certain wordings were changed and culturally relevant activities added. Similar to our study, the author found that parents relate to activities that have sociocultural meaning to them and that their goals and expectations for their children’s participation are influenced by culture, leading to meaningful occupations in the context of their children’s environment (Costa, 2014). The author concluded that the process of cultural adaption is important in ensuring a culturally sensitive and responsive practice (Costa, 2014).

Similarities and differences between perspectives of providers and parents

Providers and parents in our study shared some similar perspectives about conceptual, item and semantic equivalence and made similar suggestions to modify the YC-PEM to enhance comprehension. However, providers and parents had differing perspectives of operational equivalence. Operationally, providers commented that the YC-PEM is lengthy, difficult to understand and suggested that parents are unlikely to complete it if they do not understand its importance. In contrast, some parents commented on the length but reported being able to understand items or suggested the use of more explicit examples to improve comprehension. It took about 20 minutes for parents to complete the original version of the YC-PEM (Khetani et al., 2014). In this study, the average duration to complete the YC-PEM together with the cognitive interview was about 60-90 minutes. A systematic review and meta-analysis of studies reporting response burden in relation to questionnaire length suggested that it is the content and not the length of the questionnaire that influences response rates (Rolstad et al., 2011). The more relevant the questions are to the patients, the more motivated they are to respond and time spent on completing the questionnaire may not be perceived as burdensome (Rolstad et al., 2011). Two parents of children without developmental delay did not find the information from the YC-PEM useful to them and thus might have experienced response burden. Comparatively, parents of children with developmental delay find the information from the YC-PEM useful to them and were motivated to complete it despite its length.

This supports the need for this study to improve on the relevance of the items and not focus on the length of the YC-PEM. It is also important for service providers to educate parents on the need to complete the YC-PEM and for researchers to educate parents on the rationale of completing the YC-PEM in relation to the studies that they conduct. Parents also provided practical suggestions to modify the YC-PEM whereas providers provided comments on its potential utility in the areas of intervention planning and research. Evidently, providers and parents had distinct views and specialized areas of knowledge with regard to assessing the cultural relevance of the YC-PEM. Knalf et al (2007) found that feedback from providers helped assess the conceptual relevance of the measure while parents helped to assess the clarity and relevance of items to them. Our role as researchers is to utilize the input of these different expert sources optimally to successfully adapt the YC-PEM to this cultural context (Knalf et al., 2007). Our study also found that cognitive interviews are useful in allowing researchers to understand the thought processes of parents and yield evidence for establishing cultural equivalence of a measure, in addition to improving its reliability and validity (Irwin et al., 2009; Knalf et al., 2007). To our knowledge, this is the first study that utilized cognitive interviews to culturally adapt a measure.

Need for knowledge translation amongst providers

Our findings suggest that providers may have a more negative view about the utility of the YC-PEM than parents. Lim et al. (2014a) used the Knowledge-To-Action (KTA) Framework to understand the current assessment practice of occupational therapists in Singapore and to introduce the YC-PEM in a form of a research overview. They found that only 4 out of 25 (8%) Singaporean occupational therapists used participation and environment assessments while all used motor and visual-perception skills assessments. Occupational therapists in the study cited the lack of participation and environment measures as a reason to not using them and all were interested in using the YC-PEM. However, in this study, providers appeared to have a more negative view about the utility of the YC-PEM than parents. One reason is that 7 out of ten of the providers were not occupational therapists and they may not see the need to assess participation and environment of children because their job scopes do not focus on the activities that children participate in. The psychologists, social worker and pediatrician felt that the YC-PEM was useful for occupational therapists that work with children with developmental delay and some of the early childhood educators felt that it is not useful for typically developing children. On the contrary, the YC-PEM could be a potential tool to assist educators in assessing children at risks of developmental delay in terms of their ability to participate like other children, particularly in the childcare/preschool setting.

Since providers play a key role in the uptake of the YC-PEM questionnaire in practice, their feedback can impact the uptake of the instrument in early intervention service delivery and outcomes research. In response, knowledge translation strategies should reach out to service providers beyond healthcare, such as early childhood educators, to increase their understanding of the importance of participation and environment in the development of young children. Knowledge translation strategies include sharing the YC-PEM and its related research through different platforms such as webinar, educational outreach, audit and feedback, interactive continuing education sessions and problem-based learning groups (Law et al., 2001; MacDermid & Graham, 2009). Alternatively, uptake of the YC-PEM in large sample research in Singapore may help to increase attention to this service outcome. For example, translate clinically relevant knowledge based on the use of the tool (e.g. knowledge about disparities based on use of the tool, utility of the tool in patient care) to important stakeholders such as providers and parents to draw their attention to the importance of measuring participation (Graham et al., 2006). Several studies have used the PEM-CY to generate knowledge about participation and environmental differences between children with and without disabilities in the home, school and community settings (Bedall et al., 2013; Coster et al., 2013; Law et al., 2013) whilst Khetani et al (2014b) studied the utility of the PEM-CY for collaborative planning with parents of children with disabilities.

Limitations

Despite using various sampling methods, our sample had fewer children between 2-4 years old and therefore this age group was under represented. We also did not ensure that the sample was diversified in terms of types of disability so our sample lacked parents of children with physical disabilities. Parents of young children with physical disabilities may have different emphasis in terms of participation and environment. Due to time limitations, we did not sample until the data reaches saturation and assumed that we had sufficient data to culturally adapt the YC-PEM with the pre-determined sample size. The sample of parents may not be representative of all parents who would be asked to complete the YC-PEM. More interviews with providers and cognitive interviews with parents could also have been conducted with the YC-PEM (Singapore) to ensure that the adapted version is culturally relevant.

Conclusion

Cultural adaption of instruments should be responsive to the meaning and relevance of the activities to the daily lives of children (WFOT, 2010). Despite using the same language as its original version, the majority of the YC-PEM items needed modification to be relevant to the Singapore culture. This study also informed the importance of establishing cultural equivalence of participation measures through involvement of different expert sources such as healthcare/early childhood providers and parents. Research is underway to examine the reliability and validity of the Singapore YC-PEM. The YC-PEM (Singapore) has the potential to be used to conduct large-scale studies to build knowledge about patterns and predictors of young children’s participation in Singapore as well as environmental supports and barriers to their participation. It also has the potential to determine the effectiveness of early intervention services in achieving full participation and societal inclusion of young children with disabilities in Singapore.

Table 6.

Conceptual equivalence

Comments by:
Service Providers Occupational therapists
  • “Clinically useful”

  • “Too brief and vague for clinical usage”

  • “Useful for research”

  • “Useful for early intervention program evaluation”

  • “Large survey may be difficult”

Non-occupational therapists
  • “Clinically useful”

  • “Useful for occupational therapists who work with children with developmental delay”

  • “Useful to assess life skills of children”

  • “Useful to gather a picture of family life of the child”

  • “Not useful for typically developing children”

  • “Useful for policy planning”

  • “Useful for early intervention program planning”

  • “Too long and parents may not see the need to complete this”

Parents Parents of children with developmental delay
  • “Tells me what my child is doing now”

  • “Helps me to reflect on my child’s environmental needs”

  • “Helps me to understand my child better and what is happening in his life”

  • “Helps me to reflect if he is doing too much of something or too little of something else”

  • “Provides a bigger picture of what he does”

  • “Helps me to think of what I want more for my son. For example, more community activities”

  • “Gives me a better idea on which area to work more on”

Parents of children without developmental delay
  • “Not useful as I already know what my child does”

  • “It is just informing what my child does”

  • “Useful as it helps me to know what to improve on and what she is already good in”

  • “Helps me to analyze my child better”

  • “Puts things into perspective for me and has a better picture of what is going on in my child’s life”

Table 7.

Item equivalence

Item Equivalence Original item Modification(s)

Participation items

Home participation Cleaning up (e.g. picking up toys, throwing away trash, sweeping floor or wiping table, yard work) Removed: yard work
Added: tidy room

Taking care of other family members (e.g. sibling, pet) Added: grandparents

Laundry and dishes (e.g. load, unload, sort fold or put away) Added: washing dishes

Indoor play and games (e.g. puzzles, workbooks, stuffed animals, cars, blocks, water and sand play, pretend play and dress-up, peek-a-boo, hide-and-seek and board games) Removed: sand play and workbook

Daycare/preschool participation Field trips and events (e.g. parent night out, going to the library, school concert or fundraiser) Removed: parent night out, going to the library, fundraiser
Added: museums, aquarium, gardens

Community participation Shopping and errands (e.g. grocery, store, mall, post office, bank, pet store, car wash or auto repair shop) Removed: auto-repair shop
Added: supermarket

Dining out (e.g. dine-in or take out meals, coffee shop) Removed: dine-in or take-out meals
Added: hawker centre, food court, restaurant

Classes and lessons (e.g. music, art, language lessons) Added: reading and mathematics lessons

Organized physical activities (e.g. soccer, T-ball, gymnastics, dance, martial arts) Replaced: T-ball with swimming

Organization, groups, clubs (e.g. boy scouts, brownies/girl guides) Replaced: Interest groups (e.g. Lego, reading, robotics)

Community attractions (e.g. libraries and bookstores, museums, movie theater, aquarium, orchards, animal farms, petting zoos) Replaced: orchards with garden, petting zoos with zoo

Community events (e.g. festivals, fairs, parades, concerts, theatre, sporting events) Replaced: theatre with community centre events

Unstructured physical activities (e.g. playground and parks, beaches, hiking, bikes and scooters, sledding, fishing, ice skating) Removed: sledding and hiking
Added: swimming, beaches

Environment items

Home environment The physical layout (e.g. having organized, clean, safe space at home) Added: no curbs

The attitudes and actions of babysitters, therapists and other professionals who care for your child at home Replaced: babysitter with domestic helper

Policies (e.g. residential and workplace policies, such as family leave or working from home, time-off, hour hours) Removed: residential policies

Services in the home (e.g. therapists, babysitters, etc.) Replaced: babysitters with domestic helpers

Childcare/preschool environment Programs and services at the organized daycare or preschool (education assistant, special resources, etc.) Replaced: special resources with learning support educators, school-based therapy

Supplies (assistive devices, adapted toys, craft supplies, organic food choices, modular furniture, accessible bathrooms, classrooms and playgrounds, access to internet and technology to support learning) Removed: modular furniture and ‘organic’ in ‘organic’ food choices

Do you (and your family) have enough time to support your child’s participation at daycare or preschool (e.g. paying tuition, purchasing supplies, participating in fundraisers)? Replaced: participating in fundraisers with participating in extra classes or programs, purchasing costumes for school concert

Community environment Physical layout or amount of space outside and inside the buildings (e.g. distance to stores, presence of sidewalks, availability of ramps or elevators) Added: availability of nursing rooms

Your child’s relationships with peers Added: cousins

The safety of the community (e.g. traffic, crime, violence) Added: dengue fever and hand mouth foot disease (HMFD)

Policies (neighborhood, childcare and employer policies) Replaced: Government, neighborhood, workplace policies, (e.g. flexible work schedule to afford for time to participate in the community)

Programs and services in the community Added: (e.g. community center programs and events)

Information (e.g. about activities, services and programs) Added: (e.g. brochures, newsletters, websites etc. about community activities, services, programs)

Table 8.

Semantic equivalence

Semantic equivalence Original item Modifications
Words Field trips and events School excursions and events
Daycare/preschool Childcare/preschool
Tuition fees School fees
Subway MRT (Mass Rapid Transit)
Movie theatre Cinema
Educational assistant Assistant teacher
Director Centre supervisor/principal
Trips/vacations Trips/tours
Phrases Please describe up to three strategies that you have tried to help your child participate in this type of activity. Please describe up to three strategies that you have tried to help your child participate in this type of activity.
The physical demands of typical activities The physical skills required to do typical activities
The cognitive demands of typical activities The cognitive skills required to do typical activities
The social demands of typical activities The social skills required to do typical activities

Acknowledgments

This study was conducted with a grant awarded by Singapore Association of Occupational Therapists. We thank KK Women’s and Children’s Hospital (Singapore), Department of Child Development, especially Ms. Teo Hui Fang, for their assistance in recruiting participants and for providing logistical support. We also thank all professionals and parents who supported and participated in this study.

Footnotes

Conflict of interest statement: All authors declare that they have no conflict of interests.

Contributor Information

Chun Yi Lim, School of Rehabilitation Science, McMaster University.

Mary Law, School of Rehabilitation Science, McMaster University.

Mary Khetani, Department of Occupational Therapy, Colorado State University.

Nancy Pollock, School of Rehabilitation Science, McMaster University.

Peter Rosenbaum, Department of Pediatrics, McMaster University.

References

  1. Adolfsson M, Malmqvist J, Pless M, Granuld M. Identifying child functioning from an ICF-CY perspective: everyday life situations explored in measures of participation. Disability and Rehabilitation. 2011;33(13-14):1230–44. doi: 10.3109/09638288.2010.526163. [DOI] [PubMed] [Google Scholar]
  2. Bedell G, Coster W, Law M, Liljenquist K, Kao Y-C, Teplicky R, Khetani MA. Community participation, supports, and barriers of school-age children with and without disabilities. Archives of Physical Medicine and Rehabilitation. 2013;94(2):315–23. doi: 10.1016/j.apmr.2012.09.024. [DOI] [PubMed] [Google Scholar]
  3. Bowden A, Fox-Rushby JA. A systematic and critical review of the process of translation and adaptation of generic health-related quality of life measures in Africa, Asia, Eastern Europe, the Middle East, South America. Social Science and Medicine. 2003;57:1289–1306. doi: 10.1016/s0277-9536(02)00503-8. [DOI] [PubMed] [Google Scholar]
  4. Bult MK, Verschuren O, Gorter JW, Jongmans MJ, Piskur B, Ketelaar M. Cross-cultural validation and psychometric evaluation of the Dutch language version of the Children’s Assessment of Participation and Enjoyment (CAPE) in children with and without physical disabilities. Clinical Rehabilitation. 2010;24(9):843–53. doi: 10.1177/0269215510367545. [DOI] [PubMed] [Google Scholar]
  5. Costa UM. Translation and Cross-Cultural Adaptation of the Perceived Efficacy and Goal Setting System (PEGS): Results from the First Austrian-German PEGS Version Exploring Meaningful Activities for Children. OTJR: Occupation, Participation and Health. 2014;34(3):119–30. doi: 10.3928/15394492-20140325-02. [DOI] [PubMed] [Google Scholar]
  6. Coster W, Khetani M. Measuring participation of children with delay: Issues and challenges. Disability and Rehabilitation. 2008;30(8):639–648. doi: 10.1080/09638280701400375. [DOI] [PubMed] [Google Scholar]
  7. Coster W, Bedell G, Law M, Khetani MA, Teplicky R, Liljenquist K, Gleason K, et al. Psychometric evaluation of the Participation and Environment Measure for Children and Youth. Developmental Medicine and Child Neurology. 2011;53(11):1030–7. doi: 10.1111/j.1469-8749.2011.04094.x. [DOI] [PubMed] [Google Scholar]
  8. Coster W, Law M, Bedell G, Liljenquist K, Kao Y-C, Khetani M, Teplicky R. School participation, supports and barriers of students with and without disabilities. Child: Care, Health and Development. 2013;39(4):535–43. doi: 10.1111/cch.12046. [DOI] [PubMed] [Google Scholar]
  9. Dunst CJ, Hamby D, Trivette CM, Raab M, Bruder MB. Young children’s participation in everyday family and community activity. Psychological Reports. 2002;91:875–897. doi: 10.2466/pr0.2002.91.3.875. [DOI] [PubMed] [Google Scholar]
  10. Enabling Masterplan 2012–2016 Steering Committee. Enabling Masterplan 2012–2016: Maximising potential, embracing differences. Singapore: Ministry of Community Development, Youth, and Sports; 2012. Retrieved from http://app1.mcys.gov.sg/Policies/DisabilitiesPeoplewithDisabilities/EnablingMasterplan20122016.aspx. [Google Scholar]
  11. Gallimore R, Goldenberg CN, Weisner TS. The Social Construction and Subjective Reality of Activity Settings: Implications for Community Psychology. American Journal of Community Psychology. 1993;21(4):537–559. doi: 10.1007/BF00942159. [DOI] [PubMed] [Google Scholar]
  12. Graham ID, Tetroe J. Some theoretical underpinnings of knowledge translation. Academic Emergency Medicine : Official Journal of the Society for Academic Emergency Medicine. 2007;14(11):936–41. doi: 10.1197/j.aem.2007.07.004. [DOI] [PubMed] [Google Scholar]
  13. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. Journal of Clinical Epidemiology. 1993;46(12):1417–32. doi: 10.1016/0895-4356(93)90142-n. [DOI] [PubMed] [Google Scholar]
  14. Harkness S, Zylicz PO, Super CM, Welles-Nyström B, Bermúdez MR, Bonichini S, Mavridis CJ. Children’s activities and their meanings for parents: a mixed-methods study in six Western cultures. Journal of Family Psychology. 2011;25(6):799–813. doi: 10.1037/a0026204. [DOI] [PubMed] [Google Scholar]
  15. Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaption of HRQoL instruments: the universalist approach. Quality of Life Research. 1998;7:323–335. doi: 10.1023/a:1024985930536. [DOI] [PubMed] [Google Scholar]
  16. Irwin DE, Varni JW, Yeatts K, DeWalt DA. Cognitive interviewing methodology in the development of a pediatric item bank: a patient reported outcomes measurement information system (PROMIS) study. Health and Quality of Life Outcomes. 2009;7:3. doi: 10.1186/1477-7525-7-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Khetani M, Graham JE, Alvord C. Community participation patterns among preschool-aged children who have received Part C early intervention services. Child: Care, Health and Development. 2013;39(4):490–9. doi: 10.1111/cch.12045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Khetani MA, Orsmond G, Cohn E, Law MC, Coster W. Correlates of Community Participation Among Families Transitioning From Part C Early Intervention Services. OTJR: Occupation, Participation, Health. 2012;32(3):61–72. [Google Scholar]
  19. Khetani MA, Cohn ES, Orsmond GI, Law MC, Coster WJ. Parent Perspectives of Participation in Home and Community Activities When Receiving Part C Early Intervention Services. Topics in Early Childhood Special Education. 2011;32(4):234–245. [Google Scholar]
  20. Khetani MA, Graham JE, Davies PL, Law MC, Simeonsson RJ. Psychometric Properties of the Young Children’s Participation and Environment Measure. Archives of Physical Medicine and Rehabilitation. 2014 doi: 10.1016/j.apmr. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Khetani Ma, Cliff AB, Schelly C, Daunhauer L, Anaby D. Decisional Support Algorithm for Collaborative Care Planning using the Participation and Environment Measure for Children and Youth (PEM-CY): A Mixed Methods Study. Physical & Occupational Therapy in Pediatrics. 2014b doi: 10.3109/01942638.2014.899288. Advance online publication. [DOI] [PubMed] [Google Scholar]
  22. Khetani M, Marley J, Baker M, Albrecht E, Bedell G, Coster W, Law M. Validity of the Participation and Environment Measure for Children and Youth (PEM-CY) for Health Impact Assessment (HIA) in sustainable development projects. Disability and Health Journal. 2014c;7(2):226–35. doi: 10.1016/j.dhjo.2013.11.003. [DOI] [PubMed] [Google Scholar]
  23. King G. Perspectives on measuring participation: going forward. Child: Care, Health and Development. 2013;39(4):466–9. doi: 10.1111/cch.12083. [DOI] [PubMed] [Google Scholar]
  24. Knafl K, Deatrick J, Gallo A, Holcombe G, Bakitas M, Dixon J, Grey M. The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health. 2007;30(2):224–234. doi: 10.1002/nur.20195. [DOI] [PubMed] [Google Scholar]
  25. Larson RW, Verma S. How children and adolescents spend time across the world: Work, play, and developmental opportunities. Psychological Bulletin. 1999;125(6):701–736. doi: 10.1037/0033-2909.125.6.701. [DOI] [PubMed] [Google Scholar]
  26. LaVesser P, Berg C. Participation patterns in preschool children with an autism spectrum disorder. OTJR: Occupation, Participation, Health. 2011;31(1):33–39. [Google Scholar]
  27. Law M. Participation in the occupations of everyday life. American Journal of Occupational Therapy. 2002;56:640–649. doi: 10.5014/ajot.56.6.640. [DOI] [PubMed] [Google Scholar]
  28. Law M, Anaby D, Teplicky R, Khetani MA, Coster W, Bedell G. Participation in the home environment among children and youth with and without disabilities. The British Journal of Occupational Therapy. 2013;76(2):58–66. [Google Scholar]
  29. Law M, Missiuna C, Pollock N. Knowledge exchange and translation: An essential competency in the twenty-first century. Occupational Therapy Now. 2001;10(5):3–5. [Google Scholar]
  30. Leung GPK, Chan CCh, Chung RCK, Pang MYC. Determinants of activity and participation in preschoolers with developmental delay. Research in Developmental Disabilities. 2011;32:289–296. doi: 10.1016/j.ridd.2010.10.005. [DOI] [PubMed] [Google Scholar]
  31. Lim SM, Rodger S, Brown T. Model of Social Competence in an Early Childhood Environment. Occupational Therapy in Mental Health. 2013;29(2):114–133. [Google Scholar]
  32. Lim CY, Tan PC, Koh C, Koh E, Guo H, Yusoff ND, See CQ, Tan T. Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery-VMI): lessons from exploration of cultural variations in visual-motor integration performance of preschoolers. Child: Care, Health and Development Advance online publication. 2014 doi: 10.1111/cch.12190. [DOI] [PubMed] [Google Scholar]
  33. Lim CY, Law M, Rosenbaum P, Pollock N, Khetani M. Using the Knowledge-to-Action (KTA) Framework to introduce a newly developed participation measure for young children in Singapore. Research paper presented at the National Occupational Therapy Conference; Singapore. 2014a. [Google Scholar]
  34. Low W, Dirani M, Gazzard G, Chan YH, Zhou H-J, Selvaraj P, Au Eong K-G, et al. Family history, near work, outdoor activity, and myopia in Singapore Chinese preschool children. The British Journal of Ophthalmology. 2010;94(8):1012–6. doi: 10.1136/bjo.2009.173187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. MacDermid JC, Graham ID. Knowledge translation: putting the “practice” in evidence-based practice. Hand Clinics. 2009;25(1):125–43. doi: 10.1016/j.hcl.2008.10.003. [DOI] [PubMed] [Google Scholar]
  36. Nordtorp HL, Nyquist A, Jahnsen R, Moser T, Strand LI. Reliability of the Norwegian version of the Children’s Assessment of Participation and Enjoyment (CAPE) and Preferences for Activities of Children (PAC) Physical & Occupational Therapy in Pediatrics. 2013;33(2):199–212. doi: 10.3109/01942638.2012.739269. [DOI] [PubMed] [Google Scholar]
  37. Quek KM-T, Knudson-Martin C, Orpen S, Victor J. Gender equality during the transition to parenthood: A longitudinal study of dual-career couples in Singapore. Journal of Social and Personal Relationships. 2011;28(7):943–962. [Google Scholar]
  38. Rose Ka, Morgan IG, Smith W, Burlutsky G, Mitchell P, Saw S-M. Myopia, lifestyle, and schooling in students of Chinese ethnicity in Singapore and Sydney. Archives of Ophthalmology. 2008;126(4):527–30. doi: 10.1001/archopht.126.4.527. [DOI] [PubMed] [Google Scholar]
  39. Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: Is shorter better? A review and meta-analysis. Value in Health. 2011;14(8):1101–1108. doi: 10.1016/j.jval.2011.06.003. [DOI] [PubMed] [Google Scholar]
  40. Simeonsson RJ, Leonardi M, Lollar D, Bjorck-Akesson E, Hollenweger J, Martinuzzi A. Applying the international classification of functioning, disability and health (ICF) to measure childhood disability. Disability and Rehabilitation. 2003;25(11):602–610. doi: 10.1080/0963828031000137117. [DOI] [PubMed] [Google Scholar]
  41. Tan KP. Meritocracy and Elitism in a Global City: Ideological Shifts in Singapore. International Political Science Review. 2008;29(1):7–27. [Google Scholar]
  42. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology. 2007;60:34–42. doi: 10.1016/j.jclinepi.2006.03.012. [DOI] [PubMed] [Google Scholar]
  43. Stevelink SAM, van Brakel WH. The cross-cultural equivalence of participation instruments: a systematic review. Disability and Rehabilitation. 2013;35(15):1256–68. doi: 10.3109/09638288.2012.731132. [DOI] [PubMed] [Google Scholar]
  44. Stoffel A, Berg C. Spanish Translation and Validation of the Preschool Activity Card Sort. Physical & Occupational Therapy in Pediatrics. 2008;28(2):171–189. doi: 10.1080/01942630802031859. [DOI] [PubMed] [Google Scholar]
  45. Ullenhag A, Almqvist L, Granlund M, Krumlinde-Sundholm L. Cultural validity of the Children’s Assessment of Participation and Enjoyment/Preferences for Activities of Children (CAPE/PAC) Scandinavian Journal of Occupational Therapy. 2012a;19(5):428–38. doi: 10.3109/11038128.2011.631218. [DOI] [PubMed] [Google Scholar]
  46. Ullenhag A, Bult MK, Nyquist A, Ketelaar M, Jahnsen R, Krumlinde-Sundholm L, Almqvist L, et al. An international comparison of patterns of participation in leisure activities for children with and without delay in Sweden, Norway and the Netherlands. Developmental Neurorehabilitation. 2012b;15(5):369–85. doi: 10.3109/17518423.2012.694915. [DOI] [PubMed] [Google Scholar]
  47. Willis DB. Cognitive Interviewing: A “How To” Guide. 1999 Available at: URL: http://www.uiowa.edu/~c07b209/interview.pdf.
  48. Wong MNC. How preschool children learn in Hong Kong and Canada: a cross-cultural study. Early Years. 2008;28(2):115–133. [Google Scholar]
  49. World Health Organization (WHO) International Classification of Functioning, Disability and Health. Geneva, Switzerland: World Health Organization; 2001. [Google Scholar]
  50. World Federation of Occupational Therapists. Position statement on diversity and culture. 2010 Retrieved from http://www.wfot.org/ResourceCentre.aspx.

RESOURCES