Abstract
The questionnaire used for the U.S. Listening to Mothers II survey was translated and culturally adapted to measure Japanese women’s experience during the period of pregnancy planning through early postpartum. Methods included expert panels and two phases of cognitive interviews with 20 postpartum Japanese adult women. The number of problems with the translated questionnaire effectively decreased in the iterative process. Most problems were found in the question–interpretation stage of cognitive processing, such as wording/tone. Culture-specific concepts and unclear items were adapted to prevent erroneous interpretations in future studies. The future use of this questionnaire to generate data sets will be useful for professionals interested in developing evidence-based practices. The knowledge from this study can be helpful in improving health-care services and education for women with diverse languages and cultural backgrounds.
Keywords: Listening to Mothers II survey, pregnancy, childbirth, survey translation, Japanese women
The perinatal process, from pregnancy through postpartum, is a common event worldwide, yet social circumstances and culture play important roles in understanding that perinatal experience. In Japan, current social trends in perinatal experiences are surfacing that occurred in the United States 30 to 40 years ago (Varney, 1987). Women’s perinatal experiences have been increasingly medicalized. Various medical interventions are used commonly or even routinely for healthy perinatal women, including fetal ultrasound examination, episiotomy, and labor induction and augmentation (Sakala & Corry, 2008), which has been so in Japan as well as in the United States. Cesarean surgery rates in Japan increased from 12.8% in 1996 to 17.4% in 2005 (Mothers’ & Children’s Health and Welfare Association, 2007). Increased litigation further reduces the number of available health-care providers in Japan because providers decide not to deliver obstetric services. A severe shortage of perinatal health-care providers limits the availability and accessibility of health care for expectant mothers in Japan. As a result, Japanese women face unprecedented anxiety about the perinatal experience and perinatal health care for themselves.
In the United States, the first and second Listening to Mothers (LTM) national surveys, which examined women’s perinatal experiences, were carefully developed and implemented in 2002 and 2006, respectively, by the collaboration of Childbirth Connection, LTM National Advisory Council, Boston University School of Public Health, and Harris Interactive (Declercq, Sakala, Corry, & Applebaum, 2006; Declercq, Sakala, Corry, Applebaum, & Risher, 2002). With more than 250 questions, the U.S. LTM examined women’s experiences, beliefs, attitudes, preferences, and knowledge from pregnancy planning through postpartum. The information covered by this survey cannot be obtained from medical records or national census data in the United States. Based on the survey data, the actual experiences of mothers and their infants in the United States were compared to women’s preferences, care to which they were legally entitled, care supported by best evidence, and optimal outcomes. The information and recommendations based on the evidence have been cited by various researchers and health-care providers as rationale to promote maternity policy, practice, education, and research (Declercq et al., 2006). Furthermore, socially significant information from the survey has increased people’s awareness and stimulated interdisciplinary discussions about quality of maternal health-care practices across health professionals and lay people in the United States (C. Sakala, personal communication, March 22, 2007). The U.S. questionnaires and survey reports are publicly available from the Childbirth Connection Web site (www.childbirthconnection.org/).
To date, in Japan, surveys have not captured the perspectives of Japanese women about their experiences related to the rapid changes in prenatal care. This article describes the development of a Japanese version of the second LTM (J-LTM-II) questionnaire to measure the perspectives of Japanese women on effective and socioculturally appropriate perinatal health care. Future studies using the J-LTM-II will contribute to a better understanding of perinatal women’s experiences and to the improvement of evidence-based practice in Japan. The J-LTM-II is the first cross-national translation and cultural adaptation of the U.S. LTM-I or the LTM-II questionnaire. The source language was English, and the target language was Japanese. The perinatal period was defined as from pregnancy through postpartum.
In presenting this tool development process, we encourage further translation of the U.S. LTM-II questionnaire into additional languages to enable comparisons of women’s perinatal experiences across languages and cultures. An evaluation tool that is comparable across multiple nations and cultures is useful to explore universal and culture-specific characteristics of childbearing women’s experiences, needs, and existing maternal health-care practices worldwide.
An evaluation tool that is comparable across multiple nations and cultures is useful to explore universal and culture-specific characteristics of childbearing women’s experiences, needs, and existing maternal health-care practices worldwide.
METHODOLOGICAL ISSUES
Differences in U.S. and Japanese health-care practices and cultures are a challenge when developing questionnaires for conducting valid cross-national comparisons (Smith, 2004). Nevertheless, adaptation of the LTM-II enhances cultural context suitability (Harkness, Van de Vijver, & Johnson, 2003).
The unique requirement for translated survey questionnaires is equivalence (Smith, 2004). Equivalence refers to “the degree to which survey measures or questions are able to assess identical phenomena across two or more cultures” (Johnson, 2003, p. 351). Johnson (2003) classified 54 types of equivalence into two broad categories: interpretive and procedural. Interpretive equivalence examines the extent to which concepts are interpreted similarly across cultures, focusing on meaning (Johnson, 1998). Procedural equivalence concerns the measures and procedures that are used to make cross-cultural comparisons (Johnson, 1998). Although both interpretive and procedural equivalences are important, interpretive equivalence should be concerned with avoiding “imposed etic,” in which a culture-specific concept is wrongly regarded as culturally universal.
In our study, we used a pretesting process to explore the translated questionnaire. The U.S. Census Bureau (2005) stresses that pretesting is very important when a questionnaire is translated and/or when new questions are added to an existing questionnaire. An unverified questionnaire can mislead respondents during data collection and can produce ambiguous or erroneous interpretations in the analysis (Braun, 2003; DeMaio, Bates, Willimack, & Ingold, 2006; U.S. Census Bureau, 2005). Pretesting helps identify culture-specific concepts and unclear items, thus preventing errors (U.S. Census Bureau, 2005). The adaptation process, including the rationale for changes and remaining problems, should be fully documented to facilitate future appropriate interpretation of the instrument and promote more cross-national comparisons of women’s perinatal experience (Harkness, 2003). However, to date, “most cross-national studies failed to devote adequate time and resources to pretesting” (Smith, 2004, p. 451).
Perfect equivalence between the original and translated questionnaires was not the goal of our study; rather, the goal was that the J-LTM-II would be the most feasible and appropriate for use with Japanese women. To shorten the length of the questionnaire, the J-LTM-II targeted the period from pregnancy planning through the early postpartum discharge from the hospital (i.e., 10 days after birth). Therefore, only early postpartum questions from the U.S. LTM surveys were translated for this initial Japanese survey, including some of the original question formats from LTM-I and LTM-postpartum that are missing from LTM-II. The judgment to add, delete, and change were made according to the previously developed priority criteria. The highest priority was given to questions asking (a) information directly related to women’s health and lives that can be obtained only from women and (b) information particularly needed to improve primary health care for perinatal women. The Maternal and Child Health Handbook is a national standard handbook distributed to all pregnant women in Japan. This handbook is regarded as having the same value as a medical record for pregnancy and childbirth process. Together, women and health-care providers chart in the handbook at each perinatal visit and childbirth to share the data. Some questions were added to the J-LTM-II to be answered with data from the handbook, such as what was the rough amount of bleeding at birth.
The data for the U.S. LTM-II were collected via the Internet or telephone. In Japan, the penetration rate of the Internet and computer literacy among mothers are unknown. In general, many Japanese people use the Internet with mobile phones rather than home computers. Additionally, Japanese mothers’ preferences toward telephone or Internet surveys were unknown when our study was conducted. Therefore, although our study used in-person interviews, the final method for data collection in future national studies was explored with study participants during this phase of instrument development. The J-LTM-II was initially developed as a paper-and-pencil, self-completion format. A unique name for the J-LTM-II questionnaire will be given.
METHODS
Study Participants
Professional expert panel.
Eight Japanese health professionals (two obstetricians, four nurse–midwives, and two maternal nurses) were recruited to critique the J-LTM-II Version 1 for content validity and clarity of each item (Step I) before pretesting with women.
Lay expert panel.
Two groups of lay women were recruited. A group of 15 early-postpartum women who gave birth at a rural hospital in the 10 days preceding the interviews were recruited in-person by the primary investigator at the city hospital labor and birth ward at the time of the survey (Step II-1 and Step II-2). Another group of five women who gave birth in a hospital/clinic in an urban city in the past 2 years were recruited by snowball sampling (Step II-3). The inclusion criteria were consistent with the sample selection from the original LTM-II: women between the ages of 18 and 45 years old; birth of a single baby; baby is living at the time of interview; willingness to participate in the study; and ability to read, understand, and communicate in the English language (Japanese language, in the case of the J-LTM-II).
Overall Process
The overall steps for translation and cultural adaptation followed the U.S. Census Bureau’s (2005) “Language Translation Guidelines” and are summarized in Table 1. The methodological design of our study employed both qualitative and quantitative approaches. Methods to develop the J-LTM-II questionnaire included a review of the literature published in English and Japanese, use of multiple professional expert panels in the United States and Japan, translation by a bilingual committee, and two phases of cognitive interviews with postpartum Japanese women. The study protocol was reviewed and approved under expedited review procedures by the institutional review boards at the University of Illinois at Chicago and at a city hospital in Japan.
Table 1. Overall Steps for Translation and Cultural Adaptation.
| Step [Version] | Review Activity |
| Preparation [Original] | Translation, comparison among the three translators |
| Step I [Version 1] | Critique by eight Japanese health professionals |
| Step II-1 [Version 2] | Cognitive interviews with 10 Japanese early-postpartum women |
| Step II-2 [Version 3] | Cognitive interviews with five Japanese early-postpartum women |
| Step II-3 [Version 4] | Cognitive interviews with five Japanese later-postpartum women |
| Step III [Version 5] | Adjudication, final documentation |
Data Collection and Analysis
Parallel committee translation.
Three translators (two hired translators and the primary investigator) performed a parallel committee translation (PCT). A PCT approach was selected because it achieves informativeness, transparency, and security of translation and, although expensive, it saves time (Behling & Law, 2000; Smith, 2004). All three translators were native speakers of the Japanese language. Translator A was a Japanese obstetrician who attended high school in the United States and recently gave birth in the United States. She also has a master’s degree in maternal–child health epidemiology. Translator B was a Japanese professional translator, whose published translation work focused on women’s lives. She gave birth to four children and resides in Japan. The primary investigator was Japanese, a researcher, and a nurse–midwife who has lived in the United States for 5 years while pursuing doctoral education. Translators A and B were provided necessary information regarding the translation of LTM-II to facilitate appropriate translation. The three translators discussed each discrepancy at word, sentence, and overall communicative effect levels to reach agreement (U.S. Census Bureau, 2005). For unresolved problems, the primary investigator consulted with another Japanese translation reviewer who had extensive cross-cultural research experiences and translations and was identified by the director of the U.S. LTM project. The developer of the U.S. LTM-II was also consulted. The consensus translation of J-LTM-II Version 1 was evaluated by Translator A, and the final version of J-LTM-II Version 5 was evaluated by Translator B and the translation reviewer, according to the adopted criteria for “achieving a good translation” (U.S. Census Bureau, 2005).
Content validity index.
Content validity was examined with the Content Validity Index (Grant & Davis, 1997; Lynn, 1986; Polit, Beck, & Owen, 2007). The Japanese experts were asked whether the J-LTM-II Version 1 questionnaire captured Japanese women’s perinatal experiences and the related social context, and whether the wording of each question, format of the questionnaire, and wording of directions were appropriate for the target population. According to Polit et al.’s (2007) criteria, items rated as “1: very relevant” or “2: relevant with minor revision” by more than two of the eight reviewers are regarded as “good” relevance for the content validity. Clarity was scored by asking whether or not each item was easy to answer. Based on the reviewers’ evaluation and comments, the J-LTM-II Version 1 was revised to Version 2 for the next step: cognitive interviews with women.
Cognitive testing.
Three types of cognitive testing with two groups of Japanese women were conducted to improve the content validity and equivalence of the questionnaire. The three types of techniques included the think-aloud technique, probing questions, and debriefing (Beatty, 2004; Beatty & Willis, 2007; Willis, 2004). Cognitive testing refers to
the practice of administering a survey questionnaire while collecting additional verbal information about the survey responses; this additional information is used to evaluate the quality of the response or to help determine whether the question is generating the sort of information that its author intends. (Beatty, 2004, p. 45)
Cognitive testing assesses how people interpret and comprehend the items, recall information and events, make decisions in answering a questionnaire, and finally provide responses (Beatty & Willis, 2007; Harris-Kojetin, Fowler, Brown, Schnaier, & Sweeney, 1999; Tourangeau, 1984). Moreover, emotional reactions such as frustration or comfort level also can be observed during the interviews.
The participant women were asked to bring the Maternal and Child Health Handbook to the interviews as a memory cue. The handbook contains important information about women’s perinatal experience, such as how often they had prenatal visits; how many times, when, and where they had childbirth education; and the exact number of minutes for length of labor. Women might have forgotten some detailed information about their perinatal experience, especially about pregnancy among women interviewed 2 years after childbirth.
For a concurrent, unguided, think-aloud technique, the participants were asked to answer the J-LTM-II questions. During the interview, a woman was asked to speak aloud about what came to her mind as she answered each question. The interviewer used immediate retrospective probing, both scripted and spontaneous, for what each question meant and whether the woman had suggestions for better wording (Beatty & Willis, 2007). When a participant finished answering all of the questions, she was debriefed. The debriefing consisted of questions to ascertain her overall impressions about the questionnaire. All 20 interviews were conducted by the primary investigator in a private room. Interviews were tape-recorded. At the completion of the interview, a book for children was given to each participant.
During interviews, the interviewer took field notes for nonverbal data regarding women’s reactions to the questionnaire to note possible indicators of problems, such as procedural errors in answering, hesitancies, and facial expressions. When the first 10 interviews with the J-LTM-II Version 2 were completed, revisions were made based on women’s responses. The revised J-LTM-II Version 3 was used with the next five participants and was evaluated by the primary investigator to determine whether problematic items decreased. After the entire 15 interviews were conducted at a rural hospital in Japan, an additional five Japanese mothers who had given birth within the past 24 months were interviewed in Tokyo with the J-LTM-II Version. This step checked whether women still remembered their perinatal experience long after the early postpartum period and whether there were differences in cognitions between women from rural and urban areas.
Data from the 20 cognitive interviews were analyzed according to the four stages proposed by Tourangeau (1984): question interpretation, memory retrieval, judgment formation, and response editing (DeMaio & Landreth, 2004; Knafl et al., 2007; Sudman, Bradburn, & Schwarz, 1996; Tourangeau, 1984). When a problem was found, the primary investigator identified the types and causes of the problem, resolved the problem, and documented the decision-making process. Final decisions were made by the primary investigator as the translation adjudicator. The survey had items that asked women about their income and the amount of expenses for perinatal care. The results are reported in yen and the corresponding U.S. dollars for this English-language report (based on February 2009 exchange rate, U.S. $1 = 90 yen).
RESULTS
Demographics of Women
The 15 early postpartum women were interviewed on average at 4 days postpartum (range: 1–7 days), and the five women who gave birth within the last 2 years in Tokyo averaged 586 days postpartum (range: 572–610 days). Overall, the 20 women represented various socioeconomic backgrounds: ages (mean: 30.9 years, range: 21–45 years); parity (mean: 1.7 times, range: 1–7); education level (ranged from junior high school to undergraduate); income level (range: 2 million–4 million yen, or U.S. $22,222–$44,444, to more than 15 million yen, or U.S. $166,667, per household per year); married (95%) and single (5%); employed (75%) and unemployed (25%) during pregnancy; baby’s birth weight (mean: 2,970 g, range: 2,296–3,452 g); and gestational weeks at birth (mean: 39.0 weeks, range: 36–41 weeks). Only one of the 20 women had cesarean surgery, and none of the women had pain medication for labor pain. Interviews took an average of 64 minutes (range: 42–90 minutes).
Evaluation by Translators
Translator A and Translator B evaluated the first and final version of the translated questionnaire, respectively. Overall, both the J-LTM-II Version 1 and J-LTM-II Version 5 maintained a similar style and manner, a compatible degree of formality, and a compatible degree of literacy level compared to the original LTM-II, and produced a similar response (similar communicative effect; U.S. Census Bureau, 2005).
The received comments and suggestions from the translators were incorporated into the final revision. The translators paid careful attention to the tone and expressions of translated text, recognizing that perinatal experience is a sensitive and private issue for women. The translated text was modeled on the degree of politeness of the language used by the media, other similar questionnaires, and professional explanations for women. Regarding concerns about cultural sensitivity, the translators agreed that these concerns had been dealt with successfully. For example, bold or blunt expressions in English (“Is your baby alive?”) were strictly avoided to be culturally acceptable to Japanese women. On the other hand, words describing emotional states for the question “How did you feel while giving birth?” were particularly difficult to maintain equivalence in the translation because these words had more than one meaning. For example, the word “alert” can mean “tense/nervous” (somewhat negative) or “aware” (positive). The word “agitated” can be translated as either “confused/panic” or “annoyed/irritated.” Grammatical differences between English and Japanese, such as in word order, also had to be addressed. Some questions had to be separated into two independent questions when translated, so that the translated texts retained natural grammar in the Japanese language. As a result of repetitive description, the translated Japanese texts might become tedious. In addition, because there are many levels of politeness in the Japanese language, the translators reported that the more they described a term precisely, the less the translated text made its point. Therefore, politeness and precision were offset by loss of succinctness.
Evaluation by Japanese Professional Expert Panel
Following translation by the committee, the J-LTM-II Version 1 was evaluated by the eight Japanese health professionals. As a result, 121 of 150 items were regarded as “fitting the Japanese woman’s perinatal experience and social context,” 20 items received full scores of “very relevant” by all eight professionals, and 22 items were evaluated as “not clear” by two or more professionals. Rich, qualitative responses were provided by the evaluators. Their concerns included the following suggestions: decrease length of the survey, change medical terms to more lay terms, add questions to comprehensively measure Japanese women’s experiences, and use more natural wordings.
Revisions Made From the J-LTM-II Version 1 to Version 5
Based on the feedback from Japanese professionals and mothers, the number of modifications, additions, and deletions decreased from Version 1 to Version 4 (see Figure 1). Revisions from the original to Version 1 included the addition of 24 questions and the deletion of two questions. Revisions from Version 1 to Version 2 included the addition of 30 questions and the deletion of 10 questions. As a result, the number of questions increased by 42 in Version 2, before starting cognitive interviewing with women. The deletions and additions were made in layers as the pretesting continued; many of the deletions and additions were based in part on the findings from the cognitive interviews with Japanese women. At this stage, many suggestions by the Japanese health professionals were considered.
Figure 1.

Number of changes made in the J-LTM-II questionnaire.
Besides deleting and adding questions, minor modifications were made, most of which were rewordings. Types of problems found are summarized in Table 2. A total of 339 problems were found. Most (70%) of the problems were in the question interpretation stage of the cognitive process. Moreover, about half of the interpretation problems were caused by wording/tone. Although the translators endeavored to produce natural wording and reader-friendly translation, minor concerns about naturalness of wording persisted throughout the process of translation and adaptation.
Table 2. Types of Problems Found During the Development of the J-LTM-II*.
| Stage of Cognition Process | Type of Problem | Frequency (%) | Number of Instances |
| I. Question interpretation | 238 (70%) | ||
| Wording/Tone | 128 | ||
| Abstract level | 28 | ||
| Unclear perspective | 26 | ||
| Unclear reference | 22 | ||
| Emic/Etic concept | 13 | ||
| Comprehension difficulty | 13 | ||
| Other problems | 5 | ||
| Other | 3 | ||
| II. Memory retrieval | 25 (7%) | ||
| Accessibility | 9 | ||
| Episodic/Semantic | 7 | ||
| Memory cues | 7 | ||
| Anchoring | 2 | ||
| III. Judgment formation | 60 (18%) | ||
| Applicability of options | 44 | ||
| Response style | 16 | ||
| IV. Response editing | 16 (5%) | ||
| Unclear response format | 11 | ||
| Self-presentation | 2 | ||
| Social desirability | 1 | ||
| Interview effects | 1 | ||
| Other | 1 | ||
| Total | 339 (100%) |
Note. *J-LTM-II = a Japanese version of the questionnaire used for the U.S. Listening to Mothers II survey.
Wording/Tone
Problems with wording or tone were further classified into seven types: lengthy/repetitive (41% of the 128 wording/tone problems); unnatural (28%); medical term/jargon (18%); too formal/academic (5%); biased impression (5%); intrusive (2%); and inconsistent (2%).
Regarding the lengthy wording, many Japanese postpartum women mentioned that “it was hard to read everything. It is helpful when you [interviewer] read out loud for me.” It is a cultural belief in Japan that, although no research-based support exists, in general, postpartum women’s eyes tire easily and they should not use their eyes too much. After the first 10 interviews with women, extensive editing was used to shorten each item while maintaining the meaning that was validated in the following rounds.
Translation of medical terms required the most consideration. Many Japanese medical terms and concepts have been imported from Germany and the United States over the last century (Eto et al., 2002). The awkwardness of medical terms was a major drawback. For example, asking the mode of birth, whether vaginally or by cesarean, was confusing; most women did not understand the term vaginal birth, or keichitsu-bunben, and quite a few women stated that “because mine was not a cesarean section, I chose another option [vaginal].” Japanese women are familiar with the term normal birth, or futsu-bunben or even seijou-bunben. However, a woman who had a vacuum-assisted birth also mentioned, “Whenever I am asked this question, I am at a loss which to choose because vacuum-assisted birth is classified as ‘abnormal’ in the Maternal Handbook.” Most women had heard of vacuum-assisted birth, whereas few women understood the term forceps. As a result, the J-LTM-II combined this question with the subsequent elaborating questions (whether vaginally with or without assistance by vacuum or forceps) with the pictures of vacuum- and forceps-assisted birth.
Pictures were often used in the J-LTM-II to address medical terms. For example, women were asked which positions they used to push the baby out: lying on their back; lying on their side; upright (for example, “propped up,” squatting, or sitting); propped up (semisitting); on their hands and knees; or some other positions. The distinction between “lying on their back” and “propped up/semisitting” is important because lying on one’s back can cause pregnancy-induced low blood pressure, whereas semisitting can avoid it. The translation, however, was difficult with available Japanese words. The traditional Japanese sitting position is sitting square on the floor without reclining against a chair or the back of a chair. In fact, without the use of pictures, all women whose position was semisitting had first answered “lying on their back.” When they saw the pictures of all the various positions, they found their actual position “semisitting” and, consequently, changed their answers.
Pictures were often used in the J-LTM-II to address medical terms.
Another difficult medical term was electronic fetal monitoring (EFM). The original LTM-II asked the type of EFM, either internal or external or both. Cognitive interviews with Japanese women revealed that many women did not know about internal EFM unless they had experienced it (no woman in this study had). To add to the lack of understanding, it was found from the cognitive interviews that, because the women had two sensors with belts on their belly when they had an external EFM, some women thought one sensor externally monitored contractions and another sensor internally monitored fetal heartbeat. Thus, they answered “both internal and external” (whereas, it was actually external only). Because internal monitoring is not a common practice in Japan and valid answers regarding internal monitoring are difficult to obtain, the “internal/external” question was deleted. Moreover, a picture with two belts on a woman’s belly, as many women commonly explained EFM, was added to explain EFM.
Ambiguous Questions
Some ambiguous questions were difficult to translate because they were not specific enough for uniform interpretation. For example, “walking during labor” can be interpreted as either “walked only when necessary such as going to the toilet” or “walked and moved around actively as much as possible to progress labor.” Because women’s experiences differ, including the intention and consequence of walking, it is difficult to capture the reasons for these differences. In the J-LTM-II, the content of walking was further explored by asking their purposes of walking.
Responses without a uniform definition were also perceived as ambiguous. It was not clear how equally the sets of response categories such as “not at all,” “a little bit,” “moderate,” “quite a bit,” and “extremely” were defined across languages (Smith, 2004; Willis & Zahnd, 2007). The use of a visual aid, such as a histogram with 0% to 100% corresponding to each response category with equal intervals, is suggested for future studies.
Unclear Perspective
Problems of unclear perspective were found when a question contained multiple contexts. In most cases, further questions were added to elaborate the context. For example, when women were asked whether they drank/ate/walked around after they were admitted to the hospital and contractions became regular and well-established, many women answered “no” because they “did not want to eat because of pain” or were “not hungry” despite their providers’ recommendations. Importantly, the intention of these questions was to clarify whether there were restrictions during labor intended by the health provider or the birthing woman. Did the woman “not walk/eat/drink because she decided not to” or “not walk/eat/drink because she was told by her provider that she should not”? As a result, subsequent questions—“Did your caregiver recommend that you should do the following during your labor: Walking? Eating? Drinking?”—were added in the Japanese version.
Unclear Reference
Some questions were unclear for the women; they had a difficult time understanding the question. For example, the question “To the best of your recollection, what was the date that your baby was due?” was unclear for women who had a planned cesarean surgery date and had a “due” date scheduled. As a result, the text “(i.e., 40th week and 0 day)” was inserted as a response to this item.
Cultural-Specific Concepts
Some U.S. concepts simply do not exist in Japanese culture. In turn, some Japanese customs are unique compared to those in the United States. For example, the custom of male circumcision is unusual in Japan. The custom of satogaeri, in which women spend the birth and postpartum period with their parents, and the custom of breast massage provided by nurse–midwives are traditional practices in Japan. These topics were deleted or added in the revision process.
Doula-like support.
Many customs and ideas exist in both U.S. and Japanese cultures, although the terms used to represent them differ. For example, the original LTM-II has questions about birth doulas, usually a trained lay woman, or as a new paraprofessional, who supports perinatal women in various nonmedical ways. Because there is no doula movement in Japan, questions and the term doula were all deleted from the response categories in the J-LTM-II. However, social support has been and should be provided by someone, even though such a role as the doula has not developed in Japan. In particular, because pain medication is not used for labor pain in the Japanese culture, approaches for supporting women can play an important role for Japanese women coping with labor.
Some U.S. concepts simply do not exist in Japanese culture. In turn, some Japanese customs are unique compared to those in the United States.
Considering that Japanese women do not have a clear idea about doula support, the concept was broken down so that they could reflect on their birthing experiences in terms of who or how support might have been provided for them. Therefore, translating “make them more comfortable physically, provide emotional support, and provide information” was a relevant question, as was “stays with a woman throughout labor and birth.” The original questions were translated into seven specific questions: “explained”; “encouraged”; “praised”; “accompanied all the time so that you did not feel lonely”; “touched as much as you needed (rubbed, held hands, supported positions)”; “took care of your chores”; and “stood by you all the time.” The seven separate questions were based on the primary investigator’s concept analysis of doula support and the theoretical framework of the J-LTM-II. Two additional questions were added to examine who, if anyone, provided such support for Japanese women during the prenatal period: “During your pregnancy, did you have an experienced woman available to you who shared her perinatal experience and supported you closely?” and “Who was/were the women?”
Although obstetricians and nurse–midwives have important roles as prenatal health-care providers both in Japan and in the United States, their roles in many hospitals/clinics in Japan differ from their roles in the United States. In Japan, obstetricians and nurse–midwives (and/or maternity nurses) work together to provide perinatal care, even though nurse–midwives have a legal right to own a midwifery clinic. For the question “Once you became pregnant, which type of caregiver was most directly involved with providing your prenatal care?” Japanese women had difficulty choosing one type of provider, saying, for example, “I cannot narrow to only one. The obstetrician examined me and my baby, while the nurse–midwife took a longer time to listen to and educate me.” Others responded, “The obstetrician saw my body, and the midwife saw my mind” and “They are both primary providers, just their roles are different.” As a result, the J-LTM-II allowed multiple answers to this question.
The original LTM-II included items regarding elective cesarean surgery (Declercq et al., 2006; Sakala, 2006). However, the survey’s results revealed that only one U.S. woman decided on her own to have a cesarean surgery without medical indications, whereas more women needed to know the potential complications associated with cesarean surgery (Declercq et al., 2006; Sakala, 2006). Although elective cesarean surgery is not a social trend in Japan, medical interventions in prenatal care is the norm. More options for interventions without medical indication, such as epidural use, are becoming available for Japanese women. Full and appropriate information is necessary to guide their decision making when they are presented with new options in childbirth. While minimizing the questions on elective cesarean surgery, the J-LTM-II added questions to explore Japanese women’s beliefs and preferences: “Do you think that a woman should go through labor and birth without pain medication?” and “If you were pregnant in the future, had no medical reasons, and could decide for yourself, how likely would you be to want to have medication to alleviate labor pain?”
Malpractice and litigation.
In the original U.S.-based questionnaire with multiple-answer options, it was asked whether or not the current malpractice system causes providers of maternity care to take more protective actions such as “to order unnecessary prenatal tests to avoid being sued” and “to charge more money to cover their malpractice insurance costs.” These situations do not represent the current climate in Japan. When questioned, Japanese respondents looked confused and tired in thinking about this question after the intensive interviews. After this question was changed to an open-ended question, women provided rich thoughts in their words. It was found that women were very concerned about the current social situation related to the perinatal health-care system in Japan. Furthermore, the main reason many women enthusiastically participated in our study was not only that they enjoyed sharing their experiences, but also that they wanted to contribute to social change. Five of the 20 women even spontaneously mentioned that incentives for study participation were not necessary, as long as their participation could be helpful for future planning of health-care services for pregnant women.
Memory Retrieval
Seven percent of the problems with the translated questionnaire concerned memory retrieval, including accessibility, episodic/semantic, memory cues, and anchoring. For example, a woman who had cesarean surgery did not have access to the information needed to answer the question “During the first hour after birth, where was your baby primarily?” because she was unconscious during the first hour, having her abdomen sutured under general anesthesia after birth of the baby.
The Maternal and Child Health Handbook was identified as an important memory cue. Overall, women within 2 years after birth remembered their experiences in detail, as did early-postpartum women.
Response Styles
Cross-cultural survey literature found that Asians, particularly Japanese respondents, tend to avoid extreme values, whereas Americans do not (Chen, Lee, & Stevenson, 1995; Declercq et al., 2006; Willis & Zahnd, 2007). Survey responses reflected this tendency. For example, for the question “Overall, how would you rate your baby’s health from birth until now?” only one woman out of the first 10 chose “excellent,” whereas 75% of the U.S. women had answered “excellent” for this question (Declercq et al., 2006). Some of the reasons women gave for not answering “excellent” were “because I don’t know everything about my baby,” “I am not sure of something,” or “I avoided ‘excellent’ for no particular reason.” Because of differences in cultural response styles, a problem arises when Japanese babies are mistakenly regarded as less healthy than U.S. babies. Differences in cultural response styles created systematic biases that could compromise the cross-cultural comparison of data. One solution to extreme response style is to alleviate the extremes (Smith, 2004). As a result, when the “excellent” option was changed to “very healthy” in the second half of the sample, seven of 10 Japanese women chose “very healthy.”
Differences in the cultural response style were also implied in other questions: “Overall, how satisfied are you with the health care?” and “Overall, how satisfied are you with the maternity health care you have received?” Surprisingly, none of the 20 Japanese women answered “very satisfied,” whereas 17% and 35% of U.S. women, respectively, rated them as “excellent” (Declercq et al., 2006). The dominant reason for the low satisfaction with the Japanese maternity health care was that “medical insurance cannot be used for perinatal care.” However, even with equal health insurance, most Japanese women were “somewhat” or “not at all” satisfied with health care in Japan, saying that “it was too crowded” and “cannot find good doctors.” Considering that the Japanese health-care system is regarded as superior to many countries in the world (World Health Organization, 2000), including that of the United States, there may again be a potential bias because of cultural response styles. When women were asked to evaluate the health-care system in Japan, some avoided clear answers, saying that they did not know everything about the national characteristics of the health-care system or had not compared it with other countries. However, they were able to answer questions about the care they actually received. These findings may reflect Japanese women’s perfectionist tendency in which they often avoid making a clear, positive evaluation unless they find it perfect with enough evidence to support that it is perfect, regarding one flaw as sufficient to indicate imperfection.
Another explanation for the difference in the cultural response style may be natural reserve/humility, a cultural virtue in Japan. For the question “As you neared the end of your pregnancy, how confident were you that you could recognize the signs or symptoms of premature labor?” only one each out of the first 10 women, all of whom gave birth at term, answered “very confident” and “somewhat confident,” respectively. Half of the women answered negatively: “not very confident.” For this question, 87% of U.S. women, of whom 94% gave birth at term, had answered “very confident” (Declercq et al., 2006). One solution for these response biases may be triangulation of data (i.e., to support the subjective data with objective data such as the premature birth rate) when the data are interpreted. Further cognitive interviews are necessary to analyze the mechanism of the cultural response styles that can lead to systematic biases in cross-national surveys.
Appropriate Use of the J-LTM-II for Japanese Women
Debriefing of women revealed that the length of the questionnaire was a major problem. In particular, women were not happy with the tedious task of reading such a long instrument. Women’s reading speed and perceptions of burden varied, and it is anticipated that this will also be true for providing written responses. Computer-assisted online survey and telephone interviews were used in the original LTM-II, and the use of online surveys is spreading in Japan (Matsumoto, 2007). However, our study found that fewer Japanese women used the Internet during their pregnancy, particularly in rural areas; this will have implications for self-administered online surveys. Notably, most women refused the option of a phone interview because of concerns about loss of confidentiality, discomfort, and tasks associated with being at home with their babies. Such factors interfere with extended conversations.
All women preferred in-person interviews to the other modes of data collection, saying that “I can ask about unclear questions”; “I can remember experiences in speaking about them”; and “I feel secure in participating if you see me, guarantee [her rights in research participation and confidentiality issues in the informed consent process], and I know who you are.” Many women enjoyed being listened to, saying, “I wanted to talk about how hard and how well I did in my childbirth, but usually there is no opportunity to talk.”
Some women were further asked who would be their preferred interviewer. Female interviewers were strongly preferred. The women preferred interviewers with no generation gap and with nonjudgmental attitudes, saying that “older women may scold us, rather than listen to us.” Women who have perinatal experiences and/or expertise in perinatal health care were preferred, but not essential. One woman noted:
If the interviewer has not given childbirth, it is okay, I can talk as if I am teaching them what it is like. First meeting is okay, too. Anyone who really wants to know and wants to contribute to the improvement of the health care can be an interviewer.
The women answered that there were no uncomfortable questions in the questionnaire, such as offensive or depressing questions. However, some hesitation was seen with a question asking the women’s household income levels, especially among women with relatively low incomes. According to Smith’s (2004) study, Americans are more open to supplying personal income information. Based on the advice of the women surveyed, the option “decline to answer” was moved to the top of response categories from the end, and “if you would not mind” was added at the beginning of the question. After that, a woman was able to refuse to answer the question. Some hypothetical concerns were suggested: “If you did not have a good birth experience, participation may be tough for you” and “I wonder if, in person, some women may not answer with the truth, such as questions about smoking during pregnancy.”
DISCUSSION
Participatory Team Approach
The team approach used in our study provided more alternative options and achieved balanced decision making in the translation and adaptation of the LTM-II, whereas the primary investigator consistently coordinated and was engaged in every process (Forsyth, Kudela, Levin, Lawrence, & Willis, 2007; Harkness, 2003). In particular, the three translators and the translator reviewers provided the mix of skills and insights necessary to accomplish a difficult but rewarding task. Bilingual translators should not be mere technical assistants but should serve as experts in the languages and cultures, as an integral part of the research process (Behling & Law, 2000; Forsyth et al., 2007; Smith, 2004). The translators enthusiastically contributed their cultural and linguistic knowledge as well as their personal experiences as professional or lay women who had perinatal experiences. Immediate discussion with the original developer of LTM-II clarified translation difficulties and unclear questions. The U.S. dissertation committee members and Japanese health professionals also served as expert panels. Above all, Japanese women’s voices were fully incorporated into the J-LTM-II questionnaire, as if they were research team members rather than research subjects, and they shared their experience in a comfortable and valid manner. This participatory process is an essential criterion in planning, conducting, and evaluating primary health care.
Translation
Although the translators agreed that the final translated text was as appropriate and natural as they could make it, some women in the second half of the cognitive interviews stated that some translated text was understandable but anchored in English grammar and, therefore, somewhat unnatural. When the meanings of the original English language version were difficult to understand (e.g., unclear reference, double-barreled question), the translated items had similar problems (Smith, 2004; Van de Vijver, 2003).
There may be more unrecognized translation problems in the J-LTM-II instrument. For example, a cross-cultural study found that depression is recognized more often as a mood state by Japanese people, whereas it is characterized as being more somatic by Americans (Tanaka-Matsumi & Marsella, 1976; Van de Vijver, 2003). Another study found that the Japanese word for anxious is more unpleasant and passive than the English concept (Romney, Moore, & Rusch, 1997). However, study findings support the overall stability of the semantic structure of various emotional terms between English and Japanese, such as embarrass, anxious, excitement, happy, fear, tired, sad, and lonely (Romney et al., 1997; Rusch, 2004). Comparisons of cognitive interviews with both U.S. and Japanese women for the emotional terms in the LTM-II clarified most of the commonalities and differences in their cognitions.
Sampling
Cognitive interviewing is qualitative research (Beatty & Willis, 2007). Although appropriate sample size in cognitive interviewing has not yet been established, from eight to 30 interviews are often recommended or conducted in other similar studies (Forsyth et al., 2007; Willis, 2006; Willis & Zahnd, 2007). Our study conducted 20 cognitive interviews with women with various backgrounds, within which the number of problems effectively decreased as the questionnaire was revised (Beatty & Willis, 2007).
On the other hand, because of the small sample size and complex skip patterns in the LTM-II questionnaire, the understanding of the questions’ validity and equivalence relating to rare experiences were not assessed in depth with all study participants; examples include questions about administering epidurals for pain management, vaginal birth after cesarean, and cesarean surgery. None of the women in our study were “childbirth refugees,” a current social problem in Japan, referring to pregnant women who cannot find any perinatal provider because of the shortage of obstetricians and maternal health-care facilities. Considering that the Japanese population is rather homogeneous, the selection bias limitation in this sample of 20 women was not regarded as critical. The sample was reasonable for this study.
Notably, women whose perinatal experiences included cesarean surgery or a birth with no prenatal care were recruited but tended to decline to participate. In Japan, where natural birth with regular perinatal care is the norm, social stigma and potential negative judgment by professionals are a possibility for women who had cesarean surgery or lacked prenatal care, and this might have hindered their participation. Moreover, women who had a psychiatric disease within 5 years or who could not read or write the Japanese language were excluded in our study because of the emphasis in the developmental stage of the tool. Eventually, more recruitment and inclusion of minority women is important to improve health care for underserved populations in Japan, while maintaining voluntary participation of women in research.
In-Person Interviews
All 20 women preferred an in-person interview to the other modes of data collection. Women’s reading and writing burden and fatigue can be minimized, and data collection errors and missing data can be decreased through conversations between a woman and an interviewer; thus, validity and reliability of the data can be enhanced. On the other hand, challenges of in-person interviews, such as cost and interviewer effect bias, will need to be addressed (Braun, 2003; Rasinski, Willis, Baldwin, Yeh, & Lee, 1999; Smith, 2004). Possible respondent burden caused by the length of the survey still needs to be addressed. It was one of the reasons that the U.S. LTM chose an online survey that a mother could complete in several stages rather than having to complete it in a single sitting (C. Sakala, personal communication, May 2, 2008).
For in-person interviews, the woman–interviewer relationship can affect the quality of data obtained (Smith, 2004). A nonjudgmental, women-centered attitude of interviewers is essential to improving the recruitment and maximizing the effects of women’s reflections on perinatal experiences. Strategies used in our study included the placement of more factual, closed-ended questions in the earlier part of the interview to develop rapport and a trust relationship with the woman. More open-ended questions and questions about values and knowledge were placed later in the interview, as the women felt more comfortable in sharing their honest thoughts.
CONCLUSION
Our study supports the validity of careful translation and cultural adaptation of the LTM-II questionnaire with triangulation of methods and data sources. Methodological concerns of translation and equivalence were addressed throughout. Interpretive equivalence was prioritized rather than procedural equivalence throughout the process. Systematic and random response errors were identified and addressed; as a result, they decreased, thus supporting the validity and equivalence of the translated tool. In particular, response errors were found in the question interpretation stages of the women’s cognitive processes (Willis & Zahnd, 2007). Although some questions hold culture-specific concepts, the J-LTM-II prepared alternative questions that enabled better understanding of Japanese women’s experience and allowed qualitative cross-cultural comparisons between the U.S. and Japanese versions. The findings support the use of the J-LTM-II for Japanese women from early postpartum until 2 years after birth, in rural and urban environments, and for women with varied education and income levels.
The study findings from this survey methodology research can be useful in improving health-care services and education for women with diverse languages and cultural backgrounds. Identification of miscommunication between experts and lay people and selection of appropriate wording and modes for lay people will facilitate effective and comfortable professional–client communication and improve the quality of care provided.
Finally, the full documentation of a tool development process can optimize the future use of the tool by researchers. This is especially important when a tool is translated across languages and cultures. However, published documents are limited in this research area. Instrument translation and cultural adaptation should be included in pretesting. Documentation of the development of survey methodology will facilitate appropriate use of the tool and subsequent cross-cultural comparisons that are valid and equivalent (Beatty & Willis, 2007; Braun, 2003).
ACKNOWLEDGMENTS
Our study was supported by the American Association of University Women International Doctorate Fellowship, Fumiko Yamaji Trust Academic Nursing Education and Research Grant, and Sigma Theta Tau Alpha Lambda Chapter. We also express appreciation to Dr. Carol Sakala, Ito City Hospital, and Child Research Net.
Footnotes
RIEKO KISHI initiated this study as her doctoral dissertation at University of Illinois at Chicago (UIC). In addition to her current research in doula support for perinatal women, Dr. Kishi maintains her clinical practice in nurse-midwifery at Showa University Northern Yokohama Hospital, Japan. BEVERLY McELMURRY (1937–2010) was an associate dean and professor at UIC College of Nursing. She was the director of the World Health Organization (WHO) Collaborating Center for International Nursing Development in Primary Health Care. As Dr. Kishi’s beloved mentor and dissertation advisor, it was her inspiration that guided this study. SUSAN VONDERHEID is a research assistant professor at UIC. Her interests include the organization, quality, and costs of health services for low-income ethnic minority women and their families. SUSAN ALTFELD is a clinical assistant professor in the division of Community Health Sciences at UIC School of Public Health. BARBARA McFARLIN is an assistant professor in the Department of Women, Children and Family Health Science at UIC. JUNKO TASHIRO is a professor of St. Luke’s College of Nursing, Tokyo, and is the associate director of WHO Collaborating Center for Nursing in Primary Health Care at St. Luke’s College.
REFERENCES
- Beatty P. (2004). The dynamics of cognitive interviewing In Presser S.Rothgeb J. M.Couper M. P.Lessler J. T.Martin E.Martin J.et al. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 45–66). Hoboken, NJ: Wiley [Google Scholar]
- Beatty P. C., & Willis G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311 [Google Scholar]
- Behling O., & Law K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: Sage [Google Scholar]
- Braun M. (2003). Errors in comparative survey research: An overview In Harkness J. A.Van de Vijver F. J. R., & Mohler P. (Eds.), Cross-cultural survey methods (pp. 137–142). Hoboken, NJ: Wiley [Google Scholar]
- Chen C., Lee S. Y., & Stevenson H. W. (1995). Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science, 6(3), 170–175 [Google Scholar]
- Declercq E. R., Sakala C., Corry M. P., & Applebaum S. (2006). Listening to mothers II: Report of the second national U.S. survey of women’s childbearing experiences. New York, NY: Childbirth Connection [Google Scholar]
- Declercq E. R., Sakala C., Corry M. P., Applebaum S., & Risher P. (2002). Listening to mothers: Report of the first national U.S. survey of women’s childbearing experiences. New York, NY: Maternity Center Association [Google Scholar]
- DeMaio T. J., Bates N., Willimack D., & Ingold J. (2006). Pretesting questionnaires and related materials for surveys and censuses—U.S. Census Bureau Standard: Version 1.2.
- DeMaio T. J., & Landreth A. (2004). Do different cognitive interview techniques produce different results? In Presser S.Rothgeb J. M.Couper M. P.Lessler J. T.Martin E.Martin J.et al. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 89–108). Hoboken, NJ: Wiley [Google Scholar]
- Eto H., Kishi R., Iwasaki A., Sakamoto C., Zukawa N., Aoki M., . . .Yahiro M. (2002). A study of jargons of German origin in Japanese hospitals. Bulletin of Nagano College of Nursing, 4, 31–39 [Google Scholar]
- Forsyth B. H., Kudela M. S., Levin K., Lawrence D., & Willis G. B. (2007). Methods for translating an English-language survey questionnaire on tobacco use into Mandarin, Cantonese, Korean, and Vietnamese. Field Methods, 19(3), 264–283 [Google Scholar]
- Grant J. S., & Davis L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20, 269–274 [DOI] [PubMed] [Google Scholar]
- Harkness J. A. (2003). Questionnaire translation In Harkness J. A.Van de Vijver F. J. R., & Mohler P. (Eds.), Cross-cultural survey methods (pp. 35–56). Hoboken, NJ: Wiley [Google Scholar]
- Harkness J. A., Van de Vijver F. J. R., & Johnson T. P. (2003). Questionnaire design in comparative research In Harkness J. A.Van de Vijver F. J. R., & Mohler P. (Eds.), Cross-cultural survey methods (pp. 19–34). Hoboken, NJ: Wiley [Google Scholar]
- Harris-Kojetin L. D., Fowler F. J., Brown J. A., Schnaier J. A., & Sweeney S. F. (1999). The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer Assessment of Health Plans Study. Medical Care, 37(Suppl. 3), MS10–MS21 [DOI] [PubMed] [Google Scholar]
- Johnson T. P. (1998). Approaches to equivalence in cross-cultural and cross-national survey research. ZUMA-Nachrichten Spezial, 3, 1–40 [Google Scholar]
- Johnson T. P. (2003). Glossary In Harkness J. A.Van de Vijver F. J. R., & Mohler P. (Eds.), Cross-cultural survey methods (pp. 347–357). Hoboken, NJ: Wiley [Google Scholar]
- Knafl K., Deatrick J., Gallo A., Holcombe G., Bakitas M., Dixon J., & Grey M. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30, 224–234 [DOI] [PubMed] [Google Scholar]
- Lynn M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–385 [PubMed] [Google Scholar]
- Matsumoto W. (2007). A study on technical and cultural differences in survey methodology between Japan and the United States. International Journal of Japanese Sociology, 16(1), 23–34 [Google Scholar]
- Mothers’ & Children’s Health and Welfare Association (2007). Boshi hoken no omonaru toukei [Maternal and child health statistics of Japan] Tokyo, Japan: Mothers’ & Children’s Health Organization [Google Scholar]
- Polit D. F., Beck C. T., & Owen S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467 [DOI] [PubMed] [Google Scholar]
- Rasinski K. A., Willis G. B., Baldwin A. K., Yeh W., & Lee L. (1999). Methods of data collection, perceptions of risks and losses, and motivation to give truthful answers to sensitive survey questions. Applied Cognitive Psychology, 13, 465–484 [Google Scholar]
- Romney A. K., Moore C. C., & Rusch C. D. (1997). Cultural universals: Measuring the semantic structure of emotion terms in English and Japanese. Proceedings of the National Academy of Sciences of the United States of America, 94, 5489–5494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rusch C. D. (2004). Cross-cultural variability of the semantic domain of emotion terms: An examination of English shame and embarrass with Japanese Hazukashii. Cross-Cultural Research, 38, 236–248 [Google Scholar]
- Sakala C. (2006). Carol Sakala’s letter from North America: An uncontrolled experiment: Elective delivery predominates in the United States. Birth, 33, 332–335 [PubMed] [Google Scholar]
- Sakala C., & Corry M. P. (2008). Evidence-based maternity care: What it is and what it can achieve. New York, NY: Milbank Memorial Fund [Google Scholar]
- Smith T. W. (2004). Developing and evaluating cross-national survey instruments In Presser S.Rothgeb J. M.Couper M. P.Lessler J. T.Martin E.Martin J.et al. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 431–452). Hoboken, NJ: Wiley [Google Scholar]
- Sudman S., Bradburn N. M., & Schwarz N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass [Google Scholar]
- Tanaka-Matsumi J., & Marsella A. J. (1976). Cross-cultural variations in the phenomenological experience of depression: I. Word association studies. Journal of Cross-Cultural Psychology, 7, 379–396 [Google Scholar]
- Tourangeau R. (1984). Cognitive science and survey methods: A cognitive perspective In Jabine T.Straf M.Tanur J., & Tourangeau R. (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 73–100). Washington, DC: National Academy Press [Google Scholar]
- U.S. Census Bureau (Census Advisory Committees) (2005). Census Bureau guideline: Language translation of data collection instruments and supporting materials. U.S. Census Bureau. Retrieved from http://www.census.gov/cac/www/007585.html
- Van de Vijver F. J. R. (2003). Bias and equivalence: Cross-cultural perspectives In Harkness J. A.Van de Vijver F. J. R., & Mohler P. (Eds.), Cross-cultural survey methods (pp. 143–156). Hoboken, NJ: Wiley [Google Scholar]
- Varney H. (1987). History of nurse-midwifery in the United States. In Nurse-midwifery (pp. 19–42). Boston, MA: Blackwell Scientific Publications, Inc. [Google Scholar]
- Willis G. (2004). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage [Google Scholar]
- Willis G. (2006). Cognitive interviewing as a tool for improving the informed consent process. Journal of Empirical Research on Human Research Ethics, 1, 9–24 [DOI] [PubMed] [Google Scholar]
- Willis G., & Zahnd E. (2007). Questionnaire design from a cross-cultural perspective: An empirical investigation of Koreans and non-Koreans. Journal of Health Care for the Poor & Underserved, 18, 197–217 [DOI] [PubMed] [Google Scholar]
- World Health Organization (2000). World Health Organization assesses the world’s health systems. Retrieved from http://www.who.int/whr/2000/media_centre/press_release/en/
