Abstract
From 1949 to 1962, residents of several villages in Kazakhstan Abstract: received substantial doses of radiation to the thyroid gland resulting from nuclear tests conducted at the Semipalatinsk Nuclear Test Site. The primary source of radiation was internal from an intake of radioactive iodine by consumption of contaminated dairy products. A previous research study of childhood exposure and thyroid disease in this region gathered limited data on study participants’ dairy intake at the time of the fallout for the purpose of estimating past radiation doses. Because many subjects were too young at the time of the nuclear tests to recall dietary consumption and existing sources of archival data are limited, it was necessary to interview parents and other village residents who cared for children during this time; older adults ranging in age from 75 to 90 years old. Results from 11 focus group interviews conducted in 2007 with 82 women from 4 villages in Kazakhstan yielded group-level estimates of age-, gender-, ethnicity- and village-specific dairy consumption patterns in rural Kazakhstan during the 1950s. Children typically consumed cow’s milk with limited consumption of mare, goat, and sheep milk; and consumed dairy products such as sour milk (airan), soft cottage cheese (tvorog), and fermented mare milk (koumiss) with the greatest amounts of koumiss reported at ages 15–21. The consumption patterns differed by age and between Kazakh and Russian children, which should lead to different estimates of radiation exposure to the thyroid. This study demonstrated the utility of focus groups to obtain quantitative estimates for dietary intake in the distant past.
INTRODUCTION
In 1998, the United States (US) National Cancer Institute (NCI) collaborated with the Semipalatinsk State Medical Academy (SSMA) 1 and the Kazakh Research Institute for Radiation Medicine and Ecology to conduct a field study to determine prevalence of thyroid disease in relation to childhood radiation exposure in Kazakhstan. Between 1949 and 1962, village residents received doses primarily from internal radiation exposure to the thyroid gland as a result of fallout from atmospheric nuclear tests conducted at the Semipalatinsk Nuclear Test Site (SNTS) in northeastern Kazakhstan (1–3). The major health concern is the risk of thyroid disease resulting from internal exposures of the thyroid to radioiodine which occurred primarily through consumption of radioiodine-contaminated milk and other dairy products from animals grazing on contaminated pasture. Children are especially vulnerable as they tend to typically consume substantial amounts of dairy products and have small thyroid glands which lead the absorbed radiation dose to be higher than in adults. The amount of radiation released from the fallout-related radioactivity in the milk varied by type of animal, the interval between milking the animal and consuming the milk, and the grazing patterns of the animal. Radioiodine taken up by animals is transferred to milk at different efficiencies in different species of animal with cows being least efficient followed by horses, sheep, and goats. Although the date and time of radiation is known for each village, the internal exposure to children depends on the probability of their consumption of contaminated milk products from different animal species at different ages.
Previous data collection and objectives of the current study
At the time of the 1998 field study, participants completed a questionnaire-based interview that included a limited assessment of diet at the time of the fallout. These data, combined with supplementary information provided by collaborators (including limited archival data), were used to generate preliminary radiation doses and thyroid disease risk estimates (3). It was later realized that additional data, which could be obtained from the study population, had the potential to improve dose estimates considerably and thus quantification of thyroid disease risk among individuals exposed at young ages (3). Most importantly, all participants in 1998 were asked about the frequency (daily, weekly, never) of consumption of cow and goat milk and several milk products, but only a subset were asked about consumption (yes, no, seasonally, rarely) of fresh mare, sheep, and camel milk. Radioactive iodine is known to be concentrated in mare and sheep milk and Kazakhs tend to consume sheep and horse milk in addition to cow milk. The questionnaire did not capture any information about the quantity of milk consumed, for any milk type.
Recognizing that many subjects were too young at the time of the nuclear tests to know the frequency and amount of dairy consumption, it was consider necessary to interview their mothers to collect more detailed dairy consumption information to supplement the 1998 data and improve the dosimetry. However, because many of the 1998 study member parents were no longer living or available in 2007, it was more practical to ask surviving village residents in this age group (mothers approximately 18 to 28 years old at the time of the nuclear testing) who either had their own or cared for other children during the relevant time period to provide estimates from their own experience that could be typical of other children in the village of similar ages. Women were the primary caregivers of children at the time of the nuclear tests and are, therefore, experts regarding children’s diet during that time.
A feasibility assessment conducted in two villages and pilot study results from focus group interviews conducted in a third village led us to believe that a lengthy in-depth individual interview would be too tiresome for elderly villagers. In addition, conducting individual interviews was cost prohibitive. A written self-administered questionnaire was not feasible because of the high prevalence of vision impairment in this population. Maternal recall of early childhood dietary consumption using food-frequency questionnaires three to five decades later has limited utility (4). Focus group interviews seemed to be the best research approach available because they have been shown to increase participant comfort when individuals are gathered into homogeneous groups (5–7), generate a broader range of thoughts than individual interviews (5), and allow for a general discussion of an ordinary behavior anchored by a very memorable event among village residents (i.e., atmospheric nuclear testing and/or village evacuation by the military) along with the judicious use of recall cues that have been shown to aid autobiographical memory recall (8–15).
Use of focus group interview methods to augment field survey data
Although focus group data may not be generalized to larger populations (16–18), there may be occasions when it is the best reasonable alternative to conducting a large-scale field study of identified individuals. Previous research used focus group interview methodology both in the US and abroad to collect data about nutrition knowledge, attitudes, and intake patterns among the general population and in a village setting (19–25). Those focus groups were primarily designed to collect qualitative dietary data such as types of foods consumed, beliefs about the health and nutritional value of foods, and reasons for eating certain foods. In at least two studies, participants were also asked to quantify frequency or amount of food/drink consumed or fed to children but such quantitative data collection was limited (19, 25).
Memory and autobiographical dietary recall
When asking participants to self-report data on daily life activities, researchers are relying on autobiographical memory (26,27) and responses to questions are provided as a result of multistage recall processes (28). There are three types of autobiographical memories—personal memories, autobiographical facts, and generic personal memories. Personal memories are mental images of a single unrepeated event (e.g., consuming some very exotic food for the first and only time). Autobiographical facts reflect factual knowledge of an event that is not image based (e.g., knowing that you consumed specific food products when you were 19 years old). Generic personal memories are similar to personal memories, except that the memory reflects a recurring event and the memory is based on a typical schema as the time span between the event and recall increases (e.g., consuming an exotic food on a recurring basis).
Self-report of daily life activities or dietary intake is a complex task of cognitive processes that can be prone to distortion. Epidemiologic studies typically use food frequency questionnaires, structured interviews of foods consumed, and food diaries to ascertain dietary data (29). Although some researchers note concern for errors in autobiographical dietary recall (30–32) and others note the influence that current intake could have on the accuracy of recall (30, 32, 33–40), most highlight the utility of retrospective dietary recall as a reliable indicator when the recall period is from 1 to 10 years (28, 40–45), 15–20 years (37), and even with time periods exceeding 20 years (35, 46–48). Additionally, research supports the reproducibility of dietary recall with correlations ranging from 0.4 to 0.85 measured 1 to 10 years apart (42, 46, 47, 49–51).
Objectives of the current study
As part of a larger study to examine dairy product consumption patterns and daily life activities of children from selected villages exposed to nuclear testing in Kazakhstan in the 1950s, the objectives of this paper are to (1) describe the methodology developed to capture data through group interviews among an older population and (2) present results on estimated milk and dairy product consumption patterns by children during the 1950s.
MATERIALS AND METHODS
In 2007, eleven focus group interviews were conducted with 82 women from 4 villages (Karaul, Kainar, Dolon, and Kanonerka) in Kazakhstan. In three of the four villages, three focus groups were conducted to explore dairy consumption and daily life activities. In the fourth village (Dolon), because of the small population size, only two focus groups were conducted. Because all of the major nuclear tests were conducted in the months of August through October (1949 – 1962), focus group questions targeted dairy consumption during that time of year. Focus group interviews were conducted separately with groups of ethnic Kazakhs and Russians because of the potential effect of different consumption and living practices by these two ethnic groups. Of eight villages included in the 1998 field study, the above four villages (two primarily Russian and two primarily Kazakh villages) were selected for the focus group interviews to represent a range of moderate to high fallout exposure levels, as estimated by the fallout data (52).
Participant recruitment
Participant recruitment was conducted through a partnership between Government Semipalatinsk State Medical University (GSSMU) collaborators and local village officials and healthcare providers. Approval to conduct the study in the selected villages was obtained from the regional administration prior to participant recruitment. Participants were recruited by GSSMU collaborators working closely with local village administrators and with physicians screening for participants who met study eligibility requirements. These local contacts were integral community members, which was critical for participant recruitment.
Participant eligibility
To be eligible for the study, participants had to speak Russian or Kazakh, agree to participate in a 2-hour focus group session, and have lived in the village in the 1950s. Participants needed to be 70 years and older (approximately 18–28 years of age at the time of exposure) and have had children or provided care to children at the time of the nuclear tests. Mothers and other caregivers (i.e., older sisters) who lived in those villages at the time of the tests who could provide information about typical dietary and daily living practices were selected. When more participants were recruited than were required for the focus group interviews, NCI worked with GSSMU to select participants reflecting a range of offspring birth years. In villages where it was not possible to recruit a sufficient number of participants aged 70 and older, participants aged 65 and older were recruited. During the screening process participants were informed of the general study objectives and that there would be a token incentive for attending the focus group interview ($5 or 675 Tenge).2
Focus group moderator guide and questions
The moderator guide was divided into sections addressing dairy consumption of children and the women’s own milk consumption during early adulthood and breastfeeding. Groups were conducted in either Russian or Kazakh depending on the ethnic composition of the village. Women of only Kazakh ethnicity were interviewed in two villages (Karaul and Kainar), but only Russian ethnicity in the village of Kanonerka. In the village of Dolon, one focus group of each ethnicity was interviewed.
After consent was obtained, groups were started with participant introductions where women told us their first name and year of birth, the first name of each of their children, the child’s gender and year of birth, and if they did not have children in the 1950s, any brothers or sisters, nieces or nephews, or other children in the village whom they cared for and fed. The first questions were designed to help prompt recall from life in the 1950s and anchor their reference to the time during the nuclear tests. Using a “think back” approach to anchor the memories of participants in the 1950s, participants were asked to recall and discuss what they saw or felt at the time of the nuclear testing, whether the military contacted them, what they were told by officials, whether they were evacuated, and if evacuated, when they were allowed to return to their homes.
To obtain data required for estimating internal radiation doses, women were asked questions about dairy consumption for their children (or children they cared for) by age group. For example, “When your children were less than a year old, how much cow’s milk did your children drink?” These questions were repeated for each type of milk including goat, sheep, mare, and any other type of milk, for various age groups—birth to 12 months, 1 to 3 years, 4 to 6 years, 7 to 14 years, and age 15 and older. Participants were also asked about their own milk consumption during breastfeeding and when they were 15 to 21 years of age. During the pilot study, we determined that it was very difficult for women to recall dietary patterns of children over the age of 14 and therefore we asked women about their own consumption between the ages of 15 and 21 (i.e., during the 1950s). This process was repeated for dairy products—koumiss (fermented mare milk), tvorog (a type of cottage cheese), prostokvasha (airan in Kazakh—a sour cow’s milk), and other milk products (e.g., sour cream, cream, etc.).
Instrument and focus group interview approach
The project team piloted the focus group methodology in advance of the main data collection in an exposed village, Korostely, using a draft focus group interview guide for two groups—one women’s group to provide diet and daily life activities and one to collect background information (not presented in this paper) from male residents about pasture practices (useful for estimating radioactive iodine levels) and housing construction materials (useful for estimating shielding effects for external exposure). Results of the pilot focus group interviews led to modifications to the study methodology that were implemented for the main focus group study including the following: (1) gender matching of moderator and group was not deemed necessary; (2) when asking about the quantity and volume of dairy consumption, drinking vessels and utensils graduated in size and volume commonly used by village residents in that time period were measured by the study team and used as reference points; (3) for a more clear understanding of information being collected a large poster-sized paper wall chart was used where assistant moderators would record consumption data for the entire group to view and guide the moderator, which could improve data quality; (4) fewer and simpler questions in the interviewing guide; and (5) creation of a predictable pattern of responding to ease cognitive burden. The last technique methodically guided participants through questions for each child age group (i.e., for children from birth to 12 months old, 1 to 3 years old, 4 to 6 years old, 7 to 14 years old, and age 15 and older) by dairy consumption (i.e., first for milk type: breast milk, cow, goat, sheep, and mare; second for dairy products: koumiss, tvorog, airan, and other dairy products). During the pilot study, we also determined that it was very difficult for women to recall dietary patterns of children over the age of 14 and therefore we asked women about their own consumption between the ages of 15 and 21 during the 1950s. Additionally, lessons learned in group facilitation and note taking from the pilot study focus group interviews were integrated into training modules for local bilingual facilitators of the main study.
Following the practice described for training local bilingual facilitators (53) and general focus group procedures (6), focus group interviews were conducted by four faculty members of the GSSMU who received 68 hours of training as both lead and assistant focus group moderator. One facilitator was a native Russian speaker and the other three facilitators were native bilingual Russian/Kazakh speakers. GSSMU focus group facilitator training included basic focus group methods, protocol-specific training with several practice sessions, hands-on practice on digital audio recording and data file transfer, and training in note taking and data cleaning. Eight additional hours of focus group interviewer and assistant moderator refresher training was provided one day prior to traveling to remote villages to ensure consistency in how questions would be asked and data captured.
For each focus group interview, there was one lead facilitator, one assistant moderator dedicated to capturing data on wall-charts, and one assistant moderator dedicated to audio-recording the focus group interview session and taking written notes on a paper-based note-taking template. Audio recordings were captured on two digital recording devices: a small micro digital audio recorder and a laptop computer with digital audio recording software—both with external microphones placed in the center of the table where focus group participants were seated. Study team members from NCI, fluent in Russian, were present for focus group interviews in villages with predominant Russian ethnicity taking notes and providing feedback to facilitators and upon completion of each focus group interview.
All research instruments and procedures, including participant informed consent forms were reviewed and approved by the research team’s Institutional Review Boards and the Office of Management and Budget of the United States government. At the beginning of each focus group, participants were briefed on the purpose of the study and were asked to read and sign the informed consent form and return the form to the focus group facilitator. Focus group facilitators also read aloud the informed consent form to assist participants with limited eyesight and other conditions as needed. Participants were treated in accordance with the guidelines published by the American Psychological Association (54). No adverse events occurred during the course of this study.
Data capture, cleaning, and analysis
The focus group moderator systematically asked about each participant about her children’s consumption of each type of milk and milk product at each of the 5 age groups (birth to 12 months, 1–3 years, 4–6 years, 7–14 years, and 15–21 years). The moderator asked each participant about one milk-type or milk product before moving on to the next item. The participant was asked whether or not her children consumed the particular item. If yes, the woman was asked how often (reported per day, week, or month) and in what quantities the children consumed these products during each age group. The frequency and quantities (as well as the unit given by the participant) were recorded on a wall chart. The data recorded on the wall chart were verified against the audio recordings during the note-cleaning process. Notes, audio recordings, and data collection wall charts were reviewed upon completion of the focus group interviews in two phases. The first phase of note review was completed after interviews were completed for the first two villages (i.e., Karaul and Kainar). The second phase of note review was completed after interviews were completed for the final two villages (i.e., Dolon and Kanonerka). Data review and cleaning was completed in two phases because villages lacked the facilities for the entire project team to participate in note cleaning. All team members participated in a group debriefing session where audio recordings were played; notes reviewed, cleaned, and verified; and wall-chart data compiled into one master data file spreadsheet. Upon completion of the focus group interviews, Microsoft Excel data files were imported into SAS/STAT® software (SAS Version 9.1; SAS Institute, Cary, NC).
From the frequencies and quantities reported, a daily amount consumed (mL) was calculated. A woman’s response for a given age group could be counted multiple times (corresponding to the number of children she had who were in that age group during the relevant time period). From this, we computed the prevalence of consumption (number of children who consumed the item divided by the total number of children in a given age-stratum) and the mean daily consumption rate (obtained by summing the daily amount (mL) over all children in a given age-stratum and dividing by the number of children in that age-group who consumed the item). The mean consumption rate was rounded to the nearest 10. For both the prevalence and mean daily consumption estimates, children with missing data were excluded.
RESULTS
Participant demographics and characteristics
There were 6–8 women in each focus group with an average age of 75 years, ranging from 65 to 91 years (Table 1). Approximately 65% of the participants were ethnic Kazakhs and 35% were ethnic Russians. The mean number of children per woman in each village varied little and ranged between 2 and 5. The number of children per participant ranged from 1 to 10.
Table 1.
Focus Group Interview Participant Characteristics
| Village | Group Ethnic Composition | Average Age | Ethnicity | Average Number of Children | |
|---|---|---|---|---|---|
| Kazakh | Russian | ||||
| Kainar | Group 1: KZ | 76.6 | 8 | 0 | 3.9 |
| Group 2: KZ | 78.5 | 8 | 0 | 3.8 | |
| Group 3: KZ | 75.1 | 8 | 0 | 3.6 | |
| Karaul | Group 1: KZ | 77.9 | 8 | 0 | 4.6 |
| Group 2: KZ | 70.5 | 8 | 0 | 2.3 | |
| Group 3: KZ | 72.0 | 8 | 0 | 2.6 | |
| Dolon | Group 1: KZ | 70.8 | 6 | 0 | 4.2 |
| Group 2: RU | 74.9 | 0 | 8 | 3.0 | |
| Kanonerka | Group 1: RU | 75.4 | 0 | 7 | 2.3 |
| Group 2: RU | 75.4 | 0 | 7 | 2.3 | |
| Group 3: RU | 76.7 | 0 | 6 | 2.8 | |
| TOTAL | 75.1 | 54 | 28 | 3.2 | |
Abbreviations: KZ = Kazak; RU = Russian.
Dairy consumption
Dairy consumption data are presented in Tables 2–5. Table 2 shows the prevalence (in percentage) of each type of fresh milk consumed by children at different age ranges by village. Across all the villages, cow’s milk was the predominant type consumed, ranging from approximately 70% to 90% among toddlers, preschool and school-aged children (between 1 and 14 years). The prevalence of fresh milk consumption tended to be lower in infants, because breast feeding was common. Fresh milk consumption was also less prevalent in adolescents or young adults (aged 15–21), although the prevalence estimates were unstable because of the small number of subjects. The amount of fresh cow milk consumed was also substantial (ranging from about 200 to more than 500 mL/day) and higher than other types of fresh milk (except for mare’s milk in Kazakh villages, as presented below; Table 3). Among those who consumed cow’s milk, the mean daily amount tended to increase with age, but the patterns at older ages varied. There was very little information obtained about the amount consumed by young adults of ages 15–21. Beginning at ages 1–3 years, it was quite common for children to consume milk with tea, reaching a prevalence of nearly 100% by age 15 in Kainar and Dolon (Table 2), although the amount of cow’s milk consumed with tea was lower than that consumed alone, typically less than 100 mL/day (Table 3).
Table 2.
Prevalencea of Milk Consumption by Village and Age Group Reported by Focus Groups
| Milk | Village | Ethnicity | Age, y | |||||
|---|---|---|---|---|---|---|---|---|
| <1 | 1–3 | 4–6 | 7–14 | 15–21 | Mothers 15–21b | |||
| Cow | Kainar | KZ | 0.44 | 0.89 | 0.89 | 0.70 | 0.55 | 0.63 |
| Karaul | KZ | 0.37 | 0.85 | 0.90 | 0.83 | 0.86 | 0.47 | |
| Dolon | KZ+RU | 0.71 | 0.87 | 0.83 | 0.67 | 0.40 | 0.57 | |
| Kanonerka | RU | 0.60 | 0.89 | 0.88 | 0.91 | 0.83 | 1.00 | |
| Goat | Kainar | KZ | 0.03 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Karaul | KZ | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Dolon | KZ+RU | 0.23 | 0.20 | 0.28 | 0.00 | 0.00 | 0.00 | |
| Kanonerka | RU | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Sheep | Kainar | KZ | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Karaul | KZ | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Dolon | KZ+RU | 0.33 | 0.13 | 0.20 | 0.47 | 0.00 | 0.14 | |
| Kanonerka | RU | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Mare | Kainar | KZ | 0.07 | 0.08 | 0.08 | 0.05 | 0.00 | 0.00 |
| Karaul | KZ | 0.08 | 0.03 | 0.05 | 0.00 | 0.00 | 0.14 | |
| Dolon | KZ+RU | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.29 | |
| Kanonerka | RU | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Cow’s milk with tea | Kainar | KZ | 0.00 | 0.40 | 0.71 | 0.77 | 1.00 | 0.88 |
| Karaul | KZ | NA | NA | NA | NA | NA | NA | |
| Dolon | KZ+RU | 0.00 | 0.00 | 0.48 | 0.80 | 1.00 | 0.86 | |
| Kanonerka | RU | 0.00 | 0.48 | 0.46 | 0.43 | 0.17 | 1.00 | |
| Number of children | Kainar | KZ | 89 | 81 | 63 | 44 | 11 | 8 |
| Karaul | KZ | 76 | 71 | 42 | 29 | 7 | 15 | |
| Dolon | KZ+RU | 49 | 46 | 29 | 15 | 5 | 7 | |
| Kanonerka | RU | 50 | 48 | 41 | 32 | 6 | 5 | |
Abbreviations: KZ = Kazak; KZ+RU = Kazakh and Russian; NA = Not Applicable (focus group participants were not asked about consumption of this dairy product); RU = Russian.
The number of children for whom data were available varies slightly (by ≤6 children) across milk type. Prevalence estimates were based on the number of children for whom data were available.
Report of consumption from women who were 15–21 years of age and also mothers during the 1950’s.
Table 5.
Average Daily Consumption Ratea (mL d−1) of Dairy Products by Children in 1950s Reported by Focus Groups
| Dairy Product | Village | Ethnicity | Age, y | |||||
|---|---|---|---|---|---|---|---|---|
| <1 | 1–3 | 4–6 | 7–14 | 15–21 | Mothers 15–21b | |||
| Fermented mare milk (also referred to as koumiss) | Kainar | KZ | - | (75)c | 270 | 260 | (250) | (280) |
| Karaul | KZ | - | (250) | (250) | (35) | (250) | (650) | |
| Dolon | KZ+RU | - | 95 | (250) | (440) | (220) | (560) | |
| Kanonerka | RU | - | - | - | - | - | - | |
| Cottage cheese (also referred to as tvorog) | Kainar | KZ | (20) | (30) | (25) | (350) | DR | (35) |
| Karaul | KZ | - | (25) | (30) | 30 | DR | (200) | |
| Dolon | KZ+RU | (40) | 90 | 95 | (70) | (65) | (120) | |
| Kanonerka | RU | (7) | 15 | 30 | 30 | (20) | (30) | |
| Sour milk (also referred to as airan or prostokvasha) | Kainar | KZ | 20 | 180 | 140 | 360 | DR | (220) |
| Karaul | KZ | - | (190) | (190) | DR | DR | (280) | |
| Dolon | KZ+RU | 50 | 140 | 220 | 185 | (130) | (260) | |
| Kanonerka | RU | (20) | (50) | (45) | DR | (7) | (130) | |
| Sour creamd | Kainar | KZ | NA | NA | NA | NA | NA | NA |
| Karaul | KZ | NA | NA | NA | NA | NA | NA | |
| Dolon | KZ+RU | NA | NA | NA | NA | NA | NA | |
| Kanonerka | RU | (30) | 25 | 30 | 20 | (15) | (30) | |
Abbreviations: DR = did consume but could not recall specific amounts; KZ = Kazak; KZ+RU = Kazakh and Russian; NA = Not Applicable (focus group participants were not asked about consumption of this dairy product); RU = Russian.
Average among persons who reported consumption rate of that type of dairy product rounded to the nearest five for consumption rates from 10 to 100 (mL d−1) and to the nearest ten for consumption rates more than 100 (mL d−1)..
Report of consumption from women who were 15–21 years of age and also mothers during the 1950’s.
Values in parentheses reflect groups with fewer than 10 responses.
Units are (g d−1)
Table 3.
Average Daily Consumption Ratea (mL d−1) of Fresh Milk by Children in 1950s Reported by Focus Groups
| Milk | Village | Ethnicity | Age, y | |||||
|---|---|---|---|---|---|---|---|---|
| <1 | 1–3 | 4–6 | 7–14 | 15–21 | Mothers 15–21b | |||
| Cow | Kainar | KZ | 200 | 250 | 410 | 430 | DR | (370)c |
| Karaul | KZ | 190 | 160 | 410 | 310 | DR | (320) | |
| Dolon | KZ+RU | 240 | 290 | 400 | 510 | (200) | (390) | |
| Kanonerka | RU | 440 | 500 | 640 | 450 | (1250) | (540) | |
| Goat | Kainar | KZ | DR | - | - | - | - | - |
| Karaul | KZ | - | - | - | - | - | - | |
| Dolon | KZ+RU | (100) | DR | DR | - | - | - | |
| Kanonerka | RU | - | - | - | - | - | - | |
| Sheep | Kainar | KZ | - | - | - | - | - | - |
| Karaul | KZ | - | - | - | - | - | - | |
| Dolon | KZ+RU | 90 | DR | (250) | DR | - | DR | |
| Kanonerka | RU | - | - | - | - | - | - | |
| Mare | Kainar | KZ | (150) | (330) | (440) | DR | - | - |
| Karaul | KZ | (160) | (180) | (250) | - | - | (750) | |
| Dolon | KZ+RU | - | - | - | - | - | (830) | |
| Kanonerka | RU | - | - | - | - | - | - | |
| Cow’s milk with tea | Kainar | KZ | - | (20) | 45 | 50 | (55) | (90) |
| Karaul | KZ | NA | NA | NA | NA | NA | NA | |
| Dolon | KZ+RU | - | - | 45 | 90 | (100) | (80) | |
| Kanonerka | RU | - | 110 | 100 | 60 | DR | (100) | |
Abbreviations: DR = did consume but could not recall specific amounts; KZ = Kazak; KZ+RU = Kazakh and Russian; NA = Not Applicable (focus group participants were not asked about consumption of this dairy product); RU = Russian.
Average among persons who reported consumption rate of that type of dairy product rounded to the nearest five for consumption rates from 10 to 100 (mL d−1) and to the nearest ten for consumption rates more than 100 (mL d−1)..
Report of consumption from women who were 15–21 years of age and also mothers during the 1950’s.
Values in parentheses reflect groups with fewer than 10 responses.
Goat milk consumption was reported for a minority of children (3%–28%) in two villages, Kainar (Kazakh village) and Dolon (Russian and Kazakh mixed; Table 2). In Kainar, goat milk consumption was reported only for a small proportion of infants while, in Dolon, both goat and sheep milk consumption was reported for infants, toddlers, and preschool-aged children (<1 to 6 years) but only for Kazakhs. The prevalence of sheep milk consumption among Kazakh children in Dolon ranged from 13% to 47%. We had insufficient data regarding the amount of goat and sheep milk on which to draw any conclusions about the quantity of milk consumed (Table 3).
Mare’s milk consumption was exclusively reported for Kazakh children in Kainar and Karaul (Table 2). Although data are based on few observations, the amount of mare milk consumed seemed to increase with age up to age 6. Mean daily quantities reported for children were high, ranging from 150 to 440 mL/day, amounts comparable to those reported for cow’s milk. In Dolon, two Kazakh mothers indicated that they had consumed mare milk at ages 15–21 years (in the 1950’s) but reported no consumption by their children at any age. None of the participants reported camel milk consumption for any age group (data not shown).
Daily dairy product consumption—koumiss (fermented mare milk), tvorog (cottage cheese), and prostokvasha (or airan, a sour milk) was also asked of participants in each village (Tables 4 and 5). Consumption of fermented mare’s milk (koumiss) was only reported for Kazakh children ≥1 year and adolescents. The prevalence of koumiss consumption generally increased with age beginning at ages 1–3 (Table 4). Mean daily consumption rates are based on small numbers but suggest that those who consumed koumiss did so in fairly high quantities (comparable in some cells to that reported for cow’s milk). Cottage cheese was widely consumed (Table 4) by both Russians and Kazakhs, particularly among children ages 1 year and older. Children typically consumed ≤100 g/day (Table 5). Focus group participants also reported a high prevalence of sour milk consumption among children ages 1 year and older (Table 4), with mean daily consumption rates ranging from 7 to 360 mL/day.
Table 4.
Prevalence a of Dairy Product Consumption by Village and Age Group Reported by Focus Groups
| Dairy Product | Village | Ethnicity | Age, y | |||||
|---|---|---|---|---|---|---|---|---|
| <1 | 1–3 | 4–6 | 7–14 | 15–21 | Mothers 15–21b | |||
| Fermented mare milk (also referred to as koumiss) | Kainar | KZ | 0.00 | 0.05 | 0.31 | 0.48 | 0.70 | 0.75 |
| Karaul | KZ | 0.00 | 0.01 | 0.02 | 0.34 | 1.00 | 0.60 | |
| Dolon | KZ+RU | 0.00 | 0.22 | 0.31 | 0.27 | 0.40 | 0.57 | |
| Kanonerka | RU | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |
| Cottage cheese (also referred to as tvorog) | Kainar | KZ | 0.03 | 0.19 | 0.20 | 0.25 | 0.10 | 0.88 |
| Karaul | KZ | 0.00 | 0.24 | 0.69 | 0.90 | 0.50 | 0.40 | |
| Dolon | KZ+RU | 0.19 | 0.73 | 0.69 | 0.67 | 0.40 | 0.71 | |
| Kanonerka | RU | 0.18 | 0.70 | 0.98 | 0.58 | 0.67 | 1.00 | |
| Sour milk (also referred to as airan or prostokvasha) | Kainar | KZ | 0.16 | 0.62 | 0.75 | 0.65 | 0.80 | 0.75 |
| Karaul | KZ | 0.00 | 0.72 | 0.82 | 1.00 | 1.00 | 0.47 | |
| Dolon | KZ+RU | 0.48 | 0.69 | 0.66 | 0.93 | 1.00 | 0.71 | |
| Kanonerka | RU | 0.10 | 0.28 | 0.33 | 0.10 | 0.17 | 0.60 | |
| Sour cream | Kainar | KZ | NA | NA | NA | NA | NA | NA |
| Karaul | KZ | NA | NA | NA | NA | NA | NA | |
| Dolon | KZ+RU | NA | NA | NA | NA | NA | NA | |
| Kanonerka | RU | 0.06 | 0.38 | 0.65 | 0.68 | 0.83 | 1.00 | |
| Number of children | Kainar | KZ | 89 | 81 | 63 | 44 | 11 | 8 |
| Karaul | KZ | 76 | 71 | 42 | 29 | 7 | 15 | |
| Dolon | KZ+RU | 49 | 46 | 29 | 15 | 5 | 7 | |
| Kanonerka | RU | 50 | 48 | 41 | 32 | 6 | 5 | |
Abbreviations: KZ = Kazak; KZ+RU = Kazakh and Russian; NA = Not Applicable (focus group participants were not asked about consumption of this dairy product); RU = Russian.
The number of children for whom data were available varies slightly (by ≤6 children) across dairy product type. Prevalence estimates were based on the number of children for whom data were available.
Report of consumption from women who were 15–21 years of age and also mothers during the 1950’s.
Data were insufficient to evaluate results across focus groups within a single village. However, where there were enough data to make comparisons, responses were pooled to test for within- and between-ethnicity agreement for responses pertaining to children ages 1–3 and 4–6 years. Although there were limited data for analysis, results indicated no within-ethnicity differences but there were significant differences between ethnic Kazakh and Russian groups. There was no evidence that dietary patterns differed by gender within a given ethnicity. This indicates that the focus group methodology elicited similar responses within ethnicities as would be expected.
DISCUSSION
Overall, focus group interviews generated distributions of age- and ethnicity specific dairy consumption patterns in children from villages in Kazakhstan that received radiation exposure from atmospheric nuclear weapons tests conducted at the SNTS some 50 years ago. Typically, children consumed cow’s milk although other types of milk such as mare’s milk were also consumed by children of Kazakh ethnicity. There was limited consumption of goat and sheep milk. Mothers’ own consumption at ages of 15 – 21 tended toward cow milk although they also reported consumption of other types of milk such as mare milk. Also, children and mothers typically consumed dairy products such as airan, tvorog, and koumiss (only Kazakhs) with the greatest amounts of koumiss reported for mothers’ own consumption at ages 15–21 years.
A noteworthy result of this study is the use of focus group interview methodology to elicit individual estimates of behaviors from the 1950s. Participants who had not participated in focus group interviews prior to this study were willing and able to understand and provide meaningful response to questions asked. Use of physical examples of bowls, spoons, and cups used in these villages aided participants’ recall and ability to provide estimates of milk and dairy product consumption. Our knowledge of the exact volume for each container and utensil used as reference points in the focus groups assisted in our estimation of amounts consumed.
In contrast to previous data on dietary consumption, the focus group methodology may have improved recall through the discussions about the lifestyles and practices. Our observations suggested that involving the focus group members in discussion about village life during this period may have aided recall. The variety of food items in the diet during the 1950s was much more limited in these communities than in studies from the US in the same years. Dairy products were a main component of the diet, which may have improved the mothers’ ability to recall intakes of these foods. In addition, there was a limited amount of milk provided by the animal each day, a limited amount permitted to be kept by a family, and a known amount of time that the milk would last before spoilage. We do not, however, have any validity criterion to judge the accuracy of their reports.
Previous work from the 1998 survey data collection (3) had limited intake information about dairy products and presumed some information for the purpose of dose estimation. Consistent general answers elicited through focus group interviews should improve the estimation of dose. Although the specific amounts may not be accurate for a given individual, the probability of consuming a particular amount for a child of a specific age, gender, and ethnicity is likely to be more reliable than previous presumptions about individual-level data because we asked mothers instead of relying on the 1998 cohort members’ own recall of their childhood diet. The relative frequency and quantity of intake across these subgroups, especially ethnic and age subgroups, is especially important for dose estimation and the variability across groups is essential.
Several of our previous assumptions were modified as a result of information obtained from the focus groups. Focus group participants reported that there was no difference between boys and girls for the types of milk and dairy products consumed, children did not drink camel milk as had been reported by some interviewees in 1998, children did not consume large quantities of fresh horse and sheep milk, and milk consumption by Russian and Kazakh children differed only by the amount of milk. Finally, no information was previously available about mother’s milk intake during pregnancy and lactation, which is relevant to some individuals in the 1998 cohort who were exposed in utero or during breast feeding.
Potential limitations of this study included a reluctance to participate in the focus group interviews, hypothesis guessing, inaccurate recall, and participation bias because of mortality. In advance of focus group interviews with village residents, there was some concern as to whether village residents would actively participate in focus group interviews after having lived through an era of Soviet rule that may have discouraged an open discussion of events related to nuclear testing in the SNTS. Results from the post-focus group evaluation and facilitator debriefings indicate that all but a few participants actively participated and there was no perceivable reluctance to participate. Because of the limited number of study sites for radiation exposure, the people of selected villages in Kazakhstan may have participated in other, earlier studies, and may possess a response bias of hypothesis guessing (55) whereby participants provide responses that they believe aligns with the study objectives. However, post-focus group debriefings with focus group facilitators revealed that participants were surprised to hear that the objective of the study was to explore dairy consumption and village life in the 1950s despite being informed of the general study objectives during the screening. As a result, participants unlikely came to the focus group with a preconceived notion for what responses the research team was anticipating.
Another potential threat to the validity of the results of the study was inaccurate recall. Several studies mentioned earlier note the general validity and reproducibility of autobiographical dietary recall 40–50 years later (4,56). Chavarro et al. (4) studied the validity of maternal dietary recall using a food frequency questionnaire (FFQ) after 43 years for children aged 3–5 years. Results indicate a general lack of validity of maternal responses on a FFQ four decades later—especially for milk consumption. The food frequency questionnaire provides a different type of information and requires different cognitive tasks than those used in our study. Smith et al. (56) examined the reliability of dietary recall and concluded that when dietary recall tasks exceeds several hours, participants may be basing their reports on generic memory but add that “If dietary reports are based substantially on generic memory, perhaps generic memory is what epidemiologists should ask about” (56, p. 290). Reporting higher intakes for themselves at age 15–21 than their children at age 15–21 years suggests there was some evidence of current habits influencing their recall of the past. The distributions of intakes generated, however, are still likely to be useful for dose and risk estimation purposes since differences of reported intakes by ethnicity suggests some underlying validity to the reports. Although difficulty in recall may remain an issue, the discussions and concurrence among participants about lifestyles and events of that time period make this a unique study and not comparable to other published studies of individual recall.
As a result of the mortality of other village elders over time, it may be that participants available for the current study had different daily life practices in the 1950s than parents of 1998 study members. There is no way to discount this potential bias but questions were designed to ask about their own life practices and follow-up probes asked if their experience was different for others in the village that they knew. Conversely, village residents live such closely-knit lives that it may be difficult for one to appreciate that their experiences differed from others in the community or that if they did differ, that they would want to call attention to these differences. There was no indication in the interviews or the data collected that the participants’ responses were different from typical village life in the 1950s.
This study provides researchers with a reasonable approach for collecting data from a difficult-to-reach population with questions that pose significant cognitive demands. Other applications could include epidemiologic studies of environmental exposures or agricultural practices in the past including chemicals used in particular regions or for particular foods. Previous research demonstrates that participants tend to rely on generic memory as the recall period increases (56). Focus group interviews may help highlight the timeframe in question and could lead to more accurate recall of exposures or fields that change with time. For example, occupational physical activity may have changed over time so discussion and consensus on type of activity for particular jobs is likely to be more accurate than individual recall. It may be useful to have a focus group at the family level to determine typical food purchased at the household level for a specified period of time. The best use of such data is at the group level but, in some cases, the group data can inform the individual level, as occurred in our study.
Our data regarding milk and dairy product consumption should improve radiation dose estimation in the ongoing epidemiologic studies of thyroid disease risk following fallout radiation exposure that occurred more than 50 years ago in Kazakhstan villages or those attempting to gain a better understanding of daily life in rural villages in Kazakhstan. Even where there was information from survey data on dairy intake from the 1960s or the more recent 1998 primary data collection, information from the focus group interviews made significant contributions in understanding assumptions about consumption by age, gender, and stage of life. In many epidemiological studies of cancer and radiation exposure from past nuclear testing or accidents, retrospective dose reconstruction using very limited information presents major challenges. As in our case, past information available may not provide sufficient details needed to allow for a desired dose reconstruction with the necessary reliability. Interviews with individuals in a cohort having been exposed in the distant past may be problematic for a variety of reasons including memory that can be inaccurate, unreliable, or biased. The present effort utilizing the focus group techniques may open up a new avenue for epidemiological research on long-term radiation effects in other historical cohorts.
Acknowledgments
We gratefully acknowledge the participants of our focus group interviews, medical officials, and civil officials in Karaul, Kainar, Dolon, Kanonerka, Sarzhal, Korostely, Novopokrovka, and Bolshaya Vladimirovka who aided in the field study in August–September 2007. We also acknowledge support and guidance from Dr. Martha Linet of the National Cancer Institute and Ms. Kim Aspinwall, Ms. Dawn Ohse, Ms. Norma Kim, and Mr. John Heinrich from RTI International. We thank Dr. Deukwoo Kwon of the National Cancer Institute for statistical support. We extend our gratitude and appreciation to the anonymous reviewers for their constructive feedback.
Footnotes
The Semipalatinsk State Medical Academy (SSMA) is now the Government Semipalatinsk State Medical University (GSSMU).
In one of the four villages, Kainar, participants were not screened in advance because of the logistical complexities of reaching remote villages. In-country study collaborators met with village health care providers to review the hospital registry and generate a list of potential participants who met the screening criteria. Those identified were not screened and not informed in advance that they would receive an incentive for participating in the focus group interviews.
Contributor Information
Michael Schwerin, RTI International.
Sara Schonfeld, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Vladimir Drozdovitch, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Kuat Akimzhanov, Government Semipalatinsk State Medical University.
Daulet Aldyngurov, Government Semipalatinsk State Medical University.
André Bouville, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Charles Land, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Nicholas Luckyanov, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Kiyohiko Mabuchi, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Yulia Semenova, Government Semipalatinsk State Medical University.
Steven Simon, National Cancer Institute, Division of Cancer Epidemiology and Genetics.
Alma Tokaeva, Government Semipalatinsk State Medical University.
Zhaxybay Zhumadilov, Government Semipalatinsk State Medical University.
Nancy Potischman, National Cancer Institute, Division of Cancer Control and Population Sciences.
References
- 1.Carr ZA, Bouville A, Luckyanov N, et al. Thyroid disease prevalence and fallout exposure in Kazakhstan. In: Lindholm C, Simon S, Makar B, editors. Proceedings of the Workshop on Dosimetry of the Population Living in the Proximity of the Semipalatinsk Atomic Weapons Test Site (STUK-A187) STUK - Radiation and Nuclear Safety Authority; Helsinki, Sweden: 2002. pp. 55–61. [Google Scholar]
- 2.Grosche B, Land C, Bauer S, et al. Fallout from nuclear tests: health effects in Kazakhstan. Radiat Environ Biophys. 2002;41:75–80. doi: 10.1007/s00411-001-0136-1. [DOI] [PubMed] [Google Scholar]
- 3.Land CE, Zhumadilov Z, Gusev BI, et al. Ultrasound-detected thyroid nodule prevalence and radiation dose from fallout. Radiat Res. 2008;169:373–383. doi: 10.1667/RR1063.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chavarro JE, Michels KB, Sheherazadh I, et al. Validity of maternal recall of preschool diet after 43 years. Am J Epidemiol. 2009;169(9):1148–1157. doi: 10.1093/aje/kwp012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kitzinger J. Qualitative research. Introducing focus groups. Br Med J. 1995;311:299–302. doi: 10.1136/bmj.311.7000.299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Krueger R. Focus groups: A Practical Guide for Applied Research. 3. Thousand Oaks, CA: Sage Publications; 2000. [Google Scholar]
- 7.Lakshman M, Charles M, Biswas M, et al. Focus group discussions in medical research. Indian J Pediatr. 2000;67:358–362. doi: 10.1007/BF02820688. [DOI] [PubMed] [Google Scholar]
- 8.Brown P, Keenan JM, Potts GR. The self-reference effect with imagery encoding. J Pers Soc Psychol. 1986;51:897–906. [Google Scholar]
- 9.Ervin RB, Smiciklas-Wright H. Using encoding and retrieval strategies to improve 24-hour dietary recalls among older adults. J Am Diet Assoc. 1998;98:989–994. doi: 10.1016/S0002-8223(98)00227-2. [DOI] [PubMed] [Google Scholar]
- 10.Klein SB, Kihlstrom JF. Elaboration, organization, and the self-reference effect in memory. J Exp Psychol Gen. 1986;115:26–38. doi: 10.1037//0096-3445.115.1.26. [DOI] [PubMed] [Google Scholar]
- 11.Klein SB, Loftus J. The nature of self-referent encoding: The contributions of elaborative and organizational processes. J Pers Soc Psychol. 1988;55:5–11. [Google Scholar]
- 12.Klein SB, Loftus J, Burton HA. Two self-reference effects: The importance of distinguishing between self-descriptiveness judgments and autobiographical retrieval in self-referent encoding. J Pers Soc Psychol. 1989;56:853–865. [Google Scholar]
- 13.McDonough IM, Gallo DA. Autobiographical elaboration reduces memory distortion: Cognitive operations and the distinctiveness heuristic. J Exp Psychol Learn Mem Cogn. 2008;34(6):1430–1445. doi: 10.1037/a0013013. [DOI] [PubMed] [Google Scholar]
- 14.Rogers TB, Kuiper NA, Kirker WS. Self-reference and encoding of personal information. J Pers Soc Psychol. 1977;35:677–688. doi: 10.1037//0022-3514.35.9.677. [DOI] [PubMed] [Google Scholar]
- 15.Smith RE, Hunt RR. Distinctive processing improves memory for recent autobiographical events. In: Rosen DT, editor. Trends in Experimental Psychology Research. Nova Science Publishers; Hauppauge, NY: 2005. pp. 269–281. [Google Scholar]
- 16.Edmunds H. The Focus Group Research Handbook. Chicago, IL: NTC Business Books; 1999. [Google Scholar]
- 17.Merton R, Fiske M, Kendall P. The Focused Interview: A Manual of Problems and Procedures. 2. New York, NY: Free Press; 1990. [Google Scholar]
- 18.Morgan DL, Krueger RA. The Focus Group Kit. Thousand Oaks, CA: Sage Publications; 1997. [Google Scholar]
- 19.Edmonds VM. The nutritional patterns of recently immigrated Honduran women. J Transcult Nurs. 2005;16:226–235. doi: 10.1177/1043659605274959. [DOI] [PubMed] [Google Scholar]
- 20.Elmubarak E, Bromfield E, Bovell-Benjamin AC. Focused interviews with Sudanese Americans: Perceptions about diet, nutrition, and cancer. Prev Med. 2005;40:502–509. doi: 10.1016/j.ypmed.2004.09.021. [DOI] [PubMed] [Google Scholar]
- 21.Hargreaves MK, Schlundt DG, Buchowski MS. Contextual factors influencing the eating behaviors of African American women: A focus group investigation. Ethn Health. 2002;7(3):133–147. doi: 10.1080/1355785022000041980. [DOI] [PubMed] [Google Scholar]
- 22.Jonsson IM, Hallbert LR-M, Gustafsson I-B. Cultural foodways in Sweden: Repeated focus group interviews with Somalian women. Int J Consum Stud. 2002;26(4):328–339. [Google Scholar]
- 23.Jonsson IM, Wallin A-M, Hallbert LR-M, et al. Choice of food and food traditions in pre-war Bosnia-Herzegovina: Focus group interviews with immigrant women in Sweden. Ethn Health. 2002;7(3):149–161. doi: 10.1080/1355785022000041999. [DOI] [PubMed] [Google Scholar]
- 24.Kruger R, Gericke GJ. A qualitative exploration of rural feeding and weaning practices, knowledge and attitudes on nutrition. Pub Health Nutr. 2003;6:217–223. doi: 10.1079/PHN2002419. [DOI] [PubMed] [Google Scholar]
- 25.Satia JA, Patterson RE, Taylor VM, et al. Use of qualitative methods to study diet, acculturation, and health in Chinese-American women. J Am Diet Assoc. 2000;100:934–940. doi: 10.1016/S0002-8223(00)00269-8. [DOI] [PubMed] [Google Scholar]
- 26.Brewer WF. What is autobiographical memory? In: Rubin DC, editor. Autobiographical Memory. Cambridge University Press; Cambridge: 1986. pp. 25–49. [Google Scholar]
- 27.Brewer WF. Autobiographical memory and survey research. In: Schwarz N, Sudman S, editors. Autobiographical Memory and the Validity of Retrospective Reports. Springer-Verlag; New York: 1994. pp. 11–20. [Google Scholar]
- 28.Tourangeau R. Cognitive sciences and survey methods. In: Jabine T, Straf M, Tanur J, et al., editors. Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. National Academy Press; Washington, DC: 1984. pp. 73–100. [Google Scholar]
- 29.Willett WC, Lenart E. Reproducibility and validity of food-frequency questionnaires. In: Willett WC, editor. Nutritional Epidemiology. 2. Oxford University Press; New York: 1998. pp. 101–147. [Google Scholar]
- 30.Dwyer JT, Coleman KA. Insights into dietary recall from a longitudinal study: accuracy over four decades. Am J Clin Nutr. 1997;65(suppl):1153S–1158S. doi: 10.1093/ajcn/65.4.1153S. [DOI] [PubMed] [Google Scholar]
- 31.Marshall JR. Commentary: About that measurement problem. Int J Epidemiol. 2005;34(6):1376–1377. doi: 10.1093/ije/dyi228. [DOI] [PubMed] [Google Scholar]
- 32.Wu ML, Whittemore AS, Jung DL. Errors in reported dietary intakes. II. Long-term recall. Am J Epidemiol. 1988;128(5):1137–1145. doi: 10.1093/oxfordjournals.aje.a115056. [DOI] [PubMed] [Google Scholar]
- 33.Bakkum A, Bloemberg B, van Staveren WA, et al. The relative validity of a retrospective estimate of food consumption based on a current dietary history and a food frequency list. Nutr Cancer. 1988;11:41–53. doi: 10.1080/01635588809513968. [DOI] [PubMed] [Google Scholar]
- 34.Byers T, Rosenthal R, Marshall J, et al. Dietary history from the distant past: a methodological study. Nutr Cancer. 1983;5:69–77. doi: 10.1080/01635588309513781. [DOI] [PubMed] [Google Scholar]
- 35.Dwyer JT, Gardner J, Halvorsen K, et al. Memory of food intake in the distant past. Am J Epidemiol. 1989;130(5):1033–1046. doi: 10.1093/oxfordjournals.aje.a115404. [DOI] [PubMed] [Google Scholar]
- 36.Friedenreich CM, Slimani N, Riboli E. Measurement of past diet: review of previous and proposed methods. Epidemiol Rev. 1992:177–196. doi: 10.1093/oxfordjournals.epirev.a036086. [DOI] [PubMed] [Google Scholar]
- 37.Jensen OM, Wahrendorf J, Rosenqvist A, et al. The reliability of questionnaire-derived historical dietary information and temporal stability of food habits in individuals. Am J Epidemiol. 1984;120:281–290. doi: 10.1093/oxfordjournals.aje.a113891. [DOI] [PubMed] [Google Scholar]
- 38.Rohan TE, Potter JD. Retrospective assessment of dietary intake. Am J Epidemiol. 1984;120:876–887. doi: 10.1093/oxfordjournals.aje.a113959. [DOI] [PubMed] [Google Scholar]
- 39.Thompson FE, Metzner HL, Lamphiear DE, et al. Characteristics of individuals and long term reproducibility of dietary reports: The Tecumseh Diet Methodology Study. J Clin Epidemiol. 1990;43:1169–1178. doi: 10.1016/0895-4356(90)90018-k. [DOI] [PubMed] [Google Scholar]
- 40.van Staveren W, West C, Hoffmans M, et al. Comparison of contemporaneous and retrospective estimates of food consumption made by a dietary history method. Am J Epidemiol. 1986;123:884–893. doi: 10.1093/oxfordjournals.aje.a114318. [DOI] [PubMed] [Google Scholar]
- 41.Ambrosini GL, van Roosbroeck SAH, Mackerras D, et al. The reliability of ten-year dietary recall: Implications for cancer research. J Nutr. 2003;133(8):2663–2668. doi: 10.1093/jn/133.8.2663. [DOI] [PubMed] [Google Scholar]
- 42.Byers T, Marshall J, Anthony E, et al. The reliability of dietary history from the distant past. Am J Epidemiol. 1987;125:999–1011. doi: 10.1093/oxfordjournals.aje.a114638. [DOI] [PubMed] [Google Scholar]
- 43.Jain M, Howe GR, Harrison L, et al. A study of repeatability of dietary data over a seven-year period. Am J Epidemiol. 1989;129:422–429. doi: 10.1093/oxfordjournals.aje.a115146. [DOI] [PubMed] [Google Scholar]
- 44.Linsted KD, Kuzma JW. Reliability of eight-year diet recall in cancer cases and controls. Epidemiology. 1990;1(5):392–401. doi: 10.1097/00001648-199009000-00010. [DOI] [PubMed] [Google Scholar]
- 45.Willett WC, Sampson L, Browne ML, et al. The use of a self-administered questionnaire to assess diet four years in the past. Am J Epidemiol. 1988;127:188–199. doi: 10.1093/oxfordjournals.aje.a114780. [DOI] [PubMed] [Google Scholar]
- 46.Maruti SS, Feskanich D, Colditz GA, et al. Adult recall of adolescent diet: reproducibility and comparison with maternal reporting. Am J Epidemiol. 2005;161(1):89–97. doi: 10.1093/aje/kwi019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Pietinen P, Hartman AM, Haapa E, et al. Reproducibility and validity of dietary assessment instruments. II. A qualitative food frequency questionnaire. Am J Epidemiol. 1988;128:667–676. doi: 10.1093/oxfordjournals.aje.a115014. [DOI] [PubMed] [Google Scholar]
- 48.Wolk A, Bergstrom R, Hansson LE, et al. Reliability of retrospective information on diet 20 years ago and consistency of independent measurements of remote adolescent diet. Nutr Cancer. 1997;29:234–241. doi: 10.1080/01635589709514630. [DOI] [PubMed] [Google Scholar]
- 49.Block G, Thompson FE, Harman AM, et al. Comparison of two dietary questionnaires validated against multiple dietary records collected during a 1-year period. J Am Diet Assoc. 1992;92:686–693. [PubMed] [Google Scholar]
- 50.Frazier AL, Willett WC, Colditz GA. Reproducibility of recall of adolescent diet: Nurses’ Health Study (United States) Cancer Causes Control. 1995;6:499–506. doi: 10.1007/BF00054157. [DOI] [PubMed] [Google Scholar]
- 51.Frobisher C, Tilling K, Emmett PM, et al. Reproducibility measures and their effect on diet-cancer associations in the Boyd Orr Cohort. J Epidemiol Commun Health. 2007;61:434–440. doi: 10.1136/jech.2006.046524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Gordeev K, Vasilenko I, Lebedev A, et al. Fallout from nuclear tests: dosimetry in Kazakhstan. Radiat Environ Biophys. 2002;41:61–67. doi: 10.1007/s00411-001-0139-y. [DOI] [PubMed] [Google Scholar]
- 53.Maynard-Tucker G. Conducting focus groups in developing countries: Skill training for local bilingual facilitators. Qual Health Res. 2000;10:396–410. doi: 10.1177/104973200129118525. [DOI] [PubMed] [Google Scholar]
- 54.American Psychological Association. Publication Manual of the American Psychological Association. 5. Washington, DC: American Psychological Association; 2001. [Google Scholar]
- 55.Cook TD, Campbell DT. Quasi-experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton-Mifflin; 1979. [Google Scholar]
- 56.Smith AF, Jobe JB, Mingay DJ. Retrieval from memory of dietary information. Appl Cogn Psychol. 1991;5:269–296. [Google Scholar]