Conducting Telephone Interviews with Community-dwelling Older Adults in a State Medicaid Program: Differences by Ethnicity and Language Preference

Abstract

We document the methodological challenges of conducting a health survey of an ethnically diverse elderly community-dwelling Medicaid population by telephone. Individuals (N=5,382) 65 years and older were randomly selected from a state Medicaid Management Information System and 618 eligible participants were interviewed. Participants were classified as non-Hispanic White, English-speaking (NHW-E; 69.2%), non-Hispanic Black, English-speaking (NHB-E; 6.2%), Hispanic, Spanish-speaking (H-S; 9.2%), and Hispanic, English-speaking (H-E; 4.2%). Almost half (44.2%) of the individuals sampled were unreachable, most often because of no valid telephone number. More interviewer time was required to reach and interview Hispanic participants. On average, interviews with H-S and H-E were 11 and 8 minutes longer, respectively, than with NHW-E. Spanish-speaking Hispanic respondents reported very high rates of receipt of preventive services relative to the other groups. These high rates by Spanish-speakers may be due to actual greater utilization or biases in self-reported data due to response style differences.

People age 65 and older presently constitute approximately 13% of the U.S. population. This age group is expected to grow to 20% of the population by 2030. While currently 81% of the elderly population is non-Hispanic White, that percentage is expected to decrease to 61% by 2050 as minority populations grow as a proportion of the U.S. population. More specifically, Hispanics are expected to triple their representation in the U.S. elderly population, from 6% in 2006 to 18% in 2050.¹

The most impoverished members of the elderly population are covered by Medicaid for medical and health-related services. In 2007, it was estimated that 33% of total Medicaid spending was on long-term care, with 58.5% of Medicaid long term care spending for institutional services and 41.5% for home and community-based services. Much of what is known about the Medicaid-covered older adult population emerges from studies of nursing home residents (e.g., institutional services).² Less is known about the older adult Medicaid population who live in the community. Furthermore, even less is known about older Hispanic adults in the Medicaid program, many of whom are immigrants and many of whom have health care needs and utilization patterns that differ from those of longer-term resident populations. To evaluate the needs of the Medicaid-covered population, the Center for Adult Health of the Rhode Island Medicaid Program commissioned a needs-assessment survey of individuals 65 and older who lived in the community and who spoke English or Spanish.

Telephone interviews generally present disadvantages for collecting data from older adults, including lower response rates, less interviewer assistance, and higher numbers of don’t know responses than in-person interviews.^3-5 Less is known about the likely effect of response styles in telephone interviews with elderly individuals, many of whom may have difficulty hearing and comprehending questions asked over the telephone. Response styles are tendencies to respond systematically to questionnaire items on some basis other than what the items were specifically designed to measure. In collecting evaluation data, participant response styles may influence outcomes. Response styles include acquiescence (a tendency to agree with all the questions or to indicate positive responses), extreme responding (a tendency for respondents to select both endpoints of a response scale), use of the middle response category on ratings scales, and socially desirable responding (a tendency to reply in a manner that will be viewed favorably by others).^6-8 Prevention and detection health behaviors are particularly susceptible to socially desirable responding,^9-12 and this may be especially relevant for elderly individuals covered by Medicaid because of the fear of loss of benefits for not complying with recommended testing.

Previous studies have demonstrated that Hispanics are more likely than other racial and ethnic groups to use extreme responses to scales and are more likely to express some acquiescence.^13-17 However, these studies have been conducted primarily with non-elderly populations. Less is known about the response styles of elderly Hispanics.

Despite the limitations of telephone interviews, the costs of in-person interviews are high and likely to be prohibitive to Medicaid programs interested in assessing the health and health care needs of community-based elderly populations. Therefore, our goal was to investigate the methodological challenges of surveying an ethnically diverse elderly Medicaid population by telephone. Specifically, our objectives were to determine (1) rates of eligibility and participation; (2) response rates and optimal number of contact attempts by ethnicity and language of the interview; (3) correlates of interview duration; and (4) prevalence estimates of self-reported prevention and detection behaviors by ethnicity and language of the interview.

Methods

Sample and recruitment

The survey was conducted for the Rhode Island Department of Human Services (RI DHS) as part of a comprehensive needs assessment of elderly (age 65 and older) Rhode Island Medicaid enrollees who live in the community. Medicaid eligibility for people age 65 and older is based on limited income and resources, and a variety of institutional and community-based services are covered for people who qualify on the basis of a disability. Thus, the community-dwelling elderly Medicaid population is a mix of individuals who qualify due to poverty and/or poor health. The vast majority of participants over the age of 65 are also eligible for Medicare and are therefore said to be dually eligible.

The needs-assessment survey consisted of well validated and reliable measures supplemented by information derived from focus groups with the target population. It was designed to provide information about health status, types and prevalence of health problems and conditions, access and barriers to health care, and unmet service need.¹⁸

The Rhode Island Medicaid Management Information System (MMIS) included a total of 13,460 eligible recipients 65 years of age or older, of whom 95% were also eligible for Medicare (dually eligible). The desired final sample size was 550, which was determined by the funding available to conduct the survey during the proposed six-month study period. Based on prior experience with the RI Medicaid population and the RI MMIS, we a priori chose to draw a sample that was 10 times the desired sample size. However, of the 5,500 randomly drawn Medicaid recipients, 118 records were not sufficiently complete for inclusion in the sample. Therefore, a total of 5,382 community-dwelling individuals 65 years or older were included (40.0% of the total eligible population). A letter on university letterhead explaining the study (written in English on one side and Spanish on the other) was mailed to all randomly selected individuals. From July to December 2006, interviewers made up to 10 telephone contact attempts to screen and interview participants. Our participant contact schedule included a random order from start to finish. Participants were contacted at different days and times throughout the study period such that participants received at least one contact on each day of the week except Sunday and received at least three evening contacts. The study was approved by the Brown University Institutional Review Board.

Data collection

Upon contact with a selected participant, preferred language for the interview was determined. Next, a description of the project was provided, informed consent was obtained, and eligibility was confirmed by bilingual interviewers. As part of the informed consent process, interviewers assessed each individual’s ability to hear and understand questions asked of them. If a selected participant was unable to answer questions by telephone, information about eligibility was obtained from another individual who answered the telephone. However, due to the subjective nature of many of the survey questions, proxy respondents were not allowed to complete the survey itself.

We administered a 150-item survey that included questions about health status, health care utilization, health service needs, and prevention and detection behaviors. In the analyses presented here, we focused on prevention and detection behaviors for two reasons. First, if response style differences were to be observed, these items were ones likely to be affected due to perceived social desirability of the behaviors.^9-12 Second, we could compare rates in the selected sample to those of individuals of similar age in the Rhode Island Behavioral Risk Factor Surveillance Survey (RI-BRFSS). Specifically, we asked participants if, in the past year, they had received a general physical exam, a flu shot, and an eye exam, as well as a check of their blood pressure, cholesterol, and blood glucose levels. In addition, we asked women if they had received a breast exam and Pap smear, and men if they had received a prostate screening.

Analysis

We first computed the eligibility rate and reasons for ineligibility among the individuals randomly selected from the Medicaid roster. Second, we summarized reasons for non-participation among the eligible sample. To determine the effect of ethnicity and language of the interview on response rates, we classified participants based on the interview data as non-Hispanic White, English-speaking; non-Hispanic Black, English-speaking; Hispanic, English-speaking; and Hispanic, Spanish-speaking. We then computed the overall response rate and cumulative response rates by number of contact attempts, stratified by ethnicity and language.

We next computed the average duration of the interview by ethnicity and language of the interview. We also computed a multiple regression model to assess the effect on interview duration of participant ethnicity and language, controlling for other participant and interviewer characteristics. Finally, we fit generalized linear models to assess the relationship between ethnicity/language of the interview and the responses provided by participants in the survey about prevention and detection behaviors, controlling for age, education, and gender (where appropriate).

Results

A total of 5,382 individuals 65 years or older were randomly selected from the Rhode Island Medicaid Management Information System (RI MMIS). Of these, 2,379 (44.2%) were ineligible for a telephone interview. The reasons for ineligibility were no valid telephone number (n=1,277, 53.7%), preferred language other than English or Spanish (n=578, 24.3%), cognitive impairment (n=177, 7.4%), away from the area for an extended period of time (n=133, 5.6%), institutionalized (n=86; 3.6%), hearing or speech too limited for a telephone interview (n=72, 3.0%), and death (n=56, 2.4%). Among those who were determined to be eligible, 1,209 individuals were too ill^* or refused participation,^† and 618 were interviewed. The remaining 1,176 were classified as unknown eligibility because we were still attempting to reach them at the end of the study period. Based on the American Association of Public Opinion Research (AAPOR) definitions, the response rate was 20.5% if all those with unknown eligibility are included as eligible, 26.4% if a portion of those with unknown eligibility are excluded, and 33.8% if all those with unknown eligibility are excluded.

Overall, of the 5,382 individuals selected from the Medicaid roster, only 8.1% were successfully contacted with a final disposition recorded within two attempts, 10.8% within five attempts, and 11.5% within 10 attempts. Of the 3,003 individuals who were determined to be eligible during the study period, 14.5% were contacted within two attempts, 19.4% within five attempts, and 20.6% within 10 attempts. On the other hand, 70.2% of the interviewed sample was contacted within two attempts and 94.2% within five attempts.

Among the 1,176 who were classified as unknown eligibility, an average of 4.3 (standard deviation = 2.5) contact attempts were made during the study period. By the end of the study, contacts were suspended for only 39 individuals for whom at least 10 contacts were attempted.

Our final interviewed sample size (n=618) was greater than the initial proposed sample size (n=550) because we were able to conduct more interviews than expected within the study period with the available resources. Given the age and health status of the population as well as the length of the questionnaire, we anticipated needing to split interview administration into multiple sessions for a substantial minority of participants. However, only 16 (2.6%) participants required two or more separate sessions to complete the survey.

Using information from the survey about ethnicity and language of the interview, 412 (66.7%) participants were classified as non-Hispanic White, English-speaking (NHW-E), 37 (6.0%) as non-Hispanic Black, English-speaking (NHB-E), 25 (4.0%) as Hispanic, English-speaking (H-E), and 55 (8.9%) as Hispanic, Spanish-speaking (H-S). The remaining 89 (14.4%) individuals identified themselves as being of another ethnicity and were not included in further analyses. Figure 1 shows cumulative response rates by ethnicity and interview language among participants who were interviewed. After two attempts, the cumulative response rates were 52.7% for H-S, 64.0% for H-E, 75.7% for NHB-E, and 70.6% for NHW-E (F=2.81, p<.05). By the fifth attempt, 92.7% of H-S, 96.0% of H-E, 91.9% of NHB-E, and 93.9% of NHW-E (F=0.18, p=.91) participants had been interviewed.

Figure 1.

Cumulative response rates for interviewed participants, by number of contact attempts for ethnicity and language preference groups.

Participant demographic characteristics by ethnicity and preferred language of interview are presented in Table 1. Hispanics were younger and less likely to be female than others. Spanish-speaking Hispanics had the lowest levels of education, while NHW-E were most likely to have attended or graduated from college.

Table 1. PARTICIPANT DEMOGRAPHIC CHARACTERISTICS BY ETHNICITY AND LANGUAGE OF THE INTERVIEW.

	Total (n=529)	Hispanic, Spanish Language (H-S) (n=55)	Hispanic, English Language (H-E) (n=25)	Non-Hispanic Black English Language (NHB-E) (n=37)	Non-Hispanic White English Language (NHW-E) (n=412)	Statistical Comparison
Age	73.9 (6.9)	71.7 (5.1)	71.3 (3.5)	72.1 (6.4)	74.6 (7.2)	F=5.25
(mean, std),  range 65–100						p=.0014
Gender	78.8	54.6	64.0	83.8	82.5	χ2=26.64 p<.0001
% Female
Education
Did not graduate from   high school	45.5	83.3	58.3	32.4	41.0	χ2=42.58 p<.0001
Graduated from high school,   but not college	25.6	7.4	16.7	43.2	26.9
Attended and/or graduated   from college	28.8	9.3	25.0	24.3	32.0

H-S = Hispanic/Spanish

H-E = Hispanic/English

NHB-E = Non-Hispanic Black/English

NHW-E = Non-Hispanic White/English

As shown in Table 2, the average interview time for all participants was 39 minutes. This differed by both ethnicity and language of the interview. The average interview time for Hispanics was 47 minutes, compared with 37 minutes for non-Hispanics. Similarly, interviews conducted in Spanish were on average 11 minutes longer than those conducted in English. More specifically, interviews with H-E and H-S ranged from 5.5 to 11 minutes longer than those with NHB-E and NHW-E.

Table 2. DURATION OF INTERVIEW BY ETHNICITY AND LANGUAGE OF INTERVIEW.

	Duration of Interview
Characteristic	Range	Mean	(Std)	Median
Overall sample (n=529)	19–106	38.9	(10.4)	37
Ethnicity
Hispanic (n=80)	25–90	47.2	(12.1)	45
Non-Hispanic (n=449)	19–106	37.4	(9.3)	35
Language of interview
Spanish (n=55)	25–90	48.4	(12.0)	45
English (n=474)	19–106	37.8	(9.6)	35
Ethnicity and language of interview
Hispanic, Spanish language (n=55)	25–90	48.4	(12.0)	45
Hispanic, English language (n=25)	25–70	44.7	(12.1)	42
Non-Hispanic Black, English language (n=37)	25–60	39.1	(7.4)	39
Non-Hispanic White, English language (n=412)	19–106	37.3	(9.4)	35

In multiple regression analyses controlling for participant age, gender, education, interviewer gender, and number of contact attempts, the average interview duration was 31 minutes (see Table 3). There were no differences in interview length between NHW-E and NHB-E. Interviews with H-S were 11 minutes longer (standard error 1.6, p<.001) and those with H-E were eight minutes longer (standard error 2.1, p<.001) than those with NHW-E. Interviews with H-S were slightly longer than for H-E (3.1 minutes, standard error 2.4, p=.19; data not shown in table). None of the other variables were associated with interview duration.

Table 3. MULTIPLE REGRESSION MODEL OF CORRELATES OF INTERVIEW DURATION (N = 529).

Characteristic	Beta	Standard Error
Intercept	30.89	4.87
Participant ethnicity and language of interview
Hispanic, Spanish language*	11.09	1.57
Hispanic, English language*	7.95	2.09
Non-Hispanic, English language, Black	2.34	1.69
Non-Hispanic, English language, White	Reference
Participant age	0.08	0.06
Participant gender
Female	1.79	1.08
Male	Reference
Participant education
Did not graduate from high school	−0.93	1.06
Graduated from high school	−1.73	1.17
Attended and/or graduated from college	Reference
Interview gender
Male	− 1.25	0.90
Female	Reference
Number of contact attempts	0.22	0.25

^*p<.001

Table 4 shows the relationship between ethnicity/language of interview and the responses provided by participants about prevention and detection behaviors, controlling for age, education, and gender. Spanish-speaking Hispanics were more likely than NHW-E to indicate positive responses to six of nine measures of prevention and detection in the past year [physical exam, flu shot, eye exam, breast exam (women only), Pap smear (women only), prostate screening (men only)]. They were also more likely than NHB-E to give positive responses for eye exam and Pap smear (women only). Compared with H-E, H-S were more likely to endorse having had a physical exam, flu shot, breast exam (women only), and prostate exam (men only). However, the differences were only statistically significant for breast exam. English-speaking non-Hispanic Blacks were more likely than NHW-E to report breast exam (women only) and prostate screening (men only), although the differences were only statistically significant for breast exam. There were no differences in participant reports of having their cholesterol, blood pressure, and blood glucose checked.

Table 4. PREVENTION AND DETECTION BEHAVIORS IN PAST YEAR BY PARTICIPANT ETHNICITY AND LANGUAGE OF INTERVIEW^a.

	Hispanic, Spanish Language (H-S) (n=55)	Hispanic, English Language (H-E) (n=25)	Non-Hispanic Black, English Language (NHB-E) (n=37)	Non-Hispanic White English Language (NHW-E) (n=412)	Overall, % reported in Rhode Island Medicaid sample (n=529)	% reported in Rhode Island 2006 BRFSS (n = 1255)
Physical exam (%)	98.1b	86.9	94.5	87.6b	88.9	93.8
Flu shot (%)	80.2b	67.5	68.9	64.9b	66.9	74.2
Eye exam (%)	92.6b,c	81.6	70.8	70.8b	73.7	—
Breast exam (% of women)	93.4b,c	67.9c	82.6d	59.4b,d	63.4	81.0
Pap smear (% of women)	72.6b,c	50.4	42.4c	27.2b	33.7	44.7
Prostate screening (% of men)	82.2b	74.8	80.8	50.6b	61.1	83.7
Cholesterol check (%)	87.2	91.0	85.6	86.0	85.4	—
Blood pressure check (%)	98.3	100.0	100.0	98.1	98.3	—
Blood glucose check (%)	88.4	95.6	83.4	83.1	84.1	—

^aAll models adjust for age, education, and gender (unless item is gender-specific)

^bPercentages with identical superscripts indicate a statistically significant difference (p<.05), based on results from generalized estimating equations.

^cPercentages with identical superscripts indicate a statistically significant difference (p<.05), based on results from generalized estimating equations.

^dPercentages with identical superscripts indicate a statistically significant difference (p<.05), based on results from generalized estimating equations.

— Data not available

H-S = Hispanic/Spanish

H-E = Hispanic/English

NHB-E = Non-Hispanic Black/English

NHW-E = Non-Hispanic White/English

BRFSS = Behavioral Risk Factor Surveillance Survey

Table 4 (last column) also shows the percentages of prevention and detection behaviors for comparably aged participants in the 2006 Rhode Island Behavioral Risk Factor Surveillance System (BRFSS). The rates reported by H-S were higher than those reported in the BRFSS for four (physical exam, flu shot, breast exam, Pap smear) of the five behaviors for which data were available. On the other hand, rates of preventive services for NHW-E were lower than those in the BRFSS for all behaviors for which data were available.

Discussion

We evaluated the use of telephone interviews for assessing the medical care and service needs of a low-income, diverse, community-dwelling population of elderly people covered by Medicaid. We randomly selected individuals from the Rhode Island Medicaid Management Information System (RI MMIS). Of the selected individuals, 85% were ineligible for the survey or unable to be contacted for a telephone interview. Almost 25% were ineligible because of invalid telephone numbers. For almost 2% of respondents, our initial letter explaining the study was returned as undeliverable. Our data suggest that elderly people covered by Medicaid may change residences and have other characteristics (e.g., periods without a phone) that increase the probability that administrative lists may be outdated. The extent to which out-of-date contact information in the Rhode Island MMIS is higher or lower than administrative data bases in other state Medicaid programs is unknown. In addition, we do not have data to document how mobility rates of Medicaid-covered community-dwelling seniors in Rhode Island compare to similar populations in other states. However, in a recent review of key studies about late-life residential relocations, Sergeant and colleagues¹⁹ documented mobility rates from 5% to more than 30% when the data were rescaled to a common five-year time period across several different samples of elderly individuals. Therefore, investigators planning for future studies using contact information contained in Medicaid administrative files should assume that a substantial proportion of the older adult population may not be reachable due to outdated and/or incorrect contact information.

We were also unable to interview 22% of the sample because of health limitations. The older adult personally refused the interview, usually because of not feeling well, or an individual answering the phone indicated that the older adult was too ill to participate in a phone interview. Unfortunately, we do not have any information to determine whether some of these older adults would have been able and willing to participate in a face-to-face interview but refused to participate in the study because of the type of interactions required for a 30–40 minute telephone interview.

Substantial interviewer time was required to reach and interview participants in this study. Within two contact attempts, the response rate was only 8% for the full sample and 15% for the eligible sample. This increased to only 11% and 19%, respectively, within five contact attempts and to only 12% and 21%, respectively, within 10 attempts. Within the interviewed sample, we reached 70% within two attempts and almost 94% within five attempts. Therefore, while we made up to 10 attempts to contact participants, response rates rose minimally after five attempts.

Response rates for the interviewed sample differed by ethnicity and language of interview. After two attempts, only 53% of the Hispanic Spanish-speaking participants had been interviewed, compared with 64% of Hispanic English-speaking, 76% of non-Hispanic Black English-speaking, and 71% of non-Hispanic White English-speaking participants. However, there were no differences in the interview rate by ethnicity and language by the fifth attempt; about 94% of all participants had been interviewed. This is consistent with a survey of influenza vaccination rates among a sample of non-institutionalized Medicare beneficiaries in which Srinath and colleagues²⁰ found no substantial reduction in non-coverage bias after six contact attempts relative to the increased costs. Consequently, future studies with limited budgets should consider the cost-effectiveness of making more than five to six attempts to contact poor, older community-dwelling adults, including Hispanic, Spanish-speaking elders.

One of the reasons that response rates may have been low was because respondents did not associate our initial contact letter on university letterhead with the evaluation of services they receive from the Medicaid program. Although the text of the letter referred to the Medicaid program, respondents may have not read the text in detail and, therefore, incorrectly assumed from the letterhead that the survey was not relevant to them. To increase the perceived relevance of the survey by potential participants, future studies should consider sending the initial contact letter on letterhead associated with the Medicaid program or on joint university/Medicaid letterhead.

More interviewer time was required to interview Hispanics than non-Hispanics. The interview was intended to be 30 to 40 minutes in length and the average interview duration overall was 39 minutes. However, even after controlling for factors that could account for a longer interview, including participant age, gender, education, interviewer gender, and number of contact attempts required, interviews with Hispanic English-speakers were seven minutes longer and those with Hispanic Spanish-speakers were 11 minutes longer than those with non-Hispanic White English-speakers. Unfortunately, we do not have data to explain the reason for these findings.

Interviews of longer duration for Hispanics compared with non-Hispanics are probably not a result of more reported physical and mental health needs. Hispanics generally reported better health, lower levels of need for help with activities of daily living, and less unmet need for health-related services than non-Hispanics.¹⁸ As a result, Hispanics were generally asked fewer questions overall due to questionnaire design features.

Although we did not systematically document questions that were raised during the interview, comments made during interviewer debriefings indicated that Hispanic respondents did frequently ask questions and ask for clarification of concepts. Although we translated and back-translated the instrument from English to Spanish, the translation may not have adequately addressed the different Spanish dialects spoken in Rhode Island, including those spoken by individuals originating in the Dominican Republic, Guatemala, and Mexico. It is likely that some of the concepts and interpretations may have been so culturally determined^21-23 that they required more explanation. This may partially explain why interviews were longer with Hispanic than with non-Hispanic respondents.

We found very high reported rates of preventive services among Hispanic Spanish-speakers. The extent to which these data are accurate or the result of response styles is unclear. Rates reported by Hispanic Spanish-speakers were substantially higher than for comparably aged participants in the Rhode Island Behavioral Risk Factor Surveillance System (BRFSS) for reports of preventive services. However, rates of preventive services for non-Hispanic English-speakers in our study were generally lower than for comparably aged individuals in the BRFSS, a finding that would be expected given that participants in our study had fewer resources and were more disabled than those in the BRFSS. Therefore, acquiescence and socially desirable responding may have been particularly prevalent for older, low-income, Spanish-speaking Hispanics. There are several potential reasons for these response styles. First, Spanish-speaking individuals may have been more reluctant than English speakers to report anything negative about themselves or services provided by the Medicaid program for fear of loss of benefits. Afghani and Johnson²⁴ noted potential loss of health benefits as one important barrier to recruitment and retention of Hispanic women in a study of obesity and hypertension. Second, less acculturated Hispanics may have been more likely to respond affirmatively due to simpatia, the expectation that interpersonal relationships should be guided by harmony and lack of confrontation.²⁵ Third, Spanish-speaking individuals may have been more likely to view the survey as a test and therefore attempt to give what they thought was the right answer or to play it safe in response to the question.^26,27

There are a number of study limitations. First, there may be important potential biases in the interviewed sample of which we are unaware, limiting the external validity of our findings. External validity is also reduced because more than 10% of the randomly selected sample was deemed ineligible because they spoke a language other than English or Spanish. Third, at the end of the study period, we were still attempting to contact almost one-quarter of the sample to determine eligibility. With additional contact attempts, some of these individuals might have been interviewed or ultimately deemed ineligible. Fourth, only a small percentage of Black and Hispanic participants were interviewed. Those Black and Hispanic participants who did agree to participate may have been atypical in their health status and willingness to participate in the survey. Finally, due to the small sample size of Hispanics, we were limited in our ability to assess reasons for the differences in the required number of contact attempts and interview duration by ethnicity and language of interview.

Despite these limitations, our findings provide important information for researchers and program planners who are considering conducting interviews with older adults enrolled in state Medicaid programs. Caution should be used with telephone interviews as the method of data collection. We found that more than three-quarters of randomly selected individuals from the RI MMIS were ineligible or unable to be contacted and interviewed by telephone. While the direct costs associated with telephone interviewing may be less, it is important to consider the overall cost-effectiveness given the potentially significant selection biases, threats to external validity, and response style biases associated with telephone interviews with older, frail adults. Alternative approaches to data collection must be considered to represent adequately the experiences of the growing diversity of the older adult population. These methodological considerations include multi-mode techniques such as mailed questionnaires with in-person follow-up if necessary. In addition, future evaluations should include surveys in languages other than English and Spanish and translated questionnaires that are culturally equivalent and appropriate.^28-31