Determinants of Quality of Interview and Impact on Risk Estimates in a Case-Control Study of Bladder Cancer

Abstract

The authors evaluated potential determinants of the quality of the interview in a case-control study of bladder cancer and assessed the effect of the interview quality on the risk estimates. The analysis included 1,219 incident bladder cancer cases and 1,271 controls recruited in Spain in 1998–2001. Information on etiologic factors for bladder cancer was collected through personal interviews, which were scored as unsatisfactory, questionable, reliable, or high quality by the interviewers. Eight percent of the interviews were unsatisfactory or questionable. Increasing age, lower socioeconomic status, and poorer self-perceived health led to higher proportions of questionable or unreliable interviews. The odds ratio for cigarette smoking, the main risk factor for bladder cancer, was 6.18 (95% confidence interval: 4.56, 8.39) overall, 3.20 (95% confidence interval: 1.13, 9.04) among unsatisfactory or questionable interviews, 6.86 (95% confidence interval: 4.80, 9.82) among reliable interviews, and 7.70 (95% confidence interval: 3.64, 16.30) among high-quality interviews. Similar trends were observed for employment in high-risk occupations, drinking water containing elevated levels of trihalomethanes, and use of analgesics. Higher quality interviews led to stronger associations compared with risk estimation that did not take the quality of interview into account. The collection of quality of interview scores and the exclusion of unreliable interviews probably reduce misclassification of exposure in observational studies.

A major concern in observational epidemiologic studies is obtaining reliable information through questionnaires or personal interviews, reflecting accurate levels of exposure to the risk factors being investigated in the study population. Questionnaires should be designed and administered properly to collect accurate and unbiased responses (1–3). Training and quality control of the interviewers’ performance during the fieldwork are crucial and have been the object of methodological evaluations (4–6).

The individual characteristics of the respondents can potentially determine the quality of the information obtained. The cultural background determined by the ethnicity of the respondents may influence comprehension of the questions (7), and the respondent's behavior, such as hesitation and interruptions, has been related to more inaccurate responses (8). However, evidence is sparse regarding the role of sociodemographic factors over the quality of data collected in personal interviews. In addition, there is little evidence on the potential impact of the quality of data on the results.

We evaluated respondents’ sociodemographic characteristics, self-perceived health, and stage/grade of cancer as potential determinants for the quality of the data collected through personal interviews in a case-control study of bladder cancer. We also assessed the impact of the quality of interview on the risk estimates for a variety of bladder cancer risk factors.

MATERIALS AND METHODS

Study design and population

We conducted a multicenter, hospital-based, case-control study of bladder cancer between June 1998 and June 2001 in Spain. Study subjects were recruited at 18 hospitals: 3 in Barcelona, 2 in Vallès/Bages, 1 in Alicante, 2 in Tenerife, and 10 in Asturias. Cases were identified at diagnosis through the hospital urologic services and were defined as patients with a histologically confirmed primary carcinoma of the urinary bladder, between 20 and 80 years of age, and living in the catchment area of the participating hospitals. In addition to registers from urologic services as a source of cases, complete case ascertainment was secured by regular and frequent evaluations of hospital discharge records, pathology records, and local cancer registries. Controls were patients admitted to the participating hospitals around the time the case patients were diagnosed. They were matched individually to cases by gender and age group (5-year strata) and by area of residence. Controls were selected from patients admitted to the hospitals with diagnoses thought to be unrelated to the risk factors under investigation, such as tobacco use. The control group was admitted for the following diagnoses: hernias (37%), other abdominal surgery (11%), fractures (23%), other orthopedic problems (7%), hydrocele (12%), circulatory disorders (4%), dermatologic disorders (2%), ophthalmologic disorders (1%), and other diseases (3%). The study protocol was approved by the institutional review boards of the participating centers. All subjects provided a signed informed-consent form.

Interviews

Trained interviewers administered a structured computer-assisted personal interview (CAPI) to participants during their hospital stay. Information included sociodemographic characteristics; smoking habits; occupational, residential, and medical histories; and family history of cancer (questionnaire available online: http://www.creal.cat/fitxers/epicuro/epicuro%20questionnaires.htm). A food frequency questionnaire was self-administered. Subjects who refused to answer the CAPI were administered an abridged interview of critical items. We identified 1,457 eligible cases and 1,465 eligible controls. Among them, 84% of cases and 87% of controls responded to the questionnaire. From all the respondents, 21% of cases and 19% of controls answered the abridged questionnaire. The total study population including those responding to the abridged questionnaire was 1,219 cases and 1,271 controls.

Quality of interview

The CAPI included a final section to be completed by the interviewer after each interview. This section included questions on the respondent's perceived cooperation (“Respondent's cooperation was 1) very good, 2) good, 3) fair, 4) poor”); the perceived quality of the interview by section and overall quality of the interview (“The overall quality of this interview is 1) unsatisfactory, 2) questionable, 3) generally reliable, 4) high quality”); and the main reasons for unsatisfactory or questionable quality of information (“The main reason for unsatisfactory or questionable quality of information was because the respondent 1) did not know enough information regarding the topic, 2) did not want to be more specific, 3) did not understand or speak Spanish well, 4) was bored or uninterested, 5) seemed upset or depressed, 6) was physically ill, 7) had poor hearing or speech, 8) was confused by frequent interruption, and 9) other—specify. Enter codes for all that apply”).

Exposures

We selected exposures with consistent evidence of association with bladder cancer: smoking (9, 10), occupational exposures (11), drinking water disinfection by-product exposure (12, 13), and regular use of analgesics and nonsteroidal antiinflammatory drugs (NSAIDs) as a protective factor (9, 14). Smoking status was classified as never, occasional, former, and current smokers. Current and former smokers were defined as those who smoked at least 1 cigarette/day during 6 months or longer. Smokers who quit 1 year ago or more were defined as former smokers. Subjects grouped as never/ever worked in a priori high-risk occupations (15). Trihalomethanes are the most prevalent by-products of disinfection with chlorine and were used as markers of drinking water disinfection by-product exposure. We calculated the average level of trihalomethanes in the households from the age of 15 years until the time of interview, on the basis of the lifetime addresses obtained from the residential history combined with the retrospective annual average trihalomethane level modeled in the study municipalities from measurements in drinking water, history of water source, and chlorination status (16). Subjects were grouped by using the median level among controls. Consumption of NSAIDs was included to represent a variable from the medical history section. Subjects were grouped as never, occasional, or regular long-term users.

Statistical analysis

To evaluate potential determinants of the quality of interviews, we grouped the overall quality of the interview in 2 categories (unsatisfactory/questionable and reliable/high) and calculated the percentages and chi-squared P values by personal characteristics in cases and controls and the grade and stage in cases. To estimate the effect of the quality of interview on the risk estimates, we calculated the odds ratio and 95% confidence interval of bladder cancer for the evaluated exposures adjusting for age, sex, and area among 1) all study subjects ignoring quality of interview, 2) unsatisfactory or questionable interviews, 3) reliable interviews, and 4) high-quality interviews. In addition, we calculated odds ratios for all study subjects adjusting for overall quality of the interview. We used hierarchical models (17) with random interviewer effects to account for the fact that respondents were nested within interviewers. Analyses were conducted with STATA, version 10, software (StataCorp LP, College Station, Texas).

RESULTS

The proportion of study subjects with poor quality of interview (unsatisfactory or questionable) was 8% (Table 1). The main reasons for poor quality of information were because the interviewee did not want to be more specific (n = 49, 24%) or he/she did not know enough information regarding the topic (n = 45, 22%). Other reasons were the following: did not understand or speak Spanish well (n = 26, 13%), was physically ill (n = 13, 6%), had poor hearing or speech (n = 12, 6%), was confused by frequent interruption (n = 7, 4%), was bored or uninterested (n = 4, 2%), and other reasons (n = 46, 23%). By specific sections, the highest proportion of high quality of interview was for sociodemographics (25%), and the lowest was for occupational history (18%). Interviewers tended to score the quality of specific sections higher than the overall quality of the interview. The proportion of subjects with high overall quality was 16% compared with the range of 18%–25% for specific sections.

Table 1.

Overall Quality of Interview and Section-specific Quality (N = 2,490), Spain, June 1998–June 2001

Quality of Information	Overall Interview		Section
			Sociodemographics		Smoking		Occupational History		Residential History		Medical History
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
Unsatisfactory	26	1	19	1	13	1	20	1	13	1	12	1
Questionable	176	7	71	3	213	9	252	10	139	6	136	5
Reliable	1,823	76	1,712	71	1,699	70	1,698	71	1,742	73	1,738	73
High	390	16	613	25	486	20	421	18	488	20	495	21
Missing/don't know/missing section	75		75		79		99		108		109

Lower quality interview scores were found with increasing age, poorer self-perception of health, and low socioeconomic status, measured in both years of education and income. Age was correlated with education and health status, with older subjects having lower education and income and worse health status. The overall quality of the interview score was not associated with case-control status, gender, or marital status. Stage and grade among cases were not associated with the overall quality of the interview score (Table 2). The proportion of poor-quality interviews was higher among critical items interviews, with 13% of unsatisfactory or questionable interviews compared with 8% in nonabridged interviews. The quality of interview was correlated with the respondent's cooperation. Among subjects with a reliable/high quality of interviews, 89% had very good or good cooperation compared with 46% among the respondents with unsatisfactory or questionable interviews.

Table 2.

Overall Quality of Interview by Sociodemographic Factors, Health Status, and Grade and Stage (Cases), Spain, June 1998–June 2001

	Unsatisfactory/ Questionable (N  = 202)		Reliable/ High (N  = 2,213)		χ2P Value
	No.	%	No.	%
Case-control status
Cases	109	9	1,064	91
Controls	93	7	1,149	93	0.109
Gender
Men	182	9	1,926	91
Women	20	7	287	93	0.210
Age, years
<55	11	3	361	97
55–64	32	6	520	94
65–69	40	7	505	93
70–74	55	11	445	89
≥75	64	14	382	86	<0.01
Health status, self-perceived
Excellent	7	5	130	95
Very good	12	6	198	94
Good	43	5	891	95
Fair	44	9	464	91
Poor	13	10	119	90	0.012
Missing	83		411
Education
Less than primary	122	11	992	89
Less than high school	52	6	882	94
High school	20	6	309	94
Other	8	27	22	73	<0.001
Missing	0		8
Income, pesetas/month
≤100,000	46	8	540	92
100,001–200,000	25	3	751	97
200,001–300,000	10	4	267	96
>300,000	7	4	162	96	0.010
Missing	114		493
Marital status
Single	20	11	157	89
Married	115	7	1,457	93
Widowed	17	9	182	91
Divorced	5	6	81	94	0.339
Missing	45		336
Stage (cases)
T1	12	9	126	91
T2	19	14	119	86
T3	6	10	52	90
T4	5	10	47	90
Ta	58	8	656	92
Tis	2	33	4	67	0.120
Missing	7		60
Grade (cases)
Benign	4	10	36	90
G1	30	10	280	90
G2	21	7	299	93
G3	47	11	389	91	0.254
Missing	7		60

Abbreviations: Ta, noninvasive papillary carcinoma; Tis, carcinoma in situ.

Overall odds ratios for bladder cancer by smoking status were higher among subjects with high-quality interview scores and were highest among those whose smoking section was scored high quality (odds ratio = 7.87 for current compared with never smokers). Odds ratios and 95% confidence intervals for interviews scored as unsatisfactory or questionable were generally lower than those for reliable and high-quality interviews (Table 3). The odds ratio for current versus never smokers compared with the odds ratio for not taking quality of interview into account was 12% higher in high-quality interviews and 36% lower in poor-quality interviews.

Table 3.

Bladder Cancer Odds Ratio and 95% Confidence Interval for Smoking Status, Adjusted for Area, Sex, Age, and a Component for Random Interviewer Effects, Spain, June 1998–June 2001

Smoking	All Subjects (N  = 2,394)		Overall Quality of Interview Score
			Unsatisfactory/Questionable (n  = 195)		Reliable (n  =  1,810)		High (n  = 389)
	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval
Never	1.00		1.00		1.00		1.00
Occasional	1.42	0.93, 2.18	0.95	0.22, 4.07	1.40	0.86, 2.29	2.60	0.80, 8.51
Former	3.27	2.43, 4.40	4.79	1.71, 13.40	3.13	2.21, 4.43	4.48	2.12, 9.44
Current	6.18	4.56, 8.39	3.20	1.13, 9.04	6.86	4.80, 9.82	7.70	3.64, 16.30
			Smoking Section Quality Score
			Unsatisfactory/Questionable (n = 220)		Reliable (n  = 1,687)		High (n  = 485)
			Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval
Never			1.00		1.00		1.00
Occasional			2.69	0.61, 11.83	1.44	0.87, 2.39	1.36	0.45, 4.10
Former			2.68	1.00, 7.14	3.35	2.31, 4.84	4.14	2.20, 7.78
Current			2.54	0.94, 6.88	7.21	4.92, 10.56	7.87	4.11, 15.07

The magnitude of the association among having ever worked in high-risk occupations, average lifetime residential trihalomethane level, and regular consumption of NSAIDs was lower among unsatisfactory or questionable interviews compared with reliable or high-quality interviews (Table 4). The odds ratios for reliable or high-quality interviews compared with overall increased 2% for occupation and 6% for trihalomethane exposure but decreased 12% for NSAID consumption. In contrast, among subjects with poor-quality interviews, the odds ratios decreased 6% for occupation and 48% for trihalomethane levels; for NSAID consumption that was associated with a protective effect, the odds ratio increased 109%.

Table 4.

Bladder Cancer Odds Ratio and 95% Confidence Interval for Different Etiologic Factors, Adjusted for Smoking Status, Area, Sex, Age, and a Component for Random Interviewer Effects, Among All Study Subjects and by Overall Quality of Interview, Spain, June 1998–June 2001

	Occupation
	All Subjects (N  =  2,315)		Unsatisfactory/Questionable Interviews (n  = 186)		Reliable/High Quality Interviews (n  = 2,129)
	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval
Non-high risk	1.00		1.00		1.00
High risk	1.51	1.21, 1.89	1.47	0.66, 3.29	1.52	1.21, 1.92
	Average Residential Trihalomethane Levels
	All Subjects (N  = 1,536)		Unsatisfactory/Questionable Interviews (n  = 91)		Reliable/High Quality Interviews (n  = 1,445)
	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval
≤26 μg/L	1.00		1.00		1.00
>26 μg/L	1.76	1.25, 2.46	1.34	0.29, 6.28	1.82	1.28, 2.58
	Nonsteroidal Antiinflammatory Drugs
	All Subjects (N  = 1,749)		Unsatisfactory/Questionable Interviews (n  = 115)		Reliable/High Quality Interviews (n  = 1.634)
	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval	Odds Ratio	95% Confidence Interval
Never	1.00		1.00		1.00
Long-term users	0.49	0.23, 1.03	1.07	0.06, 18.98	0.45	0.21, 0.98

Adjustment for the quality of the interview modified the risk estimates minimally, and the quality-of-interview variable was not significant in the models (P > 0.13). The odds ratio of 6.18 among current smokers compared with never smokers increased to 6.26 after adjustment for overall quality of the interview and to 6.27 after adjustment for the quality score specific for the smoking section. Odds ratios for ever working in high-risk occupations, average residential trihalomethane level of >26 μg/L, and NSAID consumption were, respectively, 1.51, 1.75, and 0.48 after additional adjustment for overall quality of the interview.

DISCUSSION

Age, socioeconomic status, and the self-perceived health of respondents were identified as determinants of the quality of the interview. Poor-quality interview scores decreased the magnitude of the association for several risk factors for bladder cancer, as compared with the effects observed on the basis of reliable or high-quality interviews. Comparison of risk estimates among the strata of quality of interview is hampered by the small numbers in the extreme categories, particularly unsatisfactory/questionable interviews, and chance cannot be ruled out. However, an effect is clearly suggested by the systematic differences in risk estimates when comparing higher with lower quality interviews. This suggests an effect of nondifferential misclassification that results in estimates toward the null for both risk factors (smoking, occupation, trihalomethanes) and protective factors (NSAIDs).

Subjectivity of the quality of interview score is a potential concern. In our study, interviewers were trained to administer interviews in a standardized manner to avoid bias, interviewers were retrained regularly, periodic quality control was performed on fieldwork by checking tape-recorded questionnaires, and feedback was regularly provided to interviewers. However, performance and the perception of quality of the interview are likely to differ among interviewers. To minimize this effect, we conducted hierarchical models with random interviewer effects. The results were equivalent to those from logistic regression adjusting for interviewer. However, as the number of categories was considerable (n = 17 interviewers), hierarchical models were more appropriate. Given the subjective nature of quality-of-interview scores, the collection of perceived quality of interview should be a complement rather than a substitute for standard quality control procedures.

In our study, the overall quality of the interview score was similar in cases and controls even though the P value for the difference was marginally statistically significant (P = 0.11). Additional adjustment for quality of interview modified the point estimates minimally and was not significant in the models (P > 0.13). Consequently, confounding is not a likely explanation for the effect of quality of interview on the risk estimates. The most probable reason for the trend of odds ratios toward the null with decreasing quality of interview is that of nondifferential exposure misclassification. It is plausible that older subjects who appear to have less education and poorer health status have more difficulties in remembering past events and providing accurate answers to specific questions. Hesitation and vague answers among those subjects would in turn be perceived by the interviewers and coded as questionable information.

Although the collection of reliable data from questionnaires in epidemiologic studies is essential, the epidemiologic literature on the topic is sparse and based mostly on studies from the social sciences (1, 3, 8). Evaluations done in public health research are focused mainly on the determinants of nonresponse or quality control in the data-collection stage. To our knowledge, this is the first study evaluating the effect of the quality of the interview on the risk estimates, and we, therefore, cannot compare our findings with previous evaluations. Replication of our findings is warranted for confirmation, preferably including similar questions to allow comparisons. We encourage the evaluation of interviews as a general practice in epidemiologic studies, to examine the effect of the variable on the results and to consider the exclusion of unreliable interviews.

In conclusion, poorer quality interviews were associated with increasing age, lower socioeconomic status, and poorer health status. Reliable and high-quality interviews led to stronger associations than did risk estimates where the quality of the interview was not taken into account. The collection of quality-of-interview scores and the exclusion of unsatisfactory/unreliable interviews probably reduce misclassification of exposure in observational studies.

Glossary

Abbreviations

CAPI

computer-assisted personal interview

NSAID

nonsteroidal antiinflammatory drug