Methodological issues in a cross-sectional survey on cervical cancer screening using telephone interviews in Sicily (Italy): a SWOT analysis

Short abstract

Objective

A cross-sectional study on knowledge, perceptions, and adherence to cervical cancer screening was conducted using telephone interviews of Sicilian women that were performed in 2016. This study aimed to identify areas that need to be addressed to improve the validity of data collection and to minimize possible biases.

Methods

We performed a qualitative study through SWOT analysis, which is a multidimensional method based on evaluation of Strengths (S), Weaknesses (W), Opportunities (O), and Threats (T) of the research project. The contents of the SWOT forms underwent categorical, inductive, and deductive data analysis using the long table analysis method.

Results

The full availability of an updated address and phone number list was the main organizational aspect to be addressed. Socio-cultural context played a major role for understanding the questions and for acceptability of the topics. In some cases, a family member was a facilitating element, while in others, the family member hindered the interviews. Active involvement of general practitioners was considered essential for success of the interviews.

Conclusions

When performing a cross-sectional survey, organizational aspects and active involvement of general practitioners are crucial in the enrolment phase, regardless of the socio-cultural context.

Introduction

Cervical carcinoma is a leading cause of cancer in women aged 15 to 44 years.^1–4 To prevent human papilloma virus (HPV)-related diseases in 2006 and 2007, quadrivalent and bivalent vaccines against oncogenic HPV serotypes (16 and 18) were approved in Europe. Moreover, in 2015, a nonavalent vaccine that extended antigens to five additional HPV types to that contained in the quadrivalent vaccine was approved by the European Medicine Agency.^5,6

In Europe, a 30% reduction in cervical carcinoma mortality was observed in the last 30 years. This reduction was independent from the vaccination strategy adopted, particularly among women aged older than 65 years,⁷ in light of improvement in cancer therapy, and especially because of the effectiveness of the screening strategy (Pap test).^8,9 In Italy, approximately 80% of women aged 25 to 64 years undergo the Pap test by an organized screening program provided by the National Health System or personal initiative.¹⁰ Sicily, which is one of the most populous Italian regions, has a low adherence, despite organized screening programs that have been in place for several years.¹¹ Therefore, improvement of cervical screening adherence was included as a major target of the Sicilian Regional Plan for Prevention 2014–2018.¹²

Telephone interviews are often used in public health as a data collection tool for qualitative and quantitative research. Telephone surveys have the advantage of being shorter, less expensive, more standardized, requiring less training¹³ and allow reaching geographically dispersed subjects, even in rural areas, compared with face-to-face interviews.^14,15 However, telephone surveys also have some potential constraints, such as low representativeness, a high rejection rate, low responsiveness, and difficulties in administration of multiple-choice questions.¹⁶ Additionally, standardized questionnaires are not adaptable to different levels of health literacy that can be found in the general population.^17–20 In Italy, particularly in the South and the islands, there is a low level of literacy and numeracy in the adult population compared with Organization for Economic Co-operation and Development countries.²¹

A cross-sectional study was conducted (“Save Eva in Sicily” - SES survey) on knowledge, perceptions, and adherence to cervical screening in Sicilian women to promote evidence-based public health strategies to enhance the screening program in 2016²² (Table 1). On the basis of the above-mentioned concerns about surveys, using the SES survey data, researchers conducted SWOT analysis, and described the Strengths (S), Weaknesses (W), Opportunities (O), and Threats (T) of the data collection process. The current study aimed to identify areas that need to be addressed to improve the validity of data collection and to minimize possible bias in future research in the field of cervical cancer prevention.

Table 1.

The Save Eva in Sicily Survey 2016²².

General characteristics of the survey	From January to June 2016, 15 public health medical residents carried out a telephone survey among Sicilian women aged 25–64 years.
Sample enrolled	The sample of women was extracted from the lists of general practitioners (GPs) practicing in nine Sicilian Local Health Units (LHUs), using a proportional sampling scheme, stratified by age, in a total of 1,395,866 resident women. Women who matched the exclusion criteria, those who had an incorrect phone number, or those who did not answer after three contacts were replaced with a substitute with the same characteristics, and were selected with the same sampling procedure.
Survey conduction techniques	An informative letter, describing the purpose of the survey, was sent to all sampled women and their GPs. Training of all interviewers was performed to standardize the procedures and to ensure the quality of data collection. A questionnaire, which was organized into four sections (2 mutually exclusive) and 30 items, was administered to the study sample during an interview of 10–15 minutes. Before the beginning of the study, the questionnaire was validated in a convenience sample that represented approximately 10% of the target population.At the beginning of the interview, survey aims were explained, as well as methods used to ensure confidentiality of data. Informed consent was further obtained. At the end of the interview, women received health advice on preventing cervical cancer, the screening program, and healthy lifestyles, if requested.
Main results	Among the 365 interviewed women, 66% (n = 243) had a Pap test during the last 3 years. A total of 18% of the other women (n = 66) had performed at least one Pap test previously and 16% (n = 56) had never had a Pap test. In a multivariable model, GPs’ advice (adjusted odds ratio 2.55; 95% confidence interval 1.57–4.14) and perceived susceptibility (adjusted odds ratio 3.24; 95% confidence interval 1.92–5.48) increased the likelihood of having a Pap test.

Material and Methods

A qualitative study of the SES survey was conducted through SWOT analysis, which is a multidimensional method for strategic planning based on evaluation of the four main categories of strengths, weaknesses, opportunities, and threats. These are both internal attributes and external conditions of the research project.²³ SWOT analysis was carried out by 15 public health medical residents (research team) of the Department of Science for Health Promotion and Mother and Child Care of the University of Palermo. These medical residents were involved as interviewers during the SES survey.

After data collection, the interviewers carried out SWOT analysis, and examined 10 aspects of the data collection process as follows: (1) an informative letter, (2) information on the survey provided by a general practitioner (GP), (3) interaction with relatives, (4) women’s propensity to the interview, (5) understanding of the language used in the questionnaire, (6) attitude in dealing with questions about cancer and death, (7) attitude in dealing with questions about sexuality, (8) the interview as an opportunity for the woman to obtain information, (9) other relevant issues, and (10) other factors favoring/hindering the interview.

A SWOT analysis form was filled out by the interviewers, individually or in groups, for each Local Health Unit (LHU), including Agrigento, Caltanissetta, Catania, Enna, Messina, Palermo, Ragusa, Siracusa, and Trapani. Data analysis was carried out using the long table method, as described by Krueger and Casey.²⁴ The long table method is a systematic and verifiable qualitative approach without the need of computer technology, with manual categorization and classification of the emerging contents.²⁴ The text strings were systematically read, coded, and classified into categories, according to internal coherence, within the four main areas of SWOT analysis.

Any ambiguous interpretation of data was solved through discussion within the research team. Categories that included both strong and weak points were indicated by the symbol (+/−). The research team promoted a reflexive dialogue because they knew the study processes and had the opportunity to make in-depth considerations. Content saturation was reached and the study design and results were reported according to the COREQ check-list²⁴ for all items applicable to SWOT analysis.

The study was approved by the ethical committee of the University Hospital “Paolo Giaccone” of Palermo (Palermo 1) on 11 October 2017 (protocol number: 10/2017). Consent to participate in the survey was obtained through an oral consent form that was approved by the same ethical committee.

Results

From June to July 2016, 10 SWOT analysis forms were filled out by the interviewers. A total of 365 Sicilian women were surveyed. All contents were classified according to the four main areas of SWOT analysis (Table 2).

Table 2.

Results of the SES survey, which were divided into the four SWOT areas, and sorted according to the timeline of the process that they were connected to.

	STRENGTHS	WEAKNESSES
Internal	Preliminary sending of an informative letter Presentation of the interviewer as a doctor belonging to a public institution (LHUs, University, etc.) Reference to involvement of GPs of the women at the beginning of the interview Facilitating the attitude of the interviewer Possibility to postpone the interview to another date/time (+/−) Request for participation aimed at improving the screening service	Layout of the informative letter was similar to the screening invitation letter Some questions were difficult to understand (level of agreement/disagreement with the Likert scale) Some questions were perceived as repetitive Lack of involvement of GPs in the process of the investigation protocol Possibility to postpone the interview to another date/time (+/−)
	OPPORTUNITIES	THREATS
External	Women’s interest of the topic and the need/desire to understand it Interaction with family members of the women before the interview (+/−) Personal or family history associated with neoplastic disease (+/−) Opportunity for personalized information (at the end of the interview)	Unavailability of phone numbers in a certain percentage of women to be interviewed Delay or failure to deliver the informative letter of the screening program Interaction with family members of the woman before the interview (+/−) Personal or family history associated with neoplastic disease (+/−) Low health literacy Socio-cultural background Lack of trust in the health service reported by some women interviewed

SES: Save Eva in Sicily; SWOT: Strengths, Weaknesses, Opportunities, and Threats; LHU: Local Health Unit; GP: general practitioner

Strengths

The pre-interview informative letter was considered appropriate for “avoiding mistrust and in facilitating consent for the interview”. In some cases, this was considered effective in conveying the content and the purpose of the survey. Presentation of the interviewer as a medical doctor belonging to a public institution (e.g., LHUs, University) was helpful for gaining trust of the women. Quoting the GP’s name at the beginning of the interview “was really reassuring, making her (the woman) more likely to respond to the questions.” An empathetic attitude of the interviewer, aiming to listen and understand women’s opinions, and in some cases, difficulties in talking about sensitive topics, had positive effects and facilitated the interviews. Being aware of one’s involvement in improving the health service, as expressed by the interviewer at the beginning of the call, encouraged women to participate in the interview. In several cases, the telephone interview was considered as a moment of encounter between the local health service and the citizen, and an opportunity to dispel doubts and unresolved questions. The survey was perceived by the women as a demonstration of interest towards the needs of the person. In those cases where the GP personally informed the women about the survey, the responders were more motivated and willing to answer questions. However, the proactive attitude of the GPs was assigned to their spontaneous initiative.

Weaknesses

In some cases, the informative survey letter was mistaken for the letter of invitation to the screening. In fact, when asking the question “Have you ever received a letter by the LHU that invited you to cervical screening (the smear test)?”, some women answered “yes.” However, afterwards, it became clear that the women were referring to the informative survey letter.

Great difficulty was found in obtaining coherent responses to the five options in the Likert scale from “complete agreement” to “complete disagreement”. Women often misunderstood the purpose of the question and gave a notional response, thus requiring more time to repeat the questions and answers. This lengthened the interview, and in some cases, caused interruption of the call. Formulation of these questions in some cases intimidated the woman, who did not feel like answering, and she replied with phrases such as “I'm not a doctor.” Difficulty in understanding the difference between some questions, which were similar to each other in the Italian formulation, occurred (e.g., “I know at what age I should have a Pap test” and “I know how often I should do a Pap test”). In some cases, these repetitions caused annoyance, resulting in the feeling of “time-wasting.”

Involvement of GPs was limited to retrieval of telephone numbers that were not tracked through other sources and the offer of information to women for more details on cervical screening.

The telephone mode was rarely considered inappropriate for the sensitivity of topics (e.g., “[…] these are not questions to be asked by phone”). Sometimes, a particular communicative ability by the interviewers was required to overcome and limit the initial distrust. This occurred more frequently in the case of interviewers without a Sicilian accent or a different accent from that of the woman’s province, and aroused more suspicion because of confusion with call center operators.

Opportunities

Great curiosity and requirement of information were perceived in terms of knowledge of practical information and prevention issues (e.g., location of screening services, free access, utilities, timing and type of the test). This attitude acted as an empowering element and allowed conduction of the interview in a participatory and smooth manner. Sometimes the first telephone contact was made with a family member of the woman. In some cases, the role of the family was facilitating, both in finding the person (suggesting a timetable or the exact number) and urging participation in the interview. However, interaction with family members in some cases resulted in ambivalent roles (see below, threats/external risks). Every interview ended with a time for comments and questions. This was often used as a dedicated time for counseling, and information about the official telephone numbers (toll-free) to contact and institutional websites to consult (LHU sites).

Threats

The institutional sources that were available to retrieve the women’s phone numbers were not recently updated. In some cases, the numbers provided were wrong, non-existent, or no longer active. Assigning a phone number for women was sometimes not even possible. The difficulty in finding phone numbers during the data collection phase prolonged the survey time and reduced the rate of compliance. This increased the number of surveyed women who were not randomly selected (substitutes) and this information needed to be retrieved at a later time from a list by the LHU.

In some cases, not receiving the invitation letter led to mistrust in adhering to the interview. Sometimes some medical terms were not understood or were misunderstood (e.g., HPV test, fecal occult blood test). Questions in which the words “death” and “cancer” or terms related to sexuality were unwelcome to some women, and caused embarrassment and refusal to continue (e.g., “I will not answer these intimate questions”). In one case, obtaining consistent answers was difficult because of conviction that our health condition is predestined (“only believe in fate/if it has to happen, it happens/is useless”). Distrust in the regional health system regarding waiting times and the incorrect belief of cost of services offered (services are accessible free of charge within the public health system) reduced the propensity of some women for participating in the interview.

The possibility provided by the research protocol to postpone the interview resulted in various outcomes. In some cases, this turned out to be a negative element because it could be used as an opportunity to deny the interview, refuse further calls, or not be available anymore. However, in many cases, this option was necessary to avoid losing a considerable part of the women who were unable to complete the interview at first contact. Many family members were hesitant in facilitating contact with the woman to participate in the interview for several reasons. These reasons included fear that the interview was a scam and mistrust for possession of the phone number (e.g., “Who gave you the number of my daughter?”). Another reason was that, despite having received the information letter, there was reluctance for the topics “these are not questions to be asked by phone” and “we do not talk of these things at my home.”

Discussion

Telephone surveys are still an essential tool in prevention research. Nevertheless, the use of information and communication technologies by the general population has greatly changed in the last decade (e.g., a reduction in landlines and the massive diffusion of smartphones, frequent phone number changes, and an increase in unwanted calls for marketing purposes).^14,26,27 These are new challenges for healthcare systems that base their prevention and research activities on individual phone calls, and they require new methodological responses.²⁸

SWOT analysis is mainly used in strategic planning.²⁹ In our experience, SWOT analysis can be profitably used as a structured tool to analyze the methodological aspects of research in public health. SWOT analysis is performed starting from the experience of the interviewers, with particular reference to the process of data collection during telephone interviews. This analysis aimed to improve the methodology and reduce possible biases (e.g., response bias and representativeness bias) in future research in the field of cervical cancer prevention.

In our analysis, the organizational aspects to be addressed included the availability of an updated address and phone number list. This factor was relevant for the survey because it affected the arrival of the informative letters and the possibility of reaching women for telephone interviews. Two possible reasons were responsible for non-receipt of the informative letters. These reasons were postal service inefficiency and lack of valid and updated addresses within the institutional databases used (central reservations’ databases, LHUs’ assisted registry, and primary care medicine – GP’s databases). Consequently, the survey suggested difficulty in reaching the target population for organized screening programs. However, the lack of up-to-date phone numbers diminished the study quality, which increased the risk of required substitutes instead of randomly selected women, and led to possible selection bias. A telephone interview for data collection is an effective tool for reaching people, but has some limits.¹⁵ Gaining the respondents’ trust is difficult because they receive multiple daily attempts of contact from call centers. Some topics, such as death, illness in general, and more specifically cancer, and sexual health can be problematic because they may be considered sensitive topics and generate reluctance.³⁰ The active involvement of GPs, as well as pre-interview informative letters, were decisive in overcoming these obstacles, as previously reported.³¹ These situations are at least partly explained by socio-cultural aspects (e.g., beliefs, superstitions, and taboos).³² Moreover, personal or relatives’ emotionally painful experiences of cancer, in the present or in the past, and a low level of health literacy are associated with a low prevention awareness, especially among women.^33,34

In cross-sectional studies, an interview has the primary objective of data collection. However, an interview may be an opportunity to listen and to communicate individually with citizens, especially when it is addressed to large number of people and can be performed systematically according to a structured model. In fact, during the SES study, the last phase of the interview was devoted to customized information and was particularly appreciated by the women.

Finally, interpreting the results of prevention-related surveys in relation to the respondents’ level of health literacy would be useful because a relation between health literacy and preventive health behaviors has been documented.^33,34 The health literacy level could also explain the difficulties to understand and answer questions with a 5-point Likert scale (from totally agree to totally disagree) that were considered complex by some women. However, use of Likert scale questions is important to obtain comparable answers.³⁵

A limitation of this study is possible underestimation of implications of the survey because of the performance of interviewers. Because the interviewers were the same researchers who conducted SWOT analysis, they might have more easily attributed difficulties to other external factors then to themselves. In applied epidemiology, cross-sectional studies are irreplaceable tools for their simplicity, the short time to be performed, low cost, the results in terms of knowledge of health and determinants in a specific setting or subpopulation, and the operational impact.³⁶

To improve the response rate of surveys, mediation of a trusted GP (the “family doctor”) in the enrolment phase is crucial, as well as the possibility of adopting a language corresponding to the level of health literacy and to the needs of the interviewees. Cultural taboos or other aspects may affect non-adherence to the survey. Management of these difficulties by the interviewer is important for increasing compliance and to limiting selection bias. Consequently, training of interviewers is an important element for the overall survey quality. This SWOT analysis of the SES survey provides some useful recommendations for future research in the field of cervical cancer prevention.

In summary, we recommend the following six points. (1) A timely letter should be sent to inform women about the scope and purpose of the study, the enrollment procedures, and the involvement of her GP. (2) GPs should be actively involved in the enrollment phase. (3) The interviewers’ training should be carefully planned with a specific focus on communication skills. (4) Women’s participation should be promoted by emphasizing the importance of their contribution to the survey and the practical implications for improvement of the screening system. (5) The final part of the survey dedicated to counselling on information about prevention, including practical information on local health services, should be provided. (6) Risk factors should be investigated by setting or subpopulation. Methods to further simplify the language are required and a set of items to determine the level of health literacy should be included in the surveys.

List of abbreviations

SES: Save Eva in Sicily

SWOT: Strengths, Weaknesses, Opportunities, and Threats

GP: general practitioner

LHU: Local Health Unit

HPV: human papilloma virus

Authors' contributions

All individuals listed as authors have substantially contributed to designing, performing, or reporting the study, and each specific contribution is indicated as follows:

Claudio Costantino: conceptualization, methodology, validation, supervision, writing the original draft of the manuscript, and review and editing of the manuscript. Walter Mazzucco: methodology, validation, supervision, and review and editing of the manuscript. Claudia Marotta: conceptualization, methodology, formal analysis, data analysis, and writing the original draft of the manuscript. Laura Saporito, Stefania Bono, Giusy Russo Fiorino, Giorgio Graziano, Marialuisa Maniglia, Valentina Marchese, Sara Palmeri, Sandro Provenzano, Omar Enzo Santangelo, Daniele Domenico Raia, and Gianmarco Ventura: methodology, investigation, formal analysis, and data curation. Giuseppe Napoli: conceptualization, methodology, and data curation. Sofia Colaceci: conceptualization, validation, supervision, and project administration. Angela Giusti: conceptualization, validation, supervision, project administration, and review and editing of the manuscript. Alessandra Casuccio: validation, supervision, and review and editing of the manuscript. Vincenzo Restivo: methodology, formal analysis, data curation, and writing the original draft of the manuscript.

Availability of data and materials

The data that support the findings of this study are available in Italian on demand from the corresponding author.

Declaration of conflicting interest

The authors declare that there is no conflict of interest.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.