50 ways to trace your veteran: increasing response rates can be cheap and effective

Abstract

Background

while low response rates need not introduce bias into research, having a lower percentage of responders does increase the potential for this to occur. This is of particular concern given the decline that has been occurring in response rates since the 1950s. However, there are various methods that can be incorporated into the study design, which can assist in increasing levels of participation.

Objective

To outline the methods used by the King's Centre for Military Health Research (KCMHR) when conducting a recent telephone survey of serving and ex-Service military personnel.

Design

Using participants who had already taken part in a questionnaire-based study on the health effects of serving in the UK Armed Forces (n=10,272), a subsample was selected for an in-depth telephone interview-based follow-up study. The subsample consisted of 1,105 participants, selected on the basis of their mental health status. An adjusted response rate of 76% was achieved (n=821).

Results

Various methods of contact were used in this study to ensure an adequate response rate was achieved.

Conclusions

Simple research strategies increase response rates and are likely to reduce bias. Use of multiple simultaneous tracing methods and customisation of the approach to the target population increases rapport between participants, ensuring that those who take part feel valued as members of the study. In the current climate of decreasing participation in studies, research teams need to engage with their study population and devise innovative strategies to keep participants involved in the research being undertaken.

While low response rates need not necessarily introduce bias into research (Asch, Jedrziewski, & Christakis, 1997; Halpern & Asch, 2003), a lower percentage of responders increases the potential for this to occur. High response rates are regarded by journals, investigators, and readers as a proxy measure for the representativeness of a study population (Halpern & Asch, 2003) and innovative methods that can be utilised to increase levels of participation are, therefore, of interest.

Response rates trends

Response rates to postal and telephone questionnaires have been declining (De Heer, 2001; Tolonon et al., 2006). Various reasons have been suggested for this decline including changes in peoples perceptions of public investment and social cohesion (De Heer, 2001; Schumm et al., 2000; Tudor Hart & Davey Smith, 1997) and the increasing frequency of research creating “survey burden” within certain populations (De Heer, 2001).

Research has shown that there are certain groups that are at higher risk of non-response based on key characteristics such as sex, age, and health status (Eaker et al., 1998; Edwards et al., 2007; Tolonon et al., 2006; Traugott, 1987). Therefore, avoiding bias remains crucial.

De Heer, in a study of international response rate trends, found that while response rates were decreasing overall, there were specific countries in which response rates remained static (De Heer, 2001). De Heer found that the more optimistic picture in some countries had been achieved by an increase in the efforts, resources, and ingenuity given to fieldwork (De Heer, 2001) highlighting the benefits of targeted research methods.

Therefore, it seems advantageous, given the potential for bias posed by non-response, to establish and utilise methods that could encourage participation. This article will focus on methods that have been used by the King's Centre for Military Health Research (KCMHR) to trace and contact participants as well as ways to encourage participation in an often difficult research population, the UK military.

Methodology

The KCMHR has been conducting both large epidemiological studies and small in-depth qualitative studies of serving and ex-service UK military personnel since 1995. This research has included hard to reach groups such as those leaving military prison, many of whom had no-fixed abode on discharge and those still in military service, the details for whom are provided by the UK Ministry of Defence (MOD; Hotopf et al., 2006). In conducting these studies, varied response rates have been achieved—ranging from 99% for a study in a military prison, 59% for a cohort study of still-serving military personnel, and finally 0% when the MOD acted as our proxy, for confidentiality reasons, for contacting personnel who were leaving the Armed Forces for psychiatric or disciplinary reasons (Iversen et al., 2006). These substantial differences reflect the methods used to contact, trace, and engage the study population. This article will outline the methods that were used in a recent study that used multiple methods of tracing and engagement (Iversen et al., 2009).

Recent study population

A recent study conducted by KCMHR looked at the prevalence of mental health, health service usage, and stigma within both serving and ex-Service UK military personnel (Iversen et al., 2009). Using participants who had already taken part in a questionnaire-based study on the health effects of serving in the UK Armed Forces (n=10,272; Hotopf et al., 2006), a subsample was selected for an in-depth telephone interview-based follow-up study. The subsample consisted of 1,107 participants—70% who scored positive on the General Health Questionnaire-12 (GHQ-12; Goldberg et al., 1997; Goldberg & Williams, 1988) and 30% who did not. The study finished in May 2007 and an adjusted response rate of 76% was achieved (821 out of 1,107 contacts; Iversen et al., 2009).

Why is this population at high risk of non-response?

First, the group at highest risk of non-response in the general population—young, unmarried men (Eaker et al., 1998; Tolonon et al., 2006)—form a large proportion of military populations. These demographics were associated with non-response to the initial study (Hotopf et al., 2006; Tate et al., 2007). Second, an ill population based on participants’ responses to a previous study (Hotopf et al., 2006) was purposively sampled—the sample strategy required that 70% of the sample had symptoms indicative of common mental disorders when they last participated. It is known that after an initial surge in responses, those who are unwell tend to be reticent to participate in research due to a reluctance to accept that they are ill (Tudor Hart & Davey Smith, 1997) and it was, therefore, envisaged that this too would act as an obstacle to achieving a high response rate. Third, the topic of the interview was mental health. The stigma of having a possible mental disorder is ubiquitous across society and has a negative influence on help-seeking, but is even more pronounced in the “macho” culture of the Armed Forces (Hoge et al., 2004). Any health problem can potentially have a negative impact on a military career and, rightly or wrongly, mental health is seen as particularly sensitive, another factor that is known to decrease response rates (Edwards et al., 2007).

There was also the added challenge of contacting those who had left service; on average 12% of the UK Armed Forces leave each year (Defence Analytical Services Agency, 2006), which makes obtaining a valid address progressively more difficult over time. Amongst these service leavers, there will be some who experience job and accommodation transience, a factor known to be associated with mental health problems (Iversen et al., 2005). Tracing such socially excluded individuals is difficult, though important. Given all these factors, it was clear that targeted research methods would be needed to achieve a high response rate within this study population.

First step

Initially, the study population needed to be contacted so that they could either opt in or out of the study. The sample was last contacted about 3 years ago and, therefore, many of the contact details they gave at this point were no longer correct. Several methods were used to trace participants. These included: the military database that holds their last known address, the NHS address registry (although they have to be registered with a GP for this to be available/valid), and their next of kin contact details. In order to obtain information from the NHS registry, approval was required from the Patient Information Advisory Group (PIAG) to ensure compliance with the Data Protection Act, a process that can take several months.

All these sources of address and telephone information were then cross-referenced against the electoral roll (a listing of all those registered to vote in a particular area) to see if participants were resident at these addresses. Return to sender envelopes can take between 1 week and 6 months to be returned and, therefore, should not be used as the primary method through which to determine incorrect addresses. If the addresses were correct and participants had not responded to the first mail out, we sent them a second, informal letter reminding them of the study and inviting them to take part. Finally, if this elicited no response, a postcard reading “KCMHR needs you” modelled on the famous Kitchener recruiting poster (Fig. 1) was sent. This is a postcard with the relevant contact details, giving participants the final opportunity to opt into the study.

Fig. 1.

Postcard used to aid recruitment.

Table 1 outlines the tracing methods used by the KCMHR study team to identify participants (Iversen et al., 2009).

Table 1.

Methods used to contact study personnel in order of preference

Priority	Method of contact
1	Use original contact details provided by service personnel
2	Check that contact details have been entered correctly into the study database from their handwritten responses
3	If still serving personnel, contact MOD switchboard to determine if they have changed jobs and acquire updated contact details
4	If veteran, check 192.com (directory enquiries) for participant details
5	Check electoral register
6	Ask Army Personnel Centre to forward letter requesting new contact details
7	Ask DWP to forward contact details letter on our behalf to veterans who are claiming benefits
8	Search 192.com and electoral register for nearest relative and write to them requesting that they send our letter on to their relative
9	Check to see if service charities are in touch with veteran and ask them to pass on a letter from us
10	NHS tracing after PIAG approval
11	DWP tracing after DWP PIAG approval
12	Call mother

Appearance

Previous research showed that most responders (90%) had taken part in the study after receiving a study information pack and one reminder (Tate et al., 2007), suggesting that the study population needed to be engaged in the study on first contact. The aim therefore was to ensure that the invitation pack was targeted correctly to try and ensure the highest response rate possible. To this end, appearance was important.

Given the large amount of unsolicited mail potential participants receive daily, it was essential to ensure that the pack of information sent, enclosed in a standard A4 white envelope (Etter & Pernerger, 1997), did not get lost in the sea of unopened mail. For this reason, participants name and address were handwritten on the envelope and first class stamps with commemorative pictures on, ordered from the Royal Mail at no extra cost, were used in order to personalise the package (Choi, Pak, & Purdham, 1990). In addition, although the main body of the letter was typed, each person's name was hand written, the letter was hand signed, and a message reading “thank you for considering taking part” was also hand written at the bottom of the page (Maheux, Legault, & Lambert, 1989). It was made clear, by using headed paper and through text at the top of the page, that the research was being undertaken within a university, acting independently from the Ministry of Defence.

Persistence

For each study it is important to identify a point at which contact will cease and participants will be considered lost. Previous research has identified the necessity of persistence, not only to boost response rates but also to reduce bias. It can take longer to contact those who are younger and male (Traugott, 1987) suggesting that persistence can ensure a more representative final study population.

During the previous study from which this cohort was sampled, non-responder analyses identified that those who responded were no different to those who did not in terms of their health. However, the analyses did indicate that those who responded late were more likely to have missed questions, or made errors when filling in the form (Tate et al., 2007). These analyses raise the question of whether or not intensive tracing introduces a bias based on mistaken answers even if increased response rates result in a reduction of the possibility of non-responder bias. If this is the case, intensive tracing may not be of value beyond a certain threshold.

In addition to considering the study population, persistence (i.e., multiple mail outs or telephone calls) is also dependent on the financial implications of continuing to contact people and the expected increase in response rate these methods will produce. As mentioned, postal contact ended after the postcard was sent. However, telephone contact raises separate issues of when to stop calling.

For this study, attempting to make telephone contact was stopped after 10 calls to the number had been made at various times and on different days of the week. This cut-off was based on previous research, which showed that after this point few additional participants are contacted (Maheux et al., 1988). This cut-off appeared to work successfully in this study, although researchers will need to consider individual cut-off points for each study based on their own response rates and available resources.

The time of day and day of the week that telephone calls are made can have an effect on response rates. For instance, it was more productive to phone residential numbers early morning, in the evening, or at weekends and military numbers between 8 am and 10 am. These variations in time and day of the week might need to be considered when planning staffing schedules.

Encouraging participation

Participants are often nervous when talking over the phone for a variety of reasons, not least because they have never seen the research team and have negative images of researchers. To address this, a colour photograph of the research team was sent out in the initial invitation pack. The researchers were frequently asked “which one are you in the photo then?” suggests that participants had actively engaged with this method of familiarisation.

On the reverse side of this photograph was a list of possible responses to questions that participants’ were going to be asked (Aneshensel et al., 1982). This shortened the interview length. It has also been reported that inability to remember the full range of responses can lead to “primacy effect” (i.e., participants respond with the first/last option offered; Krosnick & Alwin, 1987). In addition, it acts as an aid for those who find the cognitive burden of undertaking the interview too great (Bowling, 2005), by reducing the need for them to retain all the information that is read out. This is important given the low levels of literacy of young soldiers who form a large proportion of this sample (Army Basic Skills Provision, 2007).

Remuneration: coercion or consideration?

Finally, remuneration was offered. While there are ethical debates surrounding the use of payment for interviewee's time, the offer of a small amount of compensation (in this case, £15) may have encouraged participation. From conversations with participants, it appears that the offer of payment alters their perception of the importance of research (i.e., “if they are willing to pay me for my time the study must be important)”. Rather than a form of coercion, giving financial compensation, even if slender, confirms that the research team value the time given by the participants. Participants were given the option of donating the fee to charity. This was accepted by a minority (n=83, 10%) suggesting that it is not the money itself but rather the gesture of compensation that counts.

Why use these methods?

Previous research undertaken within UK military has had various levels of success (Iversen et al., 2006). On the one hand, researchers from the University of Manchester achieved a response rate of 85% in a study of UK Gulf war veterans (Cherry et al., 2001). This study included intensive non-responder tracing and also personal visits to military bases and participant's homes. Conversely, a National Audit Office Report on those leaving military service that sent one wave of postal questionnaires achieved a response rate of 13% (Ministry of Defence, 2007). Such variations do suggest that actively attempting to contact study populations can have benefits.

Applicability to other research

Contradictory results reached in papers describing research into increasing response rates suggests that methods that are successful vary depending on the study population (Halpern & Asch, 2003). It is, therefore, important that the research team considers carefully the nature of their study population and customise their methods accordingly to maximise the response rate. Many of the generic methods outlined above have been shown to increase response rates and facilitate engagement with research, and may be helpful tools for researchers involved in contacting participants. All these methods, albeit time consuming, were relatively inexpensive and simple.

Conclusion

Simple research strategies increase response rates and are likely to reduce bias. Use of multiple simultaneous tracing methods (e.g., email, phone and post) increases response rates (Hartge, 2006) and customisation of the approach to the target population increases rapport between participants, ensuring that those who take part feel valued as members of the study. In the current climate of decreasing participation in studies, research teams need to engage with their study population and devise innovative strategies to keep participants involved in the research being undertaken.

Ethical approval

The study received approval from both the King's College Hospital NHS Research Ethics Committee (ref: 05/Q0703/155) and also from the Ministry of Defence (Navy) Personnel Research Ethics Committee (ref: 0522/22).

Conflict of interest and funding

This study was funded by the UK Ministry of Defence. The authors’ work was independent of the funders and we disclosed the paper to the Ministry of Defence at the point we submitted it for publication. SW is partially funded by the South London and Maudsley NHS Foundation Trust/Institute of Psychiatry National Institute of Health Research Biomedical Research Centre. NTF, LvS, SW and ACI work for King's College London which receives funding from the UK Ministry of Defence. SW is Honorary Civilian Consultant Advisor in Psychiatry to the British Army and a Trustee of Combat Stress, a UK charity that provides services and support for veterans with mental health problems.