What Study Design Should I Choose?

THE PROBLEM

You are a general practitioner in a five-doctor group practice. Recently you and your colleagues decided to reserve Tuesday afternoons for your diabetic patients, so as to provide more comprehensive care to them. To your dismay, you found that only half of your scheduled patients turned up for their appointments.

A few questions come to your mind:

1. Why are the patients not attending?

2. How many of them are not attending and what are the associated factors?

3. Does default result in poor outcome e.g. diabetic control?

4. Which intervention is useful to reduce the non-attendance?

All the above questions can be answered using different research methods. Question 1 can be answered using qualitative study and question 2 is to determine the prevalence for non-attendance therefore a cross-sectional study is appropriate. In question 3, we are looking for the association between the cause and effect, either case-control or cohort design could be used. In the fourth question, we are testing the usefulness of the intervention, hence a randomized control trial study design may be appropriate.

To answer your questions you decided to look through the published literature, looking for “What are the characteristics of patients who miss their diabetic appointments and why do they miss it?”

Studies in the West identified these characteristics of patients who are likely to miss their appointments: younger patients, those from lower socio-economic class, those patients who has appointments with junior doctors, and long waiting time.1,2 Other reasons quoted were: problems with the appointment system, inaccessible health care, clinic administrative errors, bad weather, lack of insurance coverage and problem with transportation.1-3

However, you are not sure if the above reasons and associated factors are applicable to your clinic. To satisfy your curiosity you decide to conduct a study among your patients. You feel that finding the answers from your own clinic is more likely to bring about a solution for your problem.

QUALITATIVE STUDY

Why are the patients not attending?

Your initial exploration may be a qualitative study. Qualitative study is a good research method if you want to understand the views and opinions of patients. Qualitative study may take several forms, all of which aim to allow the research subjects to reveal their thoughts and feelings about specific issues with as little interference as possible from the researchers (Box 1). Thus the words expressed by the research subjects are the “data” - these will need to be captured in some way for subsequent analysis.

Box 1. Some examples of qualitative study4,5

In-depth interview. This method involves a conversation between the researcher and participant or participants about the research topic. It can be done in structured, semi-structured and unstructured ways. In structured interviews, the same questions are asked in a standardised manner using the same wordings based on a prepared questionnaire. Semi-structured interviews involve a series of open-ended questions allowing the participant to express their views or feelings in their own way and own words.

Focus group discussions. In this method a facilitator (the researcher) sits with a group of participants in an effort to get information on the issue discussed. This method is used to find out the various issues related to the particular topic. Researchers observe how participants react and discuss the issues and ideas together. Ideally, the number in each group is between 8–13 participants, taking a time of between 60 and 120 minutes. The facilitator may use a prepared guide to facilitate the discussion. The discussions will be recorded either using audio or/and video tape, which later will be transcribed verbatim and later on analysed. The basis to analysis is to make sense and interpreting the data.

Participant observation. In this method, the researcher observes the research subjects in their natural environment. The researcher may want to be involved (participatory research) or take an outside role (nonparticipatory research). It is often used for studies of social roles and formal organisations. People are studied in their natural environment with minimal interference from the researcher.

OBSERVATIONAL STUDY

At times, well conducted qualitative study may provide sufficient data for you to act on. More often, the findings of the qualitative study provide the fodder for a formal questionnaire survey (see Page 32). Let’s say your qualitative study found that the reasons why your patient did not turn up for the appointment were: forgot appointment date, no transport, admission to hospital and wrong appointment date. In addition, a few patients who had defaulted in the past were also having microvascular complications and had recent admissions to hospital.

To find out how common default really is, you will need to conduct a questionnaire survey among all your diabetic patients. Questionnaire survey (and all study designs described later) collected discreet data that can be analysed further using statistical methods – they are thus called quantitative research methods.6In questionnaire survey, depending on the nature of research question and its complexity, data can be obtained in few ways: self-completion questionnaire, face-to-face interviews, telephone interviews and postal questionnaire.

The questionnaire survey (also known as cross-sectional study) allows you to determine the characteristics of the defaulters more accurately. You can also determine their reasons for not turning up and whether they had any complications or hospitalisations. Thus, cross-sectional study may show an association between default and hospitalisations. Sometimes it is difficult to decide the direction of this relationship (Is default the cause or the effect of hospitalisation?), this is because the data on default and hospitalisation were collected at the same time.

Box 2. Observational study(see Figure 1)

Cross-sectional study is also known as prevalence study. It measures cause and effect at the same time, but does not tell us the relationship, i.e. which one is the cause and which one is the effect. This is the commonest study design used in general practice and research in general. Cross-sectional studies are relatively easy to do, relatively inexpensive and can be carried out in a short time frame.

In case-control study, two groups of research subjects are chosen first: cases (e.g. diabetic patients with hospitalisation) and controls (e.g. diabetic patients without hospitalisation). The researcher goes through the past records of these subjects (both cases and controls) to find out whether they were frequent defaulters or not. Thus, in a typical case-control study, the data collection is mainly retrospective (backward in time). The selection of cases and controls is an important issue in this study design.

Cohort study begins with a group of subjects (some may be frequent defaulters, others regular attenders) but free of the condition of interest (e.g. no hospitalisation). All the subjects are followed up and observed for the occurrence of hospitalisations. In contrast to the case-control study, cohort study is usually prospective (forward in time). It provides the best information about the cause of disease plus the most direct measurement of the risk of developing that particular outcome. The difficulties of conducting a cohort study are: large number of subjects, long period of follow-up for the event of interest to develop, high cost and lost to follow up.

Figure 1.

Observation studies

Let’s say we postulate that diabetic patients who frequently default their follow-up would be hospitalised later due to diabetic complications. To prove (or disprove) this hypothesis, we can use either case control study or cohort study; these two study designs, together with cross-sectional study, are known as observational studies7 (Box 2). In observational study, the researcher attempts to record events as they have occurred, there is no attempt to modify the frequency of their occurrence. Case-control study and cohort study are technically more complicated to perform but allow more sophisticated analysis. Advice from experienced researchers or statistician is often needed before you embark on these studies. Furthermore, all the confounding variables need to be considered and taken into account. Of course, observational study may not be as complicated as we make it out to be. In fact, McWhinney and Glasziou have emphasized that observational study should be the mainstay of the research methods of family physicians.7-10

CLINICAL TRIAL

Your previous observational studies have verified that diabetic patients who frequently default their follow-up have high chance of being hospitalised for their diabetic complications. As hospitalisation is a costly health care burden, you think that reducing non-attendance (default) of diabetic follow-up will make a difference. Your previous cross-sectional study found that 50% of your diabetic patients did not turn up for their follow-up and the most important reason is forgetfulness! Therefore you wish to test the idea of sending out reminders to these patients a few days before their follow-up to see if those who receive the reminders are more likely to turn up. Clinical trial is an example of an experimental study whereby the researcher attempts to modify the outcome by introducing one or more interventions.11-12 The ideal clinical trial is called randomised controlled trial (RCTs) (Box 3). Conducting a clinical trial is a serious undertaking; one of the most important reasons is the potential harm that may befall the research subjects (depending on whether they are in the intervention or the control group). Lots of background data will have to be collected in the form of qualitative or observational studies before you can plan your RCT.

Box 3. Clinical trial

Experimental studies involve attempt to change a variable in one or more groups of people. A randomised controlled trial (RCT) is commonly used to study a new preventive or therapeutic regiment. Subjects in the study population are randomly allocated to either the intervention or control groups. The results are assessed by comparing the outcome among the various groups. A good example would be to carry out a clinical intervention such as a drug trial. In this respect, detailed attention to ethical consideration is particularly important.

In a locally conducted RCT (personal communication, Leong KC), the researchers compared the effectiveness of three reminder methods (SMS, telephone, no reminder) in reducing default in general practice follow-up. Patients who required follow-up in general practice were assigned to receive any of the three interventions. Their attendance during the appointment date was track to determine whether they have defaulted.

CONCLUSION

In this article, we have illustrated a few research methods using a common clinical problem as an example. You can see that the choice of research design depends a great deal on the research questions you have identified (see Table 1). It is important to have a clear research question, so that you can plan your research properly. Only then your question can be answered.

Table 1. Example of research questions and the appropriate research methods.

Research questions	Method
Why do patients on tuberculosis treatment default?	Qualitative – interview or FGD
What is the prevalence of default among those on anti-TB treatment?	Cross-sectional non-experimental study
Is fear of side effects of TB drugs the cause of non-attendance?”	Case-control, cohort
Does involvement of family members reduce non-attendance?”(RCT)	Randomised controlled trial
Why female patients do not do their regular pap smears?	Qualitative – interview or FGD
What are their feelings towards pap smears?	Qualitative – interview or FGD
What are the characteristics of patients that do not want to do pap smear?	Cross-sectional
Does reminder or patient education increase number of patients doing pap smear?	Randomised controlled trial