Table 2.

Factors threatening the validity and generalisability of the trial findings

Problem	How problem is realised?	Consequence	Successful strategies to overcome the problem
Restrictive exclusion criteria	Trial recruits highly selected, relatively healthy participants with single or low number of diseases, without major disabilities or other risks.	Trial may produce overly positive—although internally valid—results that are not generalisable to real-world/geriatric patients.	Mimimising exclusion criteria.
Problems with randomisation	Using randomisation techniques prone to manipulation (sealed envelopes, drawing pieces of paper from a hat, block randomisation with the same size blocks). Heterogeneity of participants leads to an imbalance in how the characteristics of the participants are distributed in the intervention and control groups.	Researcher may consciously or unconsciously choose participants that he/she wants in the intervention group, resulting in an imbalance in the intervention and control groups. Intervention and control groups differ from each other in important ways. The findings are skewed by differing characteristics.	Computer-generated random numbers, separate randomisation centre. In block randomisation the blocks should have different sizes not known to the researchers. Randomisation can be performed using stratification according to participants’ important characteristics. The analyses can be adjusted for baseline measures.
Imperfect blinding	Nonpharmacological trials are often non-blinded. Nonpharmacological trials may be single-blinded (participants in the intervention know they are in active treatment but researchers/assessors are blinded of allocation).	Researchers/assessors may consciously or unconsciously favour participants in the intervention group, leading to overly positive findings. The participants may experience the overly positive effects of an intervention knowing that they are in active treatment.	Drug trials should be double-blind. Nonpharmacological trials should at least be single-blinded, and the participants should be blinded about the main outcome measures.
Hawthorn effect	People tend to improve when they receive attention.	Intervention group improves, but it is difficult to interpret whether it is due to intervention or purely to the attention they receive.	Control group may be offered an ‘attention’ intervention. Study assessors should be equally empathetic to intervention and control participants.
Intervention too mild	It is unethical to leave control group without care, and the normal healthcare and services are very good. The intervention must be administered on top of other healthcare services. The intervention itself makes sense and seems effective, but the control group receiving normal healthcare is also receiving very good care.	The difference between new treatment and normal care is difficult to show.	Consider planning how to make the intervention as strong as possible. Consider involving older people in the planning phase. Consider how to motivate intervention participants and enhance their self-efficacy with the new treatment.
Competent, enthusiastic interventionist(s) treat(s) highly motivated volunteer participants	The problem materialises when the trial findings are implemented and distributed to healthcare providers.	Trial may produce overly positive results. Findings cannot be generalised to real-world healthcare. Trial shows good efficacy, but the findings are difficult to implement: in real life, professionals are busy, not so enthusiastic about new treatments.	Incorporate a qualitative study together with the trial. Explore and describe the essential elements important for effectiveness. This will help with the implementation phase. Consider multiple study sites and multiple methods to also recruit participants from lower social classes who may face difficulties in participating. Improve adherence in real life to fully capture trial benefits.
Contamination of the control group	The control group receives an intervention or similar treatment as the intervention group. The professionals taking care of the intervention group receive training at their healthcare unit and they move from the intervention unit to the control unit, administering their know-how there.	Common in healthcare. When professionals hear about a new treatment, they are eager to try new interventions in their own way. This dilutes the trial effects.	Plan the intervention by considering how any possible contamination might be incorporated into the trial. Consider keeping the background healthcare professionals blind about the intervention.
Missing values in measurements or missing data	Older people may have difficulties answering all questions, they become tired during long assessment sessions or refuse to take part in some assessments (e.g. cognitive tests)	Difficult to use incomplete data, and missing scales may dilute the results.	Consider a pilot phase to estimate assessment times and the amount of missing data. Prioritise scales and measures and start with the most important ones. Use alternative methods to gather data (postal surveys, home visits, telephone contacts, proxy responses). A limited number of missing values may be imputed, and researchers should be aware of problems related to imputation methods. Describe adequately the characteristics of those with missing values.
Drop-outs	Older people become tired during the follow-up. They fall ill, are hospitalised or pass away. Controls tend to drop out more easily than intervention participants due to being disappointed with their group allocation.	Drop-outs dilute the findings and decreases the power of the trial. Impossible to implement ‘intention-to-treat’ analyses. The number continuing on to the last follow-up differ between groups and may overestimate the effect of the intervention if ‘per-protocol’ analysis is used.	Ensure a pleasant trial experience for all. See Table 1 → retention. Calculate power of the trial by considering drop-out rates. Appropriate statistical analyses using two measurement points (e.g. GEE-models) may help overcome this problem. Consider allowing the intervention for the control group after the trial follow-up is over.