Consideration of treatment fidelity to improve manual therapy research

Abstract

Objectives

The purpose of this paper was to define treatment fidelity, review its use in health care research and suggest how it may be utilized in manual therapy research to improve the reliability and validity of the literature.

Results

We offer an outline and a table of how manual therapy research may benefit from the concept of treatment fidelity.

Discussion

While treatment fidelity is a newer concept, and has not been integrated into Physical Therapy or Manual Therapy research, when utilized, it can have positive effects on the reliability and validity of the techniques we evaluate.

Introduction

Treatment Fidelity (TF) involves the implementation of methodological strategies to enhance the reliability and validity of the independent variable in research.^1–4 It is an ongoing assessment of whether the treatment protocol’s core components are implemented as intended.^5–7 If a study includes high TF, we can be confident that the research process closely followed the theory or hypothesis being tested and that additional factors were neither omitted nor added.⁸ The purpose of this paper was to define TF, review its use in health care research and suggest how it may be utilized in manual therapy (MT) research to improve the quality and strength of the literature.

Components of TF have been documented in research since the 1970s as adherence or treatment integrity.⁹ In 1977, Quay¹⁰ stated that, in order to have good quality implementation of an intervention, integrity must be assessed, yet a scant 27% of the published behavioural research provided data on monitoring the independent variable.¹¹ The concept of treatment integrity grew in 1982 when psychology researchers concluded that treatment manuals alone were insufficient for protocol adherence.¹² This resulted in creating a measurement tool to determine the accuracy to which the intervention was implemented as intended. Several years later (1991) the term, ‘Treatment Fidelity’,¹³ was formally introduced into literature, and an assessment tool was created that addressed three of the five current components of TF.¹⁴ In 1999, a TF work group was formed within the United States National Institute of Health, and in 2004, the three co-chairs; Borrelli, Bellg and Czjakowski, expanded the literature to publish recommendations for behavioural intervention research.^1,8,15

The tool developed by the TF work group contained five components: treatment design, provider training, treatment delivery, treatment receipt and enactment of treatment skills. Treatment design relates theoretical models or clinical guidelines to the hypothesis being tested. It attempts to ensure that the dose of the intervention is monitored in the treatment and control conditions and that a plan is initiated for implementation setbacks. Provider training attempts to standardize the treatment protocol and minimize its fluctuation by assessing knowledge during and post-treatment.^1,2,10,16,17 Delivery of treatment is used to ensure that the content and dose are delivered adequately and that the provider adhered to the intervention protocol.^1,2

The final two components of TF focus on the participants rather than the independent variable or providers. Receipt of treatment assesses the participants’ comprehension and enactment of treatment skills assesses the participants’ ability to perform the intervention skill in real-world settings.^1,2 The NIH/ BCC treatment fidelity tool has been found to be reliable and valid;¹⁵ however, individual researchers must successfully implement the tool and document its results.^3,15

There are numerous benefits to strong fidelity including greater confidence in results, improved statistical power^1,2,8,18 and improved internal and external validity.^2,8,19,20 Additionally, TF reduces the risk of Type 1 and Type 2 errors. If the independent variable doesn’t adhere to the stated treatment protocol there will be poor internal validity. Similarly, there is a decreased possibility of replicating the study and decreased external validity^1,2 if the providers do not receive standardized training or if the protocol is not strictly followed.⁸ If TF is not assessed, one may not be sure that the significant results are attributable to the treatment rather than other, unknown factors, such as omitting a portion of the protocol, resulting in Type 1 error.^5,11 If the results are not significant, one can’t assume that the poor results are attributable to the treatment rather than addition or omission of other factors, which may lead to Type 2 error.⁵

It should be noted that, despite the TF literature, tightly controlled treatment protocols might decrease external validity. True clinical practice is not reflected in a tightly controlled trial. Therefore, when discussing external validity and TF, consideration should be given to the type of trial. Efficacy trials often take place in tightly controlled settings, not typical of clinical reality. Conversely, pragmatic studies may have higher external validity because they have less standardized, multimodal approaches, which better reflect clinical scenarios.²¹

Documentation of TF in health care literature

TF began in behavioural intervention studies in 2004, and has since been implemented in a variety of additional professions, yet its adherence is not well documented. TF was assessed in 29 studies related to second-hand smoking interventions to determine the relationship between TF adherence and statistically significant results. Studies with high fidelity were nine times more likely to obtain a significant result.²² A study of assisted living employees which monitored fidelity found a high adherence correlated with increased understanding and more success when treating participants with dementia.²³ With the exception of these few studies, it is rare for health science research to include how adhering to TF might strengthen results.

To the best of our knowledge, TF has not been implemented into MT research and is rarely mentioned in physical therapy journals. Recently, a rapid review was published on the results of group-based physiotherapy-led exercise and education interventions to promote self-management for people with osteoarthritis and chronic low back pain.²⁴ Within the rapid review, TF was assessed for adherence. The results showed the overall use of fidelity to be low, with training of providers (10%) being the least adhered to TF component.²⁵ Study design and treatment enactment were the most adhered to components at 53 and 43%, respectively. Treatment delivery was found to be 20% and treatment receipt was 33%.²⁵ These percentages are concerning considering the impact on PT research. Without successful adherence, there could be significant changes in the results if the dose of the treatment varies. For example, the difference between treating a patient for 30 min rather than the set protocol of 20 min could be substantial.

The following recommendations are provided to enhance the use of TF in current MT research. Table 1 offers a quick assessment tool to determine if the following recommendations have been successfully implemented into research or, it may be used as a guide to increase compliance with the principles of TF when developing a research protocol.

Table 1.

Monitoring of treatment fidelity in manual therapy

Treatment fidelity category	Strategies	Rate: present, absent but should be present, or N/A. If present describe strategy used
Treatment design	Dose of intervention condition: •Length of contact (min) •Number of contacts •Content of treatment •Reps/ Sets of exercise •Grade, duration and force direction of manual therapy
	Dose of control group: •Length of contact (min) •Number of contacts •Content of treatment •Reps/Sets of exercise •Grade, duration and force direction of manual therapy
	Dose is equivalent between conditions and for participants within conditions
	Specification of provider credentials
	Theoretical model or hypothesis the intervention is based on is specified
	Biopsychosocial variables are considered
Training providers	Description of provider training and standardization of training, including plan to account for provider differences
	Assessment of provider skills and maintenance of skills over time
	Specified inclusion criteria of provider qualifications
	Questionnaire used to ensure providers find the intervention acceptable, credible and potentially valuable
	Clinical Equipoise is monitored
Delivery of treatment	Content and dose of intervention are delivered as specified
	Common language used to describe manipulative technique (rate of force application, location in range of available movement, direction of force, target of force, relative structural movement and patient position)
	Assessment of non-specific treatment effects
	Use of treatment manual
	Contamination between conditions is prevented
	Standardized interactions with subjects to neutralize therapeutic alliance influence
Receipt of treatment	Questionnaire to determine degree participants understood intervention
	Assessment of participant expectations with questionnaire such as PRES
	Multicultural factors considered in the development and delivery of intervention
Enactment of treatment skills	Adherence to HEP assessed
	Assessment of return to activities and participation (participant goals)

Design of study

• •Adherence to dose is important when addressing the design of a study.^{1–3,18,20,26}Factors that may be involved in the dose of manual therapy are time, repetitions, sets, grade and direction of mobilization force.
• •Dose should be specified for both the intervention group and the control group.^{1,15,16,18,26}
• •Study design should be based upon a theoretical model or hypothesis and all objective measures should reflect this.^{1,2,8,18,23,25,27} The outcome measures should relate to previous evidence or a protocol review group should ensure that the intervention reflects the theoretical model or hypothesis.²⁵
• •Interactions and delivery methods between provider and participants should be standardized to create consistency in interaction.

Training providers

• •Providers vary in size, and may have learned manual skills in a variety of ways to accommodate for various participants. Yet, if research is being performed, all providers should perform the skill in a similar fashion. This may require testing providers to determine if they can perform the skill according to protocol.^1,8
• •Variations in experience, degree, practice hours and caseload may affect the skill level of a PT in MT. To address this concern, it is important for the lead researcher to determine clear inclusion criteria of the education and experience necessary for the research team. Consider a homogenous group of PTs to limit variation or determine if there are variations in results when considering these individual differences or include statistical analysis to objectively determine effects of variations.
• •It cannot be assumed that providers have equal understanding of a treatment based solely on their credentials or years of experience. Commonly, physical therapy research is investigating a hypothesis; therefore, the treatment protocol may be a new concept to the providers regardless of experience. If providers are not trained and assessed for adherence throughout the research, drift from the protocol can occur making it difficult to determine if the results are meaningless, due to a lack of adherence to the treatment protocol or variations in levels of training of the providers.⁵
• •Accommodate for learner differences by providing a variety of teaching methods during training. This may require more intensive training and follow-up for less experienced providers.^1,5
• •Providers should agree that the research design is acceptable, credible and potentially valuable to improve commitment to the research.⁵ It is important to consider clinical equipoise in RCT, or the assumption that one intervention is not considered superior to the other intervention(s).
• •Equipoise is difficult to achieve in manual therapy due to the increased likelihood that the provider has preconceived personal preferences based on their clinical experience.²⁸ An expertise-based RCT can be used to help improve equipoise by randomizing participants to an expert practitioner in each intervention, rather than being randomized to a treatment.²⁹

Delivery of treatment

• •The treatment should be delivered according to a detailed protocol. This can be assessed through patient questionnaires following treatment sessions, random videotaping of treatment sessions or a protocol checklist.^1,2,8
• •Common language should be used when describing a MT technique to maintain reliability between all PTs. The American Academy of Orthopedic Manual Physical Therapists’ task force proposed the following six characteristics to describe a manipulative technique³⁰:

• (1)Rate of Force Application: Describe the rate the force is applied (e.g. high velocity),
• (2)Location in Range of Available Movement: Describe if motion was intended to occur at beginning, middle or end point of available range of movement (e.g. end range),
• (3)Direction of Force: Describe the direction of the force in standard anatomical and biomechanical language (e.g. posterior-to-anterior force),
• (4)Target of Force: Describe the location the force will be applied (e.g. talocrural joint),
• (5)Relative Structural Movement: Describe which structure or region is moving and which is stable by listing the moving structure or region first, followed by ‘on’, and the stable structure or region second (e.g. lower lumbar spine on upper lumbar spine),
• (6)Participants’ position: Describe the participants’ position including any pre-manipulative positioning (e.g. prone position).

• •Assessment of non-specific treatment effects should also be included.² For example, what feedback did the participant provide during the treatment and what affect did this have on the therapist’s adherence to the treatment protocol? Additionally, thought should be given to the treatment equipment and if it affects delivery.²
• •Interactions with participants are standardized to neutralize therapeutic alliance influence.²⁹ Therapeutic alliance is a positive social connection between the participants and the PT²⁴ that provides the participants with a sense of collaboration and support.^24,31 In research, therapeutic alliance is shown to improve treatment adherence and outcomes.^24,31 Therefore, to improve standardization of the delivery of treatment, each practitioner needs to be trained to provide the same social interaction with each participant. The Working Alliance Inventory (WAI) is an example of a toll used to measure alliance to adequately determine if standardized interaction was maintained throughout the study.³¹

Receipt of treatment

• •The treatment should be well received and understood by all participants. In order to assess this, a participants’ questionnaire may be used at the end of each session.^1,2
• •Address participants’ expectations in a questionnaire and analyse whether subjective responses from the questionnaire match the objective findings from the research. The Pain Rehabilitation Expectations Scale (PRES) was introduced in a preliminary study in 2010 to address proxy efficacy, working alliance and motivation/expectations.³² Proxy efficacy is the participant’s confidence in his/her therapist’s capabilities and working alliance involves trust, respect and acceptance around a common goal for the participants.³² The PRES can be used to determine if psychosocial factors, such as motivation and trust in the therapist, changed throughout the study and whether the PRES score affected treatment outcomes.
• •Multicultural factors should be considered during the development and delivery of the intervention to increase receipt of treatment.⁵

Enactment of treatment

• •Assess whether or not participants performed interventions, such as a home exercise programme, by monitoring adherence with a daily log sheet.^33–35
• •If a study utilizes solely MT and participants are not required to perform independent exercise, enactment may not be accurately evaluated.²⁶
• •Address participants’ activity and participation goals and achievement of those goals with a questionnaire.³⁶

Limitations

There are multiple proposed reasons why TF is not consistently monitored in research including increased time and labour,^8,25,36 feasibility to implement all five components of the TF model, increased cost,^8,36 and increased participant and provider fatigue from daily protocol reminders.³⁷ One limiting factor for monitoring TF is that the stringent criteria required to maintain good TF may limit flexibility^25,38 if providers are unable to modify the intervention techniques to accommodate participants’ body type, pain level or responses to the treatment. However, modifying intervention technique puts the research at risk for poor validity. Internal validity will be decreased if the independent variable doesn’t adhere to the stated protocol and external validity will decrease if the providers do not receive appropriate training and strictly follow the protocol. This creates a challenge when transferring research results into clinical settings where treatments may be altered based on the participant’s body type, pain level and responses to treatment.^27,38

Conclusion

TF has the ability to improve the quality and strength of manual therapy research. The addition of the five components of TF to MT research, with the use of our TF tool (Table 1), may improve statistical power,^8,18 reliability, validity and reduce the risk of Type 1 and Type 2 errors.^3–5

The purpose of this paper was to define TF, review its use in health care research and suggest how it may be utilized in manual therapy (MT) research to improve the quality and strength of the literature. There are numerous articles available that explain the concept and history of TF; however, there are few articles that provide evidence that improved adherence to TF leads to improved validity of the results. More research is needed to evaluate the influence of adherence to TF principles on the outcomes of clinical trials in MT research.