Adaption and validation of the Working Alliance Inventory for General Practice: qualitative review and cross-sectional surveys

Abstract

Background

Relational aspects of primary care are important, but we have no standard measure for assessment. The ‘working alliance’ incorporates elements of the therapeutic relationship, shared decision-making, goal setting and communication skills. The Working Alliance Inventory (short form) (WAI-SF) has been used in adult psychology, and a high score on the survey is associated with improved outcomes for clients.

Objective

To adapt the WAI-SF for use between GPs and patients and to test its concurrent validity with measures of shared decision-making and the doctor–patient relationship and discriminant validity with measures of social desirability.

Methods

Two rounds of online survey feedback from 55 GPs and 47 patients were used to adapt the WAI-SF—the WAI-GP. The tool was then completed by 142 patients in waiting rooms after seeing their GP and by 16 GPs at the end of their session. Concurrent validity with measures of shared decision-making and patient–doctor depth of relationship was determined using Spearman Rho correlations. Patients also completed two social desirability surveys, and discriminant validity with WAI-GP was assessed.

Results

Following feedback, the survey was re-worded to remove phrases that were perceived as judgmental or irrelevant. The patient measure of the WAI-GP was strongly correlated with Dyadic OPTION (rho = 0.705, P = 0.0001) and Patient–Doctor Depth of Relationship scale (rho = 0.591, P = 0.0001) and not with measures of social desirability.

Conclusion

The psychometric properties of the WAI-GP support its use for measuring GP-patient alliance. Possibilities for use include assessing the influence of therapeutic alliance on the effectiveness of interventions.

Introduction

There is mounting evidence that the quality of the therapeutic relationship influences the effectiveness of health care interventions (1). Most of the work on the therapeutic relationship has been qualitative (2), thereby adding to the richness of our understanding but not enabling specific comparisons between patients or interventions. There is currently no standard, comprehensive, psychometrically sound tool for measuring the quality of the therapeutic relationship in primary care (3).

The measurement of quality in primary care is the subject of intense investigation around the world. Some of this work has focused on the measurement of ‘usual’ care rather than focusing on consultation quality (4). This is very useful for assessing primary care health systems but cannot fully evaluate how and why clinical interventions work. The measurement of quality within consultations is helpful for intervention development and translation of interventions into other health care settings.

A high-quality doctor–patient interaction is defined by a variety of factors. Factors that are associated with enhanced outcomes in primary care include patient–doctor continuity (5), trust (6), empathy (7) and communication skills (8). The process of the consultation is also enhanced by patient-centredness (9), shared decision-making, goal setting and agreed task delegation. Tools have been developed to measure some of these factors separately including empathy (7), depth of relationship (10), patient-centredness (11,12), communication skills (13) and the primary care process (14). One tool that has the advantage of assessing both the therapeutic relationship as well as elements of the consultation process is the Working Alliance Inventory (WAI).

The WAI was derived from a well-researched framework on the working alliance developed by Bordin (15). In his original paper, Bordin highlighted that the working alliance was applicable to any ‘helping relationship’ although it has been applied mainly in the discipline of psychology (15). The working alliance is composed of three parts: the bond between the two parties, collaborative goal setting and agreement on the required tasks to reach the goals. The WAI was developed based on this framework—originally as a 36-item scale and then shortened to a 12-item scale (WAI-SF) (16). The practitioner and the client each completes a different survey, and their responses are synthesized into a final rating. In adult psychology, a higher WAI score is associated with better clinical outcomes (17,18).

The WAI-SF has been investigated in a few medical settings. The WAI-SF was applied in Canadian community primary care where a strong alliance was associated with patient satisfaction, enablement and intent to adhere to treatment (19), and in the USA, it was completed by patients with chronic disease, who were recruited online and managed in either primary or secondary care, where a high WAI-SF score was also associated with patient satisfaction and intention to adhere to treatment (20). It was also used in a feasibility trial of a weight management program in Australian general practice where a strong alliance was associated with patient engagement and a trend to improved health outcomes (21). However, in this Australian trial, participants noted that some of the wording of the WAI questions did not translate well to the GP–patient context.

In Australia and the UK, doctors with specialist training qualifications in family medicine are referred to as GPs. In Australia, the bulk of primary care is delivered by GPs in the community. Given the strong theoretical basis of the WAI-SF and its association with improved outcomes in adult psychology, our aim was to adapt the WAI-SF for the GP-patient setting and to test its concurrent validity with measures of shared decision-making and the doctor–patient relationship and discriminant validity with measures of social desirability.

Methods

Adaption of the WAI-SF survey wording

We recruited GP and patient participants via the newsletter of an Australian health care consumer organization, the email list of the Australian New Zealand Academic Association of Primary Care, the GP list of the local academic family medicine unit and Twitter. The Hatcher short-form version of the WAI (16) (WAI-SF) was entered into the Qualtrics online survey platform. A five-point Likert scale ranging from ‘Strongly agree’ to ‘Strongly disagree’ was used to assess ‘How relevant is this question for you?’ and ‘How comfortable would you be to answer this question?’ Following this, the participant was then asked ‘Would you suggest any changes to the wording of this question?’ with a Yes/No option and free text box. Basic demographics of both the GP and patient groups were collected, and in addition, patients were asked for their general health status and whether they had a preferred family doctor. In discussion with a team of international, interdisciplinary researchers, the qualitative and quantitative feedback was used to revise and improve the survey statements. The revised survey was then circulated by an anonymous email link to the same participants who had agreed to be re-approached.

Clinical application of the WAI-GP

A convenience sample of patient–GP dyads was recruited from local general practices via email from the list of teaching practices at the Academic Unit of General Practice and via social media platforms. The only exclusions to participation were aged <18 years, unable to give informed consent or not able to understand spoken English.

The adapted WAI-SF, namely, the WAI-GP, was compared with current measures of shared decision-making (using the Dyadic OPTION) (22) and patient–doctor depth of relationship [using the Patient–Doctor Depth of Relationship scale (PDDR)] (10). The GPs were asked demographic information and completed the WAI-GP in relation to the patients who had agreed to participate after their consultation. Patients were asked demographic information, general health state (via the COOP/WONCA functional assessment) (23), why they attended that day and whether they were seeing their preferred GP. After their appointment, the patients were asked to complete the WAI-GP, Dyadic OPTION, PDDR, plus two measures of social desirability (eight-item Crowne–Marlowe Social Desirability Scale (24) and four-item Haghhighat Brief Social Desirability Scale) (25).

Analysis

Sample size to determine confirmatory factor analysis was a minimum of 15 GPs with 10 patients for each GP. Concurrent and discriminant validity of the patient-response section of the WAI-GP was assessed. To assess concurrent validity, the relationship between the WAI-GP and the other patient report tools (i.e. PDDR and Dyadic OPTION) was evaluated. Discriminant validity was assessed by investigating the relationship between the WAI-GP patient responses and the social desirability scales.

It was predicted that WAI-GP scores of patients would be associated with shared decision-making and depth of relationship. Conversely, it was predicted that WAI-GP scores of patients would be not be associated with social desirability. We used the Spearman’s rank correlation coefficient to assess bivariate relationships among the variables due to the positively skewed data. Pearson’s analysis was also undertaken, and results demonstrated that the pattern of correlations was unaltered.

The internal consistency of the total scale of the WAI-GP was evaluated using Cronbach’s alpha. Confirmatory factor analysis was done to determine whether the original three-factor structure (goals, tasks and bond) was replicated and whether a higher order factor of overall therapeutic alliance emerged.

We only included surveys that were sufficiently complete to score in the analysis (i.e. those missing no more than one item per instrument). We excluded three patients with incomplete WAI-GPs and one with multiple incomplete instruments.

Results

The first online survey feedback had 47 patients and 55 GP participants (Table 1). The suggestions for re-wording were detailed, with participants particularly highlighting words and phrases that they perceived as judgmental or derogatory. Some GPs did not understand that the questions would be asked of them, rather than their patient. Sixteen (out of 23) patients and 14 (out of 26) GPs gave feedback on the revised version, and the majority agreed that the wording of the questions was improved in the revised version (Table 2). Two questions from both surveys were further revised based on the second round of feedback to produce the final version (Table 2).

Table 1.

Online survey adaption of the WAI—patient and GP demographics in first round of feedback (completed 2017)

Participant responses to demographic surveys n (%)
Patients (n = 47)		GPs (n = 55)
	n (%)		n (%)
Male	8 (17.0)	Male	14 (25.5)
Female	32 (68.1)	Female	32 (58.2)
Other	1 (2.1)	Other	0
Age 25–44 years	16 (34.0)	Age 25–44 years	20 (36.4)
Age 45–64 years	22 (46.8)	Age 45–64 years	23 (41.8)
Age 65–74 years	3 (6.4)	Age 65–74 years	3 (5.5)
University educated	28 (59.6)
Language other than English at home	2 (4.3)	Language other than English at home	6 (10.9)
Health status poor	15 (31.9)	Practicing <5 years	10 (18.2)
Chronic illness	31 (66.0)	Practicing 6–15 years	14 (25.5)
Has preferred GP	38 (80.9)	Practicing 16–30 years	15 (27.3)
		Practicing 30+ years	7 (12.7)
Agree to further feedback	23 (48.9)	Agree to further feedback	26 (47.3)

Table 2.

The adapted WAI-GP survey feedback—patient and GP results (completed 2017)

Patient survey results
Element of therapeutic alliance	Original statement from WAI-short form(26)	Adapted statement following round 1 of feedback (n = 47)	Round 2 feedback: response to ‘Is this wording improved?’ on a 5-point scale (n = 16)
			Mean score (SD, CI, 95%)
Goal	As a result of these sessions I am clearer as to how I might be able to change.	As a result of seeing my GP, I am clearer as to how I can look after my health and wellbeing.	4.88 (SD 0.33, 4.7–5.06)
Task	What I am doing in therapy gives me new ways of looking at my problem.	What I am doing with my GP gives me new ways of looking at my health and wellbeing.	4.75 (SD 0.43, 4.52–4.98)
Bond	I believe___likes me.	I believe my GP cares about me.	4.81 (SD 0.39, 4.6–5.02)
Goal	___and I collaborate on setting goals for my therapy.	My GP and I work together on setting goals for looking after my health and wellbeing.	4.75 (SD 0.56, 4.45–5.05)
Bond	___and I respect each other.	My GP and I respect each other.	a2 (NA)
Goal	___and I are working towards mutually agreed upon goals.	My GP and I are working towards health goals that we both agree on.	4.63 (SD 0.6, 4.31–4.95)
Bond	I feel that___appreciates me.	I feel that my GP understands me.	4.94 (SD 0.24, 4.81–5.07)
Task	_____ and I agree on what is important for me to work on.	My GP and I agree on what is important for me to do to look after my health and wellbeing.	a1.56 (SD 0.5, 1.29–1.83)b
Bond	I feel _____ cares about me even when I do things that he/she does not approve of.	Even though I may do things that my GP does not advise or suggest, I know they still care about me.	4.19 (SD 0.81, 3.76–4.62)
Task	I feel that the things I do in therapy will help me to accomplish the changes that I want.	I feel the things I do with my GP will help me to achieve my health goals.	4.13 (SD 0.86, 3.67–4.59)
Goal	_____ and I have established a good understanding of the kind of changes that would be good for me.	My GP and I have a shared understanding of what I need to do to look after my health and wellbeing.	4.25 (SD 0.75, 3.85–4.65)
Task	I believe the way we are working with my problem is correct.	I think we’re doing the right things for my health and well-being.	4.56 (SD 0.5, 4.29–4.83)b
GPs survey results
Element of therapeutic alliance	Original statement from WAI-short form(26)	Adapted statement following round one of feedback (n = 55)	Round 2 feedback: response to ‘Is this wording improved?’ on a 5-point scale (n = 14)
			Mean score (SD, CI, 95%)
Task	___ and I agree about the steps to be taken to improve his/her situation.	My patient and I agree on the tasks required to manage his/her health and wellbeing.	4.57 (SD 0.49, 4.29–4.85)
Bond	I am genuinely concerned for ___’s welfare.	I am genuinely concerned for my patient’s welfare.	a1.93 (SD 0.26, 1.78–2.08)
Goal	We are working towards mutually agreed upon goals.	We are working towards health goals that we both agree on.	4.71 (SD 0.59, 4.37–5.05)
Task	___ and I both feel confident about the usefulness of our current activity in therapy.	My patient and I both feel confident about the effectiveness of our current approach to managing their health.	4.5 (SD 0.5, 4.21–4.79)b
Bond	I appreciate _______ as a person	I respect my patient as a person and accept them without judgment.	3.38 (SD 1.33, 2.58–4.18)bN = 13
Goal	We have established a good understanding of the kind of changes that would be good for ___.	We have a shared understanding of the kind of changes that would help my patient	4.43 (SD 0.62, 4.07–4.79)
Bond	___ and I respect each other.	My patient and I respect each other	a2 (NA)
Goal	___ and I have a common perception of his/her goals.	My patient and I have a common understanding of his/her health goals.	4.71 (SD 0.45, 4.45–4.97)
Bond	I respect ___ even when he/she does things that I do not approve of.	I care about my patient even when he/she does things that I did not recommend or advise.	4.5 (SD 0.82, 4.03–4.97)
Task	We agree on what is important for ___ to work on.	We agree on what is important for my patient to work on.	a2 (NA)

^aThe participant was asked ‘This question has stayed the same, is this OK?’, with Yes (Score 2) No (Score 1) options for response.

^bThis statement was re-written after phase 2 feedback taking into account the participants’ suggestions.

Sixteen GPs and 142 patients from seven general practices participated in the clinical application of the WAI-GP. The GPs were 50% female, mostly over the age of 40 years (62.6%), and seven had been in practice for >10 years (68.8%) (Table 3). The median number of patients included per GP was 8 with a range from 1 to 15.

Table 3.

Clinical application of WAI-GP: comparison of participant characteristics to national datasets (completed 2017)

Patient Characteristics (n = 142)	n (%)	BEACH 2015–16
Gender
Male	53 (37.3)	43%
Female	89 (62.7)	57%
Age
18–24 years	17 (12)	19%
25–34 years	22 (15.5)	23% aged 25–44 years
35–44 years	28 (19.7)	–
45–54 years	17 (12)	27% aged 45–64 years
55–64 years	20 (14.1)	–
65–74 years	17 (12)	31% >65 years
75–84 years	15 (10.6)	–
85+ years	6 (4.2)	–
Has a chronic illness	72 (50.7)	–
COOP-WONCA functional status	Mean ± SD
Hardest physical activity possible	2.78 ± 1.27	–
Very heavy–Very light (1–5)
Bothered by emotional problems	2.70 ± 1.38	–
Not at all–Extremely (1–5)
Difficulty with usual tasks	2.42 ± 1.21	–
No difficulty at all–Could not do it (1–5)
Limited social activities	2.14 ± 1.24	–
Not at all–Extremely (1–5)
Health change/2 weeks	3.05 ± 0.98	–
Much better–Much worse (1–5)
Health in general	3.12 ± 1.15	–
Excellent–Poor (1–5)
GP Characteristic (n = 16)	n (%)	BEACH 2015–16
Gender
Male	8 (50)	55%
Female	8 (50)	45%
Age
≤30 years	2 (12.5)
31–40 years	4 (25)	8.3% <35 years
41–50 years	4 (25)	46.5% 35–54 years
51–60 years	5 (31.3)	45.3% >55 years
61+ years	1 (6.3)
Years in clinical practice
≤2 years	3 (18.8)	0.8%
3–5 years	2 (12.5)	12.3% (2–5 years)
6–10 years	4 (25)	14.6%
11+ years	7 (43.8)	72.3%
Qualification
FRACGP	13 (81.3)	63%

BEACH, Bettering the Evaluation of Care and Health General Practice Dataset(27); FRACGP, Fellow of the Royal Australian College of General Practitioners.

Eighty-nine (62.7%) of the patients who completed the WAI-GP were female, most were over the age of 45 years (52.9%), and 72 (50.7%) self-reported having a chronic illness. One hundred seven patients (75.4%) said they were seeing their preferred GP that day (Table 3).

Both the patient and GP demographic data were comparable with Australian national general practice data. The ‘Bettering the Evaluation and Care of Health (BEACH)’ is a national Australian GP dataset that was collected up until early 2017 (27). In this study, the patients were similar to the national BEACH sample. There were slightly more patients in or survey that fell into the 25–44 years of age bracket. The GPs in our survey were younger than the national sample; however, more GPs in our survey had the specialty qualification of a Fellowship with the Royal Australian College of General Practice (Table 3). Compared to the national dataset, our sample included more consultations where patients were meeting that GP for the first time (Table 4).

Table 4.

Clinical application of WAI-GP: patients’ reason for consultation and consultation experience (n = 142, completed 2017)

Patients’ characteristics (n = 142)	n (%)	Comparative data
Preferred GP
Yes	107 (75.4)
No	27 (19)
Prefer not to say/no response	8 (5.6)
Number of times seen GP
First meeting today	31 (21.8)	7% new to practiceb
2nd appointment	19 (13.4)
<1 year	21 (14.8)
About 1–5 years	37 (26.1)
>5 years	34 (23.9)
Experience in consultation
Attended with a support person	21 (14.8)
GP listened carefully	141 (99.3)	75%c
GP showed respect	141 (99.3)	81%c
GP spent enough time	140 (98.6)	76%c
Patient reason for consultationa
To find out what’s wrong/diagnosis	32 (22.5)
For reassurance	11 (7.7)
To get test results	30 (21.1)
For treatment (incl scripts)	55 (38.7)
For routine check	31 (21.8)
For review	31 (21.8)
For referral	19 (13.4)	16 per 100 encountersb

^aMore than one choice permitted.

^bBettering the Evaluation of Care and Health (BEACH) General Practice Dataset 2015–16 (27).

^c2016–17 Patient Experience Survey, Australian Bureau of Statistics(28): Always listens; respect; time.

The patient measure of WAI-GP was strongly correlated with Dyadic OPTION (rho = 0.705, P = 0.0001) and PDDR (rho = 0.591, P = 0.0001). As hypothesized, the WAI-GP was not correlated with the measures of social desirability (Table 5). The patient and GP WAI-GP scores were only related in a low-moderate way (rho = 0.351, P = 0.0001).

Table 5.

Results of Working Alliance Inventory -General Practice (WAI-GP) for patients, GPs and total; Spearman Rho Correlation Coefficients among Patient WAI-GP and other scale scores (patients = 142; GPs = 139 total scores from 16 GPs; completed 2017)

WAI-GP	Number of questions, (maximum score)	Response options	Mean ± SD
Patient score (n = 142)	12, (max 5)	Strongly disagree (1) -	4.33 ± 0.59
GP score (n = 139)	10, (max 5)	Strongly agree (5)	4.27 ± 0.46
Total score (patient plus GP)	22, (max 10)		8.60 ± 0.84
	Patient WAI-GP—Spearman Rho Correlation Coefficients
	n	rho	P
Patient–Doctor Depth of Relationship scale	139	0.591	0.0001
Dyadic OPTION	127	0.705	0.0001
Crowne–Marlow Social Desirability	126	0.105	0.243
Haghighat Brief Social Desirability Scale	130	0.009	0.917

Note. Weak relationships rho = 0.01–0.34; Moderate relationships rho = 0.35–0.64; Strong relationships rho = 0.65–1.00.

The WAI-GP data from the patients were highly positively skewed (mean 4.33 ± SD 0.59) (Table 5), and this was mirrored in the positive experience reported by patients of being listened to, respected and having enough time with the GP (Table 4). Using confirmatory factor analysis, we were unable to separate the three domains in the analysis, probably due to the positively skewed data. Rather, one overall factor was identified. We explored how each item related to the overall WAI-GP using Cronbach’s alpha in the item analysis and found very high internal reliability (standardized Cronbach’s = 0.949).

Conclusions

Using online survey feedback, we developed a general practice version of the WAI (short form) for use in primary care. We found strong concurrent validity for the WAI-GP in terms of its association with shared decision-making and depth of relationship, as well as evidence for its discriminant validity in relation to socially desirable responding.

In this study, the patients’ and GPs’ scores for the WAI-GP were moderately related. It is also noteworthy that the WAI-GP total score (patients plus GPs) was normally distributed, but the patient scores were highly positively skewed. It would be helpful to further explore this finding to determine whether the patient assessment is more strongly related to health outcomes. This would seem to fit clinical experience where a patient’s assessment of the quality of the consultation and care is of primary importance (8,21,29,30). If it is found that only the patient measure is required, this would make the WAI-GP simpler to apply in both research and clinical settings.

The WAI-SF has been applied in other medical settings. In this study, we have adapted it specifically for use in general practice. This study adds to the small body of existing literature, and the findings indicate that the WAI-GP holds promise for a theoretically informed, broad and inclusive survey tool, which has thus far been lacking for use in primary care clinical research.

Strengths and limitations

Incorporating the feedback of stakeholders into the wording of the WAI-GP reflects best practice for the development of survey tools. The high education status of our patient group in the first phase is representative of our local geographic area, which has the highest level of education in Australia (31). It is possible that patients with lower education opportunity would have a different perspective on the WAI-GP. This is an area for possible future exploration with the tool.

Despite their higher education status, patient participants were aware of the potential difficulty in comprehending certain items and made comments for improving readability and understanding. This process could have been enhanced by using a ‘read aloud’ analysis with participants to further ensure that the meaning of the questions was clear and they were being answered as intended. We are aware that cognitive interviewing is the gold standard in the development of new survey tools. However, this study was an adaption of an existing tool that has previously been comprehensively tested. The research team discussed the patient-elicited alterations to the language of the tool and agreed that there was insufficient recommended change to justify the resources for a complete cognitive interviewing process.

As the recruitment strategy for this study included both formal networks as well as social media channels, we cannot measure the number of possible participants that was approached. For this reason, we do not have information on non-responders in either part of our study. The clinical application of the WAI-GP was conducted with minimal participant exclusion criteria, which is important in reflecting the diversity of patients in primary care. The demographics of patients recruited into our study reflected those of Australian national general practice datasets (27).

Implications for daily practice and research

The WAI-GP could be used in intervention development studies, in the evaluation of clinical interventions and translational research. Being able to measure the therapeutic alliance may provide insight into why some interventions work in primary care and others do not. Another application for the WAI-GP could be in the teaching and clinical setting where direct feedback from patients could assist in enhancing therapeutic relationship and consultation skills among GPs.

The WAI-GP is currently being translated and applied in the Belgian primary care setting. We envision that the WAI-GP could be useful across international health care settings to assist with clinical intervention development and translation. It is yet to be determined whether the WAI-GP will have use across different cultural settings. The original WAI has been translated into over 15 languages, so the potential for cross-cultural use of the WAI-GP does seem possible (32).

The psychometric properties of the WAI-GP are supportive of using it as a tool to measure therapeutic alliance between GPs and their patients. Therapeutic alliance is supported by a strong theoretical framework that incorporates elements known to be associated with effective primary care, such as relationships, communication skills, goal setting and shared decision-making. The WAI-GP could be used in research, clinical and teaching settings to better evaluate the influence of the therapeutic alliance on effective primary care.

Declaration

Funding: this work was supported by the Chris Silagy Research Scholarship grant from the Royal Australian College of General Practitioners. MJR is funded by National Institute for Health Research (NIHR) Post-Doctoral Research Fellowship (PDF-2014-07-013). The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

Ethical approval: the ethical aspects of this work were approved by the Australian National University Human Ethics Committee, and permission was given by Professor Horvath to adapt the WAI survey.

Conflict of interest: none.