High Patient Satisfaction with Examination by Advanced Practice Physiotherapists in an Orthopaedic Outpatient Shoulder Clinic: A Cross-Sectional Study Using Quantitative and Qualitative Methods

Abstract

Purpose: To evaluate patients’ satisfaction with being examined and diagnosed in an orthopaedic outpatient shoulder clinic, determine whether a difference exists between levels of satisfaction among patients examined by advanced practice physiotherapists (APPs) and orthopaedic surgeons (OSs), and explore patients’ experiences with being examined by APPs. Method: One hundred and thirty-three patients participated in a cross-sectional questionnaire study using the Visit-Specific Satisfaction Instrument (VSQ-9). Primary outcome was satisfaction with “The visit overall” (item 9). Nine patients participated in semi-structured interviews, which were thematically analyzed. Results:There was no significant difference in satisfaction with “The visit overall” between patients examined by an OS (median: 75, 1st–3rd quartiles: 75–100) and an APP (median: 100, 1st–3rd quartiles: 75–100). The VSQ-9 total score was not significantly different between groups, but some items regarding direct interaction with the health provider were scored significantly higher in the APP group. Patients were particularly satisfied with APPs’ ability to explain during the consultation. Conclusions: The results showed high levels of satisfaction with examinations performed by both OSs and APPs with no difference between groups concerning “The visit overall.” From a patient perspective, the results support the use of APPs to examine and diagnose selected patients in an orthopaedic outpatient shoulder clinic.

Worldwide, the burden of musculoskeletal disorders (MSD) has increased,¹ and is the second highest contributor to global disability.² MSD comprise 150 diagnoses – including shoulder disorders, which often result in reduced ability to work and have a lifetime prevalence of up to 67%.³ MSD are managed by health professionals in both primary and secondary care, with orthopaedic surgeons (OS) being the most commonly consulted specialists.⁴ However, as the majority of the patients do not need surgical intervention,⁵ they might instead be managed by an advanced practice physiotherapist (APP). Advanced practice in physiotherapy refers to a level of clinical practice where physiotherapists make complex decisions and manage risk in unpredictable contexts using advanced clinical reasoning.⁶ With the potential for reducing wait time and health care costs,^6–9 the use of APPs to examine and diagnose patients with shoulder pain has increased worldwide.^7–9 With population growth and ageing, the burden of MSD is predicted to rise markedly, which demands a multilevel response including primary prevention as well as disease management.¹⁰ Therefore, the use of APPs could be even more relevant in the future, but when implementing such a change, it is important to maintain the quality of the services provided.

Systematic reviews, comparing APPs and OSs managing patients with MSD, have reported satisfactory results regarding diagnostic agreement, costs, and patient satisfaction.^9,11 However, studies exclusively investigating the use of APPs in orthopaedic shoulder clinics are sparse.^12,13 Both Razmjou and colleagues¹³ and Lowry and colleagues¹² used VSQ-9¹⁴ in a modified version to measure and compare patient satisfaction with being examined by an APP and an OS. Razmjou and colleagues reported a statistically significant difference in favour of being examined by an APP,¹³ while Lowry and colleagues found no significant difference. Despite these positive results, some limitations exist. Razmjou and colleagues included only one physiotherapist, and data on socio-demographics and diagnoses were not available, thus limiting the generalizability of study results. In the Lowry and colleagues study, all participants were examined by both an APP and an OS. This reduces the generalizability of the results, as it does not reflect clinical practice. Also, the participants were systematically examined by the APP before the OS, with the OS giving the final diagnosis to the participant. Since, it is unclear how much information and advice was given during the examinations, the participants’ knowledge and needs might not have been similar in the two sessions. This could potentially affect their satisfaction and bias the results. Furthermore, none of the studies included a qualitative aspect, which could have given a more detailed insight into patient satisfaction.

The objectives of this study were to evaluate patients’ satisfaction with the diagnostic examination in the shoulder clinic and to determine whether a difference exists between levels of satisfaction among patients examined by APPs and OSs, and to explore the patients’ experience with being examined by APPs.

Methods

Study design

This was a cross-sectional questionnaire study supplemented with a qualitative interview study. Following usual clinical practice, participants in the questionnaire study were assigned to examination by either an OS or an APP based on first available time (Table 1). Allocation was performed by a secretary not otherwise involved in the study. Distribution of participants is presented in Figure 1. Prior to participating, all patients were informed of the purpose of the study and signed a consent form. Participants who both filled in the questionnaire and participated in an interview signed two different forms. The study complied with the declaration of Helsinki¹⁵ and was reported to the Central Denmark Region Committee on Health Research Ethics (Inquiry 58/2018). The study was approved by the Danish Data Protection Agency (ref. no: 1-16-02-724-17).

Table 1 .

Clinical Pathway for Patients Referred to the Shoulder Clinic

	Action	Details
Before examination	Triage	The referral is reviewed by an OS in the shoulder clinic. Depending on the expected treatment plan, patients are divided into three groups: 1)High probability of surgery or high complexity → examination by an OS. 2)High probability of non-surgical treatment → examination by an APP. 3)Remaining patients are considered equally appropriate to be examined by an OS or an APP. Patients are examined by the professional with first available time.
	Diagnostic imaging	All patients undergo X-ray examination of the glenohumeral and acromioclavicular joints before the clinical examination.
	Patient-reported information	Patients are asked to fill in a questionnaire, including questions about demographics and medicine use.
On day of examination	Duration of examination	Examinations last 20 minutes for OSs and 30–40 minutes for APPs.
	Physical framework	The APP and OS perform the examinations in adjacent rooms.
	Diagnosis and treatment plan	The patient’s medical history is recorded. A clinical examination, including relevant diagnostic tests, is performed. In most cases an ultrasonography examination is also performed. Based on the findings, the patients are diagnosed. Using shared decision-making, the patient and the professional agree on and initiate treatment.
	Communication	The APP and OS consult each other if questions arise regarding the diagnosis or treatment plan. It is mandatory for an APP to consult an OS if the treatment plan includes surgery, steroid injections, or referral to advanced diagnostic imaging.
Throughout the year	Coordination and meetings	Complex patient cases and diagnostic imaging results are discussed on a weekly conference between APPs and OSs. When needed, patient cases are also discussed outside scheduled conferences. Four times a year, the team at the Shoulder Clinic meets to evaluate and optimize clinical pathways.

Figure 1 .

Participant flow.

Setting

The study was conducted from May 25 to June 18, 2018, at an orthopaedic outpatient shoulder clinic (“Shoulder Clinic”) at a Danish public hospital. The clinical pathway for patients referred to the Shoulder Clinic is presented in Table 1. Three APPs and four OSs from the Shoulder Clinic participated in the study.

Through completion of a mandatory in-house education programme, all APPs at the Shoulder Clinic specialize in diagnosing patients with shoulder or elbow disorders. The duration of the education programme is more than 600 hours and consists of a theoretical part, an apprenticeship, and one year of practical training examining at least 400 patients in the Shoulder Clinic (online Appendix 1). The practical training makes up by far the majority of hours spent in the programme.

Participants

Patients were consecutively recruited from the Shoulder Clinic to participate in the questionnaire study. Inclusion criteria were: above 18 years of age, able to read and understand Danish, and considered equally appropriate to be examined by an OS or an APP (Table 1, triage: category 3). APPs more often had the first available time to examine a patient, hence the group of recruited patients examined by 
an APP was nearly twice the size of the group examined 
by an OS (Figure 1).

Among participants in the questionnaire study (n = 133), 
nine patients examined by an APP were recruited to participate in the qualitative interview study. These nine patients were purposefully selected to represent different days of examination, the different genders, different ages, reporting different levels of satisfaction, examinations from different APPs, and different employment statuses.

Data measurement and outcomes

Questionnaire study

Patients filled in the VSQ-9 questionnaire¹⁴ immediately after their examination without involvement of the health professional. VSQ-9 contains nine items related to outpatient clinic visits without encompassing questions about treatment (items can be seen in the result section, Table 3). The questionnaire measures patient satisfaction with access to care (items 1–4), direct interaction with the health care provider (items 5–8), as well as the overall experience with the outpatient visit (item 9). Each item has five response categories: poor, fair, good, very good, and excellent. VSQ-9 is standardized and can be used across different clinical settings.¹⁶

Table 3 .

Patient Satisfaction Scores from the 9 Items in the VSQ-9

	Total; N = 133		Examined by OS; n = 45		Examined by APP; n = 88		Difference*
Variable	n	Median score (1st and 3rd quartiles)	n	Median score (1st and 3rd quartiles)	n	Median score (1st and 3rd quartiles)	P
Primary outcome
9: The visit overall	131	100 (75–100)	44	75 (75–100)	87	100 (75–100)	0.06
Secondary outcomes
VSQ-9 total score	105	75 (67–86)	36	74 (64–89)	69	78 (67–86)	0.45
VSQ-9 items 5–8 mean score	131	94 (75–100)	45	81 (75–100)	86	94 (81–100)	0.011†
VSQ-9 item:
1: Waiting time for appointment	132	50 (25–75)	45	50 (25–75)	87	50 (25–75)	0.86
2: Office location	130	50 (50–75)	45	75 (50–75)	85	50 (50–75)	0.17
3: Getting through by phone	107	75 (50–75)	36	75 (50–75)	71	75 (50–75)	0.30
4: Waiting time at the office	132	75 (50–100)	45	75 (75–100)	87	75 (50–100)	0.039†
5: Duration of consultation	131	75 (75–100)	45	75 (50–100)	86	75 (75–100)	0.06
6: Explanations during consultation	132	100 (75–100)	45	75 (75–100)	87	100 (75–100)	0.008†
7: Technical skills	131	100 (75–100)	45	75 (75–100)	86	100 (75–100)	0.016†
8: Personal manners	132	100 (75–100)	45	100 (75–100)	87	100 (100–100)	0.014†

^*For all variables, Wilcoxon’s rank sum test was used to test for difference between groups (APP vs. OS).

^†A statistically significant difference between groups was found.

VSQ-9 = Visit-Specific Satisfaction Instrument.

VSQ-9 did not exist in a Danish version. We performed a translation from the English version as follows. Two persons independently translated the questionnaire into Danish, discussed their results, and agreed on a final Danish version. Subsequently, two other persons not familiar with the English version of VSQ-9 completed a translation of the Danish version back to English. Subsequently this translated English version was compared with the original version of VSQ-9 and the Danish version was modified. The Danish version of the questionnaire was test-piloted by 10 patients in the Shoulder Clinic, and we didn’t identify the need for further adjustment to the translation.

In addition to the questions on the translated questionnaire, patients were also asked to report their age, marital status, education and employment status, as well as whether they had been referred to surgery or not. Furthermore, the assigned diagnosis was collected by the health professional performing the examination.

Outcomes

The primary outcome was satisfaction with “The visit overall” (item 9).¹⁴ We chose this item in order to evaluate overall satisfaction without focusing too much on items regarding access to care, which we consider is given more weight in the VSQ-9 total score calculation.

Secondary outcomes were the VSQ-9 total score, the mean score of items 5–8 (direct interaction with the health provider), and the score on each of the individual items 1–8.

Qualitative interview study

A master’s student in Public Health (initials: EBB) who was not employed at the hospital interviewed participants using a semi-structured interview guide (online Appendix 2), comprising questions about the patients’ experiences of being examined by a health professional in relation to their shoulder problems as well as to their expectations prior to the examination. The interview guide was based on the patient-perceived dimension of the quality concept employed by Danish authorities¹⁷ and a previous study,¹⁸ along with reflections from the health care professionals in the Shoulder Clinic.

All interviews were performed within five days after the examination to minimize potential recall bias. Furthermore, the interviewer was blinded to the patient’s reported level of satisfaction at the time of the interview. Interviews were recorded and transcribed verbatim using fixed rules for transcription.

Statistics and data analysis

Sample size

We estimated sample size based on the primary outcome and parametric statistics; lacking a validated minimal clinical difference, we used estimates from a previous similar study as a benchmark¹³ to be able to detect a potential similar difference. The estimates used and inter-quartile ranges (IQR) were: APP mean score 94 (IQR: 0) and OS mean score 82 (IQR: 25). An IQR of 25 was used to calculate an approximate SD of 18.5 (25/1.35¹⁹). Power was set at 0.80 and level of significance at 0.05. Based on these parameters, each group should have had 38 patients yielding a total sample size of 76. We added 15% to ensure non-parametric tests would be able to show a potential statistically significant difference. Thus, the required sample size was estimated to be 88 participants, 44 in each group.

Quantitative analysis

Demographic information, whether patients were referred for surgery, and distribution of diagnoses were presented by descriptive statistics. Categorical variables were presented in numbers and percentages. Age was reported as means with minimum and maximum values.

To test for differences in patient characteristics between the group of patients examined by APPs compared to patients examined by OSs, we used a chi-square analysis for categorical variables and an unpaired t-test with 131 degrees of freedom for age.

According to the guidelines, the responses in each item of VSQ-9 were linearly transformed to a 0 to 100 scale, with 100 corresponding to “excellent” and 0 to “poor.”¹⁴ Data were not normally distributed and therefore reported in median and 1st and 3rd quartiles. The difference between groups was tested by the Wilcoxon's rank sum test (equal to results of the Mann–Whitney U-test).²⁰ However, since it could be argued, due to the ordinal categories, that the intervals between scale values cannot be presumed to be equal and therefore not transformed linearly, we also performed a sensitivity analysis using a chi-square test for the differences between groups.

No data imputation procedures were used. In the analyses with average measures (VSQ-9 total and VSQ-9 items 5–8), participants with missing data in one or more of the items included in the average measure were excluded from these specific analyses.

Data were analysed using STATA, version 16.0 (StataCorp LLC, College Station, TX).

Qualitative analysis

The interviewer (EBB) performed the analyses themself, but the process and methods behind it were discussed by EBB and their supervisor (initials: CPN). We used deductive content analysis²¹ to systematically analyze interview data, and deductively developed an unconstrained categorization matrix using the categories from the interview guide. Data were coded accordingly in NVivo, version 12 (QSR International, Doncaster, Australia). Data that exemplified or corresponded with the predefined categories used to structure the interview guide were coded first. Data that did not fit the deductive categorization frame were used to create new open concepts using a more inductive approach. Open coding was used and new subcategories were created. The subcategories created based on the inductive approach were grouped together when similar events appeared and were eventually grouped to one main category.²¹

Results

Quantitative results

Participants

One hundred and thirty-three of 149 eligible patients participated (Figure 1).

Descriptive data

Patient demographics, distribution of diagnoses, and number of patients referred for surgery are presented in Table 2. Eighty-nine percent received a specific shoulder diagnosis. There was no statistically significant difference in distribution of participant characteristics between the groups examined by OSs and APPs.

Table 2 .

Participant Characteristics

	No. (%)*
Variable	Total; N = 133	Examined by OS; n = 45	Examined by APP; n = 88	Difference,† P
Gender	133	45	88
Female	68 (51)	22 (49)	46 (52)	0.71
Male	65 (49)	23 (51)	42 (48)
Age	133	45	88
Years, mean (range, min-max)	52 (18–96)	49 (23–72)	53 (18–69)	0.10
Marital status	132	45	87
Married	80 (61)	29 (64)	51 (59)	0.80
Living alone	35 (27)	11 (24)	24 (28)
Cohabiting	17 (13)	5 (11)	12 (14)
Education	131	45	86
Basic (ISCED 1–2)	21 (16)	6 (13)	15 (17)	0.19
Secondary (ISCED 3)	46 (35)	12 (27)	34 (40)
Higher (ISCED 4–8)	64 (49)	27 (60)	37 (43)
Employment status	131	44	87
Enrolled in education	4 (3)	2 (5)	2 (2)	0.36
Employed	82 (63)	32 (73)	50 (58)
Unemployed	9 (7)	2 (5)	7 (8)
Retired	29 (22)	6 (14)	23 (26)
Disability pensioner, etc.	7 (5)	2 (5)	5 (6)
Referred for surgery	130	46	84
Yes	2 (2)	0 (0)	2 (2)	0.31
No	128 (99)	46 (100)	84 (98)
Diagnosis	132	45	87
Impingement	66 (50)	25 (56)	41 (47)	0.19
Rotator cuff injury	7 (5)	3 (7)	4 (5)
Glenohumeral instability	6 (5)	4 (9)	2 (2)
Glenohumeral osteoarthritis	4 (3)	1 (2)	3 (4)
Periarthritis humeroscapularis	17 (13)	2 (4)	15 (17)
Scapula instability	9 (7)	1 (2)	8 (9)
Fracture sequelae	0 (0)	0 (0)	0 (0)
Acromioclavicular joint disorder	9 (7)	4 (9)	5 (6)
Non-shoulder-related diagnoses	14 (10.6)	5 (11.1)	9 (10.3)

^*Unless otherwise stated.

^†Difference between groups (APP vs. OS). Unpaired t-test with 131 degrees of freedom was used to test for group difference in age. Among remaining variables, data distributions across subgroups were tested by chi-square analyses.

ISCED = International Standard Classification of Education.

Primary outcome

Patients were overall satisfied with their visit to the outpatient clinic. In total, the median score of the item “The visit overall” was 100 (range, min-max, 50–100) and there was no significant difference (p = 0.06) in patient satisfaction between patients examined by an OS or an APP, respectively (Table 3).

Secondary outcomes

The VSQ-9 total score was not significantly different between patients examined by OSs and APPs (Table 3). 
The mean score of VSQ-9 items 5–8 (regarding direct interaction with the health provider) showed a statistically significant difference between groups (p = 0.01) in favour of APPs (Table 3).

The median scores for items 1–8 ranged from 50 to 100 with the highest scores in the items about direct interaction with the health provider (Table 3). Some of the items showed a statistically significant difference between patients examined by OSs and APPs. In item 4, “Waiting time at the office,” patients were more satisfied with the OSs, while patients were more satisfied with the APPs in items 6–8, “Explanations during consultation,” “Technical skills,” and “Personal manner” (Table 3). The distribution of responses in items 4–9 is presented in Figure 2.

Figure 2 .

Distribution of responses in items 4–9 of VSQ-9.

Sensitivity analysis

The chi-square test revealed that two items of VSQ-9 (“Waiting time at the office” and “Technical skills”) no longer showed a statistically significant difference. The results of the primary outcome “The visit overall” and the remaining secondary outcomes remained the same.

Qualitative results

Nine patients (three from each APP) participated (Table 4).

Table 4 .

Characteristics of Interview Participants

Gender	Age (years)	Employment status	Satisfaction with “visit overall”
Female	52	Employed	Excellent
Male	23	Employed	Excellent
Male	39	Unemployed	Very good
Female	49	Employed	Excellent
Male	61	Employed	Good
Male	73	Retired	Very good
Male	57	Employed	Excellent
Female	70	Retired	Excellent
Female	42	Employed	Excellent

The length of interviews lasted between 9 and 16 minutes and in general, patients expressed several similar points. They were highly satisfied with the examinations, and the areas found to be less satisfying related to the organization and logistics of the appointments. Six main categories were identified. Five of them (APPs’ health professional profile, level of information, expectations, physical environment, collaboration in the clinic) were predefined matrix categories adapted from the interview guide (online Appendix 2). The last category concerned practical issues.

The APPs’ health professional profile

There was consensus among patients on feeling confident in and being satisfied with the APP. They considered the APP a competent examiner, with a few patients mentioning that they believed the APP to be just as competent as an OS. Three patients emphasized they felt the APP made time to both listen and do a thorough examination. Most patients felt involved in the decision made about further actions concerning their treatment, but some also expressed that there was no real alternative.

Level of information

The main topic among patients was the APPs’ ability to communicate in an understandable manner.

“I think it’s positive that [APP1] was good at explaining, and [APP1] took the time, surprisingly good time, I think”

(Female, 49 years old)

They expressed satisfaction with the APPs’ way of explaining everything during the consultation and that they received clear information about future steps; some patients associated this with the APP being competent.

Expectations

Most patients hoped for clarification of their condition and to be pain free. A few had expected to be examined by an OS, but still, they were satisfied to be examined by an APP:

“Actually, I thought that due to the name the Shoulder Clinic, it would be some very specialized consultant, but … but it was not, but it was fine. The APP seemed very competent, so there’s nothing to that at all”

(Male, 61 years old)

Only three patients were certain that an APP had examined them. Three had doubts about which type of health professional they had been examined by, and two were certain they had been examined by an OS. The fact that two patients thought an OS had seen them could indicate that this was their expectation. It could also speak to the more general acceptability of the performance and quality of the service delivered by the APP.

Physical environment

All patients expressed satisfaction with the physical environment of the clinic, such as the waiting room.

Collaboration in the clinic

Three patients experienced that the APP went out of the room to consult with an OS. They were happy to get a second opinion from an OS even though they had to wait. One expressed satisfaction at receiving the treatment (steroid injection) immediately.

Practical issues

When patients expressed being less satisfied, it concerned practical matters such as insufficient information about X-ray before examination, a need for free parking, the location of the clinic, waiting time to get an appointment, need for influence when scheduling a new appointment, and lack of collaboration between their general practitioner and the clinic concerning exchange of information (though some patients lauded the collaboration with the municipality physiotherapists).

Discussion

Main findings

Overall, the quantitative results showed high levels of satisfaction with the examinations. We found no statistically significant difference between groups neither concerning the primary outcome “The visit overall” nor in the VSQ-9 total mean score. Looking at direct interaction with the health provider (mean score of VSQ-9 items 5–8), we found a significant difference in favour of the APPs. Also, patients were more satisfied with the APPs’ explanations during the consultation, technical skills, and personal manners, while patients were more satisfied with the waiting times for OSs at the Shoulder Clinic.

The qualitative interview data showed a high overall satisfaction and patients were particularly satisfied with the APPs’ ability to explain procedures and findings during the examination. When patients expressed less satisfaction, it concerned practical issues. This was also reflected in the quantitative data, where the lowest scores were related to access to care.

Interpretation

Patients were highly satisfied with the examination at the Shoulder Clinic. The higher satisfaction level with the APPs, in some of the secondary outcomes, could be a result of the duration of the examination, because APP examinations lasted longer than those performed by OSs. This was supported by the interviews, where the APPs’ time spent to both examine and explain during the examination was highly appreciated. The high satisfaction with both the OSs and APPs is concordant with the results from both Razmjou and colleagues¹³ and a recently published study by Lowry and colleagues,¹² both using a modified version of the VSQ-9. This means that we are not able to directly compare our result on the VSQ-9 total score with the results from these studies. However, in some specific results, comparison is possible. Rating of “The visit overall” did not significantly differ between APPs and OSs in our study, which is contrary to the study by Razmjou and colleagues, where a statistically significant difference in favour of the APP was found.¹³ The causes for this difference are unclear, but could be due to having three APPs and not just one performing the examinations, unknown differences in the populations (e.g., diagnoses or socio-demographics), or a smaller sample size in our study. Both studies reported median scores related to answering in the categories “very good” or “excellent,” and both also found a significantly higher satisfaction level of patient-perceived technical skills and personal manners of APPs compared with OSs. No qualitative studies evaluating patient satisfaction with APPs examining shoulder disorders were identified, but our results are comparable with findings in a study by Fennelly and colleagues investigating the experiences of patients with musculoskeletal disorders with an APP service in an orthopaedic setting.²² Both studies report overall positive patient experiences and appreciation of the APPs’ interpersonal and professional skills.²² The high level of patient satisfaction in these areas is especially relevant, 
as patients with spinal disorders consider them important to the quality of APP services.¹⁸ The relevancy of measuring interpersonal and professional skills is supported by Waters and colleagues, who found themes such as trust, empathy, communication, expectation, and relatedness to influence patient satisfaction.²³ Other themes identified were clinic waiting time and clinical contact time.²³ In our study, patients examined by an OS were more satisfied with clinic waiting time than were patients examined by an APP, however the fact that APPs had a longer clinical contact time than OSs might have potentially had a positive impact on patient satisfaction among patients examined by APPs. Results from both Reeves and May¹⁸ and Fennelly and colleagues²² show that patients prefer a smooth and efficient patient care pathway, including short waiting times and timely access to the hospital system.^18,22 In our study, waiting time to get an appointment was the item with the lowest level of satisfaction. However, the level of satisfaction was comparable among patients examined by APPs and OSs, and one could speculate that waiting time would be even longer if the APPs were not part of the Shoulder Clinic.

Although the qualitative studies we mention^18,22,23 were not specifically conducted in populations with shoulder disorders, we believe their findings support our qualitative findings and our methods used (items from VSQ-9 and categories comprised in the interview guide) to evaluate quality and satisfaction from a patient perspective.

The quality concept employed by Danish authorities comprises three dimensions; patient perceived, quality of health professional, and organizational quality.¹⁷ The results in this study indicate that patient-perceived quality was high. Provided that the quality of the two other dimensions is high, the study supports the use of APPs to perform examinations in an orthopaedic shoulder clinic. In the Shoulder Clinic, these two dimensions were examined in a different study.²⁴ That study showed that most of the patients received the same or partly the same diagnosis and treatment plan from the APP and the OS. Furthermore, both the OSs and APPs expressed that close communication, a respectful relationship, and professional skills were important to make their shared task of examining and diagnosing shoulder patients work.²⁴ An additional and important issue for further research would be to review the costs of implementing such a model.

Strengths and limitations

One of the strengths of the current study is the use of both quantitative and qualitative methods to examine patient satisfaction. Previous research indicates that satisfaction questionnaires using Likert scales tend to overestimate satisfaction level.²⁵ In order to reduce the risk, Riiskjær and colleagues²⁵ recommend using both closed-ended and open-ended questions. In this study the qualitative research showed high patient-reported satisfaction, and the interviews gave a more detailed insight into satisfaction expressed by patients.

Another strength of this study is the response rate (89%), which is high compared with other studies.^26–29 We don’t know the reasons for the non-responses but assuming dissatisfaction with past experiences was one of them, the high response rate in this study reduces the risk of overestimating the high level of satisfaction.

Furthermore, patients filled in the questionnaire immediately after the examination without the involvement of the health professional, thereby reducing the risk of recall bias and of patients answering too positively in order to please the examiner. We also consider it a strength that a sensitivity analysis of the results based on VSQ-9 data was performed, showing only slightly different results.

As for limitations, the VSQ-9 questionnaire was translated into Danish. Even though the translation process was thoroughly performed, the Danish translation was not validated, which might have impacted the accuracy of measurements.

The fact that participants were not randomized to type of examiner induced a risk of bias when comparing the two groups. However, as the patients were allocated based on first available time slot, and by a secretary not otherwise involved in the study, we assess this risk to be low.

Finally, two of the authors (Madsen and Mikkelsen) are physiotherapists and Klebe is employed at the Shoulder Clinic, and so a potential positive cognitive bias towards OSs and APPs sharing the task could exist. We believe that this is sufficiently dealt with by involving external researchers (Bødskov and Nielsen) during the entire research process and by having them perform the main part of the analysis.

Generalizability

This study was conducted at a single location, which might reduce external validity, but we believe that the fact that patients were examined by all four OSs and three APPs employed at the Shoulder Clinic contributes to increased generalizability. Also, the population in this study was selected to avoid potential bias due to possible differences in patient satisfaction between triage groups (Table 1). Therefore the results from this study may only be generalized to patients with shoulder disorders who are not obvious candidates for surgery. Caution is warranted when comparing our results to the situations in other countries, because the professional role of the APPs as well as their education and the organizational setups differ internationally.³⁰

Conclusion

The results showed a high level of patient satisfaction with the diagnostic examination at the Shoulder Clinic by both OSs and APPs, with no difference between groups concerning “The visit overall” and the total VSQ-9 score. The mean score of items 5–8 regarding direct interaction with the health professional showed that patients were more satisfied with the examinations by APPs compared to those by OSs. Patients were in particular satisfied with the APPs’ ability to explain their findings and plan for the patients’ further clinical pathways. These results support the conclusion that from a patient perspective, APPs can be used to examine and diagnose selected patients in an orthopaedic shoulder clinic.

Key Messages

What is already known on this topic

• The use of APPs to examine and diagnose patients with shoulder pain has increased.

• Studies exclusively investigating patient satisfaction with the use of APPs in orthopaedic shoulder clinics are sparse. One study found significantly higher patient satisfaction with examination by an APP compared to an OS, but generalizability of the results is limited.

What this study adds

• Using both quantitative and qualitative methods to evaluate patient satisfaction with being examined and diagnosed by an APP provided a detailed view on the patients’ experiences.

• When comparing APPs and OSs, no difference was found in patient satisfaction concerning “the visit overall.”

• Patients were particularly satisfied with the APPs’ ability to explain and communicate their findings and plan for further action.

• The results support the conclusion that from a patient perspective, APPs can be used to examine and diagnose selected patients in an orthopaedic outpatient shoulder clinic.