Inter-rater reliability of the Shoulder Symptom Modification Procedure in people with shoulder pain

Abstract

Background

Musculoskeletal conditions involving the shoulder are common and, because of the importance of the upper limb and hand in daily function, symptoms in this region are commonly associated with functional impairment in athletic and non-athletic populations. Deriving a definitive diagnosis as to the cause of shoulder symptoms is fraught with difficulty. Limitations have been recognised for imaging and for orthopaedic special tests. 1 solution is to partially base management on the response to tests aimed at reducing the severity of the patient's perception of symptoms. 1 (of many) such tests is the Shoulder Symptom Modification Procedure (SSMP). The reliability of this procedure is unknown.

Methods

37 clinician participants independently watched the videos of 11 patient participants undergoing the SSMP and recorded each patient's response as improved (partially or completely), no change or worse. Inter-rater reliability was assessed by Krippendorff's α, which ranges from 0 to 1.

Results

Krippendorff's α was found to range from 0.762 to 1.000, indicating moderate to substantial reliability. In addition, short (3-hour) and longer (1-day) durations of training were associated with similar levels of reliability across the techniques.

Conclusions

Deriving a definitive structural diagnosis for a person presenting with a musculoskeletal condition involving the shoulder is difficult. The findings of the present study suggest that the SSMP demonstrates a high level of reliability. More research is needed to better understand the relevance of such procedures.

Trial registration number

ISRCTN95412360.

What are the new findings?

• Deriving a definitive structural diagnosis for musculoskeletal conditions involving the shoulder is difficult.

• Symptom improvement/correction/modification tests have been suggested by clinicians as one method of developing a management programme.

• This study demonstrated inter-rater reliability of the Shoulder Symptom Modification Procedure.

How might it impact on clinical practice in the near future?

• A graduated exercise programme is the most common form of management, for people with the majority of musculoskeletal conditions involving the shoulder.

• If future research demonstrates that techniques used in the Shoulder Symptom Modification Procedure confer additional benefit when incorporated into a graduated shoulder exercise programme over exercises alone, then methods such as these may have a role in the management of musculoskeletal conditions involving the shoulder.

Background

As a group, musculoskeletal conditions are associated with the second highest number of ‘years lived with disability’.¹ Within this group, conditions affecting the shoulder occur frequently in sporting and non-sporting populations,^2–4 and their prevalence increases with age.⁵ Annually, 1–2% of the general population present to their general practitioner (family physician) with a first episode of shoulder pain,^6–8 and of concern, these conditions are associated with high levels of morbidity lasting for 1 year or longer.^{3 8}

To understand the basis of the presenting shoulder symptoms, clinicians typically perform a clinical examination, which usually includes: taking a history, collecting disability and impairment data and performing special orthopaedic tests that have been designed to incriminate pathology, such as that involving the rotator cuff tendons, subacromial bursa or glenoid labrum, or to rule in conditions, such as subacromial impingement syndrome.⁹

Although orthopaedic tests are commonly used,⁹ findings from narrative^10–13 and systematic reviews^{14 15} and research investigations¹⁶ have consistently questioned the value of these procedures as a method of implicating the structures associated with the presenting symptoms. Imaging is commonly used to support the clinical assessment.¹⁷ Likewise, the certainty with which imaging findings support or confirm the clinical diagnosis is challenged by myriad studies reporting asymptomatic structural deficits, including full-thickness rotator cuff tears and glenoid labral tears, in populations including elite athletes.^18–21 One implication of current clinical practice is that people with shoulder pain may undergo operations to repair tissues that are not related to their presenting symptoms.¹³

The findings of these clinical and radiological investigations have challenged the basis on which a structural diagnosis may be achieved.^{10 13 22} This has been recognised previously and researchers have suggested that assessment and management could be based on the presenting symptoms without the need for a definite structural diagnosis.^{23 24} One such model, known as the Shoulder Symptom Modification Procedure (SSMP), was first described by Lewis¹⁰ as a systematic approach to assess clinical variables that may be associated with shoulder symptoms, to determine their relationship with the presenting symptoms. Similar to the Mulligan and McKenzie et al approaches,^{23 24} procedures identified that partially or completely improve the presenting symptoms may be considered in patient management. By placing the individual patient at the centre of the assessment and management decision process, these methods are compatible with patient-centred practice, clinical reasoning and evidence-based practice.^{22 25 26}

Shoulder Symptom Modification Procedure

The first stage of the SSMP is for the patient to identify the movements, activities or postures that reproduce symptoms. This may include symptoms experienced while sitting at a desk, lifting a pan or kettle, dressing, swimming, performing weight-bearing activities such as push-ups and in high-powered explosive activities commonplace in sport. Pain is the most commonly reported symptom, but symptoms may also include reduction in movement, instability and symptoms that may be associated with neurovascular compromise. Once defined, the component parts of the SSMP are then applied while the patient performs the symptom-provoking movements, activities or postures to determine if an immediate change is achievable. This type of ‘real-time’ process has been recommended previously,^{23 24} and evidence (albeit limited) suggests that procedures found to improve symptoms in the cervical and lumbar regions within a session may be useful in guiding treatment selection and may help predict between-session changes in symptoms.^27–29

The SSMP comprises three main sections. The first section aims to assess the relationship between thoracic posture and symptoms, the second aims to evaluate the effect of scapular position on symptoms and the third aims to assess the effect of the relationship between the humeral head and scapula on symptoms. In reality, the assessment procedures do not isolate one structure. For example, reducing the thoracic kyphosis also relatively posteriorly tilts the scapula, changes length-tension relationships of muscles, tendons and related soft tissues and may influence joint biomechanics. As all procedures involve touch, another reason for perceiving a change in symptoms may be the experience of this sensation.³⁰ Additionally, there is only very limited evidence that humeral head procedures actually influence humeral head position.³¹

The SSMP assessment form is detailed in figure 1. The specific assessment procedures have been described elsewhere.^{10 11 13} Following agreement between both parties, the person with shoulder symptoms informs the clinician if an individual procedure: partially or completely alleviates symptoms; has no change on symptoms; or makes the symptoms worse. Techniques may be combined; for example, if reducing the thoracic kyphosis and elevating the scapula independently partially reduce symptoms, then the clinician may assess the response of combining both these procedures. If the SSMP completely and consistently alleviates symptoms, then the procedures found to alleviate the symptoms are used to inform treatment.

Figure 1.

The Shoulder Symptom Modification Procedure assessment form.

It is important for clinicians to appreciate that the SSMP is not a stand-alone procedure and if the SSMP does not change symptoms or only partially alleviates them, other rehabilitation based on the clinician's clinical reasoning and the patient's acceptance of that management need to be considered, such as advice, education, rotator cuff rehabilitation exercises,^{11 13} injection therapy or surgery.^{32 33}

Although in clinical use,¹¹ the reliability of the SSMP is uncertain. The primary aim of this investigation was to evaluate the intertester reliability of clinicians in determining how people with shoulder symptoms respond to SSMP procedures. The secondary aim was to investigate the differences in reliability between those that participated in long training (over 1 day) and short training (3 hours) in the SSMP. GRAAS recommendations for reporting reliability studies were used as a guide.³⁴

Methods

Ethical approval and study registration

Ethical approval for the investigation was granted by the Faculty of Education and Health Sciences Research Ethics Committee, University of Limerick, Ireland (2015_12_13_EHS), and from the Health and Human Sciences Ethics Committee, University of Hertfordshire, UK. The investigation was registered—ISRCTN95412360.

Patients

A sample of convenience of 11 people with unilateral shoulder pain, recruited from community and clinical settings, consented to participate in the investigation. They were provided with participant information documentation and informed of their rights, including the right to withdraw from the investigation at any stage, without having to explain this decision. Prior to participation, all patients signed consent documentation, after which they provided demographic data and a rating of their present pain on a 0–10 scale (0, no pain; 10, worst imaginable pain). Once the videos were filmed, the patient participants' involvement in the study was complete.

Clinicians

A sample of convenience of 40 clinicians from physiotherapy and osteopathy were approached to participate in the investigation.

Clinician participants worked in variety of health settings, including the public and private sectors and in primary and secondary care. They had varied training in the SSMP. Some had previous experience with the SSMP and were using it in current clinical practice, while others were new to the procedure. A number of clinicians had participated in previous training (∼1 day) and to varying extents had incorporated the SSMP into their clinical practice. Others were recruited for the purpose of the investigation and received short training (3 hours duration). As such, clinician participants were not randomised into these long and short training subgroups. The clinicians were given consent documentation, and were made aware of their rights, including the right to withdraw from the study at any stage. Those providing consent also provided demographic data.

Procedure

Video analysis has been used in previous musculoskeletal conditions to investigate the reliability of assessing posture and movement,^35–39 including shoulder research,^40–42 and was determined to be the most appropriate method for the current investigation. The use of videos ensured that a large number of assessors were able to observe the patient's response to the SSMP concurrently from the same angle.

Video filming occurred in a clinical research room at the University of Limerick, Ireland. Videos were made of one of the investigators (JSL) conducting the SSMP on the 11 patient participants. All videos were filmed on the same day. The videos were filmed and audio recordings were made using two JVC Everio Camcorders (Model No. GZ-MS210BEK) cameras (Yokohama, Japan), mounted on extendable tripods positioned ∼1.5 m from the patient participants. To standardise the position, patients were instructed to stand on a cross taped to the floor in front of the cameras. To reduce distortion, the cameras were positioned as close to perpendicular to the patients as possible. Initially, the patient participants were requested to identify and demonstrate the movement that reproduced their symptoms. Following this, the SSMP assessment procedures were performed and the patients' responses filmed, and the patients were asked whether the symptoms were the same, worse or better. At the end of data collection, 167 unique video recordings were available for analysis. The video recordings were initially edited using Adobe Creative Cloud Premier Pro (http://www.abobe.com) and then converted to .avi files using PRISM video converter software (http://www.nchsoftware.com). These .avi files were played using Windows Media Player (http://www.microsoft.com). The duration of each audio and video clip ranged from 26 to 150 s, with most being under 1 min. The video recordings were uploaded onto a secure server located at the University of Limerick.

Clinician participants were provided with a unique log in and password to the server and independently watched the video clips and completed the data collection documentation. Each video was assigned a separate table on the documentation sheet and after watching each video, the clinician participants were required to record if the SSMP technique had produced no change, made the patient worse or resulted in either partial or complete improvement. The clinicians were informed that responses were to be informed by the responses provided by the patients and not by their own interpretations. Owing to technical constraints, the order of the video clips was not randomised and the clinician participants could choose to watch the video clips in any order. They were encouraged to carry out the task in their own time and in a quiet place without interruptions, and to take breaks as necessary. Clinicians were instructed that ideally they should only watch the video on one occasion but were permitted to watch on two occasions if they were unsure of the patient's responses. Confirmation of this type would occur in clinical practice in such cases of uncertainty. Clinician participants were instructed that they should record:

• ‘no-change’ if the patient reported that the technique had not changed their symptoms,

• ‘worse’ if the patient reported that the technique had increased their symptoms,

• ‘partial improvement’ if the patient reported that the technique had partially improved their symptoms, which was defined as anything between 1% and 99% improvement, and

• ‘complete improvement’ if the patient reported that the technique had completely alleviated their symptoms (ie, 100% improvement).

To reduce bias, clinicians' scores were entered into a database by a research assistant who was unaware of the purpose of the investigation. Once the data sheet was complete, the clinicians' involvement in the study was complete. At the end of the data collection period, to protect patient confidentiality, the videos were removed from the secure server and destroyed.

The focus of the analysis of the data was on inter-rater reliability. No attempt was made to assess intrarater reliability, for the following reasons:

1. intrarater reliability can be assumed to be at least as good as inter-rater reliability, and as the primary practical concern is to assess the lower limit of reliability, a separate assessment of intrarater reliability is of little interest;

2. a repeated assessment of the same videotaped technique by the same clinician would have little relevance to clinical practice;

3. given that the SSMP aims to improve symptoms, if the technique had been videotaped twice, the technique itself might have altered, such that a subsequent test of the same procedure would not be testing the same response, violating a core assumption when assessing intrarater reliability.^{35 43} Other procedures that aim to modify symptoms have reported similar immediate responses.^{44 45} This phenomenon is clearly demonstrated in other symptom modification procedures (https://www.youtube.com/watch?v=Arkxz8rabGQ&utm_content=buffer6f7c4&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer).

4. asking participating clinicians to produce assessments within and between patients would have been onerous and might have discouraged their participation.

Description of techniques

Table 1 describes the techniques assessed in the current investigation.

Table 1.

Description of techniques

Technique	Description
Thoracic extension	The patient is asked to place a finger (typically from the asymptomatic upper limb) on the sternum and while still maintaining contact with the sternum, gently ‘lift’ the finger superiorly, aiming to extend the thoracic kyphosis
Scapular elevation	The therapist gently places one hand over the lateral border of the scapular and elevates (ie, upwardly rotates) the scapula ∼1–2 cm using the other hand on top of the shoulder girdle as a guide. This then becomes the new ‘starting position’ for shoulder movement. The scapula is free to move during arm movement but starts and returns in the elevated position
Scapular depression	The opposite direction to scapular elevation
Scapular retraction	The therapist gently places one hand over the lateral border of the scapular and retracts the scapula ∼1–2 cm using the other hand on top of the shoulder girdle as a guide. This then becomes the new ‘starting position’ for shoulder movement. The scapula is free to move during arm movement but starts and returns to the new retracted position
Scapular posterior tilt	The therapist gently places one hand over the lateral border of the scapular and the thumb over the inferior angle of the scapula. The other hand on top of the shoulder girdle gently displaces the superior aspect of the scapula (and other structures) posteriorly. This then becomes the new ‘starting position’ for shoulder movement. The scapula is free to move during arm movement but starts and returns to the new posterior tilted position
Combinations	If a number of scapular positions are found to be partially reduce symptoms, they can be combined to determine if further improvement if achieved (eg, elevation and posterior tilt; retraction, depression and posterior tilt)
Depression—flexion	In sitting or standing, the patient's shoulder is flexed as close to 90° flexion as possible (maybe in less or more range, depending on symptoms), the elbow is flexed (ie, shortened lever arm). The therapist places his/her hand on the posterior surface of the distal end of the humerus, 2–3 cm proximal to the point of the elbow. The patient is then asked to push the elbow towards the ground with the therapist resisting isometrically for 5–6 s. The contraction is repeated 3–4 times and the arm gently lowered to the side and the provocative movement retested
Depression—abduction	The same as for depression—flexion but the starting position is with the shoulder in the plane of the scapula or closer to anatomical abduction if appropriate
Depression—flexion (supine)	This technique is the same as depression—flexion but is performed in supine and in addition to the muscle contraction procedure, a series of inferiorly directed gliding pressures are applied to the region of the humeral head. Following the technique, the provocative movement is retested
Depression—abduction (supine)	The same as for depression—flexion (supine), but the starting position is with the shoulder in the plane of the scapula or closer to anatomical abduction if appropriate
Eccentric flexion	In sitting or standing with the shoulder flexed just before the onset of symptoms, the hand loosely grips an elastic rubber resistance tube, which is firmly suspended from above (ie, over the top of a door). With the arm in the same position, tension is applied to the tube and then the hand firmly holds the tube. Following this, the patient is instructed to extend the shoulder ∼20–30° hold isometrically for 5–6 s and then slowly return to the starting position (ie, concentric, isometric and eccentric contractions). This is repeated 3–4 times, the tube released and the provocative movement retested
Eccentric abduction	The same as for eccentric flexion but the starting position is with the shoulder in the plane of the scapula or closer to anatomical abduction if appropriate
External rotation	If the provocative movement is shoulder flexion or abduction, the movements are performed with increased shoulder external rotator activity. This could be achieved by using the resistance of an elastic rubber band, the therapists hand or pushing against a wall using a towel, plastic bag or polishing cloth to reduce resistance
Internal rotation	The same as for external rotation with resistance aimed at increasing an internal rotation force. In addition to the suggestions above, internal rotation resistance can be achieved by asking the patient to flex the shoulders while applying pressure to a ball the size of a soccer or basketball
AP|AP with inclination	Using a mobilisation belt, heavy resistance elastic rubber band, or a neoprene strap placed over the region corresponding to the anatomical location of the humeral head a posteriorly directed force is applied by the therapist with the therapists other hand stabilising the scapula. While the pressure is applied, the provocative movement is retested. This may be shoulder abduction-external rotation as may occur in someone with an anteriorly unstable shoulder. Care needs to be taken. In addition to trialling different amounts of posteriorly directed force, the therapist can apply a posteriorly directed force with a superior inclination to assess if this combination more effectively reduces symptoms
PA|PA with inclination	The same as for AP|AP with inclination, but with the pressure applied to produce an anteriorly directed force

AP, anterior to posterior; PA, posterior to anterior.

Statistical analysis

The inter-rater reliability of clinicians' assessment of the response to SSMP procedures was calculated by analysing the responses provided by the clinicians (no change, worse, partial improvement, complete improvement) using Krippendorff's α.⁴⁶ This statistic is a reliability coefficient suitable for analysing responses from multiple raters, and accommodates missing data. It ranges from 0 to 1, with 1 representing perfect agreement, and was calculated with the ratings defined as ordinal. A 95% CI for α and the probability of not attaining a coefficient of at least 0.800 were obtained through bootstrapping (10 000 samples). The 95% CI gives a range of plausible values for the ‘true’ reliability, such that we can be 95% confident that the true reliability is at least the lower limit of the CI, while the probability value indicates the probability of the ‘true’ reliability not attaining a minimum threshold of 0.800. Reliability was only calculated where at least three patients were assessed with any one procedure. The rate of missing values for each procedure was calculated as the number of times that a rater did not provide a rating, out of the total number of possible ratings (n of patients×n of raters). Analyses were conducted in SPSS V.23.

Sample size

There do not appear to be formal methods of calculating sample size for reliability studies with ordinal outcomes and multiple raters. However, methods for continuous outcomes, such as those described by Walter et al,⁴⁷ may provide some guidance. For example, with 20 or more raters, 10 patients would provide at least 80% power to detect a coefficient of 0.800 as greater than a null value of 0.500, at a 5% significance level.

Results

Eleven patient participants consented to participate. Each presented with unilateral shoulder pain and was naïve to the SSMP procedure. The mean age was 53.7 years. Seven were men and six had symptoms involving the right shoulder. Patient participant demographic information is detailed in table 2.

Table 2.

Patient participant information

Patient	Sex	Age (years)	Height (cm)	Weight (kg)	Symptomatic side	Duration (months)	Pain score	Onset	Previous treatment
1	M	51	168	92	L	9	3	Changing swimming technique	PT
2	M	68	168	79	L	12	4	Fall onto shoulder	CS inj (×3), PT
3	M	27	180	78	L	36	5	Dislocation	CS inj (×3), PT
4	F	69	166	68	R	3	8	Insidious	CS inj
5	M	54	182	80	R	12	6	Frozen shoulder (IDDM)	CS inj, PT
6	M	69	170	90	R	18	7	Insidious	Acupuncture
7	F	48	180	76	R	72	4	Insidious	None
8	F	50	159	91	R	14	5	Insidious	NSAIDs
9	F	60	153	80	L	8	5	Insidious	CS inj
10	M	28	172	99	L	1.5	4	Trauma (rugby tackle)	PT
11	M	67	170	83	R	36	3	Insidious	PT
Mean		53.7	169.8	83.3		20.1	4.9
SD		15.2	8.8	8.8		20.7	1.6

Duration (duration of symptoms of this episode), pain score (verbal: 0, no pain; 10, worst imaginable pain).

CS inj, corticosteroid injection; F, female; IDDM, insulin-dependent diabetes mellitus; L, left; M, male; PT, physiotherapy; R, right.

Of the 40 clinicians approached (as a sample of convenience) to participate in the investigation, 37 (92.5%) provided responses. Of the three who did not respond, two cited insufficient time as being the reason for not completing the data sheets; the reason for the other clinician is unknown. There were 20 female and 17 male clinician participants (36 physiotherapists and 1 osteopath). Eighteen had participated in a short (∼3 hours) training programme to explain and practice the SSMP. Nineteen had participated in a longer training programme (∼1 day). Clinician participant demographic information is detailed in table 3.

Table 3.

Clinician participant information

Clinician	Sex	Age (years)	Occupation	Years working	Number of people with shoulder pain treated each week	SSMP 3-hour training (n=18)	SSMP 1-day training (n=19)
1	F	27	PT	0.6	25		Y
2	M	55	PT	27	45		Y
3	M	49	PT	23	4		Y
4	F	27	PT	5	25		Y
5	F	45	PT	24	6		Y
6	F	26	PT	3.5	20	Y
7	F	51	PT	26	6		Y
8	M	28	PT	2	13	Y
9	F	34	PT	10	6		Y
10	F	31	PT	8	16		Y
11	F	27	PT	1.5	15	Y
12	M	44	PT	4	10		Y
13	F	40	PT	19	8	Y
14	F	46	PT	23	10		Y
15	F	36	PT	14	5		Y
16	F	25	PT	4	26	Y
17	M	27	PT	7	20	Y
18	M	28	PT	7	25	Y
19	M	42	PT	12	5		Y
20	M	50	PT	21	30	Y
21	M	51	PT	24	10	Y
22	M	29	PT	4.5	10		Y
23	M	28	PT	4	20	Y
24	F	34	PT	6	10	Y
25	F	34	PT	7	20	Y
26	M	30	PT	8	12		Y
27	M	52	PT	21	5		Y
28	F	54	PT	26	6		Y
29	F	53	PT	6	15	Y
30	M	26	PT	5	20		Y
31	F	25	PT	3	20	Y
32	F	49	PT	29	5	Y
33	M	42	PT	17	10		Y
34	M	28	Osteopath	1	12	Y
35	F	29	PT	8	18	Y
36	M	33	PT	8	10		Y
37	F	28	PT	6	10	Y
Mean		36.8		11.5	14.4
Range		25–55		0.6–29	4–45
SD		10.3		8.9	8.8

Response to the SSMP

In total, 19 procedures and combinations were tested, representing isolated procedures (eg, scapular elevation) and, when indicated, procedures tested in combination (eg, thoracic extension and scapular posterior tilt). The responses to the procedures are detailed in table 4. Responses to each of these procedures were assessed by all 37 clinicians, though the number of patients varied from 3 to 11. On 14 (10.4%) occasions, patients reported a worsening of symptoms and no change was reported 29 (21.6%) times. On 91 occasions (67.9%), participants reported a partial or complete reduction in symptoms. The intertester reliability of the clinicians' ratings is presented in table 5.

Table 4.

Patient participant response to SSMP techniques

		Responses to technique The manner by which the patient participants responded to the technique (ie, worse, no change, partial reduction, complete reduction)
Technique	Number of patients	Worse	No change	Partial	Complete
AP pressure	10	3	0	6	1
Eccentric abduction	7	0	1	6	0
Eccentric flexion	3	0	1	2	0
AP pressure with superior translation	4	0	1	2	1
External rotation in flexion	3	0	2	1	0
External rotation in abduction	7	1	1	3	2
Internal rotation in flexion	4	1	1	2	0
Internal rotation in abduction	6	1	1	2	2
Depression in flexion	3	0	0	3	0
Depression in abduction	8	0	1	4	3
PA pressure	8	0	2	3	3
Scapular elevation	11	0	4	4	3
Scapular elevation and posterior tilt	3	0	0	2	1
Scapular elevation, retraction and posterior tilt	3	0	0	3	0
Scapular depression	11	3	3	5	0
Scapular posterior tilt	11	2	1	6	2
Scapular protraction	10	2	3	5	0
Scapular retraction	11	1	2	7	1
Thoracic extension	11	0	5	5	1
Total	134	14 (10.4%)	29 (21.6%)	71 (53.0%)	20 (14.9%)

AP, anterior to posterior; ER, external rotation; IR, internal rotation; PA, posterior to anterior.

Table 5.

Intertester reliability, whole cohort of clinicians

Movement	α	95% CI	Probability of not attaining 0.800	Number of patients	Missing ratings
AP pressure	0.846	0.802 to 0.888	0.020	10	6/370
Eccentric abduction	0.821	0.717 to 0.914	0.334	7	0/259
Eccentric flexion	0.928	0.878 to 0.970	<0.001	3	0/111
AP pressure with superior translation	0.783	0.702 to 0.857	0.647	4	4/148
External rotation in flexion	0.874	0.806 to 0.935	0.018	3	0/111
External rotation in abduction	0.826	0.775 to 0.873	0.147	7	5/259
Depression in flexion	NC	NC	NC	3	1/111
Internal rotation in flexion	0.762	0.690 to 0.828	0.863	4	0/148
Internal rotation in abduction	0.894	0.863 to 0.922	<0.001	6	1/222
Depression in flexion	NC	NC	NC	3	1/111
Depression in abduction	0.915	0.861 to 0.958	<0.001	8	1/296
PA pressure	0.837	0.772 to 0.894	0.129	8	3/296
Scapular elevation	0.905	0.854 to 0.946	<0.001	11	0/407
Scapular elevation, posterior tilt	0.920	0.839 to 1.00	0.009	3	0/111
Scapular elevation, retraction, posterior tilt	NC	NC	NC	3	0/111
Scapular depression	0.838	0.761 to 0.907	0.160	11	1/407
Scapular posterior tilt	0.911	0.876 to 0.944	<0.001	11	2/407
Scapular protraction	0.928	0.874 to 0.973	0.001	10	0/370
Scapular retraction	0.851	0.790 to 0.908	0.051	11	0/407
Thoracic extension	0.921	0.853 to 0.976	<0.001	11	0/407

The NC values were due to insufficient variation in the rating to perform calculation.

AP, anterior to posterior; NC, not calculable; PA, posterior to anterior.

Nineteen clinicians had participated in longer training (over 1 day) and 18 clinicians over a shorter period (∼3 hours). The α coefficients for these two subgroups are presented in table 6. The mean difference in estimates of these coefficients (long training subgroup minus short training subgroup) was calculated as −0.001 (95% CI −0.052 to 0.0510). Figure 2 indicates the extent of the discrepancy in the reliability of assessments between clinicians in these subgroups. The ends of each horizontal bar indicate the value of α for each subgroup, such that the length of the bar indicates the magnitude of difference between these values.

Table 6.

Intertester reliability of those participating in long and short training

	Short training (n=18)		Long training (n=19)
Movement	α	95% CI	α	95% CI	Difference (long—short)
AP pressure	0.879	0.835 to 0.918	0.819	0.772 to 0.863	−0.060
Eccentric abduction	0.869	0.759 to 0.960	0.771	0.655 to 0.872	−0.098
Eccentric flexion	0.860	0.788 to 0.921	1.000	1.000 to 1.000	0.140
AP pressure with superior translation	0.816	0.722 to 0.895	0.745	0.650 to 0.828	−0.071
External rotation in flexion	0.751	0.657 to 0.839	1.000	1.000 to 1.000	0.249
External rotation in abduction	0.846	0.780 to 0.905	0.802	0.748 to 0.852	−0.044
Internal rotation in flexion	0.733	0.657 to 0.803	0.783	0.698 to 0.860	0.050
Internal rotation in abduction	0.920	0.894 to 0.945	0.867	0.827 to 0.902	−0.053
Depression in flexion	NC	NC	NC	NC	NC
Depression in abduction	0.908	0.855 to 0.954	0.922	0.857 to 0.967	0.014
PA pressure	0.800	0.723 to 0.868	0.871	0.808 to 0.922	0.071
Scapular elevation	0.868	0.794 to 0.929	0.937	0.908 to 0.964	0.069
Scapular elevation, posterior tilt	01.000	1.000 to 1.000	0.844	0.718 to 0.953	−0.156
Scapular elevation, retraction, posterior tilt	NC	NC	NC	NC	NC
Scapular depression	0.859	0.776 to 0.932	0.816	0.737 to 0.888	−0.043
Scapular posterior tilt	0.941	0.911 to 0.966	0.883	0.841 to 0.921	−0.058
Scapular protraction	0.902	0.828 to 0.960	0.952	0.904 to 0.989	0.050
Scapular retraction	0.906	0.841 to 0.960	0.805	0.735 to 0.869	−0.101
Thoracic extension	0.918	0.852 to 0.975	0.943	0.890 to 0.988	0.025

NC, not calculable.

Figure 2.

Differences in α for raters undergoing either short (S) or long (L) training.

Discussion

Deriving a definitive structural diagnosis for an individual presenting with shoulder pain is fraught with difficulty. Suggesting care pathways based on the responses to orthopaedic tests and imaging may not correctly represent the mechanisms underlying the presenting symptoms. This is due to a poor correlation between structural changes and symptoms and poor accuracy of the clinical orthopaedic tests themselves. Clinical diagnosis is further challenged by the need to appreciate, for those presenting with pain as the main symptom, whether the symptoms have a peripheral nociceptive driver or occur as the result of altered central pain processing.⁴⁸ Owing to these complexities, for many, clinical practice is currently based on assessing the response to techniques that do not require a structural diagnosis and using the responses of the assessment procedures to inform management.^{24 49} The SSMP falls within this category of clinical assessment. The findings of the current investigation suggest that clinicians are able to assess the patient's individual responses to the components of the SSMP with a good degree of reliability; the lowest point estimate of α was 0.762, for internal rotation in flexion, which is close to the threshold value for ‘substantial’ reliability of ≥0.810 proposed by Shrout.⁵⁰ The estimates were generally similar for those clinicians who had undertaken longer training (1 day) and those who had undertaken shorter training (3 hours); the largest discrepancy was for internal rotation in flexion. Moreover, there was no consistent pattern in these differences; reliability was higher in the short training subgroup for nine techniques and was higher in the long training subgroup for eight techniques.

Although the number of raters was constant, the number of patients in whom the reliability of the assessment of each technique could be assessed varied from 3 to 11, and the precision of the estimates of α (as represented by the width of the associated 95% CI) varied accordingly. Nonetheless, owing to the large number of raters and the low rate of missing values, a reasonable degree of precision was obtained even for estimates based on just three patients.

In this investigation, reliability was assessed using video analysis playback. This was chosen as pilot work prior to this research clearly demonstrated that the response to a technique could substantially change the ‘baseline’ for the second tester and therefore confound the possibility of determining the reliability of assessment. The use of videotapes ensured that all clinicians were assessing the same response. Before the SSMP should be considered to be a reliable clinical assessment procedure, the findings of this investigation need to be repeated in a larger sample of patients, as well as testing other methods of reliability such as direct observation of patients being assessed clinically.

Our findings suggest that clinicians can learn the component techniques of the SSMP and reliably determine if they have influenced the patient's symptoms in a relatively short period of time, and there do not appear to be substantial clinical differences in reliability if training is conducted over a 3 hour period or over the course of 1 day. However, it should be remembered that clinicians were not randomly allocated to the two durations of training, and a conclusive comparison of the two subgroups cannot therefore be made. It is important to emphasise that although the findings of this investigation suggest that the SSMP is a reliable assessment process, there is no evidence to suggest that incorporating the techniques into management positively influences outcome over natural history or other treatment procedures.

In a recent large multicentre cohort study (1030 participants at baseline, 811 participants at 6-month follow-up) investigating prognostic factors for people with shoulder pain, psychosocial factors were identified as the major determinant. Of the range of biomechanical factors included in the investigation, ‘real-time’ improvement in symptoms associated with changes to scapular posture during active shoulder elevation^{10 51} was the most consistent positive biomechanical prognostic factor identified at 6 months.⁵² An improvement in symptoms and/or range of shoulder elevation was demonstrated during manual facilitation of the scapula in 41% (n=426) of participants and near-complete or complete reduction in pain and/or restoration of shoulder elevation in 12% (n=122) of participants.⁵²

One of the potential benefits of assessment and management systems such as the SSMP is that demonstrating to a patient that symptoms are modifiable may give the individual confidence to move, due to the reduction or cessation of symptoms, which, in turn, may facilitate adherence to treatment.⁵³ Poor adherence has been shown to compromise the effectiveness of treatment,^{53 54} and as self-management is required in most chronic conditions, finding a technique that reduces or alleviates symptoms may encourage the patient and facilitate the management process. Therapy-related factors are one of the five dimensions influencing adherence to treatment.⁵⁴ Although there are many subcategories within this dimension, the immediacy of beneficial effect is cited as a factor influencing adherence (p. 30). From the patient's perspective, the perception that treatment is effective in ameliorating unpleasant symptoms is a precondition for continued compliance (adherence).⁵⁵ Although there is no empirical evidence to support this contention, procedures such as the SSMP, which may demonstrate immediate improvement in symptoms, may support adherence to an agreed management plan. Of relevance, people with chronic low back pain preferred exercises that were individualised and made sense, and felt their individualised needs were addressed; they were less likely to engage with exercises that were boring or lacked challenge.⁵⁶ Appropriate and balanced communication with patients is vital to frame the entirety of the management plan.

When asked if they attribute the cause of presenting symptoms to anything specific, people presenting with shoulder symptoms commonly implicate ‘poor posture’. Although deviations in posture (from an idealised norm) are frequently cited as the cause of shoulder pain and symptoms,^{57 58} this relationship has been repeatedly challenged,^59–62 and this in turn calls into question the extent to which clinical reasoning should be based on static observation of posture. Components of the SSMP involve changing posture during symptomatic activities. If symptoms consistently change, then these changes can be incorporated into the management plan. Also of relevance is that for an individual who is convinced, or who has been convinced, that posture is a key factor underlying the presenting symptoms, demonstrating no change or a worsening in symptoms when changing posture may alter this perception and this may thereby facilitate the acceptance of alternative management strategies.

Of importance, the SSMP is not a stand-alone procedure and must be embedded within a complete patient care management programme that includes education, support, advice, consideration of lifestyle and psychosocial factors, general fitness and other local management strategies. If SSMP techniques do not positively influence symptoms, other treatments or interventions may need to be considered. These may include (but are not restricted to) graduated shoulder exercises aimed at the rotator cuff and shoulder muscles.^{11 63–66}

Limitations

The findings of this investigation need to be interpreted in the light of certain limitations. Foremost of these is that the clinicians only viewed one physiotherapist performing the SSMP in video format. If the clinician participants had observed other clinicians performing the SSMP procedures, different estimates of reliability might have resulted. The use of videos was necessary owing to the large number of assessors and the need for them to observe the same responses to each technique; however, although not practicable in this study, it would be more clinically realistic to directly observe the clinician and the patient's responses.

The use of a larger sample of patients would have allowed the reliability of the SSMP to be evaluated over a wider range of clinical presentation. There was, however, a relatively large clinician sample, which provided precise estimates of the reliability coefficients. It also allowed the relative influence of short-duration and long-duration training on reliability to be determined, though this was not a randomised comparison and is subject to confounding by other factors. In addition, it is important to emphasise that the findings only relate to the reliability of clinicians' interpretation of the SSMP procedures; the consistency with which such procedures are applied is a separate issue. Finally, being a university laboratory, the environment where the procedures were conducted and filmed was a controlled environment that may not reflect the realities of clinical practice.

Future research

The purpose of this research was to investigate the intertester reliability of the SSMP. Suggestions to assess the influence of symptom modification in a systematic way have been made^{23 24 49} and the responses used to guide treatment. There is a pressing need to understand the relevance (if any) of these types of approaches in their ability to support patient management, not only in terms of clinical outcome (type of change, magnitude of change, duration of change), but also in terms of the mechanism(s), by which they may produce a change. There is need to determine if SSMP procedures, embedded within a framework of care (advice, education, graduated exercise), add any additional value to overall management. If they have contributed positively, their continued use should be considered and if not, concepts such as this should be abandoned. There would be benefit in qualitative research to better understand patients' perceptions of SSMP procedures.

Conclusions

Deriving a definitive structural diagnosis for a person presenting with a musculoskeletal condition involving the shoulder is difficult. Limitations have been recognised for imaging as well as for orthopaedic special tests. One solution is partially to base management on the response to tests aimed at reducing the severity of the patient's perception of symptoms. One (of many) methods is the Shoulder Symptom Modification Procedure. The findings of the present study suggest that the procedure demonstrates a good level of reliability. More research is needed to better understand the relevance and importance of such procedures.