Abstract
Introduction
Neuropathic low back-related leg pain (LBLP) can be a challenge to healthcare providers to diagnose and treat. Accurate diagnosis of neuropathic pain is fundamental to ensure appropriate intervention is given. However, to date there is no gold standard to diagnose neuropathic LBLP. A Delphi study will therefore be conducted to obtain an expert-derived consensus list of clinical indicators to identify a neuropathic component to LBLP.
Methods/analysis
Included participants will be considered experts within the field as measured against a predefined eligibility criterion. Through an iterative multistage process, participants will rate their agreement with a list of clinical indicators and suggest any missing clinical indicators during each round. Agreement will be measured using a 5-point Likert scale. Descriptive statistics will be used to measure agreement; median, IQR and percentage of agreement. A priori consensus criteria will be defined for each round. Data analysis at the end of round three will enable a list of clinical indicators to be derived.
Ethics and dissemination
Ethical approval was gained from the University of Birmingham (ERN_19-1142). On completion of the study, findings will be disseminated in a peer-reviewed journal and presented at relevant conferences.
Keywords: neurological pain, neuropathic pain, leg pain, diagnosis, Delphi
Strengths and limitations of this study.
This will be the first study to develop a list of clinical indictors to identify a neuropathic component to low back-related leg pain, which will help to inform clinicians regarding treatment interventions.
This study will use national and international experts in neuropathic pain to inform the list of clinical indicators.
This study will be reported in line with Conducting and Reporting Delphi Studies recommendations.
This study is a mixed method design and thus utilises quantitative and qualitative data.
The views of Delphi panellists may differ from those experts who decline participation, and may not fully represent experts in the field of interest.
Introduction
Neuropathic pain (NP) is defined as pain caused by a lesion or disease of the somatosensory nervous system.1 NP is a largely unmet medical need due to ineffective management.2 NP is highly prevalent and has been estimated to have a global population prevalence of between 6.9% and 10%.3 NP is associated with vast economic costs, with total annual costs (including direct and indirect costs) per patient in Europe ranging from £8710 in the UK to €14 446 in Germany.4 Epidemiological surveys have shown that many patients with NP do not receive appropriate treatment; a common reason for this is misdiagnosis.5 One of the most common presentations of NP is found in those with low back pain (LBP), which is estimated to be at approximately 46.7%.6 Up to two thirds of individuals with LBP report concomitant leg pain when presenting in primary and secondary care services.7 Low back-related leg pain (LBLP) compared with LBP alone is associated with increased disability, increased pain and poorer quality of life.8 9 LBLP is generally clinically diagnosed as sciatica (lumbar radicular) or referred pain (involving non-neural structures); sciatica is considered neuropathic in nature whereas referred pain is considered nociceptive.7 However, there is evidence to suggest the coexistence of both pain mechanisms in LBLP,10 and evidence for sciatica presenting without NP and referred pain presenting with NP.11
According to the National Institute for Health and Care Excellence recent guidelines, the management of NP differs significantly to the management of LBP without sciatica.12 Thus the importance of accurate diagnosis of NP in relation to LBLP ensures that appropriate management is provided. Identification of patients with NP in LBLP is essential as pharmaceutical intervention, if indicated, may improve patient outcomes.7 To date, there is no gold standard for diagnosing NP.13 Consequently, expert opinion consensus-derived lists and a variety of patient-reported outcome measures have been developed and used as methods to determine the presence of NP in research studies.7 14 However, there is no uniformity within the literature regarding the best clinical indicators to use to identify NP in LBLP. The most common clinical indicators utilised to identify NP in LBLP research include the PainDetect,15 The Leeds Assessment of Neuropathic Symptoms and Signs16 and Douleur Neuropathique 4 (DN4).17
A cross-sectional study by Smart et al 18 18 is the one study to identify clinical indicators predictive of the presence of peripheral NP in LBP (with or without leg pain), consisting of a cluster of two symptoms and one sign: ‘pain referred in a dermatomal or cutaneous distribution’, ‘history of nerve injury, pathology or mechanical compromise’ and ‘pain/symptom provocation with mechanical/movement tests (eg, active/passive, neurodynamic) that move/load/compress neural tissue’. However, there is evidence to refute aspects of this cluster. For instance experimental research suggests that remote immune-inflammatory mechanisms can contribute to the non-dermatomal/cutaneous innervation spread of symptoms in response to mild sciatic nerve compression in rat models.19 Furthermore, evidence of non-dermatomal/cutaneous innervation spread of symptoms in both distal and proximal entrapment neuropathies have been previously reported.20 21 This highlights the possibility that pain may not always follow a dermatomal/cutaneous innervation pattern. In addition, an increasing body of literature highlights the low diagnostic validity of neurodynamic testing which consequently questions its utility as an indicator to detect NP in LBLP.22–24 These findings highlight a need for further consideration of the clinical indicators used to identify NP in LBLP, as clinical uncertainty exists and there is no established gold standard.
Objective
To conduct a three round modified Delphi study to achieve expert consensus on a list of clinical indicators to identify NP in LBLP.
Methods
This study will be conducted using a modified Delphi method. The Delphi method is an iterative multistage process used to achieve expert consensus on a given subject.25 Expert consensus-derived criteria obtained through a Delphi method have been shown to be an effective tool in situations of uncertainty to inform clinical decision making.26 The Delphi method is a low-cost, simple procedure which can be used to gain information from a large population.27 The Delphi method is anonymous and participants do not directly interact, instead they receive feedback from their peers and this minimises the risk of few individuals’ opinions dominating.28 The anonymity has also been linked to higher response rates.29 This Delphi will be conducted online which helps to accelerate data collection. Furthermore, geographical location provides no barrier to participation in the study as it will be conducted electronically.28
This three-round modified Delphi method has been designed with guidance from the Conducting and Reporting Delphi Studies (CREDES) recommendations (online supplementary appendix 1). The CREDES recommendations are featured on the Equator network and are the only reporting guidance recommendations for Delphi studies.30 Three rounds of questionnaires will be administered anonymously through RedCap (https://www.project-redcap.org). All three rounds will use a 5-point Likert scale to evaluate the level of agreement. Between rounds one and three, the results will be analysed and questionnaires constructed for subsequent rounds. Clinical indicators that do not achieve consensus will be removed at each stage. Resultant data at the end of round three will be brought together to devise a consensus derived list of clinical indicators. The stages of the Delphi and development of the expert consensus derived list of clinical indicators is summarised in figure 1.
Figure 1.
Modified Delphi processes to construct list of clinical indicators. NP, neuropathic pain
bmjopen-2019-033547supp001.pdf (262.3KB, pdf)
Steering committee
The steering committee will be located at the University of Birmingham. The committee will comprise of the lead investigator, a MRes student JM, second investigator a PhD student TN and three senior academics based at the University of Birmingham AR, DF and NH. The three senior academics have considerable experience of using the Delphi technique as well as quantitative and qualitative research methods, and will ensure study quality at each stage.
Participants
Experts will be defined as individuals with a high level of knowledge within the area of NP in LBLP which will be confirmed against the eligibility criteria outlined below. Recruited experts will be expected to develop a consensus list which will reflect a high degree of content/face validity.31 Experts will include musculoskeletal/pain specialist physiotherapists, allied healthcare professionals, doctors and academics; all of whom work within a pain setting/have a special interest in NP. JM and TN will review the profiles of potential participants to decide eligibility based on the following criteria:
International research experts who have ≥2 peer-reviewed publications relating to NP in LBLP or a related topic32.
Working within a pain clinic/musculoskeletal outpatient department for ≥10 years.
Fulfilment of ≥1 criterion will be required for participant inclusion. There is no clear evidence to support the use of a predefined criteria in ensuring expert participant recruitment into a Delphi study, but it is consistent with the CREDES recommendations as well as criteria used in previous Delphi studies.32–34 Also, the use of eligibility criteria increases the rigour and transparency of the recruitment process and minimises the risk of disputes between recruiters.
The number of participants in Delphi studies varies considerably and there is no clear indication within the literature of what the ideal number is.27 The aim for this study will be to recruit a minimum of 30 participants.35 Furthermore, no upper limit will be set as the greater the number of participants the greater the data generation. The expectation is that no more than 100 participants will be recruited based on a previous similar Delphi study.14
Recruitment
A snowballing strategy will be used to identify potential participants. JM and TN will review authorships of published systematic reviews relating to the clinical indicators of NP and identify national and international profiles. Experts will be invited to participate in the study and also requested to suggest any peers who fit the eligibility criteria.34 Call for expressions of interest will be posted on social media for participants to be nominated or to self-nominate themselves. Social media platforms will be used due to its high-quality healthcare, research and academic communities.36
Contacted individuals will receive an email with four attached documents. A Participant Information Sheet will describe the aims and objectives of the Delphi, justification for the study, eligibility requirements for experts, stages involved in the Delphi process, timeframe for each stage, assurance on anonymity and the withdrawal process. The second document will be a consent form, the third a conflict of interest form and the final document will be a participant information form for participants to detail: age, gender, country of origin, country of current employment, highest qualification, occupation, professional background/credentials and working period in NP in LBLP/related field.32 Return of the consent form and conflict of interest and participant information form signifies agreement to participate. Recruiters will collate/review returned information and email individuals to confirm participation in the study.
Ethical considerations/quality assurance
Eligible participants will be required to return a consent form and conflict of interest form. Participants will be advised at the start of the study of the withdrawal process should they need it. ID codes will be used instead of personal information to ensure participants remain anonymous throughout the process.
The data obtained in the study will be stored in a password encrypted electronic device which will only be accessible to the researchers involved in the study. Anonymity will be ensured throughout the study. Quality assurance will be achieved through three senior academics (AR, NH, DF) who are part of the steering committee. They share vast research methods experience, including the use of the Delphi method, therefore the use of their expertise in guiding/providing supervision to the lead investigator (JM) will ensure quality is maintained.
Procedure
Figure 2 details the procedure and timeline for the study. Prior to the start of the study, a pre-notification period of 6 weeks will be allocated to recruit participants.
Figure 2.

Procedure and timelines for participants in Delphi study.
Round 1
In the first-round participants will be invited to provide their level of agreement with the list of clinical indicators suggested by Smart et al 14 14 to identify peripheral NP (online supplementary appendix 2), for the purpose of this study in relation to identifying NP in LBLP. Level of agreement will be sought using a 5-point Likert scale (5=Strongly agree, 4=Agree, 3=No opinion, 2=Disagree, 1=Strongly disagree). A 5-point Likert scale will be used as any number less than 5 in a Likert scale has been demonstrated to compromise reliability and validity.37 Second, through open questions, participants will be invited to identify any missing clinical indicators. Participants will be invited to explain their reasons for their ratings and missing clinical indicators in a free text box. The use of open-ended questions allows participant freedom to volunteer any clinical indicators considered relevant, and for a richness in the qualitative data collected.26
bmjopen-2019-033547supp002.pdf (256.6KB, pdf)
Participants will be given 1 month to complete round 1. Reminder emails will be sent to non-responders at weeks 1 and 3, and in this email the withdrawal process will be highlighted if participants are unable to continue with the study.38 Only those participants who complete round 1 will be invited to round 2, in line with previous Delphi studies.34 39
Round 2
Participants will be issued with the second questionnaire constructed from the results of round 1. Participants will again be invited to provide level of agreement/disagreement using the Likert scale for clinical indicators. Participants will be invited to identify any missing clinical indicators in the format of an open-ended question. Furthermore, participants will also be given the opportunity to explain the reason for their ratings and addition of any further clinical indicators.
Participants will be given 1 month to complete round 2. Reminder emails will be sent to non-responders at weeks 1 and 3.38 Only those participants who complete round 2 will be invited to round 3.
Round 3
The questionnaire for round 3 will be constructed using the clinical indicators which achieved consensus and any additional information gained from the open-ended question from round 2. In round 3, participants will be issued with the third questionnaire constructed from the results of round 2. Participants will be asked to re-rate their level of agreement with the clinical indicators, presented in graphical format as per round 2. Participants will also be invited to rank each clinical indicator for their importance, from highest to lowest. Feedback on round 3 will not be provided to the participants. Response data will then be re-analysed for levels of agreement and consensus.40 The steering committee will then use the results to identify a consensus derived list of clinical indicators.
Data collection
Response data will be collected and quantitative data will be statistically analysed using SPSS V.25.0 (IBM Corp.) and qualitative data from the open-ended questions will be inputted into summary tables in a word document for content analysis.41 Each round will take 4 weeks with 2 weeks after each round allocated for data analysis and formulation of the subsequent questionnaire.38
Data analysis
Round 1
Level of agreement with Smart et al’s14 list will be measured using the Likert scale. Descriptive statistics including median, IQR and percentage of agreement will be gathered using data from each participant.32
Responses to open questions will be reviewed by JM and TN and content analysis will be carried out.41 Results of content analysis will be brought back to the steering group to ensure quality is maintained, also any disagreements in content analysis will be presented to the steering group. Content analysis is typically used to identify major themes in response to qualitative data.14 The questionnaire for round 2 will be constructed using the clinical indicators which achieved consensus from Smart et al’s14 list as well as using the missing indicators identified by participants in the open-ended aspect of the questionnaire.
Round 1 criteria for consensus:
Median value of participants' Likert scale data ≥3.
Percentage of agreement ≥50%.32
Round 2
Results from round 1 will be presented in a graphical format, with each criterion presented as a bar chart. Each bar chart will show the number and percentage of respondents indicating level of agreement/disagreement. Furthermore, a narrative summary will be presented in round 2 of the findings from round 1’s open ended question section. This will provide participants with feedback from round 1.
The Likert scale will be used to assess agreement/disagreement with round 2 clinical indicators. Further content analysis will be conducted from the open-ended free text boxes. The round 3 questionnaire will be constructed using the same method as round 1.
Round 2 criteria for consensus:
Median value of participants Likert scale data ≥3.5.
IQR value of participants' Likert scale data ≤2.
Percentage of agreement ≥60%.32
Round 3
As per round 2, graphical depiction of round 2 findings and narrative summaries will be presented to participants. Participants will use the Likert scale again to suggest agreement/disagreement with round 3 clinical indicators. Participants will also rank each clinical indicator from highest to lowest with respect to importance.
Round 3 criteria for consensus:
Median value of participants' Likert scale data ≥4.
IQR value of participants Likert scale data ≤3.
Percentage of agreement ≥70%.32
Agreement between participants will also be evaluated across all clinical indicators using Kendall’s W coefficient of concordance,29 statistical significance will be set at p<0.05.
Data management
All participant information and feedback will be securely stored on a password-protected computer throughout the duration of the study. Only members of the research team will be able to access the information. After completion of the Delphi study, all data will be kept securely for 10 years in the School of Sport, Exercise and Rehabilitation Sciences, University of Birmingham before being securely destroyed, in accordance with University guidelines.
Patient and public involvement
Information from patients and the public has informed the conception and requirement for this Delphi study as part of an existing programme of research that is centred on lumbar spinal surgery for LBLP and patient outcome.
Dissemination plan
This protocol will be submitted to an open access peer-reviewed journal. On completion the study findings will be disseminated in a peer-reviewed journal and presented at relevant conferences.
Discussion
The results from this study will assist both clinicians and researchers in establishing a clear reference standard for diagnosing NP in LBLP. By establishing a uniformly recognised reference standard, timely and accurate diagnosis to inform precision management of patients with NP in LBLP will be possible. In turn, timely and accurate diagnosis will enable an improved prognosis and reduce the risk of patients developing chronicity. It will be possible to use the list of clinical indicators derived from this study in combination with guidelines12 to further inform clinicians regarding the identification of NP in LBLP, thus affording greater confidence in their clinical judgement. The results from this study will provide the first list of clinical indicators specific to identifying NP in LBLP, and therefore will serve a need both clinically and within the contemporary literature to inform further research.
Conclusion
There is uncertainty within the literature when considering the clinical indicators associated with identifying NP in LBLP. In order to ascertain a consensus derived set of clinical indicators, a modified Delphi study has been designed. The clinical implications of this study will aid clinicians in identifying a neuropathic component to LBLP through a list of clinical indicators. Assisting in accurate diagnosis will ensure that appropriate treatment is carried out.
Supplementary Material
Footnotes
Twitter: @Deb_Falla, @HeneghanNicola, @abrushton
Contributors: All authors devised the focus of this Delphi study. JM is a MRes student, AR is the lead supervisor, DF and NH are co-supervisors and TN is a co-researcher. JM drafted the initial protocol manuscript with lead and co-supervisors providing guidance on methodological decisions and proposed analyses. All authors have contributed subject-specific expertise. JM and TN will recruit participants into the study. All authors will contribute to data interpretation, conclusions and dissemination. All authors have read, contributed to and agreed to the final manuscript. AR is the guarantor of the study.
Funding: It will be conducted as part of a MRes research project though the University of Birmingham.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Treede R-D, Jensen TS, Campbell JN, et al. Neuropathic pain: redefinition and a grading system for clinical and research purposes. Neurology 2008;70:1630–5. 10.1212/01.wnl.0000282763.29778.59 [DOI] [PubMed] [Google Scholar]
- 2. Baron R, Binder A, Attal N, et al. Neuropathic low back pain in clinical practice. Eur J Pain 2016;20:861–73. 10.1002/ejp.838 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. van Hecke O, Austin SK, Khan RA, et al. Neuropathic pain in the general population: a systematic review of epidemiological studies. Pain 2014;155:654–62. 10.1016/j.pain.2013.11.013 [DOI] [PubMed] [Google Scholar]
- 4. Liedgens H, Obradovic M, De Courcy J, et al. A burden of illness study for neuropathic pain in Europe. Clinicoecon Outcomes Res 2016;8:113–26. 10.2147/CEOR.S81396 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Torrance N, Ferguson JA, Afolabi E, et al. Neuropathic pain in the community: more under-treated than refractory? Pain 2013;154:690–9. 10.1016/j.pain.2012.12.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Berger A, Sadosky A, Dukes E, et al. Clinical characteristics and patterns of healthcare utilization in patients with painful neuropathic disorders in UK general practice: a retrospective cohort study. BMC Neurol 2012;12:8 10.1186/1471-2377-12-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Harrisson SA, Stynes S, Dunn KM, et al. Neuropathic pain in low Back-Related leg pain patients: what is the evidence of prevalence, characteristics, and prognosis in primary care? A systematic review of the literature. J Pain 2017;18:1295–312. 10.1016/j.jpain.2017.04.012 [DOI] [PubMed] [Google Scholar]
- 8. Konstantinou K, Hider L, Jordan L, et al. The impact of low back-related leg pain on outcomes as compared with low back pain alone: a systematic review of the literature. Clin J Pain 2013;29:644–54. [DOI] [PubMed] [Google Scholar]
- 9. Hider SL, Whitehurst DGT, Thomas E, et al. Pain location matters: the impact of leg pain on health care use, work disability and quality of life in patients with low back pain. Eur Spine J 2015;24:444–51. 10.1007/s00586-014-3355-2 [DOI] [PubMed] [Google Scholar]
- 10. Freynhagen R, Rolke R, Baron R, et al. Pseudoradicular and radicular low-back pain – a disease continuum rather than different entities? answers from quantitative sensory testing. Pain 2008;135:65–74. 10.1016/j.pain.2007.05.004 [DOI] [PubMed] [Google Scholar]
- 11. Mahn F, Hüllemann P, Gockel U, et al. Sensory symptom profiles and co-morbidities in painful radiculopathy. PLoS One 2011;6:e18018 10.1371/journal.pone.0018018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. NICE Low back pain and sciatica in over 16S: assessment and management (NICE guideline 33). National Institute for health and care excellence. available at, 2016. Available: https://www.nice.org.uk/guidance/ng59 [Accessed 10th May 2019]. [PubMed]
- 13. Smith BH, Torrance N, Bennett MI, et al. Health and quality of life associated with chronic pain of predominantly neuropathic origin in the community. Clin J Pain 2007;23:143–9. 10.1097/01.ajp.0000210956.31997.89 [DOI] [PubMed] [Google Scholar]
- 14. Smart KM, Blake C, Staines A, et al. Clinical indicators of ‘nociceptive’, ‘peripheral neuropathic’ and ‘central’ mechanisms of musculoskeletal pain. A Delphi survey of expert clinicians. Man Ther 2010;15:80–7. 10.1016/j.math.2009.07.005 [DOI] [PubMed] [Google Scholar]
- 15. Beith ID, Kemp A, Kenyon J, et al. Identifying neuropathic back and leg pain: a cross-sectional study. Pain 2011;152:1511–6. 10.1016/j.pain.2011.02.033 [DOI] [PubMed] [Google Scholar]
- 16. Walsh J, Hall T. Classification of low back-related leg pain: do subgroups differ in disability and psychosocial factors? J Man Manip Ther 2009;17:118–23. 10.1179/106698109790824703 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Ouédraogo D-D, Nonguierma V, Napon C, et al. Prevalence of neuropathic pain among black African patients suffering from common low back pain. Rheumatol Int 2012;32:2149–53. 10.1007/s00296-011-1945-4 [DOI] [PubMed] [Google Scholar]
- 18. Smart KM, Blake C, Staines A, et al. Mechanisms-based classifications of musculoskeletal pain: Part 2 of 3: symptoms and signs of peripheral neuropathic pain in patients with low back (±leg) pain. Man Ther 2012;17:345–51. 10.1016/j.math.2012.03.003 [DOI] [PubMed] [Google Scholar]
- 19. Schmid AB, Coppieters MW, Ruitenberg MJ, et al. Local and remote immune-mediated inflammation after mild peripheral nerve compression in rats. J Neuropathol Exp Neurol 2013;72:662–80. 10.1097/NEN.0b013e318298de5b [DOI] [PubMed] [Google Scholar]
- 20. Tampin B, Slater H, Hall T, et al. Quantitative sensory testing somatosensory profiles in patients with cervical radiculopathy are distinct from those in patients with nonspecific neck–arm pain. Pain 2012;153:2403–14. 10.1016/j.pain.2012.08.007 [DOI] [PubMed] [Google Scholar]
- 21. Nora DB, Becker J, Ehlers JA, et al. Clinical features of 1039 patients with neurophysiological diagnosis of carpal tunnel syndrome. Clin Neurol Neurosurg 2004;107:64–9. 10.1016/j.clineuro.2004.08.003 [DOI] [PubMed] [Google Scholar]
- 22. Nee RJ, Jull GA, Vicenzino B, et al. The validity of upper-limb neurodynamic tests for detecting peripheral neuropathic pain. J Orthop Sports Phys Ther 2012;42:413–24. 10.2519/jospt.2012.3988 [DOI] [PubMed] [Google Scholar]
- 23. Baselgia LT, Bennett DL, Silbiger RM, et al. Negative neurodynamic tests do not exclude neural dysfunction in patients with entrapment neuropathies. Arch Phys Med Rehabil 2017;98:480–6. 10.1016/j.apmr.2016.06.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Schmid AB, Hailey L, Tampin B. Entrapment neuropathies: challenging common beliefs with novel evidence. J Orthop Sports Phys Ther 2018;48:58–62. 10.2519/jospt.2018.0603 [DOI] [PubMed] [Google Scholar]
- 25. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008–15. [PubMed] [Google Scholar]
- 26. Powell C. The Delphi technique: myths and realities. J Adv Nurs 2003;41:376–82. 10.1046/j.1365-2648.2003.02537.x [DOI] [PubMed] [Google Scholar]
- 27. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical Guideline development. Health Technol Assess 1998;2:87–8. 10.3310/hta2030 [DOI] [PubMed] [Google Scholar]
- 28. Merlin JS, Young SR, Azari S, et al. Management of problematic behaviours among individuals on long-term opioid therapy: protocol for a Delphi study. BMJ Open 2016;6:e011619 10.1136/bmjopen-2016-011619 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Heiko A. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change 2012;79:1525–36. [Google Scholar]
- 30. Jünger S, Payne SA, Brine J, et al. Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med 2017;31:684–706. 10.1177/0269216317690685 [DOI] [PubMed] [Google Scholar]
- 31. McCarthy CJ, Rushton A, Billis V, et al. Development of a clinical examination in non-specific low back pain: a Delphi technique. J Rehabil Med 2006;38:263–7. 10.1080/16501970600632768 [DOI] [PubMed] [Google Scholar]
- 32. Wiangkham T, Duda J, Haque MS, et al. Development of an active behavioural physiotherapy intervention (ABPI) for acute whiplash-associated disorder (WAD) II management: a modified Delphi study. BMJ Open 2016;6:e011764 10.1136/bmjopen-2016-011764 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Blaschke S, O'Callaghan CC, Schofield P. Identifying opportunities for nature engagement in cancer care practice and design: protocol for four-round modified electronic Delphi. BMJ Open 2017;7:1–9. 10.1136/bmjopen-2016-013527 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Slade SC, Dionne CE, Underwood M, et al. Standardised method for reporting exercise programmes: protocol for a modified Delphi study. BMJ Open 2014;4:e006682 10.1136/bmjopen-2014-006682 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Sekayi D, Kennedy A. Qualitative Delphi method: a four round process with a worked example. The Qualitative Report 2017;22:2755–63. [Google Scholar]
- 36. Pezaro S, Clyne W. Achieving consensus in the development of an online intervention designed to effectively support midwives in work-related psychological distress: protocol for a Delphi study. JMIR Res Protoc 2015;4:107 10.2196/resprot.4766 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Preston CC, Colman AM. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychologica 2000;104:1–15. 10.1016/S0001-6918(99)00050-5 [DOI] [PubMed] [Google Scholar]
- 38. Zambaldi M, Beasley I, Rushton A. Return to play criteria after hamstring muscle injury in professional football: a Delphi consensus study. Br J Sports Med 2017;51:1221–6. 10.1136/bjsports-2016-097131 [DOI] [PubMed] [Google Scholar]
- 39. Taylor RM, Feltbower RG, Aslam N, et al. Modified international e-Delphi survey to define healthcare professional competencies for working with teenagers and young adults with cancer. BMJ Open 2016;6:e01136 10.1136/bmjopen-2016-011361 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Jones J, Hunter D. Qualitative research: consensus methods for medical and health services research. BMJ 1995;311:376–80. 10.1136/bmj.311.7001.376 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Patton MQ. Qualitative research methods. Thousand Oaks: Sage Publications, 2002. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2019-033547supp001.pdf (262.3KB, pdf)
bmjopen-2019-033547supp002.pdf (256.6KB, pdf)

