Abstract
Objective
To identify valid and feasible quality indicators for the primary care of osteoarthritis (OA).
Design
Systematic review and narrative synthesis.
Data sources
Electronic reference databases (MEDLINE, EMBASE, CINAHL, HMIC, PsychINFO), quality indicator repositories, subject experts.
Eligibility criteria
Eligible articles referred to adults with OA, focused on development or implementation of quality indicators, and relevant to UK primary care. An English language restriction was used. The date range for the search was January 2000 to August 2013. The majority of OA management guidance has been published within this time frame.
Data extraction
Relevant studies were quality assessed using previous quality indicator methodology. Two reviewers independently extracted data. Articles were assessed through the Outcome Measures in Rheumatology filter; indicators were mapped to management guidance for OA in adults. A narrative synthesis was used to combine the indicators within themes.
Results
10 853 articles were identified from the search; 32 were included in the review. Fifteen indicators were considered valid and feasible for implementation in primary care; these related to assessment non-pharmacological and pharmacological management. Another 10 indicators were considered less feasible, in various aspects of assessment and management. A small number of recommendations had no published corresponding quality indicator, such as use of topical non-steroidal anti-inflammatory drugs. No negative (‘do not do’) indicators were identified.
Conclusions and implications of key findings
There are well-developed, feasible indicators of quality of care for OA which could be implemented in primary care. Their use would assist the audit and quality improvement for this common and frequently disabling condition.
Keywords: Osteoarthritis, Quality Indicators, Analgesics, NSAIDs
Background
Osteoarthritis (OA) is a common reason for consultation with a general practitioner (GP): around 4% of the population aged 45 years and over will consult a GP in a year with a diagnosis of OA.1 One working definition of OA is “persistent joint pain that is worse with use [in people] age 45 years old and over [who have] morning stiffness lasting no more than half an hour” and does not require radiography for diagnosis.2 There are evidence-based interventions to reduce pain and disability in adults with OA. Guidance on the care and management of OA has been produced by the American College of Rheumatology, the European League Against Rheumatism, the Osteoarthritis Research Society International, and the National Institute for Health and Care Excellence (NICE).3–8 Although management may vary by the site of OA, core aspects of primary care management are generally common across all sites.4–6 8 If these interventions were routinely implemented by GPs, there would be a significant impact on population levels of pain and disability attributable to OA.9 However, there is evidence that such implementation is not occurring.10–15
Routine audit and feedback on provided care is needed to improve the quality of that care. Quality indicators (hereafter ‘indicators’) are one suitable tool.16 Such indicators are defined as a “measurable [element] of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided”.17 Although reviews by Hochberg18 and Strömbeck et al19 identified indicators for measuring quality of care for OA, which show promise for use in primary care, there has been no systematic review and synthesis of the development and implementation literature to identify the most promising and feasible set of primary care OA indicators. Hunter et al20 argue cogently for ‘further systematic development, implementation, and audit of quality measures.’ The objective of this systematic review was to identify existing indicators of core treatment for OA feasible for use in primary care medical records and for routine audit purposes through electronic data retrieval.
Methods
We used the methodology for systematic reviews set out by the Centre for Reviews and Dissemination.21
Review protocol
Available on request from the corresponding author.
Search strategy
A search strategy was developed to identify articles concerning the development, testing or implementation of indicators of the quality of care for OA applicable to adults in a primary medical care setting.
The systematic search strategy was customised for use in databases searchable through the UK National Health Service (NHS) Evidence portal (CINAHL, EMBASE, HMIC, MEDLINE and PsychINFO). A range of OA terms were combined with indicator terms. An English language restriction was used. The date range for the search was January 2000 to August 2013. Further studies were identified from other known repositories including the Agency for Healthcare Research and Quality.22
The search strategy for use in MEDLINE via NHS Evidence is shown in table 1.
Table 1.
MEDLINE search strategy
1 | MEDLINE | ((qualit* ADJ3 (outcome* OR indicat*))).ti,ab |
2 | MEDLINE | QUALITY OF HEALTH CARE/ |
3 | MEDLINE | QUALITY ASSURANCE, HEALTH CARE/ |
4 | MEDLINE | BENCHMARKING/ |
5 | MEDLINE | CLINICAL AUDIT/ |
6 | MEDLINE | MEDICAL AUDIT/ |
7 | MEDLINE | FACILITY REGULATION AND CONTROL/ |
8 | MEDLINE | GUIDELINES AS TOPIC/ |
9 | MEDLINE | PRACTICE GUIDELINES AS TOPIC/ |
10 | MEDLINE | TOTAL QUALITY MANAGEMENT/ |
11 | MEDLINE | exp UTILIZATION REVIEW/ |
12 | MEDLINE | exp “OUTCOME AND PROCESS ASSESSMENT (HEALTH CARE)”/ |
13 | MEDLINE | QUALITY INDICATORS, HEALTH CARE/ |
14 | MEDLINE | 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 |
15 | MEDLINE | osteoarthr*.ti,ab |
16 | MEDLINE | exp OSTEOARTHRITIS/ |
17 | MEDLINE | 15 OR 16 |
18 | MEDLINE | 14 AND 17 |
19 | MEDLINE | 18 [Limit to: Publication Year 2000-Current and English Language] |
Selection of eligible articles
The titles identified were entered in a bibliographical database and duplicates removed. Titles were assessed for relevance by a single reviewer (JJE). The resulting abstracts were evaluated independently by two reviewers (JJE and MK). All those considered relevant by one or both reviewers were entered into the next round. The full texts of the resulting articles were obtained. These were subject to dual independent review of their relevance (JJE plus MK or KSD) and, if there was disagreement on inclusion, by a third reviewer (KSD or MK). This process yielded a final set of articles for the data abstraction round.
Method of data extraction
Data extraction forms were designed using the assessment criteria below. The extraction forms were piloted and refined by three reviewers. Data were independently extracted by two reviewers (JJE plus MK or KSD). Differences in extraction were resolved by discussion or by a third independent data extraction.
Assessment of indicators
The indicators were assessed for quality against criteria used previously, and based on the Outcome Measures in Rheumatology filter (truth, discrimination, feasibility).23–28 The assessment criteria are shown in the online supplementary text S1. Indicators were considered at the level of their development group (for the evidence synthesis, consensus exercise and testing) and at the level of the individual indicator (for discrimination and feasibility).
Narrative synthesis
The clinical reviewers (two experienced GPs—JJE and MK—and an academic physiotherapist—KSD) together drafted a narrative synthesis to collate the individual indicators, which was then discussed and revised among all the authors. The indicators were mapped to OA guidance.3–8 Indicator themes developed from the best evidence and consensus method, and rated as feasible for UK primary care, were transformed into a format suitable for implementation. This included a defined numerator (the number of patients receiving a particular element of care) and denominator (those eligible for that element).
Results
Selection of articles
Ten thousand eight hundred and fifty-two unique articles were identified. The final inclusion set numbered 32. There were 10 groups of indicators in 14 development articles, and 18 implementation articles.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart setting out the review process can be found in the online supplementary figure S2. Excluded studies are listed in the online supplementary table S3.
Assessment of quality
The 10 groups of studies in which indicators for OA care had been developed are listed in table 2. The following aspects of quality assessment were common to all studies and are not included in the table.
Table 2.
Indicator studies
Indicator set Author and date |
Truth | Target population | Proposed method of measurement | Testing or implementation | Reliability | Feasibility | |
---|---|---|---|---|---|---|---|
Evidence synthesis | Consensus method | ||||||
RAND Quality of Care Assessment Tools (RAND QA) Moore (2000)29 |
Literature review, not specified to be systematic | RAND Appropriateness Method | Not specified | Medical record review | McGlynn et al30 Asch et al31 |
Tested in McGlynn et al30 and Asch et al31 —in a 4% sample reabstraction reliability was substantial at 3 levels: presence of a condition (к=0.83), indicator eligibility (к=0.76) and indicator scoring к=0.80) | McGlynn et al30 use a national sample of US citizens in metropolitan areas, using a telephone interview to collect data with subsequent analysis of medical records where consent was given. Asch et al31 use the same data as a comparator for data collected from a random sample of veterans’ health affairs clients and their records |
ACOVE -1 MacLean (2001)32 33 |
Systematic review supporting indicators produced by a content expert working with a project member knowledgeable about systematic reviews and quality indicator development | Modified RAND/UCLA Appropriateness Method | ‘Vulnerable elders’—persons ≥65 years who are at increased risk for death or functional decline | Not specified | Wenger et al34 Chodosh et al35 Higashi et al36 37 Ganz et al38 MacLean et al39 Østerås et al40 |
Tested in Wenger et al34—10% sample reabstraction: overall error rate was 1.6%; also in Chodosh et al35—‘Inter-rater reliability of chart abstraction for eligibility and scoring of indicators was 95%.’; Higashi et al36 37—10% reabstraction sample showed 97% identical eligibility and 95% identical eligibility and quality score; MacLean et al39—10% reabstraction sample with к=0.85 (93% agreement); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Wenger et al34 implemented the indicators in community dwelling VEs in the USA—medical record abstraction by trained nurses supplemented for some indicators by telephone interview. Chodosh et al,35 Higashi et al36
37 and MacLean et al39 used the same population and methods. Ganz et al used a similar population and methodology at a different time point38 Østerås et al40 implemented some of these indicators in a patient self-report format |
ACOVE-1 adapted for the ELSA Steel et al (2004)41 |
Transposition of previous ACOVE work (referenced). 26 new indicators for the set were suggested by the panel | Modified RAND/UCLA Appropriateness Method | Older patients in the UK ≥65 years) | Interviews for the ELSA | Steel et al,12 14 Broadbent et al,13 Østerås et al40 | Tested in Steel et al12—(к=0.8, 95% CI=0.7 to 0.9); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Broadbent et al13 and Steel et al12 separately implemented indicators in UK general practice, using medical record review (computerised and paper notes). Østerås et al40 implemented some of these indicators in a patient self-report format |
ACOVE-1 adapted for NH implementation (ACOVE/NH) Saliba et al (2005)42 |
Previous referenced (ACOVE) work, plus expert opinion (for modification) and additional indicator development, methodology not specified in detail | Modified Delphi; subsequent overview by ACOVE clinical committee | Long-stay NH residents ≥65 years Exclusions for advanced dementia or poor prognosis | Not specified | Cadogan et al43 Zingmond et al44 |
Tested in Cadogan et al43—к=0.65–1.00 and percentage agreement 80–100 where к could not be calculated (numbers too low) | Cadogan et al43 implemented indicators in 30 nursing homes in California using medical record review. Zingmond et al44 implemented using Medicare and Medicaid eligibility and claims data and a nursing home minimum dataset |
ACOVE-1 adapted for the HPCQI Smith et al (2007)45 |
Based on ACOVE indicators, plus some additional (non-OA) indicators. ACOVE work referenced; additional expert opinion | Modified Delphi techniques | Patients ≥60 years who are homebound | Not specified | No published examples of testing identified | No reliability testing identified | No feasibility testing identified |
ACOVE-3 ACOVE investigators (2007)46–48 |
A systematic review supporting potential indicators produced by a content expert working with a project member knowledgeable about systematic reviews and indicator development | Modified version of the RAND/UCLA Appropriateness Method | Community-dwelling individuals aged ≥65 years who are at greater risk of death or functional decline over a 2-year period | Medical records and/or administrative data, patient or proxy interview | Østerås et al40 | Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Østerås et al40 implemented some of these indicators in a modified patient self-report format |
QIGP Underwood (2002)49 |
Various sources used (Cochrane, DARE, Medline) but not clear how the evidence was assembled. Cites meta-analyses, systematic reviews, randomised controlled trials | Not stated | Not specified | Not specified | Kirk et al50 Steel et al12 Broadbent et al13 Østerås et al40 |
Tested for non-OA indicators in Kirk et al50—as OA was not included in this exercise, it is not known what degree of reliability exists for these indicators; Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Kirk et al50 implemented in 16 UK general practices in two areas using data from electronic and paper records. Steel et al12 and Broadbent et al13 separately implemented the NSAID indicator in UK general practice, using medical record review (computerised and paper notes). Østerås et al40 implemented some of these indicators in a modified patient self-report format |
Arthritis Foundation Arthritis Foundation 200451 52 |
Comprehensive literature search and expert opinion | Modified RAND/UCLA Appropriateness Method | Patients with OA | Not specified | Li et al53 | No reliability testing identified | Li et al53 used a postal survey in Canada (sampling frame from an administrative database in British Columbia) to assess non-pharmacological indicators |
PCPI (2006)54 | PCPI website refers to a methodology committee but no specific information in the indicator set to identify how it was developed | All patients aged ≥21 years with a diagnosis of OA | Medical record data extraction (detailed numerator and denominator information provided) | No published examples of testing identified | No reliability testing identified | No feasibility testing identified | |
EUMUSC.net (2012)55 | Developed from the EUMUSC.net standards of care for OA and refined by researchers and patient representatives | All adult patients with OA of hand, hip or knee | Varies. Examples include patient record or survey. Numerator and denominator clearly identified |
No published examples of testing identified | No reliability testing identified | No feasibility testing identified |
ACOVE, Assessing Care of Vulnerable Elders; DARE, Database of Abstracts of Reviews of Effects; ELSA, English Longitudinal Study of Ageing; EUMUSC.net, European Musculoskeletal Conditions Surveillance and Information Network; HPCQI, Home-based Primary Care Quality Initiative; NH, nursing home; NSAIDs, non-steroidal anti-inflammatories; OA, osteoarthritis, PCPI, Physician Consortium for Performance Improvement; QIGP, Quality Indicators for General Practice; UCLA, University of California, Los Angeles; VE, vulnerable elder.
Although not every study explicitly declared there to be no conflict of interest, the reviewers considered that no significant resulting bias of the results was likely.
No studies had an identified method of updating the indicators in light of new evidence.
External validity and sensitivity to change had not been demonstrated in any of the indicator development, testing or implementation studies.
Reproducibility, at the level of the individual indicator, is shown in table 3.
Table 3.
Narrative synthesis of exemplar indicators and their feasibility for use in primary care
Overarching theme (source) | ‘Exemplar’ indicator | Reproducibility (other sources of similar indicators) | Implementation references and comment on feasibility |
---|---|---|---|
Holistic Assessment: Pain (EULAR (all sites), NICE) | IF a VE has symptomatic OA of the knee or hip, THEN pain should be assessed when new to a primary care or musculoskeletal disease practice and annually… (ACOVE-3)46–48 | RAND QA,29 ACOVE-1,32 33 and as adapted (ELSA,41 HPCQI45), Arthritis Foundation,51 52 PCPI,54 EUMUSC.net55 |
12
13
30
31
34
35
38
40 Requires change in routine coding to improve capture of this information |
Holistic Assessment: Function (ACR (hand), EULAR (all sites), NICE ) | IF a VE has symptomatic OA of the knee or hip, THEN functional status should be assessed when new to a primary care or musculoskeletal disease practice and annually…(ACOVE-3)46–48 | RAND QA,29 ACOVE-1,32 33 and as adapted (ELSA,41 HPCQI45), Arthritis Foundation,51 52 PCPI54 |
12
13
30
31
35
38
40 Requires change in routine coding to improve capture of this information |
Education (EULAR (all sites), NICE, OARSI) | IF a patient has had a diagnosis of symptomatic OA of the knee or hip for >3 months, THEN education about the natural history, treatment, and self-management of OA should have been given or recommended at least once…(Arthritis Foundation)51 52 | ACOVE-1 (2 variations—new and pre-existing disease),32 33 and as adapted (ELSA41), EUMUSC.net55 |
12–14
34
38
40 Requires change in routine coding to improve capture of this information |
Exercise 1 and 2 (ACR (hip, knee), EULAR (all sites), NICE, OARSI | IF an ambulatory VE has symptomatic OA of the knee or hip for longer than 3 months and is able to exercise, THEN a directed or supervised muscle strengthening or aerobic exercise program should be recommended and activity reviewed annually…(ACOVE-3)46–48 |
Initial recommendation RAND QA,29 ACOVE-1 (indicators for new and pre-existing disease),32 33 and as adapted (ELSA,41 ACOVE/NH,42 HPCQI45), Arthritis Foundation,51 52 PCPI,54 EUMUSC.net55 Annual review RAND QA,29 ACOVE-132 33 |
Initial recommendation 14 30 31 34 37 38 40 43 53 Annual review 53 Requires change in routine coding to improve capture of this information |
Weight loss 1 (ACR (hip, knee), NICE, OARSI | IF a VE is obese (body mass index (BMI) ≥30 kg/m2), THEN he or she should be advised annually to lose weight… (ACOVE-3)46–48 | Arthritis Foundation51 52 | No implementation studies identified for this indicator. Should be captured from existing weight and health promotion records |
Weight loss 2 (ACR (hip, knee), NICE, OARSI | IF a patient has symptomatic OA of the knee or hip and is overweight (as defined by body mass index of ≥27 kg/m2), THEN the patient should be advised to lose weight at least annually AND the benefit of weight loss on the symptoms of OA should be explained to the patient…(Arthritis Foundation)51 52 | EUMUSC.net55 |
40
53 Consider a lower BMI threshold of 25 kg/m2 for consistency with the usual definition of ‘overweight’. Should be captured from existing weight and health promotion records. |
Aids and devices 1 (ACR (hip, knee), EULAR (hip, knee), NICE, OARSI) | IF a VE has symptomatic OA of the hip or knee and has difficulty walking that makes ADL difficult for longer than 3 months, THEN the need for ambulatory assistive devices should be assessed…(ACOVE-3)46–48 | Arthritis Foundation,51 52 EUMUSC.net55 |
40
53 Requires change in routine coding to improve capture of this information |
Aids and devices 2 (ACR (hand), NICE) | IF a VE has symptomatic OA and has difficulty with non-ambulatory ADL, THEN the need for ADL assistive devices should be assessed… (ACOVE-3)46–48 | Arthritis Foundation,51 52 EUMUSC.net55 |
40
53 Requires change in routine coding to improve capture of this information |
Paracetamol 1 (ACR (hip, knee), EULAR (all sites), NICE, OARSI) | IF a VE is started on pharmacological therapy to treat OA, THEN acetaminophen should be tried first… (ACOVE-3)46–48 | RAND QA,29 ACOVE-1,32 33 and as adapted (ELSA,41 ACOVE/NH,42 HPCQI45), QIGP,49 Arthritis Foundation,51 52 |
12–14
30
31
34
36–38
40
43
50 Requires change in routine coding to capture over-the-counter drug use |
Paracetamol 2 (ACR (hip, knee), EULAR (all sites), NICE, OARSI) | IF oral pharmacological therapy for OA is changed from acetaminophen to a different oral agent, THEN there should be evidence that the patient has had a trial of maximum dose acetaminophen (suitable for age/comorbidities)….(Arthritis Foundation)51 52 | ACOVE-1,32 33 and as adapted (ELSA,41 ACOVE/NH,42 HPCQI45) |
12
13
34
36
43 Requires change in routine coding to capture over-the-counter drug use |
Oral NSAIDs 1 (all guidance) | If NSAIDs are considered, ibuprofen should be considered for first-line treatment unless contraindicated or intolerant.* (QIGP)49 | Modifications exist in implementation studies: Steel et al,12 Broadbent et al13 to include use of COX-2 selective drugs |
12
13
50 Requires change in routine coding to capture over-the-counter drug use. |
Oral NSAIDs 2 (all guidance) | Percentage of patients aged 21 years and older with a diagnosis of OA on prescribed or OTC NSAIDs who were assessed for GI and renal risk factors. (PCPI)54 | Two indicators from ACOVE-3 refer to risks from NSAIDs and aspirin to be ‘discussed and documented’,46–48 EUMUSC.net55 |
12
13 Requires change in routine coding to capture over-the-counter drug use |
Gastroprotection (EULAR (all sites), NICE, OARSI) | IF a VE with a risk factor for GI bleeding (aged ≥75, peptic ulcer disease, history of GI bleeding, warfarin use, chronic glucocorticoid use) is treated with a non-selective NSAID, THEN he or she should be treated concomitantly with misoprostol or a PPI. (ACOVE-3)46–48 | ACOVE-1,32 33 ACOVE-3 46–48 (NSAIDs, and aspirin), QIGP,49 PCPI54 |
34–39
44
50 Should be captured from existing electronic prescribing records |
Specialist referral (EULAR (all sites), NICE, OARSI) | IF a VE has severe symptomatic OA of the knee or hip despite non-surgical therapy, THEN a referral to an orthopaedic surgeon should be made, BECAUSE joint surgery may reduce pain and improve functional status and quality of life. (ACOVE-3)46–48 | RAND QA,29 ACOVE-1,32 33 and as adapted (ELSA41), Arthritis Foundation,51 52 QIGP,49 EUMUSC.net55 |
12–14
37
38
40
50 It would be feasible to capture the presence of non-surgical therapy indicators in the record, though routine data sources cannot be used to determine the need for a surgical opinion reliably |
*It should be noted that different sources of guidance offer varying recommendations about the use of specific NSAIDs; in the UK, NICE recommend a standard NSAID or COX-2 inhibitor (other than etoricoxib 60mg) to be coprescribed with a PPI.8
ACOVE, Assessing Care of Vulnerable Elders; ACR, American College of Rheumatology; ADL, activities of daily living; COX, cyclooxygenase, ELSA, English Longitudinal Study of Ageing; EULAR, European League Against Rheumatism; EUMUSC.net, European Musculoskeletal Conditions Surveillance and Information Network; GI, gastrointestinal; HPCQI, Home-based Primary Care Quality Initiative; NH, nursing home; NICE, National Institute for Health and Care Excellence; NSAIDs, non-steroidal anti-inflammatories; OA, Osteoarthritis; OARSI, Osteoarthritis Research Society International; OTC, over the counter; PCPI, Physician Consortium for Performance Improvement; PPI, proton-pump inhibitor; QA, Quality Assessment; QIGP, Quality Indicators for General Practice; VE, vulnerable elder.
Of 10 indicator development study groups, five were based on the Assessing Care of Vulnerable Elders (ACOVE) indicators. Overall, the ACOVE series of indicators were found to most closely fulfil the assessment criteria due to their robust evidence collection and consensus development, and field testing, and update in ACOVE-3. The modifications to ACOVE-1 indicators (for use the English Longitudinal Study of Aging,41 in nursing homes42 and home-based primary care45) were minor, such as to the target population or recommended care process time frames. The degree to which modifications were subject to further empirical study and consensus varied. We judged the modified indicators to be compatible with the originals, although there was variability regarding the indicators of use of oral non-steroidal anti-inflammatories (NSAIDs) and gastroprotective agents, in terms of the drugs recommended or the target population. The RAND indicators were the earliest identified; they were based on a literature review (not identified as systematic) and high-quality consensus exercise. The developers of the Arthritis Foundation indicators had undertaken a ‘comprehensive’ literature review, and a high-quality consensus exercise. One example of implementation (of the non-pharmacological indicators) was found. The remaining indicator sets used an evidence synthesis or consensus exercise which was less rigorous, or not specified. Some had no identified evidence of implementation (eg, Physician Consortium for Performance Improvement (PCPI) indicators).
All identified articles used process-of-care measures as indicators; one indicator set (European Musculoskeletal Conditions Surveillance and Information Network; EUMUSC.net) also used three outcome measures. We identified no papers in which quality improvement over time had been investigated.
Narrative synthesis
The indicators identified in the studies were grouped into themes. A summary of exemplar indicators is shown in table 3. The basis of the exemplar choice from the truth and feasibility aspects of the evidence is shown (discrimination was not found to be empirically supported). A list of indicators suitable for routine implementation in primary care is shown in table 4. Online supplementary table S4 lists indicators which do not currently provide sufficient evidence or feasibility for implementation in primary care.
Holistic assessment
Table 4.
Proposed indicators for primary care implementation
Overarching theme | Proposal for primary care implementation |
---|---|
Holistic Assessment: Pain | % patients with a working diagnosis of OA with evidence of pain assessment within the previous 12 months |
Holistic Assessment: Function | % patients with a working diagnosis of OA with evidence of function assessment within the previous 12 months |
Education | % patients with a working diagnosis of OA with evidence of education or advice since diagnosis |
Exercise 1 | % patients with a working diagnosis of OA in the hip or knee with evidence of exercise advice or physiotherapy referral since diagnosis |
Exercise 2 | % patients with a working diagnosis of OA with evidence of an activity review within the previous 12 months |
Weight loss 1 | % patients with a BMI ≥30 kg/m2 who have a record of weight loss advice within the previous 12 months |
Weight loss 2 | % patients with a working diagnosis of OA with a BMI ≥25 kg/m2 who have a record of weight loss advice within the previous 12 months |
Aids and devices 1 | % patients with a working diagnosis of OA with evidence of functional impairment who are recorded as receiving a referral or assessment for ambulatory assistive devices within the previous 12 months |
Aids and devices 2 | % patients with a working diagnosis of OA with evidence of functional impairment who are recorded as receiving a referral or assessment for assistive devices within the previous 12 months |
Paracetamol 1 | % patients with a working diagnosis of OA with evidence of paracetamol as the first oral analgesic prescribed or advised since diagnosis |
Paracetamol 2 | % patients with a working diagnosis of OA taking oral analgesics or NSAIDs with evidence that a suitable maximal dose of paracetamol was tried beforehand |
Oral NSAIDs 1 | % patients with a working diagnosis of OA with evidence of a standard NSAID or COX-2 inhibitor as the first oral NSAID prescribed or advised since diagnosis |
Oral NSAIDs 2 | % patients with a working diagnosis of OA taking an oral NSAID with a documented risk assessment prior to first prescription |
Gastroprotection | % patients with a working diagnosis of OA taking an oral NSAID who are also prescribed a PPI or alternative gastroprotective agent |
Specialist assessment | % patients with a record of achievement of all other applicable indicators prior to specialist referral* |
*That is, the other 14 indicators above, depending on applicability of weight and therapy indicators to individual patients.
BMI, body mass index; COX, cyclooxygenase; NSAIDs, non-steroidal anti-inflammatories; OA, Osteoarthritis; PPI, proton-pump inhibitor.
There were 28 occurrences of indicators related to holistic assessment of patients.
Assessments of pain and function were relatively frequent. The ACOVE-3 examples were rated most highly. Exemplar indicators have been selected for these elements of care. Indicators for joint examination and joint aspiration arose less frequently, though were still the result of at least one high quality evidence synthesis and consensus exercise, but had not successfully been implemented.
Education and information
There were 18 occurrences of indicators for education in OA. The Arthritis Foundation indicator was selected due to its cited evidence synthesis and consensus method, and its consistency with the previously implemented ACOVE-1 and recently published EUMUSC.net indicators; no education indicator was included in ACOVE-3. There was some variation in the timeframes specified for education. It was not clear from most studies implementing this indicator theme how the required level of detail about type of education was obtained. For example, one study asked the patient in a telephone interview “Has any doctor or nurse ever talked to you about: (1) What your arthritis or joint pain will be like as time goes on, or the natural history of arthritis?, (2) How to keep your arthritis or joint pain from getting worse?, (3) How your arthritis can be treated?”38; a criterion to pass the indicator was at least one positive response. Evidence from implementation studies suggests that the indicator as worded is less feasible for implementation in primary care, requiring either a more generic indicator or a series of specific patient self-report indicators; we propose a more generic indicator.
The EUMUSC.net team includes an education indicator aimed at clinicians, which we did not include as it is not a patient-focused indicator.
Exercise and physiotherapy
There were 22 occurrences of indicators recommending or prescribing exercise or physiotherapy. One targeted patients with hand, hip and knee OA55; one self-report indicator implemented also included patients with hand, hip or knee OA40; six refer to exercise for patients with OA of the hip or knee; the remainder specify those with knee OA. There were variations between indicators on exercise, with some recommending that a programme be ‘prescribed’, ‘recommended’ or ‘considered’. Some referred to specific strengthening programmes, others to general aerobic exercise, or physical therapy. For example, one study used a record of prescription for lower extremity strengthening or ambulation with a Physical Therapist or Restorative Nursing Assistant after OA diagnosis as a criterion43; others used non-routine sources such as patient interview or unspecified sources. Evidence from implementation studies suggests that feasible indicators for primary care relate to the offer of exercise advice or physiotherapy referral, and review of current exercise activity. It would be feasible to separate two elements of the ACOVE-3 indicator into an indicator for advice, recommendation or prescription of exercise, and an indicator of annual review of activity.
Weight management
There were eight occurrences of indicators regarding weight loss in overweight patients, six for patients with OA and two for primary prevention. There was some variation in the BMI intervention threshold as well as in the type of advice or referral. There were two implementation studies identified, of the Arthritis Foundation indicator regarding weight management in symptomatic OA, in which Li et al53 used entry to a weight-loss programme or dietetics appointment as criteria for indicator achievement and the weight loss advice self-report indicator in Østerås et al40 A primary care indicator related to advice regarding weight loss to reduce the risk of OA, or to improve symptoms in people with established OA would be feasible. A further identified indicator, regarding referral to a weight-loss programme if a person has been overweight for 3 years or more, would be less feasible and desirable, due to greater difficulty establishing the denominator population.
Assistive devices (ambulatory and other)
There were nine occurrences of indicators for assessment of need for assistive devices. These covered assessment of need for ambulatory and non-ambulatory assistive devices but there were no specifically recommended interventions. Two examples of implementation were found, of Arthritis Foundation indicators (similar to and consistent with the ACOVE-3 indicators) by Li et al,53 in which credit was given when a patient had seen a physiotherapist or occupational therapist for ambulatory or non-ambulatory devices respectively within the previous year, and similar patient self-report indicators in Østerås et al.40 In line with this, general indicators for referral or assessment for ambulatory or assistive devices currently appear feasible in primary care.
Analgesics (paracetamol and oral NSAIDs)
There were 53 occurrences of indicators for use of analgesics in OA. These covered topics such as assessment of current use or consideration of analgesics; use of appropriate first-line analgesics; and risk assessment and communication. Preferred indicators generally result from at least one high quality evidence synthesis and consensus exercise, although the basis for the NSAID risk assessment indicator from the PCPI is unclear54 (though consistent with a similar indicator from the ACOVE-1 group). Where available, the ACOVE-3 indicators were chosen. Several indicators regarding use of paracetamol and NSAIDs are considered feasible for use in primary care (see table 3). Indicators regarding assessment of existing use and consideration of additional treatment from the PCPI54 and an implemented indicator regarding stronger analgesics (Østerås et al40) were not selected due to an unspecified evidence base and consensus approach; indicators regarding risk explanation were also not selected due to difficulties implementing these in routine data sources (without free text medical record analysis).
Gastroprotection
There were 13 occurrences of indicators for use of gastroprotective agents under certain conditions. However, there were variations in the triggers for prescribing a gastroprotective agent, and in the choice of agent to be used. The broadest (PCPI) indicator54 cites a meta-analysis as having indicated that use of gastrointestinal prophylaxis can be effective in reducing the incidence of adverse events. This would be consistent with the NICE recommendation that everyone over 45 years prescribed a NSAID for OA should be coprescribed a proton pump inhibitor.8 Where indicators have been implemented, they often use past medical history or co-therapy with other agents (eg, aspirin or warfarin) to determine the denominator group for this indicator. The PCPI indicator is the most feasible, although this has been narrowed to include only proton-pump inhibitor gastroprotection in line with NICE guidance.
X-rays, injections, specialist assessment and joint replacement
There were 16 instances of indicators for referral to a specialist and use of X-rays when symptoms were not improving under non-surgical care. As guidance for management of OA does not recommend routine use of X-rays, and no examples of implementation of X-ray indicators was found, this indicator was not considered feasible. A number of indicators referred to failure of other therapies as a prerequisite for specialist referral but ‘failure’ was not consistently defined. One study asked patients if they had pain and functional impairment, and had been offered a joint replacement or orthopaedic assessment.38 Another used a patient self-report to identify failure of conservative treatment leading to referral.40 An indicator mandating that all other indicators must have been recorded as appropriately met prior to referral was considered to be feasible.
There was also one indicator implemented for the consideration of steroid injections for acute symptomatic deterioration.40 This was not considered feasible for routine implementation in primary care since acute deterioration is hard to identify from the record and many injections take place in secondary care.
Outcome measure indicators
The EUMUSC.net project also identified three outcome measures55:
a 20% functional improvement within 3 months of a treatment initiation or change
a 20% reduction in pain within 3 months of a treatment initiation or change
enablement of workforce participation for people of working age.
These were considered less feasible for primary care due to the complexity of accounting for comorbidities and case-mix.
Discussion
Through a systematic review of OA indicators and a quality appraisal of the indicator development and implementation, we identified 15 indicators of the quality of primary care for OA which could be implemented, benefiting patients, clinicians and policy development.
While the conclusions of the published guidance diverge in some aspects (particularly the use of Symptomatic Slow-Acting Drugs in Osteo-Arthritis, and in some of the detail of oral NSAID use and gastroprotection), the interventions recommended by the different expert groups are broadly similar. The selected indicators were broadly applicable across all the guidance groups.
Within themes, there are differences between some of the identified indicators. Indicators sometimes target differing populations (eg, OA of the knee or any OA), frequency or threshold of assessment or intervention, type of treatment (eg, variation in oral NSAID recommended, and type of gastroprotective agent). These differences are not sufficiently major to cause difficulties in the implementation of the underlying indicator theme.
There are some limitations in this review. There may be indicators not captured by the search strategy (including any prior to 2000, and non-English language indicators). Given the thorough nature of the indicator development methodology for a number of the indicator sets, it seems unlikely that any major themes will have been omitted. In contrast with the assessment of publications on randomised controlled trials (eg, the approach taken by the Cochrane Collaboration), quality assessment of indicators themselves is not a highly developed methodology.56
We have selected indicators judged sufficiently robust and feasible for use in routine practice. The use of indicators is dependent upon systematic information capture. In the UK, approximately 90% of prescriptions are obtained with no cost to the patient, and over-the-counter analgesics are restricted in quantity.57 Analgesics and NSAIDs indicators based on data from computer-generated prescriptions are likely to be valid with no change to recording practice. Other indicators would require a change in coding practice (more detailed coded clinical information). The indicators should be generally applicable to countries with well-developed primary care systems and electronic medical records. The indicators would work best with strategic implementation, for example by inclusion in the Quality and Outcomes Framework of pay-for-performance in UK primary care.58
While there are some domains with well-developed and valid indicators, some elements do not have such indicators. For example (1) Holistic assessment: all dimensions other than pain and function, notably periodic review, a jointly formulated management plan and the effect of comorbidities; (2) Education and self-management: the development of a self-management plan and thermotherapy; (3) Non-pharmacological management: manipulation and stretching, electrotherapy, bracing, joint supports, footwear and insoles; and (4) Pharmacological management: topical NSAIDs, capsaicin and intra-articular injections. In principle, some of these areas might be suitable for the development of indicators.
We did not identify any negative (‘do not do’) indicators. There are some areas of guidance from which one might usefully derive such indicators for use in primary care. For example, the use of topical rubefacients, electroacupuncture, nutraceuticals, or intra-articular hyaluronan injections, or referral for arthroscopic lavage for OA, based on the NICE guidance.8
We found no evidence of external validity (that implementation of indicators is associated with quality improvement). Also, there is no evidence of indicators’ sensitivity to change, so this must currently be assumed. The degree to which a change in recording of the care processes actually reflects a change in the quality of care delivered has not clearly been identified. These areas warrant further investigation: an increased use of patient-reported measures such as those used by Østerås et al40 would help identify changes in process delivery and outcome.
We have identified a range of indicators for OA which have a good evidence base, are consistent with international guidance, and many of which have been implemented previously. As the disease burden of OA is high, and much of it is presented clinically to GPs, incorporation of these indicators to routine primary care practice is recommended.
Supplementary Material
Acknowledgments
The authors wish to thank the anonymous reviewers of the manuscript for their insightful and helpful observations; also Professor Peter Croft, Professor Danielle van der Windt and Dr Olalekan Uthman, of the Arthritis Research UK Primary Care Centre at Keele University for comments on an earlier draft of this manuscript.
Footnotes
Contributors: KSD developed the idea for the review. JJE and JLJ designed the search strategy. JJE, MK and KSD analysed the papers and extracted the data. All authors contributed to the analysis and revised the paper. JJE is the guarantor.
Funding: This report is independent research arising from an In-Practice Fellowship supported by the National Institute for Health Research (NIHR) with additional funding from the NIHR Programme Grants for Applied Research Programme (Grant Reference Number RP-PG-0407-10386). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work other than detailed above; JJE, MK and JB provide general medical services and benefit financially from the Quality and Outcomes Framework (which does not currently include osteoarthritis), KSD has been an invited speaker to the EULAR conference and a member of the NICE OA guideline development group—Keele University have received payments and reimbursements of travel and other expenses related to these activities. There are no other financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years. There are no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.RCGP Birmingham Research Unit. Weekly Returns Service Annual Prevalence Report 2007. http://www.rcgp.org.uk/clinical-and-research/~/media/Files/CIRC/CIRC%2076%20-%2080/BRU_Annual_prevalence_report_2007.ashx
- 2.National Collaborating Centre for Chronic Conditions. Osteoarthritis: national clinical guideline for care and management in adults. London: Royal College of Physicians, 2008. [PubMed] [Google Scholar]
- 3.Hochberg MC, Altman RD, April KT, et al. American College of Rheumatology 2012 recommendations for the use of nonpharmacologic and pharmacologic therapies in osteoarthritis of the hand, hip, and knee. Arthritis Care Res 2012;64:465–74. [DOI] [PubMed] [Google Scholar]
- 4.Jordan KM, Arden NK, Doherty M, et al. EULAR Recommendations 2003: an evidence based approach to the management of knee osteoarthritis: Report of a Task Force of the Standing Committee for International Clinical Studies Including Therapeutic Trials (ESCISIT). Ann Rheum Dis 2003;62:1145–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Zhang W, Doherty M, Arden N, et al. EULAR evidence based recommendations for the management of hip osteoarthritis: report of a task force of the EULAR Standing Committee for International Clinical Studies Including Therapeutics (ESCISIT). Ann Rheum Dis 2005;64:669–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Zhang W, Doherty M, Leeb BF, et al. EULAR evidence based recommendations for the management of hand osteoarthritis: report of a Task Force of the EULAR Standing Committee for International Clinical Studies Including Therapeutics (ESCISIT). Ann Rheum Dis 2007;66:377–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Zhang W, Moskowitz RW, Nuki G, et al. OARSI recommendations for the management of hip and knee osteoarthritis, Part II: OARSI evidence-based, expert consensus guidelines. Osteoarth Cartilage 2008;16:137–62. [DOI] [PubMed] [Google Scholar]
- 8.National Institute for Health & Clinical Excellence. Osteoarthritis: the care and management of osteoarthritis in adults. London: National Institute for Health & Clinical Excellence, 2008. [Google Scholar]
- 9.Croft P, Porcheret M, Peat G. Managing osteoarthritis in primary care: the GP as public health physician and surgical gatekeeper. Br J Gen Pract 2011;61:485–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Porcheret M, Jordan K, Jinks C, et al. Primary care treatment of knee pain—a survey in older adults. Rheumatology 2007;46:1694–700. [DOI] [PubMed] [Google Scholar]
- 11.McHugh GA, Luker KA, Campbell M, et al. A longitudinal study exploring pain control, treatment and service provision for individuals with end-stage lower limb osteoarthritis. Rheumatology 2007;46:631–7. [DOI] [PubMed] [Google Scholar]
- 12.Steel N, Maisey S, Clark A, et al. Quality of clinical primary care and targeted incentive payments: an observational study. Br J Gen Pract 2007;57:449–54. [PMC free article] [PubMed] [Google Scholar]
- 13.Broadbent J, Maisey S, Holland R, et al. Recorded quality of primary care for osteoarthritis: an observational study. Br J Gen Pract 2008;58:839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Steel N, Bachmann M, Maisey S, et al. Self reported receipt of care consistent with 32 quality indicators: national population survey of adults aged 50 or more in England. BMJ 2008;337:a957. 10.1136/bmj.a957 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.McHugh GA, Campbell M, Luker KA. Quality of care for individuals with osteoarthritis: a longitudinal study. J Eval Clin Pract 2012;18:534–41. [DOI] [PubMed] [Google Scholar]
- 16.Campbell SM, Braspenning J, Hutchinson A, et al. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care 2002;11:358–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lawrence M, Olesen F. Indicators of Quality in Health Care. Eur J Gen Pract 1997;3:103–8. [Google Scholar]
- 18.Hochberg MC. Quality measures in osteoarthritis. Clin Exp Rheumatol 2007; 25(6 Suppl 47):102–6. [PubMed] [Google Scholar]
- 19.Strömbeck B, Petersson IF, Vliet Vlieland TPM. Health care quality indicators on the management of rheumatoid arthritis and osteoarthritis: A literature review. Rheumatology 2013;52:382–90. [DOI] [PubMed] [Google Scholar]
- 20.Hunter DJ, Neogi T, Hochberg MC. Quality of osteoarthritis management and the need for reform in the US. Arthritis Care Res 2011;63:31–8. [DOI] [PubMed] [Google Scholar]
- 21.Centre for Reviews and Dissemination. Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care. York: University of York, 2009:292. [Google Scholar]
- 22.Agency for Healthcare Research and Quality. http://www.ahrq.gov [DOI] [PubMed]
- 23.University of Oxford: Centre for Evidence Based Medicine. Levels of Evidence 2009. http://www.cebm.net/index.aspx?o=1025
- 24.Campbell SM, Cantrill JA. Consensus methods in prescribing research. J Clin Pharm Ther 2001;26:5–14. [DOI] [PubMed] [Google Scholar]
- 25.Geraedts MAX, Selbmann H-K, Ollenschlaeger G. Critical appraisal of clinical performance measures in Germany. Int J Qual Health Care 2003;15:79. [DOI] [PubMed] [Google Scholar]
- 26.Marshall MN, Shekelle PG, McGlynn EA, et al. Can health care quality indicators be transferred between countries? Qual Saf Health Care 2003;12:8–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wollersheim H, Hermens R, Hulscher M, et al. Clinical indicators: development and applications. Neth J Med 2007;65:15–22. [PubMed] [Google Scholar]
- 28.Boers M, Brooks P, Strand CV, et al. The OMERACT filter for Outcome Measures in Rheumatology. J Rheumatol 1998;25:198–9. [PubMed] [Google Scholar]
- 29.Kerr E, Asch S, Hamilton E, et al. Quality of care for general medical conditions: a review of the literature and quality indicators. Santa Monica, CA: RAND Health, 2000. [Google Scholar]
- 30.McGlynn E, Asch S, Adams J, et al. The quality of health care delivered to adults in the United States. New Engl J Med 2003;348:2635. [DOI] [PubMed] [Google Scholar]
- 31.Asch S, McGlynn E, Hogan M, et al. Comparison of Quality of Care for Patients in the Veterans Health Administration and Patients in a National Sample. Ann Intern Med 2004;141:938. [DOI] [PubMed] [Google Scholar]
- 32.MacLean C. Quality indicators for the management of osteoarthritis in vulnerable elders. Ann Intern Med 2001;135:711. [DOI] [PubMed] [Google Scholar]
- 33.Shekelle P, MacLean C, Morton S, et al. Assessing Care of Vulnerable Elders: Methods for Developing Quality Indicators. Ann Intern Med 2001;135(8 Part 2):647. [DOI] [PubMed] [Google Scholar]
- 34.Wenger N, Solomon D, Roth C, et al. The Quality of Medical Care Provided to Vulnerable Community-Dwelling Older Patients. Ann Intern Med 2003;139:740. [DOI] [PubMed] [Google Scholar]
- 35.Chodosh J, Solomon D, Roth C, et al. The quality of medical care provided to vulnerable older patients with chronic pain. J Am Geriatr Soc 2004;52:756. [DOI] [PubMed] [Google Scholar]
- 36.Higashi T, Shekelle P, Solomon D, et al. The Quality of Pharmacologic Care for Vulnerable Older Patients. Ann Intern Med 2004;140:714. [DOI] [PubMed] [Google Scholar]
- 37.Higashi T, Shekelle P, Adams J, et al. Quality of Care Is Associated with Survival in Vulnerable Older Patients. Ann Intern Med 2005;143:274. [DOI] [PubMed] [Google Scholar]
- 38.Ganz D, Chang J, Roth C, et al. Quality of osteoarthritis care for community-dwelling older adults. Arthritis Rheum 2006;55:241. [DOI] [PubMed] [Google Scholar]
- 39.MacLean C, Louie R, Shekelle P, et al. Comparison of administrative data and medical records to measure the quality of medical care provided to vulnerable older patients. Med Care 2006;44:141. [DOI] [PubMed] [Google Scholar]
- 40.Østerås N, Garratt A, Grotle M, et al. Patient-reported quality of care for osteoarthritis: development and testing of the osteoarthritis quality indicator questionnaire. Arthritis Care Res 2013;65:1043–51. [DOI] [PubMed] [Google Scholar]
- 41.Steel N, Melzer D, Shekelle P, et al. Developing quality indicators for older adults: transfer from the USA to the UK is feasible. Qual Saf Health Care 2004;13:260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Saliba D, Solomon D, Rubenstein L, et al. Quality indicators for the management of medical conditions in nursing home residents. J Am Med Dir Assoc 2005;6:S36. [DOI] [PubMed] [Google Scholar]
- 43.Cadogan M, Schnelle J, Al-Sammarrai N, et al. A standardized quality assessment system to evaluate pain detection and management in the nursing home. J Am Med Dir Assoc 2005;6:1. [DOI] [PubMed] [Google Scholar]
- 44.Zingmond DS, Saliba D, Wilber KH, et al. Measuring the quality of care provided to dually enrolled medicare and medicaid beneficiaries living in nursing homes. Med Care 2009;47:536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Smith K, Soriano T, Boal J. Brief communication: National quality-of-care standards in home-based primary care. Ann Intern Med 2007;146:188. [DOI] [PubMed] [Google Scholar]
- 46.Wenger N, Roth C, Shekelle P, et al. Introduction to the Assessing Care of Vulnerable Elders-3 quality indicator measurement set. J Am Geriatr Soc 2007; 55:S247. [DOI] [PubMed] [Google Scholar]
- 47.ACOVE Investigators. Assessing Care of Vulnerable Elders—3: Quality Indicators. J Am Geriatr Soc 2007;55(Suppl 2):S464. [DOI] [PubMed] [Google Scholar]
- 48.MacLean C, Pencharz J, Saag K. Quality indicators for the care of osteoarthritis in vulnerable elders. J Am Geriatr Soc 2007;55(Suppl 2):S383. [DOI] [PubMed] [Google Scholar]
- 49.Marshall M, Campbell S, Hacker J, et al. eds. Quality indicators for general practice. London: RSM Press, 2002. [Google Scholar]
- 50.Kirk S, Campbell S, Kennell-Webb S, et al. Assessing the quality of care of multiple conditions in general practice: practical and methodological problems. Qual Saf Health Care 2003;12:421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.MacLean C, Saag KS, Solomon D, et al. Measuring quality in arthritis care: methods for developing the Arthritis Foundation's quality indicator set. Arthritis Rheum 2004;51:193. [DOI] [PubMed] [Google Scholar]
- 52.Pencharz J, MacLean C. Measuring quality in arthritis care: the Arthritis Foundation's Quality Indicator set for osteoarthritis. Arthritis Rheum 2004;51:538. [DOI] [PubMed] [Google Scholar]
- 53.Li L, Sayre E, Kopec J, et al. Quality of nonpharmacological care in the community for people with knee and hip osteoarthritis. J Rheumatol 2011;38:2230. [DOI] [PubMed] [Google Scholar]
- 54.American Academy of Orthopaedic Surgeons, Physician Consortium for Performance Improvement. Osteoarthritis: Physician Performance Measurement Set. 2006. http://www.aaos.org/research/committee/evidence/PCPI_Measures_Osteoarthritis.pdf
- 55.EUMUSC.net. Health care quality indicators for OA. 2012. http://www.eumusc.net/workpackages_wp6.cfm
- 56.Higgins JPT, Green S. eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. [Google Scholar]
- 57.House of Commons Committee of Public Accounts. Department of Health: Prescribing costs in primary care. London: The Stationery Office Limited, 2007. [Google Scholar]
- 58.NHS Employers. Quality & Outcomes Framework. 2013. http://www.nhsemployers.org/PayAndContracts/GeneralMedicalServicesContract/QOF/Pages/QualityOutcomesFramework.aspx
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.