Organisation, data evaluation, interpretation and effect of arthroplasty register data on the outcome in terms of revision rate in total hip arthroplasty

Abstract

Originally developed in Scandinavia, national arthroplasty registers have spread worldwide during the last decade. The value of registers for quality improvement in arthroplasty has frequently been documented. However, for the development of a successful register a few key points should be taken into account. Uncontrolled loss of patients from the registry area should be avoided. Registers should form an integral part of a country’s medical system. To realise the potential for improvement, it is crucial that physicians deal with the results in detail. Thus it is absolutely essential to involve the specialty societies in the interpretation and dissemination of results. With respect to revision rates, register data are usually more valid than meta-analyses of clinical studies. For every physician the most valuable data are those coming from a register in his own country; the development of national arthroplasty registers should therefore be continued.

Introduction

Arthroplasty registers were developed in Scandinavia more than 30 years ago [1] and have given impressive proof of their value for quality control and the development of arthroplasty ever since [2–11]. The last ten years have seen a rapid spread of this concept [12]. While Europe, the region of origin, has been the centre of this process, successful projects have also been developed in Australia and New Zealand. Other regions of the world, such as Asia and America, have been dealing with the issue of improving quality control through registers, but, with the exception of Canada, have never progressed beyond the initial project phase nor yet reached one or more crucial points necessary for the successful operation of an arthroplasty register.

A recently published position statement by the presidents of all leading English-speaking orthopaedic societies and a recent JBJS-B editorial show that this is also a major topic for scientific societies and journals [13, 14].

The primary goal of a register is quality improvement. This process is mainly based on information feedback for decision-makers in the health care system, such as physicians, but also public health institutions. On the basis of benchmarks, the respective decision-makers can identify potential improvements in their area of responsibility and implement them by making autonomous decisions. The impact of these decisions on the outcome is subsequently re-evaluated, which generates continuous processes of quality monitoring.

Successful registers such as those in Scandinavia have reduced the revision rate considerably over time, e.g. by approximately 50% in knee and hip arthroplasty. The range of quality deviations from the national average was diminished markedly in individual departments, with departments performing poorly in the beginning doing disproportionately better as a result of their higher potential for improvement. For patients undergoing total hip arthroplasty in Sweden between 1979 and 1991 the average revision rate at seven years was 6.5%. While 30% of departments did not exhibit statistically significant deviations from the average value, 45% performed significantly better, and 25% significantly worse. A decade later, between 1992 and 2002, the average revision rate at seven years was 4.2%, i.e. reduced by approximately one third. Moreover, only 13% of departments still had significantly worse results than the improved benchmark [15].

A major proportion of quality improvement was achieved within the first four years after the introduction of the register. The revision rate at 26 years decreased from 24% to 16%, and at ten years from 14.5% to 9%. This corresponds to a relative risk reduction for revision surgery after primary operation of about one third for both follow-up periods [16].

The effect was also observed for short-term revisions within two years, which primarily depend on the surgical procedure and infections, after specific evaluations had been included [17]. The mere fact that a quality monitoring system is introduced and the availability of valid data showing those areas in which there is potential for improvement obviously lead to quick and autonomous action within departments that may have relevant effects on the quality of patient treatment.

Registers are structurally complex organisations with a number of stakeholders. The best-known register publications, annual reports, only represent part of the evaluations and data available. Usually each participating department is given access to confidential reports comparing the specific situation of the respective department with the national average. Furthermore, it is possible to set up lists of those patients who were treated in the respective department and who had to undergo revision surgery—irrespective of where this re-operation was performed. Based on these data, individual departments are able to make further, autonomous investigations when needed, and include information not recorded in the register, such as surgeons, surgical procedures or whether patients suffering short-term infection were treated in particular operating rooms or wards or with particular instruments. On a voluntary basis, comparisons between different departments can be drawn. Since each department has full control over the data, compliance with specific requirements regarding data access and data protection can be guaranteed.

Topics of general interest can be analysed within the scope of scientific projects, which will thus contribute substantially to scientific discussion.

Complications after surgical interventions are not only extremely stressful for the patients, they also represent a major cost factor in the health care system. By avoiding these consequential costs and at the same time improving performance quality, registers are able to provide a substantial contribution to the health care system [17].

The large number of stakeholders involved, their interests, in addition to the different purposes of data use entail a multitude of possible forms of organisation.

However, some crucial points have turned out to be absolutely essential.

Organisation of an arthroplasty register

The primary purpose of a register is to record operations and complications. This is difficult sometimes when, as in arthroplasty, revision operations are relatively rare and distributed over a long period of time. The problem is aggravated by the fact that many dissatisfied patients change physicians and the instigator is not informed. This would, however, be the basic prerequisite for consequences that could induce improvement.

Therefore, the completeness of data collection is the primary goal of any register documentation. That this ambitious aim can be achieved has already been proven in a number of countries [18–23].

To attain the primary goal of completeness, the burden on the hospital staff must be kept at a reasonable level. This can be achieved in various ways, for example, by short questionnaires or avoiding double input of data which are already available. Modern IT solutions can be helpful since data already recorded can be read out. Indispensable information has already been defined by consensus in the “EFORT EAR Minimal Datasets” [24].

To a limited extent, this minimum standard can be extended for specific issues. However, it should be taken into account that every modification of the dataset will restrict the range of evaluation options. The dataset should therefore be kept as stable as possible and should only be modified after careful consideration.

Every register covers a particular geographical area, a country, a region, or even a specific hospital. To reach the primary goal of complete data collection, patients must be prevented from leaving the area covered by the register undetected. For a national register, administrative requirements, for example, that public health insurance does not cover surgery abroad, usually form a “sufficient” obstacle. For regional registers, undetected departure is often a major problem since, mainly due to lack of resources, it is impossible for them to contact the large and continuously increasing number of patients included in the register on a regular basis, as is usual in the case of clinical studies.

An initial approach to solving the problem is, for instance, data collation with administrative data. The regional Register of Emilia Romagna in Italy is a good example that this approach can lead to comprehensive data collection [23]. Usually, personal datasets are available for the remuneration for medical services. This makes it at least possible to ascertain whether patients had to undergo revision surgery and where the corresponding records could be provided. Some countries, such as Norway, go so far as to make the remuneration for services abroad conditional on the register documentation and even set up foreign-language data sheets.

A register must form an integral part of the national health care system to become effective. To be able to link a patient’s primary and revision surgery, it is necessary to record personal data. In most countries this is subject to stringent legal regulations. Relatively liberal provisions are offered only in few countries, such as Sweden, where special permission can be obtained that allows national personal data to be recorded and processed even in universities. This was one of the main reasons why many projects that were based on academic initiatives, despite the strong commitment of those involved, finally failed in the past. To organise registers of a large clinical multicentre study on a voluntary basis has not proved successful. In many countries the involvement of public institutions is therefore a prerequisite for setting up a register, which occasionally raises concern within the medical profession that this would imply direct dependency and interference with daily decisions in patient treatment. In countries where the development of a register was successful under these circumstances, it has proved useful to allocate tasks and responsibilities by consensus and in accordance with professional expertise and legal requirements.

Being complex, expensive and legally delicate, data collection and recording normally are a core competence of authorised public health institutions. Apart from the legal framework, scientific societies and academic institutions are usually neither able to provide long-term funding nor the professional infrastructure necessary to ensure successful operation in the long run. One should bear in mind that investments for development and data collection are usually required for many years before it makes sense to start with evaluation and publication activities.

Apart from that, proper interpretation of register data and results does not only require a high level of medical expertise but also a profound knowledge of daily routines in patient treatment. This can only be ensured by a national orthopaedic society. Since the competence of data interpretation and publication represents the core point of physicians’ concerns, a solution is to be considered reasonable where data collection, recording and data processing including anonymisation are the responsibility of public health authorities, whereas interpretation and publication are assigned to an expert panel of physicians. Since panel members have access to sensitive data, they should be subject to democratic control and be nominated by the respective scientific societies.

During the foundation phase of a register project the demands on the members of the expert panel are often underestimated. As a matter of course it is sensible to nominate esteemed and leading members of the scientific societies. However, these persons usually already have a variety of responsibilities so that it has proved reasonable to also involve young colleagues. This makes it possible to delegate activities such as detailed analyses or literature research and ensure long-term continuity in staff. For younger colleagues the work with register datasets also provides an opportunity to build their scientific careers.

The scientific analysis of particular issues that have appeared noticeable in the course of routine evaluations is essential for realising potential for improvement, on the one hand, based on additional specific findings, and on the other hand, by regular discussion of the results at congresses or other fora for medical discussion. The value of these activities is demonstrated by a comparison between Sweden and Finland, the countries with the longest tradition of arthroplasty registers.

Whereas in Sweden orthopaedic surgeons initiated the registers and the data were used intensively within the scientific societies, the Finnish register was run by the National Agency of Medicine. While reports were published, there was no active, central involvement of scientific societies. Feedback activities were not considered to be the register’s task. Scientific projects were possible, but not encouraged. Under these circumstances, measurable improvements of survival rates were only achievable after many years, and the latest data show that the revision burden is 50% higher in Finland than it is in Sweden. The structure of the Finnish register has only been changed recently, with considerable increase to the involvement of physicians and the scientific society (Fig. 1).

Fig. 1.

Revision burden after primary total hip arthroplasty in Finland (The 2006 Implant Yearbook on Orthopaedic Endoprostheses, http://www.nam.fi/medical_devices)

Quality improvements can only be achieved in cooperation with physicians involved in surgery by discussing the findings in detail and putting them into practice. Therefore, the scientific discussion and analysis of the data, as well as the dissemination of results are a crucial aspect in the organisation of a register and should by no means be misunderstood as an egoistic activity of individual physicians, which, unfortunately, occasionally happened on the part of the authorities in the past.

Since the scientific analysis of register findings is vital for implementing improvements and thus for ensuring a register’s success, such activities should be actively supported. Scientific societies with their various platforms such as congresses are ideally suited to undertake this task and should support these initiatives from the very beginning. This is also important to maintain the motivation of those physicians who have to cope with the documentation in their departments.

Register data are based on few, but well-standardised and objective parameters such as revision rate. The inclusion of too many parameters increases the documentation burden and, as a consequence, reduces the compliance rate. Thus, the primary goal of achieving completeness is compromised, and the success of the project is put at risk. Accordingly, any decision about the content of the questionnaire is a fairly delicate matter.

Revision rate is an essential, but not the only, parameter for assessing the quality of arthroplasty treatment. Subjective assessment of the outcome and the quality of life are without doubt of equal importance. However, it is considerably more difficult to cover these issues by means of a register. For organisational reasons it is impossible to control incoming data, for example, via a monitoring system. Patient self-assessment scores are a possible way of collecting data without increasing the physicians’ workload. Individual registers have already started successful initiatives in this context. However, there is actually no consensus at the moment about which scores are suited best and what kind of organisational structure is feasible in practice. The inclusion of such instruments should be considered critically in the initial phase of a new register since this is associated with a considerable increase in task complexity.

Interpretation of arthroplasty register datasets

Register data are intended to provide a comprehensive and representative picture of the outcome after arthroplasty interventions in a certain country or a certain region. This of course also comprises the circumstances under which the results are achieved, such as surgical concepts, requirements of the health care system and how this system is organised, or the patients treated. This specific background should therefore be taken into account when register data and results from other departments or countries are being interpreted. For example, it cannot be expected that the excellent results after cemented hip or knee arthroplasty in Sweden are easily reproducible when the long tradition and comprehensive knowledge of cementing techniques are missing in one’s own environment. Even evaluation techniques or implant designations may lead to misinterpretation if a different terminology and various definitions are used [25–27].

Annual reports of arthroplasty registers are primarily addressed to the physicians of the respective country. Therefore it does not make sense to describe the manifold influencing factors comprehensively and in every detail. This restricts the interpretation of foreign register data, just like the lack of standardisation in individual registers’ evaluation and reporting. First initiatives to improve the situation, such as the cooperation of the Scandinavian registers within NARA (Northern Arthroplasty Register Association), are therefore much appreciated.

The results of individual hospitals differ considerably even after registers have been running for many years. Sweden and Denmark publish the revision rates of individual departments. Deviations from the mean have been shown up to 300% [17, 28–30]. Apart from individual physicians’ expertise or the implants used, the deviations are also influenced by pre-existing factors that can hardly be changed. Due to a higher proportion of surgical training and occasionally more complex cases, university hospitals often have to deal with more difficult initial conditions. Innovations or the introduction of new surgical techniques and new implants may have a negative effect on the results during the learning curve.

These factors can hardly be assessed by a simple comparison of unadjusted data. The Swedish Hip Arthroplasty Register has published very good graphic representations of important factors influencing the risk profile of the patients treated in the individual departments. Quite frequently direct comparisons are made as to the performance of individual implants. In doing so one should at any rate take into account factors which are independent of the implant have at least the same influence as the quality of the product used. Correspondingly, the data should be interpreted with great care. Scientific studies on the basis of register data where the comparability of the raw data is enhanced by means of inclusion and exclusion criteria are thus reasonable and increase the quality of conclusions.

Studies based on existing datasets generally follow particular rules which should be taken into account in interpretations. Similar to clinical studies, inclusion and exclusion criteria are defined to form comparable cohorts, however retrospectively. In this case, possibly missing or desirable data cannot be collected retrospectively. Evaluations that correspond with the primary purpose of a data collection, for instance, comparisons of the performance of implants recorded in a register, usually present no problems. However, if the data are used to investigate other issues, or if a great variety of data are combined, this may lead to cohorts which are not directly comparable with regard to the question being examined, as can be demonstrated by a recently published example [31].

Patients treated with statins differ from the average population. They are often overweight, suffer from metabolic syndromes, and on average differ in lifestyles, for example, with respect to physical activity. Thus, if an analysis indicates that patients who are treated with statins show lower revision rates after arthroplasty interventions, this observation may well be correct, considering good data quality. However, the conclusion that this is an effect of the therapy is just one possible interpretation, just as the assumption that it could be a consequence of a change in metabolic processes or of the strain on the implant due to the lifestyle.

Value of arthroplasty register datasets

Two specific features account for the particularly high quality of register data. They comprise all operations performed in the respective area, which considerably reduces specific influences of individual hospitals or makes them at least calculable. As described before, these effects have a huge impact on the clinical outcome. Furthermore, registers usually include a considerably higher number of cases than clinical studies.

Register data on average show significantly lower deviations in outcome than clinical studies. Meta-analyses of conventional clinical studies can control the specific circumstances under which the results have been achieved to a considerably lesser extent than is the case for register data. Apart from general impacts, patient selection or the study design may influence the outcome. A structured comparative study of individual implants represented in clinical studies and registers has shown that the average published revision rate exhibits statistically significant and relevant differences of more than 300% for approximately 50% of implants. Conspicuously often the published results of implant developers and of studies from the United States were not reproducible in worldwide register data.

For the majority of implants the cumulative numbers of cases of clinical studies described for individual products in peer-reviewed journals were not sufficient to draw reliable conclusions [32].

L.I. Havelin calculated, in his PhD thesis, how many cases would be required in a prospective, randomised, comparative study of two implants in order to meet the standard criteria for statistical power. To identify a difference of one percentage point at ten years, 13,474 patients would be needed. To identify the relatively big difference of two percentage points would still require 3,008 patients to be included [33].

Under these circumstances conventional follow-up studies quickly reach organisational limits. The vast majority of studies published on revision rates must be regarded as statistically underpowered. For metric data, as are used in many clinical scores, considerable smaller cohorts are required.

The extent to which the outcome data available from arthroplasty registers are valid enough to make far-reaching decisions in patient treatment should therefore be critically analysed. Prospective randomised studies, which currently represent the gold standard for scientific studies, are hardly available. Published clinical studies show considerable variation in the distribution of results. Particularly in the United States, publications are dominated by implant developers; about 50% of all cases in outcome studies come from members of this group, who differ substantially from the average surgeon as regards their expertise and interests. However, there are also developers whose published results are actually reproducible in average patient treatment by register data. Therefore, the conclusion that the deviations found for 50% of developers could be explicable by higher expertise alone does not seem to be a sufficient explanation of the differences [33].

At present there is a standard evaluation scheme to assess the value of scientific studies [34]. Under this scheme prospective randomised studies are rated higher than cohort studies, for example, the category that also includes register studies. This assessment, however, is based on the assumption that samples are examined and confounders are limited as far as possible by sample selection in order to derive the most valid conclusions possible for the treatment of all patients. The possibility that comprehensive data comprising all patients treated could be available has not been considered in the current scheme. Thus, studies based on comprehensive, high-quality registers do not correspond to the prerequisites of the current scheme.

The scientific specialty societies should therefore seriously consider revising the current evaluation scheme.

Discussion

Register data can make a substantial contribution to improving the quality of arthroplasty treatment. As regards the recording of revision rates and their causes, they are superior to clinical studies.

However, registers do not compete, but complement conventional clinical studies.

For the individual surgeon the most valuable data are those coming from a register of his own country. Register development should therefore be encouraged in every country. For the interpretation of results from foreign registers methodological groundwork should be carried out in the future in order to standardise procedures and simplify them for surgeons.