Skip to main content
NCBI home page
Journal of Orthopaedics logoLink to Journal of Orthopaedics
. 2023 Nov 19;48:89–95. doi: 10.1016/j.jor.2023.11.041

Establishing clinically meaningful ranges of metal hypersensitivity in orthopaedic patients using COVID-19 vaccine-induced adaptive immune responses from fully vaccinated adults

Marco S Caicedo a,b, Vianey Flores a, Rochelle Siapno a, Michael Crosby b, Lauryn A Samelko a,b, Joshua J Jacobs b, Nadim J Hallab a,b,∗
PMCID: PMC10711032  PMID: 38089694

Abstract

Background

This study aimed to assess metal sensitization ranges among orthopaedic patients by comparing adaptive immune responses in all-comer pre- and post-operative orthopaedic adults who were COVID-19 unvaccinated or vaccinated vs patients with a painful aseptic implant by lymphocyte transformation test (LTT) to SARS-CoV-2-Spike-Protein (SP) and implant metal(s), respectively.

Methods

Data were retrospectively reviewed from three independent groups: unvaccinated COVID-19 adults (n = 23); fully COVID-19 vaccinated adults (n = 35); unvaccinated, painful aseptic implant patients with history of metal allergy (n = 98). Standard in vitro LTT for SP and implant metal(s) (nickel, cobalt) were performed and rated as negative (stimulation index [SI]<2), mild (SI ≥ 2), positive (SI ≥ 4–15), and high sensitization (SI > 15) adaptive immune responses to tested antigen.

Results

Overall, 17/23 (74%) of unvaccinated adults showed negative to mild LTT ranges, and 35/35 (100%) of vaccinated showed mild to positive LTT ranges to SP. Vaccinated individuals showed significantly higher median SI (16.1) to SP than unvaccinated (median SI, 1.7; P < 0.0001). Most vaccinated adults (94%) showed a lymphocyte SI > 4 to SP, establishing LTT SI ≥ 4 with >90% sensitivity for diagnosing effective COVID-19 adaptive immune responses. Significantly fewer painful orthopaedic patients (41%) showed comparable elevated levels of lymphocyte metal sensitivity at SI ≥ 4 compared to vaccinated group (P < 0.0001).

Conclusions

Vaccinated adults showed significantly higher lymphocyte SI to SP than unvaccinated indicating that SI ranges ≥4 should be set as unequivocally diagnostic of LTT-positive adaptive immune responses to tested antigen. This analysis supports using higher LTT SI ranges (SI ≥ 4) in diagnosing clinical orthopaedic-related Type IV metal-hypersensitivity responses among orthopaedic patients.

Keywords List: Antibodies, COVID-19, LTT, Lymphocytes, Metal sensitivity, Pain, Stimulation index, Total joint arthroplasty

1. Introduction

Local innate and adaptive immune responses to metal implant wear and corrosion products have been extensively characterized1, 2, 3, 4, 5 and are considered clinically important by the US Food and Drug Administration.6 All orthopaedic metals corrode through passive and active degradation mechanisms releasing ions and metal particles.7,8 Although metal ions and particles are not classified as sensitizers, they may activate the adaptive immune system by forming complexes with native proteins.9,10 Additionally, wide ranges of adaptive immune responses exist to metal degradation products (pre-disposition and sensitization) across individuals and orthopaedic cohorts11, 12, 13; however, 10–15% of the general population have a documented adaptive immune response to common metal sensitizers such as nickel and cobalt.5 Further, previous investigations showed a high correlation between an aseptic idiopathic painful total joint arthroplasty (TJA) and measured metal reactivity immune responses with >60% incidence of metal sensitivity among patients with certain implant types.14,15

Lymphocyte transformation testing (LTT), lymphocyte proliferation test, or lymphocyte stimulation test is a quantitative in vitro functional immune test that measures cellular adaptive immune responses to antigens, mitogens, and haptens (eg, metals, drugs).16,17 The use of LTT for drug allergy/hypersensitivity diagnosis was first used in the 1960s16,18, 19, 20 and advanced methodologically in the 1970s when the International Union of Immunological Societies recommended measuring the incorporation of 3H-thymidine for LTT.21 Drug- and metal-hypersensitivity LTT are based on detecting drug- or metal-hapten protein complex activation of specific memory T-cells in the peripheral blood of a previously sensitized individual, respectively. Historically, when lymphocyte proliferation exceeded 2-fold that of untreated controls (Stimulation Index [SI] >2), it was indicative of reactivity/sensitivity to a test agent.16 However, this reactivity/sensitivity value has been reset to higher values by some studies with an SI ≥ 3 for Beta-lactam antibiotics, including penicillin, cephalosporins, and related compounds.22

It remains uncertain what levels of metal sensitivity responses negatively impact implant performance. This assessment is complicated by individual variability in adaptive immune response and consequent downstream bioreactivity (ie, tissue damage, pain, angiogenesis). The immune system provides highly efficient and redundant protective adaptive immunity through sensitization and memory responses, and this phenomenon is used for vaccination technologies against known viral antigens. However, orthopaedic patients may have preexisting or developed metal-related adaptive immune responses (metal-sensitized TJA patients) that cannot be endogenously or exogenously mitigated, potentially resulting in a continuous inflammatory cascade.23,24 For metal-sensitized TJA patients, central questions include what level of measurable lymphocyte responses to implant metal(s) is clinically meaningful, and what level indicates detrimental adaptive immune responses leading to peri-implant tissue inflammation and untoward clinical outcomes?

The COVID-19 pandemic afforded a unique opportunity to understand what ranges of LTT responses (sensitization and memory recall to a novel antigen) are clinically meaningful. The COVID-19 coronavirus resulted in the manufacturing of highly effective vaccines, which induce productive and robust adaptive immune responses with 95% effectiveness in preventing severe disease among fully vaccinated individuals.25,26 This study compared lymphocyte sensitization responses to SARS-CoV-2-Spike-Protein (vaccine target) with implant metal-induced reactivity among unvaccinated and fully vaccinated COVID-19 adults vs patients with an aseptic painful TJA, respectively. Are LTT reactivity ranges comparable between the two types of immunogens? We hypothesized that fully COVID-19 vaccinated individuals will exhibit lymphocyte activation levels to the SARS-CoV-2-Spike-Protein at similar SI levels as that established for metal-reactivity (ie, SI > 2 by in vitro LTT assay).11, 12, 13 We used retrospective de-identified data from all-comer pre-operative and post-operative orthopaedic adults tested for adaptive immune responses by LTT to SARS-CoV-2-Spike-Protein or implant metal(s) to test this hypothesis. Lymphocyte proliferation responses to SARS-CoV-2-Spike-Protein or implant metal(s) were compared in COVID-19 unvaccinated and vaccinated adults vs patients with a painful aseptic TJA with a history of metal allergy, respectively.

2. Materials and methods

2.1. Study design and participants

Blinded, de-identified data were retrospectively reviewed from N = 156 all-comer orthopaedic adults who were referred for metal sensitivity testing either before primary or revision total joint arthroplasty (TJA [<1% of total annual TJAs performed in the United States are referred for metal-LTT]) under approval from Rush University's institutional review board (11081202-IRB01). Of the 156 adults, n = 23 were independently assigned to Group 1, COVID-19 unvaccinated adults (including adults with no implant, n = 11 [48%]; adults with a total knee arthroplasty [TKA] with no self-reported implant-referable pain, n = 12 [52%]); n = 35 were assigned into Group 2, self-reported Pfizer or Moderna fully COVID-19 vaccinated adults (including adults with no implant, n = 20 [57%]; adults with a TKA with no self-reported implant-referable pain, n = 15 [43%]; Table 1); n = 98 were assigned into Group 3, unvaccinated patients with an aseptic painful implant who have a self-reported history of metal allergy (adults with a TKA with a mean self-reported pain score of 6.3, n = 98 [100%]; Table 2). Patients in Group 3 were randomly selected at the time of analysis as a reference group to compare in vitro LTT SI to implant metal(s) vs COVID-19-Spike-Protein from Group 1 and Group 2 to evaluate clinically meaningful orthopaedic levels of metal-reactivity. Implant-referable pain for Group 3 was self-reported using the established visual analog scale (VAS) and scored from 0 to 10, with 0 indicating no pain and 10 indicating the worst possible pain.27, 28, 29, 30 Information including product name, manufacturer, and material of orthopaedic implant was not collected nor provided by participants.

Table 1.

Group 1 and group 2 patient demographics and clinical characteristics.

Characteristic Group Unvaccinated COVID-19 (n = 23)
 Group 2 Fully Vaccinated COVID-19 (n = 35)
Unvaccinated COVID-19 (n = 23) % Fully Vaccinated COVID-19 (n = 35) %
Mean (min–max) age, y 68 (53–85) – 68 (38–85) –
Gender, n
Women 18 78% 26 75%
Men 5 22% 9 25%
Implant, na
None 11 48% 20 57%
TKAb 12 52% 15 43%
Mean (min–max) age of implant, y 4.8 (1–10) – 3.9 (1–10) –
Open in a new tab

TKA, total knee arthroplasty.

a

Information including product name, manufacturer, and material of orthopaedic implant were not collected nor provided by participants.

b

Adults with a TKA in group 1 or group 2 reported no implant-referable pain.

Table 2.

Group 3 demographics and clinical characteristics of patients with a painful aseptic total joint arthroplasty and a history of metal allergya.

Characteristic Group 3 Unvaccinated, Painful TJA and Metal Allergy History (n = 98)
Painful TJA and Metal Allergy History (n = 98) %
Mean (min–max) age, y 63 (33–84) –
Gender, n
Women 95 97%
Men 3 3%
Implant, nb
TKA 98 100%
Mean (min–max) age of implant, y 1.52, (1–2) –
Mean (min–max) pain levelc 6.3 (0–10) –
Open in a new tab

LTT, lymphocyte transformation test; TJA, total joint arthroplasty; TKA, total knee arthroplasty.

a

Patients with an aseptic painful TJA and history of metal allergy were randomly selected at time of analysis as a reference group to compare in vitro LTT stimulation index to implant metal(s) vs COVID-19-Spike-Protein antigen among adults who were unvaccinated or fully COVID-19 vaccinated to evaluate clinically important orthopaedic levels of metal reactivity.

b

Information including product name, manufacturer, and material of orthopaedic implant were not collected nor provided by participants.

c

Pain level was self-reported using the established visual analog scale scored from 0 to 10.

2.2. Enzyme-linked immunoassays

Serum samples were collected from whole blood from all participants (N = 156), separated via centrifugation and stored at −80 °C until analysis. Qualitative serological IgG sandwich enzyme-linked immunoassays (ELISA) for SARS-CoV-2-Spike-Protein were performed following manufacturer's protocol using provided buffers and protocols (REF KTR-1032 RUO; Epitope Diagnostics, CA). Briefly, assay controls and human sera samples diluted 1:100 using manufacturer provided IgG negative/positive control and sample diluent, respectively, were added to the provided COVID-19 antigen coated 96-well microplate and incubated at room temperature (RT) for 30 minutes. Subsequently, the microplate was washed 5 times to remove unbound protein matrix followed by the addition of horseradish peroxidase (HRP) labeled polyclonal goat anti-IgG tracer antibody to each well, allowing the formation of an immunocomplex of “COVID-19 recombinant antigen–human anti-COVID-19 IgG antibody-HRP labeled anti-human IgG tracer antibody” if there is specific coronavirus IgG antibody present in the tested sample. Post 30-minute incubation at RT, microplate was washed 5 times removing unbound tracer antibody. HRP-labeled tracer antibody bound to the well was incubated with a provided substrate solution for 20 minutes at RT followed by the addition of provided stop solution to each well. The absorbance was measured in a Beckman Biomek spectrophotometric microplate reader at 450 nm. The enzymatic activity of the tracer antibody bound to the anti-COVID-19 IgG in the microtiter well is proportional to the amount of the anti-COVID-19 IgG antibody level in the tested sample. All samples were assayed in duplicate. Results were calculated per manufacturer's instructions. Briefly, the mean absorbance value of the negative control (xNC) was determined, and negative and positive cutoffs were calculated using provided formulas (0.9 X [xNC + 0.18], 1.1 X [xNC + 0.18], respectively). Results of tested sample were interpreted by comparing the optical density to manufacturer's provided criteria (negative interpretation, measured value ≤ negative cutoff; positive interpretation, measured value ≥ positive cutoff).

2.3. Sample collection and lymphocyte transformation test

Whole blood was collected by venipuncture from adults referred from healthcare facilities representative of 18 states in the United States using a specialized blood collection kit to ensure quality of blood draw supplies and sample temperature stability during transport (Orthopaedic Analysis LLC, Chicago, IL) within 24 h of initial collection. The process of LTT testing involved using peripheral blood mononuclear cells (PBMCs), which include 70–90% lymphocytes collected from 30 mL of peripheral blood by Ficol gradient separation. Collected PBMCs (white buffy coat) were washed in sterile 1X Phosphate Buffered Saline (PBS) and re-suspended in RPMI-1640 Medium with 10% autologous serum and cultured with 5ug/ml of SARS-CoV-2-Spike-Protein, target antigen of the first generation of the Pfizer BioNTech and Moderna vaccines (measured in unvaccinated and vaccinated adults, Group 1 and Group 2, respectively, to establish LTT–SI ranges to the original SARS-CoV-2-Spike-Protein [catalog number 130-127-041, Miltenyi Biotec, CA]) or 5ug/ml of SARS-CoV-2-Spike-Protein Omicron variant (measured in a small subset of fully first-generation vaccinated adults from Group 2—validating the specificity of LTT–SI to the target antigen of the first generation vaccine [catalog number 130-132-810, Miltenyi Biotec, CA]) or with the two most common implant-alloy metals, 0.1 mM NiCl2 and 0.01 mM CoCl2 (measured in Group 3, unvaccinated adults with an aseptic painful TJA), at 5% CO2 and 37 °C for 6 days. Previous studies showed that the selected metal ion concentrations used in this analysis are non-toxic (eg, do not induce DNA damage and cell death)10,10,31,31, 31, 32, 33, 34 within the limited context of metal–LTT testing; only the metal that the patient showed the greatest SI value was used in this analysis to represent worst-case clinical metal sensitivity. 3H Thymidine is added on day 5 of culture. 3H thymidine incorporation in unchallenged (control) and SARS-CoV-2-Spike-Protein or metal-treated PBMCs are analyzed using a beta scintillation counter on day 6. An SI of reactivity is calculated by dividing scintillation counts per minute of SARS-CoV-2-Spike-Protein or metal-challenged cells over untreated controls, and normalized SIs for all participants were used to account for variation in lymphocyte activation levels across individuals (ie, control-to-antigen ratios). LTT results were designated as no sensitization (SI < 2), mild sensitization (SI ≥ 2), positive sensitization (SI ≥ 4–15), and high sensitization (SI > 15).

2.4. Statistical analysis

The D'Agostino and Pearson omnibus normality test (alpha = 0.05) was performed to determine normality for each data set. All data was non-normal distributed, and Mann-Whitney was used for comparing two groups and Kruskal-Wallis for comparing more than two data sets for all non-normal distributed data. All statistical analysis was performed using Prism 8.0 program (GraphPad, San Diego, CA). Statistical difference was considered significant at P ≤ 0.05.

3. Results

3.1. COVID-19 IgG antibody titers

Quantifying and verifying COVID-19 vaccination status and efficacy was tested via humoral immune antibody analysis. Blood serum from adults in Group 1, unvaccinated, and Group 2, fully vaccinated, was analyzed for IgG antibodies specific for the Spike-Protein of the SARS-CoV-2 virus. Fully vaccinated adults in Group 2 (n = 35) showed an antibody positivity rate (>60U/ml) of 97% (Fig. 1a) and a median titer level of 258 U/ml of IgG specific for SARS-CoV-2-Spike-Protein (Fig. 1b). None of the individuals in Group 1 (n = 23) showed increased antibodies to SARS-CoV-2-Spike-Protein. Only 1/35 vaccinated adult in Group 2 demonstrated non-elevated IgG levels post-vaccination. Conversely, unvaccinated adults exhibited an antibody positivity rate of 0% (Fig. 1a) and a median titer level of 7.2 U/ml of SARS-CoV-2-Spike-Protein IgG (P > 0.0001; Fig. 1b).

Fig. 1.

Fig. 1

Open in a new tab

A. Rate of positive COVID-19 IgG antibody titers. B. Median COVID-19 IgG antibody titers.

3.2. Lymphocyte sensitization to COVID-19-Spike-Protein vs metal implant sensitizers

Most unvaccinated adults (74%) showed none to only mild sensitization to SARS-CoV-2-Spike-Protein with a median lymphocyte SI of 1.7 (Fig. 2a). Notably, 100% of fully vaccinated adults showed positive lymphocyte responses (SI > 2) to SARS-CoV-2-Spike-Protein ranging from mild to high sensitization (>50th percentile). Vaccinated adults also showed a significantly higher median SI of 16.1 than the unvaccinated group to SARS-CoV-2-Spike-Protein (P < 0.0001; Fig. 2a and b). Unvaccinated patients with an aseptic painful TJA with a history of metal allergy showed a median lymphocyte SI of 4.1 to nickel and/or cobalt, ranging from mild (16%) to positive (29%) and high (12%; Fig. 2). Overall, a significantly greater proportion of vaccinated adults showed higher lymphocyte reactivity at SI > 4 to SARS-CoV-2-Spike-Protein (94%) than patients with a painful TJA to implant metal(s) at a SI > 4 (41%; P < 0.0001; Fig. 2a and b).

Fig. 2.

Fig. 2

Open in a new tab

A. Patient-Specific Lymphocyte Stimulation Index to SARS-CoV-2-Spike-Protein or Implant Metal(s) by LTT. B. Median Lymphocyte Stimulation Index Response to SARS-CoV-2-Spike-Protein or Implant Metal(s) by LTT per Patient Group.

C, challenge agent; G, group; LTT, lymphocyte transformation test.

3.3. Lymphocyte sensitization to the omicron variant of COVID-19

To further demonstrate the specificity of LTT to the original SARS-CoV-2-Spike-Protein in first-generation vaccines, LTT to the Omicron COVID-19 variant was performed in a small subset of fully first-generation vaccinated individuals (n = 6). Vaccinated adults showed significantly higher LTT-SI to the original COVID-19 strain vs Omicron (23.7 vs 8.9, respectively; P < 0.015; Fig. 3). Nonetheless, LTT-SI ranges to Omicron were still classified as highly elevated ranges of adaptive immune memory responses (SI > 8).

Fig. 3.

Fig. 3

Open in a new tab

Mean Lymphocyte Stimulation Index Response to the Original Strain of COVID-19, SARS-CoV-2-Spike-Protein, or the COVID-19 Omicron Variant Among Unvaccinated and Vaccinated Adults by LTT.

C, challenge agent; G, group; LTT, lymphocyte transformation test.

4. Discussion

This is the first study comparing lymphocyte sensitization profiles to COVID-19-Spike-Protein post-vaccination and metal-reactivity among orthopaedic cohorts. Of patients with an aseptic painful TJA and a history of metal allergy, 41% exhibited lymphocyte sensitization to implant metal(s) at comparable SI ranges to the COVID-19 fully vaccinated group. These results only partially support our hypothesis that levels used to diagnose metal-lymphocyte sensitization by LTT (ie, SI > 2) are comparable to vaccinated adults’ LTT response to SARS-CoV-2-Spike-Protein, indicating that a new threshold is needed to diagnose ongoing type-IV lymphocyte response to metal-protein antigens. Base levels of acute lymphocyte sensitization acquired by COVID-19 vaccination are 95% effective in protecting against severe disease from the original strain of SARS-CoV-2-Spike-Protein.25,26 As such, more acute delayed-type hypersensitivity (DTH) response is used here as a new predicate standard comparator for more chronic metal-DTH responses that are elevated to clinically unambiguous levels of reactivity.

The level of lymphocyte sensitization to SARS-CoV-2-Spike-Protein by LTT derives from in vivo sensitization by vaccines that effectively induce a humoral immune response (B-cell derived antibodies) secondary to acute DTH responses that are requisite for downstream antibody responses and have been extensively reported on.35, 36, 37, 38 T-cell activation (as measured here) is required for B-cell class switching from naïve to antibody-producing cells. This response is further characterized here and provides novel information on the levels of a robust lymphocyte response shown by the vaccinated group to the original COVID-19-Spike-Protein corroborated with detectable elevated COVID-19 antibody titer levels. Robust immune memory reactivity to viral antigens like SARS-CoV-2-Spike-Protein is highly desirable and the goal of vaccination; however, the same level of immune reactivity against self-antigens or implanted biomaterials may lead to adverse clinical outcomes. These data indicate what threshold level(s) of lymphocyte sensitization may be more clinically meaningful in predicting aggressive immune responses leading to adverse biological and local tissue responses to biomaterial degradation products in the peri-implant interface.

These results of the current study also provide clinical insight for the provider and patient in the ability to better rely evidentially on deterministic levels of lymphocyte reactivity to metal by showing corroboration that metal-LTT SI ≥ 4 is clinically meaningful when comparing metal responses to desired levels of elevated lymphocyte reactivity to the original COVID-19 Spike-Protein antigen. This evidence-supported threshold can more reliably lead to actionable treatment with favorable clinical outcomes. Indeed, a retrospective study of patients with an aseptic painful TKA, with 85% (39/46) testing positive for metal sensitivity via LTT (50% with a SI ≥ 4), showed that a significant proportion of these patients who underwent TKA revision to a hypoallergenic component experienced significantly less pain (P < 0.001) and reported high satisfaction post-revision surgery.11 Additionally, a small subset of patients with an aseptic painful TKA who tested negative for metal sensitivity and underwent revision TKA (15%; 7/46) did not show significant improvement in postoperative pain scores.11 Notably, these results changed the senior authors’ clinical practice, including using metal-LTT when infection, instability, and malalignment are ruled out for patients with a painful TKA; if a patient with a painful TKA was confirmed metal sensitive via LTT, revision to a hypoallergenic type implant was recommended; and primary TKA patients with a known history of metal sensitivity, a hypoallergenic type component was recommended to optimally prevent any potential future complications such as metal hypersensitivity and morbidity of revision surgery.11 Overall, these results suggest that clinicians should be informed preoperatively if a patient has a known history of a metal allergy. If so, the patient will likely benefit from a metal-LTT. If the patient has an SI ≥ 4, it is recommended that additional proactive steps are taken to minimize reactivity-related risk (eg, switching to a hypoallergenic-type implant). Importantly, at this level of reactivity, the benefit/risk profile of non-standard orthopaedic procedures/implants designed to lessen metal exposure/release over the long term is warranted to minimize both the risk of potential adverse aseptic immune responses and early revision surgery. While an LTT result of a SI ≥ 2 to an implant metal(s) may indicate metal reactivity, this does not necessarily warrant the same degree of clinical consideration (eg, altered surgical approach); however, this elevated response may be used as an indication of elevated metal reactivity if strongly supported by other clinical and patient factors (eg, reactions to dermal metal exposure, peri-implant aseptic inflammation, etc). Additionally, in agreement with the Zondervan et al. study,11 these results indicate that patients receiving a metal-LTT post-operatively showing an SI ≥ 4 benefit from revision to a hypoallergenic-type implant with decreased unwanted elevated immune responses to released implant metals.

The utility of metal LTT as a downstream functional, highly quantitative, and easily replicated immune test is made more critical given the lack of other modes histologically of detecting pathological adaptive immune responses to implant metals. Some studies have correlated LTT with local histopathological evidence of reactivity,4,12,23,39,40 while others fail to link LTT metal-reactivity levels with histopathological scores of local lymphocyte accumulations.41, 42, 43 This discrepancy is expected due to the variability associated with the statistical probability of feature identification when non-homogeneous pathology exists in peri-implant tissue44, 45, 46 and/or the entire tissue is not sampled and analyzed histologically,47,48 ie, absence of evidence is not evidence of absence, and consequently conclusions may be limited when analyzing peri-implant tissues.42,49 Despite this hurdle, some studies have demonstrated the correlation between LTT and local histological lymphocyte accumulation/activation.41,42,50 The current study provides new immune-based predicate guidelines for evidential-based decision-making when establishing clinically determinant benchmarks of what is known to constitute clinical reactivity levels for exuberant/effective adaptive immune responses.

LTT, like all in vitro assays, is an indirect diagnostic tool for assessing metal-hypersensitivity and not for determining implant-referable pathology since tissue responses to immune responses also vary individually. For example, two age- and sex-matched pre-operative orthopaedic patients with identically high metal-reactivity via LTT will not likely experience similar implant immune pathology if one receives potent anti-inflammatory medications for rheumatoid arthritis and the other has osteoarthritis. Similar to drug allergy LTT assessment, diagnosis of metal-hypersensitivity to implant alloys depends on four criteria: i) classification of the clinical phenotype, ii) medical history, iii) in vitro tests, and if applicable, iv) in vivo tests or provocation tests.16 Thus, clinician evaluation post-LTT testing is central until more advanced methods (eg, radiological) and algorithms are developed using robust patient-specific data.

The direct medical risk of metal-LTT is extremely low and limited to that of a blood-draw, especially compared to patch testing, which carries the risk of sensitizing a patient to an implanted metal. LTT has a high potential for medical benefit including avoiding or delaying revision surgery and reducing health care costs by avoiding a person-specific highly reactive implant alloy/metal or implant type (eg, metal-on-metal TJA). Low-risk surveillance actions indicated by high reactivity test results include earlier/more frequent follow-ups post-surgery, avoidance of specific metal alloy components where alternatives are available and provides evidence for anti-inflammatory therapy for aseptic peri-implant inflammation. Clinical accommodation of these indicated surveillance measures carries little increased medical risk but is associated with short-term healthcare costs and may require justification by LTT or other testing. Complete avoidance of an implant or joint arthroplasty based on high LTT reactivity is not currently evidentially supported. The health benefits of restored mobility/function provided by a TJA remain paramount.51,52

Limitations of this retrospective study include a lack of data on implant specifications including product name, manufacturer, material, long-term implant function/history, and if there is an association between metal-reactivity and implant failure metrics. Additionally, these analyses cannot establish if the prevalence of metal sensitivity via LTT among patients with an aseptic painful TJA is because of a pre-existing condition preoperatively or if metal reactivity was induced post-operatively. There is a dearth of preoperative LTT information with requisite group numbers related to medium- and long-term implant clinical performance (ie, ≥5 and ≥ 10 years). Fortunately for patients with implant-related metal sensitivity, it is a subtle phenomenon as evidenced by the high overall success rates of THA and TKA established by national registries, with failure rates of 3–6% at 5 years and 8–12% at 10 years,53,54 with approximately 50% of these failures attributable to biological or possibly immune-related outcome measures55 (eg, aseptic loosening, osteolysis, arthrofibrosis, and idiopathic pain). Thus, to determine the odds ratio or risk of implant failure preoperatively due to metal sensitivity, requires a large THA/TKA group predetermined to have metal reactivity (n ≥ 100) and followed for a minimum of ≥5 and ≥ 10 years for comparison to well-established registry data (or controls, negative for metal reactivity). Notably, shorter group follow-up times (eg, <5 years) are inappropriate because of the subtle nature of metal DTH responses in vivo, as evidenced by low overall failure rates of THAs and TKAs.53,54 Alarmingly, these studies would only require 3–6% failure at 5 years and 8–12% failure at 10 years attributable to biological etiology to produce an odds ratio of 2, ie, 100% increased risk for failure if preoperatively determined to be metal sensitive. Ultimately, this odds ratio will depend on what threshold is used to diagnose positive metal sensitivity and highlights the utility of the current study. This odds ratio will likely exceed 2 given that 25–50% of failing TJAs requiring a revision show diagnostically identifiable metal sensitivity.13,15,56 Moreover, although the direct comparison of COVID-19 LTT results in vaccinated and unvaccinated vs their respective matched metal-LTT results was not performed in this study, this was not the aim of this investigation, and findings are limited to the comparison of proposed predicate standard clinical thresholds for determining elevated LTT reactivity indicative of elevated/robust adaptive immune responses.

5. Conclusion

In this retrospective analysis, we found that levels of lymphocyte reactivity via LTT (SI ≥ 4) correspond to robust, specific immune protection against an unwanted pathogen such as COVID-19. These data suggest that an LTT result of a SI ≥ 4 to an implant material is an alarm level warranting clinical proactive measures to minimize the risk of possible adverse aseptic biological responses and early revision surgery (eg, altered follow-up dates, use of less metal-releasing implants, using hypoallergenic-type implants, etc). These results do not refute the utility of preoperatively testing patients with a known history of metal sensitivity to diagnose at-risk patients and methodologically support using these clinically validated thresholds to determine excessive metal reactivity responses post-operatively. Prospective medium- (5-year) and long-term (10-year) clinical studies are needed to determine the combination of preoperative patient-specific factors combined with metal-LTT responses that best predict those at risk of early implant failure. Additionally, these results demonstrate that 33% of patients with an aseptic painful TJA with a history of metal sensitivity exhibited no lymphocyte reactivity to implant metal(s), underscoring that not all arthroplasty patients with aseptic painful implants experience metal-hypersensitivity responses. These data are consistent with previous studies showing >50% of patients with an aseptic painful TJA exhibit metal sensitivity by LTT thresholds by either cause or effect; however, prior LTT stimulation indices may need to be reevaluated considering the new threshold levels reported here.5,11, 12, 13

In summary, the threshold LTT levels established by sensitization levels to COVID-19 vaccination (ie, SI ≥ 4) provide new alarm clinical thresholds for metal-LTT levels and adjusted benchmarks for predicting poor implant performance among metal-sensitive orthopaedic populations.

Funding

Funding provided by Orthopedic Analysis, LLC; Rush University Medical Center, Department of Orthopaedic Surgery.

Informed consent

The study involved de-identified subject data and informed consent process in accordance with 45 CFR 46.101(b) Categories of Exempt Human Subjects Research, was performed.

Institutional ethical committee approval

IRB approval from Rush University Medical Center (11081202-IRB01).

CRediT authorship contribution statement

Marco S. Caicedo: Conception and design of the work, immunological assay setup and design. Vianey Flores: Acquisition and analysis of data, lymphocyte transformation testing, ELISAs. Rochelle Siapno: Acquisition and analysis of data. Michael Crosby: Acquisition and analysis of data, lymphocyte transformation testing, ELISAs. Lauryn A. Samelko: Interpretation of data, drafting and revision of work. Joshua J. Jacobs: Interpretation of data, revision of work. Nadim J. Hallab: Interpretation of data, drafting and revision of work. All authors had access to the data, participated in the interpretation of the data, drafted, and revised the manuscript, and approved the final version before submission.

Declaration of competing interest

MSC, VF, RS, MC, and LS are employees of Orthopedic Analysis, LLC. NJH is an employee of Orthopedic Analysis, LLC; has stock or stock options in Orthopedic Analysis, LLC.

Acknowledgment

None.

References

  • 1.Cobelli N., Scharf B., Crisi G.M., Hardin J., Santambrogio L. Mediators of the inflammatory response to joint replacement devices. Nat Rev Rheumatol. 2011 Sep 6;7(10):600–608. doi: 10.1038/nrrheum.2011.128. [DOI] [PubMed] [Google Scholar]
  • 2.Caicedo M.S., Desai R., McAllister K., Reddy A., Jacobs J.J., Hallab N.J. Soluble and particulate Co-Cr-Mo alloy implant metals activate the inflammasome danger signaling pathway in human macrophages: a novel mechanism for implant debris reactivity. J Orthop Res. 2009 Jul;27(7):847–854. doi: 10.1002/jor.20826. [DOI] [PubMed] [Google Scholar]
  • 3.Samelko L., Caicedo M.S., Jacobs J., Hallab N.J. Transition from metal-DTH resistance to susceptibility is facilitated by NLRP3 inflammasome signaling induced Th17 reactivity: implications for orthopedic implants. PLoS One. 2019;14(1) doi: 10.1371/journal.pone.0210336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Willert H.G., Buchhorn G.H., Fayyazi A., et al. Metal-on-metal bearings and hypersensitivity in patients with artificial hip joints. A clinical and histomorphological study. J Bone Joint Surg Am. 2005 Jan;87(1):28–36. doi: 10.2106/JBJS.A.02039pp. [DOI] [PubMed] [Google Scholar]
  • 5.Hallab N., Merritt K., Jacobs J.J. Metal sensitivity in patients with orthopaedic implants. J Bone Joint Surg Am. 2001 Mar;83(3):428–436. doi: 10.2106/00004623-200103000-00017. [DOI] [PubMed] [Google Scholar]
  • 6.Buvanendran A., Kroin J.S., Berger R.A., et al. Upregulation of prostaglandin E2 and interleukins in the central nervous system and peripheral tissue during and after surgery in humans. Anesthesiology. 2006 Mar;104(3):403–410. doi: 10.1097/00000542-200603000-00005. [DOI] [PubMed] [Google Scholar]
  • 7.Black J., Oppenheimer P., Morris D.M., Peduto A.M., Clark C.C. Release of corrosion products by F-75 cobalt base alloy in the rat. III: effects of a carbon surface coating. J Biomed Mater Res. 1987 Oct;21(10):1213–1230. doi: 10.1002/jbm.820211005. [DOI] [PubMed] [Google Scholar]
  • 8.Jacobs J.J., Skipor A.K., Patterson L.M., et al. Metal release in patients who have had a primary total hip arthroplasty. A prospective, controlled, longitudinal study. J Bone Joint Surg Am. 1998 Oct;80(10):1447–1458. doi: 10.2106/00004623-199810000-00006. [DOI] [PubMed] [Google Scholar]
  • 9.Yang J., Merritt K. Detection of antibodies against corrosion products in patients after Co-Cr total joint replacements. J Biomed Mater Res. 1994 Nov;28(11):1249–1258. doi: 10.1002/jbm.820281102. [DOI] [PubMed] [Google Scholar]
  • 10.Hallab N.J., Anderson S., Caicedo M., Brasher A., Mikecz K., Jacobs J.J. Effects of soluble metals on human peri-implant cells. J Biomed Mater Res. 2005 Jul 1;74(1):124–140. doi: 10.1002/jbm.a.30345. [DOI] [PubMed] [Google Scholar]
  • 11.Zondervan R.L., Vaux J.J., Blackmer M.J., Brazier B.G., Taunt C.J., Jr. Improved outcomes in patients with positive metal sensitivity following revision total knee arthroplasty. J Orthop Surg Res. 2019 Jun 17;14(1):182. doi: 10.1186/s13018-019-1228-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Lionberger D.R., Samorajski J., Wilson C.D., Rivera A. What role does metal allergy sensitization play in total knee arthroplasty revision? J Exp Orthop. 2018 Aug 14;5(1):30. doi: 10.1186/s40634-018-0146-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Caicedo M.S., Solver E., Coleman L., Jacobs J.J., Hallab N.J. Females with unexplained joint pain following total joint arthroplasty exhibit a higher rate and severity of hypersensitivity to implant metals compared with males: implications of sex-based bioreactivity differences. J Bone Joint Surg Am. 2017 Apr 19;99(8):621–628. doi: 10.2106/JBJS.16.00720. [DOI] [PubMed] [Google Scholar]
  • 14.Kwon Y.M., Glyn-Jones S., Simpson D.J., et al. Analysis of wear of retrieved metal-on-metal hip resurfacing implants revised due to pseudotumours. J Bone Joint Surg Br. 2010 Mar;92(3):356–361. doi: 10.1302/0301-620X.92B3.23281. [DOI] [PubMed] [Google Scholar]
  • 15.Hallab N.J., Caicedo M., McAllister K., Skipor A., Amstutz H., Jacobs J.J. Asymptomatic prospective and retrospective cohorts with metal-on-metal hip arthroplasty indicate acquired lymphocyte reactivity varies with metal ion levels on a group basis. J Orthop Res. 2013 Feb;31(2):173–182. doi: 10.1002/jor.22214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Sachs B., Fatangare A., Sickmann A., Glassner A. Lymphocyte transformation test: history and current approaches. J Immunol Methods. 2021 Jun;493 doi: 10.1016/j.jim.2021.113036. [DOI] [PubMed] [Google Scholar]
  • 17.Fatangare A., Glassner A., Sachs B., Sickmann A. Future perspectives on in-vitro diagnosis of drug allergy by the lymphocyte transformation test. J Immunol Methods. 2021 Aug;495 doi: 10.1016/j.jim.2021.113072. [DOI] [PubMed] [Google Scholar]
  • 18.Summer B., Stander S., Kapp F., Thomas P. [Role of the lymphocyte transformation test in the evaluation of metal sensitization] Hautarzt. 2016 May;67(5):380–384. doi: 10.1007/s00105-016-3791-5. [DOI] [PubMed] [Google Scholar]
  • 19.Sarkany I. Lymphocyte transformation in drug hypersensitivity. Lancet. 1967 Apr 8;1(7493):743–745. doi: 10.1016/s0140-6736(67)91362-1. [DOI] [PubMed] [Google Scholar]
  • 20.Simon N., Dobozy A., Hunyadi J. [Significance of the lymphocyte transformation test in dermatology] Berufs-Dermatosen. 1970 Aug;18(4):189–219. [PubMed] [Google Scholar]
  • 21.immunology Clinical. Report of the committee of clinical immunology of the international union of immunological Societies (IUIS) Scand J Immunol. 1976;5(1-2):1–7. doi: 10.1111/j.1365-3083.1976.tb02984.x. [DOI] [PubMed] [Google Scholar]
  • 22.Pichler W.J., Tilch J. The lymphocyte transformation test in the diagnosis of drug hypersensitivity. Allergy. 2004 Aug;59(8):809–820. doi: 10.1111/j.1398-9995.2004.00547.x. [DOI] [PubMed] [Google Scholar]
  • 23.Davies A.P., Willert H.G., Campbell P.A., Learmonth I.D., Case C.P. An unusual lymphocytic perivascular infiltration in tissues around contemporary metal-on-metal joint replacements. J Bone Joint Surg Am. 2005 Jan;87(1):18–27. doi: 10.2106/JBJS.C.00949. [DOI] [PubMed] [Google Scholar]
  • 24.Campbell P., Ebramzadeh E., Nelson S., Takamura K., De S.K., Amstutz H.C. Histological features of pseudotumor-like tissues from metal-on-metal hips. Clin Orthop Relat Res. 2010 Sep;468(9):2321–2327. doi: 10.1007/s11999-010-1372-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Polack F.P., Thomas S.J., Kitchin N., et al. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. N Engl J Med. 2020 Dec 31;383(27):2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Thomas S.J., Moreira E.D., Jr., Kitchin N., et al. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine through 6 months. N Engl J Med. 2021 Nov 4;385(19):1761–1773. doi: 10.1056/NEJMoa2110345. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Cheng L., Fan W., Liu B., Wang X., Nie L. Th17 lymphocyte levels are higher in patients with ruptured than non-ruptured lumbar discs, and are correlated with pain intensity. Injury. 2013 Dec;44(12):1805–1810. doi: 10.1016/j.injury.2013.04.010. [DOI] [PubMed] [Google Scholar]
  • 28.Rojkovich B., Gibson T. Day and night pain measurement in rheumatoid arthritis. Ann Rheum Dis. 1998 Jul;57(7):434–436. doi: 10.1136/ard.57.7.434. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Breivik H., Borchgrevink P.C., Allen S.M., et al. Assessment of pain. Br J Anaesth. 2008 Jul;101(1):17–24. doi: 10.1093/bja/aen103. [DOI] [PubMed] [Google Scholar]
  • 30.Hawker G.A., Mian S., Kendzerska T., French M. Measures of adult pain: visual analog scale for pain (VAS pain), numeric rating scale for pain (NRS pain), McGill pain questionnaire (MPQ), short-form McGill pain questionnaire (SF-mpq), chronic pain grade scale (CPGS), short form-36 bodily pain scale (SF-36 BPS), and measure of intermittent and constant osteoarthritis pain (ICOAP) Arthritis Care Res. 2011 Nov;63(11):S240–S252. doi: 10.1002/acr.20543. [DOI] [PubMed] [Google Scholar]
  • 31.Hallab N.J., Caicedo M., Epstein R., McAllister K., Jacobs J.J. In vitro reactivity to implant metals demonstrates a person-dependent association with both T-cell and B-cell activation. J Biomed Mater Res. 2010 Feb;92(2):667–682. doi: 10.1002/jbm.a.32368. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Caicedo M., Jacobs J.J., Reddy A., Hallab N.J. Analysis of metal ion-induced DNA damage, apoptosis, and necrosis in human (Jurkat) T-cells demonstrates Ni2+ and V3+ are more toxic than other metals: Al3+, Be2+, Co2+, Cr3+, Cu2+, Fe3+, Mo5+, Nb5+, Zr2+ J Biomed Mater Res. 2008 Sep 15;86(4):905–913. doi: 10.1002/jbm.a.31789. [DOI] [PubMed] [Google Scholar]
  • 33.Hallab N.J., Anderson S., Stafford T., Glant T., Jacobs J.J. Lymphocyte responses in patients with total hip arthroplasty. J Orthop Res. 2005 Mar;23(2):384–391. doi: 10.1016/j.orthres.2004.09.001. [DOI] [PubMed] [Google Scholar]
  • 34.Hallab N.J., Caicedo M., Finnegan A., Jacobs J.J. Th1 type lymphocyte reactivity to metals in patients with total hip arthroplasty. J Orthop Surg Res. 2008 Feb 13;3:6. doi: 10.1186/1749-799X-3-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Rydyznski M.C., Ramirez S.I., Dan J.M., et al. Antigen-specific adaptive immunity to SARS-CoV-2 in acute COVID-19 and associations with age and disease severity. Cell. 2020 Nov 12;183(4):996–1012. doi: 10.1016/j.cell.2020.09.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Chen Z., John W.E. T cell responses in patients with COVID-19. Nat Rev Immunol. 2020 Sep;20(9):529–536. doi: 10.1038/s41577-020-0402-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Grifoni A., Weiskopf D., Ramirez S.I., et al. Targets of T Cell responses to SARS-CoV-2 coronavirus in humans with COVID-19 disease and unexposed individuals. Cell. 2020 Jun 25;181(7):1489–1501. doi: 10.1016/j.cell.2020.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Sattler A., Angermair S., Stockmann H., et al. SARS-CoV-2-specific T cell responses and correlations with COVID-19 patient predisposition. J Clin Invest. 2020 Dec 1;130(12):6477–6489. doi: 10.1172/JCI140965. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Lieberman E.G., Barrack R.L., Schmalzried T.P. Suspected metal allergy and femoral loosening after total knee arthroplasty: a diagnostic dilemma. Arthroplast Today. 2021 Feb;7:114–119. doi: 10.1016/j.artd.2020.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Thomas P., Braathen L.R., Dorig M., et al. Increased metal allergy in patients with failed metal-on-metal hip arthroplasty and peri-implant T-lymphocytic inflammation. Allergy. 2009 Aug;64(8):1157–1165. doi: 10.1111/j.1398-9995.2009.01966.x. [DOI] [PubMed] [Google Scholar]
  • 41.Malahias M.A., Bauer T.W., Manolopoulos P.P., Sculco P.K., Westrich G.H. Allergy testing has No correlation with intraoperative histopathology from revision total knee arthroplasty for implant-related metal allergy. J Knee Surg. 2021 May 1 doi: 10.1055/s-0041-1729618. [DOI] [PubMed] [Google Scholar]
  • 42.Schneiderman B.A., Yang S., Dipane M., Lu C., McPherson E.J., Schmalzried T.P. Periprosthetic tissue reaction independent of LTT result and implanted materials in total knee arthroplasty. J Arthroplasty. 2021 Jul;36(7):2480–2485. doi: 10.1016/j.arth.2021.02.036. [DOI] [PubMed] [Google Scholar]
  • 43.Yang S., Dipane M., Lu C.H., Schmalzried T.P., McPherson E.J. Lymphocyte transformation testing (LTT) in cases of pain following total knee arthroplasty: little relationship to histopathologic findings and revision outcomes. J Bone Joint Surg Am. 2019 Feb 6;101(3):257–264. doi: 10.2106/JBJS.18.00134. [DOI] [PubMed] [Google Scholar]
  • 44.Hallab N.J., Chan F.W., Harper M.L. Quantifying subtle but persistent peri-spine inflammation in vivo to submicron cobalt-chromium alloy particles. Eur Spine J. 2012 Dec;21(12):2649–2658. doi: 10.1007/s00586-012-2251-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Hallab N.J., Bao Q.B., Brown T. Assessment of epidural versus intradiscal biocompatibility of PEEK implant debris: an in vivo rabbit model. Eur Spine J. 2013 Dec;22(12):2740–2751. doi: 10.1007/s00586-013-2904-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Besusparis J., Plancoulaine B., Rasmusson A., et al. Impact of tissue sampling on accuracy of Ki67 immunohistochemistry evaluation in breast cancer. Diagn Pathol. 2016 Aug 30;11(1):82. doi: 10.1186/s13000-016-0525-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Tipaldi M.A., Ronconi E., Krokidis M.E., et al. Diagnostic yield of CT-guided lung biopsies: how can we limit negative sampling? Br J Radiol. 2022 Feb 1;95(1130) doi: 10.1259/bjr.20210434. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Furness P.N. Acp. Best practice no 160. Renal biopsy specimens. J Clin Pathol. 2000 Jun;53(6):433–438. doi: 10.1136/jcp.53.6.433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Campbell P., Park S.H., Ebramzadeh E. Semi-quantitative histology confirms that the macrophage is the predominant cell type in metal-on-metal hip tissues. J Orthop Res. 2022 Feb;40(2):387–395. doi: 10.1002/jor.25040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Keeling P., Schneiderman B.A., Lu C., Wilson M.L., Schmalzried T.P. Lymphocyte subset ratio cannot diagnose immune failure of a TKA. J Arthroplasty. 2022 Jul;37(7):1364–1368. doi: 10.1016/j.arth.2022.03.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Kreiner C.T., Nielsen T.H., Serena B.L. Role of income mobility for the measurement of inequality in life expectancy. Proc Natl Acad Sci U S A. 2018 Nov 13;115(46):11754–11759. doi: 10.1073/pnas.1811455115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Bushnik T., Tjepkema M., Martel L. Health-adjusted life expectancy in Canada. Health Rep. 2018 Apr 18;29(4):14–22. [PubMed] [Google Scholar]
  • 53.Labek G., Thaler M., Janda W., Agreiter M., Stockl B. Revision rates after total joint replacement: cumulative results from worldwide joint register datasets. J Bone Joint Surg Br. 2011 Mar;93(3):293–297. doi: 10.1302/0301-620X.93B3.25467. [DOI] [PubMed] [Google Scholar]
  • 54.Kurtz S.M., Ong K.L., Schmier J., et al. Future clinical and economic impact of revision total hip and knee arthroplasty. J Bone Joint Surg Am. 2007 Oct;89(Suppl 3):144–151. doi: 10.2106/JBJS.G.00587. [DOI] [PubMed] [Google Scholar]
  • 55.Brown M.L., Javidan P., Early S., Bugbee W. Evolving etiologies and rates of revision total knee arthroplasty: a 10-year institutional report. Arthroplasty. 2022 Aug 25;4(1):39. doi: 10.1186/s42836-022-00134-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Caicedo M.S., Solver E., Coleman L., Hallab N.J. Metal sensitivities among TJA patients with post-operative pain: indications for multi-metal LTT testing. J Long Term Eff Med Implants. 2014;24(1):37–44. doi: 10.1615/jlongtermeffmedimplants.2014010261. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Orthopaedics are provided here courtesy of Elsevier

RESOURCES