Abstract
Summary
Patients often do not know or understand their bone density test results, and pharmacological treatment rates are low. In a clinical trial of 7749 patients, we used a tailored patient-activation result letter accompanied by a bone health brochure to improve appropriate pharmacological treatment. Treatment rates, however, did not improve.
Introduction
Patients often do not know or understand their dual-energy x-ray absorptiometry (DXA) test results, which may lead to suboptimal care. We tested whether usual care augmented by a tailored patient-activation DXA result letter accompanied by an educational brochure would improve guideline-concordant pharmacological treatment compared to usual care only.
Methods
We conducted a randomized, controlled, double-blinded, pragmatic clinical trial at three health care centers in the USA. We randomized 7749 patients ≥50 years old and presenting for DXA between February 2012 and August 2014. The primary clinical endpoint at 12 and 52 weeks post-DXA was receiving guideline-concordant pharmacological treatment. We also examined four of the steps along the pathway from DXA testing to that clinical endpoint, including (1) receiving and (2) understanding their DXA results and (3) having subsequent contact with their provider and (4) discussing their results and options.
Results
Mean age was 66.6 years, 83.8 % were women, and 75.3 % were non-Hispanic whites. Intention-to-treat analyses revealed that guideline-concordant pharmacological treatment was not improved at either 12 weeks (65.1 vs. 64.3 %, p = 0.506) or 52 weeks (65.2 vs. 63.8 %, p = 0.250) post-DXA, even though patients in the intervention group were more likely (all p < 0.001) to recall receiving their DXA results letter at 12 weeks, correctly identify their results at 12 and 52 weeks, have contact with their provider at 52 weeks, and have discussed their results with their provider at 12 and 52 weeks.
Conclusion
A tailored DXA result letter and educational brochure failed to improve guideline-concordant care in patients who received DXA.
Keywords: Aging, Clinical trials, DXA, Osteoporosis, Screening
Introduction
The World Health Organization (WHO) and others define osteoporosis as “a progressive systemic skeletal disease characterized by low bone mass and microarchitectural deterioration of bone tissue, with a consequent increase in bone fragility and susceptibility to fracture” [1–5]. Because microarchitectural deterioration cannot be directly assessed clinically [6], screening for osteoporosis prior to a fracture and the diagnosis of osteoporosis after a fracture are based on bone mineral density (BMD) testing. BMD testing is accomplished primarily via dual-energy x-ray absorptiometry (DXA) [6]. Absolute fracture risk is calculated using the Fracture Risk Assessment Tool (FRAX, online v3.6) [7]. Current National Osteoporosis Foundation (NOF) guidelines [5] recommend that “post-menopausal women and men age 50 and older presenting with the following should be considered for [pharmacotherapy] treatment: hip or vertebral (clinically apparent or found on vertebral imaging) fracture …; T-score ≤−2.5 at the femoral neck, total hip, or lumbar spine …; [or] low bone mass (T-score between −1.0 and −2.5 at the femoral neck or lumbar spine) and a 10-year probability of hip fracture ≥3 % or a 10-year probability of any major osteoporosis-related fracture ≥20 % based on the U.S.-adapted WHO algorithm” (changes from the 2010 NOF guidelines [8] in effect when this study was conducted are shown in italics). It has been estimated that about 54 million adults in the USA had either osteoporosis or osteopenia (low bone mass) in 2010, with prevalence rates among those ≥50 years old of 10.3 % for osteoporosis and 43.9 % for osteopenia [9, 10]. In 2005, more than two million osteoporosis-related fractures occurred in the USA, 14 % of which were hip fractures, resulting in $17 billion in health care costs (72 % from the hip fractures) [7]. By 2025, the number of osteoporosis-related fractures in the USA will exceed three million, with associated health care costs of $25.3 billion [7]. Moreover, fractures, especially at the hip, also result in “premature mortality, loss of independence and function, [and] reduced quality of life” [11].
Accordingly, the NOF [5], the U.S. Preventive Services Task Force (USPSTF) [11], and Healthy People 2020 [12] all have objectives for decreasing the prevalence of osteoporosis and hip fractures (e.g., Healthy People 2020 Objectives AOCBC-10 and AOCBC-11, respectively). The three main strategies involve encouraging healthy behaviors (adequate calcium and vitamin D intake, weight-bearing and muscle-strengthening exercise, fall prevention, smoking cessation, and avoidance of excessive alcohol intake) [5] and increasing DXA screening and subsequent pharmacological treatment when appropriate. But screening rates remain low (7-year cumulative incidence rates of 42.7 to 58.8 %) [13] despite Medicare’s covering DXA testing every 2 years and encouraging it as part of the one-time-only “Welcome to Medicare” and the annual “Wellness” visits [14] and notwithstanding the inclusion by the National Committee for Quality Assurance (NCQA) of two Health Effectiveness Data and Information Set (HEDIS) measures (OTO and OMW) for osteoporosis screening [15]. Pharmacological treatment rates are even lower, with a 2005–2011 Indiana study of 36,965 patients reporting that only 23.3 % received pharmacological treatment within 2 years of an osteoporosis diagnosis or a fragility fracture (82.2 % of whom were on oral bisphosphonates) [16], and a 2002–2011 USA study of 96,887 patients hospitalized for hip fracture reporting that only 28.5 % had received pharmacological treatment within 12 months post-discharge [9]. Furthermore, both studies documented important declines in pharmacological treatment rates over the course of their observation periods.
In this study, we focused on achieving guideline-concordant pharmacological treatment after DXA testing because of the barriers that patients encounter on the pathway to appropriate pharmacological treatment after DXA screening [17–19]. For patients, this pathway involves (1) receiving and (2) understanding their DXA results and (3) having subsequent contact with their provider and (4) discussing their results and options. Most efforts to improve osteoporosis treatment, however, have targeted health care providers using complex, multifaceted interventions with minimal patient involvement [20–27]. Few studies have evaluated the potential benefits of increasing patient involvement through interventions to activate patients in their health care [28, 29]. Patient activation “emphasizes patients’ willingness and ability to take independent actions” [30] by understanding their “role in the care process and having the knowledge, skill, and confidence to manage one’s health and health care” [31]. In general, strong evidence exists across a variety of diseases and conditions other than osteoporosis that higher patient activation levels predict healthy behaviors and are associated with better health outcomes and care experiences, with lesser evidence of reductions in health care costs [29, 30, 32–36]. Strong evidence also shows that well-crafted interventions that tailor the intervention to each patient lead to clinically significant improvements in patient activation levels and often result in improved health outcomes [30, 32, 37]. Although two patient-activation interventions for osteoporosis have reported improved calcium intake [38] and physical activity [39], there have been few large-scale, randomized controlled trials (RCTs) of patient-activation in osteoporosis, and none have evaluated the pathway from DXA testing to guideline-concordant pharmacological treatment as the primary clinical endpoint.
To address this gap, we conducted a pragmatic RCT—the Patient Activation after DXA Result Notification (PAADRN) study (NCT-01507662)—at three US health care centers. Our primary goal was to evaluate the impact of a simple, scalable intervention designed to improve the primary clinical endpoint—guideline-concordant pharmacological treatment—by activating patients to successfully navigate the pathway from DXA testing to guideline-concordant pharmacological treatment.
Materials and methods
Design
The PAADRN protocol has been described elsewhere [40]. In this pragmatic, double-blinded RCT, patients either received usual care plus a postal mailed tailored-letter with their DXA results accompanied by an educational brochure, or usual care alone. Inclusion criteria were patients ≥50 years old presenting for DXA between February 2012 and August 2014 at three health centers—the University of Iowa (UI), the University of Alabama at Birmingham (UAB), and Kaiser Permanente of Georgia (KPGA). Exclusion criteria were (1) patients <50 years old, (2) prisoners or patients with overt cognitive disability, (3) patients who did not speak or read English, and (4) patients who were deaf or did not have access to a telephone. Each health center’s Institutional Review Board (IRB) reviewed and approved the study protocol and consent procedures.
Randomization and intervention
At the beginning of the study, providers (93.5 % of whom were physicians, 4.4 % were nurse practitioners, and 1.8 % were physicians assistants) at each health center were rank-ordered in descending order based on their DXA volume during 2010–2011 [40]. Within each sequential block of three providers at each site, we randomly assigned one of these three providers to each of the three groups (A, B, or C) using a computerized “balls in bins” allocation program written in the “R” statistical computing language (https://www.r-project.org/about.html) [41] by the senior study biostatistician. In group A, all of the provider’s patients were assigned to the intervention. In group B, all of the provider’s patients were assigned to usual care. In group C, the provider’s patients were randomly allocated on a 1:1 basis either to the intervention or to usual care. New providers with no historical data on DXA ordering volume were randomized to one of the three groups at the time that their first patient entered the study.
Because randomization occurred after complete baseline interview data were obtained, all patients, project baseline interviewers and other staff, and investigators were initially blinded. Intervention patients may have become unblinded after receiving their letter and educational brochure, although the consenting process used deception techniques (patients were not told that they would be randomized) approved by each site’s IRB. As noted below, trained interviewers from the Iowa Social Science Research Center (ISRC), who were not project staff and who were blinded to patient assignment, performed all follow-up data collection. Patient assignment remained concealed to investigators throughout the data collection phase of the study. Providers were blinded at baseline, but could have become unblinded after one or more of their patients showed or discussed with them their intervention letter and/or educational brochure, especially providers with larger numbers of intervention patients.
Intervention patients were notified of their DXA results via a tailored letter and an educational brochure sent by postal mail. The decision to use postal mail was based on patient preferences discovered in our pilot study [17]. The intervention letter and brochure were developed using best practices in risk communication (described elsewhere [42–44]) and were sent from the central coordinating center (UI) approximately 4 weeks after their baseline DXA. The letter included the clinical impression of each patient’s DXA result (normal, osteopenia, or osteoporosis), their 10-year risk of a major osteoporotic fracture (clinical spine, forearm, hip, or shoulder) using a percentage and a visual depiction of risk using a stoplight colored bar graph for low, moderate, and high risk, as well as the suggestion that the patient bring the letter to their next provider visit for discussion. The brochure graphically explained osteoporosis, defined T-scores, and provided five steps to better bone health, including (1) discussing your results with your provider, (2) the need for proper calcium and vitamin D intake, (3) the benefits of exercise, and the dangers of (4) smoking and (5) excessive alcohol intake. Usual care patients received information about their DXA results based upon whatever were the usual care practices of their providers and their health center.
Setting
The three health centers differed substantially in the way that they communicated DXA results to their patients and in their clinical and research osteoporosis programs. UI and KPGA both used the Epic electronic health record (Verona, WI). At UI, patients had to voluntarily register in order to participate in the Epic MyChart patient portal. About 28 % of UI patients were registered for MyChart during the observation period and thus could have had limited access to their test results, including DXA, but DXA results were not routinely communicated (“pushed”) directly to patients. If UI patients chose to use MyChart to access their DXA results, they would have seen their left femoral neck, left hip, and lumbar spine T-scores and BMD g/cm2, as well as their clinical impression (normal, osteopenia, or osteoporosis). If UI patients did not use MyChart, they would only have been notified of their DXA results solely at their ordering provider’s discretion. At KPGA, all patients were routinely registered for the MyChart patient portal known to them as KP.Org. While KPGA patients could use KP.Org to access their other test results, they were not able to access any radiology test results, including DXA. At KGPA, DXA results were routinely postal mailed to patients within 4 weeks of completion using generic, non-tailored template letters that only included the clinical impression (normal, osteopenia, or osteoporosis). While KPGA providers could annotate the clinical impression to include more information like the T-scores and BMD g/cm2, this seldom occurred. While UI and KPGA both have providers with clinical interests in osteoporosis, neither had osteoporosis as either a clinical or research focus. In contrast, UAB had large clinical and research osteoporosis programs and was a regional referral center. UAB, however, had no patient portal at the time of this intervention nor did it have a standard practice of pushing DXA results to patients. Therefore, UAB patients were notified of their DXA results solely at their ordering provider’s discretion.
Baseline data collection
Consented patients had 30- to 40-min baseline interviews using REDCap (Nashville, TN) [45]. Telephone or in-person interviews were conducted by site-specific personnel up to 28 days before or up to 3 days after their index (baseline) DXA. Baseline data (described elsewhere [26]) used as covar-iates in the multivariable analyses were clinical site (UI, UAB, or KPGA), patient age (<65, 65–74, or ≥75), sex, race (non-Hispanic whites vs. all others), education (grade school, high school, college, or post-graduate), self-rated health, history of chronic obstructive pulmonary disease (COPD), history of depression, smoking status (current, former, or never), alcohol use (excessive vs. all others), engagement in weight-bearing exercise, fractures after age 40, parental hip fractures after age 50, patient prior DXA testing, index FRAX risk (low, moderate, or high), prior diagnoses for osteopenia or osteoporosis, and current or former osteoporosis medication use.
Outcomes at 12 and 52 weeks
As noted above, ISRC interviewers blinded to group assignment contacted each patient by telephone at 12 and 52 weeks post-DXA for 20- to 25-min follow-up interviews and outcome ascertainment. Our algorithm for determining the primary outcome—guideline-concordant pharmacological treatment at the 12- and 52-week follow-up interviews—was based on the 2010 NOF guidelines in effect at the time of our study, and both the hip fracture and major osteoporotic fracture risks taken from the online FRAX calculator (v3.6) [8]. These DXA and FRAX data were supplemented with self-reported data from baseline and the 12- and 52-week follow-ups, as appropriate. Table 1 portrays the final algorithm for defining guideline-concordant pharmacological treatment approved by a panel of clinicians and osteoporosis experts convened for this purpose. This algorithm is based on the two-by-two cross-classification of whether the patient was on osteoporosis pharmacological treatment (bisphosphonates, calcitonin, estrogen/hormone therapy, estrogen agonist/antagonist, parathyroid hormone, or denosumab), and whether this was guideline-concordant. Patients in cell A (appropriately on osteoporosis medication) and cell B (appropriately not on osteoporosis medication) were guideline-concordant, whereas patients in cell C (inappropriately on osteoporosis medication) and cell D (inappropriately not on osteoporosis medication) were not. This algorithm was applied to all PAADRN study participants by computer using SAS 9.4 programming code (SAS Institute Inc., Cary, NC).
Table 1.
On treatment at 12 weeks | Not on treatment at 12 weeks | |
---|---|---|
Guideline-concordant treatment | Cell A
|
Cell B3
|
Not guideline-concordant treatment | Cell C
|
Cell D
|
The supplemental self-reported information at baseline included answering the following questions: “Have you ever been told that you have osteoporosis?”, “Have you ever been told that you have osteopenia (or low bone density)?”, “What is the other bone disease?”, “Have you broken a bone since turning 40 (outside of a major trauma like a car accident)?”, “Did your mother or father have a hip fracture after the age of 50?”, “Have you taken in the past or are you now taking any of the following bone medications (Actonel, Boniva, Prolia/denosumab, Miacalcin, estrogen, Evista, Forteo, Fosamax, Relast/Zometa, or “medication for bones but unsure of the name”)?”, “Have you had a bone density test prior to this most recently scheduled DXA?”, “Do you currently take steroids (Prednisone, Cortisol, Deltasone, or Medrol) by mouth?”, and “Have you ever taken steroids (like Prednisone, Cortisol, Deltasone, or Medrol) by mouth for more than three months at one time?”. At the 12- and 52-week follow-ups, the supplemental self-reported information included re-asking the above questions, as appropriate.
We also used self-reports from the 12- and 52-week followup interviews to operationalize four of the steps along the pathway from DXA testing to the clinical endpoint of guideline-concordant pharmacological treatment. These interviews began with this focusing statement: “Today we need to ask you some more questions about the care you received after your DXA scan on [insert scan date].” The four questions were “Did you receive a letter in the mail with your DXA results?” (only asked at the 12-week follow-up), “What were the results of this DXA from (insert scan date; normal, osteopenia, or osteoporosis)?”, “Have you had contact with your provider since your DXA scan that was done on (insert scan date)?”, and “During any of these contacts with a health care provider did you discuss your DXA scan or bone health?”.
Sample size and power
PAADRN was powered on detecting differences in guideline-concordant osteoporosis pharmacological treatment as the primary clinical endpoint. We hypothesized that among osteoporosis or osteopenia patients, our intervention would increase the percentage receiving appropriate osteoporosis pharmacological treatment at follow-up and that among patients with normal BMD, our intervention would reduce the percentage of patients receiving inappropriate osteoporosis pharmacological treatment (over-use) at follow-up. PAADRN was designed to provide 99 % power to detect an 8 % absolute difference in guideline-concordant pharmacological treatment between the intervention and control groups, assuming 11,700 patients (3900 with osteoporosis) and 80 % retention at 52 weeks. During recruitment, however, it became clear that we could not achieve the original enrollment goal. Therefore, our targeted enrollment was reduced to 7500 patients providing 89 % power to detect an 8 % absolute difference.
Data analyses
Our analyses were intention-to-treat (ITT). We performed multiple imputation for participants without 12- and/or 52-week follow-up interviews, using the baseline covariates at 12 weeks, and the baseline covariates and 12-week outcomes at 52 weeks. Graphical and statistical techniques were used to evaluate each variable using appropriate bivariable methods (e.g., t test, chi-square statistic). We compared unadjusted outcomes for guideline-concordant osteoporosis pharmacological treatment as well as four of the steps (receiving and understanding their DXA results and having subsequent contact with their provider and discussing their results and treatment options) along the pathway from DXA to guideline-concordant pharmacological treatment between the intervention and usual care groups. The potential for heterogeneity of treatment effects (HTEs) was examined across a series of patient subgroups based on history of having a prior DXA or low-impact fracture, age, sex, race, and health center, which were all pre-specified (except health center) in the study protocol.
We used random-effects logistic regression to adjust for patient clustering within providers and to estimate adjusted odds ratios (AORs). The outcomes were patient-level indicators of guideline-concordant osteoporosis pharmacological treatment, recalling receipt of a DXA result letter, correct identification of the clinical impression, having had provider contact, and having had discussions with their provider. The primary independent variable was random assignment to the intervention vs. usual care groups, with the covariates noted above.
We conducted several sensitivity analyses. To address the possibility of non-random missingness and its potential adverse implications for multiple imputation, we first re-estimated the models assuming that (a) all patients with missing data on the primary outcome were not taking osteoporosis medications, and then we re-estimated the models assuming that (b) patients in the intervention group with missing data on the primary outcome were guideline-discordant but that patients in the control group with missing data on the primary outcome were guideline-concordant (i.e., a worst-case scenario). We then conducted an analysis among those with complete data only (case-wise deletion) and an analysis among those with complete data that used inverse probability of treatment weighting (IPTW) to adjust for potential selection bias. Finally, we used general estimating equations (GEE) as an alternative statistical adjustment for patient clustering within providers.
Because the outcomes were measured at 12 and 52 weeks and each time point was modeled independently (baseline to 12 weeks and baseline to 52 weeks), we used Bonferroni adjustments. All p values are two-tailed, with p values ≤0.025 deemed statistically significant, except at baseline where Bonferroni adjustments were not necessary. All statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC).
Results
Overall, 827 providers contributed patients to the study. Of these, 532 had 2010–2011 DXA ordering volume data available and 295 providers did not. Overall, 49.6 % of the providers had only one or two patients enrolled, 75.9 % had 10 or fewer patients enrolled, and 90.1 % had 25 or fewer patients enrolled, with comparable percentages within the three provider groups. The number of patients for providers randomized after enrollment began, however, was substantially smaller (range = 1–3).
Figure 1 contains the CONSORT patient flow chart. Of the 20,397 potentially eligible patients appearing on the DXA scheduling lists, 14,280 were determined to be eligible after being contacted by project staff. This excludes the 1495 potentially eligible patients for whom no project staff contact was attempted due to staffing constraints (85 at UI, 827 at UAB, and 583 at KPGA), the 1594 potentially eligible patients who were determined to be ineligible after contact with project staff, and the 3028 potentially eligible patients who were initially contacted by project staff but for whom eligibility could not be determined due to staffing constraints (146 at UI, 2876 at UAB, and 6 at KPGA). Among the 14,280 patients known to be eligible, we consented, conducted baseline interviews with, and properly randomized 7749 (54.3 %; 1685 at UI, 3080 at UAB, and 2984 at KPGA). Follow-up interviews were completed by 6728 (86.8 %) patients at 12 weeks post-baseline and by 6102 (78.7 %) patients at 52 weeks post-baseline. To gauge non-response bias, we compared PAADRN patients with those who refused on age, sex, and race at all three sites, and found that refusers were older (26.1 % were ≥75 vs. 17.7 %, p < 0.001), more likely to be men (38.4 vs. 16.3 %, p < 0.001), and more likely to be black or other than non-Hispanic whites (28.1 and 7.0 vs. 21.4 and 2.0 %, p < 0.001).
Table 2 shows that the sociodemographic, clinical, and osteoporosis-related characteristics for patients were generally similar for patients assigned to the intervention and usual care groups. Mean age was 66.6, 83.8 % were women, and 75.3 % were non-Hispanic whites. A significantly (p < 0.05) higher proportion of intervention patients had prostate cancer and also had lower T-scores on their index DXA. A significantly higher proportion of usual care patients had a history of breast cancer, prior self-reported osteoporosis, and index DXA results indicating osteopenia and osteoporosis. The prevalence of osteoporosis (19.5 %) and osteopenia (53.1 %) shown in Table 2 was higher than recent national estimates (10.3 and 43.9 %) because PAADRN participants were recruited as they presented for their DXA tests rather than from the general population [10].
Table 2.
Intervention (N = 3898) | Usual care (N = 3851) | p value | |
---|---|---|---|
Sociodemographic factors | |||
Age, mean (SD) | 66.5 (8.4) | 66.7 (8.2) | 0.246a |
Women, N (%) | 3259 (83.6) | 3230 (83.9) | 0.750b |
Race/ethnicity | |||
Non-Hispanic white, N (%) | 2930 (75.2) | 2903 (75.4) | 0.873b |
Non-Hispanic black, N (%) | 835 (21.4) | 806 (20.9) | |
Non-Hispanic other, N (%) | 59 (1.5) | 65 (1.7) | |
Hispanic, N (%) | 74 (1.9) | 77 (2.0) | |
Education | |||
Some high school, N (%) | 161 (4.2) | 140 (3.7) | 0.749b |
Completed high school, N (%) | 819 (21.2) | 836 (21.9) | |
Some college, N (%) | 1290 (33.4) | 1269 (33.2) | |
Completed college, N (%) | 785 (20.3) | 762 (19.9) | |
Graduate school, N (%) | 809 (20.9) | 814 (21.3) | |
Comorbid conditions | |||
COPD, N (%) | 259 (6.7) | 265 (6.9) | 0.680b |
Depression, N (%) | 902 (23.2) | 885 (23.0) | 0.878b |
Prostate cancer, N (%) | 117 (18.3) | 88 (14.2) | 0.048b |
Breast cancer, N (%) | 416 (10.7) | 612 (15.9) | <0.001b |
Health habits | |||
Current smoker, N (%) | 295 (7.6) | 295 (7.7) | 0.873b |
Past smoker, N (%) | 1478 (37.9) | 1388 (36.1) | 0.095b |
Current alcohol user, N (%) | 1768 (45.4) | 1808 (47.0) | 0.157b |
Self-reported health status | |||
Excellent, N (%) | 445 (11.4) | 494 (12.8) | 0.329b |
Very good, N (%) | 1443 (37.1) | 1373 (35.7) | |
Good, N (%) | 1280 (32.9) | 1253 (32.6) | |
Fair, N (%) | 571 (14.7) | 566 (14.7) | |
Poor, N (%) | 150 (3.9) | 159 (4.1) | |
Bone health | |||
Prior DXA, N (%) | 2606 (66.9) | 2590 (67.3) | 0.719b |
History of OP, N (%) | 794 (20.6) | 909 (23.8) | 0.001b |
History of OP treatment, N (%) | 1438 (36.9) | 1502 (39.0) | 0.055b |
Glucocorticoids use, N (%) | 593 (15.2) | 576 (15.0) | 0.753b |
Fracture after age 40, N (%) | 1,043 (26.9) | 1,051 (27.3) | 0.630b |
Parental hip fracture after age 50, N (%) | 585 (15.3) | 578 (15.3) | 0.999b |
Study DXA results | |||
Normal, N (%) | 1133 (29.1) | 990 (25.7) | 0.001b |
Osteopenia (low BMD), N (%) | 2052 (52.6) | 2066 (53.6) | |
Osteoporosis, N (%) | 713 (18.3) | 795 (20.6) | |
Lowest T-score, mean (SD) | −1.62 (1.1) | −1.55 (1.1) | 0.002a |
10-year risk of a major osteoporotic (clinical spine, forearm, hip, or shoulder) fracture (FRAX), mean (SD) | 12.0 (9.2) | 12.3 (9.1) | 0.101a |
p value from two-sample t test
p value is from Pearson chi-square test
Overall, 3573 participants (46.1 %) met the NOF criteria for pharmacological treatment. About a third of these patients had received osteoporosis pharmacological treatment by 12 weeks (33.0 %) or by 52 weeks (32.3 %) post-DXA. Among the 4176 participants not meeting the NOF criteria for pharmacological treatment, 8.4 % received osteoporosis medications by 12 weeks post-DXA, as did 8.1 % by 52 weeks post-DXA. There were no differences in adherence rates (75.1 % in the intervention group vs. 75.0 % in the usual care group at 12 weeks post-DXA) among those who reported having been prescribed osteoporosis medications based on their study DXA.
Table 3 contains the results from the ITT unadjusted bivariable comparisons. No differences were observed between the intervention and usual care groups on guideline-concordant osteoporosis pharmacological treatment at either 12 weeks (65.1 vs. 64.3 %, p = 0.506) or 52 weeks post-DXA (65.2 vs. 63.8 %, p = 0.250). There were, however, significant effects of the intervention along the patient pathway from DXA testing to guideline-concordant osteoporosis pharmacological treatment. At 12 weeks post-DXA, the intervention group had higher rates than the usual care group of recalling receiving a DXA result letter (82.3 vs. 61.3 %, p < 0.001) and correctly identifying their DXA clinical impression (68.9 vs. 60.8 %, p < 0.001). At 52 weeks post-DXA, the intervention group had higher rates than the usual care group of correctly identifying their DXA clinical impression (66.4 vs. 61.1 %, p < 0.001), having contact with their providers (68.3 vs. 61.5 %, p < 0.001), and discussing their DXA results with their providers (37.7 vs. 31.3 %, p < 0.001). Across the pre-specified subgroups based on history of having a prior DXA, low-impact fracture, age, sex, race, and health center, the only observed HTE effect on the unadjusted bivariable comparisons was for KPGA vs. UI patients recalling having received a DXA letter, with a larger effect for KPGA consistent with the fact that KPGA routinely sends out a generic, non-tailored letter to patients with their results.
Table 3.
At 12 weeks
|
At 52 weeks
|
|||||
---|---|---|---|---|---|---|
Intervention | Usual care | p value | Intervention | Usual care | p value | |
Patient was on guideline-concordant pharmacologic treatment | 2537 (65.1 %) | 2477 (64.3 %) | 0.506 | 2,543 (65.2 %) | 2,459 (63.8 %) | 0.250 |
Patient recalled receiving a letter with their DXA results | 3207 (82.3 %) | 2361 (61.3 %) | <0.001 | N/A | ||
Patient correctly identified the results of their baseline DXA | 2687 (68.9 %) | 2341 (60.8 %) | <0.001 | 2590 (66.4 %) | 2355 (61.1 %) | <0.001 |
Patient reported having contact with their provider after their DXA | 2255 (57.9 %) | 2212 (57.4 %) | 0.716 | 2663 (68.3 %) | 2368 (61.5 %) | <0.001 |
Patient discussed their DXA results with their provider | 1300 (33.4 %) | 1192 (30.9 %) | 0.034 | 1471 (37.7 %) | 1204 (31.3 %) | <0.001 |
Table 4 contains the AORs for the intervention group vs. the usual care group obtained from the ITT random-effects logistic regressions that adjusted for patient clustering within providers and the covariates noted above. Once again, no differences were observed for receiving guideline-concordant osteoporosis pharmacological treatment at either 12 weeks (AOR = 1.03, p = 0.635) or 52 weeks (AOR = 1.06, p = 0.354) post-DXA. After adjustment for patient clustering with providers and the covariates, however, there continued to be significant effects of the intervention on the patient pathway from DXA testing to guideline-concordant osteoporosis pharmacological treatment. At 12 weeks post-DXA, the AORs on patient recollection of receiving a DXA result letter (3.14, p < 0.001) and correctly identifying their DXA clinical impression (1.47, p < 0.001) were significant. At 52 weeks post-DXA, the AORs on correctly identifying their DXA clinical impression (1.27, p = 0.001), having contacted their provider (1.21, p = 0.010), and having discussed their DXA results with their provider (1.24, p < 0.001) were significant. Across the pre-specified subgroups, no HTE effects were observed in the random-effects logistic regressions after adjustment.
Table 4.
At 52 weeks
|
Crude Adjusted
|
||||
---|---|---|---|---|---|
Crude | Adjusted | Crude | Adjusted | ||
Patient was on guideline-concordant pharmacotherapy | AOR 95 % CI |
1.04 (0.93, 1.16) |
1.03 (0.91, 1.17) |
1.07 (0.96, 1.19) |
1.06 (0.93, 1.21) |
Patient recalled receiving a letter with their DXA results | AOR 95 % CI |
3.10*** (2.66, 3.63) |
3.14*** (2.71, 3.63) |
N/A | |
Patient correctly identified the results of their baseline DXA | AOR 95 % CI |
1.44*** (1.29, 1.62) |
1.47*** (1.31, 1.64) |
1.26** (1.10, 1.44) |
1.27** (1.11, 1.45) |
Patient reported having contact with their provider after their DXA | AOR 95 % CI |
1.02 (0.92, 1.12) |
1.01 (0.91, 1.12) |
1.21* (1.05, 1.40) |
1.21* (1.05, 1.39) |
Patient discussed their DXA results with their provider | AOR 95 % CI |
1.15* (1.02, 1.30) |
1.13 (1.01, 1.27) |
1.22** (1.09, 1.37) |
1.24*** (1.11, 1.39) |
The covariates that were adjusted for included clinical site, patient age, sex, race, education, self-rated health, COPD, depression, smoking status, alcohol use, weight-bearing exercise, fractures after age 40, parental hip fractures after age 50, prior DXA testing, index FRAX risk, prior diagnoses for osteopenia or osteoporosis, and current or former osteoporosis medication use
p < 0.025,
p < 0.005,
p < 0.0005
The four sensitivity analyses to address non-random missingness (assuming that all patients with missing data did not receive guideline-concordant pharmacological treatment, the worst-case scenario, case-wise deletion, and IPTW reweighting) yielded comparable results with one exception. The worst-case scenario sensitivity analyses resulted in the usual care group having significantly higher rates of guideline-concordant pharmacological treatment, as would be expected from a negative trial with attrition rates like those that occurred in this study. The sensitivity analysis using GEE to address patient clustering within providers rather than using the random effects model yielded comparable results.
Discussion
PAADRN’s goal was to improve guideline-concordant pharmacological treatment using a patient-activation intervention consisting of a tailored letter designed using best practices in risk communication and contained their DXA results along with an educational brochure about osteoporosis. Control group patients only received usual care. Among the 7749 PAADRN participants, neither statistically significant nor clinically meaningful differences in guideline-concordant pharmacological treatment were observed between the intervention and control groups at either 12 weeks (65.1 vs. 64.3 %, number needed to treat [NNT] = 125.0, p = 0.506) or 52 weeks (65.2 vs. 63.8 %, NNT = 71.4, p = 0.250) post-baseline. Thus, PAADRN was a negative trial with respect to its primary clinical endpoint.
The patient-activation intervention did, however, modestly increase the likelihood that patients would receive (at 12 weeks post-baseline, 82.3 vs. 61.3 %, p < 0.001) and understand their DXA results (at 12 weeks post-baseline, 66.4 vs. 61.1 %, NNT = 12.3, p < 0.001; and at 52 weeks, 68.9 vs. 60.8 %, NNT = 14.7, p < 0.001), and have subsequent contact with their providers (at 52 weeks post-baseline, 68.3 vs. 61.5 %, NNT = 14.7, p < 0.001) at which they would discuss their results and treatment options (at 52 weeks post-baseline, 37.7 vs. 31.2 %, NNT = 15.4, p < 0.001). This raises the question of why PAADRN did not improve guideline-concordant pharmacological treatment.
At least five plausible reasons warrant mention. First, by their very nature, patient-activation interventions are designed to improve “patients’ willingness and ability to take independent actions” [30]. Patients cannot, however, independently alter their osteoporosis medication regimens except by refusing to fill or take their prescribed drugs. Thus, a patient-activation intervention, by itself, may have had little opportunity to improve guideline-concordant pharmacological treatment. Second, neither the intervention letter nor the educational brochure focused on pharmacological treatment regimens, their trade-offs, or potential benefits. Rather, the intervention letter emphasized patients’ understanding of their DXA results and encouraging them to discuss these with their provider, while the educational brochure focused on lifestyle changes. Thus, the intervention may not have been properly focused on the primary clinical endpoint. Third, in keeping with the purpose of pragmatic RCTs, we developed an intervention that was highly scalable, in that it could be set up in most electronic medical record systems for automated delivery. But this underscores the tension in pragmatic RCTs between the ease of adoption and delivery (in this case, a single postal mailing of the letter and brochure) vs. providing a sufficient interventional dose (in this case, of patient-activation). Perhaps, a text-based messaging intervention that included, for example, “YouTube” video links about how to interpret DXA test results combined with repeated reminders encouraging adequate calcium and vitamin D intake, weight-bearing and muscle-strengthening exercise, fall prevention, smoking cessation, avoidance of excessive alcohol intake, and pharmacological treatment when appropriate would have been more effective. Fourth, the improvements along the patient pathway may have been too modest to bring about changes in guideline-concordant pharmacological treatment. Finally, it may be that for a silent disease that by 2025 will only result in three million fractures (0.9 % of the projected total 2025 US population of 335 million, or 4.4 % of the projected population of 67 million ≥65 at that time) with only $25 billion in associated costs (0.005 % of the projected $5.3 trillion total health care expenditures), most older adults do not take the possibility of an osteoporotic fracture very seriously and may thus be insensitive to interventions to alter their bone health and health care.
PAADRN is not without limitations. First, our power to detect small differences in the primary clinical endpoint of guideline-concordant pharmacological treatment was limited, creating the potential for type 2 error. Given PAADRN’s large size (N = 7749) and the observed absolute difference of only 0.8 % at 12 weeks and 1.4 % at 52 weeks in guideline-concordant pharmacological treatment, however, it is unlikely that our failure to reject that null hypothesis was due to statistical power. Second, our outcomes were ascertained via patient self-reports, which may involve inaccuracies that may or may not have been random. This is especially true for the primary clinical endpoint. Although we had intended to conduct a site-specific validation study of self-reports of pharmacological treatment using the pharmacy records at KPGA, we were not able to do so because of NIH budgetary reductions. Third, many PAADRN patients had a prior DXA, which may have decreased the effectiveness of the patient-activation intervention among them, although our HTE analysis did not find this to be the case. Furthermore, we could not adjust for patients who may have been on “drug holidays” at the recommendation of their treating providers. Finally, we did not collect information on whether or not patients used the internet or other sources of information either before or after their interactions with their providers.
In conclusion, we found that directly communicating patients’ DXA results to them via a tailored DXA result letter accompanied by an educational bone health brochure did not improve guideline-concordant pharmacological treatment. Thus, PAADRN was a negative trial with respect to its primary clinical endpoint. And while PAADRN did modestly improve patient attention to their results, knowledge of the clinical impression of those results, and activated patients to meet with and discuss those results with their providers, these benefits did not lead to improved bone health care.
Acknowledgments
Funding/support This work was supported by R01 AG033035 (Cram/Wolinsky) from the NIA at NIH. Dr. Cram is supported by a K24 AR062133 award from NIAMS at the NIH. Dr. Saag is supported by a K24 AR052361 award from the NIAMS at the NIH.
Role of the sponsor The NIA had no role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Footnotes
Trial Registration: clinicaltrials.gov identifier NCT01507662
Compliance with ethical standards
Conflicts of interest P. Cram, M. P. Jones, F. D. Wolinsky, S. W. Edmonds, S. F. Hall, Y. Lou, and D. W. Roblin have no conflicts of interest. N. C. Wright has received unrestricted grant support from Amgen for work unrelated to this project. K. G. Saag has received grants from Amgen, Eli Lilly, and Merck and has served as a paid consultant to Amgen, Eli Lilly, and Merck unrelated to this project.
References
- 1.World Health Organization. Assessment of fracture risk and its application to screening for postmenopausal osteoporosis. Tech Rep Ser. 1994;843:1–129. [PubMed] [Google Scholar]
- 2.Anonymous. Consensus development conference: diagnosis, prophylaxis, and treatment of osteoporosis. Am J Med. 1993;94:646–650. doi: 10.1016/0002-9343(93)90218-e. [DOI] [PubMed] [Google Scholar]
- 3.Kanis JA, McCloskey EV, Johansson H, Cooper C, Rizzoli R, Reginster JY. European guidance for the diagnosis and management of osteoporosis in postmenopausal women. Osteoporos Int. 2013;24:23–57. doi: 10.1007/s00198-012-2074-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.US Department of Health and Human Services. Bone health and osteoporosis: a report of the Surgeon General. US Department of Health and Human Services; Rockville: 2004. [Google Scholar]
- 5.National Osteoporosis Foundation. Clinician’s guide to prevention and treatment of osteoporosis. National Osteoporosis Foundation; Washington, DC: 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kanis JA. Diagnosis of osteoporosis and assessment of fracture risk. Lancet. 2002;359:1929–1936. doi: 10.1016/S0140-6736(02)08761-5. [DOI] [PubMed] [Google Scholar]
- 7.Kanis JA, McCloskey EV, Johansson H, Strom O, Borgstrom F, Oden A. Case finding for the management of osteoporosis with FRAX—assessment and intervention thresholds for the UK. Osteoporos Int. 2008;19:1395–1408. doi: 10.1007/s00198-008-0712-1. [DOI] [PubMed] [Google Scholar]
- 8.National Osteoporosis Foundation. Clinician’s guide to prevention and treatment of osteoporosis. National Osteoporosis Foundation; Washington, DC: 2010. [Google Scholar]
- 9.Johnell O, Kanis JA. An estimate of the worldwide prevalence and disability associated with osteoporotic fractures. Osteoporos Int. 2006;17:1726–1733. doi: 10.1007/s00198-006-0172-4. [DOI] [PubMed] [Google Scholar]
- 10.Wright NC, Looker AC, Saag KG, Curtis JR, Delzell ES, Randall S, Dawson-Hughes B. The recent prevalence of osteoporosis and low bone mass in the United States based on bone mineral density at the femoral neck or lumbar spine. J Bone Miner Res. 2014;29:2520–2526. doi: 10.1002/jbmr.2269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nelson HD, Haney EM, Chou R, Dana T, Fu R, Bougatsos C. Screening for osteoporosis: systematic review to update the 2002 US Preventive Services Task Force Recommendation. Agency for Healthcare Research and Quality; Rockville: 2010. [PubMed] [Google Scholar]
- 12.U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion. Arthritis, osteoporosis, and chronic neck conditions. 2015 http://www.healthypeople.gov/2020/topics-objectives/topic/Arthritis-Osteoporosis-and-Chronic-Back-Conditions/objectives Accessed 8 Apr, 2015.
- 13.Amarnath AL, Franks P, Robbins JA, Xing G, Fenton JJ. Underuse and overuse of osteoporosis screening in a regional health system: a retrospective cohort study. J Gen Intern Med. 2015;30:1733–1740. doi: 10.1007/s11606-015-3349-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. Your guide to Medicare’s preventive services. Centers for Medicare & Medicaid Services; Baltimore: 2015. [Google Scholar]
- 15.National Committee for Quality Assurance. HEDIS 2016. 2016 http://www.ncqa.org/hedis-quality-measurement/hedis-measures/hedis-2016 Accessed 13 Apr, 2016.
- 16.Liu Z, Weaver J, de Papp A, Li Z, Martin J, Allen K, Hui S, Imel EA. Disparities in osteoporosis treatments. Osteoporos Int. 2016;27:509–519. doi: 10.1007/s00198-015-3249-0. [DOI] [PubMed] [Google Scholar]
- 17.Cram P, Rosenthal GE, Ohsfeldt R, Wallace RB, Schlechte J, Schiff GD. Failure to recognize and act on abnormal test results: the case of screening bone densitometry. Jt Comm J Qual Patient Saf. 2005;31:90–97. doi: 10.1016/s1553-7250(05)31013-0. [DOI] [PubMed] [Google Scholar]
- 18.Pickney CS, Arnason JA. Correlation between patient recall of bone densitometry results and subsequent treatment adherence. Osteoporos Int. 2005;16:1156–1160. doi: 10.1007/s00198-004-1818-8. [DOI] [PubMed] [Google Scholar]
- 19.Cadarette SM, Beaton DE, Gignac MA, Jaglal SB, Dickson L, Hawker GA. Minimal error in self-report of having had DXA, but self-report of its results was poor. J Clin Epidemiol. 2007;60:1306–1311. doi: 10.1016/j.jclinepi.2007.02.010. [DOI] [PubMed] [Google Scholar]
- 20.Majumdar SR, McAlister FA, Johnson JA, et al. Interventions to increase osteoporosis treatment in patients with ‘incidentally’ detected vertebral fractures. Am J Med. 2012;125:929–936. doi: 10.1016/j.amjmed.2012.02.021. [DOI] [PubMed] [Google Scholar]
- 21.Ciaschini PM, Straus SE, Dolovich LR, et al. Community based intervention to optimize osteoporosis management: randomized controlled trial. BMC Geriatr. 2010;10:60. doi: 10.1186/1471-2318-10-60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Majumdar SR, Johnson JA, McAlister FA, Bellerose D, Russell AS, Hanley DA, Morrish DW, Maksymowych WP, Rowe BH. Multifaceted intervention to improve diagnosis and treatment of osteoporosis in patients with recent wrist fracture: a randomized controlled trial. CMAJ. 2008;178:569–575. doi: 10.1503/cmaj.070981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Majumdar SR, Johnson JA, Lier DA, et al. Persistence, reproducibility, and cost-effectiveness of an intervention to improve the quality of osteoporosis care after a fracture of the wrist: results of a controlled trial. Osteoporos Int. 2007;18:261–270. doi: 10.1007/s00198-006-0248-1. [DOI] [PubMed] [Google Scholar]
- 24.Majumdar SR, Rowe BH, Folk D, et al. A controlled trial to increase detection and treatment of osteoporosis in older patients with a wrist fracture. Ann Intern Med. 2004;141:366–373. doi: 10.7326/0003-4819-141-5-200409070-00011. [DOI] [PubMed] [Google Scholar]
- 25.Leslie WD, LaBine L, Klassen P, Dreilich D, Caetano PA. Closing the gap in postfracture care at the population level: a randomized controlled trial. CMAJ. 2012;184:290–296. doi: 10.1503/cmaj.111158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Outman RC, Curtis JR, Locher JL, Allison JJ, Saag KG, Kilgore ML. Improving osteoporosis care in high-risk home health patients through a high-intensity intervention. Contemp Clin Trials. 2012;33:206–212. doi: 10.1016/j.cct.2011.09.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Cranney A, Lam M, Ruhland L, Brison R, Godwin M, Harrison MM, Harrison MB, Anastassiades T, Grimshaw JM, Graham ID. A multifaceted intervention to improve treatment of osteoporosis in postmenopausal women with wrist fractures: a cluster randomized trial. Osteoporos Int. 2008;19:1733–1740. doi: 10.1007/s00198-008-0669-0. [DOI] [PubMed] [Google Scholar]
- 28.Warriner AH, Outman RC, Feldstein AC, et al. Effect of self-referral on bone mineral density testing and osteoporosis treatment. Med Care. 2014;52:743–750. doi: 10.1097/MLR.0000000000000170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Morisky DE, Bowler MH, Finlay JS. An educational and behavioral approach toward increasing patient activation in hypertension management. J Commun Health. 1982;7:171–182. doi: 10.1007/BF01325513. [DOI] [PubMed] [Google Scholar]
- 30.Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff. 2013;32:207–214. doi: 10.1377/hlthaff.2012.1061. [DOI] [PubMed] [Google Scholar]
- 31.Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39:1005–1026. doi: 10.1111/j.1475-6773.2004.00269.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hibbard J, Gilburt H. Supporting people to manage their health: an introduction to patient activation. The King’s Fund; London: 2014. [Google Scholar]
- 33.Greene J, Hibbard JH. Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. J Gen Intern Med. 2012;27:520–526. doi: 10.1007/s11606-011-1931-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Marshall R, Beach MC, Saha S, Mori T, Loveless MO, Hibbard JH, Cohn JA, Sharp VL, Korthuis PT. Patient activation and improved outcomes in HIV-infected patients. J Gen Intern Med. 2013;28:668–674. doi: 10.1007/s11606-012-2307-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mitchell SE, Gardiner PM, Sadikova E, Martin JM, Jack BW, Hibbard JH, Paasche-Orlow MK. Patient activation and 30-day post-discharge hospital utilization. J Gen Intern Med. 2014;29:349–355. doi: 10.1007/s11606-013-2647-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Pilling SA, Williams MB, Brackett RH, Gourley R, Weg MW, Christensen AJ, Kaboli PJ, Reisinger HS. Part I, patient perspective: activating patients to engage their providers in the use of evidence-based medicine: a qualitative evaluation of the VA Project to Implement Diuretics (VAPID) Implement Sci. 2010;5:23. doi: 10.1186/1748-5908-5-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Insignia Health. Research studies that use the Patient Activation Measure® (PAM®) 2016 http://s3-us-west-2.amazonaws.com/insignia/Research-Studies-Using-PAM.Bibliography.pdf?mtime=20150629140537 Accessed 13 Apr, 2016.
- 38.McLeod KM, McCann SE, Horvath PJ, Wactawski-Wende J. Predictors of change in calcium intake in postmenopausal women after osteoporosis screening. J Nutr. 2007;137:1968–1973. doi: 10.1093/jn/137.8.1968. [DOI] [PubMed] [Google Scholar]
- 39.Winzenberg T, Oldenburg B, Frendin S, De Wit L, Riley M, Jones G. The effect on behavior and bone mineral density of individualized bone mineral density feedback and educational interventions in premenopausal women: a randomized controlled trial. BMC Public Health. 2006;6:12. doi: 10.1186/1471-2458-6-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Edmonds SW, Wolinsky FD, Christensen AJ, Lu X, Jones MP, Roblin DW, Saag KG, Cram P. The PAADRN Study: a design for a randomized controlled practical clinical trial to improve bone health. Contemp Clin Trials. 2013;34:90–100. doi: 10.1016/j.cct.2012.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.R Development Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna: 2015. [Google Scholar]
- 42.Edmonds SW, Solimeo SL, Lu X, Roblin DW, Saag KG, Cram P. Developing a bone mineral density test result letter to send to patients: a mixed-methods study. Patient Prefer Adherence. 2014;8:827–841. doi: 10.2147/PPA.S60106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Edmonds SW, Cram P, Lu X, Roblin DW, Wright NC, Saag KG, Solimeo SL. Improving bone mineral density reporting to patients with an illustration of personal fracture risk. BMC Med Inform Decis Mak. 2014;14:101. doi: 10.1186/s12911-014-0101-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Edmonds SW, Solimeo SL, Nguyen VT, Wright NC, Roblin DW, Saag KG, Cram P. Understanding preferences for osteoporosis information to develop an osteoporosis-patient education brochure. Perm J. 2016 doi: 10.7812/TPP/16-024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Info. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]