Development of a Personalized Visualization and Analysis Tool to Improve Clinical Care in Complex Multisystem Diseases With Application to Scleroderma

Abstract

Objective

In complex diseases, it is challenging to assess a patient's disease state, trajectory, treatment exposures, and risk of multiple outcomes simultaneously, efficiently, and at the point of care.

Methods

We developed an interactive patient‐level data visualization and analysis tool (VAT) that automates illustration of the trajectory of a patient with scleroderma across multiple organs and illustrates this relative to a reference population, including patient subgroups who share risk factors with the index patient, to improve estimation of disease state. We conducted VAT usability testing with patients and clinicians. We then embedded results from internally cross‐validated, Bayesian multivariate mixed models that calculate an individual's risk of critical events, using baseline risk factors, patient‐level information in past trajectories in multiple dimensions, and known outcomes from the entire population and relevant subgroups.

Results

The web‐based application aggregates complex, longitudinal data to illustrate patient‐, subgroup‐, and population‐level health trajectories across multiple organ systems. Patients (n = 7) exposed to the VAT reported increased knowledge about their disease and confidence in medical decision‐making. Rheumatologists (n = 4) were able to access 8.6 times more data in 81.5% of the time using two‐thirds fewer clicks using the VAT compared with an electronic medical record system. Statistical modeling was successfully embedded in the VAT, enabling real‐time estimation of a patient's risks of multiple complications.

Conclusion

Systematic analysis and visualization of individual‐ and population‐level data in a complex disease has potential to improve medical decision‐making and warrants further study. Individualized risk estimation disseminated at the point of care may enable targeted screening and early intervention in high‐risk patients.

INTRODUCTION

In the clinic, physicians taking care of patients with complex rheumatic diseases use cognitive skills to integrate information across multiple parameters and organ systems, factoring in a patient's prior trajectory and baseline risk factors, to make estimates about a patient's health state, risk for complications, and need for high‐risk therapies. A key component of this mental process is that it is informed by a physician's prior experiences caring for patients with a similar expression of disease and therefore is not generalizable across providers—particularly in rare diseases or diseases with high clinical variability. Aggregating complex, longitudinal data requires a tremendous time investment on the part of the treating provider who has access to historical and current data only for the patient at hand. It is also challenging to clearly explain this information to patients during routine clinical visits to facilitate shared decision‐making. In this study, we sought to develop methods to address these challenges using a prototypic multisystem rheumatic disease, systemic sclerosis (scleroderma).

SIGNIFICANCE & INNOVATIONS.

• In complex, multisystem diseases like scleroderma, there is often significant heterogeneity in patients’ clinical manifestations, disease course, and risk of complications. There is a major unmet need to develop tools that aggregate patient‐level, longitudinal data and enable continuous learning across populations of patients to improve individualized decision‐making.

• We developed an interactive patient‐level data visualization and analysis tool that automates illustration of the trajectory of a patient with scleroderma across multiple organs relative to a reference population and calculates real‐time estimates of critical events.

• Patients exposed to historical data in the tool reported increased knowledge about their disease and felt better equipped to participate in shared decision‐making. Rheumatologists accessed >8 times more data in ~80% of the time compared with the standard electronic medical record and reported higher levels of confidence in their medical decision‐making.

• This tool represents a shift in health informatics from a clinician having access to the current patient's data only to contextualizing that patient's data with the experience of a selected subset of prior patients.

Scleroderma is a complex disease with high variability in clinical phenotype, longitudinal trajectory, treatment response, and mortality. Scleroderma can affect multiple organ systems, and most organ‐specific complications occur in ~15% of patients. ¹ , ² Several clinically available measures, such as autoantibody type and cutaneous subtype, may be useful to subgroup and risk‐stratify patients. However, these population‐level risk factors are not easily translatable to clinical practice at the patient level to inform targeted screening or early intervention. ³ , ⁴ , ⁵ For example, patients with African ancestry, diffuse scleroderma, or anti–topoisomerase‐1 antibodies have a higher risk of clinically significant interstitial lung disease (ILD). ⁶ However, for an individual patient, it remains unknown how this risk is modified by the presence of multiple risk factors, is affected by her own pulmonary function trajectory early in the disease, or changes with involvement of other organ systems.

To address these challenges, we designed a visualization and analysis tool (VAT) to effectively communicate a patient's trends across multiple organ systems. We developed interactive filters that enable a provider to compare an individual patient to a subgroup of patients who share relevant clinical and biologic characteristics. We then implemented this prototype in a web‐based application (called Patient Insight) that can be viewed within different electronic medical record (EMR) systems to bring the tool within clinicians’ workflow and enable future dissemination. ⁷ We conducted a study to evaluate effectiveness and usability of Patient Insight for patients and clinicians. Lastly, using Bayesian multivariate hierarchical models and a cross‐validated sequential prediction (CVSP) algorithm, we compute and display real‐time personalized risk estimates for major complications, harnessing knowledge from a patient's prior trajectory and known outcomes from patients with similar subgroup characteristics. ⁸ In this article, we present these design and implementation methods. Although the initial development has focused on scleroderma, we anticipate these methods, employing clinical, computational, and engineering skills, will have broad applicability across complex, multisystem diseases and health systems.

METHODS

Development of the VAT

A key objective of a VAT is to make quantitative the qualitative steps that clinicians currently use to evaluate their prior experience and knowledge when reaching a clinical decision for an individual patient. Supplemental Figure 1 describes a pipeline and series of clinical, statistical, and engineering methods that were used to develop the VAT and embed it within the clinical workflow. Expert clinicians selected clinical parameters, calculations, and interactive features to be displayed. Based on this initial sketch, statisticians developed a flexible prototype R Shiny App that visualizes patient‐ and population‐level longitudinal data, enabling quick iterations and revisions. ⁹ , ¹⁰ After clinician approval, the Johns Hopkins Medicine (JHM) Technology Innovation Center implemented R Shiny App features into a web‐based application that could be viewed within the EMR, which for our institution is Epic. This step met two objectives: (1) to generate a version of the tool physicians can use directly to test its clinical value and (2) to enable future dissemination of the tool across health systems and EMR platforms. The web‐based version, referred to as Patient Insight, can be updated outside of the EMR, allowing for rapid improvements to incorporate feedback.

Data sources

This study used clinical data from patients who consented to participate in the Johns Hopkins Scleroderma Center Research Registry, a dynamic‐entry, prospective longitudinal cohort. Data from registry participants are ingested into the JHM Precision Medicine Analytics Platform (PMAP) (Supplemental Figure 2), in which multiple data sources are harmonized. JHM's Epic EMR captures all transactions comprising clinical care: clinical visits, laboratory measurements, prescriptions, imaging, and procedures. We use real‐time Epic data from PMAP as input to our VAT.

Individualized clinical data visualization and comparison to user‐defined subgroups

Patient Insight illustrates a patient's aggregate clinical phenotype in a snapshot view; for scleroderma, this includes cutaneous subtype (ie, diffuse versus limited), ¹¹ disease manifestations, onset dates, and autoantibody status. Longitudinal data are illustrated across multiple organ systems (eg, cutaneous, cardiac, pulmonary, gastrointestinal, renal, peripheral vasculature, and musculoskeletal), including patient‐reported outcomes and laboratory data. Longitudinal immunosuppressive medication exposure is shown to assess whether drug exposure alters trajectory in these parameters. Relevant comorbid conditions, such as coronary artery disease, diabetes, hypertension, and cancer, are captured.

We defined critical events by either having longitudinal observations exceed or fall below prespecified thresholds or having a discrete event occur at a particular date. These events were plotted on a single time scale starting from scleroderma onset. ¹² Details regarding chosen variables are provided in the Supplemental Materials.

The tool illustrates the 10th, 50th, and 90th percentiles for the entire scleroderma population as a reference group. By plotting individuals’ trajectories on top of these reference lines, we can visualize a patient's disease course compared with others. Moreover, we programmed filters to compare a patient's trajectory to a user‐specified subgroup based on demographic, clinical, and biologic characteristics. This allows clinicians to easily monitor a patient's disease course relative to a group of similar patients based upon known risk factors, such as age, race, sex, cutaneous subtype, and autoantibody status.

Testing effectiveness and usability of Patient Insight

We conducted two studies to assess patients and rheumatologists’ perspectives of using Patient Insight. We sought to gain early feedback on the functionalities of visualizing historical clinical data and comparing individuals’ trajectory to a user‐defined subgroup. Of note, this component of the study did not include incorporation of personalized predictions of the future risk of critical events, which is described further below. This study was approved by the JHM Institutional Review Board (IRB000252437). Survey questions are in the Supplemental Materials.

Patient perspectives

We recruited seven patients with scleroderma who had at least one year of clinical data and were able to participate in study visits over Zoom during the COVID‐19 pandemic. Before exposure to Patient Insight, patients completed a survey to capture their understanding of their disease and its trajectory. Then, patients were introduced to their own data in Patient Insight and shown how providers would navigate the tool and share data during a clinical visit. Organ‐specific trends were illustrated relative to a population of >4,000 patients with scleroderma and relative to the subgroup of patients who shared key characteristics. Trends were explained to patients in relation to immunosuppressive therapy exposures when applicable. We conducted semistructured interviews (see Supplemental Materials) to evaluate patients’ impressions when the tool was part of the communication process. The same survey questions were administered after the interview to assess if the tool changed participants’ understanding of their scleroderma.

Interviews were audio‐recorded and transcribed verbatim. Two investigators developed an initial coding structure and identified general themes based on the interview guide and first three interviews. ¹³ , ¹⁴ The coding structure was enriched in breadth and depth based on subsequent interviews and emergent themes. After achieving thematic saturation with the fifth interview, we conducted two additional interviews to ensure no additional themes were identified. Participant comments were coded and analyzed for sentiment to summarize themes and illustrate key findings.

Provider perspectives

To assess rheumatologists’ perspectives, Patient Insight was compared with standard of care using Epic. Testing was performed in the context of four real patient cases. Two cases each were selected to reflect common issues arising in clinic: (1) whether immunosuppression needs to be initiated or changed to treat one or more organ systems and (2) whether additional, potentially invasive testing is needed to assess for complications. For each scenario, academic rheumatologists who regularly see patients with scleroderma were asked to assess, as if they were reviewing data for a clinic visit, one case using Epic and one case using Patient Insight in randomized order. Task duration, number of clicks, and data points reviewed were recorded for each task and analyzed using a paired t‐test.

Rheumatologists filled out an additional questionnaire to assess: whether Patient Insight helped them understand a patient's health state and trajectory; whether it would help them explain health state and trajectory to patients; and whether they gained any new insight into a patient's disease by seeing data visually. Throughout, a human‐factors engineer solicited feedback to understand potential areas for improvement, clarity of data elements, and user perception of task ease or difficulty. Participants were asked to elaborate on whether Patient Insight would change their clinical practice, specifically (1) the time required to prepare for seeing a patient, (2) if they would plan to review the tool with patients when discussing their clinical impression, and (3) which parameters they would most likely share with patients. Following each interview, rheumatologists were emailed a System Usability Scale (SUS) to objectively measure their satisfaction with Patient Insight. ¹⁵ , ¹⁶ SUS scores range from 0 (worst) to 100 (best), with 68 considered the threshold of acceptable usability.

Estimation of individuals’ health trajectories and prediction of future risk of critical events

A patient's estimated disease trajectories foretell the risk of having extreme values in the near future. By fully using information in multiple longitudinal measures, we can maximize the precision of estimates of patient‐specific and population trajectories. We modeled the longitudinal latent health state using a Bayesian multivariate linear mixed‐effects model ¹⁷ , ¹⁸ for cardiopulmonary parameters (percent predicted forced vital capacity [pFVC], percent predicted diffusing capacity for carbon monoxide [pDLco], left ventricular ejection fraction [LVEF], and right ventricular systolic pressure [RVSP]). Details of the model are in the Supplemental Materials. Using model estimates, we calculate and display disease trajectories for each patient for pFVC, RVSP, and EF, which are important surrogates for key outcomes (ILD, pulmonary hypertension [PH], and cardiomyopathy).

Observations falling below or rising above a clinically set threshold are useful surrogates for critical events that will likely require immediate medical attention sometimes followed by more invasive and higher risk interventions. To predict critical events, we projected each individual's trajectory into the future and then calculated the probability the person will cross the following boundaries: LVEF < 50% and LVEF < 35% (cardiomyopathy), RVSP ≥ 45 mmHg and RVSP ≥ 50 mmHg (PH), and pFVC ≤ 70% and pFVC ≤ 60% (ILD) in the next 6, 12, 18, and 24 months. We refer to the estimation of these event probabilities using our methodology as CVSP. ⁸ , ¹⁹ The CVSP algorithm sequentially produces the risks of events as additional data are observed in real‐time for a patient. The predictions are made without refitting the model to incorporate new observations for a patient using a cross‐validation method, yielding computational efficiency. We previously showed CVSP increases in precision as more data are observed for a given patient and that, even with no observations for an individual's measure, yields predictions with considerable precision compared with other methods, including random forest classification and logistic regression. ⁸ , ¹⁹

We added the cross‐validated statistical models that generate patient‐specific risk estimates of critical events to the prototype VAT. The feature is now being added to Patient Insight for external validation and calibration.

RESULTS

This study used clinical data of >4,000 patients with scleroderma. Details regarding cohort characteristics are provided in the Supplemental Materials. Figure 1 is a snapshot of Patient Insight currently used by physicians in the clinic to effectively assess patients’ past and current health status. A full view of the tool is provided in Supplemental Figure 3. In a single‐page view, the VAT displays an individual patient's longitudinal immunosuppression exposures, critical events, trajectory across multiple organ systems, disability index, and comorbidities. Users can select organ systems or variables of interest to only view selected data if desired.

Figure 1.

Patient Insight tool for scleroderma. This web application illustrates an individual patient's demographic and clinical disease manifestations and trajectory across multiple organ systems. A partial view with selected organ systems is shown in the figure. A complete view of the tool with all organ systems (cardiac, pulmonary, cutaneous, gastrointestinal, renal, peripheral vascular, and muscle) is shown in Supplemental Figure 3. The severity of disease state is indicated by red (severe) and pink (mild) regions for left ventricular ejection fraction and RVSP. Users can view values for each point by hovering over the points in the graphs. DLco, diffusing capacity for carbon monoxide; EF, ejection fraction; FVC, forced vital capacity; GLI, Global Lung Function Initiative; ILD, interstitial lung disease; mmf, mycophenolate mofetil; mRSS, modified Rodnan skin score; po, by mouth; RP, Raynaud phenomenon; RVSP, right ventricular systolic pressure; Rx, prescription; TFR, tendon friction rubs.

Trajectory relative to a reference population

An important feature of the VAT is presenting an individual patient's data relative to others with the same disease and specifically those who share key demographic, clinical, and biologic characteristics. Filters can be deployed to compare an individual patient to others who share risk factors that associate with disease severity in population studies, such as race and cutaneous and autoantibody type. Figure 2 demonstrates the use of this feature. In Figure 2A, the FVC trajectory of a White woman who developed diffuse scleroderma with anti–Scl‐70 antibodies at age 40 years is shown relative to the entire population of patients with scleroderma. In Figure 2B, her same trajectory is shown relative to other patients with similar demographic and clinical characteristics. After 10 years of disease, her pulmonary function has been poorer than the average patient with scleroderma but is typical of that expected for patients with the same race, disease subtype, and autoantibody type. The ability to interactively change the reference population provides insight into how individual and combinations of risk factors may modify a patient's likely trajectory and outcome; this approach furthers development of a continuous learning health system.

Figure 2.

Lung trajectories for an individual patient with interactive filters for the reference population. (A) Lung trajectories of a patient and tenth, median, and ninetieth percentiles reference lines of the overall scleroderma population. (B) Lung trajectories of the same patient and tenth, median, and ninetieth percentiles reference lines of selected subpopulation. On the top panel (A), this patient's trajectory is compared with that of the overall scleroderma center cohort. On the bottom panel (B), that same patient's trajectory is compared with a reference population that is similar to the patient (onset age of 30 to 50 years, diffuse type, and Scl‐70–antibody positive). These data illustrate how the reference population changes the interpretation and one's perspective. When comparing this patient to the overall scleroderma population, we observe that her percent predicted FVC trajectory declines from the ninetieth percentile to the tenth over 20 years of follow up, dropping rapidly below the fiftieth percentile line after 10 years. Relative to other similar patients, however, we see that her percent predicted FVC trajectory is better than that typically expected for the first 10 years of follow up and around the median afterwards. DLco, diffusing capacity for carbon monoxide; FVC, forced vital capacity.

Results from patient and provider surveys and interviews

Patient perspectives

Among seven patients, four were women and six had diffuse scleroderma, and 51 years was the median age. Self‐reported race and education level are provided in Supplemental Tables 1 and 2. Patients reported an increased understanding of their disease and potential prognosis after being exposed to Patient Insight (Supplemental Table 3). Figure 3 shows three main themes identified from the interviews with representative quotes. Many patients expressed that visualizing their data relative to similar patients increased their knowledge about their own disease. They shared that the tool improved recall of past symptoms and relayed clinically relevant patterns that were otherwise difficult to describe. They reported that Patient Insight would allow them to participate in their own medical care more actively, both in the context of clinical care and everyday modifiable behaviors. Some patients mentioned that the tool could help them make the most out of short appointments and increase the efficiency of medical visits. In summary, being exposed to Patient Insight satisfied patients’ curiosity about their disease state and enabled them to feel empowered dealing with their scleroderma.

Figure 3.

Themes identified from rheumatologist and patient interviews with representative quotes. Rheumatologists reported gaining new insights into a patient's disease, increased efficiency, and greater confidence in decision‐making by using the Patient Insight tool. Patients felt that using the tool increased knowledge, improved recall of their past symptoms, and instilled greater confidence in dealing with their disease. EF, ejection fraction; GI, gastrointestinal; JH, Johns Hopkins. Color figure can be viewed in the online issue, which is available at http://onlinelibrary.wiley.com/doi/10.1002/acr.25613/abstract.

Provider perspectives

Four rheumatologists participated in the study. They were in rheumatology practice (including fellowship) for 5 to 46 years, with 6 to 8 years of experience using Epic. Two described their comfort level with technology as “somewhat comfortable” and two as “very comfortable.”

The streamlining and display of historical patient data in Patient Insight resulted in more efficient examination of clinical cases (Supplemental Table 4). On average, providers accessed 8.6 times more data (t = −8.74; P < 0.001) in 81.5% of the time (t = 1.76; P = 0.123), using 66.7% fewer clicks (t = 4.17; P = 0.004). During interviews (Figure 3), the rheumatologists reported that Patient Insight helped them assess patients’ disease better, with increased efficiency and greater confidence in decision‐making. They expressed appreciation for having patient data in one place, highlighting the anticipated time‐savings and expressing excitement to share the tool with patients who value having their data presented clearly and concisely. The rheumatologists noted that they gained new insights into a patient's disease and were better able to refine diagnoses using Patient Insight. Those who recommended a change in testing or treatment reported higher levels of confidence in their decision when using Patient Insight versus Epic alone. Providers perceived usability of Patient Insight as “excellent” (SUS scores: mean 81.3 ± 7.5), expressing that the tool was easy to use, and they would like to use it frequently (Supplemental Table 5).

Estimating a patient's risk of critical events

Although visualizing a patient's historical trajectory offers tremendous value, our goal is to improve prediction of future events to make better medical decisions. We developed a prototype that enables us to estimate and visualize each patient's disease trajectory with the 95% prediction interval for pFVC, EF, and RVSP by jointly modeling multiple measures (Figure 4). The estimated risks of the three outcomes, each with two severity levels, in the next six months are displayed. We project the estimated trajectory forward and calculate the risks of the future events, shown by the shaded regions in the graphs and tables. The patient in Figure 4 is at high risk of progressive ILD (42% risk for pFVC ≤ 70%), cardiomyopathy (24% for LVEF < 50%; 8% for LVEF < 35%), and PH (29% for RVSP > 50 mmHg; 39% for RVSP ≥ 45 mmHg), suggesting more intensive screening and monitoring is warranted. The predictive analytics are now being incorporated into Patient Insight to test its value in clinical settings.

Figure 4.

Predictive modeling incorporated in prototype R Shiny App. The figures illustrate an individual patient's trajectory in pFVC, EF, and RVSP over time with the estimated risk of crossing high‐risk thresholds in these parameters within the next six months. The degree of uncertainty around these estimates is illustrated in the distribution of potential values projected forward at time 0 (the time of the current visit) and the estimated risks of crossing certain thresholds is shown in the tables on the right (denoted with lighter and darker shades of green and pink). The time of projection can be changed by the end user to illustrate predicted estimates at 6, 12, 18, or 24 months in the future. EF, ejection fraction; GI, gastrointestinal; HAQ, health assessment questionnaire; pFVC, percent predicted forced vital capacity; RP, Raynaud phenomenon; RVSP, right ventricular systolic pressure; TFR, tendon friction rubs.

An individual's estimated health trajectory and risks of critical events are calculated by harnessing information in baseline characteristics (age of scleroderma onset, sex, race, and cutaneous and autoantibody type) and longitudinal cardiopulmonary biomarkers (pFVC, pDLco, LVEF, and RVSP). In Figure 5, we display how using baseline characteristics and subsequently longitudinal biomarkers changes precision in predicted values. When no patient‐level information is available, our best estimate for pFVC one year from current visit for pFVC < 70% for very early disease (first 18 months) is depicted in Figure 5A. Probabilities of an individual having a clinically meaningful restrictive ventilatory defect suggestive of ILD is depicted by the areas under the curve. Because no individual‐level information is available, the predicted probabilities are derived from the cohort mean, and we observe a widely spread‐out bell‐shaped curve, reflecting high uncertainty in the predicted value.

Figure 5.

Risks of interstitial lung disease in the overall scleroderma population, within distinct subgroups, and for two individual patients. This series of plots show a gradual reduction in uncertainty for interstitial lung disease prediction as more information is introduced (left to right) for two separate scenarios (top and bottom row). Interstitial lung disease risks are indicated by the shaded regions and percentages. (A) shows expected pFVC distribution for a random patient with scleroderma without any baseline information or follow‐up data. The top plot in (B) shows the expected pFVC distribution for a random patient in a selected subgroup: Black women who developed diffuse scleroderma with anti–Scl‐70 antibodies. The top plot in (C) shows the interstitial lung disease risk predictions of a patient who has same baseline characteristics with her observed pFVC measurements. In contrast, the bottom plot in (B) shows the expected pFVC distribution for a random patient in a different subgroup: White men who developed limited scleroderma with anticentromere antibodies. The bottom plot in (C) shows the interstitial lung disease risk of a patient who has the same baseline characteristics with his own pFVC observations. Uncertainty in prediction, represented by the width of bell‐shaped prediction distributions, decreases when baseline information and subgroup characteristics are known and further decreases substantially when an individual patient's pFVC observations are used. pFVC, percent predicted forced vital capacity.

We then incorporate an individual's demographic and clinical characteristics to obtain subgroup‐specific ILD risk. The estimated risks of ILD for two selected subgroups are shown in Figure 5B: (1) Black women with diffuse scleroderma and anti–Scl‐70 (top) and (2) White men with limited scleroderma and anticentromere (bottom). The first subgroup has a higher ILD risk compared with the population‐level estimates in Figure 5A. The second subgroup has a lower projected ILD risk.

Individualized estimated ILD risks for two patients with very early disease from each of the subgroups are shown in Figure 5C. The top plot shows patient‐specific estimated ILD risk in the next year of an individual patient from the first subgroup. The bottom plot shows ILD risk of a patient from the second subgroup. Having pFVC observations >70% lowered the estimated patient‐specific ILD risks compared with subgroup‐specific risks for both patients. For the patient in the top plot, the estimated risk of pFVC ≤ 70% decreased from 66% to 15%. For the patient in the bottom plot, this decreased from 15% to 3%. Although the pFVC observations of the two patients are similar with upward trend, the patient in the top plot has a higher estimated ILD risk compared with the patient in the bottom plot given her baseline characteristics. This illustrates that the estimated trajectories harness information not only from each patient's pFVC observations but also from subgroup‐specific characteristics and other longitudinal biomarkers (eg, pDLco). Incorporating patient‐specific information results in increased precision of the predicted values shown by narrower bell‐shaped curves compared with Figure 5B.

Although the risks of critical events are calculated based on prespecified thresholds, risk estimates using different cutoffs can be readily calculated from our model. For instance, we can estimate an individual's risk of a 5% or 10% decline in pFVC, which could be useful for patients who have higher baseline pFVC or to identify patients who warrant more intensive monitoring, early therapeutic intervention, or consideration for clinical trial enrollment (Supplemental Figure 4).

Harnessing information across measures is particularly useful when some measures have fewer observations compared with others. In our case, patients generally have fewer observations for cardiac measures (EF and RVSP) and richer data for the pulmonary measures (pFVC and pDLco). When there are only sparse data observed for a measure or a patient, we borrow strength from other correlated measures and from the entire cohort to produce more precise estimates for a given parameter. ⁸ In Figure 6, we demonstrate how the model generates accurate risk estimates for PH using information not only in RVSP trajectory but also in pDLco, a richer measure. Two patients with similar clinical and demographic characteristics are depicted in Figure 6A and 6B, respectively. Both patients are White women with a similar age, skin subtype, and autoantibody status. Both have similar RVSP trajectories up until the time of the current visit, but the estimated risk of PH in the next two years for the patient in Figure 6A is much lower than the patient in Figure 6B. The patient in Figure 6A has an estimated risk of 14% for RVSP ≥ 45 mmHg, whereas the patient in Figure 6B has an estimated risk of 38%. This difference is driven by the patients’ pDLco trajectories. Compared with a similar reference group, the pDLco trajectory of the patient in Figure 6A is relatively stable across time, fluctuating close to the fiftieth percentile line. However, the pDLco trajectory in Figure 6B gradually declines over time and falls below the tenth percentile line for the reference group. Indeed, the patient in Figure 6B was subsequently confirmed to have PH by right heart catheterization. This demonstrates how the VAT maximally uses available data to aid a clinician's decision‐making.

Figure 6.

RVSP and pDLco trajectories and estimated risks of pulmonary hypertension of two individual patients. The tables in figures (A) and (B) show estimated risks of crossing high‐risk thresholds for PH in the next two years for two patients. Both patients are White women with diffuse cutaneous scleroderma, negative for anti–Scl‐70, anti–RNA polymerase III, and anticentromere. Onset ages are 51 and 55 years for patients in (A) and (B), respectively. The reference lines for pDLco trajectories are created using a reference population similar to the patients (White woman, onset age of 40 to 60 years, diffuse type, and Scl‐70, RNA polymerase III, anticentromere antibody negative). Smooth curves (green dotted lines) created using each patient's pDLco observations only are shown with their 95% confidence intervals to illustrate the two patients’ pDLco trends over time. pDLco, percent predicted diffusing capacity for carbon monoxide; RVSP, right ventricular systolic pressure.

DISCUSSION

In complex, multisystem diseases like scleroderma, there is often significant heterogeneity in patients’ clinical manifestations, disease course, and risk of complications. Clinicians try to incorporate knowledge from a patient's prior trajectory, the interplay across organ systems, and experiences from caring for other similar patients to make an informed estimate of a patient's likely disease course and need for high‐risk therapies. In rare or complex diseases with high clinical variability, many providers have insufficient experience to make informed decisions. Further, the challenges of information gathering and processing are substantial, and decision‐making may be particularly prone to cognitive biases, such as availability, base rate neglect, and commission biases. ²⁰ There is a major unmet need to develop tools that (1) aggregate patient‐level, multisystem data in a format that is easy for clinicians to access, (2) harness knowledge and known outcomes from other patients who share key characteristics, and (3) compute and display personalized risk estimates to improve decision‐making. ²¹

This study has several strengths. The novel patient‐level VAT shows one patient's trajectory across multiple organs in the context of a clinician‐selected, biologically relevant patient subset. The tool also provides precise individualized trajectory and future risk estimates by simultaneously modeling data in multiple organ systems, harnessing data within a patient, across organ systems, and across similar patients. This tool represents a shift in health informatics from a clinician having access to the current patient's data only to contextualizing that patient's data with the experience of a selected subset of prior patients. In our scleroderma example, we confirmed that visualizing historical data in Patient Insight can increase providers’ efficiency and enable them to see new trends and patterns in a patient's disease. We also found that the tool improves patients’ understanding of their disease. The absence of a cure or common scleroderma trajectory undermined many patients’ sense of certainty and control, which was mitigated by Patient Insight.

Limitations of this study include its small sample size, and we are expanding the study to include more rheumatologist and patient participants to test the value of embedded personalized risk estimates to aid in decision‐making. It will be important to determine whether the tool enhances a physician's judgment across a range of provider experience levels and whether the tool is interpretable by patients with varying degrees of health literacy. We internally validated the individualized prediction model before embedding it in the prototype, and the predictions’ value and usability will be tested after embedding it in the EMR. Further work is needed to externally validate and calibrate the prediction tools in diverse populations for them to be embedded in EMR systems for wide clinical use.

Development of the VAT required collective effort of clinicians, statisticians, software engineers, user design experts, and human‐factors engineers. Developing decision support tools, particularly those that embed predictive analytics, involves serial testing and improvement. We anticipate that the precision of these prediction models can be further improved by harnessing multicenter longitudinal data, adding other outcomes (eg, modified Rodnan skin score trajectory, severe gastrointestinal disease, renal crisis, and myopathy), and incorporating newly discovered molecular or imaging biomarkers. Important goals include modeling the effect of competing medication choices on a patient's likely trajectory and embedding decision support based on predicted probabilities of critical events into the VAT. We envision that sharing this tool with patients during clinical visits will allow patients to better understand their disease and the rationale for whether invasive testing or high‐risk medications are needed. Such a tool may improve medication adherence and shared decision‐making and reduce decisional regret for patients.

Personalized medicine strategies that harness and integrate knowledge within and across individuals have great potential to improve risk estimation and tailored decision‐making in complex diseases. Developing methods to bring new predictive models into clinical settings is critical to demonstrate the value of these tools, foster the creation of a continuous learning health system, and enable future dissemination. The VAT used in this study allows flexible parameterization and can be applied to display clinical data and model health trajectories for other complex diseases and from other data sources. We anticipate this framework will provide a foundation that can be scaled and generalized for multiple clinical applications and disease states.

AUTHOR CONTRIBUTIONS

All authors contributed to at least one of the following manuscript preparation roles: conceptualization AND/OR methodology, software, investigation, formal analysis, data curation, visualization, and validation AND drafting or reviewing/editing the final draft. As corresponding author, Drs Kim and A. A. Shah confirm that all authors have provided the final approval of the version to be published and take responsibility for the affirmations regarding article submission (eg, not under consideration by another journal), the integrity of the data presented, and the statements regarding compliance with institutional review board/Declaration of Helsinki requirements.

Supporting information

Disclosure Form:

Data S1 Supporting Information