How valid is a prescription-based multimorbidity index (Rx-risk) in predicting mortality in the Outcomes and Multimorbidity In Type 2 diabetes (OMIT) study? A nation-wide registry-based cohort study from Norway

Abstract

Objective

The prescription-based Rx-risk index has previously been developed to measure multimorbidity. We aimed to adapt and evaluate the validity of the Rx-risk index in prediction of mortality among persons with type 2 diabetes.

Design

Registry-based study.

Setting

Adults with type 2 diabetes in Norway identified within the ‘Outcomes and Multimorbidity In Type 2 diabetes’ cohort, with linkage to prescriptions from the Norwegian Prescription Database and mortality from the Population Registry.

Participants

We defined a calibration sample of 42 290 adults diagnosed with type 2 diabetes 1950–2013, and a temporal validation sample of 7085 adults diagnosed 2014–2016 to evaluate the index validity over time

Primary outcome measure

All-cause mortality

Methods

For the calibration sample, dispensed drug prescriptions in 2013 were used to define 44 morbidity categories. Weights were estimated using regression coefficients from a Cox regression model with 5 year mortality as the outcome and all morbidity categories, age and sex included as covariates. The Rx-risk index was computed as a weighted sum of morbidities. The validity of the index was evaluated using C-statistic and calibration plots.

Results

In the calibration sample, mean (SD) age at start of follow-up and duration of diabetes was 63.8 (12.4) and 10.1 (7.0) years, respectively. The overall C-statistic was 0.82 and varied from 0.74 to 0.85 when stratifying on age groups, sex, level of education and country of origin. In the validation sample, mean (SD) age and duration of diabetes was 59.7 (13.0) and 2.0 (0.8) years, respectively. Despite younger age, shorter duration of diabetes and later time period, the C-index was high both in the total sample (0.84) and separately for men (0.83) and women (0.84).

Conclusions

The Rx-risk index showed good discrimination and calibration in predicting mortality and thus presents a valid tool to assess multimorbidity among persons with type 2 diabetes.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study included individuals less than 65 years old, which are often not included in multimorbidity index development. The index was shown to be valid for all age groups included in the study, including younger individuals.

• A strength of this study is that the temporal validation sample included individuals who developed type 2 diabetes several years later than the index population, which confirmed index robustness over a time period with changes in prescription patterns.

• The large sample size made it possible to also evaluate the validity of the index in subgroups of age, sex, educational level and ethnicity.

• There is a risk of misclassification of specific morbidities since some drugs can be prescribed for more than one indication. However, we have limited this risk by utilising reimbursement codes for indication, whenever feasible.

• Some morbidities with high relevance for type 2 diabetes, such as obesity and mild to moderate kidney disease, are not easily captured using data on dispensed drug prescriptions.

Introduction

Multimorbidity (MM) is defined as having two or more chronic conditions¹ and has become a high priority area for healthcare providers and decision makers in relation to caring for people with type 2 diabetes^2–4 and simultaneously increasing treatment complexity.⁵ A key weakness in current clinical guidelines for treatment of type 2 diabetes is that they primarily build on evidence from clinical trials where patients with more complex disease burden are more likely to be excluded,^{6 7} and often fail to deliver clear advice on how to manage patients with more MM.^{8 9}

Most individuals with type 2 diabetes fulfil the criteria for MM. Their comorbidities are considered either concordant, that is, diseases that share pathophysiological pathways with diabetes, or discordant, that is, conditions whose treatments and pathophysiology are not directly related to diabetes.¹⁰ Individual patients with MM are at higher risk of polypharmacy, leading to higher risk of poor medication adherence and adverse drug-interactions, and sometimes conflicting treatment strategies.¹¹ MM is also associated with higher age, low socioeconomic status,¹² more frequent general practitioner visits,³ decreased quality of life and patient self-care¹³ and increased mortality.¹⁴

MM indices to measure the total burden of MM can be constructed by the use of diagnosis codes, examples of which are the Charlson,^{15 16} Elixhauser¹⁷ and the ICPC comorbidity indices.¹⁸ Similar indices can also be constructed based on records of prescribed and/or dispensed drugs, such as the Chronic Disease Score,¹⁹ which later has been renamed to the Rx-risk score, and modified several times.^{20 21} In the latest adaption by Pratt et al, individual drugs were identified based on Anatomical Therapeutic Chemical (ATC) classification codes, and subsequently a weighted Rx-index score on overall MM was computed based on 46 ATC-code-based comorbidity categories.²² Such MM indices can be used both to measure MM in research projects and for risk stratification in clinical practice to identify patients in need of more extensive follow-up. There are advantages and disadvantages both with prescription-based and diagnosis-based indices and the choice of the most appropriate index should be guided by the research question, data availability and study setting.

The Outcomes and Multimorbidity In Type 2 diabetes (OMIT) cohort is a new Norwegian registry-based observational cohort constructed to study high-risk patient groups with type 2 diabetes.²³ High-risk patients are often characterised by more complex disease patterns and by higher burden of MM.^24–26 Therefore, it is pivotal to construct a robust and valid measure of overall MM both to study the prevalence, incidence or the effects of MM in itself, but also to balance treatment groups when comparing the effect of treatment on an outcome. The objective of the present study is to adapt the previously described Rx-risk MM-index²² to reflect Norwegian clinical practice and validate its ability to predict mortality in persons with type 2 diabetes using data from the OMIT cohort. The validity in predicting mortality reflects the index’s ability to identify the most important diseases and give them appropriate weight in accordance with their prognostic importance.

Methods

Design and setting

OMIT is a cohort study including 57 515 persons with type 2 diabetes registered in the Norwegian Diabetes Register for Adults (NDR-A) during 2006–2019 and/or in the Rogaland-Oslo-Salten-Akershus-Hordaland (ROSA4) study during 2012–2014.²³

Study population

From the OMIT cohort, two separate study samples were defined: a calibration sample and a temporal validation sample. Individuals diagnosed with type 2 diabetes before 1 January 2014 and being alive and ≥18 years old by 1 January 2014 were included in the calibration sample and used to construct the index (flowchart in online supplemental figure 1). In order to be able to validate the index in a sample not used for construction of the index, we defined a temporal validation sample including individuals diagnosed during 2014–2016 who were alive and ≥18 years old by 1 January 2017. Since this validation sample had a shorter duration of diabetes, were prescribed drugs in a later time period and possibly had a better prognosis for some morbidities, it gave us the possibility to test if the index is robust for such variations. The final samples used for calibration and validation included 42 219 and 7085 individuals, respectively.

Other data sources

Data on dispensed drugs have been obtained through linkage with the Norwegian Prescription Database (NorPD), which contains information on all dispensed prescriptions in Norway from 2004 and onwards, and provides ATC-codes, date of dispensing and amount of drug dispensed. Linkage was done using the unique personal identification number available for all individuals living in Norway. In Norway, patients are entitled to reimbursement of treatment expenses if they use specific drugs for chronic conditions or severe illness. The registry contains data on both reimbursed and non-reimbursed dispensed prescriptions, but not over-the-counter drugs and not drugs administered during hospitalisations or in nursing homes. For reimbursed dispensed prescriptions, the indication for the drug is recorded as a reimbursement code. This is an ICD-10 (WHO International Statistical Classification of Diseases and Related Health Problems 10th Revision) diagnosis code if the drug is prescribed in specialist care and as an ICPC-2 (International Classification of Primary Care, second edition) diagnosis code if the drug is prescribed in primary care. Information on date of death, level of education and country of origin was obtained through linkage with data from Statistics Norway (SSB).²⁷

Definition of morbidities with modifications

We applied the methodology previously described by Pratt et al ²² to construct the Rx-risk index, although we found it necessary to implement some modifications. First, to minimise risk of backward identification of individual persons, we decided to refrain from obtaining information on drugs used for the treatment of the four relatively rare (in Norway) and potentially sensitive diseases: HIV, hepatitis B and C and tuberculosis. Second, four additional therapeutic areas with relevance for people with type 2 diabetes were included: antibiotics (excluding tuberculosis), auto-inflammatory/immune-suppression-dependent conditions, sleep disorders and endocrine antineoplastic therapies (online supplemental table 1). Thus, for the current ATC code-based Rx-risk index, we have included 46 different therapeutic areas. Third, we consulted Norwegian specialists in the field of internal medicine (TJB), psychiatry (JIR) and general practice (AKJ and ESB) (see author list) to secure that all ATC-code disease categories were defined in accordance with the current Norwegian clinical practices. This implied reviewing both certain combinations of drugs as well as reimbursement codes to get the most appropriate classification of drugs which can be prescribed for more than one indication/disease. To separate between conditions like hypertension, angina and heart failure, which all could be treated with diuretics, beta-blockers, calcium-channel blockers and ACE inhibitors/angiotensin II receptor blockers, we looked at combinations of drugs. For instance, if a person had a dispensed prescription for the loop-diuretic furosemide (ATC C03CA01), the person also needed to have a dispensed prescription for ACE-inhibitors and beta-blockers the same calendar year in order to be defined as having heart failure. If no ACE-inhibitors or beta-blockers were dispensed, the person was categorised as having hypertension. Valproic acid (ATC N03A G01) is another example of a drug prescribed for more than one indication. We categorised a person with a dispensed prescription with valproic acid as having epilepsy if an epilepsy diagnosis was registered as the reimbursement code, and as bipolar disorder if this was registered as reimbursement code. If no reimbursement code was registered, we categorised the person according to the most common indication in Norway, which in case of valproic acid is epilepsy. The final list of 46 morbidities and definitions are defined in online supplemental table 1 (A–N).

A person was required to have ≥1 drug dispensing per calendar year to be defined as having the morbidity in question. Morbidities, per 1 January 2014 and 1 January 2017 in the calibration and validation sample, were defined by assessing dispensed drugs in 2013 and 2016, respectively.

Statistical analysis

Sociodemographic characteristics of the patients in the calibration sample and the validation sample were calculated using means and SD for continuous variables and counts and percentages for categorical variables.

Weights for each comorbidity were calculated based on associations with mortality in the calibration sample. In previous validation studies of the Rx-risk index, the contribution of each individual disease entity was weighted according to the strength of association with 1 year mortality.^{22 28} In the current study, we calculated weights based on 5 year mortality since the study population was younger compared with previous studies and therefore expected to have lower mortality. Follow-up time for each person in the calibration sample was calculated from 1 January 2014 until death, emigration or censoring with a maximum of 5 years of follow-up. Among the 46 investigated comorbidities, malnutrition and pulmonary hypertension did not have sufficient data to estimate associations with mortality. We therefore estimated a multivariable Cox proportional hazard model with 44 comorbidities included as binary covariates, in addition to continuous age and sex, and report the results as HR with 95% CIs. For pain we included an overall binary variable instead of separate variables for severe and moderate pain. To get an algebraically correct index, we computed weights based on the regression coefficients from the Cox model (the natural logarithm of the HRs) and not the HRs.²⁹ In accordance with previous versions of the Rx-risk score, regression coefficients were scaled to get a maximum weight of 6 and rounded, and the Rx-risk index was then calculated as a weighted sum of the 46 binary comorbidity variables for each individual.

We also did sensitivity analyses where we defined the presence of morbidities as ≥2 dispensed prescriptions per year to investigate if this had any impact on the association with mortality

The ability of the index to predict mortality was evaluated both in the calibration sample used to estimate the index and in the temporal validation sample. In the calibration sample, the index was included as a continuous variable in a Cox regression model with 5 year mortality as the outcome, in addition to age and sex. Age was centred before estimating the Cox-model to get reasonable estimates of the baseline hazard in calculation of predicted mortality risk. Interaction terms between the index and age and sex were included if p<0.05. As a measure of discrimination, Harrel’s C index³⁰ was calculated for the model and compared against the C-index for a model only containing age and sex. The C-index is a measure of the model’s ability to rank individuals according to time to death when comparing all possible pairs of patients. For instance, in a study sample of 10 individuals, there are 45 possible pairs of patients. A C-index of 0.9 would mean that when comparing all possible pairs of patients, the person who dies first within a pair will have the highest predicted risk for 90% of the comparisons. Values above 0.7 are considered acceptable and values above 0.8 are considered excellent.

In addition, we calculated predicted 5 year mortality risk for each individual by first estimating the 5 year baseline survival function at 5 years of follow-up and using the following formula:³¹

$${Risk}_{5year}=\left(1-{Basesurv}_{5years}\bullet e^{X\bullet \beta }\right)\bullet 100$$

Basesurv_5years is the estimate of the baseline survival after 5 years, X is a vector of covariate values and β is a vector of regression coefficients for the covariates from the Cox model. The included covariates were the continuous Rx-risk index, age, sex and interaction terms if significant in the Cox model. Average predicted risk for each level of the Rx-risk index was plotted together with observed risk to evaluate if predictions were biased for low or high values of the index.

We used a similar procedure in the validation sample. The ability of the index to predict mortality was evaluated by calculating individual Rx-risk values using the weights obtained from the calibration sample and include the index together with age and sex in a Cox regression model with 3 year mortality as the outcome. A shorter follow-up time was used because of lack of 5 year data for the validation sample.

The predictive ability was assessed both in the total calibration and validation samples and in subgroups defined by age, sex, duration of diabetes, level of education and country of origin.

All statistical analyses were done using Stata V.17.

Patient and public involvement

Norwegian Diabetes Association has written a letter of endorsement and granted financial resources to support the OMIT initiative. The OMIT study group will ensure key research findings and key takeaways with relevance for patients will be communicated to patients, using both the webpage and the membership magazine of Norwegian Diabetes Association.

Results

Table 1 provides the basic characteristics for the calibration and validation sample. In the calibration sample, mean (SD) age and diabetes duration at start of follow-up was 63.8 (12.4) and 10.1 (7.3) years, respectively.

Table 1.

Description of calibration sample (n=42 290) and temporal validation sample (n=7085)

	Total	Men	Women
Calibration sample (diagnosed before 2014)
n	42 290	24 285	18 005
Age in 2014, mean (SD)	63.8 (12.4)	63.3 (11.7)	64.9 (13.0)
Years since diagnosis, mean (SD)	10.1 (7.3)	9.9 (7.3)	10.3 (7.5)
Education, n (%)
Compulsory	15 193 (35.9)	7974 (32.8)	7219 (40.1)
High school	19 045 (45.0)	11 374 (46.8)	7671 (42.6)
College/university	7308 (17.3)	4619 (19.0)	2689 (14.9)
Unknown/missing	744 (1.8)	318 (1.3)	426 (2.4)
Country of origin, n (%)
Norway	35 995 (85.1)	20 731 (85.4)	15 264 (84.8)
Europe	2233 (5.3)	1368 (5.6)	865 (4.8)
Africa	698 (1.7)	430 (1.8)	268 (1.5)
Asia inkl.Turkey	2884 (6.8)	1499 (6.2)	1385 (7.7)
Other	477 (1.1)	255 (1.1)	222 (1.2)
Temporal validation sample (diagnosed 2014–2017)
n	7085	4272	2813
Age in 2017, mean (SD)	59.7 (13.0)	59.2 (12.4)	60.4 (13.9)
Years since diagnosis, mean (SD)	2.0 (0.8)	2.0 (0.8)	2.1 (0.8)
Education, n (%)
Compulsory	2457 (34.7)	1452 (34.0)	1005 (35.7)
High school	3127 (44.1)	1950 (46.7)	1177 (41.8)
College/university	1349 (19.0)	806 (18.9)	543 (19.3)
Missing/unknown	152 (2.2)	64 (1.5)	88 (3.1)
Country of origin, n (%)
Norway	5760 (81.3)	3520 (82.4)	2240 (79.6)
Europe	460 (6.5)	279 (6.5)	181 (6.4)
Africa	188 (2.7)	118 (2.8)	70 (2.5)
Asia inkl.Turkey	556 (7.9)	286 (6.7)	270 (9.6)
Other	119 (1.7)	68 (1.6)	51 (1.8)

The individuals in the validation sample were younger, had a shorter duration of diabetes, a higher proportion with college/university education and higher proportion of patients born outside Norway. All differences were statistically significant (p<0.001). In both samples, women were older and had a higher proportion with only compulsory education compared with men.

The prevalence of various comorbidities in the calibration sample, the related HRs from the Cox model used to develop the index and the calculated weights are reported in table 2.

Table 2.

Prevalence of morbidities (measured as at least one dispensed prescription in 2013), mortality and weights for different morbidities in the calibration sample (n=42 290)

Morbidity	Prevalence, n (%)	5 year mortality, n (%)	HR (95% CI)*	Weight†
Alcohol dependency	198 (0.5)	13 (6.7)	1.45 (0.84 to 2.51)	*
Allergies	7137 (16.9)	407 (5.7)	0.84 (0.76 to 0.94)	−1
Anticoagulants	3881 (9.2)	758 (19.5)	1.59 (1.44 to 1.75)	†
Antiplatelets	17 516 (41.4)	1629 (9.3)	1.10 (1.01 to 1.19)	0
Anxiety	4332 (10.2)	484 (11.2)	1.13 (1.02 to 1.26)	*
Arrhythmia	2143 (5.1)	368 (17.2)	1.38 (1.23 to 1.56)	*
Benign prostatic hyperplasia	2003 (4.7)	245 (12.2)	0.83 (0.72 to 0.95)	−1
Bipolar disorder	427 (1.0)	33 (7.7)	1.19 (0.84 to 1.71)	*
Chronic airway disease	5976 (14.1)	585 (9.8)	1.28 (1.17 to 1.41)	*
Congestive heart failure	2273 (5.4)	315 (13.9)	1.40 (1.23 to 1.58)	*
Dementia	170 (0.4)	91 (53.5)	3.78 (3.06 to 4.68)	5
Depression	5096 (12.1)	474 (9.3)	1.31 (1.17 to 1.45)	*
Diabetes ‡	32 859 (77.7)	2391 (7.3)	1.14 (1.04 to 1.24)	0
Epilepsy	678 (1.6)	97 (14.3)	1.87 (1.52 to 2.30)	†
Gastrooesophageal reflux	9489 (22.4)	893 (9.4)	1.04 (0.96 to 1.13)	0
Glaucoma	101 (0.2)	16 (15.8)	1.88 (1.15 to 3.07)	†
Gout disease	2242 (5.3)	363 (16.2)	1.44 (1.28 to 1.61)	*
Hyperkalaemia	27 (0.1)	9 (33.3)	2.52 (1.27 to 5.00)	‡
Hyperlipidaemia	25 872 (61.2)	2027 (7.8)	0.89 (0.82 to 0.96)	0
Hypertension	27 917 (66.0)	2414 (8.7)	1.08 (0.99 to 1.18)	0
Hyperthyroidism	89 (0.2)	8 (9.0)	1.02 (0.51 to 2.05)	0
Hypothyroidism	4032 (9.5)	370 (9.2)	1.02 (0.91 to 1.14)	0
Ischaemic heart disease, angina	3048 (7.2)	439 (14.4)	1.01 (0.90 to 1.12)	0
Ischaemic heart disease, hypertension	11 350 (26.8)	1262 (11.1)	1.20 (1.10 to 1.30)	*
Incontinence	1440 (3.4)	166 (11.5)	1.05 (0.90 to 1.23)	0
Inflammation/pain	10 741 (25.4)	492 (4.6)	0.76 (0.69 to 0.84)	−1
Irritable bowel syndrome	60 (0.1)	7 (11.7)	0.78 (0.37 to 1.66)	0
Liver failure	191 (0.5)	55 (28.8)	2.16 (1.65 to 2.84)	‡
Malignancies	85 (0.2)	34 (40.0)	5.81 (4.11 to 8.21)	6
Malnutrition	0 (0.0)	0
Migraine	611 (1.4)	11 (1.8)	0.44 (0.24 to 0.79)	−3
Osteoporosis/Paget’s disease	725 (1.7)	137 (18.9)	1.30 (1.09 to 1.56)	*
Parkinsons	391 (0.9)	61 (15.6)	1.39 (1.07 to 1.79)	*
Pain mild/moderate/severe	13 249 (31.3)	1225	1.27 (1.17 to 1.38)	*
Pain mild/moderate§	7759 (18.4)	808 (10.4)
Pain severe§	9445 (22.3)	879 (9.3)
Pancreatic insufficiency	229 (0.5)	52 (22.7)	2.51 (1.89 to 3.32)	‡
Psoriasis	630 (1.5)	39 (6.2)	0.90 (0.65 to 1.23)	0
Psychotic illness	1719 (4.1)	177 (10.3)	1.77 (1.50 to 2.09)	†
Pulmonary hypertension	2 (0.0)	2 (100)
Renal disease	134 (0.3)	57 (42.5)	3.24 (2.41 to 4.36)	§
Smoking cessation	383 (0.9)	23 (6.0)	1.56 (1.03 to 2.37)	†
Steroid responsive disease	3173 (7.5)	413 (13.0)	1.20 (1.06 to 1.34)	*
Transplant	159 (0.4)	30 (18.9)	1.50 (1.01 to 2.23)	†
Infections treated with antibiotics (excl. tuberculosis)†	14 359 (34.0)	1330 (9.3)	1.18 (1.09 to 1.28)	*
Autoinflammatory/immune-suppression-dependent conditions†	1199 (2.8)	91 (7.6)	0.91 (0.74 to 1.14)	0
Endocrine antineoplastic therapies†	453 (1.1)	118 (26.1)	2.15 (1.10 to 1.11)	‡
Sleep disorders†	7126 (16.9)	802 (11.3)	1.05 (0.96 to 1.15)	0

*HR and 95% CIs from a Cox proportional hazards model with 5 year mortality as the outcomes and 44 morbidities included as covariates in addition to age and sex. Malnutrition and pulmonary hypertension were not included in the model because of lack of data.

†Regression coefficient (log(HR)) rescaled to a maximum of 6 and rounded to the nearest integer.

‡Use of insulin or glucose lowering drugs.

§Additional disease/drug groups not included in original Rx-risk index.

In total, 77.7% were medically treated with antihyperglycemic drugs or insulin and the percentage who were treated for hypertension, hyperlipidaemia or with antiplatelet drugs were 66%, 61% and 41.4%, respectively. The strongest associations with mortality were observed for malignancies (HR (95% CI))=5.81 (4.11 to 8.21)) and dementia 3.78 (3.06 to 4.68)). Some morbidities were negatively associated with mortality after adjustment for other morbidities, resulting in a negative weight for this morbidity.

Figure 1 illustrates the distribution of the Rx-risk index in the calibration and validation sample, respectively. Mean (SD) Rx-risk index was 1.4 (1.9) in the calibration and 1.2 (1.9) in the validation sample (p-diff<0.001), respectively. In both samples, <1% of individuals had values <−2 and <1% had values >7.

Figure 1.

Distribution of the weighted Rx-risk score in the calibration sample (n=42 290) and the temporal validation sample (n=7085).

The C-index (95% CI) was 0.783 (0.774 to 0.791) in the calibration sample when only using age and sex in the Cox model to predict mortality (table 3).

Table 3.

Prediction performance measured with Harrel’s C-index in the calibration sample and the temporal validation sample

	C-index base model (95% CI)*	C-index with Rx-risk†
Calibration sample‡
Total sample	0.783 (0.774 to 0.791)	0.818 (0.811 to 0.826)
By gender
Men	0.771 (0.760 to 0.781)	0.808 (0.798 to 0.818)
Women	0.798 (0.786 to 0.811)	0.832 (0.821 to 0.843)
By age group (age in 2014)
<65 years	0.644 (0.621 to 0.668)	0.733 (0.710 to 0.758)
≥65 years	0.718 (0.708 to 0.728)	0.768 (0.758 to 0.777)
By gender and age group
Men<65 years	0.632 (0.603 to 0.662)	0.719 (0.692 to 0.750)
Women<65 years	0.653 (0.613 to 0.692)	0.746 (0.703 to 0.789)
Men≥65 years	0.697 (0.682 to 0.711)	0.753 (0.741 to 0.766)
Women≥65 years	0.741 (0.725 to 0.756)	0.784 (0.770 to 0.797)
By duration of type 2 diabetes
<10 years	0.784 (0.770 to 0.797)	0.817 (0.806 to 0.830)
≥10 years	0.764 (0.754 to 0.775)	0.803 (0.793 to 0.813)
By educational level
Compulsory	0.775 (0.763 to 0.788)	0.812 (0.801 to 0.823)
High school	0.783 (0.770 to 0.795)	0.814 (0.802 to 0.826)
College/university	0.785 (0.762 to 0.808)	0.824 (0.803 to 0.845)
By country/world region of origin
Norway	0.774 (0.765 to 0.782)	0.812 (0.804 to 0.820)
Europe/Americas/Oceania	0.734 (0.684 to 0.784)	0.753 (0.705 to 0.801)
Africa/Asia	0.813 (0.764 to 0.863)	0.848 (0.801 to 0.894)
By age at diagnosis of type 2 diabetes
<40 years	0.795 (0.760 to 0.830)	0.825 (0.790 to 0.861)
40–69 years	0.744 (0.733 to 0.755)	0.790 (0.780 to 0.800)
≥70 years	0.722 (0.706 to 0.738)	0.760 (0.745 to 0.775)
Temporal validation sample§
Total sample	0.786 (0.751 to 0.821)	0.836 (0.804 to 0.868)
By gender
Men	0.780 (0.736 to 0.825)	0.830 (0.790 to 0.869)
Women	0.793 (0.736 to 0.851)	0.843 (0.789 to 0.898)
By age group (age in 2017)
<65 years	0.746 (0.663 to 0.828)	0.806 (0.724 to 0.887)
≥65 years	0.698 (0.644 to 0.753)	0.755 (0.708 to 0.802)
By gender and age group
Men<65 years	0.748 (0.669 to 0.826)	0.837 (0.758 to 0.915)
Women<65 years	0.602 (0.374 to 0.830)	0.705 (0.504 to 0.906)
Men≥65 years	0.739 (0.678 to 0.802)	0.761 (0.700 to 0.822)
Women≥65 years	0.645 (0.553 to 0.737)	0.763 (0.691 to 0.835)

*Cox model only including age, sex (and interaction between age and sex if p<0.05).

†Cox-model with age, sex and Rx-risk index as covariates. Interaction terms included if p<0.05.

‡Cox-model for 5 year mortality.

§Cox-model for 3 year mortality.

This improved to 0.818 (0.811–0.826) when including the Rx-risk index in the model in addition to age and sex. The C-index with Rx-risk included was above 0.70 in all subgroups when stratifying on sex, age, duration of type 2 diabetes, level of education and country of origin and was always improved compared with models including only age and sex.

In the validation sample, the C-index (95% CI) for the Cox-model including the Rx-risk index was 0.836 (0.804 to 0.868) and above 0.70 also in all subgroups of age and sex.

Figure 2 shows plots of average predicted mortality risk versus observed mortality as a measure of calibration, together with a distribution of number of deaths for each level of Rx-risk score in the calibration versus validation sample. Figure 3 shows results for the calibration sample stratified by level of education (figure 3A) and country of origin (figure 3B). Generally, there was a good agreement between predicted versus observed risk, except for Rx-risk values >5 in the validation sample and in those of the calibration sample with immigrant background or university/college education, likely due to a low number of observed deaths. Also, when stratifying on age and sex in the calibration sample, the agreement between predicted and observed mortality risk was good for Rx-risk values ≤5 (online supplemental figure 2).

Figure 2.

Observed and predicted risks in the calibration sample and the temporal validation sample (values of Rx-risk <−2 and >8 are truncated). The bar chart displays the percentage of individuals in the sample who are in each Rx-risk category.

Figure 3.

Observed and predicted risks by (a) level of education and (b) country of origin in the calibration sample (values of Rx-risk <−2 and >8 are truncated). The bar chart displays the percentage of individuals in the sample who are in each Rx-risk category.

Results for persons diagnosed before age 40 years (n=5596) and after age 75 years (n=1979) are reported in online supplemental figure 3. Also, in these two subgroups, the agreement between predicted and observed mortality risk was generally good, although the performance was poorer at the extreme ends of the Rx-risk distribution with few individuals and few observed deaths.

In sensitivity analyses with the presence of morbidities defined as at least two dispensed predictions, the prevalence of several of the comorbidities was reduced, especially sleep disorders and infections treated with antibiotics, but the prediction of mortality was not improved (results now shown).

Discussion

In this study, we adapted and validated the Rx-risk comorbidity index in a real-world population with type 2 diabetes. Generally, we found good agreement between observed and predicted mortality both in the calibration sample and the validation sample across the range of computed Rx-risk scores. There were a few exceptions in the higher Rx-risk-score strata in some of the smaller subgroups, where the number of study subjects were too low to perform a sound comparison. The ability of the Rx-risk to predict 5 year mortality generally remained after stratification on sex, diabetes duration, socioeconomic status (as measured by education) and ethnicity. Finally, the Rx-risk index demonstrated to be valid when stratifying on age at time of measurement of comorbidities and age at time of type 2 diabetes diagnosis.

The large sample size of the OMIT cohort is a key strength, enabling us to validate the Rx-index by the use of both a large calibration sample and a temporal validation sample. Furthermore, the index was shown to be robust to temporal changes in prescription patterns and associations between morbidities and mortality. Also, the fact that the validity of 3 year mortality prediction in the validation sample is high, even though the weights used are based on association with 5 year mortality, shows that the index is valid for prediction of mortality in general and not only for a specific length of follow-up. A key strength of the current study is that the Rx-risk was proven valid in patients with age<65 years at the time when comorbidities were measured, a feature that has not been documented previously for the Rx-risk index.^{22 28} Furthermore, the assessment of the validity of the Rx-index in subgroups with different educational levels, ethnicity and age at the time of diabetes onset adds further strength to the study.

Another strength is the employment of medication reimbursement codes, which helped us to increase the specificity and correctness of how we define different severe diseases. Examples include the distinction between severe pain and epilepsy for the drug gabapentin, or bipolar disorder and epilepsy in relation to valproic acid.

An intrinsic weakness of the Rx-risk index is that ATC codes refer to specific drugs and not diseases. The index is therefore not optimal for diseases where there is no specific drug treatment, for example, non-alcoholic hepatic steatosis, diseases where non-pharmaceutical interventions are often used (anxiety and depression) and hospital-treated diseases (cancer). In Norway, drugs administered in hospitals and nursing homes are not registered in NorPD, which would cause an underestimation of MM if persons are hospitalised for a long period or move to a nursing home, but this is unlikely a problem in this study as less than 10% were over 80 years old.

Another potential limitation is that the current Rx-risk index does not account for multiple prescriptions per year versus a single prescription, or dosing strength, which could indicate disease severity. Nevertheless, when we computed the weighted Rx-index based on a requirement of at least two as compared with one dispensation(s) per year, we found no differences in terms of the overall ability for the index to predict 5 year mortality.

Although the Rx-risk index has certain limitations, it is important to emphasise that diagnosis-code based indices of MM also have some built-in weaknesses. For instance, comorbidities, such as depression, anxiety, hypertension, insomnia and hyperlipidaemia, may be present as stable conditions, which, although being medically treated, are not subject for frequent follow-up visits and therefore may not be registered for long periods of time. But it is still possible to identify these comorbidities based on ongoing prescriptions. The optimal solution may be to combine diagnostic codes with dispensed drugs, in line with the Nordic multimorbidity index by Kristensen et al,³² but such data are not currently available in the OMIT study. As part of an updated linkage, we plan to supplement the OMIT cohort with additional data from the Norwegian Patient Registry,²³ which will allow for this type of analysis.

Further limitations include a lack of an external sample for the validation, and we were not able to validate the Rx-index in persons younger than 40 years old due to a very low number of deaths. Furthermore, while reimbursement codes may have increased the internal validity, many countries do not have this information in the prescription databases, and hence some may argue that the inclusion of reimbursement codes may at the same time diminish the external validity. However, prescription patterns, clinical practices and reimbursement rules often differ substantially between countries and/or patient populations, so the Rx-risk will always need some degree of customisation.

Despite these limitations, the current prescription-based index seems to perform on par with previously published diagnosis-based indices, which have demonstrated C-indices above 0.80 with regards to its ability to predict mortality in a population with type 2 diabetes.³²

The current study outlines for the first time a methodology by which a record of ATC codes can be used to construct a validated index on MM in a population with type 2 diabetes. The Rx-risk index provides an opportunity to describe the overall burden of MM as well as the prevalence of the specific comorbidities, and can also be useful for risk stratification in clinical practice to identify patients at particularly high risk. It is a useful tool to adjust for confounding in observational studies and for balancing treatment groups in observational data to emulate target clinical trials to study the effect of various treatment regimens.³³ The Rx-risk index will especially offer a great value in studies on high-risk patients with type 2 diabetes because these patients are often not included in clinical trials, and emulated target trials within observational studies may be the only option to obtain reasonable treatment effects.

Data availability statement

Data are available upon reasonable request. Data may be obtained from a third party and are not publicly available. Because of personal data protection regulations in Norway, no data are available. In support of collaborative research projects, access can be granted after approval by the OMIT-study group, the Regional Committee for Medical and Health Research Ethics and the data owner at the University of Oslo. Project requests can be directed to Esben Selmer Buhl at the Department of General Practice, Institute of Health and Society, University of Oslo (email: e.s.buhl@medisin.uio.no)

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

OMIT has obtained ethical approval from Regional Committees for Medical Research Ethics—South East Norway (REC South East (Ref# 74012), approval valid from 20 February 2020 until 31 December 2035) and is approved by the Data Protection Official/Officer at UiO.