Medical code embeddings from claims-based co-occurrences: a unified semantic space for ICD-10 diagnoses and ATC medications

Abstract

Objective

The analysis of care trajectories derived from electronic health records and claims data has become increasingly common in biomedical informatics. This has enabled large-scale studies of care processes, yet widely used binary code representations result in high-dimensional, sparse data that fail to capture semantic relationships between medical concepts. Learning dense vector representations (embeddings) has emerged as a promising approach to address these limitations. We aimed to construct and share joint embeddings for the International Classification of Diseases (ICD-10) and the Anatomical Therapeutic Chemical (ATC) classification system, providing reusable semantic representations of diagnoses and treatments from real-world claims data.

Materials and Methods

Using claims records from 1.5 million patients, we defined code co-occurrences within temporal windows and constructed a Positive Pointwise Mutual Information (PPMI) matrix spanning ICD-10 and ATC codes. Singular Value Decomposition (SVD) was applied to derive a low-dimensional embedding space. Evaluation combined UMAP visualization, nearest-neighbor retrieval, and a code-level classification task based on ICD chapters and ATC classes.

Results

The embeddings reflected the hierarchical organization of ICD-10 and ATC and revealed associations across coding systems, including clinically relevant diagnosis-treatment relationships. The classification task achieved mean AUCs of 0.93 for ICD-10 and 0.90 for ATC, indicating strong grouping of semantically related codes.

Discussion

The embeddings provide a reusable, code-level semantic representation that can support code retrieval, reduce manual code grouping, and be aggregated into patient-level features without training a task-specific model.

Conclusion

We release the first openly available joint ICD-10–ATC embedding space derived from real-world claims data, providing a reusable resource for biomedical informatics research.

Introduction

The use of electronic health records (EHRs) and insurance claims data for secondary research has increased in recent years.¹ The routine collection of data by insurance companies and national health systems provides information to study care trajectories. Claims databases contain the coded history of patient medical events, along with their associated timestamps. These events include diagnoses, prescribed treatments, procedures, medical devices, and are described using medical codes from classification systems and terminologies, such as ICD-10, ATC, and RxNorm.

Modeling patients’ care pathways within large-scale health data is of major interest, as it can help address important questions, such as which trajectories are associated with better outcomes, the natural history of diseases, or adherence to clinical recommendations. However, extracting such pathways remains a challenge given the complexity of claims databases. The temporal structure, heterogeneity, and sparsity of claims data pose significant methodological challenges,² particularly due to the large number of medical codes used to describe clinical events. This high cardinality problem makes prior selection and grouping of codes necessary, which can be time-consuming and error-prone. In addition, many codes are related to the same underlying medical concept, eg the ICD-10 code for type 2 diabetes is related to the ATC code for metformin. These semantic relationships are usually not captured by traditional analysis tools.

A variety of statistical learning methods have been applied to analyze care trajectories, including feature-based, distance-based, and model-based methods.³ Feature-based approaches have been applied, for example, by Righolt et al⁴ to study metformin prescription patterns. Distance-based analyses have been used by Vanasse et al⁵ to investigate care trajectories in chronic obstructive pulmonary disease. Model-based strategies, such as mixtures of Hidden Markov Models, were applied by Hilton et al⁶ to characterize subgroups of children with asthma.

While these approaches have provided valuable insights, they share a key limitation: they usually require manual selection of medical codes or key elements of the care pathway. Given the large number of codes and events in claims and EHR data, this requirement severely restricts scalability and risks overlooking clinically meaningful patterns. A central reason for this limitation lies in the sparse vector representations commonly used to encode medical codes, such as bag-of-words or TF-IDF schemes.^7,8 In such representations, codes are treated as independent features, ignoring their semantic relationships. As a result, clinically related events or pathways may appear unrelated, limiting meaningful comparison.

Recent work has therefore explored dense vector representations (embeddings) to represent medical codes.⁹ These approaches map each code to a low-dimensional vector such that semantically related codes are positioned close to one another in the latent space. Because they can be learned from standardized code co-occurrences rather than free text, such embeddings are not tied to a specific natural language. These embeddings capture semantic relationships between codes and can serve as inputs for downstream tasks. In addition, they facilitate exploration of code relationships and enable similarity-based retrieval of clinically related medical concepts.

In many recent works, representations are learned within predictive models, where embeddings are constructed in the hidden layers of neural networks and optimized for a specific outcome.¹⁰ While effective, the resulting representations are inherently task-specific and may not transfer easily across applications without retraining.¹¹ In contrast, unsupervised embeddings are learned from large collections of unlabeled care sequences independently of a prediction objective. As a result, they capture more general semantic structure in the data and can be reused as feature representations across downstream tasks.^12–14

A number of works have applied unsupervised methods to obtain embeddings for medical codes. Kartchner et al¹⁵ used GloVe¹⁶ on diagnosis codes, Choi et al¹⁷ applied Skip-Gram¹⁸ to ICD-9 and CPT procedure codes, and Hong et al¹⁹ employed PMI factorization to embed PheCodes and RxNorm codes in a shared space.

Despite these advances, there are currently, to our knowledge, no pre-trained, open-source embeddings that jointly represent codes from the International Classification of Diseases, Tenth Edition (ICD-10) and the Anatomical Therapeutic Chemical (ATC) classification system, 2 standardized coding systems widely used in EHR and claims data. For this reason, the objective of this work was to obtain unsupervised, task-agnostic, static embeddings that capture semantic relationships across both coding systems. These representations can contribute to digital health applications such as phenotyping, risk prediction, and treatment pattern analysis, by providing reusable feature representations that reduce the need for manual code grouping. In particular, they enable patient-level representations through simple aggregation of code embeddings.

This work is organized as follows. We first present the embedding method. We then describe the evaluation process, combining qualitative analyses of the embedding space with quantitative assessment through code-level classification tasks. In addition, a patient-level classification example based on longitudinal claims data is provided in the Supplementary Material to illustrate how the embeddings can be used in a real-world setting. Finally, we discuss the main findings, limitations, and directions for future work.

Methods

This section describes the methodological approach, comprising 3 components: (i) the claims data structure we consider, (ii) construction of code embeddings from claims data using Singular Value Decomposition on the Positive Pointwise Mutual Information matrix (PPMI-SVD), and (iii) their evaluation through qualitative and quantitative analyses.

Claims data structure

We consider claims data organized as patient-level sequences of time-stamped medical events. Each event corresponds to a diagnosis, treatment, procedure, or device, encoded using standard medical classification systems. For embedding computation, we use only the event codes and their temporal ordering.

Embedding through PPMI-SVD

The PPMI-SVD approach aims to capture semantic relationships between medical codes through their co-occurrence patterns. Each embedding vector represents the association profile of a code with all others, so codes are considered similar when these profiles are alike. This reflects second-order similarity rather than direct correlation. Mutually exclusive events such as alternative treatments for the same condition may be embedded close together if they occur in comparable clinical contexts. Additional details on the data structure and co-occurrence computation are provided in File S1.

PPMI matrix construction

Let $C\left(i,j\right)$ denote the number of times that codes $i$ and $j$ co-occur within the same temporal context window across all care sequences. Let $C\left(i\right)$ and $C\left(j\right)$ represent the marginal counts of codes $i$ and $j$, respectively.

The total number of co-occurrence observations is given by:

$$N=\sum _{i,j}C\left(i,j\right)$$

These counts provide empirical estimates of the underlying probabilities:

$$P\left(i,j\right)=\frac{C\left(i,j\right)}{N},\hspace{0.5em}P\left(i\right)=\frac{C\left(i\right)}{N},\hspace{0.5em}P\left(j\right)=\frac{C\left(j\right)}{N}$$

The Pointwise Mutual Information (PMI) between codes $i$ and $j$ is defined as:

$$\mathit{PMI}\left(i,j\right)=\log \frac{P\left(i,j\right)}{P\left(i\right)P\left(j\right)}$$

A higher PMI indicates that 2 codes co-occur more often than expected by chance, while a negative PMI implies the opposite. To retain only informative associations, we use the Positive PMI (PPMI):

$$\mathit{PPMI}\left(i,j\right)=\max \left(\mathit{PMI}\left(i,j\right),\hspace{0.17em}0\right)$$

The resulting PPMI matrix is a square, symmetric matrix where each entry encodes the degree of association between 2 medical codes. In practice, several temporal context window sizes, ranging from same-day to several months, were tested, and performance remained stable across the tested values. We retained a 10-day window as a short-term clinical context comparable to the scale of hospital-stay episodes.

Matrix factorization through SVD

Although the PPMI matrix captures semantic information, it remains high-dimensional and sparse: its size equals the number of distinct codes, and entries are zero for code pairs that never co-occur.

To obtain compact representations, we apply Singular Value Decomposition (SVD). The PPMI matrix is factorized as:

$$\mathit{PPMI}=U\Sigma V^{\mathrm{\top }}$$

where $U$ and $V$ are orthogonal matrices and $\Sigma$ is the diagonal matrix of singular values.

Truncating to the top $d$ singular values gives $U_d$ and ${\Sigma }_d$. The embedding matrix is then defined as:

$$W=U_d{\Sigma }_d^{1/2}$$

where each row $w_i\in {\mathbb{R}}^d$ corresponds to the embedding vector of code $i$. Dimensionality d = 150 was chosen for our final representations, as the leading singular values capture most of the information contained in the PPMI matrix (File S2).

Evaluation of resulting embeddings

Evaluating the quality of the embeddings is challenging, as no gold standard exists for a joint representation of ICD-10 and ATC codes. We therefore combined 2 complementary approaches. The qualitative evaluation assessed the medical coherence of the latent space using visualization and neighborhood analyses. The quantitative evaluation assessed whether embeddings capture coding system-level information by predicting ICD-10 chapters and ATC classes in a supervised classification task. In addition, a patient-level classification example based on longitudinal claims data is provided in File S3 to illustrate how code embeddings can be aggregated to construct patient-level representations for amyotrophic lateral sclerosis (ALS) prediction.

Qualitative assessment through neighborhood analysis and UMAP visualization

Let $\{w_i{\}}_{i=1}^n$ denote the embedding vectors associated with the $n$ distinct medical codes, each $w_i\in {\mathbb{R}}^d$. Pairwise similarity between codes is measured using cosine similarity:

$$s_{ij}=\frac{w_i^{\mathrm{\top }}{w}_j}{\parallel w_i\parallel \hspace{0.17em}\parallel w_j\parallel }$$

which defines the symmetric similarity matrix $S=\left(s_{ij}\right)$. On this basis, we carried out 3 complementary exploratory analyses:

• We applied UMAP²⁰ to obtain a 2-dimensional representation of the embeddings. UMAP preserves neighborhood relations defined by $S$, allowing both global clusters and local semantic patterns to be visualized.

• We examined local structure by retrieving the nearest neighbors of randomly selected codes according to cosine similarity, to assess whether proximity in the embedding space reflects clinically meaningful relations.

• We constructed a semantic network in which nodes represent codes and edges connect pairs with similarity above a threshold $t\in \left[\mathrm{0,1}\right]$. This sparse graph provides an interpretable, network-based view of local semantic structure that complements the UMAP projection and nearest-neighbor analysis.

Sampling variability and neighborhood stability analysis

Since co-occurrence patterns in claims data can fluctuate substantially, in particular for rare codes, we evaluated the sampling-driven variability of our embeddings. To do so, we trained 2 additional embedding spaces on non-overlapping subsamples of the original data. We then assessed how stable the local neighborhoods remain across both spaces. For each code $\mathit{i}$, we computed the Neighborhood Stability, defined as the proportion of its $\mathit{k}$-nearest neighbors preserved across the 2 spaces:

$$\text{Stability}\left(\mathit{i}\right)=\frac{\mid {\mathbf{N}}_{\mathit{k}}^{\left(\mathbf{1}\right)}\left(\mathit{i}\right)\cap {\mathit{N}}_{\mathit{k}}^{\left(\mathbf{2}\right)}\left(\mathit{i}\right)\mid }{\mathit{k}},$$

where ${\mathit{N}}_{\mathit{k}}^{\left(\mathbf{1}\right)}\left(\mathit{i}\right)$ and ${\mathit{N}}_{\mathit{k}}^{\left(\mathbf{2}\right)}\left(\mathit{i}\right)$ denote the top-$\mathit{k}$ neighbors of $\mathit{i}$ in the first and second embedding spaces. A value of 1 indicates identical neighborhoods, whereas 0 indicates no shared neighbors. Neighborhood Stability was examined as a function of code frequency.

Quantitative assessment through code chapter classification

While exploratory analysis gives a general idea of the resulting embeddings’ coherence, a common approach for embedding evaluation is to use them as input to a supervised task. We performed a code classification task, aiming to predict the ICD-10 chapter or ATC class of a code, given its embedding as input.

Code classification was performed using a multinomial logistic regression model. Separate models were trained for ICD-10 and ATC codes. In this setting, predictive performance reflects the extent to which codes from the same chapter or class form coherent groups and can be discriminated by a linear classifier in the embedding space. Since the purpose of this task was to assess whether embeddings encode semantic information, no attempt was made to optimize predictive performance beyond this simple baseline.

Model performance was evaluated using repeated stratified 11 × 10-fold cross-validation (10 folds repeated 11 times with different random seeds). For each fold, class-probability predictions were obtained, and one-vs-rest AUC values were computed for every class. Performance was summarized using the mean AUC and its standard deviation (SD) across all 110 folds, providing a robust estimate of how consistently the embeddings capture the hierarchical organization of ICD-10 and ATC. Figure 1 illustrates the overall embedding computation and evaluation pipeline.

Figure 1.

Overview of the proposed pipeline: (1) Data preprocessing: raw logs are filtered, converted into timed sequences of codes; (2) Embedding computation: co‐occurrence counts are converted into a pointwise mutual information matrix and factorized to produce low‐dimensional ICD-10/ATC code embeddings; (3) Evaluation: embedding quality is assessed using UMAP visualization, nearest‐neighbor retrieval, and supervised ICD-10 chapter/ATC class prediction.

Results

Dataset description

We computed the embeddings using the Échantillon du Système National des Données de Santé (ESND),²¹ a French national claims database covering all reimbursed healthcare consumption for a representative sample of 1.5 million individuals over a 15-year period. The ESND provides comprehensive coverage of diagnoses (ICD-10), medications (ATC), procedures, medical devices, and laboratory tests, offering the contextual information needed to learn robust code representations.

For sampling-variability analysis, we used 2 non-overlapping subsamples of the ESND, comprising 200 000 patient sequences each. They were used exclusively to compute neighborhood stability. The final embeddings released in this work were obtained from the full 1.5 million sequences.

Individual-level data from the ESND are pseudonymized and were accessed through the permanent authorization granted to the French National Institute for Research in Digital Science and Technology (Inria) by the Caisse Nationale de l’Assurance Maladie (CNAM), within the framework of the Système National des Données de Santé. Use of these data is limited to research and general-interest purposes in accordance with Article L.1461-1 of the Code de la santé publique. All processing complies with the General Data Protection Regulation (GDPR) and the French Data Protection Act.

Embeddings were obtained for codes belonging to the ESND dataset (N = 37 979). However, we only performed evaluation on ICD-10 codes (N = 10 580) and ATC codes (N = 1526), as the others are based on French classification systems. Additional details on code counts and code frequency distributions are provided in File S4.

Code embedding evaluation

Exploratory analysis

To assess whether the embeddings capture clinically meaningful structure, we examined their organization in the latent space. Semantically related codes are expected to cluster together, forming regions corresponding to broad medical concepts.

UMAP visualization

The UMAP projection was used to visualize the global and local structure of the embedding space for a subset of codes (Figure 2).

Figure 2.

UMAP projection of a representative sample of ICD-10 and ATC code embeddings in 2 dimensions. Each code is colored with respect to its ICD-10 Chapter or ATC Class. Distinct clusters correspond to relevant semantic groupings within and across chapters, illustrating how embeddings capture both intra- and inter-chapter relationships.

On the broadest scale, codes belonging to the same ICD-10 chapter or ATC class tend to group within the same neighborhood of the embedding space, forming large clusters that reflect broad medical themes. For example, codes from the ICD-10 Neoplasms chapter, such as C79.6 (Secondary malignant neoplasm of ovary) and C16.9 (Malignant neoplasm of the stomach, unspecified), occupy the same region. Similarly, codes from ATC class N (Nervous System), including N03AA03 (Primidone) and N06AA17 (Amoxapine), cluster together. This large-scale organization, which reproduces established ICD-10 and ATC groupings, indicates that the embeddings capture semantic structure consistent with existing medical classifications.

At a finer scale, the latent space contains smaller clusters that correspond to specific medical subtopics and cross-chapter associations. Neoplasm-related codes, for example, form distinct regions according to anatomical site, and these regions also include non-neoplasm codes associated with the same locations. Codes related to sinus tumors, such as C31.2 (Malignant neoplasm of frontal sinus), appear near sinus-related respiratory codes like J32.3 (Chronic sphenoidal sinusitis) and J33.0 (Polyp of nasal cavity). Similarly, D14.1 (Benign neoplasm of larynx) is located close to J37.0 (Chronic laryngitis) and J38.6 (Stenosis of larynx).

This pattern extends beyond neoplasm sites. Codes such as H68.0 (Eustachian salpingitis) and H65.9 (Unspecified nonsuppurative otitis media) cluster near J35.3 (Hypertrophy of tonsils with hypertrophy of adenoids) and J31.2 (Chronic pharyngitis), despite belonging to different ICD-10 chapters. These local structures show that relationships between conditions of the middle ear and lymphoid organs are accurately reflected in the embedding space.

Overall, these small-scale clusters, together with the large-scale organization observed earlier, demonstrate that the embeddings capture both hierarchical structure and clinically meaningful cross-chapter associations.

Nearest-neighbor search

We assessed the local coherence of the embedding space and its ability to retrieve clinically related codes across multiple coding systems by examining the nearest neighbors of 4 ICD-10 codes. For each target code, we identified the 5 closest neighbors using cosine similarity; neighbors could belong to ICD-10, ATC, or a French procedure classification (Table 1). Each neighborhood can be visualized as a local semantic network; Figure 3 shows the subnetwork for B16.9 (Acute hepatitis B).

Table 1.

Five nearest neighbors (by cosine similarity) for 4 example ICD-10 codes.

Target Code	Related Diagnoses (ICD-10)	Related Treatments (ATC)	Related procedures (French classification system)
Acute hepatitis B (B16.9)	Chronic viral hepatitis B without delta-agent (B18.1) Other specified acute viral hepatitis (B17.8) Chronic viral hepatitis B with delta-agent (B18.0) Hepatitis A without hepatic coma (B15.9) Liver disorders in infectious and parasitic diseases (K77.0)	Tenofovir + Disoproxil (J05AF07) Hepatitis B immunoglobulin (J06BB04) Rifaximin (A07AA11) Ribavirin (J05AP01) Sofosbuvir (J05AP08)	Histopathological examination of liver biopsy specimen with special staining for non-neoplastic disease Transjugular liver biopsy with ultrasound and/or fluoroscopic guidance and measurement of hepatic venous pressure Transcutaneous ultrasound of liver and biliary ducts Transabdominal ultrasound of upper abdomen with Doppler evaluation of digestive vessels Open portocaval or portorenal anastomosis (truncal), via laparotomy
Malignant neoplasm of upper lobe, bronchus or lung (C34.1)	Malignant neoplasm of middle lobe, bronchus (C34.2) Malignant neoplasm of lower lobe, bronchus or lung (C34.3) Malignant neoplasm of upper lobe, unspecified bronchus or lung (C34.8) Malignant neoplasm of unspecified part of bronchus or lung (C34.9) Malignant neoplasm of main bronchus (C34.0)	Pemetrexed (L01BA04) Erlotinib (L01EB02) Afatinib (L01EB03) Nivolumab (L01XC17) Osimertinib (L01EB04)	Thoracoscopic exploration of the chest with lung biopsy Percutaneous destruction of liver tumor by radiofrequency ablation under CT guidance Histopathologic examination for cancer of resected lung specimen Open thoracic exploration with lung biopsy (via thoracotomy) Pneumonectomy via thoracotomy (open approach)
Type 1 diabetes mellitus without complications (E10.9)	Type 1 diabetes mellitus with ketoacidosis (E10.1) Presence of endocrine implants (Z96.4) Diabetes mellitus due to underlying condition with hyperosmolarity (E10.8) Type 1 diabetes mellitus with other specified complications (E10.6) Drug-induced hypoglycemia without coma (E16.0)	Glucagon (H04AA01) Insulin lispro (A10AB04) Insulin detemir (A10AE05) Insulin aspart (A10AB05) Volanesorsen (C10AX18)	Replacement of intraperitoneal catheter of implanted infusion device for intraperitoneal insulin therapy, open approach Insertion of intraperitoneal catheter and subcutaneous implantable infusion device for intraperitoneal insulin delivery Replacement of subcutaneous implanted infusion pump Stereoscopic fundus photography with wide-field imaging (>60°), including peripheral retina Cardiac stress test using treadmill with electrocardiographic monitoring
Open-angle glaucoma (H40.1)	Degenerative myopia (H44.2) Other specified cataract (H26.8) Primary angle-closure glaucoma (H40.2) Presence of intraocular lens (Z96.1) Complicated cataract (H26.2)	Pilocarpine (S01EB01) Apraclonidine (S01EA03) Dorzolamide (S01EC03) Diclofenac (S01BC03) Brimonidine (S01EA05)	Drainage of aqueous humor of the eye Deep non-penetrating sclerectomy or deep sclerokeratectomy, without viscocanaliculoplasty Trabeculectomy (penetrating sclerotomy) Partial destruction of ciliary body Peripheral or sectoral iridotomy or iridectomy (non-laser)

Neighbors were selected from ICD-10, ATC, and a French classification system for procedure codes.

Figure 3.

Semantic subnetwork for ICD-10 code B16.9 (acute hepatitis B): nodes represent the target code and its closest neighbors with respect to cosine similarity across ICD-10, ATC, and the French classification system for procedures (threshold = 0.5).

The neighborhood of B16.9 includes ICD-10 codes for related viral hepatitis conditions such as B18.0 (Chronic viral hepatitis B with delta-agent), B18.1 (Chronic viral hepatitis B without delta-agent), and B15.9 (Hepatitis A without hepatic coma). It also retrieves ATC codes for medications frequently prescribed in hepatitis care, including J05AP08 (Sofosbuvir), A07AA11 (Rifaximin), J05AP01 (Ribavirin), and J06BB04 (Hepatitis B immunoglobulin). These treatments span several ATC chapters, indicating that the embeddings recover semantic relationships across coding systems rather than relying solely on hierarchical proximity. Procedure codes related to liver assessment, such as histopathological examination of liver biopsy specimens and ultrasound imaging of the liver and biliary ducts, are also retrieved, demonstrating alignment across diagnostic, therapeutic, and procedural concepts.

Across all 4 examples, the nearest-neighbor search consistently recovers clinically meaningful diagnosis-treatment-procedure relationships. This automated recovery of clinically related codes illustrates how the embeddings support cross-system integration by linking diagnoses, treatments, and procedures across distinct classification systems.

Sampling variability and neighborhood stability

We next examined how robust these local structures are to sampling variability. A clear link between code frequency and neighborhood stability is shown in Figure 4. Stability, defined as the proportion of top-10 neighbors retained across 2 embedding spaces learned on independent subsamples, is consistently higher for common codes. Mean stability computed on the top 5% most frequent codes was approximately 50%, whereas it dropped near 25% when considering the entire code vocabulary. As expected, rare codes are subject to sampling noise: because they appear in very few patients, unrelated co-occurring conditions in even a single individual can disproportionately affect their neighborhood, unlike frequent codes whose structure averages over many observations. This trend is observed for any value of k between 5 and 30 (File S5).

Figure 4.

Mean neighborhood stability across quantile-based frequency thresholds ($k=10$). Each point reports the mean stability over all codes whose frequencies fall above the corresponding quantile threshold (eg the top 5% most frequent codes for $x=$ 5%). Increasing the threshold progressively relaxes the frequency cutoff, leading to lower overall stability as rarer codes are added.

Quantitative assessment—classification task

To derive a numerical indicator of embedding quality, we trained a multinomial logistic regression model to predict the ICD-10 chapter or ATC class of a code from its embedding vector. Higher predictive performance indicates better alignment of embeddings with these classification systems.

Tables 2 and 3 report the AUC scores from the multinomial logistic regression model, broken down by class. We obtained a weighted average AUC of 0.93. The lowest AUC was 0.81, corresponding to Chapter R00-R99 (Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified), while the highest AUC was close to 1, achieved for Chapter P00-P96 (Certain conditions originating in the perinatal period).

Table 2.

AUC performance (+ standard deviation) for the classification task across all ICD-10 chapters.

Chapter	Name	Mean AUC (SD)	N
A00-B99	Certain infectious and parasitic diseases	0.93 (0.02)	590
C00-D49	Neoplasms	0.97 (0.01)	741
D50-D89	Blood and blood-forming organs diseases	0.94 (0.04)	163
E00-E89	Endocrine, nutritional and metabolic diseases	0.94 (0.02)	339
F01-F99	Mental and behavioral disorders	0.95 (0.02)	390
G00-G99	Diseases of the nervous system	0.93 (0.02)	337
H00-H59	Diseases of the eye and adnexa	0.97 (0.02)	260
H60-H95	Diseases of the ear and mastoid process	0.97 (0.03)	111
I00-I99	Diseases of the circulatory system	0.95 (0.02)	391
J00-J99	Diseases of the respiratory system	0.95 (0.03)	216
K00-K95	Diseases of the digestive system	0.96 (0.02)	441
L00-L99	Diseases of the skin and subcutaneous tissue	0.93 (0.03)	302
M00-M99	Diseases of the musculoskeletal system and connective tissue	0.95 (0.01)	535
N00-N99	Diseases of the genitourinary system	0.96 (0.01)	425
O00-O99	Pregnancy, childbirth and the puerperium	0.99 (<0.01)	417
P00-P96	Certain conditions originating in the perinatal period	>0.99 (<0.01)	331
Q00-Q99	Congenital malformations and chromosomal abnormalities	0.96 (0.01)	587
R00-R99	Symptoms, abnormal clinical findings, not elsewhere classified	0.81 (0.04)	330
S00-T98	Injury, poisoning and other consequences of external causes	0.91 (0.01)	1204
U00-U91	Codes for special purposes	0.84 (0.11)	38
V00-Y99	External causes of morbidity and mortality	0.93 (0.01)	1720
Z00-Z99	Factors influencing health status and contact with health services	0.84 (0.03)	625

Table 3.

AUC performance (+ standard deviation) for the classification task across ATC classes.

Class	Name	AUC (SD)	N
A	Alimentary tract and metabolism	0.86 (0.05)	183
B	Blood and blood-forming organs	0.82 (0.09)	72
C	Cardiovascular system	0.9 (0.04)	169
D	Dermatologicals	0.94 (0.05)	75
G	Genito-urinary system and sex hormones	0.92 (0.04)	91
H	Systemic hormonal preparations, excluding sex hormones and insulins	0.85 (0.12)	38
J	Anti-infectives for systemic use	0.91 (0.04)	180
L	Antineoplastic and immunomodulating agents	0.93 (0.03)	203
M	Musculo-skeletal system	0.88 (0.08)	63
N	Nervous system	0.92 (0.04)	197
P	Antiparasitic products, insecticides	0.88 (0.17)	24
R	Respiratory system	0.94 (0.04)	109
S	Sensory organs	0.97 (0.03)	77
V	Various	0.85 (0.09)	45

Overall, predictive performance was higher for chapters that group codes around a well-defined medical concept such as C00-D49 (Neoplasms), H00-H59 (Diseases of the eye and adnexa), or K00-K95 (Diseases of the digestive system) than for chapters that aggregate more heterogeneous conditions, such as Chapter R00-R99.

These results support the idea that the semantic and hierarchical structure of the ICD-10 is effectively encoded within the embedding space.

For ATC classes, predictive performance was slightly lower, with a weighted average AUC of 0.90. The lowest AUC was 0.82, observed for Class B (Blood and blood-forming organs), and the highest was 0.94 for Class D (Dermatologicals). While these results remain strong overall, they suggest that the hierarchical structure of the ATC classification may be more challenging to capture in the embedding space.

Overall, AUC scores above 0.82 for all chapters and classes indicate that hierarchical relations encoded in ICD-10 and ATC were well represented in the latent space.

Misclassified codes

Examining misclassified codes provides additional insight into the semantic coherence of the learned embeddings. Several apparent errors made by the classifier are in fact clinically meaningful. This is particularly evident for ICD-10 codes in Chapter R00-R99 (Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified). Because this chapter groups heterogeneous signs and symptoms spanning multiple organ systems, the classifier often assigns these codes to chapters that better reflect their underlying physiological or clinical context.

For example, R73.0 (Abnormal glucose tolerance test), R77.0 (Abnormality of albumin), and R73.9 (Hyperglycemia, unspecified) are predicted as belonging to E00-E89 (Endocrine, nutritional and metabolic diseases). Similarly, R25.2 (Cramps and spasms) and R25.3 (Fasciculation) are mapped to G00-G99 (Diseases of the nervous system). Finally, R00.2 (Palpitations) and R00.8 (Other abnormalities of heartbeat) are assigned to I00-I99 (Diseases of the circulatory system). In each case, the predicted chapter is clinically consistent with the underlying meaning of the code.

These semantically consistent reassignments indicate that the embeddings capture clinically meaningful relationships that are not fully reflected in the ICD-10 hierarchy. In several instances, the latent-space organization provides groupings that align more closely with clinical understanding than those defined by high-level ICD-10 categories, which are constrained by a single-parent hierarchical organization. This highlights that the embeddings encode complementary semantic structure and illustrates the inherent constraints of rigid, single-parent hierarchical classification systems.

Discussion

In this work, we introduced unsupervised embeddings for ICD-10 and ATC codes derived from 1.5 million longitudinal patient records. By leveraging 15 years of claims data containing diagnoses, treatments, procedures, and medical devices, we obtained dense representations that reproduce established hierarchical groupings while also capturing clinically grounded semantic relationships. Embedding diagnosis and treatment codes in a shared space enables downstream models to incorporate diagnosis-treatment links without manual feature engineering, and nearest-neighbor searches allow users to retrieve related codes across multiple coding systems.

Our goal was to develop general-purpose, task-agnostic embeddings that capture clinical relationships present in a broad population. These vectors can serve as reusable representations for a range of digital health applications, including risk prediction, automated phenotyping, care-trajectory analysis, and large-scale population studies. In particular, they enable the construction of patient-level features through direct aggregation of code embeddings, providing compact representations that can be readily integrated into standard statistical models, as illustrated by the patient-level classification example. All embeddings are publicly released through a GitHub repository (https://github.com/cfaujour/ICD10-ATC-Embeddings) to facilitate reproducibility and community reuse.

Because they are learned from standardized medical codes rather than free text, the embeddings are independent of natural language and may be reused in settings where ICD-10 and ATC are used, without language-specific preprocessing. However, although ICD-10 and ATC are international coding systems, the embeddings were learned from a single national claims database and may reflect French coding practices, reimbursement rules, and patterns of care. Their use in other healthcare systems remains relevant, although local validation may help assess transferability to settings with different coding or care practices.

A central methodological decision was to adopt a co-occurrence-based embedding approach rather than prediction-based methods. While most unsupervised NLP methods are designed for strictly ordered token sequences, EHR and claims data contain many simultaneous events. Sequential approaches such as Word2Vec¹⁷ require imposing an arbitrary order, which may distort clinical semantics. In contrast, the PPMI-SVD framework aggregates co-occurrences within temporal windows and naturally accommodates simultaneous codes. Given the theoretical link between Word2Vec and PMI factorization via SVD,²² this approach provides a theoretically grounded and computationally efficient alternative well suited to claims data.

Despite strong evidence from UMAP visualization, neighborhood analyses, and chapter-level classification, evaluating embedding quality remains challenging in the absence of a gold standard for clinical code semantics. Classification AUCs confirm that hierarchical structure is well captured, but they do not directly assess finer-grained associations such as diagnosis-treatment links. Prior work has highlighted the limitations of intrinsic and extrinsic embedding evaluations.²³ Manually curated benchmarks (eg expert-validated ICD-ATC links or guideline-based code pairings) could provide more targeted evaluation, though assembling such resources would be time-consuming and may vary across clinical contexts.

While many recent approaches learn contextualized representations within predictive models or sequence reconstruction frameworks,²⁴ we instead assign a single, consistent vector to each medical code. This design provides a reusable code-level representation that can be directly applied across datasets and analyses, separating representation learning from downstream modeling and leaving the composition of code-level embeddings to the user.

These embeddings can be incorporated into patient-level sequence models, such as recurrent, convolutional, or attention-based architectures,^25–27 where temporal order, event timing, and irregular intervals play an important role in modeling disease trajectories.²⁸ Beyond predictive models, these representations can also be used directly for similarity analysis, exploratory studies, and feature construction in large-scale observational data. Integrating interpretability methods, such as attention visualization²⁹ and attribution techniques,³⁰ remains essential for ensuring clinical relevance and transparency.

Conclusion

We obtained and publicly released unsupervised, task-agnostic embeddings learned from large-scale real-world claims data, jointly representing ICD-10 diagnoses and ATC medications in a shared space. These embeddings capture both the hierarchical organization of each system and clinically meaningful relationships between diagnoses and treatments. By providing a reusable semantic representation shared across ICD-10 and ATC, this work supports exploratory analysis and downstream analytic applications without requiring manual code grouping. The released embeddings provide a community resource for biomedical informatics research and a foundation for future patient-level modeling applications.

Author contributions

Corentin Faujour (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing), Anne-Sophie Jannot (Conceptualization, Methodology, Resources, Supervision, Writing—review & editing), Stéphane Bouée (Conceptualization, Methodology, Supervision, Writing—review & editing), and Corinne Emery (Conceptualization, Methodology, Supervision, Writing—review & editing)

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

This work was supported by a CIFRE doctoral fellowship funded by the French National Association of Research and Technology (ANRT). This work benefited from a government grant managed by the Agence Nationale de la Recherche under the France 2030 program, reference 22-PESN-0013.

Conflicts of interest

The authors have no conflicts of interest to report.

Data availability

The original care-trajectory data used to generate the embeddings cannot be shared due to legal, ethical, and regulatory restrictions related to patient privacy. The embeddings produced in this study are openly available to support reproducibility and further research. A synthetic dataset that reproduces the structure of the care trajectories, together with all source code used for embedding computation and exploratory analyses, is publicly accessible in the project’s GitHub repository (https://github.com/cfaujour/ICD10-ATC-Embeddings).