Comprehensive representation of health-related phenotypes in one million dogs using topic modelling of electronic health records

Abstract

Historically, veterinary studies screening for breed, age and sex predisposition to disease have relied on collating small-scale studies of clinical datasets. The availability of larger datasets through groups such as the Small Animal Veterinary Surveillance Network (SAVSNET) promise access to information regarding a wide range of clinical presentations at scale, however, methodological limitations surrounding the extraction of specific disease information or screening for disease predispositions result in a substantial reduction in the number of animals studied. These studies often address very focused hypotheses - only leveraging a small fraction of the intrinsic value of the data at any one time. Here, we implemented an unsupervised machine learning methodology, creating a representation of a large volume of clinical notes collected by SAVSNET from veterinary practices across the UK. We utilise BERTopic, a topic-modelling tool based on Bidirectional Encoder Representations using Transformers (BERT) architecture, and show it is able to surface known phenotypes, such as breed predispositions to hypoadrenocorticism, diabetes mellitus and mitral valve disease, as well as potential novel patterns of disease phenotypes. This scalable and granular modelling technique facilitates the rapid interrogation of large clinical datasets, enabling the identification of a broad range of phenotypes within the population and the early detection of temporal changes indicative of emerging infectious or environmental diseases.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40537-026-01365-0.

Introduction

In the veterinary health literature, the identification of which breeds (ages or sexes) are affected by a specific disease or which range of diseases affect a specific breed has typically been studied using comparatively small cohorts of animals with a specific condition, for example, animal predisposition to mammary cancer [44] or of breeds of animals where records could be screened for wide-ranging diseases, for instance, diseases affecting French Bulldogs or Boxers [29, 31]. Such studies typically include hundreds to small numbers of thousands of individual animals. Similarly, modelling the occurrence of disease with age is not easily achieved at scale for many breeds at one time. Ultimately, consensus about disease in dogs is usually reached through a review of multiple publications, often incorporating comparatively small groups of animals.

Recently, extremely large datasets have become available through groups such as Vet Compass [28] and the Small Animal Veterinary Surveillance Network (SAVSNET, [38]). These hold significant volumes of canine electronic health records, which in turn will embody key patterns of phenotypes displayed by dogs as represented by the clinical narratives contained therein. While these systems allow for some disease-coding by attending clinicians, they suffer from either simple single-choice coding (SAVSNET) or complex coding choices that are associated with poor compliance, and for all such coding systems, the reliability of the attending clinician to enter this data is very variable [14, 25].

The veterinary informatics sector struggles to fund work on a scale that would allow wide-ranging manual record annotation, and some form of automation of record annotation at scale would alleviate this challenge. A potential candidate for this is machine learning (ML), which has been used to create automated record classification systems for both text and image data. However to date, these are usually supervised methods requiring both prior knowledge of the phenotypes likely to be seen and reasonably large datasets of expert-annotated records in order to train these systems [10, 26], all of which require substantive investment of time on a case by case basis. Additionally, the process by which ML systems generate classifications can often lack explainability regarding the factors that influenced the classification. This can lead to systematic errors due to the model identifying unexpected features in the data necessitating the development of systems such as LIME and SHAP to generate explanations for ML outputs [4, 36]. An ideal system would involve unsupervised annotation of records according to disease phenotypes based on intrinsic characteristics of the notes. The aim would be that explainable results would be intrinsic to the method.

We have previously shown that latent Dirichlet allocation (LDA) topic modelling allowed the identification of a specific gastroenteric disease in clinical notes from dogs that would have allowed early detection of a disease outbreak [27]. LDA is based on a bag-of-words statistical method that does not differentiate word meaning according to context, potentially limiting the ability of the technique to differentiate topics. A more modern approach incorporates a combination of neural language modelling using bidirectional encoder representations using transformers (BERT) [8] to create document embeddings with subsequent dimensionality reduction using UMAP and clustering into documents with common word-composition based topics using hdbscan. This combination of techniques is implemented in the BERTopic package [12]. In addition to creating topic models, BERTopic is able to represent topic evolution across time (dynamic topic modelling) or other classifications (e.g. breed) inherent to the data under study.

Whilst novel in veterinary studies, topic modelling has been evaluated for information extraction from human electronic health records [37] and social-care notes with clinical content [41] highlighting a clear opportunity to leverage its use for veterinary data.

In the current study we evaluated the hypothesis that topic modelling with BERTopic using records from a million dogs will allow us to identify population-wide disease phenotypes which can be analysed on the basis of a number of factors including breed, age, sex and time-course.

We address the key problem that while we have access to large volumes of clinical records, identifying key disease phenotype signals in these data presents a significant bottleneck. Our objective was to evaluate how, using a minimal approach to BERTopic customisation, a topic model based on health data from one million dogs could surface disease phenotypes in an explainable and analysable form.

We use BERTopic to create topic models based on a large subset of SAVSNET clinical narratives. We evaluate the patterns of phenotypes revealed within BERTopic-generated topics, comparing how topics for disease change with time and in specific breed, age, and sex cohorts compared to known disease occurrence and how the models perform in revealing new clinical insights.

Materials and methods

We followed the STROBE and RECORD reporting guidelines [2, 43]. To enhance transparency regarding the ML methodology, we also incorporated relevant principles from reporting guidelines for artificial intelligence in health research [17].

The study design comprised collating a set of clinical narratives from veterinary consultations involving dogs. These were passed to a topic-modelling package (BERTopic) [12] for annotation with topics intrinsic to the clinical text. Topic coherence was assessed and subsequently a series of analyses of odds ratios for the association of given topics with breed, age, sex and time were assessed and presented graphically.

Endpoints of the study were to illustrate trends in the data corroborated primarily by 95% confidence intervals for topic associations differing from reference populations (with breeds this was usually cross breed)

SAVSNET datasets

SAVSNET collects data from approximately 580 veterinary premises across the UK. After each patient consultation, data are passed to SAVSNET and comprise the species, breed, age, and sex of the patients, along with the clinical narrative recorded by the attending veterinarian. The current study used a random sample of one million clinical records, each record coming from a unique dog and comprising the deidentified clinical narrative, the breed, age, sex and the date of consultation.

Topic models

Narratives from the one million record datasets were used to train a BERTopic (version 0.13.0) model [12]. Briefly, BERTopic embeds documents using the sentence transformer all-MiniLM-L6-v2 [16] followed by reduction of dimensionality using UMAP(umap-learn package v0.5.2) [24] and subsequent clustering using HDBScan(hdbscan package v0.8.37), an efficient clustering algorithm with good noise rejection and minimal parameter configuration [23]. Clusters are then analysed using term frequency/inverse document frequency (TF/IDF) to identify word weighting for each topic [33]. A trial and error method was used to adjust the UMAP and HDBSCAN parameters to generate a usable model with minimal generation of duplicated and vacuous topics (A1, A2). The topics were assessed by a clinically active academic (Noble) to attribute clinical relevance. The quality of the model was assessed using the gensim package (v4.2.0) CoherenceModel class with the ’c_v’ coherence metric [35], providing a measure of semantic interpretability. The distribution of topics by age, breed class, or consultation month was created using the topics_per_class method.

Models were trained using a Ryzen-9 12 core CPU PC with 64Gb of DDR-4 RAM equipped with an Nvidia A4000 GPU (16Gb internal memory).

Data analysis

Using this method, each clinical narrative was assigned a probability distribution for containing any topic. For subsequent analysis, each consultation was classified according to the most probable topic for that consultation.

Odds ratio calculation and confidence intervals

Odds ratios were calculated to quantify the association between topic presence and class membership relative to a specified reference class. For each topic-class combination, a contingency table was constructed where a represents the frequency of the topic in the class of interest, b represents the frequency of the topic in the reference class, c represents the frequency of all other topics in the class of interest, and d represents the frequency of all other topics in the reference class. To address zero-count cells, a continuity correction of was applied to all cell counts. Odds ratios were calculated using python code incorporated in plot generation using the following calculations:

1

Ninety-five percent confidence intervals for the odds ratios were derived using the logarithmic transformation method. The standard error of the log odds ratio was calculated as:

2

The confidence intervals were then computed as:

3

where 1.96 represents the critical value of the standard normal distribution for a 95% confidence level.

Topic word-contents were reviewed using the BERTopic visualize_barchart method. Plotly [39] was used to present a filterable line plot of proportions of narratives labeled with given topics broken down by age and date. For time series, values were normalised to peak values for each individual series to allow comparison. For breed and sex-related data, odds ratios along with 95% confidence intervals were calculated for the proportions of topic-labelled narratives against suitable references and plotted as tree plots using Plotly.

Results

The SAVSNET database contained 5,467,034 narratives from dog consultations. With the hardware in use, BERTopic would overrun memory limits in the GPU when used to train the model with more than 1,000,000 records. When the number of possible topics was unconstrained, BERTopic generated a model with over 900 topics, many of which were vacuous (sequences of words with no clear clinical correlate) or very similar in word composition to other topics (e.g. multiple combinations describing a vaccination event). As a consequence, BERTopic was run, limited to producing a maximum of 200 topics which appeared to represent diverse clinical presentations with less duplication (HDBScan and BERTopic parameters are shown in supplementary material tables A1, A2).

The coherence analysis of the BERTopic model demonstrated exceptional performance, with an average c_v coherence score of 0.8501 across 197 evaluated topics. Of the 200 topics initially generated, 184 topics (93.4%) achieved good coherence scores (0.7), while only 11 topics (5.6%) fell into the acceptable range (0.5–0.7.5.7), and merely 2 topics (1.0%) showed poor coherence (<0.5). Notably, 10 topics achieved perfect coherence scores of 1.0, indicating optimal semantic consistency within these thematic clusters. Individual topic assignment probabilities were relatively low (median maximum probability: 0.049, mean: 0.129), with only 6.9% of documents assigned with>0.5 probability to their designated topic.

The top 15 most common topic representations (15 words or word-pairs for each topic) are shown in supplementary material (Figure. SA1). Despite the count reduction, 53 topics still contained words relating to vaccination or booster.

Breed distribution

SAVSNET data comprised information about 217 unique breeds. Using the topics_per_class method of BERTopic allowed the distribution of topics for any breed to be identified and odds ratios for any breed having a given phenotype calculated with crossbreed as a reference. A wide range of potential breed predispositions could be identified. An example of four topics representing different endocrine diseases is shown in Fig. 1. Here increased occurrence of diabetes topic is seen in Samoyeds, Huskies and West Highland White Terriers whereas the description of thyroid disease is seen more commonly in standard poodles, dobermans and Scottish terriers. Hypoadrenocorticism-related narratives were seen in the standard poodle, Bearded Collies, Labradoodles and German short-haired pointers (Fig. 1). Similarly, breed associations for clinical syndromes involving the cardio-respiratory system could be evaluated with an example dataset shown in Fig. 2. Example topics representing upper respiratory signs (sneezing, reverse sneezing), cardiac findings (heart murmur) and cough are shown and illustrate potential predispositions (e.g. murmur in Cavalier King Charles spaniel and Boxer and Chihuahua, sneeze in Pug and Chihuahua) and reduced instances (e.g. murmur in Pugs, cough in French Bulldog).

Fig. 1.

Breed distribution of endocrine disease topics. Topics representing animals probably affected by diabetes (blue), hyperadrenocorticism (red), hypoadrenocorticism (green) and hypothyroidism (orange) were readily identified by topic wording. Odds ratio (reference=crossbreed) and 95% confidence interval were plotted for a selection of breeds with potential predispositions to individual endocrinopathies

Fig. 2.

Breed distribution of cough (blue), sneeze (green) and murmur (orange) topics. Here, the distribution for topics reflecting respiratory or cardiac clinical signs are shown for a selection of breed. Odds ratio (reference=crossbreed) and 95% confidence intervals for breed having consultations matched by the given topic

Age distribution

SAVSNET data comprised records from animals aged 0–18 years old. Topics illustrated a wide range of age-related variation such that topics relating to vaccination and puppy health checks occurred in younger animals. Topics relating to masses occurred more commonly in middle aged dogs and topics relating to vestibular disease, and euthanasia occurred in older dogs (Fig. 3).

Fig. 3.

Examples of topics with distinctive age distributions. Age distribution for a group of topics that represent early-life (vaccination and puppy check), mid-life (lumps and masses) and late-life (vestibular disease and euthanasia) presentations. Each topic is normalised to its maximum proportion to allow comparison of age-pattern for each topic

Sex distribution

Differences in topic distributions between male and female dogs were evaluated and demonstrated some variation. Odds ratios for male and female neutering were strongly (and appropriately) segregated for the relevant sexes such that the only topic containing the word spay (‘spay’, ‘season’, ‘pre’, ‘for spay’, ‘pre spay’, ‘spay check’, ‘op’, ‘pre op’, ‘last season’, ’lap’, ‘spey’, ‘vulva’, ‘check’, ‘mammary’, ‘lap spay’) had an odds ratio of 0.01 in males compared to females and in a similar way, a topic associated with castration (‘testicle’, ‘castration’, ‘castrate’, ‘testicles’, ‘pre’, ‘implant’, ‘suprelorin’, ‘both testicles’, ‘op’, ‘pre op’, ‘for castration’, ‘descended’, ‘check’, ‘for castrate’, ‘scrotum’) had an odds ratio of 13.7 in males compared to females. Additionally, odds ratios for muzzling (a likely marker of aggression), seizures, coughing, sneezing and skin disease topics were higher for males while odds ratio for urinary tract infection and mammary disease were lower in males (Fig. 4).

Fig. 4.

Topic distribution for male and female dogs. Selected topics are shown where odds ratios (reference=female) suggest differences between male and female dogs

Outbreak patterns

Analysis of topic occurrence with time allowed for the detection of increases in clinical syndromes on a national level (using this model). Here, a topic relating to gastrointestinal signs revealed a seasonal increase in these signs in winter with a pronounced spike in activity of this topic in winter 2020. A marked increase in activity of a topic relating to respiratory signs was seen in Autumn 2021 (Fig. 5).

Fig. 5.

Examples of topics highlighting potential disease outbreaks. The distribution of topics with time are shown for a topic most-likely representing gastroenteric disease and one representing respiratory disease. The former reflects a national outbreak of gastroenteric disease in spring 2020, the latter, a potential national increase in respiratory disease occurring in autumn 2021

Syndrome seasonality

Given the ease of screening the patterns of topics with time, it was straightforward to identify clinical syndromes with marked seasonality. Thus topics relating to grass seed foreign bodies, firework anxiety and removal of ticks from the patient showed marked, repeatable seasonality (Fig. 6).

Fig. 6.

Examples of topics that illustrate detection of seasonal patterns, in this case, firework anxiety (orange), tick infestation (blue) and grass-seed trauma (green)

Temporal trends

The key words for each topic often indicated that use of a specific drug was a core feature in that topic, allowing an evaluation of trends in use of those drugs with time. Example of these trends are seen in Fig. 7 where consultations describing meloxicam usage decrease in proportion with time where the proportion of consultations describing use of occlacitinib increased steadily with time and bedinvetmab steeply after 2020.

Fig. 7.

Examples of topic that show long-term trends with time. In this case consultations discussing use of drugs Occlacitinib (Apoquel®), meloxicam and bedinvetmab (Librela®)

Effects of lockdown

At the point of the initial lockdown for COVID in April 2020, the number of veterinary consultations decreased significantly [40] however veterinary visits continued and certain topic-labelled narratives formed a higher proportion of consults. These were identifiable as increased description of specific syndromes such as vestibular disease, torn nails, and consultations resulting in euthanasia of the dog. Other topics relating to routine health care did not occur at an increased proportion (Fig.8).

Fig. 8.

Clinical priorities during COVID-19 lockdown. During lockdown, certain topics became less common and others were over-represented as a proportion of consultations visits. These included visits for painful, distressing or terminal disease. The proportions of routine health care visits did not increase

Discussion

To realise the potential of big data, it is clear that annotation of records for specific features can not be performed manually at the million-record scale due to lack of time, compliance, limitation of the feature set that can be assessed and accuracy of annotations. We have shown that a neural language model method can be used to build a representation of clinical syndromes affecting 1 million canine patients.

As an internal validation of the method, the high proportion of highly-coherent topics in the resultant model, combined with the strong overall average, suggests that the BERTopic model successfully identified distinct and semantically meaningful themes within the corpus, indicating that the model parameters were appropriately tuned and that the resulting topic structure provided a reliable foundation for further thematic analysis and interpretation.

The contrast between high topic coherence and low individual assignment probabilities (median: 0.049) reflects the embedding-based similarity approach and the overlapping nature of clinical presentations, where document embeddings may be positioned between multiple topic clusters in the semantic space. Despite this uncertainty in similarity scores, the high topic coherence confirms that the identified topics represent clinically meaningful and semantically distinct phenotypes. This is appropriate for surveillance applications where the goal is pattern detection. Additionally, a number of known breed predispositions (e.g. Samoyed predisposition to diabetes) and for instance predictable age related topics (e.g. euthanasia) provided some external validation of the model.

The underlying text in the narratives was not preprocessed prior to tokenisation and embedding for analysis. Consequently stop words like ‘no’, ‘not’ and ‘if’ do appear as common words for several topics. We elected not to remove these, respecting the advice of the BERTopic package author [13], because the underlying BERT model was not trained with stop-words removed and because BERT models have been demonstrated to capture negation clues in clinical text [20]. Additionally, the resultant topics were coherent and overall meanings for the topics remained, albeit subjectively, easily interpretable, aligning with breed and age and temporal distributions where these were already known.

The topic model presented a rich overview of clinical features in the data, allowing a review of the prevalence of clinical syndromes across breed, age, sex and with time. As an unsupervised method, topic modelling allows the clinical phenotypes inherent to the data to surface without stipulating what those phenotypes should be. Consequently, a rich set of features can be exposed, and a representative sample of which are highlighted here.

For dogs, breed predisposition to disease is always a complex challenge, even when big datasets are available, because the clinical phenotype under study may require manual annotation. The number of records that can be annotated leads to a significant reduction in group sizes when dividing down to individual breeds, of which there are many 100 s in dogs. Thus, compromises are made in studies whereby only limited breed sets can be evaluated. With automated labelling, far more records are annotated, and more representatives from each breed can be evaluated. Additionally, Topic modelling immediately incorporates numerous phenotypes in a single study.

Here, we evaluated records annotated with topics relating to four common endocrinopathies. In the case of diabetes mellitus, breed predispositions noted in previous big-data studies were clearly identifiable for Border terriers, West Highland White Terriers and Yorkshire Terriers [15]. In that study, case numbers were limited due to a methodology requiring keyword search and manual reading. The wider sampling facilitated through topic modelling as described here also allowed the identification of other, less common, breeds with possible disease predispositions, such as Scottish terrier, Siberian Husky, Samoyed [19] and Bichon Friese, additionally highlighting Boxers, Pugs and French bulldogs as under-represented breeds.

Hypothyroidism was more common in Dobermans, Scottish terriers, Boxer and Samoyed in line with a recent review [30]. Hypoadrenocorticism was seen in Beared collies, Parson Russel Terriers, Poodles and Labradoodles and Hyperadrenocorticism in West Highland White terrier, Border terrier, Bichon Friese and Scottish Terriers.

Similar reviews of the phenotype were possible, such as the evaluation of the distribution of topics relating to heart murmur and upper and lower respiratory tract disease. In this case, heart murmurs were discussed more often in Cavalier King Charles Spaniels (a long recognised predisposition [7]) and less often in Pugs, whereas in this group of breeds, sneezing was more common in Pugs and Chihuahuas and less common in French Bulldogs. A holistic study such as shown here allows for a broader and more efficient assignment of disease processes allowing, in particular, screening for underlying genetic mutations in populations of breeds at varied risk, including rarer breeds not assessed in smaller studies.

For each consultation in this dataset, full patient data (breed, age, sex, neutering status and geographical location) was available, and a comprehensive logistic regression analysis would have been possible but outside the scope or scale of this study. Here, the key feature of this study is the range of phenotypes that can be explored. For example, where previous studies have been able to capture information about diabetes mellitus, the approach demonstrated here concurrently captured data about three other endocrinopathies with no extra investment of time. To the author’s knowledge, a comprehensive single study identifying wide ranging disease predispositions in this manner has not been performed.

The same model, when examined using the topics_per_class method using age as a class, allowed a rapid evaluation of the distribution of given topics with age revealing a variety of patterns, some of which were quite predictable (puppy vaccination in young dogs, booster vaccination with slow decline according to age, euthanasia more common in older animals, dental disease steadily increasing with age) with others less intuitive at first glance (lump/mass discussion in middle age, vestibular disease increasing extremely steeply after 10 years of age). This methodology is ready to combine classes of age and breed in order to highlight how individual breeds variably express specific phenotypes with age and can add to discussions of the different rates of ageing in different breeds [18] and complements other ML-based approaches to characterising factors leading to euthanasia in clinical notes [11].

When topics were analysed per class based on patient sex, numerous examples of topics were more common in one sex or the other. Unsurprisingly, consultations relating to male neutering and female neutering and mammary disease were strongly biased, but additionally, key known predispositions were immediately surfaced, including seizures in male dogs [9], grass seed injury in male dogs [3], urinary tract disease in female dogs [5]. Additionally, consultations describing dog aggression were more common in male dogs, in line with previous studies that indicate male dogs are more prone to human-oriented aggression [22]. Critically, a number of female dog narratives were labelled with the topic with keywords including testicles and castration. These frequently related to descriptions of undescended testicles in puppies brought in with their dam. In some cases these illustrated rare errors where the original data (from the practice management system of the contributing practice) contained the wrong sex highlighting an opportunity to feed back to our contributors on the accuracy of their clinical notes.

Utilising BERTopic’s topic_over_time method offered the opportunity to evaluate various phenotypic features temporally in three ways, namely seasonal disease, unexpected changes in proportion of labelled narratives (potential outbreaks) and long term trends. Here, we demonstrate that phenotypes with known seasonal variation were immediately identified illustrating known patterns of tick infestation, grass seed foreign bodies [1, 3, 42] and fireworks anxiety peaking around bonfire night. Critically this data can be reviewed over time to evaluate the impact of climate change on timing of these effects and in addition, better understanding of the timing of ectoparasite activity will help to inform more focused use of ectoparasiticides which are known to contaminate local environments [32].

The outbreak of canine gastrointestinal disease seen in 2020 [34] was readily identified with subsequent seasonal peaks which have since been identified [6]. Interestingly, the temporal dataset also identified a substantial change in the proportion of narratives labelled with a topic describing kennel cough-like signs (acute respiratory disease) during 2021. To the authors’ knowledge, this has not been demonstrated in any other reports and warrants further investigation. We demonstrated three examples of long term trends in drug usage (as described in clinical notes) suggesting a decrease in the use of a specific meloxicam-containing product (Metacam®), which may reflect the licensing of other products containing the same drug alongside the emergence of a number of COX-2 selective antagonists in recent years. A steady increase in described use of oclacitinib (Apoquel®, a drug used to manage allergic skin disease) was also seen, and a dramatic increase in the description of the use of bedinvetmab (Librella®, a drug used to manage arthritis), which coincided with its release in 2021.

This temporal approach also allowed for the evaluation of the impact of COVID-19 lock-downs. Thus, syndromes associated with pain (torn nails), distress (vestibular disease) and terminal illness and euthanasia represented a larger proportion of visits. These would all be associated with acute distress for the patient and their owners leading owners to pursue immediate care where perhaps vaccination and routine health care were considered acceptable to delay [21].

The sentence-embedding model used has a constrained input length of 512 tokens which means that a fraction (approximately 0.2%) of records will have been truncated prior to embedding and subsequent clustering which may have led to some loss of detail and range of topics detected by the system. In future studies, models with larger input lengths will be usable. The study’s unsupervised topic modelling on a large veterinary clinical dataset offers a broad view of clinical syndromes among one million canine patients, however, the constraints set on cluster size to avoid vacuous or duplicated topics prevent rarer topics from being exposed, which may lead to important but very uncommon syndromes being missed. While topics are often easily interpreted from their key-words, their remains a requirement to audit the underlying narratives where critical conclusions are being drawn. In future work, studies will include setting thresholds for topic probability when attributing a topic to a given consultation in order to improve labelling accuracy. Despite these constraints, the study provides a foundational exploration of clinical phenotypes, offering potential avenues for future investigations with access to full patient histories for in-depth analyses.

Conclusion

Unsupervised BERTopic topic modelling, when applied to a corpus of veterinary clinical notes, surfaces a diverse array of clinically relevant phenotypes which can be used to expose breed, age and sex predispositions to disease and highlight seasonal and outbreak variations in the occurrence of disease in a single experiment. Additionally, this approach allows the visualisation of trends in the appearance of clinical signs and treatment modalities and changes in treatment priorities during lockdown. This methodology leverages a freely available neural language model but is of particular value in this setting because of the availability of the large SAVSNET dataset with national coverage of clinical records. The model can be used to classify narratives unseen during the initial training, and the full array of topics by breed, age, sex, and date is available to view at https://public.tableau.com/app/profile/savsnet.at.liverpool/viz/topic1000000dogspjtrial/topicrisk

Finally, while this study is focused on population-level phenotypes, the resulting knowledge can directly inform the care of individual patients. Understanding population trends, such as confirmed breed predispositions to endocrinopathies, allows clinicians to perform proactive, evidence-based screening and preventative care. Knowledge of drug use trends (Fig.7) helps anticipate shifts in therapeutic standards, and timely outbreak detection (Fig. 5) allows for immediate, informed treatment of individual animals during high-risk periods. Thus, these large-scale epidemiological insights can translate directly to improved clinical decision-making and patient outcomes.

Author contributions

PJN experimental design, wrote the main manuscript. SF experimental design and BERT language model selection. NA experimental design. AR experimental design, data management and provision. All authors reviewed the manuscript.

Funding

This work was partly supported by a Dogs Trust Canine Welfare Grant. Sean Farrell is funded through BBSRC.

Data availability

The datasets generated and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request subject to ethical approval. A sample of such narratives is available at https://www.liverpool.ac.uk/media/livacuk/savsnet/SAVSNET,sample,vet,data.xlsx.

Declarations

Ethics approval and consent to participate

Owners contributing data to the SAVSNET system have the option to opt out of doing so, and the project has University of Liverpool ethics committee approval (RETH001081).

Competing interests

While this work was not directly funded by any specific body, SAVSNET does undertake some commercial work for a variety of companies in order to maintain the projects sustainability.