Early identification of persistent somatic symptoms in primary care: data-driven and theory-driven predictive modelling based on electronic medical records of Dutch general practices

Abstract

Objective

The present study aimed to early identify patients with persistent somatic symptoms (PSS) in primary care by exploring routine care data-based approaches.

Design/setting

A cohort study based on routine primary care data from 76 general practices in the Netherlands was executed for predictive modelling.

Participants

Inclusion of 94 440 adult patients was based on: at least 7-year general practice enrolment, having more than one symptom/disease registration and >10 consultations.

Methods

Cases were selected based on the first PSS registration in 2017–2018. Candidate predictors were selected 2–5 years prior to PSS and categorised into data-driven approaches: symptoms/diseases, medications, referrals, sequential patterns and changing lab results; and theory-driven approaches: constructed factors based on literature and terminology in free text. Of these, 12 candidate predictor categories were formed and used to develop prediction models by cross-validated least absolute shrinkage and selection operator regression on 80% of the dataset. Derived models were internally validated on the remaining 20% of the dataset.

Results

All models had comparable predictive values (area under the receiver operating characteristic curves=0.70 to 0.72). Predictors are related to genital complaints, specific symptoms (eg, digestive, fatigue and mood), healthcare utilisation, and number of complaints. Most fruitful predictor categories are literature-based and medications. Predictors often had overlapping constructs, such as digestive symptoms (symptom/disease codes) and drugs for anti-constipation (medication codes), indicating that registration is inconsistent between general practitioners (GPs).

Conclusions

The findings indicate low to moderate diagnostic accuracy for early identification of PSS based on routine primary care data. Nonetheless, simple clinical decision rules based on structured symptom/disease or medication codes could possibly be an efficient way to support GPs in identifying patients at risk of PSS. A full data-based prediction currently appears to be hampered by inconsistent and missing registrations. Future research on predictive modelling of PSS using routine care data should focus on data enrichment or free-text mining to overcome inconsistent registrations and improve predictive accuracy.

Strengths and limitations of this study

• This is the first cohort study to apply predictive modelling 2 years prior to persistent somatic symptoms onset, based on a large sample size (n=94 440) and at least 7 years of temporal data.

• This study used a wide range of predictors with high clinical relevance and generalisability to general practice.

• Different data-driven and theory-driven approaches for identifying candidate predictors were employed and provide insight into the utility of different approaches.

• The predictors’ generalisability to the general population and interpretation should be done with caution since predictor registration depends on consulting and registration behaviour.

Introduction

In the general population, up to 10% of adults experience persistent somatic symptoms (PSS) that cannot be fully attributed to established biomedical pathological mechanisms.^1–4 PSS are present in both patients with well-established diseases, such as cancer⁵ and cardiovascular disease,⁶ and in patients with symptoms without well-established biomedical pathology.¹ PSS are not only burdensome to the patient,⁷ but also greatly impact healthcare.⁸ For instance, in general practice, up to 50% of consultations are related to symptoms which are not clearly relatable to biomedical pathology.⁹ Most of these symptoms are self-limiting and do not need further investigation or treatment. However, identifying patients at risk of developing persistent symptoms is generally challenging.¹⁰

Definitions of PSS are ever-changing. Historically, PSS classification was based on the exclusion of well-established physical conditions.¹¹ Recent developments lack such a distinction and focus on more positive definitions (including dysfunctional symptom perceptions).^{12 13} Moreover, PSS may be defined under broad ‘umbrella’ terms or based on specific syndromes such as irritable bowel syndrome (IBS), fibromyalgia (FM) or chronic fatigue syndrome (CFS). Previous research debated the distinctness of specific syndromes.¹⁴ However, nowadays most experts accept accumulating evidence that there are both overarching common factors as well as syndrome-specific aspects to PSS.^{15 16} Similarly, differing terminology is used between healthcare professionals. For instance, in psychiatry the umbrella term ‘somatic symptom disorder’ may be used, whereas in general medicine the term ‘functional somatic symptoms’ is used.^{13 17 18} Lastly, some physicians refrain from using terms beyond well-established biomedical disorders for somatic symptoms.^{19 20} In this paper, we use the term PSS, since we aim to approach identifying the broad spectrum of patients with persistent symptoms without well-established pathophysiology, and since recent research indicates that this term is generally preferred over other umbrella terms.²¹

Ambiguity in definitions and terminology has contributed to hampered (early) identification and proactive clinical intervention of patients at risk of developing PSS.^22–24 For instance, research shows that patients with fibromyalgia are diagnosed around 6 years after symptom onset.²⁵ Consequently, PSS are related to inappropriate and relatively high healthcare utilisation and costs.^26–28 Especially in many Western countries, where general practitioners (GPs) serve as a gatekeeper for specialist healthcare.^{29 30} To prevent unnecessary referrals and medicalisation, with potential risk of iatrogenic harm, and to enable the initiation of proactive interventions, early identification is necessary.^{31 32} However, there are many barriers towards the identification of PSS in primary care.^{10 19} For example, diagnosis may be difficult due to the predominance of the biomedical disease model, fear of missing malignancy or other life-threatening conditions, the GP’s experience and knowledge relating to PSS and consultation constraints like overloaded surgery hours. Research from a European network of experts in the field stresses the need for a systemic change to overcome these challenges.³³ Furthermore, research shows that an integrative care approach (with attention to psychological, social, interpersonal and contextual factors, in addition to keeping track of any biomedical deterioration) is needed to improve care for PSS.^{34 35}

Over the years, several screening tools for patients with PSS-related issues were developed for clinical use.^{1 36–38} While diagnostic accuracy and validity have been demonstrated, the widespread use is not forthcoming. A survey of Dutch GPs showed that GPs are still in need of tools for PSS-related diagnostics.²⁰ Studies have shown that routine care data can be responsibly used for predictive modelling.^{39 40} The development of prediction models based on routine primary care data may enable screening based on readily available clinical information and support GPs in their practice. Recent studies reveal the multi-applicability of routine care data since it can be used in several different ways. Approaches range from the more classic theory-driven approaches, simple data-driven approaches⁴¹ and more complex temporal data-mining techniques.^{39 40}

This paper represents the first attempt to develop a clinical decision rule for PSS onset based on routine primary care data. The study aims to predict what patients are at risk of developing PSS 2 years prior to onset and explores different candidate predictor selection approaches. While a theory-driven approach is well-established and has a long history in science, especially in cohort studies, the use of routine care data potentially provides an approach that is more generalisable to clinical practice. Moreover, since we cannot control variable collection, we are interested in how theory-driven variable selection performs compared with non-routinely collected studies. Therefore, the present study explores different theory and data-driven approaches of variable selection, and their combinations, to identify the best approach for the predictive modelling of PSS.

Methods

Study design

A population-based retrospective cohort study was performed using data from 76 primary care practices affiliated with the extramural Leiden academic network (ELAN) of the Leiden University Medical Center (LUMC), the Netherlands. First, the onset date of PSS was determined according to the approach described below (see the Outcome section) within the period 1 January 2017 until 31 December 2018 (random ‘onset’ dates were selected for patients without PSS). Thereafter, candidate predictors were selected 2–7 years prior to the onset date (ie, for each patient 5 years of data was used to select candidate predictors). The ELAN data consists of several subsets, including demographic data (gender, year of birth), consultations (dates, coded symptomology and diagnoses according to the Dutch version of the WONCA International Classification of Primary Care (ICPC)⁴²), prescribed medication (dates and coded WHO anatomical therapeutic chemical (ATC) classification⁴³), laboratory test (dates and results) and correspondence data (dates and type of healthcare professionals (eg, profession/specialty of the other professional).⁴⁴ Part of the consultation registration is the ICPC-coded episode registration, where chronic disorders are registered. The episode data may be available up to the date of birth.

Study population

Patients aged 25–100 years from the ELAN data warehouse were used for this study. Participating practices were located in the greater Leiden and The Hague area. In general, all Dutch residents are enlisted and registered at a general practice in their neighbourhood. Primary care is included in the mandatory Dutch insurance and free of additional charge for insured citizens. The ELAN data warehouse consists of pseudonymised routine healthcare data extracted from the electronic medical records (EMRs).⁴⁵ Inclusion criteria were: registered at the general practice for at least 7 years, having at least 10 contacts and 1 ICPC code. These criteria were used to ensure availability of enough registrations per patient to enable candidate predictor construction. Furthermore, due to higher likelihood of registration errors, patients who were over 100 years of age on 31 December 2018 were excluded from the study. Because we were interested in PSS onset prediction, patients who were registered with PSS before 1 January 2017 were excluded from the analysis.

Outcome

The definition of PSS is based on an earlier analysis by our research group, for which the same ELAN database was used.³² Three approaches towards PSS identification were applied. Patients were identified as having PSS based on either having (1) ICPC codes for PSS syndromes (A04.01: chronic fatigue syndrome, D93: irritable bowel syndrome and L18.01: fibromyalgia); (2) PSS-umbrella terms, PSS-syndrome or PSS-complaint description in the episode description and/or (3) a score of ≥20 on the somatisation subscale of the four-dimensional symptom questionnaire (4DSQ), registered in the lab results. For a more detailed description of the selection criteria, see Kitselaar et al.³²

Candidate predictors

Different datasets were constructed with specific theory and data-driven candidate predictors of PSS in the ELAN data. Below a brief description of the predictor categories related to each dataset-based model will be given, see figure 1 for an overview of the data extraction steps and online supplemental table S1 for a detailed overview of candidate predictors. Two distinct theory-driven datasets were operationalised: (1) literature-based risk factors of PSS (see Kitselaar et al ³⁵ for more detail) and (2) frequencies of specific PSS-related terms and words in the free text with limited structured registration options (see online supplemental table S1). Data-driven datasets were divided into non-temporal and temporal data-driven datasets. The non-temporal datasets consist of dichotomised medical coding data (symptom/disease codes, medication codes and referrals). The coded symptom/disease dataset was based on ICPC codes categorised into WONCA chapters and code categories.⁴⁶ The coded medication dataset was based on ATC codes reduced to third level (to therapeutic/pharmacological subgroup⁴⁷). The referral dataset was based on correspondences GPs have with other healthcare professionals.

Figure 1.

Diagram showing the data extraction steps for each constructed model. ATC, Anatomical therapeutic Chemical classification; ICPC, International Classification of Primary Care; PSS, persistent somatic symptoms.

The temporal approach consists of contextualised lab results and sequential patterns in medical coding data. Due to the high number of different lab results and inconsistent availability, using reference values for this study was not feasible. Contextualisation of lab results provides a solution to enable interpretability of lab results for individual patients. In relative grounding, a lab value is comparted to its previous value to deter whether values are decreasing, increasing or have remained stable.³⁹ To avoid relatively small fluctuations in lab values as decreases or increases, variables were scaled and a minimum of 5% difference between values was required to count as a change. After relative grounding the number of stable, decreased and increased values per lab measure were used as candidate predictors.

Sequential pattern identification of medical coding data was detected using the Sequential PAttern Discovery using Equivalence classes (SPADE) algorithm.⁴⁸ The SPADE algorithm is an efficient way to find statistically significant patterns in temporal data. To identify patterns with the SPADE algorithm, sequences of registrations (ICPC, ATC and referrals) are ordered by date and subsequent registrations are associated to each object in which it occurs.⁴⁸ Thus, when a patient has multiple registrations on one day these will be separated and combined with possible subsequent registrations (eg, patient X has the following registrations on date Y: fatigue, abdominal pain, anti-constipation drug and date Z: physiotherapy, this will result in three patterns for patient X: (1) fatigue→physiotherapy; (2) abdominal pain→physiotherapy and (3) anti-constipation drug→physiotherapy). We selected frequent patterns as candidate predictors based on having at least 1% difference between patients with PSS and patients without PSS in the support value (ie, prevalence of the pattern in de dataset). Please see Zaki⁴⁸ for a more detailed description of the SPADE algorithm.

Predictive modelling

For predictive modelling, a machine learning approach by means of least absolute shrinkage and selection operator (LASSO) logistic regression was used. Relating to our dataset and aim, LASSO logistic regression has several advantages over other methods. LASSO is especially suitable for unbalanced datasets, in which the outcome classification groups differ greatly in size. Moreover, LASSO avoids overfitting in in case of a great number of candidate predictors⁴⁹ and when multicollinearity is expected.⁵⁰ Regression was chosen because of its general comprehensibility and because previous studies in EMR data have shown this generally preforms all popular methods.^{39 51}

The combined dataset was stratified into a training set (80%) and test set (20%). For training, a fivefold cross-validation, with hyperparameter tuning, was performed on the training set. For each unique model (ie, literature-review, free text, coded symptom/diseases, coded medications, referrals, contextualisation of lab results and sequential patterns) and all combined models (ie, theory-driven, data-driven non-temporal, data-driven temporal and full model), near zero-variance candidate predictors were removed (see online supplemental table S2 for total number of candidate predictors in the model and data sources). To evaluate the predictive value of each model, a sensitivity analysis was performed. This included prevalence independent measures (ie, sensitivity and specificity) and prevalence dependent measures (ie, positive predictive value (PPV) and negative predictive value (NPV)). Notably, PPV and NPV should be interpreted with caution because they are generally low when prevalence is low and their value is debatable when the prevalence in the study is not similar to general population prevalence (for a more detailed description, see 52 53). Finally, the area under the receiver operating characteristic curve (AUC) was calculated. All data was prepared and analysed using R v4.0. For the final modelling, the caret-package was used.

Final model evaluation

To evaluate the models obtained using from model training (using the training dataset) and ensure there was no overfitting of the models, the models were internally validated on the test dataset for their classification performance. Finally, predictors of the final full model were evaluated. Estimated coefficients of predictors included in the final model were presented as ORs. To verify the stability of the predictor estimates, frequencies of estimates receiving non-zero values were calculated across 1000 bootstrap samples.

Role of the funding source

The project was internally funded by the Leiden University and Leiden University Medical Centre interdisciplinary profile area ‘Health Prevention and the Human Life Cycle’. No external funding supported this study.

Patient and public involvement

GPs affiliated with the LUMC health campus were consulted during the development phase of the research design. Meetings with GPs were directed at the formulation of the outcome and construction of candidate predictors. Primary focus was the meaning and application of ICPC codes, lab measures, likelihood of missing data and general workings of EMR. Also locations to find relevant resources were discussed, to increase the knowledge of the data and the best way to interpret registrations.

Results

The total number of patients in the ELAN database we used for our research contained 306 859 patients, of which a total of 202 168 patients were excluded based on available data. A total of 10 249 patients were classified as having PSS before 1 January and therefore also excluded from the study. As a result, 94 440 patients were included in the final analysis (figure 2).

Figure 2.

Flow chart of patient inclusion in the ELAN study cohort. ELAN, extramural Leiden academic network; ICPC, International Classification of Primary Care; PSS, persistent somatic symptoms.

As shown in table 1, 0.9% (n=902) of patients in the ELAN cohort had new-onset PSS. Compared with the total cohort, patients with PSS are more likely to be women (69.0% vs 52.9%, p<0.001), are generally younger (52.6±14.4 vs. 57.2±15.4, p<0.001) and have higher consultation frequency (8.7±7.3 vs. 6.3±5.8, p<0.001). Moreover, patients with PSS are more likely to have a mental health disorder (60.3% vs 46.8%, p<0.001) while the likelihood of a physical disorder does not differ (64.6% vs 63.6%, p=0.87). The patients with new-onset PSS in the training and test sets differ on baseline variable women (68.3% vs 72.2%). Post-hoc evaluation revealed that patients with PSS in the training and test sets also differ regarding the prevalence of mental comorbidities (59.6% vs 63.3%, respectively) and physical comorbidities (65.1% vs 62.8%) (not depicted in table).

Table 1.

Patient characteristics

	Total cohort	PSS
	Full dataset	Full dataset	Training	Test
n (%)	94 440 (100.00)	902 (0.9)	772 (0.9)	180 (0.9)
Female, n (%)	49 998 (52.9)	623 (69.0)	493 (68.3)	130 (72.2)
Age, mean (SD)	57.2 (15.4)	52.6 (14.4)	52.9 (14.5)	51.3 (13.7)
Consultations, mean (SD)	6.3 (5.8)	8.7 (7.3)	7.44 (6.3)	7.2 (5.5)
Urbanisation level, n (%)
Urban area	45 567 (48.2)	404 (44.8)	326 (45.2)	78 (43.3)
Sub-urban area	43 296 (45.8)	448 (49.7)	358 (49.6)	90 (50.0)
Rural	2711 (2.9)	9 (1.0)	7 (1.0)	2 (1.1)
Disadvantage neighbourhood	67 215 (71.2)	622 (69.0)	494 (68.4)	128 (71.1)
Physical comorbidity, n (%)	60 019 (63.6)	583 (64.6)	470 (65.1)	113 (62.8)
Mental comorbidity, n (%)	44 292 (46.9)	544 (60.3)	430 (59.6)	114 (63.3)

PSS, persistent somatic symptoms.

In table 2, the predictive value based on sensitivity, specificity and the AUCs of each unique and combined model is depicted. The AUCs of the validated models varied from 0.68 for the baseline model to 0.72 for the full model. From the separate models, all models preformed equally well, based on an approximate AUC 0.70. PPV is low (ranging from 1.5% to 1.7%) and NPV is high (ranging 99.5% to 99.6%). Using the optimal cut-off selection (ie, highest number of cases selected accurately), the present model would, with 72.2% sensitivity, detect patients at-risk of PSS onset within 2 years (see table 2 for AUC’s and sensitivity analyses, and online supplemental tables S1–S3 for more details on the model contents).

Table 2.

Prediction models based on LASSO logistic regression analysis

		Training	Test
		AUC	Sensitivity	Specificity	AUC
	Baseline model*	0.66	0.73	0.54	0.68
Theory-driven	Literature-based†‡	0.70	0.61	0.68	0.71
	Free text†§	0.68	0.70	0.56	0.71
	Combined*	0.69	0.73	0.60	0.71
Non-temporal data-driven	Symptoms/diseases†¶	0.68	0.72	0.57	0.70
	Medications†**	0.69	0.76	0.58	0.70
	Referrals†††	0.66	0.71	0.55	0.69
	Combined†	0.70	0.57	0.72	0.71
Temporal data-driven	Lab contextualisation†‡‡	0.67	0.73	0.58	0.70
	Sequential patterns†§§	0.66	0.83	0.43	0.69
	Combined†	0.68	0.73	0.58	0.70
	Full model†¶¶	0.70	0.72	0.60	0.72

For a detailed description of the models, see online supplemental table S1.

*Gender, age and consultation frequency.

†It includes baseline model.

‡Variables selected based on literature search of risk factors in the general population.

§Word search through free journal text.

¶ICPC codes categorised according to the WONCA categorisation (dichotomised).

**ATC-3: therapeutic/pharmacological subgroup (dichotomised).

††Outgoing correspondence to medical specialists (dichotomised).

‡‡Relative grounded lab-results (stable, increase, decrease; dichotomised).

§§Order of ICPC, ATC and referrals over time, patterns identified with the SPADE algorithm (see online supplemental table S3).

¶¶All available candidate predictors combined; For a detailed description of the models, see online supplemental table S1.

ATC, anatomical therapeutic chemical; AUC, area under the receiver operating characteristic curve; ICPC, International Classification of Primary Care; LASSO, least absolute shrinkage and selection operator.

Final predictors were derived from the full model. From all candidate predictors used for the full model (n=545), 29 of the variables contributed to the prediction of PSS onset. Predictors stemmed from all predictor type categories, baseline (n=2), literature review (n=8), ATC (n=8), ICPC (n=3), free text (n=2), referrals (n=1), lab contextualisation (n=3) and sequential patterns (n=1). From the baseline predictors, age decreased the likelihood of PSS onset (OR=0.82) and female gender increased (OR=1.13) the likelihood of PSS onset. Baseline variable consultation frequency was not a relevant predictor in the full model, but it was an important predictor in all other models, except for the theory-driven combined model. Some other highly stable predictors using PSS-related complaint description in the free text (OR=1.12) are: having stable lymphocyte counts based on lab tests (OR=84.2); using PSS-related terminology in free text (OR=83.6%); the number of referrals for imaging (OR=1.10); number of medications (OR=1.12) and having a neurological disorder (OR=1.10) (see table 3 for the complete list of predictors and ORs). Frequencies of estimates having non-zero values across 1000 bootstrap samples indicate the level of interchangeability of predictors for other predictors (high percentage indicating higher importance of the predictor for predicting PSS onset).

Table 3.

Predictors of PSS obtained from full model LASSO logistic regression analysis

Predictors	Total cohort % or mean (SD)	PSS cohort % or mean (SD)	OR	%*
Baseline
Age	57.2 (15.4)	52.6 (14.4)	0.82	99.5
Female gender	52.9	69.0	1.13	78.1
Literature based (theory-driven)
Painful intercourse (female)†	1.1	3.1	1.17	60.8
Medications‡	2.0 (1.4)	2.5 (1.6)	1.12	94.7
Number of imaging referrals§	0.09 (0.09)	0.1 (0.1)	1.10	96.1
Fatigue¶	20.5	31.2	1.04	47.5
Mood disorder**	14.6	23.6	1.03	47.7
Number of pain sites††	0.3 (0.6)	0.5 (0.7)	1.02	63.7
Headache§§	19.8	32.6	1.02	44.8
Number of ICPC codes‡‡	2.6 (1.5)	3.3 (1.7)	1.004	13.5
Free text (theory-driven)
Complaint description¶¶	0.7 (1.0)	1.3 (1.6)	1.12	99.3
PSS terminology***	0.06 (0.15)	0.11 (0.21)	1.04	83.6
Symptom/disease codes (non-temporal data-driven)
Neurological disorder†††	18.1	27.3	1.11	77.9
Digestive symptoms‡‡‡	50.4	65.5	1.07	66.7
Female genital symptoms§§§	28.8	46.6	1.07	53.0
Female genital infection¶¶¶	8.3	15.9	1.04	48.9
Medication codes (non-temporal data-driven)
Capillary stabilisers****	0.1	0.7	1.47	57.6
Selective CA+ blockers††††	10.6	6.3	0.93	58.0
Topical contraceptives‡‡‡‡	5.5	10.5	1.06	58.8
Lipid modifier§§§§	21.4	15.6	0.95	54.2
Nasal spray, topical¶¶¶¶	40.1	51.7	1.02	51.1
Anti-constipation drug*****	28.4	40.1	1.02	52.1
Eyedrops, topical†††††	16.2	22.3	1.01	47.3
Anti-thrombotic agents‡‡‡‡‡	20.8	16.0	0.999	41.0
Referrals (non-temporal data-driven)
Physiotherapy§§§§§	30.2	39.5	1.01	43.6
Lab contextualisation (temporal data-driven)
Lymphocytes, stable	0.3 (0.5)	0.4 (0.5)	1.06	84.2
Thyroid, stable	0.5 (1.1)	0.8 (1.4)	1.04	70.3
Systolic blood pressure, stable	1.8 (3.2)	1.5 (2.8)	0.999	39.0
Sequential patterns (temporal data-driven)
Referral to Rontgen	3.1	7.1	1.10	57.6

*Frequency of estimates having non-zero values across 1000 bootstrap samples

†ICPC codes: X04, P08.02.

‡Frequency based on full ATC codes.

§Rontgen or echography.

¶ICPC code: A04.

**ICPC codes: P03, P73, P73.02, P76 and ATC codes: N06A, N05AN, D11A×04.

††Number of pain-related ICPC codes.

‡‡ICPC codes: N01, N02, N89, N90, R09.

§§All unique ICPC codes.

¶¶Fatigue, dizziness, back pain (see online supplemental table S1) for full list).

***For example, somatisation or a-specific symptoms (see online supplemental table S1) for full list).

†††ICPC: N86-99.

‡‡‡ICPC codes: D01-29.

§§§ICPC codes: X01-29.

¶¶¶ICPC codes: X70-74 and X90-92.

****ATC4-codes: C05C.

††††ATC4 codes: C08C.

‡‡‡‡ATC4 codes: G02B.

§§§§ATC4 codes: C10A.

¶¶¶¶ATC4 codes: R01A.

*****ATC4 codes: A06A.

†††††ATC4 codes: S01X.

‡‡‡‡‡ATC4 codes: B01A.

§§§§§Correspondence with physiotherapy.

ATC, Anatomical Therapeutic Chemical classification; ICPC, International Classification of Primary Care; LASSO, least absolute shrinkage and selection operator; PSS, persistent somatic symptoms.

Several of the predictors may have overlapping aetiology or overlapping variable constructs but differ in their data source. This is for instance seen in: (1) female genital symptoms (ICPC), painful intercourse (literature review), both contain ICPC code X04; (2) ‘headache’ (literature review) and neurological disorders (ICPC), both containing ICPC codes N89 and N90; (3) digestive symptoms (ICPC) and drugs for anti-constipation (ATC) and (4) ‘fatigue’ (ICPC) and ‘complaint description’ (free-text descriptors, which contains the term fatigue).

Discussion

This study provides a comprehensive overview of the effectiveness of different approaches towards predicting PSS based on routine primary care data 2 years prior to index date. Model performance based on specific predictor generation approaches does not differ greatly. Therefore, the use of the simplest approach may be most desirable. Based on the full model (including all candidate predictors), predictors associated with PSS onset stem from all predictor categories, although theory-driven and medication types (ATC) predictors were most prevalent. In line with previous literature, important predictors are related to being female (including, painful intercourse, genital infections/symptoms and contraceptives), specific symptoms (eg, digestive issues, fatigue, mood disorders and headache), healthcare utilisation (eg, number of medications or imaging, referrals or physiotherapy) and number of complaints (eg, number of pain sites or ICPC codes). Consistent with knowledge that PSS is unrelated to established biomedical pathology, results show that stable lab results (especially lymphocytes and thyroid) are important indicators of PSS. Notably, constructs of some predictors contain overlapping variables (such as: ‘neurological disorder’ and ‘headache’; and ‘fatigue’ and ‘complaint description’). This indicates that ambiguous registration may result in scattered predictors, which may have contributed to the limited predictive accuracy of the models.

Several strengths and limitations apply to this study. A major strength is the population-based cohort, with high ecological validity, with a large sample size and at least 7 years of data. Second, inclusion in our PSS cohort is based on a previously published approach which has enabled us to select patients beyond the poorly reported ICPC codes for the syndromes,³² and not limited to commonly investigated IBS, FM and CFS.⁵⁴ To our knowledge, we included a wider range of predictors than previous studies, and these are clinically relevant and generalisable to general practice. Moreover, the models were compared based on predictor categories which provides important evidence for more efficient future analyses. Lastly, we have used sophisticated machine learning techniques (temporal pattern mining and relative grounding) and analysis (LASSO regression). This allowed for optimal use of temporal data and enabled us to use all available candidate predictors in one final model. Finally, although the machine learning techniques did not improve the performance of the full model, some novel predictors were identified (ie, stable lab results: lymphocytes and thyroid). On the other hand, the use of routine care data may also limit the generalisability of the predictors to the general population since registration depends on the decision of patients to contact the physician and on the decision of physician/staff what to register. Furthermore, interpretation of predictors should be done with caution since the present analysis is directed at finding the optimal model performance, rather than explaining the outcome. For example, registration of social and psychological predictors may frequently be missing, since medical priorities might be estimated as the more important issues to code and register.^{32 41 52} Finally, the selection of patients with PSS was based on previous research on the same dataset.³² This approach enabled conservative selection of patients with PSS, but may have missed some cases.^{53 55} The aim was to enable data-driven selection and not rely on GP diagnosis, since research indicates that PSS are often missed by physicians.⁵⁶ Data-driven selection would enhance re-usability of routine care data.

To our knowledge, this is the first cohort study to predict PSS 2 years prior to onset. However, previous predictive EMR studies on PSS or PSS-subgroups show better model performance. This may be due to the 2-year prediction gap, which was not applied in previous studies or because of their use of questionnaires or physician-dependent diagnoses.^57–59 A recent study based on the ELAN data warehouse with a non-biomedical outcome showed similar predictive value,⁴¹ which could mean that routine primary care data have limited capacity for non-biomedical outcome measures. However, this study also did not apply a 2-year prediction gap. Prediction models based on other types of large cohort studies have primarily focused on PSS subtypes.^{54 59} Monden et al ⁵⁴ reported notably higher ORs, which may be related to less available confounding variables and/or to active data collection resulting in access to multidomain (ie, more complete social and psychological) data. This is in line with studies showing that GPs are less likely to report social and psychological factors^{19 20 60} and a recent systematic review demonstrating the importance of using multidomain data.³⁵ Lastly, in contrast to a body of evidence,^{59 61 62} our LASSO regression of the full model did not indicate that consultation frequency predicts PSS. Since consultation frequency was predictive in most submodels, findings imply that factors latent to consultation (such as number of imaging referrals or number of ICPC-codes) may be more precise predictors of PSS onset than consultation frequency.

Our study shows how routine primary care data can be used as a source that supports early prediction of PSS. Although predictive accuracy (in particular shown by the low PPV) indicates that it cannot be used without additional screening, relatively simple ICPC/ATC-based models can assist in this process by facilitating an initial broad distinction between PSS and well-established biomedical problems. Predictive values of free text ‘complaint description’ and ‘PSS terminology’ indicate that clinical evaluation and registration of PSS-related psychological and social constructs is important for early identification of PSS. Thus, in combination with the simple ICPC/ATC-based models, available validated screening tools such as the 4DSQ and the somatic symptom disorder - B-criteria scale (SSD-12) might further facilitate early identification of PSS. Moreover, the overlapping constructs of several predictors, which do not correlate highly, indicate a difference in registration behaviour between GPs practices, which may have limited the predictive value of the data. Although sequential patterns and lab contextualisation did not enhance model performance, the former implies that other machine learning techniques (eg, text mining) should be further explored. Especially because of the relatively fair performance of the free text-based model, for which in the present study only limited free text is used.

Results provide clear directions for both clinical and EMR research. Clinical research should be directed at the feasibility of the ICPC/ATC-based models for clinical implementation in combination with additional screening with a validated screening tool (eg, 4DSQ or SSD-12). The screening tools would provide a proxy for the difficulty to systematically register PSS-related aspects captured in the free text. Future research should evaluate criterium validity of the present outcome by selecting the outcome (ie, PSS) using validated screening tools (eg, 4DSQ, SSD-12), and further evaluate if this could enhance accuracy of routine primary care data-based predictions. Furthermore, EMR research should further develop the theory-driven and data-driven approaches. The theory-driven approach could thus be improved by more elaborate candidate predictor construction, combining variables with similar constructs more thoroughly and patient-reported outcome measures. The data-driven approach could possibly be improved using data enrichment techniques or by developing models based on more advanced approaches for free-text analysis.

Data availability statement

Data are available upon reasonable request. No additional raw data are available. Processed data can be made available to researchers upon reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants. The ethics committee of Leiden University Medical Centre supplied a waiver of ethical approval (G19.045/SB/ib), as ethical approval was not necessary for this study, exempted this study. Since there was no active data collection for this study, it was not possible to ask participants personally for informed consent. Patients were given the option to opt-out of research participation in general by their general practice. All EMR data were transferred to a trusted third party that excluded patients who opted out of research participation. The rest of the patients were pseudonymised before transferring the data to us for analysis.