Expert annotation of free-text descriptions of dream enactment behaviours associated with REM sleep behaviour disorder

Abstract

We developed a database of free-text descriptions of dream enactment behaviours (DEBs) from unstructured clinical notes of United States Veterans with computationally phenotyped diagnoses of rapid eye movement sleep behaviour disorder (RBD). We additionally examined characteristics of DEBs considered essential for RBD diagnosis according to expert consensus. Free-text DEB descriptions were manually extracted from the clinical records of 200 randomly sampled Veterans; RBD computational diagnoses were defined as ≥2 ICD codes for RBD (>30 days but <390 days apart) and procedural codes for in-lab polysomnography within 6 months. DEB descriptions were categorized by clinical features and labeled as essential or not through expert consensus. The proportions of essential versus non-essential DEB types were compared with Chi-Square of Fisher’s exact tests. Out of 195 unique DEB descriptions, experts labeled 124 (63.6 %) as essential RBD symptomatology. These included variations of “dream enactment behaviour” (e.g., “acted out dreams”); hitting, punching, or kicking bedpartners, pets, and/or bedside objects; and other mentions of sleep-related violence. Non-essential descriptions mentioned sleep-talking, thrashing, vivid dreams, nightmares, and sleeping in separate beds. Descriptions of sleep-related movements (n = 119) were more likely labeled as essential (75.6 %) versus non-essential (24.4 %; p = 0.00). Potentially violent or injurious movements were more likely labeled as essential (91.4 %) versus non-essential (8.6 %; p = 0.00). Expert labeling emphasized the significance of sleep-related movements and potentially injurious behaviours for RBD diagnosis in free-text medical records. A natural language processing algorithm that detects DEBs from free-text could further characterize sleep-related violence and injury risks associated with DEBs according to demographics and comorbidities.

1. Introduction

Rapid eye movement (REM) sleep behaviour disorder (RBD) manifests clinically with self or bed partner reported movements and/or vocalizations emerging from sleep, often as dream-enactments [1]. These dream enactment behaviours (DEBs) range from conversational sleep-talking and isolated limb twitches to yelling profanities and strangling one’s bedpartner [2–4]. Vigorous or violent forms of DEBs may result in lacerations, fractures, subdural hematomas, and other potentially lethal outcomes [4]. Accurate identification of DEBs associated with RBD facilitates appropriate treatment and, in idiopathic/isolated cases, longitudinal monitoring for the development of a degenerative synucleinopathy [5].

We previously validated computational phenotyping of RBD patients by using relevant ICD-9 and -10 codes [6]. To further define phenotypes of RBD, a greater understanding of its clinical presentations is required. Natural language processing (NLP) pipelines can extract symptomatology from millions of notes; however, NLPs need training. In addition, large language models (LLMs) may improve RBD diagnostic certainty and further assist in determining RBD phenotypes by understanding the correct terminology related to RBD.

For that purpose, and given the range of potential manifestations of DEBs, we characterized RBD symptomatology by extracting and classifying DEB free-text descriptions from the electronic medical records of a national sample of United States (US) Veterans. We additionally analyzed features of DEBs that were considered essential for RBD diagnosis according to expert consensus.

2. Methods

The study sample was derived from EMRs in the national US Veterans Health Administration (VHA) database from January 10, 1999 to 09/30/2020 and with ≥1 inpatient or outpatient encounter(s) associated with an ICD-9 or ICD-10 code(s) for any sleep disorders (N = 4,237,444) [7]. We randomly sampled 200 Veterans with computationally phenotyped diagnosis of RBD, defined as ≥2 ICD-9 (327.42) or ICD-10 (G47.52) code(s) for RBD occurring >30 days apart but within 390 days of each other [8] and a Current Procedural Terminology (CPT) code for a diagnostic, titration, or split night polysomnogram within 6 months of the initial diagnosis. As this computational definition of RBD was based on available structured data from the EMR, the presence of REM sleep without atonia was not verified through direct review of polysomnography recordings.

A subspecialist in Behavioural Neurology & Neuropsychiatry (MBJ) manually reviewed the clinical notes from the date of initial ICD diagnosis for any description of DEBs and clinical symptoms of RBD. Free text descriptions of DEBs were then reviewed in consensus meetings with two senior, board-certified experts in sleep medicine and RBD (CHS, AS) and a senior subspecialist in Behavioural Neurology & Neuropsychiatry (REJ). Final categorizations were determined through group discussion until full consensus was reached. Disagreements, which most often involved behaviors such as sleep-talking, sleeping in separate beds, and thrashing, were resolved through iterative discussion.

Free text descriptions were categorized as pertaining to movements; vocalizations; the dream itself; and/or consequences (e.g., injuries) according to consensus. The group further characterized movement descriptions as simple (e.g., thrashing) versus “complex” or seemingly goal-directed movements (e.g., punching) and by their potential for injury via similar methods. Finally, DEB descriptions were rated by group consensus as being essential (i.e., core, defining) versus non-essential (non-specific) for RBD symptomology. Overlapping categorization was permitted, such that a single phrase could contain one DEB type categorization. On the other hand, each free-text phrase was mutually exclusive and either labeled as essential or not.

The proportions of DEB categories considered essential versus non-essential were compared via Chi-square or Fisher’s exact tests. A p-value <0.05 was considered statistically significant. All statistical analyses were performed with STATA [9]. This research was approved by the Institutional Review Board at Baylor College of Medicine and Research & Development Committee of the Michael E. DeBakey Veterans Affairs Medical Center.

3. Results

The demographics of the sample were: mean age 60.6 ± standard deviation 14.1 years-old; 192 (96 %) males; 131 (65.5 %) white; and 26 (13 %) Hispanic. Chart review yielded 218 descriptions of DEBs in free-text medical records. Of these, 10 were duplicate, and 13 were determined by expert consensus to be too unclear for further categorization. The latter included mentions of sleepwalking (e.g., “got out of bed and walked to his chair”); vague motor activity (e.g., “muscle tensing”); and other uncategorizable descriptions (e.g., “REM atonia symptoms”). Removal of repeated and potentially unrelated DEB descriptions yielded 195 unique phrases for further annotation.

Examples of DEB descriptions and categorizations are shown in Fig. 1. Free-text descriptions of movements (n = 119) included various terminologies for DEBs (e.g., “dream related behaviour”); thrashing in bed; punching or kicking bedpartners; falling out of bed; and movements occurring during typical indicators of REM sleep (e.g., “movements in the early morning”). Descriptions of vocalizations (n = 11) mentioned talking, mumbling, shouting, or yelling during sleep. Descriptions of dream content (n = 30) included fighting, being chased, and combinations of movements with dreams (e.g., “kicking and dreaming”). Descriptions of consequences (n = 40) mentioned DEB-related injuries (“hurt his wife,” “injury to self or wife”); implementation of safety maneuvers (“safety proofed his room”); and needing to sleep in separate beds. Three descriptions were categorized as “unclear”: two mentioning the presence of a “parasomnia” and one describing a vague safety maneuver.

Fig. 1. Examples and expert annotations of free-text clinical descriptions of rapid eye movement sleep behaviour disorder from electronic medical records.

move. = movement; behav. = behaviour; vocal. = vocalization; conseq. = consequence.

Table 1 shows the comparison of DEB categories considered to be essential versus non-essential according to expert consensus. Experts were more likely to label descriptions of movements as essential (75.6 %) versus non-essential (24.4 %; p = 0.00). In contrast, descriptions of vocalizations, dream content, and “unclear” DEB types were more likely to be labeled as non-essential, and descriptions of consequences were not statistically significant (Table 1).

Table 1.

Comparison of dream enactment behaviour (DEB) categories labeled as essential versus non-essential by expert consensus.

		Essential
		Yes (N = 124)	No (N = 71)	p-value
All DEBs (N = 195)
Movements:	Yes	90 (75.6)	29 (24.4)	0.000
	No	34 (44.7)	42 (55.3)
Vocalizations:	Yes	3 (27.3)	8 (72.7)	0.019
	No	121 (65.8)	63 (34.2)
Consequences:	Yes	26 (65.0)	14 (35.0)	0.835
	No	98 (63.2)	57 (36.8)
Dreams:	Yes	13 (43.3)	17 (56.7)	0.012
	No	111 (67.3)	54 (32.7)
Type unclear:	Yes	0 (0)	3 (1.0)	0.047
	No	124 (64.6)	68 (35.4)
Movements (N = 119)
Complex behaviours:	Yes	75 (85.2)	13 (14.8)	0.000
	No	15 (48.4)	16 (51.6)
Potentially injurious:	Yes	64 (91.4)	6 (8.6)	0.000
	No	26 (53.1)	23 (46.9)

P-values were derived from chi-square or Fisher’s exact tests. Overlapping categorization was permitted, allowing a single phrase to include more than one DEB type classification. However, each free-text phrase was mutually exclusive and labeled as either essential or non-essential. Percentages in the table reflect row percentages, i.e., the percentage of essential by DEB type.

Of the 119 descriptions of movements, 88 were further classified as complex behaviours, 28 as non-goal directed or simple movements, and 4 as unclear movement types. Note that one excerpt, “thrashing and occasional pushing and moving around in the bed,” contained both simple/non-goal directed (“thrashing”, “moving around”) and complex (“pushing”) movements. Experts were more likely to label descriptions of complex behaviours as essential (85.2 %) versus non-essential (14.8 %; p = 0.00) for RBD symptomatology p = 0.000).

In addition, group consensus categorized 45 (37.9 %) movements as violent (e.g., kicking bedpartners); 25 (21 %) as non-violent but potentially injurious (e.g., “falling out of bed”); and 49 (41.2 %) as not possessing clear potential for injury (e.g., “dream enactment behaviours”). Movements associated with injury (both violent and non-violent) were more likely to be labeled as essential (91.4 %) versus non-essential (8.6 %; p = 0.000).

4. Discussion

This database of free-text descriptions of RBD symptoms provides insight into the phenomenology of DEBs associated with RBD and highlights specific features considered essential for diagnosis by expert consensus. Among these, descriptions of movement (particularly those characterized as complex or with injury potential) were significantly more likely to be considered diagnostically essential by experts. Expert consensus aligns with core diagnostic criteria for RBD, which emphasize the presence of purposeful or violent movements during REM sleep that typically mirror dream mentation [1,4,5].

Vocalizations and descriptions of dream content were infrequently labeled as essential, despite being commonly reported features of RBD in diagnostic criteria and symptom severity scales [1,10]. Similarly, descriptions focusing solely on consequences (e.g., sleeping in separate beds) were not consistently rated as essential. This likely reflects their relative non-specificity in isolation, as vocalizations, vivid dreams, and changes to sleeping arrangements can occur in other parasomnias or sleep disorders without the presence of REM sleep without atonia.

Importantly, this study also demonstrates the feasibility of extracting meaningful symptom data from electronic medical record free-text notes for the study of sleep disorders. Manual review and expert consensus allowed for granular classification of DEBs, which may inform the development of natural language processing (NLP) approaches to automate symptom extraction at scale. Such tools could facilitate population-level phenotyping of RBD and related parasomnias, enabling identification of comorbidities most often associated with sleep-related violence, injury potential, and neurodegeneration risk. Expert annotation of DEB descriptions provides the first structured reference dataset for NLP-based phenotyping of RBD. The resulting taxonomy of movements, vocalizations, consequences, and dream content establishes a foundation for training and validating automated classifiers. Future work will integrate these annotated data into machine-learning pipelines to test predictive accuracy and generalizability across large EMR datasets.

Several limitations merit consideration. First, our sample was limited to US Veterans with electronic codes for RBD diagnoses and polysomnography. We enhanced case ascertainment rigor by requiring at least two diagnoses and a procedural code for polysomnography [6], and our definition is consistent with other database investigations [8]. However, this computational definition does not directly confirm REM sleep without atonia, the polysomnographic hallmark of RBD. Because the present analysis was based on electronic medical record data, direct review of PSG recordings or quantification of REM sleep without atonia was not feasible. This limitation may contribute to diagnostic heterogeneity and potential misclassification within the cohort; however, our requirement of multiple RBD diagnostic codes and documentation of PSG was intended to enhance specificity and approximate clinically recognized RBD in real-world practice. Future studies are needed to validate our findings in cohorts with video-PSG confirmation. Moreover, our sample was predominantly comprised of men; while this mirrors sex differences reported in other RBD samples [11], it may limit the generalizability of our findings to women. In addition, our reliance on clinician documentation also introduces variability in how DEBs are described, with potential underreporting of vocalizations or dream content if not deemed clinically relevant at the time of encounter. Patients with small, non-violent movements may be less likely to seek or be referred to clinical care. Systematic review of video polysomnography also indicates that simple, jerking movements predominate over complex, violent movements in patients with polysomnography confirmed RBD [12]. Future research should assess agreement between clinician free-text descriptions of symptoms in medical records with patient self-report and/or documentation of DEBs with video polysomnography.

Despite these limitations, our findings offer a structured framework for understanding how DEBs manifest and are documented in real-world clinical settings. Future work should assess the utility of NLP models for automated DEB classification and explore whether specific symptom profiles predict sleep-related injury or phenoconversion to a neurodegenerative disorder. Further, validated LLM models using the data produced by our study may improve RBD diagnostic certainty and further assist in determining RBD phenotypes.