Skip to main content
Data in Brief logoLink to Data in Brief
. 2026 Aug 19;68:113175. doi: 10.1016/j.dib.2026.113175

An audiovisual dataset for emotion recognition (UTeMo) in Spanish language

Jorge-Arturo Carrasco-Jiménez a, Arturo Téllez-Velázquez a, Rosebet Miranda-Luna b, Antonio Orantes-Molina b, Juan-Pablo García-Vázquez c, Raúl Cruz-Barbosa a,⁎
PMCID: PMC13546921  PMID: 42707131

Abstract

Emotion recognition has had remarkable growth in human-computer communication because emotions play an important role in the daily lives of human beings. This task is challenging because emotions are bodily reactions to external stimuli that differ greatly between people. Societies tend to normalize the response to these stimuli depending on their culture and era. In the literature, there are databases designed for the recognition of emotions in various languages, principally English. Although it is possible to find databases in Spanish within this compendium, none of them are specialized in the variant of Mexican Spanish, and most of them contain low image resolution and audio fidelity, which limit their utility for emotion recognition through deep learning as convolutional networks. For these reasons, this data article introduces UTeMo, a new dataset in Spanish language which includes the six basic emotions based on the Paul Ekman's model (anger, sadness, joy, surprise, fear, and disgust) for facial and voice expressions plus a neutral state. The proposed dataset is qualitatively validated to provide a feasible and reliable basis for emotion recognition using machine and deep learning methods.

Keywords: Speech and facial expressions dataset, Machine learning, Deep learning, Basic emotional states, Balanced data


Specifications Table

Subject Computer Sciences
Specific subject area Multimodal emotion recognition, machine learning, deep learning, emotional state classification.
Type of data Videos (.avi) raw
Data collection Before video recordings, documentation, phrase selection and scenario configuration were carried out to have related baseline frameworks, selected Spanish phrases of basic emotions and adopted configuration, respectively. Then, the actuated audio-visual dataset was recorded in a digital media laboratory with specific settings and 14 volunteer actors. That is, an acoustically isolated cabin with a white background, a Canon XF105a video camera and Fluotec 250DMX lights and 450DMX reflectors were used. After video recordings, each video file is labeled according to its emotional state. Finally, a qualitative validation is performed to select the final samples (1801 full audiovisual recordings).
Data source location The data were collected from the Digital Media Laboratory at Universidad Tecnológica de la Mixteca, Huajuapan de León, Oaxaca, México.
Data accessibility Repository name: Mendeley Data
Data identification number: 10.17632/x5rmd28h73.1
Direct URL to data: https://data.mendeley.com/datasets/x5rmd28h73/1
Related research article None

1. Value of the Data

  • •

    The dataset represents a valuable contribution, since after reviewing public repositories such as Kaggle, Hugging Face Datasets, Mendeley Data, and Zenodo, we did not identify any publicly available multimodal audiovisual emotion dataset specifically developed for Mexican Spanish. Existing Spanish-language emotion datasets mainly provide text, audio, or, in some cases, audiovisual material extracted from soap operas, rather than recordings intentionally designed for emotion recognition in the Mexican variant of Spanish. In addition, the proposed dataset offers greater diversity than many currently available resources such as SEV and RekEmozio (see Table 5), comprising 1,801 video samples with a total duration of approximately 105 min (2.7 GB) recorded from fourteen actors (seven women and seven men) with an average age of 27 years. The dataset also provides high-quality audiovisual material, including high-resolution video recordings (1920 × 1080 pixels at 30 fps) and high-fidelity audio captured at a sampling rate of 48 kHz with a camera's built-in microphone.

  • •

    As a public database, it is accessible to anyone interested in leveraging machine or deep learning models to recognize or classify the six basic Ekman’s emotions (sadness, surprise, joy, anger, fear, and disgust) and a neutral state. Furthermore, it can be considered as a dataset whose number of samples is approximately balanced according to the seven emotion classes, so every emotional state is well represented, which is a useful feature for any classifier including deep neural networks.

  • •

    UTeMo consists of emotional vocal and facial expressions, which can be used to promote research in the area of multimodal emotion recognition (such as education through smart tutoring systems or healthcare through mental health applications) in Mexico and Spanish-American countries. Furthermore, it provides researchers with a high-quality audiovisual database in Mexican Spanish, where early, late, and hybrid information fusion strategies can be applied for bimodal emotion recognition.

  • •

    Researchers can adapt UTeMo to train machine or deep learning models as convolutional networks for emotion recognition tasks in several fields such as affective computing, human-computer interaction, or psychology, among others.

  • •

    Deep learning experimental results achieved a 95.65% accuracy on a random split and 87.44% under leave-speakers-out cross-validation with the off-the-shelf FSER architecture, demonstrating the dataset’s suitability for emotion recognition.

Table 5.

Comparison between UTeMo and other similar audio/visual databases. The symbol ``—'' means that this information is not available.

Dataset No. samples No. actors No. emotions Total recording time (hours) Video resolution FPS Audio sampling rate (Hz)
SEV [13] — 2 7 11.6 720 × 576 25 48k
RekEmozio [14] 2618 (audio) 17 7 4.93 (audio)
1.55(video)
320 × 240 — —
102 (video)
SAVEE [6] 480 (audiovisual) 44 7 0.66 — 60 44.1k
Emo-DB [7] 535 (audio) 10 7 0.38 — — 16k
EMOVO [15] 588 (audio) 6 6 0.5 — — 48k
UTeMo 1801 (audiovisual) 14 7 1.75 1920 × 1080 30 48k

2. Background

Regarding the importance of emotion recognition in different fields such as psychology, affective computing or clinical therapy [1], there are many reasons to identify the emotional state of people. However, the task of recognizing emotions is challenging since human emotions lack temporal limits and have a high degree of uncertainty. This is because individuals perceive their environment and consequently express their emotions differently when facing the same stimuli [2].

Due to this fact, several databases have been created in different languages, with the common goal of recognizing emotion. Currently, there are audiovisual databases available in English, German, Chinese, and Spanish languages. However, it's worth noting that existing Spanish databases primarily cater to the Castilian variant. Given the cultural and linguistic diversity among Spanish speakers, there is a need for databases specifically tailored to regional variations, such as Mexican Spanish. Beyond the regional aspect, the quality of the available samples could benefit from enhancements in terms of image resolution and audio fidelity. Therefore, this data article endeavors to address these gaps by introducing a newly curated dataset for bimodal emotion recognition, named UTeMo, with a specific focus on the Mexican Spanish variant—the most widely spoken Spanish variant in the world [3].

3. Data Description

The UTeMo dataset comprises 1801 video samples obtained from 14 Mexican actors (see Table 1). Each actor/actress interprets phrases corresponding to each emotional state. Most phrases were adopted from the work of [4] for the six basic emotions (sadness, surprise, joy, anger, fear, and disgust) and from [[5], [6], [7], [8]] for the neutral state (all phrases are available as a supplementary file in the UTeMo GitHub repository1). Table 2 presents a summary of the recorded statements for each emotion. That is, there are seven different statements for sadness and surprise; six different statements for anger, happiness, fear and disgust; and ten different statements for the neutral state, which makes a total of 48 phrases. During the recordings, each actor was instructed to repeat each statement three times while interpreting the corresponding emotion. This should result in a total of 144 samples per actor, assuming all actors interpreted all emotional states. Following this procedure, a total of 1296 samples were recorded across 9 actors. This occurred because the neutral state was included after the six basic emotions were interpreted, and five actors were unavailable to record it. Subsequently, the remaining samples were obtained from 5 actors interpreting 38 phrases (excluding neutral state) which were repeated three times (amounting to 570 samples). That is, a total of 1866 samples (see Table 3) were originally recorded. To calculate the original sample numbers per emotion shown in Table 3, the reader should multiply the number of statements for the corresponding emotion in Table 2 by the number of repetitions (3) and then multiply that result by the number of actors who interpreted that specific emotion (14 actors for the six basic emotions and 9 actors for the neutral state). For example, a total of 252 samples were originally recorded for Anger, since this is obtained from 6 statements repeated 3 times by 14 actors. Likewise, the neutral state consists of 270 samples, which were obtained from 10 statements repeated 3 times by 9 actors.

Table 1.

Actors’ information.

Actor ID Age Beard? Glasses? Occupation Origin (city, state)
1 24 No Yes Lawyer Huajuapan, Oaxaca
2 20 No No Acting student Huajuapan, Oaxaca
3 20 No Yes Acting student Huajuapan, Oaxaca
4 20 No No Acting student Huajuapan, Oaxaca
5 42 No No Professor Ciudad de México, CDMX
6 18 No No High school Huajuapan, Oaxaca
7 46 Yes Yes Psychologist Ciudad de México, CDMX
8 50 No No Professor Ciudad de México, CDMX
9 22 No No Undergraduate student Huajuapan, Oaxaca
10 24 Yes No Undergraduate student Oaxaca, Oaxaca
11 23 Yes No Undergraduate student Ixtepec, Oaxaca
12 25 No No Undergraduate student Oaxaca, Oaxaca
13 20 No No Undergraduate student Oaxaca, Oaxaca
14 23 No No Undergraduate student Oaxaca, Oaxaca

Table 2.

Summary of recorded statements.

Emotion No. Statement
Anger 6
Surprise 7
Happiness 6
Fear 6
Disgust 6
Sadness 7
Neutral 10
Total 48

Table 3.

Recorded sample distribution for the UTeMo dataset.

Emotion No. original samples No. discarded samples No. approved samples
Anger 252 3 249
Surprise 294 15 279
Happiness 252 10 242
Fear 252 8 244
Disgust 252 8 244
Sadness 294 5 289
Neutral 270 16 254
Total 1866 65 1801

Once files were recorded, they were sorted by emotion and converted to the AVI (Audio Video Interleave) file format. File names of samples were labelled as follows:

Emotion_ActorN_PM_FL

where Emotion corresponds to each one of the seven emotional states presented in Table 2, N ∈ [1,14] represents the actor/actress identifier, M ∈ [1,3] represents the iteration and L is the phrase identifier (L range depends on the corresponding emotion; see Table 2). For instance, the label Happiness_Actor1_P2_F5.avi indicates that the happy emotion was interpreted by Actor 1 while recording the fifth statement in the second iteration. All files using the previously described nomenclature were only compressed and publicly released in the Mendeley data repository. This means that no other folder structure was used. Fig. 1 shows some files in the root folder structure when the zip file (Utemo.zip) is unzipped. In addition, a README file which describes the file nomenclature and provides a link to the UTeMo GitHub code repository was added as a supplementary file in the Mendeley data repository.

Fig. 1.

Fig 1 dummy alt text

Simple folder structure for UTeMo video files.

4. Experimental Design, Materials and Methods

For the construction of this dataset, Mexican actors belonging to public and private theater groups, who live in Huajuapan de Leon, Oaxaca, Mexico participated. In order to build a gender-balanced database, a total of 14 people were considered (7 female and 7 male), with an average age of 27 years old. Likewise, to guarantee correct data acquisition, the RAVDESS [5] database scenario configuration has been adopted, which has a recording equipment distribution similar to Fig. 2.

Fig. 2.

Fig 2 dummy alt text

Employed scenario configuration for recordings.

The workspace and equipment used were provided by the Digital Media Lab at Universidad Tecnológica de la Mixteca (UTM, where the UTeMo name comes from). It consists of an acoustically isolated cabin with a white background. To perform recordings, the following equipment is used:

  • •

    1x CANON XF105a video camera in automatic mode (aperture: between f/1.8 at wide angle and f/8.0 at maximum zoom; shutter speed: near 1/50 sec or 1/500 sec; ISO: sensitivity is controlled through video gain, which ranges from +3 dB to +21 dB)

  • •

    2x Fluotec 250DMX lights with a light temperature of 3700K

  • •

    4x Fluotec 450DMX reflectors with neutral light

Audio was recorded using the built-in microphone of a CANON XF105a camera, which captures audio in linear PCM format (16-bit / 48 kHz). These characteristics correspond to a professional-grade digital audio standard; consequently, the recorded audio can be considered high-quality within the constraints of an integrated recording system.

Due to the importance of capturing clearer and more accentuated facial and vocal expressions, the proposed dataset was built with the help of actors. According to the literature [7], the construction of this type of data allows the acquisition of high-quality recordings since they are generated in controlled environments.

The proposed methodology used to build the UTeMo dataset (see Fig. 3) consists of the following steps:

  • 1.

    Documentation. To adopt an adequate and feasible configuration, a review of the related literature about several reference frameworks for emotion recognition, which consists of analyzing scenario configurations, phrases, and people (actors), was performed.

  • 2.

    Phrase selection. Based on the previous step, a subset of Spanish phrases was selected based on [4], which proposes sentences corresponding to the emotions of anger, sadness, happiness, disgust, fear, and surprise. In the case of the neutral state, they were taken from different databases [[5], [6], [7], [8]] and adapted to the Mexican Spanish language.

  • 3.

    Scenario configuration. The configuration presented in [5] is adopted because it uses minimal specialized equipment without sacrificing quality in recordings.

  • 4.

    Recordings. For constructing an actuated database [9], the help of volunteer actors has been requested to carry out recordings. It should be clarified that all actors have signed a letter of consent about their participation in this dataset.

  • 5.

    Labeling. Once recordings have been performed, each video file is labeled according to its emotional state.

  • 6.

    Validation. Finally, a sequential qualitative validation is performed using: (i) expert and (ii) survey validation.

Fig. 3.

Fig 3 dummy alt text

Methodology used for UteMo database construction.

According to the methodology presented above, the proposed dataset is qualitatively validated in two sequential phases. The first validation phase consists of an expert verifying that the expressions have been made correctly, that is, an expert in the subject of emotions ensures that the actors' interpretations are carried out properly and discards those badly interpreted samples. Whereas the second validation consists of a survey where 30 raters participate. The surveys were applied through the Google Forms platform. In this case, a randomly selected 10% of the database (a representative stratified subset of 181 samples) was presented for rating, where raters were instructed to validate each sample by making two types of judgments as follows:

  • •

    Categorical judgment: To validate that each sample corresponds to the interpreted emotion, each rater can choose between eight different options; one corresponding to each emotional state and an extra option with the label “none of the above”.

  • •

    Intensity judgment: For each sample, raters can express their opinion about how intense the emotion is by choosing the intensity values (Likert scale): not intense, slightly intense, moderately intense, intense, or very intense.

Here, raters were allowed to replay the video samples as needed, and they were blinded to the intended emotion portrayed by the actor.

To properly interpret the results of the judgments made by the raters, a statistical measure known as Fleiss’ Kappa [10] was utilized. Using this performance measure, it is possible to determine the level of agreement between the categorical judgments made by the raters and the actor-intended emotion labels. There are several range division criteria for interpreting the Kappa statistic; for this work, we selected the Landis & Koch [11] criteria, which are among the most established frameworks for interpreting Kappa statistics. Here, the criterion labels associated with strength of agreement are poor, slight, fair, moderate, substantial, and almost perfect. The names of these labels suggest that a ‘fair’ strength of agreement (0.21 < Kappa < 0.40) represents a considerable level of agreement.

4.1. Expert qualitative validation

The validation was carried out with the help of psychologist Ivette Jimenez García, who is a specialist in neurosciences with a master's degree in cognitive behavioral psychotherapy. She has the function of an emotion-specialized expert who validates the proposed database. This validation consists of observing each sample and confirming that the actor’s intended emotion corresponds to the emotion that is supposed to be interpreted. Consequently, a total of 65 samples were rejected, while 1801 samples were accepted, as can be seen in the last row and columns 3-4 of Table 3.

4.2. Survey qualitative validation

Another qualitative validation was carried out through a survey applied to 30 participants (raters) recruited using a non-probabilistic sampling strategy known as voluntary sampling. The survey was disseminated among the academic community of the Universidad Tecnológica de la Mixteca, as well as among acquaintances, relatives, friends, colleagues, and researchers from other institutions who were older than 18 years, and participation was entirely voluntary. Therefore, the final number of 30 raters was not determined through a formal statistical sample-size calculation but corresponds to the number of participants who completed the evaluation within the established data-collection period. The raters had an average age of 37. The survey responses (which include the raters’ demographic information) and the corresponding actor-intended emotion labels are available as supplementary CSV files in the UTeMo GitHub repository.

With respect to the categorical judgment survey results, a total of 5430 responses (181 samples evaluated by 30 raters) were collected, yielding 4216 matches between the actor-intended emotion labels and the rater responses. This represents a 77.6% agreement rate. Notice in the distribution of responses by emotion in Fig. 4, that only a small fraction of the total samples was differently perceived. However, it can also be seen that there are certain emotions that raters consider to be similar (for example, there may be confusion between happiness/surprise or sadness/neutral). Due to the inherent uncertainty of human perception, it was to be expected that some of the raters' judgments would have been ambiguous with each other.

Fig. 4.

Fig 4 dummy alt text

Distribution of rater’s categorical judgment.

To understand these results more clearly, Fig. 5 shows the percentages of agreement by emotion in a heptagonal graph. In general, all the emotions are approximately balanced, which implies that each group of samples is representative for each emotion, since there are no emotions that could cause great confusion in the raters' judgments.

Fig. 5.

Fig 5 dummy alt text

Categorical rater’s judgment percentages per emotion.

With respect to the intensity judgment, the distribution of intensity for all samples is shown in Fig. 6. The figure indicates that 58.2% of the samples were classified by the raters as moderately to very intense, while the ratio between the samples corresponding to slightly and moderately intense is almost 1.0. This trend is favorable, as it implies that despite the uncertainty of the raters' judgments, the emotions that the actors intended to interpret mostly correspond to the targeted emotions.

Fig. 6.

Fig 6 dummy alt text

Intensity rater’s judgment overall distribution.

If we break down Fig. 6 by emotions, Fig. 7 is derived. This visualization makes it possible to observe the rater's intensity judgments for each specific emotion. As illustrated, emotions such as sadness and surprise are mostly perceived as slightly to moderately intense. And emotions such as happiness, anger, fear, and disgust are predominantly perceived as moderately intense to intense, while the neutral state is perceived as not intense to slightly intense. Overall, five out of the seven emotional states are mostly perceived as moderately intense or intense.

Fig. 7.

Fig 7 dummy alt text

Intensity rater’s judgment per emotion.

Based on the results above, the Fleiss’ Kappa statistic has been calculated for the categorical judgments and interpreted using the Landis & Koch [11] criteria, which are presented in Table 4. As can be seen in this table, a substantial level of agreement has been achieved for most emotions. This demonstrates a consistent database with appropriately represented emotions and strong alignment among raters. Additionally, the interpretation of the emotions’ categorization according to the Landis & Koch criteria shows that disgust, happiness, anger, fear, and sadness are considered substantial, while emotions such as surprise and neutral are considered moderate. In general, the UTeMo database can be categorized as substantial according to these criteria. It is important to note that these values depend on the subjective judgments made by the raters; therefore, these metrics may vary depending on the population to which the survey is applied.

Table 4.

Fleiss’ Kappa statistic (K) per emotion and globally.

Emotion K Landis & Koch criteria [11]
Poor
<0
Slight 0.0-0.2 Fair 0.21-0.40 Moderate 0.41-0.60 Substantial 0.61-0.80 Almost Perfect 0.81-1.00
Disgust 0.74 X
Happiness 0.63 X
Anger 0.70 X
Fear 0.70 X
Neutral 0.50 X
Surprise 0.58 X
Sadness 0.70 X
Global 0.62 X

In order to test the UteMo dataset with deep learning classifiers, preprocessing must be performed to separate the audio and image information. In the case of emotion classification using audio data, preprocessing is achieved by first extracting the audio channel (WAV format) from each video sample using the Moviepy library. Then, environmental noise removal is performed using the Noisereduce library, which applies a spectral gating method to attenuate undesired background noise in the recordings. Additionally, silence removal is applied to emphasize audio information and reduce the file size of each sample. After that, each audio sample is amplified by 10 dB to increase the volume and strengthen the audio features. Both silence removal and volume amplification are carried out using the Pydub library. Afterward, the Librosa library is used to transform the amplified audio sample into a Mel spectrogram (a 2D image containing the audio spectral representation). After preprocessing, we obtained 1801 spectrograms, which can be provided as inputs to convolutional neural networks such as FSER [12]. Finally, to increase the generalization of the spatial relationship between frequencies and intensities inherent in each emotion and to reduce overfitting, spectrogram augmentation is applied to the training set using the Keras library, following the recommendations in [12]. The augmentation process consisted exclusively of small random rotations (±5°) and scaling transformations (±0.1), which preserve the structural properties of the time–frequency spectrogram representation while introducing controlled variability. After the spectrogram augmentation process is completed, the training set is expanded 20 times, resulting in 20 augmented spectrograms per sample.

We evaluated the UTeMo dataset using the audio spectral representation (Mel-spectrograms) for training the specialized FSER convolutional network. For all experimentation, we set the random number generator seed to 42 to ensure consistency across data splitting, weight initialization, and data augmentation. The preprocessed spectrogram dataset was randomly partitioned using stratification into 60% for training (to which spectrogram augmentation was applied), 20% for validation, and 20% for test set. This partitioning was performed before any hyperparameter exploration was carried out. Then, a coarse grid search to find the hyperparameters of FSER was implemented using the training and validation sets. The hyperparameters were searched as in [12]: evaluating epoch numbers in [200, 300, 400], batch size in [8,16,32], optimizer in [Adam, SGD], and synaptic weight initializer in [HeNormal, GlorotNormal]. Here, the found suitable hyperparameters were epoch=200, batch size=8, optimizer=SGD, and weight initializer=HeNormal. Subsequently, a fine grid search around epoch in [150, 200, 250] was performed, keeping the other hyperparameters fixed to those obtained in the coarse search. The final selected value was epoch=250. Fig. 8 shows the training and validation accuracy/loss curves for the FSER model using the found hyperparameters. To avoid overfitting, the FSER architecture incorporates dropout layers for regularization, alongside the previously described data augmentation method. Afterward, the training and validation sets were merged and used to train the final FSER model with the found hyperparameters. Finally, the FSER model was evaluated using the independent test set, where an accuracy of 95.65% and an F1-score of 95.72% were obtained.

Fig. 8.

Fig 8 dummy alt text

Training and validation accuracy/loss results for the FSER model using the found hyperparameters through grid search.

Since our previous experiments used stratified random partitioning, which means that actors were present across all splits and the data was not subject-independent, we reran the experiments using leave-speakers-out cross-validation to validate the FSER results for the emotion recognition task. Due to time constraints, we utilized the found hyperparameters identified in the previous experiments for FSER training. Following the leave-speakers-out cross-validation experiment, the average accuracy obtained for FSER was 87.44%. This result is favorable for emotion recognition, given a scenario where the network was provided solely with audio spectral information from 14 actors.

As a byproduct of this data paper, we share in the UTeMo GitHub public repository2 all preprocessing (for audio and video), training and evaluation (for FSER) scripts used in the above-mentioned experiments. Furthermore, we added supplementary material to this repository, such as the Spanish phrases used, raw survey responses and the script for Fleiss’ Kappa computation. In this way, complete reproducibility of the experiments is ensured for other researchers (see the “UTeMo_pipelineExample” Jupyter notebook).

Compared to other audiovisual datasets for emotion recognition, UTeMo contains a larger number of samples and/or higher-quality data than well-known databases in the Spanish, English, German, and Italian languages, such as SEV [13], RekEmozio [14], SAVEE [6], Emo-DB [7] and EMOVO [15] (see Table 5). Both data quality and sample count are key components for modern deep learning methods. Furthermore, UTeMo can be combined with other high-quality datasets, such as RAVDESS for cross-cultural emotion analysis or language-independent emotion recognition, while considering the dataset’s limitations.

Limitations

The only emotions considered are sadness, surprise, happiness, anger, fear, and disgust, i.e. the six basic emotions based on Paul Ekman’s model, and a neutral state.

This study is also limited by the size and demographic diversity of the actor group involved in the dataset construction. In particular, the dataset was built using a limited number of actors, which means that it should not be interpreted as fully representative of the cultural, regional, and sociolinguistic diversity of the Mexican population. This limitation may influence the variability of the emotional expressions captured in the corpus and, consequently, the extent to which the dataset can be used for broader application scenarios, such as cross-cultural studies.

In addition, the subjective perceptual validation was conducted with a limited group of raters recruited through a voluntary, non-probabilistic strategy. The number of raters was restricted in part because the evaluation instrument consisted of a lengthy survey (rating 181 videos), which made it difficult to obtain a higher response rate. The diversity of the rater group was also influenced by the academic and social networks through which the survey was disseminated. Therefore, the validation results should be interpreted within this methodological and demographic context.

Another limitation was the number of experts who participated in the validation phase. This number was restricted mainly because of the difficulty in finding available experts with sufficient time to rate all the video samples. This limitation affects the inter-coder reliability, and as a consequence, some interpretations of the video samples might not reflect the actor’s intended emotion.

Finally, all audiovisual samples were acquired exclusively using equipment at the Digital Media Lab at Universidad Tecnológica de la Mixteca, following the settings described in Section 4. No additional sources are included in the dataset.

Ethics Statement

This research involved human subjects, and informed consent was obtained from all participants. All the actors’ information was handled anonymously. Additionally, it was ensured that each image, audio or video contained no embedded labels or identifying information to maintain the highest level of confidentiality, and the authors confirmed adherence to all requirements and guidelines for scientific publication.

CRediT authorship contribution statement

Jorge-Arturo Carrasco-Jiménez: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Visualization, Project administration. Arturo Téllez-Velázquez: Methodology, Software, Validation, Formal analysis, Investigation, Resources. Rosebet Miranda-Luna: Methodology, Validation, Formal analysis, Investigation, Resources, Writing – original draft. Antonio Orantes-Molina: Methodology, Validation, Formal analysis, Investigation, Resources, Writing – original draft. Juan-Pablo García-Vázquez: Writing – original draft, Writing – review & editing, Validation. Raúl Cruz-Barbosa: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing, Supervision, Validation.

Acknowledgments

We thank the Universidad Tecnológica de la Mixteca for their support, CONAHCYT for providing the scholarship of JA Carrasco, psychologist Ivette Jiménez García, and volunteer actors and raters for their important contribution to this study.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.dib.2026.113175.

Appendix. Supplementary materials

mmc1.zip (290.8KB, zip)

References

  • 1.Guo R., Guo H., Wang L., Chen M., Yang D., Li B. Development and application of emotion recognition technology—a systematic literature review. BMC Psychol. 2024;12:95. doi: 10.1186/s40359-024-01581-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Anagnostopoulos C.N., Iliou T., Giannoukos I. Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011. Artif. Intell. Rev. 2015;43:155–177. doi: 10.1007/s10462-012-9368-5. [DOI] [Google Scholar]
  • 3.Fernández-Vítoresz D. Cervantes Virtual Center, Cervantes Institute; 2022. El Español: Una Lengua Viva. Informe 2022.https://cvc.cervantes.es/lengua/espanol_lengua_viva/pdf/espanol_lengua_viva_2022.pdf [Google Scholar]
  • 4.Morán N., Pérez J., Rodriguez W. Reconocimiento de estados emocionales de personas mediante la voz utilizando algoritmos de aprendizaje de máquina. Rev. Venez. Comput. 2018;5(2):41–52. http://caelum.ucv.ve/ojs/index.php/rev_vcomp/article/view/15858 [Google Scholar]
  • 5.Livingstone S.R., Russo F.A. The Ryerson audio-visual database of emotional speech and song (RAVDESS) PLoS One. 2018;13(5):1–35. doi: 10.1371/journal.pone.0196391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Haq S., Jackson P. Speaker-dependent audio-visual emotion recognition. Proceedings of the International Conference on Audio-Visual Speech Processing (AVSP); Norwich, UK; 2009. pp. 53–58. [Google Scholar]
  • 7.F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, B. Weiss. A database of german emotional speech. In: Interspeech, pp. 1517–1520, Lisbon, Portugal (2005). 10.21437/Interspeech.2005-446. [DOI]
  • 8.Ryerson Multimedia Research Lab., 2026 Ryerson emotion database. 2004. https://www.kaggle.com/datasets/ryersonmultimedialab/ryerson-emotion-database (accessed 29 May 2026).
  • 9.Wu C.H., Lin J.C., Wei W.L. Survey on audiovisual emotion recognition: databases, features, and data fusion strategies. APSIPA Trans. Signal Inf. Process. 2014;3(1):1–18. doi: 10.1017/ATSIP.2014.11. [DOI] [Google Scholar]
  • 10.Fleiss J.L., Levin B., Paik M.C. 1st ed. John Wiley & Sons; London: 2003. Statistical Methods for Rates and Proportions. [DOI] [Google Scholar]
  • 11.Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
  • 12.Dossou B.F., Gbenou Y.K.S. FSER: deep convolutional neural networks for speech emotion recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision; Montreal, Canada; 2021. pp. 3533–3538. [Google Scholar]
  • 13.Barra-Chicote R., Montero J., Macias-Guarasa J., Lebai Lutfi S., Lucas-Cuesta J., Fernández-Martínez F., D’Haro L., Hernández R., Ferreiros J., Cordoba R., Muñoz J. Proceedings of the 6th Conference of Language Resources & Evaluation. 2008. Spanish expressive voices: corpus for emotion research in Spanish; pp. 60–70. [Google Scholar]
  • 14.López J.M., Cearreta I., Garay N., López de Ipiña K., Beristain A. Proceedings of the 7th Spanish Human Computer Interaction Conference. Vol. 6. 2006. Creación de una base de datos emocional bilingüe y multimodal; pp. 55–66. [Google Scholar]
  • 15.Costantini G., Iaderola I., Paoloni A., Todisco M. Proceedings of the International Conference on Language Resources and Evaluation. 2014. Emovo corpus: an Italian emotional speech database; pp. 3501–3504. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.zip (290.8KB, zip)

Articles from Data in Brief are provided here courtesy of Elsevier

RESOURCES