A simultaneous EEG and eye-tracking dataset for remote sensing object detection

Abstract

We introduce the EEGET-RSOD, a simultaneous electroencephalography (EEG) and eye-tracking dataset for remote sensing object detection. This dataset contains EEG and eye-tracking data when 38 remote sensing experts located specific objects in 1,000 remote sensing images within a limited time frame. This task reflects the typical cognitive processes associated with human visual search and object identification in remote sensing imagery. To our knowledge, EEGET-RSOD is the first publicly available dataset to offer synchronized eye-tracking and EEG data for remote sensing images. This dataset will not only advance the study of human visual cognition in real-world environment, but also bridge the gap between human cognition and artificial intelligence, enhancing the interpretability and reliability of AI models in geospatial applications.

Background & Summary

Remote sensing images are important media for people to perceive and understand the real world and make decisions¹. Compared to general natural images, remote sensing images provide unique insights into human visual cognitive mechanisms with the overhead perspective and broad perspective². Remote sensing object detection, a fundamental method for large-scale monitoring of geographic environments and surface changes, is used to search for and identify specific objects or phenomena in remote sensing images^3,4. This technique is widely used in areas such as urban planning⁵, precision agriculture⁶, and disaster monitoring⁷. Recent progress in deep learning has significantly improved the ability of detection models to automatically learn and extract features from images, leading to higher detection efficiency and accuracy⁸. However, remote sensing object detection is still a data-driven statistical learning process that requires a large amount of labeled data for training⁹. Typically, training an object detection algorithm in a specific remote sensing task (e.g., detect the land cover type) requires experts to visually interpret and manually label hundreds of images and thousands of objects, which is undoubtedly a resource-intensive task. Therefore, our goal is to effectively integrate human cognitive processes in artificial intelligence (AI) approaches of remote sensing object detection. In other words, we aim to first identify and extract the main cognitive processes of the human brain (reflecting by human EEG and eye-tracking data) when detecting objects in remote sensing images, and then decode these brain and eye signals that could replace or supplement human annotations.

Previous research has shown that human eye movements and brain activities can help improve and interpret AI models for remote sensing tasks^10,11. Eye-tracking data reveal explicit patterns of visual attention, whereas EEG signals elucidate implicit cognitive processes. For example, eye-tracking data can be used to label the locations of candidate objects, which can then be identified via convolutional neural networks (CNNs)¹². EEG data are widely used in the training of BCI and object detection models to improve the efficiency of remote sensing interpretation^13,14. In addition, eye-tracking and EEG data have been used to explain and enhance deep learning models in areas such as feature extraction, attention module, and classification^15,16. However, to date, there has been no integrated eye-tracking and EEG data to investigate human visual attention for remote sensing object detection tasks. This is mainly because designing synchronized eye-tracking and EEG experiments comes with higher costs and more complex procedures, including extended setup and calibration, managing system interference, aligning data with differing temporal resolutions, and handling increased noise and artifacts during preprocessing. However, it is precisely these challenges that make such data extremely valuable. Synchronizing eye-tracking with EEG allows researchers to combine observable visual attention patterns with internal neural responses. This makes it possible to identify not only where people focus their attention in remote sensing images, but also the brain activity that occurs at the same time. This integration bridges the gap between behavioral and neural correlates of attention. It also facilitates mutual validation through complementary analytical methods, offering a more comprehensive understanding of human cognitive processes during complex tasks. Furthermore, such datasets can provide both interpretability benchmarks and valuable insights for deep learning.

In this paper, we present our EEGET-RSOD dataset, which includes synchronized EEG and eye-tracking data from 38 remote sensing interpretation experts. It contains a total of 1,000 remote sensing images with corresponding object label data. Each visual stimulus is presented within 3000 ms, aligning with the typical human process of searching for and identifying objects in remote sensing images^13,17. Experts must locate objects in these stimuli within the given time. In previous studies, rapid serial visual presentation (RSVP)^18,19 was often used to investigate the cognitive mechanisms of human visual search and recognition. Several eye-tracking and brain activity datasets (such as COCO-Search18²⁰ and BioTest EEG²¹) have also been used. In these datasets, stimuli are typically presented for 200 to 500 ms, which ensures alignment between neural activity and visual behavior but eliminates important aspects of normal human object search processes. Therefore, it is crucial to emphasize the value of simultaneously recording EEG and eye-tracking data in this study. Eye-tracking data allow us to understand which objects are being focused on during different periods of human object detection, enabling us to extract the corresponding EEG signals for each object. These signals have the potential to reflect complex visual perception, search, and target detection processes, providing valuable insights into human cognitive patterns in real world. Therefore, the EEGET-RSOD dataset has a wide range of applications and important value in the field of human visual research. It will attract the joint interest of researchers in geography, cognitive science, and artificial intelligence. Furthermore, it will promote interdisciplinary research such as geospatial brain-like intelligence⁹ and neurocognitive geography²².

Methods

Participants

To recruit qualified participants (i.e. the expert for remote sensing object detection) for the experimental tasks, potential participants need pass a remote sensing image interpretation test in addition to completing the remote sensing courses. The test consisted of 10 questions, where potential participants were required to search for airplanes in images within 5 seconds and count the number of occurrences. Participants who achieved an accuracy rate of over 90% are regarded as experts for remote sensing object detection and are eligible to participate in the formal experiment. Based on this, a total of 40 participants were involved, including 17 males and 23 females, with an average age of 22.5 years (SD = 2.03). We explained the principles of eye tracking and EEG, along with potential risks to participants, and collected their signed consent forms of the formal experiment. All participants’ vision was normal or corrected to normal, and none had a history of brain-related conditions. This study was approved by the Ethics Committee of Beijing Normal University (No: 20221024111). All participants voluntarily took part in the experiment and consented to the collection and use of eye-tracking and EEG data, including the publication and public disclosure of data that do not contain direct personal identification information.

Stimuli

The stimuli for this experiment were selected from the DIOR dataset, a large dataset for remote sensing object detection that primarily includes satellite and aerial images²³. This dataset covers various scenes, object categories, and complex backgrounds, making it well-suited for remote sensing object detection in complex environments. We selected 900 remote sensing images containing airplanes, with each image containing between 1 and 6 airplanes. Additionally, 100 images without airplanes were chosen to reduce participants’ expectation effects regarding the objects^24,25 (Fig. 1A). All the images were 800 × 800 pixels in size. The 1,000 images were evenly divided into 10 blocks, each containing 90 images with airplanes and 10 images without airplanes, with a random order for each block.

Fig. 1.

Experimental setup. (A) Examples of stimuli; (B) Eye tracking and simultaneous EEG acquisition system; (C) Experimental procedure; (D) The position of the EEG electrodes following the 10–20 system.

Experimental procedure

In the experiment, the participants first completed a 13-point eye-tracking calibration. Before the formal experiment began, the participants underwent a keypress training session in which three test images were used to familiarize themselves with the procedure. Then they viewed the first block, which contained 100 remote sensing images. Each image was displayed for 3,000 ms, during which the participants were instructed to search for and fixate on all airplanes in the image. A grayscale image, displayed for 500 ms, was shown between each pair of remote sensing images. After completing a block, participants can rest for a few minutes and start the next block when they are ready. Each participant was required to complete five blocks. While a 13-point eye-tracking calibration was used before the formal experiment began, a 5-point calibration was employed for subsequent blocks before starting. The participants were instructed to remain as still as possible during the task (Fig. 1C).

Data acquisition

The experiment was conducted in a quiet laboratory. Stimulus presentation was controlled via SMI’s Experiment Center. The participants used a laptop with a screen resolution of 1920 × 1080 and a refresh rate of 60 Hz and sat 60 cm away from the screen (Fig. 1B).

Eye-tracking data were collected using an SMI RED250 eye tracker, with a sampling rate of 250 Hz, an accuracy of 0.4°, and a spatial resolution of 0.03°. Eye-tracking calibration was performed at the start of the experiment. During the validation step, calibration was repeated until the error between any two measurements at a single point was less than 0.5° or until the average error across all points was less than 1°.

EEG data were collected using an NE Enobio 32 system with a sampling rate of 500 Hz. Thirty-two conductive silver chloride gel electrodes were applied according to the international 10–20 system (Fig. 1D), with a reference electrode placed at the right mastoid. The electrodes were positioned as follows: frontal (Fp1, Fp2, Fz, F3, F4, F7, and F8), central (Cz, C3, and C4), temporal (T7 and T8), parietal (P7 and P8), and occipital (Oz, O1, and O2). Additional electrodes were placed in the frontal (AF3 and AF4), central frontal (FC1, FC2, FC5, and FC6), central parietal (CP1, CP2, CP5, and CP6), and occipital‒parietal (PO3 and PO4) regions. Prior to recording, the impedance of each electrode was checked to ensure good contact, with the impedance maintained below 5 kΩ. Electrode impedance levels were checked before each block.

Data temporal alignment

Eye movement and EEG data were recorded using the same computer, and were synchronized via keyboard inputs. Specifically, Key events were assigned at the beginning and end of each block. At the start of the experiment, the participants were instructed to press the numeric key “1” to initiate the block. Upon completing a block, participants were required to press the numeric key “3” to finish this block. When the participant pressed the designated key, the eye-tracking and EEG acquisition software simultaneously recorded the key press and its corresponding timestamp. Each participant provided 10 synchronized keystroke records.

The conversion formula between eye movement time ${\mathrm{T}}_{\mathrm{ET}}$ and EEG time ${\mathrm{T}}_{\mathrm{EEG}}$ is as follows:

$${\mathrm{T}}_{\mathrm{EEG}}\times 2=({(\mathrm{T}}_{\mathrm{ET}}-\mathrm{b})/1000)$$ 1

Eye-tracking data is recorded using timestamps, with a precision of one-thousandth of a millisecond (0.001 ms). To convert this into milliseconds, the must be divided by 1000. In contrast, EEG data is based on sample point counts. As our data were acquired with a sampling rate of 500 Hz, the time interval between each sample point was 2 ms. Therefore, the must be multiplied by 2 to convert the EEG data’s sample point count (latency) into milliseconds. By using the timestamps of the synchronized keystrokes within the eye movement and EEG data, we can calculate the temporal discrepancy between the two systems. Specifically, we converted the timestamps of the eye-tracking and EEG data into time values measured in milliseconds. This allowed us to obtain the precise time records of the same button-press event in both the eye-tracking and EEG data. Subsequently, by subtracting these two time records, we derived the temporal discrepancy between the two systems, denoted as $\mathrm{b}$.We also provided a dataset containing the $\mathrm{b}$ values for each participant.

Figure 2 displays one example of the occurrence of blinks in the eye movement data, as represented in the EEG, along with the corresponding topographic maps. It reflects the good synchronization accuracy of our eye-tracking and EEG data acquisition.

Fig. 2.

Visualization of single-trial EEG and eye-tracking blink data.

Data Records

The dataset provided in this paper is available at Figshare²⁶. The structure of the dataset is illustrated in Fig. 3. This dataset contains six types of data: raw EEG data, preprocessed EEG data, eye-tracking data, data alignment files, saliency maps, stimulus material data, and object annotation data.

Fig. 3.

The structure of the dataset.

EEG recording

EEG data were collected using an NE Enobio 32 system with a sampling rate of 500 Hz. The reference electrode was placed on the right mastoid, and all channels were positioned according to the extended 10–20 system. A conductive gel was applied to reduce the impedance at each electrode. The file names are organized by participant ID and saved in EDF + format. All raw EEG data are stored in the “EEG.zip” file.

Additionally, we provide preprocessed EEG files (“EEG_clean data.zip”). Preprocessing was performed via EEGLAB. First, channel locations were set for the raw EEG data. A high-pass filter of 0.1 Hz, a low-pass filter of 80 Hz, and a notch filter of 50 Hz were applied. Next, rereferencing was performed using an average reference across the whole brain. Finally, independent component analysis (ICA) was used, and eye-movement artifacts were manually removed. The preprocessed EEG data are also organized by participant ID and saved in EDF format. Please note that preprocessed EEG data is only for users who wish to use it directly. It is advised to manually eliminate artifacts to achieve more effective results.

Eye-tracking recording

Eye-tracking data were collected using an SMI RED 250 binocular eye tracker, with a sampling rate of 250 Hz. File names are organized by participant ID and saved in TXT format. Each ET data frame is stored on a separate line and includes the recording time, event type (i.e., Message and Sample), gaze coordinates for both eyes, pupil size, gaze point type (i.e., Fixation, Saccade, and Blink), and corresponding stimulus material. An individual gaze plot is shown in Fig. 1.

For fixation and saccade detection, we used a method based on time and spatial thresholds to identify fixations, which is suitable for a sampling rate of 250 Hz. A fixation is defined as the gaze remaining within 0.5° of visual angle for at least 50 ms. Saccade detection is based on velocity, time and amplitude thresholds. The minimum saccade duration was set to 5 ms, the minimum saccade amplitude was set to 0.5°, and the peak velocity threshold was set to 50°/s. When the eye movement velocity, time and amplitude exceed the corresponding thresholds, the period is marked as a saccade.

Time synchronization data

This file is used for aligning the timestamps of the eye-tracking data with the EEG data. It contains the parameter b, which is required for the time conversion formula (Eq. (1)). Each participant’s data corresponds to a unique b value.

Gaze heatmap

The heatmaps generated from eye-tracking points are considered ground truths for training and testing visual saliency detection models. These heatmaps show the regions that attract the most attention from human vision. We created gaze heatmaps on the basis of kernel density estimates of all participants’ eye-tracking points. Each gaze heatmap is named the same as its corresponding stimulus material (Fig. 1).

Stimulus and object label data

The dataset includes the 1,000 selected remote sensing images, along with the object label files from the DIOR dataset (Fig. 1).

Technical Validation

Eye-tracking validation

We employed four methods to assess the quality of the eye-tracking ET data: the valid sampling rate, the object detection rate of the participants, the stability of the fixation duration, and the consistency of the fixation point distribution. Each method evaluates different aspects of the reliability of the eye-tracking data.

The valid sampling rate measures how well the eye tracker tracked the eyes during the experiment, indicating the quality of the data collection^27,28. We used Begaze’s algorithm to calculate invalid data points caused by factors such as blinking, tracking loss, or device calibration issues. The remaining data were considered valid. The valid sampling rate is the ratio of valid data points to the total number of data points. As shown in Table 1 Details of all participants in the dataset, the sampling rate exceeded 90% for 34 participants, the sampling rate was between 85% and 90% for 5 participants, and the sampling rates of the remaining 2 participants (D01 and D02) were below this threshold and were therefore excluded from the dataset.

Table 1.

Details of all participants in the dataset.

Participant No.	Sex	Age (years)	Educational level	Object detection rate	Effective sampling rate
P01	Female	21	Undergraduate student	86.11%	95.01%
P02	Female	21	Undergraduate student	81.26%	96.77%
P03	Female	21	Undergraduate student	85.13%	93.93%
P04	Male	26	Master’s student	77.78%	87.21%
P05	Female	24	Master’s student	87.87%	80.19%
P06	Male	25	Master’s student	96.59%	90.00%
P07	Male	26	Master’s student	92.03%	90.72%
P08	Male	25	Master’s student	72.04%	86.74%
P09	Female	20	Undergraduate student	93.22%	90.99%
P10	Male	20	Undergraduate student	95.43%	97.26%
P11	Male	20	Undergraduate student	94.50%	87.16%
P12	Male	21	Undergraduate student	90.21%	84.15%
P13	Male	27	Master’s student	95.83%	82.36%
P14	Male	25	Master’s student	93.86%	88.83%
P15	Female	19	Undergraduate student	97.50%	83.89%
P16	Female	18	Undergraduate student	97.11%	88.21%
P17	Female	22	Master’s student	94.19%	91.19%
P18	Female	24	Master’s student	88.15%	89.14%
P19	Female	21	Master’s student	72.85%	89.27%
P20	Female	22	Master’s student	90.91%	85.64%
P21	Female	20	Undergraduate student	94.08%	95.94%
P22	Female	20	Undergraduate student	78.35%	38.58%
P23	Female	21	Undergraduate student	88.67%	94.94%
P24	Male	21	Undergraduate student	91.78%	94.53%
P25	Male	25	Master’s student	85.34%	85.56%
P26	Female	24	Master’s student	98.83%	90.80%
P27	Female	21	Undergraduate student	95.29%	96.48%
P28	Male	24	Master’s student	89.94%	95.28%
P29	Male	24	Master’s student	84.25%	97.46%
P30	Male	24	Master’s student	96.55%	93.88%
P31	Female	24	Master’s student	92.67%	80.84%
P32	Male	25	Master’s student	84.49%	89.57%
P33	Male	25	Master’s student	91.78%	94.53%
P34	Female	18	Undergraduate student	97.31%	86.47%
P35	Male	23	Master’s student	92.67%	80.84%
P36	Female	22	Master’s student	90.26%	88.37%
P37	Female	23	Master’s student	92.30%	94.06%
P38	Female	24	Master’s student	97.11%	88.21%
D01	Female	23	Master’s student	-	75.12%
D02	Male	25	Master’s student	-	66.72%

The object detection rate measures how well participants completed the task, specifically, whether they identified all the objects in the remote sensing images. We used label data from the DIOR dataset to define the object areas. Considering the limitations of human visual fields, we defined a circular region centered on the fixation point with a radius of 50 pixels as the effective visual field. If a participant’s effective visual field overlapped with the object area, the object was considered detected. (see in Fig. 4) The detection rate was calculated as the ratio of detected objects to the total number of objects. The results are shown in Table 1. On average, the detection rate of participants was 90.11%, with the highest rate of 98.83% and the lowest rate of 72.04%. A total of 24 participants achieved a detection rate above 90%, indicating that most participants successfully completed the task as designed. Ten participants had detection rates between 80% and 90%, whereas 4 participants had rates below 80%, demonstrating that the remote sensing objects used in this experiment were challenging. A ceiling effect was not observed in the study²⁹. Furthermore, we conducted an analysis of the correlation between the effective sampling rate and the detection rate using Pearson’s correlation coefficient. The results indicated no significant correlation between the two (r = 0.232, p > 0.05). This suggests that the loss of some eye-tracking data points did not impact the accuracy of evaluating the participants’ task completion.

Fig. 4.

Objective detection rate measurement method. These images are used as illustrations. The red dots and lines are the scanpaths of the participant’s eye movement, and the blue areas are the objects that need to be searched.

The eye movements of the participants in each image reflected their visual attention during the experimental process^30,31. We used an I-DT(Identification Dispersion-Threshold) method based on time and spatial thresholds to identify fixations. The minimum fixation duration was set to 50 ms, with a maximum dispersion of 0.5°. Saccade detection was based on a peak velocity threshold of 50°/s, with a minimum saccade duration of 5 ms and a minimum saccade amplitude of 0.5°. Figure 5 shows the average fixation duration, average saccade amplitude, and average pupil size for each participant on remote sensing images with and without objects. Compared with viewing images containing objects, when reviewing images without objects, participants displayed longer fixation durations and greater saccade amplitude. This finding indicates that participants demonstrate distinct patterns in their eye movements during the remote sensing object detection task. Longer fixation duration and greater saccade amplitude typically suggest that participants engage in more complex visual processing and extensive visual searching^32,33. However, there were no significant differences in pupil size across the different images. This finding indicates that the levels of focused attention and cognitive load remained consistent for participants when viewing both object-present and object-absent remote sensing images^34,35.

Fig. 5.

Eye movement indices of the participants on object and non-object images.

Additionally, we conducted a statistical analysis of the saccade direction in remote sensing images with and without objects. The saccade direction is defined as the vector from the previous gaze point to the subsequent gaze point³⁶. We defined the positive x-axis as 0 degrees and divided the 360-degree range into 24 equal angular segments in a counterclockwise manner. We subsequently calculated the number of fixations by participants across different directions for each image. Finally, we assessed the number of fixations in both object-present and object-absent images, along with the coefficient of variation and skewness for the fixation counts across different directions. The results are shown in Fig. 6.

Fig. 6.

Saccade direction difference and visualization.

The results indicate that participants exhibited a greater number of fixations in object-absent images than in object-present images, alongside a lower directional bias in gaze. We visualized the gaze trajectories and the fixation counts for each direction. In object-present remote sensing images, participants demonstrated a more consistent gaze trajectory with a clear directional focus. Conversely, in object-absent images, participants exhibited a more structured search pattern, akin to the gaze behavior observed during text reading.

The fixation location reflects the spatial distribution of participants’ visual attention when viewing the same image^37,38. Heatmaps are commonly used to visualize and analyze the consistency of participants’ fixation distributions. We randomly selected the eye-tracking data of N participants, generated heatmaps on the basis of their fixation points, and used two similarity metrics—the Pearson linear correlation coefficient (CC) and the Kullback‒Leibler (KL) divergence³⁹—to measure the difference between these heatmap and the heatmaps of all participants’ fixation points. CC represents an intuitive measure of the linear relationship between two Heatmaps. And KL measures the difference in distribution between two Heatmaps. A CC value close to 1 or -1 indicates a strong linear correlation, whereas a CC value of 0 indicates no linear correlation. The results show that as the number of participants decreased, the CC value gradually decreased, and the variance increased (Fig. 7A). When the number of participants was fewer than 11, the CC was below 0.9. The KL divergence is often used to measure the difference between the heatmap distribution and the true distribution, with lower KL values indicating greater similarity. The results demonstrate that as the number of participants decreased, the KL values gradually increased, and the variance also increased (Fig. 7B). When the number of participants was below 9, the KL value exceeded 0.1. The results above indicate that as the number of participants increased, the variation in the saliency map decreased and gradually stabilized (Fig. 7C). This demonstrates that the current sample size is sufficient to capture consistent patterns in eye-tracking data. Specifically, the stabilization of the saliency map indicates that the eye-tracking data already reflect stable visual behavior patterns, which supports the reliability and robustness of the dataset for its intended applications.

Fig. 7.

Consistency of fixation point distributions.

EEG validation

We used event-related potentials (ERPs) as a key metric to evaluate the EEG data in this experiment. EEG preprocessing was manually performed using EEGLAB⁴⁰ and FieldTrip⁴¹. First, a bandpass filter of 0.1–80 Hz was applied to the EEG signals using EEGLAB to remove nonneural artifacts. Additionally, a notch filter of 48–52 Hz was employed to eliminate line noise interference. Next, we performed rereferencing using the average of all the electrodes. We subsequently applied blind source separation through ICA⁴² to remove artifacts such as blinks, eye movements, and electromyographic (EMG) noise. EEG data were segmented into epochs based on stimulus onset, with each epoch time-locked to events ranging from -500 ms to 1000 ms. Baseline correction was applied using the prestimulus period from -300 to 0 ms. On the basis of previous research, we selected the Pz, Cz, and Oz channels, as they are associated with visual neural responses.

The ERP is a specific type of evoked potential caused by particular psychological or cognitive activities, reflecting changes in cortical potentials. P300 is an ERP component closely associated with attention and memory processes⁴³ and is typically observed in regions such as the frontal and parietal lobes⁴⁴. In this experiment, we observed a prominent P300 component in the Pz channel (Fig. 8A), indicating that the stimuli elicited attention-related processes in the participants⁴⁵. Another common ERP component, N1, reflects the brain’s early processing of visual stimuli⁴⁶. A distinct N1 component was observed in the Cz channel (Fig. 8B), which may be related to stimulus classification and memory updating⁴⁷. In the Oz channel, we observed components resembling P1 and N2 (Fig. 8C)⁴⁸. The P1 component likely reflects the primary visual cortex’s response to early attention tasks⁴⁹. The N2 component is associated with stimulus recognition, classification, and semantic evaluation^50,51.

Fig. 8.

ERPs of different channels.

Existing research has demonstrated that the power spectral density (PSD) characteristics of EEG can be utilized to assess participants’ cognitive load⁵² and attention level⁵³. Alpha band (8–12 Hz) is associated with the resting state of individuals, and lower Alpha power indicates heightened attention and information processing⁵⁴. Theta band (3–7 Hz) is linked to memory encoding and working memory tasks, with increased theta activity often indicating higher cognitive load⁵⁵. In contrast, beta band (13–20 Hz) is related to active attention, information processing, and task execution, with increased beta activity reflecting higher levels of attentiveness⁵⁶. Consequently, we selected the Alpha/Theta ratio and Alpha/Beta ratio of the frontal channels (Fz, F3, and F4) as indicators to assess cognitive load and attention levels. These metrics were used to compare participants’ attention states during tasks involving two types of remote sensing images.

Figure 9 reveals that most participants exhibited no significant differences in attention states between object-present and target-absent remote sensing images. This indicates that participants maintained similar cognitive load and attention levels across both image types, suggesting that they exerted consistent effort in the target detection task, regardless of the presence of targets within the remote sensing images.

Fig. 9.

The mean alpha/theta and alpha/beta band power ratio of all the participants on object and non-object images.

In summary, the replication of ERP results demonstrates that our EEG data reliably capture stable neural activity patterns. This not only confirms the adequacy of both the participant sample size and the number of stimuli, but also proves the high quality of the data. In addition, the PSD validation further supports the stability of participants’ cognitive states throughout the experiment, reinforcing the data’s reliability.

Furthermore, classifying behaviors on the basis of brain signals is a common and significant task in the field of EEG⁵⁷, serving as a robust indicator of data validity. This classification involves interpreting EEG signals to identify patterns associated with visual behaviors. Such tasks have applications in BCIs and assistive technologies, where individuals can identify objects in remote sensing images solely through intent⁵⁸.

As shown in Fig. 10, we utilized labeled files to define the extent of objects within each image. Next, we categorized gaze points on the basis of their locations, distinguishing between fixations within the object region and those outside it. We then converted the fixation start times into EEG data timestamps via Eq. (1). Finally, we added both fixation types—within-object and outside-object—into the EEG dataset according to these timestamps.

Fig. 10.

EEG events based on fixation classification.

We utilized event markers to create epochs. We selected five different time windows (See Table 2 for details) for analysis. All the epochs were divided into training and testing sets at a 70:30 ratio, and classification was performed using EEGNet⁵⁹. For classification, these hyperparameters were trained: the size of the batch was 32, the stochastic gradient descent (SGD) optimizer was employed, the learning rate was 5e-4, and the maximum number of iterations was 50. The results of the classification are shown in Table 2.

Table 2.

Classification results for EEGNet at different epochs.

Time window size	Start and end time	Mean Accuracy	Maximum Accuracy
500 ms	−500~0 ms	57.85%	84.54%
	0~500 ms	59.63%	84.62%
	500~1000 ms	57.38%	82.00%
	1000~1500 ms	57.65%	84.62%
1000 ms	−500~500 ms	59.72%	80.13%
	0~1000 ms	57.60%	81.41%
	500~1500 ms	58.15%	85.90%
1500 ms	−500~1000 ms	57.94%	84.62%
	0~1500 ms	58.15%	85.90%
2000 ms	−500~1500 ms	59.04%	82.69%

In summary, the EEGET-RSOD dataset offers a valuable resource for enhancing machine learning models and deepening our understanding of human cognitive processes in the context of remote sensing object detection. However, it is essential to acknowledge certain limitations inherent in the current study. Firstly, the dataset is composed of experienced remote sensing experts, whose cognitive patterns may not fully represent those of the general population. While their shared demographic and professional backgrounds may further limit the generalizability of the findings, it offers valuable insights into expert-specific cognitive processes and strategies, which help establish a baseline for understanding domain-related expertise. Secondly, the dataset focuses specifically on aircraft detection in remote sensing imagery, which may constrain the applicability of the results to other object types or tasks involving greater complexity. Finally, the laboratory-controlled observation conditions, coupled with fixed image presentation durations, might not fully capture real-world observation behaviors. To address these limitations, future studies could incorporate a wider variety of stimuli, recruit participants with more diverse demographic and professional backgrounds, and design tasks that better simulate real-world remote sensing applications.

Author contributions

Bing He: Study design and conceptualization; Data collection; Data curation; Data validation; Manuscript writing; Manuscript correction. Hongqiang Zhang: Data curation; Data validation. Tong Qin: Study design; Data validation; Manuscript correction. Bowen Shi: Study design; Data collection; Data curation. Qiao Wang: Study design and conceptualization; Manuscript correction; Supervision. Weihua Dong: Study design and conceptualization; Data validation; Manuscript correction; Supervision. All authors reviewed and approved the final version of the manuscript.

Code availability

For the classification task, code is available at: https://github.com/Bing-1997/EEGET_RSOD/tree/main/eegnet.

For Technical Validation, code is available at: https://github.com/Bing-1997/EEGET_RSOD.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41597-025-04995-w.