EVIORA-HAR: A multimodal dataset for elderly activity recognition and safety-controlled fall detection using wearable inertial sensors and multi-view cameras

Abstract

EVIORA-HAR is a multimodal dataset for human activity recognition and fall detection that combines wearable inertial sensing with multi-view RGB video. The data were collected from 104 participants comprising 71 older adults aged 60–93 years performing activities of daily living and 33 healthy adults aged 18–24 years performing safety-controlled fall scenarios. Daily activities were recorded in residential homes and elderly care environments, whereas fall scenarios were acquired in a controlled indoor environment for participant safety. Five wearable inertial measurement unit nodes were attached to the chest, waist, right hand, right thigh, and left ankle. Visual data were captured using three camera configurations: a full-view static camera, a partially occluded static camera, and a dynamic pan-tilt-zoom camera that followed participant movement. Video was recorded at 1280 × 720 resolution and 60 frames per second, while inertial data were recorded at 60 Hz. The dataset contains untrimmed coarse-grained recordings and trimmed fine-grained activity segments representing walking, postural states, postural transitions, and forward, sideward, and backward falls. The trimmed data comprise 11 fine-grained activity classes. Using FV Cam as the reference distribution, the dataset contains 340 walk segments (68.72 min), 323 stand-to-sit (18.18 min), 314 sit-to-stand (16.60 min), 330 sit-to-lay (21.58 min), 321 lay-to-sit (20.31 min), 347 stand (48.17 min), 823 lay (118.59 min), 657 sit (91.58 min), 170 fall-forward (9.45 min), 150 fall-sideward (8.23 min), and 160 fall-backward segments (9.03 min). The released data include privacy-protected RGB videos, inertial signal files, pose-estimation outputs, activity labels encoded in the directory structure and file names, and consistent identifiers supporting modality correspondence. These data can be reused for wearable-sensor, video-based, skeleton-based, multi-view, occlusion-aware, dynamic-camera, and multimodal human activity recognition and fall-detection research.

Specifications Table

Subject	Computer Sciences
Specific subject area	Multimodal human activity recognition, wearable sensing, computer vision, and fall detection
Type of data	Table (.csv), RGB video (.mp4); raw and processed data
Data collection	Data were collected from 104 participants: 71 older adults performing activities of daily living and 33 healthy adults performing safety-controlled falls. Five MPU-6050-based wearable nodes recorded three-axis acceleration and angular velocity at approximately 60 Hz. Three RGB camera configurations recorded 1280 × 720 video at 60 FPS: full-view static camera, partially occluded static camera, and dynamic pan-tilt-zoom camera. Static views were recorded concurrently using OBS Studio, while IMU and PTZ data were acquired through a common host interface.
Data source location	Older-adult activities of daily living: residential homes and elderly-care environment in Surabaya, Madiun, and Mojokerto, Indonesia. Safety-controlled falls: Laboratory PS0808, Politeknik Elektronika Negeri Surabaya (PENS), Surabaya, Indonesia.
Data accessibility	Repository name: Zenodo Data identification number: 10.21227/2zvc-nb24 Direct URL to data: https://zenodo.org/records/22239523 Instructions for accessing these data: The DOI 10.21227/2zvc-nb24 was originally assigned to EVIORA-HAR through IEEE DataPort and is retained as the persistent dataset identifier. The same DOI is also displayed on the Zenodo record, which provides the openly accessible repository copy of the dataset. All files can be downloaded directly from the repository record. Large dataset components are provided as independently extractable ZIP archives (Part01, Part02, etc.) for easier storage and transfer.
Related research article	None

1. Value of the Data

• •The EVIORA-HAR dataset includes daily activities recorded from 71 older adults aged 60–93 years in residential and elderly-care environments. Fall recordings were collected separately from 33 healthy adults aged 18–24 years in a controlled laboratory setting. This design provides activity data from the target older population while allowing fall-related movements to be recorded without asking older participants to perform intentional falls.
• •The EVIORA-HAR dataset provides 60-Hz inertial data from five wearable IMU nodes together with 60-FPS RGB video from three camera views: full-view static camera (FV Cam), partially occluded-view static camera (OV Cam), and dynamic view (pan-tilt-zoom/PTZ Cam). These high-temporal-resolution data support wearable-based, vision-based, multi-view, occlusion-aware, dynamic-camera, and multimodal human activity recognition.
• •Two forms of the dataset are available. The untrimmed recordings contain eight coarse-grained activity protocols, while the trimmed data separate the recordings into 11 finer activity classes, including postures, transitions, walking, and directional falls. This allows researchers to work with complete activity episodes or shorter activity segments, depending on the recognition or temporal-segmentation task being investigated.
• •In addition to RGB videos and inertial signals, the repository contains pose/keypoint data, with activity labels encoded in the folder and file names. Participant faces in the released videos have been obscured to reduce identifiability. Researchers may use the original sensor representations or work directly with the provided pose data, which avoids repeating pose extraction when that processing step is not part of the research question.
• •Corresponding recordings are identified through consistent subject, activity, trial, and segment identifiers within the IMU–PTZ Cam and FV Cam–OV Cam acquisition groups. Correspondence is maintained at the annotated-segment level where matching data are available. These identifiers enable paired samples to be constructed for multimodal fusion and multi-view experiments without assuming frame-level synchronization across all four modalities.

2. Background

Human activity recognition (HAR) datasets use wearable, vision-based, and multimodal sensing for activity recognition and fall detection [1]. Elderly monitoring commonly covers ADLs and fall detection, although intentional fall-data collection involves practical and safety constraints [2,3]. Existing datasets vary in participant population, environment, modalities, resolution, and camera configuration [4], as summarized in Table 1.

Table 1.

Comparison of HAR datasets.

Dataset	Participants	Elderly	Fall	Environment	Modalities
[1]	17	–	✓	Lab	2 RGB(FV) 18 FPS + 5 IMU 100 Hz + EEG 512Hz
[5]	54	✓	–	Real world	1 IMU 50Hz
[6]	38	✓	✓	Real world, Lab	1 IMU 200Hz
[7]	100	✓	–	Real world	1 RGB-D(FV) 30 FPS
[8]	20	–	✓	Indoor	4 RGB(FV, OV) 30 FPS + IMU 50Hz
EVIORA-HAR [9]	104	✓	✓	Real world, Lab	3 RGB(FV, OV, PTZ) 60 FPS + 5 IMU 60Hz

EVIORA-HAR combines ADL recordings from older adults in residential and elderly-care environments with safety-controlled falls from healthy adults. The dataset integrates five 60-Hz IMUs with full-view, occluded-view, and dynamic PTZ RGB video at 60 FPS, together with untrimmed recordings, fine-grained segments, and pose representations. The following sections describe data organization, acquisition, temporal correspondence, annotation, quality control, usability assessment, and limitations.

3. Data Description

3.1. Repository structure and data organization

The EVIORA-HAR dataset is organized into three main directories: `Untrimmed`, `Classed`, and `Trimmed`, as illustrated in Fig. 1.

Fig. 1.

Directory structure of the EVIORA-HAR dataset.

The `Untrimmed` directory contains acquisition-level recordings with coarse-grained activity labels. The `Classed` directory reorganizes these recordings according to activity category, while the `Trimmed` directory contains shorter segments with fine-grained activity labels. The data within these directories are separated by sensing modality. Wearable motion data are stored as IMU files, while the visual data comprise recordings from the full-view static camera (FV Cam), partially occluded-view static camera (OV Cam), and dynamic view (pan-tilt-zoom/PTZ Cam). In the original directory structure, the aliases CAM 1, CAM 2, and PTZ correspond to FV Cam, OV Cam, and PTZ Cam, respectively. Pose representations derived from the video recordings using MediaPipe Pose and YOLO Pose are also included. Video recordings are stored in MP4 format, whereas IMU and pose data are provided as CSV files. This organization keeps the original activity recordings, class-based arrangement, and fine-grained segments distinct while preserving the corresponding modality-specific data. Representative examples of the multimodal recordings are shown in Fig. 2, including the FV Cam, OV Cam, and PTZ Cam views together with the corresponding IMU signals.

Fig. 2.

Representative multimodal EVIORA-HAR recordings from FV Cam, OV Cam, PTZ Cam, and IMU.

3.2. Untrimmed and class-organized recordings

The `Untrimmed` directory contains acquisition-level recordings organized by subject ID and sensing modality. Eight coarse-grained activities are included: walk (A01), stand–sit (A02), sit–stand (A03), sit–lay (A04), lay–sit (A05), fall forward (A06), fall sideward (A07), and fall backward (A08). Each recording contains the initial posture, the movement or transition, and the final posture of the activity. Table 2 summarizes the number and duration of the untrimmed recordings. FV Cam and OV Cam each contain 2109 recordings, while PTZ Cam and IMU contain 1951 and 1954 recordings, respectively. Total recording durations range from 401.73 to 433.34 min across modalities, with an average duration of approximately 12.3 s per file.

Table 2.

Distribution of untrimmed EVIORA-HAR recordings.

ID	Activity	FV Cam	OV Cam	PTZ Cam	IMU
A01	Walk	339	339	314	316
A02	Stand–Sit	323	323	311	313
A03	Sit–Stand	314	314	311	310
A04	Sit–Lay	331	331	313	313
A05	Lay–Sit	320	320	307	307
A06	Fall Forward	170	170	132	132
A07	Fall Sideward	150	150	131	131
A08	Fall Backward	162	162	132	132
Total recordings		2109	2109	1951	1954
Total duration (min)		433.34	430.97	401.73	402.66
Avg. duration/file (s)		12.33	12.26	12.35	12.36

The `Classed` directory contains the same eight coarse-grained activity categories arranged by class rather than by the original recording organization. It does not represent a separate acquisition stage; instead, it offers a class-based arrangement of the untrimmed data for users who prefer to access recordings directly by activity label.

3.3. Trimmed activity segments

The `Trimmed` directory contains shorter activity segments derived from the untrimmed recordings. A complete activity episode may therefore be represented by several fine-grained labels. For example, a stand-sit recording is represented by stand pose, stand-to-sit transition, and sit pose segments. Similarly, sit-stand recordings contain sit, sit-to-stand, and stand segments; sit-lay recordings contain sit, sit-to-lay, and lay segments; and lay–sit recordings contain lay, lay-to-sit, and sit segments. Walk recordings contain walk and stand segments, while some fall recordings also include a lay segment.

The trimmed data comprise 11 activity classes: walk, stand-to-sit, sit-to-stand, sit-to-lay, lay-to-sit, stand, lay, sit, fall forward, fall sideward, and fall backward. Their distribution across sensing modalities is summarized in Table 3. FV Cam and OV Cam each contain 3935 segments, compared with 3723 PTZ Cam segments and 3705 IMU segments. Total durations range from 388.12 to 430.43 min, with an average segment duration of approximately 6.3–6.6 s.

Table 3.

Fine-grained activity segments in the `Trimmed` directory.

Activity	FV Cam*	OV Cam*	PTZ Cam*	IMU*
Walk	340 / 68.72	340 / 68.23	405 / 56.38	405 / 56.42
Stand-to-sit	323 / 18.18	323 / 18.18	311 / 15.80	311 / 15.80
Sit-to-stand	314 / 16.60	314 / 16.60	312 / 15.82	311 / 15.77
Sit-to-lay	330 / 21.58	330 / 21.58	316 / 16.12	312 / 15.92
Lay-to-sit	321 / 20.31	321 / 20.31	312 / 15.83	308 / 15.64
Stand	347 / 48.17	347 / 47.68	332 / 48.42	331 / 48.25
Lay	823 / 118.59	823 / 117.96	716 / 107.12	712 / 106.66
Sit	657 / 91.58	657 / 90.61	624 / 94.18	620 / 93.67
Fall forward	170 / 9.45	170 / 9.45	132 / 6.67	132 / 6.67
Fall sideward	150 / 8.23	150 / 8.23	131 / 6.62	131 / 6.62
Fall backward	160 / 9.03	160 / 9.03	132 / 6.67	132 / 6.68
Total segment	3935	3935	3723	3705
Total duration (min)	430.43	427.85	389.62	388.12
Avg. duration/file (s)	6.56	6.52	6.28	6.29

^⁎Values are number of segments / total duration (min).

Compared with the untrimmed recordings, the trimmed directory separates stable postures and movement transitions into individual labels. The resulting files retain the same sensing modalities as the corresponding recordings, allowing the coarse-grained and fine-grained forms of the dataset to be accessed separately.

3.4. Data formats, file naming, and variables

Dataset files retain identifiers for the subject, activity, trial, and sensing modality. For example, S01_A01_T01_IMU.csv denotes the IMU data for subject S01, activity A01, and trial T01. Files in the trimmed directory include the segment number and fine-grained activity label, as in S01_A01_T01_IMU_seg01_walk.csv and S01_A01_T01_PTZ_seg01_walk.csv. These identifiers are also used in the corresponding pose files. Activity labels are encoded in the directory structure and file names and are also listed in the released machine-readable metadata files. The metadata table records the subject ID, cohort, age group, activity, trial, segment, modality availability, duration, relative file path, and exclusion status.

IMU data are stored in CSV format. Files in the Untrimmed and Classed directory contain 34 columns: Timestamp1, Timestamp2, Frame ID, Data Type, and 30 inertial channels from five wearable nodes. Each node contributes three-axis acceleration (Ax, Ay, Az) and three-axis angular velocity (Gx, Gy, Gz), indexed from 1 to 5 according to the IMU node. Trimmed IMU files use the same structure with an additional activity_label column.

Two pose representations are provided. MediaPipe Pose files contain 100 columns: a frame field followed by x, y, and z coordinates for 33 landmarks (x0, y0, z0 to x32, y32, z32). Empty coordinate values may occur when a landmark is unavailable. YOLO Pose files contain a frame_idx field and 17 body keypoints, each represented by x and y coordinates and a confidence value, giving 52 columns in total. Unlike the indexed MediaPipe fields, the YOLO columns use anatomical names such as nose_x, left_shoulder_y, and right_ankle_conf. The data formats and principal variables for each representation are summarized in Table 4.

Table 4.

Data formats and variables provided in EVIORA-HAR.

Data representation	Format	Principal variables
RGB video	MP4	Image sequences from FV Cam, OV Cam, and PTZ Cam
IMU	CSV	Two timestamps, frame ID, data type, and 30 inertial channels; trimmed files additionally contain activity_label
MediaPipe Pose	CSV	frame + 33 landmarks (x, y, z)
YOLO Pose	CSV	frame_idx + 17 keypoints (x, y, confidence)

In the provided pose files, MediaPipe uses one-based frame numbering, whereas YOLO Pose uses zero-based frame_idx. The frame field should therefore be used when relating pose entries to their source video rather than assuming identical row numbering between the two pose representations.

3.5. Modality correspondence and data availability

Not every recording is available in all sensing modalities. The lower availability of IMU and PTZ Cam data mainly reflects acquisition interruptions and modality-specific quality control. Some IMU–PTZ Cam recording sessions were not retained during acquisition because of IMU node connectivity problems. Further modality-specific exclusions were made during quality control. Corresponding files are identified at the recording level for the untrimmed data and at the segment level for the trimmed data. The resulting modality pairs and complete four-modality sets are summarized in Table 5.

Table 5.

Corresponding recordings and segments across EVIORA-HAR modalities.

Data level	IMU–PTZ Cam pairs	FV Cam–OV Cam pairs	Complete four-modality sets
Untrimmed recordings	1950	2109	1887
Trimmed segments	3705	3935	3489

For the untrimmed data, 1950 recordings have valid counterparts in both IMU and PTZ Cam, while 2109 recordings are paired between FV Cam and OV Cam. Among them, 1887 recordings are represented in all four modalities. At the trimmed level, 3705 IMU–PTZ Cam pairs and 3935 FV Cam–OV Cam pairs are available, including 3489 complete four-modality sets. The four-modality sets indicate correspondence at the recording or annotated-segment level and do not imply frame-level synchronization across all four modalities.

4. Experimental Design, Materials and Methods

4.1. Participants and activity protocol

EVIORA-HAR was collected from 104 participants divided into two cohorts according to the recorded activities. The ADL cohort comprised 71 older adults aged 60–93 years, while the fall cohort comprised 33 healthy adults aged 18–24 years. The demographic characteristics of both cohorts are summarized in Table 6.

Table 6.

Demographic characteristics of the participants.

Characteristic	Elderly (ADL), n = 71	Adult (Fall), n = 33
Age (years), mean ± SD	69.10 ± 7.25	21.03 ± 1.31
Age range (years)	60–93	18–24
Height (cm), mean ± SD	159.63 ± 6.55	168.55 ± 6.12
Height range (cm)	145–175	157–180
Weight (kg), mean ± SD	55.41 ± 8.00	64.06 ± 13.94
Weight range (kg)	40–86	45–105
Gender, Male	23 (32.4%)	33 (100%)
Gender, Female	48 (67.6%)	0 (0%)
Dominant hand, Right	71 (100%)	33 (100%)
Mobility aid (cane), Yes	1 (1.4%)	N/A
Mobility aid (cane), No	70 (98.6%)	N/A

The two cohorts followed different activity protocols. Older adults performed five ADLs comprising walking and transitions between standing, sitting, and lying postures, whereas healthy adults performed intentional falls in the forward, sideward, and backward directions, with the fall direction retained as part of the activity annotation. Participants followed the predefined protocols in Table 7 while being allowed to perform the movements according to their natural habits. Each activity was repeated up to twelve times depending on participant willingness and physical capability. Participants were shown an example, received verbal instructions, were cued using a 1–2–3 count, and performed the instructed fall from standing as naturally as possible. Trials of the same fall direction were repeated with 3–5 min rest intervals before proceeding to another direction. No practice trial was performed before recording.

Table 7.

Activity definitions and recording protocols.

ID	Activity	Group	Protocol
A01	Walk	Elderly	The participant starts from one end of the walking path inside the room and walks straight for approximately 2.5 m. Upon reaching the end of the path, the participant naturally turns around and walks back along the same path.
A02	Stand–Sit	Elderly	The participant starts in a standing position facing the camera, either in front of a chair or a bed. The participant then sits down naturally on the chair or bed and remains seated in a stable posture until the recording is completed.
A03	Sit–Stand	Elderly	The participant starts in a seated position on a chair or bed facing the camera. The participant then stands up naturally until reaching a fully upright posture and remains standing in a stable position until the recording is completed.
A04	Sit–Lay	Elderly	The participant starts in a stable seated position on the bed facing the camera. The participant then lies down naturally until reaching a fully lying posture and remains still until the recording is completed.
A05	Lay–Sit	Elderly	The participant starts in a stable lying position on the bed. The participant then sits up naturally until reaching an upright seated posture and remains seated in a stable position until the recording is completed.
A06	Fall Forward	Adult	The participant starts in an upright standing position on a mattress and intentionally falls forward until the body contacts the mattress. After the fall, the participant remains lying still until the recording is completed.
A07	Fall Sideward	Adult	The participant starts in an upright standing position on a mattress and intentionally falls to the right side until the body contacts the mattress. After the fall, the participant remains lying still until the recording is completed.
A08	Fall Backward	Adult	The participant starts in an upright standing position on a mattress and intentionally falls backward until the body contacts the mattress. After the fall, the participant remains lying still until the recording is completed.

4.2. Data acquisition and sensing setup

Data acquisition was conducted in indoor environments according to the activity group. ADLs were recorded in residential homes and elderly-care environments, whereas fall scenarios were recorded in a controlled laboratory environment. The acquisition areas were approximately 4–7 m in both length and width. Chairs and beds were used during the ADL recordings. Intentional falls were performed onto a 90 × 190 cm travel bed with an 8 cm padded, quilted surface. The overall acquisition setup is illustrated in Fig. 3.

Fig. 3.

Overall EVIORA-HAR data acquisition setup.

The sensing setup combined five wearable IMU nodes with three RGB camera configurations. The IMU nodes were attached to the chest, waist, right hand, right thigh, and left ankle. The cameras were positioned at a height of 180 cm and consisted of a FV Cam, an OV Cam, and a PTZ Cam. The sensor and camera specifications are summarized in Table 8.

Table 8.

Sensor specifications used for EVIORA-HAR data acquisition.

Modality	Specification	Purpose
IMU	Accelerometer ±4 g, Gyroscope ±250 °/s, DLPF 21 Hz, 100 Hz internal sampling rate, 60 Hz output rate	Wearable motion sensing
FV Cam	RGB; 1280 × 720; 60 FPS	Full-view visual observation
OV Cam	RGB; 1280 × 720; 60 FPS	Occluded-view visual observation
PTZ Cam	RGB; pan-tilt-zoom; 1280 × 720; 60 FPS	Dynamic visual tracking

Each wearable IMU node was built as an independent wireless sensing device. The node used an MPU-6050 connected through an I²C interface to an ESP8266-based Wemos D1 Mini. It was powered by a 3.7 V, 200 mAh Li-Po battery with an operating time of approximately 30 min. The components were enclosed in a 3D-printed PLA casing measuring 37 × 32 × 15 mm and weighing approximately 50 g, with a fastening strap for attachment to the participant. The wearable node is shown in Fig. 4.

Fig. 4.

Wearable IMU node used in EVIORA-HAR.

Each IMU sampling cycle recorded three-axis acceleration $\left(a_x,a_y,a_z\right)$ and three-axis angular velocity $\left({\omega }_x,{\omega }_y,{\omega }_z\right)$. For IMU node $q$ at sample $t$, these quantities are represented as

$${\mathbf{a}}_{q,t}=\left[a_{x,q,t},a_{y,q,t},a_{z,q,t}\right],{\mathit{\omega }}_{q,t}=\left[{\omega }_{x,q,t},{\omega }_{y,q,t},{\omega }_{z,q,t}\right]$$ (1)

The vectors ${\mathit{a}}_{q,t}$ and ${\mathit{\omega }}_{q,t}$ represent three-axis acceleration and angular velocity, respectively. The index $q\in \left\{1,\dots ,5\right\}$ identifies the IMU node, while $t$ indicates the sample index. The five nodes form a 30-dimensional IMU feature vector

$${\mathbf{u}}_t=\left[{\mathbf{a}}_{1,t},{\mathit{\omega }}_{1,t},\dots ,{\mathbf{a}}_{5,t},{\mathit{\omega }}_{5,t}\right]\in {\mathbb{R}}^{30}$$ (2)

Measurements from the five nodes were transmitted in real time to the receiving computer through the same Wi-Fi network using the UDP protocol in comma-separated format. At the receiving computer, measurements from the five IMU nodes were combined into common output rows, providing a shared PC-side time reference for the wearable signals. The resulting IMU output rate was 60 Hz, while all three RGB camera modalities recorded video at 60 FPS.

4.3. Multi-View recording and temporal correspondence

Data recording was organized into two acquisition groups. The first group comprised the five IMU nodes and PTZ Cam, which were captured during the same recording session through a common host acquisition interface. The second group comprised FV Cam and OV Cam, which were recorded concurrently using an OBS Studio-based system.

Temporal alignment was established within each acquisition group. The IMU samples and PTZ Cam video shared the same session time reference and were aligned using acquisition timestamps, while FV Cam and OV Cam were recorded concurrently within the same OBS Studio session. Hardware triggering or frame locking was not implemented between the two acquisition groups, and the systems were started and stopped manually. Inter-system latency, temporal offset, and drift were not measured during data collection. As a result, their numerical bounds cannot be reported.

Cross-group correspondence was established using timestamps at the recording and annotated-segment levels. Consequently, correspondence is provided across the four modalities without assuming frame-level synchronization between the IMU–PTZ Cam and FV Cam–OV Cam acquisition groups.

4.4. Annotation, trimming, and pose extraction

Fine-grained activity segments were generated from the untrimmed recordings using a semi-automated Python-based GUI designed as a video-trimming tool. The PTZ–IMU and FV–OV acquisition groups were processed separately by two annotators, one for each group. For the PTZ–IMU group, the PTZ recordings were trimmed and labeled first. The resulting temporal metadata were then applied to the corresponding IMU recordings. For the FV–OV group, the FV recordings were trimmed and labeled first. The resulting temporal metadata were then applied to the corresponding OV recordings.

Segment boundaries were assigned according to changes in movement phase. For transitional activities, a segment started when the participant began to leave the initial posture and ended when the target posture was reached. For example, a stand-to-sit transition started when the participant began to lower the body from standing and ended when the seated posture was reached. The same rule was applied to sit-to-stand, sit-to-lay, and lay-to-sit transitions. When a boundary was visually ambiguous, the recording was inspected frame by frame and the frame showing the clearest phase transition was selected. Only segments with a minimum duration of 2 s were retained. Formal inter-annotator agreement was not calculated because no independent duplicate annotation was performed.

Pose data were generated from the RGB recordings using MediaPipe Pose v0.10.5 and YOLO11m-pose implemented with Ultralytics v8.3.197. For MediaPipe Pose, video mode was used (static_image_mode=False) with model_complexity=1. Each video frame was converted from BGR to RGB before pose estimation. The 33 landmark coordinates (x, y, z) were stored when landmark visibility exceeded 0.5. Landmarks below this threshold were recorded as NA. For YOLO Pose, YOLO11m-pose was applied to every video frame using a detection confidence threshold of 0.3. The frames were passed directly to the Ultralytics inference pipeline without manual cropping or resizing. The output consisted of 17 body keypoints. Their x and y coordinates were normalized using the original frame dimensions, and the corresponding confidence values were also stored. A frame was retained when at least five keypoints had positive x, y, and confidence values. Frames with no detected pose or fewer than five valid keypoints were not included in the YOLO Pose CSV output. The resulting pose estimates were stored as CSV files associated with their source video recordings. The released dataset therefore contains both the RGB recordings and the corresponding derived pose representations for FV Cam, OV Cam, and PTZ Cam.

4.5. Data quality control and privacy processing

Data quality control was applied separately to the sensing modalities during dataset preparation. For the untrimmed IMU data, recordings containing >500 NA entries within one activity take were excluded. This quality-control screening was performed before trimming, and only the retained IMU recordings were subsequently segmented using the temporal boundaries derived from the corresponding PTZ Cam recordings. For the video modalities, recordings from FV Cam, OV Cam, and PTZ Cam were excluded when the video file could not be opened and therefore could not be used for further processing. Quality control was performed shortly after data recording, and recordings that did not meet the above criteria were directly excluded from the retained dataset. A separate file-level exclusion log was therefore not created during the original data collection process.

For the wireless IMU data, the transmitted packets during the original EVIORA-HAR acquisition did not include sequence numbers, and individual UDP packet-arrival times were not stored. Network-level packet loss and packet-arrival jitter therefore cannot be reconstructed retrospectively. Data integrity was assessed from all 1954 untrimmed IMU recordings. In these files, NA indicates that no updated measurement from the corresponding sensor node was available after the predefined two-output-cycle tolerance. Overall, 98.156% of the output rows contained non-NA values from all five sensor nodes. Rows in which all six channels of an individual node were recorded as NA accounted for 0.823% of the sensor-node rows. This NA-based measure reflects stored-data availability and should not be interpreted as a packet-loss rate.

A separate post-acquisition test was conducted to characterize the UDP-based wearable communication. The same five-node hardware and UDP-over-Wi-Fi architecture were tested during sitting, slow walking, and fast walking. For this follow-up test, sequence numbers were added to the transmitted packets specifically to enable packet-loss measurement. Packet loss was calculated from missing sequence numbers. Packet-arrival jitter was calculated as the sample standard deviation of receiver-side inter-arrival intervals. The reported mean jitter for each activity condition was obtained by averaging the per-node, per-recording jitter values. Packet-loss rates were 0.010%, 0.172%, and 0.012% for sitting, slow walking, and fast walking, respectively. The corresponding mean jitter values were 9.39, 8.75, and 7.89 ms. Packet-arrival gaps of at least 100 ms occurred 83, 75, and 13 times, with maximum gaps of 300, 375, and 223 ms. These results characterize the communication system under the follow-up test conditions rather than the original dataset acquisition.

Participant privacy was addressed before the dataset was released. Subject records use coded identifiers without personal information, and participant faces in the released RGB videos were obscured. Faces were detected using OpenCV YuNet and anonymized using pixelation followed by Gaussian blurring. The procedure was applied automatically to each video frame. Representative samples of the processed videos were visually inspected, but the entire dataset was not manually checked frame by frame. The original identifiable RGB recordings are not included in the public dataset. Audio tracks were also removed from the publicly released videos. Despite these measures, residual re-identification risk may remain from non-facial characteristics such as body shape, gait, clothing, or the recording environment.

4.6. Dataset usability assessment

The usability of EVIORA-HAR for activity recognition was assessed using CNN and LSTM as representative models. Both untrimmed and trimmed data were evaluated under four unimodal settings (IMU, PTZ Cam, FV Cam, and OV Cam) and two multimodal settings (IMU + PTZ Cam and FV Cam + OV Cam). Visual inputs used the MediaPipe Pose data provided with the dataset. Input sequences were divided into windows of $W=60$ samples with a stride of $S=30$samples using a sliding-window segmentation approach [10]. For a feature sequence $\left\{{\mathit{x}}_t\right\}$, the $k$-th input window was defined as

$${\mathbf{X}}_k=\left[{\mathbf{x}}_{kS},{\mathbf{x}}_{kS+1},\dots ,{\mathbf{x}}_{kS+W-1}\right]$$ (3)

Five-fold StratifiedGroupKFold was applied, with grouping by recording for untrimmed data and by annotated segment for trimmed data. Standardization was fitted only to the training data of each fold. For feature $j$, the standardized value was calculated as

$$z_{n,t,j}=\frac{x_{n,t,j}-{\mathrm{\mu }}_j^{\text{train}}}{{\sigma }_j^{\text{train}}}$$ (4)

where ${\mathrm{\mu }}_j^{\text{train}}$and ${\sigma }_j^{\text{train}}$ are the mean and standard deviation of feature $j$, computed from the training data only. Partial missing values were handled by temporal interpolation and forward-backward filling, while fully invalid samples were excluded. The models used class-weighted cross-entropy and AdamW optimization with a learning rate of 0.001, batch size of 32, and a maximum of 100 epochs. A random seed of 42 was used for all experiments. For class $c$, the F1-score was calculated from precision and recall following the formulation in [11]:

$$F{1}_c=\frac{2P_c{R}_c}{P_c+R_c}$$ (5)

where $P_c$ and $R_c$ are the precision and recall for class $c$. The macro-F1 score was then calculated as the arithmetic mean of the class-wise F1-scores [11]:

$$F{1}_{\text{macro}}=\frac{1}{C}\sum _{c=1}^{C}F{1}_c$$ (6)

where $C$ is the number of activity classes. The resulting accuracy and macro-F1 values are summarized in Table 9. The assessment was intended as an illustrative usability check of whether the released data representations could be used directly in common HAR pipelines, rather than as an exhaustive benchmark of model performance. The same preprocessing and evaluation procedure was applied across modalities to keep the assessment consistent. These results demonstrate the practical usability of the released data representations for common HAR models.

Table 9.

EVIORA-HAR usability assessment using CNN and LSTM models.

Modality	Untrimmed data		Trimmed data
	CNN	LSTM	CNN	LSTM
IMU	91.01 ± 1.50 / 90.65 ± 1.43	88.19 ± 2.73 / 87.47 ± 2.87	94.91 ± 0.64 / 90.71 ± 1.29	94.94 ± 0.59 / 90.05 ± 1.04
PTZ Cam	92.74 ± 2.25 / 92.89 ± 2.30	93.30 ± 1.43 / 93.03 ± 1.58	91.14 ± 1.12 / 83.77 ± 1.61	89.16 ± 1.04 / 80.85 ± 1.56
FV Cam	90.94 ± 1.20 / 91.64 ± 1.28	92.91 ± 0.57 / 93.09 ± 0.63	87.75 ± 2.18 / 80.03 ± 3.79	88.51 ± 1.13 / 82.27 ± 1.78
OV Cam	81.06 ± 1.02 / 79.97 ± 0.64	81.35 ± 2.10 / 79.16 ± 2.45	79.66 ± 1.11 / 69.70 ± 2.07	80.18 ± 1.89 / 71.65 ± 2.56
IMU + PTZ Cam	96.98 ± 0.86 / 97.00 ± 0.94	95.64 ± 1.36 / 95.53 ± 1.32	98.81 ± 0.24 / 96.00 ± 0.55	98.54 ± 0.20 / 94.54 ± 1.42
FV Cam + OV Cam	90.59 ± 1.06 / 91.44 ± 1.11	92.52 ± 0.89 / 92.79 ± 0.87	86.03 ± 0.86 / 77.90 ± 2.34	86.81 ± 1.32 / 80.58 ± 2.04

Values are accuracy ± SD / macro-F1 ± SD (%).

Limitations

Several limitations of the dataset should be considered. First, fall data were recorded from healthy adults rather than older adults because of safety considerations. Falls performed by healthy young adults may differ from those of older adults in movement characteristics, including acceleration [3]. Second, although ADLs were recorded in real-world environments, data collection was limited to indoor settings, and the two participant cohorts were not demographically balanced. All participants in the fall cohort were male. In contrast, the elderly ADL cohort consisted of 23 men and 48 women. Differences were also present in age, height, and body weight, as reported in Table 6. Because the fall and ADL classes were collected from different cohorts, model training may capture cohort-related characteristics alongside activity-related patterns. This separation creates a domain gap between the two activity groups and limits the generalization of fall-detection results to older adults. For elderly-care applications, results obtained from the controlled young-adult fall cohort should not be interpreted as direct evidence of performance on real-world falls in older adults. The cushioned landing surface may affect landing and impact characteristics compared with harder surfaces. Possible learning or movement adaptation across repeated intentional-fall trials was not formally assessed. Intentional laboratory falls may also differ from accidental real-world falls because participants were aware of the instructed fall task and the controlled landing environment. Recording start and stop for the two acquisition groups were performed manually. The IMU–PTZ Cam and FV Cam–OV Cam acquisition groups were not frame-synchronized with each other. This limits analyses that require precise temporal alignment across the two groups, including four-modality frame-wise fusion and cross-group temporal modelling. Comparisons with benchmarks based on fully frame-synchronized multimodal inputs should account for this difference. Finally, modality availability varies because some recordings were affected by acquisition failures or excluded during modality-specific quality control. Future extensions could broaden participant demographics, include additional recording environments, and use a common hardware or software trigger to improve temporal alignment across acquisition groups. Where ethically and practically feasible, observational fall data from older adults could complement the present safety-controlled fall recordings.

Ethics Statement

All participants provided informed consent before data acquisition. The research was conducted in accordance with the Declaration of Helsinki and was approved by the Research Ethics Committee of Institut Teknologi Sepuluh Nopember (ITS), under approval No. 5366/IT2.XXII/T/TU.00.08/II/2026. Participant records were stored using coded identifiers without personal information.

CRediT Author Statement

Iwan Kurnianto Wibowo: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Visualization, Writing - Original Draft. Chastine Fatichah: Conceptualization, Methodology, Supervision, Project administration, Funding acquisition, Writing - Review & Editing. Nanik Suciati: Methodology, Validation, Supervision, Writing - Review & Editing.

Declaration of Generative AI and AI-Assisted Technologies in the Manuscript Preparation Process

During the preparation of this work, the authors used ChatGPT (OpenAI) for language editing and improving clarity. The authors reviewed and edited the output as needed and take full responsibility for the content of the published article.