IDEA: Image database for earthquake damage annotation

Abstract

The data article presents the “Image Database for Earthquake damage Annotation (IDEA)”, an extended dataset of annotated real structural damage consisting of more than 5400 images, collected during post-event and ordinary field inspections. The dataset aims to fill the lack of annotated data necessary for the development of deep learning methodologies with structural damage detection and/or classification purposes. The dataset contains images annotated by structural engineers, covering different structural typologies, construction materials and damage typologies. The dataset is based on a comprehensive ontology defined by the authors, based on commonly agreed structural damage categories, which includes several types of structural and non-structural damage. Such onthology, can be used either to expand the presented dataset or to produce new ones, in order to increase the availability of data annotated according to a common standard, from the structural engineering point of view. Furthermore, the IDEA dataset is valuable as benchmark for enhancing the performance of damage classification/detection algorithms, encompassing some of the limits of currently available datasets, which cover only a few structural typologies or damage classes, or consist of classified rather than annotated images, or originate from limited laboratory experiments rather than post-event reconnaissance.

Specifications Table

Subject	Engineering & Materials science
Specific subject area	Structural damage detection and classification on images
Type of data	2D-RGB Images (.jpg); Annotation files (.xml)
Data collection	Images were collected using traditional RGB digital cameras and RGB cameras mounted on UAVs. XML files (in Pascal VOC Format) were created using an annotation tool developed for internal use by the Eucentre Foundation.
Data source location	Institution: Eucentre Foundation, Pavia, Italy Country: Italy
Data accessibility	Repository name: Zenodo Data identification number: 10.5281/zenodo.15120522 Direct URL to data: https://zenodo.org/records/15120522 Instructions for accessing these data: Data are open access and can be downloaded from the “Files” section included in the dataset’s Zenodo web page.
Related research article	Dondi, P., Gullotti, A., Inchingolo, M., Senaldi, I., Casarotti, C., Lombardi, L., & Piastra, M. (2025). Post-earthquake structural damage detection with tunable semi-synthetic image generation. Engineering Applications of Artificial Intelligence, 147, 110302.

1. Value of The Data

• •The IDEA dataset is constituted by approx. 5400 high-quality RGB images of real structures, obtained during field structural inspections from high resolution cameras and UAVs.
• •The dataset is characterized by multi-class structural damage annotation on several construction typologies and various structural element types.
• •Currently the dataset contains images describing structural damage due to earthquakes and ageing/weathering of constructions, collected in Italy. Nevertheless, the dataset could be representative of southern European countries where structural typologies are similar.
• •The broad and comprehensive IDEA’s ontology could potentially be used for the creation of other structural damage datasets annotated according to a pre-defined standard, at least from a structural engineering point of view.
• •IDEA dataset is conceived as a living lab, which could be progressively updated in the future with new images acquisitions.
• •The dataset is valuable for the implementation and development of DCNN-based damage detection systems on structures and infrastructure, as well as for testing already existing architectures thus improving their performance.
• •Some examples of contexts of application: from structure/infrastructure monitoring for maintenance purposes to post-event emergency damage recognition and situational awareness enhancement of first responders (e.g., structural risks reduction during operations).

2. Background

Buildings and infrastructure are vulnerable to damage from excessive loads, extreme events, and environmental aging-issues, potentially worsened by poor maintenance [1]. Rapid post-event inspections ensure safety, while regular surveys help prevent deterioration. However, traditional visual inspections are time-consuming, technically and logistically resource-demanding therefore expensive, and potentially unsafe. Unmanned Aircraft Systems (UAS) have improved survey capabilities by providing high-resolution imagery in hard-to-reach or hazardous areas ([[2], [3], [4]]), enabling faster and safer inspections. Yet, handling large data volumes from UAS can be challenging, especially for structures with repetitive elements. Machine learning (ML) and deep learning (DL) algorithms are increasingly integrated with UAS technology for structural assessment purposes, improving damage identification, expediting inspections, and supporting structural experts in both disaster recovery and routine maintenance. Effective training of such models requires large volumes of annotated images; however, most open-source datasets - despite having many images - cover only a few structural typologies or damage classes, or consist of classified images rather than annotated ones, or originate from limited laboratory experiments or post-event reconnaissance [5]. The presented IDEA dataset aims at expanding availability of data annotated according to a common standard for structural damage detection.

3. Data Description

The “Image Database for Earthquake damage Annotation” (IDEA) dataset contains approximately 5’400 images, collected during field inspections in Italy, and their relative annotation metadata files with approx. 10’000 structural damage instances in total.

The dataset is constituted by two subsets, depending on whether in each image there is presence (“damage” folder) or absence (“no_damage” folder) of any type of structural damage (see directory structure in Fig. 1). It is worth noting that the dataset contains images with no damage instances present on structures and infrastructure in order to facilitate false positive detection by models developed for structural damage detection. Each subset contains two subdirectories, namely “images” and “annotations” that are consisting of image files (.jpg) and relative annotation file (.xml), respectively.

Fig. 1.

IDEA directory structure.

The subdirectories “annotations” contain the XML files in PASCAL VOC [6] corresponding to each image. Each XML file includes the following fields:

• •folder and filename: path to the directory in which the relative image is stored;
• •damagePresence: classification label indicating if damage instances are present or not (i.e., labels are: “damage” / “no damage”);
• •structureType: label giving information on the typology of structure/infrastructure according to the list presented in the following Table 1;
• •
  size: information about the size of the image, derived from its original EXIF (Exchangeable Image File Format) file, in particular:

  • •
    width: width of the image in pixels,
  • •
    height: height of the image in pixels;
• •object element: information about the structural element on which damage is annotated, according to the classification in the following Table 2;
• •
  object type: information about the type of annotation (i.e. “rect” for rectangular bounding box, “polygon” for a bounding box of polygonal shape)

  • •
    name: information about the damage macro-category defined in Table 4, according to the label assigned to damage as listed in the following Table 3;
  • •
    bndbox: corner coordinates of the rectangular bounding box, provided in the form of x_min/x_max and y_min/y_max in pixels;
  • •
    polygon: edge coordinates of the polygonal bounding box, provided in the form of x_i-th/y_i-th of the i-th edge, in pixels.

Table 1.

Structural type assignments.

Structural Typology	Description
AE_BUILDING	Ordinary/residential buildings
GL_AEDES	Long-span buildings
CES-ADC_CHURCH	Churches
PES-BDP_HISTORICAL BUILDING	Heritage buildings, Palaces
BRIDGE	Bridge infrastructure

Table 2.

List of structural element typologies.

Structural Elements
1	Soil	35	Masonry Pier
2	Foundation	36	Masonry Spandrel
3	Expansion Joint	37	Masonry Abutment
4	Stairs	38	Masonry Bridge Pier
5	Bearing	39	Arch
6	Gerber Saddle	40	Vault
7	Seismic Isolator	41	Tie Rod
8	RC Column	42	Masonry Wall
9	RC Beam	43	Gable
10	RC Abutment	44	Timber Deck/Slab
11	RC Pier	45	Timber Beam
12	RC Joint	46	Timber Truss Beam
13	Floor Slab	47	Flexible Diaphragm
14	Infill	48	Plaster
15	RC Wall	49	Cladding
16	RC Pier Cap	50	Ceiling
17	RC Pedestal	51	Roof Tiles
18	RC Deck/Slab	52	Chimney
19	RC Arch	53	Ledge
20	RC Spandrel Column	54	Parapet
21	PSC Beam	55	Railing
22	PSC Pier	56	Curb
23	Wall Panel	57	Services (Water/Gas/Electric/Sewage/...)
24	Steel Deck/Slab	58	Light Pole
25	Steel Column	59	Window
26	Steel Beam	60	Overhang (Spire, Pinnacle, Etc.)
27	Steel Brace	61	Statue
28	Steel Spandrel Column	62	Steel Racks
29	Weld	63	Road Paving
30	Bolt	64	HVAC Machines
31	Connection	65	Floating Floor
32	Plate	66	Light Partition
33	Truss Beam/Arch	67	Furniture/Shelving
34	Cable Strand	68	Structure

Table 4.

Damage type aggregation in macro-categories.

Damage macro-category	Damage typology		Structural element
Crack	2	Flexural crack	All elements potentially subjected
	3	Flexural/rocking crack
	4	Shear crack
	5	Torsion crack
	6	Crushing crack
	17	Detachment crack
	26	Sliding shear crack
	27	Crack
	38	Hinge at keystone
	39	Hinge at springer
	40	Hinge at haunches
Spalling	7	Concrete spalling	All elements potentially subjected
	13	Clay block damage
	53	Non structural damage	47	Plaster
	55	Masonry spalling	All elements potentially subjected
Corrosion/Exposed rebar	8	Reinforcement oxidized/corroded	All elements potentially subjected
	9	Stirrups break
	10	Rebar deformation
	11	Rebar break
	12	Rebar buckling
	24	Corrosion/delamination
	49	Exposed and oxidized strand
Leaching	25	Leaching	All elements potentially subjected
	42	Material degradation and wear
Debris	57	Debris	All elements potentially subjected
Other damage/mechanism	1	Plastic hinge	All elements potentially subjected
	14	Large deflection
	15	Large displacement
	16	Loss of support
	20	Settlement
	22	Embankment instability
	23	Buckling
	29	Tilt/overturning
	31	Out-of-plane mechanism
	32	Rotation
	33	Pounding damage
	44	Impact damage
	51	Soft story
	52	Partial/full collapse
	56	Detachment of tympanum
Non structural damage	27	Crack	47	Plaster
			48	Cladding
			51	Chimney
			53	Parapet
			55	Curb
	52	Partial/full collapse	47	Plaster
			48	Cladding
			51	Chimney
			53	Parapet
			55	Curb
	53	Non structural damage		*

^⁎Structural element “47 – Plaster” is excluded from all elements potentially subjected to the damage typology “53 – Non structural damage”

Table 3.

Damage type list.

Damage type
1	Plastic hinge	30	Offset
2	Flexural crack	31	Out-of-plane mechanism
3	Flexural/rocking crack	32	Rotation
4	Shear crack	33	Pounding damage
5	Torsion crack	34	Liquefaction
6	Crushing crack	35	Pothole
7	Concrete spalling	36	Rutting
8	Stirrups break	37	Pile break
9	Rebar break	38	Hinge at keystone
10	Rebar deformation	39	Hinge at springer
11	Rebar break	40	Hinge at haunches
12	Rebar buckling	41	Pulling of tie anchorage
13	Slab clay block damage	42	Material degradation/wear
14	Large deflection	43	Nail pull-out
15	Large displacement	44	Impact damage
16	Loss of support	45	Permeable/missing flashing
17	Detachment crack	46	Reduction of pre-stressing strand section
18	Damage	47	Strand anchor damage
19	Break/fracture	48	Sheathing degradation/wear
20	Settlement	49	Exposed and oxidized strand
21	Scouring	50	Waterlogging in the box girder
22	Embankment instability	51	Soft story (*)
23	Buckling	52	Partial/full collapse (*)
24	Corrosion/delamination	53	Non-structural damage (**)
25	Leaching	54	Short term countermeasures
26	Sliding shear crack	55	Masonry spalling
27	Crack	56	Detachment of tympanum
28	Splitting	57	Debris
29	Tilt/overturning

(*) Damage at structure level.

(**) Single damage for all non-structural elements.

Examples of images and annotation files are described in the following paragraphs.

4. Experimental Design, Materials and Methods

4.1. Data Collection and Annotation

The dataset mainly consists of images acquired during post-earthquake surveys, collected in occasion of the last three major earthquake sequences in Italy: L’Aquila (2009), Emilia (2012) and Central Italy (2016–2017). In addition, the dataset includes images collected during buildings and infrastructures inspections carried out periodically by EUCENTRE’s teams, within the framework of the experimental and research activities financed by the Italian Civil Protection Department for emergency support and risk reduction purposes. At the time of writing, the dataset contains more than 5’400 photos, both classified and annotated.

Most of the images in the dataset have been acquired with digital cameras during traditional inspection procedures. In the years, additional high-quality images of structural portions usually hard to inspect closely, like those of bridges, have been collected with the use of UAS [1]. The resulting dataset is thus characterized by a significant variety of images in terms of image resolution and size, lighting conditions (depending on meteorological conditions and if the image is outdoor or indoor), with different points of view and geometrical scales (structural element, structural portions, or structure as a whole). Hence, the dataset is representative of different levels of observability and detectability and therefore it can be considered suitable to counteract possible imbalance of training samples in case of use for training/validation of damage detection algorithms [5].

A dedicated tool has been developed to annotate the images of the dataset, according to the damage ontology described in the following. The architectural components on which the application is based are:

• •Apache Tomcat, as application server, where the application is installed.
• •PostgreSQL, as relational database engine.
• •Apache Solr, as index, to provide good data search and consultation functions. Solr is a high-performance, highly scalable open-source indexing and search engine.
• •VGG Image Annotator (VIA), an image annotation tool [7].

4.2. Data Classification Framework

In general, damage detection is the first step of a diagnosis process, because the presence of a damage may affect the structural behaviour or safety. The localization of damage is also essential, since there are crucial elements and positions within elements for which the same damage may result more critical than for others. The damage identification process consists of several tasks, such as ontology definition, damage detection and localization and evaluation of damage severity. Each of them could be crucial in order to provide a reliable support for structural assessment.

Depending on the structural member, material and construction technology, several types and levels of damage can be detected, potentially affecting the overall safety and usability of the inspected structures. therefore, the damage classification defined in the following aims to be as detailed as possible, with a refined list of structural elements and damage types, as well as a predetermined association, allowing both a precise description of the structural response and constituting a common reference for possible future expansions of the dataset. The classification framework consists of a detailed ontology defined for distinct groups of properties, applicable both in image classification and annotation through segmentation or object detection:

• •Properties relative to structure and damage. Structure type/use, Construction technology, Element, Damage presence/class. These properties are the most relevant for classification process purposes. Structural type, construction technology and damage presence are assigned as image label (only a record for each class is allowed). Regarding the Element and Damage classes, they are annotated through bounding boxes in the area where the damage is detected.
• •Image properties. Type of image, resolution, colour. These properties are an intrinsic characteristic of the image and are assigned automatically in the relative EXIF file, when applicable.

4.3. Structural Properties and Damage Ontology

The ontology presented hereafter has been devised with the aim to create a framework open to subsequent dataset enlargement, especially for structures and damage types whose number of images may potentially be expanded with future data acquisitions.

The first properties to be defined are related to the structural type, as showed in Table 1. The classification is based on the types of first level forms used in Italy for the post-earthquake assessment of structural damage and usability.

The other properties of major interest concern the structural element (Table 2) and the damage type (listed in Table 3). It is intuitive that, in the same image, different structural elements and damage types can be found. As far as construction technology is concerned, several typologies such as reinforced concrete, reinforced/unreinforced masonry and, but not limited to, steel structures are implicitly covered in the database by the broad identification of structural types and structural elements classification.

To facilitate the proper correlation between structural elements and damage, a mapping among the two categories has been established, so that when a structural element and its construction technologies are identified only their typical damage mechanisms can be selected. Such choice implied to “force” some assignation for the case of damage related to the overall structure and for non-structural damage. Some damage type (marked as “*” in Table 3) in fact may not be associated to a specific single element, but to global phenomena affecting the whole structure, or great part of it, like collapse, instability, soft story, out-of-plane overturning mechanisms for which the “special” element “structure” has been defined. For what concerns non-structural elements, a general type of damage is considered (marked as “**” in Table 3), since it does not affect the structural response. In this case, the only important exception is that of infills, whose damage has been specifically accounted for.

It should be noted that such intentionally refined classification does not exclude the possibility of creating an ex-post “coarse” grid by aggregating the classified damage into macro-categories. Table 4 reports the aggregation of a selection of damage typologies that was applied in the research by [8], which aimed to train and validate DCNN-based algorithm developed for structural damage detection purposes in emergency contexts. Both refined and coarse damage classification labels are included in each annotation in the IDEA dataset.

Fig. 2 shows the occurrence of damage instances for each damage macro-category (listed in Table 4) in the whole dataset. Crack and spalling appear to be the most present ones as expected since these damage classes are the most recurrent for masonry and reinforced concrete (RC) structures, which constitute most of the dataset content as far as construction technology is concerned.

Fig. 2.

IDEA dataset: occurrence of instances, aggregated based on macro-categories.

4.4. Examples of Dataset Content

As previously mentioned, the dataset consists of a large amount of images, heterogeneous for what concerns the use/configuration of inspected structures, including most of the types listed in Table 1, and the typologies of damage annotated.

Some examples are reported for the sake of clarity for different types of structures and construction techniques. Fig. 3 shows the post-earthquake damage of a RC girder bridge, consisting respectively in large displacements of the bearings and an evident offset between the beams of adjacent bays. Fig. 4 shows a RC column with an extended area of concrete spalling, and evident corrosion of longitudinal rebars and stirrups. These kinds of damage are very frequently observed in reinforced concrete bridges, as showed in Fig. 5, where the pier is subjected to leaching and above all a severe concrete spalling and corrosion of steel-reinforcement. This deterioration process is mainly due to the direct exposure to atmospheric and chemical agents; in particular, factors such as humidity, freeze-thaw cycles, use of de-icing salts and proximity to the coast, in combination with lack of proper maintenance, can accelerate the material degradation. Figs. 6 and 7 show the possible effects of an earthquake to the façade of unreinforced masonry structures: the first (Fig. 6), a residential building, exhibits the typical in-plane response, with shear cracks and spalling, while the second (Fig. 7), a church, has experienced local collapse due to out-of-plane overturning of the façade together with the expulsion of the external masonry leaf of the front wall, due to lack of connection and poor quality of the masonry.

Fig. 3.

Damage to a RC girder bridge. Left: (1,2) large displacement of the bearing. Right: large displacement (lateral offset) between decks of a RC girder bridge (1), and concrete spalling of RC slab (2).

Fig. 4.

Concrete spalling (2) and reinforcement corrosion (3) at the base of a RC column, with debris (1).

Fig. 5.

Multiple damage annotation on the top of a RC bridge pier: concrete spalling (1), reinforcement corrosion (2) and leaching (3).

Fig. 6.

Shear cracks on the façade of a residential unreinforced masonry building: shear cracks in masonry piers (1, 2, 9); debris (3, 4); masonry spalling in masonry piers and spandrel (6, 7, 8, 10, 11).

Fig. 7.

Façade of the church with multiple severe damage: collapse of the top portion of the façade (3) with associated cracks (2, 4 and 8) and masonry spalling (5 and 9). On the left, a partial collapse of the external layer of the masonry wall (6), with cracks associated to the out-of-plane response of the church (1), masonry spalling (7 and 12) and non-structural damage of the window (11). In front of the church, debris from the collapsed structural portions are visible (10).

The following Figs. 8 and 9 are samples of XML annotation file for two different images with structural damage. Fig. 8 refers to a bridge image in which leaching and spalling were detected on reinforced concrete piers, annotated with rectangular bounding boxes. In Fig. 9, instead, cracks on a masonry spandrel of an ordinary building were annotated with polygonal bounding boxes. Fig. 10 reproduces a XLM annotation file of an image of a church with a “no damage” classification.

Fig. 8.

IDEA dataset: example of a bridge structure with damage instances annotated with rectangular bounding boxes and polylines. Right: annotation file content. Left: original (top) and annotated (bottom) image.

Fig. 9.

IDEA dataset: example of an ordinary building with damage instances annotated with rectangular bounding boxes and polylines. Right: annotation file content. Left: original (top) and annotated (bottom) image.

Fig. 10.

IDEA dataset: example of a church with a “no damage” classification. Right: annotation file content. Left: original image.

Limitations

The IDEA dataset is subjected to the license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International CC BY-NC-ND 4.0.

Ethics Statement

The authors have read and followed the ethical requirements for publication in Data in Brief and confirm that the current work does not involve human subjects, animal experiments, or any data collected from social media platforms.

CRediT authorship contribution statement

Ilaria Senaldi: Data curation, Methodology, Writing – original draft, Writing – review & editing. Chiara Casarotti: Conceptualization, Methodology, Supervision, Writing – review & editing. Martina Mandirola: Data curation, Methodology. Alessio Cantoni: Software.

Data Availability

• ZenodoIDEA: Image Database for Earthquake damage Annotation (Original data).