PituPhase65: An endoscopic pituitary surgery dataset for surgical phase recognition

Abstract

Surgical Phase Recognition (SPR) aims to provide the surgical team with valuable feedback while performing an intervention. In the near future, these systems will help to shorten the learning curve of residents, improve communication among professionals in the Operating Room, and obtain an automatic complete surgical report. This is an upcoming reality for surgeries such as cholecystectomy, which are highly standardized and broadly studied. Nonetheless, endoscopic pituitary surgery still has a long way to go, as it has a higher workflow variability which hinders model learning. To move forward in this field, it is necessary to publish labeled data with which to train these models. With that aim, PituPhase65 was created, a publicly available dataset including 65 endoscopic pituitary surgeries, with a mean duration of 90 minutes, performed at Hospital Universitario 12 de Octubre (Madrid, Spain) and labelled into eight different phases: out of patient, middle turbinate resection, nasoseptal flap preparation, ethmoidectomy and sphenoidal sinus opening, sellar opening, dural opening, tumor resection, and dural closure.

Background & Summary

In our new digital world, Artificial Intelligence (AI) systems have become part of our daily lives, leading to advances in multiple fields. In medicine, AI is being leveraged to incorporate new tools to bring more personalized diagnostics and treatments, and even more, to optimize workflows alleviating the burden of repetitive tasks. Specifically in surgery, AI aims to either improve preoperative, intraoperative and postoperative systems. Among others, the goals include: to make an early and accurate diagnosis of surgical outcomes through preoperative risk predictions; to use surgical navigation and intraoperative guidance systems to support the training of new surgeons; and finally, to predict postsurgical complications, scheduling hospital discharge in an efficient way¹.

The last decade has been key for the incorporation of AI in intraoperative decision making and in clinical and experimental data production, processing and storage². Particularly, Surgical Phase Recognition (SPR) is a critical component of intraoperative decision support systems for improving communications among the surgical team and optimizing the Operating Room (OR) schedule. Moreover, these systems could aid in mitigating errors giving early warnings when detecting anomalies in the surgical practice³. SPR studies allow to stablish a standardized protocol for surgery, essential for surgical education and patient postoperative monitoring⁴, as well as to automatically generate a complete operative report in an efficient way.

The development and use of this kind of applications in real clinical practice is well within our grasp. Nevertheless, we must continue investigating models to reach optimal accuracy levels. With this purpose we have built PituPhase65⁵, a database including 65 labelled endoscopic pituitary surgical videos, a procedure with a highly workflow diversity, which makes it necessary to collect as much data as possible to train robust SPR models. To date, PituPhase65 is the largest dataset for endoscopic pituitary surgery that has been published

The creation of datasets such as PituPhase65⁵ contributes to the advancement of scientific knowledge as it brings the opportunity to make an objective comparison between different methodologies, with the objective of identifying the best solutions. PituPhase65 is essential for the development of robust clinical applications not overfitted to specific techniques or to the data of a single Hospital. In this sense, this work contributes to the construction of an endoscopic surgical dataset to perform multi-center studies. Furthermore, data labelled in different surgical phases, provides an added value for the scientific community. Annotation is a very costly procedure, as it is a tedious and time-consuming task that must be performed by highly specialized surgeons.

Automated SPR assessment was tackled by our prior work⁶ that introduced SAHC⁷ in the context of endoscopic pituitary surgery, a methodology based on the use of attention and TCNs to correct misclassifications caused by ambiguous frames. This work used a smaller version of PituPhase65. This initial dataset⁸, while instrumental in enabling a proof-of-concept study, was characterized by two critical limitations: 1. It did not consider all the phases initially established by neurosurgeons, and 2. Its smaller size containing only 44 surgical procedures.

The new dataset, PituPhase65⁵ includes improvements in both areas. First, the middle turbinate resection phase, which was not considered in the first version of the dataset, is now included. Moreover, the number of videos has been increased, adding 21 new surgeries. Lastly, the inter-annotator reliability test has been updated, revising those surgeries in which the level of agreement between both annotators was lower than 80%. As a result, PituPhase65is both more complete and more accurate due to these improvements.

There are publicly available datasets from other endoscopic procedures widely studied such as Cholec80⁹, a collection of 80 cholecystectomy videos. LapEx¹⁰ is another example within laparoscopic surgeries. Moreover, other procedures have been studied in SPR and have publicly available datasets, for instance Cataracts¹¹. Nevertheless, these are standardized surgeries that do not present the workflow diversity of endoscopic pituitary surgery.

Importantly, PituPhase65 complements existing open-source endoscopic pituitary surgery datasets, such as PitVis¹², made up of 25 videos, being significatively smaller than the dataset presented in this work. Nonetheless, it was the first to be published and represents a significant contribution to the field, enabling the evaluation of several SPR methodologies in the PitVis-2023 Challenge¹³.

Aware of the importance of having accurate publicly available datasets which promote investigation and development guided to surgical safety, our multidisciplinary team, consisting of neurosurgeons and engineers, presents PituPhase65, the longest published dataset until date in the field of endoscopic pituitary surgery.

Methods

Data collection

All endoscopic procedures at our institution are routinely recorded. For this study, patients were selected from a prospectively maintained database of surgeries performed between 2017 and 2025 at Hospital 12 de Octubre, Madrid, Spain. From a cohort of approximately 160 patients who underwent endoscopic transnasal surgery for pituitary neuroendocrine tumours during this period, the most complete video recordings were selected. Although recording began in 2017, a more systematic approach was adopted in 2019 to ensure complete coverage of each procedure. All procedures were performed by two experienced neurosurgeons (A.L. and I.P.) each with more than 10 years of experience in endoscopic pituitary surgery.

Surgical visualization was achieved using 0° and 30° endoscopes (Karl Storz Endoskope, Tuttlingen, Germany - from 2017 to 2024 – and Stryker Corporation - Kalamazoo, MI, USA-from 2024 to 2025) and 4 K video cameras (IMAGE1 STM 4U, Karl Storz; resolution: 3840 × 2160 pixels; CCU 1688 AIM 4 K, Stryker).

Video recordings were captured using the AIDA system (2017–2024) in HD format, and from September 2024 onwards, using the Stryker Connected OR Hub system in 4 K UHD format. Videos were automatically segmented into separate files by the recording system to facilitate storage and transfer.

The videos were de-identified removing all the sensitive data they could contain, and instead, patients received an anonymous identifier. Videos of the same surgery were grouped in the same folder with consecutive numbering so that each video had a unique ID.

The Ethics Committee of the Hospital Universitario 12 de Octubre approved data collection with date 16^th March 2023, inside the study “Surgery AI video analytical tool” (ID 23/037). Access to the PituPhase65 dataset is governed by a Data Usage Agreement (DUA) available at the database repository. Following a risk assessment, this controlled access was implemented because the surgical videos and associated metadata present a non-negligible risk of participant re-identification. To ensure secure handling, dataset files are currently encrypted, and access is granted for any purposes that do not breach the privacy and non-identification clauses established in this DUA.

All videos were reviewed by the research team to remove any images accidentally acquired in the perioperative phase that could contain directly identifiable patient features.

Data Pre-processing

For data labelling a methodology based on audio signal was used, therefore the videos were labelled on the audio bar. For that reason all the videos must contain an audio signal, and the first pre-processing step was to add a silent audio in those videos which did not contain an audio signal. Next, the resolution was downscaled from 1920 × 1080 to 480 × 270 pixels in order to reduce memory load in the labeling tool. Before making this decision, a study was performed to verify with the neurosurgeons that the reduced resolution was suitable for video annotation.

Surgical phase definition

The surgical team involved in the study defined an annotation framework consisting of nine different phases, that are presented in Table 1.

Table 1.

Phases of transnasal endoscopic surgery.

ID	Name	Start	End
P0	Outside the body	Endoscope exits the body	Endoscope enters the body
P1	Optics Preparation	Recording connected	Enter the nose
P2	Middle turbinate resection	Optic insertion, dissection and cutting of the turbinate	Hemostasis of the turbinate
P3	Nasoseptal flap preparation	Creation of the flap from the nasal septum	Placement of the flap in the maxillary sinus, choana
P4	Ethmoidectomy and sphenoidal sinus opening	Ethmoid resection; resection of the posterior septum	Sphenoid sinus drilling and removal of its mucosa
P5	Sellar opening	Drilling of rostrum and sella turcica	Exposed sella dura
P6	Dural opening	Dura coagulation	End and beginning of resection
P7	Tumour resection	Tumor resection	Hemostasis
P8	Dural closure	Placement of fat/Duragen	Placement of the nasoseptal flap/Tisseel

In this work, a phase refers to a temporally coherent segment of the procedure defined by a specific surgical objective and workflow context. Unlike lower-level concepts such as actions or steps, phases represent higher-level procedural units comprising multiple elementary maneuvers, making them more suitable for surgical workflow analysis.

Phases were defined to reflect the routine workflow followed at our institution. We acknowledge that a Delphi consensus study has proposed standardized phase definitions for endoscopic pituitary surgery¹⁴. However, this guideline was not yet available when our annotation protocol was established.

We did not modify the original annotation scheme afterwards in order to preserve consistency across the annotated dataset, maintain a simpler and more reproducible labeling process, and account for minor local differences in surgical technique.

Video annotation

Videos were labelled using Label-studio (https://labelstud.io) by two expert neurosurgeons with more than 10 years of experience in endoscopic pituitary surgery, according to nine different phases previously defined by the surgical team (see Table 1).

Fig. 1 shows the labeling tool interface which is freely available. However, after the first round of annotations no instances of Phase 1 (Optics Preparation) were identified, and it was consequently removed. Nevertheless, it is maintained in our phase definition for consistency and for consideration for future recordings. Figure 2 shows representative frames corresponding to each class.

Fig. 1.

Labelling tool interface.

Fig. 2.

Representative frame from each phase.

Once the surgeries were labelled, the technical team revised them using visual representations (see Fig. 3) and questioned the expert’s decisions when uncertainties arose.

Fig. 3.

Visual Representation of three PituPhase84 surgeries labelling where each color corresponds to one phase.

In some cases, the order of the video files from a given surgery needed to be changed as they had been mixed up.

Data Record

The dataset PituPhase65⁵ is available at the eCienciaDatos repository (Consorcio Madroño, Madrid university library consortium [https://edatos.consorciomadrono.es/]), accessible at https://doi.org/10.21950/7JINUB.

The dataset is distributed under a Data Usage Agreement (DUA), located as a pdf file in the root directory of the dataset. To download the data, interested users must request access and agree to the terms outlined in the DUA. Both the DUA and the README files, which provide details about the dataset’s contents, are available for preview before access is requested.

The root directory of the dataset includes:

1. Data_Usage_Agreement_PituPhase65.pdf, that must be completed and signed to request data access.

2. A README.txt file, named PituPhase65_Readme.pdf, describing PituPhase65 content.

3. 65 sub-folders, one per surgery. Each folder is named with an anonymized patient identifier (e.g., ‘10XX’) and is composed of:

   • A .json file containing the surgical labels present in each video with start and end times in seconds.
   • A videos directory which contains all the video segments (.mp4 files) that together form the complete surgical procedure for that patient. These videos are named in sequential order with format “patient number_video number”.

Data description

Mean and standard deviation duration values for each phase, shown in Fig. 4, demonstrate the high unbalance among classes in the dataset. A histogram of phase occurrences is shown in Fig. 5, and Table 2 summarizes phase transition information.

Fig. 4.

Mean phase duration. The bars indicate the standard deviation. Numbers next to bars refers to mean duration in minutes (blue bar).

Fig. 5.

Histogram of phase occurrences. The number above bars indicate the exact number of instances for each phase.

Table 2.

Phase transitions in PituPhase65.

Next Phase (%)
Phase	P2	P3	P4	P5	P6	P7	P8
P2	—	75	25	0	0	0	0
P3	0	—	53.33	0	0	0	46.67
P4	0	10.71	—	78.57	3.57	3.57	3.57
P5	0	0	1.56	—	90.62	7.81	0
P6	0	0	0	7.69	—	92.31	0
P7	0	0	0	2.33	18.60	—	79.07
P8	0	88.89				11.11	—

The numbers indicate the relative (in percent) transitions from one phase (row-axis) to another (column-axis).

The high variability in the dataset is caused by the presence of incomplete surgeries, which represent almost 20% of the videos, and by the surgical variability inherent to the procedure itself, as it greatly depends on tumor and patient characteristics. For instance, depending on tumor characteristics and invasiveness, the lesion may already extend into the nasal cavity, requiring tumor resection to precede other surgical phases. Additionally, in certain cases involving small tumors with a low risk of intraoperative cerebrospinal fluid (CSF) fistula, the surgical team may decide not to harvest a nasoseptal flap, and therefore Phase 3 is omitted.

To provide further insights on the dataset, we studied the inter-annotator variability on the labelling task: two expert neurosurgeons annotated the same 19 videos extracted from the dataset, then a Kappa Coefficient was calculated. This study is detailed in subsection 1 of the Technical Validation section.

Technical Validation

Agreement score

To guarantee PituPhase65⁵ technical quality, an agreement score between both annotators was computed in those videos which had both annotations. Specifically, this corresponds to 32 surgeries in the dataset (49%). Among these, seven presented an agreement level below 80%. In those cases, both neurosurgeons reviewed the labelling, ensuring a minimum consensus in each of the surgeries, thereby generating robust labels.

Out of patient labeling

Neurosurgeons were supported in the labelling process by the technical team for the “out of patient” frames annotation. These frames are very distinct and the expertise of a surgeon is not required to detect the moments when the endoscope is out of the patient’s body. The labelling of these episodes is by far the most time-consuming task, because they can be present anywhere in the recording and are short and therefore easy to miss. For that reason, the annotations of this phase (P0) were refined in a second round of labelling carried out by the engineers in the team.

Intra-annotator test reliability

To evaluate intra-annotator reliability in the labelling process, one neurosurgeon labelled seven of the surgeries of the dataset twice, with a time period of a year and a half between the annotations. This resulted in a Cohen’s Kappa coefficient (κ) of 81%, indicating a low level of variability between both annotations. The observed ifferences can be explained in part to the neurosurgeon’s increased experience over time, which may have shifted their criteria for distinguishing surgical phases. Additionally, the inherent variability of the surgery, where phase transitions are not always clear, can hinder class separation. Figure 6 shows a visualization of the annotations of two surgeries: the one with the greatest discrepancy (6.1) and the one with the closest match (6.2). It is worth noting that differences could be amplified by the fact that the first annotations were performed in 2023 (when the project started) while the second ones were done this year (2025). The highest differences were observed in the ‘out of patient’ (P0) and ‘tumor resection’ (P5) phases. On the one hand, P5 is the largest phase of endoscopic pituitary surgery and to exactly delineate the transitions between this phase and the previous or subsequent ones is an arduous task; that explains the worst κ. On the other hand, as explained in the previous subsection, ‘out of patient’ labelling is particularly time-consuming. As a result, neurosurgeons do not label every single occurrence, but rather label only some of them, leading to differences when annotating the same surgeries.

Fig. 6.

Visual representation of the annotations with the worst and best intra-annotator agreement.

Inter-annotator test reliability

Last, an inter-annotator reliability test was performed to evaluate the concordance between both annotators. With that purpose, both neurosurgeons labeled the same 17 surgeries in a blinded way, without having access to the other annotator’s labels, avoiding to introduce bias in the annotation process. With those labels a Kappa coefficient of 84% was obtained. According to Cohen’s Kappa values published as reference¹⁵, there is a near-perfect level of agreement between both annotators. After confirming annotation similarity, it was concluded that it was possible to unify labels from both annotators into a single dataset. Thus, to reduce labeling costs, each of the expert neurosurgeons labelled half of the remaining videos.

Notably, the higher points of disagreement between both annotators are in transitions between phases, as it is extremely difficult to define the exact moment of transition. Nevertheless, given the alienation that exists between both annotators, we concluded that eventual disagreements indicate the inherent presence of ambiguous moments.

From all of the surgeries included in the dataset, those listed below include the labels of the first annotator: 1013, 1016–1020, 1031, 1032, 1035–1038, 1049–1057, 1059, 1060, 1062–1064 y 1069. The remaining ones, include the labels of the second annotator.

Limitations of the Dataset

One of the limitations of PituPhase65⁵ is the unbalanced nature of the data owing to the differences in phase durations. This difficulted model learning to perform SPR. Moreover, PituPhase65 has the intrinsic limitation of including data from uniquely one institution. Nevertheless, we believe that by sharing the data and the preprocessing and labelling procedure, we contribute to the ambitious objective of collecting a large-scale multicentric dataset.

Usage Notes

PituPhase65⁵ dataset can be downloaded from https://doi.org/10.21950/7JINUB. The dataset is shared under a Data_Usage_Agreement (DUA), that must be filled-in and emailed to the corresponding author to request access. The dua must include the names of the researchers with access to the data and a short note on the purpose of the research. After acceptance of the agreement, an email notification will be sent with the password needed to access the data.

Video files can be downloaded individually or as a single ZIP archive. To download the zip file, select the red “Access dataset” tab next to the dataset name and then the “download zip” option.

Author contributions

A.L., I.P. performed the data collection, defined the annotation protocols, annotated the videos, and reviewed the manuscript. A.G. prepared the software used for the annotations, revised and refined the labeling of the out-of-patient phase, conducted the technical validation and drafted the manuscript. P.T. contributed to data collection, coordinated the annotation process, and participated in manuscript revision. A.C. contributed to the technical validation, created the data repository and reviewed the manuscript. A.L. and P.T. conceptualized and supervised the whole project and secured the funding.

Data availability

PituPhase65⁵ dataset is available at the eCienciaDatos repository, accessible at https://doi.org/10.21950/7JINUB.

Code availability

No custom code was used to generate any elements of this paper or the dataset. Label-studio 1.13.1 was used to label the surgeries.

Competing interests

The authors declare no competing interests.