Benefits of sharing neurophysiology data from the BRAIN Initiative Research Opportunities in Humans Consortium

Abstract

Sharing human brain data can yield scientific benefits, but investigators are commonly disincentivized from sharing. We profile three successful data-sharing experiences from the NIH BRAIN Initiative Research Opportunities in Humans (ROH) Consortium and demonstrate benefits to data producers and to users.

Introduction

Research funders, journals, and institutions have increased their expectations for FAIR (findable, accessible, interoperable and reusable) neurophysiology data sharing.¹ Under the new U.S. National Institutes of Health (NIH) Data Management and Sharing (DMS) Policy, research data generated using federal funds are now required to be deposited into designated archives “as soon as possible, and no later than the time of an associated publication or the end of the award/support period, whichever comes first” (NOT-OD-21-013). The NIH-led BRAIN Initiative enforces its own data-sharing policy, with similar terms of sharing (NOT-MH-19-010).

Sharing and reusing human neural data can inform new research directions, save money, drive innovation, enhance rigor, and minimize waste.² The case for sharing invasive human electrophysiology data, such as intracranial EEG (iEEG) and single-neuron recordings from the brain, is particularly compelling, owing to the rarity of these data and the resource-intensiveness required for their collection. Yet, investigators can feel disincentivized to share because of time, resources, and training required as well as concerns related to lack of attribution.^1,2 Researchers sharing human brain data often are faced with similar disincentives, and thus, only a portion of these data is currently shared. Nevertheless, there have been many efforts to share human neuroimaging data over the past two decades, one of the most successful being the Human Connectome Project.³ In this case, unanticipated issues arose that serve as key learning opportunities for invasive human electrophysiology, even as the benefits of sharing also have been clear, with at least 1,538 publications resulting from the data as of 2021.^1,3,4

In this NeuroView, we describe scientific and social value propositions for the FAIR sharing of these data and highlight benefits of sharing for data generators. Our hope is to combat a pure compliance mindset of doing the minimum to satisfy data-sharing policies and avoid penalties. We showcase examples of successful data sharing from three invasive human electrophysiology research groups within the BRAIN Initiative Research Opportunities in Humans (ROH) consortium (herein referred to as the ‘ROH Consortium’). For each case, we describe and categorize strategies used to facilitate FAIR data sharing–for instance the use of standardized file formats—and summarize how others have reused the data, such as in education (Table 1). Finally, we emphasize how sharing has benefited data generators and offer recommendations to help maximize these benefits for generators and users.

Table 1.

Major characteristics of three successful data-sharing processes developed for invasive human electrophysiology within the NIH Research Opportunities in Humans (ROH) Consortium

	Restoring Active Memory (RAM) project: Michael Kahana Laboratory, University of Pennsylvania, and partners	Memory Intracranial Neural Dynamic (MIND) project: Ueli Rutrshauser Laboratory, Cedars-Sinai Medical Center, and partners	High-resolution single-unit recordings of cortical neurons dataset: Laboratories at Harvard Medical School and Massachusetts General Hospital
Population	patients with refractory epilepsy	patients with refractory epilepsy	participants undergoing clinical intraoperative physiological recordings
Data type(s)	• electrocorticographic recordings • patient demographics • individual electrode contact atlas location and coordinates for localization • FreeSurfer files • D-cortical surface renderings • seizure onset zones • interictal spiking • behavioral event data for ten different memory tasks • open- and closed-loop brain stimulation tasks	• Electrophysiological recordings (single neurons and local field potentials) • patient demographics • behavior • task stimulus • task event timestamps • electrode locations	• high-resolution laminar recordings
Format(s)	BIDS	NWB	• SpikeGLX • OpenEphys
Archive(s)	University-hosted webpage	DANDI DABI	Dryad
Strategies for data sharing	• standardized file formats	• standardized file formats • outreach to unique users • publication and use of open-source software	• standardized file formats • publication and use of open-source software
Selected ways data have been reused	• education, training • analytical tool development	• outreach, education, and training • analytical tool development • scientific discovery (research)	• analytical tool development • raw data export and analysis
Benefits for data generator	• earty-stage investigator training and career advancement • reanalysis of existing data • facile dataset curation	• resolution of technical issues • enhanced lab productivity • identification of collaboration opportunities • facilitates reanalysls of existing data • use in teaching	• stimulated interest in intra-species comparison of brain • multidisciplinary collaboration

Members of the ROH Consortium research groups contributed these case studies. We aimed to capture known examples of reuse for data generated by each group; however they are not exhaustive, and we expect the benefits of FAIR sharing to expand as more data are shared and our fundamental understanding of the human brain unfolds.

Penn Restoring Active Memory Project: Michael Kahana Laboratory, University of Pennsylvania, and partners

Michael Kahana’s laboratory at the University of Pennsylvania investigates the neural basis of human memory. Since 2010, the group has shared data from their multi-center brain recording studies of human memory through a university-hosted webpage, which links data and, often, analysis code to host data files and published papers. In 2014, Kahana’s team completed several public, large-scale data releases. Data were released across eight academic medical centers participating in the Restoring Active Memory (RAM) project, with funding from the BRAIN Initiative (https://memory.psych.upenn.edu/RAM).

Process of sharing the data

Data release from the RAM project included annotated data from more than 400 neurosurgical patients undergoing intracranial electrode recording for seizure mapping. These data included electrocorticographic (ECoG) recordings from >1,700 experimental sessions, mostly involving memory experiments and/or brain stimulation. The shared metadata include demographic information, individual electrode contact atlas location and coordinates for localization, FreeSurfer files, cortical surface renderings, seizure onset zone, interictal spiking, experiment design documents, session notes and behavioral event data for multiple memory tasks, and open- and closed-loop brain stimulation tasks.

All investigators on the RAM team obtained informed consent from patients to share their de-identified data. Kahana’s group also released a set of Python tools (CMLreaders) to facilitate access to data collected using multiple recording systems. Kahana’s group has begun converting the RAM data for release in Brain Imaging Data Structure (BIDS) format (https://bids.neuroimaging.io/), which was originally developed for MRI as part of OpenNeuro (https://openneuro.org/), one of the BRAIN data archives. In 2019, BIDS was extended to other data types, including EEG and iEEG.⁵

Converting the RAM data into the iEEG-BIDS format required several weeks’ effort by an experienced data scientist. Kahana’s group also has recently uploaded over 7,000 hours of scalp EEG and memory tasks in BIDS to OpenNeuro. The considerable investment of time and expertise required to convert data to the BIDS standard has been discussed in other contexts.⁶ Yet, the Kahana laboratory’s approach to sharing employing this format has led to demonstrable advantages, as noted below.

How others have reused the data and the benefits to data generators

The RAM project exemplifies how sharing of invasive human electrophysiological data may be achieved on a large scale, despite the high costs of collection and the analytical complexity of the data. More than a dozen peer-reviewed papers on the electrophysiology of human memory using publicly available data curated by the Kahana lab have been published. Many authors of these papers did not have any affiliation with the RAM research effort. Moreover, the research reported in these papers likely would not have been pursued by the investigators on the RAM team. Thus, here the act of data sharing inspired new research.

Through more than a decade of sharing, no one in the Kahana group has ever been “scooped” by another group working with shared data. Instead, data sharing has motivated other groups to pursue novel analyses of existing datasets. Additionally, the large size and volume of the datasets shared have helped the Kahana group overcome curation challenges associated with sharing other datasets.

Memory Intracranial Neural Dynamic Project: Ueli Rutishauser Laboratory, Cedars-Sinai Medical Center, and Partners

Our second case study involves Ueli Rutishauser’s laboratory at Cedars-Sinai Medical Center (https://www.cedars-sinai.edu/research/labs/rutishauser.html), which leads the Memory Intracranial Neural Dynamic (MIND) Project that also includes several academic partners. Specifically, the Rutishauser group studies mechanisms of learning, memory, and decision-making, placing particular focus on the human brain at the single-neuron level. Generated data include the activities of neurons recorded using depth electrodes in patients with intractable epilepsy.

Process of sharing the data

Members of the MIND Project on human episodic memory use the Neurodata Without Borders (NWB) format to structure their data.⁷ NWB data files are hosted on Distributed Archives for Neurophysiology Data Integration (DANDI), another BRAIN Initiative-funded archive (https://www.dandiarchive.org/). Alongside recorded electrophysiological data, NWB files include metadata related to data acquisition, task stimuli, event timestamps, participant demographics, behavioral responses, and Montreal Neurosurgical Institute coordinates of recording electrodes. The result is a comprehensive NWB file containing all data and metadata needed to analyze and reuse patient data by protocol type.

How others have reused the data and the benefits to data generators

Several major reuse cases of MIND Project data have included activities related to education, training, and tool development. For example, in 2022, datasets from the group were used at the Allen Institute’s NeuroDataReHack Hackathon (https://alleninstitute.org/events/2022-neurodatarehack-hackathon/), including activity data recorded from 1,863 neurons in the medial temporal lobe across 59 human subjects with intractable epilepsy. The International Neuroinformatics Coordinating Facility (INCF)/MATLAB Community Toolbox Training Project used another Rutishauser lab-generated dataset to teach users how to employ the NWB Application Programming Interface. The standardized datasets also have doubled as an educational resource for unique users. For example, children ages 8 to 15 years reviewed the data using NWB-based graphical user interfaces and helped translate the original publication of the dataset into a version for Frontiers for Young Minds.⁸ The interactive, web-based viewing, made possible by DANDI, enabled readers with limited programming experience to visualize the structured data and understand key findings from the parent study.

Additionally, conducting data releases has been a valuable experience for those within the laboratory. While preparing single-neuron data in the NWB format, numerous subtle inconsistencies and technical issues in data-to-be-shared have been discovered, often with the help of the automatic validators provided by NWB and DANDI. These errors were subsequently resolved and properly documented. Sometimes, these issues were not apparent in initial analyses. Thus, without the rigor of standardization for sharing, errors would have remained undetected. Second, generating standardized datasets has promoted continuity of laboratory productivity and data preservation. New laboratory members, interns, rotation students, and students conducting class projects have been easily able to reuse datasets generated by others. Third, the increased visibility and accessibility conveyed by standardized, publicly available datasets have led to other research groups approaching the laboratory to initiate collaborations. As a result, the Rutishauser team has discovered other scientists with similar interests in diverse fields.

High-resolution single-unit recordings of cortical neurons dataset: Laboratories at Harvard Medical School and Massachusetts General Hospital

A collaborative group at Massachusetts General Hospital (MGH), which includes the laboratories of Ziv Williams, Sydney Cash, and Angelique Paulk at Harvard Medical School (https://zivwilliams.mgh.harvard.edu/), is our final case study. This group has been focusing on using single-unit recordings of cortical neurons to study human cognitive processes at a cellular level.⁹ Unlike prior projects focused on generating large datasets, this project generates smaller datasets through the implementation of Neuropixels (https://www.neuropixels.org/), ultra-high density, fully integrated linear silicon microprobes. This technology enables acute, high-resolution laminar recordings from cortical neurons in participants undergoing clinical intraoperative neurophysiological recordings. As this approach is still nascent and includes a small number of participants (n<30), open access to and dissemination of the data have been important priorities for the development of the field.

Process of sharing the data

The MGH team has developed a pipeline for de-identifying data, making it freely available on Dryad (https://datadryad.org/stash/dataset/doi:10.5061/dryad.d2547d840) and DANDI (https://dandiarchive.org/dandiset/000397). The team members also place constraints on how their data may be used, consistent with the ethical guidelines for invasive human neurophysiology recently issued from the ROH Consortium.¹⁰ As described in further detail elsewhere,⁹ the raw data are re-saved in a way that ensures the surgical date cannot be traced back to individual participants. The metafiles also are edited to ensure that no dates are saved, and file names are re-coded such that there are no patient identifiers.

Finally, associated imaging scans are defaced using a manual software enabling the removal of participants’ facial features on MRI while preserving their anatomical brains. Once shared online, detailed descriptions are linked to the recorded data as well as to the open-source software and codes used to process the recordings. For example, direct links are provided to the SpikeGLX, Open Ephys, and Probe map export functions, which can allow the raw data to be easily explored. Finally, access to the motion correction software used to optimize signal registration is provided (https://github.com/evarol/dredge). These codes are paired with the recording data in a standardized (.bin) file format, which can be read by various programming languages as metadata text files on Dryad and DANDI.

How others have reused the data and the benefits to data generators

Based on total downloads/hits on Dryad and DANDI from unique users, and via direct invitations received to collaborate on new projects, it has become clear that the collaborative group pioneering work on high-resolution, single-unit recordings has catalyzed activity around the shared datasets, including for comparative analyses. These new techniques and approaches used to address human cortical recordings have generated interest from outside groups performing animal studies using similar high-density microprobes, for example. In addition, others have connected with the group to explore the cell-type identities of cortical cells and compare the extracellular action potential shapes of cortical neurons across animals and humans. Overall, the group anticipates that freely opening access to data using FAIR practices will continue to enhance not only the development of collaborative opportunities but also encourage feedback from others on how to optimize recording and analytic techniques.

Recommendations

The case studies from the BRAIN Initiative ROH Consortium presented demonstrate that shared, intracranial human electrophysiology data can be (and are) reused and that the process of sharing FAIR data has tangible benefits for generators alongside users (Table 1). Here, we highlight three benefits of sharing for data generators. To promote sharing in ways that maximize these benefits for data generators and users, we also provide three recommendations.

First, workflows for standardized data generation promote continuity of laboratory productivity and data preservation. Fewer data are lost during laboratory turnovers, and members are able to work more efficiently. The enormous efforts expended by the ROH groups to transform data into standardized formats also demonstrate the critical need for new software tools that ease the burdens of generating data and making it more useful by collating metadata, reformatting existing data, uploading it to archives, or facilitating secondary exploration. Nevertheless, the benefits of having individuals and teams in place with the expertise and experience necessary for preparing and managing data should not be underestimated. Thus, when preparing grant applications, we recommend that researchers include dedicated resources in their budgets for data-sharing training and labor, including for data scientists and programmers who can help manage data-sharing requirements. The frequency of uploading shared data will vary with the pace and volume of each project, but we recommend a minimum frequency of approximately every 6 months to ensure consistency of effort. The NIH also provides general guidelines for DMS budgeting (https://sharing.nih.gov/data-management-and-sharing-policy/planning-and-budgeting-for-data-management-and-sharing/budgeting-for-data-management-sharing).

Second, preparing invasive human electrophysiology data for sharing can reveal errors in data or code, creating opportunities to revise these errors. This is in large part because standardized file formats and associated archives (e.g., NWB and DANDI) add checks for internal consistency. Sharing open-source acquisition and analysis code also allows users to plot simple analyses as entry points into datasets, inspiring more sophisticated analyses later. Thus, sharing data using standardized formats with rich metadata and, where possible, sharing open-source acquisition and analysis code is as important as sharing parent datasets. Even widely used data formats (i.e., NWB and BIDS) presently have little interoperability with one another, which is another challenge that requires further work. Yet, labs still should consider (even if not obligated to do so by funders) releasing fully documented and standardized versions of datasets, as soon as possible after generation and after data have been appropriately protected (i.e. removal of personal identifiers). The NIH does not require that data be shared until the first publication of a dataset or the end of a funding period (whichever comes first). Yet, a common concern about “scooping” remains wherein a secondary user of data publishes key findings from a dataset before the producer of that dataset has the opportunity. Our small number of case studies presents no evidence that this has occurred, likely in part because considerable analysis of iEEG data is required to reach standards for publication. Nevertheless, this remains a valid concern, and investigators in the field will need to feel like the benefits of sharing outweigh the risks in order for this behavior to become self-sustaining.

Third, scientists who share data for reuse or reanalysis in educational settings can increase visibility, whether for datasets or for themselves and other laboratory members. Laboratories organized around preparing FAIR data can train current and future students on how to reuse them and, in turn, on how to reuse others’ data. Thus, when preparing shared datasets, we recommend that researchers also consider employing them for educational purposes, for instance, as training datasets or in hackathons, or in outreach efforts to unique users such as young learners. Such work has a cascading effect, raising awareness for research and promoting further opportunities. Researchers who share their data with the Allen Institute for Brain Science are automatically eligible for its open data science symposia, for example, catalyzing activity around shared data resources. The INCF also sponsors free online courses for data reuse, with periodic updates in line with new computational tools (https://www.incf.org/about/what-we-do). In sum, FAIR data sharing can expand the neuroscience workforce and open opportunities for collaboration. This amplification of both expertise and resources increases return on investment in data generation, especially in the longer term.

Finally, while the cases presented in this NeuroView emphasize the feasibility and benefits of data sharing for invasive human electrophysiology, participants must always be appropriately consented and data reused according to their informed choices. Thus, investigators should provide as much transparency as possible in terms of where and how the data will be shared, the relevant benefits and associated risks of sharing, and who will likely have access to data that is shared, consistent with the most up-to-date ethical considerations and recommendations.¹⁰

Conclusions and future directions

Data sharing in neuroscience can have many benefits. While only a tiny fraction of the human neural data generated so far have been shared, this situation is likely to change in the coming years, as the NIH-wide DMS policy increases sharing across all fields. Here, we have profiled successful data-sharing processes for invasive human electrophysiology from three teams in the BRAIN Initiative ROH Consortium. We highlighted three tangible benefits of sharing FAIR data for scientists generating these data. Finally, to further promote FAIR sharing, we provide three recommendations to help maximize these benefits.

ROH consortium:

Ralph Adolphs radolphs@hss.caltech.edu)

Richard A. Andersen (andersen@vis.caltech.edu)

Gordon Baltuch (gb2774@cumc.columbia.edu)

Peter Brunner (brunner@neurotechcenter.org)

Sydney S. Cash (scash@mgh.harvard.edu)

Edward Chang (Edward.Chang@ucsf.edu)

Jennifer L. Collinger (collinger@pitt.edu)

Nathan Crone (ncrone@jhmi.edu)

Evelina Fedorenko (evelina9@mit.edu)

Itzhak Fried (IFried@mednet.ucla.edu)

Josh Gold (jigold@mail.med.upenn.edu)

Jaimie Henderson (henderj@stanford.edu)

Leigh Hochberg (lhochberg@partners.org)

Matthew Howard (mhoward@razi.surgery.uiowa.edu)

Mike Kahana (kahana@psych.upenn.edu)

John Magnotti (John.Magnotti@Pennmedicine.upenn.edu)

Adam Mamelak (mamelaka@cshs.org)

Nader Pouratian (Nader.Pouratian@UTSouthwestern.edu)

Robert Mark Richardson (Mark.Richardson@mgh.harvard.edu)

Ueli Rutishauser (ueli.rutishauser@cshs.org)

Gerwin Schalk (gschalk@neurotechcenter.org)

Charlie Schroeder (schrod@nki.rfmh.org)

Krishna Shenoy (shenoy@stanford.edu)

Sameer A. Sheth (Sameer.Sheth@bcm.edu)

Nanthia Suthana (nanthia@ucla.edu)

Nitin Tandon (nitin.tandon@uth.tmc.edu)

Ziv Williams (zwilliams@mgh.harvard.edu)

Jonathan Wolpaw (wolpaw@neurotechcenter.org)