Human brain atlasing: past, present and future

Abstract

We have recently witnessed an explosion of large-scale initiatives and projects addressing mapping, modeling, simulation and atlasing of the human brain, including the BRAIN Initiative, the Human Brain Project, the Human Connectome Project (HCP), the Big Brain, the Blue Brain Project, the Allen Brain Atlas, the Brainnetome, among others. Besides these large and international initiatives, there are numerous mid-size and small brain atlas-related projects. My contribution to these global efforts has been to create adult human brain atlases in health and disease, and to develop atlas-based applications. For over two decades with my R&D lab I developed 35 brain atlases, licensed to 67 companies and made available in about 100 countries.

This paper has two objectives. First, it provides an overview of the state of the art in brain atlasing. Second, as it is already 20 years from the release of our first brain atlas, I summarise my past and present efforts, share my experience in atlas creation, validation and commercialisation, compare with the state of the art, and propose future directions.

Introduction

As the 1990s were considered the ‘decade of the brain’ and the 21st century the century of the brain, we have witnessed an explosion of initiatives and projects addressing mapping, modeling, simulation and atlasing of the human brain. Several of them are large scale, including the BRAIN Initiative (Brain Research through Advancing Innovate Neurotechnologies),¹ the Human Brain Project to develop a computer model simulating the human brain,² the HCP to map structural and functional connections in the brain,³ the Big Brain to obtain very high resolution neuroimages,⁴ and the Allen Brain Atlas to map gene expression.^5,6

Besides these large initiatives, there are numerous mid-size and small brain atlasing projects, such as Dickie et al.,⁷ Mazziotta et al.⁸ and Toga et al.⁹ Some of them have resulted in popular, commercial brain atlases, including A.D.A.M.,¹⁰ Digital Anatomist,¹¹ Voxel-Man,¹² Primal's Interactive Head and Neck¹³ and Visible Body.¹⁴

My contribution to these brain-related global efforts has been to create adult human brain atlases in health and disease, and to develop atlas-based applications. For over two decades I developed with my R&D lab 35 brain atlases, licensed to 67 companies and institutions, and made available to medical societies, organisations, medical schools and individuals in about 100 countries.¹⁵

In contrast to the ongoing initiatives and projects, my small-scale approach is guided by research, clinical and market perspectives, and it is top down, meaning that the brain is continually parcellated into a higher number of smaller and smaller pieces, and the intermediate results have been released at certain time points for use in the form of atlases without waiting until the whole process will be completed in an unpredictably long time (and uncertain results). In addition, I aim to build a virtual model of the human brain that is as much as possible complete based on the current state of knowledge, as opposed to building specific (limited to certain regions or aspects) population-based atlases. I think that only after creating a very detailed virtual, reference brain of a single specimen based on in-vivo imaging along with incorporated domain knowledge, I will be in a better position to extend it by including anatomical variability followed by functional and pathological variabilities as well as to expand it in time from development to aging.

My brain atlasing work started around 1992 and the first electronic brain atlas developed was released 20 years ago in 1997.¹⁶ It was a long awaited global product foreworded by Professor Jean Talairach, based on four classic brain atlases published earlier in a print form by Thieme.^17–20 The atlases developed by me so far, both commercial products and research prototypes, are grouped into three families, namely, derived from: Thieme's print materials, in-vivo imaging and population (including patient-specific) data, and their taxonomy is presented in Nowinski.¹⁵ Technology-wise, two main generations of the atlases are distinguished: Cerefy (and pre-Cerefy) atlases and the Human Brain in Pieces.

So far, I have created brain atlases for education,^21–23 research^24–26 and clinical applications,²⁷ mainly in neuroradiology,^28–31 neurosurgery^32–34 and neurology.^35–38 Dedicated atlas-based solutions have been proposed and developed (as products or prototypes) for brain scan interpretation,³⁰ brain cancer,²⁹ human brain mapping,^25,26,39 stroke image analysis and prediction,^40,41 psychiatry (schizophrenia)⁴² and stereotactic and functional neurosurgery.^{32–34,43–53}

I provide an overview here of the state of the art in brain atlasing. In addition, as it is already 20 years from the release of our first brain atlas, I summarise our past and present efforts, share our experience in atlas creation, validation and commercialisation, compare our work with the state of the art, and propose future directions.

Past

The initial effort was to build a multiple-atlas database with about 1000 structures and 400 sulcal patterns.⁵⁴ This database was derived from four print Thieme's classic brain atlases with complementary contents, including gross anatomy,¹⁷ deep structures,²⁰ brain connections¹⁸ and sulcal patterns.¹⁹ The original two-dimensional (2D) atlases were digitised and subsequently highly processed, enhanced, extended (including extensions to three-dimensions), labeled, converted to various representations (colour-coded bitmaps, image and geometric contours and polygonal surfaces), and spatially co-registered.⁵⁴ This was a painstaking, tedious and time-consuming effort. For instance, contouring, extending and enhancing the deep structures (in the Schaltenbrand–Wahren atlas)²⁰ took me about 2 years. The construction and features of this database are covered in detail in Nowinski³² and Nowinski et al.⁵⁴

Initially, the brain atlas database was integrated with a surgery planning system running on a workstation.⁵⁴ The system was trial licensed to several surgical companies that subsequently integrated this brain atlas database (or part of it) with their proprietary surgical workstations.³²

My first stand-alone brain atlas product was the Electronic Clinical Brain Atlas. Multiplanar Navigation of the Human Brain released jointly with Johns Hopkins in 1997.¹⁶ Its main application is education and referencing (although it has also been used in stereotactic and functional neurosurgery before the integration of my brain atlas database with surgical workstations). Its development along with the creation of the brain atlas database took about 5 years of work. As the atlas was designed to run on low-cost PC and MAC platforms (in contrast to running previously on an expensive workstation), an automatic label readout was not feasible then due to a long response time, and I placed about 17,000 pre-set (static) labels manually on 1500 atlas images, which took me about one year of work.

As this atlas product was derived from two print brain atlases created by Talairach and Tournoux, I consulted our electronic versions (extended with magnetic resonance imaging (MRI) and Brodmann's areas images) with them. My meetings in Paris with Drs Jean Talairach and Pierre Tournoux were memorable and critical to our atlasing work, being then at its early stage. In 1995, while demonstrating the electronic atlas to them, Dr Talairach requested to present it in French. Fortunately, thanks to my knowledge of the Latin brain nomenclature I managed to explain our work in my terrible French; fortunately, well enough so that Dr Talairach agreed to write the foreword to the atlas. The next meeting was even more memorable. I flew from Baltimore to Paris and the flight was delayed by a few hours. That day Paris was on strike, including taxis. Despite a few hours' delay and a cold winter time both doctors waited patiently for me in front of the Saint Anne Hospital. I was very touched. This time I was better prepared in French. Dr Talairach, then 85 years old, expressed a vivid interest in the atlas and asked difficult questions about both the electronic extended edition and its computer implementation. It was a hard time for me to explain, but it worked, as my explanations were understood by Dr Talairach who subsequently re-communicated them to the much junior Dr Tournoux.

The atlas very soon became a bestseller (700 copies sold in 3 months) acquired by medical schools, surgical companies and medical professionals. At that time, my approach for atlas development was to build ‘atlas by atlas’ and we kept developing subsequent atlases. The next product, Brain Atlas for Functional Imaging. Clinical and Research Applications, was developed jointly with Harvard Medical/MGH and released in 2000.³⁹ It provides a locus-driven mechanism for atlas-assisted analysis of activation loci.²⁶

Then, the Cerefy Atlas of Brain Anatomy. An Introduction to Reading Radiological Scans for Students, Teachers, and Researchers was released in 2002 for education with two modules for exploration and testing.²¹ The testing module provides automatic and random atlas-derived generation of questions to test the location and naming of cerebral structures,²² so it can be exploited for self and classroom testing. A Chinese, web-based version of this atlas was also developed.⁵⁵

The subsequent atlas, the Cerefy Clinical Brain Atlas on CD-ROM, published in 2004 provides automatic labeling of structures and functional description of regions.⁵⁶ I could save a substantial time otherwise required for a manual placement of the labels on the atlas images, but a huge number of anatomical and functional combinations to be carefully checked also required approximately a one man-year effort.

The next atlas, the Cerefy Clinical Brain Atlas. Extended Edition with Surgery Planning and Intraoperative Support, released in 2005⁵³ contains, additionally to surgical planning tools, a population-based functional atlas.^48,49

The strategy for the development of the first generation of the brain atlas products was to devise multiple products and to demonstrate the usefulness of atlas-based solutions in education, research and clinical applications. Consequently, the atlases have been created for anatomical referencing,¹⁶ education,²¹ human brain mapping³⁹ and stereotactic and functional neurosurgery.⁵³ All the atlases are stereotactic (with the Cartesian coordinate system), and each has been equipped with dedicated tools supporting its specific application. Namely, the atlas for referencing provides automatic labeling; the atlas for education enables an automatic testing;²² the atlas for brain mapping performs an analysis of activation loci on the user's structural and functional scans;^25,26 and the atlas for neurosurgery combines anatomical and population-based functional atlases,⁵¹ allowing the user to load a patient's image and plan a trajectory along with the target.

Despite a scientific, clinical and commercial success (until 2000, major surgical companies licensed our atlases), the limitations of the atlases were becoming more and more evident (besides a well-known sparseness of the component atlases). The brain atlas database, despite the integration of multiple atlases with complementary contents that were mutually spatially co-registered, was not extendable and an accurate match with the patient-specific data was not feasible. Moreover, having the atlases in electronic form enabled us to study their spatial consistency and quality. For instance, in the gross anatomy (Talairach–Tournoux) atlas,¹⁷ the spatial consistency among its orthogonal (axial, coronal and sagittal) sections is only 27.4%.⁵⁷ In the deep structure (Schaltenbrand–Wahren) atlas,²⁰ the major stereotactic targets were reconstructed in three dimensions from each of their axial, coronal and sagittal orientations (2D microseries). The three-dimensional (3D) models differ significantly in location, size, shape and inclusion rate among the orientations. The reconstructed subthalamic nucleus,⁵⁸ globus pallidus internus⁵⁹ and ventrointermediate nucleus of the thalamus⁶⁰ demonstrate a significant 3D inaccuracy within each orientation and across them.

Enabling atlas extendibility and feasibility of warping were the main reasons to start constructing a new generation of brain atlases. I attempted to obtain a suitable material for a new brain atlas database from several sources and groups (including the University of Colorado (the developer of the Visible Human Project) to get a visible human-type of high-resolution head data, Chinese Visible Human⁶¹ and the material from Drs Kretschmann and Weinrich),⁶² but for various reasons these projects did not materialise. A permanently available brain specimen was needed for future acquisitions and extensions of the virtual brain.

Therefore, I decided to build the new brain atlas database from multi-modal in-vivo imaging of a single (and permanently present) specimen to enable extendibility. This new database has been created gradually over a decade. Initially, the gross anatomy has been built⁶³ followed by the construction of the intracranial vasculature (first from 3 T⁶⁴ followed by 3 and 7 T together),⁶⁵ white matter tracts,⁶⁶ cranial nerves,⁶⁷ head muscles and glands,⁶⁸ skull⁶⁹ and extracranial vasculature.⁷⁰

The first atlas based on the preliminary version of this new database was the Cerefy Atlas of Cerebral Vasculature released in 2009.⁷¹ However, a major breakthrough was a prototype entitled ‘The human brain in 1,002 pieces’, which received Magna cum Laude and Excellence in Design awards from the Radiological Society of North America (RSNA) 2009.⁷² It was a starting point for creating a new generation of brain atlases, a family of the Human Brain in 1492/1969/2953 pieces consisting of five atlases (there were two editions of each of the 1492 and 1969 atlases).^73–77

Technology-wise, there are two generations of the brain atlases: (a) Cerefy (and pre-Cerefy) atlases (see Figure 1; the atlases not released by Thieme are not included) and (b) atlases of the Human Brain in Pieces family (see Figures 2 and 3 illustrating progress in content, functionality and user interface). The Cerefy brain atlas database employed in the first generation atlases contains a given (and consequently limited) number of 2D static images and some derived 3D models. Creation of the Cerefy brain atlas database involved the compilation and scanning of printed image and textual materials; segmentation (contouring or colour coding) and labeling (naming) of all structures on all the images; checking database images along with correcting, enhancing and extending them; constructing 3D versions of the component atlases; developing various representations in two and three dimensions; and spatial co-registration of all the component atlases. The brain atlas database of the second generation atlases comprises a truly 3D de/composable and extendable cerebral model. Its creation involved the acquisition of multiple multi-modal in-vivo scans, scan segmentation, 3D surface modeling, 3D model simplification (decimation or compressing), 3D surface editing, 3D model colour-coding, 3D object naming and placing the resulting virtual brain in a stereotactic coordinate system. Consequently, a first generation atlas provides a given number of 2D static images. A second generation atlas allows the user to generate an unlimited number of user-defined images (including context-dependent ones) by assembling or disassembling any 3D scene (based on a paradigm from ‘blocks to brain’) and capturing it as a high-resolution image or movie (employing publically available tools). The transition from the first to the second generation of the brain atlases required not only the creation of a new brain atlas database (both in terms of content and dimensionality), but also the development of suitable dedicated tools for its construction as well as a different development platform (an authoring tool Director to develop the first generation versus a programming language C+ + employed in the second generation) and a new atlas browser with a novel user interface. Taking into account the need for an ever-growing content of the brain atlas database, the main limitation of the first generation was (or would be if extended further) performance (speed). The main problem faced with the second generation is architecture and memory management (extensibility).

Figure 1.

Development of the first generation brain atlases released by Thieme: (a) first two-dimensional (2D) multi-atlas¹⁶ for referencing (with static labels and extended with magnetic resonance imaging (MRI) and Brodmann's areas images); (b) second 2D atlas for functional imaging³⁹ providing atlas-assisted analysis of activation loci (MRI and functional MRI scans are loaded, the brain atlas superimposed on the scans (with user controllable blending), the activation loci are labeled, and their coordinates provided); (c) third 2D atlas for neuroeducation²¹ (working in the exploration and testing modes); (d) fourth 2D multi-atlas for referencing⁵⁶ (with automatic labeling and functional description); the fifth atlas (not shown here) is its extension for surgery planning⁵³ with a population-based functional atlas; and (e) sixth three-dimensional (3D) atlas of cerebral vasculature;⁷¹ the first 3D atlas based on the new brain atlas database (also with a new user interface with the 3D view in contrast to mono or tri-planner 2D views in the previous atlases).

Figure 2.

Development of the second generation brain atlases: (a) the Human Brain in 1492 Pieces⁷³ (with structure, vasculature, and tracts); (b) the Human Brain in 1969 Pieces, second edition⁷⁶ (with structure, vasculature, tracts, cranial nerves, systems, head muscles and glands); the next atlas (in 2953 pieces)⁷⁷ with 17 tissue classes is shown in Figure 3; (c) mobile version⁷⁹ (on iPad); note a simplified user interface; (d) 3D Atlas of Neurologic Disorders;³⁸ a lesion (a white ball) is marked and labeled with pathology and the resulting signs, symptoms and syndromes, and shown along with the surrounding neuroanatomy (also labeled).

Figure 3.

The 2953 atlas of the brain, head and neck along with the user interface including the three-dimensional (3D) view (centrally), anatomical indices (right) and controls (top and left). A 3D scene assembled in the 3D view is being synchronised with the horizontally scrollable top control panels and the vertically scrollable anatomical indices, such that each single tissue class selected for display in the 3D view has its own corresponding anatomical index and top panel. The tissue classes are selectable from the matrix in the top-right panel, the left/right brain and head sides and groups from the top control panels, and individual components from the anatomical indices.

Present

Brain atlas development has been a continuous process.⁷⁸ Here I consider the recent atlas work including the Human Brain, Head and Neck in 2953 Pieces (in brief 2953);⁷⁷ 3D Atlas of Neurologic Disorders;³⁸ and their mobile editions,^79–81 see Figure 2.

The 2953 atlas is 3D, interactive, detailed, accurate, realistic, high resolution, fully parcellated, completely labeled, spatially consistent, stereotactic, user friendly, extendable (scalable), composable, dissectible, explorable and modular. This atlas content is arranged into 17 tissue classes including the central nervous system, deep gray matter structures, ventricles, white matter, white matter tracts, intracranial arterial system, intracranial venous system, extracranial arteries, extracranial veins, head muscles, glands, skull, skin, neck, visual system and auditory system, see Figure 3. The 2953 atlas, thanks to the permission and generosity of our main publisher Thieme Stuttgart–New York, is freely downloadable from http://www.thieme.com/nowinski/ or www.WieslawNowinski.com/FreeBrainAtlas.

The 3D Atlas of Neurologic Disorders³⁸ has been created from three component atlases of neurological disorders, namely, that of regional anatomy,³⁷ cranial nerves³⁶ and cerebral vasculature.³⁵ The atlas provides disorder–localisation relationships and tools for exploration. It correlates cerebral pathology with the underlying neuroanatomy and the occurring neurological deficits. The atlas presents various simulated cerebral pathological situations (lesions) in 3D that are labeled with the resulting disorders, along with the surrounding, highly parcellated neuroanatomy. Disorders are linked to textbook materials (published by Thieme) and they are described in terms of the resultant signs, symptoms and/or syndromes.

The mobile editions of the anatomical and disorders atlases are also developed (see Figure 2) based on the same principles as their desktop versions; however, with simplified contents and user interfaces.

Current impact

My brain-related R&D work has its scientific, clinical, innovative and commercial (meaning user acceptance) merits. Its impact can be considered in four areas: creation of brain databases and atlases, development of atlas-based applications, brain discoveries and innovative neurotechnology development. The brain atlases created have been delivered on multiple platforms: desktop (Windows and MAC), mobile (iPad/iPhone and Android), web-based and stand-alone libraries.

Scientifically, our work resulted in about 200 publications, including journal and conference papers, book chapters and clinical abstracts (note that the abstracts presented at clinical meetings, including RSNA, European Congress of Radiology (ECR) and American Society of Neuroradiology (ASNR), required the development of working prototypes; in total, over 50 working prototypes have been developed and presented at radiological, surgical and educational meetings).

This work has led to several discoveries. By studying relationships of the gray to white matters in brain sections, I have observed that a normal brain can be approximately characterised by a few landmarks only.⁸² This approach has potential applications in brain compression, comparison, morphometry and normalisation. It also enables quantification of healthy aging and the detection and quantification of pathologies.⁸² Consequently, this opens new avenues in computer-aided neuroradiology ranging from screening to searching of large brain databases.

I have discovered a functional laterality of the subthalamic nucleus, which is the key target in the surgical treatment of Parkinson's disease and some other neurological disorders.⁵²

I have also found that the human cerebrovasculature, despite its enormous complexity and variability, is characterised by four simple approximate proportions 1:1, 1:2, 1:3, 1:4, and their combinations 1:1:1 and 1:2:4.⁸³

Clinically, I have introduced our electronic brain atlases along with the methods of their employment to stereotactic and functional neurosurgery,^32,43,44 mainly for deep brain stimulation to treat Parkinson's disease and other neurological disorders. My atlases have been installed in over 1550 surgical workstations of 13 surgical companies (including the leading ones: Medtronic, Brainlab and Elekta). Hundreds of thousands of patients have been treated by employing my atlases, resulting in better outcomes and improved quality of life.

From the innovation standpoint, several new methods for neuroimage processing and analysis have been developed and patented (in total, I have 121 patent applications filed and granted as listed in a patent database (http://www.freepatentsonline.com); 15 patents are already granted in the USA and eight in the EU), and incorporated into the brain atlas products and working prototypes. The mathematical and computational methods devised by me along with my team for image segmentation, geometric modeling, physical modeling, atlas-to-scan registration, visualisation, interaction and virtual reality employed in building the brain atlases and developing atlas-based applications have received an overview in the accompanying paper in this issue.⁸⁴

Pioneering brain atlasing

I have been pioneering brain atlasing work^a in neuroradiology, neuroeducation, brain mapping, neurosurgery and neurology by:

• proposing novel atlas-assisted solutions in neuroradiology for automatic structure identification, localisation, delineation, labeling, quantification, reporting and communication (both doctor-to-doctor and doctor-to-patient) illustrated in nine various situations;³¹ more advanced atlas usages have been devised for pathology detection,⁸⁵ lesion quantification,⁴⁰ segmentation and labeling of pathological images,²⁹ automatic region-of-interest generation in statistical analysis,⁴² data aggregation⁴¹ and dealing with data explosion;³¹

• developing an atlas-based system for automatic interpretation of normal brain scans in less than 5 seconds;³⁰

• devising a method for building a probabilistic stroke atlas and creating such an atlas for stroke prediction;⁴¹

• developing atlas-based computer-aided design (CAD) stroke systems for diagnosis, treatment and prediction;^85,86,87

• creating a 3D interactive and stereotactic atlas of the brain, head and neck with 3000 components;⁷⁷

• employing the brain atlas in medical education for automatic assessment of students;^21,22

• creating a 3D interactive atlas of cerebral arterial variants;⁸⁸

• creating a digital atlas of blood supply territories correlated with an atlas of anatomy;⁴⁰

• providing an atlas-based solution for localisation analysis in human brain mapping (with the atlas available directly);^25,26,39

• providing an atlas-aided globally accepted solution and introducing our atlases to the marketplace in stereotactic and functional neurosurgery (the standard established);^32,43,44

• devising a method for constructing a probabilistic functional atlas (PFA) from neuroimaging and neuroelectrophysiology,⁴⁸ and employing it to build the PFAs of major stereotactic target structures;^49,50

• developing an Internet portal for stereotactic and functional neurosurgery, shifting the paradigm in atlas building from manufacturer-centric (dependent) to neurosurgical community-centric;³⁴

• proposing an operating room of the future enhanced by an atlas;⁸⁹

• proposing a concept of atlas-based do-it-yourself neurosurgery;³²

• creating a 3D atlas of neurological disorders, bridging neuroanatomy, neuroradiology and neurology.^35–38

Discussion

The creation of electronic human body atlases and the development of atlas-assisted applications is a long-term, complicated, multidisciplinary and tedious process. Here I share our R&D lab experience from research to development to validation to commercialisation.

The major challenge I faced was to build a bridge between medicine and science (and art too). Particularly, when dealing with brain atlases and their clinical applications, I had to build bridges between neuroanatomy, neuroradiology, neurosurgery, neurology, stroke and computer science. The major components of this process are mathematical methods as well as computational methods and tools, addressed in an accompanying paper in this issue.⁸⁴ Several other components have already been addressed earlier, including the principles of atlas design,¹⁵ atlas content,¹⁵ functionality,¹⁵ interaction,⁹⁰ software architecture,⁶³ aesthetic design⁶³ and validation of individual atlases.^63–70

The lessons learnt from the experience in commercialisation of our brain atlases are summarised and discussed in Nowinski,⁹¹ so this experience can guide other research groups in their translational efforts from bench to bedside. That work⁹¹ also summarised my first 10-year brain atlas efforts, from the release of the first atlas resulting then in 17 atlas products developed, 4000 CD ROMs distributed by Thieme, nine awards from clinical societies and 13 patents filed and three granted. The comparison of the 20-year to 10-year outcomes demonstrates the dynamics of our work: a jump in intellectual property generation is especially evident.

Numerous brain atlases have been created in print and electronic form, and their reviews are, e.g. in Dickie et al.,⁷ Nowinski et al.,⁵⁴ Nowinski⁹¹ and Thompson et al.;⁹² in particular, the brain atlases for stereotactic and functional neurosurgery are reviewed in Nowinski.³² The brain atlases can be classified and compared from various perspectives; for instance, source material (e.g. histology, radiologic scans, visible human data, neuroelectrophysiology, multi-modal), population of source material (deterministic⁹³ vs. probabilistic atlases),⁸ application (education, research, clinical), content, functionality, dimensionality (2D, 3D, nD), medium (print versus electronic), platform (web-based, mobile, Windows, MAC, stand-alone plug-in library) and cost (e.g. mobile vs. high-end virtual reality solutions). The atlases can capture the brain in health⁹³ or disease, being disease-specific⁹⁴ or general covering numerous disorders.³⁸ An atlas can comprise the whole brain⁹³ or any part of it, e.g. white matter.⁹⁵ cerebral arteries,⁹⁶ insula⁹⁷ or cerebellal nuclei.⁹⁸ Note that our second generation atlases are composable, so that any part of the brain can be assembled by the user along with the surrounding structures of interest for exploration and analysis. Some of the atlases have been extended beyond the brain to the head and neck, such as Primal's Interactive Head and Neck¹³ and even to the whole body, such as A.D.A.M. Interactive Anatomy¹⁰ and Visible Body.¹⁴

Various atlases have been developed from MRI for a single subject. For instance, the SPL Brain Atlas⁹⁹ and Voxel-Man¹² have been derived from 1.5 T scans, and the atlas of the basal ganglia¹⁰⁰ also exploits 1.5 T imaging along with histology. A systematic review of whole brain magnetic resonance-derived atlases is given in Dickie et al.⁷ Several atlases are based on histology,^100,101 including a few products in my first generation atlas family. Some other atlases, such as A.D.A.M., Primal's Interactive Head and Neck and Visible Body, have been created by medical visualisation professionals as artistic representations of the body rather than the real 3D models derived from imaging. A few atlases, e.g. Voxel-Man, are multilingual, whereas my atlases are in English (excluding two atlases in Chinese and a Polish edition developed as a prototype).

The development of neurosurgical atlases is an active area of research. In Nowinski,¹⁰² I have proposed an ideal atlas for stereotactic and functional neurosurgery with four major components: brain models, knowledge database, tools and clinical results. Besides my neurosurgical atlases discussed above, several other groups have developed atlases for neurosurgery.^99–101 Moreover, dedicated atlases or databases have been built from neuroelectrophysiology,¹⁰³ similar to my work on a PFA for deep brain stimulation.⁵² However, to the best of my knowledge (besides our atlases) none of these atlases has ever reached the global neurosurgery market.

We recently witnessed a dynamic development of population-based atlases^{41,96–98,104–111} for the whole brain^105,106 and specific brain regions, like cortical structures,¹⁰⁸ brainstem,¹⁰⁹ cerebellum¹¹⁰ and others.^96–98 More powerful atlases have been created in terms of population,¹⁰⁵ specimen age range span¹⁰⁶ and age appropriateness.¹⁰⁷ For instance, the atlas of Chinese adults contains 2020 specimens whose ages span from 20 to 75 years at 5-year intervals.¹⁰⁵ The multi-atlas¹⁰⁶ is an inventory of 90 atlases, ranging from 4 to 82 years of age.

None of the most popular, commercial brain atlases, including A.D.A.M.,¹⁰ Digital Anatomist,¹¹ Voxel-Man,¹² Primal's Interactive Head and Neck¹³ and Visible Body,¹⁴ has ever been used beyond education. In fact, to the best of my knowledge, none of the existing brain atlases have been accepted clinically and incorporated into commercially available workstations for daily clinical practice, as the majority of the brain atlases and databases cited in the literature are research prototypes. In contrast to the existing atlases, my atlases of the normal brain, head and neck are: highly parcellated with 3000 3D components; of high accuracy of structure fitting (0.1–0.2 mm in Nowinski et al.);⁷³ extendable; created from multiple modalities and in-vivo 3 T, 7 T and high-resolution computed tomography scans; and equipped with dedicated tools designed and developed for atlas construction, editing and extension. Moreover, I have proposed atlas-based solutions ranging from education and research to clinical applications for neurosurgery, neuroradiology and neurology. These atlases are available on four platforms: desktop (Windows and MAC), mobile (iPad, iPhone and Android), web-based, and stand-alone libraries.

Besides its educational value, the brain atlas plays multiple roles in processing and analysis of medical images. Initially, the atlas has assisted in performing typical atlas-related tasks, such as structure identification, localisation, segmentation and labeling as well as being a source of referencing. In particular, probabilistic brain atlases and multi-atlas approaches are widely used for segmentation and labeling as discussed in more detail in Nowinski.⁸⁴ With further atlas development, I have proposed more advanced atlas applications, including pathology detection,⁸⁵ lesion quantification,⁴⁰ segmentation and labeling of pathological images,²⁹ aggregation of image and clinical data,⁴¹ automatic region-of-interest generation on scans for statistical analysis,⁴² dealing with data explosion,³¹ radiology reporting³¹ and communication (for both doctor to doctor and doctor to patient).³¹

I have introduced electronic brain atlases to stereotactic and functional neurosurgery practice along with novel methods for their usage.^{32,43–45,52,53} I have also proposed new methods of atlas construction and use in this field.^32–34,47 My atlases are employed globally by 13 surgical companies in more than 1550 neurosurgical workstations. The atlases assist in preoperative planning, provide intra-operative support and facilitate postoperative evaluation.⁴⁴ I have also formulated an atlas-assisted surgical procedure supporting the future operating room⁸⁹ and proposed an atlas-guided do-it-yourself neurosurgery for patients.³²

International and big brain initiatives, such as the BRAIN Initiative,¹ the Human Brain Project,² the HCP,³ the Big Brain⁴ and the Allen Brain Atlas,^5,6 empowered by dramatic developments in diagnostic imaging, electronics and computing, propel the rapid development of not only probabilistic and multi-brain atlases but also brain repositories, biobanks, databases, libraries of spatial maps and atlases, and imaging studies.^106,112 These resources, with a recent move to fully open databases, are accessible for large studies of brain structure and function, and include BALSA,¹¹³ BRAINS,¹¹⁴ COBRA,¹¹⁵ ConnectomDB¹¹⁶ and NeuroVault.org.¹¹⁷

Recently, significant resources are being dedicated worldwide for large-scale connectome projects. These projects aim to study the structural and functional connections of the human brain at three complementary levels: macro, meso and micro. The key efforts include the HCP in the USA,³ the CONNECT project in Europe¹¹⁸ and Brainnetome project in China.¹¹⁹ Although the current activities concentrate on methodology and technology developments, these projects have already resulted in several brain atlases and the efforts will potentially lead to the creation of new and better atlases reflecting neurobiological reality. It should be noted that because of methodological limitations, it cannot be assumed that the human connectome maps produced currently are conclusive. Moreover, the patterns of our anatomical and functional connections are not permanent, but rather constantly change in response to experience.

The HCP acquires and analyses connectivity data along with other neuroimaging, behavioural and genetic data from a population of 1200 healthy adults in order to enable detailed comparisons between brain circuits, behaviour and genetics.³ Its objectives are to accelerate advances in key technologies and apply them to a large population of healthy adults to map macroscopic brain circuits and their relationship to behaviour. The HCP serves as a key resource for the neuroscience research community, enabling discoveries of how the brain is wired and how it functions. The BALSA library¹¹³ is well suited for extensively analysed HCP data, while the unprocessed and minimally preprocessed HCP data from 1200 individual subjects are better handled by the ConnectomeDB database.¹¹⁶ Some population-based tractographic atlases have been developed from the HCP data including an atlas of the brainstem from 488 subjects¹⁰⁹ and an atlas of the cerebellum from 90 subjects.¹¹¹

The CONNECT project aims to combine tractography and micro-structural measures to obtain a better estimate of the connectome. The project consists of four parts: development of methods, enhancement of tractography and connectivity, validation and application. Three atlases have already been produced from the CONNECT project: the atlas of brain connectivity, the atlas of brain micro-structure with axonal density and myelin, and the atlas of mean axonal diameter.¹¹⁸

The Brainnetome project aims to understand the brain and its disorders, develop methods of brain network analysis and techniques on different scales, and create the brainnetome atlas.¹¹⁹

Besides a wide acceptance by the research community, mainly because of publicly available atlases, repositories and tools available and useful in neuroimaging, neuroscience and neuroinformatics, I am not aware of any applications in neuroradiology and medical education yet.

The metabolic reference brain atlas is not yet available. However, recent advancements in whole brain quantitative mapping of metabolites at 3 T¹²⁰ and ultra-high resolution brain metabolite mapping at 7 T¹²¹ allow the generation of metabolite maps and metabolite ratio maps. These maps form a foundation of a metabolic database of a future reference metabolic atlas.

Future

The big brain initiatives are long term and their goals and directions are well defined, as discussed above. As we celebrate our 20th anniversary, I also ponder on what could be achievable in the next 20 years in our brain atlas niche, provided, of course, that the adequate resources will be available. Obviously, a continuous enhancement has always been embedded into my efforts⁷⁸ and overcoming the limitations of the already created atlases has been a major driving force behind atlas development (fueled, of course, by the dreams). Summarising my long-term efforts (that are impactful in terms of products, licences, global penetration, publications, awards and related patents) the question is what might be ‘wrong’ (meaning to be corrected and enhanced) this time with the current brain atlas generation? I think that the issue shall be considered in terms of whether it is possible to propose a more disruptive solution. A solution that would be multi-factor in terms of content, display, interaction, navigation, variability, applications and audience, while addressing the current shortcomings including a limited content of the image material based on a single brain specimen and restricted to non-invasive acquisitions. A solution with a transition from structure to function to dysfunction, from static to dynamic functioning of the brain, from 2D/3D to nD, from geometric to physical modeling of the brain,^122,123 and from simplified modeling of lesioned site to dynamic modeling of pathological processes.

A new solution shall integrate the latest technologies, particularly true 3D displays, game engines and virtual reality software and hardware. The current display shall shift from standard to autostereoscopic to holographic to touchable holographic displays, especially in view of the fact that almost 50% of the human brain capability deals with processing of visual information.¹²⁴ Similarly, the virtual reality shift will be from conventional (with the mouse and keyboard) interaction to gestures to natural two-hand and to whole-body interaction with voice control and tactile feedback, supporting applications for dexterous and immersive virtual reality on multiple platforms ranging from head-mounted displays to CAVE-like environments.¹²⁵ The use of powerful game development platforms (engines), such as Unity (https://unity3d.com) and Unreal (www.unrealengine.com) will also be considered, especially in atlas-based neuroeducation.

Hence, the next big step I propose shall be from the development of brain atlases to creating a multi-level molecular, cellular, anatomical, physiological and behavioural brain atlas platform extended with the variability and time over the life span (see Figure 4).

Figure 4.

Road map of brain atlas development along three directions: content, variability and time (life span). The content is represented by a hierarchical structure spanning from anatomy up to behaviour and down to molecules.

The key layer in this hierarchical content structure is anatomy (which has been and shall continuously be extended), as about 50% of knowledge in neuroradiology, neurosurgery and neurology is neuroanatomy.

Function description has already been incorporated in some previous atlases. In contrast to that, the next generation brain atlas platform shall provide visualisations of the working brain from flow of fluids (blood and cerebrospinal fluid) to physiological processes to flow of information to performing perceptual, motoric, cognitive and emotional tasks to metabolism.

Dysfunction has already been implemented in the 3D Atlas of Neurologic Disorders.³⁸ What I plan to do next, among other things, is to label with pathology not only some specific locations, but the entire brain based on the material published by Thieme.

Relationships between functional patterns and mental disorders as well as biochemistry aberrations associated with behavioural disorder are among pioneering neuroscience goals.¹²⁶ As mentioned above, one of the objectives of the HCP is to study the relationship between brain circuits and behavior.³ Moreover, ultra-high magnetic resonance fields between 14 and 20 T, much higher than currently available, can facilitate studying the patterns of neuronal connectivity correlated to human behavioural and psychiatric problems.¹²⁶ In addition, mapping of brain–behaviour relationships across the life span is a research agenda for the next decade.¹²⁷ The results of these efforts could potentially be incorporated in the future into the planned brain atlas platform.

System-wise, the most advanced atlas developed by me so far (the 2953 atlas)^77,93 contains the visual and auditory systems as separate modules, the olfactory system and components of the vestibular, oculomotor, limbic and gustatory systems. This content shall be further enhanced and parcellated, and more systems included.

A pioneering work on neocortical microcircuits is pursued by Markram et al.¹²⁸ within the Blue Brain Project (http://bluebrain.epfl.ch/). The work demonstrates the feasibility of the reconstruction of an integrated view of the structure and function of neocortical microcircuitry, including layer heights, neuronal densities, ratios of excitatory to inhibitory neurons, morphological and electro-morphological composition, electrophysiology, synaptic anatomy and physiology.¹²⁸

Realistic modeling of single neurons is a rapidly advancing field, reviewed e.g. in Almog and Korngreen.¹²⁹ There are already several tools developed for neuron modeling, including NEURON, GENESIS and MOOSE.¹³⁰

As stated earlier, my strategy has been to create a very detailed deterministic, virtual, reference brain of a single adult specimen first. This effort will be followed by: (a) the development of a variety of probabilistic atlases capturing anatomical, functional and pathological variabilities; and (b) the temporal atlas expansion across the life span from development to maturation to aging.

Within connectome projects new studies have been designed to capture age-related changes in cognition and behaviour, including the HCP Life Span with participants in six age groups from 4 to 75 years and the Chinese Color Nest Project (CCNP) Life Span with 1200 participants (scanned on 3 T and 7 T).¹²⁷ The temporal development of our brain atlas platform may potentially benefit from the results of these studies.

I have already created some population-based atlases with variability in anatomy,⁸⁸ function⁴⁸ and disorders (pathology).⁴¹ Both deterministic and population-based atlases have their own merits, so their combination is beneficial (as illustrated, for instance in Nowinski et al.,⁵¹ in which a highly anatomically parcellated atlas of the deep structures is integrated with a low parcellation, high spatial resolution and population-based atlas). Therefore, the proposed brain atlas platform can greatly benefit from the ongoing large-scale projects and the brain repositories being developed.

Reciprocally, this is quite natural to assume that the 2953 atlas and any future reference, highly parcellated brain atlases will be potentially useful in large-scale brain projects, such as the BRAIN Initiative,¹ the Human Brain Project² and the HCP.³ Then, the reference atlas can serve as a: (a) potential ‘Wikipedia’ for the brain community; (b) reference for huge amounts of data that will be generated; (c) framework for integration and interpretation of the results; (d) vehicle to present and disseminate the discoveries from science to medicine to public; and (e) training tool reducing a difficulty barrier (by making the virtual brain model easy and beautiful).

The numerous challenges I expect to be faced, among many others, are how to:

• handle a growing content complexity (resulting from both continual parcellation and integration of materials from other sources) and to preserve the integrity of the cerebral model being constructed based on the ‘from blocks to brain’ paradigm, while providing a smooth and easy navigation;

• provide a smooth transition between layers with a vast, diversified content;

• devise efficient modeling of structure and function ensuring both realism and interactivity with the right balance between clarity and complexity;

• model consistently the brain over the life span from various sources including acquisitions from multiple specimens and data from public repositories of the ongoing large-scale initiatives;

• design a variety of friendly user interfaces and environments for various computing platforms supporting a smooth and easy navigation through a dramatically growing number of objects;

• design dexterous and immersive virtual reality environments with voice control, tactile feedback and autostereoscopic and holographic displays;

• achieve a better integration of the latest technologies including game engines and display devices, while developing atlases for a wider range of applications, a broader audience (including multilingual versions), and a wider spectrum of computing platforms;

• develop new medical education, research and clinical applications based on the proposed brain atlas platform.

Summary

We have recently witnessed an explosion of large-scale, mid-size and small brain atlas initiatives and projects. The large-scale initiatives, such as the BRAIN Initiative, the Human Brain Project, the HCP, the Big Brain, the Blue Brain Project, the Allen Brain Atlas, the Brainnetome and the Chinese Color Nest Project, have resulted in the development of new methods and technologies, brain discoveries and to a lesser extent in the creation of brain atlases, predominantly as research prototypes.

Among the ongoing large-scale initiatives and numerous other projects, I have identified my own brain atlas niche differing in strategy, atlas content, developed applications and market acceptance. The first 20 (or rather 25) years of our R&D lab activities resulted in the creation of 35 brain atlases for education, research and clinical applications being distributed in about 100 countries.

Here I share my 25-year experience in brain atlas research, development and commercialisation. As described above, the process of atlas creation, validation and commercialisation is long-term, multidisciplinary, meticulous and tedious, even for a relatively simple 2D brain database. Even having an award-winning prototype developed, its refinement, validation and testing of all possible content and functional combinations (if possible at all) on multiple platforms is a very painstaking, time-consuming and costly effort. This may be one of the reasons why only a few brain atlas products from a research lab reach the global marketplace, despite numerous brain atlas prototypes being developed. Moreover, to the best of my knowledge, my atlases are the only ones being clinically applied in commercial systems worldwide.

I also present a preliminary road map projecting what optimistically can be expected in the next 20 years. The next brain atlas generation shall be disruptive by forming a multi-level platform with three directions: content spanning from molecules to behaviour; variability covering structure, function and disorders; and time across the life span.

The time spent on brain atlas development has been the most exciting and challenging period in my professional life so far, in accordance with my research motto: ‘If a clinician helps thousands of patients and a researcher could help thousands of clinicians, then doing medical research is the most rewarding job’.⁹¹ This effort can be summarised in brief as moving from dreams to globally distributed and recognised brain atlas products. In emotional terms, this move is from the pride and joy of the newly developed atlas to the shame and embarrassment about its previous edition. Nevertheless, as stated in my other quote ‘I am doubly happy that both my mind and brain serve humanity’.

Funding

The brain atlas work was funded by BMRC/ASTAR and ETPL/ASTAR, Singapore.

Conflict of interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.