The interactive presentation of 3D information obtained from reconstructed datasets and 3D placement of single histological sections with the 3D portable document format

Abstract

Interpretation of the results of anatomical and embryological studies relies heavily on proper visualization of complex morphogenetic processes and patterns of gene expression in a three-dimensional (3D) context. However, reconstruction of complete 3D datasets is time consuming and often researchers study only a few sections. To help in understanding the resulting 2D data we developed a program (TRACTS) that places such arbitrary histological sections into a high-resolution 3D model of the developing heart. The program places sections correctly, robustly and as precisely as the best of the fits achieved by five morphology experts. Dissemination of 3D data is severely hampered by the 2D medium of print publication. Many insights gained from studying the 3D object are very hard to convey using 2D images and are consequently lost or cannot be verified independently. It is possible to embed 3D objects into a pdf document, which is a format widely used for the distribution of scientific papers. Using the freeware program Adobe Reader to interact with these 3D objects is reasonably straightforward; creating such objects is not. We have developed a protocol that describes, step by step, how 3D objects can be embedded into a pdf document. Both the use of TRACTS and the inclusion of 3D objects in pdf documents can help in the interpretation of 2D and 3D data, and will thus optimize communication on morphological issues in developmental biology.

INTRODUCTION

Basic questions in developmental biology concern the mechanisms of morphogenesis and differentiation. The results of sophisticated, quantitative gene expression profiling methods, such as microarrays and next-generation sequencing, have to be verified by visualizing gene expression in the three-dimensional (3D) context of the developing organs and tissues. In cardiac embryology we have demonstrated that modern 3D reconstruction techniques to visualize morphological changes (Soufan et al., 2003) help in the identification of cardiac compartments on the basis of gene expression (Hoogaars et al., 2004; Soufan et al., 2004) and in the understanding of the role of morphogenetic parameters, such as cell size and proliferation rate, in cardiac development (Soufan et al., 2006; van den Berg et al., 2009).

Histological methods are still required to study the `where and when' of gene expression. To interpret the results of these methods, molecular biologists have to find their way in the ever-changing landscape of the developing embryo, which is far from trivial as wrongly annotated sections do appear in papers (Anderson et al., 2008). Placing individual sections into a 3D reference model would be of great help to the non-morphologist in need of histological confirmation of a gene expression profile (de Boer et al., 2007). Moreover, it can be argued that the current annotation of `arbitrary' sections is often insufficient to study genetic interactions. In the heart, the wall of a compartment can be composed of several genetically distinct regions (Soufan et al., 2004). Placing sections in a cardiac reference model will help in the exact annotation of such compartments and will thus prevent misunderstandings when such scientific concepts are communicated. Indeed, much of the disagreement in embryology might be caused by such miscommunication, as it is far from easy, even for experts, to form a correct mental image of a dynamically changing 3D structure (Soufan et al., 2004).

Understanding the intricate morphology of developing embryos ideally involves the handling of the 3D structure with ones own hands. 2D images found in the literature can only partially replace the insights gained from interactively working with 3D reconstructions on a computer screen. At most, a supplementary movie clip can be produced in an attempt to convey the insights that the researchers have gained from such 3D analyses. However, the complementation of printed journals with web-based publishing provides the opportunity to make the results of morphological studies available as interactive 3D objects. Special visualization software is no longer required because Adobe Acrobat can be used to place surface-rendered 3D objects into a scientific paper that is distributed as a portable document format (pdf) file (Murienne et al., 2008; Neusser et al., 2009; Ruthensteiner and Hess, 2008) that can be read with the freely available Adobe Reader. We have used such 3D pdfs as supplementary material with publications on heart development (Moorman et al., 2009; Postma et al., 2009; van den Berg et al., 2009; van Wijk et al., 2009) and pelvic anatomy (Wallner et al., 2008; Wallner et al., 2009).

This paper describes a computer application that we have developed to place arbitrary sections of the developing mouse heart into their proper anatomical context using high-resolution 3D reference models. Furthermore, it describes how a 3D reconstruction can be converted into an interactive 3D pdf file.

MATERIALS AND METHODS

The tracing of the anatomical context of tissue sections (TRACTS) into reference models and the creation of a 3D pdf are illustrated with an example from mouse cardiac embryology. Procedures to test and validate the placing of sections are described.

Program development

Availability

The TRACTS program and its source code, the database of reference model sections (see Appendix S2 in the supplementary material) and the reference models are available on request under a general public license. For the 3D pdf protocol and its scripts, see Appendix S1 and associated scripts file in the supplementary material.

Computer and software

The TRACTS application was developed using Matlab version 2009a (The MathWorks, Natick, MA, USA). Programming and all tests were performed on a system with an Intel P9600 2.66 GHz processor (Intel Corporation, Santa Clara, CA, USA) with 4 GB RAM running MS Windows Vista 64-bit Business as the operating system (Microsoft Corporation, Redmond, WA, USA). Amira 5.2 (Visage Imaging, San Diego, CA, USA) was used to create 3D reconstructions from serial sections. For the 3D pdf protocol we used Adobe Acrobat Pro Extended 9.3, Adobe Reader 9.3 and Adobe Illustrator CS4 (Adobe Systems, San Jose, CA, USA); we refer to these programs as Acrobat, Adobe Reader and Illustrator, respectively. The basic and advanced steps of the 3D pdf protocol were performed on a standard PC running MS Windows XP Pro as the operating system.

3D models

Biological material

Embryonic day (E) 10.0, E11.0, E11.5 and E13.5 mice (FVB) were used to generate 3D models of the heart. These were used to develop and test the performance of the program. The hearts were staged using two mouse development atlases (Kaufman, 1994; Theiler, 1972).

High-resolution reference and test models

The reference and test models were created using high-resolution episcopic microscopy (HREM) (Weninger et al., 2006). In short, this technique involves embedding specimens in a fluorescent resin and imaging the block face before sectioning. The resulting image sets were filtered to reduce out-of-plane information. The heart was segmented manually using Amira (Fig. 1A). Segmentation was performed using the Magic Wand tool, manually placing borders where the pericardial wall contacts the heart and at the pericardial reflection. The two E11.5 imaged embryos resulted in datasets of the whole embryo with voxel sizes of ∼3.5×3.5×3 μm; the E10.0 embryos with ∼1.8×1.8×2 μm; and the E13.5 embryos with ∼4×4×4 μm. We used an episcopic imaging technique [see Geyer et al. for available methods (Geyer et al., 2009)] as it enabled imaging at high resolution combined with an absence of distortion, making it an ideal technique to virtually cross-section the resulting dataset in all possible directions without loss of the original morphology and image resolution.

Fig. 1.

Reference models and reference database. (A) Reference models are based on episcopic images of E10.0, E11.5 and E13.5 mouse hearts embedded in resin. From the image stacks, the myocardium (gray) and the cardiac cushions (yellow) were segmented. The reference models are shown in a frontal and a `four-chamber' cross-sectional view. LA, left atrium; RA, right atrium; AVC, atrioventricular canal; LV, left ventricle; RV, right ventricle; IVS, interventricular septum. (B) Activity diagram listing the steps TRACTS takes when building the database of reference images. The algorithm starts at the geometric center of the first slice without applying rotation and tilting (determine first plane coordinates). An important step is obtaining the contour of the myocardium (determine contour) by detection of all myocardium pixels adjacent to the background pixels. Plane coordinates are determined by iteration of the translation, tilting angle and tilting direction (see Appendix S2 in the supplementary material). The complete set of virtual cross-sections is uniformly distributed in the 3D space of the reference model.

Paraffin test models

Two specimens (E11.0 and E11.5) were embedded in paraffin. Serial sections on which in situ hybridization (ISH) (Somi et al., 2006) was performed were used to create 3D reconstructions (Soufan et al., 2003). Every other 10 μm section was stained to visualize the myocardium using a probe to Mlc2a (Myl7) mRNA (E11.0) or cTnI (Tnni3) mRNA (E11.5). The sections were aligned and segmented using Amira. Segmentation was mainly performed by setting an intensity threshold followed by removal of nonspecific staining. The resulting 3D reconstructions had voxel sizes of 0.68×0.68×20 μm and 0.87×0.87×20 μm, respectively.

TRACTS performance tests

The performance of TRACTS was tested using (1) virtual cross-sections in arbitrary directions from a second episcopic dataset; (2) samples of paraffin sections taken from the ISH datasets, which is the type of data used for gene expression visualization; and (3) a comparison with the performance of a panel of morphology experts, who manually fitted a sample of the episcopic input set to the reference model.

Episcopic-episcopic performance

The episcopic test model (Fig. 3A) was aligned in 3D (using translation and rotation) with the E11.5 reference model (Fig. 1A) and a systematic random sample of 105 cross-sections was generated. Of these sections, 95 were deemed relevant; the other ten sections contained only a small part of the heart.

Fig. 3.

Validation of TRACTS. (A) The episcopic test model of stage E11.5 from which 95 virtual sections were taken and placed by TRACTS. (B,C) 3D reconstructions based on paraffin sections of E11.0 (B) and E11.5 (C) mouse embryos. Sections of these reconstructions were used to assess the performance of TRACTS on standard histological sections. (D) Computation time in the episcopic-episcopic test for different feature settings, when the feature is used (+) or not used (–). (E) Box plots showing the normalized fit error of TRACTS, using different feature settings. The red line denotes the median fit error and the blue box shows the interquartile range. Outliers (plus signs) are defined as more than 1.5 times the interquartile width around the interquartile range; the plotted whiskers extend to the most extreme data values that are not an outlier. (F) Box plots comparing the performance of TRACTS with the episcopic dataset and with sections of the two paraffin datasets. There was no significant difference between the performance on the E11.5 paraffin and the episcopic dataset, nor between the two paraffin sets, but there was a significant difference between its performance on the E11.0 paraffin and E11.5 episcopic datasets. NS, not significant; E11.5 paraffin-episcopic, P=0.191; E11.0 paraffin-episcopic, P<0.001; E11.5 paraffin-E11.0 paraffin, P=0.069. (G) Box plots comparing the performance of TRACTS with the result of each morphology expert and, for each section, the best-fit result of any of the experts (`best of experts'). The black plus signs in the TRACTS column indicate sections that were not included in the sample of 24 sections registered by the experts. The performance of TRACTS was significantly better than each individual expert but there was no significant difference between TRACTS and the `best of experts'.

Performance on ISH-stained paraffin sections

Because of tissue deformation and the low resolution perpendicular to the sectioning plane, it was necessary to use original sections of the paraffin models as test sections. The reference model was, therefore, aligned to each of the paraffin models (using translation, rotation and isotropic scaling). Samples of 25 and 21 sections were used from the E11.0 and E11.5 paraffin models, respectively.

Expert test

The morphology experts all have several years of experience in research in morphogenesis and heart development. In this test, the experts could use a simple user interface to interactively find the position of an input section in the 3D reference model. A randomly presented sample of 24 sections was used. This set of images was drawn from the set of cross-sections of the episcopic test model that was used for the episcopic-episcopic test of the program. There was no significant difference (P=0.110) between the fit errors of TRACTS on the subset of the images used for the expert test and the other images in the episcopic-episcopic test.

Normalized fit error

In all performance tests the reference model of an E11.5 mouse heart was used. Test images were generated as cross-sections of the aligned 3D test models. In this way, the approximate position of each cross-section of the test model is known in the reference model, which enables the computation of a fit error. This fit error takes the distance and the angle between the determined and actual cross-section into account and is normalized to values between 0 and 100 (see Appendix S2 in the supplementary material).

Statistics

Box plots show the results of the performance tests. The red line denotes the median fit error and the blue box shows the interquartile range; outliers (indicated by plus signs) are defined as results more than 1.5 times the interquartile range distance from quartile points. Because of the non-normal distribution of the normalized fit error, nonparametric statistics were used to compare the results of the different performance tests. To this end, the Kruskal-Wallis test (n groups) or the Mann-Whitney test (two groups) was used (Conover, 1980).

3D pdf

The full protocol for generating a basic and an advanced 3D pdf is given in the Appendix S1 in the supplementary material. The protocol assumes that the user is already familiar with creating 3D reconstructions from histological sections (Soufan et al., 2003).

The basic protocol is used to generate a simple 3D pdf of a reconstructed 3D object and is customized for the conversion from Amira to Acrobat. The resulting 3D pdf can be enhanced to generate an advanced 3D pdf that can be used more easily and intuitively. These independent enhancements can be tailored to the document or paper at hand. By adding annotations and preset views, the author can better convey the insights obtained from the study of the 3D object. Also, functionality and robustness can be added by applying scripts to buttons and the document itself. This advanced protocol is independent of the original 3D reconstruction program.

RESULTS

The TRACTS program

Basic principles of TRACTS

The basis of the program to `trace the anatomical context of tissue sections' (TRACTS) is a straightforward brute force pixel-based procedure, in which each (2D) input section is compared with a large number of virtual (2D) cross-sections of the (3D) reference model. In the input section, the myocardium needs to be separated from background staining using an intensity threshold or manual segmentation. Using the supplied user interface (Fig. 2), the images can be automatically or manually thresholded and small (nonspecifically stained) islands can be removed. A similarity metric based on the contour of cardiac tissue is used to select the most similar cross-section of the reference model. To avoid repetitive computations, a database was created containing pre-computed cross-sections of the reference model (Fig. 1B and see Appendix S2 in the supplementary material). To reduce misplacement and processing time, basic image features such as section size, tissue density, center of gravity and regional density are used to preselect the cross-sections from the reference model that are to be compared with the input image (de Boer et al., 2007). Details of the similarity metric, the reference model database and the image features are given in Appendix S2 in the supplementary material.

Fig. 2.

User interface of TRACTS. In the import dialog (top), the user can select the stage into which the input section has to be fitted. After import of the image the user can define a region of interest in which an (automatic) intensity threshold is employed for segmentation of the myocardium. Next, islands of an adjustable number of pixels can be automatically removed (middle). The myocardium is then mapped into the reference model of the selected age. The result window (bottom) shows the input image with annotated cardiac compartments and the position of the plane in the reference model. The latter view can be interactively viewed from different directions.

Effectiveness of image features

To test the use of image features in TRACTS, a sample of 95 test images taken from the episcopic test model was fitted (Fig. 3A). The test was performed for six different feature settings of TRACTS: purely brute force; only constraint on size; the combinations of size with center of mass, density or regional density; and all features combined. The normalized fit error was computed for each section in each test (Fig. 3E). Computation time per image is given in Fig. 3D. Performance was significantly improved when only cross-sections that are similar in size were compared. The second largest effect was observed with the regional density feature. Using additional features individually gave only a small further improvement, but each feature did result in a reduction in the time needed to register an input section. The combination of all features did not further reduce the fit error but led to a further reduction of the computation time to only 30 seconds per section.

Some of the sections that were placed at an entirely incorrect position when using only the size constraint were fitted near the correct position when preselection on image features was applied (Fig. 4A). The center of mass feature rejected a section cut through a relatively symmetrical part of the heart; because the dorsal wall of each ventricle is thicker than the ventral wall, the mirrored section that was first found was rejected (Fig. 4A, left; Fig. 4B). The overall tissue density feature served to reject a misplaced section where the high tissue density of the erroneously chosen left ventricle was replaced by the correct right atrium (Fig. 4A, middle). The regional density feature showed its effectiveness by rejecting a solution in which a right atrium was placed on a section through the left ventricular wall (Fig. 4A, right).

Fig. 4.

Effect of image features and interpretation of the fit error. (A) Added image features reduce misplacements. At the top, the test sections used as input images for TRACTS are shown. The middle row shows which reference sections were found using only a selection on heart size. The bottom row shows the reference section recovered when an additional feature is used. The normalized fit error (Fe) is given for each of the (mis)placed images. LA, left atrium; RA, left atrium; LV, left ventricle; RV, right ventricle. OFT, outflow tract. (B) Comparison of the fit errors with their corresponding planes based on the first lane of A. The blue planes are the input section and the red planes are the positions found by TRACTS, using the size constraint (top) or the center of mass constraint (bottom). (C) Examples to help with the interpretation of the normalized fit error. Lanes 1 and 2 show the translation and rotation resulting in a fit error of 1.4, which is close to the 75th percentile; 75% is fitted to positions that deviate less. Lanes 3, 4 and 5 show larger fit errors, although in all cases the correct compartments are recovered.

Interpretation of the normalized fit error

Fig. 4B shows a 3D view of the placement of an input section (shown in Fig. 4A, left) using only the size constraint (top panel) and using the size and center of mass constraint (bottom panel). The fit errors are 24.3 and 1.4, respectively. The latter fit error is close to the 75th percentile (median, 1.12; interquartile range, 0.64-1.54) of the fit errors in the episcopic-episcopic test. This error corresponds to a 57 μm distance or a 13.6° angle between the observed and correct section plane (Fig. 4B, bottom). In this respect, it is noteworthy that the discrete distance and tilting steps used to prepare the reference image database (see Appendix S2 in the supplementary material) cause a maximum fit error of ∼0.37. Fig. 4C shows examples of the locations of input and recovered sections for different fit errors. In column one, a parallel section is shown with a fit error close to the 75th percentile. In the second column, a section fitted with a similar error is shown, in which the error is mainly due to the angle between the planes. Columns 3 through 5 show significantly higher fit errors, although in each case the correct anatomical compartments are found.

Impact of stage differences and sectioning deformations

When studying cardiac development, the expression patterns of specific genes are visualized by in situ hybridization and/or immunohistochemical staining on paraffin sections. Despite the deformations resulting from paraffin embedding, sectioning and mounting, TRACTS was able to correctly place most paraffin sections into a reference model (Fig. 3F). The two paraffin models in this test were from mouse embryos of stage E11.0 (Theiler stage 18) (Fig. 3B) and E11.5 (Theiler stage 19) (Fig. 3C); the latter is the same developmental stage as the episcopic reference model. For both embryos, TRACTS showed a small median fit error (Fig. 3F). Compared with the test using episcopic test images, TRACTS performed satisfactorily when the test sections were taken from a paraffin model that was of the same stage as the reference model (P=0.191). Although the younger embryo showed a significantly poorer fit (P<0.001), the performance of TRACTS on the two paraffin models differed only slightly (P=0.069), which shows that TRACTS can cope with small differences in stage.

Comparison to manual fit by experts

A panel of cardiac embryologists was asked to manually register a subset of the images used in the above episcopic-episcopic test in the 3D reference model. They could place a plane in a 3D surface reconstruction by changing its position, tilting angle and tilting direction. The interface showed the cross-section at the position of this interactive plane.

The box plots (Fig. 3G) show the performance of TRACTS and each individual expert. The `best of experts' represents the best-fit result per input image from any of the experts. The median fit error of TRACTS was significantly lower than for each of the individual experts (all P<0.005), but there was no significant difference between TRACTS and the best of experts (P=0.741). TRACTS took 30 seconds per section, compared with several minutes for the experts.

Image space that is difficult to fit with TRACTS

Despite an overall good performance, TRACTS shows high fit errors in a few situations. Obviously, sections showing only a single atrium or ventricle can seldom be fitted to their correct position. Moreover, the left-right symmetry of the heart can lead to cross-sections through both the atrium and the ventricle and no other structures. Such sections can be fitted equally well on the left or right side of the heart. Finally, rotational symmetry along an axis through the atrium and ventricle can lead to sections showing the correct atrium and ventricle lumen without further clues as to the sectioning direction (Fig. 4C, last two lanes). In these cases, the annotation of the anatomical compartments will be correct.

Some of the high fit errors in TRACTS performance could have resulted from biological variation between the reference and test models. Sections fitted in the anatomically right position will then result in a relatively high fit error because the structures are positioned slightly differently in the reference and test models.

Visualization with 3D pdf

To disseminate 3D information, a surface-rendered 3D object can be converted into a 3D pdf using Acrobat and subsequently viewed, and interacted with, using Adobe Reader (see Fig. S3 in the supplementary material). Throughout this text we will refer to the 3D object as `object' (e.g. a 3D reconstruction of a heart) and to the parts of this object as `structure(s)' (e.g. myocardium, cushions and lumen), both in Amira and Acrobat. The object should be a 3D surface, stored by Amira as an Amira Surface file (.surf). The object should not be too complicated (i.e. composed of too many polygons) as this will slow the rendering of the object while viewing the pdf.

Basic 3D pdf

The basic protocol describes how to configure Amira and Acrobat and to properly export the 3D object into a pdf via a single command. The captured pdf can be saved as a basic 3D pdf and the embedded 3D object can be handled interactively using the default Adobe Reader user interface.

Preparation of the 3D object

The conversion of the object from Amira to Acrobat is achieved via a Capture of the Viewer window (OpenGL), which is a feature of Acrobat Pro Extended. Unfortunately, direct capture of an object that consists of several structures will result in an unusable object in Acrobat owing to the separately colored inner and outer surfaces of each structure; juxtaposed structures lead to even more surfaces that are wrongly colored, separated or shared in Acrobat. To solve this, a separate surface for each structure of the object has to be created and assigned the same inside and outside color. Because this can be very laborious, we wrote an Amira script that does this via a single command (see the scripts file in the supplementary material).

Conversion to pdf

When the surfaces are thus assigned, creating a 3D pdf from an object displayed in the Amira Viewer window becomes as simple as pressing the Print Screen button. A capture to Acrobat then contains all separate structures of the object colored correctly. The names assigned to the structures are lost by the capture procedure and reappear as numbers. These can be renamed in Acrobat. The resulting pdf (see Fig. S2 in the supplementary material) contains a 3D object that can be opened in the freeware Adobe Reader.

Basic handling

After opening Adobe Reader, the object is shown as a static image. Clicking on the object will activate the interaction with the object and display the 3D toolbar with the interaction tools (see Fig. S1e in the supplementary material). These interaction tools enable, amongst others, rotation and translation of the 3D object. When the 3D toolbar is hidden, right clicking on the object and selecting Tools>Show Toolbar will uncover it. Similarly, right clicking on the object and selecting Show Model Tree will show the tree containing the structures of the object. Structures can be shown or hidden using the check boxes.

Advanced 3D pdf

The second part of the protocol consists of advanced steps, which require some computer and scripting expertise. There are five independent additions to the basic 3D pdf: (1) creating views and annotations of the object; (2) applying layout; (3) adding buttons; (4) adding scripts for interaction and functionality; and (5) preparing and displaying cross-sections. A 3D pdf featuring these additional steps will be referred to as an advanced pdf (Fig. 5 and see Fig. S3 in the supplementary material).

Fig. 5.

Static illustration of the advanced 3D pdf. Static illustration of the 3D pdf document of a mouse E11.5 heart created with the 3D pdf protocol described in this paper (see Appendix S1 in the supplementary material). For an interactive version of this 3D pdf, see Fig. S3 in the supplementary material.

Views and 3D annotations

To circumvent the convoluted default tools of Adobe Reader we decided to create views and 3D annotations with Acrobat. Annotations of the 3D object are included to identify structures and are created in Acrobat as `3D comment'. Views are preset orientations, structure selections and cross-sections of the 3D object that can be stored in the 3D pdf. Views can be used to guide the reader to the position that, according to the maker, best displays the insights gained from the 3D reconstruction. These preset views in no way diminish the ability of the reader to interact with the 3D object and perform their own exploration of its 3D morphology.

Layout

Strictly speaking, layout is not required to disseminate the 3D objects but it will help to convey your insights and to allow intuitive interaction with your document. Layout will vary according to purpose but it generally includes a header, a placeholder for the 3D object, buttons and hyperlinks. The vector graphics editor Illustrator is the ideal application to generate the layout, as line drawings keep the file size to a minimum. Furthermore, Illustrator belongs to the same software suite, avoiding incompatibility problems. After having created a design pdf, the 3D object from the basic pdf created with the basic protocol can be pasted into this design.

Buttons and hyperlinks

The functionality of the 3D pdfs can be enhanced with buttons (see Fig. S1f in the supplementary material). These buttons simplify the default interaction tools for showing views and/or hiding structures. The behavior of buttons can be controlled by scripts. The graphics of the buttons is created in Illustrator and their functionality is added by superimposing transparent buttons in Acrobat; Acrobat also provides default functions that can be assigned to buttons. Hyperlinks can be added to buttons to link the 3D pdf to the accompanying paper and the affiliated institute.

Scripting

Scripting is the most powerful part of the enhancements, for which some programming experience is required. In our published documents (Moorman et al., 2009; Postma et al., 2009; van den Berg et al., 2009; van Wijk et al., 2009) we mainly used scripts to hide and show structures and to make them transparent. By using functions in scripts, you will centralize the functionality of your document, which makes it easier to change this functionality or apply it to multiple buttons. Furthermore, the functions can be coded to be page independent. This will ensure that the buttons keep working when the page numbers change, which occurs when appending the 3D pdf to a publication or after combining multiple 3D pdf files into one document. The scripts used are detailed in Appendix S1 in the supplementary material.

Cross-sections

Sectioning an object to display a cross-section and to expose the lumen is frequently required in the presentation of results of anatomical and embryological studies. Often, only a specific sectional view reveals the connectivity of compartments or the relation between tissues. Moreover, when the cross-sections placed by TRACTS are displayed in the reference model, the location at which the gene is expressed can be determined (Fig. 6).

Fig. 6.

Biological application. (A,D) Sections from two different mouse embryos stained for the expression of cTnI. (B,E) Adjacent sections of A and D, respectively, stained to visualize Tbx3 expression. Arrows indicate Tbx3 expression. The cTnI-positive myocardium in these images was segmented to enable TRACTS to fit these sections into the reference model. (C,F) The fit locations found by TRACTS are shown in the reference mode; cranial (C) and left lateral (F) views are shown. For an interactive version, see Fig. S3 in the supplementary material. Scale bars: 200 μm.

Cutting an object and displaying the cross-section is possible within many 3D rendering programs, but, unfortunately, not in Acrobat and Adobe Reader. Acrobat can apply a cross-section, but its interpretation is hampered by the fact that the surface of the cut is not rendered. To show a rendered surface cut, cuts have to be prepared in Amira and subsequently exported as distinct structures. These can be treated in Acrobat in such a way that cutting the object is emulated (see Appendix S1 in the supplementary material).

Biological application

To demonstrate the functionality of the combination of TRACTS and the visualization of its results in a 3D pdf, sections from two different E11.5 mouse hearts were placed into the E11.5 reference model using the user interface (Fig. 2). From both embryonic hearts, two serial sections were stained to visualize cTnI-positive (Fig. 6A,D) and Tbx3-positive (Fig. 6B,E) myocardium. Tbx3 is expressed in the precursors of the cardiac conduction system (Hoogaars et al., 2004). Without the spatial context, the compartments present in the stained sections are difficult to identify. To resolve this, the cTnI-stained sections were loaded into TRACTS and fitted into the E11.5 morphological reference model (Fig. 1A). The cTnI-stained sections were used for this purpose because the heart can be easily segmented from these sections using the user interface (Fig. 2). The coordinates of the cutting plane were exported to the 3D reconstruction program and the cross-section was visualized in the reference model. From this, an advanced 3D pdf (see Fig. S3 in the supplementary material) was created using all steps of the 3D pdf protocol. The results show that one heart was sectioned in a sagittal direction (Fig. 6C) whereas the other was transversally sectioned (Fig. 6F). From these cutting planes it becomes immediately clear that in the embryonic mouse heart, Tbx3 is expressed in the myocardium of the atrioventricular canal as well as in the cushions of the atrioventricular canal and the outflow tract.

DISCUSSION

TRACTS

The fit of sections by TRACTS results in the recovery of the correct position to within 62 μm or 14.3° for 75% of the input sections. In the process of finding this spatial position, TRACTS does not apply any deformation to the image other than anisotropic scaling and rotation. The accuracy of TRACTS is more than sufficient to help the user to identify the compartments and their boundaries in the input section. It will thus point to those compartments that warrant more detailed study to verify colocalization of expressed genes.

Moreover, with paraffin-embedded material, sections from an age-matched embryo are placed with similar precision. Because mouse embryos from the same pregnancy can differ by up to 0.5 days in developmental stage, it is reassuring that the slightly younger paraffin model showed only a small increase in the median fit error and that in this test most recovered reference model sections showed the correct annotation of cardiac compartments. These results show that TRACTS is a robust tool to trace the correct anatomical position of single tissue sections. The comparison of the performance of TRACTS with that of an expert panel showed that TRACTS outperformed each individual expert and performed similarly to the best results of these experts.

However, when comparing expression patterns one should be aware that every embryo is unique, as is every section obtained from it. Furthermore, expression patterns will vary owing to methodological variation, such as in the quality of the antibody, fixation of the specimen and exposure time to substrates.

Because TRACTS relies on the normal morphology of the reference models, fitting sections of deformed, mutant hearts might lead to unpredictable errors. However, when high-throughput mutagenesis studies (Rosenthal et al., 2004) are followed up by gene expression analysis in a mutant line with reproducible aberrant morphology, it would not be difficult to implement a mutant reference model in TRACTS. Comparison between morphology and gene expression patterns in wild-type and mutant hearts will then greatly benefit from the ability to place those reference models side-by-side in a 3D pdf document.

TRACTS can be expanded with the functionality to fit heart sections of ages other than the current series of heart models. Given the simplicity of the currently implemented image features, it is expected that they will be reasonably robust. However, the substantial change in shape that occurs during embryonic development or with malformation might require the implementation of other image features to keep computation times down.

The use of image features has a twofold purpose: (1) to prevent misplacements and (2) to reduce computation time. The use of features resulted in a reduction, from 16% to 7%, in the number of images with a fit error exceeding 5. The computation time dropped over 20-fold to only 30 seconds per section, which is adequate for the intended use of TRACTS and is fast compared with the minutes per section that experts needed to place sections. TRACTS will mainly be used by researchers who use morphological techniques to verify the spatial position of individual sections. For their benefit, the program with annotated reference models is available on request. It should be noted that the reported computation time does not include the segmentation of the input image to extract the contours of the heart. The fitting by TRACTS requires segmented heart tissue. Therefore, sections in which a gene of interest is visualized need to be either alternating or double stained with a cardiac-specific marker, otherwise the heart tissue needs to be segmented manually. This segmentation or thresholding of the myocardium is crucial for the performance of TRACTS, as it uses the contours of the segmented heart to select the best match from the model cross-sections that are selected on the basis of image features. The automated segmentation of the heart will add only seconds to the processing time using the provided user interface (Fig. 2).

Application of the current version of TRACTS to high-throughput projects would not be efficient. When large numbers of stained serial sections are available, fitting of individual sections should take advantage of the additional information, i.e. the known parallelism and distance between the sections. When complete 3D reconstructions are made, a 3D-on-3D registration technique is more appropriate because this can make use of the 3D continuity of the input set.

Besides supplying users with information about the orientation of their input sections in the reference model, TRACTS provides some extra benefits. Because the reference model is anatomically annotated, it is possible to map this annotation onto the input section after it has been placed into the reference model (Fig. 2). This facilitates the annotation of arbitrary tissue sections, avoids erroneous annotations that can be encountered even in anatomical journals, and improves scientific communication. Placing specifically stained sections into a common reference model also makes it possible to collect the visualized gene expression patterns within their anatomical context. This spatial insight can help to generate hypotheses on the genetic regulation of development.

Some of the existing gene expression atlas projects, such as EMAGE (Baldock et al., 2003; Christiansen et al., 2006), already add spatial information (e.g. anatomical context) to their gene expression database (for a review, see de Boer et al., 2009). There are two major differences between the approaches of EMAGE and TRACTS. First, EMAGE is embryo wide, whereas TRACTS is currently focused on the heart. This choice was made to overcome the morphological variations between embryos that hamper the placement of sections covering the whole embryo. Second, in EMAGE the spatial position of a section has to be found manually with MAPaint software (http://genex.hgu.mrc.ac.uk/Software/paint/), whereas TRACTS is aimed to place sections automatically.

3D pdf

Protocols to enable the incorporation of 3D objects in scientific papers have been published in the fields of astrophysics (Barnes and Fluke, 2008), molecular/protein structure (Kumar et al., 2008) and biology (Ruthensteiner and Hess, 2008). However, the protocol we present here is the first 3D pdf protocol specifically tailored to provide the novice biologist with a simple basic protocol and optional advanced steps. Similar to the existing protocols, in which PyMol (Kumar et al., 2008) or s2PLOT (Barnes and Fluke, 2008) are used as 3D visualization software, this protocol was tailor-made for use with the software packages Amira and Acrobat. Irrespective of the visualization software employed, the current protocol describes advanced steps that 3D pdf creators can use to enhance their 3D document, after the capture of the 3D object into the document. Any 3D visualization program using OpenGL should be suitable for the capture of a 3D object by Acrobat, but will require customization of the basic protocol. The protocol described in this paper can then serve to pinpoint the critical settings. Importing 3D file formats into Acrobat (e.g. .obj, .vrml and .3ds) is also possible, but will require a custom protocol to create, export and import such 3D objects, which can become very laborious (Ruthensteiner and Hess, 2008).

A basic 3D pdf, enhanced with views, annotations and cross-sections, can easily be embedded into a paper published in pdf format. However, when the 3D pdf includes layout, buttons and scripts, it is not possible to embed it into another page without interfering with the layout of the publisher. Nevertheless, such an advanced pdf can easily be added as extra pages to the publication pdf or as supplementary pdf files (as with Fig. S3 in the supplementary material). The latter option enables the reader/presenter to show the 3D pdf in full-screen mode, which is also very useful for scientific talks and education.

Although the protocol itself has a steep, but short, learning curve, implementing it is not difficult. Production time can be cut even further using templates, whereby the existing object in the original document can be replaced by another and only minor modifications are required to make the document operational.

Conclusions

The TRACTS program was shown to perform similarly to the best of five morphology experts and to be relatively robust for differences in developmental stage. In combination with the embedding of 3D objects in Adobe pdf, this will improve the annotation of sections and the communication of experimental results. The 3D pdf is an ideal means to cope with the growing demand to include complex 3D interactive information into scientific papers. Enriching documents with 3D objects is ideal in the sense that it is universal; every reader can open and directly interact with the 3D content. Furthermore, it enables researchers to better convey their observations and conclusions, allowing independent verification and avoiding miscommunication. As online publishing becomes more and more common, embedding 3D objects into online publications is the next logical step.