An evaluation of automated tracing for orbitofrontal cortex sulcogyral pattern typing

Abstract

Background:

Characterization of stereotyped orbitofrontal cortex (OFC) sulcogyral patterns formed by the medial and lateral orbitofrontal sulci (MOS and LOS) can be used to characterize individual variability; however, in practice, issues exist for reliability and reproducibility of anatomical classifications, as current methods rely on manual tracing.

New Method:

We assessed whether an automated tracing procedure would be useful for characterizing OFC sulcogyral patterns. 100 subjects from a published collection of manual OFC tracings and characterizations of patients with bipolar disorder, schizophrenia, and typical controls were used to evaluate an automated tracing procedure implemented using the BrainVISA Morphologist Pipeline.

Results:

Automated tracings of caudal and rostral segments of the medial (MOSc/MOSr) and lateral (LOSc/LOSr) orbitofrontal sulci, as well as the intermediate (IOS) and transverse orbitofrontal sulci (TOS) were found to accurately identify OFC sulci, accurately portray sulci continuity, and reliably inform manual sulcogyral pattern characterization.

Comparison with Existing Method:

Automated tracings produced visibly similar tracings of OFC sulci and removed subjective influence from locating sulci. The semi-automated pipeline of automated tracing and manual sulcogyral pattern characterization can eliminate the need for direct input during the most time-consuming process of the manual pipeline.

Conclusions:

The results suggest that automated OFC sulci tracing methods using BrainVISA Morphologist are feasible and useful in a semi-automated pipeline to characterize OFC sulcogyral patterns. Automated OFC sulci tracing methods will improve reliability and reproducibility of sulcogyral characterizations and can allow for characterizations of sulcal patterns types in larger sample sizes, previously unattainable using traditional manual tracing procedures.

Introduction

Brain imaging has great potential to reveal biomarkers that indicate pathogenic or diagnostic risk. However, current methods have failed to produce reliable biomarkers of psychiatric disease. This may be, in part, because in order to assess the large sample sizes necessary to establish biomarkers, we typically rely on registration methods and averaging procedures that align individual brains to a common coordinate system. While these registration methods can be incredibly useful for establishing differences between groups of subjects with varying diagnoses, their utility is minimal in the context of individualized patient risk assessment, or precision medicine. To establish the type of biomarkers that have greater potential for precision medicine, we may need to rely on higher-order brain features. One way to assess higher-order brain features is through the characterization of differences in sulcogyral anatomy. This method has primarily been used within the “H-sulcus” of the OFC, which is determined based on the continuity of rostral and caudal segments of the medial and lateral orbitofrontal sulci (MOS and LOS, respectively; See Figure 1A). Pattern types were originally described and labeled based on their frequency in an initial cohort of 50 human subjects, with Type I being the most common (~60–70% of hemispheres) and Type II and Type III found to be much less common (15–30% of hemispheres) (Chiavaras and Petrides, 2000). It has been repeatedly found that less common pattern types (Type II and III) confer increased risk for developing schizophrenia (Chakirova et al., 2010; Takayanagi et al., 2010; Lavoie et al., 2014; Bartholomeusz et al., 2013) and more recently, bipolar disorder (Patti & Troiani 2018).

Figure 1.

Manual and automated OFC sulci tracings. Brains with OFC sulcogyral pattern types of Type I, II, and III in both hemispheres are shown with different tracings in each row. Row A shows manual tracing, which distinguishes sulci within the OFC with different colors. Row B shows full brain sulcus tracing from the automated pipeline, with a different color for each sulcus group. Note that the green color designating the orbital sulcus is similar to a more posterior sulcus; only the more anterior group of sulci, isolated in Row C, consist the OFC sulcus group. Row C shows the isolated OFC sulcus group. Although the automated, isolated OFC sulcus group does not distinguish between sulci within the OFC, the tracings are visibily similar to the manual tracings.

Traditional classification methods include manual tracing of each OFC sulcus and characterization of each hemisphere as Type I, II, or III based on the MOS and LOS continuity. Pattern type is determined based on the continuity of the rostral and caudal portions of the MOS and LOS, with Type I having a discontinuous MOS and continuous LOS, Type II having both a continuous MOS and LOS, and Type III having a discontinuous MOS and LOS (see Figure 1). The tremendous amount of individual variability within the OFC poses challenges to developing automated labeling procedures. While the OFC sulci are labeled according to series of rules based on their relative position to surrounding sulci, additional, variable numbers of sulci also exist in this part of the cortex, further complicating the labeling procedure.

Although we describe and test this automated tracing procedure in one type of sulcogyral characterization within the OFC, other sulcogyral patterns exist elsewhere in the brain. For example, there are cortical folding patterns in the anterior cingulate cortex (ACC) that can be reliably classified (Cachia et al., 2016). The ACC patterns are most similar to those in the OFC in that each subject can be classified as a particular type, but other work also suggests that quantitative sulcogyral metrics are meaningful for characterizing individual variability in the brain. Regional shape metrics within the central sulcus (CS) are associated with attention deficit disorder (Li et al., 2015). Global sulcal morphological metrics can be used to identify sulcal changes associated with Alzheimer’s disease (Cai et al., 2017). Further, recent analyses of morphometric differences within the precuneus suggest that important variability exists within individual sulcal patterns that may not correspond to typically used brain metrics (Pereira-Pedro and Bruner, 2016). Although there are numerous other studies that have used cortical thickness, we highlight those above in order to emphasize that these higher level sulcogyral features are not widely used and may be important for capturing individual variability in surface morphology that interact with more widely used cortical surface metrics. Further, although this study is specific for OFC sulcogyral patterns, this procedure will likely be useful in the future in applications outside of the OFC.

Thus, although there could be tremendous utility in assessing this important higher order feature of OFC sulci, the procedure for manually tracing and characterizing this feature may be limiting its widespread application. This is evidenced by the relatively small number of laboratories that employ the tracing procedure, despite the publication of this method nearly two decades ago (Chiavaras and Petrides, 2000). Here, we assess whether partially automating the OFC sulcogyral characterization procedure is feasible by using automated tracing methods. Automated OFC sulci tracing differs from segmentation of other brain regions, such as the hippocampus or amygdala (Iglesias et al., 2015; Saygin et al., 2017). Segmentations often distinguish tissues and cortical or subcortical structures, but sulci tracings instead highlight empty space or distinct separations in gray matter folds. Additionally, OFC morphology is highly variable, which requires a set of rules to determine precise sulcus labeling. Therefore, we evaluate the use of a semi-automated procedure here, in which the tracing is automated, but with additional characterization of the pattern completed manually. We accomplish this by utilizing the Morphologist pipeline within BrainVISA to automatically trace OFC sulci and then assessing the utility of these tracings for pattern typing in 100 subjects that were previously traced and characterized using a manual tracing protocol (as reported in Patti & Troiani, 2018).

Materials and Methods

Subject Selection and Preprocessing

100 subjects’ structural brain images were selected from a publicly available data set (Poldrack et al., 2016) that had been previously traced using manual methods (Patti & Troiani, 2018). We also chose this data set because it contained both patients with various psychiatric disorders and controls, which would enrich the data set for the less common OFC sulcogyral patterns typically observed in patients with schizophrenia disorder (Chakirova et al., 2010; Takayanagi et al., 2010; Lavoie et al., 2014; Bartholomeusz et al., 2013) and bipolar disorder (Patti and Troiani, 2018). Randomly selected subjects in three groups from the larger UCLA dataset’s study (Poldrack et al., 2016) were included, including those with confirmed schizophrenia (N=34) and bipolar disorder (N=33), as well as subjects recruited as controls, with no evidence for schizophrenia or bipolar disorder (N=33). All subjects had manual OFC tracings for comparison and age data (group mean = 33.3 +/− 8.9 years) available from our previous work. Structural brain images were obtained from the University of California Los Angeles Consortium for Neuropsychiatric Phenomics (Poldrack et al., 2016; https://exhibits.stanford.edu/data/catalog/mg599hw5271), and subject recruitment and image acquisition information can be found in our past work with manual OFC tracings of this dataset (Patti and Troiani, 2018). Skull-intact, 1mm cubic voxel, T1 weighted structural brain images were aligned and transformed to a skull-intact MNI template to correct orientation of structural images prior to their input to the BrainVISA pipeline.

Automated OFC Tracing via BrainVISA Morphologist Pipeline

The Morphologist tools (http://brainvisa.info/web/morphologist.html), distributed with the BrainVISA software, were applied to automatically trace OFC sulci. The Morphologist-UI performs batch brain segmentation (Riviere et al., 2003) and sulcal labeling (Perrot et al., 2009; Perrot et al., 2011) on structural images. Segmentation steps included field-bias correction; intensity-based extraction of the brain from the skull, hemispheres from the brain, and grey and white matter from hemispheres; creation of grey and white matter meshes; and recognition of cortical folds. Segmentations were visually inspected to ensure brain masks did not exclude OFC voxels. Sulcal labeling was implemented with the Bayesian labeling method, Spatial Parametric Anatomical Mapping (SPAM) (Perrot et al., 2011). Cortical folds from the segmentation pipeline were aligned to probabilistic maps of labeled sulci using the standard Global+Local affine sulcal-wise transformation such that each sulcus could be probabilistically assigned a sulcus label. Morphologist labels 63 sulci on the left hemisphere and 62 sulci on the right hemisphere. Importantly, for the brain region assessed here, the OFC sulci (the MOS, LOS, TOS, and IOS) are all encompassed with one label/sulcal group within the Morphologist pipeline (collectively called Orbital Sulcus) for each hemisphere. Using the Create Sulcus Label Volume tool, the Right and Left Orbital Sulcus label allowed for the isolation of all OFC sulci from sulci with other labels. The isolated orbital sulcus was saved as a NIFTI file and was converted to a *.arg file. The *.arg file can be viewed in 2D slices like a NIFTI file and is easier to view in BrainVISA’s image viewing application (Anatomist) than NIFTI files. The isolated Orbital Sulcus in the automated tracings distinguished all OFC sulci with the same color, whereas manual tracing methods also distinguish individual OFC sulci from each other (i.e. the MOS, TOS, LOS, and IOS are the same color in automated tracings but are each a different color in manual tracings). Altogether, the pipeline output several different file types for viewing the full Orbital Sulcus for each subject for each hemisphere (See Figure 1B,C).

Evaluation of Automated Tracings

We compared the automated vs. manual tracings in several ways in order to evaluate how well the automated tracings allow for the characterization of sulcogyral pattern type. A comparison of the manual and automated pipelines is depicted in Figure 2. The below metrics were evaluated by overlaying the isolated orbital sulcus *.arg outputs from the Morphologist pipeline onto individual subject scans.

Figure 2.

Manual and automated pipelines. Both pipelines highlight the orbital sulcus so that sulcus continuity can be determined for the correct sulcus during pattern type characterization. The continuity of the sulci in the structural image ultimately determines pattern type, so different methods (manual or automated) to inform the location of sulci can both reliably inform pattern type characterization.

(1). Sulcus Presence.

The frequency with which OFC sulci were automatically traced. The MOS, LOS, TOS, and IOS are present in almost all sulcogyral patterns, so the ability of the pipeline to orient a characterizer to these sulci of interest was evaluated as the frequency these specific sulci were traced by the automated pipeline. In OFC sulcogyral pattern characterization procedures, typically all OFC sulci are traced and individually labeled. However, the medial and lateral sulci are most important when determining pattern type. Thus, we assessed how frequently the medial and lateral orbital sulci were traced using automated methods, separately for rostral and caudal segments. This was operationalized as follows: At least two voxels of manually traced sulcus had to be present in the automated tracing; when multiple fragments of a rostral sulcus were identified in the manual tracing, at least one fragment had to be present in the automated tracing to indicate presence of the rostral sulcus’ tracing; and the presence of tracing within continuous sulci (the MOS or LOS, whose caudal and rostral segments could be considered the same entity) was evaluated with two separate approaches, which we call “conservative” and “lenient”. The conservative approach evaluated caudal and rostral segments of the same sulcus independently, with presence indicated by two distinct voxels traced unique to the caudal/rostral sulcus that extended from the sulcus’ intersection with the TOS. In the uncommon case that there was either no TOS or multiple TOS’s, in order for the caudal or rostral segment’s tracing to be considered present, caudal and rostral segments had to be visible such that it could orient a person to the location of the segments. The lenient approach assumed that either the caudal or rostral tracing of a continuous sulcus could both orient a person to the full continuous sulcus, and therefore only a caudal or rostral tracing would indicate presence for the full sulcus (including both the caudal and rostral segments). Overall, sulcus presence measured the pipeline’s successes in tracing each OFC sulcus divided by the number of attempts to trace the OFC sulcus on subjects’ hemispheres.

(2). Continuity Accuracy.

The frequency with which automated tracings accurately portrayed continuity of OFC sulci. Since MOS and LOS continuity determine sulcogyral pattern type, the automated tracings’ ability to portray sulci continuity would influence the utility of the automated tracings in guiding sulcogyral pattern classifiers, whether the classifying is done by a person (as here) or by machine learning (potential future application). A continuous sulcus was portrayed in the automated tracing by having at least one axial plane in which caudal and rostral segments of the sulcus were connected by traced voxels either adjacent, diagonal, superior, or inferior (or a combination of these directions, e.g. adjacent and inferior) to each other. Consistent with the sulcal presence definitions, the caudal and rostral segments had to extend across two voxels that were unique to the sulcus segment and were oriented in the direction of the sulcus segment. In addition to these requirements, no separate, isolated fragments of the sulcus could be present for there to be apparent continuity in the tracing. Finally, continuity required direct connection between caudal and rostral segments of the same sulcus. The “Continuity Accuracy” metric, resulting from these continuity measures, was the frequency with which continuity established in the manual tracings’ aligned with the continuity information determined from the automated tracings. The “Adjusted Continuity Accuracy” metric limited these comparisons to only sulci that were traced in the automated pipeline. Therefore, “Continuity Accuracy” will be lower because any sulcus that was not traced in the automated pipeline would automatically reduce accuracy. Overall, continuity accuracy measured the pipeline’s successes in depicting each OFC sulcus’ continuity divided by the number of attempts to depict the OFC sulcus’ continuity on subjects’ hemispheres.

(3). Manual/Automated Tracing Reliability.

The reliability of sulcogyral pattern classification between automated and manual OFC sulcal tracings was used as a proxy for how well the manual tracings could be replaced by automated tracings when determining sulcogyral pattern type. Sulcogyral pattern characterization was performed on both manual tracings and automated tracings, and both sets of characterizations were compared to the accepted sulcogyral characterizations for each brain hemisphere. To maintain consistency, the same person (W.S.) characterized both automated and manual tracings, blind to the accepted characterizations. Cohen’s Kappa (κ) determined inter-rater reliability, with reliability being evaluated between manual tracing characterizations and accepted characterizations and also between automated tracing characterizations and accepted characterizations. κ was computed for both, with comparable values indicating the automated tracings could replace the manual tracings.

(4). Pattern Type Mischaracterization.

The frequency with which sulcogyral pattern types in automated tracings were characterized differently from accepted characterizations. Certain pattern types may be vulnerable to mischaracterization within the automated pipeline, either because of ambiguity in the continuity of sulci or misidentification of sulci within the OFC. In order to describe the susceptibility of pattern types to mischaracterization and to describe how well the tracings inform characterizers of the sulcogyral pattern, we calculated mischaracterization frequencies for each pattern type using automated tracings. The mischaracterization frequency for a given pattern type referred to the frequency with which automated tracing characterizations did not align with that given pattern type (i.e. false negative frequency with respect to the presence of the given pattern type was calculated, as opposed to false positive frequency for a given pattern type). For this metric, we grouped together the rarest Type IV patterns with Type III, consistent with previous work (Bartholomeusz et al., 2013; Patti and Troiani, 2018). A chi-square test for independence evaluated whether pattern type and mischaracterization were independent. Given mischaracterization and pattern type were not independent, a two-proportion z-test evaluated which mischaracterization frequencies were significantly different from each other. Overall, pattern type mischaracterization measured the pipeline’s failures characterizing a given pattern type divided by the number of times the pattern type was identified in the established data set.

Results

Results for sulcus presence, continuity accuracy, manual/automated reliability, and pattern type mischaracterization are summarized in Table 1.

Table 1.

Results for Sulcal Trace Presence, Continuity Accuracy, Reliability, and Mischaracterizations by Pattern Type.

Sulcal Trace Presence (%)	MOSc	MOSr	LOSc	LOSr	TOS	IOS
Conservative Definition	100	74.0	99.0	90.0	100	98.8
Lenient Definition	100	77.0	99.5	93.5	--	--
Continuity Accuracy (%)	MOS Left	MOS Right	LOS Left	LOS Right
Continuity Accuracy	62.0	63.0	68.0	78.0
Adjusted Continuity Accuracy	82.7	87.5	80.0	84.8
Reliability	κ	Confidence Level (%)	CI
Manual Tracings	0 .794	95	0.718 to 0.869
Automatic Tracings	0.676	95	0.582 to 0.770
Mischaracterizations by Pattern Type (%)	Type I	Type II	Type III/IV
Automatic Tracings	7.3*	34.7*	28.9*

^*Frequencies significantly varied (p<0.05) by pattern type as determined with chi-square test for independence. Type I and II and Type I and III were significantly different (p<0.05) by a two proportion z-test, however Type II and III were not found to be significantly different by the z-test.

Legend for Sulcus Abbreviations (see Figure 1 for depiction of sulci locations): MOS = medial orbital sulcus MOSc = caudal MOS segment MOSr = rostral MOS segment LOS = lateral orbital sulcus LOSc = caudal LOS segment LOSr = rostral LOS segment IOS = intermediate orbital sulcus TOS = transverse orbital sulcus

(1). Sulcus presence.

Sulcus presence was evaluated with both a conservative and lenient definition, which vary based on whether the continuity of the sulcus is considered. Examples of lower and higher quality tracings based on sulcus presence are shown in Figure 3. The conservative definition for trace presence (caudal and rostral segments evaluated independently) indicated high accuracy for the pipeline to trace the majority of OFC sulci (IOS=99%, LOSc=99%, LOSr=90%, MOSc=100%, TOS=100%); however, the MOSr was less frequently identified (74%). The lenient definition, which allows for sulcus presence to be determined if at least one of the rostral or caudal segments is traced, yielded similar results (LOSc=100%, LOSr=94%, MOSc=100%, MOSr=77%). The exclusion of the MOSr in automated tracing appeared to occur more often when the sulcus was discontinuous and/or more fragmented. Occasionally, the pipeline showed false positives for sulcus trace presence, in which sulci not within the OFC were retained in the isolated Orbital Sulcus tracing, but actually were associated with temporal and prefrontal cortices.

Figure 3.

Tracing quality and sulcus presence. In both tracings, OFC sulci that exist but were failed to be traced by the pipeline are highlighted in red ellipses. The left tracing is considered lower quality because the sulci most relevant to manual pattern characterization are not traced. The untraced sulci in the lower quality tracing are the left and right hemisphere MOSr and the left hemisphere LOSr. The untraced sulcus in the higher quality tracing was the right hemisphere IOS. The lower quality tracing still informs the manual characterizer to the locations of the untraced sulci, especially since the robust IOS is traced and is known to be located between the MOS and LOS. Lower quality tracings therefore are also useful in evaluating the sulcogyral pattern type.

(2). Continuity Accuracy.

The continuity of the MOS and LOS is used to determine sulcogyral pattern type, and the automated tracings were evaluated with two accuracy measures for how well this continuity was represented based on the tracing. The raw accuracy for the continuity representation of MOS and LOS sulci was informative (Right Hemisphere: MOS=63%, LOS=78%; Left Hemisphere: MOS=62%, LOS=68%). To establish accuracy for this measure, we assumed that continuity representation required full sulcus trace presence for a given sulcus, and so the lower MOS accuracy reflects the more frequent exclusion of MOSr in the sulcus presence determination previously described (conservative definition). We also computed an Adjusted Continuity Accuracy using the lenient definition of sulcal trace presence and confirmed expected improvement in continuity determination (Right Hemisphere: MOS=88%, LOS=85%; Left Hemisphere: MOS=83%, LOS=80%).

(3). Manual/Automated Reliability.

The goal of OFC sulcogyral tracing here is for use in OFC sulcogyral pattern characterization, so the utility of the pipeline is largely dependent on how accurately automated tracings can be characterized. To determine whether there is reliability between characterization using manual vs. automated tracings, a tracer (W.S.) trained on established criteria for OFC sulcogyral characterizations evaluated automated tracings to determine OFC sulcogyral pattern. These characterizations were then compared with OFC pattern characterizations previously completed using manual tracings. It is important to note that the role of tracing in the OFC sulcogyral pattern characterization procedure is to familiarize the tracer with the individual OFC sulcogyral anatomy. The tracings are not used to determine the pattern-rather, they are used to ensure the rater is identifying the appropriate sulci for use in pattern characterization. Manual tracings and automated tracings were both reliably characterized (κ = 0.794, 0.676, respectively). At a 95% confidence interval, the confidence intervals for the κ values overlapped ((0.718,0.869), and (0.582,0.770), respectively). While this does not entail a nonsignificant difference, this suggests the values are comparable. The discrepancy in reliability between the types of tracings could have been due to the automated tracings not distinguishing sulci within the full Orbital Sulcus with different colors, which could potentially lead to misrecognition of sulci (e.g. MOSr, LOSr) during characterization. False positives of sulcus trace presence could have also contributed to misrecognition of sulci during characterization.

(4). Pattern Type Mischaracterization.

Frequencies with which pattern types were mischaracterized were calculated for automated tracings to indicate pattern types that are susceptible to mischaracterization by the semi-automated pipeline. The chi-square test for independence indicated the mischaracterization frequencies were not independent of the pattern type (χ²=17.1, p<0.05). Automated tracings had significantly higher mischaracterization frequency for Type II (z=4.31, p<0.05) and Type III/IV (z=3.54, p<0.05) as compared to Type I. The higher mischaracterizations in Type III/IV may have been due to the fragmented sulci leading to misidentification of sulci and therefore incorrect evaluation of sulcal continuity. Significantly greater Type II mischaracterization in automated tracings was unexpected because the MOS and LOS are both continuous, presumably making it easier to identify all sulci (especially caudal and rostral segments) within the semi-automated pipeline. However, the lack of color distinction between OFC sulci in automated tracings may have still contributed to the increased mischaracterizations.

Discussion

Here, we adapted an existing Morphologist pipeline to automatically trace OFC sulci in order to establish a semi-automated process for characterizing OFC sulcogyral patterns of the H-sulcus. We evaluated several metrics, including: (1) Sulcus presence, (2) accurate determination of sulcus continuity, (3) reliability of sulcogyral pattern classification between automated and manual tracings, and (4) whether specific pattern types were misclassified using automated tracing methods. We find that automated tracing procedures identify the majority of OFC sulci (IOS=99%; LOSc=99%; LOSr=90%; MOSc=100%; MOSr=74%; TOS=100%), appropriately determine sulcal continuity (>=80% for MOS and LOS in both hemispheres), and have comparable reliability to manual tracings (manual: κ = 0.794; automated: κ = 0.676) when used in pattern classification of H-sulcus pattern type.

Current methods for characterizing OFC sulcogyral morphology not only require manual tracings of OFC sulci, but also require visual inspection of the structural brain image scan to further evaluate sulci continuity. These two layers of manual processing likely have prevented large scale use of the method due to the time consumed in carrying out the procedure. We believe that the evaluation of the Morphologist pipeline’s automated tracings indicates that the first and most time consuming layer of the processing can be reliably replaced with these automated methods. Automated tracings could likely replace the role of manual tracings in orienting one to the morphology of the OFC; however, we suggest that one trains in the full manual procedure prior to this replacement with automated tracings. Relying on automated tracings assumes the characterizer is both able to reliably differentiate the OFC sulci (MOS, LOS, IOS, TOS) and is able to evaluate continuity of the sulci in the T1 scan. This is especially true because individual OFC sulci are not differentiated by color in the automated tracings, and in order to evaluate continuity in the T1 image, one must be able to recognize these sulci within the automated tracings using the same criteria to manually differentiate and manually trace the sulci. The small difference in reliability in characterizing the manual and automated tracings and the increased mischaracterization of the Type II pattern in automated tracings were likely due to lack of color differentiation in automated tracings’ sulci which cannot always be easily resolved by the judgement of the characterizer. Nevertheless, recognition of OFC sulci in automated tracings by a trained tracer is much more rapid than recognition of OFC sulci when performing manual tracings.

The high frequency with which OFC sulci were traced and the highly informative portrayal of continuity in the automated tracings suggest future work could take advantage of the automated tracings in studies with larger sample sizes. The ability of the automated tracings to highlight the sulci of interest and to accurately inform continuity reduces the complexity involved in manual pattern characterization. As mentioned prior, it is still suggested that the characterizer has experience with manual tracing and brain anatomy, but the informative nature would reduce the strain in the second layer of manual processing (manual evaluation of sulci continuity in the T1-scan). If any sulci were not traced in the automated tracing, it would likely be readily apparent to someone with manual tracing experience, especially due to the characteristic “H” shape of the full Orbitofrontal Sulcus. Thus, an alternative pipeline could be screening of automated tracings to identify those that may need to be performed manually as a quality check in larger analyses employing the automated pipeline. Applying machine learning to recognize tracings with potential sulci trace dropout could allow for less experienced characterizers to evaluate sulci presence.

In future implementations, it will be important to recognize potential sources of error for the pipeline. The decreased recognition of more rostral OFC sulci is likely owed to the more rostral sulci lying more ventrally along the cortex. The ventral location is closer to the sinus and air-tissue boundary, producing magnetic field distortions, signal dropout, and artifacts (Holland et al., 2010). Brain-masking of distorted regions could additionally lead to dropout of the most rostral OFC regions. Automated sulci tracings are based on gray/white matter segmentations while sulci recognition is based on probabilistic maps for a more general region. Artifact and dropout issues could result in altered tracings but would likely not affect the proper identification of the full OFC sulcus group. Scanner and site variability may have the same effect. Different scanner field strengths and receiving coils influence structural MRI (Chen et al., 2015). While it is plausible varied acquisition parameters and scanner variability across sites could influence signal, we believe the gross morphology of the OFC sulci used in our analysis are robust enough to maintain reliability despite potential site differences. As automated methods are being implemented with multi-site data, it will be useful for a characterizer to be familiar with OFC morphology and manual tracings to help recognize untraced OFC sulci and to evaluate whether signal dropout in this region may influence results.

Larger analyses with automated OFC tracings could facilitate fully automated OFC pattern recognition and the identification of further characteristic morphological features within OFC sulci. While sulci throughout the brain can be reliably recognized because of characteristic primary folds (dictated by sulcal roots (Regis et al., 2005) or similarly, sulcal basins (Lohmann and Yves von Cramon, 1998)) patterns of primary folds and associated secondary or tertiary folds do not form consistent patterns with every sulcus or sulcus group. The cingulate sulcus is one such sulcus that does have consistent patterns, which vary in shape and continuity between segments, and robust automatic detection of these patterns can be captured with an automated tracings dataset consisting of only 36 brain scans (Sun et al., 2007). Given the automated tracings for OFC sulci are informative of the pattern type by our metrics, it is reasonable that the pattern types could be reliably detected within automated tracings using automated pattern characterization. The ability of automated OFC tracings to have recognizable patterns can also be extended to evaluating characteristic morphological features beyond the basic pattern types based on sulci continuity. Other features, such as secondary or tertiary folds, may be more variable (Lefèvre and Mangin, 2010) but also have functional relevance. Secondary folds within the OFC, as well as patterns of sulci morphology that may only be identified through clustering procedures requiring large amounts of tracing data, likely impact the brain region’s function, and so the involvement of the OFC in executive and social processing and value representations (Kringelbach and Rolls, 2004; Rudebeck et al., 2008) further implicates diagnostic value in understanding the more variable features of its sulci. Variance in the more stereotyped groupings into pattern types have already shown to be predictive in variance of subclinical anhedonia in schizophrenia (Zhang et al., 2016), and depressive symptoms (Whittle et al., 2014), and teasing apart more subtle features of OFC morphology may enhance our ability to predict disease. Additionally, these secondary sulcal features may contribute to the OFC’s segmentation of cellular architecture and connectivity (Kahnt et al., 2012, Brodmann, 1909, Walker, 1940, Ongür et al., 2003, Carmichael and Price, 1996, Uylings et al., 2010). These investigations would require more data to further establish reliable secondary features but have potential to further describe the individual manifestation of pathology.

Improving upon manual methods will lead to more objective classifications of OFC sulcogyral patterns. Raters for OFC pattern classification are trained under criteria that may leave room for subjectivity, and having multiple raters can reduce potential effects of subjectivity. Establishing a fully automated tracing and classification procedure would allow for any criteria internal to the automated process to be objectively and consistently followed for each classification. Because the OFC is prone to dropout in T1-weighted scans and because its sulci have high inter-individual variability, more work and the creation of more tracing data will allow us to reach a fully automated OFC tracing and sulcogyral classification procedure.

This study is not without limitations. Research in sulcogyral pattern identification typically begins with qualitative descriptions of the sulci (see Chiavaras and Petrides (2000)), and repeated identification of brains that fit these qualitative descriptions (such as in previously mentioned radiological studies, and in the following postmortem human studies: Rodrigues et al., 2015; Sarkar et al., 2016) support an objective truth to stereotyped cortical folds. However, even when these descriptions are translated into more objective criteria, a degree of subjectivity remains when making judgements of both where to trace OFC sulci and how to characterize the tracing’s pattern type. We compared reliability of pattern type characterization with respect to characterizations used in published work. Although we consider these characterizations the “accepted” characterizations, they are still based on human, and therefore subjective, judgements. Rigorous criteria for characterization judgements has allowed for a robust science of sulcogyral pattern typing, and we aim to facilitate more objectivity using these automated tracings.

Highlights.

• Automatic tracings were informative for characterization of OFC sulcogyral pattern

• Automated tracings can replace manual tracings in a semi-automated pipeline

• The semi-automated pipeline can promote larger investigations of the OFC