(
A-B) To determine the appropriate cluster neighborhood distance parameter, the fifth nearest neighbor distances for all nuclei in each wild-type brain are plotted in ascending order (right), following the convention in DBSCAN of taking the 2
·ndim−1th neighbor (
Schubert et al., 2017). (left) The zoomed view allows identification of the distance at the ‘elbow’ or ‘knee,’ or maximum curvature, typically at 20
µm or higher. (
C) Nuclei clustering within each label in the smoothed (edge-aware) atlas as a fraction of that of the original (mirrored) atlas across a range of distance values. The number of isolated, unclustered nuclei and total clusters, indicative of nuclei isolation and cluster fragmentation, respectively, decreased in the smoothed atlas relative to the original atlas, starting even prior to the minimum distance based on the max curvature seen in part ‘B.’ (
E) Example using a conservative neighbor distance of 20
µm for clustering in a brain before (top) and after (bottom) label refinement. Light gray lines show label boundaries. Unclustered nuclei are depicted as white points. All other points are clustered nuclei. Within each region, nuclei are colored by cluster size, where blue represents nuclei in the largest cluster, followed by orange, green, and red for the next successively smaller clusters. (
D) Differences in unclustered nuclei and (
E) number of clusters by region across all wild-type brains with this distance setting for selected regions across label hierarchies in bar plots (left) and as color gradients on an anatomical map (right).