Anatomic Variability Core

Polina Golland
Polina Golland, PhD
Core PI

Our Publications

The Anatomical Variability Core develops computational models of anatomical heterogeneity in large populations for the purpose of providing accurate priors for atlas-based segmentation of anatomical structures in neuroimaging. Segmentation is particularly challenging if the shape of the anatomical structure of interest varies substantially in a population, such as would be observed in a progressive neurodegenerative disorder or cancer. Building computational models that capture highly variable anatomy and using the information to improve image segmentation are the main objectives of this Core. The goal is to develop a new generation of robust segmentation tools capable of incorporating knowledge about pathology-induced anatomical variability. Such tools are essential to achieving progress in the patient-specific analysis of disease. The work of this core is organized around the following specific aims.

  1. Develop Models of Anatomical Heterogeneity in a Population to Provide Better Priors for Image Segmentation in Neuroimaging.
  2. Develop Computational Models of Local Anatomical Variability for Integration into Segmentation.
  3. Validate, Deploy, and Distribute the Segmentation Tools for Image Analysis in the Presence of Pathology.

Anatomical variability that arises as a consequence of pathology, such as the neural degeneration associated with stroke, brain tumors and ALzheimer’s disease is the clinical focus of our research, but the proposed methods will be broadly applicable to other domains of naturally occurring variability in the shape of anatomical organs, well beyond neuroimaging. 
The objective of our research is three-fold. Models of anatomical heterogeneity in a population are being developed in aim 1 to improve the availability of priors for image segmentation in neuroimaging. Our approach to modeling anatomical variability treats a heterogeneous population as a collection of relatively uniform sub-populations, each of which can be represented by a single training template in the segmentation algorithm. In aim 2, we are developing methods that explicitly account for registration uncertainty in building models of anatomical variability. Our approach assumes an atlas coordinate system into which all images are mapped and in which the local models of anatomical variability are constructed. The models we develop are complementary to the global variability models in Aim 1. We will explore a number of options for constructing local variability models and integrate them into segmentation. After validating the algorithms, we disseminate them to the broad medical image computing community in the form of segmentation tools to be used for image analysis.

Featured Technologies

Shape Priors for Cerebrovascular Disease Segmentation

AV-Figure 1

Identifying and differentiating pathologies is important for understanding the underlying mechanisms and clinical outcomes of cerebral ischemia. It is a central goal in our clinical stroke study collaboration that provides the first step in analysis and genetic association with disease. Manual delineation is infeasible in large studies of stroke that include thousands of patients. Unlike normal brain tissues and structures, the location and shape of the lesions vary significantly across patients, presenting serious challenges for prior-driven segmentation. Our generative model captures spatial patterns and intensity properties associated with different cerebrovascular pathologies in stroke patients, resulting in an inference algorithm for the automatic segmentation of cerebrovascular pathologies in clinical MR images of the brain.

Joint Modeling of Imaging and Genetic Variability


Imaging genetics studies the relationships between genetic variation and measurements from anatomical or functional imaging data, often in the context of a clinical disorder. Specifically, the use of imaging data for examining genetic associations promises new directions of analysis.
Our approach is to make use of a unified Bayesian framework for detecting genetic variants associated with disease by using image-based features as an intermediate phenotype. We jointly exploit information in the available data types, resulting in probabilistic measures of clinical relevance for both imaging and genetic markers. The resulting inference algorithm naturally handles the high dimensionality of image and genetic data to identify imaging genetic features associated with the disease.


In contrast to classical genetic correlation, we aim to predict the anatomy of a patient in subsequent scans following a single baseline image, using subject-specific genetics. Such predictions promise to facilitate novel analyses in both voxel-level studies and longitudinal biomarker evaluation.  Our approach involves a semi-parametric generative model that captures anatomical change through a combination of population-wide regression and the subject’s health based on individual genetic and clinical indicators. We provide prediction of follow-up anatomical scans in the ADNI cohort. We also explore novel analysis approaches that compare a patient’s scans to the predicted subject-specific healthy anatomical trajectory.

BrainPrint: Representing anatomical variability of cortical and subcortical structures


The shape information within ensembles of cortical and subcortical structures can be captured through novel representations. BrainPrint compactly represents shapes by solving the 2D and 3D Laplace-Beltrami operator on boundary (triangular) and volumetric (tetrahedral) meshes. BrainPrint is a discriminative representation that captures unique information about the subject’s anatomy. In particular, a robust classifier trained on BrainPrint representations from a database of brain scans can reliably identify the subject in a new scan. In an example dataset containing over 3000 MRI scans from the ADNI dataset, BrainPrint yields correct subject classification of a scan with 99.8% accuracy. We have also investigated applications to longitudinal studies and to a wide range of anatomical analyses commonly performed in clinical research. Computer-aided diagnosis of Alzeimer’s disease and it’s prodromal stage of mild cognitive impairment also benefits from using BrainPrint to characterize the shapes of brain structures. We derive features for automated diagnosis from BrainPrint by computing lateral shape differences and projections onto the principal components. Our approach is to perform classification using a generalized linear model that  includes regularization and automatic model selection. The resulting algorithm won the second prize at the ‘MICCAI 2014 Challenge on Computer-Aided Diagnosis of Dementia Based on Structural MRI Data’.

Image Imputation


Large databases of clinical images contain a wealth of information, yet medical acquisition constraints result in sparse scans that miss much of the anatomy. These characteristics often render computational analysis impractical as standard processing algorithms tend to fail when applied to such images. Our goal is to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. Our algorithm creates high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. The model captures fine-scale anatomical similarity across subjects in clinical image collections. We use it to fill in the missing data in scans with large slice spacing. This method promises to facilitate subsequent analysis not previously possible with scans of this quality.


We maintain an active collaboration with our colleagues in the Neurology Department of Mass General Hospital with the goal of analyzing clinical brain MRI scans of stroke patients to identify genetic associations of imaging biomarkers of cerebrovascular health. We also work with the National Center for Image Guided Therapy at the Brigham and Women’s Hospital to develop robust methods for segmentation and registration of brain tumor scans for surgical navigation. We have several joint projects with the Center for Functional Neuroimaging technologies at the Martinos center for Biomedical Imaging, Massachusetts General Hospital.

Outreach and Dissemination

We make our algorithms available to the broad medical image computing community by integrating them into open source software platforms such as 3D Slicer and releasing the code on github. When the computational solutions mature, we also work with our collaborators to integrate them into the clinical environment.

Our contributions to the medical image computing community go beyond scientific publications and associated open source implementations.. The members of the Core have organized and chaired the Brain Tumor Segmentation Challenge (BraTS) at MICCAI, the MICCAI Imaging Genetics Workshop, and the main MICCAI Conference. As part of NAC, we host the annual winter project weeks at MIT, where computer scientists, software developers, clinicians, and clinical researchers to work together towards better open source solutions to medical image computing.

Interactions with Other Cores

We collaborate closely with the Spatio-Temporal Modeling Core on feature-based anatomical models and on high dimensional inference from image data, with joint projects, thesis co-supervision, and joint publications.

We work with the Microstructure Imaging Core and the Slicer Core in our work to support the clinical collaborators in the National Center for Image Guided Therapy.


Adrian V Dalca, K. L. Bouman, William T. Freeman, Natalia S Rost, Mert R Sabuncu, and Polina Golland. 6/2017. “Population Based Image Imputation.” Inf Process Med Imaging., 10265, 659-71.
Christian Wachinger, Matthew Brennan, Greg C Sharp, and Polina Golland. 6/2017. “Efficient Descriptor-Based Segmentation of Parotid Glands With Nonlocal Means.” IEEE Trans Biomed Eng, 64, 7, Pp. 1492-1502.Abstract
OBJECTIVE: We introduce descriptor-based segmentation that extends existing patch-based methods by combining intensities, features, and location information. Since it is unclear which image features are best suited for patch selection, we perform a broad empirical study on a multitude of different features. METHODS: We extend nonlocal means segmentation by including image features and location information. We search larger windows with an efficient nearest neighbor search based on kd-trees. We compare a large number of image features. RESULTS: The best results were obtained for entropy image features, which have not yet been used for patch-based segmentation. We further show that searching larger image regions with an approximate nearest neighbor search and location information yields a significant improvement over the bounded nearest neighbor search traditionally employed in patch-based segmentation methods. CONCLUSION: Features and location information significantly increase the segmentation accuracy. The best features highlight boundaries in the image. SIGNIFICANCE: Our detailed analysis of several aspects of nonlocal means-based segmentation yields new insights about patch and neighborhood sizes together with the inclusion of location information. The presented approach advances the state-of-the-art in the segmentation of parotid glands for radiation therapy planning.
Yi Hong, Polina Golland, and Miaomiao Zhang. 9/2017. “Fast Geodesic Regression for Population-Based Image Analysis.” Int Conf Med Image Comput Comput Assist Interv. 20 (Pt1), Pp. 317-25.Abstract
Geodesic regression on images enables studies of brain development and degeneration, disease progression, and tumor growth. The high-dimensional nature of image data presents significant computational challenges for the current regression approaches and prohibits large scale studies. In this paper, we present a fast geodesic regression method that dramatically decreases the computational cost of the inference procedure while maintaining prediction accuracy. We employ an efficient low dimensional representation of diffeomorphic transformations derived from the image data and characterize the regressed trajectory in the space of diffeomorphisms by its initial conditions, i.e., an initial image template and an initial velocity field computed as a weighted average of pairwise diffeomorphic image registration results. This construction is achieved by using a first-order approximation of pairwise distances between images. We demonstrate the efficiency of our model on a set of 3D brain MRI scans from the OASIS dataset and show that it is dramatically faster than the state-of-the-art regression methods while producing equally good regression results on the large subject cohort.
Anne-Katrin Giese, Markus D Schirmer, Kathleen L Donahue, Lisa Cloonan, Robert Irie, Stefan Winzeck, Mark JRJ Bouts, Elissa C McIntosh, Steven J Mocking, Adrian V Dalca, Ramesh Sridharan, Huichun Xu, Petrea Frid, Eva Giralt-Steinhauer, Lukas Holmegaard, Jaume Roquer, Johan Wasselius, John W Cole, Patrick F McArdle, Joseph P Broderick, Jordi Jimenez-Conde, Christina Jern, Brett M Kissela, Dawn O Kleindorfer, Robin Lemmens, Arne Lindgren, James F Meschia, Tatjana Rundek, Ralph L Sacco, Reinhold Schmidt, Pankaj Sharma, Agnieszka Slowik, Vincent Thijs, Daniel Woo, Bradford B Worrall, Steven J Kittner, Braxton D Mitchell, Jonathan Rosand, Polina Golland, Ona Wu, and Natalia S Rost. 8/2017. “Design and Rationale for Examining Neuroimaging Genetics in Ischemic Stroke: The MRI-GENIE Study.” Neurol Genet, 3, 5, Pp. e180.Abstract
OBJECTIVE: To describe the design and rationale for the genetic analysis of acute and chronic cerebrovascular neuroimaging phenotypes detected on clinical MRI in patients with acute ischemic stroke (AIS) within the scope of the MRI-GENetics Interface Exploration (MRI-GENIE) study. METHODS: MRI-GENIE capitalizes on the existing infrastructure of the Stroke Genetics Network (SiGN). In total, 12 international SiGN sites contributed MRIs of 3,301 patients with AIS. Detailed clinical phenotyping with the web-based Causative Classification of Stroke (CCS) system and genome-wide genotyping data were available for all participants. Neuroimaging analyses include the manual and automated assessments of established MRI markers. A high-throughput MRI analysis pipeline for the automated assessment of cerebrovascular lesions on clinical scans will be developed in a subset of scans for both acute and chronic lesions, validated against gold standard, and applied to all available scans. The extracted neuroimaging phenotypes will improve characterization of acute and chronic cerebrovascular lesions in ischemic stroke, including CCS subtypes, and their effect on functional outcomes after stroke. Moreover, genetic testing will uncover variants associated with acute and chronic MRI manifestations of cerebrovascular disease. CONCLUSIONS: The MRI-GENIE study aims to develop, validate, and distribute the MRI analysis platform for scans acquired as part of clinical care for patients with AIS, which will lead to (1) novel genetic discoveries in ischemic stroke, (2) strategies for personalized stroke risk assessment, and (3) personalized stroke outcome assessment.
Miaomiao Zhang, William M Wells, and Polina Golland. 10/2017. “Probabilistic Modeling of Anatomical Variability using a Low Dimensional Parameterization of Diffeomorphisms.” Med Image Anal, 41, Pp. 55-62.Abstract
We present an efficient probabilistic model of anatomical variability in a linear space of initial velocities of diffeomorphic transformations and demonstrate its benefits in clinical studies of brain anatomy. To overcome the computational challenges of the high dimensional deformation-based descriptors, we develop a latent variable model for principal geodesic analysis (PGA) based on a low dimensional shape descriptor that effectively captures the intrinsic variability in a population. We define a novel shape prior that explicitly represents principal modes as a multivariate complex Gaussian distribution on the initial velocities in a bandlimited space. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than the state-of-the-art method such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA) that operate in the high dimensional image space.
Georg Langs, Danhong Wang, Polina Golland, Sophia Mueller, Ruiqi Pan, Mert R Sabuncu, Wei Sun, Kuncheng Li, and Hesheng Liu. 2016. “Identifying Shared Brain Networks in Individuals by Decoupling Functional and Anatomical Variability.” Cereb Cortex, 26, 10, Pp. 4004-14.Abstract
The connectivity architecture of the human brain varies across individuals. Mapping functional anatomy at the individual level is challenging, but critical for basic neuroscience research and clinical intervention. Using resting-state functional connectivity, we parcellated functional systems in an "embedding space" based on functional characteristics common across the population, while simultaneously accounting for individual variability in the cortical distribution of functional units. The functional connectivity patterns observed in resting-state data were mapped in the embedding space and the maps were aligned across individuals. A clustering algorithm was performed on the aligned embedding maps and the resulting clusters were transformed back to the unique anatomical space of each individual. This novel approach identified functional systems that were reproducible within subjects, but were distributed across different anatomical locations in different subjects. Using this approach for intersubject alignment improved the predictability of individual differences in language laterality when compared with anatomical alignment alone. Our results further revealed that the strength of association between function and macroanatomy varied across the cortex, which was strong in unimodal sensorimotor networks, but weak in association networks.
Adrian V Dalca, Andreea Bobu, Natalia S Rost, and Polina Golland. 10/2016. “Patch-Based Discrete Registration of Clinical Brain Images.” Patch Based Tech Med Imaging, 9993, Pp. 60-67.Abstract
We introduce a method for registration of brain images acquired in clinical settings. The algorithm relies on three-dimensional patches in a discrete registration framework to estimate correspondences. Clinical images present significant challenges for computational analysis. Fast acquisition often results in images with sparse slices, severe artifacts, and variable fields of view. Yet, large clinical datasets hold a wealth of clinically relevant information. Despite significant progress in image registration, most algorithms make strong assumptions about the continuity of image data, failing when presented with clinical images that violate these assumptions. In this paper, we demonstrate a non-rigid registration method for aligning such images. The method explicitly models the sparsely available image information to achieve robust registration. We demonstrate the algorithm on clinical images of stroke patients. The proposed method outperforms state of the art registration algorithms and avoids catastrophic failures often caused by these images. We provide a freely available open source implementation of the algorithm.
Miaomiao Zhang, William M Wells, and Polina Golland. 10/2016. “Low-Dimensional Statistics of Anatomical Variability via Compact Representation of Image Deformations.” Med Image Comput Comput Assist Interv, 9902, Pp. 166-73.Abstract
Using image-based descriptors to investigate clinical hypotheses and therapeutic implications is challenging due to the notorious "curse of dimensionality" coupled with a small sample size. In this paper, we present a low-dimensional analysis of anatomical shape variability in the space of diffeomorphisms and demonstrate its benefits for clinical studies. To combat the high dimensionality of the deformation descriptors, we develop a probabilistic model of principal geodesic analysis in a bandlimited low-dimensional space that still captures the underlying variability of image data. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than models based on the high-dimensional state-of-the-art approaches such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA).
Polina Binder, Nematollah K Batmanghelich, Raul San Jose Estepar, and Polina Golland. 10/2016. “Unsupervised Discovery of Emphysema Subtypes in a Large Clinical Cohort.” Mach Learn Med Imaging, 10019, Pp. 180-7.Abstract

Emphysema is one of the hallmarks of Chronic Obstructive Pulmonary Disorder (COPD), a devastating lung disease often caused by smoking. Emphysema appears on Computed Tomography (CT) scans as a variety of textures that correlate with disease subtypes. It has been shown that the disease subtypes and textures are linked to physiological indicators and prognosis, although neither is well characterized clinically. Most previous computational approaches to modeling emphysema imaging data have focused on supervised classification of lung textures in patches of CT scans. In this work, we describe a generative model that jointly captures heterogeneity of disease subtypes and of the patient population. We also describe a corresponding inference algorithm that simultaneously discovers disease subtypes and population structure in an unsupervised manner. This approach enables us to create image-based descriptors of emphysema beyond those that can be identified through manual labeling of currently defined phenotypes. By applying the resulting algorithm to a large data set, we identify groups of patients and disease subtypes that correlate with distinct physiological indicators.

Ruizhi Liao, Esra A. Turk, Miaomiao Zhang, Jie Luo, P. Ellen Grant, Elfar Adalsteinsson, and Polina Golland. 10/2016. “Temporal Registration in In-Utero Volumetric MRI Time Series.” In Int Conf Med Image Comput Comput Assist Interv, 19: Pp. 54-62.Abstract

We present a robust method to correct for motion and deformations in in-utero volumetric MRI time series. Spatio-temporal analysis of dynamic MRI requires robust alignment across time in the presence of substantial and unpredictable motion. We make a Markov assumption on the nature of deformations to take advantage of the temporal structure in the image data. Forward message passing in the corresponding hidden Markov model (HMM) yields an estimation algorithm that only has to account for relatively small motion between consecutive frames. We demonstrate the utility of the temporal model by showing that its use improves the accuracy of the segmentation propagation through temporal registration. Our results suggest that the proposed model captures accurately the temporal dynamics of deformations in in-utero MRI time series.

Miaomiao Zhang and Polina Golland. 10/2016. “Statistical Shape Analysis: From Landmarks to Diffeomorphisms.” Med Image Anal, 33, Pp. 155-8.Abstract

We offer a blazingly brief review of evolution of shape analysis methods in medical imaging. As the representations and the statistical models grew more sophisticated, the problem of shape analysis has been gradually redefined to accept images rather than binary segmentations as a starting point. This transformation enabled shape analysis to take its rightful place in the arsenal of tools for extracting and understanding patterns in large clinical image sets. We speculate on the future developments in shape analysis and potential applications that would bring this mathematically rich area to bear on clinical practice.

Bjoern H Menze, Koen Van Leemput, Danial Lashkari, Tammy Riklin-Raviv, Ezequiel Geremia, Esther Alberts, Philipp Gruber, Susanne Wegener, Marc-Andre Weber, Gabor Szekely, Nicholas Ayache, and Polina Golland. 4/2016. “A Generative Probabilistic Model and Discriminative Extensions for Brain Lesion Segmentation - with Application to Tumor and Stroke.” IEEE Trans Med Imaging, 35, 4, Pp. 933-46.Abstract

We introduce a generative probabilistic model for segmentation of brain lesions in multi-dimensional images that generalizes the EM segmenter, a common approach for modelling brain images using Gaussian mixtures and a probabilistic tissue atlas that employs expectation-maximization (EM), to estimate the label map for a new image. Our model augments the probabilistic atlas of the healthy tissues with a latent atlas of the lesion. We derive an estimation algorithm with closed-form EM update equations. The method extracts a latent atlas prior distribution and the lesion posterior distributions jointly from the image data. It delineates lesion areas individually in each channel, allowing for differences in lesion appearance across modalities, an important feature of many brain tumor imaging sequences. We also propose discriminative model extensions to map the output of the generative model to arbitrary labels with semantic and biological meaning, such as "tumor core" or "fluid-filled structure", but without a one-to-one correspondence to the hypo- or hyper-intense lesion areas identified by the generative model. We test the approach in two image sets: the publicly available BRATS set of glioma patient scans, and multimodal brain images of patients with acute and subacute ischemic stroke. We find the generative model that has been designed for tumor lesions to generalize well to stroke images, and the extended discriminative-discriminative model to be one of the top ranking methods in the BRATS evaluation.

Nematollah K Batmanghelich, Adrian Dalca, Gerald Quon, Mert Sabuncu, and Polina Golland. 7/2016. “Probabilistic Modeling of Imaging, Genetics and Diagnosis.” IEEE Trans Med Imaging, 35, 7, Pp. 1765-79.Abstract

We propose a unified Bayesian framework for detecting genetic variants associated with disease by exploiting image-based features as an intermediate phenotype. The use of imaging data for examining genetic associations promises new directions of analysis, but currently the most widely used methods make sub-optimal use of the richness that these data types can offer. Currently, image features are most commonly selected based on their relevance to the disease phenotype. Then, in a separate step, a set of genetic variants is identified to explain the selected features. In contrast, our method performs these tasks simultaneously in order to jointly exploit information in both data types. The analysis yields probabilistic measures of clinical relevance for both imaging and genetic markers. We derive an efficient approximate inference algorithm that handles the high dimensionality of image and genetic data. We evaluate the algorithm on synthetic data and demonstrate that it outperforms traditional models. We also illustrate our method on Alzheimer's Disease Neuroimaging Initiative data.

Danielle F Pace, Adrian V Dalca, Tal Geva, Andrew J Powell, Mehdi H Moghari, and Polina Golland. 2015. “Interactive Whole-Heart Segmentation in Congenital Heart Disease.” Med Image Comput Comput Assist Interv, 9351, Pp. 80-8.Abstract
We present an interactive algorithm to segment the heart chambers and epicardial surfaces, including the great vessel walls, in pediatric cardiac MRI of congenital heart disease. Accurate whole-heart segmentation is necessary to create patient-specific 3D heart models for surgical planning in the presence of complex heart defects. Anatomical variability due to congenital defects precludes fully automatic atlas-based segmentation. Our interactive segmentation method exploits expert segmentations of a small set of short-axis slice regions to automatically delineate the remaining volume using patch-based segmentation. We also investigate the potential of active learning to automatically solicit user input in areas where segmentation error is likely to be high. Validation is performed on four subjects with double outlet right ventricle, a severe congenital heart defect. We show that strategies asking the user to manually segment regions of interest within short-axis slices yield higher accuracy with less user input than those querying entire short-axis slices.
Christian Wachinger, Polina Golland, Caroline Magnain, Bruce Fischl, and Martin Reuter. 2015. “Multi-modal Robust Inverse-consistent Linear Registration.” Hum Brain Mapp, 36, 4, Pp. 1365-80.Abstract
Registration performance can significantly deteriorate when image regions do not comply with model assumptions. Robust estimation improves registration accuracy by reducing or ignoring the contribution of voxels with large intensity differences, but existing approaches are limited to monomodal registration. In this work, we propose a robust and inverse-consistent technique for cross-modal, affine image registration. The algorithm is derived from a contextual framework of image registration. The key idea is to use a modality invariant representation of images based on local entropy estimation, and to incorporate a heteroskedastic noise model. This noise model allows us to draw the analogy to iteratively reweighted least squares estimation and to leverage existing weighting functions to account for differences in local information content in multimodal registration. Furthermore, we use the nonparametric windows density estimator to reliably calculate entropy of small image patches. Finally, we derive the Gauss-Newton update and show that it is equivalent to the efficient second-order minimization for the fully symmetric registration approach. We illustrate excellent performance of the proposed methods on datasets containing outliers for alignment of brain tumor, full head, and histology images.
Christian Wachinger, Polina Golland, William Kremen, Bruce Fischl, Martin Reuter, and Martin Reuter. 2015. “BrainPrint: A Discriminative Characterization of Brain Morphology.” Neuroimage, 109, Pp. 232-48.Abstract
We introduce BrainPrint, a compact and discriminative representation of brain morphology. BrainPrint captures shape information of an ensemble of cortical and subcortical structures by solving the eigenvalue problem of the 2D and 3D Laplace-Beltrami operator on triangular (boundary) and tetrahedral (volumetric) meshes. This discriminative characterization enables new ways to study the similarity between brains; the focus can either be on a specific brain structure of interest or on the overall brain similarity. We highlight four applications for BrainPrint in this article: (i) subject identification, (ii) age and sex prediction, (iii) brain asymmetry analysis, and (iv) potential genetic influences on brain morphology. The properties of BrainPrint require the derivation of new algorithms to account for the heterogeneous mix of brain structures with varying discriminative power. We conduct experiments on three datasets, including over 3000 MRI scans from the ADNI database, 436 MRI scans from the OASIS dataset, and 236 MRI scans from the VETSA twin study. All processing steps for obtaining the compact representation are fully automated, making this processing framework particularly attractive for handling large datasets.
Christian Wachinger, Karl Fritscher, Greg Sharp, and Polina Golland. 2015. “Contour-Driven Atlas-Based Segmentation.” IEEE Trans Med Imaging, 34, 12, Pp. 2492-505.Abstract
We propose new methods for automatic segmentation of images based on an atlas of manually labeled scans and contours in the image. First, we introduce a Bayesian framework for creating initial label maps from manually annotated training images. Within this framework, we model various registration- and patch-based segmentation techniques by changing the deformation field prior. Second, we perform contour-driven regression on the created label maps to refine the segmentation. Image contours and image parcellations give rise to non-stationary kernel functions that model the relationship between image locations. Setting the kernel to the covariance function in a Gaussian process establishes a distribution over label maps supported by image structures. Maximum a posteriori estimation of the distribution over label maps conditioned on the outcome of the atlas-based segmentation yields the refined segmentation. We evaluate the segmentation in two clinical applications: the segmentation of parotid glands in head and neck CT scans and the segmentation of the left atrium in cardiac MR angiography images.
Nematollah K Batmanghelich, Ardavan Saeedi, Michael Cho, Raul San Jose Estepar, and Polina Golland. 2015. “Generative Method to Discover Genetically Driven Image Biomarkers.” Inf Process Med Imaging, 24, Pp. 30-42.Abstract
We present a generative probabilistic approach to discovery of disease subtypes determined by the genetic variants. In many diseases, multiple types of pathology may present simultaneously in a patient, making quantification of the disease challenging. Our method seeks common co-occurring image and genetic patterns in a population as a way to model these two different data types jointly. We assume that each patient is a mixture of multiple disease subtypes and use the joint generative model of image and genetic markers to identify disease subtypes guided by known genetic influences. Our model is based on a variant of the so-called topic models that uncover the latent structure in a collection of data. We derive an efficient variational inference algorithm to extract patterns of co-occurrence and to quantify the presence of heterogeneous disease processes in each patient. We evaluate the method on simulated data and illustrate its use in the context of Chronic Obstructive Pulmonary Disease (COPD) to characterize the relationship between image and genetic signatures of COPD subtypes in a large patient cohort.
Christian Wachinger and Polina Golland. 2015. “Sampling from Determinantal Point Processes for Scalable Manifold Learning.” Inf Process Med Imaging, 24, Pp. 687-98.Abstract
High computational costs of manifold learning prohibit its application for large datasets. A common strategy to overcome this problem is to perform dimensionality reduction on selected landmarks and to successively embed the entire dataset with the Nyström method. The two main challenges that arise are: (i) the landmarks selected in non-Euclidean geometries must result in a low reconstruction error, (ii) the graph constructed from sparsely sampled landmarks must approximate the manifold well. We propose to sample the landmarks from determinantal distributions on non-Euclidean spaces. Since current determinantal sampling algorithms have the same complexity as those for manifold learning, we present an efficient approximation with linear complexity. Further, we recover the local geometry after the sparsification by assigning each landmark a local covariance matrix, estimated from the original point set. The resulting neighborhood selection .based on the Bhattacharyya distance improves the embedding of sparsely sampled manifolds. Our experiments show a significant performance improvement compared to state-of-the-art landmark selection techniques on synthetic and medical data.
George H Chen, Devavrat Shah, and Polina Golland. 2015. “A Latent Source Model for Patch-Based Image Segmentation.” Med Image Comput Comput Assist Interv, 9351, Pp. 140-8.Abstract

Despite the popularity and empirical success of patch-based nearest-neighbor and weighted majority voting approaches to medical image segmentation, there has been no theoretical development on when, why, and how well these nonparametric methods work. We bridge this gap by providing a theoretical performance guarantee for nearest-neighbor and weighted majority voting segmentation under a new probabilistic model for patch-based image segmentation. Our analysis relies on a new local property for how similar nearby patches are, and fuses existing lines of work on modeling natural imagery patches and theory for nonparametric classification. We use the model to derive a new patch-based segmentation algorithm that iterates between inferring local label patches and merging these local segmentations to produce a globally consistent image segmentation. Many existing patch-based algorithms arise as special cases of the new algorithm.

Georg Langs, Polina Golland, and Satrajit S Ghosh. 2015. “Predicting Activation Across Individuals with Resting-State Functional Connectivity Based Multi-Atlas Label Fusion.” Med Image Comput Comput Assist Interv, 9350, Pp. 313-20.Abstract

The alignment of brain imaging data for functional neuroimaging studies is challenging due to the discrepancy between correspondence of morphology, and equivalence of functional role. In this paper we map functional activation areas across individuals by a multi-atlas label fusion algorithm in a functional space. We learn the manifold of resting-state fMRI signals in each individual, and perform manifold alignment in an embedding space. We then transfer activation predictions from a source population to a target subject via multi-atlas label fusion. The cost function is derived from the aligned manifolds, so that the resulting correspondences are derived based on the similarity of intrinsic connectivity architecture. Experiments show that the resulting label fusion predicts activation evoked by various experiment conditions with higher accuracy than relying on morphological alignment. Interestingly, the distribution of this gain is distributed heterogeneously across the cortex, and across tasks. This offers insights into the relationship between intrinsic connectivity, morphology and task activation. Practically, the mechanism can serve as prior, and provides an avenue to infer task-related activation in individuals for whom only resting data is available.

Christian Wachinger, Matthew Toews, Georg Langs, William M Wells III, and Polina Golland. 6/2015. “Keypoint Transfer Segmentation.” Inf Process Med Imaging, 24, Pp. 233-45.Abstract

We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ label maps. We introduce generative models for the inference of keypoint labels and for image segmentation, where keypoint matches are treated as a latent random variable and are marginalized out as part of the algorithm. We report segmentation results for abdominal organs in whole-body CT and in contrast-enhanced CT images. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm's robustness enables the segmentation of scans with highly variable field-of-view.

Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, and et al. 10/2015. “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS).” IEEE Trans Med Imaging, 34, 10, Pp. 1993-2024.Abstract

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

Adrian V Dalca, Ramesh Sridharan, Mert R Sabuncu, and Polina Golland. 10/2015. “Predictive Modeling of Anatomy with Genetic and Clinical Data.” Med Image Comput Comput Assist Interv, 9351, Pp. 519-26.Abstract

We present a semi-parametric generative model for predicting anatomy of a patient in subsequent scans following a single baseline image. Such predictive modeling promises to facilitate novel analyses in both voxel-level studies and longitudinal biomarker evaluation. We capture anatomical change through a combination of population-wide regression and a non-parametric model of the subject's health based on individual genetic and clinical indicators. In contrast to classical correlation and longitudinal analysis, we focus on predicting new observations from a single subject observation. We demonstrate prediction of follow-up anatomical scans in the ADNI cohort, and illustrate a novel analysis approach that compares a patient's scans to the predicted subject-specific healthy anatomical trajectory.

Alexandra Woolgar, Polina Golland, and Stefan Bode. 2014. “Coping With Confounds in Multivoxel Pattern Analysis: What Should We Do About Reaction Time Differences? A Comment On Todd, Nystrom & Cohen 2013.” Neuroimage, 98, Pp. 506-12.Abstract
Multivoxel pattern analysis (MVPA) is a sensitive and increasingly popular method for examining differences between neural activation patterns that cannot be detected using classical mass-univariate analysis. Recently, Todd et al. ("Confounds in multivariate pattern analysis: Theory and rule representation case study", 2013, NeuroImage 77: 157-165) highlighted a potential problem for these methods: high sensitivity to confounds at the level of individual participants due to the use of directionless summary statistics. Unlike traditional mass-univariate analyses where confounding activation differences in opposite directions tend to approximately average out at group level, group level MVPA results may be driven by any activation differences that can be discriminated in individual participants. In Todd et al.'s empirical data, factoring out differences in reaction time (RT) reduced a classifier's ability to distinguish patterns of activation pertaining to two task rules. This raises two significant questions for the field: to what extent have previous multivoxel discriminations in the literature been driven by RT differences, and by what methods should future studies take RT and other confounds into account? We build on the work of Todd et al. and compare two different approaches to remove the effect of RT in MVPA. We show that in our empirical data, in contrast to that of Todd et al., the effect of RT on rule decoding is negligible, and results were not affected by the specific details of RT modelling. We discuss the meaning of and sensitivity for confounds in traditional and multivoxel approaches to fMRI analysis. We observe that the increased sensitivity of MVPA comes at a price of reduced specificity, meaning that these methods in particular call for careful consideration of what differs between our conditions of interest. We conclude that the additional complexity of the experimental design, analysis and interpretation needed for MVPA is still not a reason to favour a less sensitive approach.
Georg Langs, Andrew Sweet, Danial Lashkari, Yanmei Tie, Laura Rigolo, Alexandra J Golby, and Polina Golland. 2014. “Decoupling Function and Anatomy in Atlases of Functional Connectivity Patterns: Language Mapping in Tumor Patients.” Neuroimage, 103, Pp. 462-75.Abstract
In this paper we construct an atlas that summarizes functional connectivity characteristics of a cognitive process from a population of individuals. The atlas encodes functional connectivity structure in a low-dimensional embedding space that is derived from a diffusion process on a graph that represents correlations of fMRI time courses. The functional atlas is decoupled from the anatomical space, and thus can represent functional networks with variable spatial distribution in a population. In practice the atlas is represented by a common prior distribution for the embedded fMRI signals of all subjects. We derive an algorithm for fitting this generative model to the observed data in a population. Our results in a language fMRI study demonstrate that the method identifies coherent and functionally equivalent regions across subjects. The method also successfully maps functional networks from a healthy population used as a training set to individuals whose language networks are affected by tumors.