Atlases are crucial to imaging statistics as they enable the standardization of inter-subject and inter-population analyses. While existing atlas estimation methods based on fluid/elastic/diffusion registration yield high-quality results for the human brain, these deformation models do not extend to a variety of other challenging areas of neuroscience such as the anatomy of C. elegans worms and fruit flies. To this end, this work presents a general probabilistic deep network-based framework for atlas estimation and registration which can flexibly incorporate various deformation models and levels of keypoint supervision that can be applied to a wide class of model organisms. Of particular relevance, it also develops a deformable piecewise rigid atlas model which is regularized to preserve inter-observation distances between neighbors. These modeling considerations are shown to improve atlas construction and key-point alignment across a diversity of datasets with small sample sizes including neuron positions in C. elegans hermaphrodites, fluorescence microscopy of male C. elegans, and images of fruit fly wings. Code is accessible at https://github.com/amin-nejat/Deformable-Atlas.

Reconstructing 3D MR volumes from multiple motion-corrupted stacks of 2D slices has shown promise in imaging of moving subjects, e. g., fetal MRI. However, existing slice-to-volume reconstruction methods are time-consuming, especially when a high-resolution volume is desired. Moreover, they are still vulnerable to severe subject motion and when image artifacts are present in acquired slices. In this work, we present NeSVoR, a resolution-agnostic slice-to-volume reconstruction method, which models the underlying volume as a continuous function of spatial coordinates with implicit neural representation. To improve robustness to subject motion and other image artifacts, we adopt a continuous and comprehensive slice acquisition model that takes into account rigid inter-slice motion, point spread function, and bias fields. NeSVoR also estimates pixel-wise and slice-wise variances of image noise and enables removal of outliers during reconstruction and visualization of uncertainty. Extensive experiments are performed on both simulated and in vivo data to evaluate the proposed method. Results show that NeSVoR achieves state-of-the-art reconstruction quality while providing two to ten-fold acceleration in reconstruction times over the state-of-the-art algorithms.

Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multiple stacks of MR slices as a sequence. With the attention mechanism, our model automatically detects the relevance between slices and predicts the transformation of one slice using information from other slices. We also estimate the underlying 3D volume to assist slice-to-volume registration and update the volume and transformations alternately to improve accuracy. Results on synthetic data show that our method achieves lower registration error and better reconstruction quality compared with existing state-of-the-art methods. Experiments with real-world MRI data are also performed to demonstrate the ability of the proposed model to improve the quality of 3D reconstruction under severe fetal motion.

Measuring and understanding functional fetal brain development in utero is critical for the study of the developmental foundations of our cognitive abilities, possible early detection of disorders, and their prevention. Thalamocortical connections are an intricate component of shaping the cortical layout, but so far, only ex-vivo studies provide evidence of how axons enter the sub-plate and cortex during this highly dynamic phase. Evidence for normal in-utero development of the functional thalamocortical connectome in humans is missing. Here, we modeled fetal functional thalamocortical connectome development using in-utero functional magnetic resonance imaging in fetuses observed from 19th to 40th weeks of gestation (GW). We observed a peak increase of thalamocortical functional connectivity strength between 29th and 31st GW, right before axons establish synapses in the cortex. The cortico-cortical connectivity increases in a similar time window, and exhibits significant functional laterality in temporal-superior, -medial, and -inferior areas. Homologous regions exhibit overall similar mirrored connectivity profiles, but this similarity decreases during gestation giving way to a more diverse cortical interconnectedness. Our results complement the understanding of structural development of the human connectome and may serve as the basis for the investigation of disease and deviations from a normal developmental trajectory of connectivity development.

The human thalamus is a highly connected brain structure, which is key for the control of numerous functions and is involved in several neurological disorders. Recently, neuroimaging studies have increasingly focused on the volume and connectivity of the specific nuclei comprising this structure, rather than looking at the thalamus as a whole. However, accurate identification of cytoarchitectonically designed histological nuclei on standard in vivo structural MRI is hampered by the lack of image contrast that can be used to distinguish nuclei from each other and from surrounding white matter tracts. While diffusion MRI may offer such contrast, it has lower resolution and lacks some boundaries visible in structural imaging. In this work, we present a Bayesian segmentation algorithm for the thalamus. This algorithm combines prior information from a probabilistic atlas with likelihood models for both structural and diffusion MRI, allowing segmentation of 25 thalamic labels per hemisphere informed by both modalities. We present an improved probabilistic atlas, incorporating thalamic nuclei identified from histology and 45 white matter tracts surrounding the thalamus identified in ultra-high gradient strength diffusion imaging. We present a family of likelihood models for diffusion tensor imaging, ensuring compatibility with the vast majority of neuroimaging datasets that include diffusion MRI data. The use of these diffusion likelihood models greatly improves identification of nuclear groups versus segmentation based solely on structural MRI. Dice comparison of 5 manually identifiable groups of nuclei to ground truth segmentations show improvements of up to 10 percentage points. Additionally, our chosen model shows a high degree of reliability, with median test-retest Dice scores above 0.85 for four out of five nuclei groups, whilst also offering improved detection of differential thalamic involvement in Alzheimer's disease (AUROC 81.98%). The probabilistic atlas and segmentation tool will be made publicly available as part of the neuroimaging package FreeSurfer.

BACKGROUND AND OBJECTIVES: While chronological age is one of the most influential determinants of poststroke outcomes, little is known of the impact of neuroimaging-derived biological "brain age." We hypothesized that radiomics analyses of T2-FLAIR images texture would provide brain age estimates and that advanced brain age of patients with stroke will be associated with cardiovascular risk factors and worse functional outcomes.

METHODS: We extracted radiomics from T2-FLAIR images acquired during acute stroke clinical evaluation. Brain age was determined from brain parenchyma radiomics using an ElasticNet linear regression model. Subsequently, relative brain age (RBA), which expresses brain age in comparison with chronological age-matched peers, was estimated. Finally, we built a linear regression model of RBA using clinical cardiovascular characteristics as inputs and a logistic regression model of favorable functional outcomes taking RBA as input.

RESULTS: We reviewed 4,163 patients from a large multisite ischemic stroke cohort (mean age = 62.8 years, 42.0% female patients). T2-FLAIR radiomics predicted chronological ages (mean absolute error = 6.9 years, r = 0.81). After adjustment for covariates, RBA was higher and therefore described older-appearing brains in patients with hypertension, diabetes mellitus, a history of smoking, and a history of a prior stroke. In multivariate analyses, age, RBA, NIHSS, and a history of prior stroke were all significantly associated with functional outcome (respective adjusted odds ratios: 0.58, 0.76, 0.48, 0.55; all p-values < 0.001). Moreover, the negative effect of RBA on outcome was especially pronounced in minor strokes.

DISCUSSION: T2-FLAIR radiomics can be used to predict brain age and derive RBA. Older-appearing brains, characterized by a higher RBA, reflect cardiovascular risk factor accumulation and are linked to worse outcomes after stroke.

Standard image based segmentation approaches perform poorly when there is little or no contrast along boundaries of different regions. In such cases, segmentation is largely performed manually using prior knowledge of the shape and relative location of the underlying structures combined with partially discernible boundaries. We present an automated approach guided by covariant shape deformations of neighboring structures, which is an additional source of prior information. Captured by a shape atlas, these deformations are transformed into a statistical model using the logistic function. Structure boundaries, anatomical labels, and image inhomogeneities are estimated simultaneously within an Expectation-Maximization formulation of the maximum a posteriori probability estimation problem. We demonstrate the approach on 20 brain magnetic resonance images showing superior performance, particularly in cases where purely image based methods fail.

Open-source software provides an economic benefit by reducing duplicated development effort, and advances science knowledge by fostering a culture of reproducible experimentation. This paper describes recent advances in the Plastimatch open software suite, which implements a broad set of useful tools for research and practice in radiotherapy and medical imaging. The focus of this paper is to highlight recent advancements, including 2D-3D registration, GPU-accelerated mutual information, analytic regularization of B-spline registration, automatic 3D feature detection and feature matching, and radiotherapy plan evaluation tools.

We propose a novel approach to identify the foci of a neurological disorder based on anatomical and functional connectivity information. Specifically, we formulate a generative model that characterizes the network of abnormal functional connectivity emanating from the affected foci. This allows us to aggregate pairwise connectivity changes into a region-based representation of the disease. We employ the variational expectation-maximization algorithm to fit the model and subsequently identify both the afflicted regions and the differences in connectivity induced by the disorder. We demonstrate our method on a population study of schizophrenia.

We present a novel method for inferring tissue labels in atlas-based image segmentation using Gaussian process regression. Atlas-based segmentation results in probabilistic label maps that serve as input to our method. We introduce a contour-driven prior distribution over label maps to incorporate image features of the input scan into the label inference problem. The mean function of the Gaussian process posterior distribution yields the MAP estimate of the label map and is used in the subsequent voting. We demonstrate improved segmentation accuracy when our approach is combined with two different patch-based segmentation techniques. We focus on the segmentation of parotid glands in CT scans of patients with head and neck cancer, which is important for radiation therapy planning.

We present a method to detect epileptic regions based on functional connectivity differences between individual epilepsy patients and a healthy population. Our model assumes that the global functional characteristics of these differences are shared across patients, but it allows for the epileptic regions to vary between individuals. We evaluate the detection performance against intracranial EEG observations and compare our approach with two baseline methods that use standard statistics. The baseline techniques are sensitive to the choice of thresholds, whereas our algorithm automatically estimates the appropriate model parameters and compares favorably with the best baseline results. This suggests the promise of our approach for pre-surgical planning in epilepsy.

Manifold learning has been successfully applied to a variety of medical imaging problems. Its use in real-time applications requires fast projection onto the low-dimensional space. To this end, out-of-sample extensions are applied by constructing an interpolation function that maps from the input space to the low-dimensional manifold. Commonly used approaches such as the Nyström extension and kernel ridge regression require using all training points. We propose an interpolation function that only depends on a small subset of the input training data. Consequently, in the testing phase each new point only needs to be compared against a small number of input training data in order to project the point onto the low-dimensional space. We interpret our method as an out-of-sample extension that approximates kernel ridge regression. Our method involves solving a simple convex optimization problem and has the attractive property of guaranteeing an upper bound on the approximation error, which is crucial for medical applications. Tuning this error bound controls the sparsity of the resulting interpolation function. We illustrate our method in two clinical applications that require fast mapping of input images onto a low-dimensional space.

We propose a unified Bayesian framework for detecting genetic variants associated with a disease while exploiting image-based features as an intermediate phenotype. Traditionally, imaging genetics methods comprise two separate steps. First, image features are selected based on their relevance to the disease phenotype. Second, a set of genetic variants are identified to explain the selected features. In contrast, our method performs these tasks simultaneously to ultimately assign probabilistic measures of relevance to both genetic and imaging markers. We derive an efficient approximate inference algorithm that handles high dimensionality of imaging genetic data. We evaluate the algorithm on synthetic data and show that it outperforms traditional models. We also illustrate the application of the method on ADNI data.

We present an analysis framework for large studies of multimodal clinical quality brain image collections. Processing and analysis of such datasets is challenging due to low resolution, poor contrast, mis-aligned images, and restricted field of view. We adapt existing registration and segmentation methods and build a computational pipeline for spatial normalization and feature extraction. The resulting aligned dataset enables clinically meaningful analysis of spatial distributions of relevant anatomical features and of their evolution with age and disease progression. We demonstrate the approach on a neuroimaging study of stroke with more than 800 patients. We show that by combining data from several modalities, we can automatically segment important biomarkers such as white matter hyperintensity and characterize pathology evolution in this heterogeneous cohort. Specifically, we examine two sub-populations with different dynamics of white matter hyperintensity changes as a function of patients’ age. Pipeline and analysis code is available at http://groups.csail.mit.edu/vision/medical-vision/stroke/.

Intensity-based image registration requires resampling images on a common grid to evaluate the similarity function. The uncertainty of interpolation varies across the image, depending on the location of resampled points relative to the base grid. We propose to perform Bayesian inference with Gaussian processes, where the covariance matrix of the Gaussian process posterior distribution estimates the uncertainty in interpolation. The Gaussian process replaces a single image with a distribution over images that we integrate into a generative model for registration. Marginalization over resampled images leads to a new similarity measure that includes the uncertainty of the interpolation. We demonstrate that our approach increases the registration accuracy and propose an efficient approximation scheme that enables seamless integration with existing registration methods.

Introducing BrainPrint, a compact and discriminative representation of anatomical structures in the brain. BrainPrint captures shape information of an ensemble of cortical and subcortical structures by solving the 2D and 3D Laplace-Beltrami operator on triangular (boundary) and tetrahedral (volumetric) meshes. We derive a robust classifier for this representation that identifies the subject in a new scan, based on a database of brain scans. In an example dataset containing over 3000 MRI scans, we show that BrainPrint captures unique information about the subject’s anatomy and permits to correctly classify a scan with an accuracy of over 99.8%. All processing steps for obtaining the compact representation are fully automated making this processing framework particularly attractive for handling large datasets.

In this paper, we use Spherical Topic Models to discover the latent structure of lung disease. This method can be widely employed when a measurement for each subject is provided as a normalized histogram of relevant features. In this paper, the resulting descriptors are used as phenotypes to identify genetic markers associated with the Chronic Obstructive Pulmonary Disease (COPD). Features extracted from images capture the heterogeneity of the disease and therefore promise to improve detection of relevant genetic variants in Genome Wide Association Studies (GWAS). Our generative model is based on normalized histograms of image intensity of each subject and it can be readily extended to other forms of features as long as they are provided as normalized histograms. The resulting algorithm represents the intensity distribution as a combination of meaningful latent factors and mixing co-efficients that can be used for genetic association analysis. This approach is motivated by a clinical hypothesis that COPD symptoms are caused by multiple coexisting disease processes. Our experiments show that the new features enhance the previously detected signal on chromosome 15 with respect to standard respiratory and imaging measurements.

We study the widespread, but rarely discussed, tendency of atlas-based segmentation to under-segment the organs of interest. Commonly used error measures do not distinguish between under- and over-segmentation, contributing to the problem. We explicitly quantify over- and under-segmentation in several typical examples and present a new hypothesis for the cause. We provide evidence that segmenting only one organ of interest and merging all surrounding structures into one label creates bias towards background in the label estimates suggested by the atlas. We propose a generative model that corrects for this effect by learning the background structures from the data. Inference in the model separates the background into distinct structures and consequently improves the segmentation accuracy. Our experiments demonstrate a clear improvement in several applications.

We propose and demonstrate an inference algorithm for the automatic segmentation of cerebrovascular pathologies in clinical MR images of the brain. Identifying and differentiating pathologies is important for understanding the underlying mechanisms and clinical outcomes of cerebral ischemia. Manual delineation of separate pathologies is infeasible in large studies of stroke that include thousands of patients. Unlike normal brain tissues and structures, the location and shape of the lesions vary across patients, presenting serious challenges for prior-driven segmentation. Our generative model captures spatial patterns and intensity properties associated with different cerebrovascular pathologies in stroke patients. We demonstrate the resulting segmentation algorithm on clinical images of a stroke patient cohort.

In this paper we construct an atlas that summarizes functional connectivity characteristics of a cognitive process from a population of individuals. The atlas encodes functional connectivity structure in a low-dimensional embedding space that is derived from a diffusion process on a graph that represents correlations of fMRI time courses. The functional atlas is decoupled from the anatomical space, and thus can represent functional networks with variable spatial distribution in a population. In practice the atlas is represented by a common prior distribution for the embedded fMRI signals of all subjects. We derive an algorithm for fitting this generative model to the observed data in a population. Our results in a language fMRI study demonstrate that the method identifies coherent and functionally equivalent regions across subjects. The method also successfully maps functional networks from a healthy population used as a training set to individuals whose language networks are affected by tumors.

Multivoxel pattern analysis (MVPA) is a sensitive and increasingly popular method for examining differences between neural activation patterns that cannot be detected using classical mass-univariate analysis. Recently, Todd et al. ("Confounds in multivariate pattern analysis: Theory and rule representation case study", 2013, NeuroImage 77: 157-165) highlighted a potential problem for these methods: high sensitivity to confounds at the level of individual participants due to the use of directionless summary statistics. Unlike traditional mass-univariate analyses where confounding activation differences in opposite directions tend to approximately average out at group level, group level MVPA results may be driven by any activation differences that can be discriminated in individual participants. In Todd et al.’s empirical data, factoring out differences in reaction time (RT) reduced a classifier’s ability to distinguish patterns of activation pertaining to two task rules. This raises two significant questions for the field: to what extent have previous multivoxel discriminations in the literature been driven by RT differences, and by what methods should future studies take RT and other confounds into account? We build on the work of Todd et al. and compare two different approaches to remove the effect of RT in MVPA. We show that in our empirical data, in contrast to that of Todd et al., the effect of RT on rule decoding is negligible, and results were not affected by the specific details of RT modelling. We discuss the meaning of and sensitivity for confounds in traditional and multivoxel approaches to fMRI analysis. We observe that the increased sensitivity of MVPA comes at a price of reduced specificity, meaning that these methods in particular call for careful consideration of what differs between our conditions of interest. We conclude that the additional complexity of the experimental design, analysis and interpretation needed for MVPA is still not a reason to favour a less sensitive approach.

We present a semi-parametric generative model for predicting anatomy of a patient in subsequent scans following a single baseline image. Such predictive modeling promises to facilitate novel analyses in both voxel-level studies and longitudinal biomarker evaluation. We capture anatomical change through a combination of population-wide regression and a non-parametric model of the subject’s health based on individual genetic and clinical indicators. In contrast to classical correlation and longitudinal analysis, we focus on predicting new observations from a single subject observation. We demonstrate prediction of follow-up anatomical scans in the ADNI cohort, and illustrate a novel analysis approach that compares a patient’s scans to the predicted subject-specific healthy anatomical trajectory.

We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ label maps. We introduce generative models for the inference of keypoint labels and for image segmentation, where keypoint matches are treated as a latent random variable and are marginalized out as part of the algorithm. We report segmentation results for abdominal organs in whole-body CT and in contrast-enhanced CT images. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm’s robustness enables the segmentation of scans with highly variable field-of-view.