We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison to common multi-atlas segmentation, while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with highly variable field-of-view.
PURPOSE: Multimodal imaging plays a key role in patient assessment and treatment planning in liver radioembolization. It will reach its full potential for convenient use in combination with deformable image registration methods. A registration framework is proposed for multimodal liver image registration of multi-phase CT, contrast-enhanced late-phase T1, T2, and DWI MRI sequences. METHODS: A chain of four pair-wise image registrations based on a variational registration framework using normalized gradient fields as distance measure and curvature regularization is introduced. A total of 103 cases of 35 patients was evaluated based on anatomical landmarks and deformation characteristics. RESULTS: Good anatomical correspondence and physical plausibility of the deformation fields were attained. The global mean landmark errors vary from 3.20 to 5.36 mm, strongly influenced by low resolved images in z-direction. Moderate volume changes are indicated by mean minimum and maximum Jacobian determinants of 0.44 up to 1.88. No deformation foldings were detected. The mean average divergence of the deformation fields range from 0.08 to 0.16 and the mean harmonic energies vary from 0.08 to 0.58. CONCLUSION: The proposed registration solutions enable the combined use of information from multimodal imaging and provide an excellent basis for patient assessment and primary planning for liver radioembolization.
We introduce Disease Knowledge Transfer (DKT), a novel technique for transferring biomarker information between related neurodegenerative diseases. DKT infers robust multimodal biomarker trajectories in rare neurodegenerative diseases even when only limited, unimodal data is available, by transferring information from larger multimodal datasets from common neurodegenerative diseases. DKT is a joint-disease generative model of biomarker progressions, which exploits biomarker relationships that are shared across diseases. Our proposed method allows, for the first time, the estimation of plausible biomarker trajectories in Posterior Cortical Atrophy (PCA), a rare neurodegenerative disease where only unimodal MRI data is available. For this we train DKT on a combined dataset containing subjects with two distinct diseases and sizes of data available: 1) a larger, multimodal typical AD (tAD) dataset from the TADPOLE Challenge, and 2) a smaller unimodal Posterior Cortical Atrophy (PCA) dataset from the Dementia Research Centre (DRC), for which only a limited number of Magnetic Resonance Imaging (MRI) scans are available. Although validation is challenging due to lack of data in PCA, we validate DKT on synthetic data and two patient datasets (TADPOLE and PCA cohorts), showing it can estimate the ground truth parameters in the simulation and predict unseen biomarkers on the two patient datasets. While we demonstrated DKT on Alzheimer's variants, we note DKT is generalisable to other forms of related neurodegenerative diseases. Source code for DKT is available online: https://github.com/mrazvan22/dkt.
The performance and diagnostic utility of magnetic resonance imaging (MRI) in pregnancy is fundamentally constrained by fetal motion. Motion of the fetus, which is unpredictable and rapid on the scale of conventional imaging times, limits the set of viable acquisition techniques to single-shot imaging with severe compromises in signal-to-noise ratio and diagnostic contrast, and frequently results in unacceptable image quality. Surprisingly little is known about the characteristics of fetal motion during MRI and here we propose and demonstrate methods that exploit a growing repository of MRI observations of the gravid abdomen that are acquired at low spatial resolution but relatively high temporal resolution and over long durations (10-30 minutes). We estimate fetal pose per frame in MRI volumes of the pregnant abdomen via deep learning algorithms that detect key fetal landmarks. Evaluation of the proposed method shows that our framework achieves quantitatively an average error of 4.47 mm and 96.4% accuracy (with error less than 10 mm). Fetal pose estimation in MRI time series yields novel means of quantifying fetal movements in health and disease, and enables the learning of kinematic models that may enhance prospective mitigation of fetal motion artifacts during MRI acquisition.
We propose and demonstrate a joint model of anatomical shapes, image features and clinical indicators for statistical shape modeling and medical image analysis. The key idea is to employ a copula model to separate the joint dependency structure from the marginal distributions of variables of interest. This separation provides flexibility on the assumptions made during the modeling process. The proposed method can handle binary, discrete, ordinal and continuous variables. We demonstrate a simple and efficient way to include binary, discrete and ordinal variables into the modeling. We build Bayesian conditional models based on observed partial clinical indicators, features or shape based on Gaussian processes capturing the dependency structure. We apply the proposed method on a stroke dataset to jointly model the shape of the lateral ventricles, the spatial distribution of the white matter hyperintensity associated with periventricular white matter disease, and clinical indicators. The proposed method yields interpretable joint models for data exploration and patient-specific statistical shape models for medical image analysis.
We present a volumetric mesh-based algorithm for flattening the placenta to a canonical template to enable effective visualization of local anatomy and function. Monitoring placental function promises to support pregnancy assessment and to improve care outcomes. We aim to alleviate visualization and interpretation challenges presented by the shape of the placenta when it is attached to the curved uterine wall. To do so, we flatten the volumetric mesh that captures placental shape to resemble the well-studied shape. We formulate our method as a map from the shape to a flattened template that minimizes the symmetric Dirichlet energy to control distortion throughout the volume. Local injectivity is enforced via constrained line search during gradient descent. We evaluate the proposed method on 28 placenta shapes extracted from MRI images in a clinical study of placental function. We achieve sub-voxel accuracy in mapping the boundary of the placenta to the template while successfully controlling distortion throughout the volume. We illustrate how the resulting mapping of the placenta enhances visualization of placental anatomy and function. Our implementation is freely available at https://github.com/mabulnaga/placenta-flattening.
The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge compares the performance of algorithms at predicting the future evolution of individuals at risk of Alzheimer's disease. TADPOLE Challenge participants train their models and algorithms on historical data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. Participants are then required to make forecasts of three key outcomes for ADNI-3 rollover participants: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog 13), and total volume of the ventricles - which are then compared with future measurements. Strong points of the challenge are that the test data did not exist at the time of forecasting (it was acquired afterwards), and that it focuses on the challenging problem of cohort selection for clinical trials by identifying fast progressors. The submission phase of TADPOLE was open until 15 November 2017; since then data has been acquired until April 2019 from 219 subjects with 223 clinical visits and 150 Magnetic Resonance Imaging (MRI) scans, which was used for the evaluation of the participants' predictions. Thirty-three teams participated with a total of 92 submissions. No single submission was best at predicting all three outcomes. For diagnosis prediction, the best forecast (team Frog), which was based on gradient boosting, obtained a multiclass area under the receiver-operating curve (MAUC) of 0.931, while for ventricle prediction the best forecast (team ), which was based on disease progression modelling and spline regression, obtained mean absolute error of 0.41% of total intracranial volume (ICV). For ADAS-Cog 13, no forecast was considerably better than the benchmark mixed effects model ( ), provided to participants before the submission deadline. Further analysis can help understand which input features and algorithms are most suitable for Alzheimer's disease prediction and for aiding patient stratification in clinical trials. The submission system remains open via the website: https://tadpole.grand-challenge.org/.
Probabilistic atlas priors have been commonly used to derive adaptive and robust brain MRI segmentation algorithms. Widely-used neuroimage analysis pipelines rely heavily on these techniques, which are often computationally expensive. In contrast, there has been a recent surge of approaches that leverage deep learning to implement segmentation tools that are computationally efficient at test time. However, most of these strategies rely on learning from manually annotated images. These supervised deep learning methods are therefore sensitive to the intensity profiles in the training dataset. To develop a deep learning-based segmentation model for a new image dataset (e.g., of different contrast), one usually needs to create a new labeled training dataset, which can be prohibitively expensive, or rely on suboptimal adaptation or augmentation approaches. In this paper, we propose an alternative strategy that combines a conventional probabilistic atlas-based segmentation with deep learning, enabling one to train a segmentation model for new MRI scans without the need for any manually segmented images. Our experiments include thousands of brain MRI scans and demonstrate that the proposed method achieves good accuracy for a brain MRI segmentation task for different MRI contrasts, requiring only approximately 15 seconds at test time on a GPU.
We present a deep learning tractography segmentation method that allows fast and consistent white matter fiber tract identification across healthy and disease populations and across multiple diffusion MRI (dMRI) acquisitions. We create a large-scale training tractography dataset of 1 million labeled fiber samples (54 anatomical tracts are included). To discriminate between fibers from different tracts, we propose a novel 2D multi-channel feature descriptor (FiberMap) that encodes spatial coordinates of points along each fiber. We learn a CNN tract classification model based on FiberMap and obtain a high tract classification accuracy of 90.99%. The method is evaluated on a test dataset of 374 dMRI scans from three independently acquired populations across health conditions (healthy control, neuropsychiatric disorders, and brain tumor patients). We perform comparisons with two state-of-the-art white matter tract segmentation methods. Experimental results show that our method obtains a highly consistent segmentation result, where over 99% of the fiber tracts are successfully detected across all subjects under study, most importantly, including patients with space occupying brain tumors. The proposed method leverages deep learning techniques and provides a much faster and more efficient tool for large data analysis than methods using traditional machine learning techniques.
Cerebral microbleeds (CMBs), a common manifestation of mild traumatic brain injury (mTBI), have been sporadically implicated in the neurocognitive deficits of mTBI victims but their clinical significance has not been established adequately. Here we investigate the longitudinal effects of post-mTBI CMBs upon the fractional anisotropy (FA) of white matter (WM) in 21 older mTBI patients across the first ~6 months post-injury. CMBs were segmented automatically from susceptibility-weighted imaging (SWI) by leveraging the intensity gradient properties of SWI to identify CMB-related hypointensities using gradient-based edge detection. A detailed diffusion magnetic resonance imaging (dMRI) atlas of WM was used to segment and cluster tractography streamlines whose prototypes were then identified. The correlation coefficient was calculated between (A) FA values at vertices along streamline prototypes and (B) topological (along-streamline) distances between these vertices and the nearest CMB. Across subjects, the CMB identification approach achieved a sensitivity of 97.1% ± 4.7% and a precision of 72.4% ± 11.0% across subjects. The correlation coefficient was found to be negative and, additionally, statistically significant for 12.3% ± 3.5% of WM clusters (p <; 0.05, corrected), whose FA was found to decrease, on average, by 11.8% ± 5.3% across the first 6 months post-injury. These results suggest that CMBs can be associated with deleterious effects upon peri-lesional WM and highlight the vulnerability of older mTBI patients to neurovascular injury.
OBJECTIVE: Deep brain stimulation (DBS) in zona incerta (Zi) is used for symptom alleviation in essential tremor (ET). Zi is positioned along the dentato-rubro-thalamic tract (DRT). Electric field simulations with the finite element method (FEM) can be used for estimation of a volume where the stimulation affects the tissue by applying a fixed isolevel (V). This work aims to develop a workflow for combined patient-specific electric field simulation and white matter tracing of the DRT, and to investigate the influence on the V from different brain tissue models, lead design and stimulation modes. The novelty of this work lies in the combination of all these components. METHOD: Patients with ET were implanted in Zi (lead 3389, n = 3, voltage mode; directional lead 6172, n = 1, current mode). Probabilistic reconstruction from diffusion MRI (dMRI) of the DRT (n = 8) was computed with FSL Toolbox. Brain tissue models were created for each patient (two homogenous, one heterogenous isotropic, one heterogenous anisotropic) and the respective V (n = 48) calculated from the Comsol Multiphysics FEM simulations. The DRT and V were visualized with 3DSlicer and superimposed on the preoperative T2 MRI, and the common volumes calculated. Dice Coefficient (DC) and level of anisotropy were used to evaluate and compare the brain models. RESULT: Combined patient-specific tractography and electric field simulation was designed and evaluated, and all patients showed benefit from DBS. All V overlapped the reconstructed DRT. Current stimulation showed prominent difference between the tissue models, where the homogenous grey matter deviated most (67 < DC < 69). Result from heterogenous isotropic and anisotropic models were similar (DC > 0.95), however the anisotropic model consistently generated larger volumes related to a greater extension of the electric field along the DBS lead. Independent of tissue model, the steering effect of the directional lead was evident and consistent. CONCLUSION: A workflow for patient-specific electric field simulations in combination with reconstruction of DRT was successfully implemented. Accurate tissue classification is essential for electric field simulations, especially when using the current control stimulation. With an accurate targeting and tractography reconstruction, directional leads have the potential to tailor the electric field into the desired region.
We address the problem of interpolating randomly non-uniformly spatiotemporally scattered uncertain motion measurements, which arises in the context of soft tissue motion estimation. Soft tissue motion estimation is of great interest in the field of image-guided soft-tissue intervention and surgery navigation, because it enables the registration of pre-interventional/pre-operative navigation information on deformable soft-tissue organs. To formally define the measurements as spatiotemporally scattered motion signal samples, we propose a novel motion field representation. To perform the interpolation of the motion measurements in an uncertainty-aware optimal unbiased fashion, we devise a novel Gaussian process (GP) regression model with a non-constant-mean prior and an anisotropic covariance function and show through an extensive evaluation that it outperforms the state-of-the-art GP models that have been deployed previously for similar tasks. The employment of GP regression enables the quantification of uncertainty in the interpolation result, which would allow the amount of uncertainty present in the registered navigation information governing the decisions of the surgeon or intervention specialist to be conveyed.
Schizophrenia has been characterized as a neurodevelopmental disorder, with structural brain abnormalities reported at all stages. However, at present, it remains unclear whether gray and white matter abnormalities represent related or independent pathologies in schizophrenia. In this study, we present findings from an integrative analysis exploring the morphological relationship between gray and white matter in 45 schizophrenia participants and 49 healthy controls. We utilized mutual information (MI), a measure of how much information two variables share, to assess the morphological dependence between gray and white matter in three segments of the corpus callsoum, and the gray matter regions these segments connect: (1) the genu and the left and right rostral middle frontal gyrus (rMFG), (2) the isthmus and the left and right superior temporal gyrus (STG), (3) the splenium and the left and right lateral occipital gyrus (LOG). We report significantly reduced MI between white matter tract dispersion of the right hemispheric callosal connections to the STG and both cortical thickness and area in the right STG in schizophrenia patients, despite a lack of group differences in cortical thickness, surface area, or dispersion. We believe that this reduction in morphological dependence between gray and white matter may reflect a possible decoupling of the developmental processes that shape morphological features of white and gray matter early in life. The present study also demonstrates the importance of studying the relationship between gray and white matter measures, as opposed to restricting analyses to gray and white matter measures independently.
RATIONALE AND OBJECTIVES: Analysis of dynamic contrast-enhanced (DCE) magnetic resonance imaging is notable for the variability of calculated parameters. The purpose of this study was to evaluate the level of measurement variability and error/variability due to modeling in DCE magnetic resonance imaging parameters. MATERIALS AND METHODS: Two prostate DCE scans were performed on 11 treatment-naïve patients with suspected or confirmed prostate peripheral zone cancer within an interval of less than two weeks. Tumor-suspicious and normal-appearing regions of interest (ROI) in the prostate peripheral zone were segmented. Different Tofts-Kety based models and different arterial input functions, with and without bolus arrival time (BAT) correction, were used to extract pharmacokinetic parameters. The percent repeatability coefficient (%RC) of fitted model parameters K, v, and k was calculated. Paired t-tests comparing parameters in tumor-suspicious ROIs and in normal-appearing tissue evaluated each parameter's sensitivity to pathology. RESULTS: Although goodness-of-fit criteria favored the four-parameter extended Tofts-Kety model with the BAT correction included, the simplest two-parameter Tofts-Kety model overall yielded the best repeatability scores. The best %RC in the tumor-suspicious ROI was 63% for k, 28% for v and 83% for K . The best p values for discrimination between tissues were p <10 for k and K, and p = 0.11 for v. Addition of the BAT correction to the models did not improve repeatability. CONCLUSION: The parameter k, using an arterial input functions directly measured from blood signals, was more repeatable than K. Both K and k values were highly discriminatory between healthy and diseased tissues in all cases. The parameter v had high repeatability but could not distinguish the two tissue types.
This paper presents an efficient approach to quantifying image registration uncertainty based on a low-dimensional representation of geometric deformations. In contrast to previous methods, we develop a Bayesian diffeomorphic registration framework in a bandlimited space, rather than a high-dimensional image space. We show that a dense posterior distribution on deformation fields can be fully characterized by much fewer parameters, which dramatically reduces the computational complexity of model inferences. To further avoid heavy computation loads introduced by random sampling algorithms, we approximate a marginal posterior by using Laplace's method at the optimal solution of log-posterior distribution. Experimental results on both 2D synthetic data and real 3D brain magnetic resonance imaging (MRI) scans demonstrate that our method is significantly faster than the state-of-the-art diffeomorphic registration uncertainty quantification algorithms, while producing comparable results.
Although brain functionality is often remarkably robust to lesions and other insults, it may be fragile when these take place in specific locations. Previous attempts to quantify robustness and fragility sought to understand how the functional connectivity of brain networks is affected by structural changes, using either model-based predictions or empirical studies of the effects of lesions. We advance a geometric viewpoint relying on a notion of network curvature, the so-called Ollivier-Ricci curvature. This approach has been proposed to assess financial market robustness and to differentiate biological networks of cancer cells from healthy ones. Here, we apply curvature-based measures to brain structural networks to identify robust and fragile brain regions in healthy subjects. We show that curvature can also be used to track changes in brain connectivity related to age and autism spectrum disorder (ASD), and we obtain results that are in agreement with previous MRI studies.
The Human Placenta Project has focused attention on the need for noninvasive magnetic resonance imaging (MRI)-based techniques to diagnose and monitor placental function throughout pregnancy. The hope is that the management of placenta-related pathologies would be improved if physicians had more direct, real-time measures of placental health to guide clinical decision making. As oxygen alters signal intensity on MRI and oxygen transport is a key function of the placenta, many of the MRI methods under development are focused on quantifying oxygen transport or oxygen content of the placenta. For example, measurements from blood oxygen level-dependent imaging of the placenta during maternal hyperoxia correspond to outcomes in twin pregnancies, suggesting that some aspects of placental oxygen transport can be monitored by MRI. Additional methods are being developed to accurately quantify baseline placental oxygenation by MRI relaxometry. However, direct validation of placental MRI methods is challenging and therefore animal studies and ex vivo studies of human placentas are needed. Here we provide an overview of the current state of the art of oxygen transport and quantification with MRI. We suggest that as these techniques are being developed, increased focus be placed on ensuring they are robust and reliable across individuals and standardized to enable predictive diagnostic models to be generated from the data. The field is still several years away from establishing the clinical benefit of monitoring placental function in real time with MRI, but the promise of individual personalized diagnosis and monitoring of placental disease in real time continues to motivate this effort.
Estimating the uncertainty in (probabilistic) image registration enables, e.g., surgeons to assess the operative risk based on the trustworthiness of the registered image data. If surgeons receive inaccurately calculated registration uncertainty and misplace unwarranted confidence in the alignment solutions, severe consequences may result. For probabilistic image registration (PIR), the predominant way to quantify the registration uncertainty is using summary statistics of the distribution of transformation parameters. The majority of existing research focuses on trying out different summary statistics as well as means to exploit them. Distinctively, in this paper, we study two rarely examined topics: (1) whether those summary statistics of the transformation distribution most informatively represent the registration uncertainty; (2) Does utilizing the registration uncertainty always be beneficial. We show that there are two types of uncertainties: the transformation uncertainty, Ut, and label uncertainty Ul. The conventional way of using Ut to quantify Ul is inappropriate and can be misleading. By a real data experiment, we also share a potentially critical finding that making use of the registration uncertainty may not always be an improvement.
External-beam radiotherapy followed by high dose rate (HDR) brachytherapy is the standard-of-care for treating gynecologic cancers. The enhanced soft-tissue contrast provided by magnetic resonance imaging (MRI) makes it a valuable imaging modality for diagnosing and treating these cancers. However, in contrast to computed tomography (CT) imaging, the appearance of the brachytherapy catheters, through which radiation sources are inserted to reach the cancerous tissue later on, is often variable across images. This paper reports, for the first time, a new deep-learning-based method for fully automatic segmentation of multiple closely spaced brachytherapy catheters in intraoperative MRI. Represented in the data are 50 gynecologic cancer patients treated by MRI-guided HDR brachytherapy. For each patient, a single intraoperative MRI was used. 826 catheters in the images were manually segmented by an expert radiation physicist who is also a trained radiation oncologist. The number of catheters in a patient ranged between 10 and 35. A deep 3D convolutional neural network (CNN) model was developed and trained. In order to make the learning process more robust, the network was trained 5 times, each time using a different combination of shown patients. Finally, each test case was processed by the five networks and the final segmentation was generated by voting on the obtained five candidate segmentations. 4-fold validation was executed and all the patients were segmented. An average distance error of 2.0 ± 3.4 mm was achieved. False positive and false negative catheters were 6.7% and 1.5% respectively. Average Dice score was equal to 0.60 ± 0.17. The algorithm is available for use in the open source software platform 3D Slicer allowing for wide scale testing and research discussion. In conclusion, to the best of our knowledge, fully automatic segmentation of multiple closely spaced catheters from intraoperative MR images was achieved for the first time in gynecological brachytherapy.
Computational biomechanics of the brain for neurosurgery is an emerging area of research recently gaining in importance and practical applications. This review paper presents the contributions of the Intelligent Systems for Medicine Laboratory and its collaborators to this field, discussing the modeling approaches adopted and the methods developed for obtaining the numerical solutions. We adopt a physics-based modeling approach and describe the brain deformation in mechanical terms (such as displacements, strains, and stresses), which can be computed using a biomechanical model, by solving a continuum mechanics problem. We present our modeling approaches related to geometry creation, boundary conditions, loading, and material properties. From the point of view of solution methods, we advocate the use of fully nonlinear modeling approaches, capable of capturing very large deformations and nonlinear material behavior. We discuss finite element and meshless domain discretization, the use of the total Lagrangian formulation of continuum mechanics, and explicit time integration for solving both time-accurate and steady-state problems. We present the methods developed for handling contacts and for warping 3D medical images using the results of our simulations. We present two examples to showcase these methods: brain shift estimation for image registration and brain deformation computation for neuronavigation in epilepsy treatment.