Segmentation of brain tissue types from diffusion MRI (dMRI) is an important task, required for quantification of brain microstructure and for improving tractography. Current dMRI segmentation is mostly based on anatomical MRI (e.g., T1- and T2-weighted) segmentation that is registered to the dMRI space. However, such inter-modality registration is challenging due to more image distortions and lower image resolution in dMRI as compared with anatomical MRI. In this study, we present a deep learning method for diffusion MRI segmentation, which we refer to as DDSeg. Our proposed method learns tissue segmentation from high-quality imaging data from the Human Connectome Project (HCP), where registration of anatomical MRI to dMRI is more precise. The method is then able to predict a tissue segmentation directly from new dMRI data, including data collected with different acquisition protocols, without requiring anatomical data and inter-modality registration. We train a convolutional neural network (CNN) to learn a tissue segmentation model using a novel augmented target loss function designed to improve accuracy in regions of tissue boundary. To further improve accuracy, our method adds diffusion kurtosis imaging (DKI) parameters that characterize non-Gaussian water molecule diffusion to the conventional diffusion tensor imaging parameters. The DKI parameters are calculated from the recently proposed mean-kurtosis-curve method that corrects implausible DKI parameter values and provides additional features that discriminate between tissue types. We demonstrate high tissue segmentation accuracy on HCP data, and also when applying the HCP-trained model on dMRI data from other acquisitions with lower resolution and fewer gradient directions.
PURPOSE: To introduce, develop, and evaluate a novel denoising technique for diffusion MRI that leverages nonlinear redundancy in the data to boost the SNR while preserving signal information. METHODS: We exploit nonlinear redundancy of the dMRI data by means of kernel principal component analysis (KPCA), a nonlinear generalization of PCA to reproducing kernel Hilbert spaces. By mapping the signal to a high-dimensional space, a higher level of redundant information is exploited, thereby enabling better denoising than linear PCA. We implement KPCA with a Gaussian kernel, with parameters automatically selected from knowledge of the noise statistics, and validate it on realistic Monte Carlo simulations as well as with in vivo human brain submillimeter and low-resolution dMRI data. We also demonstrate KPCA denoising on multi-coil dMRI data. RESULTS: SNR improvements up to 2.7 were obtained in real in vivo datasets denoised with KPCA, in comparison to SNR gains of up to 1.8 using a linear PCA denoising technique called Marchenko-Pastur PCA (MPPCA). Compared to gold-standard dataset references created from averaged data, we showed that lower normalized root mean squared error was achieved with KPCA compared to MPPCA. Statistical analysis of residuals shows that anatomical information is preserved and only noise is removed. Improvements in the estimation of diffusion model parameters such as fractional anisotropy, mean diffusivity, and fiber orientation distribution functions were also demonstrated. CONCLUSION: Nonlinear redundancy of the dMRI signal can be exploited with KPCA, which allows superior noise reduction/SNR improvements than the MPPCA method, without loss of signal information.
PURPOSE: We aimed to develop a predictive model of disease severity for cirrhosis using MRI-derived radiomic features of the liver and spleen and compared it to the existing disease severity metrics of MELD score and clinical decompensation. The MELD score is compiled solely by blood parameters, and so far, it was not investigated if extracted image-based features have the potential to reflect severity to potentially complement the calculated score. METHODS: This was a retrospective study of eligible patients with cirrhosis ([Formula: see text]) who underwent a contrast-enhanced MR screening protocol for hepatocellular carcinoma (HCC) screening at a tertiary academic center from 2015 to 2018. Radiomic feature analyses were used to train four prediction models for assessing the patient's condition at time of scan: MELD score, MELD score [Formula: see text] 9 (median score of the cohort), MELD score [Formula: see text] 15 (the inflection between the risk and benefit of transplant), and clinical decompensation. Liver and spleen segmentations were used for feature extraction, followed by cross-validated random forest classification. RESULTS: Radiomic features of the liver and spleen were most predictive of clinical decompensation (AUC 0.84), which the MELD score could predict with an AUC of 0.78. Using liver or spleen features alone had slightly lower discrimination ability (AUC of 0.82 for liver and AUC of 0.78 for spleen features only), although this was not statistically significant on our cohort. When radiomic prediction models were trained to predict continuous MELD scores, there was poor correlation. When stratifying risk by splitting our cohort at the median MELD 9 or at MELD 15, our models achieved AUCs of 0.78 or 0.66, respectively. CONCLUSIONS: We demonstrated that MRI-based radiomic features of the liver and spleen have the potential to predict the severity of liver cirrhosis, using decompensation or MELD status as imperfect surrogate measures for disease severity.
Diffusion kurtosis imaging (DKI) is a diffusion MRI approach that enables the measurement of brain microstructural properties, reflecting molecular restrictions and tissue heterogeneity. DKI parameters such as mean kurtosis (MK) provide additional subtle information to that provided by popular diffusion tensor imaging (DTI) parameters, and thus have been considered useful to detect white matter abnormalities, especially in populations that are not expected to show severe brain pathologies. However, DKI parameters often yield artifactual output values that are outside of the biologically plausible range, which diminish sensitivity to identify true microstructural changes. Recently we have proposed the mean-kurtosis-curve (MK-Curve) method to correct voxels with implausible DKI parameters, and demonstrated its improved performance against other approaches that correct artifacts in DKI. In this work, we aimed to evaluate the utility of the MK-Curve method to improve the identification of white matter abnormalities in group comparisons. To do so, we compared group differences, with and without the MK-Curve correction, between 115 individuals at clinical high risk for psychosis (CHR) and 93 healthy controls (HCs). We also compared the correlation of the corrected and uncorrected DKI parameters with clinical characteristics. Following the MK-curve correction, the group differences had larger effect sizes and higher statistical significance (i.e., lower p-values), demonstrating increased sensitivity to detect group differences, in particular in MK. Furthermore, the MK-curve-corrected DKI parameters displayed stronger correlations with clinical variables in CHR individuals, demonstrating the clinical relevance of the corrected parameters. Overall, following the MK-curve correction our analyses found widespread lower MK in CHR that overlapped with lower fractional anisotropy (FA), and both measures were significantly correlated with a decline in functioning and with more severe symptoms. These observations further characterize white matter alterations in the CHR stage, demonstrating that MK and FA abnormalities are widespread, and mostly overlap. The improvement in group differences and stronger correlation with clinical variables suggest that applying MK-curve would be beneficial for the detection and characterization of subtle group differences in other experiments as well.
Translating deep learning research from theory into clinical practice has unique challenges, specifically in the field of neuroimaging. In this paper, we present DeepNeuro, a Python-based deep learning framework that puts deep neural networks for neuroimaging into practical usage with a minimum of friction during implementation. We show how this framework can be used to design deep learning pipelines that can load and preprocess data, design and train various neural network architectures, and evaluate and visualize the results of trained networks on evaluation data. We present a way of reproducibly packaging data pre- and postprocessing functions common in the neuroimaging community, which facilitates consistent performance of networks across variable users, institutions, and scanners. We show how deep learning pipelines created with DeepNeuro can be concisely packaged into shareable Docker and Singularity containers with user-friendly command-line interfaces.
BACKGROUND: Extracellular free water within cerebral white matter tissue has been shown to increase with age and pathology, yet the cognitive consequences of free water in typical aging prior to the development of neurodegenerative disease remains unclear. Understanding the contribution of free water to cognitive function in older adults may provide important insight into the neural mechanisms of the cognitive aging process. METHODS: A diffusion-weighted MRI measure of extracellular free water as well as a commonly used diffusion MRI metric (fractional anisotropy) along nine bilateral white matter pathways were examined for their relationship with cognitive function assessed by the NIH Toolbox Cognitive Battery in 47 older adults (mean age = 74.4 years, SD = 5.4 years, range = 65-85 years). Probabilistic tractography at the 99th percentile level of probability (Tracts Constrained by Underlying Anatomy; TRACULA) was utilized to produce the pathways on which microstructural characteristics were overlaid and examined for their contribution to cognitive function independent of age, education, and gender. RESULTS: When examining the 99th percentile probability core white matter pathway derived from TRACULA, poorer fluid cognitive ability was related to higher mean free water values across the angular and cingulum bundles of the cingulate gyrus, as well as the corticospinal tract and the superior longitudinal fasciculus. There was no relationship between cognition and mean FA or free water-adjusted FA across the 99th percentile core white matter pathway. Crystallized cognitive ability was not associated with any of the diffusion measures. When examining cognitive domains comprising the NIH Toolbox Fluid Cognition index relationships with these white matter pathways, mean free water demonstrated strong hemispheric and functional specificity for cognitive performance, whereas mean FA was not related to age or cognition across the 99th percentile pathway. CONCLUSIONS: Extracellular free water within white matter appears to increase with normal aging, and higher values are associated with significantly lower fluid but not crystallized cognitive functions. When using TRACULA to estimate the core of a white matter pathway, a higher degree of free water appears to be highly specific to the pathways associated with memory, working memory, and speeded decision-making performance, whereas no such relationship existed with FA. These data suggest that free water may play an important role in the cognitive aging process, and may serve as a stronger and more specific indicator of early cognitive decline than traditional diffusion MRI measures, such as FA.
PURPOSE: To optimize diffusion-relaxation MRI with tensor-valued diffusion encoding for precise estimation of compartment-specific fractions, diffusivities, and T values within a two-compartment model of white matter, and to explore the approach in vivo. METHODS: Sampling protocols featuring different b-values (b), b-tensor shapes (b ), and echo times (TE) were optimized using Cramér-Rao lower bounds (CRLB). Whole-brain data were acquired in children, adults, and elderly with white matter lesions. Compartment fractions, diffusivities, and T values were estimated in a model featuring two microstructural compartments represented by a "stick" and a "zeppelin." RESULTS: Precise parameter estimates were enabled by sampling protocols featuring seven or more "shells" with unique b/b /TE-combinations. Acquisition times were approximately 15 minutes. In white matter of adults, the "stick" compartment had a fraction of approximately 0.5 and, compared with the "zeppelin" compartment, featured lower isotropic diffusivities (0.6 vs. 1.3 μm /ms) but higher T values (85 vs. 65 ms). Children featured lower "stick" fractions (0.4). White matter lesions exhibited high "zeppelin" isotropic diffusivities (1.7 μm /ms) and T values (150 ms). CONCLUSIONS: Diffusion-relaxation MRI with tensor-valued diffusion encoding expands the set of microstructure parameters that can be precisely estimated and therefore increases their specificity to biological quantities.
The corticospinal tract (CST) is one of the most well studied tracts in human neuroanatomy. Its clinical significance can be demonstrated in many notable traumatic conditions and diseases such as stroke, spinal cord injury (SCI) or amyotrophic lateral sclerosis (ALS). With the advent of diffusion MRI and tractography the computational representation of the human CST in a 3D model became available. However, the representation of the entire CST and, specifically, the hand motor area has remained elusive. In this paper we propose a novel method, using manually drawn ROIs based on robustly identifiable neuroanatomic structures to delineate the entire CST and isolate its hand motor representation as well as to estimate their variability and generate a database of their volume, length and biophysical parameters. Using 37 healthy human subjects we performed a qualitative and quantitative analysis of the CST and the hand-related motor fiber tracts (HMFTs). Finally, we have created variability heat maps from 37 subjects for both the aforementioned tracts, which could be utilized as a reference for future studies with clinical focus to explore neuropathology in both trauma and disease states.
BACKGROUND: The radiological differential diagnosis between tumor recurrence and radiation-induced necrosis (ie, pseudoprogression) is of paramount importance in the management of glioma patients. OBJECTIVE: This research aims to develop a deep learning methodology for automated differentiation of tumor recurrence from radiation necrosis based on routine magnetic resonance imaging (MRI) scans. METHODS: In this retrospective study, 146 patients who underwent radiation therapy after glioma resection and presented with suspected recurrent lesions at the follow-up MRI examination were selected for analysis. Routine MRI scans were acquired from each patient, including T1, T2, and gadolinium-contrast-enhanced T1 sequences. Of those cases, 96 (65.8%) were confirmed as glioma recurrence on postsurgical pathological examination, while 50 (34.2%) were diagnosed as necrosis. A light-weighted deep neural network (DNN) (ie, efficient radionecrosis neural network [ERN-Net]) was proposed to learn radiological features of gliomas and necrosis from MRI scans. Sensitivity, specificity, accuracy, and area under the curve (AUC) were used to evaluate performance of the model in both image-wise and subject-wise classifications. Preoperative diagnostic performance of the model was also compared to that of the state-of-the-art DNN models and five experienced neurosurgeons. RESULTS: DNN models based on multimodal MRI outperformed single-modal models. ERN-Net achieved the highest AUC in both image-wise (0.915) and subject-wise (0.958) classification tasks. The evaluated DNN models achieved an average sensitivity of 0.947 (SD 0.033), specificity of 0.817 (SD 0.075), and accuracy of 0.903 (SD 0.026), which were significantly better than the tested neurosurgeons (P=.02 in sensitivity and P<.001 in specificity and accuracy). CONCLUSIONS: Deep learning offers a useful computational tool for the differential diagnosis between recurrent gliomas and necrosis. The proposed ERN-Net model, a simple and effective DNN model, achieved excellent performance on routine MRI scans and showed a high clinical applicability.
We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at: https://github.com/RayRuizhiLiao/joint_chestxray.
Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.
Despite recent progress in image-to-image translation, it remains challenging to apply such techniques to clinical quality medical images. We develop a novel parameterization of conditional generative adversarial networks that achieves high image fidelity when trained to transform MRIs conditioned on a patient's age and disease severity. The spatial-intensity transform generative adversarial network (SIT-GAN) constrains the generator to a smooth spatial transform composed with sparse intensity changes. This technique improves image quality and robustness to artifacts, and generalizes to different scanners. We demonstrate SIT-GAN on a large clinical image dataset of stroke patients, where it captures associations between ventricle expansion and aging, as well as between white matter hyperintensities and stroke severity. Additionally, SIT-GAN provides a disentangled view of the variation in shape and appearance across subjects.
Evolution provides an important window into how cortical organization shapes function and vice versa. The complex mosaic of changes in brain morphology and functional organization that have shaped the mammalian cortex during evolution, complicates attempts to chart cortical differences across species. It limits our ability to fully appreciate how evolution has shaped our brain, especially in systems associated with unique human cognitive capabilities that lack anatomical homologues in other species. Here, we develop a function-based method for cross-species alignment that enables the quantification of homologous regions between humans and rhesus macaques, even when their location is decoupled from anatomical landmarks. Critically, we find cross-species similarity in functional organization reflects a gradient of evolutionary change that decreases from unimodal systems and culminates with the most pronounced changes in posterior regions of the default mode network (angular gyrus, posterior cingulate and middle temporal cortices). Our findings suggest that the establishment of the default mode network, as the apex of a cognitive hierarchy, has changed in a complex manner during human evolution - even within subnetworks.
Quantitative imaging biomarkers (QIBs) provide medical image-derived intensity, texture, shape, and size features that may help characterize cancerous tumors and predict clinical outcomes. Successful clinical translation of QIBs depends on the robustness of their measurements. Biomarkers derived from positron emission tomography images are prone to measurement errors owing to differences in image processing factors such as the tumor segmentation method used to define volumes of interest over which to calculate QIBs. We illustrate a new Bayesian statistical approach to characterize the robustness of QIBs to different processing factors. Study data consist of 22 QIBs measured on 47 head and neck tumors in 10 positron emission tomography/computed tomography scans segmented manually and with semiautomated methods used by 7 institutional members of the NCI Quantitative Imaging Network. QIB performance is estimated and compared across institutions with respect to measurement errors and power to recover statistical associations with clinical outcomes. Analysis findings summarize the performance impact of different segmentation methods used by Quantitative Imaging Network members. Robustness of some advanced biomarkers was found to be similar to conventional markers, such as maximum standardized uptake value. Such similarities support current pursuits to better characterize disease and predict outcomes by developing QIBs that use more imaging information and are robust to different processing factors. Nevertheless, to ensure reproducibility of QIB measurements and measures of association with clinical outcomes, errors owing to segmentation methods need to be reduced.
Diffusion MRI (dMRI) tractography has been successfully used to study the trigeminal nerves (TGNs) in many clinical and research applications. Currently, identification of the TGN in tractography data requires expert nerve selection using manually drawn regions of interest (ROIs), which is prone to inter-observer variability, time-consuming and carries high clinical and labor costs. To overcome these issues, we propose to create a novel anatomically curated TGN tractography atlas that enables automated identification of the TGN from dMRI tractography. In this paper, we first illustrate the creation of a trigeminal tractography atlas. Leveraging a well-established computational pipeline and expert neuroanatomical knowledge, we generate a data-driven TGN fiber clustering atlas using tractography data from 50 subjects from the Human Connectome Project. Then, we demonstrate the application of the proposed atlas for automated TGN identification in new subjects, without relying on expert ROI placement. Quantitative and visual experiments are performed with comparison to expert TGN identification using dMRI data from two different acquisition sites. We show highly comparable results between the automatically and manually identified TGNs in terms of spatial overlap and visualization, while our proposed method has several advantages. First, our method performs automated TGN identification, and thus it provides an efficient tool to reduce expert labor costs and inter-operator bias relative to expert manual selection. Second, our method is robust to potential imaging artifacts and/or noise that can prevent successful manual ROI placement for TGN selection and hence yields a higher successful TGN identification rate.
Fiber tracking produces large tractography datasets that are tens of gigabytes in size consisting of millions of streamlines. Such vast amounts of data require formats that allow for efficient storage, transfer, and visualization. We present TRAKO, a new data format based on the Graphics Layer Transmission Format (glTF) that enables immediate graphical and hardware-accelerated processing. We integrate a state-of-the-art compression technique for vertices, streamlines, and attached scalar and property data. We then compare TRAKO to existing tractography storage methods and provide a detailed evaluation on eight datasets. TRAKO can achieve data reductions of over 28x without loss of statistical significance when used to replicate analysis from previously published studies.
Glioblastoma might have widespread effects on the neural organization and cognitive function, and even focal lesions may be associated with distributed functional alterations. However, functional changes do not necessarily follow obvious anatomical patterns and the current understanding of this interrelation is limited. In this study, we used resting-state functional magnetic resonance imaging to evaluate changes in global functional connectivity patterns in 15 patients with glioblastoma. For six patients we followed longitudinal trajectories of their functional connectome and structural tumour evolution using bi-monthly follow-up scans throughout treatment and disease progression. In all patients, unilateral tumour lesions were associated with inter-hemispherically symmetric network alterations, and functional proximity of tumour location was stronger linked to distributed network deterioration than anatomical distance. In the longitudinal subcohort of six patients, we observed patterns of network alterations with initial transient deterioration followed by recovery at first follow-up, and local network deterioration to precede structural tumour recurrence by two months. In summary, the impact of focal glioblastoma lesions on the functional connectome is global and linked to functional proximity rather than anatomical distance to tumour regions. Our findings further suggest a relevance for functional network trajectories as a possible means supporting early detection of tumour recurrence.
BACKGROUND: Post-traumatic stress disorder (PTSD) is a psychiatric disorder that afflicts many individuals, yet the neuropathological mechanisms that contribute to this disorder remain to be fully determined. Moreover, it is unclear how exposure to mild traumatic brain injury (mTBI), a condition that is often comorbid with PTSD, particularly among military personnel, affects the clinical and neurological presentation of PTSD. To address these issues, the present study explores relationships between PTSD symptom severity and the microstructure of limbic and paralimbic gray matter brain regions, as well as the impact of mTBI comorbidity on these relationships. METHODS: Structural and diffusion MRI data were acquired from 102 male veterans who were diagnosed with current PTSD. Diffusion data were analyzed with free-water imaging to quantify average CSF-corrected fractional anisotropy (FA) and mean diffusivity (MD) in 18 limbic and paralimbic gray matter regions. Associations between PTSD symptom severity and regional average dMRI measures were examined with repeated measures linear mixed models. Associations were studied separately in veterans with PTSD only, and in veterans with PTSD and a history of military mTBI. RESULTS: Analyses revealed that in the PTSD only cohort, more severe symptoms were associated with higher FA in the right amygdala-hippocampus complex, lower FA in the right cingulate cortex, and lower MD in the left medial orbitofrontal cortex. In the PTSD and mTBI cohort, more severe PTSD symptoms were associated with higher FA bilaterally in the amygdala-hippocampus complex, with higher FA bilaterally in the nucleus accumbens, with lower FA bilaterally in the cingulate cortex, and with higher MD in the right amygdala-hippocampus complex. CONCLUSIONS: These findings suggest that the microstructure of limbic and paralimbic brain regions may influence PTSD symptomatology. Further, given the additional associations observed between microstructure and symptom severity in veterans with head trauma, we speculate that mTBI may exacerbate the impact of brain microstructure on PTSD symptoms, especially within regions of the brain known to be vulnerable to chronic stress. A heightened sensitivity to the microstructural environment of the brain could partially explain why individuals with PTSD and mTBI comorbidity experience more severe symptoms and poorer illness prognoses than those without a history of brain injury. The relevance of these microstructural findings to the conceptualization of PTSD as being a disorder of stress-induced neuronal connectivity loss is discussed.
The brainstem, a structure of vital importance in mammals, is currently becoming a principal focus in cognitive, affective, and clinical neuroscience. Midbrain, pontine and medullary structures serve as the conduit for signals between the forebrain and spinal cord, are the epicenter of cranial nerve-circuits and systems, and subserve such integrative functions as consciousness, emotional processing, pain, and motivation. In this study, we parcellated the nuclear masses and the principal fiber pathways that were visible in a high-resolution T2-weighted MRI dataset of 50-micron isotropic voxels of a postmortem human brainstem. Based on this analysis, we generated a detailed map of the human brainstem. To assess the validity of our maps, we compared our observations with histological maps of traditional human brainstem atlases. Given the unique capability of MRI-based morphometric analysis in generating and preserving the morphology of 3D objects from individual 2D sections, we reconstructed the motor, sensory and integrative neural systems of the brainstem and rendered them in 3D representations. We anticipate the utilization of these maps by the neuroimaging community for applications in basic neuroscience as well as in neurology, psychiatry, and neurosurgery, due to their versatile computational nature in 2D and 3D representations in a publicly available capacity.