Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
PURPOSE: Fetal brain Magnetic Resonance Imaging suffers from unpredictable and unconstrained fetal motion that causes severe image artifacts even with half-Fourier single-shot fast spin echo (HASTE) readouts. This work presents the implementation of a closed-loop pipeline that automatically detects and reacquires HASTE images that were degraded by fetal motion without any human interaction. METHODS: A convolutional neural network that performs automatic image quality assessment (IQA) was run on an external GPU-equipped computer that was connected to the internal network of the MRI scanner. The modified HASTE pulse sequence sent each image to the external computer, where the IQA convolutional neural network evaluated it, and then the IQA score was sent back to the sequence. At the end of the HASTE stack, the IQA scores from all the slices were sorted, and only slices with the lowest scores (corresponding to the slices with worst image quality) were reacquired. RESULTS: The closed-loop HASTE acquisition framework was tested on 10 pregnant mothers, for a total of 73 acquisitions of our modified HASTE sequence. The IQA convolutional neural network, which was successfully employed by our modified sequence in real time, achieved an accuracy of 85.2% and area under the receiver operator characteristic of 0.899. CONCLUSION: The proposed acquisition/reconstruction pipeline was shown to successfully identify and automatically reacquire only the motion degraded fetal brain HASTE slices in the prescribed stack. This minimizes the overall time spent on HASTE acquisitions by avoiding the need to repeat the entire stack if only few slices in the stack are motion-degraded.
Language and theory of mind (ToM) are the cognitive capacities that allow for the successful interpretation and expression of meaning. While functional MRI investigations are able to consistently localize language and ToM to specific cortical regions, diffusion MRI investigations point to an inconsistent and sometimes overlapping set of white matter tracts associated with these two cognitive domains. To further examine the white matter tracts that may underlie these domains, we use a two-tensor tractography method to investigate the white matter microstructure of 809 participants from the Human Connectome Project. 20 association white matter tracts (10 in each hemisphere) are uniquely identified by leveraging a neuroanatomist-curated automated white matter tract atlas. The mean fractional anisotropy (FA), mean diffusivity (MD), and number of streamlines (NoS) are measured for each white matter tract. Performance on neuropsychological assessments of semantic memory (NIH Toolbox Picture Vocabulary Test, TPVT) and emotion perception (Penn Emotion Recognition Test, PERT) are used to measure critical subcomponents of the language and ToM networks, respectively. Regression models are constructed to examine how structural measurements of left and right white matter tracts influence performance across these two assessments. We find that semantic memory performance is influenced by the number of streamlines of the left superior longitudinal fasciculus III (SLF-III), and emotion perception performance is influenced by the number of streamlines of the right SLF-III. Additionally, we find that performance on both semantic memory & emotion perception is influenced by the FA of the left arcuate fasciculus (AF). The results point to multiple, overlapping white matter tracts that underlie the cognitive domains of language and ToM. Results are discussed in terms of hemispheric dominance and concordance with prior investigations.
Little is known on how mild traumatic brain injury affects white matter based on age at injury, sex, cerebral microbleeds, and time since injury. Here, we study the fractional anisotropy of white matter to study these effects in 109 participants aged 18-77 (46 females, age μ ± σ = 40 ± 17 years) imaged within [Formula: see text] 1 week and [Formula: see text] 6 months post-injury. Age is found to be linearly associated with white matter degradation, likely due not only to injury but also to cumulative effects of other pathologies and to their interactions with injury. Age is associated with mean anisotropy decreases in the corpus callosum, middle longitudinal fasciculi, inferior longitudinal and occipitofrontal fasciculi, and superficial frontal and temporal fasciculi. Over [Formula: see text] 6 months, the mean anisotropies of the corpus callosum, left superficial frontal fasciculi, and left corticospinal tract decrease significantly. Independently of other predictors, age and cerebral microbleeds contribute to anisotropy decrease in the callosal genu. Chronically, the white matter of commissural tracts, left superficial frontal fasciculi, and left corticospinal tract degrade appreciably, independently of other predictors. Our findings suggest that large commissural and intra-hemispheric structures are at high risk for post-traumatic degradation. This study identifies detailed neuroanatomic substrates consistent with brain injury patients' age-dependent deficits in information processing speed, interhemispheric communication, motor coordination, visual acuity, sensory integration, reading speed/comprehension, executive function, personality, and memory. We also identify neuroanatomic features underlying white matter degradation whose severity is associated with the male sex. Future studies should compare our findings to functional measures and other neurodegenerative processes.
Tumour resection requires precise planning and navigation to maximise tumour removal while simultaneously protecting nearby healthy tissues. Neurosurgeons need to know the location of the remaining tumour after partial tumour removal before continuing with the resection. Our approach to the problem uses biomechanical modelling and computer simulation to compute the brain deformations after the tumour is resected. In this study, we use Meshless Total Lagrangian Explicit Dynamics (MTLED) as the solver. The problem geometry is extracted from the patient-specific MRI data and includes the parenchyma, tumour, cerebrospinal fluid (CSF) and skull. Appropriate nonlinear material formulation is used. Loading is performed by imposing intra-operative conditions of gravity and reaction forces between the tumour and surrounding healthy parenchyma tissues. A finite frictionless sliding contact is enforced between the skull (rigid) and parenchyma. The meshless simulation results are compared to intra-operative MRI sections. We also calculate Hausdorff distances between the computed deformed surfaces (ventricles and tumour cavities) and surfaces observed intra-operatively. Over 80% of points on ventricle surface and 95% of points on tumour cavity surface were successfully registered (results within the limits of two times the original in-plane resolution of the intra-operative image). Computed results demonstrate the potential for our method in estimating the tissue deformation and tumour boundary during the resection. This article is protected by copyright. All rights reserved.
PURPOSE: Reperfusion therapy enables effective treatment of ischemic stroke presenting within 4-6 hours. However, tissue progression from ischemia to infarction is variable, and some patients benefit from treatment up until 24 hours. Improved imaging techniques are needed to identify these patients. Here, it was hypothesized that time dependence in diffusion MRI may predict tissue outcome in ischemic stroke. METHODS: Diffusion MRI data were acquired with multiple diffusion times in five non-reperfused patients at 2, 9, and 100 days after stroke onset. Maps of "rate of kurtosis change" (k), mean kurtosis, ADC, and fractional anisotropy were derived. The ADC maps defined lesions, normal-appearing tissue, and the lesion tissue that would either be infarcted or remain viable by day 100. Diffusion parameters were compared (1) between lesions and normal-appearing tissue, and (2) between lesion tissue that would be infarcted or remain viable. RESULTS: Positive values of k were observed within stroke lesions on day 2 (P = .001) and on day 9 (P = .023), indicating diffusional exchange. On day 100, high ADC values indicated infarction of 50 ± 20% of the lesion volumes. Tissue infarction was predicted by high k values both on day 2 (P = .026) and on day 9 (P = .046), by low mean kurtosis values on day 2 (P = .043), and by low fractional anisotropy values on day 9 (P = .029), but not by low ADC values. CONCLUSIONS: Diffusion time dependence predicted tissue outcome in ischemic stroke more accurately than the ADC, and may be useful for predicting reperfusion benefit.
In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well.
Probing the cellular structure of in vivo biological tissue is a fundamental problem in biomedical imaging and medical science. This work introduces an approach for analyzing diffusion magnetic resonance imaging data acquired by the novel tensor-valued encoding technique for characterizing tissue microstructure. Our approach first uses a signal model to estimate the variance and skewness of the distribution of apparent diffusion tensors modeling the underlying tissue. Then several novel imaging indices, such as weighted microscopic anisotropy and microscopic skewness, are derived to characterize different ensembles of diffusion processes that are indistinguishable by existing techniques. The contributions of this work also include a theoretical proof that shows that, to estimate the skewness of a diffusion tensor distribution, the encoding protocol needs to include full-rank tensor diffusion encoding. This proof provides a guideline for the application of this technique. The properties of the proposed indices are illustrated using both synthetic data and in vivo data acquired from a human brain.
Using positron emission tomography, we recently demonstrated elevated brain levels of the 18kDa translocator protein (TSPO), a glial activation marker, in chronic low back pain (cLBP) patients, compared to healthy controls (HC). Here, we first sought to replicate the original findings in an independent cohort (15 cLBP, 37.8±12.5 y/o; 18 HC, 48.2±12.8 y/o). We then trained random forest (RF) machine learning algorithms based on TSPO imaging features combining discovery and replication cohorts (totaling 25 cLBP, 42.4±13.2 y/o; 27 HC, 48.9±12.6 y/o), in order to explore whether image features other than the mean contain meaningful information that might contribute to the discrimination of cLBP patients and HC. Feature importance was ranked usind SHapley Additive exPlanations (SHAP) values, and the classification performance (in terms of AUC values) of classifiers containing only the mean, other features, or all features was compared using the DeLong test. Both region-of-interest (ROI) and voxelwise analyses replicated the original observation of thalamic TSPO signal elevations in cLBP patients compared to HC (p's<0.05). The RF-based analyses revealed that while the mean is a discriminating feature, other features demonstrate similar level of importance, including the maximum, kurtosis and entropy.Our observations suggest that thalamic neuroinflammatory signal is a reproducible and discriminating feature for cLBP, further supporting a role for glial activation in human chronic low back pain, and the exploration of neuroinflammation as a therapeutic target for chronic pain. This work further shows that TSPO signal contains a richness of information that the simple mean might fail to capture completely.
PURPOSE: Diffusion-weighted MRI is sensitive to incoherent tissue motion, which may confound the measured signal and subsequent analysis. We propose a "motion-compensated" gradient waveform design for tensor-valued diffusion encoding that negates the effects bulk motion and incoherent motion in the ballistic regime. METHODS: Motion compensation was achieved by constraining the magnitude of gradient waveform moment vectors. The constraint was incorporated into a numerical optimization framework, along with existing constraints that account for b-tensor shape, hardware restrictions, and concomitant field gradients. We evaluated the efficacy of encoding and motion compensation in simulations, and we demonstrated the approach by linear and planar b-tensor encoding in a healthy heart in vivo. RESULTS: The optimization framework produced asymmetric motion-compensated waveforms that yielded b-tensors of arbitrary shape with improved efficiency compared with previous designs for tensor-valued encoding, and equivalent efficiency to previous designs for linear (conventional) encoding. Technical feasibility was demonstrated in the heart in vivo, showing vastly improved data quality when using motion compensation. The optimization framework is available online in open source. CONCLUSION: Our gradient waveform design is both more flexible and efficient than previous methods, facilitating tensor-valued diffusion encoding in tissues in which motion would otherwise confound the signal. The proposed design exploits asymmetric encoding times, a single refocusing pulse or multiple refocusing pulses, and integrates compensation for concomitant gradient effects throughout the imaging volume.
BACKGROUND: Body composition is associated with mortality; however its routine assessment is too time-consuming. PURPOSE: To demonstrate the value of artificial intelligence (AI) to extract body composition measures from routine studies, we aimed to develop a fully automated AI approach to measure fat and muscles masses, to validate its clinical discriminatory value, and to provide the code, training data and workflow solutions to facilitate its integration into local practice. METHODS: We developed a neural network that quantified the tissue components at the L3 vertebral body level using data from the Liver Tumor Challenge (LiTS) and a pancreatic cancer cohort. We classified sarcopenia using accepted skeletal muscle index cut-offs and visceral fat based its median value. We used Kaplan Meier curves and Cox regression analysis to assess the association between these measures and mortality. RESULTS: Applying the algorithm trained on LiTS data to the local cohort yielded good agreement [>0.8 intraclass correlation (ICC)]; when trained on both datasets, it had excellent agreement (>0.9 ICC). The pancreatic cancer cohort had 136 patients (mean age: 67 ± 11 years; 54% women); 15% had sarcopenia; mean visceral fat was 142 cm2. Concurrent with prior research, we found a significant association between sarcopenia and mortality [mean survival of 15 ± 12 vs. 22 ± 12 (p < 0.05), adjusted HR of 1.58 (95% CI: 1.03-3.33)] but no association between visceral fat and mortality. The detector analysis took 1 ± 0.5 s. CONCLUSIONS: AI body composition analysis can provide meaningful imaging biomarkers from routine exams demonstrating AI's ability to further enhance the clinical value of radiology reports.
Proton beam therapy (PBT) is an effective pediatric brain tumor treatment. However, the resulting microstructural changes within and around irradiated tumors are unknown. We retrospectively applied diffusion tensor imaging (DTI) and free-water imaging (FWI) on diffusion-weighted magnetic resonance imaging (dMRI) data to monitor microstructural changes during the PBT and after 8 months in a pilocytic astrocytoma (PA) and normal-appearing white matter (NAWM). We evaluated the conventional MRI- and dMRI-derived indices from six MRI sessions (t0-t5) in a Caucasian child with a hypothalamic PA: at baseline (t0), during the PBT (t1-t4) and after 8 months (t5). The tumor voxels were classified as "solid" or "fluid" based on the FWI. While the tumor volume remained stable during the PBT, the dMRI analyses identified two different response patterns: (i) an increase in fluid content and diffusivity with anisotropy reductions in the solid voxels at t1, followed by (ii) smaller variations in fluid content but higher anisotropy in the solid voxels at t2-t4. At follow-up (t5), the tumor volume, fluid content, and diffusivity in the solid voxels increased. The NAWM showed dose-dependent microstructural changes. The use of the dMRI and FWI showed complex dynamic microstructural changes in the irradiated mass during the PBT and at follow-up, opening new avenues in our understanding of radiation-induced pathophysiologic mechanisms in tumors and the surrounding tissues.
BACKGROUND: The National Cancer Institute Informatics Technology for Cancer Research (ITCR) program provides a series of funding mechanisms to create an ecosystem of open-source software (OSS) that serves the needs of cancer research. As the ITCR ecosystem substantially grows, it faces the challenge of the long-term sustainability of the software being developed by ITCR grantees. To address this challenge, the ITCR sustainability and industry partnership working group (SIP-WG) was convened in 2019.
OBJECTIVE: The charter of the SIP-WG is to investigate options to enhance the long-term sustainability of the OSS being developed by ITCR, in part by developing a collection of business model archetypes that can serve as sustainability plans for ITCR OSS development initiatives. The working group assembled models from the ITCR program, from other studies, and from the engagement of its extensive network of relationships with other organizations (eg, Chan Zuckerberg Initiative, Open Source Initiative, and Software Sustainability Institute) in support of this objective.
METHODS: This paper reviews the existing sustainability models and describes 10 OSS use cases disseminated by the SIP-WG and others, including 3D Slicer, Bioconductor, Cytoscape, Globus, i2b2 (Informatics for Integrating Biology and the Bedside) and tranSMART, Insight Toolkit, Linux, Observational Health Data Sciences and Informatics tools, R, and REDCap (Research Electronic Data Capture), in 10 sustainability aspects: governance, documentation, code quality, support, ecosystem collaboration, security, legal, finance, marketing, and dependency hygiene.
RESULTS: Information available to the public reveals that all 10 OSS have effective governance, comprehensive documentation, high code quality, reliable dependency hygiene, strong user and developer support, and active marketing. These OSS include a variety of licensing models (eg, general public license version 2, general public license version 3, Berkeley Software Distribution, and Apache 3) and financial models (eg, federal research funding, industry and membership support, and commercial support). However, detailed information on ecosystem collaboration and security is not publicly provided by most OSS.
CONCLUSIONS: We recommend 6 essential attributes for research software: alignment with unmet scientific needs, a dedicated development team, a vibrant user community, a feasible licensing model, a sustainable financial model, and effective product management. We also stress important actions to be considered in future ITCR activities that involve the discussion of the sustainability and licensing models for ITCR OSS, the establishment of a central library, the allocation of consulting resources to code quality control, ecosystem collaboration, security, and dependency hygiene.
Conventionally, as a preprocessing step, functional MRI (fMRI) data are spatially smoothed before further analysis, be it for activation mapping on task-based fMRI or functional connectivity analysis on resting-state fMRI data. When images are smoothed volumetrically, however, isotropic Gaussian kernels are generally used, which do not adapt to the underlying brain structure. Alternatively, cortical surface smoothing procedures provide the benefit of adapting the smoothing process to the underlying morphology, but require projecting volumetric data on to the surface. In this paper, leveraging principles from graph signal processing, we propose a volumetric spatial smoothing method that takes advantage of the gray-white and pial cortical surfaces, and as such, adapts the filtering process to the underlying morphological details at each point in the cortex.
White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.
Subtle alterations in white matter microstructure are observed in youth at clinical high risk (CHR) for psychosis. However, the timing of these changes and their relationships to the emergence of psychosis remain unclear. Here, we track the evolution of white matter abnormalities in a large, longitudinal cohort of CHR individuals comprising the North American Prodrome Longitudinal Study (NAPLS-3). Multi-shell diffusion magnetic resonance imaging data were collected across multiple timepoints (1-5 over 1 year) in 286 subjects (aged 12-32 years): 25 CHR individuals who transitioned to psychosis (CHR-P; 61 scans), 205 CHR subjects with unknown transition outcome after the 1-year follow-up period (CHR-U; 596 scans), and 56 healthy controls (195 scans). Linear mixed effects models were fitted to infer the impact of age and illness-onset on variation in the fractional anisotropy of cellular tissue (FAT) and the volume fraction of extracellular free water (FW). Baseline measures of white matter microstructure did not differentiate between HC, CHR-U and CHR-P individuals. However, age trajectories differed between the three groups in line with a developmental effect: CHR-P and CHR-U groups displayed higher FAT in adolescence, and 4% lower FAT by 30 years of age compared to controls. Furthermore, older CHR-P subjects (20+ years) displayed 4% higher FW in the forceps major (p < 0.05). Prospective analysis in CHR-P did not reveal a significant impact of illness onset on regional FAT or FW, suggesting that transition to psychosis is not marked by dramatic change in white matter microstructure. Instead, clinical high risk for psychosis-regardless of transition outcome-is characterized by subtle age-related white matter changes that occur in tandem with development.
This paper presents the design of NaviPBx, an ultrasound-navigated prostate cancer biopsy system. NaviPBx is designed to support an affordable and sustainable national healthcare program in Senegal. It uses spatiotemporal navigation and multiparametric transrectal ultrasound to guide biopsies. NaviPBx integrates concepts and methods that have been independently validated previously in clinical feasibility studies and deploys them together in a practical prostate cancer biopsy system. NaviPBx is based entirely on free open-source software and will be shared as a free open-source program with no restriction on its use. NaviPBx is set to be deployed and sustained nationwide through the Senegalese Military Health Service. This paper reports on the results of the design process of NaviPBx. Our approach concentrates on "frugal technology", intended to be affordable for low-middle income (LMIC) countries. Our project promises the wide-scale application of prostate biopsy and will foster time-efficient development and programmatic implementation of ultrasound-guided diagnostic and therapeutic interventions in Senegal and beyond.
Microstructure imaging seeks to noninvasively measure and map microscopic tissue features by pairing mathematical modeling with tailored MRI protocols. This article reviews an emerging paradigm that has the potential to provide a more detailed assessment of tissue microstructure-combined diffusion-relaxometry imaging. Combined diffusion-relaxometry acquisitions vary multiple MR contrast encodings-such as b-value, gradient direction, inversion time, and echo time-in a multidimensional acquisition space. When paired with suitable analysis techniques, this enables quantification of correlations and coupling between multiple MR parameters-such as diffusivity, T 1 , T 2 , and T 2 ∗ . This opens the possibility of disentangling multiple tissue compartments (within voxels) that are indistinguishable with single-contrast scans, enabling a new generation of microstructural maps with improved biological sensitivity and specificity.
Most existing algorithms for automatic 3D morphometry of human brain MRI scans are designed for data with near-isotropic voxels at approximately 1 mm resolution, and frequently have contrast constraints as well-typically requiring T1-weighted images (e.g., MP-RAGE scans). This limitation prevents the analysis of millions of MRI scans acquired with large inter-slice spacing in clinical settings every year. In turn, the inability to quantitatively analyze these scans hinders the adoption of quantitative neuro imaging in healthcare, and also precludes research studies that could attain huge sample sizes and hence greatly improve our understanding of the human brain. Recent advances in convolutional neural networks (CNNs) are producing outstanding results in super-resolution and contrast synthesis of MRI. However, these approaches are very sensitive to the specific combination of contrast, resolution and orientation of the input images, and thus do not generalize to diverse clinical acquisition protocols - even within sites. In this article, we present SynthSR, a method to train a CNN that receives one or more scans with spaced slices, acquired with different contrast, resolution and orientation, and produces an isotropic scan of canonical contrast (typically a 1 mm MP-RAGE). The presented method does not require any preprocessing, beyond rigid coregistration of the input scans. Crucially, SynthSR trains on synthetic input images generated from 3D segmentations, and can thus be used to train CNNs for any combination of contrasts, resolutions and orientations without high-resolution real images of the input contrasts. We test the images generated with SynthSR in an array of common downstream analyses, and show that they can be reliably used for subcortical segmentation and volumetry, image registration (e.g., for tensor-based morphometry), and, if some image quality requirements are met, even cortical thickness morphometry. The source code is publicly available at https://github.com/BBillot/SynthSR.