We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between O (N 2) image pairs in [Formula: see text] operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genotyping are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC = 0.97. Software is provided for efficient fine-grained curation of large, generic image datasets.
Publications by Year: 2022
Training deep learning models that segment an image in one step typically requires a large collection of manually annotated images that captures the anatomical variability in a cohort. This poses challenges when anatomical variability is extreme but training data is limited, as when segmenting cardiac structures in patients with congenital heart disease (CHD). In this paper, we propose an iterative segmentation model and show that it can be accurately learned from a small dataset. Implemented as a recurrent neural network, the model evolves a segmentation over multiple steps, from a single user click until reaching an automatically determined stopping point. We develop a novel loss function that evaluates the entire sequence of output segmentations, and use it to learn model parameters. Segmentations evolve predictably according to growth dynamics encapsulated by training data, which consists of images, partially completed segmentations, and the recommended next step. The user can easily refine the final segmentation by examining those that are earlier or later in the output sequence. Using a dataset of 3D cardiac MR scans from patients with a wide range of CHD types, we show that our iterative model offers better generalization to patients with the most severe heart malformations.
Measuring and understanding functional fetal brain development in utero is critical for the study of the developmental foundations of our cognitive abilities, possible early detection of disorders, and their prevention. Thalamocortical connections are an intricate component of shaping the cortical layout, but so far, only ex-vivo studies provide evidence of how axons enter the sub-plate and cortex during this highly dynamic phase. Evidence for normal in-utero development of the functional thalamocortical connectome in humans is missing. Here, we modeled fetal functional thalamocortical connectome development using in-utero functional magnetic resonance imaging in fetuses observed from 19th to 40th weeks of gestation (GW). We observed a peak increase of thalamocortical functional connectivity strength between 29th and 31st GW, right before axons establish synapses in the cortex. The cortico-cortical connectivity increases in a similar time window, and exhibits significant functional laterality in temporal-superior, -medial, and -inferior areas. Homologous regions exhibit overall similar mirrored connectivity profiles, but this similarity decreases during gestation giving way to a more diverse cortical interconnectedness. Our results complement the understanding of structural development of the human connectome and may serve as the basis for the investigation of disease and deviations from a normal developmental trajectory of connectivity development.