In this paper, we address the problem of 2D-3D pose estimation. Specifically, we propose an approach to jointly track a rigid object in a 2D image sequence and to estimate its pose (position and orientation) in 3D space. We revisit a joint 2D segmentation/3D pose estimation technique, and then extend the framework by incorporating a particle filter to robustly track the object in a challenging environment, and by developing an occlusion detection and handling scheme to continuously track the object in the presence of occlusions. In particular, we focus on partial occlusions that prevent the tracker from extracting an exact region properties of the object, which plays a pivotal role for region-based tracking methods in maintaining the track. To this end, a dynamical choice of how to invoke the objective functional is performed online based on the degree of dependencies between predictions and measurements of the system in accordance with the degree of occlusion and the variation of the object’s pose. This scheme provides the robustness to deal with occlusions of an obstacle with different statistical properties from that of the object of interest. Experimental results demonstrate the practical applicability and robustness of the proposed method in several challenging scenarios.
Velazquez ER, Parmar C, Jermoumi M, Mak RH, van Baardwijk A, Fennessy FM, Lewis JH, De Ruysscher D, Kikinis R, Lambin P, Aerts HJWL. Volumetric CT-based segmentation of NSCLC using 3D-Slicer. Sci Rep. 2013;3:3529.
Accurate volumetric assessment in non-small cell lung cancer (NSCLC) is critical for adequately informing treatments. In this study we assessed the clinical relevance of a semiautomatic computed tomography (CT)-based segmentation method using the competitive region-growing based algorithm, implemented in the free and public available 3D-Slicer software platform. We compared the 3D-Slicer segmented volumes by three independent observers, who segmented the primary tumour of 20 NSCLC patients twice, to manual slice-by-slice delineations of five physicians. Furthermore, we compared all tumour contours to the macroscopic diameter of the tumour in pathology, considered as the "gold standard". The 3D-Slicer segmented volumes demonstrated high agreement (overlap fractions > 0.90), lower volume variability (p = 0.0003) and smaller uncertainty areas (p = 0.0002), compared to manual slice-by-slice delineations. Furthermore, 3D-Slicer segmentations showed a strong correlation to pathology (r = 0.89, 95%CI, 0.81-0.94). Our results show that semiautomatic 3D-Slicer segmentations can be used for accurate contouring and are more stable than manual delineations. Therefore, 3D-Slicer can be employed as a starting point for treatment decisions or for high-throughput data mining research, such as Radiomics, where manual delineating often represent a time-consuming bottleneck.
Lou Y, Irimia A, Vela PA, Chambers MC, Van Horn JD, Vespa PM, Tannenbaum AR. Multimodal deformable registration of traumatic brain injury MR volumes via the Bhattacharyya distance. IEEE Trans Biomed Eng. 2013;60(9):2511–20.
An important problem of neuroimaging data analysis for traumatic brain injury (TBI) is the task of coregistering MR volumes acquired using distinct sequences in the presence of widely variable pixel movements which are due to the presence and evolution of pathology. We are motivated by this problem to design a numerically stable registration algorithm which handles large deformations. To this end, we propose a new measure of probability distributions based on the Bhattacharyya distance, which is more stable than the widely used mutual information due to better behavior of the square root function than the logarithm at zero. Robustness is illustrated on two TBI patient datasets, each containing 12 MR modalities. We implement our method on graphics processing units (GPU) so as to meet the clinical requirement of time-efficient processing of TBI data. We find that 6 sare required to register a pair of volumes with matrix sizes of 256 × 256 × 60 on the GPU. In addition to exceptional time efficiency via its GPU implementation, this methodology provides a clinically informative method for the mapping and evaluation of anatomical changes in TBI.
McDannold N, Zhang YZ, Power C, Jolesz F, Vykhodtseva N. Nonthermal ablation with microbubble-enhanced focused ultrasound close to the optic tract without affecting nerve function. J Neurosurg. 2013;119(5):1208–20.
OBJECT: Tumors at the skull base are challenging for both resection and radiosurgery given the presence of critical adjacent structures, such as cranial nerves, blood vessels, and brainstem. Magnetic resonance imaging-guided thermal ablation via laser or other methods has been evaluated as a minimally invasive alternative to these techniques in the brain. Focused ultrasound (FUS) offers a noninvasive method of thermal ablation; however, skull heating limits currently available technology to ablation at regions distant from the skull bone. Here, the authors evaluated a method that circumvents this problem by combining the FUS exposures with injected microbubble-based ultrasound contrast agent. These microbubbles concentrate the ultrasound-induced effects on the vasculature, enabling an ablation method that does not cause significant heating of the brain or skull. METHODS: In 29 rats, a 525-kHz FUS transducer was used to ablate tissue structures at the skull base that were centered on or adjacent to the optic tract or chiasm. Low-intensity, low-duty-cycle ultrasound exposures (sonications) were applied for 5 minutes after intravenous injection of an ultrasound contrast agent (Definity, Lantheus Medical Imaging Inc.). Using histological analysis and visual evoked potential (VEP) measurements, the authors determined whether structural or functional damage was induced in the optic tract or chiasm. RESULTS: Overall, while the sonications produced a well-defined lesion in the gray matter targets, the adjacent tract and chiasm had comparatively little or no damage. No significant changes (p > 0.05) were found in the magnitude or latency of the VEP recordings, either immediately after sonication or at later times up to 4 weeks after sonication, and no delayed effects were evident in the histological features of the optic nerve and retina. CONCLUSIONS: This technique, which selectively targets the intravascular microbubbles, appears to be a promising method of noninvasively producing sharply demarcated lesions in deep brain structures while preserving function in adjacent nerves. Because of low vascularity—and thus a low microbubble concentration—some large white matter tracts appear to have some natural resistance to this type of ablation compared with gray matter. While future work is needed to develop methods of monitoring the procedure and establishing its safety at deep brain targets, the technique does appear to be a potential solution that allows FUS ablation of deep brain targets while sparing adjacent nerve structures.
The accurate diagnosis of Alzheimer’s disease (AD) at different stages is essential to identify patients at high risk of dementia and plan prevention or treatment measures accordingly. In this study, we proposed a new AD staging method for the entire spectrum of AD including the AD, Mild Cognitive Impairment with and without AD conversions, and Cognitive Normal groups. Our method embedded the high dimensional multi-view features derived from neuroimaging data into a low dimensional feature space and could form a more distinctive representation than the naive concatenated features. It also updated the testing data based on the Localized Sparse Code Gradients (LSCG) to further enhance the classification. The LSCG algorithm, validated using Magnetic Resonance Imaging data from the ADNI baseline cohort, achieved significant improvements on all diagnosis groups compared to using the original sparse coding method.
Cavallari M, Moscufo N, Skudlarski P, Meier D, Panzer VP, Pearlson GD, White WB, Wolfson L, Guttmann CRG. Mobility impairment is associated with reduced microstructural integrity of the inferior and superior cerebellar peduncles in elderly with no clinical signs of cerebellar dysfunction. Neuroimage Clin. 2013;2:332–40.
While the cerebellum plays a critical role in motor coordination and control no studies have investigated its involvement in idiopathic mobility impairment in community-dwelling elderly. In this study we tested the hypothesis that structural changes in the cerebellar peduncles not detected by conventional magnetic resonance imaging are associated with reduced mobility performance. The analysis involved eighty-five subjects (age range: 75-90 years) who had no clinical signs of cerebellar dysfunction. Based on the short physical performance battery (SPPB) score, we defined mobility status of the subjects in the study as normal (score 11-12, n = 26), intermediate (score 9-10, n = 27) or impaired (score
Zhu L, Gao Y, Yezzi A, Tannenbaum A. Automatic segmentation of the left atrium from MR images via variational region growing with a moments-based shape prior. IEEE Trans Image Process. 2013;22(12):5111–22.
The planning and evaluation of left atrial ablation procedures are commonly based on the segmentation of the left atrium, which is a challenging task due to large anatomical variations. In this paper, we propose an automatic approach for segmenting the left atrium from magnetic resonance imagery. The segmentation problem is formulated as a problem in variational region growing. In particular, the method starts locally by searching for a seed region of the left atrium from an MR slice. A global constraint is imposed by applying a shape prior to the left atrium represented by Zernike moments. The overall growing process is guided by the robust statistics of intensities from the seed region along with the shape prior to capture the entire atrial region. The robustness and accuracy of our approach are demonstrated by experimental results from 64 human MR images.
Mike A, Strammer E, Aradi M, Orsi G, Perlaki G, Hajnal A, Sandor J, Banati M, Illes E, Zaitsev A, Herold R, Guttmann CRG, Illes Z. Disconnection mechanism and regional cortical atrophy contribute to impaired processing of facial expressions and theory of mind in multiple sclerosis: a structural MRI study. PLoS One. 2013;8(12):e82422.
Successful socialization requires the ability of understanding of others’ mental states. This ability called as mentalization (Theory of Mind) may become deficient and contribute to everyday life difficulties in multiple sclerosis. We aimed to explore the impact of brain pathology on mentalization performance in multiple sclerosis. Mentalization performance of 49 patients with multiple sclerosis was compared to 24 age- and gender matched healthy controls. T1- and T2-weighted three-dimensional brain MRI images were acquired at 3Tesla from patients with multiple sclerosis and 18 gender- and age matched healthy controls. We assessed overall brain cortical thickness in patients with multiple sclerosis and the scanned healthy controls, and measured the total and regional T1 and T2 white matter lesion volumes in patients with multiple sclerosis. Performances in tests of recognition of mental states and emotions from facial expressions and eye gazes correlated with both total T1-lesion load and regional T1-lesion load of association fiber tracts interconnecting cortical regions related to visual and emotion processing (genu and splenium of corpus callosum, right inferior longitudinal fasciculus, right inferior fronto-occipital fasciculus, uncinate fasciculus). Both of these tests showed correlations with specific cortical areas involved in emotion recognition from facial expressions (right and left fusiform face area, frontal eye filed), processing of emotions (right entorhinal cortex) and socially relevant information (left temporal pole). Thus, both disconnection mechanism due to white matter lesions and cortical thinning of specific brain areas may result in cognitive deficit in multiple sclerosis affecting emotion and mental state processing from facial expressions and contributing to everyday and social life difficulties of these patients.
In image segmentation, we are often interested in using certain quantities to characterize the object, and perform the classification based on criteria such as mean intensity, gradient magnitude, and responses to certain predefined filters. Unfortunately, in many cases such quantities are not adequate to model complex textured objects. Along a different line of research, the sparse characteristic of natural signals has been recognized and studied in recent years. Therefore, how such sparsity can be utilized, in a non-parametric way, to model the object texture and assist the textural image segmentation process is studied in this paper, and a segmentation scheme based on the sparse representation of the texture information is proposed. More explicitly, the texture is encoded by the dictionaries constructed from the user initialization. Then, an active contour is evolved to optimize the fidelity of the representation provided by the dictionary of the target. In doing so, not only a non-parametric texture modeling technique is provided, but also the sparsity of the representation guarantees the computation efficiency. The experiments are carried out on the publicly available image data sets which contain a large variety of texture images, to analyze the user interaction, performance statistics, and to highlight the algorithm’s capability of robustly extracting textured regions from an image.
Gao Y, Tannenbaum A, Chen H, Torres M, Yoshida E, Yang X, Wang Y, Curran W, Liu T. Automated skin segmentation in ultrasonic evaluation of skin toxicity in breast cancer radiotherapy. Ultrasound Med Biol. 2013;39(11):2166–75.
Skin toxicity is the most common side effect of breast cancer radiotherapy and impairs the quality of life of many breast cancer survivors. We, along with other researchers, have recently found quantitative ultrasound to be effective as a skin toxicity assessment tool. Although more reliable than standard clinical evaluations (visual observation and palpation), the current procedure for ultrasound-based skin toxicity measurements requires manual delineation of the skin layers (i.e., epidermis-dermis and dermis-hypodermis interfaces) on each ultrasound B-mode image. Manual skin segmentation is time consuming and subjective. Moreover, radiation-induced skin injury may decrease image contrast between the dermis and hypodermis, which increases the difficulty of delineation. Therefore, we have developed an automatic skin segmentation tool (ASST) based on the active contour model with two significant modifications: (i) The proposed algorithm introduces a novel dual-curve scheme for the double skin layer extraction, as opposed to the original single active contour method. (ii) The proposed algorithm is based on a geometric contour framework as opposed to the previous parametric algorithm. This ASST algorithm was tested on a breast cancer image database of 730 ultrasound breast images (73 ultrasound studies of 23 patients). We compared skin segmentation results obtained with the ASST with manual contours performed by two physicians. The average percentage differences in skin thickness between the ASST measurement and that of each physician were less than 5% (4.8 ± 17.8% and -3.8 ± 21.1%, respectively). In summary, we have developed an automatic skin segmentation method that ensures objective assessment of radiation-induced changes in skin thickness. Our ultrasound technology offers a unique opportunity to quantify tissue injury in a more meaningful and reproducible manner than the subjective assessments currently employed in the clinic.