Research

ASTROPHYSICS, ASTRONOMY, COSMOLOGY

A Deep Learning View of the Sensus of Galaxy Clusters in IllustrisTNG [paper] [arXiv] [doi] [bibTex] [media]
The origin of the diverse population of galaxy clusters remains an unexplained aspect of large-scale structure formation and cluster evolution. We present a novel method of using X-ray images to identify cool core (CC), weak cool core (WCC), and non cool core (NCC) clusters of galaxies, that are defined by their central cooling times. Without any spectral information, the deep learning algorithm is able to identify CC, WCC, and NCC clusters, achieving balanced accuracies (BAcc) of 92%, 81%, and 83%, respectively. The performance is superior to classification by conventional methods using central gas densities, with an average BAcc=81%⁠, or surface brightness concentrations, giving BAcc=73%⁠. We use Class Activation Mapping to localize discriminative regions for the classification decision. From this analysis, we observe that the network has utilized regions from cluster centers out to r ≈ 300 kpc and r ≈ 500 kpc to identify CC and NCC clusters, respectively.

Optical Wavelength Guided Self-Supervised Feature Learning For Galaxy Cluster Richness Estimate [Paper] [arXiv] [Poster] [Code] [Webpage] [BibTex]
Most galaxies in the nearby Universe are gravitationally bound to a cluster or group of galaxies. Their optical contents, such as optical richness, are crucial for understanding the co-evolution of galaxies and large-scale structures in modern astronomy and cosmology. The determination of optical richness can be challenging. We propose a self-supervised approach for estimating optical richness from multi-band optical images. The method uses the data properties of the multi-band optical images for pre-training, which enables learning feature representations from a large but unlabeled dataset. We apply the proposed method to the Sloan Digital Sky Survey. The result shows our estimate of optical richness lowers the mean absolute error and intrinsic scatter by 11.84% and 20.78%, respectively, while reducing the need for labeled training data by up to 60%. We believe the proposed method will benefit astronomy and cosmology, where a large number of unlabeled multi-band images are available, but acquiring image labels is costly.

Multi-Branch Attention Networks for Classifying Galaxy Clusters [paper] [poster] [webpage] [doi] [bibTex]
This paper addresses the task of classifying galaxy clusters, which are the largest known objects in the Universe. Galaxy Clusters can be categorized into cool-core (CC), weak-cool-core (WCC), and non-cool-core (NCC) clusters, defined by their central cooling times. Conventional approaches in astrophysics for conducting such classification are through measuring their surface brightness concentrations or central gas densities, which are inaccurate. Off-the-shelf deep learning approaches for solving this problem would be taking entire images as inputs and predicting cluster types directly. However, this strategy is limited in that central cooling times are usually related to only small informative regions near the center, and feeding unrelated outer regions into the network may hurt the performance. We propose multi-branch attention networks that utilize attention and bivariate Gaussian distribution to identify the galaxy cluster type. Our loss function is designed by encompassing our domain knowledge that the central cooling time of three different types of galaxy clusters (CC, WCC, NCC) varies continuously.

MEDICAL IMAGING

Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification [pdf] [bibTex] [poster] [code] [slides /video] [webpage]
Recent works have shown that deep neural networks can achieve super-human performance in a wide range of image classification tasks in the medical imaging domain. However, these works have primarily focused on classification accuracy, ignoring the important role of uncertainty quantification. Empirically, neural networks are often miscalibrated and overconfident in their predictions. This miscalibration could be problematic in any automatic decision-making system, but we focus on the medical field in which neural network miscalibration has the potential to lead to significant treatment errors. We propose a novel calibration approach that maintains the overall classification accuracy while significantly improving model calibration. The proposed approach is based on expected calibration error, which is a common metric for quantifying miscalibration. Our approach can be easily integrated into any classification task as an auxiliary loss term, thus not requiring an explicit training round for calibration. We show that our approach reduces calibration error significantly across various architectures and datasets.

Inconsistent Performance of Deep Learning Models on Mammogram Classification [bibTex] [media] [doi]
Performance of recently developed deep learning models for image classification surpasses that of radiologists. However, there are questions about model performance consistency and generalization in unseen external data. The purpose of this study is to determine whether the high performance of deep learning on mammograms can be transferred to external data with a different data distribution. Six deep learning models were evaluated on four different mammogram data sets. The models achieved between 0.71 and 0.95 auROC on the validation data set. However, when testing on the external testing set, all six models were significantly decreased, only between 0.44 (95% CI: 0.43-0.45) and 0.65 (95% CI: 0.64-0.66). Our results demonstrate performance inconsistency across the data sets and models, indicating that the high performance of deep learning models on one data set cannot be readily transferred to unseen external data sets, and these models need further assessment and validation before being applied in clinical practice.

Joint 2D-3D Breast Cancer Classification [pdf][bibTex]
Breast cancer is the malignant tumor that causes the highest number of cancer deaths in females. Digital mammo- grams (DM or 2D mammogram) and digital breast tomosynthesis (DBT or 3D mammogram) are the two types of mammography imagery that are used in clinical practice for breast cancer detec- tion and diagnosis. Radiologists usually read both imaging modal- ities in combination; however, existing computer-aided diagnosis tools are designed using only one imaging modality. Inspired by clinical practice, we propose an innovative convolutional neural network (CNN) architecture for breast cancer classification, which uses both 2D and 3D mammograms, simultaneously. Our experiment shows that the proposed method significantly improves the performance of breast cancer classification. By assembling three CNN classifiers, the proposed model achieves 0.97 AUC, which is 34.72% higher than the methods using only one imaging modality.

2D Convolutional Neural Networks for 3D Digital Breast Tomosynthesis Classification [pdf] [bibTex]
Automated methods for breast cancer detection have focused on 2D mammography and have largely ignored 3D digital breast tomosynthesis (DBT), which is frequently used in clinical practice. The two key challenges in developing automated methods for DBT classification are handling the variable number of slices and retaining slice-to-slice changes. We propose a novel deep 2D convolutional neural network (CNN) architecture for DBT classification that simultaneously overcomes both chal- lenges. Our approach operates on the full volume, regardless of the number of slices, and allows the use of pre-trained 2D CNNs for feature extraction, which is important given the limited amount of annotated training data. In an extensive evaluation on a real-world clinical dataset, our approach achieves 0.854 auROC, which is 28.80% higher than approaches based on 3D CNNs. We also find that these improvements are stable across a range of model configurations.

Automatic Hand Skeletal Shape Estimation from Radiographs
[pdf] [bibTex]
Rheumatoid arthritis (RA) is an autoimmune disease whose common manifestation involves the slow destruction of joint tissue, a damage that is visible in a radiograph. Over time, this damage causes pain and loss of functioning, which depends, to some extent, on the spatial deformation induced by the joint damage. Building an accurate model of the current deformation and predicting potential future deformations are the important components of treatment planning. Unfortunately, this is currently a time-consuming and labor-intensive manual process. To address this problem, we propose a fully automated approach for fitting a shape model to the long bones of the hand from a single radiograph. Critically, our shape model allows sufficient flexibility to be useful for patients in various stages of RA. Our approach uses a deep convolutional neural network to extract low-level features and a conditional random field (CRF) to support shape inference. Our approach is significantly more accurate than previous work that used hand-engineered features. We provide a comprehensive evaluation for various choices of network hyperparameters, as current best practices lack significantly in this domain. We evaluate the accuracy of our pipeline on two large datasets of hand radiographs and highlight the importance of the low-level features, the rel- ative contribution of different potential functions in the CRF, and the accuracy of the final shape estimates. Our approach is nearly as accurate as a trained radiologist and, because it only requires a few seconds per radiograph, can be applied to large datasets to enable better modeling of disease progression.

GANai: Standardizing CT Images Using Generative Adversarial Network with Alternative Improvement [pdf ] [bibTex] [poster]
CT is a widely-used diagnostic image modality routinely used for assessing anatomical tissue characteristics. However, non-standardized imaging protocols are commonplace, which poses a fundamental challenge in large-scale cross-center CT image analysis. One approach to address the problem is to standardize and normalize CT images using image synthesis algorithms including generative adversarial network (GAN) models. However, existing GAN models are not directly applicable to this task mainly due to the lack of constraints on the mode of data to generate. Furthermore, they treat every image equally, but in real applications, certain images are more difficult to standardize than the others. All these may lead to the lack-of-detail problem in CT image synthesis. We present a new GAN model called GANai to mitigate the differences in radiomic features across CT images captured using non-standard imaging protocols. GANai introduces a new alternative improvement training strategy to alternatively and gradually improve GAN model performance. The new training strategy enables a series of technical improvements, including phase-specific loss functions, phase-specific training data, and the adoption of ensemble learning, leading to better model performance. The experimental results show that efficiency and stability of GAN models have been much improved in GANai and our model is significantly better than the existing state-of-the-art image synthesis algorithms on CT image standardization.

Radiomic Features of Lung Cancer and Their Dependency On Ct Image Acquisition Parameters [html] [bibTex]
While image features extracted from CT images are of great importance in aiding diagnosis and personalizing therapy planning for lung cancer patients, they may highly depend on CT acquisition parameters, leading non-reproducible and redundant information. We investigate potential effects of CT imaging acquisition parameters on radiomic features. Our preliminary study shows that image features are sensitive to some CT acquisition parameter. Further study is needed to test image feature reproducibility and redundancy. The identification of reproducible, non-redundant image features are crucial to multi-center clinical trials in the radiomics of the lung cancer.