Generating Synthetic Medical Images with 3D GANs

Abstract

This thesis presents a novel approach to overcoming the challenges associated with the scarcity of annotated medical image data, a significant hurdle in cancer detection. We propose the use of Generative Adversarial Networks (GANs) to generate synthetic high-resolution 3-dimensional CT images and corresponding tumor masks, thereby enhancing the volume and diversity of available training data for machine learning models. This thesis seeks to answer two primary questions: Can GANs generate realistic high-resolution 3D CT image/mask pairs? And to what extent do synthetic CT images generated by GANs impact the accuracy of a state-of-the-art cancer segmentation model? The thesis conducts a comprehensive evaluation of various GAN models, including Vanilla GAN, Wasserstein GAN, StyleGAN2, FastGAN, and Hierarchical Amortized GAN (HA-GAN). The HA-GAN model, in particular, showed exceptional potential, demonstrating superior capacity in generating high-resolution synthetic images. These results were substantiated by multiple evaluation metrics, such as the Inception score, Frechet Inception Distance (FID), and a Visual Turing Test, where the synthesized images were presented to healthcare professionals. The results were compelling, as the professionals were frequently unable to distinguish between synthetic and real images. The synthetic data, when integrated with actual images, facilitated the training of the SegResNet model from the MONAI's Auto3Dseg framework, aiming to optimize the accuracy of tumor segmentation in 3D CT images. Notably, incorporating synthetic images into the training set significantly boosted the Dice Similarity Coefficient (DSC), thereby affirming the effectiveness of our proposed method. The proposed method surpassed the baseline score of 0.48693, achieving an improved score of 0.52193. This research's foremost contribution lies in its methodology for generating high-definition 3D CT image/mask pairs, thus significantly enriching existing medical datasets and enhancing the precision of medical image segmentation. Moreover, the thesis work has produced two unique datasets of synthesized image/mask pairs with resolutions of 128x128x64 and 256x256x128, available for distribution. The code artifact and datasets are available in appendices. The work in this thesis underscores the potential of GANs, with a special emphasis on HA-GAN, to surmount the challenge of limited medical training data, thereby pushing the boundaries of machine learning models in the realm of medical image segmentation and cancer detection.