Addressing dataset misalignment in medical image synthesis with GANs and alignment networks

Panlu You; Wanting Jing; Xiaolian Gao; Dapeng Cheng

doi:10.1117/12.3035019

11 July 2024 Addressing dataset misalignment in medical image synthesis with GANs and alignment networks

Panlu You, Wanting Jing, Xiaolian Gao, Dapeng Cheng

Proceedings Volume 13210, Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024); 1321011 (2024) https://doi.org/10.1117/12.3035019
Event: Third International Symposium on Computer Applications and Information Systems (ISCAIS 2023), 2024, Wuhan, China

Abstract

Generative Adversarial Networks (GANs)¹ have demonstrated remarkable success in generating realistic images, and are increasingly used for image-to-image translation tasks in medical imaging. However, methods like Pix2pix², which rely on pixel losses, require strict alignment of training samples. In medical MRI, strictly aligned training images are often difficult to obtain, as imaging of two modalities is not usually performed simultaneously. Although adversarial networks represented by CycleGAN³, which use consistency losses, can be trained with unpaired images, they often generate images that are biased towards low frequencies and are not clear enough, which may lead to the loss of important anatomical structures in the generated images. To address this issue, we propose a supervised GAN framework based on registration losses, AligFT GAN. The framework, based on deformation field theory, considers misaligned labels as noisy labels, models the noise with a deformation field, and trains the generator using an additional alignment network that estimates displacement vector fields to adaptively fit the misaligned noise distribution. In addition, AligFT GAN employs different types of frequency space losses to regulate the frequency content of the generated images. It combines the well-known properties of MRI K-space⁴ geometry to guide the network training process. By combining our method with GANs with cycle consistency losses, we can mitigate the impact of using misaligned data for training and frequency bias, to synthesize high-quality images.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Panlu You, Wanting Jing, Xiaolian Gao, and Dapeng Cheng "Addressing dataset misalignment in medical image synthesis with GANs and alignment networks", Proc. SPIE 13210, Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024), 1321011 (11 July 2024); https://doi.org/10.1117/12.3035019

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
11 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Gallium nitride

Education and training

Image registration

Deformation

Medical imaging

Image segmentation

Magnetic resonance imaging

Show All Keywords

Keywords/Phrases

Search In:

Publication Years