Multi-view sonar image generation via GAN trained with limited data for underwater object classification and detection

Generative Adversarial Networks (GANs) have emerged as a promising approach to address the data scarcity issue in intelligent underwater object recognition. However, existing advanced GANs still suffer from the challenge of discriminator overfitting when constrained by limited sonar data. To address this challenge, this work proposes a Dual Augmentation method for GANs (DA-GAN) to stabilize training under limited data, thereby synthesizing high-fidelity and multi-view sonar images. DA-GAN incorporates two core modules: Adaptive Adversarial Example Augmentation (AAEA) and Random Differentiable Augmentation (RDA). The AAEA module generates adversarial examples by applying imperceptible perturbations to generated sonar images using Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), and Projected Gradient Descent attack (PGD), effectively deceiving the discriminator. Adversarial examples strategically replace real samples according to the degree of discriminator overfitting. Concurrently, the RDA module applies random differentiable augmentations to both real and fake sonar samples, further mitigating the overfitting. Furthermore, leveraging the local smoothness of the latent space, we design three methods—nonlinear perturbation, sparse sign perturbation, and style-mixing to perturb the latent codes, generating multi-view sonar images. Experiment results demonstrate that DA-GAN effectively mitigates discriminator overfitting, generating sonar images with superior quality compared to StyleGAN2 on the SCTD dataset, with the FID improvement from 175.341 to 87.187. On the KLSG dataset, DA-GAN achieves an FID score of 58.067. Furthermore, after augmenting the SCTD dataset, the classification model (e.g., ResNet-34) achieves a 12.50% improvement in global accuracy, while the detection model (e.g., YOLOv5s) shows a 6.93% increase in mAP0.5:0.95.