A Novel Generative Artificial Intelligence-Empowered Data Poisoning Attack and Its Implications for Cybersecurity

Abstract

Generative Artificial Intelligence (AI) has revolutionized industries by automating creative processes and personalizing user interactions. However, alongside its groundbreaking potential, it has also presented significant challenges. For example, generative AI could be employed to expose systems to critical vulnerabilities, and novel cyberattack strategies. In fact, employing generative AI in cyberattacks requires immediate attention as it could intensify the effectiveness of malicious codes due to its high creativity and learning ability. This study introduces a novel non-targeted additive perturbation-based data poisoning attack utilizing the generative capabilities of generative AI. The proposed attack utilizes OpenAI Application Programming Interface (API) to generate diverse and unpredictable visual perturbations in image data during the training phase. The effectiveness of the attack was evaluated on a medical dataset of reticulocyte images using five deep learning classifiers, namely, VGG16, DenseNet121, MobileNetV2, ResNet50, and Xception under three poisoning levels (3.3%, 10%, and 16.7%). The results showed that VGG16 and MobileNet were the most affected by the attack, with a maximum F-score drop of 68% and 74% respectively. The results suggest that the attack effectiveness is architecture dependent with lightweight and texture-biased models exhibit larger drops in F-score and accuracy. In addition, overall higher poisoning levels were associated with higher drops in the F-score and accuracy. Finally, the impact of the attack on an experienced hematologist’s classification ability was examined, highlighting the robustness of human expertise compared to AI. This work underscores the dual edged nature of generative AI and emphasizes the need for advanced defenses against adversarial misuse in AI systems.

Keywords: Cyberattack, DALL-E, Data poisoning, Deep learning, Generative AI, Reticulocyte classification.