Geodesic
Realistic adversarial attacks on object detectors using generative models
Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part II–2, Tome 530 (2023), pp. 128-140
Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice du chapitre de livre

An important limitation of existing adversarial attacks on real-world object detectors lies in their threat model: adversarial patch-based methods often produce suspicious images while image generation approaches do not restrict the attacker's capabilities of modifying the original scene. We design a threat model where the attacker modifies individual image segments and is required to produce realistic images. We also develop and evaluate a white-box attack that utilizes generative adversarial nets and diffusion models as a generator of malicious images. Our attack is able to produce high-fidelity images as measured by the Fréchet inception distance (FID) and reduces the mAP of Faster R-CNN model by > 0.2 on Cityscapes and COCO-Stuff datasets. A PyTorch implementation of our attack is available at https://github.com/DariaShel/gan-attack. 
@article{ZNSL_2023_530_a9,
     author = {D. Shelepneva and K. Arkhipenko},
     title = {Realistic adversarial attacks on object detectors using generative models},
     journal = {Zapiski Nauchnykh Seminarov POMI},
     pages = {128--140},
     year = {2023},
     volume = {530},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/ZNSL_2023_530_a9/}
}
TY  - JOUR
AU  - D. Shelepneva
AU  - K. Arkhipenko
TI  - Realistic adversarial attacks on object detectors using generative models
JO  - Zapiski Nauchnykh Seminarov POMI
PY  - 2023
SP  - 128
EP  - 140
VL  - 530
UR  - http://geodesic.mathdoc.fr/item/ZNSL_2023_530_a9/
LA  - en
ID  - ZNSL_2023_530_a9
ER  - 
%0 Journal Article
%A D. Shelepneva
%A K. Arkhipenko
%T Realistic adversarial attacks on object detectors using generative models
%J Zapiski Nauchnykh Seminarov POMI
%D 2023
%P 128-140
%V 530
%U http://geodesic.mathdoc.fr/item/ZNSL_2023_530_a9/
%G en
%F ZNSL_2023_530_a9
D. Shelepneva; K. Arkhipenko. Realistic adversarial attacks on object detectors using generative models. Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part II–2, Tome 530 (2023), pp. 128-140. http://geodesic.mathdoc.fr/item/ZNSL_2023_530_a9/

[1] M. Arjovsky, S. Chintala, and L. Bottou, Wasserstein GAN, 2017, arXiv: 1701.07875

[2] A. Brock, J. Donahue, and K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, 2018, arXiv: 1809.11096

[3] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, Adversarial patch, 2017, arXiv: 1712.09665

[4] H. Caesar, J. R. R. Uijlings, and V. Ferrari, “Coco-stuff: Thing and stuff classes in context”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, 1209–1218

[5] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 3213–3223

[6] P. Dhariwal and A. Nichol, Diffusion models beat gans on image synthesis, 2021, arXiv: 2105.05233

[7] R. Duan, X. Ma, Y. Wang, J. Bailey, A. K. Qin, and Y. Yang, “Adversarial camouflage: Hiding physical-world attacks with natural styles”, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, 997–1005

[8] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets”, NIPS, 2014

[9] I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and harnessing adversarial examples, 2014, arXiv: 1412.6572

[10] S. M. Grigorescu, B. Trasnea, T. T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving”, Journal of Field Robotics, 37 (2019), 362–386 | DOI

[11] C. Guo, M. Rana, M. Cissé, and L. van der Maaten, Countering adversarial images using input transformations, 2018, arXiv: 1711.00117

[12] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs trained by a two time-scale update rule converge to a local Nash equilibrium”, NIPS, 2017

[13] Y. Hu, J.-C. Chen, B.-H. Kung, K.-L. Hua, and D. S. Tan, “Naturalistic physical adversarial patch for object detectors”, 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, 7828–7837

[14] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks”, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 5967–5976

[15] W.-C. Kang, C. Fang, Z. Wang, and J. McAuley, “Visually-aware fashion recommendation and design with generative image models”, 2017 IEEE International Conference on Data Mining (ICDM), 2017, 207–216

[16] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of StyleGAN”, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, 8107–8116

[17] A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks”, Neural Information Processing Systems, 25 (2012)

[18] T. Kynkääniemi, T. Karras, M. Aittala, T. Aila, and J. Lehtinen, “The role of ImageNet classes in Fréchet inception distance”, Proc. ICLR, 2023

[19] G. J. S. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. van der Laak, B. van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis”, Medical image analysis, 42 (2017), 60–88 | DOI

[20] J. Liu, A. Levine, C. P. Lau, R. Chellappa, and S. Feizi, “Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, 14973–14982

[21] Y. Liu, X. Chen, C. Liu, and D. X. Song, Delving into transferable adversarial examples and black-box attacks, 2016, arXiv: 1611.02770

[22] A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier GANs”, International Conference on Machine Learning, 2016

[23] H. Qiu, C. Xiao, L. Yang, X. Yan, H. Lee, and B. Li, SemanticAdv: Generating adversarial examples via attribute-conditional image editing, 2019, arXiv: 1906.07927

[24] S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2015), 1137–1149 | DOI

[25] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models”, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, 10674–10685

[26] C. Saharia, W. Chan, H. Chang, C. A. Lee, J. Ho, T. Salimans, D. J. Fleet, and M. Norouzi, “Palette: Image-to-image diffusion models”, ACM SIGGRAPH 2022 Conference Proceedings, 2022

[27] J. N. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, 2015, arXiv: 1503.03585

[28] Y. Song, R. Shu, N. Kushman, and S. Ermon, “Constructing unrestricted adversarial examples with generative models”, Neural Information Processing Systems, 2018

[29] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, Intriguing properties of neural networks, 2013, arXiv: 1312.6199

[30] T. Wang, T. Zhang, B. Zhang, H. Ouyang, D. Chen, Q. Chen, and F. Wen, Pretraining is all you need for image-to-image translation

[31] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-resolution image synthesis and semantic manipulation with conditional GANs”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, 8798–8807

[32] Z. Wu, S.-N. Lim, L. S. Davis, and T. Goldstein, “Making an invisibility cloak: Real world adversarial attacks on object detectors”, European Conference on Computer Vision, 2019