Geodesic
UnGAN: machine unlearning strategies through membership inference
Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part IV, Tome 540 (2024), pp. 46-60 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice du chapitre de livre

As regulatory and ethical demands for data privacy and the right to be forgotten increase, the ability to effectively unlearn specific data points from machine learning models without retraining from scratch becomes paramount. Machine unlearning aims to efficiently eliminate the influence of certain data points on a model. We propose the UnGAN, a novel approach to machine unlearning that leverages Generative Adversarial Networks (GANs) to address the growing need for efficient and reliable data removal from trained models. UnGAN proposes a unique unlearning strategy through membership inference, where a discriminator network is trained to identify whether a given input was part of the model's training set. The discriminator is a three-layer fully connected network employing ReLU activation functions, receiving inputs from the output of the model undergoing unlearning and the class label. This architecture enables the discriminator to learn the membership status of data points with high precision, thereby guiding the unlearning process. 
@article{ZNSL_2024_540_a2,
     author = {A. Zhavoronkin and M. Pautov and N. Kalmykov and E. Sevriugov and D. Kovalev and O. Y. Rogov and I. Oseledets},
     title = {UnGAN: machine unlearning strategies through membership inference},
     journal = {Zapiski Nauchnykh Seminarov POMI},
     pages = {46--60},
     year = {2024},
     volume = {540},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a2/}
}
TY  - JOUR
AU  - A. Zhavoronkin
AU  - M. Pautov
AU  - N. Kalmykov
AU  - E. Sevriugov
AU  - D. Kovalev
AU  - O. Y. Rogov
AU  - I. Oseledets
TI  - UnGAN: machine unlearning strategies through membership inference
JO  - Zapiski Nauchnykh Seminarov POMI
PY  - 2024
SP  - 46
EP  - 60
VL  - 540
UR  - http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a2/
LA  - en
ID  - ZNSL_2024_540_a2
ER  - 
%0 Journal Article
%A A. Zhavoronkin
%A M. Pautov
%A N. Kalmykov
%A E. Sevriugov
%A D. Kovalev
%A O. Y. Rogov
%A I. Oseledets
%T UnGAN: machine unlearning strategies through membership inference
%J Zapiski Nauchnykh Seminarov POMI
%D 2024
%P 46-60
%V 540
%U http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a2/
%G en
%F ZNSL_2024_540_a2
A. Zhavoronkin; M. Pautov; N. Kalmykov; E. Sevriugov; D. Kovalev; O. Y. Rogov; I. Oseledets. UnGAN: machine unlearning strategies through membership inference. Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part IV, Tome 540 (2024), pp. 46-60. http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a2/

[1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need”, Advances in Neural Information Processing Systems, 30, 2017

[2] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”, 9th International Conference on Learning Representations, ICLR, 2021

[3] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, and Others, GPT-4 Technical Report, 2023, arXiv: 2303.08774

[4] J. Lin, R. Men, A. Yang, C. Zhou, M. Ding, Y. Zhang, P. Wang, A. Wang, L. Jiang, X. Jia, and Others, M6: A Chinese Multimodal Pretrainer, 2021, arXiv: 2103.00823

[5] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models”, 2017 IEEE Symposium on Security and Privacy (SP), 2017, 3–18

[6] N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles”, 2022 IEEE Symposium on Security and Privacy (SP), 2022, 1897–1914

[7] T. Nguyen, T. Huynh, P. Nguyen, A. Liew, H. Yin, and Q. Nguyen, A survey of machine unlearning, 2022, arXiv: 2209.02299

[8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, and Y. Bengio, “Generative adversarial networks”, Communications of the ACM, 63:11 (2020), 139–144 | DOI

[9] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks”, International Conference on Machine Learning, 2017, 214–223 | MR

[10] A. Ginart, M. Guan, G. Valiant, and J. Zou, “Making AI Forget You: Data Deletion in Machine Learning”, Advances in Neural Information Processing Systems, 2019, 3513–3526

[11] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified Data Removal from Machine Learning Models”, International Conference on Machine Learning, 2020, 3832–3842

[12] T. Baumhauer, P. Schöttle, and M. Zeppelzauer, “Machine Unlearning: Linear Filtration for Logit-Based Classifiers”, Mach. Learn, 111 (2022), 3203–3226 | DOI | MR | Zbl

[13] Z. Izzo, M. Smart, K. Chaudhuri, and J. Zou, “Approximate Data Deletion from Machine Learning Models”, International Conference on Artificial Intelligence and Statistics, 2021, 2008–2016

[14] S. Neel, A. Roth, and S. Sharifi-Malvajerdi, “Descent-to-Delete: Gradient-Based Methods for Machine Unlearning”, Algorithmic Learning Theory, 2021, 931–962 | MR

[15] Q. Nguyen, B. Low, and P. Jaillet, “Variational Bayesian Unlearning”, Advances in Neural Information Processing Systems, 33, 2020

[16] Y. Wu, E. Dobriban, and S. Davidson, “DeltaGrad: Rapid Retraining of Machine Learning Models”, International Conference on Machine Learning, 2020, 10355–10366

[17] L. Bourtoule, V. Chandrasekaran, C. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine Unlearning”, 2021 IEEE Symposium on Security and Privacy (SP), 2021, 141–159

[18] B. Mirzasoleiman, A. Karbasi, and A. Krause, “Deletion-Robust Submodular Maximization: Data Summarization with "The Right to Be Forgotten”, International Conference on Machine Learning, 2017, 2449–2458

[19] Y. Cao and J. Yang, “Towards Making Systems Forget with Machine Unlearning”, 2015 IEEE Symposium on Security and Privacy, 2015, 463–480

[20] A. Krizhevsky, Learning Multiple Layers of Features from Tiny Images, Master's Thesis, University of Toronto, 2009

[21] J. Brophy and D. Lowd, “Machine Unlearning for Random Forests”, International Conference on Machine Learning, 2021, 1092–1104

[22] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 770–778

[23] V. Gupta, C. Jung, S. Neel, A. Roth, S. Sharifi-Malvajerdi, and C. Waites, “Adaptive Machine Unlearning”, Advances in Neural Information Processing Systems, 34, 2021

[24] H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. Yu, and X. Zhang, “Membership Inference Attacks on Machine Learning: A Survey”, ACM Comput. Surv., 54 (2022), 1–37

[25] M. Bertran, S. Tang, A. Roth, M. Kearns, J. Morgenstern, and S. Wu, “Scalable Membership Inference Attacks via Quantile Regression”, Advances in Neural Information Processing Systems, 36, 2024

[26] A. Sablayrolles, M. Douze, C. Schmid, Y. Ollivier, and H. Jégou, “White-Box vs Black-Box: Bayes Optimal Strategies for Membership Inference”, International Conference on Machine Learning, 2019, 5558–5567

[27] A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes, ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models, 2018, arXiv: 1806.01246

[28] L. Song and P. Mittal, “Systematic Evaluation of Privacy Risks of Machine Learning Models”, 30th USENIX Security Symposium (USENIX Security 21), 2021, 2615–2632

[29] J. Neyman and E. Pearson, “IX. On the Problem of the Most Efficient Tests of Statistical Hypotheses”, Philosophical Transactions of the Royal Society of London. Series A, 231, Containing Papers of a Mathematical or Physical Character (1933), 289–337 | DOI | Zbl

[30] H. Thanh-Tung and T. Tran, “Catastrophic Forgetting and Mode Collapse in GANs”, 2020 International Joint Conference on Neural Networks (IJCNN), 2020, 1–10

[31] M. Mirza and S. Osindero, Conditional Generative Adversarial Nets, 2014, arXiv: 1411.1784

[32] A. Ben-Israel, “The Change-of-Variables Formula Using Matrix Volume”, SIAM J. Matrix Anal. Appl., 21 (1999) | DOI | MR | Zbl

[33] A. Tarun, V. Chundawat, M. Mandal, and M. Kankanhalli, “Fast Yet Effective Machine Unlearning”, IEEE Trans. Neural Netw. Learn. Syst., 2024, 1–10

[34] M. Kurmanji, P. Triantafillou, J. Hayes, and E. Triantafillou, “Towards Unbounded Machine Unlearning”, Advances in Neural Information Processing Systems, 36, 2024

[35] M. Bellemare, I. Danihelka, W. Dabney, S. Mohamed, B. Lakshminarayanan, S. Hoyer, and R. Munos, The Cramer Distance as a Solution to Biased Wasserstein Gradients, 2017, arXiv: 1705.10743 | Zbl