My research interests encompass 1) Privacy-preserving machine learning, particularly deep learning with differential privacy, and 2) Memorization mechanisms within large-scale models.

I am a member of the joint Ph.D. program between Sun Yat-sen University and Microsoft Research Asia. My advisors are Prof. Jian Yin and Prof. Tie-Yan Liu. I received my B.S. degree in Computer Science from Sun Yat-sen University in 2019. My research is supported by Microsoft Research Fellowship.

Here is the link to my resume (last updated on January 16, 2024).

Publications

* denotes alphabetical order.

Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent, [code]
Da Yu, Gautam Kamath*, Janardhan Kulkarni*, Tie-Yan Liu*, Jian Yin*, Huishuai Zhang*
Transactions on Machine Learning Research (TMLR), 2023

Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
Jiyan He*, Xuechen Li*, Da Yu*, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, Jiang Bian
International Conference on Learning Representations (ICLR), 2023

Differentially Private Fine-tuning of Language Models, [code]
Da Yu, Saurabh Naik, Arturs Backurs*, Sivakanth Gopi*, Huseyin A. Inan*, Gautam Kamath*, Janardhan Kulkarni*, Yin Tat Lee*, Andre Manoel*, Lukas Wutschitz*, Sergey Yekhanin*, Huishuai Zhang*
International Conference on Learning Representations (ICLR), 2022

Large Scale Private Learning via Low-rank Reparametrization, [code]
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, Tie-Yan Liu
International Conference on Machine Learning (ICML), 2021

Do not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning, [code]
Da Yu*, Huishuai Zhang*, Wei Chen, Tie-Yan Liu
International Conference on Learning Representations (ICLR), 2021

Availability Attacks Create Shortcuts, [code]
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, Tie-Yan Liu
SIGKDD Conference on Knowledge Discovery and Data Mining, Research Track (KDD), 2022

How Does Data Augmentation Affect Privacy in Machine Learning?, [code]
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, Tie-Yan Liu
AAAI Conference on Artificial Intelligence (AAAI), 2021

Stabilize Deep ResNet with A Sharp Scaling Factor $\tau$, [code]
Huishuai Zhang, Da Yu, Mingyang Yi, Wei Chen, Tie-Yan Liu
Machine Learning, 2022

Gradient Perturbation is Underrated for Differentially Private Convex Optimization
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, Tie-Yan Liu
International Joint Conference on Artificial Intelligence (IJCAI), 2020

Manuscripts

Privacy-Preserving Instructions for Aligning Large Language Models
Da Yu, Peter Kairouz*, Sewoong Oh*, Zheng Xu*
Preprint, 2024

Selective Pre-training for Private Fine-tuning, [code]
Da Yu, Sivakanth Gopi, Janardhan Kulkarni, Zinan Lin, Saurabh Naik, Tomasz Lukasz Religa, Jian Yin, Huishuai Zhang
Preprint, 2023

Challenges towards the Next Frontier in Privacy
Rachel Cummings, Damien Desfontaines, David Evans, Roxana Geambasu, Matthew Jagielski, Yangsibo Huang, Peter Kairouz, Gautam Kamath, Sewoong Oh, Olga Ohrimenko, Nicolas Papernot, Ryan Rogers, Milan Shen, Shuang Song, Weijie Su, Andreas Terzis, Abhradeep Thakurta, Sergei Vassilvitskii, Yu-Xiang Wang, Li Xiong, Sergey Yekhanin, Da Yu, Huanyu Zhang, Wanrong Zhang
Preprint, 2023

Academic Service

I am a reviewer for ICML 2022-2024, NeurIPS 2022-2023, and ICLR 2022-2024. I’m awarded as a top reviewer for several times.