About Me

Dr. Juntao Li is now an associate professor at School of Computer Science & Technology, Soochow University, working with Prof. Min Zhang. Before that, he obtained a doctoral degree from Peking University in 2020, supervised by Dr. Rui Yan, Prof. Dongyan Zhao, and Prof. Dongmin Chen.

I am leading a research group on language models. We are dedicated to building application-oriented, open-source, large-size language models with transparency, reusability, and low-cost deployment. More details can be found at OpenNLG.

1. Contact

ljtsudanlp@gmail.com
ljt@suda.edu.cn

2. Research Interests and Selected Papers

(^‘*’ = equal contribution, ^’#’ = corresponding author )

2.1 Pretrained Language Models

• (OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch)[paper][Model & Code], SCIS, CCF-A
• [COLM-2024]Timo: Towards Better Temporal Reasoning for Language Models
• [ACL-24] Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization.
• [ACL-24] Exploring Reversal Mathematical Reasoning Ability for Large Language Models.
• [ACL-24] Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?.
• [ACL-24] CMD: a framework for Context-aware Model self-Detoxification.
• [ACL-24] Demonstration Augmentation for Zero-shot In-context Learning.
• [ACL-24] Rethinking Negative Instances for Generative Named Entity Recognition.
• [ICML-24] Zecheng Tang, Chenfei Wu, Zekai Zhang, Mingheng Ni, Shengming Yin, Yu Liu, Zhengyuan Yang, Lijuan Wang, Zicheng Liu, Juntao Li, Nan Duan. StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis.
• [ICLR-24] Yisheng Xiao, Juntao Li^#, Zechen Sun, Zechang Li, Qingrong Xia, Xinyu Duan, Zhefeng Wang, Min Zhang. Are Bert Family Good Instruction Followers? A Study on Their Potential And Limitations.
• [ICLR-24] Zecheng Tang, Chenfei Wu, Juntao Li, Nan Duan. LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models.
• [EMNLP-23] Yi Su, Yixin Ji, Juntao Li^#, Hai Ye, Min Zhang. Beware of Model Collapse! Fast and Stable Test-time Adaptation for Robust Question Answering. EMNLP 2023. [pdf](CCF B)
• [EMNLP-23] Chuyue Zhou, WangJie You, Juntao Li^#, Jing Ye, Kehai Chen, Min Zhang. INFORM : Information eNtropy based multi-step reasoning FOR large language Models. EMNLP 2023. [pdf](CCF B)
• [EMNLP-23] Yixin Ji, Jikai Wang, Juntao Li^#, Hai Ye, Min Zhang. Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models. EMNLP 2023 Findings. [pdf]
• [EMNLP-23] Zhaochen Su, Juntao Li^#, Zikang Zhang, Zihan Zhou, Min Zhang . Efficient Continue Training of Temporal Language Model with Structural Information. EMNLP 2023 Findings. [pdf]
• [EMNLP-23] Haoke Zhang, Yue Wang, Juntao Li^#, Xiabing Zhou, Min Zhang. G-SPEED: General SParse Efficient Editing MoDel. EMNLP 2023 Findings. [pdf]
• [EMNLP-23] Pei Guo, yisheng xiao, Juntao Li^#, Yixin Ji, Min Zhang. Isotropy-Enhanced Conditional Masked Language Models. EMNLP 2023 Findings. [pdf]
• [EMNLP-23] Lei Geng, Xu Yan, Ziqiang Cao, Juntao Li, Wenjie Li, Sujian Li, Xinjie Zhou, Yang Yang, Jun Zhang. KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model. EMNLP 2023 Findings. [pdf]
• [AI J] Yue Wang, Lijun Wu, Juntao Li^#, Xiaobo Liang, Min Zhang. Are BERT Families Zero-Shot Learners? A Study on Their Potential and Limitations. Artificial Intelligence. [pdf][code](CCF A)
• [ACL-23] Yixin Ji, Jikai Wang, Juntao Li^#, Qiang Chen, Wenliang Chen and Min Zhang. Early Exit with Disentangled Representation and Equiangular Tight Frame. [pdf][code] ACL 2023 Findings
• [EMNLP-22] Zhaochen Su^*, Zecheng Tang^*, Xinyan Guan, Juntao Li^#, Lijun Wu, Min Zhang. Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change. [pdf][code] In EMNLP’22. (CCF B)
• [COLING-22] Dan Qiao, Chenchen Dai, Yuyang Ding, Juntao Li^#, Qiang Chen, Wenliang Chen and Min Zhang. SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training. In COLING’22. (Oral) [pdf][code] (CCF B)
• [EMNLP-20] Hai Ye, Qingyu Tan, Ruidan He, Juntao Li, Hwee Tou Ng, Lidong Bing. Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training. In EMNLP’20. Full paper. [pdf][code] (CCF B)
• [IJCAI-20] Juntao Li, Ruidan He, Hai Ye, Hwee Tou Ng, Lidong Bing, and Rui Yan. Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model. In IJCAI-PRICAI’20. Full paper. [pdf] [code](CCF A)

2.2 Natural Language Generation

• [ACL-24] Efficient Domain Adaptation for Non-Autoregressive Machine Translation.
• [TPAMI] Juntao Li, Xiaobo Liang, Lijun Wu, Yue Wang, Qi Meng, Tao Qin, Tie-yan Liu, Min Zhang. Randomness Regularization with Simple Consistency Training for Neural Networks. TPAMI, 2024
• [TPAMI] Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li^#, Min Zhang, Tao Qin, Tie-yan Liu. A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond. TPAMI, 2023. [pdf] (CCF A)
• [WSDM-24] Yue Wang, Zilong Zheng, Zecheng Tang, Juntao Li^#, Zhihui Liu, Kunlong Chen, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Min Zhang. Towards Better Chinese Spelling Check for Search Engines: A New Dataset and Strong Baseline. WSDM 2024. [pdf](CCF B)
• [NeurIPS-23] Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, Zhongyu Wei, Jian Guo, Nan Duan, Weizhu Chen. AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation. WSDM 2024. [pdf](CCF A)
• [ACL-23] Xiaobo Liang, Zecheng Tang, Juntao Li^#, Min Zhang. Open-ended Long Text Generation via Masked Language Modeling. [pdf][code] ACL 2023 (CCF A)
• [ACL-23] Xiaobo Liang, Juntao Li^#, Lijun Wu, Ziqiang Cao and Min Zhang. Dynamic and Efficient Inference for Text Generation via BERT Family. [pdf][code] ACL 2023 (CCF A)
• [ACL-23] Yue Wang, Dan Qiao, Juntao Li^#, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang and Min Zhang. Towards Better Hierarchical Text Classification with Data Generation. [pdf][code] ACL 2023 Findings
• [ACL-23] Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li^#, Ziqiang Cao and Min Zhang. Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference! [pdf][code] ACL 2023 Findings
• [AAAI-23] Pei Guo, Yisheng Xiao, Juntao Li^#, Lijun Wu, Min Zhang. RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformer. [pdf][code] In AAAI’23 (CCF A)
• [AAAI-23] Yisheng Xiao, Lijun Wu, Ruiyang Xu, Juntao Li^#, Tao Qin, Tie-yan Liu, Min Zhang. AMOM: Adaptive Masking over Masking for Conditional Masked Language Model. [pdf][code] In AAAI’23 (CCF A)
• [NeurIPS-21] Xiaobo Liang, Lijun Wu, Juntao Li^#, Yue Wang, Qi Meng, Wei Chen, Tao Qin, Min Zhang, Tie-yan Liu. R-Drop: Regularized Dropout for Neural Networks.In NeurIPS’21. Full paper. [pdf][code] (CCF A)
• [AAAI-21] Meng-Hsuan Yu, Juntao Li^*, Zhangming Chan, Dongyan Zhao and Rui Yan. Content Learning with Structure-Aware Writing: A Graph-Infused Dual Conditional Variational Autoencoder for Automatic Storytelling. In AAAI’21. Full paper. [pdf] (CCF A)
• [AAAI-20] Juntao Li, Chang Liu, Jian Wang, Lidong Bing, Hongsong Li, Xiaozhong Liu, Dongyan Zhao and Rui Yan. Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce. In AAAI’20. Full paper. [pdf] (CCF A)
• [AAAI-20] Meng-Hsuan Yu, Juntao Li^*, Danyang Liu, Bo Tang, Haisong Zhang, Dongyan Zhao and Rui Yan. Draft and Edit: Automatic Storytelling Through Multi-Pass Hierarchical Conditional Variational Autoencoder. In AAAI’20. Full paper. [pdf] (CCF A)
• [AAAI-20] Danyang Liu, Juntao Li^*, Meng-Hsuan Yu, Ziming Huang, Gongshen Liu, Dongyan Zhao and Rui Yan. A Character-Centric Neural Model for Automated Story Generation. In AAAI’20. Full paper. [pdf] (CCF A)
• [AAAI-19] Juntao Li, Lidong Bing, Lisong Qiu, Dongmin Chen, Dongyan Zhao and Rui Yan. Learning to Write Creative Stories with Thematic Consistency. In AAAI’19. Full paper. [pdf] (CCF A)
• [EMNLP-18] Juntao Li, Yan Song, Haisong Zhang, Dongmin Chen, Shuming Shi, Dongyan Zhao, and Rui Yan. Generating Classical Chinese Poems via Conditional Variational Autoencoder and Adversarial Training. In EMNLP’18. Full paper. [pdf] (CCF B)

2.3 Dialogue Systems

• [FnTIR] Rui Yan, Juntao Li^#, Zhou Yu. Deep Learning for Dialogue System: Chit-Chat and Beyond. Foundations and Trends® in Information Retrieval. 2022, [pdf] (SCI Q1, IF=8.0, 129 pages + 42 pages references)
• [ACL-23] Chongyang Tao, Jiazhan Feng, Tao Shen, Chang Liu, Juntao Li, Xiubo Geng, Daxin Jiang. CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection. [pdf] ACL 2023 (CCF A)
• [TOIS-21] Juntao Li, Chang Liu, Chongyang Tao, Zhangming Chan, DongyanZhao, Min Zhang, Rui Yan. Dialog History Matters! Personalized Response Selection in Multi-turn Retrieval-based Chatbots. TOIS,2021 [pdf] (CCF A)
• [ACL-19] Lisong Qiu, Juntao Li, Wei Bi, Dongyan Zhao and Rui Yan. Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References. In ACL’19. Full paper. [pdf] (CCF A)
• [AAAI-19] Juntao Li, Lisong Qiu, Bo Tang, Dongmin Chen, Dongyan Zhao and Rui Yan. Insufficient Data Can Also Rock! Learning to Converse Using Smaller Data with Augmentation. In AAAI’19. Full paper. [pdf] (CCF A)
• [EMNLP-19] Zhangming Chan, Juntao Li^*, Xiaopeng Yang, Xiuying Chen, Wenpeng Hu, Dongyan Zhao, and Rui Yan. Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders. In EMNLP’19. Full paper. [pdf] (CCF B)

3. Grants

• 第九届中国科协青年人才托举工程项目, PI
• National Science Foundation of China, No. 62206194, PI
• Natural Science Foundation of Jiangsu Province, No. BK20220488, PI
• Alibaba Innovation Research Grant, Co-PI
• Wudao Open Fund (BAAI), PI

4. Experiences

• Microsoft Research Aisa （2023/03-2023/06）, Supervised by Nan Duan
  
• National University of Singapore （2019/09-2020/02）, Supervised by Hwee Tou Ng
  
• Alibaba Damo Academy (2018/11-2019/06), Supervised by Lidong Bing and Xiaozhong Liu
  
• Tencent AI Lab (2018/04-2018/06), Supervised by Yan Song and Lidong Bing
  
• Georgia Institute of Technology (2014/01-2014/05)

5. Tutorials and Talks

• Juntao Li and Rui Yan. Creative and Artistic Writing via Text Generation. In AAAI’20. Tutorial. [Slide]. (CCF A)
• Juntao Li and Rui Yan. Creative and Artistic Writing via Text Generation. In IJCAI’19. Tutorial. [Slide]. (CCF A)
• YSSNLP2022, Deep Learning for Dialogue System: Chit-Chat and Beyond, [video][Slide]

6. Alumni

• Chenchen Dai (Undergraduate student of Soochow University > Algorithm Engineer, Ant Group(蚂蚁金服，算法工程师))
• Yi Deng (Undergraduate student of Soochow University > University College London (UCL), Master)

7. Selected Awards

• Startrack Visiting Scholars (铸星计划), Microsoft Research Aisa
• Outstanding Graduates Awards (优秀毕业生), Peking University
• AAAI Tutorial Grant
  
• National Scholarship (国家奖学金), Peking University
  

8 Academic Services (Program Committee or Reviewer)

• ACL Rolling (Area Chair/Action Editor)
• 2023: ICLR, IJCAI (Senior PC), AACL (Website chair)
• 2022: AAAI, IJCAI, NAACL, KDD, ICLR, EMNLP (Area Chair)
• 2021: AAAI, IJCAI (Senior PC), NAACL-HLT, EACL, ACL (Area Chair), ICONIP (Tutorial, Session Chair)
• 2020: ACL, AAAI, COLING, EMNLP, AACL, CCL
• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI/PAMI)
• IEEE Transactions on Neural Networks and Learning Systems (TNNLS),
• Computational Linguistics (CL)
  
• Transactions on Knowledge and Data Engineering (TKDE)