Dongkuan (DK) Xu / 胥栋宽

Hello! I am an Assistant Professor at NC State CS, leading the NCSU Generative Intelligent Computing Lab and working on machine learning, natural language processing, and computer vision. I have been honored with the Microsoft Accelerating Foundation Models Research Award 2024, the NCSU Carla Savage Award 2024, and the Best Paper Award of ICCCN 2023. I received my Ph.D. at Penn State, and received my M.S. and B.E. at the University of Chinese Academy of Sciences and Renmin University of China, respectively.

I has been collaborating with Microsoft Research exploring trustworthiness evaluation and efficient hyperparameter-architecture search of Foundation Models, and with Google DeepMind to enable scalable and adaptive learning for Vision-Language Models. I was an intern research scientist at Moffett AI, investigating low-resource model compression. I also spent some wonderful time at NEC Labs America on contrastive learning and multi-task learning.

Other than my work, I am a big fan of American football. I love Nittany Lions, New York Giants, and Dallas Cowboys. I also like workout and soccer ball.

Email  /  CV (Oct 2023)  /  Twitter  /  Google Scholar  /  LinkedIn   

I'm looking for multiple PhDs / interns, particularly from underrepresented groups, to work on Generative AI. Feel free to send me your CV. Once we have a commitment to each other, trust me I will do my best to help you!   

My research is fundamentally grounded in exploring and advancing Artificial General Intelligence, with particular emphasis on studying the autonomy of intelligent agents, reasoning reliability, and resource efficiency in Generative AI Systems. My research group provides full-stack solutions, ranging from theoretical optimization methods and data-centric strategies to the development of efficient deep learning techniques and the co-design of algorithms and hardware. My long-term research goal is to liberate AI productivity and democratize its application to serve a broader range of populations and real-world applications, equally, sustainably, and responsibly.

• Task Planning / External Tool Use of Large Language/Vision Models

• Reliable & Scalable Reasoning of Large Language/Vision Models

• Data/Tool Generation, Retrieval & Optimization

• Algorithm-Hardware Co-design for Generative AI Acceleration

• Applications: Education, Agriculture, Networking, Scientific Discovery, Transportation, Healthcare

• MISSION: To build AGI through collective growth of generative intelligent agents

• Demo I (The Basics: Create, Config, Assemble)

• Demo II (Agent Customization and Interaction)

• Demo III (Agent Specialization and Evaluation)

News

Workshop on DL-Hardware Co-Design for Generative AI Acceleration to be held at DAC 2024 (Lead Chair)

Workshop on Dataset Distillation for Computer Vision to be held at CVPR 2024 (Co-Chair) [link]

• Topics: distillation methods, theory study, benchmarks, hardware approaches, etc

• Call for Papers: submission deadline is March 29, 2024

Socially-Relevant Computing and Analytics REU Site 2024 @ NCSU (Undergraduate Mentor) [link]

• Goal: Provide research experience to undergrad students from populations that are under-represented in computing

• Applications Now Open: deadline is Feb 15, 2024

NC State ChatGPT Workshops: Integrating ChatGPT into K-12 Classrooms 2023 (Co-Chair) [link]

• Slides & Videos: (i) Introduction [link], (ii) Risk Understanding [link], (iii) Classroom Integration [link], (iv) AI Tutoring [link]

Workshop on Resource-Efficient Learning for Knowledge Discovery at KDD 2023 (Co-Chair) [link]

Workshop on DeepLearning-Hardware Co-Design for AI Acceleration at AAAI 2023 (Lead Chair) [link]

2024

We present a modular prompting framework to solve robustness issues in code comprehension for LLMs by leveraging main-purpose reasoning guidance and iterative reasoning enhancement.

We explore cost-effective radio map generation using generative diffusion probabilistic models, applicable to both indoor and outdoor wireless network scenarios, particularly valuable in complex scenarios where obtaining comprehensive measurements is challenging.

We comprehensively investigate value alignment approaches. We first unpack the historical context of alignment tracing back to the 1920s (where it comes from), then delve into the mathematical essence of alignment (what it is), shedding light on the inherent challenges.

We investigate for the first time the reliability of sparse training from an out-of-distribution (OOD) perspective, which jointly considers OOD reliability and efficiency and has important implications for real-world deep neural network applications.

We introduce ToolNet, a plug-and-play method to organize massive tools into a directed graph, facilitating their use by large language models (LLMs) via in-context learning.

We present PGExplainer, a parameterized explainer for Graph Neural Networks (GNNs). PGExplainer adopts a deep neural network to parameterize the generation process of explanations and provide a global understanding of any GNN models on arbitrary machine learning tasks.

We propose an adaptive gradient correction method to accelerate and stabilize sparse training. Our method reduces the number of epochs up to 52.1% compared to the leading sparse training methods. Our method is compatible with both unstructured and structured sparse training pipelines.

We introduce AutoST, a training-free NAS method for Spiking Transformers, to rapidly identify high-performance Spiking Transformer architectures.

This study makes contributions to the field of computer science education, and explores the feasibility of utilizing large language models (LLMs) for automating feedback for Java programming assignments in an introductory computer science (CS1) class.

2023

We present a modular ALM framework to solve multi-step reasoning by decoupling reasoning from tool feedback and observations. Theoretical decomposition of prompt tokens establishes that our method substantially reduces prompting redundancy in prevailing Thought-Action-Observation ALM systems.

We present an augmented language model platform that enables flexible customization of agents through simple configurations, seamlessly integrating various language models, task formats, prompting modules, and plugins into a unified paradigm.

We propose two model augmentation techniques, i.e. using early-stage models and weight perturbation to learn an informative synthetic set with significantly reduced training cost. Extensive experiments demonstrate that our method achieves up to 20× speedup.

We propose a novel early exiting strategy based on cascading input similarity with valid assumptions on saturation states in visual-language models, a pioneering exploration of extending early exiting selection to encoders and decoders of sequence-to-sequence architectures.

We show that distilled data lead to not-calibratable networks due to the loss of information that is semantically meaningful but unrelated to classification tasks. We propose Masked Temperature Scaling & Distillation Training to mitigate these limitations while maintaining the efficiency.

This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures. We propose pFedHR, focusing on solving the problem of heterogeneous model cooperation.

We aim to answer: (i) What is the core to defend against adversarial attacks for sparse language models? (ii) How can we efficiently prevent the loss of pre-trained knowledge in pruning to preserve or even enhance robustness?

This paper introduces controllable randomness by generating binary masks in a specific random fashion. We aim to answer: (i) Which is better for pruning? a deterministic way or a randomized way? (ii) Can we design a consistently effective randomized pruning method?

We propose a circular training framework, Colead, which co-evolves both the data-driven synthesizer and the NLU-driven synthesizer to achieve high-quality code generation in the presence of data scarcity and domain growth.

We introduce EfficientLight, an RL-based traffic signal control method that balances model size and performance. In multi-intersection scenarios, our method outperforms all baseline methods with the lowest #paras and the smallest computational cost compared to other RL-based methods.

We develop an adaptive access point (AP) planning approach that can accurately sense the environment dynamics, reconstruct the obstacle map, and then predict the placements of mmWave APs adaptively.

This study highlights the pedagogical significance of predicting useful comments in mutual assessment to promote student learning and reduces the need to collect labeled data via domain adaptation.

How can we quantify the uncertainty in the learning process and enable reliable rare category analysis? We jointly learn the characterizations of rare categories and calibrate the confidence.

We walk through the ODQA models and conclude the core techniques on efficiency. Quantitative analysis on memory cost, processing speed, accuracy and overall comparison are given.

We for the first time identify and study the reliability problem of sparse training and find that sparse training exacerbates the over-confidence problem of DNNs. We then develop a new sparse training method, CigL, to produce more reliable sparse models.

This study explores how ChatGPT personalizes the learning experience, how it can be augmented with math and physical performance, and how educators can ensure that the LLM algorithm is unbiased.

To assist explainable sparse training, we propose important weights exploitation and weights coverage exploration to characterize sparse training. Our method does not need to train dense models, achieving up to 95% sparsity ratio and even higher accuracy than dense training, with same amount of iterations.

We propose an energy efficient spiking neural network training workflow, and design a new drop-andgrow strategy with decreasing number of non-zero weights in the process of dynamically updating sparse mask. We demonstrate extremely high sparsity (i.e., 99%) model performance in SNN based vision tasks.

We develop a gradient-like proposal for any discrete distribution without this strong requirement. Built upon a locally-balanced proposal, our method efficiently approximates the discrete likelihood ratio via a Newton’s series expansion to enable a large and efficient exploration in discrete spaces.

We propose a class-based covariance transfer method from the perspective of disentangling to transfer covariance information in long-tailed classification task.

We propose an autonomous image composition and masking method for cloud masking, a fundamental task in Earth observation problems across social sectors such as agriculture, energy, and water.

We propose an adaptive data augmentation method to avoid ad-hoc choices or painstakingly trial-and-error tuning for time series representation learning.

2022

We develop a few-shot task-agnostic NAS framework, AutoDistil, for distilling large language models into compressed students with variable computational cost. AutoDistil outperforms leading baselines with upto 3x additional reduction in computational cost and negligible loss in task performance.

We introduce the first commercial hardware platform supporting high-degree sparsity acceleration up to 32 times — S4. S4 provides a (sparse) equivalent computation power of 944 TOPS in INT8 and 472 TFLOPS in BF16, and has 20GB LPDDR4 memory with up to 72 GB memory bandwidth in a low 70 Watt power envelope. We demonstrate several-times practical inference speedup on S4 over mainstream inference platforms such as Nvidia T4.

We propose AE-BERT, an automatic and efficient pruning framework. AE-BERT achieves the inference time of a single BERT-BASE encoder on Xilinx Alveo U200 FPGA board that is 1.83x faster compared to Intel(R) Xeon(R) Gold 5218 (2.30GHz) CPU.

We study network pruning of Transformer-based language models under the pre-training and fine-tuning paradigm and propose a counter-traditional hypothesis that pruning increases the risk of overfitting when performed during the fine-tuning phase.

2021

We propose an information-aware contrastive learning framework for graph-structure data, and show for the first time that all recent graph contrastive learning methods can be unified by our framework.

We study how knowledge is transferred and lost during the pre-train, fine-tune, and pruning process, and propose a knowledge-aware sparse pruning process that achieves significantly superior results than existing literature.

We study data augmentation for cross-lingual natural language inference and propose two methods of training a generative model to induce synthesized examples to reflect more diversity in a semantically faithful way.

We utilize multi-instance learning to model the uncertainty of precursor period, and design a contrastive loss to address the issue that annotated anomalies are few.

We propose MT-RMN to dynamically learn task relationships and accordingly learn to assemble composable modules into complex layouts to jointly solve multiple sequence processing tasks.

We propose TRRN to model temporal networks by employing transformer-style self-attention to reason over a set of memories.

We propose MoveSD to model state transition in human movement from a novel perspective, by learning the decision model and integrating the system dynamics.

We introduce Longitudinal deep kernel Gaussian process regression to fully automate the discovery of complex multi level correlation structure from longitudinal data.

2020

We propose to adopt deep neural networks to parameterize the generation process of explanations, which enables a natural approach to multi-instance explanations.

We propose a semi-supervised adversarial perturbation framework that encourages the model to be more robust towards such divergence and better adapt to the target language.

We propose a deep architecture for learning trends in multivariate time series, which jointly learns both local and global contextual features for predicting the trend of time series.

We propose longitudinal kulti-level factorization machine, to the best of our knowledge, the first model to address these challenges in learning predictive models from longitudinal data.

2019

We propose an adaptive neural network for node classification in dynamic networks, which is able to consider the evolution of both node attributes and network topology.

We propose a spatio-temporal attentive RNN model, which aims to learn node representations for classification by jointly considering both the temporal and spatial patterns of the node.

DeepCC utilizes the deep autoencoder for dimension reduction, and employs a variant of Gaussian mixture model to infer the cluster assignments. A mutual information loss is proposed to bridge the training of instances and features.

2018

DMNE coordinates multiple neural networks (one for each input network data) with a co-regularized loss function to manipulate cross-network relationships, which can be many-to-many, weighted and incomplete.

Most multi-instance learning (MIL) methods that study true positive instances ignore 1) the global similarity among positive instances and 2) that negative instances are non-i.i.d.. We propose a MTL method based on positive instance graph updating to address this issue.

This paper reviews the research progress of multi-instance learning (MTL), introduces different assumptions, and categories MTL methods into instance-level, bag-level, and embedded-space. Extensions and major applications in various areas are discussed at last.

2017

We propose a self-adaptive locality-sensitive hashing encoding method for multi-instance learning (MIL), which efficiently deals with large MIL problems.

We propose a metric learning method for multi-instance classification, aiming to find an instance-dependent metric by maximizing the relative distance on neighborhood level.

2016

We propose a positive instance detection method based on multiple instance learning, of which the core idea is that true positive instances should not only be similar to themselves globally but also different from negative instances robustly.

We propose a metric learning approach called multi-metrics classification machine. We establish an optimization problem for each class (each metric) to learn multiple metrics independently.

2015

We introduce the definition of clustering, the basic elements involved in clustering process, and categorize the clustering algorithms into the traditional ones and the modern ones. All the algorithms are discussed comprehensively.

Undergraduate

This paper proposes a support vector machine-based ensemble model to forecast crude oil price based on VECM and stochastic time effective pattern modelling and recognition system (STEPMRS).

This paper presents an integrated model to forecast crude oil prices, where pattern modelling & recognition system is used to model the price trend and error correction model is offered to forecast errors. A neural network layer is employed to integrate the results.

• Panel Reviewer:
  • NSF CORE Program, 2024
  • NSF CAREER Program, 2023

• Column Editor:
  • ACM Special Interest Group on Artificial Intelligence (SIGAI) Newsletter

• Conference/Workshop Chair:
  • The First Workshop on Dataset Distillation for Computer Vision @ CVPR2024 (link)
  • Workshops of Integrating ChatGPT into K-12 Classrooms @ NSF SRCA REU Site 2023 (link)
  • The First Workshop on DL-Hardware Co-Design for AI Acceleration @ AAAI2023 (link)
  • International Workshop on Resource-Efficient Learning for Knowledge Discovery (RelKD'23) @ KDD2023 (link)
  • The Conference on Machine Learning Algorithms & Natural Language Processing (MLNLP'22, 23) (link)

• Academic Committee Member:
  • Machine Learning Algorithms & Natural Language Processing (MLNLP) (link)

• Area Chair:
  • LREC-COLING'24

• Session Chair:
  • Research Track of KDD'22
  • ADS Track of KDD'22

• Senior Program Committee Member:
  • AAI'23, 24
  • IJCAI'21

• Program Committee Member:
  • ICLR'21, 22, 23
  • ICML'21, 22
  • NeurIPS'20, 21, 22, 23
  • AAAI'20, 21, 22
  • ISQED'23
  • KDD'20, 21, 22
  • ACL Rolling Review'22
  • LoG'22
  • IJCAI'20, 22
  • NAACL'21
  • EMNLP'20, 21
  • COLING'22
  • WSDM'22, 23
  • SDM'22
  • EACL'21
  • ACM CIKM'20, 21, 22
  • AACL-IJCNLP'20, 22
  • IJCNN'18, 19, 20, 21

• Journal Reviewer:
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
  • Communications of the ACM
  • IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
  • IEEE Transactions on Knowledge and Data Engineering (TKDE)
  • IEEE Transactions on Cybernetics
  • Information Fusion
  • ACM Transactions on Knowledge Discovery from Data (TKDD)
  • Pattern Recognition
  • Neural Networks
  • Neurocomputing
  • ACM Transactions on Asian and Low-Resource Language Information Processing
  • IEEE Access
  • Neural Computation
  • Complexity
  • Soft Computing
  • Complex & Intelligent Systems
  • Multimedia Tools and Applications
  • Big Data

• Conference Volunteer:
  • The Annual Conference of NAACL-HLT, 2021
  • Backuping SDM Session Chairs, 2021
  • The 35th AAAI Conference on Artificial Intelligence, 2021
  • The 26th SIGKDD Conference on Knowledge Discovery and Data Mining, 2020

• Instructor at NC State

  • CSC 791&591: Advanced Topics in Efficient Deep Learning, Fall 2022, Spring 2024
    Course Materials: Dive into Deep Learning

  • CSC 422: Automated Learning and Data Analysis, Spring 2023, Fall 2023
    Course Materials: Introduction to Data Mining (2nd Edition)

• Teaching Assistant at Penn State

  • SRA268 - Visual Analytics, Fall 2021
    Instructor: Prof. Mahir Akgun    Course Materials: Visual Analytics with Tableau
    (Responsible for teaching lab classes of 46 students)

  • SRA450 - Cybercrime and Cyberwar, Fall 2021
    Instructor: Prof. John Hodgson    Course Materials: Cybersecurity: What Everyone Needs to Know

  • DS/CMPSC 410 - Programming Models for Big Data, Spring 2021
    Instructor: Prof. John Yen    Course Materials: Learning Spark

  • SRA365 - Statistics for Security and Risk Analysis, Fall 2020
    Instructor: Dr. James Farrugia    Course Materials: Discovering Statistics Using R

  • DS402 - Introduction to Social Media Mining, Spring 2020
    Instructor: Prof. Suhang Wang    Course Materials: Social Media Mining: An Introduction

  • SRA365 - Statistics for Security and Risk Analysis, Spring 2019
    Instructor: Dr. Katherine Hamilton    Course Materials: Theoretical Foundations of Intermediate Statistics

  • IST210 - Organization of Data, Fall 2018
    Instructor: Prof. Xiang Zhang    Course Materials: Database Systems Concepts
    (The Award for Excellence in Teaching Support)

• Guest Lecturer

  • COSI 133A: Graph Mining
    Brandeis University, 2021 Fall
    

  • COSI 165B: Deep Learning
    Brandeis University, 2021 Spring
    

• Postdoctoral Researcher

  • Zhiyuan Peng, NC State
    Topic: Augmented Large Language Model

• Ph.D. Students

  • Chengyuan Liu, NC State
    Topic: Large Language Models in Education

• Master Students

  • LiChia Chang, NC State
    Topic: Efficient Retrieval-Augmented Generation

  • John Zhu, NC State
    Topic: Trustworthy Evaluation of LLMs

  • Teddy Chen, NC State
    Topic: LLM Bias

• Undergraduate Researchers

  • Aditya Basarkar, Undergraduate at NC State
    Topic: LLM-powered Math Reasoning

  • Zihan (Z) Dong, NC State
    Topic: Large Language Models in Education

• Intern Researchers

  • Bowen Lei, Apple
    Topic: Theoretical Foundations of Efficient Learning

  • Zhengdong Zhang, Amazon
    Topic: Reliable Large Language Models

  • Shengjie Liu, PhD at NC State
    Topic: Robust Multi-modal LLM Agents

  • Xukun Liu, Master at Northwestern University
    Topic: Accelerating LLM Decoding

  • Berwin Chen, Master at University of Birmingham
    Topic: Retrieval-Augmented Generation for Scientific Discovery

  • Shanlin Liu, Master at University of Shanghai for Science and Technology
    Topic: Retrieval-Augmented Generation for Scientific Discovery

  • Shengkun Tang, Undergraduate at Wuhan University
    Topic: Multi-modal Foundation Models

  • Huanhuan Ma, Master at Chinese Academy of Sciences
    Topic: Trustworthiness Evaluation of LLMs

• The Impact of AI on Our Lives and Beyond (Talk news, in Chinese)
  Raleigh, NC, USA, March 2024
  Fo Guang Shan Buddhist Temple, North Carolina

• Advancing Environmental Strategies: Leveraging Foundation Models for Enhanced Geospatial Analytics and Conservation
  Raleigh, NC, USA, March 2024
  Forest Carbon Solutions Initiative (FCSI), NC State

• How LLMs Work and Cutting-Edge Research on Generative AI
  Online, USA, Dec 2023
  STARS AI Scholars Program (link)

• Sculpting the Future of Collective Growth in Collaborative AI (Talk news)
  Microsoft Research Asia, Online, Sep 2023
  ACE (Advance, Creativity and Empowerment) Talk

• ChatGPT in Corporate Real Estate - Unlocking the Potential (Web & Slides)
  Raleigh, NC, USA, Aug 2023
  CoreNet Global Carolinas Chapter (link)

• Testing Accuracy is Not All You Need: Less Training Cost & More Testing Reliability
  Rutgers University, New Brunswick, USA, Feb 2023
  Rutgers Efficient AI (REFAI) Seminar (link)

• Resource-efficient Deep Learning: Democratizing AI at Scale
  Online, USA, June 2022
  Pinterest Machine Learning Lunch

• Resource-efficient Deep Learning: Democratizing AI at Scale
  Online, USA, May 2022
  Amazon Search (A9)

• Parameter Efficiency: Democratizing AI at Scale (Slides)
  Waltham, MA, USA, Dec. 2021
  Brandeis University

• Chasing Efficiency of Pre-trained Language Models
  Redmond, Washington, USA, Jun. 2021
  Microsoft Research Lab

• BERT Pruning: Structural vs. Sparse (Slides)
  Waltham, MA, USA, Apr. 2021
  Brandeis University.

• BERT, Compression and Applications (Slides)
  Mountain View, USA, Apr. 2021
  Xpeng Motors

• BERT Architecture and Computation Analysis (Slides)
  Los Altos, USA, May. 2020
  Moffett.AI

• Anomaly Precursor Detection via Deep Multi-Instance RNN (Slides)
  Princeton, USA, May. 2019
  NEC Laboratories America

• North Carolina State University
  • NCSU Carla Savage Award, 2024
  • Microsoft Gift Fund, 2024
  • Microsoft Accelerating Foundation Models Research Award, 2024
  • ICCCN Best Paper Award, 2023

• The Pennsylvania State University
  • College of IST Award for Excellence in Teaching Support (top 2), 2019
  • Third place winner (Eng.) in the 37rd annual PSU Graduate Exhibition (News), 2022
  • NAACL Scholarship, 2021
  • SIAM Student Travel Award, 2021
  • KDD Student Registration Award, 2020
  • AAAI Student Scholarship, 2020
  • IST Travel Award, 2019-2021

• University of Chinese Academy of Sciences
  • President’s Fellowship of Chinese Academy of Sciences (the most prestigious award), 2016
  • National Graduate Scholarship, China (2% in university), 2016
  • Graduate Student Academic Scholarship, 2015-2017

• Renmin University of China
  • First-class Scholarship of Sashixuan Elite Fund, China (5% in university), 2014
  • Kwang-hua Scholarship of RUC, China, 2014
  • Meritorious Winner in Mathematical Contest in Modeling, 2013
  • First-class Scholarship of Social Work and Volunteer Service of RUC, 2013

• IEEE (Institute of Electrical and Electronics Engineers) Membership, 2023-Present
  
• ACL (Association for Computational Linguistics) Membership, 2021-Present
  
• President of Youth Volunteers Association of School of Information of RUC, 2012-2013
  
• Volunteer of Beijing Volunteer Service Federation (BVF), 2012-2014
  
• Leader of National Undergraduate Training Programs for Innovation and Entrepreneurship, 2011-2012

*Last updated on 04/16/2024*
This guy makes a nice webpage