Zifeng Wang

I am a research scientist at Google. I received my PhD in Machine Learning at SPIRAL Group from Northeastern University, advised by Prof. Jennifer G. Dy. During my PhD, I also work closely with Prof. Stratis Ioannidis and Prof. Yanzhi Wang. I received my BS degree in Electronic Engineering from Tsinghua University. In my college years, I also worked with Prof. Jiwen Lu (Tsinghua), Prof. Jia Deng (Princeton) on computer vision and Prof. Yong Li (Tsinghua) on big data.

I am looking for self-motivated student researchers / research interns with interests and expertise in LLMs! Feel free contact me if you would like to work on cutting-edge challenges in machine learning and LLMs.

Email  /  Google Scholar  /  LinkedIn  /  Twitter  /  CV

• Large language models
• Model adaptation and customization
• Multi-LLM collaboration and interaction
• Agents and their applications
• Continual (Lifelong) learning
• Open set recognition / novel class discovery
• Adversarial robustness and model compression

• 
  Aug 2023 - Present, Google,
  Research Scientist at Cloud AI Research
• 
  Sep 2018 - July 2023, Northeastern University,
  Research assistant at SPIRAL Group
• 
  June 2021 - Jan 2023, Google,
  Student researcher / Research Intern at Cloud AI Research
• 
  Feb 2017 - July 2018, Tsinghua University,
  Research assistant at i-Vision Group
• 
  July 2017 - Sep 2017, University of Michigan,
  Visiting researcher at Vision & Learning Lab

We propose Model Swarms, a collaborative search algorithm to adapt LLMs via swarm intelligence, the collective behavior guiding individual systems. Specifically, Model Swarms starts with a pool of LLM experts and a utility function. Guided by the best-found checkpoints across models, diverse LLM experts collaboratively move in the weight space and optimize a utility function representing model adaptation objectives.

Speculative Knowledge Distillation (SKD) is a novel approach that leverages cooperation between student and teacher models to generate high-quality training data on-the-fly while aligning with the student's inference-time distribution. In SKD, the student proposes tokens, and the teacher replaces poorly ranked ones based on its own distribution, transferring high-quality knowledge adaptively.

To enable LLMs to perform reverse thinking, we introduce Reverse-Enhanced Thinking (RevThink), a framework composed of data augmentation and learning objectives. RevThink outperforms a standard fine-tuning method trained on 10x more forward reasoning, it also exhibits strong generalization to out-of-distribution held-out datasets.

Speculative RAG is a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM. (A complementary strategy to the CaLM paper below from a very different perspective!)

TableRAG is a Retrieval-Augmented Generation (RAG) framework specifically designed for LM-based table understanding. TableRAG leverages query expansion combined with schema and cell retrieval to pinpoint crucial information before providing it to the LMs.

CaLM is a novel verification framework that leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs.

LLMs often struggle to capture relevant information in the middle of their input due to an intrinsic U-shaped attention bias, favoring tokens at the beginning and end. To address this, we propose a calibration mechanism called "found-in-the-middle" to mitigate this bias, greatly improving context relevance and RAG performance.

CodecLM is a general framework for adaptively generating high-quality synthetic data for LLM alignment with different downstream instruction distributions and LLMs. Drawing on the Encode-Decode principles, we use LLMs as codecs to guide the data generation process.

CHAIN-OF-TABLE enhances the reasoning capability of LLMs by leveraging tabular structures to express intermediate thoughts for table-based reasoning. It instructs LLMs to dynamically plan an operation chain according to the input table and its associated question.

QueryForm consists of a novel prompting-based framework for zero-shot document entity recognition with large language models (LLMs), and a large-scale weakly-supervised pre-training method on publicly available webpages.

DualHSIC consists of two complementary components that stem from the Hilbert Schmidt independence criterion (HSIC): HSIC-Bottleneck for Rehearsal (HBR) lessens the inter-task interference and HSIC Alignment (HA) promotes task-invariant knowledge sharing.

SparCL explores sparsity for efficient continual learning and achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity.

DualPrompt presents a novel approach to attach complementary prompts to the pre-trained backbone, and then formulates the continual learning objective as learning task-invariant and task-specific “instructions".

We propose a new learning paradigm for continual learning: our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions.

We investigate the HSIC (Hilbert-Schmidt independence criterion) bottleneck as a regularizer for learning an adversarially robust deep neural network classifier, both theoretically and empirically.

We propose a learn-prune-share (LPS) algorithm which addresses the challenges of catastrophic forgetting, parsimony, and knowledge reuse simultaneously.

We propose Class Discovery Kernel Network with Expansion (CD-KNet-Exp), a deep learning framework for open-world class discovery problem.

• Sparse Continual Learning on the Edge
  - ContinualAI, 2023, Remote
  - AI Time, 2023, Remote (Chinese version)
• Prompting-based Continual Learning
  - AI Talks SG, 2022, Remote
  - ContinualAI, 2022, Remote [recording]
• Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness
  - INFORMS Annual Meeting, 2022, Indianapolis, Indiana [abstract]
  - AI Time, 2022, Remote (Chinese version) [recording][blog]

• Outstanding Student Award in Research, Northeastern University, 2023
  
• Scholar Award, NeurIPS 2022
  
• Best Paper Candidate, ICDM 2020
  
• Best Paper Award, DySPAN 2019
  
• Travel Award, DySPAN 2019
  
• Travel Award, NeurIPS 2019
  
• Dean's Fellowship, Northeastern University, 2018
     Highest honor awarded to new PhD students for out standing academic background.
• Evergrande Scholarship, Tsinghua University, 2016
     Awarded to students with excellent academic performance, scientific potential and overall development.

Conference Reviewer: NeurIPS, ICML, ICLR, CVPR, ICCV, ACL ARR
PC Member: SDM
Journal Reviewer: TPAMI, TMLR, Neural Networks