Shun Zhang

Hi! This is Shun Zhang (张舜). I am a founding member of technical staff at Asari AI. My research interests lie in reinforcement learning, large language models, and value alignment.

I received my Ph.D. from the University of Michigan, advised by Prof. Satinder Singh and Prof. Ed Durfee. Prior to that, I received both my B.S. and M.S. in computer science from the University of Texas at Austin, working on research advised by Prof. Peter Stone and Prof. Dana Ballard. I have also spent time at the MIT-IBM Watson AI Lab.

My CV in HTML and PDF.

Email  /  GitHub  /  Google Scholar  /  LinkedIn  /  Twitter

Research

Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble (Short Paper)

We introduce efficient reward model ensemble approaches for reinforcement learning from human feedback (RLHF), achieving better alignment with human values under computational constraints.

Adaptive Online Replanning with Diffusion Models

When used for planning, diffusion models generate complete plans at once without incorporating new observations during the execution. Our algorithm determines when it is necessary to replan with new observations, and replans efficiently.

Planning with Large Language Models for Code Generation

Our algorithm combines Monte-Carlo tree search wth the Transformer beam search algorithm for code generation. It’s more sample-efficient than a well-accepted sampling + filtering baseline.

Hyper-Decision Transformer for Efficient Online Policy Adaptation

Using hypernets to efficiently adapt a Decision Transformer to new tasks with a handful of demonstrations.

Prompting Decision Transformer for Few-shot Policy Generalization

Using trajectory segments of different tasks as prompts for a Decision Transformer to achieve meta-offline reinforcement learning.

From Specification to Topology: Automatic Power Converter Design via Reinforcement Learning

Using Monte-Carlo tree search to automatically design electric circuits.

Efficiently Finding Approximately-Optimal Queries for Improving Policies and Guaranteeing Safety

Querying to Find a Safe Policy Under Uncertain Safety Constraints in Markov Decision Processes

An agent is uncertain about which policies are safe. It either finds a safe policy to accomplish a task or proves that no safe policies exist using a minimum number of queries.

Minimax-Regret Querying on Side Effects for Safe Optimality in Factored Markov Decision Processes

A novel formulation and a query selection algorithm for the avoiding negative side effects problem in safe reinforcement learning.

Modeling Sensory-Motor Decisions in Natural Behavior

Approximately-Optimal Queries for Planning in Reward-Uncertain Markov Decision Processes

A provably-optimal query selection algorithm to resolve reward uncertainty for better planning in reward-uncertain Markov decision processes.

Determining Placements of Influencing Agents in a Flock

Autonomous Intersection Management for Semi-Autonomous Vehicles

Design and source code from Jon Barron's website and Leonid Keselman's Jekyll fork.
CV template from Mike Lee Williams's resume.md.