Publications | Xinyi Liu's Personal Website

Imitation Learning from Human Motion Alone Does Not Guarantee Biomechanically Plausible Gait Kinetics

Thu, 12 Mar 2026 00:00:00 +0000

Mathematical Formulation

Imitation Learning Objective

The policy maximizes expected cumulative reward:

$$ J(\theta) = \mathbb{E}_{\pi_\theta} \left[ \sum_{t=0}^T \gamma^t r(s_t, a_t, q^{\exp}_t, f^{\exp}_{\mathrm{grf},t}, x^{\exp}_{\mathrm{cop},t}) \right] $$

Where $ r(\cdot) $ includes kinematics $ R_k $ and dynamics $ R_{\mathrm{dyn}} $ terms.

Reward Components

Kinematics reward:

$$ R_k = w_p R_p + w_{ee} R_{ee} + w_{rp} R_{rp} + w_{rv} R_{rv} + w_{vf} R_{vf} $$

Dynamics reward:

$$ R_{\mathrm{dyn}} = w_{\mathrm{grf}} R_{\mathrm{grf}} + w_{\mathrm{cop}} R_{\mathrm{cop}} $$

With GRF term $ R_{\mathrm{grf}} = \exp\left( -k_{\mathrm{grf}} \|\Delta f_{\mathrm{grf}}\|^2 \right) $ and CoP term $ R_{\mathrm{cop}} = \exp\left( -k_{\mathrm{cop}} \|\Delta x_{\mathrm{cop}}\|^2 \right) $.

Technical Approach

Key Components

RL-based IL with PPO and residual force control in MuJoCo.
Ablation on kinetics rewards: baseline (kinematics-only, MOIL), +GRF, +CoP, full KAIL.
Forward dynamics simulation of musculoskeletal model matching Visual3D inverse dynamics.

Performance Metrics

Joint angle RMSE <7° across conditions.
GRF correlation improves significantly with kinetics rewards (p<0.001).
Joint moment RMSE reduced, correlations increased (p<0.0001) in KAIL vs. MOIL. The figure shows simulated human gait skeletons adapting walking speeds via imitation learning, illustrating kinematic fidelity in MuJoCo environments similar to the study’s setup.

Research Significance

Motion imitation alone prioritizes kinematic matching but yields implausible GRFs and joint moments due to unphysiological foot-ground interactions. KAIL with GRF/CoP rewards ensures biomechanical plausibility, critical for biomechanics analysis, clinical diagnostics, and wearable robot design.

Implementation Highlights

Data from motion capture and force-plate treadmill on healthy subject walking at 1.2 m/s.
MuJoCo model with floating-base, impedance control, trained 900 PPO iterations.
Evaluation via GRF butterfly diagrams, CPCC, joint moment RMSE/correlations vs. inverse dynamics.

Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

Mon, 15 Jan 2024 00:00:00 +0000

Mathematical Formulation

Policy Distillation Objective

The core optimization problem is formulated as:

$$ \min_{\pi_{\theta}} \mathbb{E}_{s \sim d_{\text{expert}}} \left[ L_{\text{distill}}(\pi_{\theta}(s), \pi_{\text{expert}}(s)) + \lambda L_{\text{safety}}(\pi_{\theta}) \right] $$

Where:

$\pi_{\theta}$: Online policy network
$\pi_{\text{expert}}$: Offline expert policy
$L_{\text{distill}}$: Policy distillation loss
$L_{\text{safety}}$: Safety constraint loss
$\lambda$: Regularization coefficient

Safe Exploration Constraint

The safety constraint is defined as:

$$ P(\text{safe trajectory}) \geq 1 - \epsilon $$

Technical Approach

Key Components

Two-stage policy extraction
Offline expert policy distillation
Online fine-tuning with safety constraints

Performance Metrics

Reduced collision rate from 30% to <3%
15% improvement in driving task success rates

Research Significance

Addresses critical reinforcement learning challenges:

Mitigates conservative exploration
Enables lightweight policy networks
Improves safety in autonomous systems

Implementation Highlights

Developed versatile driving simulation platform
Implemented online distillation procedure
Extracted lightweight policies from transformer models

Developed at UC Berkeley’s MSC Lab under Prof. Masayoshi Tomizuka

Predicting Sagittal-Plane Swing Hip Kinematics in Response to Trips

Fri, 17 Jun 2022 00:00:00 +0000

Key Contributions

Developed three distinct predictive models for swing hip trip-recovery behavior
Demonstrated empirical superiority of proposed planning approach
Received Best Paper Award at BioRob for IEEE Robotics and Automation Letters