Featured Research

Although autonomous navigation in simple, static environments has been well studied, it remains challenging for robots to navigate in highly dynamic, interactive scenarios (e.g., intersections, narrow corridors) where humans are involved. Robots must learn a safe and efficient behavior policy that can model the interactions, take into account the uncertainties among the interactions during decision making, coordinate with surrounding static and dynamic entities, and generalize to out-of-distribution (OOD) situations.

To address these challenges, we have 1) introduced a novel interaction-aware decision making framework for autonomous vehicles based on deep reinforcement learning (DRL), which integrates human internal state inference, domain knowledge, trajectory prediction, and counterfactual reasoning in a principled manner [3, 7]; 2) developed a novel guided meta RL paradigm [6] to improve the generalizability of learned policies and an importance sampling based training mechansim [8] for unbiased policy learning; 3) investigated DRL methods that leverage the inferred pairwise and group-wise relations [10] for social robot navigation around human crowds; 4) proposed the first DRL framework that integrates the prediction uncertainty of pedestrians obtained from adaptative conformal inference and explicitly guides the policy learning process in a principled manner for social navigation [11]; and 5) extended social navigation to human following in crowds via constrained RL, which decomposes the task into a sparse task reward and independent, interpretable cost constraints (following distance, human safety, obstacle safety) for explicit control of the proximity-safety tradeoff, and further distills distance-constrained policies into a preference-conditioned meta-policy that enables test-time customizable following distances on real robots [12, 13]. These approaches achieve superior performance in the corresponding tasks and provide explainable, human-understandable intermediate representations to build trust with humans.

Related Publications:
13. Closed-Loop Customizable Human Following via Distilling Monotonic Constrained Policies, under review.
12. Navigating the Proximity-Safety Balance: Constraint Decomposition for Human Following in Pedestrian Crowds, IEEE/ RSJ International Conference on Intelligent Robots and Systems (IROS 2026).
11. Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling, Conference on Robot Learning (CoRL 2025).
10. Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation, arXiv preprint arXiv:2401.12275.
9. Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025).
8. Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environments, IEEE Robotics and Automation Letters (RA-L), 2025.
7. Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation, IEEE Transactions on Robotics (T-RO), 2024.
6. Robust Driving Policy Learning with Guided Meta Reinforcement Learning, ITSC 2023.
5. Game Theory-Based Simultaneous Prediction and Planning for Autonomous Vehicle Navigation in Crowded Environments, ITSC 2023.
4. Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlooks, ITSC 2021.
3. Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships, ICRA 2021.
2. Orientation-Aware Planning for Parallel Task Execution of Omni-Directional Mobile Robot, IROS 2021.
1. Safe and Feasible Motion Generation for Autonomous Driving via Constrained Policy Net, IECON 2017.

We investigate how multi-modal foundation models can provide the perception, reasoning, and learning capabilities required for general-purpose embodied intelligence. At the perception and reasoning level, VLM-3R [10] equips vision-language models with 3D reconstructive instruction tuning, enabling spatial and temporal reasoning directly from monocular observations. Beyond improving model capabilities, we also study their reliability in physical environments. HEAL [3] systematically investigates hallucinations in LLM-driven embodied agents, revealing how failures to ground language instructions in the observed environment can propagate into erroneous long-horizon behaviors. Another focus of our research is bridging high-level task understanding and low-level visuomotor execution. RDD [5] addresses planner-executor alignment by decomposing long-horizon demonstrations into sub-tasks that are aligned with the capabilities of pretrained visuomotor policies. Rather than relying on manually specified task boundaries, RDD uses retrieval-based alignment to construct executable hierarchical supervision, improving both policy learning efficiency and long-horizon execution.

We further investigate how foundation models can enable scalable, data-efficient, and human-aligned visuomotor policy learning. AutoFocus-IL [6] uses VLM-derived task-relevant visual saliency to improve imitation learning without additional human annotations, while ROVED [7] combines vision-language representations with uncertainty-aware oracle querying to substantially reduce costly human preference feedback. GUIDES [8] distills semantic guidance from foundation models into pretrained robot policies, providing a lightweight mechanism for enhancing existing visuomotor executors. Beyond robotic manipulation, Drive My Way [11] studies preference alignment in Vision-Language-Action models for autonomous driving, enabling policies to adapt to both persistent human behavioral preferences and real-time natural-language instructions.

Related Publications:
11. Drive My Way : Preference Alignment of Vision-Language-Action Model for Personalized Driving, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026).
10. VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026).
9. CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction, IEEE International Conference on Robotics and Automation (ICRA 2026).
8. GUIDES: Guidance Using Instructor-Distilled Embeddings for Pre-trained Robot Policy Enhancement, IEEE International Conference on Robotics and Automation (ICRA 2026).
7. Reducing Oracle Feedback with Vision-Language Embeddings for Preference-Based RL, IEEE International Conference on Robotics and Automation (ICRA 2026).
6. AutoFocus-IL: VLM-based Saliency Maps for Data-Efficient Visual Imitation Learning without Extra Human Annotation, IEEE International Conference on Robotics and Automation (ICRA 2026).
5. RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks, 39th Conference on Neural Information Processing Systems (NeurIPS 2025).
4. LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Planner, International Conference on Robotics and Automation (ICRA 2025).
3. HEAL: An Empirical Study on Hallucinations in Embodied Agents Driven by Large Language Models, Conference on Empirical Methods in Natural Language Processing (EMNLP 2025 Findings).
2. TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution, AAAI Conference on Artificial Intelligence (AAAI 2026).
1. Conformal Prediction and MLLM aided Uncertainty Quantification in Scene Graph Generation, International Conference on Computer Vision and Pattern Recognition (CVPR 2025).

Individual autonomous agents often have limited situational awareness due to sensor occlusions, restricted sensing ranges, and partial observability. Multi-agent cooperation can overcome these through complementary information sharing, but practical deployment requires efficient collaboration across heterogeneous models, tasks, and computational resources.

We develop scalable, real-time frameworks for heterogeneous feature sharing and fusion [2, 3], and investigate such cooperation for joint perception and motion prediction under realistic communication delays [4]. Beyond sharing observations, multi-agent systems must determine what information to acquire and how to coordinate their actions. We develop cooperative sensing policies that actively position robots to improve monitoring accuracy in dynamic indoor environments [7], together with coverage planners that prioritize critical regions while maintaining efficient full-area coverage [5]. Effective coordination must also account for human preferences and uncertainty in dynamic interactions. We investigate preference-based policy fine-tuning to align swarm behaviors with human feedback [8], and uncertainty-aware constrained multi-agent reinforcement learning for safe and robust multi-robot social navigation [10].

We have also investigated language-enabled planning and communication for heterogeneous multi-robot teams. Our research unifies language-model reasoning with heuristic search to decompose long-horizon tasks, allocate subtasks according to heterogeneous robot capabilities, and generate coordinated execution plans [1]. To improve scalability in complex, object-rich environments, we further develop structured methods that identify task-relevant actions and objects before planning, reducing unnecessary reasoning and improving plan reliability [9]. We also study decentralized collaboration, where robots exchange task-relevant observations and reasoning through natural-language messages, with uncertainty calibration used to reduce irrelevant or misleading information [6]. Together, these efforts enable scalable, reliable, and efficient multi-robot coordination across planning and communication.

Related Publications:
10. CoopUQ: Uncertainty-Aware Multi-Robot Social Navigation with Cooperative Occupancy Prediction, under review.
9. Scale-Plan: Scalable Language-Enabled Task Planning for Heterogeneous Multi-Robot Teams, IEEE/ RSJ International Conference on Intelligent Robots and Systems (IROS 2026).
8. Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025).
7. Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learning, under review.
6. CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction, IEEE International Conference on Robotics and Automation (ICRA 2026).
5. Priority-Aware Multi-Robot Coverage Path Planning, IEEE Robotics and Automation Letters (RA-L), 2026.
4. CMP: Cooperative Motion Prediction with Multi-Agent Communication, IEEE Robotics and Automation Letters (RA-L), 2025.
3. STAMP: Scalable Task And Model-agnostic Collaborative Perception, International Conference on Learning Representations (ICLR 2025).
2. CoMamba: Real-time Cooperative Perception Unlocked with State Space Models, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025).
1. LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Planner, International Conference on Robotics and Automation (ICRA 2025).

As autonomous systems increasingly interact with humans, they must learn behaviors that accommodate diverse and evolving human preferences. To achieve this goal, we develop scalable approaches to preference learning by leveraging vision-language models to generate preference labels, selectively querying humans for uncertain comparisons, and transferring learned preference representations across robotic tasks, thereby improving the scalability and feedback efficiency of preference-based reinforcement learning [2]. For personalized autonomous driving, we condition vision-language-action policies on long-term individual driving habits and real-time language instructions, enabling adaptation of behaviors such as acceleration, braking, yielding, and overtaking [3]. To systematically evaluate preference learning in robotic manipulation, we introduce LIBERO-PeRM [4], a controlled benchmark that decouples task goals from user preferences and evaluates preference learning, composition, and generalization while separately measuring task success and preference satisfaction. Extending preference alignment to multi-agent systems, we incorporate agent- and team-level preference feedback to align swarm behaviors with human preferences over collision avoidance, region constraints, and task allocation [1]. Together, these efforts enable autonomous systems to efficiently learn human preferences and translate them into adaptive, personalized, and human-aligned behaviors.

Related Publications:
4. LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation, under review.
3. Drive My Way : Preference Alignment of Vision-Language-Action Model for Personalized Driving, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026).
2. Reducing Oracle Feedback with Vision-Language Embeddings for Preference-Based RL, IEEE International Conference on Robotics and Automation (ICRA 2026).
1. Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025).

Embodied agents, including autonomous vehicles and robot manipulators, must not only recognize objects but also reason about where entities are, how scenes evolve over time, and how actions alter the physical world. While modern perception models are effective at image-level recognition, their decisions can remain weakly grounded in metric geometry, temporal dynamics, uncertainty, and cross-view consistency, limiting reliable planning and interaction.

Our research develops spatial intelligence as a bridge between perception, world modeling, language, and action. For autonomous driving, we build geometry-aware neural representations and generative models that reconstruct street scenes, synthesize realistic camera and LiDAR observations, model dynamic 3D/4D environments, and refine uncertain scene geometry. We further ground vision-language reasoning in metric and spatiotemporal structure, enabling semantic concepts to remain aligned with objects, viewpoints, motion, and feasible trajectories. Building on these representations, we study world-grounded vision-language-action models that connect multi-view perception and spatial memory directly to end-to-end driving decisions. For robot manipulation, our ongoing work incorporates explicit spatial reasoning into reinforcement-learning fine-tuning, allowing policies to reason about object-to-object and object-to-end-effector relationships, relative directions and distances, and feasible interaction geometry while learning to act. Across both domains, our goal is to learn spatially consistent representations that support more stable policy optimization, more reliable decision making, and stronger generalization across scenes, viewpoints, tasks, and embodiments.

Related Publications:
5. VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026).
4. Uncertainty-Aware Diffusion-Guided Refinement of 3D Scenes, IEEE/CVF International Conference on Computer Vision (ICCV 2025).
3. S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2025.
2. NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields, AAAI Conference on Artificial Intelligence (AAAI 2024).
1. S-NeRF: Neural Radiance Fields for Street Views, International Conference on Learning Representations (ICLR 2023).

We investigate foundation models and vision language models (VLMs) for robotics and autonomous systems to enhance their reasoning capability and reliability. For example, inferring the short-term and long-term intentions of traffic participants and understanding the contextual semantics of scenes are the keys to scene understanding and situational awareness of autonomous vehicles. Moreover, how to enable autonomous agents (e.g., self-driving cars) to explain their reasoning, prediction, and decision making processes to human users (e.g., drivers, passengers) in a human understandable form (e.g., natural language) to build humans’ trust remains largely underexplored. Therefore, we created the first multimodal dataset for a new risk object ranking and natural language explanation task in urban scenarios and a rich dataset for intention prediction in autonomous driving, establishing benchmarks for corresponding tasks. Meanwhile, our research introduced novel methods that achieve superior performance on these problems.

Related Publications:
4. Rank2Tell: A Multimodal Dataset for Joint Driving Importance Ranking and Reasoning, WACV 2024.
3. DRAMA: Joint Risk Localization and Captioning in Driving, WACV 2023.
2. Important Object Identification with Semi-Supervised Learning for Autonomous Driving, ICRA 2022.
1. LOKI: Long Term and Key Intentions for Trajectory Prediction, ICCV 2021.

We investigate dynamic relational reasoning and interaction modeling under the context of the trajectory/motion prediction task, which aims to generate accurate, diverse future trajectory hypotheses or state sequences based on historical observations. Our research introduced the first unified relational reasoning toolbox that systematically infers the underlying relations/interactions between entities at different scales (e.g., pairwise, group-wise) and different abstraction levels (e.g., multiplex) by learning dynamic latent interaction graphs and hypergraphs from observable states (e.g., positions) in an unsupervised manner. The learned latent graphs are explainable and generalizable, significantly improving the performance of downstream tasks, including more accurate and generalizable prediction as well as safer and more efficient sequential decision making and control for mobile robots. We also proposed a physics-guided relational learning approach for physical dynamics modeling, which accurately simulates and infers future evolution of physical systems.

Related Publications:
11. Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation, submitted to IEEE Transactions on Robotics (T-RO), under review.
10. Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation, IEEE Transactions on Robotics (T-RO), 2024.
9. Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Crowd Trajectory Forecasting, ICRA 2022.
8. Important Object Identification with Semi-Supervised Learning for Autonomous Driving, ICRA 2022.
7. Learning Physical Dynamics with Subequivariant Graph Neural Networks, NeurIPS 2022.
6. Interaction Modeling with Multiplex Attention, NeurIPS 2022.
5. Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking, IEEE Transactions on Intelligent Transportation Systems, 2022.
4. RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting, ICCV 2021.
3. Continual Multi-agent Interaction Behavior Prediction with Conditional Generative Memory, IEEE Robotics and Automation Letters, 2021.
2. Spectral Temporal Graph Neural Network for Trajectory Prediction, ICRA 2021.
1. EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning, NeurIPS 2020.

Trajectory and occupancy prediction is a critical research area in the field of autonomous driving. As autonomous driving technology advances rapidly, accurately predicting the trajectories and occupancy of dynamic objects such as vehicles and pedestrians has become essential for enhancing the safety and reliability of autonomous systems. Effective trajectory and occupancy prediction enables autonomous vehicles to anticipate potential hazards in their environment, thereby improving decision making processes and reducing the risk of accidents. This directly contributes to the development of more robust and safe autonomous driving technologies. In our research, we have 1) developed effective solutions to model the diverse and uncertain behavior of various traffic participants (e.g., vehicles, pedestrians, cyclists) and infer their future trajectories and occupancy of the scene in highly complex and interactive traffic scenarios; 2) investigated how to effectively detect and handle out-of-distribution (OOD) situations by improving the generalizability of prediction frameworks, which achieves state-of-the-art performance in cross-dateset OOD evaluations; 3) introduced the first-of-its-kind cooperative motion prediction framework that advances the capabilities of connected and automated vehicles (CAVs) in cooperative tracking and motion prediction, addressing the crucial need for safe and robust decision making in dynamic environments.

Related Publications:
24. Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives, Foundations and Trends in Robotics.
23. UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving, IEEE International Conference on Computer Vision (ICCV 2025).
22. CMP: Cooperative Motion Prediction with Multi-Agent Communication, IEEE Robotics and Automation Letters (RA-L), 2025.
21. Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting, IEEE Robotics and Automation Letters (RA-L), 2025.
20. TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution, under review.
19. Self-Supervised Multi-Future Occupancy Forecasting for Autonomous Driving, Robotics: Science and Systems (RSS 2025).
18. Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments, ICRA 2024.
17. Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving, IV 2024.
16. Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints, ICRA 2023.
15. Game Theory-Based Simultaneous Prediction and Planning for Autonomous Vehicle Navigation in Crowded Environments, ITSC 2023.
14. A Cognition-Inspired Trajectory Prediction Method for Vehicles in Interactive Scenarios, IET Intelligent Transport Systems, 2023.
13. Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments, IROS 2022.
12. Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking, IEEE Transactions on Intelligent Transportation Systems, 2022.
11. RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting, ICCV 2021.
10. Shared Cross-Modal Trajectory Prediction for Autonomous Driving, CVPR 2021 (Oral).
9. LOKI: Long Term and Key Intentions for Trajectory Prediction, ICCV 2021.
8. Continual Multi-agent Interaction Behavior Prediction with Conditional Generative Memory, IEEE Robotics and Automation Letters (RA-L), 2021.
7. Multi-agent Driving Behavior Prediction across Different Scenarios with Self-supervised Domain Knowledge, ITSC 2021.
6. EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning, NeurIPS 2020.
5. Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving, IEEE Transactions on Intelligent Transportation Systems, 2020.
4. Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learning, ICRA 2019.
3. Conditional Generative Neural System for Probabilistic Trajectory Prediction, IROS 2019.
2. Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling, IV 2019.
1. Wasserstein Generative Learning with Kinematic Constraints for Probabilistic Interactive Driving Behavior Prediction, IV 2019.

Human intention and motion prediction is a vital research area that focuses on improving the safety and efficiency of interactions between humans and robots. As robots are increasingly integrated into environments shared with humans, such as homes, workplaces, and healthcare settings, it becomes crucial to predict human intentions and movements accurately. Understanding human intentions allows robots to anticipate and respond to human actions in a way that is both intuitive and safe, thereby enhancing the quality of human-robot interactions. This contributes to the development of more intelligent and adaptive robotic systems that can seamlessly collaborate with humans in various real-world scenarios. In our research, we have 1) developed multi-modal prediction methods for predicting human intentions and generating future motions (e.g., trajectories, human skeletons), which leverage fine-grained semantic and human appearance information. 2) proposed a systematic framework to identify generalizable dynamic relations (pairwise, group-wise) among human crowds. 3) introduced effective deep generative models to generate diverse, realistic human motions for human behavior simulation, which enhances the performance of downstream tasks.

Related Publications:
14. Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives, under review.
13. UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving, IEEE International Conference on Computer Vision (ICCV 2025).
12. TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution, under review.
11. Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation, submitted to IEEE Transactions on Robotics (T-RO), under review.
10. MATRIX: Multi-Agent Trajectory Generation with Diverse Contexts, ICRA 2024.
9. Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints, ICRA 2023.
8. Multi-Objective Diverse Human Motion Prediction with Knowledge Distillation, CVPR 2022 (Oral).
7. Interaction Modeling with Multiplex Attention, NeurIPS 2022.
6. Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Crowd Trajectory Forecasting, ICRA 2022.
5. RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting, ICCV 2021.
4. Spectral Temporal Graph Neural Network for Trajectory Prediction, ICRA 2021.
3. LOKI: Long Term and Key Intentions for Trajectory Prediction, ICCV 2021.
2. EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning, NeurIPS 2020.
1. Conditional Generative Neural System for Probabilistic Trajectory Prediction, IROS 2019.

How to generalize the prediction to different scenarios is largely underexplored. In contrast to recent works that use the Cartesian coordinate system and global context images directly as input, we propose to leverage human prior knowledge including the comprehension of pairwise relations between agents and pairwise context information extracted by self-supervised learning approaches to attain an effective Frenet-based representation. We demonstrate that our approach achieves superior performance in terms of overall performance, zero-shot, and few-shot transferability across different traffic scenarios with diverse layouts.

Related Publications:
1. Multi-Agent Driving Behavior Prediction across Different Scenarios with Self-Supervised Domain Knowledge, ITSC 2021.

The current mainstream research focuses on how to achieve accurate prediction on one large dataset. However, whether the multi-agent trajectory prediction model can be trained with a sequence of datasets, i.e., continual learning settings, remains a question. Can the current prediction methods avoid catastrophic forgetting? Can we utilize the continual learning strategy in the multi-agent trajectory prediction application? Motivated by the generative replay methods in continual learning literature, we propose a multi-agent interaction behavior prediction framework with a graph neural network-based conditional generative memory system to mitigate catastrophic forgetting. To the best of our knowledge, this work is the first attempt to study the continual learning problem in multi-agent interaction behavior prediction problems. We empirically show that several approaches in literature indeed suffer from catastrophic forgetting, and our approach succeeds in maintaining a low prediction error when datasets come sequentially.

Related Publications:
1. Continual Multi-Agent Interaction Behavior Prediction With Conditional Generative Memory, IEEE Robotics and Automation Letters, 2021.

We proposed a constrained mixture sequential Monte Carlo method that mitigates mode collapse in sequential Monte Carlo methods for tracking multiple targets and significantly improves tracking accuracy. Since prediction is a step in state estimation, we also proposed that the prior update in the state estimation framework can be implemented with any learning-based interaction-aware prediction model. The results in complex traffic scenarios show that using the prediction model outperforms purely physical models by a large margin due to the capability of relational reasoning. In particular, our method performs significantly better when handling missing or noisy sensor measurements.

Related Publications:
4. Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking, IEEE Transactions on Intelligent Transportation Systems, 2022.
3. Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving, IEEE Transactions on Intelligent Transportation Systems, 2021.
2. Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learning, ICRA 2019.
1. Generic Vehicle Tracking Framework Capable of Handling Occlusions Based on Modified Mixture Particle Filter, IV 2018.