Start from what you remember. In memory-dependent manipulation, the evidence that decides which action to take (cue) disappears before the robot acts (now). (a) Feeding the memory to the policy as a condition still starts generation from uninformed noise. (b) RecallFlow starts generation from what it remembers: the episodic memory is the source, and the transport network sees only the present.
Abstract
Diffusion and flow-matching policies have become a standard choice for visuomotor imitation learning, owing to their ability to model multimodal action distributions. In memory-dependent manipulation, the robot must act on information that is no longer visible. Existing methods feed the observation history to the policy as an additional condition, while action generation still starts from Gaussian noise. However, the policy must then pick out the relevant target from a long and redundant history, leading to spurious correlations and unreliable decisions as the memory load grows. Additionally, the initial noise carries no information about what the robot has seen, leaving generation without guidance toward the remembered target. To address these issues, we propose RecallFlow, a flow-policy framework that builds the source distribution from episodic memory. Recall–Relocalization relates the memory to the current view through a few recall slots, so that the source reflects where the remembered targets are now. Recall Anchors then turns each slot into a Gaussian component of the source, enabling generation to start close to the intended action instead of from noise. The transport network then completes the action from the current observation alone, keeping inference nearly as cheap as a memoryless policy. We further build a suite of memory-dependent tasks grouped by memory load and compare all policies under a matched protocol. Extensive experiments show that RecallFlow achieves a state-of-the-art average success rate of 80.4% on the public MIKASA-Robo memory benchmark, and outperforms the strongest baseline by 20.3 points on our suite and by 18.8 points on four real-robot tasks.
Method Overview
Overview of RecallFlow. The present observation is encoded into tokens Zt. Recall–Relocalization lets Zt read the episodic memory bank and decodes K recall slots, each attending to the current view through a heatmap Hk. Recall Anchors turns each slot into a Gaussian anchor in action space; with a standard Gaussian, the anchors form the source p0. A transport network conditioned only on Zt carries x0 ∼ p0 to the action chunk, so memory reaches the action solely through the source.
Real-Robot Experiments
RecallFlow on the four real-robot tasks (third-person view, 2× speed). Videos play automatically without sound.
Remember Colorpick the block of the shown colorNo Repeattouch each cup onceMarked Plateplace the cube into the marked plateShell Gamelift the cup that hides the block
Real-robot experiments. (a) We deploy the policies on an xArm 7 with a wrist camera and a third-person camera. (b) We design four tasks that differ in what must be remembered. (c) RecallFlow achieves the highest success rate on all four tasks. (d) The three behaviors observed on No Repeat: repeating one cup, touching one cup twice, and touching each cup once.
Simulation Benchmarks
Evaluation tasks in simulation. (a) RecallSuite consists of five memory-dependent tasks: the cue is seen only earlier in the episode, and the candidates look identical when the robot must decide. (b), (c) Tasks from MIKASA-Robo and RoboMimic.
Method
SGT
IM
RC3
RC5
RC9
Avg.
Octo-small
46
39
45
17
11
31.6
OpenVLA‡
47
14
59
16
6
28.4
π0
33
42
35
22
15
29.4
SpatialVLA
23
27
27
17
11
21.0
CronusVLA
32
5
31
13
9
18.0
MemoryVLA
88
24
44
30
20
41.2
GMP
98
83
80
61
17
67.8
SimpleMemVLA∗
99
83
71
58
59
74.0
Chameleon
96.7
58.9
94.4
72.2
53.3
75.1
RecallFlow (ours)
100.0
87.3
100.0
78.3
36.3
80.4
Comparison with published methods on MIKASA-Robo. Success rate (%) on ShellGameTouch (SGT), InterceptMedium (IM), and RememberColor with 3, 5, and 9 colors (RC3–RC9). RecallFlow is averaged over three seeds and achieves the highest average (80.4 ± 1.1). Results of prior methods are taken from the literature. ∗Available only on arXiv. ‡OpenVLA-OFT, as reported in MemoryVLA.
Analysis
Memory decides where generation starts. (a) At a decision of a No Repeat episode, all thirty samples of RecallFlow head to the untouched container from the source on, whereas long-history DP starts from noise and commits only in its last generation steps. (b) Over 15 decisions, the source samples of RecallFlow are already at the target. (c) RecallFlow therefore keeps its success rate with a single generation step.
Which Representation Fits the Task? Adaptive Selection for Robot Learning from Human Videos
Xuji Zhang, Zichen Zhang, Wei Zhai, Jian Di, Kun Li, Yang Cao, Wenjun Lv
Project video (about 3 minutes), including real-robot rollouts on all five tasks.
Abstract
Human videos offer a scalable source of manipulation experience to complement limited robot demonstrations. Existing approaches typically commit to one intermediate representation, such as object or scene trajectories, hand keypoints, contact affordances, or latent actions, across tasks with different control requirements. We study how the level of abstraction in these representations can be matched to the information needed for robot control in each task. We introduce two diagnostics: embodiment residue, measuring the distinguishability of human and robot representations, and task information, measuring how much a representation helps predict robot actions. We compare six representation families under a common protocol to examine how these properties vary across tasks. We select the representation with the least embodiment residue among those that measurably improve action prediction, then use its predictor, trained on both data sources, to condition a policy trained only on robot demonstrations. We find that the trade-off between embodiment residue and task information varies across tasks, leading to different choices of abstraction level. Across five real-world manipulation tasks, our method achieves 64% average success, improving over the strongest external baseline by 11 percentage points and over the best fixed representation by 4 percentage points.
Method Overview
From diagnostics to a robot policy. We extract six candidate families from human (H) and robot (R) demonstrations and select f⋆ using an information gate followed by embodiment-residue ranking. The predictor gf⋆ learns from both domains and outputs the future representation; the policy πf⋆ uses it with current observations to generate the action chunk At, learning only from robot actions. The Pour Granules example selects 3D hand keypoints, and this family remains fixed during execution. Curves inside the diagnostic cards are schematic. Checkmarks and crosses denote whether the lower 95% information-advantage bound exceeds zero.
A Telepresence Mobile Robot Built from Scratch
Tsinghua University, Institute for AI Industry Research (AIR) · Research intern, 2023–2024 · Related paper: IMWUT 2025
A differential-drive mobile robot built full-stack from scratch, from mechanical design and embedded control to ROS-based navigation.
(a) System architecture, built bottom-up
Application
Remote telepresenceNavigation research for visually impaired people
Navigation
Mapping and localization with RGB-D and 3D LiDARPath planningPedestrian avoidance
Control
STM32 velocity closed loopMotors and driversHost–controller communication (serial, ROS 1/2)
Body
BuildModeled in SolidWorksFabricated by a local workshop in Beijing
SensorsRGB-D cameras (front and side)3D LiDARIMUSide force sensors
(b) Black version: 3D LiDAR
13D LiDAR
2Aluminum-alloy frame
3Electronics
4Drive wheels with independent suspension
Real-robot architecture. (a) Four layers, from the robot body up to applications. (b) Black version: 3D LiDAR, with the components that are visible in the photo labeled.
Corridor test (4× speed).
The robot navigates stably through narrow corridors with many turns and avoids pedestrians. It was used in research on remote telepresence and on navigation for visually impaired people.
Silver version: full robot (with mast)Black version: independent suspension and drive wheelsElectronics: main controller and motor drivers
Click a photo to view it at full size.
Co-authored Papers
ART: Bridging the Embodiment Gap in 3D Human-to-Robot Affordance Transfer.
Zichen Zhang, Wei Zhai, Xuji Zhang, Yang Cao, Yu Kang. IEEE Transactions on Artificial Intelligence, 2026 (accepted).