The target is the state
Whatever the loss predicts is what later planning is allowed to know. A next-frame embedding leaves velocity outside that state. Pose and motion, predicted together, put it back in.
TI-JEPA · world models
A still frame
has no velocity.
A next-frame JEPA predicts the embedding of the next frame from the current embedding and the action. A renderer without motion blur draws configuration only. The embedding of one frame is a function of configuration, so no encoder, however large, can read instantaneous velocity out of it. TI-JEPA makes one small, checkable change: split the latent into a pose code and an explicit motion code, and predict both.
55%Lower closed-loop distance on Pendulum
p = 3.2×10⁻¹⁰
p = 3.2×10⁻¹⁰
64%Lower closed-loop distance on CartPole
p = 5.1×10⁻¹⁵
p = 5.1×10⁻¹⁵
0Memoryless branch separation
exactly zero in six settings, as the theory requires
exactly zero in six settings, as the theory requires
38×Branch separation versus a 3-frame history baseline
on real Reacher photographs
on real Reacher photographs
01 · One real case
The same pixels, two real futures.
This is one pair from the trained Pendulum models, the same construction as the paper’s kill experiment. At the first step the two renderings match. The memoryless model then emits one rollout for both. TI-JEPA follows the two directions the physics actually takes.
02 · Every environment
The same test, three physical systems.
Choose an environment. Each player is another real pair from that environment’s trained models: ground truth, the memoryless baseline, and TI-JEPA. The last two are real official benchmarks, retrained on real pixels, not built for this paper.
03 · What this changes
A world model can be asked to stop.
Readable, not hidden
The motion code is a bias-free finite difference. Identical frames give exactly zero. A probe can read pose and speed separately, and a planner can ask for arrival with no leftover speed.
The next world models
Control that catches, swings, or comes to rest needs the rate of change in the object being predicted. That is a change to the target, and it travels with any encoder large enough to see the scene.