Hierarchical transformers for long-term motion in-betweening with trajectory conditioning

Abstract

Motion in-betweening is a technique that automatically synthesizes transition frames between some context frames and a target frame, significantly reducing the time required for keyframing. However, existing methods have only been applicable to short transitions where the target pose needs to be similar in both time and space, thereby limiting their use in longer sequences. In this paper, we present a hierarchical architecture consisting of two Transformers that can effectively synthesize long-term motion in-betweening. To address the optimization challenges of generating all frames at once, we divide the problem into two subproblems: predicting keyframes first and refining the remaining frames. Furthermore, we leverage the root trajectory as a conditional input to enhance our approach. By incorporating the root trajectory, our method not only enhances predictability of the generated results for users but also enables them to achieve their desired outcome through editability.