Meta AI Research: Quest 2 Body Pose Estimation Without Trackers

Meta has introduced QuestSim, which uses a neural network to display the user’s body position using the sensors in the Quest 2 helmet and the position of the controllers. This is normally done using a set of additional sensors that transmit data about the position of body parts.

True, the system doesn’t display accurate hand movements and as it stands now, it can only display other users’ poses – that is, looking at yourself, you won’t see your avatar’s own movements displayed. In addition, the delay is about 160 milliseconds, which is quite a lot for a dynamic scene. But it’s still cool, because now you’re either encased in sensors or you look like a spherical horse in a vacuum.

Experts talk about Full body tracking with Oculus Quest 2

Meta CTO Andrew Bosworth certainly seemed to hint at this last week. When asked about leg tracking in an Instagram “ask me anything” Bosworth responded:

Yeah we’ve been made fun of a lot for the legless avatars, and I think that’s very fair and I think it’s pretty funny.

Jonathan Winters III:

If Meta keeps on in the VR sector, imagine consumer VR in 20 years. They’re doing most of the heavy lifting research and development-wise.

Sven Viking:

Pity about the latency, it seems like it’d be a big step up from current IK player bodies. Still good for remote player avatars, though in fast-paced games the latency might still be a problem.