Face Tracking and Future speculation for Vive Facial Tracker


Recently I’ve picked up a Vive Eye Pro, and a Vive Face tracker. I didn’t even think to try and make anything work in VRChat because I assumed that it would be too advanced for the game unless there was a proprietary update for it.
(After some research I’m incorrect, but It requires breaking ToS (mod) for VRC and I’m not about losing my 5 year account streak.)

On thing that really sparked my attention was watching the April 2021 Developer Update stream, about 2 hours in, during the UDON portion of the stream, Momo talks about taking data from an IPhone camera to activate blendshapes (specifically for 3.0 Avatars) And I was very interested in weather or not that would also be translated to the Vive Facial tracker.

Momo goes further to explain that any device that outputs OSC Data could be used to control this, Which AFAIK, thats the main input of the Vive Facial tracker.

One issue that arises is the recent limit on Parameters. As far as I can tell, getting full face tracking (according to the non-TOS mod I found) has MANY parameters that I would imagine would quickly reach the memory cap.

All of which sounds very exciting and something I’m particularly ready for, but my main question and reason for speculation was that while the release of the face tracker and the time of the dev stream, I can see why they didn’t include it when talking about facial tracking, as the face tracker released about 2-3 weeks. And I’m more curious if Blendshape tracking is going to be a “Open Source” addon Vs one that would fully use the Vive Face tracker software.

What do you think? Am I just speculating too much? You have any concerns about the addition or are just as excited as I am?

Considering the code to implement it is open source and there’s specific branches for both unity and unreal.

Add to that that headset manufacturers like HTC, Valve, Oculus, etc, would likely closely work with VRC devs, I’d guess it’s something that’ll probably be in the pipeline to be supported natively.