Hello! Welcome to the latest VRChat Developer Update. Here, we’ll be talking about what we’re working on and what our progress looks like so far on a selection of our tasks.
You can read Friday’s update here. We had Monday off (thanks, Canada!) but we’re back on it today.
2 August 2022
Live Beta
There’s currently a Live Beta (an Open Beta that works with Live so people can use it alongside everyone else) that has a ton of features in it. Check out the post from last week to see what’s in it.
Over the weekend we continued to gather feedback, and importantly we also gathered bugs and issues with the Beta. There’s a few, which is to be expected with the speed that we integrated both brand-new features and features we’d had on hold.
We’re working on fixing those bugs, adding in some more improvements, and also continuing work on other tasks and projects.
A new Live Beta build is expected some time this week, or potentially fixing some bugs and dropping it into Live (and then moving new work into another Live Beta). We’re playing it by ear.
Movable Main Menu
A bug was found in the Live Beta where the movement handle appeared unexpectedly. This got fixed!
We found another bug where moving the Main Menu would also invisibly rotate and move the loading scene away from you. This was fixed as well.
We changed the way that this feature gets turned on the first time to help explain how it works for users. The first time a new user encounters the feature, the grab box will always be on and at 100% opacity. Hovering it will cause text to appear beneath the box reading “Grab to reposition the menu”.
After a user interacts with the grab to rotate successfully for the first time, the grab button will fade out of view. Hovering over it with the active pointer will fade the grab button back in at 50% opacity.
We also updated the QM settings toggle to read:
Responsive Menu is disabled, click to always spawn the Main Menu in your eyeline
when hovered-over and disabled, and to read:
Responsive Menu is enabled, click to keep the Main Menu in its default position
when hovered-over and enabled.
This task is currently in QA.
Horizon Adjust
Not much to say here! We put it out in the Live Beta and people liked it. There were some design concerns with people going upside down and being annoying with it, but it seemed like most of it calmed down after the novelty wore off.
There weren’t any major bugs with the implementation but there was some discussion about the UI for enabling the feature, as the warning is confusingly also a button.
Personal Mirror
Out of the things we put out on the Open Beta, this one got the most feedback! There’s a bunch of changes we put into the internal version today.
- All mirror settings save across play sessions now
- Portable mirror no longer reflects into other mirrors
- Tracking mode has been converted to a toggle (Pin to World)
- Tracking mode now sits at the top level of the menu
- Setting submenus no longer kick you out
- Fixed tracking ball reflection layers
- Renamed “transparency” to “opacity”
Additionally, a “Face Mirror” prototype was added, which is a mirror that displays your own face, attached at your HUD layer! This makes it really easy to monitor how hand gestures or Action menu settings adjust your face in real time.
This task is in iteration and is awaiting design review.
Hide Avatars by Distance / Max Number of Avatars by Distance
Last time we mentioned this one, this feature hid avatars by distance from you, so you can set “only show avatars within 5 meters of me.” Everyone outside that radius turns into a placeholder avatar.
Since then, we’ve added in the ability to limit the number of rendered avatars by distance! This means you can set “only show the closest 5 avatars”. Everyone else past that will be turned into the placeholder avatar.
There’s also been the addition of two new options: “Always Show Friends Avatars” and “Allow Override with ‘Show Avatar’”.
The first setting “Always Show Friends Avatars”, when enabled, will mean that your friends will never be affected by the distance or max number of avatars slider.
The second setting “Allow Override with ‘Show Avatar’”, when enabled, will mean that anyone you’ve clicked “Show Avatar” on will be immune to the effects of the distance or count hiding.
Avatar don’t load/unload when they’re shown or hidden by this system, the swap-over is instant.
https://gfycat.com/naughtysnappyhawk
This task is getting some more work. In particular we’re examining the possibility of showing the render range with an effect of some kind while you’re moving the slider. We also need to profile and benchmark to ensure we’re not costing more performance by implementing this system, and are doing a bit of UI polish.
Text to Text, Text to Speech, and Speech to Text
There was a sync between the engineers, designers, and producers on this task.
Internally, we’ve actually been talking about a few separate systems, so we had to separate them out into their own tasks to avoid confusion.
Text to Text
This is the “speech bubble” we talked about last time. There’s no cost associated with this, as we just build it into the application.
The effort on this is mostly done for the first iteration and a prototype will be available soon.
Text to Speech
This is generating speech based on input text. This could be relatively cheap if we used something like Amazon Polly or Microsoft Azure voice synthesis.
There’s not been effort on this yet beyond exploration.
Speech to Text
This is converting spoken text to words. This could be relatively expensive, if done in the cloud. Local solutions aren’t consistent and vary depending on OS.
For the moment, efforts will go towards an MVP (minimum viable product) TTT (Text to Text) implementation, which eventually will have TTS added on top. Once TTS is implemented and we gather feedback, we’ll look into STT. Since TTT will exist, STT may not be as useful – especially since you could input into a TTT entry field with OS-level voice input if required.
Camera Near-Clip Adjustment
This got some further iteration today.
Some concerns were voiced about having this feature on Quest. The Quest has a “lower” depth buffer (I’m pretty sure they meant “lower precision” here) which means that bad effects from adjusting this would be amplified a lot.
In implementing this, we discovered that the reference camera sometimes has issues with properly setting to the main camera, so we’re looking into fixing that alongside or before this is implemented.
It was also determined that a near-clip of lower than 1cm can be set via script directly. This could cause major issues with worlds, but users have asked for near-clip as low as 1mm (0.001m/0.1cm). This is especially useful for very small avatars. We’re looking into ways to inform users that turning down the near-clip that low will cause problems, but allow users to do so anyways.
There’s some concern that because having extremely low near-clip breaks worlds, some world authors will be hit with a bunch of invalid reports of issues. This mostly applies to worlds with large far-clip ranges.
This task is in continued development.
Noise Gating and Background Noise Removal
This is two tasks in one! Basically, it lets you adjust your mic sensitivity and lets you remove background noise as an option.
We’re implementing a cross-platform version of rnnoise, so this will work on everything from the start.
This task is in review status.
Portal Placement
This feature will allow you to place a portal with your controller instead of guessing at a position with where you’re standing.
This task is in development, has some art assets, and has a prototype!
Auto-Disable Cloning
There was a bit of confusion with this feature in the Live Beta, and we got some feedback.
Instead of being a higher-level override, we’d like this option to simply disable cloning in the normal VRChat Main Menu Settings page whenever this option is enabled and a world change occurs. Being able to be cloned in any moment will be determined by the preexisting setting.
With “Auto-Disable Cloning” on, entering a new instance with Avatar Cloning off keeps it off. Entering a new instance with Cloning on turns it off.
This was iterated on today, went through review, and through QA. It’ll make it into the next release or Live Beta build, whichever comes first.
AMD FSR
Warning: Technical stuff follows!
There was some feedback last time we posted about AMD FSR-- primarily “Why not 2.0?” and “What about DLSS/DLAA or XeSS?”
We don’t expect that we will get good results with FSR 2.0, DLSS, or Intel’s XeSS because they rely on generating accurate motion vectors. They are all temporal upscaling algorithms that combine pixels from multiple frames to produce a higher resolution image – but they require motion vectors in order to account for pixel motion between frames.
We have access to motion vectors in BIRP, but due to custom shaders, there is no guarantee they’re accurate enough to use for any of the aforementioned solutions… and without correct motion vectors, we would get various visual artifacts similar to SteamVR’s Motion Smoothing, which are bad enough that many turn it off due to said artifacting.
This is especially a problem for VRChat, because we allow custom shaders, and we have previously never required motion vectors. As such, many shaders won’t support them leading to incorrect behavior. Even beyond that, these temporal processes can come with technical limitations, with modern games often choosing to avoid certain effects because they interact poorly with temporal anti-aliasing techniques.
This would still be a problem even in a SRP such as URP (which only just had motion vector support added in the 2021 LTS version of Unity) because users can trivially make shaders that need special handling for motion vectors, not realize it, and end up causing artifacts. On top of that, motion vectors don’t handle transparency very well, so DLSS would need to be integrated into the middle of the pipeline so that it can be applied to only opaque meshes, which may not be possible in BIRP.
We hope this provided some further insight. We’re still continuing work to implement AMD FSR 1.0, as it works without motion vectors. See our previous Dev Update for further information.
Assigning CPU Affinity on Ryzen Processors
We’re currently investigating the performance benefits of adding a launch flag to VRChat that would set CCX affinity for AMD Ryzen processors.
This could potentially lead to a performance uplift. It’s possible it might lead to instability, though, so we want to make sure that’s not the case before pushing it live.
A Note About FPS
Following the launch of EAC, some users have been reporting noticeable FPS drops in VRChat. While we’ve been trying our best to hunt down these issues, we haven’t been able to replicate them on our end.
This is something we’ve kept a really close eye on. We know how important performance is to our community. Yet, after collecting about a week of data, we haven’t seen the average framerates of our users dropping – in fact, average framerate across all platforms is relatively stable. We looked very closely at our data to determine this, as the reports from users about lowered performance are plentiful, but we don’t see any downward trend.
Of course, this doesn’t mean that some users aren’t experiencing these issues. It’s totally possible! We would encourage users to first go through these troubleshooting steps and see if their issue improves. If it doesn’t, you should contact our Support Team for further troubleshooting.
It’s entirely possible that there could be a performance issue, but it might be localized to a certain piece of hardware, driver, or something else we can’t puzzle out without a bit more data.
To summarize, we’re not seeing anything to suggest EAC is degrading performance significantly. If you are having issues, please make a bug report!
Graduation Day
Alongside all this work, it turns out we had a bug in the Community Labs system that was causing worlds not to graduate properly.
We fixed the bug! And applied the fix retroactively. So, you might notice a bunch (like… 500?) worlds in the New row that graduated out of Community Labs. Go explore!
Conclusion
As always, these aren’t all the things we’ve got queued up. This is just what moved today, and we probably missed some stuff. There’s a lot going on!
We’ll get back to you tomorrow with more.