Hello, and welcome to the Developer Update for the 26th of August, 2022!
You might notice that the “feature updates” section is a little thin. p3 was a doozy to get bugfixed, and we’ve diverted a good bit of team resources over to getting the UI ready. As such, there haven’t been many things worth posting about just yet since the Tuesday Dev Update!
Schedule Update
We’re going to continue the twice-weekly updates for the next week, but starting on the week of 4 September, we will be reverting to one update a week, probably on Thursday.
We don’t want to repeat information in order to “buff up” posts, which we’ve done (accidentally) a few times. Also, “we fixed bugs” isn’t too interesting to post about…
This change in pace also allows us to cover things that are less “active development” related, such as proposals, ideas, long-range plans, or even interviews with team members. We’re also considering good ways to highlight community content that stands out.
Assuming nothing changes, we’ll have future Dev Updates on these days:
- Tuesday, Aug 30
- Friday, Sept 2
- Thursday, Sept 8
- All following Thursdays
Release Schedule
2022.2.2p3 Release
Yesterday, we got a Live Beta update for p3 out that puts us at build 1228. This build brought Quest up to date, as it had a few issues in previous builds that prevented it from getting updates.
There’s a few more outstanding bugs we need to handle before launching to Live. Once those are done, we’ll push the next build. That updated build will potentially go straight to Live early next week.
Main Menu Live Beta
Soon after p3 goes to Live, we’re going to try to push the Main Menu to Live Beta. We’ve been doing a good bit of internal testing and dogfooding. There’s a few rough spots to polish out, but we’re hopeful we can get the Main Menu into your hands soon.
We want to launch the Live Beta at the start of a week so its easy for us to manage the initial massive influx of feedback without missing anything. As such, its possible if the Live Beta gets pushed too far next week, we’ll push it to the following week.
Feature Updates
Camera Improvements
We’ve continued on iterating on the camera! There’s a TON of features and requests we’ve gotten from both the community and internally on the camera. It’s a big focus point! Here’s what we’ve been actively working on recently.
We don’t have video just yet, but you’ll probably see some video previews in an upcoming Dev Update.
Camera Flying / Camera Drone for VR
In a previous update, we talked about “Camera Flying” or “Camera Drone” (“Drone Mode” is what people seem to keep calling it), which is a feature that allows you to fly your camera lens around in Desktop mode. This is super useful for getting big group shots or setting up cinematic photos.
We’ve been working on adapting this mode for VR, too! In our prototype, you can now control “drone mode” with the joysticks on your VR controllers.
We’re not completely settled on the controls, but we’re iterating on this feature.
Visual Shutter Effect
We’ve added a tiny bit (more?) skeuomorphism to the camera! There is now a shutter effect that will occur when a photo is taken, so you have a visual cue in addition to the audio.
This is not a camera flash! It is a visual shutter effect.
Camera Menu Changes and General Fixes
We’ve reorganized the camera’s menu a bit to make it a little more usable.
The “Take Photo” button has been moved to the beginning of the list, and the “Exit” button has been moved to the camera itself.
If you have your camera set to “World” mode and move to a different world or instance, the lens will now reattach itself to the camera.
AMD FSR and nVidia Variable Rate Shading
Warning: beyond here lie very nerdy dragons! Lots of technical talk. Skip to the “short version” prompts if you want to know the tl;dr version.
AMD FSR, DLSS, and other systems have been requested by our users many times to potentially increase the performance of VRChat while minimizing damage to visual fidelity. We’ve talked about this in previous updates, but we’re ready to give you another update.
Implementing AMD FSR Natively
After a lot of evaluation, we’ve determined that AMD’s FSR system likely cannot be cleanly implemented into VRChat. The proper place to implement this feature is during post-processing, after the tonemapper but before other effects like bloom. Applying FSR at this point also has the benefit of rendering the UI at full resolution, so that text remains clear.
However, there is not a clean way to implement it with Unity’s Built-In Render Pipeline (BIRP). We tried to integrate FSR into the Post Processing stack at the appropriate point, but the BIRP doesn’t support rendering at a lower resolution while still outputting to the screen directly.
Instead, we need to output to a different lower resolution render target, apply part of the Post Processing Stack, upscale with FSR, and then apply the rest of the Post Processing Stack at the full screen resolution. Unfortunately, Unity doesn’t properly handle the switching the render texture during the render process, so the Post Processing Stack ends up trying to operate directly on the screen-- which is blank when it runs. Changing the scene render target this way also breaks worlds that rely on Camera Stacking.
Short version: Implementing FSR as documented and recommended by AMD isn’t possible in VRChat right now.
What about the OpenVR FSR DLL?
There exist other methods like the OpenVR_FSR patch, but these methods aren’t ideal. In particular, the OpenVR DLL replacement employs a “Man-in-the-Middle” approach and modifies the communication between VRChat and SteamVR. This allows it to spoof the render resolution that SteamVR provides to get VRChat to render at a lower resolution, which it then up-scales with FSR and submits to SteamVR. This is not reliable or good practice.
Moreover, this approach also has issues because it runs too late in the pipeline. According to AMD’s FSR Integration Overview (PDF warning), FSR should run immediately following anti-aliasing and tonemapping, and before any effects that introduce noise like bloom, chromatic aberration, film grain, dithering (notably, Post Processing v2 always dithers), and before the HUD and other UI.
The method used by the OpenVR FSR DLL modification runs after post-processing has completed. Applying FSR after the noise-introducing effects in PPSv2 will cause FSR to accentuate that noise, resulting in worse image fidelity and artifacting.
Since many worlds in VRChat use Post Processing and we support it natively, implementing this method lowers the visual quality of VRChat and introduces undocumented visual behavior that we cannot account for.
On top of all of this, this implementation would not work on Rift PC or Oculus Link unless they went through SteamVR, and it doesn’t work at all in Desktop mode.
Short version: The methods employed by the OpenVR FSR DLL come with significant caveats that we are not comfortable with implementing into VRChat. They will cause issues with the rest of our rendering stack and the post processing stack, going against AMD’s recommended implementation methods. The implementation significantly lowers image quality and may cause significant visual issues, while offering only a minimal improvement to performance. Finally, it only works in certain configurations and on certain platforms.
nVidia Variable Rate Shading
nVidia’s Variable Rate Shading technology increases rendering performance and quality by varying the shading rate for different regions of the frame. Any nVidia RTX GPU model 20xx or later can use VRS.
We have a prototype implementation of VRS in VRChat, and it has a significant impact on GPU times.
Testing in an empty Club Orion (a very pretty but GPU-heavy world) with these settings:
Inner: [0.4, 0.33] X1_PER_PIXEL
Middle: [0.5, 0.5] X1_PER_2X1_PIXEL
Peripheral: [1.075, 1.2] X1_PER_2X2_PIXEL
View of the Orion stage with a lot of volumetric lights. Oof ouchie my overdraw.
Blue: X1_PER_PIXEL
Green: X1_PER_2X1_PIXELS
Yellow: X1_PER_2X2_PIXELS
The normal desktop view is reasonably close to what you see while wearing the HMD but significantly understates the amount of the screen that is covered by the peripheral region with these test settings. Here’s the full HMD render view:
Due to the lens distortion correction, the peripheral region is significantly squished and the centre is stretched. Without Fixed Foveated Rendering, the periphery ends up over-sampled relative to the center of the view. While wearing the Valve Index headset normally with the eye relief dialed in all the way, the blue and green regions cover about 80-90% of your field of view resulting in something closer to the desktop mirror than the full screen view.
So, results. This shows the reduction in GPU frame times before and after enabling VRS FFR:
- Before FFR: ~10ms
- After FFR: ~6.5ms
The region radii and resolutions can easily be adjusted to handle different HMD resolutions, screen aspect ratios, etc.
Currently these settings are adjusted via the config.json file, but we want to add the settings for this into our new UI instead if we implement this as a feature.
Short version: Using nVidia’s Variable Rate Shading technology to implement fixed foveated rendering has a significant benefit to lowering GPU load in VRChat with an acceptable impact on image quality. We are prototyping and evaluating this feature.
Community Highlights
A new section to show off some cool stuff we’ve seen the VRChat Community up to lately! This time around, it’s focused on two events in VRChat, but we’re also keeping an eye on other things to feature.
Summer VKet 2022
Summer VKet 2022 is ending today! Congratulations to HIKKY for another successful Summer VKet.
If you’re not familiar with VKet, it’s a series of virtual conventions run by HIKKY. You can browse through booths of hundreds of creators making avatars, worlds, shaders, music, video, art, anything! Both independent artists and larger organizations showcase their work at VKet. Virtual Market events are a mainstay of VRChat events, and we’ve been hosting them for years.
If you didn’t catch Summer VKet 2022, keep an eye out for their future events.
Slyfest: Azuria
Slyfest: Azuria – a virtual music festival held entirely in VRChat – started today!
Slyfest features over 50 DJs and producers who are active in the VRChat music community.
Slyfest isn’t just about them music, though. The folks behind Slyfest always create a series of beautiful stages that really highlight how awesome (and immersive!) musical performances can be in virtual reality. Even if you only hop in for a bit, we strongly suggest checking out what they’ve built!
Slyfest 2022 will be going on from the 26th to the 28th. You can check out their full schedule here!
You can find the Slyfest: Azuria world in its own world row this weekend.
Conclusion
That’s it for this week! Have a good weekend, and we’ll see you on Tuesday for the next Developer Update.
(shakes in horror recalling some of the 1+GB VRAM avatars they’ve seen recently)
