HEVC/H265 video playback

General Discussion
1

Has anyone tried it? I suspect that Windows will require a codec in a few or many situations, but I’m also wondering about Quest 2 support.

I see that both AVPro and virtual desktop mention H265 but maybe I’ll have to put together a test video to just try it.

2 
--------------------------------------------------
Codec  | Unity | AVPro | Quest Unity | Quest AVPro
H264   |  yes  |  yes  |         yes | yes
H265   |  yes* |  yes* |         yes | yes³
 VP8   |  yes  |  yes  |         yes | yes³
 VP9²  |    NO |  yes  |         yes | yes³
 AV1   |    NO |  yes¹ |     skipped | skipped
.MP3   |    NO |  yes  |     skipped | yes
.flac  |    NO |  NO   |     skipped | skipped
.ogg   |    NO |  yes  |     skipped | yes
AAC.m4a|    NO |  yes  |     skipped | yes
Notes:
WMP = Windows Media Player
Unity = UnityEngine.Video.VideoPlayer, 
  Requests to webm files are from UnityPlayer User-Agent
  Otherwise MP4 playback is handed off to WMFSDK (Windows Media Foundation)
  On Quest all the requests come from ExoPlayer specifically user-agent like
  AVProMobileVideo/2023.1.2p4-1290--Release (Linux;Android 10) ExoPlayerLib/2.18.0
* H.265 requires HEVC Video Extensions on Windows.
¹ AV1 requires AV1 Video Extensions on Windows, which is free.
² Windows Store has a free "VP9 Video Extensions", I do not have this installed
³ I'm sure these work, so I haven't tested yet.

Having to go bother to get the Video Extensions seems odd to me, because many GPUs have HEVC encoding and decoding support.

This is just with testing static files. I haven’t tested h265, VP8, VP9 with AVPro players on Quest yet, but I assume they will be yes, because the Unity requests have AVPro User-Agent

April 7th revision: WMP can use LAV Filters, and “Movies & TV” doesn’t allow me to drop files into it. So I’m tossing the WMP column. Testing with the “Movies & TV” application seems to be a good way to see if the codec will work inside VRChat though. VRChat is where the decision to use “Media Foundation” or “DirectShow” with AVPro is made.

If you’re interesting in LAVFilters: winget install Nevcairiel.LAVFilters

1 Like
3

It all has to do with software licensing. It’s why there’s a difference between AVC and H264, despite them being the “same” codec. AV1 is technically supposed to be the alternative to H265, but they just made it better and it hasn’t received widespread adoption just yet. Technically any video player that has HEVC decoding support is supposed to be paying a fee to the foundation that made HEVC.

I did try to run the command and it downloaded, but it did not work in Movies and TV. Are there other steps involved?

4

Oh, the command to install LAVFilters? sorry about that. “Windows Media Player” will use the LAVFilters, and AVPro’s documentation mentions they can use LAV Filters via DirectShow, in practice VRChat is only using Windows Media Foundation.

The Movies & TV program was the only player I could easily find that would similarly not use LAV Filters.

Another interesting thing is that when I click on the “Get in Store pp” button at HEVC Video Extensions from Device Manufacturer, the Windows Store opens and informs me that “This app will not function on your device”, but it’s still installed and h265 still works in VRChat.

Although I do see reddit comments saying that that they tried start ms-windows-store://pdp/?ProductId=9n4wgh0z6vhq and get an error “The thing you are looking for isn’t here.”

I might be wrong but my point of view is that with a GTX1080 that supports H265 decoding in hardware, NVIDIA has paid to license it.

5

Yeah. I tried it, even with winget, and it doesn’t seem to work anymore. I guess Microsoft figured out people were doing it and just restricted downloads of it. Though it would be cool to know if an existing version can be copied and patched to another system.

I completely agree, but unfortunately that’s how the H265/HEVC licenses work. They are needlessly complicated. I guess it is “fair” because they made the codec from scratch and deserve some money for it. But, eh, whatever I guess.

Though it is confusing, as it specifically mentioned here that software-based decoding and encoding that is not bundled with hardware is free. This does makes sense since VLC, MPV, and other players can decode H265 and use hardware decoding.

Since VRChat seems to rely on software, I wonder about the feasibility of finding some other decoding solution (perhaps even MPV-shim?).

image
image991×206 12.7 KB

6

AVProVideo documentation shows that the Video API can be either Media Foundation or DirectShow

image

I did come across an interesting comment on reddit in a discussion about the HEVC For device manufacturers.

You need OEM flags on your system (OEM Key Activation) to install it from the store, or if you have a development / developer key flag on your system you can get it as well.

I checked my computer though, and its not OEM, and developer mode isn’t enabled. In looking into this I saw it seemed like the method to install via the store went between working/not working a bunch of times, settling down in April of this year

I think they are targeting Microsoft Windows there. I noticed that the storage page for HEVC states:

For devices that don’t have hardware support for HEVC videos, software support is provided,

I see this going one of a few ways:

  1. Microsoft should make HEVC extensions that only work with hardware support, and maybe limit installation via installation of GPU drivers.

  2. VRChat adds another video player to support a low latency video streaming protocol that is more NAT aware than RTSP :slight_smile: , and it relies on hardware support. throw the incoming data at the hardware black box.

  3. I don’t know that allowing end users to choose between Media Foundation and DirectShow is a good idea. Maybe?

1 Like