ProTV by ArchiTechAnon | Usage Guides and Walkthroughs

MediaControls Plugin Deep Dive

[WIP]

2 Likes

Playlist Plugin Deep Dive

[WIP]

Working on better docs currently. For now here’s a couple easy videos to get started with the playlist.

Setting up the Playlist

Modifying the Playlist Template

ProTV - Playlist Setup and Options (2.3 Beta 2.0 or later)

2 Likes

Queue Plugin Briefing

[WIP]

2 Likes

Skybox Plugin Briefing

This plugin encompasses a minimal plugin script that swaps the scene’s skybox material on certain TV events and the shader used to render various video modes of the skybox.
It supports equi-rectangular (aka panoramic) 360deg video and 180deg video as well as certain cubemap style 360 video renders.
It also supports Side-By-Side (SBS) and Over-Under (OU) 3D modes for VR.

Plugin Options

  • Tv
    The ProTV instance that the plugin should register to.

  • Skybox
    This is the material that the plugin should assign to the skybox material when the TV is considered ACTIVE. The required shader to use is the Video/Skybox shader that this plugin provides, as this plugin directly updates properties on the shader.

  • Fallback
    This is the material that the plugin should assign to the skybox material when the TV is considered INACTIVE. This can be left empty and the plugin will automatically use the world’s existing material as the fallback.

  • Brightness
    This is a UI Slider that is used to determine the gamma exposure of the rendered video in the shader. Helps reduce eye strain in certain situations.

  • The rest of the options are just for visual feedback of what state is active and do not have functional bearing on the plugin itself.

Events

The plugin considers the TV to be ACTIVE for the following TV events: _TvPlay, _TvMediaStart
The plugin considers the TV to be INACTIVE for the following TV events: _TvMediaEnd, _TvStop, _TvVideoPlayerError

The following events handle updating the shader options for modifying the render output:

  • _Brightness
    Applies the value of the Brightness slider to the exposure value of the shader.

  • _Not3D
    Disables any 3D mode. Make the shader render the whole video frame to both eyes in VR.

  • _SideBySide
    Enables the SBS mode. This 3D layout mode is most commonly used for 180deg video.

  • _OverUnder
    Enables the OU mode. This 3D layout mode is most commonly used for 360deg video.

  • _Flip
    Makes the shader render the video inverted vertically. This setting generally shouldn’t be needed, but sometimes the video may need to be flipped on certain platforms or videos.

  • _SwapEyes
    Makes the shader render the video to the opposing eyes when in a 3D mode. When in desktop mode, this swaps the render between each eye, since desktop can only see one eye at a time.

  • _Deg180
    Sets the render area to 1/2 the skybox.

  • _Panoramic
    Sets the render area to the whole skybox.

  • _CubeMap
    Sets the render area to use the whole skybox as well as interpret the video as having a cubemap style layout instead of the equi-rectangular layout.

Per-URL Settings

In order to allow for these settings to be applied on a per-URL basis (different videos should be handled uniquely), there is a meta info feature that the TV will extract from the URL if it’s present.
This is done by appending a # to the URL and then specifying the settings you want applied, separating each by a ;. For example https://vimeo.com/123456789#Panoramic;Standard

2 Likes

How to integrate AudioLink

Setting up AudioLink with ProTV isn’t too difficult. Here’s demonstration video showing how to connect all the pieces together.

2 Likes

Tips, Tricks and Best Practices

So you want to make your own Plugin?

[WIP]

2 Likes

How to use UdonGraph with ProTV

While ProTV generally uses U# for it’s setup, it is also provides and easy way to implement custom logic within UdonGraph as well.

[WIP]

2 Likes

Misc Notes

Understanding URLs in VRChat

All video players, generally speaking, work on Quest. Just gonna get that answer out of the way.

What doesn’t always work on Quest is the urls themselves. There are a few points to make regarding this:

  1. The urls that people are most used to aren’t actually urls for media but for websites. Most commonly is Youtube, which is mostly in the form of https://youtube.com/watch?v=VIDEOID. This url is a website url, not a media (aka direct video) url. Video players do not know how to read a website url, so they would normally fail.

    BUT! VRChat has a trick up its sleeve to handle this. They use a command line tool called YouTubeDownLoad(YTDL). What this does is fetches the content of the webpage url, and fishes around that content until it finds a url that is a valid media url, then spits that back to VRChat so the program can tell the desired video player (either UnityVideoPlayer or AVPro) to play the direct url.

  2. Due to technical limitations beyond the VRChat dev’s reach, YTDL is unable to be used natively on the Quest platform. Only PC has access to this tool currently. This means that when a user tries to play a generic youtube video using the common shortform that you see in the browser URL bar… it’s not going to work.
    There are work arounds for this problem… but the free options are generally unreliable and the other options require some level of technical skill and money to accomplish.

  3. The first workaround option we have is using the YTDL tool externally to VRChat on a PC. What this does is gives you the direct media url which you can then enter into the video player which will then sync to Quest users and they’ll able to see it at that point.
    There are some drawbacks to this method:

    • First is that it requires an external computer to run the YTDL program in order to parse the website url into the longer-form media url.

    • Second is that for pretty much all large-scale media hosting platforms (such as youtube/video/dailymotion/etc) embeds a number of various parameters within the long-form urls, one of which is an expiration timestamps. So once the expiration timestamp has passed, the URL is no longer valid an will no longer load the video. This means that it is not realistic to put the URLs into the world at build time, as they would eventually not work. This also makes it difficult to simply share streams as well because most of those urls expire after an hour or two.

  4. The second workaround is to use a custom server external to VRChat that is exposed to the internet which runs the YTDL tool for the user and then returns the media url. This helps because each time the URL is fetched, it has a new timestamp in it, so it can work for extended periods of time. The difficult thing is that it does require server admin knowledge if you set it up on your own.

    • There is a free service that does this publicly, referred to as “Jinnai”. This service is quite popular, but is generally unreliable due to it’s overuse. Videos tend to fail 50% of the time. https://t-ne.x0.to/?url=

    • There is an open source option called Vroxy if you want to host your own. It even includes setup and update scripts in the repo for any Debian-based OS (like Ubuntu) to make setup a breeze.

    • One thing to note about using a server proxy like the above two, is if there is too much suspicious traffic from the server’s IP, youtube/twitch/vimeo/etc can just rate-limit or straight up block the server’s IP causing url resolution to fail.

  5. The final workaround is much more involved as it involves hosting the videos yourself from a custom server. You’d basically use YTDL to grab the video itself, then put it on a server you have and have people access the URL directly to your server instead of the mainstream hosting platforms. This is oviously an advanced method that the average world creator is not going to do.

The cheapest option currently is to use the Jinnai service, even with its instability. Though if you are not shy of doing a bit of server stuff, deploying an instance of Qroxy will generally make the urls more reliable.

All this doesn’t even go into the actual media content, like codecs and formats, which can also contribute to video failures. But that’s for another time.

TL;DR: Youtube on Quest isn’t for the lazy.

About YoutubeDL specifically…

In order for VRChat to determine what the direct URL is for a given video, it makes use of the tool called youtubeDL. This tool is what allows desktop to be able to “watch youtube videos” (or other hosting sites like vimeo/twitch/etc). Currently due to technical reasons, this tool is not available on the Quest version of VRChat, ergo why youtube doesn’t work on quest by default.

There are two ways around this limitation when dealing with quest.

  1. On desktop, manually resolve the url outside of vrchat. This involves downloading a copy of youtubeDL onto your local machine, opening CMD (or equivalent command line program), then running either of the following commands:

    • The simplest one-liner is youtubedl -g -f best https://youtube.com/watch?v=VIDEO_ID
      This will simply get (the -g option) the url that is noted as the most compatible (the -f best option).
    • If you want to see what formats are available instead of just the “best” one, you can do youtubedl --list-formats https://youtube.com/watch?v=VIDEO_ID
      This will list all options available, examine the media codec types and resolution in the output to find your desired option, then copy the format ID (for youtube it’s a plain number on the left of each option) and replace the “best” term with that ID.
      Like so: youtubedl -g -f 312 https://youtube.com/watch?v=VIDEO_ID
      (Also if you are using vimeo with this method, I highly recommend any of the formats that start with hls-akfire_interconnect_quic- as they are good quality and compatible with Quest as well)

    Once you’ve retrieved the long-form direct URL for the youtube video, copy it and paste it into the input field for the TV.

    And additional thing to note with this method is that for most major video hosting platforms (like youtube and twitch), the long-form url has an embedded expiration timestamp in the URL. This means that the URL will only work for up to that timestamp and then will start failing for people who either late join, or reload the video. For those who already connected and are playing the video, it’ll continue to play.

  2. If you have knowledge of self-hosting stuff and a bit of programming experience, you could easily make a resolver redirect that utilizes youtubeDL remotely to resolve the URL for whoever requests it.
    This requires a server to host it on (like a VPS, AWS, or whatever), a domain (optional) and a bit of knowledge with nginx and some server language.
    The result should be where the URL is of the server and within that URL (say for example, a query parameter) is the ACTUAL url you want resolved, the server target would resolve the actual url with youtubeDL and return a 301 redirect to the requesting client which would then play that new URL.
    This would work long-term as well because every time someone tries to (re)load the URL in the world, it would cause it to be resolved to the latest expiration timestamp, thus removing the issue mentioned in the first option.
    I have tested a very rudimentary version of this and it does work, with the exception of livestreams (like twitch) on Quest. For some reason that platform does not like being redirected to live media. I’m still looking into this particular issue.

Youtube caveats

  1. If you grabbed a bunch of youtube URLs to put into a ProTV playlist and some of them don’t seem to be working, be sure that you remove any &list=PLblahblahblahblahwhatever from the URL. If there is a youtube playlist ID as a query parameter on the url, youtubeDL will attempt to fetch ALL THE DATA from every single video within that playlist. If the youtube playlist is of excessive amount (like more than 10 items) it will cause excessive lag in loading the video, or may even just straight up timeout the request causing the video load to fail completely. TL;DR: ONLY HAVE YOUTUBE PLAYLISTS IN THE URLS IF YOU ABSOLUTELY NEED THEM TO BE.

Missing script references?

If you imported ProTV and it’s dependencies (latest SDK3 and UdonSharp 0.20.3 or later), try right clicking on the ArchiTechAnon folder and selecting “Reimport”. If that doesn’t work, try closing and reopening Unity, then doing a “Reimport” on the UdonSharp folder and on the ArchiTechAnon folder as well. This should correct any missing references that SHOULD be present. If it still isn’t working, contact me on discord to help chase down the actual cause.

Random tips to be aware of

  • When you call LoadURL/PlayURL on a video player instance, if the URL is invalid, the error callback will be called IMMEDIATELY before returning from the LoadURL/PlayURL method call.
  • It appears that complex encoding (such as URL encoded characters from eastern languages) has issues on Quest. If you are doing any custom hosting, for consistency and avoiding problems in the future, it is recommended to ONLY use ASCII based characters in urls. If you are going a basic video file serve, make sure the video file names are also only ASCII based.
3 Likes

Tried this plugin and is working pretty good so far. I saw the PlaylistQueueDrawer component in the package, but simply draging it into the world doesn’t seem to have the function of a queue. Is the queue function still a WIP?

edit: nvm I got it. There’s a readme in the plugin folder. Didn’t notice that lol.

2 Likes

I have Problems with my Audio… The 3D to 2D Audio Button does not work… I need help…

1 Like

This is super helpful stuff. MANY THANKS.
I have folowing problem: Cant get stream working in Quest.

Background: Try to self host streaming server (HLS) on local computer. Exposed on the internet via Dyndns*). Streaming server seems to work as I can a) Watch in VLC locally, b) watch on mobile phone cellular, c) PCVR version of VRChat with AVPRO (Unityplayer not working).

But on Quest I cant get the stream to work in either player.
Could it be that Quest required HTTPS and will reject stream coming from http. Or do you have any other hint?

For security reasons, quest requires that ALL web requests (including video player urls) are HTTPS. This requires a custom domain with and SSL cert attached. There are multiple ways to do SSL certs, but since you are self-hosting it sounds like, I’d look into utilizing the LetsEncrypt certbot tooling as a free SSL option.

I deployed a server for parsing. I want to know how to intercept the URL in the input box, process the string, and then play it because VRC Url objects currently cannot be constructed at runtime in Udon

You cannot “intercept” the url, due the fact that it would require dynamic VRCUrl objects.
There are a couple options for dealing with user input (not going through playlists or pre-baked URLs):

  1. Have the user manually type in the redirector server link and then paste the target url as a parameter for that server’s link. (eg: https://mydomain.tld/?url=https://youtu.be/BLAHBLAH)
  2. Have a separate standard text box which the user puts the desired link in that you can then modify dynamically to prepend the server’s link format. The user then needs to copy that resulting text and put it in the target VRCUrlInputField.

Here’s the world where protv was used, but he realized my purpose, but I didn’t know how he handled the text with the VRCUrlInputField 中文补番追番动漫屋V3․12[CN] - VRChat

What sort of streaming works best to quest users? I’ve heard of things like HLS, RTMP, and http streaming, the last one when you connect you just get the data (shoutcast?)

I tried topazchat out once in a world with a different player, not sure what protocol it uses, but apparently on quest it kept giving up. Hoping for something that can retry when needed, or just keep working…

Generally speaking, the recommendation is the MPEG-TS protocol for Quest.
You’ll typically see the URL end in something like .ts.

From what I could tell in the world (as I cannot read that language), nothing out of the ordinary is happening. It looks like a standard URL input field that feeds the video into the TV as usual.

what sort of protocol for those? i guess its either like rtsp://hostname/blah.ts or https://hostname/blah.ts

i guess by not having a quest some of this research is a bit blind, trying to find out how best to construct and hand out a mpeg transport stream.

AFAIK it’s just https.
That’s at least what VRCDN uses.
I’m not super familiar with the technicals of the actual protocols/codecs.