Image loading need improvement

General Discussion
1

The Image Loading restriction is too strict for a dynamically updated Gallery System.

It has been found that there has been a discussion about this in the feedback but it has been long shelved.

A relatively possible approach would be to generate thumbnails via Video Player, however after some experimentation it was found that the API behavior was extremely system and platform dependent.

There was no guarantee of consistent behavior, not only was the API complex and difficult to debug, but the code for the Unity related discussions was actually difficult to track down the cause of the problem, and ultimately it was found that synchronization could not be effectively controlled.

Some magical values and tricks had to be applied to make it work, but absolute reliability could not be guaranteed.

What’s worse is that Udon repackages it and lacks exposure to the original built-in functions, which makes it even harder to control.

Even if it works fine in C#, there is no guarantee that it will work effectively ported to Udon, and there are small details that the lack of API exposure will make the final operation more complicated.

Additionally, you need to convert a large number of images into a single Video each time through the software, and you have to circumvent the rate limit of five seconds for this to happen… Controlled decoding and copying of textures is accomplished in the process.

I’ve basically experimented with some of the programs I could find on Unity itself, and confirmed the uncontrollability of functions like frameReady to Graphics.CopyTexture.

CopyTexture, etc. Worse yet, Blit is even more uncontrollable than CopyTexture…

Since checking whether two textures match in pixel content is far beyond Udon’s ability, it can only be implemented that when a texture copy event occurs, the behavior of an empty loop must be performed continuously on subsequent frames until a certain level of copying is performed on the next frame.

In addition, can you specify the frame? Videoplayer related functions do not expose the frame, the controllability is extremely low…

I know there are simpler ways to do this, for example Videoplayer has an available decoder built into the system for H.264 format, up to 4096*4096 with hardware acceleration.

H.265/AV1 can have 8192*8192, but I’m sorry if the user can’t see it without installing it on the system from the Microsoft Store.

Splitting is done via a simple Shader, but there is an upper limit to the sharpness, or multiple images need to be weighed.
This means that there are reasonable combinations of 81922048 or 163841024, or a variety of other combinations, but this implies some kind of limitation.

You can solve it directly through UnWrap Mesh UV without writing a shader…

2

Can you do the thumbnail first as an image? Then replace with the proper images as they load?

As the images are raw texture format I believe they should be used sparingly. Like it’s 12mb for 2k RGB. How many of those are you going to load for a gallery?

3

Dozens and hopefully it’s clear enough

Five seconds of gradual replacement is too slow

4

Dozens? How many images need to be full quality at one time? Maybe use image loader for a further away lod and then use a video player for the closeup. I’ve seen a few worlds do like a slideshow function by controlling a video player with 1 second granularity. Also seen a video player with a few options like 1 5 10 seconds.

Make sure your video keyframes line up with the images.

Personally I think a website would be better to look at a changing set of images, rather than a VRChat world

5

The main expectation is that it is easy to use and maintain, avoiding additional costs that may be incurred later.

Embedded into the world is usually used as an advertisement wall and a simple and easy to expand gallery, managed through the backend and decoupled from VRchat, avoiding repeated uploads to update the world and scrubbing the player’s cache.

The larger the world, the more it needs

6

you can download up to 100MB on the text file format, so you probably can have Base64 encoded image in a text file, loading multiple images at once. You just need to decode them from the file.

7

Have you considered Udon’s performance…

How much does it cost you in terms of performance to decode webp these?

8

it’s gonna be a couple of calls per image, creating object, and copying it from the JSON, not the worse in terms of performances.

    byte[]  imageBytes = Convert.FromBase64String(str);
    Texture2D tex = new Texture2D(2, 2);
    tex.LoadImage( imageBytes );
9

I won’t deny it needs improvements, I’m just saying, there’s solution to your Gallery System with the existing features without relying on video players.

I remember when people were storing strings in Avatars profile picture (using some encoding), because the video stream was too unreliable to be used that way.

10

I may have to look into writing a backend and experiment with it…

However, the performance of similar APIs is not very good, and will be degraded again in Udon.

I hope Udon will give less restrictions and more flexible API…

11

Have you ever actually written in Udon?

LoadImage is not exposed in Udon.

The version is 3.7.4

12

My bad, I thought it was possible based on the C# equivalent, I’ll admit I didn’t test it myself. That function not being there is indeed a serious limitation.