Performance concern with Udon heap

I think theres two problem with udon heap.

It seems that every variable in udon is allocated in Udon heap,and reference of the variable is implemented by uint address.

This means theres almost no way to implement GC in Udon,isn’t it?

Every variable in udon heap are boxed.
this will cause so many .NET GC running.

I thought a solution about this.

Use .NET reference directly instead of uint id and Udon heap.

Made a class to keep variables just in time using Dynamic Assembly.

I think we can also JIT compiles entire Udon Assembly to .NET IL(if theres no uint addresses and Jump Indirect).
This may improve Udon performance significantly.

3 Likes

The UdonHeap is designed to avoid boxing as much as possible. The UdonHeap avoids boxing by storing values inside StrongBox. The inputs and outputs for each EXTERN function are static allowing those methods to access the UdonHeap via generic methods to retrieve the specific types stored inside the UdonHeap without type casting or boxing. There are some code paths that do not know the type statically and need to cast to object but we’ve been making changes to avoid this when they cause issues. Arrays for example had some issues that should be significantly improved in the next Open Beta update.

VRChat is switching from Mono to IL2CPP with the 2018.4 engine upgrade. This has a number of upsides but unfortunately it comes with the requirement that all code is AOT (Ahead of Time) compiled. This means we cannot dynamically generate any .NET Assembly at runtime. This is definitely complicated matters but much of Udon’s development has been working around this limitation.

The UdonHeap is designed to avoid boxing as much as possible. The UdonHeap avoids boxing by storing values inside StrongBox. The inputs and outputs for each EXTERN function are static allowing those methods to access the UdonHeap via generic methods to retrieve the specific types stored inside the UdonHeap without type casting or boxing. There are some code paths that do not know the type statically and need to cast to object but we’ve been making changes to avoid this when they cause issues. Arrays for example had some issues that should be significantly improved in the next Open Beta update.

In IL,GetHeapVariable boxes variable,
but there was no allocation in editor maybe because of JIT optimization.
IL2CPP generated code never passes Mono’s JIT optimization pass,so I don’t know whether boxing is avoided in real VRC.

I also found boxing on editor when copying DateTime variables.
When copying variables. some types are type specific optimized.(like var strongBox1 = strongBox as StrongBox…)
but any other value types are boxed.

VRChat is switching from Mono to IL2CPP with the 2018.4 engine upgrade. This has a number of upsides but unfortunately it comes with the requirement that all code is AOT (Ahead of Time) compiled. This means we cannot dynamically generate any .NET Assembly at runtime. This is definitely complicated matters but much of Udon’s development has been working around this limitation.

I think IL2CPP’s performance upsides will easily pass away when Udon Program getting available in release.

All EXTERN functions use the GetHeapVariable and SetHeapVariable generic methods as the types they expect are known ahead of time (AOT). IL2CPP’s AOT compiler handles generics similarly to Mono’s JIT compiler except it uses static analysis to build a list of all possible types T can be and generates separate C++ code to handle each possible type. To avoid bloating the executable size it shares one version for all reference types, and only generates separate versions for value types as their size varies.

Unity has a couple of blog posts explaining how IL2CPP avoids boxing and how it shares generic implementations between types. https://blogs.unity3d.com/2016/08/11/il2cpp-optimizations-avoid-boxing/ https://blogs.unity3d.com/2015/06/16/il2cpp-internals-generic-sharing-implementation/

Copying heap variables is one of the less optimized code paths at the moment as it isn’t able to take advantage of the generic versions of Get/SetHeapVariable. I’ve recently added some handling for copying the main built-in C# value types (float, int, uint, etc.) without boxing that will be included in the next update to the VRChat Open Beta/Udon Open Alpha that should help with the most common cases but I plan on additional optimizations to cover the other value types like DateTime, and Unity Vectors.

I wonder more why this extern delegates are found using big switch instead of just hash map, its later cached so I guess its fine, but it still sounds like a lot of not needed branches that are hard to optimize.

And how every variable goes thro that blacklist on both read and write, and for list - all values must be passed to it. I assume blacklist is for elements of scene that should not be touched? Can’t this be ensured at call level? As such black listed objects can’t just magically appear at random call, but probably you can just add small checks for that few functions that could fetch it?
And then would be great to also get some stack memory for local variables that could be optimized even more.

I really hope that you and rest of vrchat team will later see also huge potential in one of community created compilers (for normal text scripts) and will help us by providing some additional features/bytecodes :wink: Like support for objects at bytecode level, so later we could easily exchange data between scripts in different language and use all the existing methods without any issues that would happen if we would try to simulate structures at bytecode level ourselfs.

Replacing the switch with a Dictionary is a planned change. Allegedly the C# compiler was supposed to compile a switch on a string variable into a hash table but after reviewing the IL2CPP generated C++ it clearly isn’t doing that.

2 Likes