Hello everyone! So, let me cut right to it: Our announcement of Soba / the change in direction from Udon 2 didn’t go well.
There were a lot of things we could’ve done better, and that starts with explaining why we changed direction in the first place.
Udon 2 Development Troubles
From the community’s perspective, Udon 2 was a massive upgrade to Udon that would deliver more features and significant performance improvements. However, behind-the-scenes development was more difficult than initially anticipated.
We knew about many of these issues relatively early on, but they seemed solvable. As time went on, more and more issues began cropping up with development. Some of these issues are related to the initial performance promises made of Udon 2 in the first place.
Internal benchmarking showed that, while in certain situations the initial performance improvements of Udon 2 held up, these situations were extremely specific and unlikely to mimic “real life” usage.
While math-heavy code would see an improvement (which was one of the reasons why we showed off a procedurally generated maze as having a performance increase), more standard code would not see the same huge benefits.
Diving Into Udon 2 Performance Struggles
To go a little deeper on this: Code built to be optimized in Udon uses techniques that were directly counterproductive in Udon 2. In Udon, as it stands now, if you want to write performant code, you are incentivized to outsource work to externs (functions that execute outside of Udon in the native C# environment).
In Udon 2, however, relying on externs too heavily results in significant penalties due to marshalling – the process of copying data in and out of Udon 2. Those penalties were severe enough that in some situations they would negate the execution speed bonuses of Udon 2, causing the overall performance to be worse.
We communicated this in a positive way early on: we thought that there was still reason to push forward because you would get a good amount of benefits immediately, and if you change the way you write your code you can get even more benefits. That turned out to be not entirely true, and the reality of the situation was that in the worst cases, you would need to rewrite your code just to get any benefit at all.
As development continued, it became more apparent this was a bad idea. If you are writing a “sequel” to another language used by most of your advanced creators, and you tell them they have to rewrite their code in a different style in order to get any significant benefits… that’s not exactly “creator friendly.”
Spotting Major Issues for Creators
Additionally, if you didn’t reupload your world with the Udon 2 SDK, the Udon 2 client would run in a “compatibility mode” that ran “original” Udon inside Udon 2. This was almost always slower than “original” Udon would’ve ran by itself – in internal benchmarks, this compatibility mode took anywhere from two to ten times longer to run existing code.
This created a situation where we had to decide to either include both Udon and Udon 2 in the client (which resulted in even bigger memory regressions than the ones Udon 2 already had) or to ship with this compatibility mode which would punish anyone using “old” Udon code – which would then force them to fix a bunch of compiler errors, else they wouldn’t be able to upload their world.
This was a problem that had been worked on for months without any significant progress being made, and served as yet another major roadblock from Udon 2 ever making it to creators.
The Sunk Cost Fallacy and Cutting Losses
Of course, there was a big desire here to bring Udon 2 across the finish line, and to some extent, we fell into the sunk-cost fallacy. As commenters had pointed out, two years of work had gone into this project (which initially was only supposed to take 3 - 6 months to complete), and there was still a feeling from the engineering side that these were all solvable problems.
Udon 2 was also a complex project, built with complex code. This led to significant problems with benchmarking and diagnosing issues, with engineers on the team struggling to get a stable “real world” project running using Udon 2.
That is to say nothing of fears about how negative the end user experience could have been, too. Forcing people to rewrite their code – not just to see performance gains, but to get their worlds to compile at all – became a serious concern.
Udon 2 Continues to Hit Roadblocks
Late into development, as more of Udon 2 made it to our internal QA team, these issues (and more – some of which were shared in the previous thread, such as a severe memory regression on all platforms) became more apparent.
Other problems also cropped up with differences in Udon 2’s performance between the VRChat client and the editor. While this happens with original Udon, it is fairly consistent and predictable – with Udon 2, these performance gaps were massive and inconsistent. This meant that you might tweak your code, expect it to be faster due to its performance in the editor, then run it in VRChat and see a serious regression.
Taking a Hard Look at Udon 2
Eventually, all of these issues above led to a very real internal question: Is Udon 2 something that can reasonably be released? Can we achieve the initial promises made when Udon 2 was announced?
Going all-in on Udon 2 also carried a few additional risks, which led to even more questions: If we ship Udon 2 as a complete replacement for Udon, how much content were we willing to break? How long would it take to get old VRChat content “working”?
These are questions that were made more difficult by the fact that the Udon 2 development branch was severely behind VRChat’s main development branch. It would have taken weeks – if not months – to get Udon 2 running in the latest version of VRChat. The Udon 2 internal branch had gotten so complex, that fundamentally it wasn’t workable by most of the team, creating a huge bottleneck.
This led to a large engineering review of the project. What was discovered was that continuing development on Udon 2 would be irresponsible. Shipping it, maintaining it, improving on it, or delivering more “user-friendly” tools on top of all of that would all be far out of reach.
The question then became “okay, Udon 2 is unshippable. Can we still achieve the initial goals set by the project?”
Why We Moved Forward With Soba
A proposal was then made internally for Soba. Soba’s point is simple: It fulfills most of those original promises and does so in a way that is far easier for the team to maintain and iterate on quickly. It also solves some of the engineering problems we discussed earlier. For example, Soba doesn’t require marshalling like Udon 2 does.
In short, while Soba hasn’t reached parity with Udon 2 yet, it has progressed at a much faster pace. Likewise, it is being built from the ground up with a focus on being maintainable across multiple contributors and scalable instead of trying to sew together a series of hacks to make it work.
We’ve also seen some concern about Soba failing to address how Udon (or Udon 2) is difficult to use for newer creators. While we want to do that, and we’re mindful of it, we’re trying not to fall back into the old VRChat mindset of promising the world and delivering none of it. We’ll complete this task, then ingest feedback, carefully plan our next steps, and iterate.
Lessons Learned, Moving Forward
At the same time, the above history is why we’re so leery about sharing benchmarks. It’s also, in a large way, an artifact of an older VRChat that operated differently. We’re a lot more focused now, which has led to us being able to finish long-standing projects (like Persistence) and tackle major asks from the community (like Age Verification) without spinning our wheels as we have in the past.
Without going further off the rails in this post, I know that there are other projects the creator community really cares about that haven’t made it past the finish line yet. We know. Further, we know that you want more transparency about them. A big obvious question and fear that we’re potentially facing with this post is: how do we know this won’t happen to those things, too?
The honest answer? I can’t promise they won’t. However, if they get cut, delayed, or changed, we think the community at least deserves an answer as to why right out of the gate, as detailed as this one.
During the development of Udon 2, many lessons were learned, many of them hard lessons. While this post isn’t going to make the loss of Udon 2 any easier – we know we hyped it up far too much, and we know that it was promising things that creators really wanted – but we hope it makes our decisions more understandable.
. I would highly recommend not renaming cannies to completely different names for completely different features.