Can you test your download speeds to Amazon CloudFront? Your connection could be having issues with AWS.
(Note: The link below will immediately start a speed test to AWS)
Amazon Web Services Network Test | CloudHarmony
As for possible fixes.
You can try disabling AutoTuningLevel to see if that helps at all. If it does not help, you should re-enable it as this feature can be quite important.
The link below is for Microsoft Flight Simulator, some users have reported success in disabling it for VRChat however.
https://flightsimulator.zendesk.com/hc/en-us/articles/360015761760-Slow-download-speed
EDIT (2021-05-19 JST): Some of the other proposed solutions in the link above with Microsoft Flight Simulator have been reported to help some users. If AutoTuningLevel doesn’t help you can try the other ones as well.
Another option is to try a VPN and see if that helps any.
Restarting your router is another thing you can try.