Integrating Google Dialogflow into VRChat virtual character

General Discussion
1

Is there a way to integrate Google Dialogflow into a VRChat character? We have a project going where we would like to create a character that can be interacted with via voice and typed text. We currently have several animated Ready Player Me avatars inworld as non-interactive NPCs, but don’t know how to add something like Dialogflow.

Any suggestions or guidance on how this might be done would be hugely appreciated.

Thank you!

Kaylee

2

Any dialog system you build would have to be able to run standalone, since Udon doesn’t support making arbitrary web requests. I’m not familiar with Dialogflow but from a cursory glance it appears to be a cloud service that requires an active network connection, which means it can’t work with VRChat as it currently exists…

3

Hi Fluffycritter!

Thanks so much for getting back to me.

I’ve been reading as much as I can about Dialogflow, webhooks, OSC and my head is spinning. It’s very difficult to get a clear picture of what is possible and what is not, so I appreciate the information that you’ve provided.

Based on what you’ve pointed out it seems that, for the moment at least, pursuing trying to integrate Google Dialogflow (which is a cloud-based application) into VRChat is a dead end.

Are there any chatbots that have been developed for VRChat that you know of?

Any further guidance you can offer would be again be hugely welcomed.

Thank you!

Kaylee

4

I haven’t seen any chatbots in VRC. It should theoretically be possible to implement one (that doesn’t preserve any state) using Udon/UdonSharp although interacting with it would require the players to type, since there’s no voice recognition in VRC either. You’d be better off doing something with a conversation tree with fixed dialog choices, which you could probably implement using Twine+Yarn or the like.

5

Hi again Fluffycritter!

Wow! I didn’t expect such a quick response! Thank you!

Is there somewhere I could read more about Twine + Yarn and how I might use Udon/UdonSharp to set up that kind of conversation tree? This is a steep learning curve for me (I’ve done a lot of coding in Second Life, but this is a different ball game), so any leads would be a huge help.

Kaylee

6

See https://yarnspinner.dev/ - note that the default implementation is a MonoBehaviour and to use it with UdonSharp you’ll need to port it to a UdonBehaviour. I haven’t done any of this myself though, and can’t help you any further.

7

Hi again!

Thank you for that lead Fluffycritter.

Thanks also for being willing to respond. I appreciate it a lot.