Developing an AI Vtuber: Conversation Testing Footage
3I wanted to see if it could hold up a conversation. The answer is kinda. It remembers what I say but doesn't always respond first to my most recent inquiry, responding to things I said before and fake chat messages (to simulate a real livestream chat while testing) instead. I think I can improve it. Maybe by having it stop listening to my microphone while it's mid-sentence.
It answers my first question in its second response, which is not ideal, and I'd like to reduce delay and have it give occasional shorter answers. If it gives a short answer it can do it in 2-4 seconds after asking, while longer answers are generally 5-10, but can take longer depending on other factors. Longer replies take a while to generate the text-to-speech. It can take even longer if it's in the middle of a previous message when it receives the new one, because it waits to finish talking before generating a new response and audio file. I could definitely make it do these tasks simultaneously in the future to improve speed, but I'm not sure that would actually improve its ability to respond to select the right input to respond to first. I don't expect it to be perfect though.
It is also theoretically ready to stream solo and can respond to viewers' live chats much more quickly (also theoretically) than audio input, calling out the names of chatters and responding to what they say, even responding to multiple chat messages at a time.
I can also eventually add live subtitles and make the AI itself more entertaining.
Lastly, yeah, any vroid vtubers watching will see that the ai's model has basically all stock assets. I wanted to focus on the code, what can I say? It will be easy to update the model later if I feel like it. I definitely want to replace the stock expression blendshapes, like I did with my model. The stock expressions are unbelievably bad imho. Thankfully most vtubers insist on live2d anyway and (maybe) won't notice and (maybe) won't read this far. (live2d is fantastic, but vroid is also quite capable in moderately-skilled hands)
My first video testing this AI: https://www.youtube.com/watch?v=GJvFVwODr9w
Join the Diescord: https://discord.gg/DQHtHjh9Vb
Vtuber Assets & Art by:
+Shimami / YOROZUYA - some of the clothing on my vrm model (right) (see their booth store)