NPC - speech to text and lip-syncing animation?



We have an NPC with a 7 minute long audio speech to make. I have several questions and was hoping I could get some help here. We are just delving it so I don’t know much about this yet.

  1. I made an NPC and Avatar in Fuse. Is there a way in High Fidelity to get the NPC’s mouth to move from an audio file input of a Voice over? Please tell me this doesn’t have to be manually keyframed in a 3d app! And if it does, how would we sync the mouth animation to the Voice Over?

  2. Is there a way to convert that Voice Over to On Screen Text so our viewer can read or follow along even if their speakers are low or off?

  3. Can we create 3d text in High Fidelity without the background plate? When we did this it created a solid behind the text. Can HF 3d text be made without the background plate? If so we would animate slides of 3d text from the voice over, and swap text with different VO tracks. Or does the Voice Over have to be all in one long file??

  4. If we can’t get voice to text, or 3d text without the background, we considered using an animated video. However, if its streamed from Youtube the issue is that our visitors wont all see the video at the same time, unless there is a work around for this?

  5. If we don’t stream from Youtube, can we use something like a quicktime movie Mp4 in High Fidelity? If so how do we create that asset. Does it have to be nested in a FBX or something or how can we get a video as a server side asset?

Thank you for the help. It’s great having a community you can ask for help and advice from, thanks.

Issue with eyelashes from Adobe Fuse


Is there a workflow with Adobe Fuse, Mixamo, and using Iclone for Lipsyncing?

I have to animate the mouth on my NPC to a soundtrack. Whats the easiest way to do this?