Walk.js, new avis, physics and turn detection


#1

In response to a couple of suggestions, I’ve been working on resurrecting the walk.js script. There’s been quite a lot of changes since the script was last worked on; the physics stuff has been added, voxels removed and simple surface detection made possible. I’m pleased to report that I have the script working again, have the avi walking on a simple entity surface and have re-fitted the animations to work better with the Mixamo avis.

The way in which different movement states are worked out is under some fairly significant change, as the avi is now following a simple entity surface instead of a voxel surface and is also subject to different forces due to the physics. I understand that mesh surface detection is in the pipeline and am looking forward to that…

The new physics seems to be having an effect on things, particularly regarding the avi’s point of origin - when the avi is moved forward, just after the script first loads, there is all sorts of wobbling about! In addition, when the avi turns, he seems to be pivoting around a point a metre or so behind. Since physics is still being worked on, I’ve not addressed these issues yet, so the script is a bit rough around the edges, particularly when it comes to detecting movement states.

There also seems to be an issue regarding MyAvatar.getAngularVelocity and MyAvatar.getAngularAcceleration - neither seem to be functioning right now, so turns whilst flying and turning on the spot whilst standing are not detected. I intend to investigate this shortly and submit a bug report if necessary.

The plan is to get the working version, complete with the crossed legs issue fixed released asap. I have a couple of other projects to finish off before I can get stuck back into this properly and address the newly arisen issues described above, but I am back on it…

  • davedub

#2

Ok, there is a very rough beta version of walk.js with the Mixamo avi fixes applied available here:

http://s3.amazonaws.com/hifi-public/procedural-animator/beta/walk.js

Awaiting instructions…

  • davedub

#3

not bad keep it up almost perfect


#4

Update

I’ve been working on getting the beta version of the script (1.25) back up to scratch:

  • Functionality that was either not in the original version, or not specifically requested in subsequent worklist jobs has been removed.
  • Fixed existing locomotion state detection code.
  • Further optimised (drastically simplified) the existing animation storage and replaying code.
  • Refactored the code to be easier to read and closer to HiFi’s coding standards (still work to do there)

Next Up

The two main tasks I’m considering now are:

  1. Fixing up the beta version (1.25) of the script to a point were I can confidently submit a PR. (The current release version shipping with HiFi is 1.12, and is very old and clunky!)

  2. Once the JS api has been fixed (i.e. getAngularVelocity and getAnglularAcceleration are working again) complete the script’s full functionality:

  • walk (2 directions)
  • turn on the spot (2 directions)
  • sidestep (2 directions)
  • fly (6 directions, blended + tweaks for banking and rapid flight)

The majority of the full functionality is already in place, and some will be revealed when the angular velocity fix is put in place.

Extensibility

Once complete, the script will provide a complete character animation solution and a firm foundation for further procedural animation - it’s been designed to be highly extensible, with scenarios like swimming and quadruped motion being considered from the outset and becoming possible with minimal changes to the underlying code.

this very quick, rough quadruped animation
took two only minutes to make using the
walkTools procedural animation editor

Future

Other features that may be of interest for the future include:

  • ability to specify fbx files as idle animations, blending seamlessly when starting to (proceduraly) walk
  • ability to select different sets of animations (male and female character sets would be a good start!)

Get it now!

As usual, the current beta version (1.25) is easily accessible: paste this URL into the ‘from URL’ box on the Running Scripts dialog: http://s3.amazonaws.com/hifi-public/procedural-animator/beta/walk.js

The procedural animation editor, walkTools has also been updated. The editor can be used for the following:

  • investigating the bone structure of uploaded avatars
  • modifying existing animations
  • creating new animations

walkTools is available here: https://github.com/DaveDubUK/walkTools

  • davedub

#5

Ready for release?

I've updated the beta version of the walk script with a release candidate. I'd very much appreciate any feedback before I submit a PR:

http://s3.amazonaws.com/hifi-public/procedural-animator/beta/walk.js

The script has been de-glitched, heavily optimised and thoroughly cleaned up. Aside from the major internal refactoring and the footsteps sounds being fixed, it’s essentially just a smoother running version of the previous beta.

Feedback welcome!

  • davedub

#6

How did i miss this thread :smiley: awesome work so far. Will try this out in coming weeks.


#7

I have test your walk. I like the default walk much more.

While you don t go bound foreward,
your arm s move also a bit
insteed of to fit on the Body.


#8

@Sbin - thanks for the feedback - I think you’re right about the walk looking different. I’d initially thought it was just due to the new walk playing faster, but doing a comparison (see movie below) I can see some other differences have crept in. As a result, I’m going to re-convert the old walk to the new format again before I submit the PR. The walk should (and will!) look the same when I’m done, aside from some improvements like the feet orientation and shoulders being too far back.

If you’re not seeing arms movement, it’s most likely because you have your Hydras plugged in - the script detects Hydras and does not animate the arms, leaving them free for the Hydras to control. You can turn arms animation back on by clicking the ‘Arms Free’ button on the UI or unplugging your Hydras and reloading the script.

physics artifacts?

The bounding forward thing is a different issue - it actually happens on both versions of the script. It always happens just once, when Interface has just started and the script has just been loaded. It’s like he falls off his perch and is thrust forward a metre. After this has happened, the avi is permanently stuck a metre or so forward of his origin, as can be seen when he turns on the spot with either version of the script running.

So, if you start interface and load the old version you’ll see the exact same effect. It seems to have crept in when the physics was added and we lost some js calls (MyAvatar.getAngularVelocity, MyAvatar.getAngularAcceleration, MyAvatar.getAcceleration all currently do not work)
This is something I’m currently investigating (seems to happen after the application of hips roll rotations), but because the physics is still work in progress, I’ve not prioritised it for now.

@Menithal - watch this space!

I still have non-hifi projects I have to work on, but I plan to get some more done on this during the week…

  • davedub

#9

Also, with all the work on avatars recently, can anyone make a recommendation for the best avi to use as a reference? I’d thought about using @ozan 's reference female, but this is a male walk…

  • davedub

#10

Hi Dave,
I’ve just tested your walk script and so far I’m impressed. But I’ve come across a few problems. Once I’ve landed on the surface and walked for a bit I “trip” and start to fly, I found this when walking on the next object. I can’t land and start walking, it’s hit and miss sometimes it works then sometimes I go through the land and then I’m having to fly up and down using the E/C key to try and land properly and walk.


#11

I’ve finally found some time to update the walk animation so that it matches the original. I have also optimised the short stepping / walk-to-idle transition timing, so it’s all looking a bit smoother too. As before, I’m posting here in the hope that you guys spot any faults that would otherwise stop the devs from accepting my up and coming PR:

https://hifi-public.s3.amazonaws.com/procedural-animator/beta/walk.js

@hans3000 - thank you for the feedback, and apologies for the delay in replying. I have seen the issue you’re describing come up very occasionally, but I have only been testing on a single, large surface so far. I’ve been holding off addressing this stuff until the physics is more developed, but I feel it’s time to start tackling these issues in earnest now that I’ve completed the massive re-factoring.
I plan extend my test area to include multiple surfaces so I can investigate jumping between them. Early observations seem to indicate the avi is bouncing (i.e. being subjected to some disproportionately large upwards acceleration forces) just after landing, which will be my starting point.
Footnote: If you let your avi come to a complete stop, then start walking again, the walk animation will always play - you shouldn’t need to jump up and down.

next - physics investigations

I plan to investigate both the ‘bouncing avatar’ and the ‘falling of the perch’ issues (as described in ‘physics artifacts?’ above). Any input regarding possible causes of these issues would be gratefully received.

As usual, please feel free to comment / criticise / suggest!

  • davedub

#12

Tonight ill probably play around this. Gonna see if I can make the characters tip toe with the walk animation for the heck of it.

If it is possible, then this walk script could technically support any form of avatar. Combined with standards I promoted,this technically could be applied for nearly all avatars with four limbs (or even more if expanded upon)


#13

Hey @Menithal - sadly, in the latest updates for Interface (2545), the following MyAvatar calls have stopped returning vectors (note that everything works fine on build 2542 if you are prepared to revert):

getVelocity
getAcceleration
getAngularVelocity (broken for a while now)
getAngularAcceleration (also broken for a while)

Obviously, the script can’t function without these calls! This bug has broken the existing version of walk.js too. I reported it this morning: https://worklist.net/20554

If you do revert to 2542 and want any info, please let me know. As a starter, to make the walk animation tiptoe would be very easy (once you are hosting it yourself so you can edit the files) - either:

  1. open dd-male-walk-animation.js (in the assets folder) and manually adjust the pitch offset for the feet or
  2. set up walkTools, then use the animation editor, as described here:

walkTools can be downloaded from here: https://github.com/DaveDubUK/walkTools

I hope you find things useful, especially walkTools, which I believe could be of quite some help when troubleshooting avatar armature issues.

Regarding joint naming conventions, the script has been designed to be highly extensible from the outset - I am quite certain I can animate ANY given bone structure with just a few edits - ears, tails, capes, wiggly beards - whatever :smile:

Again, if you need any info, just let me know - this has been my pet project for quite a while now, I am extremely keen to see it come to fruition, especially when it’s so close! (IMHO, The only thing that really needs final tuning now is the locomotion state detection, and that’s kinda hard to nail down whilst the physics stuff is still in such a state of flux. All the really knotty stuff has been done, tested and polished)

Enjoy!

  • davedub

#14

Yeah you are correct that the new client version doesnt work at all

In anycase, played around with this abit. its quite fun to play around with.
Some Improvements and ideas:

  1. Maybe allow numerical values to be set for the slider values as well, or holding some modifier to snap to the 0, 90, 180, 270, 360? or by 45s (or settable value)
  2. Maybe a drop down menu for custom bones adjustments and automatic left and right mapping if they arent normally defined?
  3. What exactly is the compass thingymajig? it seems to teleport me around the domain .
  4. Maybe improved camera controls while in this view?

Some Side discoveries (that arent really related to your script, but the way how hifi interprets the model bones from blender):
Assuming The Blender Skeleton is just import/re-export from market: To dirty fix for blender model bone rotations

  • Apparently the Blender skeleton model is not symmetrical according to your script (so apparently probably somehow interpreted differently by Hifi), even though the same skeleton it works fine with maya. just noticed now that there is symmetrical and opposite modes. Symmetrical works fine :slight_smile: but wasnt clear at first

  • LeftUpLeft is -180 Yaw, while Right is 180 Yaw

  • LeftShoulder has to be offset -90 Pitch, and -90 Roll, while Right shoulder has to be offset -90 Pitch, and 90 Roll (from center)

  • this also would explain why EyeBones have to be rotated up by -90 degrees on the X axis. The exporting or importing is off for bones split between the Y axis, which is the default blender mirroring axis. need to investigate some more.

As an bug fix for hifi, ui element click through needs fixing for this to work best.


#15

Hey guys,
getVelocity is working in newest build! I just tested walk.js and it works great. Keep up the amazing work davedub!


#16

update

Whilst waiting for news on VHACD and the missing js calls for angular velocity, angular acceleration and linear acceleration, I’ve been cleaning up, optimising and generally refactoring both walkTools and walk.js.

Net result - the scripts are much smaller, the functionality is the same, but it’s all now far easier to read and understand.

not walking when on a surface?

I have noticed a minor bug in walkApi.js whereby if the script is loaded whilst NOT on a surface, the hips to feet measurement can be incorrect, and so can break the surface detection functionality.
There is also an issue when the script loads before the avatar model has loaded - again, the hips to feet measurement is at fault.

I will be addressing these issues next, but in the meantime, the script needs to be loaded whilst your avi is on a surface and after your avi has downloaded. i.e. simply reloading the walk script will also fix these issues should you encounter them.

Mixamo not looking so good?

The beta walk animation has been (temporarily) tweaked to work better with @ozan’s reference avatar, so there are some minor differences. In particular, the yaw (swing forwards / backwards) on the upper arms, and the pitch offset for the thighs.
This is experimental and needs discussion as to whether or not to fit the animations to Mixamo avis or the reference avi. I personally think using the reference avi is the better option, as most ‘tweaks’ involved zeroing out values that had been added to make the Mixamo avis look right - I personally don’t think the Mixamo avi / rig is ideal for use as reference.

@Menithal - the next step on walkTools is to add a webwindow (as used for the entity properties for edit.js). The plan is to add text inputs for the currently selected joint parameters as well as incorporating the ‘view’ menu and avi orientation selector.
The walkTools script was getting pretty messy, so I decided to have a thorough refactor before adding the long overdue webwindow panel.

walk.js beta is here: https://hifi-public.s3.amazonaws.com/procedural-animator/beta/walk.js

walkTools is here: https://github.com/DaveDubUK/walkTools/

  • davedub

#17

https://alphas-new.highfidelity.io/uploads/highfidelity/3665/568937b6f4ab4f0b.jpg

works great


#18

@Judas - Thanks for the feedback, and ouch - that is worrying! I’ve just re-tested using both the Mixamo and Ozan’s avis - they seem to work fine:

So this may be an issue with your avi - would you be able to PM me a link for your avatar so I can work out what’s going wrong?

If anyone else is having similar issues, please let me know - virtual worlds animation is no good if it doesn’t work for everyone!

Am tied up this week, but plan to find some more time for HiFi stuff at the weekend…

  • davedub

#19

Anything born from Blender will never work
Maybe we need to move exclusively to closed closed source. ozan is all about mixamo


#20

@Judas I already documented how the walk.js orientation need to be like for them to work with blender exported avatars. the WalkTool.js allows for this to be done and tested with. See

https://alphas.highfidelity.io/t/walk-js-new-avis-physics-and-turn-detection/5460/14 and

https://alphas.highfidelity.io/t/blender-compendium-and-woes-of-the-blender-fbx-export-at-the-mountains-of-documentation-madness/4739/48

@davedub the issue is with the blender based avatars. their armature orientations are misinterpreted by the hifi fbx reader as the rotation methodology is different from how Maya exports the models. HiFi only currently accepts the Maya method not the Max/Blender method. The sintel model by @ozan was simply imported to maya and re-exported