Revisiting the MakeHuman - Blender- Hifi workflow


#1

Continuing the discussion from Walk.js, new avis, physics and turn detection:

Background

With the beta walkscript that is available in above thread, thanks to @davedub, there has been a hack fix Folding issue of the Blender avatars, so you can now have basic walking/flying animations applied to your avatar, regardless if you are using blender or maya exported avatars. Do note that this was a hack in the script a bit more tolerable for the time being, but it still has issues (its more of a band aid for a broken bone instead of a cast), until the issue is somehow resolved in the FBX reader or the way how transformations are interpreted…

Here are some notes, some of which will also be put later on in the Blender thread:

  • Yesterday, I made an observation with walk-script that when set bone rotations are set to 0, all avatar skeletons tend to go to an absolute T-pose (or folded T-Pose, as with Blender avatars ) and do not tend to use the rotations set earlier.
  • This means that you do not actually have to put the avatars into T pose, however it would help to test out any kinks in the bone systems.
  • It should also explain that if ones shoulders are in an angled down on the avatar when skinned, when they are, leveled to absolute 0,0,0, it will raise the shoulders up, as @Judas has been experiencing. The shoulders pivot effects the arms the most in terms of their position.

However, now that we have got the hack in to solve most of the major rotation issues with Blender Avatars with the walk.js (ONLY the walk.js, the actual issue still persists), @MetaverseCafes pointed out that there are there are rotation issues with the MakeHuman’s HumanIK default bones as well.

Revisting the Make Human Humanik rig, with the new information gathered yesterday and from earlier, I noticed that the directions are off to what the should be by the skeleton roll values which @ozan pointed out in the Sintel Thread.

So this is what generally the skeleton rolls should now look like in Blender


Notice the blue lines pointing down: these are the Z normals of the bones.

Compare to the Default Normals of the MakeHuman HumanIK rotations

The Workflow

The guidelines I stated a few months ago


Mostly said in the video still apply (Except the dropbox comment), but instead of directly exporting, there are some further steps that have to be done after rotating the eye bones.

Now, the major bones can be fixed quite easily:

  1. Select -all- Arm bones and hand bones
  2. Set their roll Z to Global -Z, (Control + N, Global -Z Axis),

    (Just noticef in the bus the image was in reverse order)
  3. Select -all- the Leg bones + Foot (no need for toe)
  4. Set their roll Z to Global -Y (Control + N, Global -Y Axis)
  5. Adjust the shoulder pivot point, so that if posed flat, the arms angled it wont make your arms go into your buttocks,

    (note, this still does result with your hands going into the buttocks, but adjusting this will get you the result you want. Needs a bit playing around but as of writing this post, I have to head to work)
  6. Do the rest of the stuff I discussed in the video
  7. Scale main skeleton to 2m aprx.
  8. Rotate main skeleton -90 on the X axis
  9. Apply rotations
  10. Rotate main skeleton to 90 X.
  11. Export to FBX,
  12. Package in HF,
  13. Upload.

This doesnt yet fix the fingers, but its something I am working on figuring out how to do it easy, as as of the moment I cant use Global -Z axis on them, as generally speaking the thumb will be incorrectly rolled.

Anyway, there are a few other details that need ironing out, but hopefully people can make better tutorials out of this.

CC: @MetaverseCafes Youll probably also find this interesting.
As a side note and answer to your question:
Make sure your widget is set to Normals selection tool, not Global when doing the Rotations. as you can see your Z rolls are pointing forwards.

This you make sure that the selected gizmo is in " Normal" or “Global” coordinates


Request: Please start locking threads that havent had any activity for longer than 6 months
WIP: MHX2 importer mods to help with makehuman to blender to hifi avatar creation
#2

Great stuff @Menithal!

I keep thinking that we could capture some of these workarounds into a Blender Add-on (for now just a helper script – to automate some of the most annoying or error-prone bits).

And given high-level command expressions, I could help implement the necessary Python glue. Here are some examples in a format that would be easy to translate into code:

  • select the armature, in pose mode
  • select all bones matching the words “leg” and “foot”
  • invoke menu item ABC
  • for all bones, remove any “mixamo:” prefix from the name

It looks like the different rigging systems have tell-tale signatures, so in theory their types can be detected and the workarounds targeted uniquely.

Also the tooltips displayed in Blender provide a useful clue for scripting – the “Python:” line is how that feature or command can be accessed from code:

You can also right-click on various settings (like checkboxes and sliders) and choose “Copy Data Path:”

… which puts the relevant part of the Python line onto the clipboard.

That might be a good way to start capturing sanity checks – for example, today I learned that when a Texture has the Influence -> Shading -> Emit checkbox enabled, it crashes my ./interface. Copying the Data Path gives me “texture_slots[0].use_map_emit” to use as a starting point:

  • for all textures, if .use_map_emit is enabled then display a warning!

Another thought is that Blender can be scripted from the command-line (like from a batch file), so the possibility exists here for a “post processing” step. That might be a good place to generate an .fst automatically from an .fbx, for example…

Hopefully this spurs some ideas!


#3

Yep, I was discussing the same thing with @Judas earlier.

Basically the Rule of thumb (heh) for avatars is:

  1. Arms and Hands Roll -Z Global, bpy.ops.armature.calculate_roll(type=“GLOBAL_NEG_Z”)
  2. Legs and Center of Mass (Hips, Spine, Neck, Head) Roll -Y Global bpy.ops.armature.calculate_roll(type=“GLOBAL_NEG_Y”)
  3. Eyes point Towards +Y, and Roll -Y Global
  4. Sanity Check: no texture names, object names, mesh names or Materials names with periods in them (this causes texture reordering issue in interface) replace all instances with ‘_’ instead of ‘.’
  5. Sanity Check: No Emit
  6. Sanity Check: Must have material
  7. Other Sanity stuff from my notes at https://alphas.highfidelity.io/t/ultimate-blender-compendium-woes-of-the-blender-fbx-export-at-the-mountains-of-documentation-madness/4739/1

As a side note The Roll doesnt seem to always work for legs, but mostly it does, Need to look into it abit more. But generally I think if a skeleton model is shared, and used as a base (such a @ozan Sintel or @Judas’s SinteM. ) it should work fine.


#4

OK, here’s a rapidly prototyped script, just to see how it could be done.

Before testing make sure you are using on a secondary copy of your .blend file (it shouldn’t do anything terrible, but best to be safe!)

gist: fufbx_ui_panel.py

The easiest way to use the script is to download and then in Blender drag a split pane window open and set it to the “Text Editor” type. Then just open fufbx_ui_panel.py and hit the “Run Script” button. The new panel and buttons should then appear under the “chain links” (aka: Constraints) properties tab:

(wasn’t sure where to put them – but that tab was relatively empty, so I commandeered it for now;))

The dots -> underscores rename button seems to work OK. (I started with that because it required scanning through all the object types in general, which helped refresh my memory of where things are in the Blender API)

The bone roll buttons are still early works in progress – all they do for now is select the bones. Getting into the right editing mode was tricky, since it depends on which state you start out in, but I think I’ve got it working reliably now from the common ones.

Hopefully it loads in other Blender versions too and lemme know if it works!

EDIT: oops, spoke too soon – just discovered some more bone selection entry states that are tripping things up; will see if I can’t get it working later tonight!

EDIT2: OK, as of “v0.0.3” the first two features now seem functional (replacing underscores and re-rolling bones along a global axis). I still couldn’t get my avatar to work with walk.js though, so to experiment more rapidly I added a second comprehensive row of axis buttons. Here is a screenshot of everything in action, including the split-plane Text Editor (which is how the script gets loaded).

(the “verbose” checkbox enables/disables the error popup (currently used like an alert box, but it might be annoying since you have to move your mouse away to dismiss); “restore bone selection” attempts to notice the bones selected going in and make them selected again afterwards (which can help to visually focus on one bone while testing the effects of various axis buttons)


#5

Remember when using the walk script, enable the prepost rotations on the script it self (use the beta script, not sure if it is yet on the main) that’s at least the first baseline test (closest to working)


#6

@Menithal - the version with the Blender fix is not live yet. I submitted the PR earlier today, so hopefully soon…


#7

So yeah as @davedub mentioned the walk script is not yet implemented:
Basically if you are testing the walk script, use the one which @davedub made here http://s3.amazonaws.com/hifi-public/procedural-animator/beta/walk.js, as its still not yet in the main client.

Ill probably take a stab at this week and check if I can make it so that its each are separate tools in the right tool bar or under the search menu so you can do. Basically it could be separated to various tools

  • HiFi: Sanity Check - Does all the materials / Name checks / Makes sure that the creator doesn’t go insane
  • HiFi: Armature Rolls - Does all the roll checks.

This is because blender can be used for a lot more than just avatars :slight_smile: so we shouldn’t really hide the tools under constrains.


#8

… been trying both walk.js scripts (main client for :walking: and beta for :running:)!

FWIW Rigify (eg: /Program Files/Blender/2.74/scripts/addons/rigify/*.py) adds UI options to pretty much everywhere possible in Blender UI, so it’s code might be a good place too look for clues.

EDIT: This is the best reference I’ve found so far for grafting things into the Blender UI: http://wiki.blender.org/index.php/Dev:2.5/Py/Scripts/Cookbook/Code_snippets/Interface

Originally I wanted to put the main options under the Help menu, but couldn’t figure out how yet. I figure the help menu might be a good place because it’d rarely get in the way of expert users and yet remain in easy reach of everyone else. Help -> HiFi -> Armature Rolls?

EDIT: @ritzo – (note: the forums won’t let me post further replies to this thread in general, so I’m editing this earlier post to address your follow-up questions)

The Center of Mass (and Arms and Hands) buttons just automate the manual process that Menithal demonstrated in his video – selecting those particular bones for you and then resetting their rolls along a particular global axis. The armature itself is not affected (or at least, shouldn’t be).

The “wide” buttons are set to the defaults used in the video notes; the smaller buttons allow choosing a different global axis (and except for axis all buttons affect the same number value – so only only one button per group is imagined to be useful in a given rigging scenario).

FWIW I no longer think bone rolls and pre-rotations can solve for contortions in general. The next easiest option is likely going to be “bone transplants” (on the Blender side – shedding whatever armature we wanted to use instead, and adopting an as-exact-as-possible replica of the reference model that informed the walking script’s animations originally).


#9

@humbletim Loads and Runs in 2.75. this will save lots of time when completed ^^^

i’m guessing  still work in progress
asssuming the rig is selected, can you clarify

  1. the ‘Centre of Mass(Hips, Spine, neck, Head) Roll -Y Global’ buttons (-x +x -y etc) will globally rotate
    the entire avatar through whichever axis (button) you choose and to the value in degrees?
    So for export to  Hifi we want to use -y and 90 degrees

[ Ref. Menithal 2. Legs and Center of Mass (Hips, Spine, Neck, Head) Roll -Y Global bpy.ops.armature.calculate_roll(type=“GLOBAL_NEG_Y”) ]

  1. ‘Arms and Hands Roll -Z Global’ button will roll Arms and Hands to whichever Global button you select?
    So for export to HiFi we want -Z  values
    as described by Menithal ? but not yet implemeted
    [Ref. Menithal 1. Arms and Hands Roll -Z Global, bpy.ops.armature.calculate_roll(type=“GLOBAL_NEG_Z”) ]

#10

Any explanation for the tongue being misplaced? Tongue position appears ok in Blender
After Eye Bones Rot hair slips, but i might have mistakenly had hair selected when i rotated eye bone, will check
But tongue visibility any suggestions ?


#11

That looks more like an alpha sorting issue ( as the shader is all glossy, not diffuse or phong.) than the tongue sliping out: check that the textures are fully opaque: Looks like the texture it self is transparent.

but otherwise, can you send me the blend file in DM and ill check it over.


#12

Yeah I went through your file, you still had your transparency enabled. see

Tested also all the bones rotations and the sort and these didnt have issues, other than the eye bone not completely pointing to the Z axis in blender…

For all parts that do not have transparent textures (pretty much, everything but hair and eyebrows) turn this off. For the transparent textures, switch to Z-transparency.


#13

Ah that’s me not even checking the transparency box was checked - who did that, not me ? No matter - smacks wrist , thanks Menithal