WIP: MHX2 importer mods to help with makehuman to blender to hifi avatar creation


#1

I’ve modified some files in MHX2 importer for blender.

Update: New version 4. (I skipped releasing the other versions).
What’s new:

  1. HF_default rig. All helper/ik bones matching the default (currently being of light) rig can be automatically generated on import. Use “hf_default” rig and NOT “hf” rig.
  2. fufbx_ui_panel_hfmod.py had been tested to work on the hf_default rig. Copy is included below.

Issues with version 4:

  1. The generated 4th digits of the finger for IK all points forward. I do not know how to rotate it in the mhx2 script. But they can be selected one by one and aligned to parent by clicking ctrl-alt-A in edit mode of the armature. Make sure that x-mirror-axis is disabled.

Downloadable files:

modded files in mhx2 importer:
http://fed-lab.com/bucket/version4-import_runtime_mhx2.zip

my copy of mhx2 exporter/importer files with the importer files patched with the mods, so you can test if it works:
http://fed-lab.com/bucket/MOD_V4-mhx2-makehuman-exchange.zip

Copy of @humbletim’s fufbx_ui_panel.py with minor additions of shoulder, foot and toe bones for rolling:
http://fed-lab.com/bucket/fufbx_ui_panel_hfmod.py

Extract and replace files in your mhx2 import folder. This is for the dev version, specifically Diffeomorphic-mhx2-makehuman-exchange-016613171fda.zip .

This mod includes an “hf” rig that comes as close to the default avatar rig except it does not have the end bones of the head, toes, or hands. All the bone names are the same as the default avatar rig and all vertex groups are also named according to the bones. The reason why I could not create this is because the rig file only references the default bones. Perhaps someone who knows how to use the plane for rotation of bones could make new bones for the rig that is not part of the default mh skeleton. This had been resolved with version 4.

I created this because the humanik rig in the old tutorials is no longer in the latest makehuman or contributed assets site. To use the rig set Rig type, to Hf in the mhx2 exporter panel of blender. And disable custom shapes, otherwise unset shapes in the skeleton panel and delete the shapes in the last layers.

To use the hf rig directly you may need to extrude end bones and name them accordingly. And do all the tweaks and bone roll rotations in @Menithal’s tutorials.

The mod also include a rough blink shapekey for left and right eye. I didn’t have time to correct the eyelashes because I created the blink target by using the eyelid bone drivers of the default skeleton. Someone may want to create a more decent blink target to include in the faceshapes.mxa file.

You can also unparent the hf rig, and append and parent the mesh to the default avatar rig. The vertex groups with weights will automatically be assigned to the bones because they have the same names.

I also have a t-pose that I tried to make as close to the default avatar so it would be easier to edit the bones if appended. This is installed in the data/poses folder of makehuman.

http://fed-lab.com/bucket/tpose4hf.zip

Btw. You’ll need the bvh exporter with Jonas Hauquier’s patches to tweak the .blend tpose and create correct poses for makehuman. Here’s is my copy that goes to the scripts/addons folder of blender:

http://fed-lab.com/bucket/io_anim_bvh.zip

See the ff. for tweaking the hf rig:

Older versions of mods
http://fed-lab.com/bucket/import_runtime_mhx2_HF_MOD_v2.zip
http://fed-lab.com/bucket/HF_MOD-mhx2-makehuman-exchange.zip


#2

As clarification, I actually made a ready made skeleton in Blender, you can find it in this thread.

You could use that in the Mod.

Other Blender tid bits available at


#3

Thanks @Menithal .
Please note that I’ve updated the link to the mod files in my op because I made a mistake in one line of the hf.json file. I’ve corrected it and posted a new link, including a copy of the patched full mhx2 package that I used for testing.
I matched the bone names in my mod to the Being of Light skeleton.
The hf rig in the mhx2 mod should match the bones in base_avatar_skeleton.blend except for ff.

  1. the eye socket bones, and the 4th digits of the hands do not exist because I do not know if bones without vertex group assignment could be referenced in the mhx2 rigs. The mhx2 custom rigs just reference and merge bones from the default makehuman skeleton.
    So the bones would have to be extruded from the relevant bones of the hf rig after it is created. But modding some files in mhx2 importer may include automatic creation of these bones; unfortunately it will take me some time to figure that out.
  2. The toes in the being of light are called LeftToeBase and RightToeBase while in the base_avatar_skeleton they are called LeftToe and RightToe. So if the hf rig is unparented from the mesh and the base_avatar_skeleton is parented instead the toe vertex groups would have to be renamed to match base_avatar_skeleton.

I’ve also noticed that the Being of Light skeleton has HeadTop_End and it map it to the headjoint in the fst file, while Head is mapped to the neckjoint.
(For other readers please see https://readme.highfidelity.com/docs/high-fidelity-avatar-standards to compare with base_avatar_skeleton)


#4

I’ll post a detailed step-by-step tutorial on my method of using the mod to create an avatar for hifi later.

I’ve already tested it successfully with the hf rig mod and the humanik tweaks (bonerolling, eye correction, etc.).
And the other method of parenting the mesh to the Being-of-Light skeleton.

My todo list would be:

  1. testing @humbletim’s fufbx_ui_panel.py to roll the bones of the hf rig mod.
  2. testing with mask modifier applied. I know the shapekeys are preserved when importing the mhx2 mesh with hf rig type but I haven’t fully created an avatar yet. Note that we cannot apply the mask modifiers to the mesh after the import in blender if the mesh had been imported in mhx2 with Modifier not set to Apply (e.g. set to Ignore) because of the shapekeys. This has relevance to pokethrough issues with clothes.

If auto bone rolling works out IMHO what still needs automation would be:

  1. extruding additional bones e.g. 4th digit, and end toes and head top bones and naming these bones according to the standard(s).
  2. rotation of axis of the entire avatar so it is z forward. (the process where the avatar is rotated 90d on its back)
  3. scaling of avatar to 100, then scaling back the armature/skeleton to 0.01, and applying scale to meshes to set it to 1.0 . (necessary for avatar bounding box issues)
  4. changing materials settings e.g. phong, transparency, emit if applicable.

#5

Ay the HeadTop_End is one of the generated tails for the skeleton. You can generate these in blender when Exporting with Leaf bones which I find unnecessary to have in the end result as it only causes more bones to be sent to the avatar mixer on animation.

The Basic Skeleton should be based on the Maya HumanIK Rig, which is the base for the Humanoid Avatars.
Most of the _End ending bones are IK Helpers, not really something you want to bind any vertex group to.

I made has some additional bones as well, I detailed the reasonings on the documentation it self, forexample the Eye Socket Bones. They should not be animated as they are for future avatar customization (eye distance). more details of that available on the Avatar Standard suggestion


#6
  1. I’ve completed the automatic generation of all the bones that match the default-being-of-light skeleton. Please see version 4 above. To match with the standard base skeleton, delete head top bone and add eyesocket bones.
  2. I’ve tested @humbletim’s fufbx_ui_panel.py and it works. I’ve added toes, foot and shoulders to the list of bones to roll. I don’t know if they are needed but I wanted to match the roll of the bones in default and standard skels.
  3. I’ve tested importing with applied mask to delete body vertex underneath the clothes and the shapekeys are preserved and functioning. See sample avatar below.

Automated rolling of spine:

Automated rolling of arms:

I guess what is left to automate are the ff.

and I forgot:
5. rotation and rolling of eye bones.
6. correction of shoulder bones.


#7

Here is a sample avatar created with the modded mhx2 importer and using fufbx_ui_panel_hf.py .

I still have the following problems:

  1. I’m unable to correct the shoulders. The imported rig is in t-pose.
  2. The teeth are all black, and the white of the eyes are too dark.

Eye blink works.
Voice works on the shapekeys if you copy-pasted the bs settings from an .fst file from another post in the forum.
I can’t remember where that post is right now. But here’s my .fst for this avatar:

name = teacher1_mask_applieddisconnect_bones
type = body+head
scale = 1
filename = teacher1_mask_applieddisconnect_bones/teacher1_mask_applieddisconnect_bones.fbx
texdir = teacher1_mask_applieddisconnect_bones/textures
joint = jointEyeLeft = LeftEye
joint = jointHead = HeadTop_End
joint = jointLean = Spine
joint = jointEyeRight = RightEye
joint = jointLeftHand = LeftHand
joint = jointRoot = Hips
joint = jointNeck = Head
joint = jointRightHand = RightHand
freeJoint = LeftArm
freeJoint = LeftForeArm
freeJoint = RightArm
freeJoint = RightForeArm
bs = BrowsD_L = brow_mid_down_left = 1
bs = BrowsD_R = brow_mid_down_right = 1
bs = BrowsU_C = brow_mid_up_left = 1
bs = BrowsU_C = brow_mid_up_right = 1
bs = BrowsU_C = brow_outer_down_left = 1
bs = BrowsU_C = brow_outer_down_right = 1
bs = BrowsU_C = brow_squeeze = 0.86
bs = BrowsU_C = brow_mid_up_left = 1
bs = BrowsU_C = brow_mid_up_right = 1
bs = BrowsU_C = brow_outer_down_left = 1
bs = BrowsU_C = brow_outer_down_right = 1
bs = BrowsU_L = brow_mid_up_left = 1
bs = BrowsU_L = brow_outer_up_left = 1
bs = BrowsU_L = brow_mid_up_left = 1
bs = BrowsU_L = brow_outer_up_left = 1
bs = BrowsU_R = brow_mid_up_right = 1
bs = BrowsU_R = brow_outer_up_right = 1
bs = BrowsU_R = brow_mid_up_right = 1
bs = BrowsU_R = brow_outer_up_right = 1
bs = CheekSquint_L = cheek_squint_left = 1
bs = CheekSquint_R = cheek_squint_right = 1
bs = ChinLowerRaise = lips_lower_out = 0.52
bs = ChinLowerRaise = mouth_open = -0.2
bs = ChinUpperRaise = lips_lower_out = 0.78
bs = ChinUpperRaise = lips_mid_lower_up_left = 0.51
bs = ChinUpperRaise = lips_mid_lower_up_right = 0.51
bs = ChinUpperRaise = lips_part = 1
bs = ChinUpperRaise = lips_upper_out = 0.31
bs = ChinUpperRaise = mouth_corner_down_left = 0.26
bs = ChinUpperRaise = mouth_corner_down_right = 0.26
bs = EyeBlink_L = EyeBlink_L = 1
bs = EyeBlink_R = EyeBlink_R = 1
bs = EyeSquint_L = cheek_squint_left = 1
bs = EyeSquint_L = cheek_squint_left = 1
bs = JawOpen = cheek_narrow_left = 0.41
bs = JawOpen = cheek_narrow_right = 0.41
bs = JawOpen = lips_lower_out = 1
bs = JawOpen = lips_mid_lower_up_left = 1
bs = JawOpen = lips_mid_lower_up_right = 1
bs = JawOpen = lips_mid_upper_up_left = 0.55
bs = JawOpen = lips_mid_upper_up_right = 0.55
bs = JawOpen = lips_part = 1
bs = JawOpen = lips_upper_out = 0.41
bs = JawOpen = mouth_open = 0.63
bs = JawOpen = mouth_wide_left = -0.3
bs = JawOpen = mouth_wide_right = -0.3
bs = LipsFunnel = lips_lower_out = 0.31
bs = LipsFunnel = lips_mid_lower_down_left = 0.41
bs = LipsFunnel = lips_mid_lower_down_right = 0.41
bs = LipsFunnel = lips_mid_upper_up_left = 0.67
bs = LipsFunnel = lips_mid_upper_up_right = 0.66
bs = LipsFunnel = lips_part = 1
bs = LipsFunnel = lips_upper_out = 1
bs = LipsFunnel = mouth_corner_down_left = 0.45
bs = LipsFunnel = mouth_corner_down_right = 0.45
bs = LipsFunnel = mouth_corner_in_left = -0.4
bs = LipsFunnel = mouth_corner_in_right = -0.4
bs = LipsFunnel = mouth_down_left = 1
bs = LipsFunnel = mouth_down_right = 1
bs = LipsFunnel = mouth_narrow_left = 0.83
bs = LipsFunnel = mouth_narrow_right = 0.83
bs = LipsLowerClose = lips_lower_in = 1
bs = LipsLowerClose = lips_part = -0.5
bs = LipsLowerDown_L = lips_mid_lower_down_left = 1
bs = LipsLowerDown_L = mouth_corner_down_left = 0.64
bs = LipsLowerDown_L = mouth_wide_left = 0.59
bs = LipsLowerDown_R = lips_mid_lower_down_right = 1
bs = LipsLowerDown_R = mouth_corner_down_right = 0.65
bs = LipsLowerDown_R = mouth_wide_right = 0.6
bs = LipsPucker = cheek_narrow_left = 0.42
bs = LipsPucker = cheek_narrow_right = 0.42
bs = LipsPucker = lips_lower_out = 1
bs = LipsPucker = lips_upper_out = 1
bs = LipsPucker = mouth_wide_left = -0.5
bs = LipsPucker = mouth_wide_right = -0.5
bs = LipsStretch_L = cheek_balloon_left = 0.5
bs = LipsStretch_L = lips_mid_upper_up_left = 0.16
bs = LipsStretch_L = lips_mid_upper_up_right = 0.16
bs = LipsStretch_L = mouth_corner_down_left = 0.67
bs = LipsStretch_L = mouth_down_left = 0.22
bs = LipsStretch_L = mouth_wide_left = 1
bs = LipsStretch_L = mouth_wide_right = 0.29
bs = LipsStretch_R = cheek_balloon_right = 0.5
bs = LipsStretch_R = mouth_corner_down_right = 0.67
bs = LipsStretch_R = mouth_down_right = 0.22
bs = LipsStretch_R = mouth_wide_left = 0.29
bs = LipsStretch_R = mouth_wide_right = 1
bs = LipsTogether = mouth_open = -0.5
bs = LipsUpperClose = lips_part = -0.5
bs = LipsUpperClose = lips_upper_in = 1
bs = LipsUpperUp_L = cheek_balloon_left = 0.3
bs = LipsUpperUp_L = cheek_up_left = 0.5
bs = LipsUpperUp_L = lips_mid_lower_down_left = 0.36
bs = LipsUpperUp_L = lips_mid_upper_up_left = 1
bs = LipsUpperUp_L = mouth_corner_down_left = 0.25
bs = LipsUpperUp_L = mouth_corner_down_right = 0.14
bs = LipsUpperUp_L = mouth_corner_in_left = 0.71
bs = LipsUpperUp_L = mouth_wide_left = 0.78
bs = LipsUpperUp_R = cheek_balloon_right = 0.3
bs = LipsUpperUp_R = cheek_up_right = 0.5
bs = LipsUpperUp_R = lips_mid_upper_up_right = 1
bs = LipsUpperUp_R = mouth_corner_down_right = 0.13
bs = LipsUpperUp_R = mouth_corner_in_left = 0.25
bs = LipsUpperUp_R = mouth_corner_in_right = 0.71
bs = LipsUpperUp_R = mouth_wide_right = 0.78
bs = MouthDimple_L = cheek_balloon_left = 0.23
bs = MouthDimple_L = lips_mid_lower_up_left = 0.3
bs = MouthDimple_L = lips_mid_lower_up_right = 0.14
bs = MouthDimple_L = lips_mid_upper_down_left = 0.3
bs = MouthDimple_L = lips_mid_upper_up_right = 0.05
bs = MouthDimple_L = mouth_corner_up_left = 0.14
bs = MouthDimple_L = mouth_up_left = 0.09
bs = MouthDimple_L = mouth_wide_left = 0.73
bs = MouthDimple_R = cheek_balloon_right = 0.23
bs = MouthDimple_R = lips_mid_lower_up_left = 0.14
bs = MouthDimple_R = lips_mid_lower_up_right = 0.3
bs = MouthDimple_R = lips_mid_upper_down_left = 0.05
bs = MouthDimple_R = mouth_corner_up_right = 0.14
bs = MouthDimple_R = mouth_up_right = 0.24
bs = MouthDimple_R = mouth_wide_right = 0.78
bs = MouthLeft = mouth_corner_up_left = 0.72
bs = MouthLeft = mouth_down_right = 0.24
bs = MouthLeft = mouth_open = -0.05
bs = MouthLeft = mouth_up_left = 0.45
bs = MouthLeft = mouth_wide_left = 2
bs = MouthLeft = mouth_wide_right = -0.3
bs = MouthRight = cheek_balloon_right = 1
bs = MouthRight = mouth_corner_in_left = -0.2
bs = MouthRight = mouth_corner_up_right = 0.72
bs = MouthRight = mouth_down_left = 0.24
bs = MouthRight = mouth_open = -0.05
bs = MouthRight = mouth_up_right = 0.45
bs = MouthRight = mouth_wide_left = -0.3
bs = MouthRight = mouth_wide_right = 2
bs = MouthSmile_L = cheek_balloon_left = 0.72
bs = MouthSmile_L = lips_part = -0.2
bs = MouthSmile_L = mouth_corner_in_left = 1
bs = MouthSmile_L = mouth_corner_up_left = 0.77
bs = MouthSmile_L = mouth_up_left = 0.37
bs = MouthSmile_L = mouth_wide_left = 2
bs = MouthSmile_R = cheek_balloon_right = 0.72
bs = MouthSmile_R = mouth_corner_in_right = 1
bs = MouthSmile_R = mouth_corner_up_right = 0.77
bs = MouthSmile_R = mouth_up_right = 0.37
bs = MouthSmile_R = mouth_wide_right = 2
bs = Puff = cheek_balloon_left = 1
bs = Puff = cheek_balloon_right = 1
bs = Puff = lips_mid_lower_up_left = 0.22
bs = Puff = lips_mid_lower_up_right = 0.22
bs = Puff = lips_part = -0.1
bs = Puff = lips_upper_in = 0.18
bs = Puff = mouth_corner_down_left = 0.17
bs = Puff = mouth_corner_down_right = 0.17
bs = Puff = mouth_narrow_left = 0.49
bs = Puff = mouth_narrow_right = 0.49
jointIndex = RightHandRing4 = 38
jointIndex = LeftEye = 71
jointIndex = LeftHandRing4 = 50
jointIndex = LeftHandRing1 = 47
jointIndex = Spine = 16
jointIndex = RightHandThumb1 = 39
jointIndex = RightLeg = 7
jointIndex = LeftShoulder = 43
jointIndex = LeftToe_End = 15
jointIndex = LeftHandThumb2 = 52
jointIndex = LeftForeArm = 45
jointIndex = LeftHandMiddle4 = 66
jointIndex = RightHandPinky2 = 24
jointIndex = LeftHandPinky1 = 59
jointIndex = Shoes03 = 3
jointIndex = RightToe_End = 10
jointIndex = LeftHandIndex3 = 57
jointIndex = RightArm = 20
jointIndex = LeftUpLeg = 11
jointIndex = RightHandRing1 = 35
jointIndex = RightEye = 70
jointIndex = Spine2 = 18
jointIndex = LeftHandMiddle2 = 64
jointIndex = Teacher1 = 4
jointIndex = LeftArm = 44
jointIndex = RightHandPinky1 = 23
jointIndex = LeftHandPinky4 = 62
jointIndex = LeftToeBase = 14
jointIndex = LeftHandIndex1 = 55
jointIndex = RightHandPinky4 = 26
jointIndex = LeftHandPinky3 = 61
jointIndex = RightHandThumb4 = 42
jointIndex = Short02 = 1
jointIndex = RightFoot = 8
jointIndex = HeadTop_End = 69
jointIndex = LeftHandMiddle3 = 65
jointIndex = LeftHandMiddle1 = 63
jointIndex = LeftHandRing3 = 49
jointIndex = RightHandIndex2 = 28
jointIndex = Head = 68
jointIndex = RightHandThumb3 = 41
jointIndex = RightHandMiddle1 = 31
jointIndex = LeftHandThumb4 = 54
jointIndex = RightHandThumb2 = 40
jointIndex = RightHandIndex4 = 30
jointIndex = RightHandMiddle4 = 34
jointIndex = Neck = 67
jointIndex = Hips = 5
jointIndex = LeftHandThumb1 = 51
jointIndex = RightHandPinky3 = 25
jointIndex = RightUpLeg = 6
jointIndex = LeftFoot = 13
jointIndex = LeftHandThumb3 = 53
jointIndex = RightHandIndex3 = 29
jointIndex = RightHandIndex1 = 27
jointIndex = RightForeArm = 21
jointIndex = RightHand = 22
jointIndex = LeftHandIndex4 = 58
jointIndex = RightHandRing3 = 37
jointIndex = Male_elegantsuit01 = 2
jointIndex = LeftHandRing2 = 48
jointIndex = LeftHandIndex2 = 56
jointIndex = RightHandMiddle2 = 32
jointIndex = Body = 0
jointIndex = RightShoulder = 19
jointIndex = RightHandMiddle3 = 33
jointIndex = RightHandRing2 = 36
jointIndex = LeftHand = 46
jointIndex = LeftHandPinky2 = 60
jointIndex = RightToeBase = 9
jointIndex = LeftLeg = 12
jointIndex = Spine1 = 17


#8

I have two questions for the community.

  1. Does it matter if bones in the spine, arms, and legs are connected or disconnected? In the default skel they are disconnected, but in the base standard skel they are connected.
  2. How can we reorder the list of bones for the fingers in the outliner panel of blender to make the middle finger the first bone listed among the fingers?

#9

Late to the party - but one thing to look out for in some of the skeletons from HF avatars… they’re not symmetrical - before I got busy with other aspects of HF I had began looking at this as I had noticed while using skeletons grafted from HF produced avatars that sometimes they’d look… off balance, like the camera is tilted a little when you’re in desktop mode… you don’t notice it in HMD so much as you, subconsciously I suppose, compensate. When I started really looking at the skeletons in detail I saw differences in various dual side bones like arms and other, not to be unkind, but, sloppiness. Maybe not a concern… just saying in case it might save some head scratching someday.


#10

The imbalance is due to the underlying mixamo auto rig or other imbalance. Normally thoughnobodys spine is absolutely perfect. I made my template avatars symmetrical for better compatibility though

@rcantada

  1. The shoulders are an exception in the t- pose. If I recall correctly. They have to be angled very slightly down like in my template. This is due to the source animation skeleton rolling the bones up slightly too. Note that the animation skeleton is different from the beingoflight, animations were done before the avatar was ever created
  2. Finger bones last I checked were the metacarpals. So finger1 is the finger’s metacarpal, not the first phallenge. I nowdays just place these on the knuckles however as the animations do not seem to agree with that any longer.
  3. Material tutorial on blender
    https://youtu.be/mH5YXpcgAfs
  4. Keep bones such shoulder-spine or hips-legs disconnected in blender. Others doesnt matter if connected, but makes it easier if they are. Otherwise the angle gets changed for animation.