Blender Asset Creation Notes - At the Mountains of Documentation Madness


#1

This post is about archiving information about Blender and howto guides on Blender.

Replies to this thread are mostly Historical, So please don’t be intimidated: Ill try to keep the OP updated with latest, and most valid information.

All of this applies for Blender 2.79. which is currently the latest. For now, use Blender Render / Opengl render instead of eevee renderer. Gltf is in the works.

Last Updated March 15 2018

  • Pointing Towards the Blender Plugin

Blender Plugins:

The blender plugin automates alot of the tiny details that when missed can cause pain and wonder why things dont work. So it is highly suggested to be using that with the general practises and information available on this page

General Observations:

  • 1 Blender Unit = 1 Unit in HF (therefore meter).
  • Imported and exportes avatars are also scaled down by 0.01 in Blender, so they actually are truly massive.
  • Export avatars Y axis - 90
  • As far as we know, the new versions of Blender seem to do automatic world orientation correctly.
  • Default FBX export settings now work fine.
  • Keep textures in same directory as the model if embedding the Textures. This make sure blender exports them correctly in fbx.

Good General Practises

  • Use PNG or JPGs only. TGA is not supported
  • Avoid having too many separate mesh groups. Keep it below 4, preferably 1. This is especially true if importing avatars from SL which use prims instead of mesh.

Good Avatar Practices :

  • ** Avatar models Must have Geometry which is weighted to the Head**: Without this the Physics will bork out as Hifi Cannot determine what physics shape the avatar must use.
  • Rigging to a Absolute T-pose so that all bone rotations are relatively same.
  • Full Body Avatar models (no textures embedded) should aim for 3 mb mesh size (less than 15k polygons total + UV + weights + standard armature).
  • Head Avatar model should aim for less than for 5 k polygons. (that is still a LOT)
  • Try to keep the total size of a full avatar with textures less than 15mb. HF does a lot of throttling so lower the better and more visibility compared to more polygons: While your avatar is priority for you, this works for everyone else, so the more resources your avatar uses the more likely its going to be cut first from the rendering que. You can ofcourse mitigate this by using LOD models.
  • Aim for more detail on the face.

Optimizations in FBX file

  • If you have your own web service or plan on reusing already uploaded textures, keep the Path Mode in Auto, and manually point to the texture directory instead.so that you can keep reusing a library of textures to save bandwidth and server space instead. You should keep textures in a consistant path.
  • If you are using dropbox a service that hashes your files to host avatar files temporarily while experimenting, make sure to set Path Mode to Copy, and Check the Embed Textures button. Credit goes to @Judas.
  • Keep only the mesh and the armature during export Everything extra adds to the bounding box size and file size.
  • You DO NOT have to bake animations unless making animations only.
  • If you are exporting animations, export only the armature. This allows the animation to be small and quick to load and cross compatible with other avatars or objects using the same body.

UV Maps

  • Multiple UV Maps on a single mesh is not supported: Instead combine your UV maps into a single one, or separate your mesh on these components to separate parts. The First UV Map in the list will be used for the textures.
  • Make sure you have as small amount of UV maps on the avatar as possible.

Textures and Materials

  • Before working on textures, it is highly suggested you have an Zone with a skybox applied to allow for reflections to show best.
  • Keep textures below 2048 px x 2048 px. Any larger size get resized and are just a waste of bandwidth for both the servers and clients
  • Use PNG or JPGs only. TGA is not supported
  • Avoid using too many separate materials. Instead favor a single material and texture over multiple smaller ones.
  • If planning to Embed Textures, make sure to have the textures in the same directory as the model before binding the textures. Embedded Textures mode in Blender doesn’t otherwise bind the textures properly to fbx
  • If you want to have transparency, use Mask Transparency.
  • You must apply a material to the model, or it will not show or behave strangely.
  • High Fidelity binds textures from
  • Alpha comes from the Albeido alpha mask
  • Binding Reference: Note, Blender Specular Color / Metallicness now allows gradients.
  • Blender Add-on now contains a Tool allowing for faster and easier binding of Textures.
  • Texture Binding video tutorial available (legacy)

Physics

Rigging

Quick Flow

Skinning

  • If a Mesh vertice does not have a weight assigned and has an armature it will fold toward a client avatars neck. This occurs also for entities

Shapekeys and FST.

  • (Incomplete) List of Shapekeys used by Faceshift
  • As long as shapekeys (in Blender, referenced as blendshapes in docs) are named the same as in the Faceshift Rig, They will work out of the box.
  • You can define maximum constraints for shapekeys in the fst file or bind Shapekeys to other shapes in the following manner:
    bs = <FS Shape Key> = <Model shape key> = <Float value>
    for example:
    bs = MouthDimple_L = Smile_Left = 0.25
    This affects ALL meshes using the shapekey Smile_Left
  • You can also do advanced selection of shapekeys ones if you have a multimesh avatar such as:
    bs = MouthDimple_L = mesh_id.Smile_Left = 0.25
    So that only mesh with the id of mesh_id will have its Smile_Left value set to 0.25 when FS shapekey Mouth Dimple L value is 1.
  • You can combine complex shape mixes for a a shape such as
    bs = MouthDimple_L = mesh_id.Left_Lip_Left= 0.25 bs = MouthDimple_L = mesh_id.Left_Lip_Up = 0.5
    To mix Left_Lip_Left and Left_Lip_Up shapekeys together when MouthDimple_L shape is being set.
  • A word of caution: avoid having an extra period in the shapekey name or mesh_id these as it will break the shapekey_binding: Default Untitled# mesh_ids can also cause this, so name your meshes.
  • Also avoid stacking the same shapekey_value in the same FS Shape key as this will cause the values to be added to each other.

Mixamo -> Blender -> High Fidelity Flow

General FST notes:

  • *Instead of binding Hips as the Root bone, bind the armature object instead as the root. This should solve the bounding box issue. Not allowing Animations to shift Hips should help after.
  • If the above does not fix the bounding box issue, you can also add an additional root bone that should be the “root” - Thanks @Triplelexx
  • You can apply absolute paths to texture and filename sources.
  • You can apply rx, ry, rz for rotation overrides, and tx, ty, tz for translation overrides.
  • Best to do all of the shapes, since you never know if a user will have Faceshift or is using the camera tracking.
  • Make sure you bind the following as freejoints on fst package for them to work with the hand tracking solutions: Confirmed.
    RightShoulder RightArm RightForeArm RightHand LeftShoulder LeftArm LeftForeArm LeftHand.
    Most of these are bound by default, however I am not sure if the Interface packager actually writes them into the file (only Free joint I see occasionally is just the first entry). Not sure if this is fully true, but the IK Constraints do not display if these are not set.
  • (Depricated) You used to be able to inject LOD models into the fst file by adding the following to it:
    lod=<PathToMediumModel>.fbx= 10 lod=<PathToLowModel>.fbx= 15 lod=<PathToLowestModel>.fbx = 20.

On Animations

Cross Suite animations now work: There are no longer any issues with rotations as long as Pre/Post Rotations setting is enabled in the client (this may be off by default).

Make sure Simplify is set to 0, and there are no FBX Root defined in the Armatures tab.

On Attachments

Cross Suite attachments work. Just make sure to follow the standard if set to be “soft”. Any extra bones in the clothing will behave as if not bound to anything if the avatar it is bound to does not have them.

Side Notes

  • You do not have to follow the standards if you just want to make a unique avatar that isnt compliant with the other avatar animations or Clothing.

  • You can play animations even if your avatar doesn’t have bones for it. If you avatar has the bones they will be animated. This means Animating an Tail or Ear or Cape can be done and then applied as an additional idle animation on your avatar if your avatar has the bones for them.


Objects with PBR textures
WIP: MHX2 importer mods to help with makehuman to blender to hifi avatar creation
3D Avatar Modeling Critique Wanted
Avatar file formats
Looking for a material guide
Interest in a Blender Avatar creation workshop
Going out Staying in Judas's Blog
Cannot get my blender model to have textures in HiFi
Mesh blender weirdness. Texture / face problem
MakeHuman to Blender to High Fidelity ATP
Hello again old acquaintances!
Cannot get Textures to import with models
Custom avatar appears, then disappears
#2

Your doing some good work here.
What i learned is to put it down and walk away from time to time or your brain starts to fizz


#3

I updated the first post with some more information regarding LOD models and more details on the reasoning behind defining the Freejoints.


#4

Wow, Menithal, this is an enormous resource. Huge amount of time you have spent.
Thank you for sharing.


#5

Heres a fun little observation while I was double checking the import settings for FBX in Blender.

If you turn off Automatic Pre/Post rotations, the imported example avatars do the exactly the same as what the walk-cycle script does to the avatars in HF, and import the avatars at with Primary being Y and Secondary being X.

This is just an observation, but it might be a red herring too.
My Second observation is based on brining in animations with the broken skeleton:

I could apply the quick and dirty animation from Mixamon but not through auto-rig. The animation it self works mostly fine (With the exception of hips not translating even if shift hips is enabled (but they do rotate): But only on the avatars that were exported from blender.

Forexample using this animation

On Blender Exported Avatar (FenFen):

On the side note, please note that this animation was designed for a slightly warped humanoid shape (has digigrade legs, and 4 fingers and originally was not in T-pose by more of Palms out A-pose), but generally still should be close when animating on other avatars, so forexample the example box avatar would look like:

On Blender Exported Avatar (Box Skel)

But Not like…

Original High Fidelity Avatars (Will):

So in theory, you can still work with the stuff you export from blender and apply animations to the custom rigs. HOWEVER do not expect existing animations to work or the procedural animation scripts to work too.

I can safely claim that for Blender Animations and Avatars to work with existing content in HF, Content creators need to do something bout the skeleton before starting to work on animations or the avatar But What exactly needs to be done still remains a mystery we have to solve. And this is something that needs to be solved during alpha.

What suggestion I have though, is that avatar standards should state that all avatars must be rigged from T-pose, so that the relative rotations of all animations can, mostly be interchanged with each other (other than for special cases)


#6

i import Maya and Mixamo fbx’s using


#7

I love this thread.
Honestly, though, I find Blender an unbelievably more complicated tool for doing most things.


#8

That might be the crucial information missing. I’ll try this today to validate the rotations.


#9

Have you looked at clara.io ? It does not have all the features of the other systems but it is free and web based. Full disclaimer, I have not tried it for rigging.


#10

@chris: I have not tried Clara.io.

But to be honest, there are issues with online tools, especially considering something as time consuming as 3D modeling and rigging:

  • Not everyone will always be able to go online or if service goes down. Sure there is offline caching, but thats a whole another ball of problems as it will mostly just limit you to which ever browser it was designed for, which I should know; having been software developer in Web Industry for half a decade. Calls back to IE as well.
  • You get bound to service terms that may change at a short notice (such as stopping being free, limitations etc), or such as getting bought by even Bigger company and changing alot of things (See OnLive).
  • Just like with software, the work you do on the service will be designed on the service. Imported elsewhere, there come issues.

I’m sure that there are others who will have the similar concerns. Out of curiosity, I will definitely check it out at some point.

But I digress, this thread I am trying to find the workflow for Blender, or debug what may be causing the issues in the first place in blender FBX export. So that proper documentation can be written. Not to find the current fastest way of doing it because of unknown variables (is it the export script, or is it just incorrect parameters). We should reveal these unknown variables.

@Judas:

I just went through all the joints from a reimported avatar with your corrected import settings: No dice however: there was no change in the armature rotations from the automatic… Could you try to see the box skeleton model and seeing if windows exporter somehow works differently? I recently updated it with fixed rotations. (I noticed I had put another test model there instead)

Let me run through the process how I did my centering with hips (I use alot with hotkeys, (On mac, change ctrl to cmd) so heres some tricks too! It also takes me less time than to write this post.):

  1. Select the bone (or its joints. in edit mode.)
  2. Set the 3D cursor to selected component. (Ctrl + S, 4)
  3. Go off edit mode. (Tab)
  4. Select the model and armature (A, while lights and camera are unselectable or manual selection)
  5. Set this as the models and the armatures origin point (Shift + Ctrl + Alt + C, 3)
  6. Center the armature to 0 0 0 (Shift + C (Centers 3D cursor), Shift + S, 2 (Snaps selection to cursor))
  7. Export

I also ran through a couple of tests through all various hip origin placements
(centering at the hips is vague), so I tried the following

  • top of Hip bone
  • bottom of Hip bone
  • middle of Hip bone

None of these had any affect when walk.js was on, but had overall effect of moving the model up and down if animations were off.


#11

Updated the OP with latest information available.

Spoke to @Judas inworld today, he tried to look into the Box Skel model I have up above, but had the same issues as I did. So there is definetly something afoot.

I created a worklist job which should allow us to debug this a bit further.


#13

I was looking through the source code of the FBXConverter on github/hifi.

This is a shot in the dark, I spotted some interesting hacks related to Mixamo: mostly related to Applying different type of coordinates instead of other models.

See Line 621 and 1993 of FBXReader.cpp

Generally, The question I asked my self is What exactly was this bug with fuse that warranted this hack? would it be similar to the issues we currently have in Blender? Unfortunately I do not have enough experience with C++ (I am a Web, Java, (visual) objC, Python dev, but never done C++) yet to build my own version of this see what is this “fix”. It seems like it skips making binding parents to nodes.

Im guessing it could boil down to having the armature (converted into a group that contains bones), instead of having straight away bones that is causing issues with blender. Unfortunately this is just conjecture as I am not that familiar with the FBX format.

Could someone who can build the client check what would happen if they’d build an additional few lines which would check if Blender exported models require with a similar hack ( especially for #L1993), by adding geometry. applicationName == “Blender (stable FBX IO)” ) and test if the model I posted earlier would unfold if using walk script?

April 8: I modified the FBX export script in Blender to inject mixamo.com into the applicationName field of the fbx file. Didn’t seem to affect a thing. Will keep testing this later today…

Also what is up with latest build (since 2122). All components seem to be… err…


Reversed in the Z buffer. (Im guessing this is an latest build thing so it might get patched quite soon., tested dev build #4607 and that didn’t have the same issue)


#14

I dug really deep into the Blender scripts used to import and export fbx, may have found a root cause to why rotations are off. Maybe even dug too deep and encountered a demon.

I found this little golden nugget of information…

The Pre/Post rotation application I found earlier? Apparently there is a reason for it:
See Autodesks own documentation on this:
http://download.autodesk.com/us/fbx/20112/FBX_SDK_HELP/index.html?url=WS1a9193826455f5ff1f92379812724681e696651.htm,topicNumber=d0e7429

I’ll probably just glance at this stuff, but its going over my head for now :stuck_out_tongue:


#15

Okay, I think I finally found the root cause. The armatures of the animations which work with the current marketplace Avatars use PreRotation (with 3D vectors) instead of Lcl Rotations (they do not have them set even).

**Blender on the other hand only exports Lcl Rotations and does not even apply PreRotation or the requires RotationActive ** parameter per bone. The issue is related to the earlier discovery with the pre/post rotations being disabled folding the avatar in blender.

When I did a quick dirty hack on my FBX file and changed all the instances of Lcl Rotations to Prerotations with 3D Vectors, I got my avatars legs working fine with the walk script.

In anycase, now would be the perfect opportunity to me hand this torch over to a developer who knows their 3D math a bit further that I can :D. Im justifying that this should be done on the HF end because Mixamo also seems to do the same thing (but there is a wierd looking hack for it in the code), so there actually might be similar issues exporting in Max FBX format.

[Here is the worklist job ticket to try solve this issue.][1]
[1]: https://worklist.net/20467


#16

How?
Could you give us a workflow for that hack?


#17

The hack is ridiculously dirty and I wouldn’t implement it in anything

Since it doesn’t solve the actual issue 100% (instead worklist item) due to going beyond my math capabilities (I only can do matrice transforms on a 2D level…) and I don’t exactly know the full requirements of each line change ( such as difference between Lcl Rotation and Vector3D for best conversion, if one of them relative? or is it just all as is?)

From what Ive read so far, the reason for the Pre/Post Rotations is to avoid gimbal locks during animations (such as the ones which occur with the shoulder of the Gangnam style animation from Mixamo). (because oddly they didn’t seem use quaternions) However, some applications simply do not export it (Blender, Max, cheetah3D, ).

As a final warning,

Do NOT rely on this hack

instead, get someone on this worklist job. Basically the **hack fixes “some rotations” but it will not make the rotations accurate, so every joint will rotate slightly wrong:
but it will not fold your avatar skeleton in half any longer. **(instead it might just arms cross…)

Im only posting this hack here to help with the development process.

  1. Center model in Blender and apply rotations, scale and location
  2. Export from blender as FBX binary, with leaf bones on ZX. (No orientation on armature)
  3. Use Autodesk FBX converter tool to convert to 7.4 ASCII.
  4. Open ASCII file with text editor
  5. Under Object Properties, within All Model LimbNode s replace instances of Lcl Rotation with Vector 3D PreRotation. and per each modified line add additional RotationActive boolean. Make sure to ONLY do these for Model, with the LimbNode param.

So that:

Model: 4328734208, "Model::Hips", "LimbNode" { Version: 232 Properties70: { P: "InheritType", "enum", "", "",1 P: "DefaultAttributeIndex", "int", "Integer", "",0 P: "Lcl Translation", "Lcl Translation", "", "A",0,5.9604651880818e-08,0.7451993227005 P: "Lcl Rotation", "Lcl Rotation", "", "A",3.23046133824941,-1.25732161001369e-06,2.22763582797049e-05 P: "Lcl Scaling", "Lcl Scaling", "", "A",1,0.999999940395355,0.999999940395355 } Culling: "CullingOff" }

Would instead be

Model: 4328734208, "Model::Hips", "LimbNode" { Version: 232 Properties70: { P: "InheritType", "enum", "", "",1 P: "DefaultAttributeIndex", "int", "Integer", "",0 P: "Lcl Translation", "Lcl Translation", "", "A",0,5.9604651880818e-08,0.7451993227005 P: "RotationActive", "bool", "", "",1 P: "PreRotation", "Vector3D", "Vector", "",3.23046133824941,-1.25732161001369e-06,2.22763582797049e-05 P: "Lcl Scaling", "Lcl Scaling", "", "A",1,0.999999940395355,0.999999940395355 } Culling: "CullingOff" }

Basically the important change are

**
P: “RotationActive”, “bool”, “”, “”,1
P: “PreRotation”, “Vector3D”, “Vector”, “”,
**

Make sure to keep formatting of the file consistent. After this.

  1. Replace Original|ApplicationName Blender ( I /O ) property with mixamo.com (this seems quite important actually)
  2. Save
  3. Use fbx tool to convert back to bin
  4. Package and Upload.

Less manual method is to crack open the export scripts in blender and add up support for prerotation and postrotation values to exported files. But that wouldn’t solve the issue on other applications exporting out in Max FBX format.


#18

Updating the OP, with some more historical info down here.

I ran into a odd bug with the collision model now that I finally got back to civilization, and got onto a computer capable of showing entities. Found out that the avatar sinks half way into the floor.

With it, I found a separate bug with the debug reseting it self if the avatar was slightly resized, but not matching the actual bounds. (See worklist ticket https://worklist.net/20489 )

Still gotta find out what its cause it: There is a remote possibility that it might be also related to the folding action, but I doubt it due to the amount of info Ive gathered from the code, the presence of a hack for mixamo models, and Autodesks own FBX documentations on difference between maya and 3dmax fbx format…

However, I tried to triage this issue by fiddling around with the origin point of blender in High Fidelity, and trying again Judas’s hack (maybe I was doing something incorrect, but considering he also tried it…): Didn’t seem to affect the avatar position at all though:

After trying out importing my model into 3Ds Max and into Maya, I noticed that there was an extra parent bone which was the root object for the armature in blender.

Next step I tried was to simply remove the Hips Joint and rename the Armature object into being the Hip joint. This didnt seem to affect the bounding box, and what ever I did with the origin, these did not affect the placement if the walking script was enabled.

I did also find out that if you turned on the walk script, the bounding box would break for the avatar… So do animations and scripts such as these actually adjust your physical server presence?


#19

Tried to chat with some of the volunteer developers at blendercoders to see if someone has done any work on Post/Pre-rotation fbx export to validate my theory of the folding avatar: it is not on list of things to do as of this moment, until Autodesk releases a more solid universal specification that work on all, if ever.

Makes one question why the FBX format was chosen in the first place over other formats? Was it mostly due to the the potential for binary format and size?

In anycase it seems like it requires a lot more work to get the reader working properly and be cross compatible with each exporter, which should be done at this state of development.

In anycase, I will continue exploring this issue further and maybe even diverge into its own thread.


#20

@ Menithal, impressive investigation. i especially like the quote “Makes one question why the FBX format was chosen in the first place over other formats? Was it mostly due to the the potential for binary format and size?” :stuck_out_tongue:


#21

we could have a whole wiki about this at how it is right now and it keeps changing every minute of the day :slight_smile: thanks for sharing