Note this a work in progress and open for anyones input as Hifi is also in Development.
After all, we should all agree on one most will be happy with, as having conflicting standards would conflict with everyones interest. Once this is in a ready format and the standard requirements have been met by High Fidelity, I will apply to have this added to documentation.
Remember that this is a recommendation to best cross-compatibility with everyone else content, who follows the standard. It is not written in stone, you do not have to follow it: but its best for the end users if it does.
Maya users can simply use the HumanIK skeleton as that has pretty much all but the optional stuff
If you using Blender, use
Proposal: Avatar Standards
Last Updated September 19 2018
- Updated Blendshape definition, along with explination what is used by engine what is not, and what is available as “fake shapekey slots” for something else entirely.
- Avatar - The being which behaves as the selected representation of a person or npc
- Mesh - Topology for the avatar model. Without this, the avatar is invisible
- Bones - A Component of a Skeleton that defines a “limb” such as an arm, leg, etc that might have to be animated as a separated limb
- Skeleton - Bones, Armature used by Avatar mesh and animations to animate to avatar the mesh
- Rigging - Creating a structure of the Skeleton of the model, and the various constraints/tools which affect that skeleton
- Weights - Per vertex definition which define how each bone in the skeleton affects the mesh.
- Skinning - The act of painting weights for the skeleton.
- UV mapping / unwrapping - Defining how the topology maps to a 2D texture
- Shapekeys / Blendshapes - variations of the topology that act targets how the mesh is modified to create various “shapes”
- FST - Faceshift Transform file - used to define the faceshift blendshape bindings
- End Users - the users of the avatar
- Avatar Base - this is the base avatar without any custom modifications.
Requirements of the Standard from High Fidelity
FBX Processing for all 3D suites must behave similarly. Currently,Now Fully Realized.
Scripts must be able to modify bone’s scale and translation as well as rotation. Currently rotation and translation is supported. Scale is supported through animation on entities, but remains untested via scripts on avatars.
Avatar animations played should not override bone scales or translations set by scripts. instead of overriding, they should be additive. Currently, animations played back on avatars do not scale avatar, but do affect entities, there also isn’t away to do relative translation that is not effected by rotation or translation overrides
Allow scripts both to lock joints from being animated by animations but also allow them to open up joints to mixing of animations and script set position
Don’t override rotation if setting translation of a bone: Currently if you set the translation of a bone, you also set the bone to its rest position, instead of adding or mixing with the animation system.
Scripts must be able to read what Blendshapes an avatar and entity mesh has available Allowing for Custom Blendshapes. Currently no such plans are in the process.
Blendshapes on Clothing, Entities and Attachments should be supported.
Allow Textures on Avatars be defined like they can be on Entities- With Material Entities this is now doable, only regulard attachments cannot have Material Entities, yet.
Pillars of the Standard:
- Bones are used to animate the characters limbs, and define the scale variation of limbs.
- Special-Bones are used to adjust the avatar’s shape.
- Blendshapes animate the face
- Naming conventions are similar if not same in all avatar skeletons.
- Animations and Avatars not using this standard cannot claim to use this standard.
- Scripts use this convention to create interfaces which end users can use to customize their avatar
Baseline Humanoid Avatar Skeleton:
The standard humanoid Skeleton of the avatar should follow HumanIK Skeleton / Mixamo.
This skeleton system will work with the input systems already in place in High Fidelity.
however with some modifications mostly to do with the eyes and additional spine / neck components:
Hips – Must always be present. This is checked when the avatar is loaded up.
- – LeftFoot
- – - LeftToe*
- – RightFoot
- – - RightToe*
- – Spine2
- – -- Neck
- – -- - Head
- – -- – LeftEye
- – -- – RightEye
- – -- – HeadTop
- – -- LeftShoulder
- – -- - LeftArm
- – -- – LeftForeArm
- – -- – - LeftHand
- – -- – -- LeftHandThumb1*
- – -- – -- - LeftHandThumb2*
- – -- – -- – LeftHandThumb3*
- – -- – -- – - LeftHandThumb4*
- – -- – -- LeftHandIndex1
- – -- – -- - LeftHandIndex2
- – -- – -- – LeftHandIndex3
- – -- – -- – - LeftHandIndex4
- – -- – -- LeftHandMiddle1*
- – -- – — LeftHandMiddle2*
- – -- – -- – LeftHandMiddle3*
- – -- – -- – - LeftHandMiddle4*
- – -- – -- LeftHandRing1*
- – -- – -- - LeftHandRing2*
- – -- – -- – LeftHandRing3*
- – -- – -- – - LeftHandRing4*
- – -- – -- LeftHandPinky1*
- – -- – -- - LeftHandPinky2*
- – -- – -- – LeftHandPinky3*
- – -- – -- – - LeftHandPinky4*
- – -- - RightShoulder - same as left, but prefixed with Right
(68 standard bone nodes)
NOTE: Finger#4 is always at the tip of the Finger. Finger#1 is NOT the metacarpal, instead its the first joint between the Proximal and Intermediate
The above standard skeletal structure is for bipedal plantigrade humanoids, but it should also work with quadrupedal stances and digitigrade (as if walking on high heels foot elonged to the toes), and unguligrade (as if angled to toes, or same as digigrade without the to bone) foot structures, especially as the procedural animation scripts for walk scripts get better:
Do expect that animations following different skeletal shapes to make you move unnaturally if the animation is designed for a different foot structure, but you may always mask out foot and toe animations for better consistency.
Following this standard will allow end users to use their input devices to control their avatars arms and fingers (if they have any),
Bones marked with an # and * are optional. Bones marked with # should NOT be animated as they are used for customization of the avatar (eye width). Unfortunately, You can a only have 0 to 2 rotating eyes, so having a multiple eyes using this standard is not possible. (I Suggest engine should support for Eye# + LeftEye and RightEye which would work same as the current eye bones (look at targets)
If The avatar’s eye distance is adjustable, The avatar skeleton should have an “eye socket” bone to allow for unified bone translation and scaled for the position of the eyes and eyelids, without the eye bones affecting the eyelids them selves.
Animations should only effect the ones defined in the baseline standard
If making only the head of an avatar, you may start Neck from but must have a root before it.
The standard must be extendable, so custom bones are allowed: just note that for most of them you must create your own animations if deviating:
If using custom bones in the avatar, make try to keep the bone amount less than 80 to save on transmission costs. Each bone animated will cost 1 KBps per transmission of either animation or script. Technically speaking you could still make a multi legged insect or winged feral dragon as an avatar, since you can simply remove the fingers and add more custom bones. Be creative! but do remember that this would deviate from the humanoid skeleton standard.
Custom Bones naming conventions:
We will adhere to a Capitalize naming convention so <Side or #> like with the HumanIK Skeleton.
_End is reserved for generated bone ends.
If you avatar has rotatable ears
or if you avatar has multiple tails
Tail1 - First bone of the first tail Tail# - # bone from the first tail #Tail1 - First bone of # Tail #TailX - X bone from the # tail
Sometimes you may to have a custom script do something with an a custom avatar’s bone, we would then describe the bone as
where as action would be the name of the script. In this case, the lowercase “my” would be the what ever unique word you want to use.
These are all bones that should not be animated by an animator.
However there should be a few reserved bone action names:
morph which should be available in avatars that want to follow a standard:
Prefix “sim” would be for physics simulated bones: Forexample, clothing, hair, tails.
Forexample A full Cape that surrounds the avatar.
simBackCape1 - first bone of the cape, center back. simBackCape# - # bone of cape, center back. etc simFrontCape1 - first bone of the cape, center front. simFrontCape# - # bone of cape, center front. etc simLeftCape1 - first bone of the cape, Left, simLeftCape# - # bone of cape, Left, simRightCape1 - first bone of the cape, Right simRightCape# - # bone of cape, Right
or a simulated tails (instead of animated through animations)
simTail1 simTail# sim#Tail1 sim#Tail#
This is a new ademendum to this standard added on 2017.
Bones currently are completely adjustable via script, however there are some caveats that have been added to the Pillars.
These will be bones which the user should be able to adjust the position of, and as they are adjusted, allowing for customization of the avatar. They should always be parented to the nearest parent bone, and following the following convention
Forexample, if I want to have an adjustable nose: I’d use
A script can then detect these and filter them accordingly and allow for an interface that the user can use to adjust their looks. More details comming
(This part is added on May 14 2017. If it still hasnt been updated, bug @Menithal… he gets easilly distracted.)
Avatar Supported Blendshapes Reference:
Unlike Bones, _ usage will be quite extensive due to the nature of FaceShift standard.
Convention is: L = Left, R = Right. Exception is Jaw. (not sure where its comming from but just go with it)
EyeBlink_L: Blinking action for the Left eye
JawOpen: Opening the Jaw
These are what are fully used during the automated voice cycles.
BrowsU_C: Center of the Brow going up
BrowsU_L: Corner of the brow up
MouthSmile_L: Corner of the Mouth up to a smile
LipsFunnel: Funneling of the lips, similar to pucker, but open. Basically O.
LipsUpperClose: Basically MMMmm shape. but for the upper lips.
They are randomly assigned a noise value as you talk depending on your volume.
Free floating Morphs / Facetracking morphs
Using these for custom morphs other than intended is relatively fine as of the until Pillar #6 is fulfilled, but notice that doing using them for anything else than as specified, is not following the standard to its fullest intent atleast with Blendshapes. Supporting these means your avatar supports facetracking.
BrowsD_L: Corner of the brow going down
JawFwd: Jaw being pushed forward
JawLeft: Jaw being rotated to the left slightly
JawChew: The act of clenching the jaw at the end of a chew
MouthFrown_L: Corner of the mouth down to a frown,
MouthDimple_L: Cheek inward
Sneer: Act of Sneering
Puff: Puffing out the cheeks
LipsPucker: Puckering of the lips
MouthLeft: Mouth oriented to the left side
LipsStretch_L: Stretching of the lip on the left side
Free floating, Subtle Detail / Microexpressions:
These are blendshapes that not everyone will notice, but lend to character if done in expressive or subtle way (note some of the images may be reverse of, just use the correct side ) If Tracking would be enabled.
Again, Using these for custom morphs other than intended is relatively fine until Pillar #6 is fulfilled, but notice that doing using them for anything else than as specified, is not following the standard to its fullest intent atleast with Blendshapes. Supporting these means your avatar supports facetracking.
Otherwise, these are just additional Blendshape slots that can be used for something else. Again, use for something else entirely at your own risk.
CheekSquint_L: Cheek in a state of a squint
EyeSquint_L: Eye Lid in a squint
EyeDown_L: Position of the eyelids when the eye is looking down
EyeIn_L: Shape of the eyelid when an eye is looking towards the nose
EyeOut_L: Shape of the eyelid when an eye is looking out of face
EyeUp_L: Shape of the eyelid when the eye is looking up
LipsUpperClose: Upper Lips rolled in
LipsLowerClose: Lower Lips rolled in
LipsUpperUp: Upper Lips rolled out
LipsLowerDown: Lower Lips rolled out
LipsUpperOpen: Upper Lips opened up slightly
ChinLowerRaise: Lower Chin Slightly up
Avatar Customization Blendshapes:
####DEPRECATED in favor of bone customization as it is taking too long for custom blendshapes to be implemented. See Thread History for detail. You can however use any of the freefloating morphs for your own projects.
Optimization and Filesize
Content creators will have limited bandwidth on servers (read small print on any unlimited host plans) so optimization is important, for both the end users and content creators. The more polygons and larger textures you use, the more bandwidth you are using from your servers per load. Optimally Best to keep Avatar models under 20 mb.
You can only have a single UV map per mesh in the object.
Make sure you fill as much as possible of the textures without too much distortion.
If you can repeat textures (such as for cloths, etc) do so. All of these help the size of the avatar, and allows you to push more detail via polygons or textures.
Compatibility between avatars and textures will be really difficult if not impossible, however the rule would be to try to keep total size of all the textures per avatar below 8 mb. They should be always smaller than 2048 squared, unless all the textures are in a single file. If using multiple texture files, then smaller the better, especially if you can make the textures smaller. Remember that you can get a lot more detail through roughness and bumpmapping, than just textures. It is suggested that you keep Albedo at a smaller size than your roughness for best detail through light reflection instead of color variation.
Avatar LOD Polygon targets:
Depricated, as of the moment, both avatars and entities do not use these information and LOD hasnt been in focus for a long time
High LOD < 30k polygons.
Medium LOD < 15k polygons.
Low LOD < 4k polygons
Lowest < 2k polygons
Generally a good rule of thumb for LOD levels would be a
Until LOD models are re-implemented into high fidelity, that is, target ones avatar between High and Medium LOD, but the lower the better.
to be continued…