Faceshift question


#1

Question for a faceshift guru @ozan @ryan

If you export a fbx from faceshift training-inspect tracking profile-export-export rig as fbx
then load said fbx into the place where you match up all the blendshapes and bones
it seems to work fine in faceshift
when i try to load it back into hf using the fbx the head moves but the face doesn’t faceshift.
I had a look in maya (not my strong suit) rigged fbx’s wont import into blender
it appears to be rigged weighted with all its blendshapes etc in maya

its in the heads as "dhead"
one i manually rigged from the obj route of the same method works and is dh in the head list
and ideas?


#2

Hi Judas,

I took a look at the files and you’re close. The mesh is looking good and it contains most of the blendshapes. I notice a couple of issues however,

  • The dhead.fst file has some unexpected data in it. I was also unable to load it in Faceshift. You need to be sure to upload the model via the Interface > File > Upload Head Model. Otherwise, it might introduce some funky data.
  • The blendshapes were partially mapped. Not sure how that was done.

I took a pass at fixing the blendshapes in Maya and then remapping them to the rig in Faceshift. I’ve uploaded the rig to interface as JudasHead_Test. Try it out. You should see the mouth open and eye blink. You can also use the .fst file as an example.

Also, here’s a snapshot of a typically workflow to create the head. I think you know most of this:

  • Create a head mesh rig with blendshapes, either by exporting the mesh from Faceshift or by creating the mesh manually in a 3rd party app such as Maya.
  • Load the head mesh rig into Faceshift via Tracking > Display > Target > Import > Load Target
  • Map the shapes via the “Blendshapes” section in the dialog box.
  • Save the mapping, typically in the same directory as the head mesh FBX file.
  • Go to Interface and Upload the head model via File > Upload Head Model.
  • Fill-out the dialog box and and upload.
  • Go to Preferences and specify the head model.

Also, here’s a link to the Faceshift user manual. It might be helpful in learning the details of mapping the rig. However, don’t hesitate to send me any additional questions about getting things up and running.


#3

[quote=“ozan, post:2, topic:788”]
The blendshapes were partially mapped. Not sure how that was done.
[/quote]. I was trying something new, I hadn’t realized that faceshift will export the complete head rigged with blendshapes
and was looking to see if theirs a simple, route to creating a personalized head achievable using pretty much only faceshift
the fbx and fst were entirely from faceshift.
What was required to fix them?
this fbx and fst load fine in faceshift
https://dl.dropboxusercontent.com/u/10483952/New%20folder.zip
im interested to know what its missing to work in hf


#4

Ah. I see. That’s a good idea for creating a personalized head.

You can also try right clicking the Tracking > Scan Area and choosing Export Fit. See image.

That way, you can export the personalized neutral scan and all the expressions, then construct the blendshapes in something like Maya.

Regarding the FST fix, I’m not sure what fixed it or why it wouldn’t load but rebuilding the blendshape node in Maya and remapping in Faceshift resolved the issue.

Ozan


#5

interesting @ozan thanks for your help. Lots of hidden things in this prog very cool


#6

@ozan ok stepforward

I can now export a fbx from faceshift the one it creates when you teach it to recognise your face.
Then import that fbx using a slightly tweaked fst file directly into hf
and have a custom head that will faceshift really well.

This is good because it will let anyone use the software to make their own head with virtually no technical know how

But and its a fairly big but.
The eyes and the teeth which are both visible in faceshift vanish on import and the head is untextured.
Faceshift does actually create a face texture which you pointed out in your above post.

Possibly the eyes and teeth have flipped normals or maybe their using nurbs.I cant seem to find away to get faceshift to spit out the head with the textures it will apply in software( yes their a bit shit but it would be a start)

I think if we can find away around these problems we could have something really cool


#7

This is great, Judas. Really good work.

A couple of notes:

  • When Faceshift exports the fitted mesh, it excludes the teeth and eyes. So that part makes sense.
  • Faceshift does export the texture but does not automatically attach the material to the mesh which may explain why you’re not seeing it in HF.

Here’s how I resolved the issue:

  • Export the fitted mesh, as you described and be sure to include the texture in the export options. I selected the “accumulated texture” option.
  • Import the mesh into a modeling app like Maya.
  • Add teeth and eye models from a 3rd party source, like Turbosquid, etc.
  • Attach the texture to the mesh.
  • Convert the mesh to an FST rig, they we already do.
  • Upload to HF.

This should get you a little further along. I agree, once we nail this workflow, we’ll have something really cool. Let me know how it goes.

Ozan


#8

Oh here’s me stuck and my problem lol
if anyone wants to take over this that would be great

If we all had the ability to create a head this way, we would have the option to create standardized heads
that if the ability to apply a texture in world was added we could make and sell skins for
and that we could all make and sell hair for.