Describe the process of adding new skins

[sskorol]

Hi @noeperez @roberottt,

I'm trying to add a new model to a simulation. Here are the steps I've completed so far:

• built it via MakeHuman app
• animated in Mixamo
• exported to .dae and .bvh via Blender and put them to models folder
• added new behaviors to Agent.msg
• extended skin_filename and animation_filename with newly added items
• added hunav_msgs handlers for the newly added behaviors

When I start all the services via launch file, I see the following error:

As it comes from Gazebo, I'm not really sure how exactly I should fix it. Where are these active/inactive animations are set?

Would be greatly appreciated any thoughts.

[noeperez]

Hi!
To create new agent models is not trivial. We struggled a bit to achieve it, but we did it.
We will update the wrapper with a set of new models soon (man, women, children, ..). They have a better look than the default bald agents.
We will also upload a guide of the steps that we followed to create them. (We are not expert in modelling, so probably an expert in this field could reach a better and easier way).

[sskorol]

Thanks! In the meantime, can you advise on the animation part for active/no_active behavior? As it seems like I'm pretty close to the solution, just want to understand the direction.

[noeperez]

Sure!
We use the "active" state to define the animation that the agent should perform while reacting to the presence of the robot.
The "no_active" is the animation while walking without the presence of the robot.

[sskorol]

But where in the code do you map these 2 states with the animation? I'm a little bit confused with the above error message as I don't see how these states correlate to animation files (.bvh) that you explicitly load in WorldGenerator.

There were similar errors related to the actual behaviors that we define in agents.xml. And I fixed them by introducing my own behaviors for a new model. But how can I do the same for the active/no_active state (which is mentioned in the error message)?

[sskorol]

To be more specific, I just added new .dae and .bvh files:

Plus corresponding processing logic:

And new behaviors:

[noeperez]

To replace an animation is not complex, while the animation is compatible with the skeleton of the model (related to your first problem).
If you also want to add a new behavior, that is not trivial, and more work is required since the behaviors are controlled by the behaviors trees. You need to program a new behavior tree and register it.

[sskorol]

But I generated both .dae and .bvh from a single .fbx. So animation should 100% match the .dae, as it works in Blender and the other tools. So I guess there's something more on the code level where some default animation is used with my model (which is incompatible with the existing animations).

[sskorol]

No, I'm stuck. Seems like there should be some special model/animation configuration to play well with Gazebo. There are a bunch of ways how you can export data from Blender. And all the tutorials are pretty much outdated. I don't know why it doesn't work. Looking forward to receiving some instructions from you. Thanks.

[noeperez]

Building a functional model with animation to play in Gazebo properly is quite tricky.
Again, as I commented in the first response, we will upload new models and to try to provide a guide of the steps we followed.

[sskorol]

Thanks, do you have any raw estimate? So that I'd understand if I want to spend more time on investigation or just wait for your update. You know, if it's a week, for example, I can wait for sure. If a month, I'd likely find a solution myself.

Note that I perfectly realize it's open-source and you have your primary job. So that's just a question, no offence.

[noeperez]

Probably, we could upload something next week.

[noeperez]

We have just added 5 new human models.
We will provide the steps we followed to create them next week.

[sskorol]

@noeperez top! Will try later today. Thanks!

[sskorol]

New models work for me. I'm looking forward to getting the instructions for custom models. Thanks!

[roberottt]

Hi!
We have just added the instructions for custom models.
You can find it here.

[sskorol]

@roberottt excellent, thanks!

I'm wondering if you also tried to change the animation. I made many attempts, but it seems like Blender exports a broken .bvh file, which can't be correctly handled by Gazebo. Or maybe I just missed something.

[noeperez]

Hi!

The models built like that are compatible with all the .bvh animations from here. We just associated the proper animations, for the "active" and "non-active" modes, to the behavior indicated.

[sskorol]

@noeperez yes, I know that dataset. However, I'm asking if you tried some custom animations built with e.g. Mixamo? It's a very simple and powerful UI tool. You can import your .fbx from MakeHuman, add any animation, then export back to .fbx and import it to Blender with a fully working animation. What I'm stuck at within this process is splitting .fbx onto .dae and .bvh. If I use only .dae, it seems to work well in Gazebo w/o animation. But when I try to export .bvh, it breaks the model: there's a monster from a horror movie walking in Gazebo. :)
So wondering if you'd have more luck than me with this process. As it should be a lot simpler. I likely missed the valid export options for the target .bvh.

[noeperez]

Sorry, we did not try with other animations or building custom animations.

[sskorol]

I tried to follow the instructions but got a broken model at the end. Have you tried adding such parts as ponytails, eyelashes, and eyebrows or changing the size of the body? It seems like walk.dae has a bit different skeleton compared to what MakeHuman produces. At least I see some significant diffs in some bones' positions. I don't know if that's the reason, but if I just scale a skeleton and leave everything else as is, some parts of the body might not be aligned or be even split.

[noeperez]

We have the same problems. When merging the skeleton of the walk.dae with the skin of a new model not always fits perfectly, and some bones positions and connections must be adjusted manually.
As we say in the document, the 3D modelling is not our field, so that's the best we have achieved for the moment.

[sskorol]

Anyway, thanks for your effort!