Fixing Problems

        I have not yet been able to find the difference between IK and FK re-targeting inside of Unity. I have not been able to find it in the documentation so far. If you know, please let me know where I can find it.

        Earlier, I tried editing the root transforms of my MoCap rigs in Blender. It failed miserably, the animations broke upon import into Unity.

        To fix the issue where the character rig was moving around inside of the character controller, and not staying centered in the view, I had to un-check apply root motion in the inspector. I will re-check it once I have all animations completely zeroed out at the origin.

        To create my states and blend trees so far, I used a combination of animations from Mixamo, Crysis, the Unity Asset Store, and my own motion capture. Any animations that had to be purchased, I did so months ago and they have been sitting on my computer, unused--until now. As you can see, the locomotion and camera issues from last week have been fixed. The character no longer moves around off-center with the camera. I used Unity IK to bind the left hand to the rifle. The character can run and sprint in all directions as well as aim and strafe when the player presses the left mouse button. The aiming is off-center, but that will be resolved once I setup aiming controls.

        As you may have seen, the arms flicker when I press the melee button. I tried setting up a crazy blend tree to blend between 14 different punching animations because I have always like the idea of randomized animations for certain things, especially melee. It has not worked so far, but the problem occurring has nothing to do with the shear number of punching animations. I know this because I tested it with a far simpler tree with only 4 animations, and the same problem persists.

Blend Tree Movement and Aiming

        Right now, I have movement working with a blend tree, blending between four running cycles, and four sprinting cycles. So, my character can run in eight directions by pressing any combination of one or two of the W, A, S, and D keys. If the Shift key is held, my character starts sprinting. Sprinting forward is still too explosive, I will need to slow that down by decreasing the value of the variable in the script. The rotation isn't that great, I will refine that when I go on to building my advanced 3rd-person character controller.

        There is a problem, the character appears to rotate when moving around. I think this is caused by the rifle aiming animation, which plays randomly because I haven't set it up properly yet. I will have to zero out the transforms of that animation in the animation editor. Observe:

        As you can see, the character rotates a bit to the right whenever the aiming animation loops, which, over time, causes the character to turn to the right like we saw earlier. I will also need to edit the position of the character's hands using Unity IK, because the rifle is not being gripped, and is being aimed too high.

Animation Re-targeting and Looping Demonstration in Unity

        The possibilities are endless because I can now re-target any humanoid animation to any character rig in Unity with the click of a couple of buttons. Here, I have re-targeted a somersaulting animation from another character to my character's rig, and set it to be looping.

Original animation:

Re-targeted and looping animation:

Applied to one of my mocap rifle aiming animations:

Going Forward

I have revised last week's plan of action a bit to account for what I accomplished this week, and what I plan to accomplish in the weeks following:

- Week 7: 

 Character's rig completed

 Reviewed re-targeting MoCap animations in Motion Builder

Learned how to use Unity Mecanim for editing and re-targeting animations. Learned how to change
root motion of animations in the inspector upon import, as well as in the Animation editor window.

-Week 8: 


 Bring Animations into Unity, Build state machine and get all animations to play on key press, animation layers, start building character controller, adding functionality, aiming, rotate spine bones (spine, chest, neck, head) to follow target.

Continue working on character controller, adding more functionality. Add Unity IK functionality, troubleshoot problems if need be.

 Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 9: 
Shoot additional Mocap, more animations are needed. Retarget all animations and get them into Unity.
Continue working on character controller, adding more functionality. Add Unity IK functionality, troubleshoot problems if need be.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 10:
Add as much functionality as possible using all Unity Mecanim tools possible.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 11:
Add as much functionality as possible in time remaining using all Unity Mecanim tools possible.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

Motion Builder and Unity Mecanim Retargeting

        Re-targeting and editing animations was going pretty well. It was nice to revisit Motion Builder. Everything was going fine in Motion Builder, until I baked my final animation from my control rig to the skeleton, and the character would be frozen on the last key frame he was on before baking. I don't know if this is a bug, but it can be a fatal error if you have spent hours to refine an animation, only to not be able to bake it back down to the skeleton and export it as an FBX.

        I took a break from MotionBuilder and hit the Unity documentation, learning how to re-target and edit animation clips in Unity. In the inspector, I learned how to retarget animations, make them looping, and edit the root transform. In order to zero out the position of the character, so that the character was moving in-place, at the origin, I had to edit the clip in the Animation editor. Imported animation clips are read-only by default, and there appears to be no way to unlock it. So, I had to create a copy of the mocap clips I wanted to use and edit the key frames of those.

        Unity documentation on looping animation clips in the engine:
https://docs.unity3d.com/540/Documentation/Manual/LoopingAnimationClips.html
https://docs.unity3d.com/Manual/LoopingAnimationClips.html

        This can be very useful, but is ideal for longer animation clips with more possibly matching parts. Otherwise, it can cause weird smooth "hicups" in the looping animation. For example, if you are trying to make a looping animation clip of someone walking or running, and the clip is not long enough, or does not have enough closely resembling parts, it can cause weird things to happen such as the feet sliding on the floor, or stuttering in the clip. Unity has nice tools to help create these looping animations very quickly, such as wire graphs. The wire graphs show if the two ends where you are cutting the clip short at will produce a seamlessly looping animation or not.

        In short, Unity offers the quickest, most efficient solution to re targeting and editing animations for the bare minimum of what one needs. However, if you want to do more, you will have to edit the animations manually in the Animation editor. Motion Builder, however, allows more precise control for editing animations, is more efficient because it allows the animator to use animation layers. Re-targeting in Motion Builder is more tedious, and so is creating looping animations, but you have more control.

Making read-only animations editable in Unity:
https://www.youtube.com/watch?v=cNnnaHqhks0

I still decided to fix my issue in Motion Builder since I may need to use it to create more precisely refined looping animation clips. The solution was very simple, don't delete the MoCap skeleton character until the animation is baked to the other character's skeleton, and the other character is not currently using the control rig, but it's original rig.

This guide to re-targeting in Motion Builder looks like it can be very helpful in the future if I were to get more in-depth into Motion Builder:
http://mocappys.com/retargeting-animation-motionbuilder/#.XHGKu-hKiUk

In the future, I think I will leverage both programs for what they do most effectively. Instead of re-targeting in Motion Builder, I think I will decide to edit the Mocap skeleton's animations in story mode to make them looping, then bring that into Unity and re-target there. However, there are still other nice features that Motion Builder offers. If I need to do something that requires precise control over part(s) of the body, such as closing the fingers to grip an object, like my character's rifle, Motion Builder allows for that. To solve this issue in Unity, I will need to use avatar masks for the hands.

In the future, I foresee that I may never need to use Motion Builder again, and will be able to pretty much do everything with re-targeting animations in Unity.

Unity Mecanim or Autodesk Motion Builder?

        I have determined that Unity's animation retargeting system in Mecanim can be very powerful, but that it is ideal if you already have finished animations. If I were, say, doing a project using primarily or almost entirely animations from the Unity asset store or some other online marketplace or community, then Unity Mecanim alone would be perfect. I would just funnel in the animations from all of these outside sources into my project and character.

        Straight away, it's pretty cool, you just drop your character rig into the project, and it automatically recognizes if if your character is in t-pose. You may need to configure the avatar, but you don't need to go through the tedious process of characterizing your character like in Motion builder. You can also correct your character, rotating the joints to fit t-pose, much like Motion Builder, but that's where the similarities end.

        I found this video training comprehensive overview of Mecanim very helpful, much of it was review, but still made certain things like Blend Trees easier to understand. Much of it was not review.

https://www.youtube.com/watch?v=wf6vtCgLk6w

        Most of what I needed to know was in the first half-hour. At 51:00, he mentions root transform offset and some other helpful things. At 1:15:00, mentions useful State Machine Behavior script components. Damping, 1:21:00.

        I discovered that one can do many interesting things in Mecanim, such as setting up muscle space, which is pretty much limiting the rotations on the character's joints, which can come in very handy if you want to make your character more or less flexible.
        I also hit Unity's documentation this week, exploring different things I can do in Mecanim.

https://docs.unity3d.com/Manual/animeditor-AnimationEvents.html

        While Mecanim is very powerful, editing animations in Mecanim seems to be very manual, setting keys and editing curves in a dope sheet much like Maya, Blender, or some other 3D package. My key-frame animation skills are not very good. In concept, I can manually edit animations in Mecanim. Even so, I feel that that would be very time consuming and inappropriate for this project and out-of-scope, considering the remaining time frame.

        Immediately after shooting MoCap, the animations are not ready to go into the engine. The MoCap rig rarely ever matches up perfectly or adequately with your character rig. So, editing always required, to ensure that the animation is in-place, and that the positioning of the limbs are correct so that there is no clipping. For this project, I need the extra functionality that Motion Builder has because it allows me to edit animations efficiently, via animation layers. The MoCap animations that I have are not finished, they are not in-place, and not yet suitable for use in Unity. For this project, I need to use Motion Builder.

         I can definitely see myself using Unity Mecanim in the future, so I do want to continue familiarizing myself with the capabilities that Mecanim has to offer, as I am very excited now that I have discovered these. I could possibly use it next quarter in New Media Project, or further along in the future, in place of Motion Builder, because Unity is free, Motion Builder is not.

Character Rigging in Maya

So far, this week has been a major comeback.
I have retopologized my character in Blender, and rigged it in Maya. I decided to use Maya for rigging for this project because Blender was giving me issue last time, when I made my own block-out animations and tried to get them working on my blender-rigged character in Unity. The mesh exploded upon the playing of the animations. So, it seems that the bones were scaling and rotating in odd directions for whatever reason.
I found that rigging in Maya isn't so hard. In fact it is much like rigging in Blender. If you set the Bind Skin setting s correctly, Maya can do most of the heavy lifting as far as applying weighting to your model goes. All you have to do is make some adjustments here and there on certain joints by weight painting.
My repopology so far looks a bit crumby. The character is UVd, so anytime I want, I can edit the position of the vertices of my mesh in order to allow for baking the detail from the high-resolution model into texture maps. I mirrored my character's right side to his left after UVing in Blender, so the UVs are directly on top of each other. This allowed me to mirror my weight painting in Maya. All in all, I was able to optimize the rigging of my character so that it took the least amount of time possible. Some of the weight painting is rough, but it will suffice for now.

So far, I am looking into retargeting animations in Unity rather than Motion Builder. The deciding factors will be how much extra functionality does Motion Builder offer? How much of it is needed? Is Unity overall faster and more efficient for what I am doing? Will editing the animations via animation layers in Motion Builder be necessary since I am using Unity IK to tweak the animations anyway? These are the questions I need to answer.

Unity Mecanim Animation Retargeting

I already know how to retarget animations in Motion Builder, so now I will examine Unity, and see if I will utilize it instead. Unity seems like it can be a cost-effective alternative. I will use Unity if it is an effective tool that I can learn to use quickly. I have gathered some tutorials and Unity documentation links.

Tutorials:
https://www.youtube.com/watch?v=dDHltGGDrAA

https://www.youtube.com/watch?v=3TOJt9tntNI

https://www.youtube.com/watch?v=7yGBu97Oej0


Unity Documentation:
https://docs.unity3d.com/Manual/Retargeting.html

Going Forward

- Week 7: 
Tue-Wed--Rigging, Basic rig for retargeting Mocap, skeleton and weight painting only.

Wed-Thurs-- MoCap Retargeting, will decide to use Motion Builder or Unity. Already know how to use Motion Builder, but Unity is free, so it would be beneficial to learn if there is time this quarter.

Thurs-Fri--Bring Animations into Unity, Build state machine and get all animations to play on key press, animation layers, start building character controller, adding functionality, aiming, rotate spine bones (spine, chest, neck, head) to follow target.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 8: 
Continue working on character controller, adding more functionality. Add Unity IK functionality, troubleshoot problems if need be.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 9: 
Shoot additional Mocap, more animations are needed. Retarget all animations and get them into Unity.
Continue working on character controller, adding more functionality. Add Unity IK functionality, troubleshoot problems if need be.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 10:
Add as much functionality as possible using all Unity Mecanim tools possible.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

-Week 11:
Add as much functionality as possible in time remaining using all Unity Mecanim tools possible.

Fri-Sat--Master's thesis research, read articles, find new ones, post to blog. What am I doing for my master's thesis? Why? How will this advance the given field?

        This week has been a battle, I have been set back considerably. I have not accomplished what I set out to finish this week. My super-soldier character has been completely modeled, but still needs to be rigged.
Aside from my other classes, one of the reasons why I have been set back is because I was sculpting my character to begin with, then realized that sculpting is taking way too long, that it will take days, or even another week to smooth out my character's armor plating. I thought that I should capture the basic form by making a high-poly retopology, then a low-poly retopology, that this would be a far more efficient way to get rid of the bumpy/wobbly surfaces on my character's mesh. In concept it seemed like a great idea, much more efficient, but it was not. I underestimated the amount of time it would take to do such a high-poly retopology, working with so many edge loops. Perhaps it was such a great idea, but was not executed properly. Maybe if I was not affraid to add more geometry early on, I would have ended up with a high-poly mesh where all of the edge loops connected properly, all quads, in a short amount of time. This is why I mentioned a new character creation workflow in the previous post. One lesson learned is that if you are going to take the time to detail sculpt a character, do it all the way. Or, figure out if you are going to be sculpting or subdiv modeling, and stick to it. If you don't spend too much time sculpting, just a sketch, getting the basic forms in then my proposed new workflow can be a great idea, I think.

        Proposed character workflow for future reference:

1. Blockout model/basemesh, "sketch" sculpt, or both consecutively.

2. High-poly retopology

3. Detail sculpt

4. Low-poly retopology

        Sculpting a character from scratch takes too long. This workflow could work, if the artist does not spend too much time detailing or refining in any one step.

My super-soldier character modeled:

My super-soldier character modeled (subdivided):

This project is more about functionality than pleasing aesthetics, so my super-soldier will not be fully textured (I will, however, bake out a normal map, since I went through the trouble to make LP and HP versions, retopologizing and all). He will have a normal map and flat colors applied.
After he is rigged, I will take him into Motion Builder to retarget my MoCap animations, then Unity Mecanim, and proceed to build his functionality as demonstrated by his animations.

I am still trucking along with creating my character. I expect he will be completed and rigged by tomorrow. Then, I will have to stop for this week. I can't wait to get back to working with Mo-Cap animations and Unity IK. I am eager to see how my character's rig works in Motion Builder when it is completed, but first things first.

Every time I create a character, I get more efficient at it, and my process becomes less haphazard. This character is a more difficult one to complete, as he is a super-soldier character with complicated organic armor plating, and this is one of my less intricate designs.

I am going to create a new 3D character creation workflow for myself, one that is suitable for a generalist. One that doesn't rely too heavily on hard surface modeling or sculpting, but utilizes both appropriately. The stages progress as so: blockout model, "sketch" sculpting, high-poly re-topology, detail sculpt, and low-poly re-topology. I will elaborate more on these in a later post.

I am proposing this type of workflow because as someone who is not a master sculptor, I have been bogged down trying to get evenly smooth surfaces in my character's armor. Many of my super-soldier's armor surfaces were very bumpy and wobbly, and I realized that this happened because there wasn't a clear separation between blockout sculpting and detail sculpting. I could tell that it would take hours, or even days to merely smooth out the sculpted armor surfaces, and even then they would not be perfect. Of course, the blockout model and high poly re-topology are not always necessary, but can be helpful steps.

So, I did a high-poly re-topology to effectively get rid of all bumpy surfaces, but I still will need to retopologize a 2nd time to bake the detail to a normal map.

Elevator Door-Closing Pitch

I have made an elevator pitch to concisely sum up why I am making this blog:

I wish to use modern game engines to correct animations and create character controllers that react more intuitively, contextually, believably, and ultimately realistically.

Ok, that is a bit too verbose, but I am cutting it down.

The creation of my character is still underway, I am closer to completing the high-poly model, then I will do the re-topology.

Unreal IK Tutorials

I looked at Unreal IK and have taken note of some tutorials:

https://www.youtube.com/watch?v=gQyIOe96IhY

https://www.youtube.com/watch?v=hmguNok7xWs

https://www.youtube.com/watch?v=rBPW7fVxEDA

https://www.youtube.com/watch?v=FbyvA71uvUY

This is out of scope of this project, but good to note for future reference.

Unreal IK Documentation

I have taken the tie to look at Unreal IK documentation:

https://docs.unrealengine.com/en-us/Engine/Animation/IKSetups

https://docs.unrealengine.com/en-us/Engine/Animation/NodeReference/SkeletalControls/TwoBoneIK

This is out of scope, but has similar concepts and may come in handy in a future project.

I am still working on a retopology my character. This is taking much longer than expected. All of the pieces are there, they just need to be connected.

Subdivided, it looks different. I can always add more subdivisions, of course, to get a higher resolution model, before I bake textures to a lower resolution model. 

This is unorthodox, and not apart of the plan, but in the future, I will spend minimal time sculpting, treating it as a sketch, only to block out the basic forms. Then, I will do the high-poly retopo, then the low-poly retopo. This type of process also allows me to do detail sculpting on the HP retopo, before doing the LP retopo.

Or, I will have to be wiser about how I sculpt characters, so that I don't get into a situation again where it will take days to smooth the sculpt. 

Yesterday, I attempted to MoCap myself. It failed, but I know I will do it correctly next time. A problem was occurring where I got the range of motion take calibrated perfectly, but when I would import the vss and vsk skeleton files into another scene and walk into the capture area, the skeletons would follow my markers, but the left arm would stay frozen in position and not move. 

My character's high-poly retopology is still underway, but I will have it completed soon.

Still working on character. I will be capturing more MoCap tomorrow.

Finalizing this character's sculpt has been troublesome. I am having trouble getting the surfaces to be smooth. This is becoming worrisome because I need to get this character completed so that I can move on to the other steps.

I am going to try retopologizing twice, once over the sculpt, in place of the sculpt, in order to capture the geometry detail. The second time to produce my low-poly mesh which the textures will be baked to.

Gyration Air Mouse

I recently ordered a Gyration air mouse. I tested it, it works. Using this, I will attempt to MoCap myself soon in order to get the remaining animations I need for my character. I am still going to be scheduling time with my MoCap actors for the soonest date possible.

My super-soldier character sculpt is still rough around the edges, but it is getting close to completion.

My super soldier character sculpt in-progress in Blender. It is still rough around the edges, quite literally.
After I finish sculpting and retopologize, I will rig the character in Maya, then re-target my motion capture animations in Motion Builder. I will not be re-rigging in Blender, because Blender rigs do not appear to be compatible with Motion Builder. It results in incorrect rotations which destroy the motion of the mocap take.
After I am done rigging, I will build a character controller in Unity and utilize Unity IK and other Mecanim features where appropriate.