Demo Video Presentation

This is my presentation, showing off my project thus far, all of the state machines, briefly going over how I utilized Unity IK to create all of the functionality in my character controller in this project:

Part 01

Part 02

Part 03

Part 04

Part 05

AIM IK Fix and Hand to Wall IK

I have made many adjustments since my last post. I fixed the foot turning animation on the lower half of the body, it now only plays when player is in idle, standing still, and looking left and right. I fixed aiming, it only activates when the player is shooting, to avoid the awful sideways over-bending of the body, and the twisting and contorting that was occurring earlier. The Aim IK weight is set to 0, unless player is firing and strafing. Next, I added a muzzle flash effect from the asset store, to highlight the presence of the character's weapon in 3rd person view, since it isn't easily visible. I added a coroutine in the script to give short delay so that player character isn't firing before the weapon is fully raised.

Finally, I have successfully built the functionality to get the character to detect and touch a wall on his left with his offhand using a raycast and IK! There is still a minor problem, since the hand doesn't make direct contact with the wall. Still, this lays the foundation for all other wall-touching actions on my task list such as creating friction while falling and ledge grabbing. However, I don't think I will get to ledge grabbing, that'll be a bit tricky, but friction while falling is worth a try! This may also pave the way for me to troubleshoot foot planting, since they both use raycasts and animation curves to blend between IK and an animation.

Above, we can see the raycast detecting the wall and causing the animation to play. On the right, we can see the original animation in the inspector, and a curve above it. This curve is what sets the weight for blending between Unity IK and the original animation.

Aiming System--Unity IK

Aiming is almost completely implemented, but is acting strange because of how the character controller and look IK conflict when they control the character. Foot planting is taking a hiatus--for now.

We can also see that turning animations play on the lower half of the body if the character is turning. This acts strangely if the character is moving around. If I keep this feature, then I will script the necessary lines to prevent that. See the extra layer added for lower body, and it's corresponding weight.

Here is the updated IK Control script in the inspector, showing the parameters that have been added for controlling the look IK.



I will have to play with these in the script, or the character controller itself in order to resolve the issues.

Foot Planting

I have been working on getting foot planting working. I have come much further since my last attempt at IK foot planting, but I am still not quite there yet. Before, I could not get it working. Then I got some jittery collision with the ground. Now, the feet are stuck down at all times, even when airborne, which is not desirable either.

Bug Fixing

So, I finished modifying the character controller code, reverting back to the original method I was using to control the character and camera, and the problem still persists. It appears to be controlling some of the IK handles in local space, and others in world space, which is very problematic.
I checked the documentation, and found that the function for moving an IK handle in space, "Animator.SetIKPosition" only works in world space by default, and there seems to be no way in the function to set it to local space. Even though I had parented the IK handles, they were still moving around in world space, making it impossible to control my character's hands' positions relative to the character. I noticed that the only IK handle that was cooperating was the left hand grab handle, located deep in the hierarchy, parented to the rifle. I have no idea why this is the case. If parenting causes IK handles to work in local space, then they should, no matter how dep in the character's hierarchy they are parented. I have parented all of the hand IK locator objects to the rifle in the character's right hand, which may produce better results anyway.

It did not achieve better results, because they are parented beneath the root bones in the hierarchy, and therefore, they don't aim directly to the center of the screen, but rather wherever the character is facing in the animation, which causes the character to aim to the right whenever I press the fire button, instead the center of the screen, which is not what I want.

I re-parented the right hand IK locator objects, which control the shooting, to the character rig, and somehow it is miraculously working. The character's hands and rifle now stay where they are supposed to be relative to the character's position/rotation in space. I have also successfully rewritten my IK Control script.

So, I am back where I was before with the hands instantaneously moving to the positions of the IK handles. The coroutine waitforseconds values are also the same as before, to try and fake smoothness. I tried multiple methods to translate the handles or to somehow blend the weights, none of them worked. So, now it is time to move on and build other features. As you may notice, I also have a test environment built using Sabre CSG, a free Unity Asset Store plugin that works similar to Unreal's built-in level builder tool, using boolean tools I will build on this test environment as needed and use it to rest my character's abilities. We will see more of it later:

Here is the new IK Control script:

Bug Fixing

I realized that in the Animator window, clicking on the animation in question and ticking the "Foot IK" checkbox helps eliminate foot sliding during animations. Of course, there is still some foot sliding during the transitions, but that is unavoidable.

I rewrote my IK Solver, it still didn't work. For some freak reason, the IK locators are not obeying parenting conventions all of a sudden. This can be seen in the previous post where I show that instead of punching or aiming in front of himself, the character is punching in different directions as I move him around, because the IK locators seem to be in world space, rather than local space, relative to the character.

It may be related to how the character is picking up the camera's rotation whenever the W or left mouse keys are pressed. However, this does not make since, since the IK locators are parented to the character's rig, beneath the object that is rotating with the direction the camera is facing. I have to find a way to automate all of the IK handle's positions, or I may have to abandon IK for the remainder of this project. If I do the latter, I will later evaluate how to properly use Unity IK, and just focus on getting as many animations in as possible and using animation layers.

Another option is to keep the parenting of the camera and character controller and all components, but revert to how the camera and character rotation was originally being controlled (in the scripts). Right now, the camera rotates around the character, and the character only faces where the camera is if being moved with one of the W,S,A,or D keys, or when the left mouse button is pressed. Originally, the mouse was controlling the rotation of the character on the Y axis (horizontal).

I feel like I am finally getting to the point where I am grasping and understanding Unity IK, that is, how the IK C# functions work, what they are doing, and how to use them. However, there are still things that I do not understand.

Bug Fixing

I have fixed most of the bugs that I was having earlier.

Re-parenting the camera and character controller components again worked as I had predicted. All components are now on the topmost object in the hierarchy, Hunter Transform.

I have fixed the root rotation issue where the character was rotating downward  with the camera. The way I fixed it is setting the camera object in the movement controller as the camera pivot null object that the camera is a child of instead of the camera itself. I did this because the camera pivot is  not pointing downward, but has all of it's rotations zeroed so that it is aligned with the floor plane.

I am still having a problem with the IK, but worse. Now, the IK is completely broken, and I have no idea why. The aim IK keeps flickering on and off, creating a jittery motion. When I move the character around in play mode, the IK locator objects seem to be moving in world space so that the character does not punch in the correct direction. However, this should not be happening since they are parented to the character and not in world space. Observe:

This is a very perplexing and confounding issue. I really hope that I do not need to re-setup the null objects, or rewrite the IK script.

State Machines

Here are all of my state machines.

This one is for the base layer, for locomotion. Aside from a bunch of unused states floating around, we can see two blend trees, and a bunch of other states. These two blend trees handle character movement in all directions, and character movement while shooting. They then transition into jumping, which begins with an initial takeoff animation, into a midair loop, and finally the landing animations. The midair jump can also transition into a shooting midair looping animation if the player decides to fire while airborne. The landing animations then transition back into the idle and movement animations.

Here is a closer look at the Blend Tree for the idle state, base character locomotion, and sprinting. The one for strafing is just like it, except it has the animations for strafing while shooting instead.

Next, is the layer, masking over the upper body which controls aiming the rifle:

Blend Tree for upper body movement during locomotion:

Blend Tree for rifle aiming and firing, blends vertically between aiming up and down:

Last, is a layer that masks over the hands, keeping them gripping the rifle. This will be expanded to include the different hand motions such as punching and grabbing:

This Week's Progress in Unity

This week I created a camera controller, re-parenting several components of my character game object's hierarchy in the scene:

Earlier, the camera was parented inside of the character, and all of the components including scripts were on the outermost object in the hierarchy. Now, there is a parent null object underneath which the character rig and camera reside in two separate game objects. The camera is now parented underneath a null object resting at the pivot point of the character, (0, 0, 0) in local space.

It's nice that my character now has a decent camera controller, however, his functionality is now suffering from a few bugs. First, the character rotates downward after moving. This is because I set the characters transform.rotation to be equal with that of the camera, and the camera is looking downward. I will have to find a way to rotate the character back up so on the x axis so that its forward vector is parallel with that of the world's.   

The second, I think, is due to the re-parenting I did of the camera and character controller, because the colliders are no longer working, so the landing animations no longer play. I will re-parent the camera, and apply the animator component and all other scripts and components to the top/root level of the hierarchy.

The last issue I have to fix is that the IK aiming is also malfunctioning, flickering instead of staying constant. The left hand is no longer moving to the position of the ik handle on the rifle. I don't know why this is taking place, but the issue started when I re-parented the camera controller object, if that is any clue. 

I have started to build the functionality for the character's melee system. There is a null object that the left-hand IK handle snaps to when it is toggled active. Right now it's a bit choppy, but I will use this to give my character the ability to strike at targets, his fist automatically moving to contact. I will need to figure out how to achieve smooth blending from null objects to IK handles for all Unity IK functionality in order for the animations to be believable.

Further Work on Aiming and Jumping

Here, I demonstrate my jumping mechanic and rifle aiming. Both are still in progress, this is an update to show the advancements that have been made so far.

I have used Unity IK to correct the position of the characters hands and rifle while aiming, so that he aims completely forward, in the direction of the camera. Some additional work is required because his hands freeze in that position, which looks awkward, and his torso also needs to be rotated to match where the rifle is aiming. There is also no rotation in the Y axis, and no animation for when he is idle and turning left and right with the camera. I will fix those issues as well to create a more animated character.

I show here my character in a test environment, downloaded from the Unity Asset store. This is just to have scale reference, and to also be able to see how fast he is moving, how high he is jumping, etc. This is where I demonstrate my jumping mechanic so far. He now falls faster and can jump higher or lower depending on how long the player holds the space bar. I still plan on adding two more types of landing, the hard landing, and rolling. I will also add jumping in all directions, because right now he only jumps forward. In addition, a shooting midair idle animation that can be transitioned to while the player is somersaulting through the air is needed.

Week 09 Progress

As you may see, my rifle fast mobile animations have been corrected, so that the rifle butt no longer clips through the character's chest while running.

My character now has jump animations. There are three animations that contribute to this jump effect total. There is an initial jump takeoff, which is hard to see at first glance, but it is there. It will take some more testing to get the transition time right. It took a bit of finagling with the speed of the animation and the transition time to get it visible. In most games, jumping is instantaneous. In others it is not, and there is an initial takeoff animation, which delays the time to jump after the key is pressed. This was difficult because I was trying to achieve the best of both effects, an instantaneous jump with a takeoff animation. Next, is the somersault/hover animation. This can be any animation that evokes free-fall or jumping in the air. For now, I chose to do a somersault. Lastly, is the landing animation, which is only the soft landing at the moment. I wish to expand this to three different types of landing: soft, hard, and rolling. The soft landing animation will play when the distance the character is falling is not too high. When the character falls from higher up, the hard landing animation will play. Finally, if the player presses the Ctrl key before impact, the rolling animation will play. 

Aiming is being corrected with Unity IK. I added to my IK solving script, which now controls the right hand. When the left mouse button is clicked, the character raises his rifle to aim directly in front of him. I plan to add to this my making the mouse control the rotation of the character's spine bone. I will also set up a blend tree for aiming up and down, and animations for turning left and right.

Getting "Apply Root Motion" Checked and Re-Structuring My Project

I had "Apply Root Motion" unchecked. That was causing some problems as we saw before. There were two main problems that I was having when apply root motion was checked, First, the camera would spin like a carousel around the character when running and moving the mouse too fast. When this would happen, I was unable to stop the camera from spinning with my mouse, and had to wait for it to slow down and stop. This could also be due to not having a proper camera controller script. I will need to revisit this and create a proper camera controller script. The second problem I was having was that over time, the character would rotate inside of controller, offsetting it's rotation from the camera in 3D space, which is also very problematic. Lastly, as we saw before, when "Apply Root Motion" was unchecked, I was getting problems with the sprinting animations looking dorky, possibly because they aren't blending properly.

In order to be able to check apply root motion, I had to do a few things, including deleting and re-importing most of my animations, with only the root translation zeroed out, will explain this later.

There is another reason why I spent a lot of time re-organizing my project. I have a lot of animations sitting on my computer, things I have purchased from places like the Unity Asset Store or Humble Bundle in the past, been drawing from many of those instead of just my MoCap. In order to go completely from MoCap, I would have to shoot a lot more MoCap, effective time management. My project had all of my animation libraries imported into it because I wanted to be able to preview them all. There were way too many animations, slowing things down, project is too big, can't move it from home to school computers. So, I separated them into 2 projects, one for previewing my animation libraries, the other for bringing chosen animations in and building functionality.

My animation preview layout:

There were way too many animations to just be sitting around in one Unity project unused:

I also worked on importing .fbx animations successfully from Blender. I found this video very helpful, from 7:35:

https://www.google.com/search?q=how+to+export+blender+animations+to+unity&rlz=1C1CHZL_enUS798US798&oq=how+to+export+blender+animations+&aqs=chrome.1.69i57j0l2.12989j0j7&sourceid=chrome&ie=UTF-8#kpvalbx=1

First thing to do is make sure all bones are selected in Pose Mode, not just the root bone or the armature object:

Next, these boxes need to be checked when exporting, all except "Force Start/End Keying."

For some reason, Blender doesn't like MoCap rigs or Vicon FBX format or something. Other rigs work, but when I try to export Vicon Blade MoCap animations/rigs from Blender to Unity, it breaks. I will need need to Use Motion Builder for those.

After that, when I have re-imported into Unity these check boxes need to be checked (all except Loop Time and Loop Pose):

The reason why these 3 check boxes need to be checked is to avoid the root rotation being applied to the character, which was causing him to rotate around in 3D space in the Unity engine.

So, sorta taken 2 steps forward, 1 step back, but it is for the better.
The reason why I deleted all of the animations from my project, zeroed out the root transforms, is so that I can check apply root motion, which fixes the problems I was having with blending between the running cycles, and also allows me to apply the root motion for some certain animations which will not be completely zeroed out on a single axis moving forward. This helps add the impression of motion to moving character so that it doesn't look like it's rolling in place while being translated on it's z axis. This is also good for me to know in case I need to use apply root motion in the future.

I also got jumping to work, but there are no animations applied, yet.

So the current state of my project is that I am still zeroing out root translation of the characters to the world origin. Once that is complete, they will be easy to add in because everything (state machines, animator, scripts, etc.) is still there.

Next week I will work more on fleshing out my character controller, adding more functionality. including falling and landing.

Fixing Problems

        I have not yet been able to find the difference between IK and FK re-targeting inside of Unity. I have not been able to find it in the documentation so far. If you know, please let me know where I can find it.

        Earlier, I tried editing the root transforms of my MoCap rigs in Blender. It failed miserably, the animations broke upon import into Unity.

        To fix the issue where the character rig was moving around inside of the character controller, and not staying centered in the view, I had to un-check apply root motion in the inspector. I will re-check it once I have all animations completely zeroed out at the origin.

        To create my states and blend trees so far, I used a combination of animations from Mixamo, Crysis, the Unity Asset Store, and my own motion capture. Any animations that had to be purchased, I did so months ago and they have been sitting on my computer, unused--until now. As you can see, the locomotion and camera issues from last week have been fixed. The character no longer moves around off-center with the camera. I used Unity IK to bind the left hand to the rifle. The character can run and sprint in all directions as well as aim and strafe when the player presses the left mouse button. The aiming is off-center, but that will be resolved once I setup aiming controls.

        As you may have seen, the arms flicker when I press the melee button. I tried setting up a crazy blend tree to blend between 14 different punching animations because I have always like the idea of randomized animations for certain things, especially melee. It has not worked so far, but the problem occurring has nothing to do with the shear number of punching animations. I know this because I tested it with a far simpler tree with only 4 animations, and the same problem persists.

Blend Tree Movement and Aiming

        Right now, I have movement working with a blend tree, blending between four running cycles, and four sprinting cycles. So, my character can run in eight directions by pressing any combination of one or two of the W, A, S, and D keys. If the Shift key is held, my character starts sprinting. Sprinting forward is still too explosive, I will need to slow that down by decreasing the value of the variable in the script. The rotation isn't that great, I will refine that when I go on to building my advanced 3rd-person character controller.

        There is a problem, the character appears to rotate when moving around. I think this is caused by the rifle aiming animation, which plays randomly because I haven't set it up properly yet. I will have to zero out the transforms of that animation in the animation editor. Observe:

        As you can see, the character rotates a bit to the right whenever the aiming animation loops, which, over time, causes the character to turn to the right like we saw earlier. I will also need to edit the position of the character's hands using Unity IK, because the rifle is not being gripped, and is being aimed too high.

Animation Re-targeting and Looping Demonstration in Unity

        The possibilities are endless because I can now re-target any humanoid animation to any character rig in Unity with the click of a couple of buttons. Here, I have re-targeted a somersaulting animation from another character to my character's rig, and set it to be looping.

Original animation:

Re-targeted and looping animation:

Applied to one of my mocap rifle aiming animations:

Unity Mecanim or Autodesk Motion Builder?

        I have determined that Unity's animation retargeting system in Mecanim can be very powerful, but that it is ideal if you already have finished animations. If I were, say, doing a project using primarily or almost entirely animations from the Unity asset store or some other online marketplace or community, then Unity Mecanim alone would be perfect. I would just funnel in the animations from all of these outside sources into my project and character.

        Straight away, it's pretty cool, you just drop your character rig into the project, and it automatically recognizes if if your character is in t-pose. You may need to configure the avatar, but you don't need to go through the tedious process of characterizing your character like in Motion builder. You can also correct your character, rotating the joints to fit t-pose, much like Motion Builder, but that's where the similarities end.

        I found this video training comprehensive overview of Mecanim very helpful, much of it was review, but still made certain things like Blend Trees easier to understand. Much of it was not review.

https://www.youtube.com/watch?v=wf6vtCgLk6w

        Most of what I needed to know was in the first half-hour. At 51:00, he mentions root transform offset and some other helpful things. At 1:15:00, mentions useful State Machine Behavior script components. Damping, 1:21:00.

        I discovered that one can do many interesting things in Mecanim, such as setting up muscle space, which is pretty much limiting the rotations on the character's joints, which can come in very handy if you want to make your character more or less flexible.
        I also hit Unity's documentation this week, exploring different things I can do in Mecanim.

https://docs.unity3d.com/Manual/animeditor-AnimationEvents.html

        While Mecanim is very powerful, editing animations in Mecanim seems to be very manual, setting keys and editing curves in a dope sheet much like Maya, Blender, or some other 3D package. My key-frame animation skills are not very good. In concept, I can manually edit animations in Mecanim. Even so, I feel that that would be very time consuming and inappropriate for this project and out-of-scope, considering the remaining time frame.

        Immediately after shooting MoCap, the animations are not ready to go into the engine. The MoCap rig rarely ever matches up perfectly or adequately with your character rig. So, editing always required, to ensure that the animation is in-place, and that the positioning of the limbs are correct so that there is no clipping. For this project, I need the extra functionality that Motion Builder has because it allows me to edit animations efficiently, via animation layers. The MoCap animations that I have are not finished, they are not in-place, and not yet suitable for use in Unity. For this project, I need to use Motion Builder.

         I can definitely see myself using Unity Mecanim in the future, so I do want to continue familiarizing myself with the capabilities that Mecanim has to offer, as I am very excited now that I have discovered these. I could possibly use it next quarter in New Media Project, or further along in the future, in place of Motion Builder, because Unity is free, Motion Builder is not.

Character Rigging in Maya

So far, this week has been a major comeback.
I have retopologized my character in Blender, and rigged it in Maya. I decided to use Maya for rigging for this project because Blender was giving me issue last time, when I made my own block-out animations and tried to get them working on my blender-rigged character in Unity. The mesh exploded upon the playing of the animations. So, it seems that the bones were scaling and rotating in odd directions for whatever reason.
I found that rigging in Maya isn't so hard. In fact it is much like rigging in Blender. If you set the Bind Skin setting s correctly, Maya can do most of the heavy lifting as far as applying weighting to your model goes. All you have to do is make some adjustments here and there on certain joints by weight painting.
My repopology so far looks a bit crumby. The character is UVd, so anytime I want, I can edit the position of the vertices of my mesh in order to allow for baking the detail from the high-resolution model into texture maps. I mirrored my character's right side to his left after UVing in Blender, so the UVs are directly on top of each other. This allowed me to mirror my weight painting in Maya. All in all, I was able to optimize the rigging of my character so that it took the least amount of time possible. Some of the weight painting is rough, but it will suffice for now.

So far, I am looking into retargeting animations in Unity rather than Motion Builder. The deciding factors will be how much extra functionality does Motion Builder offer? How much of it is needed? Is Unity overall faster and more efficient for what I am doing? Will editing the animations via animation layers in Motion Builder be necessary since I am using Unity IK to tweak the animations anyway? These are the questions I need to answer.

        This week has been a battle, I have been set back considerably. I have not accomplished what I set out to finish this week. My super-soldier character has been completely modeled, but still needs to be rigged.
Aside from my other classes, one of the reasons why I have been set back is because I was sculpting my character to begin with, then realized that sculpting is taking way too long, that it will take days, or even another week to smooth out my character's armor plating. I thought that I should capture the basic form by making a high-poly retopology, then a low-poly retopology, that this would be a far more efficient way to get rid of the bumpy/wobbly surfaces on my character's mesh. In concept it seemed like a great idea, much more efficient, but it was not. I underestimated the amount of time it would take to do such a high-poly retopology, working with so many edge loops. Perhaps it was such a great idea, but was not executed properly. Maybe if I was not affraid to add more geometry early on, I would have ended up with a high-poly mesh where all of the edge loops connected properly, all quads, in a short amount of time. This is why I mentioned a new character creation workflow in the previous post. One lesson learned is that if you are going to take the time to detail sculpt a character, do it all the way. Or, figure out if you are going to be sculpting or subdiv modeling, and stick to it. If you don't spend too much time sculpting, just a sketch, getting the basic forms in then my proposed new workflow can be a great idea, I think.

        Proposed character workflow for future reference:

1. Blockout model/basemesh, "sketch" sculpt, or both consecutively.

2. High-poly retopology

3. Detail sculpt

4. Low-poly retopology

        Sculpting a character from scratch takes too long. This workflow could work, if the artist does not spend too much time detailing or refining in any one step.

My super-soldier character modeled:

My super-soldier character modeled (subdivided):

This project is more about functionality than pleasing aesthetics, so my super-soldier will not be fully textured (I will, however, bake out a normal map, since I went through the trouble to make LP and HP versions, retopologizing and all). He will have a normal map and flat colors applied.
After he is rigged, I will take him into Motion Builder to retarget my MoCap animations, then Unity Mecanim, and proceed to build his functionality as demonstrated by his animations.

I am still trucking along with creating my character. I expect he will be completed and rigged by tomorrow. Then, I will have to stop for this week. I can't wait to get back to working with Mo-Cap animations and Unity IK. I am eager to see how my character's rig works in Motion Builder when it is completed, but first things first.

Every time I create a character, I get more efficient at it, and my process becomes less haphazard. This character is a more difficult one to complete, as he is a super-soldier character with complicated organic armor plating, and this is one of my less intricate designs.

I am going to create a new 3D character creation workflow for myself, one that is suitable for a generalist. One that doesn't rely too heavily on hard surface modeling or sculpting, but utilizes both appropriately. The stages progress as so: blockout model, "sketch" sculpting, high-poly re-topology, detail sculpt, and low-poly re-topology. I will elaborate more on these in a later post.

I am proposing this type of workflow because as someone who is not a master sculptor, I have been bogged down trying to get evenly smooth surfaces in my character's armor. Many of my super-soldier's armor surfaces were very bumpy and wobbly, and I realized that this happened because there wasn't a clear separation between blockout sculpting and detail sculpting. I could tell that it would take hours, or even days to merely smooth out the sculpted armor surfaces, and even then they would not be perfect. Of course, the blockout model and high poly re-topology are not always necessary, but can be helpful steps.

So, I did a high-poly re-topology to effectively get rid of all bumpy surfaces, but I still will need to retopologize a 2nd time to bake the detail to a normal map.

The creation of my character is still underway, I am closer to completing the high-poly model, then I will do the re-topology.