ASSIGN CONTROLS TO TRIGGER ANIMATIONS

Now I want to trigger the different animations based on the keyboard input, so to do that I need to set up some input controls in the Project Settings. So go to EDIT > PROJECT SETTINGS

 

 

And then scroll down to INPUT and then we are going to add in some new Action Mappings so click on the little plus sign next to the title.

 

Give your action a name, here I have just named it the animation title, then click the small keyboard button underneath and assign the key input for that action. Do that for all the animations you have imported.

 

 

Now we want to tell our character to use these inputs and to play the animations when it detects them, so to do that we first need to set up an animation blueprint.

We do that by going to our Content Browser and selecting ADD > ANIMATION> ANIMATION BLUEPRINT

 

and select your characters skeleton to link the animation to.

Now let’s go back to the character blueprint and link this Animation Blueprint to our character. Open up your character blueprint, select the mesh in the left hand side Components panel, and then in the right hand side Details panel scroll to the Animation tab. Make sure Animation Mode is Use Animation Blueprint and the for the Anim Class use the dropdown to select the Animation Blueprint you just created.

defining character variables based on input

So what I want to do is have the character play an animation based on the user input. So to do that I’m going to declare a few boolean values to be true or false dependent on the keys being pressed and released.

So go into your character blueprint and in the My Blueprint panel on the bottom left, go to the Variables section and click on the +Variable button to add variables. Add as many as you need. 

 

Now go into the event graph, right click and look for the first Input Action that you defined in the Project Settings Input panel and add it as a node. Mine was Defeated

 

 

Now you need to set your variables based on the button being pressed and released. We set the variables here and then in the animation blueprint we will read their value and play the required animations.

So when the button is pressed we want this character to set its variable to true and then when the button is released we will give it a delay for the length of the animation and reset it to false so that the animation can reset to the idle state. 

To set your variables, right click and search for set varName where varName is your variable name. Mine is isDefeated.

 

 

The whole node set up will look like this. Don’t forget to tick the box for the variable when you want it to be true.

 

 

I have 5 different variables to set for my 5 different animations, so my character event graph looks like this

 

setting up the animation blueprint

 

Right so our character is set up all ready to receive input an has the boolean variables updating correctly. Now we just need to tell it what animations to play in the Animation Blueprint. So go open up your animation blueprint and click on the AnimGraph tab and the first thing we’re going to do is right click to Add new state machine

 

 

And you’ll link it up to the result animation

 

Double click the New State Machine node to open it up. Then what we’re going to do in here is add all the animations that our character can move between, and we’ll set up some rules for when it can go into the animations and back again. We will be referencing the boolean variables we set up in our character blueprint for that , but for now let’s just link up our animations.

Right click and Add State and call this one Idle. This will be our default standing/hanging around doing nothing state. Drag an arrow from Entry to Idle.

 

And double click on Idle to open it up. Now on the right hand side you’ll see a tab for Asset Browser. Click on that and you will see all the animations associated with the skeleton. Choose the idle on and drag it into the graph. And then click and drag a connection between the two little human figures. So that is your Idle animation set up and since you have linked to it from the Entry node, it will go straight to it onload.

But we have a lot of animations we want to play when different keys are pressed. So we will go back out to the New State Machine and do the exact same thing to se up the other animations. We’ll link  them up and make conditions for when they play next.

So your new state machine will look something like the image below, and inside each state you will have linked up the appropriate animation as you did in the image above.

 

Now link up all the animations to the Idle so that the character can alternate between them. Just click and drag from Idle to the State until you see the little blue and red icon with Create a Transition tag.  Make transitions both to the states and back from them to the Idle state. It should end up looking something like this.

 

Now the next thing we need to do is to create some local variables that the state machine can use to know when it should trigger animations. We will define the variables here, but we will assign their value based on the value of the corresponding variable in the character blue print. IE the ones we assigned when the keys were pressed. So for me here I’m going to name them playDefeated, playDismissive, playNervous, playWalking and playHappy and I’m going to define them in the Animation Blueprint in the Variables tab in the My Blueprint panel on the left hand side. just click the + sign and type in your variable names. We will get their value in the next step.

 

Now go into the Event Graph of the Animation Blueprint and we will grab the values from out character Blueprint. To do that we need to cast to the character blueprint. So right click and type in Cast to and then your character blueprint name. Mine is AkaiCharacter so I will type Cast To AkaiCharacter. It will come up in the list as you start typing.

Link up the nodes as shown below and then drag out a pin from the cast node and get the first animation variable you defined in your character blueprint. Mine is isDefeated, and then assign it to your local Animation Blueprint variable. Mine is playDefeated. Link them up as shown.

 

Now get and assign all your other variables. It should end up looking something like this.

 

So now that we have all the variables from the Character Blueprint assign to local variables in the Animation Blueprint, we can use them to tell the state machine when to play the various animation states that we set up. So go into the state machine tab and double click on the first circle leading from the Idle state to your first animation.

Right click and GET the variable that you want to be true for the animation to play. Mine is playDefeated. Link it up to the result node.

 

Now go back out to the state machine and double click on the circle that leads back from your animation state to the idle state. Now we want the character to return to the idle when the playDefeated variable returns to false. (this is set in the character blueprint and passed through into the Animation Eventgraph remember). So we need to get the variable again, but add in a NOT to make it negative. See below.

Do the same for the other animation states and hit COMPILE.

 

Now as you hover over the circles, the conditions that you just defined will show up.

 

 

STEP 1 IMPORT CHARACTER AND ANIMATIONS

First just drag and drop your downloaded character .fbx into the Content Browser window in Unreal. Just ensure that Convert Scene Unit is selected as you do so, then click Import

 

You should see all the elements layout out as such in the Content Browser

 

 

Similarly drag and drop your downloaded animation .fbx into the Content Browser Window

When the dialogue box comes up though, you’ll need to assign the skeleton of your character. So click the dropdown next to skeleton and choose from the list it provides.

Do this step with every animation that you want to import.

 

STEP 2 CREATE CHARACTER BLUEPRINT

So now we have our character asset and animations imported let’s make our character blueprint.

Now what I’m after here is something a bit different. I’m not making a game or a first person shooter. I’m making an immersive experience (target being a Hololens 2) where you observe the characters and try to determine their feelings by body language and facial expressions. So I want the characters to play through their animations while the camera is looking at them. For the moment I will do that with keyboard input. So this entry will attempt to do that.

To do this go to your Content Browser Window and Select the green Add/Import >> Blueprint Class 

 

Then select Character and give it a name. I’ve named mine AkaiCharacter

 

Double click on your character blueprint to open it up. Now you need to assign the mesh to use. In the Components panel on the left, click on Mesh.

In the right hand Details panel, scroll down to the Mesh tab, click on the dropdown and select your mesh from the list.

 

 You might need to adjust your mesh to sit in the middle of your character blueprint nicely, so do that in the viewport.

 

 

Now in the left hand side Components panel select you character – mine is AkaiCharacter(self) and then in the right hand side Details panel, scroll down to Pawn and change Auto Possess Player to Player 0 

And hit the compile button

 

 

 

Mixamo is an Adobe library of Characters and Animations that are free to use. It is quite extensive and for learning purposes it’s definitely enough to do the job. Follow these steps to download a character and an animation to use in Unreal.

STEP 1 – LOGIN or Create account

Go to Mixamo and either log in if you already have an account, and if not, sign up because it’s free.

www.mixamo.com

 

STEP 2 DOWNLOAD CHARACTER

You’ll see two tabs at the top of the page. Characters and Animations. First go into the Characters tab and search through the vast array until you find something you like. Then simple click the orange DOWNLOAD on the top right.

 

In the download dialogue box that comes up you can just keep the default .fbx Binary as the download Format,  and T-Pose as the Pose. Click the Orange DOWNLOAD button

 

STEP 3 DOWNLOAD ANIMATION

Click on the Animations tab at the top of the page. Search through the animations until you find one that you like and click on it to load it into your character panel on the right.

Now before you download the animation, you have a panel on the right, highlighted, where you can make minor adjustments to the animation so that it suits your character. Especially helpful if your character is very fat or very skinny. Have a play around with the sliders and you’ll get to know what they do.

Then click the orange DOWNLOAD button

 

Download with the following settings. I choose not to download the skin because then I can just assign the animation to as many characters as I like. 

Click the orange DOWNLOAD button.

 

One thing to note with moving animations like walk cycles, in Mixamo they cross the floor which you rarely want them to do when you are using them in the editor. If you look at the Mixamo preview when you choose a walk animation you’ll see what I mean. 

When you have your characters in Unreal though, you pretty much always want to control where the character goes with your own code, so you need to check the In Place checkbox before you download. Once you click it you’ll see the difference.

 

MAKING MYSELF IN METAHUMAN CREATOR

I’ve been lucky enough to have early access to the Unreal Metahuman Creator and I’ve made myself! Of course she needs a lot more wrinkles which I couldn’t find any exact options for. None the less, not a bad start.

 

HOW TO GET MY METAHUMAN INTO UNREAL

 

To get your creation into Unreal so that you can start to play with it, you need to have Quixel Bridge installed. You can grab it from here. Download it and sign in using your Epic Login details. Don’t create a Quixel account because you want it to link to your Epic Metahumans. Just a note, once you do have it and you’re coming back to use it on another day, search for “Bridge”, not “Quixel” when you want to open it again.

Once you have logged in, you can find your Metahumans in the menu indicated below.

 

Each of your metahumans will show in the window and there is a “download” button in the top right corner of each. Press the one you want to download. 

 

 

While it’s downloading you can go and start up Unreal. You will need to make sure you have 4.26.2 so go and update it if you haven’t already.

Open the Unreal file that you want to import your Metahuman into. Then go back to Quixel Bridge and click the “Export” button

 

 

An “Export Successful” message will appear once it has finished exporting. 

 

Go back to UE and you will see it importing into your open file. You may need to restart to apply missing settings.

 

 

Once imported then just navigate to it in the content window, Content > Metahumans > TheNameOfYourMH and then drag the blueprint onto the stage. It might look grey at first, but just wait until the shaders have finished compiling and then you’ll see your metahuman completed.

 

Again I am following the Microsoft tutorial, just adding in extra screen grabs to help solidify the process in my mind. For the original tutorial, look here https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/tutorials/unreal-uxt-ch4

 

Spawning Hand Interaction Actors

 

I’m not going to tell you about the kinds of hand interactions you can do in Hololens 2 because I’m assuming you know about that already. This is just to tell you how to set them up for Unreal.

Hand interaction with UX elements is done with Hand Interaction Actors, which create and drive the pointers and visuals for near and far interactions.

open the MRPawn Blueprint and go to the Event Graph.

  1. Drag and release the execution pin from Event BeginPlay to place a new node.
    • Select Spawn Actor from Class, click the dropdown next to the Class pin and search for Uxt Hand Interaction Actor.
  2. From the Right arrow at the top pf the Hand Interaction Actor Box, drag and release to spawn a second Uxt Hand Interaction Actor, this time setting the Hand to Right. When the event begins, a Uxt Hand Interaction Actor will be spawned on each hand.

Your Event Graph should match the following screenshot:

Spawn UXT Hand Interaction Actors

 

The SpawnActor function requires a Transform input to avoid a compiler error, so you’ll use the default values.

Drag and release the pin off one of the Spawn Transform pins to place a new node. Search for the Make Transform node.

 

 

Drag the Return Value to the other hand’s Spawn Transform so that both SpawnActor nodes are connected.

 

 

Select the down arrow at the bottom of both SpawnActor nodes to reveal the Owner pin. Drag the pin off one of the Owner pins and release to place a new node. Search for self and select the Get a reference to self variable.

 

 

Create a link between the Self object reference node and the other Hand Interaction Actor’s Owner pin.

 

 

check the Show Near Cursor on Grab Targets box for both Hand Interaction Actors. A cursor should appear on the grab target as your index finger gets close, so you can see where your finger is relative to the target.

 

 

Compilesave, and return to the Main window. – I’m not screen grabbing that. You know how to do that by now. But double check that your Event Graph looks the same as this last screen grab.

 

Create something to interact with

 

So let’s create some things to interact with. I’ll show you how to make a cube blueprint and then you can make however many objects you like to go in the scene.

Go to the Content Browser >> Add/import >> Blueprint Class

 

choose Actor. Name the actor InteractiveCube – or whatever name you like.

 

 

Double-click InteractiveCube to open it in the Blueprint Editor, select Add Component > Scene and name it Root.

 

 

Drag-and-drop Root onto DefaultSceneRoot to replace it.

 

 

Click Add Component >Cube

 

 

Compile and save your blueprint, then head back to the main window.

 

 

Drag InteractiveCube into the viewport

 

 

Attaching Manipulators

 

A Manipulator is a component that responds to articulated hand input and can be grabbed, rotated, and translated. Applying the Manipulator’s transform to an Actors transform allows direct Actor manipulation.

Open the InteractiveCube blueprint, click Add Component and search for Uxt Generic Manipulator in the Components panel.

 

 

Expand the Generic Manipulator section in the Details panel. You can set one-handed or two-handed manipulation, rotation mode, and smoothing from here. Feel free to select whichever modes you wish, then Compile and Save InteractiveCube.

 

<<< STEP 5 – ADDING MIXED REALITY

Again I am following the Microsoft tutorial, just adding in extra screen grabs to help solidify the process in my mind. For the original tutorial, look here https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/tutorials/unreal-uxt-ch3

 

Add The Session Asset

To use a session, you need an ARSessionConfig data asset to work with. To create one of these go to your Content Browser window. Make sure you are in the root Content folder by clicking on the Content folder.

Click Add/Import > Miscellaneous > Data Asset 

 

Select ARSessionConfig, click Select

 

 

name the asset ARSessionConfig.

 

 

Double-click ARSessionConfig to open it, leave all default settings and hit Save. Return to the Main window.

 

 

Start and Stop AR Session

 

To start the AR session you use a Level Blueprint. Connecting the ARSessionConfig asset in the Level Blueprint guarantees the AR session will fire right when the game starts playing.

To do this click Blueprints > Open Level Blueprint from the editor toolbar:

 

 

This will open up the Eventgraph window. Now this is node based work, so if you’ve ever used Shadergraph or Houdini then you’ll be familiar with this kind of coding. If not then just hang in there, this is a really easy entry example to set up. Don’t worry, you’ll get used to it.

Drag the execution node (left-facing arrow icon) off Event BeginPlay and release

 

 

then search for the Start AR Session node

 

 

and hit enter. Excellent, you just connected two nodes.

 

 

Click the Select Asset dropdown under Session Config and choose the ARSessionConfig asset.

 

 

Right-click anywhere in the EventGraph and create a new Event EndPlay node. 

 

 

Just like we did for the BeginPlay Event, we’re going to drag the execution pin and release, then search for a Stop AR Session node and hit enter.

 

Hit Compile, then Save and return to the Main window.

 

 

create a pawn

At this point, the project still needs a player object. In Unreal, a Pawn represents the user in the game, but in this case it’s going to be the HoloLens 2 experience.

 

Click Add New > Blueprint Class in the Content folder

 

 

Expand the All Classes section at the bottom. Search for DefaultPawn, click Select

 

 

Name it MRPawn, and double-click the asset to open.

 

 

Click Add Component > Camera from the Components panel and name it Camera.

 

 

Make sure that the Camera component is a direct child of the root (CollisionComponent). This allows the player camera to move with the HoloLens 2 device.

 

 

Select CollisionComponent from the Components panel and scroll down to the Collision section of the Details panel.

  • Click the Collision Presets dropdown and change the value to NoCollision.
  • Do the same for the MeshComponent

 

 

Compile and Save the Blueprint, then return to the Main Window

 

 

create a game mode

 

The last puzzle piece of the mixed reality setup is the Game Mode. The Game Mode determines a number of settings for the game or experience, including the default pawn to use.

 

Click Add New > Blueprint Class in the Content folder and select Game Mode Base as the parent class. 

 

 

Name it MRGameMode and double-click to open.

 

 

Go to the Classes section in the Details panel and change the Default Pawn Class to MRPawn

 

 

Hit Compile, then Save and return to the Main window.

 

 

Select Edit > Projects Settings and click Maps & Modes in the left-hand list.

  • Expand Default Modes and change Default Game Mode to MRGameMode.
  • Expand Default Maps and change both EditorStartupMap and GameDefaultMap to Main. When you close and reopen the editor or play the game, the Main map will now be selected by default.

 

<<<< STEP 4 – CREATE A LEVEL        STEP 6 – INTERACTIVity >>>>

 

 

Again  I am taking instructions straight from a section in Microsoft’s Chess Tutorial. But I am just adding in a few extra pictures because I am a visual learner and this really helps me solidify what I’m doing in my head. So some people might find this quite slow and hand holdy, so sorry, maybe the Microsoft pages will be better for you.

https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/tutorials/unreal-uxt-ch2 The whole tutorial is great and recommend you read through it.

So,

CREATE YOUR LEVEL – Select File > New Level 

 

and choose Empty Level. The default scene in the viewport should now be empty.

 

 

CREATE PLAYER START – Select Basic from the Modes tab and drag PlayerStart into the scene.

 

 

Set Location to X = 0Y = 0, and Z = 0 in the Details tab to set the user at the centre of the scene when the app starts up.

 

Viewport with PlayerStart

 

ADD A CUBE – Drag a Cube from the Basic tab into the scene.

  • Set Location to X = 50Y = 0, and Z = 0. to position the cube 50 cm away from the player at start time. (see picture above for where you’ll find Transform settings again)
  • Change Scale to X = 0.2Y = 0.2, and Z = 0.2 to shrink the cube down.

 

 

ADD A LIGHT – Switch to the Lights tab in the Modes panel and drag a Directional Light into the scene. Position the light above PlayerStart so you can see it

 

 

SAVE YOUR FILE – Go to File > Save Current As, name your level Main, and select Save.

With the scene set, press Play in the toolbar to see your cube in action! When you’re finished admiring your work, press Esc to stop the application.

 

 

<<<< STEP 3 – ADDING PLUGINS         STEP 5 – SET UP FOR MIXED REALITY >>>>

 

Adding plugins to your project is pretty straight forward as long as you know what folders you need to copy and where to.

  1. Go to your root project folder and create a Plugins folder if there isn’t one already
  2. Unzip the zip you downloaded for the  UXTools
    1. Navigate to the Plugins folder in the unzipped files. It will be something like >> MixedReality-UXTools-Unreal-public-0.11.x >> UXToolsGame >> Plugins >>
    2. Copy the folders named UXTools  to your project Plugins folder

     

And then the final thing is you need to go into Unreal and enable your plugins. In UE to Edit >> Plugins to bring up the Plugins Editor.

Scroll down to HoloLens and check Enabled

 

Next, ensure All is selected in the top left, and in the top right grey search box, type mixed to bring up all the Mixed Reality Plugins. Click the Enabled checkbox in the Microsoft Mixed Reality Plugins. You will be prompted to restart so go ahead and press the Restart Now button in the bottom right corner.

When you restart Unreal will build the UXTools into the project

 

 

When Unreal restarts the UXTools plugin will be in your Content Browser. If you can’t see them there (and I couldn’t when I first set up my file), you can display them by adjusting the Content Browser View Options. Select >> View Options >> Show Plugin Content. You can also Show C++ Classes, but for now turn everything else off otherwise you’ll have a tonne of folders to sift through.

 

<<<< STEP 2 – FILE SET UP         STEP 4 – CREATE A LEVEL >>>>

 

YOU have everything downloaded, now what?

Launch the Unreal Engine 4.26 from the Epic Games Launcher.

 

In the Select or Create New Project Window, select Games in the New Project Category section and then hit Next

 

 

Next select a Blank Template and then hit Next

 

 

And in the Project Settings Select C++ Project, Scalable 2D or 3D and Mobile/Tablet

Choose a name and where you want to save it.

Then hit Create Project

 

 

<<<< Step 1 SET UP        STEP 3 – ADDING PLUGINS >>>>

I am following the microsoft Chess app tutorial as I do this – adding in screen grabs for extra information for proper nubies, so for the original instructions go to microsoft
https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/tutorials/unreal-uxt-ch1

 

Before starting

 

have the following PROGRAMS installed.

HAVE THE FOLLOWING PLUGINS READY TO INSTALL

Download the following plugins so that you have them ready to copy to your Plugins folder once you have created your project. I’ll remind you later, but to find instructions on how to install plugins, see the Blog post titled Adding Plugins To Your Unreal Project

HAVE THE HOLOLENS 2 TARGET PLATFORM INSTALLED

To do this,

>> go to the Epic Games Launcher

>> click on the Library button at the top of the page

>>click on the arrow button next to launch on the 4.26.1 icon

>> go to options

 

>> scroll down until you get to the Target Platform section and ensure that Hololens 2 is ticked.

>> if it isn’t ticked, tick it and hit the apply button. The download will start for you.

STEP 2 – FILE SET UP >>>>