Running Metahumans on the Hololens 2 reliably was a little more complicated than I thought it would be, but once you get the hang of the method it’s pretty easy.
We’ll be running the program, in our case Affinity on our PCs and using the Hololens Holographic Remoting app to stream Affinity to the headset. Because this is a multiplayer experience, 2 PCs and 2 headsets will be required. One will be the host/therapist who does all the controlling, and one will be the client/patient whose gaze we will be tracking. So the first thing you need to do if work out all the IP addresses of both your PCs and headsets.
STEP 1 : Find out the IP of your PC
To find out the IP of your pcs, open up a command prompt or terminal by clicking on the windows icon and typing “command”
In the command prompt type in ipconfig and hit enter
The number you want to take note of is the IPv4 Address
STEP 2 : Find out the IP of your Hololens 2 Headsets
You’ll need to start up your headsets for this. Find the Holographic Remoting app.
The number you need is the one in yellow
Okay now that you have all your numbers set, you’re ready to fire up the app!
STEP 3: Start the App as the Host/Therapist
So you have your headset on with the Holographic Remoting Player waiting for a connection. Next you need to go back to your command prompt/terminal and start up the app for streaming. So CD into the WindowsNoEditor folder where you have Affinity.exe
Now you need to stand in position because the app will load and orient itself around you. Here is how the main scene is set up. You will need to be standing where the white dot is, so that the Metahuman can load in, 2 meters directly in front of you.
Once you have loaded the scene, the second player/patient will need to stand in exactly the same place to load up their app and join your session. This way you will both be oriented in the app correctly.
It’s probably a good idea to put a marker on the ground so that you both know where to stand as it loads up.
Once you are standing in position, have an assistant run the following command in the command prompt
Affinity.exe -vr -HololensRemoting=your-headset-ip
NOTE: if the app is not orienting the way you would expect, it is a good idea to remove all holograms to clear the headset and make it remove all markers.
This is what you will see as it loads up. On the intro control panel, you will select “HOST” because you will be controlling the session.
STEP 4: Start the App as the Client/Patient
This is similar to joining as the host. First make sure the client is standing on your marker, and has started up the Holographic Remoting Player and is waiting for a connection. Next you need to go back to your second PC and in the command prompt/terminal CD into the WindowsNoEditor folder where you have Affinity.exe
The run command is slightly different because we want the app to track the user’s gaze. So type in
Affinity.exe -vr -HololensRemoting=your-headset-ip -eyetracking=WindowsMixedRealityEyeTracker
and hit Enter
The client Hololens will need to join the session that the host created above, so they will need to type in the IP address of the PC that is running the host app. PLEASE NOTE: It is the IP of the computer, not the Hololens.
If a mistake is made, the delete button can be used. For future use it might be worth putting the lab PC IP in as hard coded for testing, to remove the necessity for this step.
ADDITIONAL NOTES: If you want to test the app by yourself, without a second person joining you, you should also run this command as it will give you the eye tracking data
STEP 5: Start Testing
Now that you’re up and running, start testing. Here’s a capture of me testing it with Matt so you can see what you’ll be doing. When viewing through the headset, the avatars render fully and look realistic. It’s actually quite weird when they accidentally catch your eye as you’re studying them. Have fun playing!