A review of several “free” head-tracking alternatives to use with Flight

My favorite way to play Flight Simulator(s) is by doing sightseeing, VFR trips. I’m not an airliner person. So Flight Simulator 2020 is perfect because there’s loads of detail and everything looks great!
I’m loving the bush trips, and hope more will come soon!

The downside of it, especially when using tail draggers, is that its really heard to do a flight purely by VFR when you’re limited to a relatively small screen. It gets boring really fast if you constantly need to use the mouse to look around.

Enter head-tracking. I finally thought it would be a good time to try it. Of course everyone knows TrackIR, which uses a special camera and gear to detect head movements with great precision regardless of lighting conditions.
But I was not prepared to spend 200$+ on one, especially considering I never tried it before.

So I spent the last week just trying out different alternatives and decided to write a short blog post to summarize my findings, in case they are useful for people who might want to try this as well.

I installed and used:

  • ViewTracker
  • FaceTrackNoIR
  • OpenTrack
  • AITrack
  • FacePoseApp

Most of these options allow you to use different input sources. So let’s see one by one

ViewTracker

ViewTracker was the first one and although it’s not free, its quite affordable (12$ or so), but luckily there is a free 7-day trial to make sure it works for you.

The headtracking is probably one of the best of whole lot. But it had one big disadvantage: very high CPU usage (around 17%), which is not a good idea along side MSFS.

FaceTrackNoIR

FaceTrackNoIR seems to be one of the most famous free alternatives. It gives you plenty of flexibility in configuring different sources and outputs (you can even combine 2 sources at the same time).

The facetracking was pretty good for a free product and CPU usage about half of ViewTracker. But it had a few cons. I noticed that the facetracking algorithm didn’t play well in darker lighting or too bright, which is a problem in the place where I’m sitting. The other issue is that it tends to crash a lot.

This is where I decided to try FacePoseApp.

FacePoseApp is installed in your smartphone and uses the smartphone camera to track the face, so your CPU is spared of the work. It then communicates via UDP to FaceTrackNoIR which translates the data to 3d movement. Unfortunately, the facetracking was not as good for me, or maybe it was the light again….

AITrack

AITrack is a tool exclusively to track your face and send data via UDP to another program. So I connected it with OpenTrack. The facetracking algorithm was very good (I think viewtracker-good), but it tended to be more jumpy if I looked sideways too much. Unfortunately, like viewtracker, it consumed a lot of CPU as well… and I still needed a 2nd program to do the rest of the work.

OpenTrack

OpenTrack is the only one of this list which doesn’t include facetracking option. It has plenty of input sources which include Oculus, Valve VR, IR Cameras, Point trackers and… Aruco Paper Tracker!

I had never heard about Aruco before so I had to give it a try. You basically print a special marker to a paper, glue it to a cardboard or something hard, and stick it on top of your head. The aruco tracker looks for that layout (like a simplified QR Code) and tracks the distortion in 3D space. I was amazed at how well it worked and, best of all, very low CPU usage, which is great for gaming.

The only disadvantage is having to “wear” the paper tracker, but after 5min I forget that its there. The level of immersion is greatly enhanced compared to when you are playing without any sort of tracking. I can actually lift slightly up from my chair and look left and down to see stuff.

I even made a short video to demonstrate this

Important things to know

Tune the webcam settings. For some reason my default settings when I open OpenTrack have a “compensate low light” option enabled, which messes up the light and the tracking. I always turn it off and bring the exposure down.

Make sure you adjust the Curves Mapping ! It will basically give you the ability to fine tune how your movements and rotations translate to the in-game movements and rotation. The best way to do this is to start the game, be inside the cockpit and then start tweaking the curves. You can immediately see the effect and adjust as you need.
I turned my head left to the maximum I felt comfortable while still looking at the screen, and dragged the curve so that it would correspond to the maximum left turn in the game. Do that for up-down and then for the movements as well. (left-right, up-down and in-out)

As you can see below, my pitch curve is not linear because I don’t like to turn my head up or down too much. The first part is softer but if I’m really interested in looking up or down, then it will rotate faster in the game.

Finally, and very important, map shortcut buttons to your joystick buttons. Many people complain that they suffer from “stiff” neck after playing with head tracking. That happens because you are trying to be very still in order for the camera not to shake so much. This is solved by:

  1. tweaking the curves! – the first part of the movement can be very soft and then intensified as it grows
  2. having a quick way to “reset” the view back to the normal cockpit view. It’s inevitable that we move slightly in our chairs or adjust our position. I can quickly “center” the view, which will tell OpenTrack that this position is now position zero(center). That way I can freely move and when I’m comfortable it’s just a button-press away.

It’s also good to map a shortcut to toggle the tracking on/off. This is useful when you are using the mouse to press buttons or knobs in the cockpit. Your head will move too much and its frustrating to keep missing the target. So when I move my head in to see instruments, I quickly press the toggle button to stop tracking. Adjust knobs or whatever and when you are finished, press the toggle button again to resume tracking. Works like a charm.

My VFR has never been the same again :grin:

12 Likes

Thanks for the contribution. If I may allow myself to give a tip. I always use a linear curve but set a small dead zone at the bottom of the curve so that unintentional head movements do not cause camera movements.

Thanks for the review. I’m starting to try the different face tracking options myself, and had a edge case question.

I sometimes stream games using Steam to another room. I know I can successfully use FaceTrackNoIr using a long USB extension cord from the gaming PC in Room #1 and use powered USB Hubs to have my webcam in Room #2, but that is messy.

However I cannot get FaceTrackNoIr to work if I plug the webcam into the PC media server in Room #2

When I stream MS Flight Simulator from the Room #2 media server, the game recognises both the game controller and FaceTrackNoIr that are plugged into that client PC, however while the game controller works, the game doesn’t accept any inputs from FaceTrackNoIr. The software itself is working successfully, as I can see my face being tracked in the face tracking software on the client PC while MS Flight Sim is being streamed.

TL;DR - Does face tracking software running on a client PC work when using Steam streaming?

1 Like

You can try using DroidCam, to use your smartphone as a webcam and communicate via wireless to facetracknoir (which can be anywhere). Not sure if that would help your particular program though… it is a bit of an edge case :slight_smile:

anyone try the AeroConnector?

never heard about it. link?

Build your own head-tracking

Thanks AITrack seems to work fine with a Ryzen 7 and MSFS 2020 running :stuck_out_tongue_closed_eyes:

I don’t like wearing a hat

@daddoid1177 you’re better of using FaceTrackNoIR, it consumes less CPU and does good facetracking. All in one package

1 Like

Thanks I will give that a go.

I tried

  • Aruco Paper Tracker with OpenTrack
  • AITrack with OpenTrack
  • FaceTrackNoIR (Cost EUR3.45)

I prefer them in this order:

  1. Aruco Paper Tracker with OpenTrack
  2. AITrack with OpenTrack
  3. FaceTrackNoIR

Aruco Paper Tracker is the smoothest. I just printed out the template at 50% size and glued it onto some cardboard, and then velcroed that onto a baseball cap. Uses about 2% CPU on my Ryzen 7.

AITrack was OK, but a bit jumpy. CPU was higher but it didn’t cause me any problems.

FaceTrackNoIR was the worst even through it was a paid product. I have a 4K screen and it was written to only deal with 100% scaled screens. Mine is scaled to 150% and it was unreadable. It seemed a bit jumpy too but I didn’t persevere because of the terrible UI. Just look at it:

Happy with Aruco Paper Tracker for now even though I have to wear a hat. I thought it would bother me but I live in UK so it’s not too bad, plus I have air-con anyway.

1 Like

Yeah the facetracknoir UI is a bit bad… somehow it got jumpy on me too. I might go back to aruco again. Couldn’t try AITrack yet, crashes on me everytime I start it… :frowning:

update: downloaded a newer version of AITrack and managed to get it working. Looks cool, but maybe because of my lighting issues, its way way jumpy. It doesnt seem to have many settings to tweak the camera. Facetracknoir behaves better for me. And aruco when I feel like using a hat :smiley:

1 Like

Every head tracker should have mapping curve displays like FaceTrackNoIR. Too bad it’s not so good. I tried AITrack with OpenTrack, did not find the mapping options very helpful. and did, too, find the head tracking a bit jumpy in actual use with MSFS. I’ve ordered TrackIR from Natural Point and hope it works even better. But even with its faults, I really like AITrack and will have to give Aruco Paper a try, too. The video linked in the beginning of my related post helped me get going with OpenTrack and AITrack: OpenTrack & AITrack (help) (resolved). I think everyone should give head tracking a try - really enhances MSFS experience!

I used SmoothTrack with OpenTrack and that’s realy a smooth experience.

The basic configuration in OpenTrack and finetuning in the App is simple.
It works great with MSFS 2020.

SmoothTrack is available for Android and IOS.

2 Likes

it’s not free, but I use ViewTracker from Steam. It costs a few dollars but I found the tracking is more dependable and smoother than the free ones I tried.

I have switched off the roll, yaw, pitch because it’s hard to interact with the cockpit with these active (they shift around too much - probably fixable with settings) and I prefer to use the controller for looking around.

The cool thing with viewracker is that I can keep the X, Y, Z active. This means I can lean around the cockpit, lean in and out to get a closer look at things, Sit up and down in my seat to peer either over the cockpit or into the instruments, and lean left and right to see around outside (e.g. I can lean in and towards the side window and get a great view of the ground below me, or lean across into the passenger seat to catch a view out of the right side window, or look around the windshield supports while I’m turning to keep the runway in my line of sight.)

It has imporved the flying experience immensely.

1 Like

I usually turn off roll, but yaw and pitch are part of the experience for me, only x y z is boring!

Interacting with the cockpit can be challenging so what I did was to map a button to “freeze” the tracking. I look at the buttons/displays, freeze it, then move the mouse and interact, and when its done I unfreeze.

1 Like

Having spent a couple hours flying with the Head Tracker iOS app, I can say it really adds a lot to the game. This app (and SmoothTracker AFARK) use the TrueDepth camera of any iPhone (11 and beyond). However, I’m not entirely sure of that as tracking quality degrades heavily the darker your environment gets. In other words, it’s not suitable for flying at night in a dark room.

When there is enough ambient light in your room, it works smoothly and flawlessly. Gonna buy SmoothTracker next and see if there’s a difference …

And is there a difference?

PS3 Eye Toy Camera

I’ve found the PS3 Eye Toy camera to be far superior to the other webcams for head tracking, picked one up from eBay. Latest version of OpenTrack includes a PS3 Eye Toy Camera driver.

AiTrack

Works by tracking your face so you don’t need to wear a hat, LEDs and so on. It’s like magic!

v0.6.5-alpha this recent release is the first version that works well enough for me to replace an Aruco paper tracker (the shame :pensive::joy:) It tracks 6DOF I can move forward, backward, left, right, up and down. It’s feels so natural - superb.

Releases · AIRLegend/aitrack (github.com) https://github.com/AIRLegend/aitrack/releases

I’ve placed the webcam on top of my TV looking down. Choose 15fps for PS3 Eye camera, enter distance and custom brightness if lighting poor. 15fps is important as it reduces CPU load, better low light performance and in combination with EWMA filter and mapping properties in OpenTrack gives a perfectly smooth natural tracking.

opentrack/opentrack: Head tracking software for MS Windows, Linux, and Apple OSX (github.com) https://github.com/opentrack/opentrack

FS2020 settings for OpenTrack

Use EWMA filter. 50% Min/Max and 100% Curve.

Destination Source Invert
Yaw Yaw :x:
Pitch Pitch :heavy_check_mark:
Roll Roll :x:
X X :x:
Y Y :x:
Z Z :heavy_check_mark:

2 Likes