Sky Dolly v0.8 - FlightAware KML Import: Visualise & Replay Real-World Flights

Sky Dolly v0.7 is still in active development, but here is a preview of its major upcoming new feature: formation flight recording.

Except from RAM and CPU performance there is no limit set by Sky Dolly with regards to the number of aircrafts in the formation. In fact the only limit in the video was my patience, to fly 32 times the same 2 minutes flight :wink: And the usual limits of my iMac 27" with an Intel i7 4.2 GHz CPU, 32 GB RAM and AMD Radeon 580 Pro (with 8 GB VRAM).

To set expectations right: there is a serious “jitter-fest” going in FS2020 on when you sit in your own aircraft - or follow it with “flight following” enabled, but only for those aircrafts which are nearby: mind you: the frames per second are as usual, but aircrafts nearby jump ever slightly so “back and forth” - I call this “jitter” for now.

However as soon as you “detach” the drone camera from your user aircraft - so your drone camera remains “stationery” - all aircraft flight paths become smooth as silk, regardless of how close you move with the drone camera. As this feature is still in development I see what I can do, but for now it is good enough to shoot “cinematics”, at least with the mentioned “stationery drone camera” (and again, aircrafts “farther away” also seem to be moving smoothly, also with the “in-plane camera”).

I hope you enjoy it! At least the video for now :slight_smile:

UPDATE: Credits where credits are due, for this great “epic, cinematic soundtrack” with some “extra drama” - I love it! This is my “cinematic moment” in FS 2020 :wink:

Music:

Victorious

AShamaluevMusic

Screenshot:

2 Likes

To clarify the term “formation flying” in the context of Sky Dolly: we are talking about real formation flying, as in “Sky Dolly will be able to record and replay at the same time”.

So you start with one flight (existing or new) and “keep adding more aircrafts”, by simply clicking the Record button. So while you are flying the other (recorded up to this point) aircrafts will fly along you and your current flight gets recorded.

And again to emphasise the fact that no artifical limitation with regards to the number of aircrafts will be imposed by the upcoming Sky Dolly v0.7 (limitations might be imposed by the CPU/RAM, but in the example above I hit the “my patience ends now, after 32+ recorded aircrafts”-limit first).

This is what we are talking about here :slight_smile:

Spoiler alert: spoilers never deploy prematurely with Sky Dolly - the maximum delay has an upper bound of O(1/f) seconds, with f being the sampling frequency.

What this means - and why that is - you’ll get to know in the following small mathematical excourse about some sampling theory.

Warning: Some math ahead! If you’re more interested in “practical results” you may now scroll down to the YouTube video below.

After reading this post you will understand:

  • What sampling means
  • Why Sky Dolly replays are the exact same length as the originally recorded flight
  • Why aircraft movements in Sky Dolly replays are smooth, especially with sampling rates as low as 1 Hz
  • Why it is absolutely no problem with Sky Dolly to change the replay speed during replay (or even the recording sample rate!)
  • Why spoilers do not deploy prematurely :wink:

Sampling

Sampling data and its theory is applied whenever signals are to be processed, or typically when an analog (continuous) signal is to be converted into (discrete) “digital data”. Or in other words: samples (data values) are taken at regular (or even irregular) time intervals. The time interval is specified by the sampling frequency, with unit Hertz (Hz). 1 Hz means “one time a second”, 2 Hz means “two times a second” and so on.

Most people think of “audio signals” when talking about sampling (or informally: “recording”). But a signal can really be anything that carries data (information):

  • A two-dimensional image (where we basically sample in x- and y-direction)
  • Radio waves
  • Instrument readings which represent flight movements (aha!)

One important theorem is the Nyquist-Shannon Sampling Theorem: it basically states that any signal can be perfectly reconstructed if it is sampled with at least double the frequency than the maximum frequency of the signal itself.

Real-life example: the Red Book - the specification for audio CDs - specifies that music be sampled at 44.1 KHz (= 44,100 samples per second). Why is that? Because humans can (only) hear signals - “music” - up to roughly 22.05 KHz. So 44.1 KHz is double the maximum frequency we can possibly hear (with young ears).

Sampling Flight Data

The maximum sample rate is usually limited by the flight simulator, specifically its “simulation rate”.

What are the sample points? For instance:

  • Latitude / longitude / altitude
  • Pitch / bank / heading

But also:

  • Yoke position
  • Thrust lever position
  • And also binary (or boolean) data such as gear up/down

In a perfect world we would get a sample with a fixed frequency (equidistant time intervals) for each simulated frame during recording. And under perfect conditions during replay we would send the sampled data with the exact same frequency back to the flight simulator.

Note that we are ignoring interpolation for now, we are merely looking at the “raw sample data”.

There are various factors which make the world imperfect:

  • The simulator does not send data in regular intervals (very much like the “frames per second” vary as well - in fact there is a strong correlation between the FPS and the “simulated frames”)
  • Computer timers are not perfect (depending on the type of used timer its granularity can be as coarse as 15 ms)
  • The operating system can delay messages further, depending on CPU load

For instance, when we wanted to sample at 10 Hz (“ten samples per second”) we might end up with the recorded sample points in orange, with a total duration slightly above 1’000 milliseconds (or a bit more than a second):

A naive implemenation would assume that the recorded orange sample points would be equidistant. So if that naive implementation would somehow manage to replay them with that exact 10 Hz replay rate then the entire replay time would be exactly 1’000 milliseconds - or in other words: a couple of milliseconds too fast.

If on the other hand we somehow managed to record exactly the blue sample points and naively would replay them with a (possibly coarse) timer we would end up with the orange sample points being sent to the simulator, or in other words: the original time of 1’000 milliseconds would now be longer, or the replay would be too slow.

Improving the Naive Implemenation

There are several ways how we can improve the implemenation for recording:

  • Add timestamps to each sampled data, measuring the time (e.g. in milliseconds) from the start of the recording

And then during replay:

  • Adjust the time interval dynamically, according to when the last sample data was sent to the simulator
  • Skip one or several sample points if necessary, according to the timestamp of the sampled data

There are still some flaws when we try to stick to the sampled data “as is”, notably:

  • Rounding errors: most timers have a granularity of milliseconds only, but when we try to sample at, say 60 Hz, we need to wait 1/60 second, or 16.6666667 milliseconds, or rounded to an integer value: 16 - or 17 - milliseconds
  • Again, we can try to compensate those rounding errors by taking the elapsed time since the last sample point into account
  • But in combination with “coarse timers” those rounding erros usually “add up” over time…
  • … especially if we would - again, naively - try to “stick to the original data sample points”

Event-based Sampling

Sky Dolly hence considers the originally sampled data points nothing more as “control points which define the smooth curve” (we discuss interpolation in a moment). So conceptionally the entire signal is a smooth curve (which in reality it is, of course), and during replay we simply sample it again, with possibly (or quite likely, actually) a different replay sample rate than the original recording sample rate. During this resampling process we may or may not “touch” the originally sampled data points.

I call this also event-based sampling, because it is essentially the flight simulator that controls the (dynamic!) replay frequency, by notifying us about each “simulated frame” (which in practice is mostly equal to the displayed “frames per second”):

So in this chart we say that the orange sample points are the originally recorded samples, whereas the yellow points illustrate the sample points that are taken during replay. Again, the yellow and orange points may or may not be the same. We essentially resample the recorded data.

Sky Dolly is using a high-precision timer to resample the original curve, and since we now simply stop replay once the elapsed timer has reached (or exceeded) the last timestamp we end up with the exact same duration as the original recorded flight, replayed at the exact same speed.

Furthermore we can change the replay speed at any time during replay! Changing the replay speed essentially affects the elapsed time which we measure while we resample the curve.

Interpolation

If we sample the data at, say, 60 Hz (the “refresh rate” of most displays and hence the “magic frames per second” value) interpolation does not play any important role: for each simulated frame we would simply sent the recorded samples.

But as we sample the data with lower frequencies - Sky Dolly allows to sample at 1 Hz - interpolation becomes more and more important.

Interpolation is the process of “making an educated guess about values between known (recorded) sample points”. There exists many different interpolation methods. Besides the trivial nearest neighbour interpolation the next best - and easiest - interpolation is linear interpolation:

A noticeable drawback: the resulting curve is not smooth (mathematically speaking: the function is not differentiable).

Enter higher-order interpolation methods (where the order refers to the highest power in the mathematical equation).

Sky Dolly uses a cubic spline interpolation. Splines are a type of curve very popular in CAD (“computer aided design”) applications (e.g. also in the open source Blender). There exist many variants of splines, with different properties which make them suitable for various tasks in modelling, design and interpolation.

Sky Dolly uses Hermite Splines which are very suited for “data interpolation”, as they result in a “continuous curve”:

(Note that the shown curve here is not necessarily a cubic spline: it is simply how Excel does curve smoothening and shall hence only serve as illustration)

One important (mathematical) property of Hermite Splines: they go exactly through the “control points” (= sampled data). Or in other words: the flight path taken during replay is exactly the same flight path that has been recorded, so for instance the aircraft does the exact same “touch-down” as in the originally recorded flight.

And What Was That With The Spoilers Again?

And what does that “Big Oh” notation mean :wink: Now we have all theory together to explain why Sky Dolly never prematurely deploys spoilers - or flaps, or any other control surface, for that matter.

All aircraft and instrument data is recorded at the same recording frequency, and is then replayed at the same replay frequency (which, again, is very likely to be different than the recording frequency).

As each recorded sample data is assigned a timestamp the maximum delay at which it is replayed later on is given by the following upper bound:

 O(1/f), where f is the sample frequency (in Hertz) and the unit is seconds

Without going into all the intricacies of the “Big Oh” notation let’s explain this with an intuitive example:

  • Let’s assume the recording sample frequency is 1 Hz (“one sample per second”)
  • That simply means: f = 1

Let’s further assume the pilot deploys the spoilers just right after we have taken a sample (where the spoilers were not yet deployed). That means we don’t record the fact that the spoilers have been deployed until about (a bit less than) one second later:

...--*-+------*--------*--------*...
     0        1        1        1

So * indicates the time at which we take the sample, + indicates the moment the spoilers are deployed and 0 / 1 indicate the state of the spoilers. So it takes at maximum up to one second until we sample the fact that the spoilers have been deployed, and that’s exactly what the “Big Oh” notation is saying (with f = 1).

When we increase the recording frequency to 2 Hz, we get O(1/f), f = 2, so “at most half a second” etc.

So the higher the recording frequency, the smaller the maximum delay. But we also take note that the spoilers are never deployed too early, since we never record future events. And since we use the same timer for the aircraft’s position and attitude and all the flight instruments the implication is that all control surfaces are in sync with the aircraft’s movement (with a small delay, as given by the upper bound O(1/f)).

Sample Rate Comparison

Remember the Nyquist-Shannon theorem from above? This theorem also implies that “the less data we sample, the more details we loose”. While 1 Hz recording sample rate may be sufficient for airlines “which mostly fly straight lines” (their “turn/roll/pitch frequency is sufficiently low”) it is insufficient for “acrobatic flying”: we cannot reconstruct each flight maneuvre with a sampling rate as low as 1 Hz.

So why do we even bother to sample at lower sampling rates? Because the less samples we take, the less data storage we need. Sky Dolly reduces the required data storage (and CPU load) even more, by a technique that I call “active samples”, using a sampling window. But that shall be the topic of another post.

The following video compares various recording sample rates. But also take note that the replay is exactly the same duration as the original recording in the top-left (which was screen-captured at the same time while recording the flight with Sky Dolly):

Enjoy :slight_smile:

1 Like

To be clear: formation flying itself is hard. Very much so.

But the just released Sky Dolly v0.7 preview makes it easy to record and replay formation flights.

To illustrate, this is how you add a new aircraft to an existing (or new) flight:

  • Switch to the new “Formation” module (key F2)
  • Press the “Record” button (key R)

That’s it - really! And while you are recording all the existing aircrafts are replayed at the same time, so you know exactly where you are flying in relation to the other aircrafts in the formation.

And yes, that makes it a Sky Dolly exclusive feature :slight_smile:

While the AI aircrafts are replayed with an “auto” sample rate (interpolation every “simulated frame” event) you can still choose the recording sample rate independently, from “auto” down to 1 Hz. Switchable also during recording (because why not).

All aircrafts are controlled with a single user interface (Sky Dolly) instance. No messing around with an “infinite amount of applications instances” cluttering your precious screen space.

Talking about which: need more screen space? Simply enable the “minimal UI”, which provides everything you need for recording & replay, also for formations:

All aircraft recordings are automatically stored in the “logbook” (database) as well, so no messing around with tons of individual files. Want to replay a formation flight? Simply double-click on the flight and press the “Play” buton (or press the spacebar). All aircrafts start simultaneously, just as you would expect. As easy as that.

You can “jump from cockpit to cockpit” during replay, by switching the “user aircraft” (the aircraft being followed by the camera in FS 2020). Or you want to take control of the “user aircraft” yourself? No problem! You can enable the “Fly user aircraft manually” checkbox at any time. Got lost? Simply uncheck the checkbox again!

New aircrafts are initially positioned relative to the current “user aircraft” at the start of the recording. The user interface certainly won’t win the price for “the most sexy user interface”, but it’s functional. You can also remove existing aicrafts from a given flight.

Also refer to the official announcement made here:

Since the introduction of the new “Formation” module the download numbers on flightsim.to alone have skyrocketed from something above 2000 to over 3000 in just one week!

And thank you so much for all your constructive feedback and especially also your feature ideas! While I do take note of them I do not really want to comment on upcoming features until they are done and „ready to fly" - because I am working on Sky Dolly in my free time and I do not want to build up expectations that I possibly couldn’t meet.

So I am especially happy to hear that Sky Dolly has been nominated as one of the “Top 8 Applications for MSFS” by OverKill:

The Sky Dolly presentation starts at the 6:52 time stamp, but do watch the entire video about the other interesting tools as well - and check out OverKill’s YouTube channel, too.

Quotes:

“Now I have seen payware softwares from other developers and this freeware application Sky Dolly […] is the best that I have seen.”

“I’ve been thoroughly impressed: it records everything from aircraft control position, landing gear position, flaps, you name it. The replays are flawless.”

“It is truly fantastic, I am really impressed with how this little software has come along […] and again 100 percent completely free of charge. I am absolutely blown away by this given the fact that, again, I’ve tried two different payware applications and altough both good this one still takes the cake: it’s the least performance-impacting and provides a lot of features for the flight simmer.”

Thank you very much for those kind words :slight_smile:

Sky Dolly v0.7 now supports formation flying with an unlimited number of aircrafts. While you are recording the formation aircrafts are being replayed, so you know exactly where you are flying.

All aircrafts are controlled with a single user interface instance: play, seek & pause all aircrafts at once. Or do you feel brave enough to take control yourself and fly with the formation? You may do so at any time during replay!

The following teaser trailer illustrates the new “Formation” module:

Enjoy!

Checkout the channel Microsoft Flight Simulator 2020 Videos for some serious formation flying, featuring „The Tiger Moths“:

And a DC-3 formation:

Formation flying recorded with Sky Dolly.

I am so excited about the new features in the upcoming Sky Dolly v0.8, which is being fine-tweaked as we speak! While it was supposed to be more of an “under the hood changes” release (Sky Dolly v0.8 is now based on a plugin architecture) this release kept receiving many exciting - and hopefully useful - features.

Most of them revolve again around “formation flying”. The features will be announced once they are ready to ship, until then a few preview screenshots of a new “export plugin” (for those familiar: you will instantly recognise what this is all about ;))

Sky Dolly v0.8 introduces a new plugin architecture and many new features, most of which revolve again around formation flying:

Sky Dolly 0.8.0 square

Highlights:

  • Support for multiplayer formation flying
  • New “fly with formation” replay mode
  • New plugin architecture
  • New KML export plugin
  • Logbook filtering (by date, free text search, formation, duration, engine type)
  • Usability improvements

Also refer to the official announcement and the full CHANGELOG on github.com.

Let’s quickly dive in into one of the new features…

Multiplayer Formation Flying

While the SimConnect API does not yet give access to multiplayer aircrafts Sky Dolly now allows to import CSV (“comma-separated values”) files and add them to an existing flight:

Note that the aircraft type can now also be chosen in th CSV import plugin. For convenience all Flight Simulator 2020 stock aircrafts are already available (including the new aircrafts coming with the Simulation Update 5), but of course this aircraft table is automatically updated as new flights are recorded with new 3rd party aircrafts.

In the Formation module a time offset can then be added or subtracted to each individual aircraft, in order to synchronise their position to each other, depending on when the record button was pressed for each aircraft. The time offset result is immediately visible in Flight Simulator 2020, which is updated in realtime:

So the entire “multiplayer formation flying” process is as follows:

  • Each pilot records his/her flight on their computer
  • All but one pilot export their flight as CSV file
  • One pilot collects and imports all CSV files and adds them to his/her own recorded flight
  • Each aircraft can then be synchronised (“time-shifted”), by adding or subtracting a time offset (with millisecond granularity)

The formation flight can then be replayed like any other flight stored in the logbook.

The number of aircrafts per formation flight is technically unlimited. Of course replay, pause and seeking back and forth controls all aircrafts at one.

Sky Dolly is open source. Latest binaries are also available at:

Yay! The recent downloads just skyrocketed in the last couple of days and the 5’000 downloads mark has just been crossed:

Thank you so much, community, for the ongoing support and constructive feedback that I received over the weeks! I hope that Sky Dolly is as much use to you as it is fun for me to develop!

1 Like

:birthday::birthday::birthday: HAPPY BIRTHDAY, FLIGHT SIMULATOR 2020 :birthday::birthday::birthday:

Today a year ago the Flight Simulator 2020 was released - happy birthday!

As a present also to the community the latest bug fix release v0.8.5 also contains - exceptionally - a new feature: a KML import plugin which allows to import real-world flight data from https://flightaware.com. The real-world flights can then be visualised in Flight Simulator 2020, via Sky Dolly replay.

Here is a quick step-by-step instruction how to export KML data from FlightAware and import it into Sky Dolly:

FlightAware - Export KML

  • Go to https://flightaware.com
  • Enter a known flight number or airport ICAO
    • In this example we choose Zurich airport (LSZH)

  • Choose any completed flight
    • KML flight track data is not available for in progress flights
    • In this example we choose SWR317 from London Heathrow

  • Click on the yellow “View track log” button

  • In the Flight Track Log View click on the “Google Earth” button

  • The KML file download should now start
    • With most web browsers the file is automatically stored in the “Downloads” folder

Sky Dolly - Import KML

  • Launch Sky Dolly
  • Choose File | Import | KML (Keyhole Markup Language)
  • Click the Browse… button and locate the previously downloaded KML file
    • For example C:\Users\ [User Name] \Downloads\FlightAware_SWR317_EGLL_LSZH_20210818.kml
  • Choose an aircraft in the dropdown list that most closely matches the real-world aircraft
    • You may also add the data to an existing flight (“formation flying”)
  • Click on the Import button - done! :slight_smile:

You can now replay the newly imported flight like any other recorded flight!

Known Limitations

  • The KML file only contains position and timestamp data
  • So the aircraft attitude (pitch, bank and heading) is “reconstructed”, assuming an “A320-like aircraft” (airliner)
  • Events like flaps and gears up/down are currently also “hardcoded” and simplistic
  • The yoke inside the cockpit is not animated
  • Most real-world position data makes the aircraft “dive under the runway”, as altitude data does not 100% match with the Flight Simulator scenery data
    • This might get addressed in a future release, by adjusting the track point’s altitude to the actual scenery altitude

I hope you enjoy this new KML plugin and have fun visualising real-world aviation with Flight Simulator 2020!

P.S. Why not “join” the flight with your own aircraft (“formation flying”)? Can you do better? :wink:

1 Like