Spoiler alert: spoilers never deploy prematurely with Sky Dolly - the maximum delay has an upper bound of O(1/f) seconds, with f being the sampling frequency.
What this means - and why that is - you’ll get to know in the following small mathematical excourse about some sampling theory.
Warning: Some math ahead! If you’re more interested in “practical results” you may now scroll down to the YouTube video below.
After reading this post you will understand:
- What sampling means
- Why Sky Dolly replays are the exact same length as the originally recorded flight
- Why aircraft movements in Sky Dolly replays are smooth, especially with sampling rates as low as 1 Hz
- Why it is absolutely no problem with Sky Dolly to change the replay speed during replay (or even the recording sample rate!)
- Why spoilers do not deploy prematurely
Sampling data and its theory is applied whenever signals are to be processed, or typically when an analog (continuous) signal is to be converted into (discrete) “digital data”. Or in other words: samples (data values) are taken at regular (or even irregular) time intervals. The time interval is specified by the sampling frequency, with unit Hertz (Hz). 1 Hz means “one time a second”, 2 Hz means “two times a second” and so on.
Most people think of “audio signals” when talking about sampling (or informally: “recording”). But a signal can really be anything that carries data (information):
- A two-dimensional image (where we basically sample in x- and y-direction)
- Radio waves
- Instrument readings which represent flight movements (aha!)
One important theorem is the Nyquist-Shannon Sampling Theorem: it basically states that any signal can be perfectly reconstructed if it is sampled with at least double the frequency than the maximum frequency of the signal itself.
Real-life example: the Red Book - the specification for audio CDs - specifies that music be sampled at 44.1 KHz (= 44,100 samples per second). Why is that? Because humans can (only) hear signals - “music” - up to roughly 22.05 KHz. So 44.1 KHz is double the maximum frequency we can possibly hear (with young ears).
Sampling Flight Data
The maximum sample rate is usually limited by the flight simulator, specifically its “simulation rate”.
What are the sample points? For instance:
- Latitude / longitude / altitude
- Pitch / bank / heading
- Yoke position
- Thrust lever position
- And also binary (or boolean) data such as gear up/down
In a perfect world we would get a sample with a fixed frequency (equidistant time intervals) for each simulated frame during recording. And under perfect conditions during replay we would send the sampled data with the exact same frequency back to the flight simulator.
Note that we are ignoring interpolation for now, we are merely looking at the “raw sample data”.
There are various factors which make the world imperfect:
- The simulator does not send data in regular intervals (very much like the “frames per second” vary as well - in fact there is a strong correlation between the FPS and the “simulated frames”)
- Computer timers are not perfect (depending on the type of used timer its granularity can be as coarse as 15 ms)
- The operating system can delay messages further, depending on CPU load
For instance, when we wanted to sample at 10 Hz (“ten samples per second”) we might end up with the recorded sample points in orange, with a total duration slightly above 1’000 milliseconds (or a bit more than a second):
A naive implemenation would assume that the recorded orange sample points would be equidistant. So if that naive implementation would somehow manage to replay them with that exact 10 Hz replay rate then the entire replay time would be exactly 1’000 milliseconds - or in other words: a couple of milliseconds too fast.
If on the other hand we somehow managed to record exactly the blue sample points and naively would replay them with a (possibly coarse) timer we would end up with the orange sample points being sent to the simulator, or in other words: the original time of 1’000 milliseconds would now be longer, or the replay would be too slow.
Improving the Naive Implemenation
There are several ways how we can improve the implemenation for recording:
- Add timestamps to each sampled data, measuring the time (e.g. in milliseconds) from the start of the recording
And then during replay:
- Adjust the time interval dynamically, according to when the last sample data was sent to the simulator
- Skip one or several sample points if necessary, according to the timestamp of the sampled data
There are still some flaws when we try to stick to the sampled data “as is”, notably:
- Rounding errors: most timers have a granularity of milliseconds only, but when we try to sample at, say 60 Hz, we need to wait 1/60 second, or 16.6666667 milliseconds, or rounded to an integer value: 16 - or 17 - milliseconds
- Again, we can try to compensate those rounding errors by taking the elapsed time since the last sample point into account
- But in combination with “coarse timers” those rounding erros usually “add up” over time…
- … especially if we would - again, naively - try to “stick to the original data sample points”
Sky Dolly hence considers the originally sampled data points nothing more as “control points which define the smooth curve” (we discuss interpolation in a moment). So conceptionally the entire signal is a smooth curve (which in reality it is, of course), and during replay we simply sample it again, with possibly (or quite likely, actually) a different replay sample rate than the original recording sample rate. During this resampling process we may or may not “touch” the originally sampled data points.
I call this also event-based sampling, because it is essentially the flight simulator that controls the (dynamic!) replay frequency, by notifying us about each “simulated frame” (which in practice is mostly equal to the displayed “frames per second”):
So in this chart we say that the orange sample points are the originally recorded samples, whereas the yellow points illustrate the sample points that are taken during replay. Again, the yellow and orange points may or may not be the same. We essentially resample the recorded data.
Sky Dolly is using a high-precision timer to resample the original curve, and since we now simply stop replay once the elapsed timer has reached (or exceeded) the last timestamp we end up with the exact same duration as the original recorded flight, replayed at the exact same speed.
Furthermore we can change the replay speed at any time during replay! Changing the replay speed essentially affects the elapsed time which we measure while we resample the curve.
If we sample the data at, say, 60 Hz (the “refresh rate” of most displays and hence the “magic frames per second” value) interpolation does not play any important role: for each simulated frame we would simply sent the recorded samples.
But as we sample the data with lower frequencies - Sky Dolly allows to sample at 1 Hz - interpolation becomes more and more important.
Interpolation is the process of “making an educated guess about values between known (recorded) sample points”. There exists many different interpolation methods. Besides the trivial nearest neighbour interpolation the next best - and easiest - interpolation is linear interpolation:
A noticeable drawback: the resulting curve is not smooth (mathematically speaking: the function is not differentiable).
Enter higher-order interpolation methods (where the order refers to the highest power in the mathematical equation).
Sky Dolly uses a cubic spline interpolation. Splines are a type of curve very popular in CAD (“computer aided design”) applications (e.g. also in the open source Blender). There exist many variants of splines, with different properties which make them suitable for various tasks in modelling, design and interpolation.
Sky Dolly uses Hermite Splines which are very suited for “data interpolation”, as they result in a “continuous curve”:
(Note that the shown curve here is not necessarily a cubic spline: it is simply how Excel does curve smoothening and shall hence only serve as illustration)
One important (mathematical) property of Hermite Splines: they go exactly through the “control points” (= sampled data). Or in other words: the flight path taken during replay is exactly the same flight path that has been recorded, so for instance the aircraft does the exact same “touch-down” as in the originally recorded flight.
And What Was That With The Spoilers Again?
And what does that “Big Oh” notation mean Now we have all theory together to explain why Sky Dolly never prematurely deploys spoilers - or flaps, or any other control surface, for that matter.
All aircraft and instrument data is recorded at the same recording frequency, and is then replayed at the same replay frequency (which, again, is very likely to be different than the recording frequency).
As each recorded sample data is assigned a timestamp the maximum delay at which it is replayed later on is given by the following upper bound:
O(1/f), where f is the sample frequency (in Hertz) and the unit is seconds
Without going into all the intricacies of the “Big Oh” notation let’s explain this with an intuitive example:
- Let’s assume the recording sample frequency is 1 Hz (“one sample per second”)
- That simply means: f = 1
Let’s further assume the pilot deploys the spoilers just right after we have taken a sample (where the spoilers were not yet deployed). That means we don’t record the fact that the spoilers have been deployed until about (a bit less than) one second later:
0 1 1 1
So * indicates the time at which we take the sample, + indicates the moment the spoilers are deployed and 0 / 1 indicate the state of the spoilers. So it takes at maximum up to one second until we sample the fact that the spoilers have been deployed, and that’s exactly what the “Big Oh” notation is saying (with f = 1).
When we increase the recording frequency to 2 Hz, we get O(1/f), f = 2, so “at most half a second” etc.
So the higher the recording frequency, the smaller the maximum delay. But we also take note that the spoilers are never deployed too early, since we never record future events. And since we use the same timer for the aircraft’s position and attitude and all the flight instruments the implication is that all control surfaces are in sync with the aircraft’s movement (with a small delay, as given by the upper bound O(1/f)).
Sample Rate Comparison
Remember the Nyquist-Shannon theorem from above? This theorem also implies that “the less data we sample, the more details we loose”. While 1 Hz recording sample rate may be sufficient for airlines “which mostly fly straight lines” (their “turn/roll/pitch frequency is sufficiently low”) it is insufficient for “acrobatic flying”: we cannot reconstruct each flight maneuvre with a sampling rate as low as 1 Hz.
So why do we even bother to sample at lower sampling rates? Because the less samples we take, the less data storage we need. Sky Dolly reduces the required data storage (and CPU load) even more, by a technique that I call “active samples”, using a sampling window. But that shall be the topic of another post.
The following video compares various recording sample rates. But also take note that the replay is exactly the same duration as the original recording in the top-left (which was screen-captured at the same time while recording the flight with Sky Dolly):