Since it went live, I’ve been almost exclusively flying the PMDG 737-700.
I downloaded Sky Dolly earlier this week and could find some time to play with it yesterday. First of all, congratulations @Steeler2340, this is an incredible tool and - at least for my use case - much, much better than the embedded MSFS replay tool. Thank you for investing your time and talent to create it and share with the community. We are all in debt to you. .
During my tests I noticed something for which I would like to check if I’m the only one with this issue or not. The behavior is reproducible 100% of the time on my end.
Here is my flow:
Few minutes before landing, I start recording
Right after leaving the runway and STOPPING any plane movement, I stop recording
I drag the slider back to a point where the plane is very close (2 to 3 seconds) to the touchdown moment
Play
Pause
Set camera so I can see my landing from the runway with a stationary camera (Drone with ‘Drone Follow Mode’ OFF and ‘Drone Lock Mode’ ON)
Drag the slider back a bit more (until the plane is +/- 1 minute prior to the touchdown moment)
Unpause and enjoy
Usually, after touchdown, I drag the slider back until the plane is +/- 1 minute prior to the touchdown moment, let it play, and move the drone to see the landing from a different view point. I use to do this multiple times.
When done, I just drag the slider to the end or wait till it gets there
This all works superbly well. However, when trying to resume taxi, the plane seems to be stuck on the ground. I have checked everything to the best of my knowledge and I don’t see anything that could be causing this like auto or parking brakes applied for example. The plane will only move when N1 is around 80%. At that time, something ‘unlocks’ the plane and - as you can expect - it moves forward as a rocket.
I have tried different things without success:
After leaving the runway, stopping the plane, and before stopping recording, I confirmed the brake (be it auto or parking) was NOT applied.
I completely ignored step 2, that is, I went to step 3 when the plane was still on the runway moving at around 30kts or so. Much better this time because I can keep the plane moving by applying just a bit of thrust (the normal condition). However, if for whatever reason I have to stop the plane during taxi, I cannot make it move anymore unless I apply 80% N1.
I can be wrong here (as this is frequently the case ) but it seems to me Sky Dolly is not ‘restoring’ something when it is done replying the sequence. At least in my setup.
I can’t provide support here for specific issues, including issues related to certain 3rd-party aircraft.
However I took the liberty to create an issue for you:
This is also the preferred location when you have an in-depth issue or question, as you can easily provide more information there (free github.com account required though).
In general I can say that Sky Dolly records and replays exactly those “simulation variables” that you see in the corresponding “Simulation Variables” dialog (press key F). However those are “standard SimConnect variables”.
In case the aircraft uses custom logic to e.g. start/stop the engine then it might be that “something gets stuck” (in the aircraft simulation logic). This is very much possible - but kind of outside the reach of Sky Dolly (or any replay software, for that matter).
In the first tutorial of this series we have seen the basic steps how to connect a databbase browser such as SQLiteBrowser with the logbook. In a previous tutorial we have learnt how to create new flights from scratch, by inserting them via SQL.
In this tutorial we are looking at how to analyse - to select - existing flights and do some “data mining”.
Export Plugins
Before we begin: Sky Dolly already provides many export plugins, for various formats such as GeoJSON and KML. Specifically when exporting your flights in CSV (comma-separated values) you get your flight data in a text-based format that can be imported and processed with many applications, including spreadsheet editors such as Excel and LibreOffice Calc.
Sky Dolly supports the following CSV flavours:
Sky Dolly: the complete flight data, but soon to be deprecated and replaced by a much more powerful export format
FlightRadar24: the CSV format as used by flightradar24.com - essentially containing aircraft position, heading and speed
Position and attitude: essentially like the FlightRadar24 format, but including pitch and bank angles
But what if you wanted more data? To do some basic data aggregation and calcuations before exporting? What if none of the export formats provides the data that you are looking for?
Enter the power of SQL and the open logbook format of Sky Dolly…
Data-Mining Your Logbook
At this point we assume that you have already established a connection with your logbook (= the database). If you are unsure have a look at the first tutorial in this series.
Hint: When trying to open a logbook don’t forget to change the file selection dialog filter to All Files (*).
In the following examples we will be using SQLiteBrowser, but any similar SQLite database editor will do, including the SQLite command shell itself, at least as far as the SQL queries themselves are concerned.
Selecting a Flight
All following queries usually involve selecting some given flight from the logbook.
Remember:
The flight table contains basic flight information such as flight conditions and start- and end times (of the first recorded aircraft) as well as the sequence number of the user aircraft
The aircraft table contains basic aircraft information and the sequence number of each aircraft: a flight may have multiple aircraft (“formation flight”), but always at least one.
Finally the position table contains the recorded coordinates, velocity and attitude (*), including a timestamp (in milliseconds). Each position sample belongs to an aircraft.
There exists other tables such as engine, primary_flight_controls and others: they are on the same hierarchy level as the position table and essentially work the same.
(*) This is subject to change in some future Sky Dolly release, but the idea will remain the same
All this leads us without further ado to your first “select flight” queries. The most easiest query is simply selecting a flight by its given ID:
-- Get specific flight given by its ID
select f.id, f.creation_time, f.title
from flight f
where f.id = 63
where the value 63 needs to be replaced by an existing flight ID. In fact, the logbook in Sky Dolly shows the flight IDs, making it super-simple to select the desired flight data:
We can also query a flight recorded on a given date:
-- Get latest flight on a given date
select f.id, f.creation_time, f.title
from flight f
where strftime('%Y-%m-%d', f.creation_time) = '2021-05-26'
order by f.creation_time desc
limit 1
Here we make use of the SQLite-specific built-in function strftime that formats the date & time into a convenient ‘YYYY-mm-dd’ text format.
As there might be multiple flights on the given date we order the result set in descending order (“order by … desc”) by creation_time and limit the result set to 1 (“limit 1”).
Of course the actual date value ‘2021-05-26’ needs to be replaced again with the desired date.
Or we might simply want to select the most recent recorded flight from the logbook:
-- Get most recent flight of the logbook
select f.id, f.creation_time, f.title
from flight f
order by f.creation_time desc
limit 1
Now that we know how to select a given flight let’s step it up a little with some more interesting queries that join the flight table with other tables…
Creating Your Custom Flight Data Query
Explaining the various SQL table join operations would go far beyond of what I can show here. However excellent tutorials about SQL do exist, for example:
The following example query shows how you can select your custom data and make it available as CSV. We will see that we can also create some basic charts easily with SQLiteBrowser itself. So fasten your seatbelts…
select f.id as flight_id,
f.start_zulu_sim_time,
f.end_zulu_sim_time,
a.type,
a.flight_number,
p.timestamp,
p.latitude,
p.longitude,
p.altitude,
p.pitch,
p.bank,
p.true_heading,
p.velocity_z * 0.5924838012959 as speed_in_knots
from position p
join aircraft a
on p.aircraft_id = a.id
join flight f
on a.flight_id = f.id
and a.seq_nr = 1
-- Select the above data from the most recent flight in the logbook
where f.id = (select ff.id
from flight ff
order by creation_time desc
limit 1)
Wow, that escalated quickly Let’s analyse what we just did here:
We select the desired data columns like flight ID, aircraft type (e.g. “Asobo A320neo” etc.), position and attitude from the joined tables flight, aircraft and position
The velocity_z contains the aircraft velocity in Z direction that corresponds to the “forward” direction, in feet / seconds. We convert it to knots, to illustrate that we can also transform the data with basic mathematical functions (*)
We select the position data from the first (“seq_nr = 1”) aircraft in the flight
We can also nest SQL queries: here we select the ID of the last recorded flight in the logbook (as shown before)
Now open your favourite text editor and create a new text file. In this example I use Visual Studio Code, as it provides a wast amount of plugins, including editing CSV data in table format and even visualising GPS coordinates (in CSV format) on a world map.
Now paste (CTRL + V) the previously copied data and save your newly created text file with the *.CSV file extension. Depending on your editor and installed extension plugins this might enable syntax highlighting for CSV and enable alternative edit modes such as a CSV table editor:
Even with SQLiteBrowser we can already create some basic charts. But let’s have a look at some more queries first in order to gather some interesting data to visualise.
Flight Statistics
The following query evaluates our top 10 favourite aircraft types, according to their number of flights:
-- Select the top 10 aircraft types (number of flights)
select type as AircraftType,
count(type) as Count
from aircraft a
group by type
order by count(type) desc
limit 10
Note that we can rename colums with “as AircractType” or “as Count”, for better readability in the result set. In this query we also group result rows according to some given criteria: here we group by the aircraft type.
We are also using a so-called aggreate SQL function here for the first time, the count function. So in this example we simply count the number of used aircraft types, order the result set in descending order and limit the result set to 10 rows.
Example result:
AircraftType
Count
Pitts Asobo
46
Asobo Savage Cub
17
Fiat-Aeritalia G91 R1 AMI PLAIN
13
Airbus A320 Neo Asobo
12
Supermarine Spitfire LF Mk IXc FlyingIron G-IRTY
9
Supermarine Spitfire Mk IX RAF - 126 Sqn ML214 ‘Muscat’
6
Extra 330 Asobo
6
DV20 Asobo
5
Airbus A320 Neo FlyByWire
5
Boeing 747-8i Asobo
4
Likewise we can create a top 10 flight list according to flight duration:
-- Select the top 10 *flights* (flight duration in minutes)
select f.id, f.title, a.type, max(p.timestamp) / 1000 / 60 as minutes
from aircraft a
join position p
on p.aircraft_id = a.id
join flight f
on a.flight_id = f.id
group by p.aircraft_id
order by minutes desc
limit 10
Example result:
id
title
type
minutes
12
Flight Zurich to Geneva
Airbus A320 Neo FlyByWire
53
33
New Zealand Tour (Mount Doom)
T-45C Goshawk TW-1
49
34
Tokyo Drift with Crash Landing
Boeing 747-8i Asobo
47
14
Sion nach Samedan
Fiat-Aeritalia G91
38
29
Cliffs of Dover
Supermarine Spitfire
25
28
Rio de Janeiro Sightseeing
Icon A5 Asobo
22
11
Take-Off II from Zurich with failed start
Airbus A320 Neo FlyByWire
16
23
Paris Arc de Triomphe (Crash)
Extra 330 Asobo
16
10
Take-Off from Zurich
Airbus A320 Neo FlyByWire
15
13
Landing in Geneva
Airbus A320 Neo FlyByWire
12
By nesting the above query into the following query we can create our own top 10 aircraft type list, according to the total duration (in minutes) flown with each aircraft type:
-- Select the top 10 aircraft types (total flight time)
select a.type, sum(a.minutes) as total_minutes
from (select aa.type, max(pp.timestamp) / 1000 / 60 as minutes
from aircraft aa
join position pp
on pp.aircraft_id = aa.id
group by pp.aircraft_id) a
group by a.type
order by total_minutes desc
limit 10
Example result:
type
total_minutes
Airbus A320 Neo FlyByWire
106
Fiat-Aeritalia G91
84
Boeing 747-8i Asobo
65
Pitts Asobo
59
T-45C Goshawk TW-1
49
Supermarine Spitfire
33
Extra 330 Asobo
25
Icon A5 Asobo
22
Asobo Savage Cub
22
Supermarine Spitfire ‘Muscat’
15
Disclaimer: I am not affiliated with nor do I endorse any of the above aircraft types. All the presented example results are simply to be considered as some “random snapshot” of an older logbook of mine and are meant to be for illustration purposes only.
The Plot Thickens…
The SQLiteBrowser offers a simple Plot module that allows to visualise the current result set in an X/Y plot diagram.
We can easily visualise our previous top 10 favourite aircraft type list, by simply opening the Plot tab and selecting the desired columns for the X and Y axis, including bar chars (in case the X axis values are of type text):
Those plots can then be saved in various formats, including PDF, JPEG or PNG.
Summary
We have seen that:
With SQL we can create our own custom reports and extract the data that we need, including basic math calculations
We created several “top 10” statistics, including “your favourite aircraft type”
We can easily export our custom SQL result sets as CSV data
We can further process & visualise it in external tools, including basic plot and bar charts in SQLiteBrowser itself
Now it is up to you to come up with your own interesting statistics and logbook visualisations. With the above presented techniques and tools there are almost no limits of what you can do.
If you have any interesting SQL query feel free to share it with others here
That’s not a big problem now.
And I don’t want to come off as a teacher either.
But SkyDolly reverses the action of the aileron.
I’ve noticed this on two planes now and I suspect it’s the same on all of them.
When I fly in the sim everything is ok, in the replay the aileron works upside down.
Uhm… exactly the aileron reversal that occurred with aircraft that have a „PID controller“ like the stock Asobo A320neo and F/A-18 should have been fixed with the v0.14 release. By the way the reversal also affected the rudder and elevators for those aircraft.
So when you say „now“:
Is this a new aileron reversal behaviour that worked correctly before the v0.14?
Does it also affect rudder and elevators, or really only the ailerons?
Which aircraft exactly are we talking about?
I am asking because I thought that I had thoroughly paid attention to all aileron, rudder and elevators movements, with the aircraft that I tested with:
Asobo A320neo
PMDG 737-800
Fenix A320
Asobo F/A-18
But yes, in general I did change the implementation of those control surfaces: instead of sending back the recorded „simulation variables“ (that seem to be ignored by certain 3rd-party aircraft) I am now sending back „events“ (like those that would come from your hardware joystick or yoke) - and the sign of the previously recorded „simulation variables“ had indeed to be reversed, as it seems!
Hence it is very well possible that some (?) aircraft do react differently than others to those „events“, at least during replay… in any case, if you could tell me the exact aircraft that you were using that would already greatly limit the „search space / uncertainty“ for me
I have now examined it more closely. Now I’m 100% sure it’s a FBW problem.
Since I’m not a bus driver, I can only say that for the military jets.
I can say that as long as there is no FBW there are no problems. All planes with FBW cause problems. But only in the air when the FBW is active. Before starting, on the ground, everything works normally. After takeoff, in the air, the aileron reverses polarity.
Elevator and rudder are not affected. Unless the elevator is mixed in to assist the aileron. Then the effect is reversed here too.
But, as I said, only in the air, not on the ground.
All aircraft from IndiaFoxTecho, DC Designs, SC Designs are affected.
An exception is the F-22 from TopMachStudios, where the control surfaces don’t move at all.
In the meantime I have also tested the F-18 from Asobo.
It also has the problem described.
Just like the Gripen from Deimos.
It remains: FBW when active, on all aircraft.
Hmmm, that‘s weird! What you describe sounds exactly like the problem that should have been addressed by Sky Dolly v0.14 (CHANGELOG): the problem was caused by the „PID controller“ respectively seemingly a bug in MSFS itself that caused the PID controller to „counter-act“ the „simulation variable input“. The issue was caused when recording and sending back the rotation velocity in (the physically actually correct unit) „radians per second“. MSFS seems to expect „feet per second“ here (and I only stumbled over this because the Simvar Watcher example app from the MSFS SDK doesn‘t even let you choose „radians per second“).
The solution was simply not to record the rotation velocity at all anymore. And yes, the PID controller only takes control once airborne, that‘s why I specifically tested this both on ground and especially in the air. Also with the stock Asobo F/A-18 (because I had initial reports there).
The only difference from what you describe is that this bug affected all primary control surfaces, including rudder and elevators - according to your description only the elevators seems to be affected, so I understand.
In any case I will take another look at this with the F/A-18 and stock A320 (I don‘t own any of the other aircraft that you‘ve mentioned).
You’re right.
I was too lazy with the updates. I’ve now got the latest version and everything is fine there.
You have my full admiration for your patience with the constant changes to the sim for no apparent reason. Developing add-on content for such a system must be a nightmare. Especially if you don’t make any money with it.
Anyway, thanksalot for your work and sorry for the confusion.
Sky Dolly v0.16 “Gregarious Gee Bee” (v0.16) has just been released. This release introduces new features like new Sky Dolly logbook import & export plugins, including automated time offset calculation for formation flights.
Time offsets? What is this good for? This allows to synchronise your formation aircraft, recorded on multiple PCs by several different “live players”. Or to re-create a “busy airport day” by importing and time-synchronising real-world flights from e.g. flightradar24.com or flightaware.com.
Synchronising Live Player Formation Flights
Each formation aircraft has always had its own time offset, allowing to manually move it into the future or past, relative to all other formation aircraft. With the new time synchronisation option (available for all import formats that support real-world timestamps, such as the Sky Dolly logbook) this synchronisation can now be automated:
None - no synchronisation is done
Time - synchronisation based on time of day (but ignoring the date)
Date and time - synchronisation based on date and time
Both Time and Date and time synchronisation have proper time zone support.
The option Time (only) is useful e.g. if you wanted to import several historic real-world flights from different dates, but still want to have them replayed “on the same day” - unless of course you enjoy watching replays with a runtime greater than 24 hours. But hey, why not
But how would you create a formation flight with several “live players”?
Each pilot records his or her flight on a dedicated PC, running its own Sky Dolly instance, flying together in some MSFS “live session” and trying to push the “Record” button more or less at the same time
This results in recordings starting at slightly different times, of course, with a delay up to minutes (or possibly even longer)
After all pilots have finished their recordings they export their flights as Sky Dolly logbook (*.sdlog)
One selected pilot now collects all those recordings (best via some file sharing/synchronisation service like OneDrive), stores them in a given directory and imports them “in one go”:
Import directory: checked
Import: Add to new flight (alternatively: Add to current flight)
Time synchronisation: Date and time (for all practical purposes Time (only) synchronisation should have the same outcome)
That’s it! All aircraft are now part of a new (or the current) flight and are properly time-synchronised. Just like at the time of recording the “live players”-formation (or “community”) flight.
I do unfortunately have som issues with SkyDolly; such as engines halting at replay and buttons I cant switch (in cockpit) suddenly while recording. Also some gear-problems etc.
You can now activate the Location module (press key F3) and immediatelly teleport to any POI in New Zealand!
Bug Fix Release 0.16.2
Unfortunately the newly introduced CSV parser in Sky Dolly 0.14 introduced a regression in that its validation was a bit too strict: CSV files without header could not be imported anymore. This has now been fixed in the just released 0.16.2 bug fix release.
For issues related (mostly) with 3rd-party aircraft please have a look at the Frequently Asked section on Sky Dolly for Microsoft Flight Simulator | MSFS (you may have to search for that link a little bit on that page - unfortunately I cannot give a direct link), in short: aircraft with custom simulation logic do not properly report / react to the official “simulation variables”, as they use their own “local (custom) variables”.
That you cannot activate switches during recording is completely new to me however: Sky Dolly simply requests a set of those simulation variables (“read-only”), but does not otherwise “interfere” with any simulation logic during recording.
For more specific support you may also create an issue here (free github.com account required):
Please share additional information there (not here in the forum), such as the exact aircraft type, which controls are not working during recording, whether you might have any other 3rd-party addons that might interact with MSFS via SimConnect at the same time etc.
Every then and when I get questions here in the forum or over at flightsim.to that can essentially summarised as “what does Sky Dolly actually record?”, or more generally: “How does recording / replay work?”
But to first address another elephant in the room: “Yes, Sky Dolly is still in the makings” Development has slowed down somewhat in the last three months, because I changed my job - life happens But as I write those lines I have been adding some more code to the upcoming v0.17.
Don’t expect anything big this time around, as this will be more of a “refinment release”, polishing existing functionality and smaller issues and ideas that have been reported/suggested by community members - thank you all for your support!
Now for the questions: the latest specific question that I have received was:
“Does Sky Dolly capture and replay smoke from tires upon landing?”
Similar questions that I have received in the past were related to audio, whether Sky Dolly would record that, too.
And the answer to all those questions is a simple: “No.”
So how does it work then? Sky Dolly (and other flight recording software) records a selected set of what is known as “simulation variables”: they represent the state the simulation and the aircraft is in at any moment in (simulation) time. Those variables are part of the official SimConnect API that has been introduced back in the days with Flight Simulator X and continues to be supported by MSFS today.
The “simulation variables” represent all sorts of states, for instance:
Aircraft position and attitude (= as a result of the user input and simulated physics)
Control input: yoke, rudder and the resulitng ailerons postions, flaps, spoilers, …
Animation states: cockpit open/closed, …
Environment: temperature, wind directions, time of day, …
Not all variables can be modified, as they are a result of the calculated physics, based on other simulation variables. And to make things a bit more complicated: not all aircraft properly report (support) those “simulation variables”, let alone properly react to changes in them (but that’s another topic, for another time ;)).
So what does Sky Dolly record then? A selected set of those variables that have “the (biggest) impact on the visual representation of the recorded flight”. So for sure the exact position and attitude, but also “visual clues” such as flap/brakes/ailerons/… settings (“pilot input”). Also light switches and other “visually relevant cockpit switches” are recorded.
The exact set of those “simulation variables” can be seen in Sky Dolly, by opening the “Simulation Variables” (sic!) dialog (e.g. by pressing key V):
The variables are grouped: more or less in the same groups that Sky Dolly uses to communicate with the flight simulator. “Why is that so?” you may ask: because that’s a clever way to minimise both the required disk usage and CPU usage while replaying those variables to MSFS. Only those values that are actually changing are recorded (= less disk usage) and sent back to MSFS during replay (= less CPU usage).
For instance the gears are typically only raised and lowered once during a flight. So it would be a waste of disk space/CPU to always send a “gears up” value to MSFS. (And yet this all works magically when you seek back and forth at any time during replay ;)).
So to narrow down on the specific question, whether “gear smoke” or “audio effects” are recorded: again, no. They are the result of the above simulation variables being replayed to MSFS - or not. While most of the physics keeps running during replay (instead of you giving control input it is now Sky Dolly) every physics simulation that would affect the aircraft position and attitude is disabled during replay. Otherwise the MSFS physics would constantly “fight” the injected aircraft position and attitude, for every “simulated frame” - and the result would be a “jittery user aircraft” (there is another known issue related to “jittery AI aircraft”, aka “formation flying”: but that is yet another topic and not directly related to “physics fighting recorded position”).
Because of “position / attitude” physics are disabled - by design - it could be that some “visual effects” are not triggered (simluated) as they usually would. Also keep in mind that since we are only recording a subset of all possible simulation variables the simulation during replay is not 100% the same as when you recorded the flight. So slight simulation alterations may cause some effects to be ignored (or vice versa, you keep hearing warning signals that weren’t heard during recording).
I am still looking into recording more “simulation variables” that could have a “significant (visual) effect” on the replay, for instance I am looking into payload weights and fuel that incluences the display of external ordinance / tanks on some aircraft, such as the F-16. Also, I intend to completely re-engineer the recording of the engines: as you know MSFS now supports a total of 16 engines.
And to conclude: Sky Dolly also records some “read-only” simulation variables, but those are “for your information only”, as can be seen in the “Flight” (press key F) dialog:
P.S. Pro tip: hover the mouse over any of the above simulation variable text boxes and you will see the exact “simulation variable” name, as used via SimConnect and documented here:
And if you want to experiment yourself, there is a simple “Simvar Watcher” example application that comes with the MSFS SDK (software development kit) examples, downloadable from within MSFS with enabled “developer mode”
Here is an older post illustrating the use of the “Simvar Watcher”:
Yes, you can, as comma-separated values (CSV) text file (e.g. also editable in some spreadsheet editor like Excel or LibreOffice Calc etc.).
Two flavours are currently supported:
Little Navmap userpoints CSV format - import and edit your Sky Dolly locations in Little Navmap and plan your flights accordingly
Sky Dolly CSV - share „everything there is“ with other Sky Dolly users
Of course you can also import Little Navmap userpoints. In fact, you‘ll find „collections“ of all photogrammetric cities, all POIs of all world updates etc. on e.g. flightsim.to as Little Navmap userpoints.
But I also intend to provide import/export plugins in the Sky Dolly logbook format. Having the advantage that they would automatically be converted to the latest Sky Dolly format.
And there‘s more to come in the Location module, too
.
Let’s analyse what we just did here:
