Indeed it is not a problem in itself, and the reason they include their database IIRC is because they can’t access the FS2020 database using the SDK. In other words, when you’ll have a dozen aircraft each one having its own local database, this will be a dozen set of files to keep in sync and update too.
There remain a problem though: because the WASM sandboxed code is only given the permission to accessing a local sandboxed folder with write access just for this WASM sandboxed code, this further limits one of the strength of any other simulator: mixing add-ons.
This is one of the most fundamental architectural principle impairing creativity in my opinion and this is what I’m alluding about with this statement. And there is a reason for this: the SDK inherits the mindset of the past, which is the mindset ACE developers had when they initially build the SDK for internal use which they later opened partially to the public as the official SDK. Here is what I mean by this:
In the Q&A they are saying FS2020 is a platform like Windows, and the add-ons are the applications like Word. This is not true nor representative to me. If I’d keep the same analogy, here is what it would be instead:
FS2020 is Excel
add-ons are Excel documents.
add-ons can only use the limited set of programing functions included in Excel.
the set of programing functions available are those selected for an Excel purpose only.
you can only mix documents if you copy/paste the content of one document into the other, creating a new tab for example.
when you do, you risk formula conflicts and non-working references to the other document.
an Excel document is no more than what the Excel developer thought it could ever be.
as a 3rd party Excel document author, you can never make an Excel document be something else than what it is, a document.
if the Excel programmers are only working with 3rd party document authors doing financial amortization documents, they won’t include statistical functions.
those wanting statistical functions are not listened to nor engaged with because Excel is only sold to financial people.
X-Plane/Prepar3D are Python runtime
add-ons are Python program.
add-ons can only use the limited set of language functions included in Python.
add-ons can extend the set of functions with the use of 3rd party libraries. They can even develop their own libraries to extend the language and use the resources available in the underlying operating system.
a Python program can be many different things the Python language inventor(s) couldn’t ever have imagined their language could or would do.
as a 3rd party Python program author, you can make your program anything you’d like it to be.
P3D Python is offering more functions but at a higher abstraction level (Say DrawCircle() )
XP Python is offering less functions but at a lower level (Say MakeVertexBuffer(), DrawVertexBuffer() ).
the runtime doesn’t need to provide a function for everything, just the minimum necessary for the language to be Turing complete. If it is, they don’t need to custom implement a new function because 3rd party Python program authors can do their own.
This specific Flight Simulator SDK vision consisting in “an aircraft is the container” (the document analogy above) is very strong. In practice I find this is reducing the scope of the gauge and system SDK in my opinion, and I can clearly see this impact already in current FS2020 add-on developments. For example:
It is hard to add a 3rd party gauge in any aircraft, like the G1000 WT gauge. You can’t add the gauge in any of you aircraft easily because this requires changing the aircraft container because the SDK vision is a self-contained entity, the aircraft, rather than a composable set of modules.
If you do find a way to add the G1000 WT gauge, you risk conflicts with the aircraft autopilot system, because both can only assume, in their own right because of how the SDK is imposing this on them, they are the sole entity in the aircraft controlling the autopilot.
yourcontrols add-on requires the other add-on to publish its own list of L:vars so that it can use them for interactions. It would be much easier with introspection APIs and a modern (i.e. non inherited from FSX) event system* (1)
These are just a few examples which are equality applicable to other simulators:
to P3D for similar FSX inherited vision in the SDK (L:vars for events, aircraft container vision), but at least P3D is offering much lower level features and is not sandboxing add-ons so that 3rd party developers can make solutions to addressing this.
X-Plane on the other hand is not suffering any of these, because the SDK is supporting in its very structure and feature set the composable modular concept vision. It is a much richer environment for developing add-ons even if the set of functions is much smaller in number, because they are lower level and therefore they are enabling more.
In my opinion I find the idea to lower the entry barrier to developing add-ons for FS2020 with JS/HTML and the in-sim SDK tools commendable and good for the franchise. I really think this is good this time with the choice of technologies they are using in FS2020, whereas I never thought this was good before with the “XML Scripting Language” which was at the time meant for this.
However, delivering this idea of a lower entry barrier in using higher abstraction constructs, shouldn’t be an excuse to not delivering lower and less abstracted constructs to more experienced developers wanting to do more than what the higher abstraction constructs is provisioning for.
A good example of this to me is the drawing API. At last, Flight Simulator SDK is offering a low level basic but sufficient vector drawing API and a bliter. This is the Asobo Drawing API, very simple, to the point of building vertices and drawing the path and no need for more (2). They also include in the SDK a nanoVG implementation using this low level drawing API. On top of which they are also including in the SDK a GDI+ implementation wrapping the nanoVG implementation in a thin layer. (3)
So if the GDI+ abstraction thin wrapper ontop of NanoVG ontop of Asobo low level graphics API was good enough for the CRJ and for Microsoft/Asobo to allocating development resources just for this, why shouldn’t the same approach be more widespread for the rest and the other type of add-ons or for other add-on vendors? In other words, there is a need for high level abstraction in order to lower the barrier of entry for sure, but there is also a need for lower level API to directly interact with some deeper entry points into the simulator itself, like in other simulators. In my opinion these are really essential to making FS2020 something greater as a platform than a spreadsheet for financial analysts.
But even before going all the way to this route, there are some very basic fundamental constructs which could rapidly and not disruptively be greatly enhanced. We’re detouring the code from FS9 to P3D5 in order to support these few core concepts and to augment the core simulation capabilities (we don’t need to in X-Plane), and I’m not talking millions lines of codes either. These are very light weight very specific augmented capabilities extending their usefulness to any add-on and could help solving some of the inter-gauge and inter-aircraft mixing and communication problems existing in the FS2020 SDK carried over from the FSX area SDK.
(1) In the X-Plane world, an event is a first class citizen, with a name, any other add-on can discover and trigger, and even intercept and transform. This applies both to stock events and runtime created events by add-ons. Furthermore, X-Plane is encouraging the practice to use path delimited names so that it makes it easier to categorize and lookup events. Add-on created events and stock are therefore displaying side-by-side in a coherent tree view in the GUI for example. Any hardware device can also bind to any stock or add-on event created at runtime because they are at the same conceptual level.
In the FS world however, an event is either a number or a stock name. The number type of event is one feature inherited from the past which is perfectly illustrating that the SDK is not designed with multiple add-ons from multiple sources mixed together from the get go. In effect when you mix another 3rd party add-on, you can’t know whether the other add-on is using the same numbers as you. The named type however is limited to the stock SDK event list. The naming convention is also confusing with a some events adopting a noun+verb and others a verb+noun approach. Even in the same category, events can have disparate names and this is due to successive teams adding events with each new FS version. There is no provision for 3rd parties to create their own named events and therefore, there is no way to publish events other add-ons could consume.
This last point is illustrating the founding concept of an aircraft is a sole container, because there is no provision in the SDK to accommodate mixing add-ons. And this has some serious consequences in practice: because there is no event system, gauges are forced to use L:vars as a substitute. The problem is L:vars are meant to store data add-ons can read and write only, and they can read and write the data when it is their turn to do so. In other words, say you have an add-on publishing an L:var to let other add-ons writing 1 or 0 to “trigger” an event. If the other add-on is triggering the event twice during its turn, the publishing add-on will only see 1 event, the last one written in the L:var. A proper event system requires a FIFO queue and an L:var is only a 1 item FIFO.
(2) They are not documenting whether this is CPU rendering (Asobo, SKIA, Blend2D?) or GPU rendering (Asobo, CoherentGT, SKIA?) though.
(3) I understand they spent the time and allocated resources to implementing the GDI+ layer so that it makes it easier for their 1st party partner to release the CRJ quicker with less effort porting the existing FSX drawing code, but I can’t help thinking 3rd party developer doing WASM gauges should have had only 1 choice, which is implementing their own abstracted drawing API on top of the Asobo low level drawing API. We still can hopefuly, but I feel the SDK GDI+/nanoVG now standard has a catch: not only this is creating technical debt with an old graphics API now in the SDK, but because it is now included as standard, many 3rd party developers with legacy GDI+ code might choose to just use this instead of redoing the drawing code. My experience with add-ons is 80% of the CPU resources most gauges are consuming in airliner add-ons are graphics related, and I can’t help thinking both the nature of the GDI+ abstraction in itself (there might be more efficient abstractions than those of GDI+) and their implementation in the SDK (it is WASM code on the CPU running the thin wrapper around NanoVG on the CPU, which is a heavy wrapper around the Asobo drawing API) is not the fastest way, and therefore, add-ons using the “easy porting way” will continue using more resources than necessary and will continue affecting the simulator performance.
Ideally all databases should be the same no matter what the source. As long as the database is based on the current AIRAC cycle, it should not matter whether the database comes from LIDO, Jeppesen or NavBlue. The database is supposed to reflect the real world, based on the latest data from government AIP sources. This would be no problem if the sim’s default NavBlue database was current and accurate, but at the moment it is not. Missing procedures are aggravating, but that can happen even with real-world AIRACs - sometimes due mainly to FMS memory constraints - especially with older hardware.
But the cardinal sin for a real-world navigation database provider is having obsolete information, and NavBlue’s MSFS database has many RNAV named waypoints that were removed from the US National airspace system years ago.
For myself, I use Navigraph both for the core sim nav data (overriding NavBlue), and for the CRJ. It would be great if all aircraft in MSFS could simply use the provided free NavBlue database for all navigation, but unless NavBlue gets its act together, I don’t see that as a viable option.
If only @CptLucky8 could spend more of his time discussing these subject DIRECTLY with ASOBO, rather than having to post them in the Forum, hoping that Asobo may read them – but never receiving any reply of acknowledgement - as it is a “one way” communication.
**What a wasted resource of FS Knowledge
MSFS could be so much better if ASOBO were to communicate directly with @CptLucky8
I would very much like to see a technical Q&A were @CptLucky8 was the Technical Host, talking to the ASOBO SDK team
Very true. One of the most knowledgeable people on this forum. One really has to wonder why Asobo would not reach out to someone that has so much to offer?
@Jummivana Has Asobo taken notice of @CptLucky8 yet? Are there any plans to reach out to him, considering the vast amount of technical information he has already shared on these forums? Why is someone this knowledgeable and willing to help being so consistently ignored by Asobo, especially considering he has reached out via the official channels?
Meanwhile, all we can do is upvote this: Reality-XP
There is a absolutely large amount of information, but much of it makes a number of assumptions that don’t necessarily track.
The reality here is that named events already exist, they’re H events. Could they be more discoverable? Yes, that is known. Could a control system on top of all this be more abstracted? Yes, that is also known and is being worked on as we speak. We are also talking daily about how to evolve the API away from only being heavily var based. So, we have a pretty good awareness of those kinds of limitations, although it should be known that most of the third parties that exist in the simulator sphere at the moment are more heavily weighed towards wanting things to be as they are. For example, the reason the GDI+ API was brought forward was because third parties demanded it intently.
I share Jean-Luc’s personal opinion that some of these things should be effectively deprecated but that isn’t the majority third party view and they need to be supported. So we have to balance everyone’s interests. My goal is to democratize development, especially on the JS side, but it’s going to happen over the course of the year, and not instantly.
The general thrust of many of the comments is about building DLLs and getting direct GPU access. Those things are not on the roadmap. The sim API is not designed to use the sim as basically a glorified graphics engine or hook in arbitrary native code. Those things are just not in scope, due to sandboxing and security, and the team’s position on that is unlikely to change.
If folks have specific graphics API requests and examples of things that are impossible to draw at the moment, then those would be good to bring forward. But presently, and I know it’s disappointing to some, direct hardware access in the form of arbitrary native code loading and/or GPU access is not coming.
Are these only the third party developers recognized by Asobo, or does this include all third party developers, who may or may not be in communication with Asobo, that have expressed their opinion on the matter?
In order to improve third party access to the SDK and to extend the number of developers (which i’m sure we can all agree is a good thing), shouldn’t an emphasis be placed on the developers that currently cannot bring their addons into the sim? Sure, the ones that are already in the game might want to continue the status quo, but is that the right thing to do if it means it locks out other developers? Wouldn’t a middle ground be more favourable?
As for security, I still don’t see how sandboxing improves anything, considering a user is infinitely more likely to run malicious code during addon installation than while the addon is ran by FS2020. But, then again, i’m a nobody so Asobo doesn’t need to explain themselves to me, I suppose. It wouldn’t even be an issue if it wasn’t limiting addon capability. And, again, @CptLucky8 has proposed solutions on how sandboxing can be maintained whilst enabling dll-like functionality, all the more reason why I personally think he should be brought into the discussion:
If this is the Official word, then its good to hear it spelt out so clearly and with ambiguity.
So, unlike FSX, P3D & XP, we are not lightly to see support for the Trainer Garmin GPS system, as developed by RXP … so no point in expecting them unless there is some major change of direction by MS/ASOBO.
Time to accept that an move on … (probably to XP.)
I don’t think it’s the SDK or the sandboxing that is preventing those addons from coming to FS2020. Those make things complicated, for sure, but I think it’s the autopilot interactivity that’s the blocker, if i remember correctly.
No, of course not. A lot of this feedback is making it back up. We (WT) were a prime example. We are way outside the usual third party viewpoint. We were brought on specifically to build out some parts of the SDK and make them modernized and add our viewpoint to the mix. And this plays into your second point. As I mentioned, my goal is to democratize a lot of this with better tooling and APIs, but it is going to be evolutionary over the next year or two as there’s a lot of work to be done. And the new AnswerHub, which will be more open, will also enable better MSFS team to developer communication. There is definitely an awareness that better back and forth is needed, but doing so via the forums has become drinking the firehose, so to speak.
Sandboxing is industry standard these days; allowing processes like games that have fairly low level access to load arbitrary untrusted code is just not a good idea. Just look at the drubbing Cyberpunk took for their code load vulnerability. Consumers expect that third parties can’t load things that are unsafe. I understand that from a practical perspective that hasn’t been a problem yet (although there are a couple of historical FSX examples, they aren’t the rule), but regardless it’s important in 2021.
I agree that having some sanctioned, safe, and high performance communication solution would be a good thing. We’ve been discussing that in the context of how to move data between the JS side and the WASM side, for example. There are already some solutions getting data in and out: on the JS side you can run an external process (it doesn’t even have to be local, could be anywhere on the LAN or WAN/cloud) and communicate to JS via websockets, for example. On the WASM side you can use SimConnect client events to transmit data. The SimConnect client events solution is a bit limited, of course, but those are some options presently if you need to move data to external processes.
We are actively working on making the sim autopilot and associated vars much easier to work with and write to for overriding scenarios. I don’t know when those features will land yet, but it is something we are definitely tackling.
Speaking as someone who is presently working on producing “study level” Garmin instruments, I think the idea here is to develop high quality instruments that don’t require communication with the real Garmin trainers. Forwarding data in and out of the Garmin trainer itself is enormously limiting, as an architecture:
The trainer DB is hard to update (for some trainers not possible) and doesn’t sync with the sim nav data
The trainers themselves have a number of bugs and love to crash (NXi trainer, I’m looking at you)
Getting access to the trainers is difficult (or impossible, for some units) and sometimes the trainers are out of date compared to the actual plane installs or have wrong options
The performance of the trainers is not always very good compared to what can be achieved with good code in the sim
If folks want to make really awesome GPS units without compromises, I would definitely recommend avoiding the external trainer approach. But that’s just my opinion, of course, folks are always totally free to disagree.
Whilst i would agree that the best possible solution is to have a fully featured, 1:1 Garmin clone built with native sim code, I don’t think that’s enough reason to limit the access other developers have to the sim. After all, who can compete with you right now when it comes to marketing a G1000? You have privileged access to not only the sim internals, but you can change the course of the SDK. With great power comes great responsibility
Further more, just because some other addon built using some other methodology might have shortcomings, is that reason enough to block it? The WT G1000 currently has all sorts of shortcomings, including a hard to update database, and that it doesn’t completely sync with the sim nav data, and there are a number of bugs due to the limited implementation, and a lot of people report a significant performance impact when using a glass cockpit as opposed to steam dials. But no one said you should give up on it. We all encouraged you to keep working on it to make it better and to weed out those issues. And that is also why we all are happy that you joined Asobo, in order to produce a better G1000. But now you’re using those shortcomings to cast other developers in a negative light. That’s hardly fair. Why not think about the situation you were in before joining Asobo? With an incomplete simulation, with a wall set in front of you, blocking you from fixing some issues. Shouldn’t other developers be given at least a tenth the opportunity you got? I don’t see how this would not be beneficial for all parties involved. Because, after all, I don’t believe that this is some petty attempt at monopolizing the Garmin implementation market. I think we’re all better than that, i hope.
Again, this isn’t necessarily about you fully embracing @CptLucky8 and declaring him your beloved leader. No. This is about him being given a chance to make his argument in an official setting, with the Asobo devs, rather than on this forum with the rest of us average mortals.
Either way, i said my piece, and i won’t continue with this further, unless you have any questions for me personally. I just hope that in the same way you guys were given a massive opportunity, a very good decision made by Asobo that I appreciate, maybe more developers can be brought to the table to at least offer their opinion, especially when they have proven themselves as knowledgeable as @CptLucky8.
I think this is hardly warranted. To be super fair, our position was always that the sim was already capable of doing most of what you needed to make study level instruments, and we were able to work extremely well to show that was the case with the CJ4. Our opportunity was created because we wholly embraced the new way of working, which does, unfortunately, include a sandboxed environment. I agree with you that our G1000 is not finished (although it does use the native flight plan and AP, so it actually does sync up with the sim nav data and has no database of its own), but we’re also going to be using our learnings while building the next big version as takeaways for how to make a framework that leads people into the “pit of success” as I call it, where it is clear how to get great performance and work quickly.
I’m a huge fan of Jean-Luc, I’ve owned a number of RXP products over the years, it’s all really really great stuff! Me pointing out those architectural limitations isn’t an attempt to downplay the high quality that he has produced. Just pointing out that there are indeed other avenues for creating 1:1 recreations of those instruments that don’t involve those same architectural limitations, which may be worth considering and pursuing. Yes, the accuracy of the awesome RXP trainer based products is amazing, as are the technical achievements, but just because that specific way of working is not possible doesn’t mean that the same fidelity is not achievable via other means. And we’re happy to speak directly to Jean-Luc about his use cases so he can get his goals reached in ways that don’t require direct hardware or native code access.
My job with the MSFS team is absolutely to open the most doors possible for making development modern, easy, and streamlined with the fewest pitfalls. But part of that modernization is definitely a team focus on security, and so sometimes there are just going to have to be other ways of solutioning. That’s not me purposefully putting up walls to box out “competitors”, that’s just a reality of modern game addon design that we all have to deal with, myself included.
As an aside, you don’t have to speak with me long to hear me rue the lack of information sharing and high level of boxing out that is the standard in the FS third party community. My goal is to slowly erase those knowledge gaps and even the playing field for everyone, so that everyone from the two decades old study level gurus to the first time devs have the same opportunity to make something great for simmers. Yes, there will sometimes be non-negotiable limitations, but the goal is to make it possible to still achieve super awesome things even given those, for everybody and everyone.