Has anyone gotten SimConnect to repeatedly get data?

Well, that’s exactly what I would expect, but unless I am totally blind or stupid (or both) we seem to talk about different variants of SimConnect_RequestDataOnSimObject!

The one I am referring to:

https://www.prepar3d.com/SDKv5/sdk/simconnect_api/references/simobject_functions.html#SimConnect_RequestDataOnSimObject

does not have such a callback parameter! And again, all the provides examples seem to suggest that you need to poll for messages, with SimConnect_CallDispatch.

UPDATE: And just to be clear: the SimConnect.h header file from the FS 2020 SDK (the previous iteration - haven’t downloaded the SDK from the “December update”, in case that’s different anyway) does not have such a “callback parameter” either: so it seems to be consistent with the “Prepar3D” API docu (even though I understand the current FS 2020 SimConnect.dll supports way less functionality than the Prepar3D API).

Examples given:

All those examples have a “while(true)”-kind of loop which keeps polling for messages in 1 second (sleep(1)) intervals.

So unless you can point us to a concrete example or API documentation I just don’t see how to avoid this polling code pattern (except this funky “DLL start” voodoo magic, which I have not tried yet and which may only work with the Prepar3D specific implementation anyway…).

In fact, I looked at several “wrapper libraries” how they implemented their callbacks, and just now I also looked at this Python library (over which I already stumbled previously as well):

And lo and behold:

Python-SimConnect/SimConnect.py at master · odwdinc/Python-SimConnect · GitHub

 def _run(self):
	while self.quit == 0:
		self.dll.CallDispatch(self.hSimConnect, self.my_dispatch_proc_rd, None)
		time.sleep(.002)

A while loop with a sleep call - bang! That’s polling. :slight_smile:

And to clarify: I am totally not trying to “defend” this pattern - I am merely trying to find out how else it could be done (“without polling”), that is, to get the SimConnect.dll logic have my callback called without me having to poll for messages (which kind of defeats the idea of a callback in the first place).

Steeler APOLOGIES I totally misread the RequestDataOnSimObject() docs - you are of course completely correct and my description was mistaken.

No worries :slight_smile:

Well, there might be another way to react to messages received by SimConnect.dll, and that is the Win32 event messaging:

From the docs:

General Functions

There are 3 parameters related to Win32 event handling (which requires a hWnd instance):

hWnd
[in] Handle to a Windows object. Set this to NULL if the handle is not being used.
UserEventWin32
[in] Code number that the client can specify. Set this to 0 if it is not being used.
hEventHandle
[in] A Windows Event handle. A client can be written to respond to Windows Events, rather than use a polling and callback system, which can be a more efficient process if the client does not have to respond very frequently to changes in data in Prepar3D.

In fact, the API documentation seems to reiterate on the point that the alternative is indeed polling.

So instead of providing a callback function and polling for messages with “dispatch” one can apparently “listen” for (user defined) events injected into the Win32 event queue (which otherwise provides access to “window events” such as mouse movements, repaint events etc.).

But as the documentation also indicates this may only be preferrable if few events are expected. But if one is going to permanently track the aircraft’s position (possibly every “frame”) then the polling approach indeed seems to be the way to go…

I have yet to try this “DLL start” approach (which apparently “automatically” calls the callback, without the need to repeatedly call “dispatch”).

Sounds intresting. Please let us know if you manage to get this Win32 event approach working.

Sure thing, I’m updating my current work skeleton code now. :smiley:

I wrote a little example to show how i use RequestDataOnSimObject()
If somebody has a better way (without calling dispatch all the time) to do it please let me know…
I’m curious about Vibstroniums example.

This code prints the throttle lever percentage to the console only when the values are changed:
(boost is just used to format the prints. if you don’t have boost installed just remove the formatting of the prints.)

Currently very bare bones just connects to the sim and queries alt/lat/lon of the user and reports it back. It’s working with the single request on a 1-second period, done with C#.

It looks like you have a single-threaded app with no event loop, so you’re just spin-looping on SimConnect_CallDispatch even when there’s nothing to handle; this will ramp up your CPU usage to max out one core.

SimConnect uses a win32 window message event as the inter-process trigger to tell you there’s new messages to read and dispatch to your callbacks; thus it piggy-backs on the Windows event loop and your app can stay idle while waiting for new data to arrive.

You should be able to create a single hidden window (bare win32 calls should be fine if you don’t want to play with modern frameworks) and pass its window handle and a preferred window message ID from the user area (the examples all use 0x0402) to SimConnect_Open. Then in your window procedure, when you receive that window message call SimConnect_CallDispatch until it returns 0.

1 Like

Thanks for the pointers. I will look into those windows event loops.

1 Like

Thanks Vibstronium awesome stuff…

Are you sure about that? Or asked differently: don’t you get a user “window event” per message? Because according to the (Prepar3D) API docs:

https://www.prepar3d.com/SDKv4/sdk/simconnect_api/references/general_functions.html#SimConnect_CallDispatch

The function returns an HRESULT. Possible values include, but are not limited to, those in the following table.

Return value Description
S_OK The function succeeded.
E_FAIL The function failed.

And given that:

#define S_OK ((HRESULT)0x00000000)

so a return value of 0 simply means that the “dispatching” has succeeded (and not “no more messages to process”).

Also in your example - thanks for that - your comment about the C++ case (// In C++ you’d call SimConnect_Dispatch() in a loop here) is asking for the same question: “How long would you loop? What would the loop termination criterium be?”

Anyway, what I am actually trying to get at is the following: assuming I would setup the data retrieval for some given object, say the user’s aircraft, and asked the data to be updated (sent to the client) for every frame, as in:

res = ::SimConnect_RequestDataOnSimObject(d->m_simConnectHandler, AircraftPositionRequest, AircraftPositionDefinition, SIMCONNECT_OBJECT_ID_USER, SIMCONNECT_PERIOD_SIM_FRAME);

So messages would be sent for every “simulation frame”. And let’s further assume that I would (want to) call the SimConnect_CallDispatch only every second, or for the sake of argument, even only every 10 seconds.

So the questions:

  • Are the received messages buffered (at least for a certain amount of seconds / buffer size) in SimConnect.dll?

  • If so, is my callback called for every buffered message (since the last call to SimConnect_CallDispatch) when I call SimConnect_CallDispatch only once (“once” as in “once every ten seconds”)?

  • Or do I have to call SimConnect_CallDispatch until “no more messages are there to process”? -> But that then brings up the question again: how to detect that there are no more questions (because as already mentioned above: an S_OK = 0 return value simply means that “dispatching was okay”.

UPDATE: Oh, and a follow up question to “receive messages every frame, but dispatch only every 10 seconds”: I just set a breakpoint in my dispatch function and inspected the received data structure (for the reply to the above SimConnect_RequestDataOnSimObject), specifically the SIMCONNECT_RECV * pData structure, but I did not find any sort of “timestamp” information about when the message arrived. Is there a way to check / get such a timestamp, especially if I am processing a message only 9 seconds later?

Because if not, then my intention of polling for messages only every 10 seconds (or 1 second, to be more realistic) is moot anyway if I cannot assign every assigned message (its data) a proper timestamp… (use case: to record the user’s aircraft position).

Thanks!

Yes, I absolutely agree with this assessment:

while (quit==0) {
    SimConnect_CallDispatch(hSimConnect, MyDispatchProcRD, NULL);
}

This loop is absolutely going to burn your CPU :wink: At least but some „sleep“ statement into this loop, e.g. poll at 60 Hertz or whatever.

Also checking state changes of the thrust levers is probably a good example where you should indeed use the „win32 event loop“ („user events“) approach, as thrust levers are going to be changed relatively infrequently. And (only) processing messages once they have arrived would be more efficient, especially if you‘re only interested in state changes.

Ah okay, now I understand where this seems to come from:

SimConnect | Microsoft Docs

In the example given for SimConnect_CallDispatch there is the exact same „burn your CPU“-loop. Bad, Microsoft, bad, bad, bad :wink:

At least the Prepar3D specific documentation (which seems to be derived from the above documentation almost verbatim) added a „Sleep(1)“ statement into those polling loops :wink:

UPDATE: Nope:

General Functions

The exact same code example is given here. But at least the (most) working examples have a Sleep call…

No, you would want to call SimConnect_CallDispatch every time you get the window message, which will be at least once per simulation frame because you requested to get data every simulation frame.

[Update: to clarify – there is likely a buffer (probably a ring buffer with a maximum size) holding the incoming messages, but I don’t think it’s meant to queue things up arbitrarily long; you’re meant to dispatch the callbacks and handle incoming messages as fast as you can.

Also, unless you specifically asked for the current time in them the messages will not contain a timestamp, as far as I know. You should probably dispatch messages immediately, assign a suitable timestamp, and then queue them on your own if you only want to process them every 10 seconds or whatever.]

1 Like

SimConnect uses Windows Messaging to notify your application whenever it has a new value for any of your requested SimVariables (aka SimVars).

When this event is overridden within your code, simply identify if the message came from SimConnect then call the method to instruct SimConnect to call your appropriate method that you registered.

The hardest part is writing the message handler for SimConnect to call initially, as this would typically be built in to Windows objects such as Forms, if you want to create an application designed to run without a Form, you need to create a MessagePumpHandler.

I can provide some sample code in C# on how to achieve this, if it’s required.

This is how you use SimConnect without polling for updates.

1 Like

Yes, I agree. After thinking about it a bit longer, and given the fact that there is no timestamp in the original messages I also believe that the design of the SimConnect API is that you “pick up the message as soon as possible (after it has arrived)”.

This is actually supported by the API documentation (which is basically the same as the corresponding - and still available - Prepar3D API documentation):

https://docs.microsoft.com/en-us/previous-versions/microsoft-esp/cc526983(v=msdn.10)#general-functions (*)

For function SimConnect_RequestClientData:

RequestID
[in] Specifies the ID of the client-defined request. This is used later by the client to identify which data has been received. This value should be unique for each request, re-using a RequestID will overwrite any previous request using the same ID.

As the RequestID won’t change for requests done with a given period (e.g. “per frame” etc.) I strongly assume (without having verified it in practise) that any previously received result message will simply be overwritten - as per the documentation - with any newly arriving response.

But yes, I also assume that this response will be stored in a buffer; large enough to hold exactly one response record (as defined by SimConnect_AddToDataDefinition), until either “dispatched” or the next response for the same RequestID arrives.

Bottom line: so yes, one either needs to “poll” with the same (or higher) frequency as the messages arrive (are expected to arrive), as per the documentation of

https://docs.microsoft.com/en-us/previous-versions/microsoft-esp/cc526983(v=msdn.10)#simconnect_calldispatch

-> “to call this function sufficiently frequently”

or get informed by the Win32 event queue, with a “user event” whenever a message (of any SimConnect type) has arrived (and then call “dispatch”).

The Win32 event queue however is only favourable if events happen rarely (relatively speaking). If events are expected to change frequently (such as the aircraft’s position) then “polling” is more performant (according to the API docs).

(*) This is the original (I assume) Microsoft SimConnect API documentation, included in a “flight simulation development kit” called ESP (whatever that stands for). This product has been discontinued, and Microsoft is most likely exluding those pages from any search engine robot: even with a search like “SimConnect site:docs.microsoft.com” (and more precise keywords) you won’t find this resource anymore (neither with Bing nor Google ;)).

And I think if have now a better understanding of this “DLL voodoo magic” as well!

Again from the API docs:

“The recommended method of writing an add-on is to build it out-of-process, as an application (an .exe file) rather than in-process, as a library (a .dll file). This is because out-of-process applications provide more stability, if they crash they will typically will not crash ESP, and are easier to build, test and debug.”

Or in other words: it is (or was) apparently possible to extend the flight simulator platform (Flight Simulator X, Prepar3D, FlightSimulator 2004 etc. - not sure about FS 2020) functionality with a DLL (“plugin”) which then runs in the same process space as the flight simulator itself!

And now it all makes sense: calling “connect” from within DLLStart() is expected to always work, because at the time the plugin is instantiated (the DLL is dynamically loaded) the server part is already up and running!

And since the entire plugin logic is running in the same process space like the flight simulator I guess that SimConnect_CallDispatch - when called from that DLLStart() function - will register the callback with the flight simulator itself. Hence (I assume) it is the flight simulator (“server”) logic itself which calls our callback, just after the server has sent a new message.

Or put differently: only if we extend the flight simulator platform itself with a DLL (“plugin”) is this “DLLStart() callback registration” pattern possible. But not if we are talking about DLLs which are linked to our own, separate executable (which is the case for me).

1 Like

Yes, thank you - we have already established above (Has anyone gotten SimConnect to repeatedly get data? - #24 by Steeler2340) the fact that with the Win32 event queue and user events you can get informed about new SimConnect messages, without polling. And @Vibstronium has also already posted a working example (Has anyone gotten SimConnect to repeatedly get data? - #28 by Vibstronium) using the Win32 event queue above :slight_smile:

But again: using the Win32 event queue is only more efficient in case infrequent (rare) events are to be handled (the event queue poses a considerable overhead, too):

SimConnect | Microsoft Docs

“A Windows Event handle. A client can be written to respond to Windows Events, rather than use a polling and callback system, which can be a more efficient process if the client does not have to respond very frequently to changes in data in ESP.” [emphasis added]

1 Like

Thanks Steeler,

Apologies if I was covering old ground, I confess I hadn’t read the full thread.

With regards to the Windows Events being inefficient when frequent events are to be handled, I would have to slightly disagree.

I’m in the process of developing a client/server scenario for remote desktops, which requires many variable requests to be handled (each displayed instrument uses at least 1, usually several, SimVars), which are requested once but have many values being supplied by SimConnect.

It all depends on how you handle the WndProc events being raised.

If you are processing each raised event within the same thread or handling each request from start to end, then yes, it is inefficient and costly.

If you’re filtering the events and initiate the handler within its own thread, by invocation, then it’s very lightweight.

When debugging my code, I am monitoring the entire client/server environment, with many instruments being displayed and updated. The server component is by far the least costly in terms of resources and works extremely well.

You would only ever need to handle each event within the WndProc method if it was absolutely time-critical, but given the overall speed of how things respond in an aircraft, I cannot think of a single scenario where you must know of (and respond to) something the exact microsecond it happens - I’m happy for you to disagree.

I intend to publish the Client/Server code soon for anyone to use, once I complete the code changes needed for handling remote images (e.g. a GPS instrument). So my offer to share a snippet of code above is a little premature but was always planned.

The other benefit of using the Client/Server scenario is that it removes the obfuscation and complexities of the annoying nuances when dealing with SimConnect directly.

The code I’m developing also allows access for remote applications by exposing an API to request SimVar values and can co-exist and interact with applications developed in virtually any language, accessible from almost any PC on your network.

It also has many other features built-in, such as auto-notification when FS is running or stopped, caching the latest values for each variable requested, remembering which client has requested which variables, the last time each value was updated, the last time each client was updated, auto-resubmission of previous requests when FS restarts and many more.

I’m hopeful the code will, in some form, be helpful to any future developers wanting to use SimConnect, and perhaps provide a simpler interface for those wishing to avoid dealing with SimConnect entirely.

Dragonlaird

2 Likes