Detaching thread quits entire script when using SimConnect_CallDispatch

Hey there together,

I need a little bit help pinpointing me to what I’m doing wrong. I’m trying to using a detached thread to be able to call my dispatch function in a non blocking loop.

I’m starting my thread with

std::thread dispatcher(startDispatch,
                           std::ref(simConnection),
                           std::ref(stopThread),
                           std::ref(dispatchIsRunning));
    dispatcher.detach();

and call my translation unit which handels the dispatch calls and the dispatch itself (so it’s not running in main())
My loop looks like this

void startDispatch(SimConnection &simConnection,
                             std::atomic_bool &stopThread,
                             std::atomic_bool &isRunning)
{
    isRunning = true;
    int counter = 0;
    isConnected = simConnection.isConnected();
    quit = 0;
    HANDLE hDispatchSimConnect = simConnection.getSimConnect();
    HRESULT hr;
    while (quit == 0 && !stopThread)
    {
        hr = SimConnect_CallDispatch(hDispatchSimConnect, dispatch, nullptr);

        counter++;
        if (counter >= 30)
        {
            quit = 1;
        }
        // below should be used in a later state to quit the loop / thread when the connection to SimConnect is lost
        if (isConnected != simConnection.isConnected())
        {
            simConnection.setIsConnected(FlyShare::isConnected);
        }
        std::lock_guard<std::mutex> guard(mtx);
        // for now hold at least 1 sec for each iteration
        Sleep(1000);
    }
    isRunning = false;
}

At the end of my main() I’ll await the thread with

    stopThread = true;
    while (dispatchIsRunning)
    {
        continue;
    }

This is most likely a temporary solution on my way to a proof of concept and is just a CLI implementation (I want to use a GUI later on when I am able to handle the most basic test cases).

My problem is that after compiling in Visual Studio the CLI runs just fine (without being able to connect as I have my sim on another machine, os this is expected).
As soon as I run the .exe with my sim running it runs for ~ 3 seconds after which it silently dies (just closes).
I testwise slept for 2 secs before calling the thread which then showed that my loop actually does it’s job, BUT

  • when I remove SimConnect_CallDispatch the script works and the loop runs flawlessly
  • when I use .join() instead of .detach() (which is undesired as it is blocking) the loop also works

Could perhaps somebody explain to me what is going on here? I have a 10 sec sleep at the end of my main to keep the CLI open during testing / debugging until switching to a GUI which suits my needs as I’m just checking if the communication works in different parts. This 10 sec sleep is also ignored combining a detached thread with DispatchCall.

There is much room for improvement here, e.g. a better way to hold the loop execution other than sleep but my goal right now only is to be able to use the polling loop.

Thanks in advance,
Phil

Perhaps the sleep is not a good idea, I guess the thread has to be aware of main being alive … ???
Just a thought

I’m probably just blind again :slight_smile: but what does dispatch (which is called by SimConnect_CallDispatch) do?
And there are what seems like some global-scope variables (or class-scope if this is in a class) which could cause the loop to exit? Not clear… or why there’s both quit and stopThread which seem to do the same thing. Or why some vars are passed in by reference but others are from enclosing scope. Or what that mutex at the end does…? :slight_smile:

And where is the SimConnect handle coming from? Does whatever provides it also run a message dispatch loop?

When starting SimConnect myself I pass it a WaitHandle (from a CreateEvent() call) and use that in my threaded loop to monitor for messages, instead of any sleeping. The handle is a Win32 thing (not “standard” C++) but so is SimConnect so that’s a wash. I use another wait handle to trigger my dispatch loop to exit, so in the loop I can WaitForMultipleObjects() and either process the SimConnect message or quit the loop, depending on which event fired. But if your SimConnect instance is started by some other process, that probably doesn’t help you.

But in any case perhaps a wait handle (CreateEvent()) would help you keep the thread alive. There are probably better examples out there, but you could peruse one of the example console apps I just published and look for usage of the g_dataUpdateEvent handle which is used to pause the app while waiting for data to arrive from another thread. (Also the WASimClient in that repo uses the message dispatch method I described above, if you’re curious).

Cheers,
-Max

@Maximal6379
Hey Max back again, great :slight_smile:

You’re not blind, just missing context. I omitted this part as I just sticked to the docs, but maybe it’s better to show it.
And if you wonder why I declare the pointers before defining them: VS won’t compile if they’re declared and defined in one go, always wondered why it works for others (e.g. you).

void CALLBACK dispatch(SIMCONNECT_RECV *data, DWORD cbData, void *pContext)
{
    switch (data->dwID)
    {
    case SIMCONNECT_RECV_ID_OPEN:
        SIMCONNECT_RECV_OPEN *openData;
        openData = (SIMCONNECT_RECV_OPEN *)data;
        isConnected = true;
        break;
    case SIMCONNECT_RECV_ID_EVENT:
        SIMCONNECT_RECV_EVENT *evt;
        evt = (SIMCONNECT_RECV_EVENT *)data;
        std::cout << "evt: " << evt->uEventID << "\n";
        break;
    case SIMCONNECT_RECV_ID_EVENT_FILENAME:
        SIMCONNECT_RECV_EVENT_FILENAME *acft;
        acft = (SIMCONNECT_RECV_EVENT_FILENAME *)data;
        break;
    case SIMCONNECT_RECV_ID_SIMOBJECT_DATA:
        SIMCONNECT_RECV_SIMOBJECT_DATA *objData;
        objData = (SIMCONNECT_RECV_SIMOBJECT_DATA *)data;
        break;
    case SIMCONNECT_RECV_ID_SIMOBJECT_DATA_BYTYPE:
        SIMCONNECT_RECV_SIMOBJECT_DATA_BYTYPE *objDataType;
        objDataType = (SIMCONNECT_RECV_SIMOBJECT_DATA_BYTYPE *)data;
        break;
    default:
        break;
    }
}

var isConnected is global in the translation unit so I can communicate this back to the SimConnection class (I’m trying to contextual split my code to avoid having to much in one file, but thats another topic I guess :smiley: )

The reference vars are only for talking back to the main() function (or main thread if you wish) to know in main() if the thread is still running and to talk to the thread from inside main() to stop it’s task.
&simConnection is a reference to my “connector class” to get a reference for the SimConnect handle and to later be able to set its connection status based on events (at least in my mind).

This is where the handle is coming from:

SimConnection::SimConnection()
{
    SimConnection::m_isConnected = false;
    m_hSimConnect = nullptr;
}
// ....
HRESULT SimConnection::connect()
{
    HRESULT hr;
    // just to store the connection result for return
    HRESULT connection;
    connection = SimConnect_Open(&m_hSimConnect, m_appName, 0, 0, 0, 0);
    // trying to fetch RECV_OPEN after connecting
    hr = SimConnect_CallDispatch(m_hSimConnect, dispatch, 0);

    return connection;
}

When starting SimConnect myself I pass it a WaitHandle (from a CreateEvent() call) and use that in my threaded loop to monitor for messages, instead of any sleeping.

Fun thing is: I actually took a look at your code which reminded me of the std::mutex from one of my c++ courses I took :smiley: I’ll still have to wrap my head around those WaitHandle and CreateEvent() part, thats absolutely new to me.

What I do not understand is why .join() works with the thread and the problem occurs only with .detach() (detach works btw if I do not use SimConnect_CallDispatch in the loop. If it’s just debugging text or even checking for the handle all this stuff is working with detach).

I carefully tried to monitor possible terminating conditions but couldn’t find any that would prematurely / unwanted terminate the thread.
Even if it WOULD terminate unintended then I would expect my test Sleep() at the end of my main to keep the console open.

So you could it some up with: .detach() + SimConnect_CallDispatch = termination after ~3 secs (without any message from the actuall callback function - .join() delivers the messages)

Cheers,
Phil

Edit: Oh and for the mutex, just to put the thread on hold after each iteration to hopefully prevent race conditions. I hope thats kinda the right direction I’m moving.

@SimFlight2020

Perhaps the sleep is not a good idea, I guess the thread has to be aware of main being alive … ???
Just a thought

There sure is a better way than just sleeping, if I .join() the thread the sleeps works, though. The thread itself is not aware of main() being alive (and I don’t know if it could somehow) but I am telling the thread at the end of main() that it can finish it’s work and wait until it’s done and after that main() exits. So yeah… :smiley:

1 Like

OK that’s a bit to digest… :slight_smile: I’ll try to get back to you with more if I see anything…

But all else aside, I’d definitely recommend using the WaitHandle functionality provided by SimConnect for just such occasions (when a Window handle isn’t available, like in console/background apps).

The WASimClient uses such a handle, and also not too long ago I “fixed” John Dowson’s WASM client which was using a similar sleep loop as yours instead of a wait handle… it may be easier to read the diff than trying to follow all the code in my client: Use an event handle to check for pending SimConnect messages instead … · jldowson/FSUIPC_WAPI@cccd8dc · GitHub
EDIT: But see my recent comment on that commit and do not give the Event a name.

The important (and maybe not obvious) difference between “waiting” and “sleeping” is that you can wake up a waiting event at any time, vs. in sleep you’re basically just stuck there (or have to do a tight loop around a very short sleep). So the code can “wait” as long as it wants w/out needing fast loops or sleeps, as long as there’s a trigger to cancel the wait (or in case of SimConnect to notify that there’s data available). And one can wait on multiple events at once, like my example of a “exit thread” event alongside the SimConnect “message ready” event.

HTH,
-Max

When creating new variables inside a switch/case, you need to wrap them inside a code block:

    case SIMCONNECT_RECV_ID_OPEN: {
        SIMCONNECT_RECV_OPEN *openData = (SIMCONNECT_RECV_OPEN *)data;
        isConnected = true;
        break;
    }
    // or inside an if block, etc... contrived example:
    case SIMCONNECT_RECV_ID_EVENT:
        if (SIMCONNECT_RECV_EVENT *evt = (SIMCONNECT_RECV_EVENT *)data)
            std::cout << "evt: " << evt->uEventID << "\n";
        break;

Honestly I can’t tell exactly why it may be exiting… there could be a dozen reasons. The code you post isn’t complete (what happens in main() ? How is the thread launched? How are you waiting for it to finish?). But there is a lot of code that doesn’t do anything, which never really helps with debugging (or trying to understand someone else’s issue). As they say on Stack Overflow… “minimal reproducible example” :wink:

Have you run with the debugger attached? Just set a break point and step through what’s going on… ?

I don’t understand this… are you using the mutex somewhere else also? The thread is going to sleep and nothing is going to interrupt it at that point (until the next loop iteration).

Best,
-Max

I created a absolute minimum testcase just to see if newly created event works (which didn’t in my current loop) and everything is a single file now instead split up by context. Guess what, I can declare and define the vars even without a code block like shown in the SDK.

The code you post isn’t complete (what happens in main() ? How is the thread launched? How are you waiting for it to finish?).

In main() nothing exiting happens, I’m basically just debug logging (which I didn’t paste here on purpose) and subscribing and unsubscribing from system events. How the thread is started and awaited at the end is in the first post :stuck_out_tongue: There might be a better way of awaiting threads (I’ve seen a CreateThread in FSUIPCs code you’ve linked me, maybe better than std::thread)

Do you mean the DispatchProc?

I have a “slightly” limiting factor here. If I read it correct I would need VS and MSFS running on the same machine. I’m coding on one machine and using MSFS on a second (just to keep the MSFS system as clean as possible). So unfortunately not.

There’s a stupid error (which isn’t the reason for the console exiting, though, as I didn’t use it before). I did not use it anywhere else hence it’s kinda useless…

What I don’t understand now is: As I’ve said in the beginning, I created an absolut minimum file, just with one RECV_ID_EVENT and everything is in one file. Loop is running and working, std::thread is working with .detach().
Only difference: I didn’t use any unsubscribe or disconnects in this file. From my point of understanding using them wrong shouldn’t prematurely kill the console though :man_shrugging:

If it doesn’t bother you I would paste the complete files (which are basically just 2, leaving out the headers) stripped by any debugging outputs. Maybe you can then slap me in the face, because I did something really stupid :smiley:

Cheers Phil

Edit: I (slightly hopeful) say “disregard”. I moved my disconnect() (which basically terminates the SimConnect connection at the very end (please don’t ask why it was right below the thread start, sounds stupid looking at it…). Right now the loop seems to work, I’m getting my test messages and the console keeps running until my “timer” ends it.

Next step would propably be to study the WindowHandle you mentioned (I think it was this term) as I’m planning to use Qt for a GUI.

Uhm just one last question for now: I did understand it correctly that SimConnect basically wakes up the “CreateEvent(…)” event which is awaited for with “WaitForSingleObject(…)” as soon as it has data available?

Hi Phil,

Is this a WASM module you’re writing? I thought it was a command-line client… in which case you can debug it anywhere. I’m not sure what you’ve read, but it’s just like debugging any other program.

If it doesn’t bother you I would paste the complete files

I always like reading code :slight_smile: Here would work, but maybe something like a GitHub Gist would be easier to read. But if you have it sorted already, cool.

I did understand it correctly that SimConnect basically wakes up the “CreateEvent(…)” event which is awaited for with “WaitForSingleObject(…)” as soon as it has data available?

Exactly! You literally get each message quicker because there’s no sleeping between each iteration.

And WaitForMultipleObjects() can wait on several handles at once, eg. the SimConnect one and your own “exit thread” event to shut it down (that’s not in the commit link I posted earlier, but I do use that in WASimClient::dispatchLoop).

HTH,
-Max

I’ve read that you have to hook / select the flightsimulator.exe to be able to use stop points. I also want to develop a WASM module to access LVars. Right now it is command-line for testing purposes. Later on I want to use a GUI only.

That’s a good idea, I will come back to that if I have more to show that isn’t just fiddling and testing :smiley:

I’ve came across that looking through your code. Honenstly event handling is entirely new to me so I might take some time figuring it out and then be able to use it in a Qt app (or to be able to use “normal” handles instead of HWND handles inside Qt).

Thank you so much for your help :slight_smile: Essentially you put me in the direction to figure out how to unblock the dispatch loop :slight_smile:

Cheers,
Phil

That’s just the WASM modules. And I’ve yet to be able to “hook into” those at all. The whole timing thing is pretty absurd. Oh and using their JavaScript style “debugger” (Coherent GT) just crashes the whole sim if you click “the wrong thing.” So yea debug on WASM is old-school, but for any external app there’s no dependency on the flight sim… it doesn’t even need to be running. Really… I debug this stuff all the time.

Qt is great for GUI, indeed. And while you can get a native Window handle from Qt, and run the same message loop as in any other Windows program (handling the window messages in the huge switch/case :slight_smile: I’d personally choose not to run SimConnect on the GUI thread since that doesn’t really make any sense anyway… the GUI needs to react to the user, not process network transactions. IMHO, of course :wink: And Qt makes it very easy to do cross-thread communication with the queued signals/slots. The WASimUI relies on that quite a bit (the Client delivers callbacks from multiple threads, possibly concurrently).

Anyway, you’re very welcome!

Best,
-Max

Yeah I think I remember :smiley: I just thought when I read this statement somewhere “Oh so no real debugging for me” :smiley:

Thanks for the input :slight_smile: I just have to get Qt Creator running now. I’ve only used it for small projects during online courses so far without external linking. Compared to VS where you can just set the include path and dependencies from my novice few Qt (both qmake and cmake) looks quite complicated with manually editing CmakeLists and stuff, but … totally different subject

Yea, veering off subject, but it’s your thread… :slight_smile: Regarding Qt…

  • Stay away from Qt6, at least for now. They’ll try to push it on you but it’s not ready for prime-time.
  • With Qt5 you can use qmake or CMake, and QtCreator does have a full project manager setup GUI for both build systems (though, sure, there will be some learning curve).
  • One can use MS VS for all Qt work, just install the Qt VS Tools extension (and a version of Qt of course). Personally I way prefer the QtCreator editor, but I used VS for my last project and it does work fine.

Cheers,
-Max

Done. I actually had Qt 6 :smiley:

You mean I can “create” a design with Qt Creator and compile in VS? As Qt really tries to kill me :smiley: I somehow managed to set an include path for SimConnect with CMakeList but when compiling I’m just getting Linker errors. After ~ 7 hours I didn’t figure out how to correctly setup CMakeList as we do in VS, despite any documentation or SO questions. Sigh.

Cheers,
Phil

Actually the Qt VS Tools extension lets you run the QtDesigner tool (the form creator GUI) right inside VS. Or you can also run it externally, as a standalone thing (optionally launched from VS as well). Or yes, in QtCreator you can just edit any UI form as a standalone file. But you don’t even need QtCreator (the Designer is installed with each Qt library distro, separate from QtC). It’s a bit unfortunate that the Qt installers all force QtC to be installed.

Of course the form designer is also technically optional… I do most of my layouts in code now (though the WASimUI is an exception because it’s actually an absurd amount of stuff on one form). :wink:

As for SimConnect and CMake/qmake, just adding in the include path isn’t enough, you have to also add linker directive pointing to the SimConnect libs (the static one or dynamic, whichever you’re using). Same with VS project though.

Cheers,
-Max

Oh that’s quite nice :slight_smile:
I managed to link SimConnect.lib now but only with the legacy include_directories and link_libraries as the target_… versions don’t seem to find the files.
And actually I don’t know what I’m using… :smiley: I’m linking to SimConnect.lib inside the /lib folder (I don’t know the difference between this file and the ones in the subfolder /static and /managed)

Cheers,
Phil

Edit: With QT VS Tools this actually works like a charm and I can just use the SDK setup for it :heart_eyes: And yet again, one step further thanks to you

The libs in the root folder are for using the DLL (the DLL would need to be available to the application at runtime). The libs in /static are for static linking (the code is built into your executable, no DLL required). The /managed ones are for C#/.NET. Typically you would link to a lib by adding it’s path to the libs search path (in linker options), then add the lib name w/out the extension as a linker dependency.

:+1: