SimConnect + WASM combined project using VS2019

[HBilliet]

Chapter 8: Communicating with the WASM Module through Client Data Areas

I warn you, but this is a very long chapter!!!

We already know that it is not possible to interact with a WASM Module directly from an External Application. The only way to do that is using ClientDataAreas within SimConnect. These ClientDataAreas can be used as shareable memory areas between our External Application and the WASM Module.

In my implementation, I use 3 Client Data Areas and built a protocol between my External Application and the WASM Module. This protocol will act as an interface between my SimConnect Client (External Application) and the Gauge API (WASM Module).

• Send commands to the WASM Module
• Get answers from the WASM Module
• Get values of variables from the WASM Module

Schematically, this looks like the below:

Functions/Methods needed to work with ClientDataAreas

Setting up the ClientDataArea

There is some similarity between the functions for ClientDataAreas and the SimConnect methods we used for A-Vars in Chapter 6: Controlling the “A:”-type variables.

• MapClientDataNameToID
  We can give each ClientDataArea a “human readable name”. This name allows the different SimConnect Clients to identify the ClientDataArea. But internally in the SimConnect Client, the name is mapped to an ID, that is a lot easier to work with. If the ClientDataArea already exists, then SimConnect will send an Exception with the message “ALREADY_CREATED”. In my implementation I just ignore this error.
• CreateClientData
  This creates the ClientDataArea for this client.
• AddToClientDataDefinition
  This is the same as AddToDataDefinition used in Chapter 6: Controlling the “A:”-type variables. Although, this time we are not dealing with variable names, but we use an offset and a size in the Client Data Area.

The below schematic shows how these 3 functions work together (using 2 ClientDataAreas):

It’s important that the above 3 functions are called in all SimConnect Clients that will interact with the same ClientDataAreas, and that the Offset/Size mappings do match. If not you will obviously get unpredictable results.

ClientDataArea ID’s and Definition ID’s are local for each SimConnect Client, which means that you don’t have to use the same ID’s in both SimConnect Clients. The SimObject Definition ID’s and ClientDataArea Definition ID’s are even completely separate, which means that you can use the same ID’s without conflicts.

In the Managed Code (C#), you also need to register the data structure of each definition, as we had to do for the A-Vars. This time, we don’t use RegisterDataDefineStruct, but RegisterStruct. The signature looks as below:

public void RegisterStruct<RECV, T>(Enum dwID) where RECV : SIMCONNECT_RECV;

We’ll see further below how we use this in our code.

Setting or getting data via ClientDataArea

This works exactly the same as explained in Chapter 6: Controlling the “A:”-type variables. Once we have defined our ClientDataAreas and the Definitions of the items, we can start using them.

• SetClientData
  This function takes the ClientDataArea ID and the Definition ID together with the data that you want to set. Once the data is set, it can be read by the SimConnect Client on the receiving side.
• RequestClientData
  This function takes the ClientDataArea ID, the Definition ID and a Request ID that will identify the data that we are going to receive (again, this works asynchronously). The requested data will be sent sent to the EventHandler OnRecvClientData. (Be aware that for A-Type variables, we used RequestDataOnSimObject and results were sent to the EventHandler OnRecvSimobjectData, so it is slightly different). We can use the same concepts using Period (example: FRAME) and Flags (example: CHANGED).

Protocol between External Application and WASM Module

Before we dive into the code, let me first explain the protocol that I use between the External Application and the WASM Module. The protocol takes care of:

• Setting values of variables or trigger Events
• Getting values of variables

The Gauge API function that we will use to set and get values is execute_calculator_code:

BOOL execute_calculator_code(
    PCSTRINGZ  code,
    FLOAT64*  fvalue,
    SINT32*  ivalue,
    PCSTRINGZ*  svalue
    );

This is a very versatile function to send data or get data using a “code”-string based on RPN notation (see Reverse Polish Notation). In the current implementation, I limit the return values to FLOAT64 only. Some examples:

This will set the value 4 in the variable “L:A32NX_EFIS_L_OPTION”:

FLOAT64 val = 0;
execute_calculator_code("4 (>L:A32NX_EFIS_L_OPTION)", &val, (SINT32 *)0, (PCSTRINGZ *)0);

This gets the value of the variable “L:A32NX_EFIS_R_OPTION” and stores it in the variable “val”:

FLOAT64 val = 0;
execute_calculator_code("(L:A32NX_EFIS_R_OPTION)", &val, (SINT32 *)0, (PCSTRINGZ *)0);

Set values of variables or trigger Events

This is achieved by sending a command “HW.Set.[command to be executed]” from the External Application to the WASM Module through the ClientDataArea “HABI_WASM.Command” (example: “HW.Set.4 (>L:A32NX_EFIS_L_OPTION)”. The WASM Module will get this command, and will use execute_calculator_code with [command to be executed]. This can be used to set values but also to trigger Events (example: “HW.Set.K:(>A32NX.FCU_HDG_INC)”)

Get values of variables

This is done with a few steps:

1. First a variable is registered in the WASM Module by adding it to a list vector<LVar> LVars;. That is achieved by sending a command “HW.Reg.[command to be registered]” from the External Application to the WASM Module through the ClientDataArea “HABI_WASM.Command” (example: “HW.Reg.(L:A32NX_EFIS_R_OPTION)”. During that registration process, the WASM Module will take the next available Offset in the ClientDataArea “HABI_WASM.LVars”, and add the variable with size FLOAT64 using AddToClientDataDefinition.
2. After that, the WASM Module will send back that Offset and Size data through the ClientDataArea “HABI_WASM.Acknowledge” using a data structure LVar. That structure also has a member to immediately send back the current value of that variable.
3. The External Application will react to this acknowledge to start listening for that element (offset/size) in the ClientDataArea “HABI_WASM.LVars” using RequestClientData.
4. The WASM Module will check all registered variables every FRAME. If a variable has a different value compared to the one stored in the list, the data element in the ClientDataArea “HABI_WASM.LVars” is updated using SetClientData.
5. The External Application will get notified of this new value through its call to the EventHandler OnRecvClientData.

Although execute_calculator_code is very straight forward and easy to use, it might not be the fastest way as already mentioned by @Umberto67 in this post. I haven’t tested my code with several 100 registered variables that need to be checked every frame. But the code could be optimized by using the gauge API functions check_named_variable, get_named_variable_value and set_named_variable_value.

There seems also to exists something like gauge_calculator_code_precompile. I have no clue what this does, and I don’t know if the returned compiled string can then be used with execute_calculator_code, and if this results also in a performance gain. There is (as usual) not a lot of documentation available. If somebody has a clue, please don’t hesitate to reply in this post.

Let’s finally look at some code

In both the External Application and the WASM Module, I have added a function InitializeClientDataAreas() that does the neccessary initialization of the 3 ClientDataArea’s that we will use.

1. ClientDataArea “HABI_WASM.Command” for commands sent from the External Application to the WASM Module
2. ClientDataArea “HABI_WASM.Acknowledge” to get the acknowledge from the WASM Module when registering “custom variables” we want to listen for
3. ClientDataArea “HABI_WASM.LVars” for receiving changed values from “custom variables”

Let’s have a look at the C# code of the External Application.

// file SimConnectHUB.cs

// Client Area Data names
private const string CLIENT_DATA_NAME_LVARS = "HABI_WASM.LVars";
private const string CLIENT_DATA_NAME_COMMAND = "HABI_WASM.Command";
private const string CLIENT_DATA_NAME_ACKNOWLEDGE = "HABI_WASM.Acknowledge";

// Client Area Data ID's
private enum CLIENT_DATA_ID
{
    LVARS = 0,
    CMD = 1,
    ACK = 2
}

...

private void InitializeClientDataAreas()
{
    Debug.WriteLine("InitializeClientDataAreas()");

    try
    {
        // register Client Data (for LVars)
        _oSimConnect.MapClientDataNameToID(CLIENT_DATA_NAME_LVARS, CLIENT_DATA_ID.LVARS);
        _oSimConnect.CreateClientData(CLIENT_DATA_ID.LVARS, SimConnect.SIMCONNECT_CLIENTDATA_MAX_SIZE, SIMCONNECT_CREATE_CLIENT_DATA_FLAG.DEFAULT);

        // register Client Data (for WASM Module Commands)
        _oSimConnect.MapClientDataNameToID(CLIENT_DATA_NAME_COMMAND, CLIENT_DATA_ID.CMD);
        _oSimConnect.CreateClientData(CLIENT_DATA_ID.CMD, MESSAGE_SIZE, SIMCONNECT_CREATE_CLIENT_DATA_FLAG.DEFAULT);
        _oSimConnect.AddToClientDataDefinition(CLIENTDATA_DEFINITION_ID.CMD, 0, MESSAGE_SIZE, 0, 0);

        // register Client Data (for LVar acknowledge)
        _oSimConnect.MapClientDataNameToID(CLIENT_DATA_NAME_ACKNOWLEDGE, CLIENT_DATA_ID.ACK);
        _oSimConnect.CreateClientData(CLIENT_DATA_ID.ACK, (uint)Marshal.SizeOf<LVarAck>(), SIMCONNECT_CREATE_CLIENT_DATA_FLAG.DEFAULT);
        _oSimConnect.AddToClientDataDefinition(CLIENTDATA_DEFINITION_ID.ACK, 0, (uint)Marshal.SizeOf<LVarAck>(), 0, 0);
        _oSimConnect.RegisterStruct<SIMCONNECT_RECV_CLIENT_DATA, LVarAck>(CLIENTDATA_DEFINITION_ID.ACK);
        _oSimConnect.RequestClientData(
            CLIENT_DATA_ID.ACK,
            CLIENTDATA_REQUEST_ID.ACK,
            CLIENTDATA_DEFINITION_ID.ACK,
            SIMCONNECT_CLIENT_DATA_PERIOD.ON_SET,
            SIMCONNECT_CLIENT_DATA_REQUEST_FLAG.DEFAULT,
            0, 0, 0);
    }
    catch (Exception ex)
    {
        LogResult?.Invoke(this, $"InitializeClientDataAreas Error: {ex.Message}");
    }
}

We can clearly see the 3 ClientDataArea initializations.

• For the LVars area, we only create the ClientDataArea. We will add the definitions and requests later when we receive the acknowledge from the WASM Module.
• For the Command area, we create the ClientDataArea and add the Definition of a 256 character string. I haven’t used RegisterStruct (honestly, I forgot), but the code seems to work perfectly. It might be that RegisterStruct is only needed when we want to receive data, and because we will only send data through the ClientDataArea “HABI_WASM.Command”, it might be that it can be omitted. If somebody knows for sure, please don’t hesitate to answer in this post.
• For the Acknowledge area, we do all the calls. We create the ClientDataArea, we add the definition of the LVarAck structure (= same structure as the LVar structure in the WASM Module) register it, and finally start listening for it.

In the WASM Module, let’s first have a look at the module_init, which is called when the WASM Module is started.

// file WASM_HABI.cpp

extern "C" MSFS_CALLBACK void module_init(void)
{
	HRESULT hr;

	g_hSimConnect = 0;

	fprintf(stderr, "%s: Initializing WASM version [%s]", WASM_Name, WASM_Version);

	hr = SimConnect_Open(&g_hSimConnect, WASM_Name, (HWND)NULL, 0, 0, 0);
	if (hr != S_OK)
	{
		fprintf(stderr, "%s: SimConnect_Open failed.\n", WASM_Name);
		return;
	}

	hr = SimConnect_SubscribeToSystemEvent(g_hSimConnect, EVENT_FRAME, "Frame");
	if (hr != S_OK)
	{
		fprintf(stderr, "%s: SimConnect_SubsribeToSystemEvent \"Frame\" failed.\n", WASM_Name);
		return;
	}

	hr = SimConnect_CallDispatch(g_hSimConnect, MyDispatchProc, NULL);
	if (hr != S_OK)
	{
		fprintf(stderr, "%s: SimConnect_CallDispatch failed.\n", WASM_Name);
		return;
	}

	RegisterClientDataArea();

	fprintf(stderr, "%s: Initialization completed", WASM_Name);
}

In this call, we do 4 things:

1. Open a SimConnect connection
2. Subscribe to the SimConnect SystemEvent “Frame”. You can register for several SystemEvents. The Frame event will allow us to check all the values of the registered variables on each frame.
3. Register our DispatchProc to SimConnect
4. Register our ClientDataAreas

The registration of the ClientDataAreas is almost the same as in our External Application.

// file WASM_HABI.cpp

const char* CLIENT_DATA_NAME_LVARS = "HABI_WASM.LVars";
const SIMCONNECT_CLIENT_DATA_ID CLIENT_DATA_ID_LVARS = 0;

const char* CLIENT_DATA_NAME_COMMAND = "HABI_WASM.Command";
const SIMCONNECT_CLIENT_DATA_ID CLIENT_DATA_ID_COMMAND = 1;

const char* CLIENT_DATA_NAME_ACKNOWLEDGE = "HABI_WASM.Acknowledge";
const SIMCONNECT_CLIENT_DATA_ID CLIENT_DATA_ID_ACKNOWLEDGE = 2;

...

void RegisterClientDataArea()
{
	HRESULT hr;

	// Create Client Data Area for the LVars
	// Max size of Client Area Data can be 8192 which is 2048 LVars of type float
	// If more LVars are required, then multiple Client Data Areas would need to be foreseen
	hr = SimConnect_MapClientDataNameToID(g_hSimConnect, CLIENT_DATA_NAME_LVARS, CLIENT_DATA_ID_LVARS);
	if (hr != S_OK) {
		fprintf(stderr, "%s: Error creating Client Data Area %s. %u", WASM_Name, CLIENT_DATA_NAME_LVARS, hr);
		return;
	}
	SimConnect_CreateClientData(g_hSimConnect, CLIENT_DATA_ID_LVARS, SIMCONNECT_CLIENTDATA_MAX_SIZE, SIMCONNECT_CREATE_CLIENT_DATA_FLAG_DEFAULT);

	// Create Client Data Area for a command ("WH.Reg.[LVar]" or "WH.Set.[LVar]")
	// Max size of string in SimConnect is 256 characters (including the '/0')
	// If longer commands are required, then they would need to be split in several strings
	hr = SimConnect_MapClientDataNameToID(g_hSimConnect, CLIENT_DATA_NAME_COMMAND, CLIENT_DATA_ID_COMMAND);
	if (hr != S_OK) {
		fprintf(stderr, "%s: Error creating Client Data Area %s. %u", WASM_Name, CLIENT_DATA_NAME_COMMAND, hr);
		return;
	}
	SimConnect_CreateClientData(g_hSimConnect, CLIENT_DATA_ID_COMMAND, MESSAGE_SIZE, SIMCONNECT_CREATE_CLIENT_DATA_FLAG_DEFAULT);

	// Create Client Data Area to acknowledge a registration back to the client
	hr = SimConnect_MapClientDataNameToID(g_hSimConnect, CLIENT_DATA_NAME_ACKNOWLEDGE, CLIENT_DATA_ID_ACKNOWLEDGE);
	if (hr != S_OK) {
		fprintf(stderr, "%s: Error creating Client Data Area %s. %u", WASM_Name, CLIENT_DATA_NAME_ACKNOWLEDGE, hr);
		return;
	}
	SimConnect_CreateClientData(g_hSimConnect, CLIENT_DATA_ID_ACKNOWLEDGE, sizeof(LVar), SIMCONNECT_CREATE_CLIENT_DATA_FLAG_DEFAULT);

	// This Data Definition will be used with the COMMAND Client Area Data
	hr = SimConnect_AddToClientDataDefinition(
		g_hSimConnect,
		DATA_DEFINITION_ID_STRING_COMMAND,
		0,				// Offset
		MESSAGE_SIZE				// Size
		);

	// This Data Definition will be used with the ACKNOWLEDGE Client Area Data
	hr = SimConnect_AddToClientDataDefinition(
		g_hSimConnect,
		DATA_DEFINITION_ID_ACKNOWLEDGE,
		0,				// Offset
		sizeof(LVar)	// Size
		);

	// We immediately start to listen to commands coming from the client
	SimConnect_RequestClientData(
		g_hSimConnect,
		CLIENT_DATA_ID_COMMAND,						// ClientDataID
		CMD,										// RequestID
		DATA_DEFINITION_ID_STRING_COMMAND,			// DefineID
		SIMCONNECT_CLIENT_DATA_PERIOD_ON_SET,		// Trigger when data is set
		SIMCONNECT_CLIENT_DATA_REQUEST_FLAG_DEFAULT	// Always receive data
		);
}

The difference with the External Application is that here we only listen to the ClientDataArea “HABI_WASM.Command” for commands coming in.

The MyDispatchProc is dealing with 2 events:

• When we receive data from the ClientDataArea “HABI_WASM.Command”.
• When we are triggered for a new FRAME

void CALLBACK MyDispatchProc(SIMCONNECT_RECV* pData, DWORD cbData, void* pContext)
{
	switch (pData->dwID)
	{
		case SIMCONNECT_RECV_ID_CLIENT_DATA:
		{
			SIMCONNECT_RECV_CLIENT_DATA* recv_data = (SIMCONNECT_RECV_CLIENT_DATA*)pData;

			switch (recv_data->dwRequestID)
			{
				case CMD: // "HW.Set." or "HW.Reg." received
				{
					char* sCmd = (char*)&recv_data->dwData;
					fprintf(stderr, "%s: MyDispatchProc > SIMCONNECT_RECV_ID_CLIENT_DATA - CMD \"%s\"\n", WASM_Name, sCmd);

					// "HW.Set."
					if (strncmp(sCmd, CMD_Set, strlen(CMD_Set)) == 0)
					{
						execute_calculator_code(&sCmd[strlen(CMD_Set)], nullptr, nullptr, nullptr);
						break;
					}

					// "HW.Reg."
					if (strncmp(sCmd, CMD_Reg, strlen(CMD_Reg)) == 0)
					{
						RegisterLVar(&sCmd[strlen(CMD_Reg)]);
						break;
					}

					break; // end case CMD
				}

				default:
					fprintf(stderr, "%s: MyDispatchProc > SIMCONNECT_RECV_ID_CLIENT_DATA - UNKNOWN request %u\n", WASM_Name, recv_data->dwRequestID);
					break; // end case default
			}

			break; // end case SIMCONNECT_RECV_ID_CLIENT_DATA
		}

		case SIMCONNECT_RECV_ID_EVENT_FRAME:
		{
			ReadLVars();
			break; // end case SIMCONNECT_RECV_ID_EVENT_FRAME
		}

		default:
			break; // end case default
	}
}

The flow is pretty straight forward.

• When we receive the command “HW.Set.”, we use execute_calculator_code to execute the command
• When we receive the command “HW.Reg.”, we register a new variable with a call to RegisterLVar
• When we receive a new FRAME trigger, we read all registered variables using a call to ReadLVars

I’m not going to explain the code in more detail, because the concept is already explained earlier in this chapter, and the code is pretty self-explanatory.

The last code I want to explain is the EventHandler OnRecvClientData in the External Application. This EventHandler is called in 2 cases:

• When the WASM Module sends back an acknowledge through the ClientDataArea “HABI_WASM.Acknowledge”, the variable (that has already been added to the List Vars in AddVariable) is updated with the returned Offset and Size, after which the methods AddToClientDataDefinition, RegisterStruct and RequestClientData are being called. From now on, the External Application will be listening to changed values for these.
• When a registered variable in the WASM Module has a new value through the ClientDataArea “HABI_WASM.LVars”, it’s value gets updated.

// file SimConnectHUB.cs

private void SimConnect_OnRecvClientData(SimConnect sender, SIMCONNECT_RECV_CLIENT_DATA data)
{
    Debug.WriteLine($"SimConnect_OnRecvClientData() - RequestID = {data.dwRequestID}");

    if (data.dwRequestID == (uint)CLIENTDATA_REQUEST_ID.ACK)
    {
        try
        {
            var ackData = (LVarAck)(data.dwData[0]);
            Debug.WriteLine($"----> Acknowledge: ID: {ackData.DefineID}, Offset: {ackData.Offset}, String: {ackData.str}, value: {ackData.value}");

            // if LVar DefineID already exists, ignore it, otherwise we will get "DUPLICATE_ID" exception
            if (Vars.Exists(x => x.cType == 'L' && x.uDefineID == ackData.DefineID))
                return;

            // find the LVar, and update it with ackData
            VarData vInList = Vars.Find(x => x.cType == 'L' && $"(L:{x.sName})" == ackData.str);
            if (vInList == null)
                return;
            vInList.SetID(ackData.DefineID, VarData.AUTO_ID); // use ackData.DefineID and next available RequestID
            vInList.uOffset = ackData.Offset;
            vInList.oValue = ackData.value;

            _oSimConnect?.AddToClientDataDefinition(
                (SIMCONNECT_DEFINITION_ID)ackData.DefineID,
                (uint)ackData.Offset,
                sizeof(float),
                0,
                0);
            Debug.WriteLine($"----> AddToClientDataDefinition ID:{ackData.DefineID} Offset:{ackData.Offset} Size:{sizeof(float)}");

            _oSimConnect?.RegisterStruct<SIMCONNECT_RECV_CLIENT_DATA, float>((SIMCONNECT_DEFINITION_ID)vInList.uDefineID);

            _oSimConnect?.RequestClientData(
                CLIENT_DATA_ID.LVARS,
                (SIMCONNECT_REQUEST_ID)vInList.uRequestID,
                (SIMCONNECT_DEFINITION_ID)vInList.uDefineID,
                SIMCONNECT_CLIENT_DATA_PERIOD.ON_SET, // data will be sent whenever SetClientData is used on this client area (even if this defineID doesn't change)
                SIMCONNECT_CLIENT_DATA_REQUEST_FLAG.CHANGED, // if this is used, this defineID only is sent when its value has changed
                0, 0, 0);

            LogResult?.Invoke(this, $"{vInList} value = {vInList.oValue}");
        }
        catch (Exception ex)
        {
            LogResult?.Invoke(this, $"SimConnect_OnRecvClientData Error: {ex.Message}");
        }
    }
    else if (data.dwRequestID >= (uint)CLIENTDATA_REQUEST_ID.START_LVAR)
    {
        VarData vInList = Vars.Find(x => x.cType == 'L' && x.uDefineID == data.dwDefineID);

        if (vInList != null)
        {
            vInList.oValue = data.dwData[0];
            LogResult?.Invoke(this, $"{vInList} value = {vInList.oValue}");
        }
        else
            LogResult?.Invoke(this, $"LVar data received: unknown LVar DefineID {data.dwDefineID}");
    }
}

That’s it folks! Now we can read and write from both “native MSFS variables” and “custom variables” including triggering native and custom Events. The code includes both the External Application and the WASM Module.

Remark: The method of registration/acknowledge is different as the way of working used in the MobiFlight WASM Module. There the Offset/Size is immediately defined when adding the variable. This means that both the External Application and the WASM Module increase the Offset every time a new variable is added, and use that same value in their registration process. That works fine for MobiFlight, because (if I am not mistaken) you can only have one instance of MobiFlight connected with MSFS2020. But in my concept, I want to run my SimConnectHUB application on several PC’s (you can use SimConnect over the network). This means that I might register the same variables through the WASM Module. If I use the MobiFlight method, the variable would be registered a second time in the WASM Module, which is reducing efficiency. The Register/Acknowledge protocol guarantees that if you register the same variable, that you also get the Offset/Size of the first registration.