PushbackZoomer: my new freeware app controls pushbacks by voice commands alone!

Hi all, by bringing together experiences in various areas of app programming, it takes sometimes only a couple of days between a good idea and a finished solution. The problem I wanted to solve is the cumbersome and unrealistic (even with the existing addons) pushback procedure.

And here is the solution:

  • My new little tool, called PushbackZoomer, must be the most simplistic GUI ever. Why? Because the GUI is just a blank window. It has literally zero UI elements on it! Start it, put it in the background an forget it!
  • How do you use it? Talk with it! Using the standard audio input, you need to speak a realistic and comprehensive set of speech commands, which trigger step by step an improved pushback procedure. For each command you hear a response via the standard audio output.

This video shows the tool in action:

Features:

  • Freeware (also no-adds) Windows 10 app
  • Distributed via the Microsoft Store, was well as a *.msix file (packaged Windows app) from freeware portals
  • Zero-controls GUI
  • The only feedback, that the GUI provides, is the connection state to the sim via SimConnect (GUI in grey = idle / GUI colored = connected)
  • Minimal resources footprint. By intelligent context sensing, the app only activates voice recognition during phases, when a pushback could occur. For the rest of the flight, the app will be running idle.
  • Speech commands in english only
  • Speech commands follow the real world protocol as much as possible.
  • Freely spoken commands are not understood, but the phrases need to match a set of predefined commands (see list of minimal commands below).
  • A healthy number of variations is supported both for spoken phrases as well as for the responses.
  • Depending on the mood of the virtualpushback driver, the conversation will have a different character. For every pushback any of the moods TightLipped, Formal or Expressive will randomly be applied to our virtual pushback driver.
  • Literally hundreds of millions of different conversations with the pushback driver are possible.
  • Robust speech recognition, even as non-native English speaker I easily get confidence values of 0.95 or more
  • Any action can be canceled by saying “no, no, no”, if something was not understood correctly by the speech recognizer.

For more details please consult the manual here:

For support requests please either write to the email-address in my MS store listing or comment below my flightsim.to-offering.

This is the minimal set of commands:

  • (Request pushback -> Used to trigger the whole procedure only in these cases, when the pushback tug is not moved automatically in front of the aircraft)

  • Ground from flightdeck -> Start the converstion

  • Brakes released -> Confirmation that the parking brakes have been released

  • Turn <right | left> -> Requests a 90° turn to the right or left

  • Turn <right | left> by degrees -> Requests a turn by the specified angle to the right or left

  • Turn facing <north | east | south | west> -> Requests a 90° turn, the direction is derived from the cordinal direction

  • Stop turning -> Stop turning and continue with current heading

  • No, no, no -> Cancel started action

  • Stop pushback -> Terminate the pushback

  • Brakes set -> Confirmation that the parking brakes have been set

In case the pushback tug does not move automatically in front of the aircraft:

  • Request pushback -> Requests the pushback tug to move to a position in front of the aircraft.

To test the audio i/o of the app:

  • Test pushback zoomer -> This command is always answered and can be used to validate the audio setup, as well as get feedback about the app state

The app is my christmas present to the community and is in beta stage! Anybody who is interested to try it out, can get a private links to the MS store from me by PN.

20 Likes

This sounds great - I look forward to giving it a try soon. Do you have a github repo for this somewhere?

No, sorry, I dont have the app in Github at the moment.

The submission to flightsim.to is being checked currently…

Absolutely interested in the proper link. If this bad boy works as described, it’s bada$$!! Berry exmoose to us!

Great app especially in VR I am interested in the link thanks

Yes please

deleted post

Can’t get it to work
Opening the app and is connected (green)
But when I say ground to flightdeck nothing happens also all other command nop result ???
Did all normal procedures:
Batteries on
Ext. power on
Allign
Load flightplan
Fuel
Apu on
Exy power off
IFR clearence
Ready for pushback > open the app (turning green ) but the pushback tug does not move like in your video on you tube

From the description it looks like the microphone is not selected as standard audio input. Opening the Sound Settings after right clicking on the little loudspeaker symbol in the system tray, you should choose the microphone which you are using like this:
image

Please report if that helps.

It is set like that I use it also in Elite Dangerous with voiceattack
The main problem is when I am at the point of pushback and I start the App(green) the pushback tug does not move even after waiting several minutes

Is it the version from the MS store? If yes, update it, the state handling should be improved by a new version, that I submited today.

Alternatively try the app version 1.0.8 from flightsim.to.

The improved state handling should allow starting the app now at any time, even before MSFS is started at all. When entering the cockpit, the pushback tug should move to the aircraft within 15 to 25 seconds. After that, the voice recognition engine is enabled.

got the version from flightsim.to but the same problem. The tug stays away from the plane and no command helps.

I am using my headset and mic which work with other games

Hmm, I’ve got feedback from others, that it is principally working but it seems my state sensing requires still some tuning…

Can you tell me the aircraft and the gate at which you are trying? I would like to reproduce. I had in some cases (IIRC when reseting the flight) the issue, that the tug did not move even while the pushback was active. So even by shift-p the pushback started but the tug remained where it was.

The recommended procedure for the user is this:

  • Start the app
  • Load an aircraft at a gate

Anyway I will provide a little update today, that allows to trigger this first step by talking “request pushback”…

Deleted the app from the store redownloaded it from flightsim.to tried it on EHAM and LEMG no succes
the pushback tug does not move therefore no instuctions possible over my headphone

The new version 1.0.10 is available from flightsim.to.

It has added these 2 voice commands:

** Request pushback -> Requests the pushback tug to move to a position in front of the aircraft.

** Test pushback zoomer -> This command is always answered and can be used to validate the audio setup, as well as get feedback about the app state

Additionally the zip-file also contains a PDF manual now, including the full supported XML SRGS grammar.

Maybe the driver wasn‘t in the proper mood to do any work today - might have been too cold :man_shrugging:t3:

Latest version all working fine now thanks

1 Like

Voice recognition works great. Any way to change the language pack that the TTS is using? It’s using the German language pack which sounds weird.

Oh, good catch! Thanks for finding this bug. I will provide an improved version shortly.

1 Like