Suggestion to community when we give bug feedback to Asobo - how we can help Asobo track bugs better so they fix it faster

Discussion Hub General Discussion
1

Hi. So I was watching the Q&A today and Sebstian mentioned how he had difficulty tracking one of the bugs the community had brought up: https://www.twitch.tv/videos/784582715?t=01h00m17s

For this particular bug that Sebastian mentioned, CptLucky8 has done a nice analysis of what the problem is and I’m sure if Asobo find that thread, they will find CptLucky8’s thread with CptLucky8’s analysis.

However, I have also read a lot of other threads, even threads in the Bugs & Issues forum, where people report a bug but aren’t giving Asobo enough information to reproduce the bug. For example, for some visual bugs, people don’t even post a picture to demonstrate what the bug is.

From my experience in the software development industry, it’s 100x more helpful to the software developer if you can give details on a bug, especially how to reproduce it. Because the software developer will follow your instructions to reproduce the bug. If the software developer can’t reproduce your bug, it’s not getting fixed. So here are my suggestions on how we can help Asobo to fix the bugs faster and more efficiently:

  1. For bugs that are visual or graphics related, get a picture of before the patch, and get a picture of after the patch, in the exact same location in the game, with the exact same game settings. If you change your location or game settings, that isn’t as helpful because it could be your changed location or changed game settings that prevents Asobo from determining what the bug is.

  2. Your location in the game (ie. longitude & latitude), and your game settings need to be the same before and after the patch. Changing your location in the game, or changing your game settings, just obfuscates the problem further and doesn’t help Asobo. When you report the bug, use the same game location and game settings so the comparison is consistent, which will help Asobo.

  3. Give as much detail as you can so Asobo can reproduce a bug. Remember that Asobo can’t see your game or computer settings in the pictures you post. Try to provide as much information that you can about your game settings, your resolution, your external GPU settings (ie. Nvidia graphics settings) and even the type of CPU, GPU, and RAM for your computer. For example, there could be a bug related to AMD video cards but Asobo won’t know about it if you just post a simple picture. The more details you provide, the easier it is for Asobo to reproduce the bug.

  4. State what patch the behavior was normal and what patch where the bug was introduced. Again, this is another example of giving more information to Asobo to reproduce the bug. So if you are posting a picture comparison, state what patch the first picture is from and what patch the second picture is from, so Asobo can reproduce the bug.

TLDR: If we don’t give enough information and detail to Asobo, they won’t be able to fix the bug. If we give them better information so they can reproduce the bug on their end, they will fix the bugs faster.

11 Likes
2

I am all for this and any other ideas we can present them to succeed on our behalf.

  1. Consider creating a template that needs to be filled out (similar to the ZenDesk format) and presented here on the forum? Have an MS/Asobo member sign off on the format?

  2. Take a screen print of the incident right after is has happened.

  3. If you can duplicate it make a quick video.

  4. More work request here (Sorry, Asobo): Save every Flight Instance/Flight Plan so they can see the “exact” settings that were provided to the sim. The users could then use the auto-saved plans to send with the bug report? Then everything we do to start a flight is saved.

I agree that more needs to be done to help expedite giving them accurate information so they can duplicate it and fix it.

I hated it when a user would just say “My machine locked up on me.” My reply: What were you doing at the time? They’re reply: Nothing! :slight_smile:

3 Likes
3

Definitely. A template would be much more organized and helpful for Asobo to track down the bug! Pretty much every professional QA department uses an organized template to submit bugs.

With respect to 4, perhaps the user could turn on developer mode and when the bug happens, submit the bug from the developer mode to Asobo. Then Asobo would have all the information they need, including the hardware on the user’s computer, the game settings, the resolution, and maybe even the external graphics cards settings. And then the user can get a bug submission ID of some sort that they can post in the public forums and the bug submission ID wouldn’t give up the user’s private information about their computer specs and game settings.

3 Likes
4

I just double checked in Dev mode (I am in a ten hour flight right now…) and there isn’t anything currently that would give us a data dump, including flight plan. Am I correct?

I would be all for that. I would even take it one step further. Put a system button on the bottom left or right of the sim window, just one button, that we could press to have that data right now. Anytime we want it.

Include:
System specs (temps too).
Flight Plan with current location.
Airplane and its current info.
Weather specs.
Screen print… Yup, do a screen print then and there (we could be waiting for the next lightening strike!).

I think we can all afford losing a buttons worth of real-estate of screen to help them solve these issues faster.

1 Like
5

How do you easily find that as you are flying?

In FSX it was as easy as “Shift Z” and you had your current Lat Long. How do you do that in MSFS ?

I cannot understand why they didn’t implement all of FSX’s features as a minimum starting point and add other features later.

This is what is most frustrating to we experienced simmers.

6

Did you happen to look at the dev map and bug map? They have barely even started anything. Most is listed and not even investigated.

Seems only priority is Xbox.

8

This is a very interesting and constructive topic. Please don’t drift off topic

Thanks

3 Likes
9

In principle I support this post however there are a few points that do raise my eyebrows:

So we need to know in advance that there’s going to be a visual or graphical bug to ensure we take a screenshot in advance of the patch? Are we meant to be taking screenshots willy nilly all the time?

As other posters have asked - how are we meant to accomplish this easily?

2 Likes
10

Another suggestion which could also assist in the bugfixing process: How about having a “Send Bug Report”-function in the SIM itself? When we encounter a bug which is not a CTD we could trigger an error report to be sent online to Asobo which captures relevant parameters from the SIM and includes them in a log-file which is uploaded to Asobo. In addition, the user could be prompted to give additional comments and details which for any reason can’t be captured in the log.

I think such a functionality could help Asobo get consistent data to assist in bugfixing.

3 Likes
11

Zendesk has already updated their capture form to include pheripherals and controller details as well as mods. I think if we did develop a community template, we should attach that to all zendesk bug reports too. I frequently capture video (easily done for nVidia cards), upload to youtube and include a link in the zendesk ticket. You can’t actually upload the video to zendesk, so I hope the youtube link works. In the zendesk capture form there are all the questions above (e.g. how frequently does it occur, steps to reproduce etc.) As in the OP, being as specific as possible really helps. Location etc would be a big help (and not just “near JFK” or whatever).

I think the zendesk form could be extended to include the additional info like system components etc. It is capturing pretty much everything else. That might be a more effective solution.

EDIT - there was a previous thread about getting some kind of bug reporting dump file containing all system details etc and auto sending that to zendesk/Asobo - which would be good too.

EDIT/EDIT - @Obbis got there first!

3 Likes
12

I agree with the OP. My only comment would be this:

The bug reporting system could be more intuitive.

I for one do not have a huge amount of time to fill out the submit a request (possibly should be renamed as well and made more prominent) form every time something goes wrong.

I imagine I am not alone in having been put off from doing so when confronted by a form requiring 20 minutes of input before I submit.

I fire the sim up after work for an hour or so and don’t bank on spending another 20 minutes using my brain to describe technicalities of what went wrong where.

In these days of “Please wait whilst we report the problem to Microsoft,” couldn’t some of the software related bugs eg CTDs be automatically relayed?

And perhaps make the bug submission process a bit less clunky? One very useful thing would be for the form to remember/detect your OS, gamer tag, CPU, GPU, RAM, Peripherals and other unchanging information. Surely this would be of benefit to Asobo as well…

1 Like
13

Agree with this, although I find that autofill (i use Chrome) remembers what I’ve previously put in some of ther fields (windows version etc) which does speed up some of it.

14

I stepped back last night after we hit this topic hard and fast, and then we all let it simmer. Which is good.

This morning it kind of occurred to me that this concept of creating a bug reporting tool cannot happen “inside” of the sim but needs to be an external program, safe from any CTDs or corrupted memory. Using an internal tool to report back to Asobo the status of the sim with a screen print will not always work. Not that my proposed idea here will also work, which it won’t always.

I have not done any research on how the API/SDK works in MSFS 2020. I do not know if it is stored in memory (which I think is 99% correct), or it’s piped out to a file (too slow); or both. Real developers, not hacks like me can run with this concept better since they already understand it.

The best people to create this external program makes sense that it would be Asobo. It’s their program, and data, and GUI. They know what data snap-shot they need to get to resolve bugs in-house.

If they don’t have the time to create this bug/data dump reporting program, which we all know that they don’t, then it is up to us to create something. Asobo would need to tell us what data they want back. And, the file/data-dump layout that can expedite them reading it. With a screen print attached. Creating a video inside this app to capture something like lightening would be great, but I think we need to at least start with a screen print.

Maybe this button would be a toggle button? Left side of the button is screen print, right side of the button is video capture?

At this point I am playing the role of “Arm-Chair-Admiral”, which I detest in myself and others. I retired from business earlier this year and I don’t think I ever want to touch a compiler ever again! Shame on me for even writing this post.

It is so very hard for programmers to fix bugs if they cannot visualize the data at the time of the bug. That’s why they use break-points. And, they need to rely on the users to “describe” where they were at in the program, what it looked like at the time of the bug, “and” what they may or may have not done using the GUI. Keyboard buffer added to the quote above…*

Everyone of you is a whole lot smarter than I am. “We” are a group of people that are smarter than your average gamer. We want to fly planes. We are the ultimate geeks!

If there is anything I can do to help then I am in. As long as it does not have a compiler attached to it that is. :slight_smile:

I agree with the other post above about privacy issues, but let’s try and do “high-level, blue sky” stuff first. K.I.S.S. We will get to privacy… (Thanks for the thought, GadWin777!)

I’m going to go hide in the sim now. I’m flying from Cabo San Lucas to San Diego today.

Thanks for ignoring my post. There’s nothing to see here, move along.

2 Likes
15

Catching and dumping CTD data is very difficult to program. It creates a lot of over-head in the program that is better used for game processing. Think of it like this: Hold on to data just in case I blow up, and then dump it then?

Going down that hole of holding onto data; “if” a CTD happens, can be a huge waste of resources. 60 FPS or CTD data capture. Pick one or the other but I bet we could never have both.

We all want triple 4k at 60 fps, or VR at 2160 x 2160 at 60 fps. Something has to give…

Maybe they could do this? I am out of touch already with what is capable on the platform they used to create this sim.

16

Of course a fair point. My biggest point is really around the form itself. It takes someone like me who has very little IT knowledge quite some time to complete the bug submission to a level I would deem “helpful” for Asobo. A more intuitive process such as you describe above would mean I would just do it automiatically every time I spot an issue. Yes this might result in a higher amount of bugs being reported to Asobo, but the quality of those bug reports would gain usefulness in consistency.

17

Some kind of automated process would only really work for CTDs - for other bugs it can capture the “sim state” when you press the “report” button, but you’ll still need to add something to explain what the bug is. But that is obviously less work than filling in all your controller details every time on zendesk, which is a bit of a drag.

1 Like
18

I agree with you 110%, Wieldy.

A tool like this could over-whelm Asobo with data. I do like the idea that the user enters a comment to send with the data, like Gordon has proposed. I would also like to add to that comment that a drop-down subject be selected by the user that sorts the reports to Asobo by the title/subject of the bug report. Then internally they could resolve what team to put it under.

This is some heavy-duty stuff we are talking about here that has huge repercussion to it, aimed at Asobo. But, it could help the process move faster for them.

I usually blamed the bugs I experienced on three things:

  1. It was the keyboard actuator that did it.

  2. FNG’s always find a way to break a system. They have an un-canny ability to click on something that a programmer would have never thought possible or worth doing.

  3. Programmers are real people too and we all make mistakes. One line of code can impact a line of code two million lines later that is under a different programming department’s responsibility.

Our proposed idea could augment the current Zendesk format we have all used now. And maybe even help to streamline the process if it was web based/HTTP? My initial thoughts were just to dump the data/screen-print to the users SSD. They could then attach the file(s) to the Zendesk post.

Thanks for pointing that out, Wieldy!

19

Maybe the user can set it to developer mode and then click on the button for the data dump. It should be relatively easy for Asobo to do.

20

I’ll try contributing, first in copying/pasting the suggestion I’ve made in the Q&A discussion so that it gets centralized in a dedicated discussion here:

About Zendesk bug reports

Please add feedback for the sender about what is going on, whether discarded for a reason (lack of data to repro, inconclusive repro) or, work in progress. And for the later, it would be best to also get milestones (investigation started, trying to repro, analyzing data, postponed for further investigation, bug identified and triaged for postponed resolution, bug identified and triage for resolution now, etc…).

About repro/traceability

I’d also suggest they enable minidumps and automated crash reports with Windows Error Reporting (or other libs like Breakpad or CrashRprt). This is invaluable information.

About submission handling

It might also help they look whether the submission comes from a 3rd party developer, or at least that they add the capability to tell one’s own level of technical confidence in the report, or one’s specific technical field relevant to the report if any. Please understand I’m not trying to lessen any one’s own merits with this statement:

  • I believe when you’re a professional developer submitting a bug report, you do have a certain technological understanding of what’s going on, what could be wrong, and you can also discuss technical matters more easily in case they are contacting you back.
  • In addition, this shall save most of the 1st line questions such as “have you updated your drivers?” or any other equivalent basic question (even if sometimes developers are worse than others with these matters).
  • Similarly, someone who is a professional photographer or digital art designer might have a unique expertise regarding graphics related bugs which makes it valuable to know for the people handling the submission in the back end.
1 Like
21

I’m bumping this thread again. There is a really silly water thread now, where people just complain but aren’t posting screenshots with apples to apples comparison.

To help Asobo, post screenshots using the exact same game settings to help Asobo track the issue.
Use the same graphics settings, same location, same altitude, same weather - make everything the same in your screenshots of before and after the patch.

For the particular water thread, people are complaining that the water at release is better than the current patch but they can’t provide apples to apples screenshots to prove it.

1 Like