Please, only release well tested updates from now on

Official Microsoft Flight Simulator Wishlist
1

I think I speak for a lot of us, if I say that installing a MSFS update isn’t something I look forward to anymore. The sim could easily go from superb to abominable (be it performance and/or visuals) Like a lot of us experienced in the SU5 update.

Could you please - from now on - only release updates if they are well tested? I think most of us rather have a stable, well tested update than something that’s hardly tested at all and could wreak all kinds of havoc on our beloved sim…

Release it when it’s ready.
Not when management tells you it “should” be ready… :confused:

2

Good luck with that.

Thread will be closed in 5, 4, 3, 2, 1……

9 Likes
3

I think they were releasing updates after being well tested by the beta-testers… but, this time, I think the “x-box factor” talked louder… they wanna release MSFS for xbox already “up to dated” and maybe ignored the most basic bugs and issues…but I think it was not a good idea because they received complains from both platforms - PC and xbox.

3 Likes
4

How do you define “well-tested?” Do you mean tested on 50 of Asobo’s in-house PCs, 100 beta tester PCs, or maybe 10,000? Or now maybe 5,000 XBOXes? The problem with that is that no matter how much testing is done, no matter how many beta testers were involved, they still couldn’t come close to duplicating the potential effects of running it on over 2 million PCs, and who knows how many XBOXes.

And even if they did, there’s still going to be “that guy” who has a bad experience and says it needs to be tested more.

7 Likes
5

You have a good point of course, but the metric by which to assess success might not be a simple number. Just how many tests does it take to discover their brand new aircraft has faulty brakes? Left toe brake applies right brake… or how many tests are needed to see the world was like an over exposed photograph? They constantly miss the obvious, let alone the complex. Surely they can do better?

5 Likes
6

I would recommend one thing: an open opt-in “Insiders” program, like Windows and other Microsoft projects have.

Instead of a private beta with a small randomly picked subset of users restricted by a non-disclosure agreement, simply allow anyone who wants to to opt-in to the in-testing version, and update it regularly.

Don’t enforce an NDA – allow people running the Insiders Edition to openly discuss their problems and their triumphs, and point out new things they encounter that other people might not.

This allows problems to be discussed, evaluated, and fixed while the stakes are still low – while it’s not forced on everyone; while anyone can switch back to the regular version at any time.

8 Likes
7

Testing is not simply running the software in a bunch of machines and “see what happens”. I am an IT college teacher and in all the courses I had any contact with, there is at least a whole discipline (one semester) dedicated to software testing, more specifically Software Quality. There is quite a lot of theory, techinques, experiences and tacit knowledge involved in it.

8

I agree!!! The most updates are 3 steps bakwards, what we had before

1 Like
9

all updates are “well tested”

but when releases hit tge realworld different and unexpected issues will always crop up, especially in such a complex bit of software.

I’d rather move forward with the occasional hiccup than have one stagnant release.

note: if there was a way to add patches without issue, would all game devs not implement it?

they haven’t because it’s simply impossible at this time to cover every base or is so costly and resource consuming as to be impractical.

bugs/blockers etc are frustrating but they’re usually not forever.

patience is key.

10

Yes, it is a very complex software. Yes, it should run on PC and also in VR and as a simulator and yes, it should run on the Xbox with a controller as a game. Yes, the software can do all of this, but so far only 90 percent of all! That is probably undisputed.
But hey, we all know that just doing this last 10 percent requires an effort that can match the entire previous one. So what, wasn’t that to be expected? I play on the Xbox and simm on the PC, have an inactive pilot license since I’m already retired and I think the Flight Simulator is great.
Both Asobo and Microsoft did a very good job there. And I would like to encourage you to keep doing your best so that 100 percent satisfaction can be achieved. We hold out, but it shouldn’t take 10 years anymore, so the final spurt should go much faster. Microsoft and partners are a powerhouse and can be too. So show us what is possible, we can’t wait. But stop taking steps backwards. This is too strenuous for us and brings frustration and anger.

1 Like
11

Very much this. I don’t begrudge the OP their frustration, but this suggestion really does misunderstand the situation after what is likely to be one of MSFS’s most fundamental updates post-launch. :thinking:

1 Like
12

:rofl::joy::rofl::joy: good one!

13

No they really are not well tested.

And please stop this false narrative about how its impossible to test for the real world. I am a software developer and this IS the job. Yes we have a lot of sympathy for the complexity of the application but there are tried and tested ways to test releases. Automated testing, UAT, alpha releases, smaller change sets.

Testing is expensive, costly and both Microsoft and Asobo have made a very conscious decision to take the approach of light testing, high velocity and high level of regression and bugs. Even partners dont even know whats changing? Seb rules the development team, makes all the decisions and there simply isnt time to write automated tests, run regression testing, such is their desire to hit dates and add features.

And you know Asobo management have all made millions (its in the news)! So the approach is working. You think they care about a small but vocal minority of enthusiast PC Simmers moaning about ATC and texture downgrades. Nope. The community have helped them build a console cash cow and they are all going to be very rich.

Money talks and Asobo are raking it in, so the plan is working. Well done Seb and Jorg, now go buy that Ferrari!

For me, I did get annoyed, I just cant put the hours in like I used to and thats what really was frustrating, BETA testing for a year, but now, the sim mainly works (for me), looks amazing and runs smoothly. The UI is awful, ATC and AI is pants but the core engine is there.

Every time there is an update I just stop playing for 2-4 weeks while other poor sods find all the bugs and a hotfix comes out. I then spend a couple of days struggling with the things theyve broken and the cycle repeats. Fine. C’est la vie.

We just have to hope that someday when theyve made all the money from the shares and Xbox bonuses, they get bored of the Ferraris and holiday homes and come back and deliver what was promised to the enthusiasts who have helped beta test FS.

3 Likes
14

It has been my experience that the best testing results from users who have little to no DIRECT involvement with the development teams. This guarantees that there are no hidden or learned usage steps conducted by testers along prescribed lines. That is to say, they don’t avoid doing certain things just because that is an area of code that is incomplete or can cause problems. In other words, “conditioned avoidance”, The best test group is one that starts with no advanced knowledge of what the expected behavior should be. Any failures to success should be logged as bugs, without prejudice to “protected” or “sacred” portions of code that may exist.
That is not to say that users shouldn’t be encouraged to verify functionality that are the focus of the update, but if bugs to the overall system are discovered, they need to be examined. If the fix-it teams are too narrowly looking at how only new code incorporates into the system, the result for error regression is a risk.

3 Likes
15

I reported three bugs in the bushtrip … All were defined as real bugs by Asobo. Each time they fixed one, they created new ones. Now no more POI on the VFR maps, no route, etc … So well tested … depends on how you define well tested. They seem to ensure that the reported bug is solved but they don’t test any global functionality to ensure they didn’t create new ones. So now first leg works perfectly but second leg is more buggy than ever.

2 Likes
16

You are correct…unit testing is great, but the global functionality testing should be considered like a checklist…, so that nothing old gets broken. It’s a pain in the a—, but good companies DO test retrospectively.

6 Likes
17

If MS/Asobo did that then they would open themselves to criticism on another front. Something about lack of transparency and feedback I’d guess.

18

Not sure I follow, since what I recommend would be transparency and feedback channels.

2 Likes
19

Not going to happen. It would require a tracking system open to the public and someone to manage it. I totally agree with you it is needed.

20

Yep. That’s what I’m recommending.

This is like, industry standard stuff. If WINDOWS can do it, why can’t Flight Simulator???

2 Likes