This series of exchanges happens to me way too often:

User: X-Plane crashes when I change the flap setting.

Me: Which airplane does this happen on.

User: Any airplane.

Me: (Tries one of the default airplanes.  Crash does not happen.)  I could not reproduce this!

This has happened to me over and over again.  Invariably, when told that a bug is reproducible with “any” materials, it turns out that the bug requires something specific, like a particular airplane or scenery pack.

Bugs that occur with any materials are far less common than bugs that happen to particular airplanes or scenery because bugs that happen all of the time are more frequent and easier to spot and fix.

If we ask you which airplane or scenery pack a bug happened on, or ask you for reproduction materials, it means that either:

  • We already tried the bug on our own airplane or scenery pack and it didn’t happen or:
  • We think it’s very likely that the bug is specific to something in the airplane, based on how the code works or
  • We haven’t gotten any other reports of the bug, making it unlikely that it happens for all airplanes.

Now here’s the thing: if you have seen a bug, you saw it with a particular airplane or scenery pack.  When you report the bug, just tell us which materials you used!

(One exception: if you’re using a custom airplane, try one of the defaults first – that’ll be the first thing we ask you anyway.)

I’ll end this rant with a philosophical thought: you shouldn’t report a bug as happening with any airplane unless you have tried it with all airplanes.

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

4 comments on ““Any” Has No Place in Bug Reports”

  1. Yes, I’ve been guilty of that in the past. But this time I included a custom plane with my report (even though it does happen with all planes 🙂 )

  2. My favorite bug reports are something like “I did nothing and all of a sudden this and that doesn’t work anymore”.

    1. Quite often the user in fact *did* something. If things work correctly and stay unchanged it’s rare that they stop working all by themselves.

    2. Come on – am I supposed to guess what “it doesn’t work” means? Please tell me at least:
    – what exactly did you do (step by step) that resulted in an unexpected behavior
    – what did you expect to happen (maybe what happened was exactly what was supposed to happen even though you did expect something else)
    – what did actually happen

    Without this information everything is just like taking a stab in the dark.

    1. A psychologist locks a mathematician, a physicist and a computer user in three different rooms, each one with a crystal glass ball, a bucket of water and a measuring tape.
      After an hour, the psychologist unlocks the rooms to see what is happening:
      The mathematician has gotten bored, so he took the measuring tape and got the circumference of the crystal ball, and started calculating its surface area.
      The physicist also became bored in the meantime, so he took the measuring tape, measured the water bucket, then dipped the crystal ball into the water, and started calculating the crystal’s density.
      Finally, the psychologist looks into the room of the computer user. The crystal ball is gone and the window pane is shattered with a large hole in it. When asked what happened, the computer user answers “I didn’t change anything!!”.

Comments are closed.