11.31 and Bug Squashing

Inevitably after a large update to X-Plane like 11.30, new bugs go undetected during the beta process. We do a quick update to try to kill off these bugs as quickly as we can, e.g. X-Plane 11.11, 11.26, 11.31.

Yesterday we shipped 11.31. Unfortunately this isn’t the end of X-Plane 11.30 bugs, and in two cases, 11.31 appears to have introduced new problems.

We are working now on X-Plane 11.32, and our rough plan is:

  1. Fix all the really serious bugs (crashes, performance so bad you can’t fly) and ship that ASAP.
  2. Fix the rest of the lingering 11.30 bugs.
  3. Take a moment to question life choices.

If you are seeing crashes in X-Plane 11.30 or 11.31, the most useful thing you can do is to auto-report them, preferably with your email address in the report, so that we can contact you to run special builds.

The rest of this post is an update on the state of some of these bugs.

Driver Bugs

Older Intel OpenGL drivers contained a bug in their pre-processor that caused them to reject our HDR shadowing shaders. I rewrote the shaders to work around this bug for 11.31, and the rewrite has exposed a bug in OS X OpenGL drivers from 10.10.5. I have already fixed this and confirmed the fix with users who still run 10.10.5, so this bug fix will ship in 11.32 for sure. In the meantime, turn off HDR to work around the problem.

Also, if you are a Mac user running 10.10.5, consider updating to a newer Mac OS!

Weather Crash

The most mysterious crash we see is new to 11.31 – a mysterious crash in the weather code. This crash is mysterious to us because nothing changed anywhere near this code from 11.30 to 11.31. The crash reports also don’t make a ton of sense – Sidney and I spent a few hours last night staring at disassembly and being baffled.

I have been contacting users who auto-reported this crash, and fortunately the response to running some test builds for this has been quite positive. I’m hoping to narrow down the change that caused this so that we can wrap our heads around what went wrong.

This weather crash is the one I am most concerned about because it is both unrelated to anything we changed and introduced in 11.31 – a tiny release designed to stabilize, not destabilize the sim. I don’t have a work-around at the time because I don’t have anything like causal steps to reproduce.

Please do not contact me with “I have a crash, can I help” – if we haven’t seen your crash report, there’s no way for you to know if the crash you have is this one or something else. If your email address is in your auto-reports, we can ping you.

Running Out of Mapped GPU Memory

The largest source of instability we’ve seen recently comes from 11.30, and it’s the GPU not being able to provide X-Plane with mapped memory. Since we radically changed the rendering engine in 11.30 (as part of our port toward Vulkan) I am not surprised to see a major GPU problem, but it is still a top priority to fix it. This bug is equally common in 11.30 and 11.31, appears to affect AMD and NVidia windows users (but perhaps AMD more – we’re not sure), but isn’t something we see on our lab machines.  Sidney and I have some ideas on how to at least work around the problem so that people can fly.

Performance Problems

We’ve heard a lot of chatter about performance problems and complaints about performance loss, and we are collecting detailed performance reports from users so we can see what’s going on. Since the performance tests are automated, it’s relatively quick for us to gather this data.

So far, while we have seen a lot of mediocre performance (and mediocre hardware!), we have not yet measured via tests the kind of “catastrophic” performance problems that one might expect from the amount of complaining on forums, etc. That doesn’t mean there isn’t a serious problem out there, it just means we haven’t seen it.

For performance, my view is: if we find something truly awful (e.g. the sim used to run at 25 fps and now runs at 5 fps after the update), we’ll go fix it. I don’t want to make anyone’s copy of X-Plane impossible to use.

But if someone is seeing a 5-10% loss of performance, at this point it’s better to ignore it, because the same engineers who would analyze and fix the performance problems (Sidney and myself) are the ones who are doing the Vulkan port, and the Vulkan port gives us a better shot at fixing performance than trying to beat more performance out of the OpenGL driver stack. In fact, even analyzing the problem is more possible with Vulkan, because the OpenGL drivers do a number of expensive things that they give us no visibility into.

To give you an example of what I mean: we captured performance data from a user with an older Ryzen 16-core CPU and a GeForce 1080 GTX. Our analysis showed: 20-30 fps in the highest fps test (basically everything maxed out), 50-60 fps in medium settings, and 60-70 fps at the lowest settings. At all times, the GeForce 1080 was not maxed out and the CPU was the bottleneck.

Here’s the thing: the older Ryzen CPU has a single-core Geekbench score about as good as my 2014 iMac – that is, it’s not a top tier CPU, it’s old, and it was never optimized for single threaded performance. The user’s system is unbalanced for X-Plane (a lot more GPU than needed for that CPU), and 20-30 fps with everything maxed out is all I would expect from a CPU of that performance level.

So there’s no sign that the machine is underperforming our expectations. And the obvious thing to do to make the system faster is to be more multi-core (since the machine has 16 cores). And that means: focus on Vulkan.

So when it comes to performance, I’m going to beg patience and try to not lose momentum on the Vulkan/Metal port. In the long term it’s a better way to help everyone go faster. If we find something truly bad though, we will go and investigate more.

I don’t know what the ETA is for 11.32, but my plan is days, not weeks. The crash bugs are our top priority right now.

Posted in Development, News by | 44 Comments

X-Plane 11.31 Release Candidate 1 – Bug Squashing

X-Plane 11.31 is available to test – run our installer, update X-Plane, and check “get betas”. Full list of bug fixes here.

X-Plane 11.31 contains bug fixes that we could get done quickly, that almost made it into 11.30, and that were high priority, e.g. crashing on some Intel GPUs is fixed, and the external visuals don’t randomly lose sync.

We do still have some other fixes to get in at a later time. For example, there are a number of particle replay bugs where X-Plane isn’t saving the data needed to replay the particle effect; we will patch those in a separate patch where we can add more data to the .rep/.sit structures.

Some users have reported performance bugs, and we are gathering data and looking into them, but I’m not treating them as a five-alarm fire. We’re at a point where we are making rapid progress on the Vulkan port, and I don’t want that progress to grind to a halt as we investigate OpenGL performance problems; we already know that the long term solution to OpenGL performance problems is going to be Vulkan, not stabbing the OpenGL code repeatedly with a fork in the hope that it’s better behaved.

If we find something blatantly wrong with the OpenGL code in 11.30, we’ll fix it, but when it comes to ensuring performance, the very fact that the engine is OpenGL and not Vulkan limits us. At this point the IHVs are making their best performance analysis tools for Vulkan and Metal, not OpenGL, and Vulkan provides an API where the drivers performance is deterministic. (What we’ve seen so far is differing OpenGL performance for basically the same hardware drivers.)

X-Plane 11.30 was a really big code update to X-Plane – it had a major update on our route to re-writing the rendering engine, hence all of the rendering bugs we’re fixing. Over the next few updates I think we’ll have less code change as we stabilize, paired with art updates. We’ll take gateway airports and we have some landmarks ready to go.

Posted in News by | 45 Comments

X-Plane 11.30 Is Final

It’s out the door! (Steam should go “final” tomorrow morning, but you can get the final now – it’s still marked as a “beta”.)  Here’s everything that happened.

I expect we’ll do an 11.31 bug fix patch in the next week or two – 11.30 release candidate three was just critical fixes, e.g. fixing crashes on startup for Nvidia users with Windows 10 and the moon and stars aligned just right. Some less super-critical bugs are fixed and waiting for the bug fix patch.


Posted in News by | 77 Comments

Faking VR for fun and profit

As a developer, you’ll (hopefully) want your products to be VR compatible. To do so, you can of course use a real headset. If the need for debugging arises, that may not be very practical.

So, here’s how to fake VR in X-Plane:

  • For starters, you’ll want to bind a key combination to the “Toggle the 3-d mouse cursor in VR” command. This can be done in Settings -> Keyboard, and will allow you to switch between using the mouse in VR, and in the normal window to exit VR mode. It can by quite hard to get out of VR mode without this shortcut.
  • Start X-Plane with the --fake_vr command line argument (see “Using the Command Line Options” if unsure how to do that).
  • In the Settings menu, the VR options should now be enabled. When checking the “Enable VR Hardware” checkbox, the left-eye view will be rendered in the X-Plane window.

There are a couple of noteworthy things to keep in mind when using this. Let me start with the good news: this also works on Mac! On to the bad news… this is very much in the category of “nasty hacks”, which means:

  • Quite a number of rendering artifacts when faking VR, but it should be good enough for testing out things like user interfaces and the like.
  • No support for VR controllers
  • …or even for actually moving the fake headset about 🙁
Posted in VR by | 27 Comments

X-Plane 11.30 Release Candidate 1 and Toe Brakes

X-Plane 11.30 Release Candidate 1 is out – we fixed a lot of bugs!

As you can tell by the release name, we’re getting ready to kick this thing out the door.  Most of the team is no longer working on 11.30, and those of us who are are trying to make very small, tactical changes to not further screw things up. I wouldn’t bet against an RC2 (we’ve needed more than one try for all of the “really big” patches we’ve done in the last decade) but at this point if you haven’t tested your add-on against 11.30, you’re really late to the party.

Auto Toe Brakes

Flight simulators sometimes have to deal with unrealistic problems specific to being simulators and not real airplanes. One category of these “unrealistic problems” is dealing with user input hardware that is less complex than the real control inputs of the airplane. You might have a single throttle lever on your joystick, and not the 4 throttles with reversers of the 747. You might not have a twist axis or rudder pedals. You might be flying with a mouse!

Automatic toe brakes are a feature where X-Plane automatically applies differential toe brakes at times that we think might be useful if you don’t have toe brake hardware and no plugin is controlling the toe brakes. For example, an aircraft with a free-castering nose-wheel might be impossible to steer at low speeds without toe brakes. (You can’t turn the nose wheel directly, and the rudder will lack authority at low speed.) X-Plane can apply the left toe brake for you when you apply a lot of left rudder to help you turn.

In previous versions of X-Plane, this automatic toe-brake behavior was automatically applied based on some rules that Austin coded into X-Plane. When they worked, they really helped (and we know they sometimes worked because when we removed some of them in 11.30 we got bug reports), but at other times they weren’t tuned properly for a particular aircraft.

Starting in X-Plane 11.30 release candidate 1, aircraft authors can now explicitly control whether toe brakes are auto-applied for users without hardware, and if so, how aggressively. This control is auto-populated for older aircraft with the choices X-Plane 11.26 and earlier would have made, so you shouldn’t see a change in older aircraft behavior. The aircraft updating guide has the full details.

I believe that changes to these automatic rules may have been responsible for changes in third party aircraft ground handling in late 11.30 betas; hopefully with the new code, we should see complete compatibility.

Posted in Development, News by | 46 Comments

X-Plane 11.30 Beta 6 and 7, and Winding Down the Beta

X-Plane 11.30 beta 7 is out – full notes here. Hopefully the next beta will be a release candidate; we’re down to a small number of bugs, almost all of which are in the particle system, or are regressions (meaning the behavior was fine in 11.26 and is borked in 11.30).

If you have a bug that dates back to 11.26 or earlier and it hasn’t been fixed yet, it probably is not getting fixed in 11.30 – we’re out of time with this release.

Here are a few details on bugs we’re still working on and bugs we recently fixed.

Drawing Callbacks and Plugins

Beta 6 fixed a bug where plugins would receive extra window-phase drawing callbacks if a modern plugin had a new-style XPLMDisplay window on screen. This should fix a wide variety of plugins-interfering-with-each-others-drawing bugs, particularly where the “culprit” was a new plugin and the victim was SASL-based.

As a general guideline, please use XPLMDisplay windows with the modern SDK and new APIs whenever you can – it will give you the most compatible results going forward. The only drawing phase we recommend at this point is the panel/gauge drawing phases for custom panels.

N1/N2 for Turboprops

X-Plane’s free turboprop model uses the “N1” engine dataref to represent the gas turbine speed, and ties “N2” to the compressor tied to the prop.  When you turn on the starter but don’t bring in fuel, N1 will rise up to 16%, but N2 will stay much lower.

11.30 introduces the “new” free turboprop model – a second engine model that’s designed to more closely emulate the PT-6. For most of the beta, Austin was using an opposite convention: N2 represented the compressor turbine and N1 represented the prop turbine. Austin’s logic was that this was analogous to a high-bypass jet engine: N2 makes the high pressure and N1 spins the big fan-like thing.

In beta 7, Austin changed this; the new free turboprop model now follows the same convention as the old free turboprop model. This should make it a lot easier in the long term for authors whose aircraft have real PT-6’s to use the new model and take advantage of the improved accuracy.

Particles In the Cockpit

In beta 6, we fixed the bug where the new partilcle effects could not be seen from within the cockpit. This had the unfortunate effect of making them visible when the particles were inside the aircraft. This problem can be hard to avoid – depending on the wind and location of the exhaust on your aircraft, it’s possible the smoke just blows through the cockpit, making an artifact.

This is my plan for how we will ship particles in 11.30:

  • Particle emitters attached to objects with “outside” lighting objects will create particles that appear outside the aircraft; they will be masked out so that they don’t appear in the cabin.
  • Particle emitters attached to objects with “inside” lighting will be ignored and produce a log warning.  We are reserving this capability for a future update when we can have interior-pass particles; for now, don’t use this capability, as you can’t know how it will look in future versions of X-Plane.

This is one of the scariest “not done yet” parts of 11.30, because the logic to control interior vs exterior drawing is very complicated – it has to take into account different hardware capabilities, different rendering settings, etc.

Particle Light Levels

I am still working with Alex on particle light levels.  A recent beta included logic in the particle lighting code to match the clouds (e.g. direct sun makes them brighter) – without this, contrails don’t match the clouds.

But with this logic, particles look too dark on the ground. So we are tweaking things. The take-away here is: don’t ship your add-on with particles until we go final, as these things are subject to change during beta.

Automatic Toe Brakes and the C172

Automatic Toe Brakes is a feature where X-Plane automatically applies toe brakes when a user who does not have toe-brake hardware deflects the rudder pedals to near their maximum position. The idea is that on some aircraft, you can’t steer tightly or hold a heading in a cross-wind without the toe brakes. (This feature does not run if a plugin is controlling toe brakes or hardware is available.)

Early X-Plane 11.30 betas caused us to apply automatic toe brakes to aircraft where it was not appropriate, e.g. airliners. X-Plane 11.30 beta 6 removed this behavior from the airliners but also removed it from some third party GA aircraft where it was accidentally (but usefully) being applied.

We’re looking at this now, but at a minimum, I expect we’ll restore legacy aircraft to acting as they did before, and probably widen the steering range of our 172 a bit. I’ll write more in a separate post once we make a decision.


Posted in Development, News by | 89 Comments

Blender 2.49 files needed!

The Problem

As some of you may be aware, Blender 2.49 is old unsupported software that is becoming increasingly less usable on modern computers. To make matters worse, authors have large projects that are currently stuck on this platform, jeopardizing their work!

The Solution

We are creating a 2.49 XPlane2Blender converter inside of XPlane2Blender 2.7x, so that opening a 2.49 Blend file in 2.7x will automatically convert meshes, animations, materials, lights, textures, and XPlane2Blender properties into a modern format. This isn’t just an idea, we currently have a strong proof of concept that animations can be converted from old to new!

The 2.49 Converter Needs Your Help

The goal is

That if it exports in 2.49, it exports in 2.7x as similarly as possible.
That the conversion process is hassle free and also transparent.
That your work is safe and won’t be lost if there is a problem.

To accomplish this we’re going to need a lot of test .blend files and projects – to make it work for artists in the wild we need projects in the wild! We’ll take anything working and not working.

  • All source files will be kept secure and confidential within Laminar Research, and only used for this feature of XPlane2Blender, unless you make it open source.
  • Given that we’re trying to convert EVERYTHING, including materials and textures, it would also be extremely nice to have all files referenced as well.
  • Any description you’d like to include (preferably in a text block) would be nice too.

Again, just as with any other time someone sends me a sample to debug, it is kept secure and private, kept only for as long as needed, you can ask me to delete it at any time, and is only used for the development of that specific bug or feature of XPlane2Blender. They would never be used to develop any Laminar Research asset.

If you’re willing to submit a project to me, please get in contact via e-mail, DM, comment, or especially Github bug #397.

Please ask any questions!

Posted in Aircraft & Modeling, Modeling, Scenery, Tools by | 8 Comments

Beta 5, Default and Custom Particle Effects

First, X-Plane 11.30 Beta 5 is live. If we find it’s not a screamapiller we’ll get the Steam version going tomorrow.

Update: it is a screamapiller.  The lighting is missing for chunks of the runway, city, etc. When b6 fixes this, we’ll cut a Steam build.

There were some questions in the comments section about whether aircraft would lose the built-in default effects as part of the particle system upgrade in 11.03. The short answer is “no” and the slightly less short answer is “no, unless you do some stuff in Plane-Maker.” Here’s the details:

  • In X-Plane 11.25 an earlier, you got default effects, and there was nothing you could do about it. Every aircraft gets every effect.
  • In X-Plane 11.30, default effects are enabled by check-boxes in Plane-Maker’s “Invisible Parts” screen. Just like you can hide rendering of the aircraft’s physics model and replace it with OBJ models, you can replace the default physics-based effects and replace them with your own OBJ-based emitters.
  • In X-Plane 11.30, most of the built-in effects are built by the new particle system, but a small number (e.g. rotor down-wash) still use the old one, to take advantage of some hard-coded effects code that the new system can’t (yet) simulate.

Note that the question of how we did the default effect (new or old code) is totally unrelated to whether you (the author) want to replace it; if you turn off our effects, that check box will turn off our effects both now and when we re-implement that category of effects with new emitters (e.g. for rotor down-wash).

I still have a pile of effects bugs open, so please don’t ship aircraft based on the beta using the new particle system. (You shouldn’t ship an aircraft based on a beta anyway, but in the particle system, as of beta 5, I definitely have bugs that, when fixed, will require re-tuning of art files, e.g. lighting levels aren’t final).

Here’s a few particle bloopers.

Posted in Development by | 77 Comments

X-Plane 11.30 Beta 4 and Steam

X-Plane 11.30 Beta 4 is out, both for Laminar Customers and for Steam!  The fix list for beta 4 is pretty short because we intentionally cut back on what we added to it. Beta 4 had one job, and that job was to not be as broken as beta 1, beta 2, or beta 3.

X-Plane 11.30 Beta 4 does not have the performance improvements we have been working on for the last week and a half; Sidney and I found a bunch of things that are slowing 11.30 down, but we didn’t want to risk yet another broken beta. This stuff should be ready for beta 5 in a few days.

The public beta for 11.30 has been unusual in that it took us quite a few betas to get a stable non-crashing, usable beta, and during the waiting period we heard a lot of complaints from Steam users about being left out, and a lot of questions about when the beta would make it to Steam.

Here’s the problem: we don’t know the date when a beta will make it to Steam. Here’s the rough algorithm:

10 BETA = BETA + 1

So in the case of X-Plane 11.30, we had to go back and recut betas due to a crash in the joystick code (beta 1), a crash when the radios were on (beta 2) and airplanes falling through airports into the abyss (beta 3).

One thing to note: if there’s a beta and it’s a few days old and it’s not on Steam, it’s because the beta has a known bug that makes it so unusable that you wouldn’t want to set your Steam install to use it. I see no reason to have a beta that crashes when you start it.

(Since we collect automatic crash reports from users, we have frequency information about how often crashes are happening, and we can identify crashes that are significantly more frequent than what we’d expect in a stable beta.)

Posted in News by | 45 Comments