Category: News

We Are All Raster-farians Now

In my previous post I drew an analogy between a scenery system with its file formats and a turtle within its shell. We are limited by DSF, so we are making a new file format for base meshes so we and all add-on developers can expand the scope of our data and make better scenery for X-Plane.

The really big change we are making to base meshes is to go from a vector-centric to a raster-centric format. Let’s break that down and define what that means.

Vectors are fancy computer-graphics talk for lines defined by their mathematical end-points. (Pro tip: if you want to be a graphics expert, you just need the right big words. Try putting the word anisotropic in front of everything, people will think you just came from SIGGRAPH!) DSF started as an entirely-vector format:

  • All 3-d clutter is defined by lat/lon locations, so we have the vector outline of polygons, autogen blocks, etc.
  • The base mesh is pre-triangulated, so most base-mesh features are defined by the corners of the triangles forming lines (e.g between land and water).

This isn’t the only thing DSF does – we added raster capabilities and there is e.g. raster sound and season data in X-Plane 12, but DSF is fundamentally about vector data – saying where the edges of things go exactly.

This was great for a while, but now that we have more and more vector data (complex coastlines, complex road grids, complex building footprints) the DSFs are getting too big and slow for X-Plane.

Raster data is any data stored in a 2-d grid. This includes images (which in turn includes orthophotos) but it also includes 2-d height maps (DEMs), and the 2-d raster data we include in DSF now (e.g. sound raster data, season raster data etc). Any time we store numbers that mean something in a 2-d array, we have raster data.

Raster data has several advantages over vectors:

  • Raster data is what the GPU wants to consume.
  • Raster data has really good LOD characteristics for close detail with long view distances.
  • We can put more interesting and dense information into a raster tile without it getting bigger.

Twenty years ago, when I first worked on DSF, computers didn’t have the capacity to use lots of raster data – this was back when 8 MB of VRAM was “a lot”. But now we no longer need to depend on vectors for space savings.

Raster tiles are raster data broken into smaller tiles that get pieced together. Raster tiles have become the standard way to view GIS data – if you’ve used Apple Maps or Google Earth or OpenStreetMap or any of the map layers in WED, you’ve used raster tiles.

Raster tiles have a bunch of advantages too:

  • They have really great LOD/VRAM usage properties.
  • They can be loaded incrementally.
  • They provide an easy way to vary resolution and let authors skip providing data that they don’t need to provide. (E.g. “forest” raster data over the ocean? Just don’t provide any tiles!)

So our plan for the next-generation base mesh is “all raster tiles, all the time” – we’d like to have elevation data, land/water data, vegetation location data, as well as material colors all in raster tile form. This would get us much better LOD/streaming characteristics but also provide a very simple way for custom scenery packs to override specific parts of the mesh at variable resolution with full control.

Raster tiles are not the same thing as orthophotos. A raster tile is any data contained in a 2-d array, not just image data cut into squares (e.g. orthophotos). So while a raster-tile system may make it easier to build orthophoto scenery, it does not mean that the scenery can only be orthophotos.

Posted in Development, File Formats, News, Scenery by | 27 Comments

A Few Notes on 12.1.0 for Developers

Thomson and Dellanie posted a preview of what’s coming in X-Plane 12.1 – click over to the news blog to see the pretty pictures. Short story long, it’s a graphics-intensive release, but it’s also a big release, with weather, systems, avionics and ATC updates too. What follows here is a few details for developers.

Private Alpha Builds Soon

As with all past X-plane 12 patches, we are planning to do a private “alpha build” test run with the developer lobby before public beta. We do this both to find out about add-on compatibility and to get the worst bugs fixed with a smaller test group. As of this writing, ATC is at the end of bug fixes, the last graphics changes are being tested, but my weather work is still mid-development.

New Toys for Scenery Developers (and Aircraft Modelers)

This release turned out to have a lot for scenery developers:

  • Water bathymetry and turbidity should be sorted out, making orthophoto base meshes with 3-d water possible.
  • The material system now features normal-map decals. These decals add high-frequency ‘texture’ to a surface by perturbing the normals (direction of light bounce) instead of just “painting’ them. This means better looking materials at different lighting angles.
  • LOAD_CENTER is now usable on OBJs. This means you can set the resolution of an OBJ to be based on distance to the aircraft (just like draped polygons and orthophotos); I think this has the potential to ensure that “land mark” OBJs (models used exactly once) are at their highest resolution when the user is near them without hurting VRAM in general.
  • If it passes test (crosses fingers), particle systems can be used directly from DSF objects.

The entire DECAL system (existing and new normal map decals) are also usable in OBJs for aircraft as well, and the particle system has received some upgrades that can be used everywhere.

One warning: the old “smoke puff” directive for OBJs from X-Plane 8 is now inoperative; with 12.1.0 we finally removed the old particle system completely. I suggest using the new particle system as it will give you real control over what kind of smoke comes out of your models.

New Plugin APIs

Two new plugin APIs are planned for 12.1.0:

  • New navigation APIs
  • An extension to XPLMAvionics so add-ons can create their own glass screens.

We are still putting finishing touches on the avionics APIs now, but this tech is very close to complete, and definitely usable.

Unfortunately we will not have a low level weather API in 12.1.0 – R&D on this is ongoing, but at least in better shape than it was before due to fixes to the internal weather code.

Posted in Development, News by | 18 Comments

Blog Post Walk of Shame

After the X-Plane Developer Conference in Montreal this weekend (thanks to everyone who came and especially ToLiss for hosting/making us feel totally at home in Montreal) I thought “I should probably post something to the dev blog talking about what we’re up to.” I logged in and saw…I haven’t posted anything in four months.

So that’s not great. The truth is the X-Plane team is larger than it was in the v10 days and I spend most of my communications time talking to the internal team and third party developers.

I’m going to try to post once a week here. This sentence may be an embarrassing monument to lofty but impractical goals in another four months, but putting it in writing is a way to commit to it, and it won’t be the most embarrassing thing I’ve ever done.

Two Quick Store Notes

We announced the X-Plane Store plan in Montreal; Dellanie’s got a great FAQ there, but here for developers I just want to make two points clear:

  1. We are not locking X-Plane down. I totally get why people might think this because the iPhone app store is (1) a very prominent way to do in-app purchase and (2) the iPhone is locked down. But we are not doing that. All of the ways you get stuff into X-Plane will still work, including add-on installers, dragging folders around, you name it. Going through the store is not a requirement, and everything that works now will keep working. You won’t have to pay for freeware or re-buy anything.
  2. You will not have to be online all the time. Our current policy is that if you have an online license (an “XDD key”) you have to log in every two weeks or more to renew it. We are not going to switch to “all online all the time” – we know that this is impossible for a lot of our users and we consider maintaining this “renew the key every now and then” policy to be a requirement.

What’s Next: 12.1.0 and 12.2.0

We have two “big updates” planned right now:

  • 12.1.0 will ship next and is primarily a graphics release. RCAS, bloom, DoF, shadow softening, cloud shadow fixes, new decals, you get the idea. This update will also address a number of real weather bugs and transition to the new real weather servers.
  • 12.2.0 will ship after 12.1.0 and is primarily a flight model release. All of Austin’s advancements in prop blade dynamics, stalling and turbulence, etc.

Both releases will have a bunch of other stuff too; they’re big patches. I’m pointing to the graphics/FM divide because the decision to push one of graphics or physics in each patch is intentional to keep the scope of the beta under control.

We are aiming to get 12.1.0 into private testing this month.

What’s Up, Docs?

Probably only three people on Earth care about this, but after a decade (more walk of shame) I was trolled into updating the .net file format specification. So if this is interest to you, I apologize both for the delay and for the pain you are about to suffer. The road art file format in X-Plane is incredibly complicated and I don’t actually recommend anyone try to hack it, but it’s also not meant to be a secret.

Scenery, Now and Future

I am working to get water and orthophotos sorted out, hopefully for 12.1.0. We will also have a DSF recut (hopefully for 12.1.0 but maybe for 12.2.0) to address gateway airport boundaries and runway undulations.

In Montreal I discussed a little bit about the future of the X-Plane scenery system, but that’s complex enough to warrant another blog post.

Posted in Development, News by | 33 Comments

X-Plane 12.07 ARRsea 1 Has ARRived

Editor’s note: what follows is very, very, very, very, very silly.

Last week we shipped X-Plane 12.06; since then we have found a few straggler bugs; like typical lARRge patches from the days of X-Plane 11, a few bugs escaped us until after final, including some crashes we could see in the auto reportARR.

While it’s not great to be shipping a “hot patch” to our release, it is pretty fantastic to announce X-Plane 12.07 ARRsea 1 on International Talk Like a Pirate Day (the 19th of septembARR). The rest of this blog post only gets worse; you’ve been wARRned.

12.08 Is The New 12.07

Internally, nobody is happy about this, but the hot patch bumped all of our version numbers. So the new road map looks something like this:

  • 12.07: hot patch of 12.06.
  • 12.08: coming soon. Flight model and systems, ATC and Networking
  • 12.09: more graphics and real weathARR fixes.

12.08 (né 12.07) is almost completely integrated and should be ready for private testing as soon as 12.07 is settled.

Why Wasn’t I Notified

Since 12.07 is a new version, you won’t be auto-updated to upgrade from 12.06 while 12.07 is in testing; run the installer, update with “get betas” checked and you can get 12.07.

Crash Fixes

12.06 shipped with more sensitive internal detection for numeric problems. This is how we run the sim internally all the time – the goal is to find and squash bugs fast.

Unfortuntately there are sections of X-Plane’s simulation that are:

  • Not present in an aircraft (e.g. the real aircraft doesn’t have X)
  • Not expensive to run, CPU-wise
  • Not visible to the user (since the aircraft has e.g. no gauges to show these systems.

As it turns out, these systems are often happily running away in the background producing absolutely bonkers values; with sensitive numeric checking, the sim can crash due to problems in a system that nobody cares about.

For 12.07, I solved the problem by muting the alarm. My expectation is that we’ll have sensitive numerics in most betas, turn them off in the releases, and work through the systems code over time. (The priority on this isn’t super high because the systems aren’t consuming framerate – if they were we’d see them in our profiling.)

The other area where we saw increased crashes was with TCAS plugins. 12.07r1 has better logging and shouldn’t crash as much – the goal here is to not brick third party add-ons.

Other Bug Fixes

Multi-monitorARR: data output was not working in full screen configurations, fixed now, and manipulators work for multi-monitor systems.

OpenXARR: HP Reverb fixed, and we finally figured out why the white board would sometimes disappear. (It wasn’t gone, it was just 20 km from the hangARR.)

GARRmin Bezels

X-Plane does not have a built-in way to remove the bezels from the pop-out avionics. To solve this, some of our user and some add-ons edit the ARRtwork inside X-Plane’s “resources” folder, setting the bezels to clear.

X-Plane 12.06 introduced new bezel variants for the G1000 to cover all of the real-world panel button configurations. This fixed the bug “the real aircraft does not have these buttons, but when you pop the panel out, the buttons appear and do nothing.”

A side effect of this is: those aircraft using the new bezels use new art files, so the hacked up no-bezel art files on longer work.

This is not a bug for us to fix; if your add-on works by modifying the internals of X-Plane, we cannot guarantee that it will keep working even when we change the sim. (The only way to make that work would be to never change the sim.) You can work around this by modifying the new bezel files.

In the future, our plan for this is to provide a real way to hide bezels in the sim as a built-in part of the UI, which should make hacking unnecessary.

What Comes Next

Hopefully 12.07r1 will be “one and done”; if so, we’ll move on to private testing of 12.08 almost immediately.

If you are a third pARRty and your add-on has problems with 12.06, please tell us four weeks agoas soon as possible!

Posted in Development, News by | 25 Comments

X-Plane 12.06 – Third Time is a Charm

A few quick notes on 12.06-related chaos – it’s been a weekend.

Global Airport Errors

“Don’t rock the boat in a release candidate.” This is the lesson I should know, since I have cut at least 187 of them.

In 12.06rc1, we were missing HEKA (completely) and all of the airports at KLAS; the KLAS problem was due to a missing art asset. One of the multiple confounding factors in this bug is that the global airports don’t put up a “there was a problem with the scenery pack” pop-up message when art is missing. So, being the brainiac I am, I went “oh, that seems dumb, we’ll never know if the pack has problems, I’ll go fix that now”.

For 12.06rc2, I changed the policy so that the global airports would get a pop-up message like any other scenery pack that’s missing art.

As it turns out, the global airports are missing lots of objects – and probably have been for the entire X-Plane 12.0 run.

So coming soon to an installer you: X-Plane 12.06r3, which will be just like r2, but with the pop-up message turned off (just like the rest of the beta has been).

We have a more comprehensive plan to address these art asset issues for 12.07 – I’ll comment on why it’s important that missing scenery assets be errors in another post.

As a side note, it appears the most RCs we’ve had is X-Plane 9.40, which got up to release candidate 16 and took at least six weeks.

NVidia Users: No Zink For You (for a little bit)

In X-Plane 12.06r2 we have disabled Zink for NVidia users. We did this because there is an NVidia driver bug that interacts with Zink that will cause the sim to crash.

NVidia found the bug, is working on a fix, and they are quite responsive in pushing driver fixes. As soon as the fix is in non-beta drivers, we will change this from a “zink is banned” to a “you must have at least driver version x.y to run zink” message.

If you really really really want to run Zink on your NVidia card, you can still can, using these two steps:

  1. Raise your right hand and repeat after me: I, _____, solemnly swear that I want to crash my copy of X-Plane by running Zink with NVidia drivers before the driver fix is available. Let it be known to all of the X-Plane community that I do this of my own volition. I enjoy having my flight end abruptly with a crash report dialog box on short final, and I will not complain about this on r/flightsim.
  2. Run the sim with --zink

AMD users: you are not affected by this, so Zink is still available. Enjoy this rare chance to make fun of your green team compatriots.

We Put the ‘No’ In NOAA

Over the weekend the real weather servers went down. While we usually blame NOAA for this kind of thing, this one was super embarrassing: the server ran out of disk space (and the disk space monitor didn’t notify our ops team). The ultimate cause, I suspect, was that it was labor day weekend in the US, e.g. “a really good weekend for infrastructure issues.”

This isn’t the first real weather outage we’ve had this year, and for this reason we have a replacement weather server in development. The replacement will be able to serve the least old weather correctly when NOAA has a maintenance outage (instead of just 404ing everyone) and will hopefully clean its cache out correctly.

(The caching problem is slightly harder than it sounds: we do want to retain some old real weather files for test cases and bug reproduction. But we need to purge most of it to keep the server from filling up.)

What’s Next

If 12.06 rc3 is final, we’ll let it sit a little bit and then start private testing of X-Plane 12.07. The X-Plane 1.206 beta has taken quite a bit longer than we wanted, so 12.07 is intentionally a smaller release – about half the size in terms of code change. That’s still a lot of new stuff (and a topic for yet another separate blog post).

If 12.06rc3 is not final, I will set myself on fire.

Posted in News by | 50 Comments

ATC changes in 12.06 and beyond

Jim Keir here – today I’m going to talk about what’s been going on with ATC for the last six months or so, and what’s planned for the near future.
You’d be forgiven for thinking that the answer to the first is “not much”, simply because there’s been very little visible in the release notes for ATC, but that’s definitely not the case. Normal work on bugs and small improvements has been going on as usual, and 12.06 contained quite a backlog of these which I’ll go into more detail about below. Normal progress has been slowed, however, by time spent on adding one huge new feature. More on that later.

First though, a quick peek behind the curtain. This is a very high-level list of the changes that went into 12.06, now with added explanations! I’ve removed about half of it because they were maintenance, corrections to previous changes and other non-visible things, and edited some of the rest to protect the idiot innocent. Many of these changes, especially the ones starting with “Merge branch”, would be made up of many changes themselves and could cover days to weeks of work; others might be a change to a single letter.

New in 12.06

  • Fix incorrect voice phrase for requested altitude change
  • AI had a stupid max speed under “AI flies” at key points on the ground which was never reset
  • Fix unwanted transmission possible during spawn of AI on arrival.
    This simply clogged up the airwaves for the first minute or two of your session.
  • Fix for descent time calculations
  • Merge branch ‘fix_XPD-13780_Pushback_Messages’
  • Merge branch ‘fix_XPD-13783_Pushback_Dialog’
    These two should make the text messages and buttons in the pushback dialog clearer and more consistent with the current state. Note that you can also request pushback by radio, from ground or tower.
  • Modify altitude monitoring to try and remove inappropriate warnings
    This is a long-standing bug with repeated altitude correction calls from the controller, when you were already at the right altitude. Since this change I finally got a log from a bug report that let me nail the main cause, and a final (?) fix should hopefully be included in 12.07.
  • Allow editing of ATC flight info if required information is missing, even if a flight is under way.
  • Take two for the “AI not calling downwind” bug.
    AI may not always have called downwind. They still called final and landed correctly, this was just a missing radio message.
  • Merge branch ‘fix_tropo_jumps’
    People flying airliners found that their altitude was suddenly changing during cruise, sometimes by hundreds of feet, making their autopilot want to take a bit of a lie down in a dark room. This turned out to be two subtle bugs in creating the atmosphere’s vertical temperature profile, where it was possible to get a sudden change instead of a smooth gradient. It depended on your precise location and altitude, and having real weather which had specific temperatures at very specific altitudes.
  • Latest airport names and combined voice changes from several branches.
  • Merge branch ‘fix_XPD-13873_Wake_Sep_On_Runways’
    ATC will now delay your departure clearance until the wake from the previous aircraft, whether arriving or departing, has reduced to a safe level for your aircraft.
  • Merge branch ‘fix_XPD-13842_Departing_AI’
    The calculation for the time you would take to line up was just plain broken; sometimes it was too long, sometimes too short. This affects AI as well, of course, so this one fix should prevent you being told to line up with an A330 on short final, and also reduce AI go-arounds. It also now takes account of how far the hold-short line is from the start of the usable runway, and whether there are any sharp corners.
    Why wasn’t this seen before? Because the estimate wasn’t always way too long or way too short. Sometimes it could be just right, and the airports that habitually get used for both manual and automated testing by total coincidence were just right. This one also only was able to be found and fixed after someone sent a logfile with full details.
  • Merge branch ‘feat_XPD-12784_multi_channel_atc’
    This one’s a biggie. I’ll talk about it separately.
  • Merge branch ‘feat_ATC_AI_TCAS_Awareness’
    ATC is now aware of TCAS aircraft – read “plugin-controlled AI” – when checking your line-up time against other aircraft using the runway to work out whether to give you departure clearance or not. This means that third-party AI aircraft systems should play a little more nicely with ATC, with no changes required to the plugins.
  • Contact nag was broken entirely.
    Fixes the “Are you with me” if you are handed off to a new controller, change frequency but forget to check in with them.
  • More AI-flies changes for specific situations. Alter the test to account for fake-flow changes.
    A lot of testing is done using AI aircraft, and the “AI flies” mode for the main aircraft. Getting that transition from human to AI done and having the AI pick up an aircraft mid-flight in any state whatsoever, is not simple! This also shows one of the automated tests being changed because it correctly failed as a result of another change.
  • Fake calm flow was only using one runway even if multiple valid runways existed.
  • Allow “Missed Approach” earlier during the approach.
  • Speech for airport names not being generated if the apt.dat has no country code set for that airport
    Airport designers, please use correct ISO country codes! “USA”, “U.S.”, “America”, “Amercia”, “U S A”, “US of A” and so on.
  • Fix several problems with missed approaches
  • Add option to file and send plan to FMS, and to transfer plan from FMS
    Simple quality-of-life change for the “File a flightplan” tab in the ATC dialog. If you’ve set your FMS, maybe from a file downloaded from an online planner, you can now transfer your route to ATC in a single click. Likewise, if you’ve asked the ATC dialog to plan a route, you can send that to your FMS. Of course, you’re free to program your route into the FMS manually as before.
  • AI did not avoid user if parked on the end of the runway with no ATC interaction
    If you start a flight on the runway ATC is aware of you, you will be ‘owned’ by tower and cleared for departure. The AI, however, didn’t get the memo until you made your first radio call. Oops.
  • Further fix for OnRunwayStartMegaHack – was dependent on immediate radio access
    On-runway starts are an absolute bloody nightmare to handle! In this situation, a bug with the 737 where the radios would be unavailable for a frame or two after the sim started totally broke the AI.
  • XPD-13764: Approach clearance cancellation was not being spoken
    You’d be given go-around vectors, but not told that your approach had been cancelled, or why.
  • Synthetic flows now align to the longest runway instead of using NSEW
    When an airport definition has no runway flows defined, the sim has to invent them. Previously five flows would be created, one each for the cardinal directions and one for calm conditions. This change has the fake flows offset by the direction of the longest runway, which would usually be aligned to the prevailing wind. Hopefully this will reduce some of the valid “You’re using what runway, now?” questions – but not as effectively as airport designers actually adding flows to their airports!
  • XPD-13788: Approach cancelled if perfectly on straight-in approach a long distance away
  • Another runway-start fix for switching to AI
    I hate runway starts.
  • Route planner will consider great-circle sections
    If you ask the ATC dialog to auto-route between two airports, it was previously only considering airways or nav-to-nav. Turns out there are no defined airways across the north Atlantic, and flights from KBOS to EGLL were flying halfway down Brazil before crossing the ocean… Now it will try to pick this up and use a DCT section if following airways looks silly…
  • XPD-10555: AI can fly great-circle routes without continual corrections
    … which means the AI need to be able to actually fly them. Normally AI pop in and out of existence quite quickly and this wasn’t that noticeable – right up to the point where you ask the AI to fly a long route. Now they fly the great-circle route correctly instead of picking a straight-line heading to the destination and jink back onto the correct course every five minutes or so. This had fun consequences for trans-polar routes too; send an AI near the poles and they would just fly around it forever.
  • XPD-13768: Route generation can create routes which go past the destination, and can choose the wrong fix where multiple with the same name exist.
    Two fixes here. First, if you asked ATC to auto-route to a destination, it could be that the best fix to head for was actually beyond the destination, which makes no sense as an actual route. That was discovered at the same time as finding that in some places, two different fixes with the same name exist within a few tens of miles of each other. That feels like a bug in reality to me, but there you go.
  • XPD-13767: Crash when filing from an airport with no radio facilities
    Good ol’ fashioned crash bug. For 12.06, requesting IFR clearance on the ground from an untowered airport is disabled because the system was never intended to handle it.
  • XPD-13762: Global region now uses mbar instead of inhg for pressure.
  • XPD-13759: Adjust descent rates used to predict descent timing
    Please, folks, don’t use the “Request altitude change” over and over to manage your descent! Either wait for ATC to call it, or specifically request descent. Either way, this change pushed the decision for ATC to initiate descent out a bit.

Now for the big change – ‘feat_XPD-12784_multi_channel_atc’. This one’s been queued for quite a long time. Until now, radio frequency use was handled very simply. All controllers would both transmit and receive on all of their allocated frequencies at the same time, and you’d always be told to use the lowest frequency in their list. For small airport towers they probably only have one frequency, but for approach and regional controllers they may have more than one. In fact, some especially large controllers have dozens.

Also previously (i.e. X-Plane 11), everyone being on the same frequency really didn’t matter. AI was used less, ATC was definitely used less, and also had a lot less to say. In X-Plane 12, though, that’s changed. If you’re using both AI and ATC you’ve almost certainly found that near larger airports or even just flying cross-country legs with a controller covering a large area, you just can’t get a word in edgeways. This leads to missed turns, inability to make time-sensitive requests, avoidable go-arounds and other assorted havoc – and that was with human radio time prioritised over the AI. 

It was even worse for the AI because they are totally dependent on ATC instructions, so even short delays can put them into a situation where they suddenly need a burst of revector instructions – which increase radio load further. You can see where this is going.

With this change, a controller will actively manage frequency allocation. Instead of everyone always transmitting on a single frequency and the control facility simul-casting on all frequencies, aircraft will be allocated to other frequencies based on both anticipated and actual load. This means that, while you might still end up waiting for a burst of radio traffic to go away, you should be able to speak without too much of a delay. You might also, of course, be asked to change to a different frequency yourself (or, in betas 1 and 2, continually given this call… my bad, fixed for b3).

To be clear, this isn’t a full model of controller sectors, we just don’t have that data available. If you think of it as ‘sectors-lite’, where there are aircraft around you which are being directed by the same control facility but different staff within that facility on different frequencies, you won’t be far wrong. The short version, I hope, is that one of the key frustrations with the ATC system, a valid frustration which I saw people saying simply stopped them using ATC at all, should be gone.

And for the future? Normally of course we don’t go into too much detail about what’s planned unless it’s a hot topic, simply because shit happens and things that are planned, even if the code’s done, don’t always make it in to the next release. Or the one after that. Just this once though, I think it’s worth a little glimpse of what’s going on, just to show that outward signs of (in)activity can be misleading!

First there’s another list of fixes and improvements to come. As with the 12.06 list, here’s some of what I’m hoping will be in a very near-future release. Also as with 12.06, this is cut down and edited a little.

Coming in 12.07 (ish)

  • Ensure we issue altimeter setting for aircraft that start in the VFR/IFR approach position (i.e. 3NM/10NM from the runway, airborne).
  • XPD-14246: Fix discrepancy between flight model and ATC ISA altitude
    This is part two of the “stop whining about my altitude” bug.
  • Prevent flow changes while any aircraft is on zone 2 or higher, instead of just on final.
    You should be less likely to be revectored for a different approach if you’re already approaching.
  • Merge branch ‘feat_Untowered_Flightplan’
    This adds the ability to file an IFR flightplan with a regional controller, while on the ground at an untowered airport, including timed departure.
  • XPD-14239: Radio calls used correct indicated altitude even if altimeter setting was wrong
    When you called to check in with a controller, the altitude spoken by the pilot if you were below the transition layer was the correct one even if you’d not set your altimeter correctly.
  • Any airport with procedures can’t be a FISO.
    Airports with some kind of radio support can be full ATC or FISO. This was new for X-Plane 12 so, while this can be set per-airport in WED, most don’t have the information set about which control type they are so the sim has to guess. This simply says that if an airport has SIDs or STARs, it’s going to fully controlled.
  • Fix Austin’s AI switching control modes at intermediate hold-short points
    AI might behave differently after crossing hold points while enroute to or from the active runway.
  • Fix rotating the aircraft using the compass in the map in pause mode
    OK, I admit it, this one’s for me. I use this constantly, when testing. Previously if you used the map to change your aircraft’s heading in pause mode, the heading reset if you then changed anything else. Now the heading sticks.
  • XPD-14193: Request diversion mentions the previous destination instead of the new one.
    Duh-level typo
  • Uncertain of Position wasn’t always available when it should have been.
  • XPD-14192: Ensure the AI closes *all* aircraft doors, not just the ones it would have opened itself, when service completed.
    Yet another switch-to-AI oddity, when an AI would taxi and depart with doors open if a human opened them and the AI wouldn’t have.
  • XPD-14190: Don’t continually regenerate fake flows except at load time – if the flows are unusable, this never stops.
  • XPD-14189: LSO left/right commands were reversed
  • Disable most ATC functions in external-visuals machines.
    Only the main computer handles ATC, so why waste cycles?
  • Additional condition for allowing on-runway flightplan editing.
  • Add “Departure clearance cancelled”
    If you’re on the runway with takeoff clearance and then request taxi to parking – forgot your lunch maybe – you’ll have that clearance explicitly revoked.
  • Make com[12]_[rt]x datarefs work on aircraft with no Garmin unit
  • Fix altitudes in ATC check-in calls
  • Potential fix for being unable to contact ATC at all after cancelling services
    If you cancelled IFR in-flight then, depending on the exact state when you did it and what you do next, it was possible to end up in a situation where you couldn’t call any controller at all.
  • Change some ATC calls to allow use as VFR when filed but not cleared.
    Related to the previous bug. If you filed a flightplan, the only valid action would then be to activate it by requesting clearance. If you didn’t make this request, and took off VFR/uncontrolled, some calls were blocked.
  • Fix double call for “Cleared to Land” for VFR flights to controlled airport
  • Allow request for a different parking spot when one is already allocated. Add new “TaxiIn” state.
    You can now request a specific parking spot even after ATC has allocated a different one.
  • Expand log for inappropriate radio frequency.
    Finally, this is an example of one of the “no visible change” bugs that I’ve removed from the lists. There is a very rare problem detected in internal testing, where the system flags that an AI is on the wrong frequency. Right now there’s no clear cause, so this just adds more logging around the warning – which doesn’t appear in public builds – to help find and fix this.

Hopefully this has given an idea of some of the impact of the changes that are already in place, and what those terse bullet-points in the official release notes mean. It might also give an indication of the graceful, swan-like visible progress (hah!) compared to the furious paddling that’s normally hidden below the waterline, and I think it’s fair to say the same for all of the developers and designers, not just me working on ATC.

Looking a little further ahead

Oh yeah, nearly forgot. SIDs and STARs. If the lists of fixes and new features above look a bit sparser than you’d hope, now that Ben’s officially let the cat out of the bag I can say that this is why. SID/STAR support is written – past tense – and about to hit a lengthy period of polishing and internal testing. The key word here being “lengthy” – don’t expect to see these in 12.07, 12.08 or twelve point whatever I listed last plus one – but SID/STAR support is on the way.


Here’s something to warm the hearts of the people who complained that an overhead join with three left turns was being “vectored all over the sky”!
Posted in Air Traffic Control, News by | 30 Comments

Please Auto-Report Your Crashes

X-Plane 12.06 beta 4 is out – we pushed for a second beta this week because I’ll be out of the office next week and the installer-making machine lives in my basement, so beta five will hopefully be in about ten days. Here’s some release notes.

The team has fixed a bunch of big bugs this week, including water flashing, clicking problems with mouse-flies and a bunch of crash bugs.

The Crash Reporter…Crashed?!?

XPD-14312 was a meta-bug: the bug was that on Mac, when the sim crashed, it would hang instead of showing the auto-crash-reporting box. I’d like to use this bug as an excuse for a public service announcement:

Please always auto-report your crashes.

Our crash reporting system automatically de-duplicates crash reports but also counts the number of crashes and can tell us the number of unique users affected; we count on repeated reporting of the same crash to get a picture of the most severe crash bugs affecting the entire community. By always reporting auto-reporting the crash, you’re up-voting the importance of fixing the crash and giving us valuable insights into what’s hurting our users the most.

Do I Need to Also File A Bug?

The short answer is: no. If you auto-report a crash you do not need to file a bug.

Filing a bug can be useful if you have exact reliable steps to reproduce the bug. In this case, please file the bug, include the clear steps to reproduce, and include a Log.txt from after the crash was auto-reported. These crash logs will have a “crash UUID” at the end that lets us locate your crash inside our system.

Please do not file a bug for a crash if you do not know how to reproduce it. “I was flying and the sim just crashed, I don’t know why” provides us with no useful clues; everything we can act on is already in the auto-crash report.

What’s Next

We’re trying to wrap 12.06 up; the rest of the beta will focus on a few more rendering bugs, weather bugs, and VR bugs. While we are not going to do a massive real weather overhaul, I do have to fix the bug where it rains for no reason.

Posted in News by | 7 Comments

X-Plane 12.06 Is Full of Many Things

X-Plane 12.06 beta 1 is here! And it contains a lot of changes. Here’s a few of the bigger things we changed, and a few notes about the beta process.

Clouds and Weather

Since X-Plane 12.0 shipped, we’ve been working on the clouds and the weather system to improve performance, accuracy and quality. 12.06 ships the first two steps in this multi-step process:

  • The cloud shaders are now faster and have fewer artifacts. Daniel rewrote the way clouds are marched, fixing zebra stripes and generally making things less pixelated and ugly.
  • The cloud shaders also contain a dedicated path for cirrus clouds that should look better than what we had in 12.0 (“really thin stratus clouds up high”).
  • Alex and I rebuilt the noise functions that build each weather type to get better looking clouds of all types.

While there are a few real weather fixes included, we have not tried to comprehensively update real weather; my thinking was that without proper rendering, it would be impossible to tell if real weather was actually getting better.

Coming soon: “Minecraft clouds” (e.g. square cubic clouds, especially with real weather) should be fixed in beta 2, so enjoy them while they last. Thick prism-shaped cirrus clouds should also be fixed, and we’ll be tuning up the presets and METAR parsing.

The future: we are looking at putting a 2-d “cloud shell” behind the 3-d clouds to handle orbital view and make the planet look less silly, and we’ll be going over real weather with a fine tooth comb.

Lighting

X-Plane 12.06 fixes some constants for sky colors but is not a lighting update. We have a bunch of lighting fixes internally, but the plan is to measure twice and cut once, e.g. make one patch to update lighting once we have all of the changes.

Improving dark cockpits is high on our todo list, but we also don’t want to tweak the light levels in the cockpit over and over and over, thrashing third party developers each time we do it.

My expectation is that when we recalibrate cockpit lighting, minor aircraft updates will be needed, but third parties who have chosen to “fix” cockpit brightness themselves (by adding extra light or hacking materials) may have to undo their hacks. I’ll try to provide clear guidance and early builds when we get to this point, but lighting is still “in development”.

Rendering and VRAM

The biggest change to X-Plane 12.06 is not one you can see: we’ve converted the main rendering pipeline from 12.0 (which was hand coded) into a rendering node graph.

Rendering graphs are all the rage today; if you’re curious about this you can look at something like AMD’s Render Pipeline Shaders. But here’s the why behind this change:

The render graph allows us to double-book VRAM used to render the main frame. X-Plane 12 has a lot more stages and processing in its pipeline than X-Plane 11, and that was consuming VRAM.

In 12.06 we treat VRAM more like an AirBnB and less like a second home – at different times in the frame different parts of the pipeline are using the same chunk of VRAM, which means we need less VRAM in total for effects. This change wasn’t possible in X-Plane 11 – you can’t double-book VRAM with OpenGL.

But it also would have been too tedious to hand code aliasing – the rendering node graph automates most of this and prevents bugs.

Coming soon: we have a performance optimization for beta 2 that should help CPU-limited users.

The future: in the future the rendering node graph will also let us render different parts of the frame using different CPU cores, for better CPU utilization and higher FPS for CPU-bound users. We still have a lot of work to do on this front, but once again the rendering node graph makes it possible.

ATC and AI Aircraft

12.06 has a lot of ATC improvements – months of Jim’s work went out in beta 1. I’ll try to get Jim to write up a detailed blog post on the details.

One big improvement to ATC: Austin fixed a lot of problems with the AI pilot. This affects ATC because the AI aircraft fly more reliably and are less likely to crash and bring airport operations to a halt. We’re expecting stability improvements because numeric instability from AI aircraft crashing into mountains would sometimes crash the entire simulator.

The future: Jim has more ATC fixes coming and is working on SID/STAR support.

What Comes Next In the Development Pipeline

X-Plane development works as a pipeline: as I type this…

  • 12.05 is shipping
  • 12.06b1 is in public beta
  • 12.06b2 is being internally tested before going public
  • 12.06b3 is in development – about half of the beta 3 bugs are fixed and we’re working on the rest.
  • 12.07 development is almost finished – a mix of development and testing are going on right now.
  • We’re working on features for 12.08 and beyond.

Third parties: I believe all of the known third party compatibility bugs are scheduled to be fixed in beta 3, and most of them are fixed already. But these fixes missed the cutoff to get into beta 2, which was a few days ago. We’re hoping to get beta 3 out early next week.

We try to not hold up a beta for fixes that are almost ready – if we did the betas would never ship because there’s always one more fix that’s almost ready.

The future: Pawel’s been working quite a bit on the networking stack, and his first changes will ship in 12.07, primarily targeting pro-level customers. We also have more graphics changes coming, and some of Austin’s flight model improvement are in test right now.

Posted in News by | 66 Comments

Addressing Plugin Flickering

X-Plane 12.04b3 ships with a new feature called “Zink”, which we hope is going to alleviate a lot of the long standing issues that many users, especially with AMD GPUs have suffered from. This hopefully provides a work around for both flickering plugin drawing, eg. EFIS screens that either flicker or are missing altogether, but also potential improvements in performance by decreasing driver overhead.

If this sounds like you, you are welcome to give this a try right now by upgrading to 12.04b3, going into the Graphics settings and checking the “Use Zink plugin bridge” check box. Restart X-Plane and if everything goes to plan, X-Plane should look exactly the way you are used to but most importantly your plugins should look right and your performance might be better too. Because Zink is built on top of Vulkan, it’s only available to Windows and Linux users and is not applicable to macOS and Metal.

If you only take one thing away from this blog post, I hope it’s this image showing the difference between native Vulkan/OpenGL interop as well as Zink interop gives you an idea of why we are excited about this:

Now that I hopefully have your attention, let me elaborate how we got here and what this means for both users and developers. First of, Zink is actually a graphics driver that sits between plugins and X-Plane and translates plugin OpenGL rendering into native Vulkan commands that get executed by the same Vulkan device that X-Plane is using for rendering. Of course, X-Plane itself no longer uses OpenGL, but it’s is still very important for plugin development. A long time ago, when X-Plane was still OpenGL based and the plugin SDK was first made, it seemed like an absolute no-brainer to just expose the X-Plane OpenGL context to plugins directly and let them handle all their drawing needs themselves. It gives the most flexibility and control over everything, and it also makes it easy for the SDK because there is no need to make fancy drawing routines to be used by plugins.

Fast forward to the Vulkan and Metal port and we suddenly have an issue: Neither Vulkan nor Metal is OpenGL, but there are now hundreds of plugins out there that all assume X-Plane uses OpenGL and the OpenGL context is there. Killing OpenGL for plugins would mean countless plugins would stop working, requiring potentially lengthy update processes or maybe they’ll just never work at all again because the author lost interest in X-Plane or developing the plugin. This is how we ended up with the OpenGL Bridge in X-Plane 11.50: We create a real OpenGL context, share some memory from within Vulkan to that OpenGL context, then let plugins continue to draw how they used to do. X-Plane takes care of all of the heavy synchronization and resource creation rules under the hood and everyone is happy. Everyone? Well, no, not quite…

The problem with Vulkan/OpenGL interop

The big problem with Vulkan/OpenGL interop is that it relies on the driver to support this properly. As it turns out, driver support for this is flaky at best, if you own an AMD card you can probably tell long stories about issues. The most prominent one is flickering or altogether missing rendering, in which case plugin drawing is either completely missing or flickering in and out of existence making it basically impossible to use plugins. The other issue is performance, captured in the screenshot above, where the OpenGL bridge adds almost 10ms per frame, although I have seen numbers as bad as 30ms per frame. For reference, 33ms is the time one frame can take at most to reach 30 FPS! This wasn’t always the case, it used to run much better, but driver regressions are a real thing and this isn’t really something the hardware vendors test. Not that I can be too mad about this, X-Plane is one of the very few applications that uses this feature at all. Despite having filed numerous bug reports with the vendors, unfortunately the problems were never completely fixed, so even during the 11.50 days we started thinking about alternatives. During the X-Plane 11 days it was at least somewhat easy to tell users to switch back to OpenGL to avoid the flickering, even though that usually meant a hit in performance because the Vulkan backend was just so much faster.

There are of course other open source OpenGL drivers out there that aren’t Zink. The most well known one is probably Angle, a implementation of OpenGL ES by Google. None of them work for us however, because of another terrible decision made many years ago: X-Plane has always used a compatibility profile OpenGL context. What that means doesn’t matter, but the end result is that X-Plane and by extension plugins, rely on the existence of OpenGL commands from the 90s! Most modern implementations of OpenGL will happily ignore everything from the compatibility profile because it’s a nightmare to implement and doesn’t map very nicely to modern hardware at all. But a lot of plugins make use of these old features and rely on the fixed function pipeline. Which, by the way, is also the reason we didn’t just expose Vulkan or Metal to developers. Writing either a Vulkan or Metal renderer requires a ton of overhead, is deeply complicated and error prone. Writing correct code requires testing on different hardware, reading a few thousand pages of specification text and having a good understanding of the underlying hardware. Most plugin developers don’t venture past the fixed function pipeline, it would be absolutely unreasonable to expect them to spend the time and effort involved in this.

What’s a Zink?

Luckily we don’t have to throw any developers into the deep end because there is Zink. Zink is part of the Mesa open source project and is a backend for the Gallium driver, which is the OpenGL frontend provided by Mesa. Okay, lots of big words here, let’s simplify this a bit: Gallium implements OpenGL 1.0 to OpenGL 4.6, which basically means “all of OpenGL”. But it doesn’t actually do anything with it, it just does all of the heavy lifting required to make OpenGL work. Backends are what actually turn the intermediate data that Gallium produces into real commands to be executed by a GPU. There are a lot of these backends: For example there are multiple software renderers that just use the CPU, there are implementations that talk to AMD GPUs, Nvidia GPUs, Intel GPUs, things I never even heard about, it’s all there. But among all of those backends there is also Zink. Zink just takes what Gallium produces and builds Vulkan commands out of it to be executed by any Vulkan capable hardware. This is what makes Zink so great for us, it’s a real OpenGL implementation that includes all of the hard and complicated bits we need to make plugins work.

Of course I’ve made it sound easier than it really is, “just” writing a Gallium backend that magically speaks Vulkan is a lot of hard work. The unsung hero of all this is Mike Blumenkrantz, head honcho and lead developer of Zink. He’s spent years getting Zink to the point it is at and is contentiously working on improving it, both in terms of supported features as well as performance. He’s also an absolute legend and none of this Zink interop X-Plane stuff would exist if it wasn’t for his help, not just in getting Zink to where it is but also answering tons of our questions during the integration. Big shout out to Mike! While I’m doing shout outs, it would be unfair of me to not also mention AMD. While they are partially the reason we are here today, they also provided us with help and driver updates to make Zink work together with X-Plane and every engineer of theirs that I have met along this journey has been nothing but kind and helpful.

I’m a user, how does this affect me?

If you are on X-Plane 12.04b3, open your graphics settings and enable the Zink backend. By default we run with the native OpenGL backend for compatibility reasons, although the long term goal is to switch to Zink exclusively, but not anytime soon. There is a chance some or all of your plugins will explode, we’ve done testing with third party developers on this, but the area touched by this immense and we can’t test every plugin out there. If it breaks, no worries, please file a bug report and let us know, fall back to the native OpenGL backend and try again in the future. The idea behind Zink is to make it as much a “it just works” thing as possible, but it’s the first step and if it isn’t perfect right out of the gate, I hope you have some patience with us. I’m saying this because like I said, the surface area touched is immense here and shipping a whole graphics driver with an application isn’t really something super common and, as I have found out, sometimes breaks stuff.

The good news is that Zink is open source and we are building it ourselves, which means that for the first time we can actually see under the hood when things explode. I have very high hopes that this will help us track down issues much faster than usually. Normally drivers come with all debug data stripped, so a few times I have seen bugs that just end somewhere in the driver with no way to tell what even happened or how to reproduce it. Sometimes we don’t even have any information at all because it got so thoroughly muddied.

I’m a developer, how does this affect me?

Well, same as above: Test your plugins, file bugs, be patient. But also, there are a few things you might want to be aware of in general:

First, X-Plane now finally lets you enable a debug GL context. If you run X-Plane with --debug_gl, X-Plane will make a debug GL context for you and then wire up the necessary callbacks. By default it’ll print all messages into stdout and errors get a big in sim pop up. But you can always redirect it to something of your choosing by calling glDebugMessageCallbackARB() with your own callback. Please don’t ship plugins with a handler enabled though, because it might lead to other developers not being able to set their own callback as there can only be one. The debug GL context is available when running through Zink and also the native GL driver.

Gotcha wise, there are two that I’m aware of:

  • In theory you can make a shared GL context for background GL processing, but it might not be 100% stable. I’m still trying to track down an issue somewhere in the stack that leads to resources being incorrectly deleted, so if you run into issues with a shared GL context, this might be it. Please try it though!
  • Don’t disable and/or enable GL_FRAMEBUFFER_SRGB, this will lead to all subsequent rendering to disappear. This bug I have tracked down further into Mesa and the problem is that the driver will create a copy of the image to add or remove the sRGB bit, but it never ends up copying the result back over. In theory this should be handled by just a view on the same image without any copies, but either way this is currently broken.

If you want to test Zink as part of some CI solution, you can also run X-Plane with --zink or --no_zink through the command line and it’ll override whatever was set in the graphics settings.

I’m on macOS and I feel left out

First of all, I’m sorry to hear that. 12.04 has great things for macOS users as well that I’m sure you’ll love. Long term, we are having ideas of running Zink on macOS too, just this time on top of Metal. GL on macOS is already emulated through Metal, it’s just done by Apple without visibility or ways to tap into that. We have a bit of a pipe dream of running plugins on top of Zink on top of MoltenVK on top of Metal, but this is purely in the ideas bin right now and no code for this has been written. There would be two advantages for this approach though: First, Apple has deprecated OpenGL already in macOS and while I do believe they will continue to support it, lest their pro users will climb on their roof and shout bloody murder, they clearly have no interest in expanding OpenGL support any further. Secondly, if we get OpenGL bridging down to just Zink on all platforms, it’d be a boon for developers because they can target just one OpenGL implementation and cover all platforms in one go. It’d also finally bring OpenGL 4.6 support to macOS, which is currently held back to OpenGL 2.1.

Posted in Development, News, Plugins by | 20 Comments

Altitude Management: A Tale of Ups and Downs in the ATC System

Deciding what altitude a particular flight segment needs shouldn’t be tricky, right? You take off from somewhere near ground level, you climb to whatever you chose for cruise altitude, you start to descend when you get near the destination and eventually end up on the ground again. What could possibly go wrong?

Why, I’m glad you asked! Make yourselves comfortable, this might take some time.

A bug report arrived from someone saying they’d been vectored into a hillside on approach. That’s something that has had a lot of attention given to it for X-Plane 12, so I jumped on it and reproduced the flight. Happily the person reporting had included the correct logfile (not everyone does, we get loads where the log shows the sim sitting at the main menu!) and given the flightplan details; nothing complicated, a simple direct flight between two airports about 35 miles apart, cruising at 6000 feet.

I set it up and let the AI fly. Takeoff was fine, and then it was told to climb to 8000 feet – above the requested cruise altitude. Checking the map, okay, there was a mountain ridge in the way so that makes sense. Then the problems started. Descend to 7000, then descend to 5500, then climb to 7000, then an endless loop of climb/descend to what looked like random choices of 5000, 6000, 7000 and 8000 feet.

All of the “normal” – and a lot of the unusual – situations for approach and landing are covered in automated tests and they’d last passed the previous day, so I knew that this wasn’t a common problem but it would still be damned annoying if it happened to you.

Digging in a bit, the first thing I found was a good old-fashioned bug. Before X-Plane 12, the ATC system was pretty much exclusively written for airliners and an old bit of code had a problem where, if you were below a normal cruising altitude for an airliner, you might be told to climb slightly before starting the descent. This was kicking in because, when the AI was handed off from tower to the next level controller, it was already close enough to the destination airfield to be seen as an incoming arrival.

If you think about it, this completely breaks the simple altitude model already. The Cessna 172 climbs slowly, so it was approaching the whole “descend, approach, land” sequence from below. X-Plane 12 is wise to this though, so it shouldn’t have been a problem.

So, we’ve got one inappropriate climb instruction sorted. Re-test.

As expected, much the same as before; total indecision on the part of the controller. The next problem soon comes to light: terrain avoidance.

On the face of it, terrain avoidance should also be simple: If hill then climb. Most of the time, it is. Where that falls down is where you’re deliberately trying to get close to the ground like, say, during approach and landing.

If you’re familiar with aviation charts, you’ll know the acronym MSA – Minimum Safe Altitude. Basically the chart is divided into large squares and, in each square, there’s an MSA given. If you fly through that square, you’ll be guaranteed not to hit the ground if you stay above that altitude. It’s simple, it works, and X-Plane does pretty much exactly the same. The terrain elevation data is reduced so that a single pixel gives the MSA for whatever area that pixel covers, and this makes it relatively quick and easy to check.

This flight had a couple of steep ridges going from sea-level to about 6,200 feet which ended just inside one MSA square, and the flight was going just along the top of it at about 90 degrees to the ridges. A few hundred meters difference in location when the terrain samples were taken were enough to change the MSA between about 50 feet and 6,200 feet.

Checking a little to either side of the route can make this problem more or less go away, and that’s done in some places, but it can’t be used during approach because you’re actively trying to get close to the ground. If you went wide on the terrain checking, you’d never be allowed to land at any airport in a valley or just generally mountainous terrain.

So, back to the problem. The terrain avoidance systems were getting two very different readings depending on both exact timing and position because they were just clipping the edge of the MSA squares and one of those squares covered high ridges which finished at sea level just inside the square’s boundary. One moment it would think you had 5,000 feet of clear air beneath you, the next it was asking the poor, confused AI to climb to avoid terrain.

This problem can’t really be fixed as such, but it can be improved by reducing the sample distance and using full-resolution elevation data instead of the MSA-square summary. The problem with that is that it’s more expensive in CPU terms so it can’t be done all the time. Think of a flight from London to Delhi; checking that for every 30 meters along the route is going to take a long time, and not all of the high-res data is even in memory at once. However, high-res data is already used in some cases, notably for short legs close to the MSA or for known approach legs, but this was happening on a longer cross-country leg which happened to be the one before the actual approach started. Still, easy enough to adjust for that so let’s do that and see what happens.

The next flights were much, much better. Yay! Go for a celebratory coffee, but something nags at the back of my mind.

Coffee over, check the exact routes on the map (anyone can do this – it’s on the Developer menu as “Toggle Air Traffic Paths”). Yep. The wind had changed – I had real weather enabled – and the cross-country leg was starting a couple of miles further north, which meant that all the nonsense with clipping corners in the terrain avoidance code had simply stopped being an issue. 

Righto.

Set manual weather, re-test, and… well, it’s better, but still bouncing between two different altitudes even though the MSA’s now more or less stable.

If you remember, this is all happening during approach. When these segments are created, the terrain clearance for each one is checked and the overall approach profile is modified so that you’re not told to descend to the normal altitude, then climb again for terrain avoidance for a later part of the approach. For this flight, the destination airport was on the plains near the coast but surrounded by high mountains to the south, so this system was kicking in and keeping the approach altitudes much higher than they would normally be. However, there’s one more waypoint involved; this is created somewhere between you and the airport and used as a kind of gateway between cruise and approach. The segment after this waypoint was also being terrain-checked, but if any terrain-related adjustments were made to the subsequent approach segments, they weren’t being pushed backwards onto this pre-approach leg. Normally, this wouldn’t happen and even if the approach were modified to clear terrain, it would have to be a very unusual, large adjustment to kick the approach legs higher than the pre-approach waypoint. 

Yep, you guessed it.

Still, at least that’s a simple one to fix. Do that, re-test.

This fix is good and the next set of altitudes is… oddly low. So low, in fact, that the “oh shit” mode of the terrain avoidance kicks in. This ignores the route entirely and just uses the plane’s recent movements to predict a near-future location and checks that. This means that the MSA is being ignored somewhere, and there’s now a missing climb instruction.

More testing and this turns out to be caused by a check, during approach, which tries to avoid unnecessary climb instructions. If you’re already at low altitude and call in to an airport to land – i.e. a typical small-plane flight – you don’t want to be told to climb thousands of feet only to descend back to pretty much the altitude you’re already at. So, the controller tries to be nice and lowers parts of the approach if you’re already underneath it. That’s all fine, but… not… if… the approach has been raised to avoid terrain!

Fix that, re-test.

All is well for the first half of the flight. I know, from watching the code as it runs, that from here on in the MSA should be in the region of 5,200 to 7,200 feet depending on the exact timing. High resolution elevation data is being used now, the two mountain ridges are steep from left to right and fairly narrow back to front in relation to the flight direction, so the MSA can vary quite quickly. The expected climb instruction comes in: “Climb to FL080”. Wait; eight thousand?

This was a flight-level though. Checking the transition altitude (the point where it stops using “feet” for altitude and starts using “flight level” and a specific set of rules for choosing them) for the approach controller shows that it’s set at 6000 feet. Yet again, tiny changes in both timing and position were affecting the MSA, but this time it was because the high-resolution elevation data was being used. For the low end, a simple “add some wiggle room and round up to the next 500 feet” was coming up with either 5500 or 6000 feet. At the high end, it was being rounded up into flight-level territory and, because of the direction of flight, punted upwards to either FL070 or FL080. This was only happening because the requested altitude in the plan was exactly on the transition altitude!

Okay, so this one’s probably confusing, but not technically wrong – certainly within the limits of the system.

So the original, simple “Let the AI fly a route and see if it starts tunnelling” check has now taken three days to properly resolve, between investigation, code changes, many re-tests, and then validating the changes on a larger scale using the existing automated tests. On the plus side, it’s been an absolute torture-test for a number of systems which worked just fine in most circumstances, and has brought out problems that were able to be either fixed or at least explained. On the down-side… yeesh!

For a normal flight, none of this would have been a problem. You’d be way above the terrain, descending onto the approach from above. This flight was short, so the slow-climbing Cessna 172 was still on climbout when the approach vectors were issued for altitudes higher than it was currently at. In between the two airports there were ridges which just clipped terrain-altitude data squares so MSA checks returned altitudes which varied significantly with tiny variations in both position and timing. The destination airport had no flows defined, so X-Plane was auto-generating them, but this doesn’t look at terrain so thanks to the airport being in a steep valley, the MSA of the approach vectors unusually remained at 4,500 feet all the way in to final for an airport close to sea-level. Finally, the planned altitude was exactly the controller’s transition altitude, magnifying small differences which would normally be rounded away.

If either airport had been a few miles north or south, or further apart; if the ridges had stopped a few miles further south; if the coast had been a few miles north or south; if the aircraft in use had been able to climb to cruise altitude faster; if the plan had requested an altitude which was above the controller’s transition altitude; if the terrain MSAs hadn’t also straddled the transition altitude; if the airport had defined flows which prevented landings from the south; if the wind had been different. If any of these things had not been exactly as they were, little or none of this would have happened.

So, my – er – thanks to the person that filed this bug report! Completely by accident, you sent several systems beyond what they were able to do and, by filing the report and actually attaching the log from the same flight, allowed me to dig into the system and get rid of a good number of rough edges. It’s also given a fantastic example of “Pssh, how hard can it be?” for the obvious assumption of “well, you climb, then you descend, job done”.

Plus a couple of sleepless nights, but hey.

Oh, the “vectored into a hill” problem? Couldn’t reproduce it.

Posted in Air Traffic Control, News by | 18 Comments