Deciding what altitude a particular flight segment needs shouldn’t be tricky, right? You take off from somewhere near ground level, you climb to whatever you chose for cruise altitude, you start to descend when you get near the destination and eventually end up on the ground again. What could possibly go wrong?
Why, I’m glad you asked! Make yourselves comfortable, this might take some time.
A bug report arrived from someone saying they’d been vectored into a hillside on approach. That’s something that has had a lot of attention given to it for X-Plane 12, so I jumped on it and reproduced the flight. Happily the person reporting had included the correct logfile (not everyone does, we get loads where the log shows the sim sitting at the main menu!) and given the flightplan details; nothing complicated, a simple direct flight between two airports about 35 miles apart, cruising at 6000 feet.
I set it up and let the AI fly. Takeoff was fine, and then it was told to climb to 8000 feet – above the requested cruise altitude. Checking the map, okay, there was a mountain ridge in the way so that makes sense. Then the problems started. Descend to 7000, then descend to 5500, then climb to 7000, then an endless loop of climb/descend to what looked like random choices of 5000, 6000, 7000 and 8000 feet.
All of the “normal” – and a lot of the unusual – situations for approach and landing are covered in automated tests and they’d last passed the previous day, so I knew that this wasn’t a common problem but it would still be damned annoying if it happened to you.
Digging in a bit, the first thing I found was a good old-fashioned bug. Before X-Plane 12, the ATC system was pretty much exclusively written for airliners and an old bit of code had a problem where, if you were below a normal cruising altitude for an airliner, you might be told to climb slightly before starting the descent. This was kicking in because, when the AI was handed off from tower to the next level controller, it was already close enough to the destination airfield to be seen as an incoming arrival.
If you think about it, this completely breaks the simple altitude model already. The Cessna 172 climbs slowly, so it was approaching the whole “descend, approach, land” sequence from below. X-Plane 12 is wise to this though, so it shouldn’t have been a problem.
So, we’ve got one inappropriate climb instruction sorted. Re-test.
As expected, much the same as before; total indecision on the part of the controller. The next problem soon comes to light: terrain avoidance.
On the face of it, terrain avoidance should also be simple: If hill then climb. Most of the time, it is. Where that falls down is where you’re deliberately trying to get close to the ground like, say, during approach and landing.
If you’re familiar with aviation charts, you’ll know the acronym MSA – Minimum Safe Altitude. Basically the chart is divided into large squares and, in each square, there’s an MSA given. If you fly through that square, you’ll be guaranteed not to hit the ground if you stay above that altitude. It’s simple, it works, and X-Plane does pretty much exactly the same. The terrain elevation data is reduced so that a single pixel gives the MSA for whatever area that pixel covers, and this makes it relatively quick and easy to check.
This flight had a couple of steep ridges going from sea-level to about 6,200 feet which ended just inside one MSA square, and the flight was going just along the top of it at about 90 degrees to the ridges. A few hundred meters difference in location when the terrain samples were taken were enough to change the MSA between about 50 feet and 6,200 feet.
Checking a little to either side of the route can make this problem more or less go away, and that’s done in some places, but it can’t be used during approach because you’re actively trying to get close to the ground. If you went wide on the terrain checking, you’d never be allowed to land at any airport in a valley or just generally mountainous terrain.
So, back to the problem. The terrain avoidance systems were getting two very different readings depending on both exact timing and position because they were just clipping the edge of the MSA squares and one of those squares covered high ridges which finished at sea level just inside the square’s boundary. One moment it would think you had 5,000 feet of clear air beneath you, the next it was asking the poor, confused AI to climb to avoid terrain.
This problem can’t really be fixed as such, but it can be improved by reducing the sample distance and using full-resolution elevation data instead of the MSA-square summary. The problem with that is that it’s more expensive in CPU terms so it can’t be done all the time. Think of a flight from London to Delhi; checking that for every 30 meters along the route is going to take a long time, and not all of the high-res data is even in memory at once. However, high-res data is already used in some cases, notably for short legs close to the MSA or for known approach legs, but this was happening on a longer cross-country leg which happened to be the one before the actual approach started. Still, easy enough to adjust for that so let’s do that and see what happens.
The next flights were much, much better. Yay! Go for a celebratory coffee, but something nags at the back of my mind.
Coffee over, check the exact routes on the map (anyone can do this – it’s on the Developer menu as “Toggle Air Traffic Paths”). Yep. The wind had changed – I had real weather enabled – and the cross-country leg was starting a couple of miles further north, which meant that all the nonsense with clipping corners in the terrain avoidance code had simply stopped being an issue.
Righto.
Set manual weather, re-test, and… well, it’s better, but still bouncing between two different altitudes even though the MSA’s now more or less stable.
If you remember, this is all happening during approach. When these segments are created, the terrain clearance for each one is checked and the overall approach profile is modified so that you’re not told to descend to the normal altitude, then climb again for terrain avoidance for a later part of the approach. For this flight, the destination airport was on the plains near the coast but surrounded by high mountains to the south, so this system was kicking in and keeping the approach altitudes much higher than they would normally be. However, there’s one more waypoint involved; this is created somewhere between you and the airport and used as a kind of gateway between cruise and approach. The segment after this waypoint was also being terrain-checked, but if any terrain-related adjustments were made to the subsequent approach segments, they weren’t being pushed backwards onto this pre-approach leg. Normally, this wouldn’t happen and even if the approach were modified to clear terrain, it would have to be a very unusual, large adjustment to kick the approach legs higher than the pre-approach waypoint.
Yep, you guessed it.
Still, at least that’s a simple one to fix. Do that, re-test.
This fix is good and the next set of altitudes is… oddly low. So low, in fact, that the “oh shit” mode of the terrain avoidance kicks in. This ignores the route entirely and just uses the plane’s recent movements to predict a near-future location and checks that. This means that the MSA is being ignored somewhere, and there’s now a missing climb instruction.
More testing and this turns out to be caused by a check, during approach, which tries to avoid unnecessary climb instructions. If you’re already at low altitude and call in to an airport to land – i.e. a typical small-plane flight – you don’t want to be told to climb thousands of feet only to descend back to pretty much the altitude you’re already at. So, the controller tries to be nice and lowers parts of the approach if you’re already underneath it. That’s all fine, but… not… if… the approach has been raised to avoid terrain!
Fix that, re-test.
All is well for the first half of the flight. I know, from watching the code as it runs, that from here on in the MSA should be in the region of 5,200 to 7,200 feet depending on the exact timing. High resolution elevation data is being used now, the two mountain ridges are steep from left to right and fairly narrow back to front in relation to the flight direction, so the MSA can vary quite quickly. The expected climb instruction comes in: “Climb to FL080”. Wait; eight thousand?
This was a flight-level though. Checking the transition altitude (the point where it stops using “feet” for altitude and starts using “flight level” and a specific set of rules for choosing them) for the approach controller shows that it’s set at 6000 feet. Yet again, tiny changes in both timing and position were affecting the MSA, but this time it was because the high-resolution elevation data was being used. For the low end, a simple “add some wiggle room and round up to the next 500 feet” was coming up with either 5500 or 6000 feet. At the high end, it was being rounded up into flight-level territory and, because of the direction of flight, punted upwards to either FL070 or FL080. This was only happening because the requested altitude in the plan was exactly on the transition altitude!
Okay, so this one’s probably confusing, but not technically wrong – certainly within the limits of the system.
So the original, simple “Let the AI fly a route and see if it starts tunnelling” check has now taken three days to properly resolve, between investigation, code changes, many re-tests, and then validating the changes on a larger scale using the existing automated tests. On the plus side, it’s been an absolute torture-test for a number of systems which worked just fine in most circumstances, and has brought out problems that were able to be either fixed or at least explained. On the down-side… yeesh!
For a normal flight, none of this would have been a problem. You’d be way above the terrain, descending onto the approach from above. This flight was short, so the slow-climbing Cessna 172 was still on climbout when the approach vectors were issued for altitudes higher than it was currently at. In between the two airports there were ridges which just clipped terrain-altitude data squares so MSA checks returned altitudes which varied significantly with tiny variations in both position and timing. The destination airport had no flows defined, so X-Plane was auto-generating them, but this doesn’t look at terrain so thanks to the airport being in a steep valley, the MSA of the approach vectors unusually remained at 4,500 feet all the way in to final for an airport close to sea-level. Finally, the planned altitude was exactly the controller’s transition altitude, magnifying small differences which would normally be rounded away.
If either airport had been a few miles north or south, or further apart; if the ridges had stopped a few miles further south; if the coast had been a few miles north or south; if the aircraft in use had been able to climb to cruise altitude faster; if the plan had requested an altitude which was above the controller’s transition altitude; if the terrain MSAs hadn’t also straddled the transition altitude; if the airport had defined flows which prevented landings from the south; if the wind had been different. If any of these things had not been exactly as they were, little or none of this would have happened.
So, my – er – thanks to the person that filed this bug report! Completely by accident, you sent several systems beyond what they were able to do and, by filing the report and actually attaching the log from the same flight, allowed me to dig into the system and get rid of a good number of rough edges. It’s also given a fantastic example of “Pssh, how hard can it be?” for the obvious assumption of “well, you climb, then you descend, job done”.
Plus a couple of sleepless nights, but hey.
Oh, the “vectored into a hill” problem? Couldn’t reproduce it.
Jim this is good stuff, thanks. Will X-Plane ATC no longer vector to non-towered fields when on IFR flight plans or is this a work in process? I ask because I seem to always get a CTD in the vicinity of the non-towered field.
It should be able to handle that situation and, even if not, a crash is never the right action. Do file it.
Thanks for those explanations !
I did not send a report yet for this but the controller can be somewhat stubborn repeating all the time you were instructed to climb. Yes, __I KNOW__ but, in a GA at 90kt, fighting with some ice formation at +100fpm, it’s honestly annoying to hear that instruction in a high frequency loop…
Do you need a report for this ?
If you haven’t reported it, it’s not a bug 🙂
Please include the situation too. For example, depending on where you are in the flight it may be better to request a lower cruise altitude, for example. There’s no support at the moment for “unable” calls.
Love this stuff, it’s brilliant… thanks Jim
I think you’re approaching this issue a little bit different to how it’s meant to work in the real world, at least according to the regulations aviators are held to. And the experience described (with bouncing back and forth between MSA / LSALT / MVA levels) sounds like a junior Air Traffic Controller on their first day.
I’ll reference this in relation to the publications and orders pertinent in Australia (easily obtained from the Airservices gov au website), as well as the Air Traffic Services rules and regulations (also obtained in the same manner).
I don’t know what grid spacing X-Plane is using for MSA data (I’m guessing it’s derived from the SRTM terrain models), but the fidelity for aviation charts is provided as follows:
– ERC-H 4˚ grid intervals
– ERC-L and TAC 1˚ grid intervals
(AIP GEN 3.2), with detailed explanation in (GPA GEN 3.2, 2.2.2).
It is good practice to be continually aware of the grid LSALT for where you’re currently operating. As you’d expect to be high level and on established routes for EGLL – VIDP, the LSALT you’d need to be aware of would be the 4˚ interval one. So, the 30m check is basically useless, as you point out. The fidelity reduces to highest point within a 240NM x 240NM box (ignoring longitudinal reduction as latitude increases). You can create a lookup table of each grid square’s LSALT to match what appears on most aviation enroute and mapping charts (129,600 LSALTs at 1˚x1˚, and 8,100 at 4˚x4˚ for global coverage, but clearly a lot of ocean is in that table).
Calculation of LSALT for other operations appears in many different places, but I’ll refer to the Australian Defence Force GPA (General Planning Australia), as they routinely conduct operations off-route in controlled and uncontrolled airspace at all levels, as well as operations into and out of uncontrolled aerodromes and in locations with limited aviation data. GPA GEN 3.3, 4 provides significant detail on the calculation of LSALT, as well as confirmatory details on how route safes are determined.
Key from this is the max lateral separation from track for LSALT considerations is 50 NM, but for operations on established air routes or under positive ATC control, those considerations are going to be reduced (not sure if X-Plane has that data incorporated as part of air route data). It would seem, from the description provided, that the planned route altitude was below LSALT. In the absence of knowledge of a dedicated route existing, or special local procedures, that flight plan should have been rejected (or at least amended to the higher LSALT). Also, given the lateral separation described for the aerodromes, I would have expected a rule-of-thumb cruise altitude in the vicinity of 3000’ (before terrain and other considerations). This way, the aircraft should have had an actual cruise phase before commencing descent.
Based off the details in the post, it would seem that the flight was IFR, and the pilot had not been cleared VISUAL at any point (so terrain avoidance is provided by ATC). From the Manual of Air Traffic Services (Australia, available from Airservices), ATC has an obligation not to vector an aircraft at a pilot calculated LSALT below the actual (MATS 9.7.10.2). When in en route control (outside the terminal area, which appears to be what happened in the post), the directive is that altitudes assigned are to be no lower than the Grid LSALT, and that a 5 NM buffer is applied to the grid boundary to ensure the aircraft is established at or above the higher Grid LSALT prior to entering the higher LSALT, or outside it prior to descent below that level. (MATS 9.7.11.2).
Now, given that the lateral separation of both airfields was less than 100NM (and assuming both were controlled), ATC may have provided RTCC / MSA / MVA terrain clearance where there is a 3 NM lateral separation from the aircraft from the obstacle (5 NM on scales greater than 50 NM) (MATS 9.7.11.4). This comes back into the question of how often and how wide do you sample LSALT? In the real world, it’s continuous. In order for a decent fidelity sim, I’d say within 10 NM of the field, go high fidelity (RTCCs and highly detailed Sector Safes, accurate surveying), 10-50 medium, and outside 50, sample using the lookup table at the appropriate grid spacing for the altitude (i.e. 1×1 or 4×4). And a 30 NM leg isn’t really a long cross-country leg. It’s almost terminal to terminal…
With the transit leg being offset due to wind / departure direction, that is a failure of ATC direction. The vectoring and direction from ATC should have been tracking to intercept the flight-planned track (and establishing on it) before departing the terminal area of the departure field. This is expected real-world behaviour, and also provides reproducible tracking (again, expected). With a dedicated offset from the planned track, then the LSALT calculations balloon out significantly – since you’re no longer tracking IAW the planned methodology. Even if vectored continually between the airfields, ATC is directing headings to achieve ground tracks, so the net ground track should have been equivalent.
When it comes to the approach, MATS 11.1 covers in detail the vectoring and altitude reductions allowed to be given by ATC. But, as you point out, the airfield in question lacks published arrival procedures (no detail on IAPs available). As such, the ATC direction should be to ontop the field (basically arrive in the circuit) for transition to a visual approach, or vector to the IAF for a selected IAP (and thus terrain avoidance established by visual – pilot responsibility, or the published IAP). If the airfield is tightly constrained by terrain and lacking in STAR, then, yeah, you’re going to arrive high above it under IFR rules. That should be expected behaviour. And is, when arriving at uncontrolled airfields all the time (something which some regions do better than others).
Now, if ATC in the sim is making up a GNSS-style approach, and STAR-like vectoring, then that becomes a little issue. Particularly if it’s “lowering” the steps for aircraft arriving below it… That’s a real issue. A REALLY BAD ISSUE. The correct action should be that if an arriving aircraft is below the invented approach level, they should track accordingly, but hold their altitude until established within the procedure. So, instead of shifting the approach profile, keep the aircraft at a steady (and safe) altitude until it is established within all 3 dimensions of the profile, then continue descent and arrival. If the correct LSALT calculations and clearances have happened prior to this, then there is no way the aircraft should be in danger of hitting terrain, as the direction to descend below MSA will never be given (c.f. MATS extracts referenced above).
If you take a look at the approach plates to 01 at YBTL (Townsville), in particular the DME / GNSS arrivals, you’ll see that there are sectors where you’re entering the approach ABOVE 5000’ only 13 NM from touchdown, and the airfield is only on the order of 60’ AMSL. And there’s significant terrain in the local area to avoid. So, the concept of being held at altitude until really close to the airfield is real. This is why we have alternate planning. Because sometimes there’s no safe way to get below MSAs. Even if we “know” the terrain is clear, it’s a good way to find rock-filled clouds.
Either that or you call VISUAL when you can and be given clearance to land and maintain your own terrain separation.
So, how do you address this in a sim with tens of thousands of airfields, for which you may not have the 10 and 25 NM MSAs available (instead having to derive from SRTM)? Well, within 10 NM of the ARP (Aerodrome Reference Point), pull the highest MSA that is from fine grained data. Between 10-25 NM, pull one that is from medium-grained data. And outside 25NM it’s Grid Safe. If a flight is being directed to a NPA (Non-Precision Approach), then they are directed to the IAF and walked down Grid Safe -> 25NM MSA -> 10NM MSA -> IAF.
If, on the other hand, the sim is generating a non-published precision, which simulates a Precision Radar approach by ATC, then they can be talked down steps using the medium grained data (10-25 NM, within 3 NM of aircraft), then onto the fine grained data (within 10 NM). And the approach will look not too dissimilar to a DME / GNSS / RNAV-Z.
Another example where high MSAs are present is YCTM (Cootamundra). The airfield sits within a bowl valley, at 1100’. The 10NM and 25NM MSAs are 4200’. So, you could (and should plan to) arrive 3000’ AGL over the top of the airfield if going for a visual attempt, but without attempting the RNAV-Z (GNSS). Even on the GNSS approach, the aircraft is above 4600’ at 10NM from the airfield, on extended centreline. There are even examples where the 25NM MSA may be LOWER than the 10NM MSA. Look at YBCG (Gold Coast / Coolangatta). Approaching from over the ocean (R318˚-R166˚), the 25 NM MSA is 3000’. From R272˚ to R318˚, it’s 4000’, and 5100’ for the rest. Within 10NM, it’s 4600’. This is because of significant terrain immediately West of the airfield, despite the airfield being at 12’. So, it is possible if receiving vectors, for an aircraft to arrive at a lower altitude and then be climbed to make the 10NM MSA if not visual (or to make the IAP). Also, check the massive altitude steps for the published GNSS approach.
Other useful extracts from the AIP and MATS:
AIP ENR ENR 1.1 -18, 2.5, 2.11.3.7
AIP GEN 3.3 -6, 4
MATS 9.4.1.2 Assignable Levels – Plus RTCC
MATS 9.4.2 Ops below LSALT
MATS 9.7.11.3 Pilot terrain avoidance
MATS 9.7.11.4 Scales and obstacle clearances
It’s always worth remembering that any simulation offers an approximation of what happens in the real world. That approximation may – and should – get better over time, but there will still be technical reasons or limitations that affect it. This does show how complex the problem is to solve, which is mostly what I was driving at. Many people would quite understandably ask “how hard can it be?” for what looks like a trivial issue. Well, as you’ve pointed out, it’s not that trivial after all!
Thanks, this was very interesting to read!
Just to give some real world perspective. The MSA is very important, but not the only tool in the box for terrain clearance. On a lot of airports, the approach will start below MSA. There are different ways to legally and safely go below the MSA for pilots. First, there might be an arrival route with lower minimum enroute altitudes. These are not area-bound, but airway-bound minimum safe altitudes. Secondly, ATC can have (unpublished) minimum radar vectoring altitudes which allows them to direct planes lower than the MSA when under radar guidance. This also requires careful verification by the flight crews with the terrain and obstacles in the map. In absence of any other solution, the flight crew could also navigate to the intial approach fix above MSA, then descend at that point in the hold, and commence the approach from that point. So pilots in mountainous terrain often will get below MSA, and there are safe ways to do it. It is also completly realistic to get vectored below MSA with a tighter terrain clearance.
Right – so we’re currently working in a broadly similar way to the radar options right now. Once proper approach routes are included that should clarify things too.
Nice post, a very interesting case study. It just goes to show how a weird bug can force you to go through a lot of systems!
Hi Jim, can you please elaborate on this . “Happily the person reporting had included the correct logfile.”
Is there a dedicated log file for ATC activities or is it the normal log file located in X-Plane 12/Log.txt ?
No, just the plain ol’ Log.txt . There is a lot of ATC stuff in there at the moment which would be nice to get separated out. No, all I meant was that in probably 60% of tickets, the Log.txt shows the sim being opened, getting to the main menu, and closing again.
If you just don’t have the log any more then I get it, the bug reporter does insist on a logfile being attached so you’ve got to attach something and I’d rather the bug was filed with an empty log than not filed at all. If possible though, please do either report a bug immediately (i.e. don’t run the sim again first), or copy/rename Log.txt so that you can attach the right one later. It really does make a huge difference to ATC diagnostics, that’s the reason the log’s so busy!
Are there best practices for designing gateway airport when it comes to utilizing the built in ATC features? There are some airports where ATC is not available and reasons are unknown.
Julian Lockwood created a document about utilizing new features in XP12 for gateway airports. A design checklist in WED to makes airport ‘ATC friendly/compatible’ in that document will be valuable.
I thought there was something in there to be honest 🙂
In descending order of importance:
Always add a taxi network – that’s the invisible set of connections showing where planes should taxi, not visible polygons. And keep it simple, hundreds of tiny segments for smooth corners hinder rather than help.
Always add runway flows, and test them under various conditions to make sure they do what you think they do. Remember to add a ‘calm’ flow, that stops the active runway flipping around too much.
Add names to taxiways and parking spots if possible because these are now shown on-screen and spoken.
Hi, is it possible to add the runways in the flightplan (departure/arrival)? Sometimes Atc doesn’t choose the correct one.
Thanks
Thank you for this, VERY interesting, great job,
Yes, it IS a simulation with a ton of variables in a complex environment.
Sometimes, expectations can be TOO high.
Good read. One thing that ATC probably is not checking, is weather.
I’ve had ATC ask me to climb, probably to avoid low terrain, but I was already cleared, and it directed me into a rain cloud. Had I flown lower, which would have been a bit unsafe, but definitely safe enough, I would’ve flown much safer because I could actually see where I was going, rather than flying blindly on ATC instructions.
The instructions during that flight were all spot on though, and I cleared the clouds well before the landing site was in visual range.