For about a year there has been a subtle bug in how X-Plane draws taxiways: if you build an S-curve shape out of a single bezier and it is almost perfectly symmetrical, X-Plane would go “nah, why bother” and replace it with a single straight line segment.*
So in 10.30 I fixed it, and the result was a bunch of broken airports with missing pavement. (YMML is high on this list!)
It turns out that these airports have authoring errors – typically a segment of pavement that should be straight instead being formed by two overlapping beziers. This is definitely wrong, but due to the bug, X-Plane would simplify the overlapping S curve into a single straight segment and the layout would work. Only now that X-Plane correctly renders the S curve does the taxiway fail (because taxiways may not have overlaps). So two wrongs may not make a right, but they do make a “hey, that looks okay, let’s ship it”.
Note that the overlaps depend on the rendering setting of X-Plane – a different S curve is formed at different rendering settings; the overlap that causes the taxiway to disappear may only appear at a particular rendering setting.
For 10.31 I am going to undo my bug fix. This doesn’t make me happy, but I think it is necessary:
We have no idea how many airports have their taxiways broken by this bug.
Authors have no easy way to detect this problem, other than re-testing every airport at every rendering setting.
Even if an airport looks okay at all rendering settings, future rendering settings may cause the problem.
This is too much uncertainty to solve ‘by hand’. So my plan is:
Undo the code change for 1031. YMML and friends comes back.
Develop validation code in WED to detect this kind of authoring error.
Ship that version of WED so new authoring work will be checked.
Run the WED code on all airports and make a list of ones that need repair.
Fix all of the known problems in the airport gateway.
Redo the code change so X-Plane is correct.
This isn’t going to be a quick process, but then it can’t be, because third parties have apt.dats shipping now that only work when X-Plane has the buggy taxiway code in place. So we need to ship WED and then give third parties enough time to go back and check their layouts and fix them if necessary.
I expect to get a 10.31 beta with the taxiway code changed back this week.
For WED validation, I have some test code to detect errors but it isn’t ready yet. The problem is that it’s not good enough to detect errors with overlapping beziers**; we have to consider two bezier curves near enough to each other that with the error in rendering introduced by WED’s rendering settings, we get an overlap. (So authors, better be safe than sorry in creating your pavement.)
If there’s a moral to this story, I think it’s this: when we (LR) don’t provide good tools for authors to validate that their work is correct, the resulting body of work will end up with subtle errors.
* X-Plane renders beziers by measuring how far the mid-point of the curve is from the average of the ends. As long as this distance is ‘too far’ and the iteration count isn’t too high, X-Plane divides the bezier in half and repeats the process. In this way beziers are broken into enough line segments to approximate the bezier within a minimum error limit. The rendering settings control the error limit.
The bug: if the curve was a ‘balanced’ S curve the mid point of the curve was the average of the end points and X-Plane went “great, no error” and stopped dividing.
** Which is already not an easy problem – the analytical solution for bezier intersection is a 9th order polynomial!
August 28, 2014, Columbia, South Carolina. Laminar Research is pleased to announce the release of its Airport Scenery Gateway, as well as an updated and improved version of its airport scenery tool, World Editor 1.3. The Airport Scenery Gateway can be accessed on the internet at http://gateway.x-plane.com and the new and improved version of World Editor is available for download at http://developer.x-plane.com under Scenery Tools.
The Airport Scenery Gateway was created to facilitate the populating of X-Plane with 3D airport sceneries that are consistent in quality and appearance, error-checked, and will work with a standard installation of X-Plane 10. During the past several months Laminar Research has received hundreds of airports designed by World Editor users. World Editor is an airport scenery creation and editing tool for Laminar’s X-Plane 10 that allows the typical X-Plane user to create airport scenery.
By establishing an internet gateway for scenery submission, it is now possible with World Editor 1.3 for users to automatically submit airport scenery contributions directly to the Airport Scenery Gateway. X-Plane users can visit the Airport Scenery Gateway to search for airport scenery submissions. The scenery can then be immediately downloaded into X-Plane 10 before it appears in a scheduled X-Plane update.
If your airport is not already in X-Plane 10.30, please use the Gateway to share it via WED 1.3; do not send it directly to Robin. At this point Robin should only be getting navaid patches. You must use WED 1.3 to upload to the Gateway!
The airports that are already in 10.30 are uploaded by ‘WEDBot’, which is an account we used to transfer our existing collection into the Gateway. As users share data, we will start to get good attribution; I think this is important because we will know who to contact to sort out issues with airports.
If you could not upload an airport due to a missing ICAO, we are working on this now.
We are working on a Linux build of WED 1.3 and will post it as soon as we resolve the chaos of shipping a working app on multiple distributions. mroe has done great work to make Linux happen.
We will periodically ‘roll’ shared airports into X-Plane via point releases to the sim. I do not recommend collecting a huge pile of Gateway custom scenery packs; they will simply conflict with newer data from X-Plane when it comes out. The ability to download a pack directly from the Gateway is meant for authors who want to view the state of a pack and possibly modify it. The gateway is not meant to be a scenery file sharing site.
Finally, I think this has been discussed before, but…custom scenery authors:
Always put exclusion zones around your custom airports! Even if there are no 3-d buildings at an airport today, they may appear in the next update.
With the Gateway open, 3-d is going to appear quickly!
X-Plane 10.30 beta 8 is mirroring to our servers now; this beta contains a pile of recut DSFs, fixing some of the worst DSF bugs. I will post a complete list of recut DSFs in the release notes; what follows are just a few hilights from really prominent DSFs.
I’m not sure how long the mirror will take to finish; I’m hoping we’ll go live with the beta tonight or tomorrow.
The X-Plane updater will download scenery updates if (and only if) you have the base scenery installed. So if you have the entire world installed, you’ll get the entire set of updates (the DSFs themselves are about 500 MB); if you only have the US, for example, you’ll get some tiles but not others, and if you just have a demo, you’ll just a core file update.
If you install scenery later, simply run the updater again (the install-scenery function gives you a chance to run an update at the end of DVD scenery install) to get any fixed DSFs you might not have gotten at the time.
(Alpilotx had a good suggestion for the installer that I’d like to do in the future: an option to skip scenery updates. This would let users get a sim update immediately, and then fetch the scenery later if bandwidth is tight.)
Robin, Tyler and I have been going through the airports that have been sent in since X-Plane 10.25; the good news is that there are over 350 batches sent in!*
But – I had to throw thirteen of them overboard; they were using OpenSceneryX. For airports that are going to be sent to the gateway and become part of the default airport file for X-Plane, you can’t use third party libraries! Everything you use has to ship with X-Plane.
WED 1.3 has user interface features to help hide third party libraries, but in this case you’d think that having the objects in the opensceneryx/ folder would have been a hint.
It’s not a coincidence that this was the same thirteen entries not sent in the right format; WED 1.2.1 won’t do an export for the global database with third party libraries, and this will be true with WED 1.3 as well, where the export is uploaded directly to the gateway.
I have been working on visibility and fog-related bugs in X-Plane 10.30; two fixes will improve high-altitude long-visibility viewing:
10.25 has a bug where the fog color changes abruptly at the cut between DSF and planet rendering, particularly when atmospheric scattering is on. This is fixed in 10.30
The 64-bit version of X-Plane 10.30 will load 12 DSFs instead of 6, for longer-range drawing of detailed DSF terrain.
These comparison shots show 35k above KSEA looking at Mount Rainier with max visibility. (Note that at 35k feet, it doesn’t make a huge difference what ground visibility you pick – the planet-wide upper atmosphere model is almost entirely in control.)
These pictures are taken at “very high res” – the most my Mac Pro can survive with an old GPU; you might get a little bit better definition on “extreme” res.
A few notes on loading more DSFs:
My current plan is that the 64-bit version will always load the 4×3 DSF box – if we have to make this a setting for compatibility we will, but I think it’s preferable to not add more rendering settings.
The 32-bit build will stay with a lower DSF count for memory reasons; generally speaking, I don’t think we will have any additional visibility improvements in the 32-bit build since we are almost always going to hit a memory wall. If you have a 32-bit OS, consider upgrading to 64-bits! (In particular, if you are running Windows XP, please upgrade to Windows 7 or 8 64-bit – there are no more service packs for Windows XP, so you’re just asking to get malware!)
In 10.30 the DSF loader can load up to 4 DSfs at a time (for 4×3) or 2 at a time (for 3×2). So if you have a multi-core machine, the load time should be better even though there are more DSFs being loaded, thanks to multi-core. (The limiting factor here is that adjacent DSFs can’t be loaded at the same time because their edges need to be matched.)
I am still hoping to address other low-visibility fog issues, but that code is not complete yet.
The Planet Could Look Better
X-Plane renders nearby terrain using DSFs, but it renders the very far terrain and entire planet using a single “planet” model, which is a textured sphere displaced by a normal/height map. As of X-Plane 10.25 (and 10.30) we are not including as much detail in the 3-d planet mesh as the data on disk contains.
This pair of pictures is 45k above LOWI; the second texture has the mesh spacing on the planet artificially increased from 3 km to 2 km. (The data on disk is at 1 km spacing, but my machine can’t cope with that.)
You can see we pick up a little bit of definition in the far mountains. In the long term, I think we could ship a 500 meter planet mesh, which would make the far view in X-Plane as good as the close view used to be in X-Plane 7.
I’m not sure when we’ll ship a higher density far-planet, but I think it will have to be post-10.30.
If 1 km or 500m seems like bad resolution, do consider how far away the planet mesh is. With a 4×3 DSF box, the planet starts 100 km from the viewer. At 90 degrees FOV (an extreme case, but it makes the math easier) the screen is 200 km across at the DSF cut-over. With a 2 km planet grid, that means we will show 100 planet vertices left-to-right. At 1080p the planet triangles are thus at 20 pixel increments – not bad for a low res mesh.
Last week I spent a little bit of time looking at a bug: some ILS approach paths have tall buildings in the way. For an example, fly the ILS 22L into KBOS, and look to your right on short final. I hope the people living in those apartment buildings like watching airplanes!
A Library Bug
It looks like the bug is caused by a bad mapping of art assets in the library. The global scenery DSFs contain virtual paths for art assets, which are then mapped to real autogen art assets by the library. The virtual paths indicate the legal height ranges that the autogen can extend to, e.g. the library path
Indicates that this art asset fills a square block at least 30m deep with roads on all four sides, and the buildings should be between 32 and 40m tall for the tallest ones included. (The actual encoding of the library is partly handled by scripts we use to build the global scenery, and the encoding is byzantine enough that neither Alex nor I understand the entire thing.)
Well, if the virtual library path calls for a low building but the physical art asset contains tall buildings, you get skyscrapers in places you did not want them. And that is exactly the bug causing the ILS incursions.
I have an edit to the library paths for our next release (probably 10.30) that should fix this problem. The good news about this being a bug in the library (and not the DSFs) is that a single library fix will fix the problem everywhere.
For the Brave
(This is a hack to the sim…don’t try this if you aren’t willing to break your copy of X-Plane. If you try this and your sim dies, please don’t contact me or tech support!)
If you want to patch this bug yourself now in 10.25, simply locate the file
and replace it with this file: library.txt. Warning: because this is a mod to the sim itself, when you go to update, you will get a warning as to whether you want to overwrite the file. When the next beta or release comes out, do overwrite this file!
What About the Raw Data
I also wrote some code to attempt to ‘protect’ the ILS approach path in our DSF generator. Here’s a picture:
What you see here is a color-coded trace of height limits for autogen. Red are the strongest height restrictions, yellow less so. The height restrictions take into account not only the glide slope, but also the terrain; in the case of KSAN, the airport is at the bottom of a hill next to a mesa, so the ‘red’ (low flying planes) zone is quite large – even when a plane is on a 2 mile final, the plane is low to the ground because the ground is higher up.
The logic of the code is to let all ‘known’ tall buildings exist, but limit any generated tall buildings to follow the height restriction.
Here’s the thing: having coded it, I have not caught any cases where the autogen buildings were violating the height restrictions!
Buildings that we generate (without obstacle data) always have a maximum height that is a fraction of the FAA-spec’d tallest building. The idea is that if the FAA says there is a single 80m (roughly 20 story) building, then the nearby buildings are probably somewhat tall, but they’re not also 80m – if they were, the FAA data would include them. So we might generate some 40m buildings near the 80m building to create a plausible ‘neighborhood’.
From what I can tell, the fact that the generated buildings are so much lower than the FAA datapoints keeps them out of the approach path, and the actual cause of tall buildings has been the library path misalignment. (For example, in the KBOS 22L case, the block has an 8m height restriction and yet the buildings are either 50m or 70m tall, I can’t remember which.)
I think I am going to keep the ILS-protecting DSF generation code around – it’s useful to have approach paths noted, and it’s something we’ve been intending to do for quite a while. But the library fix is what will make a real difference.
For my final post on airport authoring, a few comments on the sharing process and moderation process. This is where we’ve gotten the most questions about procedure, etc.
A New Website
Traditionally, airports have gone to Robin Peel – he maintains a private SQL database where submissions are tracked.
We have a developer working now on a new portal website for the global airports – the website will support upload of new airports, tracking of airports by administrators, and downloading of pre-release airports before they are brought into X-Plane.
The site provides back-end tracking for us so that we can see what has been submitted and changed, etc.
To upload data on the site you’ll need a free account – this way we will have contact information for anyone sharing airports. (Very often we want to email the author and say “could you please fix the one PAPI that is facing in the wrong direction.”)
The big question is: what about quality? How will we fix problems in the shared airports, and what will the quality bar be?
My current thinking is: quality should be like a ratchet: no user submission should ever make things worse than they were before. Over time, this will allow us to continually improve the airports.
This means a few things:
If you upload a newer version of an airport, and your work is worse than what was there before in some way, your upload will probably not be used as the new official airport version. For example, if you add buildings but accidentally delete a runway, the upload is worse and won’t be used.
This also means that you have to include every type of data present before-hand. If the previous airport has buildings, you need to include these buildings in your version even if you are only editing ATC data. You can’t upload just one kind of data.*
If you upload new data (E.g. new buildings) they don’t have to be perfect. We should not reject uploads because they aren’t as perfect as possible – over time we’ll ratchet up quality. Let’s walk before we run.
On the other hand, if your upload just looks totally broken, expect it to not be used. If there are no buildings at the airport and you put a hangar on the runway itself, we can’t use it.
One reason that we want you to include all data in every upload is that it’s important to check that the data is all synchronized. If users submit apt.dat and ATC data separately, the taxi routes could be misaligned from the pavement and the authors would not see this. By having everything in your package you can see that alignment is correct between all data sources before uploading.
Moderators and Collaboration
The system is being designed to support multiple moderators; one thing that seems clear so far is that the work of keeping an eye on this data has gone way up now that we have buildings too.
If there is a conflict between two legitimate layouts, both very good but different, we have the emails of the authors – we can email both and say “you guys work out a compromise for the location”.
One final thought: I have seen a lot of postings on forums, email to me, and blog comments, all expressing concern about what to do about bad data and how to stomp it out.
I think it’s important to take a step back and not get too carried away here. The goal of the global airports is to share data, with the moderators spotting bugs. No author that I have spoken to has ever said “I really want to post bad data to your database!” The moderators will be more like editors of a book than police catching criminals.
I bring this up because I have participated in other crowd-sourced projects, some that presume the authors are innocent until proven guilty (e.g. they assume authors know what they’re doing) and some that are guilty until proven innocent (e.g. no contribution goes up without a super-thorough review).
Invariably, the more relaxed projects end up with significantly more contributions, and in the long term end up with higher quality data, driven by a larger and more motivated authoring community. By comparison, the projects that put a huge emphasis on stopping contributors from erring as a primary goal end up deterring user contributions, and end up with worse data as a whole due to a lack of man-power.
One thing Robin has done right for quite a long time with X-Plane’s airport (which might not be obvious – sometimes when you do soemthing correctly its invisible) is to not alienate contributors who have had data quality issues. I want to make sure that we preserve a positive attitude toward contributors as we grow the global airport process.
Since X-Plane 8.50, airports have had a border polygon that defines the boundary of the airport surface area. The airport boundary polygon is probably the least understood aspect of airports, particularly because it takes effect when we cut DSFs, not when you load X-Plane.
The airport boundary does a few things:
If sloped runways are not enabled, it defines (roughly) the area that X-Plane will flatten to the airport elevation. If you have ever entered a silly field elevation in WED and then turned off sloped runways, you’ll see that the flattening process is not exact.
When we create base mesh DSFs, the area of the boundary polygon gets a grass surface texture.
When we create base mesh DSFs, the area of the boundary polygon has no autogen buildings. (These two points are actually one in the same – the zoning of ‘airport’ in the DSF creation tool adds grass and removes autogen at the same time.)
When we create base mesh DSFs, the elevation under the boundary polygon has large bumps and spikes removed.
No More Bezier Curves
If you’ve used WED 1.2, you may have noticed that it refuses to validate airports with bezier curve boundaries. This is because we are trying to stop the use of bezier curves for airport boundaries.
The problem with bezier curves is that they can very easily be self-intersecting, but WED has no good test for this. The result is (apparently) valid WED airport layouts that later crash the DSF generator. Every time we cut DSFs, we find a handful of boundary polygons that need to be hand-fixed due to bad bezier outlines. There were 3 in this last batch.
Furthermore, the DSF generator cannot actually use bezier curves – it has to convert them to polygons. So the author will not get a surface area like the one they expected – there is a conversion that must take place.
Faced with a bezier curve technology that produces crashes and isn’t actually used, I decided to switch to straight polygonal airport boundaries. So: when you work on new airports, please remove bezier curves from your boundary polygons.
One of the top 5 bug reports I see: two AI airplanes starting in the same ramp start position, or worse, an AI airplane starting on top of the user’s plane. The pictures are often quite spectacular, if not what we intended for the ATC system.
The code that allocates AI start locations is much improved for X-Plane 10.30, so in the next major update, we should get less cluttered ramp starts.
But the code change isn’t enough; there’s something that you have to do now if you are working on an airports.
You need to provide more than one ramp start! In particular, if there aren’t enough ramp starts for a given airport, X-Plane cannot sanely allocate AI airplanes with space between them.
In the past, X-Plane’s UI didn’t have room to show a lot of ramp starts, so authors would include only a handful. This has been fixed for a while, so I encourage you to add lots of ramp starts to your airports. This will give the AI planes room to park and the user a choice of all of the real-world gates.
Types of Ramp Starts
X-Plane 10 introduced type and equipment codes to ramp starts. One thing Chris found while working on the ATC system was that users were putting ramp starts in silly locations. For example, some airports have ramp starts on the active runway! He had to write specific ATC code to untangle the disaster that happens when an airliner decides that “on the runway” is a good place to park.
X-Plane 10 lets you define your ramp start as a tie-down, a gate, the interior of a hanger, or ‘misc’. The misc category was designed for ramp starts that might be interesting to humans (e.g. “in the run-up area for runway 7”) that are not appropriate for AI/ATC use.
I am not a fan of ramp starts near the active runway; I figure if a user wants to minimize taxi time, X-Plane lets you start on the runway, and a start anywhere but the ramp is inappropriate for realistic flight. But whatever you make, consider marking real parking spots as gates, tie downs, etc. and the random starts as “misc”.
Static aircraft (3-d models of aircraft that don’t move) are a nice way to make an airport look busier with less impact on framerate than AI planes. But this presents a problem: if a static airplane is at gate C4 and a user wants to park there, what do we do?
I believe the solution to this is to automatically place static aircraft based on ramp starts. This is not something we have coded now, and I don’t know when we will get to it, but my design idea for static aircraft is to have the airport include a lot of start positions and no static aircraft; X-Plane can then place static aircraft at ramp starts based on a rendering setting, with the user picking more static aircraft if hardware permits.
The advantage of this scheme is that (because X-Plane places the static aircraft), X-Plane would know where they were parked and could make sure the static aircraft aren’t in the user’s spot and the AI aircraft aren’t using a spot taken by the static aircraft.
The todo item for authors to someday support sim-placed static aircraft is, fortunately, the same thing authors need to do now to prevent future AI collisions: just make sure that there are a lot of ramp starts, modeling how the real airport works.
Edit: Just to clarify, 10.30 is not scheduled to contain a big pile of ATC bug fixes. The slight improvement in AI ramp start placement that I put in was a one-off bug fix; more ATC bug fixes will come in a later patch.
Now that we have user-contributed airports with buildings in X-Plane 10.25, I have a few topics to cover regarding airport authoring. The first is taxi routes. (I will get to the contribution and sharing process in a few posts – please bear with me.)
The short of it is: if you do not provide taxi routing in an airport layout, X-Plane will generate a taxi route, and the route X-Plane generates is not particularly good. The algorithm is slow, so it hurts loading time, it produces a relatively ugly layout, often with errors, so the AI behavior is bad, and I suspect that sometimes the whole algorithm crashes.
While all of these things can be improved, there will always be a trade-off between load time and algorithm quality. For example, the layouts could be rendered using more memory, but the loading would be even slower.
And we don’t want to be generating AI layouts – we want this data to be in the apt.dat where it can be built by smart humans who can carefully author.
For example, consider an airport with a huge tarmac, where the taxi routes are simply painted onto a big slab of concrete. The AI algorithm doesn’t understand this line structure at all – it isn’t smart enough to ‘get’ the taxi instructions. So the AI network lets the plane drive straight across the slab of concrete, traffic flow be damned! The AI also has no idea what any taxiway is named, so ATC can’t provide good route names.
The pictures above show the actual generated taxi layout at KJFK – red indicates active segments where the aircraft hold short before crossing runways. Nerds will recognize that this layout is created by a straight skeleton erosion algorithm, a strategy for analyzing images that comes from OCR.
The moral of the story is: while the AI taxiing behavior can be quirky, a lot of the problems come from X-Plane trying to auto-generate layouts off of the apt.dat file without a real taxi route structure. If you create such routes, the sim loads faster, provides better directions, and provides more plausible taxi routes.