For my final post on airport authoring, a few comments on the sharing process and moderation process. This is where we’ve gotten the most questions about procedure, etc.
A New Website
Traditionally, airports have gone to Robin Peel – he maintains a private SQL database where submissions are tracked.
We have a developer working now on a new portal website for the global airports – the website will support upload of new airports, tracking of airports by administrators, and downloading of pre-release airports before they are brought into X-Plane.
The site provides back-end tracking for us so that we can see what has been submitted and changed, etc.
To upload data on the site you’ll need a free account – this way we will have contact information for anyone sharing airports. (Very often we want to email the author and say “could you please fix the one PAPI that is facing in the wrong direction.”)
The Ratchet
The big question is: what about quality? How will we fix problems in the shared airports, and what will the quality bar be?
My current thinking is: quality should be like a ratchet: no user submission should ever make things worse than they were before. Over time, this will allow us to continually improve the airports.
This means a few things:
If you upload a newer version of an airport, and your work is worse than what was there before in some way, your upload will probably not be used as the new official airport version. For example, if you add buildings but accidentally delete a runway, the upload is worse and won’t be used.
This also means that you have to include every type of data present before-hand. If the previous airport has buildings, you need to include these buildings in your version even if you are only editing ATC data. You can’t upload just one kind of data.*
If you upload new data (E.g. new buildings) they don’t have to be perfect. We should not reject uploads because they aren’t as perfect as possible – over time we’ll ratchet up quality. Let’s walk before we run.
On the other hand, if your upload just looks totally broken, expect it to not be used. If there are no buildings at the airport and you put a hangar on the runway itself, we can’t use it.
One reason that we want you to include all data in every upload is that it’s important to check that the data is all synchronized. If users submit apt.dat and ATC data separately, the taxi routes could be misaligned from the pavement and the authors would not see this. By having everything in your package you can see that alignment is correct between all data sources before uploading.
Moderators and Collaboration
The system is being designed to support multiple moderators; one thing that seems clear so far is that the work of keeping an eye on this data has gone way up now that we have buildings too.
If there is a conflict between two legitimate layouts, both very good but different, we have the emails of the authors – we can email both and say “you guys work out a compromise for the location”.
One final thought: I have seen a lot of postings on forums, email to me, and blog comments, all expressing concern about what to do about bad data and how to stomp it out.
I think it’s important to take a step back and not get too carried away here. The goal of the global airports is to share data, with the moderators spotting bugs. No author that I have spoken to has ever said “I really want to post bad data to your database!” The moderators will be more like editors of a book than police catching criminals.
I bring this up because I have participated in other crowd-sourced projects, some that presume the authors are innocent until proven guilty (e.g. they assume authors know what they’re doing) and some that are guilty until proven innocent (e.g. no contribution goes up without a super-thorough review).
Invariably, the more relaxed projects end up with significantly more contributions, and in the long term end up with higher quality data, driven by a larger and more motivated authoring community. By comparison, the projects that put a huge emphasis on stopping contributors from erring as a primary goal end up deterring user contributions, and end up with worse data as a whole due to a lack of man-power.
One thing Robin has done right for quite a long time with X-Plane’s airport (which might not be obvious – sometimes when you do soemthing correctly its invisible) is to not alienate contributors who have had data quality issues. I want to make sure that we preserve a positive attitude toward contributors as we grow the global airport process.
Update: this driver bug has since been fixed – see here.
If your X-Plane screen looks like this with HDR on, you may be running the new Catalyst 13-12 drivers. If you can, back up to the Catalyst 13-9 drivers to get back HDR mode.
(If you have a really new AMD card, you might not be able to run 13-9 – in that case please turn off HDR until AMD has a fix.)
This is a continuation of this problem – what’s new is that the 13-12 drivers are WHQL drivers and not just beta drivers.
Thanks to the users who reported this to me. I try to keep an eye on all driver combinations, but the help is appreciated!
Since X-Plane 8.50, airports have had a border polygon that defines the boundary of the airport surface area. The airport boundary polygon is probably the least understood aspect of airports, particularly because it takes effect when we cut DSFs, not when you load X-Plane.
The airport boundary does a few things:
If sloped runways are not enabled, it defines (roughly) the area that X-Plane will flatten to the airport elevation. If you have ever entered a silly field elevation in WED and then turned off sloped runways, you’ll see that the flattening process is not exact.
When we create base mesh DSFs, the area of the boundary polygon gets a grass surface texture.
When we create base mesh DSFs, the area of the boundary polygon has no autogen buildings. (These two points are actually one in the same – the zoning of ‘airport’ in the DSF creation tool adds grass and removes autogen at the same time.)
When we create base mesh DSFs, the elevation under the boundary polygon has large bumps and spikes removed.
No More Bezier Curves
If you’ve used WED 1.2, you may have noticed that it refuses to validate airports with bezier curve boundaries. This is because we are trying to stop the use of bezier curves for airport boundaries.
The problem with bezier curves is that they can very easily be self-intersecting, but WED has no good test for this. The result is (apparently) valid WED airport layouts that later crash the DSF generator. Every time we cut DSFs, we find a handful of boundary polygons that need to be hand-fixed due to bad bezier outlines. There were 3 in this last batch.
Furthermore, the DSF generator cannot actually use bezier curves – it has to convert them to polygons. So the author will not get a surface area like the one they expected – there is a conversion that must take place.
Faced with a bezier curve technology that produces crashes and isn’t actually used, I decided to switch to straight polygonal airport boundaries. So: when you work on new airports, please remove bezier curves from your boundary polygons.
One of the top 5 bug reports I see: two AI airplanes starting in the same ramp start position, or worse, an AI airplane starting on top of the user’s plane. The pictures are often quite spectacular, if not what we intended for the ATC system.
A crowded start.
The 7 engine Boing-Douglas DC-10-747 needs an escort helicopter.
A tri-tailed airliner?
The code that allocates AI start locations is much improved for X-Plane 10.30, so in the next major update, we should get less cluttered ramp starts.
But the code change isn’t enough; there’s something that you have to do now if you are working on an airports.
You need to provide more than one ramp start! In particular, if there aren’t enough ramp starts for a given airport, X-Plane cannot sanely allocate AI airplanes with space between them.
In the past, X-Plane’s UI didn’t have room to show a lot of ramp starts, so authors would include only a handful. This has been fixed for a while, so I encourage you to add lots of ramp starts to your airports. This will give the AI planes room to park and the user a choice of all of the real-world gates.
Types of Ramp Starts
X-Plane 10 introduced type and equipment codes to ramp starts. One thing Chris found while working on the ATC system was that users were putting ramp starts in silly locations. For example, some airports have ramp starts on the active runway! He had to write specific ATC code to untangle the disaster that happens when an airliner decides that “on the runway” is a good place to park.
X-Plane 10 lets you define your ramp start as a tie-down, a gate, the interior of a hanger, or ‘misc’. The misc category was designed for ramp starts that might be interesting to humans (e.g. “in the run-up area for runway 7”) that are not appropriate for AI/ATC use.
I am not a fan of ramp starts near the active runway; I figure if a user wants to minimize taxi time, X-Plane lets you start on the runway, and a start anywhere but the ramp is inappropriate for realistic flight. But whatever you make, consider marking real parking spots as gates, tie downs, etc. and the random starts as “misc”.
Static Aircraft
Static aircraft (3-d models of aircraft that don’t move) are a nice way to make an airport look busier with less impact on framerate than AI planes. But this presents a problem: if a static airplane is at gate C4 and a user wants to park there, what do we do?
I believe the solution to this is to automatically place static aircraft based on ramp starts. This is not something we have coded now, and I don’t know when we will get to it, but my design idea for static aircraft is to have the airport include a lot of start positions and no static aircraft; X-Plane can then place static aircraft at ramp starts based on a rendering setting, with the user picking more static aircraft if hardware permits.
The advantage of this scheme is that (because X-Plane places the static aircraft), X-Plane would know where they were parked and could make sure the static aircraft aren’t in the user’s spot and the AI aircraft aren’t using a spot taken by the static aircraft.
The todo item for authors to someday support sim-placed static aircraft is, fortunately, the same thing authors need to do now to prevent future AI collisions: just make sure that there are a lot of ramp starts, modeling how the real airport works.
Edit: Just to clarify, 10.30 is not scheduled to contain a big pile of ATC bug fixes. The slight improvement in AI ramp start placement that I put in was a one-off bug fix; more ATC bug fixes will come in a later patch.
Now that we have user-contributed airports with buildings in X-Plane 10.25, I have a few topics to cover regarding airport authoring. The first is taxi routes. (I will get to the contribution and sharing process in a few posts – please bear with me.)
The short of it is: if you do not provide taxi routing in an airport layout, X-Plane will generate a taxi route, and the route X-Plane generates is not particularly good. The algorithm is slow, so it hurts loading time, it produces a relatively ugly layout, often with errors, so the AI behavior is bad, and I suspect that sometimes the whole algorithm crashes.
While all of these things can be improved, there will always be a trade-off between load time and algorithm quality. For example, the layouts could be rendered using more memory, but the loading would be even slower.
And we don’t want to be generating AI layouts – we want this data to be in the apt.dat where it can be built by smart humans who can carefully author.
For example, consider an airport with a huge tarmac, where the taxi routes are simply painted onto a big slab of concrete. The AI algorithm doesn’t understand this line structure at all – it isn’t smart enough to ‘get’ the taxi instructions. So the AI network lets the plane drive straight across the slab of concrete, traffic flow be damned! The AI also has no idea what any taxiway is named, so ATC can’t provide good route names.
JFK auto-generated layout – note how irregular!
At T junctions, the aircraft ‘dip in’ to the T.
Large paved areas get near-random traversing route structures.
The pictures above show the actual generated taxi layout at KJFK – red indicates active segments where the aircraft hold short before crossing runways. Nerds will recognize that this layout is created by a straight skeleton erosion algorithm, a strategy for analyzing images that comes from OCR.
The moral of the story is: while the AI taxiing behavior can be quirky, a lot of the problems come from X-Plane trying to auto-generate layouts off of the apt.dat file without a real taxi route structure. If you create such routes, the sim loads faster, provides better directions, and provides more plausible taxi routes.
In a past life, before I worked for Laminar Research, I was a plugin developer. So I’ll admit up-front that I might be the most guilty individual when it comes to this particular problem: log spam.
What do I mean by log spam? By log spam I mean printing information to Log.txt that isn’t necessary.
Log.txt exists so that we (and third party developers) can look at a single machine-generated file and get a strong idea of what a user did with X-Plane for diagnostic purposes. Therefore log.txt needs to contain three kinds of information:
Basic configuration information. We print your graphics card, your OS, etc. (In 10.30 we’ll even get your CPU type right on OS X. 🙂 It’s more reliable to log this than to send a user into ten different control panels to gather information.
Very basic event logging, e.g. what did the user do? Where did he start his airplane, what scenery was loaded.
All abnormal conditions, including error alerts that came up.
In particular, what we do not need to do is log the detailed blow-by-blow progress of loading in excruciating detail.
If I may use my own sins as an example, older versions of XSquawkBox shipped with debug code on that dumped the contents of its entire ICAO dictionary – thousands of lines of text. This information isn’t configuration (it’s not unique to particular installs), it isn’t an event – it is part of the event of loading XSquawkBox, and its not an error condition. What a mess! (Fortunately Wade has fixed this and turned such logging off in the newest XSB betas.)
Programmers write this kind of logging code to help trace program execution, but it’s important to turn it off – if we don’t, the log files get so verbose that everyone spends a ton of time just fishing through ten tons of junk.
Some more guidelines:
Log the presence of files or sub-modules that are user-configurable, but do so briefly. It’s reasonable for SASL or Gizmo to list scripts that do not come with the base package, or for XSquawkBox or Pilots Edge to list CSLs found. (This information lets a developer see what a user has done to modify the plugin.)
Don’t log the presence of required files unless an error condition occurs. An empty log can simply mean “everything went okay.” (The sim will already confirm successful plugin loading on a per-plugin basis.)
Log rare events, e.g. a flight starting, a network connection being opened. Don’t log events that occur at high frequency, like receipt of a single network packet.
X-Plane is guilty of log-spam too; I’m slowly trying to cut down on this. For example, spurious autogen-tree warnings are now gone from the log in 10.25r1.
Support Debugging
Sometimes you need all of that logging. My suggestion is: if your add-on is complicated, make a specific interface to get verbose diagnostics that users can turn on as needed. For example, X-Plane has an option in “operations and warnings” to dump verbose play-by-play network data to Log.txt. When you enable this, you can get a ton of information about network operations, but the log will also be totally over-run. The option is meant only for developers who are integrating with X-Plane’s UDP networking, and thus it has to be turned on by hand.
The OBJ engine actually also has such a trick – if you put the single word DEBUG (all caps) at the end of your .obj file, the entire OBJ contents will be output in detail to Log.txt. Again, this is a ton of information, available when needed, but off by default.
You can support verbose logging in your add-on with a script variable, a dataref, a change to a preferences file. Pick an easy mechanism, so that you can have users turn on verbose logging only when they need it.
Not Everyone Has Grep
Finally, a quick aside to my fellow professional programmers. You might be rolling your eyes and going: “Ben you idiot, it’s easy to filter the log. Everyone should just prefix their log entry and use grep.”
If we were all programmers, I would agree 100%. But please understand that aircraft and scenery authors have to look through the log to get diagnostic information about art-asset problems, and this is an audience that is sometimes not comfortable with command-line tools, huge 25MB text files, massive amount of text, etc. So we are trying to keep the log.txt more human-readable than, for example, dmesg on Linux.
X-Aviation just posted an update to SkyMaxx Pro – the new 1.1 patch brings big performance improvements and fixes rendering problems with HDR. From the user reports I’ve read, performance in HDR mode is good. (I hate to see users have to pick between HDR and third party add-ons; we want HDR to be the basis for superior next-generation aircraft and scenery.)
Getting a plugin that draws in 3-d to work with HDR mode requires some caution; the plugin APIs for drawing were designed in X-Plane 6, when ‘deferred rendering’ didn’t even exist as a concept.
I have updated the X-Plane SDK tech-note on 3-d drawing to contain modern guidance on how to cope with 3-d drawing callbacks. In particular, you get called twice when your plugin requests to draw in 3-d:
The first callback is for drawing “solid” stuff that will be affected by spill lights; the only safe thing you can do from this callback is to call XPLMDrawObject.
The second callback is after lights have been mixed; at this point normal OpenGL drawing works reliably (albeit without spill lights being mixed in). This is the time to draw translucent prop discs, coach marks and labels, clouds, smoke and particle systems, etc.
If your plugin does any 3-d drawing (e.g. custom particle system drawing or any kind of effects code), please review the tech note, and email me if you have questions. The next-gen CSL code sample that is linked from the article is tested and works correctly too.
Update: this bug has been fixed – see here for details.
For X-Plane users with an AMD graphics* card on Windows or Linux: there appears to be a bug in the 13.11 drivers** that cause corruption in HDR mode; the symptoms are square artifacts of missing clouds and missing windshield in aircraft.
If you can run the 13.9 drivers, stick with those; if you have one of the new Rx 290 cards and you must use newer drivers, turn HDR off for now.
These bugs are Win/Lin only – the Mac AMD drivers don’t have this problem.
* AMD seems to have fully rebranded ATI’s GPUs at this point, so I guess I should get used to writing ‘AMD Radeon’.
** The bug could very well be in our code, but since the code worked on old ATI drivers and a number of other driver stacks, it may very well be a regression in the driver itself. We won’t know until it’s fixed. And even then, maybe we won’t know.
The HDR check-box turns on HDR mode in X-Plane 10. But what is HDR mode?
High Dynamic Range( HDR) mode in X-Plane actually enables many features at the same time. We put in a single check-box because the features all fit well together, and we wanted to keep the interface simple.
High-dynamic range rendering. In HDR mode, the world is rendered across a wider spectrum of brightness (just like real HDR photography)*, to better capture lighting levels. The result is then compressed to low dynamic range (which is what your monitor does).
Tone mapping with real-time exposure. The HDR image is converted to LDR via a tone mapping algorithm that attempts to preserve the character of the image while keeping the dynamic range within limits of your screen. The level of exposure changes in real-time: if you look at a bright scene and then duck the camera behind a building, you’ll see your eyes slowly adjust up, then everything gets dark when you stare at the sun. (The level-adjustment algorithm runs entirely on the GPU for performance.)
X-Plane uses deferred shading for all rendering except for translucent glass in aircraft. Deferred shading is a 2-pass approach to rendering that is commonly used in today’s shooter games, because it allows us to efficiently draw huge numbers of real 3-d spill lights. In the case of X-Plane, you get a lot more night lighting.
The individual puffs in our 3-d clouds have soften intersection with the terrain to avoid ugly cloud-ground collisions in low-cloud conditions. (See page 1 and 4 of the article for before/after pics.)
We emit heat blur from the engine particles. The ability to add blur effects is a capability of the deferred rendering engine; something I’d like to do more with in the future.
Optional atmosphericscattering. This effect enables a more advanced lighting and fog shader that runs on the GPU and models the diffusion of light through the atmosphere as it hits oxygen and water particles. Atmospheric scattering is what makes the far view ‘bluer’ than the near view.
So the moral of the story is: the HDR check-box actually enables a lot of effects!
Why Did You Put Everything in One Check-Box?
We had a few reasons to put a lot of effects into one bucket:
Some of those effects require other effects. For example, you can’t have the heat blur or soft cloud puffs without the deferred renderer.
We wanted to keep the UI simple. Our view is that the rendering settings screen is way too complicated and hard to tune, something we’re still looking to improve. So any time we add features, we have to ask: can we make this simpler?**
There are fewer combinations for us to test if there are fewer settings. Sometimes we can have bugs based on combinations of features, so having the features all enable at the same time simplifies development of testing, which is extra important when we have to test GPU features against different hardware and drivers already.
* GPU nerds: The final render surface is in 16-bit floating point format.
** If you really want to get tweaky, most of the sub-effects within HDR mode can be accessed by settings.txt. But if you edit settings.txt and your computer transforms into an alien cyborg and destroys humanity, don’t hold me responsible.
If you’ve survived downloading the rather large 10.25 patch and you want to download even more data, good news! Alpilotx has just released version 2 of his X-Plane HD Mesh Scenery.
His scenery pack is a donation-ware recut of North America and Europe. Alpilotx’s HD scenery use the latest OSM and other global data sources, and are cut with the latest DSF generator; thus they contain better DSF generation with newer data. Alpilotx then cranks the mesh settings well beyond what we can ship with the global scenery (which must fit on a DVD set and not fill an entire SSD), bringing out wonderful terrain definition and other detail that we have to cut down for the global scenery that ships as a default with the sim.
One of the cool effects of cranking up the base mesh detail: it brings new detail to the terrain shading. In X-Plane 10 we moved the process of altering terrain with slope (from forest to grass to rock, or whatever the artist defines) from a CPU process calculated ahead of time to a shader process run on the GPU. The result is that more detailed meshes actually produce better shading, as the shading responds to the 3-d detail.
If you have a computer that can run them, these meshes look great, and they get you a source of recut DSFs now.
The recut DSFs that Laminar ships will be based on the same DSF generation code improvements and the same new data, but the resolution will be reduced to produce files of similar size to the default scenery we already ship. We have not decided exactly how we will ship recuts; I think that really important fixes (e.g. Sydney with the harbor fixed) will be automatic updates; I don’t know about a wider scale yet.
