I’ve been working on a conformance test for X-Plane. The idea is simple, and not at all mine: X-Plane 945 can output a series of test images that are the same on each run. The images cover a variety of rendering conditions. If a video driver is broken, the images will be corrupted.
You can learn more about how this works here: I am working on the 945 timedemo tarball now.
The main driver for this is to help NVidia, ATI, and Apple to integrate X-Plane into their dedicated testing. With X-Plane as part of their test systems, they can catch driver bugs the easy way – the day after the code is changed, rather than months later after a series of angry web posts. X-plane 945 includes a number of new features as part of its framerate test to help with this process.
My hope is that this will benefit users (who will see less bugs) and the driver writers (who can get feedback on code changes in a uniform and reproducible manner). Here are the eight images in the sample conformance test I wrote, based on the LOWI custom airport scenery.
X-Plane 950 beta 1 should fix the stuttering present in 940. The bug was: the more you flew in one area, the worse the stutters would get – they’d clear out if you kept flying forward enough to scroll DSF tiles normally. Anyway, the bug was a book-keeping error based on the new “paged” airports from 940.* Fixed now – thanks to the users who got me a really clean test case to see this.
(Hint for users: if you can duplicate a scenery-related bug by saving a replay, or even if the bug can be reproduced in replay, that’s really, really useful to know! It makes the bug much more repeatable, since what scenery is loaded varies a lot with flight path.)
* Before 940 we would load the entire apt.dat file into memory on startup. With Robin receiving new highly detailed airport layouts every day, this was starting to use more and more memory. 940 and 950 will load a litttle bit of information about every airport and take note of the file that contains the layout – then when the sim needs a layout, it goes back to disk to pull up just the one airport in question, saving memory. We “page” the airports into memory only as needed.
My blog postings have been a little thin – but not because nothing is going on. Rather it’s a combination of working on next-gen stuff that is still in heavy development, travel, and (perhaps due to the travel, perhaps due to lack of sleep, or perhaps due to certain forms of Brandy brought on site by a certain British developer who won’t be named, but whose handle on the org starts with a P and rhymes with “Cropsman”, cough, cough) a minor fever that I’ve just about recovered from.
A user asked me about the design of the X-Plane updater – so this post is only going to be of interest to the few authors out there who are creating installers for their add-ons and need to update. We try to keep the file formats for X-Plane simple, e.g. “this one folder is your airplane”, so that if you don’t make an installer, the user won’t be overwhelmed with a complex unpack-and-install routine. But if you do need to install or update, well, here’s what we were thinking when we developed our installer.
We had a few needs for our installer:
- It had to be cross-platform to Mac, Windows and Linux.
- It had to update existing products as well as install new ones.
- The updates had to be minimal deltas, e.g. a demo can be 600 MB but an update should only be the 30 MB of files that really changed.
- We wanted the updater to do delta updates from any old version. (Some update systems require installing a number of patches in sequence. We have users who buy the DVD and then update a year later; we wanted to let them update in one shot.)
- Building the patches had to be really, really fast. We use our updater to publish our betas. So while X-Plane probably has 2-3 major patches in a single year and maybe 5 or 6 bug fix patches, we will (due to betas) cut perhaps 100 or more actual “updates”.
- On that last point, cutting separate updates by platform was a deal breaker, as it would triple the number of updates we had to cut. Having a demo install and an update be the same on the server was a big win.
Some of those needs are pretty specific to Laminar Research, but some might apply to a third party. There are commercial installers that will let you generate patches. If you don’t generate a lot of patches, you can probably use existing tools – basically you’ll spend a lot less time up front (since the tool set exists) but you might spend more time in the long run cutting separate patches by OS.
One of the first things we decided was to not use server-side technology or special protocol. That is, we don’t need any kind of smart servers to run the updater – the updates are just files hosted over HTTP on standard rented or owned apache servers. All of the work is done in the client. We did this for a few reasons:
- It lets us throw up the install anywhere – we don’t have complex needs for what’s on the server. Virtually any server will do. (As the company has grown, our server needs have grown too, so this is less of a concern now, but back then we had fewer servers in service, and were buying less overall server capacity.) If a server goes down, we can press another one into service about as quickly as we can move the file set to the server.
- We’re not server programmers. Coding the installer/updater on the client side let us leverage existing company technology, etc.
Here’s what we came up with. The basic idea of the installer is very simple. Note that since the installer uses HTTP files, you can “watch” the installer simply by downloading the files that it downloads.
- First the installer goes to a master server and gets a “version list”, which tells it what it’s going to actually get and what (multiple) mirrors are available. This one master file is the only file that must be hosted at X-Plane.com, and it allows us to change a very small file to press mirrors into service.
- Installs and updates are actually the same to us – it’s a set of files that the user should have once the install or update finishes. This starts with a file directory that lists every file (by path) that needs to be downloaded, as well as its MD5 signature.
- The directory also contains the MD5 of every old version of any file in the version, and lists files that have to be deleted to update the version (e.g. the file panel.png used to have MD5 signature 2934b..23abc2 but is now removed).
- The client, once armed with the file directory, can now download all files in the directory (install) or compare the existing files on disk to the directory by MD5 and only download changes.
- Individual files are on the server in zip form for download and decompression.
That’s pretty much it – you could write an installer in a scripting language using a tool like CURL or WGET – it wouldn’t be very complex because the installer is really just a clever, painted download+unzip tool.
Dealing With Modified Files
There is a bit of complexity in this design that you might not need for a third party installer: handling modified files. Note that the directory contains the MD5 not only of the current version of the file (to detect when no update is needed and save bandwidth) but also to see when the user has modified a file that is “managed” by X-Plane.
Before the installer overwrites/deletes a file that you own, it compares the file’s MD5 to the entire known list of MD5s for the entire version. If this file doesn’t match a known old version, it puts up the warning dialog box that you have modified a file that is about to be overwritten.
While we recommend that add-ons use scenery packs and other “safe” ways to customize the sim, this helps detect when a user has gone in and edited the cloud textures in Photoshop, and prevents us from overwriting them without warning.
Some of the newer text file formats for art assets (.lin files, roads, etc.) require you to specify how wide the texture is.
Huh? Why does X-Plane need me to tell it how wide a texture is? Doesn’t it know?
Well, first, the answer is: no! When X-Plane builds the roads and draped lines, the texture that is needed may not actually be loaded. It may only be on the todo list of textures that need loading, and if you have orthophoto scenery, that list could be very, very long. There is no guarantee that the texture is loaded before the geometry gets built. If you have a multicore machine, both might happen at the same time!
So the first reason why the road and .lin files require you to declare the size of your texture is: to allow X-Plane to build the mesh on a separate core from the one that loads the texture, for faster loading. Isn’t multi-core fun?
There’s another reason, and it’s a little bit more subtle: you can change the size of your X-Plane textures. Imagine that you get a new graphics card and you decide to “upgrade” your scenery pack. So you double the size of all of your textures, and then go in to photoshop and add more detail.
But if X-Plane were to look at the size of the texture, then all of your .lin files will be wrong! For example, let’s say you have a .lin file with a line from pixels 0 to 16 across the texture. When you double the size of your texture, you’d have to go in and change this to be 0 to 32 pixels across the texture.
But here’s the trick: since X-Plane gets the size of the texture from you and not from the actual texture, you can just leave your .lin file the way it was – with the old texture size and the old coordinates. X-Plane will do the math out and generate correct texture mappings even though the texture size is double.
In other words, the important thing about texture coordinates is that they are self-consistent, not that they reflect the actual texture size. This is because internally all texture mapping is done in ratios.
(So why not just use ratios in the file format? Try entering the size of 10 or 12 lines as ratios and tell me whether you like it. The ability to enter the line coordinates in pixels is to make it easier to set up the line files from the image file.)
In past blog posts I have tried to contrast two very different different approaches to simulation, which I would loosely call:
- “Predictive” (or reality-based). Input data comes from real world measurements, and the simulator predicts the expected behavior based on those inputs.
- “Prescriptive” (or specification based). The input data describes the desired output directly through preset formula.
In particular, when it comes to the flight model, X-Plane is predictive, via blade theory (X-Plane adds up all forces) while FS X is prescriptive (a table describes aircraft behavior).
But – not all of X-Plane is predictive. In particular, the “systems model”, which is a loose term for the modeling of electrical systems, hydraulic systems, and a number of other parts of the plane, is actually prescriptive in X-Plane, while the aerodynamics of the plane are predictive.
To illustrate: when your plane takes off, it leaves the ground because X-Plane has calculated and summed all forces and torques on the plane and has found that it has positive lift. You don’t program this in – you simply provide a number of parts that, under some conditions, influence the lift calculations. (Wings are good for this, btw! 🙂
But when you turn off the avionics master switch, X-Plane does not calculate the electron flow through the wiring harness to the glass PFD. Rather, the PFD is tagged with input data saying it requires the avionics master to be on. The PFD’s behavior with respect to the avionics switch is prescribed, not derived. (If we had a true “wiring editor” in Plane-Maker, it would be different – X-Plane would trace electrical routes and see if enough current makes it to the PFD.
In truth the distinction is a little bit fuzzier. For example, the gyroscopes actually do model the spin of the internal gyro (fail the vacuum system and watch them slowly pick up error like in a real plane) and the electrical system does internal current calculations. But in general the systems modeling is a bit of a mix, mostly prescriptive. This is in contrast to the physics of flight, which are completely predictive.
Periodically I see the question asked: why does the global scenery take up so many DVDs? Or more typically the question is: if it is so big, why is X missing, where X is the detail the user expected to see but did not find.
The size of X-Plane’s global scenery is due to two factors:
- Some aspects of the scenery are very detailed, and this takes up disk space.
- Some aspects of the scenery are computed ahead of time, not by X-Plane, and this takes up disk space.
Big Data
A few data sources make the scenery pretty big:
- The scenery is generated from 90m SRTM data for the entire world.
- US scenery includes the entire US road grid.
- Coastline data is fairly detailed in the US and for most oceanic coasts.
In other words, part of the size of the scenery comes from having reasonably accurate topography and coastlines everywhere.
Precomputation
The other source of file size is that X-Plane precomputes aspects of the scenery to reduce load on X-Plane while you fly. (We do this to improve frame rate.) In particular, the scenery uses a class of algorithms that are expensive to compute, and thus we compute them ahead of time and save the result (using up file space).
- Land class changes not on a regular grid, but along iregular polygonal boundaries following the shape of the terrain. This produces much more natural terrain transitions in the mountain, but requires more storage.
- “Auto-Gen” forests and buildings don’t follow a grid – rather, they fit into the irregular spaces made by vector roads and water. This fitting process is too slow to run inside X-Plane, so every auto-gen building and forest area must be pre-computed and saved.
Room To Grow
One interesting side effect of saving the end results of building locations (“pre-placement”, what X-Plane does) rather than the formula for how to place them (“auto-gen”, what FS X does) is that we can make the placement algorithm for buildings significantly smarter without any increase in file size or decrease in frame-rate. Having already eaten the cost of putting every building into the DSFs, we can make those locations better without hurting performance.
That algorithm to pre-place buildings also can have access to source data that isn’t available in the final DSF, allowing for effects that might not be possible in an auto-gen system.
I have blogged in the past regarding the rendering settings in X-Plane, but this seems to come up periodically, so here we go again. Invariably someone asks the question: “what computer do I have to buy to run X-Plane with all of the sliders set to maximum?”
I now have an answer, in the form of a question: “How hungry do you have to be to clean your plate at an all-you-can-eat buffet?”
There is no amount of hungry that will ever be enough to eat all of the food at an all you can eat buffet – you can always ask for more. And when it comes to rendering settings and global scenery, X-Plane is (whenever possible) the same way. You can always set more traffic, more birds, more objects, more FSAA.
Now the all-you-can-eat buffet doesn’t have infinite amounts of food in the building – just enough that they know that they won’t run out. And X-Plane is the same way. There is a maximum if you set everything all the way up, but we try to make sure that no one is going to hit a point where they want more eye candy but they’ve maxed out the settings. Eat all you want, we’ve got more.
Why on earth would we set up X-Plane like this? The answer is choice.
If you go to an all you can eat buffet, you can fill up on nothing but potatos, or you can have five pieces of chicken. It’s up to you. X-Plane is the same way – you decide if you want objects to be visible farther away or more densely. Would you rather have roads or trees? Birds or high frame-rate? You decide!
Not everyone’s appetite is the same, and not everyone’s taste is the same. This is very true when it comes to flight simulation. There are huge variations in hardware capability, target framerate (some users don’t mind 20 fps, some demand 80 fps) and in what part of the visual experience people care about most (objects vs. FSAA vs. visibility distance, etc).
Given such a heterogeneous environment, the only way to meet the needs of a wide group of users is to present choice, and make sure that we have enough of everything.
So when you go to set the rendering settings, don’t think that setting objects to anything less than maximum is like only eating half the steak you bought at a steak-house. Rather, the rendering settings are like picking which food from the buffet makes it to your plate. You choose how much you want based on what you can consume, and you pick and choose what is most desirable to you. And like an all you can eat buffet, don’t eat too much – the results won’t be pleasant!
Surely I’m not the only one with that song embedded in some deep part of my psyche, right? One thing we seem to have down with 940 is counting upward…945 is in testing now.
There are two things driving the perpetual 940 bug fix releases:
- Third parties are using the sim heavily, so the bug fix releases have been vehicles for fixing a number of small bugs that are only reproducible with custom add-ons. (The fix for night lighting and orthophotos is an example of this.)
- We’ve played a real game of whac-a-mole with the throttles. (The bug is that X-Plane will lock the two throttles together. I believe that the new 945 release candidate finally fixes this.) Having done a bit of a 3 stooges routine on this bug, we’ve tightened up our internal procedures a bit.
As always, if your third party add-on broke with 940, please file a bug report! The only case of this I know of is the throttles bug – 945 should fix the BK-117, the Mu, and any other plane suffering from throttle-glue.
Will there be a 946? There very well might be, and I sincerely hope it will not be because 945 missed its target. But if, two months from now, we’ve collected another half-dozen fixes for third party authoring, I don’t see why we wouldn’t release them.
Could The Name Beta Be Any More Confusing?
The version number gets bumped when we have a release and it still needs patching. Of course, some bugs get reported after we ship, and some address issues not even under consideration during release. Had we not had any fire drills, we would probably still be on 942 or 943 right now, because we’ve had third party reports with very specialized authoring cases after we finalized the build.
Here’s what makes things even more confusing: that check box “Check for new betas as well as updates” in the installer. At any one time we may have two versions of X-Plane availalbe for download:
- The official version (currently 944) for everyone.
- Whatever the latest in-test build is (currently 945).
(You can even see which versions are current here.) What’s confusing is that while this check box says “get betas”, in truth it gets any build that is “in staging”. At this instant that I blog, X-Plane 9.45 final is in staging. That is, we believe 9.45 fixes our two known bugs, we think it’s done, it’s a release candidate, we think it is the next American Idol^H^H^H^H^H^H^H^HX-Plane. But until we certify it, you still have to check “get betas” to download it.
What we probably need to do is re-label the check-box in some way; the “staged” approach where every build is available for test first is, I think, a valuable tool to let third party authors confirm that we fixed their bugs before the entire world gets the app.
This is my 500th post. I put off posting it all week because I wanted to post something lofty and clever. But in the end, the great is the enemy of the good – if I wait until I have a really good post, it could be weeks before I have time to write a 6000 word treatise on the relationship between quantum physics, shaders, and the price of crude oil.
The decision to publish less now or more later always comes up in software release planning – once the resource budget for a project is fixed* the only choice is ship sooner with less features or later with more features.
With both X-Plane and WorldEditor we often choose “ship now with less” for a simple reason: we are going to ship with more later, but if we ship now with less as well, people get some benefit in the meantime. WED is a perfect example: the first version could only edit airports, and shipped almost 18 months ago. Had we waited until we had overlay editing and airports, we would have had a more impressive release, but authors would have had no editing at all for 18 months. Why force the people who want to edit airports to wait 18 months for overlay features?
(An assumption in this is that the cost of actually doing a release is fairly low. Obviously we don’t want to do a new release every single week!)
There is a contrary force that might argue against frequent releases: once we change a feature to make it better, users are surprised if we don’t make it perfect. Users assume that if we fix some bugs in a feature but not all bugs, that it’s because we didn’t know about the bugs we didn’t fix. (The truth is usually that we had limited resources.) This produces a very strange situation where users are sometimes happier with a product that is less featureful/more deficient/more buggy because a small improvement in real functionality introduces an expectation of a large improvement.
A second behavioral phenomenon amplifies this: in my experience users consider new bugs to be significantly “buggier” (for lack of a term) than bugs that have been around for a while. This is perfectly understandable: humans are very adaptable and we get used to a bug over time to the point where we may not consider it as “bad” as when we first saw it. Trade the old bug for a new one, and we have to become re-acclimated.
These two behaviors argue (particularly when bugs and limited functionality are involved) to make a small number of large changes that move an aspect of the program from one ‘stable’ configuration to the next.
* If you think that more resources can break this trade-off between features and a quick release date, I strongly recommend “The Mythical Man Month“. The short version: 9 women can’t have a baby in 1 month – if you want a quicker release you have to do less.
With a new year and CES upon us, it’s go time for pundits to predict the future of the technology industry. Lori considered the question of how technology might affect X-Plane and pronounced:
Pixel shaders will get shadier.
I think she is right. It is only a question of how long until they are so shady that they are basically pitch black. (At that point, I expect a significant boost in framerates!)
