I want to revisit the question of whether (and how) the livery system should be extended. In particular, it is my opinion that the livery system should not be extended to allow:
- Replacement of OBJs used for modeling the airplane or cockpit.
- Alternate or modified ACFs*.
- Generally, the livery system shouldn’t be used for changing the plane’s behavior – it’s just paint!
I have commented previously in three parts that the livery system is meant to make easy the integration of third party paint without (a) violating copyright, (b) requiring byzantine installation instructions or (c) requiring the painter and original author to coordinate. I have received requests from a number of very talented airplane authors, asking for the livery system to cover a whole range of new features, most of them involving configuration. I will try to explain in this post how I think should should be handled.
First, to be clear: an aircraft file is the .acf file that contains the X-Plane specific data needed to simulate the plane; the aircraft package is the folder containing that .acf file. A livery is a painting scheme for the 3-d model of the airplane, and a configuration is an instance of a plane with certain features, e.g. with or without G1000, with P&W; vs. Rolls Royce engines, etc.
Configurations of an aircraft should be created by putting more than one aircraft file
(.acf) in a single aircraft package. Because the graphic and sound resources needed for the aircraft are accessed relative to the .acf file, you can build a family of aircraft with some common aspects and some unique aspects to each plane.
Files used by an aircraft fall into three broad categories:
- Files found by a fixed formula using the .acf name, e.g. be20_paint.png. Let’s call these “file-specific”.
- Files found by a fixed formula without using the .acf name, e.g. the contents of the aircraft plugins folder. Let’s call these “package specific”.
- Files that are explicitly named in the .acf file, like airfoils and OBJs. Let’s call these “flexible” (since this naming scheme could be used in any way).
Here’s how the important files in an aircraft break down:
- The aircraft paint scheme is file specific.
- The aircraft panel background is package specific (but you can effectively have each file use a different panel background by setting the panel type differently for each plane).
- Sounds are actually either, which effectively makes them flexible. Non-generic instruments are package-specific.
- Objects, generic instruments and airfoils are flexible.
In other words, if you can live with duplicating your aircraft paint files (and I suspect that in most cases either the plane is built by objects, or the modifications in each configuration warrant paint changes anyway), then every other feature can be set to package or file specific, allowing you to build a group of aircraft around a single real-world plane.
Now if there are aspects of a multi-configuration aircraft package that don’t work right now, we can look at possible changes to the sim to make this work. But it appears to me that just about everything necessary to make multiple configurations is already available in the sim now.
As a final note, the question here (livery vs. multi-file aircraft pack for configurations) is one of file formats, and thus of data organization and contracts between authors and programmers. It is not a question of user interface. The user interface can be reshaped to make multi-aircraft packages look like liveries, or liveries look like multi-aircraft packages. But I suspect that most of the interest in extended liveries is on the file-format side.
* The one exception for liveries is the tail number — given the strange state where the tail number, as painted into the livery, is also written into the ACF, it wouldn’t be bad to be able to override this property. Some people are already doing this using plugins.
Some coding problems are stubborn – I find myself looking back at a week of working realizing that all I really did was prove that a bunch of theoretical improvements don’t work in practice.
Improving OBJ throughput is one of those problems. On a high-end machine, even drastic changes to the OBJ engine make only the slightest difference in throughput – 2 or 3% at best. Every improvement counts, but a 3% improvement doesn’t change the game for how we draw scenery.
There is at least one route I haven’t had time to go down yet: object instancing. The theory is that by making many objects appear with only one object drawn, we get a multiplier, e.g. a 2x or 4x or larger amplification of the number of objects we can have.
In practice it won’t be that simple:
- To get such an amplification we have to recognize groups of the exact same object. Grouped objects will have to be culled together. So we might get a hit in performance as we draw more objects that are off-screen, just to use instancing.
- It may be that the grouping requirement is so severe that it is not practical to find and group objects arbitrarily (instead we would have to group objects that are built together, like clusters of runway lights). That might limit the scope of where we can instance.
- The objects have to look more or less the same, so some categories of very complex objects won’t be subject to instancing. (E.g. objects with animation where each object might look different.)
- I have already coded some experiments with geometry shaders, and the results are just dreadful – geometry shaders simply don’t output a huge number of vertices efficiently, so they don’t help us increase our total vertex throughput. The experience has left me with a “prove it” attitude toward GL extensions that are supposed to make things faster.
When will we know whether instancing can help? I don’t know — I suspect that I won’t be able to find time for code experiments for a bit, due to other work, particularly on scenery creation and tools.
A hidden detail of my previous post on variation and terrain textures: variation for flat textures was implemented using more triangles in the DSF in X-Plane 8, but is implemented in a shader in X-Plane 9. This means that you don’t get this feature in X-Plane 9 if shaders are off.
My guess is that this is perfectly acceptable to just about every user.
- If you don’t have shaders, you have something like a GeForce 4 or Radeon 8500, and are fighting for frame-rate. In this case, not paying the price of layer-based variation is a win.
- If you have shaders, you’re getting better performance because the shader creates variation more efficiently than the old layering scheme did.
This kind of move of a feature to the GPU can only happen at major versions when we recut the global scenery, because (to utilize the benefit) the DSFs are recut with fewer (now unneeded) layers. So features aren’t going to mysteriously disappear mid-version.
I do have a goal to move more layering-type features to the GPU for future global scenery renders. There are a number of good reasons:
- DSF file size is limited – we have distribution requirements on the number of DVDs we ship. So DSF file size is better spent on more detailed meshes than on layers.
- GPU power is increasing faster than anything else, so it’s good to put these effects on the GPU – the GPU is still hungry for more!
- If a feature is run on the GPU, we can scale it up or down or turn it on or off, for more flexible rendering settings on a wide variety of hardware. A feature baked into the DSF is there for everyone, no way to turn it off.
My hope for the next render is to (somehow) move the cliff algorithm (which is currently done with 2-4 layers) to the GPU, which would shrink DSFs, improve performance, and probably create nicer looking output.
In my previous post I discussed variation as a way to hide the artifacts of land use texturing. Now we can talk about this bug.
What are these weird artifacts that show up over the terrain when shaders are on? Well, they should (and will in 930) look like this:
But what’s going on? The answer is auto-variation.
In X-Plane 8, variation is created by using multiple layers, each one applying a texture at a different offset. This technique works on a wide range of hardware, but is inefficient – it causes overdraw (which we know is
very bad).
So in X-Plane 9 I replaced this layer-based variation with a pixel shader algorithm. This means less information in the DSF (which means smaller DSFs, faster loading and less RAM use), but it also means variation is only visible to those with shaders. Having the pixel shaders create variation dynamically on the GPU is called “auto-variation” (and is invoked via the AUTO_VARY command in a .ter file).
The artifact above was a bug in the auto-variation shader. With the code now fixed (the 930 patch will contain the fix), here are some images of how it is supposed to work:
Here we have the texture in question, at two different offsets.
This black and white texture is the “mixing mask” used to select which offset to use.
And this is the final result.
There is a little bit more disruption in the columns of green park.
I was going to post some pictures of the newly fixed “auto-vary” feature, but before I can do that in a way that makes any sense, I need to explain how X-Plane deals with texture repetition.
Texture repetition is the inevitable result of using “landuse-style” texturing (that is, a repeating single texture representing a type of land). Typical X-Plane land use textures are 1024 x 1024 at max res and repeat about every 3-5 km. Unfortunately, our brains are pattern-recognizing machines, and the result of this texturing scheme is that the “grid lines” of texture placement become apparent over wide views.
We use a number of techniques to minimize this problem.
Lots of Land Uses
Our main tool to combat repetition is to not use a given land use for too large of an area. This has the advantage of efficiently using the entire set of textures, and (because terrain textures change based on an irregular grid, based on elevation) the changes to textures are both irregular in shape and “plausible” in placement.
In this picture, you can see that the urban residential land use has been interrupted by various forest and grass textures. This is intentional – Sergio tuned hte land use rules to make sure that we wouldn’t have large regions of one land use type. Those blobs match the irregular grid, which gets its shape from the terrain’s elevation.
Variation
In the above picture, you can still see the repeating grid of the residential terrain; observe the right side – you’ll see the same repeating vertical pattern of road repeating over and over. In order to further hide repetition, we use the same texture multiple times, but in offset locations.
Here you can see that the vertical line on the right side has been broken up a bit.
More VRAM
In some cases, a terrain covers such large areas despite the rule set (e.g. for really flat areas) that we use two separate textures and can vary between them. Here you can see both input textures for our dry square crop land use, as well as the combined results.
In summary, we have three techniques:
- Add more rules to prevent large spans of a single land-use.
- Use a texture with multiple offsets (variation)
- Use two textures and vary between them.
I have blogged about this before, but I will try to create one simple explanation of what’s going on with sim/cockpit2 and sim/flightmodel2 datarefs.
Sandy and I (with the help of others who helped compile the list) created “new” datarefs (first released with X-Plane 9) , aimed at airplane modelers. These new sections are:
- sim/cockpit2/ which provides a new set of datarefs for cockpit modeling via OBJ animation and generic instruments.
- sim/flightmodel2/ which provides a new set of datarefs for airplane exterior modeling via OBJ animation.
These datarefs sometimes include new data that was not available in version 8, and sometimes they simply provide a second dataref with the same information. Why duplicate datarefs? The new datarefs have some special properties, so I wanted to have a complete set of datarefs for modelers with these new properties.
Skip to the end for the rules of thumb on how to use them.
Clean Naming
The new datarefs are designed to have longer, less confusing names; the old datarefs contained a lot of abbreviations – potentially acceptable for programmers (who are used to seeing things like fstat and chgrp on a regular basis) but not good for modelers who do not speek English as a first language. The new datarefs have long names and are more consistent in their conventions. They also contain complete documentation.
Array Sizes
You will see the array dimension of some of the new datarefs as symbolic constants, e.g. [engine] instead of [8]. This is because the dataref generation system we use knows that these new datarefs sometimes track the maximum number of parts in the aircraft structure. This tagging means that it is much simpler for Sandy and I to adjust the datarefs when Austin increases part maximums.
With the old datarefs, if Austin allows for 10 engines, Sandy and I must search for every [8] dataref and decide if it must be [10] – some will be per-engine and need to change, some will be per-battery and will not! With the new system, we simply redefine the “engine” constant to 10 and the datarefs adjust.
(Note that if your plugin really needs to run dynamically with any number of engines, the best thing to do is to read the array size using XPLMGetDatavX.)
Failure Support
There are two ways to view a dataref: before system failures (such that the dataref reflects simulated physical reality) and after system failures (such that the dataref reflects pilot indications). For example, when the pitot tube ices up, the pre-failure airspeed reflects how fast you are flying; the post-failure airspeed reflects how much crud is in your pitot tube.
Pre-failure datarefs are appropriate for animating the exterior of the airplane. For example, if the gear indicator light fails but the gear is working, you want to animate your landing gear based on the real (pre-failure) gear position, so that the gear really does look like it’s down from an outside view.
Post-failure datarefs are appropriate for animating the cockpit. For example, you want to use that post-failure indicated airspeed for your air speed indicator, so that pitot ice will affect your generic instruments and animations, as well as the built-in instruments.
The new datarefs are designed to clearly provide two different views:
- sim/cockpit2/ are all post-failure whenever possible, and are thus appropriate for cockpit modeling.
- sim/flightmodel2/ are all pre-failure, and thus are appropriate for external airplane modeling.
Be careful not to swap them! You should always be using sim/flightmodel2/ for your aircraft and sim/cockpit2/ for your cockpit. If the dataref you need is in one and not the other, email me and I will add it to the right place.
Correct Multiplayer Behavior
The older datarefs all return data about the user’s airplane. However if you build an object, attached to an ACF, and that ACF is loaded for a multiplayer plane, you will get incorrect results — the user will see his own plane’s actions visualized on the multiplayer plane.
The new sim/cockpit2/ and sim/flightmodel2/ datarefs handle this case correctly: they return data about whichever airplane is being drawn. Thus if your object is attached to airplane number 5 in a multiplayer session, that’s the airplane that will animate your control surfaces.
(Plugin developers – outside airplane drawing, these datarefs return information about the user’s flight.)
For this reason, you should always use sim/cockpit2/ and sim/flightmodel2/ – not the older sim/cockpit and sim/flightmodel/ datarefs. If the dataref you want is only in the old sections but not the new ones, email me!
What Dataref Do I Use?
Here’s the rule of thumb:
- If you are targeting X-Plane 6/7/8, you must use sim/cockpit and sim/flightmodel, otherwise
- If you are targeting X-Plane 9, use sim/cockpit2 for your generic instruments and 3-d cockpit. Use sim/flightmodel2 for your attached objects.
That’s all there is to it!
A few things are in the works:
- The X-Plane messaging system, which checks for updates, can hang up if DNS isn’t available. I should have fixed this a lot sooner, but this will be addressed in a very small 9.22 patch, in the process of being built now.
- 9.22 will also include Robin’s latest apt and nav data.
- For Linux users: 9.22 should work with threaded OpenGL on newer distros – thanks to Jan for sending me the code snippet to fix this!
And on the iphone front: the next X-Plane iphone free update should improve memory use during DSF load. This in turn will hopefully address the application suddenly quitting on “loaded” iphones (that is, iphones with a lot of email accounts or other background tasks that use memory). Memory was temporarily spiking as we optimized the DSF during load. I am not sure when this will make it to the iTunes store.
I am looking at OpenAL on Linux, but this will have to wait for 930 and a longer beta program. 922 will also not have a FADEC – 922 is a quick bug fix patch, not a feature release!
Posted in News
by
Ben Supnik |
The short answer is: this is not a very good idea.
Now with OS X, this configuration is supported, and OS X will cleverly copy graphic output from one video card to another to make the system work well. You will get a fps hit when this happens.
With Vista, this configuration isn’t supported. (Snarky comment: it is lame that Microsoft completely rewrote their video driver infrastructure and went backward in terms of configuration support.)
With Linux, I have no idea if this configuration can run. I do know that trying to change my configuration hosed Ubuntu thoroughly and I decided not to break my Linux boxes any more, having spent plenty of time doing that already in the last few days.
For X-Plane, we can’t handle this case very well (at best you get the framerate hit) because we need to share textures between the IOS screen and main screen. So if you are trying to set up an IOS screen, you really do need a dual-headed graphics card. For what it’s worth, every card I’ve gotten in the last few years has had two video outputs.
My Mac Pro has just gotten weirder – I put a Radeon HD 3870 into the second PCIe x16 slot. (The machine comes with a GeForce 8800.) I now have one monitor in each.
So here’s where things get fun:
- Start X-Plane. 60 fps.
- Drag the window to the second monitor. 30 fps.
- Quit, move the menu bar to the second monitor, restart. (X-Plane is now on the right.) 160 fps.
- Drag the window back to the primary monitor on the left. 100 fps.
What’s going on? Two things:
- On OS X, X-Plane’s graphics are rendered by one video card, and that video card (in 921) is the card that has the menu on one of its monitors.
- When an OpenGL window is displayed on a monitor that is not attached to the video card that is doing the rendering, OS X will copy the image from one video card to another, at a cost of some framerate.
So what’s going on above? Well, the 60 fps is my 8800. When I drag the window, the OS starts copying the graphics, slowing fps. When I move the menu bar, the 3870 does the rendering, and we get much higher fps. Once again, put the window on the monitor that is not attached to the video card, and fps hit.
Final note: fps tests of the 8800 vs 3870 with X-Plane 921:
Fps test 2, 8800: 46,49,51
Fps test 2, 3870: 70,75,80
Fps test 3, 8800: 24,25,25
Fps test 3, 3870: 40,41,43
In other words, the 3870 is significantly faster. I believe that this is due to the OS X drivers, not the cards themselves. Note that the 3870 is in a PCIe 1.0 slot and the 8800 is in a PCIe 2.0 slot.
I think we’ve reached the point where, if you are putting together a new computer and have X-Plane in mind:
- Get a quad-core machine if the pricing is favorable (and I think it should be now).
- Get a “Direct X 10” compatible graphics card. That would be an nVidia 8, or 9 series (or I guess that crazy new 280 card) or a Radeon HD 2000/3000/4000. DX10-type cards can be had for $100 to $150.
Quad core is easy: X-Plane 921 will use as many cores as yo have for texture loading (especially in paged scenery), uses two cores all the time, and uses 3 during DSF load. The infrastructure for this additional scalability (previous builds used two cores, more or less) will let us put 3-d generation on 4 cores or more. More on this in another post, but basically X-Plane’s utilization of cores is good and getting better, so four cores is good, particularly if it’s not a lot more expensive.
Now for DX10, first I have to say two things:
- We don’t use DirectX. We have no intention of switching to DirectX, dropping OpenGL support, or dropping OS X/Linux support. I just say “DX10” to indicate a level of hardware functionality (specified by Microsoft). The DX10 cards have to have certain hardware tricks, and those tricks can be accessed both in OpenGL and Direct3D. We will access them by OpenGL.
- We are not going to drop support for non-DX10 cards! (We’re not that crazy.)
X-Plane does not yet utilize those new DX10 features, but the DX10-compatible cards are better cards than the past generations, and are now affordable*. By making sure you get one of these, you’ll be able to use new graphic features when they come out.
* The roll-out of DX10 cards has been similar to DX9. With the first generation cards there was one expensive but fast card and one cheap but slow card. With DX10, NVidia got there first, with DX9 ATI did. Like a few years ago, now that we’re a few revs into the new spec, both vendors are making high quality cards that aren’t too expensive.
