The performance of the new X-Plane 8.50 lights using pixel shaders is looking good. However, old scenery that uses a lot of lights won’t benefit.
The problem is that the old lights were subject to ATTR_LOD limitations. Outside the last LOD the lights disappear; authors usually do some creative things with the LOD of the object to make the lights appear at the right distances.
But the new lights are drawn independently of the object LOD. This is good – it means that a low-LOD object (good for framerate) can still produce a light visible a long way away.
The problem is – these two things are not compatible. So we don’t use the new light technology for existing OBJ lights.
If you want to use the new lights in your scenery and you have a “point lights object” all you have to do is convert the object to OBJ8 format (if it isn’t already) and then substitute the old light commands for the new ones. If you have two LODs you may want to delete one, as it’s no longer necessary. Change the OBJs and leave the ENV or DSF alone and you should see a performance increase on hardware with pixel-shaders.
(Yes, you can use OBJ8s with ENVs. I don’t recommend it, but when X-Plane finds a custom object in an ENV it will read it no matter what version – 2, 700 or 800. So if you have a custom ENV package with orthophotos and point lights you can convert them.)
I’ve been asked by a few object authors to put a show/hide command in the OBJ8 format for more advanced animation. It turns out this is trickier than you might think. For example, consider this set of OBJ8 commands (based on a hypothetical ANIM_hide command):
ATTR_flat
TRIS 0 30
ANIM_begin
ANIM_hide sim/flightmodel/whatever
ANIM_smooth
TRIS 30 30
ANIM_end
TRIS 60 30
What’s tricky here is: the use of state changes inside the “hidden” part of the object. Should the triangles after the animation be smooth, flat, or change depending on the animation?
Internally X-Plane has an object optimizer; when objects are loaded we analyze it and attempt to reduce the number of total commands, so that object drawing is as fast as possible. I strongly recommend you optimize your objects heavily – if we optimize, we might make the object faster, but if you optimize you will make it faster. Our optimizer is aimed at speeding up internal commands that are generated by X-Plane as it translates the object from OBJ7 or OBJ8 to our internal memory structure.
The problem with using show/hide to optionally skip commands is that the optimizer can no longer remove state changes. Because we don’t know whether we’re flat or smooth (after a hide command) we can’t know whether the next flat or smooth attribute is needed or unnecessary. So if show/hide is truly conditional (meaning it can skip commands) then it actually can cause things to be slower because it prevents X-Plane from removing unnecessary state change.
(The case of smooth/flat might seem trivial and contrived; it is. But there are additional “hidden” attributes that are generated by X-Plane as it translates the OBJ into our in-memory internal format. So state change can come from places you might not expect. Thus it’s important that the optimizer do its best to remove state change that can’t be addressed in the OBJ itself.)
The alternative (and one that is more likely to get implemented) is show/hide as state. Under this model, the full object is completely evaluated, but the ANIM_hide command basically causes triangles to not appear. This is similar to the workaround that people are doing now: using ANIM_translate to move the unwanted geometry a very long way away (and hoping it is out of sight).
Show/hide as state makes a trade: all commands are executed always, so we run no faster by hiding, but since the execution of the object never changes its flow, it can be more heavily optimized.
I am currently leading toward the latter approach for this reason: currently objects are slow because the CPU must do the work of state change (ATTRibutes, etc.). But graphics cards are becoming more sophisticated; in the future it may be possible to run an object with state change as a single command for the graphics card, with no CPU intervention, making things faster. But the graphics card does not yet have the ability to handle conditional logic, so show/hide as conditional logic could stop us from fully utilizing the graphics card in the future.
(Some OpenGL geeks may be thinking now “but the latest graphics cards do have conditional logic”. This is true, but currently it is only usable in shaders; the cards do not yet have the ability to skip whole commands conditionally. I believe they will be able to handle static state change before they can handle conditional state change.)
Normally I try to avoid saying anything about future releases; I find that pre-announcing things just induces Murphy’s Law…something unexpected goes wrong, and people are disappointed when what we ship turns out to not be what we thought we shipped.
So at risk of getting myself into trouble, let me say this:
It is my goal to ship the first version of the graphical DSF overlay
editor concurrently with X-Plane 8.50.
That’s an aggressive goal, because X-Plane 8.50’s going to have a lot of features in it, so Austin and I are really busy right now. But hopefully by putting it out there, I can hold my own feet to the fire and get this important tool out to users!
A quick note on the AC3D plugin based on a conversation wtih Tom….
The AC3D X-Plane OBJ export plugin is not an optimizing plugin – it is meant for “finishing”.
The idea is that the plugin can edit an OBJ and preserve almost any strange sequence of attributes you come up with. If you’ve carefully ordered your polygons and used ATTR_no_blend and a bunch of other tricks to use translucency and other tricks, the plugin will not scramble your work, but will preserve it. This lets you put the finishing touches on your OBJ, hence a “finishing” plugin.
The other side of this is that in preserving your object, the plugin does not attempt to make any optimizations that would reorder your object. So if you “ask” the plugin to make an inefficient object by using lots of state changes, it will do so and your framerate will suffer.
Basicaslly there is a direct correspondence between the model in AC3D and the exporter OBJ file. So to use AC3D efficiently you must understand both what makes an OBJ fast (not using ATTRibutes among other things) and how you do these things in AC3D (not using flat shading at all, for example).
The best of both worlds would be of course to have a menu option to optimize a model, which would let you optionally change polygon order for speed. This is something I would like to do but I’m not sure when I’ll have time to code it.
850 is turning out similar to most feature-based X-Plane releases – like a tropical storm picking up speed features keep getting sucked in, creating more chaos in its core. I thought I would be working on airports and lighting only for 850, but there’s already a number of other things floating around in the release. The end result should be better visuals.
A quick note on usage of PNG: the formats of X-Plane and PNG images are not quite the same.
- To provide a non-alpha image in X-Plane, use an RGB or grayscale PNG.
- To provide an image with alpha in X-Plane, use an RGB or grayscale PNG with an alpha channel.
- To provide an alpha mask in X-Plane, use a grayscale PNG.
This last case is the strange one: as far as I can tell PNG does not allow an alpha-only image. So when X-Plane needs an alpha mask (a 1-channel image) it uses a grayscale (1-channel image).
Also please note tht you can use a color or grayscale PNG for a color texture, but internally X-Plane always operates in RGB, so you don’t save any VRAM by reducing your PNGs to grayscale. You might save some space on disk.
First the basics: you can use the ATTR_LOD command in an object to specify what distance range an object is seen from. This has two implications:
- You can have your object look different (and usually less complex) from farther away.
- Probably more important is – your object is not drawn from arbitrarily far away. All drawing is bounded!
This second aspect is actually the more important one…because at any given time most of the objects in the sim are too far away to draw due to LOD! But this begs the question: what happens if you don’t specify ATTR_LOD at all? Is your object drawn from hundreds of miles away?
The answer is: if you don’t specify ATTR_LOD X-Plane calculates an LOD for your entire object that starts at 0 meters and goes to a distance where we think no one would care if your object disappeared. The calculation is based on the bounding sphere of your object.
Our built-in algorithm usually works pretty well, but there are some cases where it gets fooled. One important case is: when you merge two small objects into one big object, the LOD distance goes up, because the total area that your object covers goes up!
What this means is: if you have a dozen small taxiway signs from XTaxiMaker, they’ll all disappear relatively quickly as you go away from the airport, protecting framerate (by keeping the number of drawn objects down). If you merge those into one big object, the performance of the sim changes in two ways:
- Framerate when the objects are drawn goes up – we now draw one object instead of five. The number of objects drawn is very important to framerate.
- The objects will be drawn more frequently, so overall framerates maybe lower.
The moral of the story is this: if you merge objects for performance, be sure to insert an ATTR_LOD statement that is apporpriate to the actual distance at which point the objects are invisible, not the total “radius” of the object! Taxi way signs are not needed after 1500 meters, even if they cover the whole airport. Similarly some packages have taxiway markings in an OBJ. These objects usually cover big areas but the lines themselves are so thin that they aren’t needed when far away.
(Also beware: LOD is calculated frmo the middle of the object, so if the object is huge, take that into account.)
As a final note, ATTR_LOD is “free”. X-Plane will calculate and check LOD distances on every object no matter what. If you don’t use ATTR_LOD, one is generated for you. So there is no harm in adding ATTR_LOD to your object. Your framerate will only go down if you pick an LOD that is larger than the sim would have picked for you.
EDIT FROM 2/19/07: during X-Plane 850 when I was developing the new lighting system (particularly used for airports, but also for the rest of scenery) I brought up the subject of “detachment”, a internal process by which the X-Plane rendering engine processes lights.
Simply put: detachment is virtually never a factor for scenery authors. Back when I wrote the article, it was possible for an author to make a few special optimizations if he or she knew about detachment. This is no longer true! The sim will correctly optimize all lighting cases as much as possible without any special help from the author.
So….the blog article in its original form follows and may be useful as an insight into how X-Plane works, but as an author, just remember: you don’t need to do anything – just make nice OBJs.
I think I’ve confused just about everybody in the last few days – especially Sergio and Austin – with work on the new lighting system. One reason I’m so scrambled: the internal representation of lights in X-Plane is very different from the commands you can put in an OBJ8 file. So there’s a whole set of terms and ideas that are important to how X-Plane works that I shouldn’t have mentioned to anyone making OBJ8s because they won’t even be part of the file format.
Then there’s detachment. I think I can explain it simply like this: detachment is when X-Plane changes the order that lights are drawn inside an OBJ and the LOD ranges in which they are drawn to improve framerate.
X-Plane 840 does not do detachment. X-Plane 850 will probably not do detachment for objects made only with the old lighting commands. But 850 will do detachment for objects that use new (to 850) lighting commands, and this will allow those objects to render very quickly even with textured lights.
This picture is actually from X-Plane 8.03 I believe. It’s a simple trick: edit the roads file to add street light objects along all roads. Make a street light object with a light command and then turn roads on and watch teh city at night appear.
The only problem is the frame-rate – this experiment brought 8.03 to a standstill.
The goal of detachment is to make textured lights in objects fast enough that this kind of scene can be rendered at reasonable framerates. I do not know if this will be possible, but detachment should certainly improve performance in cases like this.
In considering how we design new scenery features, we have to choose between a high level or a low level approach.
- In a high level approach, we describe the data abstractly as possible. “Taxiway light” is a pretty high level description.
- In a low level approach, we describe the data as specifically as possible. “Light whose RGB value is 0.1,0.1,0.9, with a halo, 0.4 meters above the ground, with texture coordinates…” is a very low level description.
In favor of a high level approach is performance and ease-of-use. For some catagories of usage, it’s easier to work with high level primitives because the description is less tedious. It’s easier to plot taxiway lights than to describe one. High level primitives also make performance easier: it gives X-Plane the freedom to draw taxiway lights in the fastest technique possible.
Low level approaches offer flexibility. Authors can make up lights we’ve never heard of before. It also means less code; there are perhaps a few dozen distinct types of lights in the airport area, all of which must be programmed, but code to handle a generic light of a given color can be written once.
The danger of a low level approach is that we could allow a scenery construction that is difficult to support in the future, and then have to choose between sacrificing performance and breaking backward compatibility. Neither option is much fun, which gives me pause in considering such design decisions.
At this particular instant for lights I am looking at a two-tiered approach:
- Simple mode lights will provide only the most basic lighting “tricks” (flashing or turning on at a certain time of day) and will be very fast. Because these lights are not very flexible, X-Plane has a lot of room to optimize how they work.
- Complex mode lights will do almost anything but be relatively slow (meaning the same speed as textured lights now). This will let us do the really tricky lights like airport beacons, which are complex but fortunately don’t have to be fast because they are relatively rare.
So to answer the general question “why don’t you let us customize XXX”, the answer is often: if we let you do whatever you want with the scenery itself, then we can’t do whatever we want with the rendering engine, and that could mean slower performance on future hardware. The design decisions are a compromise of flexibility and performance.
The new scenery tool will work with scenery packages. Here are some details:
The new scenery tool will use a “working scenery package.” This is a normal X-Plane scenery package (preferably in a custom scenery folder in an x-system folder) with an extra folder that contains the “in progress” materials – your working master copies of scenery that will be saved later as DSF files.
This way the scenery tool can easily find all of your objects (using the library system) and display them as you edit, and X-Plane can show the objects in the package. When you finish editing a scenery file, you export and the scenrey tool saves a DSF in the Earth nav data folder.
The idea is to provide a single folder containing all of the resources (in progress and finalized) for your scenery project in one place. When we were working on global scenery last year, we had several folders that could be in different places, and the result was total configuration chaos. With the package system, all of your work is together – easily backed up, shared with other authors, and kept separate from other versions or works in progress.
With regards to the scenery system, the idea of orthogonality is this: each part of the scenery system should be unaffected by other parts, such that they can be used together in any combination. This allows you the most options in creating scenery for a given set of features.
One way we try to achieve this is by only having one way to implement any particular feature. For example, because it is possible to place custom objects in overlay DSFs, we are unlikely to allow custom objects to be placed via the apt.dat file. Having two ways to place custom objects would invariably lead to subtle differences in how the two worked; not all features would be usable everywhere. It would also mean more code (read: longer time before you get to use the feature) and more subtle bugs.
Once we have a feature only available in one form, we can try to use that mechanism everywhere. For example, objects can be animated. Because any object can be animated and objects are used both for cockpits and scenery, the work we did to provide animation allows for animation in both scenery and the cockpit. When we get cars back on the road, they’ll probably be objects; navaids are also objects now, and someday hopefully ships and oil platforms will be too. Each time we recycle OBJ8 for a part of the system, animation becomes possible in a new domain.
So usually the ‘orthogonality’ strategy, code something once and use it everywhere with no duplication, means more flexibility and features being usable over a wider area. Every now and then though it bites us in the ass. Flexibility is a strength and weakness; the more we allow any part of the scenery system to work with any other part, the more likely two parts will be used together in a way that we never expected, potentially revealing bugs.
Here’s an example that I only realized existed today, after talking to Peter about animation: it turns out that if you use a textured light inside an animated object, the light (which is secretly a flat square that always points at the camera) will actually turn with the object, revealing that it is flat, and being invisible most of the time (either because it’s facing sideways and is very thin, or because it’s facing in the other direction entirely).
The problem is: when working on animation, I forgot about the textured light system and forgot to cope with the case where they interact. When the code is truly separate, an orthoganal design works well; where the code has internal interactions, it does not.
Do you care? Well, we are looking at some new features for the textured lighting system in 850; there is probably going to be some fine print about how you can use the new textured lights with animation. Even though the features are orthogonal, the code that runs it is interdependent, so some cases may not work right without extra work. How many strange cases we fix, and how many we say “just don’t do that” for will depend on how useful they really are. For example, not being able to have lights on animated objects at all would be a real limit, but perhaps not every kind of light needs to be usable within an animation. For example, VASI lights are usually just mounted on the ground.
