EDIT FROM 2/19/07: during X-Plane 850 when I was developing the new lighting system (particularly used for airports, but also for the rest of scenery) I brought up the subject of “detachment”, a internal process by which the X-Plane rendering engine processes lights.
Simply put: detachment is virtually never a factor for scenery authors. Back when I wrote the article, it was possible for an author to make a few special optimizations if he or she knew about detachment. This is no longer true! The sim will correctly optimize all lighting cases as much as possible without any special help from the author.
So….the blog article in its original form follows and may be useful as an insight into how X-Plane works, but as an author, just remember: you don’t need to do anything – just make nice OBJs.
I think I’ve confused just about everybody in the last few days – especially Sergio and Austin – with work on the new lighting system. One reason I’m so scrambled: the internal representation of lights in X-Plane is very different from the commands you can put in an OBJ8 file. So there’s a whole set of terms and ideas that are important to how X-Plane works that I shouldn’t have mentioned to anyone making OBJ8s because they won’t even be part of the file format.
Then there’s detachment. I think I can explain it simply like this: detachment is when X-Plane changes the order that lights are drawn inside an OBJ and the LOD ranges in which they are drawn to improve framerate.
X-Plane 840 does not do detachment. X-Plane 850 will probably not do detachment for objects made only with the old lighting commands. But 850 will do detachment for objects that use new (to 850) lighting commands, and this will allow those objects to render very quickly even with textured lights.
This picture is actually from X-Plane 8.03 I believe. It’s a simple trick: edit the roads file to add street light objects along all roads. Make a street light object with a light command and then turn roads on and watch teh city at night appear.
The only problem is the frame-rate – this experiment brought 8.03 to a standstill.
The goal of detachment is to make textured lights in objects fast enough that this kind of scene can be rendered at reasonable framerates. I do not know if this will be possible, but detachment should certainly improve performance in cases like this.
In considering how we design new scenery features, we have to choose between a high level or a low level approach.
- In a high level approach, we describe the data abstractly as possible. “Taxiway light” is a pretty high level description.
- In a low level approach, we describe the data as specifically as possible. “Light whose RGB value is 0.1,0.1,0.9, with a halo, 0.4 meters above the ground, with texture coordinates…” is a very low level description.
In favor of a high level approach is performance and ease-of-use. For some catagories of usage, it’s easier to work with high level primitives because the description is less tedious. It’s easier to plot taxiway lights than to describe one. High level primitives also make performance easier: it gives X-Plane the freedom to draw taxiway lights in the fastest technique possible.
Low level approaches offer flexibility. Authors can make up lights we’ve never heard of before. It also means less code; there are perhaps a few dozen distinct types of lights in the airport area, all of which must be programmed, but code to handle a generic light of a given color can be written once.
The danger of a low level approach is that we could allow a scenery construction that is difficult to support in the future, and then have to choose between sacrificing performance and breaking backward compatibility. Neither option is much fun, which gives me pause in considering such design decisions.
At this particular instant for lights I am looking at a two-tiered approach:
- Simple mode lights will provide only the most basic lighting “tricks” (flashing or turning on at a certain time of day) and will be very fast. Because these lights are not very flexible, X-Plane has a lot of room to optimize how they work.
- Complex mode lights will do almost anything but be relatively slow (meaning the same speed as textured lights now). This will let us do the really tricky lights like airport beacons, which are complex but fortunately don’t have to be fast because they are relatively rare.
So to answer the general question “why don’t you let us customize XXX”, the answer is often: if we let you do whatever you want with the scenery itself, then we can’t do whatever we want with the rendering engine, and that could mean slower performance on future hardware. The design decisions are a compromise of flexibility and performance.
The new scenery tool will work with scenery packages. Here are some details:
The new scenery tool will use a “working scenery package.” This is a normal X-Plane scenery package (preferably in a custom scenery folder in an x-system folder) with an extra folder that contains the “in progress” materials – your working master copies of scenery that will be saved later as DSF files.
This way the scenery tool can easily find all of your objects (using the library system) and display them as you edit, and X-Plane can show the objects in the package. When you finish editing a scenery file, you export and the scenrey tool saves a DSF in the Earth nav data folder.
The idea is to provide a single folder containing all of the resources (in progress and finalized) for your scenery project in one place. When we were working on global scenery last year, we had several folders that could be in different places, and the result was total configuration chaos. With the package system, all of your work is together – easily backed up, shared with other authors, and kept separate from other versions or works in progress.
With regards to the scenery system, the idea of orthogonality is this: each part of the scenery system should be unaffected by other parts, such that they can be used together in any combination. This allows you the most options in creating scenery for a given set of features.
One way we try to achieve this is by only having one way to implement any particular feature. For example, because it is possible to place custom objects in overlay DSFs, we are unlikely to allow custom objects to be placed via the apt.dat file. Having two ways to place custom objects would invariably lead to subtle differences in how the two worked; not all features would be usable everywhere. It would also mean more code (read: longer time before you get to use the feature) and more subtle bugs.
Once we have a feature only available in one form, we can try to use that mechanism everywhere. For example, objects can be animated. Because any object can be animated and objects are used both for cockpits and scenery, the work we did to provide animation allows for animation in both scenery and the cockpit. When we get cars back on the road, they’ll probably be objects; navaids are also objects now, and someday hopefully ships and oil platforms will be too. Each time we recycle OBJ8 for a part of the system, animation becomes possible in a new domain.
So usually the ‘orthogonality’ strategy, code something once and use it everywhere with no duplication, means more flexibility and features being usable over a wider area. Every now and then though it bites us in the ass. Flexibility is a strength and weakness; the more we allow any part of the scenery system to work with any other part, the more likely two parts will be used together in a way that we never expected, potentially revealing bugs.
Here’s an example that I only realized existed today, after talking to Peter about animation: it turns out that if you use a textured light inside an animated object, the light (which is secretly a flat square that always points at the camera) will actually turn with the object, revealing that it is flat, and being invisible most of the time (either because it’s facing sideways and is very thin, or because it’s facing in the other direction entirely).
The problem is: when working on animation, I forgot about the textured light system and forgot to cope with the case where they interact. When the code is truly separate, an orthoganal design works well; where the code has internal interactions, it does not.
Do you care? Well, we are looking at some new features for the textured lighting system in 850; there is probably going to be some fine print about how you can use the new textured lights with animation. Even though the features are orthogonal, the code that runs it is interdependent, so some cases may not work right without extra work. How many strange cases we fix, and how many we say “just don’t do that” for will depend on how useful they really are. For example, not being able to have lights on animated objects at all would be a real limit, but perhaps not every kind of light needs to be usable within an animation. For example, VASI lights are usually just mounted on the ground.
I’m back, but things will probably be quiet a bit. A lot of new scenery features will be going into 850, but it may be a while before 850 is ready for test due to how much is going in. I’ll post more as I know it – my work for 850 will be mostly focused around improving the look of airport areas.
I’ll be out of the office for the next eight days; please be patient with bug reports and scenery questions. An X-Plane RC should come out soon and we’ll let it sit a bit to see how it does. I think we got the fog bug fixed!
In version 840 I tried to fix a persistent bug in X-Plane where objects sometimes randomly disappear…turns out I got the math wrong.
The next 840 beta (which I think will be called “RC1”, but I am a poor predictor of these things) has a bug fix to the bug fix. I don’t know how many of these bugs this will fix (some appear to be due to driver issues) but I think we’ll be closer.
I try to avoid politics in this blog, but this article that Jonathan Harris posted to the scenery list is pretty topical to X-Plane.
Of course I am biased – low-cost high quality data (SRTM, Tiger, etc.) make it possible for a small company like X-Plane to create high quality scenery. But I would also add two more thoughts:
1. Knowledge is crucial to the healthy functioning of a democracy. Without good digital maps, you can’t apply modern tools to understanding regional trends, environmental problems, electoral districts, etc. Knowledge shouldn’t just be for the rich (or big companies).
2. Geospatial data is infrastructure – free or cheap GIS data make a country a better place to do business. I think in this global economy every national government needs to be asking “what are we doing to make sure our citizens will be competitive on a global level.” There’s no question that having good roads and transportation infrastructure are important for a national economy, but I would submit that having good digital infrastructure will become important too; in that context it’s appropriate for a government to make the data free (and take the local loss in the providing agency) to foster growth, employment, and the well-being of its citizens.
Just my 0.02…
Long time no post – it’s been a busy few weeks, helping friends with projects and traveling. There have been some lively discussions on the scenery list though!
I’ve been tossing around the idea for a fundamental change in the way the apt.dat format handles metadata. It’s subtle but I think it has some far-reaching implications.
Right now the apt.dat format is pretty high level. Let’s use taxiways as an example: a taxiway has a light code that indicates whether the taxiway has blue edge lights. This has two effects:
1. The information about whether the taxiway has lights is preserved.
2. You can’t put blue edge lights down without a taxiway.
I am exploring the idea of entirely separating edge-lights/lines* from taxiways. This would mean that:
1. We’d have no idea about whether the taxiway has lights. There may happen to be lights nearby, but we can’t be sure of where they come from and
2. We can put lights anywhere.
The shift is fundamental because it is reducing the amount of high level data and increasing file size in return for gaining flexibility with a simple system.
Anyway, email me if you have thoughts on this…food for thought.
* The exception will be runways. We will be supporting all existing data in the apt.dat format – nothing’s geting dropped, and there isn’t a real replacement for runways. Generally runways will still be built by saying “this is a runway and these are it’s many properties” and letting the sim do the rest.
X-Plane 8.40b2 introduces the ability to attach multiple objects to the aircraft. This is a very powerful new feature.
Currently these objects are not part of th physics model. This may change during the beta, or not. Please do not attempt to do real work based on the beta – an X-Plane beta is a test version of the sim and may change radically due to bugs we discover. Until X-Plane is final please try features and report bugs, but do not assume the features will continue to work the same way.