X-Plane 860 introduces a fundamental change in the way the keyboard is handled in X-Plane.
Before X-Plane 860, keyboard keys are matched to their commands by looking at the character that is prodcued by typing.
For example, spot view is produced by the ‘@’ symbol. On a US keyboard that means shift-2. On an Italian keyboard, the @ symbol is shift + the ò key (ò on its own key, cool!).
But in X-Plane 860, the matching is done on the key itself, not the symbol you get when typing. In other words, the 2 key is the 2 key no matter what. So in X-plane 860, spot view is mapped to shift-2 and shift-2 will induce spot mode whether this gives you @ or ” when typing (on Italian keyboards, shift-2 gives the double-qoute).
One advantage of this method is that the keys stay in roughly the same place…the point of those strange punctuation keys for autopilot modes is that on a US keyboard they’re all top-of-the-keyboard functions. Now that the keyboard bindings are based on the number keys plus the shift key, this will be true on international keyboards too.
One disadvantage of the new system is that if there is no “primary” key for a symbol, international keyboards can’t use it. For example, ; is a key on a US keyboard, but is made by shifiting the , key on an Italian keyboard. So right now in beta 2 Italians can’t invoke free-view (ctrl-;) at all…if they type ctrl-shift-, they get, well, ctrl-shift-,. (Of coures, mapping free view to ctrl-shift-, in the keys file does work.)
For now please just keep in mind that 860 is a beta and is still a work in progress. In the long term once we get the quirks worked out, the new keyboard system offers a lot more mapping combinations, because everything can be mapped with any combination of modifier keys. No more running out of keyboard combinations!
Also for hackers, try X-Plane –cmnds=all to see a list of all possible command bindings in the sim.
X-Plane 860 uses GLSL pixel shaders. But this doesn’t mean you need any new hardware. It does mean you might need to update your drivers. Let’s break this down a little bit.
X-Plane never talks directly to your video card. It always talks to the video driver, a piece of software that acts as a translator between X-Plane and your video card. (That’s a gross simplification – video drivers do a lot of amazing things these days.)
Your video card has certain capabilities – different cards can do different tricks. The video driver tells X-Plane what tricks the video card can do.
So there are really four possibilities for any given trick:
- Your video card cannot do that trick. The video driver will tell X-Plane “I can’t do it.”
- Your video card can do that trick. The video driver will tell X-plane “I can do it”.
- Your video card can do the trick, but the driver is old and will tell X-Plane “I can’t do it” because the driver doesn’t realize what the card can do. (This can happen because the video driver you use might come with your operating system, before you bought the new video card.)
- Your video card cando the trick, but the driver has a bug. The driver will tell X-Plane “I can do it”, but when X-Plane says “go and do it”, the video driver will crash your whole computer. This happens when drivers are buggy and can usually be fixed by getting newer drivers.
Something to note about this last case – we only crash if X-Plane actually says “go and do it”. So when we release a new version of X-Plane that utilizes new video driver tricks…
- Some users have cards that can’t do it and will see no change.
- Some users have cards that can do it, so they see the new feature.
- Some users have cards that can do it but old drivers and see no change.
- Some users have buggy drivers and see a crash for the first time.
GLSL shaders is a certain way to use the pixel shaders built into some cards. (GLSL is an interface and language to talk to those shaders.)
GLSL shaders are definitely not required for X-Plane 860. If you don’t have them, X-Plane will run the way it always did.
But if your drivers are buggy and crash when GLSL shaders are used, you will need a software update to get good drivers. (There is a way out of this…if you really can’t get a driver fix, there are command-line options that will tell X-Plane “don’t use this feature even if the card has it”.)
Traditionally it always has been real memory – system RAM or VRAM – that’s been most important to the flying equation. But recently virtual memory has been a scarce resource. What happened?
To understand this situation, consider X-Plane 6 vs 8. Back in the X-Plane 6 days, a typical system might have 256 MB of system RAM, 16 MB of VRAM, and 2 GB of virtual memory* for X-Plane. That’s a ratio of 8:1 virtual to real RAM and 128:1 virtual to VRAM.
These ratios are actually pretty useful. At any given time only part of the loaded scenery is visible, so only part of it needs to be in real RAM/VRAM. So having that extra virtual memory lets us load a big chunk of scenery at a time, rather than having a lot of complex details to load tiny bits of scenery.
Today’s computers are a bit different. 2 GB of RAM is not uncommon…that’d be a ratio of 1:1! And most gamers cards have 256 MB of VRAM now for an 8:1 ratio with VRAM. 512 MB cards are coming out and nVidia’s monstrous 8800GTX comes with (gasp) 768 MB of VRAM! Insanity!
The problem is: RAM and VRAM are getting larger, but virtual memory is not! This means the ability to have the part of the scenery you don’t see in virtual memory is getting more and more difficult.
Why can’t we get more virtual memory? Well, the problem is to get more virtual memory we need 64-bit CPUs, 64-bit operating systems, 64-bit drivers, and 64-bit applications. That’s a lot of parts that all need to be 64 bit and means a full computer upgrade and motherboard upgrade. Of the 5 machines Apple sells right now, only one is 64-bit, so even some of today’s fastest machines aren’t 64 bit. That’s slow adoption.
So in the meantime we’re adjusting X-Plane’s strategy to deal with increasingly limited virtual memory. Back in X-Plane 6 it was acceptable to leave all objects their textures cached in virtual memory for later use (which sped up reloading of scenery).
X-Plane 860 (coming soon to a beta near you) will change this – unused textures and objects will be fully purged, which will hopefully address the problem of running out of virtual memory during long flights over lots of different custom scenery.
* 2 GB? Well, technically an application can have 4 GB of virtual memory, but the OS keeps some for itself. Some OSes take 2 and leave 2, some take 1 and leave 3, but even if we had all 4 GB of virtual memory for application use (and we don’t – the graphics driver steals some too) we’ll have 4 GB machines soon, so virtual memory is disappearing.
The order objects are listed in a DSF file (either the OBJ definitions or the actual OBJ instances) is not the same as the order X-Plane will draw them! X-Plane will change the order to maximize frame-rate, and the way it organizes them has changed in 850 (and will change again in futur versions).
Until X-Plane 850 there was no way to control the order of objects (although some packages do rely on X-Plane 840’s behavior and now look funny in 850). But with 850 there is a clean way to control draw order that will work well into the future.
Starting with X-Plane 850, there is a new command ATTR_layer_group…it let’s you override what part of the rendering process your object is drawn in, which is useful in a number of cases. Here’s how it works:
X-Plane draws the scenery in a series of steps or “layer groups”. Because of the way OpenGL transparency and polygon-offset affect Z buffering, X-Plane must draw the terrain first, all decals over the terrain second, 3-d buildings third, and anything translucent (clouds, smoke puffs, etc.) last. Layer groups organize all of the scenery into these catagories.
Objects are normally drawn in the “objects” layer group, which is intended for 3-d buildings, static airplanes, jetways, and anything else you might model with an OBJ. However ATTR_layer_group will let you specify that some objects be drawn earlier or later than others.
The command’s syntax is:
ATTR_layer_group objects 1
Where the name (“objects”, but others are allowed – see the OBJ spec) specifies a group, and the number makes the object be drawn after (positive numbers) or before (negative numbers). Objects with the same group and number are drawn together (in an unspecified order), but X-Plane goes by the numbers.
So for example if you want to ensure that a transparent hanger is drawn after a static plane, use ATTR_layer_group objects 1 on the hanger to draw it after all other objects. To draw some apron lines before the terminal, use ATTR_layer_group objects -1 to draw before buildings.
A few other tips:
- Try to use as few offsets as possible – e.g. try to put all “draw last” objects into the group objects/1 – rather than using a different number for each.
- Avoid using layer groups when they are not needed – they inhibit X-Plane from optimizing framerate.
- If you do not use a layer group, but do use polygon offset, X-Plane sets the layer group to objects with a negative number. This is inhibited when you specify the layer group yourself, so be sure to use a negative offset layer group for polygon offset!
- Try to organize your textures so that one texture is not used in multiple layer groups. For example, if you have translucent buildings that must be drawn later, group them in their own texture.
- Related – try to keep polygon-offset geometry (pavement markings) in their own texture and objects. Share one texture for all markings, but don’t put the markings and object together.
X-Plane 850 was the first version of X-Plane to start employing pixel shaders — they make the new hardware-accelerated lights possible. Our next step will be to start converting existing OpenGL code over to pixel shaders.
DISCLAIMER: these are my predictions for the future of X-Plane, not promises or guarantees. Do not base your purchasing decisions on what I say, and do not come back quoting this blog saying “you promised X”. This is a statement of our intentions, but we cannot predict what limitations we could hit in the future, such as graphics problems we did not anticipate!
Right now X-Plane works based on individual selection of features – that is, each separate part of X-Plane examines our summary of a driver’s capabilities and does what it can based on those capabilities. The result is good for users in that X-Plane’s visual features turn off one-by-one as a card becomes less capable, leaving as much eye candy as possible.
What’s bad about this technique is that there are huge combinations of possible configurations, more than we can test individually, so we sometimes have bugs that show up only with a peculiar combination. Since most of the code that uses this driver spec has been relatively stable we haven’t seen a lot of bugs due to this in 830-850, but I definitely fixed a lot of configuration-related OpenGL bugs in the early v8 run when the rendering engine was new.
Pixel shaders (we use GLSL for the OpenGL nerds out there) will change that; the OpenGL shader spec is powerful enough that if we have it, we can do everything we want. So the new world of shaders will make X-Plane’s visuals be either “you have it” or “you don’t”. This definitely helps us for testing – we can test the shader version, the legacy version, and that’s it.
Remember the dark old days of X-Plane 6 when GeForce 4 MX’s had two pixel shaders, but GeForce 4 Ti’s had 4, and ATI cards had 2,3,4 or 6? Fortunately we’re past that. The last three generations of video cards have pixel shaders and every card with shaders has 16 texture units, which is more tha enough for us.
Here are the cards that will support shaders, at leaset I think from the specs posted on the web (all suffixes have equal pixel shader capabilities – the suffix usually is about the speed of the card):
- ATI Radeon 9500-9800.
- ATI Radeon X300-x850.
- ATI Radeon X1000-X1900.
- nVidia GeForce 5200-7900.
- Any card that hasn’t been released yet will surely support shaders.
Here are the cards that X-Plane will run on but don’t support shaders:
- ATI Radeon 7000-8500.
- GeForce 2,3, and 4.
- Intel GMA950 (and probably a few other Intel chipsets I don’t know abot).
I don’t know about the old Rage 128 or original GeForce 256 – technically they probably meet minimum requirement, but they’re so underpowered you wouldn’t want to run X-Plane 8 on them. Also I don’t think they ever came in 16 MB variants. I don’t know about Matrox’s compatibility.
To use pixel shaders you will also need reasonably new and modern drivers. Both ATI and nVidia ship modern drivers that support shaders fully, but very often what you get on your machine or with Windows XP does not. So if you have a card that supports shaders but don’t see them, it might be time to check the vendor website.
For Mac users Apple shipped full pixel shader support in 10.4. So: if you have a card with pixel shaders and you use 10.3.9 you should upgrade to either 10.4 now or 10.5 when it comes out! Otherwise x-Plane will not have the right driver to utilize your hardware! An X850 or 7300 on OS 9 is definitely a waste of potential. (X-Plane 8.50 already has features that are only available to pixel-shader users on 10.4!)
I am excited about pixel shaders because we’ll be able to finally fix some of the bugs we just couldn’t fix before. For example, when you turn on the landing light, X-Plane’s ambient lighting levels drop to pitch black. With pixel shaders we will finally have the capability to fix this!
Jonathan Harris sent me a test package showing how that 850 betas misplace objects at high altitude. The pyramid is 1000 meters up, and the object should be right on top of it. (Mind you DSF has a little bit of rounding error; the amount depends on the smartness of the DSF writer. DSF2Text isn’t that smart, so I can’t promise it’s better than half a meter anyway.)
The problem is that X-Plane figures out how high the ground is based on a cartesian coordinate system, not based on true round world math. This is not a problem as long as the ground and the point you are looking at the ground from are not that far apart vertically. (This is why this does not represent a physics bug; as the plane gets closer to the ground, the error in terrain height, which is also a function of slope and is thus probably zero on a good runway, gets smaller and smaller until we are dead-on at the instant the wheels touch down.)
The problem is that DSF objects do not have elevation specified in the DSF file; they are “dropped” onto the terrain. In order to drop them and know their elevation, we must have their location in cartesian coordinates, and that requires knowing their elevation. Doh! A paradox.
X-Plane 850 used to simply pick an elevation of 0 to start out the process. This leads to a lot of error at high elevation areas away from the center of mapped scenery. X-Plane 850RC4 “solves” this problem (temporarily until I can come up with something better) by dropping the object twice. The second drop is done using the height of the first drop to redo the conversion from lat-lon to cartesian coordinates. Since the first drop is only off by a little bit (a few meters usually) the error introduced in the second drop is very little.
Here’s a comparison of placement in 850RC3 and RC4 (coming to a web page near you soon I hope). The object is 5 meters tall and 1 meter across. While RC4 is still not perfect, the error should be small enough that you won’t have to hack your scenery package to get good object placement.
One of the less fun aspects of X-Plane 8.50 was dealing with driver bugs. The basic problem is that PC hardware vendors have a relatively slow cycle for updating the software they bundle with their machines; while the companies that write drivers (ATI and nVidia) have a pretty high latency from when they first ship a card to when the driver support is to the level that X-Plane needs.
The clearest case of this I saw during 8.50 was in the case of PC users with “1.5”*-type ATI drivers. These drivers will cause the terrain textures to swim when hardware-accelerated runway lights are used. Upgrading to “2.0”-type ATI drivers fix the bug. A lot of users were just fine with the 1.5 drivers that came with their machines until X-Plane 8.50 came out, and have only now updated.
8.50 started using pixel shaders; hence the need for newer drivers with bug-free pixel-shader support. But in the future we’re going to move from “DSP”-type shaders (that is, shaders written in a very low level language) to shaders writen in a GLSL, a high level abstract way to write shaders.
My fear is that this will induce a second round of “update your driver” for PC and Linux users. (Apple ships new drivers with system update, so users of the current Mac OS can usually just run the updater if it isn’t set to update automatically.) But I think it’s probably for the best that people use the latest drivers, as ATI and nVidia put software changes in the drivers that increase the throughput of the graphics card, through smarter use of the hardware.
The one group that is really stuck is Macintosh 10.3 users; Apple doesn’t provide driver updates to older operating systems. So for example users with 10.3.9 and an ATI card with pixel shaders can’t get hardware-accelerated texture lights without buying OS X 10.4. I would go as far as saying that in the future if you have a pixel-shader card it might be worth buying an OS upgrade to unlock the driver support needed to use these features.
Why move to GLSL (and require more driver updates)? In the long term it will provide superior framerate. The current shaders we have now are written in a very low level language, that roughly matches the technology of the first generation pixel-shader cards (Radeon 9700 and GeForce “FX” series). As cards gain new capabilities, Austin and I can’t take advantage of the new capabilities using this old language.
GLSL however is a higher level language; when we write a shader in GLSL, the graphics driver translates our shader into code specific to the video card at hand when X-Plane starts. This means that if the video card has some new magic trick it can do, the driver can take advantage of that trick as it translates our shader, resulting in higher framerate. And everyone loves higher framerate. 🙂
EDIT: who will be able to use GLSL? Basically any card that has hardware-based pixel shaders** from ATI or nVidia will support GLSL-based shaders if you have a modern enough driver. So if you are seeing hardware-accelerated runway lights in 850 then the GLSL changeover shouldn’t cause any more fuss than a driver upgrade if you haven’t done one rcently.
Also I should say that if I sound like I am dumping on ATI and nVidia for driver bugs, I shouldn’t be; video cards are amazingly powerful, the drivers are very complex, and the technology jumps by huge amounts in a very short time period. I believe the level of driver bugginess we see is a function of fast technology rollout and not the competence of the driver teams, who I am sure are working as hard as they can.
So we, the developers who write programs that use these cards, and you, the users who enjoy these cards, have to ask whether we’d rather have faster framerate and more eye candy or less chaos from our drivers.
*ATI drivers are numbered by ATI to match ATI’s internal versioning scheme. But X-Plane can’t really see what the ATI driver version number is (e.g. the catalyst version number). So when I say 1.5 or 2.0 I am really talking about the OpenGL specification number that the driver implements. ATI implements newer specs in newer drivers, so we can tell that an ATI driver that supports the 2.0 spec is newer than an ATI driver that supports teh 1.5 spec.
So while I have no idea of either which versions of the ATI drivers had this bug or which version it was fixed in, but we can tell from an X-Plane Log.txt file whether the drivers are going to have this bug.
**Hardware-accelerated shaders? Well, previously I’ve described a graphics card as either having shaders or not, but this isn’t totally accurate. Technically a shader can include a non-programmable shader. Furthermore the driver can present software emulation of a shader. For example, the Intel GMA950 has programmable pixel shaders in hardware. On a Mac the driver reports that it has vertex shaders, but the vertex shadings is actually done by the CPU. (This isn’t unreasonable — all vertex processing is done by the CPU on the GMA950.)
So really when we talk about “cards with shaders” we mean: any nVidia card with a number of 5000 or higher (plus some Quadros, sorry I don’t know the numbres on those), and any ATI card with a number of 9500 or higher, and any ATI card with an “X” in it, e.g. X850. At this point we’ve had shaders in cards for 3 generations, so if you buy a new computer and it has an ATI or nVidia graphics card, it should have fully programmable hardware shaders.
Of course now not all shaders are equal; the newest cards have even more powerful shaders. But this isn’t likely to matter a whole lot to X-Plane for a while. But generally I’d recommend buying the mid-range latest generation cards over high-end obsolete cards at this point.
I just fixed a bug in the mesh crack-patching code. This is a tricky subject, so here’s some information before you file bugs:
Two separate scenery tiles (whether DSF or ENV) may not line up perfectly when placed edge-to-edge. The most common reason is because their elevation data can come from different data sources, but in a few cases files from the same render can have alignment problems.
(In the case of the global scenery, if we have special circumstances like water or airports that must be flat and the features span a tile boundary, the flattening may not work the same way in both tiles. The scenery is also rendered on blocks on different computers, and the blocks may not tile perfectly.)
Whether the error is due to a bug or just two scenery packs together, the sim tries to do its best to fix any gaps in the terrain mesh. (The sim does not try to fix different textures that touch; this is beyond the sim’s capabilities of analysis.) X-Plane will take whichever file is loaded most recently and try to realign its edges to match the old edge. When this works right, you can’t see a seam. More importantly, if an airport spans that border, you can drive over the border (on a runway) without the plane hitting a bump.
I just fixed a bug in the crack-patching code where it would sometimes incorrectly fix a patch. So if you see a ‘crack’ in the terrain (basically you would see the sky color showing between terrain triangles as a sliver of light blue or grey) please report it as a bug, but be sure to include the following information:
- Please include a log.txt file so we can see what you have installed. Be sure to quit x-plane before emailing the log.txt file.
- Please include a screenshot of the crack. Turn on the framerate, plane lat/lon and camera position datarefs.
- Please set your sim resolution to 1024×1024 or smaller and send the original PNGs. Please do not crop, process, or compress them. Please do not send JPEGs.
- Please tell us the nearest ICAO airport so that we can easily go to this location.
As always our bug reporter can be found on a link from //www.x-plane.com/contact.html.
Some crack bugs may be due to a scenery defect (to be addressed the next time we make new scenery), some may be due to sim bugs, and some may be due to the combination of two scenery packages next to each other.
EDIT FROM 2/19/07: this article mentions some bugs with light visibility from X-Plane 850 betas. These bugs were fixed, and the mechanism of detachment is entirely internal to X-Plane. So please bear in mind when reading this that while this describes the internal engine of X-Plane, you as an author don’t need to DO anything special for lights.
One user reported that the taxiway lights are visible a very long way away from the airport. This is true, but it is probably not a cause for concern. By comparison, the airport beacon has a way of disappearing – hopefully less so in beta 11. Here’s what’s going on:
(When I say “light” in this blog post, I mean the little ball of color that is supposed to simulate the look of a light bulb. “Fixture” refers to the 3-d object modeling the real-world device that holds that bulb in place. The actual rays of light cast are not simulated – that’s why the approach lights don’t illuminate part of the plane no matter where you fly. Current graphics hardware just doesn’t give us that ability yet.)
In X-Plane 850 all “lights” are made via OBJ8 objects. The OBJ8 object may contain both 3-d geomemtry that forms the fixture, and one or more “lights”, created via either the LIGHT_NAMED or LIGHT_CUSTOM command. Objects may also have multiple versions based on LOD. If you don’t use ATTR_LOD, a single LOD range is assigned by X-Plane based on how big your object is. (Bigger objects will need to be visible farther away.)
Now here’s where it gets weird. Some of the lights in an object are “detached” by X-Plane from the OBJ and stored separately in the scenery. We do this for performance – the detached lights can be fed to the graphics card via a different mechanism that is much faster than regular OBJ drawing. One of the effects of light detachment is that the detached lights are no longer limited by the LOD of the object. They instead are drawn to much further distances. The fixtures of the objects, however, are never detached…thus some of the airport light OBJs in X-Plane are only visible to 500 meters. We can get away with such a low distance because the light bulb itself is detached and will remain visible.
So which lights are detached? Well, it depends on a number of factors – lights are only detached if they are not subject to animation and they are simple enough to be drawn in bulk. Which lights are these? It’s hard to predict.
So the airport beacon and taxiway light work differently; the taxiway lights are very simple and are detached – hence they are visible a long way away. The airport beacon lights are animated (the light bulbs rotate with the fixture), and are never detached. Thus the airport beacon is subject to LOD constraints and taxiway lights are not.
If you set the world level of detail setting to “low” or “very low” it simply reduces the LOD ranges of all objects. Thus on “very low” the airport beacon may be seen to disappear before the (detached) taxiway lights. For beta 11 I have tried to set the airport beacon LOD large to be enough that even on such low settings the beacon will be visible farther away.
There is one more piece to the puzzle: all lights become dimmer with distance. So in theory our hope is that the lights will become fully dim due to distance before we stop drawing them with LOD, producing a gradual fade-out rather than a sudden disappearance. But if we don’t tune all of these parameters right, lights can randomly appear to disappear in the distance instantaniously.
If there’s a moral of the story, I’m not sure what it is, except perhaps: this new light system is very new, and I am sure our artists will tune it a bit over the next few versions of X-Plane, helping to hide these implementation artifacts.
EDIT FROM 2/19/07: during X-Plane 850 when I was developing the new lighting system (particularly used for airports, but also for the rest of scenery) I brought up the subject of “detachment”, a internal process by which the X-Plane rendering engine processes lights.
Simply put: detachment is virtually never a factor for scenery authors. Back when I wrote the article, it was possible for an author to make a few special optimizations if he or she knew about detachment. This is no longer true! The sim will correctly optimize all lighting cases as much as possible without any special help from the author.
So….the blog article in its original form follows and may be useful as an insight into how X-Plane works, but as an author, just remember: you don’t need to do anything – just make nice OBJs.
I think I’ve confused just about everybody in the last few days – especially Sergio and Austin – with work on the new lighting system. One reason I’m so scrambled: the internal representation of lights in X-Plane is very different from the commands you can put in an OBJ8 file. So there’s a whole set of terms and ideas that are important to how X-Plane works that I shouldn’t have mentioned to anyone making OBJ8s because they won’t even be part of the file format.
Then there’s detachment. I think I can explain it simply like this: detachment is when X-Plane changes the order that lights are drawn inside an OBJ and the LOD ranges in which they are drawn to improve framerate.
X-Plane 840 does not do detachment. X-Plane 850 will probably not do detachment for objects made only with the old lighting commands. But 850 will do detachment for objects that use new (to 850) lighting commands, and this will allow those objects to render very quickly even with textured lights.
This picture is actually from X-Plane 8.03 I believe. It’s a simple trick: edit the roads file to add street light objects along all roads. Make a street light object with a light command and then turn roads on and watch teh city at night appear.
The only problem is the frame-rate – this experiment brought 8.03 to a standstill.
The goal of detachment is to make textured lights in objects fast enough that this kind of scene can be rendered at reasonable framerates. I do not know if this will be possible, but detachment should certainly improve performance in cases like this.
