Per-pixel lighting is something I hope to have in X-Plane soon. A number of other features will take longer, and quite possibly might never happen. This is the “pie in the sky” list – with this list, we’re looking at higher hardware requirements, a lot of development time, and potential fundamental problems in the rendering algorithm!
High Dynamic Range (HDR) Lighting
HDR is a process whereby a program renders its scene with super bright and super dark regions, using a more detailed frame-buffer to draw. When it comes time to show the image, some kind of “mapping” algorithm then represents that image using the limited contrast available on a computer monitor. Typical approaches include:
- Scaling the brightness of the scene to mimic what our eyes do in dark or bright scenes.
- Creating “bloom”, or blown out white regions, around very bright areas.
Besides creating more plausible lighting, the mathematics behind an HDR render would also potentially improve the look of lit textures when they are far away. (Right now, a lit and dark pixel are blended to make semi-lit pixels when far away as the texture scales down. If a lit pixel can be “super-bright” it will still look bright even after such blending.)
Besides development time, HDR requires serious hardware; the process of drawing to a framebuffer with the range to draw chews up a lot of GPU power, so HDR would be appropriate for a card like the GeForce 8800.
While there aren’t any technical hurdles to stop us from implementing HDR, I must point out that, given a number of the “art” features of X-Plane like the sun glare, HDR might not be as noticeable as you’d think. For example, our sun “glares” when you look at it (similar to an HDR trick), but this is done simply by us detecting the view angle and drawing the glare in.
Reflection Mapped Airplanes
Reflection maps are textures of the environment that are mapped onto the airplane to create the appearance of a shiny reflective surface. We already have one reflection map: the sky and possibly scenery are mapped onto the water to create water reflections.
Reflection maps are very much possible, but they are also very expensive; we have to go through a drawing pass to prepare each one. And reflection maps for 3-d objects like airplanes usually have to be done via cube maps, which means six environment maps!
There’s a lot of room for cheating when it comes to environment maps. For example: rendering environment maps with pre-made images or with simplified worlds.
Shadows
Shadows are the biggest missing feature in the sim’s rendering path, and they are also by far the hardest to code. I always hesitate to announce any in-progress code because there is a risk it won’t work. But in this case I can do so safely:
I have already coded global
shadow maps, and we are not going to enable it in X-Plane. The technique just doesn’t work. The code has been ripped out and I am going to have to try again with a different approach.
The problem with shadows is the combination of two unfortunate facts:
- The X-Plane world is very, very big and
- The human eye is very, very picky when it comes to shadows.
For reflections, we can cheat a lot — if we don’t get something quite right, the water waves hide a lot of sins. (To work on the water, I have to turn the waves completely off to see what I’ m doing!) By comparison, anything less than perfect shadows really sticks out.
Shadow maps fail for X-Plane because it’s a technology with limited resolution in a very large world. At best I could apply shadows to the nearest 500 – 1000 meters, which is nice for an airport, but still pretty useless for most situations.
(Lest someone send the paper to me, I already tried “TSM” – X-Plane is off by about a factor of 10 in shadow map res; TSM gives us about 50% better texture use, which isn’t even close.)
A user mentioned
stencil shadow volumes, which would be an alternative to shadow maps. I don’t think they’re viable for X-Plane; stencil shadow volumes require regenerating the shadow volumes any time the relative orientation of the shadow caster and the light source change; for a plane in flight this is every single plane. Given the complexity of planes that are being created, I believe that they would perform even worse than shadow maps; where shadow maps run out of resolution, stencil shadow volumes would bury the CPU and PCIe bus with per-frame geometry. Stencil shadow volumes also have the problem of not shadowing correctly for alpha-based transparent geometry.
(Theoretically geometry shaders could be used to generate stencil shadow volumes; in practice, geometry shaders have their own performance/throughput limitations – see below for more.)
Shadows matter a lot, and I am sure I will burn a lot more of my developer time working on them. But I can also say that they’re about the hardest rendering problem I’m looking at.
Dynamic Tessellation
Finally, I’ve spent some time looking at graphics-card based tessellation. This is a process whereby the graphics card splits triangles into more triangles to make curved surfaces look more round. The advantage of this would be lower triangle counts – the graphics card can split only the triangles that are close to the foreground for super-round surfaces.
The problem with dynamic tessellation is that the performance of the hardware is not yet that good. I tried implementing tessellation using geometry shaders, and the performance is poor enough that you’d be better off simply using more triangles (which is what everyone does now).
I still have hopes for this; ATI’s Radeon HD cards have a hardware tessellator and from what I’ve heard its performance is very good. If this kind of functionality ends up in the DirectX 11 specification, we’ll see comparable hardware on nVidia’s side and an OpenGL extension.
(I will comment more on this later, but: X-Plane does not use DirectX – we use OpenGL. We have no plans to switch from OpenGL to DirectX, or to drop support for Linux or the Mac. Do not panic! I mention DirectX 11 only because ATI and nVidia pay attention to the DirectX specification and thus functionality in DirectX tends to be functionality that is available on all modern cards. We will use new features when they are available via OpenGL drivers, which usually happens within a few months of the cards being released, if not sooner.)
Traditionally, a pilot’s priorities are: aviate, navigate, communicate.
But that might not be true for X-Plane for the iPhone.
It’s real! And it pretty much is X-Plane – there really are OBJs and DSFs in there, as well as an ACF model, all tuned for the iPhone.
In the next few posts I’ll blog a little bit about the impact of doing an iPhone port on scenery development. The iPhone is an embedded device; if you go digging for system specs you’ll see that it’s a very different beast from the desktop. The porting process really helped me understand the problems of the rendering engine a lot better, and some of the techniques we developed for the iPhone are proving useful for desktop machines as well.
Yesterday I went off on a rant about how the OpenGL 3.0 spec doesn’t matter a lot because OpenGL grows via a la carte extension. And I mentioned that this creates a dilemma for anyone designing a rendering engine that has to run on a wide range of hardware, with very different underlying capabilities.
Back in the old days of X-Plane 6, there wasn’t a lot of variety in hardware. Some cards were faster than others, but the only real difference in capability was how many texture units you had. X-Plane’s job was simple…
- First we have a runway texture.
- Still got a second texure unit? Great! Skid marks!
- Still got a third texture unit? Great! Landing light!
- Got another one? Etc…
Other than texture stacking, there wasn’t much to do.
Since then the rendering engine has become a lot more complex, as have OpenGL cards. To try to keep the combinations down, I tried to use a “bucketing” strategy for X-Plane 9. The idea of bucketing is to group cards into major buckets based on whole sets of common functionality, so that we only have to test a few configurations (the “low end” bucket and “high end” bucket), rather than having to test every single card for its own unique set of features.
The obvious bucketing choice was pixel shaders – given a card with shaders and a few other features, we can render all of the new effects. A card without shaders basically gets X-Plane 8.
So what went wrong? Driver compatibility, that’s what. Ideally we don’t want to allow every single underlying rendering engine feature to be turned on or off individually because the combinations are uncontrollable.* But in practice, being able to turn features on and off is necessary to trouble-shoot drivers that don’t always do what we expect them to.
With the GeForce 8800 and Radeon HD cards, there is a potential third bucket for DirectX-10 capable cards, capable of significantly more advanced pixel shading effects. But time will tell whether we can actually make a bucket, or we have to look at each feature individually. My suspicion is that even if we organize the shipping features into buckets, we’ll have to support a lot of combinations under the hood just to trouble-shoot our own application.
*Example: cross a standard vs. oversized panel with the presence or absence of advanced framebuffer blending, crossed with whether render-to-texture works. That’s 8 possible ways just to render the 3-d cockpit panel texture. Cross that with panel regions and 3-d cockpits and the new panel spotlighting and you have 64 configurations. Ouch!
A few people have asked me about OpenGL 3.0 – and if you read some of the news coverage of the OpenGL community, you’d think the sky was falling. In particular, a bunch of OpenGL developers posted their unhappiness that the spec had prioritized compatibility over new features. Here’s my take on OpenGL 3.0:
First, major revisions to the OpenGL specification simply don’t matter that much. OpenGL grows by extensions – that is, incremental a la carte additions to what OpenGL can do. Eventually the more important ones become part of a new spec. But the extensions almost always come before the spec. So what really matters for OpenGL is: are extensions coming out quickly enough to support new hardware to its fullest capacity? Are the extensions cross-vendor so that applications don’t have to code to specific cards? Is the real implementation of high quality?
So how are we doing with extensions? My answer would be: “okay”. When the GeForce 8800 first came out, the OpenGL extensions that provide DirectX 10-like functionality were NVidia-specific. Since then, it has become clear that all of this functionality will make it into cross-platform extensions, the core spec, or some of each. But for early adopters there was a difficult point where there was no guarantee that ATI and NVidia’s DirectX 10 features would be accessible through the same extensions.
(This was not as much of an issue for DX9-like features, e.g. the first generation of truly programmable cards. NVidia had a bunch of proprietary additional extensions designed to make the GeForce FX series less slow, but the basic cross-platform shader interface was available everywhere.)
Of more concern to me is the quality of OpenGL implementations – and for what it’s worth, I have not found cases where a missing API is standing between me and the hardware. A number of developers have posted concern that OpenGL drivers are made too complex (and thus too unreliable or slow or expensive to maintain) because the OpenGL spec has too many old features. I have to leave that to the driver writers themselves to decide! But when we profile X-Plane, we either see a driver that’s very fast, or a driver that’s being slow on a modern code path, in a way that is simply buggy.
Finally, I may be biased by the particular application I work on, but new APIs that replace the old ones don’t do me a lot of good unless they get me better performance. X-Plane runs on a wide range of hardware; we can’t drop cards that don’t support the latest and greatest APIs. So let’s imagine that OpenGL 3.0 contained some of the features whose absence generated such fury. Now if I want to take advantage of these features, I need to code that part of the rendering engine twice: once with the new implementation and once with the old implementation. If that doesn’t get me better speed, I don’t want the extra code and complexity and wider matrix of cases to debug and test.
In particular, the dilemma for anyone designing a renderer on top of modern OpenGL cards is: how to create an implementation that is efficient on hardware whose capabilities is so different. I’ll comment on that more in my next post. But for the purposes of OpenGL 3.0: I’m not in a position to drop support for old implementations of the GL, so it doesn’t bother me at all that the spec doesn’t drop support either.
The real test for OpenGL is not when a major revision is published; it is when the next generation of hardware comes out.
I believe I am getting close to a possible solution for the dreaded “Framebuffer Incomplete” errors – these error messages pop up when X-Plane starts, and you end up quitting.
If you meet these criteria, please contact me:
- You have an ATI card that has shown this error in the past.
- You can put on the latest Catalyst drivers. (I know a lot of you have put on older drivers to work around this.)
- You can run X-Plane 920 RC2.
If you’re in this crew, please email me at my XSquawkBox email address!
The rub is: despite having four machines with ATI cards, I never see this error. So I need to send you a build to get close to a fix!!! Let’s swat this bug for real!
My last post generated number of posts from both sides of the “hardware divide” (that’d be the have’s and have-not’s). I think everyone at least grasps that developer time is finite and features have to get prioritized at the cost of other features, even if not everyone agrees about what we should be coding.
I think the term “hardware divide” is the right one, because the hardware market has changed. Years ago when I bought myself a nice shiny new Dell (back when that wasn’t an idiotic idea) a medium-priced Dell had medium-priced hardware. Not only did I get a decently fast CPU (for the time), but I got a decent AGP bus, decent motherboard, etc. The machine wasn’t top-end, but it scaled.
When you look at any computer market, you need to consider what happens when consumers can no longer accept “more” and instead want “the same for cheaper”. This change in economics turns an industry on its head, and there are always winners and losers. (I have
claimed in the past that operating systems have turned that corner from “we want more” to “we want cheaper”, a shift that is very good for Linux and very bad for Microsoft.)
Desktop computers hit this point a while ago, and the result is that a typical non-gamer computer contains parts picked from the lower end of the current hardware menu. You’re more likely to see:
- Integrated graphics/graphics by the chipset-vendor.
- System memory used for VRAM.
- Slower bus speeds, or no graphics bus.
- GPU picked from the lowest end (with the fewest number of shader units).
- CPUs with less cache (this matters).
Someone commented a few days ago that computers would get more and more cores, and therefore multi-core scalability would be very important to fully utilizing a machine. I agree.
But: how many cores are those low-end PCs, aimed for general use (read: email, the web, text editing) going to have?
My guess is: not that many. Probably 2-4 at most.
These low end PCs are driven by one thing: price – the absence of VRAM or dedicated graphics hardware is all about bringing the hardware costs down – a $25 savings matters! In that situation, box-builders will want the cheapest CPU, and the cheapest CPUs will be the physically smallest ones, allowing for more chips on a wafer. A low-end PC will get no benefit from more than 4 cores – the intended use probably doesn’t even use one.*
Multiple cores are great because they give us a new way to benefit from smaller transistors (that is, by packing more cores on a chip, rather than clocking it faster, which has real limitations). But I think you’ll start to see the same kinds of gaps in CPU count that you see now with GPUs.
(In fact, the mechanics are very similar. The main differences between high-end and low-end GPUs of the same family are the number of parallel pixel pipelines – the low-end chip is often a high-end chip with a few defective pipelines disabled. Thus you can have a 4x or 8x performance difference due to parallel processing between siblings in a GPU family. Perhaps we’ll see the same idea with multi-core chips: build an 8-core chip, and if 4 of the cores fail, cut them out with the laser and sell it as a low-end chip.)
* One advantage of multiple cores is that they can take the place of dedicated hardware. For example, there is no penalty for doing CPU-based audio mixing (rather than having a DSP chip on the sound card) if the mixing happens on a second core. Being able to replace a dedicated component with a percentage of a core is a win in getting total hardware cost down, particularly if you were going to have the second core already.
In my post on 64-bit computing and X-Plane, there’s a point that’s implicit: there is a cost (in development time) to adopting any new technology, and it takes away from other things. I’ve been slow to work on 64-bit X-Plane because it would take away from things like texture paging and generic instruments. Similarly, there is a cost every time we do a build to supporting more configurations, so we pay for 64-bit continuously, by supporting six platforms instead of 3 (3 operating systems x 2 “bit-widths” of 32 and 64 bits).
We have a similar problem with graphics hardware, but it’s even more evil. Moore’s Law more or less says that in a given period of time, computer technology gets twice as fast. In the case of graphics cards, each generation of cards (coming out about every 12-18 months) is twice as fast as the last.
This has some scary implications for X-Plane. Consider these stats for video cards (taken from Wikipedia):
Card Date fill rate Bus Memory bw
GF3 01Q4 1920 MT/S 4x 8 GB/S
GF4 Ti 03Q1 2400 MT/S 8x 10 GB/S
GF5950 03Q4 3800 MT/S 8x 30.4 GB/S
GF6800 04Q2 7200 MT/S PCIe16 35.2 GB/S
GF7900 06Q1 15600 MT/S PCIe16 51.2 GB/S
GF8800 06Q4 36800 MT/S PCIe16 86.4 GB/S
GF9800 08Q2 47232 MT/S PCIe16/2 70.4 GB/S
Let’s assume we support any video card in the last 5 years (in truth we support more than that). The difference between the best card and the oldest in w006 was 13,680 MT of fill rate.
Now in 2008 the difference is 43,432 megatexels per second!
In other words, the gap between the best and worst cards we might support is over 3x larger in only 3 years!
This is no surprise – since cards get twice as fast with every revision, the gap for a given number of generations also gets twice as wide.
What this means for us, programming X-Plane, is that coming up with a single simulator that runs on the very best and worst hardware is becoming increasingly more difficult, as the performance gains at the high end run away.
We get a number of questions about whether X-Plane takes advantage of 64-bit CPUs. The short answer is: “no, not yet”, but here’s the details.
Some of the people who ask are Linux users who haven’t been able to set up 32-bit compatibility libraries for their 64-bit Linux distribution. To me, this is a portability request, e.g. can you make X-Plane work on yet another operating system. I don’t view this as a high priority because you can run X-Plane on a 64-bit operating system using 32-bit compatibility libraries for a number of distros. Linux comes in a million flavors and we can’t support them all.
Other users ask about 64-bit because they have 64-bit CPUs (and possibly paid more for a 64-bit operating system) and want to know when they get some benefit for their money. But…is 64-bits actually useful?
A 64-bit CPU is one that can deal with larger integral numbers – as a result it can access more memory than 4 GB, and it can do math with very large integer numbers more efficiently. 64-bit CPUs also have some architectural differences that are theoretically beneficial.
Is It Faster?
Joshua tried building an experimental 64-bit version of X-Plane for Linux when he first got his 64-bit AMD box and he found…
…no performance benefit at all.
Doh! This actually isn’t surprising.
- X-Plane doesn’t use very large integral number math. X-Plane uses a lot of floating point math.
- The biggest time-sinks for the CPU in X-Plane are talking to the driver and sorting through piles of scenery. The later task requires a lot of logic and memory access — neither of which work any better on a 64-bit CPU.
So the short answer is: you’re not going to get a fps boost from 64 bit. Sorry.
Memory, Memory, Memory
Now 64-bits is useful because a 64-bit app (on a 64-bit OS and 64-bit CPU) can access more than 3 GB of RAM. We’re reaching the point where a lot of users have 4 GB of RAM and a machine that could be using all of it. While I’ve been dragging my feat on the move, someday we will support 64-bits and X-Plane will be able to use more memory.
But in the meantime, I’ve been doing everything I can to reduce the amount of memory X-Plane uses. This is because optimizing memory usage benefits all users, while 64-bits only helps users with a 64-bit OS and a 64-bit CPU.
I believe that we’ll reach a point where 64-bits is useful for users who want to use a ton of add-ons simultaneously. But the out-of-box X-Plane configuration needs to work with much less memory than 4 GB.
For those who posted comments, sorry it took so long to moderate them – for some reason my spam filter decided that notifications of comments are, well, spam, so I just found them now. I should have known people would have jumped into a Vista-bashing thread. 🙂
There is an X-Plane 9.02 beta 1 posted – like 901 we’ve been pretty quiet about this, but you can get it by enabling “get betas” and running the X-Plane updater. Please give it a try. Like 901 it is a small change for the purpose of localization, but it actually has an interesting feature pair:
- True-type fonts and
- Unicode-aware.
This is part of some rework we did to provide better language support. So…you should be able to run X-Plane no matter what weird characters* are in your folder names, name your airplanes funny things, and see diacritical marks. 902 uses a font that provides all of the Latin and Greek/Cyrillic code pages.
Also I have heard reports of improvements based on drivers:
- nVidia has 175.16 drivers out and they apparently address “stuttering” issues. The stuttering issue has been on my list to investigate because it happens under Windows but not Linux. If you have stuttering performance on high-end NV hardware, particularly with forests and Windows, please try 175.16 and let me know how it goes.
- ATI has released Catalyst 8-5. Catalyst 8-3 and 8-4 were causing “incomplete framebuffer” errors for some users, but I was unable to reproduce it (after spending a good day trying to jam Windows XP onto an iMac already crammed with Windows and Linux….yet another episode of a Tale of Three Operating Sytstems). Anyway, at least one user reported the issue as fixed in Cat 8-5, so if you are having problems, please try the new driver set.
As always, bugs in the X-Plane beta should go to our bug report form, on the X-Plane contacts page.
* You might accuse me of being American-centric in decrying diacritical and greek letters weird – but the truth is I am computer-centric…anything that is not in the original ASCII set is weird. 🙂
I’ll warn you in advance: this is going to start off topic and go way off topic. “Catching up” with the changes to Mac OS, Windows and Linux has me thinking about the nature of technology. I feel a little bit guilty about this post because it’s going to turn into a rant about Vista, and ranting about Vista is like shooting fish in a barrel. On the other hand, having used Vista, well, I have a lot of rant to give.
One of the most important things to understand about technology (and computers are no exception) is that changes in the scale of the technology change the very nature of the technology. That is, as you make computers faster and cheaper, at some point the sum of all of those small improvements changes the fundamental nature of the beast. We’ve seen this as the computer transformed from main frame to desktop (which is really just a change in cost and size), finding an entirely new audience, and now again as the computer changes from what we know of now as a computer to cell phones, MP3 players, and other small, mobile devices.
“Commodification” is what happens when, as things get better, cheaper, faster, etc., consumers stop caring about the marginal improvement. Back in the days of Windows 95 and 386’s, there were ways you could improve the operating system and hardware in substantial ways; a doubling in processor speed and a rewrite of the operating system got you protected memory, which meant less data loss.
A few years ago we reached the point where desktop hardware became commodified. For the average user, 1.8 vs 2.2 ghz makes no difference at all. It’s a question of how quickly your computer can wait for keystrokes and data from the internet. (Answer: even a lowly Celleron is light-years faster than the I/O devices it typically has to talk to. Even if you’re the last kid in your class at Harvard, you’re going to be bored discussing politics with a bunch of four-year-olds.) At that point things became very difficult for major vendors like IBM (sold out), HP and Compaq (merged), Gateway (bought out of it’s misery), etc. The price of a desktop plummeted from over $1000 to less than $400.
I believe we’ve reached the point where operating systems have become a commodity as well;
- Every major operating system has all of the features of a “real” operating system – that is, protected memory, virtual memory, plug & play driver support, etc.
- The performance for normal applications is just about the same; there are some specific variations that matter in the server market, but for all practical purposes the operating system is not in the way, and the machine is much faster than users need anyway.
- Every major operating system has a similarly designed GUI experience that, once you get used to the quirks of where the close box is, is just about the same, more or less. (Mac users – keep your pants on. 🙂
And this is why life is not so good for Microsoft. In a non-commodified market, you can charge a premium for incremental improvements over the competition. That’s a game Microsoft can play – they have a lot of capital to invest in changing their operating system, as long as they are rewarded with a lot of cash for doing it. (And normally they are – about six billion dollars for a major OS revision, I’m told.)
The problem is that operating systems are now a commodity. Simply put, users don’t need a new operating system. There are no big ticket features missing from OS X 10.4, Windows XP, or Linux 2.6. This makes Microsoft’s business model fundamentally vulnerable to Linux for the first time. If the name of the game is:
- Keep costs down, as low as possible.
- Incrementally improve quality very slowly without ever causing the pain of a major OS upgrade.
That’s a game Linux, with their army of distributed bug fixers and free source code, is going to win.
When I looked at Windows XP and Ubuntu 6.06 I was afraid that Linux wouldn’t make traction into the desktop market. I blamed the adoption of X11, the KDE/GNOME schism, and the Linux communities’ being made up of Shell nerds for the tolerable desktop experience.
But look where we are now: Vista is a vehicle for bloat. Combine “we make money by shipping major features” and “there are no more major features to ship” and you get Vista…an attempt to change a lot of things when you should have left things alone.*
By comparison, Ubuntu pretty much just works – you put the live CD on your machine, it asks you some questions and installs…it knows about more hardware, has less bugs, more drivers, and a better user experience. In a commodified operating-system space, the only thing to do is try to avoid a bad user experience – if you can’t offer a really juicy carrot to users, try to avoid hitting them with a stick.
And it is in this environment that the Mac is actually gaining market share. Apple’s business model has always been at odds with the industry. Complete vertical integration meant higher costs, lack of market share, and out-of-date technology – back when having more for less meant something, that was a real weakness, and explains why the Mac never dominated in market share.
But what a difference a decade makes! Hardware is now commodified (and Apple is integrated at the system-building level, leveraging cheap third party parts like they always should have). Operating systems are commodified. But on the one frontier left, quality of user experience, Apple’s vertical integration gives it an immense advantage.
The question is: why does an operating system “just work”?
- Vista: it doesn’t. There are too many systems and not enough testers and engineers trying to solve the problem.
- Linux: massive distributed engineering. For any given hardware system, eventually a Linux nerd will integrate it. Anyone can solve the problem of poor user experience.
- Apple: they have it easy. With only half a dozen machines in production (and maybe another two dozen legacy configurations to support) they have a much smaller configuration space to worry about than anyone else.
I don’t kow what Microsoft’s future is, but it can’t be very good. At some point they are going to have to transition from a “major revision for cash” to an “incremental tuning” approach to operating systems. As long as they have market share, they still get the “Windows tax” – that is, their OEM pricing from major vendors on every new computer that is built. It’s going to be harder and harder to convince the entire world to make a major jump (see how well XP to Vista went). In this situation, they’d be better off with a more solid operating system. It’s unfortunate that they’re going to have to try to sustain market share with Vista.
Their best-case scenario is that they eventually get Vista back to an XP-quality experience, in which case all they’ve done is spend a huge amount of R&D; money and pissed off a lot of customers to maintain the status quo.
* I have mixed opinions on Vista’s video-driver-model change. But that’s a different post.
