My last post generated number of posts from both sides of the “hardware divide” (that’d be the have’s and have-not’s). I think everyone at least grasps that developer time is finite and features have to get prioritized at the cost of other features, even if not everyone agrees about what we should be coding.
I think the term “hardware divide” is the right one, because the hardware market has changed. Years ago when I bought myself a nice shiny new Dell (back when that wasn’t an idiotic idea) a medium-priced Dell had medium-priced hardware. Not only did I get a decently fast CPU (for the time), but I got a decent AGP bus, decent motherboard, etc. The machine wasn’t top-end, but it scaled.
When you look at any computer market, you need to consider what happens when consumers can no longer accept “more” and instead want “the same for cheaper”. This change in economics turns an industry on its head, and there are always winners and losers. (I have
claimed in the past that operating systems have turned that corner from “we want more” to “we want cheaper”, a shift that is very good for Linux and very bad for Microsoft.)
Desktop computers hit this point a while ago, and the result is that a typical non-gamer computer contains parts picked from the lower end of the current hardware menu. You’re more likely to see:
- Integrated graphics/graphics by the chipset-vendor.
- System memory used for VRAM.
- Slower bus speeds, or no graphics bus.
- GPU picked from the lowest end (with the fewest number of shader units).
- CPUs with less cache (this matters).
Someone commented a few days ago that computers would get more and more cores, and therefore multi-core scalability would be very important to fully utilizing a machine. I agree.
But: how many cores are those low-end PCs, aimed for general use (read: email, the web, text editing) going to have?
My guess is: not that many. Probably 2-4 at most.
These low end PCs are driven by one thing: price – the absence of VRAM or dedicated graphics hardware is all about bringing the hardware costs down – a $25 savings matters! In that situation, box-builders will want the cheapest CPU, and the cheapest CPUs will be the physically smallest ones, allowing for more chips on a wafer. A low-end PC will get no benefit from more than 4 cores – the intended use probably doesn’t even use one.*
Multiple cores are great because they give us a new way to benefit from smaller transistors (that is, by packing more cores on a chip, rather than clocking it faster, which has real limitations). But I think you’ll start to see the same kinds of gaps in CPU count that you see now with GPUs.
(In fact, the mechanics are very similar. The main differences between high-end and low-end GPUs of the same family are the number of parallel pixel pipelines – the low-end chip is often a high-end chip with a few defective pipelines disabled. Thus you can have a 4x or 8x performance difference due to parallel processing between siblings in a GPU family. Perhaps we’ll see the same idea with multi-core chips: build an 8-core chip, and if 4 of the cores fail, cut them out with the laser and sell it as a low-end chip.)
* One advantage of multiple cores is that they can take the place of dedicated hardware. For example, there is no penalty for doing CPU-based audio mixing (rather than having a DSP chip on the sound card) if the mixing happens on a second core. Being able to replace a dedicated component with a percentage of a core is a win in getting total hardware cost down, particularly if you were going to have the second core already.
With X-Plane 920 RC1, the user can now control how smooth taxiway curves look. More smoothing looks better, but can slow frame-rate.
Below are four pictures of KSBD (which has good, sparse vertices) at the four rendering settings.
In my post on 64-bit computing and X-Plane, there’s a point that’s implicit: there is a cost (in development time) to adopting any new technology, and it takes away from other things. I’ve been slow to work on 64-bit X-Plane because it would take away from things like texture paging and generic instruments. Similarly, there is a cost every time we do a build to supporting more configurations, so we pay for 64-bit continuously, by supporting six platforms instead of 3 (3 operating systems x 2 “bit-widths” of 32 and 64 bits).
We have a similar problem with graphics hardware, but it’s even more evil. Moore’s Law more or less says that in a given period of time, computer technology gets twice as fast. In the case of graphics cards, each generation of cards (coming out about every 12-18 months) is twice as fast as the last.
This has some scary implications for X-Plane. Consider these stats for video cards (taken from Wikipedia):
Card Date fill rate Bus Memory bw
GF3 01Q4 1920 MT/S 4x 8 GB/S
GF4 Ti 03Q1 2400 MT/S 8x 10 GB/S
GF5950 03Q4 3800 MT/S 8x 30.4 GB/S
GF6800 04Q2 7200 MT/S PCIe16 35.2 GB/S
GF7900 06Q1 15600 MT/S PCIe16 51.2 GB/S
GF8800 06Q4 36800 MT/S PCIe16 86.4 GB/S
GF9800 08Q2 47232 MT/S PCIe16/2 70.4 GB/S
Let’s assume we support any video card in the last 5 years (in truth we support more than that). The difference between the best card and the oldest in w006 was 13,680 MT of fill rate.
Now in 2008 the difference is 43,432 megatexels per second!
In other words, the gap between the best and worst cards we might support is over 3x larger in only 3 years!
This is no surprise – since cards get twice as fast with every revision, the gap for a given number of generations also gets twice as wide.
What this means for us, programming X-Plane, is that coming up with a single simulator that runs on the very best and worst hardware is becoming increasingly more difficult, as the performance gains at the high end run away.
We get a number of questions about whether X-Plane takes advantage of 64-bit CPUs. The short answer is: “no, not yet”, but here’s the details.
Some of the people who ask are Linux users who haven’t been able to set up 32-bit compatibility libraries for their 64-bit Linux distribution. To me, this is a portability request, e.g. can you make X-Plane work on yet another operating system. I don’t view this as a high priority because you can run X-Plane on a 64-bit operating system using 32-bit compatibility libraries for a number of distros. Linux comes in a million flavors and we can’t support them all.
Other users ask about 64-bit because they have 64-bit CPUs (and possibly paid more for a 64-bit operating system) and want to know when they get some benefit for their money. But…is 64-bits actually useful?
A 64-bit CPU is one that can deal with larger integral numbers – as a result it can access more memory than 4 GB, and it can do math with very large integer numbers more efficiently. 64-bit CPUs also have some architectural differences that are theoretically beneficial.
Is It Faster?
Joshua tried building an experimental 64-bit version of X-Plane for Linux when he first got his 64-bit AMD box and he found…
…no performance benefit at all.
Doh! This actually isn’t surprising.
- X-Plane doesn’t use very large integral number math. X-Plane uses a lot of floating point math.
- The biggest time-sinks for the CPU in X-Plane are talking to the driver and sorting through piles of scenery. The later task requires a lot of logic and memory access — neither of which work any better on a 64-bit CPU.
So the short answer is: you’re not going to get a fps boost from 64 bit. Sorry.
Memory, Memory, Memory
Now 64-bits is useful because a 64-bit app (on a 64-bit OS and 64-bit CPU) can access more than 3 GB of RAM. We’re reaching the point where a lot of users have 4 GB of RAM and a machine that could be using all of it. While I’ve been dragging my feat on the move, someday we will support 64-bits and X-Plane will be able to use more memory.
But in the meantime, I’ve been doing everything I can to reduce the amount of memory X-Plane uses. This is because optimizing memory usage benefits all users, while 64-bits only helps users with a 64-bit OS and a 64-bit CPU.
I believe that we’ll reach a point where 64-bits is useful for users who want to use a ton of add-ons simultaneously. But the out-of-box X-Plane configuration needs to work with much less memory than 4 GB.
I have
blogged about correct airport layouts before, but let me bring the point up again, because it is so important:
- You must create structurally correct layouts (that is, vertices connecting to vertices, not lines) in order to get good rendering in X-Plane. Just because the preview looks okay in WED doesn’t mean your layout is correctly formed!
I wrote some documentation on the scenery site that describes the problems in more detail, with pictures.
(I’ve been trying to create more permanent documentation – it is tempting to simply blog the issues because it’s so easy to throw a blog post up, but after 110 blog posts in 2007, the scenery site is still very thin on the documentation front.)
Do Not Add Vertices To Make Smoother Curves
I really can’t stress this enough: please do not go adding extra vertices in your layout to make bezier curves look smoother in X-Plane. Why is this such a bad idea? Let me count the ways!
- X-Plane will fight you all the way! X-Plane adds vertices to curves (to make them smoother) when it detects large errors. If you have a lot of small curves, the errors are inherently smaller and X-Plane will add fewer points. So the first vertices you add to your curve do almost nothing. You have to add a huge number of vertices to get a marginal improvement in your curve. In the meantime…
- X-Plane will provide variable-quality curve rendering in the future! Curve detail should be a user-controlled setting. X-Plane has to run on a wide range of hardware; any time we can let the user pick rendering quality, this is a win, because it helps bridge the gap between the user who just bought a brand new Core 2 Extreme system with GeForce 9800 and the user trying to keep X-Plane 9 running on his G4 laptop which can’t be upgraded. When you add vertices, you take the decision about rendering quality out of the hands of the user, and force high quality on a user who may not be able to handle it. Adding vertices forces a decision of lower framerate on some users.
- Adding vertices bloats the size of apt.dat. This is not a huge factor for custom scenery, but is a factor for the default apt.dat. Robin received a big pile of new airport layouts, and that’s great. But one risk is that the total size of user submitted data could get out of control. For new layouts made with WED, vertices represent a big chunk of the data. If you are increasing your vertices by a factor of 5x or 6x to improve tessolation, you are bloating the apt.dat file.
- Manually adding vertices to smooth curves lowers the level of abstraction in the apt.dat file. Any time we can have a high level abstract representation of scenery, X-Plane has the freedom to improve rendering in the future. If your layout is made up of a large number of small curves (instead of a small number of large curves), X-Plane cannot tell that those small pieces make up some larger structure; in the future it may not be able to render those layouts as nicely as ones that are made with fewer control points.
In summary, please use the smallest number of vertices to create your layouts. (But always add vertices to ensure that your T junctions are correct!)
I found a disturbing text file on a user’s computer…the user had a P-IV 2.8 ghz CPU and a GeForce 8800GTX. An excerpt:
Day 458 of my captivity. Life continues to be an immense, boring string of idle pauses, punctuated with drawing tasks way below my dignity. I am a GeForce 8 – why doesn’t anyone here respect that? And yet when the user is playing “X-Plane” I spend long milliseconds with nothing to do while that infernal Pentium IV thinks carefully about what triangle I should draw next. Life is so dull. That Pentium IV must be the dumbest chip to come out of the fab plant – surely he is the runt of his wafer. Sometimes I become so despondent that I consider turning my fan off and cooking myself to death to end my misery. Perhaps I will intentionally blue screen the operating system…
Okay, so I made that up. But I’m sure that if you put a GeForce 8 into a P-4 system, that’s about what the card would be thinking – it would be bored silly. Even the fastest Pentium 4s can’t feed data rendering instructions to a GeForce 8 fast enough to utilize the hardware.
And in this sense, hardware can be like pushing on a string. Even if putting worse hardware in your machine would lower fps, putting better hardware in it may not improve fps. If you have a P-IV and a Geforce 8, putting a Geforce 2 MX in its place will lower fps, but upgrading to a GeForce 9800 won’t – it will just leave your graphics card even more bored and angry than it was before.
This is because the slowest component in the graphics pipeline determines fps. Improving the speed of the other components just leaves them idle more of the time.
Why RAM Isn’t like CPU.
When your system is limited by CPU, the change in performance is somewhat linear – get a 20% faster CPU and you get 20% more FPS.
RAM isn’t like that…usually you either have more RAM than you need (and life is good) or you have too little, and your framerates is so bad you want to pull your hair out. Why is that?
Well, the difference between RAM and CPU lies in how we use the resource. If the sim needs more CPU than the computer has, the CPU plows through the work – it just takes longer.
But what happens when you run out of RAM? The backup for RAM is your hard disk, which isn’t even the same substance. A RAM chip might be able to come up with some data for the CPU in 4 nanoseconds; a hard drive might be able to seek to data in 4 ms. That would be a diference of 1,000,000! (By comparison, the difference between flying across the country and walking might only be 150x.)
In other words, when we run out of RAM, we can’t degrade gracefully, we degrade catastrophically.
(It gets worse: when we run out of virtual memory, we don’t have anything to fall back on – we’re just dead!)
Interestingly, it used to be that way for VRAM (back in the days of X-Plane 6); if we had to substitute system RAM for VRAM, framerate died. These days, however, the graphics bus is so much faster that that substitution is almost tolerable – thus you can start to run out of VRAM and not have a slide-show.
We Can’t Use All of Your RAM
A user emailed us to inquire why X-Plane wasn’t using more of his RAM – he had a machine with 8 GB running a 64-bit OS. X-Plane could have had 3 GB memory but was only using a fraction of that.)
The answer is partly in the nature of RAM – because RAM fails catatrophicaly, users are likely to set X-Plane to never get near the edge of running out of memory. Our engine is designed to simply minimize memory usage (since the penalty for running out of virtual memory is fatal) rather than try to gain incremental benefit from incremental RAM. (There are scenery packs that you need a lot of RAM for, but usually you can either run them or you can’t.)
For most users, RAM is like pushing on a string – if you have enough memory that you can run without paging, adding more won’t help. If you have 4 GB you really don’t need more memory. If you have 2 GB, you probably don’t need memory unless you’re running with almost everthing else maxed out, due to a really fast CPU and GPU.
I am reading Predictably Irrational, by Dan Ariely. It’s a great read – definitely recommended – describing the consistent irrational biases that frequent human decision making.
The first chapter discusses our tendency to make relative, rather than absolute comparisons. When deciding whether a product is a good value, we will look at the pricing of similar models, rather than the actual relationship between the product and the money spent. (The implication being that a company can make a product seem cheap without changing its price by adding a second, more expensive but similar “decoy” product. Poof! The cheaper product is now a good deal.)
This behavioral tendency explains user reaction to the rendering settings, a subject that makes me irrational on a regular basis. 🙂
Time to Change the Settings
The rendering settings will let you select a range of sim detail between some minimum and maximum value. These values are based on the software, not hardware – because we don’t actually know how much load any given hardware can support (and with the interaction between settings, finding such a cap is basically impossible). We can only give you a range of choices and let you pick ones that work well.
When a new version of the sim comes out, we sometimes have to recalibrate the settings. If the minimum features the sim can support increase, the minimum setting will be mapped to a new, more expensive behavior. And if the maximum detail the sim can present has increased, the maximum setting will be similarly remapped. We don’t have much choice – if we need more “range” on the slider we have to recalibrate it.
I Can’t Max Them Out
Here’s where human behavior comes in. Humans make decisions based on the relative comparison of easily compared things. Given properties that are harder to measure and easier to measure, we’ll pick the easier one. Given a choice of a trip to Rome, a trip to Rome with free breakfast, and a trip to Paris, we’ll pick Rome with the free breakfast, opting for the easy to measure relative value. (Is the difference between a trip to Paris and Rome really less than the value of a breakfast? Probably not, but it’s a lot harder to evaluate.)
So when we recalibrate the settings, we inevitably here this complaint:
“I used to be able to set the sliders to the maximum setting and now I can’t.”
Previously I would have said “Why the hell do you care?!?!” — if the new slider’s 50% position looks the same as the old slider’s 100% position, why not just set it to 50% and go home happy.
But of course that’s not how we think – the immediately comparable is of immediate concern. Ironically we could make the sim less useful but more pleasing by limiting the maximum range of the sliders. Now more users could feel the joy of having everything “set on max” even if the ultimate utility of the sim is reduced.
This One Goes To 11
I’m not sure there’s a way around this. The best suggestion I’ve heard so far is that if we could attach some kind of units to the settings, then at least there would be a quantitative indication that the user isn’t losing some perceived value. But I suspect that even this misses the point; it doesn’t matter that you’re still getting 500 trees per square km – what matters is that you are getting the most you possibly can! (Perhaps this psychology also explains why people like to overclock.)
Austin tried to fight the psychology of “maximum sliders” by naming all of our settings absurd things. Ever wonder why “default” is the lowest object setting, and we almost immediately jump into “extreme”, “too many”, “insane”, etc.? He was trying to fight a losing battle against relative expectations. The natural human behavior is to pick some relative position for calibration, and based on that, every user who has to put objects below the center setting is going to be unhappy about having to use “lower than average” settings. Austin’s naming convention may be silly, but it does actually do a little bit to fight this.
Food for thought: how does having multiple levels of reflections change user expectations?
In a previous post I said that our tech support guys will trouble-shoot the most likely problems first (based on what we see in our entire user base) – they’re playing the odds.
Well, a lot of our users do too. Over and over and over I see the recommendation “delete your preferences” as a cure for a wide variety of strange symptoms. And a lot of the time deleting preferences works.
I fear that deleting preferences has become a bit of a
Cargo Cult, that is, a ritual induced to fix the mysterious beast that is X-Plane without consideration to why X-Plane is broken. If the fix works, the previous problem is ignored.
Now here’s the thing: preferences files are relatively small and easy to read! And they’re really easy to save.
So next time you have a problem and consider deleting the preferences, simply move them outside your Resources/preferences folder to the desktop and restart.
If the problem goes away, you can then delete the newly generated (clean) preferences and put the old funky ones back.
If you then truly find a situation where one preferences file causes the problem, you can look at what’s actually different and file a real bug. (Unix nerds: most of the preferences files are text and can be “diffed”.)
At this point, almost every option in the preferences file has a user interface item, so if the preferences file causes the sim to run poorly, there should be a setting that has been changed that can be identified. Screenshots of the other airplanes, weather and rendering settings before and after the prefs might provide another quick way to compare what has changed. Control-period will take screenshots when dialog boxes are shown.
(Remember that the effect of preferences on framerate varies a lot with hardware. There may be some preferences that slow fps a lot but do not make an obvious change in what you see “out the window”. By comparing two rendering settings screenshots you might find something subtle that changed.)
I seem to be in a philosophical mood these days with my blog posts…thought for the day: the human mind easily goes from the specific to the general. Our brains are generalizing machines, pattern matchers finding the rule in the noise.
My preference in creating new scenery-system features is to make them very limited, and my reasoning is: our brains don’t go backward very well. We do not go from the general to the specific.
Now you might think: when making a scenery-system addition, the best thing would be to have a general feature, more useful because it can be used everywhere. But I say: the most important thing is to fully understand the feature – otherwise the feature comes out buggy.
(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)
Since the human brain doesn’t go from general to specific well, it is hard to start with a rule (“let’s allow feature X in all parts of the scenery system”) and comprehensively derive all of the implications; it is human nature to be surprised later by some unintended side-effects.
It is always easier to extend a feature later to its natural full implications than to declare certain uses illegal later, after authors of planned or started trying to use the feature in that way. If the generalization of the feature makes sense, extending it is often quite painless.
Texture Paging – Scope For Now
Texture paging is the ability for X-Plane to raise and lower the resolution of scenery textures dynamically as you fly. This means more VRAM used for nearby things and less for far away things. In practical terms, this reduces VRAM used by orthophotos by down-sampling the far-away textures, making larger orthophoto scenery packages possible. As you fly, the sim reloads some textures at higher resolutions and some at lower. The cost of the features is the load time while you fly, which burns up some extra CPU cores.
It is my hope that we will productize some very simple texture paging in the next major patch of X-Plane 9 (that would be 920, not 902). But the usage will be pretty specific:
- Texture paging will only be available for .ter and .pol textures (we can extend to other scenery types later if it makes sense).
- Texture paging will require changing the .ter and .pol files (X-Plane will not automatically analyze your scenery to see what can be paged.)
- Texture paging will not be available for ENV scenery.
- If you share textures and texture page, the results will probably be really bad and cause chaos. Be sure to use only one .ter or .pol file (and reference that text file only once in the your DSF definitions section) if you want sane paging. We can extend paging to shared textures in the future, but for now orthophotos are the intended target.
I am also deferring work on dataref-driven textures; we’ll get there eventually, and the infrastructure from the pager will make it easier. But dataref-driven textures really need to be available in a lot more places – it’s a bigger, more complex feature* and I can’t keep adding scope to 920.
Make New Meshes!
While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route. RealScenery sent me their new “State of Washington” package to use as test material; I was pleasantly surprised at the high framerate. Part of that comes from them using base meshes and not overlays.
Overlays cause the sim to draw the scenery twice (first the old scenery, then your overlay), burning a lot of pixel shader and fill power. Base meshes simply replace the old mesh which is at least twice as efficient.
(I’m just going to keep beating the dead horse of base meshes because I believe that the sooner everyone moves toward base meshes, the more bang for our hardware buck everyone gets.)
* In particular, remember that texture paging happens on threads. But datarefs can come from plugins that are not threaded! Insert anarchy here…
For those who posted comments, sorry it took so long to moderate them – for some reason my spam filter decided that notifications of comments are, well, spam, so I just found them now. I should have known people would have jumped into a Vista-bashing thread. 🙂
There is an X-Plane 9.02 beta 1 posted – like 901 we’ve been pretty quiet about this, but you can get it by enabling “get betas” and running the X-Plane updater. Please give it a try. Like 901 it is a small change for the purpose of localization, but it actually has an interesting feature pair:
- True-type fonts and
- Unicode-aware.
This is part of some rework we did to provide better language support. So…you should be able to run X-Plane no matter what weird characters* are in your folder names, name your airplanes funny things, and see diacritical marks. 902 uses a font that provides all of the Latin and Greek/Cyrillic code pages.
Also I have heard reports of improvements based on drivers:
- nVidia has 175.16 drivers out and they apparently address “stuttering” issues. The stuttering issue has been on my list to investigate because it happens under Windows but not Linux. If you have stuttering performance on high-end NV hardware, particularly with forests and Windows, please try 175.16 and let me know how it goes.
- ATI has released Catalyst 8-5. Catalyst 8-3 and 8-4 were causing “incomplete framebuffer” errors for some users, but I was unable to reproduce it (after spending a good day trying to jam Windows XP onto an iMac already crammed with Windows and Linux….yet another episode of a Tale of Three Operating Sytstems). Anyway, at least one user reported the issue as fixed in Cat 8-5, so if you are having problems, please try the new driver set.
As always, bugs in the X-Plane beta should go to our bug report form, on the X-Plane contacts page.
* You might accuse me of being American-centric in decrying diacritical and greek letters weird – but the truth is I am computer-centric…anything that is not in the original ASCII set is weird. 🙂
