X-Plane 862 RC1 is out. Just to clear up confusion, this is the next RC after 861RC1 and is just more bug fixes (all bug fixes from 861RC1 plus a few more).
There are two things hopefully working in 862:
- I think we got the optimization settings right – framerates on Windows should match 860. If they don’t, please let me know.
- There is another object-placement bug fix in 862. If you’re still seeing problems with object placement depending on which direction you approach a scenery area from, please let me know.
There have been plenty of rumors and semi-official posts regarding the upcoming major revision of X-Plane (X-Plane 9). I have been trying to keep my mouth shut about it…the problem with pre-announcing anything is that the upside to us is small (at best we do what we said) and the downside is large (at worst we don’t do what we said and people get grumpy). No one complains if XP9 turns out to have no-pause scenery load and it’s a surprise…but plenty of people complain if we say “there won’t be pauses” and then they are.
But…the situation is becoming mildly absurd…plenty of info is out there, and saying “the upcoming major release”, etc. just feels political and weaselly. Austin would be disgusted.
So listen: I am going to try to provide some info on X-Plane 9. This info is subject to change. This is what we think is going to happen to the best of our knowledge. The release is still a ways away and enormous changes will happen. When things change, do not bitch to me that “you promised X” would happen. I do not promise anything. This info is provided to try to help those making add-ons for X-Plane plan appropriately.
With that in mind…I will try to post some more details on the authoring environment in the next few days. For now, here’s some very basic guidance on compatibility and hardware requirements:
- The hardware requirements will be at least as high as X-Plane 8. If your machine is gasping and wheezing on 8, it’s not going to be any better on 9.
- X-Plane 9 will depend more heavily on pixel shaders. If your hardware doesn’t have pixel shaders (GeForce 2-4, Radeon 7000-9200) or has really crappy shaders (GeForce FX series) you will miss out on a lot of the cool stuff in v9, and possibly have the scenery look worse (but faster) than v8 (as we move features from the CPU to pixel shaders).
- Scenery that opens in x-plane should open in X-Plane 9 unmodified – if the scenery works in 8 but not 9, report it as a bug!
- Plugins that work in v8 should work in v9 without modification.
Finally, we are trying to finish up X-Plane 861…this is a bug-fix patch for version 8 – it contains no new features, but it does fix a few nasty bugs, some of which cause crashes. So if there is any new feature, it’s coming in 9, not 861. Version 8 has been out for a very long time, so I will accept no argument that v8 should have more features than it does now. It’s been a long run!
(One of my main goals with 861 is to try to fix any weird behavior for third party scenery add-ons, so that a scenery add-on looks the same in v8 and v9. If we left the bug in 861 and fixed it in v9, authors would have to hack the scenery to make it work with v8, and then remove the hack and republish for v9. By trying to fix the authoring bugs in v8, at least when possible, it lets authors publish one package for both versions. Of course, v9 will have new features, so I expect some v9-only scenery will emerge pretty quickly.)
I’ve heard a few reports that 861 is slower than 860. I think this is rather unlikely given the code changes made in the 861 patch, but I take all performance issues very seriously!!
I can’t get X-Plane 861 to run slower than 860, so here’s the data I need:
- If you are seeing this, start by getting a simple case, e.g. sitting on the runway staying still, with a known viewpoint. The conditions have to be reproducible and stable — “overall the sim is slower” isn’t quantitative, it’s an impression. Ranges of fps aren’t precise enough.
- If you are using third party add-ons, strip ’em out. Does the fps problem go away? If not, good, let’s get a case that uses only the simplest setup. If the fps problem does go away, let me know what add-on it is…perhaps a particular add-on exacerbates a performance problem.
- You can put the 860 x-plane app into an 861 folder – do this for all regressions to make sure you’re using the same prefs file, etc. etc.
- Of course most useful is: run the fps test – if you can send me comparative fps-test logs, this is the most useful thing because it controls virtually all variables!
With that in mind, please email me log files, pref files (if you must use a specific prefs combo to see a problem), etc. Please do not just tell me descriptions – that’s not precise enough!
In order to maximize X-Plane’s framerate, you need to start with the lowest settings and work your way up. Can you start with the highest settings and work your way down? No! Here’s the problem:
X-Plane’s framerate is determined by the slowest part of the system…whether it’s your CPU, AGP bus, or any of the parts of your graphics card (pixel shaders, lack of VRAM, lack of internal card memory bandwidth). Whatever the problem is, the graphics are drawn in an assembly line, and the rest of the hardware sits bored while the slowest part finishes its backlog.
So…why not start with the highest settings and turn them down? The answer is that when the framerate is slowed down by component A, changes in the load on component B won’t show as a difference of framerate, because component B isn’t the slowest. So as you turn down settings, you’re not seeing the real effect. (The safe thing to do would be: turn B back to its old setting, then retry.)
What I recommend is: start on the lowest settings possible – you’ll see a framerate that’s about as high as your system can go (limited by the FM and the speed the card can turn a frame around). Then turn settings up, one at a time, by one notch. Each time you see a fps hit, back off one setting and continue somewhere else. This way you won’t load up one part of the system so heavily that you are limiting the speed despite everything else.
Why don’t the cars drive backward when you pull the slider backward in replay mode? The short answer is “because we don’t care enough to fix it”, but a better answer might be “it would take a lot of programming time and suck up more resources from X-Plane to fix this…and we think our customers would rather that we focus our programming and your hardware resources on framerate.”
The cars are an interesting special case of a whole number of sim phenomena that we don’t attempt to track carefully in replay. Replay is designed to allow you to watch your flight – it would be cool if the scenery was doing the exact same thing during replay as during the flight, but I don’t think it’s essential for training purposes and it does come with a cost.
First remember: replay mode works by saving past values of the sim to RAM. So the more we save, the more RAM we chew up saving past history, and the less time we can save before we run out of virtual memory.
Now in some cases, the motion of dynamic sim objects is at least somewhat random. In this case we can’t easily “reverse” the algorithm that generated the motion.
But the cars are more problematic.
Not only is their motion somewhat random (each time a car makes a turn at a fork in the road it randomly decides which way to go), the cars are maintained in memory in a way that allows us to figure out who has to make the next turn very rapidly without using a lot of CPU. As much as the cars are a CPU hog, they would be much much worse without this memory structure.
The problem is, the memory structure is organized based on time flowing forward. That is, we can only tell you which car needs CPU attention next if the cars are driving forward. Put time into reverse and we now know which car needed our help last! Not useful!
So to make the cars drive backward we would have to transform this data structure every time the flow of time changed. I think it would be more annoying to have this massive CPU recomputation each time you rewound the replay than it would be nice to have the cars move backward. Why not have two data structures, one for forward, one for reverse? Well, now we’ve saved CPU but burned RAM. Either way, we’re talking about hardware resources that could be used for more scenery or more framerate.
The cars have yet another behavior that makes them hard to reverse: they are born and die! A car is born any time we realize that there aren’t enough cars on the road for the given rendering settings. A car dies any time it gets too far away from the user’s plane or reaches a point in the road where it can’t procede. (Typically these are 1-way streets that dead-end. This happens because the road data we use has very poor flow information, leading to some really strange streets.)
This cycle of cars being born and dying maintains a reasonably constant car population over time, and a car population that is near your plane as you fly. But to reverse traffic, we would have to reincarnate cars that had died previously. This would mean spending memory on remembering what cars had died. (Even if the algorithm to decide where a car is born, the algorithm to predict where a car will die is quite complex, because it requires looking at the entire set of steps the car would make during its life until it reaches the “point where it is killed”. So computing this information is out of the question.)
That’s probably more information than you wanted to know. Generally speaking, if someting unrelated to the flight model doesn’t replay in replay mode, it’s probably because it would be too “expensive” to remember its history. The cars are the most complex example, but definitely not the only example!
Austin hinted on the X-Plane news list that X-Plane will in the future use a second core to load scenery much more smoothly than it does in the past. Each time we take a whack at the scenery load code, we reduce the load time by a significant multiple, so we’re still not at truly no-pause flying, but we will be getting a lot closer in the future.
A few years ago when people asked “should I get a dual CPU machine” (this was before dual-core chips were ubiquitous) I hesitated to say yes. This is on longer the case – dual core is definitely the way you want to go.
The price performance curve is such that dual core chips are definitely in the sweet spot. In the past a true dual CPU machine costed big bucks, but now unless you buy the absolute cheapest machine, you’re going to get a dual core chip. I never recommend buying the absolute cheapest hardware — usually to get those last few dollars of price savings, the machine (or GPU) has been stripped down of major features, so you’re losing tons of performance for really marginal savings in dollars.
X-Plane’s usage of the second core has gone up too, already in 860 and the second core will be even more important in version 9.
Our usage of the second core isn’t about trying to raise X-Plane’s system requirements, it’s about not leaving “money on the table”. We need to make X-Plane run efficiently as possible on as many combinations of hardware as possible. This means giving you the best flight sim experience you can get whether you have an older single-core machine with a low-end graphics card or a new dual-core chip with the latest huge graphics card and four tons of VRAM.
The important thing here is: usage of dual core chips requires code changes. We have to write new code in X-Plane to teach it how to safely run some work on one core and other work on another. This is all about efficiency – if you have a dual-core chip there is nothing efficient about leaving one core doing nothing! We’d be leaving half the computing power of the chip idle.
So if you don’t have a dual-core chip, the sim will still run (just as it does now). But whereas the gap between single and dual-core chips is smaller now (because the dual-core users have half a chip idle a lot of the time), the gap will become larger as we utilize the second core more fully and dual-core users get to really take advantage of all of that computing power.
This goes into the bucket of “weird X-Plane behavior”: X-Plane will try both PNG and BMP file extensions when opening images, no matter what is in your file. How we got to this state is, at best, confusing.
Originally, most X-Plane image files did not contain a suffix. So an ENV file contains “grass” and X-Plane would change that to grass.bmp.
Then we added PNG support. X-Plane would try grass.png and then grass.bmp. In this case, not having the extensions turned out to be handy — authors could simply bulk-convert their images and go on with life.
With most new scenery system files, the extensions are a lot more rigid:
- The extension appears in the referencing file.
- The sim only tries that extension.
- If the format doesn’t match the extension, it’s an error.
So if you want a DSF file to reference a facade, it’s buildings.fac and if that .fac file is actually a forest file, it’s an error. The sim won’t try to decide which is more correct, the header of the file or the extension, it will just go “you’re nuts” and bail out.
But (for historical reasons) images are an exception. Keeping with the “any extension goes” theme for images, X-Plane will actually try PNG and then BMP versions of your file. The extension has to match the format…that is if you call your bmp foo.png X-Plane won’t load it at all.
We have PNG as our primary image format and BMP for backward compatibility. But it’s imaginable that we could have DDS and PNG both as primary formats — PNG for images that need lossless fidelity and DDS for images where compression is acceptable. In such an event, X-Plane’s tendency to try every extension means authors can bulk-convert from PNG to DDS (making their packages load faster) and go home happy.
Jonathan Harris suggested I look into .dds as an image format for X-Plane. I think he’s got a good idea.
First, what is DDS? It stands for “direct draw surface”, but as a file format, it’s basically a file whose structure is the same as the memory layout for a texture in DirectX. It’s a simple image format but it does two useful things for X-Plane (or any other 3-d application):
- It can contain compressed images using the same compression OpenGL and DirectX use (DXT compression).
- It can contain mipmaps – smaller versions of the same image.
Now before things get out of hand, using .dds files in X-Plane would not mean converting X-Plane to use DirectX!! .dds is simply an agreement about how to arrange a file. You can easily write code to load a .dds file into an OpenGL application, and that is what we would do.
There would be three fundamental benefits to using DDS files:
File size on disk (and download).
PNG is a losslessly compressed file format. This kind of compression (the internal algorithm used by PNG is the same as ZIP files) preserves the image, but the reduction in image size is a function of how “simple” the image is. Images with a lot of detail and high frequency information (rapid changes in color) do not become much smaller than the amount of VRAM they use when compressed in a PNG. A lot of the terrain textures we use (4 MB in VRAM) are still 3 to 3.5 MB on disk!
By comparison, the texture compression used by OpenGL is lossy – the image looks worse. But the image is reduced by a factor of 4 to 8, every time. So one of these images (4 MB in VRAM when uncompressed) would be 1 MB with S3TC compression (available in a DDS file). Even for very simple PNG files, you rarely see such a reduction in size, so .dds files would be smaller, meaning faster updates and downloads.
(Another thought: if a PNG file is smaller than its VRAM requirement, it means the PNG file has lots of areas with the same color. This means the PNG file is WASTING VRAM! We encourage our artists to make use of every little pixel in an image, and this means that PNG files that are well-created for X-Plane tend to get relatively little benefit from PNG’s internal compression.)
Load Time
Image load time is a significant factor in X-Plane. There are several reasons why DDS files could load a lot faster than PNG.
- The actual DDS file is smaller – less data to load and process means faster performance.
- Because the DDS file can contain smaller copies of the image, X-Plane doesn’t have to load a huge image and reduce its size using the CPU – it can just load the small version in the file.
- When compressed textures are used (and they almost always should be, see my previous blog post) it takes CPU power to compress the image, slowing down load. A DDS file containing a compressed image can be sent directly to the card!
Image Quality
Now one is a little bit tricky. First, please review my previous blog post, which argues that texture compression is the best way to maximize VRAM. A DDS file will look better than a PNG file when texture compression is on. Here’s why:
The S3TC compression formats specify an EXACT way to “uncompress” and draw a compressed image. But there are MANY possible ways to compress the image. Remember that S3TC is a lossy compression – the compressed image is only an approximation of the original.
Well, it turns out that algorithms that find the best approximation of an image using S3TC are slow. So we can’t use the highest quality compression inside X-Plane when compressing PNGs for the graphics card – it would be too slow.
But the image in a DDS file is already compressed — because someone else has compressed it when creating the scenery. When converting the scenery from PNG to DDS that author can use a very slow, very high quality compression algorithm to make the DDS files. It might take 12 hours to convert the entire package from PNG to DDS, but who cares? It’s something you do once and then everyone enjoys better looking images.
In other words, because the compression is done ahead of time for a DDS file, a higher quality algorithm can produce better approximations of the original images. Since we want to use compression anyway (to optimize our use of VRAM) this is a good thing.
When is DDS not appropriate?
Any time an image must not be compressed, DDS is inappropriate, and PNG is best. There are three basic cases in X-Plane where we avoid compression (see my previous blog entry for the real examples):
- Any time we need to preserve tiny detail accurately, like lettering and text.
- Any time the alpha channel needs to be smooth.
- Any time a texture is magnified so large that small color changes between two pixels is noticeable. (For example, compressing the color washes used for the sky make obvious artifacts.)
DDS wouldn’t replace PNG, it would augment it.
Speaking of compression, X-Plane doesn’t compress all textures…even if “texture compression” is on, we leave uncompressed certain textures that are flagged. A rough list:
- Most textures in the sky, like clouds and color stripes, have a lot of alpha and get wrecked by compression. They get exempted.
- Anything related to the 2-d panel, maps, and the UI get exempted.
- Some of the overlays like rain and snow, smoke puffs, and lights.
- Taxiway signs get exempted…S3TC does a number on letters and numbers.
Try this experiment: take three screen-shots (use ctlr-; to get a stable, clear view), with these settings:
- Extreme res, with tex compression
- Extreme res, without tex compression
- Very high res, without tex compression
Now consider the effect on VRAM. Each time you go down a texture resolution level, tex sizes are cut in half, and you cut VRAM use by a factor of 4.
If you use texture compression, you cut VRAM use by a factor of 4 for textures with alpha and a factor of 6 for textures without alpha. (Some textures are unaffected because they are internally marked as “never compress”.)
Now look at your visual results. I would argue two things:
- At extreme res, turning off texture compression doesn’t make the image a whole lot nicer. (It does make the machine a whole lot slower!)
- Texture compression is a lot nicer than going down a res level.
I suspect that if your machine can’t handle extreme res, you can get similar results running the whole experiment down one notch.
I think that almost everyone who has a limited machine is running with texture compression these days – we default with it on and it helps fps. I would only argue that any users who are running with a huge machine and compression off might consider turning it back on. It doesn’t hurt much and it does make the machine run faster.
(Even if you have a ton of VRAM, X-Plane can be limited by how fast texture data transfers from VRAM to the GPU, within the card. Texture compression helps this.)
