I’ve always thought we should come up with cool marketing names for new code in the sim. Look at ATI and nVidia. When they reduce the amount of VRAM on their cards and steal your system memory, they don’t call it “hey, we’re cheap bastards so we left off some chips and hijacked yours instead”. Instead they call it “TurboCache™” (nVidia) or “HyperMemory™” (ATI). Don’t those sound much better?
Perhaps we can call the new rain effect (due out in the next RC) “MegaRain XT” or, um, “HyperSpash GT” or “Virtual Droplet Technology (VDT)”? I don’t think I’m cut out for a career in product marketing. Anyway, here are a few screenshots:
There will probably still be some camera angles where the rain looks weird; it’s a compromise between image fidelity and framerate.
Apropos of nothing: Ami sent me this flash video. I must warn you…Chris described it as “the corniest flash I’ve ever seen”….and in the world of flash that truly says a lot. Think of it as the blue pill for your, um…bits. Anyone wanna make a flash video showing the virtues of MegaMoister 2.0? 😉
(I promise the next blog entry will contain more information and less attitude!)
This blog entry describes how to file a performance bug for X-Plane. You don’t have to participate in X-Plane betas, so if this procedure seems scary or too complex, I suggest simply waiting for the final sim release. But if you want to help, this procedure explains how to do it. It is not enough to just tell us “the framerate is bad now” – we will just ask for the information that this post explains how to provide.
First, to file a performance bug, you will need two clean copies of X-Plane: the current beta and the previous final release. Use the web-based installers to install clean copies of both of them. (Hint: you can download the current final release, then copy the folder and run the beta installer on the copy to save download time.) We always need a relative comparison of framerate between two versions to isolate how efficient the sim is from how fast your hardware is. If the sim is slow on your computer and has always been that way, that’s not really a problem with the sim. But if the sim used to be blazingly fast and now it is not, we can fix that.
In order to get a clean test we need to control every aspect of the simulator. Fortunately these clean installs have default preferences. But you will need to go through and make sure you have:
- The same weather settings!
- The same rendering settings. Make sure new features are off, e.g. if you are comparing 840 and 850, disable birds in 850.
- The same aircraft.
- The same number of AI aircraft.
- The same location.
- The camera facing in the exact same position and direction!
- The same view mode (e.g. forward, forward with panel, etc.).
For your framerate test you will basically run the sim in a fixed configuration and take some screenshots.
It is very important that the camera angle be fixed. Use the “takeoff” menu item to place the plane on a runway. Do not taxi the plane into position; even the slightest change in the positioning of the camera can have a large impact on framerate, so if there can be human error in moving the camera, then the comparison is not valid. By pre-placing the plane you can get the same camera angle every time.
Some important notes on screenshots:
- Control-period will take a screenshot in any screen, so you can take a screenshot of your weather settings, not just the sim screen. Always use control-period, not the built-in OS screenshot mechanism.
- Please send us the original PNG files in a big zip file; please do not crop, editor or compress them!
- Please do not run the sim at larger than 1024×1024 as the screenshots will be clipped.
For both 850 and 840 we will need three screenshots (so for each performance report we will need six screenshots):
- A screenshot of the sim running. Use the data outputs screen to view frame rate, plane latitude, longitude and altitude, and camera location on-screen, so that these are captured in the screenshot.
- A screenshot of your rendering settings.
- A screenshot of your weather settings.
These last two allow us to exactly copy what you have configured. Please do not use “real weather” when you do these tests.
For each performance report, please send us the six screenshots, the log.txt after quitting both versions of the sim (so two log.txt files) and please tell us in the email the framerates of each sim both when paused and unpaused. (Take the screenshot when unpaused.) So there will be four fps numbers to report for every given test.
Note that these clean installs will not have any third party add-ons; if your performance problem is only visible with a third party add-on, please:
- First do the performance test without them and file that pair of datapoints.
- Then install the add-on on both and make as few configuration changes as possible and then file that data too.
So for a third party bug there might be eight fps to report – all combinations of: old and new, paused and unpaused, clean and with the add-on.
I just finished answering a user’s questions about whether to upgrade from a Pentium D to a Core 2 Duo and now I am back looking at a framerate problem on a 1.4 ghz G4 with a Radeon 9200.
Let’s just consider that for a second!
One set of X-Plane users has the latest generation hardware and expects X-Plane to look as good as it can. Getting 270 fps on this hardware (yes, we have seen fps this high with the default settings) is not useful – these users want more graphics on their big machines. If we don’t give them this, X-Plane is not a competitive product.
Another set of users still has previous generation hardware. If I may single out a particular group – Mac users with the G4 processor – that previous generation wasn’t ever very impressive even when it was new. Basically Apple was stuck because the G5 was the best chip they had but couldn’t be built with low enough power to put in the places they wanted to put it. Apple also wasn’t putting terribly nice graphics cards in their machines at the time. (This was fixed when they stuck a 9700 mobility in the PowerBook.) So we have a whole group of PowerBook and Mac Mini users suffering from low framerate on older machines that were never that good as flight sim machines to begin with. (And G4 users I do understand your pain – until about a month ago my laptop was an 800 mhz G4 with a GeForce 4 MX.)
It is in this context that I say: if your framerate is low, be sure to turn the new stuff off! I did some extensive benchmarking of X-Plane 850 vs 840 and can tell you four things that very much affect framerate:
- Cars on the roads! Cars really kill framerate. 840 has no cars, so it wasn’t doing any of that work. Turn them off!
- Birds! 840 had no birds, and they represent more drawing. Birds really divide machines into the haves and have-not’s…big machines don’t even notice the cost, but if you are having framerate issues, turn them off.
- Textured lights with 3-d frames. This is one of the bigger hits. I will comment on this more below, but if you’re fighting for framerate, turn this off.
- Water reflections and shadows. This isn’t the perfect setting, because it requires turning off cloud shadows as well as water reflections, so try it last; water reflections are actually not that expensive but you may get a few fps back.
In my benchmarking my dual 1.8 ghz G5 with Radeon 9600 XT (this machine is significantlty slower than a new DuoCore MacBook Pro by the way, so this machine represents a lower performance point than ANY new hardware that anyone should buy for X-Plane) I found that with these 4 settings off (and the equivalent textured lights and water reflections off in 840) X-Plane 8.50 was consistently faster than 8.40. Turning on water reflections and textured lights (but not the 3-d cars and birds) makes 8.50 slower than 8.40 but only slightly.
A final note on textured lights with 3-d frames: there are two catagories of graphics cards:
- Graphics cards with pixel shaders. On these cards, if you turn off 3-d textured lights, we will still draw them with textures, because the card handles it, so you’re not losing any framerate for the nicer looking lights.
- Graphics cards without pixel shaders. On these cards, textured lights are done with the CPU, just like in 840. They’re slow, so they are turned off.
My point is this: the checkbox gives you the best look when on, and the best look while prioritizing for speed when off. So if your framerate is low, turn off the 3-d textured lights; we will give you any quality that we can give you for free and nothing more.
In summary: you can’t have a full bottle of wine and a drunken wife. If you want the new eye candy that was not present in the previous version of the sim (3-d birds, 3-d cars, water reflections and 3-d lights) you will have to pay for them with fps. We can give you the choice of fps or more eye candy, but not both.
A final thought: is it possible that 850 has a performance bug? Maybe! I am analyzing this now. I will post detailed instructions in my next blog entry on how to report these things. Filing a bug saying “the framerate sucks” doesn’t provide useful information; posting to a forum saying “the framerate sucks” doesn’t provide useful information either. But there are some tests you can do that will provide us with the info we need. Our biggest limitation is: we don’t have one of each kind of graphics card. So the 850 beta testers who have gotten us detailed performance numbers have been very helpful!
I just fixed a bug in the mesh crack-patching code. This is a tricky subject, so here’s some information before you file bugs:
Two separate scenery tiles (whether DSF or ENV) may not line up perfectly when placed edge-to-edge. The most common reason is because their elevation data can come from different data sources, but in a few cases files from the same render can have alignment problems.
(In the case of the global scenery, if we have special circumstances like water or airports that must be flat and the features span a tile boundary, the flattening may not work the same way in both tiles. The scenery is also rendered on blocks on different computers, and the blocks may not tile perfectly.)
Whether the error is due to a bug or just two scenery packs together, the sim tries to do its best to fix any gaps in the terrain mesh. (The sim does not try to fix different textures that touch; this is beyond the sim’s capabilities of analysis.) X-Plane will take whichever file is loaded most recently and try to realign its edges to match the old edge. When this works right, you can’t see a seam. More importantly, if an airport spans that border, you can drive over the border (on a runway) without the plane hitting a bump.
I just fixed a bug in the crack-patching code where it would sometimes incorrectly fix a patch. So if you see a ‘crack’ in the terrain (basically you would see the sky color showing between terrain triangles as a sliver of light blue or grey) please report it as a bug, but be sure to include the following information:
- Please include a log.txt file so we can see what you have installed. Be sure to quit x-plane before emailing the log.txt file.
- Please include a screenshot of the crack. Turn on the framerate, plane lat/lon and camera position datarefs.
- Please set your sim resolution to 1024×1024 or smaller and send the original PNGs. Please do not crop, process, or compress them. Please do not send JPEGs.
- Please tell us the nearest ICAO airport so that we can easily go to this location.
As always our bug reporter can be found on a link from //www.x-plane.com/contact.html.
Some crack bugs may be due to a scenery defect (to be addressed the next time we make new scenery), some may be due to sim bugs, and some may be due to the combination of two scenery packages next to each other.
I just realized that the planet-wide rendering was running at 65 fps on my G5 with clear skies but less than 20 fps with a single cirrus layer! Ooops! With X-Plane 850 we use the same OpenGL code to draw terrain whether it’s close-up (DSF) or far away (the planet render). This seemed like a good idea because it helped the two mesh. Well, it turns out cloud shadows are way too slow when applied over a huge area.
So…I simply killed them. This will be in the next beta. If you look hard you’ll be able to see a slight color change (well, worse than before) where the cloud shadows disappear at the DSF->planet transtion, but the fps are worth it. The G5 went from less than 20 fps looking at the planet to 65 fps, and the MacBook Pro went from 36 fps to 100 fps.
This was never an issue in 840, where the planet had the simplest OpenGL code you can imagine and looked like, well, just compare the planet in 840 vs 850!
Anyway, this explains why I thought I’d fixed the planet code and users cried bloody murder: my standard framerate test has no weather, because that’s Austin’s code. (Hence I shut it off to check the performance of only my code.) But sure enough it was the weather that affected the “cost” of the planet render.
There is a moral here for programmers: always test performance under real-world conditions! Often a system can have interdependencies that don’t appear in “lab” conditions. (Another example: you can’t test the speed of airpor build-up without scenery because one of the slowest parts is “draping” the airports over the scenery. When there is no scenery the ocean terrain triangles are very large and so the draping happens much faster than it can with real ground.)
I’m note sure I can even explain this…basically “polygon offset” is both the darkest voodoo in OpenGL and NOX for OpenGL programmers…usually it helps you win but sometimes it causes you to go down in flames.
The basic idea of polygon offset is this: normally you see the closer of two objects ‘on top of’ the farther one. This idea is so obvious to us humans who live in a 3-d world that it’s almost hard to explain. Imagine I am a painter. If I draw two mountains, the one I draw second will always appear “on top of” the one I draw first in my painting…thus I must draw the far mountains first and the close mountains second to get a realistic landscape.
OpenGL provides “Z buffering”…basically this lets you draw the mountains in any order and the graphics hardware takes to not draw the pieces of the far mountain that need to appear under the near mountain in order to make the 3-d image look realistic.
So where does polygon offset come in? Well, the problem is that if two pixels we’re drawing are very very close to each other in distance from the viewer, OpenGL’s ability to decide which one goes on top falls apart. It turns out there are limitations to the math OpenGL can use. In particular, if I have an object with two triangles that are exactly on top of each other…OpenGL will randomly show some pixels from one and some from the other! Yuck! This is called Z-thrash.
This is where we bring in polygon offset. Polygon offset is a trick where you tell OpenGL to handicap the competition between two triangles. You say “listen, I know that the grass and the runway are probably about equal distances from the viewer…but handicap the math by adding 2 to the runway in all cases”. What happens? The runway always wins! In other words, polygon offset is a cheat that helps assure that given two pieces of scenery on top of each other, the ‘right’ one always appears on top.
For more discussion on polygon offset, check out this article from the scenery web site.
Whew. Well that was fun…we use polygon offset on runways (to make sure they never thrash with the ground textures) and it just works great. But when it comes to roadways, things really go to hell.
Polygon offset has two strange quirks that make it tricky to work with:
1. If you cheat and use polygon offset to help a bit of scenery ‘win’ the distance competition, there is a risk that the cheat could push it through other parts of the scenery. (This mainly happens if you use too much polygon offset.) For example, if we used too much polygon offset on the runways, they would start to appear ‘on top of’ the airplane! It turns out there are two ways to manage this…we can tell OpenGL to “cheat once and forget about it” or “cheat every time you use this triangle”. Both ways have their own problems.
2. The amount that OpenGL cheats and adds some to the calculation about “who is closest” depends on the angle from the viewer to the triangle in question! Take a second to consider that…if you look straight at the runway from the top, OpenGL is only adding 1 or 2 to the calculation – but if you then sit on the runway looking down it, perhaps it’s adding 10 or 11! I can’t hope to explain this aspect of polygon offset without going into the mathematics of 3-d frustum projections, but I will say this: this property is both necessary (if it didn’t work like this, most times polygon offset wouldn’t provide a decisive winner between two triangles) and a pain in the ass, because it means that relatively large offsets can be used at certain view angles.
(In fact whenever you move the camera and see one piece of scenery “eat” another, then go away, this is what you’re seeing – we’ve used too much polygon offset and at a steep view angle the amount added to the math is too huge.)
Now I think we know enough to understand how X-Plane’s roads became the road to hell.
Generally speaking, any time you put something directly on top of the terrain mesh, you need polygon offset to assure you see it and not the terrain under it. This is true for beaches, runways, and of course roads. (Authors making airports out of OBJs know that their tarmac objects need ATTR_poly_os too!!) So I think we can at least see that we need polygon offset for roads on the ground.
Now consider bridges. They are up above the ground, so technically they don’t need polygon offset. But if a bridge segment starts on the ground and climbs up, it will be eaten by the ground. So I decided to polygon offset bridges too, and that was okay.
Then came cars. The problem is: do cars have to be polygon offset? Well, it turns out we lose either way. Recall that there are two ways to handle the ‘cheat’:
– If we only offset bridges once then they appear above terrain, but one bridge may incorrectly appear above another, because the cheat is not permanent.
– IF we offset bridges “permanently” then they will appear above the cars on top of them.
Okay so let’s offset the cars too! It turns out that that doesn’t work. Recall that the amount of offset is a function of the view angle of the triangle. Well, 99% of the time roads are horizontal, as is the terrain. So offsetting is okay – everyone gets about the same cheat.
But a car is made up of triangles going in all different directions…the result is that if you offset a car, then the roof offsets more than the doors! This just looks terrible. And even worse, the headlights, for technical reasons, always offset the minimal amount. (Geeks: they are billboards – they are quads that always look straight at you, so they always have zero view angle!) Bottom line is: you can’t cheat the cars effectively onto the roads. Up through beta 11 you have seen this awful result..the cars look like they are inside the bridges.
Now since most of our cars drive on the highways, and since the highways (at least in the US) are mostly bridges so they can go over the regular road grid, this artifact is visible just about all the time.
So for the next beta I am backing off to the lesser of two evils: the bridges will not be offset. This means the cars will look great on the bridges. What you will see is a gap, like the one shown here. What this is: right as the bridge starts to “rise” out of the ground, the polygon offset cheat is turned off, and so for a little bit the ground consumes it.
Here’s something a little bit surprising about the library: the EXPORT command in a library.txt file cannot replace another file in the same package.
For X-Plane 850 we added cars driving on the left side of the road for New Zealand, the UK, Japan, and other countries that drive on the wrong side of the road. Unfortunately what I did was export two road.net files from the same package – one with a region restriction (to these few countries) and a later one with none.
Consider the sum of these two statements…the result is that in New Zealand, the UK, etc. the sim will use either of the two road.net files, picking randomly.
850 will address this with a new library command. Whereas the “export” command normally provides one of multiple alternatives within a package (but replaces any lower ranked packages), the new “export_exclude” command will replace alternatives both within the same package and within lower ranked packages. Commands are treated sequentially, so an export_exclude should usually be at the top of the library.txt file.
I was at least as surprised as you will be to discover that…
..when you pop back to your starting location using replay mode, the airport appears (belatedly – see the previous blog entry) almost five times faster if you turn off texture compression!
What the hell?!?! 🙂 It’s almost as if some idiot programmed X-Plane to load scenery five times slower if texture compression is on. Oh wait…that was me.
The problem is: how to balance the importance of keeping the framerate up with the importance of loading airports before you get there? All background work involves stuff we might do in the background (if you enable the rendering setting), maybe on a second core/CPU if you have one, and also some talking to the graphics card. We can only talk to the graphics card between drawing frames* and doing so stops rendering.
It turns out that in X_Plane 8.30 when we developed background loading, I put in the “go 5x slower” option because compressed textures visibly halts the sim as the graphics card stops rendering and compresses each texture. Without this 5x slowdown in background work, flight becomes stuttered and bumpy. The problem is that the 5x slowdown means we’re going through airports at a much slower rate than we should be.
I am working on airport tuning now…there will still be a delay before you see airports if you use instant-replay to move around fast enough (this delay is about the length of time the sim would have spent halted during scenery load but now no longer does) but the responsiveness should be a lot better.
(Here is a discussion of dual-core for Flight Simulator X…tdragger explains the issues well, and I think they apply to any graphics-intensive real-time or near-real-time simulation, so it doesn’t surprise me that both X-Plane and FSX offload similar types of work to extra cores.)
*Newer OpenGL drivers get around this limitation to various extents; we will eventually support this in X-Plane.
Sometimes with X-Plane 850 you may reach your destination airport only to discover that…it’s not there! What’s going on???
Well, in X-Plane 850 3-d scenery is created while you fly. This includes roads, trees, buildings, and most importantly airports. So…if the flight simulator cannot build 3-d scenery as fast as you fly, you may get to the airport and discover and it’s missing. After a while, it will appear. The easiest way to see this is: go to replay mode, so you can easily move a huge distance just by dragging the replay slider. This will “thrash” the 3-d scenery build-up and a bunch of airports will be missing.
There is one exception to the “build-as-you-fly” rule: whenever you do anything that changes the scenery situation from a dialog box, X-Plane says to itself, “well, the user is already not flying, because a dialog box is open…why don’t I take the time now and create all the 3-d scenery nearby”. So when you place an aircraft at an airport, change certain rendering settings, etc. then the sim builds the airport up front and it will always be there.
I do not yet have a fix to the problem of missing airports. For now I recommend that you consider turning down your 3-d scenery; often a lot of roads or objects can cause the sim to get behind in its 3-d work. I am working on making the code more efficient.
Apple has released new iMacs. Previously the iMac was the best X-Plane experience for the dollar among Macintosh hardware offerings. It still is, mostly.
But…the cheapest smallest iMac, at an affordable $999 does not have the kick-ass ATI Radeon X1600 graphics card. Instead it has the, well, ass GMA950 integrated graphics.
If you are looking to buy an iMac and are even going to think about playing X-Plane, buy the more expensive box and get the ATI card. You can’t upgrade iMacs, at least not in any way that I’ve ever heard of, and the GMA950 is never going to perform well.
More importantly, as X-Plane progresses we will take more and more advantage of hardware shaders, and the GMA950 is missing a major set of shaders. So even though the iMac itself is blazingly fast, the video card will simply be unable to show a lot new features.