One thing I have learned during my time here in the matrix is that users often find new bugs to be significantly more annoying than existing ones; trading an existing bug for a new one is often seen as a big step backward, even if the new bug isn’t that bad. I think this is just human nature – we get used to things – it’s what humans do – and this takes the sting off of some bugs that have been around a while.
I bring this up to put in context the problem of texture paging, blurry textures, and VRAM management in 11.50. To understand what makes VRAM paging hard, you have to understand how the OpenGL driver solved the problem, and ask yourself: what was the old bug, what is the new bug?
What Is Texture Paging?
Texture paging is when textures are moved between the memory on your graphics card (VRAM) and the memory in your computer (“system memory”). At any given time, X-Plane may have more textures loaded than you can fit on your card, but it doesn’t need them all at once. As you fly, different textures are needed, and the textures have to be moved around.
If this sounds a lot like moving data from RAM to your hard drive (virtual memory), it is. When I get the question “how can X-Plane be using 2.5 GB of texture memory when I only have a 2 GB card” the answer is…texture paging.
How Did the OpenGL Driver Do It?
In an OpenGL based app, paging textures to VRAM and back is entirely up to the OpenGL driver. The algorithm goes something like this:
- Start drawing.
- Realize something you need is in system memory.
- Move something you haven’t used in a while out of VRAM.
- Move the thing you need into VRAM.
- Continue drawing.
The OpenGL driver pages during drawing when we need something new, and the result is a stutter – the frame takes longer because it cannot be completed until the texture is moved to VRAM.
Now NVidia, AMD, and Apple have spent a lot of time and effort to make this work as well as they possibly can, and I’m definitely selling short the complexity and sophistication of the drivers as they try to guess really well what should be moved out of VRAM. Still, two key facts will be true:
- If we need to page stuff into VRAM to draw, and we don’t figure that out until right when we need it, there’s going to be a stutter. Maybe a really short one you barely notice, maybe a really bad one, it’s a question of “how much”, not “if”.
- What you see on screen always looks fantastic – never blurry – because the driver won’t cut resolution to save VRAM.
So…the old bug in this case is stutters due to VRAM containing the wrong stuff.
How Does X-Plane 11.50 Do It?
With Metal and Vulkan, it’s up to us, the app, to decide what goes in VRAM and what doesn’t, and to schedule the movement of data between VRAM and system memory. Our strategy is:
- Keep an eye on how much memory is free.
- If it looks like it’s getting too close to full, shrink textures to a smaller size. Prefer to shrink textures you’re not looking at. Do the shrinking in the background.
- If it looks like VRAM has empty space free, grow textures, preferring ones you are looking at now. Do the growing in the background.
- If we have to load new textures in the background (e.g. new DSFs as you fly), wait until old textures can be shrunk.
We have a huge advantage over the drivers in this game: we know the textures we might need in the future, long before we actually need them. We can also shrink textures. This means we don’t have to stop and shuffle things around, and that means we can avoid stutters. So we can fix the old bug: stutters while you fly.
(To be clear: this is not the only source of in-flight stutters while you fly – OpenGL and 11.41 are full of them.)
The down-side is: if VRAM gets tight, we’ve made some of the textures lower resolution. This is a new behavior (blurry textures), so my fear is: users aren’t going to like it.
In particular, the OpenGL driver uses the exact optimal set of textures for every single frame – at the cost of stuttering. We may not have things shrunk and grown in exactly the best way while you fly, so for the same VRAM and no stutters, we may not be as visually sharp.
What About Plugins
Add-ons that use OpenGL (e.g. aircraft with custom glass displays) use VRAM too – this is one reason why we can’t pack 100% of VRAM with our textures; we need to have enough free that if a plugin needs VRAM, it won’t crash us.
My expectation is that fighting blurry textures and crashes due to running out of VRAM is going to be a long, iterative process and one of the main reasons to have beta testing. It’s one thing to have an algorithm that works good in the lab or on paper – it’s another to have thousands of hours of test time in real-world conditions on user machines with lots of add-ons and anarchy.
So when 11.50 is public beta and you see your blurry texture, don’t panic. VRAM management is a very new part of X-Plane, and it’s a very important part, and it’s something we will iterate on over time.
(Finally, I want to mention that while the port to Vulkan/Metal has been a joint venture that Sidney and I have worked on together, there are certain parts of it where Sidney basically did 100% of the work, and my only contribution was to complain about bugs. The shader compiler, the OpenGL plugin bridge, and the VRAM pager are all big lifts that Sidney has done on his own that make the Vulkan/Metal port possible.)