We have updated the X-Plane plugin SDK and related documentation for X-Plane 11.50. Here are some links:
New Plugin SDK: version 3.02 adds the new modern 3-d drawing callback for OpenGL/Vulkan, and deprecates the drawing callbacks that are unavailable in Vulkan/Metal. Download it here.
(Updating to the new SDK is not mandatory for your add-on to be Vulkan-compatible, but it can help catch drawing callbacks that will not work, and it is necessary to use the new 3-d drawing tech.)
Plugin Compatibility Guide:a short guide that focuses on issues existing plugins might have with X-Plane 11.50. If you are updating a plugin, read this!
OpenGL Drawing Guide:complete documentation for all aspects of drawing using OpenGL with X-Plane. If your plugin uses OpenGL, this is a must-read.
Drawing Over 3-d in 2-d: this sample code shows how to draw in a 2-d window or drawing callback with coordinates that match the 3-d world. Many 3-d plugins that draw need this kind of drawing, e.g. to label routes or aircraft with UI that matches the 3-d world.
Instancing Example Plugin:a sample plugin that draws objects using the XPLMInstancing APIs. Many add-ons will need to transition from object drawing to instancing to be compatible with Vulkan. We strongly recommend moving to instancing – it provides the fastest, most compatible drawing path, particle system support, and in the future will support FMOD sound.
Instancing is actually much easier to use than XPLMDrawObject and bypasses a lot of complexity and chaos. Instancing works with all versions of X-Plane 11, and the sample has everything you need.
X-Plane 11.50 has been out for a little bit more than 24 hours, and things have been a little bit nuts. Here are a few quick notes, in no particular order.
Bug Fixes and Work Arounds
While I don’t have work-arounds for the missing right eye in VR or older NVidia cards that won’t run in Vulkan, the good news is that we have fixes for these already. We are going to start testing beta two on Monday and try to get the fixes we have out as soon as possible. While we don’t have every major reported bug fixed, beta two should make a real difference.
Users who can’t start and have SLI setups: disable SLI in the Nvidia control panel and you will be able to run Vulkan. We are still investigating this – our goal is a bug fix so you don’t need to turn SLI off. (We do not expect X-Plane to leverage both cards – we expect it to run without failing.)
Finally, one thing I should have mentioned in the announcement: if you have scripts that modify art controls, please remove them, and don’t put them back.
The art controls are undocumented and subject to change, and they have changed a lot since X-Plane 11.41. Realistically the authors of these tweak scripts need to go back and re-evaluate every one of their tweaks in 11.50 to see if they actually help or are actually making things worse.
This is not like plugins or scenery; we tell you to get a second installation for the beta not because we want you to run add-on free but rather so that if the beta fails you are not locked out of X-Plane. We expect add-ons to work and we are taking plugin, scenery and aircraft bugs seriously.
By comparison, the art controls are “do what you want, but you void the warranty if you mess with this.” If you are running scripts that hack the art controls, we cannot tell the difference between real bugs in the early betas and art controls screwing things up.
The Road Map For Betas
Looking over the bug reports we have received, I think we are going to take on the 11.50 bugs in three phases:
Stability and compatibility. We’ll start by making sure that we run Vulkan and Metal on every platform that should be able to run them, with add-ons just working in the cases where we expect them to. We’ll start by focusing on fixing crashes, black screens, device lost, unstable plugins, etc.
VRAM use. We’ve received a number of reports that make it sound like VRAM management is not working properly. Once we can run, we’ll dig into blurry textures, running out of VRAM, etc. Sidney has built some great tools to get a good picture of how VRAM is being managed. VRAM management is one of the newest and most complex parts of 11.50 so it isn’t surprising that we’ve seen things that look buggy.
Performance. Once we are running where we should and using VRAM that we should, we can look at the cases where users are not seeing performance benefits from Metal and Vulkan, as well as remaining stutters. Once again, Sidney has built some fantastic tools that should help us dig into this quite efficiently.
This is the only order that we can reasonably approach the bugs. If the app won’t run on all qualifying hardware, we can’t test our VRAM use everywhere, and if our VRAM use isn’t correct, it can bias performance testing.
Today is the day. X-Plane 11.50 public beta 1 is now available for download to everyone. (Steam users: the beta is staged on the servers, and we’ll let it loose this afternoon if it isn’t setting people’s machines on fire.)
I want to extend my appreciation to the 50+ third party developers who participated in the eleven (!) private developer previews. This is probably the most complex single patch we have developed with regards to third party add-ons; their help testing and trouble-shooting issues was invaluable.
Like all public betas with the number “one” in the title, this is a very new, very untested beta; if you wish to try it, maybe keep your regular X-Plane install around. If you post in the comments that you got the beta and now you can’t fly and you are sad, probably the other commenters are just going to say “I told you so” a few hundred times.
Getting The Beta
Non-Steam users can get the update by running the installer, checking the “get betas” box, and running; Steam users will be able to get beta soon by picking public betas in the application properties.
If you can run X-Plane 11, you should be able to run X-Plane 11.50; if you have an old operating system or hardware, you may be stuck running X-Plane 11.50 with only OpenGL as the driver. Here’s a break-down of what you need to run with the Vulkan or Metal drivers:
Mac users need 10.13 or newer.
Windows users need Windows 10—we do not support Vulkan on Windows 7.(1)
Linux users need a distro that can run reasonably recent proprietary NVidia or AMD drivers.
GPU Hardware (Windows and Linux):
NVidia: GeForce 600 seres or newer
Any GCN or newer AMD card (E.g. HD7000 series or newer)
GPU Hardware (Mac):
Any GPU that comes with a Mac that can run 10.13 or newer or
Any eGPU supported by Apple
Hackintoshes: we have heard anecdotally that GeForce 10 series on a Hackintosh does work, but we do not have this hardware in the company, so YMMV.
Drivers (Windows and Linux):
NVidia: 440.26 or newer
AMD: 19.12.3 or newer
To run with Vulkan or Metal, there is a new check-box in the rendering settings screen – just check the box and restart. If Vulkan or Metal crashes on startup, we will turn the check-box off automatically so you are not locked out of X-Plane.
(1) In theory Vulkan should run on Windows 8, but our devs are all running Windows 10. If you have Windows 7, please update your OS – Microsoft isn’t security patching Windows 7, so it’s more than time.
The First Run Will Take a While To Load
For some users, the first time you start the sim in Metal or Vulkan you may see several extra minutes of load time while X-Plane compiles shaders. This will not happen every time you start the sim, so my advice is: get a beverage appropriate to the time of day and let the sim load.
What you’re seeing is shaders compiling. X-Plane 11.40 compiled shaders while you flew, resulting in stutters. Try this in 11.40: start the sim cold and dark at night, and turn on the battery and landing light. Feel those stutters? That’s the sim going “oh noes, I need the terrain shader with the landing light now”.
X-Plane 11.50 loads every shader it could need up front during preload of scenery, but that’s a lot of combinations. On the first load, the compiled shaders are saved to a disk cache, so load times will be back to normal the next time around.
We are working to make the first load faster, but because it’s a one-time thing it didn’t seem like a good reason to keep everyone out of the beta.
Will My Add-Ons Work?
First, we expect every add-on that does not use undocumented hacking to “just work” in OpenGL – if you find this is not the case, please report a bug.
We expect most add-ons to just work in Vulkan and Metal; we are keeping a list in the release notes of add-ons that we know do not work and require an update from the developer.
Please go easy on your third party developers – in some cases they will need to update code that has “just worked” for over a decade, and the author of a popular add-on will probably hear from users about updating to Vulkan hundreds of times a day. Some add-on authors are part-time or have day jobs, so as a community we will need to be patient.
Third party developers: we should have public docs and an SDK update by the end of the week with complete info about 11.50, but if you were not in the developer preview program and want to dig in now, ping us.
I Found A Stutter!
While we have made tremendous progress with stutters compared to X-Plane 11.41, I can tell you that as of 11.50 beta 1, we are not done, because Sidney is working on fixes for stutters that did not make the first public beta.
The good news is, we’ve reached a point where we can identify and kill stutters in a straight-forward manner – it’s never just “dark matter in the OpenGL driver” anymore. We also have tools built into the sim to collect stutter reports from users – we’ll help you use that once we kill off the remaining known ones (which appear to be in Windows itself – that’s how far we’ve come).
I Found A Bug!
The release notes do list some known bugs, some of which we already have fixed for an upcoming beta 2. If you find something you think is a bug, please file it. Commenting about the bug here, emailing us, and discussing it on a forum do not count! If you file it, it goes into our tracking system so we don’t lose it.
It has been almost a month since my last post on Vulkan and the state of development; a lot has happened since then, both for Laminar Research and the world.
First, at this point, no one working for Laminar Research has COVID-19. We have people working on X-Plane in at least six countries, including northern Italy. The team news on Slack as of Tuesday was that everyone is in some degree of lockdown and/or voluntary social distancing, varying by country. Schools in the US are temporarily closed, so both Chris and I have the kids at home.
Being sequestered at home has not affected our internal pace of development because our entire team is distributed and works at home; we have no office to close. This work-at-home pattern started almost twenty years ago when Austin had people like me working on part-time contracts to modify specific parts of the sim – it never made sense to relocate people to South Carolina and open up an office. Being remote also lets us hire people anywhere in the world who have the unique blend of skills we’re looking for.
At this point we haven’t seen any operational issues either; mostly running our business requires that our devs have internet and electricity and that the data centers we use for our cloud servers remain open and operational.
So in summary, we are very fortunate that the new coronavirus is not affecting our development and operations; other industries are paying a high price to stop the spread early. We have several things we are working on right now, and this spring is a critical time in our development schedule.
Vulkan and Metal
Update: I have gone in and changed Vulkan to Vulkan/Metal in a bunch of places – Mac users were confused as to whether we had silently dropped support for Metal at the last minute – we have not!
We are posting developer preview nine to our private testers today; hopefully this will be the last private build before a public beta candidate. We acknowledge we’re reaching public beta much later than we had hoped/wished/anticipated, but we are now aiming for a public Vulkan/Metal beta by the end of March. If you are already rolling your eyes at a public beta date that is less than two weeks away, I don’t blame you; having been this late, it’s on us to post the beta to show progress. With that in mind, I am going to describe what we’ve been doing for the last three months and why it has taken so long to get here.
The Vulkan/Metal public beta is several months later than we had anticipated because of “scope growth” – that’s manager nerd speak for “we added more stuff to it than we originally planned.” Scope growth (adding more features/code/tricks than you originally planned) is one of the big ways that projects miss original deadlines, so the big question is: what did you add and why did you add it?
The first thing we added was better handling of plugin drawing. The rewritten plugin compatibility layer provides better drawing compatibility for more plugins, including weather plugins on Windows. We went with the rewrite mostly because of the level of bugginess we saw with third party add-ons in the early developer previews.
Plugin drawing was definitely a case where we learned how to do a better job from the first version of the feature (plugin compatibility that went into the first private beta); if we had received that second-generation design via time machine we probably could have shipped faster. Adding weather support was pure feature creep–a new thing we didn’t plan on, but something that we thought was worth extra schedule time.
The second thing we added was better handling of texture paging. Once again, this was a feature where we had to rewrite the feature (multiple times in fact!) based on the feedback we got from our testers to really come up with something solid.
Our first generation texture paging around the first private developer preview was very simple: most stuff lived in VRAM, with a little bit of code to move unused stuff out of VRAM. It was a minimalist strategy that let us develop the rest of the sim and worked great on high-end cards. It was clear from day one that it wouldn’t be good enough for public beta.
Our second generation strategy added automatic movement of texture res up and down with VRAM pressure and code to page out textures that weren’t being used. This shipped about half way through the private beta, and was better, but suffered from one fatal flaw: as you turn your head, the stuff behind you isn’t being used. Under heavy memory pressure users would constantly turn their head and see blurry textures that would then “res up”. The results were distracting and unacceptable quality-wise.
We now have finished our third generation strategy: besides automatic texture res control based on VRAM pressure, we now set the relative resolution of non-orthophoto textures by distance to the aircraft. A background task on another core “crawls” the scenery near your aircraft and reevaluates the texture res of nearby scenery constantly, effectively transferring VRAM to where it is needed most. This process is completely transparent; authors do not need to modify scenery in any way for it to work, and since it runs on another core (as does the paging), it does not affect framerate.
In these pictures, you can see the new pager at work. I have parked my aircraft on the ramp at SeaTac and moved the camera across the city, so that the distant autogen (from the pilot’s perspective) is now close and the airport is in the background.
In this first picture, green represents full-res textures, with the next levels down being yellow, then magenta. You can see some nearby autogen down one level and far-off autogen down two levels; the loss of res is put far from the user.
Why is there so much green (full res) near the magenta autogen? Texture reuse. If a texture is used near the user and far from the user, it gets high res because it might be seen up-close.
In this next picture, I artifically lowered my machine from 4 to 1 GB of VRAM using a developer tool. Now you can see magneta autogen close to the airport, yellow even closer, and some really dark green (the next level down) where we had magenta. In other words, everything got a little bit lower res because we were tight on VRAM, but again nearby sceney is prioritized.
It looks like this third strategy is a keeper – it combines the careful management of VRAM to use it where it is most needed with a stable heuristic that isn’t constantly changing, which avoids chaos and unreliable behavior.
So at this point we are just fixing bugs. This is a big step forward for the private beta because past betas were dominated by coding new things, which in turn creates new bugs. At this point we just have to kill bugs off to the point where the build is high enough quality for a public beta.
I cannot promise a particular build number or date will be the public beta full stop, because I do not know yet what other bugs we may find. But I can say that we are in the part of bug fixing where things are getting locked down.
Vulkan is Not the Only Iron In the Fire
A flight simulator is more than its rendering engine (or at least, people tell me that). While we are pushing as hard as we can to get Vulkan and Metal to public beta, we also have several other irons in the fire. This includes new features for mobile, next-gen features for desktop in several feature areas, and even a few extra non-Vulkan/Metal features that have snuck into 11.50. We’ll post more about the mobile and 11.50 features as they become available; the rest is still in stealth mode. Often when the dev blog is quiet, it means we’re working on new things and putting features in the bank; this is the case right now.
As always, make backups! Well folks, we are finally feature complete, very stable, and as bug free as we can be at the moment! This update was for some small bugs and to wait and see if there would be feedback on alpha.6.
When attempting to export a project with no exportable collections or objects you’ll get an error asking if you forgot to check any Exportable Collection|Object checkboxs, Hopefully this helps beginners find those checkboxes and get started.
#504 Where, when the collection algorithm starts in the middle of the tree, walks up, and re-enters the exportable collection it started in, it now respects the visibility and ensures the XPlaneBone is re-used. For an example of this weird case see parent_out_of_collection_snake_out.test.blend
#514, #536, #538 which all related to making the Hidden or Visible feature work as intended – not including what shouldn’t be included. Hidden Lights with the Default, Strobe, Traffic, or Flashing lights no longer include the VLIGHT table entry. ATTR_manip_ is not included across split animation OBJ boundaries, Objects hidden by other means than their or their parent’s visiblility are still counted as hidden (“True WYSIWYG”)
Sorry for keeping XPlane2Blender in Alpha for so long. I develop everything with the knowledge that the artist community will jump on the latest and greatest as soon as it is out there and strive for the greatest possible quality at all times. In the future I think we’ll not use the label as much. It caused a lot of concern about whether or not it should be used, even when it was quite ready.
XPlane2Blender is developed using Blender 2.80 because some Linux Distros are slow to update Blender, and some people are slow to update Blender versions. If you are using Blender 2.82 it should work, but, tell me how it goes. I haven’t heard anything bad yet.
As always, please report any bugs you find and thank you for using XPlane2Blender!
The latest update for World Editor is out of beta testing and considered the official version. It can now be used to upload scenery to the Gateway, and is an incremental update that features the ability to preview facades in 3D, faster loading, and up to 100 “undo” operations.
For the last few weeks we’ve had the Vulkan/Metal version of X-Plane in a private test program with third party developers, letting them kick the tires to discover the limits and bugs in our plugin compatibility code.
While the initial bridge to connect OpenGL-based plugins to X-Plane worked pretty well, compatibility wasn’t perfect and we had a few bugs. We also realized that we needed to change the design of that bridge to get much better compatibility. That rework of the bridge is now done and things are looking quite a bit better on all fronts.
We found one unexpected result from this redesign: when we finished rewriting the bridge, we ended up with code that could, in the case of Vulkan only, be used for 3-d drawing. This means we can now support OpenGL-based weather/cloud plugins from inside a Vulkan render, which we did not think was possible before.
The Skymaxx and xEnviro developers are in our private test program and have been great about jumping on new builds while remaining radio silent while we test this new tech. I am pleased to now be able to say: it works!
Those pics are a special build of SkyMaxx running in 11.50 in Vulkan mode. No more having to pick – you can have your third party clouds and Vulkan at the same time.
The Details: Vulkan, not Metal
3-d drawing compatibility will be available for Vulkan, but not Metal. The technique we are using is not available on OS X, so running a cloud plugin on Mac will require the OpenGL driver, not Metal. This isn’t something we can fix with some future bit of cleverness – it’s a limitation of OS X’s APIs. (We would revisit this if Apple introduced a new API but I do not expect this to happen.)
To be clear: on Windows what we are supporting is 3-d OpenGL drawing by plugins inside the Vulkan renderer, not native Vulkan rendering by plugins inside the Vulkan renderer. Native Vulkan rendering by plugins is an incredibly complex SDK problem, and not something we are looking at for 11.50.
2-d Drawing Just Works; 3-d Drawing Requires a New Plugin
For plugins that draw in 2-d we’ve gone for a “just works” level of compatibility; while we have seen some plugins that need to update due to illegal techniques and bugs, our goal is that a well-authored add-on just works even when Vulkan and Metal are in use.
By comparison, OpenGL drawing inside Vulkan is going to require plugin authors to sign up for a new 3-d drawing callback. We’re making this opt-in because, during the beta, we’ve found a lot of add-ons using 3-d drawing callbacks inappropriately, and 3-d drawing callbacks are no longer free. By requiring plugins to opt-in when they _really_ want 3-d drawing in Vulkan, we should get better performance.
The new 3-d drawing callbacks are really only meant for custom drawing like clouds. Other techniques like adding objects to the sim and drawing markings are better handled with other APIs. Our plan is to have detailed guidance on what technique is best for developers by the time we are ready for public beta.
As always, make backups! This includes making backups before saving a 2.79 file in 2.80. There is no going back from that!
Override LODs feature and backwards compatibility
A quick LODs recap
This release includes the last must-have feature for 2.80 – LODs. For those who haven’t used LODs before, it stands for (Levels of Detail).
It is a way of defining what to draw when the camera is a certain
distance away from an OBJ. In XPlane2Blender we call those ranges
“buckets” and put Blender Objects (meshes, lights, armatures, empties)
“inside them”. For instance, suppose an OBJ of a hanger with 2 defined
ranges (buckets) 0 to 200 and 200 to 400. It has two meshes
“HangerDetailed” (bucket 1) and “HangerLowPoly” (bucket 2).
When the camera is close (0 to 199 meters), only the “HangerDetailed”
mesh will be drawn. When it is far (200 to 399), only “HangerLowPoly”
will be drawn. At 400 meters and above, nothing will be drawn. This is
very useful for increasing the performance by not drawing what is too
far away for the pilot to see.
There are two styles of Level Of Detail: Additive (where every bucket
starts at 0) and Selective (where the end of one bucket and start of
the next are equal (see example above)). Use one or the other depending
on what you need to draw when; don’t mix between them.
LODs Mode in XPlane2Blender
Using LODs in XPlane2Blender is very easy: Define the ranges
(buckets) in the OBJ settings, and tell Blender Objects which buckets
they should go in.
“LODs Mode” is considered On when you have specified a number of LOD buckets in the OBJ settings.
This means that:
LOD validations are on (see later section)
Objects must be placed into defined buckets or they will not be
drawn. This is where the new “Override LODs” checkbox on Blender Objects
comes into play.
The LODs feature works just like 2.79’s Layers Mode LODs, with a new
time saving feature and some very small backwards compatibility notes.
In the Object Properties Tab of the Properties Editor, you’ll see a new “Override LODs” checkbox.
When checked, the familiar 4 LOD bucket choice checkboxes will appear.
Specify which buckets an Object will go into. All children below it
will also go into the same buckets. Children who also override LODs will
pass their new choices on instead.
In this picture HangerDetailed has its “Override LODs” checkbox checked,
and has been placed into bucket 1. Its child, WindowDetails, will
automatically be placed into bucket 1 too.
An Object’s LOD choices will only be used if “Override LODs” is checked.
The idea is that just a few objects in the Blender Hierarchy will
need to override the LOD choices, and old projects can be quickly
reorganized to take advantage of this new feature. Inheritance will
greatly reduce data entry.
There are only a few differences from 2.79 to be aware of
Backwards Compatibility Concern
In 2.79 “No buckets chosen = Write to every LOD“. In 2.80 “No buckets chosen = Object not written“
Use the new feature to quickly specify which objects should be in all buckets
2.79 Layers Mode projects did not have an “Override LODs” checkbox, LOD choices hidden
Your old choices are remembered. Use the new Override LODs feature to quickly get the OBJ correct again
2.79 Root Objects Mode used layers 1-4 to put Objects into buckets
Quickly restore your project like this: Make an Empty for each layer
used, override the Empty’s LOD buckets, and re-parent the objects so
that all Objects are in their former buckets. If an object was used on
more layers, make more empties to organize these cases
For mistakes like not starting the first bucket at 0, having gaps or
overlaps between ranges, and etc, an error will be emitted in the log.
Other Bug Fixes
#518 – When Export Type is “Cockpit” LOD buckets is now shown.
– “Deck” (a rarely used Material property which controls how planes
collide with this surface and the area beneath it) is now renamed to
“Only Slightly Thick” implying that this collide-able surface is “Only
Slightly Thick” as opposed so something that “extends the earth up to
that surface”. Imagine a bridge that could not be flown under because
under it is “an invisible mound of rock and world surface” vs a bridge
that can be flown under because “its deck is only
slightly thick”. Get it? Well, maybe the new name will make more sense
to everyone without needing to know this example.
#511 – People will now be able to use a Particle System File again
– Some UI cleanup. “Scenery Properties” will no longer be shown when
Export Type is Aircraft or Cockpit, “Autodetect Textures” is no longer
drawn because it doesn’t do anything and confuses people on how to use
an essential feature: specifying texture paths. The value isn’t removed
and one day we’ll have that feature back in some form.
Many thanks to all the people who e-mailed me and helped me design
the new LOD feature. This is an alpha feature so if the overall response
is “this isn’t doing what we need” it can be ripped out, but, I think
it will meet expectations. Also, especially tell me if it difficult to
take a 2.79 project with LODs and make it work again. The goal is that
it an artist should be able to do it in about 5 minutes without the use
of an updater for most projects.
I’m so glad to be working on a project with users who are very
willing to send in good bug reports and great constructive feedback and
criticism! Very refreshing! As always, if something goes wrong write a
bug report (preferable) or e-mail me!
One thing I have learned during my time here in the matrix is that users often find new bugs to be significantly more annoying than existing ones; trading an existing bug for a new one is often seen as a big step backward, even if the new bug isn’t that bad. I think this is just human nature – we get used to things – it’s what humans do – and this takes the sting off of some bugs that have been around a while.
I bring this up to put in context the problem of texture paging, blurry textures, and VRAM management in 11.50. To understand what makes VRAM paging hard, you have to understand how the OpenGL driver solved the problem, and ask yourself: what was the old bug, what is the new bug?
What Is Texture Paging?
Texture paging is when textures are moved between the memory on your graphics card (VRAM) and the memory in your computer (“system memory”). At any given time, X-Plane may have more textures loaded than you can fit on your card, but it doesn’t need them all at once. As you fly, different textures are needed, and the textures have to be moved around.
If this sounds a lot like moving data from RAM to your hard drive (virtual memory), it is. When I get the question “how can X-Plane be using 2.5 GB of texture memory when I only have a 2 GB card” the answer is…texture paging.
How Did the OpenGL Driver Do It?
In an OpenGL based app, paging textures to VRAM and back is entirely up to the OpenGL driver. The algorithm goes something like this:
Realize something you need is in system memory.
Move something you haven’t used in a while out of VRAM.
Move the thing you need into VRAM.
The OpenGL driver pages during drawing when we need something new, and the result is a stutter – the frame takes longer because it cannot be completed until the texture is moved to VRAM.
Now NVidia, AMD, and Apple have spent a lot of time and effort to make this work as well as they possibly can, and I’m definitely selling short the complexity and sophistication of the drivers as they try to guess really well what should be moved out of VRAM. Still, two key facts will be true:
If we need to page stuff into VRAM to draw, and we don’t figure that out until right when we need it, there’s going to be a stutter. Maybe a really short one you barely notice, maybe a really bad one, it’s a question of “how much”, not “if”.
What you see on screen always looks fantastic – never blurry – because the driver won’t cut resolution to save VRAM.
So…the old bug in this case is stutters due to VRAM containing the wrong stuff.
How Does X-Plane 11.50 Do It?
With Metal and Vulkan, it’s up to us, the app, to decide what goes in VRAM and what doesn’t, and to schedule the movement of data between VRAM and system memory. Our strategy is:
Keep an eye on how much memory is free.
If it looks like it’s getting too close to full, shrink textures to a smaller size. Prefer to shrink textures you’re not looking at. Do the shrinking in the background.
If it looks like VRAM has empty space free, grow textures, preferring ones you are looking at now. Do the growing in the background.
If we have to load new textures in the background (e.g. new DSFs as you fly), wait until old textures can be shrunk.
We have a huge advantage over the drivers in this game: we know the textures we might need in the future, long before we actually need them. We can also shrink textures. This means we don’t have to stop and shuffle things around, and that means we can avoid stutters. So we can fix the old bug: stutters while you fly.
(To be clear: this is not the only source of in-flight stutters while you fly – OpenGL and 11.41 are full of them.)
The down-side is: if VRAM gets tight, we’ve made some of the textures lower resolution. This is a new behavior (blurry textures), so my fear is: users aren’t going to like it.
In particular, the OpenGL driver uses the exact optimal set of textures for every single frame – at the cost of stuttering. We may not have things shrunk and grown in exactly the best way while you fly, so for the same VRAM and no stutters, we may not be as visually sharp.
What About Plugins
Add-ons that use OpenGL (e.g. aircraft with custom glass displays) use VRAM too – this is one reason why we can’t pack 100% of VRAM with our textures; we need to have enough free that if a plugin needs VRAM, it won’t crash us.
My expectation is that fighting blurry textures and crashes due to running out of VRAM is going to be a long, iterative process and one of the main reasons to have beta testing. It’s one thing to have an algorithm that works good in the lab or on paper – it’s another to have thousands of hours of test time in real-world conditions on user machines with lots of add-ons and anarchy.
So when 11.50 is public beta and you see your blurry texture, don’t panic. VRAM management is a very new part of X-Plane, and it’s a very important part, and it’s something we will iterate on over time.
(Finally, I want to mention that while the port to Vulkan/Metal has been a joint venture that Sidney and I have worked on together, there are certain parts of it where Sidney basically did 100% of the work, and my only contribution was to complain about bugs. The shader compiler, the OpenGL plugin bridge, and the VRAM pager are all big lifts that Sidney has done on his own that make the Vulkan/Metal port possible.)
As I announced in my previous post, X-Plane 11.50, featuring native Vulkan and Metal driver support, is in a private beta made up of people in our dev-rel program (E.g. third party developers). We’ve gotten some really good bug reports from them – the reports are good, not the bugs – and based on that, it’s clear that 11.50 will not be ready to go public this year (meaning tomorrow).
This is disappointing to us – we burned candle at both ends to try to get all the way to public beta this year, but at this point the build just isn’t ready yet.
I’ll describe in detail below what the one bug is that is really holding back public beta, and I’ll try to do a general FAQ about Vulkan and Metal tomorrow; based on the 150+ comments from the last post, I think it’s clear that we have a lot of basic info to get out there, some of which has only been shared with third party developers until very recently.
Please don’t bombard me with “can I please be in the private beta” (or even “you’re a giant pile of moose poop for not letting me into the private beta”). Here’s the thing about the private beta: our goal is to kill off the bugs so everyone can use the beta, and to kill them off as rapidly as possible.
At this point the dozen or so third party developers who are banging on the beta are generating bug reports faster than Sidney and I can fix them – if we add more people to the beta, our bug fix rate will slow down as we have to spend more time triaging the reports coming in. So as long as we’re fixing the worst bugs productively, more people in the private beta means everyone waits longer to get to public beta. I think the best thing here is for Sidney and I to just try to fix things ASAP.
(I think there’s also a win to getting this into the hands of third party developers – in some cases we’ve seen add-ons that accidentally use the wrong APIs for processing and are thus incompatible with Vulkan when they don’t need to be – developers can get a head start on updating this code.) If you are a third party developer, we can get you into the beta program – I don’t want to favor some third party developers over others.
A short summary of performance feedback from our internal beta might go like this: smoothness and FPS are good, VRAM management is not.
Most of the crashes while flying we have seen look like the sim runs out of VRAM and isn’t able to recover, and for users with smaller GPUs (e.g. 4 GB of VRAM) X-Plane gets into a tight VRAM situation and solves it by making everything on screen low res and blurry.
The VRAM management path, which is entirely new code written for the Vulkan/Metal back-ends, is…well, it’s entirely new, and it’s going to need a bunch of testing, debugging and iteration to get solid. On an 8 GB card, VRAM is so plentiful that X-Plane’s somewhat naive VRAM allocation scheme works fine. On a 4 GB card, we’re just a tiny bit short (a few hundred MB) but the result is an alarming loss of texture resolution.
This is something we can make better! It is under our control and is thus being worked on now. (By Sidney that is – I’m writing this blog post while he gets the real work done here.) But until we have better VRAM management, it’s too soon to go public. If we did, we would just be inundated with hundreds of “my textures are blurry” reports. It’s a bug too noisy to ship with.
Why Wasn’t This a Problem on OpenGL?
One might ask: why don’t you just do whatever OpenGL did to manage VRAM? The problem is: OpenGL’s solution is to stutter.
The OpenGL driver swaps textures between VRAM and system memory based on what you really need to draw now. If you are looking away from Seattle, the space needle’s texture can live in system memory, but you might need mountain textures for Rainier.
We try to do that too – the big difference is: OpenGL will stop rendering while it moves the textures around, and we will not. The result is that OpenGL always shows you a perfect image at the texture resolution you picked – no blurred images. But you might have stutters! You can see this if you’re on the margins of VRAM by flying and then changing views or circling the camera – if your FPS go “chunk chunk chunk VROOOM” you’re eating some stutters while textures move around, then you go fast again.
The OpenGL way is a solution that we never thought was acceptable – our goal is the best image quality without sacrificing smoothness. It’s going to take a little more time to iron out low VRAM situations, but once we get there I think it will be worth it.