The latest update for World Editor is out of beta testing and considered the official version. It can now be used to upload scenery to the Gateway, and is an incremental update that features the ability to preview facades in 3D, faster loading, and up to 100 “undo” operations.
For the last few weeks we’ve had the Vulkan/Metal version of X-Plane in a private test program with third party developers, letting them kick the tires to discover the limits and bugs in our plugin compatibility code.
While the initial bridge to connect OpenGL-based plugins to X-Plane worked pretty well, compatibility wasn’t perfect and we had a few bugs. We also realized that we needed to change the design of that bridge to get much better compatibility. That rework of the bridge is now done and things are looking quite a bit better on all fronts.
We found one unexpected result from this redesign: when we finished rewriting the bridge, we ended up with code that could, in the case of Vulkan only, be used for 3-d drawing. This means we can now support OpenGL-based weather/cloud plugins from inside a Vulkan render, which we did not think was possible before.
The Skymaxx and xEnviro developers are in our private test program and have been great about jumping on new builds while remaining radio silent while we test this new tech. I am pleased to now be able to say: it works!
Those pics are a special build of SkyMaxx running in 11.50 in Vulkan mode. No more having to pick – you can have your third party clouds and Vulkan at the same time.
The Details: Vulkan, not Metal
3-d drawing compatibility will be available for Vulkan, but not Metal. The technique we are using is not available on OS X, so running a cloud plugin on Mac will require the OpenGL driver, not Metal. This isn’t something we can fix with some future bit of cleverness – it’s a limitation of OS X’s APIs. (We would revisit this if Apple introduced a new API but I do not expect this to happen.)
To be clear: on Windows what we are supporting is 3-d OpenGL drawing by plugins inside the Vulkan renderer, not native Vulkan rendering by plugins inside the Vulkan renderer. Native Vulkan rendering by plugins is an incredibly complex SDK problem, and not something we are looking at for 11.50.
2-d Drawing Just Works; 3-d Drawing Requires a New Plugin
For plugins that draw in 2-d we’ve gone for a “just works” level of compatibility; while we have seen some plugins that need to update due to illegal techniques and bugs, our goal is that a well-authored add-on just works even when Vulkan and Metal are in use.
By comparison, OpenGL drawing inside Vulkan is going to require plugin authors to sign up for a new 3-d drawing callback. We’re making this opt-in because, during the beta, we’ve found a lot of add-ons using 3-d drawing callbacks inappropriately, and 3-d drawing callbacks are no longer free. By requiring plugins to opt-in when they _really_ want 3-d drawing in Vulkan, we should get better performance.
The new 3-d drawing callbacks are really only meant for custom drawing like clouds. Other techniques like adding objects to the sim and drawing markings are better handled with other APIs. Our plan is to have detailed guidance on what technique is best for developers by the time we are ready for public beta.
As I announced in my previous post, X-Plane 11.50, featuring native Vulkan and Metal driver support, is in a private beta made up of people in our dev-rel program (E.g. third party developers). We’ve gotten some really good bug reports from them – the reports are good, not the bugs – and based on that, it’s clear that 11.50 will not be ready to go public this year (meaning tomorrow).
This is disappointing to us – we burned candle at both ends to try to get all the way to public beta this year, but at this point the build just isn’t ready yet.
I’ll describe in detail below what the one bug is that is really holding back public beta, and I’ll try to do a general FAQ about Vulkan and Metal tomorrow; based on the 150+ comments from the last post, I think it’s clear that we have a lot of basic info to get out there, some of which has only been shared with third party developers until very recently.
Please don’t bombard me with “can I please be in the private beta” (or even “you’re a giant pile of moose poop for not letting me into the private beta”). Here’s the thing about the private beta: our goal is to kill off the bugs so everyone can use the beta, and to kill them off as rapidly as possible.
At this point the dozen or so third party developers who are banging on the beta are generating bug reports faster than Sidney and I can fix them – if we add more people to the beta, our bug fix rate will slow down as we have to spend more time triaging the reports coming in. So as long as we’re fixing the worst bugs productively, more people in the private beta means everyone waits longer to get to public beta. I think the best thing here is for Sidney and I to just try to fix things ASAP.
(I think there’s also a win to getting this into the hands of third party developers – in some cases we’ve seen add-ons that accidentally use the wrong APIs for processing and are thus incompatible with Vulkan when they don’t need to be – developers can get a head start on updating this code.) If you are a third party developer, we can get you into the beta program – I don’t want to favor some third party developers over others.
A short summary of performance feedback from our internal beta might go like this: smoothness and FPS are good, VRAM management is not.
Most of the crashes while flying we have seen look like the sim runs out of VRAM and isn’t able to recover, and for users with smaller GPUs (e.g. 4 GB of VRAM) X-Plane gets into a tight VRAM situation and solves it by making everything on screen low res and blurry.
The VRAM management path, which is entirely new code written for the Vulkan/Metal back-ends, is…well, it’s entirely new, and it’s going to need a bunch of testing, debugging and iteration to get solid. On an 8 GB card, VRAM is so plentiful that X-Plane’s somewhat naive VRAM allocation scheme works fine. On a 4 GB card, we’re just a tiny bit short (a few hundred MB) but the result is an alarming loss of texture resolution.
This is something we can make better! It is under our control and is thus being worked on now. (By Sidney that is – I’m writing this blog post while he gets the real work done here.) But until we have better VRAM management, it’s too soon to go public. If we did, we would just be inundated with hundreds of “my textures are blurry” reports. It’s a bug too noisy to ship with.
Why Wasn’t This a Problem on OpenGL?
One might ask: why don’t you just do whatever OpenGL did to manage VRAM? The problem is: OpenGL’s solution is to stutter.
The OpenGL driver swaps textures between VRAM and system memory based on what you really need to draw now. If you are looking away from Seattle, the space needle’s texture can live in system memory, but you might need mountain textures for Rainier.
We try to do that too – the big difference is: OpenGL will stop rendering while it moves the textures around, and we will not. The result is that OpenGL always shows you a perfect image at the texture resolution you picked – no blurred images. But you might have stutters! You can see this if you’re on the margins of VRAM by flying and then changing views or circling the camera – if your FPS go “chunk chunk chunk VROOOM” you’re eating some stutters while textures move around, then you go fast again.
The OpenGL way is a solution that we never thought was acceptable – our goal is the best image quality without sacrificing smoothness. It’s going to take a little more time to iron out low VRAM situations, but once we get there I think it will be worth it.
Over the past month people have reported to me all sorts of weird things:
OBJ geometry moving with every export
Blender Objects moving after every export
Animations spiraling out of control
Even simple renaming Blender Objects and restructuring of the Blender hierarchy might trigger a bad export.
And the only cure was to apply all transformations or restart Blender!
Thanks to your bug reports and e-mails I was able to track down the source: all these problems were facets of the same bug!
To make a long story short: I have a fix for XPlane2Blender, but
other Blender exporters and users may not be so lucky. I’ve filed a bug
with Blender itself, but that is nothing to worry about for X-Plane
Please continue to send me your bug reports! I do read every one of them, even if I don’t get back to you right away.
X-Plane 11.40 is now final! You will be prompted to update to X-Plane 11.40 when you start X-Plane; Steam users will receive the update automatically via Steam. Here’s what’s next in the pipeline:
We are working on a bug-fix update (11.41) to catch one or two bugs that didn’t make the RC, as well as the inevitable bug that will be reported after go final. I expect to cut an 11.41 release candidate some time next week, and it should be a pretty quick release.
In the meantime, we are pushing hard to get Vulkan/Metal ready so that we can do an X-Plane 11.50 beta. We may start private testing of Vulkan and Metal before 11.41 is done, depending on what gets fixed first.
The limiting factor on getting to a public Vulkan/Metal beta will be bug fixing – it doesn’t make sense to go public beta with known bugs that will be reported over and over and make the beta unusable. There are also two remaining areas of development we need to close up:
Vulkan and Metal fight stutters by pre-loading everything they need to render the frame – this means whatever we need to draw, it’s ready. But this also means X-Plane needs to prepare every shader it might need to render a given aircraft and scenery pack.
Because X-Plane’s shaders are so flexible, this can mean tens of thousands of shaders, and that hurts load time. We have new code that tries to analyze exactly what we’ll need for a given frame. For example, if an object comes from a DSF, we never need an “interior” shader for it because the scenery object can never be inside an aircraft. By carefully applying this kind of logic, we can cut down the number of preloaded shaders and speed up load times.
With Vulkan and Metal, X-Plane manages its own use of VRAM – this is a totally new capability that we have had to code from scratch for the Vulkan/Metal port. We watch our total memory usage and reduce texture resolution dynamically (hopefully on less important and less used textures) to stay within budget.
While this code is working, it hits a wrinkle on Windows:
Third party add-ons that chew up VRAM via their OpenGL VRAM usage can use up part of our budget. For example, if you change to an aircraft with a plugin that allocates its own textures to render to, we can’t use that VRAM anymore.
The Vulkan version of X-Plane allocates VRAM in big blocks for efficiency. If we reduce our texture res to make room for the plugin, we might end up with a bunch of big pre-allocated blocks (that are mostly unused) inside Vulkan – the memory still isn’t usable by the OpenGL plugin.
Sidney is working on compacting our textures so that some of the big blocks can be given back to OpenGL – yet more new memory management code for Vulkan.
There Be Dragons
Once we have the number of shaders cut down and memory compacting working, we’ll begin heavier testing of the Vulkan/Metal build – I can’t say how close to public beta we are because we haven’t gotten the initial wave of bugs from this testing.
Release Candidate 2 is now available for LR customers, and we expect Steam to be available within 24 hours or so.
This is a very small patch with a few minor fixes. At the risk of sounding like a broken record: if you haven’t tried your add on aircraft in 11.40 yet…do it now! The reports have been pretty quiet the last couple weeks, and nobody enjoys a last minute surprise bug.
Beta 9 is now available for LR customers, and we expect Steam to be available within 24 hours or so.
This update includes a few more fixes for crashes and issues with third party add ons. The rate of bugs and crashes reported has slow down dramatically in the last week or two, and we don’t have open third party compatibility bugs, so we are getting ready to wrap up the beta.
Developers: expect r1 soon, and make sure you test your add ons.