Back in June, Ben shared some information regarding X-Plane’s next generation lighting pipeline. It’s now time to go back to the future and talk about another piece of the next generation: Vegetation. In case you have missed it, Chris and Thomson made an awesome preview video for this as well:
As you can see, vegetation got a big upgrade from what we currently have. Instead of one or two quads wedged into each other, we now have fully modeled and animated 3D trees, and lots of them. The vegetation animations are based on the current wind conditions, giving them a very nice and dynamic feeling. They also come pre-seasoned for extra flavour, so their look will match whatever local season you are currently in. Naturally this led to a lot of questions, so I’ll try to answer the two most common ones that I have seen:
How do the new trees work and what happens to my old ones?
While the vegetation engine is built from scratch, it uses the existing .for file format. Of course the .for file format received a few additions to be able to deal with 3D meshes as well as some additional parameters, but the key take away here is that if you already have custom forests, they will continue to work. It also means that if you are a scenery author, you will be able to extend your trees into the third dimension and provide updates to your users. All the vegetation that ships with X-Plane has already been enhanced with 3D versions–Petr has done an absolutely phenomenal job crafting these trees. So if you have scenery that is using the built-in forest library, you don’t have to make any changes to get them into your scenery.
The quad based trees will continue to work as before if you have a .for file that hasn’t been updated for 3D trees. They will however take advantage of the new vegetation engine for rendering. Since the forest system uses the same scenery lookup rules as before, it’s also possible to force old scenery with custom .for files to use the new 3D trees using a custom library.txt file. I’m sure there will be at least a handful of people who’ll want to do that. But the general take away here is that the new vegetation system should just work without any compatibility issues.
Lastly, the wind animations: These are done dynamically on the GPU, although they are driven by parameters provided through the .for file and additional vertex attributes on the mesh. This makes it easy to make the trunk of the tree very stiff and resist bending, for example, while the leaves can freely flutter in the wind. This means that you don’t have to rig trees in any way; as a scenery author you can just paint in the different parts of the trees in Blender for example, and let the system drive all the animations.
We will have both tooling and .for file specification updates available for scenery authors in the future, so adopting 3D trees should be very painless.
Surely this is going to be insanely slow, right?
When we set out to create the next generation vegetation engine, we knew that we’d have to build a system that can handle millions of trees. While it’s still too early to give you anything in terms of hardware requirements, I can say that the system is definitely built to be able to cope with a lot of trees. This is possible because the new vegetation engine is designed to use very little CPU processing, and instead moves the majority of the work to the GPU. A lot of the heavy processing for trees also happens very early in the frame, at a time when the GPU was mostly idle waiting for actual drawing commands from the CPU. For anyone curious about the technical aspect of this, we make heavy use of compute to cull the list of trees down to just the ones that are actually visible by the camera. We then use that information to prepare one or more indirect draw calls. This way the CPU can send forests that span multiple kilometers for processing and the GPU will generate the rest of the data all by itself.
Drawing lots of geometry is, of course, more expensive than not drawing it. Even if we only ever draw what is actually on screen, drawing anything isn’t free. So we also employ a classic LOD scheme here. Scenery authors can create multiple 3D LODs for their trees, and there is an additional billboard LOD generated at the end of the chain. The billboards are just flat tree quads that always face the camera, so they’ll still give the illusion of plenty of geometry complexity at a distance. This way we can fill the near view with maybe a few hundred or so complex 3D trees, and then fade over to the cheap billboard to fill the rest of the scene. LOD decisions are also done on the GPU on an individual, per tree basis, so the GPU is quite literally creating its own optimized workload.
The next generation tree tech makes heavy use of features that have only become possible for us thanks to Vulkan and Metal. In fact, it’s not just the tree tech, the new lighting and other next generation features make heavy use of features we finally have access to. The road to Vulkan and Metal wasn’t just to make the sim faster, but also pave the road for what’s to come next. So what does this mean for the venerable OpenGL rendering backend? Sadly, there is no next generation future for OpenGL, as it just isn’t possible to support it. But fear not, this is only the case for X-Plane’s own rendering. Plugins will continue to be able to use OpenGL, just like they are right now under Vulkan and Metal.
So if you have plugins using OpenGL that work with X-Plane 11.55 under Vulkan or Metal, it should continue to work in the next generation world as well. The one exception here being the modern3D drawing phase. Due to the new lighting engine being completely photometric, old school 3D drawing doesn’t work anymore because it’s not in on the joke that pixels aren’t plain rgb8 triplets anymore. I can’t talk about a replacement yet, but this is certainly something you might want to keep in mind if you are about to start writing a new plugin today.
Other than that, custom panel and GUI drawing will continue to work just like it does right now.
With Vulkan and Metal, X-Plane is now firmly in the driver’s seat for VRAM management. This lets us eliminate stutters that were previously present with OpenGL and almost impossible to avoid. It has, however, one big and noticeable downside: when you run out of VRAM, you get blurry textures.
Of course the goal wasn’t to replace stuttering with blurry textures, and we believe that given the normal work load of X-Plane, you should not be seeing this. The fact that so many users are seeing blurry textures, especially on big cards with lots of VRAM, points to the VRAM code being buggy in all the ways beta code can be buggy.
How much VRAM do I need?
Just to get the obvious out of the way, our system requirements have not changed for 11.50. Our minimum VRAM requirement for X-Plane 11 is still 1Gb. We expect these cards to be able to run 1080p with lowered texture resolution, providing an equal experience to X-Plane 11.41. With 2Gb we expect users to be able to run HDR with medium texture resolution on 1080p systems. 4Gb and higher should be able to run HDR at with high resolution textures with 2k monitor resolutions. With 6Gb and higher, 4k monitor resolution should be possible.
Nvidia announced their latest bitcoin graphics cards on August 20th at Gamescom this year. Among the usual increase in transistors, they also disappointed all crypto miners by adding a feature that cannot (yet) be used to calculate cryptographic hashes: Ray Tracing! Ray tracing has long been seen as somewhat of a holy grail of graphics rendering, because it’s much closer to replicating the real world than traditional rasterization and shading. However, doing ray tracing in real time has been close to impossible so far. But hey, Nvidia just announced their new RTX GPUs that can do it, so when is X-Plane going to get a fancy ray traced renderer? This and various other questions that have been asked by X-Plane users, as well as some myths, shall be answered! If you have a question that isn’t answered here, feel free to ask it in the comments.
What Nvidia has shown is absolutely impressive. Unfortunately, the fine print of all the marketing hype is that sadly it can’t just be thrown in without engineering effort. The first thing needed is actual RTX hardware, which no one at LR currently has. The second thing needed is a Vulkan-based app; we are getting there, but not in any way that would support RTX. (the whole goal of the Vulkan renderer is to not change the way the world looks, so we’ll first need a shipping production Vulkan renderer.) But then… well, it’s not entirely clear what it takes to actually write a ray traced renderer in all of its details. Nvidia has not yet published the specification for the Vulkan extension (VK_NV_raytracing), but they have published slides from presentations. One thing is very clear: you can’t just copy and paste five lines of Nvidia sample code and suddenly wake up in a ray traced world.
What Nvidia provides is the scaffolding necessary to describe a scene, as well as to provide new types of shaders that allow casting rays from point A to point B and then report back what they hit along the way. This is a huge amount of work that the hardware is providing here, but it’s not the promised “5 lines and you’ll have ray tracing in your application” that’s being promised. To adopt ray tracing you will have to write the whole ray tracer yourself, from scratch; the hardware just enables you to do so now. This is akin to implementing HDR or PBR: Shaders are the base requirement to implement both of these, but once you have shaders you still need to actually implement HDR or PBR on top of them. Another example is building a house and being provided a plot of land that can support it. Sure, it’s great, now you have a place to build your house, but you still have to come up with a blueprint, pick materials to use and then actually build the thing. Implementing ray tracing will take a great amount of engineering effort, nobody is throwing in awesome reflections with every purchase of one RTX2080Ti for free!
The other thing that’s not entirely clear is how well ray tracing will even perform in an environment like X-Plane! Worlds in X-Plane are huge and open, not small scenes from a shooter with tight spacing. Lot’s of rays are needed, and they have to travel quite far, potentially intersecting with large amounts of geometry. How good does the hardware and API scale up to these sizes? Only time will tell. That’s of course not to diminish Nvidias achievement here, it’s an incredible feat of technology in its own right and this is just the first generation!
The other thing worth mentioning is that ray tracing is not just something that Nvidia secretly cooked up in their basement for a decade. This is going to be an industry wide thing, with APIs that will work across vendors! Historically one vendor has come out with a fancy new way to do things which then became the standard adopted by other vendors. Nvidia has come forward and offered their extension as base for a core Khronos extension for Vulkan. They have a vested interested in making a cross vendor, cross platform API available.
In the foreseeable future, rasterizing renderers are unlikely to go anywhere. Rather, ray tracing for the time being can be used for additional effects that are otherwise hard to achieve. Clearly Nvidia is acknowledging this as well by providing a traditional rasterization engine that by itself is more powerful than previous generation ones. This also means that if X-Plane were to adopt ray tracing tomorrow, you could still run it on your old hardware, you’d just get extra shiny on top if you have ray tracing capable hardware.
Last but not least, this is another reason why you should stay away from the shaders! One day we’ll wake up in the glorious Vulkan future which will open the door to the glorious ray tracing future. All of this means that we’ll have to keep changing our shaders.
X-Plane has always shipped with the shaders visible to everyone as plain text in the Resources/shader directory. Partly this was due to making it convenient to load the shaders into OpenGL itself, but we also don’t have anything to hide there either so it doesn’t make much sense to try to hide them. You are more than welcome to look and poke at our shaders and if you learn something about X-Plane in the process, that’s awesome!
However, the one big caveat to that is that we never considered the shaders to be part of the publicly accessible interface and they are in no way stable across versions. X-Plane is an actively developed product and we are making a lot of changes to the codebase, including the shaders, so you should never ever distribute a plugin or tool that modifies the shaders. Since we give no guarantee that our shaders will remain stable across versions, you’ll always be left worrying that we might break your add-on.
Additionally, there is a big change to the shader system coming in 11.30 that will definitely break all existing plugins that are modifying shaders. This blog post will cover the upcoming changes and hopefully convince everyone that the shader system is in flux and not to be relied upon as a basis for add-ons. Read More
With 11.10 there is a new way aircraft only shadows are done, as well as how aircraft icons are generated. The big change is how we calculate the volume of the aircraft which up until now was based on all OBJ files that the aircraft ships with, including things like ground- and fuel trucks, stairs etc. The reason this is undesirable is because the greater the volume of the aircraft, the worse its realtime shadow quality will be because we use the volume of the aircraft to calculate our shadow map area. The bigger that area, the worse the shadow quality and the more pixelated it will look like. In an ideal world, the aircraft volume tightly hugs around the actual aircraft and we get the best shadow quality possible. With 11.10, hopefully this ideal world is finally here!
Why and how we failed before
Before 11.10 the aircraft volume was based on the volume of, well, the aircraft. However, this includes things such as the aforementioned ground trucks, fuel trucks and what have you, that artificially blow up the volume calculation. The problem is, all these objects are technically part of the aircraft (eg. we move them around with the aircraft), but they are for the most part invisible and most people wouldn’t actually consider them to be part of the aircraft proper.
In 11.05 we added a change to also consider the physical volume which kind of has the right size for the plane but doesn’t include OBJs. It is based on the physical size of the plane only, which sounds like it’s the right thing. However, as it turns out, this volume breaks badly for things such as helicopters because the rotor of some third party helicopters are attached OBJs and won’t be considered part of the physical volume of the helicopter.
At this point I should probably also quickly note what happens if the shadow volume is too small: Everything that gets clipped by the shadow volume will cast a shadow into infinity and beyond due to the way the shadow mapping works. This is especially bad for the helicopters that now have very quickly rotating bits that are constantly clipped by the shadow volume resulting in shadows flickering all over the place.
In short: What we want is a shadow volume that is as tight as possible around the aircraft for shadow quality, but not too tight because that also leads to problems.
What’s new in 11.10
In 11.10 the algorithm to compute the shadow volume has been completely changed. Instead of trying to jiggle around with the physical volume and the volume of all OBJs together and then coming up with a sane value, X-Plane now looks at what is actually being rendered. We start out with the physical bounding volume as before, but then we look at what is actually rendered! For that, we go through every OBJ that is marked as casting shadows and run the OBJ engine as if we were to render the whole thing. So OBJ animations as well as kill datarefs etc are considered. This happens during the first frame, so everything is set up the way it would be during normal rendering. Everything that is visible will be marked as such and the shadow volume will be expanded to include this OBJ.
The result is a volume with a tight fit around what is actually visible and therefore considered “aircraft”. Everything else is not included in the shadow volume and therefore stops casting shadows altogether. Of course, this is only in aircraft only shadow mode and is not used when scenery shadows are on. In that case, everything is handled like it was before and everything that is supposed to cast shadows does cast shadows. So, if you see missing shadows in aircraft only shadow mode, this is probably due to this change.
To visualize the differences, here are 4 screenshots showing the quality difference as well as the new shadow volume:
One thing that should be noted though is that going forwards these kinds of extra OBJs should really be done via the new drawing API in the 3.0 SDK! This allows us to very accurately determine the size of the aircraft but it also means that culling will become more accurate. The old method will of course continue to work, but it’s not the best or most efficient way to approach ground vehicles and other ground clutter.
The calculated volume for the whole aircraft with attached OBJs was also used for the aircraft icon generation. This led to some weird cases where the camera was positioned in such way that the aircraft was incredibly tiny due to the fact that we tried to get “everything” in at once. So far the recommendation was for authors to create a version of their aircraft without all the extra OBJs attached, but now that we have an adequate measure of the aircraft volume this is fixed as well! Aircraft icons should be correctly generated now with the camera positioned to capture the plane at the right distance.
There is one more fix for aircraft icons: Some authors created aircraft that did some clever culling based on where the pilots head is and then using the kill dataref to prevent parts of the aircraft from being rendered. Reading the view dataref now correctly reports the camera as being an external camera so that those custom culling solutions work with the aircraft icon generator. If your aircraft still doesn’t generate proper icons after 11.10, please file a bug report and let us know!