For something like six or seven years, DSF has fundamentally been a vector scenery format, meaning it contains points, lines, and connections between lines that define how scenery looks. With X-Plane 10 we’ll be adding raster data to DSF.
One way we’ll use raster data inside DSFs is to store the raw elevation data for a DSF tile. Originally we saved only the final triangulation of the mesh in 3-d; we will now save the triangulation in 2-d* and the raw elevation, which X-Plane will put back together again.
We get a few wins from storing elevation separately:
- After compression the files are actually smaller. This is because the data is more “regular” when stored in raster format than as part of the triangulation, and also because we don’t need to store normal vectors.
- Since we’ll have the elevation data in its original form, we can use it to someday enhance the mesh for graphics cards that support hardware tessellation.
If raster data is a win in both quality and file-size, why didn’t we do this originally? Two reasons:
- Originally DSFs shipped in zip files; the big win in compression with regular data comes from the more advanced 7-zip compression we started using to ship X-Plane 9.
- Raster encoding means increased load time in the sim as it “puts the mesh back together”. Today in a multi-core world this is totally moot – DSF loading happens on another core, but originally DSFs had to be loaded on single-core machines, so load time was a key performance point.**
We will also be able to put other data into the DSFs, although I’m not sure what the final file set will be. Good candidates include bathymetry data and urban-density data to affect autogen.
Finally, we get a lot of requests from plugins to access X-Plane’s elevation data; with an irregular triangulation, access via plugin isn’t practical. But with raster data, the code to locate and view the raster block inside a DSF is actually pretty easy and the data comes in a simple, easy-to-use format. This might be useful for moving maps and other such technologies.
* Technically we store the triangulation as a flat 3-d mesh; DSFs RLE encoding means that the all-zero elevation and normal-offset fields crunch down to nothing.
** The decision to make roads 2-d and set up their height at runtime is a similar decision; the original 3-d roads took up more DSF disk space to save load time.
Does that mean that Custom Scenery designers (not me) will have easier time to correct DSF data and make the scenery more appealing or accurate if they so choose (like making tunnels or correcting DSF elevation ) ?
Thanks
Easier, maybe. Easy, no. We may be able to provide a “patch elevation” overlay someday but even if you patch the raster elevation data, you’d still be hosed if:
– There are not enough triangles in the area in the base mesh to capture the desired detail or
– The land classes are inappropriate for what you’re doing (e.g. you want to make a cliff but we have a farm there).
Tunnels: that’s too complex of a topic for now, but suffice it to say you really can only make a working tunnel now by making your own base mesh and OBJs and very carefully tying them together. The chances of adding a tunnel to someone else’s mesh and having it work are about zero.
Would that limitation apply to taxiway bridges as well?
The alpha/bravo taxiway bridges at KORD come to my mind… 😉
Sort of – you can easily build the bridge out of an OBJ, but if you need to construct the “trench” that the bridge goes over (in every case I’ve seen the cars go down, the planes don’t go up by any large amount) you’d need a lot of mesh vertices, and the triangulation might not be there.
What is the potential use for bathymetry in X-Plane? Different colors/textures/etc for different water depths/regions?
Or different autogen ship classes based on the water depth? (big ships only in deep waters, for example)
Impossible to say what the potential is – it’s code, you can do anything with it given enough time. The most likely case I immediately thought of was water shading. Right now we have to put water shading properties into the mesh itself; with this we’d have the original height and could derive those properties inside X-Plane. (Again, it’s trading some multi-core CPU time for file size, gaining flexibility.)
“Since we’ll have the elevation data in its original form, we can use it to someday enhance the mesh for graphics cards that support hardware tessellation.”
*giant shiny excited puppy eyes* This statement just made my day!
Of course, my main wish is to be able to define more precise local elevation data for the ‘lumpy’ airports (such as KTEX, KAVX, the north end of KSEA, etc.). as always, the wish list will include tools to edit the data, and a simple way to consolidate it together.
Your wish is..um…sorry Robin, you’re screwed. 🙂
Seriously though, including raster data does __not__ address data density issues.
It may provide a foundation to someday deal with them, but we’re not there yet.
So, does this mean that we’ll still have climbing water? I went to a lot of trouble to eliminate that issue so far as I could when building the NZ scenery pack for 9… are you putting those algorithms in the sim?
Hrmm….that’s a damn good question! I just realized that the RF changes to “apply elevation later” in X-Plane effectively break any water flattening code in RF (which act on the triangulation). Well, now I’m really glad I didn’t solve the climbing water bug first (it’s still on the v10 DSF todo list).
One thing I had looked at, at least briefly, was letting the mesher just “mesh everything” and then changing the land/water status of coastal triangles by slope. When there is a conflict between vector and elevation data, the elevation data is often more reliable, and this technique gives you a coastline that is “where the DEM runs through 0 MSL.”
Will XP10 still support XP9 DSF? A real blow if not, as Andrew’s scenery adds significant fidelity to NZ.
Yes. Xp10 will run all v9 scenery.
Hmm… but in the cases where the vector is more reliable than the elevation, it would be really nice to be able to say so. Can we tag the geometry somehow, so we can say ‘this edge is nearly flat, keep it so’?
Well that’s an easier case…we can simply “rasterize” the water MSL from the polygon _into_ the DEM, reconditioning the DEM to match the vector data. 🙂
And..what fantasy land do you live in where the vector data is more accurate than the DEMs? 😉
That really IS the case for NZ… the SRTM data is dreadful in places, an SRTM-resolution DEM based on the vector survey data is available… but since the vectors have about a 1m horizontal resolution, they win. Especially somewhere like, say, the Waimakariri gorge, which is only about 25m wide (and around 50m deep) and therefore cuts right between the SRTM data points.
Seems to me that there’s some potential for tagging triangulation edges and faces as ‘don’t use the DEM for me, just interpolate the tri’, which can be set on areas where high resolution data exists.
Are you planning on supporting higher resolution DEMs? That would be another way around the issue.
LOL so the fantasy land is Mordor. 😉 I am assuming that an in-sim DEM must scale up at _least_ as far as MeshTool does now: 10801 x 10801. This is still only 10m though. On the other hand, 1m vectors will blow up MeshTool; you need to simplify!
In the long term I think we’ll finally break the 10m limit using raster data, not vector…
Hi Andrew,
After examining the issue, it looks like vertices will need to be tagged – as coastal, inland, or at sea; we’ll include several DEMs so that the sim can pick which DEM is sampled. This should allow for correct “flat” water and eventual 3-d water effect.
Hi Ben,
Could you construct the mesh and add textures and objects on the fly if your tiles were 1/10th the current size?
Would that allow more use of these raster files and further data size savings?
You would need to load more tiles and more often.
Alternatively you could use low resolution tiles for distant stuff… which could give you a sightline to the horizon from higher altitudes!
Could be a problem with legacy custom scenery, though…
This might require a separate post. Short version:
– Meshing is fast enough to run in background – no need to cut tile size.
– The entire part of DSF generation after the mesh would have to move into the sim too though, since that stuff depends on the mesh. That’s too much time and data for the sim.
Adding bathymetry could lead to much more interesting coastlines;
1) Still water could be ‘seen through’ giving actual depth to water
2) Shallower water causes wave shape changes and earlier wave breaking
3) You could potentially show tidal changes – such as in estuaries (very cool)
4) Lead to submarine exploration with accurate marine trenches and ridges (kidding)
This might be a silly question – but what do you need the mesh for if it is only 2D information anyway? Wouldn’t the rasterized height map be enough to build the terrain?
The height ma does completely describe the terrain shape, but the triangulation tells us what terrain type goes in which places, including land-vs-water…thus the triangulation tells us the coastline!