I just finished fixing some bugs with WorldEditor and DSF precision. A WED user was seeing draped polygon tiles exporting with alignment error, and the culprit turned out to be DSF precision. This begs the question, how precise are the lat/lon coordinates in a DSF?

DSFs are based on 16-bit coordinates, for a precision of one part in 65536. Now before you go screaming that 16-bit is dreadful, here’s the key: those 16-bit coordinates are interpreted within an arbitrary sub-rectangle of your tile, called a “pool”. If we only had one pool covering the DSF we’d have 1.5 meter precision. But by having many smaller pools, we can have high resolution within a pool.

All of the current DSF writing tools from Laminar Research are based on DSFLib, and DSFLib will create a number of pools along a pair of offset grids. In practice what this means is that you specify a number of pools across the DSF.

In WorldEditor 1.1dp2 the DSF exporter uses 8 pools. Since a DSF is about 100 km across, each pool is 12,500 meters in size, and the internal resolution of each pool is 19 cm. This is pretty good but there are cases where you can see the limits of precision. The new code (which will be in preview 3 or beta 1) uses 32 pools, for 5 cm precision.

If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want.

WorldEditor’s internal representation is 64-bit floating point, which gives precision better than 1 mm.

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

3 comments on “DSF and WED Precision”

  1. One thing I've noticed when flying in to a scenery with draped polygons (mostly converted MSFS sceneries) is that their rotational alignment is skewed when you finally land. The orientation is only correct when you Select Global Airport and load up directly at the airport. In the future will X-Plane be able to realign these ground polygons each time you enter a new 1×1 degree region so that these sceneries objects are angled correctly for where the aircraft currently is?

  3. Hello Ben,

    I tried what you said :
    "If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want."

    But it won't work…

    What is this command syntax ?
    I tried these two :

    DIVISIONS 32 ?
    PROPERTY sim/divisions 32 ?

Comments are closed.