Category: File Formats

Scheduled Maintenance and Free Passes

This may have happened to you: you import the latest approved airport scenery pack from the X-Plane Airport Gateway and modify it. When you go to export your improvements back to the gateway, WED says your work is invalid and has a problem — but not in the changes you made!

Huh? If this was the approved airport on the Gateway, how can it also be invalid in WED? How did that other author upload the airport before?

You Get a Free Pass

What do we do if we find that scenery and airports have bad data that is causing bugs? What if we can’t just fix the scenery pack for you? The policy we’ve been using is: “you get a free pass until your next scheduled maintenance; then you have to fix the bug.”

The X-Plane Scenery Gateway is a good example of this. Sometimes we find new categories of authoring mistakes – I write improvements to WED’s scenery validation function to catch these authoring errors. These are errors like:

  • Typos in taxiway signs – the old syntax makes a sign with incorrect letters.
  • Overlapping ATC routes – with the overlapping routes the AI aircraft do not taxi correctly.

In other words, these are situations where the WED scenery pack, if left alone, is causing clearly bad things to happen inside X-Plane. This isn’t a case of “old style or new style”, it’s a case of “broken or fixed.”

So what we do is we set WED to reject these scenery packs on future uploads, but we do not delete the airports from X-Plane itself. In other words, the airports with serious mistakes get a free pass until the next time an author does scheduled maintenance.

Then when an author is working in WED and is going to update the airport anyway, the author must fix the problems we have found.

No One Likes a Fire Drill

This strategy is a compromise between two extremes:

  • If we force you to fix your airport right now (e.g. “fix the airport or we remove it from X-Plane”), that’s not fun, that’s a fire drill. And having been exposed to plenty of fire drills myself dealing with the new OS X 10.11 and Windows 10 releases, I’m sympathetic to authors not wanting to have to stop everything to deal with an issue ASAP.
  • If we never require that these kinds of problems be fixed, they simply won’t be fixed. The overwhelming evidence that I have seen from working on authoring tools for X-Plane for over a decade is that some authors won’t fix problems unless the tools force them to do it.**

So requiring the change when the author does maintenance but not requiring the existing shipped scenery pack be modified strikes me as the best possible compromise: we still get quick responses to serious bugs, but we don’t force anyone to drop everything.

 

* I do realize that some authors are incredibly diligent about getting their scenery to be correct even if the tools don’t force them to do so! But the goal here is to have all scenery be correct; it only takes one broken scenery pack in the hundreds a user installs to crash X-Plane.

Posted in File Formats, Scenery, Tools by | 3 Comments

X-Plane 10.40: The New DSF Loader

I’ve been meaning for weeks now to write up some notes about 10.40. I’ve also been trying to put the hood back on 10.40 so we can get it to public beta; instead the last few weeks have been a flying circus of driver bugs, expiring certificates, etc.

But we are making progress, so I’ll start off in this post by describing a change in how DSFs are loaded and the wide DSF box.

A Tangent: Stuttering and Pauses

A post that involves X-Plane pausing during flight is going to invariably bring up a bunch of blog comments: “X-Plane stutters while I fly on my really expensive machine!” This is not good, and from what I’ve been hearing, it sounds like something got worse recently; this is something that we are investigating now.

But I also want to mention that historically, X-Plane has never been “no-pause”. What we have done is periodically made the pauses much shorter; our goal is to get down to zero pauses, ever, but this will happen by finding every source of pausing and fixing them one at a time. In other words, we’re in the middle of a process of improving smoothness, but even if one source of pausing is fixed, another source might still be causing problems.

DSF Loading: the Old Way

X-Plane has, since X-Plane 9, loaded DSFs in the background on a second core while you fly. This cuts down the amount of time it takes to change scenery. (Older versions of X-Plane would pause while scenery was loaded and shifted.)

The old DSF loader did have a few weaknesses though:

  • The sim pauses while DSFs are deleted. As DSFs become bigger, this pause is becoming more noticeable.
  • If the loader ever gets behind by two scenery shifts, it just waits until it catches up. This is where the dreaded “Async load timed out after 30 seconds” comes in – it indicates that the DSF loader was so far behind that it locked up for half a minute and didn’t catch up.
  • The old DSF loader loads one DSF at a time, tops.

DSF Loading the New Way

X-Plane 10.40 has the new DSF loader, which both loads and unloads DSFs on worker threads to keep flight smoother. It also will load more than one DSF at a time, limited only by the requirement that adjacent DSFs not be loaded at the same time.

X-Plane 10.40 also has the option of an extended DSF scenery region for sharper terrain; with this option off, two DSFs are loaded at one time during sim boot and one or two are loaded at a time while you fly. With the extended DSF region on, up to four DSFs are loaded at once during sim boot and one or two are loaded while you fly.

The extended DSF scenery region is optional; don’t use this if you’re using HD meshes or you’re short on RAM. The new DSF loader is always on.

Posted in File Formats, News, Scenery by | 60 Comments

RFC: Library Dependencies

This is another “Request for Comments” post – please discuss the proposal in the comments; if you comment asking about the OccRift your comment will be piped to /dev/null.

There’s one aspect of the library system that acts as a sharp unprotected corner, poking users on a regular basis: some scenery packs require other scenery packs to function. For example, many freeware airport scenery packs require OpenSceneryX. When the library pack is not available, X-Plane will not load the custom pack because it is missing art assets.* Users report this to us as a bug surprisingly often.

In my view, the big problem here is that a user has no way of knowing from X-Plane’s diagnostic message what library they should have installed. The diagnostic message isn’t useful because X-Plane doesn’t know either.  All X-Plane knows is that there was a library path, no one is providing art for it, and therefore life isn’t worth living.

The Proposal: Library Pack Dependencies

My proposal goes like this:

  • A library scenery pack can declare its “official name”, e.g. “opensceneryx” or “proaddons/trees” or what-ever.  Like plugin names, the goal is to pick a reasonably verbose name tied to your identity to avoid conflicts.  This directive would go in your library.txt file.
  • A scenery pack that needs that library declares a need for the library by the same name.
  • When X-Plane tries to load an art asset from the scenery pack and it is missing, if at least one dependent library is not present in the system, then the error message changes to something like

The scenery pack “KLAS Las Vegas” could not be loaded because the library “OpenSceneryX” is not installed.

The log.txt file would contain complete details, but hopefully it would be clear(er) to an end-user what has gone wrong: OpenSceneryX is missng, and thus KLAS Las Vegas cannot be used.

How Is The Link Made

In order for this proposal to work, scenery packs that require library X have to contain a directive stating so.  Therefore this proposal is not a cure-all for existing load problems. It would help in the long term as new scenery packs and libraries are created with these directives.

How would the link be made?  WorldEditor (or other scenery editing tools) would automatically write the dependency into the scenery pack by looking at the dependencies in place on the author’s machine and copying them into the scenery pack when the user picks “Build”.  Thus as long as the libraries had correct “naming” annotations (this does require library authors to update) then new packs made with WED would contain the right dependencies automatically.

Nasty Details

A few nasty details:

  • Library packs would need to contain both their “permanent” name and some kind of “human readable name” for error reporting.
  • The dependency statement in custom scenery packs would list the permanent names of needed libraries and copies of the human-readable names; if we need “librutrees” and it is missing, we don’t know that it’s real name is “Russian Trees 2.0” unless this has been copied at build time.
  • Dependencies would also need an integer version number.  This allows us to declare the case where the library is installed, but it is too old.

X-Plane’s built-in libraries would not contain dependency names because they are always available.

Dependency names for scenery packs would be written as DSF properties; there is no guarantee or need for the non-library scenery pack to have a library.txt file.

Open Issue: if a scenery pack declares a dependency and it is missing, should it be allowed to load if all of the art assets are present?  This is the more permissive use case, but it implies something fairly strange is happening on the end user’s machine. Permissiveness might be desirable during the transition into using dependency names.

Will It Work

The library system has (for quite a few years now) allowed “place-holder” objects to be declared in a scenery pack that act as fall-backs for missing objects.  The use case goes like this:

  • OpenSceneryX provides a “fallback” pack that is dumped directly into the scenery pack with blank objects for every library path.
  • If the end user has OpenSceneryX installed, they see the real art.
  • Otherwise they see nothing – the fallbacks are blank, and no error is generated.

It seems clear from the number of users who report a missing OpenSceneryX object as a “bug” that this is not working. Authors who use OpenSceneryX are not bothering to copy the “fallback” pack.  This might not even be a bug – maybe the authors don’t want their work being viewed without OpenSceneryX installed. My guess, however, is that the authors just don’t know that the fallback pack exists.  Since the authors have OpenSceneryX installed, they have no need for the fallback and can’t even tell if it is working properly.

My hope is that the library dependency scheme can be more successful in the long run because it requires no action by individual scenery authors, as long as a small number of library maintainers update their libraries.  The work of annotating scenery packs is automatic.

* Please do not try to convince me that what X-Plane should do is ignore this problem and proceed. With the RFC proposed above, we could do something less drastic, like not loading the scenery pack if the library isn’t present. But I am strongly against “load what we can and keep going”.

If X-Plane treats errors in authorship as acceptable results, then authors trying to get actual work done will have to do a lot more work to detect human mistakes in their own authorship. We need a bug to be a bug.

Posted in File Formats, Scenery, Tools by | 18 Comments

RFC: Dataref-Driven Library Regions

I have a working prototype of a proposed modification for X-Plane 10.40: dataref-driven library regions.

The idea is simple: you can define a region in a library pack, and X-Plane will only load those art assets when datarefs written into the library.txt file evaluate to true.

One of the main usages for this is to implement seasonal or winter scenery add-ons that don’t require rebooting the sim to take effect. Right now if you want to change the look of the scenery you either:

  1. Write a script to replace files inside X-Plane.  This makes updating X-Plane tricky, but it lets you mod anything.  Of course, some of those mods may not work in future versions.
  2. Create a custom library pack that replaces library paths for the default art.  This requires  reboot to take effect but doesn’t affect updates and is stable.

With this extension, method 2 can be done without a reboot.  The custom library art assets are put in a region and the region is set to only be available when certain datarefs are set to certain values.

Performance

Changes to the datarefs require a scenery re-load to take effect; that’s the cost of being able to fully change the art asset in the library.  This does allow for a lot of flexibility, however – whole objects can be added or removed based on the date, for example.  For seasonal use, if the user can decide on a season before flying, the reload should not be a facotr.

Textures vs Library Paths

The original proposal was to allow textures to be swapped by dataref. I changed to library paths because a number of the existing seasonal/winter add-ons for X-Plane change properties of art assets other than the textures; for example, they change specularity values or add normal maps that were not otherwise present.  Only changing the library art asset allows for complete customization.

Proposed Syntax

The new syntax is a single library.txt line:

REGION_DREF <dataref> <comparison> <value>

e.g.

REGION_DREF myplugin/use_snow == 1.0

Datarefs can come from the sim or a plugin; all six conditionals (< <= == != > >=) are available.  If more than one REGION_DREF line is present, all must evaluate to true to use the region.

Request For Comments

Please use the comments section to comment on this particular proposal. I’m going to be a bit fascist and nuke all off-topic comments.  This is about a specific proposed feature, not a general news update.

I have working code for this; if you’d like to try a developer build, please email me.

 

Posted in File Formats, RFC by | 12 Comments

Editing Gateway Airports (and Bit-Rot)

WorldEditor 1.4 is almost ready for beta, and among its new features (there are several big ones) one is very important to the X-Plane Airport Gateway:

  • WED 1.4 can browse airports on the gateway and download scenery packs directly into the WED workspace.

“Direct download” is really important for a few reasons, some of which might not be obvious:

  1. It saves time. Getting a gateway airport, even if you just want to look at it, is much faster when you don’t have to download the scenery pack, unzip it, install it, then import it into WED. Once you use direct download, you’ll wonder how you lived without it.
  2. It prevents mistakes. We have seen airport submissions where a user clearly downloaded a scenery pack, imported only the apt.dat (but not the DSF files) and then uploaded it. We cannot take scenery packs like that, because they fundamentally remove the 3-d data from the airport.  (A fundamental policy of the gateway is that if you upload a scenery pack and one already exists, yours can’t be worse than the existing one in some way.  We have to always move forward.)
  3. It provides ancestry information.  When you download and then upload directly from the Gateway in WED 1.4, WED provides the scenery pack ID of the original pack as the “parent” of the new pack.  This means that when Julian goes to look at an upload, he can look at the “original” and more rapidly spot problems.  If your pack is the same as the original except for taxi signs, he only needs to inspect taxi signs.
  4. It prevents data loss.  DSF is a slightly lossy data format – that is, if you get your data back out of a DSF file, it won’t be exactly the same as what you put in. (It is like a JPEG image in that regard.)

More on that last point: DSF stands for Distribution Scenery Format – it was meant as a way to make final scenery packs; it was not meant as an interchange format for continuous editing.  So users are constantly importing and exporting DSFs to do work, small rounding errors (“bit rot”) will creep into our 3-d, and features that were perfectly aligned might not be well aligned after 4 or 5 edits.

The internal format for scenery on the gateway is not binary DSF files, so doing round trips to the gateway has much less “bit rot” than importing scenery packs.

Finally, DSFs are tiles; if your airport spans a DSF boundary, all DSF features (polygons, fences, etc.) get split along the DSF boundaries.

So if you import a scenery pack that you downloaded, you’re getting the split version, which is harder to edit and work with.

The internal format of scenery packs on the gateway is not split, and thus you can edit the original in almost the same form as it was uploaded.

WorldEditor 1.4 beta should becoming in “weeks” (and hopefully not that many weeks); I’ll post here when we are ready for beta testers.

Posted in Development, File Formats, Scenery, Tools by | 25 Comments

Please Help Find Crashing Scenery

I’ve been fielding crash-related bug reports for X-Plane 10,30, and a lot of them are turning out to be caused by third party scenery packs.

Now there are two possibilities for why a third party scenery pack might crash X-Plane:

  1. There is a bug in X-Plane.  In this case, I want to fix the bug!
  2. There is something wrong with the data in the scenery pack (that is, the data doesn’t meet X-Plane’s expectations).  In this case, I want to change the result from a crash to some kind of clear error report that an author could use to fix the problem.

The Old Rules

During my work on the scenery system for X-Plane 8 and 9, my view was “garbage in, garbage out” – that is, if you feed junk data into the scenery system, the result would be unpredictable and could include a crash.  In X-Plane 8, scenery load was completely synchronous – that is, your flight just stopped while we loaded files, so my idea at the time was that validation of scenery files should be done ahead of time, perhaps by the tools that write those files.

That logic (if it was even a good idea back then) doesn’t apply now.  Loading is done entirely on other CPU cores and does not interrupt flight; we can take the CPU time to make sure incoming data is valid.  And the cost of not validating the data is clear:

  • The instability from badly formed data results in a sea of crash reports – if we have real serious sim bugs, it’s much harder to catch them in the noise.
  • Because we don’t trap bad data, the end result crash can be in the OpenGL driver (due to cascading errors), and due to the way 64-bit crash detection works, we don’t get any trace of the contributing X-Plane code path.
  • Authors who create scenery packs can’t tell if their pack is the source of crash, or how their data induced a crash.

My goal is to chase down crashes caused by third party scenery and both fix any bugs that emerge and tighten up error checking to detect the cause of crashes early on, in a way that is useful to authors.

The New Rules

Because X-Plane has not run validation over all scenery files in the past, there are probably scenery files in use now that contain data that I would consider illegal (either by the documented format, or just by my own view of how the file format was meant to work; I realize that second definition is quite useless to authors).  But for illegal data that doesn’t crash the sim, I can’t just go in and change X-Plane to reject that data in the next patch.  (“Surprise!  Your scenery packs won’t load!” isn’t cool.)

So my thinking for validation is:

  • Detected error conditions that would have caused a crash 100% of the time can be handled by any clear means possible, including quitting the sim with an error message. If the sim can handle the error and continue, that’s better, but even a controlled quit with diagnostics is an improvement over an uncontrolled crash.
  • Detected error conditions that would have crashed the sim some of the time need to be logged but can’t quit the sim or cancel loading of the entire scenery pack; while a crash is serious, users were probably still enjoying the scenery pack (and merely being annoyed at how unstable their flying experience was).  The user experience shouldn’t get worse.
  • Detected error conditions that have only minor effects (e.g. out of range values that might cause rendering artifacts) whose legal values are well-documented can be logged, perhaps with a noisy error.  The pack works, so quitting or refusing to load is not acceptable, but if the error is too quiet, the condition will never get fixed.
  • Detected error conditions that have only minor effects whose legal values were never documented need to be logged unobtrusively; it’s not fair of me to penalize scenery packs for failing to read my mind.

On those first two points: one of the issues that complicates validation of scenery packs is that some errors are not discovered until mid-flight, because scenery is processed while you fly.  This means that a large scenery pack might contain areas that are problematic to fly in; if the original author doesn’t cover every inch of scenery in a test flight, the author might not know about the problem.

The ATC system takes a more aggressive approach: it validates every single .atc file at load time even if it doesn’t need to use the data until later.  (Virtual air traffic controllers are instantiated near the aircraft as the user flies, but the data in the .atc files that will be used to set up those controllers is checked at load time.)

This approach wasn’t too expensive performance-wise: the .atc files are small and the ATC system has to read them anyway to index them for later use.  I am not sure how to best replicate this for scenery; consider Alpilotx’s HD scenery mesh – reading every single DSF in Europe at load time isn’t acceptable performance wise but without that, how can he know that his scenery doesn’t contain areas that could crash X-Plane?

Please Send Me Bogus Scenery Packs

Here’s where I need you to help: if you find X-Plane is crashing, and you find that removing a scenery pack fixes the crash, please report a bug, including the scenery pack (or where to get it) and where you had to fly/start up to induce the crash.  Even if it is the fault of the pack (and it may not be – it may be an X-Plane bug!!!!) I still want to clamp down on the failure condition.

Posted in File Formats, Scenery, Tools by | 21 Comments

Airport Authoring: Airport Boundaries

Since X-Plane 8.50, airports have had a border polygon that defines the boundary of the airport surface area. The airport boundary polygon is probably the least understood aspect of airports, particularly because it takes effect when we cut DSFs, not when you load X-Plane.

The airport boundary does a few things:

  • If sloped runways are not enabled, it defines (roughly) the area that X-Plane will flatten to the airport elevation. If you have ever entered a silly field elevation in WED and then turned off sloped runways, you’ll see that the flattening process is not exact.
  • When we create base mesh DSFs, the area of the boundary polygon gets a grass surface texture.
  • When we create base mesh DSFs, the area of the boundary polygon has no autogen buildings.  (These two points are actually one in the same – the zoning of ‘airport’ in the DSF creation tool adds grass and removes autogen at the same time.)
  • When we create base mesh DSFs, the elevation under the boundary polygon has large bumps and spikes removed.

No More Bezier Curves

If you’ve used WED 1.2, you may have noticed that it refuses to validate airports with bezier curve boundaries.  This is because we are trying to stop the use of bezier curves for airport boundaries.

The problem with  bezier curves is that they can very easily be self-intersecting, but WED has no good test for this.  The result is (apparently) valid WED airport layouts that later crash the DSF generator.  Every time we cut DSFs, we find a handful of boundary polygons that need to be hand-fixed due to bad bezier outlines.  There were 3 in this last batch.

Furthermore, the DSF generator cannot actually use bezier curves – it has to convert them to polygons.  So the author will not get a surface area like the one they expected – there is a conversion that must take place.

Faced with a bezier curve technology that produces crashes and isn’t actually used, I decided to switch to straight polygonal airport boundaries.  So: when you work on new airports, please remove bezier curves from your boundary polygons.

Posted in File Formats by | 16 Comments

The Library: Public, Private and Deprecated

10.25 is coming along well; if nothing really bad happens, we should have a beta either for this weekend or next Tuesday.  There are just a few remaining bugs left.

I’ve been looking through the collection of submitted lego brick airports, and I am impressed with what the community has done.  In the collection I count 237 customized airports, with a total of 32,152 object and .agp placements.  So I am thrilled that people are using WED and starting to bring life to their local airports.

(I’ll post some pictures and we’ll have an official list later; right now I’m still in the process of checking the airports to make sure they don’t crash the sim, etc.)

There’s a lot more to blog on the subject of airports and WED, but one thing has become immediately clear from seeing people’s work and the questions that have come up from LR employees: the library is a big cluttered mess and it makes it hard to find and use art assets properly.

The WED 1.3 code (available in GIT for nerds) already has better library filtering and sorting (another one of Ted’s contributions) but I’m trying to get a very basic quick filtering feature into WED 1.2.1 and X-Plane 10.25: categorization of the library.

In my current design I have three categories:

  • Public.  This is the stuff you should be using to make scenery packs.  The lego airport bricks are all public.
  • Private.  This is stuff we meant to use as sub-parts of other art assets; they really should not have been exposed in the library.  (The right thing to do is to use the parent art asset.)  Often the private sub-parts contain fractions of a complete art asset and don’t make any sense on their own.
  • Deprecated.  This is stuff that used to be public, but now isn’t a good idea to use.  Often the deprecated art assets contain empty stub objects, in place only to keep old scenery packs from crashing.  For example, there are actually library paths for X-Plane 7 ENV radio towers – these would be marked as deprecated.

The categories affect WorldEditor, not X-Plane!  If your scenery pack uses a private art asset, it will work just as well under 10.25 as it did under 10.20.  The goal is to filter the library in WorldEditor, so that authors can:

  1. Find the good stuff faster.
  2. Not use internals and old stuff by accident.

In its first revision, WED 1.2.1 won’t yell at you or flag you if an art asset is private; it just won’t show it in the library for additional use.  We’ll work on a more refined set of interfaces in WED 1.3.

This is a major architectural change to the library, and I’m not thrilled to have to shotgun it into X-Plane 10.25.  But the situation right now is pretty unmanageable, and I hate to see people adding private art assets into their scenery for any longer than necessary.

Authors: comments welcome!  If this is going to break how you make scenery, please say so.  If there are other ways you think this feature should work, please say so.

Everyone else: I’m going to be a hard-ass and delete off-topic comments.  This post is for authors to discuss public, private and deprecated library paths only.

Posted in File Formats, Scenery by | 11 Comments

DSF Mesh Formats – v9 vs v10

Tom asked a good question in the comment section of a previous post: what is the difference between DSF mesh formats for v9 and v10.  Here’s the story:

Mesh: In X-Plane 8 and 9, the terrain mesh is stored as a set of triangles in 3 dimensions; each corner of the triangle has a latitude, longitude, and altitude.  The shape of the mesh comes from the location of those triangles and the heights of each corner.

Mesh + DEM: X-Plane 10 can also handle a new extended DSF with raster (array) data.  In this mode, the mesh contains triangles (just like it did) but they contain only latitude and longitude.  The elevation for the entire DSF tile is stored in a 2-dimensional array of elevations (a raster DEM).  When X-Plane 10 loads this format, it reads the height for each triangle out of the array of elevations to “rebuild” the 3-d triangles at load time.

X-Plane 9 supports only the original “mesh” DSFs.

X-Plane 10 supports both the original mesh DSFs and the new Mesh + DEM DSFs.

Therefore, old scenery from v9 loads fine in v10.  But you cannot load the new v10 global scenery in v9.*

Why Did You Guys Do This?

Moving elevation data out of the triangles and into a separate raster layer actually makes the DSF smaller.**  That’s a nice-to-have, but that’s not why we did it.

DirectX 11 class graphics cards can enhance meshes on the fly, on the GPU via tessellation.  We wanted to shift the DSF elevation mesh toward raster data so that we would have the full source raster to feed into the GPU.  In this configuration, we can make a low resolution mesh, give the graphics card the full data and say “go to town.”

If the graphics card can ‘enhance’ the mesh quality, this solves a problem we have now: there is no rendering setting for mesh complexity.  Right now everyone uses the same meshes, so we have to limit mesh detail to meet the specs of low-end supported computers.  With GPU-enhanced terrain, users with more powerful systems can crank up the detail.

We’re not ready to code this yet, but one first step was modifying the DSF format to be ready for tessellation.  We did this with the X-Plane 10.0 global scenery, and a nice side effect was smaller DSFs.

What About MeshTool?

MeshTool 2.0 writes DSFs with the classic “mesh” format, and it uses the v9 global terrain definitions to fill in land classes where there are no custom orthophotos.

MeshTool 3.0 will write DSFs with the new mesh+DEM format, and it will use the new v10 terrain definitions.

Therefore, MeshTool 2.0 will make v9 scenery (that can be loaded in v10) and MeshTool 3.0 will make v10-only scenery.

* There are also a million other v10 0nly features that the global scenery requires that v9 does not support.  Besides not supporting Mesh + DEM, v9 doesn’t support the new terrain shaders, the new draped road system, or the new autogen!

** The space savings come from two places: first, we don’t need to save terrain normals.  Instead we calculate them since we have the full DEM.  Second, we use 7-zip compression, and it actually gets better compression ratios on less heavily encoded data.  So the raw raster DEM compresses better than the highly encoded triangulation mesh.  The triangle mesh encoding format was designed a long time ago for classic pk-zip, not 7-zip.

Posted in File Formats, Scenery, Tools by | 31 Comments

WED: WED’s Export Dies

Why the nerdy recursive acronym for WED?  Sometimes the data just stares you in the face.

I spent an hour today doing inventory on the build-up of WED bugs. That’s a screenshot of one of my bug screeners for the public scenery tools bug base – open bugs in WED 1.2b1 sorted by severity.  Notice a trend?  100% of the crash bugs come from the import/export module.  In fact, export bugs dominate the crash+major category too.

At this point I believe the DSF overlay export code simply needs a rewrite. The current code converts the polygons from my code to CGAL, then attempts to chop them up to fit into DSFs, and then converts them back and exports them. To put it mildly, this code is not working well.

As far as I can tell, CGAL 3.9 (the version we use) has reliability problems on Windows when processing Bezier curves.  I don’t know if it’s a build environment issue, something I did, or their bug, but I don’t trust the code path; we don’t use CGAL’s bezier curve processing for the global scenery project, so we don’t have strong data that this code will ever work.

Why Do We Need to Cut?

The current WED export code chops up your polygons to fit them into the DSF tile boundaries, because everything in the DSF must be contained within the DSF, right?

Well…

There’s actually nothing in the DSF file format to disallow ‘overhang’, e.g. a facade or some other shape that is partly in the DSF.  And for custom airports, having to cut buildings down the middle at lat/lon boundaries is unfortunate – the results can look ugly.

Unfortunately, as of this writing, X-Plane 10 requires the DSF to not “spill over” its boundaries.  But it looks like this could be fixed.

Multiple Export Fixes

This is my plan for fixing WED export, more or less:

  • WED will provide a “target” X-Plane version, e.g. 9.00, 10.00, 10.30, etc. This will select what kind of export features it can use.  If you try to export a project using a feature that cannot be supported (e.g. custom facade walls to a v9 airport) you will get a warning or error, depending on the nature of the problem.
  • WED export will not attempt to cut overlay data on-the-fly; rather it will either export the geometry spilling over the border (if allowed by export target version) or flag the span as an error.
  • An edit menu tool will be provided to cut geometry at DSF borders, for at least some DSF geometry types.

Why make border cutting a manual step?  I think that it will be more useful for authors to see the result of the border cut. For example, when a facade is border cut and has unique wall choices, the author has no control about the wall type of the wall that was introduced to span the DSF border.  With pre-cutting you can slice your facade, then pick a ‘safe’ wall type for the two walls that face each other at the DSF border.*

WED will currently refuse to border-cut some geometry; I think this will continue in 1.2, albeit this will happen in the “cut at borders” command – if you request to cut a bezier type WED may refuse.

* No matter which way you slice it, cutting facades at the DSF border sucks. This is why I think the long term solution is to allow facades to span borders, at least a little bit.

Posted in File Formats, Scenery, Tools by | 15 Comments