Category: Scenery

Smooth Airport Curves

With X-Plane 920 RC1, the user can now control how smooth taxiway curves look.  More smoothing looks better, but can slow frame-rate.
So authors: please use the minimum number of vertices to create a bezier curve.  If the user wants it to look smooth, he or she can crank the rendering settings.  If the user has the setting on “low” it’s probably due to a lack of hardware.
Below are four pictures of KSBD (which has good, sparse vertices) at the four rendering settings.

Posted in File Formats, Scenery by | 7 Comments

WED 1.1 Feature List

As 920 beta finishes up, I am working on some WED features. Future WED updates will be small and more frequent; I think it’s more useful to get at least some features into WED quickly, rather than holding them until I have a huge update.

So this list is short – that doesn’t mean that other features won’t happen – they just won’t happen as soon. In particular, I am very aware that WED needs better taxiway sign editing and better polygon editing. Those will have to wait though.

WED 1.1 will provide overlay editing. WED will not be a replacement for overlay editor – Marginal has done a really great job with his tools. However, having overlay editing in WED will allow people making airports to see complete apt.dat 850 data and overlay data at the same time. This is particularly useful for authors making custom scenery where the apt.dat format is augmented with custom DSF overlay pavement and lines.*

WED 1.1 will have 3 features:

  • Ability to edit these DSF overlay types: object placements, facades, forests, object strings, draped lines, and draped polygons (both tiled and textured with ST coordinates).
  • Very limited preview of those types. I know you will be able to see the texture for a draped polygon for orthophoto placement. I do not know if I will even have OBJ preview in 1.1. Editing will not be WYSIWYG. It will be more CAD-like.
  • Import and export of DSF overlay types to DSF overlay files.

Note that while roads and beaches are technically allowed in overlays, they are not on the list. The reason is that right now both of them require elevation data in the road and beach itself. Since WED doesn’t have a base mesh, it can’t sanely provide this information.

Eventually we will have some way to “drape” roads – at that point it will make sense to provide WED road editing too. But let’s not delay at least some overlay editing that long!

I don’t know exactly how long this work will take – my rough guesstimate is about 2 weeks. But…that depends on my working on WED for two weeks straight without interruption!

* Every type of element in apt.dat can also be created with custom-provided artwork and an overlay DSF — draped polygons for taxiways, draped lines for taxiway lines, object strings for taxiway lights, etc. The idea is to provide a way for authors who want to extend realism beyond the scope of apt.dat 850 to insert custom artwork. Apt.dat 850 is not a modeling format, so you cannot provide PNG files with it.

If you look at the LOWI demo area, you’ll see that some of the pavement in that layout is in the apt.dat file, but some pavement is in the DSF overlay. Creating this demo area required a bunch of DSF2Text hacking by Sergio and myself. With WED 1.1 it will be possible to do this completely using WED.

Posted in News, Scenery, Tools by | 5 Comments

Report It, Don’t Fix It

I just found two bugs in the scenery system in beta 7 (both will be fixed in beta 8):

  • If an object had a blank TEXTURE statement with no trailing white space (this is legal in OBJ8, at least I think) the sim would crash when it rendered the object.
  • DDS-based terrain textures with the new paging system would show junk when they were far away.

One of our users did the right thing: even though it wasn’t his scenery, he reported the problem and sent me the packages, rather than fixing the problem and going on.

My goal with scenery is to avoid regression bugs – that is, if it works in 900, it should work in 920. So I was glad to get these reports. I cringe when I read posts where people report “fixing” problems with content that are new to 920. Those are my bugs to fix!

Posted in Scenery by | 1 Comment

Why We Must Edit the Source Data

My previous post on scenery tools stirred up some discussion; y-man brought up a point fundamental enough that I think it warrants explanation.

The questions is whether to fix broken DSFs by editing the source data or the DSFs themselves.

Let me be clear: both are viable options, both have limitations, and neither are possible today. So in choosing one over the other, I am picking a strategy that I think is superior, and discounting the other not because I think it is useless, but because it is less useful and development time is very limited (doing both would take away from other good features).

Basically the two ways to fix a DSF are:

  1. Edit the DSF itself until it looks correct.
  2. Edit the source data and then rebuild the DSF from scratch.

I strongly believe we must pursue the second strategy for this simple reason: if we correct a DSF but don’t fix the source data, the same mistakes will be made in the future.

We have to keep recutting the global scenery to keep up with:

  • Higher capacities for detail in newer computers.
  • New global data that becomes available.
  • Improved generation algorithms that makes better results from existing data.

To lose any of these would be a big set-back in scenery quality, so not recutting DSFs isn’t a great option. (Furthermore, improvements in scenery often come from new global data, so picking user changes over new data would be a tough choice.)

By letting users change the source data, we can have the best of both worlds: problems are fixed while new technology is adopted.

To answer y-man’s direct questions:

But that data is not available to us users is it? If it is, where is it, and where is the spec to work with it?

It is not available yet! I am proposing to focus on making the data available and creating the infrastructure to share data and receive improvements (choice 2) rather than providing a DSF mesh editor (choice 1).

Alternatively, a user who goes thru the trouble of correcting base DSF could send in the modified DSF, or may be even a diff between before and after states of DSF text files, and attach an explanation of purpose.

Here’s the problem: we can’t preserve the diff to the DSF and apply it to future renderings.

Imagine, or example, that you relocate 500 mesh vertices from the existing DSFs to correct a coastline. (I am being generous here and assuming a clear, single, unifying edit. But some authors would more likely move the vertices to make a number of improvements.)

In the meantime, another author creates an airport nearby, and someone else improves the SRTMs (there is at least one person attributed in our about box who has been collecting void-filled SRTM tiles). The effect of the nearby airport’s pavement changing size and the raw elevation changing height is to change greatly how the mesh is generated, such that 400 of the 500 vertices that were moved simply no longer exist, and 450 new vertices are in the nearby area now that were not in the original DSF.

This case is known as a “merge conflict” in computer programming terms (and happens when two changes to a program are “merged”). The problem is that we can’t sanely know what to do with our 500 edited vertices in this case. Do we take the 100 vertices that still exist and move them without the others? What if that produces very strange results? (Triangles might become inside-out because some vertices are moved a lot but their neighbors are not because they did not match the diff.)

We could try to apply some kind of change to the new vertices similar to what happened to the old, but how do we know if this is making things better or worse? What if that change simply deforms the outline of the airport that was added?

I can go on and on with these kinds of examples, but the point is this: you can’t unbake a cake. A DSF is similar; you can’t necessarily recover the sources that were processed together to form the final product.

This is why it’s important that we create infrastructure to correct source data, rather than focus on editing the final DSFs. I can assure you that my negative attitude toward editing DSFs is not a negative attitude toward user participation!

There are still a lot of details to be worked out about how we can work together. Who will own the data, and on what terms? How will it be distributed? How will it be shared?

Unfortunately I can’t answer those questions yet. I still need to do more research into what is technologically possible – then we can figure out how to proceed.

Posted in Scenery, Tools by | 7 Comments

The Future of WED

WED 1.0 has gone RC. The onLinkly change from beta 5 is that I have the latest manual changes from Tom (including Cormac’s illustrations of taxiway signs). Both of them did some great work – the WED manual is a lot nicer than the docs I have done myself for the other tools.

A while ago I posted a tools update…has anything changed? Not a whole lot.

  • WED is definitely the future of the “heavy” scenery tools (ones with extensive UI) – it has a lot of infrastructure for features like multiple undo, multiple selection, hiearchial editing, etc.
  • The next big feature for WED will be some kind of overlay editing. I don’t think this will happen very soon though – my todo list is pretty out of control.
  • In the long term, I think WED may provide a visual way to do MeshTool-like operations. In other words, you’ll be able to build a base mesh in WED by specifying polygons for airports, applying orthophotos, and importing one big DEM per tile.
  • I don’t think that WED will ever be a DSF editor – that is, you won’t be able to open an existing DSF, move a single node, and re-save it. This just isn’t high on my priority list.

To this last point, how are you supposed to fix scenery if you can’t edit DSFs? Well, I’ll try to post more on that over the next few weeks, but the short answer is that we need structured edits to source data.

A DSF is a compiled result of a very complex set of processes. The vertex that you adjust in the mesh to fix a mountain top may not exist in the DSF if another user submits an updated airport layout, even if they are fifty kilometers apart. (All parts of the DSF affect each other through the adaptive irregular mesh.) So I don’t want to take user submitted corrections to the final DSF because those changes would be lost at the next render.

We’ve been down this road before – when the V6 global scenery was done, the plan was: now we’ll take user-edited ENVs. The problem was that this plan assumed that we’d never re-render the ENVs again, which proved totally wrong – we re-rendered them at the end of the v6 run with improved algorithms.

What we need to do is identify what components of the source data are reasonable candidates for user editing, and set up processes (similar to Robin’s collection of apt.dat data) to gather data and share it.

(We are looking at OSM – it is still under investigation!)

Posted in Scenery, Tools by | 7 Comments

What Is a File Format

I was reading this blog post on the Microsoft Office binary file formats…to quote:

A normal programmer would conclude that Office’s binary file formats:

  • are deliberately obfuscated
  • are the product of a demented Borg mind
  • were created by insanely bad programmers
  • and are impossible to read or create correctly.

Hrm – deliberately obfuscated, a product of a demented borg mind, created by an insanely bad programmer, and impossible to read or write correctly…he must be talking about DSF!

But then Joel discusses why the formats are the way they are…tell me some of these don’t apply to DSF:

  • They were designed to be fast on very old computers. (Sort of true for DSF – load time on low-end hardware was a DSF design goal.)
  • They were designed to use libraries. (True – DSF is a lot less weird when you consider the layered structure, and relatively easy to work with via DSFLib.)
  • They were not designed with interoperability in mind. (I’ll get to this later.)
  • They have to reflect all the complexity of the applications.

In fact, the only reason Joel sites for byzantine Office formats that doesn’t apply to DSF is:

  • They have to reflect the history of the applications.

In our case DSF is a new invention of X-Plane 8, so you don’t see a lot of past history artifacts in it.

The File Format as API vs. Data Container

File formats can serve two fundamentally different purposes:

  1. A file format can specify a data container, used to save information for use later. DSF fits this category: the purpose of the DSF is to hold scenery data in a compact and fast-to-read format for X-Plane. (The other goal of DSF is to be structured so that we don’t have to change it all the time.)
  2. A file format can be an interchange standard between two programs, similar to a programming API. DSF2Text text files are like that.

You pay for every design goal with more engineering time – if DSF had to be an interchange standard and a high performance data container, it would be a lot harder to design and probably not do either job terribly well.

X-Plane Owns Its Own File Formats

One of the decisions I made early on in my work with the X-Plane scenery system was to not try to use open standards for most parts of the scenery system. (OBJ8, DSF, these are X-Plane specific inventions.) Does this make me a closed-source fascist? Perhaps, but the intent was not to lock people out of X-Plane.

Instead the goal was to separate the problems of interchange and data storage in the places where the two would come into conflict. What became immediately clear was that we couldn’t meet all of our goals for performance, compatibility, and interoperability for a scenery file format with one giant scheme.

So instead we have DSF, whose goal is to be a cryptic but fast final step in the scenery pipeline, and we have DSF2Text and MeshTool, which take simpler text file formats, meant for interchange, and create DSFs.

The intended user of a MeshTool or DSF2Text text file is another program that is outputting its data format. Thus by forming a number of small links in a chain, no one file format is meant to be the “super-format” that has to do everything for everyone.

Posted in File Formats by | Comments Off on What Is a File Format

Third Party Add-on Compatibility in 920

I am in Boston visiting family for the 4th of July, so there are a few compatibility bugs that I have seen but probably won’t be able to fix for a few days. Austin is still cranking out betas. So if new betas come out and these are not fixed, don’t panic – they are definitely on my todo list.

  • Sceneries converted via FS2XPlane crash on load. It looks like this is due to a bug in the new threaded texture loader – I think you can work around this by turning off “load scenery in the background” in the rendering settings (but then loads will be slow). I have found the area where the bug occurs but haven’t isolated it.
  • A user submitted a plane to us that crashes Plane-Maker on open – the panel code gets confused. I haven’t isolated it yet. If you have planes you’ve made, save the pre-920 versions!
  • I think Benedikt’s x737 plugin should start working again in beta 2.
Posted in Aircraft, File Formats, Scenery by | 1 Comment

Clean Airport Layouts

I have blogged about correct airport layouts before, but let me bring the point up again, because it is so important:
  • You must create structurally correct layouts (that is, vertices connecting to vertices, not lines) in order to get good rendering in X-Plane.  Just because the preview looks okay in WED doesn’t mean your layout is correctly formed!

I wrote some documentation on the scenery site that describes the problems in more detail, with pictures.  

(I’ve been trying to create more permanent documentation – it is tempting to simply blog the issues because it’s so easy to throw a blog post up, but after 110 blog posts in 2007, the scenery site is still very thin on the documentation front.)
Do Not Add Vertices To Make Smoother Curves
I really can’t stress this enough: please do not go adding extra vertices in your layout to make bezier curves look smoother in X-Plane.  Why is this such a bad idea?  Let me count the ways!
  1. X-Plane will fight you all the way!  X-Plane adds vertices to curves (to make them smoother) when it detects large errors.  If you have a lot of small curves, the errors are inherently smaller and X-Plane will add fewer points.  So the first vertices you add to your curve do almost nothing.  You have to add a huge number of vertices to get a marginal improvement in your curve.  In the meantime…
  2. X-Plane will provide variable-quality curve rendering in the future!  Curve detail should be a user-controlled setting.  X-Plane has to run on a wide range of hardware; any time we can let the user pick rendering quality, this is a win, because it helps bridge the gap between the user who just bought a brand new Core 2 Extreme system with GeForce 9800 and the user trying to keep X-Plane 9 running on his G4 laptop which can’t be upgraded.  When you add vertices, you take the decision about rendering quality out of the hands of the user, and force high quality on a user who may not be able to handle it. Adding vertices forces a decision of lower framerate on some users.
  3. Adding vertices bloats the size of apt.dat.  This is not a huge factor for custom scenery, but is a factor for the default apt.dat.  Robin received a big pile of new airport layouts, and that’s great.  But one risk is that the total size of user submitted data could get out of control.  For new layouts made with WED, vertices represent a big chunk of the data.  If you are increasing your vertices by a factor of 5x or 6x to improve tessolation, you are bloating the apt.dat file.
  4. Manually adding vertices to smooth curves lowers the level of abstraction in the apt.dat file.  Any time we can have a high level abstract representation of scenery, X-Plane has the freedom to improve rendering in the future.  If your layout is made up of a large number of small curves (instead of a small number of large curves), X-Plane cannot tell that those small pieces make up some larger structure; in the future it may not be able to render those layouts as nicely as ones that are made with fewer control points.

In summary, please use the smallest number of vertices to create your layouts.  (But always add vertices to ensure that your T junctions are correct!)

Posted in File Formats, Scenery, Tools by | 7 Comments

I Hate It When My Padding’s Not Integral

“Warning: padding is not integral” is a warning message that the airport sign code renderer can spit out when your airport is loaded.  Unlike all of the other warnings that you might see, this one doesn’t actually tell you that there’s a problem with your sign.  It’s actually telling you that there’s a problem with my code, and today I finally fixed it.  (The fix will ship in X-Plane 920, not X-Plane 902.)

The problem is that the official FAA sizes (0.76 meters, for example) suffer from tiny rounding errors when put through the sign creation process.  (Use 0.33 as a proxy for 1/3rd.  Divide something in thirds  and add it up.  You lose a tiny sliver, right?)  That warning message was the renderer noticing that it was being asked to fill in a really tiny sub-pixel area (which it then did, hence you can’t see a problem from your airplane).
With 920, the sign heights are changed ever so subtly (by less than 0.1%!) so that they don’t cause rounding errors.  This is the equivalent of calling the sign 0.33 meters instead of 1/3rd meter.
If this all seems like decimal point hell, well, welcome to my world!
Bottom line: you can ignore this warning, it’ll go away in 920.
Posted in Scenery by | Comments Off on I Hate It When My Padding’s Not Integral

Irrational Sliders

I am reading Predictably Irrational, by Dan Ariely. It’s a great read – definitely recommended – describing the consistent irrational biases that frequent human decision making.

The first chapter discusses our tendency to make relative, rather than absolute comparisons. When deciding whether a product is a good value, we will look at the pricing of similar models, rather than the actual relationship between the product and the money spent. (The implication being that a company can make a product seem cheap without changing its price by adding a second, more expensive but similar “decoy” product. Poof! The cheaper product is now a good deal.)

This behavioral tendency explains user reaction to the rendering settings, a subject that makes me irrational on a regular basis. 🙂

Time to Change the Settings

The rendering settings will let you select a range of sim detail between some minimum and maximum value. These values are based on the software, not hardware – because we don’t actually know how much load any given hardware can support (and with the interaction between settings, finding such a cap is basically impossible). We can only give you a range of choices and let you pick ones that work well.

When a new version of the sim comes out, we sometimes have to recalibrate the settings. If the minimum features the sim can support increase, the minimum setting will be mapped to a new, more expensive behavior. And if the maximum detail the sim can present has increased, the maximum setting will be similarly remapped. We don’t have much choice – if we need more “range” on the slider we have to recalibrate it.

I Can’t Max Them Out

Here’s where human behavior comes in. Humans make decisions based on the relative comparison of easily compared things. Given properties that are harder to measure and easier to measure, we’ll pick the easier one. Given a choice of a trip to Rome, a trip to Rome with free breakfast, and a trip to Paris, we’ll pick Rome with the free breakfast, opting for the easy to measure relative value. (Is the difference between a trip to Paris and Rome really less than the value of a breakfast? Probably not, but it’s a lot harder to evaluate.)

So when we recalibrate the settings, we inevitably here this complaint:

“I used to be able to set the sliders to the maximum setting and now I can’t.”

Previously I would have said “Why the hell do you care?!?!” — if the new slider’s 50% position looks the same as the old slider’s 100% position, why not just set it to 50% and go home happy.

But of course that’s not how we think – the immediately comparable is of immediate concern. Ironically we could make the sim less useful but more pleasing by limiting the maximum range of the sliders. Now more users could feel the joy of having everything “set on max” even if the ultimate utility of the sim is reduced.

This One Goes To 11

I’m not sure there’s a way around this. The best suggestion I’ve heard so far is that if we could attach some kind of units to the settings, then at least there would be a quantitative indication that the user isn’t losing some perceived value. But I suspect that even this misses the point; it doesn’t matter that you’re still getting 500 trees per square km – what matters is that you are getting the most you possibly can! (Perhaps this psychology also explains why people like to overclock.)

Austin tried to fight the psychology of “maximum sliders” by naming all of our settings absurd things. Ever wonder why “default” is the lowest object setting, and we almost immediately jump into “extreme”, “too many”, “insane”, etc.? He was trying to fight a losing battle against relative expectations. The natural human behavior is to pick some relative position for calibration, and based on that, every user who has to put objects below the center setting is going to be unhappy about having to use “lower than average” settings. Austin’s naming convention may be silly, but it does actually do a little bit to fight this.

Food for thought: how does having multiple levels of reflections change user expectations?

Posted in Scenery by | 4 Comments