As 920 beta finishes up, I am working on some WED features. Future WED updates will be small and more frequent; I think it’s more useful to get at least some features into WED quickly, rather than holding them until I have a huge update.
So this list is short – that doesn’t mean that other features won’t happen – they just won’t happen as soon. In particular, I am very aware that WED needs better taxiway sign editing and better polygon editing. Those will have to wait though.
WED 1.1 will provide overlay editing. WED will not be a replacement for overlay editor – Marginal has done a really great job with his tools. However, having overlay editing in WED will allow people making airports to see complete apt.dat 850 data and overlay data at the same time. This is particularly useful for authors making custom scenery where the apt.dat format is augmented with custom DSF overlay pavement and lines.*
WED 1.1 will have 3 features:
- Ability to edit these DSF overlay types: object placements, facades, forests, object strings, draped lines, and draped polygons (both tiled and textured with ST coordinates).
- Very limited preview of those types. I know you will be able to see the texture for a draped polygon for orthophoto placement. I do not know if I will even have OBJ preview in 1.1. Editing will not be WYSIWYG. It will be more CAD-like.
- Import and export of DSF overlay types to DSF overlay files.
Note that while roads and beaches are technically allowed in overlays, they are not on the list. The reason is that right now both of them require elevation data in the road and beach itself. Since WED doesn’t have a base mesh, it can’t sanely provide this information.
Eventually we will have some way to “drape” roads – at that point it will make sense to provide WED road editing too. But let’s not delay at least some overlay editing that long!
I don’t know exactly how long this work will take – my rough guesstimate is about 2 weeks. But…that depends on my working on WED for two weeks straight without interruption!
* Every type of element in apt.dat can also be created with custom-provided artwork and an overlay DSF — draped polygons for taxiways, draped lines for taxiway lines, object strings for taxiway lights, etc. The idea is to provide a way for authors who want to extend realism beyond the scope of apt.dat 850 to insert custom artwork. Apt.dat 850 is not a modeling format, so you cannot provide PNG files with it.
If you look at the LOWI demo area, you’ll see that some of the pavement in that layout is in the apt.dat file, but some pavement is in the DSF overlay. Creating this demo area required a bunch of DSF2Text hacking by Sergio and myself. With WED 1.1 it will be possible to do this completely using WED.
I just found two bugs in the scenery system in beta 7 (both will be fixed in beta 8):
- If an object had a blank TEXTURE statement with no trailing white space (this is legal in OBJ8, at least I think) the sim would crash when it rendered the object.
- DDS-based terrain textures with the new paging system would show junk when they were far away.
One of our users did the right thing: even though it wasn’t his scenery, he reported the problem and sent me the packages, rather than fixing the problem and going on.
My goal with scenery is to avoid regression bugs – that is, if it works in 900, it should work in 920. So I was glad to get these reports. I cringe when I read posts where people report “fixing” problems with content that are new to 920. Those are my bugs to fix!
My previous post on scenery tools stirred up some discussion; y-man brought up a point fundamental enough that I think it warrants explanation.
The questions is whether to fix broken DSFs by editing the source data or the DSFs themselves.
Let me be clear: both are viable options, both have limitations, and neither are possible today. So in choosing one over the other, I am picking a strategy that I think is superior, and discounting the other not because I think it is useless, but because it is less useful and development time is very limited (doing both would take away from other good features).
Basically the two ways to fix a DSF are:
- Edit the DSF itself until it looks correct.
- Edit the source data and then rebuild the DSF from scratch.
I strongly believe we must pursue the second strategy for this simple reason: if we correct a DSF but don’t fix the source data, the same mistakes will be made in the future.
We have to keep recutting the global scenery to keep up with:
- Higher capacities for detail in newer computers.
- New global data that becomes available.
- Improved generation algorithms that makes better results from existing data.
To lose any of these would be a big set-back in scenery quality, so not recutting DSFs isn’t a great option. (Furthermore, improvements in scenery often come from new global data, so picking user changes over new data would be a tough choice.)
By letting users change the source data, we can have the best of both worlds: problems are fixed while new technology is adopted.
To answer y-man’s direct questions:
But that data is not available to us users is it? If it is, where is it, and where is the spec to work with it?
It is not available yet! I am proposing to focus on making the data available and creating the infrastructure to share data and receive improvements (choice 2) rather than providing a DSF mesh editor (choice 1).
Alternatively, a user who goes thru the trouble of correcting base DSF could send in the modified DSF, or may be even a diff between before and after states of DSF text files, and attach an explanation of purpose.
Here’s the problem: we can’t preserve the diff to the DSF and apply it to future renderings.
Imagine, or example, that you relocate 500 mesh vertices from the existing DSFs to correct a coastline. (I am being generous here and assuming a clear, single, unifying edit. But some authors would more likely move the vertices to make a number of improvements.)
In the meantime, another author creates an airport nearby, and someone else improves the SRTMs (there is at least one person attributed in our about box who has been collecting void-filled SRTM tiles). The effect of the nearby airport’s pavement changing size and the raw elevation changing height is to change greatly how the mesh is generated, such that 400 of the 500 vertices that were moved simply no longer exist, and 450 new vertices are in the nearby area now that were not in the original DSF.
This case is known as a “merge conflict” in computer programming terms (and happens when two changes to a program are “merged”). The problem is that we can’t sanely know what to do with our 500 edited vertices in this case. Do we take the 100 vertices that still exist and move them without the others? What if that produces very strange results? (Triangles might become inside-out because some vertices are moved a lot but their neighbors are not because they did not match the diff.)
We could try to apply some kind of change to the new vertices similar to what happened to the old, but how do we know if this is making things better or worse? What if that change simply deforms the outline of the airport that was added?
I can go on and on with these kinds of examples, but the point is this: you can’t unbake a cake. A DSF is similar; you can’t necessarily recover the sources that were processed together to form the final product.
This is why it’s important that we create infrastructure to correct source data, rather than focus on editing the final DSFs. I can assure you that my negative attitude toward editing DSFs is not a negative attitude toward user participation!
There are still a lot of details to be worked out about how we can work together. Who will own the data, and on what terms? How will it be distributed? How will it be shared?
Unfortunately I can’t answer those questions yet. I still need to do more research into what is technologically possible – then we can figure out how to proceed.
WED 1.0 has gone RC. The on
ly change from beta 5 is that I have the latest manual changes from Tom (including Cormac’s illustrations of taxiway signs). Both of them did some great work – the WED manual is a lot nicer than the docs I have done myself for the other tools.
A while ago I posted a tools update…has anything changed? Not a whole lot.
- WED is definitely the future of the “heavy” scenery tools (ones with extensive UI) – it has a lot of infrastructure for features like multiple undo, multiple selection, hiearchial editing, etc.
- The next big feature for WED will be some kind of overlay editing. I don’t think this will happen very soon though – my todo list is pretty out of control.
- In the long term, I think WED may provide a visual way to do MeshTool-like operations. In other words, you’ll be able to build a base mesh in WED by specifying polygons for airports, applying orthophotos, and importing one big DEM per tile.
- I don’t think that WED will ever be a DSF editor – that is, you won’t be able to open an existing DSF, move a single node, and re-save it. This just isn’t high on my priority list.
To this last point, how are you supposed to fix scenery if you can’t edit DSFs? Well, I’ll try to post more on that over the next few weeks, but the short answer is that we need structured edits to source data.
A DSF is a compiled result of a very complex set of processes. The vertex that you adjust in the mesh to fix a mountain top may not exist in the DSF if another user submits an updated airport layout, even if they are fifty kilometers apart. (All parts of the DSF affect each other through the adaptive irregular mesh.) So I don’t want to take user submitted corrections to the final DSF because those changes would be lost at the next render.
We’ve been down this road before – when the V6 global scenery was done, the plan was: now we’ll take user-edited ENVs. The problem was that this plan assumed that we’d never re-render the ENVs again, which proved totally wrong – we re-rendered them at the end of the v6 run with improved algorithms.
What we need to do is identify what components of the source data are reasonable candidates for user editing, and set up processes (similar to Robin’s collection of apt.dat data) to gather data and share it.
(We are looking at OSM – it is still under investigation!)
I am in Boston visiting family for the 4th of July, so there are a few compatibility bugs that I have seen but probably won’t be able to fix for a few days. Austin is still cranking out betas. So if new betas come out and these are not fixed, don’t panic – they are definitely on my todo list.
- Sceneries converted via FS2XPlane crash on load. It looks like this is due to a bug in the new threaded texture loader – I think you can work around this by turning off “load scenery in the background” in the rendering settings (but then loads will be slow). I have found the area where the bug occurs but haven’t isolated it.
- A user submitted a plane to us that crashes Plane-Maker on open – the panel code gets confused. I haven’t isolated it yet. If you have planes you’ve made, save the pre-920 versions!
- I think Benedikt’s x737 plugin should start working again in beta 2.
I have
blogged about correct airport layouts before, but let me bring the point up again, because it is so important:
- You must create structurally correct layouts (that is, vertices connecting to vertices, not lines) in order to get good rendering in X-Plane. Just because the preview looks okay in WED doesn’t mean your layout is correctly formed!
I wrote some documentation on the scenery site that describes the problems in more detail, with pictures.
(I’ve been trying to create more permanent documentation – it is tempting to simply blog the issues because it’s so easy to throw a blog post up, but after 110 blog posts in 2007, the scenery site is still very thin on the documentation front.)
Do Not Add Vertices To Make Smoother Curves
I really can’t stress this enough: please do not go adding extra vertices in your layout to make bezier curves look smoother in X-Plane. Why is this such a bad idea? Let me count the ways!
- X-Plane will fight you all the way! X-Plane adds vertices to curves (to make them smoother) when it detects large errors. If you have a lot of small curves, the errors are inherently smaller and X-Plane will add fewer points. So the first vertices you add to your curve do almost nothing. You have to add a huge number of vertices to get a marginal improvement in your curve. In the meantime…
- X-Plane will provide variable-quality curve rendering in the future! Curve detail should be a user-controlled setting. X-Plane has to run on a wide range of hardware; any time we can let the user pick rendering quality, this is a win, because it helps bridge the gap between the user who just bought a brand new Core 2 Extreme system with GeForce 9800 and the user trying to keep X-Plane 9 running on his G4 laptop which can’t be upgraded. When you add vertices, you take the decision about rendering quality out of the hands of the user, and force high quality on a user who may not be able to handle it. Adding vertices forces a decision of lower framerate on some users.
- Adding vertices bloats the size of apt.dat. This is not a huge factor for custom scenery, but is a factor for the default apt.dat. Robin received a big pile of new airport layouts, and that’s great. But one risk is that the total size of user submitted data could get out of control. For new layouts made with WED, vertices represent a big chunk of the data. If you are increasing your vertices by a factor of 5x or 6x to improve tessolation, you are bloating the apt.dat file.
- Manually adding vertices to smooth curves lowers the level of abstraction in the apt.dat file. Any time we can have a high level abstract representation of scenery, X-Plane has the freedom to improve rendering in the future. If your layout is made up of a large number of small curves (instead of a small number of large curves), X-Plane cannot tell that those small pieces make up some larger structure; in the future it may not be able to render those layouts as nicely as ones that are made with fewer control points.
In summary, please use the smallest number of vertices to create your layouts. (But always add vertices to ensure that your T junctions are correct!)
“Warning: padding is not integral” is a warning message that the airport sign code renderer can spit out when your airport is loaded. Unlike all of the other warnings that you might see, this one doesn’t actually tell you that there’s a problem with your sign. It’s actually telling you that there’s a problem with my code, and today I finally fixed it. (The fix will ship in X-Plane 920, not X-Plane 902.)
The problem is that the official FAA sizes (0.76 meters, for example) suffer from tiny rounding errors when put through the sign creation process. (Use 0.33 as a proxy for 1/3rd. Divide something in thirds and add it up. You lose a tiny sliver, right?) That warning message was the renderer noticing that it was being asked to fill in a really tiny sub-pixel area (which it then did, hence you can’t see a problem from your airplane).
With 920, the sign heights are changed ever so subtly (by less than 0.1%!) so that they don’t cause rounding errors. This is the equivalent of calling the sign 0.33 meters instead of 1/3rd meter.
If this all seems like decimal point hell, well, welcome to my world!
Bottom line: you can ignore this warning, it’ll go away in 920.
I am reading Predictably Irrational, by Dan Ariely. It’s a great read – definitely recommended – describing the consistent irrational biases that frequent human decision making.
The first chapter discusses our tendency to make relative, rather than absolute comparisons. When deciding whether a product is a good value, we will look at the pricing of similar models, rather than the actual relationship between the product and the money spent. (The implication being that a company can make a product seem cheap without changing its price by adding a second, more expensive but similar “decoy” product. Poof! The cheaper product is now a good deal.)
This behavioral tendency explains user reaction to the rendering settings, a subject that makes me irrational on a regular basis. 🙂
Time to Change the Settings
The rendering settings will let you select a range of sim detail between some minimum and maximum value. These values are based on the software, not hardware – because we don’t actually know how much load any given hardware can support (and with the interaction between settings, finding such a cap is basically impossible). We can only give you a range of choices and let you pick ones that work well.
When a new version of the sim comes out, we sometimes have to recalibrate the settings. If the minimum features the sim can support increase, the minimum setting will be mapped to a new, more expensive behavior. And if the maximum detail the sim can present has increased, the maximum setting will be similarly remapped. We don’t have much choice – if we need more “range” on the slider we have to recalibrate it.
I Can’t Max Them Out
Here’s where human behavior comes in. Humans make decisions based on the relative comparison of easily compared things. Given properties that are harder to measure and easier to measure, we’ll pick the easier one. Given a choice of a trip to Rome, a trip to Rome with free breakfast, and a trip to Paris, we’ll pick Rome with the free breakfast, opting for the easy to measure relative value. (Is the difference between a trip to Paris and Rome really less than the value of a breakfast? Probably not, but it’s a lot harder to evaluate.)
So when we recalibrate the settings, we inevitably here this complaint:
“I used to be able to set the sliders to the maximum setting and now I can’t.”
Previously I would have said “Why the hell do you care?!?!” — if the new slider’s 50% position looks the same as the old slider’s 100% position, why not just set it to 50% and go home happy.
But of course that’s not how we think – the immediately comparable is of immediate concern. Ironically we could make the sim less useful but more pleasing by limiting the maximum range of the sliders. Now more users could feel the joy of having everything “set on max” even if the ultimate utility of the sim is reduced.
This One Goes To 11
I’m not sure there’s a way around this. The best suggestion I’ve heard so far is that if we could attach some kind of units to the settings, then at least there would be a quantitative indication that the user isn’t losing some perceived value. But I suspect that even this misses the point; it doesn’t matter that you’re still getting 500 trees per square km – what matters is that you are getting the most you possibly can! (Perhaps this psychology also explains why people like to overclock.)
Austin tried to fight the psychology of “maximum sliders” by naming all of our settings absurd things. Ever wonder why “default” is the lowest object setting, and we almost immediately jump into “extreme”, “too many”, “insane”, etc.? He was trying to fight a losing battle against relative expectations. The natural human behavior is to pick some relative position for calibration, and based on that, every user who has to put objects below the center setting is going to be unhappy about having to use “lower than average” settings. Austin’s naming convention may be silly, but it does actually do a little bit to fight this.
Food for thought: how does having multiple levels of reflections change user expectations?
If you are developing an add-on for X-Plane, you should always check the log file (Log.txt) after running. (Remember, the sim must quit before the log file is completely written to disk.)
There are three times you should check the log:
- Any time your content doesn’t look the way you expect. (Perhaps the error is described in more detail in the log file.
- Any time there is an error (especially the “a problem with the package X” message, which is always accompanied by details in the log).
- Right before posting your work as a last check.
Posting errors to the log gives us a way to provide verbose feedback to authors when the sim hits a problem without making the user experience too horrible; minor errors are logged and major errors are mentioned to the user only once per package and logged.
The flip side of this is that if you are working on content, you need to seek out these error log messages; the alternative is to have the sim quit every time something goes wrong.
I seem to be in a philosophical mood these days with my blog posts…thought for the day: the human mind easily goes from the specific to the general. Our brains are generalizing machines, pattern matchers finding the rule in the noise.
My preference in creating new scenery-system features is to make them very limited, and my reasoning is: our brains don’t go backward very well. We do not go from the general to the specific.
Now you might think: when making a scenery-system addition, the best thing would be to have a general feature, more useful because it can be used everywhere. But I say: the most important thing is to fully understand the feature – otherwise the feature comes out buggy.
(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)
Since the human brain doesn’t go from general to specific well, it is hard to start with a rule (“let’s allow feature X in all parts of the scenery system”) and comprehensively derive all of the implications; it is human nature to be surprised later by some unintended side-effects.
It is always easier to extend a feature later to its natural full implications than to declare certain uses illegal later, after authors of planned or started trying to use the feature in that way. If the generalization of the feature makes sense, extending it is often quite painless.
Texture Paging – Scope For Now
Texture paging is the ability for X-Plane to raise and lower the resolution of scenery textures dynamically as you fly. This means more VRAM used for nearby things and less for far away things. In practical terms, this reduces VRAM used by orthophotos by down-sampling the far-away textures, making larger orthophoto scenery packages possible. As you fly, the sim reloads some textures at higher resolutions and some at lower. The cost of the features is the load time while you fly, which burns up some extra CPU cores.
It is my hope that we will productize some very simple texture paging in the next major patch of X-Plane 9 (that would be 920, not 902). But the usage will be pretty specific:
- Texture paging will only be available for .ter and .pol textures (we can extend to other scenery types later if it makes sense).
- Texture paging will require changing the .ter and .pol files (X-Plane will not automatically analyze your scenery to see what can be paged.)
- Texture paging will not be available for ENV scenery.
- If you share textures and texture page, the results will probably be really bad and cause chaos. Be sure to use only one .ter or .pol file (and reference that text file only once in the your DSF definitions section) if you want sane paging. We can extend paging to shared textures in the future, but for now orthophotos are the intended target.
I am also deferring work on dataref-driven textures; we’ll get there eventually, and the infrastructure from the pager will make it easier. But dataref-driven textures really need to be available in a lot more places – it’s a bigger, more complex feature* and I can’t keep adding scope to 920.
Make New Meshes!
While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route. RealScenery sent me their new “State of Washington” package to use as test material; I was pleasantly surprised at the high framerate. Part of that comes from them using base meshes and not overlays.
Overlays cause the sim to draw the scenery twice (first the old scenery, then your overlay), burning a lot of pixel shader and fill power. Base meshes simply replace the old mesh which is at least twice as efficient.
(I’m just going to keep beating the dead horse of base meshes because I believe that the sooner everyone moves toward base meshes, the more bang for our hardware buck everyone gets.)
* In particular, remember that texture paging happens on threads. But datarefs can come from plugins that are not threaded! Insert anarchy here…
