I’m back from vacation and trying to catch up on email and close out 920. I’ve received a number of emails regarding the 3-d cockpit, the big questions being:
- What can I do about the lousy lighting for 3-d object textures and 3-d panel textures via ATTR_cockpit_region?
- Can I use the new 2-d panel spot lights in the 3-d cockpit?
The answer is unfortunately “not much for now” and “no”. Let me explain what’s going on with the 3-d cockpit and what we’re thinking for a long term strategy.
First, I try to organize my feature work around one part of the sim for each patch. 920 is a bit too big of a patch for us (featuring both a lot of cockpit/instrument work and some big rendering engine changes). Austin has been on the road a lot this year, and in his absence I went a little nutty. I wanted to do some work on the 3-d cockpit, but it’s different code, and with 920 in beta so long, this work will have to wait.
3-d Lighting
Our long term approach to the 3-d cockpit is “real 3-d lighting”. By that I mean: multiple light sources, acting on all of the cockpit geometry based on 3-d positioning. This means a few things:
- Providing some way to specify multiple light sources inside the cockpit, as well as how they are controlled (e.g. how do you dim the flood light)?
- Specifying which attached objects are considered part of the interior of the plane vs. the exterior.
- Providing a way to provide emissive lighting vs. elements that must be lit by light sources.
The 3-d cockpit lighting environment must work the same for the panel texture and object textures. This is necessary to keep the brightness of the finished cockpit consistent between the two textures sources. With the current 3-d system (e.g. what you can do in 864) often the brightness of the panel texture and the rest of the object don’t match.
To this end, ATTR_cockpit_region, which is targeted at the new system, gives you the same lighting model for the object texture and panel texture. Now that model isn’t very useful right now, but it will get better with future patches, and it will always be consistent.
Why Not 2-D Panel Lighting
We can’t use the new 2-d panel lighting features (spot lights), etc. because they are not scalable. Most of the advanced 3-d cockpits I have looked at use a lot of textures, quite possibly several 1024×1024 object textures, as well as a panel texture.
Now the panel texture is very expensive, so there is a penalty for letting it be any bigger than necessary. Given this, we’d only have two options:
- Provide the 2-d spot light features on the OBJ textures, effectively making them dynamic. This would be very expensive, performance-wise.
- Have authors only use one huge panel texture. This would limit them to 2048×2048 and be really slow.
If we don’t provide the spot lighting features for all panel textures then we have the problem of inconsistent lighting, which makes the feature fairly useless.
So instead I have withheld spot lighting (and _LIT replacement textures) from the 3-d panel; the cockpit object will instead end up featuring real 3-d light sources to create these kinds of effects in a 3-d correct manner.
The idea here is to avoid providing features that will be unnecessary, inferior, slow, or unsupportable in the future. Otherwise such a “stop-gap” measure would just end up breaking existing airplanes.
So for now, the 864 system for 3-d cockpits still works as it always did. It will be at least one more patch before 3-d cockpits get a serious upgrade.
It’s that time of the year again…I’ll be out of the office next week, and more importantly, away from all cell and email contact.
Austin and Randy will be at EAA – stop by and say hi if you’re there. Once Austin gets back, the last few betas should resume – we’re close to an RC on 920 but not quite there.
I just found two bugs in the scenery system in beta 7 (both will be fixed in beta 8):
- If an object had a blank TEXTURE statement with no trailing white space (this is legal in OBJ8, at least I think) the sim would crash when it rendered the object.
- DDS-based terrain textures with the new paging system would show junk when they were far away.
One of our users did the right thing: even though it wasn’t his scenery, he reported the problem and sent me the packages, rather than fixing the problem and going on.
My goal with scenery is to avoid regression bugs – that is, if it works in 900, it should work in 920. So I was glad to get these reports. I cringe when I read posts where people report “fixing” problems with content that are new to 920. Those are my bugs to fix!
My previous post on scenery tools stirred up some discussion; y-man brought up a point fundamental enough that I think it warrants explanation.
The questions is whether to fix broken DSFs by editing the source data or the DSFs themselves.
Let me be clear: both are viable options, both have limitations, and neither are possible today. So in choosing one over the other, I am picking a strategy that I think is superior, and discounting the other not because I think it is useless, but because it is less useful and development time is very limited (doing both would take away from other good features).
Basically the two ways to fix a DSF are:
- Edit the DSF itself until it looks correct.
- Edit the source data and then rebuild the DSF from scratch.
I strongly believe we must pursue the second strategy for this simple reason: if we correct a DSF but don’t fix the source data, the same mistakes will be made in the future.
We have to keep recutting the global scenery to keep up with:
- Higher capacities for detail in newer computers.
- New global data that becomes available.
- Improved generation algorithms that makes better results from existing data.
To lose any of these would be a big set-back in scenery quality, so not recutting DSFs isn’t a great option. (Furthermore, improvements in scenery often come from new global data, so picking user changes over new data would be a tough choice.)
By letting users change the source data, we can have the best of both worlds: problems are fixed while new technology is adopted.
To answer y-man’s direct questions:
But that data is not available to us users is it? If it is, where is it, and where is the spec to work with it?
It is not available yet! I am proposing to focus on making the data available and creating the infrastructure to share data and receive improvements (choice 2) rather than providing a DSF mesh editor (choice 1).
Alternatively, a user who goes thru the trouble of correcting base DSF could send in the modified DSF, or may be even a diff between before and after states of DSF text files, and attach an explanation of purpose.
Here’s the problem: we can’t preserve the diff to the DSF and apply it to future renderings.
Imagine, or example, that you relocate 500 mesh vertices from the existing DSFs to correct a coastline. (I am being generous here and assuming a clear, single, unifying edit. But some authors would more likely move the vertices to make a number of improvements.)
In the meantime, another author creates an airport nearby, and someone else improves the SRTMs (there is at least one person attributed in our about box who has been collecting void-filled SRTM tiles). The effect of the nearby airport’s pavement changing size and the raw elevation changing height is to change greatly how the mesh is generated, such that 400 of the 500 vertices that were moved simply no longer exist, and 450 new vertices are in the nearby area now that were not in the original DSF.
This case is known as a “merge conflict” in computer programming terms (and happens when two changes to a program are “merged”). The problem is that we can’t sanely know what to do with our 500 edited vertices in this case. Do we take the 100 vertices that still exist and move them without the others? What if that produces very strange results? (Triangles might become inside-out because some vertices are moved a lot but their neighbors are not because they did not match the diff.)
We could try to apply some kind of change to the new vertices similar to what happened to the old, but how do we know if this is making things better or worse? What if that change simply deforms the outline of the airport that was added?
I can go on and on with these kinds of examples, but the point is this: you can’t unbake a cake. A DSF is similar; you can’t necessarily recover the sources that were processed together to form the final product.
This is why it’s important that we create infrastructure to correct source data, rather than focus on editing the final DSFs. I can assure you that my negative attitude toward editing DSFs is not a negative attitude toward user participation!
There are still a lot of details to be worked out about how we can work together. Who will own the data, and on what terms? How will it be distributed? How will it be shared?
Unfortunately I can’t answer those questions yet. I still need to do more research into what is technologically possible – then we can figure out how to proceed.
For X-Plane 920, I modified the HUD code to draw the HUD into the 3-d cockpit’s panel texture. This turned out not to be such a good idea. The problem is one of opting in: if we change the behavior of the sim without authors being able to “opt in” to that new behavior, we break old planes.
The first plane we’ve seen with this problem is the F-22. The F-22 uses a 3-d object for the cockpit all the time (even in 2-d), and uses the 2-d panel simply for texturing. (In this sense, the F-22 is ahead of its time – this structure gives some of the advantages of the new 3-d panel texture).
The problem is that the F-22 uses the panel texture to map the 2-d HUD to the 3-d HUD, with the expectation that only the HUD background (and not the writing) will appear. The sim then draws the HUD on top. When we start burning the HUD into the panel texture, we get two HUDs.
To fix this, we won’t draw the HUD to the panel texture if you have a real 3-d cockpit object. This change will go into beta six or seven.
I may eliminate the HUD on the default 777 as well…the problems with this feature are numerous:
- As discussed above, for any plane with an existing 3-d object, adding the HUD is more likely to break the plane than add value..authors could not plan for this.
- For a plane with no cockpit object like the 777 the HUD may not be very well aligned with anything – it’s just a 2-d drawing.
- Even if the HUD texture is useful, because the HUD is drawn to a texture, the result is blurry and unfocused due to texture filtering.
In other words, even for new authors I don’t think that drawing the HUD to a texture is a good way to make a HUD.
WED 1.0 has gone RC. The on
ly change from beta 5 is that I have the latest manual changes from Tom (including Cormac’s illustrations of taxiway signs). Both of them did some great work – the WED manual is a lot nicer than the docs I have done myself for the other tools.
A while ago I posted a tools update…has anything changed? Not a whole lot.
- WED is definitely the future of the “heavy” scenery tools (ones with extensive UI) – it has a lot of infrastructure for features like multiple undo, multiple selection, hiearchial editing, etc.
- The next big feature for WED will be some kind of overlay editing. I don’t think this will happen very soon though – my todo list is pretty out of control.
- In the long term, I think WED may provide a visual way to do MeshTool-like operations. In other words, you’ll be able to build a base mesh in WED by specifying polygons for airports, applying orthophotos, and importing one big DEM per tile.
- I don’t think that WED will ever be a DSF editor – that is, you won’t be able to open an existing DSF, move a single node, and re-save it. This just isn’t high on my priority list.
To this last point, how are you supposed to fix scenery if you can’t edit DSFs? Well, I’ll try to post more on that over the next few weeks, but the short answer is that we need structured edits to source data.
A DSF is a compiled result of a very complex set of processes. The vertex that you adjust in the mesh to fix a mountain top may not exist in the DSF if another user submits an updated airport layout, even if they are fifty kilometers apart. (All parts of the DSF affect each other through the adaptive irregular mesh.) So I don’t want to take user submitted corrections to the final DSF because those changes would be lost at the next render.
We’ve been down this road before – when the V6 global scenery was done, the plan was: now we’ll take user-edited ENVs. The problem was that this plan assumed that we’d never re-render the ENVs again, which proved totally wrong – we re-rendered them at the end of the v6 run with improved algorithms.
What we need to do is identify what components of the source data are reasonable candidates for user editing, and set up processes (similar to Robin’s collection of apt.dat data) to gather data and share it.
(We are looking at OSM – it is still under investigation!)
I’ve blogged in the past about the difference between key frames to animate changes to a dataref and key frames to animate an event. Simply put, key frames let you relate the position of your mesh to a dataref value, but they do not let you start a sequence of animations based on a change to a dataref.
In other words, you can make an aniamtion that puts the gear handle up when the gear dataref is 0 and down when the dataref is 1. If that dataref were ever 0.5 you’ d see something in the middle. (This dataref will always be 0 or 1 though.)
But you can’t use animation to make the handle slowly move from up to down over a few seconds when the dataref instantly jumps from 0 to 1. If the dataref moves instantly, so does your animation. There can be no time delay between the dataref and animation.
To get this “triggered sequence” effect, you need something else to detect the event and create a dataref that moves slowly. That something could be a plugin; X-Plane also provides a built-in mechanism: the “misc. slider”.
In an attempt to put the information somewhere useful, I have described the feature on the X-Plane wiki.
No Spare Datarefs (Usually)
In that last blog post I went on a rant about “spare” datarefs. Basically people would ask us to make some datarefs that did absolutely nothing, simply to wire parts of their panels and animations together.
And while I’m going off on a tangent about authoring and datarefs, we will not be adding any “spare” datarefs.
Well, the slider_on and slider_ratio datarefs are sort of “spare” datarefs – they’re 24 slots that have no default function.
I relented on my usual hard-line position on this (and a few other) datarefs for two reasons:
- These datarefs are tied to a real (albeit really simple) bit of simulation code – that is, the code that moves the sliders over time. We had to add these datarefs to make the slider mechanism accessible.
- Because sliders are an airplane feature, there is a logical scope: the person who writes the airplane gets to decide what the sliders do, and no one else gets to muck with them. (Since the cycle time is an aircraft file attribute, no one else could sanely hope to utilize them.)
I still maintain my original position that if you need a dataref that doesn’t exist at all, you should use a plugin to create it.
Someone in the X-Plane community can write a (really simple) open-source plugin whose sole purpose is to create trivial datarefs for aircraft use from a text configuration file, letting aircraft authors make as many datarefs as they need. (It is possible that this plugin exists and I just don’t know about it.)
Aircraft authors: to make sure that datarefs from differing add-ons do not have name conflicts, Sandy and I established a registry of plugin dataref prefixes. If you are going to add custom datarefs to your airplane, be sure to register a prefix, and only use that prefix. Do not start your datarefs with sim/XXX – that prefix is for X-Plane’s use only.
(Side note: the X-Plane SDK website has been under some kind of automated attack for the last week or so with lots of bogus user sign-ups. I haven’t had time to address this properly; if you are not registered on the SDK Wiki, I may not be able to approve new sign-ups for a few more days.)
I was reading this blog post on the Microsoft Office binary file formats…to quote:
A normal programmer would conclude that Office’s binary file formats:
- are deliberately obfuscated
- are the product of a demented Borg mind
- were created by insanely bad programmers
- and are impossible to read or create correctly.
Hrm – deliberately obfuscated, a product of a demented borg mind, created by an insanely bad programmer, and impossible to read or write correctly…he must be talking about DSF!
But then Joel discusses why the formats are the way they are…tell me some of these don’t apply to DSF:
- They were designed to be fast on very old computers. (Sort of true for DSF – load time on low-end hardware was a DSF design goal.)
- They were designed to use libraries. (True – DSF is a lot less weird when you consider the layered structure, and relatively easy to work with via DSFLib.)
- They were not designed with interoperability in mind. (I’ll get to this later.)
- They have to reflect all the complexity of the applications.
In fact, the only reason Joel sites for byzantine Office formats that doesn’t apply to DSF is:
- They have to reflect the history of the applications.
In our case DSF is a new invention of X-Plane 8, so you don’t see a lot of past history artifacts in it.
The File Format as API vs. Data Container
File formats can serve two fundamentally different purposes:
- A file format can specify a data container, used to save information for use later. DSF fits this category: the purpose of the DSF is to hold scenery data in a compact and fast-to-read format for X-Plane. (The other goal of DSF is to be structured so that we don’t have to change it all the time.)
- A file format can be an interchange standard between two programs, similar to a programming API. DSF2Text text files are like that.
You pay for every design goal with more engineering time – if DSF had to be an interchange standard and a high performance data container, it would be a lot harder to design and probably not do either job terribly well.
X-Plane Owns Its Own File Formats
One of the decisions I made early on in my work with the X-Plane scenery system was to not try to use open standards for most parts of the scenery system. (OBJ8, DSF, these are X-Plane specific inventions.) Does this make me a closed-source fascist? Perhaps, but the intent was not to lock people out of X-Plane.
Instead the goal was to separate the problems of interchange and data storage in the places where the two would come into conflict. What became immediately clear was that we couldn’t meet all of our goals for performance, compatibility, and interoperability for a scenery file format with one giant scheme.
So instead we have DSF, whose goal is to be a cryptic but fast final step in the scenery pipeline, and we have DSF2Text and MeshTool, which take simpler text file formats, meant for interchange, and create DSFs.
The intended user of a MeshTool or DSF2Text text file is another program that is outputting its data format. Thus by forming a number of small links in a chain, no one file format is meant to be the “super-format” that has to do everything for everyone.
I am in Boston visiting family for the 4th of July, so there are a few compatibility bugs that I have seen but probably won’t be able to fix for a few days. Austin is still cranking out betas. So if new betas come out and these are not fixed, don’t panic – they are definitely on my todo list.
- Sceneries converted via FS2XPlane crash on load. It looks like this is due to a bug in the new threaded texture loader – I think you can work around this by turning off “load scenery in the background” in the rendering settings (but then loads will be slow). I have found the area where the bug occurs but haven’t isolated it.
- A user submitted a plane to us that crashes Plane-Maker on open – the panel code gets confused. I haven’t isolated it yet. If you have planes you’ve made, save the pre-920 versions!
- I think Benedikt’s x737 plugin should start working again in beta 2.
We’re about to do something strange: any time now, Austin should announce 920 beta 1.
But what happened to the 902 beta program?
The answer is simple: it turns out we don’t need it.
Both 901 and 902 were tiny releases whose sole purpose was to add new languages to X-Plane. (901 added French/German, 902 Russian). The idea was to get our foreign distributors a localized version of the sim faster than we could do the next major patch.
Well, 920 is here, and the Russian port is still being debugged, so 920 will contain Russian and also a huge host of new features. There will never be a 902 final; simply update your beta to 920 when it’s out – any tiny changes in 902 are baked into 920.
Don’t Panic
As always, my advice for beta 1 is: don’t panic. Remember that we are a small company, and thus we have a small number of computers between all of our employees. The first few betas usually involve one or two serious bugs (the sim explodes and sets your computer on fire, etc.) that only happen on the one hardware configuration none of us have.
If you are a user, consider waiting for a later beta, or upgrade a copy of the sim.
If you create third party add-ons, please get a copy of the beta early and start filing bugs! It is much easier for us to fix compatibility bugs when we get the reports early.
Multi-Lingual
A number of users have offered to help translate X-Plane. With 920 we finally have the “new” localization infrastructure in place; I will finally be able to get users started on this process.
