In designing interfaces for building planes, writing plugins, etc. one of the main design questions that keeps coming up is: to copy or to reference? Should authors simply refer to an art asset, piece of data, or code in order to utilize it, or should the author copy a snapshot into the custom add-on. There isn’t one right answer. Here are the main considerations.
Performance and Efficiency
One of the obvious considerations is efficiency: in some cases we might be able to provide better performance when an art asset is referred from a common source.
For example, in some cases X-Plane will consolidate VRAM use based on actual files, so a library object is loaded once no matter how many packages use it, but is loaded many times if a package copies it.
(In other cases X-Plane will actually merge multiple copies of a resource – referencing is only a win in some cases.)
An indirect consideration: if an art asset is provided by Laminar Research and is used by reference, then a new update can provide a new, better optimized art asset – see below.
Dependencies and Contracts
When someone uses an art asset, algorithm, etc. by reference, it creates an implicit contract by the provider of the asset by reference to not change the properties of the asset. By comparison, when the asset is copied, the contract is only to support the format that the asset is encoded in.
This is the main reason why I am often against providing new assets by reference, whether it is a new dataref, texture, etc. Often I will simply send a user a snippet of code, rather than making X-Plane’s version available via a dataref. The idea is that copying does not create a new interface (and thus a new “contract”) between X-Plane and the add-on.
Copyright and Legal Issues
For historical reasons, the US legal system describes the privileges of intellectual property owners by regulating the act of copying. (To say that this is a bit quaint in the digital age doesn’t even scratch the surface, but that’s a rant for another post.) The result of this particular regulation of copying (but not of referencing) is that the decision to provide an asset by copy vs. reference has legal implications. If the author does not want to go through licensing, referencing may be the only option.
I’m not sure if these will make it into X-Plane 9.30 (we’re trying to close down features, but it doesn’t do much good to hold off features that help people make airplanes) but…while I was in Italy I created a few more manipulator types.
The set of manipulators let you change the value of a dataref directly with a mouse-click – the various flavors control how the dataref is changed.
These manipulators are the natural corollary to the command manipulator, which runs a command on a click.
Why have both? Commands are good, but they don’t cover 100% of sim functionality (just like datarefs don’t cover 100% of sim functionality). By having both, it will be possible to control switches and buttons that are best accessed by a dataref change.
For the basics on commands vs. datarefs, see
here.
The generic instruments already write to both commands (via the trigger instrument) and datarefs (via the rotary instrument) – these new manipulators provide the same functionality for 3-d cockpits.
It seemed like opposite-day at Laminar Research…Austin saying that an error shouldn’t quit the sim and me saying the error was never okay, ever. Well, I relented: with X-Plane 930 beta 8, if your scenery pack is missing objects, you can still fly. Instead you get a single error message like this:
That is the “non-fatal” error dialog box – you will see it only once for each scenery pack with a problem for each time you run the sim. It means that at least one thing is wrong with your scenery pack, but you need to look at Log.txt to see what’s wrong. For example, you might see this in the Log.txt file:
Failed to find resource 'KSBD_example.obj' at 'Custom
Scenery/KSBD Demo Area/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj' at 'Custom
Scenery/KSBD Demo Area/custom objects/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj' at
'Resources/objects/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj'
at 'Resources/KSBD_example.obj'
***Error with scenery file "Custom Scenery/KSBD
Demo Area/Earth nav data/+30-120/+34-118.dsf"
(/Volumes/RAID/code/design/HLutils/Files/io_dsf.cpp: 503.)
Unable to locate object: KSBD_example.obj
In this case, X-Plane couldn’t find the object KSBD_example.obj – the sim is also listing all of the places it looked. Note that only the first location is a good location – the other 3 are legacy search paths that date all the way back to version 6. It is likely that in the next major version we will trim down our search paths significantly.
A few comments on this whole situation:
-
Authors, do not ignore error messages like the dialog box above – every one of them indicates a condition serious enough that we think you should fix it. Non-fatal errors like these may crash future versions of the sim, or your content may simply stop working.
If you file a bug against a future version of the sim saying your scenery pack used to work and is now broken, and we find that the old scenery pack had errors, we’re not going to fix the bug – we’re going to laugh maniacally and dance around you in a circle while singing “told you so”.
Okay – we’re very unlikely to do that – but if you have errors in your scenery pack, you’re doing something wrong and you need to fix it – treatment of illegal data is not stable between versions of the sim!!
-
I was never very sympathetic to this whole bug report because X-Plane has never accepted a DSF with missing objects – this has been a fatal* error since X-Plane 8.0 when DSF was introduced. So I simply don’t understand why there are any scenery packs floating around with objects missing. Why would you place an object if you don’t want to see it? The whole issue strikes me as a total failure to check quality by authors, since even running your pack once would reveal this kind of problem every time!
-
The motivation to make missing objects illegal comes from version 7 and ENVs. When looking at ENV scenery, I found that a large number of ENV scenery packs were missing at least some of their objects (an error that was silently ignored in ENV). It seemed like we were hiding an error and the result was authors not noticing simple mistakes that might “lose” an object (e.g. renaming an OBJ file).
Hence the “harsh” policy for DSFs – it was in response to a real problem with existing scenery!
- You don’t need to have missing objects just because you use library objects from another scenery pack (that might not be around). Use the EXPORT_BACKUP command in your library and a single blank OBJ as a place-holder for the objects you want from a library that might be missing. OpenSceneryX provides a stub library that authors can include so that their scenery will load without errors even if the OpenSceneryX library is not installed.
Anyway, Austin was right to make the error non-fatal. Besides being a little bit nicer for users who don’t know (or care) why their pack is gone) it lets authors get a list of all missing objects with only one run of the program.
* Fatal? In computer terms, a fatal error is one that makes the program quit, e.g. the error is fatal because it kills the program.
I have said this before, but now it’s finally true: new file specifications are subject to change in the middle of beta!
In particular ATTR_light_level has changed slightly from beta 7 to beta 8. If you are using this feature in your objects, you will need to update your objects.
A new ac3d beta will be posted later today that supports the updated syntax.
You can read about the syntax here.
I get asked about the maximum mesh density in X-Plane a lot.
First, I must note that X-Plane’s mesh is adaptive – the triangles are not arranged in a grid, but rather they’re arranged to maximize the quality of the mesh with minimal triangle count, while preserving the outlines of water bodies and airports. These pictures show a dynamic mesh fit to a section of the grand canyon (both with and without the shaded DEM for reference):
Second, the actual maximum mesh density is almost certainly limited by system resource constraints. The maximum spatial resolution in a DSF is about 0.6 millionths of a foot. You’re going to exceed the maximum number of vertices, or much more likely, the maximum amount of available memory in X-Plane long before you have a grid at that resolution.
(That doesn’t meant that resolution isn’t usable – remember, the idea of the adaptive mesh is to have very high densities in very improtant places, but not everywhere.)
But in practice this is all moot if you don’t code your DSFs by hand – the question that really matters is: what is the highest density mesh that you can use with the current scenery tools.
Until recently, MeshTool was limited to a 1201×1201 (90 meter) input DEM. The input DEM is a simple grid, and the finished DEM is never higher resolution than the input. So until now, it would have been just about impossible to make a really high res mesh, even if X-Plane could handle it.
The most recent changes to MeshTool fix this – MeshTool can now handle a 10801 x 10801 (10 meter) input DEM, which X-Plane should be okay with as well.
This new version will hopefully go into beta relatively soon.
What is the difference between a dataref and a command? They serve different purposes in X-Plane, but it’s easy to get them confused, especially because the names can look so similar. If you only take one thing away from this comparison, it should be:
- Datarefs are information.
- Commands are actions.
Datarefs
A dataref is a single bit of published information. For example, the user’s indicated airspeed, as seen by the pilot, is a dataref, stored in:
sim/cockpit2/gauges/indicators/airspeed_kts_pilot
Datarefs have names that do not change. Datarefs made available by X-Plane start with sim/ while datarefs made available by plugins start with another prefix. Datarefs have been in X-Plane since the release of the plugin system in version 6.70.
You can always read a dataref, but sometimes you can change it. Trying to change a dataref usually has one of three actions:
- If the dataref is not writable at all, nothing happens.
- If the dataref is writable, it will change.
- Sometimes a dataref may be writable, but only after changing some other sim configuration. For example, you can only “write” to the control surface deflection datarefs after setting the control surface override dataref to 1. (If you don’t set this override, X-Plane will constantly write its own ideas of the control surface positions to the control surface datarefs and your changes will be lost.)
You can read and write datarefs:
Commands
A command is an action that the sim can take on your behalf. For example, the command
sim/autopilot/altitude_arm
arms the autopilot for altitude hold.
Like datarefs, commands have permanent names, starting with sim/ for X-Plane or other prefixes for plugins. Commands have been available in X-Plane since version 9.0.
You can always actuate a command, but there is no guarantee that it will do anything. For example, the engine starter command won’t start the engine if the plane has electrical starters and the battery is dead.
You can use commands by:
Plugins
Plugins can add both new datarefs and new commands to the sim. Plugins can also change the behavior of all built-in sim commands, and can change the information in some datarefs.
Where Do I Find Datarefs And Commands
X-Plane’s default commands and datarefs are listed in the text files Commands.txt and Datarefs.txt in the Resources/plugins folder. (Note: providing the command list is new to X-Plane 930.) The dataref list is also available on the X-Plane SDK Wiki.
Up next: when should I use a command and when should I use a dataref?
I’ve been hacking at the scenery tools over the last few days…and I think we’re making some progress. I mention this because it can be hard to see the work getting done – there is nothing for months, then a new version of a tool. (Well, now you can watch the code change as it happens. If you don’t think this like watching paint dry, you might be a geek!) I also mention it because two of the biggest improvements aren’t my own – they’re insanely helpful user contributions.
The AC3D Plugin
Version 3.2 is now in beta. Version 3.2 catches the plugin up with all of the latest OBJ tricks, including dataref-controlled LIT levels, manipulators, hard surfaces, etc. This might be more useful for airplane modelers than scenery modelers, but I think the variable _LIT channels can be used for some interesting scenery effects too.
WorldEditor
WED 1.1 will go beta once X-Plane 930 is out of beta, and will feature (very primitive) overlay editing. It’s not a thing of beauty (yet) – the overlay editing features are leaner than the airport editing tools were. But 1.1 will allow you to edit an overlay and airport in one workspace.
I can tell you now that the area where WED 1.1 is weak is preview. But I don’t want to hold off the ability to edit overlays in WED “at all” just because it’s not as nice as it could be (and someday will be).
It should be possible to work on an overlay project in both WED and OverlayEditor – for example, you could build a taxiway layout in WED, using the overlay capabilities for custom pavement and lines, but use OverlayEditor to place 3-d objects (with a 3-d view).
MeshTool
To be blunt, MeshTool in its current form is really hard to use. Part of that is expected – MeshTool is a “back-end” tool like DSF2Text, aimed to simplify the problem of creating base a base mesh for some other program. But even as a back-end, MeshTool is difficult to use because it has bugs in its polygon processing, which often result in cryptic and hard-to-diagnose crashes when used with even relatively simple input data.
The good news is that I have my base mesh generation code running with a new polygon processing library (
CGAL) – the switchover from my old buggy code to CGAL was done by Andrew and contributed back as a patch. I’ve been throwing all sorts of weird data at the polygon cutting algorithms and they are rock solid.
The mesh generation code is part of MeshTool, so the next build of MeshTool will feature these processing improvements. That’s where Janos’ extensive work on the build environment comes in – he’s basically turned the scenery code from a big pile of my mad ramblings into a sane multi-platform project. This will mean a much simpler, faster process for releasing tools, particularly on Windows and Linux.
(Previously once I got a tool working, I would have to waste a few days trying to get the code to work on Windows too, slowing the whole process down.)
I expect to recut the “scenery tool” pack (that is, MeshTool, DSF2Text, and all of those other smaller apps) some time during WED beta, after 930 goes final. If all goes well, there should be native versions of the command-line tools for Linux.
MeshTool Needs a UI
Fixing MeshTool’s polygon processing is necessary, but not at all sufficient to make it a useful tool; to be really useful, it needs a real user interface. I think there are two strong possibilities for this:
- WED could develop base mesh editing. Under this scheme, you would be able to draw land use polygons in WED and create a “DEM” object for the elevation.
- MeshTool could be an output option for QGIS (via some kind of plugin or script).
I don’t know which it will be yet. The advantage of WED is that an author could have all scenery elements in one place. The advantage of QGIS is that it contains advanced GIS features that I will probably never put in WED. It may be that QGIS’s UI is simply not intuitive to less advanced users – I don’t have good perspective on this.
Posted in Tools
by
Ben Supnik |
I just posted the new X-Plane AC3D plugin (3.2 beta 1). For the info, please subscribe to the x-plane-scenery yahoo mailing list. I will post links on the scenery website once the plugin has undergone more testing; during early beta I only need a few testers to tell me I broke things.
Please read the README that comes with the download completely!
An important note for anyone using an existing .ac file to make airplanes with panels:
The new plugin gives you direct control over manipulations. But older .ac files don’t have the manipulator set on any of the objects. Thus if you export your airplane, your panel texture will work, but the panel will not be clickable.
To fix this, for each object in the hierarchy that has panel texture, select the object, open the X-Plane Object Properties… dialog box, and change the manipulator from “None” to “Panel”. (If you don’t see this option, make your properties window a bit taller.)
Note that if you don’t need your panel to be clickable, setting the manipulator to “none” is slightly faster in X-Plane 930 and a lot faster in X-Plane 922.
Other details: you’ll need the Commands.txt and DataRefs.txt file from X-Plane’s Resources/plugins folder.
Panel sub-regions are now handled quite a bit differently – please be sure to read the README completely. If you were using the 3.1 plugin with panel regions, you may need to update your .ac file a bit.
Thanks to Janos (“sothis” on the .org) we now have a GIT repository of the scenery tools, with public browsing of the scenery tools code.
(Non-programmers – this basically means that source code updates for the scenery tools will now be available every hour, rather than every now and then when I get around to it. The rest of this post is for programmers.)
The X-Plane tools code have always been open source, in that the LR-created code is distributed under the MIT/X11 license (which basically says “do whatever you want, don’t sue us”). The public repository makes the process of getting the code a lot simpler:
- The master code is actually in CVS, but this public GIT repository is updated from CVS once an hour, so this code is very close to the latest we have.
- The full version history, tags, and other information that might be useful to a programmer are all present.
- The web interface supports online browsing of the code, as well as downloading a “snapshot” of the entire tree (as a zip, gz, or bz2 file).
Git is, to put it mildly, a confusing tool if you don’t already use it. However, the web interface allows you to simply fetch the code from a given date. If you are a git user, git cloning is supported via http, and we are working on getting the git daemon running too. The repo is read-only; if you want to send us a patch, contact me.
(Git users will note that most of my checkin comments are really lame. This is a bad habit that comes from using CVS too much. CVS’s checkin comments are per-file, not per-group, which makes them somewhat useless to search on. Typically CVS users rely heavily on tags. The bridge from CVS to git tries to group them into a single commit, which helps reveal the actions taken on the source code.)
In particular, when a new spec is being developed, during beta it may change. (During a beta of a future version, old specs won’t change.)
So…please do not release non-beta aircraft and scenery based on beta builds like 930.
Here’s an example: the current spec for attached objects is that the draw order is based first on lighting mode, then on the order listed in Plane-Maker.
It turns out that if we do that, polygon offset can’t be used in a number of weird cases. So the rules will have to change. I’m not sure what they will change to, but the decision will be finalized when 930 is finalized.
