Update: the comments form is fixed – sorry about that!
I’ve been working a bit on scenery tools this week. Here’s a road map of some of what’s coming up.
WED and the X-Plane Scenery Gateway
We are working on a release of the latest scenery gateway airports for X-Plane 10.45. Like all releases, we’ve found problems with airports that WED does not detect. So I will try to release a new version of WED with stricter validation soon.
Airport Parking Spots
Austin is working on new code to place static aircraft at unused ramp parking spots. I’l describe this in more detail in another post, but here are some key points:
As an airport author, you simply place ramp starts, not static aircraft.
X-Plane will feature some new static aircraft in the update.
Third parties can further add to the static aircraft and have them be used via the library system.
There will be a revision to the apt.dat format that stores more data per ramp spot and a WED update to support this new data.
Since there may be scenery now with static aircraft on top of ramp starts*, we will auto-remove static aircraft from ramp starts in the gateway export as a temporary measure until authors can resolve the situation and post the fixes to the gateway.
We are not removing the old static aircraft from the library, but we will be hiding them from WED’s library so that authors use ramp starts instead of placing them by hand.
Static aircraft in parking spots will ship next year, not this year, but is planned as a v10 update.
Scenery Tools on OS X
I made significant progress this week porting WED for OS X to 64-bit and modern Mac APIs. This effort basically meant rewriting the OS-level UI code in WED to be new AppKit based calls instead of the old Carbon API.
What this means is that developers will be able to build all of the scenery tools on a modern Mac running Yosemite or El Capitan with X-Code 7. Since Chris has already updated the projects to compiled WED in MSVC, a developer can build all of the common scenery tools from the most widely used open source IDEs.
It also brings us closer to a 64-bit WED on Mac, which is desperately needed since OS X provides the least amount of usable address space to 32-bit apps. I am not sure of the 64-bit status of the Windows WED build; it may need more work on the libraries.
Will It Blend
This might be the most significant scenery tools development: we’ve been working with Ondrej (der-On on Git-Hub) on new features for the Blender 2.7 exporter. This will include:
Much better animation support. Complete WYSIWYG animation of data blocks and bones, with no work-arounds needed. Bone animations match Blender 2.49 so you can move your projects forward.
Modernized material support including instancing for a really clean work-flow that takes advantage of X-Plane 10’s features.
Updates for the latest OBJ features.
Our plan is to create a migration script to bring 2.49 projects into 2.7, converting the special tags used for an X-Plane 2.49 export into 2.7.
Many of our artists still use 2.49, but there’s no question that 2.49’s days are numbered. It’s a question of when it stops working on OS X, not if. With the migration step, we can move our projects forward without artists having to redo work.
Like the previous 2.7 exporter, this one is open source and will be free, and should be able to export existing 2.7-type projects with no modifications – we are trying to maintain full backward compatibility.
* This situation is slightly silly – if the user picks the ramp start to fly, the user will be on top of a static aircraft.
This may have happened to you: you import the latest approved airport scenery pack from the X-Plane Airport Gateway and modify it. When you go to export your improvements back to the gateway, WED says your work is invalid and has a problem — but not in the changes you made!
Huh? If this was the approved airport on the Gateway, how can it also be invalid in WED? How did that other author upload the airport before?
You Get a Free Pass
What do we do if we find that scenery and airports have bad data that is causing bugs? What if we can’t just fix the scenery pack for you? The policy we’ve been using is: “you get a free pass until your next scheduled maintenance; then you have to fix the bug.”
The X-Plane Scenery Gateway is a good example of this. Sometimes we find new categories of authoring mistakes – I write improvements to WED’s scenery validation function to catch these authoring errors. These are errors like:
Typos in taxiway signs – the old syntax makes a sign with incorrect letters.
Overlapping ATC routes – with the overlapping routes the AI aircraft do not taxi correctly.
In other words, these are situations where the WED scenery pack, if left alone, is causing clearly bad things to happen inside X-Plane. This isn’t a case of “old style or new style”, it’s a case of “broken or fixed.”
So what we do is we set WED to reject these scenery packs on future uploads, but we do not delete the airports from X-Plane itself. In other words, the airports with serious mistakes get a free pass until the next time an author does scheduled maintenance.
Then when an author is working in WED and is going to update the airport anyway, the author must fix the problems we have found.
No One Likes a Fire Drill
This strategy is a compromise between two extremes:
If we force you to fix your airport right now (e.g. “fix the airport or we remove it from X-Plane”), that’s not fun, that’s a fire drill. And having been exposed to plenty of fire drills myself dealing with the new OS X 10.11 and Windows 10 releases, I’m sympathetic to authors not wanting to have to stop everything to deal with an issue ASAP.
If we never require that these kinds of problems be fixed, they simply won’t be fixed. The overwhelming evidence that I have seen from working on authoring tools for X-Plane for over a decade is that some authors won’t fix problems unless the tools force them to do it.**
So requiring the change when the author does maintenance but not requiring the existing shipped scenery pack be modified strikes me as the best possible compromise: we still get quick responses to serious bugs, but we don’t force anyone to drop everything.
* I do realize that some authors are incredibly diligent about getting their scenery to be correct even if the tools don’t force them to do so! But the goal here is to have all scenery be correct; it only takes one broken scenery pack in the hundreds a user installs to crash X-Plane.
I’ve been meaning for weeks now to write up some notes about 10.40. I’ve also been trying to put the hood back on 10.40 so we can get it to public beta; instead the last few weeks have been a flying circus of driver bugs, expiring certificates, etc.
But we are making progress, so I’ll start off in this post by describing a change in how DSFs are loaded and the wide DSF box.
A Tangent: Stuttering and Pauses
A post that involves X-Plane pausing during flight is going to invariably bring up a bunch of blog comments: “X-Plane stutters while I fly on my really expensive machine!” This is not good, and from what I’ve been hearing, it sounds like something got worse recently; this is something that we are investigating now.
But I also want to mention that historically, X-Plane has never been “no-pause”. What we have done is periodically made the pauses much shorter; our goal is to get down to zero pauses, ever, but this will happen by finding every source of pausing and fixing them one at a time. In other words, we’re in the middle of a process of improving smoothness, but even if one source of pausing is fixed, another source might still be causing problems.
DSF Loading: the Old Way
X-Plane has, since X-Plane 9, loaded DSFs in the background on a second core while you fly. This cuts down the amount of time it takes to change scenery. (Older versions of X-Plane would pause while scenery was loaded and shifted.)
The old DSF loader did have a few weaknesses though:
The sim pauses while DSFs are deleted. As DSFs become bigger, this pause is becoming more noticeable.
If the loader ever gets behind by two scenery shifts, it just waits until it catches up. This is where the dreaded “Async load timed out after 30 seconds” comes in – it indicates that the DSF loader was so far behind that it locked up for half a minute and didn’t catch up.
The old DSF loader loads one DSF at a time, tops.
DSF Loading the New Way
X-Plane 10.40 has the new DSF loader, which both loads and unloads DSFs on worker threads to keep flight smoother. It also will load more than one DSF at a time, limited only by the requirement that adjacent DSFs not be loaded at the same time.
X-Plane 10.40 also has the option of an extended DSF scenery region for sharper terrain; with this option off, two DSFs are loaded at one time during sim boot and one or two are loaded at a time while you fly. With the extended DSF region on, up to four DSFs are loaded at once during sim boot and one or two are loaded while you fly.
The extended DSF scenery region is optional; don’t use this if you’re using HD meshes or you’re short on RAM. The new DSF loader is always on.
There’s currently no good way to get feedback from other users.
This is a shame, because when you upload scenery to a site like X-Plane.org, you get to hear from real people using your scenery—they might tell you how much they appreciate your work, or suggest ways you could improve it.
There’s a tradeoff here: because all airports on the Gateway are periodically exported to X-Plane, users don’t have to seek out your scenery in order to enjoy it; they get it automatically.
That means your work has a much wider impact—it benefits hundreds or even thousands of times as many users compared to posting to a download site. But, it also means people who use your scenery probably don’t know your name.
So, if you’re a Gateway artist, I’d like to hear your thoughts on a couple things:
How do you feel about the current situation? Do you want a way to get feedback (and kudos) on your Gateway submissions?
If so, what would be the best way(s) to give that feedback? A few possibilities I can think of include:
A “like” button for your scenery pack or the airport as a whole
Pros: Easy to understand, zero friction for people leaving feedback (means more people are likely to use it)
Cons: Impersonal (compared to text-based comments like “I love this scenery!”), doesn’t solve the problem of hearing from X-Plane users who never visit the Gateway
The ability for other users to leave comments on your scenery pack
Pros: Very personal, the “standard” way to get feedback on other download sites
Cons: Higher friction (fewer people will use this compared to a “like” button, for instance), doesn’t solve the problem of hearing from X-Plane users who never visit the Gateway
Automatic estimates of how many X-Plane users see your scenery pack each month/year—for instance, you submit a scenery pack for KBOS, and you see that x,000 people fly there each month
Pros: Gets “feedback” (such as it is) from users who never visit the Gateway, gives a very clear indication of how many users you impact
Something else entirely?
Disclaimer: As with any discussion of future features, I can’t promise we’ll implement any of the above… but I can tell you we’ll consider it.
So, drop a comment below and let me know what you think!
UPDATE: As of July 11, we now have a basic discussion system on the Gateway which integrates with the existing bug reports. Give it a try and let me know what you think!
I just posted WorldEditor 1.4 – “WED”, as we seem to call it most of the time – release candidate one for Mac, Windows and Linux – see the WorldEditor page for download links, report bugs on the gateway bug reporter.
Besides fixing a few bugs, this build has a new certificate so uploading/downloading from the gateway will work.
WED has a validate function that finds authoring problems in your scenery packs; two notes on this function:
First, I have been trying to improve the validation every time an issue comes up with a scenery pack. Therefore WED 1.4’s validation is quite a bit more strict than 1.3’s validation.
Once WED 1.4 goes final, it will become the required version to upload to the gateway. This ensures that we have the strongest validation, and saves Julian from having to hand check authoring issues.
This means that when you go to make a modification to your existing work, you might not be able to resubmit it until you fix existing problems. Your project didn’t change, the validation just became tighter.
Second, I have added the ATC layout error checks to the validation step. These checks (select crossing routes, select doubled nodes, select degenerate route lines) have been available since WED 1.2, but you now have to have a ‘clean’ layout to build your project. I’m hoping that this stricter validation will help fix layout bugs; when the layout is wrong, the AI aircraft’s taxi routes become even weirder than normal.
Update: WorldEditor 1.3.2 is out now and has new certificates to work with the gateway; get it here!
I screwed up: WorldEditor 1.3.1 contains a certificate that allows it to authenticate that the X-Plane Scenery Gateway is who it says it is before WED transmits your user name and password during an airport upload. And this certificate expires in about two hours.
Last night we cut a new build (1.3.2) with a new certificate with a much longer time range, but Tyler said that for some reason the new certificate did not work. So it’s most likely that we’re going to run out of time before we get a new WED build posted. Here’s what this means:
You will not be able to upload airports to the gateway with WED 1.3.1.
Once WED 1.3.2 is available, you will be able to upload airports using WED 1.3.2.
Every other function of WED will keep working.
The Gateway’s public web page will keep working.
I’ll post an update here when we can get WED 1.3.2 “live” – unfortunately it will probably be more than two hours. I’m hoping to have this solved by the end of the day.
I will also cut a new WED 1.4.0 beta with the latest bug fixes and an updated certificate. That should be available tonight.
As a side note, I think that everything that is “must fix” for WED 1.4 is fixed, so this will be a WED 1.4 release candidate. We are deferring jpeg-2000 support out of WED 1.4 entirely so we can ship it.
Last night I posted new versions of WorldEditor, MeshTool, and our command line tools. Follow the “tools” menu on this page to find them – developer.x-plane.com is the official home of our Laminar Research’s scenery tools. (We’re migrating scenery.x-plane.com and wiki.x-plane.com to just be on this site.)
Please remember that the scenery tools bug share the gateway bug reporter; please search the gateway bug reporter before you report a new bug – maybe your bug has already been reported. (You do not need to log in to Jira to search!)
WED 1.4 Beta 2
I have packaged a README with WorldEditor, and including release notes on all fixes since beta 2; bugs filed to the gateway reporter are also updated. Unfortunately I do not have release notes dating back to 1.0.
At this point there are still a number of Linux UI bugs, and Geo-JPEG 2000 support is still out. At this rate I expect to not ship Geo-JPEG 2000 support in WED 1.4 at all.
MeshTool 3.0 Beta 1
This is the first public beta of MeshTool 3 – so far there aren’t new features compared to MeshTool 2; the main change is that MT3 builds X-Plane 10 style DSFs using X-Plane 10 land classes. It is therefore appropriate for add-ons that target X-Plane 10 only.
I have linked to the config and land class files for both versions of MeshTool on the download page; it is important you use the right land class and config files for your project. (Upgrading MeshTool without replacing the config file or land class data won’t work.)
Scenery Tools 15-3
This is a recut of the command line tools. Not much has changed.
ObjConverter is no longer included; right now we don’t have a compiling version for Windows, and frankly I can’t think of a good reason to use this tool ever.* If ObjConverter was, for some reason, part of your workflow, you can still download it from the 12-2 tools release; email me and I can also explain how to use a different, better option to convert your objects.
DDSTool now defaults to sRGB gamma on input files. Both the old and new version read gamma information from your source PNG file, but if the PNG file is not tagged properly (and it’s very easy to have tagging go wrong, particularly when Photoshop** is involved) you would get classic Mac gamma in the old version. This is basically never what you want; the new recommended work-flow is to always work in sRGB, use the new DDSTool, and you’ll always get the right results.
* ObjConverter tried to convert 3DS and DXF files directly to OBJ. But since there is no standard encoding of animations, materials, and other X-Plane specific data in 3DS, the converter could only copy the mesh and texture map. This made it appropriate only as a way to get from one 3-d program to another.
But even this use is not a good way to move your data, because it strips out animation and meta-data. My suggestion is that you export directly from 3DS using an export script, or open the source 3DS or DXF file in a modeler that has native X-Plane support like Blender or ac3d.
X-Plane’s OBJ format is not meant as a way to move your models between programs; it is meant only as a final destination format for shipping your scenery.
** The problem with photoshop isn’t that it writes the PNG files incorrectly, rather the problem is that Photoshop is way too smart; it writes a full color profile when libpng only understands a few simple constructs like sRGB and gamma. So what I was seeing was libpng not understand the sRGB color profile from Photoshop because the encoding was too complex.
Defaulting to sRGB is a bit of a band-aid, but it also is what everyone should use all of the time.
X-Plane 10.40 will have an option to load a larger local region of DSF scenery. For as long as I have been involved in X-Plane (back to X-Plane 6 as a user) the local scenery region was 3×2 tiles (each 1×1 degree in latitude and longitude). With this option, the region is 4×3.
What this gets us is the option for a longer viewing distance before we have to transition from the higher detail DSF scenery tiles to the lower resolution whole-planet render. In X-plane 10 the planet render actually has shape, but the resolution is low; if you see it up close, it does not look good.
Some fine print:
You will only be able to use this option in the 64-bit build of X-Plane. The 32-bit version does not have enough memory.
Combining extended DSFs with heavy third party scenery may be unacceptably slow. For example, Alpilotx was able to run extended DSFs with the HD meshes, but his computer has monstrous amounts of RAM (64 GB I think??). I’m pretty sure extending with the UHD meshes is a non-starter.
Load time shouldn’t be too bad; this change also includes a re-work of the DSF loader that takes better advantage of multi-core hardware. If you have a 4-core machine your DSF load time shouldn’t be worse, even with extended DSFs.
Here are two sets of pictures taken over the demo area at extreme res on my PC; this shows the interaction between atmospheric scattering and loading more DSFs. The camera is at about 30k feet.
The combination of pushing the transition to the planet “out” away from us and using scattering to remove color detail starts to get something that looks more like the real world.
Note that to get the match-up in the lower right, you must have Earth Orbit textures (which come with any full install) and you must be in extreme res or the planet starts to get fuzzier.
Here’s another set.
In the long long term, I expect the planet to improve in render quality (with at least a 2x boost in image quality, and perhaps better than that in mesh shape), and I expect scattering and other lighting to improve in quality.
I do not expect to further extend the DSF box beyond 4×3; I think that the planet can improve to further “bridge the gap.”