If you haven’t participated in the 10.21 betas, run the updater and click “get betas” to get it. Release notes here; report bugs here.
If you have a custom scenery pack, please try it on this RC! There have been a few changes to the facade engine to fix cases that were clearly broken in 10.20; please check your scenery to make sure these changes didn’t cause some other problem that I didn’t see in my testing.
If you are creating an airplane or scenery pack and you need to create a fully customized spill light (visible in HDR mode only in v10) there are two ways to do it.
First, we have a parameterized light called “full_custom_halo” that takes nine parameters, e.g.
LIGHT_PARAM full_custom_halo 0 20 0 1 0.5 0.5 1 50 0 -1 0 0.5
(Phew!) The first 3 numbers 0,20, are the XYZ position (this light is 20 meters above our object origin). The next four are the RGBA color (100% red, 50% yellow, 50% green, 100% brightness*), so it’s a very light red. RGB colors for spills are in linear space, so you’ll have to experiment to get your colors perfect. The size is 50 meters (a huge light throw), and it points in the direction 0, -1, 0 (straight down). 0.5 is the “semi-width” of the cone – 60 degrees in this case. Basically cosine(cone width * 0.5) gives you this width parameter, or pass 1.0 for an omnidirectional light.
This light is on all the time – use full_custom_halo_night for the night version.
There’s a second option: OBJ files have a custom spill command:
LIGHT_SPILL_CUSTOM 0 20 0 1 0.5 0.5 1 50 0 -1 0 0.5 your/dataref/here
The position and 9 light parameters all have the same meanings, but in this case you get to provide a dataref. The dataref must be a float-array dataref of size 9; when your dataref callback is called, the array is pre-initialized to the 9 params, and you can change them at will. With this dataref, your plugin can change the light parameters in real-time.
Which technique should you use?
For scenery the param-light version may be notably faster when used many times in an object. So if you’re building a light used a lot (a streetlight, a taxiway light, an airport lighting fixture) you really want that param version.
Update: this was not documented at the time, but if you are using datarefs with LIGHT_CUSTOM_SPILL, the dataref format does not match the OBJ8 format. This page now documents the right format for plugin objects!
* The mathematically minded will note that there is no need to have an alpha on lights because they are additive. This is true – just set alpha to 1.0 and use darker colors!
This came up multiple times in the comment section, and is probably important enough for its own post: how are conflicts resolved between user-submitted airport buildings (via the lego bricks and Robin’s database) and custom scenery?
The answer is two-fold:
The user submitted airports are in a single pack; the intention here is that they be given the lowest priority of all custom scenery.* If you install a custom scenery pack after getting 10.21, this will happen automatically (because X-Plane installs newly discovered stuff at the highest priority). If you already have custom scenery installed, you can change its priority in the .ini file or simply remove and re-add your packs if you prefer to sort them alphabetically.
Airport layouts for custom packs will automatically replace the default ones – we only load apt.dat from one pack – this is just like before.
For the actual buildings, custom scenery packs should include exclusion zones to remove anything below it.
This is not new. Custom scenery always should exclude what is below it. I’ve been meaning to mention that in a separate post: custom scenery should exclude anything that would “screw it up”, not just things that have already caused problems! If you make custom scenery and you do not “defensively” exclude, you’re likely to get broken by other scenery packs, as well as the default scenery (either via a user submitted airport or a recut).
If you squint and look at this situation, it isn’t really new — we’ve had a scenery pack with all of Robin’s submitted apt.dat layouts for several years now, and it has never been possible to add/remove individual airports. Instead custom scenery goes on top and replaces. We are doing more of the same for buildings.
* We would have liked to have put it in “Global Scenery”, but it is important that the airports be higher priority than any custom base meshes that might be installed.
First, please read these posts regarding OSM water and recuts. I’ll be here when you’re done.
Okay good! So the short version of it is that there are three reasons* why water might be missing from the global scenery:
In this last case, you can help; if the OSM water data is wrong, the global scenery will be wrong in the exact same way. Our goal is: if it looks blue on the OSM main map, it should be wet in X-Plane.
With that in mind, I was surprised to find out about this edit: //www.openstreetmap.org/browse/way/80118306. That’s a change to the dry lake bed in Edwards Air Force Base by alpilotx. Surprisingly, as of 24 hours ago the tagging in OSM was still wrong, just as it was in July of 2011. I had assumed that Edwarads was ‘wet’ due to an import bug by us, but apparently it was due to incorrect tagging.
So if the base map doesn’t look blue, you know what to do: add the missing water. It’s good for OSM, it’s good for X-Plane, it’s good for pretty much any project using OSM data (and there are a lot of them!) and once OSM is fixed, the bug can be fixed in X-Plane forever because OSM is the data we use to get water/land/coastline info.
* There’s actually a forth reason: intentional removal of water that we thought was too small/detailed for the data size of the global scenery. In that cases there’s nothing to be done to put it back — the global scenery has to be able to cover the whole world without being too many tens of GB.
I have a lot to blog about this week, but before I can get into any of it, let me describe the process of recutting DSFs.
Alpilotx and I are now working on DSF recuts. The recuts will incorporate a few important changes:
At this point you are asking the two most important questions: which tiles will you recut and how will you get them? The answer to both is: we are starting the process by recutting a small number of tiles that we think are most important to recut, and pushing them out over the network as part of the update process.
At this point we have enough bug data from users that we have a pretty good idea of which tiles need to be recut first. For example, trees on Chicago’s runway, Botany Bay not existing, and Edwards AFB containing a real (and not dry*) lake have all been reporting about 5,290 times each. So we will pick the first set of recut tiles ourselves to fix the most visible, highly reported bugs.
But the goal here is also to start an assembly line that can mostly run by itself. That is, once we get these bugs fixed, we can potentially recut a fixed number of DSF tiles every month and include them in an update, which will move us closer to having the scenery be “live” like the sim and not frozen for the entire version run.
I think moving to a live model for the base DSFs is an important step that we have to make. Back in X-Plane 6 when I first started flying X-Plane as a user, Austin sending out patches with a new version of the sim every month or two was viewed as crazy. Look at how far we’ve come: at this point application updates are built into the Apple app store on iPhone (and the Mac app store), and if you don’t push updates, users want to know if you’ve been hit by a bus. Updates are the norm.
That change is completely logical: back when I started working on commercial software (in the 20th century) distribution was by CD-ROMs, which were mostly sold in stores. Once your software went out, that was it, so “done” was “done”.
Now software is mostly distributed over the internet; with constant connectivity and increasing bandwidth, it would be crazy for a company to not respond to its customers needs and requests by issuing updates. One of the most important properties of software as a “building material” is its flexibility — if users want software to do something different, you can easily change that software. With widespread internet connectivity we have the matched distribution to go with the flexibility of the software itself.
This recut is not a global world-wide recut; I do not know when or even if this will happen. Instead the recut is a first step into keeping DSFs “current”, and the beginning of an on-going process.
The other question that I hear a lot (besides which DSFs and how will I get them) is “when do I need to finish my OSM/apt.dat mods to get them into the recut.”
My goal with incremental periodic recuts is to avoid this issue entirely. Once we recut a DSF, the cost of recutting it again is very low — we just hit “go” on the DSF generator and replace the files in the update.
In other words, don’t worry about when you “have” to get your updates in by. We will recut tiles, and if you make a useful change after the tile is recut, we’ll recut it again. Recuts should be like a train that leaves the station every 15 minutes, not once a week.
(With Robin republished the apt.dat on a regular basis again, apt.dat updates are already like this.)
TL;DR? Here you go:
* I’m still not sure what happened at AFB, but my guess is that our importer saw “lake”, got very excited, and ignored the word “dry”.
It looks like 10.21 will not contain new autogen art assets. Propsman has a pile of stuff partly ready to go, but had to go on a pre-scheduled trip and wasn’t able to upload the files before he ran out of time. We’ll cut 10.22 to drop autogen in – it’s not hard for us to run an update to push art assets.
But he did send me a fix for one autogen bug that we’ve received some rather vocal complaints about:
In X-Plane 10.20 Propsman accidentally mapped some industrial warehouses to a land-class that is mostly used for small towns and rural populations. The result is a lot of big ugly industrial buildings in places that don’t make sense.
The second shot shows the same area modified with the new patch, which puts smaller individual buildings into place. This should help the land-scape look less disrupted.
The mapping from land-class to autogen is part of the autogen library, so as we get more art assets into place we will continue to tweak the mapping to try to get the best looking results.
Please note that this change may affect frame-rate, possibly not in a good way. Depending on what your hardware is good or bad at, changes to the autogen will cause frmaerate changes.
Why the nerdy recursive acronym for WED? Sometimes the data just stares you in the face.
I spent an hour today doing inventory on the build-up of WED bugs. That’s a screenshot of one of my bug screeners for the public scenery tools bug base – open bugs in WED 1.2b1 sorted by severity. Notice a trend? 100% of the crash bugs come from the import/export module. In fact, export bugs dominate the crash+major category too.
At this point I believe the DSF overlay export code simply needs a rewrite. The current code converts the polygons from my code to CGAL, then attempts to chop them up to fit into DSFs, and then converts them back and exports them. To put it mildly, this code is not working well.
As far as I can tell, CGAL 3.9 (the version we use) has reliability problems on Windows when processing Bezier curves. I don’t know if it’s a build environment issue, something I did, or their bug, but I don’t trust the code path; we don’t use CGAL’s bezier curve processing for the global scenery project, so we don’t have strong data that this code will ever work.
The current WED export code chops up your polygons to fit them into the DSF tile boundaries, because everything in the DSF must be contained within the DSF, right?
Well…
There’s actually nothing in the DSF file format to disallow ‘overhang’, e.g. a facade or some other shape that is partly in the DSF. And for custom airports, having to cut buildings down the middle at lat/lon boundaries is unfortunate – the results can look ugly.
Unfortunately, as of this writing, X-Plane 10 requires the DSF to not “spill over” its boundaries. But it looks like this could be fixed.
This is my plan for fixing WED export, more or less:
Why make border cutting a manual step? I think that it will be more useful for authors to see the result of the border cut. For example, when a facade is border cut and has unique wall choices, the author has no control about the wall type of the wall that was introduced to span the DSF border. With pre-cutting you can slice your facade, then pick a ‘safe’ wall type for the two walls that face each other at the DSF border.*
WED will currently refuse to border-cut some geometry; I think this will continue in 1.2, albeit this will happen in the “cut at borders” command – if you request to cut a bezier type WED may refuse.
* No matter which way you slice it, cutting facades at the DSF border sucks. This is why I think the long term solution is to allow facades to span borders, at least a little bit.
I had a chance to work this week on the Blender 2.49 scripts; here’s a quick update on the status of the scripts and what the next steps are.
Relatively early in X-Plane 10 development, we picked Blender 2.49 for our scenery development environment. We did this for a few reasons:
At the time, some developers were already migrating to Blender 2.5, but Ondrej’s exporter was only partly written. Since we didn’t need any Blender 2.5 features and schedule time was of primary importance, we decided to stick with 2.49 (to avoid the delay of our artists learning a new environment and the delay of having to wait for or develop for ourselves a complete and fully optimized exporter).
The result is that I augmented Jonathan’s scripts to:
There are other variants of the scripts floating around: both Ondrej and Ben Russell added partial manipulator export to the 3.09 scripts to help aircraft authors. Ondrej has also created a new branch (the “3.20” scripts) that target Blender 2.50 and newer.
There’s one technical point tot note about Blender that might not be obvious: for all practical purposes. Blender 2.4x and Blender 2.5x might as well be completely different programs! They have totally different scripting interfaces and they don’t even use compatible versions of the Python language. Supporting Blender 2.5x is a from-scratch effort and the scripts are totally unrelated to anything you can use on Blender 2.49.
There is no way to use my Blender 2.49 scripts on Blender 2.5; this is just a decision that the Blender developers made – they created a major fork in the Blender world. So when you read about Blender 2.49 vs Blender 2.50, consider that difference to be as major as Blender vs AC3D or Blender vs 3DS.
This week I had time to integrate both Ben and Ondrej’s manipulator export code into my 2.49 scripts, and I also added the remaining missing OBJ8 commands that are useful for aircraft developers. This means that my scripts are now more or less ready for aircraft use as well as scenery use, and they provide a forward path for Blender 2.49 users who use either Jonathan, Ondrej, or Ben’s scripts. This brings me closer to my goal: making a complete set of 2.49 scripts, current to X-Plane 10.
I have a few more loose ends to tie up, but I am hoping to get the scripts pushed to github for Jonathan (and anyone else who wants them) in the next week. There is still a lot of documentation to add.
I have blogged before about the importance of not shipping products that target beta builds of X-Plane, and the importance of waiting to the end of beta for file formats to settle down.
This also applies to the library! While we are in beta, objects in the library may be renamed or disappear in ways that could break your scenery.
So if you are making custom scenery that uses our library, you have two options:
In other words, if your custom scenery works in 10.11 but fails in 10.20, please file a bug – we screwed up.
But if your custom scenery works in one 10.20 beta but not another 10.20 beta, don’t use 10.20 beta objects yet – they’re not fully cooked!
X-Plane 10.20 beta 9 is out and (hopefully) fixes missing library paths. Alex and I spent a good chunk of yesterday trying to clean up the tools that generate the autogen library, but it is a very complicated problem, as indicated by the 4786 distinct entries in the library.
Beta 8 and 9 introduce a change in how the rendering settings work; my suspicion is that users seeing a framerate drop in the new betas are seeing the effects of the intermediate rendering settings acting differently; the minimum and maximum for the settings should produce nearly identical results.
I’m not going to try to explain how the settings used to work – the actual results were extremely convoluted due to quirks in the engine. This change is one of several we want to make to both improve how the settings work (from a usability standpoint) and improve the look of the autogen at lower settings.
A few tuning tips:
Our eventual goal is to keep the autogen base footprints even when the object count is very low; the footprints of the autogen contain views of the building tops, allowing for the equivalent of ‘orthophotos only’ for users with very low rendering capabilities. This will help avoid the ‘sea of green’ effect on lower settings. (The behavior now, where whole city blocks disappear) will eventually go away, to be replaced by the “footprint” with roofs of the city block when 3-d is turned down.
These pictures illustrate the effect of world detail distance. Note that the ground images attached to the buildings are still present even as the buildings begin to disappear with lower settings. The last picture shows the autogen with most 3-d buildings stripped away to reveal the ground tiles underneath.
As we keep working on the autogen system we’ll get more and better ground textures in place, more and better ground tiles on the autogen, and yet more buildings; the result should be a reasonably seamless urban experience at a range of rendering settings.
Some cities are “over-green” in their underlying data in the DSF – this happens when the source land class is difficult to ‘fit’ into the grid system efficiently. (Typically a little bit of vegetation or water area affects zoning and ‘greenifies’ a large area.) We will have to address this problem in DSF tile recuts. (In other words, zoning errors are a failure of accuracy compared to the source data, to be recut; adding more autogen improves plausibility by providing a wider variety of “stuff”.)