Category: Scenery

Customizing Spill Lights – Two Ways

If you are creating an airplane or scenery pack and you need to create a fully customized spill light (visible in HDR mode only in v10) there are two ways to do it.

First, we have a parameterized light called “full_custom_halo” that takes nine parameters, e.g.

LIGHT_PARAM full_custom_halo 0 20 0   1 0.5 0.5 1    50  0 -1 0   0.5

(Phew!)  The first 3 numbers 0,20, are the XYZ position (this light is 20 meters above our object origin).  The next four are the RGBA color (100% red, 50% yellow, 50% green, 100% brightness*), so it’s a very light red.  RGB colors for spills are in linear space, so you’ll have to experiment to get your colors perfect.  The size is 50 meters (a huge light throw), and it points in the direction 0, -1, 0 (straight down).  0.5 is the “semi-width” of the cone – 60 degrees in this case.  Basically cosine(cone width * 0.5) gives you this width parameter, or pass 1.0 for an omnidirectional light.

This light is on all the time – use full_custom_halo_night for the night version.

There’s a second option: OBJ files have a custom spill command:

LIGHT_SPILL_CUSTOM 0 20 0   1 0.5 0.5 1    50  0 -1 0   0.5  your/dataref/here

The position and 9 light parameters all have the same meanings, but in this case you get to provide a dataref.  The dataref must be a float-array dataref of size 9; when your dataref callback is called, the array is pre-initialized to the 9 params, and you can change them at will.  With this dataref, your plugin can change the light parameters in real-time.

Which technique should you use?

  • Use LIGHT_SPILL_CUSTOM if you need the dataref to do clever customization.
  • Use the PARAM_LIGHT with the nine params if you don’t need the dataref.

For scenery the param-light version may be notably faster when used many times in an object.  So if you’re building a light used a lot (a streetlight, a taxiway light, an airport lighting fixture) you really want that param version.

Update: this was not documented at the time, but if you are using datarefs with LIGHT_CUSTOM_SPILL, the dataref format does not match the OBJ8 format.  This page now documents the right format for plugin objects!

* The mathematically minded will note that there is no need to have an alpha on lights because they are additive.  This is true – just set alpha to 1.0 and use darker colors!

Posted in Modeling, Scenery by | 2 Comments

Conflict Resolution and User-Submitted Airport Buildings

This came up multiple times in the comment section, and is probably important enough for its own post: how are conflicts resolved between user-submitted airport buildings (via the lego bricks and Robin’s database) and custom scenery?

The answer is two-fold:

  1. Scenery pack prioritization
  2. Exclusion zones

The user submitted airports are in a single pack; the intention here is that they be given the lowest priority of all custom scenery.*  If you install a custom scenery pack after getting 10.21, this will happen automatically (because X-Plane installs newly discovered stuff at the highest priority).  If you already have custom scenery installed, you can change its priority in the .ini file or simply remove and re-add your packs if you prefer to sort them alphabetically.

Airport layouts for custom packs will automatically replace the default ones – we only load apt.dat from one pack – this is just like before.

For the actual buildings, custom scenery packs should include exclusion zones to remove anything below it.

This is not new.  Custom scenery always should exclude what is below it.  I’ve been meaning to mention that in a separate post: custom scenery should exclude anything that would “screw it up”, not just things that have already caused problems!  If you make custom scenery and you do not “defensively” exclude, you’re likely to get broken by other scenery packs, as well as the default scenery (either via a user submitted airport or a recut).

If you squint and look at this situation, it isn’t really new — we’ve had a scenery pack with all of Robin’s submitted apt.dat layouts for several years now, and it has never been possible to add/remove individual airports.  Instead custom scenery goes on top and replaces.  We are doing more of the same for buildings.

* We would have liked to have put it in “Global Scenery”, but it is important that the airports be higher priority than any custom base meshes that might be installed.

Posted in Scenery by | 8 Comments

If It’s Not Blue, You Know What To Do

First, please read these posts regarding OSM water and recuts.  I’ll be here when you’re done.

Okay good! So the short version of it is that there are three reasons* why water might be missing from the global scenery:

  1. Our OSM data importer had a bug.  What can you do to help in this case?  Nothing; it’s on us to fix the bug.
  2. The OSM base map was missing water back in July of 2011, but the water is in OSM now.  What can you do to help in this case?  Nothing; we just need to get this new data.  (Or perhaps it is because of you that the water is in OSM, in which case thanks!)
  3. The OSM base map is missing water now!

In this last case, you can help; if the OSM water data is wrong, the global scenery will be wrong in the exact same way.  Our goal is: if it looks blue on the OSM main map, it should be wet in X-Plane.

With that in mind, I was surprised to find out about this edit: //www.openstreetmap.org/browse/way/80118306.  That’s a change to the dry lake bed in Edwards Air Force Base by alpilotx.  Surprisingly, as of 24 hours ago the tagging in OSM was still wrong, just as it was in July of 2011.  I had assumed that Edwarads was ‘wet’ due to an import bug by us, but apparently it was due to incorrect tagging.

So if the base map doesn’t look blue, you know what to do: add the missing water.  It’s good for OSM, it’s good for X-Plane, it’s good for pretty much any project using OSM data (and there are a lot of them!) and once OSM is fixed, the bug can be fixed in X-Plane forever because OSM is the data we use to get water/land/coastline info.

* There’s actually a forth reason: intentional removal of water that we thought was too small/detailed for the data size of the global scenery.  In that cases there’s nothing to be done to put it back — the global scenery has to be able to cover the whole world without being too many tens of GB.

Posted in Scenery by | 12 Comments

DSF Recuts – Some Basics

I have a lot to blog about this week, but before I can get into any of it, let me describe the process of recutting DSFs.

Alpilotx and I are now working on DSF recuts.  The recuts will incorporate a few important changes:

  • Bug fixes to the OSM import code.  Some of the major cases of ‘missing water’ (e.g. Botany Bay) were due to a bug in the code that imports vector water day from OSM into our DSF generator. Once we fix this code, recut DSFs will get back water that sould have been there but was “lost on import.”
  • Newer OSM data.  The OSM data in the simulator now comes from approximately July of 2011.  OSM is growing fast and being improved every day; for the recuts, we will take the latest data.  If water was missing because it was not in the map in 2011, but that water is present now, recut tiles will get the new water.
  • Airport boundaries from the latest airport file from Robin.  Due to a last-minute data screw-up by me, many airports in the original DSFs were cut with very old airport data, and thus the airport boundary is wrong or missing.  Even the airports that were cut with current data have the airport areas from November of 2011.  Recut tiles will use the lastest (March 2013 when I last checked) data.
  • Better land-class allocation.  While the source land-class data that alpilotx prepared for me is very, very good, the resulting land class in the DSFs is sometimes inaccurate due to the limits of the algorithm I apply to it.  I am working on improvements to that algorithm.  The details of these changes are probably too technical for this blog, but the short of it is that I am trying to fix as many of the algorithmic problems that we’re not happy with as I can in the short amount of time I have been allocated.

A Tactical Recut

At this point you are asking the two most important questions: which tiles will you recut and how will you get them?  The answer to both is: we are starting the process by recutting a small number of tiles that we think are most important to recut, and pushing them out over the network as part of the update process.

At this point we have enough bug data from users that we have a pretty good idea of which tiles need to be recut first.  For example, trees on Chicago’s runway, Botany Bay not existing, and Edwards AFB containing a real (and not dry*) lake have all been reporting about 5,290 times each.  So we will pick the first set of recut tiles ourselves to fix the most visible, highly reported bugs.

But the goal here is also to start an assembly line that can mostly run by itself.  That is, once we get these bugs fixed, we can potentially recut a fixed number of DSF tiles every month and include them in an update, which will move us closer to having the scenery be “live” like the sim and not frozen for the entire version run.

I think moving to a live model for the base DSFs is an important step that we have to make.  Back in X-Plane 6 when I first started flying X-Plane as a user, Austin sending out patches with a new version of the sim every month or two was viewed as crazy.  Look at how far we’ve come: at this point application updates are built into the Apple app store on iPhone (and the Mac app store), and if you don’t push updates, users want to know if you’ve been hit by a bus.  Updates are the norm.

That change is completely logical: back when I started working on commercial software (in the 20th century) distribution was by CD-ROMs, which were mostly sold in stores.  Once your software went out, that was it, so “done” was “done”.

Now software is mostly distributed over the internet; with constant connectivity and increasing bandwidth, it would be crazy for a company to not respond to its customers needs and requests by issuing updates.  One of the most important properties of software as a “building material” is its flexibility — if users want software to do something different, you can easily change that software.  With widespread internet connectivity we have the matched distribution to go with the flexibility of the software itself.

This recut is not a global world-wide recut; I do not know when or even if this will happen. Instead the recut is a first step into keeping DSFs “current”, and the beginning of an on-going process.

The Next Train Leaves in 10 Minutes

The other question that I hear a lot (besides which DSFs and how will I get them) is “when do I need to finish my OSM/apt.dat mods to get them into the recut.”

My goal with incremental periodic recuts is to avoid this issue entirely.  Once we recut a DSF, the cost of recutting it again is very low — we just hit “go” on the DSF generator and replace the files in the update.

In other words, don’t worry about when you “have” to get your updates in by.  We will recut tiles, and if you make a useful change after the tile is recut, we’ll recut it again.  Recuts should be like a train that leaves the station every 15 minutes, not once a week.

(With Robin republished the apt.dat on a regular basis again, apt.dat updates are already like this.)

Executive Summary

TL;DR?  Here you go:

  • We are recutting a few tiles at a time, not the whole world.
  • Recut tiles will come as part of the automatic X-Plane update process.
  • Tiles may be recut multiple times to gather new improvements, so don’t worry about “missing” an update when you improve source data.
  • We are fixing import/programming problems in the first set of recuts.
  • We have enough bug reports on tiles now, we don’t need more yet.

* I’m still not sure what happened at AFB, but my guess is that our importer saw “lake”, got very excited, and ignored the word “dry”.

Posted in News, Scenery by | 29 Comments

A Quick Fix for Autogen

It looks like 10.21 will not contain new autogen art assets.  Propsman has a pile of stuff partly ready to go, but had to go on a pre-scheduled trip and wasn’t able to upload the files before he ran out of time.  We’ll cut 10.22 to drop autogen in – it’s not hard for us to run an update to push art assets.

But he did send me a fix for one autogen bug that we’ve received some rather vocal complaints about:

In X-Plane 10.20 Propsman accidentally mapped some industrial warehouses to a land-class that is mostly used for small towns and rural populations.  The result is a lot of big ugly industrial buildings in places that don’t make sense.

The second shot shows the same area modified with the new patch, which puts smaller individual buildings into place.  This should help the land-scape look less disrupted.

The mapping from land-class to autogen is part of the autogen library, so as we get more art assets into place we will continue to tweak the mapping to try to get the best looking results.

Please note that this change may affect frame-rate, possibly not in a good way.  Depending on what your hardware is good or bad at, changes to the autogen will cause frmaerate changes.

Posted in Scenery by | Comments Off on A Quick Fix for Autogen

WED: WED’s Export Dies

Why the nerdy recursive acronym for WED?  Sometimes the data just stares you in the face.

I spent an hour today doing inventory on the build-up of WED bugs. That’s a screenshot of one of my bug screeners for the public scenery tools bug base – open bugs in WED 1.2b1 sorted by severity.  Notice a trend?  100% of the crash bugs come from the import/export module.  In fact, export bugs dominate the crash+major category too.

At this point I believe the DSF overlay export code simply needs a rewrite. The current code converts the polygons from my code to CGAL, then attempts to chop them up to fit into DSFs, and then converts them back and exports them. To put it mildly, this code is not working well.

As far as I can tell, CGAL 3.9 (the version we use) has reliability problems on Windows when processing Bezier curves.  I don’t know if it’s a build environment issue, something I did, or their bug, but I don’t trust the code path; we don’t use CGAL’s bezier curve processing for the global scenery project, so we don’t have strong data that this code will ever work.

Why Do We Need to Cut?

The current WED export code chops up your polygons to fit them into the DSF tile boundaries, because everything in the DSF must be contained within the DSF, right?

Well…

There’s actually nothing in the DSF file format to disallow ‘overhang’, e.g. a facade or some other shape that is partly in the DSF.  And for custom airports, having to cut buildings down the middle at lat/lon boundaries is unfortunate – the results can look ugly.

Unfortunately, as of this writing, X-Plane 10 requires the DSF to not “spill over” its boundaries.  But it looks like this could be fixed.

Multiple Export Fixes

This is my plan for fixing WED export, more or less:

  • WED will provide a “target” X-Plane version, e.g. 9.00, 10.00, 10.30, etc. This will select what kind of export features it can use.  If you try to export a project using a feature that cannot be supported (e.g. custom facade walls to a v9 airport) you will get a warning or error, depending on the nature of the problem.
  • WED export will not attempt to cut overlay data on-the-fly; rather it will either export the geometry spilling over the border (if allowed by export target version) or flag the span as an error.
  • An edit menu tool will be provided to cut geometry at DSF borders, for at least some DSF geometry types.

Why make border cutting a manual step?  I think that it will be more useful for authors to see the result of the border cut. For example, when a facade is border cut and has unique wall choices, the author has no control about the wall type of the wall that was introduced to span the DSF border.  With pre-cutting you can slice your facade, then pick a ‘safe’ wall type for the two walls that face each other at the DSF border.*

WED will currently refuse to border-cut some geometry; I think this will continue in 1.2, albeit this will happen in the “cut at borders” command – if you request to cut a bezier type WED may refuse.

* No matter which way you slice it, cutting facades at the DSF border sucks. This is why I think the long term solution is to allow facades to span borders, at least a little bit.

Posted in File Formats, Scenery, Tools by | 15 Comments

Blender 2.49 Scripts

I had a chance to work this week on the Blender 2.49 scripts; here’s a quick update on the status of the scripts and what the next steps are.

Blender 2.49 for Internal Scenery Development

Relatively early in X-Plane 10 development, we picked Blender 2.49 for our scenery development environment.  We did this for a few reasons:

  • The two artists doing scenery (Alex for cities, Tom for airports) both used Blender 2.49 as their primary 3-d tool.
  • There was already a completely functional OBJ exporter (Jonathan’s XPlane2Blender scripts, version 3.09) that produced highly optimized OBJ output.

At the time, some developers were already migrating to Blender 2.5, but Ondrej’s exporter was only partly written.  Since we didn’t need any Blender 2.5 features and schedule time was of primary importance, we decided to stick with 2.49 (to avoid the delay of our artists learning a new environment and the delay of having to wait for or develop for ourselves a complete and fully optimized exporter).

The result is that I augmented Jonathan’s scripts to:

  • Fully support X-Plane 10 OBJ extensions like instancing, the new GLOBAL attributes and draped geometry.
  • Export some of the other v10 data types (autogen, facades, roads).  Exporting non-OBJ stuff is really complicated, and the rules to use Blender to make the right meta data are not trivial.

Other Scripts

There are other variants of the scripts floating around: both Ondrej and Ben Russell added partial manipulator export to the 3.09 scripts to help aircraft authors.  Ondrej has also created a new branch (the “3.20” scripts) that target Blender 2.50 and newer.

There’s one technical point tot note about Blender that might not be obvious: for all practical purposes. Blender 2.4x and Blender 2.5x might as well be completely different programs!  They have totally different scripting interfaces and they don’t even use compatible versions of the Python language.  Supporting Blender 2.5x is a from-scratch effort and the scripts are totally unrelated to anything you can use on Blender 2.49.

There is no way to use my Blender 2.49 scripts on Blender 2.5; this is just a decision that the Blender developers made – they created a major fork in the Blender world.  So when you read about Blender 2.49 vs Blender 2.50, consider that difference to be as major as Blender vs AC3D or Blender vs 3DS.

Recent Work

This week I had time to integrate both Ben and Ondrej’s manipulator export code into my 2.49 scripts, and I also added the remaining missing OBJ8 commands that are useful for aircraft developers.  This means that my scripts are now more or less ready for aircraft use as well as scenery use, and they provide a forward path for Blender 2.49 users who use either Jonathan, Ondrej, or Ben’s scripts.  This brings me closer to my goal: making a complete set of 2.49 scripts, current to X-Plane 10.

I have a few more loose ends to tie up, but I am hoping to get the scripts pushed to github for Jonathan (and anyone else who wants them) in the next week.  There is still a lot of documentation to add.

 

Posted in Development, Tools by | 15 Comments

The X-Plane 10 Beta Library is Not Stable!

I have blogged before about the importance of not shipping products that target beta builds of X-Plane, and the importance of waiting to the end of beta for file formats to settle down.

This also applies to the library! While we are in beta, objects in the library may be renamed or disappear in ways that could break your scenery.

So if you are making custom scenery that uses our library, you have two options:

  • Ignore the beta; develop against the last stable release (10.11) and use that library, which we will continue to support.
  • Use the beta, but don’t release yet, and be prepared for objects to come and go.

In other words, if your custom scenery works in 10.11 but fails in 10.20, please file a bug – we screwed up.

But if your custom scenery works in one 10.20 beta but not another 10.20 beta, don’t use 10.20 beta objects yet – they’re not fully cooked!

Posted in File Formats, Scenery by | 7 Comments

Beta 9 and How the Rendering Settings Now Work

X-Plane 10.20 beta 9 is out and (hopefully) fixes missing library paths. Alex and I spent a good chunk of yesterday trying to clean up the tools that generate the autogen library, but it is a very complicated problem, as indicated by the 4786 distinct entries in the library.

Beta 8 and 9 introduce a change in how the rendering settings work; my suspicion is that users seeing a framerate drop in the new betas are seeing the effects of the intermediate rendering settings acting differently; the minimum and maximum for the settings should produce nearly identical results.

I’m not going to try to explain how the settings used to work – the actual results were extremely convoluted due to quirks in the engine.  This change is one of several we want to make to both improve how the settings work (from a usability standpoint) and improve the look of the autogen at lower settings.

  • The “objects” popup controls the amount of autogen and 3-d stuff in general.  Besides adding more autogen, it adds more detail to the autogen as it appears and can even add more detail to custom scenery.  (This behavior isn’t new.)  In particular, if you use an a complex library element like an “AGP” scene (e.g. the control tower or FAA building) the amount of detail of that mini-scene will increase with the objects setting.  The goal here is to have the objects slider control the amount of 3-d.
  • The “forests” popup now strictly controls the density of all vegetation in the sim.  This includes both the natural forests (.for files for DSF nerds) and the vegetation that is part of the autogen (e.g. the trees that Alex puts down on the lawns of houses).  Lower forest settings tend to prioritize the rural forests over autogen trees to save framerate.
  • The distance to which 3-d is drawn is controlled by “world detail distance” – this is how far anything 3-d is drawn out to.  Finally with this change, this affects the forests as well as the autogen and the roads – everything runs to the same distance.

A few tuning tips:

  • If you are running 32-bits, turn “world detail distance” down by at least one notch; having a shorter distance for 3-d will allow the sim to save some memory.
  • If you run with full autogen, moderate forests, and highest world detail distance, you may now need to turn forests up and world detail distance down to create the equivalent of what you used to see.  In older betas, the forest setting modified world detail distance in some very strange ways.

Our eventual goal is to keep the autogen base footprints even when the object count is very low; the footprints of the autogen contain views of the building tops, allowing for the equivalent of ‘orthophotos only’ for users with very low rendering capabilities.  This will help avoid the ‘sea of green’ effect on lower settings.  (The behavior now, where whole city blocks disappear) will eventually go away, to be replaced by the “footprint” with roofs of the city block when 3-d is turned down.

These pictures illustrate the effect of world detail distance.  Note that the ground images attached to the buildings are still present even as the buildings begin to disappear with lower settings.  The last picture shows the autogen with most 3-d buildings stripped away to reveal the ground tiles underneath.

As we keep working on the autogen system we’ll get more and better ground textures in place, more and better ground tiles on the autogen, and yet more buildings; the result should be a reasonably seamless urban experience at a range of rendering settings.

Some cities are “over-green” in their underlying data in the DSF – this happens when the source land class is difficult to ‘fit’ into the grid system efficiently.  (Typically a little bit of vegetation or water area affects zoning and ‘greenifies’ a large area.)  We will have to address this problem in DSF tile recuts.  (In other words, zoning errors are a failure of accuracy compared to the source data, to be recut; adding more autogen improves plausibility by providing a wider variety of “stuff”.)

Posted in News, Scenery by | 18 Comments

X-Plane 10.20 Beta 6:

10.20 Beta 6 is out – this build fixes the missing paths in the library that were causing the dreaded “Could not load fac/forest/beach” error. This build also puts in some important changes to 64-bit plugis, so below I will try to point out what matters to select audiences:

Users:

If you don’t make an add-on but you are still testing 10.20, there is still a kind of testing that we need (but you may not like it): is anything broken in the 32-bit edition of 10.20 that worked in 10.11?

This kind of bug (it worked in the old version but is broken now) is called a regression bug in the tech lingo, because the sim has “regressed” – it has gone backward.  This is the most important kind of bug for us to find for three reasons:

  1. No one wants to go backward.
  2. It’s easier for us to find the bugs in new features than random bugs in features we didn’t think we had changed.
  3. If you report a regression bug against two versions of the sim (it worked in 10.11, broken in 10.20) we have tools to see only the code changes from 10.11 to 10.20, which makes finding the bug much quicker.

So: please try the 32-bit 10.20 and if it fails at something that 10.11 did, please let us know ASAP!

Aircraft Developers

This build fixes the prop disc and airspeed indicator, and finally provides more datarefs for the anti-ice system; I suggest rechecking your airplane with these systems.  At this point I think I only have two airplane issues floating around on my radar:

  • The new light billboards and spill can be tricky to use, particularly when lights crash through the fuselage of an airplane.
  • The manifold pressure indication is FUBAR on helicopters.

Both of these bugs were present in 10.11.

In fact, we have no regression bugs on file in the airplane system between 10.11 and 10.20 32-bit.  So if you know of something broken with your airplane that is newly broken in 10.20 32-bit, please report it ASAP.  (You do not need to re-report bugs that were already present in 10.11 like the manifold pressure.)

Plugin Authors

This build integrates a fix for LuaJIT on 64-bit Macs; see the release notes and plugin SDK for how to use it.

This build also strips out the symbols of libpng, libcurl, and libfreetype2 on Mac and Linux.  These symbols never should have been visible in X-Plane, but they have been for years.

  • If your 32-bit plugin used to work on X-Plane 10.11 but stops working on 10.20 32-bit beta 6, please file a bug and we will figure out what to do.
  • If your 64-bit plugin use to work on 10.20 beta 5 but stops working on beta 6, your makefile or X-Code settings have a bug; post to the x-plane-dev mailing list to get help.

There may be future changes to 64-bit like this one; this is why I recommend not releasing while we are in beta!  Our goal is to ‘get 64 bit right’ now during beta while we can try and test.

Scenery

There are no open regression bugs from 10.11 against scenery; facade textures were borked and are now fixed, and the library should be restored.  So if your scenery pack worked in 10.11 and fails in 10.20, please report this ASAP!

I do have one open bug (present in all of the v10 run) that line markings are losing line segments; I hope to get that fixed by the end of beta.

(We also have a report that certain incorrect ATC/apt.dat files will hard-crash the sim – I think Chris will probably be able to get some better validation in place before beta ends.)

Posted in Development, News, Scenery by | 2 Comments