Category: Plugins

XPlane2Blender v3.4.0-beta.5 arrives

Download here:

//github.com/der-On/XPlane2Blender/releases/tag/v3.4.0-beta.5

Change Log

New Features

Build Version Numbers

XPlane2Blender’s new version number system will allow us to debug .blend and .obj file even faster. It should also make making updates to the data model easier to implement.

  • Every exported .obj file’s footer contains information about the addon version, when it was compiled, and why. For example: 3.4.0-beta.5+1.20170922151901

A break down of the components

  • 3.4.0: The traditional Blend addon number
  • beta: The type of build cycle we’re in. Other choices you may see are dev (a developer build), alpha, rc (for full release)
  • 5: the build type version we’re on. Since this is beta 5, the build type version is 5.

The elements after the + are generally less meaningful to the average user

  • 1: The version of the data model (the properties and settings for XPlane2Blender, used for comparing changes
  • 20170922151901: The “build number” date this source code was packaged and released in the form of YYYYMMDDHHMMSS in UTC.
Communication

The build version number is partially shown (elements after the + are hidden) in the scene settings under Composite Normal-Textures, potentially along with warnings about the stability of the build you are using.

Examples

When starting this version of the beta, you will see this:
scene_version_comminucation

A future stable version of 3.4.0 for the general public will show this:
scene_version_communication_rc_ex
Notice the green check mark and lack of any warnings.

In a future pre-alpha cycle for 3.4.1, two types of people will see the following:
scene_version_communication_dev_ex

  1. A developer writing and testing unreleased code
  2. Someone who didn’t head the warning against using such code, or get scared off by the nuclear symbol and extra warning about a lack of a build number

It is the worst case scenario for stability and traceability, hence the nuclear symbol.

Why the extra warning about a lack of a build number?

A lack of a build number indicates that you do not have a good dialog (e-mail, chat, this release page, or other channels) with a knowledgeable and attentive developer. This means your work is more likely to be run through a bad version of the code and damaged, or your bugs will be harder to diagnose and repair.
scene_version_comminucation_rc_no_build_nmbr
In this case, despite the code appearing to come from a stable era of the code near a release, there is potential for something to be wrong and have very poor ways of tracing what it could be – hence the lack of green check mark and big red warning symbol.

If you see some new pre-alpha feature you’d like to try, just ask us about it first. Going forward, we want to start with a dialog about potential dangers, testing, and intentions of an incomplete feature rather than an autopsy later on. Don’t be scared, we always love hearing from users before there is a problem, not after!

Build Version History

Also, all .blend files will be keeping a log of every different version of XPlane2Blender that they are opened and saved with. This is automatic and needs no involvement from the user. Those who are curious can look in the Plugin Development Tools section at the bottom of the scene settings and see the history of their file.

Note: This does not record any history data about Blender versions, other addon versions used, timestamps opened or saved, or changes made to it (including XPlane2Blender settings changes). It is purely useful as a debugging tool, and is not fool proof.
build_history

This .blend file started as a legacy 3.4.0 pre-beta.5 file, and was then with a copy of 3.4.0-beta.5, with no build number. Then it was used with 3.3.12, then finally, used with a build of 3.4.0-beta.5 created on 09/18/2017 at 01:27:30am.

One could use this information to guess what transformations the data could possibly have gone through along each step of its journey.

Posted in Development, Plugins by | 1 Comment

Plugin Compatibility and New XPLM 3.0 Features

Tyler posted about some of the new features coming to the X-Plane SDK version 3.0 API (available with 11.10 once we find the last bug and move it somewhere else). Here’s how compatibility works for the windowing system.

First, since the 2.0 SDK, XPLMCreateWindowEx has taken all of its arguments in a structure, a XPLMCreateWindow_t. That structure, in turn has a structSize variable that is meant to be initialized like this:

XPLMCreateWindow_t my_win = { 0 };
my_win.structSize = sizeof(my_win);
my_win.froblinator = xplmFull_Of_Eels; // etc.
XPLMCreateWindowEx(&my_win);

Now when we revise the SDK, the size of the XPLMCreateWindow_t structure grows when you #define XPLM300. This results in new plugins compiled with the new SDK sending in a large struct (with more callbacks) and older plugins compiled against the old SDK sending in a small struct.

This is how the SDK “knows” what SDK you are using and provides new behavior to new plugins but provides compatible behavior to existing plugins, compiled years ago.

(This technique isn’t in any way new or unique to the X-Plane SDK – you can tell by the coding conventions that it is cough, cough, borrowed from the Win32 SDK.)

Posted in Development, Plugins by | 8 Comments

Two “Gotchas” Developing Plugins

These both come from third party developers – one I was aware of and one was completely new to me.

The Debug Heap Is Slow

Veteran Windows C++ developers know this, but if you don’t spend a lot of time with MSVC, this might be new.

The default configuration of MSVC is to force use of the debug heap when the debugger is attached; this applies even to release builds of binaries that aren’t yours.

What this means is: if you do nothing else but start your plugin “inside” X-Plane, X-Plane will use Microsoft’s debug heap.

We avoid doing this internally because it is really, really, really slow. The debug heap provides some great tracking for finding memory problems, but the cost is really, really slow load time.

The fix is simple: add _NO_DEBUG_HEAP=1 to the environment variables for your debugger. You can do this in the control panel as shown in the linked article, or you can add it to MSVC’s debugger properties.

Other Things To Go Fast

As a side tangent, when debugging a plugin, find ways to load X-Plane faster. A few things that can help:

  • Turn off AI aircraft.
  • Pick a simple aircraft to fly (e.g. not the 747).
  • Go somewhere without scenery (or without much scenery, e.g. PMDY).
  • If you have to have scenery, turn 3-d all the way off.

My Plugin Won’t Reload

We receive periodic bug reports that plugins won’t reload on Windows. This is miserable for you as a third party developer – it means rebooting the sim instead of just reloading your plugin. But why does it happen?

I thought we had a bug, but a third party developer found this:

I did searched the registry for the names I’ve used for X-Plane 11 folder and found this items at
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windws NT\CurrentVersion\AppCompatFlags\Layers

Each name used was included there with “$IgnoreFreeLibrary<win.xpl>”
value, after removing them and restarting X-Plane I got able to compile the plugin using the names again.

I have not found any official Microsoft documentation on this, just a lot of ranting from really annoyed programmers who want to know why the hell Microsoft would do this. If anyone has better Google-Fu than me and can find official docs, please do share. So what follows is my wild speculation of what’s going on.

In some old version of Windows, FreeLibrary probably didn’t actually (or immediately) free a DLL under certain circumstances. Application developers then wrote buggy apps that relied on this behavior, e.g. they freed a library while it was in use and their app didn’t crash.

Microsoft updated Windows and made FreeLibrary do what it says on the label, always, and old buggy apps started crashing left and right.

Microsoft, being Microsoft, did not say “you app developers are idiots, why don’t you go rewrite your buggy apps”. Instead, they added a feature to the App Compatibility Wizard that watches your app and, if it crashes after a DLL was unloaded and the moon is the right shade of Rondele, it adds a registry key marking this library as “never to be unloaded, even if this app says so”.

From that point on, the app doesn’t crash because the old semantics of FreeLibrary (“ha ha ha, you don’t mean that do you”) are restored in that one case.

The above is entirely made up – it’s my mental model for what might have happened, unless Raymond Chen someday gives us some back story.

Still, you can see what has gone wrong here – Windows decided that for compatibility reasons, plugins should not actually be unloaded, and since all of your DLLs are named win.xpl, this one plugin crash has caused plugins to never be unloaded. Thus the plugin unload/reload never releases the DLL and you can’t swap in new code.

Nuke the registry key and you’re back in business, at least until you crash again.

 

Posted in Development, Plugins by | 9 Comments

New SDK Features Coming in X-Plane 11.10

The upcoming X-Plane 11.10 release (and before you ask, we’ll let you know as soon as we have an ETA! 🙂 ) will include Version 3.0 of the X-Plane SDK (XPLM).

NB: The code samples linked below will not work yet—in part because X-Plane 11.10 isn’t in beta yet, and in part because we haven’t updated the sample code downloads with the new XPLM300 headers. But, that doesn’t mean you can’t look at the code itself right now!

There are a handful of really important features here for plugin developers:

  • Instanced drawing (via the new XPLMInstance header). This is a really important one for improving plugin drawing performance. More info on the theory behind this in Ben’s recent post. The good news for developers still working on X-Plane 10 plugins is that we’ve created a “wrapper” to provide backward compatibility with old versions of the SDK. Using the wrapper, you don’t get the performance benefit that you’d see in X-Plane 11, but you’ll at least be able to use the same API. See the sample project here.
  • Map APIs (via the new XPLMMap header). Based on our RFC, this provides an interface for drawing text labels, PNG icons, and arbitrary OpenGL within the X-Plane 11 maps. See the sample project here.
  • Two minor features for menus (in the XPLMMenus API; see the new menu SDK sample for examples):
    • Aircraft-specific menus. Plugins that get loaded with the user’s aircraft will now have access to XPLMFindAircraftMenu(), to which you can append new menu items or submenus.
    • Menu items that show keyboard shortcuts. When you add a menu item via XPLMAppendMenuItemWithCommand(), if the user has a key bound to that command, the key will be displayed on the right-hand side of the menu, just like X-Plane’s native menus.
  • More joystick axes & buttons, to match X-Plane 11.10’s support for 20 USB devices (up from the previous cap of 10).
  • Lots and lots of new features for plugin-created UI in the XPLMDisplay API, including:
    • Support for styling windows like the built-in X-Plane 11 windows (sample project here)
    • Support for “popping out” windows into first-class OS windows (demoed in the same sample project above)
    • Support for automatic UI scaling of all drawing in your window (this comes for free in all windows created with XPLMCreateWindowEx that are compiled against the XPLM300 API)—this means users with hi-DPI/4k monitors who have set a 150% or 200% scale for the X-Plane UI will get the experience they do with built-in windows.
    • Support for windows that automatically “stick” to certain edges of the screen, via the XPLMSetWindowGravity() API (sample project here)

Just to be 100% clear, to get any of these features (with the exception of the backward-compatibility wrapper for instanced drawing, of course), you’ll need to compile against the XPLM300 API.

[Edited to add:] Using the XPLM300 API is 100% optional. Old plugins will continue to function, and you could even write new plugins and compile them against the old API (I’m not sure why you would you want to…), and they’ll work in X-Plane 11.10 and beyond.

Posted in Plugins by | 27 Comments

XPlane2Blender v3.4.0-beta1 is out!

XPlane2Blender v3.4.0-beta1 is out!

The next version of XPlane2Blender is right around the corner, come test it! Highlights of this release are Optimized Animations, Increased Usability, and X-Plane 11 OBJ8 features (mainly Blend Glass mode and Normal Metalness). Read more about what has been changed on the release page and download it!

(Link to beta removed until major breaking bug has been fixed. Make backups of files before using any beta product.)

As with any beta, make backups before using a partially tested product. We don’t predict there should be anything breaking in it, but its never a bad idea to be safe.

Bugs and feed back on Github is preferred over this comment section, but most of all I want to hear your feedback. To stay focused, only comments related to XPlane2Blender, the beta, and Blender will be responded to for this section. The status of other 3D Modeling Plugins/VR/Weather Systems/etc is off topic.

Posted in Aircraft & Modeling, Development, Modeling, Plugins by | 9 Comments

In the Long Term, Plugins Shouldn’t Draw In 3-D Directly

Plugins can use OpenGL to draw directly inside X-Plane; this capability is almost fifteen years old and goes back to X-Plane 6. Plugin drawing typically falls into four categories:

  1. Drawing user interface (floating windows, HUD-like UI, map details, etc.).
  2. Drawing custom panel instruments (particularly 2-d glass displays).
  3. Drawing effects in 3-d (custom smoke or fire, or maybe diagram marks showing lift).
  4. Drawing solid stuff in 3-d (e.g. pushback trucks, parts of the airplane, etc.).

These first two use cases work pretty well; the third case works barely, and the last one is a bit of a train wreck.

Problems With Plugin Drawing

OpenGL an 3-d hardware was very different when Sandy and I created the original plugin SDK 1.0. Hardware had a fixed function pipeline built into the GPU itself, and your computer had one CPU core, so the issues I’m going to describe weren’t a problem back then. Here are some problems with plugin drawing:

  1. Increasing overhead just to draw. It used to be that we could just send a message to the plugin system at “good times” and let drawing happen. X-Plane now has to do a bunch of work to prepare OpenGL for plugin use; this synchronization work slows down X-Plane, and it’s getting worse as X-Plane’s internal design diverges from the canonical OpenGL 1.5 pipeline. (Example: we have to copy our matrices into the fixed function pipeline matrices before we call you, then copy them back if you call us with a drawing routine. Slow!)
  2. Plugins don’t have access to the fastest drawing paths. The less our drawing looks like OpenGL 1.5, the less efficient plugins are in comparison to X-Plane. This is getting worse over time too. (For example: if we want to render two VR eyes at once, we have to call your plugin twice, once for each one.)
  3. Plugins don’t have access to our lighting environment or G-Buffer formats, and thus can’t easily draw in a way that integrates with our 3-d world. This is getting worse as our rendering engine becomes more complicated.

So…this is not fantastic. The first two use cases (UI and panel) aren’t that badly affected because the two of them require only 2 or 3 call-outs to plugins, drawing in a very simple manner.  The second two (3-d use cases) are quite problematic.

These costs are going to get worse when X-Plane moves to Metal and Vulkan drives; the cost of syncing over to OpenGL will be higher, plugins won’t even be drawing on the fastest APIs, and we may have to do some expensive texture synchronization.

Fixing The Problem

My goal isn’t to completely eliminate the costs of plugin drawing; rather just to minimize the costs by providing better alternatives for common tasks.

So in the long term, my plan is to add something to the SDK that we should have had for a while: the ability to create persistent object and effect systems in the X-Plane world.

Right now if you want to draw an object, you call XPLMDrawObjects – hopefully you’re doing this from just the right callbacks, but if you get this wrong, I forgive you; the rules for how to do this correctly in the v10 and v11 renderers are insanely complex, because the plugin API was invented fifteen years ago.

In the future, you will be able to simply create an object and X-Plane will draw it at all the right times, for any rendering mode, handling all of the special cases for HDR, shadows, reflections, you name it. You’ll just need to tell us when it moves. The same idea can be applied to particle systems for effects.

The one open issue with this scheme is how animation will work. Currently animations is handled by reading datarefs – if you want to animate multiple instances from a plugin, you have to figure out (inside your dataref handler!) which instance is being drawn and return the right animation values.

Not only is this a complicated mess, but it’s also slow; it requires us to call your dataref callback many times from inside the drawing loop, slowing everything down.

What I’d like to have in the future is some kind of ‘animation block’ where:

  1. Your plugin queries the object to find out which animation values it needs.
  2. Each time you update a particular instance of that object, you provide a packed block of the values of all of those animations.
  3. We use the block directly and never have to call datarefs.

This technique isn’t just faster, it also is easier to make multi-core and will play nicely when we can instance animation in the future.

The time frame for this is “during the v11 run hopefully” – that is, a new object API is not going to be in 11.0, and I can’t guarantee when it will be. I expect to support the old way (XPLMDrawObject and drawing hooks) at least through the entire v11 run, if not longer; our expected failure mode is ‘if you do it the old way, the sim might be really slow’. The new APIs will be designed to be completely friendly with multi-monitor, VR, Vulcan, and multicore.

 

Posted in Development, Plugins by | 49 Comments

Mac Issues: SASL Crashes and Red Lines

I have two updates on Mac-specific bugs. Windows users, now is a good time to go get another cup of coffee. Update: it looks like neither of these bugs can be fixed or bypassed within X-Plane, so they could be with us for a while. So I am adding instructions on work-arounds for Mac users.

Red Red Lines

There is a bug on recent Nvidia cards (specific to OS X) where, at high rendering settings, you’ll see a red outline around the horizon. At this point I believe that this is a driver bug.

Here’s the interesting part: this bug is fixed in the Mac drivers that Nvidia provides. (Yes – NVidia provides a driver you can run on your Mac instead of the OS X shipping one.)

Now here’s the down-side, before you go install this thing. When I used this driver, I managed to repeatedly hang my entire machine while using X-Plane. This doesn’t happen using Apple’s driver.  So at this point I’d say “use at your own risk.”

(In Nvidia’s defense, the driver is targeted at after-market PCIe cards for pro users; use in a RetinaBook is “beta” and that’s what I did.)

We don’t have a work-around for the red line bug in sim – if I did, I’d have applied it. So this bug will probably be in 1040, and it will be up to you to either try the NV driver, wait for an Apple fix, or use different rendering settings.

Options for Work-Around:

  • Turn down shadow settings until the artifacts go away or
  • Install the Nvidia drivers and enable them in the menu bar control panel. (In the driver picker, pick OS X and you’ll find Mac after-market drivers.)

Crash Changing Planes or Quitting

Updated 11/20/15: this bug has been fixed – see here for the details.

If you get a crash when changing aircraft or quitting X-Plane, and the plane you were flying uses SASL, please don’t file a bug with me or send me a crash log. The crash is SASL crashing inside OpenAL. The crash is specific to El Capitan, and we don’t know what causes it.

This one is an area of on-going investigation; you can basically live with the annoyance of “crashing” out of every SASL aircraft or not use El Capitan. You already know how I feel about being an early adopter to new operating systems.

Options for Work-Around:

The problem is that the crash is when you unload an aircraft running SASL, so the bug appears to be “loading any plane crashes” – it’s really the unloading of the aircraft before that is causing a problem.  So the work-around is:

  1. Launch X-Plane.
  2. Pick Quick Flight and enable “Show this window on every startup of X-Plane”.
  3. Quit X-Plane – it may crash since you are unloading a SASL aircraft.
  4. When you re-start X-Plane, pick the default Cessna and you will no longer be “stuck”.

If you are stuck on El Capitan and you need to fly SASL aircraft and you are seeing crashes, my suggestion is to keep Quick Flight enabled at startup. If you have Yosemite and fly SASL-enhanced aircraft, I suggest not upgrading.

Posted in News, Plugins by | 21 Comments

Update: X-Plane 10.32r1 Steam Edition and Gizmo Do Get Along Now

There is a bug in 10.32r1 Steam Edition – some kind of interaction between Steam and Gizmo causes Gizmo to crash. Since Gizmo is loaded on startup, this means users of popular add-ons like Skymaxx Pro can’t fly.

We are working on this now and I am hopeful we’ll have some kind of fix tomorrow. I’ll also post more details about the bug once we have more info. The crash affects X-Plane 10.32r1, Steam edition on OS X only, as far as we know.

In the meantime: if you get a crash on start with X-Plane 10.32r1, please file a bug. Please include your Log.txt file and any crash logs that you see go by. In particular, if a plugin is having problems only on the Steam edition (but not the Global edition of 10.32r1) or if a plugin besides Gizmo crashes, I would like to see it!

UPDATE: We have determined that the crashes are caused by Steam introducing an ABI breakage of the libstdc++ runtime when we use one of their distribution tools. We are now working with an engineer at Valve software to solve this. In the meantime, the Steam distributed X-Plane has been rolled back to 10.31.

UPDATE 2015-01-21: Thanks to quick help from Valve software, we were able to re-release X-Plane 10.32r1 on Steam today which now gets along fine with plugins again.

Posted in News, Plugins by | 18 Comments

Mac Plugin Developers – You Should Be Using Native Paths!

TL;DR version: if your plugin runs on OS X, you you should be setting the capability “XPLM_USE_NATIVE_PATHS” to 1, like this:

XPLMEnableFeature("XPLM_USE_NATIVE_PATHS",1);

This sets your plugin to use Posix-style file paths on OS X.

I am going through X-Plane looking for APIs that Apple has deprecated and replacing them.  Aliases to custom scenery on other hard drives stopped working in Yosemite because we were using the Alias Manager to resolve the aliases, and the API is deprecated; in Yosemite Apple actually made it stop working.  So now I’m looking to see what other deprecation problems we might be sitting on.*

One thing I noticed in my search is that kCFURLHFSPathStyle is marked deprecated in OS X 10.9.  I don’t know when it will actually stop working, but we’re not supposed to be using it.

And here’s the thing: the only use case we have for it is incredibly silly: if your plugin doesn’t tell us that you can support Posix paths, we’ll convert to HFS paths so that you can then (in your plugin) convert back to Posix paths.  In this use case, both the XPLM and your plugin are using a deprecated API to temporarily convert a file path to a silly format, and then back again.

Why Are Unix Paths Opt-In

The original XPLM API dates back to X-Plane 6 and ran on the classic Mac OS under the HFS file system.  In this environment, all file system paths were HFS paths, e.g. Volume:directory:then:filename.

For a while, X-Plane could run under OS 9 and OS X using the Carbon APIs and CFM file formats; in this environment, the SDK continued to provide HFS file paths to all plugins at all times.

When we introduced the ability for plugins to use the underlying Posix file paths (which makes life much easier for the plugin developer, since Posix paths are what the OS really wants) we had to make it opt-in; a plugin tells us it wants this new thing by setting a new feature.  Plugins that don’t opt in are assumed to be old and are assumed to expect the old convention.

HFS Paths Are Now Obsolete

Here’s the thing: at this point Apple has changed ABIs twice and changed CPU architectures; they have also changed executable formats.  Simply put, no plugin code that runs in X-Plane 9 or 10 can possibly be using HFS file paths directly, because all running plugins are only capable of running on OS X.

But because it was possible to write a plugin that worked both ways (by opting in only when X-Plane was ready) there are still plugins that will run in HFS mode if and only if X-Plane can’t support Posix (e.g. if they are running on X-Plane 9.)

So in order to fully dispose of HFS paths, we need your plugin to start opting in to Posix paths.  Doing so is really easy – it generally involves adding the one line above and deleting your HFS code (which is fun).

When Can You Use Posix Paths

Posix paths in the XPLM are available to plugins starting with X-Plane 10.0.  If your plugin requires X-Plane 10 or the XPLM 2.1, posix paths are always available.

The XPLMEnableFeature APIs are available to plugins with the XPLM API 2.0, starting with X-Plane 9.0.  So if your plugin only runs on X-Plane 9 and 10, you can attempt to set this (because the API is always available).

* Because our minimum OS is OS X 10.6 for X-Plane 10, X-Code normally doesn’t tell us about most deprecations.  To find all of the issues, I temporarily set our minimum to OS X 10.10 just to see the warning list.

Posted in Development, Plugins by | 5 Comments