NVidia: 4 Ben: 0

When it turns out that a bug that we thought was in an OpenGL driver is actually in X-Plane, I try to make a point of blogging it publicly; it’s really, really easy for app developers to blame bugs and weird behavior on the driver writers, who in turn aren’t in a position to respond.  The driver writers bust their nuts to develop drivers quickly for the latest hardware that are simultaneously really fast and don’t crash.  That is not an easy task, and it’s not fair for us app developers to blame them for our own bugs.

So with that in mind, what did I screw up this time?  This is a bug in the framerate test that caused me to mis-diagnose performance of hardware instancing on NVidia drivers on OS X.  Thanks to Rob-ART Morgan of Barefeats for catching this – Rob-ART uses the X-Plane framerate test as one of his standard tests of new Macs.

Here’s the TL;DR version: hardware instancing is actually a win on modern NVidia cards on OS X (GeForce 4nn and newer); I will update X-Plane to use hardware instancing on this hardware in our next patch.  What follows are the gory (and perhaps tediously boring) details.

What Is Hardware Instancing

Hardware instancing is the ability to tell the graphics card to draw a lot of copies of one  object with a single instruction.  (We are asking the GPU to draw many “instances” of one object.)  Hardware instancing lets X-Plane draw more objects with lower CPU use.  X-Plane’s rendering engine will use hardware instancing for simple scenery objects* when available; this is what makes possible the huge amounts of buildings, houses, street signs, and other 3-d detail in X-Plane 10.  X-Plane has supported hardware instancing since version 10.0.

The Bug

The bug is pretty subtle: when we run the framerate test, we do not set the world level of detail explicitly; instead it gets set by X-Plane’s code to set up default rendering settings for a new machine.  This “default code” looks at the machine’s hardware capabilities to pick settings.

The problem is: when you disable hardware instancing (via the command line, explicit code in X-Plane, or by using really old hardware) X-Plane puts your hardware into a lower “bucket” and picks lower world level of detail settings.

Thus when you disable hardware instancing, the framerate test is running on lower settings, and produces higher framerate numbers!  This makes it look like turning off instancing is actually an improvement in fps, when actually it’s just doing better at an easier test.  On my RetinaBook Pro (650M GPU) I get just over 20 fps with instancing disabled vs 17.5 fps with instancing enabled.  But the 20 fps is due to the lower world LOD setting that X-Plane picks.  If I correctly set the world LOD to “very high” and disable instancing, I get 16.75 fps.  Instancing was actually a win after all.

Was Instancing Always a Win?

No. The origin of this mess was the GeForce 8800, where instancing was being emulated by Apple’s OpenGL layer.  If instancing is going to be software emulated, we might as well not use it; our own work-around when instancing is not available is as fast as Apple’s emulation and has the option to cull off-screen objects, making it even faster.  So I wrote some code to detect a GeForce 8800-type GPU and ignore Apple’s OpenGL instancing emulation. Hence the message “Disabling instancing for DX10 NV hw – it is software emulated.”

I believe the limitations of the 8800 are shared with the subsequent 9nnn cards, the 1nn, 2nn and 3nn, ending in the 330M on OS X.

The Fermi cards and later (4nn and later) are fundamentally different and can hardware instance at full power.  At the time they first became available (as after-market cards for the Mac Pro) it appeared that there was a significant penalty for instancing with the Fermi cards as well.  Part of this was no doubt due to the framerate test bug, but part may also have been a real driver issue.  I went back and tried to re-analyze this case (and I revisited my original bug report to Apple), but X-Plane itself has also changed quite a bit since then, so it’s hard to tell how much of what I saw was a real driver problem and how much was the fps test.

Since the 480 first became available on the Mac, NVidia has made significant improvements to their OS X drivers; one thing is clear: as of OS X 10.9.5 instancing is a win and any penalty is an artifact of the fps test.

What About Yosemite?

I don’t know what the effect of instancing is on Yosemite; I wanted to re-examine this bug before I updated my laptop to Yosemite.  That will be my next step and will give me a chance to look at a lot of the weird Yosemite behavior users are reporting.

What Do I Need To Do?

You do not need to make any changes on your own machine.  If you have an NVidia Mac, you’ll get a small (e.g. < 5%) improvement in fps in the next minor patch when we re-enable instancing.

 

* In order to draw an object with hardware instancing, it needs to avoid a bunch of object features: no animation, no attributes, etc.  Basically the object has to be simple enough to send to the GPU in a single instruction.  Our artists specifically worked to make sure that most of the autogen objects were instancing-friendly.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Development, Hardware | 8 Comments

Sibling Rivalry

When I was very young, it was hard to watch my younger brother get presents on his birthday. I was jealous! Why should he get all of the attention?  I was here first!

When I was just a little bit older, I realized that my brother’s birthday was actually a pretty good day for me too. You see, my brother and I had one big pile of toys, so whatever my brother received as a gift would be available to me too; all I had to do was be patient and not snatch his toys for a few days.

The new announcement for X-Plane 10 Mobile is clearly trying to whip up a little bit of sibling rivalry: “X-Plane 10 Mobile will make our desktop users jealous.” Besides being a chance to plug X-Plane 10 Global to mobile users, it is also a reference to the inevitable emails we did get (for X-Plane 9 vs X-Plane 9 Mobile) and will get from desktop users who are jealous of the development resources we spend on the mobile product.

Here are a few notes on X-Plane desktop and mobile and the relationship of the two products.

First, X-Plane 9 Mobile funded the development of X-Plane 10 Global.  Had we not shipped X-Plane 9 Mobile, there would not be an X-Plane 10 Global, and I probably would not still be working at Laminar Research.  So even if you ignore leverage and synergies between the code and you consider mobile products to be a distraction from the true purpose of X-Plane (desktop flight simulation), you can’t ignore that mobile is part of our business, perhaps a part that should not be discarded.

Second, we have been moving to a “two fronts” strategy where we can actively develop both products at the same time, and we have hired more developers so that we can do so.  X-Plane 9 waited while X-Plane 9 Mobile was developed, and then the mobile product was more or less frozen for years while we worked on X-Plane 10 desktop.

The level of ping-pong hasn’t been as bad for X-Plane 10 mobile.  We shipped 64-bit support, deployed the airport gateway, ported to Steam and shipped a brand new GPS while developing X-Plane 10 Mobile.*  This has not been easy, but I think it indicates that we’re making progress towards “two fronts”.  Both desktop and mobile suffer if they have to sit in the penalty box for years on end while the other is developed.

Finally, mobile devices are now powerful enough that we can share code and art assets between the two code bases.  A few examples:

  • Our minimum OpenGL requirements for the mobile product is OpenGL ES 2.0; our minimum desktop OpenGL version is OpenGL 2.0.  And the capabilities required by both (render-to-texture via VBOs, vertex and fragment shaders) really are as similar as they sound.
  • I have an iPhone 6 and a 2008 8-core Mac Pro.  I ran the particle system performance test code on both and they run at almost the same speed.  Obviously the Mac Pro is 6 years old and the iPhone 6 is a top-end phone.  But there is no gap.  This means that an older but supported desktop device will have similar characteristics to the top end of mobile devices.  It’s just one big spectrum of computers now.

A lot of code was moved from desktop to mobile for X-Plane 10 mobile.  But code was also developed for X-Plane 10 mobile with the intention of moving it back to the desktop version of X-Plane.

My take-away point: if you use our desktop product, you don’t need to actually be jealous of new toys in X-Plane 10 Mobile.  Those toys are meant to be your toys too.

* One thing to note about this list: our ability to do more than one thing at once is mostly limited by who will do the work.  Different developers and artists in the company have different skill sets; our developers are not interchangeable robots.  Well, one of our developers is a robot, but I’m not going to name names.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Development, iPhone, Mobile Devices, News | 22 Comments

How To Tell Whose Plugin Crashed?

In X-Plane 10, we started listing the plugin that crashed in Log.txt when a crash happens. This post is for plugin authors and provides one or two tricks to determine more information.

How Do We Know If a Plugin Crashed?

The mechanism for determining whether X-Plane or a plugin crashed is very simple: every time X-Plane dispatches to plugin code, we make a note that we are jumping into plugin land; when we jump back we clear the note.  This way when we get a crash, if the crash is in plugin-land, rather than collect an automatic crash report (which we won’t be able to interpret) we let the OS crash handling go through.

This mechanism is not perfect.  For example, when the sim reads an animation from a plugin, it does not mark the dataref read as plugin land, so a crashing dataref handler in a plugin will appear as an X-Plane crash.

How Do We Know Which Plugin Crashed?

The log from a plugin crash will say something like “X-Plane crashed because of a plugin: XSquawkBox.”  (This is of course hypothetical; XSquawkBox would never crash. ;-)  How does X-Plane know who crashed?

The answer is: it asks the X-Plane Plugin Manager (the XPLM).  The XPLM tracks which plugin is executing at any given time; every time a plugin callback is dispatched, the XPLM notes which plugin owns the callback and marks that plugin as current.  (This is also how the XPLM knows who owns resources; the plugin that is current when a resource is allocated owns the resource).

Note that if your plugin calls into X-Plane and X-Plane crashes, you still get blamed.  For example, if you call us to draw an object (with XPLMDrawObject) but you pass us a bogus object handle, the sim will probably crash and you’ll take the wrap.  This is probably the most useful case from a debugging standpoint; if your plugin calls our API and it crashes, we’re not going to look at it without some info about what your plugin was doing.  You (the plugin author) get the crash report and can thus know in context how the call was happening.

(If these crashes were reported to Laminar Research, we’d have no idea what was happening in the plugin because we have no plugin symbols.)

Note that dataref reads are tracked by plugin, so if, in your flight loop callback, you call XPLMGetDataRef on a dataref provided by another plugin, and that plugin’s read handler crashes, they get logged, not you.

Who Is XPLANE_PLUGIN_XPLANE

X-Plane is actually registered as a plugin, because it provides datarefs.  Therefore if you can crash in callback code registered by X-Plane (that was called from a major callback from a plugin, like a flight loop callback), we’ll log that we crashed due to a plugin, and that X-Plane is the plugin!

An example to illustrate this: your plugin registers a flight loop callback.  From your flight loop callback you call XPLMGetDatavf to read an array dataref.  The array dataref is owned by X-Plane but you pass an illegal pointer to XPLMGetDatavf, such that saving the values crashes.

X-Plane (the app) will crash inside X-Plane’s dataref handler (because writing to illegal memory will cause a crash).  Because we’re in your flight loop callback, a plugin is going to get blamed, and because X-Plane’s “plugin” code (the dataref accessor functions ) were running, XPLANE_PLUGIN_XPLANE gets blamed.

There are only two bits of code that I can think of off hand where this can happen:

  1. Reading X-Plane built-in datarefs.
  2. Using the built-in 64-bit Lua allocator by sending plugin messages to X-Plane.

If you can consistently reproduce a crash blamed on XPLANE_PLUGIN_XPLANE, please let me know; it either means we have a bug in our datarefs, or your code is doing something illegal (that we should probably trap with error detection if it’s affordable). Note that we can’t catch the bogus-memory-for-array-dataref case without killing performance, so that’s going to remain a crash.

Crashing In a Worker Thread

Plugins do not have officially registered worker threads in X-Plane 10; therefore if a worker thread crashes, we don’t know if it is ours or yours, and we cannot issue approrprate blame.  Currently X-Plane assumes that all worker thread crashes are sim crashes and auto-reports them.

If you do have a plugin that uses a worker thread and you think it might be crashing, you’d have to run from the command line with –no_crash_reporter to capture native OS crash information.

We (Laminar Research) cannot contact you to notify you of worker thread crashes caught by our crash reporter.  Besides not having contact info for all plugins, the crash reports can’t trace the stack properly to determine who crashed, because we don’t have symbols.

Finding the Plugin At Fault in an Apple Crash Report

If our crash reporter catches a crash, we log the plugin at fault, and we make sure the complete Log.txt is written to disk.  Similarly, if you log something with XPMDebugString, the log is written to disk immediately.

However, sometimes a crash will happen an we won’t catch it.  Causes include a worker thread crashing, running on Yosemite (where, as of this writing, automatic crash detection is inoperative), etc.  If you are on OS X the user will get a crash report, and it can actually be pretty useful.

The crash report does have one weakness: all of the plugins are named mac.xpl.  How do you know who crashed?

It turns out each plugin’s binary gets a unique ID (UUID) which will be logged in the crash report along with its address range.  You can calculate your own plugin’s UUID with

dwarfdump --uuid mac.xpl

The output will look like this;

bash-3.2$ cd DataRefEditor/
bash-3.2$ dwarfdump --uuid mac.xpl
UUID: 0C9A8216-95CB-2F45-D8B6-51B6168AC05D (i386) mac.xpl
UUID: 5A0D5A18-992D-FD89-3CF1-6299CBC8F03C (x86_64) mac.xpl
UUID: 66A96F33-3FFC-A36B-660C-533F006B590F (ppc7400) mac.xpl

This is the UUID for DataRefEditor.  Note that DataRefEditor runs on three platforms (PowerPC, 32-bit Intel, and 64-bit Intel – you can actually run PluginAdmin in X-Plane 8, 9, and 10) and therefore it has 3 UUIDs, one for each architecture.

So if you see a mac.xpl loaded in an Apple Crash Report whose address range covers the crash and has your UUID, then you know you are the one who crashed.

(If I had known that so many crash report tools would delete the path to a crashing DLL, I would have considered a different naming convention for fat plugins.)

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Development | 3 Comments

Please Help Find Crashing Scenery

I’ve been fielding crash-related bug reports for X-Plane 10,30, and a lot of them are turning out to be caused by third party scenery packs.

Now there are two possibilities for why a third party scenery pack might crash X-Plane:

  1. There is a bug in X-Plane.  In this case, I want to fix the bug!
  2. There is something wrong with the data in the scenery pack (that is, the data doesn’t meet X-Plane’s expectations).  In this case, I want to change the result from a crash to some kind of clear error report that an author could use to fix the problem.

The Old Rules

During my work on the scenery system for X-Plane 8 and 9, my view was “garbage in, garbage out” – that is, if you feed junk data into the scenery system, the result would be unpredictable and could include a crash.  In X-Plane 8, scenery load was completely synchronous – that is, your flight just stopped while we loaded files, so my idea at the time was that validation of scenery files should be done ahead of time, perhaps by the tools that write those files.

That logic (if it was even a good idea back then) doesn’t apply now.  Loading is done entirely on other CPU cores and does not interrupt flight; we can take the CPU time to make sure incoming data is valid.  And the cost of not validating the data is clear:

  • The instability from badly formed data results in a sea of crash reports – if we have real serious sim bugs, it’s much harder to catch them in the noise.
  • Because we don’t trap bad data, the end result crash can be in the OpenGL driver (due to cascading errors), and due to the way 64-bit crash detection works, we don’t get any trace of the contributing X-Plane code path.
  • Authors who create scenery packs can’t tell if their pack is the source of crash, or how their data induced a crash.

My goal is to chase down crashes caused by third party scenery and both fix any bugs that emerge and tighten up error checking to detect the cause of crashes early on, in a way that is useful to authors.

The New Rules

Because X-Plane has not run validation over all scenery files in the past, there are probably scenery files in use now that contain data that I would consider illegal (either by the documented format, or just by my own view of how the file format was meant to work; I realize that second definition is quite useless to authors).  But for illegal data that doesn’t crash the sim, I can’t just go in and change X-Plane to reject that data in the next patch.  (“Surprise!  Your scenery packs won’t load!” isn’t cool.)

So my thinking for validation is:

  • Detected error conditions that would have caused a crash 100% of the time can be handled by any clear means possible, including quitting the sim with an error message. If the sim can handle the error and continue, that’s better, but even a controlled quit with diagnostics is an improvement over an uncontrolled crash.
  • Detected error conditions that would have crashed the sim some of the time need to be logged but can’t quit the sim or cancel loading of the entire scenery pack; while a crash is serious, users were probably still enjoying the scenery pack (and merely being annoyed at how unstable their flying experience was).  The user experience shouldn’t get worse.
  • Detected error conditions that have only minor effects (e.g. out of range values that might cause rendering artifacts) whose legal values are well-documented can be logged, perhaps with a noisy error.  The pack works, so quitting or refusing to load is not acceptable, but if the error is too quiet, the condition will never get fixed.
  • Detected error conditions that have only minor effects whose legal values were never documented need to be logged unobtrusively; it’s not fair of me to penalize scenery packs for failing to read my mind.

On those first two points: one of the issues that complicates validation of scenery packs is that some errors are not discovered until mid-flight, because scenery is processed while you fly.  This means that a large scenery pack might contain areas that are problematic to fly in; if the original author doesn’t cover every inch of scenery in a test flight, the author might not know about the problem.

The ATC system takes a more aggressive approach: it validates every single .atc file at load time even if it doesn’t need to use the data until later.  (Virtual air traffic controllers are instantiated near the aircraft as the user flies, but the data in the .atc files that will be used to set up those controllers is checked at load time.)

This approach wasn’t too expensive performance-wise: the .atc files are small and the ATC system has to read them anyway to index them for later use.  I am not sure how to best replicate this for scenery; consider Alpilotx’s HD scenery mesh – reading every single DSF in Europe at load time isn’t acceptable performance wise but without that, how can he know that his scenery doesn’t contain areas that could crash X-Plane?

Please Send Me Bogus Scenery Packs

Here’s where I need you to help: if you find X-Plane is crashing, and you find that removing a scenery pack fixes the crash, please report a bug, including the scenery pack (or where to get it) and where you had to fly/start up to induce the crash.  Even if it is the fault of the pack (and it may not be – it may be an X-Plane bug!!!!) I still want to clamp down on the failure condition.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in File Formats, Scenery, Tools | 21 Comments

X-Plane 10.31 Is Out – Steam Soon

X-plane 10.31 is now final; if you’ve run X-Plane 10.30 you probably already know that. :-)

Steam Users: the Steam Edition of 10.31 for OS X was broken, so we rolled it back to 10.30.  We’ll get 10.31 out on Steam as soon as we can fix the problem – it should be a few days at worst.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in News | 8 Comments

Diagnosing Crashes in X-Plane 10.30

Since X-Plane 10.30 shipped, we have received a lot of feedback about the stability of X-Plane 10.30 – or rather, the lack thereof.  We are getting more bug reports that the sim crashes, more tech support requests, and more discussion of crashing in the forums.

We don’t have a good way to characterize with hard data how much less stable 10.30 may be – unfortunately even the crash reports we gather don’t give us a statistically clear picture.*  But with this much user reporting, stability is our first line of inquiry. 10.31 will come out shortly (and will fix a few issues) and we’ll keep investigating problems and pushing small patches (to fix stability issues without introducing new ones) until we’re back on solid ground.  The rest of this post discusses some of what I’ve seen so far looking into stability issues.

How To Use the Automatic Crash Reporter

Since X-Plane 10.10, we’ve had the ability to send crash reports** directly to Laminar Research.  Here are a few suggestions:

  1. If you need help, contact tech support!  This is the most important thing. We do not read every single crash report!  Instead we use statistical queries to locate and identify common crashes.  If you crash and write in the comment field “please help me”, your cry for help will probably be lost.
  2. Please do report every time X-Plane crashes, even if the crash fields are blank.  The reporter can handle huge quantities of crashes, and we’re trying to gather a statistical picture of crashes.  If you get a crash every day, we want to know that it wasn’t a one-time thing.
  3. If you hit a reproducible crash (e.g. every time you follow a simple set of steps the sim crashes with a crash-auto-report dialog box), please file a bug and include the crash report (from Output/crash_reports).  Reproducible does not mean “fly the sim and it crashes”.  It means “start the sim with no prefs, open the C172, flip the avionics switch 3 times and the sim crashes” – specific steps with a known crash outcome every time.

Reproduction steps are gold to us – sometimes we’ll see a crash and the proximate cause is “hrm – the sim crashed drawing an OBJ”.  Sadly this doesn’t tell us much about how to reproduce the crash – and clearly the sim does not crash every time you draw an OBJ; it wouldn’t start up.

The Problem with OpenGL Drivers and 64-Bit

Our crash reporter can only trace the origins of a crash through X-Plane code.*** Unfortunately, of the top 5 Windows 64-bit crashes in 10.30r2 (and this represents most of the crashing going on), 87% of the crashes originated in an OpenGL driver from AMD, NVidia or Intel.  (This does not mean the driver is bad – it could mean that X-Plane sent the driver junk data.  Or it could be the driver’s fault.  We don’t know.)

Unfortunately that means that for the vast majority of 64-bit crashes, we have no idea what the sim was doing when it crashed – the trace from the driver will contain junk.  All we know is that we were, in fact, in the driver at the time of death.  (We also have the log.txt but often it just indicates the user was flying.)

This is a long term problem that we need to address as developers with our crash report technology; we have some ideas on how to fix it but they will take time to implement. Clearly we’ve hit a point where the low hanging fruit (crashes that affect a lot of users but are entirely in X-Plane code) are gone.

Coping With Add-On Crashes

We do not have good statistics (yet) for what percentage of submitted crashes were on X-Plane installations heavily augmented with add-ons, but it is not uncommon to see a user with a custom aircraft running a custom plugin, an online flight plugin, tons of custom scenery packs, and general utility plugins.

This kind of customization is great!  We wouldn’t have added the SDKs if we didn’t want people to use them.  But when it comes to isolating a crash it does make things quite a bit more difficult.

My goal for X-Plane is to not have X-Plane crash due to bad third party add-on data, no matter how badly it is formed.  To that end, when I find cases where bad data will crash the sim, I am adding code to protect, document, and shut down based on the error.  (Some errors are recoverable, some are not, but the important thing is to give the author some feedback as to what is going on.)

In the long term I think we need to add better diagnostics to our crash reporting to show what add-ons are involved in a crash. If you have 1000 scenery packs, I can’t go download them all in an attempt to reproduce a crash.

In the short term, if you are seeing instability and you have third party add-ons installed:

  • Please run without the third party add-ons – the goal here is not to have a more boring flight, and it is not a cure – it is a diagnostic tool.
  • By running with different sets of add-ons you can isolate which add-on causes a problem with a given flight.
  • If you find a problem with a plugin (whether it’s in an aircraft or the add-on itself is a plugin), you probably need to contact the plugin author – the plugin contains its own code that we don’t have source to.
  • If you find a problem with data (e.g. a scenery pack or airplane without plugins), please file a bug, including where to get the scenery pack or aircraft, and how to reproduce the bug.

Reproduction steps are gold to us!  The combination of the add-on and simple steps to reproduce let us find and fix problems, even with third party add-ons.  If you have a flight that crashes, please try to simplify it.  Try going directly to the location that you were at where you crashed, starting with the engines on, etc.

Yosemite

While we’re talking about stability, I have to mention Yosemite: on OS X the vast majority of bug reports we’re getting that involve “weird stuff” and crashes involve Yosemite.  We’re still trying to sort out the issues, but Yosemite is definitely the most disruptive OS X major version release we’ve had in a long time, and because it’s free, a lot of people downloaded it.

  • If you haven’t updated to Yosemite on you’re using X-Plane, consider waiting until things settle down.  We’ll work around problems, and Apple will issue patches.
  • If you have an older machine, consider never updating.  Perhaps this is heresy but often the maximum OS version your machine can run is limited by hardware requirements, e.g. this is the OS that just about knocks your machine over.  How well is it going to run X-Plane?  How much of the hardware resources are left for us?
  • When a new OS comes out, don’t just click “download” even if it’s free; wait a week or two and see how well it works for other people first.

A major OS upgrade is when the OS vendor can make major changes, drop old technology, etc.  On OS X it also means new OpenGL drivers (because the drivers are always packed with the OS), so it’s a potentially disruptive update.

Update: it turns out Yosemite also broke our automatic crash reporter, which would explain why none of the common Yosemite crashes show up in our top crash list. :-(  Some users have sent me the Apple crash reports that appear (since our crash reporting doesn’t run) so that we can diagnose what’s going on.

Next Steps

10.31 should be out in a day or so (including Steam); as we continue to find and fix stability problems we’ll continue to issue small patches.  We’re looking to do a 10.35-type release with small features and new airports; the current plan is that Julian and Tyler will do a public beta of the new airports (as a single separate scenery pack) so that people can try the new airports and spot problems before we get into beta.

 

* Crash reports don’t tell us how many times the sim was run overall; users with stable systems don’t send us any data, so an increasing crash report incidence would be meaningless – we don’t have data about increasing or decreasing use of the sim.

** Here we’re talking about true program crashes – the uncontrolled exit of the sim for unknown reasons.  If you have a controlled failure, e.g. you load scenery and it’s missing objects and the sim refuses to run, you don’t get a chance to auto-report that.  We also don’t view this as a stability issue – X-Plane can be very unforgiving about broken third party content, and we can have a separate argument about whether this is a necessary evil or terrible user experience (or both!), but there’s no diagnostic information to gather.

Update 2: Blog comments are now working again.

*** The problem is that for 64-bit there is no standard for back-tracing a stack without the debug symbols to decode the stack linkage; there is no standard link register.  I’m guessing that the ABI designers thought they were helping the compiler guys optimize, but in practice it makes debugging other people’s DLLs miserable.  Microsoft publishes debug symbols for every DLL they make ,which is a big help, but the driver writers do not.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Development, News | 19 Comments

Please Try X-Plane 10.31r2

Quick note: 10.31r2 is up on servers now.  Please try it!!  We think this is a keeper 10.31 – it has just a few more bug fixes.  A list of fixes is here.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in News | Comments Off

Blender 2.49 Users: Don’t Get Yosemite

If you develop 3-d models for X-Plane using Blender 2.49 on OS X, do not upgrade to OS X 10.10 (Yosemite).  Something changed in the OS that causes the python scripts to not load, which means you won’t be able to export.

(As a side note, I am of the opinion that if you use your computer for work, you should not install major OS upgrades without investigating first.  If Apple is going to break support for something intentionally, it will be in a major version.  Once you install the new OS, going back is hard.  So why jump in so fast?  These days I don’t believe that any one OS update is so much better than what was there before that it trumps keeping your productivity software working.  But then, as Chris points out, I am older than dirt.)

Update: Chris found a way to make Blender 2.49 work with Yosemite.  For reasons unclear to us, the Python 2.5 framework that comes with the OS in Yosemite is just a symlink to version 2.6 (rather than the real Python 2.5).  If you delete the symlink and put in place the old 2.5 binary from an older OS, Blender starts working again.  I’ll post instructions when we have something written up in readable form, but this is a tricky mod at best – you have to change admin-writeable files in your library.

Update #2: Hi guys, Chris here. Here’s an outline of the steps that you’ll need to take…and you’ll need to be an administrator of course since you’ll be running sudo. A basic disclaimer….you should have a full bootable and recent backup of your system anytime you mess with system files (even though this should be relatively straight forward). These instructions worked for us but you should follow them at your own risk.

  1. Go into your time machine archives or find a friend that has 10.8 or whatever version of OSX you used to be on, and obtain a copy of the Python 2.5 version framework at this path and put it somewhere accessible:
    /System/Library/Frameworks/Python.framework/Versions/2.5
  2. Open Terminal
  3. Change directories by running
    cd /System/Library/Frameworks/Python.framework/Versions/
  4. To be safe, type:
    ls -al

    to get a full directory listing. If you see 2.5 -> 2.6, you have a symlink from 2.5 to 2.6 which is what’s causing Blender to use the wrong Python version. IF YOU DON’T SEE THIS, YOU DON’T HAVE A PROBLEM!

  5. Delete the symlink by running
    sudo rm -rf 2.5
  6. Copy the old version of 2.5 into this location (pay attention to the dot at the end!):
    sudo cp -R some/path/to/the/good/2.5 .
  7. Now as good practice, let’s put the permissions the way they should be:
    sudo chown -R root 2.5
    sudo chgrp -R wheel 2.5
  8. As one of our artists discovered, if Step #7 fails for you because “wheel” is not a known group name, it might be a good idea to run Disk Utility and let it “Fix Permissions” on your drive.
  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Tools | 3 Comments

X-Plane 10.31 Release Candidate 1 – A Few Fixes

X-Plane 10.31 release candidate one is now live.  To get it, you need to run the X-Plane installer and check “Check for new betas as well as updates” – 10.31 isn’t final so it won’t auto-notify you.

Please do get 10.31 Release Candidate 1 and try it.  We’d like to go final with it shortly.

The release notes are here – that’s basically a list of the bugs fixed.  Go read them if you want to know what is fixed!  (If the release notes don’t say it is fixed, then there is no change from 10.30.)

Report bugs here.

Please do not report bugs in the comment section – a little bit of me dies inside every time you do.

It looks like we have some bug fixes that will go into a 10.32, but for 10.31 I wanted to get out smaller fixes that we could ship quickly.  So hopefully 10.31r1 will be the only 10.31 patch and then we can get fixes in for 10.32 in another week or two.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in News | 9 Comments

HDR + Instructor Screen = Yuck (But Only For a Few More Days)

Just a quick note: last night I fixed the bug in 1030 where the sky is corrupt when you use the instructor’s station and HDR together.  This is the last of the ‘quick’ fixes for 1031, so I’m hoping to have 1031 cut over the weekend.

When I first saw pics of this bug I thought it might have been an old driver, but the number of users seeing it made it clear that it was a real thing; as it turns out, it was a subtle OpenGL screw-up by me.  Frankly the amazing thing is that we’re not seeing it on every Windows driver.  (From what I can tell, the AMD drivers “helpfully” work around my error, hiding the bug.)

We’ll post release notes with the beta with a complete bug fix list.

  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • Google Buzz
  • LinkedIn
Posted in Development, News | 1 Comment