X-Plane’s log file is our attempt to capture everything we could possibly want to know about your computer in case something goes wrong. (There’s no personal information in there – we want to know things like what kind of CPU and graphics card do you have.) We do this because a fair amount of tech support comes from configuration problems; having the sim tell us the configuration saves tech support time in explaining how to gather configuration information and eliminates the risk of user error.
This matters because a fair number of bugs and tech support requests come from strange system configurations…often we don’t even know what was wrong until the user later reports fixing the problem by adjusting a piece of hardware that we didn’t know existed. Here’s one example from last week:
Hi Guys, I just wanted to let you know, I got X-Plane working! It turned out it was my sound card. I don’t actually have a sound card…I replaced my RealTek USB speakers with a small gnome that lives on top of my motherboard. The gnome yodels what he thinks I should be hearing.
Well, it turned out that last week the gnome went on a wicked bender and passed out and shorted out my PCIe bus…I think that’s what was causing X-Plane to freeze. I gave the gnome some charcoal pills and black coffee and the sim runs great!
BTW when is X-Plane 10 coming out? Do you need beta testers? 🙂 🙂 🙂
Okay, I admit, I made that up. But…that’s not far off from reality – just replace “gnome” with your favorite barely-compatible-with-DirectSound pro-level recording sound card. Anyway, our response to this kind of thing is visible in the log…e.g.:
Sound Card: gnome (B.A.C. = 0.02%)
There is one down-side to the log: some users seem to like to report every single line of the log to us, with the question “is this a problem?” We’ve been trying to make it more obvious what’s an error and what’s just information. (Hint: most of it is information…except for the lines with the word ERROR in big bold letters.)
Therefore I make this suggestion to plugin developers re: the use of the log file.
- Do identify all log output as coming from your plugin.
- Do log serious errors that will be needed in crash forensics.
- Do not log errors that are reported to the user. If the user opens a file and it’s the wrong version, and your plugin presents a dialog box, you probably don’t need to also log the error.
- Do log configuration information needed to triage a system.
- Do not log routine status from normal operation – not in a release build please!
A month ago the blog was quiet because there was a lot I couldn’t talk about; now it’s quiet because I am up to my eyeballs in X-Plane 10. I’ll try to get out a few posts I’ve been meaning to write, but it’s definitely crunch time.
I have received a number of emails from plugin developers who wanted to know if they could modify some of the sim/aircraft datarefs, or why modifying them had no effect/an unintended effect.
The short answer is this: in some cases X-Plane will pre-process and cache data that comes from the .acf file. In this case, a sim/aircraft dataref (most of these come from the .acf file) can be read, but writing it will have no effect because the sim has already had its one look at the dataref.
This is a design limitation; it was never the intention of the SDK to allow complete Plane-Maker-level editing of the aircraft on the fly in the sim.
I was discussing plugin-controlled prop discs with a third party developer. The developer wanted to know if custom prop disc control would end up inside Plane-Maker. It may end up doing so, but I don’t think this would be nearly as useful as it would seem. What follows is my explanation to him of why this is.
Let me draw an analogy: when it comes to systems modeling, using a plugin is to Plane-Maker as using Blender is to using Plane-Maker.
Users who cannot use Blender are frustrated because they cannot make something as nice as those who are building planes out of OBJs. Sometimes they ask for more features in Plane-Maker, like: more stations! This new editing mode! Make the UI better!
But…you tell me: will Plane-Maker’s UI ever be as flexible and powerful as Blender? And if it ever did get to be that good, would that have turned out to be a good use of LR’s time, when Blender is already available?
The motivation for OBJ-based airplane geometry via third party tools is that what users want to do cannot be easily generalized into a few simple cases. Every plane is different, so a truly flexible platform is needed.
The prop disc (and other systems modeling problems) are the same way. In developing the prop disc graphics, I spoke with a number of third party developers who had already tried to push prop discs as far as they could go, were using plugins, were drawing themselves, as well as 25 other crazy hacks. I also spoke to our internal art team. And what I found was: no one had any consensus on how the prop disc system should work. Everyone wanted to tune a very specific set of behaviors to their peculiar art assets.
That’s what drove me to put it into a plugin. When we need an equation or a strategy we reach the point where we need more flexibility than Plane-Maker can exhibit. A plugin can encapsulate a strategy or technique in a way that Plane-Maker radio buttons cannot.
Consider what would happen if custom prop disc parameters were built into Plane-Maker. Everyone would have to wait until Austin implemented the prop disc algorithm they wanted. How would this be bad? Let me count the ways?
- How many algorithms do you think Austin has time to code? Not more than he has fingers on his right hand. Only the five lucky third party developers who get their algorithm coded will be happy with this.
- Austin code exactly what you want? Don’t get your hopes up.
- , what you asked for wasn’t what you wanted? We can’t change the behavior now, that would break compatibility!
- oh…your email got eaten by a spam filter? Too bad…no custom prop disc for you!
- Sorry, we don’t have a release vehicle lined up for the next 3 months. You’ll have to wait.
This problem is already happening across pretty much every aspect of systems modeling: airplane have unique, quirky systems which are usually useful for exactly one plane. It is not even remotely sustainable for X-Plane to code these things one at a time with a set of check-boxes. We might as well have a pop-up menu for every airplane ever invented, and simulate every single airplane inside the sim itself. Imagine the development costs…if a single high quality MSFS add-on sells for $30-$50…
Think of the prop disc via plugins situation (and the strobe lights are the same way) as an experiment in generic instruments for systems. By transitioning to a generic abstaction for instruments we’ve let a lot more users do exactly what they want with a small, high performance piece of code. The original instrument strategy (one of everything) reached a point where we simply couldn’t meet user needs in a cost-effective manner.
Now that Wade has XSquawkBox 1.0.3 out, I’ve been thinking about CSLs – that is, the collections of simplified airplanes that XSquawkBox uses to draw the other users. The CSL system was invented back in the days of X-Plane 6, and it’s getting a bit long in the tooth. You can’t use OBJ8 files, and it doesn’t understand a lot of the modern rendering tricks that authors use with the standard tool chain.
Plane-Maker has advanced quite a bit since then too – to make the original CSL, I had to create a special one-off hacked build of Plane-Maker to export aircraft as OBJs. This capability is now built into Plane-Maker, works a lot better, and even supports animation.
X-Plane now exports a native OBJ drawing interface to plugins. Besides giving plugins access to the fully optimized native OBJ draw code, this also means that plugins can draw objects with per-pixel lighting.
One more piece of the puzzle: in France Austin announced that we were working on a new ATC engine. One goal of this new engine is to provide ATC to the AI planes, as well as your plane, so that the other aircraft interact seamlessly in one simulated environment. (In X-Plane 9, ATC only directs you, and the AI are rogue planes that try not to buzz you when you’re on the runway.)
This makes me wonder: should there be a next-gen CSL format that is shared between X-Plane and third parties, based on X-Plane doing the rendering work?
Version 2.0 of the plugin API (available in all versions of X-Plane 9) introduces three new routines that, for the first time, allow plugins to work with OBJs.
XPLMLoadObject and
XPLMUnloadObject load an OBJ file (a model mesh) into memory, and then purge it when done. That part most users understand. The routine that causes confusion is
XPLMDrawObjects. When you draw an OBJ with
XPLMDrawObjects, you are not creating anything long-lasting or persistent in the world. Your object will be visible for one frame on screen, and then will disappear unless your draw it again. Your object is not part of any of the physics calculations.
To put this in perspective: when you make a new window using the Widgets API, the window is persistent – it exists until (1) you delete it or (2) your plugin is unloaded for some reason. You don’t have to do anything per frame to “maintain” the window – you make it and it exists.
Objects are not like that. You cannot make an object “exist” in the X-Plane world – you can only draw it once per frame using the drawing callbacks. Essentially the draw-objects API is a lower level API.
Building a Layered System
Plugins operate in a layered environment, with lower level code on the bottom and your plugin at the top. The layer stack might only be 2 layers deep (XPLM on the bottom, plugin on top), or there might be several layers. Consider XSquawkBox:
- The UI is drawn using the XPWidgets API. The XPWidgets API gets its drawing from the XPUIGraphics API, and the XPUIGraphics API changes OpenGL state using the XPLMGraphics API. So we’ve built up a layered system: basic OpenGL supports drawing, drawing supports user interface, user interface supports the plugin.
- Similarly, multiplayer is done using a library that isn’t part of the basic plugin system (but is open source): libXplaneMP. So here the XPLM supports drawing airplanes, libXplaneMP uses that to create a full multiplayer API, then XSquawkBox uses it.
The alternative to a layered system would be a “monolithic” one. Under a monolithic system, the only API for airplanes would be libXplaneMP, and the only way to create user interface would be widgets. Sandy and I usually prefer the layered approach because it provides a lot more flexibility. If you like widgets, great, use them. If not, no problem – roll your own on top of XPLMDisplay.
The Plugin System Is a Foundation
When Sandy and I cannot provide all of the layers, we have a strong bias toward providing the bottom layer, for an obvious reason: if the bottom layer isn’t in the plugin system, it may be impossible for anyone else to create it. So typically if we have a choice between a high level vs. low level API, we’ll put the low level API in first.
This is precisely what is happening with object drawing – we have the low level API (“draw an object”) but not the high level one (“create an object in the scenery system”). Since we have provided the lowest level, it is possible to code persistent objects in your plugin by layering on top of our API. By comparison, had we only provided “create an object” it would be pretty close to impossible to draw an object for one frame – if you didn’t want a scenery object, the API would be inflexible and useless.
This blog post is for amateur plugin developers. By amateur I mean: some plugin developers are professional programmers by day, and are already familiar with all aspects of the software development progress. For those developers, the SDK is unsurprising and performance is simply a matter of applying standard practice: locate the worst performance problem, fix it, wash-rinse-repeat.
But we also have a dedicated set of amateur plugin developers – whether they had programming experience before as hobbyists, or learned C to take their add-ons to the next level, this group is very dedicated, but doesn’t have the years of professional experience to draw on.
If you’re in that second group, this post is for you. Explaining how to performance tune code is well beyond the scope of a blog post, but I do want to address some fundamental ideas.
I receive a number of questions about plugin performance (to which the answer is always “that’s not going to cause a performance problem”). It is understandable that programmers would be concerned about performance; X-Plane is a high performance environment, and a plugin that wrecks that will be rejected by users. But how do you go from worrying about performance to fixing it?
Measure, Measure, Measure, Measure.
If I had to go crazy and recite a sweaty and embarrassing mantra about performance tuning so that I could be humiliated on YouTube it would go: measure, measure, measure, measure.
If you want your plugin to be fast, the single most important thing to know is: you have to find performance problems by measurement, not by speculation, guessing or logic.
If you are unfamiliar with a problem domain (which means you are writing new code or a new algorithm – that is, doing something interesting), there is no way you are going to make a good guess as to where a performance problem is.
If you have a ton of experience in a domain, you still shouldn’t be guessing! After 5 years of working on X-Plane, I can make some good guesses as to where performance problems should be. But I only use those guesses to search in the likely places first! Even with good guesses, I rely on measurement and observation to make sure my guess wasn’t stupid. And even after 5 years of working on the rendering engine, my guesses are wrong more often than they are right. That’s just how performance tuning is: it’s really hard for us to guess where a performance problem might be.*
Fortunately, the most important thing to do, measuring real performance problems, is also the easiest, and requires no special tools. The number one way to check performance: remove the code in question! Simply remove your plugin and compare frame-rate. If removing the plugin does not improve fps, your plugin is not hurting fps.
It is very, very important to make frame-rate comparison measurements under equal conditions. If you measure with your plugin in the ocean and without your plugin at LOWI, the results are meaningless. Here’s a trick I use in X-Plane all the time: I set my new code to run only if the mouse is on the right half of the screen. That way I can be sitting at a fixed location, with the camera not moving, and by mousing around, I can very rapidly compare “with code”, “without code”. The camera doesn’t move, the flight model is doing the same thing – I have isolated just the routine in question. You can do the same thing in your plugin.
Understand Setup Vs. Execution
This is just a rule of thumb, and you cannot use this rule instead of measuring. But generally: libraries are organized so that “execution” code (doing stuff) is fast, while setup and cleanup code may not be. The SDK is definitely in this category. To give a few examples:
- Drawing with a texture in OpenGL is very fast. Loading up a texture is not fast.
- Reading a dataref is fast. Finding a dataref is not as fast.
- Opening a file is usually slower than reading a file.
- You can run a flight loop per frame without performance problems. But you should only register it once.
If you want to pick a general design pattern, separate setup from execution, and performance-tune them separately. You want things that happen all the time to be very fast, and you can be quite intolerant of performance problems in execution code. But if you have setup code in your execution code (e.g. you load your textures from disk during a draw callback) you are fighting the grain; the library you are using probably hasn’t tuned those setup calls to be as fast as the execution code.
Math And Logic Is Fast
Modern computers are astoundingly fast. If you are worried that doing a slightly more complex calculation will hurt frame-rate, don’t be. One of the most common questions about performance I get is: will my systems code slow down X-Plane. It probably won’t – the things you calculate in systems logic are trivial in computer-terms. (But – always measure, don’t just read my blog post!)
In order to have slow code you basically need one of two things:
- A loop. Once you start doing some math multiple times, it can add up. Adding numbers is fast. Adding numbers 4,000,000,000 times is not fast. It only takes one for-loop to make fast code slow.
- A sub-routine. The subroutine could be doing anything, including a loop. Once you start calling other people’s code, your code might get slow.
This is where the professionals have a certain edge: they know how much a set of standard computer operations “cost” in terms of performance. What really happens when you allocate a block of memory? Open a file? If you understand everything going on to make those things happen, you can have a good idea of how expensive they are.
Fortunately, you don’t need to know. You need to measure!
SDK Callbacks Are Fast (Enough)
The SDK’s XPLM library serves as a mediator between plugins and X-Plane. Fortunately, the mediation infrastructure is reasonably fast. Mediation includes things like requesting a dataref from another plugin, or firing off a draw callback. This “callback” overhead contains no loops internally, and thus it is fast enough that you won’t have performance problems doing it correctly. One draw callback that runs every frame? Not a performance problem. Read a dataref? Not a performance problem. (Read a dataref 4,000,000 times inside a for-loop…well, that can be slow, as can anything!)
However you should be aware that some plugin routines “do work”. For example, XPLMDrawObject doesn’t just do mediation (into X-Plane), it actually draws the object. Calls that do “real work” do have the potential to be slower.
Be ware of one exception: a dataref read looks to you like a request for data. But really it happens in two parts. First the SDK makes a call into the other plugin that provides the data (often but not always X-Plane itself) and then that other plugin comes up with the data. So whenI say “dataref reads are fast” what I really mean is: the part of a dataref read that the SDK takes care of is fast. If the dataref read goes into a badly written plugin, the read could be very, very slow. All of the datarefs inside X-Plane vary from fast to very fast, but if you are reading data from another plugin, all bets are off.
Of course, all bets are off anyway. Did I mention you have to measure?
* Why can’t we guess? The answer is: abstraction. Basically well structured code uses libraries, functions, etc. to hide implementation and make the computer seem easier to work with. But because many challenging problems are hidden from view (which is a good thing) it’s hard to know how much real work is being done inside the black box. Build a black box out of black boxes, then do it again a few time, and the information about how fast a function is has bee
n obscured several times over!
A quick tip to anyone using OpenGL and a plugin to make custom instruments. (The most common case of this I can imagine is customized glass displays, where it may not be practical to use 10,000 generic instruments, and the display being emulated is basically a computer display.)
OpenGL is not pixel exact, its per-primitive anti-aliasing (e.g. draw me a smooth line) is inconsistent and weird, and you probably won’t have full-screen anti-aliasing on your panel. (FSAA is out of your control, as X-Plane sets up the panel render.
For this reason, I recommend “texture anti-aliasing”. Basically in this technique you draw everything as rectangles, – for your lines, you use textures that have alpha at the edge. The linear intepollation of the texture forms the anti-aliased edge.*
For a detailed examination of the problem, I recommend this page. (arekkusu is a freaking genius and his treatment of the problem is very thorough!) I have also tried to explain how the hell OpenGL decides what it’s going to draw here. There are a few important cases where you can be pretty sure about what OpenGL will draw, and a lot of cases where you’re playing pixel roulette.
The “texture” technique also has the advantage that it leads nicely into the use of texture artwork. For example, if you need a line with an arrow-head, well, you’re already drawing a line as a textured rectangle with pixels in the center and alpha on the sides. Just ask your artist to draw an arrow-head in photoshop!
* Texture FSAA actually produces better results than FSAA (with non-AA pixels). The reason is that FSAA produces a finite number of intermediate alpha levels. The number of finite levels depends on the FSAA scheme, but it is always discrete levels, hence 16x FSAA looking better than 2x. By comparison, when we use textures, we get a smooth linear blend between our transparent and opaque pixels, giving us the smoothest possible anti-aliasing.
Normally I try to make the X-Plane scenery and modeling system as opaque as possible — I want to make sure that nobody ever actually uses the rendering features that I spend weeks and weeks developing.
But the other night I had a little bit too much to drink, got distracted, and posted these:
In all seriousness, I have been trying to find time to put more documentation up on the Wiki. For these features, you will find an explanation of how the planes work, as well as a link to the planes (with plugins) to download, and a link to the plugin source code (on the SDK site, with sample makefiles for 3 operating systems).
Plugins? Do not panic! While plugins are necessary for some of the features demonstrated here, others can be created without additional programming.
BTW, if the existing documentation uses a concept that is not explained anywhere, please email me. I sometimes leave holes in the documentation by accident.
We’ve made some progress in transitioning XSquawkBox to be under Wade’s primary management – read more here.
I just found out from a user that Chase View Delux doesn’t work with X-Plane 940 beta 4. Here’s what’s going on:
- Plugins often end up running X-Plane code – when you make a call to the X-Plane SDK, the SDK sometimes passes the call right to X-Plane.
- X-Plane 940 beta 4 has heavy error checking on all of the X-Plane code.
Put these together and…X-Plane 940 beta 4 is putting heavy checking on plugins that wasn’t there before.
Typically the kinds of problems caught by the error-checking code inside a plugin are not critical, but aren’t bad things for authors to fix. Since we will turn off heavy error checking as soon as we get the systems code fixed, plugins will start working again..there’s no need for plugin authors to recut anything.
