Plugin developers: the title says it all – don’t register a drawing callback that doesn’t actually draw anything.
When I write it like that, it sounds a little bit silly, but some plugins register a drawing callback on startup, because they might draw something – but then only do the drawing when certain things happen. I am guilty of this myself – libxplanemp registers its drawing callbacks at init time even if there are no multiplayer planes to draw.
Most plugins did this for simplicity and to avoid any strange rules about when you could register things – the original 1.0 SDK was a lot pickier than what we have now. And back in 1537 when the French invaded Flanders, the cost of a no-op a draw callback was pretty cheap – just the cost of the function call-out and a few instructions of book keeping.
Over time, as X-Plane’s renderer has become more complex, this has become not the case. X-Plane now does some significant work to save and restore GL state when we hit a drawing callback, and that overhead is going to become significantly more expensive when we move to Vulkan/Metal.
To optimize this, Tyler modified the drawing callback code in 11.10 – starting with 11.10 if we find that no plugin has registered a given drawing callback, we skip the entire process, including the OpenGL book keeping.
So if your plugin goes in and registers a callback for every single drawing callback type then we have to fall back to the slow path, carefully stashing our GL state and prepping for your running at each drawing callback point.
Here’s my guidance:
Never register for a drawing callback you’re not going to use.
Do delay registering for a drawing callback until it is possible to need it.
Don’t try to optimize your drawing callbacks by registering and de-registering per frame.
So as an example, it would be reasonable for a multiplayer API to register a drawing callback when the first airplane enters the system and then keep it around until the last airplane goes away, even if that airplane is off screen.
(The reason for that last rule is: if we have to make an expensive graphics transition once we discover that drawing callbacks will be needed, we don’t want to get thrashed by drawing callbacks disappearing and then re-appearing over and over.)
I don’t think there’s been a subject that has attracted as much attention on the developer blog as VR – 274 comments on the announcement and another 90 on the next VR topic. We’ve also received tons of bug reports, emails, and Chris has even sat down and watched endless video commentary on Youtube about our VR preview. (Just a reminder, this is a PREVIEW, not a Beta. In a Beta, we expect the features to be mostly complete, though perhaps with some bugs. In this preview, we’re still actively working on the features!)
A lot of the discussion has been about how you interact with the virtual world; Chris and I have spent literally hours discussing this since VR1. In this post, I want to share our thinking and what we’re working on for VR2.
We can view VR interaction in terms of four use cases: fully virtual, virtual with touch and hardware, no touch controllers, and HOTAS. Let’s look at them in detail.
The case that works best in VR1 is a “fully virtual” work-flow – take two touch controllers and sit in your room and you can operate the entire aircraft. There are some rough edges that we are looking at.
Lots of users complained that the yoke movement in VR is always a joystick motion (rotation controls pitch and roll), even if the aircraft has a yoke you pull toward you to climb. Chris’s always-a-joystick choice was based on ergonomics: since it’s fully based on angle, you can move your hand to rest on something while flying without changing your pitch. This reduces the fatigue that would come with having to hover your hand over the exact position if you were using the yoke in a realistic way. With that said, we’re looking at providing both a “realistic” mode (where motion tracks the virtual yoke, matching our other manipulators) and an “ergonomic” mode, where the control is always a joystick and you can fly from any virtual hand position.
There’s no way to input rudder right now if you don’t own physical pedals. Tyler is working on making the touch controllers configurable as joystick devices. With this, you can map rudder onto your joystick or D-pad, and also use some of those buttons for common functions. (This capability is useful for two-controller use cases; users with one controller and physical hardware will probably want to keep most of the VR controller buttons mapped to VR-specific functions, as some of these cannot be moved to a conventional joystick.)
Controllers and Hardware
We heard about this case in the private beta: users with touch controllers and physical hardware have a problem when the virtual control is located inside the physical control in the real world. This happens if you are flying the Cessna and have a CH Products Yoke, for example.
To solve this, we’re adding the ability to “touch” cockpit elements from a distance via the controller lasers. It’s a little harder to use than grabbing the controls directly (because small wrist motions are amplified by distance) but it lets you get at controls that would otherwise be blocked.
Laser grabbing also turns out to be really handy for airliners – the gear handle is a stretch from the captain’s chair in the 737; now you can grab it by laser.
Laser grabbing should work seamlessly with conventional direct manipulation of controls, so for users who are happy with the direct interaction in the default fleet, there should be no loss here, and no need to revert to a mouse because some of the controls are physically blocked.
Mouse Control to Major Tom
Some users don’t have touch controllers, just an HMD, a mouse and real flight hardware (E.g. a USB yoke or something). For these users, we’re working on 3-d mouse interaction with the sim. I am not sure how capable it will be, but if you used to fly with a monitor, stick, and mouse, now you can fly with an HMD, stick, and mouse; anything you would have done with the mouse before (UI clicking, 3-d cockpit interaction) stays on the mouse.
Some users fly with only physical hardware (E.g. a Warthog or some other HOTAS setup). This is the group of users that Chris and I are, frankly, kind of perplexed by. If you flew X-Plane HOTAS before VR, X-Plane is, right now, just as capable in VR; anything mapped to your USB hardware still works on your USB hardware, and if you have to go pick up a touch controller, you would have had to pick up the mouse anyway. Once we have mouse support, the mouse can be “the thing you get for UI” instead of the touch controller.
We are looking at HMD-mounted UI selection, e.g. look at the UI and click a button – Rift users may be familiar with this from the “look at this warning message for 2 seconds” notice at startup.
My personal opinion here is: having used half-written mouse code, touch controllers and HMD-aimed selection, personally I thought HMD-aimed selection was not only the worst of the three ways to interact, but it was not in the same league as mouse or controllers in terms of precision. But it does provide a way (not possible right now without VR) to stay HOTAS for the entire use of X-Plane, which could be valuable.
I’m not sure how soon we’ll cut VR preview 2 – I wanted to get it done this week, but I think it’s going to be some time next week; we’re working on these issues now, but it’s hard to say when it’s “done” – we might try all of our “really good ideas” and realize they need more work. That’s how user-interface is…you have to try it to know whether the idea works.
For airplane authors, the biggest question VR brings up is: what do I have to do to my aircraft to make this work? This is the first of three posts explaining not only how manipulation works with VR, but why we made our decisions and how we got here. Bear with me – this will end with specific recommendations for what to do with your aircraft.
A Long Time Ago…
Old-timers like me will remember that when the 3-d cockpit first came out, almost all of the “brains” of it were powered by clicking on the 2-d panel, which was “mapped” to the 3-d cockpit. X-Plane would find out where your mouse click intersected the 3-d cockpit, find the texture location on that triangle, and map that back to the 2-d panel.
This seemed pretty cool when it came out 15 years ago, but as authors started making more sophisticated 3-d cockpits, it became clear that we needed a way to specify how the mouse worked directly with the 3-d cockpit shape, because the mapping back to the 2-d wouldn’t allow the user to do basic things like dragging a throttle.
Thus in X-Plane 9 we added the first set of manipulators – tags on the 3-d mesh of a cockpit specifying what happens when the user clicks on it. The manipulator could specify a dataref to modify, a command to actuate, and some data about what kind of mouse click operation was happening.
The key words here are mouse click. The original manipulators were designed entirely in terms of a mouse on a 2-d screen; they were designed to be exactly as powerful as a 2-d panel system, which was of coarse also totally mouse-centric. The relationship between the generic instruments and the original manipulators is pretty tight.
Moving to Mechanisms
The good part of the original manipulator system was that it was very flexible – mouse-centric handlers were a low level tool around which authors could do whatever they wanted.
The down-side of this design was that mouse-centric handlers were a low level tool around which authors could do whatever we want. We examined our own default fleet of a dozen or so aircraft and found that no two aircraft’s cockpits operated the same way, and almost all of them had at least some aspect that was hard to use – a poor manipulator choice for the job.
Knobs were, in particular, quite difficult – the original 2-d panel system used click-and-hold over a knob to actuate it, but authors working in 3-d had often used drag actions to do knobs, and the drag actions would respond differently depending on camera angle. We received approximately 8,452,124 bug reports that the knobs in the 747 were hard to use specifically because of this.
So during X-Plane 10, we added some new manipulators to X-Plane, and they had a very different philosophy: the manipulator described what was happening in the simulated aircraft, and X-Plane picked a mouse behavior to make that work. The new manipulators described knobs that turned and switches that either flipped left-right or up-down. These manipulators reacted to the scroll wheel automatically because X-Plane knows what the knob is and therefore what a reasonable scroll-wheel interaction should be. (By comparison, with the old-style manipulators, the author has to specify what a reasonable scroll-wheel action is.)
Real Interaction Things Before It Was Cool
As it turns out, mechanism-centric manipulators are a lot better for VR than mouse-centric ones. Consider two cases:
The 2-d axis manipulator (ATTR_manip_drag_xy) specifies what happens when the mouse moves up-down and left-right, relative to the screen. What on earth does that mean in VR? Even if we made something that tracks up-down and left-right relative to the screens of the HMD, the results would be completely weird because the “thing” in the cockpit would not move the way your hand was moving. With a mouse, a mis-match between mouse and aircraft isn’t surprising, but in VR, it’s weird to reach out and touch something and have it behave the wrong way.
The knob manipulator. It describes a knob that does things when turned clockwise or counter-clockwise. This is pretty easy to deal with – you click it with your controller and turn your wrist clock-wise or counter-clockwise and it turns. Because we know what the mechanism is (and not just what the mouse should do) we can make a good decision about how to handle this manipulator in VR.
As it turns out, we ran into this problem of not doing what to do with the mouse, and needing to know what the mechanism was before we even started looking at VR. The exact same problem (“I want to touch the 3-d cockpit as if it was a thing and have it respond in the expected way”) exists in VR and on X-Plane Mobile!
Because X-Plane mobile runs on a touch screen and you have your physical finger on a switch, there are a lot of cases where the switch has to track really well and work the right way. If the switch goes up and down, you want to flick your finger up to turn the switch on; if it goes left-right you want to flick left and right, and anything else is astonishing.
So X-Plane mobile set us on this path toward mechanism-based manipulators, and VR further drove us in that direction, since both have the same physical, non-mouse environment where a user directly interacts with the 3-d cockpit.
Guidance For Authors
So as an author, what should you do when working on a 3-d cockpit? My answer is: use some of the manipulators, but not others, to model the way things work in 3-d.
Use these manipulators ALWAYS
These manipulators are the best way to create certain physical mechanisms – use them every time this case applies to your aircraft.
ATTR_manip_drag_axis. Use this for a physical mechanism that slides along a linear path, like the throttle handle of a Cessna.
ATTR_manip_drag_rotate. Use this for a physical mechanism that rotates around a fixed point, like the throttle handles of a 737, or a trim wheel. (Prefer this to the old technique of putting a drag axis along the edge of the throttle handles.)
ATTR_manip_noop. Use this to make a manipulator that blocks the mechanism behind it from being touched, e.g. the safety guard plate over a switch.
ATTR_manip_command. Use this for push buttons that actuate momentarily while a button is held down, like a button to motor a starter that you push in and hold down.
ATTR_manip_command_knob, ATTR_manip_command_switch_up_down, ATTR_manip_command_switch_left_right. Use these for knobs and switches with 3 or more positions that rotate or move. Pick the right manipulator for the mechanism!
ATTR_manip_axis_knob, ATTR_manip_axis_switch_up_down, ATTR_manip_axis_switch_left_right. These provide an alternative to the above manipulators when you must use a dataref. We recommend commands over datarefs any time you have a choice; the command system provides better interoperability between custom plugins, custom aircraft, and custom hardware.
ATTR_manip_command_switch_left_right2, ATTR_manip_command_switch_up_down2, ATTR_manip_command_knob2. Use these for knobs and switches with exactly two positions. You can use these to get a simple click-to-toggle with the mouse (quicker and easier for mouse users) while getting real 3-d movement in VR.
Use These Manipulators Sometimes
These manipulators are not perfect fits with physical motions and may require tweaking for VR, but they’re the best options we have now for certain problems. Try to use something from the always list instead if you can. Do not use these manipulators if the mechanism you are simulating matches something from the list above.
ATTR_manip_drag_xy. This is the only 2-axis drag we have, and is the best choice for eye-ball vents and the yoke. The yoke is special-cased in VR and should be based off of a 2-d xy manipulator. We are looking into more powerful multi-dimensional manipulation in VR, but this work won’t be complete for 10.20.
ATTR_manip_command_axis. This manipulator runs a command once based on moving a lever down or up. You should probably use a drag axis or command up-down switch, but there may be some odd mechanisms in aircraft where this manipulator is a good fit. It should not be your first-choice go-to manipulator.
ATTR_manip_push, ATTR_manip_radio, ATTR_manip_toggle. These manipulators can serve as alternatives for push-button style controls when you absolutely have to write to a dataref and not use a command. WARNING: Do not ever use these manipulators for things that move, e.g. don’t use these for banana switches, spinning knobs, or things like that.
Do not use these manipulators
These manipulators were for mouse-centric movement and should be replaced with knob or switch manipulators.
ATTR_manip_delta, ATTR_manip_wrap. These were designed to simulate knobs by click-and-hold – use the knob manipulators instead.
ATTR_manip_drag_axis_pix. This was designed to simulate a knob by click-and-drag – use the knob manipulators instead.
We Love Commands
As you may have noticed from this list and the ever-growing size of X-Plane’s built-in command list, we love commands – they have become our go-to mechanism for our aircraft. Commands have a few advantages over data-refs:
There is a clear design in the plugin system for changing a command’s behavior. So a third party aircraft can easily “take over” the existing engine start command. This is very, very difficult to accomplish via datarefs.
Commands can be mapped directly to joystick and keyboard hardware. There is no way to bind a joystick to a dataref.*
Commands can be mapped to sound to capture the act of the user “doing” the thing, regardless of whether it works. For example, if you want the click of the starter switch regardless of whether the starter motor engages (maybe the battery is dead), commands make this easy to do. Often there is no dataref for ‘the user is trying but it’s not working’.
In X-Plane 11 we’ve been moving to commands to get our planes ready for VR, the new manipulators, and FMOD sound all at once.
In the next post I’ll cover some of the issues specific to VR and the 3-d cockpit.
* Developers who write plugins that provide external interfaces to X-Plane: make sure you have a way to provide access to commands and not just datarefs! There are things you can do only with commands and not datarefs.
X-Plane 11’s native VR support uses SteamVR, so let me clear this up now:
You will be able to use VR directly in X-Plane 11 with any distribution of our sim:
An X-Plane 11 digital download from Laminar Research (or any other reseller) if you have a 24-digit product key.
An X-Plane 11 DVD set from Laminar Research (or any other reseller) if you have DVDs.
X-Plane 11 on Steam if you bought X-Plane on Steam.
VR will work for X-Plane 11 no matter how you bought it. You do not have to have bought X-Plane from Steam to use VR.
(This post does not contain any guidance on supported hardware – I’d like to wait until we get further into beta and get some feedback on how system requirements affect actual users. This is just a reassurance that you didn’t buy X-Plane “the wrong way” for VR.)
What Is SteamVR
SteamVR is a free download from Steam’s app store that X-Plane (and many other games) use to talk to your VR hardware. We use SteamVR for both the HTC Vive and the Oculus Rift – X-Plane’s native VR support requires that you have both the Oculus drivers and SteamVR.
X-Plane 11.11 is now final – you’ll be notified to auto-update. 11.11 is a small bug fix release that fixes a few key issues that we didn’t find out about in time to get into 11.10. Here’s the release notes.
Our VR private beta program is underway – we’ve been cutting VR private betas in parallel to 11.11 testing; I’ll post more on VR over the next few days.
X-Plane 11.10 went final Thursday night – you’ll be notified to update. Jennifer has a good write-up of the high level features here, or you can read the release notes.
11.10 was a big patch for us – new airports, new autogen, new lego bricks, the G1000, joystick profiles, London landmarks, and significant engine enhancements to both the physics and rendering engines.
We may do an 11.11 patch next week – we have a few bug fixes that didn’t make the RC. Jennifer gets a lot of bug reports over the weekend, so we’ll evaluate Monday.
VR Is Coming!
The rendering engine work in 11.10 gets the sim “VR-ready”, and Chris has been working on the actual VR features of the app in parallel to the 11.10 release.
The next major patch of X-Plane will be 11.20; it will feature native VR support for the Oculus Rift and Vive on Windows, and the beta is coming very soon.
An author asked me how to make a sound cone in FMOD for X-Plane.
For example, our Cessna features recordings of the engines from four different mic positions (front, left, right, and back) – our engine sound event contains four sub-sound events, with sound cones modulating when the sound can be heard. Think of a 90 degree cone sticking out from the engine to the front, the right, etc. Only if the listener is in the sound cone can you hear it.
Here’s the trick: X-Plane provides two special parameters that give you the bearing of the listener relative to the event. From this you can create your attenuation shapes.
In X-Plane 11.10, annunciator sounds automatically play even if you are using FMOD. This is confusing, particularly compared to 11.05 – here’s what’s going on.
We have two categories of sounds that are not handled by FMOD as of 11.10:
Radio sounds (where you have no way in FMOD to create the exact sound that’s going to happen, because it’s based on the ATC system or speech synthesis).
Cockpit warning bells and other “event” sounds that are triggered directly from the flight model code.
We are planning to make this second set of sounds FMOD-accessible in the future, but to do so we need to create a system for “events” in X-Plane that is accessible to plugins and the authoring interfaces.
My current thinking is that an “event” system would let plugins subscribe to notifications of events, wire up sound events to sim events, and create new events via the plugin system. We may even be able to eventually play particle effects in response to events. The events would have path names (similar to commands and datarefs) for flexibility.
I had hoped to get to this in 11.10 but we just didn’t have time, so this will have to wait for a future patch. When we do have sound events we’ll have a way to indicate in an FMOD .snd file that (1) you want to wire up sound to particular sim events and (2) you don’t want the built-in sounds anymore.
Here’s the current set of sounds that are played even if FMOD is in use. If you don’t want them, include silent waves in your aircraft’s “sounds” folder.
I hate finding out about these kinds of bugs late in the beta. But this one was pretty big.
X-Plane uses a “metalness” material model. Metalness is a fairly standard material model that recycles the albedo of your material to implement both dialectrics (non-metals) and metals. It works like this:
Non-metal: diffuse light is tinted by the albedo texture. Specular light is not tinted.
Metal: there is never diffuse light. Specular light is tinted by the albedo texture.
In other words, since metals don’t have a diffuse component, we recycle the albedo to save texture space.
The bug in 11.05 is that for a pure metal, the albedo was tinting ambient light but not the sun itself. A third party developer sent me a test model that showed the problem – here’s the before and after.
That weird set of colors on the top of the helicopter body is due to the white sunlight being added to the red body. The red aircraft body is lit by ambient light reflected directly off of the environment. The runway stripes are not visible because the metal is about 30% rough, diffusing the reflection.
(One of the confusing things about PBR models if you are not used to them is that the environment itself can cast both diffuse light and specular reflections off of a material, and if the material is rough, it can be hard to tell them apart. Diffuse light is always widely diffused, no matter how glossy the surface.)
Alex and I took a look at some of our legacy aircraft and found that fixing the lighting didn’t make too much of a difference for metals that are not tinted.
Since our aircraft don’t have any “tinted” metal there isn’t much change. Note that in real life heavily tinted metals aren’t common. The cases you might have are:
Approximations of metals with complex spectrum-dependent fresnel like gold.
Composite paint, e.g. like for your car, with fleks of metal and translucent dielectric polymer mixed together – that’s the case where this really helps.
As I wrote this up I realized that the spot lights are still wrong in beta 8 – they’ll be fixed for RC1 unless people really need the old model, in which case we’ll have to put some versioning in. My guess though is that anyone doing composite paint realized it was impossible in 11.05 and hasn’t shipped anything like that yet.
In these comparisons we are under an airport light viewing various materials. In the multi-color case the bottom row of materials are various “metals” with light tinting to simulate copper, iron, gold, etc.
In the second set of cases (all red) we have a 2-d grid: metal on the bottom, dialectric on top and rough on the left and glossy on the irght. Note that the reflection of the overhead light is reddish for the metals and white for the plastics in 11.10 (correct) but all white in 11.05 (incorrect).