For airplane authors, the biggest question VR brings up is: what do I have to do to my aircraft to make this work? This is the first of three posts explaining not only how manipulation works with VR, but why we made our decisions and how we got here. Bear with me – this will end with specific recommendations for what to do with your aircraft.

A Long Time Ago…

Old-timers like me will remember that when the 3-d cockpit first came out, almost all of the “brains” of it were powered by clicking on the 2-d panel, which was “mapped” to the 3-d cockpit.  X-Plane would find out where your mouse click intersected the 3-d cockpit, find the texture location on that triangle, and map that back to the 2-d panel.

This seemed pretty cool when it came out 15 years ago, but as authors started making more sophisticated 3-d cockpits, it became clear that we needed a way to specify how the mouse worked directly with the 3-d cockpit shape, because the mapping back to the 2-d wouldn’t allow the user to do basic things like dragging a throttle.

Thus in X-Plane 9 we added the first set of manipulators – tags on the 3-d mesh of a cockpit specifying what happens when the user clicks on it. The manipulator could specify a dataref to modify, a command to actuate, and some data about what kind of mouse click operation was happening.

The key words here are mouse click. The original manipulators were designed entirely in terms of a mouse on a 2-d screen; they were designed to be exactly as powerful as a 2-d panel system, which was of coarse also totally mouse-centric. The relationship between the generic instruments and the original manipulators is pretty tight.

Moving to Mechanisms

The good part of the original manipulator system was that it was very flexible – mouse-centric handlers were a low level tool around which authors could do whatever they wanted.

The down-side of this design was that mouse-centric handlers were a low level tool around which authors could do whatever we want. We examined our own default fleet of a dozen or so aircraft and found that no two aircraft’s cockpits operated the same way, and almost all of them had at least some aspect that was hard to use – a poor manipulator choice for the job.

Knobs were, in particular, quite difficult – the original 2-d panel system used click-and-hold over a knob to actuate it, but authors working in 3-d had often used drag actions to do knobs, and the drag actions would respond differently depending on camera angle. We received approximately 8,452,124 bug reports that the knobs in the 747 were hard to use specifically because of this.

So during X-Plane 10, we added some new manipulators to X-Plane, and they had a very different philosophy: the manipulator described what was happening in the simulated aircraft, and X-Plane picked a mouse behavior to make that work. The new manipulators described knobs that turned and switches that either flipped left-right or up-down.  These manipulators reacted to the scroll wheel automatically because X-Plane knows what the knob is and therefore what a reasonable scroll-wheel interaction should be. (By comparison, with the old-style manipulators, the author has to specify what a reasonable scroll-wheel action is.)

Real Interaction Things Before It Was Cool

As it turns out, mechanism-centric manipulators are a lot better for VR than mouse-centric ones. Consider two cases:

  • The 2-d axis manipulator (ATTR_manip_drag_xy) specifies what happens when the mouse moves up-down and left-right, relative to the screen. What on earth does that mean in VR? Even if we made something that tracks up-down and left-right relative to the screens of the HMD, the results would be completely weird because the “thing” in the cockpit would not move the way your hand was moving. With a mouse, a mis-match between mouse and aircraft isn’t surprising, but in VR, it’s weird to reach out and touch something and have it behave the wrong way.
  • The knob manipulator. It describes a knob that does things when turned clockwise or counter-clockwise.  This is pretty easy to deal with – you click it with your controller and turn your wrist clock-wise or counter-clockwise and it turns.  Because we know what the mechanism is (and not just what the mouse should do) we can make a good decision about how to handle this manipulator in VR.

As it turns out, we ran into this problem of not doing what to do with the mouse, and needing to know what the mechanism was before we even started looking at VR. The exact same problem (“I want to touch the 3-d cockpit as if it was a thing and have it respond in the expected way”) exists in VR and on X-Plane Mobile!

Because X-Plane mobile runs on a touch screen and you have your physical finger on a switch, there are a lot of cases where the switch has to track really well and work the right way.  If the switch goes up and down, you want to flick your finger up to turn the switch on; if it goes left-right you want to flick left and right, and anything else is astonishing.

So X-Plane mobile set us on this path toward mechanism-based manipulators, and VR further drove us in that direction, since both have the same physical, non-mouse environment where a user directly interacts with the 3-d cockpit.

Guidance For Authors

So as an author, what should you do when working on a 3-d cockpit? My answer is: use some of the manipulators, but not others, to model the way things work in 3-d.

Use these manipulators ALWAYS

These manipulators are the best way to create certain physical mechanisms – use them every time this case applies to your aircraft.

ATTR_manip_drag_axis. Use this for a physical mechanism that slides along a linear path, like the throttle handle of a Cessna.

ATTR_manip_drag_rotate. Use this for a physical mechanism that rotates around a fixed point, like the throttle handles of a 737, or a trim wheel.  (Prefer this to the old technique of putting a drag axis along the edge of the throttle handles.)

ATTR_manip_noop. Use this to make a manipulator that blocks the mechanism behind it from being touched, e.g. the safety guard plate over a switch.

ATTR_manip_command. Use this for push buttons that actuate momentarily while a button is held down, like a button to motor a starter that you push in and hold down.

ATTR_manip_command_knob, ATTR_manip_command_switch_up_down, ATTR_manip_command_switch_left_right. Use these for knobs and switches with 3 or more positions that rotate or move. Pick the right manipulator for the mechanism!

ATTR_manip_axis_knob, ATTR_manip_axis_switch_up_down, ATTR_manip_axis_switch_left_right. These provide an alternative to the above manipulators when you must use a dataref. We recommend commands over datarefs any time you have a choice; the command system provides better interoperability between custom plugins, custom aircraft, and custom hardware.

ATTR_manip_command_switch_left_right2, ATTR_manip_command_switch_up_down2, ATTR_manip_command_knob2. Use these for knobs and switches with exactly two positions. You can use these to get a simple click-to-toggle with the mouse (quicker and easier for mouse users) while getting real 3-d movement in VR.

Use These Manipulators Sometimes

These manipulators are not perfect fits with physical motions and may require tweaking for VR, but they’re the best options we have now for certain problems. Try to use something from the always list instead if you can. Do not use these manipulators if the mechanism you are simulating matches something from the list above.

ATTR_manip_drag_xy. This is the only 2-axis drag we have, and is the best choice for eye-ball vents and the yoke. The yoke is special-cased in VR and should be based off of a 2-d xy manipulator. We are looking into more powerful multi-dimensional manipulation in VR, but this work won’t be complete for 10.20.

ATTR_manip_command_axis. This manipulator runs a command once based on moving a lever down or up. You should probably use a drag axis or command up-down switch, but there may be some odd mechanisms in aircraft where this manipulator is a good fit. It should not be your first-choice go-to manipulator.

ATTR_manip_push, ATTR_manip_radio, ATTR_manip_toggle. These manipulators can serve as alternatives for push-button style controls when you absolutely have to write to a dataref and not use a command.  WARNING: Do not ever use these manipulators for things that move, e.g. don’t use these for banana switches, spinning knobs, or things like that.

Do not use these manipulators

These manipulators were for mouse-centric movement and should be replaced with knob or switch manipulators.

ATTR_manip_delta, ATTR_manip_wrap. These were designed to simulate knobs by click-and-hold – use the knob manipulators instead.

ATTR_manip_drag_axis_pix. This was designed to simulate a knob by click-and-drag – use the knob manipulators instead.

We Love Commands

As you may have noticed from this list and the ever-growing size of X-Plane’s built-in command list, we love commands – they have become our go-to mechanism for our aircraft. Commands have a few advantages over data-refs:

  1. There is a clear design in the plugin system for changing a command’s behavior. So a third party aircraft can easily “take over” the existing engine start command.  This is very, very difficult to accomplish via datarefs.
  2. Commands can be mapped directly to joystick and keyboard hardware. There is no way to bind a joystick to a dataref.*
  3. Commands can be mapped to sound to capture the act of the user “doing” the thing, regardless of whether it works. For example, if you want the click of the starter switch regardless of whether the starter motor engages (maybe the battery is dead), commands make this easy to do. Often there is no dataref for ‘the user is trying but it’s not working’.

In X-Plane 11 we’ve been moving to commands to get our planes ready for VR, the new manipulators, and FMOD sound all at once.

In the next post I’ll cover some of the issues specific to VR and the 3-d cockpit.



* Developers who write plugins that provide external interfaces to X-Plane: make sure you have a way to provide access to commands and not just datarefs!  There are things you can do only with commands and not datarefs.

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

12 comments on “Manipulators and VR Part 1: From Mouse to Mechanism

  1. Hi Ben,

    Many thanks for that informative explanation. It makes complete sense!

    My only worry now is how on earth are you going to be able to implement the mouse pointer back in for those of us who don’t have controllers? I’m sure you will be able to think of something since so far your talents have astounded me, but never-the-less it is a worry. As mentioned in a separate post on the initial VR open beta announcement, DCS has managed to implement it with their VR cockpits a treat, and since they have had VR out for quite a while now, they’ve also fine tuned how it works pretty well. Perhaps you should look at how they’ve got it to work?

    Anyway, really looking forward to seeing what you manage to come up with as I have every faith in you guys!…


    1. You won’t want to use the mouse in vr ever again after trying the touch controllers. I know, it’s hard to believe until you use them. But once you’ve tried you won’t give the mouse a second thought. The laser pointer works way better than a mouse for menus and the physical manipulation is way, way beyond anything else you’ve seen.

      1. Yes…but not if you still want to use your hardware like yoke or throttle quadrant. It will always be in the way as it’s almost impossible to place it like in a real plane.
        A laser pointer for the touch controller would be a good idea then! So you can manipulate the knobs you reach (the overhead panel, for example) and the others with the laser pointer.

        1. Right. We’re looking at allowing the laser pointer on controls. The idea is that you can directly manipulate most controls, then use the laser for controls that are either really far away (the gear handle for a 737 captain is a bit of a nightmare for my virtual passengers 🙂 or blocked by hardware (e.g. the landing light switch in the C172 if you have a CH yoke).

          We’re also looking at the mouse, for users who don’t have any touch controllers, and at using the HMD and pointing, for users who have no free hands.

          I don’t know how this will all fit together yet, but it’s been one of our main areas of active development since preview 1.

          1. “We’re also looking at the mouse, for users who don’t have any touch controllers, and at using the HMD and pointing, for users who have no free hands.”

            That certainly puts my mind at rest, thank you! 🙂


          2. i thought the object of flight sims was to be as realistic as possible
            I have never seen a real pilot using touch controllers to fly an aircraft i would love to see him or Her try to handle a chopper with tose things strapped to his hands
            btw, are you implementing these things in to you”commercial “release , i dont think it would be accredited
            ps i need the mouse so i can get past the first screen

      2. Agreed.
        It is a wonderful experience.

        Unfortunately though, I’ve got a lot of aircraft that have 2d manipulators which are now unusable. Even a laser-pointer or something in the cockpit would bring them back… it doesn’t need to be a mouse per se.

        I’d love to ditch the mouse. I’d rather it didn’t come back. I do want to use my planes though.

  2. Great news on the release of native VR for XP11. I have a PFC C2 Professional Flight Console. I therefore am able to control a lot of what I need to via the console with little other interaction in the cockpit other than the radios and GPS. I have been using Flyinside XP for sometime now (I was a day one Kickstarter contributor to the original FSX release). I know where every switch and knob is on the console by feel. I therefore will never need the touch controllers if I can manipulate the radios and GPS by mouse. So mouse manipulation would be very much welcomed so that I can start using the native VR .

    Lastly just a great big thank you to all of you at Laminar – the work you do and the way in which you do it with such enthusiasm, passion and fun is truly appreciated. The sim is just stunning now.

    cheers and a Happy New Year

  3. Hey Ben,
    The ivap plugin for x-plane is basically useless in VR… it displays in the very bottom left of the headset view about two inches from your face, and cannot be interacted with. On the desktop, the plugin is drawn incorrectly, the text overlay is not aligned with the graphical background. It’s also very annoying to switch to the desktop, or lift the hmd to interact with the plugin. I mentioned this at the IVAO forums but they said to contact Laminar. It’s unclear by looking at the plugin or sourceforge to know who is behind it.

    1. Hi,

      This isn’t surprising, but it’s way too soon to look at this kind of thing. At this point we have ZERO support for plugins in VR. We DO expect to have plugin support for VR, but we have to make VR work “at all” before we can add third party interfaces.

      In the long term, they will need to adopt the 3.0 SDK and then set a few extra flags to work in VR – that is, they’ll need to “opt in” to VR. The extra work for VR should be pretty trivial if they can run on the 3.0 SDK.

    1. The manipulators are in the OBJ docs and 11.10; support is in the GIThub code base but not in 3.4rc1; we’ll start a 3.5 beta for VR I think.

      (Ted has done the 2-way switch knobs and new rotate manipulator – detents on a linear axes is in process.

Comments are closed.