Category: VR

Faking VR for fun and profit

As a developer, you’ll (hopefully) want your products to be VR compatible. To do so, you can of course use a real headset. If the need for debugging arises, that may not be very practical.

So, here’s how to fake VR in X-Plane:

  • For starters, you’ll want to bind a key combination to the “Toggle the 3-d mouse cursor in VR” command. This can be done in Settings -> Keyboard, and will allow you to switch between using the mouse in VR, and in the normal window to exit VR mode. It can by quite hard to get out of VR mode without this shortcut.
  • Start X-Plane with the --fake_vr command line argument (see “Using the Command Line Options” if unsure how to do that).
  • In the Settings menu, the VR options should now be enabled. When checking the “Enable VR Hardware” checkbox, the left-eye view will be rendered in the X-Plane window.

There are a couple of noteworthy things to keep in mind when using this. Let me start with the good news: this also works on Mac! On to the bad news… this is very much in the category of “nasty hacks”, which means:

  • Quite a number of rendering artifacts when faking VR, but it should be good enough for testing out things like user interfaces and the like.
  • No support for VR controllers
  • …or even for actually moving the fake headset about 🙁
Posted in VR by | 27 Comments

We Are Now Just Like Star Trek

Fellow nerds^H^H^H^Htrekkies^H^H^H^H^Hnerds are aware of the even-odd phenomenon – for nine films, every other Star Trek movie was pretty bad, then the next one would be okay. VR previews are now like that, with VR2 and VR4 being fatally broken. We’re planning to just skip 6 entirely.

Anyway, I broke VR4 – my code to put a loading screen in the Oculus Rift had a side effect of killing the render resolution on the HTC Vive and WMR-based headsets. Since I only have the Rift (Chris has one of everything) I never saw the bug; since I thought my change wouldn’t affect anything outside the Rift, Chris didn’t re-try the Vive.

So, VR5 is now up – this should restore the Vive/WMR headsets to the performance of VR3.

We can confirm that ASW is not working in VR4 or VR5 – we’re looking at this now, but for VR4 and VR5 you’ll see ATW but not ASW, regardless of Oculus debug settings.

(Edit: the first version of this post confirmed no ASW in VR5. ASW is off in both VR4 and VR5 because the native rift SDK is not permitting ASW for X-Plane. A number of commenters were concerned that they would lose ASW by going from VR4 to VR5. You will not! ASW is already off in VR4; if you like VR4’s performance, you clearly don’t miss ASW.)

(Edit 2: We have a very promising lead on the ASW issue for Oculus. We have it working again here but I want to do more testing before I declare victory. We do not need any more information. Nothing you can do on your end will fix this. If you think you’re fixing it by “jiggling the handle” you’re probably just falling into placebo effect. If everything goes well we will have an update to fix this over the next day or so).

Posted in Development, News, VR by | 146 Comments

X-Plane 11.20 VR4 Is Live

****EDIT: There’s apparently an issue for people running the Vive and WMR where they’re seeing reduced resolution. We’re looking into it and will post an update as soon as possible.

****EDIT2: We’ve found the issue affecting Vive and WMR. We’re testing a fix internally and will release an update hopefully in the next 24 hours. Please do not submit any more bug reports about Vive/WMR resolution.

11.20 VR4 is Live on the servers. Aside from the usability fixes that Ben already mentioned, the major ‘feature’ in VR4 is…Oculus users will no longer need SteamVR. If you downloaded it just for X-Plane, go ahead and remove it. It will no longer be necessary.

As we said we would do from the very beginning, we investigated the relative performance of X-Plane through the native Oculus SDK versus SteamVR and what we found, through data collection, is that the overall experience for Oculus users was better by going through the Oculus SDK directly. I know many of you are thinking “Duh! I told you that a month ago ya big dummy!” and yes…yes you did. Fortunately/Unfortunately, we try not to make engineering decisions based on gut feelings and anecdotal evidence when we have a way to collect actual numbers. We wanted to tackle a majority of the usability issues affecting everyone before we looked into performance.

In the various A/B tests that we performed, we found that going to the Oculus SDK directly got us about 25% improvement in frame rate. This does not necessarily indicate that there’s anything wrong with SteamVR itself. There are several factors influencing the performance in VR. First, Oculus has their “home”, that little bachelor pad where you hang out while waiting for games to load. SteamVR has their “home” as well. When you use SteamVR, BOTH are running on your machine. Those houses are not free and X-Plane is already CPU bottlenecked so anything consuming CPU resources is going to directly affect performance. (I noticed an Autodesk updater in my task manager that was stealing 5% of my CPU consistently. That too was decreasing my performance….every bit matters!). Going directly to the Oculus SDK removes the SteamVR house from the equation.

Sure, getting 25% improvement is a huge win, but that’s NOT the biggest win. The biggest win, in my opinion, is that Asynchronous Space Warp (ASW) works MUCH better even at very low frames rates down to about 22.5fps. It appears as though the timing of the frames is critical for ASW to work properly. Being at 22.5, 30, 45, 90fps feels smooth! Being in between those frame timings seems to make ASW lose its mind creating an annoying judder; the opposite of what ASW is supposed to be doing for us. Oculus seems to be V-Syncing us to hit those intervals, allowing their algorithms to make reliable timing decisions and predictions. It’s my suspicion that SteamVR was just not hitting those intervals, causing ASW to flip out.

TLDR; Performance for Oculus will be on par with what Vive users have been seeing all along. The smoothness of the rendering seems consistent even down to 22.5fps. If you’re a Vive user, you will still, of course, need SteamVR as that IS your native SDK. If you’re a WMR user, you will still need SteamVR. I have not seen any reprojection issues with WMR like we have with Oculus. Supposedly the upcoming versions of SteamVR have some performance improvements coming for WMR users as well so we’ll be sticking with SteamVR for all headsets other than Oculus. That can always change in the future…based on data.

Posted in Development, News, Uncategorized, VR by | 106 Comments

X-Plane Desktop VR2 Preview Released!

Set your updaters to grab the latest Beta and you’ll find yourselves with VR2 Preview Release. We put a lot of time into this release in an attempt to tackle as many of the usability issues of the VR1 release as we could. (Steam users — VR2 should be available as a public beta on Steam sometime this weekend, as soon as Philipp gets it uploaded.)

Important Note: If you are trying to run VR2 on an X-Plane install that was previously running FlyInside’s plugin, you might experience a crash on startup. We suggest installing VR2 to a fresh copy of X-Plane on your hard drive. If you can’t do that, you might need to uninstall FlyInside and reset your X-Plane preferences.

EDIT: There is currently a known issue with the Thrustmaster HOTAS causing the sim to crash. We’re looking into this and once it’s fixed, we will release VR3 which will address this issue and possibly others should they arise in the next 48 hours or so. For now, if you want to use VR2, you’ll need to do so without your HOTAS.

I’ll enumerate the major headliner features/improvements and talk about them individually, but first, let me be clear what is NOT in this release. This release will NOT dramatically improve the performance or reduce judder for any VR platforms. The goal of VR2 is to address usability for all users. Read More

Posted in News, VR by | 126 Comments

Interacting With X-Plane in VR

I don’t think there’s been a subject that has attracted as much attention on the developer blog as VR – 274 comments on the announcement and another 90 on the next VR topic. We’ve also received tons of bug reports, emails, and Chris has even sat down and watched endless video commentary on Youtube about our VR preview. (Just a reminder, this is a PREVIEW, not a Beta. In a Beta, we expect the features to be mostly complete, though perhaps with some bugs. In this preview, we’re still actively working on the features!)

A lot of the discussion has been about how you interact with the virtual world; Chris and I have spent literally hours discussing this since VR1. In this post, I want to share our thinking and what we’re working on for VR2.

We can view VR interaction in terms of four use cases: fully virtual, virtual with touch and hardware, no touch controllers, and HOTAS. Let’s look at them in detail.

Fully Virtual

The case that works best in VR1 is a “fully virtual” work-flow – take two touch controllers and sit in your room and you can operate the entire aircraft. There are some rough edges that we are looking at.

  • Lots of users complained that the yoke movement in VR is always a joystick motion (rotation controls pitch and roll), even if the aircraft has a yoke you pull toward you to climb. Chris’s always-a-joystick choice was based on ergonomics: since it’s fully based on angle, you can move your hand to rest on something while flying without changing your pitch. This reduces the fatigue that would come with having to hover your hand over the exact position if you were using the yoke in a realistic way. With that said,  we’re looking at providing both a “realistic” mode (where motion tracks the virtual yoke, matching our other manipulators) and an “ergonomic” mode, where the control is always a joystick and you can fly from any virtual hand position.
  • There’s no way to input rudder right now if you don’t own physical pedals. Tyler is working on making the touch controllers configurable as joystick devices. With this, you can map rudder onto your joystick or D-pad, and also use some of those buttons for common functions. (This capability is useful for two-controller use cases; users with one controller and physical hardware will probably want to keep most of the VR controller buttons mapped to VR-specific functions, as some of these cannot be moved to a conventional joystick.)

Controllers and Hardware

We heard about this case in the private beta: users with touch controllers and physical hardware have a problem when the virtual control is located inside the physical control in the real world. This happens if you are flying the Cessna and have a CH Products Yoke, for example.

To solve this, we’re adding the ability to “touch” cockpit elements from a distance via the controller lasers. It’s a little harder to use than grabbing the controls directly (because small wrist motions are amplified by distance) but it lets you get at controls that would otherwise be blocked.

Laser grabbing also turns out to be really handy for airliners – the gear handle is a stretch from the captain’s chair in the 737; now you can grab it by laser.

Laser grabbing should work seamlessly with conventional direct manipulation of controls, so for users who are happy with the direct interaction in the default fleet, there should be no loss here, and no need to revert to a mouse because some of the controls are physically blocked.

Mouse Control to Major Tom

Some users don’t have touch controllers, just an HMD, a mouse and real flight hardware (E.g. a USB yoke or something). For these users, we’re working on 3-d mouse interaction with the sim. I am not sure how capable it will be, but if you used to fly with a monitor, stick, and mouse, now you can fly with an HMD, stick, and mouse; anything you would have done with the mouse before (UI clicking, 3-d cockpit interaction) stays on the mouse.

HOTAS

Some users fly with only physical hardware (E.g. a Warthog or some other HOTAS setup). This is the group of users that Chris and I are, frankly, kind of perplexed by. If you flew X-Plane HOTAS before VR, X-Plane is, right now, just as capable in VR; anything mapped to your USB hardware still works on your USB hardware, and if you have to go pick up a touch controller, you would have had to pick up the mouse anyway.  Once we have mouse support, the mouse can be “the thing you get for UI” instead of the touch controller.

We are looking at HMD-mounted UI selection, e.g. look at the UI and click a button – Rift users may be familiar with this from the “look at this warning message for 2 seconds” notice at startup.

My personal opinion here is: having used half-written mouse code, touch controllers and HMD-aimed selection, personally I thought HMD-aimed selection was not only the worst of the three ways to interact, but it was not in the same league as mouse or controllers in terms of precision. But it does provide a way (not possible right now without VR) to stay HOTAS for the entire use of X-Plane, which could be valuable.

When

I’m not sure how soon we’ll cut VR preview 2 – I wanted to get it done this week, but I think it’s going to be some time next week; we’re working on these issues now, but it’s hard to say when it’s “done” – we might try all of our “really good ideas” and realize they need more work. That’s how user-interface is…you have to try it to know whether the idea works.

Posted in Development, VR by | 156 Comments

Manipulators and VR Part 1: From Mouse to Mechanism

For airplane authors, the biggest question VR brings up is: what do I have to do to my aircraft to make this work? This is the first of three posts explaining not only how manipulation works with VR, but why we made our decisions and how we got here. Bear with me – this will end with specific recommendations for what to do with your aircraft.

A Long Time Ago…

Old-timers like me will remember that when the 3-d cockpit first came out, almost all of the “brains” of it were powered by clicking on the 2-d panel, which was “mapped” to the 3-d cockpit.  X-Plane would find out where your mouse click intersected the 3-d cockpit, find the texture location on that triangle, and map that back to the 2-d panel.

This seemed pretty cool when it came out 15 years ago, but as authors started making more sophisticated 3-d cockpits, it became clear that we needed a way to specify how the mouse worked directly with the 3-d cockpit shape, because the mapping back to the 2-d wouldn’t allow the user to do basic things like dragging a throttle.

Thus in X-Plane 9 we added the first set of manipulators – tags on the 3-d mesh of a cockpit specifying what happens when the user clicks on it. The manipulator could specify a dataref to modify, a command to actuate, and some data about what kind of mouse click operation was happening.

The key words here are mouse click. The original manipulators were designed entirely in terms of a mouse on a 2-d screen; they were designed to be exactly as powerful as a 2-d panel system, which was of coarse also totally mouse-centric. The relationship between the generic instruments and the original manipulators is pretty tight.

Moving to Mechanisms

The good part of the original manipulator system was that it was very flexible – mouse-centric handlers were a low level tool around which authors could do whatever they wanted.

The down-side of this design was that mouse-centric handlers were a low level tool around which authors could do whatever we want. We examined our own default fleet of a dozen or so aircraft and found that no two aircraft’s cockpits operated the same way, and almost all of them had at least some aspect that was hard to use – a poor manipulator choice for the job.

Knobs were, in particular, quite difficult – the original 2-d panel system used click-and-hold over a knob to actuate it, but authors working in 3-d had often used drag actions to do knobs, and the drag actions would respond differently depending on camera angle. We received approximately 8,452,124 bug reports that the knobs in the 747 were hard to use specifically because of this.

So during X-Plane 10, we added some new manipulators to X-Plane, and they had a very different philosophy: the manipulator described what was happening in the simulated aircraft, and X-Plane picked a mouse behavior to make that work. The new manipulators described knobs that turned and switches that either flipped left-right or up-down.  These manipulators reacted to the scroll wheel automatically because X-Plane knows what the knob is and therefore what a reasonable scroll-wheel interaction should be. (By comparison, with the old-style manipulators, the author has to specify what a reasonable scroll-wheel action is.)

Real Interaction Things Before It Was Cool

As it turns out, mechanism-centric manipulators are a lot better for VR than mouse-centric ones. Consider two cases:

  • The 2-d axis manipulator (ATTR_manip_drag_xy) specifies what happens when the mouse moves up-down and left-right, relative to the screen. What on earth does that mean in VR? Even if we made something that tracks up-down and left-right relative to the screens of the HMD, the results would be completely weird because the “thing” in the cockpit would not move the way your hand was moving. With a mouse, a mis-match between mouse and aircraft isn’t surprising, but in VR, it’s weird to reach out and touch something and have it behave the wrong way.
  • The knob manipulator. It describes a knob that does things when turned clockwise or counter-clockwise.  This is pretty easy to deal with – you click it with your controller and turn your wrist clock-wise or counter-clockwise and it turns.  Because we know what the mechanism is (and not just what the mouse should do) we can make a good decision about how to handle this manipulator in VR.

As it turns out, we ran into this problem of not doing what to do with the mouse, and needing to know what the mechanism was before we even started looking at VR. The exact same problem (“I want to touch the 3-d cockpit as if it was a thing and have it respond in the expected way”) exists in VR and on X-Plane Mobile!

Because X-Plane mobile runs on a touch screen and you have your physical finger on a switch, there are a lot of cases where the switch has to track really well and work the right way.  If the switch goes up and down, you want to flick your finger up to turn the switch on; if it goes left-right you want to flick left and right, and anything else is astonishing.

So X-Plane mobile set us on this path toward mechanism-based manipulators, and VR further drove us in that direction, since both have the same physical, non-mouse environment where a user directly interacts with the 3-d cockpit.

Guidance For Authors

So as an author, what should you do when working on a 3-d cockpit? My answer is: use some of the manipulators, but not others, to model the way things work in 3-d.

Use these manipulators ALWAYS

These manipulators are the best way to create certain physical mechanisms – use them every time this case applies to your aircraft.

ATTR_manip_drag_axis. Use this for a physical mechanism that slides along a linear path, like the throttle handle of a Cessna.

ATTR_manip_drag_rotate. Use this for a physical mechanism that rotates around a fixed point, like the throttle handles of a 737, or a trim wheel.  (Prefer this to the old technique of putting a drag axis along the edge of the throttle handles.)

ATTR_manip_noop. Use this to make a manipulator that blocks the mechanism behind it from being touched, e.g. the safety guard plate over a switch.

ATTR_manip_command. Use this for push buttons that actuate momentarily while a button is held down, like a button to motor a starter that you push in and hold down.

ATTR_manip_command_knob, ATTR_manip_command_switch_up_down, ATTR_manip_command_switch_left_right. Use these for knobs and switches with 3 or more positions that rotate or move. Pick the right manipulator for the mechanism!

ATTR_manip_axis_knob, ATTR_manip_axis_switch_up_down, ATTR_manip_axis_switch_left_right. These provide an alternative to the above manipulators when you must use a dataref. We recommend commands over datarefs any time you have a choice; the command system provides better interoperability between custom plugins, custom aircraft, and custom hardware.

ATTR_manip_command_switch_left_right2, ATTR_manip_command_switch_up_down2, ATTR_manip_command_knob2. Use these for knobs and switches with exactly two positions. You can use these to get a simple click-to-toggle with the mouse (quicker and easier for mouse users) while getting real 3-d movement in VR.

Use These Manipulators Sometimes

These manipulators are not perfect fits with physical motions and may require tweaking for VR, but they’re the best options we have now for certain problems. Try to use something from the always list instead if you can. Do not use these manipulators if the mechanism you are simulating matches something from the list above.

ATTR_manip_drag_xy. This is the only 2-axis drag we have, and is the best choice for eye-ball vents and the yoke. The yoke is special-cased in VR and should be based off of a 2-d xy manipulator. We are looking into more powerful multi-dimensional manipulation in VR, but this work won’t be complete for 10.20.

ATTR_manip_command_axis. This manipulator runs a command once based on moving a lever down or up. You should probably use a drag axis or command up-down switch, but there may be some odd mechanisms in aircraft where this manipulator is a good fit. It should not be your first-choice go-to manipulator.

ATTR_manip_push, ATTR_manip_radio, ATTR_manip_toggle. These manipulators can serve as alternatives for push-button style controls when you absolutely have to write to a dataref and not use a command.  WARNING: Do not ever use these manipulators for things that move, e.g. don’t use these for banana switches, spinning knobs, or things like that.

Do not use these manipulators

These manipulators were for mouse-centric movement and should be replaced with knob or switch manipulators.

ATTR_manip_delta, ATTR_manip_wrap. These were designed to simulate knobs by click-and-hold – use the knob manipulators instead.

ATTR_manip_drag_axis_pix. This was designed to simulate a knob by click-and-drag – use the knob manipulators instead.

We Love Commands

As you may have noticed from this list and the ever-growing size of X-Plane’s built-in command list, we love commands – they have become our go-to mechanism for our aircraft. Commands have a few advantages over data-refs:

  1. There is a clear design in the plugin system for changing a command’s behavior. So a third party aircraft can easily “take over” the existing engine start command.  This is very, very difficult to accomplish via datarefs.
  2. Commands can be mapped directly to joystick and keyboard hardware. There is no way to bind a joystick to a dataref.*
  3. Commands can be mapped to sound to capture the act of the user “doing” the thing, regardless of whether it works. For example, if you want the click of the starter switch regardless of whether the starter motor engages (maybe the battery is dead), commands make this easy to do. Often there is no dataref for ‘the user is trying but it’s not working’.

In X-Plane 11 we’ve been moving to commands to get our planes ready for VR, the new manipulators, and FMOD sound all at once.

In the next post I’ll cover some of the issues specific to VR and the 3-d cockpit.

 

 

* Developers who write plugins that provide external interfaces to X-Plane: make sure you have a way to provide access to commands and not just datarefs!  There are things you can do only with commands and not datarefs.

Posted in Cockpits, Development, Plugins, VR by | 12 Comments

You Can Use VR With Any Edition of X-Plane 11

X-Plane 11’s native VR support uses SteamVR, so let me clear this up now:

You will be able to use VR directly in X-Plane 11 with any distribution of our sim:

  • An X-Plane 11 digital download from Laminar Research (or any other reseller) if you have a 24-digit product key.
  • An X-Plane 11 DVD set from Laminar Research (or any other reseller) if you have DVDs.
  • X-Plane 11 on Steam if you bought X-Plane on Steam.

VR will work for X-Plane 11 no matter how you bought it. You do not have to have bought X-Plane from Steam to use VR.

(This post does not contain any guidance on supported hardware – I’d like to wait until we get further into beta and get some feedback on how system requirements affect actual users. This is just a reassurance that you didn’t buy X-Plane “the wrong way” for VR.)

What Is SteamVR

SteamVR is a free download from Steam’s app store that X-Plane (and many other games) use to talk to your VR hardware. We use SteamVR for both the HTC Vive and the Oculus Rift – X-Plane’s native VR support requires that you have both the Oculus drivers and SteamVR.

Read More

Posted in Development, VR by | 100 Comments

X-Plane Public VR Preview Released!

After an intense year of development and a few demos of the technology at various Flight Simulator conferences, it’s finally time to let you all in on a preview of X-Plane 11 with VR support built-in! I will admit that I personally thought this was another technologic fad that was going to fade without ever gaining traction but I’m happy to say that I think I was wrong. Once you try a VR headset, you’ll never want to fly without one again. For the first time EVER, you can fly precisely and accurately by looking around and interacting with the cockpit without being anchored to a narrow field of view and small clickable hotspots viewed from unnatural angles. VR lets you have unlimited freedoms to move your head and body around naturally. You also get a sense of scale for the first time. Objects and manipulators are the right sizes and distances…just stand next to the tires or engines on one of the airliners and you’ll understand what I mean by scale. Read More

Posted in News, VR by | 279 Comments