My preferred way to fix X-Plane problems is to fix the problem first, then post second.  That way there is no question that what we ship is what we announce.

But there’s a bunch of issues floating around with X-Plane 10, some of which will either take a while to fully fix, or which will be fixed in steps, with some steps happening now and some happening over a much longer time frame as the sim evolves.  So I’m going to write some of them up now and get the information out there.  A lot of users have taken the time to send in good bug reports, so it’s not fair to then sit on the plan.

A lot of these plans are just that: plans.  Plans change.  I believe that one of LR’s strengths is that we can change the plan quickly – if we come up with a plan and it turns out to be dumb we don’t have to just go through with it, we can come up with something better.

With that in mind, I had the following conversation with Austin yesterday – over a rather poor cell phone connection:

Me: Heya!  Do you want to talk about BAD ALEX?*
Austin: Wait, what?  What did Alex do???
Me: Oh…no no.  Bad ALLOCs!  A-L-L-O-C-S.  Running out of memory…

A number of you have seen this error.  When X-Plane runs out of memory one of three things usually happens:

  1. Some low level part of the sim seg faults because the request for memory failed and wasn’t properly detected.
  2. The OpenGL driver blows up because it can’t function without memory.
  3. The sim tries to allocate memory and throws an “uncaught exception: std::bad_alloc” error.

So first, let’s make one thing clear: we are running out of virtual address space, not physical RAM or VRAM.  X-Plane is a 32-bit process, and thus it can only access 2-4 GB of virtual memory (address space) regardless of what is in your computer.  The specific limits:

  • 32-bit Windows: 2 or 3 GB, depending on OS settings.
  • 64-bit windows: 4 GB
  • OS X: 3.5 GB
  • Linux: 3 GB – I think.

This address space has to contain a few big things:

  • A copy of every texture X-Plane is using.  (Most OpenGL drivers will “back up” textures in virtual memory at least some of the time if not all of the time.)
  • Every mesh that will be drawn for all scenery, planes, etc.
  • The weather system (physical and drawn).
  • The physical mesh that is collision checked.
  • The sim itself.
  • etc.

So if you increase the combination of screen size, texture res, turn on HDR (which allocates a bunch of textures to render to), crank up the autogen, the sum can exceed X-Plane’s virtual address space limit.

The Short Term: Surviving With 3 GB

In the short term there are a few things that you can do and a few things that we can do.  If you are seeing crashes or bad_alloc errors:

  • Turn down rendering settings.  The major ones are: airport detail (set to default), forests (set to something moderate if you are using autogen too), and texture res (first run with compressed textures).  Don’t use 4x SSAA when in HDR mode – use FXAA instead.
  • If you are on 32-bit Windows, consider moving to 64-bit Windows, at least in the long term.
  • If you are on 32-bit Windows with 2 GB per process, increase that limit to 3 GB as shown here.

Note that buying more RAM will not help!  We are out of virtual address space, not physical RAM chips!!

In the short term we can:

  • Improve the error reporting to make it more obvious what has happened.
  • Attempt to detect a lack of memory earlier and give warnings.
  • Tighten our memory belts – I need to go on another memory hunt.  Unfortunately we went on a pretty serious one before we shipped so there may not be any quick wins.
  • Detect 32-bit Windows that are running without the 3 GB warning and flag this for users clearly with help instructions.  If you’re running with 2 GB on 32-bit Windows it’s hard to see this in the OS itself, but getting that extra GB makes a big difference!

The Long Term: 64 Bits

The long term solution is clear – we need to migrate X-Plane to 64 bits.  This is not going to be particularly easy or fun.

  • On Mac, we have to port all of our platform specific Window/UI setup code from Carbon/C++ to ObjectiveC/Cocoa because Apple doesn’t provide our APIs in a 64-bit variant.  Thanks Steve.
  • On Windows we have to change compilers, as the one we use now is old and doesn’t build x64 apps.
  • Sandy and I will have to make a 64-bit variant of plugins.
  • 32-bit plugins won’t run in a 64-bit app.  As far as I know none of the three operating systems provides a way to bridge DLLs like this, so plugin authors will need to create 64-bit compiled versions of their plugins.

It’s not a trivial amount of work.  But it is what we need to do.  It’s crazy to have a video card with 3 GB of VRAM and an app with only 3 GB of address space – in that situation by definition an app could only fill VRAM with textures if it had no code (let alone data).  For years the cards, CPUs, buses, everything has been getting faster, bigger, more, except for virtual address space limit, which we have finally smashed into face first.

There have been a bunch of posts in which I have said “we know we need to do 64 bit” but now we’re saying “it really needs to be on the next train that leaves the station.”  We will probably start the porting work once we get the 10.0.x series of patches stabilized.

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

59 comments on “Bad Alex! Bad!!! (The Road Map For Memory)”

  4. would it be just an update if this materializes during beta phase or would it be formally launched in xplane 11. Although i wouldn’t mind buying xplane10 64bit version 😀

  6. What do you suppose porting to x64 will buy in terms of performance gains (if any)? I know there are lots of user with more than that under the hood. Either way, keep up the good work.

    1. I would assume none. Some users have pointed out that you get more registers with x64, but note that all pointer-based data structures take 2x the size. Joshua once ran X-Plane Linux in 64-bit modes on an AMD – this was years ago, and saw zero fps change.

    1. I am not providing _any_ estimation of the time frame that this will be completed. I don’t have enough information to give an answer that is useful. I am not going to take a guess. If we guess low, users will be annoyed that we were late. If we guess high, the initial guess will be disheartening (“four decades?!? why are you guys so slow?!”). So there’s nothing to be gained by guessing. No guesses.

  8. HI Ben – I know it is a lot of work, but I’m really happy to see 64-bit on the radar and soon, by the looks of it. I have a 64 bit OS, so can use the full 4gb for x-plane. Good luck.

  9. I’m really tempted to say ” THIS IS GRRRRRREAT” and suddenly transform into the frosted flakes tiger. However, I’m not quite sure I totally understand what all of this means. I’ve seen the Video Driver stop to respond and X-plane crashing. I have 24 Gb of RAM, and only a 1Gb video card. What does that mean in your explanation ? Is the fact that we are out of memory space the cause for poor fps ? (I’ve mentioned before that I get poor FPS with GPU not maxed and CPUs below 15% at all times).

    thank you for all this
    and 64 Bit is definitely the way to go.
    I am quite happy.

    Patrick B.

      1. Thanks I understood that. My question was more : “is the problem you were describing is the source of the poor performance some people are getting, even though they have pretty good systems”

        Patrick

          2. It’s too soon. I don’t have enough data. Patrick, please be patient. I have a lot of bugs and crashes an perf issues to go through – I really can’t dig into any one user’s perf issues on the blog.

  10. Wow, porting it over to Cocoa finally? I’ve been wondering when you guys would get around to it, I guess 64-bit was the answer to that. Sucks though, right in the middle of a new launch. Won’t you have to redo all the windowing environments? How will this play out for Plane Maker and Airfoil Maker?

    PS- Do you want help?

    1. Our preferred time frame would be never: we get no benefit from an API change – it’s just work.

      We will have to redo all of the Os-specific window setup code. This will have no impact on the UI of Plane-Maker as its UI is built in our cross-platform OGL code.

      Help…um, sure?? 🙂 If you like writing ObjC code who am I to say no.

  11. Ha ha :-). I prefer writing it enough that I’m not sure why you guys stick with carbon (I’m sure you have plenty of reasons…). Can I write you an email perhaps? We can talk about it a bit more.

    1. Sure, go ahead. We tend to view ObjC like all company-proprietary languages: annoyingly non-cross-platform. We use Carbon because we’ve always used it, and we haven’t particularly wanted to rewrite a ton of code just for entertainment.

  12. Just curious…since you guys have seen this coming for a while…why wasn’t it one of the core changes for XP10? I feel it would have been easier to identify this as a necessary change, then build XP10 from the ground up (roughly) as a 64-bit app. As opposed to shipping a 32 bit XP, then going back and rewriting it for 64 bit.

    Hindsight is 20/20, of course…no one would have guessed 5 years ago that you could get 16GB DDR3 for like $150, or that newegg would sell 8x8GB RAM kits.

    1. 1. We’re not dropping 32 bit. We’re not making 64 bit mandatory. We don’t want to change the tech under X-Plane that fast.
      2. There isn’t going to be a ground-up rewrite. See Joel on Software – he has a great essay on why “throw it out and rewrite it all” is usually a poor idea.
      3. X-Plane 10 already has so much in it we didn’t want to cram even _more_ into it.
      4. Hindsight is 20/20 but foresight is, well, not quite that good – we thought we’d be able to function in 32 bits.

  13. So if I absorb all this geeky goodness, and goodness it is, the key takeawayfrom a transition to the addressing capability afforded by 64 bit executable is that we’re adding to the potential complexity of the rendered frame, not enhancing frame rate itself. So if your CPU is currently your limiting factor on frame rate, it will still be your limiting factor and you won’t really gain from throwing more addresses at it. Am I close, Ben?

    I’m ready for this….one way or another!

    1. Sort of, yeah. At some point if the rendering engine is efficient and the hardware is voracious, we need to feed the beast more data than we can fit into 3 GB. I think this is why we are seeing these memory-related crashes now, 3 weeks after ship. It took users a few weeks to test the waters and find which options would bury framerate and which would not. Once people found that with nice hw they could run with max autogen _and_ max texture res, they found a combination that would max out memory. The more we fix our perf and the more hw grows, the more address space we need.

  14. One question comes to mind, Ben…. if running X-Plane on multiple systems for a cockpit configuration, will 64 bit X-Plane be able to comunicate seamlessly with 32 bit X-Plane on slave systems?

      1. Okay, I can live with that!

        [yoda] Always in motion, is the future. Yeessss![/yoda]

        But is it a fair statement that even if X-Plane 10 finds it’s way into a 64 bit version, will there still be a 32 bit version alongside?

        Thanks for all the incredibly hard work, and adding this blog, with replies, on top of all that! Sir, I’d like to buy you a cup of coffee!!! 😉

  15. Seriously? After all this development time, you’ve never thought to start using the latest coding technology from the beginning?

    You guys have barely matched the sim quality of FSX. Not to mention MS Flight.

    Am I the only one that feels X-Plane 10 as barley matching the quality of FSX? And with all the bugs and issues on a shipping product, why would I stay?

    God help you when MS Flight is released.

    1. Steve you’re entirely within your right to spend your money elsewhere if you prefer FSX or MS Flight. In fact, I would encourage you to continue to support those applications if that’s how you feel.

      I do think that the judgment that you’re passing on our programming and business practices is unfair since you of course have no inside information into the company’s inner-workings. And, you’ve obviously also missed the point that Ben was trying to make which was that until now, a 64bit version of the product was completely unnecessary. It’s only now that the amount of VRAM on video cards has increased to the level where the application’s RAM usage has started to come even close to using it 32bit addresses.

      1. While I certainly do not agree with Steve – on the contrary – I have to say the limits of X-Plane 9 beeing 32-bit was obvious for me since I own the Mid-2010 Mac Pro. I COULD use 9 with higher settings and great FPS, but the 32-bit limit is in the way. I am sure that I am not only the only one begging for X-Plane 10 beeing 64-bit for more than a year now. I’m sooo happy about this news. Please bring it sooner than later ;):)

      2. QUOTE:
        “You guys have barely matched the sim quality of FSX. Not to mention MS Flight.
        Am I the only one that feels X-Plane 10 as barley matching the quality of FSX? And with all the bugs and issues on a shipping product, why would I stay?
        God help you when MS Flight is released.”

        I completely disagree. MS Flight will be just another Microsoft Mess. they will need three service packs to bring us from 5 fps to 12 fps. And your comment was brutal. X-plane 10, when the patches will come out, will be a masterpiece with a professional “non-gaming” look that cannot be compared to Microsoft Flight. How can you even compare Microsoft flight when it’s not even in BETA Testing yet. These debates belong to the appropriate forums, not on the development blog I think.

    2. What are you smoking ? Please share with your brother because I would love for FSX to look like XPX ! Especially since I have a ton of ADD-ONS for FSX.

      I did some comparison shots of FSX/XPX in PNW. Look for it on Avsim forum under X-Plane.

  16. It sounds like creating a 64 bit version for windows should be much less work than for Apple.

    Perhaps you could release and perfect Windows 64 first before burning carbon… especially if there is a chance Apple is secretly working on a new compiler.

    1. That’s a really good question: is there some subset of a FULL 64 bit port that would make a difference.

      But one hitch: win 64 (the only windows OS subset that can go 64 bit) already provides 4 GB of address space to the process – win64 users are the _least_ likely to have this problem.

      I’ve been tuning up the logging in each beta to try to get a better picture of who sees crashes – we definitely see a fair amount of win32 users – a 64 bit port helps none of those users until they (oh boy) reinstall Windows.

      I am also getting the gut feeling that the win/lin driver OGL stacks are a little bit more efficient about address space than Apple’s, but I don’t have hard evidence on that.

  17. Alex,

    Why you guys create a Flight Sim where we when start the X-Plane the app delivers us to the last location or airport we’ve been last time? Why you don’t create a menu? It’s so simple for us to select the aircraft; the flight plan; and the airport with the apron location …
    About other issue, when I select to go to another airport but instead to start on runway I prefer to start on apron gates and the App crashes to desktop? Bugs, Crashes and so …
    If you guys think I’m gonna order it with this issues and migrate from MS Flight Sim to X-Plane … think again. I want a stable product where I can make a flight without problems about crashes, bugs and even performance. My rig is up-to-date with the last components on market and X-Plane still make errors and I can only reach 10-20 FPS??? Guys be careful: Microsoft on January 2012 is starting the MS Flight program beta. You are selling a product not stable!!! Please Alex, solve this problem. If you want info about my system I also send it on bug report page.

    2. Seriously? What kind of crap is this? And the “Guys be careful”-part?

      Do you think Laminar are making a simulator based on elementary school algebra? X-plane is by far the most advanced and realistic sim for personal computers, and with version 10, in my opinion, the most good looking too.

      Give’em a break!

      Looking forward to have my dvd’s shipped. Laminar rocks!

  18. Ehm just curious: if using non-portable APIs or writing non-portable code (I mean the GUI) sucks so bad why don’t you switch to (for example) Qt? Qt is most definitely portable.

    1. Or SDL? 🙂 So far we have had two reasons:
      1. We’ve never had to do a total rewrite before, so there wasn’t a compelling reason to do a total rewrite before. (In other words, until Apple dropped Carbon for 64-bit, there was no high OS maintenance cost.)
      2. So far I haven’t seen an x-platform kit that does some of the things X-Plane needs.

      Now if someone has a good idea, I am all ears!! But here are a few of the more esoteric things we’d like that we’d need to get from a third party kit. (And note that developing a “stronger” HAL API isn’t that bad for us – we have all of the code – I’ve discussed being able to show the HAL source with the Linux community before.)

      – Strong HID support – we’d like to be able to see a _lot_ of HID information about joysticks so we can do a better joystick interface. What we have now is lowest common denominator of three OSes. We’d like to be able to get the manufacturer/device ID and look pretty closely at the HID metadata for the whole device.

      – Fluid support of full screen and in-window OpenGL without a restart and without having to throw the GL context out (which would force a complete reload of everything, might as well restart the sim). We went to “no reload for full screen” in v9 – I am of the opinion that this is a nice interface and we shouldn’t go back. You don’t have to restart any other app to go full screen, why should you have to restart X-Plane?

      – OpenGL support _including_ multi-thread-safe worker contexts on 3 platforms. (This implies having a bit of access and control over how GL contexts get created.)

      – Access to monitor/desktop topology information – we only barely do this now and we need to do better.

      So if SDL has gotten a lot better or some framework does this, let me know!! But when I last looked, SDL didn’t provide the level of polish we already had in our own code.

      1. Ben,

        Have you looked at wxWidgets? It works on Windows, Linux and Mac. I have used it as a GUI for a Python program (wxPython is a wrapper for the C version) and it works on all three platforms. I have had a look at their 64 bit support and I think you might be interested in this page in the Wiki:
        //wiki.wxwidgets.org/Development:_wxMac
        I hope that is helpful.

        Michael

  19. Hi Ben, Chris and all LR gang.

    Thank you very much for the 3GB memory hint for Windows XP.
    I was patient because I knew that you are trying very hard but I was getting so frustrated by constant bad_alloc error.

    Now, I am not experiencing any crashes.
    My settings are not superb but I am getting 30-40FPS which is good for a smooth simulation.

    64 Bit …. yes, if that is what I have to use for better performance down the road.

    Excellent job and thank you again.

    Cheers, Andrzej

    1. Glad that worked. We’re looking to make the check at startup and direct users more actively to get the system tuned up. That 3rd GB is just low hanging fruit.

    1. I hope that’s enough. 😉

      I say that only because when I started using X-Plane my _nice_ laptop had 256 MB of RAM, 8 MB of VRAM, X-Plane needed 128 MB of RAM and we had 2 GB of address space…that 2 GB seemed infinite, just like 16 TB does now.

      (To be clear, 4 GB is for a 32-bit WoW process on Win 64…clearly when you are native 64 bit you get enough more that it’s moot.)

  21. The longer you wait to do 64-bit the more problems you will have when you do it, just get it done! You must do it sooner or later.

  22. X-Plane is 32 bits.
    Never realised this simple fact.
    IT DEFINITELY HAS TO GO X64!!!
    And in Q1 if you ask me (which you don’t)…

  23. YES!

    As one of the lucky ones having a x64 monster computer I’m really looking forward to this 🙂 By the time this is released I guess my Linux monster-pc will have VRAM in the 4GB or more. (It’s 2GB VRAM to day).

    The more power XP can take advantage of, the better. I like the fact that XP can be run on “old” computers, and now seems to be able to fully use the power of the most modern systems as well.

    And, thank you for providing a realy good flightsim for Linux 🙂

  24. If my knowledge is correct, wouldn’t a lot of the c++ code stay the same except that you have to somehow bridge cocoa for the top layer GUI and calls(ex. mouse and other calls onto the window and asking the os for information) and other platform specific code but for the most part cocoa is just used for talking directly to the OS and not really interfering with the actual flight sim logic? Because you can compile 64bit c++ under a thin layer of cocoa and it works just fine. Where I was working we would code almost everything in c++ for portability and have a thin cocoa layer for interacting with the UI.

    1. Yes. This is not a total rewrite – just a rewrite of the code that calls OS-specific Carbon APIs to build windows, process events, and set up OpenGL. There are other OS-specific APIs we call that I have not yet checked for 64-bit happiness (HID and friends, etc.) so I’m not sure what else needs a rewrite.

  25. Hi Ben, I can feel with you.
    I did a Windows/Mac cross-platform OGL abstraction before, with multi-thread contexts, and I too absolutely [[[dispise this:brain] damage:of a:language] which:is Objective:C].

    Are you on MingGW for Windows? There is a MinGW64 port out there, so no need to switch to Visual Studio.

  26. You show how to increase ram from 2 to 3 gb
    to help reduce the “bad_alloc” in win xp
    but how do you do that in win 7?

    Thank you

  27. I have 32bit Windows 7. Does that mean that mean that X-Plane has 3gb of address space? I tried the Vista tweak and got an error message.

Comments are closed.