EDIT: when this article was originally written, the 2.1.2 plugin SDK was not available, which caused a lot of confusion. The new SDK is now posted, and the building instructions are completely updated.
Beta 11 is out and this hopefully has the final set of SDK changes for 64-bit plugins. Besides a bunch of other fixes (see the release notes), here’s the rough state of plugins:
- 32-bit plugins should just work. If you have a plugin that worked in X-Plane 10.11 but is broken in 10.20, please report a bug – even if it’s not your own plugin! I really want to hear about any of these cases.
- 64-bit plugins should just work; if they don’t, it may be due to programmer error, so please only report a 64-bit plugin problem if you wrote the plugin.
- A new SDK (2.1.2) is cut with real frameworks for the Mac; Sandy and I spent a pile of time working on this, but I need to update the wiki instructions. Docs coming in the next 48 hours I hope.
- Name and shame is gone for linking, so the logs should be clean. If your plugin crashes, you still get named and shamed.
X-Plane 10.20 beta 9 is out and (hopefully) fixes missing library paths. Alex and I spent a good chunk of yesterday trying to clean up the tools that generate the autogen library, but it is a very complicated problem, as indicated by the 4786 distinct entries in the library.
Beta 8 and 9 introduce a change in how the rendering settings work; my suspicion is that users seeing a framerate drop in the new betas are seeing the effects of the intermediate rendering settings acting differently; the minimum and maximum for the settings should produce nearly identical results.
I’m not going to try to explain how the settings used to work – the actual results were extremely convoluted due to quirks in the engine. This change is one of several we want to make to both improve how the settings work (from a usability standpoint) and improve the look of the autogen at lower settings.
- The “objects” popup controls the amount of autogen and 3-d stuff in general. Besides adding more autogen, it adds more detail to the autogen as it appears and can even add more detail to custom scenery. (This behavior isn’t new.) In particular, if you use an a complex library element like an “AGP” scene (e.g. the control tower or FAA building) the amount of detail of that mini-scene will increase with the objects setting. The goal here is to have the objects slider control the amount of 3-d.
- The “forests” popup now strictly controls the density of all vegetation in the sim. This includes both the natural forests (.for files for DSF nerds) and the vegetation that is part of the autogen (e.g. the trees that Alex puts down on the lawns of houses). Lower forest settings tend to prioritize the rural forests over autogen trees to save framerate.
- The distance to which 3-d is drawn is controlled by “world detail distance” – this is how far anything 3-d is drawn out to. Finally with this change, this affects the forests as well as the autogen and the roads – everything runs to the same distance.
A few tuning tips:
- If you are running 32-bits, turn “world detail distance” down by at least one notch; having a shorter distance for 3-d will allow the sim to save some memory.
- If you run with full autogen, moderate forests, and highest world detail distance, you may now need to turn forests up and world detail distance down to create the equivalent of what you used to see. In older betas, the forest setting modified world detail distance in some very strange ways.
Our eventual goal is to keep the autogen base footprints even when the object count is very low; the footprints of the autogen contain views of the building tops, allowing for the equivalent of ‘orthophotos only’ for users with very low rendering capabilities. This will help avoid the ‘sea of green’ effect on lower settings. (The behavior now, where whole city blocks disappear) will eventually go away, to be replaced by the “footprint” with roofs of the city block when 3-d is turned down.
These pictures illustrate the effect of world detail distance. Note that the ground images attached to the buildings are still present even as the buildings begin to disappear with lower settings. The last picture shows the autogen with most 3-d buildings stripped away to reveal the ground tiles underneath.
As we keep working on the autogen system we’ll get more and better ground textures in place, more and better ground tiles on the autogen, and yet more buildings; the result should be a reasonably seamless urban experience at a range of rendering settings.
Some cities are “over-green” in their underlying data in the DSF – this happens when the source land class is difficult to ‘fit’ into the grid system efficiently. (Typically a little bit of vegetation or water area affects zoning and ‘greenifies’ a large area.) We will have to address this problem in DSF tile recuts. (In other words, zoning errors are a failure of accuracy compared to the source data, to be recut; adding more autogen improves plausibility by providing a wider variety of “stuff”.)
I hate to put time estimates on anything. The reason: timing is subject to change and always has a ton of uncertainty. What we internally is not a schedule. A railway has a schedule; we have a plan, a roadmap, or some goals. Betas are full of uncertainty. We don’t know what bugs are in the sim – if we did, beta would be over a lot faster.
So: this is my current thinking on the timing of the rest of the 10.20 beta, as of now. It will be obsolete within five minutes. Since most of the time schedules get screwed up by things being late (if we were ahead of schedule we could just choose to not release to be on schedule 🙂 this is post may be useful for third party developers trying to plan releases.
If things go well, I believe we will be able to build a 10.20 release candidate this year. (That’s a tighter goal than it sounds – there are only two weeks left.) I think we will not finalize that release candidate this year; more likely we’ll sit on the release candidate into a little bit of January. The release candidate will hopefully be settled enough that there are not any engineering surprises for third parties if we have to recut the release candidate.
If things go badly, we won’t be RC this year – there’s always the chance that someone could have a sick kid, a power outage, new bugs could be found, you name it. Life is full of uncertainty!
I do not know what autogen will get dropped into the beta or when. If we have autogen that is almost ready for 10.20 but misses the release deadline, we’ll drop new art assets into a 10.21 or 10.22 – it’s very easy for us to do a bug fix patch with more “stuff” in it.
If I was gambling, I’d bet that there will be a 10.21 anyway – what we found with 10.10 is that a lot of users didn’t participate in the beta, and thus we got a flood of bug reports right after 10.10 went final. This can be frustrating for us (in that it means that our beta coverage isn’t as good as we’d like in terms of hardware configurations) but it’s also good in that it means that the entire community isn’t in on the beta (something we don’t want – we don’t want to expose every X-Plane user to beta bugs!).
10.20 Beta 6 is out – this build fixes the missing paths in the library that were causing the dreaded “Could not load fac/forest/beach” error. This build also puts in some important changes to 64-bit plugis, so below I will try to point out what matters to select audiences:
Users:
If you don’t make an add-on but you are still testing 10.20, there is still a kind of testing that we need (but you may not like it): is anything broken in the 32-bit edition of 10.20 that worked in 10.11?
This kind of bug (it worked in the old version but is broken now) is called a regression bug in the tech lingo, because the sim has “regressed” – it has gone backward. This is the most important kind of bug for us to find for three reasons:
- No one wants to go backward.
- It’s easier for us to find the bugs in new features than random bugs in features we didn’t think we had changed.
- If you report a regression bug against two versions of the sim (it worked in 10.11, broken in 10.20) we have tools to see only the code changes from 10.11 to 10.20, which makes finding the bug much quicker.
So: please try the 32-bit 10.20 and if it fails at something that 10.11 did, please let us know ASAP!
Aircraft Developers
This build fixes the prop disc and airspeed indicator, and finally provides more datarefs for the anti-ice system; I suggest rechecking your airplane with these systems. At this point I think I only have two airplane issues floating around on my radar:
- The new light billboards and spill can be tricky to use, particularly when lights crash through the fuselage of an airplane.
- The manifold pressure indication is FUBAR on helicopters.
Both of these bugs were present in 10.11.
In fact, we have no regression bugs on file in the airplane system between 10.11 and 10.20 32-bit. So if you know of something broken with your airplane that is newly broken in 10.20 32-bit, please report it ASAP. (You do not need to re-report bugs that were already present in 10.11 like the manifold pressure.)
Plugin Authors
This build integrates a fix for LuaJIT on 64-bit Macs; see the release notes and plugin SDK for how to use it.
This build also strips out the symbols of libpng, libcurl, and libfreetype2 on Mac and Linux. These symbols never should have been visible in X-Plane, but they have been for years.
- If your 32-bit plugin used to work on X-Plane 10.11 but stops working on 10.20 32-bit beta 6, please file a bug and we will figure out what to do.
- If your 64-bit plugin use to work on 10.20 beta 5 but stops working on beta 6, your makefile or X-Code settings have a bug; post to the x-plane-dev mailing list to get help.
There may be future changes to 64-bit like this one; this is why I recommend not releasing while we are in beta! Our goal is to ‘get 64 bit right’ now during beta while we can try and test.
Scenery
There are no open regression bugs from 10.11 against scenery; facade textures were borked and are now fixed, and the library should be restored. So if your scenery pack worked in 10.11 and fails in 10.20, please report this ASAP!
I do have one open bug (present in all of the v10 run) that line markings are losing line segments; I hope to get that fixed by the end of beta.
(We also have a report that certain incorrect ATC/apt.dat files will hard-crash the sim – I think Chris will probably be able to get some better validation in place before beta ends.)
Two new add-ons out in the last few days:
The first is AlpilotX’s New Zealand Pro scenery. This is a recut of all of NZ using higher quality global data, the global scenery algorithms run at higher settings (no need to tone them down if the whole world doesn’t have to fit on 8 DVDs) and some custom GIS algorithms Andras came up with to handle various features.
For me it’s exciting to see what the engine can do when someone let’s it a bit off of its leash; the global scenery is always a compromise between the possible and what can be done for all users over the entire planet – it’s a compromise to make a base layer.
NZP is donation-ware. Andras* is not a Laminar Research employee – he is an X-Plane user who has gone so far with his scenery work that he works directly with us on the global scenery; we are very lucky to have his help — his contributions make X-Plane so much better, both when he works on the global scenery and on his custom work.
* Albert, who does the textures, is in this same category – they are two of a number of X-Plane users who have made outsized contributions to the sim!
Second, the McPhat ATR-72 is out. I had a chance to meet these guys in Mallorca, and see some of their work-in-progress and the stuff on their laptops was really phenomenal. I always enjoy seeing people push the envelope; it gives us an idea of where we need to provide more ‘headroom’ inside X-Plane.
On the subject of betas: I don’t know when beta six will come out – I think sometime next week but I don’t know if it will be early or late in the week. We do have a potential fix for the library problems with autogen (the 10.20b5 drop with new autogen is missing some library definitions) so that should help people using custom scenery who are getting the dreaded “could not load forest/facade/beach” message. (We changed our tools for the new autogen and accidentally lost some library paths which we are putting back.)
The old library did accidentally have a lot of library paths with the word “test” in them; I really hope no one is using them in their add-ons. As a general rule, if the word “test”, “temporary”/”temp” or “hack” is in a library path don’t use it!
Posted in News
by
Ben Supnik |
I probably should have said this earlier.
Please do not release 64-bit add-ons.
If you want to beta test your add-on in 64-bits, great! If you have gotten 64 bits working, that’s totally cool!
But, please don’t ship 64-bit code. The problem is: we are still in beta and there are still open issues with 64-bit plugins that have not been resolved. If we make changes to the plugin system during beta to make 64-bits work right and your add-on is broken, you won’t be able to react to it if it is already released.
So wait – make sure to test against the RC. Like new features and scenery file formats, while the new features is in beta, it is unstable and we’re not going to try to ‘protect’ you from change. Once we go final, things will be set in stone but we are not there yet.
(In beta 3 we changed our address space layout on Mac…that should give you an idea of how new this stuff is.)
X-Plane 10.20 beta 5 is out now – beta 4 was defective (installer-kitting problem). This build drops in a lot of new autogen.
The most immediate bug report I’ve gotten is third party scenery packs that are broken because they use library paths that are no longer available. I need to investigate what’s going on here, but I believe that we will need to post an updated library file that is not missing library definitions in a future beta. It may take a few betas to get this set up.
I also have not had time to attack most of the ‘meat and potato’ bugs that have been reported, so again it may take a few betas. We’re not rushing hard on the 10.20 run; I think it’s better to give third parties time to get their feet wet with 64 bits and make sure we address all of the technical concerns that come up from third party plugins.
About the New Autogen
The new autogen included in 10.20 beta 5 includes some of the new buildings that Alex has been working on; there is more that is not yet complete or published. The autogen art development actually consists of two tasks:
- Creating buildings and fragments of buildings (both mesh and texture) that look good and have high performance. Because we want to make the autogen fast, Alex tries to reuse sets of autogen where possible.
- Taking that “lego kit” of fragments and putting them into specific autogen art assets – autogen art assets are not just individual buildings – a single art asset builds an entire city block, encoding the rules for how a subset of the building fragments are used.
The main problem from 10.11 and earlier is that we had no autogen that provided urban buildings to irregularly shaped blocks – hence we fell back to the ranch houses, which just looked silly. The new drop has more irregular autogen with higher density urban buildings.
There is another chunk of autogen Alex is working on that is not done yet: buildings that respond to vertical information. This is why the Manhattan skyline just isn’t that “tall” – the DSF contains the height information. But while we have art assets that cover the shapes of the city without ranch houses, we don’t have enough art assets that can respond to the “tall” directive. When Alex drops that in we should see more vertical development.
I do not know what the timeline is for more autogen, but the general plan is simple: Alex will keep working on more art assets and we’ll periodically drop them in as we get them, each time bringing more realism to the existing global scenery.
X-Plane 10.20 beta 3 is out. This build went out without new art assets. We have a pile of new art assets waiting to go out the door: autogen from Alex, more terrain textures from Albert, an airplane upgrade from Javier, and fixes to airplanes and aircraft from Tom.
But I held them back and kept them out of this release. Why am I such a bastard?
The answer is: two complex low level changes went into this beta, and I want to capture at least one beta’s worth of crash report data to check for destabilization. The new art assets may use more memory (especially if users crank them up to see what’s going on); if we rolled everything into one beta, we wouldn’t be able to tell art-induced crashes from code-change induced crashes. So we’ll roll things out in steps. As I have said before, the purpose of betas is not to have an awesome flying experience, it’s to find bugs!
Bear in mind that not all of the bugs reported are fixed; we cut beta 3 to get these two specific code changes fixed; more bug fixes are coming soon. Please see the release notes for what is fixed. please do not re-report bugs unless they are listed as fixed but still show up as broken on your machine.
I’m hoping to get a beta 4 out over the weekend with art assets if beta 3 goes smoothly.
So What Did You Do?
So what are the two bug fixes that are major enough to warrant their own beta?
- A change in the load order between X-Plane’s sound code and OpenAL. I realized that the old way we load sound (e.g. in 10.10) makes it virtually impossible for a globally installed plugin (one in Resources/plugins) to share OpenAL, even if it follows all of the rules set out in the OpenAL plugin tech-note. This change fixes that, apparently fixes conflicts between Gizmo and SASL, and hopefully should let plugins do the right thing. Plugin developers, see here for more info. (The original load order assumed that OpenAL would be more robust to two copies being opened; it now appears that this is not the case and OpenAL can be quite picky at times.)
- The OS X 64-bit memory layout has been modified to be compatible with LuaJIT 2.0, the Lua engine inside most Lua-based plugins (e.g. Gizmo, SASL, FlyLua, etc). This does not entirely fix the porting problems developers have been seeing with LuaJIT, but it at least makes the plugins work under light memory conditions. (These plugins would fail 100% of the time in beta 2.) Here’s the technote; I am in contact with most of the plugin developers using Lua, but if you use Lua and haven’t heard from me, email ben at x-plane dot com.
These are not the only bug fixes in beta 3, but they are the two that are weird enough that we want ‘clean’ bug data, without additional art assets to cloud our view.
So: hopefully these two changes will go smoothly and we’ll get on to beta 4 and art assets soon. The artists work their butts off to create this stuff and we want to roll it out!
Will Lua Ever Work With 64-Bit OS X?
A few developers who depend on SASL and Gizmo have asked me whether Lua will work with 64-bit X-Plane on OS X. The only answer that I can give is that I do not know, but I am optimistic that we will engineer a solution. I consider making Lua work to be a very high priority.
Lua is in almost every commercial third party airplane we’ve seen in the last few years; the market has spoken and third party developers have said “we like Lua.” So I think it’s worth doing some serious engineering to keep Lua working. Making Lua work with 64-bit OS X is currently my top priority bug.
As many have noted, QuickTime video recording is not available in the 64-bit version of X-Plane. This is not a bug; we will not be ‘porting’ QuickTime to 64 bits.
What we are going to do is:
- Maintain QuickTime capture in every place we have always had it: 32-bit Mac and 32-bit Windows.
- Someday we will transition from QuickTime to “something else” that will run on every platform, 32 and 64 bits. At that point we will drop QuickTime and have one universal capture solution. I don’t know what release this will be but it won’t be in 10.20.
My preference is for something that captures a fairly common and widespread format; photo-JPEG-compressed AVI is a candidate; remember the goal of video capture in X-Plane is not to make the best finished movies but to get the movies out with low overhead, for high-fps capture. That’s why we do QuickTime-JPEG and not MP4/H264, for example. Really good output formats like H264 require very long encode times to get their high quality high ratio compression. On the fly our goal with capture is just to get the frames out to disk for later processing.
Why No 64-Bit QuickTime?
The rest of this is just for programming nerds, feel free to let your eyes glaze over.
The problem is that QuickTime has two APIs:
- A 32-bit C API available on Mac and Windows (original “QuickTime”) – this is the original C API that dates back to Mac OS 8 or earlier.
- A 32/64-bit Objective C API (“QTKit”) available only on Mac (e.g. the only platform with ObjC).
We currently ship with that first API, hence no Linux support, and no 64-bit support – the API just isn’t available in those flavors.
I believe the work of porting to the new ObjC QTKit API would be about the same as implementing any other library-based capture solution, but it would only get us one OS in 64-bit. So when we do spend time to extend video capture, we’ll do it in way that works everywhere.
We’re not going to hold up 64-bit going final (10.20 final) for this; I think that there are more important features to move forward with: 64-bit, better rendering settings for autogen, DSF recuts, scenery tools. We’ll get to next-gen video capture, but we’re not going to try to jam it into this release.
Posted in News
by
Ben Supnik |
…a new operating system.
Here’s how I think about 32 vs. 64 bit: supporting both 32 and 64 bit ABIs in X-Plane is like supporting Mac and Windows. It means a little bit more work for programmers to cover more executable types but it lets users get more out of X-Plane and add-ons on a wider variety of hardware.
We’re not switch from 32 to 64-bit. We’re adding 64-bit native execution to our list of supported configurations. (Linux nerds have been asking us for this for years! 🙂
So I view the question of “when will my add-ons support 64-bit” as similar to “when will they support a different OS.” It will come with time, but no one is dropping the existing setup.
Developers don’t have the choice to pick 32 or 64 bits; the question is whether to extend support to 64 bits.
Did any plugin developers drop Windows support when they ported their plugins to Linux? I am guessing no. And I am guessing that 64-bits will be the same way.
This is already covered in the X-Plane 64-bit FAQ, but at a minimum we will maintain 32-bit support all the way through the v10 run. My guess is 32-bits will be around for the next major version too, but we’ll figure out those system requirements when the time comes.
BTW: beta 2 is out. Mac users, you have keyboards and mice again. (The bug only happened when you start the sim in full-screen mode.) Windows users, no spinning balls. And GeForce 200 users, your shaders compile. So at a minimum most of the face-palm bugs from beta 1 are fixed.
Posted in News
by
Ben Supnik |
