A Beautiful Beta-Stopping Bug
While working on the HDR pipeline, I ended up with these. Clearly not what we want to ship, but I do think they’re pretty cool looking.
While working on the HDR pipeline, I ended up with these. Clearly not what we want to ship, but I do think they’re pretty cool looking.
At this point the only thing holding up a public beta of 10.10 is, well, me. I am currently working on a set of related aircraft rendering bugs, and as soon as I can get the car off of jacks, we can go beta.
One of my goals or 10.10 is to close out the issues that stop payware authors from releasing final conversions of their aircraft to 10.10. By final, I mean conversions that won’t need any additional future editing. Right now X-Plane 10.05r1 has a few bugs that cause v10 to look different from v9, different depending on rendering settings, or just wrong. I want to fix those problems in a permanent manner so that authors can release aircraft and not worry about having to update.
Here are my goals for 10.10, roughly in priority order:
The switch of priority between item 2 and 3 is a fundamental change in priority from what I originally intended for X-Plane 10.
Originally I thought that the best thing to do would be to keep non-HDR rendering as close to v9 as possible, so that at least content would look correct with HDR off.
My new thinking for 10.10 is that we should aim for consistency between HDR and non-HDR modes. Realistically, an author is going to have to make at least a few adjustments in moving v9 content to v10; better to have the cost of rendering engine changes be borne out in a v9->v10 upgrade than to have every v10 airplane require double authoring to cope with HDR vs. non-HDR differences. In the long term, it’s best to not have v9 haunt v10 like a ghost years after authors are making exclusive v10 content.
This begs the question: why is HDR rendering so weird? Why does it look different from non-HDR rendering, and why doesn’t it look the same as v9? What did you guys do?
There are a number of changes to how we render in X-Plane 10, some specific to HDR rendering, and some sim-wide.
X-Plane maintains two separate rendering paths for HDR vs. non-HDR and they are quite different in both what happens at each stage and when the stages occur.
Fixing some of the authoring bugs has required further changing the HDR pipeline to allow for correct rendering. The up-side of this change in pipeline is that the new one supports better HDR tone mapping and possibly even better fill rate performance. The down-side is that it’s a lot of complex code to touch and it may take a few betas to work out the bugs.
So I just realized that it’s been a full month since I’ve posted anything on the dev blog, which is a bit ridiculous since we’re working on a ton of stuff for 10.10 and a number of announcements have also come out in the computer industry. I’ll try to catch up on various topics over the next week.
I’m pretty sure that that’s the money question: where is 10.10? The answer is: we are working on it, mainly fixing bugs pre-beta. When will the public beta program start? I don’t know – that will be determined by how quickly we can knock out some of the current bugs. My view is that it’s a waste of everyone’s time to go beta on 10.10 with known bugs that we can fix without going beta – going beta too soon means users waste their own time reporting bugs we know about, and we are distracted from fixing those bugs with the task of going through the duplicate reports.
I think we’ve described this road map before, but in case there is any confusion let me be absolutely clear:
X-Plane 10.10 will not be 64 bit!
We have already made significant progress toward a 64-bit X-Plane and we will continue working on this front. But the plan was never to ship 10.10 with 64 bit support. Rather, X-Plane 10.10 ships with a number of changes to our compilers (as well as a ton of other stuff). The next major patch (10.20) will support 64 bit on all three platforms at once, and we will know that any problems will be due to the 64-bit-ness (and not the changes to compiler, runtime, makefiles etc.) because those will have been vetted in 10.10.
The reason to ship 10.10 in 32-bit is to get out all of the other changes we’ve made so far.
This is not a complete feature list – when we do the first public beta we’ll run through our source control log to scrape out all changes. But here are some fairly big things:
The artists have been working this entire time and we’ve built up a pretty good pile of art assets to ship too – I’m not going to try to enumerate them right now because I’m not up to date on what they’ve created.
One goal I have for 10.10 is to close out all of the bugs that are stopping authors from converting their payware add-on planes from X-Plane 9 to 10. Some of these are already fixed and some are still on my todo list. I’ll post more about some of the stickier remaining issues in another post.
This post will pretty much only be of interest to Linux users – everyone else, talk amongst yourselves. Topic of discussion: X-Plane.com is neither a plane nor a communist. (Sorry, work on the road processing code has drained me of any remaining sanity.)
The brief history of X-Plane for Linux is that Joshua Wise ported X-Plane to Linux a while ago because he felt like it (and being too young to drive at the time, apparently had some free time). The Linux port was never a commercial effort. – Laminar Research didn’t say “we’ll make a ton of money by porting to Linux, so let’s hire someone”. It just sort of happened. (Well, I wasn’t in the room so I don’t know exactly what was said.)
Linux has steadily represented about 5% of our user base from that point on, and this gets to the “commercial problem” of Linux: even with the work we do to try to keep platform specific code in X-Plane to a minimum, Linux sales don’t justify the development cost to maintain the platform.
Yet there still is a Linux version of X-Plane. We keep renewing the Linux port for “one more version” because while the Linux community doesn’t buy a ton of copies of X-Plane, Linux users do contribute in an outsized way by providing code back to the scenery tools and other development efforts. In other words, the Linux community doesn’t contribute dollars, but they contribute a lot of code.
This came up during the X-Plane 9 and 10 Linux discussions. Our primary request to Linux users has been “please support each other”. There are too many Linux distributions for us to provide support; our dedicated support staff does not even have Linux expertise, so Linux support immediately hits engineering.
While a number of Linux users have really done a great job of helping other users out, another issue came up. One of the ways we try to minimize Linux-specific costs is to keep the OS-specific code very basic and rudimentary. In the past we’ve taken patches from users to fix OS specific bugs, but this is an awkward process at best.
Some Linux users asked if we could find a way to release source to the OS-specific parts of X-Plane so that they could improve the Linux-specific code (and thus the platform-specific experience) themselves. Right now this is a very awkward process: I have to tell the user “give me a snippet that does X” and then hope that I can get the snippet integrated and tested on my own.
One of the suggestions that came up at the time was to separate out some of the code that abstracts the OS/hardware so that Linux users could patch it more easily.
Separating out the platform specific code is a good idea, but it wasn’t one we were going to drop everything to pursue – if we did, the end result would be no improvement to X-Plane after some number of weeks of heavy development. Instead, we are slowly moving the code in this direction as we work on it.
The first code to get a work-over is the joystick code; Chris has been refactoring the joystick code for a few weeks now. 10.10 won’t show you any new behavior; it runs the existing UI on top of new low level code. Once we are sure this works, Chris can implement a bunch of new joystick UI features on top of the new low level code quite easily.
The new joystick code separates out the OS-specific joystick interface code from the rest of X-Plane. In the long term, this will make it practical for us to share the hardware abstraction code with users as source, take patches, and even allow developers to swap it out on their local development machines via a shared object. We’re not there yet with this level of flexibility, but the new code makes these things possible.
There is one immediate win to the new code that will be apparent to Linux users in X-Plane 10.10: the new code (which is based on input.h instead of joystick.h) recognizes hat switches as a set of buttons, rather than a pair of axes; this is almost certainly better from a UI configuration standpoint.
I fear one of the main points of my last post was perhaps lost in the excited discussion of how to cope with conflicting crowd-sourced airport submissions. The engineering principle is:
Fix real bugs, not hypothetical bugs. Solve real problems, not hypothetical problems.
By this I mean: it’s invaluable to wait until you actually have evidence of a problem before you design the solution. Without a real data point showing what the bug is, the solution might not actually fit the problem as it actually happens. In a program as complex as X-Plane, particularly when third party content is thrown in, it’s hard to predict in advance what is going to go wrong.
In the previous case (coping with conflicts) the key here is that I want to see a few conflicts before I choose a plan. I see proposals for a revision system, public logs, a registery, a voting system…any one of these ideas might be the right solution, but we don’t know yet because we don’t have two people submitting buildings.
The same principle applies to another issue that was brought up a while ago:
Won’t all of the default airports look the same since they all use the same library?
First, to be clear: we’re not trying to make the default library airports look like the real world airports, we’re not trying to duplicate custom scenery, and if people submit buildings for all 20,000 airports in X-Plane there’s no way we’re going to have 20,000 control tower libraries in the library.
But there is a valid issue: what if people heavily use the assets? Won’t we start to get a “sameness” either between two airports or within an airport as we use the library more and more?
This brings up the second half of “fix real bugs”:
Identify possible solutions, but don’t use them until necessary.
I am not worried about collisions of airport because there are a huge number of possible solutions, many of which are easy to implement and have been proven to work well in the real world in other problems. In other words, the problem of merging crowd-sourced data is a solved problem, and we can use the right existing solution when we have enough data to pick that solution.
The same thing applies to repetition in the library: we have ways of fixing repetition. I want to develop a few airports and see what looks repetitive before asking our artists to spend their time making more stuff* but we at least know how we can fix the problem when the time comes.
So what are the safety valves here? How can we deal with repetition in the library? There are a few ways:
All of these changes can be put into an X-Plane update.
In summary, there are a lot of ways we can make the library more diverse and less repetitive, and as we see the library get used (and certain elements become over-used) we can add more variation to keep things looking fresh.
* For example, it wouldn’t make sense for me to ask Tom to make 50 variants of the administration building if almost no one places it in any airport. We need to see what kinds of assets get used the most and then make decisions.
A quick note on an issue that apparently came up during the X-Plane Town Hall (I have not had a chance to listen to any recording or read a transcript; Austin just pinged meo n this):
What if two people submit library buildings for the same airport?
We actually already have to cope with this since we already crowd-source taxiway layouts.
My view is: we’ll put infrastructure in place when the problem occurs; there are a lot of of ways to cope with collisions of contribution, and the problem has been solved in many cases already. So I want to wait and see what kind of collisions occur and then pick the best method (whether technological or organizational) to deal with them.
Robin already tracks who submitted data, so we have the basis to identify when there has been a change or a possible collision.
WED 1.2 is in beta now — thanks to the users who have taken the time to submit WED bugs (which go here, btw). Tyler is working on an update to the manual, and I’ll try to find some time to squash bugs.
But how do you share an airport with terminals with the rest of the X-Plane community? At the X-Plane developer’s conference, we reiterated that we want to extend our crowd-sourced airport data collection to include terminal buildings and ATC routes, as well as the pavement layouts that Robin already curates for us.
To be clear: you do not have to share your WED creation to use WED! WED can be used for custom scenery and your own private creations; submitting your work for sharing with the community is optional!
Here is our thinking on submitting airports:
You will also still be able to say “Build Scenery Pack” to create a custom scenery pack for your own use – WED 1.2 is meant to support both the crowd-sourcing and custom scenery work-flows.
The “package for submission” work is still in progress; Robin and I will test this out privately while WED is in beta so that test submissions don’t flood the data collection process.
Let me anticipate the inevitable “why don’t you just use Google Maps” question: to the best of my understanding, tracing your airport vectors off of Google Maps imagery is an unlicensed use of their service and would result in a derived product owned by Google. That’s pretty much a deal breaker.
OSM’s use of background imagery to trace (in Potlatch) comes from specific license agreements between OSM and the image provider – in other words, the image providers are “donating” to OSM by giving them a specific license to trace without ownership issues.
Anyway, I am open to other ideas. (Except for “no, you didn’t read the license right, Google really doesn’t mind if you use their imagery for free to create your own data set”. I am not open to that! 🙂
I have updated the WorldEditor page to link to the new builds of WED 1.2 beta 1. This is the first beta of WED 1.2, which we showed at the X-Plane developer’s conference in SC a week ago. A few hilites:
Tyler and I are working on a 1.2 manual but it’s still chaotic enough that I think it isn’t even ready as a beta document.
Last weekend I was in Columbia, South Carolina for the second X-Plane developer’s conference – that is, the US meeting. I’ll try to write that up later, but first a quick note on WED 1.2. We had Tom Kyler on site, and his demonstration of WED 1.2 with the new lego brick objects turned out to be the surprise hit of the weekend. It’s one thing to say “we’re going to crowd-source airports” – it’s another to show the pieces in action.
WorldEditor 1.1 is released, and WED 1.2 is available as a “developer preview” – it’s that developer preview we showed at the conference. A real “beta 1” will be out shortly.
What’s a developer preview? It’s an incomplete beta. In this case we didn’t have all of the features of 1.2 in place, but I wanted to get it out there for people to poke at. Tom set up Seattle using the developer preview, so clearly it’s “usable” – albeit with a heavy dose of caution.
My plan is to get WED 1.2 beta 1 released some time this week, with every planned feature except for “submit-to-Robin”, which he and I should probably test internally a bit while we make sure that WED’s output is solid.
Note that if you are making custom scenery, there’s no reason why you can’t use WED 1.1 for now – it’s finished and stable. WED 1.2 has usability and v10 updates, but overlay editing has been available in WED 1.1 for a while.
After WED, the next scenery tools priorities will be:
It’ll take a bit to get through the scenery tools because we are also mid-patch for X-Plane. 10.10 will be the next “patch” with new features. It looks like we will post a 10.06 first with some new translation files that have come back to us from a number of sources.