I have posted the first “combined” scenery tools release. Basically ObjView, ObjConverter, and DSF2Text (now named DSFTool) are packaged together with DDSTool (for making DDS images from PNGs) and the new MeshTool (for making base meshes); a UI application called XGrinder provides drag & drop support for most of the converters.
The tools releases will be by month (e.g. March 2008) and each release will contain the latest for all of the tools in the release. WED and the AC3D plugin will continue to be separate downloads.
I have also posted another snapshot of the source code, and generally I’ll try to do a source code post whenever I do a tools post to keep the two in sync.
The AC3D plugin has been updated; I hope to have a new WED posted in the next few days.
Last night I created the “seed” files for MeshTool. Let me explain what these files are and why we’ll need them.
MeshTool is a wrapper around our irregular-mesh generation code. It will allow third parties to create base mesh scenery without having to create triangulations. Just like DSFTool saves people the work of having to encode DSFs (with point pools, command lists, and all that ugly stuff), MeshTool saves people the work of having to create their own triangulations.
MeshTool is a low level tool – you provide a text input file and some data. It’s designed to be an engine underneath tools like PhotoSceneryX, not an end in itself.
MeshTool will create “default” land-use terrain that approximately matches the global scenery, water, and custom orthophoto-based terrain. You (or a program you use) provide a text file that describes the boundaries between custom photos, land, water, and airports. You must also provide a SRTM-style HGT file for elevation.
How does X-Plane know what land-use should go on what terrain? That’s where the seed files come in. Our global scenery is generated from a set of rules that take into account morphology (land height and slope), approximate climate, and general land use. You provide the terrain shape via the HGT file, and we provide you with a seed file that contains climate and land use for that DSF tile.
Why do we provide the seed file rather than letting you find and create climate data? Well, our rules are tuned for a very specific pair of data sets; by providing the exact climate and land use data that we use, we assure that the rules files work correctly. The purpose of MeshTool is not to customize land use terrain, and we do not provide a mechanism for it. The purpsoe of MeshTool is to let you put orthophotos and new coastlines into the base mesh.
The good news is: seed files are tiny. They are typically 4 kb-8kb each; the entire data set is 322 MB total. That’s because the climate data is only 3×3 per DSF and the land use is only 121×121.
I hope to get MeshTool into some kind of testing within the next few weeks; if you are a programmer and would like to feed MeshTool from your own program, please contact me and perhaps I can arrange an alpha copy. I will also post the seeds as soon as I can.
Sandy and I are working on a major revision to the plugin SDK (all the old stuff will work, we’re just adding new things) that should be available in X-Plane 9 soon. The new “2.0” SDK APIs include some new functionality for working on scenery.
- Plugins can find the height of the ground at a given location, which is necessary to draw in the 3-d world in a realistic way (e.g. vehicles that drive on the ground in a sloped airport environment).
- Plugins can load and draw OBJ files using X-Plane’s built-in facilities. I’ve posted OBJ drawing code in the past, but this makes things even easier.
- Plugins can lookup virtual paths in the library to find artwork from scenery packages.
This makes a number of scenery-system concepts available to plugins.
I’ve been resisting OBJ-drawing support in the SDK for a while, but a few things changed my mind:
- We’ve moved as much drawing in X-Plane to OBJs as possible and it’s been a big win. A lot of the dynamic elements are OBJs, they’re used in scenery and cockpits and airplanes. Using OBJ files means our artists (who are not programmers) can customize just about every aspect of the sim. So by providing a file format with a rich tool chain to plugins, hopefully we are helping third parties streamline content development.
- With pixel shaders, X-Plane’s 3-d drawing environment has become complex and hard for third parties to safely augment. By encoding drawing at a higher level via pre-built OBJs (which can be animated via plugin-driven datarefs) we can insulate plugins from drawing-environment changes.
If you have content that worked in 864 but does not work in 9, please file a bug! Please do not assume we intentionally changed the file formats to break compatibility…I would rather get the bug report so we can fix X-Plane than have you change your work.
For every author who changes his or her work in response to a bug (an accidental breaking of compatibility), we’ll have dozens of users who find their add-ons not working but not knowing how to fix it themselves.
With that in mind, beta 16 will have two fixes:
- Some panels were showing the default instrument backgrounds over custom panels. This will also be fixed in the next beta.
- Lit customized overlay parts were not showing. This will be fixed in the next beta.
So if you see those bugs, please check your plane against the next beta when it comes out. If you find other problems, please file a bug!
I’ve spent perhaps the last two weeks almost entirely looking at driver-related graphics bugs. I should note that these are often not driver bugs, and the issue of whether a bug is in the sim or a driver is a lot stickier than you’d imagine, but I’ll blog about that later.
In the meantime, where do we stand on scenery development tools? Here’s a rough summary.
General Strategy
The scenery tools are not targeted to X-Plane 8 or 9 – they’ll export content for both (but if you use new V9-only features, your content will be v9 dependent).
There’s been one structural change in v9 that’s worth mentioning: version 9 allows road networks in DSF overlays. This cleans up the scenery model a lot…roads are more like objects and forests than like the base mesh, so now all of the 3-d elements can be in an overlay. This is particularly useful because you might want to adjust the road and objects nearby all at once.
Given that we have no road editing tools right now, I think there is one case where the tools will be v9-centric: I do not think I will ever provide a road-editing tools for DSF base meshes with roads. I do think I will add road editing to the list of overlay features that will eventually end up in WED.
This means that if you want to customize roads, you’ll have to do it in version 9. This is an inevitable outcome of the scenery system structure…replacing the entire base mesh to edit the roads is not a great idea.
WED
WED is in beta 4, and I think it will go final as soon as I have time to make it official. At this point most of the open bugs I get are usability issues and enhancement requests, not real bugs. We’ll get more usability features into WED in the future.
Please don’t send me WED bugs now (or attach them as blog comments). I’ll put out a formal last-call for WED bugs when I have time to dig in.
The future plan for WED is to add DSF overlay editing with all legal overlay types (which is now everything except beaches and base mesh).
AC3D
The AC3D export plugin already supports new v9 OBJ features – key frames and panel regions. These tools are available now!
Tools
The “tools” are the set of other scenery apps: DSF2Text, ObjConverter and ObjView. We’ll be adding two new tools soon:
- DDSTool is a PNG->DDS converter. There are already other programs (many free) for both platforms that can do this conversion, but we wanted something easy and drag & drop when we were doing the in-house conversion, so I coded DDSTool (based on the Squish library). We’ll make it available to authors.
- MeshTool is a base-mesh generation program. We’re still working out the details on this, but basically it will generate DSF base meshes from raw DEM data and polygonal coastlines.
I am restructuring the tools to be entirely command-line-based (except for ObjView) with a single “drag & drop” UI that wraps all of the tools. So if you use the mouse, you’ll use the one drag & drop application to do all your DSF2Text converions, 3DS->OBJ conversions, PNG->DDS conversion, etc. This is mainly to cut down the number of apps I have to build, but it should also declutter workflow a bit.
MeshTool
I’ll post more on MeshTool once we have more details, but here are the main points:
- MeshTool will not have a UI. It will take text files and process them. It is designed to provide a building block on which people can write more advanced tools. In other words, MeshTool gives other programmers our mesh algorithms just like DSF2Text gives other programmers our DSF-packing algorithms.
- MeshTool will not be an editor. It will create NEW meshes from RAW DATA, not from previous meshes. There are a lot of reasons why I do not want the mesh-creation workflow to be “take the global scenery and hack it”. So MeshTool lets you build your own.
- This means you will need your own elevation and coastline data for MeshTool! You can’t use what shipped in the global scenery! However, since the data that ships the global scenery is freely downloadable and widely available, I don’t think this is a problem.
- MeshTool will be able to “burn in” orthophotos into a mesh. This is probably its most important feature. Right now the only way to make orthophoto-based scenery is to use overlay polygons, but the performance of .pol files with a lot of scenery is really poor under version 9. MeshTool will let people build photo-sceneries that take full advantage of the framerate that X-Plane can deliver!
MeshTool should address some of the common issues I’m hearing with orthophoto sceneries (slow performance with a ton of .pol files, no ability to fix the coastlines or remove underlying base terrain, no ability to fix elevation).
MeshTool will create new default landuse terrain where there are no orthophotos or water. This is important because the elevation data can be customized. If you take an existing DSF and somehow edit the elevation points (for example, use DSF2Text and just start editing) and raise a grassy field, you’ll have grass going up a cliff. But if you pass the raw elevation data to MeshTool, it will build a rocky cliff over steep elevation, creating more plausible scenery.
MeshTool should be in the post 9.0 final time-frame, as will all the other tool reposts.
Posted in Tools
by
Ben Supnik |
I’ve posted the first beta of the AC3D X-Plane export plugin version 3.1 on the scenery web site. This plugin contains support for key frames and panel regions, two new version 9 features.
Generally, the scenery tool code is not part of the X-Plane code. There aren’t version 9 and version 8 scenery tools. There will always be one set of scenery tools that let you optionally use the v9 features or not.
An example is the AC3D plugin. Everyone can use the new plugin (and a few small features apply to X-Plane 8 too, like a less sluggish animation UI when editing large objects). If you use key frames, your object will only work with v9 – if you don’t, it will work with v8.
More specifically on key frames: a traditional v8 animated object is really just a key framed object with only two key-frames. So the UI has changed very little – basically where you only had two key frames (points of correspondence between the dataref and animation) in the old plugin, you can now use the “add” button to add more.
I will try to add features to the scenery tools to warn when v9 features are being used, so that if you want your scenery to run on v8 and v9, you can check for v9-only feature usage.
For this blog entry I will try to show how to put together taxiway segments to avoid WED problems.
Here we have a 3-way junction, that is, one taxiway ending with another. First, what not to do:
First we have an overlapped solution. This is not good because overlap is wasteful of graphics card power. We also have the problem that the edge markings will overlap. So in the second and third picture, we “cut away” the horizontal taxiway markings by adding vertices. (In picture two we have overlap, in picture 3 we try to end the taxiway right at the intersection.)
The problem with technique two is: we still have overlap (and imprecise markings). The problem with picure three is that we can’t guarantee that the vertices of the vertical taxiway are exactly “on top of” the horizontal taxiway edge. So you may get something in X-Plane that looks more like this:
(Gap exagerated for clarity.) This is a “slivering” problem. The solution is one that is both efficient and nice looking:
In this situation, I have inserted two vertices in the edge of the horizontal taxiway and then used “snap to vertex” to assure that the corners of the vertical taxiway are exactly the same as the “cut points” in the horizontal one. This gives me a connection that will look clean in x-plane with no cracks or slivering and no overlap. My lines also end at just the right places.
(Hint: once you do this, to move all of the overlapped vertices at once, select them all with the marquee tool and drag.)
Another issue I see sometimes with WED layouts is how to draw the “fillets”, or curved taxiways that touch runways. Because the lines are not as smooth in x-plane as WED, often users are surprised when the approximations of the bezier curves cause an overlap and theere is no pavement.
Note that we’re stuck – we can’t use the technique described above because we can’t insert vertices into the runway. So we have to overlap. Less overlap is better – the overlap here is exaggerated to make things clear.
In this curve, we have a control handle only on the bottom vertex. It will overlap as soon as the control handle reaches the other side of the top line. By pulling the control handle down, we restore the curve.
Here we have two control handles – now it is a question of whether the top control handle crosses the top of the taxiway.
Here is one solution – at the expense of more overlap we can simply give ourselves some breathing room with our bezier curve. However, my preference would be to try to keep the control handles below the taxiway top by a wide margin.
I don’t actually know all of the best ways to make an apt.dat file. The format is new and we haven’t had a ton of time to performance test it. But here are some thoughts:
- One thing I know is…it is best to describe the shape you are using with the fewest number of nodes possible. In other words, let bezier curves do their thing, don’t add a ton of vertices to them. And please don’t add vertices to try to make the curves smoother. The best thing is to leave the layout simple and let X-Plane render it. In the future we will allow the user to set the level of smoothness based on his or her hardware. Adding more nodes will hurt quality – we will be able to pick better nodes at runtime than you can in the apt.dat file.
- Avoid overlapping large areas of pavement. Overlap is bad – it hurts framerate by requiring the graphics card to draw the same pixels over and over. As X-Plane supports more advanced shaders for nicer lighting, this cost will be a lot heavier on users.
- Ignoring overlap, the question of whether to have several smaller chunks of pavement vs. one large one is a difficult question. I would suggest moderation – try not to make too many individual pavement elements, but don’t make the entire element one giant pavement either. You may have to experiment to find the best framerate.
I posted previously in my blog that we use airport boundaries to flatten airports when we make the DSFs, as well as place airport grass. Let me clarify this a bit: there are two stages of flattening that we do.
- The flattening in the DSF is a “soft” flattening – we do not completely flatten the area, but rather we try to remove high frequency spikes in the elevation data, so that the airport maintains its overall shape without being unusable. This flattening process is very precise in that it only happens inside the airport area. Because we decide the mesh when making the DSF, we can shape the mesh to flatten only the airport area.
- If flattening in the sim is enabled, we then do a “hard” flatten of an area including the airport and some surrounding area. This flatten is truly flat and destroys any topography for the airport and a lot of surrounding terrain. Because the mesh is already defined, X-Plane is limited in its choices of where it can choose to flatten or not flatten.
Now that first soft flattening is what we use the airport boundary for. In X-Plane 800 the US DSF flattening worked very poorly. For the 820 flattening it worked well for most cases, but had some very specific bugs. (Typically the confluence of water, airport, and/or a DSF tile in close proximity would cause the three different processors for water, airport, and boundaries to come into conflict.)
I am going to try to further improve the flattening algorithm for the v9 render, with the hope of fixing some of the buggy cases from the 820 render. I am sure there will still be some broken cases – with 14,000+ DSF tiles, one of them is bound to be weird…but my hope is that with each re-render we can get closer to a render where users can run with sloped runways enabled and enjoy realistic non-flat runways.
I don’t like it when overlay scenery depends on the underlying base mesh…I’ve been telling authors for a while now on the lists, “don’t assume the base mesh is the global scenery”, and while this actually has been a safe assumption for version 8, I suspect that once we get v9 out the door I’ll have some time to write mesh-editing tools, at which point the user will have a choice of base meshes.
Roads are currently built in MSL space (that is, absolute elevation on the planet) and thus don’t change when the base mesh changes. In a previous post I discussed the idea of having them be AGL based (draped) so that they would “hug” whatever base mesh you have.
First I should say: this isn’t coming any time soon and won’t be in 9.0. But also, upon closer examination, there are a number of ugly issues just below the surface. Consider all of the things that the global-scenery generator does with roads:
- Road type is selected not only from source road data, but from surrounding conditions, some of which are not obviously determined by the final DSF.
- Road bridging is determined based on nearby roads, water, gradient constraints, and available bridge templates.
- Road type is further influenced by bridging considerations.
- Roads are smoothed to try to build the most even access, “hiding” rapid changes in underlying terrain from a high-speed roadway.
- Powerline tower locations are picked from terrain peak to peak with constrained maximum lengths. (The individual segments also controls the span of wires, so introducing new nodes in the draping process might be undesirable.)
The problem therefore with a draped road is that the decision to bridge (both how high and what kind) are dependent on the underlying terrain.
The eventual solution would be to do all of the road processing in X-Plane while draping – but this implies that the author of a custom road grid doesn’t actually have control over how the roads come out. One advantage of the current system is that total control over the roads is provided in the DSF – given appropriate tools, a modeler could very carefully hand-craft the roads with a lot of detail.
