Heightmap fun and hints


#1

Was experimenting with height maps, using 512x512 basis to split my overall terrain into 4 quadrants. Should have been simple…

512x512 height map implies 2^12 scale for 8192M coverage.

Place lower left - translation 0,0,0
Place upper left - translation 8192,0,0
Place lower right - translation 0,0,8192
Place upper right - translation 8192,0,8192

Left gaps until I remembered another engine where I have to make my maps 513x513 (or 1025x1025) to make happy. Tried it and at 513x513 scale 2^12 the edges land at 8192 on X, Y as needed. Yay!

Guess that’s why the heightmap helper I have for blender has the +1 pixel sizes as options. Still experimenting, but starting to get great results, so good I may have to give up my precious mesh terrain.


#2

Now the next issue becomes the drop in elevation at the edges… leaves a trench at edges… fill with water? :wink: More to come once I sort that bit out.


#3

Looking at the following image you can see the rolloff at the join - what you can’t see was a 8192 M mesh box to verify the edges land at correct place (they do)… if the sinking edge wasn’t there it would stitch perfect. Now to find what I’m doing wrong to create that dip. It’s like a fun puzzle! :wink:


#4

I guess you used insert what @chris told me to use.
I tried insert, but theres no heightfield terrain visible or so tiny you barly see it. Something is incompatible between the heightfield importer and heightfield insert with size. Need to look closer at it. Still also not sure how many meters 1 pixel on the heightfield map is. I guess 1 meter. Need to puzzle more. looks like your stretching maps very much.


#5

Think of it like a painting… you begin by filling in a base, it’s just color covering the canvas then you add detail with finer brush strokes layering upon the base.

Or, you create 4096 maps and stitch together if you want the same look as SL/OS. Why 4096? 16384/256 = 64 then 64*64 = 4096. That’s the only way it will look similar to resolutions seen in those places. That’s math, not opinion. And doing that would be madness.

My goal is to achieve a low render cost fill for the entire domain, then layer more detail upon it using additional metavoxel and mesh parts, so, the stretched textures work for me, not against in that context.

The tools work generally well, just need to learn how to use them effectively.

My only issue is the height drop at the two edges - looking at it from an in world origin of 0,0,0, that would be a drop of height at the top and right edges. I’m going to assume it’s an issue on my side for now and will experiment more. Overall it seems a promising way to create terrain over large areas quickly and, with practice, will enable some really appealing visuals.

By the way… meters per pixel. Simple. 16384/512 = 32. Assuming 2048x2048 is the maximal heightmap size that’s viable that would give 16384/2048 = 8 meters per pixel. That would define the best you can do using one heightmap to fill an entire domain in one piece.


#6

The latest metavoxel update (weekend December 6, 2014) makes some good changes - it requires re-generation for your metavoxels but is well worth it. Have also discovered the drop off I’m seeing on edges is my problem, not the metavoxel importer/render engine. Will sort that out.

Scaling is a lot easier to understand for the less geeky. When doing an insert height map, the spacing parameter determines the size based upon height map pixel resolution. So… a 512x512 at 1.00 spacing gives you a 512x512M insert with a 1M per pixel resolution. Height scale should be something close to how you modeled your height map source, if the peak height is 500M then that’s what it needs to be, but depending on how made, it might look better with some experimenting there above/below that value. Also as you change from spacing = 1.000 your height ratio can change visually.

What I’m seeing is… I model a 2KM x 2KM with a peak height of 180M mesh in Blender, convert to a 512x512 (or 513x513 :D) and drop in HiFi… it needs to be spacing 4.0, height 180M and if I want the lowest point to be at the domain floor a height offset of -90 is indicated.

But, depending on model, it might be better to bring it up a bit from there, say -89 or so. Depends on what you want and will be far less critical once physics come into play.

Cool thing is, it’s all logical and simple math. The scaling, texture spread and offsetting in 3D space all work as expected. Let the real fun begin!


#7

The metavoxel editor (heightfield map importer) seems to got some improvements with scale. also understand better how todo some things. Insert i dont touch.
Also when you fly over your terrain "we need a plane !) it looks like minecraft with terrain generation in the distance.

So, overal it looks good. :slight_smile: Only bad i cannot apply to the imported terrain in my domain = crash.


#8

More notes to self and anyone else it may be of use to.

Still seeing that I need +1 pixel sizes to get expected in world sizes. For instance, a 512x512 map at scale = 1 does not create a 512x512M metavoxel area. A 513x513 does (same goes for other sizes). This is seen with other engines working from height map graphics as well. Could it be coordinates being 0 based index and decoding of png placing in a 1 based array? : ) I can confirm metavoxel field sizes using mesh objects of known size (my ruler sets). Given that mesh agrees with all the base voxel sizes, metavoxel sizing is the odd man out. So it’s either 2 wrongs and 1 right or 1 wrong and 2 rights.

Extrapolation can be your best friend. Given a height delta of relatively small peak value, it is far better to have less data points. Given a height map of 1024x1024 vs 64x64 for a small in world space (128M plot test) the 64x64 height map is superior as the dots it connects are more spread leading to extrapolation to a smoothed line versus a stair step. It’s well worth creating maps at much lower resolution, testing and only moving upwards in size if the geometry stops following the desired shaping. Also the engine seems to happily support small size maps - I’ve tested down to 32x32 pixels.

Updating metavoxel items can be very slow - as is initial rendering upon login. This does not, so far, seem related to complexity of metavoxels present. I’m seeing about 5 to 10 seconds lag in displaying metavoxel data - turning in a circle does not speed up the process as seen with entities. This may be peculiar to my metavoxel server, it’s an ancient core 2 duo laptop. It’s not network related as this is happening on a high speed local LAN. Will see how it fares on the production machine (i7) eventually.

Edit: Promoted build to much faster primary server - seeing same thing, metavoxel landscape takes 30 seconds or more to appear.


#9

I think this problem can be fixt to see where the avatar is inworld and only load that part from the server, expect the are doing this already or have it on the list. (i hope) you cannot load a 100sqkm terrain at once. :open_mouth:


#10

Glad to see you’re doing so much with the metavoxel stuff! Needless to say, there are a lot more improvements in the pipeline.

First, the delay you’re seeing seeing on load is probably the download speed (or at any rate, that is a problem, even if it’s not the one you’re seeing). Since I switched to lossless compression on the terrain (which removes the compression artifacts we were seeing, as we were using eight bit JPG for both height and color data), we’re sending much more data, and we’re sending it through our own reliable UDP layer (as opposed to TCP), which may be relatively inefficient. The plan is to switch back to lossy compression at least for the color data, and also to cache terrain data to avoid resending. There are also some tweaks to be made in terms of adjusting the LOD computation (which determines both what gets rendered and what gets sent over the wire). You can see if metavoxels are downloading by opening the extended stats (forward slash, then click on the stats display) and looking for a download percentage under the metavoxel node counts.

As I think you discovered, the “Import Heightfield” and “Insert Spanner” (with type Heightfield) options now do the same thing; “Insert Spanner” just gives you more control over the transformation (allows you to rotate the heightfield, basically) at the cost of some convenience. With “Insert Spanner,” the heightfield starts out as a 1x1x1 (meter) cube, with the vertical range (0 to 255 for an eight bit import, 0 to 65535 for a 16-bit one) spread out over the vertical (Y) range. Scale scales everything at once, aspectY controls the ratio of Y:X, and aspectZ controls the ratio of Z:X. With “Import Heightfield,” the “Spacing” option controls the distance (in meters) horizontally between height samples (so, setting it to 1.0 with a 1024 height map will extent to about one kilometer). The “Height Scale” is the range in meters corresponding to the imported range (so, 1.0 would map 255 to 1.0 for an eight-bit import), and the “Height Offset” is just a Y offset in meters.

Aside from what I mentioned (fixes for download delays), my current project is coming up with a better solution for uniting the heightfields with the dual contour representation. Currently, the dual contour representation (generated piecemeal when you use the “voxel” tools like the Sculpt Brush) doesn’t quite align with the heightfields, and always uses a fixed resolution, which means that it’s very easy to overload the server by attempting to tunnel into a large, low-resolution heightfield. The plan is to use a more memory and bandwidth efficient format that combines both heightfield and voxel data and uses the same horizontal resolution for both.

Other things on the horizon are masks to control how different terrain textures blend together and baked lighting computation on assignment clients.


#11

@Andrzej

Still pondering the time to download being the delay. I’m seeing no difference between loading a scene from my local development machine on a gigabit LAN vs loading from my production machine located in a data center then accessed by my 35mb cable connection, the machine in colo center is on a very fat/fast data pipe - it saturates my 35mb cable on an SFTP download. :smile:

What I do see, after watching it enough times is a fairly long delay, then the ping time to server goes to around 500 ms then metavoxels render almost as soon as ping time goes up. This is same LAN vs WAN and slow local serving machine vs fast remote machine. Could there be something leading to a delay in triggering metavoxel render event on connect to a domain?


#12

I wouldn’t think so, but I’ll keep an eye out. When I load a large terrain data set on a server running on my local machine, or on another machine on the LAN, it loads within a couple seconds (of download time). When I load the same data set from an external server, it can easily take up to a minute (of painfully slow download time).

I should also mention that there’s an issue specific to Windows with importing large heightfields that’s related to the problems we’ve been having with loading large models on that OS. It’s some kind of memory allocation issue with Qt, and we’re continuing to look for a fix.


#13

Thanks - I’m using a 256x256 height map using a 1024x1024 texture for diffuse. I’ll time the load from LAN vs remote to be sure I’m not nuts thinking there’s no difference in lag to render and correct my assertion if needed.


#14

@Andrzej

After investigating more - it’s no question it’s the download time - sorry for attempting to throw a red herring.

It’s odd that it’s so slow via LAN to LAN access, but either LAN:LAN or LAN:WAN I can see the Mbps holding at 1 to 2 Mbps then drops to 0.07 or so as the metavoxel field appears.


#15

Thanks @Andrzej.

I wish i could test more with metavoxels. but i cannot apply the terrain, its not getting saved. https://worklist.net/20227 Hard to test loading speed if theres no terrain. :wink:


#16

@Andrzej

If the latest mac stack manager and windows interface is using the 1450 MTU size for metavoxels then I’m happy to report it’s helped drastically. I’m seeing mine load in about 5-7 seconds now vs 30ish before. If it’s not why I’m seeing faster load times then I’ll look forward to the patch and cutting that in half! lol

Thanks!


#17

@Andrzej , intressting. because L3DT have some sort of problem to with saveing large textures, in L3DT that only works as BMP because the creator rewrote that image format routine. . Maby there some idea’s in this topic.

http://www.bundysoft.com/phpBB2/viewtopic.php?t=1128


#18

One thing that would be a huge help is a way to move camera about scene while inserting a spanner. When importing a height field I’m able to use cam controls (by turning on newEditEntites.js) and see what I’m doing, even moving my AV while working to, for instance, align ground plane with feet/structures.

Really want to use spanners, adding patches of metavoxel details… it’s the promised land, but, a bit futile for me as I can’t see what the bleep I’m doing if I move it outside my view or want to do fine adjusts before committing.

Maybe I’m missing something, if so, do tell! If not - we really need some cam controls for those of us deprived of goggles. : - )


#19

Oh, finaly have a heightfield map loaded and saved. but saving takes a long time.

Done more testing, I see now the problem, if you press APPLY and close the metavoxel editor directly after that. Nothing is saved and the terrain diappear. Theres no feedback when its saved.

Note: it did feel like its not saved, but the terrain sems saved fine now.
Better option would be that you can close the metavoxel editor not sooner then the sterrain is saved.

But it appears a few seconds later as saved. We need some visual feedback what happens.


#20

Yeah, it takes some time after pressing “Apply” first to process the heightfield images (for which there’s no feedback at present), then to upload them to the server (if you have the bandwidth display active, you should see the upstream rate increase for a while, then decrease; in the extended stats, you’ll see the upload rate at 0% until finished–that’s a bug that will be fixed soon). Eventually I’m sure we’ll add some user-friendly feedback for the entire process.