Your terrain generation is looking good. This is something that was/is on my list of things to try out in Flax but some other pre-steps had to occur before doing this. This was one of the things I wanted to do for the community here on Flax during my 1.6 hard core test (feedback given). Since you are doing this I can give you all the information I have to help. There is a lot of information below but all is specific to game engine terrains. They are handled differently…something I did not expect and found out the hard way. Flax’s terrain system seems to be very good and you seem to have an excellent start.
Thank You for doing this. Whether or not it becomes community or not it is very much needed. I would be happy to donate via Paypal once at the next stage. I am sure others would donate too.
No TLDR here…I kept most everything as condensed as possible though.
If you want/need further things on this please PM me. I can upload things to my FTP for you to grab to help with. UE4 has an excellent water shader too (the original UE4 Water…not the latest one). I have great normals for waves. You are welcome to these. I can also add more screen clips here in this topic to keep this open for the community.
Some thoughts here from doing a lot of landscaping in both Unity and UE4 (I have 15 10km x 10km) regions done in both using the same height maps filled out with 3D vegetation, snow drifts, and 3D mesh roads. Procedural is great but only does 80% of the work, but is a huge win to help make things look natural without the human tendency for personal replicated taste. Best if the human works with the last 20% after in my humble opinion.
I guess the main advice here if you make this terrain system is…Keep It Simple Stupid…KISS. Unity’s Gaia and CTS had it almost perfect before it had far too much scope creep. UE4 has a couple automatic landscape products on the marketplace and I gave up on them since they are performance hogs and modifying them when something is outside of what the designer’s vision is (example…the vision is mostly to replicate earth forests and grasslands with mostly rolling hills…makes sense I guess…they are pretty). MicroSplat in Unity had the best solution and was very performance friendly. The ‘Make Terrain Button’ does not exist. LOL!!
For the Terrain Auto Material I can help get you started with what needs to be done. In theory it should transfer well. This is a UE4 landscape material I developed from migrating from Unity then watching a couple videos on how to work with the landscape layering system in UE4. Flax ‘should’ have these but unsure. Might need some Flax support to make sure these options are available or the landscape layering will be a mess like in Unity (very difficult to work with).
Unity Refs used in this design: MicroSplat (mainly) and an old version of CTS.
A small note on Splat Mapping style terrain layering from experience. The best success of a good splat map was using L3DT to output a Splat Map along with the seamless tiled height maps. Works great…but…I always had to go back and hand paint to get rid of a lot of artifacts (both Unity and UE4). In light of this I decided to simply skip the splat and hand paint. I think you will find with doing a landscape material like below hand painting is extremely fast with very few artifacts. Rocks are great artifact fixers in any case. 
Terrain Textures (I have textures I could donate/help with if needed):
- PNG (RGB no Alpha so not RGBA)
- No larger than 2048x2048 (or you will hit texture memory limits fast)
- Bake your AO into the texture (I will briefly explain how below this)
- No need for SM (specualr map). Simply set the specular to 15,15,15 or 32,32,32 RGB to start.
- No need for displacement maps or any other map…saving valuable texture memory!
- Use GIMP 2.10 and the Normal Map Filter to generate your normal maps (Photoshop does terrible Normals).
Making terrain textures look almost 3D:
- Each texture has what is called Texture Fog.
- Drag your texture into GIMP 2.10, Layer, Brightness/Contrast.
- Slide your Contrast left and right. Takes practice but you will see where the texture transitions from soft to sharp…this is where all the texture fog is removed. Slide to slightly right to add some softness back in. You will see what happens but the texture will have ‘Baked in AO’ and will look almost 3D.
- Now slide the Brightness down a little to add some AO to the texture.
- Done. Export (Note: the GIMP xcf format is a very good method to store your texture libraries)
GIMP Normal Maps:
Search for GIMP Normal Map Plugin. Here is the github link for it. Same filters found in the NVIDIA plugin.
GitHub - hhirsch/gimp-normalmap: Automatically exported from code.google.com/p/gimp-normalmap
I find this works best. Normal Maps should be subtle…especially for terrain textures.
- Set your Scale from 3.0 to 5.0. Start with 5.0 and test to see. Varies per texture and lighting model. More than 5.0 and the normals start to show artifacts. Prewitt 3x3 also works well.
Details:
The Flax Manual does not have many details on the Terrain Material Layering. In fact it is very vague on how to do layering. The manual does show the base layer then hand painting layers on top. The layer material looks like a standard material in the Terrain Domain. Not sure what the limits of layer blends Flax has. UE4 is 8 (more it gets frisky and causes a lot of issues…remember that SM5 is limited to 16 Texture Samples…which all games engines are limited to for now…more than 8 will run out of texture space and create very large CPU and GPU texture loads).
So this Auto-Landscape Layering is 8 Layer Blended (almost no tiling) and works simply by painting the texture layers on the terrain. You can automate the layer texturing but I find painting by hand works best after the terrain itself is shaped. For example: a 10,000m x 10,000m take about 8hrs to fully paint from start to finish. So it is very fast. It also had grass (2 layers with any 3D plant) and snow cover masking built it so it covers most any terrain needed.
A note about things like trees and rocks. I spray these down in a 3D brush then hand adjust since procedural tools do most of the heavy work to keep clicking down and great randomization but always need some sort of adjustment on a few items, especially on slopes which are everywhere naturally.
The Output section: 8 Landscape Layers Blended.
Auto Slope Function:
Note: Below here you will find that each layer is very similar, which they are. This makes everything consistent and simple to adapt.
Base Terrain Layer (Rock Layer…this is what everything paints on top of)
Dirt Layers (1 & 2):
Grass Layers (1&2):
Sand Layers (1&2):
Snow Layer:
Procedural Grass/Foiliage (tied to both of the Grass Layers):
A sample of what this will do. This is only the material and terrain mesh (done with L3DT) and with Ultra Dynamic Sky (UE4). I am positive this can be done in Flax.
how these things work once I return to them or…