I'm new to the engine and want to write custom shaders

Graphics
#1

Hi,

After spending a few months with Unity, I’m now considering switching to a different engine.
I tested Flax and got a positive first impression.
So far, I have only written an FPS character controller as a test.

Specific shaders are essential for my project, as they need to create a specific aesthetic.

I haven’t yet figured out how to write custom shaders. Ideally, I would like to write my own HLSL shaders with vertex and fragment stages.

Maybe someone experienced can help me understand how to achieve the following features:

  • Vertex-Snapping (PSX like)
  • Vertex Colors (from mesh)
  • Vertex-Lighting
  • Affine Texture Mapping (also PSX like)
  • Scalable Pixelation¹
  • PSX-like dithering and color space transformation²
  • Quantized Light-/Shadowmapping³
  • User-adjustable fullscreen shader (parameters like scanline-strength, etc.)

¹ Scalable Pixelation

The resolution limit is scalable and done this way in my Unity shader:

// Apply resolution limit to the base texture.
int targetResolution = (int)log2(_ResolutionLimit);
int actualResolution = (int)log2(_BaseMap_TexelSize.zw);
int lod = actualResolution - targetResolution;

half4 baseColor = SAMPLE_TEXTURE2D_LOD(_BaseMap, sampler_PointRepeat, i.uv, lod) * i.color;

I don’t know if there’s a similar LOD texture sampling system in Flax?

² PSX-like dithering and color space transformation

I won’t use dithering on single textures since my textures are already have dithering applied.
However, I’m using an user-adjustable fullscreen dithering shader:

col is the fullscreen blit, colLow is a downsampled (pixelated) version sampled with Unity’s LOD texture sampling as seen above. This results in a very nice dithering effect without calculating colors.

float3 PsxDitherFullscreen(float3 col, float3 colLow, float strength, uint2 p)
{
    col    *= 255.0;
    colLow *= 255.0;
    uint dither_u = psx_dither_table[p.x % 4][p.y % 4];

    col   += ((dither_u / 2.0 - 4.0) * strength);
    col    = lerp((uint3(col)    & 0xf8), 0xf8, step(0xf8,col)); 
    colLow = lerp((uint3(colLow) & 0xf8), 0xf8, step(0xf8,colLow));

    col    = lerp(col, colLow, step(0xf8, (dither_u / 15.0 * strength)));

    col /= 255.0;
    return col;
}

You can see my fullscreen shader in action here:
https://youtu.be/yauQqGNjKvA

For the colorspace transformation (on textures, not fullscreen) I’m using modified parts of this code:
https://github.com/Mortalitas/GShade/blob/master/ComputeShaders/PSXDither.fx

// truncate to 5bpc precision via bitwise AND operator, and limit value max to prevent wrapping.
// PS1 colors in default color mode have a maximum integer value of 248 (0xf8)
col = lerp((uint3(col) & 0xf8), 0xf8, step(0xf8,col)); 

// bring color back to floating point number space
col /= 255;

³ Quantized Light-/Shadowmapping
I already have shaders in Unity which support quantized mapping. It’s done using this technique:
https://www.reddit.com/r/Unity3D/comments/1gdj7ik/comment/lu4csgp/

The image of the initial reddit post shows the desired effect. This is, of course, only intended for use with fragment stage lighting, not vertex lighting. It is a nice solution to simulate baked shadows on lowres textures like some PS1 games did. Metal Gear Solid and Alien Resurrection come to my mind.

Thanks a lot for reading!

#2

Go ahead and see our docs about shaders: Shaders | Flax Documentation

In that sections, you can find code examples for custom geometry drawing (custom Vertex and Pxiel shaders too) or Compute Shaders. You can use typical HLSL code with some Flax-specific macros. Code from Unity should be rather easy to transfer.

For example, that SAMPLE_TEXTURE2D_LOD is Unity-specific macro which could be done as:

_BaseMap.SampleLevel(MySampler, uv, lod), where you need to define that texture and sampler to be bonded from code int oa specific registers. See HLSL official docs to learn more.

#3

I figured out how to import HLSL files and implemented a first post processing shader.
Now I’m trying to implement a custom surface shader but as far as I see, there’s no documentation on it?
This is something really essential, so I guess it’s possible, somehow?
Where can I find some information on that topic?

#4

We have an in-built material generation that uses templates depending on material domain:

You can output material properties from custom shader to be passed there (as MaterialLayer connection).

#5

Thanks! This looks like a solution I’d get along with very good.

Is there a way for me to create my own material template within my project? Like a modified version of the surface material template? This would be very nice. Can you please provide a more detailed advice on how to do this?