Sprite Projection

[philstifts]

In the last months I got a little distracted from my game project and tried to come up with a solution to one of my most aching problem with modern rendering. Especially in regard to recent Unreal-Engine fads to completely throw overboard all subtle artistic sense and to just agree on Quixel Megascans to be the one ultimate way to present better looking landscapes, nature and the likes.

As impressive as it looks it is a hilarious overkill and absolutely nothing you can work with as an artist beside just copy things. It is in analogy to the step from paintings to photographs.

The question arise: How can you render many stupid, small, random things like small rocks and foliage in a landscape without providing painstakingly crafted 3D-Models for it that need to be reduced to an unrecognisable and badly lit geometric cube-thingy as soon as the distance and amount would kill performance? How can you get the feel of such things and don’t completely kill flexibility.

You can try to project a sprite in 3D space.

Hey, this is stupid! Why the heck should I use such a monstrosity of a shader when I can just use a flat 3D model for my sprites and billboard like every game with sane graphic programmers do?

Let me explain…

[sinsro]

Not sure I caught the question here, you would like the engine to support something like speedtree which would automatically use billboard impostors at a given range?

[philstifts]

Yeah, impostor-algorithm is a very close one. But the twist is - it’s the complete opposite. Where with impostor you project a detailed 3D model on a 2D sprite, here you project a 2D sprite on a detailed 3D model.

What sounds extremely silly at first is really nice in practice. Cause you can do something like this:

You can render unlimited sprites in full detail with no performance drops. The core algorithm is so simple it should run even on potatoe-hardware.

From there on, you can raytrace shadows, calculate distance AA with mipmaps, integrate shadow maps (that’s a lil bit complicated in FLAX), and even animate with spritesheets.

And on the mathematical side it looks like this:

In my example I use a textured model for the ground. But just cause I am a cheapskate. This shader is meant the render even the ground on sprite basis.

[Ray_Rune]

Sounds like a showcase. Can we see your reproducible example?

[philstifts]

If you want to know the limits of this shader, it is simple. There are of course a lot. It is basically raymarching through projection layers. With all the known problems that come with raymarching.

First and foremost: It is very dependent on precise hit detection of the sent ray. If you don’t find your sprite you will waste all the power of the gpu for nothing. That is the reason I use a 3D model as basis. The raymarcher transforms into a fur shader basically.

If you are a really mad lad you can try to go the “Media Modecule” route and prerender a SDF-Map with information on vague ray hit detection every frame. FLAX even uses something like this for the more complex lighting stuff. You can write a whole graphic engine on this information alone.

Second and also very important: The axis space needs to be uniform. Makes the calculations super simple and fast. You can rotate the whole space without problem, but never only one sprite-layer. This will kill your gpu and is nearly impossible the control coding wise.

As long as I want to use this stuff in a game I don’t give away code, but I can post binaries of the showcase for examination of the restrictions it comes with.

Besides that it is really not that hard to implement. Nearly every shader on shadertoy is more elaborate. If someone knows basic raymarching it should be absolutely doable.

And yeah, ‘unconventional’ foliage and landscape rendering is more or less the only use case that makes sense. That is the reason I wrote this shader. ; )

[Ray_Rune]

Not sure I caught the question here (2)…

[philstifts]

The question is: How can you bridge the uncanny valley in landscape rendering without:

• brute force your way through with an excessive amount of workforce and hardware strain in traditional ways
• resignate and either go totally abstract and low-tech or just copy without creativity

I play games since the 90’s and landscapes never looked so dead, sameish, soulless ever before. In nearly every game. Even though the technical means are absolutely impressive nowadays. And if there is no million dollar budget behind a game it is even worse. Sometimes it looks like all real artists left the industry. And what’s left is a mechanical fabricate.

I don’t want this for my games. And I really don’t care about newest tech, highest resolutions, sharpest shadows and so on. But the expressiveness should be there.