How do shaders like 'Ocean Skybox' work?

[DustyWizard]

Can someone explain to me how the Ocean Skybox animations work? eg this item:

Ocean Skybox - High Fidelity Marketplace

If I wanted to go about writing custom shaders for a skybox like this how would you set this up in High Fidelity? Are there any example projects I can look at where I can see all the code?

Thanks!

[Tim.Crook]

I’d like to know the same thing. I got this shader from the marketplace:

Water - High Fidelity Marketplace

And it only lasts for an hour or so before it disappears from my domain and I have to re- rez it.

It looks like doing my own shader is the way to go.

[Adrian]

Go to this site:
https://www.shadertoy.com/
Browse to find a shader you like and have a fiddle with the editor. The code is a whole learning curve. That will give you a starting point, however porting them into Hifi is yet another thing, it can be done but there is a lot to cover.
Some of the shaders (.fs files) can be used right out of shadertoy, loading them as you would the water one, so long as they dont use an external image or noise (you can see which ones use image and which ones dont).

There are a couple of threads on this forum about them. Here is an old one but the images seem to be not loading.
adapted cloud shaders/
For more info search for "procedural shaders, fractal shaders, fragment shaders, procedural textures and procedural skies.
I have found them to be somewhat unreliable in Hifi.

[Adrian]

Seems like that thread is so old none of the links still work.
Here is prob the most valuable part of that thread, here is the bare bones of the skybox shader with a simple color for sky, simple color for ground and a simple fog area at the join.
Edited to fix the colors

#line 2

“#define SKYCOLOR vec3(0.29, 0.68, 0.98)”
“#define FOGCOLOR vec3(0.68, 0.85, 0.98)”
“#define OCEANCOLOR vec3(0.07, 0.05, 0.98)”

//Resolution X, Y, Z (ratio , by default 1.0)
//vec3 iResolution = vec3(iWorldScale.xz, 1.0);

// We set the seconds (iDate.w) of iDate to iGlobalTime, which contains the current date in seconds
//vec4 iDate = vec4(0, 0, 0, iGlobalTime);

vec3 getSkyboxColor()
{
vec3 rayDir = normalize(_normal);
float pos = rayDir.y;
vec3 col = SKYCOLOR;
float r = sqrt( dot(pos,pos) ); //vector length = radius
float smooth_to_fog = 0.0;

/***** Calculate Sky *****/
if( pos > 0.0 )
{
    col = SKYCOLOR;
    smooth_to_fog = 1.0 - smoothstep(0.025, 0.2, r);
}

/***** Calculate Ocean *****/
else if( pos <= 0.0 )
{
    col = OCEANCOLOR;
    smooth_to_fog = 1.0 - smoothstep(0.01, 0.02, r);
}

/***** Mix sky or ocean with fog *****/
col = mix( col, FOGCOLOR, smooth_to_fog);

return col;

}

Here is the code for the 2D water shader modded for Hifi by Summer4me

#line 2
// Found this on GLSL sandbox. I really liked it, changed a few things and made it tileable.
//
// by David Hoskins.

// Water turbulence effect by joltz0r 2013-07-04, improved 2013-07-07
// Redefine below to see the tiling…
//#define SHOW_TILING

// by summer4me…
// I have found the script here https://www.shadertoy.com/view/MdlXz8
// and have adapted it for procedural entities in Highfidelity

#define TAU 6.28318530718
#define MAX_ITER 5

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{ vec2 iResolution;
iResolution.x=7.0;
iResolution.y=7.0;
float time = iGlobalTime * .5+23.0;
// uv should be the 0-1 uv of texture…
vec2 uv = fragCoord.xy / iResolution.xy;

#ifdef SHOW_TILING
vec2 p = mod(uvTAU2.0, TAU)-250.0;
#else
vec2 p = mod(uv*TAU, TAU)-250.0;
#endif
vec2 i = vec2§;
float c = 1.0;
float inten = .005;

for (int n = 0; n < MAX_ITER; n++)
{
    float t = time * (1.0 - (3.5 / float(n+1)));
    i = p + vec2(cos(t - i.x) + sin(t + i.y), sin(t - i.y) + cos(t + i.x));
    c += 1.0/length(vec2(p.x / (sin(i.x+t)/inten),p.y / (cos(i.y+t)/inten)));
}
c /= float(MAX_ITER);
c = 1.17-pow(c, 1.4);
vec3 colour = vec3(pow(abs(c), 8.0));
colour = clamp(colour + vec3(0.0, 0.35, 0.5), 0.0, 1.0);

#ifdef SHOW_TILING
// Flash tile borders…
vec2 pixel = 2.0/ iResolution.xy;
uv *= 2.0;

float f = floor(mod(iGlobalTime*.5, 2.0)); 	// Flash value.
vec2 first = step(pixel, uv) * f;		   	// Rule out first screen pixels and flash.
uv  = step(fract(uv), pixel);				// Add one line of pixels per tile.
colour = mix(colour, vec3(1.0, 1.0, 0.0), (uv.x + uv.y) * first.x * first.y); // Yellow line

#endif
fragColor = vec4(colour, 1.0);
}

// This is the function that is being called by us
vec4 getProceduralColor() {
// retrieve the position to get the color
vec2 position = _position.xz;
// add a half to all the axes to adjust them to our method
position += 0.5;
// invert the y axis
position.y = 1.0 - position.y;
// initialize the result value
vec4 result;
// We call shadertoy their entry point here, which is mainImage for normal viewports
// This function writes to the result value, as input we enter the position multiplied by the current worldscale
mainImage(result, position * iWorldScale.xz);
// Return the colour vector to our renderer in Interface
return result;
}

If you believe the problem with the water shader might be the hosting or marketplace intrusions then host this file as water.fs or something and call it into your own object via the userData field.

userData:

{
“ProceduralEntity”: {
“version”: 2,
“shaderUrl”: “http://www.your_ip/folders/water.fs”
}
}

[Tim.Crook]

excellent, thanks @Adrian. I’ll give it a shot.

[DrFran]

Whatever happened to Summer4me? They made nice stuff, and they were nice as well.

[Tim.Crook]

{
“ProceduralEntity”: {
“version”: 2,
“shaderUrl”: “http://www.your_ip/folders/water.fs”
}
}

nope, I tried that and it didn’t work. Any suggestions?

[Adrian]

Maybe. I have to assume you didnt just paste it in as shown but put in your own URL?!

It worked last time I tried but like I said, it is unreliable which was a polite way of saying good luck with this bucket of black snakes. Sometimes it works and sometimes it doesnt but its nearly always going to bite you, the only constant is that nobody ever has any good answers, welcome to Hifi,.
I dont know what posessed me to post anything, I promised myself I was done. Its my own fault.
I have already wasted far too much of my life on this.

All I will suggest is to revert back to version 1 instead of version 2

{
“ProceduralEntity”: {
“shaderUrl”: “http://www.dontForgetToUseAReliableHost.com/hifi/scripts/water.fs”
}
}

(use both http://www. and type it in with your own path, dont copy paste)
It seems like your real problem Tim is stuff disappearing from the marketplace, probably nothing to do with the procedural shaders at all, but some unknown unexplained completely misunderstood elements of the code that makes these things work (or fail to work).

I am sorry but Hifi cannot be trusted to play well.
Life is too short for this, I am sorry.

[Tim.Crook]

haha, yes I did put the files url from my bucket in the web address.

it is, stuff I get from the marketplace usually disappears within a week, but the water shader doesn’t even last a day in my sandbox before it disappears.

I get that hi-fi is unreliable, early days and all that, I guess I’ll have to keep plodding along and ask questions when I get really stuck.

Thanks for your help Adrian.

Cheers,

Tim.

[SamGondelman]

Hi everyone! I need to put together a docs page on this stuff but here’s all the details I’ve collected on how our shaders work (for now):

• Shaders can only be applied to Shape and Zone entities

v1 shaders:

• must implement vec3 getProceduralColors() or vec4 getProceduralColors() (alpha is ignored)
• are completely emissive (unlit)

v2 shaders:

• must implement float getProceduralColors(inout vec3 diffuse, inout vec3 specular, inout float shininess)
• diffuse/specular/shininess are set inside the function to the desired values, not returned
• returned value is emissiveAmount
• if emissiveAmount > 0.0, unlit
• shininess goes from 0 to 128

v3 shaders:

• must implement float getProceduralFragment(inout ProceduralFragment proceduralData)
• ProceduralFragment looks like:
  struct ProceduralFragment {
  vec3 normal; // world space normal
  vec3 diffuse;
  vec3 specular;
  vec3 emissive;
  float alpha;
  float roughness;
  float metallic;
  float occlusion;
  float scattering;
  };
• returns emissiveAmount, unlit if > 0
• everything has default values but can be set like proceduralData.alpha = 0.5;

v4 shaders:

• same as v3 but with per-fragment position
• only use if position is really needed, otherwise will be less performant than v3
• must implement float getProceduralFragmentWithPosition(inout ProceduralFragmentWithPosition proceduralData)
• ProceduralFragmentWithPosition looks like:
  struct ProceduralFragmentWithPosition {
  vec3 position; // world space position
  vec3 normal; // world space normal
  vec3 diffuse;
  vec3 specular;
  vec3 emissive;
  float alpha;
  float roughness;
  float metallic;
  float occlusion;
  float scattering;
  };

Available global variables

v1:
uniform float iGlobalTime; // shader playback time (in seconds)
uniform vec3 iWorldScale; // the dimensions of the object being rendered

v2/v3/v4:
// Unimplemented uniforms
const vec3 iResolution = vec3(1.0); // Resolution doesn’t make sense in the VR context
const vec4 iMouse = vec4(0.0); // Mouse functions not enabled currently
const float iSampleRate = 1.0; // No support for audio input
const vec4 iChannelTime = vec4(0.0); // No support for video input

uniform float iGlobalTime; // shader playback time (in seconds)
uniform vec4 iDate;
uniform int iFrameCount;
uniform vec3 iWorldPosition; // the position of the object being rendered
uniform vec3 iWorldScale; // the dimensions of the object being rendered
uniform mat3 iWorldOrientation; // the orientation of the object being rendered
uniform vec3 iChannelResolution[4];
uniform sampler2D iChannel0; // these 4 channels are set by the “channels” section in the userData
uniform sampler2D iChannel1;
uniform sampler2D iChannel2;
uniform sampler2D iChannel3;

all versions:
vec3 _normalWS;
vec3 _normalMS;
vec4 _color;
vec2 _texCoord0;
vec4 _positionMS;
vec4 _positionES;

// For retro-compatibility
#define _normal _normalWS
#define _modelNormal _normalMS
#define _position _positionMS
#define _eyePosition _positionES
ProceduralEntities are also supposed to support transparency, but for it to work, you need to set the alpha on the entity to be < 1, and it’s not exposed in the UI yet.

For ProceduralSkyboxes:

• mostly the same
• must have vec3 getSkyboxColor()
• built in uniforms:
  vec3 _normal;
• also support same built in uniforms as Procedural Shapes, depending on version

Also, for all types:

• write discard; in your shader to completely discard the fragment. That pixel won’t be colored at all, it’ll be like a hole through the entity.

Add to Entity

• set userData to something like this:

{ "ProceduralEntity":
    {
        "version": 4,
        "shaderUrl": "https://gist.githubusercontent.com/SamGondelman/932c8c03fd39b0549f0a07c0a736abd9/raw/85e047ff15e68c29c730823421b52a4024311753/raymarch.fs",
        "uniforms": { "scene": 0 }
    }
}

• you can define custom uniforms and up to 4 textures by adding something like this to the ProceduralEntity object in userData:

        channels: [ <texture1>, <texture2> ]

I realize this is a lot of information! I want to get a docs page set up with examples and everything but haven’t had the time. Let me know if you have any questions!

[Tim.Crook]

I realize this is a lot of information! I want to get a docs page set up with examples and everything but haven’t had the time. Let me know if you have any questions!

That’s great Sam, thank you. I managed to get the water shader working late last night but I’ll have a poke through the code and see what else I can come up with.

Cheers

[DustyWizard]

Thanks Sam, this was very helpful. Some more examples would be helpful to learn how this works. I’m hoping to use this to create a more realistic dynamic sky.

Further questions:

1. What variables/functions are available in the shader code? Can we pass variables from an entity script so that we can vary the shader based on user input?
2. I notice in the raymarch.fs example you shared you access a function called ‘getEyeWorldPos()’ - is there a list of what functions are accessible?

[SamGondelman]

Yeah, I would love to put together some examples sometime soon!

To answer your questions:

1. You can pass variables from entity scripts to shaders via uniforms in userData. In my example, I pass a variable called “scene”, which is accessed in the shader. Uniforms can be sent as floats, vec2s, vec3s, or vec4s. For example: uniforms: { aFloat: 0.0, aVec2: [0.0, 1.0], aVec3: [0.0, 1.0, 2.0], aVec4: [0.0, 1.0, 2.0, 3.0] }
2. As I mentioned, you can also pass up to 4 textures via the channels
3. In addition to the global variables I mentioned, here’s what you have access to in the shaders:
   • any built in glsl functions (be mindful that some of these may behave differently on different graphics cards or machines)
   • const float DEFAULT_ROUGHNESS = 0.9;
     const float DEFAULT_SHININESS = 10.0;
     const float DEFAULT_METALLIC = 0.0;
     const vec3 DEFAULT_SPECULAR = vec3(0.1);
     const vec3 DEFAULT_EMISSIVE = vec3(0.0);
     const float DEFAULT_OCCLUSION = 1.0;
     const float DEFAULT_SCATTERING = 0.0;
     const vec3 DEFAULT_FRESNEL = DEFAULT_EMISSIVE;
   • TransformCamera getTransformCamera(), where a TransformCamera is:
     struct TransformCamera {
     mat4 _view;
     mat4 _viewInverse;
     mat4 _projectionViewUntranslated;
     mat4 _projection;
     mat4 _projectionInverse;
     vec4 _viewport;
     vec4 _stereoInfo;
     };
   • vec3 getEyeWorldPos()
   • bool cam_isStereo()
   • float cam_getStereoSide()
   • miscellaneous noise helper functions, which are all defined in this file: https://github.com/highfidelity/hifi/blob/master/libraries/gpu/src/gpu/Noise.slh

[Adrian]

Thank you Sam for stepping up.

If you do make some examples and I really hope you actually do this, please make a few simple and basic ones so real users can actually get some use out of them, and try not to make them as technical and involved as they might get. It can easilly look like a lot of sensless jargon to the average coder/builder.

If you really want people to get the most out of this and produce really nice results, you must let us learn how it works first, and only you can teach us. Really we cant teach ourselves from docs of technical specifications.
This information is quite extensive but can be quite useless to the average user, its all just dry technical data and says nothing about how to actually make stuff work.

Some more examples would be helpful to learn how this works

Nothing is as valuable as simple working examples, along with a basic explanation.

Thanks again for getting involved.

[SamGondelman]

Completely understood! To be honest, the reason I have been holding off on doing this is because I am pushing for us to support shaders on meshes first, which can be applied with material entities. Once we have that, this old system will likely be deprecated, and it would be nice if the examples could just cover the new method from the get go.

[Tim.Crook]

Add to Entity

• set userData to something like this:

{ ProceduralEntity:
    {
        version: 4,
        shaderUrl: "https://gist.githubusercontent.com/SamGondelman/932c8c03fd39b0549f0a07c0a736abd9/raw/85e047ff15e68c29c730823421b52a4024311753/raymarch.fs",
        uniforms: { scene: 0 }
    }
}

Hey @SamGondelman I tried to get this working but couldn’t, any hints?

[SamGondelman]

My bad, you need the quotes:

{
    "ProceduralEntity":
    {
        "version": 4,
        "shaderUrl": "https://gist.githubusercontent.com/SamGondelman/932c8c03fd39b0549f0a07c0a736abd9/raw/85e047ff15e68c29c730823421b52a4024311753/raymarch.fs",
        "uniforms": { "scene": 0 }
    }
}

[Phil]

Correction:

v3 shaders:

• must implement float getProceduralFragment(inout ProceduralFragment proceduralData)

ProceduralFragmentData seems to have changed name to ProceduralFragment.

Thanks

[Phil]

ProceduralEntities are also supposed to support transparency, but for it to work, you need to set the alpha on the entity to be < 1, and it’s not exposed in the UI yet.

Do we have any way of setting an entity’s alpha < 1, so that translucency from the shader will work?
Thanks

[SamGondelman]

Ahhh sorry, you’re right about ProceduralFragment. Fixed in my post!

And the easiest way to set the alpha is with this snippet that you can run from the developer console, just swap out the id of the entity you want to make transparent: Entities.editEntity("<id of entity in quotes>", { alpha: <desired alpha from 0 - 1> });