High Fidelity's lighting - how to improve it


#1

We had a conversation with Philip last night and one of the topics that came up was how High Fidelity’s lighting could be improved.

I’ll start with the obvious.

  1. Shadows and light are just simply not being generated correctly. I see light bleeding under walls, through floors, through roofs, etc. The biggest culprit seems to be the Key Light of the zone entity. I had to turn it off in my domain and put point lamps everywhere instead (they don’t have this issue).

Notice how the keylight is bleeding under the wall into the room interior At first I thought this might be because I had an edge split modifier on, so I remade the entire building so that everything was completely connected (walls and floor) with no holes or sharp edges. This did not fix the issue. Even with everything completely sealed the light bleeds through. (There’s another problem in the shot above I forgot to mention - point lamps don’t seem to cast shadows. Look underneath those leaf chairs, for example. There’s no shadow on the ground underneath. And yes, I do have shadows turned on.)

  1. The other main issue is that meshes with alpha textures do not cast correct shadows. For example, if you have a fence texture with holes of transparency in it, those holes are not reflected in the shadow cast by the fence. You just see a flat solid shadow instead.

  2. The THIRD main issue is that shadows disappear when you get far away from them. This is jarring if you are traveling across a large terrain. It reminds me of the obvious LOD pop-in of old N64 games. Instead of making a shadow disappear outright, High Fidelity should just load a lower-res version of it instead as you get far away.

If these three issues could be fixed I would be satisfied. The lighting still won’t be up to par with other platforms but it at least won’t look like the rendering engine is drunk.


However - if High Fidelity DOES want to reach parity with the other platforms in this area, here’s how you do it. The main issue is that High Fidelity lacks any sort of global illumination. Here’s an example showing the difference between a shot rendered with no global illumination vs one with it.

Notice that the second shot looks brighter underneath the monkey head. That’s because light is bouncing off the ground in the same way it would in the real world. In High Fidelity, this doesn’t happen. Instead, HiFi tries to artificially make it brighter with the Ambient Light from a zone entity. This is a one-size fits all approach, and is obviously less realistic. It’s an approximation that has to be guessed by the content creator.

Unlike Sansar and VRChat (Unity), all lighting in High Fidelity is generated dynamically, in real time. This is by design, as it allows you to change anything in the scene and not have to go through a time-consuming light-baking process. In those other platforms, you have to rebake all of the lighting if you change even a single object. This is the norm in more traditional game engines (Unreal, Unity, Source, etc.) There is obvious benefit to HiFi’s way of doing this, but also the obvious drawback - Lighting generated in real-time every single frame is NEVER going to look as nice as lighting that took hours and hours to bake. .

Because High Fidelity is designed around real-time editing, it would be unfair and unrealistic to expect global illumination in High Fidelity. Every game engine that supports global illumination has to bake it (Sansar, UE4, all of them do this, and in game development, complex scenes can take hours or even a whole day for a single scene). However, there’s a workaround that High Fidelity could open to us. By easing up on the texture limits (we’re capped at 2048x2048, and only 4 of them max on one mesh), HiFi could allow us to use global illumination that was baked to a texture externally, through Blender’s Cycles for example. High Fidelity should let us turn off texture limits within our own domains if we want to. This platform is all about creative freedom, right? Let us decide for ourselves how to design our scenes. Mozilla would never limit the size of images that Firefox could load. Neither would Google with Chrome. There would be uproar if they even tried. So why is the web browser of VR, High Fidelity, doing it?

That ends my rant. Thanks for reading this giant wall of text.


#2

Textures are capped t 2048x2048 ? Well that sucks.

I expected 4096x4096 and i need that for my mesh terrain. Especially as long high fidelity does not have inworld terrain.

Sl instead of one 4096 texture high fidelity prefer that you use more smaller textures and make things more complex ? in this case i need to use 2048x2048 for the same mesh.

We need 4096 textures !

Oh, i think shadows are still turned off by default. need to turn it on to check.


#3

Just a side note:

I think your shadows and AO are actually turned off (which by default are off). If I consider that scene that is. because otherwise the keylight could cast an aggressively draw-distanced shadow (you can see it fading out in the distance like you talked about), like this

Or did they break it recently?

As mentioned, Sansar and unity generate the GI and Ambient Light prior to the scene being rendered in real time. High Fidelity’s Ambient Light drives a similarish, functionality, but is limited by the fact that you cannot generated those baked maps on the fly, or after the scene has loaded, so its static to what ever the ambient URL is set to be. there is a race condition to decide "when " to do the baking too.

I was thinking there should be a button that takes a screenshot / generates light probes for a scene, allowing us to set it for a domain ambient url, within high fidelity.

Working on something similar for the non-community version of the blender plugin, done through cycles on scene export. It is not GI though, and thats something I’d like to see as well, but I doubt we will see it until they have more crew in the graphics team (now only being sam and samuel, which have done great work with the setting of materials on objects they showed yesterday, including texture animations via scripts and tiling of textures etc)

Now regarding texture size: this limit should be completely client dependant not set in stone at 2048. So that those who can, should run it as what ever size they are set, and those who cant run at the more capped scales.

I think they reduced it because there are so many unique textures out there, and most avatars tend to ignore memory budgets, probably will try to go for the largest possible detail possible, and that will impact performance, since game engines tend to be more restrained in texture size.


#4

My shadows are definitely enabled. I even toggled them off and then back on before taking the screenshot just to make sure the menu wasn’t lying to me.

I intentionally don’t have AO turned on (because the current implementation looks awful), it’s not relevant/related to the issues I showed in the screenshots above.

I like that light probe idea, sounds like it could work well.


#5

Oh wierd. I am not used to not seeing any shadows at all even if it’s on


#6

Well planned statement. We live in a world with UHD and 4k everything; why are we being restricted to 2k textures…


#7

I rolled the texture limit question around in my head for a while, because while my first reaction can tend to be, “That’s silly! Gimme MOAR,” sometimes that’s rather rash. :wink:

Seems to me that texture limits have their place, but should be determined by a domain owner. If content coming in over something other then their asset service(s) is over a limit they set, then it could be resized, blocked, or replaced. A universal limit made a lot of sense somewhere like Second Life™ where you have continents filled with people of varying skills and knowledge concerning content. Here, we’re all the gods of our own domains. It might make sense for a place like hifi://earth that offers free land to set limits on textures or other assets. Another domain may be carefully and skillfully crafted that can make very good use of 4096 textures.

This is a lesson I learned from my (omg long time) spent in OSgrid. In that open grid, it’s the simulator owners that determine limits on physical objects, prim counts, script functions, etc. Those sim owners are also responsible for their regions and the content they contain, not OSgrid. If their regions run badly due to poor content or a lack of backend resources, that is on them. The same principles should apply here in High Fidelity.


#8

Nah, its not matter of backend resources.

Its a matter of rendering power, which is why the limit should be on the client side and adjustable via settings. After all its all rendered by the client