The Road to PBR (I): HDR and Bloom

I knew from the start that supporting PBR (Physically-based rendering) was going to be one of the biggest (and hardest) milestones of the rendering system refactor. The fact that I’m finally starting working on it (and also writing about it) indicates that I’m seeing the finish line after all this time.

There’s still a lot of work ahead, though.

Supporting PBR is not an easy task in any engine, since it involves not only creating new shaders and materials from scratch, but also to perform several render passes before and after the actual scene is rendered in order to account for all of the lighting calculations. Some of those will make things a bit faster in later stages, like pre-computing irradiance maps, while others will improve the final render quality, like tone mapping.

The road is long, so I’m going to split the process into several posts. In this one, I’ll focus on HDR and Bloom. 

HDR

Adding support for HDR was the natural first step towards PBR, since it’s extremely important in order to handle light intensities, which can easily go way above 1.0. Wait! How come light intensities can have values beyond 1 if that is the biggest value for an RGB component? Think about the intensity of the sun compared with a common light bulb. Both are light sources, but the Sun if a lot brighter than the light bulb. When calculating lighting values using a method such as Lambert or Phong, both of them will end up having the maximum possible value (that is, of course, 1 or white), which is definitely not physically correct. 

HDR stands for “High Dynamic Range” and it basically means we are going to use 16 or 32 bits float values for each color component (RGBA) in our frame buffer. That obviously leads to brighter colors because the values are no longer clamped to the [0, 1] interval, as well as more defined dark areas because of the improved precision. 

As an example, this is what a scene looks like without HDR when having lights with values above 1.0:

The above scene consists of a long box with a few lights along the way. At the very end we have one more light with a huge intensity value (around 250 for each component). What’s important to notice in this image is that we’re almost blinded by the brightest light and there’s no much else visible. In fact, we can barely see the walls of the box.

Now, here’s how the same scene looks like with HDR enabled:

The light at the end is still the brightest one, but now we can see more details in the darker areas, which makes more sense since our eyes end up adjusting to the dark in real world.

Tone Mapping

At this point you might be wondering how those huge intensity values translate to something that our monitors can display, since most of them still work with 8-bit RGB values.

Once we compute all of the lighting in HDR (that might include post-processing too), we need to remap the final colors to whatever our monitor supports. The process is called tone mapping, and it involves a normalization process which maps HDR colors to LDR (low definition) colors, applying some extra color balance in the process to make sure the result looks good.

There are many ways to implement tone mapping depending on the final effect we want to achieve. The one I’m using is the Reinhard tone mapping algorithm, which balances all bright lights evenly and applies gamma correction.

Bloom

While not really a requirement with PBR, but at this point it was really easy to implement and probably the best way to test HDR colors.

Bloom is an image effect that produces those nice auras around bright colors, as if they were really emitting light.

Below, you can see the same scene with and without bloom:

Bloom Off
Bloom On

The technique require us to filter colors in the image using a threshold value. If we’re using HDR, it’s as simple as keeping the colors that are equal or greater to 1.0. Then, we blur the image as many times depending on the desired effect (the more blur we applied, the bigger the final aura around objects). Finally, we blend both the original scene image and the blurred one together. That’s it.

Next up…

In the next post I’m going to talk about the new PBR materials and shaders and how them approximate the rendering equation in real-time.

Cascading Shadow Maps

I mentioned in my previous post that calculating shadows for directional lights was the simplest case of shadow mapping. Well, I’m proud to announce that it’s no longer the case, since the process has become a lot more complex now thanks to cascading shadow maps.

Shadow mapping for directional lights (lights that are very far away from the scene and which rays are considered to be parallel) is fairly straightforward in theory. The shadow map is computed by calculating the closest distance from a light source to the objects in our scene. And it affects the entire scene, so every single object must be processed when calculating the shadows. An orthographic projection is used in this case to simulate the parallel rays.

But the problem with this approach is that it doesn’t scale well when there are multiple objects spread all over the place, since the resolution of the shadow texture is limited. Yes, we could increase the texture resolution but the GPU has a hard limit there too (I think it’s 8k for high-end GPUs at the time of this writing).

Cascading shadow maps work by following a simple idea: we need shadows that are closer to the camera to have the greatest resolution. At the same time, we don’t really care if the shadows that are far away look pixelated (up to a point, of course). Therefore, we need to generate multiple shadow maps, each of them processing a different section of the scene, using a slightly different projection to render different objects based on their distance to the camera.

For example, let’s assume our camera has a maximum viewing distance of 1000 units (meters, feet, light-years, etc…). Then, we could split the scene objects into four groups based on the distance from any object to the camera. Each of those groups will be rendered into a different shadow textures, as follows:

  • Group 1: objects that are less than 10 units away from camera. These are the objects that are closest to the camera and the ones ending up with the higher resolution shadows.
  • Group 2: objects that are less than 100 units away from the camera. These objects should still get a pretty decent shadow resolution.
  • Group 3: objects that are less than 500 units away from the camera. For these objects, the shadow resolution won’t be great, but it might not be that bad either.
  • Group 4: objects that are farther than 500 units. These are the objects in the background. Here, the shadows will look really pixelated and smaller objects might not even cast shadows at all.

Here’s how it looks like in action:

Notice how shadows look pixelated at first since they are farther away from the camera, but they start to look better and better as the camera gets closer.

The following video shows how the different objects are grouped together into cascades of different colors:

Notice how shadows become less and less pixelated as the camera gets closer.

If you paid attention on those videos above, you might have noticed that some shadows disappear completely when the camera gets closer (look at the cyan cascade in the second video). The reason for that problem is very simple to explain: for each cascade, we need to "zoom into" the objects that we care about. So, only objects that are currently visible from the point of view of the camera are actually processed when computing the shadow maps. It might be possible that one object is affected by shadows from objects that are not visible from the current viewpoint. In the videos above, some of the cubes get culled by the camera when they're not visible and therefore not rendered in the shadow map. I still need to fix this behavior. 

If you want to know more about this technique, here’s an excellent article from GPU Gems 3 which I used as basis for my implementation.

Shadows Everywhere!

I spent the last couple of weeks working on improving shadows for each of the different types of lights in Crimild. The work is far from over, but I wanted to share this anyway since there are some visible results already.

Historically, support for shadows have always been poor in Crimild, often limited only to directional or spot lights. Now that I’m refactoring the entire rendering system, it was a good time to implement proper shadow support for all light types.

Light types in Crimild

There four different light types supported by Crimild at the time of this writing:

  • Ambient: This is not really a light source, but rather a color that is applied to the entire scene, regardless of whether the objects are under the influence of any other light or not. It’s supposed to serve as an indirect light source (think about light that bounces from walls or other bodies), since there’s no support for global illumination in Crimild (yet)
  • Directional: The simplest light source. It simulates a light that is very far away and therefore all rays are assumed to be parallel (think about the Sun). Directional lights do not have a position in the world and influence the entire scene.
  • Spot: This is a light that has a position and a direction. The most straightforward example are street lamps or a flashlight. Also, spots may define a cone of influence for the light, only lightings objects inside that area.
  • Point: This is a light source that has a position in space and cast light rights in all directions (a torch or a light bulb). Point lights have an area of influence as well, defined as a sphere.

Shadow Mapping

The technique I used for shadows is the same one everyone’s been using in games for the past 15 or more years: shadow mapping. This technique requires to render the scene at least once from the point of view of the light, producing an image where each pixel is defined as the distance between the light and its closest geometry. Another way to say this is, if the light is casting rays from its origin (or in a given direction), we want to know the distance to the very first objects that are intersected by those rays.

After the shadow map is created, we render the scene as usual in a different pass (from the camera’s point of view this time) and, for each visible object, we calculate its distance (distV) to a given light and we compare that value with the one stored in the shadow map for the same light (distS). If distV > distS, it means something else is closer to the light and therefore the visible object is in shadow.

That was an extremely simplified description of what shadow mapping is. Check this link if you want to know more about this technique.

In the videos below, the white rectangles in the lower-right corner show the computed shadow maps for each of the light types. Darker objects are farther away from the light source.

Shadow Atlas

If there are several lights that need to cast shadows, we need to create a shadow map for each of them, of course. In order to optimize things a bit (and make the shader code simpler), all shadow maps are stored in a single shadow atlas (which is a big texture, basically).

The shadow atlas is not organized in any particular at the moment, though. All shadows are computed in real time, every frame, and the atlas is split into regions of the same size. This is not ideal, but it works.

A future update will split the altas into regions of different sizes. The bigger the region, the more resolution the shadow map will get and the better the final shadow will look. But, how can we define which region is given to which light? Simple: by predefining priorities for each light source. For example, directional lights should be rendered with as much resolution as possible (since they pretty much need to contain the entire scene). So, they should use the biggest available region. Point and spot lights, on the other hand, can be sorted based on distance to the camera. The closer the light source, the bigger the region it has in the shadow atlas.

Enough theory. Let’s see this in motion.

Directional Lights

This is the simples scenario (since I haven’t implemented cascade shadow maps yet). The scene is rendered once from the point of view of the directional light. Since that particular light type is supposed to be far away, we use an orthographic projection when creating the shadow map, meaning are not deformed when projected in the ground.

If more than one light is casting shadows, we need to render the scene once per light, computing the corresponding projection on each case.

The video below shows the shadow atlas in action. All shadow maps are rendered in the same texture.

Spot Lights

Shadows for spot lights are computed in a similar way as for directional ones, except that in this case we use a perspective project when rendering the shadow map since light rays are emitted from a given position in space. The final effect is that shadows are stretched with distance.

If we have multiple spot lights, each of them is rendered individually.

Point Lights

Computing shadows for point lights is the most expensive one, since we need to render the scene six times. Why? Since point lights cast rays in all directions, the shadow map is actually a cube with six faces. It’s like having six spot lights, pointing up, down, left, right, forward and backward.

As you might have guessed, this complexity increases even more as more point lights are added to our scene. In that case, we’re rendering the scene 6*N times, where N is the number of point lights casting shadows.

Next Steps

As I mentioned before, the work is not yet completed. At the moment, I’m working on cascading shadow maps, which is a technique to improve shadow resolution for directional lights (I’ll talk about it when it’s ready).

Also, I want to make some optimizations both on the shadow atlas organization, as well as on each of the light sources. For example, I can use frustum culling to avoid rendering objects that are not actually visible for a given light source. But I’ll leave all that until after implementing physically based rendering (hopefully before the end of the year).

Stay tuned.