Epic games recently displayed a fragment of the rendering technology powering Unreal Engine 5 running on a PlayStation 5. The demo presents a locale exhibiting realistic lighting and shadows as light and color bounce around the environment illuminating the scene and demonstrating an insane level of detail on the background and models not seen before in games. The two primary technologies that Epic games focused on for the demo are Lumen, a procedure for generating real-time global illumination and Nanite, a new technique for geometry to be rendered in games. Lumen Global Illumination Lumen is a ray-traced form of global illumination (aka bounce lighting) that uses a custom type of tracing to achieve a convincing appearance. Ray-traced global illumination works by having a light source emit light rays that hit a surface such as a stone wall, which then bounces the light. The manner in which it is colored from the wall interacts with other objects in the vicinity of the object. Games have used ray-traced global illumination for years but only for static lights and objects. Developers currently use baked lighting or lightmaps, by rendering the lighting then storing the light information via texture maps. While this allows games to be affected by global illumination, the lighting is static (cannot be altered) and permanently affixed to the items in the setting. This results in unrealistic shading if the game has a dynamic day and night cycle or leads developers to need a perpetual time of day. Cube Mapping Another drawback to using texture maps for lighting is any object that would be affected by specular lighting (reflections) such as wet asphalt, glass, or ceramic tiles must use cube maps. Cube mapping is a method of environmental mapping that utilizes a cube’s six sides as a map shape. The environment is projected onto the faces of the cube then placed in the scene to imitate reflections. While results can be impressive there are limits to the practicality of using cube maps. To solve cube map limitations, Nvidia designed graphics cards with real-time ray tracing at the forefront. Microsoft developed Direct X-Ray Tracing, available only with the Direct X12 API, for graphics card manufacturers to bring dynamic realistic lighting to games. By allowing lights to interact with the geometry of the character and environmental models, developers have created extensively realistic lighting in games, unfortunately at a heavy cost to performance. Voxels Instead of bouncing rays from the geometry of the game scenes, Lumen bounces rays from voxels. A voxel signifies a coordinate value in three-dimensional space. Unlike polygons that have their coordinate positions explicitly programmed, voxels infer their position in the environment based upon its relative position to other voxels in the scene. Voxels allow for differing levels of detail mixed with the bounce lighting to save on GPU overhead when compared to direct hardware ray tracing. A downside that can occur (for reasons I am not currently versed in) though is the global illumination can have dithering, noise, and artifacts occur in the bounce light, affecting the game’s color and shadow intensity. Despite these imperfections, Lumen is shaping up to be a fantastic option to achieve realistic lighting in games while not blowing up our GPUs. Nanite Nanite is a micro-polygon render technology that provides a geometric level of detail not before experienced on consoles or PC. When games are in development, the character and environmental artists create extremely high detailed models that would be impossible, with current rendering engines, to animate in real-time. Developers create these high-end models and then reduce the ‘quality’ of the assets to fit their rendering target. The main purpose is that it is easier to reduce the polygon count of a model than to increase it. Developers will take the high-end model and export it as a texture, called a normal map, to be attached to the low-poly model to fake the level of geometric detail. Developers create a multitude of low-poly, low detailed models and textures that render when furthest from the game camera and slowly increase in detail as the character moves closer towards them. This can cause a very unnatural warping effect on the geometry in the game that can be quite undesirable. Epic’s Nanite render technology does not require the use of the low-poly models and normal maps. Nanite lets developers import their artwork directly from ZBrush, a high-end 3D modeling/sculpting program. Unreal Engine 5 can dynamically scale the quality of the model depending on the positioning of the player camera, thus eliminating texture and geometry pop-in and allowing games to look more ideal to real-life scenarios. Conclusion Overall Epic Games is forcing tremendous strides with their advancements in-game rendering technologies. Lumen appears to be a fantastic and less demanding alternative to full hardware ray tracing. Nanite will be a graphics fidelity game changer as it will allow a level of detail yet to be realized in modern games. As both technologies mature and developers can combine them in future games, we may finally hit the uncanny valley where digital graphics look more realistic than reality.