Shaders in Computer Graphics

In computer graphics, a shader is a computer program that calculates the appropriate levels of light, darkness, and color during the rendering of a 3D scene — a process known as shading. Shaders have evolved to perform various specialized functions in the fields of computer graphics special effects and video post-processing, as well as general-purpose computation on graphics processing units.

Traditional shaders compute rendering effects on graphics hardware with a high degree of flexibility. Most shaders are coded for (and run on) a graphics processing unit (GPU), , although this is not a strict requirement. Shader languages are used to program the GPU rendering pipeline , which has largely replaced the fixed-function pipeline of the past, which only allowed generic geometry transformation and pixel-shading functions ; with shaders, custom effects can be used. The position and color ( hue , saturation , brightness , and contrast ) of all pixels , vertices, and/or textures used to construct the final rendered image can be altered using algorithms defined in the shader, and can be modified by external variables or textures introduced by the computer program calling the shader.

Shaders are widely used in film post-production , computer graphics, and video games to create various effects. Beyond simple lighting models, more complex shader applications include: adjusting an image's hue , saturation , brightness ( HSL/HSV ), or contrast; creating blur , light bloom , volumetric lighting , normal mapping (for depth effects), bokeh , cel shading , posterization , bump mapping , distortion , chroma keying (for so-called "blue screen/green screen" effects), edge and motion detection , as well as psychedelic effects such as those seen in the demoscene .

Illustration of the division of tasks between the CPU and the GPU.

This usage of the term "shader" was introduced to the public by Pixar in version 3.0 of the RenderMan interface specification , originally published in May 1988.

As graphics processors developed, major graphics software libraries such as OpenGL and Direct3D began supporting shaders. The first shader-capable graphics processors supported only pixel shading , but vertex shaders were quickly introduced once developers realized the power of shaders. The first video card with a programmable pixel shader was the Nvidia GeForce 3 (NV20), released in 2001. Geometry shaders were introduced with Direct3D 10 and OpenGL 3.2. Ultimately, graphics hardware evolved toward a unified shader model .

An example of two types of shading: flat shading on the left and Phong shading on the right. Phong shading is an improvement on Gouraud shading and was one of the first computer shading models developed after the basic flat shader, significantly improving the appearance of curved surfaces in renders. Shaders are most commonly used to create lit and shaded areas when rendering 3D models .

Another application of shaders is for special effects, even on 2D images (for example, a webcam photo ). The unaltered, unshaded image is on the left, while a shader has been applied to the same image on the right. This shader works by replacing all light areas of the image with white color, and all dark areas with a brightly colored texture.

Shaders are simple programs that describe the characteristics of a vertex or a pixel . Vertex shaders describe the attributes (position, texture coordinates , colors, etc.) of a vertex, while pixel shaders describe the characteristics (color, z-depth, and alpha value) of a pixel. A vertex shader is invoked for each vertex in a primitive (possibly after tessellation ); thus, one vertex in, one (updated) vertex out. Each vertex is then rendered as a series of pixels onto a surface (a block of memory) that will ultimately be sent to the screen.

Shaders replace a piece of graphics hardware commonly known as the Fixed Function Pipeline (FFP), so called because it performs lighting and texturing in a hard-coded manner. Shaders provide a programmable alternative to this hard-coded approach.

The basic graphics pipeline is as follows:

• The central processor sends instructions (compiled shader-language programs) and geometric data to the graphics processor located on the video card.
• In the vertex shader, the geometry is transformed.
• If a geometry shader is present in the GPU and active, certain changes to the scene's geometry are performed.
• If a tessellation shader is present in the GPU and active, the geometry in the scene can be subdivided .
• The calculated geometry is triangulated (broken down into triangles).
• Triangles are broken down into quad fragments (one quad fragment represents a 2 × 2 primitive fragment).
• Quad fragments are modified according to the fragment shader.
• A depth test is performed; fragments that pass it are written to the screen and may be added to the frame buffer .

The graphics pipeline uses these steps to convert three-dimensional (or two-dimensional) data into useful two-dimensional data for display. In general, this is a large pixel matrix, or "frame buffer".

Initially, video cards were equipped with several specialized processors supporting different instruction sets. Shaders were divided into three types depending on which processor would execute them (depending on which instruction sets were available):

• vertex (eng. vertex shader);
• geometry (eng. geometry shader);
• pixel or fragment (eng. pixel shader, fragment shader).

There are three types of shaders that are commonly used (pixel, vertex, and geometry shaders), and several new ones have been added recently. While older video cards use separate processing units for each shader type, newer cards have unified shaders , which are capable of executing any type of shader. This allows video cards to use computing power more efficiently.

Later, video cards began to be equipped with universal processors (GPUs) supporting the instruction sets of all three shader types (unifying the shader architecture). The division of shaders into types has been retained to describe the shader's purpose. It became possible to perform general-purpose computations on the GPU (not related solely to computer graphics), such as mining and neural networks.

2D shaders operate on digital images , also called textures in computer graphics. They modify pixel attributes . 2D shaders can take part in the rendering of 3D geometry . Currently, the only type of 2D shader is the pixel shader.

Pixel shaders, also known as fragment shaders, compute the color and other attributes of each "fragment": a unit of rendering work affecting at most one output pixel . The simplest types of pixel shaders output a single screen pixel as a color value; more complex shaders with multiple inputs/outputs are also possible. Pixel shaders range from simply outputting the same color to applying a lighting value , performing bump mapping , shadows , specular highlights , translucency, and other phenomena. They can alter the depth of a fragment (for Z-buffering ) or output more than one color if multiple render targets are active . In 3D graphics, a pixel shader alone cannot create certain kinds of complex effects, since it operates on only one fragment, without knowledge of the scene's geometry (i.e., vertex data). However, pixel shaders are aware of the screen coordinate being rendered and can sample the screen and nearby pixels if the entire screen's contents are passed to the shader as a texture. This technique can provide a wide range of two-dimensional post-processing effects, such as blurring or edge detection/enhancement for cartoon/cel shaders. Pixel shaders can also be applied at intermediate stages to any two-dimensional images — sprites or textures — in the pipeline, whereas vertex shaders always require a 3D scene. For example, a pixel shader is the only type of shader that can act as a post-processor or filter for a video stream after it has been rasterized .

A pixel shader operates on fragments of a raster image and on textures — processing data associated with pixels (such as color, depth, texture coordinates). A pixel shader is used at the final stage of the graphics pipeline to form an image fragment.

Example code for a pixel shader in DirectX ASM:

ps.1.4
texld r0, t0
mul r0, r0, v0

3D shaders act on 3D models or other geometry, but can also access the colors and textures used to draw the model or mesh . Vertex shaders are the oldest type of 3D shader, typically making changes on a per-vertex basis. Newer geometry shaders can generate new vertices from within the shader. Tessellation shaders are the newest 3D shaders; they operate on batches of vertices at once to add detail, for example, subdividing a model into smaller groups of triangles or other primitives at runtime, enhancing things like curves and bulges , or modifying other attributes.

Vertex shaders are the most established and widespread type of 3D shader and are run once for each vertex given to the graphics processor. The goal is to transform the 3D position of each vertex in virtual space into the 2D coordinate at which it appears on the screen (as well as a depth value for the Z-buffer). Vertex shaders can manipulate properties such as position, color, and texture coordinates, but cannot create new vertices. The output of a vertex shader passes to the next stage of the pipeline, which is either the geometry shader, if present, or the rasterizer . Vertex shaders can provide powerful control over the details of position, motion, lighting, and color in any scene involving 3D models .

A vertex shader operates on data associated with the vertices of polyhedra, such as the coordinates of a vertex (point) in space, texture coordinates, vertex color, the tangent vector, the binormal vector, and the normal vector. A vertex shader can be used for the view and perspective transformation of vertices, for generating texture coordinates, for calculating lighting, and so on.

Example code for a vertex shader in DirectX ASM:

vs.2.0
dcl_position v0
dcl_texcoord v3
m4x4 oPos, v0, c0
mov oT0, v3

Geometry shaders were introduced in Direct3D 10 and OpenGL 3.2; they were previously available in OpenGL 2.0+ using extensions. This type of shader can generate new graphics primitives , such as points, lines, and triangles, from the primitives that were sent to the start of the graphics pipeline .

A geometry shader, unlike a vertex shader, is capable of processing not just a single vertex, but an entire primitive. A primitive can be a line segment (two vertices) or a triangle (three vertices), and with adjacency information for a triangular primitive, up to six vertices can be processed. A geometry shader is capable of generating primitives "on the fly" (without involving the central processor).

Geometry shaders were first used on Nvidia series 8 video cards.

Geometry shader programs execute after vertex shaders. They take an entire primitive as input, possibly with adjacency information. For example, when working with triangles, three vertices are the input to the geometry shader. The shader can then emit zero or more primitives, which are rasterized, and their fragments are ultimately passed to the pixel shader .

Typical applications of geometry shaders include generating point sprites, geometric tessellation , shadow volume extrusion, and single-pass rendering to a cube map . A typical real-world example of the benefits of geometry shaders is automatic mesh complexity adjustment. A series of line strips representing control points for a curve is passed into the geometry shader, and depending on the required complexity, the shader can automatically generate additional lines, each providing a better approximation of the curve.

Starting with OpenGL 4.0 and Direct3D 11, a new class of shaders called tessellation shaders was added. It adds two new shader stages to the traditional model: tessellation control shaders (also known as hull shaders) and tessellation evaluation shaders (also known as domain shaders), which together allow simpler meshes to be subdivided into smaller meshes at runtime according to a mathematical function. The function can be tied to various variables, in particular the distance from the viewing camera, to provide scaling of the active level of detail . This allows objects close to the camera to have finer detail, while more distant objects can have coarser meshes while still appearing comparable in quality. This can also drastically reduce the required mesh bandwidth, allowing meshes to be tessellated directly within the shader units instead of downsampling very complex meshes from memory. Some algorithms can perform upsampling of any arbitrary mesh, while others allow "hints" to be embedded in meshes to indicate the most characteristic vertices and edges.

Around 2017, AMD's Vega microarchitecture introduced support for a new shader stage — primitive shaders — somewhat similar to compute shaders, with access to the data needed for geometry processing.

In 2018, Nvidia introduced mesh and task shaders with the Turing microarchitecture , which are also modeled on compute shaders. Nvidia Turing is the world's first GPU microarchitecture to support mesh shading via the DirectX 12 Ultimate API, several months before the release of the Ampere RTX 30 series.

In 2020, AMD and Nvidia released the RDNA 2 and Ampere microarchitectures, which support mesh shading through DirectX 12 Ultimate . These mesh shaders allow the graphics processor to handle more complex algorithms, shifting much of the workload from the central processor to the graphics processor, and, with intensive rendering algorithms, can increase frame rates or the number of triangles in a scene by an order of magnitude. Intel announced that its Intel Arc Alchemist GPUs, shipping in the first quarter of 2022, would support mesh shaders.

A unified shader is a combination of a 2D shader and a 3D shader.

Compute shaders are not limited to graphics applications, but use the same execution resources for GPGPU . They can be used within graphics pipelines, for example, for additional stages in animation or lighting algorithms (such as tiled forward rendering ). Some rendering APIs allow compute shaders to easily share data resources with the graphics pipeline.

Ray tracing shaders are supported by Microsoft through DirectX Raytracing , the Khronos Group through Vulkan , GLSL, and SPIR-V , and Apple through Metal . In real hardware implementations, a single ray tracing core contains many ray tracing shaders (ray tracing ALUs).

Tensor shaders can be integrated into an NPU or a GPU . Tensor shaders are supported by Microsoft through DirectML , the Khronos Group through OpenVX , Apple through Core ML , Google through TensorFlow , and the Linux Foundation through ONNX . In real hardware implementations, a single tensor core contains many tensor shaders (tensor ALUs).

Shaders are written to apply transformations to a large set of elements at once, for example, to every pixel in a screen area or to every vertex of a model. This lends itself well to parallel processing , and most modern graphics processors have multiple shader pipelines to facilitate this, significantly boosting computational performance.

The shader programming model resembles a higher-order function for rendering, taking shaders as arguments and providing a defined flow of data between intermediate results, enabling both data parallelism (across pixels, vertices, etc.) and pipeline parallelism (across stages). (see also map reduce ).

The language in which shaders are programmed depends on the target environment. The official shading language of OpenGL and OpenGL ES is the OpenGL Shading Language , also known as GLSL, while the official shading language of Direct3D is the High Level Shader Language , also known as HLSL. Cg , a third-party shading language that outputs both OpenGL and Direct3D shaders, was developed by Nvidia ; however, it has been deprecated since 2012. Apple released its own shading language called Metal Shading Language as part of the Metal framework.

Modern video game development platforms , such as Unity , Unreal Engine, and Godot, increasingly include node-based editors that can create shaders without the need for actual code; instead, the user is presented with a directed graph of connected nodes that allows them to route various textures, maps, and mathematical functions into output values such as diffuse color, specular color and intensity, roughness/metallic, height, normal, and so on. Automatic compilation then turns the graph into an actual compiled shader.

• GLSL
• SPIR-V
• HLSL
• Compute kernel
• Shading language
• GPGPU
• List of common shading algorithms
• Vector processor