Graphics

• in Euclidean geometry we use Cartesian coordinates
• in projective geometry we use Homogeneous coordinates
• we can express all affine transforms and projections as one matrix
• transformations can be composed by matrix multiplication
•  for points,  for vectors

• Example: homogeneous matrix  of a rotation matrix  and a position vector
• Homogeneous coordinates -> Cartesian coordinates
• Cartesian coordinates -> Homogeneous coordinates

Rotation about a fixed point

• move  to the origin, rotate, move back
• post-multiply order -> from the right or from bottom to top in the code
• the origin of the object matters

OpenGL example:

• a unit vector that points in the same direction as the camera
• if the dot product between lookAt vector and the normal vector of a polygon is lower than zero, the polygon is facing the camera

• parametric curve defined by a set of control points
• most common - cubic curve, 4 points, 2 points provide directional information

• the simplest type of polygons, always planar
• all GPUs are designed around triangle rasterization

• Constructing a triangle mesh
  • winding order (clockwise, counter-cw)
  • triangle lists, strips and fans
  • mesh instancing - shared data

• Vertex Fetch: driver inserts the command in a GPU-readable encoding inside a pushbuffer
• Poly Morph Engine of SM fetches the vertex data
• Warps of 32 threads are scheduled inside the SM
• Vertex shaders in the warp are executed
• H/D/G shaders are executed (optional step)
• Raster engine generates the pixel information
• data is sent to ROP (Render Output Unit)
• ROP performs depth-testing, blending etc.

Vertex shader phase

• handles transformation from model space to view space
• full access to texture data (height maps)

Tesselation shader phase (optional)

• two shader stages and a fixed-function tessellator between them
• Tesselation Control Shader - determines the amount of tessellation
• Tesselation Evaluation Shader - applies the interpolation

Geometry shader phase (optional)

• operates on entire primitives in homogeneous clip space

Rasterization phase

• Assembly - converts a vertex stream into a sequence of base primitives
• Clipping - transformation of clip space to window-space and chopping off triangles that are outside the frustum
• Culling - discards triangles facing away from the viewer
• Rasterization - generates a sequence of fragments (window-space)

Fragment shader phase

• input: fragment, output: color, depth value, stencil value
• can address texture maps and run per-pixel calculations

Final phase

• Additional culling tests
  • pixel ownership - fails if the pixel is not owned by the API (OpenGL)
  • scissor test - fails if the pixel lies outside of a screen rectangle
  • stencil test - comparing against stencil buffer
  • depth test - comparing against depth buffer
• Color blending
  • combines colors from fragment shader with colors in the color buffers
• writes data to framebuffer
• swaps buffers

• conventional rendering converts each triangle into pixels on a 2D screen
• RayTracing provides realistic lighting by simulating the physical behavior of light
• not possible in real-time until recently
• the light traverses the scene, reflecting from objects, being blocked (shadows), passing through transparent objects (refractions), producing the final color
• APIs: OptiX, DXR, VKRay

Rasterization

Ray Tracing

Texture mapping

• a piece of bitmap that is applied to an object
• may be used as a look-up table for calculations

UV coordinates

• texture coordinates, range from [0, 0] (bottom-left) to [1,1] (top-right)
• uv mapping - projecting a texture onto an object

Texel

• an individual texture element

Texture Mapping

• application of a texel on a 3D model

Types

• diffuse map
• height map
• normal map
• specular map
• ...

Bump mapping

• heightmap is used to generate normals
• simple but not very accurate, heightmap uses only gray colors

Normal mapping

• specifies a surface normal direction vector at each texel
• uses RGB information as a 3D vector to give more accurate bump effect

Displacement mapping

• heightmap is used to adjust vertices
• provides realistic edges but requires a dense mesh

• there is not a clean one-to-one mapping between texels and pixels
• GPU has to sample more than one texel and blend the resulting colors

Mipmapping

• for each texture, we create a sequence of lower-resolution bitmaps
• objects further from the camera will use low-res textures

Nearest neighbor

• the closest texel to the pixel center is selected

Bilinear filtering

• the four texels surrounding the pixel center are sampled, and the resulting color is a weighted average of their colors

Trilinear filtering

• bilinear filtering is used on each of the two nearest mipmap levels

Anisotropic filtering

• samples texels within a region corresponding to the view angle

• used to smooth sharp edges of vertices

FSAA/SSAA (Super-Sampled Antialiasing)

• uses sub-pixel values to average out the final values

DSR (Dynamic Super Resolution)

• scene si rendered into a frame buffer that is larger than the actual screen
• oversized image is downsampled
• the pixel shader is evaluated multiple times per pixel

MSAA (Multisampled Antialiasing)

• comparable to DSR, half of the overhead
• the pixel shader only needs to be evaluated once per pixel

MFAA (Multi-frame sampled Antialiasing)

• sample locations using MSAA across multiple frames

CSAA (Coverage sample Antialiasing)

• NVidia's optimization of MSAA
• new sample type: a sample that represents coverage