Shaders and Multiplayer

Czech technical University in Prague
Faculty of Information Technology
Department of Software Engineering
© Adam Vesecký, MI-APH, 2019

Shaders and filters

Shaders and filters

  • PIXI.Shader
    • helper class for shaders
    • stored in separate files, *.vert for vertex shaders and *.frag for fragment shaders
    • Shader languages support for VS Code - plugin, provides intellisense for shader files
  • PIXI.Filter
    • a special type of WebGL shader that is applied to a screen
    • built-in filters: AlphaFilter, BlurFilter, NoiseFilter, ColorMatrixFilter,...
    • in PixiJS v3, a filter was always applied to a whole screen
    • in PixiJS v4, a filter can be applied only to a part of a screen
    • in PixiJS v5, both approaches are possible
    • documentation link
    • pixi-filters - library with additional filters

Basic vertex shader

#ifdef GL_ES

precision highp float;

#endif

 

// variables from PIXI

attribute vec2 aVertexPosition;

attribute vec2 aTexturePosition;

uniform mat3 translationMatrix;

// variable for fragment shader

varying vec2 vTextureCoord;

 

void main() {

    // map texture coordinate from UV coordinates

    vTextureCoord = aTexturePosition;

    gl_Position = vec4((translationMatrix * vec3(aVertexPosition, 1.0)).xy, 0.0, 1.0);

}

Basic fragment shader

uniform sampler2D texture;

 

varying vec2 vTextureCoord;

 

void main(void){

   gl_FragColor = texture2D(texture, vTextureCoord);

}

Variables

  • uniform - per-primitive parameters (constant during an entire draw-call)
  • attribute - per-vertex parameters (positions, normals, colors, UVs,...)
  • varying - per-fragment (per-pixel) parameters - are interpolated
  • Complete GLSL specification

Shader setup

  1. load assets, including vertex and fragment shader
  2. define PIXI.Geometry with two triangles and texture coordinates on which the shaders will be executed
  3. define uniform variables (float, integer, vec2, vec3, mat3, mat4,...)
  4. create shader via PIXI.Shader.from(vertex, fragment, uniforms)
  5. create PIXI.Mesh with defined geometry and shaders
  6. modify uniform variables in the game loop as you like

Basic Example

  • src/labs/lab06/example-shader-basic.ts

Noise example

  • src/labs/lab06/example-shader-noise.ts
  • two input attributes - current time for animation and the resolution of the screen

Noise example

  • fragment shader that calculates output color for each coordinate

    • void main(){

        // get current pixel coordinate

        vec2 st = gl_FragCoord.xy/u_resolution.xy;

        // put aspect-ration into consideration

        st.y *= u_resolution.y/u_resolution.x;

     

        float t = u_time*0.5;

        // calculate noise

        float df = 1.0;

        df = mix(hex(st,t),hex(st,t+1.),fract(t));

        df += snoise(vec3(st*75.,t*0.1))*0.03;

        gl_FragColor = vec4(mix(vec3(0.),vec3(1.),step(0.7,df)),1.0);

    }

Wave example

  • src/labs/lab06/example-shader-wave.ts

Mandelbrot set example

  • src/labs/lab06/example-shader-mandelbrot.ts
  • palette texture used as a look-up table

Easing functions

Easing functions

  • : current time
  • : start value
  • : change in value
  • : duration
  • Linear tween:
  • Quadratic ease-in:
  • Quadratic ease-out:
  • Cubic ease-in:
  • Sine ease-in:
  • Sine ease-out:
  • Sine ease in/out:
  • Exponential ease-in:

Tweening example

  • src/labs/lab06/example-tween.ts

    • Linear

    Ease in-out

    Sine-in

Tweening example

  • src/labs/lab06/example-tween2.ts

    • Linear

    Quadratic

    Exponential

Multiplayer

Multiplayer

  • skins, virtual items, chat server, virtual shops and custom assets are usually HTTP-based
  • streaming of the virtual world is managed by UDP protocol
  • continuous values (those that change very often) don't need to be confirmed
  • commands and events that have an impact on the game model need to be confirmed
  • multiplayer libraries provide a hybrid architecture based on UDP that simulates TCP for the sake of reliability - not every message is considered reliable
  • issues to deal with
    • compressed game state - entropy encoding
    • lower frequency of updates - interpolation
    • security - server is the main runner
    • latency - extrapolation, prediction, rewind
    • unreliable connection - queue of to-be-confirmed messages

Network emulator

  • libs/network-emulator
  • a simple multiplayer library that emulates UDP
  • supports message confirmation, streaming, interpolation

Messaging

  • everything needs to be serialized into a byte stream
  • it doesn't matter how do we store our variables on target platform. What matters is their range (a variable that has 4 possible values can be stored in 2bits instead of a whole byte)
  • Javascript's number is 64bit long
  • floats are serialized by a simple "multiply-divide hack"

Example: Custom message serialization

class NetworkExampleMessage extends Net.NetData {

  positionX: number;

  positionY: number;

  rotation: number;

 

  loadFromStream(reader: Net.NetReader) {

    this.positionX = reader.readFloat();

    this.positionY = reader.readFloat();

    this.rotation = reader.readFloat();

  }

 

  saveToStream(writer: Net.NetWriter) {

    writer.writeFloat(this.positionX);

    writer.writeFloat(this.positionY);

    writer.writeFloat(this.rotation);

  }

 

  getDataLength() {

    return 8 + 8 + 8;

  }

}

Synchronization Example

  • src/labs/lab06/example-network-basic.ts
  • open the console to see the logs
  • press Q/W to change streaming frequency
  • press R to reset the rotation
  • press S to pause the streaming

Client

Server

Synchronization example

  • src/labs/lab06/example-network-steering.ts
  • notice the "teleport" issue when server changes values instantly

Client

Server

Packet loss

  • for linear movement, packet loss is not a problem
  • the client extrapolates the value if there are no future samples

Packet loss

  • packet loss for more complex movement results in a weird speed-up due to a wrong prediction

Synchronization Example

  • src/labs/lab06/example-network-anim.ts

Client (lag = 0)

Client (lag = 500 ms)

Server

Synchronization Example

  • src/labs/lab06/example-network-anim.ts

Client 25% packet loss

Client 500ms lag + 75% packet loss

Server

Synchronization of new objects

  • the client creates a new object
  • the server confirms it
  • both the client and the server edit an existing object
  • the server doesn't accept a change from the client

Synchronization of new objects

  • the server creates a new object, the packet gets lost
  • the server updates an existing object, the packet arrives
  • the client can't edit an object that doesn't exist -> the message lingers in the queue
  • the server re-sends old messages that haven't been confirmed yet
  • when the first message arrives, the client applies all other messages in the queue

Object creation example

  • src/labs/lab06/example-network-commands.ts
  • only the server can create/remove objects
  • messages are confirmed by the client but the model is not consistent - why?

Client

Server

Interactive example

  • src/labs/lab06/example-network-interactive.ts
  • both the client and the server can create new objects
  • click on the screen to add new objects

Client

Server