3D Game Resources & WebGL

onGameStart 2011 - Brandon Jones

A little bit about me

• Currently a Software Developer for Motorola
• Been playing with WebGL pretty much since the first beta release
• Work with rendering various video game resources in the browser
• Built the glMatrix library
• See my demos at TojiCode.com

Warning! Technical Jargon Ahead!

This presentation assumes a basic knowledge of 3D rendering concepts. If you are new to graphics development and want to learn more about WebGL, check out LearningWebGL.org for some excellent beginners tutorials!

What we'll be covering today

• Looking at some of the demos that I've done in the past
• Examining their file formats
• Talking about what worked about their formats, and what didn't
• Discuss things to consider when building your own WebGL-oriented formats

So where do we start?

The same place we ALWAYS start with any new technology

Get id tech to run on it!

Rendering Quake 3

• First put online a little over a year ago
• Goal was to render the level as accurately as possible, all effects included, at a "gaming" speeds
• Still missing a few visuals, but for the most part all the shaders work!
• Includes some basic movement physics/collision detection

Quake 3 - details

• No server! Everything happens client side.
• All the heavy-duty parsing takes place in a Web Worker
  • Same with movement system
• Using as much of the original formats as possible
• Only thing that was changed was normalizing the textures to power-of-two PNGs
• Shaders are parsed from Q3 material format and compiled into GLSL on the fly

No geometry culling

• Original format broke down geometry into small chunks by area for visibility culling
• At the time, I wasn't able to find a culling method that didn't slow the renderer down
• The cost of the culling logic and extra draw calls actually outweighed the cost of rendering brute-force
• I re-sort the geometry by material on load. Everything that shares a material is drawn in a single call, visible or not.
• This was around Chrome 5/6, things have changed a bit since then...

Things that made life difficult

• Several of the game's resources relied on a "test-and-fallback" system (Textures)
• They also load large datasets but only use small portions. (Materials)
  • I condensed all materials into a single file by hand
• Used charCodeAt() for binary parsing!
  • Typed arrays didn't work in Web Workers at the time. Fortunately that's fixed now.
  • In your own formats either require Typed Array support or just use JSON.
• Shader format is designed to do all the effects as multi-pass. This could be improved when compiling the shaders, but I'm not.

Looking for something more streamlined...

iOS RAGE Level

• Tackled this format because I wanted something that would work well with browser limitations
• More concerned about input and bandwidth limitations than graphics
• The graphics resources themselves are very simple
• Trickiest bit was reverse engineering the format

iOS RAGE Level - details

• Everything shares a single shader: Flat diffuse texturing on every surface.
• It's a testament to the artists and tool-builders at id that this looks so great
• We are doing geometry culling here
• Movement follows a pre-defined path (rail) through the level. Visibility is based off of your position on the rail.

Megatexture "Lite"

• The PC/Console game utilizes a virtualized texturing method called "Megatexture"
• Scene is pre-rendered to gather feedback about which textures are visible and what mip level is needed
• That information is used to build texture atlases on the fly that contain only the textures needed for the current scene
• RAGE iOS still has Megatexturing, but everything is pre-computed
• Each point along the rail references a static list of texture atlases that are built when the level is compiled.

RAGE is an awesome example of everything that WebGL games should strive to be

• It knows it's limitations. Rendering and gameplay are both built with mobile in mind
• Controls are built to embrace the platform, not fight with it.
• Game sessions are quick, gratifying, and self contained. Nobody wants to play a 20 hour epic in their browser. (yet)
• It's designed to allow data to stream in as you go. Reduces up-front load time

Reverse engineering the files

• Came down to a couple of "shot in the dark" assumptions, lots of time spent looking at hex codes, and lots of experimenting
• Spent a lot of time reading interviews about the game as well.
• Starting point: I knew that there had to be some XYZ coords in there somewhere!
• Built a quick WebGL viewer that stepped through the file with a given offset, stride, and length and dumped point clouds to the screen

• Biggest misstep was assuming that positional data would be floating point. (They're actually shorts)
• Once I located the vertex block, found a value near the beginning of the file that corresponded with it's byte offset
• Extrapolated other "lump" offsets from there.
• Used a lot of min/max testing to determine value meaning
  • Three -1.0 to 1.0 values are probably a normal
  • If min is 0 and max numTextures-1 then it's probably a texture index
• Find my full notes on the format on Google Docs

Musings on file formats

When talking about file formats for WebGL, you've got two semi-competing goals:

• Download Fast
• Parse Fast

Downloading files fast

• The Google Body team has done some incredible work with their WebGL-Loader project
• Compresses vertex data into a very compact UTF-8 stream
• Performance is still very good! An approach like this will work for most cases.

But if you need to parse the files really fast...

• With the ability to have XHR requests come back as ArrayBuffers, the door is opened to have super efficient model and level loading if the format was designed correctly.
• The model format can basically be broken down into:
  • A single, large vertex buffer (Binary!)
  • A single, large index buffer (Binary!)
  • A list of meshes, broken up by material and bone groups, that contain offsets into the previous two buffers
  • Materials, skeletal structure, etc.

It's entirely possible that getting the appropriate buffers into your video memory would take nothing more than a couple of subarray calls.

var byteArray = new Uint8Array(binaryBuffer);
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, 
    byteArray.subarray(vertOffset, vertOffset + count), 
    gl.STATIC_DRAW);

var indexBuffer = gl.createBuffer();
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, indexBuffer);
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, 
    byteArray.subarray(indexOffset, indexOffset + count), 
    gl.STATIC_DRAW);
            

For elements of your model that are not simple arrays of numbers, it's probably a lot faster to simply use JSON

{
    materials: [
        water: {
            baseTexture: "texture/waterbase.png",
            shader: "shaders/water.shader"
        },
    ],
    rootBone: {
        pos: [0, 0, 0],
        orient: [0, 0, 0, 1],
        children: [
            // and recurse!
        ]
    },
    // etc.
}
            

Downloading fast and Parsing fast

It may be that a best case scenario is to download a compressed version of your resources, expand them in script, and cache the expanded version with local storage options like the filesystem API.

Performance Tips

I am primarily talking about desktop browsers, mobile is still very early with it's WebGL support. That said, most of these concepts are pretty universal.

Use RequestAnimationFrame!

Every time you do a draw call with setTimeout, somewhere a kitten cries.

Do you hate kittens?

Rendering optimization: No big surprises here

• Everything that applies to desktop graphics still holds true here
• The biggest difference is that it's more important than ever to offload to the GPU (when reasonable)

But at the same time...

• Browser change fast, so it's rarely wise to assume anything about performance. Benchmark, benchmark, benchmark!

Change state as little as possible

• Pack multiple meshes verts into a single buffer
• Avoid multiple passes where possible
• Sort your draw calls by state. (As a pre-process, if you can)
• Try to draw all geometry that shares a material in one call
• Don't break up draw calls based on visibility.
• It's usually cheaper to overdraw a bit in a single call than to make two separate draw calls.
• Don't take my word for it! Benchmark!

Take advantage of spare GPU cycles!

In many scenarios, you won't be GPU limited.* That means that you can get a couple of features "for free".

• If antialiasing is available, use it!
• Don't bother turning off texture filtering.
• Anisotropic filtering isn't available yet, but if it's added crank it up!
• (Benchmark it!)

Pre-compute, pre-compute, precompute!

If you can calculate something "offline", do it! The browser is going to be racing full tilt to keep up with your game logic, it really doesn't need to be calculating your normals for you too.

Many existing formats require lots of processing to get to a displayable point. This can help allow for things like LOD or hardware optimizations.

That stops making sense in a Web-centeric environment. Just send the client the right file type!

Cross platform format support

There will be cases where different platforms will perform best with different formats. Like Mobile vs. Desktop or various texture compression formats.

In these cases, store all the different file permutations on the server! Let the client tell you it's capabilities, and send it the appropriate types based on that.

1 Artist > 1,000 GPU Cores

It's worth considering that all the shader tricks in the world will never yield the results that you'll get from one good artist.

Yes, we all love our fancy water shaders and refractive glass, but a solid art direction can make even flat polygons look spectacular! (See: iOS RAGE)

Things to avoid

There are a couple of things that WebGL makes more painful than usual:

• Uploading large datasets (textures, vertex buffers, etc) to the GPU is expensive, especially on Chrome.
  • Push as much as you can before you start rendering your scene
  • When uploading while rendering, try and limit how much you push in a single frame.
• Bandwidth is a valuable resource!
  • Only make the client download what it must
  • Utilize local cache and filesystem APIs when you can
  • Be considerate of the fact that some of your players will be on capped connections

• Javascript makes it very easy to build inefficient data structures
  • Your data locality can be all over the place with traditional objects
  • Try storing data that you will be looping over in script in a TypedArray. Lookups are much faster.
• Don't go crazy with classes/functions/events/recursion
  • The fastest render procedures are going to be the ones that happen in a single, tight for loop
  • Jumping out to various object's draw methods is going to eat up cycles you don't want to waste
  • If you need to do complex scene-graph tracing, do it outside the main render loop, and preferably in a worker thread.

Thank you!

Q&A

• Blog: TojiCode.com
• Twitter: @Tojiro
• Google+: Brandon Jones (Motorola Mobility)
• Email: tojiro@gmail.com