3D Graphics

Native Client applications use the OpenGL ES 2.0 API for 3D rendering. This document describes how to call the OpenGL ES 2.0 interface in a Native Client module and how to build an efficient rendering loop. It also explains how to validate GPU drivers and test for specific GPU capabilities, and provides tips to help ensure your rendering code runs efficiently.

Validating the client graphics platform

Native Client is a software technology that lets you code an application once and run it on multiple platforms without worrying about the implementation details on every possible target platform. It’s difficult to provide the same support at the hardware level. Graphics hardware comes from many different manufacturers and is controlled by drivers of varying quality. A particular GPU driver may not support every OpenGL ES 2.0 feature, and some drivers are known to have vulnerabilities that can be exploited.

Even if the GPU driver is safe to use, your program should perform a validation check before you launch your application to ensure that the driver supports all the features you need.

Vetting the driver in JavaScript

At startup, the application should perform a few additional tests that can be implemented in JavaScript on its hosting web page. The script that performs these tests should be included before the module’s embed tag, and ideally the embed tag should appear on the hosting page only if these tests succeed.

The first thing to check is whether you can create a graphics context. If you can, use the context to confirm the existence of any required OpenGL ES 2.0 extensions. You may want to refer to the extension registry and include vendor prefixes when checking for extensions.

Vetting the driver in Native Client

Create a context

Once you’ve passed the JavaScript validation tests, it’s safe to add a Native Client embed tag to the hosting web page and load the module. As part of the module initialization code, you must create a graphics context for the app by either creating a C++ Graphics3D object or calling PPB_Graphics3D API function Create. Don’t assume this will always succeed; you still might have problems creating the context. If you are in development mode and can’t create the context, try creating a simpler version to see if you’re asking for an unsupported feature or exceeding a driver resource limit. Your production code should always check that the context was created and fail gracefully if that’s not the case.

Check for extensions and capabilities

Not every GPU supports every extension or has the same amount of texture units, vertex attributes, etc. On startup, call glGetString(GL_EXTENSIONS) and check for the extensions and the features you need. For example:

• If you are using non power-of-2 texture with mipmaps, make sure GL_OES_texture_npot exists.
• If you are using floating point textures, make sure GL_OES_texture_float exists.
• If you are using DXT1, DXT3, or DXT5 textures, make sure the corresponding extensions GL_ANGLE_texture_compression_dxt1, GL_ANGLE_texture_compression_dxt3, and GL_ANGLE_texture_compression_dxt5 exist.
• If you are using the functions glDrawArraysInstancedANGLE, glDrawElementsInstancedANGLE, glVertexAttribDivisorANGLE, or the PPAPI interface PPB_OpenGLES2InstancedArrays, make sure the corresponding extension GL_ANGLE_instanced_arrays exists.
• If you are using the function glRenderbufferStorageMultisampleEXT, or the PPAPI interface PPB_OpenGLES2FramebufferMultisample, make sure the corresponding extension GL_CHROMIUM_framebuffer_multisample exists.
• If you are using the functions glGenQueriesEXT, glDeleteQueriesEXT, glIsQueryEXT, glBeginQueryEXT, glEndQueryEXT, glGetQueryivEXT, glGetQueryObjectuivEXT, or the PPAPI interface PPB_OpenGLES2Query, make sure the corresponding extension GL_EXT_occlusion_query_boolean exists.
• If you are using the functions glMapBufferSubDataCHROMIUM, glUnmapBufferSubDataCHROMIUM, glMapTexSubImage2DCHROMIUM, glUnmapTexSubImage2DCHROMIUM, or the PPAPI interface PPB_OpenGLES2ChromiumMapSub, make sure the corresponding extension GL_CHROMIUM_map_sub exists.

Check for system capabilites with glGetIntegerv and adjust shader programs as well as texture and vertex data accordingly:

• If you are using textures in vertex shaders, make sure glGetIntegerv(GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS, ...) and glGetIntegerv(GL_MAX_TEXTURE_SIZE, ...) return values greater than 0.
• If you are using more than 8 textures in a single shader, make sure glGetIntegerv(GL_MAX_TEXTURE_IMAGE_UNITS, ...) returns a value greater than or equal to the number of simultaneous textures you need.

Vetting the driver in the Chrome Web Store

If you choose to place your application in the Chrome Web Store, its Web Store manifest file can include the webgl feature in the requirements parameter. It looks like this:

"requirements": {
  "3D": {
    "features": ["webgl"]
  }
}

While WebGL is technically a JavaScript API, specifying the webgl feature also works for OpenGL ES 2.0 because both interfaces use the same driver.

This manifest item is not required, but if you include it, the Chrome Web Store will prevent a user from installing the application if the browser is running on a machine that does not support OpenGL ES 2.0 or that is using a known blacklisted GPU driver that could invite an attack.

If the Web Store determines that the user’s driver is deficient, the app won’t appear on the store’s tile display. However, it will appear in store search results or if the user links to it directly, in which case the user could still download it. But the manifest requirements will be checked when the user reaches the install page, and if there is a problem, the browser will display the message “This application is not supported on this computer. Installation has been disabled.”

The manifest-based check applies only to downloads directly from the Chrome Web Store. It is not performed when an application is loaded via inline installation.

What to do when there are problems

Using the vetting procedure described above, you should be able to detect the most common problems before your application runs. If there are problems, your code should describe the issue as clearly as possible. That’s easy if there is a missing feature. Failure to create a graphics context is tougher to diagnose. At the very least, you can suggest that the user try to update the driver. You might want to linke to the Chrome page that describes how to do updates.

If a user can’t update the driver, or their problem persists, be sure to gather information about their graphics environment. Ask for the contents of the Chrome about:gpu page.

Document unreliable drivers

It can be helpful to include information about known dubious drivers in your user documentation. This might help identify if a rogue driver is the cause of a problem. There are many sources of GPU driver blacklists. Two such lists can be found at the Chromium project and Khronos. You can use these lists to include information in your documentation that warns users about dangerous drivers.

Test your defenses

You can test your driver validation code by running Chrome with the following flags (all at once) and watching how your application responds:

• --disable-webgl
• --disable-pepper-3d
• --disable_multisampling
• --disable-accelerated-compositing
• --disable-accelerated-2d-canvas

Calling OpenGL ES 2.0 commands

There are three ways to write OpenGL ES 2.0 calls in Native Client.

Use “pure” OpenGL ES 2.0 function calls

You can make OpenGL ES 2.0 calls through a Pepper extension library. The SDK example examples/api/graphics_3d works this way. In the file graphics_3d.cc, the key initialization steps are as follows:

• Add these includes at the top of the file:

  #include <GLES2/gl2.h>
  #include "ppapi/lib/gl/gles2/gl2ext_ppapi.h"
  

• Define the function InitGL. The exact specification of attrib_list will be application specific.

  bool InitGL(int32_t new_width, int32_t new_height) {
    if (!glInitializePPAPI(pp::Module::Get()->get_browser_interface())) {
      fprintf(stderr, "Unable to initialize GL PPAPI!\n");
      return false;
    }
  
    const int32_t attrib_list[] = {
      PP_GRAPHICS3DATTRIB_ALPHA_SIZE, 8,
      PP_GRAPHICS3DATTRIB_DEPTH_SIZE, 24,
      PP_GRAPHICS3DATTRIB_WIDTH, new_width,
      PP_GRAPHICS3DATTRIB_HEIGHT, new_height,
      PP_GRAPHICS3DATTRIB_NONE
    };
  
    context_ = pp::Graphics3D(this, attrib_list);
    if (!BindGraphics(context_)) {
      fprintf(stderr, "Unable to bind 3d context!\n");
      context_ = pp::Graphics3D();
      glSetCurrentContextPPAPI(0);
      return false;
    }
  
    glSetCurrentContextPPAPI(context_.pp_resource());
    return true;
  }
  

• Include logic in Instance::DidChangeView to call InitGL whenever necessary: upon application launch (when the graphics context is NULL) and whenever the module’s View changes size.

Use Regal

If you are porting an OpenGL ES 2.0 application, or are comfortable writing in OpenGL ES 2.0, you should stick with the Pepper APIs or pure OpenGL ES 2.0 calls described above. If you are porting an application that uses features not in OpenGL ES 2.0, consider using Regal. Regal is an open source library that supports many versions of OpenGL. Regal recently added support for Native Client. Regal forwards most OpenGL calls directly to the underlying graphics library, but it can also emulate other calls that are not included (when hardware support exists). See libregal for more info.

Use the Pepper API

Your code can call the Pepper PPB_OpenGLES2 API directly, as with any Pepper interface. When you write in this way, each invocation of an OpenGL ES 2.0 function must begin with a reference to the Pepper interface, and the first argument is the graphics context. To invoke the function glCompileShader, your code might look like:

ppb_g3d_interface->CompileShader(graphicsContext, shader);

This approach specifically targets the Pepper APIs. Each call corresponds to a OpenGL ES 2.0 function, but the syntax is unique to Native Client, so the source file is not portable.

Implementing a rendering loop

Graphics applications require a continuous frame render-and-redraw cycle that runs at a high frequency. To achieve the best frame rate, is important to understand how the OpenGL ES 2.0 code in a Native Client module interacts with Chrome.

The Chrome and Native Client processes

Chrome is a multi-process browser. Each Chrome tab is a separate process that is running an application with its own main thread (we’ll call it the Chrome main thread). When an application launches a Native Client module, the module runs in a new, separate sandboxed process. The module’s process has its own main thread (the Native Client thread). The Chrome and Native Client processes communicate with each other using Pepper API calls on their main threads.

When the Chrome main thread calls the Native Client thread (keyboard and mouse callbacks, for example), the Chrome main thread will block. This means that lengthy operations on the Native Client thread can steal cycles from Chrome, and performing blocking operations on the Native Client thread can bring your app to a standstill.

Native Client uses callback functions to synchronize the main threads of the two processes. Only certain Pepper functions use callbacks; SwapBuffers is one.

SwapBuffers and its callback function

SwapBuffers is non-blocking; it is called from the Native Client thread and returns immediately. When SwapBuffers is called, it runs asynchronously on the Chrome main thread. It switches the graphics data buffers, handles any needed compositing operations, and redraws the screen. When the screen update is complete, the callback function that was included as one of SwapBuffer‘s arguments will be called from the Chrome thread and executed on the Native Client thread.

To create a rendering loop, your Native Client module should include a function that does the rendering work and then executes SwapBuffers, passing itself as the SwapBuffer callback. If your rendering code is efficient and runs quickly, this scheme will achieve the highest frame rate possible. The documentation for SwapBuffers explains why this is optimal: because the callback is executed only when the plugin’s current state is actually on the screen, this function provides a way to rate-limit animations. By waiting until the image is on the screen before painting the next frame, you can ensure you’re not generating updates faster than the screen can be updated.

The following diagram illustrates the interaction between the Chrome and Native Client processes. The application-specific rendering code runs in the function called Draw on the Native Client thread. Blue down-arrows are blocking calls from the main thread to Native Client, green up-arrows are non-blocking SwapBuffers calls from Native Client to the main thread. All OpenGL ES 2.0 calls are made from Draw in the Native Client thread.

SDK example graphics_3d

The SDK example graphics_3d uses the function MainLoop (in hello_world.cc) to create a rendering loop as described above. MainLoop calls Render to do the rendering work, and then invokes SwapBuffers, passing itself as the callback.

void MainLoop(void* foo, int bar) {
  if (g_LoadCnt == 3) {
    InitProgram();
    g_LoadCnt++;
  }
  if (g_LoadCnt > 3) {
    Render();
    PP_CompletionCallback cc = PP_MakeCompletionCallback(MainLoop, 0);
    ppb_g3d_interface->SwapBuffers(g_context, cc);
  } else {
    PP_CompletionCallback cc = PP_MakeCompletionCallback(MainLoop, 0);
    ppb_core_interface->CallOnMainThread(0, cc, 0);
  }
}

Managing the OpenGL ES 2.0 pipeline

OpenGL ES 2.0 commands do not run in the Chrome or Native Client processes. They are passed into a FIFO queue in shared memory which is best understood as a GPU command buffer. The command buffer is shared by a dedicated GPU process. By using a separate GPU process, Chrome implements another layer of runtime security, vetting all OpenGL ES 2.0 commands and their arguments before they are sent on to the GPU. Buffering commands through the FIFO also speeds up your code, since each OpenGL ES 2.0 call in your Native Client thread returns immediately, while the processing may be delayed as the GPU works down the commands queued up in the FIFO.

Before the screen is updated, all the intervening OpenGL ES 2.0 commands must be processed by the GPU. Programmers often try to ensure this by using the glFlush and glFinish commands in their rendering code. In the case of Native Client this is usually unnecessary. The SwapBuffers command does an implicit flush, and the Chrome team is continually tweaking the GPU code to consume the OpenGL ES 2.0 FIFO as fast as possible.

Sometimes a 3D application can write to the FIFO in a way that’s difficult to handle. The command pipeline may fill up and your code will have to wait for the GPU to flush the FIFO. If this is the case, you may be able to add glFlush calls to speed up the flow of the OpenGL ES 2.0 command FIFO. Before you start to add your own flushes, first try to determine if pipeline saturation is really the problem by monitoring the rendering time per frame and looking for irregular spikes that do not consistently fall on the same OpenGL ES 2.0 call. If you’re convinced the pipeline needs to be accelerated, insert glFlush calls in your code before starting blocks of processing that do not generate OpenGL ES 2.0 commands. For example, issue a flush before you begin any multithreaded particle work, so that the command buffer will be clear when you start doing OpenGL ES 2.0 calls again. Determining where and how often to call glFlush can be tricky, you will need to experiment to find the sweet spot.

Rendering and inactive tabs

Users will often switch between tabs in a multi-tab browser. A well-behaved application that’s performing 3D rendering should pause any real-time processing and yield cycles to other processes when its tab becomes inactive.

In Chrome, an inactive tab will continue to execute timed functions (such as setInterval and setTimeout) but the timer interval will be automatically overridden and limited to not less than one second while the tab is inactive. In addition, any callback associated with a SwapBuffers call will not be sent until the tab is active again. You may receive asynchronous callbacks from functions other than SwapBuffers while a tab is inactive. Depending on the design of your application, you might choose to handle them as they arrive, or to queue them in a buffer and process them when the tab becomes active.

The time that passes while a tab is inactive can be considerable. If your main thread pulse is based on the SwapBuffers callback, your app won’t update while a tab is inactive. A Native Client module should be able to detect and respond to the state of the tab in which it’s running. For example, when a tab becomes inactive, you can set an atomic flag in the Native Client thread that will skip the 3D rendering and SwapBuffers calls and continue to call the main thread every 30 msec or so. This provides time to update features that should still run in the background, like audio. It may also be helpful to call sched_yield or usleep on any worker threads to release resources and cede cycles to the OS.

Handling tab activation from the main thread

You can detect and respond to the activation or deactivation of a tab with JavaScript on your hosting page. Add an EventListener for visibilitychange that sends a message to the Native Client module, as in this example:

document.addEventListener('visibilitychange', function(){
  if (document.hidden) {
    // PostMessage to your Native Client module
    document.nacl_module.postMessage('INACTIVE');
  } else {
    // PostMessage to your Native Client module
    document.nacl_module.postMessage('ACTIVE');
  }

}, false);

Handling tab activation from the Native Client thread

You can also detect and respond to the activation or deactivation of a tab directly from your Native Client module by including code in the function pp::Instance::DidChangeView, which is called whenever a change in the module’s view occurs. The code can call ppb::View::IsPageVisible to determine if the page is visible or not. The most common cause of invisible pages is that the page is in a background tab.

Tips and best practices

Here are some suggestions for writing safe code and getting the maximum performance with the Pepper 3D API.

Do’s

• Make sure to enable attrib 0. OpenGL requires that you enable attrib 0, but OpenGL ES 2.0 does not. For example, you can define a vertex shader with 2 attributes, numbered like this:

  glBindAttribLocation(program, "positions", 1);
  glBindAttribLocation(program, "normals", 2);
  

  In this case the shader is not using attrib 0 and Chrome may have to perform some additional work if it is emulating OpenGL ES 2.0 on top of OpenGL. It’s always more efficient to enable attrib 0, even if you do not use it.

• Check how shaders compile. Shaders can compile differently on different systems, which can result in glGetAttrib* functions returning different results. Be sure that the vertex attribute indices match the corresponding name each time you recompile a shader.
• Update indices sparingly. For security reasons, all indices must be validated. If you change indices, Native Client will validate them again. Structure your code so indices are not updated often.
• Use a smaller plugin and let CSS scale it. If you’re running into fillrate issues, it may be beneficial to perform scaling via CSS. The size your plugin renders is determined by the width and height attributes of the <embed> element for the module. The actual size displayed on the web page is controlled by the CSS styles applied to the element.
• Avoid matrix-to-matrix conversions. With some versions of Mac OS, there is a driver problem when compiling shaders. If you get compiler errors for matrix transforms, avoid matrix-to-matrix conversions. For instance, upres a vec3 to a vec4 before transforming it by a mat4, rather than converting the mat4 to a mat3.

Don’ts

• Don’t use client side buffers. OpenGL ES 2.0 can use client side data with glVertexAttribPointer and glDrawElements, but this is really slow. Try to avoid client side buffers. Use Vertex Buffer Objects (VBOs) instead.
• Don’t mix vertex data and index data. By default, Pepper 3D binds buffers to a single point. You could create a buffer and bind it to both GL_ARRAY_BUFFER and GL_ELEMENT_ARRAY_BUFFER, but that would be expensive overhead and it is not recommended.
• Don’t call glGet* or glCheck* during rendering. This is normal advice for OpenGL programs, but is particularly important for 3D on Chrome. Calls to any OpenGL ES 2.0 function whose name begins with these strings blocks the Native Client thread. This includes glGetError; avoid calling it in release builds.
• Don’t use fixed point (GL_FIXED) vertex attributes. Fixed point attributes are not supported in OpenGL ES 2.0, so emulating them in OpenGL ES 2.0 is slow. By default, GL_FIXED support is turned off in the Pepper 3D API.
• Don’t read data from the GPU. Don’t call glReadPixels, as it is slow.
• Don’t update a small portion of a large buffer. In the current OpenGL ES 2.0 implementation when you update a portion of a buffer (with glSubBufferData for example) the entire buffer must be reprocessed. To avoid this problem, keep static and dynamic data in different buffers.
• Don’t call glDisable(GL_TEXTURE_2D). This is an OpenGL ES 2.0 error. Each time it is called, an error messages will appear in Chrome’s about:gpu tab.