Understanding the Video WAM extension

A quick overview of how video WAMs work today.

In brief, video WAMs have an additional render API call on the main thread. It uses WebGLTextures as input and output. render is called once per video WAM per frame.

I added a small extra API that I call a WAM extension to the interface between a WAM and it’s host.

All wam extensions are accessible via window.WAMExtensions. If the host implements window.WAMExtensions.video, then when a video WAM plugin loads, it registers itself with the extension by calling window.WAMExtensions.video.setDelegate(pluginId, delegate).

In theory, implementing the delegate for that extension is all a WAM needs to do to support video. In practice, it will probably take a bit of work to get your video code working. Most 3d libraries for the web do not support chaining shader programs on the same GL context out of the box, I had to modify the ISF library and merge a patch to Three.JS to get it working. If you are having trouble please post here and I’ll try to help, although I’m no webgl expert. Take a look at the code here for some guidance around what GL calls need to occur every render call VS initialization:

The ThreeJS and ISF shader WAMs also use a second WAM extension which enables collaborative editing of text documents. If you don’t need that in your video WAM you can ignore the use of collaboration extension, MultiplayerHandler, etc.