Thursday, June 2, 2011

html5 and QML 3D video fx!

Andrew Wason has been working on a video effects engine, called WebVfx, that allows effects to be implemented using HTML or QML. It includes producer, filter and transition plugins for MLT.

Video effects can be authored using web technologies like CSS3, Canvas, WebGL etc. Video frames decoded by MLT are available to the effect as JavaScript Image or ImageData objects. QML is supported as an alternative to WebGL for 3D based transitions, by extending it with QtQuick3D.

3D scenes can be modeled in a tool like Blender, the scene can be rendered and baked to a texture and the 3D camera path can be animated and exported using a Blender plugin included with WebVfx. Then video frames can be applied as textures to portions of the scene. Here is the source to a simple 3D example with demo video.

Here is an example of a more sophisticated 3D birthday card theme developed internally, rendered with WebVfx and MLT using the avformat consumer.

Advanced 2D image effects can be implemented using WebGL and a GLSL fragment shader. WebVfx includes a support framework so only the shader code needs to be written. Here is the source to PageCurl and CrossZoom GLSL transitions (with links to demo videos).

Source code and docs are available on github: