Sunday, June 2, 2013

v0.9.0 released with new property animation API

Earlier in the year we announced the introduction of OpenGL-based image processing on the GPU. However, now it is finally available in a release version of the software. Not only that, but we have also added a new, properly integrated property animation API to replace the limited, awkward mlt_geometry API. Here are the features of the new API:
  • The time of a keyframe can be set using frame number, time clock string, or a timecode string.
  • A negative time value makes it relative to the end of an object's duration.
  • Keyframes support 3 forms of interpolation: discrete (no interpolation), linear, and a smooth Catmull-Rom  spline.
  • Supports a new mlt_rect propery type in addition to string (discrete only), integer, and floating point.
  • API for most apps is just a few new, simple methods on the Properties object.
  • Integrated with the OpenGL-based effects only at this time to ease the migration - will be extended to other plugins in the next version or two.
Above is a plot of the curve described by the following property value.
0|=50; 50|=100; 100=200; 200~=60; -7:00~=180; -2:00~=100; -1=220
This shows that it is possible to combine a number of things in the same curve. It starts out with discrete keyframes (|), switches to a linear interpolated keyframe (=), and ends with a few smooth spline keyframes (~) set using timecode relative to the end/width (-).

Here are other highlights of the release:
  • Improved pause behavior when using buffered rendering in mlt_consumer.
  • Added mlt_color type.
  • Deprecated mlt_geometry API.
  • Support for the latest versions of FFmpeg and Libav (but dropping support for 0.5 and 0.6 versions).
  • Added alpha channel output to avformat consumer.
  • Added reconnect and exit_on_disconnect properties to avformat producer.
  • Added qglsl consumer to use opengl with avformat, sdi, and decklink.
  • Added avsync module with blipflash producer and consumer for testing.
  • Added new "count" producer to gtk2 module.
  • Changed frei0r to use index-based property names making it impervious to param name changes (param name still accepted for compatibility). 
  • Added default parameter values to frei0r metadata.
  • Added more python example web services.
  • Started a unit test suite.

Friday, February 22, 2013

MLT Adds GPU Image Processing

OpenGL logo
For a few months, we have been working on a set of image processing filters and transitions that utilize the OpenGL Shader Language (GLSL) of your video card/chip. This means that not only are they very fast but also cross-platform and widely supported. GLSL is generally faster and better supported than OpenCL and NVIDIA CUDA because those technologies are rather new or vendor-specific and designed for more general purpose computing. The new filters are also high quality by using 16-bit linear floating point per color component. All of this means that you can apply more effects without a huge reduction in frame rate.

This work has now been merged into the master code branch and will be available in the next release. They have already been integrated into the Shotcut daily builds for all 3 major OS platforms! However, a caveat - there is not much to see there yet because Shotcut does not yet have filters. :-) Nevertheless, it  has served as a feasibility exercise of the cross-platform integration. Meanwhile, one can open a clip, save as XML, edit the XML to put something like <filter mlt_service="movit.mirror"/> inside of the <producer></producer>, open the XML in Shotcut, and see the result. But first you will need to enable GPU Processing in the Settings menu.

What is currently available?

  • movit.blur
  • movit.convert (colorspace conversion)
  • movit.crop
  • movit.diffusion
  • movit.glow
  • movit.lift_gamma_gain (color correction)
  • movit.mirror
  • movit.opacity
  • movit.rect (position and scale)
  • movit.resample
  • movit.resize (pad)
  • movit.saturation
  • movit.sharpen
  • movit.vignette
  • movit.white_balance
  • movit.mix (transition)
  • movit.overlay (transition)
  • qglsl (a wrapper to multi consumer that uses Qt to abstract platform-specific OpenGL context)
More information can be found in the documentation section of the website.

What still needs to be done?

  • A new threading model to improve concurrency between CPU and GPU
  • More work on qglsl to support more multi-consumer use cases
  • Keyframable parameters throughout


We thank Christophe Thommeret for providing the proof-of-concept which provided many ideas about how to integrate something like this with MLT.
We thank Steinar H. Gunderson for providing the new Movit library, which is an external dependency that provides all of the heavy lifting here.

Sunday, January 20, 2013

version 0.8.8 released

This is purely a bugfix/maintenance release. If you are using a very recent version of FFmpeg or libav that has  switched to using planar audio in its API you will need this.