Buffered attribute landed for HTML5 video

Last night I landed support for the buffered HTML5 video attribute in Firefox. This is cool because we can now accurately determine which time-segments of a video we can play and seek into without needing to pause playback to download more data. Previously you could only get the byte position the download had reached, which often doesn't map to the time ranges which are playable very well, especially in a variable bit rate video. This also can't tell you if there are chunks which we skipped downloading before the downloaded byte position. Once the video controls UI is updated, users will be able to know exactly which segments of their video are downloaded and playable and can be seeked into without pausing playback to download more data.

To see this in action, download last night's Firefox nightly build or later, and point your browser at my  video buffered attribute demo. You'll see something like the screenshot below, including an extra progress bar (implemented using canvas) showing the time ranges which are buffered.

 

I've implemented the buffered attribute for the Ogg and WAV backends. Support for the buffered attribute for WebM is being worked on by Matthew Gregan, and is well underway. At the moment we return empty ranges for the buffered attribute on video elements playing WebM and raw video.

My checkin just missed the cutoff for Firefox 4 Beta 3, so the first beta release that the video buffered attribute will appear in is Firefox 4 Beta 4.

Easy way to create WebM videos

If you're looking to create some WebM videos to try out the new WebM support in Firefox 4 Beta 1, an easy tool to use is Miro Video Converter. Because WebM is such a new format, existing video editing software doesn't encode in WebM yet, so if you want to make a WebM video, you must create it in another format and then convert it to WebM using a tool like Miro Video Converter.

Miro Video Converter provides a very simple interface for ffmpeg, one of the leading open source video encoding tools. Miro Video Converter configures ffmpeg to output converted video to the same quality as the input, and it uses the excellent xiph.org libvorbis audio encoder, rather than ffmpeg's own broken Vorbis encoder (if you hear poor quality audio on YouTube WebM videos, it's probably because they're still using ffmpeg's vorbis encoder).

Miro Video Converter is freely available for download on Windows and Mac, and can convert to other formats besides WebM.

WebM video support in Firefox 4 Beta

The new Firefox 4 Beta 1 includes support for HTML5 video elements playing WebM videos. This is exciting, as much of the industry is getting behind WebM. Opera is shipping WebM support in Opera 10.6, Google Chrome's "early access release channel" builds include WebM support. Microsoft said they'd support WebM in their HTML5 video implementation in their upcoming IE9, provided appropriate codecs are installed on the user's system. Adobe has announced that they'll support WebM playback in Flash, which will provide fallback playback of WebM in any older or otherwise non WebM supporting browsers. Intel says it will move towards hardware support for WebM once it becomes popular.

Google has freely licensed the VP8 video codec used in WebM, and provided a royalty free patent grant. This is great news for the future of the internet. We now have a royalty free video codec, with quality which is competitive with proprietary alternatives. This means anyone can freely use high quality internet video, without having to worry about getting sued or having to negotiate a patent license.

All this will hopefully contribute to increased adoption of WebM and HTML5 video, coupling all the power of the modern web browser's rendering pipeline with high quality video.

If you want to try out our new WebM support in Firefox, the easiest way is to download the Firefox 4 beta, and watch WebM videos on YouTube's "HTML5 Experiment" program.

Improved WebM performance on x86_64 Linux and Win64

Last week we landed patches from Makoto Kato to enable optimized assembly in libvpx on Win64. We also landed a similar patch from Tim Terriberry to enable optimized assembly in libvpx on Linux x86_64. This will improve decoding performance of WebM videos on these platforms. We're still being hit by scaling performance, particularly on high resolution and "full-screen" YouTube HTML5 experiment WebM videos. We're aware of this, and are working to improve it.

WebM has landed on Firefox nightlies

Today I landed Firefox's WebM support on mozilla-central, our Firefox development branch. It should appear in nightly builds from tonight onwards.

Credits to the development effort go to Chris Double for writing the decoder backend, Matthew Gregan for writing the ISC style licensed libnestegg WebM demuxer, and myself for integrating all the moving parts into the Mozilla tree and build system. Also thanks need to go to the releng team which promptly installed YASM on the Mozilla Linux and Mac build machines, and to Roc, Shaver and many others for cracking the whip and working behind the scenes.

Firefox should build with WebM support without needing any extra changes to your build configuration, unless you're building on Win32, where you'll need to have MASM installed in order to compile libvpx's optimized assembly. MASM ships with the Windows 7 SDK, and with Visual Studio Pro. If you've got neither of those installed, you can also download MASM directly.

If you're building on Linux x86, Mac x86 or Mac x86_64 and you've got YASM installed, you'll automatically build VP8 decoder's optimized assembly code from libvpx. If you don't have YASM, you'll fallback to using the generic C code, which won't be as fast, but still performs acceptably. We don't have WebM support building on Win64 yet, you can disable if you reconfigure with --disable-webm.

If you're looking for some WebM videos to test with, you can view WebM videos on YouTube's HTML5 Experiement with nightly builds from tonight onwards.

A new unwitting ally in the fight against random orange

I have discovered a new tool in our toolbox in the fight against random test failures: crashinjectdll.dll on Windows, nptest.so on Linux (I don't know what it's called on MacOS...). I suspect we're randomly injecting crashes into plugins, in order to test recovery from plugin crashes with out-of-process plugins (correct me if I'm wrong). If a mochitest run times out on Tinderbox, and a crash has been injected, when the mochitest run is killed we'll get call stacks in the output log for all the threads that were running. This can give us some details of the state of Firefox when the timeout occurred, and may give us a clue as to why we've timed out.

Maybe we should make always inject a plugin crash at the start of a mochitest run, so that we always have call stacks when we kill a timed-out mochitest run?

Next time someone comments in your random orange bugs with a log, check to see if it's got call stacks in it!

New Ogg video decoder for Firefox

Today I finally landed the new Ogg video decoder for Firefox/Mozilla. It should appear in Firefox nightlies from 2 April 2010 onwards. Chris Double did the initial implementation for the new Ogg decoder, and I took over in December, implemented seeking, and bashed it into shape.

This replaces our old Ogg decoder which used liboggplay. We'll now be able to implement new features easier, and have greater control over our decode pipeline. We can now control our audio and video decode separately, and this allows us to do things like not decoding video interframes if the video decode is taking too much time and causing the audio decode to fall behind. This prevents audio stutter when decoding large-frame-sized videos. The new backend also has more reliable seeking, and can even load a few weirdly formed Ogg files that the previous backend couldn't handle.

It was a large patch, and it's a testament to our robust set of video and audio unit tests that when I checked it into mozilla-central, there was basically only one small fix required. It has been a long grind getting the new decoder to work robustly, but our unit tests have been an invaluable, if fickle, task-mistress.


The new backend should at least be feature-equivalent to the old backend. If you spot a regression in Ogg video playback in new nightly builds, please file a bug in Core in the "Video/Audio" component!