HTML5 video playbackRate and Ogg chaining support landed in Firefox

Paul Adenot has recently landed patches in Firefox to enable the playbackRate attribute on HTML5 <audio> and <video> elements.

This is a cool feature that I've been looking forward to for a while; it means users can speed up playback of videos (and audio) so that you can for example watch presentations sped up and only slow down for the interesting bits. Currently Firefox's <video> controls don't have support for playbackRate, but it is accessible from JavaScript, and hopefully we'll get support added to our built-in controls soon.

Paul has also finally landed support for Ogg chaining. This has been strongly desired for quite some time by community members, and the final patch also had contributions from "oneman" (David Richards), who also was a strong advocate for this feature.

We decided to reduce the scope of our chaining implementation in order to make it easier and quicker to implement. We targeted the features most desired by internet radio providers, and so we only support chaining in Ogg Vorbis and Opus audio files and we disable seeking in chained files.

Thanks Paul and David for working on these features!

Changes to DOM full-screen API in Firefox 11

We've made some changes to how the HTML full-screen API exits full-screen mode in Firefox 11, which is scheduled to ship in March 2012. Previously Document.mozCancelFullScreen() would fully-exit full-screen and return the browser to "normal" mode. Starting in Firefox 11, Document.mozCancelFullScreen() will restore full-screen state to the element that was previously full-screen. If there is no previous full-screen element in either the document or a parent document (full-screen mode isn't restored to former full-screen elements in child documents), then the browser will "fully-exit full-screen", and return the browser to normal mode.

To see how this is useful, consider the case of a PowerPoint clone or presentation web app that wants to run full-screen. One way to implement such a web app would be to have a full-screen <div> element where the slides are shown. The developer may want to be able to switch full-screen mode seamlessly between the slide deck <div> and (say) a <video>, and then return to having the slide deck <div> as the full-screen element so that the user can carry on with the presentation. Before this change, if the <video> was in a cross-origin subdocument (like a YouTube embedded player in an <iframe>) returning full-screen mode to the slide deck <div> from the <video> was a two-step process; users would have to fully-exit full-screen, and re-request full-screen mode on the slide deck element. Now developers can simply call Document.mozCancelFullScreen() and seamlessly switch back. The browser won't drop out of full-screen mode during the transition.

Note that if users press the escape key they will always fully-exit full-screen, i.e. Firefox won't restore the previous full-screen element to full-screen state on escape key press. So to seamlessly restore full-screen to the previous full-screen element, developers must explicitly call Document.mozCancelFullScreen(), they can't rely on the user pressing the escape key.

We've also added webconsole logging upon full-screen request failures to Firefox 11, to make debugging denied full-screen requests easier.

Another change coming in Firefox 11 is we'll no longer deny full-screen requests in web pages which contain windowed plugins. Now we'll exit full-screen when a windowed plugin is focused instead (on Windows and Linux, MacOSX is unaffected).

Firefox's HTML full-screen API enabled in Nightly builds

A few days ago I enabled the HTML full-screen API in Firefox nightly builds. This enables developers to make an arbitrary HTML element "full-screen", hiding the browser's UI and stretching the element to encompass the entire screen. This will be particularly useful for HTML5 video and games.

If all goes well, this feature will ship in Firefox 10 at the end of January.

The API has changed slightly since I last blogged about it. The current API is Mozilla-specific, but is similar to the W3C's Fullscreen draft specification.

To enter full-screen mode, call the following method on the HTML Element you'd like to enter full-screen:

• void mozRequestFullScreen() : posts an asynchronous request to make the HTML element the full-screen element. If the request is granted, some time later a bubbling "mozfullscreenchange" event is dispatched to the element which requested full-screen. If the request is denied, a "mozfullscreenerror" event is dispatched to the element's owning document. We only grant requests for full-screen when:
• mozRequestFullScreen() is called in a user-generated event handler, e.g. a mouse click handler, and
• the requesting element is in its document, and
• there are no windowed plugins present in any document/iframe in the current page, and
• all iframes containing the requesting element (if any) have the mozallowfullscreen attribute.

We added the following method and attributes to HTML Document:

• void mozCancelFullScreen() : exits the document from full-screen mode. This dispatches a "mozfullscreenchange" event to the document containing the (now former) full-screen element. Note that the "mozfullscreenchange" event which is dispatched when you enter full-screen is targeted at the full-screen element, so if you want to receive the "mozfullscreenchange" on both entering and exiting full-screen in the same listener you should add your listener to the document, rather than the full-screen element.
• readonly attribute boolean mozFullScreen : true when the document is in full-screen mode.
• readonly attribute Element mozFullScreenElement : reference to the current full-screen element.
• readonly attribute boolean mozFullScreenEnabled : returns true if calls to mozRequestFullScreen() would be granted in the current document. This returns false if there are any windowed plugins present in any document/iframe in the current page, or if any iframes containing this document don't have the mozallowfullscreen attribute present, or if the user has disabled the API by preference. If this returns false you may want to not show the user your enter-full-screen button in your page, since you know it won't work!

We also added the :-moz-full-screen css pseudo class, which applies to the full-screen element while in full-screen mode.

We added the mozallowfullscreen attribute to iframe elements. Without this, full-screen requests made by script in the iframe's content (i.e embedded ads, or a YouTube player in an iframe for that matter) will be denied.

While in full-screen mode, the user can press the ESC key (or F11) to exit. Alpha-numeric keyboard input while in full-screen mode causes a warning message to pop-up to guard against phishing attacks. The only key input which doesn't cause the warning message to pop up are: left, right, up, down, space, shift, control, alt, page up, page down, end, home, tab, and meta.

Navigating, changing tab, changing app (ALT+TAB) while in full-screen mode will cause full-screen mode to exit.

Here's about a simple example, which will work in current Firefox nightly builds:

(Press ESC to exit full-screen)

The code for that button's onclick handler is simply:
document.getElementById('bruce_video').mozRequestFullScreen();

How is Firefox's full-screen API different from Webkit/Chrome/Safari's full-screen API? Firefox's API adds a "width: 100%; height: 100%;" CSS rule to the element which requests full-screen, so that it's stretched to occupy the entire screen. Chrome's API does not do this, but instead it centers the full-screen element in the window and blacks-out the underlying webpage. So the full-screen element won't occupy the entire screen with Chrome's API unless you specify a "width: 100%; height: 100%;" rule yourself. Conversely if you want to vertically and horizontally center something while in full-screen with Firefox's API, you need to make the containing element of your desired centered element full-screen instead, and apply CSS rules to vertically and horizontally center the contained element.

For a cross-browser full-screen API example, see html5-demos.appspot.com's full-screen demo.

Edit: 11 Nov 2011, clarified Document.mozCancelFullScreen() and Document.mozFullScreenEnabled, fixed typos.

Mozilla full-screen API progress update

Update 10 November 2011: the full-screen API has been changed slightly and enabled in Firefox Nightly builds, see http://blog.pearce.org.nz/2011/11/firefoxs-html-full-screen-api-enabled.html for details.

I've been working on implementing Robert O'Callahan's HTML full-screen API proposal in Firefox (bug 545812). Support for the base API has landed, disabled by default, in Firefox nightly builds. To enable the full-screen API, set the pref full-screen-api.enabled to true.

We have implemented a general purpose full-screen API which can make any HTML element the full-screen element (it seems WebKit based browsers' full-screen API allow only making <video> elements full-screen).

This feature makes the following API changes to HTML Element:

1. void mozRequestFullScreen() : makes an HTML element the full-screen element. Causes browser chrome to hide, and expands the element to encompass the entire screen. Upon success, this dispatches a "mozfullscreenchange" event to the requesting full-screen element, or the element's owner document if the element is not in a document. We only grant requests for full-screen when running in user-generated event handlers, e.g. a mouse click handler.

This feature makes the following API changes to HTML Document:

1. void mozCancelFullScreen() : exits the document from full-screen mode.
2. readonly attribute mozFullScreen : true when the document is in full-screen mode.
3. readonly attribute mozFullScreenElement : reference to the current full-screen element, if it's in the current document.

This feature adds the :-moz-full-screen css pseudo class, which applies to the full-screen element while in full-screen mode.

For a request for full-screen to be granted in content inside an iframe, the containing iframe needs to have the mozallowfullscreen attribute present. This is a boolean attribute, so the attribute only needs to be present, it doesn't matter what value it's set to.

Keyboard input is restricted in full-screen mode. When alpha-numeric key input occurs in full-screen mode, full-screen mode immediately exits. This is to help protect against phishing attacks.

We also plan to deny requests for full-screen mode when windowed plugins are present (since we can't easily monitor key events to windowed plugins on non-MacOSX platforms). We will exit full-screen mode when a windowed plugin is added to a document as well. I have a patch for this, but its dependencies haven't landed yet.

Work remaining to be done before this can be enabled:

1. Adding a warning message when we enter DOM full-screen mode (on desktop Firefox, and on Fennec too).
2. Making the full-screen API work in multi-process Firefox/Fennec (bug 684620). This requires a way of getting the PBrowserParent from C++ in the chrome process to be implemented, there's not a way to do that yet unfortunately.
3. Make change/open tab cause full-screen mode to exit (bug 685402).
4. A security review must be completed, and concerns raised there must be addressed. This could involve changing the API.

We also want a clearer transition effect when entering full-screen, to somehow show the full-screen element "stretching out" to encompass the screen.

You can test out our work-in-progress full-screen implementation, by grabbing the latest Firefox nightly build, setting the pref full-screen-api.enabled to true, and pointing your browser at my not-very-exciting full-screen API demo page.

New media element APIs and better media seeking resolution

French intern Paul Adenot has recently implemented the seekable and played attributes on the HTML5 video and audio elements in Firefox. The seekable attribute enables script to see what regions of the media can be seeked into (particularly handy with live streams), and the played attribute enables script to see what regions of the media has already been played. Paul has also done some work improving the built in controls on media elements. Thanks for your hard work Paul! These should be available in release builds in November (Firefox 8).

Also in Firefox 8 are my changes to media seeking resolution. Now media seeking should be accurate to the nearest microsecond. It's been reported elsewhere how important accurate seeking for video is. We were previously accurate to the nearest video frame, but we could still be up to one audio packet off (often between 4 and 8 ms out). Now we prune audio samples when seeking so we're down to microsecond resolution.

Simple rate limited HTTP server for testing HTML5 media/streaming

While working on the Firefox HTML5 video and audio support, I've found it extremely useful to have an HTTP server on which the transfer rate is reliably limited. Existing servers are either too heavy weight, like apache, or have inconsistent rate-limiting, like lighttpd which I found to have very "bursty" rate limiting.

I ended up taking the educational route, and implementing a simple HTTP server in C++. It supports the following features:

1. Support for HTTP1.1 Byte Range Requests. This means you can seek into unbuffered data when watching HTML5 video.
2. Rate limiting, configurable on a per request basis by passing the "rate=x" HTTP query parameter, where x is the transfer rate of the connection in kilobytes per second. The server will send x/10 KB ten times per second to maintain this rate smoothly.
3. Simulated live streaming, configurable on a request basis by passing the "live" query parameter. When in "live" mode, no Content-Length header is sent, and the server doesn't advertise or perform byte range requests - so you can't seek into unbuffered video/audio, just like in a live stream.
4. Cross platform; tested on Windows (runs on port 80) and Linux (runs on port 8080). I haven't test it on MacOS yet.
5. Simply serves all files in the program's working directory, making it easy to use (and abuse).
6. Open source! Get the code at https://github.com/cpearce/HttpMediaServer, or download a pre-built win32 binary.

For example, if you wanted to simulate a live stream being served at 100KB/s, your test URL might look something like http://localhost:80/video.ogg?rate=100&live.

I've been using it for quite a while, and over the weekend I finally cleaned it up and put it up on GitHub. Check it out.

HTML5 Video painting performance statistics in Firefox 5

I've landed video frame paint performance counters for HTML5 video onto mozilla-central. This should ship in Firefox 5, barring any disasters. This work was a combined effort by Chris Double and I. These are Mozilla specific fields which will only be available in Firefox.

The new statistics enable us to measure the performance of the video decoding and frame painting pipeline.

This adds the following fields to the HTMLVideoElement:

• mozParsedFrames - A count of the number of video frames that have been demuxed/parsed from the media resource. If we were playing perfectly, we'd be able to paint this many frames.
• mozDecodedFrames - A count of the number of deumxed/parsed video frames that have been decoded into Images. We skip decoding of parsed/demuxed frames if the decode is falling behind the playback position (this can happen if it takes a long time to decode a keyframe for example).
• mozPresentedFrames - A count of the number of decoded frames that have been presented to the rendering pipeline for painting (set as the current Image on the video element's ImageContainer). We may not present decoded frames if the frame arrives for presentation late.
• mozPaintedFrames - A count of the number of presented frames which were painted on screen. We may end up not painting presented frames if another frame is presented before the graphics pipeline has time to paint the previously presented frame, or if the video is off screen. 
• mozFrameDelay - The time (as a floating point number in seconds) which the last painted video frame was rendered late by. This is the time duration between the decoder saying "paint frame X now", and the graphics pipeline physically getting frame X displayed on the screen. The value is accurate on desktop Firefox, but not on mobile. Improvements in the graphics pipeline, and the integration with the graphics pipeline, will show up as a decrease in this number.

Here's a demo of the video paint statistics in Firefox 5. You'll need a recent Firefox trunk nightly build for the demo to work.

Firefox 4 video decoder architecture

To assist others coming up to speed on the architecture of the video decoder, I've put together a diagram of Firefox 4's video playback engine. We rewrote our video architecture for Firefox 4 in order to give us better control over the complete stack.

Click on the image for a larger diagram.

The key classes in our architecture are:

• nsHTMLMediaElement - This manages the JavaScript/HTML accessible HTMLMediaElement interface, and implements the resource selection, load, and preload logic.
• nsBuiltinDecoder - Manages a main thread accessible snapshot of the state of the underlying decoder. The decoders run on non-main threads, and we don't want to block the main thread to dig into the decoders when JS queries playback state, so we maintain a snapshot of the playback engine's state in this class. This inherits from nsMediaDecoder. You can also implement playback support for a new format by inheriting and implementing nsMediaDecoder. nsWaveDecoder is currently implemented this way, but we're in the process of reimplementing that as a sublcass of nsBuiltinDecoderReader.
• nsBuiltinDecoderStateMachine - Manages the decode, state machine, audio-push threads, frame queueing, A/V sync, and buffering logic. This ensures that all the HTML5 events get dispatched at the appropriate time, and that behaviour is consistent and sane across different media types. Demuxing is handled abstractly by subclasses of nsBuiltinDecoderReader. This way all media types can share as much playback logic as possible, reducing our maintenance overhead.
• nsOggReader/nsWebMReader - Demuxing and codec specific functionality is implemented by subclassing nsBuiltinDecoderReader. This reduces the amount of work required to implement and maintain support for new codecs. When a new codec is implemented as a nsBuiltinDecoderReader subclass, support for HTML events, buffering, and playback logic does not need to be reimplemented, since it already exists in nsBuiltinDecoderStateMachine. To add support for a new codec, it's easiest to implement support as a new nsBuiltinDecoderReader subclass.
• nsAudioStream - Our cross platform audio API wrapper. It is based on libsydneyaudio, which operates on a push model rather than a (more commonly used) callback-based model, which has brought in a whole raft of headaches. Matthew Gregan is in the process of rewriting our audio layer to a more sane callback based model. We also provide a cross-process nsAudioStreamRemote, which proxies audio commands to an audio stream in another process. This is required on mobile.
• ImageContainer - When it comes time to present a video frame, nsBuiltinDecoderStateMachine sets it as the "current image" of the video element's ImageContainer object. This then propagates through the Layers/2D scene rendering system, and it eventually gets rendered on the screen. The Layers compositing runs on the main thread, and ImageContainer provides a thread-safe wrapper. The images contained in the ImageContainer can be in OpenGL/D3D surfaces, so we can take advantage of hardware accelerated scaling, rendering, and YCbCr to RGB conversion.
• nsVideoFrame - This resides in layout, and manages the dimensions/reflow of the video, as well as its poster image.
• nsMediaStream - Our network code runs on the main thread, but the underlying libraries we use for media decoding (libvpx, libtheora, etc) assume synchronous reads. We can't afford to do blocking reads on the main thread, so we cache the media data downloaded into the nsMediaCache, and provide a thread-safe wrapper synchronous wrapper for reading in the nsMediaStream class. We use Necko for our networking, so we can take advantage of all the existing security and load-group functionality it implements.

The advantage of controlling the entire playback engine are many. We can easily control frame dropping, memory allocation, the threading model, what, when, and how we decode, and we can integrate more tightly with our network stack.

Google to remove H.264 support from Chrome

Google is removing H.264 support from Chrome in the coming months. This is good news, as it means we're closer to having a high quality patent unencumbered video format for use in HTML5 video that works across all browsers. Now we're just left with Safari, Apple's iDevices, and IE9 as natively supporting patent encumbered video formats. Microsoft has said that IE9 will play "VP8 video when the user has installed a VP8 codec on Windows", it will be interesting to see how that pans out.

How to stop a video or audio element downloading

Say you've started playback of an HTML5 audio or video element, and you decide you really want to cancel playback and downloading of the media resource. Stopping playback is easy, just call the pause() method. But the network connection won't be stopped until the media element gets garbage collected, and even if you release all references to the media element, it won't be destroyed until the browser decides to runs its garbage collector. How can you stop the download of the media resource in the meantime? Here's a quick hack to achieve this: just reset the element's src attribute to the empty string. This destroys the element's internal decoder, and stops the network download.

For example:

<audio id="audio" controls>

  <source src="funky-music.ogg">

  <source src="funky-music.mp3">

</audio>
<!-- ... Some time later, we decide we should stop the audio element playing and downloading... -->
<script>
var audio = document.getElementById("audio");

audio.pause();

audio.src = ""; // Stops audio download.
audio.load(); // Initiate a new load, required in Firefox 3.x.
</script>


Be aware that this will destroy the media element's decoders, so the element won't be playable anymore, and it will be rendered as an "error cross" if it's in a document. Also in Firefox 3.x you need to call load() after changing the source, whereas in Firefox 4 the load is scheduled to run when you change the src attribute, and the extra load() call is not required.

Changes to HTML5 video/audio load() function in Firefox 4

I've updated the media load() implementation in Firefox 4 to match the current WHATWG media load algorithm specification. There are three main changes that web developers using media elements should be aware of.

Firstly, error reporting has slightly changed. When a media element fails to load from its <source> children, an "error" event is dispatched to every child element which failed to load. Previously in Firefox 3.x you'd receive only one "error" event dispatched to the media element once all of its child <source> elements had failed to load. Now you only receive "error" events in the child <source> elements, and not in the media element itself.

For example, suppose you have the following markup:

<video>

<source id="mp4_src"

        src="video.mp4"

        type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</source>

<source id="3gp_src"
        src="video.3gp"
        type='video/3gpp; codecs="mp4v.20.8, samr"'>
</source>

<source id="ogg_src"
        src="video.ogv"
        type='video/ogg; codecs="theora, vorbis"'>
</source>

</video>

Firefox 4 does not support the playback of patent encumbered content, so you'll receive "error" events in the <source> elements with the MP4 and 3GP resources, before the Ogg resource is loaded. Note also that the <source> children are loaded in the order in which they appear in the markup, and if one <source> child successfully loads and is playable, the children after it won't be loaded.

To detect that all child <source> elements have failed to load, check the value of the networkState attribute of the media element; if its value is HTMLMediaElement.NETWORK_NO_SOURCE, you know all child <source> elements have failed to load.

If you add another child <source> element to a media element which is in networkState HTMLMediaElement.NETWORK_NO_SOURCE, it will attempt to load the resource specified by the newly added <source>.

Secondly, when the load begins has changed. When you set the src attribute of a media element, or add a <source> child element to a media element, a load will be scheduled to run asynchronously as soon as the current JavaScript context exits (basically the load starts the next time we return to the browser application's main event loop). So for example, suppose you have some timeout set such as:

<script>

setTimeout(
  function() {

    var v = document.createElement("video");

    v.src = "video.ogg";
    document.appendChild(v)

    // Do some other stuff...
  },
  1000);

</script>

When the function runs, the load for the video element won't begin until after the function returns, and control returns to the browser. This is important, because in Firefox 3.x, the load is started (and could even run to completion!) as soon as you set the media element's src attribute.

Lastly, the media element's events now no longer bubble. Previously they bubbled, and this was a bug in Firefox 3.x and in violation of the spec.

HTML5 Video preload attribute supported in Firefox 4, autobuffer attribute removed

Late last week I landed support on Firefox trunk for the HTML5 video 'preload' attribute. This replaces the 'autobuffer' attribute, which we previously supported. If you were previously using the autobuffer attribute on your HTLM5 videos, you need to update to using the preload attribute as well.

The preload attribute provides a hint to the browser as to how much downloading is sensible for a given media. The preload attribute can have three values:

1. "none" - suggests to the browser that it doesn't need to load this media at all until the user plays the resource. The browser will delay any network traffic required to load the media until the users tries to play the resource, or explicitly loads the resource. I suggest using this preload value, along with the poster attribute, when it's unlikely that the user will play the resource. This is probably most useful in a mobile environment, where data can be expensive.
2. "metadata" - suggests to the browser that it isn't necessary to load the entire resource in advance. The browser will suspend the load after loading metadata, displaying the first video frame (if there's no poster image), and ensuring it can play the media. This is the default behaviour, and prevents excess network traffic when the web developer isn't certain the video will definitely be played. If you don't specify a preload value, Firefox will automatically do this. This was also the default behaviour in Firefox 3.5 and 3.6 in the absence of the autobuffer attribute. This default behaviour is a suitable compromise between bandwidth saving and user convenience.
3. "auto" - suggests to the browser that it should load as much of the resource as possible. As long as the browser's own media cache isn't full, it will keep on downloading. I suggest this is most useful in the "YouTube" case, when you've got a media which the user is almost certainly going to watch, and so having the user download the media is not likely to be wasting server bandwidth. This behaviour is the same as using the autobuffer attribute in Firefox 3.5 and 3.6

Since the autobuffer attribute is no longer present in Firefox 4, and the preload attribute is not present in Firefox 3.5 and 3.6, if you want a media to download completely, you should include both preload="auto" and autobuffer in the video element, e.g.:

<video autobuffer preload="auto" src="video.ogg"></video>

If you want the video to only be downloaded if the user actually plays the video, you should not include either the preload or the autobuffer attribute. The default behaviour in Firefox 3.5, 3.6, and the upcoming Firefox 4 is to only load up to the first frame and then suspend the download.

Keyframe indexed Ogg files supported in Firefox 4

Seeking in Ogg files served over the internet has historically been slow. This is because traditional Ogg files do not contain a keyframe index, so you've got no idea where in the media you need to begin decoding from in order to playback from any given time, particularly in variable bitrate media. Seeking typically has to be implemented as a bisection search over the media, which can be slow when performed over a high-latency network such as the internet.

I've been working with the people at the Xiph foundation to develop a keyframe index for Ogg files to alleviate this problem, and I'm delighted to announce that today I finally landed support for indexed Ogg files in Firefox trunk. The keyframe index format I developed has been included in the newly minted Ogg Skeleton 4.0 metadata track.

How much of an impact does seeking with an index make? Download a current Firefox 4 nightly build, and point it at my keyframe indexed Ogg seek demo, and see for yourself. It speeds up seeking dramatically.

If you're using ffmpeg2theora version 0.27 or later to encode your Ogg media, you'll automatically get a Skeleton 4.0 track with keyframe indexes when you encode your media. If you have existing Ogg media which you want add keyframe indexes to, you can use my OggIndex command line tool to add indexes to your Ogg media. The author of ffmpeg2theora, J^, is kindly building and hosting nightly builds of OggIndex, and the OggIndex source code is also available via git under a BSD license.

Indexed Oggs are also supported in GStreamer trunk thanks to the work of Sebastian Dröge.

I've been working on this sporadically for almost a year now, so it's great to finally get this landed!

Buffered attribute landed for HTML5 video

Last night I landed support for the buffered HTML5 video attribute in Firefox. This is cool because we can now accurately determine which time-segments of a video we can play and seek into without needing to pause playback to download more data. Previously you could only get the byte position the download had reached, which often doesn't map to the time ranges which are playable very well, especially in a variable bit rate video. This also can't tell you if there are chunks which we skipped downloading before the downloaded byte position. Once the video controls UI is updated, users will be able to know exactly which segments of their video are downloaded and playable and can be seeked into without pausing playback to download more data.

To see this in action, download last night's Firefox nightly build or later, and point your browser at my  video buffered attribute demo. You'll see something like the screenshot below, including an extra progress bar (implemented using canvas) showing the time ranges which are buffered.

 

I've implemented the buffered attribute for the Ogg and WAV backends. Support for the buffered attribute for WebM is being worked on by Matthew Gregan, and is well underway. At the moment we return empty ranges for the buffered attribute on video elements playing WebM and raw video.

My checkin just missed the cutoff for Firefox 4 Beta 3, so the first beta release that the video buffered attribute will appear in is Firefox 4 Beta 4.

Easy way to create WebM videos

If you're looking to create some WebM videos to try out the new WebM support in Firefox 4 Beta 1, an easy tool to use is Miro Video Converter. Because WebM is such a new format, existing video editing software doesn't encode in WebM yet, so if you want to make a WebM video, you must create it in another format and then convert it to WebM using a tool like Miro Video Converter.

Miro Video Converter provides a very simple interface for ffmpeg, one of the leading open source video encoding tools. Miro Video Converter configures ffmpeg to output converted video to the same quality as the input, and it uses the excellent xiph.org libvorbis audio encoder, rather than ffmpeg's own broken Vorbis encoder (if you hear poor quality audio on YouTube WebM videos, it's probably because they're still using ffmpeg's vorbis encoder).

Miro Video Converter is freely available for download on Windows and Mac, and can convert to other formats besides WebM.

WebM video support in Firefox 4 Beta

The new Firefox 4 Beta 1 includes support for HTML5 video elements playing WebM videos. This is exciting, as much of the industry is getting behind WebM. Opera is shipping WebM support in Opera 10.6, Google Chrome's "early access release channel" builds include WebM support. Microsoft said they'd support WebM in their HTML5 video implementation in their upcoming IE9, provided appropriate codecs are installed on the user's system. Adobe has announced that they'll support WebM playback in Flash, which will provide fallback playback of WebM in any older or otherwise non WebM supporting browsers. Intel says it will move towards hardware support for WebM once it becomes popular.

Google has freely licensed the VP8 video codec used in WebM, and provided a royalty free patent grant. This is great news for the future of the internet. We now have a royalty free video codec, with quality which is competitive with proprietary alternatives. This means anyone can freely use high quality internet video, without having to worry about getting sued or having to negotiate a patent license.

All this will hopefully contribute to increased adoption of WebM and HTML5 video, coupling all the power of the modern web browser's rendering pipeline with high quality video.

If you want to try out our new WebM support in Firefox, the easiest way is to download the Firefox 4 beta, and watch WebM videos on YouTube's "HTML5 Experiment" program.

Improved WebM performance on x86_64 Linux and Win64

Last week we landed patches from Makoto Kato to enable optimized assembly in libvpx on Win64. We also landed a similar patch from Tim Terriberry to enable optimized assembly in libvpx on Linux x86_64. This will improve decoding performance of WebM videos on these platforms. We're still being hit by scaling performance, particularly on high resolution and "full-screen" YouTube HTML5 experiment WebM videos. We're aware of this, and are working to improve it.

WebM has landed on Firefox nightlies

Today I landed Firefox's WebM support on mozilla-central, our Firefox development branch. It should appear in nightly builds from tonight onwards.

Credits to the development effort go to Chris Double for writing the decoder backend, Matthew Gregan for writing the ISC style licensed libnestegg WebM demuxer, and myself for integrating all the moving parts into the Mozilla tree and build system. Also thanks need to go to the releng team which promptly installed YASM on the Mozilla Linux and Mac build machines, and to Roc, Shaver and many others for cracking the whip and working behind the scenes.

Firefox should build with WebM support without needing any extra changes to your build configuration, unless you're building on Win32, where you'll need to have MASM installed in order to compile libvpx's optimized assembly. MASM ships with the Windows 7 SDK, and with Visual Studio Pro. If you've got neither of those installed, you can also download MASM directly.

If you're building on Linux x86, Mac x86 or Mac x86_64 and you've got YASM installed, you'll automatically build VP8 decoder's optimized assembly code from libvpx. If you don't have YASM, you'll fallback to using the generic C code, which won't be as fast, but still performs acceptably. We don't have WebM support building on Win64 yet, you can disable if you reconfigure with --disable-webm.

If you're looking for some WebM videos to test with, you can view WebM videos on YouTube's HTML5 Experiement with nightly builds from tonight onwards.

New Ogg video decoder for Firefox

Today I finally landed the new Ogg video decoder for Firefox/Mozilla. It should appear in Firefox nightlies from 2 April 2010 onwards. Chris Double did the initial implementation for the new Ogg decoder, and I took over in December, implemented seeking, and bashed it into shape.

This replaces our old Ogg decoder which used liboggplay. We'll now be able to implement new features easier, and have greater control over our decode pipeline. We can now control our audio and video decode separately, and this allows us to do things like not decoding video interframes if the video decode is taking too much time and causing the audio decode to fall behind. This prevents audio stutter when decoding large-frame-sized videos. The new backend also has more reliable seeking, and can even load a few weirdly formed Ogg files that the previous backend couldn't handle.

It was a large patch, and it's a testament to our robust set of video and audio unit tests that when I checked it into mozilla-central, there was basically only one small fix required. It has been a long grind getting the new decoder to work robustly, but our unit tests have been an invaluable, if fickle, task-mistress.


The new backend should at least be feature-equivalent to the old backend. If you spot a regression in Ogg video playback in new nightly builds, please file a bug in Core in the "Video/Audio" component!

Don't serve Ogg media with gzip/deflate compression

One common way to reduce the load on a web server is to use gzip or deflate compression when serving to a supporting web browser. When Firefox requests an Ogg media, it advertises that it can handle a gzipped or deflated response; the HTTP request includes the Accept-Encoding: gzip,deflate header. But despite Firefox advertising that it supports gzip/deflate, you probably don't want your web server to gzip or deflate Ogg media. If you serve an Ogg media compressed, Firefox won't be able to seek in the media, or determine its duration. Since the video/audio data in Ogg files is already compressed, gzip/deflate won't actually save you much bandwidth anyway, so you probably want to disable compression when serving Ogg files.

I have added this tip to the Mozilla Developer Center article on configuring servers for Ogg media.