Firefox's Gecko Media Plugin & EME Architecture

For rendering audio and video Firefox typically uses either the operating system's audio/video codecs or bundled software codec libraries, but for DRM video playback (like Netflix, Amazon Prime Video, and the like) and WebRTC video calls using baseline H.264 video, Firefox relies on Gecko Media Plugins, or GMPs for short.

This blog post describes the architecture of the Gecko Media Plugin system in Firefox, and the major class/objects involved, as it looked in June 2019.

For DRM video Firefox relies upon Google's Widevine Content Decryption Module, a dynamic shared library downloaded at runtime. Although this plugin doesn't conform to the GMP ABI, we provide an adapter to allow it to be run through the GMP system. We use the same Widevine CDM plugin that Chrome uses.

For decode and encode of H.264 streams for WebRTC, Firefox uses OpenH264, which is provided by Cisco. This plugin implements the GMP ABI.

These two plugins are downloaded at runtime from Google's and Cisco's servers, and installed in the user's Firefox profile directory.

We also ship a ClearKey CDM, which is the baseline decryption scheme required by the Encrypted Media Extensions specification. This mimics interface which the Widevine CDM implements, and is used in our EME regression tests. It's bundled with the rest of Firefox, and lives in the Firefox install directory.

The objects involved in running GMPs are spread over three processes; the main (AKA parent) process, the sandboxed content process where we run JavaScript and load web pages, and the sandboxed GMP process, which only runs GMPs.

You can view a Diagram of Firefox's Gecko Media Plugin online, or download a PDF version of Firefox's Gecko Media Plugin architecture.

The main facade to the GMP system is the GeckoMediaPluginService. Clients use the GeckoMediaPluginService to instantiate IPDL actors connecting their client to the GMP process, and to configure the service. In general, most operations which involve IPC to the GMPs/CDMs should happen on the GMP thread, as the GMP related protocols are processed on that thread.

mozIGeckoMediaPluginService can be used on the main thread by JavaScript, but the main-thread accessible methods proxy their work to the GMP thread.

How GMPs are downloaded and installed

The Firefox front end code which manages GMPs is the GMPProvider. This is a JavaScript object, running in the front end code in the main process. On startup if any existing GMPs are already downloaded and installed, this calls mozIGeckoMediaPluginService.addPluginDir() with the path to the GMP's location on disk. Gecko's C++ code then knows about the GMP. The GeckoMediaPluginService then parses the metadata file in that GMP's directory, and creates and stores a GMPParent for that plugin. At this stage the GMPParent is like a template, which stores the metadata describing how to start a plugin of this type. When we come to instantiate a plugin, we'll clone the template GMPParent into a new instance, and load a child process to run the plugin using the cloned GMPParent.

Shortly after the browser starts up (usually within 60 seconds), the GMPProvider will decide whether it should check for new GMP updates. The GMPProvider will check for updates if either it has not checked in the past 24 hours, or if the browser has been updated since last time it checked. If the GMPProvider decides to check for updates, it will poll Mozilla's Addons Update Server. This will return an update.xml file which lists the current GMPs for that particular Firefox version/platform, and the URLs from which to download those plugins. The plugins are hosted by third parties (Cisco and Google), not on Mozilla's servers. Mozilla only hosts the manifest describing where to download them from.

If the GMPs in the update.xml file are different to what is installed, Firefox will update its GMPs to match the update.xml file from AUS. Firefox will download and verify the new GMP, uninstall the old GMP, install the new GMP, and then add the new GMP's path to the mozIGeckoMediaPluginService. The objects that do this are the GMPDownloader and the GMPInstallManager, which are JavaScript modules in the front end code as well.

Note Firefox will take action to ensure its installed GMPs matches whatever is specified in the update.xml file. So if a version of a GMP which is older than what is installed is specified in the update.xml file, Firefox will uninstall the newer version, and download and install the older version. This is to allow a GMP update to be rolled back if a problem is detected with the newer GMP version.

If the AUS server can't be contacted, and no GMPs are installed, Firefox has the URLs of GMPs baked in, and will use those URLs to download the GMPs.

On startup, the GMPProvider also calls mozIGeckoMediaPluginService.addPluginDir() for the ClearKey CDM, passing in its path in the Firefox install directory.

How EME plugins are started in Firefox

The lifecycle for Widevine and ClearKey CDM begins in the content process with content JavaScript calling Navigator.requestMediaKeySystemAccess(). Script passes in a set of MediaKeySystemConfig, and these are passed forward to the MediaKeySystemAccessManager. The MediaKeySystemAccessManager figures out a supported configuration, and if it finds one, returns a MediaKeySystemAccess from which content JavaScript can instantiate a MediaKeys object. 

Once script calls MediaKeySystemAccess.createMediaKeys(), we begin the process of instantiating the plugin. We create a MediaKeys object and a ChromiumCDMProxy object, and call Init() on the proxy. The initialization is asynchronous, so we return a promise to content JavaScript and on success we'll resolve the promise with the MediaKeys instance which can talk to the CDM in the GMP process.

To create a new CDM, ChromiumCDMProxy::Init() calls GeckoMediaPluginService::GetCDM(). This runs in the content process, but since the content process is sandboxed, we can't create a new child process to run the CDM there and then. As we're in the content process, the GeckoMediaPluginService instance we're talking to is a GeckoMediaPluginServiceChild. This calls over to the parent process to retrieve a GMPContentParent bridge. GMPContentParent acts like the GMPParent in the content process. GeckoMediaPluginServiceChild::GetContentParent() retrieves the bridge, and sends a LaunchGMPForNodeId() message to instantiate the plugin in the parent process.

In the non multi-process Firefox case, we still call GeckoMediaPluginService::GetContentParent(), but we end up running GeckoMediaPluginServiceParent::GetContentParent(), which can just instantiate the plugin directly.

When the parent process receives a LaunchGMPForNodeId() message, the GMPServiceParent runs through its list of GMPParents to see if there's one matching the parameters passed over. We check to see if there's an instance from the same NodeId, and if so use that. The NodeId is a hash of the origin requesting the plugin, combined with the top level browsing origin, plus salt. This ensures GMPs from different origins always end up running in different processes, and GMPs running in the same origin run in the same process.

If we don't find an active GMPParent running the requested NodeId, we'll make a copy of a GMPParent matching the parameters, and call LoadProcess() on the new instance. This creates a GMPProcessParent object, which in turn uses GeckoChildProcessHost to run a command line to start the child GMP process. The command line passed to the newly spawned child process causes the GMPProcessChild to run, which creates and initializes the GMPChild, setting up the IPC connection between GMP and Main processes.

The GMPChild delegates most of the business of loading the GMP to the GMPLoader. The GMPLoader opens the plugin library from disk, and starts the Sandbox using the SandboxStarter, which has a different implementation for every platform. Once the sandbox is started, the GMPLoader uses a GMPAdapter parameter to adapt whatever binary interface the plugin exports (the Widevine C API for example) to the match the GMP API. We use the adapter to call into the plugin to instantiate an instance of the CDM. For OpenH264 we simply use a PassThroughAdapter, since the plugin implements the GMP API.

If all that succeeded, we'll send a message reporting success to the parent process, which in turn reports success to the content process, which resolves the JavaScript promise returned by MediaKeySystemAccess.createMediaKeys() with the MediaKeys object, which is now setup to talk to a CDM instance.

Once content JavaScript has a MediaKeys object, it can set it on an HTMLMediaElement using HTMLMediaElement.setMediaKeys().

The MediaKeys object encapsulates the ChromiumCDMProxy, which proxies commands sent to the CDM into calls to ChromiumCDMParent on the GMP thread.

How EME playback works

There are two main cases that we care about here; encrypted content being encountered before a MediaKeys is set on the HTMLMediaElement, or after. Note that the CDM is only usable to the media pipeline once it's been associated with a media element by script calling HTMLMediaElement.setMediaKeys().

If we detect encrypted media streams in the MediaFormatReader's pipeline, and we don't have a CDMProxy, the pipeline will move into a "waiting for keys" state, and not resume playback until content JS has set a MediaKeys on the HTMLMediaElement. Setting a MediaKeys on the HTMLMediaElement causes the encapsulated ChromiumCDMProxy to bubble down past MediaDecoder, through the layers until it ends up on the MediaFormatReader, and the EMEDecoderModule.

Once we've got a CDMProxy pushed down to the MediaFormatReader level, we can use the PDMFactory to create a decoder which can process encrypted samples. The PDMFactory will use the EMEDecoderModule to create the EME MediaDataDecoders, which process the encrypted samples.

The EME MediaDataDecoders talk directly to the ChromiumCDMParent, which they get from the ChromiumCDMProxy on initialization. The ChromiumCDMParent is the IPDL parent actor for communicating with CDMs.

All calls to the ChromiumCDMParent should be made on the GMP thread. Indeed, one of the primary jobs of the ChromiumCDMProxy is to proxy calls made by the MediaKeys on the main thread to the GMP thread so that commands can be sent to the CDM via off main thread IPC.

Any callbacks from the CDM in the GMP process are made onto the ChromiumCDMChild object, and they're sent via PChromiumCDM IPC over to ChromiumCDMParent in the content process. If they're bound for the main thread (i.e. the MediaKeys or MediaKeySession objects), the ChromiumCDMCallbackProxy ensures they're proxied to the main thread.

Before the EME MediaDataDecoders submit samples to the CDM, they first ensure that the samples have a key with which to decrypt the samples. This is achieved by a SamplesWaitingForKey object. We keep a copy in the content process of what keyIds the CDM has reported are usable in the CDMCaps object. The information stored in the CDMCaps about which keys are usable is mirrored in the JavaScript exposed MediaKeySystemStatusMap object.

The MediaDataDecoder's decode operation is asynchronous, and the SamplesWaitingForKey object delays decode operations until the CDM has reported that the keys that the sample requires for decryption are usable. Before sending a sample to the CDM, the EME MediaDataDecoders check with the SamplesWaitingForKey, which looks up in the CDMCaps whether the CDM has reported that the sample's keyId is usable. If not, the SamplesWaitingForKey registers with the CDMCaps for a callback once the key becomes usable. This stalls the decode pipeline until content JavaScript has negotiated a license for the media.

Content JavaScript negotiates licenses by receiving messages from the CDM on the MediaKeySession object, and forwarding those messages on to the license server, and forwarding the response from the license server back to the CDM via the MediaKeySession.update() function. These messages are in turn proxied by the ChromiumCDMProxy to the GMP thread, and result in a call to ChromiumCDMParent and thus an IPC message to the GMP process, and a function call into the CDM there. If the license server sends a valid license, the CDM will report the keyId as usable via a key statuses changed callback.

Once the key becomes usable, the SamplesWaitingForKey gets a callback, and the EME MediaDataDecoder will submit the sample for processing by the CDM and the pipeline unblocks. 

EME on Android

EME on Android is similar in terms of the EME DOM binding and integration with the MediaFormatReader and friends, but it uses a MediaDrmCDMProxy instead of a ChromiumCDMProxy. The MediaDrmCDMProxy doesn't talk to the GMP subsystem, and instead uses the Android platform's inbuilt Widevine APIs to process encrypted samples.

How WebRTC uses OpenH264

WebRTC uses OpenH264 for encode and decode of baseline H.264 streams. It doesn't need all the DRM stuff, so it talks to the OpenH264 GMP via the PGMPVideoDecoder and PGMPVideoEncoder protocols.

The child actors GMPVideoDecoderChild and GMPVideoEncoderChild talk to OpenH264, which conforms to the GMP API.

OpenH264 is not used by Firefox for playback of H264 content inside regular <video>, though there is still a GMPVideoDecoder MediaDataDecoder in the tree should this ever be desired.

How GMP shutdown works

Shutdown is confusing, because there are three processes involved. When the destructor of the MediaKeys object in the content process is run (possibly because it's been cycle or garbage collected), it calls CDMProxy::Shutdown(), which calls through to ChromiumCDMParent::Shutdown(), which cancels pending decrypt/decode operations, and sends a Destroy message to the ChromiumCDMChild.

In the GMP process, ChromiumCDMChild::RecvDestroy() shuts down and deletes the CDM instance, and sends a __delete__ message back to the ChromiumCDMParent in the content process.

In the content process, ChromiumCDMParent::Recv__delete__() calls GMPContentParent::ChromiumCDMDestroyed(), which calls CloseIfUnused(). The GMPContentParent tracks the living protocol actors for this plugin instance in this content process, and CloseIfUnused() checks if they're all shutdown. If so, we unlink the GMPContentParent from the GeckoMediaPluginServiceChild (which is PGMPContent protocol's manager), and close the GMPContentParent instance. This shuts down the bridge between the content and GMP processes.

This causes the GMPContentChild in the GMP process to be removed from the GMPChild in GMPChild::GMPContentChildActorDestroy(). This sends a GMPContentChildDestroyed message to GMPParent in the main process.

In the main process, GMPParent::RecvPGMPContentChildDestroyed() checks if all actors on its side are destroyed (i.e. if all content processes' bridges to this GMP process are shutdown), and will shutdown the child process if so. Otherwise we'll check again the next time one of the GMPContentParents shuts down. 

Note there are a few places where we use GMPContentParent::CloseBlocker. This stops us from shutting down the child process when there are no active actors, but we still need the process alive. This is useful for keeping the child alive in the time between operations, for example after we've retrieved the GMPContentParent, but before we've created the ChromiumCDM (or some other) protocol actor.

How crash reporting works for EME CDMs

Crash handling for EME CDMs is confusing for the same reason as shutdown; because there are three processes involved. It's tricky because the crash is first reported in the parent process, but we need state from the content process in order to identify which tabs need to show the crash reporter notification box.

We receive a GMPParent::ActorDestroy() callback in the main process with aWhy==AbnormalShutdown. We get the crash dump ID, and dispatch a task to run GMPNotifyObservers() on the main thread. This collects some details, including the pluginID, and dispatches an observer service notification "gmp-plugin-crash".  A JavaScript module ContentCrashHandlers.jsm observes this notification, and rebroadcasts it to the content processes.

JavaScript in every content process observes the rebroadcast, and calls mozIGeckoMediaPluginService::RunPluginCrashCallbacks(), passing in the plugin ID. Each content process' GeckoMediaPluginService then goes through its list of GMPCrashHelpers, and finds those which match the pluginID. We then dispatch a PluginCrashed event at the window that the GMPCrashHelper reports as the current window owning the plugin. This is then handled by PluginChild.jsm, which sends a message to cause the crash reporter notification bar to show.

GMP crash reporting for WebRTC

Unfortunately, the code paths for WebRTC handling crashes is slightly different, due to their window being owned by PeerConnection. They don't use GMPCrashHelpers, they have PeerConnection help find the target window to dispatch PluginCrashed to.

Firefox video playback's skip-to-next-keyframe behavior

One of the quirks of Firefox's video playback stack is our skip-to-next-keyframe behavior. The purpose of this blog post is to document the tradeoffs skip-to-next-keyframe makes.

The fundamental question that skip-to-next-keyframe answers is, "what do we do when the video stream decode can't keep up with the playback speed?

Video playback is a classic producer/consumer problem. You need to ensure that your audio and video stream decoders produce decoded samples at a rate no less that the rate at which the audio/video streams need to be rendered. You also don't want to produce decoded samples at a rate too much greater than the consumption rate, else you'll waste memory.

For example, if we're running on a low end PC, playing a 30 frames per second video, and the CPU is so slow that it can only decode an average of 10 frames per second, we're not going to be able to display all video frames.

This is also complicated by our video stack's legacy threading model. Our first video decoding implementation did the decoding of video and audio streams in the same thread. We assumed that we were using software decoding, because we were supporting Ogg/Theora/Vorbis, and later WebM/VP8/Vorbis, which are only commonly available in software.

The pseudo code for our "decode thread" used to go something like this:
 
while (!AudioDecodeFinished() || !VideoDecodeFinished()) {
  if (!HaveEnoughAudioDecoded()) {
    DecodeSomeAudio();
  }
  if (!HaveEnoughVideoDecoded()) {
    DecodeSomeVideo();
  }
  if (HaveLotsOfAudioDecoded() && HaveLotsOfVideoDecoded()) {
    SleepUntilRunningLowOnDecodedData();
  }
}
 
This was an unfortunate design, but it certainly made some parts of our code much simpler and easier to write.

We've recently refactored our code, so it no longer looks like this, but for some of the older backends that we support (Ogg, WebM, and MP4 using GStreamer on Linux), the pseudocode is still effectively (but not explicitly or obviously) this. MP4 on Windows, MacOSX, and Android in Firefox 36 and later now decode asynchronously, so we are not limited to decoding only on one thread.

The consequence of decoding audio and video on the same thread only really bites on low end hardware. I have an old Lenovo x131e netbook, which on some videos can take 400ms to decode a Theora keyframe. Since we use the same thread to decode audio as video, if we don't have at least 400ms of audio already decoded while we're decoding such a frame, we'll get an "audio underrun". This is where we don't have enough audio decoded to keep up with playback, and so we end up glitching the audio stream. This sounds is very jarring to the listener.

Humans are very sensitive to sound; the audio stream glitching is much more jarring to a human observer than dropping a few video frames. The tradeoff we made was to sacrifice the video stream playback in order to not glitch the audio stream playback. This is where skip-to-next-keyframe comes in.

With skip-to-next-keyframe, our pseudo code becomes:

while (!AudioDecodeFinished() || !VideoDecodeFinished()) {
  if (!HaveEnoughAudioDecoded()) {
    DecodeSomeAudio();
  }
  if (!HaveEnoughVideoDecoded()) {
    bool skipToNextKeyframe =
      (AmountOfDecodedAudio < LowAudioThreshold()) ||
       HaveRunOutOfDecodedVideoFrames();
    DecodeSomeVideo(skipToNextKeyframe);
  }
  if (HaveLotsOfAudioDecoded() && HaveLotsOfVideoDecoded()) {
    SleepUntilRunningLowOnDecodedData();
  }
}

We also monitor how long a video frame decode takes, and if a decode takes longer than the low-audio-threshold, we increase the low-audio-threshold.

If we pass a true value for skipToNextKeyframe to the decoder, it is supposed to give up and skip its decode up to the next keyframe. That is, don't try to decode anything between now and the next keyframe.

Video frames are typically encoded as a sequence of full images (called "key frames", "reference frames", or  I-frames in H.264) and then some number of frames which are "diffs" from the key frame (P-Frames in H.264 speak). (H.264 also has B-frames which are a combination of diffs of frames frames both before and after the current frame, which can lead the encoded stream to be muxed out-of-order).

The idea here is that we deliberately drop video frames in the hope that we give time back to the audio decode, so we are less likely to get audio glitches.

Our implementation of this idea is not particularly good.

Often on low end Windows machines playing HD videos without hardware accelerated video decoding, you'll get a run of say half a second of video decoded, and then we'll skip everything up to the next keyframe (a couple of seconds), before playing another half a second, and then skipping again, ad nasuem, giving a slightly weird experience. Or in the extreme, you can end up with only getting the keyframes decoded, or even no frames if we can't get the keyframes decoded in time. Or if it works well enough, you can still get a couple of audio glitches at the start of playback until the low-audio-threshold adapts to a large enough value, and then playback is smooth.

The FirefoxOS MediaOmxReader also never implemented skip-to-next-keyframe correctly, our behavior there is particularly bad. This is compounded by the fact that FirefoxOS typically runs on lower end hardware anyway. The MediaOmxReader doesn't actually skip decode to the next keyframe, it decodes to the next keyframe. This will cause the video decode to hog the decode thread for even longer; this will give the audio decode even less time, which is the exact opposite of what you want to do. What they should do is skip the demux of video up to the next keyframe, but if I recall correctly there was bugs in the Android platform's video decoder library that FirefoxOS is based on that caused this to be unreliable.

All these issues occur because we share the same thread for audio and video decoding. This year we invested some time refactoring our video playback stack to be asynchronous. This enables backends that support it to do their decoding asynchronously, on another own thread. So since audio decodes on a separate thread to video, we should have glitch-free audio even when the video decode can't keep up, even without engaging skip-to-next-keyframe. We still need to do something like skipping the video decode when the video decode is falling behind, but it can probably engage less aggressively.

I did a quick test the other day on a low end Windows 8.0 tablet with an Atom Z2760 CPU with skip-to-next-keyframe disabled and async decoding enabled, and although the video decode falls behind and gets out of sync with audio (frames are rendered late) we never glitched audio.

So I think it's time to revisit our skip-to-next-keyframe logic, since we don't need to sacrifice video decode to ensure that audio playback doesn't glitch.

When using async decoding we still need some mechanism like skip-to-next-keyframe to ensure that when the video decode falls behind it can catch up. The existing logic to engage skip-to-next-keyframe also performs that role, but often we enter skip-to-next-keyframe and start dropping frames when video decode actually could keep up if we just gave it a chance. This often happens when switching streams during MSE playback.

Now that we have async decoding, we should experiment with modifying the HaveRunOutOfDecodedVideoFrames() logic to be more lenient, to avoid unnecessary frame drops during MSE playback. One idea would be to only engage skip-to-next-keyframe if we've missed several frames. We need to experiment on low end hardware.

How to prefetch video/audio files for uninterrupted playback in HTML5 video/audio

Sometimes when you're playing a media file using an HTML5 <video> or <audio> element, or with WebAudio, you really want to be sure that the whole audio/video file is totally downloaded before you start playing it. For example, you may be writing a game, and you want to be sure all your sound effects are preloaded, so there's no delay between your animations and your sound effects while the network downloads the remainder of the file.

So how can you be sure a media resource is fully downloaded before beginning playing it? You could wait for the "canplaythrough" event to fire on all your media elements, but that event is not fired correctly by Chrome.

A more reliable solution is to prefetch the video/audio file using XHR/AJAX requests, and play the video/audio from a Blob URI.

Here's a simple snippet of a JS file that downloads a file using XHR. The function accepts callbacks to return results.

function prefetch_file(url,
                       fetched_callback,
                       progress_callback,
                       error_callback) {
  var xhr = new XMLHttpRequest();
  xhr.open("GET", url, true);
  xhr.responseType = "blob";

  xhr.addEventListener("load", function () {
    if (xhr.status === 200) {
      var URL = window.URL || window.webkitURL;
      var blob_url = URL.createObjectURL(xhr.response);
      fetched_callback(blob_url);
    } else {
      error_callback();
    }
  }, false);

  var prev_pc = 0;
  xhr.addEventListener("progress", function(event) {
    if (event.lengthComputable) {
      var pc = Math.round((event.loaded / event.total) * 100);
      if (pc != prev_pc) {
        prev_pc = pc;
        progress_callback(pc);
      }
    }
  });
  xhr.send();
}

When the file is successfully downloaded, the fetched_callback is called with an argument which is the blob URI. You can simply set this as the src of an audio or video element and can then play the fully-downloaded resource. You can also set the same blob URI as the src of multiple audio/video elements, and the downloaded data won't be re-downloaded or duplicated/copied in memory.

There's also a progress_callback that's called with a percentage complete parameter as the file is downloaded, and an error_callback that's called when the download fails.

For a working demo: prefetching a video file before playback using HTML5 demo

The rise and fall of Movie Rotator

Several years ago my Dad came to me and asked me how to rotate a video he'd recorded on his camera. He'd turned the camera sideways to record a video of someone standing and talking (in portrait orientation) but when he played the video back on his computer it was rendered in landscape orientation by his media player, i.e. not the same we he'd shot it, so the picture was rendered off by 90 degrees.

My Dad was running Windows XP, and so didn't have a platform video editor, so I set about making one.

I built and released Movie Rotator version 1.0, in March 2008. I learned a lot, and I shipped my own first complete product. By May 2010 I was consistently getting 4,000 downloads per month. Movie Rotator 1 was free and open source, and used the Quicktime runtime to set a rotation matrix in the MP4/MOV container causing the video to be rotated by the media player. It was a simple solution, and didn't require the video the be re-encoded, but some media players didn't honour that matrix stored in the container, so it didn't work for some video players. Most notably YouTube. There were other problems that became apparent later too.

Movie Rotator 1

I never wanted to charge for Movie Rotator 1, I was never sure whether it would take off, if there would be enough demand for it.

Fast forward to late 2012, and I learned how to use Windows Media Foundation to playback videos as part of my H.264/AAC/MP4 support work in Firefox on Windows. I had been looking forward to being able to learn this on "company time", as I could then use those skills in making Movie Rotator 2, a newer version that reencoded the videos rotated, to solve all the problems in version 1. Movie Rotator 1 was still getting 4000 downloads per month, and had steady Google Adsense income. Competitors existed. They charged for their software. In January 2013 I started work on Movie Rotator 2. This time I intended to try selling it, to make some money for my family.

I sacrificed to work on Movie Rotator 2. My daughter was only a year old at the time, so others had to fill in for me while I was working nights and weekends on this project. I started to skip the Saturday morning black belt class to work on Movie Rotator

After months of work I had learned a huge amount working on Movie Rotator 2. I learned all about about encoding and playback with WMF. I also had to learn about Direct3D9, Direct2D, and a bunch of other C++11x features. I learned a little about crypto and how software license keys are made and validated. I also learned to ruthlessly prioritize. In the evenings I was exhausted from work and parenting, so I saved the hard problems for Saturdays when I worked on the weekend. Of course that would be greatly inconveniencing my family. So I spent my time carefully.

Then in August 2013 I checked my websever's logs again. They were down, way down. I was down to only 1900 downloads per month. Traffic was dropping steadily. In all likelihood, I extrapolated and Movie Rotator didn't have long left.

I'd always assumed that Movie Rotator was a product only good for a few years, but I'd expected a few more yet.

What to do? Development of the software was almost finished. I'd been on the cusp of engaging an accountant to incorporate and a lawyer to sort out a software license/purchase agreement... But now I wasn't certain I'd recoup the expense. I was gutted. I'd sacrificed so much, and rolled up snake eyes.

So I released a free version of Movie Rotator 2 in September 2012, and today scrubbed the code and committed it to posterity as open source. Movie Rotator 2's code is now available on GitHub.

Movie Rotator 2

I benefited greatly building this product. I learned an awful lot, way more than on Movie Rotator 1. I really learned how to focus and prioritize. But it is still a bitter pill; all the sacrifice I made, to not achieve my goal.

I guess the world has changed. My theory is that people are mostly recording and watching their videos on their mobile phones now, and those handle video rotation just fine, they have to. At least now that my code is open sourced, it may still help someone someday, possibly me even. I'd like to take more time to clean it up more, and experiment with the design to see how various patterns work. It's good to have a moderately sized codebase to test such things on. You never know what may happen. My DirectShow Firefox code languished for years before finally being resurrected this year to ship in Firefox, so who knows...

What does the H.264/avc1 codecs parameters for video/mp4 mime types mean?

The HTMLMediaElement.canPlayType() API enables you to query what video formats a user agent can play. For "video/mp4", the container for H.264/AAC, you can specify a "codecs" parameter that denotes the the H264 profile and level. Firefox doesn't currently handle MP4 codecs parameter very well, so I took it upon myself to figure out what the codecs parameters mean for H.264.

According to RFC6381 The 'Codecs' and 'Profiles' Parameters for "Bucket" Media Types, codecs paremeters for H.264 are contained in the "avc1" sample entry, and are is represented as follows:

avc1.PPCCLL

That is, the string "avc1." (or "avc2.", I'm not sure what the difference is yet), followed by 3 bytes represented in hex without the "0x" prefix, where the bytes represent the following:

PP = profile_idc
CC = constraint_set flags
LL = level_idc

These fields are defined in in Annex 1 of ITU-T H.264 and ISO/IEC 14496-10:2012  twinned standards. ITU-T H.264 can be downloaded for free.

profile_idc defines the H.264 profile. ITU-T H.264 doesn't have a single table listing what the different profile_idc values mean, but handily, Microsoft defines an eAVEncH264VProfile enumeration on the decimal values of the profile_idc in Codecapi.h (available on Win7):

enum eAVEncH264VProfile {
  eAVEncH264VProfile_unknown                    = 0,
  eAVEncH264VProfile_Simple                     = 66,
  eAVEncH264VProfile_Base                       = 66,
  eAVEncH264VProfile_Main                       = 77,
  eAVEncH264VProfile_High                       = 100,
  eAVEncH264VProfile_422                        = 122,
  eAVEncH264VProfile_High10                     = 110,
  eAVEncH264VProfile_444                        = 144,
  eAVEncH264VProfile_Extended                   = 88,
  eAVEncH264VProfile_ScalableBase               = 83,
  eAVEncH264VProfile_ScalableHigh               = 86,
  eAVEncH264VProfile_MultiviewHigh              = 118,
  eAVEncH264VProfile_StereoHigh                 = 128,
  eAVEncH264VProfile_ConstrainedBase            = 256,
  eAVEncH264VProfile_UCConstrainedHigh          = 257,
  eAVEncH264VProfile_UCScalableConstrainedBase  = 258,
  eAVEncH264VProfile_UCScalableConstrainedHigh  = 259
};

So for example, avc1.4D401E has a profile_idc of 0x4D, which is 77 in decimal, so it's main profile.

constraint_set flags are encoded as a bit flags 6 bitfields named constraint_set0_flag through to constraint_set5_flag. The meaning of a constraint_setN_flag being set depends on the profile being represented. The bits are stored in with constraint_set0_flag in the high bit, and so there are two padding 0s in the low bits. So for example, avc1.4D401E, the contraint_set flags are 0x40, so that's 01000000b, thus this means constraint_set1_flag is set.

The level number, level_idc, is defined as fixed point, from 1 to 5.2, with an oddball 1b level, as defined in ITU-T H.264 in Table A-1 - Level Limits. The level_idc is encoded in the codecs parameter as 10 times the level number, i.e. level 5.1 is represented as decimal 51, or 0x33 hex. So continuing our example avc1.4D401E, the level_idc is 1E, or 30 decimal, so level 3.0.

Level 1b is an oddball encoding, and is encoded as 11 decimal with the constraint_set3_flag equal to 1. If the constraint_set3_flag is not equal to 1, level 1.1 is encoded.

Microsoft also conveniently define this in the eAVEncH264VLevel enumeration.

enum eAVEncH264VLevel {
  eAVEncH264VLevel1    = 10,
  eAVEncH264VLevel1_b  = 11,
  eAVEncH264VLevel1_1  = 11,
  eAVEncH264VLevel1_2  = 12,
  eAVEncH264VLevel1_3  = 13,
  eAVEncH264VLevel2    = 20,
  eAVEncH264VLevel2_1  = 21,
  eAVEncH264VLevel2_2  = 22,
  eAVEncH264VLevel3    = 30,
  eAVEncH264VLevel3_1  = 31,
  eAVEncH264VLevel3_2  = 32,
  eAVEncH264VLevel4    = 40,
  eAVEncH264VLevel4_1  = 41,
  eAVEncH264VLevel4_2  = 42,
  eAVEncH264VLevel5    = 50,
  eAVEncH264VLevel5_1  = 51,
  eAVEncH264VLevel5_2  = 51
}

Hardware accelerated H.264 decoding landed in Firefox on Windows Vista and later

I have been hard at work getting our H.264 support on Windows Vista and later hardware accelerated using DXVA2. I'm happy to say that this finally landed in Firefox 23 Nightly builds. With this patch we'll use the GPU to accelerate H.264 video decoding when possible on Windows, which greatly reduces our CPU and power usage.

If you spot a bug in H.264 video in Firefox Nightly builds, please file a bug in Core: Audio/Video. Bonus points if you toggle the pref "media.windows-media-foundation.use-dxva" to false, reload the video, and tell me whether the bug still happens!

Notes:

1. H.264/AAC/MP3 support on Windows 7 and later is shipping in Release builds in Firefox 21 next week, Vista gets it in Firefox 22.
2. Edwin Flores is working on our H.264/AAC/MP3 support on Mac and Linux.
3. I'm currently working on getting MP3 support for Windows XP using DirectShow, it will probably land next cycle (Firefox 24).

H.264/AAC/MP3 support now enabled by default in Firefox Nightlies on Windows 7 and later

Support for playing H.264/AAC in MP4 and support for playing MP3 audio files in HTML5 <video> and <audio> elements has now been switched on by default for Windows 7 and later in the Firefox Nightly channel. You no longer need to set the pref to enable it. Please test MP4/MP3 support in Firefox Nightly builds, and file bugs in the "Core:: Video/Audio" component in Bugzilla.

Assuming no catastrophic bugs are found, H.264/AAC/MP3 support in Firefox on Windows 7 and later should ship in Firefox 22 21, which is scheduled to be released around June 25 May 14.

Edit 25 Feb 2013: Corrected target release, it's Firefox 21, not Firefox 22.

HTML5 video playbackRate and Ogg chaining support landed in Firefox

Paul Adenot has recently landed patches in Firefox to enable the playbackRate attribute on HTML5 <audio> and <video> elements.

This is a cool feature that I've been looking forward to for a while; it means users can speed up playback of videos (and audio) so that you can for example watch presentations sped up and only slow down for the interesting bits. Currently Firefox's <video> controls don't have support for playbackRate, but it is accessible from JavaScript, and hopefully we'll get support added to our built-in controls soon.

Paul has also finally landed support for Ogg chaining. This has been strongly desired for quite some time by community members, and the final patch also had contributions from "oneman" (David Richards), who also was a strong advocate for this feature.

We decided to reduce the scope of our chaining implementation in order to make it easier and quicker to implement. We targeted the features most desired by internet radio providers, and so we only support chaining in Ogg Vorbis and Opus audio files and we disable seeking in chained files.

Thanks Paul and David for working on these features!

Experimental H.264,AAC, and MP3 support in Firefox Nightly builds on Windows 7 and later

As the Internet has already discovered, recently I landed patches to add a Windows Media Foundation playback backend for Firefox. This is preff'd off by default.

This allows playback of H.264 video and AAC audio in MP4 and M4A files, and MP3 audio files in HTML5 <audio> and <video> elements in Firefox on Windows 7 and later.

To test MP4/MP3 playback, download the latest Firefox Nightly Build, and toggle the pref "media.windows-media-foundation.enabled" to "true" in about:config.

There are a few bugs that I'm aware of, which is why this landed preff'd off by default, but if you spot any bugs, please file a bug in Firefox's "Core :: Video/Audio" component.

Firefox's HTML full-screen API enabled in Nightly builds

A few days ago I enabled the HTML full-screen API in Firefox nightly builds. This enables developers to make an arbitrary HTML element "full-screen", hiding the browser's UI and stretching the element to encompass the entire screen. This will be particularly useful for HTML5 video and games.

If all goes well, this feature will ship in Firefox 10 at the end of January.

The API has changed slightly since I last blogged about it. The current API is Mozilla-specific, but is similar to the W3C's Fullscreen draft specification.

To enter full-screen mode, call the following method on the HTML Element you'd like to enter full-screen:

• void mozRequestFullScreen() : posts an asynchronous request to make the HTML element the full-screen element. If the request is granted, some time later a bubbling "mozfullscreenchange" event is dispatched to the element which requested full-screen. If the request is denied, a "mozfullscreenerror" event is dispatched to the element's owning document. We only grant requests for full-screen when:
• mozRequestFullScreen() is called in a user-generated event handler, e.g. a mouse click handler, and
• the requesting element is in its document, and
• there are no windowed plugins present in any document/iframe in the current page, and
• all iframes containing the requesting element (if any) have the mozallowfullscreen attribute.

We added the following method and attributes to HTML Document:

• void mozCancelFullScreen() : exits the document from full-screen mode. This dispatches a "mozfullscreenchange" event to the document containing the (now former) full-screen element. Note that the "mozfullscreenchange" event which is dispatched when you enter full-screen is targeted at the full-screen element, so if you want to receive the "mozfullscreenchange" on both entering and exiting full-screen in the same listener you should add your listener to the document, rather than the full-screen element.
• readonly attribute boolean mozFullScreen : true when the document is in full-screen mode.
• readonly attribute Element mozFullScreenElement : reference to the current full-screen element.
• readonly attribute boolean mozFullScreenEnabled : returns true if calls to mozRequestFullScreen() would be granted in the current document. This returns false if there are any windowed plugins present in any document/iframe in the current page, or if any iframes containing this document don't have the mozallowfullscreen attribute present, or if the user has disabled the API by preference. If this returns false you may want to not show the user your enter-full-screen button in your page, since you know it won't work!

We also added the :-moz-full-screen css pseudo class, which applies to the full-screen element while in full-screen mode.

We added the mozallowfullscreen attribute to iframe elements. Without this, full-screen requests made by script in the iframe's content (i.e embedded ads, or a YouTube player in an iframe for that matter) will be denied.

While in full-screen mode, the user can press the ESC key (or F11) to exit. Alpha-numeric keyboard input while in full-screen mode causes a warning message to pop-up to guard against phishing attacks. The only key input which doesn't cause the warning message to pop up are: left, right, up, down, space, shift, control, alt, page up, page down, end, home, tab, and meta.

Navigating, changing tab, changing app (ALT+TAB) while in full-screen mode will cause full-screen mode to exit.

Here's about a simple example, which will work in current Firefox nightly builds:

(Press ESC to exit full-screen)

The code for that button's onclick handler is simply:
document.getElementById('bruce_video').mozRequestFullScreen();

How is Firefox's full-screen API different from Webkit/Chrome/Safari's full-screen API? Firefox's API adds a "width: 100%; height: 100%;" CSS rule to the element which requests full-screen, so that it's stretched to occupy the entire screen. Chrome's API does not do this, but instead it centers the full-screen element in the window and blacks-out the underlying webpage. So the full-screen element won't occupy the entire screen with Chrome's API unless you specify a "width: 100%; height: 100%;" rule yourself. Conversely if you want to vertically and horizontally center something while in full-screen with Firefox's API, you need to make the containing element of your desired centered element full-screen instead, and apply CSS rules to vertically and horizontally center the contained element.

For a cross-browser full-screen API example, see html5-demos.appspot.com's full-screen demo.

Edit: 11 Nov 2011, clarified Document.mozCancelFullScreen() and Document.mozFullScreenEnabled, fixed typos.

New media element APIs and better media seeking resolution

French intern Paul Adenot has recently implemented the seekable and played attributes on the HTML5 video and audio elements in Firefox. The seekable attribute enables script to see what regions of the media can be seeked into (particularly handy with live streams), and the played attribute enables script to see what regions of the media has already been played. Paul has also done some work improving the built in controls on media elements. Thanks for your hard work Paul! These should be available in release builds in November (Firefox 8).

Also in Firefox 8 are my changes to media seeking resolution. Now media seeking should be accurate to the nearest microsecond. It's been reported elsewhere how important accurate seeking for video is. We were previously accurate to the nearest video frame, but we could still be up to one audio packet off (often between 4 and 8 ms out). Now we prune audio samples when seeking so we're down to microsecond resolution.

Simple rate limited HTTP server for testing HTML5 media/streaming

While working on the Firefox HTML5 video and audio support, I've found it extremely useful to have an HTTP server on which the transfer rate is reliably limited. Existing servers are either too heavy weight, like apache, or have inconsistent rate-limiting, like lighttpd which I found to have very "bursty" rate limiting.

I ended up taking the educational route, and implementing a simple HTTP server in C++. It supports the following features:

1. Support for HTTP1.1 Byte Range Requests. This means you can seek into unbuffered data when watching HTML5 video.
2. Rate limiting, configurable on a per request basis by passing the "rate=x" HTTP query parameter, where x is the transfer rate of the connection in kilobytes per second. The server will send x/10 KB ten times per second to maintain this rate smoothly.
3. Simulated live streaming, configurable on a request basis by passing the "live" query parameter. When in "live" mode, no Content-Length header is sent, and the server doesn't advertise or perform byte range requests - so you can't seek into unbuffered video/audio, just like in a live stream.
4. Cross platform; tested on Windows (runs on port 80) and Linux (runs on port 8080). I haven't test it on MacOS yet.
5. Simply serves all files in the program's working directory, making it easy to use (and abuse).
6. Open source! Get the code at https://github.com/cpearce/HttpMediaServer, or download a pre-built win32 binary.

For example, if you wanted to simulate a live stream being served at 100KB/s, your test URL might look something like http://localhost:80/video.ogg?rate=100&live.

I've been using it for quite a while, and over the weekend I finally cleaned it up and put it up on GitHub. Check it out.

Reducing the memory overhead of thread stacks

I recently landed bug 664341 into mozilla-central, which adds an API to specify the amount of virtual address space reserved for the thread stacks of nsIThreads. The new API looks like this:

  extern NS_COM_GLUE NS_METHOD
  NS_NewThread(nsIThread **result,
               nsIRunnable *initialEvent = nsnull,
               PRUint32 stackSize = nsIThreadManager::DEFAULT_STACK_SIZE);

The default stack size is the default for whatever platform your running on, so behaviour is unchanged for existing uses of NS_NewThread.

This new API is important, as x86 Linux by default reserves 8MB of virtual address space per thread stack. Windows and OSX use 1MB and 64KB respectively. If you have a lot of threads, their stacks can hog the virtual address space, and malloc will fail; we had at least one media mochitest that could fail in this way.

If you have code that creates threads, you should consider using this API. It's an easy way to reduce perceived memory usage. 

I've also recently concluded a major refactoring of the media playback engine in Firefox. This reduces the number of threads required to play <audio> and <video> elements by roughly one third. We now only require two threads per playing media element (plus one extra thread for sound playback on Linux at least until bug 623444 lands and we can refactor to take advantage of that). Media elements which are paused now shut down their threads where possible, resulting in lower overall memory usage. If you have a page with 100 <audio> elements on it, you no longer have 300 threads lying around using up virtual address space!

HTML5 Video painting performance statistics in Firefox 5

I've landed video frame paint performance counters for HTML5 video onto mozilla-central. This should ship in Firefox 5, barring any disasters. This work was a combined effort by Chris Double and I. These are Mozilla specific fields which will only be available in Firefox.

The new statistics enable us to measure the performance of the video decoding and frame painting pipeline.

This adds the following fields to the HTMLVideoElement:

• mozParsedFrames - A count of the number of video frames that have been demuxed/parsed from the media resource. If we were playing perfectly, we'd be able to paint this many frames.
• mozDecodedFrames - A count of the number of deumxed/parsed video frames that have been decoded into Images. We skip decoding of parsed/demuxed frames if the decode is falling behind the playback position (this can happen if it takes a long time to decode a keyframe for example).
• mozPresentedFrames - A count of the number of decoded frames that have been presented to the rendering pipeline for painting (set as the current Image on the video element's ImageContainer). We may not present decoded frames if the frame arrives for presentation late.
• mozPaintedFrames - A count of the number of presented frames which were painted on screen. We may end up not painting presented frames if another frame is presented before the graphics pipeline has time to paint the previously presented frame, or if the video is off screen. 
• mozFrameDelay - The time (as a floating point number in seconds) which the last painted video frame was rendered late by. This is the time duration between the decoder saying "paint frame X now", and the graphics pipeline physically getting frame X displayed on the screen. The value is accurate on desktop Firefox, but not on mobile. Improvements in the graphics pipeline, and the integration with the graphics pipeline, will show up as a decrease in this number.

Here's a demo of the video paint statistics in Firefox 5. You'll need a recent Firefox trunk nightly build for the demo to work.

Firefox 4 video decoder architecture

To assist others coming up to speed on the architecture of the video decoder, I've put together a diagram of Firefox 4's video playback engine. We rewrote our video architecture for Firefox 4 in order to give us better control over the complete stack.

Click on the image for a larger diagram.

The key classes in our architecture are:

• nsHTMLMediaElement - This manages the JavaScript/HTML accessible HTMLMediaElement interface, and implements the resource selection, load, and preload logic.
• nsBuiltinDecoder - Manages a main thread accessible snapshot of the state of the underlying decoder. The decoders run on non-main threads, and we don't want to block the main thread to dig into the decoders when JS queries playback state, so we maintain a snapshot of the playback engine's state in this class. This inherits from nsMediaDecoder. You can also implement playback support for a new format by inheriting and implementing nsMediaDecoder. nsWaveDecoder is currently implemented this way, but we're in the process of reimplementing that as a sublcass of nsBuiltinDecoderReader.
• nsBuiltinDecoderStateMachine - Manages the decode, state machine, audio-push threads, frame queueing, A/V sync, and buffering logic. This ensures that all the HTML5 events get dispatched at the appropriate time, and that behaviour is consistent and sane across different media types. Demuxing is handled abstractly by subclasses of nsBuiltinDecoderReader. This way all media types can share as much playback logic as possible, reducing our maintenance overhead.
• nsOggReader/nsWebMReader - Demuxing and codec specific functionality is implemented by subclassing nsBuiltinDecoderReader. This reduces the amount of work required to implement and maintain support for new codecs. When a new codec is implemented as a nsBuiltinDecoderReader subclass, support for HTML events, buffering, and playback logic does not need to be reimplemented, since it already exists in nsBuiltinDecoderStateMachine. To add support for a new codec, it's easiest to implement support as a new nsBuiltinDecoderReader subclass.
• nsAudioStream - Our cross platform audio API wrapper. It is based on libsydneyaudio, which operates on a push model rather than a (more commonly used) callback-based model, which has brought in a whole raft of headaches. Matthew Gregan is in the process of rewriting our audio layer to a more sane callback based model. We also provide a cross-process nsAudioStreamRemote, which proxies audio commands to an audio stream in another process. This is required on mobile.
• ImageContainer - When it comes time to present a video frame, nsBuiltinDecoderStateMachine sets it as the "current image" of the video element's ImageContainer object. This then propagates through the Layers/2D scene rendering system, and it eventually gets rendered on the screen. The Layers compositing runs on the main thread, and ImageContainer provides a thread-safe wrapper. The images contained in the ImageContainer can be in OpenGL/D3D surfaces, so we can take advantage of hardware accelerated scaling, rendering, and YCbCr to RGB conversion.
• nsVideoFrame - This resides in layout, and manages the dimensions/reflow of the video, as well as its poster image.
• nsMediaStream - Our network code runs on the main thread, but the underlying libraries we use for media decoding (libvpx, libtheora, etc) assume synchronous reads. We can't afford to do blocking reads on the main thread, so we cache the media data downloaded into the nsMediaCache, and provide a thread-safe wrapper synchronous wrapper for reading in the nsMediaStream class. We use Necko for our networking, so we can take advantage of all the existing security and load-group functionality it implements.

The advantage of controlling the entire playback engine are many. We can easily control frame dropping, memory allocation, the threading model, what, when, and how we decode, and we can integrate more tightly with our network stack.

How to stop a video or audio element downloading

Say you've started playback of an HTML5 audio or video element, and you decide you really want to cancel playback and downloading of the media resource. Stopping playback is easy, just call the pause() method. But the network connection won't be stopped until the media element gets garbage collected, and even if you release all references to the media element, it won't be destroyed until the browser decides to runs its garbage collector. How can you stop the download of the media resource in the meantime? Here's a quick hack to achieve this: just reset the element's src attribute to the empty string. This destroys the element's internal decoder, and stops the network download.

For example:

<audio id="audio" controls>

  <source src="funky-music.ogg">

  <source src="funky-music.mp3">

</audio>
<!-- ... Some time later, we decide we should stop the audio element playing and downloading... -->
<script>
var audio = document.getElementById("audio");

audio.pause();

audio.src = ""; // Stops audio download.
audio.load(); // Initiate a new load, required in Firefox 3.x.
</script>


Be aware that this will destroy the media element's decoders, so the element won't be playable anymore, and it will be rendered as an "error cross" if it's in a document. Also in Firefox 3.x you need to call load() after changing the source, whereas in Firefox 4 the load is scheduled to run when you change the src attribute, and the extra load() call is not required.

Changes to HTML5 video/audio load() function in Firefox 4

I've updated the media load() implementation in Firefox 4 to match the current WHATWG media load algorithm specification. There are three main changes that web developers using media elements should be aware of.

Firstly, error reporting has slightly changed. When a media element fails to load from its <source> children, an "error" event is dispatched to every child element which failed to load. Previously in Firefox 3.x you'd receive only one "error" event dispatched to the media element once all of its child <source> elements had failed to load. Now you only receive "error" events in the child <source> elements, and not in the media element itself.

For example, suppose you have the following markup:

<video>

<source id="mp4_src"

        src="video.mp4"

        type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</source>

<source id="3gp_src"
        src="video.3gp"
        type='video/3gpp; codecs="mp4v.20.8, samr"'>
</source>

<source id="ogg_src"
        src="video.ogv"
        type='video/ogg; codecs="theora, vorbis"'>
</source>

</video>

Firefox 4 does not support the playback of patent encumbered content, so you'll receive "error" events in the <source> elements with the MP4 and 3GP resources, before the Ogg resource is loaded. Note also that the <source> children are loaded in the order in which they appear in the markup, and if one <source> child successfully loads and is playable, the children after it won't be loaded.

To detect that all child <source> elements have failed to load, check the value of the networkState attribute of the media element; if its value is HTMLMediaElement.NETWORK_NO_SOURCE, you know all child <source> elements have failed to load.

If you add another child <source> element to a media element which is in networkState HTMLMediaElement.NETWORK_NO_SOURCE, it will attempt to load the resource specified by the newly added <source>.

Secondly, when the load begins has changed. When you set the src attribute of a media element, or add a <source> child element to a media element, a load will be scheduled to run asynchronously as soon as the current JavaScript context exits (basically the load starts the next time we return to the browser application's main event loop). So for example, suppose you have some timeout set such as:

<script>

setTimeout(
  function() {

    var v = document.createElement("video");

    v.src = "video.ogg";
    document.appendChild(v)

    // Do some other stuff...
  },
  1000);

</script>

When the function runs, the load for the video element won't begin until after the function returns, and control returns to the browser. This is important, because in Firefox 3.x, the load is started (and could even run to completion!) as soon as you set the media element's src attribute.

Lastly, the media element's events now no longer bubble. Previously they bubbled, and this was a bug in Firefox 3.x and in violation of the spec.