One common way to reduce the load on a web server is to use gzip or deflate compression when serving to a supporting web browser. When Firefox requests an Ogg media, it advertises that it can handle a gzipped or deflated response; the HTTP request includes the Accept-Encoding: gzip,deflate header. But despite Firefox advertising that it supports gzip/deflate, you probably don't want your web server to gzip or deflate Ogg media. If you serve an Ogg media compressed, Firefox won't be able to seek in the media, or determine its duration. Since the video/audio data in Ogg files is already compressed, gzip/deflate won't actually save you much bandwidth anyway, so you probably want to disable compression when serving Ogg files.
I have added this tip to the Mozilla Developer Center article on configuring servers for Ogg media.
Friday, 12 March 2010
Sunday, 7 March 2010
Switched blog to custom domain
I have switched my blog to being on a custom domain hosted at Blogger. The address for my blog is now blog.pearce.org.nz. Previously I was using Blogger's publish via FTP to host my blog on my own domain, but they have discontinued that service. I tried running Wordpress on my own domain, but I couldn't get plugins to work, so without WP-Cache my blog was slow. I'm glad to have my blog restored to its former snappy look and feel.
Monday, 11 January 2010
Indexing keyframes in Ogg videos for fast seeking
Seeking in Ogg videos in HTML5 <video> over the network currently can be very slow. This is because when we seek an Ogg/Theora video to a target time, we must perform a bisection search over the file in order to find the target Theora video frame. If this is an interframe (which just records what's changed since its preceding frame), we must then perform another bisection search to find the interframe's keyframe, and then decode forwards to the target frame in order to completely construct it.
A reasonable bisection search implementation may require half a dozen HTTP requests to complete, so if we need to do two bisection searches per seek (once for the target frame, and once for the target frame's keyframe), we actually need to do about a dozen HTTP requests per seek...
If we knew in advance where the keyframes were, we wouldn't need to do any bisection searches; we could make just one HTTP request to the last keyframe preceeding the target frame. Clearly making fewer HTTP requests is faster.
Enter the Skeleton 3.1 with Keyframe Index track. This extends the existing Skeleton 3.0 metadata track to provide an index for every Theora video and Vorbis audio track in an Ogg media. This will enable players to make the optimal HTTP request when seeking in media files served over the internet, resulting in as fast seeking as possible when viewing online video.
It's nice if a video player's UI can display the playback duration of the Ogg media. Unfortunately the raw Ogg format does not store the duration either, so it must be calculated, which requires additional HTTP requests, and slows down the video loading. The Skeleton 3.1 track also includes the playback duration of its Ogg containing file, to eliminate this overhead and speed up loading.
I developed the Skeleton 3.1 with Keyframe Index track in conjunction with the folks at Xiph.org. See the Xiph.org Skeleton 3.1 with Keyframe Index wiki page for the work-in-progress specification. Any comments on the specification would be much appreciated, please send them to the Theora mailing list.
I have developed a prototype Ogg indexer, OggIndex, and also recent ffmpeg2theora nightlies will encode keyframe indexes if you specify the command line option -–seek-index. OggIndex and experimental indexing ffmpeg2theora nightlies are available for download.
To see how keyframe indexing improves network seeking performance of HTML5 Ogg/Theora <video>, you can download a development version of Firefox which can take advantage of indexes here:
http://pearce.org.nz/video/firefox-indexed-seek-linux.tar.bz2
http://pearce.org.nz/video/firefox-indexed-seek-macosx.dmg
http://pearce.org.nz/video/firefox-indexed-seek-win32.zip
If you already have a Firefox instance running, you'll need to either close your running Firefox instance before starting the index-capable Firefox, or start the index-capable Firefox with the --no-remote command line parameter.
To compare the network performance of indexed versus non-indexed seeking, point the index-capable Firefox to the indexed seek demo page.
You should notice a clear speed difference when seeking to an unbuffered position in the indexed media.
The Skeleton 3.1 with Keyframe Index specification is still being developed, but we hope to lock it down soon. We are planning to ship support for keyframe index-assisted seeking in Firefox 3.7.
A reasonable bisection search implementation may require half a dozen HTTP requests to complete, so if we need to do two bisection searches per seek (once for the target frame, and once for the target frame's keyframe), we actually need to do about a dozen HTTP requests per seek...
If we knew in advance where the keyframes were, we wouldn't need to do any bisection searches; we could make just one HTTP request to the last keyframe preceeding the target frame. Clearly making fewer HTTP requests is faster.
Enter the Skeleton 3.1 with Keyframe Index track. This extends the existing Skeleton 3.0 metadata track to provide an index for every Theora video and Vorbis audio track in an Ogg media. This will enable players to make the optimal HTTP request when seeking in media files served over the internet, resulting in as fast seeking as possible when viewing online video.
It's nice if a video player's UI can display the playback duration of the Ogg media. Unfortunately the raw Ogg format does not store the duration either, so it must be calculated, which requires additional HTTP requests, and slows down the video loading. The Skeleton 3.1 track also includes the playback duration of its Ogg containing file, to eliminate this overhead and speed up loading.
I developed the Skeleton 3.1 with Keyframe Index track in conjunction with the folks at Xiph.org. See the Xiph.org Skeleton 3.1 with Keyframe Index wiki page for the work-in-progress specification. Any comments on the specification would be much appreciated, please send them to the Theora mailing list.
I have developed a prototype Ogg indexer, OggIndex, and also recent ffmpeg2theora nightlies will encode keyframe indexes if you specify the command line option -–seek-index. OggIndex and experimental indexing ffmpeg2theora nightlies are available for download.
To see how keyframe indexing improves network seeking performance of HTML5 Ogg/Theora <video>, you can download a development version of Firefox which can take advantage of indexes here:
http://pearce.org.nz/video/firefox-indexed-seek-linux.tar.bz2
http://pearce.org.nz/video/firefox-indexed-seek-macosx.dmg
http://pearce.org.nz/video/firefox-indexed-seek-win32.zip
If you already have a Firefox instance running, you'll need to either close your running Firefox instance before starting the index-capable Firefox, or start the index-capable Firefox with the --no-remote command line parameter.
To compare the network performance of indexed versus non-indexed seeking, point the index-capable Firefox to the indexed seek demo page.
You should notice a clear speed difference when seeking to an unbuffered position in the indexed media.
The Skeleton 3.1 with Keyframe Index specification is still being developed, but we hope to lock it down soon. We are planning to ship support for keyframe index-assisted seeking in Firefox 3.7.
Friday, 20 November 2009
Replay Debugging mochitest failures with VMWare Workstation 7
Ever since my last escapades with Replay Debugging in VMWare Workstation 6.5, I've been looking forward to improvements in this awesome technology. Thankfully the guys at VMWare have been hard at work, and now VMWare Workstation 7 now boasts improved Replay Debugging. I've found it much more robust and reliable, and Roc and I have already used it to debug some random orange bugs.
I've documented how to produce a Replay Debugging setup for debugging intermittent test failures in Mozilla mochitests, and put it up on MDC:
https://developer.mozilla.org/En/Debugging/Record_and_Replay_Debugging_Firefox
Now anyone can setup a machine to record and replay debug intermittent mochitests! A word of warning: you need a modern CPU in order to get good performance. I had poor performance when running on my two-year-old Core2Duo laptop, but replay performance is almost at real-time speeds on my shiny new Intel i7 950 box.
I still have two patches that need to be refined and then checked in, to facilitate replay debugging. The first enables the mochitest harness to loop forever on a test directory. The second enables you to set break points on specific JavaScript dump() calls, so you can break during replay close to where the action is.
We're far from having a fully automated record and replay setup, but we've made a start!
I've documented how to produce a Replay Debugging setup for debugging intermittent test failures in Mozilla mochitests, and put it up on MDC:
https://developer.mozilla.org/En/Debugging/Record_and_Replay_Debugging_Firefox
Now anyone can setup a machine to record and replay debug intermittent mochitests! A word of warning: you need a modern CPU in order to get good performance. I had poor performance when running on my two-year-old Core2Duo laptop, but replay performance is almost at real-time speeds on my shiny new Intel i7 950 box.
I still have two patches that need to be refined and then checked in, to facilitate replay debugging. The first enables the mochitest harness to loop forever on a test directory. The second enables you to set break points on specific JavaScript dump() calls, so you can break during replay close to where the action is.
We're far from having a fully automated record and replay setup, but we've made a start!
Monday, 5 October 2009
Problems building Firefox with Visual Studio 2005 and Windows 7 SDK
When I tried to update my 1.9.2 tree this morning on my Windows box I quickly discovered that Ben Hearsum had recently updated our build system to require the Windows 7 SDK. However once I installed the new SDK and tried to build (using Visual Studio 2005's compiler), I got the following error:
shell32.lib(shguid.obj) : fatal error LNK1103: debugging information corrupt; recompile module.
This is a compatibility issue between Visual Studio 2008 and Visual Studio 2005 which only occurs when you compile in debug mode. There's a hotifx to resolve this issue. Once I installed this, I could magically build again. Thanks to biesi for pointing out the hotfix!
shell32.lib(shguid.obj) : fatal error LNK1103: debugging information corrupt; recompile module.
This is a compatibility issue between Visual Studio 2008 and Visual Studio 2005 which only occurs when you compile in debug mode. There's a hotifx to resolve this issue. Once I installed this, I could magically build again. Thanks to biesi for pointing out the hotfix!
Friday, 18 September 2009
Ogg video seek performance improvements
I've recently landed bug 501031 on mozilla-central and on 1.9.2 which roughly cuts Ogg seeking time in half. In Firefox 3.5, seeking Ogg Internet video was very slow, often taking 20 seconds or more to seek. This patch will make seeking in Ogg media in Firefox 3.6 much faster! This patch should also reduce the likelihood of encountering visual artefacts after a seek.
Currently the Ogg format doesn't contain any kind of keyframe index, so when you want to seek to a given time you typically do a bisection search over the entire file, reading little chunks as you go to figure out where your bisection has landed. This works fine for files on disk, but when seeking in files served over the Internet this can be slow, especially when viewing media which is hosted half a world away.
To play Ogg files, Firefox uses liboggplay, which in turn uses liboggz to seek. Unfortunately liboggz's seek bisection is in need of some maintenance (it's currently being rewritten by the maintainer). Its bisection is erratic and it fails to terminate its search appropriately, so it makes many more bisections than required. Most bisections or non-sequential reads result in a new HTTP connection, which is what makes this process slow for Internet video.
My patch fixes liboggz's seek to bisect sensibly, and to end the bisection search when it lands within 500ms of the seek target. This means that once we land close enough to the seek target, we'll just keep downloading from there. This is typically faster than continuing the bisection search due to the latency in setting up HTTP requests. We subtract 500ms from the seek target before we begin the bisection search, so that we don't finish the seek after the actual seek target.
Theora stores its video data as keyframe and interframes. In order to seek to and display a frame at a given time, we need to seek to the previous keyframe and decode forward to the target frame. The suggested approach is to extract the keyframe's position from the frame's granulepos field, and then seek again to the keyframe. This second seek needs to be exactly right, and that's hard due to some nasty edge cases with regards to stream muxing. Doing another bisection search in our case is also slow due to the latency in setting up HTTP requests. So now we just calculate the maximum possible time-offset that a frame can be from its keyframe, and subtract that from our seek target. This means we will often download more data than necessary, but for us that's typically faster than doing another bisection search.
The moral of the story is that if you want your video to seek quickly, include regular keyframes!
Currently the Ogg format doesn't contain any kind of keyframe index, so when you want to seek to a given time you typically do a bisection search over the entire file, reading little chunks as you go to figure out where your bisection has landed. This works fine for files on disk, but when seeking in files served over the Internet this can be slow, especially when viewing media which is hosted half a world away.
To play Ogg files, Firefox uses liboggplay, which in turn uses liboggz to seek. Unfortunately liboggz's seek bisection is in need of some maintenance (it's currently being rewritten by the maintainer). Its bisection is erratic and it fails to terminate its search appropriately, so it makes many more bisections than required. Most bisections or non-sequential reads result in a new HTTP connection, which is what makes this process slow for Internet video.
My patch fixes liboggz's seek to bisect sensibly, and to end the bisection search when it lands within 500ms of the seek target. This means that once we land close enough to the seek target, we'll just keep downloading from there. This is typically faster than continuing the bisection search due to the latency in setting up HTTP requests. We subtract 500ms from the seek target before we begin the bisection search, so that we don't finish the seek after the actual seek target.
Theora stores its video data as keyframe and interframes. In order to seek to and display a frame at a given time, we need to seek to the previous keyframe and decode forward to the target frame. The suggested approach is to extract the keyframe's position from the frame's granulepos field, and then seek again to the keyframe. This second seek needs to be exactly right, and that's hard due to some nasty edge cases with regards to stream muxing. Doing another bisection search in our case is also slow due to the latency in setting up HTTP requests. So now we just calculate the maximum possible time-offset that a frame can be from its keyframe, and subtract that from our seek target. This means we will often download more data than necessary, but for us that's typically faster than doing another bisection search.
The moral of the story is that if you want your video to seek quickly, include regular keyframes!
Wednesday, 5 August 2009
Configuring web servers for HTML5 Ogg video and audio
When serving HTML5 Ogg <video> or <audio> from your web server, there's a number of things you can do to make videos load faster. This post outlines how to configure your web server to improve HTML5 video and audio playback performance.
1. Serve X-Content-Duration headers
The Ogg format doesn't encapsulate the duration of the media. So for the progress bar on the video controls to display the duration of the video, we need to somehow determine the duration. You can either support HTTP1.1 Byte-Range requests (see 2. below) or better yet, serve an X-Content-Duration header for your Ogg videos. This provides the duration of the video in seconds (not HH:MM:SS format), as a floating point value. For example, a video which is 1 minute and 32.6 seconds would you'd serve the extra header: "X-Content-Duration: 92.6".
When an Firefox requests an Ogg media, if you should serve up the X-Content-Duration header with the duration of the media. This means Firefox doesn't need to do any extra HTTP requests to seek to the end of the file to calculate the duration so it can display the progress bar.
You can get the duration using oggz-info, which comes with oggz-tools. oggz-info gives output like this:
Note that you can't just serve up the Content-Duration line that oggz-info outputs, it's in HH:MM:SS.ss format, you need to convert it to seconds only, and serve it as X-Content-Duration.
Be warned, it looks like oggz-info makes one read pass of the media in order to calculate the duration, so it would be wise to store the duration value, and not to calculate it for every HTTP request of every Ogg video.
Also be aware that oggz-info does not calculate the duration of videos that start at a non-zero time correctly. oggz-info reports the duration as the time of the last frame, not the time of the last frame, minus the time of the first frame. Edit - 6 Aug 2009: Looks like this was only true for old versions of oggz-info, current versions use the presentation time from the skeleton track to calculate the duration correctly.
2. Handle HTTP1.1 byte range requests correctly
In order to seek to and play back regions of the media which aren't yet downloaded, Firefox uses HTTP1.1 Byte Range requests to retrieve the media from the seeek target position. Also if you don't serve X-Content-Duration, we use byte-range requests to seek to the end of the media (provided you're serving Content-Length) to determine the duration of the media.
Your server should serve the "Accept-Ranges: bytes" HTTP header if it can accept byte-range requests. It must return "206: Partial content" to all byte range requests, else Firefox can't be sure you actually support byte range requests. Remember you must return "206: Partial Content" for requests for "Range: bytes=0-" as well.
If you're curious, see bug 502894 comment 1 for more details of the HTTP requests Firefox can make and why.
3. Include regular key frames
When we seek, we have to seek to the keyframe before the seek target, and then download and decode from there until we reach the actual target time. The further your keyframes are apart, the longer this takes, so include regular keyframes. ffmpeg2theora's default of one keyframe every 64 frames (or about every 2 seconds) seems to work ok, but be aware that the more keyframes you have, the larger your video file will be, so your mileage may vary.
4. Serve the correct mime-type
For *.ogg and *.ogv files containing video (possibly with an audio track as well), serve the video/ogg mime type. For *.oga and *.ogg files which contain only audio, serve the audio/ogg. For *.ogg files with unknown contents, you can serve application/ogg, and we'll treat it as a video file. Most servers don't yet serve the correct mime-type for *.ogv and *.oga files.
5. Consider using autobuffer
If you have the autobuffer attribute set to true for your video or audio element, Firefox will attempt to download the entire media when the page loads. Otherwise, Firefox only downloads enough of the media to display the first video frame, and to determine the duration. autobuffer is off by default, so for a YouTube style video hosting site, your users may appreciate you setting autobuffer="true" for some video elements.
1. Serve X-Content-Duration headers
The Ogg format doesn't encapsulate the duration of the media. So for the progress bar on the video controls to display the duration of the video, we need to somehow determine the duration. You can either support HTTP1.1 Byte-Range requests (see 2. below) or better yet, serve an X-Content-Duration header for your Ogg videos. This provides the duration of the video in seconds (not HH:MM:SS format), as a floating point value. For example, a video which is 1 minute and 32.6 seconds would you'd serve the extra header: "X-Content-Duration: 92.6".
When an Firefox requests an Ogg media, if you should serve up the X-Content-Duration header with the duration of the media. This means Firefox doesn't need to do any extra HTTP requests to seek to the end of the file to calculate the duration so it can display the progress bar.
You can get the duration using oggz-info, which comes with oggz-tools. oggz-info gives output like this:
$ oggz-info /g/media/bruce_vs_ironman.ogv
Content-Duration: 00:01:00.046
Skeleton: serialno 1976223438
4 packets in 3 pages, 1.3 packets/page, 27.508% Ogg overhead
Presentation-Time: 0.000
Basetime: 0.000
Theora: serialno 0170995062
1790 packets in 1068 pages, 1.7 packets/page, 1.049% Ogg overhead
Video-Framerate: 29.983 fps
Video-Width: 640
Video-Height: 360
Vorbis: serialno 0708996688
4531 packets in 167 pages, 27.1 packets/page, 1.408% Ogg overhead
Audio-Samplerate: 44100 Hz
Audio-Channels: 2
Note that you can't just serve up the Content-Duration line that oggz-info outputs, it's in HH:MM:SS.ss format, you need to convert it to seconds only, and serve it as X-Content-Duration.
Be warned, it looks like oggz-info makes one read pass of the media in order to calculate the duration, so it would be wise to store the duration value, and not to calculate it for every HTTP request of every Ogg video.
2. Handle HTTP1.1 byte range requests correctly
In order to seek to and play back regions of the media which aren't yet downloaded, Firefox uses HTTP1.1 Byte Range requests to retrieve the media from the seeek target position. Also if you don't serve X-Content-Duration, we use byte-range requests to seek to the end of the media (provided you're serving Content-Length) to determine the duration of the media.
Your server should serve the "Accept-Ranges: bytes" HTTP header if it can accept byte-range requests. It must return "206: Partial content" to all byte range requests, else Firefox can't be sure you actually support byte range requests. Remember you must return "206: Partial Content" for requests for "Range: bytes=0-" as well.
If you're curious, see bug 502894 comment 1 for more details of the HTTP requests Firefox can make and why.
3. Include regular key frames
When we seek, we have to seek to the keyframe before the seek target, and then download and decode from there until we reach the actual target time. The further your keyframes are apart, the longer this takes, so include regular keyframes. ffmpeg2theora's default of one keyframe every 64 frames (or about every 2 seconds) seems to work ok, but be aware that the more keyframes you have, the larger your video file will be, so your mileage may vary.
4. Serve the correct mime-type
For *.ogg and *.ogv files containing video (possibly with an audio track as well), serve the video/ogg mime type. For *.oga and *.ogg files which contain only audio, serve the audio/ogg. For *.ogg files with unknown contents, you can serve application/ogg, and we'll treat it as a video file. Most servers don't yet serve the correct mime-type for *.ogv and *.oga files.
5. Consider using autobuffer
If you have the autobuffer attribute set to true for your video or audio element, Firefox will attempt to download the entire media when the page loads. Otherwise, Firefox only downloads enough of the media to display the first video frame, and to determine the duration. autobuffer is off by default, so for a YouTube style video hosting site, your users may appreciate you setting autobuffer="true" for some video elements.
Subscribe to:
Posts (Atom)