Thursday, 22 September 2011

Mozilla full-screen API progress update

Update 10 November 2011: the full-screen API has been changed slightly and enabled in Firefox Nightly builds, see http://blog.pearce.org.nz/2011/11/firefoxs-html-full-screen-api-enabled.html for details.

I've been working on implementing Robert O'Callahan's HTML full-screen API proposal in Firefox (bug 545812). Support for the base API has landed, disabled by default, in Firefox nightly builds. To enable the full-screen API, set the pref full-screen-api.enabled to true.

We have implemented a general purpose full-screen API which can make any HTML element the full-screen element (it seems WebKit based browsers' full-screen API allow only making <video> elements full-screen).

This feature makes the following API changes to HTML Element:
  1. void mozRequestFullScreen() : makes an HTML element the full-screen element. Causes browser chrome to hide, and expands the element to encompass the entire screen. Upon success, this dispatches a "mozfullscreenchange" event to the requesting full-screen element, or the element's owner document if the element is not in a document. We only grant requests for full-screen when running in user-generated event handlers, e.g. a mouse click handler.
This feature makes the following API changes to HTML Document:
  1. void mozCancelFullScreen() : exits the document from full-screen mode.
  2. readonly attribute mozFullScreen : true when the document is in full-screen mode.
  3. readonly attribute mozFullScreenElement : reference to the current full-screen element, if it's in the current document.
This feature adds the :-moz-full-screen css pseudo class, which applies to the full-screen element while in full-screen mode.

For a request for full-screen to be granted in content inside an iframe, the containing iframe needs to have the mozallowfullscreen attribute present. This is a boolean attribute, so the attribute only needs to be present, it doesn't matter what value it's set to.

Keyboard input is restricted in full-screen mode. When alpha-numeric key input occurs in full-screen mode, full-screen mode immediately exits. This is to help protect against phishing attacks.

We also plan to deny requests for full-screen mode when windowed plugins are present (since we can't easily monitor key events to windowed plugins on non-MacOSX platforms). We will exit full-screen mode when a windowed plugin is added to a document as well. I have a patch for this, but its dependencies haven't landed yet.

Work remaining to be done before this can be enabled:
  1. Adding a warning message when we enter DOM full-screen mode (on desktop Firefox, and on Fennec too).
  2. Making the full-screen API work in multi-process Firefox/Fennec (bug 684620). This requires a way of getting the PBrowserParent from C++ in the chrome process to be implemented, there's not a way to do that yet unfortunately.
  3. Make change/open tab cause full-screen mode to exit (bug 685402).
  4. A security review must be completed, and concerns raised there must be addressed. This could involve changing the API.
We also want a clearer transition effect when entering full-screen, to somehow show the full-screen element "stretching out" to encompass the screen.

You can test out our work-in-progress full-screen implementation, by grabbing the latest Firefox nightly build, setting the pref full-screen-api.enabled to true, and pointing your browser at my not-very-exciting full-screen API demo page.

10 comments:

discon said...

I understand the reasoning behind blocking alphanumeric input but doesn't this way make it useless for the applications that would benefit from a full-screen the most? There can be no full-screen office applications or serious games without keyboard support. No WebGL MMORPG or FPS with WASD movement. Ever.

Blocking alphanumeric input isn't enough anyway. A malicious page can just show a virtual keyboard and ask the user to use that for "increased security" (like many banks do), along with a line that says it's a cool new beta feature.

Without giving it much thought, wouldn't it be better to simply overlay the domain name along with it's secure status on a fixed (or not) position on the screen every time a key event occurs and auto-hide it after 10 secs? Full-screen video would still work and games would learn not to use that part of the screen for anything crucial.

I'm sure this has it's own problems but I'm also sure that Mozilla can come up with something better.

Chris Pearce said...

@discon: we haven't ruled out supporting alphanumeric key input - the original proposal includes it - we've just not implemented it in this first round.

I'm fully aware WASD is desirable for FPS full-screen games. Most of the benefit of going full-screen for games is mouse capture anyway, and with the mouse capture API under development (https://bugzilla.mozilla.org/show_bug.cgi?id=633602) the need for full-screen for games is diminished.

We plan to show a warning when a user enters full-screen, and requests for full-screen are only granted in the first second of a user generated event handler, so a user should be aware that they've gone full-screen mode and are thus vulnerable to phishing. The transition effect to "stretch" the full-screen element to encompass the screen should also make it obvious the user has gone full-screen.

Francois said...

Hi Chris. Not sure if you've come across our project yet but we really have hopes for Fullscreen API.

We're building a Web Game Platform that enables you to use your Phones/Tablets as Controllers for Web Games. See our video:
http://brassmonkeysdk.com/

See video of Emotely, now a part of Brass Monkey:
http://emotely.com/

The hope for us would be that you can initiate fullscreen mode even without typical user input, otherwise you'll have to get off the couch to do so. For security I assume this means there'd have to be a permissions dialog that came down like geo-location and asked if the user was ok with our website initiating full screen mode. Hopefully only the first time and then it is remembered.

Because of our approach we don't the mouse lock issues that web based First Person Shooters might have, full screen or no. While I agree with your point that that is one of the biggest benefits of fullscreen. I think besides immersive-ness a larger benefit that full screen mode can have is performance. My understanding though is that full screen mode can't set pixel resolution so is missing out on that benefit. Games/Apps are more enjoyable often at lower resolutions but higher frame-rates.

I should probably be talking about this on Mozilla's IRC or something.

We're about to submit our app in the next week or so. Keep an eye out.

Pike said...

FWIW, I've added fullscreen support to https://github.com/maccman/flarevideo. Still buggy, I misread the mozCancelFullScreen api, but I'll still patch that.

Sadly, video perf in that player seems to be much worse than in a plain video element :-(

Chris Pearce said...

@Francois: emotely looks cool!

If full-screen can be initiated from non-user-generated event handlers, then it may not be obvious to the user they've entered full-screen mode. Displaying a permission dialog alleviates this, but then you require two clicks to enter full-screen mode, rather than one.

Re: screen sizes, you can still choose your canvas size!

@Pike: Full-screen performance is not good when you've got hardware accelerated graphics disabled, as is the default in Firefox on Linux due to crappy drivers.

Pike said...

I'm on a mac. And I compare the themed player to our default video controls, and it just starts to stop every other second. I suspect it does some overlay/composition thing in a way that makes us allocate/deallocate/gc or something too badly too frequently.

Chris Pearce said...

@Pike: Interesting. Can you post a link to a page which exhibits this problem?

Francois said...

@Chris I'd love to chat more. Mind doing a phone call soon. I think I have some ideas about this, of course we have our own motivations being a controller company, but the same can be said for any controller project people. So many right now.

I meant more, that if a call is made to programmatically go full-screen that isn't from user input then it asks you if you want to give permission to the page to be able to initiate full screen without direct input. This could be a 'yes just this time, no, , and yes always' choice set.

About the screen resolution thing. Setting the canvas size can work, but canvas doesn't scale up efficiently. Also we have this other issue with leveraging browser full screen for Flash/Unity-plugin based games where they also don't scale up efficiently. I highly doubt changes would go in that helps alleviate Flash/Unity issues, but it is something to consider.

Please add me on Skype at 'emotely', I'd really appreciate the chat. There are many angles to this that I've been thinking about inside and out for a year. Kind of a mess right now.

I'd love to get you a sneak peak at it, to show you by example what we're missing out by not having it. Manually setting fullscreen once and then sitting back on the couch isn't perfect but it's pretty good. For our situation you need to manually open the web page as it is, so have regular input initiated fullscreen at that point is sufficient and in browsers were it isn't supported programmatically we're gonna just have a pop down dialog with instructions.

Pike said...

I have http://people.mozilla.org/~axel/fullscreen/ up, I don't think the video is huge enough to show it. Maybe just grab that site and update it with some HD stuff? Don't feel like putting that on people :-/

Robert said...

Francois: scaling up a canvas (or video) is practically free in Firefox when GPU acceleration is enabled.