Exposure of audio and video elements to assistive technologies

This blog has been quiet recently due to me being away at the AEGIS forum and workshop near London last week (more on that to come soon), and us preparing everything for the Firefox 3.5 release.

If you’re using Firefox 3.5b4, you may already have been notified of the Firefox 3.5b99 update. This update brings exposure of the HTML5 audio and video elements through the MSAA/IAccessible2 and ATK/AT-SPI accessibility interfaces, further strengthening our commitment to Open Video.

What this means is that, using NVDA or Orca, you now have access to an HTML5 audio or video file embedded in a web page. If the controls attribute is specified within the audio or video tag, Firefox 3.5 creates its own set of playback controls. These are now exposed to screen readers using MSAA or AT-SPI to traverse our trees. Exposure through iSimpleDOM interfaces on Windows will come at a later stage (see this bug). Previously, these controls were unreachable. They did not appear inside the virtual buffer.

In addition, a keyboard navigation bug was also fixed allowing control of most of the functions of the embedded controls. Note that the individual buttons and sliders are not yet tabbable, we’ll be working on a good keyboard interaction model and hopefully improve the current model in a point release.

To test the video element, with NVDA as an example, do this:

  1. Go to this video.
  2. Using NVDA’s quick key b, find the Play button.
  3. Press Space.
  4. To interact with the audio/video element directly, press NVDA+Space to turn to Focus Mode temporarily, and Shift+Tab once to land on the video grouping.
  5. You can now use the following keyboard shortcuts:
    Shortcut Action
    Space Play/Pause
    right arrow and Left arrow Move through the video in 5 second intervals.
    Ctrl+Right Arrow and Ctrl+Left arrow Move through the video by 1 minute increments.
    Up arrow and Down arrow Increase and decrease the volume. This does not change the volume of your speech synthesizer!
  6. After you’ve finished interacting with the video controls, press NVDA+Space again to turn virtual cursor back on.

Note that the ultimate goal is to not require you to turn focus mode on and off with NVDA+Space, but to allow you to interact with each scrubber/slider using the regular Focus mode command Enter and Escape, however we’re not quite there yet (see the linked bug above).

BTW: the above keyboard shortcuts also work if you don’t have a screen reader running. You need to tab to the video element grouping and then use the shortcuts as described.

Note also that due to our not exposing the elements through iSimpleDOM yet, some screen readers may not see the controls yet. In that case, turn off your virtual cursor and tab to the video element to use the keyboard shortcuts to control your video playback.

Stay tuned for more updates as they come along!

Show Comments