New accessibility features in Firefox 3.5

Firefox 3.5 is fast approaching, and it’s time to list all the user-visible changes to the accessibility support in this new version!

Support for text attributes, formatting and spell checking

Firefox 3.5 exposes text attributes such as bold, underlined, and color information through the AT-SPI and IAccessible2 attributes properties of their respective AccessibleText interfaces. Information about formatting such as left aligned, centered etc., is also provided. In addition, in rich editing and other text entry environments where the spell checker is enabled, when a word is misspelled, this information is also provided so screen readers can pick up and notify the user. Since my original blog post on this subject, Orca 2.28, NVDA 0.6p3 and JAWS 10 have added support for this feature, allowing a seamless proof reading of entered text in both Firefox 3.5 and the upcoming Thunderbird 3 release. This works also when writing a message in GMail or other rich editing environments, not just textboxes or textareas.

Better compliance with WAI-ARIA 1.0

We’ve made sure that the WAI-ARIA 1.0 spec is adhered to to the most extent possible, removing attributes that are no longer in the spec, and adding/changing those that were agreed upon in the progression towards finalizing ARIA 1.0, which is currently in a late review stage. One of the most significant attributes added is aria-label, which allows any text to be associated with a widget that doesn’t appear anywhere else within the web app. For extension devs: This also works in XUL, not just HTML. One project that makes heavy use of this is Firebug 1.4 in the accessibility UI enhancements that Hans from the Paciello Group has put in. This is also the reason why Firebug 1.4 works better in Firefox 3.5 than 3.0, what accessibility is concerned, since Firefox 3.0 doesn’t support this new attribute.

Also, aria-expanded can be used on all elements now, allowing better exposure of states for buttons that drop down a list of items, for example.

Support for the exposure of embedded HTML 5 audio and video controls

As recently announced, we’re also supporting the exposure of the embedded controls of the HTML 5 audio and video elements to assistive technologies.

Better event firing in dynamic web applications

We fixed a significant issue with firing proper events when nodes get hidden or made visible through JavaScript or Ajax calls. This should allow a much better experience with accurate visibility of nodes within virtual buffers of various screen readers.

And a ton of bug fixes for stability

Of course, each cycle also goes with a ton of bug fixes that improve stability and accuracy. These are mostly under the hood and often deal with edge cases, but these are no less important to our user base.

When Firefox comes out, I encourage everyone to upgrade as soon as possible, since it will provide an even more rich experience when browsing the web than Firefox 3.0 already did. Probably the most important extension for blind users, WebVisum, already works with this release, so you won’t lose anything on that front! Also, other extension devs are working hard to make their projects work with Firefox 3.5.


My first experience using an accessible touch screen device

Yes, you read correctly: An accessible touch screen device! This morning, I went to a retail store carrying mostly Apple products and had a look at the new iPhone 3G S that was released in Germany on Friday. Apple revealed during the WWDC keynote two weeks ago that it would have a built-in screen reader named the same as is included in Mac OS X: VoiceOver. This is a feature not available on the regular iPhone 3G, as its hardware capacity is insufficient.

I was not at all sure what to expect. From reading a bunch of posts on the VIPHone Google Group, I knew that people were going through a learning curve, a steep one at times. Up to now, something usable via a touch screen or touch-only keys would always mean a dead-end to me and other blind people. The iPhone 3G and the iPod Touch are not usable for me. Likewise, elevators that have keys you only need to touch, not press, to get toa different floor, are a real challenge. In fact I once tripped an alarm while trying to use such an elevator, alone int he cabin and touching the emergency button accidentally.

When I arrived at the store, I had already made arrangements with them to be allowed to take an in-depth look at the 3G S. As we went over to the iPhone stand, one of the sales assistants already knew how to turn on VoiceOver. Apple are documenting this in the regular iPhone user’s manual, no special docs needed. The assistant helping me turned it on, and a clear crisp voice came out of the built-in speakers. She was a bit confused by the changed gestures. I had done some reading, and took over from there.

And I must say this was an amazing experience! My fingers definitely need to get used to gestures such as flicking or tapping, or using a rotor. But having an iPod Nano 4th generation helped with that, since moving the finger over the screen like on a dialer felt most like tracking around the iPod’s click wheel. Even the sound the rotor makes is the same. 🙂

Responsiveness to gestures was amazing. I own an Nokia N82, which is to date probably the handset that reacts fastest to keyboard commands with the Talks or MobileSpeak screen readers, but the responsiveness on the iPhone beats that by lengths!

Finding my way around the iPhone’s UI took some getting used to. Traditional mobile screen readers, also like most Windows or Linux screen reader solutions, give the blind user a filtered view of the world, by default constrained to the focus location. Only on demand can one explore the screen using mouse emulation or similar techniques. On the iPhone, you interact with the real thing right from the start. You touch the screen in the lower half, somewhere on the right, and you’re told that the Safari or iPod symbol is there on the Home screen. You move your finger to the left, and you’re told what’s right next to it. To interact with the menu bar of the Phone app, you need to move your finger down to the bottom and move from left to right to hear the options such as “Contacts” or “Phone pad”. Yes, there are VoiceOver gestures to explore the screen top to bottom, left to right. You do this by flicking left to right anywhere, and the accessible controls are being walked one by one. But the interaction model is very close to the actual screen layout most of the time. This tremendously helped when I walked through a couple of applications with the sales assistant standing next to me. She could literally point me to the correct spot, and VoiceOver would speak what I needed to hear. Or she could give me verbal directions, and my finger would find the controls.

Typing is probably going to take the most adjusting. It is nothing like typing on the number pad of my N82. James Craig’s typing tip for VoiceOver on iPhone helps a lot: You look for the correct key with one hand, keep your finger there, then tap somewhere on the screen with another finger from the same or the other hand, and the character is input. Gladly, the keyboard doesn’t change position, and after a few letters I had a very good idea where each letter should be, and my typing sped up within 10 typed letters already. In addition, one can turn on word prediction/completion, which is another accessibility feature that can also aid people with motor impairments make typing easier. It plays nicely with VoiceOver.

This is by far not a comprehensive review or comparison. I couldn’t use many of the features since the SIM card in that exhibited model was locked, and I don’t have my own model yet.

Apple are speeding ahead and breaking down conventions in accessibility, or as Mike Calvo of Serotek wrote: They’re getting to the future first. They’re the first to include a screen reader for the blind on one of their mainstream models. Google are going to do something similar with their G1 efforts. The API is there, and some basic console work seems to be working already, but this is by far not as comprehensive as what Apple are doing. RIM also have an accessibility API, but from what I’m told, the screen reading solution that has been hinted every now and then over the past couple of weeks is going to cost extra money, which Apple’s solution does not. The traditional mobile accessibility solutions on Windows Mobile and Symbian S60 all require an additional payment of $200 to $350 for a screen reading solution, or in some cases even proprietary hardware that then costs $2000 or even more.

And this, of course, opens up other possibilities for future implementations of touch screen use cases, not just by Apple, but by other companies as well.

And one more bit of info: The gestures and touchy interface also come to VoiceOver in Snow Leopard with compatible new MacBooks with the multi-finger trackpad. So whenever a colleague tells me to lok for something in the top right quadrant of the screen, I can do that once I have Snow Leopard running on my MacBook. I’ll just put my finger there and let VoiceOver tell me what’s there!

Now my only problem is to get an iPhone. It would appear that my current contract doesn’t allow me to upgrade, since I upgraded it only recently, but too long before I knew the 3G S was coming. We’ll see how I get my hands on a device, it’s not freely available without contract in Germany.

My first touch screen experience was an amazing one!

Exposure of audio and video elements to assistive technologies

This blog has been quiet recently due to me being away at the AEGIS forum and workshop near London last week (more on that to come soon), and us preparing everything for the Firefox 3.5 release.

If you’re using Firefox 3.5b4, you may already have been notified of the Firefox 3.5b99 update. This update brings exposure of the HTML5 audio and video elements through the MSAA/IAccessible2 and ATK/AT-SPI accessibility interfaces, further strengthening our commitment to Open Video.

What this means is that, using NVDA or Orca, you now have access to an HTML5 audio or video file embedded in a web page. If the controls attribute is specified within the audio or video tag, Firefox 3.5 creates its own set of playback controls. These are now exposed to screen readers using MSAA or AT-SPI to traverse our trees. Exposure through iSimpleDOM interfaces on Windows will come at a later stage (see this bug). Previously, these controls were unreachable. They did not appear inside the virtual buffer.

In addition, a keyboard navigation bug was also fixed allowing control of most of the functions of the embedded controls. Note that the individual buttons and sliders are not yet tabbable, we’ll be working on a good keyboard interaction model and hopefully improve the current model in a point release.

To test the video element, with NVDA as an example, do this:

  1. Go to this video.
  2. Using NVDA’s quick key b, find the Play button.
  3. Press Space.
  4. To interact with the audio/video element directly, press NVDA+Space to turn to Focus Mode temporarily, and Shift+Tab once to land on the video grouping.
  5. You can now use the following keyboard shortcuts:
    Shortcut Action
    Space Play/Pause
    right arrow and Left arrow Move through the video in 5 second intervals.
    Ctrl+Right Arrow and Ctrl+Left arrow Move through the video by 1 minute increments.
    Up arrow and Down arrow Increase and decrease the volume. This does not change the volume of your speech synthesizer!
  6. After you’ve finished interacting with the video controls, press NVDA+Space again to turn virtual cursor back on.

Note that the ultimate goal is to not require you to turn focus mode on and off with NVDA+Space, but to allow you to interact with each scrubber/slider using the regular Focus mode command Enter and Escape, however we’re not quite there yet (see the linked bug above).

BTW: the above keyboard shortcuts also work if you don’t have a screen reader running. You need to tab to the video element grouping and then use the shortcuts as described.

Note also that due to our not exposing the elements through iSimpleDOM yet, some screen readers may not see the controls yet. In that case, turn off your virtual cursor and tab to the video element to use the keyboard shortcuts to control your video playback.

Stay tuned for more updates as they come along!