De-XPCOM-ing the Accessible module

XPCOM is used to communicate to the Mozilla core, getting information out of and into it from languages such as JavaScript, Java, Python and C++. Unfortunately, due to historic reasons, modules internal to the Gecko platform also used to use XPCOM to get information. One of these modules was (and partially still is) the Accessible module. XPCOM, while very powerful, also has some performance limitations when querying for a lot of information via QUERY_INTERFACE. Therefore, over the past couple of months, Alex and David have been working on de-XPCOM-ing our module to make it more performant and ready for the future.

To the end user, this will feel more performant especially on complex pages.

Event management

Firing events, and calculating when and how to fire them, has been a big performance killer for the Accessible module in the past. While this was for the most part not particularly noticeable for screen reader users, since we are sort of limited by the rate our synthesizers talk, recently more and more technologies have started using the Accessibility services. Fingerprint devices to enter passwords into Firefox, tablet PC interfaces etc., all use parts of the Accessible module. Since there is currently no way to turn parts of it on or off, as soon as any piece of software accesses just a single accessibility service, the whole engine gets started, and from that point on, all calculations take place as if a screen reader for the blind was present.

Again, since this cannot currently be selectively turned off, and it is not certain that this will ever be possible, it is our goal to make this fact the least noticeable to users. To that end, we’ve started a project called event coalescing. Event coalescing will, as the name suggest, coalesce the stream of events we get from the DOM to only fire those event assistive technologies absolutely need. However, the rules have not been finalized yet, and the code as it is in the current platform based on some initial ideas and feedback, is not very performant, in parts even less performant than in Firefox 3.6, which fires more events but calculates less before sending them.

For those interested in the very technical details, the wiki page for event coalescing is here.

HTML5 form element enhancements

While we’re working hard on improving performance, work in other areas of Gecko is also progressing, requiring us to work together to make sure these enhancements are also accessible. One of these is the work that needs to be done to make new additions that HTML5 brings to form elements accessible. I wrote up a summary with bug links and some information here. If you have feedback on the ideas and proposed roles, you’re welcome to contact us by leaving a comment down here or on the #accessibility IRC channel.

UI work

As the road to the next major release of Firefox is being walked, there are also some UI redesigns happening. One, which has already landed, was the conversion of the tab strip to a real toolbar. My job was to make sure screen readers can still cope with them after this change. Some minor regressions were found, but all in all this still works great.

Another, much bigger change, is the redesign of the Add-Ons Manager to include support for language packs, Jetpacks and other ways to extend Firefox. This work is still underway, and I recently took a first round of testing. Some keyboard navigation issues, and one XUL markup issue that needs to be addressed, but so far, although this is a major UI change, things look very accessible, and I’ll make sure it stays that way.

Other areas

Work that is not directly related to the Gecko platform, but which is also important in the field of accessibility, is, among others:

• Thunderbird 3.1 Beta2 has been released recently. Thunderbird 3.1 comes with the same platform version as Firefox 3.6. Due to the table enhancements in the Gecko 1.9.2 platform, the message list and other areas will be much more accessible to screen readers. Information such as the unread status, the presence of attachments etc. can now be queried using the IAccessibleTable2 interface or equivalent on other platforms. The control is now a real table control rather than a flat ListView on Windows, giving much more accurate semantics.
• The Instantbird team has also done a great job at providing good XUL markup in their multi-instant-messaging client. We’re currently working together to work out some very specific problems with buddy status and others, but Instantbird is a great multi-messenger worth trying!

As you can see, lots of exciting stuff happening, some of it very user-visible, other more technical and very low-level in nature, but all exciting!

Stay tuned!

Update: Article translation

Thanks to community member Patricia Clausnitzer, this blog post is now available in Belorussian. How cool is that! Thank you very much! And BTW: This is a first!

The Mozilla Foundation had a booth at CSUN for the fourth year in a row. David, Alexander Surkov and I were present to man the booth, talk to people, and also participate in a couple of general sessions at the conference to gather information and news, and also to network.

Adobe announces broad range support for IAccessible2

One of the biggest news bangs to come out of the conference is Adobe’s announcement to support the IAccessible2 and WAI-ARIA standards in thenext versions of their Flash and Flex products. Both these standards were heavily driven by, among others, Mozilla, IBM and several assistive technology vendors such as NV Access of the NVDA project. Support for the native GNOME and Mac OS X accessibility APIs is also in the works.

In addition, Adobe announced that they will also include IAccessible2 support in their Acrobat and Reader products.

This means that another big player in the software industry is coming forward and supports these widely recognized standards. It is good to see Adobe getting behind the over-all accessibility efforts and helping to drive adoption in this manner!

Three Firebug-related sessions

Hans Hillen of the Paciello Group had two very successful talks about the UI accessibility support in Firebug. The first was a demo of many of the features, using NVDA as the screen reader to demo them. the second was a use-case talk, where Hans explained in some more technical detail how he went about making the Firebug UI accessible to screen reader users.

Both talks were very well received. The first one had quite a broad audience, while the second audience was smaller, but very focused and involved.

In addition, Jon Gunderson of the University of Illinois at Urbana-Champaign held a talk on the Accessibility Testing Extension for Firebug. But unfortunately, due to my travel schedule, I did not have a chance to visit this talk.

It was good to see two Mozilla grantees doing talks at this year’s CSUN, giving visibility to the many facets of Mozilla’s accessibility strategy.

Newer mobile accessibility technologies marching forward

Apple, RIM and Google, the three vendors of mobile devices with well-defined accessibility APIs, all had well-visited talks at CSUN. In addition, I am aware of at least two talks involving the accessible iPhone and iPod Touch 3rd generation that put these technologies to good use to provide a new generation of assistive software, built on mainstream devices.

Well-visited booth

The Mozilla Foundation booth was well visited on all three days that I helped staff it. Comments and questions ranged from the very flattering “I love Firefox and I love what you guys are doing for accessibility!” to “What’s a browser vendor doing at this conference?”. When we then explained why we attended, many of them were keen on trying out Firefox when they got home or back to thheir hotel rooms.

Also, this conference made quite a number of people aware of other Mozilla products than Firefox. While many had heard about Firefox, they had not heard at all about Thunderbird before. But with the better accessibility in Thunderbird, we can now change this and spread Thunderbird in the accessibility community even more!

I personally had a very moving moment on Friday when a deaf/hard of hearing gentelman and his interpreter stepped up to our booth. He was very interested in what we do for accessibility. Before I knew it, I was talking to him through his interpreter, but wasn’t actually noticing it until well into the conversation. At some point, I mentioned Thunderbird, at which point he started joking about the Ford Thunderbird. David, who was present at this conversation, can probably tell a bit more about this, since this was very visual and I only got a third of what he was actually meaning.

David and Alex also took a lot of pictures, which they’ll hopefully upload and share very soon so you all can get a better picture about what CSUN 2010 was like! Mozilla received a big big chunk of good attention, our funding of other accessibility-related open-source projects such as NVDA, Orca and others, definitely is being recognized in the industry as being exemplary. Also, we got a very nice compliment from a gentleman from the Office of homeland security, who told us that he thought our Voluntary Product Accessibility Template is among the best he has encountered so far.

One big failure is there, though

One big problem, which I think should not go unmentioned, is the lack of good internet connectivity in the exhibition hall. For a 2010 information technology conference, having no useable WIFI connection down in the exhibition hall at all is simply unacceptable. The internet connections that were offered were hideously priced, almost like in the mid 1990s when internet connectivity was still not as common as today. Up in the session rooms, the situation was a bit better, at least there were hotspots one could use most of the time.

For next year, one thing I’d like to see is a well thought-through strategy for free wireless internet connectivity throughout all conference locations. A technology conference lives and breathes with the buzz people can create around it by tweeting, uploading pictures etc. People with disabilities are no exception, and instead of roadblocking it, the responsible powers at CSUN should embrace this trend and encourage people to get the word out as easily and hazzle-free as possible!

In summary

I can only say that it was worthwhile going to CSUN yet again, and I am hoping we’ll have a chance to participate next year as well!

Support for voice dictation software in Windows Vista and Windows 7

Firefox 3.6 introduces support for the Microsoft Text Services Framework. Among other things, this allows users to dictate into text fields on the web using the Microsoft voice dictation software that comes with Windows Vista and Windows 7.

Because this is fairly new technology to Firefox, and because there are undoubtedly quirks to iron out, this support has to be specifically enabled in the advanced configuration editor. To do this:

1. In an empty tab, type about:config in the awesome bar.
2. Acknowledge the warning by clicking the “I’ll be careful, I promise!” button.
3. Type the letters tsf in the filter textbox
4. In the list that appears, select the preference intl.enable_tsf_support. This is off by default (its value is FALSE).
5. Right-click that preference and choose the “Toggle” menu option. This will change the option to read TRUE.
6. Restart Firefox.

This setting will now be saved in your profile and the functionality is available to you.

This expands the range of supported accessibility-related APIs yet again and offers good integration with one more native feature of the Windows Vista and 7 operating systems. Now, users with typing difficulties can use Firefox in a more efficient manner than before.

Windows 7 task bar integration

The new task bar integration in Windows 7 is accessible. If you have more than one tab open, and you navigate the task bar using the keyboard, a screen reader such as NVDA will announce “sub menu” for the Firefox task bar icon. This means that you can use the up and down arrow keys to select the tab you want to bring to the foreground when you switch to Firefox. This is just as seamless as sighted users will choose the tab preview from the Windows 7 task bar using the mouse.

More consistent focus handling

This affects everyone, keyboard and mouse users alike, but is mentioned here nevertheless since it brought us a few bug fixes for free like more consistent tabbing on pages that have the tabindex attribute on some elements but not others. Also, when saving executable files on Windows, the dialog that comes up prompting to save the file is now automatically being announced by screen readers.

Support for the IAccessibleTable2 interface

I blogged about this in more detail here. This gives screen readers access to all kinds of table structures, be it ordinary data tables, ARIA tree grids, XUL tree tables and other possible table constructs, in a unified and consistent manner.

More consistent and maintainable naming rules

Also in line with the user agent implementor’s guide for WAI-ARIA, we’ve improved the way we calculate the accessible names (often similar to on-screen text) of various elements in HTML and XUL. This makes our code more robust, predictable and maintainable, and therefore will allow easier adding of new features/element support in the future.

Notifying screen readers when an object attribute’s value changes

For better support of WAI-ARIA Drag And Drop, we’ve added support for the IAccessible2 object attribute changed event. This event notifies screen readers when an accessible’s object attribute’s value has been changed by the page. This usually happens if a certain HTML element’s attribute is exposed via its corresponding accessible’s object attributes, and that element’s attribute value is changed by a user action (usually done via JavaScript).

And again tons of bug fixes

Of course, there have also been a good number of fixes for existing features that were reported to use by users and assistive technology vendors alike. We’ve also kept track and participated in last-minute changes to the WAI-ARIA spec and kept our implementation up to date.

The whole accessibility team hopes that you’ll enjoy using this new version of Firefox as much as we enjoyed creating and testing it!

And boy, was I jealous! All my sighted friends could try out this cool new thing, an alternative browser to the omnipresent, and some would even say, omnipotent, Internet Explorer! An alternative! But it was not yet fully accessible. There were first glimpses: When I ran Firefox 1.0 with JAWS 6.0 back then, I could navigate the menus, and most dialogs talked. But the main functionality, the browsing experience, was not yet available to me.

That all changed with Firefox 1.5 and the donation IBM gave to the Mozilla project in August 2005. At the same time, Window-Eyes 5.5, and a bit later JAWS 7.0 were released with initial support for Firefox 1.5. Suddenly, the virtual cursor worked, quick navigation keys brought you from heading to heading or form field to form field, just like in Internet Explorer 6! And boy, was it fast!

It also had a ton of problems. Pages not rendering completely, esp pages with frames giving users huge headaches by not rendering at all often times, etc. Many users including myself had to decide that Firefox was just not ready for prime time use on a daily basis for screen reader users just yet.

But I kept up with development and also tested Firefox 2.0, which came out in October 2006. Many of the problems were still there, but some were a lot better. And the initial support for what back then was still known as “dynamic HTML” surfaced, giving a first glimpse of what later became WAI-ARIA and which has already changed the landscape of accessible web applications.

In December 2007, I started as Mozilla’s Accessibility QA engineer and took matters into my own hands. For Firefox 3, the accessibility community accomplished a huge lot: Firefox 3 was the first UI browser to become accessible on Linux. There was now an open-source screen reader taking full advantage of all the new features of the IAccessible2 standard on Windows. WAI-ARIA compliance was very high. All content-related problems were a thing of the past and the browsing experience was as reliable in Firefox as it was in IE. Moreover, value was added by projects such as WebVisum, which is a social tagging and captcha solving service exclusive to blind users using Firefox. When Firefox 3 was released in June of 2008 and we got in the Guinnes book of world records, I’m sure there were quite a number of blind folks participating in that effort!

But we didn’t stop there. For Firefox 3.5, we added support for text attributes, giving blind users the ability to in-line spell check their entries on the web just like any sighted people do. The community worked together to give open audio and video to everyone, not just sighted people, but also screen reader and keyboard users. WAI-ARIA compliance was increased to nearly 100%. Stability was further increased. Two more Windows screen reader vendors stepped up to support Firefox, making the total number of screen readers for the blind supporting Firefox 3.x 5 on the Windows platform.

And on its 5th birthday, my colleagues in the QA team and I are working hard to get a beta refresh out to all of you of the next major release, Firefox 3.6. In fact I’m writing this blog post using the candidate build. And we have a cool set of features assembled for Firefox 3.6 on the accessibility side, about which I’ll blog next.

All I can say is: Thank you Firefox! Thank you to everyone in the community who has been and is working so hard to make the web a better place for everyone! I’m grateful and proud to be a part of this community!

Happy birthday, Firefox!

We initially fixed this for Firefox 3.6 alpha and now also backported the fix to the 3.5 branch.

WAI-ARIA landmark support

When in virtual buffers, D and Shift+D can be used to skip between WAI-ARIA landmarks. Landmarks are also announced while reading a web page. The new Elements List also has a section for landmarks. Even the possible nesting of landmarks is announced.

WAI-ARIA Drag And Drop support

NVDA now supports WAI-ARIA Drag and Drop, with some help from Firefox 3.6 and later.

More features

• Sounds can now indicate the switching on and off of Focus Mode. Sounds are the default setting, but you can switch back to using indication via speech.
• N and Shift+N can be used to skip past blocks of links to the next/previous block of non-link text.
• On pages that take longer than 1 second to load, you can interact with your system while the page is being rendered. NVDA will tell you that it is processing the page, and it will no longer block the system while doing so.

Also, the Flash and Java interaction model discussed in an earlier post are included in this beta.

For more new feature information, I suggest studying the What’s new document and try out the beta for yourself!

Traditionally, Adobe Flash content is being rendered into the virtual buffer in Windows screen readers such as JAWS. Over the years, this has proven to cause several issues:

• Dynamic content frequently updating causes the virtual buffer to either get out of date, or to update so frequently that reading the content is close to impossible.
• Accessing content can be cumbersome if Forms Mode or similar concepts are not properly handled.

For these and other reasons, the WebAIM screen reader survey taken last year ranked Flash as the technology posing the most accessibility obstacles on the web for blind users. 71% of all participants found Flash to be difficult or extremely difficult to use. Inaccessible Flash applets, which take up the vast majority of Flash content out in the wild, are doing the rest to strengthen this view.

With Java applets, things get even more complicated. For one, one has to install the Java Access Bridge from Sun Microsystems, to get Java to be accessible at all, inside the browser and elsewhere. Once that hurdle is taken, Java applet content is not rendered inside the virtual buffer, but is present somewhere within the browser window, and one usually has to try to tab to it and get focus to it that way, outside the context of the virtual buffer. Accessible Java applets are very rare and currently hardly play any role when considering accessibility on the web. If at all, they’re viewed as obstacles and something to be avoided.

However, this could change with the approach the NV Access team is taking. In their latest snapshot build, they are introducing an interaction model that is remotely similar to what blind Mac users have come to know and appreciate from Apple’s VoiceOver screen reader. What you do is this:

1. You load a web page that contains Flash content. For example, take any YouTube video.
2. You navigate to a spot that says “embedded object clickable” with the normal virtual buffer navigation methods. For easiest access, NVDA provides the quick navigation key o to get to embedded objects.
3. Press Enter.
4. What this does is zoom in on the embedded Flash object and give it focus. Now, use Tab and Shift+Tab to navigate around the Flash app. Other keys such as the arrow keys also will perform differently now, for example left and right will scrub through the video on YouTube.
5. When done, press NVDA+Space to leave the embedded object and zoom out, returning to the parent web page. Your virtual buffer navigation will now function the same way as it did before you zoomed in on the Flash.

One note of caution: I fell into the trap that I thought the content would be rendered in the virtual buffer, as is traditionally done with Windows screen readers. To be honest, I didn’t read the note on this feature before I played with it. But if you don’t tab after pressing Enter, you will immediately leave the embedded object and continue navigating with the virtual cursor. This is because Flash does not focus any particular object inside the applet by default when the applet itself gains focus.

When I tried this on YouTube earlier, I had the feeling I had never seen so many details of the YouTube player before!

One more thing: The above technique will work in Firefox 3.5.x and the latest Minefield nightly builds, and it will also work in the 3.6b1 that’ll be available some time soon, but is not going to work in the 3.6alpha release we issued beginning of September, due to a regression that only recently got fixed in the 3.6 codebase.

With this new, in my opinion more user-friendly approach to accessing Flash content and Java applets, making sure your Flash or Java applets are accessible is becoming even more important than it already is, since blind users will be able to interact with applets more seamlessly than before.

And while we’re at it, the better support in NVDA for Flash should also be an incentive to Adobe to make Flash accessible on other platforms such as Linux and Mac. For the Mac, maccessibility.net have a petition to Adobe for making Flash accessible on the Mac. If you haven’t already done so, I encourage you to show your support by signing that petition!

First of all, the bad news is that getting the remaining quirks out of Mac accessibility is not going to happen this or next quarters.

The main reason is that our code currently needs to be made ready for future enhancements. Our recent work on the IAccessibleTable2 interface showed us that painfully: The refactoring and reorganization needed took much longer than we had estimated and required us to touch much more code than we had hoped. We also have some nasty event firing problems that have been a pain in the neck for those ATs supporting us that we desperately need to attack these now.

However, the good news is that since early May, the accessibility unit tests (Mochitests) have been running mostly successfully on the Mac on both the mozilla-central and mozilla-1.9.2 branch tinderboxes. Those tests had been running successfully on Windows and Linux since December already. They’ve been intermittently showing some flakiness (what we call random orange) in especially the events area, which is what we’ll be working on in the immediate future anyway, but otherwise been quite stable.

On top of that, at least investigating and driving the Mac accessibility effort forward has been sort of a pet project of mine all the while, even though Mac a11y is no longer part of our team’s official goal set for this year.

What I’ve been doing is build Mac builds with accessibility enabled on my MacBook and test them whenever I had some cycles. On Mac OS X 10.5 (Leopard), VoiceOver and the Firefox builds played terribly together, with lots of sluggishness and delayed responses that made it impossible to get any really useful testing done.

This, however, has changed with Snow Leopard. Without consciously changing things on our end, a build done under Snow Leopard is much more responsive with VoiceOver. We’re nowhere near the good performance blind people have come to appreciate from Safari yet, but we’re doing much better now. Whether it’s the improved VoiceOver or the fact that we’re now building against the MacOS10.5 SDK, or both, I don’t know.

The following are what I’d consider the most pressing issues before we could even think about sending this off to a bunch of voluntary testers:

• VoiceOver is not speaking most accessible roles of UI and web page elements. Strangely enough, when pressing a button, it will say “Press button” and the like.
• VoiceOver skips over plain text on pages, and the text is also not visible in VoiceOver’s Obbject Chooser menu.
• Other inconsistencies having to do with roles like VoiceOver not recognizing our HTML area as such and not automatically starting to read pages etc.
• Performance problems: Right now, navigating from one element to the next takes about half a second, which is unacceptable.

And this is where you might be able to help us! Are you a developer who is fluent in Objective-C, and (as a bonus) versed in Apple’s accessibility APIs? We could use your voluntary help in trying to nail this down!

So if you’d like to help improve Firefox on the Mac and have an idea about what might be going on here, go grab the source and build it! In order for accessibility to be turned on, in addition to all the stuff you need to put into your MOZCONFIG file, you need to add the following line:

ac_add_options --enable-accessibility

With this exception, commence as described in our docs.

Once you’ve got Firefox built, you can press CMD+F5 to turn on VoiceOver. A friendly male voice named Alex will start talking to you immediately and give you feedback on your actions through your Mac’s speakers. CMD+F5 is a toggle, so the same keystroke will turn VoiceOver off and leave no trace of it having been active.

Once VoiceOver is running, the following is a list of hot keys that can be used to navigate. Note that VO refers to the VoiceOver modifier, which is both Ctrl and Option held down.

More information is available on Apple’s accessibility site. A Key commands chart (PDF) will give you more keystrokes, and the linked-to bugs above will give exact steps to repro the bugs themselves.

Since Firefox is not a native COCOA application on the Mac in the sense that we implement all our UI elements ourselves, we have to expose the whole accessibility contract and are obviously doing some things wrong there. The initial work that was done was done by a Mozilla Foundation grantee back in 2006, but this project was never finished and the project had been in limbo since then.

If you are willing to help, feel free to connect with me through e-mail (available from my About page) or via IRC on the #accessibility channel. I’ll be glad to assist with getting involved. The accessibility team definitely would appreciate it, and so would many community members I’m sure!

The goal was to:

1. Refactor our code to become more maintainable.
2. Expose the same kind of interface to assistive technologies regardless of what lies underneath is a true HTML table, an ARIA table, an ARIA data grid, a XUL tree table etc. There are so many commonalities that ATs should be able to expect similar, if not identical, information from any of these table types.
3. Implement the IAccessibleTable2 interface that was defined within the IA2 group over the summer.
4. Get rid of many “todo” items in our Mochitest unit tests.

For The Minefield (Firefox 3.7a1pre) nightly build that will come out today, Friday September 11, the patch that implements IAccessibleTable2 has landed. All header/table cell relations are now defined through proper method calls rather than special forms of accessible relations. This was a big patch with almost 550 kb in size. A quarter of this were IDL changes for IA2, but the rest was all code that had to be reviewed and super-reviewed. It’s definitely the biggest patch I’ve been working on so far since I started working for Mozilla.

Over the past few weeks, Alex also refactored our XUL tree exposure to better meet our goal. With 235 KB, this was the second biggest patch I have been working on so far.

Other changes that went in concern selection of cells, rows, unselecting parts of tables etc. Some of these have caused our long “to do” item list of around 80 items to drop to a mere 7 throughout the whole a11y test suite. The table tests were among the first written when we started doing automated tests on a11y some 20 months ago, and it’s cool to finally see this area of the code properly addressed!

With these changes, our number of passing tests is now at 10205 and a total of 7 to-do items.

And this is where you come in! If you’re an accessible tables junkie, know a lot about HTML table make-up, or know of very weird tables out in the wild, go download the latest nightly build and throw the tables at it! Let us know your results, for example by commenting here on the blog, and if you find something that definitely isn’t exposed right, file a bug! We appreciate any and all help you can give us here!

If you’re an AT vendor and plan on implementing support for IAccessibleTable2, here’s a model you can use!

Finally, I’d like to thank all the cool people (module peers and super reviewers) who helped us accomplish what we’ve accomplished so far! With their knowledge and wisdom about the inner workings of Gecko and their respective modules, they helped make our code faster, better and more robust. Keep on rockin’!

There even is a workaround for IE, but it doesn’t work in IE 8. To quote a friend from Germany: “Maybe IE 17″.

As JJB mentions in his post, UI accessibility was provided for many of the Firebug features by Hans Hillen of the Paciello Group. The Mozilla Foundation funded the first of this work, and the Mozilla Corporation is now continuing funding to finish the remaining UI pieces like the “Net” panel, fix remaining issues, and also to work with the University of Illinois to develop Firebug features that will help web developers check their sites and applications for accessibility and fix issues in that area.

I am not going to iterate over the features one by one, since the documentation on the accessibility features is very comprehensive. I urge all developers wanting to utilize these features to RTFM (read the fantastic manual) in its entirety before starting to use accessible Firebug.

Once you are up and running with it, you’ll find that all of the described features work as advertised in the documentation. And even more: The documentation currently still states that the script panel’s source code viewer cannot be read with NVDA. However, Hans, Jamie from the NVDA team, and I managed to fix that problem shortly before beta 1 of Firebug 1.4. I notified Hans to update the documentation.

Compared to Firebug 1.3, Firebug 1.4 offers a wealth of accessible web development tools. Firebug 1.3 was effectively not accessible, Firebug 1.4 is, with a few small exceptions, fully accessible, which is an almost 100% leap forward within a single version number increment!

This opens up whole new opportunities for blind developers. To my knowledge, the development tools offered by the Redmond browser do not offer this wealth of features. In testing I did while fixing my example form to also work in IE 8, I found the tools to be unreliable and shaky in many areas. Firefox 3.5 and Firebug 1.4, on the other hand, together with NVDA, which currently best supports the former, are a vehicle to open up new job opportunities! With Firebug, a blind person has full control over the styling and layout of a web application or page. For example, taking my example form from the Easy ARIA Tip #3, and looking at the label and input elements for the “name” entry, I can immediately be told that the label is to the left of the input, and how many pixels there are between the two. I can make sure they’re visually at the same height. The box model of the Layout sidepanel of the Firebug HTML panel is fully accessible and gives me a feel for a page I’ve never had before!

Communication of search results within the HTML panel, or when encountering a breakpoint, is just awesome! Being able to inspect the DOM or a single element’s properties, being able to in-line edit properties and immediately be able to test the effect a certain ARIA attribute might have on your screen reader output is just productivety at its best!

What Hans accomplished here is done almost entirely through WAI-ARIA, and by implementing very intelligent keyboard navigation mechanisms. Through ARIA, such widgets as the log rows for the console, when you enter something such as document;, or dir();, are pushing the boundaries of known desktop widgets. The fact that tabs have popup menus attached to them is being communicated to the user by NVDA straight away, thanks to a very open and flexible architecture that does not assume certain facts about traditional static desktop widgetry such as “tabs never have menus attached to them”. The pure visual box representation of the Layout side panel of the HTML panel is another great example. This even blew David Bolter away when I showed it to him during the Mozilla all-hands in late April, and David is a long-standing and accomplished a11y guru!

Implementing some of these features in desktop applications usually requires very customized implementations of the platform’s accessibility APIs. ARIA, however, is so open and flexible that it can be easily applied to make such visually complex tools like Firebug accessible without ever touching the Mozilla core codebase. Firebug is a mix of XUL (the Mozilla UI language) and HTML+CSS.

So, Victor Tsaran of Yahoo! is already firebugging, and so am I. When are you?

The following are the candidates:

• NVDA, using the latest snapshot build, which actually does not behave much different from the current 0.6p3 release.
• JAWS 10.0.1154.
• Window-Eyes 7.1.
• SuperNova 11.02.
• System Access in the form of the free System Access To Go service.

The scoring is simple: For each important feature that is fully supported, each screen reader gets 1 point. A particular web app may have more than 1 feature, so it is possible to score multiple points for web apps.

Note that, even if WAI-ARIA support is not explicitly documented, it is still possible to score points because Firefox exposes many widgets through MSAA and IAccessible2 that are not standard HTML widgets. The interesting question here is: Are the various forms of Forms/Focus mode flexible enough to deal with these?

Landmarks

WAI-ARIA landmarks are one of the most widely used features of the spec already. They allow another means of navigating a web page, finding things such as the banner, main content, search, complementary or footer information. The newly relaunched Mozilla Add-Ons website uses them now, as does this blog.

NVDA

No.

JAWS

Yes. Landmarks are announced, they can be navigated to using the Semicolon quick navigation key, and there’s a list of landmarks available through JAWSKey+Ctrl+SemiColon. 1 point

Window-Eyes

No.

SuperNova

No.

System Access To Go

No.

So after the first round, JAWS is in the lead with 1 points.

Contact form from my Easy ARIA tips

The example contact form I created for my Easy Aria tip #3 offers several features that can be incorporated without having to create widgets, and which have appeared in some form or another on pages throughout the web already:

• Does the fact that the Name, E-Mail and Message fields are required get indicated by anything other than the label saying “required”?
  • By navigating the virtual buffer
  • When in forms/focus mode and tabbing around
• When entering an invalid name by just entering the first name:
  • Does the alert get spoken when tabbing away?
  • When tabbing back, does the field get indicated as having an invalid entry?
  • Does the fact that this field has an invalid entry get indicated when navigating in the virtual buffer?

In total, there are 5 points to score for this test.

NVDA

NVDA indicates the field as being required in v cursor mode and when tabbing around. 2 points. It speaks the alert. 1 point. It indicates the invalid attribute both when navigating the virtual buffer and when in focus mode and tabbing around. 2 points. Total: 5 points

JAWS

While the label gets spoken in virtual cursor mode, when JAWS switches to forms mode automatically when hitting the entry field, the plopping sound drowns out every indication of attributes such as required or invalid. Only when deviating from default settings and turning AutoFormsMode off one will hear those attributes in V cursor mode. No points for these two. The alert gets spoken. 1 point. When tabbing around, the attributes such as required and invalid do get announced with the default settings. 2 points for these. Total: 3 points

Window-Eyes

The fact that the field is required gets spoken in both browse and focus modes. 2 points. The alert gets spoken. 1 point. The fact that the field has an invalid entry gets spoken in both browse and focus modes. 2 points. Total: 5 points

SuperNova

None of the asked for features work. Sorry, 0 points.

System Access To Go

The alert gets spoken. 1 point. None of the attributes are spoken when navigating or tabbing. Total: 1 point.

After round 2, NVDA and Window-Eyes take the lead with 5 points each, JAWS follows on third place with a total of 4 points, System Access has 1 point, and SuperNova has 0 points.

Yahoo! Search

The new Yahoo! Search is an interactive widget allowing browsing of possible search terms and related concepts that fit the currently selected search term. It uses a whole range of WAI-ARIA widgets, properties and states, live regions etc. When performing a search, the following things should be working:

• When focusing the textbox:
  • Does the screen reader speak the name “Search query”?
  • Does the screen reader announce the description “Use the up and down arrow keys to select suggestions, or press down and then right to explore concepts.”?
• When typing, does the screen reader announce that search suggestions are available?
• When search suggestions are available, does pressing DownArrow properly announce that focus shifted to the list of suggested search terms, and what to do to get back to the search field?
• Does pressing RightArrow announce the shift to the “related concepts” list and the selected item?
• When arrowing through either list, does the highlighted/focused item get spoken, and does the search that will be performed when pressing Enter get announced by the screen reader?

So, there are 7 points to score for this one.

NVDA

It speaks the “Search query” label. 1 point. It speaks the “Use the..” description. 1 point. When search suggestions are available, the fact is announced. 1 point. When pressing DownArrow, the transition to the list of suggested terms is announced along with the full instructions and the selected item. 1 point. When arrowing left and right to the related concepts and back, each focus transition is properly announced and the highlighted item read. 1 point. When arrowing up and down through either list, the newly highlighted search term is announced, and the search that is going to be performed is announced automatically. 2 points. Total: 7 points

JAWS

When focusing the search field, the “Search query” label is announced. 1 point. The “use …” description is not announced automatically. It is also not being announced when pressing JawsKey+Tab or Insert+F1. The only way to get to it is to use their HomeRow utility functions and cycling to the “Description” item with HomeRow+F10 and then listening to it with HomeRow+F9. For this almost non-discoverability I can’t give a point, sorry. When search results are available, this gets announced. 1 point. When pressing DownArrow, the transition to the list is announced along with the prompt. 1 point. When RightArrowing, the transition to the “Explore related concepts” list is announced. 1 point. When arrowing up and down, the newly highlighted item is not announced, and neither is the search that is going to be performed. One can get the currently focused item by using Insert+Tab, but the description is once again burried in HomeRow. I’m willing to give half a point for this one since initially it will be confusing to users that they don’t hear anything when arrowing up and down. Total: 4.5 points

Window-Eyes

The label “Search query” is announced. 1 point. The “Use…” description is announced. 1 point. The availability of search results is not announced. The transition to the search term suggestions is partially announced: The focused item is, but the prompt is not. Half a point. The transition to the “Related concepts” and back is announced partially: The newly focused item is, but the prompt isn’t. half a point. When arrowing up and down, both the search suggestion and the search that is going to be performed are being announced. 2 points. Total: 5 points.

SuperNova

Announcing the “Search query” label works. 1 point. But unfortunately, that’s where the fun ends. The description is not announced, the availability of search term suggestions is neither. And the rest of the functionality of this widget is broken. DownArrow is captured by SuperNova and will not fall through to the widget, getting one stuck inside the textbox. Tabbing around will only get up to the “Submit your site” link, but the search terms aren’t reachable. SuperNova will say “bottom”, and no further can one go. Total: 1 point.

System Access To Go

The picture is roughly the same as with SuperNova. The label “Search query” is spoken. 1 point. The description is not spoken. The availability of search term suggestions neither. DownArrow gets you to the “Search” button instead of the list of search terms. In fact, this virtual buffer also ends at the “Submit your site” link. Total: 1 point.

At the end of this round, NVDA leaps ahead with 12 points. Window-Eyes is second with 10 points, followed by JAWS with 8.5 points. System Access scores a total of 2, and SuperNova got their first point!

GMail Chat

GMail has an integrated Google Talk widget that I talked about before. The following should be working:

• Ability to activate the “Set status here” label by pressing Enter on it to input a personal status message.
• Ability to activate the “status menu” and navigate inside it with speech output.
• Navigate inside the list of buddies and hear their names and status.
• Inside the Chat window, announce typed and incoming messages.
• Track going to the Chat window toolbar.

Once again, there are 5 points to score. Let’s see how everyone fares!

NVDA

Pressing Enter on “Set status here” works fine, and one can input a status message. 1 point. Activating and navigating in the status menu works fine. 1 point. The list of buddies talks fine. 1 point. Chatting works fine. 1 point. Trying to access the toolbar items by first going out of focus mode with Escape made NVDA hang each time I tried it. It somehow has a conflict with the chat widget. Sorry, no point for this one. Total: 4 points

JAWS

The label to input a status message is not activable by pressing Enter. It can only be activated using the JAWS cursor emulation. Since this is a well-known workaround, I’m giving half a point. The Status menu is activable and works fine. 1 point. The list talks fine. 1 point. The incoming and typed messages are spoken in the chat output. 1 point. The chat toolbar to pop out the chat into its own window is accessible. 1 point. Total: 4.5 points.

Window-Eyes

Accessing the label to input a status message works with workaround of routing WE cursor to element, then mouse to WE cursor, and clicking with the mouse. However, I cannot input a status message afterwards, even though I hear the prompt for it. a quarter of a point for that. The status menu cannot be activated through any means. The list talks fine. 1 point. The chat window works with restrictions: It can be activated and typed in, but incoming messages are not read. half a point for that. Trying to access the toolbar items of the chat window sort of works by turning browse mode back on, and then searching, but since the last position is not retained, I can only give half a point for this one. Total: 2.25 points.

SuperNova

Activating the “Set status here” works. I can input a new status. 1 point. The status menu button does not work, cannot be activated or found through other means. The list of buddies talks. 1 point. Activating a chat with a buddy does not work. Consequently, since the chat window never comes up, the toolbar items for the chat window obsolete themselves. Total: 2 points.

System Access To Go

The “Set status here” and Status menu items are not accessible. The list talks fine. 1 point. Activating a chat works. 1 point. Finding the toolbar buttons is not possible, because the cursor gets stuck within the textbox of the chat window and there’s no way to move it out. Total: 2 points.

…and the winner is…

Congratulations go to the NV Access team and their screen reader! In this WAI-ARIA shootout, you scored 16 points.

Number 2 is JAWS by Freedom scientific, scoring a total of 12.5 points.

Window-Eyes by GW Micro is third with a total of 12.25 points.

Fourth place goes to Serotek with their System Access screen reader product line, with a total of 4 points.

And SuperNova by Dolphin receives 3 points.

In summary

This was a close match, although there is clearly a dividing line between the three screen readers that have been supporting Firefox for a longer period of time, and those that came on board fresh within the past year or so.

I hope this little competition encourages each of the vendors to better themselves for the benefit of the users. We’re here to help each and everyone of you with technical advice and discussion on how things should be implemented.

Keep on rockin’!

Support for text attributes, formatting and spell checking

Firefox 3.5 exposes text attributes such as bold, underlined, and color information through the AT-SPI and IAccessible2 attributes properties of their respective AccessibleText interfaces. Information about formatting such as left aligned, centered etc., is also provided. In addition, in rich editing and other text entry environments where the spell checker is enabled, when a word is misspelled, this information is also provided so screen readers can pick up and notify the user. Since my original blog post on this subject, Orca 2.28, NVDA 0.6p3 and JAWS 10 have added support for this feature, allowing a seamless proof reading of entered text in both Firefox 3.5 and the upcoming Thunderbird 3 release. This works also when writing a message in GMail or other rich editing environments, not just textboxes or textareas.

Better compliance with WAI-ARIA 1.0

We’ve made sure that the WAI-ARIA 1.0 spec is adhered to to the most extent possible, removing attributes that are no longer in the spec, and adding/changing those that were agreed upon in the progression towards finalizing ARIA 1.0, which is currently in a late review stage. One of the most significant attributes added is aria-label, which allows any text to be associated with a widget that doesn’t appear anywhere else within the web app. For extension devs: This also works in XUL, not just HTML. One project that makes heavy use of this is Firebug 1.4 in the accessibility UI enhancements that Hans from the Paciello Group has put in. This is also the reason why Firebug 1.4 works better in Firefox 3.5 than 3.0, what accessibility is concerned, since Firefox 3.0 doesn’t support this new attribute.

Also, aria-expanded can be used on all elements now, allowing better exposure of states for buttons that drop down a list of items, for example.

Support for the exposure of embedded HTML 5 audio and video controls

As recently announced, we’re also supporting the exposure of the embedded controls of the HTML 5 audio and video elements to assistive technologies.

Better event firing in dynamic web applications

We fixed a significant issue with firing proper events when nodes get hidden or made visible through JavaScript or Ajax calls. This should allow a much better experience with accurate visibility of nodes within virtual buffers of various screen readers.

And a ton of bug fixes for stability

Of course, each cycle also goes with a ton of bug fixes that improve stability and accuracy. These are mostly under the hood and often deal with edge cases, but these are no less important to our user base.

When Firefox comes out, I encourage everyone to upgrade as soon as possible, since it will provide an even more rich experience when browsing the web than Firefox 3.0 already did. Probably the most important extension for blind users, WebVisum, already works with this release, so you won’t lose anything on that front! Also, other extension devs are working hard to make their projects work with Firefox 3.5.

Enjoy!

If you’re using Firefox 3.5b4, you may already have been notified of the Firefox 3.5b99 update. This update brings exposure of the HTML5 audio and video elements through the MSAA/IAccessible2 and ATK/AT-SPI accessibility interfaces, further strengthening our commitment to Open Video.

What this means is that, using NVDA or Orca, you now have access to an HTML5 audio or video file embedded in a web page. If the controls attribute is specified within the audio or video tag, Firefox 3.5 creates its own set of playback controls. These are now exposed to screen readers using MSAA or AT-SPI to traverse our trees. Exposure through iSimpleDOM interfaces on Windows will come at a later stage (see this bug). Previously, these controls were unreachable. They did not appear inside the virtual buffer.

In addition, a keyboard navigation bug was also fixed allowing control of most of the functions of the embedded controls. Note that the individual buttons and sliders are not yet tabbable, we’ll be working on a good keyboard interaction model and hopefully improve the current model in a point release.

To test the video element, with NVDA as an example, do this:

1. Go to this video.
2. Using NVDA’s quick key b, find the Play button.
3. Press Space.
4. To interact with the audio/video element directly, press NVDA+Space to turn to Focus Mode temporarily, and Shift+Tab once to land on the video grouping.
5. You can now use the following keyboard shortcuts:
   

   Shortcut	Action
   Space	Play/Pause
   right arrow and Left arrow	Move through the video in 5 second intervals.
   Ctrl+Right Arrow and Ctrl+Left arrow	Move through the video by 1 minute increments.
   Up arrow and Down arrow	Increase and decrease the volume. This does not change the volume of your speech synthesizer!

6. After you’ve finished interacting with the video controls, press NVDA+Space again to turn virtual cursor back on.

Note that the ultimate goal is to not require you to turn focus mode on and off with NVDA+Space, but to allow you to interact with each scrubber/slider using the regular Focus mode command Enter and Escape, however we’re not quite there yet (see the linked bug above).

BTW: the above keyboard shortcuts also work if you don’t have a screen reader running. You need to tab to the video element grouping and then use the shortcuts as described.

Note also that due to our not exposing the elements through iSimpleDOM yet, some screen readers may not see the controls yet. In that case, turn off your virtual cursor and tab to the video element to use the keyboard shortcuts to control your video playback.

Stay tuned for more updates as they come along!

Exposing HTML 5 audio and video elements

The initial exposure for the HTML5 audio and video elements to screen readers landed, causing a minor regression that was quickly fixed. In testing this with NVDA, I found that the button labels weren’t properly exposed and that the slider values were not really useful. The progress meters were showing the number of bytes downloaded, or the milliseconds elapsed instead of useful percentage values. Along those lines, Alex also added a bug to expose proper names for each progress meter, so a screen reader user knows which slider is for what purpose.

Except for the last patch, all others have landed on mozilla-central and will be available for testing starting with the 11th of May nightly build.

To make it clear: This is for those HTML5 audio and video elements that have the controls attribute set, indicating that the internally available controls should be used. Other forms of controlling the media playback, such as from external HTML controls/widgets, already worked in the past since these were not part of the actual audio or video element itself.

Tree view item rectangle exposure

We received a report that in Thunderbird 3 beta on Windows, the rectangles for tree view items were not exposed correctly. The rectangle was too small, not encompassing the whole item. Alex investigated this and fixed the bug, putting an optimization in as a second step for all platforms. This also landed on mozilla-1.9.1 after having baked on mozilla-central for a while, and is available since the May 9th nightly builds of Shiretoko, Thunderbird and SeaMonkey.

The ARIA live region background tab leakage

David has been taking different stabs at bug 444644, with some good results thanks to feedback from Roc and BZ during the Mozilla all-hands week. However, we’re still fighting a situation where the creation of virtual buffers by NVDA is causing the live region updates from background tabs to be spoken again. Investigation is ongoing

Other ARIA-related triage

David’s also been a busy bee clearing out some ARIA-related bugs, gathering feedback here and there, closing others as they’ve been solved by other bugs, etc.

Firebug accessibility

This is not strictly inside the “Accessible” module of the platform, but very closely related to the Mozilla eco system. Accessibility of the Firebug UI has been shaping up very nicely over recent weeks. I spent a fair amount of time last week pounding the different alpha releases to help make sure things stayed in shape.

On Friday, Hans from the Paciello Group, Jamie from the NVDA team and I also managed to get the biggest outstanding problem solved in a very productive meeting on IRC, and that’s the reading of the Firebug JS panel by NVDA. Watch this space for a review once Firebug 1.4 goes to beta!

That’s it for this week, thanks for the read!

In Firefox 3.0.x, we expose the screen text, in this case the word “regression” as the accessible name. This is the piece that screen readers speak when focus lands on the link, and which usually also gets rendered into the virtual buffer.

If there is also a title attribute, in this case “@title attribute no longer exposed on accDescription”, this will be translated into the accessible description of the link object. This is additional information that can be spoken by the screen reader on demand. For example, in NVDA, focus the link or arrow to it in the virtual buffer, and hit NVDA+Tab. NVDA will speak first the name, followed by the fact that this is a link which is linked to something, followed by the description.

The one exception where we do not expose a description is when the screen text and title attribute contents would match. This is considered bad practice anyway, because it is redundant information, and thus we suppress it.

Firefox 3.5b4, and in fact all builds that date back to mid October last year, have a bug in that the title attribute is no longer exposed as the accessible description. Jamie from the NVDA team found this recently and notified us.

I’m happy to report that this functionality got restored in the Firefox 3.5b5pre nightly builds starting with the May 7, 2009 build. Sorry for any inconvenience this may have caused!

Actions for sorting and expansion/collapsing

After some minor setbacks, David’s patch on exposing actions for ARIA sort and expand/collapse attributes finally landed today. This means that:

• An element that has aria-sort set, will expose an action of “sort” to assistive technologies.
• An element that has aria-expanded set to “true” will expose an action of “collapse”, one that has aria-expanded set to “false” will expose an action of “expand”.

These can be used to exactly determine what action will be performed once it is being performed.

Exposure of the HTML5 audio and video element controls

Alexander’s patch to expose the embedded controls of the HTML 5 video and audio elements has landed on mozilla-central. With NVDA, one can now see the grouping where the controls are, and invoke the action on each of the buttons. One can even switch to focus mode on the sliders and use the arrow keys to manipulate them. Note: Due toa different approach in reading our information, JAWS does not yet expose these controls despite this patch. Other screen readers are pending tests.

There are a few problems still which will be addressed soonish: For one, the buttons don’t have text labels yet, and the slider percentage values reflect times rather than actual percentages, so we need to see how we’re going to expose this properly.

In other news

The team, along with a number of community members, has worked on a new high-level accessibility strategy document. Frank Hecker has a blog post explaining this in greater detail.

Spreading the good work of ARIA to mainstream open-source CMS

Peter Krantz, accessibility expert from Sweden, has started an effort to contribute WAI-ARIA landmark roles to mainstream open-source content management systems. If you know one of the CMS that don’t have patches yet, feel free to jump in!

That’s it for this week, see you next week for a new edition!

In the version that is currently on addons.mozilla.org, there are several problems with missing label7control associations in the Options dialog as well as problems navigating the tree, and more importantly, opening a feed article via the keyboard.

Not too long ago, I contributed a patch to the project to fix these problems, and Chris has accepted it and put it into a recent beta version of Feed Sidebar. He also made it possible to access the sorting options from the context menu.

The latest beta brings a better updating mechanism that is less resource hungry.

For those of you who have asked me about a way to view feeds in a tree like structure, this is definitely worth a try! Go download the latest beta version here! I’ve found it to be very stable and accessible. Of course, feedback is welcome! You can either contact Chris directly of course, or leave a comment here, I’ll then forward it to him.

Enjoy!

This article can be found on the front page of my blog under the “Pages” section, in the “Articles” sublist.

The article is meant as an introduction, not as a replacement for the NVDA user guide, and it is certainly not meant to replace other accessibility testing tools you might use for your website testing, just as an additional tool to help you get a feel for how blind users interact with your web sites or web applications.

I plan to update the article periodically as new versions of NVDA become available, features are added and other info relevant to the article might change.

Enjoy the read, and feel free to leave feedback!

The example works visually, but has a number of accessibility issues which make testing IE 8 with it a not so pleasant experience:

• Neither aria-required nor aria-invalid take any effect with either JAWS or NVDA. It’s as if the attributes weren’t set, yet the IE DOM exposes them correctly, as JAWS’s Element Info keystroke, Insert+Shift+F1, clearly indicates.
• Neither JAWS nor NVDA see the alerts come up, and thus don’t speak them. The alerts appear visually, so the JavaScript is working, but the DOM mutation is not being picked up. Only after a refresh in the respective screen readers is the content of these visible in the virtual buffer.

For testing, I used the latest update to JAWS 10, build number 10.0.1142, and NVDA trunk snapshot 2828. And of course, the release version of IE 8.

For those of you doing web application development and testing, this is an indication that your best bet to get proper results is definitely the combination of a strong implementation of ARIA in Firefox and a supporting screen reader.