The try server has saved the accessibility team members’ butts so many times over the past two years that I’ve stopped counting. Just now, as we’re working hard to fix our performance problems on trunk, and putting in some other much needed reorganization, this try server build system has proven to be an invaluable tool for finding regressions before they even become such.

For example on this document handling reorganization bug, I tested various try-server builds before confirming that this huge code refactoring didn’t break us. Much of it is caught by Mochitests, but not all of it is, because some things are in platform-dependent wrapper classes that cannot be tested by the Mochitest unittest framework.

Likewise, we’re currently fighting a regression in the patch for a performance booster bug that would break a popular screen reader on many pages if it landed as the patch is now. I found this while testing the try server build for this patch. Tests pass and the code looks correct, but something is wrong nevertheless. The try server allows us to go through as many incarnations of the patch as needed until the problem is resolved without affecting the production of nightly builds. It avoids having to file follow-up bugs and having to go back to the same area of the code weeks later.

For my job as Mozilla’s accessibility QA engineer, the try server has become an invaluable tool as part of my day job. I encourage everyone doing testing in tandem with a developer to utilize this tool heavily!

Since this is the first issue, and my last update on a concrete new feature/major change has been a while, here’s a broad overview over what we’ve been up to over the past two months or so.

Automated tests on all three active Firefox development branches

Since December 18, 2008, all three Firefox branches that are under development run accessibility mochitests. These branches are:

• Gecko 1.9.0, off which Firefox 3.0.x is being released. These tests actually have been running since roughly the Firefox 3.0.4 timeframe in October.
• Gecko 1.9.1 AKA Firefox 3.1. This is the branch Firefox 3.1 will be released from.
• Mozilla-central. This is the development branch where the next major release after Firefox 3.1 is being developed, with new features going in and more experimental stuff is happening right now.

The most interesting of these is definitely the mozilla-central branch. Both Gecko 1.9.1 and 1.9.0 receive back ports of important features/bug fixes from this branch. The number of tests that run in the accessibility code has surpassed the 2,000 mark two to three weeks ago. We started in December with about 1100 tests.

ARIA 1.0 compliance patches

David has been working on ARIA 1.0 compliance patches and started to put in some good infrastructural stuff as well. Interesting items are:

• The change from aria-grab to aria-grabbed
• The removal of role=”description”
• Some urgently needed fixes for when the document node gets an ARIA role
• Some infrastructural changes and fixes to which ARIA roles can receive what states, and what these states should be

Alex implemented the ARIA spec’s name from subtree calculation algorithm. Of course, despite all the tests, a regression had to creep in, but after that’s fixed, I believe we’re in good shape. The visible result, especially for authors of ARIA-enabled we bapps, is that the Firefox 3.2a1pre nightlies should now always behave according to spec when calculating the name from those elements/widgets that call for its subtree to be aggregated as its name.

As a side note for those interested in the effect our automated tests now have: When Alex tried to land a fix for bug 463645, it clashed with the above mentioned name from subtree bug in an unexpected way, causing an immediate orange flag on the unittest tinderboxen. We’re looking for a solution to that problem right now. Hadn’t we had these tests in place, it would have taken a while for this regression to creep up in manual testing. This way, due to an immediate backout of the offending code, no nightly build ever saw this happen.

New for text attributes

David implemented the conversion to pt algorithm that IAccessible2 calls for on Windows. This change is also in Firefox 3.1.

Alex implemented a faster way of retrieving text attributes,a nd he and I teamed up on better better test coverage for this area.

Accessible relations improvements

Alex worked on better accessible relations support, allowing for a relation to have multiple targets. This will allow assistive technologies to get at aria-labelledby relations correctly. As a sequel, 1:1 relations between tabs and tabpanels were also implemented, allowing ATs to better identify which tab a tab panel belongs to.

Proper reorder events

Alex, our man for big patches , also created a fix for a long-standing bug involving reorder events on DOM mutation changes. Several screen readers need these events to keep track of dynamic changes on web pages to keep their virtual buffers up to date. Some recently inconsistent behavior that often required the user to refresh their virtual buffers manually, should now work much better. This change also is included in the Gecko 1.9.1 branch already, waiting to debut in the upcoming beta 3 of Firefox 3.1.

Thunderbird 3 reading panes

The Thunderbird 3 reading panes received a fix to the way they expose the “from: ” etc. header information. When tabbing through these header fields, one now immediately hears which type of header this is with a compatible screen reader.

This fix was done by Yuen Hoe, a student at the University of Singapore, as part of the NUS Mozilla students project. He picked this bug specifically. Thanks Moofang!

Because this patch relates very closely to SeaMonkey, it was ported there as well, however users can’t take full advantage of this yet in SeaMonkey because of a keyboard navigation issue in the message reading panes.

Better accessibility in SuMo’s live chat

SuMo, Mozilla’s support community, offers a live chat facility that alows users to get help quickly. This live chat was previously not very accessible.

However, Gijs Kruitbosch, the mastermind behind ChatZilla’s accessibility features, worked on a fix, which will see the light in one of the next Sumo updates.

I know this is not strictly the Gecko Accessible module, but nevertheless very important to the over-all Mozilla eco system, and that’s why I’m mentioning it here.

On other fronts

I’m currently working through some of the earliest test files to make them use the common accessibility retrieval and events structures that were implemented more recently. This will help maintainability and make sure that if we add new features to these common functions and classes, every test will benefit from them.

On Windows, all active branches will be more correct in what service IDs they accept when calling the QueryService function. This insures better compatibility with Windows 7, among other things.

Thanks for sticking with me until now! The next reports will be shorter, I promise!

Of course, we welcome your feedback on this kind of post, or on specific areas. So feel free to comment here!

The problem: The code that calculates the names for any created accessibles has been growing over time and became largely unmaintainable. New features suich as adding the aria-label property support requires code duplication for HTML and XUL, and in general the code has many stylish un-niceties.

So, our team has started a code cleanup and code refactoring series to get the code into better shape and maintainability.

As with any refactor, the result should be identical in output with what we started out from. However, as those of you familiar with software development know, the risk of regressions is there and should not be discounted.

While we do have test cases for many of these instances already, there may still be cases we’ve missed. So any help we get from the community will help make sure that the refactor goes smoothly, but also help fill in any possible gaps in our testcases.

So, how can you help? By downloading and installing the latest nightly builds of Firefox 3.1 for Windows or Linux, and testing the heck out of them. Use your favorite screen reader, use your familiar web sites, use it for day-to-day surfing. Obviously the most likely pages you’ll find differences on, if any, will be those pages you visit frequently, sites you know what the output should be.

If you find something that is different from what you know, you can download the last Windows or last Linux build before the refactor, unzip it into a separate folder, and compare your findings using that build.

If you find differences you did not expect to find, you have two main choices that will get the developer team’s attention:

1. File a bug in Bugzilla:
   • Component: Disability Access APIs
   • Version: Trunk
   • Platform: PC or whichever you use
   • OS: Windows or Linux, depending on where you found the bug.
2. Post a message on the mozilla.dev.accessibility newsgroup (Google Groups mirror).

In any case, your bug report should contain the URL of the page you are experiencing the difference with, the expected output of the element, and the output you’re now getting. Also, is that element a graphic, link, heading, form field etc.? Also, you should mention what screen reader you’re using. If posting to the newsgroups, it will also help to mention the operating system.

How to update the nightly builds to pick up latest code changes: The builtin Check for Updates feature, if invoked from a nightly build, will always grab an update to the latest nightly build and install it for you. So, you only need to download and go through the installation process once. You can then daily check for updates and get the latest code that way.

The first build to see changes will be the October 11 build, build ID 1.9b2pre/20081011.

Working with different profiles: If you don’t want to put your regular profile into the hands of the Firefox nightly builds, you can start Firefox.exe or the ./firefox executable with the -p option to bring up Profile Manager. You can then create a new profile and start Firefox with that profile. That way, your Firefox 3.0.x profile won’t be touched by the 3.1 nightlies if you choose not to. I’ve found, however, that the nightlies are very stable already, and I often flip back and forth between 3.0.x and 3.1 builds on the same profile without problems. The one thing that most certainly would happen is that some extensions may not work in 3.1 yet.

I’d like to thank all of you in advance who decide to participate in this effort and help everyone who relies on Firefox accessibility by testing out the code refactor. You can make a real difference because we obviously can’t test all of the web pages out there, and yours may just be the one we might miss out on.

Obviously, getting these tests running on the production tinderboxes so we immediately see when we broke something is the next step. But until that can be done, we need to find a solution for bug 441974. Basically what is happening is that tests pass when each test file is run stand-alone, but some of these tests fail randomly when running all files in one big batch. But I made some good connections at the Mozilla summit last week, and as soon as we get these passing we’ll start running those tests. They’ll then run along with the many other unit tests we have for Firefox and the Mozilla platform.

I’d like to thank our intern Lukas Blakk and a bunch of other members of the QA and build teams to help me with getting these configs for buildbot right!

For RC1 and the upcoming RC2, I’ve now created testcases for accessibility areas. These tests should be performed using Firefox on either Windows or Linux, and using a screen reader like NVDA, JAWS, or Window-Eyes on Windows, or Orca on Linux. To sighted people not using a screen reader, these expected results do usually not make much sense since they are especially tailored towards output generated by screen readers for the blind.

If you’re interested in helping out testing Firefox on the accessibility side, go get a Litmus account and run the Firefox 3.0 Accessibility test run.

There are a lot of other test runs to perform if you’re interested. You may have to find keyboard equivalents for certain mouse-driven actions, but that should be no problem if you know your Firefox!

Look forward to your results!