Blog maintenance on Saturday

On Saturday, August 23, starting at 9 AM GMT+02:00 (3 AM Eastern, midnight Pacific), this blog will undergo some much needed maintenance. Afterwards it will hopefully be faster, and also have a new theme. I’ll try to keep the interruption as brief as possible. But just in case, so you know. :)

Posted in General | Leave a comment

Started a 30 days with Android experiment

After I revisited the results of my Switching to Android experiment, and finding most (like 99.5%) items in order now, I decided on Tuesday to conduct a serious 30 days with Android endeavor. I have handed in my iPhone to my wife, and she’s keeping (confiscating) it for me.

I will also keep a diary of experiences. However, since that is very specific and not strictly Mozilla-related, and since this blog is syndicated on Planet Mozilla, I have decided to create a new blog especially for that occasion. You’re welcome to follow along there if you’d like, and do feel free to comment! Day One is already posted.

Also, if you follow me on Twitter, the hashtag #30DaysWithAndroid will be used to denote things around this topic, and announce the new posts as I write them, hopefully every day.

It’s exciting, and I still feel totally crazy about it! :) Well, here goes!

Posted in Accessibility, Mobile | Tagged , , , | Leave a comment

Revisiting the “Switch to Android full-time” experiment

Just over a year ago, I conducted an experiment to see whether it would be possible for me to switch to an Android device full-time for my productive smartphone needs. The conclusion back then was that there were still too many things missing for me to productively switch to Android without losing key parts of my day to day usage.

However, there were several changes over the past 15 or so months that prompted me to revisit this topic. For one, Android itself has been updated to 4.4 Kitkat, and Android L is just around the corner. At least Google Nexus devices since the Nexus 7 2012 get 4.4, and the newer Nexus 7 and Nexus 5 models will most likely even get L.

Second, after an 8 month hiatus since the last release, TalkBack development continues, with the distribution for the beta versions being put on a much more professional level than previously. No more installing of an APK from a Google Code web site!

Also, through my regular work on testing Firefox for Android, I needed to stay current on a couple of Android devices anyway, and noticed that gesture recognition all over the place has improved dramatically in recent versions of Android and TalkBack 3.5.1.

So let’s revisit my points of criticism and see where these stand today! Note that since posting this originally on August 3, 2014, there have been some great tips and hints shared both on here as well as on the eyes-free mailing list, and I’ve now updated the post to reflect those findings.

The keyboard

Two points of criticism I had were problems with the keyboard. Since TalkBack users are more or less stuck on the Google Keyboard that comes with stock Android, there were a few issues that I could not resolve by simply using another keyboard. Both had to do with languages:

  1. When having multiple languages enabled, switching between them would not speak which language one would switch to.
  2. It was not possible to enter German umlauts or other non-English accented characters such as the French accented e (é).

Well what can I say? The Google Keyboard recently got an update, and that solves the bigger of the two problems, namely the ability to enter umlauts and accented characters. The button to switch languages does not say yet which language one would switch to, but after switching, it is clearly announced.

To enter accented characters, one touches and keeps the finger on the base character, for example the e, and waits for the TalkBack announcement that special characters are available. Now, lift your finger and touch slightly above where you just were. If you hit the field where the new characters popped up, you can explore around like on the normal keys, and lift to enter. If you move outside, however, the popup will go away, and you have to start over. This is also a bit of a point of criticism I have: The learning curve at the beginning is a bit high, and one can dismiss the popup by accident very easily. But hey, I can enter umlauts now! :)

Another point is that the umlaut characters are sometimes announced with funny labels. The German a umlaut (ä), for example, is said to have a label of “a with threema”. Same for o and u umlauts. But once one gets used to these, entering umlauts is not really slower than on iOS if you don’t use the much narrower keyboard with umlauts already present.

This new version of the keyboard also has some other nice additions: If Android will auto-correct what you typed, space, and many relevant punctuation characters will now say that your text will be changed to the auto-corrected text once you enter this particular symbol. This is really nice since you immediately can explore to the top of the keyboard and maybe find another auto-correct suggestion, or decide to keep your original instead.

The new version of the keyboard was not offered to me as a regular software update in the Play Store for some reason. Maybe because I had never installed a Play Store version before. I had to explicitly bring it up in Google Play and tell the device to update it. If you need it, you can browse to the link above on the desktop and tell Google Play to send it to your relevant devices. ;)

Two points gone!

Editing text

The next point of criticism I had was the lack of ability to control the cursor reliably, and to use text editing commands. Guess what? TalkBack improved this a lot. When on a text field, the local context menu (swipe up then right in one motion) now includes a menu for cursor control. That allows one to move the cursor to the beginning or the end, select all, and enter selection mode. Once in selection mode, swiping left and right will actually select and deselect text. TalkBack is very terse here, so you have to remember what you selected and what you deselected, but with a bit of practice, it is possible. Afterwards, once some text is selected, this context menu for cursor control not only contains an item to end selection mode, but also to copy or cut the text you selected. Once something is in the clipboard, a Paste option becomes available, too. Yay! That’s another point taken off the list of criticism!

Continuous reading of lists

TalkBack added the ability to read lists continuously by scrolling them one screen forward or back once you swiped to the last or first currently visible item. In fact, they did so not long after my original post. This is another point that can be taken off my list.

The German railway navigator app

This app has become accessible. They came out with a new version late last year that unified the travel planning and ticketting apps, and one can now find connections, book the tickets and use the Android smartphone to legitimize oneself on the train from start to finish on Android like it is possible on iOS. Another point off my list!

eBooks

While the Google Play Books situation hasn’t changed much, the Amazon Kindle app became fully accessible on Android, closely following the lead on iOS. This is not really a surprise, since FireOS, Amazon’s fork of Android, became much mure accessible in its 3.0 release in late 2013, and that included the eBook reader.

Calendaring

This was originally one of the problematic points left, however comments here and in the eyes-free list lead to a solution that is more than satisfactory. I originally wrote:

The stock Calendar app is still confusing as hell. While it talks better somewhat, I still can’t make hells of tales out of what this app wants to tell me. I see dates and appointments intermixed in a fashion that is not intuitive to me. Oh yes, it talks much better than it used to, but the way this app is laid out is confusing the hell out of me. I’m open to suggestions for a real accessible calendar app that integrates nicely with the system calendars coming from my google account.

As Ana points out in a reply on eyes-free, the solution here is to switch the Calendar to Agenda View. One has to find the drop down list at the top left of the screen, which irritatingly says “August 2014″ (or whatever month and year one is in), and which lead me to believe this would open a date picker of some sort. But indeed it opens a small menu, and the bottom-most item is the Agenda View. Once switched to it, appointments are nicely sorted by day, their lengths are given and all other relevant info is there, too. Very nice! And actually quite consistent with the way Google Calendar works on the web, where Agenda View is also the preferred view.

And if one’s calendaring needs aren’t covered by the Google Calendar app, I was recommended Business Calendar as an alternative.

Currency recognition

Originally, I wrote:

The Ideal Currency Identifier still only recognizes U.S. currency, and I believe that’ll stay this way, since it hasn’t seen an update since September 2012. Any hints for any app that recognizes Canadian or British currencies would be appreciated!

I received three suggestions: Darwin Wallet, Blind-Droid Wallet, and Goggles. All three work absolutely great for my purposes!

Navigation and travel

I wrote originally:

Maps are still not as accessible as on iOS, but the Maps app got better over time. I dearly miss an app like BlindSquare that allows me to identify junctions ahead and other marks in my surroundings to properly get oriented. I’m open to suggestions for apps that work internationally and offer a similar feature set on Android!

I received several suggestions, among them DotWalker and OsmAnd. Haven’t yet had  a chance to test either, but will do so over the course of the next couple of days. The feature sets definitely sound promising!

One thing I am definitely missing, when comparing Google Maps to Apple Maps accessibility, is the ability to explore the map with my finger. iOS has had this feature since version 6, which was released in 2012, where you can explore the map on the touch screen and get not only spoken feedback, but can trace streets and junctions with your finger, too, getting an exact picture of whichever area you’re looking at. I haven’t found anything on Android yet that matches this functionality, so I’ll probably keep an iPad around and tether it if I need to look at maps in case my next main phone is indeed an Android one.

Ana also recommended The vOICe for Android as an additional aid. However, this thing is so much more powerful in other senses, too, that it probably deserves an article of its own. I will definitely have a look (or listen) at this one as well!

Other improvements not covered in the original post

Android has seen some other improvements that I didn not specifically cover in my original post, but which are worth mentioning. For example, the newer WebViews became more accessible in some apps, one being the German railway company app I mentioned above. Some apps like Gmail even incorporate something that sounds a lot like ChromeVox. The trick is to swipe to that container, not touch it, then ChromeVox gets invoked properly. With that, the Gmail app is as usable on Android now as the iOS version has been since a major update earlier this year. It is also no longer necessary to enable script improvements to web views in the Accessibility settings. This is being done transparently now.

Other apps like Google+ actually work better on Android even than on iOS. ;)

Other third-party apps like Facebook and Twitter have also seen major improvements to their Android accessibility stories over the past year, in large parts due to the official formation of dedicated accessibility teams in those companies that train the engineers to deliver good accessible experience in new features they introduce.

And one thing should also be positively mentioned: Chrome is following the fox’s lead and now supports proper Explore By Touch and other real native Android accessibility integration. This is available in Chrome 35 and later.

Things that are still bummer points

There is really only one item from the blockquoted items of my original post that is still problematic, and that is the Gallery. The built-in Camera on my Nexus 5 talks much better indeed than it used to, I can now find out whether I’m taking a picture, shooting a video etc. The problematic part is if I want to go back and share the media I took later. The Gallery doesn’t give me any info on the different items I have there. This also includes apps like ABBYY TextGrabber, where if I wanted to choose a photo from the gallery, I’d be just as lost.

However, as Piotr comments below (on the original post content), other device manufacturers use different skins or apps for the same purpose, and it seems that the Camera and Gallery on the Samsung Galaxy S5 are much more powerful in the accessibility sense than the stock Google-provided ones on the Nexus 5.

In conclusion

Over the past year, Android has become a much more viable alternative for me and possibly other users. The gap is closing, and that is a good thing for user choice and market diversity! Show stoppers? No, there aren’t really any any more. The stuff with the Camera or rather the Gallery is an annoyance, but taking pictures is something I do occasionally, not regularly. So yes, I can now see myself using an Android device, be it a Nexus 5 or maybe a modern Samsung device, as my primary phone.

Welcoming your comments!

Posted in Accessibility, Mobile | Tagged , , | 22 Comments

Quick tip: Add someone to circles on Google Plus using a screen reader

In my “WAI-ARIA for screen reader users” post in early May, I was asked by Donna to talk a bit about Google Plus. Especially, she asked how to add someone to circles. Google Plus has learned a thing or two about screen reader accessibility recently, but the fact that there is no official documentation on the Google Accessibility entry page yet suggests that people inside Google are not satisfied with the quality of Google Plus accessibility yet, or not placing a high enough priority on it. That quality, however, has improved, so adding someone to one or more circles using a screen reader is not that difficult any more.

Note that I tested the below steps with Firefox 31 Beta (out July 22) and NVDA 2014.2. Other screen reader/browser combos may vary in the way they output stuff or switch between their virtual cursor and focus/forms modes.

Here are the steps:

  1. Log into Google Plus. If you already have a profile, just go ahead and find someone. If not, create a profile and add people.
  2. The easiest way to find people is to go to the People tab. Note that these currently have no “selected” state yet, but they do have the word “active” as part of the link text.
  3. Once you found someone in the list of suggestions, find the “Add to circles” menu button, and press the Space Bar. Note that it is very important that you press Space here, not Enter!
  4. NVDA now automatically switches to focus mode. What happened is that a popup menu opened that has a list of current circles, and an item at the bottom that allows you to create a new circle on the fly. The circles themselves are checkable menu items. Use the up and down arrows to select a circle, for example Friends or Acquaintances, and press the Space Bar to add the person. The number of people in that circle will dynamically increase by one, and the state will be changing to “checked”. Likewise, if you want to remove a person from a particular circle, press Space Bar just the same. These all act like regular check boxes, and the menu stays active so you can shuffle that person around your circles as you please.
  5. At the bottom, there is a non-checkable menu item called “Add new circle”. Here, you have to press Enter. If you do this, a panel opens inside the menu, and focus lands on a text field where you can enter the name of a new circle, for example Web Developers. Press Tab to reach the Create Circle button and press Space Bar. The new circle will be added, the person you’re adding to circles will automatically be added to that circle, and you’re back in the menu of circle checkboxes.
  6. Once you’re done, press Escape twice. The first will end NVDA’s focus mode, the second will close the Add to Circles menu. Focus will land back on the button for that person, but the label will change to the name of a single circle, if you added the person to only one circle, or the label “x Circles”, where x is the number of circles you just put that person into.

The above steps also work on the menu button that you find if you opened the profile page of an individual person, not just in the list of suggested people, or any other list of people. The interaction is exactly the same.

Hope this helps you get around in Google Plus a bit more efficiently!

Posted in Accessibility | Tagged , | 1 Comment

Accessibility in Google Apps – an overview

I recently said that I would write a blog series about Google apps accessibility, providing some hints and caveats when it comes to using Google products such as GMail, Docs, and Drive in a web browser.

However, when I researched this topic further, I realized that the documentation Google provide on each of their products for screen reader users is actually quite comprehensive. So, instead of repeating what they already said, I’m going to provide some high-level tips and tricks, and links to the relevant documentation so you can look the relevant information up yourself if you need to.

There is really not much difference between Google Drive, GMail and the other consumer-related products and the enterprise-level offerings called Google Apps for Business. All of them are based on the same technology base. The good thing is that there is also no way for administrators to turn accessibility off entirely when they administrate the Google Apps for Business setup for their company. And unless they mess with default settings, themes and other low-vision features should also work in both end user and enterprise environments.

A basic rule: Know your assistive technology!

This is one thing I notice pretty often when I start explaining certain web-related stuff to people, be they screen reader users or users of other assistive technologies. It is vital for your personal, but even more your professional life, to know your assistive technology! As a screen reader user, just getting around a page by tabbing simply isn’t enough to get around complex web applications efficiently and deal with stuff in a timely fashion. You need to be familiar with concepts such as the difference between a virtual document (or virtual buffer or browse mode document) and the forms or focus mode of your screen reader, especially when on Windows. You need to know at least some quick navigation commands available in most browse/virtual mode scenarios. You should also be familiar with what landmarks are to navigate to certain sections of a page. If you just read this and don’t know what I was talking about, consult your screen reader manual and key stroke reference now! If you are a person who requires training to memorize these things and isn’t good at self-paced learning this, go get help and training for this, especially in professional environments. You will be much more proficient afterwards and provide much better services. And besides, it’ll make you feel better because you will have a feeling of greater accomplishment and less frustrations. I promise!

Now with that out of the way, let’s move on to some specific accessibility info, shall we?

GMail

One of the most used things you’l be using is GMail. If you want to use a desktop or mobile e-mail client because that is easiest for you, you can do so! Talk to your administrator if you’re in a corporate or educational environment, or simply set up your GMail account in your preferred client. Today, most clients even won’t require you to enter an IMAP or SMTP server any more, because they know what servers they need to talk to for GMail. So unless your administrator has turned off IMAP and SMTP access, which they most likely haven’t, you should be able to just use your preferred client of choice. Only if you want to add server-side e-mail filters and change other settings will you then need to enter the web interface.

If you want to, or have to, use the web interface, don’t bother with the basic HTML view. It is so stripped down in functionality that the experience by today’s standards is less than pleasant. Instead, familiarize yourself with the GMail guide for screen reader users, and also have a look at the shortcuts for GMail. Note that the latter will only work if your screen reader’s browse or virtual mode is turned off. If you’re a screen reader user, experiment with which way works better for you, browse/virtual mode or non-virtual mode.

Personally, I found the usability of GMail quite improved in recent months compared to earlier times. I particularly am fond of the conversation threading capabilities and the power of the search which can also be applied to filters.

Note that in some documentation, it is said that the chat portion of GMail is not accessible. However, I found that this seems to be outdated information, since the above guide very well states that Chat works, and describes some of its features. Best way to find out: Try it!

Contacts

Contacts are accessible on the web, too, but again you can use your e-mail client’s capabilities or extension to sync your contacts through that as well.

Calendar

Google Calendar’s Agenda View can be used with screen readers on the web, but it, too, allows access from desktop or mobile CalDav clients as well. The Google Calendar guide for screen reader users and Keyboard Shortcuts for Google Calendar provide the relevant info.

Google Docs and Sites

This is probably the most complicated suite of the Google offerings, but don’t fear, they are accessible and you can actually work with them nowadays. For this to work best, Google recommends to use either JAWS or NVDA with Firefox, or IE, or Chrome + ChromeVox. I tested, and while Safari and VoiceOver on OS X also provided some feedback, the experience wasn’t as polished as one would hope. So if you’re on the Mac, using Google Chrome and ChromeVox is probably your best bet.

Also, all of these apps work best if you do not rely on virtual/browse modes when on Windows. In NVDA, it’s easy to just turn it off by pressing NVDA+Space. For JAWS, the shortcut is JAWSKey+Z. Bt since this has multiple settings, consult your JAWS manual to make this setting permanent for the Google Drive domain.

The documentation on Drive is extensive. I suggest to start at this hub and work your way through all linked documentation top to bottom. It’s a lot at first, but you’ll quickly get around and grasp the concepts, which are pretty consistent throughout.

Once you’re ready to dive into Docs, Sheets, Slides and the other office suite apps, use the Docs Getting Started document as a springboard to all the others and the in-depth documentation on Docs itself.

One note, in some places, it is said that creating forms is not accessible yet. However, since there is documentation on that, too, those documents stating that creating forms isn’t accessible yet are out of date. One of those, among other things, is the Administrator Guide to Apps Accessibility.

I found that creating and working in documents and spreadsheets works amazingly well already. There are some problems sometimes with read-only documents which I’ve made the Docs team aware of, but since these are sometimes hard to reproduce, it may need some more time before this works a bit better. I found that, if you get stuck, alt-tabbing out of and back into your browser often clears things up. Sometimes, it might even be enough to just open the menu bar by pressing the Alt key.

Closing remarks

Like with any other office productivity suite, Google Docs is a pretty involved product. In a sense, it’s not less feature-rich than a desktop office suite of programs, only that it runs in a web browser. So in order to effectively use Google Apps, it cannot be said enough: Know your browser, and know your assistive technology! Just tabbing around won’t get you very far!

If you need more information not linked to above, here’s the entry page for all things Google accessibility in any of their apps, devices and services. From here, most of the other pages I mention above can also be found.

And one more piece of advice: If you know you’ll be switching to Google Apps in the future in your company or government or educational institution, and want to get a head start, get yourself a GMail account if you don’t have one. Once you have that, all of Google Drive, Docs, and others, are available to you as well to play around with. There’s no better way than creating a safe environment and play around with it! Remember, it’s only a web application, you can’t break any machines by using it! And if you do, you’re up for some great reward from Google! :)

After some requests, I also produced a little audio demo (about 40 minutes) where I demonstrate a few Gmail, Drive and Docs features.

Enjoy!

Posted in Accessibility, GMail | Tagged , , , , , , , , | 12 Comments

WAI-ARIA for screen reader users: An overview of things you can find in some mainstream web apps today

After my recent post about WAI-ARIA, which was mostly geared towards web developers, I was approached by more than one person on Twitter and elsewhere suggesting I’d do a blog post on what it means for screen reader users.

Well, I’ve got news for all my blind and visually impaired readers: You’re not getting one blog post, you’re getting a whole series instead! :-)

This blog post will kick it off, and I will cover some general uses where you will find that WAI-ARIA improves your user experience. These examples are most useful when using modern screen reader/browser combinations, as is the case with most web stuff today anyway. So if you’re using NVDA or JAWS on Windows, Orca on Linux, or VoiceOver on the Mac, most, if not all, example uses below should work for you in one way or another. Browsers on Windows are Firefox and Internet Explorer, on Linux it’s also Firefox, and on OS X most likely Safari. Chrome and ChromeVox may or may not work in a similar way, but I’ll leave that to those using that combination to test it.

Some WAI-ARIA basics

WAI-ARIA stands for Web Accessibility Initiative – Accessible Rich Internet Applications. It is a World Wide Web Consortium (W3C) standard. It allows web authors to tell assistive technologies that certain constructs of HTML markup — the stuff that makes up web pages — mean something that is not actually available in normal HTML, but maps to some desktop control that users are probably familiar with.

WAI-ARIA has three pillars: Roles, States and Properties, and Live Regions. I will briefly cover each of them below, but not extensively, since this is mostly for end users.

Roles

Roles tell the screen reader that a particular element of a web page is actually meant to be something else. For example, if assigning a role of “button” to a fancy looking clickable element made up of one or more generic HTML elements, screen readers will simply read it as a button in virtual buffer. If the author did a good job of providing keyboard accessibility, you can even tab to it and press Space to activate. All examples below do that.

Roles are oriented along known control types from the desktop. Using WAI-ARIA, you can mark up even fancy stuff like whole tree views, similar to what the folder tree in Windows Explorer is. In a later article, we’ll see such a tree view in action.

States and Properties

States and Properties tell the screen reader something more about a particular control than normal HTML can. For example, even in older web pages, they can tell the screen reader that a field is required or invalid. They can tell whether a button opens a popup or sub menu, if something has an auto-complete, if a fancy checkbox is checked or not, etc. We’ll see a lot of these in action in this and other articles.

Live Regions

Live Regions allow the web author to make the screen reader tell the user about some dynamic updates. For example, a live region can be used to make the screen reader say information like “auto-complete results are available”, or “Your message has been sent successfully”. Status information that is good to know at a particular moment, but not important enough to stick around for long, or even be exposed in a dialog form.

Live regions can be either polite, meaning they speak once your screen reader has finished speaking what it’s currently speaking, or assertive, meaning they will interrupt whatever the screen reader is currently speaking, to immediately make you aware of a certain status change.

OK, now that we’ve got these pillars of WAI-ARIA laid out a bit, let’s get started looking at some examples! it is important that you have JavaScript in your browser turned on. Yes: On, not off! Modern web apps live only because of JavaScript, and turning it off nowadays is like cutting off an arm or foot, it will leave you mostly immobilized on most modern web sites. So throw away those old beliefs that javaScript is no good for accessibility, and turn it back on or uninstall that NoJS add-on you’ve been keeping around!

Each of the below examples will open in a new tab or window on purpose, so you can play around with them a bit and return to the article when you’re done.

Twitter

Twitter got a huge accessibility boast over the last year to a year and a half. Twitter even has a dedicated accessibility team now making sure their web site and mobile apps are becoming more accessible, and stay that way.

When you open Twitter and log in — assuming you have an account –, you can do several things to try out its accessibility.

First, focus on the Tweet Text edit field and invoke focus or forms mode, or whatever that mode is called in your preferred screen reader. Even that alone will give you several bits of information. It will tell you that the edit field is collapsed, that it has a sub menu, that it has an auto-complete, and that it is multi-line. The bits about having a sub menu and being collapsed are coming from WAI-ARIA markup. The other info could be encountered in just any multi-line edit field on the web.

Next, start typing some text. Notice that on occasion, Twitter will speak numbers as the number of available characters that you have left in the tweet decreases. Twitter has a limit of 140 characters per tweet.

Now, let’s add a user name. Type the @ sign. My Twitter handle is MarcoInEnglish (in one word, capitalization is not important). So after the @ sign, type the letter m. If you’re following me, at least one result should come up. The first thing you’ll hear now is that the edit field changes state from “collapsed” to “expanded”. After a moment, depending on your internet connection, you will also hear something like “3 results available” (number may vary). This means that Twitter has found matching handles that start with, or contain, the letter m.

Now, press DownArrow. You will now hear that you are in a list, that a certain list item is selected and what it contains, and that it’s item 1 of 3 (or whichever number of results are being displayed). All this is WAI-ARIA magic. You can press Up and Down to select an entry, Tab or Enter to insert that into your tweet, or Escape to dismiss the AutoComplete and return focus to the edit field without inserting anything, and resume typing.

If you decided to press Enter or Tab, you’ll hear that focus returns to the edit field, and that it is now collapsed again. Your cursor is positioned after the Twitter handle you just inserted, and a space has been added for you so you can continue typing uninterrupted.

Next, let’s look at something else on the Twitter page. Press Escape or whichever hotkey gets you out of forms or focus mode back into browse mode/virtual cursor mode. Find the heading level 2 that says “Tweets”, then DownArrow to the first tweet. After the tweet text and possible links, you’ll find a list of 4 items. The first three items are normal buttons named Reply, Retweet and Favorite. The fourth, however, is a menu button that has a sub menu. Screen readers will announce it as such. Press it. Focus will now shift to a menu, and you have 3 items: Share via E-Mail, Embed, and Report. These are announced as menu items within a menu. Press Escape to dismiss this menu without selecting anything.

These are also WAI-ARIA menus. Menu items, or menu buttons with attached sub menus are normally not available in HTML. However, through WAI-ARIA markup and some focus/keyboard handling via JavaScript, these widgets can be made fully keyboard accessible and expose all relevant information to screen readers, including menus.

Want to get even more app-like? If you’re on Windows, which probably most of you are, turn off your virtual cursor. With NVDA, you do this by pressing NVDA+SpaceBar. JAWS’s shortcut is Insert+Z. You may have to press it twice to also prevent JAWS from reinvoking virtual PC cursor when a new page loads. Once you’ve turned off your browse mode or virtual cursor, start pressing J and K (yes, the letters) to move through tweets. You’re now moving the actual keyboard focus, and through WAI-ARIA labeling, and some live regions on occasion, you are getting a fully accessible experience. Press your Question Mark key to hear which other keyboard shortcuts you have available. You can reply, favorite, retweet tweets, expand conversations and jump to the new tweets that might, in the meantime, have been added while you were browsing your timeline. You can also quickly jump to the compose edit we were in earlier, to write a new fresh tweet. In essence, you might hardly ever need virtual cursor at all on Twitter, because the app-like experience is so good!

On Mac OS X, you don’t even have to worry about switching modes, because there is no virtual cursor, and you can use those Twitter shortcuts right away. In fact, this might be a much more efficient way to navigate the Twitter experience than the VoiceOver commands for web browsing.

All the while, keyboard focus will make sure that pressing Tab will actually move into the tweet you’re currently reading.

Facebook

Facebook has made similar advancements in making their regular web presence more accessible in the last 18 to 24 months. If you log in with your Facebook account and start exploring from the top, you’ll immediately encounter a few menu buttons that have popups attached to them. These are the Friend Requests, Messages, and Notifications buttons. If you have the newest design, the Privacy and Account Settings buttons will also be there.

A bit further below, you will find the Search field. It is announced as a combo edit field, one of those edits that you can either type in or choose from a list. Focus it, and start typing, for example the name of the Facebook Accessibility page. You will automatically hear announcements as results become available. Like you would expect, you can arrow up and down through the results, and if you found the one you were looking for, press Enter to go to that page.

But let’s stay on the main page for now, and find the edit field that alllows you to post a status update. This is not only a normal edit field. It, too, has the possibility to auto-complete names or locations as you type them. If you start typing the name of a friend, a list will pop up that allows you to select from the available auto-complete results, and press Tab to insert that name into the text field, including tagging that person once you post the status update. Unlike we’ve seen on Twitter, the list comes up automatically, but you can continue typing without the danger of selecting something. You will only insert a search result via the Tab key.

Again, listen to what your screen reader tells you about the results, the list items, etc. Also, the widget to post a status update has a few buttons at the top that allow you to switch whether you’re posting a news item, a photo or video. These are called toggle buttons. They are a bit similar to radio buttons, but because they will immediately perform an action once you press them, they’re buttons. Radio buttons normally only change a selection, but don’t cause whole widget environments to change. You will hear the announcement that one, by default the Story item, is pressed, meaning this is the button that is currently active. An analogous construct could be in your word processing toolbar where you have Left, Center, Right, and Justified buttons. Only one of them can be active — or pressed — at a particular time. If one is pressed, the pressed state of another goes away. Same here in this Facebook widget.

All of this is, again, WAI-ARIA. In earlier years, you would not have known which item was the active one, or that these were toggle buttons at all. The auto-complete results would not have spoken as such, either.

There’s one more thing I would like you to try: Find a friend who will be your guineapig and send them a private message. After you send it, leave your focus in the edit field and wait for their reply. Ideally, you’d choose a friend who is currently showing as online. Once she or he replies, you should automatically hear the message and can start replying right away. This is again a live region at work. Once new messages come in, even those that you just sent, they’ll be read out aloud.

Microsoft OneDrive and Office Online

Microsoft also has made some great strides in making their online services more accessible. Their cloud storage is called OneDrive, and it is a web frontend to the files and folders you have stored in their cloud.

If you have an account at Microsoft OneDrive, log in and look at the list that is available on that page. It will list all your files and folders. Here, you can see a not yet fully complete implementation (as of this writing) of WAI-ARIA. For one, the list items are in multiple columns, so you can not only use up and down, but also left and right to move to items. Second, when pressing Enter, screen readers are not yet notified of changes. You have to press an arrow key again to hear that the contents has actually changed. Now, select a file, and press the context menu key. This is also called the windows key and is located to the left of the right control key on most keyboards. You should now hear a context menu open, and when you press up and down, you should hear the selected menu item. Yes, you should, but you won’t. :-) Because this is currently still a bug in the implementation. The context menu appears, you can move up and down, but the newly focused item is not yet being communicated to screen readers. Yup, it’s a bug, and I let Microsoft know about it when I found it just now. :-)

As a workaround, you can turn virtual mode back on manually (NVDA+Space or for JAWS the NUMPAD PLUS key), move to the end of the virtual document, and find the menu items there. Arrow to the one you want and press Enter to select and activate it.

The Word document, if you selected one, will now open in Office Online. I wrote a more detailed review of Office Online already, so I strongly suggest you read that for an in-depth look at what Word in the browser has to offer! And it’s all done through a combination of modern web technologies, including WAI-ARIA. The tool bars and ribbons etc. are all rich internet application stuff.

More goodies are coming

The use of WAI-ARIA is spreading. Since many web sites use re-usable components such as jQueryUI and others nowadays, making these accessible will bring better accessibility to many web sites automatically. Other components, such as the TinyMCE and CKEditor rich web editors, have also made great strides in accessibility in the last year or two, and sites using these will get that accessibility for free. If you have a WordPress blog on version 3.9, try using the visual editor for a change, and from the edit area, press Alt+F10, for example! :-)

Sometimes, as shown in the Microsoft OneDrive case above, things may not be fully implemented yet. Yes, these things can happen, and they happened on the desktop before, too. The best is to provide constructive feedback to the site owners and point out where the problems lie exactly. That way, they can be fixed most easily.

In the next few blog posts on this series, I will take an in-depth look at Google apps like GMail, Contacts and calendar, Google Drive, Docs, Sheets and Slides, etc. These have become more and more important in many work as well as educational environments, and Google has made great progress in the last 6 to 9 months alone making their offerings much more accessible than they were before. Their documentation is also generally pretty good, but I’d like to provide my own perspective nevertheless, and provide a few tips and tricks and point out possible caveats. So stay tuned!

I hope this article helped you get a bit of a glimpse into what WAI-ARIA is and what good it can do for you as a blind or visually impaired user when used properly. More and more web sites, components and companies are using it.

And if anyone tries to tell you that WAI-ARIA is not important or its importance is greatly overstated, ask them if they can do what you can do, without using WAI-ARIA, and without offering any 3rd party app, just the web browser and assistive technology. WAI-ARIA is important, and it is being enhanced to provide even richer and more accessible experiences in the future. More and more apps are moving to a web-based user interface for both desktop and mobile operating systems, and it is important to have a technology that can make these rich applications as accessible as their native counterparts or legacy offerings. The technology is there, it’s working, so let’s use it!

So keep your JavaScript turned on, fear not, and embrace the future of web apps!

Posted in Accessibility, ARIA | Tagged , , , , , | 13 Comments

Improvements to the handling of the aria-pressed attribute

On Monday this week, Heydon Pickering brought to my attention that Firefox has a problem with the way it handles the aria-pressed attribute in some circumstances. aria-pressed is used on buttons (or elements that have a WAI-ARIA role of “button”) to turn it into a toggle button which can be either pressed or not. Think of the Bold, Italic, etc. toolbar buttons in your favorite word processor as an example.

The problem Heydon uncovered in his simple toolbar example was that we didn’t expose the “pressed” state in his case. Upon investigation, both Steve Faulkner and I found that a) a native button element was used and b) adding the role of “button” fixed the problem.

This was certainly not the way we should handle a native button which has the aria-pressed attribute added, when we already turned its role from “button” into “toggle button”. Because we’re dealing with a native button, adding role=”button” should not at all be necessary.

I decided to dive into the code myself and fix the problem. This was my first dive into the C++ core of Firefox in way over a year, and it turned out to be a much bigger project than I originally thought, in which I learned a lot about some of the new structure Alex and Trevor had been introducing. But in the end, we now have:

  1. You can use the aria-pressed state attribute on a native button, and its states will correctly be exposed.
  2. When state changes, assistive technologies will be notified via an event, so NVDA’s virtual buffer will immediately reflect the change, for example.
  3. While I was there, I also removed the “checkable” state from the toggle button accessible. Checkable is something reserved for checkboxes and radio buttons. Toggle buttons are slightly different not only in visual appearance, but also in that they can be either: Stand-alone like your attribute toolbar buttons “Bold”, “Italic” etc., or part of a group where only one can be checked at a time, like “Left justified”, “right justified” etc. Applying the checkable state to these is not appropriate, and it caused screen readers to say nonsense like “A to Z toggle button pressed not checked”. Well, that nonsense is no more! :)
  4. Finally, i also updated the bridge code between the Gecko core and the Android and Firefox OS output to work with this new set of changes, so there, toggle buttons will now also properly speak and braille.

I just checked this code into our tree called Mozilla-Inbound, from where it will travel to Mozilla-Central within the next day or so, from where Firefox Nightly builds are made. So those of you on the bleeding edge will probably see this appear in a build around Sunday April 6, or Monday April 7 or so. This will then be in the Firefox 31 release.

Thanks to Heydon for finding this bug in Firefox! And thanks to Alex for his support while I muddled through some of the new stuff in core! :) This was fun, and it felt good to write a real patch again after such a long time where I mostly did testing and evangelism.

Posted in ARIA | Tagged , | 1 Comment

What is WAI-ARIA, what does it do for me, and what not?

On March 20, 2014, the W3C finally published the WAI-ARIA standard version 1.0. After many years of development, refinement and testing, it is now a web standard.

But I am often asked again and again: What is it exactly? What can it do for me as a web developer? And what can it not do?

I often find that there are assumptions made about WAI-ARIA that are not correct, and the use of it with such wrong assumptions often lead to sites that are less accessible than when ARIA is not being used at all.

In addition, Jared W Smith of WebAIM just yesterday wrote a very good blog post titled Accessibility Lipstick on a Usability Pig, highlighting another related problem: Even though a website may suck usability-wise, pouring WAI-ARIA sugar on it somehow forces it into compliance, but it still sucks big time.

So with all these combined, and after receiving encouragement by Jason Kiss on Twitter, I decided to write this post about what WAI-ARIA is, what it can do for you as a web developer, and what it cannot do. Or rather: When should you use it, and more importantly, when not.

I realize such articles have been written before, and these facts have also all been stressed time and again in talks by various good people in the field of web accessibility. But it is such an important topic that it is finally time for this blog to have such an all-encompassing article as well.

So without further due, let’s jump in!

What is it?

WAI-ARIA stands for “Web Accessibility Initiative – Accessible Rich Internet Applications”. It is a set of attributes to help enhance the semantics of a web site or web application to help assistive technologies, such as screen readers for the blind, make sense of certain things that are not native to HTML. The information exposed can range from something as simple as telling a screen reader that activating a link or button just showed or hid more items, to widgets as complex as whole menu systems or hierarchical tree views.

This is achieved by applying roles and state attributes to HTML 4.01 or later markup that has no bearing on layout or browser functionality, but provides additional information for assistive technologies.

One corner stone of WAI-ARIA is the role attribute. It tells the browser to tell the assistive technology that the HTML element used is not actually what the element name suggests, but something else. While it originally is only a div element, this div element may be the container to a list of auto-complete items, in which case a role of “listbox” would be appropriate to use. Likewise, another div that is a child of that container div, and which contains a single option item, should then get a role of “option”. Two divs, but through the roles, totally different meaning. The roles are modeled after commonly used desktop application counterparts.

An exception to this are document landmark roles, which don’t change the actual meaning of the element in question, but provide information about this particular place in a document. You can read more about landmarks in my WAI-ARIA tip #4. Also, if you’re using HTML5, there are equivalent elements you might want to use as well.

The second corner stone are WAI-ARIA states and properties. They define the state of certain native or WAI-ARIA elements such as if something is collapsed or expanded, a form element is required, something has a popup menu attached to it or the like. These are often dynamic and change their values throughout the lifecycle of a web application, and are usually manipulated via JavaScript.

What is it not?

WAI-ARIA is not intended to influence browser behavior. Unlike a real button element, for example, a div which you pour the role of “button” onto does not give you keyboard focusability, an automatic click handler when Space or Enter are being pressed on it, and other properties that are indiginous to a button. The browser itself does not know that a div with role of “button” is a button, only its accessibility API portion does.

As a consequence, this means that you absolutely have to implement keyboard navigation, focusability and other behavioural patterns known from desktop applications yourself. A good example can be read about in my Advanced ARIA tip about tabs, where I clearly define the need to add expected keyboard behavior.

When should I not use it?

Yes, that’s correct, this section comes first! Because the first rule of using WAI-ARIA is: Don’t use it unless you absolutely have to! The less WAI-ARIA you have, and the more you can count on using native HTML widgets, the better! There are some more rules to follow, you can check them out here.

I already mentioned the example with buttons versus clickable divs and spans with a role of “button”. This theme continues throughout native roles vs. ARIA roles and also extends to states and properties. An HTML5 required attribute has an automatic evaluation attached to it that you have to do manually if you’re using aria-required. The HTML5 invalid attribute, combined with the pattern attribute or an appropriate input type attribute will give you entry verification by the browser, and the browser will adjust the attribute for you. All of these things have to be done manually if you were using aria-invalid only. A full example on the different techniques for form validation can be found in a blog post I wrote after giving a talk on the subject in 2011.

Fortunately, this message seems to finally take hold even with big companies. For example, the newest version of Google Maps is using button elements where they used to use clickable divs and spans. Thanks to Chris Heilmann for finding this and pointing it out during an accessibility panel at Edge Conf (link to Youtube video) in March 2014!

Here’s a quick list of widget roles that have equivalents in HTML where the HTML element should be preferred whenever possible:

WAI-ARIA role Native element Notes
button button use type=”button” if it should not act as a submit button
checkbox input type=”checkbox” -
radiogroup and radio fieldset/legend and input type=”radio” fieldset is the container, legend the prompt for which the radio buttons are the answer for, and the input with type of “radio” are the actual radio buttons
combobox select size=”1″ Only exception is if you need to create a very rich compond widget. But even then, combobox is a real mess which warrants its own blog post.
listbox select with a size greater than 1 Only exception is if you create a rich auto/complete widget
option option As children of select elements or combobox or listbox role elements
list ul or ol Not to be confused with listbox! list is a non-interactive list such as an unordered or ordered list. Those should always be preferred. Screen readers generally also supporting nesting them and get the level right automatically.
spinbutton input type=”number” If the browser supports it.
link a with href attribute Should, in my humble opinion, never ever ever be used in an HTML document!
form form Nobody I know from the accessibility community can actually explain to me why this one is even in the spec. I suspect it has to do primarily with SVG, and maybe EPUB.

The reason all these roles to native elements mappings are in there at all are because of the fact that WAI-ARIA can also be applied to other markup such as EPUB 3 and SVG 2. Also, some elements such as spin buttons and others are new in HTML5, but because WAI-ARIA was originally meant to complement HTML 4.01 and XHTML 1.x, and HTML5 was developed in parallel, roles, states and properties were bound to overlap, but got more defined behaviors in browsers for HTML5.

Likewise, you should prefer states such as disabled and required over the WAI-ARIA equivalents aria-disabled and aria-required if you’re writing HTML5. If you write HTML 4.01 still, this rule does not apply. If you’re specifically targetting HTML5, though, there is not really a need for the aria-required, aria-disabled and aria-invalid states, since the browsers take care of that for you. And yes, I know that I am in disagreement with some other accessibility experts, who advise to use both the HTML5 and WAI-ARIA attributes in parallel. Problem with that is, in my opinion, that it is then extra work to keep the WAI-ARIA attributes in sync. Especially with the aria-invalid attribute, this means that you’ll still have to put in some JavaScript that responds to the state of the HTML5 form validation state.

Why this duplication? Why can’t the browser just also react to the WAI-ARIA attributes as it does to the HTML ones?

This is a question I am getting a lot. The simple answer is: Because WAI-ARIA was never meant to change browser behavior, only expose extra information to assistive technologies. The more complex answer is: WAI-ARIA can be applied to XHTML 1.0 and 1.1, HTML 4.01, and HTML5. The HTML5 attributes can only be applied to HTML5 capable browsers, including mobile, but will give you all kinds of extra features defined in the standard for these attributes. If the WAI-ARIA attributes were suddenly made to influence actual browser behavior, the level of inconsistency would be enormous. To keep WAI-ARIA clean and on a single purpose, it was therefore decided to not make WAI-ARIA attributes influence browser behavior ever.

When should I use it?

Whenever you create a widget that is not indiginous to your host language, e. g. HTML. Examples of such widgets include:

  • tree with treeview children: When creating something the user should be navigating in a hierarchical fashion. Examples can be seen in the folder tree in Windows Explorer or the list of topics and books in the Windows HTML help viewer.
  • menu with menuitem, menuitemcheckbox, and menuitemradio children: A real menu system as can be seen in Windows or OS X apps.
  • grid or treegrid, with gridrow, columnheader, rowheader, and gridcell children: An editable grid similar to a spreadsheet. Not to be confused with data tables which use native HTML table, tr, th, and td elements with proper scope and other attributes.
  • toolbar container with actual buttons or other interactive elements as children.
  • dialog as container for a message or input prompt that needs to be acknowledged or dismissed in a modal fashion, e. g. no other part of the web page or app can be interacted with at the moment. The role alertdialog is related, although I recommend not using it since its support is inconsistent.
  • application, in which you control every single focus and keyboard interaction. For a detailed discussion, see my article on the use of the application role.

In all of the above cases, it is your responsibility to:

  1. Familiarize yourself with the expected user interaction with both mouse and keyboard.
  2. Make sure all elements can be reached via the tab key, and that focus is always visible. Use the tabindex with a value of 0 to put stuff in the tab order at their natural position within the document.
  3. Make sure some items are not necessarily reachable via the tab key. For example in a tool bar, items should be navigated via left and right arrows, and tab should jump straight to the next container (tool bar or other compound widget), and Escape should return to the main application area. Likewise, a tab list should be navigated with left and right arrows, too, and tab itself should jump into the tab panel that is active.
  4. You manage focus appropriate to the type of widget you are in. For example, when opening a dialog (role “dialog”), it is your responsibility to set keyboard focus to it or its first focusable child element, and to manage modality, meaning you must make sure tab does not wander outside the confines of the dialog as long as it is open. Escape should generally cancel, and a dedicated OK/Accept/Save button should close with changing. Make sure once you’re done with a dialog, you return focus to a defined position in your application/document, like the button that opened the dialog in the first place, or keyboard and screen reader users might get lost.

If you do not adhere to the common interaction patterns associated with certain of these roles, your WAI-ARIA sugar might very quickly turn into sour milk for users, because they get frustrated when their expected keyboard interaction patterns don’t work. I strongly recommend studying the WAI-ARIA 1.0 Authoring Practices and keeping them handy for reference because they provide a comprehensive list of attributes and roles associated to one another, as well as more tips on many of the things I just mentioned. Another very good resource is the afore mentioned Using WAI-ARIA in HTML document which provides an extensive technical, yet easy to understand, reference to best practices on how to apply WAI-ARIA code, and also, when not to do it.

Closing remarks

I realize that to someone just getting started with web accessibility, these topics may seem daunting. However, please remember that, like learning HTML, CSS and JavaScript, learning the intricacies of web accessibility means you’re learning a new skill. The more often you use these techniques, the more they become second nature. So don’t be discouraged if you at first feel overwhelmed! Don’t be shy to ask questions, or pull in people for testing! The accessibility community is generally a very friendly and helpful bunch.

One thing that may also help with motivation, and it’s been thankfully mentioned more and more often: Accessibility and usability go hand in hand. The more you improve the usability, the more it gets you there in terms of accessibility, too. And first and foremost, it’s about people! Not about some WCAG technique, not about a law that needs to be fulfilled, but about people actually using your web site or web application. Fulfilling legal requirements and WCAG techniques then come naturally.

So: Make WAI-ARIA one of your tools in your arsenal of tools for web development, but take its first rule to heart: Don’t use it unless you absolutely have to. Get the usability right for keyboard users, make sure stuff is properly visible to everyone even when they’re standing outside with the sun shining down on their shiny mobile phones or tablets (thanks to Eric Eggert for that tip!), and use WAI-ARIA where necessary to provide the extra semantic screen readers need to also make sense of the interactions. With people in mind first, you should be good to go!

Happy coding!

Posted in Accessibility, ARIA | Tagged , | 19 Comments

Easy ARIA Tip #7: Use “listbox” and “option” roles when constructing AutoComplete lists

One question that comes up quite frequently is the one of which roles to use for an auto-complete widget, or more precisely, for the container and the individual auto-complete items. Here’s my take on it: Let’s assume the following rough scenario (note that the auto-complete you have developed may or may not work in the same, but a similar way):

Say your auto-complete consists of a textbox or textarea that, when typing, has some auto-complete logic in it. When auto-complete results appear, the following happens:

  1. The results are being collected and added to a list.
  2. The container gets all the items and is then popped into existence.
  3. The user can now either continue typing or press DownArrow to go into the list of items.
  4. Enter or Tab select the current item, and focus is returned to the text field.

Note: If your widget does not support keyboard navigation yet, go back to it and add that. Without that, you’re leaving a considerable amount of users out on the advantages you want to provide. This does not only apply to screen reader users.

The question now is: Which roles should the container and individual items get from WAI-ARIA?Some think it’s a list, others think it’s a menu with menu items. There may be more cases, but those are probably the two most common ones.

My advice: use the listbox role for the container, and option for the individual auto-complete items the user can choose. The roles menubar, menu, and menuitem plus related menuitemcheckbox and menuitemradio roles should be reserved for real menu bar/dropdown menu, or context menu scenarios. But why, you may ask?

The short version: Menus on Windows are a hell of a mess, and that’s historically rooted in the chaos that is the Win32 API. Take my word for it and stay out of that mess and the debugging hell that may come with it.

The long version: Windows has always known a so-called menu mode. That mode is in effect once a menu bar, a drop-down menu, or a context menu become active. This has been the case for as long as Windows 3.1/3.11 days, possibly even longer. To communicate the menu mode state to screen readers, Windows, or more precisely, Microsoft Active Accessibility, uses four events:

  1. SystemMenuStart: A menu bar just became active.
  2. SystemMenuPopupStart: If a SystemMenuStart event had been fired before, a drop-down menu just became active. If a SystemMenuStart event had not been fired before, a context menu just became active. If another SystemMenuPopupStart preceeded this one,  a sub menu just opened.
  3. SystemMenuPopupEnd: The popup just closed. Menu mode returns to either the previous Popup in the stack (closing of a sub menu), the menu bar, or falls out of menu mode completely.
  4. SystemMenuEnd: A menu bar just closed.

These events have to arrive in this exact order. Screen readers like JAWS or Window-Eyes rely heavily on the even order to be correct, and they ignore everything that happens outside the menus once the menu mode is active. And even NVDA, although it has no menu mode that is as strict as that of other “older” screen readers, relies on the SystemMenuStart and SystemMenuPopupStart events to recognize when a menu gained focus. Because the menu opening does not automatically focus any item by default. An exception is JAWS, which auto-selects the first item it can once it detects a context or start menu opening.

You can possibly imagine what happens if the events get out of order, or are not all fired in a complete cycle. Those screen readers that rely on the order get confused, stay in a menu mode state even when the menus have all closed etc.

So, when a web developer uses one of the menu roles, they set this whole mechanism in motion, too. Because it is assumed a menu system like a Windows desktop app is being implemented, browsers that implement WAI-ARIA have to also send these events to communicate the state of a menu, drop-down or context or sub menu.

So, what happens in the case of our auto-complete example if you were to use the role menu on the container, and menuitem on the individual items? Let’s go back to our sequence from the beginning of the post:

  1. The user is focused in the text field and types something.
  2. Your widget detects that it has something to auto-complete, populates the list of items, applies role menuitem to each, and role menu to the container, and pops it up.
  3. This causes a SystemMenuPopupStart event to be fired.

The consequences of this event are rather devastating to the user. Because you just popped up the list of items, you didn’t even set focus to one of its items yet. So technically and visually, focus is still in your text field, the cursor is blinking away merrily.

But for a screen reader user, the context just changed completely. Because of the SystemMenuPopupStart event that got fired, screen readers now have to assume that focus went to a menu, and that just no item is selected yet. Worse, in the case of JAWS, the first item may even get selected automatically, producing potentially undesired side effects!

Moreover, the user may continue typing, even use the left and right arrow keys to check their spelling, but the screen reader will no longer read this to them, because their screen reader thinks it’s in menu mode and ignores all happenings outside the “menu”. And one last thing: Because you technically didn’t set focus to your list of auto-complete items, there is no easy way to dismiss that menu any more.

On the other hand, if you use listbox and option roles as I suggested, none of these problems occur. The list will be displayed, but because it doesn’t get focus yet, it doesn’t disturb the interaction with the text field. When focus gets into the list of items, by means of DownArrow, the transition will be clearly communicated, and when it is transitioning back to the text field, even when the list remains open, that will be recognized properly, too.

So even when you sighted web developers think that this is visually similar to a context menu or a popup menu or whatever you may want to call it, from a user interaction point of view it is much more like a list than a menu. A menu system should really be confined to an actual menu system, like the one you see in Google Docs. The side effects of the menu related roles on Windows are just too severe for scenarios like auto-completes. And the reason for that lies in over 20 years of Windows legacy.

Some final notes: You can spice up your widget by letting the user know that auto-complete results are available via a text that gets automatically spoken if you add it in a text element that is moved outside the viewport, but apply an attribute aria-live=”polite” to it. In addition, you can use aria-expanded=”true” if you just popped up the list, and aria-expanded=”false” if it is not there, both applied to your input or textarea element. And the showing and hiding of the auto-complete list should be done via display:none; or visibility:hidden; and their counterparts, or they will appear somewhere in the user’s virtual buffer and cause confusion.

A great example of all of this can be seen in the Tweet composition ContentEditable on twitter.com.

I also sent a proposal for an addition to the Protocols and Formatting Working Group at the W3C, because the example in the WAI-ARIA authoring practices for an auto-complete doesn’t cover most advanced scenarios, like the one on Twitter and others I’ve come across over time. Hope the powers that may be follow my reasoning and make explicit recommendations regarding the use of roles that should and shouldn’t be used for auto-completes!

Posted in Accessibility, ARIA | Tagged , | 7 Comments

Why screen reader detection on the web is a bad thing

On February 26, 2014, the webAIM project published the results of their 5th screen reader user survey. Two questions were new in this survey that pertain to a recently growing desire of some web developers to know whether they’re dealing with assistive technologies on the other end or not. The results were rather shocking to me as a representative of a browser vendor and experienced assistive technology user:

78.4% of respondents were very comfortable or somewhat comfortable with letting a web site know they are using a screen reader. And even 86.5% were very or somewhat comfortable if doing so resulted in better accessibility of the website. For each question, only roughly 2% of respondents answered “I don’t know” to that question.

In my opinion, it should have been the other way around. I took the questionaire myself back in January, and because I am involved in making a web browser and knowing about the various desires by web developers and some standards people, I knew what was meant here, and also knew the implications. I submit, carefully, and without wanting to diminish anyone’s maturity in this matter, that most of those who responded didn’t actually know what the screen reader detection actually means. There was just too little context given, and information on this subject is also not so easy to find on the web.

For one, letting a website know you’re using a screen reader means running around the web waving a red flag that shouts “here, I’m visually impaired or blind!” at anyone who is willing to look. It would take away the one place where we as blind people can be relatively undetected without our white cane or guide dog screaming at everybody around us that we’re blind or visually impaired, and therefore giving others a chance to treat us like true equals. Because let’s face it, the vast majority of non-disabled people are apprehensive in one way or another when encountering a person with a disability.

Second, this is just a modern form of the text-only alternative web sites of the 1990s when screen readers were still learning to stumble around graphical user interfaces, let alone the web, and those text-only alternatives were a necessary evil to get anything done in most cases. In the early 2000s, as screen readers became better and browsers more modern, those faded away. And only in recent years has the trend sort of reverted, primarily because people just don’t know, or don’t care, to make their new shiny dynamic web sites and widgets properly accessible, even though the techniques exist to do just that. The current worst of the bad examples is the “accessible alternative” of the Audible.com web site. For one, its feature set is limited, and second, it constantly throws you back to the standard web presence, when doing searches, for example, and other totally screwed user experience snafoos. Or is there anyone out there who is actually wanting to tell me they enjoy this and find it even remotely useful? And to add insult to injury, the stuff that is currently a bit tricky on the standard web site can easily be fixed by some WAI-ARIA markup and some consistent keyboard navigation JavaScript.

And the old arguments still apply: Alternative versions of web sites tend to get out of date, no longer being maintained, and creating and maintaining them is more expensive than properly applying markup and learning the web accessibility skill properly once.

Third, there are security implications. Ever since I started working for Mozilla in 2007, there have been repetitive inquiries about exposing whether a screen reader is running with the browser, and if so, also granting access to the accessibility APIs to content JavaScript to manipulate what screen readers read to the user. Either of these would give web sites knowledge or exposure of operating system bits that, in every other case, would be considered unacceptable. You wouldn’t want to give web site JavaScript access to your file system, which type of keyboard you’re using, what brand your processor is, or if your machine still has a floppy drive or not, would you? Accessibility APIs hook into all kinds of channels to expose all the needed information to assistive technologies. Granting a web site access to these APIs would open a level of access to a computer that is just unacceptable, and unnecessary, for a web site to have.

Some might argue that apps on mobile devices have access to that information, even through defined APIs, and the same is true on desktops. Yes, but the entry for an app is much higher. There is usually an installation process, and apps have by definition access to more information. They aren’t as casually visited as a web site is. Even on Firefox OS, there is a 3 level app privilege model, even though Firefox OS is built of web technologies, and not all of these levels will have access to the information whether the screen reader is running or not.

Aural style sheets never gained acceptance because the expected gain was just not there, and the desire to have equal access to the same content as everybody else was much stronger. More recent requests all are just the same thing in different disguise.

WebAIM should either have given more context to this question, especially since it was not a topic even most advanced screen reader users would be familiar with, or they should have asked a third question highlighting the privacy/security context. Funny enough, the question about plain text alternatives was answered with “seldom or never” by almost 30% of respondents, so the desire to use such sites in general is much lower than the two screen reader detection questions might suggest. So I again submit that only the lack of proper context made so many people answer those questions differently than the one about plain text alternatives.

It is my sincere hope that the work currently done by the IndieUI group at the W3C will provide enough incentive for web authors not to abuse the querying mechanisms described, and that this will only be used by very well-informed sites where the developers provide actual value. The privacy examples in that document look at least promising. Let’s hope it all holds up until this becomes a standard one day. I, for one, will strongly discourage web developers who ask me from using this unless I am personally convinced they know what they’re doing. And yes, like with Location sharing, I will personally be very restrictive about allowing sites access to that particular information if I’m asked.

Posted in Accessibility | 29 Comments

WAI-ARIA showcase: TinyMCE 4.17

Today, on February 26, the TinyMCE team released version 4.17 of the JavaScript/HTML editor for the web. In the release notes, the first item mentioned is much improved accessibility support. For those of you not familiar with TinyMCE: It is the WYSIWYG (What You See Is What You Get), AKA visual or rich editor used in many popular content management systems such as the WordPress blogging software.

Over the past few weeks, I had the pleasure of testing the improved accessibility features and provide feedback to the developers. Here are a few hints to get you going, but for a full experience, I suggest you try it out yourself!

I tested with Firefox and NVDA and JAWS, but you should also get good results with Internet Explorer and other browser/screen reader combinations. Since the two mentioned work so well, it can be trusted that the implementation is very solid.

When you navigate or tab to the sample editor instance linked to above, you’ll land in the content area with some sample text. Arrowing up and down will give you paragraphs, headings and other items such as links. Note that NVDA and Firefox treat links as embedded characters, you have to actually navigate into them to read their texts, but they are accessible.

From the content, you can press Alt+F9 to open a menu bar. Left and Right arrow navigate through the top menu items like in your average desktop app, and Down Arrow will actually open a menu. Note that from there, you cannot go Right or Left through the various drop-down menus, you first have to press Escape once to go back to the top level. When on sub menus, Right Arrow will walk into them, escape will back up.

Alternatively, you can press Alt+F10 to go into the toolbars. You’ll land on the first tool bar and can tab or arrow through the items. Sub menu items will have the sub menus or panels opened via Down Arrow, and closed via Escape. When you reach the end of one toolbar, Right Arrow and LeftArrow wrap to the next and previous tool bars respectively. Escape will bring you back into the main content area.

From either the menus or toolbars, you can open dialogs such as Insert/Edit Image/Video. These are multi-page dialogs with at least two tabs at the top. When in the tab list, Right and Left arrow will move between the tabs, Space will make the current tab active. Tab moves into the panel and through the controls. OK will accept changes, Escape or Cancel will discard.

On the tool bars and in the formatting menu, you can insert emoticons or change the text and background colors. These are panels you open by pressing DownArrow or Enter on their respective menu/toolbar items, and then use the arrow keys to select the actual item you want applied/inserted. All of these items are also made to talk, so a blind person can change the text color as well and know what they’re doing.

All of this is due to the great use of the WAI-ARIA (Accessible Rich Internet Applications) markup. As soon as content management systems start integrating this new version, they’ll automatically benefit from these improvements. This opens the door for even more sites offering better than plain-text editing on the web in an accessible fashion. I cannot even start to count how often I was frustrated in the past because while composing a blog post in plain text, some single symbol missing or fat-fingered markup caused whole posts to fall apart. I personally favor rich editing over plain text editing because I do not have to worry about the markup any more and can concentrate on what I want to say instead. If I want to insert a link or change the formatting of something, I just select the text and apply the formatting I want.

I congratulate the TinyMCE team on this new release and am looking forward to even more good accessible features in the future!

Posted in Accessibility, ARIA | Tagged , | 9 Comments

Some do’s and dont’s we gathered from making the Firefox OS UI more accessible

In my last post, I mentioned that we learned a few things from making the Firefox OS UI, code-named Gaia, more accessible over the past few months. This produced quite a few questions about what these things were. So rather than adding them to that blog post, here’s a dedicated one to just that topic.

First, a big thank you to Yura and Eitan for their input on this. It is mostly their patches that I learned these things from, I only played with some of the code myself and gave up when I was notified how much of the CSS broke by my code changes. :)

Use buttons or links, not clickable divs and spans

Should be pretty obvious, but you would be surprised at how many clickable divs and spans there can be inside a single web app. Why that is, escapes me really. And despite what Google’s accessibility evangelists say in their videos, the rest of the accessibility community agrees with me that clickable spans and divs instead of buttons or links are not good practice!

Especially buttons are no longer hard to style to one’s liking nowadays. Moreover, buttons are focusable by default, they respond to the “disabled” attribute and are taken out of the focus order, and you get the onClick mechanism for free that usually responds to mouse, keyboard and touch. Moreover, if you just add the type=”button” attribute to a button element inside a form, it no longer is a submit button.

Second preferred thing should be links, however links don’t respond to the disabled attribute and may be harder to style. But there are places where they are definitely the appropriate thing to choose and should not be replaced by anything else.

Don’t use WAI-ARIA roles in CSS selectors

Although technically possible, this is extremely bad practice in our experience. First, WAI-ARIA markup has a very specific purpose. it is there to make HTML elements accessible to screen readers when these are used in a way they were not originally designed for. An example are tabs. HTML has no tabs widget, so compound HTML blocks must be used to simulate them. There is a very specific path to follow for these. If CSS selectors were now based on that markup, and that markup needs to change, suddenly a whole bunch of CSS needs to be touched to, or the layout will break.

In our case, WAI-ARIA roles had been inserted without actual knowledge of what they were meant for, and CSS selectors were based on that. Moreover, other parts of markup then used those CSS selectors by making simple rows of buttons inherit some of the tab styling, which made those buttons expose totally inappropriate semantics to screen readers.

So whatever you do, do not use WAI-ARIA roles or attributes as selectors for your CSS. Keep the visuals clearly separated from the screen reader-intended markup!

Hide inactive content properly

If your app is made in a way where every screen gets its own HTML page, you’re out of the woods. If, on the other hand, several screens are put within one HTML file, you need to absolutely make sure that content is hidden properly. Like the sighted user, you want to trap the screen reader user on one particular touch screen at a time. All touch-screen-supporting screen readers support a sequential method of going through available items on a screen. If you don’t hide the inactive/invisible parts of your app from the screen reader user, they are now suddenly able to move to parts you don’t intend them to go. They might even try to activate items in parts that are currently not appropriate, potentially breaking your app’s logic. We found three methods to work best when hiding content:

  1. Use styling like display: none; or visibility: hidden; for those parts you don’t want shown. That is the most reliable method to hide content, since screen readers don’t ever expose such content. There is a more thorough explanation of the mechanisms and things to consider is described in another post on this blog.
  2. If the above does not work for you for some reason, because you may encounter performance or reflow issues, resort to using aria-hidden. Hide everything except the active screen.

Use common accessibility practices

Most accessibility practices you’ve learned for desktops also apply for mobile. Use alt text, or if that doesn’t work out for some reason, aria-label for images, form labeling techniques as usual, and make sure you have a logical order of your HTML, because on mobile, too, screen readers infer the reading order from the order of the elements in the document object model.

As if on queue, the BBC accessibility team have published their mobile accessibility guidelines today. And although tailored towards the BBC primarily, all of the techniques and examples found there are valid for any mobile web app and its accessibility. I strongly recommend you check them out! If I was your teacher, I’d say: Mandatory read! :)

Posted in FirefoxOS, Mobile | Tagged , , | 6 Comments

Accessibility in Firefox OS: An update

After my introductory video, quite some things happened in the realm of Firefox OS accessibility, and although we’re still not quite ready yet to release builds to beta testers, we’re getting closer to a state where this will be the case.

In the interim, I’d like to share a few things that have happened over the past few months that may get you a bit excited! We certainly are! ;)

A screen reader toggle

It recently became possible to toggle the screen reader and thus gain access to a Firefox OS device running the Feb 12 or later nightly builds of Firefox OS 1.4/Master. The way this works is as follows:

  1. Press the Power button three times, with close to one second pause between each press. So unlike iOS, which some of my readers may be familiar with, here it’s slowness that matters. The reason is that currently, the Power button doesn’t register properly when you press it too fast in rapid succession.
  2. You’ll then hear an announcement instructing you to press the Power button three more times in the same manner to toggle screen reader on.
  3. Do so, and you’re ready to go.

This even works in the Welcome wizard/First Time User experience screen. In essence, I can now setup a Firefox OS phone as a blind user.

Turning the screen reader off works in a similar fashion: Press the Power button three times in slow succession, confirm that you want to end it, and off it is switched.

Speech property controls

Also, recently, speech controls were added to an Accessibility panel in the Settings app. You can control the screen reader on/off state itself, the speech rate and the volume.

The Accessibility panel is enabled when the screen reader is turned on, or it can be enabled from Developer settings, which in turn can be enabled from Device Info/More information. This, of course, is only interesting for sighted folks interested in trying things out, a blind user will most likely use the toggle. The important thing to note is that the screen reader setting has been moved out from the developer settings into the Accessibility panel in 1.4. So the occurrences that we had over the past couple of weeks where people turned on the screen reader accidentally and then didn’t know how to work their devices, should occur less. ;)

Automatic focusing on new elements

When switching pages, bringing up an application or a website, the first element is now automatically focused after load. This is something Firefox for Android 30 and later will also do when a new web page loads. With that, speech response is much moe intuitive.

Accessible Settings

The Settings app in Firefox OS is the first to become fully accessible. It was the ground work laying app for us and turned out pretty well. We learned a lot about how to style, or not style, stuff, how to properly use techniques to only have items to swipe to that are actually relevant, and other details.

Problems, problems, problems

Oh yes, there are still a lot of problems, primarily in the user interface itself. The whole user interface of Firefox OS is written in HTML5, JavaScript and CSS, and when we started making it accessible, some of the code was pretty rough in terms of semantics. But Eitan and Yura sank their teeth into it, and despite several refactorings and such, we’re getting to a point where important work is being done that will make a lot of apps at once much more accessible. Along the way, we’re setting some rules for everyone on the Gaia team to follow, and implementing tests to make sure breakage occurs less often in the future. Some of these rules include things like “don’t use WAI-ARIA roles as parts of CSS selectors”, for example.

Everyone who has made a so far mostly inaccessible web app accessible knows how much tedious work this can be. And even more so with a dynamic project like Gaia, the Firefox OS user interface. But we’re getting a lot better, and seeing other team members pick up knowledge and attention on proper accessibility semantics. :) Gaia’s code is also improving a lot over-all, and that definitely helps, too!

In summary, we’re working hard, with me testing and other team members coding their butts off to give everyone the best possible user experience in Firefox OS in the near future! Stay tuned for more updates!

Posted in FirefoxOS | Tagged , | 15 Comments

Accessibility in Firefox for Android: Some more technical background, Part II

A long while back, I wrote a post explaining some of the more technical details of the implementation of the accessibility in Firefox for Android. If you want to read the whole post, feel free to do so and then come back here, but for those of you who don’t, here is a short recap of the most important points:

  1. What made the accessibility possible at all in the first place was the fact that Firefox for Android started to have a native Android UI instead of a custom XUL one.
  2. The only thing that needed to be made accessible was the custom web view we’re using, all the rest of the browser UI gained accessibility from using native Android widgets.
  3. The switch to a native UI also gave us the possibility to talk directly to TalkBack and other assistive technology apps.
  4. At the core is the well-known accessibility API also used on the desktop, written in C++. On top of that sits a JavaScript layer, code-named AccessFu, which pulls information from that layer and generates TalkBack events to make everything speak. It also receives the keyboard commands from the Android side, and as we’ll see below, has been substantially extended to also include touch gestures.
  5. There is now also an extended layer of accessibility code on the native Android layer, which I’ll come to below.

Making Explore By Touch work

After that last post, we had to make Explore By Touch work, for both Ice Cream Sandwich and, as it was released shortly after, the all-new Jelly Bean and beyond accessibility enhancements. For that, the JavaScript layer receives touch events from the Android side of things and queries the core engine for the element at the touch point. Also gestures like tapping, swiping, double-tapping and two-finger scrolling are received and processed accordingly.

For Jelly Bean and beyond, we had to do a special trick to make left and right swipes work. Because we’re implementing everything ourselves, we have to fake previous and next accessible elements, making TalkBack and others believe they have true native accessible elements to the left and right of the current element. TalkBack, in effect, only knows about three accessibles on the page. The currently spoken one is always the one in the middle of these three. When the user swipes right to go to the next element, the element to the right becomes the new middle one, gets spoken, and at the same time, the previous middle one becomes the one to the left, and the next regular page element is fetched and becomes the new right element. This particular logic sits in our Android accessibility code and queries the JavaScript portion for the relevant accessibles.

The fact that we have so much control over this stuff, in fact we have to do it this way or it wouldn’t work, allowed us to port the swiping back to Ice Cream Sandwich AKA Android 4.0, which doesn’t natively support that, and even Gingerbread AKA Android 2.3, which doesn’t have Explore By Touch support at all. But in the Firefox web views, it works! Including swiping and double-tapping, two-finger scrolling etc. Unfortunately, there was no way to make the rest of the browser UI accessible by touch on Gingerbread, too, so you’ll still have touse the keyboard for that.

On Jelly Bean and above, we are restricted a bit by what gestures Android supports. In effect, we cannot change the swiping, dluble-tapping, exploring, or two-finger scrolling behavior. We also cannot change what happens when the user swipes up and down. In those instances, Android expects a certain behavior, and we have to provide it. This is why, despite popular request, we currently cannot change the 3 finger swipes left and right to something more comfortable to execute quick navigation. Single-finger swipes left and right are strictly reserved for single object navigation. We’d love it to be different, but we’re bound in this case.

BrailleBack support

Some of the above techniques were used, in a slightly different fashion, to also implement BrailleBack support. As for TalkBack and friends, we have to implement everything ourselves. You have to implement two protocols: com.googlecode.eyes-free.selfbraille.SelfBrailleClient and com.googlecode.eyes-free.selfbraille.WriteData. This isn’t documented anywhere. Our summer intern Max Li did a terrific job dissecting the BrailleBack code and grabbing the relevant pieces from the GoogleCode project, and the result can be seen in Firefox for Android 25 and later. Max also added separate braille utterances, so the output isn’t as verbose as for speech, and follows better logic for braille readers. A few weeks ago, a review of using Android with braille was posted on Chris Hofstader’s blog by a deaf-blind user highlighting how well he could work with Firefox for Android in large part because of its excellent braille support. To reiterate: max was a summer intern at Mozilla last year. He is sighted and had never been in contact with braille before this as far as I know. He implemented this all by himself, occasionally asking me for feedback only. Imagine that, and then getting this review! I’m proud of you, Max!

And Max didn’t stop there: In Firefox 29 and above, an improvements to the way unordered and numbered lists are being presented in braille.

Much of all of this is good for Firefox OS, too

Because of the layered nature of our code, much of what has been implemented for Firefox for Android can be re-used in Firefox OS as well. The differences are mainly in what comes on top of the JavaScript/AccessFu layer. Talking to a synth directly instead of generating TalkBack events, of course using the new WebSpeech API, and in the future we’ll also “only” need to implement BrlTTY or something similar support and connectivity for braille displays. The abstraction allows us to then put the utterances to good use there as well. The main problem we’re having with Firefox OS right now is the actual UI written in HTML, JS, and CSS, code-named Gaia. Getting all the screens right so you cannot swipe to where you’re not supposed to at any given moment, making everything react to the proper activation events etc., and teaching the Gaia team a lot about accessibility along the way are the major tasks we’re working on for that right now. But thanks to the layering of the accessibility APIs and implementations, the transition from Firefox for Android was, though not trivial, not the biggest of our problems on Firefox OS. :)

In summary

The Android API documentation was not much help with all of this. As mentioned, the portion about SelfBrailleClient and friends wasn’t documented anywhere at all, at least I didn’t find anything but source-code references, among them that of Firefox for Android, in a Google search. But also the exact expectations of TalkBack aren’t retrievable just by studying the docs. Eitan had to dive into TalkBack’s source code to actually understand what it was expecting of us to deliver to make it all work.

Here’s to hoping that Google, despite its quest to close-source Android more and more, will keep BrailleBack and TalkBack open sourced so we can continue to read its source code in future updates to keep providing the good support our users have come to expect from us.

Posted in Accessibility, FirefoxOS, Mobile | Tagged , , , | Leave a comment

Switching back to Windows

Yes, you read correctly! After five years on a Mac as my private machine, I am switching back to a Windows machine in a week or so, depending on when Lenovo’s shipment arrives.

You are probably asking yourself, why I am switching back. In this post, I’ll try to give some answers to that question, explain my very personal views on the matters that prompted this switch, and give you a bit of an insight into how I work and why OS X and VoiceOver no longer really fit that bill for me.

A bit of history

When I started playing with a Mac in 2008, I immediately realised the potential this approach that Apple was taking had. Bundling a screen reader with the operating system had been done before, on the GNOME desktop, for example, but Apple’s advantage is that they control the hardware and software back to front and always know what’s inside their boxes. So a blind user is always guaranteed to get a talking Mac when they buy one.

On Windows and Linux, the problem is that the hardware used is unknown to the operating system. On pre-installed systems, this is usually being taken care of, but on custom-built machines with standard OEM versions of Windows or your Linux distro downloaded from the web, things are different. There may be this shiny new sound card that just came out, which your dealer put in the box, but which neither operating system knows about, because there are no drivers. And gone is the dream of a talking installation! So, even when Windows 8 now allows Narrator to be turned on very early in the installation process in multiple languages even, and Orca can be activated early in a GNOME installation, this all is of no use if the sound card cannot be detected and the speech synthesizer canot output its data through conected speakers.

And VoiceOver had quite some features already when I tried it in OS X 10.5 Leopard: It had web support, e-mail was working, braille displays, too, the Calendar was one of the most accessible on any desktop computer I had ever seen, including Outlook’s calendar with the various screen readers on Windows, one of which I had even worked on myself in earlier years, and some third-party apps were working, too. In fact, my very first Twitter client ran on the Mac, and it was mainstream.

There was a bit of a learning curve, though. VoiceOver’s model of interacting with things is quite different from what one might be used to on Windows at times. Especially interacting with container items such as tables, text areas, a web page and other high-level elements can be confusing at first. If you are not familiar with VoiceOver, interacting means zooming into an element. A table suddenly gets rows and columns, a table row gets multiple cells, and each cell gets details of the contained text when interacting with each of these items consecutively.

In 2009, Apple advanced things even further when they published Snow Leopard (OS X 10.6). VoiceOver now had support for the trackpads of modern MacBooks, and when the Magic TrackPad came out later, it also just worked. The Item Chooser, VoiceOver’s equivalent of a list of links or headings, included more items to list by, and there was now support for so-called web spots, both user-defined and automatic. A feature VoiceOver calls Commanders allowed the assignment of commands to various types of keystrokes, gestures, and others. If you remember: Snow Leopard cost 29 us Dollars, and aside from a ton of new features in VoiceOver, it obviously brought all the great new features that Snow Leopard had in store for everyone. A common saying was: Other screen readers needed 3 versions for this many features and would have charged several hundred dollars of update fees. And it was a correct assessment!

In 2011, OS X 10.7 Lion came out, bringing a ton of features for international users. Voices well-known from iOS were also made available in desktop formats for over 40 languages, international braille tables were added, and it was no longer required to purchase international voices separately from vendors such as AssistiveWare.. This meant that users in more countries could just try out VoiceOver on any Mac in an Apple retail store or a reseller’s place. There were more features such as support for WAI-ARIA landmarks on the web, activities, which are either application or situation-specific sets of VoiceOver settings, and better support for the Calendar, which got a redesign in this update.

First signs of trouble

But this was also the time when first signs of problems came up. Some things just felt unfinished. For example: The international braille support included grade 2 for several languages, including my mother tongue German. German grade 2 has a thing where by default, nothing is capitalized. German capitalizes many more words than English, for example, and it was agreed a long time ago that only special abbreviations and expressions should be capitalized. Only in learning material, general orthographic capitalization rules should be used. In other screen readers, capitalization can be turned on or off for German and other language grade 2 (or even grade 1). Not so in VoiceOver for both OS X and iOS. One is forced to use capitalization. This makes reading quite awkward. And yes, makes, because this remains an issue in both products to this date. I even entered a bug into Apple’s bug tracker for this, but it was shelved at some point without me being notified.

Some other problems with braille also started to surface. For some inexplicable reason, I often have to press routing buttons twice until the cursor appears at the spot I want it to when editing documents. While you can edit braille verbosity where you can define what pieces of information are being shown for a given control type, you cannot edit what gets displayed as the control type text. A “closed disclosure triangle” always gets shown as such, same as an opened one. On a 14 cell display, this takes two full-length displays, on a 40 cell one, it wastes most of the real estate and barely leaves room for other things.

Other problems also gave a feeling of unfinished business. The WAI-ARIA landmark announcement, working so well on iOS, was very cumbersome to listen to on OS X. The Vocalizer voices used for international versions had a chipmunk effect that was never corrected and, while funny at first, turned out to be very annoying in day-to-day use.

OK, the enthusiastic Mac fan boy that I was, thought, let’s report these issues and also wait for the updates to trickle in. None of the 10.7 updates really fixed the issues I was having.

Then a year later, Mountain Lion, AKA OS X 10.8, came out, bringing a few more features, but compared to the versions before, much much less. Granted, it was only a year between these two releases, whereas the two cycles before had been two years each, but the features that did come in weren’t too exciting. There was a bit polish here and there with drag and drop, one could now sort the columns of a table, and press and hold buttons, and a few little things more. Safari learned a lot new HTML5 and more WAI-ARIA and was less busy, but that was about it. Oh yes and one could now access all items in the upper right corner of the screen. But again, not many of the previously reported problems were solved, except for the chipmunk effect.

There were also signs of real problems. I have a Handy Tech Evolution braille display as a desktop braille display, and that had serious problems from one Mountain Lion update to the next, making it unusable with the software. It took two or three updates, distributed over four or five months, before that was solved, basically turning the display into a useless piece of space-waster.

And so it went on

And 10.9 AKA Mavericks again only brought a bit polish, but also introduced some serious new bugs. My Handy Tech BrailleStar 40, a laptop braille display, is no longer working at all. It simply isn’t being recognized when plugged into the USB port. Handy Tech are aware of the problem, so I read, but since Apple is in control of the Mac braille display drivers, who knows when a fix will come, if at all in a 10.9 update. And again, old bugs have not been fixed. And new ones have been introduced, too.

Mail, for example, is quite cumbersome in conversation view now. While 10.7 and 10.8 very at least consistent in displaying multiple items in a table-like structure, 10.9 simply puts the whole mail in as an embedded character you have to interact with to get at the details. It also never keeps its place, always jumping to the left-most item, the newest message in the thread.

The Calendar has taken quite a turn for the worse, being much more cumbersome to use than in previous versions. The Calendar UI seems to be a subject of constant change anyway, according to comments from sighted people, and although it is technically accessible, it is no longer really usable, because there are so many layers and sometimes unpredictable focus jumps and interaction oddities.

However, having said that, an accessible calendar is one thing I am truly going to miss when I switch back to Windows. I know various screen readers take stabs at making the Outlook calendar accessible, and it gets broken very frequently, too. At least the one on OS X is accessible. I will primarily be doing calendaring from my iOS devices in the future. There, I have full control over things in a hassle-free manner.

iBooks, a new addition to the product, is a total accessibility disaster with almost all buttons unlabeled, and the interface being slow as anything. Even the update issued shortly after the initial Mavericks release didn’t solve any of those problems, and neither did the 10.9.1 update that came out a few days before Christmas 2013.

From what I hear, Activities seem to be pretty broken in this release, too. I don’t use them myself, but heard that a friend’s activities all stopped working, triggers didn’t fire, and even setting them up fresh didn’t help.

Here comes the meat

And here is the first of my reasons why I am switching back to Windows: All of the above simply added up to a point where I lost confidence in Apple still being dedicated to VoiceOver on the Mac as they were a few years ago. Old bugs aren’t being fixed, new ones introduced and, despite the beta testers, which I was one of, reporting them, were often not addressed (like the Mail and Calendar problems, or iBooks). Oh yes, Pages, after four years, finally became more accessible recently, Keynote can now run presentations with VoiceOver, but these points still don’t negate the fact that VoiceOver itself is not receiving the attention any more that it would need to as an integrated part of the operating system.

The next point is one that has already been debated quite passionately on various forums and blogs in the past: VoiceOver is much less efficient when browsing the web than screen readers on Windows are. Going from element to element is not really snappy, jumping to headings or form fields often has a delay, depending on the size and complexity of a page, and the way Apple chose to design their modes requires too much thinking on the user’s part. There is single letter quick navigation, but you have to turn on quick navigation with the cursor keys first, and enable the one letter quick navigation separately once in the VoiceOver utility. When cursor key quick navigation is on, you only navigate via left and right arrow keys sequentially, not top to bottom as web content, which is still document-based for the most part, would suggest. The last used quick navigation key also influences the item chooser menu. So if I moved to a form field last via quick navigation, but then want to choose a link from the item chooser, the item chooser opens to the form fields first. I have to left arrow to get to the links. Same with headings. For me, that is a real slow-down.

Also, VoiceOver is not good at keeping its place within a web page. As with all elements, once interaction stops, then starts again, VoiceOver starts interaction at the very first element. Conversations in Adium or Skype, and even the Messages app supplied by Apple, all suffer from this. One cannot jump into and out of the HTML area without losing one’s place. Virtual cursors on Windows in various screen readers are very good at remembering the spot they were at when focus left the area. And even Apple’s VoiceOver keystroke to jump to related elements, which is supposed to jump between the input and HTML area in such conversation windows, is a subject of constant breakage, re-fixing, and other unpredictability. It does not even work right in Apple’s own Messages app in most cases.

Over-all, there are lots of other little things when browsing the web which add up to make me feel I am much less productive when browsing the web on a Mac than I am on Windows.

Next is VoiceOver’s paradigm of having to interact with many elements. One item where this also comes into play is text. If I want to read something in detail, be it on the web, a file name, or basically anything, I have to interact with the element, or elements, before I get to the text level, read word by word or character by character, and then stop interaction as many times as I started it to get back to where I was before wanting to read in detail. Oh yes, there are commands to read and spell by character, word, and sentence, but because VoiceOver uses the Control+Option keys as its modifiers, and the letters for those actions are all located on the left-hand side of the keyboard, it means I have to take my right hand off its usual position to press these keys while the left hand holds the Control and Option keys. MacBooks as well as the Apple Wireless Keyboard don’t have Control and Option keys on both sides, and my hand cannot be bent in a fashion that I can grab these keys all with one hand. Turning on and off the VoiceOver key lock mechanism for this would add even more cumbersome to the situation.

And this paradigm of interaction is also applied to the exploration of screen content by TrackPad. You have to interact or stop interacting with items constantly to get a feel for the whole screen. And even then, I often feel I never get a complete picture. Unlike on iOS, where I always have a full view of a screen. Granted, a desktop screen displays far more information than could possibly fit on a TrackPad without being useless millimeter touch targets, but still the hassle of interaction led to me not using the TrackPad at all except for some very seldom specific use cases. We’re talking about a handful instances per year.

Next problem I am seeing quite often is the interaction braille gives me. In some cases, the output is just a dump of what speech is telling me. In other cases, it is a semi-spacial representation of the screen content. In yet another instance, it may be a label with some chopped off text to the right, or to the left, with the cursor not always in predictable positions. I already mentioned the useless grade 2 in German, and the fact that I often have to press the routing button at least twice before the cursor gets where I want it to go. The braille implementation in VoiceOver gives a very inconsistent impression, and feels unfinished, or done by someone who is not a braille reader and doesn’t really know the braille reader’s needs.

Next problem: Word processing. Oh yes, Pages can do tables in documents now, and other stuff also became more accessible, but again because of the paradigms VoiceOver uses, getting actual work done is far more cumbersome than on Windows. One has, for example, to remember to decouple the VoiceOver cursor from the actual system focus and leave that inside the document area when one wants to execute something on a tool bar. Otherwise, focus shifts, too, and a selection one may have made gets removed, rendering the whole endeavor pointless. Oh yes, and one has to turn the coupling back on later, or habits will result in unpredictable results because the system focus didn’t move where one would have expected it to. And again, VoiceOver’s horizontally centered navigation paradigm. Pages of a document in either Pages or Nisus Writer Pro appear side by side, when they are visually probably appearing below one another. Each page is its own container element. All of this leaves me with the impression that I don’t have as much control over my word processing as I have in MS Word or even the current snapshot builds of OpenOffice or LibreOffice on Windows. I also get much more information that I don’t have to look for explicitly, for example the number of the current page. NVDA, but probably others, too, have multilingual document support in Word. I  immediately hear which spell checking is being used in a particular paragraph or even sentence.

There are some more issues which were not addressed to this day. There is no PDF reader I know of on OS X that can deal with tagged (accessible) PDFs. Even when tags are present, Preview doesn’t do anything with them, giving the more or less accurate text extraction that one gets from untagged PDFs. As a result, there is no heading navigation, no table semantic output, and more that accessible PDFs support.

And the fact that there is no accessible Flash plug-in for web browsers on OS X also has caused me to switch to a Windows VM quite often just to be able to view videos embedded in blogs or articles. Oh yeah, HTML5 video is slowly coming into more sites, but the reality is that Flash is probably still going to be there for a couple of years. This is not Apple’s fault, the blame here is solely to be put on Adobe for not providing an accessible Flash plug-in, but it is one more thing that adds to me not being as productive on a Mac as I want to be on a desktop computer.

Conclusion

In summary: By all of the above, I do not mean to say that Apple did a bad job with VoiceOver to begin with. On the contrary: Especially with iOS, they have done an incredibly good job for accessibility in the past few years. And the fact that you can nowadays buy a Mac and install and configure it fully in your language is highly commendable, too! I will definitely miss the ability to configure my system alone, without sighted assistance, should I need to reinstall Windows. As I said above, that is still not fully possible without assistance. It is just the adding up of things that I found over the years that caused me to realize that some of the design decisions Apple has made for OS X, bugs that were not addressed or things get broken and not fixed, and the fact that apps are either accessible or they aren’t, and there’s hardly any in-between, are not compatible with my way of working with a desktop computer or laptop in the longer term. For iOS, I have a feeling Apple are still full-steam ahead with accessibility, introducing great new features with each release, and hopefully also fixing braille problems as discussed by Jonathan Mosen in this great blog post. For OS X, I am no longer as convinced their heart is in it. As I have a feeling OS X itself may become a second-class citizen behind iOS soon, but that, again, is only my personal opinion.

So there you have it. This is why I am going to be using a Lenovo notebook with NVDA as my primary screen reader for my private use from now on. I will still be using a Mac for work of course, but for my personal use, the Mac is being replaced. I want to be fast, effective, productive, and be sure my assistive technology doesn’t suddenly stop working with my braille display or be susceptible to business decisions placing less emphasis on it. Screen readers on Windows are being made by independent companies or organizations with their own business models. And there is choice. If one does no longer fit particular needs, another will most likely do the trick. I do not have that on OS X.

Posted in Apple | Tagged , , , , | 45 Comments

WAI-ARIA showcase: Microsoft Office web apps

Prompted by the recent Microsoft and GW Micro partnership announcement, I took a long overdue look at Microsoft’s Office 365 product offerings. The Home Premium edition not only gives you five installations of full Office Professional versions in your household, Windows and Mac combined, but also the apps for iOS and Android on up to five mobile devices, extra SkyDrive cloud storage space, and access to the Office in the browser offerings. Considering the cost of shelf Office products, the subscription prices are an amazing end user benefit!

Quick test: Office on the desktop and in mobile apps

I first tested the current desktop versions. Office 2013 for Windows is, of course, largely accessible to NVDA, JAWS, Window-Eyes and others. I heard that there seem to be some problems with Outlook, but I didn’t test that.

The Mac version of Office is, although superficially seeming accessible, not usable because in Word 2011 for Mac, for example, you don’t see the document content with VoiceOver.

The iOS version of Office Mobile has some major accessibility issues as well, primarily because of flaky document content and appearing and disappearing toolbars.

Firefox for Windows

I was hesitant to look at the Office in the Browser offerings at first, given the dismal situation of both Google Docs and Apple iWork in the browser. But boy, was I surprised! I brought up SkyDrive and created a new Word document from within Firefox on Windows and using NVDA.

The first thing I heard was the fact that I landed in the editing area, and that I could leave this by pressing control+f6. And if I wanted more help on the accessibility, I should press Alt+Shift+a. The latter brings you to this help page, which contains a lot of good information.

When I started typing into Word, I immediately noticed that my braille display was following along nicely. This can not be said for Google Docs which uses pure live region magic, which is completely speech-only, to convey information. I then also found out that NVDA was reporting to me font information. So when I bolded text and then asked NVDA to report to me the font and color, I heard the formatting information. Also when I formatted one paragraph as a heading, I was told the different font info.

I then pressed the above mentioned Control+F6 and cycled to the status bar. I saw the word count, language my document was in, and other info. I pressed the keystroke again and was taken to the Ribbon controls. NVDA immediately picked up all the info MS provides in their web app, that this is a ribbon toolbar, that I am on this and that tab of so many tabs, etc. The only means to navigate currently is by using the tab and shift+tab keystrokes, and Space or Enter to activate buttons. You first tab through all tabs, the Share button and the Minimize Ribbons button. Focus then moves into the ribons of the actually selected tab. While tabbing through, I noticed that all tabs were announced as selected. This seems to be a small bug still, in that the aria-selected=”true” should only be present on one of the tabs, meaning the tab whose content is shown below. All others should have aria-selected=”false”. Also, MS could optimize the keystrokes by allowing left and right arrows between the tabs, and the adjacent buttons which are always visible, and let the Tab key move straight into the active ribbon, like is done in the desktop version of Word.

Speaking of the ribbons: NVDA speaks the grouping information when passing through each ribbon. So you always hear when transitioning between different sub groups inside the ribbon. This helps immensely when you want to find the right group quickly.

Another press of Control+F6 brought me back to the document, and I could continue editing right away. Many of the shortcuts familiar from the desktop version of Word also work here, for example Control+B to bold text.

A slightly technical note: MS always feed the current paragraph only to the screen reader. This guarantees quite good performance. The fact that they’re doing this with all formatting intact means that they are using something more powerful than a simple textarea element. It is pretty amazing!

And all this time, I did not have to switch modes with NVDA. MS were mindful of the application role and used it wisely while developing this app. They provide all keyboard access to all controls, and since the document is in editing mode, there is also no problem reading the document content.

As described in the help document linked above, the best way to view your document is by going to Reading Mode from the View ribbon, clicking the “Accessible reading mode” link, and viewing the generated accessible PDF in Adobe Reader. Yup, that’s right, MS create a tagged PDF right out of the Word Web App for you! Note that if you’re using Firefox, you’ll probably first get a web view with pdf.js presenting the PDF. pdf.js does not yet use Accessibility tags, so the best is to click Tools, Download, and then save or open the PDF in Adobe Reader. This is the Tools item in the virtual buffer, not the Firefox menu item.

After I finished some testing with NVDA, I went back and did the same with JAWS and Window-Eyes. For both screen readers, it is recommended that you follow the hint given in the MS Office web app help document to turn off virtual mode. Both screen readers have some difficulty handling role “application” correctly, so they need to be forced into non-virtual mode for everything to work correctly.

The Window-Eyes experience was rather dismal despite of this, though. It barely picked up changes in the text content, navigation was slow and unpleasant. It spoke the ribbons OK, but not as fully fledged as NVDA did. Most importantly, it didn’t pick up changing group info.

JAWS did very well with Office web apps and Firefox. It even picked up the paragraph format used each time the paragraphs changed. Nice touch! It did, however, capture the Ctrl+F6 keys, so it would not allow Word to process them correctly. Navigation between the document and other elements was therefore quite cumbersome, since one needed to tab from the browser controls back to the document and into the ribbons. Since Control+F6 is no keystroke in Firefox, it is probably some funky scripting that is intercepting this keystroke and doing what would otherwise be done with F6 alone. I consider this a pretty annoying bug on Freedom Scientific’s part. JAWS also spoke the ribbon groups in a flattened manner, leaving out group transitioning mostly. Formatting information in the document was picked up just like by NVDA.

Internet Explorer

After I finished tests with Firefox, with those overwhelmingly pleasant experiences with NVDA in particular, I also went to see how Microsoft’s own browser and screen readers which, with the exception of NVDA, focus primarily on IE, would fare.

NVDA worked, but was very slow when navigating the document. Again, it handled role “application” best without the need to switch any modes. Formatting info was picked up.

JAWS worked OK once I turned off virtual mode. Here, it also didn’t capture Control+F6. Its speaking of the ribbons was rather flat, though, not picking up changes in ribbon groups. It also picked up formatting information.

Window-Eyes, again, left the most flaky impression, with document content not always being picked up, navigation being slow, and focus often being stuck or lost.

With the exception of the NVDA sluggishness I saw, which is probablz something that can be fixed in a timely manner, I would compare the results as almost equal between Firefox and IE, with a slight edge for Firefox.

Safari on OS X

After I completed my testing on Windows, I looked at OS X, where the most popular combination is Safari and VoiceOver. The result was rather disappointing. Both latest Safari on Mountain Lion and Mavericks only pick up the very first paragraph, and if you type something new. Everything else simply isn’t spoken when navigating through the document with up and down arrow. The ribbons are spoken rather flat, again, with grouping information not being picked up inside the individual ribbons. If you are looking to edit Word documents on Mac, I recommend you use Pages, Nisus Writer Pro or such. Especially the new Pages on Mavericks is much better in terms of accessibility than it was previously.

On mobile devices

In Firefox for Android, MS doesn’t render a full Word UI. Instead, one gets only the loosely formatted document and a page number indicator. Trying to request the desktop site from the menu brings you to the general Office.com web site instead. The document can be read, and the login process into your Microsoft Account works as expected.

QuickOffice, available on the Play Store, seems to be pretty accessible, from a quick test with my simple document opened from the MS SkyDrive app.

On an iPad using Safari and VoiceOver, you do get a full UI, and tabs and buttons, combo boxes etc., in the ribbons are spoken when touching or swiping, but grouping information is once again not available. Also, it is not possible to get into a consistent editing mode to work on a document. It is possible in theory, and outside of VoiceOver usage, may even work just fine, but once VoiceOver comes into play, support is not really available. Either the keyboard doesn’t come up, or if it does, the cursor cannot be tracked. Also, the shortcuts like Control+F6 don’t work with an attached keyboard.

If you want to use an iPad to do office document processing, I suggest you grab Pages, Numbers, and Keynote from the App Store and edit documents there. The new versions of these apps are amazing in terms of accessibility, in some points even surpassing their Mac counterparts.

Conclusion

Microsoft did an awesome job with the Office web apps. I only tried Word in this test, but according to the documentation, there is also support for PowerPoint, and at least some for Excel. The fact that Firefox and NVDA work so seamlessly with this web app shows that Microsoft did the coding right, and that their implementation of the WAI-ARIA standard is correct. I was particularly pleased that braille is also working. While it may not be important in some areas of the world, braille isn’t dead, and the fact that this works is very important to many users.

This is an excellent mainstream example of a well-implemented web app using WAI-ARIA! It should be an incentive to Google and Apple to also implement proper support into Docs and iWork respectively. While Docs has some live region magic, this leaves out braille users completely, and it doesn’t transport formatting information. I can edit Google Docs documents, yes, but I have no control over the formatting, unless I go into the tool bars, tab through to the various combo boxes and buttons and slowly gather the needed information that way. ChromeVox may or may not be able to gather some more information from Chrome than other screen readers do in all other browsers, but ChromeVox isn’t the only solution people are using to access documents, and the solution implemented by Google should be universal, not singling out any one browser/AT combo.

And Apple’s iWork in the browser isn’t accessible. Nuff said.

It is awesome to see how Microsoft has come along in accepting and implementing web standards. Office Web apps is one great example, and as someone who has worked on improving the WAI-ARIA support for Firefox and helped flesh out various user scenarios by providing testing, blog posts and such for years, it makes me extremely proud to see that Firefox and NVDA are the best to work with is at the time of this writing!

Posted in Accessibility, Apple, ARIA, Firefox, Mobile, NVDA, Testing | Tagged , , , , , , , , | 7 Comments

What got me into programming

On Nov 20, 1983, Borland released Turbo Pascal 1.0. At a price of revolutionary 50 us$, developers received the world’s first fully integrated development environment (IDE) for the PC. It combined the compiler and code editor, and later also debugger, in a package that ran as a single program and didn’t require developers to go in and out of different applications under MS-DOS constantly, immensely increasing productivity.

We take integrated development environments for granted today, but it all had to start from somewhere.

And my personal reason to celebrate this birthday is this: Turbo Pascal was what got me into programming in 1988. I received Turbo Pascal 3.0 for CP/M and was able to run it on my Commodore 128D. I learned the basic programming concepts from developing a program together with one of the social workers at my boarding school. The rest was learned by studying an old programming handbook for Basic, and translating the knowledge to Turbo Pascal. My father then read the programming reference to me, I made a note of each operator, reserved word and available procedure/function in braille, and started developing programs myself from there.

I also got hold of the source of a scientific calculator, which taught me concepts such as reference parameters.

When i got my first MS-DOS PC in 1991 together with a Dolphin 1 synthesiser and the screen reader HAL 4.0, I also got Turbo pascal 5.0, later 6.0, and tweaked HAL to work with it as best as possible. That version accompanied me way into my computer science studies. I think the last DOS program I wrote was in 1998 or so.

In parallel, beginning in 1995, I also started experimenting with Windows software development, when I got my first version of JAWS for Windows. The programming environment was called Delphi 1.0, which was the Windows equivalent of Pascal (then called Object Pascal) by Borland. I tweaked JAWS to work better with the Delphi IDE, and shared my experiences on a Henter-Joyce CompuServe forum board. That, and some other work I did early on with JAWS scripting, eventually got me the job at the German company Omni PC, which later was integrated into Freedom Scientific in 2001.

I stuck with Delphi for years, doing all my personal programming work there. Oh I also ventured into Visual Basic and later C# and .Net with Visual Studio 2003, but I always returned to Delphi for actual real productive Windows work.

I even beta tested Delphi 2007, codenamed Spaceley, in late 2006 and early 2007, when I was sort of in-between jobs and recovering from burnout syndrome. During that time, and you can call it my last project with Delphi, I integrated MSAA support into a popular Delphi component called VirtualTreeView. The last version of Delphi I ever used was RAD Studio 2007. I still have a DVD of that sitting on a shelf somewhere, but haven’t used it in years. my current work simply takes place in a completely different realm.

But even today, I still occasionally receive questions about current Delphi versions, based on the scripts I wrote and contributed to JAWS in the mid 2000s. And sometimes I am very sorry to have to tell people that I cannot help them any further nowadays, since I totally am out of touch with this part of the software world. Yes, Delphi is still around, now under the Embarcadero umbrella, and includes a lot of different platforms besides Windows.

So today, it was time to celebrate a little. I raise a glass to all the great folks who work and worked on Turbo Pascal and Delphi. Thank you, without you, I might not have started programming at all!

Program Celebrate;

begin
  writeln('Cheers!');
end.

Posted in General | Tagged , , , , , , , | 4 Comments

Why accessibility APIs matter

This morning, Victor from payPal and I got into an exchange on Twitter regarding the ChromeVox extension. ChromeVox is a Chrome extension which provides screen reading functionality for blind users. Through keyboard commands, the user can navigate page content by different levels like object by object, heading by heading, form control by form control, etc.

Wait, you might say, but this is screen reader functionality I know from NVDA or JAWS on Windows, or VoiceOver on the Mac! And you’re absolutely right! The difference is that ChromeVox is working in the web content area of Chrome only, not in the menus or tool bars.

That in itself is not a bad thing at all. The problem comes with the way ChromeVox does what it does. But in order to understand that, we need to dive into a bit of history first.

Back in the late 1990s, when the web became more important every day, and Windows was the dominant operating system. With the big browser war coming to an end with Microsoft’s Internet Explorer as the great winner, it was apparent that, in order to make the web accessible to blind users and those with other disabilities, assistive technologies had to deal with IE first and foremost. Other browsers like Netscape quickly lost relevance at that time in this regard, and Firefox wasn’t even thought about yet.

Microsoft took sort of a mixed approach. They were bound by their own first implementation of an accessibility API, called Microsoft Active Accessibility, or MSAA. MSAA had some severe limitations in that, while it knew very well how to make a button or checkbox accessible, it had no concept of document-level accessibility or concepts such as headings, paragraphs, hyperlinks etc. Microsoft managed to evolve MSAA bit by bit over time, giving it more role mappings (meaning different types of content had more vocabulary in saying what they were), but there were still severe limitations when it came to dealing with actual text and attributes of that text.

But screen readers wanted more than just knowing paragraphs, links, form fields and buttons. So the Windows assistive technologies were forced to use a combined approach of using some information provided by MSAA, and other information provided by the browser’s document object model (DOM). There were APIs to get to that DOM in IE since version 5.0, and for the most part, the way rich internet information is accessed in IE by screen readers has not changed since 1999. Screen readers still have to scrape at the DOM to get to all the information needed to make web content accessible in IE.

In 2001, Aaron Leventhal of Netscape, later IBM, started working on a first implementation of accessibility in the Mozilla project, which later became Firefox. To make it easier for assistive technology vendors to come aboard and support Mozilla in addition to Firefox, a decision was made to mimic what IE was exposing. That part of interface is with Firefox until today, and being used by some Windows screen readers, although we nowadays strongly evangelize for use of a proper API and hope to deprecate what we call the ISimpleDOM interfaces in the future.

In 2004/2005, accessibility people at Sun Microsystems and the GNOME foundation, as well as other parties, became interested in making Firefox accessible on the Linux desktop. However, this platform had no concept similar to the interfaces used on Windows, and a whole new and enhanced set of APIs had to be invented to satisfy the needs of the Linux accessibility projects. Other software packages like much of the GNOME desktop itself, and OpenOffice, also adopted these APIs and became accessible. While some basic concepts are still based on the foundation laid by MSAA, the APIs on Linux quickly surpassed these basics on a wide scale.

Around the same time, work began on the NVDA project, the first, and to date only, open-source screen reader on Windows. The NVDA project leaders were interested in making Firefox accessible, giving users a whole open-source combination of screen reader and browser. However, they were not planning on building screen-scraping technology into NVDA that was used by other Windows screen readers, but wanted API-level access to all content right from the start. Out of this requirement, an extension to MSAA, called IAccessible2, was born, which enhanced MSAA with stuff already present in the Linux accessibility APIs. As a consequence, they are very similar in capability and nomenclature.

In parallel to that, Apple had been developing their own set of APIs to make OS X accessible to people with visual impairments. Universal Access and the NSAccessibility protocol are the result, accessibility from an API level that also does not require the screen reader to scrape video display content to get to all the information. This protocol is in many ways very different in its details, but offers roughly the same capabilities.

Within Firefox, this meant that gaps that were previously only pluggable by using the browser DOM directly, needed to be closed with proper API mappings. Over time, these became very rich and powerful. There is a platform-independent layer with all capabilities, and platform-specific wrappers on top which abstract and slightly modify (on occasion) the exposed information to make it suitable for each platform. Both Firefox for Android and Firefox OS JavaScript bridges to Talkback and a speech synthesizer respectively, use the platform-independent layer to access all information. Whenever we find the JavaScript code needs to access information from the DOM directly, we halt and plug the hole in the platform-independent APIs instead, since there will no doubt be a situation where NVDA or Orca could also run into that gap.

So to re-cap: In IE, much information is gathered by looking at the browser DOM directly, even by NVDA because there is no other way. In Firefox, some of the more legacy screen readers on Windows also use this technique provided by Firefox as a compatibility measure, but all newer implementations like NVDA use our IAccessible2 implementation and no DOM access to give users the web experience.

Safari on OS X uses the Apple NSAccessibility protocol obviously. It has since been discontinued on Windows, and never had much of an  MSAA support to speak of.

Google Chrome also exposes its information through Apple’s NSAccessibility protocol on OS X, and uses MSAA and IA2, at least to some degree, on Windows.

And what does ChromeVox use?

Here’s the big problem: ChromeVox uses DOM access exclusively to provide access to web content to blind users. It does, as far as I could tell, not use any of Chrome’s own accessibility APIs. On the contrary: The first thing ChromeVox does is set aria-hidden on the document node to make Chrome not expose the whole web site to VoiceOver or any other accessibility consumer on OS X or Windows. In essence, both Crome and ChromeVox perform their own analysis of the HTML and CSS of a page to make up content. And the problem is: They do not match. An example is the three popup buttons at the top of Facebook where the number of friend requests, messages, and notifications are displayed. While Crome exposes this information correctly to VoiceOver, ChromeVox only reads the button label if there is a count other than 0. Otherwise, the buttons sound like they are unlabeled.

In my half hour of testing, I found several pages where there were these inconsistencies between what Chrome exposes, and what chromeVox reads to users. An example quite to the contrary is the fact that Google Docs is only accessible if one uses Chrome and chromeVox. What Chrome exposes to VoiceOver or NVDA is not sufficient to gain the same level of access to Google Docs.

If you are a web developer, you can imagine what this means! Even if you go through the trouble of testing your web site with Chrome and Safari to make sure they expose their information to VoiceOver, it is not guaranteed that ChromeVox users will benefit, too. Likewise, if you use chromeVox exclusively, you have no certainty that the other APIs are able to cope with your content.

Web developers on Windows have also learned these lessons the hard way with different screen readers in different browsers: Because with IE, each is forced to do their own interpretation of HTML, at least on some level, results will undoubtedly differ.

There is a harmonization effort going on at the W3C level to make sure browsers interoperate on what they expose on each platform for different element types. However, if prominent testing tools like ChromeVox or some legacy screen readers on Windows hang on to using their own HTML interpretation methods even when there are APIs available to provide them with that information, this effort is made very very difficult and puts a big extra burden on those web developers who are making every effort to make their sites or web apps accessible.

When we started work at Mozilla to make both Firefox for Android and Firefox OS accessible, we made a conscious decision that these platforms needed to use the same API as the desktop variants. Why? Because we wanted to make absolutely sure that we deliver the same kind of information on any platform for any given HTML element or widget. Web developers can count on the fact that we at Mozilla will always ensure that if your stuff works in a desktop environment, it is highly likely that it will also speak properly on Firefox OS or through TalkBack in Firefox for Android. That is why our JS bridge does not use its own DOM access methods, so there are no interpretation gaps and API diversities.

And here’s my pledge to all who still use their own DOM scraping methods on whichever platform: Stop using them! If you have an API available by the browser, use that whenever possible! You will make your product less prone to changes or additions in the HTML spec and supported elements. As an example: Does anyone remember earlier versions of the most prominent screen reader on Windows which suddenly stopped exposing certain Facebook content because Facebook started using header and footer elements? The screen reader didn’t know about those and ignored everything contained within the opening and closing tags of those elements. It required users to buy an expensive upgrade to get a fix for that problem, if I remember correctly. NVDA and Orca users with Firefox, on the other hand, simply continued to enjoy full web access to Facebook when this change occurred, because Firefox accessibility already knew how to deal with the header and footer elements and told NVDA and Orca everything they needed to know.

On the other hand, if you are using DOM scraping because you find that something is missing in the APIs provided by the browser, work with the browser vendor! Identify the gaps! Take the proposals to close those gaps to the standards bodies at the W3C or the HTML accessibility task force! If you’re encountering that gap, it is very likely others will, too!

And last but not least: Help provide web developers with a safer environment to work in! Web apps on the desktop and mobile operating systems are becoming more and more important every day! Help to ensure people can provide accessible solutions that benefit all by not making their lives unnecessarily hard!

Posted in Accessibility, ARIA, Firefox, FirefoxOS, Mobile | Tagged , , , | 13 Comments

Twitter now has a dedicated accessibility team

Ever since I joined Twitter in March of 2008, at my first CSUN under the Mozilla banner, Twitter’s own web presence was always a bit, or even a lot, of a challenge to use for me as a screen reader user. While the initial version was still pretty straight-forward, as time went by and Twitter added more features and turned their web presence into a web app, the interaction became increasingly cumbersome. Fortunately, there are clients on various platforms that allowed me to access the service without having to rely on the web site. Even after the more strict API 1.1 roll-out a year ago, this situation hasn’t really changed for me.

A few months ago, I learned that Todd Kloots had joined Twitter. I knew Todd from when he was still at Yahoo! doing awesome things for web accessibility there. As time went on, members of the accessibility team at Mozilla, the NVDA developers, and Todd discussed various aspects of web accessibility, making it apparent that something good was happening at Twitter!

A few weeks ago, after a couple of tweets by Todd and others, I tried out the Twitter web app for the first time after a long while. And as was hinted in the tweets, the J and K keys suddenly started working much more nicely with the various browser/screen reader combinations I tried.

Yesterday, now, a post was published on the Twitter blog detailing some of the recent changes in accessibility of twitter.com, and an official accessibility team Twitter account was also announced. You can find the team at @a11yteam, and they welcome suggestions for further improvements.

If you are blind and want to try things out, you can do so right away by logging onto Twitter and reading your timeline. J and K will move you up and down through the timeline. Further keystrokes are available if you press the ? (question mark) key. Just remember, if you’re on Windows, you’ll have to turn off virtual cursor mode to be able to properly use them, as they will most likely be captured by the quick navigation key system of your screen reader otherwise. If you’re on the Mac, you do not need to do anything special.

I hope that Twitter will also pay equal attention to detail in their mobile web app, so the app you can get from the Mozilla Marketplace for both Firefox for Android and Firefox OS will be equally accessible. First signs of improvement are there, with tabs and buttons properly labeled nowadays. In addition, starting in Firefox 25, we have improved the way we execute touch events from within TalkBack and Firefox OS, so interaction should be just smooth!

This is exciting news, and I wish the Twitter accessibility team all the best in their future work on the web and native applications! As far as the web and mobile web presences are concerned, we at the Mozilla accessibility team will be here, ready to help wherever needed!

Posted in Accessibility | Tagged , , | 3 Comments

What’s New for TalkBack users in Firefox 25 for Android

Firefox 25 for Android, out as an official release end of October, and on the Aurora channel since August 9, will include a number of new features for visually impaired users. This continues a series of new features introduced in Firefox 24, which I blogged about earlier.

New reading order

As hinted in my blog post about Firefox 24 new features, we have now made a change to the default reading order of web elements. If you did not make any adjustments, you will get this new reading order as soon as you update to Firefox 25. Now, the current item’s label is spoken first, followed by its role and states, and any changed hierarchical information is now appended at the end. This allows for a much more efficient browsing experience in our opinion.

If you want to change it back, follow the steps given in the other blog post.

Support for landmarks

Firefox for Android now supports landmarks. It speaks the landmark that you just entered, and an option has been added to the three finger swipe up and down menu to move by landmark quick navigation. If you use a keyboard, the quick navigation key is d and Shift+d, similar as in the NVDA screen reader for Windows.

Support for data tables

Firefox for Android 25 supports data tables. If you enter a data table, column and row headers are spoken, if a cell spans multiple columns or rows, this is also announced, as well as the table dimensions and any caption and/or summary, too. When traversing a table element by element, only the necessary information that changed is read. So if you move from column 2 to column 3, but stay within the same row, the new column header is read, but the row header is not repeated. Likewise if you stay in column 3 and move to the next row, the new row header will be announced, but the column header will not, because it did not change.

Suppression of layout tables

Firefox has had a heuristic to determine if a table on a web page might be there solely for layout purposes. While not encountered very often nowadays, it still happens occasionally. Now if such a layout table is encountered, Firefox will suppress speaking of any table or table cell information as to not flood you with unnecessary information.

Speaking of additional information, if available

Sometimes, web authors provide additional information via the title attribute, an aria-describedby statement or some other means that is not the primary label or text for an element, but which, on the desktop, becomes visible when hovering the mouse over the element. This information is stored for screen readers in something we call the AccessibleDescription. Unlike the AccessibleName, which is the primary label or text, you can think of the AccessibleDescription as an addendum. Since there is no real concept of an AccessibleDescription in Android itself, we made a compromise and add this AccessibleDescription if it is there, and if it is different from the AccessibleName. Some web authors unnecessarily put the same text of a link or image in the title as they do in the enclosing text or alt text respectively. We detect that and suppress the redundant info. But if there is true additional information, it will now be appended to the name in the now default reading order. In the old reading order, it will be prepended, since it will honour the setting.

Support for switching movement granularities

It is now possible to traverse through text on a web page by character, word, or paragraph using the usual TalkBack gestures in Jelly Bean. Switch your granularity to one of these modes and swipe left and right to move, spell, or read word by word.

More braille features

In addition to the braille output introduced in Firefox 24, Firefox 25 also brings a cursor indicator, and you can move the cursor via braille routing keys. Also, if you type something into a text field on the web, braille will now update properly to show you what you typed.

Better support for web applications reacting to touch events

Mobile web applications such as Mobile Twitter will now work much better for TalkBack users. Before, it could happen that if you double-tapped nick names or tweets in your timeline, nothing would happen. Now, we correctly pass on touch events, which Mobile Twitter expects, so you can do things such as reply, retweet and other actions pertaining to the current tweet, view the user profile etc.

Better touch targets

Firefox for Android now allows for a better explore by touch experience by improving the discoverability of certain web elements such as plain text.

…and more

There have also been bits and pieces added, changed, and optimised to make reading web content more reliable, especially if there are frames and iframes involved.

I hope you’ll enjoy these new features! Happy browsing!

Posted in Accessibility, Firefox, Mobile | Tagged , , | 2 Comments