Archive for the ‘Apple’ Category

The current state of accessible Firefox on the Mac, your help is appreciated!

Tuesday, September 22nd, 2009

I’ve been asked time and time again about what the current state is of the possibility of an accessible Firefox on the Mac that interacts with the VoiceOver screen reader. Well, let me recap what the current affair is.

First of all, the bad news is that getting the remaining quirks out of Mac accessibility is not going to happen this or next quarters.

The main reason is that our code currently needs to be made ready for future enhancements. Our recent work on the IAccessibleTable2 interface showed us that painfully: The refactoring and reorganization needed took much longer than we had estimated and required us to touch much more code than we had hoped. We also have some nasty event firing problems that have been a pain in the neck for those ATs supporting us that we desperately need to attack these now.

However, the good news is that since early May, the accessibility unit tests (Mochitests) have been running mostly successfully on the Mac on both the mozilla-central and mozilla-1.9.2 branch tinderboxes. Those tests had been running successfully on Windows and Linux since December already. They’ve been intermittently showing some flakiness (what we call random orange) in especially the events area, which is what we’ll be working on in the immediate future anyway, but otherwise been quite stable.

On top of that, at least investigating and driving the Mac accessibility effort forward has been sort of a pet project of mine all the while, even though Mac a11y is no longer part of our team’s official goal set for this year.

What I’ve been doing is build Mac builds with accessibility enabled on my MacBook and test them whenever I had some cycles. On Mac OS X 10.5 (Leopard), VoiceOver and the Firefox builds played terribly together, with lots of sluggishness and delayed responses that made it impossible to get any really useful testing done.

This, however, has changed with Snow Leopard. Without consciously changing things on our end, a build done under Snow Leopard is much more responsive with VoiceOver. We’re nowhere near the good performance blind people have come to appreciate from Safari yet, but we’re doing much better now. Whether it’s the improved VoiceOver or the fact that we’re now building against the MacOS10.5 SDK, or both, I don’t know.

The following are what I’d consider the most pressing issues before we could even think about sending this off to a bunch of voluntary testers:

  • VoiceOver is not speaking most accessible roles of UI and web page elements. Strangely enough, when pressing a button, it will say “Press button” and the like.
  • VoiceOver skips over plain text on pages, and the text is also not visible in VoiceOver’s Obbject Chooser menu.
  • Other inconsistencies having to do with roles like VoiceOver not recognizing our HTML area as such and not automatically starting to read pages etc.
  • Performance problems: Right now, navigating from one element to the next takes about half a second, which is unacceptable.

And this is where you might be able to help us! Are you a developer who is fluent in Objective-C, and (as a bonus) versed in Apple’s accessibility APIs? We could use your voluntary help in trying to nail this down!

So if you’d like to help improve Firefox on the Mac and have an idea about what might be going on here, go grab the source and build it! In order for accessibility to be turned on, in addition to all the stuff you need to put into your MOZCONFIG file, you need to add the following line:

ac_add_options --enable-accessibility

With this exception, commence as described in our docs.

Once you’ve got Firefox built, you can press CMD+F5 to turn on VoiceOver. A friendly male voice named Alex will start talking to you immediately and give you feedback on your actions through your Mac’s speakers. CMD+F5 is a toggle, so the same keystroke will turn VoiceOver off and leave no trace of it having been active.

Once VoiceOver is running, the following is a list of hot keys that can be used to navigate. Note that VO refers to the VoiceOver modifier, which is both Ctrl and Option held down.

Key Description
VO+Arrows Navigate in all 4 directions. Left and Right from item to item sequentially.
VO+Shift+DownArrow Interact with the currently spoken item. Interacting is examining an item in more detail. For example, interacting with a table will expose the individual cells to VoiceOver and restrict navigation inside this container.
VO+Shift+UpArrow Stop interacting and return to the parent level of navigation.
VO+Space Press or activate the current item. This will perform the default action, which is usually identical to clicking the mouse.

More information is available on Apple’s accessibility site. A Key commands chart (PDF) will give you more keystrokes, and the linked-to bugs above will give exact steps to repro the bugs themselves.

Since Firefox is not a native COCOA application on the Mac in the sense that we implement all our UI elements ourselves, we have to expose the whole accessibility contract and are obviously doing some things wrong there. The initial work that was done was done by a Mozilla Foundation grantee back in 2006, but this project was never finished and the project had been in limbo since then.

If you are willing to help, feel free to connect with me through e-mail (available from my About page) or via IRC on the #accessibility channel. I’ll be glad to assist with getting involved. The accessibility team definitely would appreciate it, and so would many community members I’m sure!

Posted in Accessibility, Apple, Firefox | 18 Comments »

My first experience using an accessible touch screen device

Monday, June 22nd, 2009

Yes, you read correctly: An accessible touch screen device! This morning, I went to a retail store carrying mostly Apple products and had a look at the new iPhone 3G S that was released in Germany on Friday. Apple revealed during the WWDC keynote two weeks ago that it would have a built-in screen reader named the same as is included in Mac OS X: VoiceOver. This is a feature not available on the regular iPhone 3G, as its hardware capacity is insufficient.

I was not at all sure what to expect. From reading a bunch of posts on the VIPHone Google Group, I knew that people were going through a learning curve, a steep one at times. Up to now, something usable via a touch screen or touch-only keys would always mean a dead-end to me and other blind people. The iPhone 3G and the iPod Touch are not usable for me. Likewise, elevators that have keys you only need to touch, not press, to get toa different floor, are a real challenge. In fact I once tripped an alarm while trying to use such an elevator, alone int he cabin and touching the emergency button accidentally.

When I arrived at the store, I had already made arrangements with them to be allowed to take an in-depth look at the 3G S. As we went over to the iPhone stand, one of the sales assistants already knew how to turn on VoiceOver. Apple are documenting this in the regular iPhone user’s manual, no special docs needed. The assistant helping me turned it on, and a clear crisp voice came out of the built-in speakers. She was a bit confused by the changed gestures. I had done some reading, and took over from there.

And I must say this was an amazing experience! My fingers definitely need to get used to gestures such as flicking or tapping, or using a rotor. But having an iPod Nano 4th generation helped with that, since moving the finger over the screen like on a dialer felt most like tracking around the iPod’s click wheel. Even the sound the rotor makes is the same. :)

Responsiveness to gestures was amazing. I own an Nokia N82, which is to date probably the handset that reacts fastest to keyboard commands with the Talks or MobileSpeak screen readers, but the responsiveness on the iPhone beats that by lengths!

Finding my way around the iPhone’s UI took some getting used to. Traditional mobile screen readers, also like most Windows or Linux screen reader solutions, give the blind user a filtered view of the world, by default constrained to the focus location. Only on demand can one explore the screen using mouse emulation or similar techniques. On the iPhone, you interact with the real thing right from the start. You touch the screen in the lower half, somewhere on the right, and you’re told that the Safari or iPod symbol is there on the Home screen. You move your finger to the left, and you’re told what’s right next to it. To interact with the menu bar of the Phone app, you need to move your finger down to the bottom and move from left to right to hear the options such as “Contacts” or “Phone pad”. Yes, there are VoiceOver gestures to explore the screen top to bottom, left to right. You do this by flicking left to right anywhere, and the accessible controls are being walked one by one. But the interaction model is very close to the actual screen layout most of the time. This tremendously helped when I walked through a couple of applications with the sales assistant standing next to me. She could literally point me to the correct spot, and VoiceOver would speak what I needed to hear. Or she could give me verbal directions, and my finger would find the controls.

Typing is probably going to take the most adjusting. It is nothing like typing on the number pad of my N82. James Craig’s typing tip for VoiceOver on iPhone helps a lot: You look for the correct key with one hand, keep your finger there, then tap somewhere on the screen with another finger from the same or the other hand, and the character is input. Gladly, the keyboard doesn’t change position, and after a few letters I had a very good idea where each letter should be, and my typing sped up within 10 typed letters already. In addition, one can turn on word prediction/completion, which is another accessibility feature that can also aid people with motor impairments make typing easier. It plays nicely with VoiceOver.

This is by far not a comprehensive review or comparison. I couldn’t use many of the features since the SIM card in that exhibited model was locked, and I don’t have my own model yet.

Apple are speeding ahead and breaking down conventions in accessibility, or as Mike Calvo of Serotek wrote: They’re getting to the future first. They’re the first to include a screen reader for the blind on one of their mainstream models. Google are going to do something similar with their G1 efforts. The API is there, and some basic console work seems to be working already, but this is by far not as comprehensive as what Apple are doing. RIM also have an accessibility API, but from what I’m told, the screen reading solution that has been hinted every now and then over the past couple of weeks is going to cost extra money, which Apple’s solution does not. The traditional mobile accessibility solutions on Windows Mobile and Symbian S60 all require an additional payment of $200 to $350 for a screen reading solution, or in some cases even proprietary hardware that then costs $2000 or even more.

And this, of course, opens up other possibilities for future implementations of touch screen use cases, not just by Apple, but by other companies as well.

And one more bit of info: The gestures and touchy interface also come to VoiceOver in Snow Leopard with compatible new MacBooks with the multi-finger trackpad. So whenever a colleague tells me to lok for something in the top right quadrant of the screen, I can do that once I have Snow Leopard running on my MacBook. I’ll just put my finger there and let VoiceOver tell me what’s there!

Now my only problem is to get an iPhone. It would appear that my current contract doesn’t allow me to upgrade, since I upgraded it only recently, but too long before I knew the 3G S was coming. We’ll see how I get my hands on a device, it’s not freely available without contract in Germany.

My first touch screen experience was an amazing one!

Posted in Accessibility, Apple | 97 Comments »

Why I would not buy an iPod just yet

Wednesday, October 1st, 2008

Recently, Apple announced better accessibility features in iTunes 8 and the 4th generation iPod Nano. This is a major breakthrough in accessibility because now, the maker of a mainstream hardware media player is taking steps to make sure the device is useable also by people with vision impairments. Previously, the only way to make certain types of MP3 players accessible was through projects like Rockbox, which provides a custom firmware for these players. This approach is never fully complete, because for the supported iPods, it does not allow the very specific features to work like DRM-protected media playback.

On the other hand, there are new approaches taken by assistive technology vendors such as HumanWare who manufacture new, modern devices like the VictorReader Stream, which is both a DAISY (Digital accessible information system) and Audible audio book player, and also an MP3 and Ogg Vorbis player. The revolutionary element of this is that it comes with a speech synthesizer, bilingual if you so choose, to read the titles or meta information of your MP3 or Ogg files to you.

The question I was asking myself when I read Apple’s announcement was if this now obsoletes such hardware specifically designed for the blind. And the answer came quickly, too: We’re not quite there yet. And unless Apple and other hardware manufacturers of mainstream players are ready to include blindness specific, or seemingly blindness specific, features into their players, there will always be a place and necessity for these specialized hardware devices.

I also asked myself whether I would choose a Stream or an iPod if I wanted to buy a new portable media player today. I already own a Stream, bought it in March of this year, so am in no immediate need to purchase a new device. And after some thinking, I came to the conclusion that a Stream today is still the better choice for me and possibly other blind users. Here are the reasons why I think this is the case.

More flexibility in speech output

The Stream offers in its firmware bilingual voice synthesis output. One is always English, the other is the language of one’s choosing. Since I am German, in my case a German voice is also on board. This allows me to listen to the meta data or file names of both English and German books or media files without having to compromise on one end. Apple’s approach, to synthesize the menu items and file names on the host computer via the speech APIs is monolingual. In other words, if I choose to synthesize the samples in German, all my English language MP3 meta data will also be read to me with German pronounciation rules, sounding a bit awkward.

No purchase of a speech synthesizer required

The speech synthesis voices are on board when you purchase a Stream. With the iPod solution, you are required to purchase at least one additional Microsoft Speech API 5 voice from somewhere. Granted, users of the JAWS screen reader are in luck, since Freedom Scientific ships RealSpeak Solo voices in multiple languages with their JAWS 8 and 9 releases, which can then be used to synthesize the menu prompts and file data. But others are required to purchase synthesis engines for varying prices to get the same functionality. And to be honest, I wouldn’t want my iPod to speak to me in the Microsoft Sam voice. If you’re using XP, open Control Panel, and there the Speech input/output panel, and let the default voice speak a sample to you. You’ll know what I mean… :-) And the voice selection on Vista is only slightly improved over that. And they’re all English only.

Support for DAISY audio books

The Apple iPod does not support the playback and navigation options that a modern dedicated DAISY player offers. DAISY books can be listened to in MP3 format, but no navigation among headings, phrases or the like is possible without the additional meta data present in DAISY books. Since one paradigm I live by is to always have the greatest flexibility when getting my content, I would not want to limit myself by excluding the possibility to read DAISY content.

Since more and more mainstream download shops also offer DAISY audio books now, bringing this format out of the niche of blindness-specific libraries, it is my hope that mainstream music players will soon adopt this format, too, and offer a wider choice in this regard.

Support for the Flac and Ogg Vorbis formats

OK, this is a strictly personal opinion: I prefer to rip my CD collection in Ogg Vorbis format, since I feel it is substantially superior to MP3. The iPod does not support Ogg Vorbis even in the latest of their devices.

Support for various text file formats

The Stream can read various types of text files to me: Plain text, HTML and XML, RTF, and even some types of braille formatted files, using its on-board speech synthesis engine.

Notetaking feature

The Stream allows me to take notes using either an external or the built-in microphone. Depending on the memory card that’s inserted, I can even record whole presentations with this. Granted, this, like the previous one, is a very specific feature, but an essential one for blind users. Quickly being able to record something without having to fiddle with the notetaking facility of one’s mobile phone is even more efficient than grabbing pen and paper to take a note.

More flexibility in memory usage

The Stream operates using standard SD or SDHC memory cards you can buy in any store, giving it virtually unlimited memory capacity. The Apple iPods have their Flash memory built in, restricting one to 8 or 16 gigabytes (in the case of the 4th generation Nano). In addition, because these speech prompts get copied to the iPod to make it accessible, using up memory as well, which counts against your available music storage space.

Freedom of choice when purchasing digital content

With the iPod, one is bound to Apple’s iTunes software and music store. Even though there is also DRM-free content available, the vaast majority of content is protected, and encoded in the not too impressive 128 kbit/s Aac format. For the record: Other music stores use protected 128 kbit/s Windows Media Audio (WMA) format, which I find equally unimpressive. If I purchase music for download, I usually go for DRM-free, high quality MP3 format. If I don’t find it, I go to Amazon and buy the CD to rip it in my favorite Ogg format. Call me old-fashioned, but I’m a guy who prefers to purchase whole albums rather than individual tracks. Feels incomplete somehow.

The Stream offers me that flexibility, the iPod limits me in that it won’t accept my Ogg files and requires me to always use Apple’s music store, or use iTunes to copy files across. Using the Stream, I can simply insert the SD card into my reader and copy the files over using my preferred file management tool, reinsert the card into the Stream, and off I go.

Do you have no arguments for the iPod in store?

The only thing I can currently think of is the price. The Stream costs about twice as much as the more expensive iPod Nano 4G with 16 gigabytes of memory.

One other argument some might think of is the trendiness of the device. For a blind person however, other than the feeling that one might look trendy, that should never be a criteria to choose a device. Granted, the Stream looks a bit–and I cite a friend’s 8 year old son–like a mobile phone of the mid 90’s, with its speaker and mike and the rather big keys and the edged case. The iPod for some might indeed have a much nicer form factor.

Looking at other features the iPod offers, I can only say that viewing fotos is sort of beside the mark, and being able to watch videos is only interesting if one wants to buy or rent TV shows or movies from the store. If you record from TV or have DVDs, there are always ways to extract the audio only, which is enough for a blind person.

Having said all the above, I’m not saying that Apple’s move was a bad one or that this is an invain endeavor. I can only repeat that I find it fantastic that Apple has taken these steps, and can only encourage them to continue this venture, and encourage others to follow this example. The more choice everyone has, the better.

And if someone from Apple reads this and thinks of making me a Christmas present, I’d be very grateful for the gesture, and write a positive review on my blog. :-)

Tags: , , , ,
Posted in Accessibility, Apple | 12 Comments »