My first experience using an accessible touch screen device

Yes, you read correctly: An accessible touch screen device! This morning, I went to a retail store carrying mostly Apple products and had a look at the new iPhone 3G S that was released in Germany on Friday. Apple revealed during the WWDC keynote two weeks ago that it would have a built-in screen reader named the same as is included in Mac OS X: VoiceOver. This is a feature not available on the regular iPhone 3G, as its hardware capacity is insufficient.

I was not at all sure what to expect. From reading a bunch of posts on the VIPHone Google Group, I knew that people were going through a learning curve, a steep one at times. Up to now, something usable via a touch screen or touch-only keys would always mean a dead-end to me and other blind people. The iPhone 3G and the iPod Touch are not usable for me. Likewise, elevators that have keys you only need to touch, not press, to get toa different floor, are a real challenge. In fact I once tripped an alarm while trying to use such an elevator, alone int he cabin and touching the emergency button accidentally.

When I arrived at the store, I had already made arrangements with them to be allowed to take an in-depth look at the 3G S. As we went over to the iPhone stand, one of the sales assistants already knew how to turn on VoiceOver. Apple are documenting this in the regular iPhone user’s manual, no special docs needed. The assistant helping me turned it on, and a clear crisp voice came out of the built-in speakers. She was a bit confused by the changed gestures. I had done some reading, and took over from there.

And I must say this was an amazing experience! My fingers definitely need to get used to gestures such as flicking or tapping, or using a rotor. But having an iPod Nano 4th generation helped with that, since moving the finger over the screen like on a dialer felt most like tracking around the iPod’s click wheel. Even the sound the rotor makes is the same. :)

Responsiveness to gestures was amazing. I own an Nokia N82, which is to date probably the handset that reacts fastest to keyboard commands with the Talks or MobileSpeak screen readers, but the responsiveness on the iPhone beats that by lengths!

Finding my way around the iPhone’s UI took some getting used to. Traditional mobile screen readers, also like most Windows or Linux screen reader solutions, give the blind user a filtered view of the world, by default constrained to the focus location. Only on demand can one explore the screen using mouse emulation or similar techniques. On the iPhone, you interact with the real thing right from the start. You touch the screen in the lower half, somewhere on the right, and you’re told that the Safari or iPod symbol is there on the Home screen. You move your finger to the left, and you’re told what’s right next to it. To interact with the menu bar of the Phone app, you need to move your finger down to the bottom and move from left to right to hear the options such as “Contacts” or “Phone pad”. Yes, there are VoiceOver gestures to explore the screen top to bottom, left to right. You do this by flicking left to right anywhere, and the accessible controls are being walked one by one. But the interaction model is very close to the actual screen layout most of the time. This tremendously helped when I walked through a couple of applications with the sales assistant standing next to me. She could literally point me to the correct spot, and VoiceOver would speak what I needed to hear. Or she could give me verbal directions, and my finger would find the controls.

Typing is probably going to take the most adjusting. It is nothing like typing on the number pad of my N82. James Craig’s typing tip for VoiceOver on iPhone helps a lot: You look for the correct key with one hand, keep your finger there, then tap somewhere on the screen with another finger from the same or the other hand, and the character is input. Gladly, the keyboard doesn’t change position, and after a few letters I had a very good idea where each letter should be, and my typing sped up within 10 typed letters already. In addition, one can turn on word prediction/completion, which is another accessibility feature that can also aid people with motor impairments make typing easier. It plays nicely with VoiceOver.

This is by far not a comprehensive review or comparison. I couldn’t use many of the features since the SIM card in that exhibited model was locked, and I don’t have my own model yet.

Apple are speeding ahead and breaking down conventions in accessibility, or as Mike Calvo of Serotek wrote: They’re getting to the future first. They’re the first to include a screen reader for the blind on one of their mainstream models. Google are going to do something similar with their G1 efforts. The API is there, and some basic console work seems to be working already, but this is by far not as comprehensive as what Apple are doing. RIM also have an accessibility API, but from what I’m told, the screen reading solution that has been hinted every now and then over the past couple of weeks is going to cost extra money, which Apple’s solution does not. The traditional mobile accessibility solutions on Windows Mobile and Symbian S60 all require an additional payment of $200 to $350 for a screen reading solution, or in some cases even proprietary hardware that then costs $2000 or even more.

And this, of course, opens up other possibilities for future implementations of touch screen use cases, not just by Apple, but by other companies as well.

And one more bit of info: The gestures and touchy interface also come to VoiceOver in Snow Leopard with compatible new MacBooks with the multi-finger trackpad. So whenever a colleague tells me to lok for something in the top right quadrant of the screen, I can do that once I have Snow Leopard running on my MacBook. I’ll just put my finger there and let VoiceOver tell me what’s there!

Now my only problem is to get an iPhone. It would appear that my current contract doesn’t allow me to upgrade, since I upgraded it only recently, but too long before I knew the 3G S was coming. We’ll see how I get my hands on a device, it’s not freely available without contract in Germany.

My first touch screen experience was an amazing one!

This entry was posted in Accessibility, Apple. Bookmark the permalink.
Skip to top


103 Responses to My first experience using an accessible touch screen device

  1. Pingback: jim0203 (jim0203)

  2. Hello Marco,

    thank you so much for these insights! I would never have expected a touch screen device reacting so sleek, if you can’t even see anything on it!

    Now, I’m crossing fingers, that you will hopefully soon find a way out of your current contract to get one of these shiny new iPhones. :)

    I’m really excited about this.

    Nice to hear that Google has similar plans with the G1. I got mine on me all the time, so next time around, feel free to check it out. I’m sure there will be news. :)

    One last Question:
    You’re a power user. How do you think the iPhone will work for newbies?


  3. Lucas says:

    Hello, Marco.

    I’m from brazil and I’m not so acostumed speaking english. besides this little problem I read a lot of post in this blog and that’s amazing this one!

    I’ve read one week ago that apple that developed an acessible screem reader, and the most important: it’s included on the iphone 3gs.

    I will wait this product come to brazil to take a look on it and have my own impressions, but I think I will like.

    I found your e-mail to enter in contact with you in particular, but I didn’t find.
    If possible, I would talk to you about some tips to build a blog using wordpress because I’m having problems.

    If it’s possible:
    this is my messenger, if you can add me to change some ideas, I will be glad to you.

    thanks for atention, Lucas.

  4. Kaj Rietberg says:

    In which language did you use the voice-over?

  5. Marco says:

    Hi Sandra:

    You’re a power user. How do you think the iPhone will work for newbies?

    To be honest, I think it’ll work just fine. Primarily because a newbie isn’t biased, set in his/her ways of having used a Symbian or Windows Mobile-based phone for ages and being used to typing on a number pad or some QWERTY keyboards. For them, I think learning the iPhone will be totally different from a newbie learning a Symbian phone, but since there is no previous knowledge, it’ll feel natural I would think.

  6. Marco says:

    @Lucas, you can find my contact info on the “About” page. I didn’t do anything special setting up WordPress, and when I stumbled I had the support forums to help me out.

  7. Marco says:

    Hi Kaj,

    I used it with the German language voice at first, being from Germany and in a German store. But during that time, I also brought up an application that by default speaks in English. The voices used are those of Nuance, originally RealSpeak Solo, but nowadays called Vocalizer I believe. The English voice is that of Samantha, which you can also find in the Victor Reader Stream. The German voice is Yannick, the newest male voice offered by Nuance for this language. I would think that in Dutch, it would use one of those voices available from Nuance.

  8. Pingback: SteveALee (SteveALee)

  9. Hi.
    well I do some unofficialarticles for the jared rimer network tech site.
    your article was linked to from that site.
    to be honest I have got those that say keyboards are still the way to go.
    however the touch screen thing does put out a few more things.
    the main one being that there will never have to be any translation to text, of say computer language.
    when someone says click the black circle in the middle of the screen on the access toolbar or something well its not like accessible at the moment but with this tech I could find it and tap it.
    will really improve accessibility.
    then there is gaming in general a touch screen with this function would really rock

  10. Pingback: Wait till I come! » Blog Archive » TTMMHTM: Religion lulz, 60s computer labs, blind dogs and touchscreens, a new UK power plug and badass babies

  11. Pingback: ste8 (Stefano Ottaviani)

  12. Eitan says:

    Excellent review, this is exciting stuff!

    I am wondering if you could receive the added benefit of a longer battery life by shutting off the screen but still have it’s touch interface. This would also be helpful for privacy reasons.

  13. Marco says:

    Hi Eitan,

    I don’t have my iPhone yet (still!), but read on the VIPhone mailing list that people are using it with a screen that is completely darkened. So I think this is possible.

  14. Pingback: The Accessible iPhone 3GS | ATMac

  15. Pingback: The Accessible iPhone 3GS « The BAT Channel

  16. Pingback: The Accessible iPhone 3GS « AccessTech News

  17. Pingback: Touch Usability

  18. Pingback: chrisjf (ChrisJF)

  19. Kaj Rietberg says:


    Thanks for the reply.

  20. Pingback: iPhone and Voice Over – Blind Wiki Page « AccessTech News

  21. Pingback: iPhone and Voice Over – Blind Wiki Page « The BAT Channel

  22. Pingback: lucia_costa (Lucia Costa)

  23. Pingback: Software Quality Digest - 2009-07-13 | No bug left behind

  24. Pingback: ericbrinkm (Eric Brinkman)

  25. Pingback: Max Design - standards based web design, development and training » Some links for light reading (15/9/09)

  26. Twitter Comment

    RT @gonsalves_r “My first experience using an accessible touchscreen device.” [link to post] #a11y #accessibility #iphone

    Posted using Chat Catcher

  27. Twitter Comment

    First impressions of the accessible touch-screen on the new iPhone, from @MarcoZehe [link to post]

    Posted using Chat Catcher

  28. Pingback: Thoughts around universal access on mobile from Accessibility 2.0 » iheni :: making the web worldwide

  29. Twitter Comment

    Accessible touch screen interface, it’s possible and Apple have already done it – [link to post]

    Posted using Chat Catcher

  30. Twitter Comment

    A user’s first experience with an accessible touchscreen device: the iPhone 3GS [link to post]

    Posted using Chat Catcher

  31. Pingback: Wait till I come! » Blog Archive » Finite Incatatem – my keynote at Accessibility 2.0

  32. Twitter Comment

    An accessible touch screen device! iPhone with built in screen reader [link to post]

    Posted using Chat Catcher

  33. Pingback: Gesture-based VoiceOver on the iPhone and iPod touch | Webs Developer

  34. Pingback: Gesture-based VoiceOver on the iPhone and iPod touch - Programming Blog

  35. Twitter Comment

    Old post, but that’s why I love Apple [link to post]

    Posted using Chat Catcher

  36. Pingback: Wait till I come! » Blog Archive » Mein Vortrag am A-Tag 2009 in Wien: Alles aendert sich, warum nicht wir?

  37. Pingback: Il Web per non vedenti: qual è il giudizio degli utilizzatori di Screen Reader? -

  38. Pingback: Marco’s accessibility blog « BookBolé

  39. Pingback: Marco’s accessibility blog « Inclusive Planet

  40. Pingback: Accessibilità » Blog Archive » Il Web per non vedenti: qual è il giudizio degli utilizzatori di Screen Reader?

  41. Touchscreen says:

    Marco, thank you for shared experience.

  42. Pingback: Funktion für alle » Blog Archive » Tillgänglighet pÃ¥ Iphone 3GS

  43. Twitter Comment

    Wow iPhone is useful to a completely blind person and the guy that uses one [link to post]

    Posted using Chat Catcher

  44. Twitter Comment

    @iheart2code She held the phone up to her head and listened while she touched the screen. Maybe like this description. [link to post]

    Posted using Chat Catcher

  45. Twitter Comment

    @fuzzyraygun I’ve seen that in OS X, but didn’t know the iPhone had that capability

    Posted using Chat Catcher

  46. Pingback: The Blind Buzz on Accessibility « The Blind Buzz

  47. Pingback: The Blind Buzz on Accessibility « AccessTech News

  48. Pingback: The Blind Buzz on Accessibility « The BAT Channel

  49. Muscle Mass says:

    What about your second accessible touch screen device the iPad! Cant wait for it to come to Aus.

  50. Is this for real? How can you navigate on a touch screen that’s so tiny? box cup

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>