Last week, I published a story here highly critical of NFBCS, NFB and Curtis Chong as leaders in technology related to blindness. The piece, “Accessibility And NFBCS” described a number of incredibly important issues in technological accessibility for people with vision impairment in which the largest advocacy organization in the world of blindness remains absent and asks how they can be effective leaders if they ignore the most important events of the day.
The article also discussed the question of whether or not it would have been better if Microsoft had made its own “end to end” screen reader. I believe that, as Apple provides on iOS and Macintosh and Google includes on Android, that all OS should have a fully functional screen reader shipping out-of-the-box. Sighted people don’t need to pay extra for the graphical UI they use, blind people should not need to pay extra for the UI we use either.
In last week’s article, I discuss the NFB role in pressuring Microsoft into not doing its own screen reader, favoring instead the high priced, third party solutions from Freedom Scientific, GW Micro, Dolphin and other companies. Last week’s article was specifically about NFBCS and Curtis Chong’s writings in Braille Monitor. It, therefore, described the NFB role in the third party screen reader story with little context. In the early drafts of that piece, I did include much more historical context but those early drafts of the article contained more than 6000 words and the final version that I actually published still had more than 3600 and was “too long” for some of my readers.
After publishing the story last week, I spent a few hours talking on the phone with NFB insiders who, like me and the other sources I used for that article, were actually present for some of the meetings with Microsoft and were observers to this history. While I feel that the story I told last week about NFBCS and its role is true, I also think it’s important that I tell the rest of the story.
I try to publish here every Tuesday. In some weeks, I have a lot of time to do a lot of research and write fairly formal pieces. Some weeks, like this one, I’ve less time to devote to the blog and, therefore, will be telling this story largely from memory. Unlike some articles here, it will not contain a lot of links to outside sources and such because I just haven’t the time to do so this week.
Ted Henter’s Speech
In the late nineties, Ted Henter, founder of Henter-Joyce and the inventor of JAWS, took the stage at an NFB convention general assembly and made a speech detailing exactly why he felt that screen readers will always be best if developed by small companies dedicated entirely to access technology. One may believe that Ted said these things out of a purely cynical desire to protect the profits of his own company but, while tis may be partially so, having worked for, talked to, hung out with and been friends with him for more than a decade now, I’m confident that Ted was speaking honestly to what he felt was a greater good.
When Ted made that speech, there were no fully functional screen readers built into operating systems. IBM had made two screen readers, Screen Reader/DOS and Screen Reader/2 but neither had ever gathered much popular appeal. Vocal-Eyes from GW Micro had been the most popular DOS screen reader among American users, followed by the DOS version of JAWS. When Windows came along, JAWS for Windows (JFW) and Window-Eyes would together dominate the market. Thus, when Ted made his speech, there were no examples of a fully functional screen reader having been accepted broadly by the user community, thus, no evidence of an OS vendor making a tool that our community would actually enjoy using.
The Others In The Room
Starting around 1995, MS held meetings on its campus up in Redmond to which as many accessibility oriented stakeholders were invited. This, of course, included the screen reader vendors, advocacy organizations including NFB, ACB and AFB and other notables from the world of technology and disability. As I state in the introduction, I am writing this from memory and the two NFB insiders to whom I spoke last week were also telling the story from their memories so, please realize, the following may be a bit foggy as that’s how human memory works.
As far as I can tell, everyone in the room at those meetings, the AT companies trying to protect our profits and the advocacy organizations speaking on behalf of their constituents, agreed that the third party screen reader system would provide the greatest access for the vast majority of users. I will contend that, then and for a number of years into this century, this model was probably the right path to take.
Life Before Accessibility API
If you are one of the many blind people who enjoy using an iOS device from Apple (iPhone, iPad, iPod Touch), you are benefitting from Apple’s very well designed accessibility API and compliance with it in the apps you use. In the late nineties, though, there were no examples of a functional API driven screen reader anywhere.
The Big Hack
In the days before modern accessibility API had been invented, Windows screen readers used a technique called “driver hooking.” In brief, a part of JAWS, HAL, Window-Eyes and the others pretended it was a video card driver and gathered its data in what we called an “off screen model” (OSM). In brief, the OSM was a database of information sorted primarily on where things appear on the screen. Using some other techniques, the screen readers knew which Window, hence, application they were in and, along wit painstakingly developed heuristics for each program they hoped to support, they would then speak and/or braille the information for their users.
A little example of how this worked that I can recall of the top of my head is how JAWS tracked focus in Excel. To “know” which cell a user had in focus, the JAWS Excel support would “see” that the software had issued a series of “LineTo” graphical drawing commands to the video driver. JAWS had no actual idea about what was going on in Excel, so it had to jump through heuristic hoops to do something as simple as tracking focus.
Needless to say, a system built on having human programmers spend so much time figuring out the strange details of how a rectangle is drawn on the screen to track focus had severe limitations. Anytime an application made the slightest change, it would likely break the screen reader.
The OSM and screen scraping techniques also introduced major stability problems as it was such a non-standard way of doing things that neither Windows nor the applications running on it were aware of the screen reader as a device driver and, for their own purposes, also used non-standard techniques to put information on the screen. An application would try to do its own optimizations and would, as a result, cause a screen reader to crash.
Microsoft’s was the first OS vendor to attempt to build an accessibility API. It was called Microsoft Active Accessibility (MSAA) and, to be frank, it was little more than a demo of things to come. MSAA, even in its 2.0 version, did not provide a screen reader with enough information to provide its users with a usable solution. Thus, all screen readers using MSAA back then also had to include non-standard techniques to provide information properly to their users.
This fact alone is a large part of what made it virtually impossible for MS to make its own screen reader, until the advent of UIA (later in this piece), it was impossible to use anything resembling standard techniques to gather information.
The VBA Hack
At Henter-Joyce, Glen Gordon, still the top developer on JAWS, realized that using the automation API designed for VisualBasic programmers to extend Microsoft Office and other applications could also be used to get data into a screen reader. The JAWS team took Glen’s idea and ran with it. This was what the JAWS developers used to do all of the amazing things we did in MS Office back then.
The good thing about using the VBA approach was that we could gather very specific information in the context of the application being employed. We no longer had to follow graphics commands to determine focus, we could get the precise coordinates by simply asking Excel. There were two major downsides to this approach, it required that each application supported in this way have hand coded scripts written for it, hence, it also required that the screen reader had a scripting facility, something that, back then, Window-Eyes didn’t have and proudly boasted that they didn’t even want.
The Window-Eyes Versus JAWS Approach
At this point in our story, I must acknowledge that GW Micro, in what felt to me like business suicide, chose to embrace the MSAA path while we, at Freedom Scientific, continued to reject it. History shows us that GW Micro was on the right side of the technological discussion then but, in my mind, they arrived at that conclusion too early.
While GW Micro would provide no cost consulting advice to billion dollar corporations on how to get MSAA implemented in their software, we, at FS continued down the path of non-standard solutions. In many ways, this is what put JAWS on top of the heap, it could do things no other screen reader could and it provided the best experience for its users as a result.
The Hole In The JAWS Approach
In order to provide the best possible collection of data to its users, JAWS employed v very non-standard techniques mixing its OSM with the VBA support and MSAA where available. The outcomes were terrific for the end uses but, inside FS, continuing to support each application using custom code (scripts and internal) was an expensive proposition. In the world of generic accessibility API, a screen reader like VoiceOver gets all of its information from said interface, hence, when the app changes, no one needs to go in and change the code in the screen reader. This was not true then and to a lesser extent now for JAWS.
As I’ve written here many times before, by 2003, JAWS had reached a monopoly marketshare at around 85% of all new product sales. FS, therefore, had no incentive to continue investing in JAWS as it had won the race. Thus, with JAWS receiving less and less funding annually, these custom solutions started to deteriorate.
The First Real Accessibility API
While MSAA was a pretty poor bit of technology, Microsoft should be recognized for even giving it a try. As there were no accessibility API in existence before MSAA, they had no point of reference and MSAA stood as a solid prototype for things to come.
In my opinion, the first truly usable accessibility API was the one led by Peter Korn at Sun Microsystems. It not only provided specific information about a control, it had the facility to provide its context, hence, enable a screen reader to provide a more complete picture to its users. In effect, the Gnome Accessibility API was the first of its breed.
Apple would follow with its API and Microsoft would craft something called User Interface Automation (UIA) that serves as both an accessibility API and a test framework for the software. Today, with comprehensive accessibility API on all major OS, it’s reasonable to expect a fully featured screen reader also to be included with of them.
On iOS, OSX and Gnome, there’s one accessibility API on each. On Windows, the OS with the largest number of users, blind or otherwise, there is a second one called iAccessible2 (iA2). Some experts would argue that iA2 is the best and most important of the various accessibility but, in all honesty, I’m not expert enough to describe either its benefits or any pitfalls within it. The Mozilla Foundation chose iA2 over UIA or MSAA in the Windows versions of it’s applications and it’s iA2 that you’re enjoying when you use FireFox with NVDA.
When iA2 was developed, it was intended to be a cross platform accessibility API. The goal was to permit application developers who build software on multiple OS to write their accessibility code once and be able to reuse it on other systems. Sadly, while iA2 is “owned” by the Linux Foundation, it has never been implemented on any system other than Windows, something I think is a shame.
Another huge question is whether or not Microsoft will support iA2 in its “end to end” Narrator. They may elect to only support UIA/MSAA and, therefore, leave FireFox and the other iA2 enabled applications out of the sphere of its support. This could be another reason third party screen readers like NVDA may stick around into the future.
The only conclusion I can draw about the entire third party screen reader debate is that the story is much more foggy than one might think. NFB did insist on promoting the third party, mom and pop company solutions but they didn’t do so alone. HJ/FS, GW Micro, AFB, ACB and everyone else in those meetings except for MS agreed that this was the best path forward. History and Apple have demonstrated that it is, indeed, possible to deliver a high quality, no cost screen reader along with the OS and, holding that high priced, proprietary third party screen readers are a favorable solution in the 21st century is purely anachronistic thinking.
I do not mean to suggest that third party screen readers will or even should disappear entirely, they will likely fulfill an important set of requirements for a lot of people, third party screen readers might even become the luxury products of the Windows world, supporting applications too old to have included MSAA/UIA or by providing a user experience different from and preferred by some to the generic Narrator. Of course, I’ve been predicting the demise of the commercial third party screen reader since I wrote an article slamming JAWS 7 on my old blog so I’m likely not the best prognosticator of things to come.
I would like to express my thanks to the loyal NFB members to whom I talked on the phone and exchanged emails with last week. I appreciate the insight you guys gave me, a different perspective on the different meetings up in Redmond and the help you provided in writing this article. I appreciate honest and sincere dialogue and also thank the NFB faithful and everyone else who posted comments on last week’s article. In my opinion, the more discussion we can have about this community within this community the better.
Fortunately for all involved, technology progresses. At the AFB Leadership conference in Phoenix last week, Rob Sinclair announced that Narrator would, in Windows 10, be an “end to end” screen reader. Only the future can tell us how it will work out.