Saturday, March 27, 2010

How to speak with your fingers



Now for the big question: Why do this? Why spend time and money developing a free program to let people type at 250 words per minute? What is steno good for?

I can think of a few groups that might benefit from free steno technology:

* People who don't use their voices to communicate, who would benefit from using a synthesized voice that can speak at conversational speeds.

* Writers, programmers, and other people who spend a lot of time working with text, who would like to set down their thoughts in a more fluent, natural way, rather than having to type them out laboriously, letter by letter.

* People who want to avoid or ameliorate the risk of repetitive motion disorders such as tendonitis or carpal tunnel syndrome.

* People interested in mobile/wearable computing and augmented reality.

* People who want to break onto the high score tables of online typing games, or who spent months teaching themselves the Dvorak keyboard layout for -- at most -- a 20% increase in speed, because they want to do everything as quickly and efficiently as possible.

* People who might be interested in court reporting, captioning, or CART (communication access realtime transcription) as a career, but who don't want to invest thousands of dollars in equipment, software, and training fees before knowing for sure whether it's the job for them.

I'll take the first group first and follow up on the other ones in subsequent blog posts.

Many people communicate without using their voices. Some are Deaf, some have cerebral palsy or other muscular disorders, some are on the autistic spectrum, and some have issues with their brain, tongue, jaw, larynx, or lungs which prevent them from producing comfortable or intelligible speech. Some of these people communicate using sign language. Others use assistive technology, including speech synthesizing software.

For example:

* Stephen Hawking

* Roger Ebert

* Schuyler Rummel-Hudson

* Alan Doherty

Not everyone who uses a speech synthesizer could benefit from steno. People with paralysis, like Stephen Hawking, don't have full control over their fingers, so they input text using rocker switches, sip-and-puff devices, or eye-gaze cameras. Schuyler Rummel-Hudson, who's 10, currently uses a small computer with pictographic symbols that stand in for common words and phrases. As she gets older, she might choose to switch to an alphabet-based system, but pictographs are sometimes more useful for people who have difficulty with fine motor control.

Steno would almost certainly be a great help to people like Roger Ebert or Alan Doherty, though. They don't have lower jaws, but their fingers work just fine. Currently they're forced to type everything they want to say on an ordinary keyboard, either letter by letter or using predictive text systems, which require around three to four keystrokes per word. The very best qwerty typist can get up to around 130 words per minute, but normal conversational speed is usually around 180, and often goes as high as 260 words per minute. People who rely on qwerty keyboards to communicate face the choice either of playing back pre-written sentences or requiring long pauses between each sentence. Either way disrupts the natural rhythm of conversation.

Steno, by design, can be written as quickly as English can be spoken. It also allows for greater fluency of thought (which I'll touch on more in the segment about writers and programmers), because it works syllabically rather than letter by letter. For instance, I'll take a random sentence:

"Whether or not the application is completed depends on your full cooperation."

That's 78 keystrokes on a qwerty keyboard. On a steno keyboard? Twelve strokes in all, making it over six times more efficient.

Has anyone used steno for accessibility before? Well, obviously it's been used for nearly two decades by CART providers like me on behalf of Deaf and hard of hearing people, but none of my research has turned up any accounts of disabled people using steno on their own behalf to communicate. (If you do know of anyone, though, please contact me! I'd love to hear about it.) The reason is not too hard to guess: Steno is ridiculously expensive, and it's got a relatively steep learning curve. Steno software (which costs around $4,000) is designed primarily for court reporters, and is not completely compatible with text-to-speech applications. Steno machines ($2,000 to $5,000) are -- with a few exceptions -- heavy, bulky, and anachronistic-looking.

Virtually everyone who learns steno these days does it because they intend to make a career out of it. The cost of the technology is prohibitive to dabblers, hobbyists, and people who don't have the time or inclination to undergo intensive court reporter training. Plover is an attempt to eliminate the $4,000 software cost. I'm currently looking into low-cost solutions for steno hardware. Once those two barriers are removed, I think the training will largely take care of itself. It took me a year and a half of intensive practice to get from 0 to 225 words per minute, but I was writing at 100 words per minute after only a few months, and that was true of nearly all my classmates as well. Steno is so vastly more efficient than qwerty that even a beginning stenographer can outstrip the best qwerty typist relatively quickly.

People who use assistive technology are uniquely suited for the task. They're often at their computers for many hours a day and have a high incentive to learn everything they can about their equipment. It's almost a truism that people with disabilities are usually the earliest adopters and most dedicated power users of almost any new technology. Voice-to-text software, which has proven invaluable to people who are unable to type, requires as much if not more training time than steno, as I mention in my post on Voice Writing.

Steno can provide the same benefits in the opposite direction. In addition to people like Mr. Ebert and Mr. Doherty, who can't speak but are able to hear, steno technology could do a lot for my own clients, who are primarily Deaf and hard of hearing. Without having to hire a CART provider, people who are hard of hearing and don't know sign language can speak to each other using steno as quickly as thought, with no potential for misunderstanding. People who are Deaf and do use sign language rather than spoken English can communicate with Hearing English speakers by writing what they want to say in steno and using the voice synthesizer to speak for them, rather than having to rely on hand-written notes when interpreters aren't available.

Even people who don't have any problems with hearing or speaking might start to use steno, as a way to communicate in noisy clubs or libraries, or as a high-speed substitute for texting. I'll write more about that in my wearable computing post, but the bottom line is this: The more people communicate using steno, the more universally accessible our society will be.

Qwerty is a venerable and popular input system, but at its best it's only a third as efficient as human speech. Steno is every bit as efficient as human speech, and it's been forced into undeserved obscurity by its high entry cost and inaccessible design. Plover is an attempt to fix that. I'll keep you posted.

11 comments:

Sonja Elen Kisa said...

One big area where conventional theories might be lacking, if we integrate steno writing to computing, is briefs or ways to handle common computing keystrokes or punctuation:

Ctrl-Alt-Del
Curly braces
Angled brackets
Alt-Tab
Win-D
F11
PgDn
etc.

I started a thread on this topic in depoman.com in "theories

Mirabai Knight said...

I actually defined Alt-Tab in my dictionary (TLA*BLT) and then discovered that it didn't actually do anything, because the access that Eclipse gives steno input is not low enough to mimic the access that a qwerty keyboard has to the operating system. That was actually a big spur to the development of Plover; I wanted my steno machine to do everything a qwerty keyboard could do, and commercial software, despite the $4,000 price tag, wouldn't let me do it.

I do use angle brackets successfully with Eclipse, though -- BRBGT for open bracket and BR*ABGT for close bracket. I don't think that coming up with outlines for commands is much different from coming up with outlines for other words and phrases. I talk about some of my command outlines in this Depoman thread.

Roger Ebert said...

Consider me interested. I've downloaded everything, but at first glance the instructions look daunting. I will see..

Thanks for thinking of people with this problem!

RE

Mirabai Knight said...

Mr. Ebert, I'm so sorry that I haven't responded until now, but I wanted to wait until I had a video demonstration for you. I've just uploaded it:

http://plover.stenoknight.com/2010/12/plover-espeak-conversational-realtime.html

I've also added the same video to the top of this entry. I'm afraid that Plover isn't currently compatible with Macs (it only runs on Linux), but Mac compatibility should be coming along very soon, and we'll be sure to contact you as soon as it does. Plover definitely will work with the custom voice you had built, because it's designed to precisely simulate the qwerty keyboard -- only much, much faster.
Thank you for dropping by the blog. It's an honor. I'm extremely excited by your interest, because I think this could be a life-changing program for you and for many other people who want to speak at conversational speeds without using their voices. In your early days as a newspaperman, did you ever learn shorthand? If so, learning machine stenography should be easy; the principles are very similar. Even if not, I've found that most people can outstrip their qwerty typing speeds with only a few months of practice. Plover's in constant development, and some really exciting stuff is just around the corner, so please stay tuned.

StevenBhardwaj said...

Amazing blog and software! My initial googling of stenotype found nothing but admonitions against using a steno machine without paper output was.

Do you know of anyone that uses steno for realtime voice-to-text interlingual interpretation?

StevenBhardwaj said...

So I'm not yet a steno geek... (emphasis on the 'yet'!!)
And I'm not a coder geek either, or a linguist.
But I am a language geek, so I 'brief'-ly looked up steno in my favored non-English languages, and here's what I found.

Spanish has the Spanish Association of Stenotypers:
http://www.estenotipiaasociacion.com/
Spanish is very phonetic so I imagine it would be relatively easy application

of stenotype, having only skimmed Mirabai's 101 series and watched her

interview videos.

Chinese amazingly has the 亚伟中文速录机, the "Ya Wei Chinese Stenotype Machine,"

which literally means the "Powerful Asia" Chinese Fast Note-taking Machine.

Apparently it's selling for 5,800 Taiwanese dollars... thats ~USD$200. Looks like the English language will need you guys, Mirabai and Josh and Co!
Here's the original link I found:
http://baike.baidu.com/view/445512.htm
and here's the google translate URL:
http://translate.google.com/translate?sl=auto&tl=en&js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Fbaike.baidu.com%2Fview%2F445512.htm
And here's the company website with the price:
http://www.suji.com.cn/onet/slj/index.htm#
And here's the google translate url of the company website:
http://translate.google.com/translate?sl=auto&tl=en&js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Fwww.suji.com.cn%2Fonet%2Fslj%2Findex.htm%23

French
The French stenotype website emphasizes that stenotype was invented by a Frenchman, and although there is no professional organization linked from the wikipedia page, I found this:
http://www.stenotype-grandjean.com/index.php?option=com_frontpage&Itemid=1

So all my languages of interest have a stenotype thing going, very encouraging! Also on my hit-list, Swahili has a version of Pitman shorthand, but Swahili and Kinyarwanda are very phonetic like Spanish so they might not be hard to adapt a theory for.

Big question for all you amazing folks: Would it be possible to adjust Plover to apply directly to Spanish steno (start with the easier one) by just changing the dictionary?

:D
-Steven

StevenBhardwaj said...

I was in error, it's 5800 元/台 which is not taiwanese yuan, but mainland renminbi. So let's put that price back to a more comparable $1000 / machine.

Mirabai Knight said...

Do you know of anyone that uses steno for realtime voice-to-text interlingual interpretation?

I know it's been done, but I don't know any specific names of people who offer it. I wish I was multilingual; it would be a great thing to be able to offer. (My parents speak six languages between them, but they didn't teach me anything but English, so sadly I fall into the Ugly American stereotype. Sigh.)

I know of several Spanish stenographers working, and I think Plover should be pretty adaptable to various language layouts; it's just a matter of changing the settings in the layout table. That's definitely a great thing about open source. People can customize the software to suit their needs by just opening one file in a text editor and swapping out some letters.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.