Friday, April 2, 2010

Ploverwinder takes flight!



I have to edit an entire play script tonight (I'm captioning a matinee tomorrow), so I don't have much time for a long post, but I just wanted to show off the fruit of this morning's labors. Some lovely people in the #python IRC channel gave me a brilliant line of code that was the last thing I needed to turn my Sidewinder X4 into an actual working steno machine... As long as you don't mind pressing enter after every stroke. Next I have to figure out how to make the program recognize the interval between when a key or group of keys is pressed and when they're all released so that it can delineate strokes automatically. For now, though, it's recognizing every chord I throw at it and translating them as cleanly as if it were coming from my Revolution Grand.



Anyone who wants to play around with it can download stenowinder.py and ploverbd.py (the dictionary file) at the github. Put them in the same directory. If you've got Python 3 installed, you should be able to just run the program without installing anything else. Let me know if that isn't the case. If your keyboard doesn't have anti-ghosting like the Sidewinder X4, you'll probably have to press each key individually rather than chording them, but at least you can get a taste of how it's supposed to work. I am ridiculously excited, but I'd better go do what I actually get paid for. More later!

Saturday, March 27, 2010

How to speak with your fingers



Now for the big question: Why do this? Why spend time and money developing a free program to let people type at 250 words per minute? What is steno good for?

I can think of a few groups that might benefit from free steno technology:

* People who don't use their voices to communicate, who would benefit from using a synthesized voice that can speak at conversational speeds.

* Writers, programmers, and other people who spend a lot of time working with text, who would like to set down their thoughts in a more fluent, natural way, rather than having to type them out laboriously, letter by letter.

* People who want to avoid or ameliorate the risk of repetitive motion disorders such as tendonitis or carpal tunnel syndrome.

* People interested in mobile/wearable computing and augmented reality.

* People who want to break onto the high score tables of online typing games, or who spent months teaching themselves the Dvorak keyboard layout for -- at most -- a 20% increase in speed, because they want to do everything as quickly and efficiently as possible.

* People who might be interested in court reporting, captioning, or CART (communication access realtime transcription) as a career, but who don't want to invest thousands of dollars in equipment, software, and training fees before knowing for sure whether it's the job for them.

I'll take the first group first and follow up on the other ones in subsequent blog posts.

Many people communicate without using their voices. Some are Deaf, some have cerebral palsy or other muscular disorders, some are on the autistic spectrum, and some have issues with their brain, tongue, jaw, larynx, or lungs which prevent them from producing comfortable or intelligible speech. Some of these people communicate using sign language. Others use assistive technology, including speech synthesizing software.

For example:

* Stephen Hawking

* Roger Ebert

* Schuyler Rummel-Hudson

* Alan Doherty

Not everyone who uses a speech synthesizer could benefit from steno. People with paralysis, like Stephen Hawking, don't have full control over their fingers, so they input text using rocker switches, sip-and-puff devices, or eye-gaze cameras. Schuyler Rummel-Hudson, who's 10, currently uses a small computer with pictographic symbols that stand in for common words and phrases. As she gets older, she might choose to switch to an alphabet-based system, but pictographs are sometimes more useful for people who have difficulty with fine motor control.

Steno would almost certainly be a great help to people like Roger Ebert or Alan Doherty, though. They don't have lower jaws, but their fingers work just fine. Currently they're forced to type everything they want to say on an ordinary keyboard, either letter by letter or using predictive text systems, which require around three to four keystrokes per word. The very best qwerty typist can get up to around 130 words per minute, but normal conversational speed is usually around 180, and often goes as high as 260 words per minute. People who rely on qwerty keyboards to communicate face the choice either of playing back pre-written sentences or requiring long pauses between each sentence. Either way disrupts the natural rhythm of conversation.

Steno, by design, can be written as quickly as English can be spoken. It also allows for greater fluency of thought (which I'll touch on more in the segment about writers and programmers), because it works syllabically rather than letter by letter. For instance, I'll take a random sentence:

"Whether or not the application is completed depends on your full cooperation."

That's 78 keystrokes on a qwerty keyboard. On a steno keyboard? Twelve strokes in all, making it over six times more efficient.

Has anyone used steno for accessibility before? Well, obviously it's been used for nearly two decades by CART providers like me on behalf of Deaf and hard of hearing people, but none of my research has turned up any accounts of disabled people using steno on their own behalf to communicate. (If you do know of anyone, though, please contact me! I'd love to hear about it.) The reason is not too hard to guess: Steno is ridiculously expensive, and it's got a relatively steep learning curve. Steno software (which costs around $4,000) is designed primarily for court reporters, and is not completely compatible with text-to-speech applications. Steno machines ($2,000 to $5,000) are -- with a few exceptions -- heavy, bulky, and anachronistic-looking.

Virtually everyone who learns steno these days does it because they intend to make a career out of it. The cost of the technology is prohibitive to dabblers, hobbyists, and people who don't have the time or inclination to undergo intensive court reporter training. Plover is an attempt to eliminate the $4,000 software cost. I'm currently looking into low-cost solutions for steno hardware. Once those two barriers are removed, I think the training will largely take care of itself. It took me a year and a half of intensive practice to get from 0 to 225 words per minute, but I was writing at 100 words per minute after only a few months, and that was true of nearly all my classmates as well. Steno is so vastly more efficient than qwerty that even a beginning stenographer can outstrip the best qwerty typist relatively quickly.

People who use assistive technology are uniquely suited for the task. They're often at their computers for many hours a day and have a high incentive to learn everything they can about their equipment. It's almost a truism that people with disabilities are usually the earliest adopters and most dedicated power users of almost any new technology. Voice-to-text software, which has proven invaluable to people who are unable to type, requires as much if not more training time than steno, as I mention in my post on Voice Writing.

Steno can provide the same benefits in the opposite direction. In addition to people like Mr. Ebert and Mr. Doherty, who can't speak but are able to hear, steno technology could do a lot for my own clients, who are primarily Deaf and hard of hearing. Without having to hire a CART provider, people who are hard of hearing and don't know sign language can speak to each other using steno as quickly as thought, with no potential for misunderstanding. People who are Deaf and do use sign language rather than spoken English can communicate with Hearing English speakers by writing what they want to say in steno and using the voice synthesizer to speak for them, rather than having to rely on hand-written notes when interpreters aren't available.

Even people who don't have any problems with hearing or speaking might start to use steno, as a way to communicate in noisy clubs or libraries, or as a high-speed substitute for texting. I'll write more about that in my wearable computing post, but the bottom line is this: The more people communicate using steno, the more universally accessible our society will be.

Qwerty is a venerable and popular input system, but at its best it's only a third as efficient as human speech. Steno is every bit as efficient as human speech, and it's been forced into undeserved obscurity by its high entry cost and inaccessible design. Plover is an attempt to fix that. I'll keep you posted.

Friday, March 26, 2010

The Sidewinder X4 is here

It arrived today. This evening I cut up a couple of my old leather steno keypads, and laid out a steno machine on its keys.

This is a terrible webcam shot (I'll try to get a proper one with my partner's digital camera later), but you can sort of see the shape of the thing if you look for the non-illuminated black squares:



And here it is with the equivalent steno letters superimposed on the image by means of Microsoft Paint. Yeah, we're pretty high tech up in here.



I've tried it out, and the touch is actually not that bad. Some of the twistier steno outlines are a little awkward because the keys are quite a bit closer together than on a regular machine, but the anti-ghosting is working like a dream. I wrote the phrase "So is this working?" in steno six times and got this:

avarwdvlru
vaarwdlvru
avawrldvru
avarwldvru
avarwdvlru
avarwvdlru

AV or VA is "so" (SO)
A is "is" (S-)
WR or RW is "this" (TH-)
DVL or DLV or LDV or VDL is "working" (WOG)
RU or UR is "?" (HF)

The order doesn't matter in steno, so this amount of key-recognition consistency should really be all we need to make a relatively functional machine. I'm very excited. My Python tutor can't meet next week, but the week after that we might try to get the Sendkeys issue resolved, or we might backtrack a bit and see if we can get this $60 steno machine (plus $10 worth of keypads) working with our dumb terminal version of Plover. How amazing would that be, huh?

Tuesday, March 23, 2010

Figuring it out

Well, my tutor and I didn't get any code written today; the lesson was taken up with trying to make SendKeys work, with next to no success. The problem is that SendKeys was written for Python 2.1 and was last updated in 2003. We've been writing Plover in Python 3.1, so it doesn't work. Our options are:

* Figuring out a way of updating SendKeys so it's compatible with Python 3.
* Revising Plover so that it's compatible with SendKeys as it is (might be tricky because pyserial requires Python 2.3 or later; plus my heart bucks against the idea of regressing backwards into the unsupported Pythonic past.)
* Finding an alternative to SendKeys -- possibly using the script found in the last comment of this?

On the hardware side of things, I just ordered a Sidewinder X4, which I mentioned in an earlier post. If it's really true that multiple keys of the keyboard will be recognized as chords, maybe I can train my fingers to ignore the slight misalignment between rows and learn how to play the qwerty keyboard like a steno machine. That is:


QWER TY UIOP[
ASDF GH JKL;'
CV NM

would map to

STPH ** FPLTD
SKWR ** RBGSZ
AO EU

I tried a little bit experimentally on my laptop's keyboard, just mashing keys into a Vim window ("just mashing keys into a Vim window" came out as "mapfdsxkejrlscn;wxoianfmpdimn", but I think some of the letters weren't recognized, since my Lenovo's keyboard doesn't have any anti-ghosting abilities), and it feels weird, but I think it might be possible with practice. The main barrier I'm worried about is that of determining spaces between strokes. Steno machines register keypresses once all of the keys have been lifted; qwerty keyboards register keypresses as soon as each one is pushed down. Will it be possible to put in some sort of software hack that registers each stroke as finished every time the keyboard realizes that none of its keys are being pressed?

Tuesday, March 16, 2010

Wacom + Keypads = Steno?

Could this be an answer to our steno hardware problem? It talks about unlimited multitouch, up to 20 fingers or more. If you program in a steno keyboard layout and stick some keypads on there for ergonomic feedback, could that serve as a poor man's Lightspeed? The question is whether it'll actually be cheaper. Low-end drawing tablets have gotten quite reasonable in price; I saw one at Staples the other day that boasted multitouch (meaning up to three inputs at a time) and went for only $50. But this looks like a flagship model, and I know some of the top-of-the-line drawing tablets go for several thousand. Still, there's a lot bigger market out there for those than there currently is for steno machines, so if the price comes down in a year or two we might have our solution.

Monday, March 15, 2010

Numbers are working. Now what?

I just had a great Python session with my tutor, and we've gone about as far as we can go with our current dumb terminal version of Plover. Everything is translating properly, including numbers. The new version of the dictionary contains all the entries I use with my proprietary software. It's all very exciting.

But I'm still the only person using Plover. It's very useful when I'm outputting my CART display to one computer using my proprietary software and echoing it on another computer using Plover. That's a pretty specific activity, and if there's anyone besides me who thinks it would come in handy for them, they haven't said anything to me about it. No, in order for this thing to do what it needs to do, it has to output to the operating system, just like any ordinary qwerty keyboard does when you plug it in and start typing.

My first volunteer beta tester uses Fedora, so whatever I use to rig the output has to work on Linux. I'm on XP, so we need a Windows version too. I wish I could use the same code for both -- and if anyone reading this has an idea of how to do that, please let me know -- but from the brief research my Python tutor and I did before the end of today's session, it looks like we're going to have to use something like SendKeys for the Windows version and... I don't know. Whatever GOK uses in Linux? This is all very much uncharted territory for me, so I'd love advice from anyone who knows how to get input from a Python program and use it to emulate keyboard output. If you've got any ideas, comment here or email me: plover@stenoknight.com.

Plover is getting uglier...

But it's all to the good! In the first version of the dictionary file I uploaded, I just took out all the entries that weren't plain text, so Python wouldn't blarf on them. This time I've actually gone through and changed the metacharacters to typographic symbols like ^ (caret); this makes the output of Plover look much uglier than before, but it means I now have my fingerspelling alphabets back, and as soon as we pipe this stuff out to external programs, we'll be able to use those characters to tell Plover when to capitalize a word, when not to insert spaces between translations, and all that good stuff. So hold your nose and have a screenshot!

New Plover screenshot