Mobile and Wearable Computing

What is Steno Good For?

Part One: How to Speak With Your Fingers
Part Two: Writing and Coding
Part Three: The Ergonomic Argument
Part Four: Mobile and Wearable Computing
Part Five: Raw Speed
Part Six: CART, Court, and Captioning


When I was 10 years old, my big brother William came to visit from California. He was 29, had a mohawk and mirrored sunglasses, worked as an electronic engineer for a tech company in Silicon Valley, and lived in a drainage tunnel because he didn't believe in rent.



My brother William, just after graduating from Cal Poly

He was, needless to say, the coolest human being on the planet. One day he came to talk to my 6th grade class about careers in computer science. He walked into the school wearing a head mounted display that projected green glowing lines of ASCII onto his eyeballs, controlled by a clunky beige laptop strapped to his chest (this was 1991), with a numeric keypad peripheral tied around his right leg, which he controlled using a chording system he'd invented himself. Actually, he'd built all of it himself, putting it together out of spare parts on a whim, and the instant I saw it, I wanted one just like it. It was my first exposure to the idea of writing via chording and my first taste of the dorktastic awesomeness that is wearable computing.

It shouldn't be too surprising that my brother abandoned his wearable rig shortly after he got it all working. It was too heavy and too hot, the display gave him eye strain, and the chording thigh pad was hopelessly slow and uncomfortable.



Not my brother. A guy named Steve Mann.

I've railed often enough against the inefficiencies of qwerty and its tedious one letter per keystroke input ratio, but the one letter per chord ratio of nearly all one-handed keypad systems (Twiddler, Frogpad, Chordite, et al.) is even slower, less accurate, and less ergonomic. The jacked-in cyberfuture of the '90s failed to materialize, and while computers continued to get faster and smaller, they remained external objects, migrating from our desks to our backpacks to our pockets, but refusing to become part of our wardrobes.

Fast forward 19 years. I'm now the age my brother was when he blew my mind by walking into Washington Middle School like an awesome apocalyptic cyborg. Every day I carry a 26-pound bag from my apartment in Upper Manhattan to the subway and settle in for an hour long commute to my office in downtown Brooklyn.



This is actually an old picture. I've gotten a new steno machine and a new laptop since then, and I don't carry the wireless router anymore, since I now use Bluetooth instead. But you get the general idea.

The bag contains one laptop (a Lenovo SL400) and one tablet PC (a Samsung Q1), both running Windows XP; two tripods; a Revolution Grand steno machine; a USB foot pedal for audio transcription; and a bunch of wires and cables. Sometimes I want to get my transcription work done on the train during my commute, so I set up the steno machine on its tripod, press the Samsung Q1 onto its heavy-duty velcro holder, plug in the USB foot pedal and my Sennheiser HD 280 Pro headphones, and start writing. It sounds less awkward than it is. When the train is crowded, that maneuver is all but impossible, and I'm forced to leave all my fancy transcription gear in my bag. Instead, I reach for my phone. I can't do transcription work on it, but it keeps me entertained.

Part three of What is Steno Good For? was composed entirely on my phone (a Blackberry Curve 8330) with two thumbs and a lot of patience. This is what mobile computing means in 2010: Hunting and pecking on a teeny-tiny qwerty keyboard at 20 WPM. Oh, my 10-year-old cyberpunk self would be weeping. But why are these pocket-sized systems currently the most convenient form of mobile text input? Is this as good as it's going to get? Will we be stuck tip-tapping on our phones with our thumbs forever? I sure hope not. The basic problem is this: If it's small enough to fit in a pocket, it's too small to type on efficiently. If it's too big to fit in a pocket, it's too inaccessible to be available on the spur of the moment. There are two potential solutions to the problem. One, clothing-integrated or clothing-mounted text input. Two, virtual space text input. The first one is easy enough to visualize. The second one is pretty far out there, so I won't be addressing it at any length, especially since I'm concerned that its usefulness will be limited without at least a minimal amount of haptic feedback.

As you've probably guessed, this is the least reality-based article in the What is Steno Good For? series. None of my ideas for wearable steno systems are anywhere close to currently available. My scheme is to get people hooked on free software, free steno training, and $60 steno machines first. Then maybe once there's a critical mass of steno-savvy consumers, some company will recognize the demand for efficient mobile computing and manufacture the wearable computing technology I've always dreamed of. My first job is to convince you that steno is actually an ideal solution to the finger size versus pocket size paradox I referred to before. But first, a quick detour on the subject of head-mounted displays. It's a problem that still hasn't been solved to anyone's satisfaction, even after several decades of trying. They're too heavy, too fragile, too stupid-looking, too headache-inducing. But let's posit that someday soon the problem will be solved, and we'll be able to go out and buy lightweight, stylish augmented reality overlay monitors that look just like ordinary pairs of eyeglasses.

So you're walking down the street wearing your AR specs, watching text hover gracefully over all of the local landmarks. What if you want to interact with that text? What if you want to work on your novel while doing your grocery shopping? Text chat with your friend in Belize while walking your dog? Or, you know, write a blog post on a crowded subway? If you're reading this on a desktop computer, put your qwerty keyboard in your lap. If you're not, pretend. Imagine the keyboard split in half and made flexible, melding with the fabric of your jeans and wrapping around each leg. Then imagine trying to type on it while walking. Do you see how many buttons that is? Even if you leave out some of the metakeys, you've got 33 keys plus enter, shift, and space. Make them big enough to fit your fingers, and they wrap clean around to the back of your thigh. Make them small enough to fit completely in the region where your fingers rest naturally, and it's impossible to type on them accurately. The human hand, the human leg, and the English alphabet seem to have irreconcilable differences.



See? Just... No.

But instead of 36 tiny little squares, what about two panels of ten nested rectangles, plus two more resting under each thumb? Compared to the qwerty layout it's both narrower and shorter, and those 24 panels (22 letters plus two asterisks) can be used to produce every letter of the alphabet plus punctuation, commands, and special characters to boot. At 260 words per minute, I might add -- but this article isn't about speed; I'll get to that in the next one. This is about walking and writing or sitting and writing or lying down and writing or doing the cha-cha or riding bumper cars or running a marathon and writing. Unlike on the qwerty keyboard, where you're constantly moving your hands up or down, left or right, leaving and returning to the home row, in steno the only fingers that ever leave their fixed positions are the right pointer for the asterisk and the right pinky for the D and Z keys. All the others stay put, making touch typing much easier, even under bumpy conditions.



Okay, this is really not doing it justice, but I don't have Photoshop, so it's the best you're gonna get.

Don't want to type on the tops of your thighs? Wear a hoodie and type on your abs with your hands in the pockets. Cross your arms and type on your biceps. The limited surface area required by the steno layout means that you could hypothetically write in steno more or less anywhere you can rest your fingers. Now, because it's a two-handed system it's still going to be inconvenient in some situations, but it's got a good shot at succeeding where the qwerty keyboard was inevitably doomed to fail.

I know this clothing-integrated stuff is still many years away, but even on the slightly less mobile front, I'm excited about the dual screen multi-touch laptops I've posted about recently. Putting steno on those seems like a good compromise between my bulky three-part transcription system and all these full-on wearable pipe dreams, and I know they'd be a big help to me in my daily commute. We take our technology with us almost everywhere we go these days, and we desperately need a more elegant way of interacting with it. I think steno could fit the bill.



PS: Just for laughs, here's my current most successful attempt at mobile steno computing. It is not a practical everyday solution. I use it to CART meetings and other events with Deaf or hard of hearing clients who need to walk around a lot; it worked quite well for a meet-and-greet tour of a client's potential grad department, for instance. But this sort of thing is way too bulky and weird-looking for anything but special occasions. It uses my Samsung Q1, my Revolution Grand, some gaffer's tape, and a Connect-a-Desk. Far from perfect, but all I've got for now.