A few months ago I had an RSI problem so bad - able to type only a minute at a time, even sitting with hands on keyboard hurt - that I started down this route. This video was, literally, a life-altering motivator for me, and I was quite obsessed with it.

Ironically, after seeing a physical therapist - which, let me tell you, you should do at the first sign of pain, because while they can't help some people I personally am batting 1.000 with PTs for RSI over my many-year career - my recovery is now so complete that I've totally fallen off the voice-computing path... for now. But I intend to keep going, not just because it is hilarious but because, well, RSI happens and it really pays to vary the routine sooner rather than later. There is nothing like trying to do a ton of emergency scripting on Python and emacs at the lowest possible point of your productivity.

The most important hint I have so far is: do not waste time with Mac OS. You need a PC running the Windows version of Dragon. The Mac version is pretty good for occasional email but lousy for emacs because it doesn't have the Python hook into the event loop that a saint hacked into the PC version years ago before leaving Dragon.

The speechcomputing.com forums are your friend.

Yeah, they say there is an open-source recognition engine that works okay, and time spent improving free recognition engines is time that really improves the world for all kinds of injured people, but here's the problem: when you need a speech system you really need it, and there are a lot of moving parts. Dragon, and Windows, and a super PC to run it on are super cheap compared to your time, especially when your time is in six-minute increments punctuated by pain.

I guess it depends on the type of software you're working on, but input speed has never been close to being the bottleneck with coding for me...

Most of the time I'm trying to figure out what to do or how to implement an algorithm. Rarely do I get those mad-scientist frenzies where I'm typing away frantically trying to get all the words down as they come into my mind in a flash of inspiration.

Tangentially related, but I'll throw it in here, since so many developers aren't taking ergonomics seriously. RSI can happen to you if you are not careful, and it can wreck your career (almost happened to me). Several years ago, I started having aches in my arms. Over half a year it got gradually worse, until it was so bad, I thought I had to give up coding altogether. Fortunately, I managed to get it under control, mostly with the aid of a break program, and an ergonomic keyboard and mouse. I'm now completely over it, but I still need to be careful not to get it back. A lot more details in this post: http://henrikwarne.com/2012/02/18/how-i-beat-rsi/

My counter-argument to voice-driven coding has been primarily around the input bandwidth and the fact that you must work from home with that kind of setup.

I guess the presenter conducted the "faster than the keyboard" test under very controlled circumstances (e.g. only working on his own code, so one doesn't have to deal with non-english-word variables/functions).

I don't mean to be a hater, because that was an _amazing_ demo, but I don't believe it's the holy grail the title implies it is.

"Emacs pinkie" is a non-issue if you use a keyboard with thumb clusters, e.g a Maltron or a Kinesis model. Investing in a good keyboard is just as crucial as investing in a good chair, especially if you make a living by coding. The time that you spend compensating for a bad input device by hacking your own workarounds can be more costly then spending money on a proper solution.

Once you are an adequate touch typist typing speed is only beneficial if you use a language that requires you to type a lot of boilerplate. Even then, you can use an IDE for auto-completion. I can type at very high speeds — as fast as others can input text by using their voice — but I can't remember the last time I needed to type for more than a minute at a time. If you use a language that requires you to spend more time thinking about code than it does to actually type it, typing speed really doesn't matter. Code is like speech in that it is judged by the eloquence, not the speed, of its delivery.

I was trying to work something like this out to try about a month ago but had to put it aside for later. Running my speech recognition inside a virtual machine was a dealbreaker, but not all that uncommon for people doing this sort of thing. I really, really wanted to get Julius[1] running in OS X but after a couple tries I couldn't get it to build (problem on my end– this is a good reminder to get it sorted out). If you're looking for an alternative to CMU Sphinx that's still FOSS, you really should check Julius out. There are plenty of docs on getting it running with languages other than Japanese. If you're curious about how well it can work, check out this[2] demo (requires Chrome).

[1] http://julius.sourceforge.jp/en_index.php [2] http://www.workinprogress.ca/KIKU/dictation.php

~99% of my time coding is spent working through the stuff in my head

Now if they could optimize that...

Where is it backed up that it's faster than the keyboard?

For the couple of minutes I watched of him demoing it... I type waaaay faster than that. In fact, I can't possibly imagine how I could speak faster than I can code on the keyboard.

(Regular English sentences are another story, but code is full of important punctuation, exact cursor positioning, single characters, etc...)

I mean, this is awesome for people with trouble typing (which was my own case a few months back), but I don't think it needs to be over-sold by being "better"...

Whenever I see posts about voice controlling your computer, I spontaneously think "thank the heavens I don't have to share an office with you." I realize some people work alone, at home or in a sound proof office, but every work environment I've worked in has had a shared acoustic space.

These voice control schemes almost always end up as a cool gimmick, and rarely as a productivity boosting solution.

While I've never been able to adapt to using voice to code, what I have done successfully is use Dragon to document my code. I set up some macros that could move forwards and backwards between methods in Eclipse, added a "start doc" macro...Eclipse does a lot of very smart completion so basic features in Dragon handled it without difficulty.

Dictating your javadoc is pretty damn convenient.

This reminded me of the guy who tried some Perl scripting using Windows Vista voice recognition.

http://www.youtube.com/watch?v=MzJ0CytAsec

I like it a lot. I wish there would be solution to tie this with say Google Glass, and be able to go on a walk or sit in the woods and code or make notes with it, hands free. Or while doing cooking or laundry, etc.

It's unfortunate he couldn't get the OSS speech recognition to work, though.

Reminds me of VimSpeak.

https://github.com/AshleyF/VimSpeak http://www.youtube.com/watch?v=TEBMlXRjhZY

What I think is interesting is that a lot can be done to make typing easier and more human when you can type like you speak (and think).

For example: we say/think

  for each item in list

but in a lot of languages you need to type something like

  foreach(item in list) {

A step further: we say/think

  let a be the substring of b from 1 to the end

we need to type

  a = b.substring(1)

Ofcourse the last example is much shorter and even more readable (to the machine for sure) but maybe code could be a little more human.

That was a fun talk to watch. Someone should try something similar using some kind of brainwave detecting glass gear to make it possible to code by simply thinking. That'd be awesome.

Question (halfway on topic) --

Who makes the best speech recognition software in the world? Regardless of whether it is available to consumers ... who is the best at it?

In particular, how do Apple (Siri) and Google (Google Now) compare to Nuance's stuff? Is Nuance so far ahead of everyone else that they're the clear leader? Or is their codebase "legacy" and vulnerable to better, more accurate software which can be built now due to better algorithms and approaches?

A word of warning -- I started dictating all of my email and Facebook replies on my Android using Google's voice keyboard on my Nexus One a few years ago in response to RSI pain in my hands from overusing my cell phone. Within a month, I started losing my voice.

RSI comes in multiple forms; using your voice exclusively is not going to fix the problem. The trick is to switch things up, which involves having alternatives in the first place.

In the video he mentions that he wish he had known about the previous talk. Looked it up - http://pyvideo.org/video/1706/plover-thought-to-text-at-240-.... Pretty interesting. They are applying court reporter techniques to coding, cutting down on the keystrokes immensely.

Anyone else find speaking commands out loud to distract from thought?

"slap... slap... jog... dot... word... chk... slap... snore"

This is amazing!

If you could speak a bit softer with this, maybe throw in some noise-cancelling headphones, I could totally see this being useful even in an office situation.

I could see a potential pseudo-language developing out of this to abstract a lot of the individual characters, functions and common invocations used while coding.

Okay, here's the million dollar question that isn't on the FAQ and no one in the audience asked.

How the hell did he code it without using his hands? With help?

To his amanuensis: Slap. York. Tork. Jorb. Chomp.

Or maybe he felt his hands going, and he spent the last few months of his pre-RSI existence coding this up.

Here's an open source Python script i wrote a few years ago that allows you to type with your voice. It's based off of CMU Sphinx. The accuracy is almost certainly not as good as Dragon, and it doesn't have a macro facility, so you cannot code as fast as typing. I haven't improved it much over the past few years because my hands got better and i don't need it anymore.

https://sourceforge.net/projects/voicekey/ (tarball, includes language model) https://github.com/bshanks/voicekey (repo, does not include language model)

Hi, I'm the guy in the video. You might also be interested in a presentation I gave last Sept at Strangeloop with a much longer demo of coding in Clojure and Elisp: http://www.infoq.com/presentations/Programming-Voice

There's also this lightning talk http://www.youtube.com/watch?v=qXvbQQV1ydo from PolyglotConf (warning: crappy audio from a shaky cell phone cam).

I promised to release my duct tape code later this year. I'm a bit behind schedule with that but it should be out in a month or two.

There's a lot of potential for multimodal gamified programming using tablets. A combination of gesturing, shaking the tablet, face expression, hand drawing, myo sensing, as well speech, in addition to machine learning in the compiler and for regular expression building. Within the next year a whole raft of apps along these lines will be coming online in the app stores. Big opportunity for Indie developers on the app store, you can easily charge $20+ if they're good and disrupt the emacs/vi/eclipse monopoly/monotony.

This is a cool project, as I think a voice interface would be the ultimate in computing, something like in "2001, A Space Odyssey," or "Star Trek."

I remember first playing with voice recognition and voice command on a PPC Mac back in 1994.

That the technology hasn't progressed along the same lines as cell phones and processors is testament to how difficult voice recognition actually is when dealing with a wide variation of dialect within any given language.

I would love to be able to use my voice as my main input to my computers and other devices.

It's very different for different programmers.

I've worked with people who are skilled developers and who can't even touch type. They have a slow-pace, methodical way of working. Many look at the keyboard over glasses and hunt and peck. The professor who ported plan 9 to raspberry pi (recent video here) is an example of this approach.

On the other hand, I have a shocking memory and can't hold context for long. Sometimes I come to write a piece of code and find that I wrote it last week and can't remember a thing about it.

I work by crashing through. I stalk the problem, procrastinate, drink tea, write short essays about what's stopping me from getting started. Eventually I get the whole problem in my head, and then need to get it down and done before I get tired. When I'm in this state and I need to solve a problem that I could use a standard library function for, often I'll just hammer out code to make the problem go away (list comprehension, string manipulation and the like) in order not to cause any extra load on my short term memory or distraction. Raw typing speed is very important. A drop in pace would hurt a lot.

It's one thing to not be able to write code as fast as you can type. It's another to use a speech to text input method that's designed for long-form prose and try to use it to code. Can you imagine the frustration of trying to enter longCamelCaseVariableNames without a special macro to do so? I don't know the usual commands in Dragon, but I imagine it would be something like: "long delete space uppercase camel delete space uppercase case delete space upper case variable delete space uppercase names", possibly with a few false starts and undos in there as it interprets some of your words as commands rather than code.

To experience something like it, try using your phone keyboard, with word prediction on, to write code. It will be slow, and frustrating, and have a lot of false starts.

There's a big difference between "not the fastest way to enter text" and "so slow it's unusable", and the impression I get is that without extensive macros like this, most speech to text systems are so slow as to be unusable for writing code.

This; if I'm typing and jumping around the code at full speed for long periods it usually means either:

1. I haven't understood the problem

2. I haven't understood the solution

3. I need to spend five minutes improving my Vim macros

No, but the less time you spend going through the mechanical action of translating thought to code the more time you have to focus on solving problems.

It usually happens during refactoring, or when you're doing something you've done before, so you already know pretty much exactly what needs to be done and you're just executing.

It also helps to stop coding for a minute, think what you want to do, then code until you stop typing for more than 5 seconds, repeat.

People say the same thing about learning a good text editor. Personally I find that while I spend most of my time thinking, when it comes time to enter or edit code it helps a lot if I can do it as quickly as possible. That way I stay in flow instead of getting bored and clicking over to HN.

I was never sure which side of this argument I came down on, and then I switched to a Kinesis Advantage keyboard.

I had to slow down my typing for a couple weeks to get the finger positions right. The whole time, I felt like I was coding with a hangover. I felt like I couldn't think properly, just because of the reduced brain->computer bandwidth.

Yep. I only wish my brain had the bandwidth to produce code at the rate my fingers consume it.

It's similarly much less an issue when you map your keys correctly. Control goes to the left of "A", meta below "/". Much less pinky travel. Sun got this right way back in the 80's with the Type 3 keyboard (vi users prefer its placement of ESC too).

Do those keyboards really make a difference? I'd be happy to try them and give it a fair chance, but I'm not okay with a 455 pounds gamble.

The pedals look like a good idea (I'm an emacs user)... but they do seem pretty goofy.

I used a Kinesis for many years and had a great chair and ergonomic setup before developing the RSI that I describe in the video.