Take the 2-minute tour ×
Personal Productivity Stack Exchange is a question and answer site for people wanting to improve their personal productivity. It's 100% free, no registration required.

I'm looking for ways to augment mouse and keyboard control with another form of input.

We use Dragon to trigger macro expansion. For instance, to create a voice command to write a loop template in Python, we can use the following Advanced Scripting command:

Sub Main
    Clipboard("for i in range():" & vbCrLf)
    SendKeys "^v{TAB}"
End Sub

To return the variable that is in the clipboard in Python:

Sub Main
    originalClipboard = Clipboard
    Clipboard("return " & originalClipboard)
    SendKeys "^v"
    Wait(1)
    Clipboard(originalClipboard)
End Sub

enter image description here

To add a title and a legends to a figure in Matlab:

Sub Main
    Clipboard("title('')" & vbCrLf & "xlabel('')" & vbCrLf & "ylabel('')")
    SendKeys "^v"
    SendKeys "{Enter}"
End Sub

And so on. I'm looking for any information (tips/ideas/libraries/etc) to use Dragon NaturallySpeaking to code more efficiently.

share|improve this question

closed as too broad by AsheeshR, Raystafarian, THelper, Rory Alsop Mar 21 at 10:30

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

3  
Although this wasn't exactly your question, there is some research into speech dictation of code, for disabled people, but it's not mature. See e.g. research.microsoft.com/en-us/um/people/abegel/qual/node16.html. See also related question programmers.stackexchange.com/questions/47933/…. –  Derrick Coetzee Jul 16 '12 at 1:27
    
Thanks a lot for the links! –  Franck Dernoncourt Jul 16 '12 at 1:47

5 Answers 5

Alexandria Carstens, who is a speech recognition (Dragon) trainer & consultant in Vancouver, gave me a couple of advice:

Any or all of the following techniques:

  • Start with a blank vocabulary and add required verbiage to it (definitely take advantage of written form and spoken form).
  • If also require a fairly basic vocabulary as well, start with a small vocabulary, and add required verbiage to it.
  • Make use of text commands for repetitious text.
  • Make use of computer control commands (i.e. step-by-step, Advanced VB scripting) for speeding up processes such as compiling etc.

(if you are a Dragon NaturallySpeaking user, you will probably be interested in her website which gives useful advice)


Also, here is an interesting video/demo on programming by voice (using Dragon NaturallySpeaking + DragonFly): http://pyvideo.org/video/1735/using-python-to-code-by-voice

Summary

I dictate my code using a voice recognition system with Python embedded in it. In a fast paced live demo, I will code a small system and deploy it without touching the keyboard. I hope to convince you that voice recognition is no longer a crutch for the disabled or limited to plain prose.

Two years ago I developed a case of Emacs Pinkie (RSI) so severe my hands went numb and I could no longer type or work. Desperate, I tried voice recognition. At first programming with it was painfully slow but, as I couldn’t type, I persevered. After several months of vocab tweaking and duct-tape coding in Python and Emacs Lisp, I had a system that enabled me to code faster and more efficiently by voice than I ever had by hand.

The source code for voice commands isn't online yet, it should be released in June or July on https://github.com/tavisrudd?tab=repositories.


Another nice resource: Can I write SQL using speech recognition?

share|improve this answer
    
Is this up to date? Source still "not online"? –  SamtheBrand Apr 11 at 6:21
    
@SamtheBrand Did my previous comments get deleted? –  Franck Dernoncourt Jun 3 at 20:03
    
were there previous comments here? (I vaguely do remember this being the case.) –  SamtheBrand Jun 4 at 1:27
    
@SamtheBrand Actually I see I replied in the comments following Alex' answer, that's why I got confused :) –  Franck Dernoncourt Jun 4 at 2:50

I think the power and potential of voice activation tools lies in using spoken word to trigger actions and events, but not to replace typing.

For example, if you can configure a voice recognition package to trigger commands like "open", "save", "build", "run", etc., then you can save yourself time hopping through menus or typing hotkeys.

However, I wouldn't recommend putting any effort whatsoever into using voice recognition to actually generate your source code keyword-by-keyword. Unless you are a very fast speaker or a very slow typist, you probably won't be able to increase your speed, accuracy, or overall efficiency much by this method. Imagine having to say "dot", "open-bracket", "semi-colon", and so on every time you need the special markup characters that are so much more prevalent in programming than in prose. Without a highly specialized approach (one that will take much more effort than tweaking a package like NaturallySpeaking), I doubt you will increase efficiency.

One of the most promising areas of increasing efficiency as a programmer is in avoiding context shifts. For example, doing as much as possible without removing your hands from the keyboard (using hotkeys instead of mousing) helps programmers stay focused and effective. A voice activation system might be effective if it reduces time using the mouse, exiting your active program, but trying to replace typing with speaking does not sound likely to make coding any easier.

share|improve this answer
    
Thanks! Reducing mouse usage is indeed a key to increase productivity: I am using SmartNAV to move the mouse cursor (superuser.com/a/435266/116475) and I am currently looking for a device to perform mouse clicking without using hand/foot/voice (quora.com/Computer-Hardware/…). –  Franck Dernoncourt Jun 25 '12 at 20:03
1  
For mouse clicking, an option you might want to consider is AutoHotkey - it's a system that allows you to write simple scripts including hotkeys that replace some keystrokes with others. You should be able to find a hotkey that you can map to send left-click, right-click, etc. –  asfallows Jun 25 '12 at 20:15
1  
In fact I'm already using hotkeys to emulate mouse clicks, as well as foot switch, voice recognition and dwell clicking. But I would like to do it without using hand/foot/voice. I am therefore looking into brain computer interactions and eye tracking. I know there is also mouth-operated mouse, I have never tried this but it sounds pretty intrusive. I am not aware of any alternatives but I would love to hear of other devices if they exist. Maybe I should open a question on the matter here. –  Franck Dernoncourt Jun 25 '12 at 21:07

I wrote a system using a similar approach to Tavis', and have been using it for a few weeks. Basically it runs Dragon with NatLink and Dragonfly in a VM and executes the actions on my Linux system. It's definitely still duct tape coding, but I'm quite happy with coding in it. https://github.com/calmofthestorm/aenea. Many of the grammars should be of use with just Dragonfly on Windows with only minor changes.

Am I faster dictating than typing code? No, but I am faster with a combination of voice and keyboard, and at any rate I spend enough time thinking anyway that a small slowdown at code entry due to voice doesn't have much of an effect on overall productivity (and I'm sure with more work I can improve)

Tristan Hayfield also recently posted about a similar project on the Dragonfly group: https://github.com/TristenHayfield/damselfly

You may also want to take a look at VoiceCode; I played with it a bit and while it wasn't really what I was looking for it's a very interesting project.

There's definitely a lot of promising stuff floating around, but I feel like at present you need to be prepared to do some substantial customization and hackery of your own in order to get a system working you'll be happy with.

I really hope Tavis releases his code soon. That video was astounding and I'm still not sure how he does some of those things (like that list comprehension).

share|improve this answer
    
Hey great answer, but which of Tavis instances impresses you most. Why? And can you link to a particular Tavis project that you're talking about? –  SamtheBrand Apr 11 at 6:19
    
@SamtheBrand The source code for voice commands isn't online yet, it should have been released in June or July on github.com/tavisrudd?tab=repositories but from what I can see it's yet another conference's vanished promise... Btw do you know to which SE I can migrate this question? It got recently closed on this SE, and my last question that got closed here was deleted afterward without any warning, with no respect at all for user created content. Otherwise I'll take everything to Quora. –  Franck Dernoncourt Apr 11 at 15:23
    
Hi @FranckDernoncourt. I believe this is the right site, though I suppose Programmers could host it as well. I think your best bet is to flag it for reopen. Give that a try? –  SamtheBrand Apr 11 at 15:26
    
@SamtheBrand OK I'll give a try, we'll see what happens... in the meantime I wish there was some application or web service to alert me whenever one of my StackExchange question is put on hold / gets on closed. –  Franck Dernoncourt Apr 11 at 15:35
1  
@SamtheBrand 3 days after nobody else than me cast a re-open vote. I think that barely anyone is interested in voice recognition here, and I think something else but that would be banned as a comment. –  Franck Dernoncourt Apr 14 at 15:48

I've used it extensively for a code type task, in the end I had many macros that would save me 10 or 20 mouse clicks and moves! It would take me a bit less time and I found it extremely convenient.

It was for coding, something called reaktor DSP environment. I know the program pretty well. I do believe that it would be EXTREMELY efficient at coding.

What I think you need to do though, is not use any normal text. Just make a specialised command mode that doesn't interpret any normal phrases, and copy into it about 2000 words that you use in programming; "foreach, var, Vector3()" etc.

You'll need to make some commands like logical if if(){}

loop ALPHA to loop a++

After that you will be coding 1 language A LOT faster I think, unless you use predictive code typing which is very fast anyway. But you will have to train it as a command mode only, ignorning normal prose text and teach it a mass of code words. I will do this myself soon because my hands need a rest from coding for a week or 2. It will be a safe solution for me.

Remember, Dragon naturally speaking PRO is the only serious solution for making a very specialised command mode. You will be able to add commands like reference search to open a browser and read refs about a highlighted text, etc. Premium version is limited.

share|improve this answer
    
Thanks for your answer! Just 3 questions: 1) Can we create a custom mode in Dragon NaturallySpeaking? 2) How do you deal with variables (how do you call them) ? 3) Dragon NaturallySpeaking sometimes add automatically spaces: how do you avoid this (e.g. if you say "variable name open bracket" there might be a space between the variable name and the bracket, which you probably want to avoid)? –  Franck Dernoncourt Oct 11 '12 at 1:42

More efficiently in what sense?

IMO voice input for programming is substantially less efficient than typing, particularly in an decent editor with good macro and template facilities. I think you'd be better served figuring out the domain(s) you normally work in, figuring out what code gen tools you already have access to or can create, and nail down every possible keyboard and template efficiency you can find.

Programming vocabulary is specialized and focuses on a different set of terminals than "normal" speech does. Context- and language-sensitive editors have access to the AST of whatever language, environment, and framework you're working on--Dragon and similar voice input software does not.

This is something I've played with every couple of years for over a decade now, and it rarely takes more than an hour or two before I give up in disgust. The abstractions we deal with as developers simply don't map to the English vocabulary. Even assuming voice macros I have yet to see a system, or combination of systems, that doesn't make me want to punch my monitor in the face.

share|improve this answer
    
Thanks for your feedback Dave! What I meant by efficiently is having the program done as quickly as possible. –  Franck Dernoncourt Jun 24 '12 at 22:31
1  
@FranckDernoncourt Voice just doesn't cut it then, AFAIC, although some languages may be more amenable to voice than others. IMO not worth it at all, at least not yet. –  Dave Newton Jun 24 '12 at 23:18
    
Maybe Dragon software is much better than Window's Voice recognition software.. but I can hardly get Windows Voice to do things normally! I imagine it would be hell to try and code with. –  Gaʀʀʏ Jun 26 '12 at 1:33
1  
@le_garry Voice recognition is pretty good these days. I used it for this reply, in fact. But for curly-brace languages, no thanks. still, I could see it being useful for higher level language is perhaps. just not yet. –  Dave Newton Jun 26 '12 at 1:40
1  
Voice recognition works extremely well. Demonstration in French: youtu.be/FODUScE6-Ek (hopefully I'll be able to do the same soon in English). Here is an excerpt of an e-mail I received today: "amazingly after just the intro session with dns12 (on dilbert) he could read a paper abstract with heavy scientific jargon and get 99% accuracy (two words missed)!". Google voice recognition has also become very good. –  Franck Dernoncourt Oct 11 '12 at 1:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.