Generating code from natural language is closer than you think(blog.stephenwolfram.com) |
Generating code from natural language is closer than you think(blog.stephenwolfram.com) |
I teach computer science and have a particular fondness for introductory CS. The reason Stephen Wolfram is wrong, wrong, wrong about this is that people that have never been taught programming can't express themselves precisely enough in their native language, either; and even among those of us that have been programming for decades, when we express ourselves in natural language we can be very precise but it takes a lot more work and becomes a lot more unwieldy than just writing out our instructions in [pseudo]code.
CS educators have been wishing for a long time that "intro to CS" didn't equate to "intro to programming". And it doesn't have to, not quite, but the reason it always seems to revert there is that the prerequisite for every other thing in CS is, not programming itself, but a certain precision of thought that is easiest to teach just by teaching students to program. In a programming language. Because if you try to make them write out instructions in a natural language, and you notice that they aren't being precise and therefore deliberately misinterpret that instruction, they just think you're being a dick about it. They sometimes even think this if you honestly misinterpret them. (This is true even in a non-CS context.)
Saying that we will soon be "generating code from natural language" is, at best, misleading. It implies that people who couldn't learn a programming language will be able to program, which is quite untrue---I promise that with the possible rare exception of a few pathological edge cases, when people can't learn to program, the language is the least of their problems. And for those of us that can and do learn programming languages, all but the simplest sorts of programs will probably be easier to write in a programming language (which was designed for that sort of thing) than in a natural language (which was not).
(And holding up Mathematica as an exemplar is particularly egregious; it is so loaded with syntax that "just works" that you need to either have a deep familiarity with traditional mathematical notation or else a degree-level CS background in programming language theory if you want to have a good shot at learning the language in anything more than a pattern-matching fill-in-the-blank way.)
With the interactive nature of this thing people are going to learn if they want, just as people learned to type grammarless queries into google.
I don't think the point of this is for random-street-smart-kid to program complex systems seamlessly, it's more for random-physics-loving-student to try out things he still doesn't understasnd, and stuff like that.
Hell, there are many programmers who code things this way - try it, change it, repeat. Natural language would just allow non-programmers who aren't familiar with the syntax to join in on this process. Continued practice would likely improve the reasoning abilities of the user and make the process faster.
Thank you, you have just pointed out a likely cause for my improvement in writing scientific papers. I learned how to program.
You already see this in places like hacker news here where people often use constructs like "s/thing/other thing/" because it's more concise and useful than writing out the natural language version.
"thing" → "other thing" is the same length, and while not natural language, it isn’t a computer language.
I think using the sed-like (right?) language is more useful as a signaller. Check it out, yo, I grep shit all the time.
X->Y makes just as much sense, but the s (for "substitute") makes it mnemonic - I read it as "sub X for Y".
Stross is also an occasional commenter here on HN.
Sure, you eliminate the first big hurdle in programming, but learning the syntax of programming language is usually one of the easier parts of software development.
The advantage of natural language over a high level programming language would appear to be analogous to that which a high level programming language has over assembly.
I think you may be conflating programming with software development. People still develop software in assembly language, but few people use it in lieu of javascript on the web.
The inherent problem with NLP is that human languages are imprecise and ambiguous. For an example of the problems with NLP, try using Wolfram Alpha. While it returns useful results for many queries, as soon as you start off the beaten path it can become an exercise in frustration as you try to figure out exactly which format of words it will accept, especially as I know the mathematica command that would accomplish the desired goal (or can look it up quickly).
Obviously Alpha isn't the end-game of NLP and improvements can be made to accept more constructions. But ultimately, you're going to have to restrict yourself to a subset of your natural language's syntax, grammar and vocabulary, and I believe that learning what that subset is is far more difficult and frustrating than just learning a programming language.
Furthermore, merely knowing a NL programming language isn't enough to be a programmer--you still need to learn how to think logically and algorithmically, which seems to be the hard part for people new to programming.
Yes, but once you have to say "Draw a circle in this shade of red, give it a black border this thick, make it this big and put it at these coordinates" things are starting to get unwieldy. After a few different circles I'd probably start longing for draw_circle(fill_colour, radius, (x,y,z), border_colour, border_width).
Edit: Added function arguments to make the comparison less biased.
CLS
SCREEN 12
CIRCLE (200, 300), 50, 4Winograd: https://hci.stanford.edu/~winograd/shrdlu Wolfram: http://blog.stephenwolfram.com/data/uploads/2010/11/conesphe...
After a few minutes of thought, the biggest benefactor of natural language coding would be what I'd call late-bloomers: people who always thought like coders, but never actually did programming.
Maybe he'll call it 'A New Kind of Programming Language'.
Natural language, however, is conversational. You say A, the person you're talking to interprets that as Z and asks for clarification, you explain the difference between A and Z, now the other person thinks M and asks more clarifying questions, etc.
Computers are fully capable of working conversationally, instead of specificationally; it just requires that instructions be stored in a form that's a bit more complicated than a linear tape (e.g. a database of constraints, like Prolog.)
* Read/eval(uate)/print loops
Prolog (also Erlang and some others) is mixed; you reload a file of rules read as a whole, but can easily prompt the system for easy testing, and reloading is very fast. It's very convenient with a Prolog shell terminal and a vi window, two buffers in Emacs, etc.
There's a reason physicists express their concepts in mathematics, and that's because math is the language humans devised to express those things, having found natural language inadequate.
Programming is similar in that regard.
Why not just create a DSL (e.g. in Scala) with a simple standardized NL-like syntax that can give meaningful "compile errors". There is no need to impress the general public.
1. I'm not sure what problem so-called natural language programming is trying to solve.
2. Though I admire this man's building of Mathematica and the company that sells it, I'm generally not a fan of what I perceive as his history of "discovering the obvious" and self-promotion. Or rather, rediscovering or making things sound like he invented them or came up with them for the first time. Cellular automata and it's implications in his book "A New Kind of Science", and now this piece sounds like more of the same. I give him a little slack because he's in business and so there's the self-promotion angle, but not much.
Secondly, I think eventually programming will have to become more mainstream. And this will be done through some form of a natural language interface. Programming is best a tool to solve problems. It's way too limited now where only an elite group gets to control its use.
Eventually programming will have to become a tool as common as mathematics. The problems we face are only going to continue to grow in complexity where the only way to get a handle on them is through automated computation. The user will need the instant, iterative feedback that only a self-made program can provide. For this to happen, the interface to programming will need a radical change.
Furthermore, I'm not talking about your average office assistant. I'm talking about professionals in a science-related field. Being able to use programming as a tool is going to become invaluable.
But would you call using a text adventure (put blue ball in red box) programming?
Here's the actual source code of a game: http://inform7.com/learn/eg/bronze/source.html
Of course, it's really a precise programming language that happens to read like English; the same text might not work if it were rearranged into semantically equivalent English.
We can create a database with the different phrases that people use to suggest a command, for example:
Calculate the limit, find the limit ... all of this is Limit, and so on.
To sell a product, this kind of ability is well received. Is like telling to your telephone: Please call this number for me, the number of my friend Alfred. And the computer lookup Alfred in its database and connect with that number, that is not deep IA but is a useful trick for selling products.
But for most people unfamiliar with Mathematica syntax, typing, "draw a red circle" and having the computer choose sensible (or any) defaults yields a template of the exact code one would have needed to type. Which saves Googling or reading the manual for circles, and teaches the syntax in a very natural way.
"A natural language query system for a prolog database" (Hadley) http://ir.lib.sfu.ca/handle/1892/7254
Not general code generation I suppose, but there's been quite some work on natural language to database queries systems in AI research elsewhere.
Which by the way was a stupid premise but hey :)
A non-programmer cannot write code, neither in English nor in programming language.
What's easier, "5 + 10" or "five plus 10", or even worse "five added to ten"?
Dijkstra had an article about that, titled: On the foolishness of "natural language programming"[1].
[1]: http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EW...
Result: Affirmative, Dave. I read you.
Now how many of us have seen a specification that was unambiguous, complete, and efficient? Have a look at the trouble that progammers have implemented browsers capable of passing Acid 3. And that's for a special case where the spc writers are programmer themselves, and have even provided a set of programmatic test cases to test validity.
Or alternatively, how many times have you tied to help a friend do a complex computing task (automated backups, transferring an iPod from one computer to another, whatever) and you start asking questions to find out exactly what they want, which tradeoffs they want to make etc, and they just get frustrated with you?
Those are three major flaws with natural language programming
- not even humans can correctly compile natural language specs
- it is incredibly difficult to write a natural language spec that is correct, unambiguous and efficient
- people already dislike dealing with the complexity of computer programming, even when they have a programmmer holding their hand, asking all of the right questions.
Natural languages already have tokens, syntax, and grammar whereas visual fields do not have them. All those elements must be imposed onto a visual language before it can be translated into the machine language (e.g. there is no obvious convention for visual commands or visual conditionals).
Note in particular the differing levels of what you might call "eventual feasibility" for each of those things... but none of that changes the fact that they are all demoware, and another demo isn't really all that intrinsically interesting until they push it somewhere actually useful, and thereby, new. Demoware isn't necessarily a permanent status, but another demo is not sufficient to escape.
I don't know about that. Within a particular domain, you can certainly come close to representing these things, or presenting the right controls for a user to represent commands and conditionals. For an example, check out QuickFuse http://quickfuseapps.com
Before we built this, we thought about common ways people "drew" voice apps, and commands and conditionals both have certain natural visual representations in the "voice app" space, e.g., blocks with branching arrows and writing text to indicate spoken words.
http://strlen.com/aardappel-language
(There are otheres there, Wouter did some extensive research)
The arbitrariness of the mapping between text tokens and graphic tokens quickly increases beyond convention the ability to establish a convention (example: "Number of cheeseburgers shipped from Peoria to Toledo last month."
x->y suggests lambda to many people in our community.
Edit: I meant to provide some additional information to other readers, not to disagree in any way.
draw_circle(...)
circle_draw(...)
circle.draw(...)
Graphics::circle.draw(...)
shape_draw('circle',...)
or any of the million ways to ask the computer to draw a circle.
and you don't know if you need to do:
import graphics.package
#include <graphics.h>
use graphics.io
...
Just witness the myriad ways there are to tell the computer to print some text to standard out (echo, print, write, cout, ...), and the myriad ways to formulate a conditional or a for loop (e.g. see http://rosettacode.org/wiki/99_Bottles_of_Beer)
If the compiler/interpreter had at least some level of intelligence we could just say "draw a circle" or "output this text" and the compiler/interpreter would know which function to call, which libraries to include, and the exact syntax to use.
EDIT: Though, now that I think about it, this may be a great add-on to IDE's or text editors like Emacs, instead of having the compiler deal with it. Emacs knows which programming mode you are in, and you could just type 'M-x code: draw circle' and it inserts the appropriate code, depending on the language.
* "I set the brake up by connecting up rod and lever."--Yes, given the whole of the rest of the mechanism. Only in conjunction with that is it a brake-lever, and separated from its support it is not even a lever; it may be anything, or nothing."* -- Wittgenstein
Think of playing a text-based adventure game and how frustrating it can be trying to work out the exact phrase the designers had in mind. To make this work you'd essentially need a machine that could converse with you to build the program - the way that an engineer talks to a client to draw up a specification - which would be very cool.
I like your idea for an editor add-on. Snippets taken one step further.
Draw a 300 pixel DarkRed circle with a thin black border centered in the blue square.
Wolfram's point is that conventional languages won't get your circle to that location as easily, e.g. if the blue square is named arbitrarily and its color is bluish but not literally <color>blue</color>.
I'm not suggesting that high-level languages aren't useful, but the reason people often storyboard designs with felt-tips and tracing paper rather than Photoshop is to avoid interrupting the flow of thoughts with the task of translation.
Inform is an exapmple of a natural-language-ish, rule-based system (for programming text adventures), that could efficiently re-declare rules in a REPL (if not for its basis in virtual machine image formats that expect to be compiled from complete specifications.) Inform guarantees efficiency by using a hub-and-spoke system of rules: rather than every rule having the possibility to interact with every other rule, rules can only interact with rules in their own "rulebook" (module), the core rulebooks (standard library), and the "meta" rulebook (monkeypatches to re-specify libraries.) Thus, integrating a new definition only takes O(k + n + e) time—where k, n, and e should all be small—rather than O(n^2). This works well for Inform, but I'm not sure whether it would be as effective in a general-purpose programming environment.
* Case in point: Loading a file is "consult"; I assume this is historically because Prolog was originally a language for doing NLP in French. (See e.g. HoPL-2.)
If there were an obvious visual convention, the start button would not need to say "start" and the hang up button would not need to say "hang up."
That's not to say that arbitrary visual conventions can't have great utility (the alphabet being a case in point). But to illustrate the issues with graphic conventions, Quickfuse does not use the long established conventions for flowcharting. It uses natural language instead.
Also, you'll find that that where we departed from conventions for flowcharting, we did it to save pixels, or make the UI more accessible. There is a tradeoff in visual programming between ease of editing and clutter--compare with Max MSP, which has a stark UI, at the cost of having you memorize certain textual commands. If you draw a bare flowchart, it's not obvious how to manipulate it until you draw other GUI controls on top of it or make some modifications. However, it is the general paradigm we are tapping into.
The context of my initial context was in response to "fully visual programming" in the ancestor comment and the implication that natural language programming faced similar challenges.
The expediency provided by labels is a result of the arbitrariness of graphic interpretation. I am not suggesting that the use of labels isn't helpful, only that the use of labels doesn't really differentiate "visual programming" as a subset of programming, i.e. in and of itself a text label is not significantly more visual than text on a terminal screen.
For example, the label is necessary because the visual convention for "start" is ambiguous. Left, right, top and bottom are all used as a starting position depending on the arbitrary conventions of the context. Likewise, a MaBell styled receiver, green flag, a vertical stroke or a hand with index finger extended as if to press a button may all be used to indicate start.
I used your departure from flowcharting conventions as an illustration of the unique problems with graphical communication conventions. I am not implying that deviating from flow charting convention is a bad idea.
My point is that your deviation from flowcharting conventions is arbitrary in the sense that it was driven by factors irrelevant to the process of flow charting (i.e. the limitations of the medium on which the flowchart is presented rather than concerns about the mapping of graphic symbols to processes).
Draw a circle.
Color it red.
Make it smaller.
No, 5% bigger.
Make the radius 5 units.
circle = Circle(); // default origin, radius, color, etc.
circle.set_visible(true)
circle.color = RED
circle.make_smaller()
circle.set_radius(5)
And get rid of the set_visible.
"I have to say that something I thought would be a big issue is the vagueness of natural language. That one particular natural language input might equally well refer to many different precise programs.
"And I had imagined it would be a routine thing to have to generate test examples for the user in order to be able to choose between different possible programs.
"But in reality this seems to be quite rare: there is usually an “obvious” interpretation, that in typical Wolfram|Alpha style, one can put first—with the less obvious interpretations a click away."
When those things matter, more information will be required in both natural language and a high level programming language. On the other hand, when none of that really matters a high level programming language still requires all those items to be specified.
Like any level of abstraction, there are tradeoffs. On the other hand to explain how to submit a reply, I don't say:
<form method="post" action="/r">
<input type="hidden" name="fnid" value="7YwrNY6bwe"/>
<textarea name="text" rows="6" cols="60"/>
<br/>
<br/>
<input type="submit" value="reply"/>
</form>
edit: html example to fit natural language model.
* http://images.google.com/images?q=enso
(The results for "bring me some delicious pho" are similarly unsatisfactory - it just tells me that delicious means "greatly pleasing or entertaining". Not news to me, pal!)
http://www.wolframalpha.com/input/?i=draw+a+green+filled+red...