A bot disguised as a human software developer fixes bugs

A bot disguised as a human software developer fixes bugs(technologyreview.com)

38 points by mkm416 7 years ago | 5 comments

An interesting prospect, but the patches appear to consist mostly of wrapping a block of code that throws a NullPointerException in a check for null. This is non-trivial -- consider https://github.com/aaime/geowebcache/pull/1/files, where the initializer form has to be broken up and the bot needs some kind of knowledge of data dependencies to know how much code to wrap in the if-block -- but it is also help that humans already receive from machines in the form of lints.

Much more interesting is this: https://github.com/eclipse/ditto/pull/151/files. How did it know to use this.isConsuming()?

Here's the list of PRs it made: https://github.com/Spirals-Team/repairnator/issues/758

I want to take issue with a quote from the paper:

> There is one more aspect to consider. It has been shown that human engineers do not accept contributions from bots as easily as contributions from other humans, even if they are strictly identical [5]. The reason is that humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer.

Even if a machine and a human submit the exact same patch, the human's carries the implicit promise that it was the result of a (more or less :) intelligent process. The human actually thought about it and came to the conclusion that not only did it make the build pass but that it was, taking into account as much context as might be necessary, the right patch to make. If you review a machine's patch, you are obliged to perform the whole intelligent evaluation of whether it is the right patch yourself, which is almost never easier than simply writing the patch yourself.

kapep 7 years ago | |

In their paper it sounds like they only focus on automating the process of analyzing build failures and submitting patches, while the actual repair is done by "[running] different program repair prototypes coming from the latest academic research" to produce possible patches that fix the build failure. It would be interesting to know which tools are used here that are able to produce solutions like adding the check for 'isConsuming'.

There also is a human analyzing these patches and who is submitting the PRs. They say they don't want to waste the developers time, so while it is not explicitly stated, I would guess that they may touch up patches that fix the build failure but would otherwise be rejected because of code quality/conventions/formatting. I could see that automated tools may come up with solution that are equivalent to checking 'isConsuming', but would place that check at other locations than a human would (for example directly where the error is thrown).

krageon 7 years ago | |

For a code review you need to understand and follow the process regardless, or you won't be able to make a good call about whether or not the patch makes sense. Even if you do that, having the code already written is almost always faster in my experience.

quelltext 7 years ago | |

Wait, how is the code in the first link passing definite assignment analysis checks for `params`? The Java compiler is intentionally very restrictive about this (even if by means of more elaborate analysis definite assignment could be proven).

quelltext 7 years ago | | |

My hunch is that it doesn't, so the fix was actually wrong.

The file here has the necessary initialization for params for this to work: https://github.com/GeoWebCache/geowebcache/blob/master/geowe...