AI chatbots were tasked to run a tech company

AI chatbots were tasked to run a tech company(businessinsider.com)

4 points by kmote00 2 years ago | 5 comments

kmote00 2 years ago |

Summary: Researchers in a new study tasked an AI-powered tech company (nicknamed "ChatDev") with developing 70 different programs. They found AI could develop software in under seven minutes for less than $1 in costs, on average. AI bots were assigned roles and were able to talk, make logical decisions, and troubleshoot bugs.

kmote00 2 years ago | |

Original paper: https://arxiv.org/pdf/2307.07924v3.pdf

az09mugen 2 years ago |

"The study said 86.66% of the generated software systems were "executed flawlessly.""

What's that susupposed to mean ?

Unit tests were written to have this percentage ? By human or AI itself ?

86.66% of how many LOC ?

How long will it take for a human (because AI can badly do) to debug the code ?

What was the purpose of generated code ?

Lack of useful context here. Maybe I missed information fast-reading the article.

IMHO, it looks like just another rant on how "good" LLMs can "write code".

EDIT : Sorry OP, I didn't see the arxiv link.

I'm on mobile and read a 10 Mo pdf isn't worth it. I'll try to read it on a computer though, looks interesting.

az09mugen 2 years ago | |

I don't want to undermine the work of the researchers because they did a good one with a solid protocol.

But the problem to me is the specification, basically one of the tasks was : "design a basic Gomoku game". Which implies a set of pre-existing rules, something already known and well documented on the web. For example on github you can find 1.2k repos in python with the tag 'Gomoku'.

In a company you will be less likely asked to code a Gomoku from scratch, maybe for an interview, at most.

So in my POV, it's just code synthesis from already existing stuff. And reviewed.

I think my job is safe.

archo 2 years ago |

https://archive.is/ku2xx