Implementing GPTZero from scratch – Reverse engineering GPTZero

Implementing GPTZero from scratch – Reverse engineering GPTZero(github.com)

99 points by burhanultayyab 3 years ago | 18 comments

We've gone through the original implementation of GPTZero and successfully reverse engineer it. (it gives the same results as original GPTZero). We've also recorded the implementation process which can be found below.

Youtube Implementation Video: https://youtu.be/x9H-aY5sCDA

Github: https://github.com/BurhanUlTayyab/GPTZero

Website: https://gptzero.sg

Discord: https://discord.com/invite/F3kFan28vH

We're also working on a GPTZerov2 (inspired by LLM based transformers and GANs), which would be more accurate, and can detect lines changed by humans.

Please give some feedback on our work.

Thanks

forgotpwd16 3 years ago |

>our implementation gets the exact same results

Considering https://news.ycombinator.com/item?id=34556681 that isn't really a good thing to have.

That said, good work replicating it.

burhanultayyab 3 years ago | |

Thanks for the complement. We would be more cautious in the future.

rvz 3 years ago |

Good. A detector should have been written and open-sourced before releasing GPT in the first place, especially by OpenAI themselves. A shame that someone else had to do the work other than them.

Better late than never to open source it now, unlike the original creator of GPT Zero or OpenAI doing it.

cwmoore 3 years ago | |

They would be in a position to log hashes of generated substrings, multiple of which in a document would indicate a high likelihood indicator of the source. Solving for the Turing Test appears to have been a poorly chosen goal.

burhanultayyab 3 years ago | | |

You are correct, that's why we're working on a version 2 of it where we're using LLM Transformers fighting adverserially to make it better.

camjw 3 years ago |

A bit weird that you claim 100% accuracy to the real GPTZero when most sentences I tried gave different results.

Also, kind of cringe to copy the original GPTZero down to the domain name - you are gptzero.sg right?

alexcdot 3 years ago |

the model at https://gptzero.me is different now, it does sentence-level prediction as opposed to the old GPTZero

disclaimer: i'm familiar with creator

burhanultayyab 3 years ago | |

It's amazing what you guys are working on :)

We're working on v2 where two large LLMs fight with one another.

sdrg822 3 years ago |

Given that GPTZero was an undergrad’s side project it’s not that surprising?

bablawexler 3 years ago | |

where can I see your code if that’s so easy

nicholaswork 3 years ago |

Interesting and the website is more responsive than the original.

forgotpwd16 3 years ago | |

Complementing the project in which you're one of the two devs is a bit strange imo.

https://github.com/BurhanUlTayyab/GPTZero/commits/main

dboreham 3 years ago | | |

Why?

burhanultayyab 3 years ago | |

Thanks for the complement.

nacs 3 years ago | | |

Do you/your dev-partner give each other compliments like this while developing too or just at the end?