Ask HN: Best tool for autonomous quick coding assessments?

6 points by marttilaine 1 year ago | 5 comments

We place 100s of software development students in internships in the Nordics, and are looking for a way to perform a basic coding skill assessment before matching an intern with a company.

Unfortunately the schools submit to us some students who cannot write code and who will claim otherwise in their written applications, and we find out about this at the latest when the intern is in a real project and unable to perform at all.

The goal of this light-weight assessment would be to simply answer the question: *can this intern read an assignment and proactively write some code?*

So the assessment should have: - a built-in editor - a time limit, like 1 hour - detecting the use of AI / massively copy-pasting from another window to the editor - live preview of the result, e.g. a React UI the intern completed based on instructions

What we don't need/want: - live interview scenario with a supervisor - a full interview (just one assignment per student)

Any suggestions of tools to use? Experiences?

I just submitted for a demo with Coderpad but don't really know where else to look.

vunderba 1 year ago |

Short of an actual on-site interview, there's zero chance of detecting LLMs unless the prospective applicant is completely oblivious.

y34h 1 year ago |

I also thought about coderpad, I have seen some codingame community challenge results and liked the report.

android521 1 year ago |

at this point, it is not possible. Whatever you develop, the student can just use another phone /tablet to take a screenshot and upload it to llm to generate solution and then he/she can just type it out.

999900000999 1 year ago | |

True, but if someone is smart enough to figure that out over just putting in code that doesn't work, they're at least trainable.

Most jobs are starting to push AI coding anyway. Your expected to use Chat GPT/copilot, but your ultimately responsible for understanding how that code works.

As long as they aren't literally stupid enough to try to open up another tab and just paste the code in, I would still look at someone typing in a solution they memorized/ got from an LLM as promising.

I don't really see a difference between a person who has the ability to read through cracking the coding interview, and memorize every solution, to regurgitate them, versus using an LLM.

And even if someone can pass your code exam that doesn't mean you have to interview them, I actually feel a bit weird because I had a code test recently, which I was able to pass. But since it was a very brute force solution, and I wasn't able to optimize it I guess they decided to not interview me.

tacostakohashi 1 year ago |

Check out HackerRank?