We're finally seeing that all of the answers to our company's top-of-the-funnel coding screen are nearly identical. It's fine that people are using LLMs, but it's clear that the value of our current screening questions are diminished. Once you have a small pool of candidates, you can interview them in person, ask questions that deal with ambiguity, etc. There are many opportunities to deconvolve the human from the AI.

But, what techniques are people using for automated screening to evaluate technical/programming skills?