AI Has Broken Hiring(hbr.org) |
AI Has Broken Hiring(hbr.org) |
Maybe the relentless pursuit of "efficiency" at all costs has broken the world?
I remember when I applied for my first job. I got dressed up and my mom drove me to the interview because I didn't have a driver's license or car at the time. It wasn't "efficient" for me and I suppose it wasn't "efficient" for the company but much to my surprise, I got an offer and that was my first "tech job"...before tech jobs were cool.
It's very strange that the authors talk about how "making a bad hire is terribly expensive" but then call out "travel time and costs". Well, if B < A for each role filled, is it really so bad?
And yeah, I get that huge companies like Google and Facebook hire from around the world and not everyone is located in close proximity to Mountain View and Palo Alto, but that speaks more to the oligopolistic world we're living in than anything else.
If a small number of companies weren't distorting the labor markets, this might matter less.
A friends' company has even ended remote hiring altogether after auditing their remote hires and discovering a lot of connections from countries they didn't expect.
There's even a growing scam where people get recruited to lend their identities and bank accounts to someone else to get the job. Then they're asked to install some software on the company laptop and leave it open and powered on during the workday so someone can operate it remotely. Remote work is wild right now.
The cost of a bad hire they're referring to includes things like opportunity cost of not having a good hire in that position, damage they've done to the product (codebase, design, etc.), and second-order effects like demoralizing the rest of the team.
The actual hiring costs of a bad hire are a rounding error compared to the damage they can do.
Have you ever been on a team that was great until they hired one wrong person who made every work week a miserable slog? Attrition goes up as the good employees start to leave. The codebase starts accumulating a lot of tech debt. Even after they're gone it can take a long time to recover.
This is why it's so important to be able to fire fast, but that's another topic rife with difficulties.
If the cost of a bad hire is huge (which I agree it is), why is the hiring process optimized, in part, around reducing the travel costs? It would seem that these costs are modest in comparison.
In the Olden Days [tm], jobs were advertised through recruiters, physical media (eg the paper) and connections. You had to review applications and conduct interviews. The cost of applying was relatively high, your reach was relatively low and the investment per applicant was relatively high. So imagine that there were enough jobs for everyone in a simplified model. 10 people applied for 10 jobs. It's not the same 10 people for each job. But there was a decent ROI on effort. It kinda just worked.
Fast forward to now and the cost of applying is essentially zero in terms of registering interest and submitting a CV. And you apply for a lot more jobs so instead of 10 applicants for 10 jobs, you have 200 applicants for 200 jobs. Still the same applicant to job ratio but way more inefficient for everyone involved. Applicants can't put in the same effort for 200 applications that they did for 10 and employers can't review 200 applications the same way they did 10.
So what happens? Employers, who have the power in this relationship, put up roadblocks in the name of efficiency. Now you have to survive ATS before ever going in front of a human. That ATS uses inscrutable logic that may filter you out for not including enough keywords or some other specious reason. You now have hiring assignments.
The net effect is that an applicant puts in 200 applications, get automatically filtered out from 180 of them and then has to do upwards of 20 take home assignments.
Plus there are more and more layers added. More rounds of interviews. Phone screens. Remote interviews then on-site interviews. All of this wastes time and, like you allude to, I don't think it's effective. But it's a natural response to the illusion of choice.
Let me give an online dating analogy. In years gone by, you'd rely on meeting people in person. Now, less so. And speaking in a strictly heteronormative sense, how it tends to go is that women on average have hundreds of choices and men have on average far fewer. A gender imbalance plays into this.
So what does a woman in this scenario do? They start adding filters to just make the numbers more manageable. Height, salary, location, same interests and so on. So the net effect is that that a lot of people are indepndently applying filters and filtering down to a pool with a lot of crossover. Conventionally attractive men, for example, will tend have far more options.
So I think the same happens with hiring. If you're a Big Tech company, you start adding filters. Did this person go to a top school? What internships did they have? Do these things matter for on-the-job performance? Barely (IMHO). But what you'll probably find is that a handful of people have a ton of options while others struggle. And it's simply the product of employers trying to make their applicant pool manageable but they're all doing it in very similar ways.
And I honestly don't know what the solution is.
This has always been a problem: Candidate applies with an amazing resume but then flails when you ask them questions or “can’t remember”.
I can remember a few interviews where I asked candidates about something I read on their resume (which I study before every call) and they corrected me to explain that they did something different. Then I held up their resume and pointed to their exact words and they turned bright red while they tried to come up with a new explanation.
That was rare, though. You could catch a lot of little cases of stretching the truth, but it wasn’t common to feel like you were reading a resume that didn’t match the candidate.
What has changed in the age of AI is that more people are feeling more brazen about letting the AI speak for them. These situations are happening more frequently. You get the feeling that people are less shy about trying to cheat and manipulate because it feels like the AI is doing the cheating and writing the words, so it’s done at arm’s length.
I spend some time helping with resume reviews occasionally. It’s getting sad to see in the general discussion of the group when people go from elated that they got an interview for their dream job to embarrassed when the interviewers saw right through their AI written resume and ended the hiring cycle. I wonder if we’re seeing a peak in AI resume junk while everyone tries it out, but before it becomes common knowledge that an AI junk resume is a way to shoot yourself in the foot when applying to companies you actually want to work for.
It makes me wonder why so many otherwise successful companies let HR bungle the hiring process.
Hiring has always been broken. May be not completely at the FAANG level, but below that, and more importantly across the globe it's seriously broken, and there's a high variance when it comes to hiring consultants quality.
The widespread use of AI vy applicants is very likely surfacing how comfortable consultants were doing the bare minimum when hiring.
Source: I've been working for 10+ years for a company that has an ATS for mostly European clients.
I know for a fact how crappy work around hiring is.
P.S.: the article focuses mostly on one direction of hiring. The opposite direction is also suffering from this (briefly explained in the article about AI fueled hiring bias). In my opinion, that is an even greater problem.
I will say that I'm not surprised by this at all. I think a ton of people have been convinced that basically all languages are more or less the same, so they are confident putting languages they barely know on their resumes. "I know python and Java, how hard can C++ be?". This isn't a new problem, or even a "coding bootcamp problem"
I studied computer science at a small university in 2006, several of my friends went to a much larger university and studied Software Engineering
They didn't learn pointers back then either. They learned Uncle Bob Java and that was basically it.
Leetcode was always weak, but now that it is easy to cheat on it is a negative selector, because the cheaters do best. Leetcode was originally supposed to be done in-person on a whiteboard to assess a candidate's collaborative problem solving skills, but with remote interviewing it has evolved into writing passing code with minimal or no feedback.
The real problem is that engineering departments are now filled with leetcode grinders and cheaters, who all live in permanent fear of being replaced by AI, and so any candidate who doesn't fit that paradigm is a threat that must be eliminated at all costs.
It's really easy to screen out people when you say "Hey - login to this VM and show me how to import raw data into postgres and run a report."
Or do whatever you're going to do.
My favorite story is from a particular sean who had a candidate that said they'd been using VM for 20 years, and when he went into a document the candidate hit j 200 times to go line 200.
I have seen this phrase structure before.
Lol. I'm not sure this person has ever given an interview before
Just had a "guess the teachers password" moment at some interview as a senior and the interviewer didn't understand my answer and didn't ask questions.
The problem is incentives. A lot of people probably need to be fired who are gate keeping by blocking hiring.
All interviews should be bilateral win win recommendation chats.
They should not end because one person didn't understand the other or someone who was not yet interested in the job did g remember some weird detail of something.
Our memories are getting worse with AI and augmentation.
We need to judge marginal add and make recommendations.
In practical terms Problem: AI made "skill-fishing" easy, and previous signals like good cover letters, well-crafted CV, even correct answers in interviews now don't have their old signalling power - because anyone can do it.
Solution: If this is the case, a) now recruiters need to assess AI skills (exactly what I'm working on - but won't link as it's flagged anytime I link it - but you can search for "aisa test")
b) we need to move on to a system where we accept it's agents talking to each other. CV is for human-human communication but now agent writes, another agent reads. If THAT'S THE CASE - we need an updated protocol for representative agents of each party to contact. (this is the product I'd be working on if I wasn't working on the former)
We took a chance on a flash recruiting session our canton organized. 35 interviews in 2 hr 15 mins. Crazy. But excellent signal, because if you are looking for it, and give the candidate a hint to show it ("tell me a story about how you solved a computer problem for your self/friend/family/club"), you can find the candidates with a spark. And I would not have detected it from their CVs or cover letter alone.
More human connection. Less machines. There, I fixed it for you.
So many interviews still demand absolute perfection so they just optimize for people that are dishonest and get away with it.
Which itself is a symptom of companies getting drowned in AI generated resumes. It's becoming more common for people to use AI tools that will operate browsers to mass-submit resumes for them. When you receive 1000 resumes you have to start filtering somewhere.
What I'm worried about now is that we're moving to a situation where some level of proof-of-work that an AI can't easily do is going to become necessary to have some filtering. I don't know what that looks like, but I don't like it.
> Quite odd this is all happening when ostensibly the unemployment rate is very low, which should make it an employee's market.
Unemployment rate is not evenly distributed. If you were a licensed electrician or qualified as a home healthcare aid then you could walk from one job to another in many cities.
If you're trying to get a $200K or more tech job, then you're competing with everyone else for a shrinking pool of openings.
Yeah, but it's now 1000x worse. Before you needed actual skill (or luck) to create a good looking CV, especially for niche positions.
Now you take their job description, the company's "About us" webpage, your old CV and have LLMs generate a CV with pretty solid grammar and most of the verbiage they expect.
In the past the average unqualified person wouldn't even know the right words for a specific niche domain, let alone how to use them.
Oh, and single LLMs are kind of inherently multilingual, this makes it even worse, because you can have people that barely understand the target language generate a reasonable CV in that language.
The CV quality floor has been raised but the candidate floor has fallen through the pits of hell.
I’ve interviewed hundreds of people over the last few years as a peer, hiring manager, and as a “bar raiser”, and it’s just a lot of work no matter who does it…
"What does bad recruiting cost us?" is very hard number to quantify because it's just sand that gets thrown into so many gears, but cost of that sand is across a ton of departments and so measuring for it is very difficult.
Hiring is exactly the same thing, even when trying to do it on merit, people are simply poor judges of character, ability and the rest.
Most of society is governed by people who simply kept getting lucky and kept doubling down because their ego demanded it and their last roll of the dice didn't drive them to poverty or happiness.
Biases are a strange thing. “High performers” aren’t one homogenous group; take a staff engineer at a FAANG and plop them in a role at a startup or vice versa and you’ll find very quickly that high performers are a product of environment (IME). The people you need to ship something at a big company will sink your startup, and the people who will lead a startup to unicorn levels of success will flounder in frustration in a big corp.
Finding high performers is really hard, as you said it’s a filtering problem, and it’s very much based on vibes and feelings. Leetcode, take home tests, on site tests, discussions about projects all filter for specific things - some or many of which aren’t related to the job at hand. If we removed the “risk of leaving current job element” the only way to do it would be to give someone a 3 month trial and see if they’re a fit. Honestly you probably know in your gut by week 2 if it’s going to work or not.
The time commitment is real, but on-site travel is almost always reserved for the last round on-site. Often as a final pass verification, or when the company is down to a couple of final candidates. Companies aren't flying every applicant out for all of the interviews. If you get to that point, you're close to the job.
Also, in terms of the costs to the applicants, this touches on the oligopolistic nature of so many industries today, which has resulted in high concentrations of the most desirable jobs in places with the highest costs of living.
Basically, unless you already have a FAANG job or are independently wealthy, it's not easy to up and move to Silicon Valley, Seattle, etc. and job hunt.