In defense of coding interviews(biggestfish.substack.com) |
In defense of coding interviews(biggestfish.substack.com) |
I'm finishing up a personal open-source project and am looking for a new position focused on ruby or rails, so if you're hiring and are open to evaluating me based on something cool I've written, hit me up.
And here's where our industry isn't a "real" profession yet. There are plenty of qualifications one can get, but none that gates the ability to call oneself a software engineer. The result: we have to prove it in interviews.
And I don't know whether this is better or not. It is different though. The price of not having to sit through exams is to have to jump through hoops.
The thing with software is people tend to think of it as something that needs maintenance to stay fresh, perhaps a bit like piloting an airplane.
Airplanes have logs and pilots recertify on an ongoing basis, but software doesn't work like that. Someone might have stopped coding but still have a title.
I'm not saying software actually does work like that, but I'm sure it's a view some people subscribe to.
Obviously it’s pants on head stupid to do that if you want a web dev to make a pretty app, tie frameworks together and liaise with 3rd party integrators. However, I think many employers want the crème of such devs who are also good at algos.
Which is not going to work for the vast majority of devs and jobs.
Most devs either don’t care or need to care about algorithms, and the few devs who do, spend most days not using them.
This would be akin to asking a civil engineer to build a wooden road bridge. Having the skills is great, but how often is the company going to need them?
Maybe not in the 21st century however, and I don’t think your analogy is perfect.
It might be more like wanting to hire an architect who is also great at civil eng. The skills definitely synergize and can help prune the architect’s search space of good designs, but such a candidate is probably just going to go for an engi job, and lean on their architecture skills there: for example an algo dev that can do mock-ups.
I think that many employers don’t have a clue what makes a good dev, in general, or in the positions they’re trying to fill.
A completely fictitious language is developed. Simple enough that you can become fluent within an hour. From this you read a bunch of problems in said language and must solve them.
A set of assertions about the problems are written down and you must verify them. Finally a full program, created from a dialect of the given language is given and you try to understand
I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?
Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.
a) Its like saying a Tobacco company has a paper saying smoking is healthy
b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale.
c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
https://news.ycombinator.com/item?id=31544634
How do you test knowledge indeed?
Lol be careful not to ask for a loop, nobody uses loops in real programming problems
dont mind talking over code casually but dont put me on the spot or give me homework. not doing that.
I just want to put this out there to anyone who gets involved in interviews that this is a very important rule -- if your question can basically be redefined as some really clever solution you found on a problem you put > 30 minutes into, it's probably not a good question.
A lot of the seniors I work with would come up with questions like this, and I get their logic:
1. I'm a senior and this was a challenge for me 2. The logic is clear when you see it and look at it 3. I want people who can do work similar to me 4. I know how to get to the conclusion, so I can judge based on the path the candidate takes Conclusion: It's a good question
While I understand how we can get to such a conclusion, I think that in most cases the only takeaway you can get is "can this person think like I did in this situation?", which maybe has some value for judging workplace compatibility, but I don't think it's a fair assessment of someone's technical aptitude. If you yourself required a lot of time and brainpower to sort through a tricky issue like this with the benefit of the problem scope and system needs that led to this issue, it's very unlikely you'll be able to judge any assessment
Point 4 seems good at first because we can say "the goal is to just see them think and use their experience"; there is truth to it but without the rest of the context it's not really a good assessment because ultimately, you're looking to see "did you get the same conclusion I did? Did you avoid the same pitfalls I hit?"
Instead, focus on a general scenario that tests their understanding of basic principles. How do they use the fundamental knowledge for the subject matter to work through a new problem that you yourself haven't really dealt with. It removes your bias a bit and opens you up to those "wow why didn't I think of that?" moments that you'd have working on tough projects together.
Try your best to give a good faith interpretation to any path the candidate takes and help take it to natural conclusions, and see how the candidate reacts; if you can see they've made a workable but problematic solution, nudge them towards the problems you see and see how they discuss it.
For me a candidate that can really process their own thoughts and do a good analysis of their own solution (its benefits and shortcomings) is extremely valuable; I usually lead off with "I know how I'd do it, but that doesn't mean it's the right answer, so focus on your strategy and walk me through it." If you want tot test compatibility or you feel you have a stronger solution, accept their solution (if it's valid) then go ahead and discuss your thoughts a bit and see how they respond and how they discuss it. The more you make it a technical conversation against peers and less a university exam, the more you have to see how they think and operate and the better you can understand them.
It amazes me how many people will defend the current interviewing process and then openly admit that the majority of their engineers could not pass the same process without spending dozens of hours preparing.
[1] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
But other than that the study didn't find any evidence that men have any issues with being observed while coding. So unless a better study comes around we can dismiss complaints about observed interviews being stressful unless it comes from a woman, it is in the study you linked after all. (I do believe there is a stress effect, but that study isn't showing it, need a bit more samples and probably several different observers to get better data)
What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company.
This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems.
To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call:
You are NOT Google.
Read up on HN post on Fast. Someone was commenting one how they failed the interview because the HM wanted someone that wasn't a cowboy. They wanted someone that build microservices like Google. Being a cowboy is what you need to find PM fit. Focusing on microservices when you don't even have enough customers is a huge waste of money and detrimental to your company because of operational complexity. Any changes that should take 5 mins, now takes 3 hours. Fast had so little customers that an engineer suggested that they run the system under a single AWS monolith and was quickly shot down.
But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. They can always use more developers to help optimize the profits of those algorithms or reduce runtime costs, the ROI on that is extremely high so that is what their hiring process is optimized for. Some of their other projects might have done better with a different process, but they don't have much weight compared to the revenue generating projects.
One of the Marques Brownlee videos mentions that when he met Eric Schmidt, Schmidt mentions that every problem Google have were scaling. If you look at it from his perspective, it would make a lot of sense why algo is so important for Google interviews. He applied it not only in tech, but with hiring as well.
Funny how the "objective tests" that leaders apply to grunts never get applied to the leaders themselves.
Ultimately, that's how I think these companies eventually mature. The people that know how to steer the cargo ship retires and you are left with workers that only know how to do the work assigned. Bureaucracy ensues and no one dares to steer for fear it would sink it or get fired for causing unneeded risk. And plus, they don't know how to.
"Do coding interviews work in the actual world in which we live in?" No, fundamentally not. Almost nobody is sufficiently capable to actually analyze code, and writing code with an audience is absolutely a stupid anxiety inducing high stress situation that makes candidates into babbling children who can't define an array - EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE. Add in one idiot on the interview panel who can't stop "tsk"-ing or asking obnoxious questions ("Why did you use a foreach and not a for loop?") and you have an hour long anxiety sandwich where the only thing you are getting from the candidate is whether or not they can dance in front of a sufficiently hostile crowd on demand.
With an exception of one, which was a bit too much like a puzzle, I think the assignments were completely fair examples of what I might experience on daily basis. Not at all tailored to people doing competition style programming, which was what I expected after being exposed to HN for years.
I have to say I enjoyed both preparing for the interview as well as the interviews themselves. And I believe that overall the interviewers were able to grasp how I would code.
With zero preparation, I could go to an interview with you right now and tell you anything you wanted to know about designing a database schema, how to get any kind of data out of that schema, how to make that data available via an API, how to cache the data and perform permissions checks, how to deploy code to a cloud environment, how to set up a build pipeline, or any other question you might want to ask about the day-to-day process of developing and deploying web applications. I can answer those questions because I already work in the industry and I broadly know what I'm doing.
However, I probably couldn't pass a FAANG interview without months of prep work, precisely because whiteboard interviews bear almost no resemblance to the reality of software development.
The questions that are asked have little to no basis in my daily work as an engineer. Same is true for the hundreds of other engineers I’ve worked with who have gone through the same hoops that I have.
In my experience, you have a lot more than an hour to do it in the real job. And while perhaps 1% of SW folks will deal with in on a daily basis, most will not. I certainly have gone years without needing to use graph algorithms (including even DFS/BFS). And I've needed a BST only once in a decade - and I didn't need to code it, just needed to understand the data structure and complexity to know that "binary_search" in C++ was a superior option than using a map for my use case (even though the complexity is the same).
Some context: At work I recently solved a problem where I had to build a graph (DAG), and do some queries on it mildly efficiently[1]. I did it quickly thanks to wasting time on Leetcode, but no one knew how quick, and people were impressed that I "got it done in less than a day". The reality in most SW work is that whether it takes you a day or an hour to solve this problem, the overall rate of progress will not be impacted.
[1] Mildly efficiently because for the first POC pass, I didn't memoize, and then I got lazy and decided I'd optimize it only after getting a real world data set and seeing if it was slow. It wasn't, even for a decently large data set. The reason? Business logic dictated that the max distance between two nodes did not exceed 8, so from a complexity stand point, you're not increasing the work N-fold by not memoizing, but it's just changing the constant factor.
Somehow, I suspect that in half of the FAANG interviews, had I coded this and argued that the complexity didn't change, I would not get a pass.[2]
[2] Not trying to be cynical: I did argue this for a different problem in a Google phone screen - I had been asked to write some code + data structures to handle an Manager/Report relationship, and the interviewer pointed out my data structure was less than optimal. I countered with "In most companies, the number of employees a manager has under him/her is usually bounded by a small number." Surprisingly, the interviewer gave in.
Curious, were there any questions about what you think of their company ethics, and if you have a problem with it?
You learn a bit about the company during the interview, and if that bit is bad, then move on. If they ask shitty leetcode questions and the interviewers are hostile or hung up on unimportant details - move on to another company.
Take for example the "can't define an array" crowd. Are there people virtually sitting in front of me who just forget the basic syntax? Absolutly, but given we a) tell them that correct syntax isn't dreafully important, and b) let them pick the language, a [] would probably suffice, I don't think it's a completely unreasonble ask.
Moreso, when we added a prestep of basic questions that one can google, the "can't define an array" crowd mostly disappeared. Sure, there may be some other causation there, but it's hard to hire a coder that you know, can't demonstrate coding.
Finally, anecdata, since we started coding interviews, the quality of developer is simply better. We have had internal debates about this, but the truth is, everyone is overall happier with what we're doing, and I'm a lot happier that false positives are way down.
I'm sure there's other ways of course. :)
Knowledge generally isn't a great indicator of future job performance, intelligence is. There might be some correlation between the two (you probably need a level of intelligence to acquire the knowledge for LC-style questions), but most jobs are layering proxy upon proxy to test something that has little to do with job performance. Being able to fuddle syntax or search basic solutions and mutate, is leagues different from what LC in an isolated environment is.
Then you add onto that the alternatives tend to be even worse. Personality testing works when done right, but the vast majority of interviewers play pretend-psychiatrist and make the entire thing way too subjective to be effective beyond filtering obvious red flags. Take-homes work, but most companies won't compensate for time and there's no framework to show which take-homes individuals have succeeded (kind of like a certification process), making them an incredible time sink.
And then there are always the cases who have little experience yet manage to pick everything up in 2-3 months and become anywhere from average to top performers.
Tell that to the people who are always the false negative.
I wonder at what point does it stop being "whiteboard interview" and just a "few sprinkle of algorithmic questions to test logical skill".
Also when the talents pool is large enough, obviously you can do as many brutal questions as you can and still fill the headcounts with good developers. So I think saying "but it still works" is not very productive.
I look forward to seeing you disrupt every major tech company in the world via your superior hiring strategy.
> EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE.
You don’t seem to have thought very hard about what employers are optimising for. Everyone realises that programming interviews produce lots of false negatives. Even the strongest programmers have bad days. From the employer’s point of view, that’s okay - minimizing false negatives is not the goal. The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance.
And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.
Have you ever seen a hiring process that's simply not effective?
The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do.
I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too.
I (and others probably) are eager to hear what companies you feel do a good job at hiring.
> Everyone realises that programming interviews produce lots of false negatives.
Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role.
You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge.
Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully.
> The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases.
The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens.
But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?
Hire for strengths, not to avoid weaknesses.
you should be able to explain why you wrote something the way you did; that's not an absurd ask.
perf characteristics of different iteration styles are often significant to the task at hand.
This ridiculously untrue.
It's also a great example of how dumb current interviews are. Someone like this poster could easily interview you, and now what? Do you just play along or begin to explain how incredibly wrong they are?
See, that's just not right. I mean, we all agree there's a hard problem to solve here, right? The world is populated by engineers of widely varying skill levels. It just is.
So how do you detect them? Well, the ability to bang out a correct solution on a whiteboard is a verifiably correct way to do that. It may not be optimal (passing candidates with other issues that make them bad fits), and it may have undesired failure modes (rejecting people who have trouble performing in the high pressure environment). And those are problems worth discussing.
But... it works. It clearly works. We've all seen it work. Companies that use these techniques aggressively tend strongly to be organizations with a long track record of producing working software. The FAANG employers are desirable because FAANGs ship code and make money!
So I just don't get the absolutism here. If there's a clearly better way that produces a more effective workforce, where's the darwinian example? Who's winning by hiring better people than Apple or Meta? I just don't see it.
Once you get into scale where you need to hire 20 devs a year and get 100s of CVs per week it all breaks down.
[0] side note - I work in games where we're often filtering resumes for specific roles, e.g. a gameplay programmer or an online programmer. The filtering here is most of the time bucketing candidates based on their experience and sorting that way.
If you have a better way on how to assess a software developer, I would love to hear it.
Instead I try to find an isolated problem/task we have in our backlog that the candidate should try to implement on their own time, so I can review what kind of solution they will actually deliver once hired. We then go through the solution with them and do a code review as they would go through if they got hired. You then see both the problem-solving skills as well as how they take feedback.
I would recommend checking out DuckDuckGo's excellent "How we hire" guide[0] which describes some of the best hiring process I have come across (albeit it may be too extensive to do in full for some companies).
Step 1: An initial phone interview with the candidate regarding technologies of choice, war stories, etc
Step 2: A small coding exercise that is similar to what we work on. The candidate can solve this on their own time and email us the solution.
Step 3: Review the solution together and ask questions about their methods.
Step 4: Meet the boss & offer.
The coding exercise is only sent _after_ the initial phone call, so we don't waste their time or ours.
The only problem here is that we can't virtue signal that we follow FANG whiteboard interviews and pretend that everyone working at the company can invert a Trie Tree in a 30 minute coding session..
1st interview - Go over the role etc, both parties ask question to see if they are a good fit. If both parties are happy to move to the next stage then there is a take home exercise to do.
Tech Exercise - Something simple which touches the core competencies that the role requires.
2 Second interview - Go through the exercise discussing the design choices etc. If everyone is happy offer the job.
I think the probation period should be used to determine if the candidate is the right fit for the role. That way the business gets a much better idea of who it is they are hiring and the same goes for the candidate.
I'm sure Google wouldnt be as successful as they are if their answer to "how do you define an array" was "well think about what you're trying to do here"
One of the things I like to do, especially when a candidate is super anxious, is just ask them to talk about a recent project they've done that they like and why they like it.
It’s about the person’s behavior, the way he/she interacts with you. It’s about the person’s culture, tabs vs space, and that kind of things. And it’s mostly about whether you like the person or not.
And I think it’s fine. You mainly need to be sure that the person knows what a for loop is, other than that, most people are ok at programming. What really matters is that you can work with that person.
Unfortunately interviewers who don’t realize this will unconsciously tweak the interview to make it harder for people they don’t like and easier for people they do like. And then they have an “objective” reason to not hire the person: “oh my god, he didn’t know merge sort”.
I give all kinds of random questions. And many times I recommend hiring people who bombed it, and many times I recommended bailing on ones who nailed it (overconfidence, poor communication, etc).
Yet most companies interview as if they are inventing novel storage/processing mechanisms. Theory is important to understand which tools to leverage when, of course, but not to the extent that it's typically prioritized in the typical interview.
I've had plenty of people crush the theory portion of the interview and be poor performers on the job (slow coding, low volume output), but I've never had somebody crush the coding portion (implement very fast and proficiently), and perform poorly on the job
Leetcode-style interviews became popular in the mid 00s, primarily because they were used by hot tech companies of the time. The thing to understand is that at that time, the idea of asking people to write code during an interview what sort of revolutionary. Prior to that interviews had little structure. It wasn't unusual for the hiring manager to make the decision, some times based on credentials, recommendations, or trivial-like questions.
This type of interview became wildly popular because it allowed budding unicorns to hire programmers of high quality at scale. The process was less biased than the alternative, reproducible and scalable. Here you have two blog posts [1][2] that show the line of thought at the time.
The reality is that big tech has elevated leetcode type interview to an art. They have reached a near local optimal through years of experiments and refinements. It is working well for them so they don't have the need to take big risks such as completely revamping their hiring process.
I love the topic of hiring and interviewing and I'd love to truly get at the bottom of which method works best. I like this article because it explicitly calls out shortcomings with typical alternatives that are not usually mentioned. I hope in the future a new crop of unicorns can take these practices to the next level and do a fair comparison.
[1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... [2] https://sites.google.com/site/steveyegge2/five-essential-pho...
Best I can think of is take home tests but I wouldn't want to rely entirely on that, and they have their own problems too.
I have a decade of experience, working as a senior and staff engineer at megacorporations, have experience as a research scientist, am in a PhD program for computer science research, but lets just double check that I know how to use a hash table.
Should it be from university CS programs? Which are supposed not to be vocational? And would discriminate against those who are not degree holders?
Should it be from internships, which are nowadays subject to the same Leetcode examinations as actual work?
Should it be from junior developer roles, which are an endangered species, as every business in desperate need for seniors, and lack the patience to train?
Maybe open source can be a sort of free apprenticeship to teach developers these good practices?
Or maybe grinding away at Leetcode, Project Euler, and arbitrary coding puzzles is really the only pedagogical solution.
It is how people pass these tests, but it’s pretty poor pedagogy.
We have no issue whatsoever hiring juniors (we hire about as many as mid-levels), but we don't have to settle for someone who doesn't have internships or projects and the openings can be closed in 2 weeks so it seems like nobody is hiring, but actually the bar has risen (and there are plenty of candidates who clear the bar).
I am confused - how is this question materially different from asking how engineers are expected to know what they will need in order to properly do their jobs? Just how are engineers expected to know programming languages, or domain-specific languages, or standard libraries, or platform apis? Is this what you are asking?
Corporations are not testing your knowledge of algorithms per se. While that knowledge is obviously important, what's more important is how dedicated you are when it comes to reaching a goal. Essentially, they are testing your will power. You know the rules. Can you achieve success and not give up in the middle of the road by skipping, say, Dynamic Programming? They could've tested your level of dedication by asking to bake some fancy cake, but throwing CS-related questions just makes much more sense.
That reasoning completely changed my perspective about algo-heavy coding interviews.
I have two very memorable no's. They were for pretty senior/advanced IC roles. Juniors would get far more slack. I usually ask an expression evaluator question.
One person, used eval(string). Brilliant, absolutely brilliant. full points for answering, but how do we make sure the string is safe to eval? Long frustrating discussion around regexes that that maybe a junior dev is responsible for maintaining. Man, just write the parser. I know you can do it. Give me something that's something that's not _crazy_ to put into prod. Give me something that won't be a bug tarpit. Sorry man, I loved your answer. I hated your response to how this can be operationalized. Clearly super smart. Maybe I should have given the green light, but the response to criticism was so bad. There are lots of people I disagree with, that I'm happy to work with. I think I made the right choice, but I wonder from time to time.
Another person I absolutely would have loved to work with, and would have learned a ton. Very into formal methods, or slightly relaxed versions of verifying reliability. My organization at the time was pretty fast and loose. I believed they would be miserable. In the interview I told them I'd give them the green light, but they'd have a huge uphill battle.
it's so hard to tell. People are adaptable, small mismatches can be papered over, but big mismatches - it's so tough to say. Relaxing and being vulnerable on both sides is so hard to do well. Maybe I was just a jerk. I think I made the right calls, but it haunts me.
Personally I don’t mind a whiteboard session or two during a loop but what’s wild to me is how, especially at big companies, you’re expected to do four or five of these to get an offer.
How often do these companies decide “well they understand when to use DFS and they can merge-sort a linked-list and they knew to use dynamic programming but we can’t hire them because they couldn’t remember how to implement a heap”?
The more casual, flexible, subjective coding interview described by the OP is not the prevalent form of whiteboarding interviews, which has been superseded by the weeder version.
I agree that this is unreasonable but not for the reason that you mention.
Once you can pass this kind of interview, you can throw me 10 more and I will still pass them.
Edit: on further thought. I think I’d be ok with coding interviews if the interviewee got to bring in their own coding question and the interviewer had to do it as well. That way the they also knows the people they are working with can live up to their ideal of a good coder.
It's saying coding interviews are not spectacularly awful if you (thoughtfully) change them.
Great! That's what everyone complaining about them has been saying in one way or another all along, change them.
I wrote about my own experiences with coding challenges during my recent job search on my blog at https://blog.urth.org/2022/04/19/software-job-search-2022-re...
I'll summarize my conclusion about live coding challenges, which is that I'm not convinced that my performance on these challenges reflected any abstract skill I have. Instead, they mostly reflected the fact that the problems I was given were either nearly identical to work I've done in the past, or were similar enough to things I'd done recently that they felt pretty easy.
I guess there's _some_ signal in that, but I don't know if it really says anything about how good I am at coding in general.
Here's some data models, and here's how you can reference eachother by properties. Cool. Let's do 3 levels of nested loops to extract the relationships and map them. That's like 99% of what they'll be doing.
The signals I'd look for are "do they start the entire thing by looking at structures or not", etc.
But unfortunately this breaks down when you're interviewing 10-year+ candidates. It doesn't yield anything useful. So sticking for those positions with a really really basic problem and pseudocoding it, and using the remaining time to instead have them map out solutions they did in the past seems to be useful.
Most things leet code interviews screen is "does this person know how to cram" because I tell you, everyone I know who aced leet code interviews crammed for 3 months and literally knew any one you can think of, and every variation. Juniors tend to do better in those, ironically enough.
Is it ironic or intentional? Seems like a great way to hide age discrimination under the guise of "fairness" (everyone gets the same test).
As I commented above, I successfully passed multiple leetcode interviews in the last four years, including FAANG. I did not spend a single minute on prep.
A) SWEs are shit behavioral interviewers in any other way. You are not about to chnage that even if you are google.
B) It is crucial that newcomers be hired by their peers and by their direct manager. This has solid research. People hate it when they get an hire and they hadn't a meaningful part in the decision... The new hire is more likely to fail.
A+B means it is better to have a terrible process lead by SWEs (e.g. coding interviews), than a good one lead by professional recruiters (which will fail, no matter their capabilities and methodology) (and yes, I do believe that without considering A, some non technical recuiters trained at workplace psychology will have a much better success rate without needing any technical interview. They will need someone technical to have a talk with the candidate, but definitely not to ask leetcode etc).
It is an interview about your communication, whether the code you write can be maintained, whether you can inspire/build trust (can you explain what you're doing? does that feel reasonable to the other person?).
Where people fail is to think that it's a test of experience, or a "heads down I must make every test case pass", that the outcome matters more than the process.
It's all about communication.
The "write code" types of interviews fit for juniors who might not know how to even write code properly. If a person doesn't know how to do that and is a junior they usually have the right energy and can put 100% into the job to adapt/learn. I hired such juniors who had no experience in the languages/platforms and were productive within a month. They had good character and discipline.
Distilling a question like that for an advanced coder is impossible. No wonder the article didn't provide any sample of a "good interview". We'd all burn him at the stake because we'd all hate it. There's no such piece of code that would provide a valuable data point about a candidate. So this is a waste of your time when dealing with a junior and with an advanced coder.
Coding interviews are difficult. You will make mistakes, this is unavoidable. I think the best mistakes to make are the ones where you hire people who are part of your team and can grow with it. Even if they aren't the best coders around, being part of the team will help them hone those skills and evolve. If I had to pick a 10/10 coder or a team player I'd pick the latter every time.
Got the job.
Then I did a slight pivot to get into BigTech via cloud app dev consulting - “application modernization” [2] - still with no coding interview although I still do the same type of enterprise/cloud app dev I did in the latter part of my career in the “real world” - and a shit ton of yaml.
Even in 2020, if I couldn’t have gotten a job at either Azure or AWS, I probably would have done what it takes and practiced to increase my income by an easy six figures or more.
Coding interviews are a “gravity problem”. You can not like either. But they still exist.
That being said, if I were starting out today and could break the can’t get a job <-> don’t have experience cycle by “grinding leetcode” for few months and and end up in the top 10% of income earners in the US right out of college, I would have definitely done it and I encourage my younger relatives graduating in CS to do it.
However, if you’re just hiring for your standard yet another senior CRUD developer and paying as such ($90K-$160K) - don’t expect me to do the monkey dance.
I think algorithm style coding interviews are the great equalizer. Even though I at 48 probably couldn’t pass one without some serious practice.
[1] https://www.hanselman.com/blog/dark-matter-developers-the-un...
[2] yes it’s a full time position with the same compensation structure.
Not in FAANG or famous startups it's not. You're either gonna solve 3-6 medium hard Leetcode questions perfectly or you're out. Sure, liking you will help you better not come off as a douche - but you're not gonna get an offer without being able to solve (unless it's some kind of a diversity hire).
* read code
* extend some working code with a new feature
* parse some semi-structured text into a data structure
What folks should do is practice coding in front of someone. Writing little exploratory snippets and understand the base library of the language they claim to know. I don't even ask people to vocalize while they are working on the problem. And I solve it at the same time they are and if they get stuck, I show them some of my code.
I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
The idea that everything boils down to answering some intro to programming questions quickly is absurd.
of course i interview for my specific situation. the candidate will be working with me directly, so they should at least be able to interact with me. if i don't like them then that's almost a showstopper, but if the candidate otherwise qualifies i should try to figure out what my problem is, and see if i can get over it.
The analogs of the traditional coding interview in the law school world are the LSAT and classroom timed memory tests. Both have absolutely nothing whatsoever to do with being a lawyer or the kinds of activities that lawyers actually do.
What they are are intelligence tests. They are easy ways for judges and top tier law firms to tell if you are "one of them." You can get your foot in the door if you can perform, at a minimum, at a certain level. There are plenty of things you can do without 97+ percentile LSAT and A's in law school, but you are screened from the top echelon of work.
People object that this isn't fair and screens good people out -- false negatives. But experience has shown that it is effective and easy. Perhaps there is a better way, but this way seems to work well enough. There is little incentive, beyond some people complaining about it, to change a system that is working well.
I see the traditional leetcode interview in the same light. The point is to see if you can perform at a certain minimal level. Yes, it's arbitrary, and often unconnected to real life. But it is an effective way to gauge a minimum level of intelligence and competence. Not perfect, but easy and effective.
Finally, there is ego element to this. People who can perform at this level see these skills as a minimum qualification. They are uncomfortable seeing themselves in relation to others who cannot perform at this level. Knowing that their peers have similar minimum qualifications makes them feel safe and secure about their status. This is not a criticism, just an observation about human nature. People in a club need a shared identity.
Especially if the interviewers make comments like "um how much do you actually practice law?" if you stumble on one of their pet quiz questions.
How are these analogous? LSAT is the Law School Admission Test. It gets you into law school. It doesn't get you a job as a lawyer. (The bar exam doesn't get you a job as a lawyer either.) And you only have to take it once, not for every job interview.
You can go to a top tier university, land a great job for several years, have positive references, and still be required to do leet code for a recruiter... that seems like ongoing gatekeeping, rather than a one-time effort/pass.
When I'm interviewing a senior developer that will guide other juniors, they will need to prove they know more than a for loop. They will need to show they understand the underlying concepts and technologies. Else it's a no hire, tough luck.
We also fired plenty of people that weren't up to the quality we expect.
Were are you working? At some government?
If they come up with, or start walking with an interesting solution, I listen to them.
Tabs and spaces don't matter.
I actively work against the biases of "whether I like the person or not" because I'm aware of that as a factor.
We can absolutely minimize all of those peripherals when we explicitly train against it and a good programming interview doesn't have to be about that.
But to be perfectly unbiased, the process would have to double blind. And I don’t know anyone who likes to hire people (or anyone who wants to be hired) following a double blind process.
So at least there is _some_ non-technical bias that gets in the process.
It’s hilarious listening to people review candidates on third coding interviews. Usually it’s “this person didn’t know this one thing I think makes me really smart”.
This is why I lean so heavy in OSS to review candidates. I can see how they interact with people, and whether they are actually able to build software that _people want_
I find with this approach it’s a lot harder to justify nonsense.
I hear people say maybe they don’t have time for OSS and then hand them a Leetcode interview, which as we all know is just about memorizing problems. OSS is such a better metric and has the added benefit of improving the world.
Yup. I just went through a round of interviews at multiple companies. In the past I successfully interviewed at a FAANG.
I agree completely.
This.
There is absolutely no need to ask such ridiculous algorithms and advanced equations questions that you will never use or implement from scratch yourself, or asking the candidate to create a formal proof by rote for a entry-level software engineering position or even a senior level position.
Such interviewers who ask these questions have both googled the answers to these questions before the interview and also have never used them practically, but only do this to appear smarter than they actually are. Worse part is not even FAANG asks any of these formal proof questions, but only some certain startups that the last time I have checked, they have failed to make money and have shutdown. Why? Because they seriously thought they were a FAANG company.
Open-source cuts through the algorithms nonsense, saves time and gets a realistic idea of what the candidate has actually worked on in the open. I can ignore the typical hello world, or demo projects but can ask for if they have created open-source libraries or contributions to any serious projects like compilers, OS projects, or any related project to the job description etc.
A take home project is also a fine metric, but asking questions about algorithms and finding out that the interviewer (or the company) doesn't even use them tells you that the interviewer doesn't know what they are looking for.
If you don't answer the 2 leetcode mediums in optimal complexity within 45 minutes then you don't pass. It's about coding skills.
Even as their manager it’s not my job to lead a death march.
Somebody who produces 2x at same quality of work is worth 2x the other person, fairly objectively.
It's common to find people who produce up to 10x the average. Anyone in a startup or growth oriented company is smart to optimize for these people.
The most proficient and fast coders often write higher quality code too, in my experience.
People who over index on comp sci theory often are lacking in theory underlying good code. But it's not mutually exclusive. Just that if you have to optimize for one, coding proficiency matters much more for startup or growth type companies.
Exception might be like an AI/ML based startup with a lot of theoretical challenges. Most companies much more business/rule oriented though
Being able to come up with an algorithm in a reasonable time doesn’t make it crappy. No wonder you can’t pass those.
I think the reason for that observation relates to why these interview tests are generally a waste of time.
Based on the tests that I've seen, they are 'recall lots of useless information' tests. That means it's actually possible to study for these interview tests (if you have the desire) and if you can ace the test by just spending time memorizing answers.
However, 'slow coding' has very little to do with coding and everything to do with problem solving. You only learn how to problem solve through lived experience and many people never learn that skill.
These interview tests generally fail only because they're not actually testing for the skills needed to do the job.
What does that even mean?
In an interview context, I've seen cases where somebody fully understands the logic underlying the solution, yet it takes them 30m to write it, while another person can do it in 5m or less.
Essentially how long it takes to translate from mental understanding to working code
And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping.
During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).
I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting.
Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job.
I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure.
The interview I had at google was much more humane than what people are attempting to cargo cult.
I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate.
- Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity?
- Scalability: can you evaluate many candidates fast with high throughput?
- Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately.
- Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies?
Leetcode format does good in all criteria above, or at least better than other formats below in aggregate.
1. Take home assignments.
- Objectivity: bad
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates.
- Accuracy: good.
- Meritocratic: good
2. Coding questions with heavy emphasis on practicality (Stripe & Square)
I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google.
- Objectivity: good
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily.
- Accuracy: good.
- Meritocratic: good
3. Knowledge / Trivia questions about programming language or framework
This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions.
- Objectivity: good
- Scalability: good
- Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily.
- Meritocratic: good
4. Pure behavioral
- Objectivity: bad
- Scalability: good
- Accuracy: bad
- Meritocratic: bad
5. Ex-coworker
- Objectivity: bad
- Scalability: bad
- Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well.
- Meritocratic: bad
> Meritocratic: good
I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.
If it is standard at the company to give all candidates that same question, this is actually a good way to reduce bias in hiring decisions. Otherwise, you might bias toward hiring, say, people with college degrees or people who worked at large corporations, for example. And this might unintentionally bias against certain demographics that obtain college degrees at lower rates (for example).
Also, typically the point of these questions is to see if the candidate can solve a novel problem not if they can use a hash table.
This is a hyperbolic rhetorical question but consider how much of modern work really is CRUD API gluing, and how much of that work is not rigorous engineering, but tedious implementation as per documentation and wading through hastily-thrown together legacy cruft written during previous crunch time.
How much of that work involves novel programming and memorized understanding? Most of it requires study of existing codebases, not given a blank REPL to hash out something new.
When I started interviewing I started off by jumping into what I considered to be not insulting questions, but I very quickly learnt how awkward that can be if the candidate just had no clue.
Now I start with for loops and preface it with "apologies if this seems too simple" or something like that.
You would be surprised how many people can't do a simple for loop. I mean really simple.
I really hate the contrived problems that interviewers pull from HackerRank and its ilk. Even if you could glean useful information from watching someone program a knight hopping around a phone dial pad (a real problem I've gotten, which I'm highly skeptical of its utility), it comes off as pretty irritating to the interviewee. You're not solving little programming riddles on day to day work, you're solving engineering problems, and I think that these leetcode questions don't reflect that.
I think there's evidence of this too; you can study for these leetcode questions and get better at them, so what exactly are we testing? You're not testing the person's experience, you're testing how long they spent on hackerrank the night before, or how many interviews they had recently.
I think one can fairly accurately determine my level of skill from my resume, and thus I don’t think that the leetcode interviews are all that useful. I keep hearing about senior engineers who can’t code, and maybe I’m just lucky, but that really has not been my experience when interviewing, and I don’t think “spent the previous night practicing on hackerrank” really gives you any useful information.
That's an interesting theory. And they pick the goal that happens to be best suited to fresh/current undergrads from Stanford who weren't working a draining job to put themselves through school, so can afford the additional person-months to achieve the gatekeeping that was set up by people like them, for people like them.
I and many other experienced software engineers have demonstrated perseverance and rising to the occasion of extremely challenging goals. But that theory suggests they're not interested in that. Maybe they're interested in the class shibboleth that was established when computers became less of a meritocracy for people with a passion for it, and more of a go-to status job for the upper middle class?
I'm not sure how to make this come through but I'm rather sympathetic. I think practical experience & can-do coder know-how count for most of the marbles. But I also think it's impressively hard to assess by, that most of what we do is just taste & path dependence on the specific techs we've seen that happened to leave good impressions on us: not science, not knowledge, not truth: most coding, most practice is almost entirely happenstance.
But again I want to empathize. Because I think it's cruel to disregard engineers that have seen so much. But I also think you're hyperbolizing overmuch & discrediting the discourse when you make your argument so sharply pointedly. Because there's some truth, absolutely, but it's also no-where near as lopsided. For example, you talk about fresh/current undergrads at Stanford. But this is a pretty regular, basic CS path that any university student should be familiar with. I only had two undergrad classes that really talked to algorithms (Algorithms and then Data structures; OS kind of somewhat), albeit there was some computational thinking already in play at that point. Understanding complexity & algorithms is understanding the basics, it really is. If you can't see time flowing, don't know the tally & impact of the work you're asking for, you're lacking knowledge to avoid a lot of bad decisions. This isn't made up fake shit taught only to the elite.
What's alluring about this system is that it has some hard facts & theory underneath it. It's not just opinion & engineering pop-culture. The answers don't change, the rules don't change, the material stays the same, and it's all rooted in being able to analyze and understand problems, rooted in comprehension. Being able to see & understand & analyze, being able to apply some basic principles: this is a pretty sure way to find people who will be able to Not Mess It Up, who are capable of looking, assessing, & navigating through scenarios computationally.
And last and perhaps most key to me: it's also material that someone with even a modest bit of aptitude can cram for. Pick up a book like "Cracking the Coding Interview" and you can semi-accurately reproduce the output of four years of undergrad in two or three months of occasional light practice & trying. Projects like Leetcode can get you the experience. You may feel bad that what you want yourself to be measured on doesn't count here, but to people trying to go get hired, it's enormously helpful to them to have well defined expectations, to have specific kinds of tests to expect & be able to prepare & study for. I see so many protests that it's not fair, that it favors only those with the luxury of time, but those views to me massively undermine how wonderfully vastly accessible it is that there's a known, well-described, well-supported subject-area we can study. The industry overflows with things to learn, study, & know, but here's something concrete & specific, which undergirds it all, which alone might not determine your coding skill, but which does indicate you at least have some raw basic intellectual capacity to go understand problems put before you & apply some sensible computational thinking to tackling them.
There's tons of things not tested for, but by having a well defined, technology-neutral set of computational thinking tests, based on actual science with objective, factual answers, rather a much wider corpus of future/current/legacy pop-engineering-of-the-day built on nothing but wishy-washy subjective opinion, I think we achieve one of the best possible wins we can against an enemy we both hate: class-based discrimination, cruel tests that reject outgroups.
There's other ways to tease out if someone is "dedicated to reaching a goal" like actually asking them questions about their life, daily routine, accomplishments etc. I think these are much better signals than "this person can implement many sorting algorithms and solve towers of hanoi or the egg drop puzzle without a google search to refresh their memory" with an N=1 sample size used to gauge how reliably they can do that. How many people if asked to do this on a job rather than an interview a year later would then go and implement it from memory without double checking they didn't forget some edge case on stackoverflow?
I know grinding leetcode is fundamentally useless because you will immediately start losing your ability to interview once you get hired. If you don't change jobs within a year you'll need to start studying all that crap again for the next interview.
An excellent question.
If the interviewer is to ask such algorithms questions looked up on leetcode, etc, then they must be prepared for when the candidate asks if they also use it themselves on the job. If the interviewer admits they don't use it, then they also admitted that they don't know what they are looking for and really are wasting the candidate's time.
If it were me, I would look at open-source contributions (no hello-world or demo projects) where that is enough proof for me to evaluate an entry-level candidate and cut through the algorithms nonsense and ask questions to the candidate based on that which will save everyone time.
That this isn't the bulk of what we do doesn't change the fact that these challenges do test computational thinking acuity. Having the ability to see & speak computationally is a good skill, one that connects our day-to-day abstract practice with actual real processes. Being able to break down problems & analyze how to tackle them shows an objective ability to assess & work through problems. I want to work with people who can be clear, who can model & explain & step through situations.
And these skills are, generally, learnable, and relatively quickly. I disagree that these abilities fade, but yes, some re-familiarizing & re-training is probably important, especially because, as you point out, the sample size is indeed often N=1, and that's pretty wild.
If you are a no-name company paying whatever your HR department defines as "market rate" and you don't get hundreds of very qualified candidates for every role then interviewing like Google might not be a great idea.
"Great" filtering for parents or people caring for ailing family or any other non-work commitment.
I guess it's not ageism at all, right?
That may sound good in theory, but in practice I've been rejected quite a few times for not delivering the correct or most optimal solution despite trying my best.
it always amuses me when people use these seemingly stupid questions (out of context) to prove how broken tech interviews are. but of course, if your job will be write high perf code, then suddenly they question is incredibly relevant.
to be clear, i agree that there are a lot of stupid questions that get asked which are not relevant to the actual job you'll be doing (exceedingly few SWEs will need to implement a merge sort as part of their job)
These questions, while certainly aimed at identifying core competency areas, are also an opportunity for the candidate to demonstrate an aptitude for problem solving. Nobody really cares "how many piano tuners operate in Seattle". What they care about is, given that kind of question, how a candidate responds. You might be surprised by the number of people totally unable to even generate a reasonable estimate:
- How many people live in Seattle? - What percentage have pianos? - How often does a piano need tuning? - How long does it take to tune a piano? - How many pianos can be tuned per work day (given commuting time)? - etc.. driving towards a number.
Did the candidate attack the problem and even attempt to solve it? Did they use metrics? The list goes on... Most LC-style questions (save the "aha! moment" ones) have a similar impetus, but are also much more practical and efficient because they test for many things at once.
And if I'm being honest... most FAANGs aren't looking for engineers that require quiet and time to solve problems. They are looking for elite engineers who can write a solution nearly as fast as the problem is presented (I'm obviously exaggerating a little bit here). The difficulty of the questions presented to me (6 technical interviews) were what I consider semi-trivial. I was able to describe the solution-space within seconds of reading the prompt and code an optimal solution nearly as fast as I could type in all cases. It honestly made me wonder about how poorly the average programmer must perform such that these were the questions they needed to evaluate my potential.
Another commenter posted this article [0] about the interviewing philosophy back in the early '00s. I found it very enlightening, and reassuringly, while reading through it I knew the answer to nearly every question (and could expand upon the topics) just off the top of my head. I'm not trying to sound arrogant here but the idea that a FAANG, who pays top-dollar for engineers, would settle for "10 years experience" as a suitable replacement for "demonstrates ability" is what's absurd. A false positive is way more costly than a false negative.
[0] https://sites.google.com/site/steveyegge2/five-essential-pho...
Getting into law school is the first part of the "interview." If you don't have the LSAT score, you won't get into a prestigious law school, full stop. If you aren't in a prestigious school, then a huge swath of elite positions are permanently closed to you. (In general -- yes there are a sprinkling of exceptions.) The bar exam has nothing to do with it -- everyone has to pass that, no matter what you do.
The second part of the interview is law school grades and certain extra curricular signals. You're right, there is only one "interview." But the analogy still holds up, with respect to the nature of the intelligence and memory tests.
I asked “do we want to have consistency or availability for this?”, to which the interviewer said “I want both”, to which I said “umm, I’m pretty sure you can’t have both, CAP theorem and whatnot”. We spent two minutes arguing which led to me taking out my phone and pulling up the CAP Theorem Wikipedia page, and eventually the interviewer begrudgingly admitted I was right.
I surprisingly did get an offer after that interview but I found it a little annoying to be interviewed by a person with less experience than me who I have to argue with because his understanding of computer science is wrong.
The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included.
But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
Well, I'd like to have a word with the Apple Mac Mail app team. And the Disk Utility team. And some others...
Congrats, you just invented whiteboard interview.
Coding is the act of writing code, not ideating an algorithmic solution. Its almost completely orthogonal to computer science problem solving.
The principles of high quality code (from a design perspective) are also completely orthogonal to runtime complexity of an algorithm.
I think computational thinking is a good objective measure, and is learnable. That comes first to me. I also think acquiring & practicing computational thinking demonstrates will power, a willingness & capacity to tackle something, and that that something is valuable. And it's an objective, common, topical, & agreed upon/expectable test, one that doesn't rely on socially convincing someone in 5 minutes you indeed have willpower.
If you only think of this as trying to get good at Leetcode, I think you risk ignoring the actual lessons. Leetcode is an ok practicing grounds, but what's being tested relies on a mix of computer science and problem solving, and just going to leetcode & trying to get good there will probably eventually work out, but you'll miss the computer science that would actually be helping you.
The interview results need to be reasonably repeatable and comparable to assess interviewer skew and bias.
The result is that the interviews themselves inevitably end up being somewhat predicable. And what is predicable you can prepare for, giving you predictably better results.
If you are top 0.1%, sure you will nail them without much prep, but most of the candidates would spend some time preparing.
Well... yeah. To pedantically elaborate that, clearly the hypothesis is that the measured traits (doing well on a coding interview and doing well at coding) are co-causal, and that measuring one (which is comparatively cheap to do) is a useful proxy for measuring the other (extremely expensive!).
That's a really compelling hypothesis. The counter point requires, to my eyes, a clear working example: a company with a uniform hiring process involving some other technique which has a track record of product development as robust as all the existing leaders. And I don't see one. Calling them monopolies or cargo cults seems to be filling in for evidence, and I don't see that as persuasive.
But there are plenty of smaller companies who do well for themselves, relatively speaking, without cargo culting on BigCo hiring. And there are plenty of companies who cargo cult on BigCo hiring but who have not become a market leader. In fact there are graveyards full of failed startups who cargo culted BigCo hiring. So the hiring process doesn't clearly explain the financial results of the BigCos.
And on the other side: are there major employers who implement rigorous coding interviews who... don't have a history of producing good software? I really can't think of any.
That's my point: the evidence is really stacked in favor of the FAANG scheme. So pointing to a handful of small outfits doesn't really sway my impression; those are just expected outliers. I've worked at smaller companies with terrible hiring success, FWIW; they ended up successful because of a few lucky hires.
I don't think that's right. If a small town only has one gas station, and the next closest is 100 miles away, isn't that gas station owner a monopolist?
Moreover, "good software" is also highly subjective and subject to dispute. I feel that the software coming from Apple and Google has gone downhill in the past decade. Are we to explain that by hiring?
The financial numbers are indisputable for Apple, Microsoft, Google, Amazon, and Facebook. (Not sure the term FAANG even makes sense anymore.) But something like "reputation" is another matter entirely.
This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts.
> Hire for strengths, not to avoid weaknesses.
I guess you didn’t read the part of my comment where I wrote:
1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.”
This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems.
If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall.
It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing.
But my default position is scepticism, because if it was possible it would probably already be happening.
Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage.
We can get into a discussion/argument about what is the competitive advantage of Google/Microsoft/Apple (which collectively own near 100% of the market for mobile OS, desktop OS, web browser, search, etc.), but that's kind of a tangent IMO.
I will say that Google/Microsoft/Apple have a competitive advantage specifically in hiring (that is, getting many of the top candidates to apply) because of their prestige (a product of the aforementioned market dominance) and vast wealth. The latter in particular allows them to offer more compensation, especially stock compensation, than other companies.
It's important to mention though that the formalized hiring practices of these companies are largely ex post facto and could only be put in place after they were already successful. It's unlikely they hired exactly in the same way during the early years. Steve Jobs was introduced to Steve Wozniack by a friend in high school! Nobody can plausibly claim that the success of the Apple II was due to "coding interviews". Larry Page and Sergey Brin met in college, where they were already working on the ideas that led to Google. Hiring doesn't just turn into success, that's not a reliable formula.
You can set the bar as high as you like, but you won't be able to hire anyone if candidates just walk away from your obstacle course. Talented people put themselves through a lot of crap to work at BigCos because there's a big payoff working for a BigCo. That's what tends to make the BigCo hiring pool better than for other companies, not the hiring bar itself. You can afford to be extremely picky if you have a wealth of options to pick from.
You're conflating some things here. Without unpicking it all, the main thing I'd say is that you've only demonstrated that amazing hiring is not sufficient for competitive advantage. You have not demonstrated that it's not necessary.
Ideally without an increase of time/cost requirements.
I can imagine it, because I've been that person.
It's simple interview anxiety, which causes temporary but severe performance degradation. Then when the interview is over, the anxiety suddenly disappears, and you can even go back and do the same problems much more easily.
But I suppose interviewers will just assume that my resume is a lie or that I managed to incompetently hold onto previous jobs.
If these people are finding work elsewhere and doing productive work for years what does it say about your process?
Or do you think companies never fire people?
We already fired plenty of people that we hired. So yes, our hiring process is not perfect. But if you're a JavaScript developer and can't explain the difference between var, let and const, I'm not going to hire you. If you claim these people are great developers, you are free to hire them.
Let me know the details of your company. Whenever we find a candidate that doesn't satisfy our requirements (which is around 90% of them), I'll send them to you so you can hire them.
But, I don't think those are vultures. They are bald eagles. I'm not sure if owls scavenge, but I do know they primarily hunt. Eagles both hunt and scavenge. Vultures primarily scavenge.
On the other hand I felt like a sample of 8 different interviewers from different offices and backgrounds felt representative. If it is true that the interviews are overwhelmingly negatively received, what is the probability that I would get more than half of reasonable interviews? Let alone 7/8.
The interactions and power dynamics between people differ wildly from person to person. While an interviewer might be the kindest person in the world for candidate A, they might be a total bully for candidate B. I was bullied constantly throughout school snd, whatever cues prompted those kids to pick on me are probably still there, but adults are a bit more subtle, because they can hurt you in other ways. And this is on top of the day-to-day variations, because they might be hungry and the current interview is sitting between them and lunch or maybe they just got slapped down by their boss and want to punch someone or whatever. Based on my own experience, out of the typical 5 rounds of in-person interviews that I went through many times for FAANG companies, one or two ended up being a total shitshow each time, where the interviewer was condescending at best and I don't think they realised it or cared. I was told several times "how easy / trivial" the problem is and that "they're sure I'll have an easy time solving it based on my CV". Maybe the worst ones are when the interviewer smirks at me when they spot dome issue and I can't cope with that.
Now add to this a long streak of never ever getting an offer after dozens of such interviews in spite of putting myself through training sessions and mock interviews over and over. I'm sure I'd get in eventually, if I just kept trying, but do I want a job where I'll be asked to interview people in a similar manner? Even if I refuse to interview people as part of my job, will I really thrive in there? I'm certain this process permeates in one form or another throughout the company culture in various internal management, decisionmaking and promotion processes, but that's a separate discussion.
Are you sure you want to work at a FAANG company? You sure you would be happy? What motivates you to try so hard?
The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it.
But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
You're also maximizing the odds they will get bored and feel baited. With all due respect, you could have picked a better example to illustrate your point.
This is exactly why you can't make such bold claims. You're looking at this from one perspective without considering all the variables which go into the workplace equation. So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place. For various reasons, this isn't really happening in software development, making people all to eager to grasp at proxies which they swear will do the same thing. Maybe whiteboarding is in line with the day-to-day at FAANGs, but tons of companies ask questions which are not in line with the day-to-day at all. Both vertically and horizontally. It creates dissatisfaction, burnout, bad cultures, sometimes even animosity, as if hiring managers are playing pranks on candidates.
If you're only looking at this from the perspective of "get someone who can do the job", you're going to miss the forest for the trees. As others say, the company is also being interviewed.
I like this point. This is an especially big problem in my area (data science), in which people think they are being hired to do deep representation learning, and are subsequently mortified to learn that the job is principally about transpiling ”product manager” to ”SQL”. The cure is to be up front with job candidates. Tell them the job involves no deep learning. Explain that you give hard interview problems because you need to know that they can reason about eg asymptotic complexity on the rare times where it arises.
> So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place.
I don’t necessarily disagree, but I believe you want to optimize for the hardest day rather than the average day. If your team is optimized for who can best solve an average problem, you will eventually have a problem that nobody can solve, and the business will suffer greatly as a result. Hence the discrete math problems.
You might enjoy Work Rules by Laszlo Bock. They actually did do this at Google to validate their interview approach.
In all seriousness, thank you for bringing this up - it’s an interesting story and I will probably look into it. But I also think it’s pretty obvious that coding interviews work extremely well, and any discussion of ”how can we create a procedure that is better for everyone” that doesn’t acknowledge this is hopeless.
We have an initial Hacker Rank 'filter', that if you don't pass that, we don't even do an interview. A few weeks ago we interviewed someone who passed that filter very well. But in the interview, it was really below par. Not even the basics were understood. So I'm very skeptical about these kinds of homework where they could get help.
What I currently do is go through a very simple coding example that I wrote (very generic). Then we go through the code and I ask questions like "can you explain what this line is doing", or "If I move this line over here, is the behavior still the same", etc. Some bugs are in there and I ask what is going wrong and how to solve them.
Not ideal, but the problem is that nothing really is :(.
You may have good intentions but what do you equates to "I don't trust you, bro/sis". Not a great way to start a relationship imho
That is possible of course. But if they wrote the code together, and know they will question later, you could teach the other person all the trade-offs etc. I could see myself doing that with a bad developer and get them through such an interview.
> Not a great way to start a relationship imho
Well, you could take the benefit of the doubt, and then when it seems they did cheat, just fire them. But firing is always such a big decision.
It's a difficult problem.
How often would you say it ever happens? This feels like a rather extreme level of paranoia.
And in this case you're eliminating people who aren't willing and/or able to spend time preparing for your interview process. One of my previous jobs had a clause in the contract that all my programming work belonged to my employer; is your company going to sign a waiver that says they take responsibility if a previous employer sues? Doesn't matter whether or not it's enforceable or not.
What if you use that solution in production, do you compensate for their time in helping solve the problem like you would if the task was assigned to you?
Also, you're not really setting a level playing field here. How do you gauge a candidates experience/seniority versus the other 10 candidates you hired if you're not asking them the same questions or looking for the same things. Maybe I've solved that bug in my current job.
All this is to say that this doesn't "just fix" hiring - I don't know if it's any better or worse than giving people on-site tests, but you're at the very least trading one set of problems for another.
But try that in Java and you'll often see a 3x-10x difference, especially if you run into problems with boxing. Streams are known to be notoriously slower than for loops with index. I once have even doubled the speed of a for loop by unrolling it manually (Java 8). Not all compilers are equally strong at optimization.
Anyway, asking such questions in an interview for a junior/regular developer is just a distraction.
at least for JS VMs, this is true today. this answer will certainly vary for other languages/runtimes.
i profile and optimize a lot of JS/TS code that handles datasets with millions of datapoints at my day job. but you don't need to take my word for it; this claim is not exactly difficult to verify.
> Do you just play along or begin to explain how incredibly wrong they are?
in this case, i'd prefer to be proven wrong with code, rather than prose.
1. require a complete rewrite of an existing, massive codebase
2. where the primary task is rendering interactive charts for a React web app
Total nonsense. It is not true of JS today, not has it ever been true.
No modern web app's or website's performance is dominated by the performance of what loops you choose. It isn't even remotely meaningfully impacted by that; by this I mean, any differences are totally unmeasurable beyond some 3 line micro-benchmark.
You have never benchmarked or profiled your code.
Of course, tight inner loops for image processing or ML, etc. yes, this can make a difference. But we're only talking about a handful of loops that matter out of many tens of thousands an application may have, you are almost certainly never writing those loops, and that's a specialized field that basically has nothing to do with the web. And it isn't what you're talking about.
> in this case, i'd prefer to be proven wrong with code, rather than prose.
What absurdity. And this is exactly when interviews suck. Because people have strongly held beliefs that are ungrounded in reality, and since there are no objective measures by which to rate interviews you need to pretend that they're real, because a lot of interviewers are basically just testing "cultural fit".
But hey! I have to say. That if you brought this up in an interview. I would at least know to never take the job!
Having a set of common knowledge to interview about is in fact very useful.
It's like a dance. Your are supposed to lead deriving the solution, interviewer is supposed to follow closely and judge your performance.
If you expect to ace an interview just like that then you also need to be lucky not accidentally getting asked to stuff, which you always need to Google an answer to, basic algorithms and patterns, which you usually kind of know but could slow you down during the precious 40 minutes you have to show off your skills.
And lastly, people do prepare for these. If you do not do that, you might simply make a slightly worse impression than someone who did.
That being said, I mostly did not prepare for the coding interviews. I focused on the design and behavioral interviews.
Also, I would also be quite surprised if Google didn't allow candidates to Google information during the interview...
I did a Google onsite several years ago (probably 7 or 8 years ago at this point). Each interview was just me in a room with a whiteboard and a person sitting at a table across from me with a notepad. No computer present.
I suppose I could have dug out my phone and Googled, but that feels unprofessional to me, moreso than if I had access to a computer.
Also the part I got stuck on was coming up with the trick needed to solve the problems. Most I was fine, but one in particular would have taken some decent searching to find something equivalent if I just Googled, and would have taken some time to grok the answer. Also that guy was not very forthcoming with hints or information when I asked him, so I doubt he'd have appreciated a Google search.
I don't remember the exact question, but it had to do with converting a 3D city model into a 2D skyline by drawing a single line. I got stuck on trying to visualize how to deal with partial overlapping models and irregular building shapes and ran out of time.
At FAANG, most people are not doing any of these things. Think of people working on EC2, Ads or Maps.
I only worked at multiple FAANG and I never had to design a database schema or make data available via API.
It's not hard to do those.
And I suppose for most business cases out there, writing data to the DB and reading it back later is all it takes.
However, there's plenty of other problems where the skillset you mentioned falls short.
For instance, you talked about designing a database schema. Do you know how to design and build a database itself from the ground up? What if that DB has to be distributed, with thousands of writers? How do you ensure consistency and consensus? (These are all solved problems).
Think about Linux kernel developers. Do you think DB schema design is an especially relevant skill for them to have?
Can you come up with a compiler, or your own programming language?
Try building your own text editor, with features like syntax highlighting, undo/redo etc (it takes clever algorithms).
I'm not claiming I can do all of that; just mentioning that the whole scene is much more interesting beyond the horizon.
I also think the skill cap on this sort of question is super low.
(I’d put design questions in a different category though)
And yes, people who know technology will do better.. in a technology interview. I suppose I could ask about gardening but that doesn't help much does it?
I’m not talking about people who know technology generally. I’m talking about people who know different technology. The process you’ve proposed, if implemented at, say, PHP shop, will rule out plenty of great .NET developers since the interviewer won’t have the context.
I’m not convinced it’s really an improvement. I’ve worked with too many people who are great bullshitters but terrible engineers, who sailed through on their charisma.
But that’s why most companies have a mix of technical and behavioral interviews.
Talking about higher level stuff doesn't mean you can comprehend certain development problems.
Indeed. The only way such questions are useful, is as a starting point, maybe they make the candidate more comfortable (less nervous), and they also ensure that you're not asking something the candidate is completely unfamiliar with.
But in any case, any such "experience" or generally "subjective" question must be accompanied by further detail-oriented questions - why did you do this and not that, tradeoffs of different solutions, "how does this framework/library work underneath", questions about understanding performance/security etc characteristics and why, etc.
Anyhow, several years ago, I added a disclaimer to my public profile on LinkedIn which states that I refuse any form of live / timed coding assessments during interviews and I'm not going to make any exceptions on it.
Some places have phone screens, and some don't. One company gave an easy leetcode problem as part of the phone screen, while in another case, one person actually spoke characters over the phone to me ("LinkedList", "left angle bracket", "String", "right angle bracket"). Some give take home questions. Some do a code review interview. Some dig into my resume during an interview, and some don't. Some companies had multiple interview sessions where I was asked the same question about my resume in each session by different people (wasting time). And so on.
So in general, I can and have passed over companies based on their hiring process.
The variations you’re talking about are quite minute. Whether someone asks you about your resume or asks you about your most technically challenging project is all relatively mundane stuff.
No company has thought through every little bit of their system and processes because no company has the time to be completely original, just like no human idea only has ideas that they thought up themselves instead of borrowing a bunch of ideas from school, books, and overheard conversation because they sound right.
It's ok to not have an answer.
Google gives even junior developers multi month projects to own, manage and complete. No Scrum shop gives developers that level of autonomy, not even to senior developers.
So to me the time I've spent working on Scrum projects feels like a sweatshop compared to how I could plan and structure development at Google. At Google I am free to collaborate with stakeholders, build prototypes and get feedback etc, as I see fit to complete the project. Or not do it when I don't feel it is needed, the important part isn't how I run the project but that I run it well. If you don't give your developers that level of autonomy then I'm not sure why you'd care much about developer competence at all. (I quit Google a few years ago though due to how the company was changing, but I'll never join a Scrum development team again)
This is a sign that the hiring process is broken. It's not testing skill level, it's testing anxiety level.
People who break down and cry in interviews are rare but if they do it, they are best avoided and it was good that the interview revealed this. You cannot have that happening on the job but if they can't handle being asked to program something whilst up against a time limit or being watched, they're going to have breakdowns in other situations too.
Because you asked them question out of syllabus. Was the person informed that the interview wouldn't involve Leetcode?
The person spent months prepping Leetcode just to get into BigTech and then you come out of woodwork with your "special" interviewing material. Imagine prepping for a calculus test for months and then the teacher comes out and takes a "simple" test on trignometry. Would you be able to do it? Not only that, all his/her friends got a normal calculus test which they passed with flying colors.
This had to do with the pressure one puts on themselves for a job a BigTech company.
Good for you.
Now seriously I'm not sure what you're arguing for - you're saying FAANGS don't do Leetcode style questions? I get it that YOU don't do it to candidates in those roles, but the reality is for SWE roles this is how it is. People who aren't very good in Leetcode will not succeed. There's hundreds, maybe thousands of candidates for these roles and some of them can do the Leetcode thing - they are the ones who get hired. I'm not advocating for this method because I'm not particularly good at it or like solving these things on my spare time, nor do I think this method is particularly useful - but that's reality.
A long time ago I managed a developer who was a genius at finding and fixing bugs - so I thought I would reward him by writing a new module we needed from scratch. He could not do it - when faced with a complex mess of existing code he was perfectly happy and could dive in and fix stuff, faced with a blank sheet of paper he froze and didn't know where to start.
Developers are people and all people are different - we all have our strengths and weaknesses.
It's _very_ easy to measure how fast it takes a ticket to go from "in progress" to "done" but measuring whether code has bugs, or leads to downtime or is just impossible to parse is a different problem.
If you never need code to be readable or documented I can write it quite quickly. If you want something other humans can have a hope of maintaining, that might take a bit.
That I can’t tell you to two decimal places exactly how much better the first list is than the second doesn’t mean I can’t tell who’s on which list with high fidelity/repeatability.
I know a bunch of people like that, but I've only worked with so many people. A startup would exhaust their friends pretty fast and be back to interviewing strangers.
Over time it has made me question if my short list is as good as I think it is. Maybe I am biased and only observed good/bad work for people on either list. Also people change over time and take on new responsibilities that change their work.
In my experience the top 20% of employees do 80% of the work
That would make them 16x as productive compared to the bottom 80% who does 20% of the work. So 10x seems to be roughly the same thing as the 80/20 rule.
Besides a 10x developer cannot be uncovered through a leetcode test or a coding test. A 10x developer pushes back on requirements, bridges gaps.. identifies issues early.. it's about making things happen not typing faster or knowing the stack well
The worst are net negative productivity
I'd say there's a big difference between a company finding engineers and engineers throwing themselves at a company. The latter is what often occurs with BigCos. They have a big pool of candidates throwing themselves at the company, and the company gets to pick whomever they like from the pool. How much credit to due to the company for picking? Whereas I think you'd give a company more credit for going out and specifically recruiting/poaching one person they want, even if that person wasn't applying for a job with them.
Lots. Firstly for becoming a place people want to work. Secondly for sifting through all of those people.
I think this is a huge mistake. I work in data science, and nearly universally my colleagues are motivated by opportunities to learn. If you're testing for the very rare day, so that your employee has the right answer in their front pocket when that happens they are never going to feel challenged, feel like they're growing, and they're going to jump ship.
I think you're doing snobby filtering, and it's not to your benefit.
On the hard days, the answer has never been seen before, and therefore can’t be in anyone’s pocket. You need to know that you have employees who are smart enough to discover it. Technical interviews are how you find them.
The observation that your employees are motivated by learning opportunities is useful for how you manage them, but this doesn’t tell us how to hire people who can do the job.
There are a few problems with this. If your "hardest day" deviates far from your "average day", while being very infrequent, you're still not doing a good job enticing the candidate. Worse, by looking for people who have already acquired the skills to combat the worst you can throw at them, you're severely stagnating their growth.
Then comes the cherry on top, if your company is doing it while not providing absolutely stellar benefits to function as external motivation, you're further incentivizing people to work on ditching your place. Many companies do not give total comps in line with the skill level tested for, setting themselves up for failure in this regard. Altogether, you're still looking at this primarily from the job interviewer's perspective, not the perspective of the candidate which translates into the company's long term (lack of) health.
>you will eventually have a problem that nobody can solve
No. Someone has to be the first person to solve a question. This strategy doesn't take into account the capacity of individuals to learn and overcome challenges. What's more, most individuals will not be on the highest level, allowing mentors to train them to higher levels.
>I don’t necessarily disagree
It's not about disagreement. Schmidt and Hunter did a meta-analysis on this way back in 1998, and psychology has yet to find the silver bullet. Look at table 1[1]. Most studies checking the market find similar things. It is ridiculously difficult to find correlations between job performance and selection criteria. All corporates, including FAANG, are not exempt from making the very human mistake of doing something which seems great, but ends up being part of survivorship bias. In something as multi-faceted as the job market, combined with the young and everchanging nature of software development, making statements on correlations shouldn't be taken for granted.
Notice how the study above points out years of experience as one of the worst predictors. Most of the job market still uses YoE as if it is gospel. The market is nowhere near as rational as people like to believe it is.
[1]: https://www.researchgate.net/publication/232564809_The_Valid...
I've said that even if you hire people who do know the basics of the language, and some algorithms, and even could solve a toy problem on paper, you might still hire someone who can't code at scale. Coding interviews typically don't test design skills, documentation skills, ability to not get lost on a larger code base, debugging skills, tendency to avoid hacks and increase tech debt etc.
The question changes to: is what you are testing the actual skill you need? Are you hiring a teacher who is going to explain var,let,const as part of their job or are you hiring someone who needs to know how properly to use var,let,const?
Send me over any developer who fails your teaching requirements for var,let,const but has code samples with proper usage.
I've never seen a senior developer fired for poor coding. I've seen them get fired for being part of a department cut or because they got into a fight with the owner/boss. I've seen them fired because they make too much compared to outsourced resources.
I'm sorry what? I don't expect them to teach me, I just want them to tell the difference. By the way, the ones that can't say: "they told me to always use let". I'll send them to you :D. Best of luck.
> I've never seen a senior developer fired for poor coding.
I've seen plenty. If a junior dev needs to help you out of trouble every time, you know there is something wrong.
Some people just reach their limit very soon, and can hide away in some comppanies. Probaly companies where you seem to work. Other companies probably just have a higher bar, no disrespect meant.
A junior helping a senior? Shame on them? Fire the senior? That sounds really foolish. You realize that junior/senior/etc are labels and someone you hire a junior can know more syntax than a senior in a framework. That's okay that the senior seeks helo.. it's natural and what a senior should do. p You are presenting your company in a bad light and making yourself sound inexperienced.
Could you have reached your limit? Would your methods fly elsewhere? Apply the len to yourself and see where you end up.
What I read is a knee jerk justification for your hiring methodology without being open to any feedback.
You should take a breath. Clearly people disagree with you. Try to empathize with where they're coming from and understand the point they're trying to make.
No doubt there are situations where hugely optimized code is necessary. An interview isn't one of them unless you're hiring someone to write exactly that code. Most companies aren't.
knowing the difference, and consciously deciding that it doesn't matter for a given task is often a strong positive signal. hopefully a good interviewer will not actually ask you this question if it doesnt matter for the given task.
The question you're asking involves 10k * 3 * 30 = 900,000 data points. If a candidate wants to simply loop over all of them multiple times a second, I'd say that's a much bigger problem.
unless the task is to animate them in a rAF loop (which runs at ~60Hz), right? the task matters. in some cases the question is stupid & irrelevant, and in the other case it means getting the job done, or not.
the difference between the brain-dead way and the fast way is literally 30x (53ms vs 1.8ms on my machine): https://jsfiddle.net/3ho6f5k1/
Isn't that why managers get paid the big dollars, to make the big decisions?
The whole rhetoric of the tech industry — taking risks, creative destruction, etc. — seems to completely fall apart when it comes to hiring, where everyone turns fearful and ultraconservative.
A bad manager completely tanks the productivity and morale of the entire team too. And you can't fire your boss, you can only quit your job.
> I’ll also point out you are not doing that person any favors by hiring them and firing them one month later (if your org is even capable of doing that).
Not sure how it's relevant whether you're doing the incompetent person a favor or not? Hiring isn't about favors, it's an exchange of money for work. If there's no useful work, then the money has to stop.
> Bad hires are disastrous.
Seems a bit overstated to me. Do you have any examples of companies put out of business by a bad programmer hire?
Coders come and go. Is it a "disaster" if your best coder leaves for another job? We're treating hiring as if it were like marriage and divorce, but it's not. When you hire someone, it's not "til death do us part". Breakups are expected.
Too many companies make hiring much more difficult than it needs to be. They overthink it, and also waste way too much time on it. Find someone who seems right for the job, and hire them. If they don't work out, get rid of them and hire someone else. You might get unlucky at times with a bad hire, but you might also get lucky with an unexpectedly great hire. If you can't correct your mistakes quickly, that's a problem with your company's culture, not a problem with job candidates. I would argue that it's not a bad hire that demoralizes a team but rather a bad company culture that's incapable of correcting mistakes.
Knowing how to implement prefix tree or a hash table shows that you've learned solutions, but not that you can write good code in a team environment. Frankly I'd take a worse engineer that worked better in a team over a 10x that ends up silo'ed in 95% of cases.
A good algorithmic interview does not test algorithm trivia. The question should be solvable with only strong structured thinking and fundamentals.
For example, the much-derided “reverse a binary tree” question (I don’t ask it but I think it’s reasonable) doesn’t really require a ton of algorithm trivia. The data structure can be explained in a few minutes and the problem definition can be made super clear and crisp.
https://news.ycombinator.com/item?id=23848039
Maybe the content, despite it being able to be gamed and drilled via Leetcode et al, is sound, but the format in which it is tested may not be.
People cargo cult because there isn't enough time to exhaustively research every topic from scratch. Civilization exists because we blindly trust whatever methods seem to work, and then iterate on them.
> Cargo cult programming can also refer to the practice of applying a design pattern or coding style blindly without understanding the reasons behind that design principle. Some examples are adding unnecessary comments to self-explanatory code, overzealous adherence to the conventions of a programming paradigm, or adding deletion code for objects that garbage collection automatically collects.
Installing an app from the store is not cargo culting. Playing with a new framework is not cargo culting. Trying a new order at the donut shop is not cargo culting.
the problem really is that results for interview process does not seem to reliably add up to anything, there are no reasons behind any interview methodology in the traditional cargo culting sense because there is no interview methodology really founded on reason, it's mumbo jumbo all the way down.
But back to my original point some posts up, everybody has at some point taken something on faith as a best practice that works without understanding why.
Some examples where cargo cult programming are concerned are adding unnecessary comments to self-explanatory code - but how about also one I've had requesting the removal of explanatory comments from code because the code should explain itself (comment was something like, have done it this way instead of what might seem like the more reasonable way ..example code.. because of a bug in Safari that means X happens, etc. etc, if you want to removal or change to better method please check that Safari works now and this place actually wanted it removed because they preferred not to have comments in code!)
Or how about this one which I am old enough to have done: putting your script tags in the head of the document because this was a best practice (before async and defer attributes, I'm talking 1999-2000 here)
Hell many of the best practices around CSS usage have depending on the year been the exact opposite of some other earlier best practice. And many frontend developers do it because they don't have the time or evidently the inclination to really learn CSS.
Hey, many best practices regarding CSS at various times are based around the fact that frontend developers don't want to learn to use it.
Now you might say those are not really examples of cargo culting, but I think they are somewhat close - if you are using a naming convention to avoid learning how something works then you don't really know why you are using the naming convention because the purpose of that convention is to keep you from that knowledge.
anyway getting off track here, I just think that saying you are not going to work at a place that has cargo culted its interview process does not tell you anything about the overall quality of the place, because everyone has cargo culted something, just like many fine developers when setting up their React projects have said oh, everyone uses Webpack we'll do that as our build process - they know they need a build, but why Webpack? They don't really know (me neither, every time I've ever encountered Webpack I've always ended up wondering why!? Generally the answer is because that's what the tutorials had, everyone used it, they didn't give any thought, but now we're stuck with it)
If they mentioned the fact that React 18 batches updates in concurrent mode which would greatly improve the responsiveness of a dashboard app that's updating lots of small components frequently, or if they suggested the graphs might be better written using GLSL shaders to move the rendering to the GPU so JS only needs to update a uniform array, then I'd pay attention. Potential new ways to approach a problem or changes to architecture to avoid the problem entirely rather than micro-optimizations are much more interesting signals about someone's deeper knowledge of building things.
Look, I have a lot of experience with the hiring process. It's very difficult. I will listen to someone with a lot of hiring experience. But just saying "it's broken" isn't going to cut it.
And remark that a big part of our hiring practice is to put the candidate at ease. I even say that they won't get everything right, and that's no problem. I help them out when they get stuck, etc.
I can't really believe that people here think when you have more years under your belt, you somehow must be good. People get fired in companies because they don't reach the expected level. If you keep them, other colleagues will get frustrated and leave on their own. And then you're stuck with the underperformers.
And yes, everyone has limitations. Some sooner than others. Welcome to reality.
I find the discussions here very silly that somehow you must hire everyone without a technical interview, and never fire anyone because people don't have limitations. Not going to respond anymore.
People out there who can’t code interviewing for SWE roles, I can’t speak to, but there are definitely folks being asked to be fluent in code for non SWE roles who aren’t SWEs, so it makes sense everyone gets put through the leetcode grinder rubric to calibrate their level. I don’t think this is about misrepresentation, but expectation misalignment and a crude tool being necessary to evaluate human skills against a baseline.
(nearing 40, 22 years in tech, getting out asap, ymmv)
I’m a staff engineer and tech lead at a MANGA company. I code in up to 5 languages a day. I’ve written Network stacks, compression software, virtualization software, and the backbone of multiple PaaS platforms. And they were focused on whether I knew niche algorithms (apparently based on EXT3) and other stupid stuff.
That's a new one to me, what's it made of?
(All that's coming to mind is I presume totally unrelated: https://en.wikipedia.org/wiki/Manga )
I interview tons of software engineering candidates (pre pandemic it was at least one or two a week). At least half of my senior candidates fail basic phone screener questions.
These are questions that reflect something they'd be doing day to day in the role, but also are incredibly standard in our field.
The issue in most of them is that they've taken their support network for granted. The tons of developers before them who laid groundwork, and the others currently who take care of many tasks they consider junior.
Well that leaves them in a weird spot where they've not actually done a lot of the foundational work their predecessors set up for them, and they've let a lot of their skills atrophy by delegating to juniors. They'll even say as much and say "oh so and so does that, or they had it set up and it's great"
But then what am I hiring them for? Certainly they're not bringing a wealth of programming ability over, so maybe soft skills? That's hard to hedge bets on for a SWE, maybe as an product manager or something.
Again, at least half of software engineers I talk to for the screening round are not strong developers for the reasons I mentioned above.
At my last company, I joined as their lead developer and discovered they already had another developer in the pipeline they were thinking of hiring. They thought the guy was super senior, and when I read his CV sure enough he came across that way. He emphasized on his CV that he was not only a Java expert but an expert on the internals of HotSpot itself.
Probably he'd been saying that for years, and given how many firms use the JVM it's probably a great line. Unlucky for him he got an interviewer (me) who actually was an expert in the HotSpot internals and liked to relax by reading its source code. Guess what? He knew nothing about the JVM, not even stuff you'd learn by reading the user guide. I got the sense he'd read one or two popular press articles about it 20 years ago, put it on his CV and never updated it.
So we progress to some coding. He can't code. He struggles with even the basics of starting a new program in an editor and compiling it.
The guy was demanding a huuuge salary and great perks. After I wrote up what I'd found from the coding interview he was quietly dropped from the pipeline. They'd been about to accept because they were so dazzled by his claims but weren't checking them.
The "knew it once maybe sorta" problem is common, especially with C++ where lots of people were struggling with it in the 90s and then jumped to Java/C# as soon as they came out. But they still list C++ on their CV. Ask them to write a C++ program (that does anything at all) and they'll just refuse or state point blank they'd feel really uncomfortable doing that.
Lots of people lie about their experience unfortunately. They probably don't think of it as lying. They just write down every possible thing they've ever thought or read about and exaggerate, gambling that they'll never be called on it. And, they're probably right. That's why coding interviews took off.
Well, job postings are infamous for listing a ridiculous number of "requirements" that aren't really requirements. Those who are familiar with hiring know that job postings themselves are lies and encourage people to ignore the requirements and apply anyway. Even the people who work for the hiring company say this! So it's a two-way street. There are already perverse incentives created by the hiring companies. And we also know that resumes are screened for specific buzzwords. You may not be recruited or interviewed unless unless your resume has the right buzzwords.
I did a bunch of really quick 15~20 min pre-interviews to check if the person was entering the right process before we both wasted hours of our time. Some who were supposed to have coded and deployed whole applications didn’t know how their versioning system worked. Or they reviewed PR/MRs a lot and tweaked a few lines here and there but had little practical knowledge of issues you’d regularly hit when developing, and only heard of popular tools and resources by name.
We could still had them onboard and give them time to get up to speed, but it’s harder to do when they weren’t upfront about it and you’re there trying to guess what else they’re bulshitting on their resume.
there are a lot of them
Unless you're suggesting that knowing the performance of a loop helps with that?
I kindly disagree. You seem to have worked at a very stressfull place. In my experience that has nothing to do with it being "true scrum TM" or not. In any case, happy for you that you got out.
Also the leetcode interview is cargo-culted from big tech companies like Google and Scrum is cargo-culted from big tech consultants. They don't fit well together, having both means your company just picked the most popular/simple way to do each part without considering why Google uses that interview or tech consultants uses Scrum. If you want your engineers to work on problems similar to at Google then you wouldn't use Scrum, and if you want your engineers to work on similar problems as big tech consultants then you wouldn't use the leetcode interview.
If as you say, you had no autonomy/freedom, this has nothing to with scrum, that's just regular bad management.
To me it seems like Scrum was designed to make developers have as little individual ownership/responsibility as possible while still being able to produce relevant code. I see the merit in that, you don't have to tell me why minimizing individual ownership/responsibility could be a good thing. But I strongly believe that individual ownership is necessary to properly capitalize on the creativity of individual talent, otherwise its mostly wasted.
Is this actually true? It's been widely and frequently claimed over the years that "199 out of 200 applicants for every programming job can't write code at all. I repeat: they can't write any code whatsoever." https://blog.codinghorror.com/why-cant-programmers-program/
> frankly a bit insulting to people who learned to control their emotions
It's frankly a bit ignorant of human psychology to think that people can just "learn" to control their emotions. We're humans, not robots. Yes, you could go to a professional therapist, if you have both the time and money; it could take a lot of both. But if many people have to go to a therapist for the sole purpose of dealing with audition-style coding interviews, then maybe there's something seriously wrong with audition-style coding interviews. It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews.
> and work under pressure
I explained at length in the link from my previous comment that working under pressure is entirely different from interview pressure.
I've done hundreds of interviews as the interviewer. I've had people seriously panic maybe once or twice in that entire time. I've heard occasionally that the rates for other interviewers were higher (but still low), if that's the case then they just sucked at interviewing.
Where did therapists come into it? I've never heard of anyone going to a therapist to pass coding interviews, that would be ridiculous. Learning to control your emotions is something people are meant to learn as children. It's a standard skill you're supposed to have to interact with others, like basic politeness. If you're having breakdowns as adults then it's expected that the situation is really intense, like the death of a loved one, breakdown in a relationship etc. Job interviewing is a normal part of life and shouldn't be as stressful as those things.
"It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews."
The whole "cramming leetcode" thing is way overblown, or possibly a modern feedback loop in which crammers are making it harder to detect genuine skill and experience. Again I've done a lot of interviews and nobody ever mentioned having to prepare for them. That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms. Normal software jobs should ask people to demonstrate their skills, but it shouldn't be something that requires exam-like prep for any working programmer.
For a given job most candidates actually don't get hired, often because they are not a good fit but many times because they are too nervous to think straight.
How do you know their level of nervousness? You can't read minds.
For an array of 5-10 items the performance hit of foreach is worth it for readability. Maybe an interesting discussion could occur about when to sacrifice readability for performance but you won’t get that by grilling people about minutiae.
As always it depends, maybe that component is being called 500 times on the page making it actually 5,000 invocations.
I'd FULLY expect a candidate to understand the difference and would ask in a whiteboard session. There is a lot of bloated slow javascript libraries out there and they almost always stem from the fact that most devs don't realize how much overhead function calls introduce.
For the interview, there is no discussion necessary, the correct answer is simply:
"I used a foreach for readability, because performance doesn't matter in this case because X."
or
"I used a for because performance matters in this case because Y"
End of story.
100% agree. my point is more about OP's example being used to prove that dumb questions get asked. if it's being asked by a good interviewer, it's usually relevant.
100% trivia.
Edit: And a job ad where they say you should be well versed in Scrum I take it as they will force Scrum on me regardless if it makes sense or not, because I doubt a team just doing Scrum for themselves would put that in a job ad. Companies loves putting this in their job ads though so it is very easy to spot which companies works like this.
I'm not saying that this way is strictly superior, but you undeniably have way more freedom under it than Scrum.
I think you're making too much of the example of the candidate crying. That is indeed an outlier case, not the norm. But the crying case is an undeniable sign that job candidates can get extremely anxious during an interview, in a way that affects their performance. The issue isn't "breakdowns" per se, the issue is that coding tests are testing anxiety level rather than skill level. Ironically, most job candidates are in fact "controlling their emotions", as you say: they may show no outward sign of breakdown or panic, but on the inside, their mind and stomach are churning heavily, and this makes it difficult to perform at their normal level. Everyone is trying to act calm and collected during an interview, but often it's merely an act, and they're actually not calm and collected at all. What you perceive from the outside as incompetence may be a whole different story on the inside for the candidate. https://www.engr.ncsu.edu/news/2020/11/11/tech-sector-job-in...
Anxiety isn't all or nothing. There's a wide spectrum between totally calm and crying breakdown. When there's a crying breakdown, it's easy to recognize, but anxiety is not so easy to recognize if there's not a breakdown.
> I've done a lot of interviews and nobody ever mentioned having to prepare for them.
Why would they mention it, even if they did prepare?
> That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms.
Regardless, we're living in the present now, under current conditions, not living in the past, so for the purposes of getting a job, it doesn't really matter when the idea originated.
You seem to be suggesting something totally unfalsifiable - that candidates who do a bad job aren't actually bad programmers, but are just nervous inside in undetectable ways. There's no way I or anyone else can argue against this notion because there's no way to prove a negative like that, but suffice it to say that the sort of issues I'd associate with nervousness - minor slips that they then notice and correct, stumbles, brief moments of forgetfulness etc - are not why people tend to fail interviews. They fail interviews because their coding skills suck.
Heh. It's only unfalsifiable if you ignore a person's previous experience and accomplishments, refusing to acknowledge anything other than your own interview as an accurate gauge of the person's skills.
Also, I'm literally telling you that it happens to me. (And I've heard many people say it happens to them.) So either I'm a liar, or you have to admit that it happens. Go ahead and call me a liar if that's what you want to believe, but I'm not even trying to get a job anymore. I'm happily self-employed now, and my customers also seem very happy with my coding skills.
The term "control your emotions" is somewhat vague. You can hide your emotions, but that doesn't mean you're not experiencing the emotions. An extreme example of that is if you "put on a brave face" after a personal tragedy. You may not cry and break down, but that doesn't mean the sadness doesn't affect you severely. The reason I mentioned therapy is that you'd probably need therapy in order to somehow train yourself to not even feel the emotions in the first place in situations where they arise automatically for you, i.e., to not have certain phobias at all.
> the sort of issues I'd associate with nervousness
Have you considered that you might not have a full understanding and appreciation for human psychology and the wide variance among humans?
Which is often the right thing to do unless you have very specific knowledge of that experience e.g. a referral, because so many people flat out lie about their experience and accomplishments. If you can't directly verify it, the accuracy is so low it's best ignored.
As for your experience, I'm glad to hear you're now happily self employed (so am I!). How are you proving your skills to customers, if you can't handle job interviews? Or are you making a product?
"Have you considered that you might not have a full understanding and appreciation for human psychology and the wide variance among humans?"
I have an average such understanding, having lived an average life. But the problem remains: someone who claims to be able to do a job fine but can't demonstrate that capability when asked, is from an employer's perspective best avoided. That isn't a flaw in the interviewing process, that's the point of it! The reasons why someone can't demonstrate it are irrelevant because there's no better alternative available, and anyway, someone who can't handle the pressure of a job interview is very likely to crack in other situations. You claim it's not true and there's something psychologically unique about interviewing, but, I don't really believe this and clearly neither do other people doing hiring.
You're missing the forest for the trees. This is not a job interview, it's just the two of us talking, and I'm stating that many people who are not stage performers experience "stage fright" in an audition-style interview, which severely but only temporarily hinders their performance.
If you acknowledge this fact, then you have to start to wonder whether audition-style interviews are accurately measuring skill level, or whether they're mainly measuring anxiety. And you may have to reevaluate whether a flubbed interview indicates that a candidate "can't code" and is a "faker", as opposed to just freezing from stage fright.
This is not even to claim that there are zero liars out there. But you have to wonder about the percentage of liars, if there are alternative explanations for poor interview performance. The level of paranoia in hiring programmers seems totally absurd to me.
> How are you proving your skills to customers
I am making a product, as well as contracting sometimes. But nobody wants to test me. They trust my experience and aren't ultra-paranoid that I might be a faker who can't code.
> if you can't handle job interviews
This is not accurate. I can handle job interviews. I'm not good at auditions. An audition is a very specific kind of interview, and rather perverse for hiring people who mainly work sitting by themselves.
> That isn't a flaw in the interviewing process, that's the point of it!
Disagreed. It seems that coders tend not to recognize how coding test interviews are not the norm in the world for interviews. They're quite unusual. Most people don't have to audition for a job. An interview is a two-way street: both the interviewer and interviewee are trying to determine whether the job is right for them, and whether they want to work together. A test is very one-sided.