1. Dunning Kruger in 1999 on overconfidence when unskilled.
2. Selection bias in favor of “quick thinkers”
3. Compounding effect of selection bias in capitalism—-often survivorship in decision making theories.
Uh huh
I guess you want to say that it’s a poor metric for intelligence.
Not so sure. IQ is measured in standardized tests. Participants should have the same amount of time to answer the questions. If intelligent people take longer to reach the correct answers than non-intelligent, then, it seems paradox.
The idea could be to prolong the test. So we better see who reaches the correct answers without much time pressure (but still with the same amount). Or go even further and ask really difficult questions without any limit.
These would probably also measure the ability to concentrate, or motivation to continue, … which might not be what you want for an IQ test.
However, if your question would be true, designers of IQ test could fix them in light of this research: they plan for enough time so that even the most intelligent have time to answer the questions. It’s of course a problem to find a couple of really intelligent people if your test is flawed in the first place.
A standard IQ test take an hour or so to administrate.
For a very long time, I just have not been smart enough to go to sleep in time. (Metacognition related "intelligence"?)
That's how you explore the space.
You can't expect a good solution before you've fully explored the space.
(This is not to say its false)
The article title refers to something that's been long known even in pop science since Daniel Kahnemann's "Thinking Fast and Slow". In fact, the original paper[0] even cites Daniel Kahnemann's work:
> Simulation results indicate that decision-making speed is traded with accuracy, resembling influential theories from the fields of economy and psychology on fast and slow thinking.
Kahnemann's work might be summarized (very roughly) as follows: "System 1" often quickly suggests an intuitive (and frequently wrong) answer, whereas "System 2" is the part that does the slower, more rational, and conscious thinking. "Intelligent" people tend to be those who control their System 1 and thus their urge for intuitive answers more effectively.
And indeed this is what the article says, too:
> Resting-state functional MRI scans showed that slower solvers had higher average functional connectivity, or temporal synchrony, between their brain regions. In personalized brain simulations of the 650 participants, the researchers could determine that brains with reduced functional connectivity literally “jump to conclusions” when making decisions, rather than waiting until upstream brain regions could complete the processing steps needed to solve the problem.
> “In more challenging tasks, you have to store previous progress in working memory while you explore other solution paths and then integrate these into each other. This gathering of evidence for a particular solution may sometimes takes longer, but it also leads to better results.
However, if I understand correctly, the thing about the research here that's actually novel is that they now have a better understanding of the neural processes underlying System 1 vs. System 2 ("we identified a mechanistic link between functional connectivity, intelligence, processing speed and brain synchrony for trading accuracy with speed in dependence of excitation-inhibition balance") and that they in fact simulated the brain digitally, see my other comment.
That doesn't work well in interviews, especially with how terrible most interviewers at time management. I sometimes get 10 minutes for a system design problem because the interviewer was expected to get signals on my non-technical competencies as well as system design.
This is never enough time to ask clarifying questions, diagram things, and get a good solution out unless it's similar to a problem I've already solved.
It's often OK to not solve the problem as long as you give an interviewer insight into how you think, but some interviewers expect a miracle.
It's not, though. Many people have no clue how to interview well, and way too many tech interviewers are obsessed with whether or not the candidate got the "right" answer.
Anyone who dropped multiple mediocre solutions before a good one in an interview with me would likely get a strong hire — I love to see this kind of iterative thinking, and finding people who can role model a healthy exchange of rough-draft ideas is always a great boon for the team's psychological safety (and by extension, their creativity and productivity).
But I think, industry-wide, it's not great. As a SWAG, probably ~50% of interviewers are more interested in the correctness of the answer rather than the caliber of the thought process and communication.
And this is why most the interview processes as practiced today are a joke. It's far more this weird sort of cargo-cult hazing process than any actual sort of reasonable assessment. Few people give challenging problems that they expect won't get solved to step through thought processes, they have some predisposed ideal solution or perhaps probably optimal algorithmic solution on the spot. That lends itself well to a combination of assessing rote memorization and chance, I suppose.
There are definitely engineers hunting max salary/equity over everything and they are going to jump ship if they can get a FAANG role. Being able to spit out rote memorized leetcode and systems design questions is probably correlated with someone who wants to maximize their salary.
I was a little irritated though because, though I was early in my career, I have no doubt that I'd have been a productive member of their team within a month. I had thought the interview went well - we discussed the problem they proposed in detail, we arrived at a reasonable solution by the end, I asked lots of questions and responded well when I was prompted about edge cases.
I was just fuzzy on implementation details and aspects of database design that I hadn't had direct experience with.
Anyway, I'm not bitter, not getting that job led to a fantastic gig that I still have today. But I did feel like they were focused on the wrong things in the interview.
On the other hand, many interviewers are just normal people who have no idea how to gauge "how the candidate thinks", but like to think they are. I saw this a lot in the heyday of Google's brainteaser type questions. (And middle managers seem to revel in it.)
So the right answer is as good a proxy as they can hope for. It still sucks, but there you go.
Some advantages - Interviewer and the interviewee are at ease. There is no rush to solve a problem. - You can easily spend 90 minutes to 2 hours on System design, Spend 2-3 hours coding and another 2 hours in behaviour/leadership what not. - The interviews can be progressive, meaning you don't make it through the first 2 hours - good bye. - This can be done remotely as well as in person. Of course, in person would be better, hosting expenses etc.
End of the day, decision is made and you can verbally convey an offer/reject.
This calls for a lot of discipline and commitment from the companies and their interview panel, I mean so be it. Dedicate 1 month for hiring and be done, at least for senior positions. Just like you allocate time for your projects, allocate dedicated time for interviewing every 6 months, every quarter whatever.
Do they pay you for your time for those long multi-hour interviews?
Is that knowledge available in real time? When I interviewed at Google last year, the next thing I heard back from the recruiter was that I'd passed the interviews and should expect a job offer. She wished me congratulations. A couple weeks later, she informed me that my interview scores were too low to get a job offer. It remains unclear to me why there would be separate thresholds for "passing" and "eligible for hire".
In "Thinking Fast and Slow", Daniel Kahnemann describes the "instant" answer as being given by "System 1", which the far slower (and more rational) "System 2" might distrust.
Deep wisdom here, indeed. Don't you dare say this during an actual interview, though -- especially when you're asked to do the system design part in 10 minutes or less. The last thing these companies could possibly want is to have reality to intrude.
And then proceed with my non-optimal solution. They often don’t care about optimality at all. I’ve passed interviews by implementing a bubble sort before. And I have no shame in doing so. I’ve never needed to implement a sort in an actual job, and if I did need to I would be looking up how to do it.
In the past I put it down my experienced approach and what I need from others: make it work (mediocre solutions that deliver 80% value are fine!), then make it better (reflect and refactor until readable/approachable/idiomatic) before finally (stretch goal) make it faster (break idiomatic paradigms, vectorize etc.)
At interviews they stop me at my first attempt.
Of course not - interviews aren't about assessing actual problem-solving ability (in the face of, you know, actual real problems). But rather your ability to recite stock answers to made-up problems the interviewer found on some website somewhere. That, and your ability to politely tolerate their horrible time management skills, laugh at their jokes -- and feign belief / interest in their product.
Experienced candidates give two solutions in seconds, juniors cannot figure out the task, and overqualified are visibly annoyed by being asked to come to a whiteboard.
"People with lower IQ jump to conclusions"
If you don't think about edge cases, you get "solutions" faster.
That's why we have so much technical debt.
Maintaining/fixing/managing the code that the star developer produces is very ungrateful work.
Surely a group of people who deem themselves 'intelligent' can broaden their perspectives a little?
From article: "Participants were asked to identify logical rules in a series of patterns"
The more intelligent you are, the more different rules you can come up with for a pattern. It then becomes a task of figuring out which rule the interviewer thinks is the (only) correct one.
Magnus Carlsen does not move his pieces faster than bad chess players. He thinks thru more creative options and see more useful "edge cases" than the bad players.
A lot of this thread is this sort of thinking, that the comment author is the bright one and everyone else is a dunce.
Inexperienced dev code the first solution they have.
Experienced dev challenge their first solution with one (or more) other solution(s).
I find such summary ambiguous. Difficult for who? Isn't it very common that a difficult problem for me is a piece of cake for thee, like the PDE that Bezos couldn't solve for hours yet his classmate could solve in a matter of seconds?
In 1697 Jean Bernoulli challenged the mathematicians and scientists in Europe to solve the brachistochrone problem. Bernoulli was particularly proud of his solution after working on it for weeks. Yet Newton, who was 55 years old at that time and hadn't worked on science or math for years, worked out a brilliant solution overnight and submitted the solution anonymously. Reading the solution, Bernoulli famously said "I I recognize the lion by his paw". Not only did Newton solve the problem, he also invented the Calculus of Variations.
This is spelled out in the article ... difficulty is relative among problems, with increasingly difficult problems within the same problem space.
What is "solve"? The example given was finding a route on a map. Is any route a valid solution, or only the best one? Does the time differ for finding a route of the same quality, or does it differ only for how long it takes until the brain is satisfied with the solution?
Abstraction solves a lot of hard problems, but it is also costly.
A typical IQ test (WAIS) takes roughly an hour to administrate.
Huh, I hadn't heard of this before. Does anyone know more about how exactly those "in silico" brains work and how they compare to their real-world counterparts? I mean, the article makes it sound as if the researchers fully understood how the brain works and had managed to create a faithful digital copy, which I find difficult to believe.
EDIT: The original paper says
> To study neuronal processing in silico we created BNMs [brain network models] for the 650 subjects using a tuning algorithm that fits each participant’s simulated FC with their empirical FC (Figs. 2 and 3). The BNMs use coupled neural mass models to simulate the electric, synaptic, firing, and hemodynamic (fMRI) activity of a 379-nodes whole-brain network. Each node consists of one excitatory and one inhibitory population that mutually and recurrently interact. To simulate long-range white matter coupling, the neural masses were connected by each participant’s SC, which were estimated by dwMRI tractography. Importantly, we added feedforward inhibition to increase biological realism
Sounds like they used small neural networks for simulation and adjusted the weights between the neurons to what they saw in participants' MRI measurements.
All I'm saying is I think it has more to do with familiarity of material than "intellect." Intelligence to me is being able to rapidly comprehend and assimilate new systems of logic and signification as they are approached. Biology is going to have different rules from Set Theory which is going to have different rules than Political Science. Someone might note that these things are all interrelated: which they are, of course. But I think intelligence here would be noticing the contradictions that appear when they are set in relation, not simply trying to understand how they all stand together in some grand, unified fashion; nor (on the other hand), making humble claims (like formalism) which are not truly held by anyone.
One student solved it in class, but the rest of us weren't told what the solution was. I tried solving it a bit in high school but still couldn't come up with a solution.
I really hope there is a solution that I couldn't find, and not that we were expected to discover that there is no solution. I expect that problems in the real world may not have a solution, but if you're going to pull that sort of thing in a 4th grade classroom (even a gifted one) I expect some setup so that the kids know this is a possibility.
Lower response time indicates jump to the conclusion that happened to be correct. The percentage of correctly solved higher-level problems was predictably lower for people with lower g-factor.
Not shocking.
> needed more time to solve challenging tasks but made fewer errors
I don't think this really has anything to do with "intelligence" as much as it does patience.
Who would you want to be your airline pilot: the person who solves 70% of problems quicker or who solves close to 100% of the problems slower? The ability to error check has a time cost but also is of infinite value for any position of consequence.
Fascinating..what a time to be alive!
1: https://collections.archives.caltech.edu/repositories/2/acce... Can someone verify that?
That's a remarkable generalization. In reference to the article, perhaps you should have thought longer about it.
(Not all of them, of course, but in general they are at least as intellectually-curious and open as anyone else I know.)
I gather you may not know any of these types yourself but they’re certainly abundant here on HN, dismissing Dropbox as easily replicated with rsync and other such sentiments.
I'm 6 months into my current role and only just starting to feel the confidence and comfort to question the approaches to problems we've dealt with in that time, in order to, hopefully, modify the approaches we take in future in the hopes of improving the quality of the output.
It takes time to "find the water level" and also working through applying that knowledge to each problem, and if time is 'pressurised' it can lead to suboptimal resolutions. On the flipside, no one will wait forever - and it feels as if the world is currently oversensitive to waiting time.
Find the water level. But that also takes time.
Dialogue is a necessity for the correct development of knowledge. To engage in dialogue we have recourse to language and a vocabulary of discourse. As knowledge is gained, dialogues branch off and specialize. The system of thought inherent to the ~unique arrangement of the elements of these dialogues partition thought into acceptable (an opening) and unthinkable (a closing).
Throughout history you will find minds that navigated these intellectual currents by stepping away from their dominant belief system and gaining knowledge of the universality of meaning, and saw hidden (filtered) vistas previously unseen, and then step back into their home grounds to contribute to the development of the field, in a positive manner, adding new elements into vocabulary of the dialogue. They extend it, and create the possibility of synthesis in the future.
Equally, you will find that the further necessity of establishing schools, cults, churches, and institutions, which lend social prestige to its members and satellites, introduce incentives contrary to that of pure love of knowledge, and this attracts a certain type of people, beyond the already present danger of vanity and self regard.
What is to be done about it?
Helpfully suggest that tolerance of this necessary evil may be the remedy for your gripe.
There is almost nothing to be done but just wait patiently.
Data driven hiring practices require hard definitions. One of these hard definition of intelligence is IQ. IQ is very correlated with job success and also various other skill sets related to job success.
From the company perspective, successful hiring using a data driven scientific approach is the right call simply because its the best metric we have on intelligence.
Obviously their are aspects to intelligence that currently aren't quantified but is it wise for a company to bet their future on gut feelings that are subject to bias and aspects of intelligence not in the realm of science?
Unfortunately no.
I'm pretty sure the Google interview process is somewhat of a data driven process but I can't be sure. Are any googlers in the know? Are specific interview question and answer pairs measured and correlated with the success of the employee? Or is it all qualitative judgement?
IQ, I believe, is actually not legal to measure for an occupation. But we know that this is the metric that faangs are actually attempting to replicate.
He actually ended up teaching me some calculus so I could grok trig identities, even though we were still in the precalc year. And let me use it on the tests to prove stuff that would have been expected to be proved geometrically.
The other big gripe I have with the way kids are tested is the way people are locked out of life because they're only good at one or two subjects to the extent they fail some of the others. Congratulations, you have failed at school because you, a teenager, can't meet our minimum requirements for being able to write inane, shallow analysis of 19th century poetry, can't run n meters in less than m minutes, and have failed to learn German. Good luck pursing your prolific talents with no high school diploma kthxbye.
I completely failed an English course and still graduated. Even went on to fail classes in college. Had to retake a class but still got a degree.
I haven't taken a test in years, but the number of abstractions my mind would cycle through was a major disadvantage on timed tests.
This may explain what happened to me. Of the various standardized tests that I have taken in life (SAT in 8th grade, SHSAT, PSAT, SAT, ACT, LSAT, GMAT), the LSAT is the only one that I did not score in the 99th percentile on, and its logic games section was my weakest. I have never had any test prep training of any kind other than taking old exams; if I had taken an LSAT class perhaps I might have approached the section differently.
At work you get stuff done rather than answer questions in a obscenely short amount of time
THEN I stopped, and thought:
"Hey, waait a minute! Is the article trolling me with a 'difficult problem' here? which I will now proceed to take a long time to 'solve'? because I'm 'intelligent'? well, screw you, troll - I can solve this quickly!", and so, I hastily clicked on something in order to make the popup go away.
Thus proving that I'm not intelligent.
I win.
HNer isn't at all intimidated by the wall of text and knows exactly what it mean but reads every word to check they haven't snuck any particularly egregious privacy violations and goes down the rabbit hole of evaluating the likely implications of blocking 'necessary' cookies and the virtues of doing that via the cookie popup or browser plugin before making their selection...
Could we please get an ISO for usability, and regulation that requires compliance?
(Should still be styled as a button, however.)
The charitable interpretation is that the irony here was intentional
What's even worse though, you can show up, pass the final exam for a subject and still fail the subject due to "insufficient attendance" throughout the year. Which is what happened to me. I was skipping class too much to write code in my room, which ironically meant I couldn't get into a CS course even though I could code circles around the well-behaved neruotypical kids who showed up for class and did their homework.
According to Jony Ive, Steve Jobs was exceptional at nurturing this kind of creative environment. He describes it perfectly in this WSJ article [1] from a couple years ago:
> As thoughts grew into ideas, however tentative, however fragile, he recognized that this was hallowed ground. He had such a deep understanding and reverence for the creative process. He understood creating should be afforded rare respect—not only when the ideas were good or the circumstances convenient.
> Ideas are fragile. If they were resolved, they would not be ideas, they would be products. It takes determined effort not to be consumed by the problems of a new idea. Problems are easy to articulate and understand, and they take the oxygen. Steve focused on the actual ideas, however partial and unlikely.
The other part of it is that ideas, as fragile as they are, can blossom and transform if you can get multiple people and multiple different perspectives involved in the early stages. That's why it's so powerful to have this kind of culture on a team; it lifts everyone up.
The problem with polishing the idea into a final draft before sharing it is that — whether or not it's good — it's almost guaranteed to not be as good as it could have been. And, by the time the "final draft" is ready, it's too late to get involved — all the interesting rabbit holes and tangents and possibilities have already been pruned away.
[1]: https://www.wsj.com/articles/jony-ive-steve-jobs-memories-10...
The problem you describe is the part that is directly addressed by psychological safety. If you can provide a wrong answer and have people critique it but without ridicule, then you have a “safe” environment where people are comfortable to provide solutions without certainty on their optimality.
The point is that participants feel safe challenging an idea.
What probably happened was that the recruiter looked at your profile, read the comments, and expected you to pass the HC (and told you the next day)… but you didn’t in the end.
Now last year Google implemented a hiring freeze, so you could have fell into that. I would believe that for easier-to-fill roles, during the hiring freeze they suddenly had a glut of candidates who “passed” with no job to give them, so the suddenly “failed” and we’re sent away.
I think this was earlier than the freeze. It was certainly earlier than any awareness of a freezing job market made it onto HN; the email from the recruiter promising to give me good news on a phone call tomorrow -- a rare example of a Google recruiter putting something in writing -- arrived on January 12 of last year. There was a very long lag between that call, described in my earlier comment and presumably taking place on Jan 13, and the followup telling me I'd been rejected on March 2. I undersold that by describing it as "a couple weeks".
On the other hand, according to my memory I was informed that I should see a job offer by the end of February - I noted at the time (to myself) that the schedule was surprisingly long - and in the light of your comment, that and the timing of my eventual rejection do tend to support the idea that the hiring committee's next scheduled meeting was at the end of February.
This is different from the more linear directed thoughtpatterns promoted in institutions. I often find people in the HN crowd to be more in the latter crowd than the former, but nonetheless intelligent.
When I speak to artists or people in more creative fields, I find that, typically, the conversations are more improvisational in nature. The conversation just ebbs and flows naturally, with energy and a lot of spontaneous thought building. The strange book Impro is a good read for this.
Whereas, when I speak with engineers or scientists, it feels like the conversation has a much higher chance of lurching, especially when you start to veer off the beaten path.
I consider myself to be a bit more divergent than most, but being in a technical field is a bit of an impedance mismatch. I think a component of divergent thinking is sometimes extreme thinking, wondering about what if and ignoring, for the time being, how to get there.
The book impro, who is it’s writer? I see a couple of results coming up.
> I have no doubt that I'd have been a productive member of their team within a month.
Did you consider that they might have skipped you not because you were "fuzzy on implementation details" but because you projected that vibe "I don't know much about what you do but I bet I can be ask good as you at it in a month"?
When I interview junior candidates, attitude, interest, and capacity for learning are more important to me than specific knowledge.
Anyway though, it was their choice, and I clearly didn't demonstrate what they wanted to see. Not getting that job left me in the right place at the right time for a job that I really love, so... ¯ \ _ ( ツ ) _ / ¯
It's ironic that the dynamics of targeting higher quality candidates are not that different from personal dating. While for larger organizations they seem to stress how much more damage a bad hire can be they have the capacity to absorb losses better than substantially smaller organizations on a purely mathematical basis. Likewise, individuals that have higher resources, emotional intelligence, capacity for grace, etc. can much better tolerate "bad" relationships yet are among the most stringent at avoiding "bad" relationships in the first place.
I put it into DDG but didn't find much.
And no, I am not exaggerating the time needed to fix. The weekend bender, maybe. Average case is started on a weekend and in production two weeks later. The code did not improve over those two weeks.
I’ve dealt with this half a dozen times (maybe more) in my career. Enough that I want to struggle the next “rockstar” that shows up in Slack. They generate problems and support issues faster than a room full of monkeys on typewriters.
I get even more cranky when they decide to use the opportunity to learn Go or Haskell, a programming language no one else on the team knows. And of course they fuck it up because they haven’t used a statically-compiled, strongly-typed (or functional) language before.
I’m generally a careful programmer, but I can definitely write fast and cut corners if necessary. The difference is I know what corners to cut and how to document them.
It might take me 3 weeks instead of 2, but the reduced support cost is far worth the extra week of development.
The key problem is always the same. The business wants to move as fast as possible, and developers are extremely concentrated in either the overly defensive "let us discuss ad nauseam" camp or the willing/passive "let's develop something and deliver it" camp. I don't agree with either, but frankly, I've had far more trouble convincing the former of their problem (analysis paralysis and wasting time) than the latter of theirs, and the former group is extremely convinced they have the high ground over the latter.
>I’ve dealt with this half a dozen times (maybe more) in my career.
And I've dealt with 'seniors' willingly sitting in meetings not achieving anything and pushing silly refactors more than half a dozen times only to run into performance issues, bugs and terrible deployments. User complaints don't lie, the grass ain't greener on the other side.
No, as far as the star dev goes, he's done with the project and has moved onto the next one. He's far to valuable to the company to be fixing bugs. The buggy code is already in production because he promised it was done. The fact that others have to fix the problems implies that the star dev is done.
I wrote about a specific problem, not a generalization that all fast phased development is bad. Also, nothing in my story implies that I'm justifying overanalysing.
I've seen cases that if code is not ready by Monday then the company goes bust. The code is shipped riddled with bugs. Totally justified in that case.
But more often than not "people are done after their code hits production".
"Let's have discussions until 2 weeks before the deadline for something that needs 2 weeks of development time!"
"..wait, why are we having performance issues? Why did we not think of this one case that's occurring every 5 seconds?"
People are very eager to take article's example and use it to slam individuals they have a vendetta against, while looking away from the other extreme which has the exact same problem. Going too fast is a problem. So is going too slow and LARPing as a clairvoyant, getting close to a deadline and then business forces you to get disciplined and deliver something not battle-tested.
The more right on the curve you are, the higher the chances that you met people that decided too fast than too slow.
But yeah, your probably right.
"Intellectual" is a fine word for an intelligent person who does the kind of thinking college professors do. Indeed, Wikipedia defines one as "a person who engages in critical thinking, research, and reflection."
We similarly use the term "anti-intellectual" to refer to those in our society who choose not to engage in this kind of critical thinking.
I dislike turning it into an insult, so think we should use a different word to refer to such people.
I’m certain most of us know what the parent is talking about on an intuitive, vs verbal level (i.e. the idea was communicated, even if the words used to communicate it were not 100% on point)
Did the joke go over my head?
And they all seem to want to reply to HN comments.
Yes.
> dismissing Dropbox as easily replicated with rsync
This is a strawman of the actual post. Please don't use it as an example like this.
A good example of an "intelligent crowd" are the Windows wizards who will helpfully advise how something is done so the user can get on with their day.
That doesn't mean I won't take easy potshots when the occasion arises, though. If they (Linux neckbeards) don't like it, they can work on improving their act and reputation.
This is 100% what it is. A "We went through it so you do too." sort of thing, combined with the fact that management can't interview everyone and SWE's aren't good at reading people to determine if they're lying.
I have a friend who's a manager at a FAANG and while there's a lot of good, smart people there, there are many, many people who are terrible at what they do and are very difficult to deal with.
My company does not do whiteboarding and makes an effort to get to know the person. We rarely get bad hires. Maybe 1 out of 100? Give me 10 to 20 minutes just to talk to someone and I'll tell you if they'll be a good employee. One of the first things I do is assess if they're lying on their resume. If they're not, then I'm comfortable believing in their skill set.
Every other practice we have in our interview process is justified based upon historical precedent as well as a team decision for where the team wants to go next.
This is pretty much the opposite in an interview situation, even when it's a subject people know well, there's way too much "unknown" about your particular setup that the interview doesn't/can't know.
2. I usually ask candidates questions and scenarios based upon their submitted code samples and will presume an OS or cloud provider they’ve listed as something they’re familiar with on their resume. If you have lied for any reason it will be very obvious very fast as I keep adapting the question to a narrower and narrower scope to being a purely toy problem that is now useless for measuring anything beyond whether they’ve seen the problem before. Usually we can spot some errors in code samples submitted and make it clear to candidates that we do not expect perfect or necessarily even working code! The test isn’t about necessarily being bug free but about the attitude one has about their work output and how they’ve thought about various failure modes. I’ve had candidates surprisingly often say “that can’t happen, I made sure of it in my test cases” and then I show with a quick test run I think of that their submitted code does indeed have some flaws that would have had issues in production causing, say, OOM issues, segfaults, etc. A successful candidate’s attitude is accepting / welcoming of criticism as a team effort, has strong but loosely held opinions when encountering data contradicting their position appropriately, and is able to accept a challenge from junior engineer with respect and sincerity. Red flags = arguing with the interviewer for asking an admittedly irrelevant question, getting very defensive about a technical decision, and saying adamantly “this should have been caught in a code review so it’s not relevant” and being dismissive about the question. Yes, I’ve seen all these responses before. Nowhere in this process does “is expert at X” enter the picture, and we are usually evaluating the quality of the questions asked by the candidate as well. Many of us on the team have respectfully asked the relevance and presumptions of a question and that’s a big plus IMO - they have courage, respect, and critical thinking skills under pressure. I make sure to tell them that during the interview because people are so used to taking orders and shutting up that it’s bad for the organization.
Another set of red flags I’ve seen is “I’ve never had to do X, it’s taken care of by Y” and being unable to reason or even conjecture about how Y could be designed and reassured it’s not about correctness when it’s on their resume. Like I had someone that was pretty clearly a competent, skilled programmer but was applying to be an SRE and didn’t know how to work on a system below the container level with any tools proprietary or OSS - they were recommended to a different team but not for the applied position.
I assure you that everyone I’ve interviewed for years now has said they’ve had a great experience in all the follow-ups with recruiters, felt that the interview was a technical conversation rather than a hazing / torture test, and that nobody felt the questions were not relevant to the job once hired. I’m not an asshole as an interviewer trying to push people to some limit or something that seems to be the result of most technical interviews in our industry.
When that happens, do you tell them they have 5 minutes and if they don't figure it out they're fired? Try that sometime and see how they perform.
Either that, or the amount of time you subject potential hires is just a "signal" of your power and a filter for the needy, desperate and intellectually invested. I guess some companies are looking for that.
I do a lot of interviewing for our company, and I used to think this too. But I've been totally wrong a few times. Sometimes someone I've been 100% sure about has been let go after a few months, while some I've been very iffy about have turned out to be excellent coworkers.
The best 2-3 hires I ever made as a restaurant manager I knew they would be great within minutes of starting the interview. Other than those outliers I would say a great hiring manager was about twice as likely to have a productive hire as a bad hiring manager. Great hiring managers were batting around 0.600, almost regardless of if it were a relatively “technical” position or not. At some level it was just a crapshoot.
Even when fresh out of school one should see what someone is capable of.
I would filter out the ones that have the criterias, and then see how they fit with current culture rather than roasting them with tests.
Like you do in any other business setting..
It turned into: "an initial interview; meet potential future co-workers and manager; have them see me do some actual work they do; meet the company owner(as he happened to be there);meet the accounting person and haggle over proposed pay; meet the HR person and haggle over minor changes in the employment contract;finally sign the contract" all in about ~7h. Coming in I was expecting to spend an hour there, but I was pleased with the outcome. I spent 6 years working there leaving only because of a move to another city.
And then slow builds and decisions would have to be rewritten anyway. Usually with many assumptions baked in.
So I highly prefer fast iteration where you get at least something out for the customers to use as soon as possible.
Blaming people is a sure sign of broken processes.
Paying for someone’s time is… tricky. If a company doesn’t pay, and the candidate is willing to do the interviews, it is a signal that both are expecting positive financial outcomes overall (ie both sides foresee fit).
The moment you start paying for people to interview, that signal gets weaker. Likely compensated by stricter resume filters. Its a disadvantage for people with uncommon profiles or less experience. (And an advantage for those that have more experience and some big names on their CV)
If vJ is the perceived value of getting the job, pJ the perceived probability of getting the job, vI the value of the literal payment for going through the interview process, and vT the perceived value of one's time (and any other cost if it exists, such as travel) required to go through the process, we can represent the expected value of going through the interview process as:
EV = pJ * vJ + vI - vT
I also assume that the candidate would choose where and whether to apply based on EV for their various options.
One one end, if your pJ is close to 1 (you're highly qualified for the job and will likely get it), the result is dominated by pJ * vJ which is ~= vJ; vI and vT matter little. This means that if you will probably get the job, it doesn't really matter much whether the interview is paid (and it also doesn't matter as much how much time it takes). For top candidates, the difference in pJ * vJ for different companies should be the dominant factor, i.e. they will apply for the best jobs.
On the other end, if your pJ is close to 0 (you're applying on a long shot), then vI and vT become much more significant factors. If your chance to get the job is really low, then the interview being paid makes it significantly more attractive, and it also matters more how much time it takes. The companies that pay for interviews, and companies that are easy to apply and interview for are likely the ones with the highest EV for the poorer candidates.
Basically the worse of a candidate you are for the job, the more important it is for the interview to be paid, because with a low enough probability of getting the job, this payment becomes a big factor in the expected value of doing the interview.
Of course it's not as simple as that, because people are not machines chasing pure financial interest and have feelings about how you treat them. Also, a highly qualified candidate is more likely to have a job that is closer to the one they're applying for, while an unqualified candidate might have a much worse job, making vJ higher for the less qualified candidate. But it is likely that the relative difference in pJ is much greater between a qualified and unqualified candidate than the relative difference in vJ. The candidate's own perception of the probability of being hired (pJ) might also be unrealistic in either direction, and I'm assuming it is a good predictor of the true probability of being hired. But I think in general the rule should hold, paid interviews would decrease the quality of the candidate pool.
I mean, would you pay money to a handyman if he'd show up near your roof and showed you how he uses his tools and how he WILL EVENTUALLY work, but, not really fixing it?
Having said that, I think paying people to interview for 6-figure salary jobs (AUD) is a non-starter to me. If a company is stretching your interviews out to the extent it's a waste of time, that's a signal that it's not a place you want to work.
If that was my idea not the handyman’s? Yes. I would expect them to charge for their time.
This is for Staff+ level positions.
EDIT: Just to state, companies do give enough time to prepare for the interview. I am only talking about the interviewing duration, not the prep time.
The culture is so different here in Australia and New Zealand.
I've done a lot of interviewing of candidates over the last 12~ years or so - here are some generalisations based on my experience and that of my peers for tech roles:
- 95% of the time it's 2-3 interviews / meetings. I have seen a single interview be enough when the person is already known to the team and the interview went well confirming and clarifying existing knowledge of the person.
- The first is usually a call with people and culture / the internal recruiter - high level intro and general culture fit. These tend to be between 15-30 minutes. You should usually know if you're being offered another interview by the end of the call - or within a few days at most and have the next (main) interview setup for the next week or two at most.
- Then if the role is for a developer / programmer or design / UX the candidate is usually sent a coding (or design) test which can be done remotely and will usually take 30-120 minutes but this obviously varies depending on the test, role, and persons abilities.
- Second is often a technical and team fit interview with two people in or working closely with the team / department you're hiring for. These are usually around 45-90 minutes. You should usually hear back about this within a day or two, and if another interview is required you would hopefully have that booked in for some time in the next week or two.
- A third is often done if the team/tech lead wasn't in a previous session - or if the interviewers can't agree or get a good feel for the candidate.
I don't think many handymen offered a chance for a two year, full-time building gig would say "sure, wire me $100 for the consultation and I'll come over", and if they did they'd be smart enough to realise they were politely turning down the job.
Frankly, "too preoccupied with the idea of getting paid for a single day of producing no output to take into account the potential to earn $xx,xxx more over the course of the year", sounds like an excellent way for employers to filter out candidates who have low interest in or confidence of getting the job or poor decision-making ability...
However, if a for-profit company wanted me to spend essentially a full day of my time off work so that I can go and interview to make them more money than they'd be paying me - yeah that doesn't sound right at all unless you're straight out of school.
I know what you're saying with your handyman argument but that is a false equivalency.
Yes, entitlement is possibly a good description there. It’s far too one-sided. If they had to pay they’d suddenly find ways to get “signal” much more efficiently and effectively.
Entitlement cuts both ways. Software development pays very well and largely has excellent working conditions, it’s basically a golden ticket. I know people who are essentially stuck in minimum or low wage jobs for life.
Don't for a moment miss that if "software development pays very well" then it pays the company even more than it pays you. Being employed by them is a win for the company they're not giving you free handouts, they should be willing to spend money to make money.
Entitlement doesn't "cut both ways", it's a mostly meaningless conversation stopping dismissive insult, but as far as power imbalances go one side is a company with millions of dollars and lawyers on hand or on retainer. If it's a big FAANG they likely union bust or block which is against employees interests, they possibly have stack ranking or other methods to divide and conquer employees by pitting them against each other, possibly has 'unlimited' vacation time which excuses them from rollover or payout for untaken vacation and maybe they don't approve vacation because everyone is always busy. Often they will be lobbying politicians to increase their power and decrease employee's power. Healthcare is tied to employment, which is in the company's interest and against the employee's interest because it makes leaving harder for the employee but does not affect firing or redundancy for the employer.
On the other hand you have a single lone human.
The employer also wants loyalty, often ownership of anything the employee creates on their own time while employed, often an NDR, and control over what the employee wears and says and the times they work.
And they want long days of interviews, often with terrible interview practises which don't give them good data (much discussed on HN year after year) - and they expect this effort and time which is ultimately for their benefit, for free.
Most tech jobs are not in FAANG.
> If you say "I need to spend a day doing work things on a company's terms for a potential deal which would benefit the company a lot, and I want to be paid for my time and effort" that's ... not entitlement.
I frankly don’t find it plausible that the work being done in a day or two during the interview process is valuable labor. Who is going to deploy that code and make it production ready? How would you even organise and plan that work? Is an engineer opening a PR internally with the interview code or something?
> Healthcare is tied to employment, which is in the company's interest and against the employee's interest because it makes leaving harder for the employee but does not affect firing or redundancy for the employer.
Fair. I live in a country with a public healthcare system so this doesn’t factor in for me.
It's the hiring of an employee who will make wealth for the company over their employment period which is the financial benefit, not the work done on the day. But the work done on the day is still value - it's signals of the quality of the candidate which is something the company wants (and therefore values, and therefore should buy). Second, if someone arranges for your time and tells you to do some task they choose, they don't then get to say 'I can't get value from your work so I'm not going to pay you'", that's not how things work.
> "they also deliver compensation packages that essentially guarantee a wealthy, comfortable lifestyle."
And what about the people who spent a day on the interview and did not get hired? The company gets time, effort, signal information which benefits the company ("no hire" has value too), and the candidate gets their time and effort wasted. Imagine if you had two contractors do a task you set, then turned to one and said "I'm not using your work so I'm not paying for it", how well would that go down? "Fuck you, pay me"[1] I expect. It's not like going to a shop and not buying some things, it's like asking the shops to deliver things to you and then refusing to pay for delivery on things you don't buy because the shop had a chance of getting your business so they should be grateful. (Worse if the company is constantly interviewing just to keep the pipelines warm, without any interest in actively hiring anyone right now).