These comments are completely missing the point. HackerRank's value is as a screening process. The point of such problems in the early phases of an interview is not to find out how good a programmer you are, but how bad a programmer you are.

There are an ASTOUNDING number of candidates for programming jobs who can barely code. FizzBuzz eliminates a lot of them, but sometimes you need something just a bit more difficult. That's where HackerRank comes in.

When talking to people that are "professional programmers actively seeking work" I find that the majority of them can't do simple tasks. "Simple" isn't meant in an arrogant way, either; examples include reversing an array (with no space constraint), iterating a dictionary/map (regardless of understanding map implementation, a lot of people don't understand that iteration order isn't predictable), and basic use of nested lists/non-scalar data structures. Those aren't esoterica that "nobody uses" on a daily basis, they're bread and butter of programming work.

These people have hobby projects/OSS work, it's just . . . not sufficient, in many cases, to indicate basic competence on the job. I don't know why that doesn't correlate. I don't think most of these people are lying and publishing work that isn't theirs. It might be that they have a very long time to write the stuff in their GitHubs or whatever and can't do basic things in a hurry otherwise. It might be something else. Regardless, they turn out to be unqualified when tested on the basics.

If you find the problems insultingly easy, choke it down and consider it equivalent to the question of "are you comfortable being asked to show up to work on time?"--patronizing, but an easy box to check, and not meant for you.

There are companies that rely entirely on very hard/complex algorithms problems for interviewing, with the full knowledge of what they're testing for and why they're using those tests. I'm not talking about those companies; there are good and bad reasons for doing that. I'm talking about simple coding challenges presented early on in the candidate's process.

There are also companies that get confused as to what they're using the HackerRank-equivalent tests for, and give really hard problems (or, worse, puzzle problems that block all progress and rely on some sort of "gotcha" trivia knowledge to even approach), but evaluate candidates' performance as if those problems were FizzBuzz++ basic competence screenings. That's a mistake, but also not what I'm talking about here.

For example, I haven't had to write my own algorithm to parse a b-tree or implement binary search since I left school. I could explain how these algos work, but to actually implement it perfectly in python or java under timed conditions with no outside docs, I would most certainly fail and the company would miss out on a pretty good generalist.

For example, the correct answer does not appear in the list of available options, or where the problem statement is not precise enough to solve in an unambiguous manner.

There is no human I can talk to and ask clarifying questions to resolve the ambiguity because, you know, it is a HackerRank test!

After going through these experiences, I have decided that HackerRank test is just not worth the time. As a rule, I decline any request to take a HackerRank test citing my reasons as politely as possible. Surprisingly, more often than not, the recruiter is happy to setup a telephone round where I can talk to an actual human.

It's good to see someone take the initiative to express this, but IMHO, it would be even better to just take the initiative to contribute to one of the proposed open source projects and put it on your resume, and participate in the Hackerrank part of the interview.

I like the proposals, and I'd probably land on the side of impressed if a candidate I was interested in sent me a letter like this. But, I still need a quick way to screen people, and I also need a set of standard questions that all candidates get so I can compare & rank them. I don't use Hackerrank, but I do ask easy weeder questions in the same style.

Then, after the candidate passes the easy part, I look at their portfolio of work projects and side projects, and consider everything they have to offer.

If you want to be evaluated in a more holistic manner (including side projects) you'll probably have better luck with very small companies. When the people in charge will actually be the ones working with you every day, and they are only filling a small number of positions, they will be more likely to find it worthwhile to get to know you as a whole person during the hiring process.

Unfortunately, once a company is beyond a certain size this just isn't possible because it doesn't scale. They have to fill hundreds or thousands of positions and also maintain a universal set of hiring standards. Things degrade into a game of numbers. This makes cookie-cutter tests almost inevitable.

While frustrating in their own right, for me what makes things worse is that you have to do one of these screens for every single company you apply to. And most companies have the same evaluation goals for the phone screen stage (e.g., do you actually understand data structures & algorithms).

What I'd really like to see is the emergence of something like a "common app" for software hiring. Outsource technical screening to a reputable third-party, and once you've proven yourself by obtaining a good enough "score" you can move immediately to on-sites (what constitutes a "good" score would vary by company). This doesn't solve the problem of the efficacy of technical screens, but it at least cuts down on all the redundant effort.

Two services I've tried so far that are doing something like this are Interviewing.io and A-List. Unfortunately it seems these services merely make it easier to get a phone screen, rather than replacing them. But it seems like a step in the right direction.

The value proposition of investing time and effort into interviewing with this company appears to be less than, say, interviewing companies whose priorities are well-aligned with OP from the get-go. That being said, these kinds of "coding puzzle" screeners aren't going anywhere, and may lock him out of great opportunities at great companies.

"I'd rather have it and not need it, than need it and not have it."

I have taken a number of HackerRank screens myself, and I have never seen one that either (a) tests beyond the coding level that would be expected of a fresh CS graduate or (b) is not vastly exceeded in scope and complexity by the coding challenges given during the onsite, which do allow considerably more flexibility and relax some of the artificial constraints.

Why attempt to circumvent a test that is meant as a slightly less trivial FizzBuzz?

This response seems like an attempt to engineer that path out of nothing. Good luck, but you can also avoid these interviews if you can make those connections/build your reputation to the right extent.

I tried to build my own coding test that is based on refactoring based on changing requirements: https://alexrohde.com/public/zennish/index.html#/challenges

That being said, depending on the type of company and the number of candidates they are dealing with, I wouldn't be surprised if they simply ignore your letter rather than trying to deal with it. If you have 60+ more or less qualified applicants for a single job opening your first job is to get that down to less than 10 as quickly as possible and throwing out anything that doesn't fit into your template is a quick and popular first step.

What are the alternatives?

They need to minimize time, give everyone a fair chance, and improve the signal. I certainly wouldn't say the current method is great at meeting those criteria.

IMO If employer is adamant to rely only on cookie cutter first round interviews, then it’s probably not worth joining such places. It’ll be filled with people who are great at cracking such interviews and believing that these cookie cutter interviews are the only way to go.

There will be people on both sides of this debate, pro and anti.

If you don’t believe in solving algorithms/ hackarank type initial interviews, it’s good to challenge such requests.

If you are fine with it, that’s great. You and the employer have the same thinking. And it’ll/ May work great!

Without even addressing the sour grapes attitude about coding interviews, the sheer arrogance with which you try to dictate someone else's process is a red flag in the first place. That, plus the fact that you're refusing to do what they want, is really bad for your chances of being hired.

However unless they really like your profile they will immediately next you after that kind of response because companies also don't have time for this nonsense. They just want a standardised tests for all candidates so they can rank them and then hire the best.

I almost got the job, they randomly canceled the on-site at the last minute. Needless to say, I think my effort at least got me to that final step.

I think an email like this would be just fine, for them.

I think it would go over badly at companies where HR runs the hiring process.

With a tailored take-home, the company has a chance to evaluate candidates based on more realistic, holistic challenges than “solve this academic comp sci problem that you won’t actually encounter on the job”. And candidates can get a better sense of the type of work and challenges they’d actually be tackling on the job.

The take-home isn’t tailored to each candidate. The opposite, actually: it needs to be the same for all so that candidates’ solutons can be compared.

Here’s the tl;dr example of a challenge we give web engineering candidates:

“Here’s our API and API docs: <link>. Create an interactive data visualization using D3 and any other tools you’d like. Your submission should include instructions on how to run it.”

By keeping the challenge relatively unconstrained, we allow candidates to show some creativity and prompt them to make the same feature/quality/speed tradeoffs that they’d have to make in the role. We look for autonomous individuals, so this turns out to be a good filter. We encourage them to spend less than 4 hours on it, but our time limit is a pretty generous 1 week since they likely already have a full time job and other interviews.

Yes, side projects can be fake but thats why you interview for multiple rounds. Think even there is no proof a candidate really took the HRank test.

Ideas for how to do hiring differently are cheap and plentiful.

Nowadays I say up-front "I just want to mention this because it's generally been a sticking point and I don't want to waste your time, I'm looking for a salary in the range of $X". That has saved me a whole bunch of time.

Or at least I would consider hiring them if the rest does fit (culture and other such squishy interpersonal things).

HackerRank test can be as standard or as non-standard, and as easy or as difficult as the individual employer wishes it to be.

When it was over I asked the interviewer if there was ever a time when he had to implement such an algorithm on the job during a production outage event and his answer: No.

In my experience working within devops teams maintaining and developing public cloud infrastructure I don't think I've ever had the pleasure either.

In retrospect I don't think it was a bad experience. I think there are roles which exist in software that do require a higher level of rigor than others. I'm often amazed how easily many developers will dismiss performance concerns (premature optimization!) for example. I would expect a hiring process to be upfront and honest about such requirements and candidates should be aware of their own skills and background before applying.

But most software development jobs at ABC Corp are not the SRE Team at Google and they hire as if they were.

I think that's the risk we take with standardizing and automating hiring processes.

update: just markup.

I have never been in a startup that fears arguments. Every startup I have been a part of has lively debates and arguments almost every other day. It's only the large slow moving corporate giants where I see such lack of debate and arguments on what is obviously wrong.

I would go to the extent of saying that a startup that does not encourage arguments in a harmonious fashion would die very soon anyway. Debates and arguments help in weeding out stupid ideas and focus on what that really matters.

This person's email shows that he can have a qualitative argument in a coherent, polite, and non-confrontational fashion - exactly the kind of people you need in startups!

I wish I had more teammates who could write as effectively.

You hire people to solve problems, not churn out code. Code is only a tool to solve a problem.

Some tests will seem impossible to complete in the alotted time. Often that's intentional. It saves time for the candidates and gives a better spread of scores (very few will get 100% in some test batteries). But it can be disheartening. For these initial online tests I try to give enough time that most people won't really run out of time unless they get stuck.

Why are you so quick to extrapolate an entirely new character based on this person not wanting to use HackerRank?

I'd be willing to waste 5 minutes tops, unless the employer has put some skin in the game (fizzbuzz is ok - nothing much more than that tho).

There's a 1,000 other employers out there who might be a better a fit. I am not going to do a 1 hour, 30 minute or even 15 minute task if they can't at least even be bothered to pick up the phone to talk to me or glance at my github profile.

Companies that expect completion of a weekend-filling exercise before granting you an interview are universally run by entitled assholes. You're actually doing yourself a favor by screening them out: they'll be bad to work for, you won't get to work with other 10x engineers and the company will likely achieve nothing impressive anyway.

Agreed, this is why I give my own coding quizzes, so I can have a discussion about it. I'm more interested in the thinking than the answer.

But, I do have sections in my interview that test knowledge and not skill, so I can still see the usefulness of letting Hackerrank be a part of the process.

The one thing I don't claim is that HackerRank or coding quizzes are perfect. This is why coding quizzes are only one of maybe six phases of an interview.

If the implication is false, it would be an exciting data point!

HN being what it is, here is quite a non-zero chance to meet a person with real hiring power who really could have made an offer to the person in question.

At the same to be cognizant of your business, profit margins and by that definition what you can pay and remain profitable is a good thing. Not every shop has to be a world changing venture, and if you can pay low and hire people good enough to run your business is just perfectly Ok.

There are small financial firms which will conduct significantly harder interviews than Facebook/Google. The offers are also commensurately higher (significantly so).

This kind of approach is fine for a junior. I'd even encourage it because there's an attempt to change the parameters of the problem which is something we often have to teach people as they progress.

For a senior position, not so much. I mean, it might be that they're ignorant or stupid, in which case you've probably dodged a bullet when they don't hire you. Or it might be something else entirely.

A more sensible approach, in my view, would be to ask what they were looking for with the HackerRank test in a subsequent face to face. If they say that they don't have a better way to determine developer competency then, sure, suggest that you may be able to help with that when you join.

If it were me interviewing, I'd be particularly interested in suggestions that a) were developed from successful prior hiring practices and/or b) not so bleeding obvious as "look at their Github page".

This reminds me of this recent HN post: https://news.ycombinator.com/item?id=16337434

One should be able to devise a solution to a simple problem like this without any CS course at all.

or code units/code points/grapheme clusters/emojis. Thanks to hard work of many developers string handling becomes easier over time, but I don't think it's trivial if you don't exactly specify what you mean by "string" and "character".

> "Difficult to work with" is perhaps too harsh a statement

That's exactly what the OP meant to say, and verbatim what they would have said in an interview debrief. The interviewee wanted some amount of compromise, and all of a sudden they're "difficult to work with". Now everyone else in the room is framing this potential hire as an asshole. There's no coming back from that. I've seen this happen many times. One term of phrase like that and instantly a qualified candidate is out because someone latched onto a single fault and made wild extrapolations about it.

> just a low-investment screen

You have no idea how much of an investment it is. I've had hacker rank problems that I was expected to spend 3 hours on. That's a pretty big investment just to get my foot in the door. Sometimes (read: often) the juice just isn't worth the squeeze. The employer wants me to give it my best when they're not even willing to come up with their own questions.

> uniform across all candidates

I see this a lot as the panacea of interviewing. Sounds good to have everyone on a level playing field. But if you start out with a crappy process, applying it to everyone equally isn't going to get you good talent. As an interviewee, I'll still be bitter about the bullshit you put me through, even if everyone else had to do it.

Low-investment on the part of the employer. They still expect me to spend 90 minutes on proving I have seen a REPL before, which is time I, frankly, don't have. I'm happy to answer/talk about things that relate to my job, but working on puzzles that just happen to be solved more easily by programming has exactly nothing to do with it.

> the candidate is asking for a completely custom treatment

What's next? Having to actually look at someone's CV? That's insanity.

10/10 !

I don't know about you, but I would not want to hire someone who can take any task and complete with minimal issues. Where I work, one is encouraged to argue about what is not right in the meetings and offer better solutions.

This candidate is doing exactly that: Argue why a certain hiring methodology is not effective and what a better alternative is. Considering that this candidate does not just follow orders but challenges the status quo and the fact that he has volunteering-based projects to show makes this author very likely a good fit for my organization.

Maybe at a consultancy and only if you're far downstream of dealing with clients.

That's not normal, nor should it be.

Yet by forbidding the use of external research and libraries they remove the two main tools I would want some to use "in an environment and context where you don't have the advantage of being an expert ahead of time"

It's extremely hypocritical to criticize someone for the very thing you are doing (assuming the other person doesn't know how to do their job). Couple this with the fact that they are not asking for any advice on their hiring process, it's a pretty rude thing to do.

A little bit of fluff added to the wording doesn't make it suddenly polite.

> I would not want to hire someone who can take any task and complete with minimal issues.

Someone who can't do straightforward tasks without having an issue is incompetent, pedantic, or both.

These tests aren't useless. Tons of candidates have never seen a REPL before and are totally unable to solve even a trivial coding assignment.

I suppose your company is too clever to waste their time on such useless tests? Who at your firm wastes hours interviewing candidates who have no ability to code?

If their hiring process involves jumping through hoops before being granted permission to interview and if the tasks assigned to you bear almost no relationship to the job at hand, the chances of the job actually being good are low.

Employers that do this kind of thing typically have an entitlement complex and not the smartest. People with an entitlement complex who are not very clever are bad to work for.

>Why would you assume someone else knows anything about your accomplishment and credentials and trusts the skills you list on your resume before you've demonstrated anything?

In my case because I have a bunch of open source available that they can just read.

If a company expects me to complete a 1 hour badly thought through exercise that is at best barely related to the job before they grant me permission to talk to them, just exactly how well do you think they'll treat me once I'm actually hired?

It's not a matter of fear or offense. It's a matter of judiciously investing my time on the right opportunities.

I get about 3 to 4 calls from recruiters everyday. If I agree to go through this kind of interviewing process every day (even if the bar is low), I would be spending more than 20 hours every week just doing HackerRank tests. This is not feasible for me. Therefore just like the recruiters need a way to filter candidates out, I need a way to filter recruiters out.

Requesting a HackerRank test is just one of the many filters I use.

It's still easy to spot the liars — e.g. I've interviewed people who worked at the NSA and even they could talk about the skills they used, just not which projects or data — and the process of deciding which things to talk about is a pretty good way to explore their communications style, too.

The more senior someone gets, the more expensive. It's worth the company's time to screen for basic aptitude, especially when someone claims greater expertise.

> As a consequence, I rather start my own company

By all means, you should definitely do that and stop worrying about job interviews!

> if you employed me with a bit of a good will on your part

Good will is something you earn. And you earn it by doing things the company needs without complaining. For starters, they need to be able to compare candidates against each other when hiring. Fighting that, and asking the company to evaluate something you've done that doesn't allow comparing you against other people, isn't something that will benefit the company.

> or by simply reading my CV and clicking on the links there.

You're expecting people to spend time reading your projects before they screen you? If you got the call, it's probably because someone read your CV. That's all you can expect at this stage. If you want them to click the links and read the rest, then you take 30 minutes to do the phone screen, and another 30 to do the coding quiz. Then you get to have a conversation about your projects.

If they never ask you about your projects, then yeah, maybe you shouldn't work there. The coding quiz is only one small part of a many-part process.

Yes, this was true of every job I've ever had. Maybe I've been lucky.

I don't really understand why asking someone to do a small amount of programming for a programming job interview, when they claim to be an experienced programmer, is any sort of "extreme" practice.

You've expressed a lot of complaints about coding quizzes, some that many people here share, and some that I agree with.

I would urge you to spend your energy making a specific alternative proposal that companies can actually use. The OP's proposal isn't something companies can actually use for all candidates, even if they could use it for him.

I'm very interested in how to better screen and interview devs. I want to improve my process. But no process I've ever been exposed to is even close to perfect.

Make sure your proposal considers the company's perspective. The ideal process will benefit both the candidate and the company, but something that's pleasant for the candidate and unpleasant for the company will never get adopted no matter how good it sounds.

Sounds great. I've done this too, but I've never done it before screening candidates. I don't have enough time to screen candidates by programming with them.

> "Mimic real life" means no puzzles (unless I've literally experienced them in real life), no binary tree reversals and no big O notation questions.

Understood. My goal with my coding quizzes and knowledge questions is not to mimic real life. It's to asses the boundaries of the candidate's education and experience, without regard to skill.

> I think that type of thinking leads to embarrassments like this

There always have been and always will be false negatives, even with lengthy face-to-face interviews, even with pair programming, and even with paid internships.

One high profile false negative anecdote, while unfortunate, doesn't imply there's a widespread or unexpected problem. Applying for jobs always comes with a risk of not getting the job for a wide variety of factors that are outside the candidate's control. If it's a job you really want, all you can do is try hard.

I love homebrew, and I'm sure @mxcl is a fabulous coder, but the tweet you shared does make it sound like he expected to get the job without an interview, and might have come across that way, or might have not prepared at all.

Imagine you have strong interest from Google, FB, Uber, Amazon etc. Google wants to waive their interview process as you are an open source contributor (they actually read your CV and clicked on the links). FB wants you to lead some ML team they have doing cool things etc. Other companies beg you to work for them even if you don't consider them interesting, willing to overpay you and pamper you.

Now comes your unknown company/startup. In order to even talk, you require passing some HackerRank coding test. I look at your Glassdoor reviews, you either have none, or few, or your compensation seems to be low etc. You might be working in interesting area, maybe I should give you a shot? Or I just want to see what current crop of interviews in your industry looks like, maybe I agree on going through the process? Maybe I even visit some interesting city you are located in and scratch it off my bucket list?

In the end I won't work for you. I won't consider anything you offer. I've gotten from you what I wanted - glimpse of the area you are working on for more ideas, keeping my brain up to date to interview requirements, visiting city I've never been before. You wasted time and money on me. You didn't get anything. I puzzled your head because you thought you'd have a shot at getting me. And that is the best outcome you'd get from this; most likely I wouldn't even talk to you after your initial requirements and go with other choices available to me.

* Argument from authority (because Norvig)

* "We might want to put a front ender in a back end job and vice versa so we need an interview process that accounts for that"

* Tests of "abstract skills" - as in, skills you won't actually use - are more important than tests of non-abstract skills which you will.

* "It's their money"

This is moving the goalposts. That is not the same thing as a wanton disregard for realism.

Have you been asked interview questions and set tasks which were not related to what you actually do day in day out? I have. LOTS.

>I don't necessarily disagree with this, but could you elaborate more on what bad things are actually happening that affect people? Are good coders, by and large, not able to get jobs? Are good coders having statistically significant problems getting paid or finding enjoyable work? I don't think so. Please elaborate on what actual damage is being done, I'm not seeing any.

Um, more false positives and more false negatives. It honestly feels weird having to justify why realism in testing is important. It feels so damned obvious to me. Would you design a purposefully unrealistic scientific experiment? Create a deliberately unrealistic automated test? What's special about interviewing that realism is of secondary or tertiary concern? What is more important than realism?

>I'm not automatically on the side of @mxcl because he was turned down

It feels kind of like you're automatically on Google's side.

The part that made me go what the fuck was when you said "he makes it sound like he expected to get the job without an interview" when he neither said nor implied anything of the sort.

And, earlier you took it as a kind of article of faith that everybody at google was obviously super smart, because Google.

There's definitely some bias there.

>I'm interested in whether they paid attention in school.

That's cool. I'm interested in whether they can do their job and I think it's kind of weird how people are seemingly so keen on setting tasks that test anything but that.

>None of that precludes asking some realistic questions

Strictly speaking asking what their favorite kind of chocolate is doesn't preclude that either, but there's a limited time available to interview and a limited amount that can be learned from asking them that - or indeed - anything else of tangential relevance.

You expect people to do research, but you clearly haven't listened to anything I've said about my process, which I've commented on multiple times in this thread. Your summary is a straw man unrelated to anything I do when I hire people.

And you still haven't yet proposed anything better!

Even the top notch devs I know can't do anything meaningful without an internet connection. Unless of course you know how to invent several layers of libraries, instantly in hours without bugs. Know the documentation of the libraries memorized to the last full stop.

In my exposure at least I know 0 devs who can do this. And I'd bet even the interviewer wouldn't be able to write any meaningful without a network connection. This isn't 1960s. By very definition almost anything today requires an internet connection.

The only thing I'm really defending here is using a 2nd stage screening process that only takes 30 minutes, before I go to the 3rd stage that takes a half day or full day. @BitL seems to be fighting the screening process, which is a necessary practicality and isn't going away.

The first stage is reading resume and glancing briefly at any side projects, which is ~5 minutes. The first stage is invisible, and filters out more people than any other stage. If anything unfair is happening, it's happening here, and there's little a candidate can do about it.

My thoughts on your outline:

A - getting rid of the whiteboard is of course optional. In my case, having graded exercises doesn't mean I stop using the whiteboard. I only provide a whiteboard for people who want one, and I don't ask people to whiteboard things better done in an IDE.

B - Yes! I do like having exercises that range from super easy to not solvable. It helps to know where people get stuck.

C - Yes! That's very helpful for language specific jobs. Personally, I also like exercises that are language agnostic, to let the candidate pick the language they're most comfortable with. I've loved doing coding quizzes in Python during interviews, and would prefer that over, say, C++.

Why is stating what meta-characteristics you're looking for some sort of requirement for you? My job ad does say "passionate and talented", and I (always) reserve the right to interpret that any way I choose.

> It's entirely possible that they rejected him for a perfectly reasonable reason, though there's nothing to really imply that that is so.

Yes, exactly. There's also nothing I know of to imply it's not so either. I can't pass any judgement, I can't call it embarrassing, and I don't know if the question was irrelevant or dumb or fair & smart. If you do know it for a fact, then spill the beans. So far, nothing you've chosen to share addresses this factually.

> You don't know more than what he tweeted

That's what I've said from the beginning, and is precisely why I can't side with him.

Your argument is a false dilemma, you've made assumptions and jumped to the (incorrect) conclusion that because I don't side with @mxcl, then I side with Google. You've assumed that if I'm not with @mxcl, then I'm against him. Your incorrect assumptions only reflect on you, not me.

https://en.wikipedia.org/wiki/You%27re_either_with_us,_or_ag...

> To me, it implied that he was pissed at being asked irrelevant questions.

In my mind, that's not meaningfully different from what I said, other that you're claiming that "irrelevant" is absolute and objective. To make that claim, you need to know Google's side of the story. Since you know more than the tweet, tell me... what's the whole story?

> in a situation where you didn't really have enough information to form a judgment you instinctively sided with the large, powerful, faceless corporation and showed distrust of the little guy who made a great piece of software for free that people love.

You just admitted and rationalized your own bias. What justification do you have for continuing to talk about mine, which may or may not exist, given yours?