November 5, 2009

Recruiting Puzzles

One of the most interesting parts of Peter Seibel's Coders at Work (see my review on Slashdot) was Peter Norvig's discussion of the Google programmer hiring process.

The impact of a bad hire on a programming team can be extremely negative - vastly increased code maintenance costs and reduced morale and productivity. "Hire slowly" is now becoming an ingrained mantra in software companies. Despite this, effective processes for evaluating candidate programmers still seem to be little known.

Interview puzzle questions (made (in)famous by Microsoft) as a tool in the hiring process came up in Norvig's interview. Not surprisingly he is against the idea. For a while puzzle questions were a fad in programmer interviews, based on the (unfounded and unverified) assumption that they were a predictor of coding prowess.

To examine why puzzle questions are a dumb idea, you need to look at the objectives you are trying to fulfill when hiring programmers:

  1. Hire someone who is good at programming.

  2. Hire someone who is good at programming as part of your team.


There is no need to resort to unrelated tasks with a low degree of correlation to see whether someone will help fulfill these objectives. A candidate can be assessed directly and efficiently by inviting them for three hours of pair programming.

Why are you still wasting time asking how many golf balls can fit in an airplane?

1 comment:

Vladimir Sedach said...

I came across an article in The Business Insider about the author's experience interviewing for an Associate Product Marketing Manager position at Google. I think these sorts of analytical puzzle questions are actually important for a marketer - the job requires a lot more unconventional thinking, creativity, and decision-making based on incomplete/unavailable metrics than programming does. I agree with the comments in the discussion criticizing the author's performance.

Unrelated to puzzle questions, one comment in particular stood out because it described a particularly clever and applicable method for judging the hiring process:

We reviewed our hiring process because we were not getting the hard work and ability to learn that we had with our current top go-to people.

As a test, we sent our top performers' OLD resumes when they were hired 10 years ago to HR with different names.

What did we find?

8 out of our 9 top performers did not make it through HR to a hiring managers desk.