Hiring Tip: Graph Interview Performance Vs Years of Experience To Spot Outliers

When hiring your initial engineering team, the more tools you have to discern who the truly amazing engineers are the better.  One approach we took at Mixer Labs (a company I founded that Twitter bought last year) was to graph # of years of experience vs how the person did in their interviews (this was an arbitrary scale where we rated people 1-10, focused on performance on the same set of questions, rather then raw intelligence alone.)  They key is to have the same group of people on your team do the ratings, and to have the list of prior interviewees listed on the same ranking scale, so that it is easy to compare people.

To our surprise, we found that in general, the way people did on their interview roughly corresponded to the number of years of experience they had.  Indeed, when you graph things out, the graph looked roughly like this:

This graph helped us quickly hone in on anyone who was an outlier on the graph (i.e. their performance/knowledge in the interview outweighed their years of experience).  For example, from this perspective the person with 4 years experience who scored a "7" was a much better hire then the people with 7 or 8 years experience who scored a "7" or even an "8".  The interview performance outliers also tended to be amongst the most productive people on the team once they joined the company.

This also suggests that if you do not correct interview feedback for years of experience, you may hire more experienced people who are actually more "average" relative to their work experience level then you might expect (since you would normally compare them to *all* candidates in the overall hiring pool, which consists of people with less experience).

Obviously, anyone who scored close to a "10" was worth hiring irrespective of number of years of experience.  There are obviously a number of people who are both very experienced, and very very good.

Any other simple tools or tips for hiring?  Leave them in the comments section.

Other posts on hiring:

• How To Get Your First 3 Employees
• Your First 5 Engineering Hires: How To Select For Awesome People
• 5 Myths To Building an Awesome Mobile Team

You can follow me on Twitter here.