I admit, that this is traditionally a security blog, however I am going to digress for a post. I truly hate the concept of a coding test during an interview. I hate them as both an interviewer and an interviewee. Joel Spolsky & Jeff Atwood would probably disagree with me, however I truly believe programing tests during an interview are pointless. I have come to this conclusion after 8 years in the software industry both, taking tests as well as giving them. The reasons that I believe these tests are pointless are: The tests don’t really prove anything; coding tests don’t simulate a real work environment; they’re a waste of time.
1. Prove nothing
I’ve often heard the arguments that you wouldn’t hire a contractor, any other contractor without previously seeing their work. Or you need to see and evaluate one’s style to see if the candidate is a good fit, you want to ensure that the candidate can actually code.
The test during an actual interview does nothing to really prove or establish any of these three points. Often times during the interview the candidate gets X amount of time to accomplish a specific task. Software today has become so diverse with so many different industries writing many types of software across many different technologies that the candidate may not be versed in all the specific technologies require to accomplish their task in the given time frame.
Does that mean that they’re a bad programmer? Of course not. Does it mean they’re incapable of working for your company? of course not.
When you truly consider the role of a developer or engineer within your organization, writing the code is actually the easy part and the end solution to the problem. This raises the question are you hiring an engineer to solve problems or are you hiring an engineer to crank out code. The obvious answer should be the former. Therefore the way an engineer actually writes code is less relevant then how they solve the problem especially in today’s industry where every company has their own standards, practices and format.
If the candidate has been employed for longer then an year, at a previous software/technology company and provided that they’re not a fresh graduate it stands to reason that they can actually write code. So too isn’t this why you check their references and follow up with their past.
2. Doesn’t model your environment
In what organization does a member of your development team actually only have X amount of time to solve a problem and write some actual code. So why are you asking a candidate to perform under pressure, already being stressed out from the interview, to write code and be evaluated in a short amount of time? Do you really think that you’re getting the best possible result from that candidate, now if one doesn’t handle interview situations well, you might pass them over because they didn’t perform on their test well enough yet, you might hire a weaker engineer, because they are capable of performing in that 1 situation better.
In the environment the engineer has a chance to think through the problem clearly design an excellent solution and lastly write the code which will illustrate the situation as needed. So why not design a test and circumstances that mirror an actual environment and allow the candidate to actual produce in the best situation.
3. Waste of Time
Often the candidate is going to write bad code. Often time the developer shows up isn’t mentally prepared to write the code. Now you as an interviewing team have to evaluate bad code that may or may not actually solve the problem that your test was designed to evaluate.
Often time these tests are designed to be so short, such that the candidate can complete it within the given time frame. The test is pointless, without meaning, often times it doesn’t expose the candidate to real problems & technologies that your organization actually works with. Nor does it actually allow the organization a chance to evaluate a candidate’s architectural abilities.
A Solution
Rather then springing a coding test on the candidate during the interview, why not E-mail the candidate the test a few days before as part of the screening process or the night before. This approach will give the candidate the time required to think about the problem plan out a solution, and actually implement the feature. They may or may not actually finish it early and therefore have time to refine and refactor it into an even better solution on your behalf.
Once the candidate has sent the test back the interviewing team can perform a code review analyze the test and prepare a standard set of questions to ask the candidate for the in person interview. Or the team can perform the code review during the interview itself. It will quickly become obvious through a discussion whether the candidate implemented the test themselves or if they hired someone else to implement the test for them. This approach would actually allow the time for a more detailed test which would test architecture abilities as well as, coding abilities, giving a much more complete and reliable solution.
Given the onset of DevExpress for visual studio, or other free IDE’s these days there’s really no excuse why a potential candidate could not grab an IDE, if they don’t already have one installed, which is telling in and of itself. There’s really no reason why a candidate cannot pick up an IDE and actually write some code at home in the evening before the test.
What am I advocating for? Scrapping the test during the interview and allowing for a more detailed thorough take home test that models your work environment and exposes the candidate to the technologies that your organization actually uses.
A coding test where you test for knowledge of specific technologies and actually look at the result is stupid, I agree with this. (I for one need to dig through the whole Spring documentation again and again each time I set up a new project.) But there’s a good reason to do a coding test, although not one for testing knowledge of specific technologies.
You most often don’t hire a programmer for doing a contract job. You hire him to do work on a daily basis as a member of a team. An easy coding test, so the candidate doesn’t get stressed out, during which you can experience how it is to work with him, provides IMO valuable information to decide if he’ll be a good fit for a team or not.
One of the more difficult examples for such a test which I’ve given was “generate all permutations of n”, with the entire Internet available for searching for solutions. Then I’d go on discussing how and if it would make sense to build a recursive or an iterative solution, what happens when n is large, and stuff. Many people didn’t even get to a working solution, but that’s OK, if we like their way of thinking and their approach to the problem we still hire them. Maybe I’m challenged in some way, but I don’t see a way to get the same feedback during a non-coding interview.
This may also have something to do with expectations. At the time I was practixing that, we were expecting to take in fresh programmers without a lot of knowledge, and educate and train them to become highly knowledgeable and skilled programmers, able to work independently on just about anything, if needed, but also able to be just a cog in a larger, well functioning mechanism. As such, our focus on specific technologies was low. If, however, you urgently need an addition to a team, with specific technological knowledge, a coding test in that specific technology makes sense, IMO. But that’s not a good strategy for a long time hire – it’s risky at best for the team health to add people based on momentarily technical needs, IMO.
Anonymous,
You mentioned a lot here. I am not advocating for the total removal of the coding test, I am advocating for changing the parameters. Let me ask you this, take your permutation n question, if you give a candidate 2 -3 hrs in the interview to do it, you’re 100% right, they’re probably not going to get a working solution. However if you gave it to them before the interview and you were able to analyze the results better.
The potential candidate would be able to analyze the problem, develop you a high quality solution or a low quality solution, you’d gain a much more in depth understanding of what they were capable of, how they thought what they did. For example, if it were me, taking that test, and I had a day before my interview or some time line. I’d first seek to get a working solution and then I’d work on refining it. This would demonstrate 1. How I thought, I could solve a tough problem. 2. I would demonstrate how it could be improved. The interviewer would get a much clearer picture. of what I am truly capable of.