Provide the candidate with a genuine programming test in the interview - a laptop hooked up to a projector with a broken project, with a mixture of basic to complex bugs, and ask them to add some functionality, again ranging from basic to moderately complex. This is something nobody can really study for and will actually provide a showcase of their skills. Have technical people there talking to the candidate to get a feel for their personality and how they approach the problems faced.

For the job interview for my current role, I had spent 2 weeks studying programming principles, design patterns, database stuff etc. I was able to answer all the theoretical questions well and think I came across well

I was then handed a laptop which was connected up to the projector and shown a solution which contained a broken website with a defined number of errors which I had to debug and add some simple functionality. While I wasn't very familiar with the front-end framework used by the company, I was able to fix about half the bugs and then give my best guess as to what the causes were for the remaining bugs based on what I could see, and this was good enough to get me in the door.

The whole thing, between the theoretical questions and the website debugging session lasted about 2 hours.

once you actually get the job, you'll most likely never run into any of those problems from the job interview book again

If their job doesn't involve solving those kinds of problems, why test them on them in the first place?

Why not give them a real problem they would encounter on the job instead?

That's my recommendation. Surely you will have many examples of real on-the-job problems to choose from to find a suitable one.

About a year ago I started to have discussions with candidates. There are many topics that are controversial in the industry – for example:

• What framework is the best?


• Is a certain design pattern actually useful?


• Focus on new features or fix bugs first?

First I ask the candidates about their opinion. Once they got their point across I simply pick the opposite and argue against it. The following discussion will tell me a lot.

• Did they only memorize some facts for the interview or do they actually have experience with this area and good examples to prove their point? Actually, I was very surprised by how often their argument is: because I always did it that way
  


• How well are they able to bring their arguments and the complex technical details across? Are they good in teaching?


• How do they handle resistance? Do they try to convince, do they try to please or do they start to be aggressive or arrogant? Are they open to learning?

I am aware that this might be very stressful for the candidate. Therefore the has to be done very carefully.

• Have multiple interviewers. Your strongest engineers on the team should participate in the interview process - they definitely want to work with people better than the one's that are struggling, right? Let other engineers conduct their own 1 on 1 interview. And have another engineer do the initial phone screens.




• Ask an open ended design question. for a problem of medium complexity. Ask the candidate to write out the header file, class declarations, box diagrams, etc... Does the candidate ask clarifying questions? After he's done with a basic design, introduce a new requirement that would break his design. Is he able to pivot? Pick one part of his design and probe deeper.



• 
  

  Ask a question that probes if they have a deep understanding of the platform fundamentals. Some examples I've asked.

  
  
  
  
  
  • "How does the C++ compiler implement virtual methods?"
  
  
  • "How do closures work in Javascript?"
  
  
  • "Explain when you would ever want to use UDP instead of TCP."
  
  
  • "After you type a www.foo.com into a browser address bar, tell me about all the network activity and protocols involved to get the content on the screen."
  
  
  
  



• Probe for genuine interest in programming. "Tell me about a program you wrote for fun that wasn't for work or school." A good candidate will tell you about an app, website, or hack that he did for his own personal pleasure. A weaker candidate will only tell you about the program he wrote to figure something out for work/school.

I've been most of my life as an electrical engineer, and back in the 90s many of the companies I'd interview with stopped asking me questions about electrical engineering. Between my schooling and my work resume, it was obvious to them that I knew enough about electrical engineering to meet their needs. What did they ask about?

My personality. My ability to work in (and lead) a team. My cross-discipline skills. My capacity to learn.

To be ruthlessly blunt, engineering skills (especially programming skills) are a dime a dozen. There are so many basically competent people out there that there is nothing about programming or engineering that's worth asking (unless you're interviewing a new graduate with no internship experience).

What's important is being sure the candidate is suitable for your environment. How well will they work with the existing personalities in your company? Do they bring more to the table than the skills needed for the basic job? How quickly can they come up to speed with changes in company direction? technology? or design standards? Are they willing to work beyond the basic job, providing (e.g.) scholarly articles, conference presentaions, patents, and other "bring honor to the company" activities? And can they show that they can do any of this?

I can sum this up with a comment I made to Philips Semiconductor recruiters long ago (as an employee). I can teach a 10-year-old to connect the dots when it comes to microelectronic design. Teaching them WHY you connect those dots is another matter. Therefore, you can't judge a new employee by how well they connect the dots. You have to find ways to discover how well they know WHY they're doing what they're doing. That's where teamwork and extra-curricular activities come in. They show depth of understanding.

Use Whiteboard interviews, and ask problems which are related to the skills and business domain that you're interested in. Look for people that you can work with, even if they aren't perfect matches for the position, because you can always train the right person if they have potential. Ask existing employees for recommendations (and offer a finders-fee)

After getting through enough basic questions to show that they have some minimal (by my needs) ability to write code, I move on to the tougher part of the interview, based on two subject areas, their work and my work.

Ask for information about a recent project at their current job. Even with respect to NDAs and IP restrictions, they should be able to explain the business problem, the complexity of the problems and how they overcame them. If they can't explain their own work to me so that I can understand and evaluate, then I worry about how successful they will be.

Ask them to white board a design/solution to one of your current problems. The expectation should not be a working solution but how they approach the problem. Provide enough information that they have a high level scope for the problem. They should ask good questions, and you should provide good answers so they can proceed. They should be able to sketch out some sort of solution. In the end their solution may not work out, but you will be able to see how they work on a problem and whether or not they should be able to work on the projects you will have.

Simply. You just grill the applicants on the implementation details and change requirements on them to see how they respond. Make them use what they've memorized as building blocks.

To be frank, problem memorizing isn't an issue. Memorization is actually one of the key techniques chess players use to learn the game. By memorizing algorithms your applicants are developing a library of solutions from which to work from. This means that they'll be able to solve problems better in the future, if the patterns and implementation of the patterns they're learning are actually useful.

What you need to test is that last part. By varying the problem in real time you get to see how well the applicant actually understands how to use what they've memorized. This means that you can ask them about better ways to solve the question and you can even ask them to compose their solution into a work flow to do something completely different.

What you want is someone who can solve problems. What you don't want is someone who solves problems the way you want to see them solved. If everyone's memorizing solutions, all you need to do is test their ability to use what they've memorized. When it comes down to it, everyone uses what they know to break down what they don't know. If your devs can use patterns they've already mastered to break down problems of increasing complexity, then they would have been able to reverse a linked list if they've never seen anyone do it before.

As a recruiter, this is as far as you can reasonably be expected to go without trying to get free labor out of your applicants.

Temporary paid trials.

Welcome new user, I've come to believe that the only way to find software engineers is, once you've found someone who seems to know what they're talking about in the specific technology at hand,

• Simply pay them for a one or two week freelance contract. Actually have them jump in and do a real project on your real product with the real team.

This is becoming more and more common.

I think that's it.

It's the only way to know.

Everything else is useless, as the OP points out.

The amount of time/money you'll spend on the sundry alternatives ... might as well just put them on the project for a week.

(Depending on your style and project, you might retain 'em for a time period (a week) or some "task" for $x. Either way.)

If you consistently take this approach, you'll find after a year or two that, the money you wasted doing this approach, was indeed far less than the sundry other costs of search and hiring.

"Because I am frankly tired of hiring promising software developers who answer each question perfectly in job interview questions, and then when you actually see them code in a real-world situation, you realize how little they actually know."

Temporary paid trials.

Some further thoughts, the tenor of this QA is

• For software engineers, recruitment processes fail - indeed they're often "hopeless". (One apex of this, and the particular one raised in frustration by the OP, is that hopeful programmers now just "game" the "technical questions" phase of things.)

• Note that even "google-style" absurdly detailed/extended recruitment processes ............ often completely fail. Programmers (in particular) often "just don't work out" no matter how careful the recruitment processes.

• By all means in any industry or mode of occupation, folks sometimes "don't work out" even after careful recruitment. However, this is especially a problem for software engineers - it is notorious.

{You may ask - why? How come it's particularly a problem with programmers? I know precisely why this is, and some people have their own theories on it, but no need to start another argument!}

Even more thoughts!

This answer has a big pile of downvotes and a big pile of upvotes.

Apparently

• for many folks this is a totally obvious idea




• but for other folks it is freaky crazy stuff.

After all, setting aside just programming per se, back in the 90s one of the keys to the staggering success of General Electric at the time was Jack Welch's scheme where each manager simply fired the bottom 10% of people each year! (His book is terrific BTW.)

And I cringe to type the phrase "the gig economy" but in "the gig economy" we are all on "a trial basis" - from day to day and forever.

Winnowing programmers is incredibly, notoriously, difficult.

In some ways, studying for the interview is a good thing. You probably already ask that people study about four years in a University to prepare for the interview.

Now, if your interviewing process is scripted, and the script lacks any kind of variation, then it is just a matter of time before the answers become memorized in the population. Solutions include:

1. As questions that are generated, with variable fields that require the work to be done specific to this asking of the question.


2. Change the question set from time to time, to minimize the time to build the pool of answers.


3. Change the entire approach, giving them a trivial project with specifically added bugs, asking them to fix it and add a known feature.

Also, consider the problem in the larger sense, if you are asking for information in an interview, and the information was memorized, then that should be an ideal candidate, unless you are asking for the wrong information.

You are deciding that the information isn't what is wanted, so you might be able to fix the problem by asking for the right kind of information.

I believe you misunderstand the reason why big companies ask algorithm questions. Writing out the algorithm for quicksort is probably not a very useful skill for the majority of developers. Likewise figuring out the best way to parse a 1 million character string is not something people routinely do at work. However knowing how to solve these problems is a great way of filtering out the dumb candidates. If someone is too dumb for the job, they won't be able to coherently answer any of the interviewing puzzles commonly asked at job interviews. Even if they memorize all the answers, you could easily break their pattern by asking a slightly different variation of the question or adding a bonus question on the spot. In contrast, a good candidate would have no issues preparing for the "test" and would pass with flying colors.

Interviewing is about filtering out the bad candidates, rather than about finding the best candidates per se. There's no harm in having someone prepare in advance, as this wouldn't help someone who's not prepared to work on the right level.

The best interview I ever gave was one of the simplest interviews I gave and was also the best my skills were ever measured. This is what its 5 rounds were:

1. Telephonic with the recruiter: who asked me to verbally go about explaining the answer to a simple coding problem. (The recruiter was not technical.) Answering him demonstrated that I can explain my solution to non-technical people, proving that there is a real connect between the skills mentioned on my resume and practicality.




2. On-site 1: One of the company's senior dev demonstrated their product to me. He paid attention to the questions I was asking during and after knowing their product. This helped them accurately gauge my interest in that domain and whether I was genuinely curious.




3. On-site 2: A tech-manager from the company brought one of their own pieces of code and asked me what I think of it. The code had basic issues - like not using try-catch, not closing resources, poorly named classes and variable etc. The code was working, but was not written responsibly. The corrections I suggested, demonstrated my design skills. If one is not a responsible programmer, this interview would end up being the hardest. Especially for solution muggers.




4. On-site 3: A senior developer discussed with me the design of Merge Sort and after correct design was reached, asked me to implement it in my language of choice. The design discussion was relaxing as I was sure that him and I were on the same page about what we were going to implement.




5. On-site 4: Another senior dev asked me to explain one of my own projects to him. This demonstrated that I actually understood things being done in my current job instead of doing short term and nonsensical code patching.

All the rounds were very relevant and nowhere did I feel that I was being asked to do something out of the blue especially for this interview. Every task was gauging how good a software developer I was, rather than how many answers had I memorized.

Not surprisingly, this company was very highly rated on Glassdoor and had very low attrition. Employees didn't write much about what they were doing but they all were really invested in their company and their work.

In my job, the right solution to difficult problems has come after discussion between teammates. That is what a good interview would try to simulate. Just asking difficult problems in an interview is generally not helpful.

If your candidates are able to pass your interviews by studying, isn't that a strong indication of their ability to study and learn? If you're hiring fresh grads, that's really the most valuable skill to hire for.

If an experienced candidate is doing it, at the very minimum it means they're highly motivated to study for your interview. Which means they're strongly interested in coming to work for your company. And also, it's a strong signal that they haven't lost their ability to learn and grind after leaving college. You should still ask them questions about their work experience, achievements, and collaborations with co-workers to get a better sense of their overall ability.

It's problem-solvers that we want, not problem-memorizers.

Very few people can come up with quicksort, or Floyd's cycle detection algorithm, or finding the longest increasing subsequence in the timespan of an interview having never studied the solution. Those who can are unlikely to be interviewing at your company for regular dev jobs. No offense, I'm sure it's a great company, but statistically most of your candidates are not going to be of that caliber - because very few programmers in the world are.

On the other hand, if your candidates are able to apply solutions to problems they've seen before to variations of that problem or to related problems, it's a signal that they're good at pattern-matching and identifying what type of problem they're looking at. Conversely, if they've never seen that type of problem before you also have signal that they know how to research a solution and understand it - because after all, they studied for, and passed your interview.

Being able to come up with a solution on the spot isn't really that great a qualification, either, at least IMHO. I've certainly come up with a lot of on-the-spot solutions that in retrospect weren't all that great, and have sometimes spent weeks or months finding a decent solution to a really hard problem. (And sometimes the solution is just remembering that I saw something similar in a code example somewhere...)

You don't say whether you're trying to hire new grads or experienced developers, but in either case I think you really have to find some way to look at what they've done previously. Sometimes it's looking at published papers (even if the person isn't the principal author), sometimes it's a personal recommendation, sometimes just asking them to show you something they've done that they're proud of. FWIW, I don't think I've ever gotten a job that I interviewed for that didn't have at least one of those things going for it.

As an applicant, there are a few things I noticed which can still be tested during an interview, even if you're not in the IT field yourself:

• Behaviour-based questions

The objective is not to nail the person you have in front of you, but actually see what kind of person they might be. This gives a short window of whether or not they might be suited for working in the company.

• Past experience questions

[Could you] Tell me about a situation when...

For this one, the objective is to invite the person into telling you more about their experience in a specific situation. This type of question can assert their skills as well as their self-being.

It can be useful if you know the IT teams has some situations which can be difficult to handle (having to handle a lot of tasks in short notice, having to handle tired clients, etc).

• Problem-solving based question

You mentioned you are looking for problem-solver.

I once had an interview with a company where the interviewers didn't ask any programming based question. In order to assert my problem-solving skills, they instead told me about a situation, and I was told to think aloud so they can witness my problem-solving skills in action. For example:

We have a plane with 50 seats, and 50 passengers awaiting to come in. The first two passengers lost their cards, so they will sit in a random seat. The next passenger will sit in their seat if they can (if it's not already occupied); otherwise, they will sit in a random seat. What is the probability for the last passenger to sit in his own seat?

The objective is not for them to find the solution, but to show you how they tackle the problem.

~~~~~~~~~~

For each of these questions, remember to take note of the applicant's answer. This way, you can better discuss about them with the IT department manager.

I don't think you need to change the questions, just how and what you ask.

It's gotten to the point where you pretty much can't ask a question without the applicant already having pre-studied at least a version of that same task and hence knows, roughly, what the solution is.

This isn't a zero sum thing, you can feed off the solution for an entire interview on it's own if you want, and get a deep understanding of the interviewee's skills (assuming as the interviewer you have your own technical skills).

You present the question, the interviewee creates a solution (maybe code, maybe whiteboard). This is the start:

• Take me through the way this works


• What factors led you to choose this solution?


• Are there any other ways of doing this?


• Why is your solution better than the others?


• Are there any potential advantages in the other solutions missing from your solution? what scenarios might that apply?


• Having just written this off the cuff, any refactorings you'd like to do now you've looked it over?


• If you didn't do this test driven, what tests would you be looking to write/do to ensure this is correct?


• Any bugs you've noticed?


• Do you think the code is production quality? Could another developer support/refactor it without knowledge transfer from you? What would you need to change to ensure it's intent is understood?

Someone who has memorized an algorithm or pattern without understanding it will not be able to drill down into detail, but a good developer should be able to have discussions on these topics, even if their raw solution wasn't perfect (and seeing someone discover a better solution while talking to you and be prepared to call it out shows a maturity over code/design reviews and refactoring which are highly desirable).

The problem is that you're not testing for the skills you're looking for.

A process I've used both as an interviewer and interviewee that has worked well:

1. A quick 15 minute phone screening to find out if the person is competent and someone you could see yourself hiring.




2. A simple coding exercise for the applicant to complete on their own time. This is where you get a feel for actual problem solving and coding skills. The exercise should not take more than a few hours and you can optionally pay them for their time.




3. The actual technical interview, which involves reviewing the coding exercise and discussing the solution and the decision making process.

Most of the (currently) highly rated answers to this question suggest some sort of live coding as a part of the interview. I would highly discourage this as it is not at all representative of a real world scenario. It's a high pressure, unnatural situation where a candidate is likely to produce an inferior solution than they otherwise would, with no additional benefit.

A number of people have answered "The questions don't match the needs of the job" - and that the solution is to change the questions to something that more accurately reflects what's needed to know how to do the job.

I'll take a different angle: This probably has a lot to do with recruiters/interviewers not putting in the time to create their own questions.

In our group, we're currently interviewing .NET devs, and here are some of the questions from our interviewing process:

• We've got these two SQL tables with these columns. Write a query that gathers data X, filtering down based on Y, with a corner case of Z being handled in a specific way.


• Here's a badly-written 8-line C# function; Code review it.


• Write a C# function that takes an input of a filename, and outputs how many bytes there are before the first null byte in that file.

The answers to those probably aren't going to be found in a '140 questions to memorize before an interview' - because we came up with the questions from scratch. We didn't copy them from another company, or crib them off a 'Good Interview Questions' site. Yeah, it takes more work... but we don't have to worry about someone already having the answer memorized.

I've had to recruit 5 developers of different skill levels over the course of the past year and it's not been an easy process. The best method we found was to make use of online programming tests to filter out the weaker candidates before we interviewed.

We picked a coding test website (CodinGame, though other alternatives are available) and asked candidates to sit a test before we interviewed them.

Pros:

• Filtered out the 'looks good on paper' candidates who didn't live up to their CV / resume.


• Allowed us to see the candidate's working - we offered interviews to people who didn't score highly on the test but by reviewing what they had done to solve the problem gave us a good idea of their analytical skills.


• We viewed candidates who weren't serious enough about the position to do a 50 minute online test as not worth interviewing, this filtered them out.


• We could set tests at different skill levels.

Cons:

• Some candidates, when faced with several interviews, would go for the path of least resistance and choose the the ones that didn't involve a coding test.


• You need a few candidates to take the test to get a benchmark of what is a decent score.


• There is a cost involved in taking the test.