Whiteboard-style Coding Interviews Might Not Be as Bad as You Think

Photo by ThisisEngineering RAEng on Unsplash

Whiteboard-style coding interviews has a bad reputation, and they are being perceived as "bad interview practice" in general within the software industry (e.g. "Why whiteboard interviews are BROKEN", "Coding Interviews are Broken", and there are many others like these). Despite this perception, many companies (especially the top-tier tech companies, a.k.a. FAANG and the likes) still hire software engineers through this interviewing process.

What I mean by "whiteboard-style coding interviews" is an assessment through algorithms and data structure based coding challenges which are performed within a coding environment that doesn't reflect the day-to-day experience of an engineer, e.g. whiteboard, rudimentary online coding editor, etc. This is also known as "leetcode-style", interviewing through CS trivia questions, etc. So, a physical whiteboard doesn't necessarily need to be present, but it's a name that's often associated with this type of interviews.

As someone who has conducted 100+ tech interviews within the last 4 years and also as someone who has failed this type of interviews nearly dozen times while being successful with a few of them in the last 10+ years, I would like to express my thoughts on why this perception might not be the most accurate one, why organizations still stick with this type of interviewing style despite the negative perception, and also why this type of interviews can and should even be perceived positively by candidates.

I am a person who believes that there is no cookie-cutter approach to the problem of hiring software engineers effectively. Each organization should essentially do what's working best for them, and most importantly it should be encouraged to experiment with different approaches to find the near-perfect balance based on what the organization is trying to optimize for. Therefore, my intention with this post is not to try to reject the hypothesis of whiteboard-style interviews being "bad interview practices". That said, in general, I do believe that gaining a different perspective on this type of interviewing processes will be helpful to all of us, mainly to understand the problem which is aimed to be solved through this type of interview processes. This is not only for us to eventually accept the process as is, and move on with our lives. However, if we understand why the process exist today in the way it's, we will then be able to form a better judgment of the process from the perspective of both sides, and we can then challenge the assumptions, and influence a change if necessary.

⚠️ Disclaimer 1: the content of this blog post is based on my personal opinions, and by no means represent the view of my current or former employers. So, just keep that in mind while reading the post, and be nice 🙂

⚠️ Disclaimer 2: I am acknowledging that interviewing is an area where quantifying the success of the certain interviewing processes is really challenging. Also, I am aware that all the information I will unleash here are not backed by any scientific evidence. However, they come from an experience. So, I still see it valuable to share. Nevertheless, take all this info in this post with a grain of salt.

⚠️ Disclaimer 3: I am aware that there are organizations in many shapes and sizes. It would be naive to think that there would be one interviewing style that would work for all of them. In this post, I am mainly focusing on organizations which work on tech-centric products, where the tech side of the business is not there to just implement features and fix bugs, but instead it's at the center of your organization to drive product decisions and innovations (in these organizations, tech becomes the organization's competitive advantage in the market).

In some parts, I might be implicit about this fact. So, keep this in mind. When I am referring to an "organization", you should now know what I mean by that.

There are Various Hiring Strategies

Let's start seeing the problem of hiring software engineers from the organization's view point. I personally believe that defining what type of software engineer you want to hire into the organization is a multifaceted problem, and plays a crucial role on understanding the reasons behind the interview process of an organization. I want to touch on three of these dimensions to narrow down the focus of this post. I am super aware that this is not an exhaustive list. However, based on my experience, they play a significant role on shaping up the interview process that software engineers go through. My hope here is that the information in this section will set foundational knowledge for us to understand why a certain interview process might be set in the way it is today, which will/should eventually make it more clear what the problem of "whiteboard"-style interviews are trying solve (more on that later).

Today's Problems vs. Tomorrow's

The hiring criteria usually differs a lot depending on whether you are hiring Software Engineers to solve today's problems vs. tomorrow's. This one indeed plays the biggest role on what type of interview process and what type of coding assessment Software Engineers may need to go through.

Today's Problems

Today's problems for an organization are well-known, and we are already aware of the challenges of those problems even if they may have not been solved yet. If you are hiring engineers on solving today's problems, meaning that you know what technologies, technical challenges, and architectural structure you will be working in the long run based on the today's view, you will have a pretty good idea on what you want from a Software Engineer.

For instance, if you as the organization are on k8s and you intend to stay on k8s for the rest of your existence as an organization, it's highly logical to assess the candidate's knowledge and experience on k8s as part of your interview process. Another example could be given here for a specific domain. Let's assume that organization's products are centered around a search functionality. So, it could be acceptable to think that organization can aim for hiring Software Engineers who have prior experience on search domain, and the assessment criteria could be centered around search domain during the interview.

This type of interview processes are relatively easy compared to the next topic we will discuss, because the organization has a high chance to be able to solidify the assessment criteria while also leaving less room for personal judgment. On the other hand, this type of hiring approach comes with a great deal of risk, and the reason is quite simple: tech-centric product design and development is highly volatile. It's not just that the problems, which we have to deal with, keep evolving, but also the technologies we use to solve these problems. These sometimes even change in the direction which haven't been imagined yet. Therefore, if your assessment criteria during an interview process is centered around a specific area, regardless of this being a specific technology, domain, or problem solving technique, you are betting on the successful candidates to adopt to these changes, be able to solve the unique problems which come with those, while also being able to innovate. Or, maybe as an organization, you are more naive and thinking that things will stay still for the forseeable future, and you will just be fine. However, I am hoping that no organization is betting on the latter.

Tomorrow's Problems

What I mean by tomorrow's problems is the notion of uncertainty from the user, business and technology centric challenges perspective. It's critical to being able to deal with this uncertainty potentially within an ambiguous environment (uncertainty often brings out ambiguity regardless of whether the organization is well-structured or not by default). Based on my experience, this is much more close to the reality within the current tech industry where things are changing fast, and you are required to adopt and move fast.

As an organization that wants to hire Software Engineers who will be able to cope with the challenges of both today's and tomorrow's problems, you want to hire engineers that fit into the creative thinking process through their sharp problem solving skills with critical and analytical thinking. This may sound great, but assessing engineers against these criteria is much, much harder compared to the process to hire engineers for today's problems. There are a few reasons for this, again, based on my own experience:

• The criteria described above itself doesn't reveal a specific domain or technology for you to be able to pick for the assessment. This eventually guides the organization to boil down the evaluation to be performed on the basis of fundamental knowledge and problem solving skills (more on that later).
• As there is no specific technology or domain here to assess, individual interviewer judgment might be more significant. This is especially the case when the interview process is set up correctly and effectively, meaning that the decision is not binary i.e. it's not "solved the coding problem. Therefore, it's a pass!", or vice-versa.
• It's a challenge to make this type of interview process work in scale, both for the organization and the candidates. This is especially the case for big tech companies, which have thousands of engineers working for them.

Mixed Hiring Strategies

There is also a room for a hiring strategy which is mixed with both of the criteria mentioned above. At the end of the day, majority of the work for any tech-centric organization is going to be around problem solving through a creative thinking process. However, there is probably still small amount of (I would like to unscientifically say around ~10-15%) work which needs to be completed in the short-medium term, and requires specific skills or knowledge and experience on a specific technology. When you have this need, it's common to see organizations adopting a different hiring process for that to hire employees for the short term (while also still hiring full time employees to solve tomorrow's problems), i.e. as contractors. This is completely valid, and works well for both sides as long as this is kept in a minimum, and doesn't become your default and only hiring strategy. The reason is that:

• The organization knows what they want in terms of hiring criteria, which happens to be a specific and easy to assess one.
• Expectations are set correctly for both sides, i.e. short-term employees know they are hired to execute on the work based on their existing skills and organization knows that there are folks who are hired purely for execution.

Hiring Into a Team vs. Company

Depending on where you want to land a prospective employee after the successful outcome from the interview process, your hiring strategy can also differ, and can even be multiplied. If you are an organization where your teams have longevity, and work on specific domains, it's valid to hire into a specific team. This gives autonomy to each team to be creative about their own hiring strategy while also allowing the team to be much more specific about the assessment criteria, which will potentially lead to multiple hiring strategies to exist within an organization for the same role.

The other option is to hire software engineers into the organization based on a generic criteria, and defer the team selection to a later point. This often works better based on my experience, as it makes it much easier for engineers to move between teams within an organization which further helps for an organization to retain the talent under circumstances where the employee wants to change their team for one reason or another.

Specialist vs. Generalist

It's common that software engineers sometimes end up specializing within an area, e.g. Backend, Frontend, Test, QA, iOS, Security, and so on or and so forth. The list can go on, but the truth is that each of these roles requires different skill sets even if all these will require the person to write code and implement software to a certain extent.

On the other hand, some organizations purposely hire "Software Engineers". This can vary from organization to organization what this mean but in general this refers to the generalist software engineer, where they are well-versed on solving problems through designing, and implementing software without necessarily constraining themselves within a workflow. In my experience, even these hires end up specializing in one area of software engineering. However, these engineers can still contribute to pretty much throughout the whole lifecycle of the software delivery process.

In general, an organization's chosen interview process can be very different when it comes to hiring a specialist vs. generalist, but they can still have some fundamental common characteristics. I will touch on this later in this post.

Hiring Strategy Relationship with Whiteboard-style Coding Interviews

What does this information have to do with coding interviews, more specifically the whiteboard-style coding interviews? Having an understanding on what type of hiring strategies are out there, and which one is being used by the organization I am interviewing with helps me gain a wider perspective about their interview process, and makes me emphasize with the strategy. If I am convinced that hiring strategy and interview process aligns, this motivates me more. I am hoping this will at least be similar for you.

So, for the rest of the post, I will be making the assumption that the organization's hiring strategy has the following criteria:

• Hiring engineers to solve tomorrow's problems, not just today's problems.
• Hiring engineers into the organization, not particularly into a team. Even if the intention is to land them into a specific team, you are making the assumption that they can move within the organization.
• Hiring generalist engineers, not specialist.

I made these choices here not because the alternatives are somehow bad. The main reason is that these are the criteria that the many organizations generally choose as part of their software engineer hiring strategy, and these contribute heavily to why whiteboard-style coding interviews become the core part of the assessment throughout the interviews.

Whiteboard-style Coding Interviews

There seems to be notable amount of speculation about the negative side of whiteboard-style coding interviews. I say speculation here on purpose, not because it's people's intention to speculate, and all of the information out there is completely speculative. I say that because genuinely there are a lot of misconceptions and lack of rationale out there when judging these interviews. As humans we sometimes forget that most things are not black and white, and trade-offs matter. I believe that's what really is happening here. We end up judging these processes from one side: the candidate's.

While acknowledging that every organization's intentions are different with this type of interviews, I personally believe, with the experience I have both as an interviewer and interviewee, I have a pretty solid idea on why these interviews are shaped up in the way they are today in general. Most importantly, it's actually largely positive for the candidates that these interviews are set up the way they are today.

These Interviews are Intentionally Structured

This type of interviews are assessing the candidate's core problem solving ability through coding while also assessing the candidate's fundamental computer science knowledge around algorithms, data structures, and complexity analysis. Throughout the interview, interviewers will look for signals to give them a higher confidence on these fronts, and this is the key to understand! The evaluation here is not binary, and it will likely be linked to how you communicate your ideas as well as how you execute on them, but let's ignore that for now to purely look at why these matter. If we go back to the tomorrow's problems point we touched on above, the reasoning will make much more sense. Even if the organization accepts the fact that tomorrow's problems are unpredictable and full of uncertainty, they need to be able to form some assumptions around some commonalities of these problems. One of those assumptions happen to be that tech-centric problems will eventually require understanding of the core algorithms, data structures as well as being able to solve problems effectively through coding.

They also assess the candidate's critical thinking ability within context of a problem that needs to be solved through coding. This gives a pretty solid idea on what type of thought process you have, and how you reason about problems as a Software Engineer. This extends from thinking about edge cases to seeing opportunities within your solution to optimize it proactively in terms of various aspects (e.g. modularizing your solution, improving the runtime performance, choosing your test cases, etc.).

These coding challenges also set up a pretty good environment to be able to assess your analytical ability, to a certain extent. Can you reason about how your solution would perform with different input sizes? Can you ask the correct questions upfront to gather an accurate enough analytical reasoning, and proactively determine the optimum solution according to the info collected? All of this can give pretty accurate signals.

One of the things that these questions don't asses is the rote knowledge of the candidate on a particular technology such as a programming language. Unless you are being hired as a specialist on a specific programming language, you will likely be given the freedom to choose the language that you are most familiar with, even if that language is not among the languages that the organization is using. This is great, as it gives you the choice for the language which you are the most comfortable with. However, this doesn't mean that you don't even need to be fluent in that programming language. It's quite the opposite. As you are given the choice to choose the best language which you feel comfortable with, you are expected to be fluent with the basics of the language, and it's very likely that you will only need the basics throughout the interview.

As a contrary to point above, one other thing that's not being evaluated is your rote knowledge on the syntax of the programming language you have chosen. As you will likely be performing for these interviews within an unfamiliar environment such as an online code editor that won't have autocompletion, or any other IDE features that you might already be familiar with day to day, it's understandable that you may make some syntax mistakes here and there, and your code may not actually compile. This is totally OK, and you should not worry about this too much. That's one other reason why you will likely not be given any option to be able to run your code. The core reason for this is to prevent to noise around retrieving the accurate signals that the interviewer is looking for, and get you spend as much time as possible on the code logic, implementation, and testing. There is also a fundamental assumption here: if you are an engineer who is able to perform well within these interviews by solving the given problems effectively, you will almost always be able to find syntax problems and solve them. So, the potential time that you might have spent fixing these issues would have actually lowered your chances because you would not be giving any useful signals to the interviewer during that time.

Hiring Strategy and Coding Interview Structure

If we relate the structure of the coding interviews with the hiring strategy I mentioned in the previous section, we should now be able to see how things start to make more sense (🤞🏼):

The organization, which we are basing our assumptions on in this post, wants to hire generalist software engineers into the organization (not into a specific team) to solve tomorrow's problems, not just today's problems. So, the assessment is not restricted according to the work the organization is taking on today. The software engineering candidates are evaluated according to their core problem solving and computer science knowledge, based on several different coding challenges in varying degree of difficulty to maximize the chance of accurate, and high quality signals to be retrieved to reduce the chance of a false-positive, or false-negative. Coding interviewers are only one part of the entire interview process. However, they on its own give pretty accurate signals for the minimum bar around he candidate's:

• skills to use a programming language to solve a particular problem through coding, while being able to form an algorithm
• ability to be able to critically think about the problem, while being able to ask the right questions to widen their understanding on the problem, proactively finding out the edge cases, and potential optimizations
• analytical reasoning for a given problem, according to requirements which may or may not be ambiguous to start with.

Whiteboard-style Coding Interviews are not Perfect

Whiteboard-style coding interviews are no silver bullet. The hiring is a complicated problem to solve in general, and this becomes much harder when it comes to hiring talented people to fit into an environment where creative thinking, and core problem solving skills matter the most, if not as much as the core technical skills. It would have been very naive to think that one solution would work perfectly to solve this complicated, multifaceted problem. It's again worth emphasizing that every organization is different. However, we should have a shared understanding by know what type of organization I am referring to.

As applicable to nearly everything else in our precious World, this type of interviews comes with their own trade-offs, and it's worth acknowledging them so that we can do as much as possible to mitigate them. The following list of negatives about these interviews are based on my own experience, and not meant to be a definitive and exhaustive list.

• They require significant amount of preparation for candidates. This is a legitimate one, and I would like to believe that it's one of the biggest negatives that's felt commonly among all the candidates, including myself. As it's likely that you are not working directly with the all data structures day to day, and spending all your time around algorithmic problems, you need to at least refresh your memory around these concepts, while also making sure that you can give the accurate signals to the interviewers within 45-60 minutes. So, you gotta have the knowledge and skills, and prove that you have it fast. This requires preparation, and not everyone has the privilege to dedicate this amount of effort.
• The structure can be copied by some organizations without actually forming an hiring strategy at the first place. This is the worst negative about these interviews, by far! As this interviewing style is adopted by top tech companies, they must be working effectively and efficiently, correct? Great, let's adopt the same process, and call it a day. Well, not so much. I hope that we now have a shared understanding by now that hiring strategy plays a huge role on why these interviews exist in the way they are today. So, without understanding the why, it's going to be a miserable experience for your organization and the candidates if you end up copying the process. Don't do that, understand your organization's unique problems first, at least then see if this type of coding interviews are a good fit for the problems you are trying to solve as part of your hiring strategy.
• They are occasionally executed poorly, and they lead to either false-negatives or false-positives. Sometimes the execution doesn't meet with the intentions or the expectations, and this type of interviews may occasionally be performed poorly. This could be a one-off occurrence due to the interviewer's own performance, or it might be a systematic issue (it could actually be related to the point above). For instance, the candidate might be rejected because they didn't get the syntax of the programming language correctly. It could also be that the candidate has performed really well on a coding interview because they already knew the questions, and executed flawlessly on paper without actually giving much high quality accurate signals around the problem solving ability. Highly calibrated interviewers have the ability to retrieve the correct signals under these circumstances to understand when this is happening, and can often mitigate this type of situations really well. However, less-calibrated interviewers may sometimes overlook this, and potentially lead to a false-positive outcome.

💡 A word on false-negatives: The topic of false-negatives is outside the purpose of this post, but comes up a lot when coding interview topic is brought up. False-negative interview outcomes are the ones which concludes the candidate's rejection when the same candidate would have actually performed well on the job (speculatively) even if the candidate's performance was less than strong during the interview itself. These are super common in software engineering interviews, and more common especially in coding interviews. These outcomes might potentially cause an organization to miss out on incredible talent. However, when you think deeply about this, false-negatives are actually much more favorable compared to false-positives for the organization's, their employees' success and happiness in the long run.

The reason is simple: false-positives will lead to a potential decline of the talent within the organization, and this can grow exponentially. Once this starts happening, the talent you retained in your organization might also start being unhappy about this because talented employees tend to want to work with talented colleagues, and this will eventually lead to attrition of the talent and high turnover rate.

Let's assume that within the last month you had a hundred borderline outcomes through your interview process, and you let half of them slip while taking the risk on these employees, as you didn't want to miss out on a potentially great talent. Let's again assume half of these hires turn out to be false-positives, and they start interviewing candidates within 6 months or so. They will likely end up hiring engineers under the bar, and good portion of those engineers will likely also turn out to be false-positives. Those hires will also end up interviewing engineers within the next 6 months or so, and this cycle will keep continuing, and it won't be much longer to see the impact of this exponential cycle.

You should also remember that once you hire the wrong person into your organization, letting go of that person is super painful and a long process, unless you are adopting a structure similar to what Netflix operates under (which I personally respect and relate to).

• These interviews favour new graduate software engineers more than the seniors. The assumption seems to be that the new graduate software engineer candidates can perform much better in this type of interviews as they have the fresh knowledge from the college around all the core algorithms and data structures subject, whereas the senior engineers won't have the fresh knowledge around these. This might reflect the truth depending on what type of job a senior engineer is performing day-to-day. If you are not making use of these concepts during your daily job, you might need more preparation. However, going back to the core evaluation criteria, it is also likely the case that senior engineers potentially have the edge when it comes to critical thinking ability, and core problem solving. So, this one is a bit up in the air for me even if there is some truth to it.

• These interviews not at all in favour of software engineers who don't have a formal computer science education. This is largely correct, but also largely depends on what type of learning you have accumulated. If we were to speak commonly though, it's going to be very rough until these candidate can feel comfortable around this type of coding challenges, let alone feeling confident to perform well on the interviews. I am one of these folks without a formal CS education background. I still remember the time where I bombed a coding interview long time ago when the interviewer asked me to implement a stack data structure without using any collection types. I didn't really know how to it, and wasn't also able to reason about the problem. However, I took this as a challenge, and over the years, I have accumulated tons of knowledge around algorithms, data structures, and complexity analysis thanks to this interview style. More importantly, this knowledge proved to be immensely useful many times while I was designing, implementing and reviewing systems, and code changes. Long story short, this is not really a negative. It's either an excuse or opportunity for you to learn and grow yourself. In an era where the entire lectures of formal CS courses are published online (e.g. see Introduction to Algorithms and Advanced Data Structures from MIT OpenCourseWare), this knowledge is much more easily acquirable than you might think.

Untold Truth?

⚠️ OK, this section is purely based on assumptions, and totally speculative. If you are type of person who likes to focus on facts, or at least the opinions formed based on experience, you may wanna skip this. If not, buckle up!

We should be open about the fact that this style of coding interviews require us to prepare, and that takes significant amount of time commitment from the candidate's point of view. It's a privilege to have that time, but let's assume you have that, it also still takes significant amount of diligence and perseverance to be able to perform strongly in these interviews. The outcome is also not left to luck. It's common that the candidate goes through 4-6 coding challenges throughout the hiring process, and one less than strong performance could be enough for them to get rejected.

I personally believe that the preparation characteristics of these interviews on its own puts off great deal of candidates. These candidates could actually perform well on the job if they were to get it, but they never try because they don't have the enough enthusiasm and perseverance to go through this process (even if they might have the time). I also speculatively believe that this is a good thing for the organization, especially the ones which can already attack significant amount of candidates, because they are implicitly evaluating the candidate's enthusiasm for the role, and general attitude on perseverance through this characteristic of the interview style.

I don't know whether this is the correct assumption or not, but wanted to put it out there here. Considering that software engineering hiring game favouring false-negatives over false-positives (for the right reasons in my opinion), this assumption might not be too far off.

Common Misconceptions

Based on the common rationale and understanding we have laid down, I believe it's now fair to be able to judge some of the common misconceptions about whiteboard-style coding interviews. Here are some of them that I am aware of which are worth specifically highlighting, this is not meant to be an exhaustive list:

• You are being asked about questions which don't reflect the day to day experience of the engineers working in the organization, and there needs to be a system that should close the gap between the interview and the actual job that is being interviewed for.
  • If you are hiring software engineers only for today's problems, by all means, you should absolutely do this. If, however, you want to hire software engineers to solve not just today's problems, but also the tomorrow's ones, this advice introduces a lot of risks for the business.
• Interviewing environment doesn't reflect the reality, e.g. you are left with a whiteboard or code editor which doesn't have autocompletion and even a compiler.
  • This ties back to assessment criteria and the signals which the interviewer is trying to retrieve from the candidate throughout the interview. Being able to get the syntax of the programming language, and how to use the compiler is not one of those criteria. The potential time that you might have spent fixing the potential issues related to these areas would have actually lowered your chances because you are not giving any useful signals to the interviewer during that time. So, it is actually in your favour that you are forced to use a rudimentary coding environment for the test.
• The candidates with interviewing anxiety is being passed on unfairly.
  • Who doesn't have interviewing anxiety? Interviews are stressful as hell, but being able to control the stress and anxiety is also part of our lives. I also have to put this here: if actors with millions of dollars net worth are willing to take auditions, I am sure we, as Software Engineers, should be able to justify to ourselves that it might actually be reasonable to take the job interviews 🙂

Conclusion

TL;DR; is that whiteboard-style coding interviews exist for certain specific reasons, mainly to be able to assess the candidate's core knowledge around algorithms and data structures, and how well they will likely fit into the creative thinking process through their sharp problem solving skills with critical and analytical thinking to solve not just today's problems, but also tomorrow's unique problems within a potentially ambiguous environment. I believe this is largely a good thing when these interviews are setup and executed effectively by the organizations and interviewers. They often give a strong indication on what type of organization you are going to be jumping into, which usually tends to be the one who sees software engineers not just execution machines, but also as part of their product development and innovation process, and values their abilities much more strongly.

They surely have some downsides, biggest one of which is that these interviews require significant upfront preparation, and not everyone has the privilege to commit a significant amount of time for this. The process itself also favours false-negatives over false-positives, which is a very good thing for the organization and their teams, but can be super demoralizing for the candidates.

They are not perfect for the problem that they are trying to solve, for sure. However, I am yet to see an alternative solution to this problem that works as effectively and efficiently as this one when hiring at scale according to the criteria we have just laid down (yes, take-home assessments are not an alternative!).

Further Resources

• Coding Interviews are Broken
• Why whiteboard interviews are BROKEN
• Thread based on Jaana Dogan's recent Tweet on this topic Very informative thread, it can be useful if you want to gain further perspective on how different people think about this.
• Coding Interviews are NOT Broken
• Hiring Without Whiteboards, it seems to be curated list for companies (or teams) that don't do "whiteboard" interviews
• Get that job at Google Steve Yegge's very informative post on hiring at Google. Very old now, but surprising still a lot of relevant bits. It's worth checking out.
• Moishe Lettvin - What I Learned Doing 250 Interviews at Google
• Coding Interview Problem: Least Disruptive Subrange Very helpful example coding challenge walk-through with full of rationale from Jackson Gabbard, an ex-Facebook Software Engineer.