A Deeply Flawed Metric
Recently I have seen more and more recruiting functions focusing on the quality of hire or even using it as the primary measure of their success.
This is unfortunate.
One of the most difficult aspects of creating a metric is a precise, clear, measurable definition. And here is where the challenge lies when talking about the quality of a candidate or a hire.
What exactly do we mean when we say quality of hire? It is how long they work for us. Is it how well they are liked? If so, by their boss, their team, or their peers? Are we measuring performance? Against what standard?
As you can already see, nailing down a definition is just the beginning of our challenge.
Quality, as defined by manufacturing standards, is the elimination of variance. In other words, the more an object or a process is exactly like a similar product or process, the higher the quality. Variance is measured in thousands of percentages with a goal of at least 99.999%, realizing that 100% is never attainable. This is called Six Sigma and means only 3.4 defects for every one million opportunities.
But can we define human performance in this way?
Perhaps in some cases. We might say that high-quality coders, for example, write N lines of code in an hour or a day with fewer than 3.4 errors for every one million lines of code. But how do we define such a measure for leadership? For creative writing? For marketing success? For human resources? For building relationships with candidates? In some cases, we can count how many of something (e.g., novels or stories written, relationships formed on LinkedIn), but we cannot understand the value or efficacy of these numbers.
Do We Have Data?
Once we have a definition and a target measure, there needs to be a data set large enough to be objective and reliable. For example, testing a vaccine requires thousands of people to take the vaccine to determine if it is effective. The sample size needs to be quite large. One of the fallacies in statistics is called the “statistics of small numbers,” where we use a small number of instances and extrapolate those to an entire group. Hiring managers often do this by basing their ideal hire on one or two others they have hired. The only objective way would be if that manager had hired thousands of people and tracked each of them.
If we adopt the definition of quality that is the hallmark of manufacturing, quality is not about perfection or opinion; it is about reducing variations. In recruiting, this would mean reducing the variations or differences in skills between candidates as much as possible for a similar position. We would need hundreds or thousands of exactly the same type of hire even to begin to judge quality. And we would have to show that not having variations in skills leads to better outcomes.
Taken to the extreme, this would mean having to find and hire people who are virtual clones of each other. This is an inherent danger in using artificial intelligence to determine the best candidate. As it functions today, A.I. matches specific historical data such as pedigree, skills, or previous experience to those asked for in a job description. But we have little objective data to show that these are the best indicators of quality or performance.
Trying to remove the differences between candidates works against us because it reduces diversity, potentially leads to groupthink, and inhibits innovation.
If a job is completely defined and quantifiable, a robot would be a better choice than a human. This is often the case in jobs where rules and procedures apply, and there is little room for judgment.
Without a well-defined, objective measure, we are left with subjective judgment. We all have been in situations when one manager thinks an employee is great, and another manager feels the opposite. Virtually every way recruiters define quality has flaws that make the measure useless.
Flaws In Typical Measures of Quality of Hire
#1 Flaw: Speed to Productivity
The first flaw is measuring how quickly a new hire is productive. Why is speed important, and how do you determine the standard? Is productivity more important than judgment or innovation? In a handful of repetitive jobs measuring speed could be used fairly, but for most jobs, it does not provide useful or valid information. Perhaps an in-depth understanding of the work by observing and learning and getting a thorough grasp of details would be more useful and critical to long-term success than speed.
#2 Flaw: Definitions
The second flaw is not defining precisely what productive means. This is very hard to do for positions where the output is not tangible or varies from time to time. Is it possible or fair to tie the level of an employee’s productivity back to the hiring process? The variables that influence productivity include training, the work environment, the new employee’s teammates, and the corporate culture. There are too many variables to say that the recruiting process alone causes or even has any real impact on performance.
#3 Flaw: Performance Ratings
The third flaw is measuring quality by looking at a new employee’s performance rating (assuming your firm still has performance ratings). Once more, this is a post hoc ergo propter hoc fallacy. It does not follow that recruitment leads to poor or good performance. Once again, there are too many other variables that affect performance far more than recruitment. Performance ratings are subjective and depend on personality, and the relationship the new hire and the manager have with each other as on actual performance. Many organizations have recently abandoned performance ratings because they are not objective, increase employee dissatisfaction, and do not improve performance.
#4 Flaw: Turnover
And finally, the fourth flaw is using turnover rates as a measure of quality. Whether someone leaves in a short period of time or a longer one is much more likely to be caused by the manager, the economy, the organization’s product vision, economics, the corporate culture, or some other factor than because of the type of person recruited.
Quality of hire remains elusive and impossible to measure validly and objectively.
We need to either find measures that we can all agree are fair or stop trying to measure quality in ways that lead to biased and unjustifiable exclusions.
Let’s Keep This Sponsor Free
It really helps to have your financial support to generate articles each week and bring you unique features occasionally. Your small investment makes a huge difference to me and lets me keep this publication free of vendors and sponsors.
As a paid subscriber, you can access past articles and white papers over the past four years. This is not available to free subscribers.
Paid subscribers can start threads using the new Chat feature.
Your subscription keeps this newsletter ad-free and unbiased.
A few times a year, I publish something just for paid subscribers – a white paper, report, or podcast – I try doing two or three of these yearly.
Thanks for considering upgrading to a paid subscription.
Book a Speech, Briefing or Webinar
If you would like me to speak at an event, virtually or in person, or give your team a briefing, I would be pleased to discuss it with you.
Could You Use A Trusted Advisor?
If you are looking for guidance or help in becoming a more strategic leader, we may be able to help. For the past twenty-five years, I have been helping recruitment leaders in major corporations, non-profits, and NGOs to redesign, improve, or transform their talent acquisition functions. I work with you as a partner to assess and improve your processes, find and remove constraints, create more engaging career sites, and choose the most useful and relevant technology. I will work with you as a coach, mentor, or consultant – whichever meets your needs. I have only one goal – to make recruiting strategic and pertinent to your organization. Let me know if I can help. Please send me an email at kwheeler@futureoftalent.org.
Recent Comments