It's Not THAT Hard to Hire Data Scientists

We recently started working with a client that we have coveted for years. Their data science team (which has a great reputation) powers the company’s core product, so it is not only important to the business… it is the business. They have 65 people on the team: 10 or so engineers, 10 or so people in data acquisition / data management and the rest are modelers. They have a fully funded mandate to double the team in the next 12-18 months. They appear to be a perfect client for us.

They started us with 2 searches: Data Scientist and Sr. Data Scientist. They interviewed 11 people over 3-weeks who met all of the qualifications. Feedback on our candidates included:
• No wow factor
• Solid but not a superstar
• Can do the job but probably won’t raise the bar
• We want people with MS or PhD in Statistics only (no candidates with advanced degrees in other quant disciplines).
• We have several people from xxx (former employer) that worked with him. Nobody raved about him.

None of the candidates made it to an on-site interview. Unfortunately, this is par for the course in 2018. Weeks of work go down the drain as we strain relationships with a dozen qualified candidates.

There is a disease of thinking among data science leadership that tells them that nobody is good enough, smart enough or talented enough to work for their team. This has grown into an epidemic in the past couple of years.

Our close rates have plummeted over the past 2-years while companies that claim to have many vital data science openings treat recruiting like Excalibur and the sword in the stone, waiting for a divinely appointed true king of data science to step forward and show his pure heart, mind and royal lineage.

If you look at the LinkedIn profiles of the data scientists currently on the team you will usually see no trace of the superstar qualifications that are now required. None of these people could get hired into their own jobs today. Positions go unfilled for many months. Thousands of man hours and millions of dollars are spent on recruiting processes that seem designed (or at least destined) to go nowhere.

This is the new normal among top data science teams. Flaming hoops that burn up 99.99% of the candidates that dare to jump through. Why? What is happening here?

Sure, there are nuances to each data science group that make success there different from success elsewhere… but that does not seem to be the issue. Many of our candidates are not being shot down for failing to satisfying these intricacies of qualification. They are being shot down for not having “X Factor.” They are competent and qualified but not “superstars.”

This begs the question: how are you going to fill your dozens of jobs and get all of your work produced if you are holding out for subjective superstar qualities? At some point don’t you have to hire qualified people who can do a good job? At some point doesn’t senior leadership hold hiring managers and HR teams accountable for failing to fill their roles… for failing to iterate the search to match the reality of the marketplace and the needs of the business? Apparently not. Apparently it has become acceptable to fail and just say: “It is really hard to find great data science talent.” It’s not THAT hard.

One of the interesting things that I have noticed is that the people that DO get hired onto teams like these are often missing the key qualifications. When we ask how this happened the answer is usually that the person they hired had worked with some of their colleagues at another company. They saw him in action and were comfortable with him. So I guess it is not surprising that getting hired onto a top team is more a function of who you know than what you know. Why should it be different from the rest of the world?

I am personally interested to see where all of this leads because it is not sustainable. Companies are burning through capital and time. At some point they are going to have to build the products, produce the insights, move the needle on the business… or the entire promise of a data-driven future won’t be met.

From my experience recruiting data science talent and working with dozens of companies I have seen three general reasons for unfilled roles:

Ignorance

a. People don’t understand the marketplace well enough to create roles that require combinations of skills and experience that exist… OR they don’t understand the marketplace well enough to create roles that a talented and qualified person would actually want.
b. People don’t understand what they need. For example, they think they need a superstar data modeler but after months of interviewing they come to understand that they really need someone with basic modeling skills who can understand business issues and communicate effectively with stakeholders.

Politics

a. If I hire lots of other smart data scientists into the company who can do what I do my success and job security becomes less certain.
b. If I hire lots of data scientists I am going to have to produce lots of results. If I never hire I can just be a victim of the lousy recruiting process or the impossibly tight talent market.

Disconnect

My recruiters don’t understand these roles. I am looking at resumes that are too technical, too quant, too academic, too (fill in the blank) and talking to irrelevant candidates.

There is no doubt that the cost of hiring the wrong people is high. Being thorough and disciplined in your recruiting process is important. This has to be weighed against the high cost of empty seats. You should not let the perfect be the enemy of the good.