Data Scientists Want Autonomy, ‘to Build Things’

LinkedIn gave data scientist Jonathan Goldman enough free rein that he developed algorithms to find connections between people, eventually creating “People You May Know” ads that produced a click-through rate 30 percent higher than the rate obtained by other prompts to visit more pages on the site.

That’s an example of how data scientists can provide business value to their companies, according to an article at Harvard Business Review.

“If your organization stores multiple petabytes of data, if the information most critical to your business resides in forms other than rows and columns of numbers, or if answering your biggest question would involve a ‘mashup’ of several analytical efforts, you’ve got a big data opportunity,” write the authors, Thomas H. Davenport, a visiting professor at Harvard Business School and D.J. Patil, data scientist in residence at venture capitalists Greylock Partners.

The labor pool for data scientists is very tight, they say, so much so that Greylock Partners, which backed companies such as Facebook, LinkedIn, Palo Alto Networks and Workday, has set up its own specialized recruiting team for companies in its portfolio.

The lengthy article describes the skills required and offers 10 tips for finding the data scientists you need. The most basic skill, they say, is the ability to code; don’t consider anyone who can’t at least code at a rudimentary level. Beyond that, look for intense curiosity, a trait I’ve written about before — and associative thinking enabling them to look at one problem and see another. They mention a data scientist studying a fraud problem who realized it was similar to a type of DNA sequencing problem. They write:

“More than anything, what data scientists do is make discoveries while swimming in data. It’s their preferred method of navigating the world around them. At ease in the digital realm, they are able to bring structure to large quantities of formless data and make analysis possible. They identify rich data sources, join them with other, potentially incomplete data sources, and clean the resulting set. In a competitive landscape where challenges keep changing and data never stop flowing, data scientists help decision makers shift from ad hoc analysis to an ongoing conversation with data.”

Data scientists often wind up creating their own tools – that’s how Hadoop grew out of Yahoo. Among their recruiting suggestions:

• Look at user groups – Groups focused on the languages R and Python are good places to start.
• Make sure a candidate can find a story in the data set and communicate it. Test whether he or she can communicate with numbers, visually and verbally.
• Be wary of candidates who are too detached from the business world. Make sure the candidate can apply data to the company’s challenges.

Expect candidates to make their employment decisions based on how interesting the data challenges are. They have a lot of options; their pay will reflect that. And while data scientists need to be closely tied to upper management in charge of products and services, they also need to associate with their own kind to keep their skills keen. They need autonomy to experiment and explore possibilities. The authors write:

“The data scientists we’ve spoken with say they want to build things, not just give advice to a decision maker.”