Will Artificial Intelligence Replace the Data Scientist?

Five Ways Automation Speeds Up Big Data Deployments

Who needs a data scientist when you can have a robot analyze your data? No, seriously, that’s an actual question enterprises may be asking if this Computerworld article on artificial intelligence is right.

Technically, I guess artificial intelligence isn’t a robot until you add a body, but the question still stands: Can artificial intelligence solve the data deluge better than humans? AI experts certainly think so.

“The notion that a human analyst can look at all of this data unaided becomes more and more implausible,” Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence, told senior reporter Sharon Gaudin.  “You can’t have a person sitting there watching Twitter to protect your brand. … You need A.I. tools.”

The obvious use case is with security, where humans are already failing to keep up with the ever-changing threat. Algorithms can “learn” from the data and flag deviations.

That same ability to detect patterns and deviations also makes AI useful in data search, scheduling and other tasks currently handled half by technology and half by humans, the piece points out. AI is already being used to prioritize email, speech recognition, culling through Big Data, and planning and scheduling functions. Siri is also based on AI, which does not make me feel better about all this.

It can also be profitable, as David Selinger proved at Amazon. He shares his experience with an Amazon test pilot in this IT Business Edge guest column.

No one actually goes so far as to suggest that AI could replace data scientists, but that does seem to be the implication here — unless they’ll be the ones writing the algorithms. Then again, if machines learn to program other machines — as is sometimes predicted with AI — then data scientists could join the growing list of jobs we’ll lose to robots and automation.

Artificial intelligence has also been applied to integration. Two years ago, I interviewed Robert Fox, who was then the senior director of Software Engineering for Liaison Technologies. Fox explained that Liaison’s Map Intuition mapping tool used inferences based on Bayesian Artificial Intelligence techniques.

“This is a very different way to accomplish ‘auto-mapping,’” he shared via email. “Map Intuition… looks at the data and model together to infer mappings. We have found that this approach produces a more completed translation map without the work of defining terms.”

Map Intuition also “learned,” improving the mappings as you added more data sources or target sample files, he said.

Webinars and Events:

“How UnitedHealth Group Optimized Its Data Services,” Thursday, March 26, 1 – 2 p.m. ET.  UnitedHealth Group (UHG) used MongoDB and integration vendor Talend in building its OptumInsight platform. OptumInsight gives health care analysts a single view of Medicare and retirement claims data, which allows them to audit, research and analyze this data in a much simpler way than before. The event will feature representatives from Talend and MongoDB.

“Deeper Questions: How Interactive Visualization Empowers Analysts,” The Briefing Room, Tuesday, March 17, 4 p.m. ET. Analyst Phil Bowermaster explains how data visualization can reveal valuable business insights.  Ellie Fields will provide a demonstration of Tableau 9.0.

“Semantics in Financial & Business Systems,” Wednesday, April 1, is a one-day forum hosted during this year’s Enterprise Data World, which runs March 26 — April 3 in Washington, D.C. The conference will cover how semantic technologies are enabling risk management and regulatory initiatives in financial services such as the Financial Industry Business Ontology (FIBO) and Legal Entity Identifiers (LEI).

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.