The Under-Appreciated Value of Big Data Diversity

Loraine Lawson
Slide Show

Big Data: Not Just for Big Business Anymore

The tricky part about data is learning to accept what it says without imposing your own agenda.

It seems Big Data is no exception — at least, when it focuses on traditional, structured data, according to a Harvard Business Review Blog post written by Prof. Theos Evgeniou, Assoc. Prof. Vibha Gaba and consultant/visiting professor Joerg Niessing of international business school, INSEAD.

“A large body of research shows that decision-makers selectively use data for self-enhancement or to confirm their beliefs or simply to pursue personal goals not necessarily congruent with organizational ones,” they write. “Not surprisingly, any interpretation of the data becomes as much an evaluation of oneself as much as of the data.”

You may be able to avoid this pitfall by worrying less about the volume of your data, and focusing on growing the variety of your data, they argue.

“Big is old – retailers and financial institutions have had big data for decades. But Diversity is new,” they say.

The business reason for adding diversity is simple: It illuminates links in behavior that traditional data simply can’t reveal.

Marketing is already proving this by linking data from in-store loyalty programs to data from public websites, such as car or movie sites — basically, anything with a cookie. Toss in social media data and some in-depth research, and it can reveal more about data than volumes of traditional customer data.

For example:

“A leading Telco company we have worked with was able to increase market share by more than 20 percent in some countries without increasing the marketing budget by leveraging behavioural and transactional data from social and general media.”

Professors aren’t the only ones who have realized that focusing too much on volume can restrict what you’re able to achieve with Big Data.

Yves de Montcheuil, Talend’s vice president of marketing, identifies focusing on volume as one of the five major pitfalls of Big Data.

“When dealing with big data management, forget volume, de Montcheuil writes. “No matter the quantity, it is important to go after the ‘right’ data and identify all the sources that are relevant.”

Beyond the typical social media and SaaS data, de Montcheuil suggests you focus on so-called ‘dark data.’ He identifies two types:

  1. “Exhaust data,” which includes data from sensors and logs that’s usually purged rather than stored.
  2. Public data, which includes social media and open data.

As you think through Big Data, you’re probably going to begin with what you know: relational data. From the start, though, you can plan to move toward data diversity and Big Data Management maturity, suggests an October TDWI Best Practices Report, “Managing Big Data.”

“You have to start somewhere, so start with relational data, then move on to other structured data, such as log files that have a recurring record structure,” the report advises. “Carefully select a beachhead for unstructured data, such as text analytics applied to call center text in support of sentiment analysis.”

Next, add in semi-structured data that may be mission-critical, such as procurement and other B2B transaction data in the form of XML documents.

“Diversity, if managed well, yields divergent thinking and the pooling of a broader base of knowledge results often in better strategic choices,” write Evgeniou, Gaba and Niessing. “The point we stress here is that diverse data confers similar benefits.”

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post
Oct 26, 2013 6:24 PM SusanO SusanO  says:
Big Data is definitely the buzz phrase of the year, and the points you make about diversity of data are very valid. It isn't enough to have a lot of information. If you can't use this information to build your business, attract and retain customers, improve customer satisfaction and revenue and competitive advantage, why are you spending precious money and resources to gather that data? Once you capture your business data, you must present it in a way that is meaningful to your audience. That often means using graphics to provide a swift, clear picture so that your audience can approve your recommendation or understand the point you are trying to make. In your article you mention that executives and managers often use data to advance a personal opinion or goal. Our team at ElegantJ BI has found that our clients are often unaware of the lack of clarity and objectively in their decision making process. When data is presented objectively and with appropriate metrics, it can help the business to avoid falling into the trap of 'favored projects', 'personal agendas' and misguided opinions. Gathering, filtering, analyzing and taming big data using a business intelligence tool and a go Reply
Oct 27, 2013 5:17 AM Will Worthington Will Worthington  says:
thanks for highlighting an important aspect of leveraging multiple data sources to round out information insights. Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.