Does Everyone Lose When Twitter Takes Back Its Data Firehose?

Millennials Expect Brands to Use Their Unique Data for Hyper-Targeted Experiences

Friday, Twitter announced plans to terminate agreements with its remaining data resellers, DataSift and NTT Data, by mid-August. At that point, any company using Twitter for analytics will need to go directly through Twitter. The reaction was swift and… well, here are some samples:

• “Twitter just gave the bird to startups, and I am not talking about their logo,” wrote Danny Crichton of TechCrunch in “The Valley Of The Sith Lords.”
• “Sad news today that Twitter is cutting off access to its ‘firehose,’ the stream of Tweets and related information that have been utilized by third party vendors for years,” wrote technology consultant Ben Kepes in a Forbes column with an equally emotive title, “How To Kill Your Ecosystem. Twitter Pulls An Evil Move With Its Firehose.”
• “3 Ways Cutting Us Off Harms Twitter’s Business – DataSift CEO” read the headline on Diginomica’s coverage.

Yikes!

Much of this coverage seems to have originated with DataSift CEO Nick Halstead, who told TechCrunch he was “blindsided” by the news. Halstead wasn’t shy about criticizing Twitter for abruptly ending the partnership. He also wrote a much quoted blog post Friday and then updated it on Saturday.

As if that weren’t enough, DataSift investor Mark Suster took up the discussion on his Both Sides of the Table blog. His take was more focused on proving that DataSift isn’t revenue-dependent on Twitter.

“DataSift passes 95% of the data revenue back to Twitter. Ninety five,” Suster writes.  “This is not about us making money off of Twitter data. We make money by helping customers reduce complexity in data streams and they pay us SaaS licenses and volumetric processing fees to do so.”

In other words, the problem is not in DataSift, but in Twitter, he assures us.

“We never built our product on one partner’s data platform but on many (20 primary data sources plus 100+ smaller ones) and Twitter’s move doesn’t ‘shut down DataSift,’ it simply takes 1,000+ customers who consume Twitter data through DataSift and makes their life more difficult,” Suster writes.

Along the same line, Diginomica notes that 80 percent of DataSift’s revenue comes from what the company does with the data rather than simply reselling it. That aligns with a common industry truism: What matters isn’t the data, but the business insights you derive from it. On the other hand, it’s hard to imagine so much discussion over something that matters so little to revenue.

What lesson, then, are we to take from this?

It depends on how things play out. It’s possible, for instance, that this will actually have a chilling effect on companies that plan to leverage Twitter, and possibly other proprietary data sources.

“Twitter has a long history of making moves that are counter to the best interests of its ecosystem,” Kepes writes on Forbes. “Interestingly Twitter recently invested in Dataminr, another analytics vendor who tracks news and financial data from the Twitter stream – perhaps other third party vendors in this space should start worrying?”

It’s also possible that the social media company has cut off its nose to spite its face. Steven Willmott, CEO of API management vendor 3Scale, did the best job of summarizing why this might be a poor play for everyone.

“While it makes sense at some level to want to be closer to the consumers of data (that’s valuable and laudable from a product perspective), removing other channels is an innovation bust,” Willmott writes. “Twitter will no doubt do a great job on a range of use-cases but it’s severely damaging not to have a means to enable full firehose access for others.”

He then presents three arguments to support his premise for expansion:

1. Twitter won’t be able to cover all the use cases that would otherwise be covered by third parties.
2. Few organizations can handle the full Twitter firehose.
3. The most interesting data use cases involve combining public data rather than relying on one source like Twitter. “…this is very hard to do well and just got a lot harder,” he adds.

Whatever the impact, it’s certainly upset many in the data industry. What’s unclear is whether this move will have far-reaching implications for data as a product.

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.