Valhalla has been looking over the Big Data market, trying to answer the question: “how far along is the market?” Are there really only four or so Big Data users — the likes of Google, Yahoo, Facebook, and Twitter — or are there more? Is it an Early Adopter (or even merely a Tech Enthusiast market), or has it crossed the chasm? What are the use cases?
Here are some of our findings:
1.The Big Data market is an Innovator/Early Adopter market overall, with possible Early Majority beachheads in web analytics and adtech
Although our interviewees described a larger number of use cases – “voice of the customer” analytics in marketing, M2M sensor processing, fraud and risk analysis, predictive analytics of various types – there was no hard evidence for widespread uses of Big Data today in these use cases, and many of the interviewees described them as “nascent” or “near-future” use cases.
There was, however, agreement that web analytics and adtech platforms were much further along in terms of using Big Data techniques for projects which were important to the customers’ businesses and mainstream today.
· AdTech users employ Big Data technologies for real-time bidding (RTB) and managing and matching 3rd-party data to ad inventory or online user data (this area seems to be called “data management platforms”, an area where DemDex (which was acquired by Adobe for $xxxM) is perhaps the poster child.
· Web analytics users employ Big Data technologies for indexing web pages and extracting performance indicators from raw weblogs.
2. Informants believe that Hadoop and its stack is likely to remain the central platform for the Big Data market, but there is contradictory evidence
I don’t personally agree with this finding, but our interviewees all said, implicitly and explicitly, that the Hadoop stack was going to be the basis for Big Data technologies going forward.
One very thoughtful analyst said explicitly that the MapReduce/Hadoop stack would evolve over time, and that new technologies – like Dremel or Storm or Spanner and so forth – would be incorporated into the Hadoop ecosystem rather than creating new ecosystems of their own.
The only problem with this point of view is that “legacy” Big Data techniques – data warehousing, RDBMS, classic Business Intelligence suites – have a vast market share and a long history of productive use cases. How these platforms will interoperate in the future is unknown, and whether an approach like Hadapt’s (where a “classic” RDBMS or BI technology suite runs within the Hadoop stack) will prevail is still too early to call.
3. Wikibon’s analysis sizes the Big Data market today at $5B
A quantitative Wikibon analysis, which is quite thoughtful, concludes that $480M of this revenue comes from what they call “pure play” vendors (i.e., Hadoop infrastructure vendors and some other NoSQL or NewSQL) and the balance from legacy players.
Very curious about your thoughts on this.