In God we trust.
All others bring data.

J. Edwards Deming

Data, we hear, is “…the new oil“.  It is the next big thing, the new realm of business opportunity, the them thar hills where we can all dig for gold.

Big Data, it is claimed,  shows us flu outbreaks before they happen, tells how the stock market is going to move today and knows if you are pregnant before your father does.

So does this justify the reported $125bn (or $50.1bn) (or $17bn) which is forecast to be spent on Big Data in 2015?  Perhaps, but I doubt it.

I remember another time when data promised us a brave new world. In the olden days of the mid-nineties, the story, cited by sources such as the Financial Times,  was how analysis of shopping baskets for a mid-range US store on Friday nights showed a previously unknown correlation between buying beer and buying diapers.   This story and others like it made the idea of data mining credible as it promised a whole new world of customer insight and understanding.

The beer/diaper story, it turns out, wasn’t true, but it didn’t stop the stampede.

Companies spent fortunes on things called data warehouses, invested in stuff called knowledge management and we were all supposed to make lots of money by having greater insight into customer behaviour.

And did we?

Did we heck.

But who did make money?

Consultants, vendors and the media who were all beating the data mining drum.

The consultants, who told us that this was the next big thing and that we would go out of business without it.

The vendors, who sold us the kit and the software and the maintenance contracts and the patches and who told us that the issue we were complaining about “…was a known bug that would be fixed in the next release…”.

The journalists and the advertisers, who sold ad space and wrote articles about “Making the case for knowledge management” and held conferences with titles like “Data Warehousing: A strategic imperative” or some such.

So we went to the conferences and read the articles and called in the consultants and bought the kit and installed the software and did the management of change and ran the benefits realisation exercises and employed new people with strange job titles and tried, as the dust settled, to see the step change in performance promised to us.

While the drum-majors walked away to do the same thing again with new clients.

Result? Billions of dollars siphoned out from the pockets of companies who were doing real things for customers and into the coffers of these drum-beating companies.

At the time, I recall that the consultants, tech companies and media companies in this space all posting phenomenal revenues.

What I don’t recall is reading so much about the great profits their clients made from all this knowledge management and data mining.

As far as I can tell, the promise of data mining and knowledge management was a promise unkept.

And now we have Big Data.

Perhaps Big Deja Vu might be a better term, because it all sounds very familiar.

Look: I get it. Big Data is a real thing.

The scale, depth, density and timeliness of customer data available to us is magnitudes greater than ever before.   Mobile data, geolocation, behavioural antecedents, digital payments, social media and evolution of the web are streets ahead of the consumer purchasing information upon which the promise of data warehousing used to rely.

I just feel we need to think harder about how we want to take advantage of it.  The same people are beating the drum and more and more companies are falling into step.

But the drum beat isn’t our drum beat. It’s the beat of market hype, of technologists seeking a market, of consultants seeking The Next Big Thing.

It’s not the drum beat of the customer.

What customer insight are we missing? Why does it matter? What could we do better for customers if we had it?

And (this is the kicker) what difference will it make? Really?

If we are to avoid being vendor victims, we should begin – as always – with the customer. And if we can answer these questions properly, then perhaps Big Data may really be of value to customers and to the companies that sell to them – and not just to those who are selling tickets to the Big Data bandwagon.