A Little Problem With Big Data

Courtesy RetailWire.comA STIMULATING DISCUSSION in RetailWire.com this morning led me to once again think deeply about how retailers are confronting so-called Big Data and applying it to their businesses.

The question posed was an intriguing one, given the continuing hype and mysticism ascribed to Big Data over the past several years.

What is your take on the advancements (or not) retailers are making in the use of data capture and analysis? Is it all leading to significantly improved customer experiences down the road, or something less?

The responses mostly seemed to accept two tacit assumptions: One, that all store data is Big Data. Two, that the primary goal of Big Data analytics is the creation of targeted promotional offers. I have a little problem with that.

When did retail POS data suddenly become Big Data? We’ve been collecting it (and mostly discarding it) for decades. Now that storage costs have finally declined, we can capture and hold it long enough to run a few queries and design a few models. Shopper in-store data really hasn’t changed much, but our ability to mine its potential has certainly advanced.

Certainly data flows from the POS and frequent shopper programs continue to expand. There are even some new sources, like in-store shopper tracking, entering the mix. Yes there’s lots of data. But is this really Big Data?

Read more