Let's Talk About Big Data

Please allow me to start by saying I am entering into this discussion as if you had no idea what big data is. If you are familiar already, skip this paragraph. I'm going to define big data by quoting SAS, one of my personal favorite analytics software: big data is "a popular term used to describe the exponential growth and availability of data, both structured and unstructured." Now pause. People in marketing are already pretty familiar with the idea that data is always being collected and analyzed to make decisions, but let's look at some of the more subtle implications of this definition.

Exponential Growth
Because this term is so often misused, you'll forgive me for putting it into context as well. Exponential growth does not just mean something is big. Exponential grow is almost unfathomable to people who don't work with numbers. A quick example, 10 times 10 is 100. It creates a much larger number than you started out with and it yields a 900% increase, very large by most metrics. On the other hand, 10 raised to the 10th exponent is 10,000,000,000. For anyone who doesn't feel like counting those zeros, that's 10 billion, a 99,999,999,900% increase. The increases yielded by exponents are almost laughable and it implies growth that is practically unstoppable.

Now, back to data. An exponential increase in the amount of data available for analysis is extremely significant. It is safe to assume that all internet traffic generates data. The amount of time you spend online, any website you visit, any browser you use, any searches you perform, and any links you click are all turned into numbers that describe you as a person, and for businesses, you as a consumer. This allows researchers to draw all sorts of conclusions about your skills, interests, and behaviors. Every action you take online defines you and the shear amount of actions you take creates an exponentially larger profile.

Structured and Unstructured
So what does this mean? It means a lot of the aforementioned data that is collected is not connected to anything and is not organized in a usable way. One of the key tenets of market research, for example, is to only collect data after determining your end goal. In other words, you collect for a purpose: to give specific answers to specific questions. Big data does not always do this. Often times, it simply collects and stores mass amounts of data with which no conclusions can be drawn because the information was not collected in a structured manner. However, there is still a sizable amount of data that is valuable to the collector even if there was no original purpose.

What's the point?
Ultimately, the goal is to create meaning out of a tangled web of information, and then apply it. Used correctly, it's a big asset for businesses. However, many businesses do not take the time to appropriately collect and interpret this mass of information. Resources are invested to obtain large stores of data, and then it sits. Fortunately for researchers, many more businesses have figured out ways of using this data to create better customer interactions. Predicting purchases, for example, so that when an individual finally initiates the transaction, the experience has already been optimized for ease and simplicity. As long as analysts are responsible with individuals' privacy and uphold their confidentiality, there is no reason this latest advancement in information analytics can't benefit everyone. A brief word of caution: if the necessary care is not taken to keep collected data separate from the individuals from which it comes, there could be serious ethical implications. Individuals' right to privacy must be respected above all.

---

Sources:
http://www.sas.com/en_us/insights/big-data/what-is-big-data.html