Big Data: Data Divination

Every era coins a new term that becomes a house hold name and the word of this era is – Big data.

Using collected data to predict events shouldn’t blind us to the humans behind it.

Anybody belonging to any substantial workplace or organization cannot afford to not know about it. It’s the brand new flavour and everybody is talking about it. But the question is, what is big data and how is it affecting us and our lives and what are the limitations of this Data divination?

Human nature always wants to know what is in store for them in the future! They want to be able to know about the outcome of their decisions. Even on the small scale, for instance, when you see a person everyday of your life; by watching his daily habit and level of work he puts in, you make the prediction about his future. Now you don’t know the future of course. It’s the pattern that you followed and studied. Same is the case with big data. You can use the big data to predict the outcome of any action, but that solely depends on the size of the data you are using. (Paul, Jan 2017)

No one knew that Brexit’s outcome would be opposite to what was anticipated, according to Paul McFedries. And according to him, this only happened because of the quantity of big data that was used to predict the aftermath of the decision. As McFedries puts it, the data was medium sized comprising of a few thousands individuals. In his view, you can successfully predict the outcome of such data divination if your big data encompasses million entities. That is the size needed to be able to make the correct assessment about what is going to happen in, let’s say, election. That’s what happened in US in their latest elections. No one ever divined about the notion of Trump’s being the next President of United States. As Paul puts it, the surprise came because the data sample being used was medium sized, comprising of just only thousands of individuals. Had it been a real big data, using data of millions of individuals, outcome wouldn’t have been a surprise.

This brings forward concept of big data divided into different categories depending on their quantitative and qualitative nature. Let’s look into them briefly:

Types of “Big Data”

BIG DATA: Encompass at least millions of entities

MEDIUM DATA: Comprises of thousands of entities.

FAST/HOT DATA: Data which is readily available to manipulate and is being used frequently.

SLOW/COLD DATA:  Data which is being collected for a substantial period of time and is not in frequent use.

So there are multiple kinds of data that are used for the survey purposes as well as for making prediction about the outcome of some major event that have major implications on the fate of large population.

To get better at forecasting big political events, we need both better data and sharper reporting, a clearer read on the numbers, and a more penetrating portrait of on-the-ground realities.

—Aaron Timms

References

Fedries, P. (2017, January 19). IEEE Spectrum. Retrieved February 20, 2017, from Data Divination: Spectrum.ieee.org