What is Big Data?
When I was young, we would go to Uncle Haci’s clothing store at the start of each semester. The moment we walked through his store’s doors, he would show my mom the latest offerings in her favorite colors, comment on how I grew a few inches taller since the last time we saw him and lay a cotton pink shirt that my sister was sure to fall in love with. We would leave his store with overflowing bags, happy and satisfied. These days, I do a good deal of my clothes shopping online. Some websites greet me with a small “Hello Ozgur” icon at the top of the page, show me immediately “The Most Popular Items” and let me know that others who have clicked on this blue striped shirt I am browsing, have also looked at these skinny black jeans – all in an effort to personalize my highly sterile online shopping experience. At the end of the day, the goal is to make me feel like I have walked into Uncle Haci’s store with his personable smile and keen knowledge of what I might want. And what is behind the curtain of all these personalization efforts? Big Data.
Big Data is one of the forefront runners for the “Buzzword of the Decade”; you hear it everywhere. Data itself is nothing new for humanity. We have been recording our lives more than 17 millennia ago, started by animal paintings in the monumental Lascaux cave, southern France. There are records to suggest that six thousand years ago, Babylonians undertook the first census to count the number of people and livestock, as well as quantities of butter, honey, milk, wool and vegetables.
What makes the data that we have been collecting for ages “Big” all of sudden?
There are three important qualities to it: Volume, Velocity and Variety. It is not surprising that we can now collect tons of data from the Internet considering that over three billion people are surfing it every day. When we examine global figures, we see that in one minute, more than 200,000 photos are uploaded on Facebook, almost 100 hours worth video is shared on YouTube and around 350,000 tweets are broadcast on Twitter. In the very same minute, the number of searches performed on Google is close to 4 million. At this head-spinning rate, it’s been calculated that the data we produced in the last couple of years account for more than 90% of all data created since recorded human history. Furthermore, data is no longer a quantitative simple concept; we can now treat every phone conversation, every video recording and every WhatsApp exchange as data and analyze it to produce actionable information.
Technological advances in the electronics and computing fields are a major force behind this Big Data avalanche. First of all, storing data has become incredibly inexpensive: a hard disk of 5MB capacity used to cost around $400,000 in 1950s; nowadays we can buy a 128 GB USB drive from the local tech shop for under $100. Add to this, faster processors that are capable of parallel computing, open source platforms such as Hadoop that makes distributed data architecture easier, it is no wonder we are living in the Golden Era of Data.
The benefits of the information that can be extracted from Big Data are not just limited to the online marketing domain either. In recent years, Center for Disease Control in the U.S. has teamed up with Google data scientists to predict flu epidemics before the cases start hitting the hospital emergency rooms, simply by tracking the prevalence of search keywords such as “Runny Nose”, “Itchy Throat” and “High Fever”. By collecting data from the chips it has installed in its Fusion brand vehicles, Ford Motor Company is exploring ways to optimize driving routes to reduce a vehicle’s impact on local air quality in specific locations, such as near hospitals, schools and in high density residential areas.
As a result of all of these developments, “Data Analytics” is becoming on of the most popular professional fields all over the world. According to a report by McKinsey, in the next 5 years the U.S. only will need 140,000-190,000 workers with advanced skills in data analytics and 1.5 million managers capable and knowledgeable enough to make decisions based on detailed data exploration.
At Verite Analytics, our goal is to become one of the pioneers in the Data Analytics field both in Turkey and globally, especially in the area of Smart Cities – which we call Analytic Cities, living spaces that thrives on the data it generates.