Big data : The next generation of innovation, competition, and productivity

What is big data?

Big data usually includes data sets with sizes beyond the ability of commonly-used software tools to capture, curate, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set. With this difficulty, a new platform of "big data" tools has arisen to handle sense making over large quantities of data, as in the Apache Hadoop Big Data Platform.

Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This data is big data.

Big data spans three dimensions: Volume, Velocity and Variety.

Volume:  

Enterprises are awash with ever-growing data of all types, easily amassing terabytes—even petabytes—of information.

• Turn 12 terabytes of Tweets created each day into improved product sentiment analysis
• Convert 350 billion annual meter readings to better predict power consumption

Velocity: 

 Sometimes 2 minutes is too late. For time-sensitive processes such as catching fraud, big data must be used as it streams into your enterprise in order to maximize its value.

• Scrutinize 5 million trade events created each day to identify potential fraud
• Analyze 500 million daily call detail records in real-time to predict customer churn faster

Variety: 

Big data is any type of data - structured and unstructured data such as text, sensor data, audio, video, click streams, log files and more. New insights are found when analyzing these data types together.

• Monitor 100’s of live video feeds from surveillance cameras to target points of interest
• Exploit the 80% data growth in images, video and documents to improve customer satisfaction

 

Big data is not only a matter of size; it is an opportunity to find insights in new and emerging types of data and content, to make your business more agile, and to answer questions that were previously considered beyond your reach. Until now, there was no practical way to harvest this opportunity.