Transforming Big Data Challenges Into Opportunities
We create 2.5 quintillion (or 2.518) bytes of data every day, according to IBM, with 90 percent of the data in the world created in the last two years alone. What does that mean in context?
What does this mean in context? Every hour, Walmart handles 1 million transactions, feeding a database of 2.5 petabytes, which is almost 170 times the data in the Library of Congress. The entire collection of the junk delivered by the U.S. Postal Service in one year is equal to 5 petabytes, while Google processes that amount of data in just one hour. The total amount of information in existence is estimated at a little over something called a zettabyte, yet the U.S. Office of Weights and Measures has coined a term for the amount of data too big to yet imagine – a yottabyte.
While corporations across all industries globally are struggling with how to retain, aggregate and analyze this mounting volume of what the industry refers to as big data, it also provides a unique opportunity for innovative startups that recognize the business prospects big data presents. Interactivity is driving big data, with people and machines both consuming and creating it. Digital companies focused on becoming good at aggregating and analyzing the data created by the end users of their product, who then provide their customers with solid insights taken from that data are at a distinct competitive advantage over others in the marketplace. Big data is not just unlocking new information but new sources of economic and business value.
There are two distinct challenges this information overload of Big Data presents. The first is the old “needle in the haystack” problem of uncovering the information you need amongst the massive amounts of data you have collected. The second is making use of the information once you discover it by developing ways to find meaning in it. The key is to turn the data into understandable information that can be analyzed and made into knowledge that can serve as the wisdom that allows people to make better decisions.
Both challenges have solutions rooted in technology with aggregation, social discovery and search algorithms to aid in discovery and visualization, summarization and new classes of powerful analytics to help with consumption. It is clear that companies are turning big data challenges into business opportunities by helping to find and collect information to improve the quality in decisions, segment audiences and embed data into products to create new opportunities –but what is next? With all of the data out there it is clear that more innovation is needed to make sense of it all.
For the next wave of big data innovation, the opportunities are more focused around how to assess the quality and impact of the data. Consider privacy, for example. Companies need to control who needs or has a right to know the data and ensure the proper protections are in place. There is also the question of rights management and who actually owns the data - the source, the company, the customer? It all begs the question of data integrity as well. Once you collect and uncover the data, how can you ensure it is accurate? Lastly is the question of relevance and determining what data is really worth knowing and how to filter out the noise.
It is clear that Big Data has gone mainstream and there are countless ways for those who understand the challenges it brings to seize the opportunities. As Gordon Gekko said almost 25 years ago now, “The most valuable commodity I know of is information.” And while even he would not have been able to predict the vast amounts at our disposal today, the sentiment still reigns true. We have entered into the industrial revolution of data, and entrepreneurs ready to be at the forefront must act today or risk being left behind.
Larry A. Bettino is a General Partner at StarVest Partners. This story originally appeared on Information Management, a sister publication to Health Data Management.
While corporations across all industries globally are struggling with how to retain, aggregate and analyze this mounting volume of what the industry refers to as big data, it also provides a unique opportunity for innovative startups that recognize the business prospects big data presents. Interactivity is driving big data, with people and machines both consuming and creating it. Digital companies focused on becoming good at aggregating and analyzing the data created by the end users of their product, who then provide their customers with solid insights taken from that data are at a distinct competitive advantage over others in the marketplace. Big data is not just unlocking new information but new sources of economic and business value.
There are two distinct challenges this information overload of Big Data presents. The first is the old “needle in the haystack” problem of uncovering the information you need amongst the massive amounts of data you have collected. The second is making use of the information once you discover it by developing ways to find meaning in it. The key is to turn the data into understandable information that can be analyzed and made into knowledge that can serve as the wisdom that allows people to make better decisions.
Both challenges have solutions rooted in technology with aggregation, social discovery and search algorithms to aid in discovery and visualization, summarization and new classes of powerful analytics to help with consumption. It is clear that companies are turning big data challenges into business opportunities by helping to find and collect information to improve the quality in decisions, segment audiences and embed data into products to create new opportunities –but what is next? With all of the data out there it is clear that more innovation is needed to make sense of it all.
For the next wave of big data innovation, the opportunities are more focused around how to assess the quality and impact of the data. Consider privacy, for example. Companies need to control who needs or has a right to know the data and ensure the proper protections are in place. There is also the question of rights management and who actually owns the data - the source, the company, the customer? It all begs the question of data integrity as well. Once you collect and uncover the data, how can you ensure it is accurate? Lastly is the question of relevance and determining what data is really worth knowing and how to filter out the noise.
It is clear that Big Data has gone mainstream and there are countless ways for those who understand the challenges it brings to seize the opportunities. As Gordon Gekko said almost 25 years ago now, “The most valuable commodity I know of is information.” And while even he would not have been able to predict the vast amounts at our disposal today, the sentiment still reigns true. We have entered into the industrial revolution of data, and entrepreneurs ready to be at the forefront must act today or risk being left behind.
Larry A. Bettino is a General Partner at StarVest Partners. This story originally appeared on Information Management, a sister publication to Health Data Management.
More for you
Loading data for hdm_tax_topic #reducing-cost...