Using Data Analysis to Predict the Future
post by Chris Curran on April 6, 2011Guest post by Zach Sachen
Your ability to predict the future is relative to the knowledge you have and what you do with it when compared to your competitors. If competitors do not have access to the same knowledge at the same time as you, to them you are essentially predicting the future. For example, if you know before your competitors that a consumer trend is emerging, and you act on it, you will benefit more from your prescience. The question is how do you know before everyone else does?
Required Prediction Ingredients
Nostradamus relied on astrology. It made him famous but his forecasting accuracy has been widely debunked. CIOs have much more reliable tools at their disposal: data, technology and people who know how to use them to generate actionable information and reliable forecasts.
Companies make investments to capture, manage, and analyze data in hopes of turning their data into information with which they can gain knowledge and a competitive advantage. By having a large breadth of data and the ability to manage and analyze it, a company can come close to predicting the future. After all, if a firm is the first to identify a trend and has statistically accurate data to support whether or not that trend will continue then it has an advantage over other companies that don’t have the same information.
Being a Prediction Leader
In 2005 many companies began to think about how, and if, they should capitalize on underutilized “dark fiber.” Making the right decision about whether to buy up the fiber and at what price requires a predictive capability. This decision in particular involves predicting future internet usage. Having access to past internet usage information - e.g. what demographics, regions, volume, etc. - is a start and would greatly reduce risk when making the dark fiber decision. Taking that same information and developing statistically relevant predictive scenarios reduces that risk even more. This historic internet usage information is only available to a few companies, and they are the ones that essentially have the potential to predict the future use of the fiber.
Let’s consider the characteristics of a company that could make the best decision in this case.
The first step in making a solid prediction and profitable decision is making the company has a large relevant internal (and, ideally, external) data set. The internal data set provides the differentiator - the competitive advantage. A company will also need some of the best technology to be able to manage and analyze the data. This too can be a competitive advantage if the technology is internal or “home grown.” The last thing it needs is information-centric employees who can marry the first two ingredients to help drive the decision. Making investments in these ingredients would make a company extremely well positioned to accurately predict the value of the fiber, and also to use the prediction capability to inform other corporate decisions. The harvested knowledge could inform its strategic decisions well ahead of when those decisions will produce benefits.
In this example, a large portion of the dark fiber was purchased by a large internet services firm that had characteristics very similar to those of described. The decision was probably made by very good strategists, and perhaps it was also informed by publicly available data, but I suspect it was also informed by coupling data that was available internally with other information. The firm used this information to understand internet usage and volume trends across the globe and more importantly what it would look like in the future.
What Is Next
What we are going to experience in terms of predictive power could be bigger than the internet, bigger than mobile, and bigger than social media. Every day more and more of what used to escape into only the memories of those that witnessed it is now being recorded and digitized—digitally enabled conversations, movements tracked by a global positioning system (GPS), and online communications. More and more of history is digitally recorded and made available for analysis, for baselines, for comparisons. Many companies and governments are betting on “big data” because they know it is the critical catalyst that will spark the power of prediction. These companies are gearing up to collect and transform huge amounts of data into information, and take action on what they see happening.
The CIO will play a critical role in giving a company the power of prediction. This is a powerful ability - let’s hope it impacts the world for the better. In the meantime, think about how well positioned your company is when it comes to prediction.
Photo shared by bitterjug
Pingback: Links for August 21 2011 | Eric D. Brown()