Note: This post was featured in our “Top 5 posts of 2012″. The post ranked #1 on the list and was originally published June 5, 2012.
You know big data has gone big time when your mom sends you a story from the local newspaper and asks, “Is this the same big data thing you do?”
David Corrigan, director of product marketing for InfoSphere, which includes IBM’s big data portfolio, recently spoke with us to answer the question, “What is big data?”
Big data is getting a lot of hype from trade publications and IT blogs to more mainstream media. “There’s a lot of market hype in both business and IT circles…but it’s real,” Corrigan asserts.
Big data describes the growing volume, variety and velocity of information. As Corrigan explained, “Data’s coming at organizations more quickly than ever before, sometimes as an information stream. And in some cases, the insight has a really short shelf life.”
The trick is analyzing data in its native format – whether that’s PDF, video, text or something else – in real time.
Listen to the “What is big data?” podcast with David Corrigan for a great discussion – in just 13 minutes – of what is big data, its uses and potential. Here are a few highlights:
Big data isn’t just for big companies
Whether you’re a 5 person shop or part of the Fortune 500, you can have big data. Corrigan says that any company can find an opportunity to analyze new sources of data and use the insights lurking within. Companies of all sizes must “cut through the noise” created by so much data to find valuable insights.
To remain competitive, all organizations need to analyze both internal and external data, as quickly and cost effectively as possible. As the world becomes more instrumented, with RFID tags, sensors and other sources, companies are creating more and more data. When paired with external data – like that generated by social media sites – there’s incredible opportunity that is largely untapped and unanalyzed.
Big data improves analysis
Corrigan detailed how companies leverage big data to improve their existing processes. Financial services companies, for example, already do fraud analysis. But incorporating data from additional sources can help improve the process and detect potential problems more quickly. Similarly, big data technology helps law enforcement analyze streaming video in real time and detect portions for review, rather than waiting for human eyes to watch each frame.
Adding social media data to the mix helps companies understand how their customers feel about their products or services. In an increasingly competitive market, this is tremendously valuable.
Big data means new opportunities
While improving existing analysis, big data technology also helps companies do something entirely different, offering “entirely new market opportunities” from existing, previously untapped data.
Corrigan explained that this happens in a few different ways. Some companies can turn their information into an asset that lets them partner with other companies, or create an add-on product for existing customers.
Telco companies, for example, can now analyze location data gleaned from mobile devices. This data helps them improve their own processes and networks, but it also presents an opportunity to partner with retailers.
Big data technology
“The underlying point of all of these new technologies is the ability to process and analyze big data in a cost-effective manner,” Corrigan summarized. “There’s nothing more cost effective than understanding data before you move it, or as you encounter it as a stream.”
Indeed, since big data technology lets you analyze data as it flows toward your company, you can determine what’s relevant for deeper analysis. Hence, you can store only that more important data in your data warehouse, improving efficiency and saving storage costs.
Listen to the podcast for a nice overview of big data, Hadoop and more.
To effectively compete in today’s changing world, it is essential that companies leverage innovative technology to differentiate from competitors. Learn how you can do that and more in the Smarter Computing Analyst Paper from Hurwitz and Associates.
I firmly believe this is the future in the technological field. From a purely analytical point of view, big data can improve upon all the existing process and more. And what's better is that, it's not only restricted to big companies, but also accessible to smaller firms. Thanks for sharing this insightful post with us.
Crysta, excellent post yet I'm missing the social aspect of big data… I strongly believe that having more insights and exploiting the data we have available (and now we can with this new set of Big data technologies) is going to help us making a better world!
Thanks! I agree that we'll see some huge implications on the social side, as well. Projects like IBM's Smarter Cities program are already beginning to have an impact, and my own town is using data to do things like improve water supply management during our current drought.
Nice Post. Thanks.
Great to see IBM adopting the “Vs” of Big Data that Gartner first defined over 12 years ago. For future reference and the courtesy of proper attribution, here is a link to the piece I wrote describing them from 2001: http://goo.gl/wH3qG. Thanks. –Doug Laney, VP Research, Gartner, @doug_laney
I was looking at this for my research paper, and through reading this blog it literally blew my mind that big data can be applied to small companies too! In my estimation, this is indeed one of the biggest concepts (if not THE biggest concept) for the future.
However, on a side note, I believe that since it can be used for the betterment of humanity, it can also be used for its demise. There is not just a need of advancement in technology, but also a need of advancing care & love for one another. Take care, and keep up putting up such great posts!