Note: This post was featured in our “Top 5 posts of 2012″. The post ranked #1 on the list and was originally published June 5, 2012.
You know big data has gone big time when your mom sends you a story from the local newspaper and asks, “Is this the same big data thing you do?”
David Corrigan, director of product marketing for InfoSphere, which includes IBM’s big data portfolio, recently spoke with us to answer the question, “What is big data?”
Big data is getting a lot of hype from trade publications and IT blogs to more mainstream media. “There’s a lot of market hype in both business and IT circles…but it’s real,” Corrigan asserts.
Big data describes the growing volume, variety and velocity of information. As Corrigan explained, “Data’s coming at organizations more quickly than ever before, sometimes as an information stream. And in some cases, the insight has a really short shelf life.”
The trick is analyzing data in its native format – whether that’s PDF, video, text or something else – in real time.
Listen to the “What is big data?” podcast with David Corrigan for a great discussion – in just 13 minutes – of what is big data, its uses and potential. Here are a few highlights:
Big data isn’t just for big companies
Whether you’re a 5 person shop or part of the Fortune 500, you can have big data. Corrigan says that any company can find an opportunity to analyze new sources of data and use the insights lurking within. Companies of all sizes must “cut through the noise” created by so much data to find valuable insights.
To remain competitive, all organizations need to analyze both internal and external data, as quickly and cost effectively as possible. As the world becomes more instrumented, with RFID tags, sensors and other sources, companies are creating more and more data. When paired with external data – like that generated by social media sites – there’s incredible opportunity that is largely untapped and unanalyzed.
Big data improves analysis
Corrigan detailed how companies leverage big data to improve their existing processes. Financial services companies, for example, already do fraud analysis. But incorporating data from additional sources can help improve the process and detect potential problems more quickly. Similarly, big data technology helps law enforcement analyze streaming video in real time and detect portions for review, rather than waiting for human eyes to watch each frame.
Adding social media data to the mix helps companies understand how their customers feel about their products or services. In an increasingly competitive market, this is tremendously valuable.
Big data means new opportunities
While improving existing analysis, big data technology also helps companies do something entirely different, offering “entirely new market opportunities” from existing, previously untapped data.
Corrigan explained that this happens in a few different ways. Some companies can turn their information into an asset that lets them partner with other companies, or create an add-on product for existing customers.
Telco companies, for example, can now analyze location data gleaned from mobile devices. This data helps them improve their own processes and networks, but it also presents an opportunity to partner with retailers.
Big data technology
“The underlying point of all of these new technologies is the ability to process and analyze big data in a cost-effective manner,” Corrigan summarized. “There’s nothing more cost effective than understanding data before you move it, or as you encounter it as a stream.”
Indeed, since big data technology lets you analyze data as it flows toward your company, you can determine what’s relevant for deeper analysis. Hence, you can store only that more important data in your data warehouse, improving efficiency and saving storage costs.
Listen to the podcast for a nice overview of big data, Hadoop and more.
To effectively compete in today’s changing world, it is essential that companies leverage innovative technology to differentiate from competitors. Learn how you can do that and more in the Smarter Computing Analyst Paper from Hurwitz and Associates.