Original Post Date: Thursday, August 25, 2011 

Check out this Report on Big Data from McKinsey & Company published in June 2011. 

Back in the day, when personal computers were becoming widely accepted and Microsoft Windows© was the new cool thing, SneakerNet was a common means of sharing data.   Certainly the introduction and subsequent improvements of networking technology and the Internet have made data sharing a whole lot easier and quicker.  But the concept of Big Data creates a whole new level of opportunity and potential for collecting and using data in ways heretofore unthinkable.

So what is Big Data?   According to the report referenced above – “Big Data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage and analyze.  The authors are cautious not to declare a specific size since lightening fast technology advances may quickly make their number wrong thought they do propose currently that datasets in the few terabytes to several petabytes (1000’s of terabytes).  (hmm… I just had to add petabytes to my Word® 2007 dictionary).  The concept of Big Data requires the collection of data from multiple sources (sensors, smartphones, GPS’s, social media postings, data collected by government agencies, etc.) and performing analysis of that data in ways that will make life better and bring more value to businesses, consumers, citizens, etc.
Some examples of Big Data types of applications include;

* Marketing  initiatives like Amazon.com’s “you might also want …” admonition based on information available about your buying patterns and the buying patterns of those purchasing the same item

* Applications like RedLaser which lets a shopper scan a bar code of an in-store item with their smartphone and get immediate price and product comparisons

* Improved supply chain management through access to data across the supply chain helping manufacturers optimize planning and delivery of new products

* Mobile phone apps that allow merchants to track customers from the moment they enter the store to determine traffic patterns and flows through shelves and displays

* Capabilities like OnStar ® where sensors in the automobile send real time data to the service if the airbags deploy or to alert that a system is malfunctioning.

* Capabilities like ShopAlert that sends coupons or offers to smartphones of subscribers when they are in the vicinity of a store, restaurant or bar

The report actually contains a lot more examples across various platforms and sectors.  They have specifically Health Care, Public Sector, Retail, Manufacturing and the ever growing business of using personal location data.  Within each of these categories – opportunities for creating value as well as potential barriers to adoption are presented.

While the technology and the possibilities create lots of excitement – there are also some areas of concern – how much personal data is too much and how close do we want “Big Data” to look like “Big Brother”.   Certainly there are lots of issues that need to be addressed at all different levels before Big Data goes wildly mainstream – but it seems to me that the possibilities for value and capability will, in many cases, be worth it. 

How have you seen Big Data used?  What possibilities can you see for Big Data applications?  Leave a comment to this post to share your Big Data stories.