The ocean of data to be analyzed is getting deeper and wider.  The New York Times recently published an article on the data deluge in DNA sequencing.  They reported that the world's largest genomics research institute, BGI, generates so much data that it can't send it to clients over the Internet anymore, as that would take weeks. So they send computer disks containing the data, via FedEx!   The article highlights that data handling is now a key constraint for their industry.  Data quickly becomes dated or runs into an overwrite issue so making sense of data "quickly" is paramount.  It’s hard to believe but according to the NY Times article, "It costs more to analyze a genome than to sequence a genome."  It's not at all clear what you do with all that data.  Every industry is heading towards this problem.

So, what’s the solution?  In addition to large scale distributed storage system, there needs to be concerted effort towards disciplined approach to analytics. The typical used "Christopher Columbus" mentality of sailing into data and hoping to find insights isn't going to work anymore. Suppose you are given a task of finding sunken treasures from the Pacific Ocean.  The Pacific is deep and huge--how would you approach finding the lost treasure/gold.  Would you rent a submarine and start cruising the ocean like an explorer might?  Or would you look for clues to where the gold could be and use that as guide to direct your attention and effort like a detective would?

If you go the explorer route, you are guaranteed the views: nice corals, beautiful wildlife, emerald-green waters, but the chances of you finding any sunken treasure is slim. Essentially you are not directing your effort of finding treasure, just “something”. Your actions of exploration are independent of what you are looking for, you could have been tasked with looking for killer whales and you would still take the same action.

If on the other hand, you choose to be a detective, looking at historical ship routes and ship wrecks, using depth as a way to eliminate, you can start identifying potential areas to explore. Once you have identified a dozen plus potential location, you can prioritize the top 3 areas and then use submarine or deep sea diver to go explore. You are more likely find gold and in much shorter time. You will either succeed fast or you will fail faster and then re-strategize to attack the problem again.

Good analysts are detectives using business/corporate goals and priorities as guides to direct their effort to go looking for answers which are relevant to business and thereby finding gold nuggets and driving positive financial impact for the organization.