Big Data is Not a Technical Issue, it’s About People
For 95 percent of all problems, the technology and tools are available to handle everything you need to do. It really comes down to people, politics, and policies. The technology is there to implement, but you need buy in, you need the talent, you need the budget. Big data culture today is really in the Stone Age. The technology has gotten way ahead of what we can do culturally. We’re good at cranking out MBA spreadsheet jockeys who don’t really understand the raw material they use. They understand how to manipulate data, but they don’t understand it fundamentally. When there’s a problem with a Big Data implementation, it’s usually a not a technology problem but a people problem. People don’t want to have the way they work scrutinized. In failed Big Data projects, the people affected often were misread or ignored. The key to outcomes that matter is tying Big Data into people’s real agendas. If you leave the person out, you will not solve the right problem. The best analytics are ones that can give insight into who to talk to.
Beyond the practical considerations, technology professionals have a responsibility to be cognizant of the possible effects of the data we collect and analyze. The big ethical issue is that nobody thinks this is an ethical issue. The consequences are very real. We will see some really sad, heart-wrenching uses of data that will destroy an individual, and possibly groups of people.
Collecting Can Create Big Liabilities
As our technology outstrips our laws, key questions have yet to be addressed: Who has the real ownership of a given data set, technologically, legally, and societally? Who takes responsibility when agreements are violated? We simply don’t have the body of law to deal with those questions, and they are not moot. Like many other Big Data collections, the U.S. Census is protected by law from being used for any other purpose. But, laws can be fungible. Those laws didn’t stop the government from using Census data to identify Japanese Americans before sending them internment camps during World War II, for example. On a practical basis, improving your data collection and analytics may mean being held to a higher standard of quality. Just because you collect the data, does that mean you are responsible for analyzing it? This is uncharted territory. What are you required to keep and what are you required not to keep?
The Chicken or the Egg? (Data or the Question?)
One view of Big Data holds that organizations should gather as much information as possible. When you start to dig into the data, you see things. You can find things even when you don’t know you’re looking for them. That does not mean that your sole intent should be collecting masses of data in hopes that it will solve a business problem. Big Data by itself is kind of useless. You need big analytics to make a big impact. Besides, you don’t really even need Big Data for big analytics. Lots of important findings come from relatively small data sets.
Big Data is “a genie without a bottle,” because information once made digital can never be called back. No matter what legal or other protections it may appear to have, at some point someone you may not trust could have access to that data, because data’s lifecycle extends beyond our control.