Behavioral economists and cognitive scientists seem to have made a career out of showing how our judgments tend to be wrong in glaring ways. In the case of big data—the pervasive term for the expanded and growing pool of data available to power business decisions—they may once again be onto something. When it comes to identifying the right data, we’re more often impulse buyers than paragons of analytical skill. The effectively infinite amount of data available to modern number crunchers is no guarantee of greater insight. Cynics might liken it to giving a child on Halloween a vastly bigger bag of candy in order to get to a better nutritional decision.
Those daunted by behavioral economists may find solace in another impressive character, the data scientist. The skills needed to convert raw data into usable information, insights, and predictions are distinct from the software engineering skills needed to wrangle large, messy data: We can’t simply throw processing power at a world of data and expect a meaningful picture to emerge. Far more crucial is the ability of an organization to identify the right data and the right analysis to perform. In Too Big to Ignore, the authors remind us that the oft-cited Moneyball account is not a tale of using terabyte or petabyte class data but rather “an inspired use of the right data to address the right business opportunity.”
Moreover, we tend to look to big data to solve existing problems and in doing so overlook opportunities to rewrite the playbook. In Big Data 2.0, the authors consider three perspectives—customer, product, and ecosystem strategy—to frame the selection, interpretation, and sharing of data in ways that go beyond the quest to perfect the as-is. How can we use data to shape customer behavior, conceive new products and services, or even redraw the playing field?
The third of our trio of articles on data managers behaving badly, Telling a Story with Data, takes us from the fledgling infographic startup of Florence Nightingale to a discussion of how modern analysts often miss the opportunity to communicate their findings in a way that might influence decision makers or lead to a more valued outcome. The compelling story that emerges through big data and analytics, author Tom Davenport suggests, is less about R-squared and more about the journey from problem to insight: The entire process can serve as a vehicle by which to engage stakeholders who may be bored mightily by an otherwise charming display of charts produced at the end of that journey.
We can’t, then, automate our way to omniscience, but the struggle to derive insights from data itself may be something to embrace and share.