The Next Big Thing

What about all that small data?

As we turn the page on another year it seems to be an appropriate time for all of us who are involved in process improvement to reflect on the big picture of what we are trying to accomplish, and the smaller picture of how we can be more effective. The excerpt below helps put things in perspective:

"When the Lord created the world and people to live in it — an enterprise which, according to modern science, took a very long time — I could well imagine that He reasoned with Himself as follows: "If I make everything predictable, these human beings, whom I have endowed with pretty good brains, will undoubtedly learn to predict everything, and they will thereupon have no motive to do anything at all, because they will recognize that the future is totally determined and cannot be influenced by any human action. On the other hand, if I make everything unpredictable, they will gradually discover that there is no rational basis for any decision whatsoever and, as in the first case, they will thereupon have no motive to do anything at all. Neither scheme would make sense. I must therefore create a mixture of the two. Let some things be predictable and let others be unpredictable. They will then, amongst many other things, have the very important tasks of finding out which is which." —From E. F. Schumacher's Small is Beautiful

That pretty much captures the essence of Lean Six Sigma, doesn't it?

Enterprises everywhere take this sentiment to heart in a big and serious fashion, and leaders are enamored by the promise of big data and the analytic models it can yield. Big data certainly offers enormous potential, and on the surface, the message supports the notion that more data means more insight. But it's not quite that simple.

There's irony in the source of the quote above, a book called "Small is Beautiful," because the current infatuation with large big data may neglect a larger and more immediate source of value — the small data. It's alluring to build predictive models of customer behavior, but if we can't use the operational data to effectively manage the processes that fulfill that customer demand, are predictive models as useful as they could be? And if management structures and practices don't take advantage of the myriad meaningful signals available from the small data surrounding us, are the disciplines really in place to take advantage of big data?

Secondly, big data efforts can resemble the Analyze phase of a DMAIC project conducted in isolation — without the Define, Measure, Improve, and Control activities that turn statistical conclusions into new practical realities. You've probably run into people who have been enchanted by statistical tools but can't translate the number‐crunching into sustained positive change. Analytical insights from big data analysis are like potential energy: necessary but insufficient. The energy must be released in a controlled fashion and turned into useful action. Analyze without DMI & C is an exercise, not a process.

So in the coming year, it may be useful to re‐examine how we use the available small data to actually manage our core processes while linking our wider "analytics" exercises to a realization process that is proven to yield results: DMAIC.

My 2 cents.

—Bill Hathaway