Tuesday, May 10, 2011

Data-Fueled Decision-Making

I would like to coin a term, right here in front of God and everybody. Go ahead: search the Internet. Search the texts. Search Google Books. You will not find this term, except in links to this post. This is a PhilosFX exclusive, and a Dr. Dave original!

Data-Fueled Decision-Making

We begin with three fundamental concepts regarding processes:

  • Every human activity can be thought of as a process; 
  • Every process can be improved; and
  • Any process can be modeled with data. 

Next we present two caveats regarding modeling processes:

  • Obviously, some processes are easier to model than others, either because the process itself is simple, or because the data are readily available; and 
  • The effort of modeling a problem (aka, decision opportunity), is best undertaken when the process under investigation is sufficiently complex to warrant the investment in building a useful model. 

And now, a point of clarification: D-FD-M is distinct from its infamous cousin, D-DD-M.

The phrase "data-driven decision-making" has been associated with No Child Left Behind (NCLB). Search D-DD-M and virtually all the hits will point to NCLB. In relation to NCLB, D-DD-M refers to using test scores as a performance indicator for students, teachers, and schools. D-DD-M in this narrow sense has come under fire as short-sighted or unfair for two main reasons:

  • test scores are not a comprehensive measure of education quality; and 
  • focusing on test scores encourages teaching to the test (and worse).

Here is the problem with D-DD-M: any time one places higher importance on Numbers than Values, one will distort reality and subvert human intellect and aspiration.

"Computers are incredibly fast, accurate and stupid. On the other hand, a well trained operator as compared with a computer is incredibly slow, inaccurate and brilliant."  --Not sure who said it, but pretty sure it was NOT Albert Einstein!  

 We need a term that speaks to the use of data as a common language, instead of a directive. Numbers do not dictate human actions. Numbers quantify. Numbers translate. Numbers scale for comparison. Numbers are the beginning of the conversation, not the end! Data-driven decision-making? No. We'd like to introduce a new term, "Data-fueled decision-making."

This post is about Data-Fueled Decision-Making (D-FD-M) and the models that a decision analyst might employ to aid decision makers. The scope is any human activity / process which is sufficiently complex as to make the investment in modeling it worthwhile, yet sufficiently simple as to make the model construction and data collection feasible.

Computer models are not necessary for trivial decision-making scenarios. On the other hand, a good model can be very helpful in complex decision making situations, especially with multiple stakeholders and a desire for transparency, which is to say, the ability for all stakeholders to audit the decision. 

In its broadest sense, and for the purposes of this post, D-FD-M is using measures of process performance to inform decision making tasks such as:
  • problem identification and definition, 
  • course of action development, 
  • analysis of alternatives, 
  • project selection, and 
  • portfolio management. 

Different types of decision analysis techniques are suited to different leadership and decision-making styles. Again, we are not referring to "fight or flight" decisions, or decisions where the choices are fairly obvious or trivial. We are referring to a class of decisions where the importance and the uncertainty are significant enough to warrant the resources required to to build and maintain a model.
  • Some decision-making is based on the informed opinion of the senior leader.  Naturalistic decision-making  relies on senior-leader experience for split-second decisions. Autocratic and bureaucratic systems, such as dictatorships and top-heavy institutions use this. 
  • Some decisions are the result of an empowered committee or a direct vote. Democratic and Republic (representative democracy) systems are like this. 
  • Some decisions are a sort of hybrid. The Military Decision-Making Process is an example of a participatory process where the senior leader retains override authority.
Data-Fueled Decision-Making is often a combination of the senior leader's strategy and scoring system plus the data supplied by a distributed workforce.

In D-FD-M, data may be used to either:

  • indicate current process performance, usually with a score; or
  • test the efficacy of proposed process improvement projects.
More later, but remember folks, YOU HEARD IT HERE FIRST!