Way back in 2007, I wrote about the tension between sophisticated analytics and human judgment. No matter what data tells people -- and it can tell us a lot -- many of us are uncomfortable relying strictly on data when making decisions. Sometimes our guts simply tell us to go a different way.

So it's perhaps not that surprising that a recent Accenture survey, mentioned in this TDWI piece, found that 40 percent of companies base important business decisions largely on judgment rather than analytics. In many cases, of course, it's not so much that companies think human judgment trumps the data, it's because of data shortcomings. Sixty-one percent of respondents cited a lack of good data, for example, while 60 percent mentioned absence of historical data.

The quality of data can't be discounted, said Ian Ayres, author of "Super Crunchers: Why Thinking-by-Numbers is the New Way to be Smart," whom I interviewed in 2007. He said it's important to make data mining contestable, that is, not to "just rely on a unified prediction from a genius in the corner." He suggests having analytic audits, where company outsiders come in and independently crunch numbers to see if they produce different results.

Yet the Accenture survey also seemed to indicate that, in some cases, companies simply preferred to rely on their guts. The experts contributing their opinions in the TDWI piece said that data and human judgment aren't an either/or proposition. Said Neil Raden, author of "Smart (Enough) Systems":

... I don't think you can separate gut from analytics, because all analytics can do is inform your decision and at some point you have to apply your gut to the analytics.

Perhaps the realization that human experience and opinion can complement data helps explain the growing popularity of prediction markets, a decision-making tool I wrote about a few months ago. Bo Cowgill, a quantitative marketing manager at Google, told me such markets bring in more diverse opinions, which makes decisions more reliable. Google typically has two dozen internal prediction markets running at a given time. He said:

With more traditional methods, you rely on a single analyst, or maybe a team of them, crunching numbers. They are typically not going to be as exposed to as much information as the crowd.

Ayres said humans are less effective in making more complicated decisions. Their success rate tends to fall as the number of underlying factors involved in a decision grows. Yet that's exactly when prediction markets can be helpful, believe Cowgill and others, because they bring in opinions and knowledge sets that might not otherwise be considered.

The counter view: When making decisions, humans often assign too much weight to certain factors or include irrelevant factors. In our interview, Ayres offered the example of a loan officer who focuses on an applicant's race. Said Ayres:

Give me any individualized loan approach and let me run a race with it; the statistical approach is going to make better lending decisions. Discretionary systems tended to make loans to their friends other than to qualified candidates.