Analytics on Steroids

In light of the publication of Senate Majority Leader George Mitchell's report on illegal performance-enhancing substances in Major League Baseball, it's a little embarrassing that Billy Beane's success with the Oakland A's figures so prominently in the early pages "Competing on Analytics: The New Science of Winning," by Tom Davenport and Jeanne Harris.*I say "a little" because the fact that the A's were apparently competing not only on analytics but, at least to some extent, on steroids, does not negate the book's argument. It does, however, invite meditation on the limits of the analytic approach to competition and may also clarify questions of "art versus science" in the use of analytics for automated decision-making.

Beane's success, popularized in Michael Lewis' "Moneyball," clearly owes a great deal to his analytical approach, despite the use of steroids by some of his players. In any case, his success is only one of many examples of analytical competition adduced by Davenport and Harris.

Ultimately, the application of analytics to measure various kinds of performance is little more than an extension of accounting. One wouldn't rely on intuition to track business transactions, and all the advocates of analytical competition are arguing is that one take a similarly empirical and mathematical approach to measuring what works or doesn't. The emergence of this approach is thus not the result of brilliant insight so much as it is simply a natural consequence of the recent availability of large amounts of data for analysis. Given insurers' increasing ability to access enormous stores of potentially valuable data hitherto locked away in their systems, the concept of analytical competition-often through the vehicle of Davenport and Harris' book-is justifiably popular among insurance technology executives.

However, the Moneyball example shows that it's possible that variables other than the ones identified can distort the outcomes of analysis. And as with the use of any metrics, one has to choose the right ones: irrelevant hypotheses generate worthless conclusions. In that respect, intuition may supply what a purely analytical approach cannot. Also, one must never lose sight of the effects of chance.

"In an area where you haven't found an independent variable closely linked to a dependent variable, there's room for chance and other factors," is how David West, research director, TowerGroup, puts it.

While individual anomalies are very unlikely to trump statistical tendencies in the long run, West's point underscores the need to understand precisely what one is and is not measuring, and what the full range of possibility-not just probability-is when applying analysis.

The general principle is that "'on any given Sunday' any team can beat any other team," West says. In application of that principle, he adds that, "this Sunday, Miami is will ultimately remain the only undefeated team in history when they spoil the Patriots' season."

*Patriots coach Bill Belichick is also mentioned in the book (p78), but his cheating involved the effective use of analytics rather than their distortion."In an area where you haven't found an independent variable closely linked to a dependent variable, there's room for chance and other factors," is how David West, research director, TowerGroup, puts it. The general principle is that "'on any given Sunday' any team can beat any other team," West says. In application of that principle, he adds that, "this Sunday, Miami is will ultimately remain the only undefeated team in history when they spoil the Patriots' season>."

Anthony O'Donnell has covered technology in the insurance industry since 2000, when he joined the editorial staff of Insurance & Technology. As an editor and reporter for I&T and the InformationWeek Financial Services of TechWeb he has written on all areas of information ... View Full Bio