Battling That Pesky Weight Problem

In life-and-death decisions, people want facts. Health insurance with “medium” coverage doesn’t cut it at the emergency room, and the nation’s vague multicolor terror-alert system has generated plenty of nervous confusion.

Still, despite all of the money at stake, many businesses rely on simplistic weighted scores to measure and protect against risk. But a risk-assessment scale of “1 to 5” or “low to severe” is about as effective as a placebo, warns Doug Hubbard, chief executive of Hubbard Decision Research in Glen Ellyn, Ill.

Mathematical formulas are used to measure risks in everything from insurance coverage to stock portfolios, but CIOs rarely take this approach when trying to assess their information-technology risks, Hubbard says. “I.T. isn’t some special case that defies all risk analysis,” he says. “It’s just the last place to use it.”

Hubbard believes in training managers to measure risk the way actuaries do. Instead of rating a project’s chance of technical success on a scale of 1 to 5, for example, a manager would forecast a 75% chance a project will cost between $1 million and $1.3 million, or an 85% chance it will be completed within 10 to 12 months.

Scientific measurements are ideal in theory, agrees Edward Hill, a managing director of Protiviti Inc., a risk and audit consulting firm based in Menlo Park, Calif. But most managers aren’t well versed in statistical research, and the time it would take them to learn it might not be worth the investment for their companies, he says.C

Another obstacle: assembling the hard numbers needed for more-exact measurements. Take information-security risks, for example. Data are often unreliable or unavailable because most computer crimes aren’t caught, and when they are, companies try to keep the details quiet, says Ed Roche of Barraclough Ltd., a New York information-technology consulting firm. “You can’t make up the data,” he says. “If it’s not there, it’s not there.”

Hubbard thinks of it this way: Would you rather fly in a plane designed using mathematical formula or weighted score?