Given their monstrous cost and time commitment, data warehouses are the last projects a company can afford to have fail. But that's just what often happens to these behemoths. Analysts agree that somewhere between 60 and 70 percent of all data warehousing projects fail.
These kinds of statistics are enough to put some companies off starting a data warehouse, but even with the risk, many companies conclude that it's worth it to try. No one can guarantee a project won't fail, but with a little careful planning, you can avoid at least some of the common roadblocks.
Believe it or not, technology has little to do with the success or failure of most data warehousing projects. Everyone can make unfortunate mistakes, but many projects are doomed before building even begins. It might start when executives hear about the wonders of data warehousing without knowing anything about what they require. They'll pull together $200,000 and a two-month deadline and expect to have a functioning product at the end of it.
"$200,000 won't even get you into the ballpark," says Kerri Apple, currently Vice President of Sales at Dallas-based B-Trade, and who has built data warehouses for the likes of Wal-Mart and Home Depot in the past. "Companies implement new projects all the time, and they don't even know what's up with the existing one."
Sandy Stoker, Vice President of Business Development at Dun & Bradstreet, agrees that most underestimate the cost and time involved, although admittedly fewer now than in previous years. She says, for example, that on average about a quarter of resources required for a data-warehousing project go into data migration alone.
"We found that 61 percent of managers said their projects went over budget," she said. "Many are multinational companies with lots of data, and they overlook the challenges required to analyze it."
Analysis is impossible if the warehouse spits out worthless data. Yet this is something that Mike Jette, director of consulting at Tigris Consulting, sees time and time again.
"Too often the technical side will work, but the data just stinks," he says. "People are so concerned about getting things working that they never consider whether the data being loaded into the warehouse makes sense. They'd rather ignore the problem than address it."
This is a fault of ownership, he explains. Departments are so involved in getting processes working on their end that they don't like to claim responsibility for what comes out of the warehouse. Stoker says successful projects address this by putting an executive champion in charge, or at least a project manager who has full support from above.
Otherwise, the project is likely to disintegrate into a morass of inter-departmental politics, another sure killer of an otherwise promising data warehouse. "People get unbelievably territorial about the data," Stoker says.
Stoker, Jette and Apple all agree that once a project is finally running, companies often shirk upkeep. Apple says too many companies accept the pilot project for the final version and never go any further. Stoker says Dun & Bradstreet has found information decays about two or three percent per month, and small business data decays even faster.
"Area codes alone are changing at a phenomenal rate," she says. Dun & Bradstreet copes with information shifts by updating its databases thousands of times per day. "You need to keep it fresh, otherwise you're back where you started."
