Before an organization decides to build a data warehouse, it is important to understand the value that a data warehouse can provide. I have said for a while now that 90 percent of data warehouses are used for validation and the other 10 percent are used for exploration. (Actually, 7 years ago I used 95/5, so we are going in the right direction!) Validation is the corroborating of something that we already know or believe to be true. Think of the examples of what people are saying about data warehouses:
- Who are our most profitable customers? If you are not validating what you think you already know, then maybe you should not be in the business. Most companies define profitable as who spends the most with us. I doubt that this is true.
- How many widgets were sold this month vs. last month? If you don't have something in place that can answer that today, I am not sure how you have survived this long.
- Are purchases of my product made by impulse or planned buying patterns? In marketing we studied that in larger cities the buying of candy bars are impulse whereas in rural areas it was more planned. There is empirical evidence to suggest that this is correct, except in Denver where the pattern is more like a rural area (this was few years ago and most likely has changed!). This is a good validation, but probably not something that you would want to spend $1 million understanding.
I believe that the value of data warehousing is in exploration. Exploration is the searching of data that can then return knowledge. Knowledge can be in the form of "I found this" or "I did not find this". Examples of these are:
- What are the four types of purchases made within a certain time frame that is a 90% indicator that someone is about to declare bankruptcy? Is there a fifth?
- What are the grocery items that people buy on a regular basis and should we assess this behavior as a high risk?
- There is no relationship between pressure and temperature being outside the tolerance levels on the quality of the chemical.
So, what are you using (or going to be using) your data warehouse for? Is it valuable to your organization? Don't get me wrong. I still think that validation is useful, but the data warehouse can add much more value.
The Problem
Organizations have lots of data: sales data, production data, shipping data, accounting data, manufacturing data, etc. There are many places where this data can reside: on-line transaction processing (OLTP) systems, spreadsheets, documents, etc. It can be stored on disks, tape, CDs, etc. There is so much data but very little knowledge.
The building of the data warehouse is not hard, but is difficult. It is difficult to determine what you need due to the roar of the over-hyped markets. It is difficult because the information in the source systems is not quite good yet (that is why there are 64 different US States in some systems!). It is difficult because there are not good representations for metadata (and metadata gets pushed to the "nice to have" requirements). It is difficult because the data warehouse is sometime nebulous to explain to accountants in return on investment (ROI) terms. It is difficult because...? Well, you get the picture.
Let's all Share
One of the values that a data warehouse can provide is not only the sharing of data, but also the sharing of knowledge. The data warehouse can provide, if you are wiling to explore it, a major source of sustainable competitive advantage: knowledge. The faster you can gain and put that knowledge into use, the more valuable that knowledge is to your organization.
What are some of the reasons that organizations need a data warehouse? How about:
- improved customer service
- focused marketing efforts
- optimized distribution
- survival
These boil down to one thing: getting better at what your organization is doing today and preparing for tomorrow. This is done by shared knowledge to create value.
The Real Value
The more that you can share amongst the people in your organization, the more value your data warehouse adds. Some of my friends (Dale Mietla and John Sell of Synfusion (www.synfusion.com), to name a few) and I claim that:
The Value of Data Warehouse = Quality of Knowledge times the Organizational Reach
Think about it: If someone gains a great piece of knowledge but holds it to themselves, what is the value? Not much. But if you take that knowledge and share it with everyone, the value is a multiplier of the quality. That is Real Value.
Is your data warehousing providing value? Is it greater than the sum of the data? How much reach into your organization does the data warehouse and its results have?
What is the Value of your data warehouse?
About the Author
Chuck Kelley is president and founder of Excellence In Data, Inc. and an internationally known expert in database technology. He has more than 20 years of experience in designing and implementing operational/production systems and data warehouses. Kelley has worked in some facet of the design and implementation phase of more than 35 data warehouses and data marts. He also teaches seminars, co-authored a book with W. H. Inmon on data warehousing and has been published in many trade magazines on database technology, data warehousing and enterprise data strategies. Please feel free to email him at chuckkelley@usa.net with comments (negative or positive) about this column or ideas for future columns.
For More Information
