Problem solve Get help with specific problems with your technologies, process and projects.

Data warehousing project for large insurance company

I am consulting for a very big U.S.-based insurance company on their data warehousing project. Let me describe the scenario: They already have their old data storage application on a mainframe/DB2, and ETL applications are written on Cobol and schedulers with JCLs. We planned to use the ETL tool Ab Initio for ETL, Cognos for reporting, and CA Unicenter Scheduler for scheduling purposes. But these all are technical details... What else do I need to know about this project to make it succesful in one shot? What is the exact methodology used, or steps we need to follow, in the industry for this sort of DW project?

Well, my first inclination is to suggest you tap into some consultancies that can provide your organization the fundamentals for "best practice" data warehouse architectures and project methodologies. Ideally, you need to get assistance from reliable systems integrator companies that work closely with the tools you have selected. Look for consulting companies that provide certified training as well as consulting services that specialize in the business intelligence space and demonstrate proficiency with the tools. Also look for companies that can come in with "tried and true" implementation templates. These templates can cut to the chase because you won't have to go through the learning curve of how to set up your development shop and standards, etc. to ensure effective and efficient use of the tools. You can get right down to developing your solutions.

Regarding approaches for requirements gathering, I urge you to use what I call "facili-modeling". This is where a seasoned data architect uses a combination of group facilitation techniques and process and data modeling techniques to drive out the business intelligence requirements for your business. The models are then used as the starting point for your target data warehouse and the information needs requirements are the starting point for your information delivery design.

Regarding project management methods, make sure you chunk the work up into projects that can produce meaningful results back to the business in 90-120 days. So be careful to scope each project and prioritize the sequence of these projects (increments) such that you are delivering the most value early on. This will garner the additional business support (and funding) you need to gain momentum to do the subsequent increments. I would also recommend finding the best "dog on the bone" project manager you have and put that person in charge of the first increment. Keep your project teams manageable. I recommend no more than four to six people. Be sure your project team has people that possess skills in more than 1 area. For example: Make sure your Ab Initio person also has strong physical database skills. Make sure your Cognos person has a strong command of SQL. Make sure your data architect can put on an ETL hat, DBA hat and Cognos hat as needed to be effective across the project life cycle.

Regarding architecture, be sure you take the steps necessary to understand exactly how this new "beast" is going to happily co-exist with your operational world. You are bringing in tools that work real well but they also like to be well fed (computing resources). You'll also be introducing data movement patterns like you've not seen before, so be sure to take a good look at your network fabric between the source applications and your data warehouse applications. Also look into your network between the end-users and the servers supporting the data warehouse databases and the Cognos applications.

Good Luck.

Dig Deeper on Oracle data warehousing