Problem solve Get help with specific problems with your technologies, process and projects.

How do other DW teams deal with in-and-out flows?

In-and-out flows: In the 'traditional' DW architecture, your normal operations are not affected by DW issues. The business keeps running. This has impacts on DW staffing -- there is typically no need for 24/7 operational-type support. What happens when the information is so good that other groups want to utilize it as input to their operational systems? My specific examples are cleansed and transformed revenue source data being used for commissions and cleansed and transformed backlog and shipments being used for material planning. The DW can become lodged in the middle of the flow of operational data, which has staffing impacts, and again, will detract us from our mission of focusing on analysis and executive decision-making. How do other DW teams deal with in-and-out flows?

You hit at the root of the challenges of implementing an enterprise-wide Corporate Information Factory (CIF). There are several components and layers of the CIF, including the suite of operational support systems (OSS), the Operational Data Store (ODS), the Atomic History and the suite of Data Marts. The suite of OSS applications collectively are the Systems of Record for every data element in your enterprise. This suite of applications is used by the business to conduct business.

The Operational Data Store is the integrator of everything, owner of nothing. It is a non-temporal snapshot repository of operational detail. This data structure provides a means to support tactical decisions, provide operational reports, forms the foundation for feeding cleansed and integrated operational data into the atomic history and data marts and forms the "hub" of a hub and spoke/publish and subscribe data integration architecture.

The Atomic History is the enterprise data-warehouse where historical details are maintained to support data mining across the enterprise and provide the historical details for drilling from the data marts.

The Data Marts provide departmental views of trended and aggregated facts organized by the analysis dimensions most regularly used by those departments. Marts are also used as a means to provide fast response to typically requested trending information. (A classic example is the data marts used to support executive dash boards and balanced score cards).

Let's talk about the ODS. Of the CIF components, the ODS is your optimal structure to consider as a hub for sharing cleansed and integrated data across your suite of OSS applications.

  • If the majority of the data interoperability needs (moving data updates across OSS applications) can be satisfied through scheduled-based updates, the ODS along with your ETL tool and a scheduling package will probably work well for you.
  • If you have a significant amount of data that needs to be updated across your OSS applications on an business-event/message basis, you'll need to look at bringing an Enterprise Application Integration (EAI) type tool into your architecture. The next generation EAI tools are showing great promise for enabling interoperability across applications. They are rules based and provide a means for maintaining those rules in your metadata repository. They provide a fast and relatively non-intrusive means for building peer-to-peer data interfaces.
  • If your mission is strictly to address interoperability, you can implement these EAI tools without the hub and spoke architecture (without an ODS).
  • However, if you also need a means for providing tactical and strategic decision support reporting on an enterprise-wide basis, I encourage you to keep the ODS in the architecture and build an extra "subscriber" to the ODS for each data element being shared across your applications. The ODS then provides a means for enterprise tactical reporting/viewing and provides the foundation of cleansed and integrated data to the atomic history and data mart layers of the Corporate Information Factory.

I wish you the best of success in your efforts.

Dig Deeper on Oracle data warehousing

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.