Data loading

When moving data from a database to a warehouse, the data transformation phase is followed by the data loading process. This is a seemingly straightforward process, but there are some things to look

    Requires Free Membership to View

at before making the move according to Ralph Kimball's book The Data Warehouse Lifecycle Toolkit (Wiley Computer Publishing):

The capabilities you need during the data loading process are, in large part, a function of the target platform. Some of these capabilities are:

1. Support for multiple targets. The atomic data mart may be on one DBMS, and the business process data marts may be on another. Each target will probably have its own syntax and idiosyncrasies, and your load process should know about these differences and use or avoid them as appropriate.

2. Load optimization. Most DBMSs have a bulk loading capability that includes a range of features and can be scripted or invoked by your data staging tool through API. Every database product has a set of techniques and tricks that optimize its load performance. These include steps like avoiding logging during loads and taking advantage of bulk loader capabilities like creating indexes and aggregates during the load.

3. Entire load process support. The loading services also need to support requirements before and after the actual load, like dropping support requirements before and after the actual load, like dropping and re-creating indexes and physical partitioning of tables and indexes.

For More Information

This was first published in June 2001

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.