The first step in getting a handle on storage is to understand each point at which it is managed. This visibility will begin to provide insight into why so much raw storage ends up as so little usable storage. We'll use a common storage hierarchy (Figure 1) to explain how storage is allocated.
Figure 1: Storage allocation hierarchy
We begin to trace storage allocation at its entry point into the infrastructure. This first layer, total disk storage, is the raw physical storage sitting in your data center. Regardless if it is internal computer storage or external storage such as direct-attached storage (DAS), network-attached storage (NAS) or a storage area network (SAN), the result is the same -- you've purchased more than you need. Some reasons why include:
- Vendor discounts provide incentives to buy in volume.
- Lengthy procurement processes incent administrators to over-purchase storage.
- Industry is trending toward bigger disks, forcing customers to
- reluctantly buy fewer higher-density disks.
- Disk performance degrades well before 100% utilization.
Next, a portion of the total disk storage is allocated to support the customer request. This is known as configured storage. Configured storage is divided into three functional areas:
- Primary storage is the space made available to the customer.
- Redundant storage is the space needed to protect primary storage from physical corruption and media loss. Redundant storage, referred to as RAID (redundant array of independent disks), has several levels and can equal or double the size of primary storage.
- Recovery storage contains all the files necessary to recover your database. Its size is largely dependent on your recovery strategy. A recovery strategy that calls for a full disk backup of the database along with archive log files will more than equal the primary storage size.
Following our hierarchy, primary storage is presented to the operating system as LUNs (disk groups). When these allocations, called raw devices, are allocated in fixed sizes, the customer's space request must be rounded up to the closest LUN size. This practice can be the biggest contributor to unused space.
Logical volumes that ease administration and virtualize raw devices carry overhead. You can lose as much as 10% of the LUN size for this convenience. File systems allocated from logical volumes are presented to the database as physical storage. These again carry 5% to 10% overhead. Additionally, they achieve best performance when they are less than 90% utilized. Alternatives to logical volumes and file systems include direct management of raw devices and storage software that combines these layers into a single management layer.
The remaining storage allocation layers are managed within the database software. File system space is consumed by the database as database files. Database files can be configured to automatically extend when under space pressure. Segments represent table and index objects in the database. These objects are allocated in extents. A segment extent is a logically contiguous number of database blocks. A database block either contains data, in which case it is considered a used block, or can be empty or unused. A used block is 1% to 100% filled with actual data. The amount of data in a used block is dependent on a segment's configuration and its update/delete activity.
The psychology of organizational behavior helps further explain why we have so much storage and why it is under-utilized and difficult to track. Companies usually share storage management responsibilities between several different administrators. Storage, system and database administrators often create a "black box" into their domain. Each administrator will reserve or hold excess storage at each layer in the hierarchy beyond what is requested and allocated because they don't trust the customer. Additionally, customers will ask for more storage than they need because they aren't very good at estimating or forecasting storage needs. In both cases, this is a learned or acquired behavior based on the negative experiences of an untrusting and imperfect system. This tendency is actually incented by leadership that penalizes contingency request and provisioning.
Controlling storage usage
Here are the top 10 ways to maximize storage usage:
- Establish a governance process for new storage requests.
- Institutionalize management policies, procedures, standards and best practices.
- Improve storage planning, estimation and forecasting.
- Reduce the procurement cycle time by moving to an "on demand" model.
- Change leadership practices to incent reasonable contingency behavior.
- Reduce the number of management layers with alternative software solutions.
- Reorganize responsibilities to support a self-service database storage administrator.
- Increase transparency into storage management domains.
- Offer a range of storage allocation units (small, medium and large).
- Educate your organization on storage usage.
Bonus: 10 additional ways to maximize storage usage:
- Learn from experience by reviewing past requests and provisions.
- Monitor and report space consumption and proactively request space.
- Implement an information lifecycle management (ILM) program.
- Renegotiate storage vendor contracts.
- Utilize data compression where appropriate.
- Revisit and verify your recovery strategy.
- Revisit your data protection scheme.
- Implement a storage reclamation program.
- Purchase smaller-sized disks.
- Configure database structures for maximum usage.
After reading this article you may be surprised by the amount of unused storage in your infrastructure. I hope you've gained some insight into why this is and how you can make improvements. Understanding how storage is managed in your company is critical to controlling storage budgets.
About the author
Jeff McCormick is a Senior Data Architect at a financial services company and Executive Director of the Connecticut Oracle User Group. Jeff has worked in IT for almost 20 years as a system, storage and database architect/administrator and has over 15 years of experience with DB2, Sybase, SQL Server and Oracle relational database technology. He holds several certifications including Oracle Database 10g Administrator Certified Professional, Microsoft Certified Product (SQL Server) and Certified Sybase Professional Database Administrator. Jeff has performed extensive work in the area of high availability and disaster recovery, speaking and authoring several papers on availability architecture.
This was first published in April 2007