Oracle disk I/O tuning: The SCSI bus, part 1

The following is part of a series on the different aspects of disk I/O performance and optimization for Oracle databases.

The following is part of a series on the different aspects of disk I/O performance and optimization for Oracle databases. Each tip is excerpted from the not-yet-released Rampant TechPress book, "Oracle disk I/O tuning," by Mike Ault. Check back to the main series page for upcoming installments.

Mike Ault

Mike Ault is one of SearchOracle.com's Oracle Internals experts. Mike is senior Oracle consultant with Burleson Consulting, and one of the leading names in Oracle technology.

To view Mike's expert responses or to ask him a question, click here.


Peripheral buses

The two peripheral buses on Sun systems are Sbus and PCI bus. Sbus typically runs at 20-25MHz and provides 32 or 64 bit sizes. The peak Sbus bandwidth is 200 MB/s. The PCI bus runs at 33 or 66MHz and may be 32 or 64 bit. The peak PCI bus bandwidth is 528 MB/s. Almost all add-on SCSI adapters will plug into the PCI bus.

SCSI bus

SCSI buses on SUN can operate various bandwidths, speeds and throughputs (see table 3-2 to see the various bus specifications for SUN). The prtconf Solaris command is used to report information that can be used to determine the speed of a particular SCSI device. Most of the SCSI systems I have seen in recent years have been SCSI3 based.

Protocol Bus Width Bus Speed Throughput
SCSI1 8bit 5MHz 5MB/s
Fast SCSI, SCSI2 8bit 10MHz 10MB/s
Fast Wide SCSI, SCSI3 16bit 10MHz 20MB/s
Ultra SCSI 8bit 20MHz 20MB/s
Wide Ultra SCSI, Fast20 16bit 20MHz 40MB/s
Ultra2 SCSI 8bit 40MHz 40MB/s
Wide Ultra2 SCSI, Fast40 16bit 40MHz 80MB/s
Ultra3, Ultra160, Fast80 16Bit 80MHz 160MB/s
Ultra320, Fast160 16Bit 80MHz 320MB/s

Table 3-2: SUN SCSI Specifications

The scsi_options parameter can be set in the /etc/system file to limit bus speed or set other characteristics. You should check device documentation to determine if these settings need to be specified for a specific device. The default scsi_options variable allows the widest range of capabilities that the SCSI host adapter can provide to be supported.

The default scsi_options value on Solaris 2.x works for both 5MB and 10MB devices. The driver will negotiate with each device to determine if it is 10MB transfer capable or not. If they are 10MB devices, 10MB transfer will be used. If not, 5MB transfer will be used.

SCSI subsystem options - on SUN Solaris, a global word of options are available. The bits of the global word are broken down as:

Bit(s) Use
0-2 Reserved for debugging/informational level
3 Reserved for a global disconnect/reconnect switch
4 Reserved for a global linked command capability switch
5 Reserved for a global synchronous SCSI capability switch

The rest of the bits are reserved for future use.

The bits, which can be set are set using the hexadecimal values are shown in Table 3-3.

Option tag Option Value Description
SCSI_DEBUG_TGT 0x1 Debug statements in target drivers
SCSI_DEBUG_LIB 0x2 Debug statements in library
SCSI_DEBUG_HA 0x4 Debug statements in host adapters
SCSI_OPTIONS_DR 0x8 Global disconnect/reconnect
SCSI_OPTIONS_LINK 0x10 Global linked commands
SCSI_OPTIONS_SYNC 0x20 Global synchronous xfer capability
SCSI_OPTIONS_PARITY 0x40 Global parity support
SCSI_OPTIONS_TAG 0x80 Tagged command support
SCSI_OPTIONS_FAST 0x100 FAST scsi support
SCSI_OPTIONS_WIDE 0x200 WIDE scsi support

Figure 3-3: SCIS Options Hexadecimal Values

The values for the various options are added to achieve the desired combination of features. For example, a line in /etc/system that read:

        set scsi_options=0x3f8 
means that the default options would be to allow WIDE SCSI, FAST SCSI, tagged commands, global parity, synchronous transfer, linked commands and global disconnect/reconnect, i.e. all currently supported options.

SCSI chains may be made of single-ended or differential connections. They should not be mixed, as this may damage the equipment. Differential connections permit longer chains, but the hardware is usually more expensive. Single-ended chains must be less than 6 m in length; differential chains must be less than 20 m for synchronous connections or 25 m for asynchronous connections.

A Single ended SCSI connection uses "Normal" electrical signals, uses an open collector to the SCSI bus. The maximum length for SCSI-1 is a 6 meter cable with stubs of max 10cm allowed to connect a device to the main-cable. Most devices are single ended.

A differential SCSI connection uses two wires to drive one signal. It has a maximum cable length of 25 meters. Differential SCSI is electrically incompatible with single ended devices, allowed in SCSI-1 and upwards based systems.

The SCSI target numbers represent attachment points on the SCSI chain. Each target number may include as many as 8 devices (luns or logical unit numbers). Embedded SCSI devices only include one lun.

Higher target numbers receive better service and on a narrow bus, the target priorities run 7 -> 0. On a wide bus, they run 7 -> 0, then 15 -> 8. The host adapter is usually 7. This can cause problems where busy disks and tape devices share a SCSI bus, since tape devices are usually assigned target 6. If possible, isolate tape devices to their own SCSI bus.

If you are running older drives with Solaris 2.7 and above, you may run into a situation where there are many bus over-runs and errors. This is usually caused by having one or more disks on the SCSI bus that improperly implement the tagged queuing option (or don't support it at all!).

For a quick fix, append this line to /etc/system and reboot:

set scsi_options & ~0x80 
This turns off the Tagged Command Queuing. Tagged command queuing has also been seen to cause problems between Suns and some RAID implementations. However, the best fix is to replace the older drives with newer drives that will probably offer higher speed and greater capacity anyway.

Nevertheless, verify that your drives are incapable of supporting tagged queuing before making this change, since this will seriously degrade performance on disks that do properly support tagged command queuing. Setting the SCSI options for the target drives that can't support the option is therefore the preferred solution.

In Solaris 2.4 and later versions, you can set those options per SCSI bus. See your systems man entries or paper manual instructions for isp(7) and esp(7).

For some disks, decreasing the maximum number of queued commands by setting an option for the sd driver (SCSI Driver) using the host configuration variable sd_max_throttle, is all that is necessary:

set sd:sd_max_throttle=10 

Click to buy the book, "Oracle disk I/O tuning," by Mike Ault.


About the author

Mike Ault is a SearchOracle.com expert and a senior Oracle consultant with Burleson Consulting, and one of the leading names in Oracle technology. The author of more than 20 Oracle books and hundreds of articles in national publications, Mike Ault has five Oracle Masters Certificates and was the first popular Oracle author with his landmark book "Oracle7 administration and management." Mike also wrote several of the "Exam Cram" books, and enjoys a reputation as a leading author and Oracle consultant. Ask Mike a question today!


This was first published in August 2004

Dig deeper on Oracle database performance problems and tuning

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchDataManagement

SearchBusinessAnalytics

SearchSAP

SearchSQLServer

TheServerSide

SearchDataCenter

SearchContentManagement

SearchFinancialApplications

Close