The following is part of a series on the different aspects of disk I/O performance and optimization for Oracle...
databases. Each tip is excerpted from the not-yet-released Rampant TechPress book, "Oracle disk I/O tuning," by Mike Ault. Check back to the main series page for upcoming installments.
The two peripheral buses on Sun systems are Sbus and PCI bus. Sbus typically runs at 20-25MHz and provides 32 or 64 bit sizes. The peak Sbus bandwidth is 200 MB/s. The PCI bus runs at 33 or 66MHz and may be 32 or 64 bit. The peak PCI bus bandwidth is 528 MB/s. Almost all add-on SCSI adapters will plug into the PCI bus.
SCSI buses on SUN can operate various bandwidths, speeds and throughputs (see table 3-2 to see the various bus specifications for SUN). The prtconf Solaris command is used to report information that can be used to determine the speed of a particular SCSI device. Most of the SCSI systems I have seen in recent years have been SCSI3 based.
|Protocol||Bus Width||Bus Speed||Throughput|
|Fast SCSI, SCSI2||8bit||10MHz||10MB/s|
|Fast Wide SCSI, SCSI3||16bit||10MHz||20MB/s|
|Wide Ultra SCSI, Fast20||16bit||20MHz||40MB/s|
|Wide Ultra2 SCSI, Fast40||16bit||40MHz||80MB/s|
|Ultra3, Ultra160, Fast80||16Bit||80MHz||160MB/s|
Table 3-2: SUN SCSI Specifications
The scsi_options parameter can be set in the /etc/system file to limit bus speed or set other characteristics. You should check device documentation to determine if these settings need to be specified for a specific device. The default scsi_options variable allows the widest range of capabilities that the SCSI host adapter can provide to be supported.
The default scsi_options value on Solaris 2.x works for both 5MB and 10MB devices. The driver will negotiate with each device to determine if it is 10MB transfer capable or not. If they are 10MB devices, 10MB transfer will be used. If not, 5MB transfer will be used.
SCSI subsystem options - on SUN Solaris, a global word of options are available. The bits of the global word are broken down as:
|0-2||Reserved for debugging/informational level|
|3||Reserved for a global disconnect/reconnect switch|
|4||Reserved for a global linked command capability switch|
|5||Reserved for a global synchronous SCSI capability switch|
The rest of the bits are reserved for future use.
The bits, which can be set are set using the hexadecimal values are shown in Table 3-3.
|Option tag||Option Value||Description|
|SCSI_DEBUG_TGT||0x1||Debug statements in target drivers|
|SCSI_DEBUG_LIB||0x2||Debug statements in library|
|SCSI_DEBUG_HA||0x4||Debug statements in host adapters|
|SCSI_OPTIONS_LINK||0x10||Global linked commands|
|SCSI_OPTIONS_SYNC||0x20||Global synchronous xfer capability|
|SCSI_OPTIONS_PARITY||0x40||Global parity support|
|SCSI_OPTIONS_TAG||0x80||Tagged command support|
|SCSI_OPTIONS_FAST||0x100||FAST scsi support|
|SCSI_OPTIONS_WIDE||0x200||WIDE scsi support|
Figure 3-3: SCIS Options Hexadecimal Values
The values for the various options are added to achieve the desired combination of features. For example, a line in /etc/system that read:
set scsi_options=0x3f8means that the default options would be to allow WIDE SCSI, FAST SCSI, tagged commands, global parity, synchronous transfer, linked commands and global disconnect/reconnect, i.e. all currently supported options.
SCSI chains may be made of single-ended or differential connections. They should not be mixed, as this may damage the equipment. Differential connections permit longer chains, but the hardware is usually more expensive. Single-ended chains must be less than 6 m in length; differential chains must be less than 20 m for synchronous connections or 25 m for asynchronous connections.
A Single ended SCSI connection uses "Normal" electrical signals, uses an open collector to the SCSI bus. The maximum length for SCSI-1 is a 6 meter cable with stubs of max 10cm allowed to connect a device to the main-cable. Most devices are single ended.
A differential SCSI connection uses two wires to drive one signal. It has a maximum cable length of 25 meters. Differential SCSI is electrically incompatible with single ended devices, allowed in SCSI-1 and upwards based systems.
The SCSI target numbers represent attachment points on the SCSI chain. Each target number may include as many as 8 devices (luns or logical unit numbers). Embedded SCSI devices only include one lun.
Higher target numbers receive better service and on a narrow bus, the target priorities run 7 -> 0. On a wide bus, they run 7 -> 0, then 15 -> 8. The host adapter is usually 7. This can cause problems where busy disks and tape devices share a SCSI bus, since tape devices are usually assigned target 6. If possible, isolate tape devices to their own SCSI bus.
If you are running older drives with Solaris 2.7 and above, you may run into a situation where there are many bus over-runs and errors. This is usually caused by having one or more disks on the SCSI bus that improperly implement the tagged queuing option (or don't support it at all!).
For a quick fix, append this line to /etc/system and reboot:
set scsi_options & ~0x80This turns off the Tagged Command Queuing. Tagged command queuing has also been seen to cause problems between Suns and some RAID implementations. However, the best fix is to replace the older drives with newer drives that will probably offer higher speed and greater capacity anyway.
Nevertheless, verify that your drives are incapable of supporting tagged queuing before making this change, since this will seriously degrade performance on disks that do properly support tagged command queuing. Setting the SCSI options for the target drives that can't support the option is therefore the preferred solution.
In Solaris 2.4 and later versions, you can set those options per SCSI bus. See your systems man entries or paper manual instructions for isp(7) and esp(7).
For some disks, decreasing the maximum number of queued commands by setting an option for the sd driver (SCSI Driver) using the host configuration variable sd_max_throttle, is all that is necessary:
Click to buy the book, "Oracle disk I/O tuning," by Mike Ault.
About the author
Mike Ault is a SearchOracle.com expert and a senior Oracle consultant with Burleson Consulting, and one of the leading names in Oracle technology. The author of more than 20 Oracle books and hundreds of articles in national publications, Mike Ault has five Oracle Masters Certificates and was the first popular Oracle author with his landmark book "Oracle7 administration and management." Mike also wrote several of the "Exam Cram" books, and enjoys a reputation as a leading author and Oracle consultant. Ask Mike a question today!