Kindred Healthcare is taking a slightly different approach enterprise storage. Although expanded capacity was important,...
the greater concern lay in recovery and storage of critical operating system data used to run hundreds of remote servers.
The company did a self-installation of ERDisk for Windows, a product by Dublin, Ohio-based Aelita Software Corp., to aid in automated and centralized backup and storage of system operating files. As a result, Kindred has been able to reduce the downtime associated with rebuilding entire operating systems, says Tim Hesson, Kindred's corporate manager of operating systems.
"We've reduced what generally had been a 24-hour issue to less than an hour," says Hesson.
Kindred's storage needs encompass both corporate financial data and clinical data. The company operates nearly 500 hospitals and healthcare facilities in 48 U.S. states. Local servers are used to power most of Kindred's outlying facilities, with tape drive technology used to store and backup data. The company's multi-pronged strategy also includes a storage area network (SAN) with about 20 terabytes of capacity at its Louisville, Ky., headquarters.
"Our remote facilities are almost a completely different architecture than our corporate (network)," says Hesson. "The corporate SAN stores a lot of centralized applications, including clinical applications, that are used by people at our field facilities. By the same token, it also stores accounts receivable records and other corporate financial data."
The remote facilities posed the biggest storage challenge, for two main reasons. First, Kindred had "gotten into a line of servers with inferior tape drives," says Hesson, exposing data when they failed. This heightened the importance of storing backup data for system recovery. In addition, the servers ? though local to the individual facility ? were managed remotely by information technology personnel in Louisville.
Second, Kindred had to rely on nurses' aides, office workers and other non-IT staff to be conscientious enough to swap out new backup tapes routinely. For one reason or another, however, backups often weren't performed. Kindred tried using technologies such as Backup Exec by Veritas to prompt better tape backups, to no avail. Even a well-documented procedure and special training curricula didn't help. This meant that, when a system failed, it often would have to be rebuilt from scratch ? a costly and time-consuming endeavor. "Technically you're supposed to do backups every time you make a change, but we have so many systems it's difficult (for non-IT staff) to be that conscientious," says Hesson.
Kindred uses redundant array of independent disks, or RAID configurations, for storage. Each time an operating system was rebuilt, important information for applications, registry components, connectivity and network services was at risk of being lost or corrupted. The resulting unplanned downtime ran into hours. Hesson says Kindred began looking for a way to centralize backup of operating system files, where 70% to 80% of its failures originated.
A special feature of ERDisk is its creation of emergency repair disks that keep operating system data in a central storage location on a network, says Ratmir Timashev, Aelita's president and chief executive officer. These special disks include registry components of the system state and can be created for all network computers in a Windows environment, including domain controllers, servers and workstations.
ERDisk for Windows was attractive to Kindred because it enabled operating system files to be backed up automatically over very low bandwidth. Kindred programmed the software to perform backups on Saturday evenings. Backing up from the stored operating system files, as opposed to tape backups, minimized risk and reduced the IT management burden. "We're still missing some of those (tape) backups but it's not hurting us as badly because we're able to replace the system state and get it back online with the existing data," says Hesson. "It's also reduced the pressure those inferior tape drives were exerting on the enterprise because we don't have that exposure as much."
In addition, data recovery rates have gone from about 40% "even if we had the tapes, which we generally didn't," to around 98-99%, Hesson says.
That's not to say there weren't technical problems. When ERDisk was first installed, a glitch prevented the software from recognizing Mylex SCSI controllers. "It would sort of bypass it. We immediately called Aelita, which resolved it in two hours" by e-mailing Kindred a patch, and patching its own application in about two weeks, says Hesson. "That was impressive."
Aelita says licensing for ERDisk for Windows starts at $99 per server. The company also makes a companion product, ERDisk for Active Directory, which Kindred is using during a Windows 2000 migration.
For more information about Kindred Software, visit its Website.
For more information about Aelita's ERDisk, visit its Website.
Garry Kranz is a freelance business and technology writer based in Richmond, Va.
This Featured Topic explores Disaster Recovery policy
Have questions about backup? Ask our backup expert