When Steve Helle became chief technology officer of NextAction Corporation, his 15-plus years of database management...
experience immediately told him that Microsoft's SQL Server and Windows Server operating system simply couldn't meet the ever-increasing demands of his firm's homegrown database modeling applications.
That was in March 2004, and the Westminster, Colorado-based NextAction, which uses database modeling techniques to assist customers with mailing campaigns, was facing the prospect of significant revenue loss due to increased downtime of mission-critical systems.
"When I came in, my assessment was that we had run out of gas in a variety of fashions," Helle said. "We were having system failures, machines would crash, SQL Server would crash, the application would crash, and it was a production-critical application so it had a lot of impact on our revenues if the crash was significant."
NextAction had about 13 terabytes of data and eight databases running on five Windows boxes at the time, and it was clear to Helle that something needed to be done. It was a realization that eventually led the year-and-a-half-old company to re-architect its entire IT infrastructure.
Helle said that he had never seen SQL Server scale to that size and believed it was time for a change. He was also concerned about the operating system and the hardware.
"It just was a matter of the company growing so fast and the data volume growing so fast that it just outpaced SQL Server," Helle said. "[And] I like Windows for my desktop, but I didn't like Windows running a 13 terabyte database environment."
Following a lengthy evaluation and testing process, the company decided to dump Microsoft Windows and SQL Server for Oracle Database 10g running on the open source Suse Linux operating system from Novell Inc. During the six-month implementation which lasted from December to June of 2005, Helle said the firm also moved off direct attached SCSI storage in favor of an HP-based storage area network (SAN) offering from a local reseller.
Helle says the biggest benefit of the now-completed initiative -- other than protected revenue as a result of reduced downtime -- is the fact that things run a lot faster now. He added that the new systems store data much more efficiently, resulting in smaller file sizes.
"We have a much faster system now and a much more scalable system," he said. "Right now that primary database is up to about seven terabytes, and we're on a SAN that can grow to about 330 terabytes."
Deciding on a new platform meant testing out the options, and according to Helle, that is just what NextAction did.
The firm purchased the latest version of 64-bit SQL Server and got it running on a 64-bit Itanium 4-way box from Dell. They also obtained the latest version of Oracle Database 10g and got that running on a 64-bit Sun Microsystems server. After running some processes, Helle said the Oracle/Opteron box proved to be the clear winner.
"Everything averaged out to about eight times faster than the 64-bit SQL Server environment," he said.
Not long after this, Helle spoke with an HP representative who sold Helle on the idea of an Opteron box with separate data and memory bus.
"We loaded up Suse and Oracle on [the new Opteron box] and ran out benchmarks, and it was across the board two times faster than Sun running Oracle," Helle said. "We ended up going with the Opteron environment running Suse, Oracle 10g and back-ended by an HP SAN."
Once it was clear that Linux would be the operating system of choice for NextAction's IT overhaul, Helle said the next thing to do was choose the best Linux distribution for the project.
Helle said that Novell Suse turned out to be the best option for his firm's large database application because "Oracle was more comfortable on the platform at the time" than Red Hat's Linux distribution. He added that NextAction's systems administrators also had a hand in choosing Suse.
"Our systems administration group had had some prior experience with both Red Hat and Suse, and they felt more comfortable with Suse," he said.
Helle said that one of the more tedious challenges of migrating NextAction's homegrown database modeling application from Windows Server to Suse involved converting all of the firm's customer data files from Windows to Unix format -- a process that generally involves a simple DOS-to-Unix command.
"It's a one-line command and it's pretty simple, but with our [sizable] data sets it could take 20 to 30 minutes per file to do that," he said.
Experts will attest that all major technology projects come with their own sets of challenges and unforeseen problems, and NextAction's IT overhaul was no exception.
Helle said that choosing a SAN and the right reseller to purchase it from proved to be one of the most troubling experiences of all. He said that the first HP-based storage area network the firm ordered wasn't delivered for three months into a six-month development cycle.
"I kept having to tell my DBAs to just take that 200 gigabyte hard drive that's on the server and test it out before we get the SAN, so that really was the biggest problem," Helle said. "We ended up switching over to a different HP storage environment."
The considerable demands placed on Helle and his team of four DBAs during the project also proved taxing.
"Maintaining a production environment that was 30 terabytes while developing a new application to replace it was a big challenge," he said.
Results measured in protected revenue
Helle said he couldn't exactly quantify the return on investment he expects to reap as a result of the IT overhaul, but he said that one thing is for sure -- Oracle Database 10g licensing fees are "expensive."
He added, however, that the increased fees his firm pays for licensing will likely be offset by the amount of revenue they'll "protect" as a result of increased uptime.
"The total cost of the solution was more, largely because we took the multiple terabytes and migrated it to a scalable SAN that could grow to 330 terabytes," Helle said. "But more importantly was that if our old database crashed during the last month of the year, one 24-hour outage could have cost several hundred thousand dollars."