SAN FRANCISCO - Oracle’s largest in-house database is a 101-terabyte database running Beehive, the company’s in-house email and collaboration software, running on nine Oracle Exadata boxes.
The database handles more than 140,000 accounts and 3.5 million emails per day, according to Campbell Webb, vice president of Oracle product development IT.
“It is our largest application in-house,” he said. “It is Oracle’s largest backend database.”
The application runs on nine full-racks of Exadata, all wired with InfiniBand. It has 24 instances of Oracle Database, although Webb said only 16 are live right now. He said they plan to increase the number of live instances in the next year.
The systems handle almost 1,5000 user transactions per second on average; at peak it’s more than 5,000 transactions per second. The environment also handles 176,000 I/O operations per second (IOPS).
Webb said the Beehive environment was migrated from Exadata Version 1, the HP-Oracle joint venture that came out in 2008 and saw significant improvements as a result – 20-times improvement in user I/O wait and 80x in database cluster wait, for example.
Webb revealed the details at Oracle OpenWorld this week in a session with two other Oracle shops that have consolidated on Exadata.
One of them, Whole Foods Market, consolidated four different server platforms and five different versions of Oracle Database onto two quarter-rack Exadata boxes, one for each data center and providing standby environments for each other.
Jerry Gregoire, DBA team leader for Whole Foods, said all the various platforms were 4-7 years old. It reached a point where the servers hosting databases had maintenance contracts that were “best effort agreements.” It was time to upgrade. The alternative to Exadata was to take each server and application group and upgrade each one-by-one. Gregoire estimated that would take three years, and further estimated that that was unacceptable.
“That’s assuming it wouldn’t get derailed midway through,” he said. “I’m three months impatient, never mind three years.”
Consolidating all those old servers to the Exadata wasn’t the radical part, however. Gregoire said the biggest part of the project was getting the OK from senior management and then issuing an ultimatum to the applications teams that they had to upgrade to the latest versions of their application.
“Their reaction, as you can imagine, was less than enthusiastic,” Gregoire said. “We asked all of them to upgrade to an edition none of them had any direct experience on, onto a database none of them had worked on, onto new hardware none of them knew about.”
But, Gregoire said, he offered some promises to the application teams. Among them: Database performance would skyrocket, and the support model was going to be streamlined and improved.
The company had the Exadatas – which also included a third quarter-rack non-production system – delivered in January. By April testing had been done and production migrations began. They plan to hit their goal of migrating everything by the end of this year.
Gregoire said the company had certain processes that normally took three days; now they take 14 hours.
“That actually creates complications for our apps teams because now they have to fill in two more days a week,” Gregoire joked.
Bronwyn Altizer, senior DBA consultant for Cardinal Health, had a similar story. The company retired more than 20 physical servers in favor of two half-rack Exadatas that went live in January. Like Whole Foods, the boxes serve as DR for one another.
Altizer said the initial obstacle the company had to overcome was its applications and production managers weren’t used to sharing their environments. They didn’t want to worry about what another applications time was doing. They didn’t want to deal with shared downtime. Altizer and the DBA team were eventually able to convince them that their production wouldn’t suffer.
While some IT shops say that having one vendor to go to for all programs leads to vendor lock-in that they don’t like, in Cardinal Health’s situation Altizer said they liked it.
“Having one place to go to for support has probably been one of the best experiences,” she said. “We don’t have to call the SAN (storage-area network) guys, we don’t have to call the server guys. We just call Oracle.”