This is Part 2 of a two-part series on Oracle license audits. Part 1, beware the Oracle license audit, introduced the reader to Oracle license audits, explaining the reasons behind them happening, and why Oracle is so aggressive with them.
An Oracle license audit can be daunting, but knowing exactly what happens and what tends to trigger violations can be helpful in avoiding unnecessary extra fines.
Howard Latham, the IT infrastructure manager for a U.K. market research company, was surprised when he found out his shop was getting audited. According to him, the company has a limited amount of databases and continuously crunches the same data.
“We’re not growing,” he said. “We’re not increasing our revenue. We provide a service in a fairly limited market, and we’ve got it pretty much sewn up.”
It’s important to know some reasons why Oracle might initiate a license audit; these were outlined in the first part of this series. It’s also important to know exactly what happens when Oracle ends up pointing its finger at your IT shop.
Oracle declined to comment.
One of the typical examples (of a licensing infraction) and most common is that customers are using features of the diagnostic and features packs. In (Oracle Database) 11g R2 there is a way to disable them using parameters, but not many customers are aware of this. This is especially relevant to bigger, more conservative organizations with big database footprints. This could hit them hard.
Alex Gorbachev, CTO, The Pythian Group
The Oracle license audit process
It starts with a letter from Oracle License Management Services (LMS). LMS is the company’s auditing division, though you’ll probably never hear them say the word audit—it doesn’t like the term. The letter usually says something about the customer being selected for a license review, and then asks the customer to respond to the letter within some time period, which is commonly 30 days or less.
Once the audit starts, Oracle gives the customer a spreadsheet and flowchart document. It will depict the customer’s entire server deployment and requires the customer to fill in wherever it has Oracle software running. It also includes 30 to 40 questions on the customer’s Oracle environment, according to Eliot Arlo Colon, president of Miro Consulting.
Usually the audit goes deeper than this, but occasionally Oracle will see the spreadsheet and realize that the customer’s Oracle environment isn’t as complex as imagined. For example, a customer might own 50 processor licenses. Someone from the company fills a spreadsheet showing it has 30 licenses installed with no other Oracle software. It might end there.
But according to Colon, about 80% of the time it doesn’t. From there Oracle will send the customer SQL scripts that the customer runs in the customer’s environment. The scripts spider their way through the customer’s IT infrastructure, detecting any use of Oracle software.
Then Oracle reviews the results.
Oracle license violations: How they happen
In some cases, Oracle finds no violations. Actually, according to a recent poll by Alex Gorbachev from The Pythian Group Inc., Oracle finds license violations a little more than half the time. But when it does find problems, oftentimes they’re big—as in expensive—ones to fix.
One Oracle database administrator (DBA) described work he did for a U.K. governmental agency. The DBA, Norman, only wanted his first name used because of how much the agency had to pay Oracle after the review: around €3 million. Norman said his company normally filled in and sent back the auditing spreadsheet whenever Oracle requested. But the person in charge of it left the agency and no one replaced him for some time. When someone did, he sought to sort out licensing with Oracle.
“The spreadsheet was filled, in and it all hit the fan,” Norman said. The agency spent months dealing with Oracle and an Oracle partner named 2e2. Eventually they came to a resolution whereby the agency would pay Oracle a boatload of money, and Oracle would leave them alone.
There is a bunch of areas where Oracle might find license violations. Colon said his company once tried to assemble a pamphlet on the myriad violations. They stopped after 80 pages. But there are a few major things to watch for.
First, it can happen when IT staff takes Oracle software from development to production to shelfware. Those intermediary steps might trigger license violations. Developers might inadvertently turn on features during an installation or upgrade that essentially takes the product from a free development platform to a pricey production platform.
Second, upgrading applications can often lead to higher licensing costs because what was included before in the standard price is now considered a premium package.
“One of the typical examples and most common is that customers are using features of the diagnostic and features packs,” Gorbachev said. “In (Oracle Database) 11g R2 there is a way to disable them using parameters, but not many customers are aware of this.”
“This is especially relevant to bigger, more conservative organizations with big database footprints,” Gorbachev added. “This could hit them hard.”
This is one thing that happened to the U.K. governmental agency recently, according to Norman. Following the audit, IT staffers were careful to ensure that all new Oracle installs got only what was licensed. But when patching recently from 10.2.0.3 to 10.2.0.5, IT discovered that part of the patching process added back in all the non-licensed options that they had carefully switched off at install time.
“Nice little earner for Oracle that!” Norman said.
As Oracle improves the product with each upgrade, the new features often become part of a new management pack that, though installed by default, trigger extra licensing costs. So features that end users might think are free actually cost something.
“One thing you might ask is why Oracle is introducing more features separately,” Gorbachev said. “Well, it’s a great way for Oracle to add additional revenue. Oracle Database already dominates the market, and so it’s harder for them to get more revenue. So packs are a great way to find another revenue stream.”
That raises another issue—that of an installed option vs. a used option.
“There is a very fine line between installing features and using features,” Gorbachev said. “It depends which documentation you’re looking at.”
Norman said Oracle issued his company a document saying that the company had to license any feature with Oracle that was being used. But Oracle itself had another document on its website saying that customers had to license a product if it was installed, whether it was used or not. That led to some confusion about how much the agency had to pay Oracle.
Third, a company may incur extra licensing costs during a merger or acquisition. Colon gave the example of one company that approached Miro saying it planned to make an acquisition in the next 60 days and wanted to know what the impact to software licensing would be. By the time Miro went through the company’s infrastructure, it discovered a $2 million liability if it made the acquisition.
Finally, the licensing contract that a customer signs with Oracle may be a static document, but it often refers to documentation on Oracle’s website that is dynamic—it could change at any time.
Colon pointed to the processor core factor table that Oracle has on its website. Oracle made a few changes to that table last year, changes that could affect license costs for customers who are, for example, running Oracle databases and applications on Intel Itanium chips.
“You may have a purchaser that says they want to buy four copies of a product and completely miss the fact that Oracle can change the license value based on what hardware the software is deployed on,” Colon said. “Even the savvy IT buyers don’t understand some of it.”
On the flipside, there may be opportunities for savings, particularly if a company is running Oracle-Sun server hardware and processors. Colon gave the example of one company that was looking to purchase some Sun hardware and had previously kept a budget number based on old core factors. Then last year Oracle lowered the core factor for Sun Sparc processors. As a result, the company was able to buy more Sun hardware than it had anticipated and accelerated a project timeline by six months.
Finally, Colon added that when the dot-com bubble burst, the first people companies often cut were IT asset managers, who had responsibilities such as tracking Oracle license use.
“Most of those functioned disappeared because of cutting costs,” he said. “But now more than ever that function is needed.”