Sometimes a simple plan to save time can really backfire. In Oracle DBA Ron Horjus' case, a mistake he made while implementing a new file path naming standard was so memorable, and miserable, that the details are vivid more than a decade later. At the start, Horjus simply intended to take advantage of the wildcard. He explained for SearchDatabase.com readers how this seemingly simple procedure went astray, and resulted in his company putting in place a new a new policy regarding DBA permissions.
"I was employed as a DBA and maintained all the Oracle/Unix servers for this manufacturer. One of these servers hosted an application for plant maintenance.
I needed to install an Oracle upgrade, so the application, one of several on that server, was shut down. After completing the upgrade, I decided to change the ownership of all the Oracle-related parameter files and executables to "Oracle", with group "dba" -- the new standard at the shop.
To facilitate a faster install and to reduce "account" hops, I was using the root account, a common procedure in those days. I changed directories to "/opt/oracle" and entered the "change owner" command with the following file path wild card: " .*". What I didn't know before, but what I found out very quickly after hitting the "enter" key, is that this wildcard not only traces the directory tree down (as I intended), but also up the chain.
Within seconds, the entire server crashed to a halt because all files and executables were owned, inexplicably, by Oracle, not by root or sys.
Fortunately, the system administrator had just recently taken a backup of the system and managed to restore the file system in eight hours.
Needless to say, since that experience, another new standard was put in place: "The DBA shall not have access to the root password".
For more true DBA bloopers, click:
Have your own tale of woe to share? Submit your backup/recovery snafus, tuning disasters and ugly upgrades. Stories of good intentions gone bad, over-ambitious and under-trained newbies, clueless consultants, and even more clueless managers will all be accepted. The submitter of the most amusing or wince-inducing blooper of the month will receive a free copy of Craig Mullins' new book Database administration: The complete guide to practices and procedures. Send your bloopers to us today!