This tip is an excerpt from the paper "Keep downtime short on 11i migration" and is brought to you by the International Oracle Users Group (IOUG). Become a member of the IOUG to access the paper referenced here and a repository of technical content created for Oracle users by Oracle users.
This tip continues from Evaluating which tasks can done before and after and Applications database migration using the exp/imp method.
It would be naive to hope that a freshly upgraded applications system can successfully go live without any patching, often extensive patching. Actually patching is always needed, the maintenance pack database driver (11.5.8 for example) always has to be applied. And this patch usually has pre-requirement patches, there can be quite a number of patches needed before you can even complete your basic upgrade. If you want to update the modules you use to latest version, even more, literally gigabytes of patches have to be applied. Thus, speeding up patching is a critical part in techniques of reducing upgrade downtime. Again, here we use similar approach as in the whole upgrade process, first we try to minimize the number and overhead of tasks to be run, then try to speed up the tasks left which definitely have to be run.
As the name says, Oracle Applications utilities provide us a way of merging several different patches together to one file. This allows us to save much time, reducing task duplication overhead of patches. Merging makes patching easier, allowing us to apply only few, big patch bundles, not tens of separate patches. Also, quite time consuming load of package revision cache is done only once per merged patch bundle, not for all individual patches – again, saving more time. (The package revision cache issue is relieved in AD.G).
The patch merging process is simple:
- Create directory for source patches.
- Unzip source patches directly under the created directory.
- View readme files to check for additional prerequisites, issues.
- Run AD Merge Patch (admrgpch) in source patch parent directory.
admrgpch source_dir target_dir [ –merged_name <name>]
- Check admrgpch logfile.
- Apply the merged patches.
Note that admrgpch creates separate files for c, d and g drivers. AD merge patch can:
- Merge patches of different source character sets
- Eliminate redundant tasks
- Check dependencies to include only highest revision of fix
- Can merge platform specific patches with generic patches
- Notify you if you try to merge incompatible patches
- Merge together different NLS translations of a patch
- Merging together of merged patches
- Consolidate README files into readme directory
- Retain original bug numbers in merged patch
AD merge patch cannot:
- Merge patches of different Apps releases
- Merge patches of different platforms
- Merge patches of different parallel modes
- Merge c,d,g patches together (available in future ver.)
By characteristics, we could divide patches applied during an upgrade to four:
- 10.7/11.0 preupgrade patches
- 11i Consolidated Upgrade Patches (CUP)
- 11i maintenance pack
- Family Packs / Mini-Packs / One-offs
Naturally, all those patch types should be merged to different sets because they are applied in different environments. Even if it would be possible to merge 11i maintenance pack with newer family and minipacks, it would be unsupported and noone could guarantee the result of applying this kind of patch.
When applying patches, significant amounts of time can be spent on starting up adpatch session, answering start-up questions, validating passwords, etc. This is especially true when there are lots of patches to apply, when patch merging isn't possible or used. Therefore Oracle provides some features to speed up patching by allowing storing default answers to start-up questions. Also password checking for hundreds of schemas can be disables if we are sure that all of them defined in Apps match the actual ones in database.
Creating a defaultsfile
For storing answers to adpatch initializing questions, we have to create a defaultsfile. The easiest way to do it, is by running adpatch with defaultsfile option. The result file is a simple text file, which can be edited later on or copied to another nodes or installations, if needed. The file must be located under $APPL_TOP/admin/<SID> directory, thus a correct syntax for creating the responsefile is:
Thus, an example for Unix would be:
The same for Windows:
Note that in Windows Oracle uses LOCAL environment variable to specify default database location, in unix it is TWO_TASK (ORA_DFLT_HOSTSTR in VMS).
Autopatch will be started as normal, except all answers to usual questions will be stored in specified defaultsfile. You should proceed until the prompt "Enter the directory where your Oracle Applications patch has been unloaded", then exit adpatch by typing "abort". After exiting adpatch, you should verify whether the defaultsfile was created to right location, checking contents of the file could be useful, too. You can create any number of defaultsfiles in case of need, just by specifying different file name from command line when running adpatch.
From now on, when you run adpatch with defaultsfile option pointing to location of newly created file, answers are provided to start-up questions according to contents of defaultsfile. (Note that immediately next time you run adpatch after creating defaultsfile, the questions are already answered, because we terminated adpatch with abort).
Running in non-interactive mode
Despite of several startup questions can now be answered automatically, the most important questions asking for the location of patch driver need still be answered. This can be done using adpatch command line parameters, leaving the defaultsfile independent of patch drivers we apply. Also, if we set the command line parameter interactive=no, no further questions are asked during the patching process if everything is configured properly. Few more command line options are introduced to syntax, to have greater flexibility for patching.
adpatch defaultsfile=<defaultsfile location> workers=<number of parallel workers> logfile=<log file location> patchtop=<directory where patch has been unloaded> driver=<driver name> interactive=no
If the non-interactive patching process should fail, we can continue applying the patch after correcting the problem by adding restart=y to command line options. Other command line options have to remain same, e.g. you can't continue applying patch from different location than the location specified at first run.
Skipping schema password validation
Every time you run adpatch, a schema password check is executed to check whether the schema passwords in database match to the ones defined in Applications. This naturally takes some additional time and is necessary if we are not sure whether the passwords have been manually changed at database level. But at least during a coordinated upgrade work we can be pretty sure, that the passwords aren't changed, thus we can skip the check after the first run. This can be done, by passing an option novalidate to adpatch.
adpatch defaultsfile=<defaultsfile location> workers=<number of parallel workers> logfile=<log file location> patchtop=<directory where patch has been unloaded> driver=<driver name> interactive=no options=novalidate
A simple script for running patches non-interactively with minimal overhead would look like this:
adpatch defaultsfile=$APPL_TOP/admin/$TWO_TASK/def.txt workers=4 logfile='echo $1 | cut -d. -f1`.log patchtop=`pwd` driver=$1 interactive=no options=novalidate
A script with contents written above could be saved to $AD_TOP/bin/adp (or adp.bat in windows). If you go to your patch directory and run the script adp with the patch driver file name as only parameter, the patch is automatically applied, if no problems occur. Note, that the command has to be run for all three drivers (c,d,g) where relevant. For better use in scripts, full path names for patch driver locations must be specified. The following is an example of script, which can handle full path names. This script takes also just one argument, which is the full path name of c,d or g driver.
adpatch defaultsfile=$APPL_TOP/admin/$TWO_TASK/def.txt workers=4 logfile=`basename $1 | cut -d. -f1`.log patchtop=`dirname $1` driver=`basename $1` interactive=no options=novalidate
After patching, it is often required in readme files and instructions that some maintenance task has to be run using adadmin. Adadmin also supports defaultsfile feature, thus allowing us to speed it up as well. We could create a separate defaultsfile for every different operation we want to run in non-interactive mode, or one file for series of operations. The syntax for creating the defaultsfile is following:
When at adadmin prompt, run all your necessary tasks, adadmin will save all the commands into specified defaultsfile. When defaultsfile is created, the maintenance tasks in defaultsfile can be run non-interactively using the interactive=no parameter.
adadmin defaultsfile=$APPL_TOP/admin/$TWO_TASK/admin.txt logfile=admin.log workers=4 interactive=no
Note that if any errors occur during maintenance tasks, the non-interactive adadmin will be terminated and adadmin has to be run in interactive mode to respond to the prompt. Also, following maintenance tasks, which aren't meant to be run regularly, cannot be run in non-interactive mode:
- Convert to Multiple Reporting Currencies
- Create Applications environment file
- Convert character set
- Convert to MultiOrg
- Copy files to destinations (can be run non-interactively starting from AD.H)
When applying multiple patches, it would be reasonable to defer all the maintenance tasks mentioned in readme files to run only after all patches have been applied. There is no requirement for them to be run immediately after patching, but the tasks definitely have to be run before using the system.
Finding patches and their prerequisites
Although searching and downloading patches isn't exactly a task to do during production downtime, there are few nice features in recent Apps versions for easier patch maintenance.
Patch prerequisite checking
Starting from AD.G autopatch is able to verify whether all prerequisite patches have been applied when applying c-driver of a patch. No check is done for d or g drivers. This only works for recent patches, which have a .ldt file containing patch metadata with them. When adpatch determines, that some prerequisites are missing, it will list the missing patches and fails. The automatic patch checking feature also includes 'compatible feature prereq' line in driver files, thus preventing these patches from being applied using older versions of autopatch. Also, if adpatch determines, that the patch being applied has already been applied in past, it asks whether you want to reapply it. In order to apply any patches with prerequisite checking you have to run "Maintain snapshot information" under "Maintain Applications Files" for each APPL_TOP in adadmin. This operation gathers current file, file version and bugfix information that adpatch uses for prerequisite checking.
Patch advisor functionality
Starting from 11.5.9 there will be Patch Advisor functionality in Oracle Applications Manager. Patch advisor will recommend patches based on information downloaded from Oracle, the specific products installed on a system, and other customizable criteria. The patch advisor will also be able to analyze a specific patch to identify the delta between that patch's prerequisites, and the patches already installed on a given system. Both of these features are helpful, but neither one frees us from reading through the readme files yet, because those mechanisms are very fresh and not very sophisticated yet.