If you have been patching your Oracle Enterprise Manager Environment frequently you were struggling with patching the WLS and all other components.
Since the latest RU for OEM (13.5.0.22) there is a new patching feature available. The holistic patching, as explained in this blog post (https://blogs.oracle.com/observability/post/oracle-enterprise-manager-13ru22-available). This patching allows you to patch all involved components in one shot.
The first Holistic Patch was release on May 21 (Patch #36635982). This patch is 3.5GB (extracted 5.3GB) in size, compared to the latest RU for the Enterprise Manager which is “only” 1.3 GB it is huge. But this patch is a “Generic Platform” patch an it includes all the patches for all available OEM platforms.
As an example, these are the OHS patches included:
Apply the patch
After you’ve downloaded the patch and verified all prerequisites – if you are already on 13.5.0.22 you do not have to update the omspatcher – you could start with applying the patch.
As always you have to unzip the patch first and then run the analyze part, not that there is a new omspatcher parameter -spb_patch. This step took ~1min in my case.
omspatcher apply <tmporary_dir>/36635982 -spb_patch -analyze
After that I strongly suggest to verify the WLS Console is up and running and you can login to the console. Beside that you have to be prepared for the patching itself. Because the first thing which is done during the patching phase is update the JDK in the $ORACLE_HOME:
Wed May 29 08:18:30 2024 :: The Env version is 1.8.0_261
Wed May 29 08:18:30 2024 :: The installed java vendor is Oracle Corporation
Wed May 29 08:18:30 2024 :: The tar file is /u01/tmp/36635982/binary_patches/jdk/linux64/jdk-8u411-linux-x64.tar.gz
Wed May 29 08:18:30 2024 :: The new jdk file is jdk1.8.0_411
Wed May 29 08:18:30 2024 :: The new version is 1.8.0_411
Wed May 29 08:18:30 2024 :: The input version after formatting 1.8.0
Wed May 29 08:18:30 2024 :: The input release after formatting 411
Wed May 29 08:18:30 2024 :: return code is 1
Wed May 29 08:18:30 2024 :: The Env version after formatting 1.8.0
Wed May 29 08:18:30 2024 :: The Env release after formatting 411
Wed May 29 08:18:30 2024 :: return code is 1
Wed May 29 08:18:30 2024 :: Final Input version is 1.8.0.411 and Final Env version is 1.8.0.261
Wed May 29 08:18:30 2024 :: The compare val is 1
Wed May 29 08:18:30 2024 :: The Env version is lower as compared to JDK included in patch :: JDK will be updated
Wed May 29 08:18:30 2024 :: Updating the JDK now...
Wed May 29 08:18:30 2024 :: Copy command is cp -r /u01/app/oracle/product/mdlw135//.omspatcher_storage/temp/jdk1.8.0_411 /u01/app/oracle/product/mdlw135/oracle_common
Wed May 29 08:18:30 2024 :: Move command is mv /u01/app/oracle/product/mdlw135/oracle_common/jdk1.8.0_411 /u01/app/oracle/product/mdlw135/oracle_common/jdk_temp
Wed May 29 08:18:30 2024 :: emInstanceMapping.properties Exists..Stopping the oms..
Wed May 29 08:18:30 2024 :: /u01/app/oracle/product/mdlw135/bin/emctl stop oms
Wed May 29 08:18:30 2024 :: Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved.
Stopping Oracle Management Server...
Node Manager Not Running
Oracle Management Server is Up
JVMD Engine is Up
Wed May 29 08:18:30 2024 :: Old java folder renamed to /u01/app/oracle/product/mdlw135/oracle_common/jdk_1716963516
Wed May 29 08:18:30 2024 :: Updated Java Successfully
I missed that my WLS console was not up and running and the apply phase failed at the phase where I had to enter the password for the console. Normally that wouldn’t be a problem, because I just restart my OEM environment but after the start of the Enterprise Manager I tried to login and got the following error:
An internal error has occurred. Check the log file for details.
This error pops-up if you have an incompatible JDK version and even a rollback of the patch didn’t rollback the new JDK version!
But there is a easy solution if you have to revert the jdk to the old jdk. There is a jdk_update…log file in $ORACLE_HOME/cfgtoollogs/omspatcher/SPB_PATCH in this logfile is written to which directory ($ORACLE_HOME/oracle_common/jdk_<a date>) the old jdk was copied. you can easily copy this folder back to jdk. This is described in this MOS note: OEM 13c: How To Apply/Rollback Holistic Patch (SPB, CPU, and JDK) On OEM 13.5 (Doc ID 3002450.1)
In my situation the solution to overcome that is to start the admin server only:
emctl start oms -admin_only
After that I was able to start the patching with:
omspatcher apply <tmporary_dir>/36635982 -spb_patch
If you have a mutli-oms environment and the patch is not located on a shared environment you’ll see the following message at the end of patching:
...
[May 29, 2024 9:51:36 AM] Applying Patches :: End
[May 29, 2024 9:51:36 AM] Starting the oms....
[May 29, 2024 9:51:36 AM] The command after checking the OS is /u01/app/oracle/product/mdlw135/bin/emctl start oms
[May 29, 2024 9:53:43 AM] Oracle Enterprise Manager Cloud Control 13c Release 5
[May 29, 2024 9:53:43 AM] Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved.
[May 29, 2024 9:53:43 AM] Starting Oracle Management Server...
[May 29, 2024 9:53:43 AM] WebTier Successfully Started
[May 29, 2024 9:53:43 AM] Oracle Management Server Successfully Started
[May 29, 2024 9:53:43 AM] Oracle Management Server is Up
[May 29, 2024 9:53:43 AM] JVMD Engine is Up
[May 29, 2024 9:53:43 AM] The file does not exist /u01/tmp/36635982/version.txt
[May 29, 2024 9:53:43 AM] We will not be patching the add omses......generating the addomsScripts
[May 29, 2024 9:53:43 AM] totalCmd : /u01/app/oracle/product/mdlw135/perl/bin/perl /u01/app/oracle/product/mdlw135/OMSPatcher/oms/createAddOMSScript.pl /u01/app/oracle/product/mdlw135 apply /u01/tmp/36635982 scripts_09-53-43 1 NA
[May 29, 2024 9:53:43 AM] The addoms script is generated
/u01/app/oracle/product/mdlw135/.omspatcher_storage/oms_session/scripts_09-53-43/addOMSPatcherScript.sh
[May 29, 2024 9:53:43 AM] Copy the script to the add oms and run it
[May 29, 2024 9:53:43 AM] File deleted successfully : /u01/app/oracle/product/mdlw135/linux64_patchlist.txt
You have to copy the unziped to the same directory on the additional OMS or to change the created addOMSPatcherScript.sh accordingly and execute it.
