ODA Upgrade from 19.16 to 19.20
Patching Oracle Database Appliance (ODA) Bare Metal Systems involves applying updates to both the operating system and Oracle Software Components.
Starting with Oracle Database Appliance release 19.11, Oracle Database Appliance uses the out-of-place patching model. This means that a new software Oracle Home is created when patching an Oracle Grid Infrastructure or Oracle Database home. Starting with Oracle Database Appliance release 19.11, the patches for Oracle Grid Infrastructure are available as images. All procedures remain the same.
The Oracle Database Appliance Patch Bundle contains the latest patches for DCS Admin, DCS Components, OAK, Oracle ILOM, Firmware, enhancements, and fixes.
The Oracle Database Appliance Release Notes include information about the latest Oracle Database Appliance patch bundle and a list of component versions in the patch bundle. Oracle recommends updating the appliance with the latest Oracle Database Appliance software version to take advantage of new features, fixes, and the latest supported component versions. See the Oracle Database Appliance Release Notes for the upgrade paths.
In this blog we will upgrade ODA (Single Node) from 19.16 to 19.20, always remember, if we need to upgrade to 19.20 your current version has to be 19.16 or later.
Minimum patch requirement for ODA releases:
Download and stage 19.20 version patches in the staging area:
In this activity, we are downloading the server patch, Grid infrastructure clone, and database clone. If you need patches for additional components refer to the below link for patch numbers:
https://docs.oracle.com/en/engineered-systems/oracle-database-appliance/19.20/cmtrn/oda-patches.html#GUID-220DA05B-0F52-4EDA-84C9-BFD15F43802D
Update the repository with the downloaded patches:
Unzip the downloaded patches and update the repository.
Update the repository with the server software file and Confirm that the repository update is successful.
/opt/oracle/dcs/bin/odacli update-repository -f /tmp/oda-sm-19.20.0.0.0-230802.1-server.zip
Update the repository with the Oracle Grid Infrastructure clone file and the Oracle Database clone file and Confirm that the repository update is successful.
/opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.20.0.0.0-230720-GI-19.20.0.0.zip /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.20.0.0.0-230720-DB-19.20.0.0.zip
Update DCS Admin:
/opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.20.0.0.0
Update DCS Components:
The command updates the Zookeeper, and MySQL, and updates other DCS components such as the DCS Agent, DCS CLI, and DCS Controller on Oracle Database Appliance.
If the DCS components are updated, then the message “status”: “Success” is displayed on the command line.
/opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.20.0.0.0
Update DCS Agent:
/opt/oracle/dcs/bin/odacli update-dcsagent -v 19.20.0.0.0
Create a pre-patch report and ensure pre-checks run successfully:
[root@test01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v 19.20.0.0.0
Job details
----------------------------------------------------------------
ID: 24184885-1966-4523-b610-1bcfaec6cc1f
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER, SERVER]
Status: Created
Created: February 8, 2024 6:39:04 PM AST
Message: Use 'odacli describe-prepatchreport -i 24184885-1966-4523-b610-1bcfaec6cc1f' to check details of results
[root@test01 ~]# odacli describe-prepatchreport -i 24184885-1966-4523-b610-1bcfaec6cc1f
Patch pre-check report
------------------------------------------------------------------------
Job ID: 24184885-1966-4523-b610-1bcfaec6cc1f
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER, SERVER]
Status: SUCCESS
Created: February 8, 2024 6:39:04 PM AST
Result: All pre-checks succeeded
Node Name
---------------
test01
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.20.0.0.0.
Is patch location available Success Patch location is available.
Verify OS patch Success Verified OS patch
Validate command execution Success Validated command execution
__ILOM__
Validate ILOM server reachable Success Successfully connected with ILOM
server using public IP and USB
interconnect
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.20.0.0.0.
Is patch location available Success Patch location is available.
Checking Ilom patch Version Success Successfully verified the versions
Patch location validation Success Successfully validated location
Validate command execution Success Validated command execution
__GI__
Validate GI metadata Success Successfully validated GI metadata
Validate supported GI versions Success Successfully validated minimum version
Validate available space Success Validated free space under /u01
Is clusterware running Success Clusterware is running
Validate patching tag Success Validated patching tag: 19.20.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate ASM in online Success ASM is online
Validate kernel log level Success Successfully validated the OS log
level
Validate minimum agent version Success GI patching enabled in current
DCSAGENT version
Validate Central Inventory Success oraInventory validation passed
Validate patching locks Success Validated patching locks
Validate clones location exist Success Validated clones location
Validate DB start dependencies Success DBs START dependency check passed
Validate DB stop dependencies Success DBs STOP dependency check passed
Validate space for clones Success Clones volume is already created volume
Evaluate GI patching Success Successfully validated GI patching
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
__SERVER__
Validate local patching Success Successfully validated server local
patching
Validate command execution Success Validated command execution
Please note if the pre-patch report does not take a filesystem snapshot, use the below steps to take the filesystem snapshot using odabr utility (which needs to be installed explicitly)
# df -Ph / /opt /u01
# pvs
# /opt/odabr/odabr backup -dryrun
- Specify size based on dryrun output
# /opt/odabr/odabr backup -snap
Or
# /opt/odabr/odabr backup -snap [-osize 30 -rsize 5 -usize 70]
# /opt/odabr/odabr infosnap
# lvs
Until this step, all the activities can be performed online, from now the downtime starts, and make sure the application is down.
Update Server/Server Patching:
Please note that the server reboots 5 minutes after the patching is completed
[root@test01 etc]# /opt/oracle/dcs/bin/odacli update-server -v 19.20.0.0.0
{
"jobId" : "2b4c40ef-b50e-42d1-b43f-a76c1bdb8e4a",
"status" : "Created",
"message" : "Success of server update will trigger reboot of the node after 4-5 minutes. Please wait until the node reboots.",
"reports" : [ ],
"createTimestamp" : "February 08, 2024 18:48:51 PM AST",
"resourceList" : [ ],
"description" : "Server Patching",
"updatedTime" : "February 08, 2024 18:48:51 PM AST",
"jobType" : null
}
[root@test01 ~]# odacli describe-job -i 2b4c40ef-b50e-42d1-b43f-a76c1bdb8e4a
Job details
----------------------------------------------------------------
ID: 2b4c40ef-b50e-42d1-b43f-a76c1bdb8e4a
Description: Server Patching
Status: Success
Created: February 8, 2024 6:48:51 PM AST
Message: Successfully patched GI with RHP
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validating GI user metadata February 8, 2024 6:49:06 PM AST February 8, 2024 6:49:06 PM AST Success
Validate ILOM server reachable February 8, 2024 6:49:06 PM AST February 8, 2024 6:49:08 PM AST Success
Validate DCS Admin mTLS setup February 8, 2024 6:49:08 PM AST February 8, 2024 6:49:08 PM AST Success
Configure export clones resource February 8, 2024 6:49:09 PM AST February 8, 2024 6:49:09 PM AST Success
Creating repositories using yum February 8, 2024 6:49:09 PM AST February 8, 2024 6:49:13 PM AST Success
Updating YumPluginVersionLock rpm February 8, 2024 6:49:13 PM AST February 8, 2024 6:49:13 PM AST Success
Applying OS Patches February 8, 2024 6:49:14 PM AST February 8, 2024 7:04:13 PM AST Success
Creating repositories using yum February 8, 2024 7:04:14 PM AST February 8, 2024 7:04:14 PM AST Success
Applying HMP Patches February 8, 2024 7:04:14 PM AST February 8, 2024 7:04:30 PM AST Success
Patch location validation February 8, 2024 7:04:30 PM AST February 8, 2024 7:04:30 PM AST Success
Oda-hw-mgmt upgrade February 8, 2024 7:04:30 PM AST February 8, 2024 7:05:00 PM AST Success
OSS Patching February 8, 2024 7:05:00 PM AST February 8, 2024 7:05:01 PM AST Success
Applying Firmware Disk Patches February 8, 2024 7:05:01 PM AST February 8, 2024 7:05:03 PM AST Success
Applying Firmware Controller Patches February 8, 2024 7:05:03 PM AST February 8, 2024 7:05:05 PM AST Success
Checking Ilom patch Version February 8, 2024 7:05:06 PM AST February 8, 2024 7:05:06 PM AST Success
Patch location validation February 8, 2024 7:05:06 PM AST February 8, 2024 7:05:06 PM AST Success
Save password in Wallet February 8, 2024 7:05:06 PM AST February 8, 2024 7:05:07 PM AST Success
Disabling IPMI v2 February 8, 2024 7:05:07 PM AST February 8, 2024 7:05:07 PM AST Success
Apply Ilom patch February 8, 2024 7:05:07 PM AST February 8, 2024 7:14:18 PM AST Success
Copying Flash Bios to Temp location February 8, 2024 7:14:18 PM AST February 8, 2024 7:14:18 PM AST Success
Starting the clusterware February 8, 2024 7:14:18 PM AST February 8, 2024 7:16:20 PM AST Success
Registering image February 8, 2024 7:16:20 PM AST February 8, 2024 7:16:21 PM AST Success
Registering working copy February 8, 2024 7:16:21 PM AST February 8, 2024 7:16:21 PM AST Success
Registering image February 8, 2024 7:16:21 PM AST February 8, 2024 7:16:21 PM AST Success
Creating GI home directories February 8, 2024 7:16:21 PM AST February 8, 2024 7:16:21 PM AST Success
Extract GI clone February 8, 2024 7:16:21 PM AST February 8, 2024 7:16:21 PM AST Success
Provisioning Software Only GI with RHP February 8, 2024 7:16:21 PM AST February 8, 2024 7:16:21 PM AST Success
Patch GI with RHP February 8, 2024 7:16:22 PM AST February 8, 2024 7:22:44 PM AST Success
Updating .bashrc February 8, 2024 7:22:44 PM AST February 8, 2024 7:22:45 PM AST Success
Updating GIHome in the repository February 8, 2024 7:22:45 PM AST February 8, 2024 7:22:45 PM AST Success
Updating GIHome version February 8, 2024 7:22:45 PM AST February 8, 2024 7:22:49 PM AST Success
Validate GI availability February 8, 2024 7:22:53 PM AST February 8, 2024 7:22:53 PM AST Success
Patch KVM CRS type February 8, 2024 7:22:53 PM AST February 8, 2024 7:22:54 PM AST Success
Patch VM vDisks CRS dependencies February 8, 2024 7:22:54 PM AST February 8, 2024 7:22:54 PM AST Success
Patch DB System domain config February 8, 2024 7:22:54 PM AST February 8, 2024 7:22:54 PM AST Success
Update System version February 8, 2024 7:23:35 PM AST February 8, 2024 7:23:35 PM AST Success
Cleanup JRE Home February 8, 2024 7:23:35 PM AST February 8, 2024 7:23:35 PM AST Success
Add SYSNAME in Env February 8, 2024 7:23:35 PM AST February 8, 2024 7:23:35 PM AST Success
Starting the clusterware February 8, 2024 7:23:35 PM AST February 8, 2024 7:25:30 PM AST Success
Setting ACL for disk groups February 8, 2024 7:25:30 PM AST February 8, 2024 7:25:42 PM AST Success
Enable LKCE February 8, 2024 7:27:20 PM AST February 8, 2024 7:30:51 PM AST Success
Update lvm.conf file February 8, 2024 7:30:53 PM AST February 8, 2024 7:30:53 PM AST Success
Update previous workarounds February 8, 2024 7:30:53 PM AST February 8, 2024 7:30:53 PM AST Success
Generating and saving BOM February 8, 2024 7:30:53 PM AST February 8, 2024 7:31:22 PM AST Success
PreRebootNode Actions February 8, 2024 7:31:22 PM AST February 8, 2024 7:32:07 PM AST Success
Reboot Ilom February 8, 2024 7:32:07 PM AST February 8, 2024 7:32:07 PM AST Success
Update Storage/Storage patching:
Run pre-patch report
[root@test01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -st -v 19.20.0.0.0
Job details
----------------------------------------------------------------
ID: 2beee6d6-c776-44a1-b2b6-5d2c091c5efe
Description: Patch pre-checks for [STORAGE]
Status: Created
Created: February 8, 2024 7:41:32 PM AST
Message: Use 'odacli describe-prepatchreport -i 2beee6d6-c776-44a1-b2b6-5d2c091c5efe' to check details of results
[root@test01 ~]# odacli describe-prepatchreport -i 2beee6d6-c776-44a1-b2b6-5d2c091c5efe
Patch pre-check report
------------------------------------------------------------------------
Job ID: 2beee6d6-c776-44a1-b2b6-5d2c091c5efe
Description: Patch pre-checks for [STORAGE]
Status: SUCCESS
Created: February 8, 2024 7:41:32 PM AST
Result: All pre-checks succeeded
Node Name
---------------
test01
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__STORAGE__
Validate patching tag Success Validated patching tag: 19.20.0.0.0.
Patch location validation Success Verified patch location
Patch tag validation Success Verified patch tag
Storage patch tag validation Success Verified storage patch location
Verify ASM disks status Success ASM disks are online
Validate command execution Success Validated command execution
Update Storage
[root@test01 ~]# /opt/oracle/dcs/bin/odacli update-storage -v 19.20.0.0.0
{
"jobId" : "8f2a025b-4cfe-44d2-88be-e6e027d5e850",
"status" : "Created",
"message" : "Success of Storage Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "February 08, 2024 19:44:46 PM AST",
"resourceList" : [ ],
"description" : "Storage Firmware Patching",
"updatedTime" : "February 08, 2024 19:44:46 PM AST",
"jobType" : null
}
[root@test01 ~]# odacli describe-job -i 8f2a025b-4cfe-44d2-88be-e6e027d5e850
Job details
----------------------------------------------------------------
ID: 8f2a025b-4cfe-44d2-88be-e6e027d5e850
Description: Storage Firmware Patching
Status: Success
Created: February 8, 2024 7:44:46 PM AST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Applying Firmware Disk Patches February 8, 2024 7:44:50 PM AST February 8, 2024 7:44:53 PM AST Success
Applying Firmware Controller Patches February 8, 2024 7:44:53 PM AST February 8, 2024 7:51:07 PM AST Success
Generating and saving BOM February 8, 2024 7:51:07 PM AST February 8, 2024 7:51:12 PM AST Success
PreRebootNode Actions February 8, 2024 7:51:12 PM AST February 8, 2024 7:51:12 PM AST Success
Reboot Ilom February 8, 2024 7:51:12 PM AST February 8, 2024 7:51:12 PM AST Success
Note: If disks are not up-to-date, it may be related to ILOM and BIOS patches.
Update DBHomes/Patch DBHomes:
Run pre-patch report
[root@test01 ~]# odacli list-dbhomes
ID Name DB Version DB Edition Home Location Status
---------------------------------------- -------------------- ---------------------------------------- ---------- --------------------------------------------- ----------
0f6c2c52-3d0c-4d9b-8acc-167bbbe80195 OraDB19000_home1 19.16.0.0.220719 EE /u01/app/odaorahome/oracle/product/19.0.0 CONFIGURED
[root@test01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 0f6c2c52-3d0c-4d9b-8acc-167bbbe80195 -v 19.20.0.0.0
Job details
----------------------------------------------------------------
ID: 0b0f53e2-4ae8-41de-a3ff-82f8c9b24a04
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home1
Status: Created
Created: February 8, 2024 7:58:24 PM AST
Message: Use 'odacli describe-prepatchreport -i 0b0f53e2-4ae8-41de-a3ff-82f8c9b24a04' to check details of results
[root@test01 ~]# odacli describe-prepatchreport -i 0b0f53e2-4ae8-41de-a3ff-82f8c9b24a04
Update DBHome
[root@test01 etc]# /opt/oracle/dcs/bin/odacli update-dbhome --id 0f6c2c52-3d0c-4d9b-8acc-167bbbe80195 -v 19.20.0.0.0 -f
{
"jobId" : "0d7fd66f-6c9a-4b7e-93ea-5fbec5af34d9",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "February 08, 2024 21:24:29 PM AST",
"resourceList" : [ ],
"description" : "DB Home Patching: Home Id is 0f6c2c52-3d0c-4d9b-8acc-167bbbe80195",
"updatedTime" : "February 08, 2024 21:24:29 PM AST",
"jobType" : null
}
[root@test01 etc]# odacli describe-job -i 0d7fd66f-6c9a-4b7e-93ea-5fbec5af34d9
Job details
----------------------------------------------------------------
ID: 0d7fd66f-6c9a-4b7e-93ea-5fbec5af34d9
Description: DB Home Patching: Home Id is 0f6c2c52-3d0c-4d9b-8acc-167bbbe80195
Status: Success
Created: February 8, 2024 9:24:29 PM AST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Creating wallet for DB Client February 8, 2024 9:24:36 PM AST February 8, 2024 9:24:36 PM AST Success
Patch databases by RHP - [TLHHBP] February 8, 2024 9:24:36 PM AST February 8, 2024 9:29:51 PM AST Success
Updating database metadata February 8, 2024 9:29:51 PM AST February 8, 2024 9:29:51 PM AST Success
Set log_archive_dest for Database February 8, 2024 9:29:51 PM AST February 8, 2024 9:29:54 PM AST Success
Update System version February 8, 2024 9:29:54 PM AST February 8, 2024 9:29:54 PM AST Success
Generating and saving BOM February 8, 2024 9:29:54 PM AST February 8, 2024 9:30:35 PM AST Success
TDE parameter update February 8, 2024 9:30:35 PM AST February 8, 2024 9:30:35 PM AST Success
Cleanup Patchrepo:
# odacli cleanup-patchrepo -cl -v 19.16.0.0.0 # cd /backup/ODA_patches # rm -f oda-sm-19.16.0.0.0-date-server.zip odacli-dcs-19.16.0.0.0-date-GI-19.16.0.0.zip odacli-dcs-19.16.0.0.0-date-DB-19.18.0.0.zip
Post Patching Tasks:
Add your additional rpms manually if needed Put back your profile scripts for grid and oracle users Remove the snapshots done with odabr. Note: Run on both nodes # odabr delsnap # /opt/odabr/odabr infosnap # lvs # pvs
Remove OLD DB home if not needed
odacli delete-dbhome -i db244e9d-9d3e-446f-9e15-461913aaf1b6
Run datapatch and release the environment for validation.