Introduction

Hope previous blog posts were useful in addressing ODA upgrade issues for the virtualization platforms. If you are planning on the 19.8 journey please read the below-mentioned article to ease the 19.8 journey.

 

Article about the upgrade and issues: 

 

19.13 will be the last upgrade for the virtualized platform, After that oracle going to discontinue OVM and they are moving to KVM-based virtualization. This is going to be a game changer for ODA for performance.

To be compliant with an Oracle license is very important. When configuring virtualized ODA set up the core as per the purchased license core count. 

If you misconfigured this there is a way to correct this. But you need to take complete downtime to perform this activity. 

In this article, I will illustrate how to adjust CPU core count in virtualized ODA.

Oracle documentation: https://docs.oracle.com/cd/E75549_01/doc.121/e74838/GUID-98E3071C-8278-420D-86F5-72E3B950918E.htm#CMTAR-GUID-98E3071C-8278-420D-86F5-72E3B950918

 

Adjust Core Count 

The ODA-BASE core count can be adjusted via dom0, Before making any changes make sure to take a backup of the ODA-BASE. For this CPU downscale, we need complete downtime. 

Pre-requisites Steps

  • Gather CPU 
  • Shut down all the VMS
  • Shut down all the repositories
  • Shutdown oda-base
  • Take the backup of the ODA-BASE

 

Steps to Perform

Adjust the CPU core count for the ODA-BASE ( This will change the CPU core count on both nodes)

 

Startup ODA-BASE

Validate ODA-BASE running services. ( Oracle cluster and database services)

 

CPU Core Count Configuration

Note: Make sure the ODA-BASE is down for both nodes before performing this activity

 

Validate the ODA-BASE Status

First, gather the ODA-BASE details to note down the current CPU core count

[root@ecl-oda-DOM0-1 ~]# oakcli show oda_base
ODA base domain
ODA base CPU cores      :12
ODA base domain memory  :80
ODA base template       :/OVS/NEVER_DELETE_12.1.2.12.tar.gz
ODA base vlans          :['net1', 'net2', 'vbr1']
ODA base current status :Running

[root@ecl-oda-DOM0-1 ~]# oakcli show oda_base
ODA base domain
ODA base CPU cores      :12
ODA base domain memory  :80
ODA base template       :/OVS/NEVER_DELETE_12.1.2.12.tar.gz
ODA base vlans          :['net1', 'net2', 'vbr1']
ODA base current status :Stopped
[root@ecl-oda-DOM0-1 ~]#

As per this example, I’m reducing the ODA-BASE CPU core count from 12 to 10

Below mentioned command output shows the 

[root@ecl-oda-DOM0-1 Repo_backup]# oakcli configure oda_base
Core Licensing Options:
        1. 2 CPU Cores
        2. 4 CPU Cores
        3. 6 CPU Cores
        4. 8 CPU Cores
        5. 10 CPU Cores
        6. 12 CPU Cores
        7. 14 CPU Cores
        8. 16 CPU Cores
        9. 18 CPU Cores
        10. 20 CPU Cores
        11. 22 CPU Cores
        12. 24 CPU Cores
        13. 26 CPU Cores
        14. 28 CPU Cores
        15. 30 CPU Cores
        16. 32 CPU Cores
        17. 34 CPU Cores
        18. 36 CPU Cores
        Current CPU Cores       :12
        Selection[1 .. 18](default 36 CPU Cores)        : 5
        ODA base domain memory in GB(min 16, max 491)(Current Memory 80G)[default 160]  : 80G
WARNING: Please enter a valid option for memory size
        ODA base domain memory in GB(min 16, max 491)(Current Memory 80G)[default 160]  : 80
Additional vlan networks to be assigned to oda_base ? (y/n) [n]:
Vlan network to be removed from oda_base ? (y/n) [n]:
INFO: Configure VNC password for oda_base
Please input your password:
ERROR: Invalid password, password should have uppercase lowercase and special characters and numbers
ERROR: password length should be longer than 8
ERROR: please enter a valid password
Please input your password:
Please confirm your password:
INFO: Node 0:Configured oda base pool
INFO: Node 1:Configured oda base pool
INFO: Node 0:ODA Base configured with new memory
INFO: Node 0:ODA Base configured with new vcpus
INFO: Node 0:ODA Base configured with new VNC passwd
INFO: Changes will be incorporated after the domain is restarted on Node 0.
INFO: Node 1:ODA Base configured with new memory
INFO: Node 1:ODA Base configured with new vcpus
INFO: Node 1:ODA Base configured with new VNC passwd
INFO: Changes will be incorporated after the domain is restarted on Node 1.
INFO: Updating /etc/sysctl.conf in oda_base domain with parameter "vm.nr_hugepages=21520"
ERROR: Odabase Agent on node 0 is down
INFO: Please update /etc/sysctl.conf in oda_base domain on node 0 with parameter "vm.nr_hugepages=21520"
ERROR: Odabase Agent on node 1 is down
INFO: Please update /etc/sysctl.conf in oda_base domain on node 1 with parameter "vm.nr_hugepages=21520"
INFO: Updating /etc/security/limits.conf in oda_base domain with new memlock value 60000000
ERROR: Odabase Agent on node 0 is down
INFO: Please update /etc/security/limits.conf in oda_base domain on node 0 with new memlock value 60000000
ERROR: Odabase Agent on node 1 is down
INFO: Please update /etc/security/limits.conf in oda_base domain on node 1 with new memlock value 60000000
You have new mail in /var/spool/mail/root
[root@ecl-oda-DOM0-1 Repo_backup]#

 

Validate Core Count

After changing run the show oda_base command to validate the cpu change.

[root@ecl-oda-DOM0-0 Repo_backup]# oakcli show oda_base
ODA base domain
ODA base CPU cores      :10
ODA base domain memory  :80
ODA base template       :/OVS/NEVER_DELETE_12.1.2.12.tar.gz
ODA base vlans          :['net1', 'net2', 'vbr1']
ODA base current status :Stopped
[root@ecl-oda-DOM0-0 Repo_backup]# oakcli start oda_base


[root@ecl-oda-DOM0-1 Repo_backup]# oakcli start oda_base
INFO: Starting ODA base domain...
INFO: Started ODA base domain
[root@ecl-oda-DOM0-1 Repo_backup]# oakcli show oda_base
ODA base domain
ODA base CPU cores      :10
ODA base domain memory  :80
ODA base template       :/OVS/NEVER_DELETE_12.1.2.12.tar.gz
ODA base vlans          :['net1', 'net2', 'vbr1']
ODA base current status :Running
[root@ecl-oda-DOM0-1 Repo_backup]#

 

Validate Settings via Dom0 Log

I would recommend checking the dom0 oakcli logs. Which gives you a clear understanding of the CPU change.

2023-07-12 15:45:37,800 [Cmd_EnvId] [Thread-4021] [odaBaseActions] [INFO] [176] Getting existing vlan list for ODA BASE
2023-07-12 15:45:37,801 [Cmd_EnvId] [Thread-4021] [odaBaseActions] [INFO] [186] Existing Vlan List ['net1', 'net2', 'vbr1']
2023-07-12 15:46:00,194 [Cmd_EnvId] [Thread-4023] [odaBaseActions] [INFO] [176] Getting existing vlan list for ODA BASE
2023-07-12 15:46:00,194 [Cmd_EnvId] [Thread-4023] [odaBaseActions] [INFO] [186] Existing Vlan List ['net1', 'net2', 'vbr1']
2023-07-12 15:47:56,907 [Cmd_EnvId] [Thread-4024] [odaBaseActions] [INFO] [130] ODA Base pool being configured with 20 cpus
2023-07-12 15:47:56,907 [Cmd_EnvId] [Thread-4024] [cpupoolactions] [DEBUG] [90] Checking if the pool odaBaseCpuPool exists
2023-07-12 15:47:56,908 [Cmd_EnvId] [Thread-4024] [cpupoolactions] [DEBUG] [93] Updating cpu pool odaBaseCpuPool to 20 cpus
2023-07-12 15:47:56,908 [Cmd_EnvId] [Thread-4024] [oakCpuPoolManage] [DEBUG] [75] Trying to set cpu pool odaBaseCpuPool to 20 cpus
2023-07-12 15:47:56,909 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [1327] getNumCpusDom0 called
2023-07-12 15:47:56,909 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [723] Executing command xm vcpu-list |grep Domain-0 | wc -l
2023-07-12 15:47:57,179 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [1334] Number of cpus pinned to dom0 is 20
2023-07-12 15:47:57,180 [Cmd_EnvId] [Thread-4024] [oakCpuPoolManage] [DEBUG] [81] Number of cpus allocated to dom0 is 20
2023-07-12 15:47:57,180 [Cmd_EnvId] [Thread-4024] [cpupoolDb] [DEBUG] [77] cpu 43 assigned to default-unpinned-pool pool
2023-07-12 15:47:57,180 [Cmd_EnvId] [Thread-4024] [cpupoolDb] [DEBUG] [77] cpu 42 assigned to default-unpinned-pool pool
2023-07-12 15:47:57,181 [Cmd_EnvId] [Thread-4024] [cpupoolDb] [DEBUG] [77] cpu 41 assigned to default-unpinned-pool pool
2023-07-12 15:47:57,181 [Cmd_EnvId] [Thread-4024] [cpupoolDb] [DEBUG] [77] cpu 40 assigned to default-unpinned-pool pool
2023-07-12 15:47:57,181 [Cmd_EnvId] [Thread-4024] [oakCpuPoolManage] [DEBUG] [124] Completed allocation of cpus
2023-07-12 15:47:57,182 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [181] Created xml string 0Cpupool configured
2023-07-12 15:47:57,184 [Cmd_EnvId] [Thread-4024] [cpupoolactions] [DEBUG] [53] Cpulist string for cpu pool odaBaseCpuPool is 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39
2023-07-12 15:47:57,184 [Cmd_EnvId] [Thread-4024] [cpupoolactions] [DEBUG] [62] Updating vmcfg file of VM oakDom1 with cpus: 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39
2023-07-12 15:47:57,184 [Cmd_EnvId] [Thread-4024] [odaBaseActions] [INFO] [101] ODA Base configure operation called
2023-07-12 15:47:57,184 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [567] Updating cfg params values
2023-07-12 15:47:57,185 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [540] Post conversion, the configuration parameter updated to {'cpus': '20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39'}
2023-07-12 15:47:57,185 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [578] Updating parameter cpus to 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39
2023-07-12 15:47:57,185 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [624] Writing vmcfg to /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg
2023-07-12 15:47:57,185 [Cmd_EnvId] [Thread-4024] [agentutils] [DEBUG] [181] Created xml string 0ODA Base configured vm
2023-07-12 15:48:07,223 [Check_Shared_Repo] [MainThread] [repoactions] [INFO] [162] Checking for shared repos
2023-07-12 15:48:07,224 [Cmd_EnvId] [Thread-4025] [odaBaseActions] [INFO] [56] Configuring memory for ODA BASE
2023-07-12 15:48:07,239 [Cmd_EnvId] [Thread-4025] [agentutils] [DEBUG] [488] Updating the memory parameter to 81920
2023-07-12 15:48:07,240 [Cmd_EnvId] [Thread-4025] [agentutils] [DEBUG] [624] Writing vmcfg to /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg
2023-07-12 15:48:10,250 [Check_Shared_Repo] [MainThread] [repoactions] [ERROR] [182] Error encountered while checking for shared repos: OAKERR:7084The HAVIP 192.168.19.21 is not pingable
2023-07-12 15:48:10,251 [Check_Shared_Repo] [MainThread] [agentutils] [DEBUG] [181] Created xml string 0OAKERR:7084The HAVIP 192.168.19.21 is not pingable
2023-07-12 15:48:17,249 [Cmd_EnvId] [Thread-4026] [odaBaseActions] [INFO] [71] Configuring vcpus for ODA BASE
2023-07-12 15:48:17,249 [Cmd_EnvId] [Thread-4026] [agentutils] [DEBUG] [524] Updating the vcpus parameter to 20
2023-07-12 15:48:17,249 [Cmd_EnvId] [Thread-4026] [agentutils] [DEBUG] [624] Writing vmcfg to /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg
2023-07-12 15:48:27,261 [Cmd_EnvId] [Thread-4027] [odaBaseActions] [INFO] [86] Configuring VNC passwd for ODA BASE
2023-07-12 15:48:27,261 [Cmd_EnvId] [Thread-4027] [agentutils] [DEBUG] [624] Writing vmcfg to /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/vm.cfg
2023-07-12 15:48:57,313 [Check_Shared_Repo] [MainThread] [repoactions] [INFO] [162] Checking for shared repos
2023-07-12 15:48:57,313 [Cmd_EnvId] [Thread-4030] [odaBaseActions] [INFO] [176] Getting existing vlan list for ODA BASE
2023-07-12 15:48:57,329 [Cmd_EnvId] [Thread-4030] [odaBaseActions] [INFO] [186] Existing Vlan List ['net1', 'net2', 'vbr1']
2023-07-12 15:49:00,340 [Check_Shared_Repo] [MainThread] [repoactions] [ERROR] [182] Error encountered while checking for shared repos: OAKERR:7084The HAVIP 192.168.19.21 is not pingable
2023-07-12 15:49:00,341 [Check_Shared_Repo] [MainThread] [agentutils] [DEBUG] [181] Created xml string 0OAKERR:7084The HAVIP 192.168.19.21 is not pingable
2023-07-12 15:54:47,554 [SYS_DISC_-108646_OvmType] [Thread-4034] [repoactions] [DEBUG] [74] show all repos called
2023-07-12 15:54:47,555 [SYS_DISC_-108646_OvmType] [Thread-4034] [agentutils] [INFO] [79] Re-initializing repos
2023-07-12 15:54:47,555 [SYS_DISC_-108646_OvmType] [Thread-4034] [oakvmagentxml] [DEBUG] [56] Initializing oakagentxml object for repo /OVS/Repositories/odarepo2
2023-07-12 15:54:47,556 [SYS_DISC_-108646_OvmType] [Thread-4034] [repoactions] [DEBUG] [82] writing repo xml for odarepo2 repository to /tmp/fileApuAH8 file
2023-07-12 15:54:47,559 [SYS_DISC_-108646_OvmType] [Thread-4034] [repoactions] [DEBUG] [87] removing temporary file /tmp/fileApuAH8
2023-07-12 15:54:57,572 [Check_Shared_Repo] [MainThread] [repoactions] [INFO] [162] Checking for shared repos
2023-07-12 15:54:57,572 [SYS_DISC_-108646_OvmType] [Thread-4035] [vmactions] [INFO] [348] vm discovery operation called for odarepo2 repo
2023-07-12 15:54:57,589 [SYS_DISC_-108646_OvmType] [Thread-4035] [agentutils] [INFO] [79] Re-initializing repos
2023-07-12 15:54:57,591 [SYS_DISC_-108646_OvmType] [Thread-4035] [oakvmagentxml] [DEBUG] [56] Initializing oakagentxml object for repo /OVS/Repositories/odarepo2
2023-07-12 15:55:00,602 [Check_Shared_Repo] [MainThread] [repoactions] [ERROR] [182] Error encountered while checking for shared repos: OAKERR:7084The HAVIP 192.168.19.21 is not pingable
2023-07-12 15:55:00,603 [Check_Shared_Repo] [MainThread] [agentutils] [DEBUG] [181] Created xml string

 

Conclusion

It’s very important to be compliant with an Oracle license. There is the possibility that you lose control of the number of databases due to the high demand for business needs. Having these core count numbers eases the planning of coming workloads. If anything beyond the cpu license take a small downtime window and adjust as per the licensing policy.