OCI Storage Gateway – For RMAN Backups

Chanaka Yapa Apr 27, 2023 5:30:00 AM

OCI Storage Gateway – For RMAN Backups – Eclipsys

25:20

Nov 2, 2023 12:30:00 PM

OCI DB Server – Troubleshoot Missing LVM – Eclipsys

Jan 6, 2023 11:15:00 AM

Backup Oracle Database to OCI Object Storage using Bck2Cloud – Eclipsys

Nov 22, 2022 10:00:00 AM

Configure Storage Domain (OLVM) – Part 4/6 – Eclipsys

oci

Acronym Definition

SG – Storage Gateway
OSS – Object Storage Service

Introduction

Syncing data from on-prem to the cloud is always challenging. Oracle introduced a storage gateway to address the primary use case of Object Storage Service (OSS) Storage Gateway to Data Transfer between On-premises applications and Oracle Cloud Infrastructure (OCI).

Apart from this, OSS Storage Gateway is a highly usable solution for Backups, Archival, and Disaster Recovery. Storage Gateway is the evolution of the Storage Software Appliance which was available with Oracle Cloud Infrastructure Classic (OCI-C) and on OCI it’s enhanced with file-to-object transparency and improved scale and performance.

In this article, I will cover the steps to configure the storage gateway.

Shared the Oracle documentation which covers all the pre-requisite steps for SG.

https://docs.oracle.com/en-us/iaas/Content/StorageGateway/Tasks/installingstoragegateway.htm

Pre-requisites for Storage Gateway

Two dual-core CPUs or better. Oracle recommends 4-core CPUs
Minimum memory requirements:

– 16 GB is required for any Storage Gateway file system

– 32 GB for File Systems with up to 50 Million Files
– 64 GB for File Systems with up to 100 Million Files

The recommended local storage disk size is 600 GB, which includes

– 500 GB for the File System Cache

– 80 GB for Metadata Storage

– 20 GB for Log Storage

Note: Oracle Linux 8 is not supported. The Storage Gateway installation software automatically installs Docker and the NFS protocol.

Setup the VM for Installation

As per the above-mentioned requirements set up the VM.

Figure 1: Elaborates the file system of the VM

[opc@sw-gatewat-poc-01 ~]$ df -h
Filesystem                                 Size  Used Avail Use% Mounted on
devtmpfs                                    16G     0   16G   0% /dev
tmpfs                                       16G     0   16G   0% /dev/shm
tmpfs                                       16G  8.8M   16G   1% /run
tmpfs                                       16G     0   16G   0% /sys/fs/cgroup
/dev/sda3                                   39G   11G   28G  28% /
/dev/sda1                                  200M  7.4M  193M   4% /boot/efi
/dev/mapper/sgw--cache--vg-sgw--cache--lv  100G   33M  100G   1% /ocisg/metadata
/dev/mapper/sgw--vg--data-sgw--data--lv    512G   33M  512G   1% /ocisg/cache
/dev/mapper/sgw--log--vg-sgw--log--lv       50G   33M   50G   1% /ocisg/log
tmpfs                                      3.2G     0  3.2G   0% /run/user/0
tmpfs                                      3.2G     0  3.2G   0% /run/user/994
tmpfs                                      3.2G     0  3.2G   0% /run/user/1000
[opc@sw-gatewat-poc-01 ~]$

SG software can be downloaded from the below-mentioned link: https://www.oracle.com/downloads/cloud/oci-storage-gateway-downloads.html

Copy the downloaded file to the newly created server and extract the installation and execute the ocisg-install.sh to start the installation. This will create a docker container and provide you URL for storage gateway configuration and display nfs port for mounting.

Sample Installation Log

[opc@sw-gatewat-poc-01 ocisg-1.4]$ ./ocisg-install.sh
*************************************
Install must be run with sudo or as root user
*************************************
[opc@sw-gatewat-poc-01 ocisg-1.4]$ sudo ./ocisg-install.sh

*************************************
Imported temporary env vars from opc to this install session
*************************************
Checking that docker is installed and using the correct version
Found docker version Docker version 19.03.11-ol, build 9bb540d
WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release.
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
         Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release.
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
         Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.

*************************************
Checking host prerequisites
*************************************

Detected linux operating system
Checking kernel version
Found kernel version 5.4.17-2136.314.6.2.el7uek.x86_64
Checking NFS version
Found NFS version 4

*************************************
All prerequisites have been met
*************************************


*************************************
Begin installation
*************************************

Installing to destination /opt/ocisg/
Previous install detected, checking version
OCISG version 1.4 is already installed.
Are you sure you want reinstall?  WARNING: This will remove all previously mapped data volumes. (y/N) y
Stopping OCISG
Cleaning up
Installing to destination /opt/ocisg/
Copied install scripts
Copied OCISG image
Starting configuration script

Enter the path for OCISG file system cache : /ocisg/cache

Enter the path for OCISG metadata storage : /ocisg/metadata

Enter the path for OCISG log storage : /ocisg/log
Writing configuration
Importing image
93076e6b905d: Loading layer [==================================================>]  235.2MB/235.2MB
d17d69a27f00: Loading layer [==================================================>]  8.192kB/8.192kB
619970d251a2: Loading layer [==================================================>]  8.192kB/8.192kB
bcb436c183d1: Loading layer [==================================================>]  307.2MB/307.2MB
e4be562dc80c: Loading layer [==================================================>]  1.891MB/1.891MB
5c1f1cdc83c6: Loading layer [==================================================>]  3.072kB/3.072kB
6ca724698b2c: Loading layer [==================================================>]  3.584kB/3.584kB
a594b6d55852: Loading layer [==================================================>]  3.072kB/3.072kB
517390192e6e: Loading layer [==================================================>]  4.096kB/4.096kB
257691076299: Loading layer [==================================================>]  1.477GB/1.477GB
197893d934e3: Loading layer [==================================================>]  3.072kB/3.072kB
9aa636049fe7: Loading layer [==================================================>]  3.072kB/3.072kB
cc0bed915e96: Loading layer [==================================================>]  63.91MB/63.91MB
c4e8e14ad37c: Loading layer [==================================================>]  227.9MB/227.9MB
e48596a66911: Loading layer [==================================================>]   2.56kB/2.56kB
e1fac4243073: Loading layer [==================================================>]  3.584kB/3.584kB
9a671469b9d3: Loading layer [==================================================>]  34.49MB/34.49MB
e02426d58273: Loading layer [==================================================>]  75.39MB/75.39MB
2c0c24ae8689: Loading layer [==================================================>]  3.903MB/3.903MB
25ca008cdb97: Loading layer [==================================================>]  102.9kB/102.9kB
2351114749e3: Loading layer [==================================================>]  25.09kB/25.09kB
17f4ccee9d77: Loading layer [==================================================>]  4.096kB/4.096kB
a905489f8786: Loading layer [==================================================>]  4.096kB/4.096kB
ed0ab8967a40: Loading layer [==================================================>]   7.68kB/7.68kB
2fe7f76a8bb1: Loading layer [==================================================>]   1.67MB/1.67MB
e624258f4755: Loading layer [==================================================>]  10.86MB/10.86MB
a227a04bcc2f: Loading layer [==================================================>]  2.793MB/2.793MB
0ad062f0009b: Loading layer [==================================================>]  1.168MB/1.168MB
a3835d50ca3b: Loading layer [==================================================>]  3.584kB/3.584kB
a3c9fbd174c7: Loading layer [==================================================>]  3.584kB/3.584kB
f634e79112d6: Loading layer [==================================================>]  3.584kB/3.584kB
3107ff305045: Loading layer [==================================================>]   2.56kB/2.56kB
1f98d0b8baab: Loading layer [==================================================>]   7.68kB/7.68kB
725be63a5e98: Loading layer [==================================================>]   7.68kB/7.68kB
b48f1da08dae: Loading layer [==================================================>]   7.68kB/7.68kB
8bd3f88a6026: Loading layer [==================================================>]   7.68kB/7.68kB
03bb5b8d0700: Loading layer [==================================================>]  3.584kB/3.584kB
bd7450835c5f: Loading layer [==================================================>]  205.8kB/205.8kB
2187992728c8: Loading layer [==================================================>]  3.072kB/3.072kB
caee8ec3f4ed: Loading layer [==================================================>]  17.41kB/17.41kB
27f37a80cc79: Loading layer [==================================================>]  3.584kB/3.584kB
b8b1163911c4: Loading layer [==================================================>]  3.584kB/3.584kB
1bf1d8f75b88: Loading layer [==================================================>]  12.29kB/12.29kB
2f5bbeea865e: Loading layer [==================================================>]  9.728kB/9.728kB
f79c38ced372: Loading layer [==================================================>]  9.728kB/9.728kB
7a9683b765b6: Loading layer [==================================================>]  3.584kB/3.584kB
7810ff2c6ef2: Loading layer [==================================================>]  4.096kB/4.096kB
63c86c31894d: Loading layer [==================================================>]  4.096kB/4.096kB
419197a73f06: Loading layer [==================================================>]  4.608kB/4.608kB
695501c9fef1: Loading layer [==================================================>]  4.608kB/4.608kB
a1a0023cb6e8: Loading layer [==================================================>]  4.096kB/4.096kB
c3b9822d7246: Loading layer [==================================================>]  4.608kB/4.608kB
40bdb7fd38c5: Loading layer [==================================================>]   7.68kB/7.68kB
7515b9a5b3d3: Loading layer [==================================================>]  6.656kB/6.656kB
2cd8ec8eeab8: Loading layer [==================================================>]  6.656kB/6.656kB
1eaee59b8d2a: Loading layer [==================================================>]   5.12kB/5.12kB
a94526c34a72: Loading layer [==================================================>]  3.584kB/3.584kB
f2c0b9399083: Loading layer [==================================================>]  12.29kB/12.29kB
027658e7b6fc: Loading layer [==================================================>]  6.144kB/6.144kB
fa6ac8c3e10e: Loading layer [==================================================>]  4.608kB/4.608kB
34a0282a5540: Loading layer [==================================================>]  69.68MB/69.68MB
ffa4e11cc6af: Loading layer [==================================================>]  90.74MB/90.74MB
89af104bbefc: Loading layer [==================================================>]  2.048kB/2.048kB
5ec2372a7d0e: Loading layer [==================================================>]  4.997MB/4.997MB
Loaded image: ocisg:1.4
Loaded image: oraclelinux:7.3
Creating OCISG Volume with args -v /cloudsync/:/cloudsync/:shared -v /ocisg/cache:/usr/share/oracle/ -v /ocisg/metadata:/usr/share/oracle/system/ -v /ocisg/log:/var/log/gateway
Applying configuration file to container
Starting OCISG [ocisg:1.4]
Setting up config file port with nfs
Setting up config file port with admin
Setting up config file port with rest
Management Console: https://sw-gatewat-poc-01:32770
If you have already configured an OCISG FileSystem via the Management Console,
you can access the NFS share using the following port.

NFS Port: 32771

Example: mount -t nfs -o vers=4,port=32771 sw-gatewat-poc-01:/OCISG FileSystem name /local_mount_point
[opc@sw-gatewat-poc-01 ocisg-1.4]$

Use this URL to configure object storage: https://sw-gatewat-poc-01:32770

oci2

Figure 1: Storage Gateway Configuration Page

To configure this you need to feed under mentioned detail

Compartment OCID
Tenant OCID
User OCID
Public Key’s Fingerprint
Private Key
Private Key Passphrase

This information can be gathered from login to your profile. Now select a resource (API Key) and generate an API key.

Download the generated private key and public key, or else you can copy your own key.

oci3

Figure 2: Add API Key

Now view the key details, this will help you to gather all other above-mentioned details such as Compartment OCID, Tenant OCID and etc.

oci4

Figure 3: API Key Details

Then select “Create a file System” on the storage gateway URL, which will navigate you to Figure 4.

Feed all the required detail which we gather using API details.

oci5

Figure 4: Create a File System Page

This will show it looks once you add and create the file system. It should be green in color to access the file system.

oci6

Figure 5: Storage Gateway File System Configuration

Now the major configuration is complete, the next step is to add an ingress rule to access the storage nfs port and execute the below-mentioned commands to mount the NFS share.

oci7

Figure 6: Ingress Firewall Rule.

First, I tested this mount on the storage gateway server.

[root@sw-gatewat-poc-01 ~]# sudo mount -t nfs -o vers=4,port=32770 10.0.0.212:/OBJ-POC /obj_bucket_content

Expected output

[root@sw-gatewat-poc-01 ~]# df -h
Filesystem                                 Size  Used Avail Use% Mounted on
devtmpfs                                    16G     0   16G   0% /dev
tmpfs                                       16G     0   16G   0% /dev/shm
tmpfs                                       16G  457M   16G   3% /run
tmpfs                                       16G     0   16G   0% /sys/fs/cgroup
/dev/sda3                                   39G   12G   28G  30% /
/dev/sda1                                  200M  7.4M  193M   4% /boot/efi
tmpfs                                      3.2G     0  3.2G   0% /run/user/0
/dev/mapper/sgw--vg--data-sgw--data--lv    512G  1.1G  511G   1% /ocisg/cache
/dev/mapper/sgw--cache--vg-sgw--cache--lv  100G   33M  100G   1% /ocisg/metadata
/dev/mapper/sgw--log--vg-sgw--log--lv       50G   33M   50G   1% /ocisg/log
/dev/dm-4                                   10G  2.0G  8.1G  20% /var/lib/docker/devicemapper/mnt/5110d06d9a28da86d250bb89769453f7bcdc0a70cffe2888492f7a8686b4c1cc
tmpfs                                      3.2G     0  3.2G   0% /run/user/994
tmpfs                                      3.2G     0  3.2G   0% /run/user/1000
10.0.0.212:/OBJ-POC                        8.0E     0  8.0E   0% /obj_bucket_content
[root@sw-gatewat-poc-01 ~]#

Now test this mount option on the database server.

[root@dbsdpl21 /]# mkdir BACKUP
[root@dbsdpl21 /]# mount -t nfs -o vers=4,port=32770 10.0.0.212:/OBJ-POC /BACKUP/
[root@dbsdpl21 /]# df -h
Filesystem                           Size  Used Avail Use% Mounted on
devtmpfs                              30G     0   30G   0% /dev
tmpfs                                 30G  1.4G   29G   5% /dev/shm
tmpfs                                 30G  593M   29G   2% /run
tmpfs                                 30G     0   30G   0% /sys/fs/cgroup
/dev/mapper/VolGroupSys0-LogVolRoot   45G  8.7G   34G  21% /
/dev/sda2                            1.4G   99M  1.2G   8% /boot
/dev/sda1                            486M  7.5M  478M   2% /boot/efi
/dev/sdi                             197G   33G  155G  18% /u01
tmpfs                                5.9G     0  5.9G   0% /run/user/102
/dev/asm/commonstore-484             5.0G  656M  4.4G  13% /opt/oracle/dcs/commonstore
tmpfs                                5.9G     0  5.9G   0% /run/user/54322
tmpfs                                5.9G     0  5.9G   0% /run/user/101
tmpfs                                5.9G     0  5.9G   0% /run/user/0
10.0.0.212:/OBJ-POC                  8.0E     0  8.0E   0% /BACKUP           -- Mounted storage 
[root@dbsdpl21 /]#

Config NFS Mount Point for RMAN Backup

As the object storage is a mount point via nfs protocol, you can configure this mount point as the default disk location. The backup will store on OCI object storage.

RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';

using target database control file instead of recovery catalog
new RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT   '/BACKUP/PWHSE01/%U';
new RMAN configuration parameters are successfully stored

Backup Database

Now you can allocate a channel to nfs share and back up the database.

RMAN> run
2> {
3> ALLOCATE CHANNEL CH1 DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';
4> ALLOCATE CHANNEL CH2 DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';
5> ALLOCATE CHANNEL CH3 DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';
6> ALLOCATE CHANNEL CH4 DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';
7> ALLOCATE CHANNEL CH5 DEVICE TYPE DISK FORMAT '/BACKUP/PWHSE01/%U';
8> BACKUP DATABASE PLUS ARCHIVELOG;
9> RELEASE channel c1;
10> RELEASE channel c2;
11> RELEASE channel c3;
12> RELEASE channel c4;
13> RELEASE channel c5;
14> }
15>
using target database control file instead of recovery catalog
allocated channel: CH1
channel CH1: SID=328 instance=PWSH011 device type=DISK

allocated channel: CH2
channel CH2: SID=629 instance=PWSH011 device type=DISK

allocated channel: CH3
channel CH3: SID=628 instance=PWSH011 device type=DISK

allocated channel: CH4
channel CH4: SID=1089 instance=PWSH011 device type=DISK

allocated channel: CH5
channel CH5: SID=17 instance=PWSH011 device type=DISK


Starting backup at 13-APR-23
current log archived
channel CH1: starting archived log backup set
channel CH1: specifying archived log(s) in backup set
input archived log thread=2 sequence=163 RECID=321 STAMP=1086026489
input archived log thread=1 sequence=159 RECID=323 STAMP=1086030087
input archived log thread=2 sequence=164 RECID=322 STAMP=1086030086
input archived log thread=2 sequence=165 RECID=325 STAMP=1086033686
input archived log thread=1 sequence=160 RECID=324 STAMP=1086033686
input archived log thread=1 sequence=161 RECID=326 STAMP=1086036200
input archived log thread=2 sequence=166 RECID=327 STAMP=1086036200
input archived log thread=1 sequence=162 RECID=328 STAMP=1086036323
input archived log thread=2 sequence=167 RECID=329 STAMP=1086036323

List database backup to validate

List of Backups
===============
Key     TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
3425    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3426    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3427    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3428    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3429    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3430    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3431    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T163159
3432    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3433    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3434    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3435    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3436    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3437    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3438    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3439    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3440    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3441    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3442    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T171752
3443    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T174938
3444    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T174938
3445    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T174938
3446    B  A  A DISK        13-APR-23       1       1       NO         TAG20230413T174938
3447    B  F  A DISK        13-APR-23       1       1       NO         TAG20230413T180124

Verification

To validate that backups are stored in object storage, navigate to the object storage bucket folder from OCI URL.

oci8

Figure 6: Validate backups are in object storage

Conclusion

A storage gateway is a great way to extend to capabilities of object storage. This easy the mounting of object storage to any VM, because all system engineers and database engines are comfortable with it NFS.

Even this can be used to copy database backup to the different regions without making any major configuration. As a disaster recovery method, we can replicate object storage content.

Nov 2, 2023 12:30:00 PM

OCI DB Server – Troubleshoot Missing LVM – Eclipsys

Jan 6, 2023 11:15:00 AM

Backup Oracle Database to OCI Object Storage using Bck2Cloud – Eclipsys

Nov 22, 2022 10:00:00 AM

Configure Storage Domain (OLVM) – Part 4/6 – Eclipsys

Azure VM Selection Made Easy: A Script Identifying Best Constrained CPU VMs for High Memory/Low CPU Workload

ExaCC: Create a VM Cluster Network using OCI CLI

Oracle Managed Services

Oracle Engineered Systems

Oracle Cloud

Oracle Licensing

Development Services

Oracle Database

Oracle Systems Portfolio

Application Services

Other Data Services

Eclipsys Joins A Brand With Multi-National Reach Following Acquisition By DSP UK

Resources

Life at Eclipsys

Getting to know us

OCI Storage Gateway – For RMAN Backups

Contents

Introduction

Setup the VM for Installation

Sample Installation Log

Config NFS Mount Point for RMAN Backup

Backup Database

Verification

Conclusion

Leave a Comment

Oracle Managed Services

Oracle Engineered Systems

Oracle Cloud

Oracle Licensing

Development Services

Oracle Database

Oracle Systems Portfolio

Application Services

Other Data Services

Eclipsys Joins A Brand With Multi-National Reach Following Acquisition By DSP UK

Resources

Life at Eclipsys

Getting to know us

OCI Storage Gateway – For RMAN Backups

Contents

Introduction

Setup the VM for Installation

Sample Installation Log

Login

Config NFS Mount Point for RMAN Backup

Backup Database

Verification

Conclusion

Leave a Comment