Introduction
In today’s data-driven era, businesses leverage data analysis to seize opportunities and gain a competitive edge. This makes it crucial for organizations to have a robust disaster recovery (DR) plan. Most organizations leverage cloud backup features to store their data securely. When backups are in the public cloud, it is essential to ensure they are not altered and can be retrieved even after a ransomware attack. Oracle Cloud Object Storage provides an excellent feature of object versioning to ensure backups are safe and can be restored at any time during ransomware attacks.
Here are several reasons why immutable backups are important:
Ransomware Protection:
Immutable backups protect against ransomware attacks by ensuring that backup data cannot be altered or deleted by unauthorized users or malicious software. Since immutable backups cannot be modified or deleted using standard backup management tools, they serve as a reliable recovery point in the event of a ransomware attack.
Data Integrity:
Immutable backups provide a tamper-evident data protection mechanism, ensuring that backup data remains unchanged and unaltered over time. This guarantees the integrity of backup data, making it suitable for compliance requirements and regulatory standards.
Compliance Requirements:
Many industries and organizations are subject to regulatory requirements that mandate the implementation of immutable backup solutions. Immutable backups help organizations comply with data protection regulations by ensuring the immutability and integrity of backup data.
Protection Against Insider Threats:
Immutable backups protect against insider threats and malicious actions taken by employees or privileged users. By preventing unauthorized modifications or deletions of backup data, immutable backups safeguard against internal security breaches and unauthorized data tampering.
Disaster Recovery:
Immutable backups serve as a reliable disaster recovery mechanism, enabling organizations to recover data in the event of data loss, corruption, or system failures. Since immutable backups cannot be altered, they provide a secure and trustworthy recovery point for restoring critical data and systems.
Retention and Legal Holds:
Immutable backups facilitate long-term data retention and legal holds by ensuring that backup data remains unchanged and immutable for a specified retention period. This is essential for organizations that need to retain data for compliance, legal, or archival purposes.
Data Resilience:
Immutable backups enhance data resilience by protecting backup data from accidental or intentional deletion, corruption, or tampering. By maintaining a secure and immutable copy of critical data, organizations can mitigate the risk of data loss and ensure business continuity.
In this article, I will elaborate on how to test object storage version control.
For testing purposes, I will create a tablespace called IMMUTABLE_TEST, a user named IMMUTABLE_USER, and a table named test_table, and insert one row into the table.
Next, I will back up this tablespace and remove the data file from the database. Using object storage versioning, we can restore the data file and validate the data.
Create Test User and Table
Sample Commands :
CREATE TABLESPACE IMMUTABLE_TEST DATAFILE SIZE 1G AUTOEXTEND ON NEXT 200M MAXSIZE UNLIMITED;
CREATE USER "IMMUTABLE_USER" PROFILE "DEFAULT" IDENTIFIED BY "******" DEFAULT TABLESPACE "IMMUTABLE_TEST" TEMPORARY TABLESPACE "TEMP" ACCOUNT UNLOCK;
ALTER USER IMMUTABLE_USER QUOTA UNLIMITED ON IMMUTABLE_TEST;
CREATE TABLE IMMUTABLE_USER.TEST_TABLE
(
id integer,
firstname varchar(10),
lastname varchar(10),
age integer,
income integer,
primary key (id)
);
INSERT INTO IMMUTABLE_USER.TEST_TABLE VALUES (10,'John','Smith', 22, 45000);
commit;
Validate Table
SQL> select * from IMMUTABLE_USER.TEST_TABLE;
ID FIRSTNAME LASTNAME AGE INCOME
---------- ---------- ---------- ---------- ----------
10 John Smith 22 45000
SQL>
Backup Immutable Tablespace
RMAN> run
{
allocate channel ch01 device type disk format '/rman_backups/PWHSE01/%d_IMMUTABLE_TEST_%M%D%Y_%d_%p_%s';
backup tablespace PWHSE01_PDB1:IMMUTABLE_TEST;
}2> 3> 4> 5>
allocated channel: ch01
channel ch01: SID=196 device type=DISK
Starting backup at 08-JUL-24
channel ch01: starting full datafile backup set
channel ch01: specifying datafile(s) in backup set
input datafile file number=00013 name=/u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/o1_mf_immutabl_m08ocgoj_.dbf
channel ch01: starting piece 1 at 08-JUL-24
channel ch01: finished piece 1 at 08-JUL-24
piece handle=/rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39 tag=TAG20240708T130843 comment=NONE
channel ch01: backup set complete, elapsed time: 00:00:01
Finished backup at 08-JUL-24
Starting Control File and SPFILE Autobackup at 08-JUL-24
piece handle=/u03/app/oracle/fast_recovery_area/PWHSE01_3C9_YYZ/autobackup/2024_07_08/o1_mf_s_1173791324_m8r78ww6_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 08-JUL-24
released channel: ch01
RMAN>
Verify Object Versioning is Enabled.
Before deleting anything from object storage, we need to ensure that object version control is enabled.
Figure 1: Verify object version control is enabled
As shown in Figure 2, validate the backup piece in object storage.
Figure 2: Validate backup
Assume a hacker gains access and removes or encrypts the backup from the object storage.
To simulate this, I will remove the backup from the object storage.
[oracle@dbsdpl55 PWHSE01]$ cd /rman_backups/PWHSE01/
[oracle@dbsdpl55 PWHSE01]$ ls -lrth
total 0
-rw-r----- 1 oracle oinstall 1.2M Jul 8 13:08 PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39
[oracle@dbsdpl55 PWHSE01]$ rm PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39
[oracle@dbsdpl55 PWHSE01]$ pwd
/rman_backups/PWHSE01
[oracle@dbsdpl55 PWHSE01]$ ls -lrth
total 0
[oracle@dbsdpl55 PWHSE01]$
Now enable show deleted object and check the deleted backup piece is there.
Figure 3: Enable show deleted backups
Now let’s retrieve the deleted backup piece.
Copy the deleted piece to object storage.
Figure 5: Copy deleted backup
You need to provide destination object storage to copy the deleted backup piece.
Figure 6: Provide destination object location.
Now let’s copy the backup piece backup object storage.
Figure 7: Copy worker status.
Let’s validate the retrieved backup piece.
Validate copy backup from OS level
[root@dbsdpl55 PWHSE01]# pwd
/rman_backups/PWHSE01
[root@dbsdpl55 PWHSE01]# ls -lrth
total 0
-rw-r----- 1 oracle oinstall 1.2M Jul 8 13:08 PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39-2024-07-08T17:12:09.158Z
[root@dbsdpl55 PWHSE01]#
Restore
To verify if we can restore from this backup piece, let’s move the current data file and attempt to restore the data file.
[root@dbsdpl55 datafile]# mv o1_mf_immutabl_m08ocgoj_.dbf o1_mf_immutabl_m08ocgoj_.dbf.old
[root@dbsdpl55 datafile]# ls -lrth
total 2.3G
-rw-r----- 1 oracle oinstall 158M May 8 01:00 o1_mf_temp_m06oys0x_.tmp
-rw-r----- 1 oracle oinstall 5.1M Jul 8 11:52 o1_mf_users_m06oypj2_.dbf
-rw-r----- 1 oracle oinstall 1.1G Jul 8 13:08 o1_mf_immutabl_m08ocgoj_.dbf.old
-rw-r----- 1 oracle oinstall 561M Jul 8 13:52 o1_mf_sysaux_m06oxpkf_.dbf
-rw-r----- 1 oracle oinstall 441M Jul 8 14:05 o1_mf_system_m06oy36x_.dbf
-rw-r----- 1 oracle oinstall 266M Jul 8 14:05 o1_mf_undotbs1_m06oyh67_.dbf
Restore Errors
We encountered the RMAN-06010 error because the database control file doesn’t recognize this backup piece. To resolve this issue, we need to catalog the backup piece.
RMAN> restore datafile '/u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/o1_mf_immutabl_m08ocgoj_.dbf ';
Starting restore at 08-JUL-24
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=193 device type=DISK
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 07/08/2024 14:21:38
RMAN-20201: datafile not found in the recovery catalog
RMAN-06010: error while looking up datafile: /u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/o1_mf_immutabl_m08ocgoj_.dbf
RMAN>
Catalog Backup
RMAN> catalog start with '/rman_backups/PWHSE01/';
searching for all files that match the pattern /rman_backups/PWHSE01/
List of Files Unknown to the Database
=====================================
File Name: /rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39-2024-07-08T17:12:09.158Z
Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done
List of Cataloged Files
=======================
File Name: /rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39-2024-07-08T17:12:09.158Z
RMAN>Now let’s try to perform the restored datafile again. As we deleted datafile from PDB we need to connect to PDB.
[oracle@dbsdpl55 ~]$ export ORACLE_PDB_SID=PWHSE01_PDB1
[oracle@dbsdpl55 ~]$ rman target /
Recovery Manager: Release 19.0.0.0.0 - Production on Mon Jul 8 14:33:51 2024
Version 19.22.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: PWHSE01:PWHSE01_PDB1 (DBID=3302517169)
RMAN> RUN {
ALTER DATABASE DATAFILE 13 OFFLINE;
RESTORE DATAFILE 13;
RECOVER DATAFILE 13;
ALTER DATABASE DATAFILE 13 ONLINE;
}2> 3> 4> 5> 6>
Statement processed
Starting restore at 08-JUL-24
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00013 to /u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/o1_mf_immutabl_m08ocgoj_.dbf
channel ORA_DISK_1: reading from backup piece /rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39
channel ORA_DISK_1: errors found reading piece handle=/rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39
channel ORA_DISK_1: failover to piece handle=/rman_backups/PWHSE01/PWHSE01_IMMUTABLE_TEST_07082024_PWHSE01_1_39-2024-07-08T17:12:09.158Z tag=TAG20240708T130843
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:15
Finished restore at 08-JUL-24
Starting recover at 08-JUL-24
using channel ORA_DISK_1
starting media recovery
media recovery complete, elapsed time: 00:00:00
Finished recover at 08-JUL-24
Statement processed
Validate the datafile after restoring and table
[oracle@dbsdpl55 ~]$ ls -lrth /u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/*immutabl*
-rw-r----- 1 oracle oinstall 1.1G Jul 8 14:34 /u02/app/oracle/oradata/PWHSE01_3c9_yyz/PWHSE01_3C9_YYZ/0F133E16B5531E5EE0630503F40A9588/datafile/o1_mf_immutabl_m8rd9g55_.dbf
[oracle@dbsdpl55 ~]$
SQL> select * from IMMUTABLE_USER.TEST_TABLE;
ID FIRSTNAME LASTNAME AGE INCOME
---------- ---------- ---------- ---------- ----------
10 John Smith 22 45000
SQL>Conclusion
Overall, immutable backups are essential for protecting data integrity, ensuring compliance with regulatory requirements, safeguarding against ransomware attacks, and enabling reliable disaster recovery. By implementing immutable backup solutions, organizations can enhance their data protection capabilities and mitigate various security risks associated with data management and storage.