Understanding Journal Mode in Real-Time Replication

In VMware environments, replication can be implemented using one of two modes for handling I/O changes: snapshots or journaling. Each approach has advantages and disadvantages. The key differences lie in how they manage data consistency, change tracking and recovery.

With the snapshot approach, VM replication occurs at scheduled intervals, so any changes made after the last snapshot (taken for VM replication) and before a VM failure can be lost. In contrast, the journal mode captures every write operation in real time and stores it in a journal, enabling recovery to any point in time. This mode is used for real-time replication to minimize data loss for critical workloads. This blog post explains how the journal mode works for real-time replication of VMware virtual machines.

NAKIVO for VMware Replication

Efficient replication of VMware vSphere VMs onsite or offsite. Instant automated failover for stronger resilience to incidents. High availability and low RTOs.

DISCOVER SOLUTION

What Is Journal Mode in Real-Time Replication?

The journal mode for real-time replication is a method that tracks and logs every change made to the primary VM in a continuous stream or near real-time manner. The logged changes recorded in the journal are applied to the VM replica incrementally, which provides the ability to perform VM recovery using the replica to specific points in time. Real-time replication with journal mode is used to ensure zero or near-zero data loss. Real-time replication is also called continuous replication.

How Journal Mode Works

The working principle of the journal mode for real-time replication includes the following main stages:

• Change tracking. A journal is created to log all modifications to the primary VM’s state, including file system changes, application data updates and system configurations.
• Continuous replication. The I/O journal data is continuously applied to the replica VM, maintaining synchronization between the primary and the replica VM.
• Point-in-time recovery. The journal’s retention window enables recovery to any specific point in time within the window.

The journal mode for real-time replication in NAKIVO Backup & Replication is powered by a universal data protection solution NAKIVO Backup & Replication.

Records of input/output (I/O) operations are recorded on a per-disk basis for replicated VMs in NAKIVO Backup & Replication. A new journal extent is created in the I/O journal for every real-time replication run. There are internal logical parts of the journal extent that contain 256 MB of data, which are called frames. A journal extent can contain one or more frames. Frames can contain up to 65536 data blocks and each block contains 4096 bytes of data.

Key features of Journal Mode

The journal mode for real-time VM replication has the following key features:

• Continuous synchronization. The primary VM changes are continuously logged, ensuring near-real-time replication to the replica VM.
• Granular point-in-time recovery. The journal provides fine-grained recovery options, enabling restoration to any state within the journal’s retention window. This is useful for recovery after failures, including data corruption or accidental deletions.
• Flexible retention period. The storage duration of the journal logs depends on the system’s storage capacity and configuration. Shorter retention periods reduce storage needs but limit recovery points.

Journal Mode Benefits

Real-time replication with the journal mode provides the following benefits:

• Minimal data loss. Real-time replication with Journal mode allows you to achieve zero or near-zero recovery point objectives (RPO) due to continuous tracking and replication.
• Fine-grained recovery. The ability to recover to specific points in time offers flexibility for disaster recovery.
• Efficiency. Real-time replication with Journal mode replicates only changes rather than entire VM states or snapshots, reducing replication time and network load.
• Reduced downtime. In case of a failure, the ability to quickly recover to a recent state minimizes downtime.

Journal Mode vs Traditional Replication

Journal mode and traditional snapshot-based VM replication differ in terms of data loss in case of failure, system requirements, configuration complexity and use cases.

Traditional replication using snapshot mode operates by periodically creating snapshots to copy the VM state from the source VM to the replica VM. The minimum interval between snapshot creation in data protection applications can be a few minutes. This means that the data written to the original VM after creating a VM snapshot can be lost if the original VM fails. Snapshot-based replication is optimal for VMs when the recovery point objective (RPO) is not strict. The advantages of traditional snapshot-based VM replication are ease of configuration and light system requirements that make this method affordable for most organizations.

Journal mode used for real-time replication or continuous replication operates with a continuously updated journal and these updates are applied to the replica VM. The recovery point objective (RPO) for VMs with real-time replication using Journal mode can be near-zero or zero. The disadvantage of Journal mode is higher system requirements and more complex configuration.

How to Set Up Real-Time Replication Journal Mode in NAKIVO

First, you need to install the Journal Service and then you can configure its options to perform a real-time replication job for VMware VMs.

Journal service installation

Target ESXi hosts where VM replicas (made with real-time replication) are stored must have the Journal service installed. You can use the Transporter to install or configure the Journal service.

To install the real-time replication journal service automatically (starting with NAKIVO Backup & Replication v11.1), follow the steps below:

1. Complete the Real-Time Replication Job Wizard to create a new real-time VMware VM replication job.
2. After the Real-Time Replication Job Wizard completion:
   • The NAKIVO solution installs the journal service on the required transporter virtual appliances (VAs) that do not have the journal service.
     The journal service is installed on /opt/nakivo/journalservice with 755 permission and it runs under the “bhsvc” user (“bhsvc” group).
   • Journal services on multiple transporter VAs are installed simultaneously (up to 10 VAs at a time).
   • In case a target host has multiple transporter VAs, the product will use the first VA in the list to install the journal service.

To install the Journal service manually (available for all NAKIVO Backup & Replication versions), open the web interface of NAKIVO Backup & Replication, go to Settings > Nodes, click the Download icon and hit Journal service for Real Time Replication Installer.

The NAKIVO virtual appliance for VMware vSphere is based on Ubuntu Server and the journal service installer has the .sh file extension. The version number in the file name depends on your version and build of NAKIVO Backup & Replication. After the download, the installer appears in the recent downloads of your browser.

Journal settings for a replication job

When all requirements are met (such as ESXi cluster, I/O filter, etc.) and all components are configured in your environment, you can create a real-time replication job for VMware VMs.

The I/O journal settings for a real-time VMware replication job can be configured at the Retention step.

You can configure the following I/O Journal settings:

• Journal mode:
  • Rollback journal. New data changes are saved directly to the VM replica. Old data in the VM replica is saved to the journal. Old data is removed from the journal according to the journal settings, such as the history limit.
  • Roll forward journal. New data changes are saved to the I/O journal. Old data is merged to the VM replica based on the settings.
• Journal history limit. Optionally, you can set a limit for journal history. The range is between 1 hour and 30 days.
• Journal size limit. You can set this limit between 1 GB and 20 TB. If the journal size limit is not set, it will be equal to the size of the datastore. If this datastore is larger than 20 TB, the journal size will be limited to 20 TB.

  

Real-time replication using journal mode provides near-zero data loss for business-critical VMware virtual machines. NAKIVO Backup & Replication supports both snapshot-based and real-time replication, giving you the flexibility to protect your data with traditional VM replication and real-time replication.

Try NAKIVO Backup & Replication

Get a free trial to explore all the solution’s data protection capabilities. 15 days for free. Zero feature or capacity limitations. No credit card required.

Try for Free