April 22, 2019
Backup to Google Cloud with NAKIVO: Choose Your Method
In the modern digital world, having data backed up is the basis of success in business. No business infrastructure is completely immune from disasters, be they the result of human error or malice, or perhaps a force majeure, and your only way of guaranteeing protection of your valuable data if disaster occurs is with a reliable backup solution in place. At present, cloud environments are popular in large part due to their affordability, availability (in about 99.99% of cases), and convenience, making them especially useful tools for storing backups. The 3-2-1 backup rule recommends that you have at least three copies of data, two of which are stored on different forms of media, one of which is stored offsite - in the cloud (remote backup), for example.
Google Cloud Platform is one of the most popular and reliable cloud platforms, as it provides competitive pricing and a convenient user interface. Backup to Google Cloud is a highly practical means of saving and securing data, and can certainly be considered when composing a disaster recovery plan, business continuity plan and different site recovery scenarios.
NAKIVO Backup & Replication is a fast, reliable and affordable data protection solution that can back up data across your local infrastructure as well as perform backup to Google Cloud. Today’s blog post explores how you can carry out VM backup to Google Cloud with NAKIVO Backup & Replication using two methods.
Before you begin, make sure that a full solution of NAKIVO Backup & Replication (Director+Transporter) is installed on a physical machine, virtual machine or NAS device in your local infrastructure. Visit this page to read about installation details in full. Note that you can enable encryption for data transferred between Transporters which makes the process of backup to Google Cloud with NAKIVO Backup & Replication much more secure. Below, you can see a simplified scheme of the components used for the methods explained in this blog post.
Method 1 – Administer Backup to Google Cloud by Using a VM Disk Attached to a VM Instance
When using this first method, you must attach a persistent virtual disk whose size can be up to 64 TB to a Google VM instance running Linux, and create a backup repository on that disk.
If you wish to back up a VM to Google Cloud by using this method, perform the following actions:
- Create a VM instance running Linux with an additional virtual disk to be used as a backup repository
- Prepare the additional virtual disk for creating your backup repository
- Mount the virtual disk and edit /etc/fstab
- Install NAKIVO Backup & Replication Transporter on your Linux VM instance running in Google Cloud
- Configure a firewall in Google Cloud which allows for possible communication with the Transporter
- Add the remote Transporter to the configuration of NAKIVO Backup & Replication
- Create a new backup repository on the remote virtual disk attached to Linux running in Google Cloud
Preparing Google Cloud Environment
Open https://cloud.google.com/ in your web browser. You need to have a Google account and you must be logged in to go forward. Click Go to console. You need to activate your account to use Google Cloud Platform by entering the parameters of your bank card. Google provides a 1-year free trial and gives you $300 with which to try out Google Cloud services (valid at the time of this blog post’s writing - potentially subject to change in future). You can upgrade a trial account to a full account for using Google Сloud without any limitations.
Once you have configured your Google account for Google cloud services, go to the home page of the Google Cloud Platform management interface in your web browser on which you can create a new project. The system will ask you to create a new project if no projects have been created yet. In the current example, the name of the project is NAKIVO-test01.
Creating a new VM instance in Google Cloud
After you have created and selected your project on the Google Cloud Platform page, click Compute Engine in the left pane of the web interface and click VM instances in the opened menu. A VM instance in Google Cloud is also called a Google Cloud Instance.
If you have an account as explained in the current example, Google Cloud Platform will ask you to create a new VM instance. In order to create a new VM instance, simply click Create.
In the Create an instance screen, configure all needed options.
Enter the instance name, for example, g-transporter (the name must start with a lowercase letter followed by up to 62 lowercase letters, numbers or hyphens and cannot end with a hyphen). NAKIVO Transporter will be installed on this VM later.
Select region and zone. In this example, Europe-west4 (Netherlands) is used as the region and europe-west4-a is used as the zone. You can select the region that is nearest to your geographical location to minimize latency in your network connection.
Machine type. Customize your virtual machine (instance) configuration. According to the minimal requirements for installing NAKIVO Transporter, at least two CPU cores and 2 GB of RAM are needed. Click Customize and select 2vCPUs and 2 GB memory.
Boot disk. Debian GNU/Linux 9 is selected by default. Click the Change button and select Ubuntu 18.04 LTS Minimal with 10GB standard persistent disk. 10 GB is enough to run Linux (without GUI) and NAKIVO Transporter. Google Cloud Platform provides pre-configured Linux images to save time spent for deploying the operating system manually.
Access scopes. The Allow Full access to all cloud APIs option must be enabled.
Click to expand the Management, security, disks, section in order to add one more virtual disk to your VM instance in Google Cloud.
Hit the +Add new disk button to add the second virtual disk which will be used as a backup repository to store backups made by NAKIVO Backup & Replication. Configure the following settings:
Name. Enter the name of your virtual disk, for example, disk-1.
Description. Enter disk description (optional).
Type. Select Standard persistent disk as the disk type, as this disk type costs less than other faster disks (such as SSD) but is more than enough to store backups.
Source type. Select the Blank disk option.
Mode. Select the Read/write mode.
Deletion rule. Select Keep disk. This option saves your disk even if you delete your Google VM (instance). Later you can mount a virtual disk to another VM instance and attach a backup repository to the appropriate instance of NAKIVO Backup & Replication. Attaching and detaching backup repositories can be useful for site recovery scenarios.
Size. Define 200 GB or more for your virtual disk.
Other disk settings can be left by default.
Click Done to finish creating a new virtual disk.
Check the options for your new VM and click Create to create the VM instance in Google Cloud. Wait for a few seconds until the instance is created.
A new instance is powered on automatically. You can open the Linux console remotely by using SSH. Hit the SSH icon for your VM instance and click Open in browser window. Remember the external IP address of your Linux VM instance (22.214.171.124 in this example) that is displayed on the VM instances screen.
Now that you have created a new VM instance running Linux in Google Cloud and attached a second virtual disk, the second disk (200-GB disk in this example) must be prepared to be used in Linux. You need to create a partition, create a file system on that partition (format the partition) and mount said partition onto the appropriate directory.
Mount your virtual disk and prepare it for use as a backup repository
In the Linux console to which you have connected via SSH, you need to execute some commands as root. Enter the following command to get root privileges in the console:
Note: Press Ctrl+D if you need to exit the root mode in console.
First, list the disks attached to the Linux VM by using the fdisk -l command. The name of the second disk device in Linux is /dev/sdb in this case, and the first disk device is /dev/sda. Remember that partitions must not be edited while they are in use. Instead, you should create a new partition on your second (200-GB) disk with fdisk (by using the command mode of fdisk).
Type n to create a new partition table, and answer the questions provided by the console wizard of fdisk. Press Enter if you agree to use the default parameter.
- Select the partition type you require. Set the new partition as your primary partition. Press p – primary (default) then press Enter, or just press Enter to use the default parameter.
- Select the required partition number (the default is 1).
- Select the first sector (default 2048).
- Select the last sector. Use the default in order to create one partition per disk.
- You can see that you have created a new partition, number 1, type 'Linux' and sized at 200 GiB. This string tells you that the partition has been created successfully, but these changes are currently stored in memory. You will now need to write those changes to the disk.
Type w to write changes, then press Enter, after which the changes are applied to the disk, and a new partition is created.
Now you have to format the new partition with the ext4 file system.
Create the ext4 file system on your new partition (/dev/sdb1 is the name of the recently created partition) by using the following command:
Add the partition label, for example, nakivo200:
e2label /dev/sdb1 nakivo200
Check the labels of your disk partitions to make sure the proper label is applied for the partition you created before.
Create a new directory that will be used as a backup repository by NAKIVO Backup & Replication.
Mount the 200-GB /dev/sdb1 disk partition you previously created to the /mnt/nakivo-repo directory:
mount /dev/sdb1 /mnt/nakivo-repo
In this case, the partition has been mounted to the directory successfully, but will only remain mounted until you reboot the Linux machine. Now, we need to make Linux mount this partition automatically and permanently on each boot of the operating system by editing /etc/fstab.
Update the package list for your package manager to ensure that the package manager has got the greatest possible amount of information about available packages and their dependencies.
Install the vim text editor:
apt-get install vim
Edit /etc/fstab by using the following command:
As you can see, the partitions are mounted by partition labels in default on this Linux VM that is running in Google Cloud Platform.
Add the following string to your /etc/fstab file:
LABEL=nakivo200 /mnt/nakivo-repo ext4 defaults 0 0
Write the changes to the file and quit before rebooting your Linux VM instance.
After your Linux VM instance reboot, verify whether the partition that you will use as a backup repository was mounted automatically:
Installing a Transporter on the Linux VM running in Google Cloud
Download NAKIVO Backup & Replication from the official website if you have not done so already, selecting the installer for Linux and saving the file to any directory on your local machine (for example, C:\NAKIVO).
Now you need to transfer the NAKIVO Backup & Replication installer to your Google cloud instance running Linux. In the remote console window of your Linux which has been opened in your web browser, click the configuration icon in the right top corner. Click Upload file, select the installer file (NAKIVO_Backup_Replication_v8.1.0_Installer_TRIAL.sh in this case) saved on your local machine and click Open. Wait for the file to be copied onto your Linux VM instance running in Google Cloud.
By default, the file is copied to the home directory of your user on the Linux machine (a default user name is based on the name of your Google account).
Go to the home directory of your user (the name of the user is michael_nakivonbr in this example).
Make the installer file executable:
chmod +x ./NAKIVO_Backup_Replication_v8.1.0_Installer_TRIAL.sh
Run the installer:
Read the license agreement (press Enter to list/scroll the text) and type y if you agree with the terms of the agreement.
Press t to install the Transporter only on the remote Linux machine (which is running in Google Cloud).
Make sure that there are enough issuances of permission for NAKIVO Transporter to write data to the directory in use as a backup repository:
chown -R bhsvc:bhsvc /mnt/nakivo-repo/
chmod -R 0775 /mnt/nakivo-repo/
Configuring firewall in Google Cloud
Default firewall rules in Google Cloud don’t allow connections to NAKIVO Transporter running on a VM instance in Google Cloud from outside the cloud. Here, you must add a new rule to fix this, in the Google Cloud Platform menu clicking VPC network and hit Firewall rules.
On the Firewall rules page, you can see existing default firewall rules. Click Create firewall rule to create a new rule.
Select the following parameters for a new firewall rule:
Direction of traffic: Ingress (Inbound traffic)
Actions on match: Allow
Targets: All instances in the network
Target tags: nakivo
Source filter: IP ranges
Source IP ranges: Add your external IP addresses from which you will connect to Google Cloud Linux VM running NAKIVO Transporter
Protocols and ports: Specified protocols and ports – TCP 9446, 9448-10000
Click Create to create the firewall rule.
Now you can see the new rule which permits connections for the Transporter in the list of firewall rules used in Google Cloud Platform.
Adding the Transporter to the configuration of NAKIVO Backup & Replication
The Transporter is installed on the Linux VM instance running in Google cloud and every necessary component is now configured in Google Cloud. For your next step, you need to connect the Transporter to the Director of your NAKIVO Backup & Replication instance used in the local (onsite) infrastructure.
Open the web interface of NAKIVO Backup & Replication running in your local infrastructure.
Go to Configuration > Transporters. Click Add Existing Transporter and select Installed service.
On the Add Existing Transporter configuration screen, set the following parameters:
Hostname or IP: Enter the external IP address of your Linux VM running in Google cloud on which the Transporter is running. You can check this IP address on the VM instances page of the web interface of Google Cloud Platform. In this example, the IP address of the machine with remote Transporter is 126.96.36.199.
Transporter name: Google Cloud Transporter.
For the rest of the parameters, you can keep the default values. Then click Add.
Creating a new backup repository on Linux running in Google Cloud
Once the remote Transporter running in Google Cloud has been added, you can create a new backup repository on the Google VM instance running the Transporter. In the web interface of NAKIVO Backup & Replication, go to Configuration > Repositories, then click the Add Backup Repository button and click Create new backup repository.
On the Create Backup Repository screen, set the following parameters:
Name: Google Cloud Repo
Assigned Transporter: Google Cloud Transporter
Location: Local folder on assigned transporter
Path to the local folder: /mnt/nakivo-repo (select the directory on your Linux VM instance running in Google Cloud that was created for using as a backup repository).
Compression: Best (cloud storage costs money and you can use the maximum possible compression to save storage space in this case).
You can also enable encryption and other options if needed. Then click Add.
Now your Google Cloud backup repository is shown in the list of backup repositories used in NAKIVO Backup & Replication.
Creating a job for VM backup to Google Cloud in NAKIVO Backup & Replication
Now you can create a new backup job or backup copy job and select your Google Cloud repository as a backup destination for starting the process of VM backup to Google Cloud.
The best practices recommend you to make local backups and store backup copies in the cloud. This approach also meets the 3-2-1 backup rule. Creating a backup job and then copying the VM backup to Google Cloud with NAKIVO Backup & Replication is explored in this current example.
Now you can create a VM backup job and back up your VM (VMs) to a backup repository in your local infrastructure, in this case, backing up the VMware VM running Windows to the CIFS backup repository stored in the local infrastructure (local site) and setting the name of the backup job as “VMware backup job Light”.
You have now opened the New Backup Copy Job Wizard which consists of 5 steps.
1. Backups. Select the existing VM backup job (VMware backup job Light in this case).
2. Destination. In the drop-down menu, select the Google Cloud backup repository that you created previously (Google Cloud Repo is used in this example).
3. Schedule. Set scheduling options for your job used for copying your VM backup to Google Cloud.
4. Retention. Set retention settings for your backup copy job. You can select how many recovery points can be kept per the appropriate time period.
5. Options. Configure job options, set the name (for example, Backup copy job to G-Cloud) and hit Finish & Run to start the process of copying your VM backup to Google Cloud.
Run the process of copying your local VM backup to Google Cloud and wait until the process is finished.
Congratulations! Now you know how to run VM backup to Google Cloud Storage as well as backup copy with NAKIVO Backup & Replication using the first method.
Method 2 – Backup to Google Cloud by Using a Bucket
The second method used for performing a VM backup to Google Cloud with NAKIVO Backup & Replication is similar to the method described, with the primary difference being that a Google Cloud Storage bucket is used for creating a backup repository instead of a standard persistent disk attached to the VM instance.
When latency and disk performance are not the top priority for tasks such as backup from your local infrastructure to cloud infrastructure (for example, VMware VM backup to Google Cloud), you can use a Google Cloud Storage bucket that provides scalable, flexible and durable storage. When using storage for backup, you can generally assume that storage space will not be used as intensively as it would be for applications such as a database server, application server etc. Google Cloud Storage is a serious service for hosting backups, boasting an unlimited bucket size with a 5-TB maximum size of a file that can be stored in a bucket. You are likely to find the price quite attractive if you are using buckets in Google Cloud.
You can upload and download files in the web interface of Google Cloud Platform as well as mounting a bucket to a VM instance running in Google Cloud. But there are some features of mounting buckets to Google VM instances comparing to mounting disks to VM instances as file systems. You should use FUSE, a special tool instead of traditional Linux mounting tools.
In order to perform VM backup to Google Cloud bucket with NAKIVO Backup & Replication you should do the following:
- Create a Google Cloud instance (VM) running Linux. Ubuntu 18.04 LTS Minimal is used in the current example (explained for method 1 above)
- Install NAKIVO Backup & Replication Transporter on your Linux VM instance running in Google Cloud (see method 1)
- Create a bucket that will be used as a repository for backup to Google Cloud Storage
- Install gcfuse on your Linux VM instance
- Mount the bucket to the Linux VM instance
- Add a Transporter (see method 1, make sure that firewall rules allow for connecting the Director to the Transporter).
- Create a backup repository in the local directory of your Linux VM instance to which the bucket is mounted
- Back up your VMs or make a copy of your VM backup to Google Cloud by using the created backup repository
Let’s explore the configurations needed for using this method in detail by applying the Linux VM instance configured during exploration of method 1.
Create a bucket
Go to the Google Cloud Platform console and select your project.
In the left pane of the Google Cloud Platform management page, go to Storage > Browser to open a bucket management page. Then click Create bucket.
On the Create a bucket page, configure the settings for your bucket.
Name. Enter a name that is unique across Google Cloud Storage, for example, nakivo-bucket-01.
Default storage class. There are four options for storage class:
- Multi-regional. This option is appropriate for storing frequently accessed data. The maximum availability is provided but the price is the highest of the four classes.
- Regional. This option can be used for storing data that is accessed quite frequently. The availability is high, yet the price is lower than for the Multi-regional storage class.
- Nearline. Select this option if you don’t expect to access the data frequently (about once per month). The cost per GB is low, but data retrieval costs are present.
- Coldline. Select this storage class option for rarely-used data (about once per year). It is a good option for storing backups for disaster recovery and providing the lowest cost per GB stored .
Let’s select the Coldline storage class for this tutorial.
Location. Select the location that is the nearest to you. The European Union is used as the location in the current example (the Linux VM instance running in Google Cloud is also located in one of the European datacenters, as you may recall).
For all remaining settings, you can leave them in default values. Click Create after configuring all needed options.
Now that you have successfully created your bucket, you should upload a temp.png file via the web interface to the nakivo-bucket01 previously created for testing.
Install gcsfuse on your Linux VM instance
FUSE (filesystem in userspace) is a software interface, a framework that allows Linux users to create their userspace file systems for use on Linux machines without editing the kernel code. FUSE can be also used for FreeBSD, OpenSolaris and macOS.
Google Cloud Storage FUSE (gcsfuse) is an open source tool that works as a storage adapter for mounting Google buckets as file systems on Linux and macOS machines. A standard semantic file system is used to upload and download files (Cloud Storage Objects).
In the Google Cloud Platform management web interface, go to Compute Engine > VM instances, select your Linux VM instance (g-transporter in this case), make sure that the instance is running and connect it to said VM instance via SSH.
Upon receiving the Linux console, make note that most of the following actions should be done in said console of your Linux VM instance, and the commands must be executed with root privileges.
Add a repository that contains gcsfuse by running the following commands:
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Update the list of available packages used by your package manager. After running this command, your package manager will be aware that a new repository that contains gcsfuse has been added.
Install gcsfuse from the repository you have previously added.
apt-get install gcsfuse
Mount the Google bucket to the Linux VM instance
Now that the gcsfuse tool has been installed, let’s mount the Google bucket to Google Cloud instance and continue using this bucket for backup to Google Cloud Storage.
Create a directory to which you are planning to mount the bucket, for example, /mnt/g-bucket.
Mount the bucket by using gcsfuse:
gcsfuse nakivo-bucket01 /mnt/g-bucket/
Verify whether the bucket is mounted to the specified directory:
List the files located within the bucket. As you recall, the temp.png file has been uploaded to the nakivo-bucket01 in the web interface of Google Cloud Platform.
ls -al /mnt/g-bucket
Important: Full access to the appropriate Cloud APIs must be enabled in the settings of your Linux VM instance that is running in Google Cloud (check the VM instance details page). Otherwise, you will not be able to write files to a Google bucket from the Linux console. You cannot administer the owner and permission settings for a directory used as a mounting point for the bucket by using traditional commands like:
chown -R username:groupname /mnt/g-bucket/
chmod -R 0775 /mnt/g-bucket/
After mounting a bucket, permissions are overwritten.
Try to copy any files to the Google bucket directory (/mnt/g-bucket/), or create a new file.
echo $PATH > /mnt/g-bucket/test1.txt
Verify whether the file has been created:
ls -al /mnt/g-bucket/
Check uid (user id) and gid (group id) of NAKIVO user:
cat /etc/passwd | grep bhsvc
In this example, the output is the following:
In this case, user id (uid) is 999 and group id is 1003.
To make mounting of the bucket automatic and persistent, edit /etc/fstab for this purpose.
Add the string
nakivo-bucket01 /mnt/g-bucket gcsfuse rw,user,implicit_dirs,uid=999,gid=1003,noatime,allow_other 0 0
nakivo-bucket01 is the bucket name in Google Cloud Storage.
/mnt/g-bucket is the directory used as a mounting point for the bucket.
The mounting options are as follows:
gcsfuse - mount the file system using gcsfuse.
rw - mount with permissions.
user – any user can mount the file system.
implicit_dirs - makes possible the allowance for the implicit existence of directories.
uid=999 - override the uid field set by the file system. 999 is the uid of the bhsvc user created by NAKIVO Backup & Replication. This user must have permissions to write the files. Otherwise, a backup repository cannot be created.
gid=1003 - override the gid field set by the file system. 1003 is the gid of the bhsvc group used by NAKIVO Backup & Replication. Similarly, this group must have permissions to create a backup repository and write the files.
noatime - information about accessing the files is not updated in the file system (the time of writing the file, last editing of the file). In the case of using Google Cloud Storage, this approach helps you reduce I/O and, as a result, increase performance.
Allow_other - all users can access the files in the mounted bucket (not only root user).
0 - this field sets whether the dump utility is enabled to back up the file system (0 - disabled, 1- enabled).
0 - this field sets fsck in order to check the file systems (0 - do not check, 1 – check this partition first, 2 – check this partition next).
Save changes to /etc/fstab and reboot your Linux VM instance.
Make sure that your Google bucket is mounted automatically after the VM reboot:
df -h | grep g-bucket
Check whether there are enough permissions for NAKIVO Transporter to write files in the /mnt/gbucket/ directory
ls -al /mnt
drwxr-xr-x 1 bhsvc bhsvc 0 Mar 28 15:51 g-bucket
Try to copy any files to the Google bucket directory, or create a new file:
echo $SHELL > /mnt/g-bucket/test2.txt
Check whether the file has been created and file content written successfully:
ls -al /mnt/g-bucket/
Go to the web interface of Google Cloud Platform and check the content of your bucket (nakivo-bucket-01 in this case) to make sure that files are also visible in the web interface. Everything should be OK.
Go to the web interface of NAKIVO Backup & Replication (the Director side). In the current example the address of NAKIVO Backup & Replication web interface is https://192.168.17.50:4443
Deploy a Transporter on your Linux VM instance (explained for the method 1 above). Go to Configuration > Transporters, click Add Existing Transporter and select Installed service. Enter the external IP address of your Linux VM instance running in Google Cloud on which the Transporter is installed. Enter the Transporter’s name, making sure to check your firewall settings in Google Cloud Platform and verifying that the Director is able to connect to the remote Transporter.
In this example, we have used the existing Transporter that was added during explanation of method 1 of backup to Google Cloud (Google Cloud Transporter).
Add the new backup repository to the local folder of the machine running the Transporter. Go to Configuration > Repositories, click Add Backup Repository and select Create new backup repository (much like that shown for method 1).
In this case, select /mnt/g-bucket/ as the directory to be used as a backup repository (a path to the local folder).
Hit the Add button to finish creating a backup repository in the Google Cloud bucket and get ready for remote backup.
Now you can create a backup job or a backup copy job and send your VM backup to Google Cloud.
Backup to Google Cloud Storage can be considered a type of remote backup which allows you to protect your data against disasters in your onsite infrastructure. Configuring VM backup to Google Cloud with NAKIVO Backup & Replication is not difficult and can be done by using one of two methods. The first method requires an additional virtual disk mounted on a Google Cloud instance running Linux. If the second method is used, then a Google bucket must be mounted on the Google instance instead of the virtual disk to create a backup repository. You can compare speed and price for these two methods and select the method which best meets your needs. Try the backup-to-cloud functionality, which comes as a part of the Free Edition of NAKIVO Backup & Replication and allows you to easily and cost-efficiently send backups and backup copies of your production machines.