Chapter 7 Backing Up and Recovering a vSphere Deployment

2V0-21.19 EXAM OBJECTIVES COVERED IN THIS CHAPTER:

• Section 4 - Installing, Configuring, and Setting Up a VMware vSphere Solution

  • Objective 4.6 - Deploy and configure VMware vCenter Server Appliance (VCSA)

• Section 5 - Performance-tuning and Optimizing a VMware vSphere Solution

  • Objective 5.1 - Determine effective snapshot use cases

• Section 7 - Administrative and Operational Tasks in a VMware vSphere Solution

  • Objective 7.7 - Perform different types of migrations
  • Objective 7.9 - Create and manage VMs using different methods

This chapter covers backing up the vCenter Server Appliance (VCSA) and virtual machines as well as replicating virtual machines. Backups are important for a lower-cost method of recovering (partially or completely) failed components as well as tracking changes of components over a period of time. Replication is important in order to provide rapidly available duplicates of components, either locally or remotely.

Replication is normally costlier than backups as replication tasks cause additional WAN network traffic when run to remote sites, and replication tasks are often run at a higher rate. While you might back up your virtual machines nightly, replications are often set to run every hour. With VMware's backup and replication solutions, only the data that changed since the last process ran is affected, making both processes very efficient.

While backup tasks are often run on most virtual machines in the environment, you might only replicate the most critical virtual machines.

VCSA Backup and Restore

With vSphere 6.5, the vCenter Server Appliance has the ability to make a file-based backup. While the backup is currently triggered manually from the appliance GUI and cannot be scheduled, the resulting backup is much smaller than a copy of the whole appliance would be. The primary downside to the file-based backup is that in order to restore the backup, you need to use the deploy utility to create a new appliance that used the backup files as the data source.

To start a backup of the VCSA, you need to log into the management interface of the appliance, which is https://:5480. The login credentials are root plus the password created when the appliance was deployed.

Once logged in, you will see the Backup button on the summary page (Figure 7.1).

Clicking the Backup button will launch the Backup Appliance wizard. The first screen (Figure 7.2) configures the destination for the backup. The choices are HTTP, HTTPS, SCP, FTP, and FTPS. If you select HTTP or FTP, you will be warned that those protocols are not secure.

Note that the Location field requires the server FQDN or IP plus a directory (or folder) at a bare minimum. You can also supply a path. The directory doesn't need to exist, but if it does exist, it must not have an existing backup in it. The directory name will be used to set the virtual machine name during the restore process, so best practice would be to use the name of the appliance as the directory name.

You have the option of encrypting the files, which will add the .enc extension and a layer of encryption to each of the backup files ( Figure 7.3 ).

The size of the backup will vary depending on the size of the inventory and the amount of historical data on the vCenter server; however, as shown in Figure 7.4, exporting the historical data is optional.

Backing up the appliance is much more straightforward than restoring it. To restore, you run the graphical deploy utility (the CLI deploy utility does not offer restore) and choose the Restore option ( Figure 7.5 ).

The Restore process will prompt you for the location of the backup files (Figure 7.6).

If you added a password during the backup, you will be prompted to enter the password during Stage 2 (Figure 7.7). The Windows system running VCSA-Deploy will download the backup-metadata.json file from the backup directory and check the supplied password against the PasswordValidator entry to verify that your entered password is correct before prompting to complete the restore.

The restore process is very similar to the VCSA upgrade and migrate processes, where a new appliance is deployed and then the configuration and data are imported. While this process takes longer to recover a VCSA appliance compared to restoring a copy of the VCSA virtual machine, the data backed up is considerably smaller and the backup process should be faster.

Backing Up Virtual Machines by Using VDP

VMware includes a license for vSphere Data Protection (VDP) with all versions of vSphere except the lowest-price vSphere Essentials license. Based on (or “Powered By” according to the plug-in) EMC's Avamar product, VDP provides virtual-disk-level backups of virtual machines, either powered on or powered off. Additional features include data deduplication, backup replication, user-controlled file-level restore, and guest-level backups of Microsoft services including Exchange, SQL, and SharePoint.

There are a few key “maximums” to know for VDP that will affect how they are deployed. The greatest number of virtual machines that can be protected by one VDP appliance is 400, and the greatest number of VDP appliances that can be supported per vCenter is 20. This means you can only protect up to 8,000 virtual machines on a single vCenter server. Of course, an environment that large should probably be using a third-party backup solution because each VDP appliance runs independently. With multiple appliances in use, you would have to manually keep track of which VMs are being backed up on which appliance and you would have at least one backup job and destination per appliance.

Installing VDP

To install vSphere Data Protection, you need a vCenter server, an ESXi host, and at least 873 GB of storage space. When you deploy the OVA file for the VDP appliance, you will be prompted to add IP information for the appliance. After deploying, connect to the new appliance using https://:8543/vdp-configure with a username of root and password of changeme.

During installation, you can “create” new storage for VDP or use (or migrate) existing VDP storage. Creating new storage adds VMDK files to the appliance to be used (Figure 7.8).

When creating storage for VDP your options are 0.5, 1, 2, 4, 6, or 8 tebibytes (TiB) as shown in Table 7.1. The appliance will deploy with a single 200 GB VMDK for the appliance plus additional VMDKs depending on your storage choice.

When adding storage to the appliance, you can choose to have all the storage VMDKs created in the same directory as the appliance, or you can distribute them among the available datastores (Figure 7.9).

If you initially select a storage amount less than 8 TiB, you can expand the storage later using the same https://<appliance FQDN or IP>:8543/vdp-configure link used for the initial setup. Select Expand Storage under the Storage tab (Figure 7.10) and then choose the new size and place the disks.

If the new storage size utilizes larger VMDKs (see Table 7.1), the existing VMDKs will be resized. When you initially add storage to the appliance, you can choose thick lazy-zeroed, thick eager-zeroed, or thin provisioned. This choice can be changed later using the Expand Storage wizard (Figure 7.11).

During the initial installation, you can choose the CPU and RAM settings for the appliance, which default to 4 CPUs and 4096 MB of RAM with the minimum storage setting. If you are choosing a higher amount of storage, the wizard may suggest more memory. If you later expand the storage, you may again be prompted to increase the memory for the appliance (Figure 7.12). While the CPU and Memory window for both the initial and Expand Storage wizards shows an entry for CPU, the value is set to 4 and can't be changed.

Creating Backup Jobs

Once the appliance has been initially configured, you can access the vSphere Data Protection plug-in using the vSphere client (Figure 7.13). If you do not see the plug-in on the client, you may need to log out and back in.

There are two types of backup job types: Applications and Guest Images. Applications jobs allow you to back up Exchange, SQL, and SharePoint servers (Figure 7.14) using a client installed in the guest. Note that the applications do not need to be running on virtual machines; as long as the VDP client can access the IP address of the VDP appliance, you can back up one of the supported applications. Guest Images jobs back up entire virtual machines or select VMDKs.

The application backup clients can be downloaded from the Downloads section of the Configuration tab (Figure 7.15).

Installation doesn't require much more than the VDP server's IP address, but each client will need to be configured after installation. Figure 7.16 shows the configuration tool for the Exchange client.

The tool will create the user specified with the permissions it needs. You will need to enter the information (username, password) into the backup job during creation. See Figure 7.17 for an example of creating an Exchange backup job.

When creating Guest Images backup jobs, you have the choice of backing up the virtual machine with all of its drives (Full Image) or selecting specific drives to back up with the Individual Disks option. Both methods will back up the virtual machine's .vmx configuration file, but Individual Disks is useful if you want different backup schedules for different drives or have incompatible drives on a virtual machine. Virtual machine disks set to Independent mode, RDMs, and VMDKs stored on VVols are not compatible with VDP backups. The backup job wizard will display an alert (Figure 7.18) if an incompatible disk is selected.

VDP has two options for backing up the VMDKs: HotAdd and network block transport (NBT). For HotAdd, the VDP appliance will take a snapshot of the VMDKs it is backing up (Figure 7.19) and then mount the VMDKs from the virtual machines as IndependentNonPersistent disks (Figure 7.20).

If the virtual machine is powered on, the snapshot process of the HotAdd will include a quiesce attempt. When a snapshot is flagged for quiesce, the guest disk buffers will be flushed so that there are no outstanding writes. This prevents guest disk corruption. If the quiesce request is not successful, a crash-consistent snapshot will be taken, but noted in the backup log.

HotAdd is the default mechanism for backing up VMDKs; however, it requires the datastore the VMDK is on to be presented to the host the VDP appliance is on, and vSphere Flash Read Cache (vFlash) cannot be enabled for the virtual machine. If these requirements are not met, VDP will use NBT to back up the virtual disks and will leverage the ESXi server hosting the virtual machine to back up the VM over the network to the VDP appliance.

VDP utilizes Change Block Tracking (CBT) to improve backup and restore times. With CBT, the ESXi hosts can identify what disk sectors have changed since the last backup. Also, per VMware KB article 1020128, “On VMFS partitions, CBT can also identify all the disk sectors that are in use.” This technology allows backup jobs to know which data to back up without scanning the data or to restore a backup by only replacing the blocks that changed. CBT is enabled on virtual machines by default.

After a backup job is run, you can create a Backup Verification job to ensure that the backup worked successfully.

Restoring from Backup

There are several options for restoring from backup. For Guest Images, you can overwrite the existing VMDK with the backup or overwrite the entire VM with the backup. You can also spin up a copy of the VM with the VMDKs attached. Using the VDP Restore client, you can also restore individual files from the Guest Images backups. Application restores can be full or partial, depending on the backup options. Unlike a backup job, a restore job can include Guest Images and Application restores.

If the virtual machine was not successfully quiesced before a Guest Images backup runs against it, the backups will be differentiated in the backup list with a red icon (Figure 7.21).

If you create a restore that includes backup jobs that were not quiesced, a warning will be displayed alerting you to the crashconsistent backup (Figure 7.22).

To restore individual files from a backup, access the VDP File Level Restore (FLR) client using https://<IP or FQDN or VDP appliance>:8543/flr. There are two methods of login, Local Credentials and Advanced Login.

Local Credentials gives an administrator on the VM that was backed up the ability to restore files locally. Only files backed up on the machine the FLR is running from can be restored using Local Credentials. If you attempt to use Local Credentials from a guest that has not been backed up by the VDP appliance, you will get an error message: “Cannot locate vm in vCenter” (Figure 7.23).

For username and password, enter the credentials for the local PC you are running the FLR client from to get a list of available backups (Figure 7.24). Only backups for the current VM are available when using Local Credentials.

Selecting a backup job and clicking Mount will display the file structures of the VMDKs. You can restore files to their original locations by selecting the root disk or choose a new folder to restore to; however, the folder must already exist. You can monitor the restore from the Monitor Restores tab (Figure 7.25).

If you would like to restore files to a VM other than the original, or if the UUID of the virtual machine has changed from a restore of the VM or by moving it, you can use the Advanced Login. The Advanced Login screen takes the same local credentials but also requires the vCenter credentials used to register the VDP appliance with vCenter.

While any system administrator with the URL for the VDP appliance can restore backups for systems they administer, only a user with the proper vCenter credentials can restore files from the backup jobs of other systems.

A few notes on restoring using the FLR client: You cannot restore ACLs, symbolic links, or more than 5,000 files or folders in one restore job. Browsing directories is also limited to no more than 5,000 files/folders. Also, you cannot use FLR if there is a NAT in front of the VM or if there is a firewall between the VDP and the VM. There are also several limitations to which VMDKs FLR can pull files from; GPT disks, Windows dynamic disks and VMDKs with multiple partitions are not supported.

Deploying Proxy Servers

VDP can also be provisioned with external proxy servers. The VDP appliance installs with an internal proxy server capable of 8 simultaneous backups. However, you can deploy up to 8 external proxy servers, although there is a limit of 24 concurrent backups per VDP appliance when external proxies are used. The internal proxy server will be disabled if an external proxy server is added ( Figure 7.26 ).

Proxies can be deployed to increase the simultaneous number of backups running, to spread the backup and restore footprint across multiple appliances and hosts, or to provide a single VDP appliance with the ability to back up datastores that are not accessible on all hosts. Proxies are also required for backing up and restoring LVM and ext4 file systems.

Replicating Virtual Machines

VMware vSphere offers a solution that can replicate virtual machines between hosts. With vSphere Replication, you can create copies of virtual machines in a different cluster on the same vCenter, to a different vCenter in the same SSO domain, or to a different vCenter and SSO domain. The destination can also be cloud platforms like VMware Cloud on AWS. All licenses of vSphere except Essentials include a license for vSphere Replication.

vSphere Replication uses an appliance (at least one vSphere Replication appliance per vCenter) to manage and monitor the replication between the hosts. There is a plug-in to add the management options to the web client.

Offering features such as recovery point objectives (RPOs) down to 5 minutes and multiple point in time recovery for protected virtual machines, vSphere Replication can be a very effective tool to migrate workloads or prepare for disaster recovery. VMware's Site Recovery Manager (SRM) product can leverage vSphere Replication in addition to storage replication to create a complete disaster recovery environment.

To deploy vSphere Replication, you need to download the ISO from VMware.com and use the web client to deploy the proper files. The /bin directory of the ISO image contains the files to deploy several versions of vRealize appliances, including the SRM and Cloud Service versions. The Cloud Service version can be leveraged by vCloud Director, vCloud Air, or other cloud services.

Deploying a Replication Appliance

For a basic implementation of vSphere Replication, you only need to deploy the vSphere Replication OVF with the files selected ( Figure 7.27 ).

The other OVF file set - vSphere_Replication_Addon is needed to deploy “add-on” appliances that can help with scaling vSphere Replication.

During the deployment you will be prompted for the compute resource, how many CPUs the appliance is deployed with (4 is the default but can be changed to 2), storage, and network. You are also required to enter at least one NTP server and a new password for the root account of the appliance during the OVF deployment.

Once the appliance is deployed, you can connect to its management console at https://<IP or FQDN or vSphere Replication>:5480 and log on as root with the password you set during deployment. From the configuration screen, verify that the information is accurate and enter the password for the SSO administrator (Figure 7.28). Click the Save and Restart Service button to register the appliance with vCenter.

The appliance deploys with a self-signed certificate, but you can add a certificate generated from VMware Certificate Authority (VMCA) or the certificate authority used by your infrastructure. Once the appliance is registered with vCenter and the services are restarted, the Service Status at the bottom of the configuration page will change to “VRM service is running” (Figure 7.29).

The appliance allows the administrator the ability to manually check the VMware download site for updates and then install them. This feature is found in the configuration site for the appliance on the Update tab (Figure 7.30).

You can also schedule automated updates from the default repository, a CD, or a custom repository (Figure 7.31).

Configuring Replication

You can use vSphere Replication to replicate virtual machines between hosts in the same vCenter with no additional configuration. To replicate virtual machines to hosts connected to a separate vCenter, you will need a second replication appliance deployed and connected to the other vCenter, and the vSphere Replication servers need to be linked together.

When linking vSphere Replication servers, there are two site options: local and remote (Figure 7.32). Local sites are vCenter servers that share the same SSO domain, and remote sites are vSphere Replication servers that are connected to a vCenter server and have a different SSO domain.

To replicate a virtual machine, right-click the VM and select Configure Replication from the All vSphere Replication Actions menu ( Figure 7.33 ). Virtual machines cannot have Fault Tolerance enabled to configure them to be replicated and must be powered on before replication will start.

You can also select multiple VMs and create replication tasks for each of the selected VMs at one time. The tasks will share the same settings, although the storage location can be set for each VM during the Configure Replication wizard (Figure 7.34).

Also on the Configure Replication window, you are prompted to enable Guest OS quiescing and/or Network Compression. We would suggest enabling Guest OS quiescing for all OSs that support it, to prevent issues with the replicated image. However, you may find that certain virtual machines do not respond well to quiescing attempts and may need that disabled.

The Network Compression setting compresses the data to replicate before it is sent to the target vSphere Replication appliance. According to a VMware blog on the topic (blogs.vmware.com/vsphere/2015/03/vr60-compression.html), you can expect compression ratios of 1.6:1 to 1.8:1. While this results in a higher CPU load on the hosts running the source and target vSphere Replication appliances, the selected virtual machines will sync faster and there will be lower network utilization. While compression is a trade-off between network and CPU utilization, most environments have CPU resources to share, and we would suggest using network compression.

If there are ESXi 5.5 hosts in the environment, the Network Compression setting only works as advertised when the source and destination datastores are accessible by version 6.x ESXi hosts. If the virtual machine is running on a vSphere 5.5 host, then no compression will take place. If the destination datastore is only available by vSphere 5.5 hosts, the vSphere Replication appliance at the target site will decompress the data and send it to the ESXi 5.5 host.

This can also affect vMotion and by extension DRS. Virtual machines that are being replicated with compression enabled cannot be moved to ESXi 5.5 hosts because the new host does not support compression

Recovering Replicated VMs

The recovery settings of the virtual machine replication task include an adjustable RPO as well as optional point in time instances. The RPO setting is a balance between the most time you can lose and the system resources (CPU and networking primarily) it takes to replicate the changes. An RPO of 5 minutes means that in theory the most you can lose would be the last 5 minutes of changes. Every 5 minutes, the machine is replicated, so the most the destination could be behind is 5 minutes. However, this may be impractical for many servers as it could take 5 minutes to quiesce the guest or the CPU, and network impact of constantly replicating data may exceed the usefulness of the limited loss.

The multiple points in time feature allows multiple options for recovering the replicated VMs (Figure 7.36).

To recover a virtual machine at the target site, use the vSphere Replication Section of the Monitor tab for the destination vCenter. Right-click the virtual machine in Incoming Replications and choose Recovery (Figure 7.37).

The Recovery wizard will prompt you to either use the latest sync or initiate a sync as the first step of the recovery. If the virtual machine is still available, then Synchronize Recent Changes is the preferred choice.

After recovery, the virtual machine will be powered off and the network connections will be disconnected. If the replication was configured with multiple points in time, those points will be available as snapshots on the recovered virtual machine (Figure 7.39).

To fail the virtual machine back to the original configuration without Site Recovery Manager, you will need to create a vSphere Replication task for the recovered virtual machine with a destination of the original source.

Summary

This chapter has covered backing up the VCSA appliance and backing up and replicating virtual machines. Backing up the configuration of VCSA is important to ensure quick recovery in the event of a problem, and the files can be reasonably small (depending on the size of your events and performance database).

The vSphere Data Protection Appliance will back up your virtual machines to storage locally to the appliance using deduplication to reduce the storage footprint of the backups. The backup jobs can be replicated to VDP appliances at other sites to ensure recoverability in the event of a site loss.

vSphere Replication will make duplicates of virtual machines that can be brought up fairly quickly for testing or in the event the original VM is unavailable. While backups are not intended to run very often, replications can be set down to every 5 minutes. vSphere Data Protection offers data deduplication to reduce the storage used; vSphere Replication offers compression to reduce the network bandwidth used.

A combination of vSphere Replication and vSphere Data Protection will ensure that your virtual machines are recoverable from issues on the local machines or in the local site while balancing storage and network impact.

Exam Essentials

Know how to back up the VCSA. New to vSphere 6.5, you can back up a VCSA appliance into a set of files with options to only back up the configuration or include the historical data.

Understand how to deploy vSphere Data Protection. VDP has limitations on the number of virtual machines it can back up simultaneously and how many VMs it can back up in total. While 1 VDP appliance can back up 8 virtual machines at once, you can deploy up to 8 proxy servers to back up a total of 24 virtual machines at a time. However, each VDP appliance is limited to a total of 8,000 virtual machines, so larger environments will need a different solution.

Know how to replicate virtual machines. vSphere Replication manages virtual machine replication between clusters or vCenter servers. You should be able to describe deploying Replication, setting up a VM to replicate, and then recover that virtual machine at the target.

Know how Data Protection and Replication compare. Know that both are included with all vSphere licenses except Essentials. Both have appliances that need to be deployed and both have primary or manager appliances that can only be installed once per vCenter. Data Protection offers data deduplication and Replication has network compression. Data Protection jobs can run daily; Replication jobs can run from every 5 minutes up to once per day.

Review Questions

1. What features are offered by vSphere Data Protection? (Choose two.)

   1. Compression
   2. Deduplication
   3. Encryption
   4. Replication

2. What feature is offered by vSphere Replication?

   1. Compression
   2. Deduplication
   3. Encryption
   4. Guest agent

3. What is the minimum number of VDP proxy servers needed to back up 16 virtual machines simultaneously?

   1. 1
   2. 2
   3. 4
   4. 8

4. What product should be used to ensure that your Exchange server can be recovered in the event of a storage failure?

   1. vSphere Data Protection
   2. vSphere Replication
   3. vSphere High Availability
   4. vSphere storage clusters

5. What should be used to ensure that your MySQL database can be recovered in the event of a storage failure?

   1. vSphere Data Protection
   2. vSphere Replication
   3. vSphere High Availability
   4. A third-party utility

6. What option can be enabled to allow a virtual machine administrator to choose from multiple restore times after a virtual machine has been recovered?

   1. Snapshots
   2. Data deduplication
   3. Point in time instances
   4. Recovery point objective

7. What option reduces the amount of changes that can be lost for a virtual machine in the event of a recovery?

   1. Snapshots
   2. Data deduplication
   3. Point in time instances
   4. Recovery point objective

8. What options can be used to ensure that copies of your virtual machines are available in a remote environment in the event of a site loss? (Choose two.)

   1. vSphere Data Protection
   2. vSphere Replication
   3. vSphere High Availability
   4. vSphere storage clusters

9. What option can be enabled to reduce the likelihood of guest corruption?

   1. Guest OS quiescing
   2. Data deduplication
   3. Point in time instances
   4. Recovery point objective

10. What option should be disabled if the recovery point objective cannot be met?

    1. Guest OS quiescing
    2. Data deduplication
    3. Point in time instances
    4. Network compression

11. Why is the icon next to the object named “08/16/2018 1:02 AM” shown in red here?

    1. It is the master for that sequence.
    2. It has not been replicated yet.
    3. It was not quiesced.
    4. It has been encrypted.

12. What would prevent a virtual machine from being replicated?

    1. Fault Tolerance enabled
    2. Unsupported guest OS
    3. VMDKs on virtual SAN
    4. VM power state is on

13. What steps would reduce the network traffic used by vSphere Replication?

    1. Increasing the RPO time
    2. Disabling point in time instances
    3. Decreasing the RPO time
    4. Enabling data deduplication

14. What steps would reduce the storage used by vSphere Data Protection?

    1. Increasing the RPO time
    2. Disabling point in time instances
    3. Decreasing the RPO time
    4. Enabling data deduplication

15. Unchecking which option would minimize the storage needed by VCSA backups?

    1. Stats, Events, Alarms, and Tasks
    2. Common
    3. Configuration, Events, and Tasks
    4. Configuration, Events, Tasks, and Performance Metrics

16. What is the best method for ensuring that a Windows vCenter server can be recovered in the event of a storage issue?

    1. vCenter backup wizard
    2. vSphere Data Protection
    3. vSphere High Availability
    4. vSphere datastore cluster

17. What step could be taken to resolve the issue shown here?

    1. Select Use Latest Available Data.
    2. Unselect Point in Time Recovery.
    3. Power on the virtual machine.
    4. Quiesce the virtual machine.

18. What steps are required to regain use of a virtual machine after it has been recovered using vSphere Replication? (Choose two.)

    1. Power on the VM.
    2. Quiesce the virtual machine.
    3. Connect the network adapters.
    4. Select the point in time.