Common Format For A Time Machine Backup Disk

• Best Time Machine Backup Drive
• Erase Backup Disk Time Machine

Time Machine only recognizes the drives formatted as Mac OS Extended (Journaled), also known as HFS+. If the drive is formatted with different format types, the Time Machine will offer t o erase the disk and reformat before use. To know the current format of the external drive, use the Disk Utility app. The most common format for a Time Machine backup disk is Mac OS Extended format(Journaled) format, but Time Machine also supports Mac OS Extended (Case-sensitive, Journaled) and Xsan formats. Important: You can back up from an HFS+ or APFS-formatted disk to an HFS+ disk; however, Time Machine can't back up to an APFS-formatted disk.

Previously, to back up a virtual machine's disk, you needed to:

1. Create a temporary snapshot.
2. Copy the snapshot file in its current format, either RAW guest data or QCOW2 file data.
3. Delete the temporary snapshot.

It was possible to do a limited version of an incremental backup by creating one snapshot every time a backup ran, copying the snapshot disk, and deleting the previous snapshot. But the copied data was in QCOW2 format, so restoring it required merging QCOW2 files before uploading to storage.

1. Incremental Backup
   1. The Incremental Restore Flow
   2. Backup REST API
      1. enable POST
   3. list GET
      1. Checkpoints REST API
   4. imageio backup API
      1. Example - getting data and zero ranges

High level Design

Now, using the Incremental Backup API, you can:

• Perform full or incremental backups for disks using QCOW2 format without any temporary snapshots
• Backup raw guest data instead copying QCOW2 data for QCOW2 disks
• Restore raw guest data to disk onto raw or QCOW2 disks

Backups are simpler, faster and more robust, and integration with backup applications is improved, with new support for backing up and restoring raw guest data, regardless of the underlying disk format.

Limitations

• Only disks in QCOW2 format can be backed up incrementally, not RAW format disks. The backup process saves the backed up data in RAW format.
• Only backed up data in RAW format can be restored.
• Incremental restore does not support restoring snapshots as they existed at the time of the backup. This limit is common in backup solutions for other systems.

Creating a Virtual Machine With Incremental Backup Enabled

For a backup application to be able to include a disk during incremental backups, you must enable incremental backup for that disk. When adding a disk, you should enable incremental backup for every disk. You can back up disks that are not enabled for incremental backup using full backup or in the same way you did previously.

Because incremental backup requires disks to be formatted in QCOW2, use QCOW2 format instead of RAW format. For more information, see Disk Format.

Enabling Incremental Backup on an Existing Virtual Machine

Because incremental backup is not supported for disks in RAW format, a QCOW2 format layer must exist on top of any RAW format disks in order to use incremental backup. Creating a snapshot generates a QCOW2 layer, so creating a snapshot will allow to enable incremental backup on all disks that are included in the snapshot, from the point at which the snapshot is created.

WARNINGIf the base layer of a disk uses RAW format, deleting the last snapshot and merging the top QCOW2 layer into the base layer converts the disk to RAW format, thereby disabling incremental backup if it was set. To re-enable incremental backup, you can create a new snapshot, including this disk.

Disk Format

The following table shows how enabling incremental backup affects disk format:

The Incremental Backup Flow

1. The backup application uses the REST API to find virtual machine disks that should be included in the backup. Only disks that are enabled for incremental backup, using QCOW2 format, are included.
2. The backup application starts a backup using the [VmBackup API]: http://ovirt.github.io/ovirt-engine-api-model/4.3/#services/vm_backup 'VmBackup API', specifying a virtual machine ID, an optional previous checkpoint ID, and a list of disks to back up. If you don't specify a previous checkpoint ID, all data in the specified disks is included in the backup, based on the state of each disk when the backup begins (full backup).
3. The engine prepares the virtual machine for backup. The virtual machine can continue running during the backup.
4. The backup application polls the engine for the backup status, until the engine reports that the backup is ready to begin.
5. When the backup is ready to begin, the backup application creates an image transfer for every disk included in the backup. For more information, see the [ImageTransfer API]: http://ovirt.github.io/ovirt-engine-api-model/4.3/#services/image_transfer 'ImageTransfer API'.
6. The backup application gets a list of changed blocks from ovirt-imageio for every image transfer. If a change list is not available, the backup application gets an error.
7. The backup application downloads changed blocks in RAW format from ovirt-imageio and stores them in the backup media. If a list of changed blocks is not available, the backup application can fall back to copying the entire disk.
8. The backup application finalizes all image transfers.
9. The backup application finalizes the backup using the REST API.

The Incremental Restore Flow

1. The user selects a restore point based on available backups using the backup application (not part of oVirt).
2. The backup application creates a new disk or a snapshot with an existing disk to hold the restored data.
3. The backup application starts an upload image transfer for every disk, specifying format=raw. This enables format conversion when uploading RAW data to a QCOW2 disk.
4. The backup application transfers the data included in this restore point to imageio using the API.
5. The backup application finalizes the image transfers.

Checkpoint deletion

1. Backup application finds the oldest checkpoint of a VM.

2. Backup application remove the checkpoint

Restoring snapshots

Incremental restore will not support restoring snapshots as existed atthe time of the backup. This limit is common in backup solutions forother systems.

Handling an Unclean Shutdown or Storage Outage During Shutdown

If a virtual machine is shut down abnormally, bitmaps on the disk might be left in an invalid state. Creating an incremental backup using such bitmaps leads to corrupt virtual machine data after a restore.To detect bitmap in invalid state we can use the in-use bit in thebitmap, but this info is not exposed to the management layer.

To recover from an invalid bitmap, you need to delete the invalid bitmap and all previous bitmaps, and the next backup needs to include the entire disk contents (full backup).

Backup REST API

enable POST

Enable incremental backup for a virtual machine's disk.

For example, to enable incremental backup for a new disk on a virtual machine with id 123, send a request like this:

With a request body like this:

The response is:

Parameters Summary

Finding disks enabled for incremental backup

list GET

For the specified virtual machine, list the disks that are enabled for incremental backup, filtered according to the backup property.

For example, for a virtual machine with the id 123, this request:

Gets this response:

Parameters summary:

Start a full backup POST

Start full backup. The response phase indicates that the backup is 'initializing'. You need to poll the backup until the phase is 'ready'. Once the backup is ready the response will include which should be used as the in the next incremental backup.

For example, to start a full backup of a disk with id 456 on a virtual machine with id 123, send a request like this:

With a request body like this:

The response includes backup 789 with 999:

Parameters summary:| Name | Type | Direction | Summary |:—- | :—- | :—- | :—- |disk| ? | out | Required. Specify the UUID of a disk. |

Start an incremental backup POST

Start incremental backup since checkpoint id . The response phase indicates that the backup is 'initializing'. You need to poll the backup until the phase is 'ready'. Once the backup is ready the response will include which should be used as the in the next incremental backup.

For example, to start an incremental backup of disk 222 on virtual machine 111, send a request like this:

With a request body like this:

The response includes backup 789 with 999 and 666:

Get backup info GET

Gets information about a backup.

Gets the following information about a specific backup:

• the id of each disk that was backed up
• the ids of the start and end checkpoints of the backup
• the id of the disk image of the backup, for each disk included in the backup
• the phase of the backup
• the date the backup was created

When the value of is ready, the response will include which should be used as the in the next incremental backup and you can start downloading the disks to back up the virtual machine storage.

Best Time Machine Backup Drive

For example, to get info about a backup with id 456 of a virtual machine with id 123, send a request like this:

The response includes the backup with id 456, with 999 and 666. The disk included in the backup has id 444, and the id of the disk image is 555.

finalizing backup POST

To finalize a backup, use the finalize backup service method.

For example, to finalize backup with id 456 of a virtual machine with id 123, send a request like this:

Creating an image transfer object for downloading an incremental backup POST

When the backup is ready to download, an imagetransfer object should be created.To correlate the transfer with the backup, property should be specified with the relevant backup id.The transfer property should be raw (this indicates that NBD is used as the ImageTransfer backend).For information on creating an imagetransfer object, see the [add method for ImageTransfers]:http://ovirt.github.io/ovirt-engine-api-model/4.3/#services/image_transfers/methods/add 'add method' in the REST API Guide.

For example, to initiate transfer for an incremental backup send a request like this:

With a request body like this:

Creating an image transfer object for incremental restore POST

In order to restore raw data backed up using the incremental backup API to a QCOW2-formatted disk, an imagetransfer object should be created. What version of imovie do i have. To restore an incremental backup, specify the key in the transfer. For information on creating an imagetransfer object, see the [add method for ImageTransfers]:http://ovirt.github.io/ovirt-engine-api-model/4.3/#services/image_transfers/methods/add 'add method' in the REST API Guide.

For example, to initiate restoring an incremental backup send a request like this:

With a request body like this:

When uploading into a snapshot, replace with .

When the transfer format is RAW and the underlying disk format is QCOW2, uploaded data is converted on the fly to QCOW2 format when writing to storage.Uploading data from a QCOW2 disk to a RAW disk is not supported.

Checkpoints REST API

Get all created checkpoints for a VM GET

Erase Backup Disk Time Machine

To get all the created checkpoints of a VM :

For example, to get all the created checkpoints of a virtual machine with id 123 send a request like this:

The response includes all the virtual machine checkpoints, each checkpoint contains the checkpoint's disks, , and the virtual machine it belongs to :

Get a specific VM checkpoint GET

For example, to get a specific checkpoint with id 456 of a virtual machine with id 123 send a request like this:

Response:

Remove the root checkpoint of a specific virtual machine DELETE

To remove a checkpoint of a virtual machine, the removed checkpoint should be the oldest checkpoint of the VM (root checkpoint).

For example, to remove the root checkpoint with id 456 of a virtual machine with id 123 send a request like this::

imageio backup API

Map request

Get a map of zeros and data ranges on storage.

Query options:

• dirty=y - return only ranges modified since backup checkpoint id

Returns list of JSON objects with these keys:

• data: true for allocated ranges, false for zero or unallocated ranges
• start: offset of range in bytes
• length: number of bytes

Example - getting data and zero ranges

Request:

Response:

Example - getting only dirty data and zero ranges

Request:

Response:

Note: This will erase all data on your backup disk!!!

Launch Disk Utility

Open a Finder window and navigate to Applications > Utilities and double click on Disk Utility.

The remaining steps vary considerably depending on the operating system you are running. Choose About This Mac from the Apple menu to determine your current OS, then make a selection below.

Instructions for Sierra and El Capitan

Select the backup disk

Click to select the disk that you would like to use for your backup. This disk should not be the same as your startup disk.

The name of a new disk will often include the manufacturer's name (e.g. WD My Book 111D Media..). A startup disk will often include the manufacturer's serial number in the title (e.g. TOSHIBA MK50..).

Erase the backup disk

Click on the Erase button in Disk Utility's toolbar, then configure the name, format, and partitioning scheme of your backup disk. You can set the name to whatever you like, but set the Format to Mac OS Extended (Journaled) and set the Scheme to GUID Partition Map, then click the Erase button.

Don't Use Time Machine

Click Don't Use. You may use the same backup disk for both Time Machine and CCC backups, but if you do so, you must use a dedicated partition for the Time Machine backup. Otherwise Time Machine will consume all available space on the backup volume and make it impossible for CCC to use the backup volume.

Your new hard drive is now ready to accept backups created by Carbon Copy Cloner!

Instructions for Mountain Lion, Mavericks, or Yosemite

Select the backup disk

Click to select the disk that you would like to use for your backup. This disk should not be the same as your startup disk.

The name of a new disk will often include the storage capacity and manufacturer's name (e.g. 500.07 GB WD My Passp..). A startup disk will often include the manufacturer's serial number in the title (e.g. 320.07 GB TOSHIBA MK3255GSXF Media).

Partition the backup disk

Click on the Partition tab.

Choose 1 Partition from the Partition Layout popup menu (or more if desired).

Click on Options.

Choose GUID Partition Table, then click the OK.

Name the Volume

Format the Volume

Select Mac OS Extended (Journaled) from the Partition Format popup menu.

Click Apply.

Ensure that you have selected the correct disk for your backup drive. This step will delete all data from the selected disk. Click Partition.

Now skip ahead to the remainder of the instructions that are not OS-specific.

Related Documentation

'My disk is already formatted HFS+, why am I getting this warning?'
Video: Preparing a drive for a backup of the macOS in macOS 10.11 or higher
Video: Creating a new/additional partition (OS X 10.10 and earlier)
Support for third party filesystems (e.g. NTFS, FAT32)