Initial Steps: Create a Protection Group for Volumes
These initial steps demonstrate how to create a protection group for volumes on the array using the FlashArray user interface and T-SQL commands. This protection group will be used in examples for the following use cases.
-
Create a protection group by clicking the + icon for Source Protection Groups. The Create Protection Group dialogue will appear.Figure 1. The Protection Groups tab in the FlashArray user interface.
-
Name the Protection Group, and then click Create.Figure 2. The Create Protection Group dialogue.
-
To add volumes to the protection group, click the ellipsis icon for Members, and then select Add Volumes.Figure 3. Adding volumes to a protection group.
-
Select all volumes relevant to the database. This should be done for both application- and crash-consistent snapshots.Figure 4. The Add Members dialogue.
-
After adding selected members to the protection group, use the example protection group "UserDB-PG" for all use case samples going forward.Figure 5. The UserDB-PG protection group.
Crash-consistent Backup and In-Place Restore to the Same Host
Scenario: This use case demonstrates how to perform a crash-consistent backup and restore on the same host. The example shows how to use a crash-consistent snapshot to perform an in-place restore of a user database on the same SQL Server instance. This scenario recovers the protection group snapshot to the original volumes from where they were taken.
Creating a Crash-consistent Snapshot
-
Enter Optional Suffix information for the snapshot's name (the snapshot name will default to "volume name.snapshot-number" without a suffix being set), and then click Create.Figure 7. Naming a snapshot in the Create Snapshot dialogue.
-
The snapshot name (with optional suffix) is displayed for Source Protection Group Snapshots.Figure 8. The snapshot created for the source protection group.
Recovering a Crash-consistent Snapshot to the Source Host
-
If the database being restored is available, online, and usable, ensure that it is set offline using the following command:
ALTER DATABASE [UserDB] OFFLINE WITH ROLLBACK IMMEDIATE; -
The Windows disks that the volumes correspond to also need to be offline prior to restoring the volume. snapshot. To offline the disks, in the Disk Management application on the host, right-click on the disk, and then select Offline.Figure 9. Volumes attached to the SQL Server host as offline drives.
-
In the FlashArray user interface, select the protection group snapshot from the previously created snapshot, and then select it to open the detail view for the volume snapshots contained within it.Figure 10. Selecting the protection group snapshot.
-
To restore the snapshots to the original volumes, click the overwrite icon.Figure 11. Restoring a snapshot using the overwrite icon.
-
In the Restore Volume from Snapshot prompt, review the volume name, and then select Restore to confirm.Figure 12. The Restore Volume from Snapshot dialogue.
-
Repeat steps 4–5 for each volume in the protection group snapshot.
-
Overwriting volumes with snapshots does not immediately destroy the state of a volume when it is overwritten. Instead, a snapshot of the state of the volume is taken and then placed in the Destroyed Volume Snapshots section, which is then retained for a set period prior to being eradicated.Figure 13. The Destroyed Volume Snapshots section of the Volumes tab.
-
If the volumes are in an offline state, they need to be set online. In Disk Management, right-click each offline disk, and then select Online.Figure 14. Bringing the disks online in the Windows Disk Manager.
-
If the disk does not have a drive letter assigned, right-click the disk, and then select Change Drive Letter and Paths.Figure 15. Volumes attached as drives on the SQL Server host.
-
If the disks come online in a read-only state, right-click each disk and set it to read/write.
-
If the database was set offline in step 1, and the database paths remain consistent, then set the database to online by using the following command:
ALTER DATABASE [UserDB] ONLINE; -
If the database was deleted or is being recovered to a different set of paths, attach the database to the current instance by running the CREATE DATABASE T-SQL command with the FOR ATTACH property and the appropriate paths to each datafile.
CREATE DATABASE [UserDB_temp] ON ( FILENAME = 'E:\SQL\UserDB.mdf' ),( FILENAME = 'F:\SQL\UserDB_log.ldf' ) FOR ATTACH
Creating an Application-consistent Snapshot Backup
Scenario: This use case shows how to create an application-consistent snapshot for a protection group and volume(s) for a specific point in time. These steps capture the snapshot and a backup of the transaction log from SQL Server. This procedure should be used when a snapshot is needed for a FlashArray protection group and its volume(s). A SQL-native log backup is used to perform a point-in-time restore when the data is restored to the current location or a new location (for example, cloning). The benefit of using snapshots for backup in this way includes instantly recovering data, regardless of the size, without waiting for a full backup and restore.
Prerequisites: Running SQL Server 2022 or later with a database and FlashArray with a protection group and volume. For more information about T-SQL snapshot backups, see Create a Transact-SQL snapshot backup.
Usage notes: Each section of the example is meant to be run sequentially. Note that although this process uses the FlashArray user interface and T-SQL commands, the same result can be accomplished using PowerShell. Because this process freezes write input/output to the database, using PowerShell or other tools to automate this process is preferred so that database write input/output can resume as quickly as possible. See the Everpure OpenConnect SQL Server Scripts on GitHub for more information.
-
Prior to creating a Protection Group snapshot, the write operations on the database need to be suspended and the database prepared for a storage snapshot backup. The database will be frozen for write input with no impact to read output, until the BACKUP DATABASE [UserDB] TO DISK...WITH METADATA_ONLY command is run. This step should be automated to reduce the duration of the write input/output freeze. To do this, execute the following T-SQL command, substituting the database name for [UserDB]:
ALTER DATABASE [UserDB] SET SUSPEND_FOR_SNAPSHOT_BACKUP = ON -
Go to Source Protection Group Snapshots, click the ellipsis, and then select Create to create a snapshot.Figure 16. Creating a snapshot.
-
Enter Optional Suffix information for the snapshot's name, and then click Create. The snapshot name (with optional suffix) is displayed for Source Protection Group Snapshots. Take note of this snapshot name for reference in other examples in this document.Figure 17. Naming a snapshot in the Create Snapshot dialogue.
-
After the protection group snapshot has been created, finish the snapshot backup process by executing the following com-mand, substituting the database name and location for the metadata file (the backup file is required to put the database into restoring mode for point in time recovery):
BACKUP DATABASE [UserDB] TO DISK='C:\SQL\UserDB.bkm' WITH METADATA_ONLY
Creating Manual Log Backups
A log backup is only required if you are performing a point-in-time recovery. Log backups are scheduled separately from snapshot backups; however, if a manual log backup needs to be taken, execute the following command, substituting [UserDB] for the relevant database name:BACKUP LOG [UserDB] FROM DISK = 'C:\SQL\UserDB-log.bkm'
Recovering An Application-consistent Snapshot Backup to the Same Host
Scenario: Snapshots are highly efficient and quick when data must be restored directly to the original database instance or host. These are the steps to recover data from an application-consistent snapshot to the original volume on the same array attached to the original host. This scenario uses T-SQL snapshot backups, introduced in SQL Server 2022.
Prerequisites: Running SQL Server 2022 or later with a database and FlashArray with volumes assigned to a protection group.
Usage notes: Each section of this example is meant to be run sequentially.
-
If the database being restored is available, online, and usable, ensure that it is set offline using the following command:
ALTER DATABASE [UserDB] SET OFFLINE WITH ROLLBACK IMMEDIATE -
The Windows disks that the volumes correspond to also need to be offline prior to the volume snapshot being restored. To offline the disks, in the Disk Management application on the host, right-click on the disk, and then select Offline.Figure 18. Volumes attached to the SQL Server host as offline drives.
-
In the FlashArray user interface, select the protection group snapshot, and then select the snapshot link to open the detail view for the volume snapshots contained within the protection group.Figure 19. The snapshot created for the source protection group.
-
To restore the snapshots to the original volumes, select the overwrite icon.Figure 20. Restoring a snapshot using the overwrite icon.
-
At the Restore Volume from Snapshot prompt, review the volume name, and then select Restore to confirm.Figure 21. The Restore Volume from Snapshot dialogue.
-
Repeat steps 4–5 for each volume in the protection group snapshot.
-
Overwriting volumes with snapshots does not immediately destroy the state of the volume when it is overwritten. Instead, a snapshot of the state of the volume is taken and then placed in the Destroyed Volume Snapshots section, which is then retained for a set period prior to being eradicated.Figure 22. The Destroyed Volume Snapshots section of the Volumes tab.
-
If step 2 has previously set the volumes to offline, these volumes now need to be set online once more. In Disk Management, right-click each offline disk, and then select Online.Figure 23. Bringing the disks online in Windows Disk Manager.
-
If the disk does not have a drive letter assigned, right-click the disk, and then choose Change Drive Letter and Paths.Figure 24. Volumes attached as drives on the SQL Server host.
-
If the disks come online in a read-only state, right-click each disk and set it to read/write.
-
Restore the database with METADATA only using the NORECOVERY option:
RESTORE DATABASE [UserDB] FROM DISK = 'E:\SQL\UserDB.mdf' with METADATA_ONLY, REPLACE, NORECOVERY; -
(Optional) For point-in-time recovery, restore the transaction log to the database on the new SQL Server host by executing the following command on the new host. Replace the database name and transaction log backup location with the relevant values:
RESTORE LOG [UserDB] FROM DISK = 'C:\SQL\UserDB-log.bkm' WITH NORECOVERY; -
Finish the database snapshot restore:
RESTORE DATABASE [UserDB] with RECOVERY;
Using Crash-consistent Snapshots to Clone to a New User Database
Scenario: FlashArray snapshots offer a fast and efficient method to clone SQL Server databases for a variety of use cases. As database sizes increase, administrators can use crash-consistent snapshots to reduce the complexity of copying large amounts of data and cloning that data for different consumers. For example, the volumes supporting one or more databases can be cloned to different SQL Server database instances for development, test, and reporting environments.
Prerequisites: Running SQL Server with a database and FlashArray with a protection group and volume. This procedure is based on a Everpure volume database refresh script that is available on GitHub.
-
See the section on Creating a Crash-consistent Snapshot to create a crash-consistent snapshot.
-
To copy the volumes, select the protection group snapshot from the previously created snapshot, and then select the copy icon to copy to a new volume.Figure 25. Volumes attached to the SQL Server host as offline drives.
-
Copy the selected snapshot to a new volume by creating a name and then clicking Copy.Figure 26. The snapshot created for the source protection group.
-
Repeat steps 3 and 4 for each volume in the protection group, providing a unique name for each volume.Figure 27. Restoring a snapshot using the overwrite icon.
-
Once the snapshot is cloned/copied (a data reduced copy), verify that a new host or hosts are available. These hosts can be part of a dev or test environment or a reporting server. In the FlashArray user interface, select Storage in the navigation pane, and then click the Hosts tab.Figure 28. The Restore Volume from Snapshot dialogue.
-
Verify that the new host appears in the Hosts group, and then attach the host from the Hosts tab.
-
On the new host, open the Disk Management application. The volumes appear as offline disks.Figure 29. The Destroyed Volume Snapshots section of the Volumes tab.
-
Right-click each offline disk, and then select Online.Figure 30. Bringing the disks online in Windows Disk Manager.
-
If the disk does not have a drive letter assigned, right-click the disk, and then choose Change Drive Letter and Paths.Figure 31. Volumes attached as drives on the SQL Server host.
-
If the disks come online in a read-only state, right-click each disk and set it to read/write.
-
Attach the database to the current instances by running the CREATE DATABASE T-SQL command with the FOR ATTACH property and the appropriate paths to each datafile.
The database is now available on the new host. The snapshot can now be cloned to other dev, test, or reporting SQL Server hosts.CREATE DATABASE [UserDB] ON ( FILENAME = 'E:\SQL\UserDB.mdf' ),( FILENAME = 'F:\SQL\UserDB_log.ldf' ) FOR ATTACH
Seeding an Availability Group from a Snapshot
Scenario: SQL Server availability groups provide high availability and disaster recovery for SQL Server databases by enabling database administrators to replicate databases across multiple SQL Server instances and create a failover environment for a group of databases. However, the process of seeding replicas can be time-consuming. Even on fast networks, the sheer size of the data involved in the operation can result in seeding secondary replicas taking hours or even days. This challenge is further exacerbated if the primary replica contains very large databases or if seeding occurs across slow or wide area networks between data centers.
Everpure snapshots and the new T-SQL snapshot backup capability in SQL Server 2022 can help alleviate the difficulties in seeding secondary replicas.
Prerequisites:
-
An existing SQL Server availability group: This example requires at least two SQL Server 2022 replicas configured in an availability group, with one server acting as the primary replica, and the other acting as the secondary replica.
-
Volumes assigned to a protection group: See the section on Initial Steps: Create a Protection Group for Volumes to create a protection group and add volumes to the protection group. These volumes are already attached to the hosts and assumed to be online.
-
Asynchronous snapshot replication between two FlashArray systems: The FlashArray systems replicate the volumes in the protection group.
-
Log backups disabled: Log backups must be disabled on the primary replica during the seeding process because the log backup must be restored on the secondary replica for the availability group to synchronize.
-
PowerShell modules: This procedure was implemented using the Everpure PowerShell SDK2 and dbatools modules. The dbatools modules were used to open and hold a connection to the SQL Server instance.
Setting Up the Environment
Import-Module dbatools
Import-Module PureStoragePowerShellSDK2
$Credential = Get-Credential # Get the user credentials
$PrimarySqlServer = 'SqlServer1.example.com' # SQL Server primary replica
$SecondarySqlServer = 'SqlServer2.example.com' # SQL Server secondary replica
$AgName = 'ag1' # Name of the availability group
$DbName = 'UserDB' # Name of database to place in the
#availability group
$BackupShare = '\\FileServer1\SHARE\BACKUP' # File location for metadata backup file.
$PrimaryArrayName = 'flasharray1.example.com' # FlashArray containing #the primary replica
$SecondaryArrayName = 'flasharray2.example.com' # FlashArray containing the volumes
for # secondary replica
$SourcePGroupName = 'SqlServer1_Pg' # Name of the protection group on
#primary array
$TargetPGroupName = 'flasharray1:SqlServer1_Pg' # Name of the protection group replicated
FlashArray1 to FlashArray
formatted as #ArrayName:ProtectionGroupName
$PrimaryFaDbVol = 'Fa1_Sql_Volume_1' # Volume name on FlashArray containing
# the primary replica database files
$SecondaryFaDbVol = 'Fa2_Sql_Volume_1' # Volume name on FlashArray containing
# the primary replica database files
$TargetDisk = 'serialnum' # The serial number of the Windows # volume containing database files
Taking a Snapshot on the Primary Replica FlashArray Protection Group
The next step is to take a snapshot on the FlashArray that contains the primary replica's database file. This is done by freezing the write input/output on the database, taking a snapshot of the protection group that replicates to the secondary replica's FlashArray, and then creating a metadata backup that contains the contents of the backup.
The following commands connect to the primary replica FlashArray, pause the write input/output on the primary replica, and take a snapshot of the primary replica's protection group. The final command takes a metadata backup of the database and unfreezes the database if it is successful.
# Connect to the primary replica FlashArray
$FlashArrayPrimary = Connect-Pfa2Array –EndPoint $PrimaryArrayName -Credential $Credential -IgnoreCertifica-teError
#Suspend write operations to the database on the primary replica
$Query = "ALTER DATABASE [$DbName] SET SUSPEND_FOR_SNAPSHOT_BACKUP = ON"
Invoke-DbaQuery -SqlInstance $SqlInstancePrimary -Query $Query -Verbose
#Create a snapshot of the primary replica's protection group
$SourceSnapshot = New-Pfa2ProtectionGroupSnapshot -Array $FlashArrayPrimary -SourceName $SourcePGroupName
-ForReplication $true -ReplicateNow $true
#Create a metadata backup, which will unfreeze the database if successful
$BackupFile = "$BackupShare\$DbName$(Get-Date -Format FileDateTime).bkm"
$Query = "BACKUP DATABASE $DbName
TO DISK='$BackupFile'
WITHMETADATA_ONLY,MEDIADESCRIPTION='$($SourceSnapshot.Name)|$($FlashArrayPrimary.ArrayName)'"
Invoke-DbaQuery -SqlInstance $SqlInstancePrimary -Query $Query -Verbose
Replicating the Snapshot from the Primary Replica FlashArray to the Secondary Replica FlashArray
# Connect to the secondary replica FlashArray
$FlashArraySecondary = Connect-Pfa2Array –EndPoint $SecondaryArrayName -Credential $Credential -IgnoreCer-tificateError
#Write-Warning "Obtaining the most recent snapshot for the protection group..."
$TargetSnapshot = $null
do {
Write-Warning "Waiting for snapshot to replicate to target array..."
$TargetSnapshot = Get-Pfa2ProtectionGroupSnapshotTransfer -Array $FlashArraySecondary -Name $TargetP-GroupName |
Where-Object { $_.Name -eq "$TargetPGroupName.$($SourceSnapshot.Suffix)" }
if ( $TargetSnapshot -and $TargetSnapshot.Progress -ne 1.0 ){
Write-Warning "Snapshot $($TargetSnapshot.Name) found on Target Array...replication progress is $($Tar-getSnapshot.Progress)"
Start-Sleep 3 }
} while ( [string]::IsNullOrEmpty($TargetSnapshot.Completed) -or ($TargetSnapshot.Progress -ne 1.0) )
Write-Warning "Snapshot $($TargetSnapshot.Name) replicated to Target Array. Completed at $($TargetSnapshot.Completed)"
Cloning the Volumes on the Secondary Replica FlashArray
# Connect to the secondary replica FlashArray and take the volume offline
Invoke-Command -Session $SecondarySession -ScriptBlock { Get-Disk | Where-Object { $_.SerialNumber -eq $us-ing:TargetDisk } | Set-Disk -IsOffline $True }
# Overwrite the volumes on the secondary replica FlashArray with the snapshot
New-Pfa2Volume -Array $FlashArraySecondary -Name $SecondaryFaDbVol -SourceName ($TargetSnapshot.Name + ".$PrimaryFaDbVol") -Overwrite $true
#Bring the volume online on the secondary replica FlashArrayInvoke-Command -Session $SecondarySession -ScriptBlock { Get-Disk | Where-Object { $_.SerialNumber -eq $us-ing:TargetDisk } | Set-Disk -IsOffline $False }
Restoring the Database and Log Backup on the Secondary Replica
# Restore the database with the NORECOVERY option
$Query = "RESTORE DATABASE [$DbName] FROM DISK = '$BackupFile' WITH METADATA_ONLY, REPLACE, NORECOVERY"
Invoke-DbaQuery -SqlInstance $SqlInstanceSecondary -Database master -Query $Query -Verbose
#Take a log backup from the primary replica
$Query = "BACKUP LOG [$DbName] TO DISK = '$BackupShare\$DbName-seed.trn' WITH FORMAT, INIT"
Invoke-DbaQuery -SqlInstance $SqlInstancePrimary -Database master -Query $Query -Verbose
#Restore the log backup to the secondary replica
$Query = "RESTORE LOG [$DbName] FROM DISK = '$BackupShare\$DbName-seed.trn' WITH NORECOVERY"
Invoke-DbaQuery -SqlInstance $SqlInstanceSecondary -Database master -Query $Query -Verbose
Start the Seeding on the Secondary Replica
# Set the seeding mode on the secondary replica to manual
$Query = "ALTER AVAILABILITY GROUP [$AgName] MODIFY REPLICA ON N'$PrimarySqlServer' WITH (SEEDING_MODE = MANUAL)"
Invoke-DbaQuery -SqlInstance $SqlInstancePrimary -Database master -Query $Query -Verbose
#Add the database to the availability group
$Query = "ALTER AVAILABILITY GROUP [$AgName] ADD DATABASE [$DbName];"
Invoke-DbaQuery -SqlInstance $SqlInstancePrimary -Database master -Query $Query -Verbose
#Start the availability group replication process
Invoke-DbaQuery -SqlInstance $SecondarySqlInstance -Database master -Query 'ALTER DATABASE [$DbName] SET HARD AVAILABILITY GROUP = [ag1];'
#Check the status of the availability group to see if the primary and secondary replicas are synchronizedGet-DbaAgDatabase -SqlInstance $SqlInstancePrimary -AvailabilityGroup $AgNameOnce the availability group status shows that the replicas are synchronized, the availability group is functioning.