Category: Backup & Recovery

  • SQL Server – Backup Verification Script – Based on conditions

    Problem: There are scenarios where you have to verify the multiple SQL Server instances backup within limited time frame whether it meet your organization backup criteria standards or not.

    Solution:
    You can execute below T-SQL script to check the backup status as per your organization standard by assigning backup conditions value to backup criteria variable. You can execute below script either by connecting each SQL Server Instance or execute it by registering all the SQL Server Instances in Central Management Server (CMS).

    Below T-SQL Script work with all the versions of MSSQL Server and verify FULL, Differential and Transaction Log Backup. It will check the max backup date and evaluate result based on condition as backup is Passed or Not.

    Backup Verification Script will return below values. Column names are self-explanatory.
    • SQLInstanceName
    • DatabaseName
    • db_create_ddate
    • DB_Recovery_Model
    • dbstatus
    • windowsservername
    • productversion
    • productlevel
    • edition
    • current_datetime
    • last_full_backup_date
    • last_diff_backup_date
    • last_tran_backup_date
    • days_since_last_full_backup
    • days_since_last_diff_backup
    • hours_since_last_tranlog_backup
    • Full_Backup_Stauts
    • Diff_Backup_Stauts
    • Log_Backup_Stauts
    • full_backup_location
    • diff_backup_location
    • tlog_backup_location

    You have to specify the backup condition based on your organization standard by assigning values to below variables.

    <br />
    declare @full_backup_criteria_in_days int<br />
    declare @diff_backup_criteria_in_days int<br />
    declare @log_backup_criteria_in_hours int</p>
    <p>-- specify the backup criteria<br />
    set @full_backup_criteria_in_days = 7 -- 7 day older Full Backup is Pass<br />
    set @diff_backup_criteria_in_days = 1 -- 1 day older Diff Backup is Pass<br />
    set @log_backup_criteria_in_hours = 2 -- 2 hours older Log Backup is Pass<br />
    

    Complete Backup Verification Script

    <br />
    declare @full_backup_criteria_in_days int<br />
    declare @diff_backup_criteria_in_days int<br />
    declare @log_backup_criteria_in_hours int</p>
    <p>-- specify the backup criteria<br />
    set @full_backup_criteria_in_days = 7 -- 7 day older Full Backup is Pass<br />
    set @diff_backup_criteria_in_days = 1 -- 1 day older Diff Backup is Pass<br />
    set @log_backup_criteria_in_hours = 2 -- 2 hours older Log Backup is Pass</p>
    <p>select<br />
               serverproperty('servername') as SQLInstanceName,<br />
               quotename(bkup_full.[database_name]) as DatabaseName,<br />
                ( select    sdb.crdate<br />
                  from      [master]..[sysdatabases] sdb<br />
                  where     sdb.name = bkup_full.[database_name]<br />
                ) as [db_create_ddate],<br />
               databasepropertyex(bkup_full.[database_name],'recovery')  as DB_Recovery_Model,<br />
               databasepropertyex(bkup_full.[database_name],'status')  as dbstatus,<br />
               case serverproperty('isclustered')<br />
                            when 1 then cast(serverproperty('computernamephysicalnetbios') as varchar)<br />
                            when 0 then cast(serverproperty('machinename') as varchar)<br />
               end as windowsservername,<br />
               serverproperty('productversion') as productversion,<br />
               serverproperty('productlevel') as productlevel,<br />
               serverproperty('edition') as edition,</p>
    <p>           current_timestamp as current_datetime,</p>
    <p>		   bkup_full.[backup_finish_date] as [last_full_backup_date],<br />
    		   bkup_diff.[backup_finish_date] as [last_diff_backup_date] ,<br />
    		   bkup_log.[backup_finish_date] as [last_tran_backup_date] ,</p>
    <p>           datediff(dd, bkup_full.[backup_finish_date], current_timestamp) as [days_since_last_full_backup] ,<br />
               datediff(dd, bkup_diff.[backup_finish_date], current_timestamp) as [days_since_last_diff_backup] ,<br />
               datediff(hh, bkup_log.[backup_finish_date], current_timestamp) as [hours_since_last_tranlog_backup] ,</p>
    <p>           case<br />
                            when datediff(dd, bkup_full.[backup_finish_date], current_timestamp) &lt;= @full_backup_criteria_in_days<br />
                            then 'Pass'<br />
                            else 'Fail'<br />
                            end as Full_Backup_Stauts,</p>
    <p>            case<br />
                            when datediff(dd, bkup_diff.[backup_finish_date], current_timestamp) &lt;= @diff_backup_criteria_in_days  then 'Pass'<br />
                            else case when quotename(bkup_full.[database_name]) IN ('[master]') then 'N/A' else 'Fail' end<br />
                            end as Diff_Backup_Stauts,</p>
    <p>            case<br />
                            when datediff(hh, bkup_log.[backup_finish_date], current_timestamp) &lt;= @log_backup_criteria_in_hours  then 'Pass'<br />
                            else case when databasepropertyex(bkup_full.[database_name],'recovery') = 'SIMPLE' then 'N/A' else 'Fail' end<br />
                end as Log_Backup_Stauts,            </p>
    <p>            ( select top 1 [physical_device_name]<br />
                  from      [msdb]..[backupmediafamily] bkup_media_family<br />
                  where     bkup_media_family.[media_set_id] = bkup_full.[media_set_id]<br />
                ) as [full_backup_location] ,<br />
                ( select top 1  [physical_device_name]<br />
                  from      [msdb]..[backupmediafamily] bkup_media_family<br />
                  where     bkup_media_family.[media_set_id] = bkup_diff.[media_set_id]<br />
                ) as [diff_backup_location] ,<br />
                ( select top 1  [physical_device_name]<br />
                  from      [msdb]..[backupmediafamily] bkup_media_family<br />
                  where     bkup_media_family.[media_set_id] = bkup_log.[media_set_id]<br />
                ) as [tlog_backup_location]</p>
    <p>    from    [msdb]..[backupset] as bkup_full<br />
                left join [msdb]..[backupset] as bkup_log on bkup_log.[database_name] = bkup_full.[database_name]<br />
                                                           and bkup_log.[server_name] = bkup_full.[server_name]<br />
                                                           and bkup_log.[type] = N'L'<br />
                                                           and bkup_log.[backup_finish_date] = ( (select  max([backup_finish_date])<br />
                                                                                                from    [msdb]..[backupset] b2<br />
                                                                                                where   b2.[database_name] = bkup_full.[database_name]<br />
                                                                                                        and b2.[server_name] = bkup_full.[server_name]<br />
                                                                                                        and b2.[type] = N'L') )<br />
                left join [msdb]..[backupset] as bkup_diff on bkup_diff.[database_name] = bkup_full.[database_name]<br />
                                                           and bkup_diff.[server_name] = bkup_full.[server_name]<br />
                                                           and bkup_diff.[type] = N'I'<br />
                                                           and bkup_diff.[backup_finish_date] = ( (select max([backup_finish_date])<br />
                                                                                                from    [msdb]..[backupset] b2<br />
                                                                                                where   b2.[database_name] = bkup_full.[database_name]<br />
                                                                                                        and b2.[server_name] = bkup_full.[server_name]<br />
                                                                                                         and b2.[type] = N'I') )<br />
        where   bkup_full.[type] = N'D'<br />
                and bkup_full.[backup_finish_date] = ( (select  max([backup_finish_date])<br />
                                                     from   [msdb]..[backupset] b2<br />
                                                     where  b2.[database_name] = bkup_full.[database_name]<br />
                                                            and b2.[server_name] = bkup_full.[server_name]<br />
                                                            and b2.[type] = N'D') )<br />
                and exists ( select [name]<br />
                             from   [master]..[sysdatabases]<br />
                             where  [name] = bkup_full.[database_name] )<br />
                and bkup_full.[database_name] &lt;&gt; N'tempdb'<br />
    

    This script is the “Sustain” dimension

    A backup you haven’t verified is a hope, not a plan — and this script is exactly the kind of continuous, boring verification that keeps a data platform honest. In my AI risk framework, this is Sustain: monitor and govern, so the other controls hold up in production. The framework: R-I-S-K-S: The 5-Dimension Framework for AI Risk Management

  • How to rerun the SQL Agent job step automatically in case of failure?

    How to rerun the SQL Agent job step automatically in case of failure?

    Take an example of long running the job with the multiple steps where each steps has dependency of the previous steps successful execution. In case if any of the job step fails, entire job will fail and DBA has to manually run the job to complete it and job will run from the beginning.

    There could be multiple reason of the job failure for example deadlock, network issue, requested backup file is not available as backup is still in progress etc.

    You can avoid this situation using he job step advanced option Retry Attempts and Retry Interval (minutes)

    Before the job step considered to have failed, you can set the number of times that the job step should be repeated using “Retry attempts” option. You can also use the Retry interval (minutes) option the number of minutes interval between retry attempts.

    Default value of “Retry attempts” and “Retry interval (minutes)” is 0

    For example if I will specify the “Retry attempts” to 3 and “Retry interval (minutes)” to 15 minutes, job step will re-run of up to 3 failure at 15 minutes interval.

    Capture

  • Best Practices SQL Server Transaction Log

    Background
    In SQL Server Database Recovery model will decide how the transaction log will be logged in transaction log file. Transaction log file extension is .LDF

    Full – Transaction log is fully logged (Can take log backup)
    Bulk Logged – Bulk transaction is minimally logged (Can take log backup)
    Simple – Transaction log will be truncated on checkpoint

    In transaction log file transactions are sequentially logged, every record in transaction log file is uniquely identified by log sequence number (LSN). LSN data type is Numeric (25,0)

    You can follow below best practices for the transaction log file

    1. Don’t create multiple log files : As transactions will be logged into log file sequential manner it would not help for data stripping across multiple files
    2. Keep the transaction log file on the separate drive
    3. Identify the RPO and RTO for the database and according to that choose the recovery model and correct log backup strategy
    4. RAID 1 + 0 is high recommended for transaction log
    5. AUTO SHRINK should be always off on the database
    6. Pre-allocate the space to transaction log file, it will improve the performance. Don’t depend on the auto growth option.
    7. Always set the values of Initial size, max size and growth property of the transaction log file
    8. Always set auto growth value, don’t set in percentage
    9. Transaction Log file internal fragmentation can also lead the performance and database recovery issue. Database should not have an excessive number of Virtual Log Files (VLFs) inside the Transaction Log. Having a large number of small VLFs can slow down the recovery process that a database goes through on startup or after restoring a backup. Make sure transaction log initial size and log growth defined well to avoid internal fragmentation
    10. External fragmentation can be removed by using disk defragmentation utility
    11. In case of Transaction log full, please use below query to check the cause of the log full and take the decision accordingly.

    SELECT name ,
    recovery_model_desc ,
    log_reuse_wait_desc
    FROM sys.databases
    WHERE name = @DatabaseName

    Your transaction log is an AI pipeline’s raw material

    Everything on this list still applies. What is new is who is reading the log: Change Data Capture scans this same transaction log to feed real-time pipelines — including the embedding and RAG pipelines that need fresh data. Keep the log healthy (right-sized, right recovery model, no VLF sprawl) and your downstream AI stays fresh. On the retrieval side: The 9 Types of RAG, Explained

  • Stripping SQL Server Database Backup to Multiple Files

    Stripping Database backup to multiple files and on different drives will make the backup speed faster and will reduce the backup duration.

    Check the below sample script for the backup and restore. You can perform the same task using SSMS GUI as well.
    Backup Script

    BACKUP DATABASE [SQLDBPool] TO  
    DISK = N'C:\JSpace\Backup\SQLDBPool1.bak',  
    DISK = N'C:\JSpace\Backup\SQLDBPool2.bak',  
    DISK = N'C:\JSpace\Backup\SQLDBPool3.bak'
    WITH NOFORMAT, 
    NOINIT,  
    NAME = N'SQLDBPool-Full Database Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10
    GO
    

    Restore Script

    RESTORE DATABASE [SQLDBPool] FROM  
    DISK = N'C:\JSpace\Backup\SQLDBPool1.bak',  
    DISK = N'C:\JSpace\Backup\SQLDBPool2.bak',  
    DISK = N'C:\JSpace\Backup\SQLDBPool3.bak'
    WITH  FILE = 1,  NOUNLOAD,  STATS = 10,replace
    GO
    
  • LiteSpeed – Restore Script

    Script to check the backup file – data and log file information

    exec master.dbo.xp_restore_filelistonly
    @filename ='<BackupFilePath>\<BackupFileName.bak>'
    

    Script to check the backup file header information

    exec master.dbo.xp_restore_headeronly
    @filename ='<BackupFilePath>\<BackupFileName.bak>'
    

    Script to verify the backup file if the backup file is valid or not

    EXEC master.dbo.xp_restore_verifyonly @filename ='<BackupFilePath>\<BackupFileName.bak>'
    

    Script to restore database using Full backup with the default options

    exec master.dbo.xp_restore_database
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.bak>'
    

    Script to restore database using Full backup with file move option

    exec master.dbo.xp_restore_database
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.bak>',
    @with = 'move "logical filename" to "physical file location.mdf"',
    @with = 'move "logical filename" to "physical file location.ldf"'
    

    Script to restore database using Full backup with replace option

    exec master.dbo.xp_restore_database
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.bak>',
    @with = 'replace',
    @with = 'move "logical filename" to "physical file location.mdf"',
    @with = 'move "logical filename" to "physical file location.ldf"'
    

    Script to restore Full backup with no recovery

    exec master.dbo.xp_restore_database
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.bak>',
    @with = 'replace',
    @with = 'move "logical filename" to "physical file location.mdf"',
    @with = 'move "logical filename" to "physical file location.ldf"',
    @with='NORECOVERY'
    

    Script to restore log backup with no recovery

    EXEC master.dbo.xp_restore_log
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.trn>',
    @with ='NORecovery'
    

    Script to restore log backup with recovery

    EXEC master.dbo.xp_restore_log
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.trn>',
    @with ='Recovery'
    

    Script to do point in time recovery

    EXEC master.dbo.xp_restore_log
    @database = '<dbname>',
    @filename = '<BackupFilePath>\<BackupFileName.trn>',
    @with ='Recovery',
    @with = 'STOPBEFOREMARK = <LogMark>'