In this blog post we will discuss 7 points to remember while taking effective backup for SQL Server.

1. Always take backup during off business hours, as backup operations generate IO activities.
2. Take backup on a different drive from your data and log file, to get maximum IO efficiency from your database.
3. Make sure to check the status of the backup job frequently. If the status of the backup job is failed, you may want to fix it as soon as possible.
4. Check the database recovery model under Database Properties >> Recovery Model. You want to set it appropriately based on your business needs.
5. Take frequent log backups so your log files do not grow too big.
6. Set appropriate frequency of your database backup. (e.g. Full backup – daily, differential backup – hourly, log backup – every 10 minutes)
7. Test your database backup by restoring it in your test environment.

I want to specifically stress on the 7th item. I frequently do consulting for organization for Performance Tuning. While we are doing SQL Server Performance Tuning work, we often end up taking about backup. I often see that most of the organizations follow many of the first 6 points, but the last 7th point is very often implemented in many organizations.

I have seen when disaster happens, DBA team trying to restore their database to production just to realize that there is some error in the backup operation. When they realize this error it is often too late.

It is always better to proactive about your database backup strategy. I recommend everyone to read following article which discusses backup timeline.

What is a Backup Timeline for Restore Process? – Interview Question of the Week #092

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – 7 Important Things to Remember While Taking Effective Backup

What are Ghost Records and How to Clean Them Up?

I loved this question when I received it last week in my SQL Server Performance Tuning Practical Workshop.

Well, in simple possible words – Ghost records are those records which are deleted logically from the table but physically still exists on the disk.

For example, if there is a large amount of insert, update or delete happens on your database. SQL Server indeed reflects the changes when you select the data. However, it does not delete the record physically. These records are called Ghost Records.

Here is the script which you can use to identify the ghost record count.

SELECT
db_name(database_id),
object_name(object_id),
ghost_record_count,
version_ghost_record_count
FROM sys.dm_db_index_physical_stats(DB_ID(N'SQLAuthority'), OBJECT_ID(N'TestTable'), NULL, NULL , 'DETAILED')
GO

Here is another script which you can use to clear all the ghost records from your database.

USE master;
GO
EXEC sp_clean_db_free_space @dbname = N'SQLAuthority'

I see lots of people executing above script before they take backup. Honestly, I think it is alright to take backup without executing above SP as SQL Server frequently clears up all the ghost records. However, with that said, if you feel comfortable running above SP before taking a backup, I think it is totally fine too. Just an extra few minutes before the backup would not hurt you if your backup is finished in your maintenance window.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – What Are Ghost Records and How to Clean Them Up?

In this blog post we will discuss how to Enable Backup Compression on by default.

SQL Server 2008 had introduced compressed backup features and nowadays I see quite often enabled at most of my customers where I go for consulting. When we install SQL Server, by default this option is turned off. So if you want to take backup with compression you can write following command.

BACKUP DATABASE [AdventureWorks2014] TO DISK = N'D:\AW1.bak' WITH COMPRESSION
GO

Pay special attention to keyword Compression in the above syntax. This becomes very tedious and cumbersome after a while as every single time when we have to take backup we have to specify that particular command. Additionally, when people upgrade their system and have 100s of the different backup job, they wish that they would have a single switch which they can enable or disable and the effect is applied to all the database backup by default.

The good part is that SQL Server has a default switch at the server level which we can enable or disable.

EXEC sys.sp_configure N'backup compression default', N'1'
GO
RECONFIGURE WITH OVERRIDE
GO

When you run above script, it will enable compression by default on all of your database, even though the keyword “WITH COMPRESSION” is not used.

If you have enabled this switch at server level and if you want to disable any particular database for a single database, you can specific “WITH NO_COMPRESSION” keyword to disable the compression for that database.

You can enable the same settings with SQL Server Management Studio as well. Here is the property of the server where you can enable backup compression setting by default.

Please note when you enable compression you may notice your backup takes less space and completes faster along with that there will be a marginal increase in the CPU consumption. Personally I have enabled Backup Compression on all of my databases.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – How to Enable Backup Compression On by Default

Question: How to Delete a Single Table From SQL Server Backup File (.bak)?

Answer: It is not possible by directly editing the backup file .bak.

I had actually heard this question during a recent interview, however, the only reason I have not included this question in the interview question and answer series is that I believe it is not worth that important. A good interviewer will never ask you this question in an interview as he knows that there is no value to this question in the real world.

I can totally see why sometimes users want to delete a particular backup from the backup file, however, it is not possible to do so at all. If you want to delete any table in backup, well, first you should delete that from your database and take backup.

If your intention is to reduce the size of the backup, I suggest you opt for compressed backup option available by default in SQL Server.

I guess, that’s it. I have nothing more to add in this subject. If you thought, I have a negative tone of this blog post, honestly that is not my intention at all.

Here are few related blog posts which you should read:

• Configure the Backup Compression Default Server Configuration Option
• How to Enable Backup Compression On by Default
• SQL SERVER – Playing with Backups and Compression
• Effect of Compressed Backup Setting at Server Level on Database Backup

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – How to Delete a Single Table From SQL Server Backup File (.bak)

Here is an email I received about database corruption and that has lead me to write this quick blog post about database corruption.

“Hi Pinal,

We learned that our database got corrupted. Once we learned this, we immediately ran CHECKDB with Repair option and now we are not sure what is happening. We are not able to access our database at all.

What should I do?”

Well, this is just a one message, but quite often I get similar messages. With the information provided in the email, one can’t move forward so they had to engage me in the consultation and we eventually recovered their database. However, during the consultation, I realized that they had done a quite a lot of attempts to recover the database before they reached out to me. Honestly, instead of doing all the efforts if they would have reached out to me without any changes to their system, I would have helped them faster.

During this database corruption recovery engagement, I realized that I should write this blog post where I will write what one should not do when they detect database corruption.

Don’t Shutdown SQL Server

When the database corruption is detected, the very first reaction of users is to shut down the SQL Server. The most popular argument which I hear is that they want to shutdown SQL Server to prevent the further corruption of the database. This is indeed a good intent, but it is not necessarily the best suggestion. There are enough cases where I have seen database is not able to come back up after it was shut down. You can render your database in an inconsistent state if the database recovery is going on.

Don’t Restart SQL Server

We all have habits when something does not work consistently we restart it. It may be our phone, computer, network or our sometimes I have seen people applying this to their vehicles too. This does not apply for corrupted database on SQL Server. There are zero chances that your corruption will be recovered when you restart your SQL Server. You may end up with some other problems when you restart without understanding what is going on behind the scene.

Don’t Upgrade Your SQL Server

Before I write I must admin that in my career, I have only heard this once so far. However, there are chances that many might have thought about this one, or have done in the past but never shared the story. One of my customers tried to upgrade their server when they faced corruption. They thought that by upgrading to the latest version of SQL Server, they may get rid of the corruption as SQL SErver now might know how to handle corruption better in the next version of SQL Server. Honestly, this is not the route you want to go when you are facing corruption situation.

Repairing Corrupted Database

You should consider the CHECKDB option to repair eventually if your all the other efforts fail. In the most cases without understanding what actually this option do, many DBA end up loosing their data. One of my customers once allowed repair with data loses on their database and their database was not showing any errors with CHECKDB. However, their application did not work as it missed the important table in the database.

Use this option once, you are absolutely sure that you have no other route left besides repairing the corrupted database.

Third Party Tools

There are lots of third party tools available, which claim that they can repair your database. There are cases where I have seen them working perfectly fine. They often attract customers by showing a part of the database during trial, but when the user tries to attempt to recover the database after purchasing the full license they often do not produce desired results.

Instead of downloading any third party tool and risking your database, I suggest you reach out to an expert consultant first and under their guidance, use the tool. I also keep a list of good tools with me. If you want to know more about it, you may reach out to me directly at my email address.

Summary

Well, above the list was was about things which you should not do when you encounter database corruptions. You must plan your disaster recovery situation such a way that when you encounter such situation you exactly know what you would do and how you can recover.

If you are going to reach out to me for database, corruption, I am going to ask you one simple question – “How old is your last good full database backup?” Make sure that you have answer of this question in your email and I will be happy to help.

Here are two related blog posts:

• Backup Timeline and Understanding of Database Restore Process in Full Recovery Model
• Database Disaster Recovery Process
• Database going into Recovery Pending Automatically and Coming Back
• How to Recover Truncated or Deleted Data When a Database Backup is or is not Available?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – 5 Don’ts When Database Corruption is Detected

Just the other day one of my customers asked me that they are facing a situation where they have specified in their backup script RETAINDAYS option but it is actually not deleting their older database backups.

I get this question quite a lot of time. I believe the word RETAINDAYS gives impressions to user that when the SQL Server backup is created they remains available till the days the RETAINDAYS option suggests and after that it automatically gets deleted. 

Well, here is the truth – the assumption is absolutely WRONG.

If you want to delete your backup files which were created earlier, you may want to go with the route of sqlcmd or powershell.

The option RETAINDAYS achieves a very different purpose with Backup. It just prevents users to overwrite the backup file if the user is trying to do it with INIT option. Otherwise, it really does not do anything else. If the user is using the FORMAT option, the backup will be overwritten anyway. This means the use of RETAINDAYS is very much limited.

I guess that’s it. There is nothing much we can write about this option as it has very limited capabilities. I hope now onwards I will be able to point users to this blog post when they reach out to me with the question about this option.

Here is the working script of how this option is used in SQL Server.

BACKUP DATABASE AdventureWorks TO DISK = 'd:\adv.bak' WITH RETAINDAYS = 7

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – RETAINDAYS Does Not Delete Backup After x Days

One of my clients sent email to know my quick suggestion on one of the error messages. Let us learn how to fix error related to worker thread.

Cannot create worker thread.
BACKUP LOG is terminating abnormally.

As per my client, this error was not consistent, but it does happen multiple times a day.  They also told that they had many databases backing up at the same time.

WORKAROUND/SOLUTION

I told them to increase worker threads.

Below is the script to find current value

USE master
GO  
EXEC sp_configure 'show advanced options', 1;  
GO  
RECONFIGURE;
sp_configure 'max worker threads' 
GO  

If the value is zero, then SQL would create threads based on number of CPUs.

As per the article the default values for worker threads is set for 576 for 8 processors. I have asked to increase to 640 threads as they had many databases.

EXEC sp_configure 'max worker threads', 640;  
GO  
RECONFIGURE;  
GO  
EXEC sp_configure 'show advanced options', 0;  
GO  
RECONFIGURE;  
GO 

Have you ever changed default value of parameter? This is very unique scenario and I have not seen quite often in the industry. I would love to know your feedback about this blog post and share your story if you have ever faced such situation in the production environment.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Msg 3013 – Cannot Create Worker Thread. BACKUP LOG is Terminating Abnormally

One of my blog readers was having trouble with backup to URL using SAS token. He followed my below blog to fix his issue. SQL SERVER – Backup to URL error: Operating system error 50(The request is not supported.) Now let us look in this blog post how we can fix errors related to credentials.

So far, so good! Later he was unable to take a backup using maintenance plan. He contacted me, and it was sounding very interesting to me. He informed that he is getting below errors when he configures backup to URL using maintenance plan.

Executing the query “BACKUP DATABASE [master] TO URL = N’https://sqlau…” failed with the following error: “Use of WITH CREDENTIAL syntax is not valid for credentials containing a Shared Access Signature.

BACKUP DATABASE is terminating abnormally.”. Possible failure reasons: Problems with the query, “ResultSet” property not set correctly, parameters not set correctly, or connection not established correctly.

So, I decided to reproduce the issue and created a maintenance plan.

As he explained me, the backup failed! Looked into history

Use “View T-SQL” hyperlink and found below query.

BACKUP DATABASE [master] TO URL = N'LONG_URL_HERE_'
WITH CREDENTIAL = N'https://sqlauthsqlbackups.blob.core.windows.net/sqlauthcontainer',
NOFORMAT, NOINIT, NAME = N'master_backup_2017_12_07_222756_5893177',
SKIP, REWIND, NOUNLOAD, STATS = 10
GO

When I ran above in SSMS, it failed with error

Msg 3225, Level 16, State 1, Line 1
Use of WITH CREDENTIAL syntax is not valid for credentials containing a Shared Access Signature.
Msg 3013, Level 16, State 1, Line 1
BACKUP DATABASE is terminating abnormally.

WORKAROUND/SOLUTION

I looked around in the documentation and found this document.

To create a striped backup set, a SQL Server file-snapshot backup, or a SQL credential using Shared Access token, you must use Transact-SQL, Powershell or C# rather than the Backup task in Maintenance Plan Wizard.

This confirms that Microsoft doesn’t support taking backup to URL using SAS token via maintenance plan. Now, the only option left is to use T-SQL and schedule that as a job. Painful!

Well, not to worry! You can use below script to generate backup file name with date time and schedule it in the job. How to Add Date to Database Backup Filename? – Interview Question of the Week #109

Please comment if you have encountered a similar error and found some other solution.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Msg 3225: Use of WITH CREDENTIAL Syntax is Not Valid for Credentials Containing a Shared Access Signature

While preparing for a demo session for my client, I was playing with SQL Server backups on my lab server. I encountered an error which I am going to blog here. Here is the error about backup or restore history details.

Msg 3009, Level 16, State 1, Line 3
Could not insert a backup or restore history/detail record in the msdb database. This may indicate a problem with the msdb database. The backup/restore operation was still successful.

Here are the steps to reproduce the error.

USE [master]
GO
ALTER DATABASE [msdb] SET READ_ONLY WITH ROLLBACK IMMEDIATE 
GO

Once above command is completed successfully, we can try to take a backup of any database. I have taken backup of the master database using the command mentioned below.

BACKUP DATABASE MASTER TO DISK = 'master.bak'

Here is the error which we would receive.

Here is the complete text of the error message, which is truncated in the image.

Processed 456 pages for database ‘MASTER’, file ‘master’ on file 1.
Processed 2 pages for database ‘MASTER’, file ‘mastlog’ on file 1.
Msg 3906, Level 16, State 1, Line 3
Failed to update database “msdb” because the database is read-only.
Msg 3009, Level 16, State 1, Line 3
Could not insert a backup or restore history/detail record in the msdb database. This may indicate a problem with the msdb database. The backup/restore operation was still successful.
BACKUP DATABASE successfully processed 458 pages in 0.045 seconds (79.427 MB/sec).
The statement has been terminated.

WORKAROUND/SOLUTION

As we can see in the message, the backup has been successful but only logging to MSDB history tables didn’t happen. I must mention that above is just one of the causes of the error 3009. We need to look the error before that which is the root cause of the error. There are various probable causes and MSDB read-only was just one of them. Here are the possible errors:

1. Error: Time-out occurred while waiting for buffer latch type 4 for page (1:2516), database ID 4.

Action Needed: This error indicates corruption in the database and we should run DBCC CHECKDB for MSDB database.

2. Error: Database ‘msdb’ cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server ERRORLOG for details.

Action Needed: This error indicated that msdb is not opened. We should check ERRORLOG and find the cause.

If you are not bothered about the logging in MSDB database, then you can use trace flag 3001.

DBCC TRACEON(3001,-1)

Above will stay active til SQL is stopped. To keep it ON permanently, you need to put it as a startup parameter. Read my earlier blog about Trace Flags. SQL SERVER – What is Trace Flag – An Introduction

CLEANUP

Here is the command to cleanup MSDB mode.

USE [master]
GO
ALTER DATABASE [msdb] SET READ_WRITE WITH ROLLBACK IMMEDIATE 
GO

Have you used such undocumented trace flags in SQL Server in production?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – FIX: Msg 3009 – Could Not Insert a Backup or Restore History Detail Record in the MSDB Database

A prevalent question since many years often resurfaces in my mailbox. I am writing this blog here so in the future, I can direct users to this blog posts when they ask about Tail-Log Backups.

Tail-Log Backups were introduced in SQL Server 2005, and I find it handy feature for the backup process.

Before we start any explanation, let me say this first that, this feature will only work if you have taken once full backup of your database and your database is in either full or bulk-logged recovery models. If you have not taken the full backup of your database, this feature will not work for you. Now let us continue understanding what Tail-Log Backups is actually.

Tail-Log Backups

A tail-log backup captures any log records which has not yet been backed up by the last transactional log backup.

There are multiple scenarios when we need to take this tail-log backup, but we will today focus on the one scenario which is restoring the online database.

When your database is online after taking the backup, there may be more transactional operations happen. This means when we try to restore the database, it may be possible that we may lose some transactions and often you also see that error that your database should take the tail-log backup before restoring the database. This has often confused quite a many DBA so far.

Well, to avoid losing the last few transactions, it is always a good idea to take tail-log backup with a norecovery option.

Here is the script to take the tail log backups.

BACKUP LOG SQLAuthority
TO DISK = 'D:\Data\SQLAuthority_NORECOVERY.TRN'
WITH NORECOVERY, STATS = 10

Once you take your tail log backup, your database will move to restoring state, and no further transactions will be possible on it. Please note that when you restore your database, your sequence of restoring your databases should be as following:

• Restore your full backup
• Restore any differential backups if available (if not, move to next step)
• Restore all the transactional log in the order since last full backup
• Restore the Tail-Log Backup with recovery option

If you face any issue while restoring your database, do reach out to me and I will be happy to help you.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – What is Tail-Log Backups?

One of my blog readers contacted me via email for an error message which he was receiving while using the backup to URL feature. Using this feature, SQL Server can take database backup directly to Azure Blob Storage. I have already written a blog about few errors which I received earlier but this error was new. I replied him asking more details and it was an easy error to fix. In this blog we would discuss how to fix 3271, The remote server returned an error: (403) Forbidden.

Here is the command which I have used to reproduce the error.

BACKUP DATABASE SQLAuthority 
TO URL='https://sqlprodbkups.blob.core.windows.net/sqldbbackups/SQLAuthority.bak'
WITH CREDENTIAL ='AzBackupCred'

And the error message is below

Msg 3271, Level 16, State 1, Line 1
A nonrecoverable I/O error occurred on file “https://sqlprodbkups.blob.core.windows.net/sqldbbackups/SQLAuthority.bak:” Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (403) Forbidden..
Msg 3013, Level 16, State 1, Line 1
BACKUP DATABASE is terminating abnormally.

If you notice the error 3271 can come because of multiple reasons. An important piece of the error message is the error returned by the remote server. In this case, we are getting error 403 – Forbidden. Below is one earlier blog where I discussed Error – 400 – Bad Request SQL SERVER – Backup to URL Fails in Azure Resource Manager (ARM) Deployment Model with Error – (400) Bad Request

WORKAROUND/SOLUTION

As you can see there is nothing wrong with the command. As soon as we run the command, SQL Server uses the credential to connect to storage and performs a backup. When I checked the credential, it was created using Access Keys in the portal. In our case, we have given container name instead of storage account name while creating credential.

Below image should give a clear picture of what needs to be given at which place.

Credential Name = any name which we need to use in BACKUP command

Identity = Storage Account Name which can be found under “Access Keys”.

Password and Confirm Password = Either Key1 or Key2.

It didn’t take much time to realize that my client has used some random name in Identity instead of storage account name and that was the cause of the error. We were able to fix that using ALTER command as below

USE [master]
GO
ALTER CREDENTIAL [AzBackupCred] WITH IDENTITY = N'sqlprodbkups', 
SECRET = N'Value In Key1'
GO

Hope this would help you in fixing error 403, forbidden while taking backup to URL.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Fix Error 3271: A nonrecoverable I/O error occurred on file. The remote server returned an error: (403) Forbidden

In SQL Server one error message can be caused due to many reasons. One of such error message is 3271 which is caused during failure of backup to URL command. Here are two earlier blogs about same error message 3271. (404) Not Found.

SQL SERVER – Fix Error 3271: A nonrecoverable I/O error occurred on file. The remote server returned an error: (403) Forbidden

SQL SERVER – Backup to URL Fails in Azure Resource Manager (ARM) Deployment Model with Error – (400) Bad Request

In this blog, we would discuss error 404 – Not Found. Here is the complete error message which can be seen.

Msg 3271, Level 16, State 1, Line 1
A nonrecoverable I/O error occurred on file “https://sqlprodbkups.blob.core.windows.net/sqldbbackups1/SQLAuthority.bak:” Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (404) Not Found..
Msg 3013, Level 16, State 1, Line 1
BACKUP DATABASE is terminating abnormally.

And here was the command.

BACKUP DATABASE SQLAuthority 
TO URL='https://sqlprodbkups.blob.core.windows.net/sqldbbackups1/SQLAuthority.bak'
WITH CREDENTIAL ='AzBackupCred'

WORKAROUND/SOLUTION

In this case, we again need to look at exception message which is 404 – Not Found.

It didn’t take much time to figure out that the container name which was given in the command was incorrect. The container name should be picked from the Azure portal. We need to go to Home > Storage accounts > sqlprodbkups > Blob service. (our storage account name is sqlprodbkups)

If you re-look at the command which gave an error, we can clearly see that I misspelled the container name (sqldbbackup1 instead of sqldbbackup) to cause the error. Once you pick right name, this exception 404 would disappear.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Fix Error 3271: A nonrecoverable I/O error occurred on file. The remote server returned an error: (404) Not Found

One question I always ask all of my customers before I start Comprehensive Database Performance Health Check is that if they have taken backup of their server. The answer to this question honestly does not matter to me during consultancy I never change anything which brings to the situation where they need to restore the database. However, when I asked this question to my client, they said they have an issue to identify a version of SQL Server from the backup file.

Let me elaborate on this subject. My client had multiple versions of SQL Server. They had SQL Server 2008 R2, SQL Server 2012, SQL Server 2014 and SQL Server 2016 (they have yet not moved to SQL Server 2017). Each of this version has a database named “DevDepartment”. Now before the consultancy started they took backup of this database in a single folder for all the versions. Now their backup folder had 4 backups with a similar name – “Backup-DevDepartment+[DynamicTimeStamp]”. They were indeed very much confused with all the files and were not sure which file belongs to which database server.

Well, the answer is very simple. We can figure out details about the server, login and database version by just checking the header details of the backup files.

Here is the script which can give you important server details:

RESTORE HEADERONLY
FROM DISK = N'D:\data\SQLAuthority.bak'

When you run above script, it gives quite a few important details.

Pay attention to columns – ServerName, UserName, DatabaseName, DatabaseVersion. We can figure out what was the original server name and database version from those columns.

Here are the quick list lf all the SQL Server versions and their compatibility levels from SQL Server 2008 to the latest version of SQL Server.

You can compare the column DatabaseVersion from the resultset with Internal Database version and can determine the version of the database. Well, I hope this was a very simple tutorial where we can figure out the version of SQL Server from the backup file.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Identify Version of SQL Server from Backup File

While preparing for a demo for my upcoming session, I encountered an interesting error. I thought I have written a blog about it, but I was wrong. My earlier blog was about a different error which looks very similar. Here is my earlier blog for error 3132. SQL SERVER – FIX: Msg 3132, Level 16 – The Media Set has 2 Media Families, but Only 1 is Provided. All Members Must be Provided. In this blog post, we will learn how to fix the error The Media Loaded on “Backup” is Formatted to Support 1 Media Families, but 2 Media Families are Expected According to the Backup Device Specification.

In this blog, we would learn about error 3231. Here is the complete message of the error.

Msg 3231, Level 16, State 1, Line 5
The media loaded on “C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\Backup\SQLAuthority.bak” is formatted to support 1 media families, but 2 media families are expected according to the backup device specification.
Msg 3013, Level 16, State 1, Line 5
BACKUP DATABASE is terminating abnormally.

Here are the steps to reproduce the error.

CREATE DATABASE SQLAuthority
GO
BACKUP DATABASE SQLAuthority TO DISK = 'SQLAuthority.bak'
GO
BACKUP DATABASE SQLAuthority 
TO DISK = 'SQLAuthority.bak',
   DISK = 'SQLAuthority_1.bak' 
WITH DIFFERENTIAL

WORKAROUND/SOLUTION

As per error message, there is an existing backup available in SQLAuthority.bak, but in next backup, we have given two files to take split backup. One of the files is an earlier backup file which has only one media family.

The solution would be to use “FORMAT” and “INIT” to erase existing backup data in the existing backup. You might want to be careful as we are formatting existing media which has a backup. Here is the script with modification.

CREATE DATABASE SQLAuthority
GO
BACKUP DATABASE SQLAuthority TO DISK = 'SQLAuthority.bak'
GO
BACKUP DATABASE SQLAuthority 
TO DISK = 'SQLAuthority.bak',
   DISK = 'SQLAuthority_1.bak' 
WITH DIFFERENTIAL, FORMAT, INIT

If you want to use UI then below gives you the options.

CLEANUP SCRIPT

Here is the script to cleanup database backup history and dropping the database.

EXEC msdb.dbo.sp_delete_database_backuphistory @database_name = N'SQLAuthority'
GO
USE [master]
GO
ALTER DATABASE [SQLAuthority] SET  SINGLE_USER WITH ROLLBACK IMMEDIATE
GO
USE [master]
GO
DROP DATABASE [SQLAuthority]
GO

Have you encountered such errors while taking a backup? Please share via comments.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – FIX: Msg 3231 – The Media Loaded on “Backup” is Formatted to Support 1 Media Families, but 2 Media Families are Expected According to the Backup Device Specification

If you have worked with various SQL Server editions, then you would know that SQL Server Express edition does not have SQL Server Agent service. Due to this, we lose capabilities to automate a lot of things. Let us learn in this blog post how to delete old backup files in SQL Express.

This article is not specific to SQL Express but can be implemented in any edition of SQL Server. Since we can use maintenance plan in other SQL editions, it may not be very useful there. One of the main things DBAs want to automate is taking backups on a regular basis. I have gone through this world of internet and there are a million ways to automate SQL backups and the most basic, easy and popular one adopted is using a Task Scheduler method. Microsoft has a Knowledge Base Article to achieve the same.

How to schedule and automate backups of SQL Server databases in SQL Server Express

While doing consulting with a client, I found that the server was having E drive almost full. Later I learned that they have adopted the above approach. Now, they asked me if there is an automated way to delete backup files which are older than ‘X’ number of days. This is because their disk space was getting filled quickly.

SOLUTION/WORKAROUND

Below is the script which can be used to achieve deletion of the files. We need to provide two parameters, Path and DaysToKeep. The below script will hold the latest 5 days backup and delete all the older files from the folder provided in Path variable.

$Path = "D:\Backups\"
$DaysToKeep = "-5"
$CurrentDate = Get-Date
$DatetoDelete = $CurrentDate.AddDays($DaysToKeep)
Get-ChildItem $Path -Recurse | Where-Object { $_.LastWriteTime -lt $DatetoDelete } | Remove-Item -Recurse

Get-ChildItem $Path -Recurse | Where-Object { $_.LastWriteTime -lt $DatetoDelete } | Remove-Item -Recurse

Note: this would delete any file in that location so if you want, you can add a filter to delete only .bak or .trn file.

USAGE STEPS

1. Save the above script to a folder. for example: c:\temp\cleanupjob.ps1
2. Open the Windows Task Scheduler. In the Task Scheduler, select the Create Task option
3. Enter a name for the task, and give it a description (the description is optional)
4. Click on the Triggers tab and set your schedule or event that will trigger the running of your PowerShell script
5. Next, go to the Actions tab and click New to set the action for this task to run. Set the Action to Start a program.
6. In the Program/script box enter “exe“
7. In the Add arguments box enter the value ->

-ExecutionPolicy Bypass c:\temp\cleanupjob.ps1

6. Save your scheduled task and run it.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – PowerShell Script – Delete Old Backup Files in SQL Express

Recently I was hired for my On Demand (55 minutes) Service by an organization and they had a very interesting situation. To be very honest, I was initially lost for few moments when they described this situation to me and wanted me to solve their error. They were facing a very unique situation where their SSMS was working just fine but when they try to restore any database, SSMS Restore Database Wizard will take forever to open.

The DBAs there were very much surprised with this issue and they had already spent a lot of time on the server already. After unsuccessful attempts, they decided to reach out to me and wanted me to solve their error. Let us see what are the various efforts which we attempted during the call to solve the problem and which one exactly worked.

Failed Attempt 1: Re-install SSMS

This did not work for us.

Failed Attempt 2: Install the latest version of SSMS

Again, just like reinstallation SSMS, the latest version of SQL Server Management Studio did not work.

Failed Attempt 3: Increased the memory to OS

As we were on a virtual machine, we tried to increase the memory on OS but that was not successful as well.

Failed Attempt 4: Attempt to restore another database

It does not matter which database we tried to restore the SSMS Restore Database Wizard would not come up easily.

Failed Attempt 5: Tried when the server is not busy

Honestly, that did not work out as well.

Successful Attempt: Delete Backup History

Well, after spending over 30 minutes on various different unsuccessful attempts, just like a senior DBA, I was also a bit lost for the moment. However, when noticed that only their restore database wizard is slow but no other part in the SSMS, I suddenly remembered about backup history.

Whenever we open Restore Database Wizard at that time SQL Server retrieves entire history of the database backup from the table msdb.dbo.backupset and displayed under the section Backup sets to restore. I checked with the DBA and they were not deleting any backup history at all as a part of the maintenance process.

The next thing which we did was to delete the backup history with the following command, where I deleted all the history before the date January 1, 2018.

USE msdb;
EXEC sp_delete_backuphistory @oldest_date = '01/01/2018';

Once I deleted their history, suddenly their restore database wizard started to open very quickly.

Lesson learned that you must have a proper maintenance plan in place. Deleting the backup history is just one of the step of the maintenance task, you can reach out to me at pinal @ sqlauthority.com and in an hour we can put a very robust maintenance plan which will not give you unexpected surprises as such described in today’s blog post.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Restore Database Wizard in SSMS is Very Slow to Open

If you are a SQL DBA then this blog might be a quick refresher for you. I am writing this for Data Protection Manager (DPM) Administrators who are not aware of the cause of the issue. In this blog, we would learn about how to fix error raised by DPM – “The backup failed because of a gap in the LSN the Log files”

One of my client who was not an expert in SQL contacted me for below error which was reported in DPM log.

Description: Last 1 recovery points not created. DPM tried to do a SQL log backup, either as part of a backup job or a recovery to a latest point in time job. The SQL log backup job has detected a discontinuity in the SQL log chain for database since the last backup. All incremental backup jobs will fail until an express full backup runs. For more information, open DPM Administrator Console and review the alert details in the Monitoring task area.

He had no idea about what needs to be done. So, he contacted me for On Demand (55 minutes) services. Even if I didn’t know much about DPM, I took this request and learned something new about DPM.

WORKAROUND/SOLUTION

In SQL Server full recovery model transaction log file gets truncated after transaction log backup. It looks like if the DPM server finds that there is a discontinuity in the log backup and it raises an error. This also means that some other backup application (mostly SQL Server Native Backup) is taking the log backup LSN is not in sequence for backups taken by DPM. If you are an SQLDBA, you can think it as LSN chain break which might happen in log-shipping when we take out-of-band backup.

SQL SERVER – Log Shipping Restore Job Error: The file is too recent to apply to the secondary database

So, I asked him to use the script from the following blog to find out if there are other backups running.

SQL SERVER – Get Database Backup History for a Single Database

Sure enough, SQLDBAs were not aware of DPM and they configured native SQL Server backup. Since DPM was already protecting the server, we disabled the job which was taking T-Log backups. After this issue was resolved and same error never came back.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – DPM Backup Failure – E_COORD_LOG_CHAIN_BROKEN – The Backup Failed Because of a Gap in The LSN the Log Files

Question: How to Find Last Full Backup Time and Size for Database?

Answer: What makes me unhappy during any of my Comprehensive Database Performance Health Check is DBA and Developers often not taking backup of their database. It is really sad to see that individuals who care about their database performance would be so careless for their database backup (safety). Often DBAs get into action as soon as point them that they do not have a full backup. However, there are times when I have seen after pointing this out to DBAs just stay do not care about it.

In any case, here is the script which when you run gives you two of the most important details about your database backup.

• Backup Date
• Backup Size

SELECT
CONVERT(CHAR(100), SERVERPROPERTY('Servername')) AS Server,
msdb.dbo.backupset.database_name,
msdb.dbo.backupset.backup_finish_date,
CAST(msdb.dbo.backupset.backup_size AS NUMERIC(35,2))/1048576.0 AS backup_size_MB
FROM msdb.dbo.backupmediafamily
INNER JOIN msdb.dbo.backupset ON msdb.dbo.backupmediafamily.media_set_id = msdb.dbo.backupset.media_set_id
WHERE msdb.dbo.backupset.type = 'D'
ORDER BY
msdb.dbo.backupset.database_name

When you run the above query and do not see your database in this list, that means your database is not being backup at all. If you see your database in this list, I suggest you take look at the Backup Date and make sure that there is always a recent backup available for the database.

If you are using any such script, I would be happy to connect with you and know what script you run to check your database’s last full backup.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on How to Find Last Full Backup Time and Size for Database? – Interview Question of the Week #173

I have been seeing many customers moving their SQL Server to Cloud. Few of them have chosen Microsoft Azure platform to host their production server and few others have selected other cloud platforms like AWS. When they host SQL in Azure, they can choose various backup methods. In this blog, we would learn about fixing an interesting error “Account Verification Failed” while using Microsoft SQL Server Backup to Microsoft Azure Tool.

This is a free tool from Microsoft which can be downloaded from the link here Microsoft SQL Server Backup to Microsoft Azure Tool

One of my clients has downloaded and installed it on the SQL Server Azure Virtual Machine. Here is how they were trying to configure. As soon as they launch, we get below screen.

Click on “Add” and provide a backup path and file extension.

Hit next and provide storage account details.

As soon as we click on “Verify account” we get an error – Account verification failed. This is not a very informative error as we have triple checked the account name, keys, and container multiple times.

WORKAROUND/SOLUTION

We used another account and it was working fine. Then we compared each property of both storage accounts and found that this was caused due to below setting – “Secure transfer required”. In good case it was disabled and in bad case it was enabled.

Once we made it “Disabled” we were able to proceed.

Have you encountered such error where there is no meaningful solution in the message? I hope Microsoft fixes it soon.

Another cosmetic issue: If you look at the tool installer and usage it says “Microsoft SQL Server Backup to Windows Azure Tool” whereas download link says “Microsoft SQL Server Backup to Microsoft Azure Tool”. This is something like what I blogged here SQL SERVER – SSMS 17.7 Observation – Two Miscellaneous Category in Database Property Options

Hope it would reach Microsoft team via this blog and they provide some meaningful error message.

Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Backup to Microsoft Azure Tool – ERROR: Account Verification Failed

Backup to URL is one of the common methods used in SQL Server perform backup to Azure Blob Storage. In this blog, I am going to share a script to generate the create credential and backup command using access keys.

If you don’t know already, Backup to URL also has two methods to connect to the storage account

1. Credential by using Access Keys.
2. Credential by using SAS token.

In this blog, I would show the first method – Backup using Access Keys.

WORKAROUND/SOLUTION

To use below script, you need to copy some values from Azure Portal. Go to Azure Portal Home > Storage accounts > Choose Storage account > Access keys as shown below:

To get a container name, you can refer below screenshot. You need to click on “Browse Blobs”. If you don’t have container created already then click on “+” symbol and create a new one. In my Azure, I have already created one called “dailybackups” as shown below.

---- Backup To URL (with Credentials) :
--- =================================== ---
DECLARE @Date AS VARCHAR(25)
	,@TSQL AS NVARCHAR(MAX)
	,@ContainerName AS NVARCHAR(MAX)
	,@StorageAccountName AS VARCHAR(MAX)
	,@DatabaseName AS SYSNAME
	,@StorageKey AS NVARCHAR(MAX)
	,@CredentialName AS SYSNAME;
SELECT @StorageAccountName = '';--- Find this from Azure Portal
SELECT @ContainerName = '';--- Find this from Azure Portal
SELECT @StorageKey = '';--- Find this from Azure Portal
SELECT @DatabaseName = 'master';
SELECT @CredentialName = 'Cred' + @StorageAccountName;
IF NOT EXISTS (
		SELECT *
		FROM sys.credentials
		WHERE name = '' + @CredentialName + ''
		)
BEGIN
	SELECT @TSQL = 'CREATE CREDENTIAL [' + @CredentialName + '] WITH IDENTITY = ''' + @StorageAccountName + ''' ,SECRET = ''' + @StorageKey + ''';'
	-- PRINT @TSQL
	EXEC (@TSQL)
END  
SELECT @Date = REPLACE(REPLACE(REPLACE(REPLACE(CONVERT(VARCHAR, GETDATE(), 100), '  ', '_'), ' ', '_'), '-', '_'), ':', '_');
SELECT @TSQL = 'BACKUP DATABASE [' + @DatabaseName + '] TO '
SELECT @TSQL += 'URL = N''https://' + @StorageAccountName + '.blob.core.windows.net/' + @ContainerName + '/' + @DatabaseName + '_' + @Date + '.bak'''
SELECT @TSQL += ' WITH CREDENTIAL = ''' + @CredentialName + ''', COMPRESSION, STATS = 1;'
-- PRINT @TSQL
EXEC (@TSQL)

Above script would create a container and take backup of the master database. Once I ran the script, I could see Credential create and backup was also taken as shown below.

Here is the backup

Soon, I would also share a script to use the backup to URL using SAS token via another blog. If you find this useful, please comment and let me know.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

First appeared on SQL SERVER – Backup to URL – Script to Generate the Create Credential and Backup Command using Access Keys