During my consultancy engagement, my client’s DBA team was checked ERRORLOG and asked me the possible cause of below error about service broker.

DateTime spid74s      Error: 9642, Severity: 16, State: 3.
DateTime spid74s      An error occurred in a Service Broker/Database Mirroring transport connection endpoint, Error: 8474, State: 11. (Near endpoint role: Target, far endpoint address: ”)

From the above message, we can figure out that there is some system session in SQL which is trying to connect to some endpoint. I checked and they confirmed that they were not using service broker. But they were using Always On Availability Group.

I checked data synchronization and it was fine. Rows which were modified on primary was reaching to secondary as well. This means there was no issue with data movement also.

While enquiring further I learned that they are using a Read-Only Routing feature that was not working properly.

WORKAROUND/SOLUTION

As soon as I checked the availability group properties and checked Read-Only Routing configuration, everything has fallen in place and started making sense.  Here is what I saw in the properties.

Do you see a problem here?

When the request hits primary, it finds the routing and request try to go to 5022 port on secondary. That port is not meant for client connectivity and hence the error.

Whenever we try to connect to a listener and perform read-only routing at the very same time, we were seeing mirroring endpoint error.

To fix the issue, we corrected the port number to the ports on which SQL Server was listening, After fixing the issue, routing started working and we also found that error disappeared.

The same error can be reproduced if you try to connect to SQL Server on a port on which SQL is listening but not for client connectivity.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error: 9642 – An error occurred in a Service Broker/Database Mirroring transport connection endpoint

This was one of an interesting error which I received while preparing for a demo. I wanted to demonstrate a disaster recovery scenario by doing backup and restore. In this blog we would learn how to fix Msg 4353 – Conflicting file relocations have been specified for the file. Only a single WITH MOVE clause should be specified for any logical file name.

To reproduce this error, we need to create a database which has two files which the same “logical” name and different physical name. Creating that is also a tricky and for me, it was created due to my copy/paste mistake. If you or your developers are lazy like me, you can run into this error.

Before running below, make sure C: has a folder called Temp.

CREATE DATABASE [SQLAuthority]
ON PRIMARY
(NAME = 'SQLAuthority', FILENAME = 'C:\Temp\SQLAuthority.mdf')
LOG ON
(NAME = 'SQLAuthority_log', FILENAME = 'C:\Temp\SQLAuthority_log.ldf')
GO

Now, I am adding 2 data files and 2 log files.

ALTER DATABASE SQLAuthority ADD FILE
(NAME = 'SQLAuthority_D1', FILENAME = 'C:\Temp\D1.ndf'),
(NAME = 'SQLAuthority_D1', FILENAME = 'C:\Temp\D2.ndf')
GO
ALTER DATABASE SQLAuthority ADD LOG FILE
(NAME = 'SQLAuthority_L1', FILENAME = 'C:\Temp\L1.ndf'),
(NAME = 'SQLAuthority_L1', FILENAME = 'C:\Temp\L2.ndf')

Do you notice any problem in above? Yeah, two physical files having a same logical name.

I took a backup of this database as below.

BACKUP DATABASE SQLAuthority TO DISK = 'C:\Temp\SQLAuthority.bak'

Here is the output

Processed 328 pages for database ‘SQLAuthority’, file ‘SQLAuthority’ on file 1.
Processed 8 pages for database ‘SQLAuthority’, file ‘SQLAuthority_D1’ on file 1.
Processed 8 pages for database ‘SQLAuthority’, file ‘SQLAuthority_D1’ on file 1.
Processed 3 pages for database ‘SQLAuthority’, file ‘SQLAuthority_log’ on file 1.
Processed 0 pages for database ‘SQLAuthority’, file ‘SQLAuthority_L1’ on file 1.
Processed 0 pages for database ‘SQLAuthority’, file ‘SQLAuthority_L1’ on file 1.
BACKUP DATABASE successfully processed 347 pages in 0.050 seconds (54.140 MB/sec).

You can clearly see that backup is happening for all the files, including duplicates. Now, I wanted to restore the database on a different server. Since there was no C:\Temp on the destination server, I decided to relocate the files to a different folder, so I used UI to restore it. Here is the error which I got.

Conflicting File Story

When I looked at T-SQL, here is the one which is failing

USE [master]
RESTORE DATABASE [SQLAuthority]
FROM DISK = N'D:\SQLAuthority.bak'
WITH FILE = 1
,MOVE N'SQLAuthority' TO N'F:\DATA\SQLAuthority.mdf'
,MOVE N'SQLAuthority_D1' TO N'F:\DATA\D1.ndf'
,MOVE N'SQLAuthority_D1' TO N'F:\DATA\D2.ndf'
,MOVE N'SQLAuthority_log' TO N'F:\LOG\SQLAuthority_log.ldf'
,MOVE N'SQLAuthority_L1' TO N'F:\LOG\L1.ndf'
,MOVE N'SQLAuthority_L1' TO N'F:\LOG\L2.ndf'
,NOUNLOAD
,STATS = 5
GO

As we can see, we have the same logical name twice in the restore statement.

WORKAROUND/SOLUTION

The error message is saying a pretty obvious thing but why did it allow me to do that in the first place? ALTER command which I ran, in the beginning, should have shown the error message to me about conflicting files. However, there is no error for the conflicting files.

Anyways, the workaround would be to create the same path as the source on this server. Basically, we need to avoid MOVE command and let it restore files at the same location where they wanted to go.

Another workaround would be to rename the logical file name using below command.

USE [SQLAuthority]
GO
ALTER DATABASE [SQLAuthority] MODIFY FILE (NAME=N'SQLAuthority_D1', NEWNAME=N'SQLAuthority_D2')
GO
USE [SQLAuthority]
GO
ALTER DATABASE [SQLAuthority] MODIFY FILE (NAME=N'SQLAuthority_L1', NEWNAME=N'SQLAuthority_L2')
GO

Interestingly, the above command changes name only for the one file (out of two in this case). Once renamed then another backup is needed to be restored.

Reference : Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – FIX: Msg 4353 – Conflicting File Relocations Have Been Specified for File. Only a Single WITH MOVE Clause Should be Specified for Any Logical File Name

While preparing for a disaster recovery demo for my client, I observed an interesting behavior. In this blog, I would show a quick demo where you can learn how the full backup size can be different on the primary and secondary replica for an availability database in Always On availability group.

Here is the quick script to understand the behavior.

1. Create a Database and take a full backup and create a wide table (8KB char column)

   USE master
   GO
   CREATE DATABASE SQLAGDB
   GO
   BACKUP DATABASE SQLAGDB TO DISK = 'SQLAGDB.bak'
   GO
   USE SQLAGDB
   GO
   CREATE TABLE SQLAuthority (j CHAR(8000))
   GO
   

2. Configure AlwaysOn Availability Group using SSMS – Always On Wizard for database SQLAGDB. Makes sure it is healthy and secondary in sync.
3. Insert lots of rows in the table. You can use the below script

   -- connect to database 
   USE SQLAGDB
   GO
   -- insert 100 rows
   SET NOCOUNT ON
   GO
   INSERT INTO SQLAuthority
   VALUES ('A') 
   GO 100
   -- 
   SET NOCOUNT ON
   GO
   -- insert from same table to same table (8 times)
   -- total rows should be 100+100+200+400+800+1600+3200+6400+12800
   INSERT SQLAuthority
   SELECT *
   FROM SQLAuthority 
   GO 8
   SELECT COUNT(*) from SQLAuthority -- 25600
   

4. Take COPY_ONLY backup from both replicas.

   --primary 
   :connect sqlserver-0
   BACKUP DATABASE SQLAGDB TO DISK = '\\sqlserver-0\share\primary_before_log_backup.bak'
   WITH copy_only,format
   GO
   -- secondary 
   :connect sqlserver-1
   BACKUP DATABASE SQLAGDB TO DISK = '\\sqlserver-0\share\secondary_before_log_backup.bak'
   WITH copy_only,format
   GO
   

   Here is the output
   Connecting to sqlserver-0…
   Processed 25944 pages for database ‘SQLAGDB’, file ‘SQLAGDB’ on file 1.
   Processed 28629 pages for database ‘SQLAGDB’, file ‘SQLAGDB_log’ on file 1.
   BACKUP DATABASE successfully processed 54573 pages in 6.755 seconds (63.115 MB/sec).
   Disconnecting connection from sqlserver-0…
   Connecting to sqlserver-1…
   Processed 25944 pages for database ‘SQLAGDB’, file ‘SQLAGDB’ on file 1.
   Processed 2 pages for database ‘SQLAGDB’, file ‘SQLAGDB_log’ on file 1.
   BACKUP DATABASE successfully processed 25946 pages in 2.259 seconds (89.729 MB/sec).
   Disconnecting connection from sqlserver-1…
   Notice processed pages for ‘SQLAGDB_log’.

5. Let’s take a log backup (on any replica)

   BACKUP LOG [SQLAGDB] TO  
   DISK = N'\\sqlserver-0\share\Log-backup.trn' 
   WITH NOFORMAT, NOINIT,  SKIP, NOREWIND, NOUNLOAD,  STATS = 10
   GO
   

   Processed 28630 pages for database ‘SQLAGDB’, file ‘SQLAGDB_log’ on file 1.
   BACKUP LOG successfully processed 28630 pages in 3.533 seconds (63.309 MB/sec).

6. Now take the same full backup on both replicas.

   --primary 
   :connect sqlserver-0
   BACKUP DATABASE SQLAGDB TO DISK = '\\sqlserver-0\share\primary_after_log_backup.bak'
   WITH copy_only,format
   GO
   -- secondary 
   :connect sqlserver-1
   BACKUP DATABASE SQLAGDB TO DISK = '\\sqlserver-0\share\secondary_after_log_backup.bak'
   WITH copy_only,format
   GO
   

   and here is the output:
   Connecting to sqlserver-0…
   Processed 25944 pages for database ‘SQLAGDB’, file ‘SQLAGDB’ on file 1.
   Processed 4 pages for database ‘SQLAGDB’, file ‘SQLAGDB_log’ on file 1.
   BACKUP DATABASE successfully processed 25948 pages in 3.182 seconds (63.706 MB/sec).
   Disconnecting connection from sqlserver-0…
   Connecting to sqlserver-1…
   Processed 25944 pages for database ‘SQLAGDB’, file ‘SQLAGDB’ on file 1.
   Processed 2 pages for database ‘SQLAGDB’, file ‘SQLAGDB_log’ on file 1.
   BACKUP DATABASE successfully processed 25946 pages in 2.365 seconds (85.707 MB/sec).
   Disconnecting connection from sqlserver-1…
   

7. A picture is worth a thousand words:
   

This situation would come when we don’t perform log backup and T-Log continues to grow. From above we can understand that full backup with copy_only has different logic to take a portion of the log file on primary and secondary. Interesting enough the additional size (208 + 229 = 437) is equal to LDF size of the database file. Once log backup is taken there is no additional log file to be backed up and size became the same. I hope you learn about Different Size of COPY_ONLY Full Backup on Primary and Secondary Replica in Always On.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Different Size of COPY_ONLY Full Backup on Primary and Secondary Replica in Always On

I always recommend my client to go with the latest and great version of tools like SQL Server Management Studio (SSMS). In this blog, we would learn about a situation where an older version of SSMS was giving an error while adding a new database to existing availability group – This BACKUP or RESTORE command is not supported on a database mirror or secondary replica.

THE SITUATION

My client had an existing availability group which was set up and deployed by a vendor. Now, they wanted to add a new database to an existing availability group. They were using the wizard to add the database. They selected the option to do a fresh backup/restore. Due to old SSMS version, automatic seeding option was not available. Here was the error which they were getting in the UI.

Here is the text of error message:

Restoring database log resulted in an error. (Microsoft.SqlServer.Management.HadrTasks)
——————————
ADDITIONAL INFORMATION:
Restore failed for Server ‘SQLSERVER-1’. (Microsoft.SqlServer.SmoExtended)
——————————
System.Data.SqlClient.SqlError: This BACKUP or RESTORE command is not supported on a database mirror or secondary replica. (Microsoft.SqlServer.Smo)

The surprising part was that even after the error came, the database was added to the availability group successfully. This means the error was a benign error and can be ignored but I want to know what error was appearing. To dig further I generated the script of Always On and executed it manually. It failed with the exact same error.

WORKAROUND/SOLUTION for RESTORE command

It was clear that the database was getting added successfully due to automatic seeding. We can see below in ERRORLOG.

Primary Replica:

• Database backed up. Database: OneMoreDatabase, creation date(time): 2019/06/11(13:21:51), pages dumped: 355, first LSN: 38:2800:1, last LSN: 38:2824:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {‘\\sqlserver-0\Share\OneMoreDatabase.bak’}). This is an informational message only. No user action is required.
• Database backed up. Database: OneMoreDatabase, creation date(time): 2019/06/11(13:21:51), pages dumped: 522, first LSN: 38:2712:1, last LSN: 38:2824:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {‘{3AFC5950-4E79-4225-B572-0A70AD876E45}’}). This is an informational message only. No user action is required.
• Log was backed up. Database: OneMoreDatabase, creation date(time): 2019/06/11(13:21:51), first LSN: 38:2712:1, last LSN: 38:2824:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {‘\\sqlserver-0\Share\OneMoreDatabase_20190626054651.trn’}). This is an informational message only. No user action is required.

Secondary Replica:

• Database was restored: Database: OneMoreDatabase, creation date(time): 2019/06/11(13:21:51), first LSN: 38:2712:1, last LSN: 38:2824:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {‘{E9D35CEB-D6FD-40D6-A73C-06560EFD95D9}’}). Informational message. No user action required.
• Database was restored: Database: OneMoreDatabase, creation date(time): 2019/06/11(13:21:51), first LSN: 38:2800:1, last LSN: 38:2824:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {‘\\sqlserver-0\Share\OneMoreDatabase.bak’}). Informational message. No user action required.

If we look at the backup and restore commands, there are two backups. Once on TYPE=DISK and another on TYPE=VIRTUAL_DEVICE. The second backup comes when there is an automatic seeding happening for the database.

Running script manually fails with below (on secondary). This is the same what we get from the UI.

Connecting to sqlserver-1…
Msg 3059, Level 16, State 2, Line 29
This BACKUP or RESTORE command is not supported on a database mirror or secondary replica.
Msg 3013, Level 16, State 1, Line 29
RESTORE LOG is terminating abnormally.
Disconnecting connection from sqlserver-1…
Connecting to sqlserver-1…
Msg 41145, Level 16, State 1, Line 67
Cannot join database ‘DWConfiguration’ to availability group ‘XAG’. The database has already joined the availability group. This is an informational message. No user action is required.
Disconnecting connection from sqlserver-1…

When I compared the script generated from SSMS for an old and new version, I found the difference which explained the issue clearly. This was the extra command in the new version of SSMS.

ALTER AVAILABILITY GROUP [XAG]
MODIFY REPLICA ON N'sqlserver-1' WITH (SEEDING_MODE = MANUAL)

This means that SSMS is smart enough to change the mode to MANUAL when below option is selected.

So, to conclude, the seeding was set to AUTOMATIC for the replica. When we were using an older version of SSMS, it was doing both automatic seeding and backup/restore. This was causing restore to fail because it was seeded already and added successfully.

You can use below query to find replica seeding mode.

SELECT ag.name
	,replica_server_name
	,seeding_mode_desc
FROM sys.availability_replicas ar
	,sys.availability_groups ag
WHERE ag.group_id = ar.group_id
	AND name = 'AGSQLAGDB'

Have you seen such errors of SSMS? Please comment and let me know.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Add Database to Availability Group Failure – This BACKUP or RESTORE Command is Not Supported on a Database Mirror or Secondary Replica

Recently one of my blog readers contacted me and ask if I knew about wait type parallel_backup_queue. I have often got the same question during the Comprehensive Database Performance Health Check. My search on the internet was not giving me a lot of details. Only official documentation which I could find was about wait statistics related to Parallel Backup Queue is very little available online.

Occurs when serializing output produced by RESTORE HEADERONLY, RESTORE FILELISTONLY, or RESTORE LABELONLY.

I asked him, my friend, what exact command he is running, and he informed that he is running below.

RESTORE FILELISTONLY FROM DISK='F:\Backups\ProductionDB.bak'

This is matching with documentation but what is “serializing output”? He informed that he was checking sys.dm_exec_requests DMV and filtering for the SPID which was running this command.

I asked him to check sys.sysprocesses and we found that there were two threads for that SPID. One of them was in a suspended state with wait type parallel_backup_queue and another was running and showing wait of BACKUPIO. I captured process monitor to check IO and I was surprised to see that the above command was reading the whole file in chunks. The database size was close it 1.5 TB.

So, I asked him to leave it running as it was reading the backup file. Finally, we found that it failed with an error:

The media family on device is incorrectly formed.

This made me believe that if there is a problem with the backup media, the commands would take a long time and we might see wait of parallel_backup_queue.

I have not been able to reproduce this in my lab even with 50 GB backup. If you find a way, please share via comments. Or if you have some other findings of Parallel Backup Queue, please share that as well.

Here is the link to the blog post where I discussed SQL Server Wait Stats in detail.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – What is Wait Type Parallel Backup Queue?

One of my clients contacted me for whom I assisted in configuring Always On Availability Groups for the first time. They were getting an error whole taking log backup on secondary replica: Log backup for database “SQLAuthority” on secondary replica failed because the new backup information could not be committed on the primary database.

During implementation, I explained to them various topologies, pros, and cons and assisted them in doing the end-to-end implementation as this was their first availability group. If you want any such implementation related assistance, feel free to contact me.

Secondary Replica

After I implemented their first availability group, they got comfortable and implemented distributed availability group on their own. They contacted me and informed that things are going well but they are seeing an error when they take log backup of an availability database on a secondary replica of the secondary availability group. Below is the message from job history.

Log backup for database “SQLAuthority” on secondary replica failed because the new backup information could not be committed on the primary database. Check the database status in the SQL Server error log of the server instance that is hosting the current primary replica. If the primary database is participating in the availability group, retry the operation. Check the database status in the SQL Server error log of the server instance that is hosting the current primary replica. If the primary database is participating in the availability group, retry the operation. [SQLSTATE 42000] (Error 35296)  BACKUP LOG is terminating abnormally.

To make things simple, you can imagine a deployment like this.

myAG1 (mynode1, myNode2) <- myDistributedAG –> myAG2 (myNode3, myNode4)

Distributed availability group is created between myAG1 and myAG2. myNode1 is global primary and myNode3 is a forwarder. The failure of the job is happening on myNode4.

SOLUTION/WORKAROUND

I tried this in my lab, and I was able to get exactly the same error. I searched further and found that this is by design. I found below information in this link.

In a distributed availability group, backups can be performed on secondary replicas in the same availability group as the active primary replica, or on the primary replica of any secondary availability groups. Backups cannot be performed on a secondary replica in a secondary availability group because secondary replicas only communicate with the primary replica in their own availability group. Only replicas that communicate directly with the global primary replica can perform backup operations.

When I informed this to my client, they were able to understand this limitation and modify the scripts to take backup accordingly.

Have you seen such errors while working with the availability group? Please share via comments.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error 35296: Log Backup for Database “SQLAuthority” on Secondary Replica Failed Because the New Backup Information Could Not be Committed on Primary Database

Question: How to Forecast the Size of Restore of Backup in SQL Server?

Answer: Recently, I had discussed this in consulting engagement: Comprehensive Database Performance Health Check. We had a very unique scenario where we had a huge backup file of the SQL Server and we had to restore it on the server. The issue is very simple that we had a limited amount of the disk and were not sure what would happen if we try to restore backup which will eventually build a database larger than the available space.

While we were working on the server’s performance, the DBA brought up this question to me. Fortunately, the answer to this problem is very simple and I pretty much every day face this scenario.

Let us assume that we have a backup with the filename adv.bak. Let us check how much space will it contain on the drive with the help of the command RESTORE FILELISTONLY.

RESTORE FILELISTONLY
FROM DISK = N'H:\data\adw.bak';

Once you run the command it will display the details of the MDF, LDF or any other file contained in the backup. Pay attention to the column Size. This column actually contains the size of the file in bytes once the backup is restored on the database.

In our example, if you notice there are two different files MDF and LDF. The respective sizes in the bytes are as following:

• MDF: 1216348160
• LDF: 679477248

Now you can covert the same using the following formula:

• 1 byte = 8 bits
• 1 kilobyte (K / Kb) = 2^10 bytes = 1,024 bytes
• 1 megabyte (M / MB) = 2^20 bytes = 1,048,576 bytes
• 1 gigabyte (G / GB) = 2^30 bytes = 1,073,741,824 bytes
• 1 terabyte (T / TB) = 2^40 bytes = 1,099,511,627,776 bytes
• 1 petabyte (P / PB) = 2^50 bytes = 1,125,899,906,842,624 bytes
• 1 exabyte (E / EB) = 2^60 bytes = 1,152,921,504,606,846,976 bytes

Or just use Google Calculator (which I do most of the time), which also displays the formula divide the digital storage value by 1e+9.

• MDF: 1216348160 = 1.21 GB
• LDF: 679477248 = 0.67 GB

Essentially, I will need around 2 GB of free space if I want to restore my backup. Well, that’s it, sometimes a complex looking problem of how to forecast the Size of Restore has a very simple solution.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on How to Forecast the Size of Restore of Backup in SQL Server? – Interview Question of the Week #265

One of my clients contacted me and asked me the reason for this message? And my first reaction was, did you keep the database in very cold weather? It might be freezing over there and that’s why SQL is saying:   “I/O is frozen on database” and “I/O was resumed on database”? Did you check the air conditioner temperature?

Jokes apart. Let’s discuss the message. Let me paste the complete message and I am showing only one such combination.

This sequence of messages would be present for all databases which got backed-up. I had only four system databases on my test server. I fired disk shadow to take a backup of C drive and once the backup was completed, we can see three such pairs, each with the same four messages. Since we can’t take the backup of tempdb, we are seeing only master, model, and msdb. I have highlighted them with a different color so that it is easy to focus.

THE ANSWER – Frozen

Four messages which are shown in this blog are generated when there is any software taking volume-level backup. I have used DIskshadow to take C drive backup and all the database for which the files were in C drive got backed up. These software use VSS (Volume Shadow Copy Services) which interacts with SQL Server Writer to get backup of those files. The key part is the backup type mentioned in message # 3 – VIRTUAL_DEVICE = {GUID}

If you are seeing such messages, they are information messages and generally can be ignored unless the backups are causing issues with the application. These messages indicate that SQL Server has frozen the IO so that snapshots can be taken for the volume. This is coordination between VSS and SQL Server so that consistent database backup is taken across the volumes. If you are using software like BackupExec, Veeam, Microsoft Data Protection Manager (MSDPM), Azure Backups then it is normal to see these messages.

Are there any such interesting messages which you can share?  Please comment and let me know.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – What do These Messages Mean – “I/O is Frozen on Database” and “I/O was Resumed on Database”?

Every single day, when I write blog posts, I am not sure what I will write tomorrow but I must express my gratitude to all of you, who keep on asking questions new interesting questions which made me write more. Today we will talk about the Impact of Recovery Model on Insert Workload Stress Test and Wait Statistics. Today’s question was sent to me by one of my clients who works at a large technology firm that deals with the stock market. I had earlier consulted for the last month while engaging in the Comprehensive Database Performance Health Check.

Note: This blog post is not discussing the bulk-logged recovery model, so please do not post a comment about it.

Recovery Model – Simple or Full

Earlier on this blog, I wrote a blog post about SQL SERVER – Wait Statistics Generated by oStress – Insert Workload.  I got lots of emails and tweets about this blog post. In the blog post, I had created a test table and used oStress Utility to insert the data in parallel, during the insert, there was a huge amount of the WRITELOG Wait Statistics. Lots of people wrote back to me asking what was the Recovery Model during the insert operation.

The most common argument as following –

You are seeing a lot of WRITELOG wait statistics because you might have a recovery model set to FULL recovery and if you change it to simple, you will not see thew WRITELOG wait statistics as well the query will run faster.

Very interesting argument indeed. I often get this argument as well and I explain to them the reality behind it. However, yesterday when one of my clients reached out to me, I also explained the science behind the reality but he said his CTO would not believe the theory and would like to see the demonstration. Finally, we went on GoToMeeting call and looked at the demonstration.

As many you had the same concern, I decided to also blog about it. Here is the few blog posts, I suggest you read it before continuing today’s blog post as they are all related, and reading earlier blog post will give you the context behind the whole setup.

• SQL SERVER – Performance Test – sqlcmd vs SSMS
• SQL SERVER – Performance Test – oStress vs SSMS
• SQL SERVER – Stress Testing with oStress – Load Testing
• SQL SERVER Management Studio and SQLCMD Mode
• SQL SERVER – Wait Statistics Generated by oStress – Insert Workload

Now I will be running two different tests on my server and after both the tests, I will be measuring the time to complete the execution and also the wait statistics of the single operation. Between both the execution, I will truncate the table and reset the wait statistics.

Set up

First, create a simple table in the database.

USE [SQLAuthority]
GO
CREATE TABLE [dbo].[TestTable](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Description] [varchar](500) NOT NULL DEFAULT ('I hope you have a great day!')
) ON [PRIMARY]
GO

Now that we have created a table with the identity key in it. Let us move to the next step where we will run our queries with the stress.

Before running each of the following tests, we reset the table with empty data by running the following truncate command.

TRUNCATE TABLE [SQLAuthority].[dbo].[TestTable];

Now rest the wait statistics on your development or test server where you are going to run the query with the following command.

DBCC SQLPERF('sys.dm_os_wait_stats', CLEAR);
GO

Test 1: Full Recovery Model

First set the recovery model of the database to the full recovery model.

USE [master]
GO
ALTER DATABASE [SQLAuthority] SET RECOVERY FULL WITH NO_WAIT
GO

Now after truncating the table and clearing the wait statistics, we will run the following command stress command.

To run the test make sure that you have installed RML Utility in your system. Next, you can run the oStress command RML Utility folder. In most systems, it will be here: C:\Program Files\Microsoft Corporation\RMLUtils.

ostress -S"Quick\SQL19" -E -Q"INSERT INTO [SQLAuthority].[dbo].[TestTable] VALUES(DEFAULT);" -n100 -r10000 -q -dSQLAuthority

As you can see in the code above, I am running the test of inserting a single row 10000 times (one at a time) in 100 parallel threads in parallel.

Now let us measure the time to execute the query and also the wait statistics. If you do not have a query to measure the wait statistics, you can get that from here: SQL Server Wait Statistics collection script.

The query took around 1 min and 18 seconds and wait statistics for write log is around 3090 seconds. 

Test 2: Simple Recovery Model

Now for this test, we will set the recovery model of the database to the full recovery model.

USE [master]
GO
ALTER DATABASE [SQLAuthority] SET RECOVERY SIMPLE WITH NO_WAIT
GO

As this is the second test, make sure to truncate the table and clear the wait statistics. Now after truncating the table and clearing the wait statistics, we will run the following command oStress command. The command is the same as what we had ran before.

ostress -S"Quick\SQL19" -E -Q"INSERT INTO [SQLAuthority].[dbo].[TestTable] VALUES(DEFAULT);" -n100 -r10000 -q -dSQLAuthority

I am running the test of inserting a single row 10000 times (one at a time) in 100 parallel threads in parallel, just like earlier time.

Now let us measure the time to execute the query and also the wait statistics.

The query took around 1 min and 21 seconds and wait statistics for write log is around 3029 seconds. 

Honestly, there is not much difference in either duration of the query or even in the wait statistics.

Summary – Insert Workload Stress Test

I have run the test nearly 10 different times with 10 different workloads for each of the workloads. That means I have run 100 + 100 total of 200 tests for this scenario. Every single time I found that the results are very similar. The difference and variation between both the tests are absolutely negligible.

The performance of the queries are the same irrespective of your Recovery Model. 

The reality is that in simple as well as in the full recovery model, all the data is always written in the log file first. The only difference between the recovery model is that the simple recovery model does not keep the transaction once it has completed successfully (or failed successfully).

I hope this helps you to understand how the recovery model has very little or no impact on query performance for insert workload stress test.

Note: This blog post is not discussing the bulk-logged recovery model, so please do not post a comment about it.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Impact of Recovery Model on Insert Workload Stress Test and Wait Statistics

Question: How to Know Backup Speed in SQL Server?

Answer: It is very simple to know the backup speed in SQL Server.

You can get the answer to this question with the help of SQL Server Management Studio. Let us run the following script which is taking backup of the database.

BACKUP DATABASE [AdventureWorks2017] TO
DISK = N'D:\data\adw.bak'
GO

Now when you run the above script it will give you the output response in the messages window. When you look at the window, you can see the last line which demonstrates the speed of the current backup.

The last line shows the efficiency in MB/sec.

It depends on many different parameters like – speed of the drive, backup options, size of the backup, and activities going on the server from where the backup is initiated. Additionally, backup also generates wait for statistics which often impacts the current server’s performance.

The best practices say one should take backup only when there are not many activities are going on the server. There are multiple types of backups – full backup, differential backup, and log backup. You will have to schedule them as per your business needs.

Here are my earlier articles on this subject, which you may find interesting.

• SQL SERVER – Backup Timeline and Understanding of Database Restore Process in Full Recovery Model
• What is a Backup Timeline for Restore Process? – Interview Question of the Week #092
• SQL SERVER – 7 Important Things to Remember While Taking Effective Backup
• How to Find Last Full Backup Time and Size for Database? – Interview Question of the Week #173
• SQL SERVER – Restore SQL Database using SSMS – SQL in Sixty Seconds #044 – Video
• SQL SERVER – Configure, Monitor and Restore Backups from Mobile & Web browser
• SQL SERVER – Database in RESTORING State for a Long Time

If you have an interesting question, please share and I will be happy to blog about it with due credit to you.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on How to Know Backup Speed in SQL Server? – Interview Question of the Week #277

Restoring the database with Just Log File is not possible. If you are looking for the answer to this question, you have bad news. I am writing this blog post for a simple reason, when I get an email asking for the same question again, I answer the question quickly with a link to the blog post.

Here are the top 5 questions I have received in the last 3 months about restoring a database with just a log file.

Q1: Is it possible to recreate the database with just a log file, I have do not have a data file?

Q2: How to generate mdf file from the ldf file?

Q3: I have multiple data file in my database and I have all the datafiles except one file and all the log file, can I restore my database?

Q4: I have all my log file backup and not datafile backup, is there any way to stitch the log files to generate data files?

Q5: I have no backup and ransomware is encrypted all my data and log file and I have no backup, what is the workaround?

Unfortunately, the answer to all the questions is – No, it is not possible.

If you do not have your data file or its backup, only having a log file or log file’s backup is not good at all. If you only have a log file (and no data files), you can’t generate your database from log files even if you have a third party tool that claims that they will regenerate the database for you. At the most third party tool can read the log file and give you some idea about the recent data which was modified there is no way, it can help you to regenerate the database at all.

Let me re-iterate if there is no good backup with you, in most cases when the accident happens, the situation is extremely unfortunate. Always take your backup and test it regularly. Here is my most popular blog post about the backup timeline: SQL SERVER – Backup Timeline and Understanding of Database Restore Process in Full Recovery Model.

Here are blog posts which you may find relevant to this topic:

• SQL SERVER – Restore Database Backup using SQL Script (T-SQL)
• SQL SERVER – Fastest Way to Restore Database
• SQL SERVER – Restore Database Without or With Backup – Everything About Restore and Backup
• SQL SERVER – Backup and Restore Database Using Command Prompt – SQLCMD
• SQL SERVER – Database Backup and Restore Job Management
• SQL SERVER – Restore SQL Database using SSMS – SQL in Sixty Seconds #044 – Video
• SQL SERVER – Restore Database Wizard in SSMS is Very Slow to Open
• SQL SERVER – How to Restore Corrupted Model Database from Backup?
• SQL SERVER – A Quick Script for Point in Time Recovery – Back Up and Restore
• SQL SERVER – Database Stuck in Restoring State

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Restore Database With Just Log File

Question: What does Verify Backup When Finished do in SQL Server?

Answer: It will check up if the backup media is in the place and also checked the header information to make sure that it looks like the header of the backup file.

While it looks like a very important step in the entire backup and restores operation honestly, I consider it just a great start. If you have only specified this option while taking your backup, actually, you have a great start but you can do much more if you have also enabled performing checksum along with it. Contradictory to what many people think this option of Verify Backup When Finished, does not do any integrity check.

Remember that the option of verifying backup only checks limited header information of the backup. It is very much possible for back to get validated (verified or passed) the test of VerifyOnly but during restore, you may still face errors.

Final Suggestion – Verify Backup

If you are really serious about your SQL Server backup, I strongly recommend that you have a routine where you go and restore your backup to your server frequently. There is no better test then restoring the database on the server. If your restore is successful, you have passed your test, otherwise, no matter what feature you have enabled, there are always chances of the disaster.

Here are few recent blog posts on similar topics:

SQL SERVER – Restore Database With Just Log File

Restoring the database with Just Log File is not possible. If you are looking for the answer to this question, you have bad news. It is not possible.

How to Know Backup Speed in SQL Server? – Interview Question of the Week #277

Question: How to Know Backup Speed in SQL Server?
Answer: It is very simple to know the backup speed in SQL Server. Let us discuss today.

SQL SERVER – Msg 3254 – The Volume on Device Path\File is Empty. RESTORE DATABASE is Terminating Abnormally

Let us fix today – Msg 3254 The volume on device <path\file>is empty. Followed by Msg 3013 – RESTORE DATABASE is terminating abnormally.</path\file>

How to Forecast the Size of Restore of Backup in SQL Server? – Interview Question of the Week #265

Question: How to Forecast the Size of Restore of Backup in SQL Server? Answer: Recently, I had discussed this in consulting engagement.

SQL SERVER – Drop failed for Availability Group – Failed to Destroy the Windows Server Failover Clustering Group Corresponding to Availability Group

While working with one of the clients to recover from a disaster, I encountered an error. In this blog we would talk about error Drop failed for Availability Group. If you want any such assistance, you can hire me for quick consultation using On-Demand offering.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on What does Verify Backup When Finished do in SQL Server? – Interview Question of the Week #283

In this blog post, we are going to talk about how to undo human errors in SQL Server with the help of Point in Time Recovery. Let me start asking some questions –

• Have you ever dropped your table by mistake?
• Have you ever wrote an UPDATE statement without the WHERE condition?
• Have you ever deleted the data which are critical to the business?

If the answer to this question is yes. I am sure you understand the importance of rolling back the transactions. In SQL Server you can always Undo Human Errors with the help of Point in Time Recovery. Let us learn how you can do that in this video.

Here are a few things to remember. You can rollback your transaction if your recovery model is Full Recovery Model and also if you have taken one full backup before you made the error. If either of the conditions is not satisfied you may not be able to recover your data again.

One of the most common questions I have received for this video is what happens if the database has many other important transactions going on other tables and you want to keep them rolling forward. If that is the case, you should restore your database on another system or with a different name on the same system and restore your database to the point right before you made the error.

Once you do that, you can retrieve your data from the table where you have accidentally faced the disaster and insert into your original live database.

In any case, it is always a good idea to perform all of your experiments on the development environment first. Once you are satisfied with the results and output, you should try them in the production environment. It is always a good idea to test your work before deploying it.

Here is the sample script which I have used in this blog post.

CREATE DATABASE SQLAuthority
GO
ALTER DATABASE SQLAuthority
SET RECOVERY FULL
GO
USE SQLAuthority
GO
CREATE TABLE TestTable (ID INT)
GO
BACKUP DATABASE [SQLAuthority] 
TO DISK = N'D:\SQLAuthority.bak'
GO
INSERT INTO TestTable (ID)
VALUES (1), (2), (3)
GO
SELECT GETDATE() BeforeTruncateTime;
-- 2020-08-18 19:45:38.717
SELECT *
FROM TestTable
GO
-- Disaster
TRUNCATE TABLE TestTable
GO
SELECT *
FROM TestTable
GO
BACKUP LOG [SQLAuthority] TO
DISK = N'D:\SQLAuthorityLog.trn'
GO
-- ------------------
-- Restore operation
USE master
GO
RESTORE DATABASE [SQLAuthority] 
FROM DISK = N'D:\SQLAuthority.bak'
WITH NORECOVERY, REPLACE
GO
RESTORE LOG [SQLAuthority] 
FROM DISK = N'D:\SQLAuthorityLog.trn'
WITH STOPAT = '2020-08-18 19:45:38.717', RECOVERY
GO
USE SQLAuthority
GO
SELECT *
FROM TestTable
GO
USE master
GO
DROP DATABASE SQLAuthority
GO

Please run the script above with the video which is included in this blog post. If you accidentally run the script without understanding what each command do, you may damage your system.

Remember final thing: When something goes wrong, do not worry or panic. Just note down the time when it happened. You can always rollback if you have a full back and full recovery model using the technique explained in this video.

If you still need help you can reach out to me here:  Comprehensive Database Performance Health Check.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Undo Human Errors in SQL Server – SQL in Sixty Seconds #109 – Point in Time Restore

Recently, I got a very interesting question from my client of the Comprehensive Database Performance Health Check. As many of you know that I primarily focus on the SQL Server Performance issue. If your server is running slow, I would like to help it to run faster. My client recently noticed that their server was running slow and they investigated the slowness in the system. After investigating their server, they found out that they have a full backup running for one of the largest servers and at the same time they are running a log backup on the same server. Now their question was is that fine or will that will have a negative impact on their backup?

Let me answer the main question – It is totally fine to have Log Backup and Full Backup running at the same time. While they both are running for the safety of the database and they are related to each other with the timeline, it is totally fine to have them running overlapping and they do not impact the integrity of the database or the backup.

Running Log Backup

The full backup does not truncate or remove data from the transaction log. Your transaction log file operates kind of independent from the full backup file. When you take full backup and transaction backup both at the same time, the transaction log backups are cleaned only after the full backup completes. One more thing, if you only run the full backup, your transactions log will remain intact and will not clean up. Only transaction log backup can clean the transaction file.

Let me know if you like what we discussed today. I am always looking forward to your question and often I created videos on them as well. Here are my recent 25 SQL in Sixty Seconds Video.

Reference: Pinal Dave (https://blog.sqlauthority.com) 

First appeared on SQL SERVER – Running Log Backup While Taking Full Backup

Most of the time during the Comprehensive Database Performance Health Check, I focus on SQL Server Performance Tuning. However, often the client also asks for help when they have a question about SQL Server Engine. Recently I received a question from a client that they want to move their database from the Full Recovery Model to the Simple Recovery Model, can they do it?

The answer to this one is difficult as well as simple. When I asked the reason to my client why they want to move to the Simple Recovery Model, their answer was that they have heard it is better for performance. Honestly, it is not true entirely. I have been doing consulting for over a decade and I have yet to see a single situation when we have changed the recovery model and it the server has performed miraculously well.

The Recovery model is directly connected with the data loss and secondary related to SQL Server features. There are few features of the SQL Server that are not possible when your model is set to Simple. Here is a list of the features which are not supported if you are using the Simple Recovery Model.

• Log shipping
• Always On
• Database mirroring
• Point-in-time restores

If you are going to use any of the features listed above, they are not supported by the Simple Recovery Model. Additionally, you should make the decision of using the recovery model on your work loss exposure and not on any other factor primarily.

Here are few recent blog posts where I have posted SQL in the Sixty Seconds videos.

• CONCAT and NULL – SQL in Sixty Seconds #123
• Sleeping vs Suspended Process – SQL in Sixty Seconds #122
• Pinned Tab – SSMS Efficiency Tip – SQL in Sixty Seconds #121
• Split Screen for SSMS Efficiency – SQL in Sixty Seconds #120
• Nonclustered Primary Key – SQL in Sixty Seconds #119
• Recent Execution of Stored Procedure – SQL in Sixty Seconds #118
• Full-Screen Execution Plan – SQL in Sixty Seconds #117

Reference: Pinal Dave (https://blog.sqlauthority.com) 

First appeared on SQL SERVER – Simple Recovery Model and Restrictions

Yesterday one of my clients of Comprehensive Database Performance Health Check reached out to me asking for help in attaching the database files to the server. Let us see today we can attach a database with T-SQL.

There are two different methods to attach the database. Let us see both of them.

Method 1: Create a Database with T-SQL

This is my preferred method and I use it all the time.

USE [master]
GO
CREATE DATABASE [AdventureWorks] ON 
( FILENAME = N'D:\Data\AdventureWorks.mdf' ),
( FILENAME = N'E:\Log\AdventureWorks_log.ldf' )
 FOR ATTACH
GO

Method 2: sp_attach_db

This is used to be my preferred method before I started to use Method 1.

USE [master]
GO
EXEC sp_attach_db @dbname = N'AdventureWorks',   
    @filename1 =   'D:\Data\AdventureWorks.mdf',   
    @filename2 =   'E:\Log\AdventureWorks_log.ldf'
GO

Well, that’s it. It is pretty simple to attach databases with the T-SQL script. Here is a few associated blog post on the same topic, which you may find useful.

• SQL SERVER – Attach or Detach Database – SQL in Sixty Seconds #068
• SQL SERVER – Unable to Attach Database – File Activation Failure – The Log Cannot be Rebuilt
• SQL SERVER – Unable to Attach Database Files – The PageAudit Property is Incorrect – Ransomware Attack
• How to Attach MDF Data File Without LDF Log File – Interview Question of the Week #078
• How to Move SQL Server MDF and LDF Files? – Interview Question of the Week #189
• SQL SERVER – Attaching and Restoring Database in Clustering Generates An Error – Notes from the Field #115

If you have any such questions, you can reach me via Twitter.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Attach a Database with T-SQL

Earlier I wrote a blog post about SQL SERVER – Attach a Database with T-SQL. Right after reading this blog post, my client of Comprehensive Database Performance Health Check sent me an email asking if there is any other method when we have to attach an In-Memory Database with T-SQL. The question is very valid as SSMS graphical user interface does not support attaching an in-memory database. Let us learn how we can do that today.

Here is the script which works with In-Memory Database.

CREATE DATABASE [AdventureWorks] ON
( FILENAME = N'D:\Data\AdventureWorks.mdf' ),
( FILENAME = N'E:\Log\AdventureWorks_log.ldf' ),
(FILENAME = N'G:\InMemoryFolder')
FOR ATTACH
GO

When you are moving the in-memory database, make sure that you remember the name of the file stream folder where the in-memory objects are stored and you can specify that after specifying data and log file as a filename in the script.

I have not found much help on the internet on this topic, hence this blog post. If there is any inaccuracy in the script or any other method to attach the in-memory database via T-SQL, please leave a comment and I will be happy to share the same blog readers with due credit to you.

I like to create small videos teaching valuable lessons about T-SQL, you can subscribe to my channel SQL in Sixty Seconds to learn new stuff about technology.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Attach an In-Memory Database with T-SQL

Recently I was asked by one of my clients of Comprehensive Database Performance Health Check recently asked if there is any way to Check Backup Reliability. Well, honestly, yes there is a way and that is to restore the backup on your database.

Check Backup Reliability

Lots of people often ask me if there is a way to figure out if the backup of the database is fine or not. Honestly, there is no magic trick or there is no shortcut for it. You will have to restore your database on your production server or on the development server and see if the data is valid or not.

There is honestly no other shortcut. I have seen so many organizations who think that their backup is valid and good but when they try to restore it, it fails.

Here are my few recent videos and I would like to know what is your feedback about them. You can subscribe to my youtube channel here.

• Zoom in SSMS – SQL in Sixty Seconds #153
• Transfer Schema of Table – SQL in Sixty Seconds #152
• Find a Table in Execution Plan – SQL in Sixty Seconds #151
• Transactions and Variables – SQL in Sixty Seconds #150
• Count Table in Cache – SQL in Sixty Seconds #149
• List All Sessions – SQL in Sixty Seconds #148
• Line Numbers for SSMS Efficiency – SQL in Sixty Seconds #147
• Slow Running Query – SQL in Sixty Seconds #146
• Change Database and Table Collation – SQL in Sixty Seconds #145
• Infinite Loop – SQL in Sixty Seconds #144
• Efficiency Trick – Query Shortcut – SQL in Sixty Seconds #143
• SQL SERVER – 16 CPU vs 1 CPU : Performance Comparison – SQL in Sixty Seconds #142
• SQL SERVER – TOP and DISTINCT – Epic Confusion – SQL in Sixty Seconds #141

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Check Backup Reliability

One of my long-term clients (Comprehensive Database Performance Health Check) recently asked me question if I know a way of creating multiple backup files for one backup files. Of course, there is a way and it is a very easy way. Let us learn it today.

Creating Multiple Backup Files

It is very easy to take one backup and split it and distribute it. At the same time, it is equally easy to take multiple backups files from SQL Server itself. There are two different ways to do it. One via SSMS and another via T-SQL. Let us learn both of them here.

SSMS

In SSMS, right-click on the database, go to tasks and select the option backup. Over here is the destination where you can provide multiple locations where you want to take backup. On the screen, I have provided the same directory but you can also select another directory as well.

Well, that’s it. It is that simple.

T-SQL

If you want to do that with T-SQL, here is the script.

BACKUP DATABASE [CollationTest] TO
DISK = N'D:\data\backup1.bak',
DISK = N'D:\data\backupcopy.bak'
WITH STATS = 10
GO

Let me know if you want me to create a video on this topic or not on backup files. Remember, here you are splitting one file into multiple files and you will need all the files to restore your backup. 

Here are my few recent videos and I would like to know what is your feedback about them. You can subscribe to my youtube channel here.

• Forwarded Records and Performance – SQL in Sixty Seconds #155
• Hide Code in SSMS – SQL in Sixty Seconds #154
• Zoom in SSMS – SQL in Sixty Seconds #153
• Transfer Schema of Table – SQL in Sixty Seconds #152
• Find a Table in Execution Plan – SQL in Sixty Seconds #151

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Creating Multiple Backup Files – Stripped

Yesterday I blogged about SQL SERVER – Creating Multiple Backup Copies. I got an amazing response to the blog post. Lots of people wanted me to create a video about it. I can totally understand the interest in the video over a simple blog post. This question was asked to me by one of my clients of Comprehensive Database Performance Health Check.

I have included the video over here about Multiple Backup Copies:

If you are interested in the backup and restore topic, here are a few of my blog posts.

• Backup Timeline and Understanding of Database Restore Process in Full Recovery Model
• What is a Backup Timeline for Restore Process? – Interview Question of the Week #092
• How to Know Backup Speed in SQL Server? – Interview Question of the Week #277
• 7 Important Things to Remember While Taking Effective Backup
• How to Find Last Full Backup Time and Size for Database? – Interview Question of the Week #173
• What is Copy Only Backup in SQL Server? – Interview Question of the Week #128
• Restore Database With Just Log File
• When was Database Last Backed Up in SQL Server? – Interview Question of the Week #10

Here are my few recent videos and I would like to know what is your feedback about them. You can subscribe to my youtube channel here.

• Forwarded Records and Performance – SQL in Sixty Seconds #155
• Hide Code in SSMS – SQL in Sixty Seconds #154
• Zoom in SSMS – SQL in Sixty Seconds #153
• Transfer Schema of Table – SQL in Sixty Seconds #152
• Find a Table in Execution Plan – SQL in Sixty Seconds #151

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Multiple Backup Copies Stripped – SQL in Sixty Seconds #156