I recently downloaded Idera’s SQL virtual database, and tested it. There are a few things about this tool which caught my attention. Let us learn about Retrieve and Explore Database Backup without Restoring Database.

My Scenario

It is quite common in real life that sometimes observing or retrieving older data is necessary; however, it had changed as time passed by. The full database backup was 40 GB in size, and, to restore it on our production server, it usually takes around 16 to 22 minutes, depending on the load server that is usually present. This range in time varies from one server to another as per the configuration of the computer. Some other issues we used to have are the following:

1. When we try to restore a large 40-GB database, we needed at least that much space on our production server.
2. Once in a while, we even had to make changes in the restored database, and use the said changed and restored database for our purpose, making it more time-consuming.

My Solution

I have heard a lot about the Idera’s SQL virtual database tool.. Well, right after we started to test this tool, we found out that it really delivers what it promises. Using this software was very easy and we were able to restore our database from backup in less than 2 minutes, sparing us from the usual longer time of 16–22 minutes. The needful was finished in a total of 10 minutes. Another interesting observation is that there is no need to have an additional space for restoring the database. For complete database restoration, the single additional MB on the drive is not required anymore. We can use the database in the same way as our regular database, and there is no need for any additional configuration and setup.

Let us look at the most relevant points of this product based on my initial experience:

• Quick restoration of the database backup
• No additional space required for database restoration
• virtual database has no physical .MDF or .LDF
• The database which is restored is, in fact, the backup file converted in the virtual database. DDL and DML queries can be executed against this virtually restored database.
• Regular backup operation can be implemented against virtual database, creating a physical .bak file that can be used for future use.
• There was no observed degradation in performance on the original database as well the restored virtual database.
• Additional T-SQL queries can be let off on the virtual database.

Well, this summarizes my quick review. And, as I was saying, I am very impressed with the product and I plan to explore it more. There are many features that I have noticed in this tool, which I think can be very useful if properly understood.

I had taken a few screenshots using my demo database afterwards. Let us see what other things this tool can do besides the mentioned activities. I am surprised with its performance so I want to know how exactly this feature works, specifically in the matter of why it does not create any additional files and yet, it still allows update on the virtually restored database. I guess I will have to send an e-mail to the developers of Idera and try to figure this out from them.

I think this tool is very useful, and it delivers a high level of performance way more than what I expected. Soon, I will write a review for additional uses of SQL virtual database.. If you are using SQL virtual database in your production environment, I am eager to learn more about it and your experience while using it.

The ‘Virtual’ Part of virtual database

When I set out to test this software, I thought virtual database had something to do with Hyper-V or visualization. In fact, the virtual database is a kind of database which shows up in your SQL Server Management Studio without actually restoring or even creating it. This tool creates a database in SSMS from the backup of the same database. The backup, however, works virtually the same way as original database.

Potential Usage of virtual database:

As soon as I described this tool to my teammate, I think his very first reaction was, “hey, if we have this then there is no need for log shipping.” I find his comment very interesting as log shipping is something where logs are moved to another server. In fact, there are no updates on the database from log; I would rather compare it with Snapshot Replication. In fact, whatever we use, snapshot replicated database can be similarly used and configured with virtual database. I totally believe that we can use it for reporting purpose. In fact, after this database was configured, I think the uses of this tool are unlimited. I will have to spend some more time studying it and will get back to you.

Click on images to see larger images.

virtual database Console

Harddrive Space before virtual database Setup

Attach Full Backup Screen

Backup on Harddrive

Attach Full Backup Screen with Settings

virtual database Setup – less than 60 sec

virtual database Setup – Online

Harddrive Space after virtual database Setup

Point in Time Recovery Option – Timeline View

virtual database Summary

No Performance Difference between Regular DB vs Virtual DB

Reference: Pinal Dave (http://blog.SQLAuthority.com)

The post SQL SERVER – Retrieve and Explore Database Backup without Restoring Database – Idera virtual database appeared first on Journey to SQL Authority with Pinal Dave.

I recently received following comments on my earlier blog about Making database to read only.

“Today I was trying to attach the (MDF, NDF, LDF) SQL server 2008 database which i have received from my client. After attachment the database status is showing (Read-Only) (Eg.database name (Read-Only). How do I make to normal mode for the data updating. Is there any query available to resolve this problem. Your help will be highly helpful.”

Let’s learn Making Database to Read Only and Changing Database to Read/Write with the help of T-SQL scripts.

Here is the answer:

Make Database Read Only

USE [master]
GO
ALTER DATABASE [TESTDB] SET READ_ONLY WITH NO_WAIT
GO

Make Database Read/Write

USE [master]
GO
ALTER DATABASE [TESTDB] SET READ_WRITE WITH NO_WAIT
GO

If you face an error that if the database is already in use, you can resolve the same by making database in single user mode – here is the guideline SQL SERVER – ALTER DATABASE dbname SET SINGLE_USER WITH ROLLBACK IMMEDIATE.

Let me know if you have ever faced this situation in your business or life by leaving note in the comment section in the blog. I read every single comment.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

The post SQL SERVER – Making Database to Read Only – Changing Database to Read/Write appeared first on Journey to SQL Authority with Pinal Dave.

When a novice DBA or Developer create a database they use SQL Server Management Studio to create new database. Additionally, the T-SQL script to create a database is very easy as well. You can just write CREATE DATABASE DatabaseName and it will create new database for you. The point to remember here is that it will create the database at the default location specified in SQL Server Instance (this default instance, can be changed and we will see that in future blog posts). Now, once the database files goes in production it will start to grow.

It is not common to keep the Database on the same location where OS is installed. Usually Database files are in SAN, Separate Disk Array or on SSDs. This is done usually for performance reason and manageability perspective. Now the challenges come up when database which was installed at not preferred default location and needs to move to a different location. Here is the quick tutorial how you can do it.

Let us assume we have two folders loc1 and loc2. We want to move database files from loc1 to loc2.

USE MASTER;
GO
-- Take database in single user mode -- if you are facing errors
-- This may terminate your active transactions for database
ALTER DATABASE TestDB
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
GO
-- Detach DB
EXEC MASTER.dbo.sp_detach_db @dbname = N'TestDB'
GO

Now move the files from loc1 to loc2. You can now reattach the files with new locations.

-- Move MDF File from Loc1 to Loc 2
-- Re-Attached DB
CREATE DATABASE [TestDB] ON
( FILENAME = N'F:\loc2\TestDB.mdf' ),
( FILENAME = N'F:\loc2\TestDB_log.ldf' )
FOR ATTACH
GO

Well, we are done. There is little warning here for you: If you do ROLLBACK IMMEDIATE you may terminate your active transactions so do not use it randomly. Do it if you are confident that they are not needed or due to any reason there is a connection to the database which you are not able to kill manually after review.

Reference : Pinal Dave (http://blog.SQLAuthority.com)

The post SQL SERVER – Move Database Files MDF and LDF to Another Location appeared first on Journey to SQL Authority with Pinal Dave.

Backup and Restore is one of the core tasks for DBAs. They often do this task more times than they would have ideally loved to do so. One thing I noticed in my career that every successful DBA knows how to automate their tasks and spend their time either playing games on a computer or learning something new!

Let us see how a DBA can automate their task about Backup and Restore using SQLCMD. I am demonstrating a very simple example in this blog post. Please adapt the script based on your environment and needs.

Here is the basic script for Backup and Restore

Note: name of my database server is touching and I am connecting it with windows authentication.

Backup

BACKUP DATABASE AdventureWorks2012 TO DISK='d:\adw.bak'

Restore

RESTORE DATABASE AdventureWorks2012 FROM DISK='d:\adw.bak'

Here is the basic script for Backup and Restore using SQLCMD

Backup

C:\Users\pinaldave>SQLCMD -E -S touch -Q 
"BACKUP DATABASE AdventureWorks2012 TO DISK='d:\adw.bak'"

Restore

C:\Users\pinaldave>SQLCMD -E -S touch -Q 
"RESTORE DATABASE AdventureWorks2012 FROM DISK='d:\adw.bak'"

Please leave a comment if you are facing any issue. As mentioned earlier the scripts are very basic scripts, you may have to adapt them based on your environment. For example, if you are facing error there are chances that database files are already open or exists on the drive. You you should also have necessary permissions to do backup and restore as well file operations in your server.

Watch a 60 second video on this subject

Reference: Pinal Dave (http://blog.sqlauthority.com)

The post SQL SERVER – Backup and Restore Database Using Command Prompt – SQLCMD appeared first on Journey to SQL Authority with Pinal Dave.

SQL Server backups are essential for any business that store their data in SQL Server. The following is a list of best practices you should be following if you are the person in charge with maintaining your organization’s databases. Let us see some of the SQL Server Backup Tips.

1. Backups should not be stored in the same physical location as database files

This very simple rule will save your business from a lot of difficult situations in the event that a physical drive becomes faulty.

In case something like this occurs, you sahould have the possibility to use other drives or to remote to a different location that has stored a backup of your database so that you are able to restore your database from that location.

2. Backup schedules are set up

Another good safety precaution is for your backups to always be accompanied by proper schedules which have been established to meet the application needs and your particular business requirements.

Creating backup schedules is very important because as time passes and your backups get older, the risk of data loss becomes higher, unless you are protected and you have a way to reestablish all the data up to the point of failure.

Backup schedules will provide you a consistent and constant history of your data which will always be useful, not only in disaster situations.

3. Test the restore procedure of your backup on a test server

You should always try to restoring your backups on a test server and make sure that you can recover all of the data with all the options you need to use during a restore on the main server.

Just because a backup process has finished successfully, this will not guarantee that the backup can also be restored. You might not be aware that your backup was not created correctly, and when trying to restore it on the main server, the process might fail due to media corruption or other factors.

4. Make use of all available verification options when doing backup

Another good practice is to use all options provided in the process of backing up your database in order to make sure that the end result is a transactionally consistent backup.

If you are using T-SQL scripts to back up your database, then make sure that when using the BACKUP command you also add the CHECKSUM parameter, in the WITH clause, which will imply that each page will be passed through a checksum operation to ensure the consistency of your backup on the backup media.

BACKUP DATABASE MyDatabase
TO DISK = 'Z:\MyBackups\MyDatabase.bak'
WITH CHECKSUM;

If you prefer doing your backups manually through a visual interface like the one available in SQL Server Management Studio (SSMS), make sure to check the first two checkboxes in the Reliability section, on the Options page.

These two options, Verify backup when finished and Perform checksum before writing to media will add a level of verification to your backup process which will ensure your backups are consistent.

If you choose a different wizard to do your backups, like the one available through SQL Server Maintenance Plans, make sure to tick the box that corresponds to Check Database Integrity option.

A different piece of software which I prefer to use to use for this operation, mainly because of its simplicity in use, is SQLBackupAndFTP. It has all the most important options in an easy to access and intuitive interface.

5. Do a FULL backup daily

Based on the needs of your organization you should choose a recovery model that will allow you to protect your organization’s against data loss. If your organization can afford to lose 15, 30, 60 minutes worth of data, choosing a simple recovery model is the option you should go for.

In this case, having a FULL database backup is the best method of protection against data loss in any data protection plan and in conclusion, most cases would require it to be performed daily, despite the overhead added by the time required for such an operation.

If your organization’s databases are small with easily recoverable data, then a FULL backup should be the way to go.

As I am already using SQLBackupAndFTP for my backup process, I find it very convenient that it has, by default, set its backup schedule to do a FULL back up every 24 hours.

This is very helpful and this way the risk that a detail that is most important will be eliminated.

The biggest advantage of this best practice is that the recovery is easier as the backup is just one file and no database log is needed, while the downside for this type of backup is that data that can be recovered is only up to the time of the backup and depending on the size of your database, it can take up a lot of disk space.

6. Do differential backups more frequently

If you consider doing FULL backups are too expensive for your organization, from a resource availability and data loss point of view, there is the option of doing differential backups of your databases, which can and should be done more frequently than FULL backups.

In terms of advantages, differential backups will take less time to complete and also less disk space as they contain just the data from the last FULL backup operation. No database log is needed in this case either and the restores are more precise in terms of data.

The disadvantages of this would be that still, the data that can be recovered is only up to the time of the restore and the restoring process is a bit more complicated as there are two files involved.

7. Do transaction log backups even more frequently

The following most important step in doing backups, after FULL and differential backups, would be to back up the transaction log. As the transaction log contains all the recent activity in the database, it could be used to make a restore of the database to a specific point in time, which is its biggest advantage.

Doing a transaction log backup also truncates the log, which keeps it from becoming full. Similar to database backups, transaction log backups can be done while the system is active.

If your organization has a high activity with some of its databases, doing a transaction log backup every 10 minutes is recommended, while other databases which are less active might have their transaction logs backed up every 30 minutes or 60 minutes.

The negative side of this type of backup is that the database is required to have transaction logging activated which will increase the size of the database and the effort required when doing the restore process.

8. Practice recovery operations

A successful company is flexible and quickly adapts to the changes in the market. In such a case, where business requirements could change unexpectedly, this could mean that your backup strategies can become obsolete.

Testing your backup strategies on a frequent basis and covering different scenarios that might appear, scenarios that include both system and individual database restores will ensure that your backup plans will have the expected efficiency and will work at the time they are needed.

9. Regularly back up system databases

Even though backup strategies will save you from losing user data, it is incomplete without a backup plan for your SQL Server system databases, master, model and msdb databases.

These databases are essential because they contain system configuration along with SQL Server job information which has to be restored in case of a total system restore.

You should also keep a strict and frequent plan of backing up your system databases, preferably on a daily basis, if you’re frequently changing instances. Otherwise, for installations which are more stable, you can do this backup operation with less frequency.

If you have already considered taking a look at SQLBackupAndFTP for your backup process, I still have good news regarding this software.

As you can see above, it also offers you the possibility to backup, along with your user databases, the system databases just with one extra click. You still have the options to back these databases up with a schedule and to your favorite location, be it on the same disk or in a different location.

Conclusion

If you are to follow these best practices in your backup process you will eventually find an optimum and the most efficient combination of steps to manage your organization’s databases. Software is here to help us and make our lives easier, while making us more efficient. This is why, for some of the steps above I recommend using SQLBackupAndFTP, which in my opinion has the most complete and easy to use set of tools necessary to manage your databases.

Also, by incorporating any of the steps above in your backup practices you are going to improve your organization’s efficiency against data loss and speed up the recovery process of your data.

Reference: Pinal Dave (http://blog.sqlauthority.com)

The post 9 Things You Should be Doing with Your Backups – Backup Tips – SQL Server Backup Tips appeared first on Journey to SQL Authority with Pinal Dave.

At this rate, I think I am going to exhaust the whole error set available in SQL Server. I am glad that I am able to get into unique situations and then resolve them too. And this is more of a diary of the error messages I am getting into. I was playing with my Demo database on SQL Server 2016. Once done, I started restoring from the previous copy.  But is failing with below error 3169:

Msg 3169, Level 16, State 1, Line 1
The database was backed up on a server running version 13.00.0801. That version is incompatible with this server, which is running version 12.00.4213. Either restore the database on a server that supports the backup, or use a backup that is compatible with this server.
Msg 3013, Level 16, State 1, Line 1
RESTORE DATABASE is terminating abnormally.

Then I realized that I have taken back-up from 13.00.0801 which is SQL Server 2016 and I was trying to restore on 12.00.4213 which is SQL Server 2014. The error message shows both versions, source and destination.

So, it’s is clear that there is no direct way to downgrade from a higher version to a lower version and it would fail with an error message. Why? When an instance of SQL is upgraded, not only do the binaries for the database engine change, the schema level for the databases also changes. So it would not be possible to attach a database with a higher schema level to an instance on a lower version of SQL.

What can be done? One possible approach is to manually export and import the data. You can follow the below steps:

Create an Empty Database

• In Object Explorer, expand Databases, right-click a database, point to Tasks, and then click Generate Scripts. Follow the steps in the wizard to script the database objects.
• On the Choose Objects page, select “Script entire database and database objects”.
• On the Set Scripting Options page, select Save scripts to a specific location.
• Select the Advanced button; under “Types of data to script” select “Schema Only” and under “Script for server version” select “SQL Server 2008” (or appropriate version)
• Click on Next and finish to complete the process.
• Now you can use the generated script file and execute this against the SQL server 2008 instance to create the complete database schema without data.

Move the data

To move the data, you have a couple of options, but all of these are a bit tricky.

1. Script out the data exactly the way described in the above steps, just by selecting “Types of data to script” as “Data Only”.

Or

2. Script out the data for every object individually one by one instead of complete database if you have a large database.

Or

3. BCP to export the data into files and then again use BCP to import the data back into the tables onto the new server.

In short, moving back-up from higher to lower version is not possible. Only option we have is to move the data using the ways described above. Do you know any more ways?

Reference: Pinal Dave (http://blog.sqlauthority.com)

The post SQL SERVER – FIX Error 3169 – The database was backed up on a server running version appeared first on Journey to SQL Authority with Pinal Dave.

ApexSQL Backup is a powerful yet simple tool for Microsoft SQL Server which represents a fully equipped command center for database backup and restore jobs management. It enables users to create and schedule backup related jobs (backup, restore, log shipping, index defragmentation…) and monitor all scheduled backup activities or overview complete backup history on multiple servers across the domain.

ApexSQL Backup is designed to create and work with native Microsoft SQL Server backups and databases, and supports all backup options which can be used in the SQL Server Management Studio and encompasses them in a simple and user friendly UI which revolves around simple wizards for each particular job.

In addition to improvements to already existing native features, ApexSQL Backup offers some unique functionalities that make a backup jobs creation and scheduling a fun and easy tasks. ApexSQL Backup users can create specific policies which include complete information and configuration of backup jobs as well as the scheduling setup, and then apply them to different databases across multiple SQL Servers, both local and remote. This enables fast implementation of full, differential or transaction log backup jobs for many databases at once. Furthermore, multiple policies can be applied to the same database, which means that after creating and fine-tuning the backup policies, they can be applied to all applicable databases with just few clicks.

ApexSQL Backup components

ApexSQL Backup is composed of three components:

• GUI – an interface for job scheduling, manipulation and execution. It contains ApexSQL Backup service which communicates with all remote Agents
• Agent – a simple component (service) which allows ApexSQL Manage to audit SQL Servers on the local system by allowing communication
• Central repository database – stores information and configuration data, very small database since the information stored in it is minimal

One central repository database can be used to store data for unlimited number of SQL Servers (connected to multiple ApexSQL Backups agents), as well as multiple GUI all over the network – which allows multiple users to create and execute backup/restore jobs across all servers in the domain.

How does it work

As mentioned before, to use ApexSQL Backup, one needs to install ApexSQL GUI on the workstation which will be used to work with the tool and configure/monitor all jobs. ApexSQL Backup central repository database can be installed on any accessible SQL Server, and ApexSQL Backup Agent must be installed on the machine which hosts SQL Server you want to manage with ApexSQL Backup.

Note: ApexSQL Backup Agent can be installed from the installer file, or directly from the application GUI when connecting to the SQL Server which you will work with.

After installing all ApexSQL Backup components, the only remaining task is to add SQL Servers you want to manage from the main GUI form. Simply click on the Add button in the left pane and provide connection information in the Add SQL Server dialog.

Once the server is added, maintenance tasks implementation can be started since there are no additional configuration requirements.

Maintenance tasks with ApexSQL Backup

As mentioned before, ApexSQL Backup is based on simple step by step wizards which are designed to quickly guide the user through the task creation and help them create the most appropriate jobs with just a few clicks, while enabling those with more complex tasks to create specific backup jobs with many customizable options. Here is how:

While the main ribbon offers variety of maintenance tasks, we’ll choose a Backup job for this example, but keep in mind that all other jobs have similar wizards for the job and scheduling with small differences to accommodate job-specific options and features.

So, after selecting the backup job from the main ribbon, the backup creation wizard starts and the first requirement is to choose one or more databases to backup, choose a backup type as well as whether to backup the database itself, or to create backups of files/filegroups.

The next step allows you to customize naming policy by utilizing various wildcards or by typing a specific name for the backup file.

In addition to the naming policy, destination and mirror paths should be specified, and the user has an opportunity to specify some output options.

ApexSQL Backup offers various options for all supported maintenance tasks, and all related to the backup job can be configured in the following step of the wizard. The user can specify backup file information, choose verification, compression and encryption settings and fine-tune the backup creation.

In the final step of the wizard, users can specify execution schedule(s) for the backup task, ranging for immediate execution, manual execution to unattended, fully automated execution regular basis. Once the scheduling is completed, the wizard can be finished, and task is immediately added to the schedules view which shows information on all existing schedules.

In the same manner, once the scheduled job has been executed, it will be shown in the activities view, regardless of the job status, full information will be shown even for the jobs that have failed due to some (un)expected reasons

For more information and details on scheduling backup jobs, you can check out this article.

In addition to previously mentioned views, there is a third view – Backup history – which allows users to check and track all existing backups on the database backups. This includes all existing backups created for the specific database – even those created without ApexSQL Backup, and even before ApexSQL Backup and Agent were installed. While this overview allows users to historically see all existing backups, it also allows them to restore selected database or it’s files/filegroups to a specific point in time – simply select the backup file from the list and choose the appropriate restore option.

Aside from the Backup maintenance task, other jobs can be scheduled through similar wizards and appropriate jobs will be automatically added to the Schedules and Activity panes.

Restore backups – schedule and execute database restore, file/filegroups restore, or point in time restore.

Ship logs – schedule and execute a combination of backup and restore jobs to create a solution that provides disaster recovery protection at the database level and for off-site reporting – see more in this article.

Cleanup history – schedule a cleanup of msdb database to remove the backup/restore information history from the msdb database.

Rebuild & reorganize indexes – schedule and index maintenance tasks to diminish index fragmentation and increase database performance.

Execute SQL Scripts – pretty much self-explanatory – schedule an execution of T-SQL script file or T-SQL command on the server.

Another convenience of ApexSQL Backup is the ability to notify user(s) upon task completion. In addition to storing the information and displaying it in the Activities view, all jobs can be configured to trigger a notification email to specified email addresses when the job is successfully executed, or a failure has occurred. This is typically configured in the last step of the wizard on each maintenance task.

For all those that prefer to work via command line interface, ApexSQL Backup fully supports CLI, and all GUI functionalities can be achieved by the usage of appropriate switches in the cmd.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

The post SQL SERVER – Database Backup and Restore Management and Automation with ApexSQL Backup appeared first on Journey to SQL Authority with Pinal Dave.

Solving puzzle is something everyone would love to do. It gets our creative thoughts and the things that it challenges how we can solve problems. Recently I bought a puzzle which my kid had to finish in her school holidays. I was amazed to how complex it was. I saw that my kid and wife had taken almost a week to solve it. But once they finished, the expression in their faces to show-off was amazing. I felt it was worth every penny I had spent on it. Few people like detective series because of the same reason. And I also got a similar chance while working with a client in the recent past. Here is the puzzle which they wanted me to solve for Error 112.

As per my client, the problem started a few weeks ago. On random basis the backup of the database was failing with below error in maintenance plan history.

Executing the query “BACKUP DATABASE [PRODDB] TO DISK = N’\\…” failed with the following error: “The operating system returned the error ‘112(There is not enough space on the disk.)’ while attempting ‘SetEndOfFile’ on ‘\\\\fs01\\_SQL_Backups\\proddb_backup_2016_04_06_231005_3910233.bak’.

BACKUP DATABASE is terminating abnormally.

As we can see, the backups were taken at network location. They took a backup locally and it was lesser than free space available so backup would have succeeded. I have asked if there has any special setting done for backup. All they told me that they are using compressed backup.

Resolution for Error 112

I did some research and found that the backup size of compressed backup would be different in the beginning and at the end. As we can think, the size completely depends how compressing the data is and SQL Server cannot determine that beforehand. Based on my research, compressed backup completely works on the pre allocate, it may succeed if disk space is less and may fail if disk space is more depending on the pre-allocation file size and final compressed backup size.

I have asked to enable the trace flag 3042. By enabling this trace flag, backup bypasses the “pre-allocation algorithm”. This means it will have the most optimized space beforehand.

Here is the code to enable the traceflag 3042.

DBCC TRACEON (3042,-1);
GO

If due to any reason, you want to turn of the traceflag, you can use following code.

DBCC TRACEOFF (3042,-1);
GO

As a general rule of thumb, I have asked my client to have at least 1/3 of the volume of backup. So in this situation, the size of backup database is 50 GB then 1/3 should be at least 17 GB. I have asked them to make sure that they have that much space available in the backup drive. Again, this is not an algorithm, it is general guidance. True space can only be determined after the compressed backup is completed.

After enabling the trace flag 3042, SQL Server really grew the backup file bit by bit.

You can read more about this feature in my blog below.

• Introduction to New Feature of Backup Compression
• Data and Page Compressions – Data Storage and IO Improvement
• Effect of Compressed Backup Setting at Server Level on Database Backup

Reference: Pinal Dave (http://blog.sqlauthority.com)

The post SQL SERVER – Backup Randomly Failing with Error 112 (There is not enough space on the disk.) appeared first on Journey to SQL Authority with Pinal Dave.

Using scripts can be a powerful way to automate things that a GUI will fail in many cases. I was working with a client who wanted to write a PowerShell script that would help him automate his backups. This is a pretty simple requirement that can be easily achieved using SMO as shown below in my scripts. I have been using these scripts for a number of years at various places. Let us see how we can backup every database in SQL Server.

I really was not having anything in mind, but went the extra mile to understand why the DBA was asking such script as he had maintenance plans already in place. The DBA told me this script was useful because he would use it in places when he needs to do some sort of upgrade to the SQL instances and before applying the hotfixes or service packs, he would take an ethic out of the cycle backup of his databases to make sure he has a fall back option in case the installations failed because of any reason. I personally felt this was a nice use case for such scripts.

In the below script, I have used the default instance running on my machine and feel free to change the same accordingly. Also create a folder where the backups need to land. Make sure you have the required space on the drive where this backup will be taken.

$SQLInstance = "." 
$BackupFolder = "C:\Data\" 

$timeStamp = Get-Date -format yyyy_MM_dd_HHmmss 

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | Out-Null 
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SmoExtended") | Out-Null 

$srv = New-Object ("Microsoft.SqlServer.Management.Smo.Server") $SQLInstance 
$dbs = New-Object Microsoft.SqlServer.Management.Smo.Database 
$dbs = $srv.Databases 

foreach ($Database in $dbs) 
{ 
 $Database.name 
 $bk = New-Object ("Microsoft.SqlServer.Management.Smo.Backup") 
 $bk.Action = [Microsoft.SqlServer.Management.Smo.BackupActionType]::Database 
 $bk.BackupSetName = $Database.Name + "_backup_" + $timeStamp 
 $bk.Database = $Database.Name 
 $bk.CompressionOption = 1 
 $bk.MediaDescription = "Disk" 
 $bk.Devices.AddDevice($BackupFolder + "\" + $Database.Name + "_" + $timeStamp + ".bak", "File") 
 
TRY {
 $bk.SqlBackup($srv)
 } 
CATCH 
 {
 $Database.Name + " backup failed." 
 $_.Exception.Message
 } 
} 

When I ran this on the PowerShell ISE window, I got an output with the list of databases.

You can see that the tempDB database will fail with an error, but that is absolutely fine. In this case, we have not done any filter on our list of databases. We will also backup the system databases as shown in the figure below.

With the powershell script displayed in this blog post we can backup every database in SQL Server.

What automation have you done in your environments for this requirement? Do you have something similar running on your servers? Do share the scripts and spread the happiness along. Let us learn together more.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

The post PowerShell Script – Backup Every Database In SQL Server appeared first on Journey to SQL Authority with Pinal Dave.

I have been lately writing on powershell and didn’t know there are many more scenarios that can be enabled. I was traveling for a consulting assignment over the weekend for a customer location. Though this exercise was around performance tuning their systems, it was amazing to see that they had some of the basics not done either. In this blog we will be discussing about how to backup SQL Server System Databases.

The post PowerShell – Backup SQL Server System Databases appeared first on Journey to SQL Authority with Pinal Dave.

The other day, I received an email from a user, who had dropped master database in their system. I told him directly that he should have not dropped master database as that is a very important system database. It is really strange that after so many years quite a few of the people do not know what is a master database in SQL Server.

Question: What is a Master Database in SQL Server?

Answer: Master database is system database and it contains information about running server’s configuration. When SQL Server is installed, it usually creates master, model, msdb, tempdb resource and distribution (last two depends on version of SQL Server) system database by default. This is the one which is absolutely must have a database. Without this database SQL Server cannot be started.

If any database is corrupted it should be restored from the latest backup, considering SQL Server was able to start with a damaged database. Sometimes when this database is corrupted so much it cannot start SQL Server at all and it cannot be restored. SQL Server should rebuild using command prompt and restored from the latest backup once again.

If you ever lose your master database or got it corrupted you can easily fix it as well. I have documented in this link about how to fix master database.

Additionally, if your master database size ever goes big, here is interesting blog post about that subject. Additionally, you should always backup master database just like any other database.

Backup of the this database should be made for:

• Changing server-level configuration settings;
• Changing database-level configuration settings; and
• Changing any logon accounts details.

When someone talks about master database, I always think of Yoda. Here are a few interesting quotes from Yoda.

• “A Jedi uses the Force for knowledge and defense, never for attack.” The Empire Strikes Back
• “Adventure. Excitement. A Jedi craves not these things.” The Empire Strikes Back
• “Judge me by my size, do you?” The Empire Strikes Back
• “Fear is the path to the dark side…fear leads to anger…anger leads to hate…hate leads to suffering.” The Phantom Menace
• “Do. Or do not. There is no try.” The Empire Strikes Back

The last one is the most popular and my favorite quote from Yoda. Yoda also said “Always pass on what you have learned.”, well, I request you to share this blog post with your friends and you will make Yoda happy.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

First appeared on What is a Master Database in SQL Server? – Interview Question of the Week #076

Recovering lost data that was deleted or truncated can be fairly quick and easy, depending on the environment in which the database resides, recovery measures implemented before and after the data loss has occurred, and the tools used for the job. In this blog post we will learn about how to recover truncated or deleted data when a database is (or not) available.

In general, there are three specific scenarios for recovery when the data is lost:

Scenario #1

Full backup has been created before the unintended delete/truncate operations have occurred, and no additional database changes have been performed since. This is by far the easiest data loss to solve, since it can be recovered with the simple restoration of the full database backup, since it will bring the database to the pre-disaster state, and since no additional changes have occurred, the recovery process will be fully valid and completed with the simple restore.

Scenario #2

Same as in the first scenario, the full database backup has been created prior to the deletes/truncates, but these were followed by the additional changes which need to be preserved. If we would restore the full database backup as in the first case, we would lose all of these additional changes, so another solution should be implemented. The most obvious solution is to restore our full database backup as a different database, and then to extract the tables affected by truncate/delete operations, and to import them into the original database.

Even though this solution is easy for smaller databases, in some environments, it is not possible to fully restore full database backups, due to the resources consumed (backup and database size, restoration time, restoration performance impact…). Logical solution here would be to extract those tables for recovery from the full database backup, but this cannot be done via SQL Server mechanisms, nor via SQL Server Management Studio.

In order to extract specific tables from a database backup, we can use ApexSQL Recover, a powerful recovery tool for Microsoft SQL Server which can recover data and structure lost due to delete, drop and truncate operations, recover deleted BLOBs, or extract BLOBs or specific tables directly from database backups without the need to restore them first.

To extract the tables:

1. Install ApexSQL Recover on your workstation (the tool can be used to perform recovery both locally or remotely, so it can be installed directly on the production server to locally complete the recovery/extraction, or it can be installed on the workstation of the user’s preference to remotely connect to the SQL Server and perform recovery from there)
2. Start ApexSQL Recover and choose the option to extract data from a database backup in the main ribbon
   
3. Add database backup file by clicking on the ‘Add file’ button and selecting your full database backup. If you have consecutive transaction log backups which need to be used for table extraction, add them in this step, and check them together with the full database backup
   
4. In the next wizard step, ApexSQL Recover will show the list of tables in the database/backup, and it is recommended to exclude all tables which are of no interest for recovery and to check only tables which were affected by the truncate/delete operations
   
5. In the next step of the wizard, between the choice of saving the recovery script to the .sql file and recovering to a new database, let’s opt for the first option, which enables us to inspect the recovery script, or to execute the recovery script against the production database to complete the recovery. We can also specify the script location.
   
6. In the final step, we should select the output type. Since we want to recover only data lost due to truncate/delete operations, choose the ‘Extract data only’ option and click next
   
7. After ApexSQL Recover completes processing, the application will inform the user on the recovery outcome, and users can now inspect the script directly from the application internal editor, or open it manually in a SQL Server Management studio or similar tool, from where it should be executed against the production database to complete the recovery process
   

Scenario #3

A different scenario, and by far the worst one, is when a disastrous truncate/delete occurs, followed by additional important transactions which we also do not want to lose, but there is no full database backup to use as the recovery source. Since we do not have a database backup, restoring it, or extracting table data with ApexSQL Recover is not an option.

Luckily, we can use ApexSQL Recover to perform the recovery of truncated/deleted data, since it does not require full database backup in the recovery process, but instead utilizes information in the LDF and MDF files to rollback unintended changes. With this in mind, knowing the internal mechanisms of the SQL Server itself, prior to engaging the actual recovery, due to the fact that SQL Server overwrites MDF file information frequently, it is important to ensure that the information within is kept safe, and not overwritten by the upcoming traffic.

In general, here are the immediate steps to take once the disaster is detected:

1. Change database mode to ‘Read only’ to prevent overwriting of the information in the MDF file
2. Take the database offline and make copies of database LDF and MDF files, then take the database back online – these copies can then be used to create additional copies of the database which can be used to read recovery information from, and enable the production database to continue to receive changes while the recovery process is prepared and performed.

Even though both of these measures will mean that your database will have some downtime, it is important to take these precautionary steps whenever possible to secure the highest change of successful recovery.

As said above, we’ll use ApexSQL Recover to connect to the database affected by unintended truncate/delete operations and generate a recovery script. Here is how:

1. Start ApexSQL Recover and choose to recover ‘Deleted data’ or ‘Truncated data’, depending on the operation which you need to recover from – the remaining steps of the wizard will be the same regardless of this choice
   
2. In the next step, provide the connection details to your SQL Server instance, provide connection credentials and choose the database for recovery
   
3. In the next step, we can go with any of the three options, depending on our case:
   1. ‘Help me decide’ option will guide the user through a quick series of questions in order to provide the most appropriate resources
   2. ‘Add transaction logs’ option should be used if we are creating transaction log backups so we can use them in addition to the MDF and LDF files as the recovery source
   3. ‘No additional transaction logs are available’ option should be chosen if our transaction log is not being backup-ed
      
4. Regardless of the choice in the previous step, once we proceed, we will arrive to the step which is used to determine the recovery window – a time frame which will be considered for the recovery. If we do have a knowledge of the time frame where a disaster has occurred, we should specify the time frame, as close as possible, which will result in faster processing, and in a more accurate recovery (additional delete/truncate operations may have occurred close to the disastrous once, so we will want to exclude those from the recovery process by using proper date/time filters)
   
5. The next step allows us to further filter-out the recovery and narrow it down to specific tables, so it is recommended to include only tables affected by disastrous truncates/deletes
   
6. Same as when we were extracting table information in the scenario #2, we will choose to create a recovery script, and once the processing is finished, our recovery script can be inspected and finally executed against the production database to complete the recovery process
   

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – How to Recover Truncated or Deleted Data When a Database Backup is or is not Available

Here is a recent email which I received from Madhu. He is a beginner in the SQL Server and when he tried to take a backup from SSMS (SQL Server Management Studio), he got error related to backup file. As soon as he sent me an email I knew what was the exact problem, he was facing and I was able to help him quickly. Let us first recreate the same error and later on see how to fix the same.

When he tried to take backup of his database, he received following error:

System.Data.SqlClient.SqlError: Cannot use the backup file because it was originally formatted with sector size 4096 and is now on a device with sector size 512. (Microsoft.SqlServer.Smo)

You can see in the following image the error window in SSMS.

Well, the solution for the same is very easy.

Solution / Workaround:

The reason for the error is that the backup file which was created was created with different sector size and it is not possible to use it now. In the SSMS we can see there is already one file in the Destination.

First remove the existing file in SSMS.

Next, add a new file to the backup location.

That’s it! We are done!

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Fix Error – Cannot use the backup file because it was originally formatted with sector size 4096 and is now on a device with sector size 512

ApexSQL Backup is a tool for Microsoft SQL Server, intended for database backup and restore job management. The application supports all native SQL Server backups (full, differential and transaction log backups), and allows users to easily create, save and manage all backup related jobs. ApexSQL Backup can be used to run, schedule and monitor all backup operations across the domain from a single location.

Database Backup and Restore Job Management

Creating and maintaining full backup chains is a priority for any disaster recovery plan. But running these jobs manually on a daily basis could be quite challenging and time consuming. ApexSQL Backup enables the automation of these jobs by applying schedules, or by running custom scripts in the command line interface. Furthermore, ApexSQL Backup supports the automation of other daily maintenance tasks such as deleting old backup files, rebuilding and reorganizing fragmented indexes, verifying created backups and much more.

ApexSQL Backup components

ApexSQL Backup consists of four main components:

• Windows application – main part of the program, managed via GUI. Used for creating, managing and executing database backup and restore maintenance jobs in SQL Server.
• Command line interface aka CLI – a console application that enables users to automate backup, restore, and log shipping maintenance jobs by running custom scripts. All jobs in a script are easily configured by using simple CLI switches.
• Central repository database – SQL Server database that stores parameters for all created jobs as well as configuration settings data for the application
• Agent – simple windows service that is used for communication between the application and SQL Server instances.

How does it work

In order to work properly, ApexSQL Backup needs all four components to be installed and connected. The application and central repository database are usually installed on the machine that will be used for managing and monitoring SQL Server instances inside the domain or a workgroup. ApexSQL Backup agent must be installed on each server machine that hosts one or more SQL Server instances that need to be managed. After all agents are running, each SQL Server instance can be added to the list of manageable servers.

To add the local SQL Server instance, click on the Add button in Home tab, and select local instance from dropdown menu. Select the authentication type for the selected server, and provide necessary credentials if SQL Server authentication was selected.

To add a remote instance, type the name or IP address of the instance in SQL Server field. Alternatively, click on the Network server button to obtain the list of available network servers, and select the one that should be added to the list (in order to have all remote servers listed, make sure that SQL Server Browser services are enabled and running on the remote servers).

As soon as all instances are added to the server list, they can be managed through the application interface. ApexSQL Backup contains many useful features that make database maintenance tasks fun and easy.

Backup job management

Making a foolproof disaster recovery plan should be one of the main priorities for every database administrator. Creating uninterrupted backup chains that consist of regular full, differential and transaction log backups is the base concept behind any serious disaster recovery plan. ApexSQL Backup enables the user to configure and automate the entire process of creating any type of SQL Server database backups.

To make a database backup click on the Backup button in the ribbon of the Home tab

In the main tab of Backup wizard, provide the main parameters for the backup job:

• Select SQL Server instance that needs to be managed
  
• Click on the Browse (…) button in Databases box, and check the boxes in front of the databases that you wish to back up
  
• Select the backup type (full database, differential database, transaction log, or filegroups backup) from the drop down menu in the Type box
• Set name and description parameters for the job name in Name and Description boxes
• To change default SQL Server backup path, and assign custom backup destination path, click the Browse button (…) under the Path In the opened form, set the backup folder, and specify the naming rules for the backup file.
  
• If you want to execute the backup job only once, select the Execute immediately radio button at the bottom of the main tab in Backup wizard. If defined job needs to be executed on regular basis, set the schedule for the job, by selecting Schedule radio button. Set the frequency and duration for the schedule, and click Ok to save the schedule settings.
  

It is possible to set additional job parameters in Advanced tab of the backup wizard, such as: media, verification, compression, encryption and cleanup settings.

Optionally, configure the email notifications for the job status.

Click Ok to schedule or execute defined backup job.

Backup templates

Another interesting and practical way of creating backup jobs is by using backup templates. Users can save all preferred settings for a backup job as a template, and apply it to one or more databases whenever necessary. Backup templates support all features that are supported for the regular backup jobs.

To create a new backup template, navigate to Home tab, and click on the Manage button in Backup templates group.

• Click on the New button to configure the template.
  
• Provide the custom name and description for the template. Select the backup type for the template: full, differential, or transaction log backup. Naming rules may be defined for operation name and operation description (this will be displayed in Activities view), as well as for the backup filename. Schedule also needs to be set for every backup template.
  
• Use the Advanced tab to set media, verification, compression and cleanup options. If necessary, set the email notifications in Notifications tab, and click OK when finished to save the template settings.
  

To deploy created template, click on Deploy button from Backup templates group in the Home tab.

In Deploy templates form, you need to specify:

• One or more templates that you wish to deploy – select previously created templates by clicking on Browse button in Templates text box.
• Databases that will be targeted by the template – select the target databases by clicking on Browse button in Databases text box
• Destination path for the backup files – type the path manually, or browse for the destination folder

After all targets have been selected, click on Deploy button to apply the template(s).

All backup jobs configured this way are created as schedules, and they can be managed in Schedules view as such.

Restore job management

Other important feature of ApexSQL Backup is the ability to configure and manage restore operations.

To start the restore job configuration, click on the Restore button in Home tab.

In the main tab of Restore wizard, provide all required parameters for the job configuration:

• Select the targeted SQL Server instance, and the database that needs to be restored from the drop down menus in SQL Server and Database
• Choose to restore from a folder scan or from a specific backup files by selecting the respective radio button.
• Select the desired restore type: Full; Full and differential; or Full, differential and transaction log restore. Note that you need to have suitable files selected in previous step to be able to execute any of the selected restore type operations.
• Configure the restore job to run immediately, or set the schedule if job needs to be executed frequently.

In Advanced tab of restore wizard, you can set the custom location for data and log files, include various DBCC operations in restore job, or provide the custom T-SQL script and run it after the restore job is done.

Optionally, set email notifications in Notifications tab, and click Ok to schedule or execute the configured job.

Log shipping jobs

Setting the regular log shipping jobs is one of the simple, cost effective ways of running the successful disaster recovery plan. It may not be as powerful as failover clustering and Always ON Availability Groups, but it is much easier to configure, and has much lower cost in human and server resources.

To configure the log shipping job, click on the Log shipping button in the main ribbon of the Home tab.

In the main tab of Log shipping wizard, several parameters must be provided:

• The primary (production) server is selected from dropdown menu in SQL Server textbox
• The source database on a primary server is selected in Database box
• The paths for the backup and shared folder must be provided
• Backup and restore schedules have to be configured at the bottom of the form.

• To configure the settings for the secondary (destination) server, click on Add destination In the Log shipping destination form, specify the destination server, destination database, recovery model and local folder (note that destination server has to be added to the list of manageable servers in the main form). If needed, multiple destination servers and/or databases can be added to the list. Click Ok to save the settings.

• To complete job configuration, click OK. The wizard scheduled two operations: backup of transaction log on primary server, and restore of the created file on secondary server.

• To start log shipping immediately, go to Schedules view, select both created schedules, and click Run now from the context menu

Reorganize and Rebuild indexes

ApexSQL Backup can handle some basic index maintenance operations. Rebuilding and reorganizing indexes regularly can significantly improve the performance of high traffic servers. On the other side, heavily fragmented indexes may degrade the overall query performance on the server.

To start the rebuild or reorganize index operation, click on the respective buttons in the Tasks group of the Home tab.

Reorganize index wizard allows users to specify SQL Server instance and database, as well as to manually select tables and views for the reorganize job. The schedule may also be set for the job, so it can be automated to run on a regular basis. Email notification are configured in Notification tab.

Rebuild index wizard works in a similar way. SQL Server instance and database are selected from the drop down menu, and tables and views are added from the list. The operation can be run both in online and offline mode. Rebuild index operation also supports schedules and notifications, and can therefore be automated.

Monitoring backup jobs

All described jobs and operations can be easily monitored and managed in Schedules, Activities, and History views.

Schedules view shows all scheduled maintenance jobs and job details in a grid. The jobs include all types of backup and restore schedules, rebuild and reorganize index schedules, log shipping jobs, etc. All schedules located in the grid can easily be managed by using the commands from the context menu. They can be executed immediately, enabled, disabled or deleted.

Activities view displays detailed information about all operations that were executed by ApexSQL Backup, either directly, or by the scheduled jobs. The form keeps track for both successful and failed operations, and can therefore be very useful in diagnosing various issues. Furthermore, all error messages that were returned by SQL Server are saved in the Message column. All activities can be grouped, sorted, or filtered by the values from any column. If the need arises, activities can be exported as CSV document.

History view shows the complete backup history (since the moment ApexSQL Backup is installed) for any selected database. The backup history can be viewed in the grid and on the timeline. This is very useful for monitoring backup chains, and identifying best restore point in the case of the disaster. Database can be restored easily from the context menu by selecting any of the available backup points.

Related links to Database Backup and Restore Job Management:

• What is SQL Server log shipping?
• SQL SERVER – Select the Most Optimal Backup Methods for Server
• Reorganize and Rebuild Indexes

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Database Backup and Restore Job Management

Question: What is a Backup Timeline for Restore Process?

Answer: One of the most popular interview question about Backup and Restore Database process. I find it very interesting that after so many years of this concept introduced, but after so many years users do not know the different types of backup and process to restore the database.

In general, databases backup in full recovery mode is taken in three different kinds of database files.

1. Full Database Backup
2. Differential Database Backup
3. Log Backup

What really perplexes most people is differential database backup.

After interacting with many DBAs I have realized that it is quite confusing to most of the people how a differential database is useful and many are not aware of the correct method to restore the full recovery model. Before I start explaining please understand the following diagram where I have demonstrated time line when a backup was taken.

Let us remember the golden rule for restoring first.

‘After restoring full database backup, restore latest differential database backup and all the transaction log backup after that to get database to current state.’

Here is the article which you want to read to understand this concept in detail.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on What is a Backup Timeline for Restore Process? – Interview Question of the Week #092

I break a lot of things on my SQL environment and I believe that is a very good way to learn things. Today, I wanted to perform a restore of a database using a backup which I took earlier. In this blog post, let us learn how I fixed Property BackupDirectory is Not Available for Settings.

As soon as I clicked database, I was welcomed with the below error message.

Here is the text of the message.

Event ID: 7011
Property BackupDirectory is not available for Settings ‘Microsoft.SqlServer.Management.Smo.Settings’. This property may not exist for this object, or may not be retrievable due to insufficient access rights. (Microsoft.SqlServer.Smo)

I was not sure from where I would get BackupDirectory property so I capture profiler trace while clicking on restore option and found below when I searched for BackupDirectory

declare @BackupDirectory nvarchar(512)
if 1=isnull(cast(SERVERPROPERTY('IsLocalDB') as bit), 0)
select @BackupDirectory=cast(SERVERPROPERTY('instancedefaultdatapath') as nvarchar(512))
else
exec master.dbo.xp_instance_regread @HkeyLocal, @InstanceRegPath, N'BackupDirectory', @BackupDirectory OUTPUT

After debugging further, it was easy to find that its looking for a registry key for my SQL instance and then I recalled that I was playing with a registry setting for one of my previous blog posts.

Solution

I found that below setting was empty.

As soon as I entered a value in the highlighted text box of “backup” and hit OK, I was able to get to the restore UI without any error.

Have you ever used default setting for backup? Do you know the use of it?

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Error: Property BackupDirectory is Not Available for Settings

I always like to keep a number of scripts handy in my USB drive with my Laptop. These scripts are used for my internal use when I am suddenly looking for something interesting in customer environments. One of the scripts that I use is shown below. Many a times I have seen customers don’t really have any backup strategy. And when I start working with them, I call out some of the basic backup techniques that they need to be doing.

I have learnt and advised on a number of occasions that the backup strategy is not to be thought about in isolation but needs to be built keeping the restore strategy in mind. Your restore strategy and the RTO, RPO conversation with the stakeholders will decide how a typical backup plan needs to be done.

Let me get back to the fundamental problem statement that I wanted to share. The below powershell script is used to identify the backups taken for each of the databases.

PowerShell Script to Listing all Backups

</gs>Import-Module SQLPS
$ServerName = "BigPinal"    #"Replace with your <gs id="6563431d-bbce-4739-9279-780f3366dd87" ginger_software_uiphraseguid="72e9cd3c-57f5-4072-9401-0a73c9fb23cf" class="GINGER_SOFTWARE_mark">servername</gs>"
$InstanceName = "Default"  #"Default"
$ConnectionString = "SQLSERVER:\SQL\"+$ServerName+"\"+$InstanceName+"\Databases"
<gs id="639d042d-12fa-46c2-ab73-7c5076cd39a5" ginger_software_uiphraseguid="72e9cd3c-57f5-4072-9401-0a73c9fb23cf" class="GINGER_SOFTWARE_mark">dir</gs> $ConnectionString | %<gs id="7c3d4b49-704c-482d-a88e-55639226824a" ginger_software_uiphraseguid="72e9cd3c-57f5-4072-9401-0a73c9fb23cf" class="GINGER_SOFTWARE_mark">{</gs>$_<gs id="b5a1fed9-8b3c-4f2f-b649-d7b8f341d772" ginger_software_uiphraseguid="72e9cd3c-57f5-4072-9401-0a73c9fb23cf" class="GINGER_SOFTWARE_mark">.</gs>EnumBackupSets<gs id="baa4cdce-97d3-4780-b666-bb4ce951ec74" ginger_software_uiphraseguid="72e9cd3c-57f5-4072-9401-0a73c9fb23cf" class="GINGER_SOFTWARE_mark">(</gs>)} | ft 

The output for the same would look like below:

You might be of the opinion on why you would use. This is a simple yet powerful script you can use to initially take a dump to understand the various backups. I take a quick look at this output with the list of databases to understand if there are databases which are not being backed up or if there is a plan for backups or not.

I know this is a simple task, but you will be surprised to how many still make basic mistakes in their environments. Do let me know what have you got from your environments and what were some of your findings? I am sure want to learn from you too.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

First appeared on PowerShell – Script to Listing all Backups Taken with SQL Server

The era of innovation with SQL Server administration side of things keep happening, it is strange that a lot of us miss these capabilities getting added into the system. I have seen processes define the way how DBA’s work. In a lot of places, the processes, scripts and behaviors are brought forward without a second thought. I wanted to share a recent story that happened at one of my client’s place. Let us talk my experience about playing with backups and compressions.

I was brought in by one of my clients for performance tuning exercise. This was a routine procedure that I undertake. When I was at their office, one of the DBA asked if I could also look at their standard procedures used for daily Agent jobs. As I had a few hours left in that exercise, I thought to spend time with them. I went about looking at some of the maintenance plans and the scripts they were using. I went about asking when these were made, how they were brought forward from various migrations and so on. This is when I realized, some of the scripts for backups used by them were created for SQL Server 2005 time and were running easily even after 2 upgrades.

I went about asking why they were not using the backup compression feature of SQL Server? This is when the DBA asked me about how easy it change the script and if there are advantages that I need to know. I helped them change the script and this was a simple yet powerful change for them. The backup size reduced drastically. To showcase the same, let me run a script and show you.

-- Normal Backup performed on the Database
BACKUP DATABASE [AdventureWorks2016] TO DISK  
= 'C:\Backup\AdventureWorks_uncompressed.bak' WITH STATS = 10, NO_COMPRESSION, FORMAT;
GO
-- Normal Backup (with Compression enabled) performed on the Database
BACKUP DATABASE [AdventureWorks2016] TO DISK  
= 'C:\Backup\AdventureWorks_compressed.bak' WITH STATS = 10, COMPRESSION, FORMAT;
GO

As you can see, the above command changes just in a single word and it can start using the backup compression abilities.

Our example was the easiest and you can see from the backup sizes that this has got us 5 times disk compression. This is a powerful and lesser appreciated feature when it comes to working with SQL Server.

I am sure you would have used this feature in your environment too, do let me know how efficient this has been for you? Send a note via comments below.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Playing with Backups and Compression

It is second to human nature to compare when given options. When I wrote about the blog, SQL SERVER – Playing with Backups and Compression, I did compare some of the space savings one will achieve when working with backups and compression. Though the space savings were significant, one of the blog readers wanted to understand how this would be on time? Is it faster or slower to do the same. I went about taking the same script and I collected the time taken. Let us learn about Performance Analysis of Backup to Azure.

SQL Server parse and compile time:
CPU time = 0 ms, elapsed time = 0 ms.
10 percent processed…. <<<removed for blog>>
70 percent processed.
Processed 53423 pages for database ‘AdventureWorks2016’, file ‘AdventureWorks2016CTP3_Log’ on file 1.
100 percent processed.
BACKUP DATABASE successfully processed 93263 pages in 1.970 seconds (369.852 MB/sec).
SQL Server Execution Times: CPU time = 140 ms, elapsed time = 2180 ms.

SQL Server parse and compile time:
CPU time = 0 ms, elapsed time = 0 ms.
10 percent processed……<<<removed for blog>>
Processed 53425 pages for database ‘AdventureWorks2016’, file ‘AdventureWorks2016CTP3_Log’ on file 1.
100 percent processed.
BACKUP DATABASE successfully processed 93264 pages in 2.149 seconds (339.052 MB/sec).
SQL Server Execution Times: CPU time = 109 ms, elapsed time = 2373 ms.

As you can see, there is hardly any difference in time when it comes to working with backup compressions. Even for large backups, the difference is not significant that one can complain as a DBA.

On a completely different note, I wanted to play around and see what will be the effect when working with Azure based backup to the URL. I wanted to test the same database and check on performance. I used the following script:

On completion of the above script, the backup was taken to a blob as mentioned in the URL. I took the same performance metrics of time to check the difference. It is shown below:

SQL Server parse and compile time:
CPU time = 0 ms, elapsed time = 0 ms.
10 percent processed.
20 percent processed….
100 percent processed.
BACKUP DATABASE successfully processed 93266 pages in 3.809 seconds (191.293 MB/sec).
SQL Server Execution Times:
CPU time = 94 ms,  elapsed time = 4385 ms.

As you can see, in this case the time difference is significant for sure. This is fundamentally because as the backup is being taken, it must be streamed onto Azure in parallel. This increased network bandwidth usage is going to take a toll on the overall process. As you can see based on my small database, it is close to twice the time. In this case, I wanted to bring this to notice because as a DBA we need to make sure this has been considered when working with Backup to Azure.

I hope you enjoyed Performance Analysis. On a completely side note, I would love to learn if you are using the backup to Azure capability in your environments? How big are these databases? Do let me know via comments about your experience.

Reference: Pinal Dave (http://blog.sqlauthority.com)

First appeared on SQL SERVER – Performance Analysis of Backup to Azure

Question: When was Database Last Backed Up in SQL Server?

Answer: I had received this question doing my recent training on SQL Server Performance Tuning. Though this training is about performance tuning, I do enjoy answering pretty much all the questions for SQL Server.

Here is the script which will list all the different types of backup for your server.

-- Get Backup History for required database
SELECT TOP 100
s.database_name,
CASE s.[type] WHEN 'D' THEN 'Full'
WHEN 'I' THEN 'Differential'
WHEN 'L' THEN 'Transaction Log'
END AS BackupType,
m.physical_device_name,
CAST(CAST(s.backup_size / 1000000 AS INT) AS VARCHAR(14)) + ' ' + 'MB' AS bkSize,
CAST(DATEDIFF(second, s.backup_start_date,
s.backup_finish_date) AS VARCHAR(4)) + ' ' + 'Seconds' TimeTaken,
s.backup_start_date,
s.server_name,
s.recovery_model
FROM msdb.dbo.backupset s
INNER JOIN msdb.dbo.backupmediafamily m ON s.media_set_id = m.media_set_id
ORDER BY backup_start_date DESC, backup_finish_date
GO

When you run above query, it will display following resultset:

You can clearly see above query will display the backup name and type of the backup. One of the my favorite columns is time taken column to complete the backup as that also tells us lots about the entire backup process is set up.

Earlier I have written a blog post about backup timeline and it is one of the most popular blog posts so far. I encourage everyone to read this blog post.

A Must Read: SQL SERVER – Backup Timeline and Understanding of Database Restore Process in Full Recovery Model

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on When was Database Last Backed Up in SQL Server? – Interview Question of the Week #107