Sometimes we have an error in SQL Server job history which are not very accurate to find the cause of an issue. In this blog we would explore a situation where rebuild index job was failing with error 9002 – The transaction log for database ‘PinalDB’ is full due to ‘LOG_BACKUP’.
THE PROBLEM
In this situation, my client wanted to know the cause of rebuild index job failure. The complete error message in the job history was as follows:
Executing the query “ALTER INDEX [PK_auditID] ON [dbo].[tbl_audi…” failed with the following error: “The transaction log for database ‘PinalDB’ is full due to ‘LOG_BACKUP’.
The statement has been terminated.”. Possible failure reasons: Problems with the query, “ResultSet” property not set correctly, parameters not set correctly, or connection not established correctly.
THE ROOT CAUSE ANALYSIS APPROACH
If we look at the message it is clear that LDF file was full and that cause the rebuild index job to fail. My client informed that they are taking regular log backup so why LDF is full due to log backup reason? As a normal way to find the cause, I always ask to see SQL Server ERRORLOG and look at an interesting message at the same time when an issue was reported. If you are not familiar with SQL Server ERRORLOG then you must read my earlier blog on the same topic.
SQL SERVER – Where is ERRORLOG? Various Ways to Find ERRORLOG Location
In ERRORLOG, we could see below messages.
2018-09-06 05:00:07.36 Backup Log was backed up. Database: PINALDB, creation date(time): 2017/06/07(11:11:11), first LSN: 3384654:6547:1, last LSN: 3384654:6550:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {‘J:\Backup\PINALDB\PINALDB_backup_2018_09_06_050005_5978479.trn’}). This is an informational message only. No user action is required.
2018-09-06 05:51:10.57 spid160 Error: 9002, Severity: 17, State: 2.
2018-09-06 05:51:10.57 spid160 The transaction log for database ‘PINALDB’ is full due to ‘LOG_BACKUP’.
2018-09-06 06:00:01.76 spid16s Error: 9002, Severity: 17, State: 2.
2018-09-06 06:00:01.76 spid16s The transaction log for database ‘PINALDB’ is full due to ‘LOG_BACKUP’.
2018-09-06 06:00:01.76 spid16s Could not write a checkpoint record in database PINALDB because the log is out of space. Contact the database administrator to truncate the log or allocate more space to the database log files.
2018-09-06 06:00:01.77 spid16s Error: 5901, Severity: 16, State: 1.
2018-09-06 06:00:01.77 spid16s One or more recovery units belonging to database ‘PINALDB’ failed to generate a checkpoint. This is typically caused by lack of system resources such as disk or memory, or in some cases due to database corruption. Examine previous entries in the error log for more detailed information on this failure.
Last two error messages giving more information and confirming that we were running out of log space. Further, I looked into Event Log and found a breakthrough message there.
09/06/2018 05:51:39 AM Warning 2013 srv The M: disk is at or near capacity. You may need to delete some files.
The time is EXACTLY the same as a message in SQL ERRORLOG. I checked their database configuration using sys.database_files and found that M drive has only LDF file for this database.
Using the above information, I was able to provide Root Cause of the issue. Have you been into a situation where you have to look at multiple logs to find the root cause?
Reference: Pinal Dave (https://blog.SQLAuthority.com)
First appeared on SQL SERVER – Rebuild Index Job Failed – Error: 9002 – The Transaction Log for Database ‘PinalDB’ is Full Due to ‘LOG_BACKUP’