Disaster Recovery (Soft Recovery) - Exchange 2003

After an unexpected shutdown of the Exchange Server, the Information Stores in One Storage Group goes dismounted. Gave an error message while mounting.
Checked the state of the databases by eseutil /mh and there it shows dirty shutdown.

Here it can be that one of the database is not up-to-date and the server got rebooted when it was processing one of the transactions and got corrupted.

Our next step in this case is to identify the store that has been corrupted.

We will try to replay the logfiles on the each of the stores individually:

• Renamed priv2.edb and priv2.stb.
• Replayed the logfiles: eseutil /r /i E00
• Error Encountered. So log files cannot be replayed for Information Store 1 (priv1.edb, priv1.stm)
• Now Renamed priv1.edb and priv1.stm.
• Replayed the logfiles: eseutil/ r /i E00
• Completed Successfully. So the Information Store 2 is up to date now and we are able to mount it.

Now as the Information Store 1 is corrupted, the best practice is to restore from the last backup and then make the database up to date by recovering from the present log files.Eseutil /p should be the last resort as it performs the hard recovery and punches off the corrupted tables or entries from the database. So there is always a good chance of loss of some data.

As we have renamed priv1.edb and priv1.stm, the database restore from the backup will create new set of these files. We should not overwrite these files from the restore.

The Restore of backup (Veritas in this Case) should be set with the following options:

• Do not replay logs automatically
• Do not mount databases
• No Loss restore (don't overwrite or delete anything)
  

After the restore, we have the new set of edb and stm (a day old) and the current log files that will make them up to date.
Now we will use "ESEUTIL /CC" command to replay to restore.env file and continue on to production log files. There are temporary log files and restore.env file created when we run the restore. In case of Veritas restore, they are created in C:\temp\

Restore.env file is just a temporary environmental file holding path information about the data. Its purpose is to help the Exchange 2003 restore process find its files and match them with the corresponding email stores.

Now run the command eseutil /cc "path of restore.env"

Mount the store back after it completes successfully.

Calculate amount of disk space to be reclaimed by Offline Defrag...

By using the command eseutil /ms, we can calculate the exact amount of disk space that can reclaimed by doing the offline defrag.

Here are the steps:

1) Run eseutil /ms command from the bin directory on the dismounted database. It will always be run on the edb file.

2) The first section of the space report is the SLV SPACE DUMP. This reports on free space in the streaming database file (.stm).
TOTALS:
Free: 1528814
Reserved: 2238
Deleted: 228
Committed: 11888
Unknown: 0

There are 1528814 free pages in the .stm file, and each page is 4096 bytes in length. Therefore, there are 6,262,022,144 bytes of empty space in the file (1528814 x 4096).

3) As for the EDB file dump you'll see the following information in the last line in the column 'Available'
-------------------------------------------------------------------------------
3244272

The number at the lower right of the output (3244272) is the total of all free pages in the database. If you multiply this number by 4096, you will see that defragmenting this database will recover 12.3 gb of space.

4) Hence, based on the above example you will recover 12.3 GB + 6GB after you do the offline defrag.(this is including the STM also)

System Attendant Mailbox Missing: Users getting NDRs, Cant Move Mailboxes...

In this scenario, the users are getting NDRs that System Attendant cannot be reached... and when we try to move the mailbox to and from the server, the error says:
"The attempt to log on to the Microsoft Exchange Server computer has failed. The MAPI provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0512-00000000"

The application log has the following error messages:

Event ID : 9175
Raw Event ID : 9175
Record Nr. : 1167709
Category : MAPI Session
Source : MSExchangeAdmin
Type : Error
Message : The MAPI call 'OpenMsgStore' failed with the following error:
The attempt to log on to the Microsoft Exchange Server computer has failed.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0512-00000000

These symptoms indicate that the ‘mailbox store’ that holds the System Attendant mailbox is down or that the SA mailbox is disconnected, but actually all the mailbox stores are up and running on the server.
There is only one mailbox store in an Exchange server that holds SA mailbox... but you cannot find this mailbox on any of the stores.
And when you verify the attributes of System Attendant using ADSIEdit under Microsoft System Attendant, it is found that homeMDB was ‘Not Set’

Recommended Plan:
=================

• Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Information Store -, on the right side you’ll see a list of mailbox stores, click on any one of them and copy the value of ‘Distinguished Name’ attribute
• Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Microsoft System Attendant, right click here and find the attribute ‘homeMDB’, paste the value that you copied earlier (From ‘Distinsguished Name’) here
• Restart ‘Microsoft Exchange System Attendant’ service
• After the exchange services come up, browse to the store whose DN value you copied earlier and find the System Attendant mailbox under ‘Mailboxes’ list, if you don’t find it run a ‘Cleanup Agent’

If you don’t see the SA mailbox in the ‘Mailboxes’ list on the store even after performing ‘Run Cleanup Agent’. Using ADSIEdit, once you get to the properties of the ‘Microsoft System Attendant’ you can note the value of the the attribute ‘mail’. This is essentially the primary SMTP address of the SA mailbox. You can send an email to this address, that should initialize the mailbox and you should see it under ‘Mailboxes’ tab in ESM.