Out of Office Assistant not Working

The OOF message is a hidden entry in a user’s mailbox. The Out of Office Assistant creates a set of two rules in the Inbox subtree. The first contains a Message Class of IPM.Note.Rules.Oof.Template.MicrosoftMessage, whilst the other rule contains a Message Class of IPM.Note.Rules.OoFTemplate.Microsoft. If one or both of these rules are corrupted or unsynchronized with the OOF-enabled indicator, then it’s possible that the Out Of Office notification may no longer work.

To resolve this, use the Mdbvue32 (Microsoft Exchange Server Information Store Viewer) utility which can downloaded from here to delete the two entries for Out of Office.

Here are the instructions:

1. Run mdbvu32.exe.
2. Click OK to clear the first window that pops up.
3. Make sure that your profile is selected in the Choose Profile window and click OK.
4. Click on the MDB menu option.
5. Click on the OpenMessageStore option.
6. Make sure that "Mailbox- [user's full name]" is selected and click on Open.
7. Click on the MDB menu option again.
8. Click Open Root Folder.
9. In the Child Folders box - double click on "Top of Information Store".
10. In the next Child Folders box - double click on "Inbox".
11. Look in the Associated Messages in Fld box. All of your rules are in this box.
12. Examine each row of entries that appear under Associated Messages in Fld by double-clicking the CB items one at a time.

You are looking for two items that have the following message properties:

• The item that contains a message property Ox65EB that displays either OOF Rules or MSFT:TDX OOF Rules.
• The items that contain a message property PR_MESSAGE_CLASS that displays IPM.Note.Rules.OofTemplate.Microsoft.

13. Select both CB: values. To do so, press and hold down CTRL while you click both CB values under Associated Messages in Fld.
14. In the "Operations available (select operation, then push Call Function button)" text box, push the drop down button to reveal the list of functions.
15. Scroll down the list of functions until you see "lpFld -> deleteMessages() (ON SELECTED MSGS)" and then click on it to select it.
16. Next press the Call Function button. This will delete the rule that you selected.
17. Press the Close button to exit the MAPI_FOLDER window.
18. Press Close again to exit all the windows.

Reference: http://support.microsoft.com/kb/248709

Mail Delivery Slow: Messages Waiting to be Routed Queue Filled up

This issue stuck us 2 weeks back and we went all over by this. Suddenly we found out that the emails sent are being received after a lot of delay. Looking at the Exchange Server, I saw thousands of messages in Messages Waiting to be Routed Queue for Default SMTP Virtual Server and the number kept on increasing and the messages were going out very slow.

Slow mail delivery and mass queuing of mail in the Messages Waiting to be Routed queue in Exchange is typically caused by either Anti-Virus software on the Exchange server, by Distribution List expansion problems, or by connectivity problems between Exchange and Active Directory.

Antivirus was already ruled out as we tried disabling it from the registry as well but with no affect.

I opened up a case with MS PSS... turned up the diagnostics logging for MS Transport and MS DSAccess but nothing conclusive from the logs.

WE monitored the LDAP read and search times, SMTP categorizer queue length as it seemed to be the performace issue. Here is excellent MS guide for Troubleshoting Exchange Server Performance Issues.

Then we collected the Hang Dumps for Store.exe and Inetinfo.exe.

In the Store dump we see that we are waiting on WLAP calls to the GC’s.
From the Inetinfo dumps, we are waiting for the HrCheckRestrictions which means that the mail was probably to a DL that had restrictions placed on it.

So we figured out that Delviery Restrictions might be the cause of mail delivery being slow as we have applied the delivery restrictions on some Distribution Lists quite recently.

This problem occurs when lots of Lightweight Directory Access Protocol (LDAP) searches are initiated. Lots of LDAP searches are initiated when you send mail to distribution groups that include lots of users who have delivery restrictions configured on their mailboxes.

If you send a message to a group that contains many recipients, and if each of those recipients is also configured with a delivery restriction to reject messages from the members of a distribution group that contains many members, Exchange 2000/2003 Server must expand the restricted distribution group one time for each member of the group to which you sent the message. Also, if a failure that can be retried occurs during this process, Exchange Server stops the group expansion process, and then retries the connection an hour later. This causes the messages to be held in the categorizer queues, delays message processing, puts load on transports' Advanced Queuing and SMTP components, and eventually causes system queues to start backing up.

Here is the Excellent MS Exchange Team Blog for Performance issues due to connector restrictions.

Well, the solution to this above was the hotfix (already included in Exchange SP2) and change in one registry entry that defines the new Expansion Logic for restricted Distribution Group.

Here are the details:

1. Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeTransport
2. On the Edit menu, point to New, and then click Key
3. Type Parameters, and then press ENTER to name the new registry subkey.
4. Right-click Parameters, point to New, and then click DWORD Value.
5. Type RestrictionMethod, and then press ENTER to name the new registry entry.
6. Right-click RestrictionMethod, and then click Modify.
7. Type 2, and then press ENTER .

Reference: http://support.microsoft.com/default.aspx?scid=kb;EN-US;895407

Mailbox Move: Disconnected Mailbox remains on source Mailbox Store

I faced this issue while moving the mailbox from one mailbox store to a mailbox store on another server.
The mailbox is moved successfully and the user can access it fine on the server it was moved to.

However, the mailbox remains on the source mailbox store in a disconneced state and has small size. The number of items are shown as 0. The mailbox is disconnected and it does not allow you to purge or reconnect it to another user giving the message "This mailbox is already connected to a user."

To be able to purge the mailbox from the Target Store, try to move the mailbox back to that (original) Store. Move Mailbox will fail and gives you the message:

"A duplicate mailbox was found due to problems during a Move Mailbox procedure. The duplicate mailbox has been deleted. Try again later."

Hence when you run the cleanup agent on the original source mailbox store... that duplicate mailbox will be purged.

Reset the language for the folder names inside the Mailbox

In Exchange 2000/2003, when one of the following conditions takes place, the default language names of the folders inside the user's mailbox might change and names of standard folders in an Exchange mailbox, such as the Inbox, Sent Items, Public Folders, and so on, appear in different languages when seen from the client program:

1. You move a mailbox from one Exchange server to another Exchange server.
2. You create a new mailbox.

Exchange creates the folders based on the language of the Outlook client that first touches the mailbox, or the language of IE if OWA is first to touch the mailbox.

To reset the folder names, here is the registry tweak

In HKEY_CURRENT_USER\SOFTWARE\Microsoft\Office\11.0\Outlook\Setup, create a DWORD value ResetFolderNames and set its value to 1.

Here is more information about this from Microsoft.

Greylisting and Exchange

There is a potential issue between the Mail Servers that implement Greylisting and MS Exchange SMTP Servers.

Greylisting is used on some mail servers to tempfail first attempt of an email, asking the sending server to retry later. When Exchange tries to send mails to certain domains that implement ‘greylisting’, the mails fail to get delivered and an NDR is generated. Here is an example of what that NDR looks like:

"You do not have permission to send to this recipient. For assistance, contact your system administrator. server.domain.com #4.7.1 smtp;450 4.7.1 <recipient@greylistdomain.com>: Recipient address rejected: Greylisted"

The problem is that the sending Mail Servers are not delaying in response to a 450 "mailbox unavailable" response. The standard (RFC2821) specifies this as a transient condition and the sender should re-queue the message and resend it later. While it's reasonable to fail a message after receiving a number of these "transient failure" responses, the timeout before resending should be higher than 1 second - 10, 15 or 30 minutes are usual values.

By defaut, messages receiving a 4xx SMTP response are processed as a "glitch" 3 times before being put back into the queue for processing on the retry interval. So the problem is when the server resend the message 2 more times with a 1-second delay between attempts and then (presumably) fails delivery and notifies the sender that an error has occurred.

So as a workaround, we need to assertively set GlitchRetrySeconds to a value that allows the greylisting conditions to be satisfied, 120 seconds would do good in most of the cases.

How to Configure Glitch Retry Interval in Exchange Server 2003

Public Folders and Exchange 2007

Public folders are slowly being pushed out of Exchange into SharePoint 2007 and thus is no longer available in OWA from Exchange 2007.

Exchange 2007 client access server has some limitations in public folder support: no IMAP, NNTP, nor OWA access to Public Folders.

Only Public Folders that are stored on an Exchange 2003/2000 server can be accessed via a browser. There is a speculation that PF access via OWA 2007 will be made available when Exchange 2007 SP1 rolls out.
Also with the release of Exchange 2007 SP1, there will be the Public Folder Management Console for Public folders management. As of now public folder management can only be done through the command shell.

Exchange 2007 is surely de-emphasizing public folders. Public folders may not be included in future releases, but support for public folders will be maintained through at least 2016.

Here is what MS Exchange Team say about Exchange 2007 and Public Folders.

Restrict expanding a Distribution List for users

I have heard this question a lot... Here is way we can restrict expanding a distribution list so that the users can't see its members.

Use ADUC. Right-click the DL and click on "Exchange tasks...". From there, select "Hide membership".

This is useful when we have the Message restrictions applied on the properties page for the DL and it fails when the user expands the list to its members, then the message is sent again to everybody.

Exchange Management Shell

It may first appear as if the Exchange Management Shell is nothing more than a standard command prompt. But if you look at its prompt, you will notice that the letters "MSH" appear in brackets just ahead of the path.

The [MSH] tells you that you are not running in a true command prompt environment, but rather within a Microsoft Scripting Host shell. The Exchange Management Shell is nothing more than a Microsoft Scripting Host environment that has been extended to support Exchange Server commands.

There are many management tasks that are only performed on the Exchange Managament Shell and CANNOT be performed in Exchange Management Console. Some of them are:

• All Public Folder management.
• Give permissions to user's mailbox, entire database or the server.
• Advanced database, mailbox and recipient management.
• Advanced Transport Settings like setting a maximum message size limit for incoming and outgoing messages on the organization or connector, Set advanced SMTP connection settings etc.
• Certain Client Access settings also like set connection time-outs for POP3/IMAP4 servers (Set-IMAPSettings / Set-POPSettings), Prevent previous versions of Outlook from connecting to Exchange (Set-CASMailbox –MAPIBlockOutlookVersions), Enable/disable POP3 or IMAP4 for a user (Set-CASMailbox) etc...

Here is some collection of scripts for managing Exchange Server 2007 that are very useful for day to day operation and management.

Sender Policy Framework (SPF)

The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent sender address forgery. It is an attempt to control forged e-mail. SPF is not directly about stopping spam – junk email. It is about giving domain owners a way to say which mail sources are legitimate for their domain and which ones aren't.

The current version of SPF — called SPFv1 or SPF Classic — protects the envelope sender address, which is used for the delivery of messages.

Sender authentication protocols are designed to protect against forgery of e-mail sender identities, either in the envelope or in the header. In the envelope, first there is the "HELO" identity, which names the mail server (MTA) that is sending the message. The "MAIL FROM" identity is the e-mail address that is responsible for sending the message and where delivery errors (bounces) will eventually be reported. And the "RCPT TO" identity is the message's recipient address. The header contains another set of identities (besides other meta information about the message, such as the subject and the sending date).

SPF authenticates the envelope HELO and MAIL FROM identities by comparing the sending mail server's IP address to the list of authorized sending IP addresses published by the sender domain's owner in a "v=spf1" DNS record.

SPFv1 allows the owner of a domain to specify their mail sending policy, e.g. which mail servers they use to send mail from their domain. The technology requires two sides to play together:
(1) the domain owner publishes this information in an SPF record in the domain's DNS zone, and when someone else's mail server receives a message claiming to come from that domain, then
(2) the receiving server can check whether the message complies with the domain's stated policy. If, e.g., the message comes from an unknown server, it can be considered a fake.

Example of SPF Record

mydomain.com. TXT "v=spf1 mx a:machine1.mydomain.com include:gmail.com -all"

The parts of this SPF record mean the following:

v=spf1: SPF version 1
mx: the incoming mail servers (MX's) of the domain are authorized to also send mail for mydomain.com
a:machine1.mydomain.com: the machine machine1.mydomain.com is authorized, too
include:gmail.com: everything considered legitimate by gmail.com is legitimate for mydomain.com
-all: all other machines are not authorized

For detailed information on SPF records and their syntax, please refer www.openspf.org
The SPF Setup Wizard: http://old.openspf.org/wizard.html

Troubleshooting Fast Growing Transaction Logs

Here is a pretty good technet that details various causes:

http://technet.microsoft.com/en-us/library/aa996191.aspx

Also when we mount/dismount the stores manually, the transaction logs files are replayed on to the database thus they get committed and we are able to mount stores.

We can check this status by using eseutil: eseutil /mk "path to checkpoint file"

It will give you the transaction log file uptill which all the logs have been committed. This is helpful in the case we are running out of disk space and when the full backup is also a long way to finish… so that we can move out the committed logs and keep them on a different location. In case of recovery scenario, we need these log files to make database up-to-date.

Disaster Recovery (Soft Recovery) - Exchange 2003

After an unexpected shutdown of the Exchange Server, the Information Stores in One Storage Group goes dismounted. Gave an error message while mounting.
Checked the state of the databases by eseutil /mh and there it shows dirty shutdown.

Here it can be that one of the database is not up-to-date and the server got rebooted when it was processing one of the transactions and got corrupted.

Our next step in this case is to identify the store that has been corrupted.

We will try to replay the logfiles on the each of the stores individually:

• Renamed priv2.edb and priv2.stb.
• Replayed the logfiles: eseutil /r /i E00
• Error Encountered. So log files cannot be replayed for Information Store 1 (priv1.edb, priv1.stm)
• Now Renamed priv1.edb and priv1.stm.
• Replayed the logfiles: eseutil/ r /i E00
• Completed Successfully. So the Information Store 2 is up to date now and we are able to mount it.

Now as the Information Store 1 is corrupted, the best practice is to restore from the last backup and then make the database up to date by recovering from the present log files.Eseutil /p should be the last resort as it performs the hard recovery and punches off the corrupted tables or entries from the database. So there is always a good chance of loss of some data.

As we have renamed priv1.edb and priv1.stm, the database restore from the backup will create new set of these files. We should not overwrite these files from the restore.

The Restore of backup (Veritas in this Case) should be set with the following options:

• Do not replay logs automatically
• Do not mount databases
• No Loss restore (don't overwrite or delete anything)
  

After the restore, we have the new set of edb and stm (a day old) and the current log files that will make them up to date.
Now we will use "ESEUTIL /CC" command to replay to restore.env file and continue on to production log files. There are temporary log files and restore.env file created when we run the restore. In case of Veritas restore, they are created in C:\temp\

Restore.env file is just a temporary environmental file holding path information about the data. Its purpose is to help the Exchange 2003 restore process find its files and match them with the corresponding email stores.

Now run the command eseutil /cc "path of restore.env"

Mount the store back after it completes successfully.

Calculate amount of disk space to be reclaimed by Offline Defrag...

By using the command eseutil /ms, we can calculate the exact amount of disk space that can reclaimed by doing the offline defrag.

Here are the steps:

1) Run eseutil /ms command from the bin directory on the dismounted database. It will always be run on the edb file.

2) The first section of the space report is the SLV SPACE DUMP. This reports on free space in the streaming database file (.stm).
TOTALS:
Free: 1528814
Reserved: 2238
Deleted: 228
Committed: 11888
Unknown: 0

There are 1528814 free pages in the .stm file, and each page is 4096 bytes in length. Therefore, there are 6,262,022,144 bytes of empty space in the file (1528814 x 4096).

3) As for the EDB file dump you'll see the following information in the last line in the column 'Available'
-------------------------------------------------------------------------------
3244272

The number at the lower right of the output (3244272) is the total of all free pages in the database. If you multiply this number by 4096, you will see that defragmenting this database will recover 12.3 gb of space.

4) Hence, based on the above example you will recover 12.3 GB + 6GB after you do the offline defrag.(this is including the STM also)

System Attendant Mailbox Missing: Users getting NDRs, Cant Move Mailboxes...

In this scenario, the users are getting NDRs that System Attendant cannot be reached... and when we try to move the mailbox to and from the server, the error says:
"The attempt to log on to the Microsoft Exchange Server computer has failed. The MAPI provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0512-00000000"

The application log has the following error messages:

Event ID : 9175
Raw Event ID : 9175
Record Nr. : 1167709
Category : MAPI Session
Source : MSExchangeAdmin
Type : Error
Message : The MAPI call 'OpenMsgStore' failed with the following error:
The attempt to log on to the Microsoft Exchange Server computer has failed.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0512-00000000

These symptoms indicate that the ‘mailbox store’ that holds the System Attendant mailbox is down or that the SA mailbox is disconnected, but actually all the mailbox stores are up and running on the server.
There is only one mailbox store in an Exchange server that holds SA mailbox... but you cannot find this mailbox on any of the stores.
And when you verify the attributes of System Attendant using ADSIEdit under Microsoft System Attendant, it is found that homeMDB was ‘Not Set’

Recommended Plan:
=================

• Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Information Store -, on the right side you’ll see a list of mailbox stores, click on any one of them and copy the value of ‘Distinguished Name’ attribute
• Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Microsoft System Attendant, right click here and find the attribute ‘homeMDB’, paste the value that you copied earlier (From ‘Distinsguished Name’) here
• Restart ‘Microsoft Exchange System Attendant’ service
• After the exchange services come up, browse to the store whose DN value you copied earlier and find the System Attendant mailbox under ‘Mailboxes’ list, if you don’t find it run a ‘Cleanup Agent’

If you don’t see the SA mailbox in the ‘Mailboxes’ list on the store even after performing ‘Run Cleanup Agent’. Using ADSIEdit, once you get to the properties of the ‘Microsoft System Attendant’ you can note the value of the the attribute ‘mail’. This is essentially the primary SMTP address of the SA mailbox. You can send an email to this address, that should initialize the mailbox and you should see it under ‘Mailboxes’ tab in ESM.