Messaging @ Microsoft Exchange Server

This blog has been moved!

2010-08-09T08:42:00.000-07:00

This blog has been moved to www.roundtheclockit.com on wordpress platform. Its been a while since I hv posted on this as I have been very busy with few of the stuff. In the future posts, you will see more stuff on windows, citrix and virtualization apart from the messaging posts also and we promise to be regular :-).

Please visit www.roundtheclockit.com.

Message Redundancy - Hub Transport Servers | Shadow Redundancy vs Transport Dumpster

2010-03-20T01:22:00.000-07:00

As we discussed in the last article, High Availability is becoming very critical for every organization, message loss is one very critical and important aspect that we cannot ignore. By message loss, we mean here the message loss that can incur at the time of failure of one or more servers.

With introduction of Microsoft Exchange Server 2007 role based model, Hub Transport Server role provided the centralized transport pipeline through which all messages had to pass, thus making it possible and very efficient to set transport rules and policies.
For standalone non clustered mailbox servers, there is no built in protection as the message loss can occur if a server fails. Here on the Hub Transport Server, the message is stored in the transport database and is deleted as soon as it is sent to the next hop.

However, the feature called Transport Dumpster was introduced in Exchange Server 2007 that protect against message loss for the mailboxes that reside on a CCR cluster. Transport Dumpster holds the messages that are sent to the recipients whose mailbox resides on a clustered mailbox server, and recent sent items are retransmitted back in the event of failure of one cluster node so that messages are not lost during failover.

In Exchange 2010, there's a new feauture Shadow Redundancy that provides redundancy for messages for the entire time they are in transit. With Shadow Redundancy in exchange 2010, the deletion of a message from the transport databases is delayed until the transport server verifies that all of the next hops for that message have completed delivery. If a successful delivery is not reported, the hub server will try to resend the message.

Shadow redundancy uses the SMTP service extensions that allows the SMTP hosts to negotiate Shadow Redundancy support in Exchange 2010.

So the key points here are:

Transport Dumpster in Exchange 2007 only safeguards messages for the mailboxes that reside on CCR node, however Shadow Redundance is inbuilt in Exchange 2010 that can be enabled or disabled for the entire organization.
Shadow Redundancy makes hub transport servers more resilient against message loss. Exchange Server 2007 deletes messages from the database as soon as they were sent to the next hop, however, Shadow Redundancy keeps messages in the database until Exchange confirms that they were been delivered.
Earlier versions of Exchange were not designed to verify message delivery, however Microsoft has extended SMTP service in Exchange 2010 that allows this now.

Understand more indepth about Shadow Redundancy. Also, here's an excellent technet that explains detailed Shadow Redundancy Mail Flow Scenarios. This topic explains in detail what happens for each specific message flow scenario that can involve Exchange.

High Availability | From Exchange 2007 to Exchange 2010

2010-02-01T02:30:00.000-08:00

High Availability is one of the most important factors considered today in almost all the messaging deployments as email is becoming the mission critcal applcation and the backbone for all the businesses.

Exchange has come a long way from its earlier versions - Exchange 2003 that uses typical Windows Clustering technologies based on a shared storage model, and then to Exchange 2007 that brought new dimensions with the introduction of log replication technology in LCR, CCR and SCR. With this new technology of continuos replication where the transaction logs are shipped from one copy of a database to another, the exchange 2007 deployment offers high availability in
various scenarios like Local Continuous Replication (LCR) on a single server deployment, Cluster Continuous Replication (CCR) available accross different servers and Standy Continuous Replication (SCR) spread across different sites.

Exchange 2010, introduces the concept of Database Availability Groups (DAG) that takes the high availability to the next level for mailbox servers. A DAG is as the name suggests a group or a collection of mailbox servers (upto a maximum of 16) that uses the continuous replication technology that was first introduced in Exchange 2007 and are effectively a combination of Cluster Continuous Replication (CCR) and Standby Continuous Replication (SCR). It also makes use of
some of the components of Windows Failover Clustering to achieve high availbility and these cluster elements are installed automatically when a mailbox server is added to a DAG and managed completely by Exchange.

To achieve full high availibility solution for all the roles, with the introduction of Exchange Server 2007 SP1, it can be achieved by deplyoing a total of minimum of 4 servers - two servers installed as a single CCR environment, giving high availability for the users’ mailboxes and the other two servers deployed as combined Hub Transport and Client Access Servers, and configured as a load-balanced pair.

But with Exchange 2010, a full high availability solution can now be deployed by using a minimum of 2 servers as it’s now possible to combine the mailbox server role with other roles such as the Hub Transport and Client Access Server role. And with futher IO reductions in Exchange 2010 and RAID-less/JBOD support, it support much larger mailboxes with reduced storage costs.

To summarize Exchange 2010 high availibilty improvements:

Combines the capabilities of CCR and SCR into one platform.
Easier than traditional clustering to deploy and manage.
Allows each database to have up to 16 replicated copies.
Provides full redundancy of Exchange roles on two servers.
Further IO reductions.
RAID-less / JBOD support.

So exciting times ahead for the folks who are planning migration to Exchange 2010 :)

Troubleshooting Back Pressure Issues | Exchange Server 2007 Hub/Edge Transport Servers

2009-11-20T06:39:00.000-08:00

Back Pressure is an important resource monitoring feature of Microsoft Exchange Transport Service that exists on Hub/Edge Transport servers. The following system resources are monitored by this:

Free space on the hard disk drive that stores the message queue database.
Free space on the hard disk drive that stores the message queue database transaction logs.
Number of uncommitted message queue database transactions that exist in memory.
Memory that is used by the EdgeTransport.exe process.
Memory that is used by all processes.

All configuration options and threshold values for back pressure are available in the EdgeTransport.exe.config application configuration file that is located in the C:\Program Files\Microsoft\Exchange Server\Bin directory. You can even enable or disable this resource monitoring by changing the EnableResourceMonitoring parameter in EdgeTransport.exe.config file.

There might be situations and i also ran into one when you notice that the mail has stopped flowing from the Hub server and the queues are disabled. In the event logs, there is a warning message from MSExchangeTransport with event id 15004 which states:

Resource pressure increased from Normal to Medium.

Resource utilization of the following resources exceed the normal level:

Version buckets = XXX [Medium] [Normal=80 Medium=120 High=200]

Back pressure caused the following components to be disabled:

Inbound mail submission from the Internet

Mail submission from the Pickup directory

Mail submission from the Replay directory

Mail delivery to remote domains

Please note that all these threshold values for normal, medim and high are defined in the EdgeTransport.exe.config file. There can be number of reasons that can cause resource pressure to jump high and disable the queues... here are most expected reasons for this:

Running out of disk space or disk performance issues.
Running out of memory
Antivirus software running on Exchange Server with incorrect configuration
Virus issues.
A very large message being processed by Exchange Server.

The first 4 are are very visible and you can easily verify if one or the other is running out or spyking anything. In most of the cases, its the fifth one that is likely to hit. Once that large message gets in the submission queue, it is being processed by exchange server and this causes the problem.

There are various things that can prevent this type of scenario... for e.g. Outlook 2010 wouldn't even let you attach the attachment that is larger than the limits set on an exchange server, however if you are running earlier version of Outlook and Exchange Server is also not at current update level, this can happen.

Here are the few steps that you need to do for the quick resolution for this:

Find out the mailbox that has sent the large email.
Disable that mailbox.
Restart the Information Store service on the mailbox server.
Change the location of Queue Database path and log file path. It can be done from EdgeTransport.exe.config file by changing the value for "QueueDatabasePath" and "QueueDatabaseLoggingPath".
Restart the MSExchange Transport Service on the Hub Transport Server.
You might need to perform the recovery on the queue database to get the emails that are in uncommitted state on the previous queue database from the logs.

If you want to learn more on backup pressure, here's the technet for you.

Understanding Back Pressure

Exchange 2010! Released to Manufacturing

2009-10-09T01:26:00.000-07:00

Great news for everyone... described as the most advanced and the best messaging application by many - Exchange 2010 has been Released to Manufacturing... announced on the MS Exchange Team blog.

Update Rollup 9 for Microsoft Exchange Server SP1 Released

2009-07-18T10:05:00.000-07:00

Microsoft has released Update Rollup 9 for Microsoft Exchange Server 2007 Service Pack 1 (SP1).

This update rollup contains some imortant bug fixes.

KB 970162 details about the issues fixed in this release of update rollup.

Read more on MSExchange Team Blog.

Standby Continuous Replication - On Standalone Mailbox Servers - Part 3

2009-07-11T04:16:00.000-07:00

In Part 2, we discussed about the Preperation of the SCR Target server, in this Part 3, we will talk about SCR Activation for the Database Portability scenarios and in the end, we will discuss about SCR Activation using the Recover Server switch.

Database Portability is a new feature of Exchange 2007 and it allows us to mount the exchange database on any of the exchange servers in same organization. So in case of a database corruption say in SG1, we can manually perform the activation of the SCR target database.

Here are detailed steps for SCR Activation for the database portabilty scenarios:

1. Dismount the SCR Source database.

2. Now we need to disable the SCR and need to make the target database available for mounting.

This is done by the following command:

Restore-StorageGroupCopy SourceServerName\SG1 -StandbyMachine TargetServerName

Note: If the SCR source is not available, the Force parameter should be used with the Restore-StorageGroupCopy command.

3. After this has completed, we need to verify whether the database is in a Clean Shutdown state. If the database is in a Dirty Shutdown state, we can bring the database to a Clean Shutdown state by running Eseutil recovery mode (Eseutil /r) against the database.

4. Now after the database is in the Clean Shutdown state, we need to update the Active Directory with the new locations of the Storage Group and database files.

Here are the commands to change the paths for SG1Target and DBTarget from the temp paths to the paths for the SCR target's storage group and database files:

Move-StorageGroupPath TargetServerName\SG1Target -SystemFolderPath Path -LogFolderPath Path -ConfigurationOnly

Move-DatabasePath TargetServerName\SG1Target\DBTarget -EdbFilePath Path -ConfigurationOnly

5. Now we need to make the database to allow itslef to be overwritten during a restore operation. We can do this from its properties from the EMC.

6. Next, we can mount this database.

7. Ok now, the database is mounted, but we need to rehome the mailboxes to point to SCR Target Server. Please note that Microsoft Exchange System Attendant and System mailboxes should not be included in this.

Here is the command to do this:

Get-Mailbox -Database SourceServerName\SG1\DB I where {$_.ObjectClass -NotMatch '(SystemAttendantMailboxExOleDbSystemMailbox)'} I Move-Mailbox -ConfigurationOnly -TargetDatabase TargetServerName\SG1Target\DBTarget

So now this information has been replicated across the Active Directory, users can access the mailboxes now.

Please note if the Source server is not accessible, for clients running Outlook 2003 and earlier versions, we need to change their outlook profile to point to new target server.

Now, there might be scenarios where the whole source server is completely down and is not accessible pertaining to any hardware failure or something else, in these cases we can use Recover Server to activate the SCR target.

Here are the detailed steps for SCR Acttivation using Recover Server:

1. Disable the SCR and make the target databases available for mounting.
This is done by the following command:

Restore-StorageGroupCopy SourceServerName\SG1 -StandbyMachine TargetServerName -Force

2. Uninstall exchange server from the target server.

3. Rename the target server as the name of the source server if you have Outlook 2003 clients and want them to rehome automatically.

4. Run Setup /recoverserver. This should be run from the location where exchange binaries are located.

5. If the databases are in Dirty Shutdown state, use Eseutil to recover the databases.

Standby Continuous Replication - On Standalone Mailbox Servers - Part 2

2009-07-11T03:37:00.000-07:00

In the Part 1 of this article, we discussed about SCR and how to enable it. In this Part 2, we will focus on the second step i.e. Prepare the SCR Target Server.

We can have the target server prepared initially so that in case of a failure, the SCR activation can be done pretty quickly that will save us precious time.

For all the Storage Groups (from the source server) whose copy has been setup on the SCR target server, we will create the Storage Groups on the Target server and create the mailbox database on it. We can keep the path of the log files and the database on any temp location as we will mount this store, dismount it and remove that temp directory.

Steps and Commands:

As in the part 1 of this article, we have enabled the SCR for SG1 on the Source Server. Here we will create the Storage Group SG1Target and mailbox database DBTarget on the Target Server.

New-StorageGroup -Server TargetServerName -name SG1Target -LogFolderPath C:\SG1Target -SystemFolderPath C:\SG1Target
New-MailboxDatabase -StorageGroup TargetServerName\SG1Target -Name DBTarget -EdbFilePath C:\SG1Target\dbtarget.edb
Mount-Database DBTarget
Dismount-Database DBTarget
Delete everything from C:\SG1Target

Please note that these storage groups that we have set up on the target server are only temporary objects that will be used to carry out the database portability recovery during the SCR Activation Process. We can create them later on also but creating them now will save us the precious time while performing the recovery.

Standby Continuous Replication - On Standalone Mailbox Servers

2009-07-11T02:44:00.000-07:00

SCR - Standby Continuous Replication, the wonderful new feature introduced in Exchange Server 2007 SP1, introduces a new dimension in High Availabilty for Exchange Server 2007. SCR is designed for scenarios that use standby recovery servers and allows replicating a copy of Storage Group to a number of remote servers.

SCR uses the same log shipping and replay technology as local continuous replication (LCR) and cluster continuous replication (CCR).

It is very useful in the DR planning especially if we don’t want to go for full fledged clustering with automatic failover technologies like CCR because of various reasons like cost constraints of servers, OSs that are needed to have these types of setups.

SCR introduce redundancy in two scenarios for the Mailbox Servers:

One Storage Group or Multiple Storage Groups go corrupt - Use Database Portability to activate SCR target databases.
Full Server failure - Use Server Recovery to activate SCR target databases.

The starting point for SCR is called the source, which is any storage group, except a recovery storage group and the endpoint for SCR is called the target. SCR can support multiple targets per storage group. Although SCR can be combined with CCR and LCR, in this article we will focus and discuss its configuration on the standalone Mailbox Servers.

The whole process revolves around three activities:

Enable SCR
Prepare the SCR Target Server
SCR Activation

Enable SCR

Requirements:

The paths must be the same for both soource and the target like if source server is C:\Exchange\Data and C:\Exchange\Logs then these paths must be available on the target server.
There is a hard coded 50 log lag between the Source and Target, by default there is a 24 hour replay time which is configurable.
There can be only 1 database per storage group.

Command:

Enable-StorageGroupCopy SourceServerName\SG1 -StandbyMachine TargetServerName -ReplayLagTime 0.0:0:0

Parameters:

ReplayLagTime specifies how long the Replication Service waits before replaying replicated transaction logs to the replica Database (EDB) on the target. Default - 1 day

TruncationLagTime sets a lag time for truncating log files on that replica. Default - 0.

We can validate and check the status of the replication by running:

Get-storagegroupcopystatus -standbymachine TargetServerName

We will discuss the next two steps in the next parts of this article.

451 4.4.0 DNS query failed

2009-07-07T03:24:00.000-07:00

I was working on this annoying issue few days back, the mails sent to few domains are getting stuck in the queues on the Hub Transport server. The error message on the queue states 451 4.4.0 DNS query failed.

Well, it looked liked from here that there might be some issue with the DNS, ran few nslookup tests from the HUB servers itself, and surprisingly the domain and its MX records were resolving fine and i was able to connect to them absolutely fine.
More interestingly, if you hardcode the entry of the MX Record for that domain in the hosts file, the mails went through straight away.

I researched around few forum posts stated below, it seem like that this is a Bug with Exchange Server 2007 on Windows Server 2008, some issue related to IPv6.

Reference Posts:

http://social.technet.microsoft.com/Forums/en-US/exchangesvrtransport/thread/2e9a6263-7863-4d13-9480-7611db222e9e
http://social.technet.microsoft.com/Forums/en-US/exchangesvrtransport/thread/5ac614ee-e4fb-41d9-88f0-363f912634b1

Update Rollup 8 for Microsoft Exchange Server SP1 Released

2009-05-20T03:55:00.000-07:00

Microsoft has released Update Rollup 8 for Microsoft Exchange Server 2007 Service Pack 1 (SP1).

It contains the important bug fix which happened after Update Rollup 7 which caused the sender of the Delivery Status Notification to be unresolved.

KB 968012 details about the issues fixed in this release of update rollup.

Exchange Server 2007 SP2... Arriving Soon!

2009-05-13T02:09:00.000-07:00

Exchange Server 2007 SP2 is set to arrive in Q3 this year... it contains many new features but some great news on the backup front. It is providing the new backup VSS-based plugin that will enable users to take Exchange aware VSS backups from Windows Server 2008. So we will be able to backup and restore Exchange 2007 SP2 databases from Windows Server 2008 backup tool.

For more details and other features, please read this MSExchange Team blog.

Outlook Anywhere or RPC over HTTPS configuration from Script

2009-03-27T04:24:00.000-07:00

I was reading this interesting post from Oz Casey Dedeal about RPC over HTTPS Script. This prompted to me to write this post as i was working on this some time back and had another way of scripting RPC over HTTPS and Outlook Anywhere configuration.

This VB script will do the configuration for both Outlook 2003 and Outlook 2007. This script basically aims at changing the registry entries for your Outlook Profile that do all the outlook over https or outlook anywhere configuration.

Here are the Registry Keys and values that we are targetting at for doing this configuration in an Outlook Profile:

HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles\\13dbb0c8aa05101a9bb000aa002fc45a

"00036601" - for Cached Mode
"00036623" - Enable Exchange Proxy settings and control various check boxes like "on fast networks…" etc.
"00036627" - For Authentication - Basic or NTLM
"001f6622" - for specifying Exchange proxy server name
"001f6625" - for specifying Exchange proxy server name according to principal cert name.

So the script will go like:

Const HKEY_CURRENT_USER = &H80000001
strComputer = "."
Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & _
strComputer & "\root\default:StdRegProv")
strValue1Name = "00036601"
arrValue1 = Array(&H00,&H00,&H00,&H00)
strValue2Name = "00036623"
arrValue2 = Array(&H00,&H00,&H00,&H00)
strValue3Name = "00036627"
arrValue3 = Array(&H00,&H00,&H00,&H00)
strValue4Name = "001f6622"
arrValue4 = Array(&H00,&H00,&H00,&H00)
strValue5Name = "001f6625"
arrValue5 = Array(&H00,&H00,&H00,&H00)
strKeyPath = "Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles"
oReg.EnumKey HKEY_CURRENT_USER, strKeyPath, arrSubKeys

For Each subkey In arrSubKeys
strFullPath = strKeyPath & "\" & subkey & "\13dbb0c8aa05101a9bb000aa002fc45a"
oReg.SetBinaryValue HKEY_CURRENT_USER,strFullPath,strValue1Name,arrValue1
oReg.SetBinaryValue HKEY_CURRENT_USER,strFullPath,strValue2Name,arrValue2
oReg.SetBinaryValue HKEY_CURRENT_USER,strFullPath,strValue3Name,arrValue3
oReg.SetBinaryValue HKEY_CURRENT_USER,strFullPath,strValue4Name,arrValue4
oReg.SetBinaryValue HKEY_CURRENT_USER,strFullPath,strValue5Name,arrValue5
Next

Replace the value of 00 after &H in arrValue1, arrValue2, arrValue3, arrValue4 and arrValue5 with your values by looking into the registry under mentioned reg keys.

Update Rollup 7 for Microsoft Exchange Server SP1 Released

2009-03-18T22:45:00.000-07:00

Microsoft has released Update Rollup 7 for Microsoft Exchange Server 2007 Service Pack 1 (SP1).

This update has important fixes for SCR and IMAP4 issues.

KB 960384 details about the issues fixed in this release of update rollup.

Read more on MSExchange Team Blog.

Integration Of Exchange 2007 and OCS 2007

2009-03-07T10:50:00.000-08:00

In the last post, I discussed about the Microsoft's core solutions on Unified Communications - Exchange Server 2007 UM and OCS 2007. I had a chance to setup a test environment where they both operate together. So here we will talk about their intergration and take a look how it looks like.

To have both of them work together, you need to have Exchange Server 2007 SP1. Here are the key steps that you need to perform to configure them:

Configure the new dial plan. To do this, run this command from the powershell:
New-umdialplan -name OCSPlan -Uritype "Sipname" -Voipsecurity "SipSecured" -Numberofdigitsinextension 4

Go to properties of this dial plan and set the subscriber access number. Subscriber access is used by users to access their individual mailboxes to retrieve e-mail, voice messages, contacts, and calendaring information. Outlook Voice Access is the new Exchange 2007 Unified Messaging feature that lets subscribers access their Exchange 2007 mailbox.

Apply this dial plan to the UM server:
set-umserver -id UM_Server_Name -dialplans OCSPlan

Run Powershell script "EXCHUCUTIL.PS1" from Exchange Management Shell. This is installed when you apply SP1 for Exchange 2007. To run this successfully you might need to modify the properties of the Start Menu shortcut for the Exchange Management Shell so that it launches directly from the Scripts folder "Program files\Microsoft\Exchange Server\Scripts".

On the OCS server, run OCSumutil with the following command line: ocsumutil /domain:yourdomain.com

Now you can configure your users for Unified Messaging. You can do this from Exchange Management Console and right click on the user to Enable Unified Messaging and select the dial plan.

So now when you log on with this user to Communicator 2007, you will see the additional call menu from where you can forward your calls or redirect incoming calls to voicemail.

And now you get a Missed Conversation notification when someone IMs you and you are offline or away:

Also, You get the missed call notifications or voice mails when someone calls you via OCS client:

The world of Unified Communications

2009-02-15T20:17:00.000-08:00

Over the recent times, this has been one of the hottest topic of discussion in various organizations and among various professionals.

Microsoft and third-party vendors such as Cisco Systems, Avaya, Nortel, and Polycom offer products that support Unified Communications (UC), and have their own definitions for UC.

If we talk about Microsoft here, Microsoft offers two integrated solutions: Microsoft Exchange Server 2007 and Office Communications Server 2007 that are the key players in Microsoft's UC strategy. Together they bridge the gap between computers and telephones. They integrate with your company's exisiting phone system and deliver complete communication services using your existing data network.

So in the world of Unified Communications, Microsoft offers Exchange as the Unified Messaging component, handling voicemail, fax, and telephone access to messaging. OCS is the component that offers IM, conferencing, presence, and voice services.

Microsoft Exchange Server 2007 Unfied Messaging

Here are the features that Unified Messaging Role provides in Exchange Server 2007:

Outlook Voice Access (OVA): Outlook Voice Access allows a phone to be a Unified Messaging client. A text-to-speech engine is built into the UM server to allow email message to be read from a users mailbox. A UM-enabled user can call into their company's UM server and can access Voicemail, Email, Calendar, Contacts, OOO messages and security and personalization settings.

Call Answering: Say if a user receives any external call, this call will be routed via IP/PBX or a VoIP gateway, if the destination user is not available, this call will be routed to Unified Messaging Server. The external user will be given an option to leave a voice mail and it will be delivered to the mailbox server, else the user will be notified of the missed call.

Auto Attendant: The Auto Attendant in the Exchange Unified Messaging Server allows an organization to create customized menus, custom greetings, searching directory etc.

Fax Receiving: The UM server provides the ability to receive faxes in .tiff format via IP/PBX or VoIP gateway.

Microsoft Office Communications Server 2007

Office Communications Server 2007 manages all real-time (synchronous) communications including: instant messaging, VoIP, audio and video conferencing. It works with existing tele-communications systems, so business can deploy advanced VoIP and conferencing without tearing out their legacy phone networks. Microsoft Office Communications Server 2007 also powers Presence. With presence, you can see at-a-glance if someone is available and contact them with a click using instant messaging, a phone call or a video.

Microsoft unified communications technologies delivers voice and video conferencing through Microsoft Office Communications Server 2007 via IP. This means that conference call audio and video streams are captured and logged just like an instant message or an email.

The complete conferencing solution requires:

Office Communications Server 2007
Microsoft Office Communicator 2007
Microsoft Office Live Meeting service (2007)
Video-equipped computers and/or VoIP phones

Integration of Exchange 2007 and OCS 2007

Microsoft Exchange Server 2007 Unified Messaging and Microsoft Office Communications Server 2007 can be deployed together to provide voice messaging, Instant Messaging (IM), enhanced user presence, audio-video conferencing, and an integrated e-mail and messaging experience for users in your organization.

Here are the key features and benefits that they will provide together:

You get the missed call notifications or voice mails when someone calls you via OCS client.
You get a notification when someone IMs you and you are offline or away.
OCS automatically uses the out-of-office (OOF) message text you set in Exchange.
Presence information displayed in Microsoft Outlook 2003/2007 and in Windows Sharepoints Services team sites and document libraries.

So in a nutshell, you are Connected... Always!

Update Rollup 6 for Microsoft Exchange Server SP1 Released

2009-02-10T20:39:00.000-08:00

Microsoft has released Update Rollup 6 for Microsoft Exchange Server 2007 Service Pack 1 (SP1).

This update has a fix for the critical security issue and a fix to allow IE 8 to be used for OWA 2007. This does not include OWA 2007 S/MIME control.

KB 959241 details about the issues fixed in this release of update rollup.

Read more on MSExchange Team Blog.

Some mailboxes show up as Shared or Linked... after Migrating to Exchange Server 2007

2009-01-25T03:29:00.000-08:00

Here's a scenario where after migrating users from Exchange Server 2003 to Exchange Server 2007, some of the user mailboxes show as "Shared" Mailbox or "Linked" Mailbox within the Exchange Management Console... instead of showing them as "User" Mailbox.

A Linked mailbox is a mailbox that is associated with an external account. Thus Linked mailboxes are normal mailboxes that can hold an External account in a separate resource forest.

Few of these mailboxes here were shown as Linked mailboxes after migration because an orphan Active Directory Account has the "Associated External Account" right on the mailbox in question.

To fix this and have them show up as normal "User" mailboxes remove that orphan Active Directory account that has the "Associated External Account" right on the mailbox in question and run the following command from the Exchange Management Shell:

set-mailbox -id MAILBOX_NAME -ApplyMandatoryProperties

Now we come to why few mailboxes are shown up here as Shared mailboxes. If you have Self permission with associated external account checked, you get the shared mailbox on migration. That is the reason these mailboxes are showed up as shared.

To fix this, remove the "Associated External Account" permission from the Self and again run that command:

set-mailbox -id MAILBOX_NAME -ApplyMandatoryProperties

To read and understand more about the ApplyMandatoryProperties Parameter, read this excellent blog entry from Evan Dodds.

Exchange Server Remote Connectivity Analyzer

2009-01-13T01:43:00.000-08:00

This looks pretty exciting... in beta though but worth keeping an eye on as it can be very useful for exchange admins... especially the amount of troubleshooting we need to do... :-)

https://www.testexchangeconnectivity.com/

Entourage for EWS

2009-01-12T05:13:00.000-08:00

Entourage for Exchange Web Services is targeted at organizations using Exchange Server 2007. This means that we can finaly say goodbye to WebDav and hopefully to all the issues that we face as EWS is a stronger and more robust protocol than WebDAV, that enables client applications to communicate with the server running Microsoft Exchange Server 2007 SP1 or later – for better compatibility, performance and reliability.

Entourage EWS will launch a public beta this month. The final release will be available later this year as a free update to Office 2008 users.

More Details: http://www.microsoft.com/mac/itpros/entourage-ews.mspx

Configuring Autodiscover for Exchange 2007

2008-12-29T23:34:00.000-08:00

The Autodiscover service provides a mechanism to automatically configure Exchange 2007 client applications to access the Client Access server.

It is used to set up client applications like Outlook to work with Exchange by returning configuration data that is necessary for applications to function properly without requiring users to know where to fetch the data, such as discovery of the server that hosts a user's mailbox.

Configuring Autodiscover for External Access

Outlook 2007 tries the following two URLs to try to connect to the Autodiscover service:

https://mydomain.com/autodiscover/autodiscover.xml

https://autodiscover.mydomain.com/autodiscover/autodiscover.xml

Step 1: Configuring DNS

We will discuss the two simple and most commonly used scenarios here:

1. Using a SSL Certificate That Supports Multiple DNS Names
There are third-party Certification Authorities (CAs) that currently support Subject Alternative Names. In this you provide all the necessary DNS names like mail.mydomain.com, autodiscover.mydomain.com etc. in the same certificate by using a Unified Communications certificate that supports the Subject Alternative Name field.

For the Autodiscover service to function correctly, you must add an additional host record (autodiscover.mydomain.com) so that Outlook 2007 clients can locate and connect to the Autodiscover Service when they use the Outlook Anywhere feature from the Internet. The host record you create should map to the Public IP Address that will be used as the entry point to your Client Access server.

2. Using One Single-Name Certificate and the Autodiscover SRV Record
This solution is to use one single-name certificate installed on the Default Web Site.
If your DNS provider supports SRV records, this solution is the simplest and least expensive way to deploy Outlook Anywhere in hosted and non-hosted Exchange 2007 environments.

Create an SRV record:
Service: _autodiscoverProtocol: _tcpPort Number: 443 Host: mail.mydomain.com

For more information on Outlook 2007 and Autodiscover SRV records configuration, please read this knowledgebase article: http://support.microsoft.com/?kbid=940881

Step 2: Modify the Service Connection Point (SCP)

By default, the URL for the Autodiscover Service stored in the SCP object in Active Directory will reference the internal FQDN for the Client Access server during Exchange 2007 Setup. You will use the Set-ClientAccessServer cmdlet to modify this URL so that it points to the new location (FQDN) for the Autodiscover service.

In the Exchange Management Shell, run the following command:

Set-ClientAccessServer -identity CAS_servername -AutodiscoverServiceInternalUri https://mail.mydomain.com/autodiscover/autodiscover.xml

You can point this to https://autodiscover.mydomain.com/autodiscover/autodiscover.xml if you are using DNS host record for autodiscover.mydomain.com.

Step 3: Configuring Exchange Services for Autodiscover

1. Enable Outlook Anywhere for the external host name.

Enable-OutlookAnywhere -Server CAS_servername -ExternalHostname "mail.mydomain.com" -ExternalAuthenticationMethod "Basic" -SSLOffloading:$False

2. Configure the external URL for offline address book for the Autodiscover service.

Set-OABVirtualDirectory -identity "CAS_servername\OAB (Default Web Site)" -externalurl https://mail.mydomain.com/OAB -RequireSSL:$true

3. Configure the external URL for Unified Messaging for the Autodiscover service.

Set-UMVirtualDirectory -identity "CAS_servername\UnifiedMessaging (Default Web Site)" -externalurl https://mail.mydomain.com/UnifiedMessaging/Service.asmx -BasicAuthentication:$True

4. Configure the external URL for Exchange Web Services for the Availability service and Out of Office services.

Set-WebServicesVirtualDirectory -identity "CAS_servername\EWS (Default Web Site)" -externalurl https://mail.mydomain.com/EWS/Exchange.asmx -BasicAuthentication:$True

Update Rollup 5 for Exchange 2007 SP1

2008-11-25T09:56:00.000-08:00

The Update Rollup 5 for Exchange 2007 SP1 has been released and is available at Microsoft Download Center.
So would recommend that you should plan for its install as this has fixed quite a lot of issues.
More details at MS Exchange Team Blog.

Information Store Crashing, Exchange 2007

2008-11-12T07:49:00.000-08:00

The Microsoft Information Store Service (Store.exe) is repeatedly crashing an Exchange 2007 Mailbox Server.
In the application log, we get the following error message and store.exe is crashing due to EXCDO.DLL file:

Faulting application store.exe, version XXXX, time stamp XXXX, faulting module EXCDO.DLL, version XXXX, time stamp XXXX, exception code XXXX, fault offset 0x00000000001ed390, process id 0x1fc8, application start time XXXX.

Have opened up a case with MS PSS and had them analyze the crash dumps. They have identified it as a Bug. The issue is casued by Mac's, Entourage clients that does calendar bookings in the ICS format and the crash is occuring due to Entourage Client accessing a badly created ICS.
MS will provide a fix to this issue in Update Rollup 6.

If anyone is experiencing this issue, make sure you call MS PSS first thing and have them provide the temp bug fix to you.

Web Services, Exchange 2007

2008-11-01T11:59:00.000-07:00

Now that we have finished the final phase of our migration and we have fully migrated the Public Folders to the new Exchange 2007 environment, want to talk about this exciting new component of Exchange 2007, that is the Web Services. Well I think this is the most exciting and amazing new feature in Exchange 2007 that makes life very easy of an Exchange Admin.

The Web Services that are included in Microsoft Exchange Server 2007 provide an XML messaging interface for managing Exchange store items and accessing functionality on a computer that is running Exchange 2007 from client applications.

Exchange 2007 includes the following Web services:

Exchange Web Services
Autodiscover Service
Unified Messaging Web Service

1. Exchange Web Services

Exchange Web Services provides the functionality to enable client applications to communicate with the Exchange server. Exchange Web Services is deployed with the Client Access server (CAS) role. Microsoft Exchange Server 2007 clients connect to the computer that is running Exchange 2007 that has the Client Access server role installed in an Active Directory directory service site by using an HTTPS connection.

Some of the main operations that Exchange Web Services provide are:

The Availability Service: Outlook 2007 clients can use HTTPS to connect and download free and busy data for other users through the Availability Service. Unlike previous versions of Exchange, free/busy data does not have to be stored in public folders, instead we access the target mailbox’s free/busy data directly from the calendar (via the Availability service).
OOF: The ability to set Out-Of-Office messages is controlled by a new web service.
OAB Distribution: Outlook can use HTTPS to locate and download the Offline Address Book (OAB) from a web distribution point.
The Exchange Data Service operations: The Exchange Data Service operations enable the handling and organizing of items, folders, and attachments, as well as ambiguous name resolution and distribution list expansion.
Notification and Synchronization Operations.

2. Autodiscover Service

The Autodiscover service provides a mechanism to automatically configure Exchange 2007 client applications to access the Client Access server.

3. Unified Messaging Web Service

The Unified Messaging Web Service provides an extensibility point for clients to read and change information about Unified Messaging properties. If your mailbox is enabled to use Exchange 2007 Unified Messaging, you can use Outlook 2007 to play a voice message back to a phone.

Will discuss about their configuration in detail in the coming posts :-)

Get HUB server stats from Message Tracking

2008-10-22T23:47:00.000-07:00

Now in Exchange 2007, from Command Shell using Message Tracking, you can pull out some of the very exciting statistics which were never so easy from the previous versions of exchange.

Here we are talking about pulling up records like:

Number of emails sent by HUB servers over a given period of time.
Number of emails received by HUB servers over a given period of time.
Number of emails sent by a particular sender or received by a particular recipient over a given period time

Example: To get the number of emails sent by a HUB server, run this command from EMS:

Get-MessageTrackinglog -server -start "MM/DD/YYYY hh:mm am/pm" -End "MM/DD/YYYY hh:mm /pm" -event SEND -ResultSize 999999 I measure-object

Similarly you can add the switches like -event RECEIVE to get the number of message recieved, -Sender "Email address" to get from a particular sender etc.

Read this article from Bharat Suneja to get more details on Message Tracking from Exchange Server 2007

Playing with RSG | Database Restore

2008-09-09T02:22:00.000-07:00

This came across few days back when one of our production Exchange 2003 backend server went down due to hardware failure and was totaly dead.The Backup team was shortly in action to carry out the restore. The restore was done on the different server in a Recovery Storage Group (RSG).

Now the question is how to merge these mailboxes in RSG with the production ones... certainly direct merge was not possible as we didnt opt for Dialtone method as the production server was totaly dead and we didnt set up a new server to replace that one.

The one option was to to extract the mailboxes from RSG using Exmerge, delete the existing mailboxes of users and create new ones on another server, and then import that extracted mailboxes. But this is very time consuming.
However here's an another option to tweak the RSG to make it behave as the production store and move that mailboxes over.

The Recovery Storage Group is identified by the msExchRestore Attribute. The msExchRestore = TRUE property tells us if a database is a recovery database.
As such we cannot directly connect the mailbox from the RSG to user direcly. We have to modify this msExchRestore Attribute and make its value to Not Set so that it cannot be identified as RSG and we can connect the mailbox to the user.

The detailed steps are as follows:

1. Reset the MsExchRestore attribute on both the Recovery Storage Group, and the recovered mailbox store using Adsiedit:

Expand the Configuration Container node, and then browse the hierarchy to:
DC=Domain
DC=com
CN=Configuration
CN=Services
CN=Microsoft Exchange
CN=Administrative Groups
CN=Administrative Group Name
CN=Servers
CN=ExchangeServerName
CN=InformationStore
CN=Recovery Storage Group

Right click for Properties on the Recovery Storage Group, ensure that the checkbox for "Show Optional Attributes" is checked. Scroll down to the "MsExchRestore" attribute. Double-click the MsExchRestore attribute and check the Not Set parameter. Click OK, Apply, and OK to exit the properties pane.
Right click the "CN=MailBox Store(ServerName)" under the Recovery Storage Group, and select Properties. Ensure that the checkbox for "Show Optional Attributes" is checked. Scroll down to the "MsExchRestore" attribute. Double-click the MsExchRestore attribute and check the parameter. Click OK, Apply, and OK to exit the properties pane.
Close the AdsiEdit mmc

2. Now from the ADUC>Exchange Tasks, delete the mailboxes of the users that are in question.

3. In Exchange System Manager select the Recovery Storage Group, right click and select Refresh. Expand the Recovery Storage Group, then the mailbox store.

4. Select "Mailboxes" under the Mailbox Store, then right click the mailbox to be recovered, and select "Reconnect". The applet "Select a New User for this Mailbox" applet will appear. Enter the alias of the user you wish to associate with the recovered mailbox into the "Enter the Object Name to select" data entry field, the click "Check Name". The alias of the user should be resolved to the full display name. Click OK. You will see a pop-up stating "The operation has completed successfully".

5. Once its done for all the users, you can successfully move the mailboxes over to any of the servers.

IMAP not working for users | Mailbox server in different AD site than CAS

2008-08-27T00:54:00.000-07:00

Here we have a cross site scenario where users from one site are trying to connect to the CAS that is on the different site.

By default, POP3 and IMAP4 connectivity between a Client Access server in one Active Directory site and a Mailbox server in another Active Directory site is not enabled. The Client Access server prevents users from logging on to their mailboxes by using POP3 or IMAP4 if their mailboxes are located on a Mailbox server in a different Active Directory site.

And we get the following warning in the application log of the Client Access Server:

User "User-Name" was prevented connecting to his/her mailbox because the Mailbox Server resides in a different ActiveDirectory site (CN=Site A,CN=Sites,CN=Configuration,DC=domain,DC=com) than the Client Access Server (CN=Site B,CN=Sites,CN=Configuration,DC=domain,DC=com) to which he/she connected. Either arrange for the Client Access Server and the Mailbox Server to reside in the same site, or change the configuration setting of AllowCrossSiteSessions to true.

To enable cross-site connectivity for POP3 and IMAP4 clients:

On the Client Access server, locate the POP3 and IMAP4 configuration files. By default, the location is C:\Program Files\Microsoft\Exchange Server\ClientAccess\PopImap.
Use the text editor to open the configuration files. The two configuration files are as follows:
Microsoft.Exchange.Pop3.exe.Config
Microsoft.Exchange.Imap4.exe.Config
Search for the key AllowCrossSiteSessions. add key="AllowCrossSiteSessions" value="false" /
Edit the line of the configuration file as follows. add key="AllowCrossSiteSessions" value="true" /
Repeat the steps in this procedure for each protocol for which you want to enable cross-site connectivity.
Restart the services for each configuration file that you have updated to enable cross-site connectivity.

More Details at Microsoft Technet.

500 Internal Server Error when using OWA with /Exchange

2008-07-31T01:30:00.000-07:00

You get this 500 - Internal Server error while you access Outlook Web Access from https://CASServer.domainname.com/exchange while https://CASServer.domainname.com/owa works fine.
This happens when you have seperate Exchange 2007 Mailbox and CAS servers. Ideally the request should be redirecetd to /owa but you get 500 - Internal Server Error right after typing in your credentials in the forms login page.

This happens due to the fact that redirection is not working because ISAPI Extensions are not installed on the Mailbox Server. ISAPI extensions handle specific incoming requests to the IIS server. Extensions are loaded when they are first needed and kept in memory until the host process shuts down.

To fix this issue, please install the ISAPI Extensions on the mailbox server.

Here is the command that you have to run from the EMS to install them:

ServerManagerCmd -i Web-ISAPI-Ext

Make sure to do an IISRESET after this.

Storage Group not responding | Version Store Issues

2008-07-21T23:37:00.000-07:00

This happend few days earlier on our exchange server... all users in a particular Storage Group were not able to connect to Outlook/OWA. The mails queued up for them on the server...
On looking at the server logs... there were lot of logon error messages... users were not able to logon to their mailbox... and at the beginning when it all started, there was one error message with Event Id 623. It says:

Information Store (2984) Storage Group 1: The version store for this instance (3) has reached its maximum size of 155MB. It is likely that a long running transaction is preventing cleanup of the version store and causing it to build up in size. Updates will be rejected until long-running transaction has been completely committed or rolled back.

Analysis

The Version Store keeps an in-memory list of modifications made to the database. It gives ESE the ability to track and manage the current transactions. Thus the Version Store is where transactions are held in memory until they can be written to disk.

Event 623 is the result, typically of a long running transaction. The result of this long running transaction is to exhaust resources in the version store. As a result, the Version Store no longer reaps deleted records causing unneeded data, which is marked as deleted, to accumulate in the database. The accumulation of unneeded data can exacerbate performance problems which can lead to event id 623. No more transaction can continue until this is clear.

Thus we will see 623 event indicating that the maximum Version Store size has been reached . All the Write operations to the database will fail because there's no more version store space to record the operation.

Why this happens

This can happen for one of the two reasons:

1) In order to properly reconcile write-conflicts and properly support repeatable reads, a given entry in the version store cannot be cleaned up until it is older that the oldest active transaction.
2) Version Store cleanup simply cant keep up with the load on the machine.

Possible Causes

- Online Maintenance Tasks running at peak times.
- Backups running at peak times.
- Disk I/O performance.
- Large Mails

Any of these can add up simultaneously and add to the performance degrade of the server.

More details and troubleshooting on this.

Useful Exchange Management Shell Cmdlets

2008-07-14T23:44:00.000-07:00

Convert a legacy Exchange 2003 "resource" into a room mailbox in Exchange 2007

Move the mailbox to an Exchange 2007 server
Run: set-mailbox [MailboxName] –type Room
Then run: set-mailboxcalendarsettings [MailboxName] –AutomateProcessing AutoAccept

Configure the conference room resource mailboxes to remove attached files from meeting requests

Get-Mailbox - RecipientTypeDetails RoomMailbox Set-MailboxCalendarSettings -DeleteAttachments:$true

Get a list of Exchange ActiveSync users

$mbx = get-casmailbox where {$_.hasactivesyncdevicepartnership -eq $true -and $_.identity -notlike "*CAS_{*"} ; $mbx foreach {$name = $_.name; $device = get-activesync devicestatistics -mailbox $_.identity; $device foreach {write-host $mbx.name, $_.devicemodel, $_.devicephonenumber, $_.deviceid, $_.FirstSyncTime, $_.LastSuccessSync} }

Get details of single EAS mailbox

Get-ActivesyncDeviceStatistics -mailbox user@domain.com

An inappropriate message that has the subject line of exchange is sent to all users on the Exchange server 2007 mailbox server named Exch1. You need to delete this message from all the mailboxes.

Create a new mailbox named TempMailbox that has a folder named Export. Run the Get-Mailbox -Server Exch1 Export-Mailbox -TargetMailbox TempMailbox -TargetFolder Export -SubjectKeywords "Exchange" -DeleteContent cmdlet.

Get list of all user mailboxes with a list of email addresses attached to each user.

get-mailbox select-object name, alias, primarySmtpAddress Export-Csv C:\mailboxes.csv -NoTypeInformation

Generic User Accounts | Exchange 2007 Shared Mailboxes

2008-06-14T03:51:00.000-07:00

In an organization, there are mailboxes required that are shared by many individuals in a particular department... I will say them as generic accounts that a group of people use for common mail access. However, the Information Security team may take it as offensive as they have an active mailbox enabled user account... and if you decide to close on them, it becomes really difficult as users are used to them as it helps them to organize and manage their tasks efficiently... and instead creating the Distribution List with the same email address won't solve the purpose.

In Exchange 2007, we have concept of shared mailbox recipient type. When we create a mailbox as 'shared' it creates a disabled active directory account to which the mailbox is connected. Shared mailboxes do not have an associated password so we must grant mailbox permissions for the users requiring access to that mailbox.

Advantages

Disabled accounts act as a security measure.
Since the user account is disabled by default no initial password is required.
If desired, existing mailboxes can be converted to shared mailboxes. Users can still be able to access emails and continue to receive emails on that mailbox.
Will let the users to continue their operations more or less the same way.
No need to setup Distributions Lists on account of closure of their respective generic accounts.

Public Folder database dismounted after you move all the Public Folder Replicas from Exchange 2003 server to Exchange 2007 server

2008-04-08T00:11:00.000-07:00

Issue with Public Folders

It has been observered that when we decommision the Exchange 2003 server after moving all the Public Folders Replicas from it to the Exchange 2007 Server, the Outlook stops connecting for all the clients... Public folders doesn't show anything from Outlook Web Access and even Public Folder Management Console on Exchange 2007 server doesnt show any of the Public Folders and the Public Folder database doesnt mount.

This typically happens when we manually remove the Exchange 2003 server hosting earlier all the Public Folder replicas.

This happens because the msExchOwningPFTree attribute had its value missing. This msExchOwningPFTree attribute controls the Public Folder database path and contains a list of all the stores in the hierarchy.

We need to specify its correct value in order to make Public Folders work. We can do it from ADSI Edit tool.

Start the ADSI Edit tool
Expand the Configuration container, then expand out each container as follows: CN=Services then CN=Microsoft Exchange then CN=YourExchangeOrg then CN=Administrative Groups then CN=AdminGroupofyourPFServer then CN=Servers then CN=YourPFServer then CN=InformationStore and then click on CN=StorageGroupContainingYourPFStore. In the pane on the right you see the public store object listed. Right click on the object and click on properties.
In the list of attributes, double click on the msExchOwningPFTree attribute and specify the correct value.
Restart Microsoft Exchange Information Store service.

Here is the Microsoft KB article that will help you to determine the correct value of msExchOwningPFTree attribute and set that up.

Find Users not using Default Exchange Storage Limit set via Mailbox Policy

2008-04-01T04:28:00.000-07:00

Here's an LDAP query that I use frequently to search and display all the users in the domain that do not have the default mailbox store policy set.

(&((mailNickname=*)(mDBUseDefaults=FALSE)))

Windows update available that will turn off the SNP

2008-03-19T00:33:00.000-07:00

There is a new High Priority Windows update available that will turn off the Scalable Networking Pack features which are installed by default in Windows 2003 SP 2.
http://msexchangeteam.com/archive/2008/03/12/448421.aspx
You have to reboot the server after applying this update.

SMTP Connectors | Exchange 2007

2008-02-12T21:29:00.000-08:00

Microsoft has split connectors in Exchange 2007 into Receive Connectors and Send Connectors.

This transition to a separate send/receive configuration scheme makes it a lot easier to tell Exchange how you want it to behave, while minimizing the chances that you're going to change a parameter and cause unintended consequences.

By default, when you install the Exchange Hub Transport role, it creates two default receive connectors. If you've installed Exchange 2007 into an existing environment with 2003, then you probably already have a Send Connector (SMTP Connector).
While Receive connectors represent an inbound connection point for SMTP, Send connectors relay outbound communications.

The two default receive connectors are configured for authenticated SMTP transactions only.
The "Default" receive connector on Hub is configured for other Exchange servers to authenticate, but it does not accept anonymous email by default.

So users receive the following NDR when they try to send mail from any external address:

This is an automatically generated Delivery Status Notification

Delivery to the following recipient failed permanently:

test@Ex2007domain.com

Technical details of permanent failure:
PERM_FAILURE: SMTP Error (state 12): 530 5.7.1 Client was not authenticated

The easiest way to address this is to add the "Anonymous users" on the Default Receive Connector.

Here is an excellent MSExchange Team Blog for Configuring Exchange 2007 Hub Transport role to receive Internet mail.

Public Folders not able to receive External Emails

2008-02-06T21:41:00.000-08:00

When we set the Mail Enabled Public Folders, it receive all the internal emails fine but is unable to receive any mail from outside and throws the NDR to the sender:

This is an automatically generated Delivery Status Notification.

Delivery to the following recipients failed.
abc@xyz.com

Final-Recipient: rfc822;abc@xyz.com
Action: failedStatus: 5.2.1
X-Display-Name: abc

This happens when Permission to Anonymous is set to None. Make sure Anonymous has atleast Contributor rights.

Stats Setup for monitoring the Website

2008-01-08T21:50:00.000-08:00

Few days back, I needed to set up the monitoring for my front-end webmail server for bandwidth usage, number of hits, tracking of IP addresses, browsers etc... I looked into the number of options but inarguably found that Awstats is the best in this business.

AWStats is a free powerful and featureful tool that generates advanced web, streaming, ftp or mail server statistics, graphically. This log analyzer works as a CGI or from command line and shows you all possible information your log contains, in few graphical web pages.

The Setup consists of 5 major steps:

1. Installing Perl
2. The AWStats Setup
3. Setup IIS Logging
4. Setup AWStats Config File
5. Scheduling Log File Analysis

1. Installing Perl

Download the Perl binary for Windows. Grab the MSI version. GO ahead and run the install.
It will automatically create the Web Service Extension mapping in IIS. However you need to allow that from IIS manager.

2. The AWStats Setup

Now that Perl is installed, we can get AWStats all setup.

1. Run the Setup. Select all the defaults.
2. Copy the contents of its Bin directory to C:\Perl\Bin Folder.
3. Go into IIS Manager, create the virtual directory named stats under default Website for C:\Perl\Bin.
4. Give Execute Permissions, select Scripts and Executables from the dropdown.

3. Setup IIS Logging

Right click the website to be logged, go to properties, and then on the Configuration tab, check off
Enable Logging (if it isn’t already). Select W3C Extended Format from the dropdown, and then click Properties.
Proceed to the advanced tab and uncheck everything. There are certain items here that we’ll tick back off. Namely, the following items should be checked:

Date (date)
Time (time)
Client IP Address (c-ip)
Username (cs-username)
Method (cs-method)
URI Stem (cs-uri-stem)
Protocol Status (sc-status)
Bytes Sent (sc-bytes)
Protocol Version (cs-version)
User Agent (cs(User-Agent))
Referrer (cs(Referrer))

The IIS end is now done, the final step is to setup the AWStats.conf file.

4 . Setup AWStats Config file

It is already copied in the C:\Perl\Bin Folder directory. Copy the existing config file and save it as awstats.mywebsite.com.conf.
*mywebsite.com = name of the website you want to monitor.

The first parameter to setup is your log file item. The only important part is the final piece at the end with the date time codes. Here's the line from my config file below:

LogFile="C:\WINDOWS\system32\LogFiles\W3SVC1\ex%YY-0%MM-0%DD-0.log"

The config file and documentation say that one should specify LogFormat=2 for IIS. Unfortunately, IIS 6 does not follow this predefined format, so we’ll have to specify our own:

LogFormat="date time cs-method cs-uri-stem cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes"

Then we have to specify the value of Site Domain. Site Domain must contain the main domain name, or the main intranet web server name, used to reach the web site.

SiteDomain="www.mywebsite.com"

5. Scheduling Log file Analysis

AWStats only analyzes log files when told to do so. It can easily be toggled from the command line, and running a scheduled task to do this is the best solution.

Here’s a command that look like:

c:\perl\bin\perl.exe awstats.pl -config=mywebsite.com –update

All set now, you can access the stats from http://mywebsite.com/stats

Out of Office Assistant not Working

2007-11-01T04:06:00.000-07:00

The OOF message is a hidden entry in a user’s mailbox. The Out of Office Assistant creates a set of two rules in the Inbox subtree. The first contains a Message Class of IPM.Note.Rules.Oof.Template.MicrosoftMessage, whilst the other rule contains a Message Class of IPM.Note.Rules.OoFTemplate.Microsoft. If one or both of these rules are corrupted or unsynchronized with the OOF-enabled indicator, then it’s possible that the Out Of Office notification may no longer work.

To resolve this, use the Mdbvue32 (Microsoft Exchange Server Information Store Viewer) utility which can downloaded from here to delete the two entries for Out of Office.

Here are the instructions:

1. Run mdbvu32.exe.
2. Click OK to clear the first window that pops up.
3. Make sure that your profile is selected in the Choose Profile window and click OK.
4. Click on the MDB menu option.
5. Click on the OpenMessageStore option.
6. Make sure that "Mailbox- [user's full name]" is selected and click on Open.
7. Click on the MDB menu option again.
8. Click Open Root Folder.
9. In the Child Folders box - double click on "Top of Information Store".
10. In the next Child Folders box - double click on "Inbox".
11. Look in the Associated Messages in Fld box. All of your rules are in this box.
12. Examine each row of entries that appear under Associated Messages in Fld by double-clicking the CB items one at a time.

You are looking for two items that have the following message properties:

The item that contains a message property Ox65EB that displays either OOF Rules or MSFT:TDX OOF Rules.
The items that contain a message property PR_MESSAGE_CLASS that displays IPM.Note.Rules.OofTemplate.Microsoft.

13. Select both CB: values. To do so, press and hold down CTRL while you click both CB values under Associated Messages in Fld.
14. In the "Operations available (select operation, then push Call Function button)" text box, push the drop down button to reveal the list of functions.
15. Scroll down the list of functions until you see "lpFld -> deleteMessages() (ON SELECTED MSGS)" and then click on it to select it.
16. Next press the Call Function button. This will delete the rule that you selected.
17. Press the Close button to exit the MAPI_FOLDER window.
18. Press Close again to exit all the windows.

Reference: http://support.microsoft.com/kb/248709

Mail Delivery Slow: Messages Waiting to be Routed Queue Filled up

2007-10-16T00:06:00.000-07:00

This issue stuck us 2 weeks back and we went all over by this. Suddenly we found out that the emails sent are being received after a lot of delay. Looking at the Exchange Server, I saw thousands of messages in Messages Waiting to be Routed Queue for Default SMTP Virtual Server and the number kept on increasing and the messages were going out very slow.

Slow mail delivery and mass queuing of mail in the Messages Waiting to be Routed queue in Exchange is typically caused by either Anti-Virus software on the Exchange server, by Distribution List expansion problems, or by connectivity problems between Exchange and Active Directory.

Antivirus was already ruled out as we tried disabling it from the registry as well but with no affect.

I opened up a case with MS PSS... turned up the diagnostics logging for MS Transport and MS DSAccess but nothing conclusive from the logs.

WE monitored the LDAP read and search times, SMTP categorizer queue length as it seemed to be the performace issue. Here is excellent MS guide for Troubleshoting Exchange Server Performance Issues.

Then we collected the Hang Dumps for Store.exe and Inetinfo.exe.

In the Store dump we see that we are waiting on WLAP calls to the GC’s.
From the Inetinfo dumps, we are waiting for the HrCheckRestrictions which means that the mail was probably to a DL that had restrictions placed on it.

So we figured out that Delviery Restrictions might be the cause of mail delivery being slow as we have applied the delivery restrictions on some Distribution Lists quite recently.

This problem occurs when lots of Lightweight Directory Access Protocol (LDAP) searches are initiated. Lots of LDAP searches are initiated when you send mail to distribution groups that include lots of users who have delivery restrictions configured on their mailboxes.

If you send a message to a group that contains many recipients, and if each of those recipients is also configured with a delivery restriction to reject messages from the members of a distribution group that contains many members, Exchange 2000/2003 Server must expand the restricted distribution group one time for each member of the group to which you sent the message. Also, if a failure that can be retried occurs during this process, Exchange Server stops the group expansion process, and then retries the connection an hour later. This causes the messages to be held in the categorizer queues, delays message processing, puts load on transports' Advanced Queuing and SMTP components, and eventually causes system queues to start backing up.

Here is the Excellent MS Exchange Team Blog for Performance issues due to connector restrictions.

Well, the solution to this above was the hotfix (already included in Exchange SP2) and change in one registry entry that defines the new Expansion Logic for restricted Distribution Group.

Here are the details:

1. Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeTransport
2. On the Edit menu, point to New, and then click Key
3. Type Parameters, and then press ENTER to name the new registry subkey.
4. Right-click Parameters, point to New, and then click DWORD Value.
5. Type RestrictionMethod, and then press ENTER to name the new registry entry.
6. Right-click RestrictionMethod, and then click Modify.
7. Type 2, and then press ENTER .

Reference: http://support.microsoft.com/default.aspx?scid=kb;EN-US;895407

Mailbox Move: Disconnected Mailbox remains on source Mailbox Store

2007-09-18T22:52:00.000-07:00

I faced this issue while moving the mailbox from one mailbox store to a mailbox store on another server.
The mailbox is moved successfully and the user can access it fine on the server it was moved to.

However, the mailbox remains on the source mailbox store in a disconneced state and has small size. The number of items are shown as 0. The mailbox is disconnected and it does not allow you to purge or reconnect it to another user giving the message "This mailbox is already connected to a user."

To be able to purge the mailbox from the Target Store, try to move the mailbox back to that (original) Store. Move Mailbox will fail and gives you the message:

"A duplicate mailbox was found due to problems during a Move Mailbox procedure. The duplicate mailbox has been deleted. Try again later."

Hence when you run the cleanup agent on the original source mailbox store... that duplicate mailbox will be purged.

Reset the language for the folder names inside the Mailbox

2007-09-05T02:16:00.000-07:00

In Exchange 2000/2003, when one of the following conditions takes place, the default language names of the folders inside the user's mailbox might change and names of standard folders in an Exchange mailbox, such as the Inbox, Sent Items, Public Folders, and so on, appear in different languages when seen from the client program:

You move a mailbox from one Exchange server to another Exchange server.
You create a new mailbox.

Exchange creates the folders based on the language of the Outlook client that first touches the mailbox, or the language of IE if OWA is first to touch the mailbox.

To reset the folder names, here is the registry tweak

In HKEY_CURRENT_USER\SOFTWARE\Microsoft\Office\11.0\Outlook\Setup, create a DWORD value ResetFolderNames and set its value to 1.

Here is more information about this from Microsoft.

Greylisting and Exchange

2007-08-30T23:17:00.001-07:00

There is a potential issue between the Mail Servers that implement Greylisting and MS Exchange SMTP Servers.

Greylisting is used on some mail servers to tempfail first attempt of an email, asking the sending server to retry later. When Exchange tries to send mails to certain domains that implement ‘greylisting’, the mails fail to get delivered and an NDR is generated. Here is an example of what that NDR looks like:

"You do not have permission to send to this recipient. For assistance, contact your system administrator. server.domain.com #4.7.1 smtp;450 4.7.1 <recipient@greylistdomain.com>: Recipient address rejected: Greylisted"

The problem is that the sending Mail Servers are not delaying in response to a 450 "mailbox unavailable" response. The standard (RFC2821) specifies this as a transient condition and the sender should re-queue the message and resend it later. While it's reasonable to fail a message after receiving a number of these "transient failure" responses, the timeout before resending should be higher than 1 second - 10, 15 or 30 minutes are usual values.

By defaut, messages receiving a 4xx SMTP response are processed as a "glitch" 3 times before being put back into the queue for processing on the retry interval. So the problem is when the server resend the message 2 more times with a 1-second delay between attempts and then (presumably) fails delivery and notifies the sender that an error has occurred.

So as a workaround, we need to assertively set GlitchRetrySeconds to a value that allows the greylisting conditions to be satisfied, 120 seconds would do good in most of the cases.

How to Configure Glitch Retry Interval in Exchange Server 2003

Public Folders and Exchange 2007

2007-08-05T23:50:00.000-07:00

Public folders are slowly being pushed out of Exchange into SharePoint 2007 and thus is no longer available in OWA from Exchange 2007.

Exchange 2007 client access server has some limitations in public folder support: no IMAP, NNTP, nor OWA access to Public Folders.

Only Public Folders that are stored on an Exchange 2003/2000 server can be accessed via a browser. There is a speculation that PF access via OWA 2007 will be made available when Exchange 2007 SP1 rolls out.
Also with the release of Exchange 2007 SP1, there will be the Public Folder Management Console for Public folders management. As of now public folder management can only be done through the command shell.

Exchange 2007 is surely de-emphasizing public folders. Public folders may not be included in future releases, but support for public folders will be maintained through at least 2016.

Here is what MS Exchange Team say about Exchange 2007 and Public Folders.

Restrict expanding a Distribution List for users

2007-08-01T23:45:00.000-07:00

I have heard this question a lot... Here is way we can restrict expanding a distribution list so that the users can't see its members.

Use ADUC. Right-click the DL and click on "Exchange tasks...". From there, select "Hide membership".

This is useful when we have the Message restrictions applied on the properties page for the DL and it fails when the user expands the list to its members, then the message is sent again to everybody.

Exchange Management Shell

2007-07-18T01:11:00.000-07:00

It may first appear as if the Exchange Management Shell is nothing more than a standard command prompt. But if you look at its prompt, you will notice that the letters "MSH" appear in brackets just ahead of the path.

The [MSH] tells you that you are not running in a true command prompt environment, but rather within a Microsoft Scripting Host shell. The Exchange Management Shell is nothing more than a Microsoft Scripting Host environment that has been extended to support Exchange Server commands.

There are many management tasks that are only performed on the Exchange Managament Shell and CANNOT be performed in Exchange Management Console. Some of them are:

All Public Folder management.
Give permissions to user's mailbox, entire database or the server.
Advanced database, mailbox and recipient management.
Advanced Transport Settings like setting a maximum message size limit for incoming and outgoing messages on the organization or connector, Set advanced SMTP connection settings etc.
Certain Client Access settings also like set connection time-outs for POP3/IMAP4 servers (Set-IMAPSettings / Set-POPSettings), Prevent previous versions of Outlook from connecting to Exchange (Set-CASMailbox –MAPIBlockOutlookVersions), Enable/disable POP3 or IMAP4 for a user (Set-CASMailbox) etc...

Here is some collection of scripts for managing Exchange Server 2007 that are very useful for day to day operation and management.

Sender Policy Framework (SPF)

2007-07-10T00:15:00.000-07:00

The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent sender address forgery. It is an attempt to control forged e-mail. SPF is not directly about stopping spam – junk email. It is about giving domain owners a way to say which mail sources are legitimate for their domain and which ones aren't.

The current version of SPF — called SPFv1 or SPF Classic — protects the envelope sender address, which is used for the delivery of messages.

Sender authentication protocols are designed to protect against forgery of e-mail sender identities, either in the envelope or in the header. In the envelope, first there is the "HELO" identity, which names the mail server (MTA) that is sending the message. The "MAIL FROM" identity is the e-mail address that is responsible for sending the message and where delivery errors (bounces) will eventually be reported. And the "RCPT TO" identity is the message's recipient address. The header contains another set of identities (besides other meta information about the message, such as the subject and the sending date).

SPF authenticates the envelope HELO and MAIL FROM identities by comparing the sending mail server's IP address to the list of authorized sending IP addresses published by the sender domain's owner in a "v=spf1" DNS record.

SPFv1 allows the owner of a domain to specify their mail sending policy, e.g. which mail servers they use to send mail from their domain. The technology requires two sides to play together:
(1) the domain owner publishes this information in an SPF record in the domain's DNS zone, and when someone else's mail server receives a message claiming to come from that domain, then
(2) the receiving server can check whether the message complies with the domain's stated policy. If, e.g., the message comes from an unknown server, it can be considered a fake.

Example of SPF Record

mydomain.com. TXT "v=spf1 mx a:machine1.mydomain.com include:gmail.com -all"

The parts of this SPF record mean the following:

v=spf1: SPF version 1
mx: the incoming mail servers (MX's) of the domain are authorized to also send mail for mydomain.com
a:machine1.mydomain.com: the machine machine1.mydomain.com is authorized, too
include:gmail.com: everything considered legitimate by gmail.com is legitimate for mydomain.com
-all: all other machines are not authorized

For detailed information on SPF records and their syntax, please refer www.openspf.org
The SPF Setup Wizard: http://old.openspf.org/wizard.html

Troubleshooting Fast Growing Transaction Logs

2007-07-04T02:49:00.000-07:00

Here is a pretty good technet that details various causes:

http://technet.microsoft.com/en-us/library/aa996191.aspx

Also when we mount/dismount the stores manually, the transaction logs files are replayed on to the database thus they get committed and we are able to mount stores.

We can check this status by using eseutil: eseutil /mk "path to checkpoint file"

It will give you the transaction log file uptill which all the logs have been committed. This is helpful in the case we are running out of disk space and when the full backup is also a long way to finish… so that we can move out the committed logs and keep them on a different location. In case of recovery scenario, we need these log files to make database up-to-date.

Disaster Recovery (Soft Recovery) - Exchange 2003

2007-06-20T23:21:00.000-07:00

After an unexpected shutdown of the Exchange Server, the Information Stores in One Storage Group goes dismounted. Gave an error message while mounting.
Checked the state of the databases by eseutil /mh and there it shows dirty shutdown.

Here it can be that one of the database is not up-to-date and the server got rebooted when it was processing one of the transactions and got corrupted.

Our next step in this case is to identify the store that has been corrupted.

We will try to replay the logfiles on the each of the stores individually:

Renamed priv2.edb and priv2.stb.
Replayed the logfiles: eseutil /r /i E00
Error Encountered. So log files cannot be replayed for Information Store 1 (priv1.edb, priv1.stm)
Now Renamed priv1.edb and priv1.stm.
Replayed the logfiles: eseutil/ r /i E00
Completed Successfully. So the Information Store 2 is up to date now and we are able to mount it.

Now as the Information Store 1 is corrupted, the best practice is to restore from the last backup and then make the database up to date by recovering from the present log files.Eseutil /p should be the last resort as it performs the hard recovery and punches off the corrupted tables or entries from the database. So there is always a good chance of loss of some data.

As we have renamed priv1.edb and priv1.stm, the database restore from the backup will create new set of these files. We should not overwrite these files from the restore.

The Restore of backup (Veritas in this Case) should be set with the following options:

Do not replay logs automatically
Do not mount databases
No Loss restore (don't overwrite or delete anything)

After the restore, we have the new set of edb and stm (a day old) and the current log files that will make them up to date.
Now we will use "ESEUTIL /CC" command to replay to restore.env file and continue on to production log files. There are temporary log files and restore.env file created when we run the restore. In case of Veritas restore, they are created in C:\temp\

Restore.env file is just a temporary environmental file holding path information about the data. Its purpose is to help the Exchange 2003 restore process find its files and match them with the corresponding email stores.

Now run the command eseutil /cc "path of restore.env"

Mount the store back after it completes successfully.

Calculate amount of disk space to be reclaimed by Offline Defrag...

2007-06-13T03:06:00.000-07:00

By using the command eseutil /ms, we can calculate the exact amount of disk space that can reclaimed by doing the offline defrag.

Here are the steps:

1) Run eseutil /ms command from the bin directory on the dismounted database. It will always be run on the edb file.

2) The first section of the space report is the SLV SPACE DUMP. This reports on free space in the streaming database file (.stm).
TOTALS:
Free: 1528814
Reserved: 2238
Deleted: 228
Committed: 11888
Unknown: 0

There are 1528814 free pages in the .stm file, and each page is 4096 bytes in length. Therefore, there are 6,262,022,144 bytes of empty space in the file (1528814 x 4096).

3) As for the EDB file dump you'll see the following information in the last line in the column 'Available'
-------------------------------------------------------------------------------
3244272

The number at the lower right of the output (3244272) is the total of all free pages in the database. If you multiply this number by 4096, you will see that defragmenting this database will recover 12.3 gb of space.

4) Hence, based on the above example you will recover 12.3 GB + 6GB after you do the offline defrag.(this is including the STM also)

System Attendant Mailbox Missing: Users getting NDRs, Cant Move Mailboxes...

2007-06-12T04:49:00.001-07:00

In this scenario, the users are getting NDRs that System Attendant cannot be reached... and when we try to move the mailbox to and from the server, the error says:
"The attempt to log on to the Microsoft Exchange Server computer has failed. The MAPI provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0512-00000000"

The application log has the following error messages:

Event ID : 9175
Raw Event ID : 9175
Record Nr. : 1167709
Category : MAPI Session
Source : MSExchangeAdmin
Type : Error
Message : The MAPI call 'OpenMsgStore' failed with the following error:
The attempt to log on to the Microsoft Exchange Server computer has failed.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0512-00000000

These symptoms indicate that the ‘mailbox store’ that holds the System Attendant mailbox is down or that the SA mailbox is disconnected, but actually all the mailbox stores are up and running on the server.
There is only one mailbox store in an Exchange server that holds SA mailbox... but you cannot find this mailbox on any of the stores.
And when you verify the attributes of System Attendant using ADSIEdit under Microsoft System Attendant, it is found that homeMDB was ‘Not Set’

Recommended Plan:
=================

Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Information Store -, on the right side you’ll see a list of mailbox stores, click on any one of them and copy the value of ‘Distinguished Name’ attribute
Using ADSIEdit, browse to the location – Configuration – Microsoft Exchange - - Administrative Groups - - Servers – – Microsoft System Attendant, right click here and find the attribute ‘homeMDB’, paste the value that you copied earlier (From ‘Distinsguished Name’) here

Restart ‘Microsoft Exchange System Attendant’ service

After the exchange services come up, browse to the store whose DN value you copied earlier and find the System Attendant mailbox under ‘Mailboxes’ list, if you don’t find it run a ‘Cleanup Agent’

If you don’t see the SA mailbox in the ‘Mailboxes’ list on the store even after performing ‘Run Cleanup Agent’. Using ADSIEdit, once you get to the properties of the ‘Microsoft System Attendant’ you can note the value of the the attribute ‘mail’. This is essentially the primary SMTP address of the SA mailbox. You can send an email to this address, that should initialize the mailbox and you should see it under ‘Mailboxes’ tab in ESM.