The top row is going to coordinate with the S_.value that you are going to use in the following Exchange Shell command:

Import-CSV “C:\Mailboxes.csv” | foreach {new-mailbox –Name $_.name –Alias $_.alias –UserPrincipalName $_.userprincipalname –Database $_.Database –OrganizationalUnit $_.organizationalunit –password (ConvertTo-SecureString $_.password –AsPlainText –force)}

And you should see the mailbox’s created below:

That’s it.  You can see how the values map with their respective column names.  You can add as many users as you want, and change it so they go to different database’s.

You can even create an automated job to export from your production servers, and them import them to your DEV Exchange Servers for testing. 

To view disconnected mailbox’s, essentially mailboxes that have been deleted from their user accounts, you need to first ensure that Exchange has gone through and cleaned the database.  This is done to ensure that it marks that mailbox as deleted.  If your database is MDB36, run the following command:

Clean-MailboxDatabase MDB36

Exchange gives no result from the command.  But now you can view Disconnected mailboxes through the “Disconnected Mailbox” view in the EMC:

You can also view it in the shell by running the following command:

Get-MailboxStatistics –Database MDB36 | where {$_.disconnectdate –ne $null}

And you will receive the following output:

By default, Exchange 2010 keeps disconnected mailbox’s in the DB for 14 days.  But say you want to remove this mailbox now and return it’s white space to use in the DB.  You need to remove the mailbox from the shell. 

You can do this by getting the GUID for the mailbox by running the command:

Get-MailboxStatistics –Database MDB36 | where {$_.disconnectdate –ne $null} | select displayname,MailboxGUID

And you will receive the following output:

Now run the following command to remove the mailbox:

Remove-Mailbox –Database MDB50 –StoreMailboxIdentity 7b40b106-5941-4de0-9fce-27ede21c474e

You’ll receive a confirmation prompt, just accept it, and your all set:

Enjoy!

Lets say you have the following environment:

ExchangeResource.corp hosts all the Microsoft Exchange 2010 servers, and linked mailbox accounts.  The actual user accounts are stored in the Tailspin.corp and Mantech.corp forests.  The Tailspin.corp and Mantech.corp forests have a one way forest trust with ExchangeResource.corp so that users in the Tailspin.corp and Mantech.corp forests can access their linked mailboxes in the ExchangeResource.corp domain.

Now to make things easy on the users, you set the OWA directory to use UPN suffix names instead of Domain\user:

This will allow users in Tailspin to login using username@tailspin.corp and users in Mantech to use username@mantech.corp.

Everything works fine, but then you add a UPN suffix to each individual forest that makes the UPN suffix match the users email address. Below is an example shown with the user Tom Jones in the Tailspin forest:

Now users in Tailspin login using username@tailspin.com and users at Mantech login using username@mantech.com.

A user goes to login with the new UPN and is greeted with an error message that they could not login:

But using the old UPN still works fine, so what’s going on?

Well, if we check the event logs of the DC in the ExchangeResource.corp domain we find EventID 6034 for LsaSrv in the security event log:

The DC is telling us that it does not know how to route the Tailspin.com suffix.  It notes that it has been added to the forest tailspin.corp, as it learns it through the forest trust, but that the name suffix is not enabled.  It does very nicely tell us how to fix this.  Go to Active Directory Domains and Trusts->Right click on ExchangeResource.corp->Properties

Go to the Trusts tab.  Here you will see all the forests that you have trusts with.  Highlight the tailspin.corp forest and click on properties:

Navigate to the Name Suffix Routing tab:

Here we can see the new tailspin.com suffix has been added, it even has a status of “New”, but the Routing is disabled.  Highlight the suffix and then click Enable:

If you do not see the new suffix you created listed here, simply click the Refresh button and it should appear.

After hitting apply both names should be enabled:

Now if a user try’s to login, they should be all set!

Keep in mind you will need to do this each time you add a new UPN Suffix to one of the domains that are being trusted by ExchangeResource.corp.

Enjoy!

Recently had a colleague that ran into an issue with an Exchange 2010 migration.  He could fail over the mailbox databases with no issue to DR, but that’s where the trouble started.

The production database would start to report that there was a high copy queue length that would increase as more activity occurred on the DB.  The production database pure and simple was not receiving the transaction logs from the newly activated database in DR. 

The setup was simple, two nodes, one in production, one in DR with a FSW.  The nodes were all-in-one 2010 boxes, with one NIC for MAPI and one NIC for replication.

My colleague also informed me that he had some trouble initially seeding the database.  All roads pointed to an issue with replication.  We quickly checked his replication network settings and found the following setup:

His DAG network had both replication networks for the separate sites under one object.  Once we moved them to their own separate networks:

Everything went back to normal!

Till the next time!

I recently ran into an issue where users were reporting that messages to their Blackberry handhelds were being delivered in clumps, and were also extremely delayed.  These are symptoms of a Blackberry Server rescan.  We were running BES 5.0.2 and Exchange 2010 SP1.  We upgraded the BES to 5.0.2 MR4, but the issue remained.

This essentially means that the BES server cannot keep up with email delivery, and thus cannot deliver the emails as they arrive.  It is forced to queue them, and deliver them in chunks.

I had already configured the Exchange 2010 throttling policy as per Blackberry’s documentation:

BES 5 and Exchange 2010 Throttling Policy

So throttling wasn’t the issue.  The issue ended up being that the BES server was only creating one mailbox agent in the BES server.  This has to do with the DAG model in Exchange 2010, and the way around it is to create static mailbox agents for EVERY user, or force the BES server to create multiple mailbox agents.  We can do this with two registry keys.  The first one:

HKLM\Software\Research In Motion\Blackberry Enterprise Server\Dispatcher

If it is not there, create a DWORD named MaxUsersPerAgent and set the decimal value to the maximum number of users per agent, in my case 40 users:

The second registry edit is:

HKLM\Software\Research In Motion\Blackberry Enterprise Server\Agents

If it is not there, create a new DWORD with the name MaxMailboxesPerSession and set the decimal value to the value that you want the maximum number of mailboxes to be piped through a single MAPI session.  This is separate from the agent above.  For instance, I set mine to 35, which means at the 36th mailbox, the BES will create a new MAPI session.

After you make those changes, restart your BES server.  After making the above changes, your performance should increase, and you should see extra instances of the BlackberryAgent.exe process:

Just an FYI, SQL Express and MSDE are limited to five agents on the server.  For more agents, you will need to be using SQL Standard or higher, and edit the key:

HKLM\Software\Research In Motion\Blackberry Enterprise Server\Agents\NumAgents and change its decimal value to a number higher than 5:

Again, this value will be ignored higher than a value of 5 on SQL Express and MSDE.  If you have 1000 users, and you set the MaxUsersPerAgent value to be 40 as above, your agent breakdown will be as follows:

BlackberryAgent (1) = 40 Users

BlackberryAgent (2) = 40 Users

BlackberryAgent (3) = 40 Users

BlackberryAgent (4) = 40 Users

BlackberryAgent (5) = 840 Users

So be careful to set the MaxUsersPerAgent value appropriately for your environment if your limited to the number of agents.

As for the BES server itself, we saw slightly higher memory utilization due to the extra agents (around 400 MB), but much lower CPU usage.

Hope you guys find this helpful!

When performing a database *over in Exchange 2010, especially a planned one, it is suppose to be as seamless as possible to the end user.  An Outlook user for example, will receive a small pop up informing them that the connection to Exchange has been lost, and their Outlook may hang for a couple of seconds before reconnecting and resuming normal behavior.

I recently ran into an issue where this was not the case.  Users would call the helpdesk as soon as a database failover was initiated with complaints that their outlook was prompting them for a login:

Well, needless to say this is not expected behavior.  After a little troubleshooting that involved a packet capture, it seemed that the Outlook clients were making an HTTPS call to the CAS servers at that moment.  Turns out it was an attempt to connect to them over Outlook Anywhere, and this was the reason of the login prompt.  When I checked the Outlook client, they in fact had the Outlook Anywhere Settings enabled.  This was due to Autodiscovery.  To check the settings in Outlook 2007 navigate to Tools->Account Settings->Change->More Settings->Connection  If yours is enabled, it will look like the following:

Under Exchange Proxy Settings, you’ll find the settings enabled:

Since these were internal clients, and had no need to use Outlook Anywhere, simple unselect the Connect to Microsoft Exchange using HTTP:

This will disable the Outlook Client from connecting.  The only issue is, if you use automatic profile generation through Group Policy, this leverages autodiscovery, so it will continue to put the setting back.  You can do one of two things.  The first is to delete the Outlook Anywhere provider using the Remove-OutlookProvider command, which is NOT recommended.  This will stop Autodiscovery from publishing Outlook Anywhere GLOBALLY. 

The second is to use Group Policy.  Create a blank GPO named something like Disable Outlook Anywhere Settings.  Download the Outlook Anywhere ADM template from here, and import it into the template under User Settings.  You’ll now have the Outlook Anywhere (RPC/HTTP) options available in Group Policy:

The only value you need to edit here is the RPC/HTTP Connection Flags setting:

Edit the setting, set it to Enabled and No Flags

This will disable the Connect to Microsoft Exchange Using HTTP in the outlook clients after its been applied, notice how its greyed out:

Once this GPO has applied to all your users, you should now be able to failover databases without the users receiving a log in prompt. 

Thanks to both user comments and NetApp themselves, we have determined that there is an easier way to add disks to members of a DAG without removing them from the DAG.  You can simply stop the Windows Clustering service (run the command “net stop ClusSvc” from the command line).  Obviously you should move any active mailbox databases on this DAG member to another DAG member before doing this, as stopping the clustering service will cause the databases to dismount and move on their own.  From there, you should be able to add the disk’s, start the clustering service, and your DAG member will automatically return to a normal operating mode, participating in the DAG.

Thanks to all who sent this in!!

**Original Article**

I recently ran into an issue where I was unable to add an RMD LUN to a Windows Guest running on VMWare vSphere.  Here is my setup.

I had a Windows 2008 R2 guest that was running Exchange Server 2010 SP1.  The guest was a Mailbox server that was a member of a Database Availability Group.  I was attaching the LUN’s to iSCSI RDM’s that were based on a NetApp FAS 3140 running ONTAP 7.3.2.  The guest was running version 6.3 of Snapdrive.

The guest had 12 iSCSI RDM’s working properly for month’s, but the issue arose when I tried to add more.  I would be able to select the volume, create the LUN, size, the mounting location of the LUN.  The issue was when in Snapdrive I was presented with where to store the RDM file for the VM.  See the screen below:

The problem was the console starting freezing up, and generally not responding.

After several minutes, I eventually received an error stating there was an “error in fetching number of vmfs datastores”

I tried all the basic’s, re-installing Snapdrive, upgrading to Snapdrive 6.3 PP1, rebooting the host, stopping and starting the service.

Turns out there is a bug in Snapdrive that causes the error above, when the Guest is a member of a Windows Cluster.  Since all DAG members utilize Windows Clustering, this applied to me.   The resolution was easy.

I moved all the databases off of the server in question.  Then, in Exchange Management Tools, I went to Organization Config->Mailbox and selected the Database Availability Tab.  Right click your dag and select Manage Database Availability Group Membership:

Right click the server in question, select Delete and then the manage button.  This will remove the server from the DAG.

[This will not cause any issue with the existing databases as we’ll see below]

Now, go back into Snapdrive and add your LUN’s, all should be working now.

After your done, add the server back to the Database Availability group, almost the same way you removed it, this time select Add, and then select the previously removed server and add it back.

Next, for each MailboxDatabase that the server has copies of, run this command in the Exchange Management Shell:

Add-MailboxDatabaseCopy –Identity MB01 –MailboxServer NYDAGNODE1

Or in the EMC, go to Organizational Configuration->Mailbox and right click each Mailbox Database and select Add Database Copy.  Then select your server.

Since the server still has copies of the Mailbox Databases, it will start to resynchronize with the DAG, and bring itself up to date.  That way you won’t need to reseed your entire DB which can take some time.

Hope someone finds this useful!

When doing a migration from Exchange 2007 to Exchange 2010, one of the biggest item’s you need to watch out for is the migration of the ActiveSync environment, and be aware of how it can affect your end users.  You should also be aware of potential issues depending on the TYPE of active sync device you are using, as some will work, and other’s will have issues.

First we’ll start with the migration.  Our current Exchange 2007 ActiveSync environment is as follows:

Here, we have one internet facing site, the NY site.  There is a DNS record for the CAS server in NY, activesync.company.com that points to the IP of the CAS server on the internet.  We have set the –externalurl to activesync.company.com and the –internalurl to newyork2007.company.local.  Note that newyork2007 is the NETBIOS name of the CAS server in NY.  We set both of these values with the following command:

Set-ActiveSyncVirtualDirectory –Identity “newyork2007\Microsoft-Server-ActiveSync” –InternalURL newyork2007.company.local –ExternalURL activesync.company.local

In London, we have set the InternalURL attribute to the local name of the server, but leave the ExternalURL attribute blank.  We do not populate the ExternalURL attribute because London is not accessible directly from the internet. 

Setting the –internalurl attribute updates the SCP in active directory, so that any system that query’s AD itself, will be able to return the internal URL the user should access.  For instance, in our above scenario, LON07 the user configures his active sync device from the internet.  He put’s in activesync.company.com as the server address, which is the external DNS name of the NY CAS server.  The NY CAS server, as part of the Active Sync process, query’s Active Directory for the home mailbox of LON07, and then determines which site LON07 is in.  Since LON07 is not in the same site as the NY CAS, the NY CAS then returns the value for ExternalURL.  If we had entered a value here, such as lonactivesync.company.com, the users device would be redirected to it (as long as the device supported auto discovery, more on that later), and the user would connect, as long as that was configured properly.  In our case, since there isn’t, the NY CAS uses the InternalURL entry to determine what address the NY CAS should use to proxy on behalf of the LON07 user.  Essentially the NY CAS connects to the London CAS, and returns the Active Sync info to the users device, all seamless to the LON07 user.

Now, we start to introduce Exchange 2010 to the equation.  Microsoft’s high level recommendation is to create a new namespace, called legacy.company.com and have this entry point to the 07 CAS, and slide the 2010 CAS into the existing activesync.company.com namespace.   See the below diagram:

So we would need to reconfigure the –ExternalURL and –InternalURL attributes of the NY 07 CAS servers, as well as the NY 2010 CAS servers.  They can all be done by changing the values of the command listed earlier in this article.  The logic here is the same as 07-07 proxy.  If the NYC07 user enters in activesync.company.com into his/hers server address on their ActiveSync device, the 2010 CAS server will query AD, and determine that he is a 2010 user, but in the same AD site.  It will then query to see if an ExternalURL setting is populated, in which case ours is.  That users device, if it supports activesync, will automatically be redirected to legacy.company.com and their profile loaded, all seamless to the end user.

If LON07 enters in activesync.company.com the NY 2010 CAS server query’s Active Directory, finds his mailbox is in another site, and checks to see if there is an ExternalURL entry.  Since their isn’t, like before, it proxies the connection to the London 07 CAS server, all seamless to the end user. 

Now, this is all great, but what happens if your device does not support auto discovery?  Some active sync devices don’t work properly with auto discovery, and in that case, Microsoft recommends that you manually change their profile to point to legacy.company.com.  Maybe not even that, but for security purposes you don’t allow external devices to use auto discover to determine the settings.  In this case, you again have to manually point those devices to legacy.company.com  If you have any significant number of users, this can be insanely time consuming. 

Let me show you an example.  I had configured everything as it was in the above diagram.  I was configuring his active sync on an Apple iPad, a device that supports activesync.  Problem was, his account wasn’t working.  The following log file was taken from the NY 2010 CAS server for the NYC07 user, they are located at c:\inetpub\logs\LogFiles\W3SVC1:

Here, we can see that the NY 2010 CAS server is telling the device that it has the wrong URL, and is redirecting it to legacy.company.com.  This is because the device has advertised that it can do auto discover.  In our example, since auto discover is disabled, or because the device doesn’t handle auto discover properly, the user was getting a connection to server error on the iPad.

Now, with NO changes, let’s try configuring the same user, but not using the iPAD, but using Touchdown for the Android.  Now, all work’s without issue, here is the log files:

In this case, the configuration worked without issue.  Notice how it also says PrxTo:newyork07.company.local.  This is because since Touchdown did not advertise to Exchange that it could do auto discovery, Exchange knew it would have to proxy the connection back to the NY 07 CAS server to allow this to complete successfully.

The funny thing is, if we were to configure LON07 on the iPAD it would work fine.  Why?  Because since the London 07 CAS server does NOT have a value for ExternalURL, Exchange knows it HAS to proxy to London 07 CAS for all London users.

So, we want the same behavior for the NY users on 07.  To do so, we simply need to clear the ExternalURLvalue on the NY 07 CAS server.  We would do so with the following command:

Set-ActiveSyncVirtualDirectory –Identity “newyork2007\Microsoft-Server-ActiveSync” –InternalURL newyork2007.company.local –ExternalURL $null

This would wipe out the ExternalURL value.  The downside to this, is that auto discover for this URL would not be included, so if you used Outlook Anywhere, or other devices to connect using Auto discover, it would cause issues.  If you didn’t though, for instance you disable auto discover, this fixes all your issues.  Now when you try to connect NYC07’s mailbox to the iPad, since there is no ExternalURL entry for the NY 07 CAS server, the NY 2010 CAS server is forced to proxy:

Now, all existing 07 users will continue to have access to their mailbox’s via active sync and will not need any changes when their mailbox’s are moved to 2010.

Exchange 2010 SP1 has been released, and comes with a slew of new and exciting features.  Since we are all clamoring to get this installed in our environments, we should discuss how exactly we upgrade the members of our DAG so as to provide zero downtime to our users, and get our systems patched correctly.

From a high level view, remember we should be patching servers in the following order:

1. Client Access Servers
2. Hub Transport Servers
3. Edge Transport Servers
4. Mailbox Servers

The process we discuss in this article can and should be applied to ALL updates to members of a DAG, not just major updates like service packs.

In our environment we have three total nodes:

1. NYDAGNODE1
2. NYDAGNODE2
3. BOSDAGNODE1

Let’s start with BOSDAGNODE1.  Step one would be to move all active databases on BOSDAGNODE1 to another server.  You want to ensure that it does not have any active databases on it.  We can accomplish this in two ways.

In the EMC navigate to Server Config –>Mailbox.  Select the server in question and then select Switchover Server:

Then you can either automatically choose a target server, or specify one yourself:

Or, through the shell, you can run the command:

Move-ActiveMailboxDatabase –Server DKPBOSDAGNODE1

This will do the same thing as the console, and will automatically choose a target server.

Next, we want to block DKPBOSDAGNODE1 from activating it’s databases.  This will prevent other servers from failing over to it.

Set-MailboxServer DKPBOSDAGNODE1 –DatabaseCopyAutoActivationPolicy:blocked

Now you can upgrade the server to Exchange 2010 SP1!

After its finished, run the following command to re-enable activation of the node:

Set-MailboxServer DKPBOSDAGNODE1 –DatabaseCopyAutoActivationPolicy:unrestricted

Now, you can proceed on the other nodes until your finished!

One caveat you need to be aware of.  Exchange 2010 RTM servers can failover TO a dag node member running Exchange 2010 SP1, but a server running Exchange 2010 SP1 cannot failover to a dag node member NOT running Exchange 2010 SP1.  So make sure your entire DAG is upgraded in a timely fashion!

As for your schedule, note that I chose to do Boston, which is our DR site first, because NY would still be able to fail over to it in case of a DR situation.  Next, I can do one node at a time in NY.  This allows me to keep my mailboxes local to my production site, while ensuring that I am covered should a DR situation arise. 

With the recent release of the Apple iPad, the new iPhone, not to mention the numerous Google Android phones available, there has been a dramatic increase in interest in using Exchange ActiveSync along with Exchange Server 2010 or Exchange Server 2007. 

Along with using these devices, comes certain questions regarding security.  One of those topics, covered by this post, is how to restrict end users to a specific ActiveSync device.  Some ActiveSync devices do not support certain features, that Exchange Admins may want to ensure don’t connect to their systems.

For this example, we’ll run the Get-ActiveSyncDeviceStatistics –Mailbox pponzeka command to determine the DeviceID of the users current ActiveSync device:

Note the DeviceID listed, 413030303030313542354533744.  This is akin to a serial number for this particular active sync device, its unique per device.  We can lock down this use, so that he can only use THIS device to connect to his mailbox via activesync.

To do so, we simple run the command Set-CasMailbox pponzeka –ActiveSyncAllowedDeviceIDs number1,number2

If we had multiple devices, you would just list both numbers separated by a comma.

If you ever want to remove the restriction, simply enter the null value:

Set-CasMailbox pponzeka –ActiveSyncAllowedDeviceIDs:$null

This will set this users mailbox back to the default of allowing all activesync device’s to connect!