I was working with a customer who changed some TLS 1.0/1.1 setting on their Exchange server that broken mail flow to Exchange Online. They were in the process of migrating to Exchange Online from their four server Exchange DAG. I found that two servers were unable to send messages to the Edge server and then Exchange Online:
421 4.4.2 Connection dropped due to Socket Error Attempted failover to alternate host, but that did not succeed. Either there are alternate hosts, or delivery failed to all alternate hosts.
In this situation I was able to place the server component Hub Transport in maintenance mode by running:
I recently completed a migration from Exchange 2013 to Exchange 2016 and after the Exchange 2013 server was uninstalled that’s when the issues with OAB started. I confirmed that all the mailbox databases were set with the Offline Address Book.
I confirmed that the OAB Virtual Directory was set with the correct URL.
I ran the Outlook Test Email AutoConfiguration and noticed that in the output it didn’t have a line for OAB.
It turns out that the Offline Address Book needed to be set for Global Web Distribution.
SCOM Alerts can be related to SSL certificates and it is worth checking the IIS BackEnd Site Binding to see if the certificate is valid. One example of this is the alert for OutlookRpcDeepTestMonitor. Also note that if the server alerting is getting a “proxying to unknown” error that the Certificate issue is likely on a different Exchange Server.
Open IIS, browse down to Site and Exchange Back End. Click bindings and edit the site bindings on port 444. The site should be bound with the certificate called “Microsoft Exchange”. When you view the certificate I found the certificate being used had an error “The CA Root certificate is not trusted”.
To fix this issue the self signed certificate needs to be exported from the Personal Store and imported into the Trusted Root CA.
Add the Snap-in for Certificates
Browse down to Personal and Certificates and Export the self-signed certificate where the friendly name is “Microsoft Exchange”.
Export it using the format P7B and select the option to “Include all certificates in the certification path if possible”
Name the file and Save it anywhere you like.
Browse down to Trusted Root Certification Authorities and right click Certificates -> All Tasks and Import…
Select the certificate you exported, click next and ensure the certificate is placed in the Trusted Root CA.
Now back to IIS when you view the certificate that is bound to the Exchange Back End Site it should look like this:
Now you need to restart the Exchange Health Manager service MSExchangeHM on the server that reported the issue or restart it across all the Exchange Servers:
Write-Host Restarting Health Manager Service on$Server
The SCOM alert for MSExchangeDelivery service is failing due to this exception:
Microsoft.Forefront.Monitoring.ActiveMonitoring.Smtp.Probes.MailboxDeliveryAvailabilityProbe+MailDeliveryAvailabilityProbeException:Multiple different exceptions
at Microsoft.Forefront.Monitoring.ActiveMonitoring.Smtp.Probes.MailboxDeliveryAvailabilityProbe.DoWork(CancellationToken cancellationToken)
at Microsoft.Office.Datacenter.WorkerTaskFramework.WorkItem.Execute(CancellationToken joinedToken)
It turns out the health mailbox was full and unable to accept new messages.
To resolve this issue I stopped the MSExchange Health Manager service and deleted all the AD accounts for Health Mailboxes. Health Mailboxes can be found in the Monitoring Mailboxes OU which is inside the Microsoft Exchange System Objects OU by default. After removing the AD objects I restarted the health manager service and new health mailboxes are created automatically.
I was recently asked to determine how much of the mailbox database is taken up by recover deleted items?
It’s good to understand that the recoverable items folder resides in the non-IPM subtree of each mailbox. The non-IPM subtree is a storage area within the mailbox that contains operational data about the mailbox. This subtree isn’t visible to users using Outlook, Microsoft Office Outlook Web App, or other email clients.
With Exchange 2013 this provides the following key benefits:
When a mailbox is moved to another mailbox database, the Recoverable Items folder moves with it.
The Recoverable Items folder is indexed by Exchange Search and can be discovered using In-Place eDiscovery.
The Recoverable Items folder has its own storage quota.
Exchange can prevent data from being purged from the Recoverable Items folder.
Exchange can track edits of certain content.
The Recoverable Items folder contains the following subfolders:
Deletions This subfolder contains all items deleted from the Deleted Items folder. (In Outlook, a user can permanently delete an item by pressing Shift+Delete.) This subfolder is exposed to users through the Recover Deleted Items feature in Outlook and Outlook Web App.
Versions If In-Place Hold or Litigation Hold is enabled, this subfolder contains the original and modified copies of the deleted items. This folder isn’t visible to end users.
Purges If either Litigation Hold or single item recovery is enabled, this subfolder contains all items that are purged. This folder isn’t visible to end users.
Audits If mailbox audit logging is enabled for a mailbox, this subfolder contains the audit log entries.
DiscoveryHolds If In-Place Hold is enabled, this subfolder contains all items that meet the hold query parameters and are purged.
Calendar Logging This subfolder contains calendar changes that occur within a mailbox. This folder isn’t available to users.
If you wanted to know how much data you had in recoverable items you can try the following:
The above shows each of the recoverable item folders for each mailbox
Now I have the total size of all the recoverable items for all the mailboxes in Bytes. Thankfully powershell has an easy way to convert that to GB for me. You don’t need to divide by 1024 then divide by 1024 then divide by 1024 etc. You can copy the result of the Sum and divide it by 1GB.
54GB is the total amount of space used by Recoverable Items.
Now the question came around how much space is taken up by Legal using In-Place Hold. I ran the following command:
I was recently working with a customer who had issues with logging into OWA. The users would get the following error:
Something went wrong
We can’t get that information right now. Please try again later.
Date: 8/3/2017 4:13:24 PM
In the event viewer under the application logs I found the following warnings:
Log Name: Application
Source: MSExchange OAuth
Date: 8/3/2017 11:13:08 AM
Event ID: 2004
Task Category: Configuration
Unable to find the certificate with thumbprint EF6392A5E64713AD43598B7A0FF75080964FB096 in the current computer or the certificate is missing private key. The certificate is needed to sign the outgoing token.
To find the existing certificate for which the authentication configuration is looking you can run:
I found that the certificate returned wasn’t listed when I ran the command Get-ExchangeCertificate. I was required to create a new exchange certificate by running the following commands:
New-ExchangeCertificate-KeySize2048-SubjectName"cn= Microsoft Exchange ACS Certificate"-FriendlyName"Microsoft Exchange Server ACS Certificate"-PrivateKeyExportable$true-Services SMTP-DomainName yourdomain.com
Now to set the AuthConfig to the newly created certificate we need to run the following:
Set-AuthConfig-NewCertificateThumbprint&lt;paste your thumbprint here&gt;-NewCertificateEffectiveDate$date
Now when I check the AuthConfig you can see the update certificate: