I’ve recently worked with a client to troubleshoot RADIUS authentication issues between their Cisco Nexus as a RADIUS client and their Microsoft Windows 2012 R2 NPS (Network Policy Server) server as the RADIUS server and after determining the issue, the client asked me why I never wrote a blog post on the steps that I took to troubleshoot issues like these so this post serves as a way to demonstrate the process.

The first place I look when a RADIUS client is not able to successfully authenticate against Active Directory through a Windows 2012 R2 NPS server is the directory:

C:\Windows\System32\LogFiles

Where the following IN####.log (YYMM) logs are found:

Each of the IN logs contain connects from RADIUS clients over a month so if I am troubleshooting RADIUS issues that are occurring at that point in time then I would:

1. Open the latest log file
2. Move the cursor to the last entry of the log file
3. Use the Find feature (CTRL + F) and search for the RADIUS client’s IP address

The following is an example of an entry from a client with an IP address 10.92.9.11 that I am troubleshooting:

Being able to locate the IP address of the problematic client in the log above allows me to:

1. Verify that the client is indeed reaching out to NPS server with the RADIUS request
2. Determine the exactly time of the request

Item #2 is important to have because the next step is to open the event logs of the NPS server and navigate into the Security events:

Those who have ventured into these logs would know that a lot of entries are written into this log especially if it was a domain controller.  The timestamp obtained in the log file above will allow us to navigate to the section of the logs where we’ll find the relevant entries.  Since the error message on the Nexus login was an “invalid password or user name”, I went ahead and filtered the Security events with the Keywords: Audit Failure:

Navigating to the entries with the same timestamp displays event IDs 6273 and 4625 entries that provide information about why the login failed:

Network Policy Server denied access to a user.

Contact the Network Policy Server administrator for more information.

User:
Security ID:
domain\argotest
Account Name:
argotest
Account Domain:
domain
Fully Qualified Account Name:
domain.internal/domain/Users/Test Accounts/argotest

Client Machine:
Security ID:
NULL SID
Account Name:
-
Fully Qualified Account Name:
-
OS-Version:
-
Called Station Identifier:
-
Calling Station Identifier:
-

NAS:
NAS IPv4 Address:
10.92.9.11
NAS IPv6 Address:
-
NAS Identifier:
-
NAS Port-Type:
Virtual
NAS Port:
0

RADIUS Client:
Client Friendly Name:
NX-1
Client IP Address:
10.92.9.11

Authentication Details:
Connection Request Policy Name:
Use Windows authentication for all users
Network Policy Name:
Connections to other access servers
Authentication Provider:
Windows
Authentication Server:
SVRARDC01.domain.internal
Authentication Type:
PAP
EAP Type:
-
Account Session Identifier:
-
Logging Results:
Accounting information was written to the local log file.
Reason Code:
65
Reason:
The Network Access Permission setting in the dial-in properties of the user account in Active Directory is set to Deny access to the user. To change the Network Access Permission setting to either Allow access or Control access through NPS Network Policy, obtain the properties of the user account in Active Directory Users and Computers, click the Dial-in tab, and change Network Access Permission.

Network Policy Server denied access to a user.

Contact the Network Policy Server administrator for more information.

User:
Security ID:
NULL SID
Account Name:
argotest
Account Domain:
domain
Fully Qualified Account Name:
domain\argotest

Client Machine:
Security ID:
NULL SID
Account Name:
-
Fully Qualified Account Name:
-
OS-Version:
-
Called Station Identifier:
-
Calling Station Identifier:
-

NAS:
NAS IPv4 Address:
10.92.9.11
NAS IPv6 Address:
-
NAS Identifier:
-
NAS Port-Type:
Virtual
NAS Port:
0

RADIUS Client:
Client Friendly Name:
NX-1
Client IP Address:
10.92.9.11

Authentication Details:
Connection Request Policy Name:
Use Windows authentication for all users
Network Policy Name:
-
Authentication Provider:
Windows
Authentication Server:
SVRARDC01.domain.internal
Authentication Type:
PAP
EAP Type:
-
Account Session Identifier:
-
Logging Results:
Accounting information was written to the local log file.
Reason Code:
16
Reason:
Authentication failed due to a user credentials mismatch. Either the user name provided does not map to an existing user account or the password was incorrect.

An account failed to log on.

Subject:
Security ID:
SYSTEM
Account Name:
SVRARDC01$
Account Domain:
domain
Logon ID:
0x3E7

Logon Type: 3

Account For Which Logon Failed:
Security ID:
NULL SID
Account Name:
argotest
Account Domain:
domain

Failure Information:
Failure Reason:
Unknown user name or bad password.
Status:
0xC000006D
Sub Status:
0xC000006A

Process Information:
Caller Process ID:
0x384
Caller Process Name:
C:\Windows\System32\svchost.exe

Network Information:
Workstation Name:
 
Source Network Address:
-
Source Port:
-

Detailed Authentication Information:
Logon Process:
IAS
Authentication Package:
MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Transited Services:
-
Package Name (NTLM only):
-
Key Length:
0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

Reviewing the events above should give you more insight as to why the authentication process is failing but note that the information could be misleading at times as if you read all of the events above, it would seem like the authentication process is failing because the user name and password is incorrect but the problem in the situation above was that an incorrect group was placed into the actual NPS policy thus causing the authentication to fail.

Hope this helps anyone who may be looking for information on how to troubleshoot NPS authentication issues.

I was recently asked by a colleague of mine as to where he could download the Wyse USB Firmware Tool / Dell USB Firmware Tool as shown in one of my previous blog posts:

Attempting to pull and image from a Wyse thin client fails with: “Copy image to USB failed Press Enter key to reboot now.”
http://terenceluk.blogspot.com/2014/09/attempting-to-pull-and-image-from-wyse.html

… because he searched all over the internet and couldn’t find the following page I included in the post above:

I haven’t had to work with Wyse thin clients for a while but after searching through the Wyse and redirected to Dell site, I noticed that this tool was no where to be found. It was not until about 15 minutes later when I finally found it so I thought I’d write this quick blog post in hopes that such an activity wouldn’t waste other administrator’s time.

Many of the Wyse tools such as PCE, TXC, USBFT, WCM and WDM have been moved to the following URL:

https://downloads.dell.com/wyse/

This URL brings you to the following no nonsense and no frills directory:

The directory we’re interested in is the USBFT directory where you will find the USB Firmware Tool:

https://downloads.dell.com/wyse/USBFT/

For those who are interested, I found this information in the following post:

http://en.community.dell.com/techcenter/enterprise-client/wyse_general_forum/f/4949/t/19629018

I’ve recently worked with a client who wanted a website load balanced by a Citrix NetScaler to have access filtered based on the source IP address of the incoming client.  There are probably quite a few ways to accomplish this but the two that I am aware of is either through ACLs or a Responder Policy.  I personally do not like to use ACLs for situations like this because of the “global-ness” that they are configured so the purpose of this post will demonstrate how to accomplish this with a Responder Policy.

Begin by navigating to AppExpert > Responder > Policies:

The first policy we’ll be creating is the catch all policy that will drop all connections:

Name: Drop_All_IPs_Traffic

Action: Drop

Express: TRUE

This policy can also created with the following command:

add responder policy "Drop_All_IPs_Traffic" TRUE DROP

The next step is to create a policy that allows workstation subnets to access the site:

Name: Allow_Wkstns_Subnet

Action: NOOP

Express: CLIENT.IP.SRC.IN_SUBNET(10.43.3.0/24) || CLIENT.IP.SRC.IN_SUBNET(10.43.4.0/24)

This policy can also created with the following command:

add responder policy "Allow_Wkstns_Subnet" q/CLIENT.IP.SRC.IN_SUBNET(10.43.3.0/24) || CLIENT.IP.SRC.IN_SUBNET(10.43.4.0/24)/ NOOP

Note that the above expression allows the subnets: 10.43.3.0 and 10.43.4.0 to access the site.  If the requirement is to specify a range, you can use the following expression:

CLIENT.IP.SRC.BETWEEN(10.43.3.1,10.43.3.100)

If the requirement is to specify an individual IP, you can use the following expression:

CLIENT.IP.SRC.EQ(10.43.3.136)

With the Responder Policies created, proceed by opening the properties of the load balancing virtual server of the website and add a new Responder Policy:

Choose Policy: Responder

Choose Type: Request

Proceed and select the catch all policy that denies all IPs:

Select Policy: Drop_All_IPs_Traffic

Priority: 100

Goto Expression: End

Invoke LabelType: None

With the policy binded, proceed to click on Add Binding to bind the policy that allows access for specified IP address or subnets:

Select Policy: Allow_Wkstns_Subnet

Priority: 90

Goto Expression: End

Invoke LabelType: None

Notice that the priority is configured to be 90 for this policy because we want the responder policy to evaluate this policy first to determine whether the incoming request is from these source IP addresses, if it is then allow access, otherwise proceed to the next responder policy which denies access to all IP addresses.

Once the configurations are applied, you should now either be able to access the webpage if the device is on the workstation subnet defined in the responder policy or unable to resulting in a page similar to the following:

A client recently reached out to me to assist with a requirement that was requested after receiving a penetration test from an external vendor for their older Windows Server 2008 R1 Citrix XenApp 6.5 environment where when attempting to navigate to a page that did not exist, the browser would display a HTTP Error 404.0 – Not Found page that reveals the IIS version along with some other details of the web server:

What the client wanted to do was simply redirect the page to a custom page that did not reveal any information about the server.  To do this, he placed a 404-Copy.htm page in the C:\Inetpub\wwwroot\Citrix directory:

… then redirecting the page via the following field in the 404 Custom Error Page:

/Citrix/404-Copy.htm

The problem with the configuration above is that the user would now be presented with the following HTTP Error 500.19 – Internal Server Error page with the message:

Absolute physical path "C:\inetpub\custerr\" is not allowed in system.webServer/httpErrors section in web.config file. Use relative path instead.

Searching for this error brought me to the following Microsoft blog post:

Custom Error Pages – HTTP Error 500.19 – Internal Server Error
https://blogs.msdn.microsoft.com/benjaminperkins/2012/05/02/custom-error-pages-http-error-500-19-internal-server-error/

Which suggested to used the Configuration Editor to configure the allowAbsolutePathsWhenDelegated to true but this option was not available in the IIS administration console:

The following is from another server with IIS 7.5 on Windows Server 2012 that has the Configuration Editor available:

After trying to find another way to set the variable but unable to find a way to, I found that we could get around this by simply place the 404-Copy.htm page in the root directory C:\Inetpub\wwwroot:

Then referencing the page via the path:

/Citrix/404-Copy.htm

Providing us with the result that we wanted:

Another method which is not preferred is to completely remove the 404 error page as such:

Which would result in the following page displayed:

Troubleshooting this issue on this older Windows 2008 R1 server was a but of a nuance so I hope this post would save someone else a bit of time.

Problem

You’re currently migrating from Exchange Server 2010 to 2016 and have just reconfigured the Send Connectors to route outbound SMTP mail through Exchange 2016 but notice that sent emails are stuck in the Exchange 2010 queues with the following error:

451 4.4.0 Primary target IP address responded with: “421 4.3.2 Service not available.” Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.

Solution

One of the reasons why this error would be thrown is if the receive connectors configured on Exchange 2016 are not allowing Exchange 2010 servers to relay email through them.  To correct the problem, either locate an existing or create a new FrontendTransport receive connector and configure the following permissions:

Authentication:

• Transport Layer Security (TLS)
• Integrated Windows authentication
• Exchange Server authentication

Permissions Group

• Exchange servers
• Legacy Exchange servers

Ensure that this receive connector’s Remote network settings is configured to allow the IP addresses of the Exchange 2010 hub transport servers.

With the above configured, the queued up mail on the Exchange 2010 servers should retry with success.

I’ve recently been involved with configuring a client’s Citrix NetScalers to load balance inbound SMTP connections to Exchange and thought I’d take this opportunity to blog the process.

#1 – Configure Exchange Server Objects

Begin by creating the Exchange Server objects in Traffic Management > Load Balancing > Servers:

#2 – Create SMTP Monitor

Create an SMTP monitor object by navigating to Traffic Management > Load Balancing > Monitors:

Name: EXCH_MONITOR_SMTP

Type: SMTP

Interval: 5 second

Response Time-out: 2 second

Destination Port: Bound Service

Down Time: 30 second

Click on the Special Parameters tab and configure the following:

Script Name: nssmtp.pl

Dispatcher IP: 127.0.0.1

Dispatcher Port: 3013

Note that the nssmtp.pl script bundled with the NetScaler will go as far as attempting to open a connection to confirm that the service is up.  The script and the actual code can be found in the following directory of the NetScaler:

/netscaler/monitors

#!/usr/bin/perl -w

################################################################

##

## Copyright 1998-2016 Citrix Systems, Inc. All rights reserved.

## This software and documentation contain valuable trade

## secrets and proprietary property belonging to Citrix Systems, Inc.

## None of this software and documentation may be copied,

## duplicated or disclosed without the express

## written permission of Citrix Systems, Inc.

##

################################################################

## This is a netscaler supplied script. Please dont modify this as it will be overwritten during

## reboot. If you want to modify, please make a copy of this script and modify.

## This script is used to do smtp monitoring using KAS feature.

use strict;

use Net::SMTP;

use Net::SMTP6;

use Netscaler::KAS;

sub is_ipv4_address

{

my $address = $_[0];

if ($address =~ m/^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/) {

return 1;

}

return 0;

}

## This function is a handler for performing smtp probe in KAS mode

sub smtp_probe

{

## There must be at least 4 arguments to this function.

## 1. First argument is the IP that has to be probed.

## 2. Second argument is the port to connect to.

## 3. Timeout, it is present in index 3

if(scalar(@_) < 2)

{

return (1,"Invalid number of arguments");

}

## Try to connect to the server

my $smtp;

if (is_ipv4_address($_[0])) {

$smtp=Net::SMTP->new($_[0].":".$_[1],Timeout=>$_[3])

or return (1,"Unable to connect to server - $!");

## Probe succeeded

$smtp->quit;

return 0;

}

else { #IPV6 adress

$smtp=Net::SMTP6->new($_[0], PeerPort => $_[1], Timeout=>$_[3])

or return (1,"Unable to connect to server - $!");

## Probe succeeded

$smtp->quit;

return 0;

}

}

## Register smtp probe handler, to the KAS module.

probe(\&smtp_probe);

#3 – Configure Service Group

Proceed to configure the Service Group object by navigating to Traffic Management > Load Balancing > Service Groups:

Name: Exchange_2016_SMTP

Protocol: TCP

Cache Type: SERVER

Click on the No Service Group Member to add the Exchange server objects that were created in Step #1:

Click on the Monitors option on the right to add the SMTP monitor created in Step #2:

Complete the creation of the Service Group and you should now see the group listed with the State and EffectiveState as being up:

Step #4 – Create the Load Balancing Virtual Server

Continue to configure the load balancing virtual server object by navigating to Traffic Management > Load Balancing > Virtual Servers:

Add the Service Group created in Step #3:

Complete the creation of the load balancing virtual server and you should see State and Effective State listed as being up:

Step #5 – Lockdown Open Relay for Exchange Receive Connector

One of the common mistakes often overlooked when configuring SMTP load balancing via the NetScaler is inadvertently allowing open relay on the Exchange Server’s receive connector traffic coming from the NetScaler would appear to be an internal IP to the Exchange server.  One of the ways to test whether the receive connector allows for open relay is to execute the following commands via telnet:

telnet exchangeServerFQDN 25

220 EXMB02.contoso.com Microsoft ESMTP MAIL Service ready at Thu, 1
2 Jan 2017 14:20:03 -0400
ehlo bogus.com
250-EXMB02.contoso.com Hello [10.21.1.32]
250-SIZE 37748736
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-8BITMIME
250-BINARYMIME
250 CHUNKING
mail from:bogus@bogus.com
250 2.1.0 Sender OK
rcpt to:validperson@domain.com
250 2.1.5 Recipient OK

Note that the mail from email address has a domain that is not hosted on the Exchange server and the rcpt to address is meant to be an email address that is also not hosted on the Exchange server.  If the response to these commands is Recipient OK then your receive connect is allowing open relay.  To correct this, ensure that the receiving connector has the Externally secured (for example, with IPsec) setting disabled:

Once the connect has been locked down, the following response is what the telnet commands would yield:

220 EXMB01.contoso.com Microsoft ESMTP MAIL Service ready at Thu, 1
2 Jan 2017 14:15:47 -0400
ehlo bogus.com
250-EXMB01.contoso.com Hello [10.21.1.32]
250-SIZE 37748736
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-8BITMIME
250-BINARYMIME
250 CHUNKING
mail from:bogus@bogus.com
250 2.1.0 Sender OK
rcpt to:validperson@google.com
550 5.7.54 SMTP; Unable to relay recipient in non-accepted domain

Step #6 – Lockdown SMTP Load Balancing Virtual Server Connectivity

Another often overlooked issue that load balancing SMTP requests through a NetScaler creates is that the Exchange server’s receive connectors no longer see the true source IP address because all of the requests now originate form the NetScaler’s NSIP address which means a malicious or unauthenticated internal device could potentially relay mail off of the load balancing virtual server and be able to successfully have the Exchange server deliver the email.  This could be addressed by either configuring the Direct Server Return (DSR) feature on the NetScaler or simply locking down which IP addresses can connect to the load balancing virtual server.  I won’t cover the configuration of DSR and will point to one of my previous blog posts to demonstrate how to lock down the load balancing virtual server:

Filtering a Citrix NetScaler load balancing virtual server access based on source IP address
http://terenceluk.blogspot.com/2017/01/filtering-citrix-netscaler-load.html

Those who are familiar with the Citrix NetScaler’s administrative console would be familiar with the upgrade button in the Systems menu that allows the administrator to upload the upgrade package and have the appliance automatically apply the firmware update:

While this feature makes the upgrade process quite easy, I’ve also found that it is sometimes unreliable because the upgrade progress status window could freeze at a certain step and not update which then leaves us wondering if we should close and start over or continue waiting.  What I’ve typically done in the past is manually apply the update by uploading the package onto the appliance and using command line to execute the upgrade either via the console or SSH session.

Step #1 – Download and upload firmware package

Begin by logging onto the the Citrix website and download the upgrade package normally named similar to the following:

build-11.1-51.21_nc.tgz

Launch your preferred SFTP client such as WinSCP, connect to the secondary appliance and navigate to the following directory:

/var/nsinstall

Create a directory for the new package and copy the firmware into the directory:

Repeat the same for the active appliance.

Step #2 – Backup and save the NetScaler configuration

Log onto the active NetScaler’s administration console and proceed to backup and save the configuration:

A hypervisor snapshot could also be created as well.

Step #3 – Unpack and install firmware upgrade

With the NetScaler backed up and upgrade firmware package uploaded, proceed with accessing the console or opening an SSH session to the secondary node, enter the shell mode by executing shell, navigate to the /var/nsinstall/<firmwareUpdate> directory then execute the following to extract the package:

tar -zxvf ns-x.0-xx.x-doc.tgz

For this example, the command to execute would be:

tar -zxvf build-11.1-51.21_nc.tgz

Once the files have been extracted, proceed to install by executing:

./installns

Continue and restart the appliance once the installation has completed.

Step #4 – Confirm HA status, confirm synchronization is disabled and force failover

With the secondary appliance upgraded and restarted, log back into the console and execute show ver to confirm that the version has been upgraded:

Continue and review the HA status by executing:

show ha node

… to confirm that this node is listed as secondary and synchronization is disabled:

**Note that both Sync State and Propagation is currently configured as AUTO DISABLED because build 51.21 automatically disables these settings during the upgrade.

In the event that synchronization is not disabled, execute the following to disable it:

set node -hasync disable

Execute show ha node again to confirm the status then force the failover:

force failover

Step #5 – Upgrade primary NetScaler node

Repeat the steps outlined in #3:

With the NetScaler backed up and upgrade firmware package uploaded, proceed with accessing the console or opening an SSH session to the secondary node, navigate to the /var/nsinstall/<firmwareUpdate> directory then execute the following to extract the package:

tar -zxvf ns-x.0-xx.x-doc.tgz

For this example, the command to execute would be:

tar -zxvf build-11.1-51.21_nc.tgz

Once the files have been extracted, proceed to install by executing:

./installns

Continue and restart the appliance once the installation has completed.

Step #6 – Verify upgrade of appliances and failover to original primary node

With the second appliance upgraded, proceed by logging onto the appliance and execute show ver to confirm that the version has been upgraded:

Execute the following command to check the status:

show ha node

Proceed by failing back the primary role back to the previous primary appliance with the command:

force failover

Step #7 – Enable synchronization on secondary appliance

Log onto the secondary appliance and execute the following to verify it is in secondary state:

show node

**Note that both Sync State is labeled as SUCCESS and Propagation is labelled as ENABLED because build 51.21 automatically enables these settings after the upgrade.

If Sync State and Propagation is not enabled then execute the following command to enable synchronization:

set node -hasync enable

Execute the following command to verify that the configuration of the secondary appliance is synchronized with that of the primary appliance

show ns runningconfig

Problem

You’ve completed an upgrade of VMware App Volumes Manager from 2.6.0.1226 to 2.12.0.70:

… by uninstalling the old version then reinstalling the new version using the same database but notice that launching the Apps Volume Manager now display the following HTTP 500 Internal Server Error:

Reviewing the logs on the App Volumes Manager server in the folder:

C:\Program Files (x86)\CloudVolumes\Manager\log

Reveals quite a few large 200MB+ logs:

Opening these logs file in Notepad++ shows the following error repeatedly logged:

[2017-01-17 00:25:04 UTC] P2236R698508  INFO Started GET "/cv_api/version" for 127.0.0.1 at 2017-01-16 20:25:04 -0400

[2017-01-17 00:25:04 UTC] P2236R698508  INFO Processing by CvApi::VersionsController#show as */*

[2017-01-17 00:25:04 UTC] P2236R698508 ERROR    Manager: Unhandled request exception: ODBC::Error: 42S02 (208) [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'ldap_domains'.: EXEC sp_executesql N'SELECT TOP (1) 1 AS one FROM [ldap_domains]'

[2017-01-17 00:25:04 UTC] P2236R698508 ERROR    Manager: Inspecting Array (21028620) (from log block)

Solution

I’m not much of an expert with App Volumes so I called support to have an engineer review the logs and was told that the line:

[2017-01-17 00:25:04 UTC] P2236R698508 ERROR    Manager: Unhandled request exception: ODBC::Error: 42S02 (208) [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'ldap_domains'.: EXEC sp_executesql N'SELECT TOP (1) 1 AS one FROM [ldap_domains]'

… indicates that a table in the SQL database named ldap_domains was missing.  Reviewing the database via Microsoft SQL Server Management Studio confirms the missing table:

The engineer classified this as a catastrophic error during the upgrade and said that it looks like many of the other tables were not created during the upgrade.  He wasn’t completely sure but the issue may have been due to the major jump in version and the schema changes between the versions.  At this point, the suggestion was to restore a database backup and attempt to reinstall or reinstall with a new database and reconfigure the configuration with the existing AppStacks.  This environment was only used by 20 users so I decided to perform a reinstall and reconfigure.

To ensure that the previous created AppStacks are imported, ensure that you select the same datastore containing all the AppStacks during the initial configuration after the new install:

Not exactly the best solution but since I’ve configured Active Directory groups to grant access to the applications, it was easier for me to go this route than try to perform a restore and then upgrade again:

Problem

Companies federated with your Lync / Skype for Business environment have recently noticed that the status contacts in your organization briefly show as Updating…:

… then eventually changes to Presence unknown:

You’ve confirmed that the services on the Edge server are all Running:

However, reviewing the event logs show numerous errors logged in reference to the Lync Standard Server:

Log Name: Lync Server

Source LS Web Conferencing Edge Server

Event ID: 41987

Level: Error

Web Conferencing Server connection failed to establish.

Over the past 30 minutes Lync Server has experienced incoming TLS connection failures 120 time(s). The error code of the last failure is 0x80096004 (The signature of the certificate cannot be verified.

) and the last connection was from the host "".

Cause: This can occur if this box is not properly configured for TLS communications with remote Web Conferencing Server.

Resolution:

Check your topology configuration to ensure that both this host and remote Web Conferencing Server can validate each other TLS certificates and are otherwise trusted for communications.

Log Name: Lync Server

Source LS Protocol Stack

Event ID: 14428

Level: Error

TLS outgoing connection failures.

Over the past 86 minutes, Lync Server has experienced TLS outgoing connection failures 15 time(s). The error code of the last failure is 0x80096004(TRUST_E_CERT_SIGNATURE) while trying to connect to the server "svrlyncstd02.domain.internal" at address [10.1.1.66:5061], and the display name in the peer certificate is "Unavailable".

Cause: Most often a problem with the peer certificate or perhaps the host name (DNS) record used to reach the peer server. Target principal name is incorrect means that the peer certificate does not contain the name that the local server used to connect. Certificate root not trusted error means that the peer certificate was issued by a remote CA that is not trusted by the local machine.

Resolution:

Check that the address and port matches the FQDN used to connect, and that the peer certificate contains this FQDN somewhere in its subject or SAN fields. If the FQDN refers to a DNS load balanced pool then check that all addresses returned by DNS refer to a server in the same pool. For untrusted root errors, ensure that the remote CA certificate chain is installed locally. If you have already installed the remote CA certificate chain, then try rebooting the local machine.

Log Name: Lync Server

Source LS Protocol Stack

Event ID: 14366

Level: Error

Multiple invalid incoming certificates.

In the past 480 minutes the server received 30 invalid incoming certificates. The last one was from host 10.1.1.66.

Cause: This can happen if a remote server presents an invalid certificate due to an incorrect configuration or an attacker.

Resolution:

No action needed unless the number of failures is large. Contact the administrator of the host sending the invalid certificate and resolve this problem.

Attempting to browse the the FQDN of the Lync Standard Server from the Edge server displays the following webpage with a certificate warning:

· If you arrived at this page by clicking a link, check the website address in the address bar to be sure that it is the address you were expecting.

· When going to a website with an address such as https://example.com, try adding the 'www' to the address, https://www.example.com.

For more information, see "Certificate Errors" in Internet Explorer Help.

Solution

One of the reasons why presence would stop working and the event ID errors above to be thrown is if the Lync Edge server’s internal network interface certificate was recently updated by the issuing Root certificate issuing the updated certificate is not installed onto the Lync Edge server.  Note that the Lync Edge server is never joined to the domain so if the internal network interface certificate is issued by an internal CA then the root CA certificate along with the chain must be manually imported into the Trusted Root Certification Authorities certificate store on the Edge server:

Once this has been completed, proceed to restarting the Lync Edge server’s services and confirm that the following informational event logs are written:

Verify that the following page is now displayed when you browse the the FQDN of the Lync Standard Server from the Edge server:

Server Error

403 - Forbidden: Access is denied.

You do not have permission to view this directory or page using the credentials that you supplied.

Problem

You’ve downloaded Windows Essential 2012 and installed Windows Movie Maker 2012 on a Windows 7 desktop fails to launch and displays the following error message:

Sorry, Movie Maker can’t start. Make sure your computer meetings the minimum system requirements before trying to start Movie Maker again, and then try to update the driver for your video card if Movie Maker still doesn’t start.

Searching through the internet leads you to the following Microsoft KB:

You cannot start Windows Movie Maker 2012 when a graphics card that only supports DirectX 9 is installed on a Windows 7 or Windows Server 2008 R2-based computer
https://support.microsoft.com/en-us/help/2741355/you-cannot-start-windows-movie-maker-2012-when-a-graphics-card-that-only-supports-directx-9-is-installed-on-a-windows-7-or-windows-server-2008-r2-based-computer

… but attempting to install the update displays the following message:

The update is not applicable to your computer.

Downloading other video editing applications such as Shotcut also fails to launch indicating the video card does not support OpenGL.

Solution

In order to get the Windows 7 virtual desktop to successfully launch Windows Movie Maker, 3D support needs to enabled as shown in the following virtual machine properties:

Video card

3D graphics

Enable 3D support

It is important to note that VMware Horizon View desktops cannot be simply powered off and have the Video Card settings configured because if the pool properties are not configured to enable 3D then the settings of the virtual desktop would eventually be reverted back.

Proceed by logging onto the View administration console, open the desktop pool properties, navigate to the Remote Display Protocol tab and review the following settings:

Default display protocol: PCoIP

Allow users to choose protocol: Yes

3D Renderer: Disabled

If the Allow users to choose protocol setting is configured as Yes then 3D support would be disabled:

In order to enable 3D support, we’ll need to ensure that the following settings are configured:

Remote Display Protocol

Default display protocol: PCoIP

Allow users to choose protocol: No

With the Allow users to choose protocol configured as No, we will now be change to change the 3D Renderer setting:

Proceed and change the settings to the following:

Remote Display Protocol

Default display protocol: PCoIP

Allow users to choose protocol: No

3D Renderer: Automatic

The VRAM size can be adjusted from the default if required:

VRAM Size: 96MB

The last important step is to completely shut off the desktop so that View would send the commands to vCenter to reconfigure the VM.  A simple restart or reset of the virtual desktop will not change the configuration so make sure you initiate a full guest shutdown.  Once 3D support is enabled, both Windows Movie Maker (DirectX 11) and Shotcut (OpenGL) would now launch:

I’ve noticed that many of my clients and colleagues have been calling me about a specific step during the migration of Exchange 2010 Public Folders to Exchange 2016.  Given the frequency of the calls I’ve gotten, I thought it would be a good idea to write this short blog post about it.

Problem

You’re in the process of migration from Exchange Server 2010 to 2016 and have gotten to the public folder portion of the migration.  The following TechNet article is what you are using for the migration:

Migrate public folders from Exchange 2010 to Exchange 2016
https://technet.microsoft.com/en-us/library/mt463355(v=exchg.150).aspx

You’ve been able to execute all of the steps but notice that as you approach Part 4 and attempt to execute Create-PublicFolderMailboxesForMigration.ps1 to create target public folder mailboxes on Exchange 2016:

… the cmdlet fails with the following error:

[PS] C:\PFMigration>.\Create-PublicFolderMailboxesForMigration.ps1 -FolderMappingCsv PFMailboxes.csv -EstimatedNumberOfC

oncurrentUsers:100

C:\PFMigration\Create-PublicFolderMailboxesForMigration.ps1 : Existing Public Folder deployment is not locked for migra

tion. The script cannot continue unless all Public Folder mailboxes are deleted first. Please, make sure the existing m

ailboxes have no data before deleting them.

At line:1 char:47

+ .\Create-PublicFolderMailboxesForMigration.ps1 <<<< -FolderMappingCsv PFMailboxes.csv -EstimatedNumberOfConcurrentUs

ers:100

+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException

+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Create-PublicFolderMailboxesForMigrati

on.ps1

[PS] C:\PFMigration>Get-PublicFolder

Solution

One of the most common reasons I find administrators encounter this error during the public folder migration process is because they are executing the Create-PublicFolderMailboxesForMigration.ps1 cmdlet on the Exchange 2010 server.  All the steps prior to Part 4 is usually executed on the Exchange 2010 server so some administrators forget that they need to execute this cmdlet to create new Exchange 2016 public folders on the new Exchange 2016 Management Shell instead:

Executing the Create-PublicFolderMailboxesForMigration.ps1 cmdlet should complete successfully and output the following:

PS] C:\PFMigration>.\Create-PublicFolderMailboxesForMigration.ps1 -FolderMappingCsv PFMailboxes.csv -EstimatedNumberOfC

oncurrentUsers:100

Do you want to run software from this untrusted publisher?

File C:\PFMigration\Create-PublicFolderMailboxesForMigration.ps1 is published by CN=Microsoft Corporation, OU=MOPR,

O=Microsoft Corporation, L=Redmond, S=Washington, C=US and is not trusted on your system. Only run scripts from trusted

publishers.

[V] Never run [D] Do not run [R] Run once [A] Always run [?] Help (default is "D"): A

Creating a new session for implicit remoting of "Get-OrganizationConfig" command...

Public Folder mailbox updates.

Creating 5 Public Folder mailbox(es) and updating 0. Total mailboxes to serve hierarchy will be 1. Would you like to

proceed?

[Y] Yes [N] No [?] Help (default is "Y"): Y

Total mailboxes created: 5. Total mailboxes updated: 0. Total serving hierarchy: 1.

Here is a list of Public Folder mailboxes created:

Name IsServingHierarchy IsMigrationTarget

---- ------------------ -----------------

Mailbox1 False True

Mailbox2 True True

Mailbox3 False True

Mailbox4 False True

Mailbox5 False True

[PS] C:\PFMigration>

Quite obvious when you realize it but we all tend to get a bit lost when we’re too immersed into a deployment at times.

Problem

You attempt to create a CSR on Avaya Session Border Controller for Enterprise 6.3 000-19-4338:

… but notice that you are unable to generate the CSR if the Subject Alt Name is left blank:

The Subject Alt name field can not be empty.

Attempting to fill it in with the Common Name fails with:

Subject Alt Name is not properly formatted. See here for more information.

Solution

The reason why the above 2 errors are thrown is because the entry is not supposed to be the FQDN as most Windows administrators are used to but rather an IP address and SIP domain.  The Avaya Session Border Controller should have 2 interfaces, 1 for internal and 1 for external so depending which interface this CSR is for, the format should either be:

IP:1.1.1.1,DNS:external.domain.com

… or:

IP:10.1.1.1,DNS:internal.domain.com

**Note that the certificate for the public interface should be have the public address accessible from the internet and not the internal NAT address.

The CSR should complete once this has been corrected:

Problem

You’re attempting to upgrade your VDP appliance from 6.1.2 to 6.1.3:

… but notice the following message after mounting the upgrade ISO and navigating to the Upgrade tab:

To upgrade your VDP appliance, please connect a valid upgrade ISO image to the appliance.

You’ve confirmed that the upgrade ISO is not corrupted and can see the files in the file when using utilities such as WinRAR to browse it.

Solution

A quick search on the internet appears to suggest that this has been a problem since version 6.1 of the VDP appliance and the way to get around this issue is to manually mount the ISO via commands through an SSH session.  The following are two posts that I found helpful:

VMware vSphere Data Protection – Upgrade 6.1.2 to 6.1.3 ISO not detected
http://www.stephenwagner.com/?p=1107

ISO Package Not Available During VDP Upgrade From 6.1
http://www.virtuallypeculiar.com/2016/12/iso-package-not-available-during-vdp.html

My first attempt to resolve the issue was to use the first post but while using VI to edit the /etc/auto.mnt file to mount the ISO worked to get the ISO mounted for the install, it was too cumbersome to repeat during the install when you had seconds to remount the ISO because it gets dismounted.  The single line command supplied in the second post was much easier.  The following are steps to perform the upgrade:

Step #1 – Backup VDP Appliance

It is extremely important to backup the appliance before performing the upgrade appliance as every failed upgrade I experienced rendered the appliance unusable so begin by shutting down the VDP appliance and change ALL the disks aside from Disk 1 to Dependent – Dependent disks are included in snapshots:

Once the disks have been changed to Dependent mode, snapshot the virtual machine to create a rollback point.

Step #2 – Power on VDP Appliance and Mount ISO

Ensure that the upgrade ISO is uploaded to a datastore to avoid an upgrade failure due to mounting the ISO through a remote console and either the desktop you’re upgrading with crashes or vCenter gets restarted causing the ISO to disconnect from the VM.

SSH to the VDP appliance’s IP address and execute the command:

df -h

Note the lack of the device /dev/sr0 in the screenshot below:

Mount the attached ISO by executing the command:

mount /dev/sr0 /mnt/auto/cdrom

Note the confirmation:

mount: block device /dev/sr0 is write-protected, mounting read-only

Executing df -h will now display the line item:

/dev/sr0 5.1G 5.1GB 0 100% /mnt/auto/cdrom

Browsing the directory with the commands:

cd /mnt/auto/cdrom

ls

Will display the contents of the upgrade ISO:

vSphereDataProtection-6.1.3.avp

version.properties

Step #3 – Start the Upgrade

Viewing the VDP Upgrade tab on administration console at https://<VDPapplianceIP>:8543/vdp-configure/ will now display the following:

Package verification in progress. This may take a few minutes…

The page should display the upgrade package after a few minutes:

VSphereDataProtection7280

Status: ready

Priority: normal

Version: 6.1.3.70

Start the upgrade:

Step #4 – Prepare to remount ISO at 71%

This step is extremely important because it takes quite a bit of time for the upgrade process to reach 71% which will eventually dismount the ISO.  Failure to remount the ISO within the 10 second span would cause the upgrade to fail and the appliance to be unusable requiring you to revert back to the previous snapshot.

Copy the following command to prepare to remount the ISO:

mount /dev/sr0 /mnt/auto/cdrom

Monitor the progress bar and as soon as it reaches 71%:

… execute df -h to determine whether the ISO has been dismounted and as soon as it has, execute the command to mount the ISO.  Note that the command may appear to fail the first few times but keep repeating the execution and you will eventually mount the ISO:

mount: mount point /mnt/auto/cdrom does not exist

The installation should continue to 72% if you have mounted the ISO in time:

At around 84%, the console would go back to the login page allowing you to log back in showing services are stopped and ISO not attached in the Upgrade tab but executing df -h in the SSH you have opened indicates it still is.  Reviewing the console of the VDP doesn’t show any changes:

Wait a bit longer and you will eventually get kicked out again and this time logging in will show more services started with the ISO attached again.  vCenter should show tasks performed on the as well:

From this point, give appliance maybe 10 more minutes just to be safe and proceed to reboot it.  You should be able to successfully connect to the appliance via the vSphere Web Client and see the version is now 6.1.3 in the console:

Step #5 – Verify VDP and delete snapshots

Proceed to verify that VDP appliance to ensure that it operates as expected, then proceed to delete the snapshots and change the disks back to Independent – Persistent.

I recently had to assist a client with migrating their Exchange Server 2010 public folders to Exchange Server 2016 and ran into a situation where the mailbox database storing the public folder mailboxes would throw the following error message when I attempt to delete it via the GUI or PowerShell:

This mailbox database is associated with one or more active PublicFolderMailboxMigration requests. To get a list of
all PublicFolderMailboxMigration requests associated with this database, run Get-PublicFolderMailboxMigrationRequest |
?{ $_.RequestQueue -eq "<Database ID>" }. To remove a PublicFolderMailboxMigration request, run
Remove-PublicFolderMailboxMigrationRequest <Recipient ID\Request Name>.
    + CategoryInfo          : InvalidOperation: (empfdb01:DatabaseIdParameter) [Remove-MailboxDatabase], AssociatedMRS
   RequestExistsException
    + FullyQualifiedErrorId : [Server=BMEXMB01,RequestId=65decc2b-2925-48cf-91d0-aba27ab1329f,TimeStamp=1/30/2017 1
   :20:19 PM] [FailureCategory=Cmdlet-AssociatedMRSRequestExistsException] A32E1F9E,Microsoft.Exchange.Management.Sys
  temConfigurationTasks.RemoveMailboxDatabase
    + PSComputerName        : bmexmb01.contoso.com

After ensuring that there were no Public Folder Mailbox Migration requests by executing the Get-PublicFolderMailboxMigrationRequest cmdlet, I did a quick search on the internet and found the following post:

https://social.technet.microsoft.com/Forums/office/en-US/54fd0db4-11d3-421c-92e8-d4050338a907/trouble-removing-2016-mailbox-database?forum=Exch2016Adm

… where a person indicated that there may be a lingering object in the Configuration container that is not returned by the PowerShell cmdlets.  I then went ahead and launched ADSIedit, connected to the Configuration container then browsed to:

Configuration > Services > Microsoft Exchange > ExchOrganization > Mailbox Replication displayed the following:

The node that appeared to be out of place was the one named:

CN=PublicFolderMailboxMigrationRequestsCNF:11714980-d0a3…

Browsing into that node displayed the following items:

Opening these items showed that they were objects that corresponded to the day I had started the migration batch for the public folder migration so I went ahead and deleted these items in the folder then forced replication via repadmin /syncall /AdeP on a domain controller and was then able to delete the mailbox database.

I’ve found that many of my clients with services that rely on Microsoft Certificate Authorities deployed within the internal network have frequently asked me whether there was a way to monitor the expiry of these issued certificates and the answer to that is yes, with the Certificate Expiration Alerting tool found here:

Certificate Expiration Alerting
https://blogs.technet.microsoft.com/nexthop/2011/11/17/certificate-expiration-alerting/

The next common question that usually pops up shortly after testing the tool is whether there was a way to filter out issued certificates that have blank common names as shown in the following screenshot:

CertExpAlerter.exe -c "cert01\Company-CA" -d 312

Note that the command above queried for certificates that expire in 312 days and 3 certificates were returned where 2 had blank common names.  The way to filter the common name as described in the TechNet article is with the use of RegEx and the only reason why I am familiar to it is because I used to work with Lync Enterprise voice quite a bit which forced me to learn it for creating translation rules. The RegEx expression we’re interested in is the following:

^(?!\s*$).+

What the above RegEx command matches is any string that contains at least one non-space character which results with the exclusion of blank common names:

Hope this helps anyone who is unfamiliar with RegEx and is looking for the expression to filter out blank common names.

Problem

You’ve created a new scheduled task in Task Scheduler to execute a batch file but notice that the task does not complete successfully and the Last Run Result is (0x1):

Reviewing the History tab of the scheduled task shows the following log entries:

Level: Information

Task Category: Action completed

Operation Code: (2)

General:

Task Scheduler successfully completed task "\Microsoft\Daily Certificate Expiry Notification" , instance "{ae87616f-a564-4c34-ab21-0a7f7cc2a99c}" , action "C:\Windows\SYSTEM32\cmd.exe" with return code 2147942401.

Level: Information

Task Category: Task completed

Operation Code: (2)

General:

Task Scheduler successfully finished "{ae87616f-a564-4c34-ab21-0a7f7cc2a99c}" instance of the "\Microsoft\Daily Certificate Expiry Notification" task for user "NT AUTHORITY\NETWORK SERVICE".

Solution

One of the possible causes to this issue is if the Start in (optional): of the configured actionis not filled out. To correct this, open the properties of the scheduled task, navigate to the Actions tab and then edit the action:

Notice that the Start in (optional): field is not filled in:

Fill in Start in (optional): with the path to the batch file:

With the above in place, the job should now run with the Last Run Result as The operation completed successfully. (0x0):

Problem

You’ve recently upgraded vCenter 5.5 from Update 3c to 3e and immediately noticed that your existing VDP 6.1.2.19 appliance backup jobs fail and the closest KB you can find related to this issue is the following:

825Communication fails in vSphere Data Protection with vCenter Server 5.5 Update 3 (2146825)https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2146825

However, attempting to download VDPHotfix.sh, uploading it to the VDP appliance and applying the hotfix does not appear to work:

login as: root
Using keyboard-interactive authentication.
Password:
Last login: Wed Jan 25 12:20:34 2017
root@vdp1prd:~/#: cd /home
root@vdp1prd:/home/#: cd admin
root@vdp1prd:/home/admin/#: ls
0.0           autorestart  cert.pem  getnodelogs  gsan.out  key.pem    truncate
VDPHotfix.sh  bin          cgsan     gsan         hfsclean  timerange
root@vdp1prd:/home/admin/#: .VDPHotfix.sh
-bash: .VDPHotfix.sh: command not found
root@vdp1prd:/home/admin/#: ./home/admin/VDPHotfix.sh
-bash: ./home/admin/VDPHotfix.sh: No such file or directory
root@vdp1prd:/home/admin/#: ls
0.0           autorestart  cert.pem  getnodelogs  gsan.out  key.pem    truncate
VDPHotfix.sh  bin          cgsan     gsan         hfsclean  timerange
root@vdp1prd:/home/admin/#: ./home/admin/VDPHotfix.sh
-bash: ./home/admin/VDPHotfix.sh: No such file or directory

Attempting to upload the full 2146825_vdp-hotfix.zip file onto the appliance and using the tar -zxvf command to extract the file does not work either:

admin@vdp1prd:/tmp/#: unzip 2146825_vdp-hotfix.zip
Archive:  2146825_vdp-hotfix.zip
  inflating: 2146825_vdp-hotfix.tar
admin@vdp1prd:/tmp/#: tar -zxvf 2146825_vdp-hotfix.
2146825_vdp-hotfix.tar  2146825_vdp-hotfix.zip
admin@vdp1prd:/tmp/#: tar -zxvf 2146825_vdp-hotfix.tar

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
admin@vdp1prd:/tmp/#: sh VDPHotfix.sh
sh: VDPHotfix.sh: Permission denied
admin@vdp1prd:/tmp/#: sh VDPHotfix.sh
sh: VDPHotfix.sh: Permission denied
admin@vdp1prd:/tmp/#: ls

Solution

Prior to calling VMware support, the only solution that worked for this environment where I encountered the problem was the workaround mentioned in the following forum post:

VDP 6.1.2 appliance can't connect to vCenter after initial configuration
https://communities.vmware.com/thread/542344?tstart=0

The workaround involved modifying the following file in the directory /usr/local/avamar/lib:

mcsutils.pm

… and adding the following line highlighted in red in between the two lines in black:

. "-Dfile.encoding=UTF-8 "
. "-Dsecurity.provider.rsa.JsafeJCE.position=last "
. "-Dlog4j.configuration=file://$mcsvar::lib_dir/log4j.properties "; # vmware/axis

Once this workaround was applied, the backup jobs began to run and complete.

Knowing that this workaround probably wasn’t the best solution, we opened a support call with VMware then was asked to upgrade the VDP appliance from 6.1.2 to 6.1.3 and noticed that the upgrade does indeed fix the issue.  I hope this post helps anyone who may come across this issue as I did not see an official KB article indicating upgrading was the solution.

One of the most common questions I’ve been asked by clients with Microsoft RDS deployments is how to hide the Library, Favorites and redirected local drives for RDS published RemoteApp applications.  The reason for this is because many administrators have noticed that users are unable to differentiate the Desktop, Downloads and Recent Places folders listed in the Save As dialog box for RemoteApp applications, which reside on the RDS server, and ones located on their local desktop:

This isn’t the users fault so to avoid this confusion, it is best to hide these folders and the following demonstrates how to accomplish this.

Hiding the Favorites Menu

Hiding the favorites menu as shown in the screenshot below:

… requires modifying registry keys as shown in the following:

For 32-bit applications

Navigate to the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Classes\CLSID\{323CA680-C24D-4099-B94D-446DD2D7249E}\ShellFolder]

Modify the DWord Attributes to a9400100:

For 64-bit applications

Navigate to the following key:

[HKEY_CLASSES_ROOT\CLSID\{323CA680-C24D-4099-B94D-446DD2D7249E}\ShellFolder]

Modify the DWord Attributes to a9400100:

Hiding the Libraries Menu

Hiding the libraries menu as shown in the screenshot below:

… requires modifying registry keys as shown in the following:

For 32-bit applications

Navigate to the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Classes\CLSID\{031E4825-7B94-4dc3-B131-E946B44C8DD5}\ShellFolder]

Modify the DWord Attributes to b090010d:

For 64-bit applications

Navigate to the following key:

[HKEY_CLASSES_ROOT\CLSID\{031E4825-7B94-4dc3-B131-E946B44C8DD5}\ShellFolder]

Modify the DWord Attributes to b090010d:

Hiding the Network Menu

Hiding the libraries menu as shown in the screenshot below:

… requires modifying registry keys as shown in the following:

For 32-bit applications

Navigate to the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Classes\CLSID\{F02C1A0D-BE21-4350-88B0-7367FC96EF3C}\ShellFolder]

Modify the DWord Attributes to b0940064:

For 64-bit applications

Navigate to the following key:

[HKEY_CLASSES_ROOT\CLSID\{F02C1A0D-BE21-4350-88B0-7367FC96EF3C}\ShellFolder]

Modify the DWord Attributes to b0940064:

Hiding the Favorites Menu

Hiding the redirected local drives as shown in the screenshot below:

… requires applying the following group policy configuration to the server object:

Computer Configuration > Policies > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Device and Resource Redirection

Enabling Do not allow drive redirection will prevent the local redirected drives from being accessible:

Problem

You attempt to export a user’s mailbox to PST via the Exchange 2013 ECP console but receive the following error:

The call to

‘net.tcp://exchangeServer.domain.com (15.0.1178.0caps:1FFF)’ timed out. Error details: This request operation sent to ‘net.tcp://exchangeServer.domain.com/Microsoft.Exchange.MicrosoftReplicationService did not receive a reply within the configured timeout (00:01:00). The time allotted to this operation may have been a portion of a longer timeout. This may be because the service is still processing the operation or becase the service was unable to send a reply message. Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.

Solution

I’ve had clients call me about this error message in the past and most of them would have Google-d the error and come across the following KB:

Mailbox import or export fails in Exchange Server 2010
https://support.microsoft.com/en-us/help/2675690/mailbox-import-or-export-fails-in-exchange-server-2010

Many other forum posts also suggests the two updating the RpcClientAccessServer to point to one of the CAS servers but if an attempt was made to do that with the cmdlet:

Set-MailboxDatabase “Mailbox Database Name” -RpcClientAccessServer casServer.FQDN.com

… you’ll quickly notice that this cmdlet no longer exists and the only way to change this attribute is to use ADSIedit.

What I’ve noticed is that there have been plenty of times where the above KB does not apply so prior to going down the route of either creating a host record or adjusting the RpcClientAccessServer variable, check to ensure that the path specified in the following step for the export is correct:

Export to a .pst file

*Specify the path to export the .pst file to (example: \\server\folder\ExportFile01.pst)

The error message above could also be thrown if you specify a path that is not reachable.

Problem

You have an account that was recently disabled in Active Directory and need to access it via OWA so you proceed to enable in Active Directory Users and Computers then log on via the webmail webpage but receive the following error:

:-(
something went wrong
Your account has been disabled.
X-ClientId: 19F7108344BF45D0BD829021E1B17519
X-OWA-Error: Microsoft.Exchange.Data.Storage.AccountDisabledException
X-OWA-Version: 15.0.1178.4
X-FEServer: EX04
X-BEServer: EX04
Date: 13/03/2017 14:13:58
Fewer details...

Solution

I’ve come across this quite a few times when clients have asked me to troubleshoot this and as obvious as the solution is, it is also very easy to overlook.  What I’ve noticed over the past year is that the majority of the time administrators run into this issue and are absolutely sure that the user’s AD account is enabled is when the following configuration in Exchange is disabled:

Email Connectivity

Outlook Web App: Disabled

Some organizations may disable this for various reasons.  Hope this helps anyone who may encounter this issue.