Problem

There are situations where an existing GPO is configured to restrict users from launching the MMC and the available snap-ins for non-administrative users to enhance security. To achieve this, an administrator can create a GPO with the following settings:

User Configuration > Administrative Templates > Windows Components > Microsoft Management Console > Restrict users to the explicitly permitted list of snap-ins:Enabled

Re-enabling select MMCs are fairly straight forward as a list of consoles are provided in the sub folder: Restricted/Permitted snap-ins

As the list provided in the default Group Policy ADM only includes default Windows Server snap-ins, attempting to add a MMC such as the following SQL Server 2919 Configuration Manager would become a problem:

Solution

There are two methods to address this issue. The first is to create a custom ADMX file and import it into Active Directory and the second is to apply the registry key that a custom ADMX would. This document serves to demonstrate the latter method as it is simpler for one-off configurations. To achieve this, we will need to create a GPO that will create a [HKCU]\Software\Policies\Microsoft\MMC\{MMC snap-in GUID} and set the Restrict_Run key to 0, which represents enable (1 is disabled):

Identify the custom MMC snap-in GUID

The first step is to identify the custom mmc snap-in GUID. For the purpose of this example, we will use the snap-in SQL Server 2919 Configuration Manager. Begin by navigating to:

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MMC\SnapIns\

Then locating and copying the custom snap-in’s GUID, which is the folder’s name:

{84a016c0-5617-4a05-ae8c-c806c5ff3e20}

Create a GPO with an update registry configuration

With the mmc snap-in GUID identified, proceed to create a GPO, navigate to User Configuration > Preferences > Windows Settings > Registry:

Create a registry configuration as such:

Action: Update
Hive: HKEY_CURRENT_USER
Key Path: Software\Policies\Microsoft\MMC\{84a016c0-5617-4a05-ae8c-c806c5ff3e20
Value name: Restrict_Run
Value type: REG_DWORD
Value data: 0

The policy should look as such once configured:

With the above policy created, have the user with this policy applied log off and back on to access the custom mmc snap-in.

I recently ran into an issue with a Citrix ADC / NetScaler NS13.0 36.27.nc after a reboot where the following error is displayed upon successfully authenticating:

Http/1.1 Internal Server Error 43531

The URL displayed ends with /cgi/dlge:

https://workspace.contoso.com/cgi/dlge

No configuration changes have been made for months. I combed through the configuration but could not determine why this error was being thrown so a ticket was opened with Citrix. The engineer went through the configuration and decided to change the Web Interface Address FQDNin the Citrix Gateway Session Profile to use the IP address instead of the DNS of the StoreFront server as shown in the screenshot below, which immediately corrected the issue:

We originally thought that there was something wrong with DNS but a DIG for the storefront.contoso.com URL returned the correct IP address for the Load Balancing Virtual Server that load balanced the two StoreFront servers:

root@CTXNETSCALER# dig storefront.contoso.com

; <<>> DiG 9.10.6 <<>> storefront.contoso.com

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31561

;; flags: qr aa rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 1280

;; QUESTION SECTION:

;storefront. contoso.com. IN A

;; ANSWER SECTION:

storefront. contoso.com. 3600 IN A 10.0.1.17

;; Query time: 0 msec

;; SERVER: 127.0.0.2#53(127.0.0.2)

;; WHEN: Tue Sep 29 19:25:36 UTC 2020

;; MSG SIZE rcvd: 69

root@CTXNETSCALER#

I haven’t gotten to the root cause of this issue but noticed that there were no recent posts for this error and thought I’d write a post in case someone else encounters this issue. We were told that an upgrade from the current version 13 Build 36.28 to version 13 Build 64.35 would resolve the issue so I will update this post when I determine whether it resolves the issue.

One of the more common questions I’ve been asked by colleagues and clients over the past few months is in regards to integrating Microsoft Teams with an on-premise Exchange mailbox so that users could access their calendars within the Teams client. Without a supported on-premise Exchange or hybrid configuration, the Teams client will not display the Calendar button. Given the frequency of this issue, I would like a write a quick post for items to check if the Calendar button is not present.

The following is the official documentation of the requirements to verify:

Requirements to create and view meetings for mailboxes hosted on-premises
https://docs.microsoft.com/en-us/microsoftteams/exchange-teams-interact#requirements-to-create-and-view-meetings-for-mailboxes-hosted-on-premises

If mailboxes are hosted on-premises, to create and view meetings, the following requirements must be met:

1. The required Teams license needs to be assigned for the Azure Active Directory synced user.
2. Users must be synchronized to Azure Active Directory. For information about how to use Azure AD Connect to synchronize with Azure Active Directory, see Hybrid identity documentation.
3. Mailboxes are hosted in Exchange Server 2016 Cumulative Update 3 or later.
4. Autodiscover and Exchange Web Services is published externally.
5. OAuth authentication is configured preferably via the Exchange Hybrid Configuration Wizard running a full hybrid configuration (Classic or Modern). If you are not able to use the Hybrid Configuration Wizard, configure OAuth as described here: https://docs.microsoft.com/en-us/exchange/configure-oauth-authentication-between-exchange-and-exchange-online-organizations-exchange-2013-help.

Note Exchange trusts OAuth Token from Teams service which is known as EvoSTS. Step 1 should be enough, but just the EvoSTS; ACS is used for Free/Busy lookup in the calendar.

1. The checkbox for the Exchange Hybrid Deployment feature in Azure AD Connect is set.
2. For calendar app support and Teams Outlook Add-In for Mac, Exchange Web Service URLs must be configured as SPNs in Tenant Azure AD for the Exchange Service Principal. This step is done with Hybrid Configuration Wizard or following manual steps for Hybrid Modern Authentication.

In short, verify the following:

1. A Teams license must be assigned to the user (this can’t be missed as the user wouldn’t be able to log on without one)
2. Verify that the on-premise account for the users are synced into Azure AD (do not have separate on-premise and cloud accounts or there would be no way for Teams to know they are linked).
3. I have not been able to get the calendar feature to work with any Exchange versions earlier than 2016 CU3 (check your Exchange version here: https://docs.microsoft.com/en-us/exchange/new-features/build-numbers-and-release-dates?view=exchserver-2019)
4. Autodiscover and EWS should already be published if mailbox via internet access is allowed but if it isn’t, make sure these services are published and tested with https://testconnectivity.microsoft.com/
5. There are two ways to configure OAuth authentication:

a) Use the Exchange Hybrid Configuration wizard via the on-premise Exchange EAC:

b) Or manually via this link: https://docs.microsoft.com/en-us/exchange/configure-oauth-authentication-between-exchange-and-exchange-online-organizations-exchange-2013-help

It is very unlikely that you need to use the link to manually configure it so use the Exchange Hybrid Configuration wizard to configure this.

6. Verify that the follow checkbox for the Exchange Hybrid Deployment feature in Azure AD Connect is enabled:

7. This step is the same as step #5 and I would recommend to use Exchange Hybrid Configuration wizard for the configuration.

I’ve found that the calendar button can take upwards to a few hours if I had just enabled the Exchange Hybrid Deployment for AD Connect so be patient. Once all the requirements above have been met, you should see the calendar button in the Teams client as shown below:

Problem

You have AD FS (Windows Server 2019) claims-based authentication configured for Exchange Server 2019 Version 15.2 (Build 464.5) OWA (Outlook on the Web) and has had it working in production for months but received complaints from users that the following error is presented after successfully authenticating at an AD FS portal to access OWA:

This page isn’t working
If the problem continues, contact the site owner.
HTTP ERROR 440

Attempting to access /ECP also throws the same error.

This appears to only happen in Chrome but not Internet Explorer or Chromium Edge (or regular Edge).

The following are versions of Chrome that presents the error above:

Version 85.0.4183.121 and 86.0.4240.75

The following are the version of IE and Edge Chromium that did not exhibit this issue:

Edge Chromium: 85.0.564.70

Internet Explorer 11: 11.1082.18362.0

Solution

A quick search for this issue on the internet returned the following thread on Reddit:

https://www.reddit.com/r/exchangeserver/comments/iyirls/owa_440_error_in_chrome/

It appears the quickest way to load the page in Chrome is to navigate to the following setting:

chrome://flags/#reduced-referrer-granularity

Then set the Reduce default ‘referrer’ header granularity. From default to disable:

This can also be configure via Group Policy via the registry as shown in the following document: https://cloud.google.com/docs/chrome-enterprise/policies?policy=ForceLegacyDefaultReferrerPolicy

Having worked with a lot of clients to implement missing security headers identified by the following popular scan by Scott Helme: https://securityheaders.com/, I felt that there must be a better way of addressing this at the AD FS server level rather than the client level. The Referrer Policy header was familiar to me because I had recently implemented for another client’s Citrix portal and for those who are not familiar with it can find more information at Scott Helme’s site: https://scotthelme.co.uk/a-new-security-header-referrer-policy/

The short description of it as described by Scott is: Referrer Policy is a new header that allows a site to control how much information the browser includes with navigations away from a document and should be set by all sites.

After reviewing the authentication and redirect process between the AD FS portal and OWA, then testing the various Referrer Policy options, it appears the most restrictive one that worked was the no-referrer-when-downgrade because the full URL was passed with this option.

The following is the official Microsoft documentation that explains how headers are configured for AD FS:

Customize HTTP security response headers with AD FS 2019
https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/operations/customize-http-security-headers-ad-fs

The document does not include the Referrer Policy in the list but you can view and configure it as shown in the following cmdlets executed on the AD FS server:

To view the response headers:

Get-AdfsResponseHeaders

To expand the truncated list of response headers:

PS C:\> Get-AdfsResponseHeaders | Select -ExpandProperty ResponseHeaders

Key Value
--- -----
Strict-Transport-Security max-age = 31536000
X-Frame-Options DENY
X-Content-Type-Options nosniff
X-XSS-Protection 1; mode=block
Content-Security-Policy default-src 'self''unsafe-inline''unsafe-eval'; img-src 'self'; frame-src *.duosecurity....

PS C:\>

To configure the Referrer Policy:

Set-AdfsResponseHeaders -SetHeaderName "Referrer-Policy" -SetHeaderValue "no-referrer-when-downgrade"

Confirm that the Referrer-Policy is configured with the following cmdlet:

Get-AdfsResponseHeaders | Select -ExpandProperty ResponseHeaders

Key Value
--- -----
Strict-Transport-Security max-age = 31536000
X-Frame-Options DENY
X-Content-Type-Options nosniff
X-XSS-Protection 1; mode=block
Content-Security-Policy default-src 'self''unsafe-inline''unsafe-eval'; img-src 'self'; frame-src *.duosecurity....
Referrer-Policy no-referrer-when-downgrade

To remove the Referrer Policy:

If the Referrer-Policy needs to be removed then the following cmdlet can be executed:

Set-AdfsResponseHeaders -RemoveHeaders "Referrer-Policy"

Problem

You’ve received reports that browsing a Citrix ADC / NetScaler published Exchange OWA displays the following warning message in Chrome and Edge Chromium:

Google Chrome

Your connection is not fully secure

This site uses an outdated security configuration, which may expose your information (for example, passwords, messages, or credit cards) when it is sent to this site.

NET::ERR_SSL_OBSOLETE_VERSION

Edge Chromium

Your connection isn't secure

This site uses an outdated security configuration that might expose your personal information when it's sent to this site (for example, passwords, messages, or credit cards).

NET::ERR_SSL_OBSOLETE_VERSION

It is possible to proceed in both browsers but a Not secure message will be displayed in the address bar:

Solution

One of the reasons why this warning message would be displayed is because Google Chrome 72 and later versions have deprecated support for legacy TLS versions, which include TLS 1.0 and 1.1. If the Load Balancing Virtual Server for the Exchange OWA configured on Citrix ADC / NetScaler has only TLS 1.0 and TLS 1.1 enabled as shown in the screenshot below then the warning message above will be displayed:

To correct the issue, simply enable TLSv12 in addition to TLSv1, TLSv11:

Or just enable TLSv12 if there are no legacy devices with browsers accessing the website (those should be upgraded if they exist):

Once updated, the site should load and the Security tab of Google Chrome’s Developer Tools should display a message indicating the site is using TLS 1.2:

Problem

You’re attempting to use the Set-MsolUser cmdlet to configure the immutableId attribute for a user in Azure Active Directory but receive the following error:

PS C:\> Set-MsolUser -UserPrincipalName jsmith@contoso.com -ImmutableId "zxGeOiOTdkivMtgkOsuvKA=="

Set-MsolUser : Uniqueness violation. Property: SourceAnchor.

At line:1 char:1

+ Set-MsolUser -UserPrincipalName jsmith@contoso.com -ImmutableId ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : OperationStopped: (:) [Set-MsolUser], MicrosoftOnlineException

+ FullyQualifiedErrorId : Microsoft.Online.Administration.Automation.UniquenessValidationException,Microsoft.Onlin

e.Administration.Automation.SetUser

PS C:\>

Reviewing the properties of the user account that you are trying to assign the immutableID value to confirms that it is null:

Get-MsolUser -UserPrincipalName jsmith@contoso.com | FL immutableId

Using Get-MsolUser to search for an account with the immutableID does not return any results:

Get-MsolUser | Where-Object {$_.ImmutableId -eq "zxGeOiOTdkivMtgkOsuvKA=="} | select UserPrincipalName

Solution

One of the most common reasons I’ve found for this error is if a deleted user object has the same immutableID assigned to it. A typical scenario would be:

1. An effort was made to merge on-premise Active Directory accounts with Azure AD but Azure AD Connect created a new account with a random number following the name rather than merge the two accounts
2. The administrator deletes the new account and attempts to assign the ObjectGUID (converted to base 64) of the on-premise Active Directory account to the Azure AD account

To confirm whether there is an account in the deleted users container, execute the following cmdlet:

Get-MsolUser -ReturnDeletedUsers

The following cmdlet can return the UPN along with the immutableID of the user accounts found in the deleted users container:

Get-MsolUser -ReturnDeletedUsers | FL UserPrincipalName,immutableID

Once the account with the conflicting immutableID is identified, the following cmdlet can be used to delete it:

Remove-MsolUser -UserPrincipalName jsmith@contoso.com -RemoveFromRecycleBin

With the account removed, you should now be able to assign the immutableID.

I recently had a client who used Office 365 for Exchange Online and Teams while also having an on-premise Active Directory but did not appear to have AD Connect deployed to synchronize their on-premise Active Directory accounts with their Office 365 Azure AD accounts. This became a problem when we started the WVD deployment so the client agreed to have us focus on synchronizing the two directories before commencing with other planned projects.

Azure AD Connect Soft Match vs Hard Match

As some may already know, there are two ways of matching accounts in on-premise Active Directory and Azure AD:

1. Soft Match
2. Hard Match

Soft Match uses the properties of userPrincipalName and ProxyAddresses to determine (evaluated by Azure AD) whether an account in the on-premise Active Directory can be merged with an Azure AD account. This essentially means that if an account is found to have the same userPrincipalName or the same primary SMTP address of ProxyAddresses, then AD Connect will merge the two accounts.

More information about ProxyAddresses matching with the primary SMTP address can be found here:

How to use SMTP matching to match on-premises user accounts to Office 365 user accounts for directory synchronization
https://support.microsoft.com/en-us/help/2641663/use-smtp-matching-to-match-on-premises-user-accounts-to-office-365

Hard Match uses the property sourceAnchor/immutableID to match the on-premise Active Directory account to the Office 365 account. The default sourceAnchor that Azure AD Connect uses for the on-premise Active Directory is the objectGUID property and the immutableID property is a value assigned to the Azure AD user account. To perform a Hard Match between two accounts, we would set the immutableID property of the existing user in Azure AD to the Base64 encoded string of the ObjectGUID of the user in the on-premise AD.

More information about soft and hard match can be found here:

Azure AD Connect: When you have an existing tenant
https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-install-existing-tenant#sync-with-existing-users-in-azure-ad

The Problem

The plan for merging the accounts between on-premise and the cloud was to use the soft match method and the initial test with a newly created account worked without any issues. However, proceeding to accounts that have existed in the on-premise Active Directory would consistently throw a DuplicateAttributes and AttributeValueMustBeUnique error in the AD Connect export job. After reviewing the KB article below without finding a resolution, I decided to proceed with the hard match method.

Troubleshooting Errors during synchronization
https://docs.microsoft.com/en-us/azure/active-directory/hybrid/tshoot-connect-sync-errors

The immediate issue I had with the hard match method was that it required a lot of manual labour to extract the ObjectGUID from the on-premise AD account, convert it to a Base64 encoded string, then import it into the Azure AD account’s immutableId attribute.

Extract Immutable ID

$UPN = "jsmith@contoso.com"

$user = Get-ADUser -Filter 'UserPrincipalName -eq $UPN'

$name = $user.name

$immutableid = [System.Convert]::ToBase64String($user.ObjectGUID.tobytearray())

Import Immutable ID

Set-MsolUser -UserPrincipalName jsmith@contoso.com -ImmutableId <convertedObjectGUID>

Having to perform the above for 50, 500 or 5000 and more accounts would not be efficient.

Solution

As with many laborious tasks in technology, the best way to minimize the amount of manual labour is to script the process and the following demonstrates this.

Step #1 – Extracting accounts in on-premise Active Directory

We will need to first obtain a list of users we’ll be hard matching with the accounts in Azure AD. If you are matching accounts that are nicely organized in OUs, you can use the following PowerShell cmdlet to export them into a CSV file:

Get-ADUser -Filter * -SearchBase "OU=Toronto,OU=Contoso-Users,DC=contoso,DC=com" -Properties * | Select-Object name,UserPrincipalName,emailaddress | export-csv -path c:\temp\Toronto.csv

The PowerShell cmdlet above will export all of the accounts in the specified OU with the fields:

1. Name
2. UserPrincipalName
3. emailAddress

Repeat the above for any other OUs that need to be matched.

Step #2 – Extracting accounts in Azure AD

The environment I worked in did not consistently have mirroring accounts in the on-premise Active Directory and Azure AD so I decided to export the list of accounts in Azure AD to compare in a spreadsheet:

Get-MsolUser | Select-Object DisplayName,UserPrincipalName,@{Name=’ProxyAddresses’;Expression={[string]::join(“;”, ($_.ProxyAddresses))}},ImmutableId | export-csv -path c:\temp\Toronto-MSOL.csv

Step #3 – Compare the accounts in on-premise AD and Azure AD

With the two CSV files extracted, I placed the UPNs into an Excel spreadsheet and used Home >Conditional Formatting > Highlight Cells Rules > Duplicate Values feature to identify which accounts have matching ones in the other directory.

Step #4 – Extract on-premise AD accounts’ ObjectGUID and Convert to ImmutableID

Now that we have a list of UPNs for users who we intend on matching from the on-premise Active Directory, proceed to create a txt file with just the UPNs separated by line breaks and use the following PowerShell script to extract the ObjectGUID then convert the value to Base 64 so it can be assigned as the immutableID:

foreach ($UPN in Get-Content C:\temp\TorontoAccounts.txt) {

$user = Get-ADUser -Filter 'UserPrincipalName -eq $UPN'

$name = $user.name

$immutableid = [System.Convert]::ToBase64String($user.ObjectGUID.tobytearray())

#$UPN,$name,$immutableid

#Store the information from this run into the array

[PSCustomObject]@{

UPN = $UPN

Name = $name

ImmutableID = $immutableId

} | Export-Csv C:\temp\UPNsWithImmutableID.csv -notype -Append

}

Step #5 – Import the UPNs with ImmutableIDs into Azure AD

The final step in this process is to use the CSV created in step #4 to assign the immutableIDs to the identified UPNs in Azure AD with the following script:

$onPremADUsers = Import-Csv c:\temp\UPNsWithImmutableID.csv

foreach ($row in $onPremADUsers)

{

#$row.UPN, $row.Name, $row.ImmutableID

Set-MsolUser -UserPrincipalName $row.UPN -ImmutableId $row.ImmutableID

}

The above will read the CSV file and assign the immutableID.

Step #6 – Confirm that the ImmutableIDs have been assigned

The last step, which is more of a verification step, is to review the UPNs that were assigned with the immutableIDs:

$onPremADUsers = Import-Csv c:\temp\UPNsWithImmutableID.csv

foreach ($row in $onPremADUsers)

{

Get-MsolUser -UserPrincipalName $row.UPN | Select-Object DisplayName,UserPrincipalName,ImmutableId

}

Hope this helps anyone who is looking for a quicker way of extracting and assigning sourceAnchor/immutableID so AD Connect can hard match accounts.

Problem

You’ve signed into Office 365 with the appropriate PowerShell modules and cmdlets:

Import-Module SkypeOnlineConnector

$sfbSession = New-CsOnlineSession

Import-PSSession $sfbSession

… and proceed to use the Set-CsUser cmdlet to enable a user for Teams Enterprise Voice but it fails with the following error:

PS C:\> Set-CsUser -Identity mesheiah@contoso.com -EnterpriseVoiceEnabled $true -HostedVoiceMail $true -OnPremLineURI tel:+7324

Management object not found for identity "mesheiah@contoso.com".

+ CategoryInfo : NotSpecified: (:) [Set-CsUser], ManagementException

+ FullyQualifiedErrorId : Microsoft.Rtc.Management.AD.ManagementException,Microsoft.Rtc.Management.AD.Cmdlets.SetO

csUserCmdlet

+ PSComputerName : admin0b.online.lync.com

The user being enabled for Teams Enterprise Voice has been previously enabled for an on-premise Skype For Business Enterprise Voice so you use the following cmdlet to list the uri properties and notice that the LineURI is still configured with the on-premise deployment:

PS C:\> Get-CsOnlineUser -Identity mesheiah@contoso.com | FL *uri

OnPremLineURI :

LineServerURI :

LineURI : TEL:+14165555555;ext=1324

Including the OnPremHostingProvider to the cmdlet will display the following SRV: value for it:

Attempting to use the Set-CsUser cmdlet to configure the LineURI to null will fail:

PS C:\> Set-CsUser -Identity mesheiah@contoso.com -LineURI $null Unable to set "LineURI". This parameter is restricted within Remote Tenant PowerShell.

+ CategoryInfo : InvalidArgument: (:) [Set-CsUser], ManagementException

+ FullyQualifiedErrorId : LineURI,Microsoft.Rtc.Management.AD.Cmdlets.SetOcsUserCmdlet

+ PSComputerName : admin0b.online.lync.com

PS C:\>

You’ve confirmed that the user has been removed from the on-premise Skype for Business Server.

Solution

One of the possible reasons why this error would be thrown is if the user account still has the msRTCSIP-DeploymentLocator attribute populated with the SRV: value:

This attribute and its value is required for the on-premise Skype for Business configuration but would prevent the user account that is synced into Azure AD to be enabled for Teams Enterprise Voice. To correct the issue, simply open the properties and Clear the value:

Issue a synchronization between the on-premise AD and Azure AD:

Start-ADSyncSyncCycle -PolicyType Delta

Wait for a few minutes and the account should have the LineURI and OnPremHostingProvider attributes empty.

Proceeding to enable the account for Enterprise Voice will now complete.

Problem

I was recently asked by a colleague of mine who was setting up Search Filter on a Citrix ADC / NetScaler Gateway so that only users of a Active Directory group could log into the published portal. The Citrix article that demonstrates the configuration is the following:

How to Restrict Access to NetScaler Gateway for only Members of one Active Directory Group
https://support.citrix.com/article/CTX111079

The article is very straight forward and he wanted to use the configuration that allowed for nested groups as describes here:

Nested Groups - By default, NetScaler will only search for usernames that are direct members of the Active Directory group. If you want to search nested groups, then add the Microsoft OID :1.2.840.113556.1.4.1941: to the LDAP Search Filter. The OID is inserted between memberOf and = as shown below:
memberOf:1.2.840.113556.1.4.1941:=CN=Citrix Remote,OU=Citrix,DC=corp,DC=local

So he proceeded to add 1.2.840.113556.1.4.1941 after the memberOf then added the domain users group into the Active Directory group he was going to use to restrict access.

He tested the configuration but noticed that it didn’t work. Reviewing the aaaa.debug logs shows that correct group is being used to filter (I will paste the full debug log at the bottom of this post:

/home/build/rs_130_36_15_RTM/usr.src/netscaler/aaad/ldap_common.c[1332]: ns_ldap_search 0-161: Searching for <<(| (objectClass=domainDNS) (& (sAMAccountName=tluk@contoso.com) (memberOf:1.2.840.113556.1.4.1941:=cn=ACL - Citrix Access,OU=Access Groups,OU=Groups,OU=contoso,DC=corp,DC=contoso,DC=com)))>> from base <<dc=corp,dc=contoso,dc=com>>

However, the test login with tluk@contoso.com which is in the domain users group that is placed in the Citrix Access group eventually fails with:

/home/build/rs_130_36_15_RTM/usr.src/netscaler/aaad/ldap_drv.c[509]: receive_ldap_user_search_event 0-161: User tluk@contoso.com not found on the LDAP server

Sat Oct 3 12:36:22 2020

Attempting to configure the Nested Group Extraction configuration as demonstrated in the following KB does not work:

How to Configure LDAP Nested Group Extraction on NetScaler Gateway
https://support.citrix.com/article/CTX216282

Solution

The short answer to this problem is that attempting use Domain Users to nest into a group used by the Netscaler to filter access will not work because this group is almost certainly configured as the Primary Group of the users and such a group is not used for group extraction. The following is the explanation described by Citrix:

Active Directory Group Extraction Does Not Work on NetScaler
https://support.citrix.com/article/CTX135603

Problem Cause

The Active Directory group which was set as primary group is not used for group extraction. All the other groups associated with the user account are extracted. This is not a bug or issue on a NetScaler appliance. This is as per Active Directory design. Active Directory does not return its primary group as a group therefore the primary group cannot be extracted by a NetScaler appliance. This is true even if you make the Department group as a primary group instead of Domain users. In the search filter on the NetScaler LDAP search filter column you might have to modify the search rule as follows:
memberof=CN=domain users,dc=lab, dc=sumagee, dc=com

Because the primary group is dept only, Domain Users group is extracted by the NetScaler appliance.

It would be nice if this was included in the original KB demonstrating the configuration as this may not be obvious to administrators configuring the restrictions so I hope this post will help anyone who may encounter this scenario.

Problem

Users have complained that they no longer see published apps and desktops after successfully logging onto Citrix as they only see the message:

There are no apps or desktops available to you at this time.

Reviewing the Citrix Delivery Services event logs on the Citrix StoreFront server displays the following errors:

None of the Citrix XML Services configured for farm Controller are in the list of active services, so none were contacted.

Log Name: Citrix Delivery Services
Source: Citrix Store Service
EventID: 4012
Level: Error

Failed to launch the resource 'Controller.GP' as it was not found.

Log Name: Citrix Delivery Services
Source: Citrix Store Service
EventID: 28
Level: Warning

None of the Citrix XML Services configured for farm Controller are in the list of active services, so none were contacted.

Log Name: Citrix Delivery Services
Source: Citrix Store Service
EventID: 4012
Level: Error

Solution

The most important entry in the event logs written for this issue could easily be missed because the entry that provides the cause of the issue is actually labeled as Information. Continuing to move to earlier logs will reveal the following entry indicating that the SSL certificate on the Delivery Controller has expired:

The Citrix XML Service at address svr-ctxdc-02.ccs.int:443 has failed the background health check and has been temporarily removed from the list of active services. Failure details: An SSL connection could not be established: The server sent an expired security certificate. The certificate *.ccs.int, *.ccs.int is valid from 10/29/2018 9:37:20 AM until 10/28/2020 9:37:20 AM.. This message was reported from the Citrix XML Service at address https://svr-ctxdc-02.ccs.int/scripts/wpnbr.dll[UnknownRequest].

You would not be able to see this entry if you are reviewing the logs in the Administrative Events, which does not display Information entries.

To correct the issue, simply issue a new SSL certificate to replace the expired certificate on the Delivery Controller (or controllers if there are more than one), then update the bindings in IIS Manager:

Successfully updating the SSL certificate will re-establish communication between the StoreFront server and the Delivery Controller(s).

Now that working from home has become more popular for organizations during the pandemic, many of the clients I work with have decided to deploy Microsoft Teams’ Direct Routing feature that allows users to receive and make calls out via the PSTN network. A few of the larger organizations I’ve had the opportunity to work with have still kept their messaging on-premise for various reasons and one of the most common complaints I’ve received is the voicemail notification. I still remember researching how it worked for on-premise Exchange Server deployments and found I had to combine several sources to understand what worked and didn’t so this blog post serves as a quick summary of the behavior.

The short story of the type of experience a user with an on-premise Exchange Server mailbox would have when they are enabled for Enterprise Voice in Teams is that voicemail notifications are sent to them via email. The email originates from Office 365 Exchange Online and delivered to the on-premise Exchange server and into the user’s mailbox via SMTP. The only issue I’ve seen where users do not receive this notification is if the organization uses a SPAM service such as Mimecast and it is not configured to allow Office 365 IP addresses for relay (https://community.mimecast.com/s/article/Maintaining-Authorized-Outbound-Addresses) and therefore the emails are stuck in the Exchange Online’s queue failing to be delivered.

Users with an on-premise Exchange Server mailbox will see the following message on their smartphones when they receive a voicemail but unable to do anything else within the menu:

We have your voicemail covered.

You can listen to voicemail or read the transcript.

Clicking into the Voicemail menu in the Calls section will display the following:

We can’t get your voicemail right now

Please check back soon.

All of the above are default behaviors of a Teams user with an on-premise Exchange Server mailbox and the only way around this is to migrate them to Exchange Online.

More information about this can be found at the follow Microsoft documentation:

Set up Cloud Voicemail for Exchange Server Mailbox Users
https://docs.microsoft.com/en-ca/microsoftteams/set-up-phone-system-voicemail#set-up-cloud-voicemail-for-exchange-server-mailbox-users

Hope this helps anyone looking for an explanation of what works and what doesn’t when a user isn’t on Exchange Online.

As mentioned in my previous posts:

Enabling user for Teams Enterprise Voice fails with: "Management object not found for identity"
http://terenceluk.blogspot.com/2020/10/enabling-user-for-teams-enterprise.html

… a user who has previously been enabled for an on-premise Skype for Business Server may still have the msRTCSIP-DeploymentLocator attribute populated with the value SRV: even after they have been removed from the on-premise SfB Server.

Having this value in their user account would prevent them from being enabled for Enterprise Voice in Microsoft Teams and the quick resolution for this would be to remove the attribute, for and AD Connect sync, then try enabling them again.

This solution may be viable for a few accounts but not practical when the amount goes up to the 100s or 1000s so I decided to create a few PowerShell cmdlets to automate the process.

Find SfB Enabled with msRTCSIP-DeploymentLocator Configured Accounts

Get-ADuser -Properties msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled -Filter {msRTCSIP-DeploymentLocator -like "*" -and msRTCSIP-UserEnabled -eq $true } | Select UserPrincipalName,msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled

Find SfB Enabled with msRTCSIP-DeploymentLocator Configured as SRV: Accounts

Get-ADuser -Properties msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled -Filter {msRTCSIP-DeploymentLocator -eq "SRV:" -and msRTCSIP-UserEnabled -eq $true } | Select UserPrincipalName,msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled

Find SfB Disabled with msRTCSIP-DeploymentLocator Configured as SRV: AND empty msRTCSIP-PrimaryHomeServer Accounts Export as CSV

Get-ADuser -Properties msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled -Filter {msRTCSIP-DeploymentLocator -eq "SRV:" -and msRTCSIP-UserEnabled -notlike '*' -and msRTCSIP-PrimaryHomeServer -notlike '*'} | Select UserPrincipalName,msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled | Export-CSV C:\temp\SRVaccounts.csv

Find SfB Disabled with msRTCSIP-DeploymentLocator Configured as SRV: AND empty msRTCSIP-PrimaryHomeServer Accounts and Clear value

**The following cmdlet modifies accounts and I would recommend to export them to a list and review prior to making the changes**

Get-ADuser -Properties msRTCSIP-DeploymentLocator,msRTCSIP-PrimaryHomeServer,msRTCSIP-UserEnabled -Filter {msRTCSIP-DeploymentLocator -eq "SRV:" -and msRTCSIP-UserEnabled -notlike '*' -and msRTCSIP-PrimaryHomeServer -notlike '*'} | Set-ADUser -Clear msRTCSIP-DeploymentLocator

The following are the output of some of the cmdlets above.

I’ve recently been involved with a few projects involving security vulnerabilities from penetration scans and one of the clients were asked to secure their Exchange OWA portal load balanced behind a Citrix ADC so that 2 of the 6 headers that are identified to be missing from the Security Headers (https://securityheaders.com/) scan are addressed:

• X-Content-Type-Options
• Permissions-Policy

Please refer to my previous blog post for instructions on how to implement the other headers identified to be present in the scan above:

Securing a Citrix ADC (formally known as NetScaler VPX) to score an A rating on Security Headers - March 2020
http://terenceluk.blogspot.com/2020/02/securing-citrix-adc-formally-known-as.html

X-Content-Type-Options

The X-Content-Type-Options header is fairly easy to implement as it is described in Citrix’s knowledge base article:

How to create rewrite policy for content security headers , XSS protection, HSTS, X-Content-Type-Options &amp; Content-Security-Policy.
https://support.citrix.com/article/CTX233095

The following are the CLI commands:

add rewrite action rw_act_insert_Xcontent_header_Policy insert_http_header X-Content-Type-Options "\"nosniff\""

add rewrite policy rw_pol_insert_XContent_Policy TRUE rw_act_insert_Xcontent_header_Policy

bind lb vserver mail.college.bm_external -policyName rw_pol_insert_XContent_Policy -type RESPONSE -priority 120 -gotoPriorityExpression NEXT

The following is the Rewrite Action in the Citrix ADC / NetScaler GUI:

The following is the Rewrite Policy in the Citrix ADC / NetScaler GUI:

With the Rewrite Action and Policy in place, and having the policy bound to the virtual server, the scan should now return the header as being present:

Permissions-Policy

The Permissions-Policy, which replaces the Feature Policy, isn’t as straight forward as Citrix does not have a KB for the implementation of this header. However, Scott Helme provides a very detailed explanation for this header:

Goodbye Feature Policy and hello Permissions Policy!
https://scotthelme.co.uk/goodbye-feature-policy-and-hello-permissions-policy/

The following are the CLI commands to create the Rewrite Action, Policy and bind it to the virtual server:

add rewrite action rw_act_insert_Permissions_Policy insert_http_header Permissions-Policy "\"vibrate=(self), sync-xhr=(self \'https://<owa.domain.com>\')\""

add rewrite policy rw_pol_insert_Permissions_Policy "HTTP.RES.HEADER(\"Permissions-Policy\").EXISTS.NOT" rw_act_insert_Permissions_Policy

bind lb vserver mail.contoso.com_external -policyName rw_pol_insert_Permissions_Policy -type RESPONSE 110 -gotoPriorityExpression NEXT

The following is the Rewrite Action in the Citrix ADC / NetScaler GUI:

The following is the Rewrite Policy in the Citrix ADC / NetScaler GUI:

With the Rewrite Action and Policy in place, and having the policy bound to the virtual server, the scan should now return the header as being present:

**Note that the reason why the score in the screenshot above is A rather than A+ is because the Content-Security-Policy header configured contains the ‘unsafe-inline’ in the script-src directive. I have yet to successfully omit ‘unsafe-inline’ for sites such as a Citrix Gateway or Exchange OWA without causing the login page to fail to load.

Problem

You attempt to create a new SQL Server 2019 Availability Group after configuring a Windows Failover Cluster:

… but the following error is presented:

TITLE: Microsoft SQL Server Management Studio

------------------------------

• The local node is not able to communicate with the WSFC cluster.

• No quorum set across the WSFC cluster.

(Microsoft.SqlServer.Management.HadrTasks)

-----------------------------

BUTTONS:

OK

------------------------------

Clicking on the Show details button displays the following:

===================================

The local node is not part of quorum and is therefore unable to process this operation. This may be due to one of the following reasons:

• The local node is not able to communicate with the WSFC cluster.

  • No quorum set across the WSFC cluster.

For more information on recovering from quorum loss, refer to SQL Server Books Online.

------------------------------

Program Location:

at Microsoft.SqlServer.Management.Hadr.CreateAvailabilityGroupWizardController.CreateDataModel()

at Microsoft.SqlServer.Management.Hadr.CreateAvailabilityGroupWizardController.Initialize()

at Microsoft.SqlServer.Management.TaskForms.SqlWizardController.Microsoft.SqlServer.Management.TaskForms.ISqlWizardController.Initialize(ISqlWizardManager wizardManager)

at Microsoft.SqlServer.Management.TaskForms.SqlWizardManager.LoadWizardController()

at Microsoft.SqlServer.Management.TaskForms.SqlWizardManager.Initialize(String moniker, IContext context, ISqlWizardInfo wizardInfo)

at Microsoft.SqlServer.Management.ActionHandlers.ShowWizardActionHandler.RunTaskForm(IContext context)

at Microsoft.SqlServer.Management.ActionHandlers.DialogBasedActionHandler.RunTaskFormThread(Object contextObject)

Solution

One of the common cause of this error is if the Enable Always On Availability Groups configuration for the SQL Server Services was turned on before you have installed and configured the Windows Failover Cluster:

If this is the case, simply disable the Enable Always On Availability Group option, restart the SQL Server Services, re-enable the option, and then restart the service:

The New Availability Group should now launch and allow you to create the Always On Availability Group:

The New Availability Group should now launch and allow you to create the Always On Availability Group:

Problem

You’re attempting to create a Availability Group Listener after successfully creating a SQL Server 2019 Availability Group but receive the following error:

TITLE: Microsoft SQL Server Management Studio

------------------------------

Create failed for Availability Group Listener 'PRD_TX_Listener'. (Microsoft.SqlServer.Smo)

For help, click: https://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=16.100.44091.28+(SMO-master-A)&EvtSrc=Microsoft.SqlServer.Management.Smo.ExceptionTemplates.FailedOperationExceptionText&EvtID=Create+AvailabilityGroupListener&LinkId=20476

------------------------------

ADDITIONAL INFORMATION:

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

------------------------------

The WSFC cluster could not bring the Network Name resource with DNS name 'PRD_TX_Listener' online. The DNS name may have been taken or have a conflict with existing name services, or the WSFC cluster service may not be running or may be inaccessible. Use a different DNS name to resolve name conflicts, or check the WSFC cluster log for more information.

The attempt to create the network name and IP address for the listener failed. If this is a WSFC availability group, the WSFC service may not be running or may be inaccessible in its current state, or the values provided for the network name and IP address may be incorrect. Check the state of the WSFC cluster and validate the network name and IP address with the network administrator. Otherwise, contact your primary support provider. (Microsoft SQL Server, Error: 19471)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=15.00.2000&EvtSrc=MSSQLServer&EvtID=19471&LinkId=20476

------------------------------

BUTTONS:

OK

------------------------------

===================================

Create failed for Availability Group Listener 'PRD_TX_Listener'. (Microsoft.SqlServer.Smo)

------------------------------

For help, click: https://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=16.100.44091.28+(SMO-master-A)&EvtSrc=Microsoft.SqlServer.Management.Smo.ExceptionTemplates.FailedOperationExceptionText&EvtID=Create+AvailabilityGroupListener&LinkId=20476

------------------------------

Program Location:

at Microsoft.SqlServer.Management.Smo.SqlSmoObject.CreateImpl()

at Microsoft.SqlServer.Management.SqlManagerUI.AGListenerConfigurationNewModeViewData.DoWorkCore(AGListenerController controller)

at Microsoft.SqlServer.Management.SqlManagerUI.AGListenerConfigurationControl.OnRunNow(Object sender)

at Microsoft.SqlServer.Management.SqlMgmt.PanelExecutionHandler.Run(RunType runType, Object sender)

at Microsoft.SqlServer.Management.SqlMgmt.SqlMgmtTreeViewControl.DoPreProcessExecutionAndRunViews(RunType runType)

at Microsoft.SqlServer.Management.SqlMgmt.SqlMgmtTreeViewControl.ExecuteForSql(PreProcessExecutionInfo executionInfo, ExecutionMode& executionResult)

at Microsoft.SqlServer.Management.SqlMgmt.SqlMgmtTreeViewControl.Microsoft.SqlServer.Management.SqlMgmt.IExecutionAwareSqlControlCollection.PreProcessExecution(PreProcessExecutionInfo executionInfo, ExecutionMode& executionResult)

at Microsoft.SqlServer.Management.SqlMgmt.ViewSwitcherControlsManager.RunNow(RunType runType, Object sender)

===================================

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

------------------------------

Program Location:

at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteNonQuery(String sqlCommand, ExecutionTypes executionType, Boolean retry)

at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteNonQuery(StringCollection sqlCommands, ExecutionTypes executionType, Boolean retry)

at Microsoft.SqlServer.Management.Smo.ExecutionManager.ExecuteNonQuery(StringCollection queries, Boolean retry)

at Microsoft.SqlServer.Management.Smo.SqlSmoObject.ExecuteNonQuery(StringCollection queries, Boolean includeDbContext, Boolean executeForAlter)

at Microsoft.SqlServer.Management.Smo.SqlSmoObject.CreateImplFinish(StringCollection createQuery, ScriptingPreferences sp)

at Microsoft.SqlServer.Management.Smo.SqlSmoObject.CreateImpl()

===================================

The WSFC cluster could not bring the Network Name resource with DNS name 'PRD_TX_Listener' online. The DNS name may have been taken or have a conflict with existing name services, or the WSFC cluster service may not be running or may be inaccessible. Use a different DNS name to resolve name conflicts, or check the WSFC cluster log for more information.

The attempt to create the network name and IP address for the listener failed. If this is a WSFC availability group, the WSFC service may not be running or may be inaccessible in its current state, or the values provided for the network name and IP address may be incorrect. Check the state of the WSFC cluster and validate the network name and IP address with the network administrator. Otherwise, contact your primary support provider. (.Net SqlClient Data Provider)

------------------------------

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=15.00.2000&EvtSrc=MSSQLServer&EvtID=19471&LinkId=20476

------------------------------

Server Name: BMA-PROD-SQL1

Error Number: 19471

Severity: 16

State: 0

Line Number: 1

 ------------------------------

Program Location:

at Microsoft.SqlServer.Management.Common.ConnectionManager.ExecuteTSql(ExecuteTSqlAction action, Object execObject, DataSet fillDataSet, Boolean catchException)

at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteNonQuery(String sqlCommand, ExecutionTypes executionType, Boolean retry)

Solution

This error is typically displayed if the Cluster Name Object (CNO) in Active Directory does not have permissions to create and read the listener computer object. The CNO name can be found in the Failover Cluster Manager:

To correct the issue, simply navigate to the OU containing CNO, which would be where it will attempt to create the listener computer object, right click on the OU and select Delegate Control… to launch the Delegation of Control Wizard:

Add the CNO computer object:

Select Create a custom task to delegate and click Next:

Select This folder, existing objects in this folder, and creation of new objects in this folder then click Next:

Select General and Creation/deletion of specific child objects, then locate the following and select the following:

• Read all properties
• Create Computer objects
• Delete Computer objects

With the CNO computer object granted the respective permissions, the listener should now create successfully:

One of the most common questions I still get asked when it comes to Skype For Business is when an administrator attempts to edit an account but receives the following error:

Active Directory operation failed on "dc1.contoso.local". You cannot retry this operation: "Insufficient access rights to perform the operation

This error has been around since Lync Server 2010 and the two ways around it are:

1. Use PowerShell to edit the account
2. Enable Inheritance for the Active Directory account

Most administrators would prefer to opt for #2 as this would allow the account to be configured in the GUI so what needs to be done is to go into the Security properties of the account, click on Advanced button, then click on Enable inheritance so that the account inherits permissions:

This isn’t too much effort for one or two accounts but not very practical in the 10s, 100s or 1000s. The following are PowerShell scripts that allows to better identify and configure these accounts:

List Accounts and their Inheritance Status and Enable Accounts that have Inheritance disabled

$ou = 'ou=User Acccounts,ou=Contoso Organization,dc=contoso,dc=local'

$userslist = Get-ADUser -ldapfilter “(objectclass=user)” -searchbase $ou

foreach($account in $userslist)

{

# Bind the accounts

$ou = [ADSI](“LDAP://” + $account)

$sec = $ou.psbase.objectSecurity

if ($sec.get_AreAccessRulesProtected())

{

$isProtected = $false ## turn on inheritance

$preserveInheritance = $true ## retain inherited permissions

$sec.SetAccessRuleProtection($isProtected, $preserveInheritance)

$ou.psbase.commitchanges()

Write-Host “$account is now enabled for inheritance”;

}

else

{

Write-Host “$account already has inheritance enabled”

}

}

Export Accounts and Inheritance Status to CSV

$CSVexportPath = 'C:\temp\AccountInheritanceStatus.csv'

$ou = 'ou=User Acccounts,ou=Contoso Organization,dc=contoso,dc=local'

$userslist = Get-ADUser -ldapfilter “(objectclass=user)” -searchbase $ou

foreach($account in $userslist)

{

# Bind the accounts

$ou = [ADSI](“LDAP://” + $account)

$sec = $ou.psbase.objectSecurity

$UPN = $account.UserPrincipalName

$name = $account.name

$DistinguishedName = $account.DistinguishedName

#Store the information from this run into the array

[PSCustomObject]@{

UPN = $account.UserPrincipalName

Name = $account.name

DistinguishedName = $account.DistinguishedName

InheritanceEnabled = $sec.get_AreAccessRulesProtected()

} | Export-Csv $CSVexportPath -notype -Append

}

List Inheritance Status to Table

$ou = 'ou=User Acccounts,ou=Contoso Organization,dc=contoso,dc=local'

$userslist = Get-ADUser -ldapfilter “(objectclass=user)” -searchbase $ou

foreach($account in $userslist)

{

# Bind the accounts

$ou = [ADSI](“LDAP://” + $account)

$sec = $ou.psbase.objectSecurity

$UPN = $account.UserPrincipalName

$name = $account.name

$DistinguishedName = $account.DistinguishedName

#Store the information from this run into the array

[PSCustomObject]@{

UPN = $account.UserPrincipalName

Name = $account.name

DistinguishedName = $account.DistinguishedName

InheritanceEnabled = $sec.get_AreAccessRulesProtected()

} | Format-Table -AutoSize

}

Problem

You’re attempting to upgrade an Exchange Server 2019 to Cumulative Update 7, which requires .NET Framework 4.8:

https://support.microsoft.com/en-ca/help/4571787/cumulative-update-7-for-exchange-server-2019

The prerequisites identifies that .NET Framework 4.8 is not present and provides a link to the following installer:

https://support.microsoft.com/en-us/help/4503548/microsoft-net-framework-4-8-offline-installer-for-windows

The following is the offline installer:

Executing it on the Exchange server loads the installer but stays stuck at the following prompt:

File security verification:

All files were verified successfully.

Installation progress:

Solution

Before begin troubleshooting, review the current version of .NET Framework installed with the following PowerShell cmdlet:

PS C:\Windows\system32> Get-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full" | Format-List

CBS : 1

Install : 1

InstallPath : C:\Windows\Microsoft.NET\Framework64\v4.0.30319\

Release : 461814

Servicing : 0

TargetVersion : 4.0.0

Version : 4.7.03190

PSPath : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework

Setup\NDP\v4\Full

PSParentPath : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4

PSChildName : Full

PSDrive : HKLM

PSProvider : Microsoft.PowerShell.Core\Registry

PS C:\Windows\system32>

This issue is one that I’ve come across multiple times in the past as demonstrated in one of my previous posts:

Installing .NET Framework 4.7.1 on Windows Server 2016 as a prerequisite for patching Exchange 2016 CU8 to CU12 remains stuck at: “File security verification: All files were verified successfully.”
http://terenceluk.blogspot.com/2019/04/installing-net-framework-471-on-windows_4.html

There are times when waiting 20 minutes or more will proceed with the install:

However, what I’ve found that the MSU from the Microsoft Update Catalog installs much faster:

https://www.catalog.update.microsoft.com/Search.aspx?q=4486153

Once installed, execute the following cmdlet again to verify that .NET Framework 4.8 is installed:

Get-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full" | Format-List

Hope this helps anyone who may run into this issue as updating Exchange Server to a CU is usually performed after hours with limited time for troubleshooting.

I do not have many clients with an on-premise Exchange Server but there was one who needed their Exchange Server 2019 CU3 patched to the latest CU7.

Error:

The following error was generated when "$error.Clear();

start-SetupService -ServiceName MSExchangeIS

" was run: "Microsoft.Exchange.Configuration.Tasks.ServiceDisabledException: Service 'MSExchangeIS' is disabled on this server. ---> System.InvalidOperationException: Cannot start service MSExchangeIS on computer '.'. ---> System.ComponentModel.Win32Exception: The service cannot be started, either because it is disabled or because it has no enabled devices associated with it

--- End of inner exception stack trace ---

at System.ServiceProcess.ServiceController.Start(String[] args)

at Microsoft.Exchange.Management.Tasks.ManageSetupService.StartServiceWorker(ServiceController serviceController, String[] serviceParameters)

--- End of inner exception stack trace ---

at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)

at Microsoft.Exchange.Management.Tasks.ManageSetupService.StartService(ServiceController serviceController, Boolean ignoreServiceStartTimeout, Boolean failIfServiceNotInstalled, Unlimited`1 maximumWaitTime, String[] serviceParameters)

at Microsoft.Exchange.Management.Tasks.ManageSetupService.StartService(String serviceName, Boolean ignoreServiceStartTimeout, Boolean failIfServiceNotInstalled, Unlimited`1 maximumWaitTime, String[] serviceParameters)

at Microsoft.Exchange.Management.Tasks.StartSetupService.InternalProcessRecord()

at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__91_1()

at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)".

Attempting to restart the CU7 patch detects the unfinished patching and restarts but then fails again:

There aren’t any Microsoft KBs or blog posts with the error but the message implies the Microsoft Exchange Information Store (MSExchangeIS) could not be started and reviewing the services console does show that it is disabled so I manually enabled it, started it, restarted the installer but it would fail again with the MSExchangeIS service disabled.

A bit of searching on the internet lead me to the following blog post:

https://oddytee.wordpress.com/2017/11/09/cannot-start-service-msexchangeservicehost-on-computer-during-exchange-cu-update/

… which suggested that we chronically change any Microsoft Exchange services set to disabled to automatic with the following PowerShell command:

While (1 -le 2) { Sleep 1 ; Get-Service | Where { $_.DisplayName -Like ‘Microsoft Exchange*’ } | Set-Service –StartupType ‘Automatic’ }

I gave this a shot and can confirm that it allowed CU7 to complete patching the Exchange server.

I’ve recently had to perform a bit of licensing management for a client because they had a set of users who have Office 365 E1 licenses assigned but the Apps configured were inconsistent across the board. Using the Microsoft 365 admin center to perform this is possible but isn’t the most efficient method for large batches.

https://admin.microsoft.com/

What I ended up using was PowerShell for the assignments and thought it would be useful to share the PowerShell cmdlets for some common licensing tasks that may be useful for others.

All of the cmdlets below require you to connect to the O365 tenant via Connect-MsolService cmdlet.

Obtaining licenses available in the tenant

The cmdlet to obtain the licenses available in the tenant is Get-MsolAccountSku and as the output below shows, the AccountSkuId name will not directly reflect what you see in the portal. Case in point, STANDARDPACK is actually the name for Office 365 E1:

Most of the names for these SKUs can be found at the following TechNet article:

Product names and service plan identifiers for licensing
https://docs.microsoft.com/en-us/azure/active-directory/enterprise-users/licensing-service-plan-reference

Obtaining the enabled or disabled apps for the license assigned to a user

Each Office 365 license actually bundles a group of applications that can be enabled or disabled for a user. Below is an example of what Apps are available when Office 365 E1 is assigned:

The following cmdlet allows us to obtain the Apps and their status for the specified user:

(Get-MsolUser -UserPrincipalName shaejames@contoso.com).Licenses.ServiceStatus

Note that this cmdlet will list all the Apps and their status. Success and Disabled is self-explanatory, while PendingProvisioning and PendingActivation is reflected in the follow 3 app entries with the same description:

This app is assigned at the organization level. It can’t be assigned per user.

Retrieving a list of all users with E1 license

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "STANDARDPACK"}

Enabling or disabling apps for a user assigned with an E1 license

The following are examples of how to enable and disable various applications:

Only disable SWAY and enable all apps

$E1_DefaultApps = New-MsolLicenseOptions -AccountSkuId reseller-account:STANDARDPACK -DisabledPlans SWAY

Set-MsolUserLicense -UserPrincipalName "shaejames@contoso.com" -LicenseOptions $E1_DefaultApps

Disable SWAY and Exchange Online and enable all apps

$DisabledApps=@()

$DisabledApps+="SWAY"

$DisabledApps+="EXCHANGE_S_STANDARD"

$E1_DefaultApps = New-MsolLicenseOptions -AccountSkuId reseller-account:STANDARDPACK -DisabledPlans $DisabledApps

Set-MsolUserLicense -UserPrincipalName "shaejames@contoso.com" -LicenseOptions $E1_DefaultApps

Enable all apps

$DisabledApps=@()

$E1_DefaultApps = New-MsolLicenseOptions -AccountSkuId reseller-account:STANDARDPACK -DisabledPlans $DisabledApps

Set-MsolUserLicense -UserPrincipalName "shaejames@contoso.com" -LicenseOptions $E1_DefaultApps

Disable Skype for Business Online (Plan 2) and Exchange Online and enable all apps

$DisabledApps=@()

$DisabledApps+="MCOSTANDARD"

$DisabledApps+="EXCHANGE_S_STANDARD"

$E1_DefaultApps = New-MsolLicenseOptions -AccountSkuId reseller-account:STANDARDPACK -DisabledPlans $DisabledApps

Set-MsolUserLicense -UserPrincipalName "shaejames@contoso.com" -LicenseOptions $E1_DefaultApps

Set all users with E1 license to disable Skype for Business Online (Plan 2) and Exchange Online (Plan 1) options

$DisabledApps=@()

$DisabledApps+="MCOSTANDARD"

$DisabledApps+="EXCHANGE_S_STANDARD"

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "STANDARDPACK"} | Set-MsolUserLicense -LicenseOptions $E1_DefaultApps

Get all users with Microsoft Teams Commercial Cloud and E1 license

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "TEAMS_COMMERCIAL_TRIAL" -and ($_.licenses).AccountSkuId -match "STANDARDPACK"}

Get all users with Microsoft Teams Commercial Cloud and E1 license AND remove Microsoft Teams Commercial Cloud license

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "TEAMS_COMMERCIAL_TRIAL" -and ($_.licenses).AccountSkuId -match "STANDARDPACK"} | Set-MsolUserLicense -RemoveLicenses reseller-account:TEAMS_COMMERCIAL_TRIAL

Get all users with Microsoft Teams Exploratory and E1 license

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "TEAMS_Exploratory" -and ($_.licenses).AccountSkuId -match "STANDARDPACK"}

Get all users with Microsoft Teams Exploratory and E1 license AND remove Microsoft Teams Exploratory license

Get-MsolUser -All | Where-Object {($_.licenses).AccountSkuId -match "TEAMS_Exploratory" -and ($_.licenses).AccountSkuId -match "STANDARDPACK"} | Set-MsolUserLicense -RemoveLicenses reseller-account:TEAMS_Exploratory

------------------------------------------------------------------------------------------------------------------------

I hope these cmdlets will help anyone who may need to undertake a similar task.

I’ve recently had a few clients reach out to me about the configuration for publishing Office 365 SaaS applications via Citrix Workspace because most quickly realize that it isn’t as easy as using the SaaS templates in the Citrix Cloud portal as soon as the service is provisioned. Doing so would present the application icon:

… but clicking on it will display the following error:

Sign in

Sorry, but we’re having trouble signing you in.

AADSTS50107: The requested federation realm object 'https://citrix.com/dcintd0a28a9' does not exist.

Troubleshooting details

If you contact your administrator, send this info to them.

Copy info to clipboard

Request Id: 63e1ab2a-8c53-48d9-9bbd-a775b8e61a02

Correlation Id: 80a8cfd9-acfd-4faf-995c-d91afe14a5b5

Timestamp: 2020-11-09T14:18:15Z

Message: AADSTS50107: The requested federation realm object 'https://citrix.com/dcintd0a28a9' does not exist.

Advanced diagnostics:Enable

If you plan on getting support for an issue, turn this on and try to reproduce the error. This will collect additional information that will help troubleshoot the issue.

One of the important prerequisites that many administrators may not be away of is that Citrix Workspace requires a domain in Azure AD to be federated with it in order to successfully publishing Office 365 as a SaaS application. The choices for federation is to federate the primary domain or another domain that is verified within Azure AD. Most of the clients I work with would not be able to federate their primary domain because that would cause users who attempt to log into Office 365 via https://office.com or https://login.microsoftonline.com/ to be redirected to the Citrix Workspace portal for authentication:

https://accounts.cloud.com/core/company/prompt

With this in mind, what I typically recommend is to review and select a domain that the organization owns and can be verifiable but is not used for any Office 365 authentication to federate with Citrix.

The following Prerequisites section of the document provides a more detailed explanation about this:

https://docs.citrix.com/en-us/advanced-concepts/design-guides/citrix-gateway-o365-saas.html#prerequisites

The rest of the document provides instructions on how to configure the federation:

Citrix Gateway SaaS and O365 Cloud Citrix Validated Reference Design
https://docs.citrix.com/en-us/advanced-concepts/design-guides/citrix-gateway-o365-saas.html

What I’ve been told by a lot of administrators who may not be completely familiar with Citrix Cloud is that a few of the steps can be confusing so I would like to write this short blog post to demonstrate the configuration.

Prerequisites

Before beginning the federation configuration, ensure that a domain has been selected and verified in the Office 365 portal (Azure AD) as we will need this for the configuration:

Step #1 – Install Azure AD Modules and Connect to Azure Active Directory

Begin by launching PowerShell as an administrator and install the AzureAD modules then connect to Azure Active Directory with the following cmdletse:

Install-Module AzureAD -Force

Import-Module AzureAD -Force

Install-Module MSOnline -Force

Import-module MSOnline -Force

Connect-MsolService

Step #2 – Define the required variables

The following are the variables that will need to be defined and where to obtain the information:

$dom = "contoso.com"< replace contoso.com with the domain that has been selected and verified in Azure AD to federate with Citrix Workspace

$IssuerUri = “https://citrix.com/dcintd0a28a9” < replace the dcintd0a28a9with the Customer ID, which can be located here in the Citrix Cloud portal:

$fedBrandName = "CitrixWorkspace"< Leave this without any changes

$logoffuri = "https://app.netscalergateway.net/cgi/logout" < Leave this without any changes

$uri = "https://app.netscalergateway.net/ngs/dcintd0a28a9/saml/login?APPID=27e051ca-42c2-478d-a460-9d512a4e8dab"< Replace the full https:// URI with the following:

Assuming that the Office 365 SaaS has been published, click into the Library Offerings:

Edit the Office 365 application:

Expand the Single sign on heading and copy the Login URL string:

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("c:\cert\dcintd0a28a9_cert.crt")> The certificate can be confusing for more administrators and the location to download it is in the same place as the Login URL above where you will find the Certificate available to download in multiple formats. Proceed to select CRT and download the file:

$certData = [system.convert]::tobase64string($cert.rawdata)< Leave this without any changes

The following are all the variables with a sample configuration and the items that need to be changed highlighted in red:

$dom = "contoso.com"

$IssuerUri = "https://citrix.com/dcintd0a28a9"

$fedBrandName = "CitrixWorkspace"

$logoffuri = "https://app.netscalergateway.net/cgi/logout"

$uri = "https://app.netscalergateway.net/ngs/dcintd0a28a9/saml/login?APPID=27e051ca-42c2-478d-a460-9d512a4e8dab"

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("c:\cert\dcintd0a28a9_cert.crt")

$certData = [system.convert]::tobase64string($cert.rawdata)

Step #3 – Configure the Federation between Azure AD and Citrix

Configure the federation with the following PowerShell cmdlet without any changes:

Set-MsolDomainAuthentication -DomainName $dom -federationBrandName $fedBrandName -Authentication Federated -PassiveLogOnUri $uri -LogOffUri $logoffuri -SigningCertificate $certData -IssuerUri $IssuerUri -PreferredAuthenticationProtocol SAMLP

Step #4 – Confirm the Federation between Azure AD and Citrix

Confirm that the federation has been configured by executing the following PowerShell cmdlet:

Get-MsolDomainFederationSettings -DomainName contoso.bm

------------------------------------------------------------------------------------------------------------------------

The following is a sample output in the PowerShell window after a successful configuration:

With the federation in place, clicking on the Office 365 application will redirect the user to Office 365 to authenticate.