I don’t usually provide operational support but have a client who still reaches out to me with issues related to Azure because of my role but I absolutely welcome it as it continues to give me exposure to day-to-day operational problems. The recent question I received a couple days ago was one that I’ve come across in the past and felt it is worth writing a quite blog post about it to help anyone who may come across this.

Problem

You’re attempting to change an App Service Plan from, say, the Premium Tier to Standard Tier but the process fails with the message:

Failed to change to App Service plan: 'Cannot use the SKU Standard with File Change Audit for site <AppServicePlan>. See https://aka.ms/supported-log-types for more information.’

A quick search on the internet will return the following blog post: https://nexxai.dev/cannot-use-the-sku-basic-with-file-change-audit-for-site/, which outlines a series of steps with PowerShell cmdlets to remediate the issue. I’ve used this procedure in the past and it has worked for me but the post was written back in January 2021 and I suggest the solution below which uses the portal.azure.com GUI should be attempted before using PowerShell.

Solution

1. Login into portal.azure.com and navigate to the App Service.

2. Click on Diagnostic settings and look for a rule set that refers to App Service Audit Logs, then click on the Edit setting link:

3. With the settings of the rule displayed, scroll down the list and look for AppServiceAuditLogs, deselect it, save the configuration and confirm that you want to delete the setting:

Once the configuration has been updated, proceed and try changing the App Service Plan.

Hope this helps anyone looking for the solution to this issue.

A colleague of mine recently reached out to me to ask why my script from this post:

PowerShell script to assign users in an on-premise AD group to an Azure Enterprise Application's Users and Groups

http://terenceluk.blogspot.com/2022/02/powershell-script-to-assign-users-in-on.html

… did not work not work when a service principal was used as the Get-AzureADServiceAppRoleAssignment did not return the Users and Groups of the Enterprise Application that is passed to the cmdlet as the ObjectId. This brought me back to the time when I first worked with this cmdlet and spent hours over the weekend to figure out why this was the case so in hopes of saving others some grief, this post will explain the issue and resolution.

Let me begin by starting with the environment I’ll use for demonstrating this.

Environment

An Application registration named Enterprise-Apps-Permissions-PS is created in this tenant and an Enterprise Application (Service Principal) with the same name has been created from it.

Note that the Object ID of the Enterprise Application is: f984b219-756b-4454-bbf6-4290968836e4

If we wanted to retrieve the Enterprise Application’s Users and Groups via PowerShell, we can simply use the following cmdlets:

# Log into Azure AD PowerShell With Admin Account

Connect-AzureAD

# Retrieve Users and groups assignment

Get-AzureADServiceAppRoleAssignment -all $true -ObjectId "f984b219-756b-4454-bbf6-4290968836e4"

However, if you log in with a service principal using certificate thumbprint to authenticate, executing the same command gives you something different:

# Set variables to connect with service principal

$tenant = "84f4470b-3f1e-4xxxx-9f95-xxxxxxx24f"

$thumb = "D4F02xxxxxx60422BxxxxxxxE60Fxxxxxxxx"

$appId = "14b57135-6bf6-451f-aabb-faa3822dd5e8"

# Connect to Azure AD with service principal

Connect-AzureAD -TenantId $tenant -ApplicationId $AppId -CertificateThumbprint $thumb

# Obtain users and groups

Get-AzureADServiceAppRoleAssignment -all $true -ObjectId "f984b219-756b-4454-bbf6-4290968836e4"

Explanation

The reason why the output is different is because the Get-AzureADServiceAppRoleAssignment looks up and returns different configuration for a user and a service principal.

When logged in as a user and execute Get-AzureADServiceAppRoleAssignment: the Enterprise Application’s Users and Groups will be returned

When logged in as a service principal and execute Get-AzureADServiceAppRoleAssignment: the App registration’s API permissions is returned

Let’s review the output with some side-by-side screenshots.

In this environment the App registration named Enterprise-Apps-Permissions-PS has 3 Microsoft Graph permissions assigned to it, which is what Get-AzureADServiceAppRoleAssignment returns when executed under a service principal login. The screenshot has the top window showing the 3 Microsoft Graph permissions assigned to the App registration (Azure Active Directory > App registration > API permissions):

• AccessReview.Read.All
• AccessReview.ReadWrite.All
• AccessReview.ReadWrite.Membership

… while the bottom window shows 1 user account assigned to the Enterprise Application linked to the App registration (Azure Active Directory > Enterprise Application > Users and Groups):

• Terence Luk

If you use a regular user account to connect to Azure AD via Connect-AzureAD, then proceed use the cmdlet Get-AzureADServiceAppRoleAssignment (https://docs.microsoft.com/en-us/powershell/module/azuread/get-azureadserviceapproleassignment?view=azureadps-2.0) and execute the following:

Get-AzureADServiceAppRoleAssignment -all $true -ObjectId "f984b219-756b-4454-bbf6-4290968836e4"

You’ll receive the user and groups assigned to the Enterprise Application as shown in the following PowerShell output and screenshot:

However, if you connect into Azure AD with a Service Principal using certificate authentication as such:

$tenant = "84f4470b-3f1e-4xxxx-9f95-xxxxxxx24f"

$thumb = "D4F02xxxxxx60422BxxxxxxxE60Fxxxxxxxx"

$appId = "14b57135-6bf6-451f-aabb-faa3822dd5e8"

Connect-AzureAD -TenantId $tenant -ApplicationId $AppId -CertificateThumbprint $thumb

Then execute the same cmdlet:

Get-AzureADServiceAppRoleAssignment -all $true -ObjectId "f984b219-756b-4454-bbf6-4290968836e4"

You’ll receive the API Permissions assigned to the App registration as shown in the following PowerShell output and screenshot:

Solution

The Microsoft documentation isn’t very clear (or maybe I’m not reading them properly) but all the examples show Get-AzureADServiceAppRoleAssignment as the cmdlet to use to list the Enterprise Application’s Users and Groups. However, if you’re using a service principal because you’re trying to automate the execution of a script, you actually need to use this cmdlet:

Get-AzureADServiceAppRoleAssignedTo
https://docs.microsoft.com/en-us/powershell/module/azuread/get-azureadserviceapproleassignedto?view=azureadps-2.0

Note the follow output when executing as a service principal:

Hope this helps anyone who might run into this issue as it took quite a big chunk of my weekend when I was troubleshooting this.

Also note that the cmdlet New-AzureADUserAppRoleAssignment to assign a user or group to the Enterprise Application does not change when using a service principal.

I’ve been working with a lot of automation in Azure lately and wanted to capture as many configuration activities I have gone through to share through my blog, and today’s post is to demonstrate how to set up a service principal with a secret that can be used when using Connect-AzAccount (https://docs.microsoft.com/en-us/powershell/module/az.accounts/connect-azaccount?view=azps-7.1.0), which can be used for automation activities.

The full script that I created can be found at my following Github repo: https://github.com/terenceluk/Azure/blob/main/PowerShell/Connect-AzAccount-with-Service-Principal.ps1

Below is the breakdown of the script and its process.

Begin by authenticating into Azure with:

Connect-AzAccount

Proceed to set the variables (name and password) for creating the App Registration and Enterprise Application that will be named "TestApp"

$spName = "TestApp"

$spPassword = "ChM7Q~fbYA934Q.nFxihDrSfBov3vqhh4g5OG"

$passwordLifeInYears = 3

Use the $spPassword variable containing the password we defined to create an object that is used to create a secret with an expiry date in the TestApp App Registration. For the purpose of this example, the secret will be valid for today’s date + 3 years.

$credentials = New-Object Microsoft.Azure.Commands.ActiveDirectory.PSADPasswordCredential `

-Property @{StartDate=Get-Date; EndDate=Get-Date -Year (Get-Date).AddYears($passwordLifeInYears).year; Password=$spPassword};

$spConfig = @{

DisplayName = $spName

PasswordCredential = $credentials

}

Use the cmdlet below to create the App Registration with a client secret configured and Enterprise Application (Service Principal) named TestApp:

$servicePrincipal = New-AzAdServicePrincipal @spConfig

With the cmdlet above executed, you should now be able to see the following TestAppApp Registration:

Note the Application ID as you’ll see the Enterprise Application reference it shortly:

Navigate to the Certificates & secrets area, then select Client secrets, and you’ll see a secret created for it. This secret contains the password we defined earlier but is not retrievable from this portal:

Navigate to the Enterprise Applications and you should see the TestApp created:

Note the same Application ID as the TestApp App Registration in the screenshot above confirming this Enterprise Application (Service Principal) is derived from it:

With the TestApp service principal created, we can now use it to authenticate with Connect-AzAccount but note that we’ll need the following:

1. The tenant ID we’re connecting to
2. The configured secret for the TestApp App Registration
3. The Service Principal Application ID, which can be found from the App Registration or Enterprise Application properties

Begin by defining the variables required to connect:

$tenantId = "84f4470b-3xxx-4xx9-xx95-abxxxxx3024f"

$spPassword = "ChM7Q~fbYA934Q.nFxihDrSfBov3vqhh4g5OG"

$servicePrincipalAppID = "b27b779c-02ab-4911-aab9-4a4f43a4be45" # This is the Application ID of the Enterprise App

Convert the Service Principal secret to secure string:

$password = ConvertTo-SecureString $spPassword -AsPlainText -Force

Create a new credentials object containing the application ID and password that will be used to authenticate:

$psCredentials = New-Object System.Management.Automation.PSCredential ($servicePrincipalAppID, $password)

Authenticate with the credentials object:

Connect-AzAccount -ServicePrincipal -Credential $psCredentials -Tenant $tenantId

Hope this helps anyone looking for instructions on how to do this as the instructions I found from Microsoft documentation did not work for me so changes had to be made to a few cmdlets.

Below is the full script pasted into this blog post:

<# Begin by creating the service principal and client secret #>

# Connect-AzAccount - https://docs.microsoft.com/en-us/powershell/module/az.accounts/connect-azaccount?view=azps-7.1.0

# Sign in with a service principal - https://docs.microsoft.com/en-us/powershell/azure/authenticate-azureps?view=azps-7.1.0#sign-in-with-a-service-principal

# Connect to Azure with an authenticated account for use with cmdlets from the Az PowerShell modules

Connect-AzAccount

# Set the variables (name and password) for creating the App Registration and Enterprise Application that will be named "TestApp"

$spName = "TestApp"

$spPassword = "ChM7Q~fbYA934Q.nFxihDrSfBov3vqhh4g5OG"

$passwordLifeInYears = 3

# Use the $spPassword variable containing the password to create an object that is used to create a secret in the TestApp App Registration

# The object will also specify when the date of this secret will expire and that is today's Month and Date in 3 years

$credentials = New-Object Microsoft.Azure.Commands.ActiveDirectory.PSADPasswordCredential `

-Property @{StartDate=Get-Date; EndDate=Get-Date -Year (Get-Date).AddYears($passwordLifeInYears).year; Password=$spPassword};

$spConfig = @{

DisplayName = $spName

PasswordCredential = $credentials

}

# Create the App Registration with a client secret configured and Enterprise Application (Service Principal) named test app

$servicePrincipal = New-AzAdServicePrincipal @spConfig

##############################################################################################################

<# Try connecting with the newly created Service Principal (Enterprise Application) named TestApp with a secret configured #>

# Define the variables required to connect

$tenantId = "84f4470b-3xxx-4xx9-xx95-abxxxxx3024f"

$spPassword = "ChM7Q~fbYA934Q.nFxihDrSfBov3vqhh4g5OG"

$servicePrincipalAppID = "b27b779c-02ab-4911-aab9-4a4f43a4be45" # This is the Application ID of the Enterprise App

# Convert the Service Principal secret to secure string

$password = ConvertTo-SecureString $spPassword -AsPlainText -Force

# Create a new credentials object containing the application ID and password that will be used to authenticate

$psCredentials = New-Object System.Management.Automation.PSCredential ($servicePrincipalAppID, $password)

# Authenticate with the credentials object

Connect-AzAccount -ServicePrincipal -Credential $psCredentials -Tenant $tenantId

As a follow up to one of my previous posts:

PowerShell script to assign users in an on-premise AD group to an Azure Enterprise Application's Users and Groups
http://terenceluk.blogspot.com/2022/02/powershell-script-to-assign-users-in-on.html

A PowerShell script can be used to manually add user accounts to an Azure Enterprise Application’s User and Groups, and to take it one step further, it can be automated with Azure Automation Runbooks or Azure Functions. This post serves as a continuation of how to automate PowerShell scripts.

One of the first and most important task to do when automating a PowerShell script is to handle how authentication happens. Interactive logon with modern or legacy basic authentication is not and never really an option. The recommended method for scripts is to create a service principal and either use a secret (see my previous post: http://terenceluk.blogspot.com/2022/02/using-azure-service-principal-to.html or certificate (as shown in this post) to authenticate.

Microsoft has documentation (https://docs.microsoft.com/en-us/powershell/azure/active-directory/signing-in-service-principal?view=azureadps-2.0) for how to configure a Service Principal with a certificate to authenticate but I feel the process can be elaborated it a bit so below is a detailed walkthrough of what happens.

Before I begin, the PowerShell script can be found here at my Github repo: https://github.com/terenceluk/Azure/blob/main/PowerShell/ServicePrincipalCertificateAuthentication.ps1

Step #1 – Connect to Azure AD and define the variables:

Start by importing the AzureAD module and authenticating with your Global Admin account, then define the variables with values that we’ll be using later on to create the certificate.

# Import AzureAD module with -UseWindowsPowerShell switch for PowerShell 7

Import-Module AzureAD -UseWindowsPowerShell

# Login to Azure AD PowerShell With Admin Account

Connect-AzureAD

# Define variables that will be used to create the certificate and export it

$password = "YourPassword123$" # certificate password

$certDNSname = "EnterpriseApps.contoso.com" # the CN (common name) of the certificate

$certificateLifeInYears = 3

$certificateExportPath = "c:\temp\"

$certificateExportName = "enterpriseAppsCert.pfx"

$certificatePathandName = $certificateExportPath + $certificateExportName

$enterpriseAppName = "Enterprise-Apps-Permissions-PS" # This is the App Registration object that the Enterprise Application created from

Step #2 – Create a self-signed certificate on the local Windows desktop and export it to PFX with the private key

Create a self-signed certificate on the local Windows desktop these cmdlets are executed and then export it to the computer’s local C:\temp as a PFX:

# Create the self signed cert on the local Windows machine's Computer Store (Not User Store)

$currentDate = Get-Date

$notAfter = $currentDate.AddYears($certificateLifeInYears)

$thumb = (New-SelfSignedCertificate -CertStoreLocation cert:\localmachine\my -DnsName $certDNSname -KeyExportPolicy Exportable -Provider "Microsoft Enhanced RSA and AES Cryptographic Provider" -NotAfter $notAfter).Thumbprint

### Certificate is now created in the computer's local store

# ^^^ Convert the password to a secure string and export the newly created certificate on the computer's Computer Store to a PFX file

$password = ConvertTo-SecureString -String $password -Force -AsPlainText

Export-PfxCertificate -cert "cert:\localmachine\my\$thumb" -FilePath $certificatePathandName -Password $password

### ^^^ Certificate now exported to the local computer

Step #3 – Create the App Registration and Enterprise Application (Service Principal) objects

Begin by creating the certificate as an object, create the App Registration, upload the certificate, create the service principal that is linked to the app registration:

# Create and load the certificate into variables

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate($certificatePathandName, $password)

$keyValue = [System.Convert]::ToBase64String($cert.GetRawCertData())

# Create the Azure Active Directory Application (Azure Active Directory > App Registrations) and upload certificate during creation

$application = New-AzureADApplication -DisplayName $enterpriseAppName # include URL if there is one or omit it -IdentifierUris "https://TestApp"

New-AzureADApplicationKeyCredential -ObjectId $application.ObjectId -CustomKeyIdentifier $enterpriseAppName -StartDate $currentDate -EndDate $notAfter -Type AsymmetricX509Cert -Usage Verify -Value $keyValue

### ^^^ App registrations should now show the newly created application with the certificate uploaded to "Certificates and secrets" so it would allow a client with the

### certificate and its private key to authenticate

# Create the Service Principal (Enterprise Application) and linked to the Application (App Registration)

$sp=New-AzureADServicePrincipal -AppId $application.AppId

### ^^^ Enterprise Applications should now show the newly created application with filter set to "All Applications"

Step #4 – Assign Roles to the Service Principal

The service principal will not have any permissions to the directory so grant the required permissions:

# Give the Service Principal Reader access to the current tenant (Use Get-AzureADDirectoryRole to list the roles - Azure Active Directory > Roles and Administration > Directory readers)

Add-AzureADDirectoryRoleMember -ObjectId (Get-AzureADDirectoryRole | where-object {$_.DisplayName -eq "Directory Readers"}).Objectid -RefObjectId $sp.ObjectId

# Give the Service Principal Reader access to the current tenant (Use Get-AzureADDirectoryRole to list the roles - Azure Active Directory > Roles and Administration > Application administrator)

Add-AzureADDirectoryRoleMember -ObjectId (Get-AzureADDirectoryRole | where-object {$_.DisplayName -eq "Application administrator"}).Objectid -RefObjectId $sp.ObjectId

### ^^^ The Enterprise App (Service Principal) should now be added to the "Application administrator" and "Directory Readers" roles

Step #5 – Test authentication with Service Principal

With the service principal configured, test logging in as it with the same computer that has the certificate used for authentication:

<#

We can now test authentication with the Service Principal and the certificate stored on the local computer we have just created

#>

# Get Tenant Detail

$tenant=Get-AzureADTenantDetail

# Now you can login to Azure PowerShell with your Service Principal and Certificate

Connect-AzureAD -TenantId $tenant.ObjectId -ApplicationId $sp.AppId -CertificateThumbprint $thumb

Please see my next post for how to create an Azure Function App where it will use this service principal authenticate and run a PowerShell script on a timer trigger.

Problem

You’ve completed configuring an Azure Function App with a PowerShell script that authenticates against Azure AD with Connect-AzureAD using a certificate but noticed that attempting to use the Test/Run button shows that the authentication fails with the following:

The full error log output is the following:

2022-02-07T12:12:42.895 [Error] ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMicrosoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADException: CertificateNotFoundInStoreStack:

2022-02-07T12:12:43.179 [Error] ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMicrosoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADException: CertificateNotFoundInStoreStack:

2022-02-07T12:12:43.269 [Information] Executed 'Functions.Update-Compliance-Cloud-Users' (Succeeded, Id=7e7de1c7-1674-4144-a68a-92417853733f, Duration=4742ms)

2022-02-07T12:12:47.153 [Information] Executing 'Functions.Update-Compliance-Cloud-Users' (Reason='This function was programmatically called via the host APIs.', Id=4f8eff73-c3e7-424a-9f82-96555ab7d1a2)

2022-02-07T12:12:47.266 [Warning] WARNING: Module AzureAD is loaded in Windows PowerShell using WinPSCompatSession remoting session; please note that all input and output of commands from this module will be deserialized objects. If you want to load this module into PowerShell Core please use 'Import-Module -SkipEditionCheck' syntax.

2022-02-07T12:12:47.642 [Error] ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMicrosoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 163SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : AuthenticationError: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : Connect-AzureAD,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADException: CertificateNotFoundInStoreStack:

2022-02-07T12:12:47.885 [Error] ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMicrosoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: ERROR: CertificateNotFoundInStoreOriginInfo : localhostException :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord :Exception :Type : System.Management.Automation.RemoteExceptionSerializedRemoteException : System.ArgumentException: CertificateNotFoundInStoreat Microsoft.Open.Azure.AD.CommonLibrary.DiskDataStore.GetCertificate(String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\DataStore\DiskDataStore.cs:line 117at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.AcquireTokenWithCertificate(AdalConfiguration config, String appId, String thumbprint) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 62at Microsoft.Open.Azure.AD.CommonLibrary.ServicePrincipalTokenProvider.GetAccessTokenWithCertificate(AdalConfiguration config, String clientId, String certificateThumbprint, AccountType credentialType) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\TokenProvider\ServicePrincipalTokenProvider.cs:line 40at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\AuthenticationFactory.cs:line 59at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 127at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\RMProfileClient.cs:line 55at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\bt\1218881\repo\src\dev\PowerShell.V2\CommonLibrary\ConnectAzureAD.cs:line 193at System.Management.Automation.CommandProcessor.ProcessRecord()SerializedRemoteInvocationInfo : System.Management.Automation.InvocationInfoErrorRecord : …Message : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADMessage : CertificateNotFoundInStoreHResult : -2146233087CategoryInfo : NotSpecified: (:) [Connect-AzureAD], ArgumentExceptionFullyQualifiedErrorId : System.ArgumentException,Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureADException: CertificateNotFoundInStoreStack:

Solution

There could be various reasons why this error is thrown but the components to check is ensure that the certificate used for authentication is uploaded into the Function App via TLS/SSL settings > Private Key Certificates (.pfx) > Upload Certificate:

Verify that the thumbprint used in the Connect-AzureAD is correct.

Another reason that this could happen is if you had forgotten to add the Application setting:

WEBSITE_LOAD_CERTIFICATES

The Microsoft documentation can be found here: https://docs.microsoft.com/en-us/azure/app-service/configure-ssl-certificate-in-code#make-the-certificate-accessible

Below is a screenshot of the application setting:

Upon successfully getting the above configured, the Test/Run should execute successfully with the output below:

As stated in the following Microsoft documentation:

How Exchange Online uses TLS to secure email connections
https://docs.microsoft.com/en-us/microsoft-365/compliance/exchange-online-uses-tls-to-secure-email-connections?view=o365-worldwide

By default, Exchange Online always uses opportunistic TLS. Opportunistic TLS means Exchange Online always tries to encrypt connections with the most secure version of TLS first, then works its way down the list of TLS ciphers until it finds one on which both parties can agree. Unless you have configured Exchange Online to ensure that messages to that recipient must use secure connections, then by default the message will be sent without encryption if the recipient organization doesn't support TLS encryption. Opportunistic TLS is sufficient for most businesses. However, for businesses that have compliance requirements such as medical, banking, or government organizations, you can configure Exchange Online to require, or force, TLS.

I have worked with organizations in the past which have had extremely strict requirements for messaging delivery and have asked for connectors to be configured to force inbound and output TLS connections. For inbound connections, it could be a bit labour intensive if there is a long list of domains to be configured. For outbound connections, it can also be laborious to configure multiple domains with corresponding smarthosts to for where to establish a TLS connection and send the email (this covers organizations that do not have SPF records configured).

Manually configuring inbound and outbound connectors with the GUI may not be the best for an abundance of domains so I’ve created the following two scripts that uses an Excel file to import the configuration settings.

Inbound Connector

The purpose of this script is to create an inbound connector for Exchange Online that forces the defined incoming domains to require TLS.

This script will import a list of domains from an Excel spreadsheet with a column named domains.

https://github.com/terenceluk/Microsoft-365/blob/main/Exchange-Online/Create-Inbound-Connector-For-TLS.ps1

The used for the spreadsheet should look as such:

Note that there is a limit on the length of sender domains passed so if there are too many domains then you’ll need to split them out:

Exception: Cannot bind parameter 'SenderDomains' to the target. Exception setting "SenderDomains": "SenderDomainString: The length of the

property is too long. The maximum length is 2243 and the length of the value provided is 5666."

Outbound Connector

The purpose of this script is to create an outbound connector for Exchange Online that forces the defined destination domains to require TLS and use a defined smarthost.

This script will import a list of domains from an Excel spreadsheet with a column named domains.

https://github.com/terenceluk/Microsoft-365/blob/main/Exchange-Online/Create-Outbound-Connector-For-TLS.ps1

The used for the spreadsheet should look as such:

As a follow up to my previous post:

Creating a Service Principal to connect to Azure AD using a Certificate to authenticate
http://terenceluk.blogspot.com/2022/02/creating-service-principal-to-connect.html

This post serves to demonstrate how to create a Function App to automate the execution of a PowerShell script that is ran via a timer trigger.

Step #1 – Create the Function App

Begin by creating a Function App in the Azure portal:

Provide the required parameters:

Resource Group: <Name of resource group to place function app>

Function App name: <Name of Function App>

Publish: Code

Runtime stack: PowerShell Core

Version: 7.0

Region: <Desired region>

Select the desired storage account and the plan type:

As this is the consumption plan, the Enable network injection is not available:

Application Insights is not available for the consumption plan:

Proceed to create the Function App:

Step #2 – Update Function App platform to 64 Bit

The default PowerShell Core settings for the Function App is set to 32-bit so navigate to Configuration > General settings > Platform, change the setting to 64-Bit and save:

Step #3 – Configure application settings

It is imperative that we do not include any sensitive parameters such as the tenant ID, application ID and certificate thumbprint in the PowerShell script to authenticate against Azure AD so these can be configured in the Function App’s Application settings as they would be encrypted and transmitted over an encrypted channel. Navigate to Configuration > Application settings then click on New application setting:

Configure the following 3 settings:

1. tenant
2. WEBSITE_LOAD_CERTIFICATES
3. appId

Note that the second application setting named WEBSITE_LOAD_CERTIFICATES represents the certificate thumbprint the PowerShell script will use to authenticate against Azure AD and the reason why it is named as such is because the name will have the Azure Function load the certificate PFX when it starts. We’ll be referencing these variables within our PowerShell script. Proceed to save the new settings.

You should now see the 3 settings configured:

Step #4 – Load the exported PFX of the certificate used to authentication against Azure AD

With the application settings configured, proceed to upload the PFX export of the certificate that will be used by the PowerShell script to authenticate against Azure AD. Navigate to TLS/SSL settings > Private Key Certificates (.pfx) and click on Upload Certificate:

Proceed to upload the exported PFX certificate:

The certificate should be displayed under Private Key Certificates:

Step #5 – Define modules to load so the script can import them at runtime

Modules required by the PowerShell script will need to be loaded by the Function App. Proceed and navigate to App files then select requirements.psd1 from the drop down list:

From within this file, define the modules to load. For the purpose of this example, we will load the latest version 2 AzureAD module: https://www.powershellgallery.com/packages/AzureAD/2.0.2.140

Proceed to save the configuration:

Step #6 – Create the Function

Proceed to create the actual function containing the PowerShell script by navigating to Functions > Create, then enter the appropriate settings:

Development environment: we’ll use the portal for this example

Template: we’ll configure this function with a Timer trigger

New Function: Enter a name for the function

Schedule: As this is timer triggered, we’ll need to use NCRONTAB expressions to set the timer schedule (https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-timer?tabs=csharp#ncrontab-expressions)

For the purpose of this example, the timer will be set to run every 3 hours Monday to Friday between 8a.m. to 6p.m. (8a.m., 11a.m., 2p.m., 5p.m.) so the NCRONTAB expression for this will be: 0 0 8-18/3 * * 1-5

Sample schedule:

2022-02-08 08:00:00

2022-02-08 11:00:00

2022-02-08 14:00:00

2022-02-08 17:00:00

2022-02-09 08:00:00

2022-02-09 11:00:00

2022-02-09 14:00:00

2022-02-09 17:00:00

2022-02-10 08:00:00

2022-02-10 11:00:00

2022-02-10 14:00:00

2022-02-10 17:00:00

2022-02-11 08:00:00

2022-02-11 11:00:00

2022-02-11 14:00:00

2022-02-11 17:00:00

2022-02-14 08:00:00

2022-02-14 11:00:00

2022-02-14 14:00:00

2022-02-14 17:00:00

Click on the create button to create the function.

Note that the default timezone used with the CRON expressions is Coordinated Universal Time (UTC) and if the schedule defined is to be ran on a different timezone, use the WEBSITE_TIME_ZONE app setting to change the timezone. Refer to the following document for more information: https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-timer?tabs=csharp#ncrontab-time-zones

Step #7 – Configure the PowerShell script to run

Navigate into the Code + Test and switch to the run.ps1 script:

There will be a sample script provided and we’ll only need the first 2 lines:

# Input bindings are passed in via param block.

param($Timer)

Remove the remaining lines and add the lines declaring and setting the variables that will reference the application settings and the Connect-AzureAd cmdlet to connect to Azure AD:

Import-Module AzureAD -UseWindowsPowerShell

$tenant = $ENV:tenant

$thumb = $ENV:WEBSITE_LOAD_CERTIFICATES

$appId = $ENV:appId

Connect-AzureAD -TenantId $tenant -ApplicationId $AppId -CertificateThumbprint $thumb

As shown above, the $ENV: prefix is used to reference the application settings configured for the Function. At this point, save the function, so we can use the Test/Run button to test connectivity to the tenant:

The output should display a successful connection to Azure AD:

Now that we have confirmed connectivity, proceed to add the rest of the PowerShell script to the run.ps1 code and test that the Azure Function executes and completes as desired.

One of the common topics I am asked about Azure is how to best monitor the resources with Azure Monitor. There isn’t really a short answer for his because Azure Monitor encompasses a full suite of components that that collect, analyze, and act on telemetry from cloud and on-premises environments. Barry Luijbregts has a great walkthrough of the when and what when monitoring applications and services in an Azure Fridays video:

What to use for monitoring your applications in Azure | Azure Friday
https://www.youtube.com/watch?v=Zr7LcSr6Ooo

In the video, he provided a matrix of mappings for the monitoring options in Azure and the use case (I recreated the table with newer icons):

I intend on finding time to write a series of blog posts to demonstrate the monitoring options and today I would like to start with how to monitor and alert for an Azure virtual machine with Azure Monitor. The following are the topics I’ll be covering for this blog post.

1. Create and configure a Log Analytics Workspace
2. Enable Insights for a virtual machine
3. Configure Log Analytics to collect logs (Windows event logs, Windows performance counters, Linux performance counters, Syslog and IIS Logs)
4. Query Log Analytics with Kusto query language (KQL)
5. Setting up an Alert to notify an email address based on the Kusto query
6. Monitor Virtual Machine with the Activity Log

Create and configure a Log Analytics Workspace

Before proceeding to enable monitoring on the virtual machines, we’ll should create and configure a Log Analytics Workspace for data ingestion. A single subscription can use any number of workspaces depending on the requirements, you can have multiple virtual machines and resources logging into the same Log Analytics workspace, and the only requirement of the workspace is that it be located in a supported location and be configured with the VMInsights solution. Once the workspace has been configured, you can use any of the available options to install the required agents on virtual machine(s) and virtual machine scale set and specify a workspace for them to send their data. VM insights will collect data from any configured workspace in its subscription.

Navigate to the Log Analytics workspaces service in the Azure portal:

Create a new workspace:

We’ll be back here to configure the settings after onboarding the virtual machines.

Enable Monitoring for Virtual Machine

There are several ways to onboard the virtual machines into Azure Monitor and install the required agents on the VMs.

You can navigate to the Virtual machines section in the Log Analytics workspace and connect the virtual machine but this will not enable Insights:

Then click on the Connect button:

The alternative, which I prefer because it automatically turns on Insights is to navigate into the Monitor service:

Then to Virtual Machines, into the Not monitored tab and either click into the VM or click on the Enable button:

You will be asked to select the Log Analytics Workspace that was created in the previous step or use a default one:

The agent will take a few minutes to install and the following will displayed within the Windows Programs and Features when completed:

See the following documentation for more information about agents and their features: https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agents-overview

**Note that Dependency Agent will not be installed if the VM was added by connecting to the Log Analytics workspace. To get the Dependency Agent installed, enable Insights by navigating into the virtual machine to the monitored:

Under Monitoring, click on Insights and then Enable:

Configure Log Analytics to collect logs (Windows event logs, Windows performance counters, Linux performance counters, Syslog and IIS Logs)

Log Analytics will automatically ingest virtual machine metrics provided by the host but not anything within the operating system. This means that if you load up a query window in the Logs, you can query for heartbeat as such:

Heartbeat

| where Resource == "<vmname>"

**Note that we’re using resource for the query rather than computer as the latter is the FQDN of the Windows virtual machine.

But you won’t be able to query for any information in, say, the event logs of the Windows operating system.

In order to be able to query the event logs, you’ll need to navigate to the Settings section then into the Agents configuration:

You’ll be able to add Windows event logs, Windows performance counters, Linux performance counters, Syslog and IIS Logs. For the purpose of this example, we’ll add the Application and Systemlogs for Windows but remember to add other logs (e.g. Performance) or attempting to query for them will yield no results:

Query Log Analytics with Kusto query language (KQL)

With Log Analytics configured and Windows Event Logs collected, we can now test querying for the data captured. There are plenty of Kusto query blog posts available so I won’t go into the details of syntax or a large variety of queries. For the purpose of this example, we’re going to query for a specific event in the system logs that is written when a service named Advanced Monitoring Agent is abruptly terminated as shown in the following screenshot:

The Advanced Monitoring Agent service terminated unexpectedly. It has done this 1 time(s). The following corrective action will be taken in 60000 milliseconds: Restart the service.

To query for this event in Log Analytics, navigate into the Log Analytics workspace for the virtual machine, scroll to the General heading and navigate to Logs:

A new empty query window will be displayed:

The query we’ll execute is the following:

Event

| where EventLog == "System" and EventID == 7031 and Source == 'Service Control Manager' and EventLevelName == "Error" and ParameterXml contains "Advanced Monitoring Agent" and TimeGenerated > ago(5m)

| parse kind=relaxed EventData with * '<Data Name="param1">' Windows_Service_Name '</Data><Data Name="param2">' Windows_Service_State '</Data>' *

| project Computer, Windows_Service_Name, Windows_Service_State, EventLevelName, TimeGenerated

The query’s requirements (where line) are as follow.

Event needs to have all of these properties:

1. Event log is from System
2. The Event ID is 7031
3. The Source of the event is Service Control Manager
4. The Event is an error
5. The field containing the service name needs to contain Advanced Monitoring Agent (this field contains more than the name)
6. The TimeGenerated must be within minutes (we are comparing the timestamp of the event with the timestamp derived by subtracting 5 minutes from the time now)

**Note that I only have one virtual machine configured for this Log Analytics workspace but most will likely have more than one computer so the following additional where should be included so only the intended VM is returned:

and Computer == 'computerName.contoso.com'

The purpose of the parse line is to retrieve the values between the param1 and param2 tag, name them with Windows_Service_Name and Windows_Service_State so they can be used in the following project line.

The purpose of the project line is to display the following 5 fields when the record is expanded:

1. Computer
2. Windows_Service_Name
3. Windows_Service_State (1 is stopped)
4. EventLevelName
5. TimeGenerated (UTC)

The following are the fields mappings found in the event details tab:

Setting up an Alert based on the Kusto query

Setup an Action Group

Before we can set up an alert, we’ll need to set up an Action Group, which will be associated to the an Alert Rule. Navigate to Monitor > Alerts > Create > Action Group:

Fill in the appropriate fields for the action group and click on Notifications:

We will be configuring an email notification for this action group so select:

Notification type: Email/SMS message/Push/Voice
Name: A name for the notification type
Email: The email address for the user or group to notify

We will be creating an action in a separate post but the Actions tab is where we can specify an action to execute. There are quite a few actions available so the only limitation is your creativity:

• Automation Runbook
• Azure Function
• Event Hub
• ITSM
• Logic App
• Secure Webhook
• Webhook

Navigate to the Review + create tab to review the action group and use the Test action group (preview) feature to test the alerting. Note that it doesn’t matter what sample type is selected for the test as anyone of them would send an email:

Proceed with the test:

The test should complete and a test email should come in shortly:

Complete creating the Action Group and it should show up in a minute or two:

An email should also be sent to the email address configured indicating it has been added to an Action Group:

Setup an Alert Rule

With the Action Group created, we can now proceed to create the alert. There are several ways to create the Alert Rule and depending on where it is initiated, additional options would be required.

Option #1 – Azure Monitor Level

If you create the Alert Rule at the Azure Monitor level via Monitor > Alerts > Create > Alert Rule then you’ll be asked to select the scope at the subscription level:

Option #2 – Log Analytics Level

If you already know which Log Analytics Workspace this alert rule is going to be created, you can navigate directly into the Log Analytics Workspace via Log Analytics Workspaces > TheLogAnalyticsWorkspace > Alerts >Create > Alert Rule:

Note that the tab displayed is Condition as the Scope is already filled out:

You can navigate to the Scope tab and see the defined settings prepopulated with the Log Analytics Workspace:

For the purpose of this example, we’ll be using a custom Kusto query so select Custom log search:

The log query window will be displayed and for the purpose of this example, I’ll be using the same query demonstrated above:

Event
| where EventLog == "System"
and EventID == 7031
and Source == 'Service Control Manager'
and EventLevelName == "Error"
and ParameterXml contains "Advanced Monitoring Agent"
and TimeGenerated > ago(5m)
and Computer == 'computerName.contoso.com'
| parse kind=relaxed EventData with * '<Data Name="param1">' Windows_Service_Name '</Data><Data Name="param2">' Windows_Service_State '</Data>' *
| project
Computer,
Windows_Service_Name,
Windows_Service_State,
EventLevelName,
TimeGenerated

Proceed to execute the query to validate it with the Run button:

Click on the Continue Editing Alert to close the validated query:

The Condition options will be displayed:

Whether to use the Measurement and Split by dimensions are dependent what we are querying, but for this example, we’re only going to configure the Alert logic to:

Operator: Greater than
Threshold value: 0
Frequency of evaluation: 5 minutes

Our query is supposed to return any unexpected service terminations that are within 5 minutes:

Navigate to the Actions tab and select the previously created Action Group:

Select the Details tab and fill in the fields as required:

We’ll be configure this as a critical severity:

Proceed to create the rule:

Once created, navigate to Alert rules to view the newly created rule:

With the rule creation confirmed, proceed to test. I’ll be terminating the service as such:

The following event ID 7031 should be logged:

You should receive an alert within a few minutes:

And this is how you would monitor a Windows VM with Log Analytics. One of the potential challenges for using event logs to monitor Windows Services is that you need to know exactly the log you’re looking for. Another method for monitoring Windows services, which I feel is more efficient, is to use Change Tracking and Inventory: https://docs.microsoft.com/en-us/azure/automation/change-tracking/overview which I will demonstrate in another post.

Monitor Virtual Machine with the Activity Log

Having gone through using Log Analytics to create a custom query to determine the health of a virtual machine, I’d like to provide another method where instead of using the Signal type of a Log, we can use the Activity Log instead to trigger an alert. Recall that when we create an alert via Virtual Machine > Alerts > Create > Alert rule:

We are presented with the Condition configuration where we select a signal. The signal is essentially the incoming information source that triggers alert. Other than the Custom log search, which we used previously to provide a custom Kustoquery, there are also the options:

Metric– Virtual metrics that the platform / hypervisor collects
ActivityLog– The log entries entered into the Activity Log of the virtual machine

For the purpose of this example, we’ll be using the Signal type Activity Log Power Off Virtual Machine (Microsoft.Compute/virtualMachines) as shown in the screenshot below:

It is important to note that:

Power Off Virtual Machine (Microsoft.Compute/virtualMachines)

… is not the same as:

Deallocate Virtual Machine (Microsoft.Compute/virtualMachines)

The difference is that if you use the Stop button in the portal UI, it will deallocate the virtual machine and stop the compute instance charges for the VM:

The Power Off Virtual Machine (Microsoft.Compute/virtualMachines) does not fire off an alert when the Stop button is used because of it is looking for scenarios where, say, the following Azure CLI command is executed:

az vm stop --resource-group rg-prod-infraServers --name dc03

The command above will stop the operating system but charges for the instance will continue, which is probably not what is intended.

----------------------------------------------------------------------------------------------------------------------------

With the monitoring signal type explained, let’s proceed to finishing creating the alert by leaving the rest of the condition default:

Select the previous action group we defined, which sends an email, for the action group:

Fill in the fields for the Details tab:

Review and create the alert:

I’ve noticed that the alert can take a few minutes to show up in the portal so give it a bit of time:

With the alert configured, proceed to power off (not deallocate) the VM:

You should see the following entry logged in the Activity Log for the virtual machine:

Clicking into the Alerts for the virtual machine will display the alert generated by powering off the VM:

You should also receive an email notification alert:

I hope this blog post is able to provide information on how Azure Monitor works when monitoring a virtual machine and a clear demonstration on how to set up the monitoring and alerting. A follow up post for automation will be written soon.

In my previous post:

Monitor and Alerting for an Azure Virtual Machine with Azure Monitor
Terence Luk: Monitor and Alerting for an Azure Virtual Machine with Azure Monitor

I demonstrated how to set up Log Analytics to monitor the event log for a system event ID 7031 with a specific string that represents a Windows Service we wanted to monitor for and detect if it stops. While this method is certainly a viable option, it isn’t very straight forward if you’re not familiar with Windows and do not know what system events are triggered when a service stops. Case in point, when a service is abruptly terminated, an event ID 7031 error is logged but if the service is gracefully stopped, the event logged will be a 7036 information. Having to capture all types of events with a query leaves a lot of room for error so I would like to demonstrate a different method for monitoring Windows or Linux services.

The Azure feature I typically use to monitor services from within a virtual machine is Change Tracking and Inventory in Azure Automation. This feature tracks changes in virtual machines hosted in Azure, on-premises, and other cloud environments. Items that are tracked by Change Tracking and Inventory include:

• Windows software
• Linux software (packages)
• Windows and Linux files
• Windows registry keys
• Windows services
• Linux daemons

Change Tracking and Inventory overview

https://docs.microsoft.com/en-us/azure/automation/change-tracking/overview

I find this feature extremely powerful and opens up many monitoring opportunities for all sorts of use cases. For the purpose of this example, we’ll use it to monitor Windows service status.

Creating an Automation Account

The Change Tracking and Inventory is a feature of Azure Automation and you’ll therefore need to create an Automation account that is linked to a Log Analytics workspace. Begin by navigate to Automation Accounts:

Then create an automation account:

Enabling Change Tracking for the Automation Account

Navigate to Configuration Management > Change Tracking, select a supported Log Analytics workspace, then click Enable:

The following console will be displayed once the deployment has successfully completed:

Adding VMs for Change Tracking

With Change Tracking ready in the Automation Account, proceed to add the VMs:

Select the virtual machine(s) you would like to enable Change Tracking and then click Enable:

With the virtual machine added, proceed to adjust the settings by clicking on Edit Settings:

Navigate to Windows Services and note how the frequency is set to 30 minutes:

This is likely not frequent enough and for the purpose of this example, we’ll use the lowest frequency of 10 seconds to collect the Windows Services changes:

With the change tracking configured, proceed to stop the service you intend to test with:

Refresh the Change tracking console and you should see the Windows Services change logged:

Note the details for the Advanced Monitoring Agent service we stopped:

Proceed to click on the Log Analytics button:

The query window will automatically execute the ConfigurationChange query without any other requirements, which will provide all results for any configuration change. We’ll refine it to only list the service we want to track:

ConfigurationData
| where SvcName =~ "Advanced Monitoring Agent"
| project SvcName, SvcDisplayName, SvcState, TimeGenerated
| where SvcState != "Running"

Now that we have a query to search for a specific service, we can create a new alert by clicking the New alert rule button:

Proceed to configure the Condition settings:

Update the Thresholdvalue to 0 and Frequency of evaluation to 1 minute to capture any service status that is not “Running”:

Select an action group for the notification:

Fill in the details for the rule:

Complete creating the rule:

As with all rules, it may take a bit of time before it shows up in the console:

Note that although the rule was created within the Automation Account, it is actually configured and associated with the Log Analytics Workspace of the Automation Account:

Proceed to test stopping the monitored service and you should see an email notification similar to the one below:

Hope this provides a good overview of how to use Change Tracking and Inventory to monitor Windows Services. What’s great about this feature is that it allows you to track other changes such as file and registry, which opens up so many possibilities for monitoring.

Having monitoring and alerting set up for failed login attempts to any identity directory services (e.g. on-premise AD and Azure AD) have always been important yet often times neglected for many environments I’ve worked in. Those who have worked with on-premise Active Directory would know the pain of going through a domain controller’s security event logs and how difficult it is to obtain what you need as all security events are recorded. This is why there are third party monitoring and alerting products that have had so much success for years. The built-in Azure AD Sign-in Logs provides a display of sign-in attempts for troubleshooting but is also rarely reviewed and audited. If configuration has been made for the Azure AD logs, the retention can range from 7 days (Azure AD Free) to 30 days (Azure AD Premium P1 or P2) as stated here in the documentation: https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/reference-reports-data-retention#how-long-does-azure-ad-store-the-data

I’ve always bundled in the step for increasing the retention of these logs in my projects and the action required to ensure logs are retained for a longer retention is to route them to an Azure storage account using monitor as described in the following Microsoft documentation: https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/quickstart-azure-monitor-route-logs-to-storage-account. With the retention of the logs in place, the next critical component is to have monitoring and alerting in place to capture events that are of concern to the organization. A common use case is to monitor the first global admin account created for the tenant as that is usually one that isn’t associated with a particular user and may be used as the Emergency Break Glass Account that does not have MFA. Another common monitoring that should be set up is for potential malicious login attempts.

With the above in mind, what I would like to do in this post is demonstrate how to:

1. Set up monitoring of Azure AD with Log Analytics
2. Set up an alert using Kusto to query Azure AD Sign-In Logs
3. Set up reporting of Azure AD failed sign in attempts with Logic Apps

Configure Log Analytics Workspace for Azure AD

I won’t go into how to create a Log Analytics workspace so proceed to create one that will be used to ingest Azure AD logs as such:

Proceed to navigate into Azure Active Directory > Diagnostic settings and then click on Add diagnostic setting:

The following are the log category options:

• AuditLogs
• SignInLogs
• NonInteractiveUserSignInLogs
• ServicePrincipalSignInLogs
• ManagedIdentitySignInLogs
• ProvisioningLogs
• ADFSSignInLogs
• RiskyUsers
• UserRiskEvents
• NetworkAccessTrafficLogs
• RiskyServicePrincipals
• ServicePrincipalRiskEvents

For the purpose of this example, we’ll just capture the AuditLogs and SignInLogs and click on the Save button:

Note the following requirement:

In order to export Sign-in data, your organization needs Azure AD P1 or P2 license. If you don't have a P1 or P2, start a free trial.

With the AuditLogs and SignInLogs configured for collection, navigate to the corresponding Log Analytics workspace > Usage and estimated costs > Data Retention to configure the required retention:

Set up an alert using Kusto to query Azure AD Sign-In Logs

As mentioned earlier, a common use case is to monitor the first global admin account created for the tenant as that is usually one that isn’t associated with a particular user and may be used as the Emergency Break Glass Account that does not have MFA. To set this up, navigate to Monitor > Alerts > Create > Alert rule:

Select the log analytics workspace as the scope:

Select Custom log search for the Condition so we can define our own Kusto query:

The following query can be used to look for break glass account login:

SigninLogs
|where UserPrincipalName contains "cspadmin@contoso.onmicrosoft.com"

With the Condition defined, proceed to assign an Action Group that will alert the appropriate group an appropriate communication such as email.

Set up reporting of Azure AD failed sign in attempts with Logic Apps

Another common scenario I’ve come across is to provide a report of failed sign-in attempts to a group of administrators to review on either a hourly, daily or weekly basis. A frequent report can be very useful for an active attack while daily and weekly can be great for monitoring attacks. For the purpose of this example, I will use a directory with the following failed sign-in attempts from all sorts of Asia countries for a user:

The kusto query we’ll be using to look for failed sign-ins is the following:

SigninLogs

| where Status.errorCode != 0

| extend City=LocationDetails.city, State=LocationDetails.state, Country=LocationDetails.countryOrRegion, Error_Code=Status.errorCode, Failure_Reason=Status.failureReason

| project TimeGenerated, UserDisplayName, AppDisplayName, IPAddress, City, State, Country, AuthenticationRequirement, Failure_Reason, ConditionalAccessStatus, ConditionalAccessPolicies, Error_Code

You are free to adjust the query to include or exclude additional fields of the failed sign-in attempts.

With the kusto query created and tested, the next step is to create a Logic App that will generate and send a report to an email address for administrators to review. Create a Logic App as such:

**Note that the Enable log analytics option for the creation of the Logic App is to get richer debugging information about the logic apps during runtime.

Next, navigate into the Logic app and click on Logic app designer:

We’ll be creating 3 steps for this Logic App where:

1. Recurrence: This will configure a recurring schedule for this Logic App to execute
2. Run query and visualize results: This will allow us to run the Kusto query, set a Time Range and specify a Chart Type
3. Send an email (V2): This will allow us to send the Kusto query results via email

Recurrence:

I wanted to send this report every day at 5:00p.m. EST:

Run query and visualize results:

The query I wanted to execute is:

SigninLogs

| where Status.errorCode != 0

| extend City=LocationDetails.city, State=LocationDetails.state, Country=LocationDetails.countryOrRegion, Error_Code=Status.errorCode, Failure_Reason=Status.failureReason

| project TimeGenerated, UserDisplayName, AppDisplayName, IPAddress, City, State, Country, AuthenticationRequirement, Failure_Reason, ConditionalAccessStatus, ConditionalAccessPolicies, Error_Code

The time range I wanted to query for is the last 12 hours and the Chart Type I want is an HTML Table:

Send an email (V2):

The report will include the Attachment Content and Attachment Name derived from the query with the subject Failed Login Report. The email will look pretty barebone so you are free to add HTML code to pretty it up.

Proceed to save the Logic App:

Use the Run Trigger to test the Logic App and confirm that an email is sent:

Hope this helps anyone who may be looking for a way to monitor, alert, and report Azure AD logins with Log Analytics and Logic Apps.

In my previous post:

Using Azure Change Tracking and Inventory to monitor Windows Services
http://terenceluk.blogspot.com/2022/02/using-azure-change-tracking-and.html

I demonstrated how to set up Change Tracking and Inventory in Azure Automation to monitor Windows services in a virtual machine and alert when the service was no longer in a running state. With monitoring and alerting in place, the next step is to incorporate automation so that issues can be immediately remediated without requiring manual intervention. Azure provides multiple methods for automation and for this post, I will demonstrate how to achieve this with the following:

• The PowerShell cmdlet Invoke-AzVMRunCommand (https://docs.microsoft.com/en-us/powershell/module/az.compute/invoke-azvmruncommand?view=azps-7.2.0)
• PowerShell script that uses the Invoke-AzVMRunCommand cmdlet to restart a virtual machine’s Windows service
• Automation Account Runbook
• Managed Identity (https://docs.microsoft.com/en-us/azure/automation/enable-managed-identity-for-automation)
• Test a Automation Account Runbook
• Create an Alert to monitor for service down and create Action Group with Automation Account Runbook
• Test Alert and Automation Account Runbook

To avoid recreating the same content that demonstrate how to set up monitoring for a virtual machine’s Windows service, let’s assume that we’ve gone through the same steps as we did for my previous post:

Using Azure Change Tracking and Inventory to monitor Windows Services
http://terenceluk.blogspot.com/2022/02/using-azure-change-tracking-and.html

Monitoring and alerting has already been setup and what needs to be done is to incorporate automation as shown in the following.

PowerShell cmdlet Invoke-AzVMRunCommand

The PowerShell cmdlet Invoke-AzVMRunCommand is a cmdlet that allows running PowerShell scripts or commands remotely on an Azure Virtual Machine and this will be the method we’ll be using to remotely restart a stopped Windows service in a automation runbook. For more information about this cmdlet, see the following documentation:

Invoke-AzVMRunCommand
https://docs.microsoft.com/en-us/powershell/module/az.compute/invoke-azvmruncommand?view=azps-7.2.0

Run scripts in your Windows VM by using action Run Commands
https://docs.microsoft.com/en-us/azure/virtual-machines/windows/run-command

PowerShell script that uses the Invoke-AzVMRunCommand cmdlet to restart a virtual machine’s Windows service

Next, we’ll incorporate the cmdlet that allows us to restart a virtual machine’s Windows service into the following script:

# This PowerShell using Invoke-AzVMRunCommand runs the PS cmdlet with a parameter directly that specifies the Windows Service name on the target VM

Connect-AzAccount

$resourceGroupName = "yourResourceGroupName"

$vmName ="ServerName"

$scriptToRun = "Start-Service -DisplayName 'Remote Registry'"

Out-File -InputObject $scriptToRun -FilePath ScriptToRun.ps1

Invoke-AzVMRunCommand -ResourceGroupName $resourceGroupName -Name $vmName -CommandId 'RunPowerShellScript' -ScriptPath ScriptToRun.ps1

Remove-Item -Path ScriptToRun.ps1

This script will need the Virtual Machine Contributor role to execute the Invoke-AzVMRunCommand cmdlet and it will be provided with the Managed Identity for the AutomationAccount (configured a bit later) so the final script will have the following lines inserted to execute Connect-AzAccount with the Managed Identity:

# Ensures you do not inherit an AzContext in your runbook

Disable-AzContextAutosave -Scope Process

# Connect to Azure with system-assigned managed identity

$AzureContext = (Connect-AzAccount -Identity).context

# set and store context

$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext

# This PowerShell using Invoke-AzVMRunCommand runs the PS cmdlet with a parameter directly that specifies the Windows Service name on the target VM

$resourceGroupName = "yourResourceGroupName"

$vmName ="ServerName"

$scriptToRun = "Start-Service -DisplayName 'Remote Registry'"

Out-File -InputObject $scriptToRun -FilePath ScriptToRun.ps1

Invoke-AzVMRunCommand -ResourceGroupName $resourceGroupName -Name $vmName -CommandId 'RunPowerShellScript' -ScriptPath ScriptToRun.ps1

Remove-Item -Path ScriptToRun.ps1

Automation Account Runbook

With the PowerShell script for restarting a Windows service prepared, proceed to create a runbook that will execute the script when an alert. Navigate to the Automation Account > Runbooks and Create a runbook:

The options available for the Runbook type are as follows:

1. PowerShell
2. Python
3. PowerShell Workflow
4. Graphical PowerShell
5. Graphical PowerShell Workflow

This example will use a PowerShell script so we’ll select PowerShell with the runtime version as 5.1 and then create the runbook:

With the runbook created, navigate into the newly created runbook and click on the Edit button:

Insert the prepared script into the Runbook:

Proceed to click Save and then Publish to publish the PowerShell Runbook. Note that we won’t be testing this just yet because we haven’t configured the managed identity yet and therefore the runbook does not have the appropriate permissions to use the Invoke-AzVMRunCommand to start a Windows service on the VM.

Managed Identity

With the runbook created, we’ll need to configure a managed identity for the Automation account to run the PowerShell script. More documentation about managed identity can be found in the following documentation:

Using a system-assigned managed identity for an Azure Automation account
https://docs.microsoft.com/en-us/azure/automation/enable-managed-identity-for-automation

Navigate to the Automation Account > Identity > System assigned and switch the Status to On to enable a system assigned managed identity:

Next, click on the Azure role assignments and add the Virtual Machine Contributor role to the assignment:

The Automation Account now has the assigned role for the PowerShell script to execute with.

Test a Automation Account Runbook

With the managed identity configured, we can now proceed to test the PowerShell script and verify that it indeed starts the Windows service. Navigate into the runbook and click on the Edit button:

Click on the Test pane button:

Click on the Start button to execute the PowerShell script using the managed identity:

The runbook will now execute the script:

Queued..

Streams will display when the test completes.

Wait for the test complete and verify that the output indicates it succeeded without errors:

Proceed to verify that the Windows service on the VM has restarted:

Create an Alert to monitor for service down and create Action Group with Automation Account Runbook

With the Automation Account Runbook tested, let’s proceed to create an alert to detect when the Remote Registry service (or any service of your choice) has stopped. Navigate to the Log Analytics workspace that was created to monitor the service, click on Create> Alert rule:

Select Custom Log Search to provide a custom Kusto query:

Many posts, including a previous one I wrote, simply use the following query to look for when a service has stopped:

ConfigurationData

| where SvcName =~ "RemoteRegistry"

| project SvcName, SvcDisplayName, SvcState, TimeGenerated

| where SvcState != "Running"

I find that the issue with using this query is that it will return all records of when the service has not been running within the specified period. This means that if the service was restarted and is running afterwards then this query will not show it and therefore the alert would continue to be fired. After giving it bit of thought, what I wanted to was run a query to get the last time the service was running and the last time it was stopped then compare the two TimeGenerated. Given my lack of experience with Kusto query, I cannot figure out how I can take the time and compare them so I decided to capture the two queries in variables, join them together, then compare the time stamps as shown below:

let LastStopped =

ConfigurationData

| where SvcName =~ "RemoteRegistry"

and SvcState != "Running"

| project SvcName, SvcDisplayName, SvcState, TimeGenerated

| order by TimeGenerated desc

| limit 1;

let LastRunning =

ConfigurationData

| where SvcName =~ "RemoteRegistry"

and SvcState == "Running"

| project SvcName, SvcDisplayName, SvcState, TimeGenerated

| order by TimeGenerated desc

| limit 1;

LastRunning

| join LastStopped on SvcName

// LastRunning time is earlier than LastStopped time

| where TimeGenerated <= TimeGenerated1

The intention of this query is to only return a result if the most recent event was when the service was not running and to not return anything if the most recent event was when the service is running. I’m completely open to recommendations if anyone happen to read this and think there is a better way of doing it.

With the query in place, proceed to create the Alert by clicking on Continue Editing Alert to use the query, leave the rest of the conditions as default and click on Actions:

Create a new action group by clicking on Create action group:

Provide a name for this action group and click on Notifications:

Configuration the notification setting and then click on Actions:

Select Automation Runbook for the Action type:

Select the runbook that was created earlier and click OK:

Proceed to create the Action Group:

The newly created Actiongroup should automatically be added to the Alert:

Fill in the Details tab for the alert:

Complete creating the Alert:

Test Alert and Automation Account Runbook

Proceed to test and confirm that an alert is fired when the service is down and the runbook has successfully executed to restart the service:

I hope this blog post is able to provide information on how to set up an Automation Account to monitor a virtual machine’s Windows service and a runbook that will execute a PowerShell script using the Invoke-AzVMRunCommand to restart a stopped service.

I recently had to perform routine patching of various Citrix ADC / NetScalers to ensure that they are up-to-date with the latest NS13.1 17.42.nc build (released Feb 23, 2022) and one HA pair in particular displayed the following message when the expected Duo MFA authentication prompt was supposed to be displayed:

Additional Information Required

Please type your response below

Attempting to enter your passcode or other information into the prompt leads to an authentication failure.

Searching on the internet displays the following Duo KB:

Why do I see the Citrix Gateway or Netscaler error "Additional Information Required" during Duo authentication?
https://help.duo.com/s/article/3506?language=en_US

… which did not apply for my situation and other forum posts suggest that the theme configured for the gateway was the culprit and that did not apply for this appliance either. What I ended up having to do was downgrade from the updated firmware:

NS13.1 17.42.nc

… back down to:

NS13.1 12.50.nc

Then I opened up a ticket with Duo and the response I received was the following:

Hi Terence,

Thanks for getting back to me.
As per the docs here:
https://duo.com/docs/citrix-netscaler
If you have Citrix Gateway build 12.1-51.16 or later with an "Advanced" or "Premium" edition license, or Citrix Gateway build 13.0-67.x or later with "Standard", "Advanced", or "Premium" edition license, try adding Duo 2FA with nFactor.
After updating past 13, you must use the nfactor version of the integration.
This involves one single change in the proxy.
the line that is:
[radius_server_iframe]
type=citrix_netscaler
will become:
[radius_server_iframe]
type=citrix_netscaler_rfwebui

The way the Citrix ADC / NetScaler is configured to integrate with Duo is as shown in the following Duo document:

Duo for Citrix Gateway Basic Primary Authentication
https://duo.com/docs/citrix-netscaler#:~:text=Log%20in%20to%20the%20Duo,information%20to%20complete%20your%20setup.

I’ve seen other methods such as configuring Duo as:

Duo for Citrix Gateway - Basic Secondary Authentication Instructions
https://duo.com/docs/citrix-netscaler-alt

… and the nFactor method as the support engineer provided but noticed that the nFactor article indicates that it suggests to “try” so I did not realize it was mandatory. I asked the support engineer to clarify this and the response I receive was:

Hi Terence,

Thanks for getting back to me.
You are not wrong, the wording is definitely ambiguous.
However in all the previous cases with this exact issue I have seen, I’m confident in the solution, given I know it works.
I suspect it was worded this way as there is some overlap where both will actually work for various versions/license editions, however given your experience you do not seem to be in the overlap given it is not working.
Please follow through and make the change and I expect the issue will be resolved.
Please let me know if this helps.
Have a great day!
Kind regards,

Raphael

I haven’t actually tried switching over the nFactor yet but will do so and update this post in the future. Hope this helps anyone who may encounter the same issue as I have.

Problem

You’ve recently cleaned up Log Analytics Workspaces that had Virtual Machines connected to it and wanted to re-add those VMs to new workspaces but notice that you see the message:

Cannot enable – Workspace not found (Why?)

Clicking on the link provides the following explanation:

The workspace (e5a604e7-8908-4a30-9d15-5bc4aa81ca48) associated with this virtual machine could not be found. This usually happens if you do not have access to this workspace or the workspace is deleted. Please verify the existence of the workspace with your administrator. If you need to connect to a different workspace, please remove the monitoring agent extension from this virtual machine and try again.

You proceed to remove the monitoring agents from the Windows VM:

Then restart the OS but the previous message still persists.

Attempt to enabling Insights on the VM also fails with the following message:

The workspace (e5a604e7-8908-4a30-9d15-5bc4aa81ca48) associated with this virtual machine could not be found. This usually happens if you do not have access to this workspace or the workspace is deleted. Please verify the existence of the workspace with your administrator. If you need to connect to a different workspace, please remove the monitoring agent extension from this virtual machine and try again.

Solution

I’m not sure if the Azure portal requires more time to refresh but the error message was still present after 2 hours of uninstalling and restarting the VMs I worked on and what I found was to navigate into any Log Analytics Workspace, navigate to Virtual Machines and click into the VM in question:

Then click on the disconnect button:

The process took about a minute and eventually displayed the VM from being Disconnecting to NotConnected:

Navigating back into Azure Monitor > Virtual Machines > Not monitored will now allow you to enable the VM:

I’ve recently been asked about why Log Analytics was not able to capture Security logs from a Windows server as it is not an option under the Log name list:

The short answer is because this is not a feature included natively within a Log Analytics Workspace as described in the following Microsoft documentation:

Configuring Windows Event logs
https://docs.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-windows-events#configuring-windows-event-logs

You can't configure collection of security events from the workspace. You must use Microsoft Defender for Cloud or Microsoft Sentinel to collect security events.

In this post, I will demonstrate how to set up Azure Sentinel to capture a Windows Active Directory Domain Controller event logs and query them.

I will also write a separate post to demonstrate how to use Microsoft Defender for Cloud in the future.

Create Log Analytics Workspace

Begin by creating a Log Analytics Workspace that will store the Azure Sentinel streamed Active Directory Domain Controller logs:

Enable monitoring for Active Directory Domain Controller and attached it to the Log Analytics Workspace

Navigate to Azure Monitor > Virtual Machines > Not monitored and enable the domain controllers:

Enable Insights and select the Log Analytics Workspace that was created earlier to capture the logs when the following window is displayed:

The following agents will be installed:

Repeat the process for any additional domain controllers.

Add Microsoft Sentinel to the new Log Analytics Workspace

With the Domain Controllers added to the Log Analytics Workspace, proceed to create and add a new Microsoft Sentinel with the Log Analytics Workspace:

Configure Azure Sentinel Data Connector to collect Windows Security events

Navigate to Data Connectors, type in Security Events in the filter text field, select Windows Security Events via AMA then click on Open connector page:

In the opened connector page, navigate to the Configuration section and click on the button Create data collection rule button:

Fill in the field for the rule name, subscription and the resource group for the data collection rule:

Click on the +Add resource(s) button and note that This will also enable System Assigned Managed Identity on these machines, in addition to existing User Assigned Identities (if any).

Select the domain controller(s) to that this data collection rule will apply to then click Apply button:

The selected list of VMs will be listed as resources. Proceed to click on the Collect tab:

The following options are presented:

• All Security Events
• Common
• Minimum
• Custom

Hovering over the information button displays the following:

• All events - All Windows Security and App Locker events.
• Common - A standard set of events for auditing purposes.
• Minimal - A small set of events that might indicate potential threats. By enabling this option, you won't be able to have a full audit trail.
• Custom - Allows you to filter and select the security events to stream by using Xpath queries.

A more detailed breakdown of what is included in Common and Minimal can be found here:

https://docs.microsoft.com/en-us/azure/sentinel/windows-security-event-id-reference?WT.mc_id=AZ-MVP-5003408

• All events - All Windows security and AppLocker events.
• Common - A standard set of events for auditing purposes. A full user audit trail is included in this set. For example, it contains both user sign-in and user sign-out events (event IDs 4624, 4634). There are also auditing actions such as security group changes, key domain controller Kerberos operations, and other types of events in line with accepted best practices.

The Common event set may contain some types of events that aren't so common. This is because the main point of the Common set is to reduce the volume of events to a more manageable level, while still maintaining full audit trail capability.

• Minimal - A small set of events that might indicate potential threats. This set does not contain a full audit trail. It covers only events that might indicate a successful breach, and other important events that have very low rates of occurrence. For example, it contains successful and failed user logons (event IDs 4624, 4625), but it doesn't contain sign-out information (4634) which, while important for auditing, is not meaningful for breach detection and has relatively high volume. Most of the data volume of this set is comprised of sign-in events and process creation events (event ID 4688).
• Custom - A set of events determined by you, the user, and defined in a data collection rule using XPath queries. Learn more about data collection rules.

Minimal

1102, 4624, 4625, 4657, 4663, 4688, 4700, 4702, 4719, 4720, 4722, 4723, 4724, 4727, 4728, 4732, 4735, 4737, 4739, 4740, 4754, 4755, 4756, 4767, 4799, 4825, 4946, 4948, 4956, 5024, 5033, 8001, 8002, 8003, 8004, 8005, 8006, 8007, 8222

Common

1, 299, 300, 324, 340, 403, 404, 410, 411, 412, 413, 431, 500, 501, 1100, 1102, 1107, 1108, 4608, 4610, 4611, 4614, 4622, 4624, 4625, 4634, 4647, 4648, 4649, 4657, 4661, 4662, 4663, 4665, 4666, 4667, 4688, 4670, 4672, 4673, 4674, 4675, 4689, 4697, 4700, 4702, 4704, 4705, 4716, 4717, 4718, 4719, 4720, 4722, 4723, 4724, 4725, 4726, 4727, 4728, 4729, 4733, 4732, 4735, 4737, 4738, 4739, 4740, 4742, 4744, 4745, 4746, 4750, 4751, 4752, 4754, 4755, 4756, 4757, 4760, 4761, 4762, 4764, 4767, 4768, 4771, 4774, 4778, 4779, 4781, 4793, 4797, 4798, 4799, 4800, 4801, 4802, 4803, 4825, 4826, 4870, 4886, 4887, 4888, 4893, 4898, 4902, 4904, 4905, 4907, 4931, 4932, 4933, 4946, 4948, 4956, 4985, 5024, 5033, 5059, 5136, 5137, 5140, 5145, 5632, 6144, 6145, 6272, 6273, 6278, 6416, 6423, 6424, 8001, 8002, 8003, 8004, 8005, 8006, 8007, 8222, 26401, 30004

If the Custom option is desired, use the following documentation to fine tune the events collected:

Limit data collection with custom XPath queries
https://docs.microsoft.com/en-us/azure/azure-monitor/agents/data-collection-rule-azure-monitor-agent#limit-data-collection-with-custom-xpath-queries

An example of only collecting event ID 4625, which is an event logged for failed attempts is to use:

Security!*[System[(EventID=4625)]]

You can also configure to only capture successful login Event ID

Security!*[System[(EventID=4624)]]

Or perhaps both:

Security!*[System[(EventID=4624 or EventID=4625)]]

For the purpose of this demonstration, we’ll select Common and then create the rule:

The ingestion and availability of the data should be present within minutes.

To query for security events, navigate to the Logs blade:

We can start with a query as simple as the following, which would retrieve all events with the user name tluk:

SecurityEvent

| where TargetUserName == 'tluk'

Another more in depth query can be to use Event ID 4625, which is an event that is logged for any logon failure. With this in mind, we can reference the following Microsoft documentation:

https://docs.microsoft.com/en-us/windows/security/threat-protection/auditing/event-4625

… to create the following query which will query for event ID 4625, then map the SubStatus code with the reason so it is displayed in the results. Note that kusto queries are case sensitive and the hexadecimal code is in all lower case:

SecurityEvent

| where EventID == 4625

| extend Reason = case(

SubStatus == '0xc000005e', 'There are currently no logon servers available to service the logon request.',

SubStatus == '0xc0000064', 'User logon with misspelled or bad user account',

SubStatus == '0xc000006a', 'User logon with misspelled or bad password',

SubStatus == '0xc000006d', 'The cause is either a bad username or authentication information',

SubStatus == '0xc000006e', 'Indicates a referenced user name and authentication information are valid, but some user account restriction has prevented successful authentication (such as time-of-day restrictions).',

SubStatus == '0xc000006f', 'User logon outside authorized hours',

SubStatus == '0xc0000070', 'User logon from unauthorized workstation',

SubStatus == '0xc0000071', 'User logon with expired password',

SubStatus == '0xc0000072', 'User logon to account disabled by administrator',

SubStatus == '0xc00000dc', 'Indicates the Sam Server was in the wrong state to perform the desired operation.',

SubStatus == '0xc0000133', 'Clocks between DC and other computer too far out of sync',

SubStatus == '0xc000015b', 'The user has not been granted the requested logon type (also called the logon right) at this machine',

SubStatus == '0xc000018c', 'The logon request failed because the trust relationship between the primary domain and the trusted domain failed.',

SubStatus == '0xc0000192', 'An attempt was made to logon, but the Netlogon service was not started.',

SubStatus == '0xc0000193', 'User logon with expired account',

SubStatus == '0xc0000224', 'User is required to change password at next logon',

SubStatus == '0xc0000225', 'Evidently a bug in Windows and not a risk',

SubStatus == '0xc0000234', 'User logon with account locked',

SubStatus == '0xc00002ee', 'Failure Reason: An Error occurred during Logon',

SubStatus == '0xc0000413', 'Logon Failure: The machine you are logging on to is protected by an authentication firewall. The specified account is not allowed to authenticate to the machine.',

SubStatus == '0x0', 'Status OK',

strcat('Unknown reason substatus: ', SubStatus))

The following is an event I intentionally generated by typing in an incorrect password:

It is important to note that the amount of data collected can vary depending on the size of the environment so to avoid unexpected charges due to the amount of data ingested by Sentinel and into Log Analytics, start with minimum to get an idea of the amount of data per day and fine tune the configuration. Hope this helps anyone looking for a demonstration on how to set up security events collection in Azure. Kusto query is extremely powerful and will be able to provide all sorts of useful data for troubleshooting, auditing, and reporting.

Problem

You attempt to use Remote Desktop to log into a domain controller but the attempt fails with the following message:

An authentication error has occurred.
The specified network password is not correct.

Attempting to log in via the console is not an option.

Solution

The workaround to successfully RDP to this domain controller is to disable the Allow connections only form computers running Remote Desktop with Network Level Authentication (recommended) in the RDP settings:

If console access is available, try to log in via direct console access and disable the configuration. For situations where the domain controller is hosted in cloud providers such as Azure, console access will not an option. In scenarios where console access is not available, one of the potential workarounds is to use the Registry Editor to remotely connect to the domain controller and disable this setting via the registry.

Launch the Registry Editor, select the File tab and choose Connect Network Registry:

Enter the remote domain controller’s name:

The connection should succeed:

Navigate to the following registry path:

dc2\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp

Locate the REG_DWORD named UserAuthentication and change the value from 1 (enabled) to 0 (disabled):

Changing this registry key should now have the NLA configuration disabled as such:

You should now be able to log into the domain controller:

Proceed to review the event logs of the domain controller and correct any health issues.

As a follow up to my previous post:

Monitoring, Alerting, Reporting Azure AD logins and login failures with Log Analytics and Logic Apps
http://terenceluk.blogspot.com/2022/02/monitoring-alerting-reporting-azure-ad.html

If your organization already uses Azure Sentinel as a (SIEM Security information and event management), it would be preferred to use the Azure Active Directory Data Connector available in Sentinel to capture and monitor Azure AD logs. This post serves to demonstrate how to achieve the same results by leveraging Azure Sentinel to capture and query the Azure AD events.

Create Log Analytics Workspace

Begin by creating a Log Analytics Workspace that will store the Azure Sentinel streamed Active Directory Domain Controller logs:

Add Microsoft Sentinel to the new Log Analytics Workspace

With the Domain Controllers added to the Log Analytics Workspace, proceed to create and add a new Microsoft Sentinel with the Log Analytics Workspace:

Configure Azure Sentinel Data Connector to collect Windows Security events

Navigate to Data Connectors, type in Azure Active Directory in the filter text field, select Azure Active Directory then click on Open connector page:

The type of Azure AD logs that Microsoft Sentinel can capture are listed in the connector page. Note that in order to export Sign-in data, your organization needs Azure AD P1 or P2 license.

Detailed information about the options can be found here:

Connect Azure Active Directory (Azure AD) data to Microsoft Sentinel
https://docs.microsoft.com/en-us/azure/sentinel/connect-azure-active-directory

You can use Microsoft Sentinel's built-in connector to collect data from Azure Active Directory and stream it into Microsoft Sentinel. The connector allows you to stream the following log types:

• Sign-in logs, which contain information about interactive user sign-ins where a user provides an authentication factor.

The Azure AD connector now includes the following three additional categories of sign-in logs, all currently in PREVIEW:

• Non-interactive user sign-in logs, which contain information about sign-ins performed by a client on behalf of a user without any interaction or authentication factor from the user.
• Service principal sign-in logs, which contain information about sign-ins by apps and service principals that do not involve any user. In these sign-ins, the app or service provides a credential on its own behalf to authenticate or access resources.
• Managed Identity sign-in logs, which contain information about sign-ins by Azure resources that have secrets managed by Azure. For more information, see What are managed identities for Azure resources?
• Audit logs, which contain information about system activity relating to user and group management, managed applications, and directory activities.
• Provisioning logs (also in PREVIEW), which contain system activity information about users, groups, and roles provisioned by the Azure AD provisioning service.

The options:

• ADFS Sign-In Logs (Preview)
• User Risk Events (Preview)
• Risk Users (Preview)

… are new sources that Microsoft has released after the initial options.

ADFS Sign-In Logs requires Azure AD Connect Health to correlate Event IDs from AD FS to provide information about the request and error details if the request fails. More information about ADFS Sign-In Logs can be found here:

AD FS sign-ins in Azure AD with Connect Health – preview
https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-health-ad-fs-sign-in

What is Azure AD Connect Health?
https://docs.microsoft.com/en-us/azure/active-directory/hybrid/whatis-azure-ad-connect#what-is-azure-ad-connect-health

Download Azure AD Connect Health Agent for AD FS
https://portal.azure.com/#blade/Microsoft_Azure_ADHybridHealth/AadHealthMenuBlade/QuickStart

User Risk Events (Preview) and Risk Users (Preview) pertain to Azure Identity Protection, which are made available with the Azure Premium P2 licenses. More information about the details of these events can be found here: https://docs.microsoft.com/en-us/azure/active-directory/identity-protection/concept-identity-protection-risks and more information about the type of risk data that can be queried can be found here: https://docs.microsoft.com/en-gb/azure/active-directory/identity-protection/howto-export-risk-data

For the purpose of this example, we’ll select

• Sign-in logs
• Audit logs
• Non-interactive user sign-in logs
• Service principal sign-in logs
• Managed Identity sign-in logs
• Provisioning logs

Note that you may need to wait upwards to 30 minutes before the connector’s status switches from Not connected to Connected and the Data types begin to be highlighted in green:

Once connected, you should see the following tables under Log Management:

• SigninLogs
• AuditLogs
• AADNonInteractiveUserSignInLogs
• AADServicePrincipalSignInLogs
• AADManagedIdentitySignInLogs
• AADProvisioningLogs

From here, the sky is really the limit as we have access to various logs and able to query for any type of information we want with Kusto. One of the examples I demonstrated in my previous post is the following which would look for failed sign-ins from within the SigninLogs:

SigninLogs

| where Status.errorCode != 0

| extend City=LocationDetails.city, State=LocationDetails.state, Country=LocationDetails.countryOrRegion, Error_Code=Status.errorCode, Failure_Reason=Status.failureReason

| project TimeGenerated, UserDisplayName, AppDisplayName, IPAddress, City, State, Country, AuthenticationRequirement, Failure_Reason, ConditionalAccessStatus, ConditionalAccessPolicies, Error_Code

With Azure Sentinel configured for the Azure AD logs, we can configure Logic Apps to send out daily reports or other automation tasks for alerting. Please see my previous post for configuration demonstration.

Monitoring, Alerting, Reporting Azure AD logins and login failures with Log Analytics and Logic Apps
http://terenceluk.blogspot.com/2022/02/monitoring-alerting-reporting-azure-ad.html

I’ve recently had to look into how the Certificates & secrets configured for their App Registrations could be monitored so administrators could be warned well ahead of the expiry so applications using these Enterprise Applications / service principals would not cease to work unexpectedly. Microsoft Azure unfortunately does not provide a native way for this monitoring (yet) but I managed to find a PowerShell script by Christopher Scott (https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/app-registration-expiration-monitoring-and-notifications/ba-p/2043805) that uses the PowerShell cmdlets Get-AzADApplication, Get-AzADServicePrincipal, and Get-AzADAppCredential to extract the information and send them to Log Analytics. Chris does a great job walking through the steps but I did not completely understand the why and how every component work. I also could not get his script to work when used in an automation account as the data sent to Log Analytics would display all Certificates & secrets as expired and the start and end times were not displayed. Blindly using a script without understanding it isn’t something I recommend or do myself so I took the time to review as well as compare it to the sample script that Microsoft provides here: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#powershell-sample. One of the components that Chris’ script was different than Microsoft was how the signature and post to Log Analytics function was written so I decided to use Microsoft’s sample script for building the signature for authorization and sending data to log analytics, and Chris’ App Registration Certificates & secrets logic to obtain the data.

The purpose of this blog post is to demonstrate how one can send custom log data to Log Analytics by breaking down and understand the components in the finalized working script that captures the Certificates & secrets configured for their App Registrations and uses the HTTP Data Collector API to send log data to Azure Monitor from a PowerShell REST API call. The opportunities for sending data to a Log Analytics Workspace so it can be queried are limitless and going through this hands on learning exercise was very exciting for me.

Prerequisites

The components we’ll need for this script are as follows:

Az.Accounts and Az.Resources modules will be required for this script and the versions I have installed are:

Az.Accounts – 2.5.1
Az.Resources 4.2.0

An account with the Global Reader role, which we’ll use to interactively log onto Azure to retrieve the App Registration configuration details. Using a service principal to run this script is the obvious better choice so I will also include the PowerShell script that uses a Service Principal to log into Azure. It can be found here: https://github.com/terenceluk/Azure/blob/main/PowerShell/Get-AppRegistrationExpirationServicePrincipal.ps1

For the purpose of this post, I’ll continue with logging in interactively with Connect-AzAccount

Workspace ID: This is the Workspace ID of the Log Analytics that will be storing the data. The value can be found by navigating to the Log Analytics workspace > Agents management > Workspace ID

Shared Key: This is the Primary Key of the Log Analytics that will be storing the data. The value can be found by navigating to the Log Analytics workspace > Agents management > Primary key

The two variables above are similar to Storage Account access keys, where having them would allow us to send log analytics data into the workspace.

The full PowerShell script can be found at my GitHub repo: https://github.com/terenceluk/Azure/blob/main/PowerShell/Get-AppRegistrationExpirationInteractive.ps1

As well as pasted at the bottom of this post.

Functions

Two functions are required for this script.

The first function is used to build the signature that will be used as an authorization header when a request is sent to the Azure Monitor HTTP Data Collector API to POST data. This function requires the following parameters to build the signature:

1. Workspace ID ($customerID)– The Workspace ID of the Log Analytics Workspace
2. Primary Key ($sharedKey)– The Workspace Primary Key of the Log Analytics Workspace
3. Date– The current date time
4. Content Length– The character length of JSON formatted data we are sending to Log Analytics
5. Method– “POST” is the method that will be sent
6. Content Type– “application/json” is the content type that will be sent
7. Resource– “/api/logs” is the resource that will be sent

Once the above parameters are collected, the function will build the signature that will be used in the authorization header and return it to the function that will post data to the Log Analytics Workspace.

# The following function builds the signature used to authorization header that sends a request to the Azure Monitor HTTP Data Collector API

Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource)

{

$xHeaders = "x-ms-date:" + $date

$stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource

$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)

$keyBytes = [Convert]::FromBase64String($sharedKey)

$sha256 = New-Object System.Security.Cryptography.HMACSHA256

$sha256.Key = $keyBytes

$calculatedHash = $sha256.ComputeHash($bytesToHash)

$encodedHash = [Convert]::ToBase64String($calculatedHash)

$authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash

return $authorization

}

The next function is what will be used to actually send data to the log analytics workspace. This function receives the following parameters:

1. Workspace ID ($customerID)– The Workspace ID of the Log Analytics Workspace
2. Primary Key ($sharedKey)– The Workspace Primary Key of the Log Analytics Workspace
3. Body– The data to be sent to the Log Analytics workspace in JSON format
4. Log Type– The name of the log the data should be sent to

Once the above parameters are collected, the function will send the required parameters to the function that builds the signature for the authorization header (the function above), then uses inserts the required custom Log Analytics Workspace ID to build the URI / API endpoint (https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#request-uri):

https://<CustomerId>.ods.opinsights.azure.com/api/logs?api-version=2016-04-01

It will then create the required header containing the authorization signature, the custom log name in the Log Analytics workspace, the current date and time, and a optional timestamp field that could be an empty string which will have Azure Monitor assume the time is the message ingestion time.

Finally, the cmdlet Invoke-WebRequest (https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.2) to send the HTTPS requests to the URI / API endpoint.

# The following function will create and post the request using the signature created by the Build-Signature function for authorization

Function Post-LogAnalyticsData($customerId, $sharedKey, $body, $logType)

{

$method = "POST"

$contentType = "application/json"

$resource = "/api/logs"

$rfc1123date = [DateTime]::UtcNow.ToString("r")

$contentLength = $body.Length

$signature = Build-Signature `

-customerId $customerId `

-sharedKey $sharedKey `

-date $rfc1123date `

-contentLength $contentLength `

-method $method `

-contentType $contentType `

-resource $resource

$uri = "https://" + $customerId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01"

$headers = @{

"Authorization" = $signature;

"Log-Type" = $logType;

"x-ms-date" = $rfc1123date;

"time-generated-field" = $TimeStampField;

}

$response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing

return $response.StatusCode

}

Here is an example of what the $headers variable would contain:

Connecting to Azure

The cmdlet Connect-AzAccount is used to interactively connect and authenticate the PowerShell session.

# Log in interactively with an account with Global Reader role

Connect-AzAccount

Defining Variables

As mentioned in the prerequisites, we’ll need the following variables assigned with their values.

1. Workspace ID ($customerID)– The Workspace ID of the Log Analytics Workspace
2. Primary Key ($sharedKey)– The Workspace Primary Key of the Log Analytics Workspace
3. Log Type– The name of the log the data should be sent to
4. Time Stamp Field – This variable is optional and we’ll be leaving it empty for this script

# Replace with your Workspace ID

$customerId = "b0d472a3-8c13-4cec-8abb-76051843545f"

# Replace with your Workspace Primary Key

$sharedKey = "D3s71+X0M+Q3cGTHC5I6H6l23xRNAKvjA+yb8JzMQQd3ntxeFZLmMWIMm7Ih/LPMOji9zkXDwavAJLX1xEe/4g=="

# Specify the name of the record type that you'll be creating (this is what will be displayed under Log Analytics > Logs > Custom Logs)

$LogType = "AppRegistrationExpiration"

# Optional name of a field that includes the timestamp for the data. If the time field is not specified, Azure Monitor assumes the time is the message ingestion time

$TimeStampField = ""

Obtaining list of App Registrations and Enterprise Applications / Service Principals

The list of App Registrations and Enterprise Applications / Service Principals for the Azure AD tenant will be retrieved and stored in variables.

# Get the full list of Azure AD App Registrations

$applications = Get-AzADApplication

Here is a sample of what the variable would contain:

# Get the full list of Azure AD Enterprise Applications (Service Principals)

$servicePrincipals = Get-AzADServicePrincipal

Here is a sample of what the variable would contain:

Filter for App Registrations that have Certificates & secrets configured

Next, an array will be created to store applications that have Certificates & secrets configured. Then the array will be populated with the following fields:

1. DisplayName
2. ObjectId
3. ApplicationId
4. KeyId
5. Type
6. StartDate
7. EndDate

# Create an array named appWithCredentials

$appWithCredentials = @()

# Populate the array with app registrations that have credentials

# Retrieve the list of applications and sort them by DisplayName

$appWithCredentials += $applications | Sort-Object -Property DisplayName | % {

# Assign the variable application with the follow list of properties

$application = $_

# Retrieve the list of Enterprise Applications (Service Principals) and match the ApplicationID of the SP to the App Registration

$sp = $servicePrincipals | ? ApplicationId -eq $application.ApplicationId

Write-Verbose ('Fetching information for application {0}' -f $application.DisplayName)

# Use the Get-AzADAppCredential cmdlet to get the Certificates & secrets configured (this returns StartDate, EndDate, KeyID, Type, Usage, CustomKeyIdentifier)

# Populate the array with the DisplayName, ObjectId, ApplicationId, KeyId, Type, StartDate and EndDate of each Certificates & secrets for each App Registration

$application | Get-AzADAppCredential -ErrorAction SilentlyContinue | Select-Object `

-Property @{Name='DisplayName'; Expression={$application.DisplayName}}, `

@{Name='ObjectId'; Expression={$application.ObjectId}}, `

@{Name='ApplicationId'; Expression={$application.ApplicationId}}, `

@{Name='KeyId'; Expression={$_.KeyId}}, `

@{Name='Type'; Expression={$_.Type}},`

@{Name='StartDate'; Expression={$_.StartDate -as [datetime]}},`

@{Name='EndDate'; Expression={$_.EndDate -as [datetime]}}

}

Here is a sample what the array would contain:

Adding additional fields to specify whether certificate & secret has expired

It is possible to immediately send the information already collected to the Log Analytics workspace but Chris Scott too it one step further and appended additional fields for whether the certificate or secret was expired, the timestamp used to check the validity and the days until expiry. This can be accomplished while using Kusto query but I find this to be very handy to add.

# With the $application array populated with the Certificates & secrets and its App Registration, proceed to calculate and add the fields to each record in the array:

# Expiration of the certificate or secret - Valid or Expired

# Add the timestamp used to calculate the validity

# The days until the certificate or secret expires

Write-output 'Validating expiration data...'

$timeStamp = Get-Date -format o

$today = (Get-Date).ToUniversalTime()

$appWithCredentials | Sort-Object EndDate | % {

# First if catches certificates & secrets that are expired

if($_.EndDate -lt $today) {

$days= ($_.EndDate-$Today).Days

$_ | Add-Member -MemberType NoteProperty -Name 'Status' -Value 'Expired'

$_ | Add-Member -MemberType NoteProperty -Name 'TimeStamp' -Value "$timestamp"

$_ | Add-Member -MemberType NoteProperty -Name 'DaysToExpiration' -Value $days

# Second if catches certificates & secrets that are still valid

} else {

$days= ($_.EndDate-$Today).Days

$_ | Add-Member -MemberType NoteProperty -Name 'Status' -Value 'Valid'

$_ | Add-Member -MemberType NoteProperty -Name 'TimeStamp' -Value "$timestamp"

$_ | Add-Member -MemberType NoteProperty -Name 'DaysToExpiration' -Value $days

}

}

Here is a sample of what the array will contain:

Converting data to be sent to Log Analytics to JSON

The HTTP Data Collector API expects the data to be in JSON format so the collected information is converted:

# Convert the list of each Certificates & secrets for each App Registration into JSON format so we can send it to Log Analytics

$appWithCredentialsJSON = $appWithCredentials | convertto-json

## The following commented lines is a sample JSON that can be used to test sending data to Log Analytics

<#

$json = @"

[{

"DisplayName": "Vulcan O365 Audit Logs",

"ObjectId": "058f1297-ba80-4b9e-8f9c-15febdf85df0",

"ApplicationId": {

"value": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57",

"Guid": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57"

},

"KeyId": "2ea30e24-e2ad-44ff-865a-df07199f26a5",

"Type": "AsymmetricX509Cert",

"StartDate": "2021-05-29T18:26:46",

"EndDate": "2022-05-29T18:46:46"

},

{

"DisplayName": "Vulcan O365 Audit Logs",

"ObjectId": "058f1297-ba80-4b9e-8f9c-15febdf85df0",

"ApplicationId": {

"value": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57",

"Guid": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57"

},

"KeyId": "259dbc4d-cdde-4007-a9ed-887437560b15",

"Type": "AsymmetricX509Cert",

"StartDate": "2021-05-29T17:46:22",

"EndDate": "2022-05-29T18:06:22"

}]

"@

#>

Use the Post-LogAnalyticsData function to send the collected data to the Log Analytics Workspace

With the data collected, proceed to use the Post-LogAnalyticsData function to send the data:

# Submit the data to the API endpoint

Post-LogAnalyticsData -customerId $customerId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($appWithCredentialsJSON)) -logType $logType

Here is a successful POST with a return code of 200:

More information about the return codes can be found here: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#return-codes

------------------------------------------------------------------------------------------------------------

Note that it will take a bit of time before the data is displayed in the Log Analytics Workspace. I’ve found that I sometimes have to wait upwards to 15 minutes or more before it is displayed.

When it is displayed, you should see the a table with the log name you specified in Custom Logs:

You will also see the table listed under Custom Logs:

Querying the log will display the following:

The fields available as you scroll across are:

1. TimeGenerated [UTC]
2. Computer
3. RawData
4. DisplayName_s
5. ObjectId_g
6. ApplicationId_value_g
7. ApplicationId_Guid_g
8. KeyId_g
9. Type_s
10. StartDate_t [UTC]
11. EndDate_t [UTC]
12. Status_s
13. TimeStamp_t [UTC]
14. DaysToExpiration_d
15. Type
16. _ResourceId
17. TenantId
18. SourceSystem
19. MG
20. ManagementGroupName

You might be wondering the following:

Question: Why are there _s, _g, _d appended to some of the variables?

Answer: These are record types (string, Boolean, double, date/time, GUID) that are automatically added.

See the following link for more information: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#record-type-and-properties

Question: Why are there extra fields?

Answer: Some of them are reserved properties (e.g. tenant, TimeGenerated, RawData) and others are other default fields added.

------------------------------------------------------------------------------------------------------------

I hope this post provides a bit more information about how to send custom log data to a Log Analytics Workspace. The Microsoft documentation:

Send log data to Azure Monitor by using the HTTP Data Collector API (preview)
https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api

… does a fantastic job of explaining all the components in detail albeit a bit of a long read so I hope this blog post helps provide a shorten version of the mechanics.

I will be following up with another post that demonstrates how to automate the use of this script to collect App Registrations’ Certificates and Secrets expiration into Log Analytics, then use a Logic App to create and send a report out via email so stay tuned.

------------------------------------------------------------------------------------------------------------

# The following function builds the signature used to authorization header that sends a request to the Azure Monitor HTTP Data Collector API

Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource)

{

$xHeaders = "x-ms-date:" + $date

$stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource

$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)

$keyBytes = [Convert]::FromBase64String($sharedKey)

$sha256 = New-Object System.Security.Cryptography.HMACSHA256

$sha256.Key = $keyBytes

$calculatedHash = $sha256.ComputeHash($bytesToHash)

$encodedHash = [Convert]::ToBase64String($calculatedHash)

$authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash

return $authorization

}

# The following function will create and post the request using the signature created by the Build-Signature function for authorization

Function Post-LogAnalyticsData($customerId, $sharedKey, $body, $logType)

{

$method = "POST"

$contentType = "application/json"

$resource = "/api/logs"

$rfc1123date = [DateTime]::UtcNow.ToString("r")

$contentLength = $body.Length

$signature = Build-Signature `

-customerId $customerId `

-sharedKey $sharedKey `

-date $rfc1123date `

-contentLength $contentLength `

-method $method `

-contentType $contentType `

-resource $resource

$uri = "https://" + $customerId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01"

$headers = @{

"Authorization" = $signature;

"Log-Type" = $logType;

"x-ms-date" = $rfc1123date;

"time-generated-field" = $TimeStampField;

}

$response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing

return $response.StatusCode

}

# Log in interactively with an account with Global Reader role

Connect-AzAccount

# Replace with your Workspace ID

$customerId = "b0d472a3-8c13-4cec-8abb-76051843545f"

# Replace with your Workspace Primary Key

$sharedKey = "D3s71+X0M+Q3cGTHC5I6H6l23xRNAKvjA+yb8JzMQQd3ntxeFZLmMWIMm7Ih/LPMOji9zkXDwavAJLX1xEe/4g=="

# Specify the name of the record type that you'll be creating (this is what will be displayed under Log Analytics > Logs > Custom Logs)

$LogType = "AppRegistrationExpiration"

# Optional name of a field that includes the timestamp for the data. If the time field is not specified, Azure Monitor assumes the time is the message ingestion time

$TimeStampField = ""

# Get the full list of Azure AD App Registrations

$applications = Get-AzADApplication

# Get the full list of Azure AD Enterprise Applications (Service Principals)

$servicePrincipals = Get-AzADServicePrincipal

# Create an array named appWithCredentials

$appWithCredentials = @()

# Populate the array with app registrations that have credentials

# Retrieve the list of applications and sort them by DisplayName

$appWithCredentials += $applications | Sort-Object -Property DisplayName | % {

# Assign the variable application with the follow list of properties

$application = $_

# Retrieve the list of Enterprise Applications (Service Principals) and match the ApplicationID of the SP to the App Registration

$sp = $servicePrincipals | ? ApplicationId -eq $application.ApplicationId

Write-Verbose ('Fetching information for application {0}' -f $application.DisplayName)

# Use the Get-AzADAppCredential cmdlet to get the Certificates & secrets configured (this returns StartDate, EndDate, KeyID, Type, Usage, CustomKeyIdentifier)

# Populate the array with the DisplayName, ObjectId, ApplicationId, KeyId, Type, StartDate and EndDate of each Certificates & secrets for each App Registration

$application | Get-AzADAppCredential -ErrorAction SilentlyContinue | Select-Object `

-Property @{Name='DisplayName'; Expression={$application.DisplayName}}, `

@{Name='ObjectId'; Expression={$application.ObjectId}}, `

@{Name='ApplicationId'; Expression={$application.ApplicationId}}, `

@{Name='KeyId'; Expression={$_.KeyId}}, `

@{Name='Type'; Expression={$_.Type}},`

@{Name='StartDate'; Expression={$_.StartDate -as [datetime]}},`

@{Name='EndDate'; Expression={$_.EndDate -as [datetime]}}

}

# With the $application array populated with the Certificates & secrets and its App Registration, proceed to calculate and add the fields to each record in the array:

# Expiration of the certificate or secret - Valid or Expired

# Add the timestamp used to calculate the validity

# The days until the certificate or secret expires

Write-output 'Validating expiration data...'

$timeStamp = Get-Date -format o

$today = (Get-Date).ToUniversalTime()

$appWithCredentials | Sort-Object EndDate | % {

# First if catches certificates & secrets that are expired

if($_.EndDate -lt $today) {

$days= ($_.EndDate-$Today).Days

$_ | Add-Member -MemberType NoteProperty -Name 'Status' -Value 'Expired'

$_ | Add-Member -MemberType NoteProperty -Name 'TimeStamp' -Value "$timestamp"

$_ | Add-Member -MemberType NoteProperty -Name 'DaysToExpiration' -Value $days

# Second if catches certificates & secrets that are still valid

} else {

$days= ($_.EndDate-$Today).Days

$_ | Add-Member -MemberType NoteProperty -Name 'Status' -Value 'Valid'

$_ | Add-Member -MemberType NoteProperty -Name 'TimeStamp' -Value "$timestamp"

$_ | Add-Member -MemberType NoteProperty -Name 'DaysToExpiration' -Value $days

}

}

# Convert the list of each Certificates & secrets for each App Registration into JSON format so we can send it to Log Analytics

$appWithCredentialsJSON = $appWithCredentials | convertto-json

## The following commented lines is a sample JSON that can be used to test sending data to Log Analytics

<#

$json = @"

[{

"DisplayName": "Vulcan O365 Audit Logs",

"ObjectId": "058f1297-ba80-4b9e-8f9c-15febdf85df0",

"ApplicationId": {

"value": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57",

"Guid": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57"

},

"KeyId": "2ea30e24-e2ad-44ff-865a-df07199f26a5",

"Type": "AsymmetricX509Cert",

"StartDate": "2021-05-29T18:26:46",

"EndDate": "2022-05-29T18:46:46"

},

{

"DisplayName": "Vulcan O365 Audit Logs",

"ObjectId": "058f1297-ba80-4b9e-8f9c-15febdf85df0",

"ApplicationId": {

"value": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57",

"Guid": "ac28a30a-6e5f-4c2d-9384-17bbb0809d57"

},

"KeyId": "259dbc4d-cdde-4007-a9ed-887437560b15",

"Type": "AsymmetricX509Cert",

"StartDate": "2021-05-29T17:46:22",

"EndDate": "2022-05-29T18:06:22"

}]

"@

#>

# Submit the data to the API endpoint

Post-LogAnalyticsData -customerId $customerId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($appWithCredentialsJSON)) -logType $logType

As described in my previous post:

Using PowerShell to send custom log data to Log Analytics for Azure Monitor alerting and Kusto Query
http://terenceluk.blogspot.com/2022/03/using-powershell-to-send-custom-log.html

I had difficulty using Christopher Scott’s script to work when used in an automation account as the data sent to Log Analytics would display all Certificates & secrets as expired and the start and end times were not displayed and ended up spending most of my weekend troubleshooting why so in an effort to help anyone who may encounter the same issue as I did, this will be a quick blog post writeup that will outline the symptoms and resolution.

Problem

You attempt to use Christopher Scott’s PowerShell script (https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/app-registration-expiration-monitoring-and-notifications/ba-p/2043805) to extract App Registration’s Certificates & secrets to send to Log Analytics but noticed that the data returned when querying for displays:

1. All certificates and secrets as Expired
2. The DaysToExpiration are all set to -738,241
3. The StartDate and EndDate fields are expired

Solution

What was determined after troubleshooting this was that the default Az.Resources module for the Automation Account was version: 5.4.0.

It appears the cmdlet Get-AzADServicePrincipal and Get-AzADAppCredential returns null for a lot of the fields in this newer version.

To fix this issue, try downgrading it to 4.2.0 by downloading the nupkg package here:

https://www.powershellgallery.com/packages/Az.Resources/2.5.0

Rename the extension from nupkg to zip.

Import the package in the Modules of the Automation Account:

Try running the Runbook again to confirm that the data sent to Log Analytics is displayed properly as such:

As a follow up to my previous post:

Using PowerShell to send custom log data to Log Analytics for Azure Monitor alerting and Kusto Query
http://terenceluk.blogspot.com/2022/03/using-powershell-to-send-custom-log.html

I recently looked into how the Certificates & secrets configured for Azure AD App Registrations could be monitored so administrators could be warned ahead of the expiry because of a recommendation I proposed for a client where their Azure team would experience outages due to expired credentials. This client had hundreds of partners that authenticate against their Azure AD and the unplanned downtimes were a major pain point.

As indicated in my previous post and for those who have attempted this will know that Azure does not provide a native built-in method for monitoring them but there are two solutions provided by the following professionals:

App Registration Expiration Monitoring and Notifications – By Christopher Scott
https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/app-registration-expiration-monitoring-and-notifications/ba-p/2043805

Use Power Automate to Notify of Upcoming Azure AD App Client Secrets and Certificate Expirations – By Russ Rimmerman
https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/use-power-automate-to-notify-of-upcoming-azure-ad-app-client/ba-p/2406145

I liked the solution that Christopher Scott created so I went ahead and tested it but ran into an issue caused by the updated Az.Resources module as described in this post:

Attempting to use Chris Scott's App Registration Expiration Monitoring and Notifications displays data with "DaysToExpiration" set to "-738,241"
http://terenceluk.blogspot.com/2022/03/attempting-to-use-chris-scotts-app.html

I also noticed that the kusto query he provided did not cover App Registrations that had more than one secret or certificate configured. Lastly, I preferred to have a pre-scheduled report sent out with a list of expiring certificates and secrets so this post serves to outline the configuration and some minor tweaks to this great solution.

Create Log Analytics Workspace

Begin by creating a Log Analytics Workspace that will store the Azure Sentinel streamed Active Directory Domain Controller logs:

With the Log Analytics workspace created, navigate to the Agents management blade to collect the following information:

1. Workspace ID
2. Workspace Primary ID

These two parameters will be required for setting up the Automation Account so it can send custom Log Analytics data to this workspace.

Create and Configure App Registration for Automation Account Runbook

The Automation Account Runbook that we’ll be creating shortly will be using a service principal to execute a PowerShell script that will collect every App Registration and Enterprise Applications / Service Principal along with their configured certificates and secrets. This service principal will need Global Reader rights to obtain the information.

**Note that we can also use a Managed Identity for the execution of the PowerShell script as shown in one of my previous blog posts: http://terenceluk.blogspot.com/2022/02/using-automation-account-to-monitor-vms.html

With the App Registration created, document the following information:

1. Application (client) ID <- this represents the service principal’s user name
2. Directory (tenant) ID

Proceed to create a client secret in the Certificates & secrets blade and document the value of the secret before navigating away from the page as it would not be displayed again:

Next, navigate to the Roles and administrators blade for the Azure AD tenant, search for Global Reader and add the service principal to the role:

Create Automation Account

With the prerequisites for the Automation Account configured, the next is to create the new Automation Account that will use a PowerShell script to extract the App Registrations along with the configured certificates and secrets, and their expiry dates to send into the Log Analytics Workspace:

Configure Variables and Credential Automation Account

A runbook using a PowerShell script will be used to obtain the certificates and secrets of App Registrations by authenticating against Azure AD and accessing the Log Analytics Workspace to send data. It is best practice to store variables and credentials outside of the script so we’ll store them securely within the Variables and Credentials blade of the Automation Account.

Navigate to the Variables blade and configure the following 3 variables:

1. MonitoredTenantID
2. WorkspaceID
3. WorkspacePrimaryKey

It is possible to store these variables as encrypted to further protect the values and for the purpose of this example, I will only store the Workspace Primary Key as encrypted:

Navigate to the Credentials blade and configure the App Registration / Service Principal credential that the PowerShell script will use to authenticate against Azure AD:

Create and Configure an Automation Account Runbook

With the credentials and variables for the PowerShell script configured, navigate to the Runbooks blade and create a new runbook:

Provide the following:

Name: <name of choice>
Runbooktype: <PowerShell>
Runtimeversion: <5.1>
Description: <description of choice>

You will be brought into Edit window of the Runbook after creating it:

Proceed to paste the following PowerShell script into the window: https://github.com/terenceluk/Azure/blob/main/PowerShell/Get-AppRegistrationExpirationAutomation.ps1

A more detailed breakdown of what the script does can be found in my previous blog post: http://terenceluk.blogspot.com/2022/03/using-powershell-to-send-custom-log.html

I will also include the PowerShell script at the end of this blog post.

Note how the script references the credentials and the variables we defined earlier for the Automation Account:

Use the Testpane feature to execute a test and verify that the script runs without any errors:

A return code of 200 is what we’re looking for:

Proceed to publish the script:

Next, schedule the Runbook to run accordingly to a day, time, and frequency of your choice:

With the runbook configured, proceed to test executing the Runbook and confirm that the desired data is being sent to the Log Analytics Workspace:

Set up reporting with Logic Apps

With the Automation Account and Runbook successfully set up and verified, proceed to create a new Logic App that will query the data in the Log Analytics Workspace, generate a HTML report and send it to the desired email address.

Navigate into the Logic app and create a new Logic App:

Once the Logic App has been created, click on Logic app designer:

We’ll be creating 3 steps for this Logic App where:

1. Recurrence: This will configure a recurring schedule for this Logic App to execute
2. Run query and visualize results: This will allow us to run the Kusto query, set a Time Range and specify a Chart Type
3. Send an email (V2): This will allow us to send the Kusto query results via email

Recurrence:

Configure the desired frequency of this Logic App:

Run query and visualize results:

Fill in the following fields:

Subscription: <your subscription>
Resource Group: <desired resource group>
Resource Type: Log Analytics Workspace
Resource Name: The Log Analytics Workspace containing the custom log data>
Query: See the query in my GitHub: https://github.com/terenceluk/Azure/blob/main/Kusto%20KQL/Get%20Expiring%20and%20Expired%20App%20Reg%20Certs%20and%20Secrets.kusto

AppRegistrationExpiration_CL

| summarize arg_max(TimeGenerated,*) by KeyId_g // arg_max is used to return the latest record for the time range selected and the * is to return all columns, records with unique KeyId_g will be returned so expired and multiple credentials are returned

| where DaysToExpiration_d <= 1000 or Status_s == "Expired" // this specifies a filter for the amount of days before expiry

//| where TimeGenerated > ago(1d) // the TimeGenerated value must be within a day

| project TimeGenerated, Display_Name = DisplayName_s, Application_Client_ID = ApplicationId_Guid_g, Object_ID = ObjectId_g, Cert_or_Secret_ID = KeyId_g, Credential_Type = Type_s, Start_Date = StartDate_value_t, End_Date = EndDate_value_t, Expiration_Status = Status_s, Days_To_Expire = DaysToExpiration_d, Directory_Tenant_ID = TenantId // the columns have been renamed to easier to understand headings

Time Range: Last 24 hours
Chart Type: HTML Table

Let’s break down the Kusto query line by line:

Look up data in the following log:
AppRegistrationExpiration_CL

Retrieve only the latest records based on the TimeGenerated field using the unique KeyID_g field that represents the certificate or secret ID as a filter (this will cover App Registrations that have multiple certificates or secrets configured):
| summarize arg_max(TimeGenerated,*) by KeyId_g // arg_max is used to return the latest record for the time range selected and the * is to return all columns, records with unique KeyId_g will be returned so expired and multiple credentials are returned

Only list records where the certificate or secret will be expiring in 50 days or if it has already expired:
| where DaysToExpiration_d <= 50 or Status_s == "Expired" // this specifies a filter for the amount of days before expiry

Only list records with the TimeGenerated field within a day:
//| where TimeGenerated > ago(1d) // the TimeGenerated value must be within a day

Project the fields of interest for the report and rename them to more meaningful names:
| project TimeGenerated, Display_Name = DisplayName_s, Application_Client_ID = ApplicationId_Guid_g, Object_ID = ObjectId_g, Secret_ID = KeyId_g, Credential_Type = Type_s, Start_Date = StartDate_value_t, End_Date = EndDate_value_t, Expiration_Status = Status_s, Days_To_Expire = DaysToExpiration_d, Directory_Tenant_ID = TenantId // the columns have been renamed to easier to understand headings

Send an email (V2):

The report will include the Attachment Content and Attachment Name derived from the query with the subject Certificate & Secrets Expiry Report. The email will look pretty barebone so you are free to add HTML code to pretty it up.

Proceed to save the Logic App and use the Run Trigger to test the Logic App and confirm that an email is sent: 

**Note that I have set the kusto query to return records with an expiry of 1000 days or less so more records would return.

Hope this helps anyone who may be looking for a way to set up a Logic App for sending recurring reports of expiring App Registrations’ Certificates and Secrets information that were sent to a custom Log Analytics data.

I’ve recently been involved in a few virtual desktop architecture design and one of the topics I was asked to discuss was around the monitoring of the virtual desktops. Having worked with Citrix and VMware VDI solutions, I’ve always enjoyed the “out of the box” monitoring solutions Citrix Director (shown in the screenshot below) and VMware Horizon Help Desk Tool that were included.

The metrics and visualization that these tools provided were extremely valuable for understanding the overall health and troubleshooting when issues arise. Those who worked with Azure Windows Virtual Desktop in the very beginning will remember that these metrics were possible but an investment of time were needed to develop the kusto queries to retrieve the capture metrics from Log Analytics so dashboards could be created. The professionals in the Azure community provided many prewritten queries, workbooks and even Power Bi dashboards. Fast forward today, Microsoft has made it extremely easy to capture and include many “out-of-the-box” dashboards with just a few clicks. Furthermore, the log analytics data is still available for anyone who would like to add additional logs and metrics to capture to create customized reports.

There are many great posts available that show up to configure Log Analytics to capture data but some of these manual steps aren’t really necessary today so the purpose of this blog post is to demonstrate how to quickly set up Log Analytics with baseline configuration defined by Microsoft. Upon completion of the setup, we’ll see how many dashboards for monitoring and reports are already available.

Create Log Analytics Workspace to store Azure Virtual Desktop events and metrics

As always, we’ll begin by creating a Log Analytics Workspace that is dedicated to storing the logs collected from the Azure Virtual Desktop components (it is best not to much other logs into this workspace):

Configure the required retention for the data (the default is 30 days) and be mindful of how much data being ingested according to the size of the deployment:

Set up Configuration Workbook

It is possible to proceed to the Azure Virtual Desktop’s Host pools and Workspaces to enabling logging, then configure event and performance metrics monitoring for each session host but this can all be completed by using the Configuration Workbook. Navigate to Azure Virtual Desktop in the Azure portal:

Select the Insights blade, select the Host Pool to be configured and click on Open Configuration Workbook:

Select the Log Analytics workspace that was configured in the first step and then click on Configure host pool:

The template for configuring Host pool diagnostic settings will be displayed indicating the following categories will be captured:

• Management Activities
• Feed
• Connections
• Errors
• Checkpoints
• HostRegistration
• AgentHealthStatus

Proceed to deploy the template:

Note that the Host Pool’s Resources diagnostic settings is now configured:

Proceed to scroll down to the Workspace and click on Configure workspace:

The template for the workspace diagnostic settings will be displayed:

• Management Activities
• Feed
• Errors
• Checkpoints

Proceed to deploy the template:

Note that the Workspace’s Resources diagnostic settings is now configured:

Next, navigate to the Session host data settings tab:

Click on Configure performance counters to capture the recommended baseline counters as displayed on the right under Missing counters:

Click on Apply Config:

Note the performance counters that have been successfully added:

Proceed to configure the recommended event logs to be captured by clicking on Configure events:

Click on the Deploy button:

The Windows event logs that will be captured will be listed. Note that the Microsoft-Windows-GroupPolicy/Operational log is not included in the baseline but is one I added (more on this a bit later).

With the Resource diagnostic settings and Session host data settings configured, proceed to the Data Generated tab and a summary of the amount of Perf Counters, AVD Diagnostics and Events billed over the last 24hrs will be displayed. I’ve waited for a few days before capturing the screenshot so metrics would be displayed:

If we navigate to the Host Pool blade of the Azure Virtual Desktop deployment and click on the Diagnostic settings, we’ll see the configuration we have just completed. Some administrations will know that this is something that can be configured manually as well.

The same is for the Workspace as well:

Adding new Session Hosts to Log Analytics Workspace

It is important to remember to add new session hosts (VDIs) as they are added to the AVD deployment so they are monitored. To add new hosts, navigate to Azure Virtual Desktop > Insights:

A message indicating There are session hosts not sending data to the expected Log Analytics workspace. will be displayed if any are unmonitored. Otherwise the following dashboard will be displayed:

Out-of-the-Box Dashboards

Microsoft provides many out-of-the-box dashboards after completing the configuration.

• Connection diagnostics: % of users able to connect
• Connection performance: Time to connect (new sessions)
• Host diagnostics: Event log errors
• Host performance: Median input latency
• Utilization
• Daily connections and reconnections
• Daily alerts

Navigating to the Connection Diagnostics tab will provide the following metrics:

• Success rate of (re)establishing a connection (% of connections)
• Success rate of establishing a connection (% of users able to connect)
• Potential connectivity issues in Last 48 hours
• Connection activity browser for Last 48 hours
• Ranking of Errors impacting Connection activities in Last 48 hours

Navigating to the Connection Performance tab will provide the following metrics:

• Top 10 users with highest median time to connect
• Top 10 hosts with highest median time to connect
• Time to connect and sign in, end-to-end
• Time for service to route user to a host
• Round-trip time
• RTT median and 95th percentile for all hosts

Navigating to the Host Diagnostics tab will provide the following metrics:

• Host pool details
• Performance counters
• Events
• Host browser
• CPU usage
• Available memory

Navigating to the Host Performance tab will provide the following metrics:

• Input delay by host
• Input delay by process

Navigating to the Users tab will allow you to interactively search for a user and then provide the following metrics:

• Connections over time for tluk@contoso.com
• Feed refreshes by client and version
• Feed refreshes over time for tluk@contoso.com
• Connections by client and version
• Key usage numbers
• Connection activity browser for Last 48 hours
• Ranking of errors impacting Connection activities for tluk@contoso.com in Last 48 hours

Navigating to the Utilization tab will provide the following metrics:

• Sessions summary
• Max users per core
• Available sessions
• CPU usage
• Monthly active users (MAU)
• Daily connections and reconnections
• Daily connected hours
• Top 10 users by connection time
• Top 10 hosts by connection time

Navigating to the Clients tab will provide the following metrics:

• Active users by client type over time
• Usage by client version for all clients
• Users with potentially outdated clients (all activity types)

Navigating to the Alerts tab will provide the following metrics:

• Alerts over time
• Details filtered to all severities

And there you have it. It’s truly amazing the amount of dashboards made available with minimal amount of configuration for the environment.

Custom Monitoring of Metrics

I had indicated in one of the previous screenshots that I included the log Microsoft-Windows-GroupPolicy/Operational for the events captured and the reason for this is because I’ve worked in many projects for VDI deployments in the past where the virtual desktop solution was blamed for slow logon performance. One of the metrics I used quick frequent is the GPO processing that the Citrix Director dashboard provides and this value can be easily obtained by capturing the Microsoft-Windows-GroupPolicy/Operational log and using the following kusto queries:

User GPO processing:

// This query will retrieve the amount of time required for computer logon policy processing to complete by parsing ParameterXML

// The logon details can also be retrieved from EventData but we're using ParameterXml instead to demonstrate how to parse it

Event

| where EventLog == "Microsoft-Windows-GroupPolicy/Operational"

| where Computer contains "Server-or-Desktop-Name"

| where EventID == "8001"

| parse ParameterXml with * "<Param>" GPO_Processing_Seconds "</Param><Param>" Digit1 "</Param><Param>" Server_or_Computer "</Param><Param>" Digit2 "</Param><Param>" Boolean

| project TimeGenerated, Server_or_Computer, GPO_Processing_Seconds, RenderedDescription

https://github.com/terenceluk/Azure/blob/main/Kusto%20KQL/Get-User-Logon-Policy-Processing-Duration.kusto

Computer GPO processing:
// This query will retrieve the amount of time required for user logon policy processing to complete by parsing ParameterXML

// The logon details can also be retrieved from EventData but we're using ParameterXml instead to demonstrate how to parse it

Event

| where EventLog == "Microsoft-Windows-GroupPolicy/Operational"

| where Computer contains "Server-or-Desktop-Name"

| where EventID == "8001"

| parse ParameterXml with * "<Param>" GPO_Processing_Seconds "</Param><Param>" Digit1 "</Param><Param>" User "</Param><Param>" Digit2 "</Param><Param>" Boolean

| project TimeGenerated, Computer, User, GPO_Processing_Seconds, RenderedDescription

https://github.com/terenceluk/Azure/blob/main/Kusto%20KQL/Get-Computer-Logon-Policy-Processing-Duration.kusto

Other metrics can also be collected using kusto and your imagination is really the limit.

I hope this post served as a good refresher for anyone who hasn’t looked at Azure Virtual Desktop monitoring for a while and would like to know what features are available with minimal configuration. The following are Microsoft Azure Virtual Desktop related documentation I would highly recommend reading:

Sample Kusto Queries:
https://docs.microsoft.com/en-us/azure/virtual-desktop/diagnostics-log-analytics#example-queries

Using Log Analytics to Monitor AVD:
Walkthrough of setting up the diagnostics, events, performance, workbooks:
https://docs.microsoft.com/en-us/azure/virtual-desktop/azure-monitor

Monitoring Virtual Machines:
https://docs.microsoft.com/en-us/azure/azure-monitor/vm/monitor-virtual-machine

Diagnostic Logs References (fields):
WVDConnections
https://docs.microsoft.com/en-us/azure/azure-monitor/reference/tables/wvdconnections

WVDErrors
https://docs.microsoft.com/en-us/azure/azure-monitor/reference/tables/wvderrors

Troubleshoot Azure Monitor for Azure Virtual Desktop
https://docs.microsoft.com/en-us/azure/virtual-desktop/troubleshoot-azure-monitor

Log data ingestion time in Azure Monitor
https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-ingestion-time