Problem

You have a VMware Horizon View Connection Server 7.5.0 that was recently upgraded from 7.4.0 and you noticed that the VMware Horizon View Connection Server service does not start. Attempting to start the service shows that it starts but then stops a few seconds later:

Navigating to the VMware Horizon View Connection server logs directory at:

C:\ProgramData\VMware\VDM\logs

… and reviewing the wsnm_starts.txt file:

… show that the service starts and stops:

Reviewing the most recent log file log-2018-06-25.txt:

… shows a series of errors:

Navigating to the bottom of the log file reveals the following errors as the service stops:

2018-06-25T10:42:47.052-03:00 INFO (0714-1280) <logloaded> [MessageFrameWork] Plugin 'ws_winauth - VMware Horizon View Framework Windows Authentication Support' loaded, version=7.5.0 build-8583568, buildtype=release

2018-06-25T10:42:47.052-03:00 INFO (0714-0E90) <Service Main Thread> [wsnm] The VMware View System Service is starting

2018-06-25T10:42:47.068-03:00 INFO (0A94-12E8) <logloaded> [MessageFrameWork] Plugin 'ws_java_bridgeDLL - VMware Horizon View Framework Java Bridge' loaded, version=7.5.0 build-8583568, buildtype=release

2018-06-25T10:42:47.068-03:00 INFO (0A94-123C) <Main Thread> [MessageFrameWork] Program 'ws_MessageBusService - VMware Horizon View Java Component Service' started, version=7.5.0 build-8583568, pid=0xA94, buildtype=release, usethread=1, closeafterwrite=0, sessionId=0

2018-06-25T10:42:47.068-03:00 INFO (0A94-12D4) <Service Main Thread> [ws_MessageBusService] The service 'MessageBusService' is started

2018-06-25T10:42:47.068-03:00 INFO (0A94-13E4) <logloaded> [MessageFrameWork] Plugin 'ws_javaview - VMware Horizon View Framework Java Diagnostics' loaded, version=7.5.0 build-8583568, buildtype=release

2018-06-25T10:42:47.070-03:00 INFO (0A94-0CE4) <logloaded> [MessageFrameWork] Plugin 'mfw_java - VMware Horizon View Framework Java Native Support (64-bit)' loaded, version=15.0.0 build-8440871, buildtype=release

2018-06-25T10:42:47.211-03:00 ERROR (0A94-112C) <javabridge> [mfw_java] Java Bridge Managed failed on tmp =jniEnv->FindClass("com/vmware/vdi/mfwj/binaryResp")

2018-06-25T10:42:47.211-03:00 ERROR (0A94-112C) <javabridge> [mfw_java] Exception in BagConv::Init

2018-06-25T10:42:47.211-03:00 WARN (0A94-112C) <javabridge> [ws_java_bridgeDLL] Exception

2018-06-25T10:42:47.211-03:00 WARN (0A94-112C) <javabridge> [ws_java_bridgeDLL] in thread "main"

2018-06-25T10:42:47.211-03:00 ERROR (0A64-09F4) <Service Main Thread> [ws_ConnectionServer] Wait for READY state FAILED for dependent 'wsmsgbus', error: The specified service is stopping

2018-06-25T10:42:47.243-03:00 INFO (0A94-123C) <Main Thread> [ws_MessageBusService] The service 'MessageBusService' is stopped

2018-06-25T10:42:47.726-03:00 ERROR (0A64-09F4) <Service Main Thread> [ws_ConnectionServer] wsnm OpenProcess FAILED, error: 87 (The parameter is incorrect.)

2018-06-25T10:42:47.726-03:00 ERROR (0A64-09F4) <Service Main Thread> [ws_ConnectionServer] wsnm OpenProcess FAILED, error: 87 (The parameter is incorrect.)

2018-06-25T10:42:47.726-03:00 INFO (0714-0E90) <Service Main Thread> [wsnm] The VMware View System Service is shutting down

2018-06-25T10:42:47.726-03:00 WARN (0A64-0D40) <NodeManagerWatcher> [MessageFrameWork] Connection to Node Manager lost

2018-06-25T10:42:47.898-03:00 INFO (0714-04B4) <Main Thread> [wsnm] The VMware View System Service has stopped

2018-06-25T10:42:47.960-03:00 INFO (0A64-0440) <Main Thread> [ws_ConnectionServer] The service 'Broker' is stopped

2018-06-25T10:43:21.813-03:00 WARN (07D4-0C84) <3204> [v4v_broker_agent_svc] SocketChannel: Unable to connect to contoVV02:32111

Solution

There wasn’t much information available only aside from this old KB listed for version 6.2.x and older:

View Administrator portal fails to launch after upgrading from VMware Horizon View 5.2 to 5.3 (2075114)

https://kb.vmware.com/s/article/2075114

The environment for this example did not have vRealize Operations Manager but I went ahead and uninstalled the VMware Horizon 7 Connection Server component:

Ensured that I left the AD LDS InstanceVMwareVDMDS intact:

Then reinstalled VMware Horizon 7 as a replica server using the same AD LDS instance, which corrected the problem.

Problem

You have an existing Windows Server 2012 R2 server that you would like to manually sysprep:

So you proceed to navigate to the directory: C:\windows\system32\sysprep to execute the sysprep.exe:

Select the Generalize option and click OK:

The process starts but quickly fails with the error:

System Preparation Tool 3.14

A fatal error occurred while trying to sysprep the machine.

You navigate to the directory:

C:\windows\system32\sysprep\Panther

.. and find the following the content in the setuperr.log file:

2017-06-26 10:18:22, Error [0x0f0073] SYSPRP RunExternalDlls:Not running DLLs; either the machine is in an invalid state or we couldn't update the recorded state, dwRet = 0x1f

2017-06-26 10:18:22, Error SYSPRP WinMain:Hit failure while processing sysprep re-specialize internal providers; hr = 0x8007001f

2017-09-18 15:11:15, Error [0x0f0073] SYSPRP RunExternalDlls:Not running DLLs; either the machine is in an invalid state or we couldn't update the recorded state, dwRet = 0x1f

2017-09-18 15:11:15, Error SYSPRP WinMain:Hit failure while processing sysprep re-specialize internal providers; hr = 0x8007001f

2018-06-26 13:21:48, Error [0x0f0073] SYSPRP RunExternalDlls:Not running DLLs; either the machine is in an invalid state or we couldn't update the recorded state, dwRet = 0x1f

2018-06-26 13:21:48, Error SYSPRP WinMain:Hit failure while processing sysprep re-specialize internal providers; hr = 0x8007001f

2018-06-26 13:42:57, Error [0x0f0073] SYSPRP RunExternalDlls:Not running DLLs; either the machine is in an invalid state or we couldn't update the recorded state, dwRet = 0x1f

2018-06-26 13:42:57, Error [0x0f00ae] SYSPRP WinMain:Hit failure while processing sysprep cleanup external providers; hr = 0x8007001f

Solution

Most of the posts I found relating to this error message indicates that the rearm limit has been reached but executing slmgr.vbs /dlv to review the Remaining Windows rearm count does not appear to suggest this:

From there, I decided to try using the configuration that I usually use for a server that has exceed an arm count to see and it surprisingly corrects the issue.

Begin by launching the registry editor and navigating to:

HKEY_LOCAL_MACHINE\SYSTEM\Setup\Status\SysprepStatus\

Verify that the CleanupState registry key is set to 2:

Verify that the GeneralizationState is set to 7:

Uninstall and reinstall the MSDTC via the following commands:

msdtc -uninstall

msdtc –install

Proceed by navigating to the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SoftwareProtectionPlatform\

Locate the SkipRearm key and change the value to 1:

Rerunning the sysprep.exe executable to start the sysprep process should work as expected now:

Once the sysprep completes, use a utility such as PsGetsid64.exe to verify that a new SID was generated for the server.

Problem

You’re attempting to install a new role, such as mediation services, on an existing front end server but the process fails with the following message:

> Bootstrap-CsComputerLogging status to: C:\Users\a-tluk\AppData\Local\Temp\2\BootstrapFull-[2018_06_23][13_45_16].htmlChecking prerequisites for bootstrapper...Checking prerequisite WMIEnabled...prerequisite satisfied.Checking prerequisite NoBootstrapperOnBranchOfficeAppliance...prerequisite satisfied.Checking prerequisite SupportedOS...prerequisite satisfied.Checking prerequisite NoOtherVersionInstalled...prerequisite satisfied.Host name: drlyncstd01.contoso.comDisabling unused roles...Executing PowerShell command: Disable-CSComputer -Confirm:$false -Verbose -Report "C:\Users\a-tluk\AppData\Local\Temp\2\Disable-CSComputer-[2018_06_23][13_45_24].html"Checking prerequisites for roles...Checking prerequisite SupportedOS...prerequisite satisfied.Checking prerequisite SupportedOSNoDC...prerequisite satisfied.Checking prerequisite DotNet35...prerequisite satisfied.Checking prerequisite SupportedSqlRtcLocal...prerequisite satisfied.Checking prerequisite WMIEnabled...prerequisite satisfied.Checking prerequisite NoOtherVersionInstalled...prerequisite satisfied.Checking prerequisite PowerShell...prerequisite satisfied.Checking prerequisite SupportedServerOS...prerequisite satisfied.Checking prerequisite KB2533623Installed...prerequisite satisfied.Checking prerequisite SupportedSqlLyncLocal...prerequisite satisfied.Checking prerequisite SupportedSqlRtc...prerequisite satisfied.Checking prerequisite IIS...prerequisite satisfied.Checking prerequisite IIS7Features...prerequisite satisfied.Checking prerequisite KB2982006Installed...missingChecking prerequisite ASPNet...prerequisite satisfied.Checking prerequisite KB2646886Installed...prerequisite satisfied.Checking prerequisite BranchCacheBlock...prerequisite satisfied.Checking prerequisite WCF...prerequisite satisfied.Checking prerequisite WindowsMediaFoundation...prerequisite satisfied.Prerequisite not satisfied: Before you install Skype for Business Server 2015, you must install an update for Windows Server 2012 R2. For details about the update, see Microsoft Knowledge Base article 2982006, "IIS crashes occasionally when a request is sent to a default document in Windows 8.1 or Windows Server 2012 R2" at http://go.microsoft.com/fwlink/?LinkId=519376

You navigate to the provided URL:

http://go.microsoft.com/fwlink/?LinkId=519376

… which redirects you to the following KB:

IIS crashes occasionally when a request is sent to a default document in Windows 8.1 or Windows Server 2012 R2
https://support.microsoft.com/en-us/help/2982006/iis-crashes-occasionally-when-a-request-is-sent-to-a-default-document

Last Updated Dec 9, 2014

… download the update but realize that it will not install:

The update is not applicable to your computer.

Solution

The supplied KB is outdated and have since been updated with the following correct one:

Prerequisite (KB2982006) not satisfied when you try to install Skype for Business Server 2015
https://support.microsoft.com/en-us/help/4056288/can-t-install-skype-for-business-server-2015-due-to-missing-kb2982006

Cause

This issue occurs because the WMI query result of the prerequisite of KB2982006 isn't reliable.

To correct the issue, navigate to the following URL and install the January 2018 cumulative update 6.0.9319.514 for Skype for Business Server 2015, core components or later CU:
https://support.microsoft.com/en-us/help/4074705

I went ahead and installed the March 2018 CU:

Business Server 2015 Cumulative Update KB3061064 Mar 2018

… and was able to proceed with the install:

This is a follow up to a previous post I wrote about poor VMware Horizon View video, keyboard and mouse performance when adding a 3^rd monitor to the Dell Wyse 7020 Windows 10 IoT thin client:

Adding a 3rd monitor to a Dell Wyse Z90QQ10 thin client connecting to a VMware Horizon View 6.2 virtual desktop causes slow video performance with mouse movement and typing delays

http://terenceluk.blogspot.com/2018/04/adding-3rd-monitor-to-dell-wyse-z90qq10.html

The poor performance issue began affecting our Microsoft Windows 2016 RDS services, which escalated the priority so I was finally able to take 2 days to troubleshoot the problem. I had also opened up a ticket with VMware and Dell prior to beginning troubleshooting the issue myself and the VMware EUC engineering said this was likely a Dell Wyse issue since it does not happen on a full thick PC while Dell never got back to me at all after a ticket was opened. I’ve never been impressed with Dell support for the Wyse devices but figured I’d give them a try but they have yet to call me days after the ticket was opened and I don’t have much confidence that I would receive a call back soon.

Before I begin, the following are the environment details:

Hardware

Thin Client: Dell Wyse Z90QQ10 Thin Client

Operation System: Windows 10 Enterprise 2015 LTSB

Monitors: 3 x Dell P2414Hb (1920 x 1080) and 3 x Dell U2417H (1920 x 1080)

View Horizon Client: VMware Horizon View Client 4.8.0 build-8547331

Troubleshooting

The first item I looked at the start of the troubleshooting was the PCoIP Client Session Variables configuration provided by VMware to apply to the Horizon View Client and in particular the Configure PCoIP client image cache size policy and Configure the PCoIP session bandwidth floor settings:

Both of these settings did not help with the problem so I moved on to upgrading the VDI’s virtual machine version (no improvement), increasing resources (no improvement) then reviewing the drivers available for the two video cards in the thin client but noticed that there was an 2 month newer update for the AMD Radeon HD 8330E but no update for the AMD Radeon E6460:

Realizing I’m not going to have much luck with the drivers or the Horizon View Client application tweaking, I started reviewing the display output connections of the thin client:

After trying different combinations to connect the 3 monitors, I concluded that the degrade in performance happens when both of the video cards are used. Independently using the video cards to display 2 monitors at a time was fine but using 1 output connector from each video card for 2 monitors degraded performance immediately. Having ran out of ideas, I decided to review the specifications to see if I had missed something:

https://www.dell.com/en-us/work/shop/wyse-endpoints-and-software/wyse-7000-series-thin-clients-high-performance-virtual-desktop/spd/wyse-z-class

What I immediately noticed was that the thin client actually supported 6 displays as indicated under the Display specifications:

Wyse 7020
DisplayPort: 2560 x 1600 @32bpp
Dual DisplayPort: 2560 x 1600 @32bpp
DVI-I: 1920 x 1200 @32bpp
Dual Display: 1920 x 1200 @32bpp
Four Displays (DVI & DisplayPort): (1) 1920x1200@32bpp, (3) 2560x1600 @32bpp
Four Displays (DisplayPort daisy-chain): 3840x2160@32bpp
Six Displays (DisplayPort daisy-chain): 2560x1600@32bpp

I have to admit that I don’t really follow the advancements of monitor outputs so I was unfamiliar with DisplayPort daisy-chain but taking it literally made me realize that I could potentially connect more than 2 monitors to the same video card. The Dell P2414Hb (1920 x 1080)

monitors we had in the office did not support daisy chaining so we ended up purchasing 3 x Dell U2417H (1920 x 1080) to test this capability and I immediately noticed that daisy chaining 3 monitors to the AMD Radeon HD 8330E video card provided optimal performance in Horizon View. I then went and tried to the do the same with the AMD Radeon E6460 and received an error indicating it only supported 2 monitors via a single daisy chained display port.

It was a bit absurd to find that this was the problem and I could not find any information about this anywhere on the internet so I hope this post would help anyone who may encounter the same problem. Below is a photo of the Dell Wyse 7020 back panel with the video cards labeled and a table of the test results:

AMD Radeon E6460

AMD Radeon HD 8330E

**Any combination of mixing the AMD Radeon E6460 and HD 8330E resulted in poor performance and would generated the following low on memory message for the VMware Remote MKS service:

Close programs to prevent information loss

Your computer is low on memory. Save your files and close these programs:

VMware Remote MKS

Problem

You need to add an additional RD Session Host Server to an existing RDS deployment then create a new RDS Collection with the server so you proceed to add the server:

Once you’ve successfully added the server, you proceed to add the server as a RD Session Host Servers in the Remote Desktop Services management console:

Once successfully added as a RD Session Host server, you proceed to create a new collection with the server:

The collection is successfully created by the add server operation fails with the following error message:

<ServerName>.FQDN Unable to configure the RD Session Host server <ServerName>.FQDN. Invalid operation

Solution

As per the following KB article:

You cannot create a session collection and an error occurs in Windows Server 2012
https://support.microsoft.com/en-us/help/3014614/you-cannot-create-a-session-collection-and-an-error-occurs-in-windows

… this can be caused by GPOs configuring one of the following two settings:

Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Security

Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Session Host\Licensing

In this example, the environment had a GPO that configured the RDS Licensing server:

To correct the issue, move the computer object to another OU to prevent the GPO from being applied thus unconfiguring the licensing server:

Delete the collection and recreate it:

Many of the clients I work with have been taken an interest in implementing warning banners for all emails received from outside of their organizations in an effort to combat the ever increasing phishing emails that make it through their SPAM filters.  A great write up by Joe Palarchio can be found here:

Office 365 – Providing Your Users Visual Cues About Email Safety
https://blogs.perficient.com/2016/04/04/office-365-providing-your-users-visual-cues-about-email-safety/

The blog post provides a great description for the configuration but many of the clients I work with felt the colours and border weren’t as prominent as they would like so I’d like to share the HTML code I use to generate the following banner for anyone who may want to use it:

Note that this is a mail flow rule:

The following is the HTML code:

<div style='border:dotted #003333 1.0pt;padding:1.0pt 1.0pt 1.0pt 1.0pt'>

<p class=MsoNormal style='background:#F4DF11'><span lang=EN-GB
style='font-size:10.0pt;font-family:"Cambria",serif;color:red;mso-ansi-language:
EN-GB;mso-fareast-language:EN-GB'>CAUTION:</span><span lang=EN-GB
style='font-size:10.0pt;font-family:"Cambria",serif;color:black;mso-ansi-language:
EN-GB;mso-fareast-language:EN-GB'> This email originated from outside of the
organization. Do not click links or open attachments unless you recognize the
sender and know the content is safe.</span><span lang=EN-GB style='font-size:
12.0pt;font-family:"Times New Roman",serif;mso-ansi-language:EN-GB;mso-fareast-language:
EN-GB'><o:p></o:p></span></p>

</div>
<BR>

-----------------------------------------------------------------------------------------------------------------------------------------

Note that my experience with the implementation banner was well received by IT infrastructure teams but often poorly received by users because such a banner fills up the body preview on smart phones rendering that feature to become useless so it would be wise to pilot this with users before implementing it globally.

Problem

You’ve deployed a new Windows Server 2016 RDS environment and published RemoteApps but received complains that when a user’s session times out after the configured idle limit, they receive the following Windows and unable to click the OK button:

Idle timer expired

Session has been idle over its time limit.

It will be disconnected in 2 minutes.

Click OK to stay connected.

The problem with the window above is that the RDS RemoteApp session has disconnected but the window indicating the end of the session is stuck behind this warning window. There is really nothing the user can do to get the window in the background to get on top of this one so they need to terminate the RDS session via the task bar.

One of the solutions that correct this issue is to disable the Use advanced RemoteFX graphics for RemoteApp configuration found:

Computer Configuration > Policies > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Remote Session Environment

Disabling this feature corrects the problem but it creates a new problem where if the user’s desktop launching the RemoteApp does not have the left most monitor as their primary monitor:

What happens with this setup for users is that they would launch the RemoteApp without any issues:

The application will work as expected but if the user minimizes it:

Then attempts to restore the RemoteApp, the application will attempt to be restored on the left non-primary monitor and display a black screen that the user cannot interact with:

This appears to only be a display issue because the user can right-click on the application in the task bar, close it, relaunch the application and not lose any work. What’s strange is that this does not appear to affect applications that are not maximized meaning if the application was launched and then minimized as such:

The application would restore without any issues. Another alternative workaround is to configure the left most monitor as the primary:

Another workaround I was able to find was to limit the amount of monitors for the RemoteApps with the Limit number of monitors configuration:

While this corrects the issue, it restricts the application to the primary monitor disallowing the user to drag the window to the left or any other monitor and this was likely going to be very annoying.

I had opened a case with Microsoft about two weeks back, which was closed because I couldn’t replicate it on my desktop (I always use my left most monitor as primary) but have been since reopened this week after figuring this out. The engineer hasn’t called me back yet but knowing the cause allowed me to find this forum post discussing the same problem on Windows 10:

[Windows 10 1709] Issues when maximizing RDP App

https://social.technet.microsoft.com/Forums/lync/en-US/831fda26-1336-4806-a3eb-8b989e023a52/windows-10-1709-issues-when-maximizing-rdp-app?forum=win10itprogeneral

I had this issue too. I found that re-enabling remotefx on session servers made the issue go away. But now window focus is messed up, when a new window pop up in a remote application it goes behind the main application until user clicks out of the app, the pop up will appear.

The environment I’m experiencing this problem uses Windows 7 as the desktop so I can confirm that this isn’t limited to Windows 10. There isn’t a resolution in the forum post so I hope to get to the bottom of this and share the resolution.

Problem

You’ve just used the Skype for Business Server 2015 – Deployment Wizard to request a certificate for the internal interface of the Lync Edge server:

The process completes with warnings:

Viewing the logs displays the following warning:

Warning: The chain of the certificate "CA6EEEC4F50136BCDF70F2A6369C3189F4B7F980" is invalid.

Clicking the Next button displays the following message:

A certificate with thumbprint CA6EEEC4F50136BCDF70F2A6369C3189F4B7F980 has been added to the local certificate store.

The certificate has been issued by the online certification authority and is installed to the local certificate store, however it is not valid. Make sure that the Root certificate, and necessary certificate chain is installed on this server.

You notice that it is not available to be assigned and launching the properties of the certificate display the following:

The integrity of this certificate cannot be guaranteed. The certificate may be corrupted or may have been altered.

Clicking on the Certification Path tab displays the following Certificate status:

This certificate has an invalid digital signature.

Clicking on the Root CA shows This certificate is OK for the Certificate status indicating there are no issues with the Root Certificate Authority:

However, opening the properties of the Root certificate displays properties that does not appear to be expected root CA. In this example, I noticed that the Valid from date was incorrect (it appeared to belong to the old root CA’s certificate):

The Signature algorithm and Signature hash algorithm was also incorrect because I had upgraded them to sha256:

What I did was launch the Certification Authority management console on the root CA and locate the issued certificate:

The properties showed that the status was in a healthy state:

Opening the properties of the root CA certificate confirmed my suspicion that the Skype for Business Edge server is displaying a different root CA as the issuer of the certificate because the Valid from dates were different:

The Signature algorithm and Signature hash algorithm was shown and expected to be sha256:

Solution

One of the causes of such an issue is if you haven’t updated the root certificate in the Trusted Root Certification Authorities on the Skype for Business Edge server:

In this example, the server had the older sha1 certificate but not the newer sha256 so it seemingly decided to display the newly issued certificate as one created by the old CA. To correct this problem, export and import the new sha256 root certificate into the Trusted Root Certification Authorities on the Skype for Business Edge server and the newly issued certificate for the Edge server will be displayed properly so you can assign to the Edge service.

Problem

You’ve received complaints from users that they are experiencing out of sync messages between devices such as their mobile phones compared to the Skype for Business clients.  Logging onto the Skype for Business front-end server and reviewing the Lync Server logs show that event ID 32054 errors are logged and refer to the users who have complained about the issue:

Log Name: Lync Server
Source: LS Storage Service
Event ID: 32054

Storage Service had an EWS Autodiscovery failure.

StoreWebException: code=ErrorEwsAutodiscover, reason=GetUserSettings failed, smtpAddress=ssteele@contoso.com, Autodiscover Uri=https://autodiscover.contoso.com/autodiscover/autodiscover.svc, Autodiscover WebProxy=<NULL>, WebExceptionStatus=NameResolutionFailure ---> System.Net.WebException: The remote name could not be resolved: 'autodiscover.contoso.com'

   at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)

   at System.Net.HttpWebRequest.GetRequestStream()

   at Microsoft.Exchange.WebServices.Autodiscover.AutodiscoverRequest.InternalExecute()

   --- End of inner exception stack trace ---

   at Microsoft.Rtc.Internal.Storage.Exchange.ExchangeContext.SendGetUserSettingsRequest(StoreContext ctx, String smtpAddress)

   at Microsoft.Rtc.Internal.Storage.Exchange.ExchangeContext.GetUserEwsSettings(StoreContext ctx, String smtpAddress, CacheMode cacheMode)

Cause: Autodiscovery Uri was not correctly configured or unreachable, that there is a problem with the Proxy, or other errors.

Resolution:

Check event details.  Check autodiscovery Uri is properly configured and reachable. Check that proxy setting is properly configured and reachable.  Validate Skype for Business to Exchange Autodiscovery configuration by following the trouble shooting guide. If problem persists, notify your organization's support team with the event details.

Solution

One of the reasons why this event error would be logged causing the out of sync messages is if the Exchange autodiscover URL is unreachable as indicated in the event ID details:

StoreWebException: code=ErrorEwsAutodiscover, reason=GetUserSettings failed, smtpAddress=ssteele@contoso.com, Autodiscover Uri=https://autodiscover.contoso.com/autodiscover/autodiscover.svc, Autodiscover WebProxy=<NULL>, WebExceptionStatus=NameResolutionFailure ---> System.Net.WebException: The remote name could not be resolved: 'autodiscover.contoso.com'

The environment where I encountered this issue did not use the standard Exchange autodiscover URL and therefore caused this issue for their SfB users.  One of the workarounds for this issue is to manually configure Skype for Business to use a different autodiscover URL but before doing so, confirm that there isn’t already an existing manually created configuration by executing the cmdlet:

Get-CsOAuthConfiguration

Note that a standard configuration should provide output similar to the above.  Having confirmed that the existing configuration is standard, proceed to execute the following cmdlet to manually configure a reachable Exchange autodiscover URL:

Set-CsOAuthConfiguration -ExchangeAutodiscoverUrl "https://<TheAutoDiscoverDomain>/autodiscover/autodiscover.svc" -Realm "<theSIPdomain>" -Exchange AutodiscoverAllowedDomains "*.<sipDomain>.com"

The event log errors should clear and the following information event will be logged once the SfB front-end server is able to reach the Exchange autodiscover service via the newly configured autodiscover URL.

Log Name: Lync Server
Source: LS Autodiscover
Event ID: 32007
Level: Information

Autodiscover OAuth configuration was successfully retrieved.

Problem

You’re attempting to upgrade a HP VC FlexFabric-20/40 F8 Module from firmware version 4.45 2015 to 4.62 Feb 2018 (https://support.hpe.com/hpsc/swd/public/detail?sp4ts.oid=6894628&swItemId=MTX_7fae1562133f483abb29f204b7&swEnvOid=4184#):

You download the HP Virtual Connect Support Utility and use the update command to upgrade the firmware but the process fails with:

Error: The signature verification of specified package has failed

Using the packageinfo command on the vcfwall462.bin file returns the same Package Signature: Invalid output:

Downloading the older 4.50 firmware (https://support.hpe.com/hpsc/swd/public/detail?sp4ts.oid=6894628&swItemId=MTX_1f352fb404f5410d9b2ca1b56d&swEnvOid=4184#) and using the packageinfo command display a Package Signature: Valid output:

The Active Onboard Administrator firmware version of the chassis is at version 4.70 May 18, 2017:

Solution

One of the reasons why the upgrade would exhibit such a behavior is if you are using an older version of the HP Virtual Connect Support Utility. In the case of this example, the older version 1.11.1 was used and updating the utility to version 1.13.0 (https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_7df0afcd8cef4984a78f4a888c#tab-history) corrects the issue:

Problem

You’re attempting to renew the Skype for Business Server 2015 front-end server with an internal Microsoft Enterprise Certificate Authority but receive the following error:

> Request CertificateRequest-CSCertificate -New -Type Default,WebServicesInternal -CA "DC1.corp.contoso.bm\Contoso CA" -Country "BM" -State "Hamilton" -City "Hamilton" -FriendlyName "Skype for Business Server 2015 Default certificate 9/12/2018" -KeySize 2048 -PrivateKeyExportable $False -Organization "Contoso" -OU "IT" -AllSipDomain -Verbose -Report "C:\Users\administrator\AppData\Local\Temp\Request-CSCertificate-[2018_09_12][13_59_43].html"Creating new log file "C:\Users\ccs.contosoCORP\AppData\Local\Temp\Request-CSCertificate-[2018_09_12][13_59_43].xml".Create a certificate request based on Skype for Business Server configuration for this computer.Creating new log file "C:\Users\ccs.contosoCORP\AppData\Local\Temp\Request-CSCertificate-[2018_09_12][13_59_43].html". WARNING: Request-CSCertificate failed. WARNING: Detailed results can be found at "C:\Users\ccs.contosoCORP\AppData\Local\Temp\Request-CSCertificate-[2018_09_12][13_59_43].html".Command execution failed: Error Parsing Request A required certificate is not within its validity period when verifying against the current system clock or the timestamp in the signed file. 0x800b0101 (-2146762495 CERT_E_EXPIRED)

Solution

One of the reasons why this error would be thrown if you are using an internal Microsoft Enterprise CA is if the issuing Root CA’s certificate has expired. If you are able to confirm that this is the cause then simply log onto the Root CA’s Certificate Authority administration console and renew the certificate then request a new certificate:

I was recently involved in assisting a customer with removing Sophos Endpoint Security and Control from their desktops and laptops because it was being replaced with Cylance Protect.  Anti-Virus products aren’t something that I typically deal with so I had to do a bit of searching on the internet to come up with script that would check for the services and then subsequently remove them.  What I noticed during the process was that the Sophos KB article:

How to uninstall Sophos Endpoint Security and Control from the command line or with a batch file
https://community.sophos.com/kb/en-us/109668

… wasn’t very helpful because I wasn’t able to find all of the components in the registry to extract the uninstall GUID to pair with the msiexec.exe /x command.  Another item I noticed was that there are a lot of sample scripts and batch files available but they only worked for specific versions.

So after spending a good half day on this, I thought it would be a good idea to share what finally worked for me in case anyone out there looking for this.  As the title of this post indicates, this works for version 10.7 and 10.8.

Sophos Install Script – Copy and paste below into a .cmd file

REM Check for the Sophos service

@Echo off

REM --- Check for an existing installation of Sophos Agent

IF NOT EXIST "C:\Program Files (x86)\Sophos\Remote Management System\ManagementAgentNT.exe" (

REM --- Check for an existing installation of Sophos Anti-Virus

IF NOT EXIST "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\SavService.exe" (

REM --- Check for an existing installation of Sophos Anti-Virus status reporter

IF NOT EXIST "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\SAVAdminService.exe" (

REM --- Check for an existing installation of Sophos AutoUpdate on 32-bit (the 'Sophos AutoUpdate Service' process)

IF NOT EXIST "C:\Program Files\Sophos\AutoUpdate\ALsvc.exe" (

REM --- Check for an existing installation of Sophos AutoUpdate on 64-bit (the 'Sophos AutoUpdate Service' process)

IF NOT EXIST "C:\Program Files (x86)\Sophos\AutoUpdate\ALSVC.exe" (

REM --- Check for an existing installation of Sophos Message Router

IF NOT EXIST "C:\Program Files (x86)\Sophos\Remote Management System\RouterNT.exe" (

REM --- Check for an existing installation of Sophos Network Threat Protection

IF NOT EXIST "C:\Program Files\Sophos\Sophos Network Threat Protection\bin\SntpService.exe" (

REM --- Check for an existing installation of Sophos System Protection Service

IF NOT EXIST "C:\Program Files (x86)\Sophos\Sophos System Protection\ssp.exe" (

REM --- Check for an existing installation of Sophos Web Control Service

IF NOT EXIST "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\Web Control\swc_service.exe" (

REM --- Check for an existing installation of Sophos Web Intelligence Service 64-bit

IF NOT EXIST "C:\ProgramData\Sophos\Web Intelligence\swi_update_64.exe" (

REM --- Check for an existing installation of Sophos Web Intelligence Service 32-bit

IF NOT EXIST "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\Web Intelligence\swi_service.exe" (

REM --- Check for an existing installation of Sophos Anti-Virus on Vista+ (the SAV adapter config file)

IF NOT EXIST "C:\ProgramData\Sophos\Remote Management System\3\Agent\AdapterStorage\SAV\SAVAdapterConfig" (

goto _End

))))))))))))

REM --- Disabling Sophos Services

sc config "Sophos Agent" start= disabled

sc config "SAVService" start= disabled

sc config "SAVAdminService" start= disabled

sc config "Sophos AutoUpdate Service" start= disabled

sc config "Sophos Message Router" start= disabled

sc config "SntpService" start= disabled

sc config "sophossps" start= disabled

sc config "Sophos Web Control Service" start= disabled

sc config "swi_service" start= disabled

sc config "swi_update_64" start= disabled

REM --- Stopping Sophos Services

net stop "Sophos AutoUpdate Service"

net stop "Sophos Agent"

net stop "SAVService"

net stop "SAVAdminService"

net stop "Sophos Message Router"

net stop "Sophos Web Control Service"

net stop "swi_service"

net stop "SntpService"

net stop "sophossps"

net stop "swi_filter"

REM --- Taskkill all services just in case services such as AutoUpdate is running and will not stop

taskkill /im ManagementAgentNT.exe /f

taskkill /im SavService.exe /f

taskkill /im SAVAdminService.exe /f

taskkill /im ALsvc.exe /f

taskkill /im RouterNT.exe /f

taskkill /im SntpService.exe /f

taskkill /im ssp.exe /f

taskkill /im swc_service.exe /f

taskkill /im swi_update_64.exe /f

REM --- Disable Tamper Protection

reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\TamperProtection\Config" /v SAVEnabled /t REG_DWORD /d 0 /f

reg add "HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Sophos\SAVService\TamperProtection" /v Enabled /t REG_DWORD /d 0 /f

REM --- Begin uninstall Sophos Components

REM --- Sophos Network Threat Protection

start /wait MsiExec.exe /X{66967E5F-43E8-4402-87A4-04685EE5C2CB} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-NTPLog.txt

REM --- Sophos System Protection

start /wait MsiExec.exe /X{1093B57D-A613-47F3-90CF-0FD5C5DCFFE6} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-SPLog.txt

REM --- Sophos Anti-Virus

start /wait MsiExec.exe /X{23E4E25E-E963-4C62-A18A-49C73AA3F963} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-AVLog.txt

REM --- Sophos Anti-Virus 10.7

start /wait MsiExec.exe /X{65323B2D-83D4-470D-A209-D769DB30BBDB} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-SAVlog.txt

REM --- Sophos Anti-Virus 10.8

start /wait MsiExec.exe /X{6654537D-935E-41C0-A18A-C55C2BF77B7E} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-SAVlog.txt

REM --- Sophos Remote Management System

start /wait MsiExec.exe /X{FED1005D-CBC8-45D5-A288-FFC7BB304121} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-RMSLog.txt

REM --- Sophos AutoUpdate

taskkill /im ALsvc.exe /f

start /wait MsiExec.exe /X{AFBCA1B9-496C-4AE6-98AE-3EA1CFF65C54} /qn REBOOT=SUPPRESS /L*v %windir%\Temp\Uninstall_SOPH-AULog.txt

REM --- Sophos Endpoint Defense

"C:\Program Files\Sophos\Endpoint Defense\uninstall.exe"

REM --- End of the script

:_End

-----------------------------------------------------------------------------------------------------------------------------------------------

Here are also a few useful items that I had to document during the process of developing the command above

List of Services and Executable Locations

Obtaining the GUIDs of Applications

You can obtain the GUID applications using the PowerShell cmdlet:

wmic product get > C:\InstalledPrograms.txt

Locate the GUID of the application that needs to be removed and modify the script to include correct GUID.

Problem

You attempt to start a VMware vSphere virtual machine but notice that the Windows operating system fails to boot:

Windows Boot Manager

Windows failed to start. A recent hardware or software change might be the cause. To fix the problem:

Insert your Windows installation disc and restart your computer.

Choose your language settings, and then click “Next.”

Click “Repair your computer.”

If you do not have this disc, contract your system administrator or computer manufacturer for assistance.

Info: There isn’t enough memory available to create a ramdisk device.

Status: 0xc0000017

Solution

I’ve received numerous calls in the past 2 years about this error and found that this is usually caused by an incorrect memory configuration in the virtual machine settings where the value is too low for the Windows operating system (Windows Server 2016 in this case) to start. The settings window presented to an administrator in vSphere 6.5 tends to use MB for the Memory value and therefore causes situations where an administrator would inadvertently put in a value assuming it was GB. Below is an example of a VM inadvertently configured with 64MB of memory instead of the intended 64GB:

Problem

You attempt to connect to the login page of a vCenter 6.5 server via the vSphere Web Client but receive the following error message from the browser:

503 Service Unavailable (Failed to connect to endpoing: [class Vmacore::Http::NamedPipeServiceSpec:0x00000263204438b0] _serverNamespace = / action = Allow _pipeName =\\.\pipe\vmware-vpxd-webserver-pipe)

Solution

While there are a possible causes for this error, one of the reasons why the message is presented is if the Postgres database is unable to start thus causing the vCenter login portal to not load.  For this example, the reason why the Postgres database engine would not start is because the Windows Server hosting the vCenter services and Postgres database had the security permissions for the Log on as a batch job overwritten by a group policy in the domain.  This is what was displayed when I checked the vCenter Windows Server Log on as a batch job policy accounts:

Adding the following local server accounts back in corrected the issue:

• vapiEndpoint
• vmware-vpostgres
• vscan-health
• vsm
• vsphere-client
• vsphere-ui

Troubleshooting

This issue took quite a bit of time traversing through various logs to determine the root cause and I did not want to include all the information in the solution section in case the reader needed the remediation quickly so I will put them here.

The first logs I reviewed were the vCenter logs on the Windows server located here:

C:\ProgramData\VMware\vCenterServer\logs\vmware-vpx

I opened up the latest vpxd log and managed to find a line indicating there was an error authenticating the administrator@vsphere.local account:

2018-10-06T05:44:52.744+01:00 error vpxd[04128] [Originator@6876 sub=User opID=5aa4e999] Failed to authenticate user <administrator@vsphere.local>

Doing a bit of searching for this error directed me to this KB article:

Logging in to the vCenter Server Appliance fails with the error: Failed to authenticate user (2147174)
https://kb.vmware.com/s/article/2147174

… but I quickly determined that it did not apply to this issue.

Continuing to review the logs revealed the following lines that appear to suggest there may be a problem with the Postgres SQL database:

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] SQL execution failed: DELETE FROM VPX_IP_ADDRESS WHERE ENTITY_ID = ?

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] Execution elapsed time: 4113 ms

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] Statement diagnostic data from driver is 57P01:0:34:Could not receive the response, communication down ??;

--> Could not send Query(connection dead)

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] Bind parameters:

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0

2018-10-06T05:44:46.752+01:00 error vpxd[09488] [Originator@6876 sub=Default opID=HB-host-15@26976-e46a4d] [VdbStatement] value = 115

2018-10-06T05:44:47.152+01:00 warning vpxd[06528] [Originator@6876 sub=VpxProfiler opID=ProcessTaskNotifications-56e509fe] ProcessTaskNotifications [TotalTime] took 5070750068 ms

2018-10-06T05:44:47.153+01:00 warning vpxd[09280] [Originator@6876 sub=VpxProfiler opID=ProcessEventNotifications-6058ed8] ProcessEventNotifications [TotalTime] took 5070750084 ms

2018-10-06T05:44:47.228+01:00 warning vpxd[09432] [Originator@6876 sub=VpxProfiler opID=ProcessScheduledTaskFiring-1c06dac8] ProcessScheduledTaskFiring [TotalTime] took 5070750086 ms

2018-10-06T05:44:49.538+01:00 info vpxd[09552] [Originator@6876 sub=SupportMgr] Wrote uptime information

2018-10-06T05:44:49.603+01:00 error vpxd[09552] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1

2018-10-06T05:44:49.603+01:00 error vpxd[09552] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: SELECT count(1) as TOTAL FROM VPX_DATACENTER

2018-10-06T05:44:49.603+01:00 error vpxd[09552] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 30 ms

2018-10-06T05:44:49.603+01:00 error vpxd[09552] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is 08S01:0:34:Could not receive the response, communication down ??;

From here, I went ahead to launch the ODBC Data Source Administrator (64-bit) to check the System DSN that connected to the PostgreSQL database, ran the test connection and immediately noticed that it was not accessible:

A bit more searching lead me to the following KB:

Connecting to the embedded vPostgres Database in a Windows installed vCenter Server 6.0 (2108848)
https://kb.vmware.com/s/article/2108848

Using the instructions provided in the KB, I navigated to the following directory to review the vcdb.properties file with the PostgreSQL database information:

C:\ProgramData\VMware\vCenterServer\cfg\vmware-vpx

Then launched the command prompt, navigated to the following directory:

C:\Program Files\VMware\vCenter Server\vPostgres\bin\

Then attempted to connect to the database with the following command:

psql -U username vc VCDB

… but quickly got the same error as the ODBC system DSN connection attempt:

From here, I navigated to the vCenter’s bin folder:

C:\Program Files\VMware\vCenter Server\bin

… then used the following command to list the status of the vCenter services:

service-control --status

The output above is not what is to be expected from a healthy vCenter and seeing the vPostgres service Stopped confirms the unavailability of the database that vCenter would require to start up properly.

-------------------------------------------------------------------------------------------------------------------------------------------------------

Note that this is what the status on a healthy VCSA service status should look like:

Command> service-control --status

Running:

applmgmt lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-psc-client vmware-rhttpproxy vmware-sca vmware-sps vmware-statsmonitor vmware-sts-idmd vmwa re-stsd vmware-updatemgr vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmwar e-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-client vsphere-ui

Stopped:

vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmwa re-vcha

Command>

-------------------------------------------------------------------------------------------------------------------------------------------------------

Attempting to manually start the vPostgres service with the following command shows that it immediately fails:

service-control --start vPostgres

C:\Program Files\VMware\vCenter Server\bin>service-control --start vPostgres

Perform start operation. vmon_profile=None, svc_names=['vPostgres'], include_coreossvcs=False, include_leafossvcs=False

2018-10-06T12:56:32.655Z Service vmware-vpostgres state STOPPED

Error executing start on service vmware-vpostgres. Details {

"resolution": null,

"detail": [

{

"args": [

"vmware-vpostgres"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'vmware-vpostgres'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

Service-control failed. Error {

"resolution": null,

"detail": [

{

"args": [

"vmware-vpostgres"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'vmware-vpostgres'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

C:\Program Files\VMware\vCenter Server\bin>

I was able to find this following KB after searching for failed PostgreSQL database but it did not apply to this situation:

Starting vPostgres fails after vCenter Server Appliance failure (2106584)
https://kb.vmware.com/s/article/2106584

Trying to initiate the command to start all of the services will stall and eventually fail when attempting to start services dependent on the database:

service-control --start –all

Attempting to stop all services successfully completes:

service-control --stop --all

Starting all the services again would have the process stuck on vmware-psc-client:

Eventually timing out with this output message:

Service-control failed. Error Failed to start vmon services.vmon-cli RC=1, stderr=Failed to start eam, vapi-endpoint, vsphere-client, vpxd-svcs, vsphere-ui, vmware-vpostgres services. Error: Operation timed out

From here, I navigated to the PostgreSQL logs directory:

C:\ProgramData\VMware\vCenterServer\logs\vpostgres

… opened the latest log file and found the following output:

2018-10-06 05:40:19.419 BST 5b6adb7d.1344 0 LOG: Writing instance status...

2018-10-06 05:40:19.420 BST 5b6adb7d.1344 0 LOG: Wrote instance status successfully.

2018-10-06 05:40:19.420 BST 5b6adb7d.1344 0 LOG: Updated instance status successfully.

2018-10-06 05:41:19.422 BST 5b6adb7d.1344 0 LOG: Updating instance status...

2018-10-06 05:41:19.422 BST 5b6adb7d.1344 0 LOG: Writing instance status...

2018-10-06 05:41:19.424 BST 5b6adb7d.1344 0 LOG: Wrote instance status successfully.

2018-10-06 05:41:19.424 BST 5b6adb7d.1344 0 LOG: Updated instance status successfully.

2018-10-06 05:41:32.146 BST 5bb80915.577c 0 VCDB vc LOG: duration: 20170.000 ms bind _PLAN000001331116CC90: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME, USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME, COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME, DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME, NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME, STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID FROM VPXV_EVENT_ALL WHERE COMPUTERESOURCE_ID=$1 AND CREATE_TIME>=$2 AND (EVENT_TYPE IN ($3,$4,$5,$6)) ORDER BY EVENT_ID DESC LIMIT 100;

2018-10-06 05:41:32.146 BST 5bb80915.577c 0 VCDB vc DETAIL: parameters: $1 = '7', $2 = '2018-10-06 00:41:11.969+01', $3 = 'vim.event.DrsVmMigratedEvent', $4 = 'vim.event.DrsExitedStandbyModeEvent', $5 = 'vim.event.DrsVmPoweredOnEvent', $6 = 'vim.event.DrsEnteredStandbyModeEvent'

2018-10-06 05:41:41.189 BST 5bb80915.577c 0 VCDB vc LOG: duration: 9031.000 ms bind _PLAN000001331116B490: SELECT EVENT_ID, ARG_ID, ARG_TYPE, ARG_DATA, OBJ_TYPE, OBJ_NAME, VM_ID, HOST_ID, COMPUTERESOURCE_ID, DATASTORE_ID, NETWORK_ID, NETWORK_TYPE, DVS_ID, DATACENTER_ID, RESOURCEPOOL_ID, FOLDER_ID, ALARM_ID, SCHEDULEDTASK_ID FROM VPXV_EVENT_ARG WHERE (EVENT_ID IN ($1,$2,$3,$4,$5))

2018-10-06 05:41:41.189 BST 5bb80915.577c 0 VCDB vc DETAIL: parameters: $1 = '677130', $2 = '677036', $3 = '676762', $4 = '676753', $5 = '675555'

2018-10-06 05:42:19.426 BST 5b6adb7d.1344 0 LOG: Updating instance status...

2018-10-06 05:42:19.430 BST 5b6adb7d.1344 0 LOG: Writing instance status...

2018-10-06 05:42:19.431 BST 5b6adb7d.1344 0 LOG: Wrote instance status successfully.

2018-10-06 05:42:19.431 BST 5b6adb7d.1344 0 LOG: Updated instance status successfully.

2018-10-06 05:43:19.433 BST 5b6adb7d.1344 0 LOG: Updating instance status...

2018-10-06 05:43:19.433 BST 5b6adb7d.1344 0 LOG: Writing instance status...

2018-10-06 05:43:19.434 BST 5b6adb7d.1344 0 LOG: Wrote instance status successfully.

2018-10-06 05:43:19.434 BST 5b6adb7d.1344 0 LOG: Updated instance status successfully.

2018-10-06 05:44:19.436 BST 5b6adb7d.1344 0 LOG: Updating instance status...

2018-10-06 05:44:19.436 BST 5b6adb7d.1344 0 LOG: Writing instance status...

2018-10-06 05:44:19.437 BST 5b6adb7d.1344 0 LOG: Wrote instance status successfully.

2018-10-06 05:44:19.437 BST 5b6adb7d.1344 0 LOG: Updated instance status successfully.

2018-10-06 05:44:29.038 BST 5b6adb7d.1344 0 LOG: Updating instance status...

2018-10-06 05:44:29.184 BST 5b6adb7d.1c30 0 LOG: received fast shutdown request

2018-10-06 05:44:29.465 BST 5b6adb7d.1344 0 ERROR: canceling statement due to user request

2018-10-06 05:44:30.372 BST 5b6adb7d.1c30 0 LOG: aborting any active transactions

2018-10-06 05:44:33.229 BST 5b6adb7d.1fd0 0 ERROR: canceling statement due to user request

2018-10-06 05:44:33.231 BST 5bb82535.68ec 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:35.589 BST 5bb82535.58dc 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:38.297 BST 5bb82535.5160 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:39.128 BST 5bb82530.6420 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:41.974 BST 5bb80915.577c 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:44.535 BST 5bacb856.5f24 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:47.545 BST 5bacb855.62a4 0 VCDB vc FATAL: terminating connection due to administrator command

2018-10-06 05:44:50.366 BST 5b8f9b4c.5c74 0 VCDB vc FATAL: terminating connection due to administrator command

After a bit more troubleshooting from here was when I decided to check the Log on as a batch job configuration to ensure the following local server accounts were present and was able to confirm that they were not:

• vapiEndpoint
• vmware-vpostgres
• vscan-health
• vsm
• vsphere-client
• vsphere-ui

Proceeding to add the appropriate accounts back corrected the issue:

The following is what a healthy Windows vSphere Server status should look like:

C:\Program Files\VMware\vCenter Server\bin>service-control --status

Running:

VMWareAfdService VMWareCertificateService VMWareDirectoryService VMwareComponentManager VMwareDNSService VMwareIdentityMgmtService VMwareSTS VServiceManager rhttpproxy vPostgres vapiEndpoint vimPBSM vmon vmonapi vmsyslogcollector vmware-cis-config vmware-license vmware-perfcharts vmware-psc-client vmwareServiceControlAgent vpxd vpxd-svcs vsan-health vsphere-ui vspherewebclientsvc

Stopped:

EsxAgentManager VMWareCAMService content-library mbcs vmware-autodeploy-waiter vmware-imagebuilder vmware-network-coredump

C:\Program Files\VMware\vCenter Server\bin>

Note that having the PostgreSQL database up will still throw an error when you attempt to test the connection:

FATAL: password authentication failed for user “vc”

It has been a while since I’ve deployed a VDP appliance and since I had the opportunity to do so this week, I took the time to screenshot the process so I can write what would likely be the last post for this application since VMware will no longer be releasing new versions in the future.  The deployment hasn’t changed much throughout the versions so this post serves more to demonstrate what the deployment looks like rather than how.

Deploying the OVA

One of the features I’ve enjoyed using since vSphere 6.0 is Content Libraries but I’d like to note that you can only deploy OVF packages and notOVA files from Content Libraries so the best way to start the appliance deployment is simply use the Deploy OVF Template… option from within the vSphere Client or vSphere Web Client to select the download OVA file:

Select the Local file radio button and click on Choose Files to select the downloaded OVA VDP appliance:

vSphereDataProtection-6.1.9.ova

Proceed by selecting the folder location and configuring a meaningful name for the appliance:

Select the appropriate cluster for the appliance:

Review the details:

Accept the EULA as we always do:

Select the datastore that will store the appliance:

Configure the port group (VLAN) for the appliance:

Configure the network properties for the appliance:

Review the configuration and commence the deployment:

Configuring the appliance

Power on the VDP appliance once the OVA deployment has completed and wait until the operating system has completed resulting with the following screen displayed:

Navigate to the appliance via https://<IP Address>/vdp-configure and login with:

root / changeme

The following configuration will be presented:

Configure the IP settings as well as create an A (forward) and PTR (reverse) record for the appliance in your DNS:

Forgetting to create the DNS records will throw the following error:

IP/DNS could not be resolved. Check the IP, DNS values for forward and reverse lookup validation.

Configure the appropriate time zone:

Configure a password for the appliance that will change the default changeme password:

Configure the connect to the vCenter that this VDP appliance will connect to:

Click on the Test Connection button before attempting to hit the Next button as it is a requirement:

This part of the wizard is where you define the type of storage the VDP appliance will use.  The first option is to create new storage by adding VMDK disks to the VDP appliance to store backups, the second option is to attached existing disks from another VDP appliance and the third option is to migrate VDP storage data from another VDP release.  For the purpose of this example, we’ll be creating new storage that is 0.5TB in size because the intention is to attached this VDP deployment to a Data Domain.

Leave the Device Allocation settings as the default unless you want to separate the disks from the actual appliance:

Configure the CPU and Memory according to the recommended settings for the size of your backups as noted here:

https://docs.vmware.com/en/VMware-vSphere/6.5/vmware-data-protection-administration-guide-61.pdf

Decide whether you want to join the feedback program (not sure whether this is relevant since this would be the last release:

Complete the deployment and decide whether to run the performance analysis on the storage configuration.  I will not be selecting this because we’ll be attaching this to a data domain:

The deployment will display the login screen during the configuration but don’t bother trying to log in as you won’t be able to:

Instead, proceed to the console of the appliance and wait until it restarts and boots back into the OS:

Log back into the appliance and you should see the appliance successfully deployed:

Note the size of the disks that are configured for a 0.5TB deployment:

boot – 200GB

data01 – 256GB

data02 – 256GB

data03 – 256GB

For comparison purposes, these are the size of the disks for a 1TB deployment:

boot – 200GB

data01 – 512GB

data02 – 512GB

data03 – 512GB

One of the things I eventually realized at the end of this deployment was that the Data Domain in this environment had a DDOS (Data Domain Operating System) that was at a version that no longer supported the VDP appliance since VMware was no longer going to release this product so when I attempted to connect to the DD, I will receive the following error:

Failed to add the Data Domain.

Reasons: Failed to connect to Data Domain system. The hostname, user name and/or password may be invalid.

Here is a are two snippets from the EMC site:

On April 5th, 2017, VMware announced the End of Availability (EOA) of the VMware vSphere Data Protection (VDP) product. At this time the latest release of VDP uses DDboost library 3.1.0.1 which is compatible up to DDOS 5.7.X. Due to the End of Availability announcement there are no plans to introduce additional DDboost libraries to allow support of later DDOS releases.  
For additional information on VDP/Data Domain compatibility please see the DDBoost for VMware compatibility Matrix found here:

DDBoost Compatibility Matrix

For additional information on the end of service life for VMware Data Protection please see the following documentation from VMware:

End of Availability (EOA) of VMware vSphere Data Protection (2149614)

----------------------------------------------------------------------------------------------------------------------------------------------------

I’ve had several colleagues reach out to me in the past to ask about how I normally troubleshoot an Agent unreachable issue when all of the usual checks were verified (e.g. desktop being powered on, an IP address was obtained, etc) so I thought I’d take the opportunity to write this blog post using a recent issue I encountered at one of the environments I work with.

Problem

You’ve noticed that desktops in a Desktop pool has the Status reported as being Agent unreachable:

Clicking on the ellipsis displays the following details:

Status: Agent unreachable

Pairing state:Paired and secured

Configured by: <viewConnectionServerFQDN>

Attempted theft by:

Troubleshooting

Assuming the following obvious items have already been verified:

1. IP address was obtained
2. There is connectivity between the View Connection server and VDI
3. Desktop is powered on
4. All Horizon View Agent services have been started
5. Desktop’s domain secure connection is not broken

… the first step for troubleshooting issues that have limited information provided in the error details is to review the Horizon View Agent logs on the problematic virtual desktop via the following folder:

C:\ProgramData\VMware\VDM\logs

What I typically do when in the log folder is to sort the files by Date modified so the most recently modified files are displayed on top.  For this example, the latest debug log that had been modified was:

debug-2018-10-09-154215.txt

… so I opened the file in Notepad and began reviewing the entries:

Parsing through the logs allowed me to identify the following 3 suspicious entries:

2018-10-09T15:42:25.726-03:00 DEBUG (0F6C-09CC) <Thread-4> [BrokerUpdateUtility] Published CHANGEKEY request

2018-10-09T15:42:34.277-03:00 DEBUG (0724-0728) <Main Thread> [wsnm_desktop] WTS_CONSOLE_CONNECT, sessionId=1

2018-10-09T15:42:35.745-03:00 DEBUG (0F6C-09CC) <Thread-4> [BrokerUpdateUtility] Timeout waiting for success response

2018-10-09T15:42:16.553-03:00 WARN (0724-079C) <PluginInitThread> [ws_configmgr] Update rejected, attempt to reconfigure as paired with missing key information

2018-10-09T15:44:23.232-03:00 INFO (0724-0F38) <JavaBridge> [wsnm_jmsbridge] wsnm_jms died, restarting in a minute

After reviewing the agent logs of the virtual desktop, I proceeded to log onto the VMware View Horizon Connection Server to review the server logs located in the folder:

C:\ProgramData\VMware\VDM\logs

The most recent modified log file was:

log-2018-10-09.txt

… so I opened the file in Notepad and began reviewing the entries until I found the following:

2018-10-09T00:00:08.386-03:00 WARN (0EC8-0A34) <DesktopControlJMS> [JMSMessageSecurity] Message could not be validated: Signature invalid for identity agent/4f631619-2cf1-4676-a63e-a37ecc4b9e80

2018-10-09T00:00:08.386-03:00 WARN (0EC8-0A34) <DesktopControlJMS> [JMSMessageSecurity] Identity validation failed: UNKNOWN

2018-10-09T00:00:08.386-03:00 WARN (0EC8-0A34) <DesktopControlJMS> [DesktopTracker] CHANGEKEY message from agent/4f631619-2cf1-4676-a63e-a37ecc4b9e80 is discarded as it cannot be validated

The entry references a GUID of a desktop and not its name so to determine which desktop this GUID belongs to, I launched adsiedit.msc on the View Connection server:

Then used these instructions to open the local VMware Horizon View ADAM database:

Connecting to the View ADAM Database (2012377)
https://kb.vmware.com/s/article/2012377

Name: View ADAM Database

DistinguishedName: dc=vdi,dc=vmware,dc=int

Server: localhost:389

Once connected to the ADAM database, navigate to the Servers OU then locate the unique GUID in the right hand pane:

Open the properties of the object and navigate down to the attribute named pae-DisplayName to locate the name:

Solution

Quickly searching for the identified log entries above brought me to the following KB:

After reinstalling or upgrading the View agent, the View Administrator console reports the message: Agent Unreachable (2038679)
https://kb.vmware.com/s/article/2038679

Using the provided command::

vdmadmin -A -d desktop-pool-name -m name-of-machine-in-pool -resetkey

… located in the directory:

C:\Program Files\VMware\VMware View\Server\tools\bin

Resolved the issue.

The following are the screenshots of what the output for the command would look like:

Note that a restart is not required for the status of the VDIs to return to normal:

Problem

You’ve noticed that the Content Library service on your Windows vCenter Server 6.5 has stopped:

C:\Program Files\VMware\vCenter Server\bin>service-control --status

Running:

VMWareAfdService VMWareCertificateService VMWareDirectoryService VMwareComponentManager VMwareDNSService VMwareIdentityMgmtService VMwareSTS VServiceManager rhttpproxy vPostgres vapiEndpoint vimPBSM vmon vmonapi vmsyslogcollector vmware-cis-config vmware-license vmware-perfcharts vmware-psc-client vmwareServiceControlAgent vpxd vpxd-svcs vsan-health vsphere-ui vspherewebclientsvc

Stopped:

EsxAgentManager VMWareCAMService content-library mbcs vmware-autodeploy-waiter vmware-imagebuilder vmware-network-coredump

C:\Program Files\VMware\vCenter Server\bin>

**Note that the content-library service on a Windows vCenter is named content-library while the VCSA has it named vmware-content-library so if you attempt to start the service with the supplied command in the KB then you’ll receive the error below:

C:\Program Files\VMware\vCenter Server\bin>service-control --status vmware-content-library

Failed to get service vmware-content-library status. Err Given service name vmware-content-library is invalid

Service-control failed. Error Given service name vmware-content-library is invalid

C:\Program Files\VMware\vCenter Server\bin>

Proceeding to start the service on the Windows vCenter 6.5 server failed with the following error:

C:\Program Files\VMware\vCenter Server\bin>service-control --start content-library

Perform start operation. vmon_profile=None, svc_names=['content-library'], include_coreossvcs=False, include_leafossvcs=False

2018-10-19T16:19:40.231Z Service content-library state STOPPED

Error executing start on service content-library. Details {

"resolution": null,

"detail": [

{

"args": [

"content-library"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'content-library'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

Service-control failed. Error {

"resolution": null,

"detail": [

{

"args": [

"content-library"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'content-library'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

C:\Program Files\VMware\vCenter Server\bin>

Attempting to start the Content Library Service from within the vSphere Web Client (Home > Administration > System Configuration > Services > Objects > Services > Content Library Service) will also fail:

The "Start service" operation failed for the entity with the following error message.

Error (com.vmware.vapi.std.errors.error) => {

messages = [],

data = <null>

}

Reviewing the content library logs show that it has not been updated during the time of the troubleshooting (this is because it is unable to start so no logs would be written):

C:\ProgramData\VMware\vCenterServer\logs\content-library

Solution

One of the reasons why the content library service on a Windows Server vCenter 6.5 server won’t start is if the appropriate local account created during the vCenter 6.5 server install no longer has the Log on as a batch job permission on the Windows server. In the case of this example, checking the properties of the permissions showed that the local server content library account was missing:

Manually adding the account back into the security permission corrected the issue:

It is also important to note that the accounts listed in the screenshots above are incomplete for all the vCenter Server services to function properly as there are many more accounts that need to be added as shown in the list below:

• cm
• content-library
• eam
• imagebuilder
• mbcs
• netdumper
• perfcharts
• rbd
• vapiEndpoint
• vmware-vpostgres
• vsan-health
• vsm
• vsphere-client
• vsphere-ui

Note that the list above can be found in this VMware KB:

Error "Logon failure: the user has not been granted the requested logon type at this computer" (2148054)
https://kb.vmware.com/s/article/2148054

The properties of the Log on as a batch job should look something like the screenshots below:

With the appropriate account added, the content library service should start as expected:

C:\Program Files\VMware\vCenter Server\bin>service-control --start content-library

Perform start operation. vmon_profile=None, svc_names=['content-library'], include_coreossvcs=False, include_leafossvcs=False

2018-10-19T17:00:34.224Z Service content-library state STOPPED

Successfully started service content-library

It has been a while since I’ve deployed a vSphere Replication Appliance and since I had the opportunity to do so this week, I took the time to screenshot the process so I can write this blog post demonstrated how to use the Content Library for the deployment.

Begin by downloading the vSphere Replication appliance from the VMware portal:

VMWare-vSphere_Replication-8.1.0-9466424.iso

As the supplied download is an ISO file, you’ll need to unpack the package into a folder as such:

The files we’ll need to upload into the Content Library is located in the bin folder:

With the contents ready for upload, log into the vSphere Client, navigate into a Content Library that has been created, select Templates, click on Actions and then Import item:

Select Localfile and then click on the UPLOAD FILE button:

Select the vSphere_Replication_OVF10.ovf file in the folder and click Open:

What’s nice about the import process is that it will notify you if you forget to include the required VMDK files as shown in the screenshot below:

Proceed by clicking on the UPLOAD button again and select the required files:

The wizard should now have the IMPORT button active:

Clicking on the IMPORT button will proceed with the upload:

Once the import / upload has completed, the ovf file should now be available under the Templates section:

Proceed by right clicking on the vSphere_Replication_OVF10.ovf file and select New VM from This Template…:

Provide a name for the appliance in the Virtual machine name text field:

Select a compute resource:

A brief details summary will be provided

Agree to the EULA as we always do:

Select the appropriate vCPU configuration for the appliance:

The following VMware document provides information on the decision for either a 2 or 4 vCPU configuration:

https://docs.vmware.com/en/vSphere-Replication/6.5/com.vmware.vsphere.replication-admin.doc/GUID-E654F2D8-7D56-4A81-9568-E85172A7022D.html

Note:

Selecting higher number of vCPUs ensures better performance of the vSphere Replication Management Server, but might slow down the replications that run on ESXi host systems that have 4 or less cores per NUMA node. If you are unsure what the hosts in your environment are, select 2 vCPUs.

I’ll be selecting 2 vCPU for this example:

Select the datastore for the appliance:

Select a VLAN for the management network:

Fill in the required fields for the appliance:

Review and click Next:

Review the deployment summary and click on Finish to begin the deployment:

The Recent Tasks should now display the deployment status of the appliance:

The new appliance should also be displayed in the client:

Proceed by powering on the appliance:

Wait until the following is displayed in the console:

Launch a browser and navigate to:

https://<replication appliance IP/FQDN>:5480

Log in with the user name root and the password configured earlier in the wizard:

Click on the Configuration button and fill in the appropriate fields for the vCenter information:

**Note that the VRM service is stopped and it is expected as the configuration has not completed.

Proceed to click on Save and Restart Service once the configuration parameters have been configured and you should see the status Verifying LookupService’s SSL certificate to be displayed:

The following Confirm SSL Certificate window will be displayed if you are using a self-signed certificate on the vCenter server so proceed by clicking on the Accept button:

The status Saving configuration will be displayed:

The process should completed with Successfully saved the configuration displayed and the VRM service shown to be running:

The Site Recovery icon will not immediately show up:

Attempting to log out and back in immediately will likely who the following message:

There are plug-ins that were installed or updated. They will be ready for use next time you log into vSphere Client.

After logging out a second time and back in should now show the Site Recovery icon:

Problem

You’re attempting to deploy and OVF within a VMware vCenter Server 6.5 installed on a Microsoft Windows Server but noticed that the deployment within the vSphere Web client fails with the following error:

This version of vCenter Server does not support Deploy OVF Template using this version of vSphere Web Client. To Deploy OVF Template, login with version 6.5.0.0 of vSphere Web Client.

You noticed that an attempt to use the vSphere Client (HTML5) will show that the Deploy OVF Template… is greyed out:

You’ve confirmed that all of the services within the Windows services console are started:

Searching the internet with the error message and symptoms returns plenty of results for the VCSA (vCenter Server Appliance) pointing to the following KB:

OVF deployment fails after upgrading to vCenter Server Appliance 6.5 U1 (2151085)
https://kb.vmware.com/s/article/2151085

However, the vCenter Server for this example is not an appliance but rather Windows and the version is 6.5 U2c (Build: 8815520):

Build numbers and versions of VMware vCenter Server (2143838)
https://kb.vmware.com/s/article/2143838

Troubleshooting

Having no luck finding any other articles or blog posts on the internet that applied to issue with the Windows version of vCenter, I went ahead and checked the Content Library service as noted in the VCSA KB article and noticed that it was indeed stopped:

C:\Program Files\VMware\vCenter Server\bin>service-control --status

Running:

VMWareAfdService VMWareCertificateService VMWareDirectoryService VMwareComponentManager VMwareDNSService VMwareIdentityMgmtService VMwareSTS VServiceManager rhttpproxy vPostgres vapiEndpoint vimPBSM vmon vmonapi vmsyslogcollector vmware-cis-config vmware-license vmware-perfcharts vmware-psc-client vmwareServiceControlAgent vpxd vpxd-svcs vsan-health vsphere-ui vspherewebclientsvc

Stopped:

EsxAgentManager VMWareCAMService content-library mbcs vmware-autodeploy-waiter vmware-imagebuilder vmware-network-coredump

C:\Program Files\VMware\vCenter Server\bin>

**Note that the content-library service on a Windows vCenter is named content-library while the VCSA has it named vmware-content-library so if you attempt to start the service with the supplied command in the KB then you’ll receive the error below:

C:\Program Files\VMware\vCenter Server\bin>service-control --status vmware-content-library

Failed to get service vmware-content-library status. Err Given service name vmware-content-library is invalid

Service-control failed. Error Given service name vmware-content-library is invalid

C:\Program Files\VMware\vCenter Server\bin>

Proceeding to start the service on the Windows vCenter 6.5 server failed with the following error:

C:\Program Files\VMware\vCenter Server\bin>service-control --start content-library

Perform start operation. vmon_profile=None, svc_names=['content-library'], include_coreossvcs=False, include_leafossvcs=False

2018-10-19T16:19:40.231Z Service content-library state STOPPED

Error executing start on service content-library. Details {

"resolution": null,

"detail": [

{

"args": [

"content-library"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'content-library'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

Service-control failed. Error {

"resolution": null,

"detail": [

{

"args": [

"content-library"

],

"id": "install.ciscommon.service.failstart",

"localized": "An error occurred while starting service 'content-library'",

"translatable": "An error occurred while starting service '%(0)s'"

}

],

"componentKey": null,

"problemId": null

}

C:\Program Files\VMware\vCenter Server\bin>

Attempting to start the Content Library Service from within the vSphere Web Client (Home> Administration > System Configuration > Services > Objects > Services > Content Library Service) will also fail:

The "Start service" operation failed for the entity with the following error message.

Error (com.vmware.vapi.std.errors.error) => {

messages = [],

data = <null>

}

Attempting to locate the ts-config.properties file as shown in the VCSA KB article will show that it exists but the corresponding ts-config.properties.rpmnew does not:

C:\ProgramData\VMware\vCenterServer\cfg\content-library\config

The content library logs also has not been updated during the time of the troubleshooting (this is because it is unable to start so no logs would be written):

C:\ProgramData\VMware\vCenterServer\logs\content-library

Solution

One of the reasons why the content library service on a Windows Server vCenter 6.5 server won’t start is if the appropriate local account created during the vCenter 6.5 server install no longer has the Log on as a batch job permission on the Windows server. In the case of this example, checking the properties of the permissions showed that the local server content library account was missing:

Manually adding the account back into the security permission corrected the issue:

It is also important to note that the accounts listed in the screenshots above are incomplete as there are many more accounts that need to be added as shown in the list below:

• cm
• content-library
• eam
• imagebuilder
• mbcs
• netdumper
• perfcharts
• rbd
• vapiEndpoint
• vmware-vpostgres
• vsan-health
• vsm
• vsphere-client
• vsphere-ui

Note that the list above can be found in this VMware KB:

Error "Logon failure: the user has not been granted the requested logon type at this computer" (2148054)
https://kb.vmware.com/s/article/2148054

The properties of the Log on as a batch job should look something like the screenshots below:

With the appropriate account added, the content library service should start as expected:

C:\Program Files\VMware\vCenter Server\bin>service-control --start content-library

Perform start operation. vmon_profile=None, svc_names=['content-library'], include_coreossvcs=False, include_leafossvcs=False

2018-10-19T17:00:34.224Z Service content-library state STOPPED

Successfully started service content-library

You should now be able to deploy an OVF from either the vSphere Web Client or vSphere Client (HTML5):

Problem

You’ve completed configuring VMware Horizon View with SecurEnvoy but when authentication fails with AccessDenied:

Reviewing the SecurEnvoy logs reveal the following error:

Incorrect Soft Token Code Received From ClientIP=10.34.30.58 RemoteID=

Solution

One of the possible reasons why authentication would not work and this message is logged in the Log Viewer is if the Shared Secret configured on the VMware Horizon View Connection Server does not match the one configured in the corresponding Radius server in SecurEnvoy:

The following message should be logged once the authentication succeeds:

Access Accepted with Soft Token From ClientIP=10.34.30.58 RemoteID=