Tag Archives: failed

VMware STS Clients Failed SSL Certificate of STS Service Cannot Be Verified

“Initialization of STS Clients failed. Root Cause: The SSL certificate of STS service cannot be verified” is an error which put a delay in deployment of the vShield Manager.

VMware STS Clients Failed Error

During the configuration of the Lookup Service Information, we encountered this particular error. It important to understand how the environment was designed when we hit this error and why it didn’t seem to make sense at first .

There are two sites, Site A and Site B, in a hybrid vCenter 5.1 configuration running vCenter 5.5 Single Sign-On and Web Client on their own dedicated virtual machines, SSO1 and SSO2. vCenter 5.5 Single Sign-On and the Web Client both reside on the same server, one in each site. There are a total of 5 vCenter Servers that are at 5.1 U1/U2 versions. Each vCenter is pointed at their corresponding site/geographic regions’ vCenter 5.5 Single Sign-On and Web Client server.

VMware Single Sign-On SSO Architecture

This model is fully supported by VMware per KB2059249 and has proven to be an ideal deployment model in the vCenter 5.1 product family than the initial release of Single Sign-On 5.1.

The vShield Manager was deployed at Site B and we used Site B’s SSO and Web Server address when configuring the Lookup Service. After research, internet forums indicated that the certificate of the SSO server, chain and root certificates needed to be bundled into a single certificate and installed on the STS server. This did not make sense since no certificates were manually generated for use by the SSO servers. All SSO certificates were generated during installation and we’re self signed by the VMware SSO installer.

VMware STS Clients Failed Error

While working with a co-worker to troubleshoot the issue above, it occurred to me to list all services that the SSO server see’s to determine what STS service that the SSO server was using. After issuing the following command on the SSO server:

ssolscli listServices https://cgvccore2.fqdn:7444/lookupservice/sdk


VMware STS Clients Failed Error Proof

The urn:sso:sts service was listed with Site A’s registered URL! It completely slipped my mind that there was only one STS server listed in any SSO instance. We updated the Lookup Service Information Host URL and the “Initialization of STS Clients failed. Root Cause: The SSL certificate of STS service cannot be verified” issue was resolved!

VMware STS Clients Failed Error Resolved

Note: This is single point of failure, it would be best to load balance the STS service. There are articles to update where the STS service is pointing to the event of a failure if a load balance model is not implemented initially.

Tagged , , , , , ,

vSphere Inventory Search 403 Query Service Failed Forbidden

The 403 error which I encountered was tied to the vSphere Client Login Screen. If the “Use Windows Session Credentials” is checked this would cause the 403 errors when searching for a virtual machine.

The work around is to type in the username and password you are authenticating as which bypasses and saved session credentials.

There are many articles about the 403 Forbidden error using the Search Inventory feature within the vSphere Client. There are a differnt range of solutions and work arounds including reinstalling vCenter! Reinstalling vCenter just didn’t sit well with me since that can be a long process depending on your environment and how your organization responds to certain changes.

Login to the query service failed. The server could not interpret the communcation from the client. (The remote server returned an error: (403) Forbidden.)

404 Error vSphere Login Screenshot

When investigating the log files on the vCenter Server for the Inventory Service located at:

       “C:ProgramDataVMwareInfrastructureInventory ServiceLogsds.log”

The following two errors stood out when related to this issue:

       “WARN com.vmware.vim.vcauthorization.impl.AuthorizationManagerImpl]
Unable to find user data for user: DOMAINUser”

       “ERROR com.vmware.vim.vcauthorization.impl.PrincipalContextImpl]
Failed to get group memembership”

Root Cause:
After reaching out to support, it turns out that the issue is at login. If the following option is used, “”. It can cause a 403 Error when using the Inventory search.

Work Around: Type in the username and password, even if it is the same identity you are logging in as.

vSphere Login 404 Workaround Screenshot

VMware Support Suggested Permanent Fix: Upgrade to vCenter Server 5.1 Update 1.

Tagged , , , , ,

Failed to Start Migration Pre-copy Error 0xbad003f vMotion Migration Fix

“A general system error occurred: Failed to start migration pre-copy. Error 0xbad003f. Connection closed by remote host, possibly due to timeout.”
“A general system error occurred: Failed to start migration pre-copy. Error 0xbad004b. Connection reset by peer.”

Another issue, that I recently came across was a live vMotion issue where the vMotion migration would fail during the pre-copy and always at 10%. The following issues were either one of the two:

VMware vCenter vSphere Event Log

I performed some basic troubleshooting such as a vmkping. I used the ping command and watched the response times remain consistent during the attempted vMotion migration. No packets were being lost which I thought that there would be packet loss if there was an issue with Layer 3 IP addressing.

VMware ESXi vmkping

While still on the command line with the ESXi host, I decided to look for any arp entries anyways regardless of my logic to rule it out. I ran the following:

cat /var/log/vmkernel | grep arp

I was wrong, there was another host on the network that had the same IP address!

VMware ESXi Log

I found a new IP address for my VMKernel, updated DNS then updated the IP address on the ESXi host and my issue was resolved!

Tagged , , , , , ,