May 27, 2022

# Intro

As part of our ongoing work on our Attack Surface Management platform we are continually researching new and relevant vulnerabilities. In some cases, we’ve experienced other talented researchers finding vulnerabilities in software we audited. This blog post attempts to trace the steps of someone else’s vulnerability research with the ultimate goal of better understanding how to discover these issues ourselves in the future.

This mentality is something that I’ve really appreciated from a friend of mine, Steven Steely, who sees every vulnerability missed during research as a true blessing, a gift, an opportunity to become better at vulnerability research. We love this culture, and we want to do our part at propagating it.

The vulnerability was found by Bruno López of Innotec Security and there is a writeup on this vulnerability by the Horizon3 team, which can be found here.

However, after our team read it, we were itching for more details about the root cause, so this is our blog post on the issue, which aims to scratch that itch if you felt the same way. The blog post linked above demonstrates the issue, but does not have the vulnerable code path, but rather the mitigation details. While it’s great to see how this vulnerability was mitigated, we’re curious about what caused it in the first place, so we know what to look for next time.

# Hunting for the controller

We’ve got a copy of the vulnerable version of VMWare Workspace One Access, and we’ve gone through the extremely boring process of setting it up (oh the joys of vulnerability research). At this stage, we want to try and narrow down exactly where this vulnerability exists in code.

A great place to start is inspecting the logs for any activity or exceptions when executing the exploit. This might give us some great insight into where we can discover our affected controllers.

The logs for VMWare Workspace One Access are located at /opt/vmware/horizon/workspace/logs. We navigate to this directory and we run tail -f * which will tail all the logs.

While monitoring the logs, we fire off a request reproducing the PoC with a non-existent user, to understand what logs return in response to an invalid user

Request:

POST /SAAS/auth/login/embeddedauthbroker/callback HTTP/1.1
Host: 727onc12i6e997lb8nlja4e5cwip6e.oastify.com

… omitted for brevity …


Logs:

==> horizon.log <==
2022-05-28T12:17:37,020 WARN  (Thread-178) [ACCESS;-;192.168.1.38;aff62942f834035:127.0.0.1;-] com.vmware.horizon.service.controller.BaseController - Destination provided was a malformed URL 324fa2f7-b09b-4481-a69e-f9ed244c70e1
2022-05-28T12:17:37,056 INFO  (Thread-178) [ACCESS;-;192.168.1.38;aff62942f834035:127.0.0.1;-] com.vmware.horizon.frontend.service.SessionSupport - Not authenticated. Invalidating session for request: https://727onc12i6e997lb8nlja4e5cwip6e.oastify.com/SAAS/auth/login/embeddedauthbroker/callback with query string:null
2022-05-28T12:17:37,222 INFO  (Thread-178) [ACCESS;-;192.168.1.38;aff62942f834035:127.0.0.1;-] com.vmware.horizon.service.controller.auth.LoginController - Did not find valid auth relay info in persistence. This must be a new request. Generated a new auth relay cd5c36f4-a8ad-408c-afa7-2a6fb7f6c31c
2022-05-28T12:17:37,231 INFO  (Thread-178) [ACCESS;-;192.168.1.38;aff62942f834035:127.0.0.1;-] com.vmware.horizon.service.controller.auth.LoginController - Start authenticating null with embedded auth broker
2022-05-28T12:17:39,440 INFO  (Thread-178) [ACCESS;-;192.168.1.38;aff62942f834035:127.0.0.1;-] com.vmware.horizon.federationbroker.FederationBrokerService - JIT is not enabled and no users found with attribute value: test.
com.vmware.horizon.common.api.components.exceptions.UserNotFoundException: user.not.found



Based off the logs, we want to see the code for com.vmware.horizon.service.controller.auth.LoginController.

During the search for this controller, we realised why this controller may not have been missed by other people. The workflow we were following when we decompiled the source code was the following:

cd SAAS/WEB-INF/lib
for f in *.src.jar; do unzip -d "${f%*.src.jar}" "$f"; done


Turns out, using these decompilation steps, you would not end up with the source code for LoginController.java. The source code for this controller actually exists in the SAAS/WEB-INF/classes directory:

Looking at LoginController.java, we can see the following snippet of code:

/* 1596 */       log.info(String.format("Start authenticating %s with embedded auth broker", new Object[] { username }));
/* 1597 */       federationResults = this.federationBrokerService.doEmbeddedAuthBrokerLogin(orgRuntime, httpServletRequest, httpServletResponse, model, embeddedAuthContextInfo
/* 1598 */           .getIdpInfo(), inputParams, userStoreDomain, embeddedAuthContextInfo.getAuthMethodIdsToUse(), skipUserLookup);


We cannot see any HTTP requests being made within the LoginController, so we follow through and check out the code for federationBrokerService.doEmbeddedAuthBrokerLogin. After reading this code, it seems like it delegates authentication to different types of authentication adapters.

This brings our search to LocalPasswordAuthAdapter, which is also hinted at in the VMWare advisory (an issue to do with local authentication). In order to find this JAR file, we triggered a stack trace by providing a host header with an unresolvable host, causing the following logs:

2022-05-28T14:27:30,110 WARN  (Thread-115) [ACCESS;-;192.168.1.38;35a220fd88b6ebd:192.168.1.38;-] com.vmware.horizon.adapters.local.LocalPasswordService - Failed to authenticate user
java.net.UnknownHostException: aaaa: No address associated with hostname


Ahah! local-password-auth-adapter-0.1.jar

Searching the entire file system for this file pointed us directly to where it was located:

access:/ # find . -name 'local-password*'


We compressed all of the authentication adapters and decompiled them locally. But focused our efforts on the LocalPasswordAuthAdapater due to the exceptions in the logs.

During the login flow the LocalPasswordAuthAdapter.login function is called and is responsible for determining if a user is authenticated or not and tracking failed login attempts.

/*    0 */   @Nonnull
/*    0 */   public AuthnAdapterResponse login(@Nonnull String tenantId, @Nonnull Map<String, String> config, @Nullable HttpServletRequest request, @Nullable HttpServletResponse response, @Nullable Map<String, String> inputParameters) throws AuthAdapterConfigException {
/*  143 */     Locale locale = (request == null) ? Locale.getDefault() : request.getLocale();
/*  145 */     this.messages.setLocale(locale);
/*  146 */     if (inputParameters == null)
/*  147 */       inputParameters = new HashMap<>();
/*  151 */     if (null == inputParameters.get("numAttempted")) {
/*  152 */       inputParameters.put("numAttempted", "0");
/*  153 */       return generateResponseForCollectingUserInputFirstTime(inputParameters, locale, authnAdapterResponse);
/*    0 */     }
/*    0 */     try {
/*  157 */       assureRequiredUserInput(inputParameters, locale);
/*  158 */     } catch (AuthAdapterConfigException e) {
/*    0 */     }
/*  166 */     String domain = inputParameters.get("domain");
/*  168 */     String endpoint = getLocalUrl(request); // sour
/*  169 */     String userAgent = request.getHeader("User-Agent");
/*  171 */       if (response != null)
/*  172 */         response.setStatus(200);
/*  174 */       return generateSuccessResponse(username, domain);
/*    0 */     }
/*  177 */     Integer numAttempted = Integer.valueOf(1 + getNumAttempted(inputParameters).intValue());
/*  179 */     Integer allowedNumAttempts = getNumAttemptsAllowed(config);
/*  180 */     inputParameters.put("numAttempted", numAttempted.toString());
/*  181 */     if (numAttempted.intValue() >= allowedNumAttempts.intValue())
/*  182 */       return generateErrorResponse(username, domain);
/*  185 */     ConfigAttribute errorAttribute = createErrorAttribute("error.local.password.authFailed", locale);
/*    0 */   }


The interesting parts of the login function are on lines 168 and 170.

On line 168 we can see where the user controlled input is parsed by the getLocalUrl function which is responsible for constructing a url out of the request.

The getLocalUrl function does this by utilizing the HttpServerletRequest getter methods.

Get Local URL code:

/*    0 */   private String getLocalUrl(HttpServletRequest request) {
/*  194 */     if (null == request)
/*  195 */       return null;
/*    0 */     try {
/*  200 */       return (new URL(SSLConst.HTTPS, request.getServerName(), request.getServerPort(), request.getContextPath() + "/API/1.0/REST/auth/local/login")).toString();
/*  201 */     } catch (MalformedURLException e) {
/*  202 */       log.error("Failed to create URL: " + e.getMessage(), e);
/*  203 */       return null;
/*    0 */     }
/*    0 */   }



The getServerName getter returns the value of the host header which the user controls

getServerName() code:

    /**
* Returns the host name of the server to which the request was sent.
* It is the value of the part before ":" in the <code>Host</code>
* header value, if any, or the resolved server name, or the server IP
*
* @return a <code>String</code> containing the name of the server
*/
public String getServerName();


After the request is constructed we see a call to the getLocalPasswordService function to return the LocalPasswordService

getLocalPasswordService(config) code:

    @VisibleForTesting
}


Due to the user being able to control the host header, the poisoned endpoint variable is then passed into the LocalPasswordService.authenticate function on line 170. Where we finally see our source reach the sink.

The LocalPasswordService.authenticate function then executes the request with the user controlled host header on line 80 and validates if the user is authenticated by checking if the response is a 200 and if so, then the user is successfully authenticated.

LocalPasswordService.authenticate code:

/*    0 */   public Boolean authenticate(@Nonnull String endpoint, @Nonnull String tenantId, @Nonnull String username, @Nonnull String password, @Nullable String domain, @Nonnull String userAgent) {
/*   65 */       return Boolean.valueOf(false);
/*    0 */     }
/*   67 */     LOGGER.debug("Attempting to authenticate user " + username + " via " + endpoint);
/*    0 */     try {
/*   77 */       this.idmXRay.beginHttpSubsegment("SAAS", endpoint, "POST", Optional.of(requestBuilder));
/*   80 */       HttpResponse httpResponse = this.httpClient.execute(requestBuilder.build());
/*   81 */       EntityUtils.consumeQuietly(httpResponse.getEntity());
/*   83 */       int responseCode = httpResponse.getStatusLine().getStatusCode();
/*   84 */       if (200 == responseCode) {
/*   85 */         LOGGER.debug("Successfully authenticated user " + username);
/*   86 */         return Boolean.valueOf(true);
/*    0 */       }
/*   88 */       LOGGER.debug("Failed to authenticate user " + username + ", status code " + responseCode);
/*   90 */     } catch (IOException e) {
/*   91 */       LOGGER.warn("Failed to authenticate user ", e);
/*    0 */     } finally {
/*   94 */       this.idmXRay.endSubsegment();
/*    0 */     }
/*   96 */     return Boolean.valueOf(false);
/*    0 */   }


We see two issues with the behaviour of the LocalPasswordService.authenticate function.

The first issue is that the endpoint which is requested to confirm whether or not the authentication credentials are valid, is controllable by the user via the Host header. This allows a user to input an arbitrary host, which VMWare Workspace One Access will then make a request to.

Being in control of the server these requests are sent to, ultimately allows you to control the flow of the application to bypass authentication.

The second issue is that it is possible to make HTTP requests to internal hosts through the Host header. This is not that impactful as the path of the request is not controllable and redirects are not being followed. At best, you could potentially tell if an internal host is online or offline.

# Bonus Points

In order to set up a debugger, we can modify the file located at /opt/vmware/horizon/workspace/bin/setenv.sh and add the following to the JVM options:

-agentlib:jdwp=transport=dt_socket,server=y,address=5005,suspend=n


Run service horizon-workspace restart, and then setup an iptables rule on the appliance to allow traffic to all ports:

iptables -P INPUT ACCEPT && iptables -P OUTPUT ACCEPT


To prepare for the debugging we compiled a jar file using https://github.com/mogwailabs/jarjarbigs/blob/master/jarjarbigs.py - thank you to frycos’s blog post (https://frycos.github.io/vulns4free/2022/05/24/security-code-audit-fails.html) for this tip.

python3 ~/tools/jarjarbigs/jarjarbigs.py ./webapps/ ./webapps.jar


# Conclusion

When auditing large enterprise applications, it is often possible to miss code paths due to complexity. It is critical to understand if you have all of the application’s source code decompiled and available. The three reasons why someone would have missed this vulnerability seem to be because:

• The code path to the local authentication adapter is complex
• The JAR file for the local authentication adapter is located in a different folder than the tomcat webapps directory + the LoginController is not in the decompiled JARs
• It was not obvious that request.getServerName() returns the value of the Host header (poorly documented)

In this blog post we mapped the sources and sinks, which was a valuable learning experience for us. We’ll use the lessons learned from this exercise when approaching future source code review work.