Umbraco Headless - performance fix

Umbraco headless (currently in béta) looks very promising for a multi-platform approach. No fuss, just content. Under heavy load, however, it - or SOMEthing at least - buckled, so here's how we found and solved the problem. (Until the next version arrives with a definitive fix)

Starting Headless

Having done a couple of projects in Umbraco Cloud, we were itching to try out Umbraco Headless (currently in béta). An opportunity presented itself in a small project - fully React - with a moderate amount of content to manage.

Spinning up a test-instance was fast and easy, but for some reason, you can't add one to your existing collection of projects. I needed to register with a new email. Probably just part of it being béta.

Once created, you get the same Umbraco backend that you are (or should be) used to, except for some options that have been removed.
Under Settings, there's no Templates, no Partial Views, no Scripts, but that makes sense, as your Umbraco is reduced to a big box of content, no longer worrying about presentation.

I quickly created a few document-types, added some content and wrote an API on top of that (hosted in Azure), using the Umbraco.Headless.Client (.Net Framework, in my case). Using the client is rather straightforward, as you're usually just fetching one item or a list of children/descendants.

You could do without this extra API layer in between, but I do not like to have all business logic in the frontend code. Plus, you get all the advantages of Azure API Management.

Under load

System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.Net.WebException: Unable to connect to the remote server
---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

However, when the React app neared completion and we started testing, we experienced intermittent hickups in responsetime. When the system started doing 10-20 API-request per second, each in turn doing multiple requests towards Umbraco, requests started to drop out. Delays up to 42 seconds (and always 42!) suggested something timing out.

Another advantage of the extra API layer : Application Insights traced the problem to a Dependency-call towards Umbraco.

The culprit

My initial suspicion was that it would be a resource issue. It's still in béta, so maybe these test-instances were shared on a server, low on resources at certain times?

Turns out it was NOT the Umbraco backend having issues. No errors whatsoever in the logs. I went as far as transferring my content (a nightmare on it's own, as Content Transfer is not supported in Headless) to a regular Umbraco Cloud project (where we know what to expect, resource-wise) and adding the required dll's and config-changes to fake it being Headless. I immediately faced the same issue.

Digging around (i.e. decompiling) the Umbraco Headless client, I saw it used Refit under the hood.

Refit is Open Source, so debugging was easy.
The default implementation - as used by Umbraco - creates a new HttpClient for every request. In .Net Core this is not a problem, but in .Net framework this is a big no-no. There's enough written about the topic already, so I will not reiterate. (e.g. YOU'RE USING HTTPCLIENT WRONG AND IT IS DESTABILIZING YOUR SOFTWARE).

I was in brief contact with someone from Refit, to be told that this is just the default implementation and Refit has overloads allowing you to provide your own HttpClient as you see fit.

The solution

So, you can provide Refit with your own client. Good in theory, if the Umbraco.Headless.Client was open source. (I fully understand why it is not)

The current version of the Headless-client (0.9.7-CI-20180905-01) has no way of being manipulated. I have received confirmation from Umbraco HQ that the next version will create an instance of the HttpClient, which will have the same lifespan as the service, with an option of adding your own HttpClient instance.

In the meantime, what I did was get a copy of Refit (4.6.30) and changed its default implementation to reuse a static HttpClient every time. Build and deployed the new dll and put it in my API-project. All problems disappeared immediately.

Here's what I added, if you're interested. To be absolutely safe, I implemented multiple static clients (one per host you're calling) instead of just the one. If it were a permanent solution, I would probably fiddle with it some more, but it's only needed until the next release of Umbraco.Headless.Client.

private static readonly Dictionary<string, HttpClient> clients = new Dictionary<string, HttpClient>();

private static HttpClient Client(string hostUrl, HttpMessageHandler innerHandler = null)
{
    HttpClient client;
    if (!clients.TryGetValue(hostUrl, out client))
    {
        client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };
        clients.Add(hostUrl, client);
    }

    return client;
}

And calling this inside public static T For<T>(string hostUrl, RefitSettings settings)

//Default Refit-implementation
//var client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };

//Temporary workaround
var client = Client(hostUrl, innerHandler);