Pages

Wednesday, 10 April 2019

Umbraco - Upload SVG as Image

In Umbraco, you can upload multiple images into the media library by simply dragging and dropping them all together unto a Media Folder.


This works fine for JPG, GIF, PNG and the like, but when you drop an SVG, it is uploaded as a File instead of an Image.

When you subsequently want to use a MediaPicker to select that image in one of your content items, you cannot select the SVG, because only Images are allowed, not Files.

It's perfectly possible to click Create > Image, then drop the SVG on the Upload image pane, but as a developer, I pride myself on my laziness. I will not upload 28 images one by one, if I can solve this once and for all.

"Hidden" setting

The list of file extensions that is recognized as an Image (everything else is a File) is defined in settings/content/imaging/imageFileTypes.

For some reason, however, this setting is not present in config/umbracoSettings.config by default.
When not present, the code falls back to jpeg, jpg, gif, bmp, png, tiff, tif. We just want to add svg to the list.

There are many settings omitted like this. Maybe they do not want to overload the configs with settings that are falling back to default anyway? I've seen the same in other CMS's.

So, all you need to do, is go to config/umbracoSettings.config and add the following :

<?xml version="1.0" encoding="utf-8" ?>
<settings>   

  <content>
    <imaging>
      <!-- what file extension that should cause umbraco to create thumbnails -->
      <imageFileTypes>jpeg,jpg,gif,bmp,png,tiff,tif,svg</imageFileTypes>     
    </imaging>

    ......

  </content>
</settings>

Now, when you drop one or more SVG's on a folder, it will be recognized as an Image.

Monday, 8 April 2019

Umbraco Courier - Azure Storage

When you use Umbraco Courier to transfer items from development to production or vice-versa, you may run into a small problem when you store your media in AzureStorage, as is highly recommended.

The problem

When transferring media items with Courier, the data and the blob (= the actual image) are transferred to the target environment, but at the target, Umbraco is saving the blob to the default location on disk, not checking its FileSystemProviders, telling it to transfer the blob to AzureStorage.

The solution

I made a small modification to Hugo's solution on the Umbraco forum, where he raises the issue, then solves it himself.

You'll need to provide 3 EventHandlers.

The first one will run on the sending environment, telling the target to queue a postprocessor for the image.

public class MediaEventTrigger : ApplicationEventHandler
{
  protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
  {
    ExtractionManagerEvents.ExtractedItemResources += ExtractedItemResources; ;
  }

  private void ExtractedItemResources(object sender, ItemEventArgs e)
  {
    var provider = sender as RevisionExtraction;

    //make sure the media event only fires for media items, not other resources
    if (e.Item.ItemId.ProviderId == ItemProviderIds.mediapropertyDataItemProviderGuid)
    {
      var media = e.Item.Dependencies.Single(i => i.ItemId.ProviderId == ItemProviderIds.mediaItemProviderGuid);
      provider.Context.ExecuteEvent("QueueCourierMediaEvent", e.ItemId, new SerializableDictionary<string, string>());
    }
  }
}

The second one will be running at the receiving end. It will queue the third, instructing it to move the blob to Azure Storage.

public class MediaEventListener : ItemEventProvider
{
  public override string Alias => "QueueCourierMediaEvent";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    ExecutionContext.QueueEvent("UpdateMediaPath", itemId, parameters, EventManagerSystemQueues.DeploymentComplete);
  }
}

And now for the actual moving of the blob.

public class MediaEventQueueListener : ItemEventProvider
{
  public override string Alias => "UpdateMediaPath";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    var fs = FileSystemProviderManager.Current.GetFileSystemProvider<MediaFileSystem>();
    var ms = ApplicationContext.Current.Services.MediaService;

    var media = ms.GetById(Guid.Parse(itemId.Id));
    var oldPath = GetMediaPath(media);
    if (!string.IsNullOrEmpty(oldPath))
    {
      var absolutePath = HttpContext.Current.Server.MapPath(oldPath); //path to file on filesystem
      using (var fileStream = System.IO.File.OpenRead(absolutePath))
      {
        fs.AddFile(oldPath, fileStream); //will be picked up by FileSystemProvider and saved to AzureStorage
      }
     Directory.Delete(Path.GetDirectoryName(absolutePath), true);
     }
  }

  private string GetMediaPath(IMedia media)
  {
    //Image may use ImageCropper (or not), so test presence of Json
    var umbracoFile = media.GetValue<string>(Constants.Conventions.Media.File);
    if (!string.IsNullOrWhiteSpace(umbracoFile))
    {
      if (umbracoFile.StartsWith("{"))
      {
        //the umbracofile property contains the src for the image, which at the beginning is the location of the file on the filesystem.
        var umbracoFileJson = JsonConvert.DeserializeObject<ImageCropDataSet>(umbracoFile);
        return $"~{umbracoFileJson.Src}";
      }
      else
      {
        //Contains the path directly
        return umbracoFile;
      }
    }

    return string.Empty;
  }
}

Shouldn't we rename?

No. Or, not really.

Let's say we are transferring media/1028/flower.jpg from Dev to Prod. The path will remain media/1028/flower.jpg, because that just makes life easy.

You could argue that we need to rename each item's folder with a Guid or something, so it becomes unique, but then you would have to write another handler, parsing and updating all your content, as you'd need to update all media-paths in grid-content referencing your media.

If you always migrate in one direction, this becomes a non-issue, as their will be (almost) no name-collisions on the media folder.

Name-collisions?

Suppose someone manually added tree.jpg to Prod directly.
It is possible that - after transfer from Dev - we now have media/1028/flower.jpg and media/1028/tree.jpg in the same folder on Azure storage. I can live with that.

The only scenario where this becomes a problem is, if you uploaded a different flower.jpg and the blob is overwritten when you transfer the one from Dev.

In my case, this is extremely unlikely, but it's good to at least be aware of this possibility.

Umbraco Courier - Https

This is going to be a short one, but one that took me far too long to figure out.

Install Courier

First off, when you install Courier in your Umbraco, make sure you install the package in each environment. Don't just add the configs and dlls to your source and deploy it. It requires database changes that are only deployed through the package installer.

Configure Courier

I'd suggest giving the documentation a quick run-through. It's rather thorough.

The most important is configuring your repositories (i.e. the environments you want to be able to send data to).

<repository alias="DEV" name="Development" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-dev.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="ACC" name="Acceptance" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-acc.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="PRD" name="Production" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-prd.azurewebsites.net</url>
  <user>0</user>
</repository>

Https not working

Out of the box, post-install, this should just work.

If you're unlucky - like I always am - when trying to use Courier to transfer a few items, you may get

Cannot connect to the destination
"https://xxxx//umbraco/plugins/courier/webservices/repository.asmx"

Take note of the https. There may be other reasons why it's not working for you, but if it works over http and not over https, the solution is simple.

According to Umbraco Support, https is not supported by Courier, but I found that a bit hard to believe. Comparing with other projects - where it WAS working over https - I found the culprit to be the SecurityProtocol.

The only thing you need to do to get this working, is change the default Securityprotocl to TLS1.2 (or higher, if you have this option and are reading this a few years in the future).

Find a startup-class in your project (ContainerInitializer or WebApiConfig or whatever, you'll find something) and add the following line:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

Friday, 1 February 2019

Winning the Google Hackathon!

On January 16th, I participated (with a lot of my colleagues) in the first and only European Google Mobile Speed Hackathon. We spent the day learning how to improve the speed of some of our projects.
To motivate is further, the winning team would win a small prize and eternal worship of our peers.

Long version of this post : https://www.the-reference.com/en/blog/jeroenvantroyen/2019/google-hackathon

Expectations

I'm used to using PageSpeed Insights. Most of our projects already score in the 80 range. So, to be honest, I didn't anticipate huge improvement in score. 

Boy, was I in for a surprise!

Turns out Google recently changed the scoring system and is now expecting a lot more from us.
The tool we were to measure speed with was Lighthouse, built into Chrome DevTools (F12).

Lighthouse immediately makes a bunch of suggestions on how to improve your score. 

Techniques

If you're interested in the techniques we applied to our project, you can read the technical details here : https://www.the-reference.com/en/blog/jeroenvantroyen/2019/google-hackathon

The result?

The resulting score at the end of the day was very satisfying. We now scored 90% on performance.


Putting theory to practice

After all we'd learned, I was itching to put my newfound knowledge to good use, so I revisited my last project (Skyn) and checked it's initial score and suggestions.


Wait, 17%? AND I forgot to check minimization of Javascript AND css?


With an effort of less than 2 hours, the result is below. The site also feels more snappy than it used to.

When there's more time, I plan to split scripts in a critical and a non-critical bundle, then defer the last one. Split CSS in critical inline and loadCSS the rest.

Play with it!

How about you? Let me know if a few simple tricks, a couple of hours well-spent, can improve your pagespeed. 
Good luck!

Tuesday, 27 November 2018

Umbraco Headless - Optional Images

When you use the Umbraco.Headless.Client on top of an Umbraco Headless instance, you are accessing Umbraco through a REST API. This means your content is serialized (to JSON) on the Umbraco side and deserialized inside the Headless-client.

BUT : I ran into a problem when using optional Images.

First steps

Create a very simple Document Type, with 2 properties.


Create a class in your Visual Studio project that maps to this DocumentType.

using Umbraco.Headless.Client.Net.Models;

namespace Models
{
    public class Test : ContentItem, IContentBase
    {
        public string Title { get; set; }
        public string Description { get; set; }
    }
}

Fetch a node that uses this template. The quickest way is to get it via its Id.
Normally, the contentService would be injected, but for simplicity's sake, let's keep it like this.

  var umbracoHeadlessUrl = ConfigurationManager.AppSettings[umbracoHeadless:url];
  var contentService =  new PublishedContentService(umbracoHeadlessUrl);

  var testNode = await contentService.GetById<Test>(id);

The resulting JSON looks like this :

{
    "Title": "Title 1",
    "Description": "<p>A very long description.</p>",
    "id": "1940908b-460c-47dd-825a-010381a57abd",
    "parentId": "7a0e6214-7802-4130-b77c-0965148b0a33",
    "name": "Test",
    "path": null,
    "contentTypeAlias": "test",
    "createDate": "0001-01-01T00:00:00",
    "updateDate": "0001-01-01T00:00:00",
    "url": null,
    "writerName": null,
    "creatorName": null,
    "properties": null,
    "_links": null
}

Adding an Image

Add a property to your template using datatype MediaPicker.
Add a property to your Test-class of type Image. I'm only interested in the Url-property of this image, so that's the only one I created.

public class Image
{
    public string Url { get; set; }
}
public class Test : ContentItem, IContentBase
{
    public string Title { get; set; }
    public string Description { get; set; }
    public Image Image { get; set; }
}

When I leave the Image-property empty in Umbraco and run my changes, I get the following error:

Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Models.Image' 
because the type requires a JSON object (e.g. {\"name\":\"value\"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {\"name\":\"value\"}) or change the deserialized 
type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> 
that can be deserialized from a JSON array.


The reason for this error, is that the JSON for the empty image is

    "image": [],

which would require your property to become a List<Image> instead.
However, if you put something in the Image property, the JSON comes out as

    "image": {
        "writerName": "Jeroen Vantroyen",
        "creatorName": "",
        "writerId": 1,
        "creatorId": 1,
        "urlName": "question",
        "url": "/media/1001/images.png",
        ...
        //truncated for brevity
    },

So, when left empty, it's an (empty) array. When filled in, it's a simple JSON object.
How to fix this?

JsonConvertor to the rescue

The solution is simple and elegant, using a JsonConvertor, which can handle both arrays and single objects.
I do not take credit for the solution below, but will save you the hunt for the code.

    public class SingleOrArrayConverter<T> : JsonConverter
    {
        public override bool CanConvert(Type objectType)
        {
            return (objectType == typeof(List<T>));
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            JToken token = JToken.Load(reader);
            if (token.Type == JTokenType.Array)
            {
                return token.ToObject<List<T>>();
            }
            return new List<T> { token.ToObject<T>() };
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            List<T> list = (List<T>)value;
            if (list.Count == 1)
            {
                value = list[0];
            }
            serializer.Serialize(writer, value);
        }

        public override bool CanWrite
        {
            get { return true; }
        }
    }

To use this, I modify my test-class one last time.

public class Test : ContentItem, IContentBase
{
    public string Title { get; set; }
    public string Description { get; set; }

    [JsonConverter(typeof(SingleOrArrayConverter<Image>))]
    public List<Image> Image { get; set; }
}

To use an instance of this class, I would resort to Image.FirstOrDefault().

Umbraco Headless - MultiLingual with Vorto

You want to fully embrace the advantages offered by Vorto inside your Umbraco solution. Unfortunately, the current Umbraco Headless béta does not include it out of the box (yet!).
Let's install it ourselves.

Vor-who?

New to Umbraco or living under a rock? Vorto is a nice, little module, that transforms your text-properties into multi-lingual text properties.

Yes, Umbraco 8 will make it obsolete as it will have a built-in mechaniscm for multi-language. Since we're not there yet, let's just carry on.

If your past solution to multi-language was to create TWO properties for each and every text-field (one English, one Spanish, for example), it's not only annoying to maintain and keep track, but it's a nightmare when the customer suddenly decides to add a 3rd language, and a 4th ...

To take advantage of Vorto, all you have to do is :
  • Install it. (see below)
  • Add one or more datatypes in the Developer-section (e.g. one for Textstring, one for Richtext editor).
  • Define properties in your document types with this datatype.

When used on a normal TextString it looks like below. Easy tabs to switch language. Checkmarks indicate which languages have a value.

It automatically grabs all defined languages, but you can override this behaviour, set the default language ...  Just go read about it here.



Normal Installation

For on-premise Umbraco or Umbraco Cloud, you could install this as a nuget-package. If you do not have any custom code, you can download the package and install it in the Umbraco backend.

Install-Package Our.Umbraco.Vorto

The installation contains plugin-files and a dll, so no database-changes.
  • App_Plugins/Vorto/css/vorto.css
  • App_Plugins/Vorto/js/jquery.hoverIntent.minified.js
  • App_Plugins/Vorto/js/vorto.js
  • App_Plugins/Vorto/views/vorto.html
  • App_Plugins/Vorto/views/vorto.languagePicker.html
  • App_Plugins/Vorto/views/vorto.languageSourceRadioList.html
  • App_Plugins/Vorto/views/vorto.mandatoryBehaviousPicker.html
  • App_Plugins/Vorto/views/vorto.poropertyEditorPicker.html
  • App_Plugins/Vorto/views/vorto.rtlBehaviourPicker.html
  • bin/Our.Umbraco.Vorto.dll

Umbraco Headless installation

We were as good as assured by Niels Hartvig (founder and CEO @ Umbraco) that Vorto would become part of Umbraco Headless, but it wasn't just yet.

So, if it's so easy to install, why am I writing about it?
Well, Umbraco Headless doesn't provide the option to install packages, nor does it allow you access to the repository.

If this last mention of the repository doesn't mean anything to you, I'd suggest reading up on how Umbraco Cloud handles deployment. In a nutshell : once your Umbraco Cloud installation is done for you, you get access to the git repository. Commit something on it and it gets deployed to your website.

So, how DO you deploy Vorto?

Access the git repository

In a normal Umbraco Cloud project, you can go to https://www.s1.umbraco.io/project/<your-project-name>, right-click an environment and go to "Power tools (Kudu)".



In Umbraco Headless, this option is removed, but you can still manually go to https://<your-project-name>.scm.s1.umbraco.io.

From there, go to Source control info and copy the Git-url.



Clone and commit

Clone the repository locally, add the files required for Vorto (see above) and commit/push them back to the repository.

You will have to restart your environment (again from the project management page) for the changes to be picked up.

And you're good to go.

Tuesday, 6 November 2018

Umbraco Headless - performance fix

Umbraco headless (currently in béta) looks very promising for a multi-platform approach. No fuss, just content. Under heavy load, however, it - or SOMEthing at least - buckled, so here's how we found and solved the problem. (Until the next version arrives with a definitive fix)

Starting Headless

Having done a couple of projects in Umbraco Cloud, we were itching to try out Umbraco Headless (currently in béta). An opportunity presented itself in a small project - fully React - with a moderate amount of content to manage.

Spinning up a test-instance was fast and easy, but for some reason, you can't add one to your existing collection of projects. I needed to register with a new email. Probably just part of it being béta.

Once created, you get the same Umbraco backend that you are (or should be) used to, except for some options that have been removed.
Under Settings, there's no Templates, no Partial Views, no Scripts, but that makes sense, as your Umbraco is reduced to a big box of content, no longer worrying about presentation.



I quickly created a few document-types, added some content and wrote an API on top of that (hosted in Azure), using the Umbraco.Headless.Client (.Net Framework, in my case). Using the client is rather straightforward, as you're usually just fetching one item or a list of children/descendants.

You could do without this extra API layer in between, but I do not like to have all business logic in the frontend code. Plus, you get all the advantages of Azure API Management.

Under load

System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.Net.WebException: Unable to connect to the remote server
---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

However, when the React app neared completion and we started testing, we experienced intermittent hickups in responsetime. When the system started doing 10-20 API-request per second, each in turn doing multiple requests towards Umbraco, requests started to drop out. Delays up to 42 seconds (and always 42!) suggested something timing out.

Another advantage of the extra API layer : Application Insights traced the problem to a Dependency-call towards Umbraco.

The culprit

My initial suspicion was that it would be a resource issue. It's still in béta, so maybe these test-instances were shared on a server, low on resources at certain times?

Turns out it was NOT the Umbraco backend having issues. No errors whatsoever in the logs. I went as far as transferring my content (a nightmare on it's own, as Content Transfer is not supported in Headless) to a regular Umbraco Cloud project (where we know what to expect, resource-wise) and adding the required dll's and config-changes to fake it being Headless. I immediately faced the same issue.

Digging around (i.e. decompiling) the Umbraco Headless client, I saw it used Refit under the hood.

Refit is Open Source, so debugging was easy.
The default implementation - as used by Umbraco - creates a new HttpClient for every request. In .Net Core this is not a problem, but in .Net framework this is a big no-no. There's enough written about the topic already, so I will not reiterate. (e.g. YOU'RE USING HTTPCLIENT WRONG AND IT IS DESTABILIZING YOUR SOFTWARE).

I was in brief contact with someone from Refit, to be told that this is just the default implementation and Refit has overloads allowing you to provide your own HttpClient as you see fit.

The solution

So, you can provide Refit with your own client. Good in theory, if the Umbraco.Headless.Client was open source. (I fully understand why it is not)

The current version of the Headless-client (0.9.7-CI-20180905-01) has no way of being manipulated. I have received confirmation from Umbraco HQ that the next version will create an instance of the HttpClient, which will have the same lifespan as the service, with an option of adding your own HttpClient instance.

In the meantime, what I did was get a copy of Refit (4.6.30) and changed its default implementation to reuse a static HttpClient every time. Build and deployed the new dll and put it in my API-project. All problems disappeared immediately.

Here's what I added, if you're interested. To be absolutely safe, I implemented multiple static clients (one per host you're calling) instead of just the one. If it were a permanent solution, I would probably fiddle with it some more, but it's only needed until the next release of Umbraco.Headless.Client.

private static readonly Dictionary<string, HttpClient> clients = new Dictionary<string, HttpClient>();

private static HttpClient Client(string hostUrl, HttpMessageHandler innerHandler = null)
{
    HttpClient client;
    if (!clients.TryGetValue(hostUrl, out client))
    {
        client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };
        clients.Add(hostUrl, client);
    }

    return client;
}

And calling this inside public static T For<T>(string hostUrl, RefitSettings settings)

//Default Refit-implementation
//var client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };

//Temporary workaround
var client = Client(hostUrl, innerHandler);