Pages

Tuesday, 21 April 2020

Easy tool for Pagespeed optimization

Last october, I talked at Duugfest about Google Pagespeed and how to optimize your page for scoring high ...  having a fast, mobile-friendly website.

I even blogged about it in februari, after winning the Google Hackathon and in more technical detail  here.

One of the things I mentioned at Duugfest was that I had made a "tool" (an MVC ActionFilterAttribute) that would post-process all rendered HTML and "do something" with all the images.
Hang tight, I'll get less vague in a minute.

More than one of you asked if I could share the code for the filter, and it took me until now to get around to that. So, I created a nuget package with my ActionFilter and called it Our.Umbraco.PageSpeed. The intention is to add more tricks as I devise them along the way.

The reason for this one-filter-catches-all approach is 2-fold.

1. Lazy-loading

Google Pagespeed Insights recommends lazy loading your images. It's very easy to use LaySizes for it. So easy, in fact, that I won't repeat anything you can easily read in their README.

Now, what you'll have to do, is go through all your code (views, partial views, ...) and tweak your images a bit.
Basically, your images look like this:

<img data-src="image.jpg" class="lazyload" />

There are reasons NOT to do this however:

  • it's boring, repetitive work, prone to errors.
  • you may miss an image here and there. Nobody will notice, until you run Pagespeed insights.
  • you don't control ALL places where images are rendered. E.g. a content editor can put an image inside a richtext-field.

A filter will just parse the HTML and replace all images. None are missed, no typo's are made.

2. Serve images in better formats

Google suggest serving images in WEBP, as it is a better format than jpg, png, ...
Not all browsers support it, however, so you may want to resort to using <picture> instead of <img> and provide multiple images, out of which your browser can choose, depending on it's capabilities.

This would mean something along the lines of:

<picture>
  <source type="image/webp" src="webp-version of image.jpg" />
  <source data-src="image.jpg" class="lazyload" />
</picture>
If only there was a way of converting images to webp without requiring our content editors to manage multiple versions of each image.

But wait, there IS : ImageProcessor is baked in in Umbraco and has a plugin for Webp.


Combining it all 

Now, if we combine the possibility of

  • replacing all our img-tags on-the-fly by source-tags,
  • providing a way to convert to Webp,
  • lazy loading the result.
we get the following:

<picture>
  <source type="image/webp" data-src="image.jpg?format=webp&quality=70" class="lazyload" />
  <source data-src="image.jpg" class="lazyload" />
</picture>

How to achieve this?

  1. Install LazySizes in your project as per their instructions.
  2. Install nuget ImageProcessor.Plugins.WebP
  3. Install nuget Our.Umbraco.PageSpeed
  4. Decorate your controller with the LazyLoadFilter-attribute
[LazyLoadFilter]
public class MyRenderMvcController : RenderMvcController
{
    public override ActionResult Index(RenderModel model)
    {
        //Do stuff here. Or not, see if I care.
        return base.Index(model);
    }
}

And for all those DocTypes that you didn't write a Controller for. How do you make sure they get their fair share of the fun?

For these, you can set the default Controller to your MyRenderMvcController, and it will also benefit from the LazyLoadFilter-attribute.

DefaultRenderMvcControllerResolver.Current.SetDefaultControllerType(typeof(MyRenderMvcController));

If you want to give this a go and it doesn't work for you for any reason, give me a shout and we'll figure something out.

Wednesday, 15 April 2020

Umbraco 8 - Language fallback for Grid properties

In my previous post (which was hanging around in draft for months until I finally got around to finishing it) I mentioned that my way-of-working with Language Fallback does not work with Grid properties.

Grids are never really empty

The reason for this is that when you use property.Value("propertyAlias", fallback: Fallback.ToLanguage), the only scenario where fallback occurs, is when the value of that property is empty.
When you make a new language version of an item and you do not put anything in the grid, it still is not empty, because what constitutes as an "empty" grid, actually looks like this.

{
  "name": "1 column layout",
  "sections": [
    {
      "grid": "12",
      "rows": []
    }
  ]
}

A second problem I encountered, is that the HtmlHelper-extension that is used to render your grid in a view only accepts a propertyAlias and under the hood it goes straight for the property in question, not through .Value(), preventing you from adding a similar workaround as in my previous article.

Fortunately Umbraco is Open Source, and the code for GetGridHtml is easily located.

public static MvcHtmlString GetGridHtml(this HtmlHelper html, IPublishedContent contentItem, string propertyAlias, string framework)
{
    if (string.IsNullOrWhiteSpace(propertyAlias)) throw new ArgumentNullOrEmptyException(nameof(propertyAlias));

    var view = "Grid/" + framework;
    var prop = contentItem.GetProperty(propertyAlias);
    if (prop == null) throw new NullReferenceException("No property type found with alias " + propertyAlias);
    var model = prop.GetValue();

    var asString = model as string;
    if (asString != null && string.IsNullOrEmpty(asString)) return new MvcHtmlString(string.Empty);

    return html.Partial(view, model);
}

A new way of rendering

So, all I had to do was create my own class with my own extension method.

Now what happens is:
  • I get the value of the grid-property. With language fallback indicated, but that never happens, because a grid is never empty.
  • If the grid is functionally "empty", I get the English value of the same grid-property.
  • Normal processing resumes.

In my views, where - until now - I used the standard way of rendering grids, I had to add a using statement and change the name to my own extension method.

@using MyCode.Umbraco.Web.Extensions;
@inherits UmbracoViewPage<ContentPage>
@{
    Layout = "BasePage.cshtml";
}
<div>
    @Html.GetFallbackGridHtml(Model, ContentPage.GetModelPropertyType(c => c.Body).Alias, "site")
</div>

For those interested, here's the entire class. No rocket science, but it made my job just a little easier again.

using System.Linq;
using System.Web.Mvc;
using System.Web.Mvc.Html;
using Newtonsoft.Json.Linq;
using Umbraco.Core.Models.PublishedContent;
using Umbraco.Web;
using Umbraco.Web.Composing;

namespace MyCode.Umbraco.Web.Extensions
{
    public static class GridExtensions
    {
        public static bool IsGridEmpty(object gridContent)
        {
            var asJToken = gridContent as JToken;
            if (asJToken != null)
            {
                //Check all sections
                var sections = asJToken["sections"] as JArray;
                if (sections != null && sections.Any())
                {
                    //If any section has any row --> not empty
                    foreach (var section in sections)
                    {
                        var rows = section["rows"] as JArray;
                        if (rows != null && rows.Any())
                        {
                            return false;
                        }
                    }
                }
            }

            return true;
        }

        public static MvcHtmlString GetFallbackGridHtml(this HtmlHelper html, IPublishedContent content, string propertyAlias, string framework = "bootstrap3")
        {
            var gridContent = content.Value(propertyAlias, fallback: Fallback.ToLanguage);

            //If gridcontent is "empty", do a fallback to defaultLanguage
            if (IsGridEmpty(gridContent))
            {
                var defaultLanguage = Current.Services.LocalizationService.GetDefaultLanguageIsoCode();
                gridContent = content.Value(propertyAlias, defaultLanguage);
            }

            return html.GetFallbackGridHtml(gridContent, framework);
        }

        public static MvcHtmlString GetFallbackGridHtml(this HtmlHelper html, object gridContent, string framework = "bootstrap3")
        {
            if (gridContent == null)
            {
                return new MvcHtmlString(string.Empty);
            }

            var view = "Grid/" + framework;
            return html.Partial(view, gridContent);
        }
    }
}

Umbraco 8 - ModelsBuilder with Language Fallback


In Umbraco 8, they introduced the much-anticipated multi-language feature called Variants.
I won't describe this in detail, just check the documentation on Variants.

Edit: the approach below is superseded by changing the default fallback behaviour of Umbraco as described here.

Language fallback

In concert with Variants, we can now have language fallback. This basically means that when you have an English and a Dutch version of a page, you do not HAVE to fill in every field on the Dutch version. When left empty it can fall back to the English version (assuming English is marked as Fallback Language in your language settings).


Now, fallback doesn't just magically work, it involves some work on your part.

Assume that message is a contentItem (derived from IPublishedContent) and we want to get the value of the title property.

The traditional way is to get it by calling the Value() method.
var title = message.Value<string>("title");

This will give you the value of the title in the current culture. If this is empty in Dutch, you will get an empty string.

Enable language fallback

To enable language fallback, you would add a fallback parameter.
var title = message.Value<string>("title", fallback: Fallback.ToLanguage);

Verify VariationContext

If - for some reason - the current culture in your context is not set or plain wrong, you can give the culture as a parameter to the Value() method, but this is not the way to go.

I use a lot of API's (derived from Umbraco.Web.WebApi.UmbracoApiController) and they usually don't set the current Umbraco culture.
Given a parameter language containing the culture you need (coming for instance from a request header), you can set the correct culture for Variant fallback as follows:

Current.UmbracoContext.VariationContextAccessor.VariationContext = new VariationContext(language);

You now get the title in the correct language or - when empty - the fallback langauge.

ModelsBuilder

I hear you thinking : "Dude, we've been using ModelsBuilder to have strongly-typed content models for years. Do you really expect us to go back to .Value("propertyName")?"

No, I do not. But it requires some more work.

I won't go into the details of Extending the Modelsbuilder, but here's the gist of it.
The generated class (simplified for brevity) for our Message model would be
[PublishedModel("message")]
public partial class Message : PublishedContentModel
{
    [ImplementPropertyType("title")]
    public string Title => this.Value<string>("title");

    [ImplementPropertyType("someProperty")]
    public string SomeProperty => this.Value<string>("someProperty");
}

We cannot modify this class, because it is generated by the ModelsBuilder. But because it is a partial class, we can extend it.

Below is the custom class you add to your project (same name and namespace as the generated one).
public partial class Message
{
    [ImplementPropertyType("title")]
    public string Title => this.Value<string>("title", fallback: Fallback.ToLanguage);
}

You'll need to remove the implementation for title from the generated class. The next time the generator runs, it will skip the title property because of the ImplementPropertyType-attribute in your custom class.

It is now possible for you to use the following code to get the title (with fallback) from the message item.
var title = message.Title;

Random thoughts

  • The above approach does not work for Grid content, but that's subject for a different post.
  • When creating a new language version of an item in Umbraco, the name of the item is a mandatory field. It would be nice if this could be left empty and also fallback to the English version.
  • If you make a field mandatory, it is mandatory in all languages, so you can't leave it empty and depend on fallback.
    Personally, I made many mandatory fields optional again, because I feel the advantage of fallback outways the need for mandatory fields. (your call)
Feedback or questions? Give me a shout.

Wednesday, 10 April 2019

Umbraco - Upload SVG as Image

In Umbraco, you can upload multiple images into the media library by simply dragging and dropping them all together unto a Media Folder.


This works fine for JPG, GIF, PNG and the like, but when you drop an SVG, it is uploaded as a File instead of an Image.

When you subsequently want to use a MediaPicker to select that image in one of your content items, you cannot select the SVG, because only Images are allowed, not Files.

It's perfectly possible to click Create > Image, then drop the SVG on the Upload image pane, but as a developer, I pride myself on my laziness. I will not upload 28 images one by one, if I can solve this once and for all.

"Hidden" setting

The list of file extensions that is recognized as an Image (everything else is a File) is defined in settings/content/imaging/imageFileTypes.

For some reason, however, this setting is not present in config/umbracoSettings.config by default.
When not present, the code falls back to jpeg, jpg, gif, bmp, png, tiff, tif. We just want to add svg to the list.

There are many settings omitted like this. Maybe they do not want to overload the configs with settings that are falling back to default anyway? I've seen the same in other CMS's.

So, all you need to do, is go to config/umbracoSettings.config and add the following :

<?xml version="1.0" encoding="utf-8" ?>
<settings>   

  <content>
    <imaging>
      <!-- what file extension that should cause umbraco to create thumbnails -->
      <imageFileTypes>jpeg,jpg,gif,bmp,png,tiff,tif,svg</imageFileTypes>     
    </imaging>

    ......

  </content>
</settings>

Now, when you drop one or more SVG's on a folder, it will be recognized as an Image.

Monday, 8 April 2019

Umbraco Courier - Azure Storage

When you use Umbraco Courier to transfer items from development to production or vice-versa, you may run into a small problem when you store your media in AzureStorage, as is highly recommended.

The problem

When transferring media items with Courier, the data and the blob (= the actual image) are transferred to the target environment, but at the target, Umbraco is saving the blob to the default location on disk, not checking its FileSystemProviders, telling it to transfer the blob to AzureStorage.

The solution

I made a small modification to Hugo's solution on the Umbraco forum, where he raises the issue, then solves it himself.

You'll need to provide 3 EventHandlers.

The first one will run on the sending environment, telling the target to queue a postprocessor for the image.

public class MediaEventTrigger : ApplicationEventHandler
{
  protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
  {
    ExtractionManagerEvents.ExtractedItemResources += ExtractedItemResources; ;
  }

  private void ExtractedItemResources(object sender, ItemEventArgs e)
  {
    var provider = sender as RevisionExtraction;

    //make sure the media event only fires for media items, not other resources
    if (e.Item.ItemId.ProviderId == ItemProviderIds.mediapropertyDataItemProviderGuid)
    {
      var media = e.Item.Dependencies.Single(i => i.ItemId.ProviderId == ItemProviderIds.mediaItemProviderGuid);
      provider.Context.ExecuteEvent("QueueCourierMediaEvent", e.ItemId, new SerializableDictionary<string, string>());
    }
  }
}

The second one will be running at the receiving end. It will queue the third, instructing it to move the blob to Azure Storage.

public class MediaEventListener : ItemEventProvider
{
  public override string Alias => "QueueCourierMediaEvent";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    ExecutionContext.QueueEvent("UpdateMediaPath", itemId, parameters, EventManagerSystemQueues.DeploymentComplete);
  }
}

And now for the actual moving of the blob.

public class MediaEventQueueListener : ItemEventProvider
{
  public override string Alias => "UpdateMediaPath";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    var fs = FileSystemProviderManager.Current.GetFileSystemProvider<MediaFileSystem>();
    var ms = ApplicationContext.Current.Services.MediaService;

    var media = ms.GetById(Guid.Parse(itemId.Id));
    var oldPath = GetMediaPath(media);
    if (!string.IsNullOrEmpty(oldPath))
    {
      var absolutePath = HttpContext.Current.Server.MapPath(oldPath); //path to file on filesystem
      using (var fileStream = System.IO.File.OpenRead(absolutePath))
      {
        fs.AddFile(oldPath, fileStream); //will be picked up by FileSystemProvider and saved to AzureStorage
      }
     Directory.Delete(Path.GetDirectoryName(absolutePath), true);
     }
  }

  private string GetMediaPath(IMedia media)
  {
    //Image may use ImageCropper (or not), so test presence of Json
    var umbracoFile = media.GetValue<string>(Constants.Conventions.Media.File);
    if (!string.IsNullOrWhiteSpace(umbracoFile))
    {
      if (umbracoFile.StartsWith("{"))
      {
        //the umbracofile property contains the src for the image, which at the beginning is the location of the file on the filesystem.
        var umbracoFileJson = JsonConvert.DeserializeObject<ImageCropDataSet>(umbracoFile);
        return $"~{umbracoFileJson.Src}";
      }
      else
      {
        //Contains the path directly
        return umbracoFile;
      }
    }

    return string.Empty;
  }
}

Shouldn't we rename?

No. Or, not really.

Let's say we are transferring media/1028/flower.jpg from Dev to Prod. The path will remain media/1028/flower.jpg, because that just makes life easy.

You could argue that we need to rename each item's folder with a Guid or something, so it becomes unique, but then you would have to write another handler, parsing and updating all your content, as you'd need to update all media-paths in grid-content referencing your media.

If you always migrate in one direction, this becomes a non-issue, as their will be (almost) no name-collisions on the media folder.

Name-collisions?

Suppose someone manually added tree.jpg to Prod directly.
It is possible that - after transfer from Dev - we now have media/1028/flower.jpg and media/1028/tree.jpg in the same folder on Azure storage. I can live with that.

The only scenario where this becomes a problem is, if you uploaded a different flower.jpg and the blob is overwritten when you transfer the one from Dev.

In my case, this is extremely unlikely, but it's good to at least be aware of this possibility.

Umbraco Courier - Https

This is going to be a short one, but one that took me far too long to figure out.

Install Courier

First off, when you install Courier in your Umbraco, make sure you install the package in each environment. Don't just add the configs and dlls to your source and deploy it. It requires database changes that are only deployed through the package installer.

Configure Courier

I'd suggest giving the documentation a quick run-through. It's rather thorough.

The most important is configuring your repositories (i.e. the environments you want to be able to send data to).

<repository alias="DEV" name="Development" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-dev.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="ACC" name="Acceptance" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-acc.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="PRD" name="Production" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-prd.azurewebsites.net</url>
  <user>0</user>
</repository>

Https not working

Out of the box, post-install, this should just work.

If you're unlucky - like I always am - when trying to use Courier to transfer a few items, you may get

Cannot connect to the destination
"https://xxxx//umbraco/plugins/courier/webservices/repository.asmx"

Take note of the https. There may be other reasons why it's not working for you, but if it works over http and not over https, the solution is simple.

According to Umbraco Support, https is not supported by Courier, but I found that a bit hard to believe. Comparing with other projects - where it WAS working over https - I found the culprit to be the SecurityProtocol.

The only thing you need to do to get this working, is change the default Securityprotocl to TLS1.2 (or higher, if you have this option and are reading this a few years in the future).

Find a startup-class in your project (ContainerInitializer or WebApiConfig or whatever, you'll find something) and add the following line:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

Friday, 1 February 2019

Winning the Google Hackathon!

On January 16th, I participated (with a lot of my colleagues) in the first and only European Google Mobile Speed Hackathon. We spent the day learning how to improve the speed of some of our projects.
To motivate is further, the winning team would win a small prize and eternal worship of our peers.

Long version of this post : https://www.the-reference.com/en/blog/jeroenvantroyen/2019/google-hackathon

Expectations

I'm used to using PageSpeed Insights. Most of our projects already score in the 80 range. So, to be honest, I didn't anticipate huge improvement in score. 

Boy, was I in for a surprise!

Turns out Google recently changed the scoring system and is now expecting a lot more from us.
The tool we were to measure speed with was Lighthouse, built into Chrome DevTools (F12).

Lighthouse immediately makes a bunch of suggestions on how to improve your score. 

Techniques

If you're interested in the techniques we applied to our project, you can read the technical details here : https://www.the-reference.com/en/blog/jeroenvantroyen/2019/google-hackathon

The result?

The resulting score at the end of the day was very satisfying. We now scored 90% on performance.


Putting theory to practice

After all we'd learned, I was itching to put my newfound knowledge to good use, so I revisited my last project (Skyn) and checked it's initial score and suggestions.


Wait, 17%? AND I forgot to check minimization of Javascript AND css?


With an effort of less than 2 hours, the result is below. The site also feels more snappy than it used to.

When there's more time, I plan to split scripts in a critical and a non-critical bundle, then defer the last one. Split CSS in critical inline and loadCSS the rest.

Play with it!

How about you? Let me know if a few simple tricks, a couple of hours well-spent, can improve your pagespeed. 
Good luck!