Pages

Showing posts with label Umbraco. Show all posts
Showing posts with label Umbraco. Show all posts

Friday, 16 July 2021

Umbraco - Copy Grid content

In Umbraco 7, we had Vorto to handle multi-lingual fields.

In Umbraco 8, Variants were introduced, as well as the split screen editor where you can have 2 languages side by side, for easier translation.

My problem?

Normal text fields can be easily copied left to right by selecting them and copy/pasting the content.
What's been bugging me since the very beginning, is that it's a real hassle to copy an entire grid to a new language.

As a Content Editor in charge of translating this content, I am required to rebuild the entire grid-structure. To translate the grid in the screenshot above, I would have to :

  • add an XL row
  • add a headline component
    • copy/paste the text (or translate it on the fly)
  • add a richtext component
    • copy/paste the text (or translate it on the fly)
  • add an M-M row
  • add an image component
    • figure out which image they used, if I want to use the same one
  • add a richtext component
    • copy/paste the text (or translate it on the fly)
  • ... and so on
This gets tedious very fast. What I really want to be able to do is copy the entire grid structure from left to right and then just translate the portions of text that I need to change.

This feature was requested in October 2019. It became up-for-grabs February 2020 (meaning, anyone who felt like it, could do an attempt at implementing it and making a pull request).
April 2021 someone picked it up and started working on it. The pull request is still pending further improvements.

Disclaimer(s) about my "solution"

  • It was tested on Umbraco 8.13. It should work on older versions and will probably remain working on future versions (minor tweaks could be needed, but are unlikely since it's so isolated).
  • When upgrading to a newer version, you'll probably need to redo the modifications.
  • It is a bit hacky (modifying files in Umbraco), but works fine for all the projects I'm working on.
  • It does not take into account that custom propertyeditors might subscribe to certain events emitted by the grid when e.g. adding a new row or control.

My solution

My quick and dirty solution to have the option of copy/pasting the entire structure of the grid from left to right (or even from item A to item B), consists of adding custom code to 2 files in the Umbraco codebase.

What will happen is:
  • A button "Copy Grid" is displayed next to the "Reorder" button. When pressed, the JSON describing the entire grid is copied to localStorage.
  • A button "Paste Grid" is displayed when no rows have yet been added to the grid and will simply grab the JSON from localStorage again. 

The good news is that these files are .js and .html, so there's no compilation step necessary, making this change something a 5 year old (with advanced computer skills) could do.

\Umbraco\js\umbraco.controllers.js

We have to place are code somewhere, so find the definition of the method called "toggleSortMode" (around line 19.000) and insert the following 2 function-definitions above it.
// Start Grid Copy/Paste
var gridCopyAlias = "grid-value-copy";

$scope.copyGridModel = function copyGridModel() {
    localStorageService.set(gridCopyAlias, $scope.model.value);

    var msg = 'Grid successfully copied to clipboard.';
    notificationsService.success(msg);		
}

$scope.pasteGridModel = function pasteGridModel() {
    try {
        var modelValue = localStorageService.get(gridCopyAlias);
        if (modelValue) {
            $scope.model.value = modelValue;							
        } else {
            var msg = 'Couldn\'t find a copied Umbraco Grid. Make sure to copy an existing Grid first.';
            notificationsService.error(msg);    
        }
    } catch (err) {
        var msg = 'Something went wrong pasting the Grid.';
        notificationsService.error(msg, err);
    }
}
// End Grid Copy/Paste

Then, scroll up a bit (~250 lines) to find the definition of Angular controller "Umbraco.PropertyEditors.GridController".
'use strict';
angular.module('umbraco').controller('Umbraco.PropertyEditors.GridController', function ($scope, localizationService, gridService, umbRequestHelper, angularHelper, $element, eventsService, editorService, overlayService, $interpolate) {

and inject 2 additional services at the end:
  • notificationsService : to be able to add warnings, errors in the backoffice
  • localStorageService : something to serve as a clipboard while copy/pasting
It should now look like 
'use strict';
angular.module('umbraco').controller('Umbraco.PropertyEditors.GridController', function ($scope, localizationService, gridService, umbRequestHelper, angularHelper, $element, eventsService, editorService, overlayService, $interpolate, notificationsService, localStorageService) {

\Umbraco\Views\propertyeditors\grid\grid.html

All the way at the top, you will find the definition for the ReOrder button.
    <umb-editor-sub-header appearance="white" ng-if="showReorderButton()">
        <umb-editor-sub-header-content-right>
            <umb-button action="toggleSortMode()" button-style="link" icon="icon-navigation" label-key="{{reorderKey}}" type="button">
            </umb-button>
        </umb-editor-sub-header-content-right>
    </umb-editor-sub-header>

We will add 2 of our own buttons here, to call the methods we just added to the controller.
Just replace the entire section with the code below
    <umb-editor-sub-header appearance="white" ng-if="!showReorderButton()">
        <umb-editor-sub-header-content-right>
            <umb-button action="pasteGridModel()" button-style="link" icon="icon-arrow-down" label="Paste Grid" type="button">
            </umb-button>
        </umb-editor-sub-header-content-right>
    </umb-editor-sub-header>

    <umb-editor-sub-header appearance="white" ng-if="showReorderButton()">
        <umb-editor-sub-header-content-right>
            <umb-button action="copyGridModel()" button-style="link" icon="icon-arrow-up" label="Copy Grid" type="button">
            </umb-button>

            <umb-button action="toggleSortMode()" button-style="link" icon="icon-navigation" label-key="{{reorderKey}}" type="button">
            </umb-button>
        </umb-editor-sub-header-content-right>
    </umb-editor-sub-header>

Make sure to put these modified files in your solution, or they will be overwritten the next time you do a deploy.

Wednesday, 1 July 2020

Umbraco - Enabling Language Fallback by default

Sometimes you need a fresh pair of eyes to see an obvious solution when you're stuck looking in the wrong direction.

(Skip the backstory to the technical implementation if you prefer technical stuff over the story)

Backstory

Earlier, I talked about using Modelsbuilder partial classes to enable Language Fallback on all variant properties, so your View Designers would have an easier time of using fallback without even realizing it.

The reason I want this is:

I want (Razor) View Designers to be able to use

Model.Title

and not

Model.Value<string>("title", fallback: Fallback.ToLanguage)

I want them to
  • not need to know HOW to get fallback on a poperty
  • not need to know WHEN to use it (when a property is variant)
  • not have an excuse to FORGET to use it
I ALSO don't want to use my previously discussed approach where I need to implemented each Variant property in a partial class.

I wanted to find a better solution. I wanted Modelsbuilder to generate the .Value() method using Language Fallback by default.
I went as far as implementing it and making a Pull Request for ModelsBuilder, based on an AppSetting to enable this behaviour.

The PR was basically refused and a long discussion ensued wether or not what I wanted had merit and my solution was the correct approach.

Until one day, Ronald Barendse commented (paraphrased) "Why don't you just override the default Umbraco behaviour?"

My eyes opened and I went to work (and completed it unexpectedly fast).

Implementation

The default behaviour resides in the PublishedValueFallback class. To enable a solution just for this situation, they made the TryGetValue method virtual, so you can override it with whatever you need.

using System.Linq;
using Umbraco.Core.Models.PublishedContent;
using Umbraco.Core.Services;
using Umbraco.Web.Models.PublishedContent;

namespace Dpw.Eworld2.Foundation.Umbraco.PublishedContent

{
    public class CustomPublishedValueFallback : PublishedValueFallback
    {
        public CustomPublishedValueFallback(ServiceContext serviceContext, IVariationContextAccessor variationContextAccessor)
            : base(serviceContext, variationContextAccessor)
        {
            
        }

        public override bool TryGetValue<T>(IPublishedContent content, string alias, string culture, string segment, Fallback fallback, T defaultValue, out T value, out IPublishedProperty noValueProperty)
        {
            //When no fallback, use ToLanguage by default
            if (!fallback.Any(f => f == Fallback.DefaultValue || f == Fallback.Language || f == Fallback.Ancestors))
            {
                fallback = Fallback.ToLanguage;
            }

            return base.TryGetValue(content, alias, culture, segment, fallback, defaultValue, out value, out noValueProperty);
        }
    }
}

Bascially, what I do is : when no fallback of any sort is requested, I put fallback to Language Fallback. Nothing else needs to change.

To enable this custom class instead of the default one, you register it in your Startup class as follows:

composition.RegisterUnique<IPublishedValueFallback, CustomPublishedValueFallback>();

I was then able to toss my PR, my custom built ModelsBuilder-version, all remaining partial classes and never need to even think about which property will be Variant.

Thursday, 11 June 2020

Remote debug Umbraco on Azure Web App

The title is a bit misleading, because you can debug ANY Azure Web App, not just when running Umbraco, but I had to keep it a bit Umbraco related.

When everything works

If everything was always working, I wouldn't be writing about it today. There's plenty of guides to show you how to do remote debugging, when everything is fine.

In my experience - more often than not - there's always some crucial little step, preventing you from quickly attaching a debugger and finding the exact cause of that elusive error that you just can't reproduce on your local environment.
Remote debugging has saved me HOURS of trying to get to that one error that only one of your users is experiencing in production.

Under normal circumstances, you would
  • publish your Web App from Visual Studio in debug mode. 
  • go to your Cloud Explorer, scroll down to your web app
  • click Attach debugger (this will activate remote debugging if not already active)
 

What can possible go wrong?

Well, you could:
  • not be able to publish for some reason
  • have forgotten to put it in Debug mode
  • not see your Web App in Cloud Explorer, even though your seeing everything else and you DO have access in the Azure Portal and IT already verified that everything is working
    (clearly, I'm speaking hypothetically, because this never happens)

Do it all yourself

Enable remote debugger

First, you'll need to enable remote debugging on your Web App.
  • Go to your Web App in the Azure Portal.
  • Go to Configuration > General Settings
  • Enable Remote Debugging and choose your version of Visual Studio

Download the PublishProfile

The PublishProfile contains all endpoints and credentials you'll need for accessing this Web App.
You can download it from the Overview blade.


The file looks like this.
Obviously, I've fudged it a bit, so you wouldn't get any ideas of trying to mess with my app.
<publishData>
	<publishProfile profileName="[yourApp] - Web Deploy" publishMethod="MSDeploy" publishUrl="[yourApp].scm.azurewebsites.net:443" msdeploySite="[yourApp]" userName="$[yourApp]" userPWD="[yourAppPassword]" destinationAppUrl="http://[yourApp].azurewebsites.net" SQLServerDBConnectionString="" mySQLDBConnectionString="" hostingProviderForumLink="" controlPanelLink="http://windows.azure.com" webSystem="WebSites">
		<databases />
	</publishProfile>
	<publishProfile profileName="[yourApp] - FTP" publishMethod="FTP" publishUrl="ftp://[yourAppFTP].ftp.azurewebsites.windows.net/site/wwwroot" ftpPassiveMode="True" userName="[yourApp]\$[yourApp]" userPWD="[yourAppPassword]" destinationAppUrl="http://[yourApp].azurewebsites.net" SQLServerDBConnectionString="" mySQLDBConnectionString="" hostingProviderForumLink="" controlPanelLink="http://windows.azure.com" webSystem="WebSites">
		<databases />
	</publishProfile>
	<publishProfile profileName="[yourApp] - ReadOnly - FTP" publishMethod="FTP" publishUrl="ftp://[yourAppFTP]dr.ftp.azurewebsites.windows.net/site/wwwroot" ftpPassiveMode="True" userName="[yourApp]\$[yourApp]" userPWD="[yourAppPassword]" destinationAppUrl="http://[yourApp].azurewebsites.net" SQLServerDBConnectionString="" mySQLDBConnectionString="" hostingProviderForumLink="" controlPanelLink="http://windows.azure.com" webSystem="WebSites">
		<databases />
	</publishProfile>
</publishData>
The interesting bits are 
  • url - this should be "[yourApp].azurewebsites.net" (make sure there's no ".scm." part)
  • userName - make sure to take the fully qualified one : "[yourApp]\$[yourApp]"
  • userPWD

Attach your debugger

  • In Visual Studio, go to Debug > Attach to process (CTRL-ALT-P)
  • In the Connection target, paste your Url plus port 4022 (for VS2017, port 4024 for VS2019)
  • In the popup-window, paste username and password 
If your credentials are correct and there is no company firewall blocking your outgoing request, you should be able to see all running processes on the remote machine.
We want to attach to the w3wp.exe. If running a .Net Core application, this will be dotnet.exe.


And that's it, you are now remote debugging Umbraco (or whatever you're working on) and can put breakpoints where you think the problems occur.

Troubleshooting

Now, there are numerous things in the preceding steps that can go wrong and possibly many more that I haven't even encountered yet myself.

Cannot attach debugger

You get an error after entering the url of the Web App, or the popup asking for credentials does not appear.

It's most likely a connectivity issue.
  • Check your corporate firewall.
  • Check if it works from home (while not on any VPN), or tether via your mobile.
  • Check your Web App for IP filtering.

Cannot activate breakpoints

The version of your code doesn't match the version running on your Web App.
  • Try republishing the latest version.
  • Try finding the source branch that matches this release.
If the problem you're investigating is in a specific class library, try building a new version of that project only and push that .dll and it's accompanying .pdb via Kudu or FTP (you have the ftp-credentials in the PublishProfile you downloaded earlier).

Make a backup of the existing .dll, so you can quickly revert.

Thursday, 4 June 2020

DocTypeGridEditor - Remove default DocType option

As I mentioned in my previous post, I'd be checking to get rid of the default "Doc Type" option in the popup below.
It allows your editors to choose any DocType (marked as "Element") and we can think of reasons why you wouldn't want this option to be available to them.


Even if you didn't mention it in your grid.editors.config.js, the "Doc Type" option is always there.

Turns out it's defined in the manifest-file (App_plugins\DocTypeGridEditor\package.manifest). If you make that gridEditors-property into an empty array, the option will no longer be there.

{
  "gridEditors": [],
  "javascript": [
    "~/App_Plugins/DocTypeGridEditor/Js/doctypegrideditor.resources.js",
    "~/App_Plugins/DocTypeGridEditor/Js/doctypegrideditor.services.js",
    "~/App_Plugins/DocTypeGridEditor/Js/doctypegrideditor.controllers.js",
    "~/App_Plugins/DocTypeGridEditor/Js/doctypegrideditor.directives.js"
  ],
  "css": [
    "~/App_Plugins/DocTypeGridEditor/Css/doctypegrideditor.css"
  ]
}

I hope anyone can benefit from this. See you around!

DocTypeGridEditor - partial view was not found

We've been working with DocTypeGridEditor for years, but just once in a while, there's an issue that takes more than 5 minutes to isolate.

I'm going to assume you know what I'm talking about, for an intro to DTGE will take too much time and can be found elsewhere.

Below is the popup that the users get when adding a control to the grid.


Embed, Image, Macro and Rich text are there by default. Headline and Quote is something we replaced in our base project. Widget is something specific for the project I'm working on.

The problem is the "Doc Type" option. It's not configured in the grid.editors.config.js where you control which controls the editors can pick. It must be added behind the scenes by the DTGE implementation somehow.

For some reason, however, if one of our editors picked "Doc Type" instead of one of the others, it yields the following error:
System.InvalidOperationException: The partial view 'richTextBlock' was not found or no view engine supports the searched locations. The following locations were searched:
~/Views/ContentPage/richTextBlock.aspx
~/Views/ContentPage/richTextBlock.ascx
~/Views/Shared/richTextBlock.aspx
~/Views/Shared/richTextBlock.ascx
~/Views/ContentPage/richTextBlock.cshtml
~/Views/ContentPage/richTextBlock.vbhtml
~/Views/Shared/richTextBlock.cshtml
~/Views/Shared/richTextBlock.vbhtml
~/Views/Partials/richTextBlock.cshtml
~/Views/MacroPartials/richTextBlock.cshtml
~/Views/richTextBlock.cshtml
   at System.Web.Mvc.HtmlHelper.FindPartialView(ViewContext viewContext, String partialViewName, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at Our.Umbraco.DocTypeGridEditor.Web.Extensions.HtmlHelperExtensions.RenderDocTypeGridEditorItem(HtmlHelper helper, IPublishedElement content, String editorAlias, String viewPath, String previewViewPath, Boolean isPreview)
   at ASP._Page_app_plugins_doctypegrideditor_render_doctypegrideditor_cshtml.Execute() in C:\Projects\DPW\Website\ProjectV8.Site\app_plugins\doctypegrideditor\render\doctypegrideditor.cshtml:line 28
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at Umbraco.Web.Mvc.ProfilingView.Render(ViewContext viewContext, TextWriter writer) in d:\a\1\s\src\Umbraco.Web\Mvc\ProfilingView.cs:line 25
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at ASP._Page_Views_Partials_grid_editors_base_cshtml.Execute() in C:\Projects\DPW\Website\ProjectV8.Site\Views\Partials\grid\editors\base.cshtml:line 20

Using the richTextBlock directly works fine, so after some head-scratching I discovered that we usually declare our components as follows:
  {
    "name": "Headline",
    "alias": "headline",
    "view": "/App_Plugins/DocTypeGridEditor/Views/doctypegrideditor.html",
    "icon": "icon-coin",
    "render": "/App_Plugins/DocTypeGridEditor/Render/DocTypeGridEditor.cshtml",
    "config": {
      "allowedDocTypes": [
        "headlineBlock"
      ],
      "nameTemplate": "",
      "enablePreview": true,
      "viewPath": "/Views/Partials/Grid/Editors/",
      "previewViewPath": "/Views/Partials/Grid/Editors/",
      "previewCssFilePath": "",
      "previewJsFilePath": ""
    }
  }
Some debugging showed me that the "Doc Type" definition, which is added automatically, even if you want it or not, is this:
  {
    "name": "Doc Type",
    "alias": "docType",
    "view": "/App_Plugins/DocTypeGridEditor/Views/doctypegrideditor.html",
    "render": "/App_Plugins/DocTypeGridEditor/Render/DocTypeGridEditor.cshtml",
    "icon": "icon-item-arrangement",
    "config": {
      "allowedDocTypes": [],
      "nameTemplate": "",
      "enablePreview": false,
      "viewPath": "/Views/Partials/Grid/Editors/DocTypeGridEditor/",
      "previewViewPath": "/Views/Partials/Grid/Editors/DocTypeGridEditor/Previews/",
      "previewCssFilePath": "",
      "previewJsFilePath": ""
    }
  }
The difference is the viewPath-property, ending in /DocTypeGridEditor/, which we do not have. Historical reasons, I guess.

The only way to override this default behavious is by manually adding the DocType configuration to the grid.editors.config.js, with the correct path of course.

Now, if I find a way to eliminate the Doc Type option altogether, I'll let you know.

Tuesday, 21 April 2020

Easy tool for Pagespeed optimization

Last october, I talked at Duugfest about Google Pagespeed and how to optimize your page for scoring high ...  having a fast, mobile-friendly website.

I even blogged about it in februari, after winning the Google Hackathon and in more technical detail  here.

One of the things I mentioned at Duugfest was that I had made a "tool" (an MVC ActionFilterAttribute) that would post-process all rendered HTML and "do something" with all the images.
Hang tight, I'll get less vague in a minute.

More than one of you asked if I could share the code for the filter, and it took me until now to get around to that. So, I created a nuget package with my ActionFilter and called it Our.Umbraco.PageSpeed. The intention is to add more tricks as I devise them along the way.

The reason for this one-filter-catches-all approach is 2-fold.

1. Lazy-loading

Google Pagespeed Insights recommends lazy loading your images. It's very easy to use LaySizes for it. So easy, in fact, that I won't repeat anything you can easily read in their README.

Now, what you'll have to do, is go through all your code (views, partial views, ...) and tweak your images a bit.
Basically, your images look like this:

<img data-src="image.jpg" class="lazyload" />

There are reasons NOT to do this however:

  • it's boring, repetitive work, prone to errors.
  • you may miss an image here and there. Nobody will notice, until you run Pagespeed insights.
  • you don't control ALL places where images are rendered. E.g. a content editor can put an image inside a richtext-field.

A filter will just parse the HTML and replace all images. None are missed, no typo's are made.

2. Serve images in better formats

Google suggest serving images in WEBP, as it is a better format than jpg, png, ...
Not all browsers support it, however, so you may want to resort to using <picture> instead of <img> and provide multiple images, out of which your browser can choose, depending on it's capabilities.

This would mean something along the lines of:

<picture>
  <source type="image/webp" src="webp-version of image.jpg" />
  <source data-src="image.jpg" class="lazyload" />
</picture>
If only there was a way of converting images to webp without requiring our content editors to manage multiple versions of each image.

But wait, there IS : ImageProcessor is baked in in Umbraco and has a plugin for Webp.


Combining it all 

Now, if we combine the possibility of

  • replacing all our img-tags on-the-fly by source-tags,
  • providing a way to convert to Webp,
  • lazy loading the result.
we get the following:

<picture>
  <source type="image/webp" data-src="image.jpg?format=webp&quality=70" class="lazyload" />
  <source data-src="image.jpg" class="lazyload" />
</picture>

How to achieve this?

  1. Install LazySizes in your project as per their instructions.
  2. Install nuget ImageProcessor.Plugins.WebP
  3. Install nuget Our.Umbraco.PageSpeed
  4. Decorate your controller with the LazyLoadFilter-attribute
[LazyLoadFilter]
public class MyRenderMvcController : RenderMvcController
{
    public override ActionResult Index(RenderModel model)
    {
        //Do stuff here. Or not, see if I care.
        return base.Index(model);
    }
}

And for all those DocTypes that you didn't write a Controller for. How do you make sure they get their fair share of the fun?

For these, you can set the default Controller to your MyRenderMvcController, and it will also benefit from the LazyLoadFilter-attribute.

DefaultRenderMvcControllerResolver.Current.SetDefaultControllerType(typeof(MyRenderMvcController));

If you want to give this a go and it doesn't work for you for any reason, give me a shout and we'll figure something out.

Wednesday, 15 April 2020

Umbraco 8 - Language fallback for Grid properties

In my previous post (which was hanging around in draft for months until I finally got around to finishing it) I mentioned that my way-of-working with Language Fallback does not work with Grid properties.

Grids are never really empty

The reason for this is that when you use property.Value("propertyAlias", fallback: Fallback.ToLanguage), the only scenario where fallback occurs, is when the value of that property is empty.
When you make a new language version of an item and you do not put anything in the grid, it still is not empty, because what constitutes as an "empty" grid, actually looks like this.

{
  "name": "1 column layout",
  "sections": [
    {
      "grid": "12",
      "rows": []
    }
  ]
}

A second problem I encountered, is that the HtmlHelper-extension that is used to render your grid in a view only accepts a propertyAlias and under the hood it goes straight for the property in question, not through .Value(), preventing you from adding a similar workaround as in my previous article.

Fortunately Umbraco is Open Source, and the code for GetGridHtml is easily located.

public static MvcHtmlString GetGridHtml(this HtmlHelper html, IPublishedContent contentItem, string propertyAlias, string framework)
{
    if (string.IsNullOrWhiteSpace(propertyAlias)) throw new ArgumentNullOrEmptyException(nameof(propertyAlias));

    var view = "Grid/" + framework;
    var prop = contentItem.GetProperty(propertyAlias);
    if (prop == null) throw new NullReferenceException("No property type found with alias " + propertyAlias);
    var model = prop.GetValue();

    var asString = model as string;
    if (asString != null && string.IsNullOrEmpty(asString)) return new MvcHtmlString(string.Empty);

    return html.Partial(view, model);
}

A new way of rendering

So, all I had to do was create my own class with my own extension method.

Now what happens is:
  • I get the value of the grid-property. With language fallback indicated, but that never happens, because a grid is never empty.
  • If the grid is functionally "empty", I get the English value of the same grid-property.
  • Normal processing resumes.

In my views, where - until now - I used the standard way of rendering grids, I had to add a using statement and change the name to my own extension method.

@using MyCode.Umbraco.Web.Extensions;
@inherits UmbracoViewPage<ContentPage>
@{
    Layout = "BasePage.cshtml";
}
<div>
    @Html.GetFallbackGridHtml(Model, ContentPage.GetModelPropertyType(c => c.Body).Alias, "site")
</div>

For those interested, here's the entire class. No rocket science, but it made my job just a little easier again.

using System.Linq;
using System.Web.Mvc;
using System.Web.Mvc.Html;
using Newtonsoft.Json.Linq;
using Umbraco.Core.Models.PublishedContent;
using Umbraco.Web;
using Umbraco.Web.Composing;

namespace MyCode.Umbraco.Web.Extensions
{
    public static class GridExtensions
    {
        public static bool IsGridEmpty(object gridContent)
        {
            var asJToken = gridContent as JToken;
            if (asJToken != null)
            {
                //Check all sections
                var sections = asJToken["sections"] as JArray;
                if (sections != null && sections.Any())
                {
                    //If any section has any row --> not empty
                    foreach (var section in sections)
                    {
                        var rows = section["rows"] as JArray;
                        if (rows != null && rows.Any())
                        {
                            return false;
                        }
                    }
                }
            }

            return true;
        }

        public static MvcHtmlString GetFallbackGridHtml(this HtmlHelper html, IPublishedContent content, string propertyAlias, string framework = "bootstrap3")
        {
            var gridContent = content.Value(propertyAlias, fallback: Fallback.ToLanguage);

            //If gridcontent is "empty", do a fallback to defaultLanguage
            if (IsGridEmpty(gridContent))
            {
                var defaultLanguage = Current.Services.LocalizationService.GetDefaultLanguageIsoCode();
                gridContent = content.Value(propertyAlias, defaultLanguage);
            }

            return html.GetFallbackGridHtml(gridContent, framework);
        }

        public static MvcHtmlString GetFallbackGridHtml(this HtmlHelper html, object gridContent, string framework = "bootstrap3")
        {
            if (gridContent == null)
            {
                return new MvcHtmlString(string.Empty);
            }

            var view = "Grid/" + framework;
            return html.Partial(view, gridContent);
        }
    }
}

Umbraco 8 - ModelsBuilder with Language Fallback


In Umbraco 8, they introduced the much-anticipated multi-language feature called Variants.
I won't describe this in detail, just check the documentation on Variants.

Edit: the approach below is superseded by changing the default fallback behaviour of Umbraco as described here.

Language fallback

In concert with Variants, we can now have language fallback. This basically means that when you have an English and a Dutch version of a page, you do not HAVE to fill in every field on the Dutch version. When left empty it can fall back to the English version (assuming English is marked as Fallback Language in your language settings).


Now, fallback doesn't just magically work, it involves some work on your part.

Assume that message is a contentItem (derived from IPublishedContent) and we want to get the value of the title property.

The traditional way is to get it by calling the Value() method.
var title = message.Value<string>("title");

This will give you the value of the title in the current culture. If this is empty in Dutch, you will get an empty string.

Enable language fallback

To enable language fallback, you would add a fallback parameter.
var title = message.Value<string>("title", fallback: Fallback.ToLanguage);

Verify VariationContext

If - for some reason - the current culture in your context is not set or plain wrong, you can give the culture as a parameter to the Value() method, but this is not the way to go.

I use a lot of API's (derived from Umbraco.Web.WebApi.UmbracoApiController) and they usually don't set the current Umbraco culture.
Given a parameter language containing the culture you need (coming for instance from a request header), you can set the correct culture for Variant fallback as follows:

Current.UmbracoContext.VariationContextAccessor.VariationContext = new VariationContext(language);

You now get the title in the correct language or - when empty - the fallback langauge.

ModelsBuilder

I hear you thinking : "Dude, we've been using ModelsBuilder to have strongly-typed content models for years. Do you really expect us to go back to .Value("propertyName")?"

No, I do not. But it requires some more work.

I won't go into the details of Extending the Modelsbuilder, but here's the gist of it.
The generated class (simplified for brevity) for our Message model would be
[PublishedModel("message")]
public partial class Message : PublishedContentModel
{
    [ImplementPropertyType("title")]
    public string Title => this.Value<string>("title");

    [ImplementPropertyType("someProperty")]
    public string SomeProperty => this.Value<string>("someProperty");
}

We cannot modify this class, because it is generated by the ModelsBuilder. But because it is a partial class, we can extend it.

Below is the custom class you add to your project (same name and namespace as the generated one).
public partial class Message
{
    [ImplementPropertyType("title")]
    public string Title => this.Value<string>("title", fallback: Fallback.ToLanguage);
}

You'll need to remove the implementation for title from the generated class. The next time the generator runs, it will skip the title property because of the ImplementPropertyType-attribute in your custom class.

It is now possible for you to use the following code to get the title (with fallback) from the message item.
var title = message.Title;

Random thoughts

  • The above approach does not work for Grid content, but that's subject for a different post.
  • When creating a new language version of an item in Umbraco, the name of the item is a mandatory field. It would be nice if this could be left empty and also fallback to the English version.
  • If you make a field mandatory, it is mandatory in all languages, so you can't leave it empty and depend on fallback.
    Personally, I made many mandatory fields optional again, because I feel the advantage of fallback outways the need for mandatory fields. (your call)
Feedback or questions? Give me a shout.

Wednesday, 10 April 2019

Umbraco - Upload SVG as Image

In Umbraco, you can upload multiple images into the media library by simply dragging and dropping them all together unto a Media Folder.


This works fine for JPG, GIF, PNG and the like, but when you drop an SVG, it is uploaded as a File instead of an Image.

When you subsequently want to use a MediaPicker to select that image in one of your content items, you cannot select the SVG, because only Images are allowed, not Files.

It's perfectly possible to click Create > Image, then drop the SVG on the Upload image pane, but as a developer, I pride myself on my laziness. I will not upload 28 images one by one, if I can solve this once and for all.

"Hidden" setting

The list of file extensions that is recognized as an Image (everything else is a File) is defined in settings/content/imaging/imageFileTypes.

For some reason, however, this setting is not present in config/umbracoSettings.config by default.
When not present, the code falls back to jpeg, jpg, gif, bmp, png, tiff, tif. We just want to add svg to the list.

There are many settings omitted like this. Maybe they do not want to overload the configs with settings that are falling back to default anyway? I've seen the same in other CMS's.

So, all you need to do, is go to config/umbracoSettings.config and add the following :

<?xml version="1.0" encoding="utf-8" ?>
<settings>   

  <content>
    <imaging>
      <!-- what file extension that should cause umbraco to create thumbnails -->
      <imageFileTypes>jpeg,jpg,gif,bmp,png,tiff,tif,svg</imageFileTypes>     
    </imaging>

    ......

  </content>
</settings>

Now, when you drop one or more SVG's on a folder, it will be recognized as an Image.

Monday, 8 April 2019

Umbraco Courier - Azure Storage

When you use Umbraco Courier to transfer items from development to production or vice-versa, you may run into a small problem when you store your media in AzureStorage, as is highly recommended.

The problem

When transferring media items with Courier, the data and the blob (= the actual image) are transferred to the target environment, but at the target, Umbraco is saving the blob to the default location on disk, not checking its FileSystemProviders, telling it to transfer the blob to AzureStorage.

The solution

I made a small modification to Hugo's solution on the Umbraco forum, where he raises the issue, then solves it himself.

You'll need to provide 3 EventHandlers.

The first one will run on the sending environment, telling the target to queue a postprocessor for the image.

public class MediaEventTrigger : ApplicationEventHandler
{
  protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
  {
    ExtractionManagerEvents.ExtractedItemResources += ExtractedItemResources; ;
  }

  private void ExtractedItemResources(object sender, ItemEventArgs e)
  {
    var provider = sender as RevisionExtraction;

    //make sure the media event only fires for media items, not other resources
    if (e.Item.ItemId.ProviderId == ItemProviderIds.mediapropertyDataItemProviderGuid)
    {
      var media = e.Item.Dependencies.Single(i => i.ItemId.ProviderId == ItemProviderIds.mediaItemProviderGuid);
      provider.Context.ExecuteEvent("QueueCourierMediaEvent", e.ItemId, new SerializableDictionary<string, string>());
    }
  }
}

The second one will be running at the receiving end. It will queue the third, instructing it to move the blob to Azure Storage.

public class MediaEventListener : ItemEventProvider
{
  public override string Alias => "QueueCourierMediaEvent";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    ExecutionContext.QueueEvent("UpdateMediaPath", itemId, parameters, EventManagerSystemQueues.DeploymentComplete);
  }
}

And now for the actual moving of the blob.

public class MediaEventQueueListener : ItemEventProvider
{
  public override string Alias => "UpdateMediaPath";

  public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
  {
    var fs = FileSystemProviderManager.Current.GetFileSystemProvider<MediaFileSystem>();
    var ms = ApplicationContext.Current.Services.MediaService;

    var media = ms.GetById(Guid.Parse(itemId.Id));
    var oldPath = GetMediaPath(media);
    if (!string.IsNullOrEmpty(oldPath))
    {
      var absolutePath = HttpContext.Current.Server.MapPath(oldPath); //path to file on filesystem
      using (var fileStream = System.IO.File.OpenRead(absolutePath))
      {
        fs.AddFile(oldPath, fileStream); //will be picked up by FileSystemProvider and saved to AzureStorage
      }
     Directory.Delete(Path.GetDirectoryName(absolutePath), true);
     }
  }

  private string GetMediaPath(IMedia media)
  {
    //Image may use ImageCropper (or not), so test presence of Json
    var umbracoFile = media.GetValue<string>(Constants.Conventions.Media.File);
    if (!string.IsNullOrWhiteSpace(umbracoFile))
    {
      if (umbracoFile.StartsWith("{"))
      {
        //the umbracofile property contains the src for the image, which at the beginning is the location of the file on the filesystem.
        var umbracoFileJson = JsonConvert.DeserializeObject<ImageCropDataSet>(umbracoFile);
        return $"~{umbracoFileJson.Src}";
      }
      else
      {
        //Contains the path directly
        return umbracoFile;
      }
    }

    return string.Empty;
  }
}

Shouldn't we rename?

No. Or, not really.

Let's say we are transferring media/1028/flower.jpg from Dev to Prod. The path will remain media/1028/flower.jpg, because that just makes life easy.

You could argue that we need to rename each item's folder with a Guid or something, so it becomes unique, but then you would have to write another handler, parsing and updating all your content, as you'd need to update all media-paths in grid-content referencing your media.

If you always migrate in one direction, this becomes a non-issue, as their will be (almost) no name-collisions on the media folder.

Name-collisions?

Suppose someone manually added tree.jpg to Prod directly.
It is possible that - after transfer from Dev - we now have media/1028/flower.jpg and media/1028/tree.jpg in the same folder on Azure storage. I can live with that.

The only scenario where this becomes a problem is, if you uploaded a different flower.jpg and the blob is overwritten when you transfer the one from Dev.

In my case, this is extremely unlikely, but it's good to at least be aware of this possibility.

Umbraco Courier - Https

This is going to be a short one, but one that took me far too long to figure out.

Install Courier

First off, when you install Courier in your Umbraco, make sure you install the package in each environment. Don't just add the configs and dlls to your source and deploy it. It requires database changes that are only deployed through the package installer.

Configure Courier

I'd suggest giving the documentation a quick run-through. It's rather thorough.

The most important is configuring your repositories (i.e. the environments you want to be able to send data to).

<repository alias="DEV" name="Development" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-dev.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="ACC" name="Acceptance" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-acc.azurewebsites.net</url>
  <user>0</user>
</repository>
<repository alias="PRD" name="Production" type="CourierWebserviceRepositoryProvider" visible="true">
  <url>https://xxx-prd.azurewebsites.net</url>
  <user>0</user>
</repository>

Https not working

Out of the box, post-install, this should just work.

If you're unlucky - like I always am - when trying to use Courier to transfer a few items, you may get

Cannot connect to the destination
"https://xxxx//umbraco/plugins/courier/webservices/repository.asmx"

Take note of the https. There may be other reasons why it's not working for you, but if it works over http and not over https, the solution is simple.

According to Umbraco Support, https is not supported by Courier, but I found that a bit hard to believe. Comparing with other projects - where it WAS working over https - I found the culprit to be the SecurityProtocol.

The only thing you need to do to get this working, is change the default Securityprotocl to TLS1.2 (or higher, if you have this option and are reading this a few years in the future).

Find a startup-class in your project (ContainerInitializer or WebApiConfig or whatever, you'll find something) and add the following line:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

Tuesday, 27 November 2018

Umbraco Headless - Optional Images

When you use the Umbraco.Headless.Client on top of an Umbraco Headless instance, you are accessing Umbraco through a REST API. This means your content is serialized (to JSON) on the Umbraco side and deserialized inside the Headless-client.

BUT : I ran into a problem when using optional Images.

First steps

Create a very simple Document Type, with 2 properties.


Create a class in your Visual Studio project that maps to this DocumentType.

using Umbraco.Headless.Client.Net.Models;

namespace Models
{
    public class Test : ContentItem, IContentBase
    {
        public string Title { get; set; }
        public string Description { get; set; }
    }
}

Fetch a node that uses this template. The quickest way is to get it via its Id.
Normally, the contentService would be injected, but for simplicity's sake, let's keep it like this.

  var umbracoHeadlessUrl = ConfigurationManager.AppSettings[umbracoHeadless:url];
  var contentService =  new PublishedContentService(umbracoHeadlessUrl);

  var testNode = await contentService.GetById<Test>(id);

The resulting JSON looks like this :

{
    "Title": "Title 1",
    "Description": "<p>A very long description.</p>",
    "id": "1940908b-460c-47dd-825a-010381a57abd",
    "parentId": "7a0e6214-7802-4130-b77c-0965148b0a33",
    "name": "Test",
    "path": null,
    "contentTypeAlias": "test",
    "createDate": "0001-01-01T00:00:00",
    "updateDate": "0001-01-01T00:00:00",
    "url": null,
    "writerName": null,
    "creatorName": null,
    "properties": null,
    "_links": null
}

Adding an Image

Add a property to your template using datatype MediaPicker.
Add a property to your Test-class of type Image. I'm only interested in the Url-property of this image, so that's the only one I created.

public class Image
{
    public string Url { get; set; }
}
public class Test : ContentItem, IContentBase
{
    public string Title { get; set; }
    public string Description { get; set; }
    public Image Image { get; set; }
}

When I leave the Image-property empty in Umbraco and run my changes, I get the following error:

Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Models.Image' 
because the type requires a JSON object (e.g. {\"name\":\"value\"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {\"name\":\"value\"}) or change the deserialized 
type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> 
that can be deserialized from a JSON array.


The reason for this error, is that the JSON for the empty image is

    "image": [],

which would require your property to become a List<Image> instead.
However, if you put something in the Image property, the JSON comes out as

    "image": {
        "writerName": "Jeroen Vantroyen",
        "creatorName": "",
        "writerId": 1,
        "creatorId": 1,
        "urlName": "question",
        "url": "/media/1001/images.png",
        ...
        //truncated for brevity
    },

So, when left empty, it's an (empty) array. When filled in, it's a simple JSON object.
How to fix this?

JsonConvertor to the rescue

The solution is simple and elegant, using a JsonConvertor, which can handle both arrays and single objects.
I do not take credit for the solution below, but will save you the hunt for the code.

    public class SingleOrArrayConverter<T> : JsonConverter
    {
        public override bool CanConvert(Type objectType)
        {
            return (objectType == typeof(List<T>));
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            JToken token = JToken.Load(reader);
            if (token.Type == JTokenType.Array)
            {
                return token.ToObject<List<T>>();
            }
            return new List<T> { token.ToObject<T>() };
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            List<T> list = (List<T>)value;
            if (list.Count == 1)
            {
                value = list[0];
            }
            serializer.Serialize(writer, value);
        }

        public override bool CanWrite
        {
            get { return true; }
        }
    }

To use this, I modify my test-class one last time.

public class Test : ContentItem, IContentBase
{
    public string Title { get; set; }
    public string Description { get; set; }

    [JsonConverter(typeof(SingleOrArrayConverter<Image>))]
    public List<Image> Image { get; set; }
}

To use an instance of this class, I would resort to Image.FirstOrDefault().

Umbraco Headless - MultiLingual with Vorto

You want to fully embrace the advantages offered by Vorto inside your Umbraco solution. Unfortunately, the current Umbraco Headless béta does not include it out of the box (yet!).
Let's install it ourselves.

Vor-who?

New to Umbraco or living under a rock? Vorto is a nice, little module, that transforms your text-properties into multi-lingual text properties.

Yes, Umbraco 8 will make it obsolete as it will have a built-in mechaniscm for multi-language. Since we're not there yet, let's just carry on.

If your past solution to multi-language was to create TWO properties for each and every text-field (one English, one Spanish, for example), it's not only annoying to maintain and keep track, but it's a nightmare when the customer suddenly decides to add a 3rd language, and a 4th ...

To take advantage of Vorto, all you have to do is :
  • Install it. (see below)
  • Add one or more datatypes in the Developer-section (e.g. one for Textstring, one for Richtext editor).
  • Define properties in your document types with this datatype.

When used on a normal TextString it looks like below. Easy tabs to switch language. Checkmarks indicate which languages have a value.

It automatically grabs all defined languages, but you can override this behaviour, set the default language ...  Just go read about it here.



Normal Installation

For on-premise Umbraco or Umbraco Cloud, you could install this as a nuget-package. If you do not have any custom code, you can download the package and install it in the Umbraco backend.

Install-Package Our.Umbraco.Vorto

The installation contains plugin-files and a dll, so no database-changes.
  • App_Plugins/Vorto/css/vorto.css
  • App_Plugins/Vorto/js/jquery.hoverIntent.minified.js
  • App_Plugins/Vorto/js/vorto.js
  • App_Plugins/Vorto/views/vorto.html
  • App_Plugins/Vorto/views/vorto.languagePicker.html
  • App_Plugins/Vorto/views/vorto.languageSourceRadioList.html
  • App_Plugins/Vorto/views/vorto.mandatoryBehaviousPicker.html
  • App_Plugins/Vorto/views/vorto.poropertyEditorPicker.html
  • App_Plugins/Vorto/views/vorto.rtlBehaviourPicker.html
  • bin/Our.Umbraco.Vorto.dll

Umbraco Headless installation

We were as good as assured by Niels Hartvig (founder and CEO @ Umbraco) that Vorto would become part of Umbraco Headless, but it wasn't just yet.

So, if it's so easy to install, why am I writing about it?
Well, Umbraco Headless doesn't provide the option to install packages, nor does it allow you access to the repository.

If this last mention of the repository doesn't mean anything to you, I'd suggest reading up on how Umbraco Cloud handles deployment. In a nutshell : once your Umbraco Cloud installation is done for you, you get access to the git repository. Commit something on it and it gets deployed to your website.

So, how DO you deploy Vorto?

Access the git repository

In a normal Umbraco Cloud project, you can go to https://www.s1.umbraco.io/project/<your-project-name>, right-click an environment and go to "Power tools (Kudu)".



In Umbraco Headless, this option is removed, but you can still manually go to https://<your-project-name>.scm.s1.umbraco.io.

From there, go to Source control info and copy the Git-url.



Clone and commit

Clone the repository locally, add the files required for Vorto (see above) and commit/push them back to the repository.

You will have to restart your environment (again from the project management page) for the changes to be picked up.

And you're good to go.

Tuesday, 6 November 2018

Umbraco Headless - performance fix

Umbraco headless (currently in béta) looks very promising for a multi-platform approach. No fuss, just content. Under heavy load, however, it - or SOMEthing at least - buckled, so here's how we found and solved the problem. (Until the next version arrives with a definitive fix)

Starting Headless

Having done a couple of projects in Umbraco Cloud, we were itching to try out Umbraco Headless (currently in béta). An opportunity presented itself in a small project - fully React - with a moderate amount of content to manage.

Spinning up a test-instance was fast and easy, but for some reason, you can't add one to your existing collection of projects. I needed to register with a new email. Probably just part of it being béta.

Once created, you get the same Umbraco backend that you are (or should be) used to, except for some options that have been removed.
Under Settings, there's no Templates, no Partial Views, no Scripts, but that makes sense, as your Umbraco is reduced to a big box of content, no longer worrying about presentation.



I quickly created a few document-types, added some content and wrote an API on top of that (hosted in Azure), using the Umbraco.Headless.Client (.Net Framework, in my case). Using the client is rather straightforward, as you're usually just fetching one item or a list of children/descendants.

You could do without this extra API layer in between, but I do not like to have all business logic in the frontend code. Plus, you get all the advantages of Azure API Management.

Under load

System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.Net.WebException: Unable to connect to the remote server
---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

However, when the React app neared completion and we started testing, we experienced intermittent hickups in responsetime. When the system started doing 10-20 API-request per second, each in turn doing multiple requests towards Umbraco, requests started to drop out. Delays up to 42 seconds (and always 42!) suggested something timing out.

Another advantage of the extra API layer : Application Insights traced the problem to a Dependency-call towards Umbraco.

The culprit

My initial suspicion was that it would be a resource issue. It's still in béta, so maybe these test-instances were shared on a server, low on resources at certain times?

Turns out it was NOT the Umbraco backend having issues. No errors whatsoever in the logs. I went as far as transferring my content (a nightmare on it's own, as Content Transfer is not supported in Headless) to a regular Umbraco Cloud project (where we know what to expect, resource-wise) and adding the required dll's and config-changes to fake it being Headless. I immediately faced the same issue.

Digging around (i.e. decompiling) the Umbraco Headless client, I saw it used Refit under the hood.

Refit is Open Source, so debugging was easy.
The default implementation - as used by Umbraco - creates a new HttpClient for every request. In .Net Core this is not a problem, but in .Net framework this is a big no-no. There's enough written about the topic already, so I will not reiterate. (e.g. YOU'RE USING HTTPCLIENT WRONG AND IT IS DESTABILIZING YOUR SOFTWARE).

I was in brief contact with someone from Refit, to be told that this is just the default implementation and Refit has overloads allowing you to provide your own HttpClient as you see fit.

The solution

So, you can provide Refit with your own client. Good in theory, if the Umbraco.Headless.Client was open source. (I fully understand why it is not)

The current version of the Headless-client (0.9.7-CI-20180905-01) has no way of being manipulated. I have received confirmation from Umbraco HQ that the next version will create an instance of the HttpClient, which will have the same lifespan as the service, with an option of adding your own HttpClient instance.

In the meantime, what I did was get a copy of Refit (4.6.30) and changed its default implementation to reuse a static HttpClient every time. Build and deployed the new dll and put it in my API-project. All problems disappeared immediately.

Here's what I added, if you're interested. To be absolutely safe, I implemented multiple static clients (one per host you're calling) instead of just the one. If it were a permanent solution, I would probably fiddle with it some more, but it's only needed until the next release of Umbraco.Headless.Client.

private static readonly Dictionary<string, HttpClient> clients = new Dictionary<string, HttpClient>();

private static HttpClient Client(string hostUrl, HttpMessageHandler innerHandler = null)
{
    HttpClient client;
    if (!clients.TryGetValue(hostUrl, out client))
    {
        client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };
        clients.Add(hostUrl, client);
    }

    return client;
}

And calling this inside public static T For<T>(string hostUrl, RefitSettings settings)

//Default Refit-implementation
//var client = new HttpClient(innerHandler ?? new HttpClientHandler()) { BaseAddress = new Uri(hostUrl) };

//Temporary workaround
var client = Client(hostUrl, innerHandler);