Golang plugins

December 21, 2021

While working on my static site generator, I stumbled upon the following one-man requirement:

It would be great if I could execute custom template functions

Now, while Go provides a way to define and use any kind of function in templates via template.FuncMap and template.Funcs, the challenge here was to be able to define such functions on the fly and link them to the already existing main executable (the static site generator).

Dynamically-linked libraries

What we need is a way to create our own library of template functions, which can be linked to the main executable on demand, at runtime. This allows us to update the library without having to recompile the main executable. This concept is quite similar to dynamic linking. Dynamic linking, in a nutshell, means binding a (shared) library into a running process. The main reason this technique exists is to save memory: multiple processes using the same shared library can point to a single copy of that library in physical memory. In our particular use case, however, the main advantage is the ability to update the shared library without recompiling the main executable.

Go plugins

While googling “Go dynamic linking”, I came across a doc called Go Execution Modes. The document (originally from 2014 and updated in 2016) outlines future (now current) directions for Go’s support of shared libraries and related ways of building Go programs. Among all the build modes described, the plugin build mode seems to be the most suitable for our original requirement.

An example plugin

Let’s say we want our h1 elements to be rendered with a fancy capital letter. The final result should look like this:

Hello world

The template that generates the previous HTML should be as easy as:

<h1>{{ &quot;Hello world&quot; | fancyTitle }}</h1>

The implementation of fancyTitle should be straightforward:

File plugins/plugin.go

package main // Plugins must live within the main package

import (
    "fmt"
    "html/template"
)

var MyFuncs = template.FuncMap {
    "fancyTitle": func(s string) template.HTML {
        if len(s) < 2 {
            return template.HTML(s)
        }

        return template.HTML(
            fmt.Sprintf(`<span class="fancy-capital-letter">%s</span>%s`, s[0:1], s[1:]),
        )
    },
}

We put our function within a template.FuncMap because that’s what template.Funcs expects as parameter. The main executable can then load this plugin as follows:

File main.go

package main

import (
    "fmt"
    "html/template"
    "os"
    "os/exec"
    "plugin"
)

const PluginFilepath = "plugins/plugin.so"

func main() {
    var myFuncs template.FuncMap
    myFuncs = loadFuncs()

    t, err := template.New("mypage").Funcs(template.FuncMap(myFuncs)).ParseFiles("hello.html")
    if err != nil {
        panic(err)
    }

    err = t.ExecuteTemplate(os.Stdout, "hello.html", nil)
    if err != nil {
        panic(err)
    }
}

// LoadFuncs loads the custom template functions from a hardcoded location
func loadFuncs() template.FuncMap {
    // Hardcoded location of plugins
    cmd := exec.Command("go", "build", "-buildmode=plugin", "-o", PluginFilepath, "plugins/plugin.go")
    err := cmd.Run()
    if err != nil {
        fmt.Fprintf(os.Stderr, "Could not build plugin: %s", err)
        os.Exit(1)
    }

    plugin, err := plugin.Open(PluginFilepath)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Could not load plugin: %s", err)
        os.Exit(1)
    }

    myFuncsSymbol, err := plugin.Lookup("MyFuncs")
    if err != nil {
        fmt.Fprintf(os.Stderr, "Could not find the symbol MyFuncs: %s", err)
        os.Exit(1)
    }

    var myFuncs *template.FuncMap
    myFuncs, ok := myFuncsSymbol.(*template.FuncMap)
    if !ok {
        fmt.Fprintf(os.Stderr, "MyFuncs must be of type template.FuncMap")
        os.Exit(1)
    }

    return *myFuncs
}

Explanation of the function loadFuncs:

  1. We build the plugin located at plugins/plugin.go into the file plugins/plugin.so
  2. We open the plugin plugins/plugin.so
  3. We search for the symbol MyFuncs in the plugin
  4. We assert that the type of MyFuncs is *template.FuncMap1, and if so, we return it

In this toy example, there are some assumptions about the structure of the plugin. The relative location of the plugin must be known in advance, as well as the name and type of the exported symbol. In a more realistic scenario, things could be a bit more flexible.

Caveats

There are multiple reasons why Go plugins have not gained widespread adoption in the Go community. To name a few:

Notes

[1]. When using Lookup to search for a global variable, you get a pointer (see this GitHub issue).