Event Transformation in WebAssembly with Fermyon Spin

WebAssembly (aka WASM) is a new binary format that started getting a lot of attention when CloudFlare announced that their workers could run WebAssembly functions allowing multiple languages to be used to write functions. Recently Docker announced a beta that allows users to run WASM binaries via Docker, extending the options for developers to deploy their applications.

While containers are great, WASM binaries also offer isolation, re-usability of components and start fast, as such they could be a great format for serverless functions.

The ecosystem around WASM is moving fast and current limitations (e.g support in Go) will most likely be moot point several months from now. With Fermyon you can already write and deploy WASM applications using their Spin framework. Recently Fermyon also released a Python SDK which is great because I love Python :)

Since serverless is about running functions and routing events to them to build an event-driven application it makes perfect sense to see how a TriggerMesh event-flow could benefit from running a WASM function.

In this blog post I want to show you how to write an event transformation with WASM (using Fermyon Spin) and hook it up to a local event-flow. We will inject an event manually into the broker, configure triggers to invoke the WASM function and route all events for display in a Google CloudRun service.

Let's get started:

Installing Spin and TriggerMesh

The Fermyon blog is great to get all the WASM bits in place and create a skeleton of a Python app that will compile into a WASM module. For brevity here are the steps:

curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
sudo mv ./spin /usr/local/bin/spin
spin plugin install py2wasm
spin templates install --git https://github.com/fermyon/spin-python-sdk

‍

If you have not used TriggerMesh before, simply go through our quickstart. Our local environment relies on Docker being present on your machine and makes use of our CLI tmctl which you can install via brew:

brew install triggermesh/cli/tmctl

‍

Building and Running your WASM application

spin new --accept-defaults http-py wowow
tree wowow/
wowow/
├── Pipfile
├── README.md
├── app.py
└── spin.toml
spin build
spin up

‍

The spin build command will compile the Python application into a WASM binary and you will see a .wasm file in your directory. The spin up will load the wasm module into the Spin framework making an HTTP route available to invoke your wasm module.

In order to make it useful with TriggerMesh and transform incoming events, you need to modify the application to set the CloudEvents headers. These headers are used by the TriggerMesh event broker to route events to their target destination. CloudEvents is a specification from the CNCF serverless working group.

Let's add CloudEvent metadata to the function.

Adding CloudEvent Headers and Transforming the Event

Open the app.py file, import the json module. In the Response object set the headers, ce-type, ce-specversion and ce-source. The modified app.py looks like this:

import json
from spin_http import Response

def get_header_value(headers, attribute):
    header_dict = {key.lower(): value for key, value in headers}
    return header_dict.get(attribute.lower())

def create_response(headers, body):
    content_type = ("content-type", "application/json")
    ce_type = ("ce-type", f"{get_header_value(headers, 'ce-type')}.transformed")
    ce_source = ("ce-source", "spin-app")
    ce_specversion = ("ce-specversion", get_header_value(headers, "ce-specversion"))

    return Response(200, [content_type, ce_type, ce_source, ce_specversion], body)


def handle_request(request):
    print(request.headers)

    body = request.body.decode('utf-8')
    data = json.loads(body)
    data["hello"] = "wasm"

    response_body = bytes(json.dumps(data), "utf-8")
    response = create_response(request.headers, response_body)

    return response

‍

You can now rebuild and run the application

spin build
spin up

‍

At this point you are ready to include this function into a TriggerMesh event-flow using the tmctl CLI.

Create a Broker and Add the WASM function

We will create the simplest flow there is: Send an event of type io.wasm.event into a broker, transform that event by adding a new key in the payload and display all the events by forwarding them to a Google CloudRun service. This overall flow is depicted below.

tmctl create broker wasm

‍

Since the WASM function is running locally and returns CloudEvents we can use the so-called CloudEvents target and use the local Docker network:

tmctl create target cloudevents --endpoint http://host.docker.internal:3000

‍

You will see a new Docker container running which runs the TriggerMesh CloudEvents adapter and acts as an event forwarder to send the event to the Spin/WASM function endpoint. The event broker also runs by default as an in-memory broker and you will see the container for it as well.

docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED          STATUS          PORTS                     NAMES
f10f47f0fde2   gcr.io/triggermesh/cloudeventstarget-adapter:v1.24.0   "/ko-app/cloudevents…"   6 seconds ago    Up 5 seconds    0.0.0.0:53441->8080/tcp   wasm-cloudeventstarget
3ddd5e685700   gcr.io/triggermesh/memory-broker:v1.2.0                "/memory-broker star…"   14 seconds ago   Up 13 seconds   0.0.0.0:53435->8080/tcp   wasm-broker

‍

To display all the events we are also going to add an event Display running in Google CloudRun

tmctl create target cloudevents --name cloudrun --endpoint https://triggermesh-console-tu4luqbmqq-uc.a.run.app/

‍

The routing is done by creating Triggers, so we will create two triggers: a wildcard trigger to display all the events in CloudRun and a specific Trigger to execute our wasm function when events of type io.wasm.event are sent to the broker

tmctl create trigger --target cloudrun
tmctl create trigger --eventTypes io.wasm.event --target wasm-cloudeventstarget

‍

You describe the full event-flow that we just setup with the describe command like so:

tmctl describe
Broker     Status
wasm       online(http://localhost:53435)

Trigger                   Target                     Filter
wasm-trigger-0aedded8     cloudrun                   *
wasm-trigger-3c46c8c4     wasm-cloudeventstarget     type is io.wasm.event

Target                     Kind                  Expected Events     Status
wasm-cloudeventstarget     cloudeventstarget     *                   online(http://localhost:53441)
cloudrun                   cloudeventstarget     *                   online(http://localhost:53449)

‍

And now to see it in action, send an event

tmctl send-event --eventType io.wasm.event '{"hi":"world"}'

‍

In the display you will see the original event and the transformed event with the added key and the new event type.

Congratulations, you just created a Python function compiled it to WASM run it as a module using the Fermyon Spin framework and added it to an event-flow with TriggerMesh.

Check out @sebgoa taking this for a spin :)