Wick: An overdue introduction

8 min readAug 28, 2023

This is an intro to wick, an application meta-framework I started building almost two years ago.

Background

Two years ago, I hit a wall. I had worked in software for twenty+ years and spent seven of them helping Shape Security grow from a small startup to a unicorn.

I was ready to start something new. But I couldn’t.

I knew what lay ahead and it was daunting. I couldn’t bear to start the same tasks I’d completed a hundred times before. New languages and frameworks, the cloud, serverless, microservices, etc. didn’t change it. It was still the same work with new names. Integrate a dozen APIs here, make a hundred random libraries work together there, and rewrite the same logic one more time.

The work wasn’t difficult. It’s that I didn’t think we should have to do it anymore.

Most of every application is software components connecting to other software components. Why couldn’t we have the USB experience for software? Swap any dependency in and out, anywhere, anytime. If an ultra-widescreen, 4k monitor can connect to my computer with the same port as my mouse, surely we can make software libraries act as seamlessly.

If we did, we could build real applications more rapidly while reusing more. We could automatically generate documentation and even examples. We could secure everything the same way, deploy it the same way, and test it the same way. Furthermore, if everything was asynchronous, we could scale from a single process to a distributed network without changing code.

These ideas are not new. It’s a lot of what’s found in functional languages, reactive frameworks, and ideas like flow-based programming. The problem is that they’ve mainly applied within the boundaries of a language or runtime. But businesses don’t operate within a language or runtime. Teams have different use cases, varying environments, and myriad tools to integrate with.

With advancements like WebAssembly and tools like Kubernetes and Docker, it seemed like a good time to bring these ideas to a new layer.

Turns out I wasn’t alone in that thought. I was able to secure pre-seed funding and dedicate myself full-time to building what we now call wick.

Wick is an application framework that exists outside of a language and — thanks to WebAssembly — across languages.

At the core of Wick is how it defines a component. A wick component is a collection of operations and each operation takes in and produces an arbitrary number of streams.

Wick components are like libraries in any programming language. They’re collections of functions.

Except they start as YAML.

Wick components

Component definitions are written in YAML and they define configuration, test cases, package metadata, operations, and an operation’s inputs and outputs.

The actual logic for a component comes in four (current) flavors:

WebAssembly: The Wick component definition defines the operations a WebAssembly module handles and Wick delegates to a WASM function on invocation.
Composite: Composite components build their logic from other components. They define logic in a flow — a functional data pipeline —by connecting together other components’ operations. Think of composite component operations as Unix pipes or using map() on arrays or streams.
SQL: SQL components define operations as SQL queries. Wick binds inputs to query arguments, and each row is output to a stream. Wick supports MS SQL Server, Postgres, and Sqlite out-of-the-box.
HTTP: An HTTP component defines the configuration necessary to make an HTTP request. Wick uses an operation’s inputs to build the URL path, query string, and body, and the operation’s output is the response metadata and the body (as raw bytes or JSON).

The best way to understand how this all works is with examples. Building a dynamic application out of a bunch of YAML might seem nuts.

Quickly make a new component

The command above creates an example component with one operation, echo.

The default echo operation has one input stream appropriately called input and an output stream called output. Here's what it looks like:

kind: wick/component@v1
name: my_component
component:
  kind: wick/component/composite@v1
  operations:
  - name: echo
    inputs:
    - name: input
      type: object # an "any" type
    outputs:
    - name: output
      type: object
    flow:
    - <>.input -> <>.output

These configuration files are the source of truth. Even WebAssembly components — which can define their own exports — can’t run in a wick context with anything but the interfaces defined in the definition. This constraint lets tools query and assert against components without parsing myriad languages or inspecting any of a component’s internals. Tools like wick list, for example.

The wick list command loads a component and reports its signature. It's simple, sure, but how many languages or frameworks give you such a tool?

All wick commands accept a --json flag to integrate with other tools. Component definitions also support descriptive fields on many elements, so you can use this output to generate documentation and examples, like we do on our docs site:

Running components from the command line

Since every wick component follows the same general API, the wick invoke command can invoke any operation in any component or any application from the command line.

That’s right, you can invoke a function deep inside an application straight from the command line. No extra configuration or setup necessary.

That means every operation in every application is now a command-line utility.

Need to expose functionality to another team? Already done.

Need to validate data in another time and setting? Use the application itself in a bash script.

Need to troubleshoot why part of an app is failing? Execute an operation from the command line and test it in isolation.

Since we’re on the subject of testing…

Testing components

While you’re free to write unit tests in the language you compile to WebAssembly, wick comes with its own test framework to make adding tests trivial.

Wick test cases can exist as separate files or embedded within a component definition. The test cases define what to pass to an operation and what to assert against in the output.

tests:
  - name: basic_tests
    cases:

      - name: basic_equality
        operation: echo
        inputs:
          - name: input
            value: 'Hello, world!'
        outputs:
          - name: output
            value: 'Hello, world!'

      - name: assertions
        operation: echo
        inputs:
          - name: input
            value:
              string_value: Hello! 
              number: 42
        outputs:
          - name: output
            assertions:
              - operator: Contains
                value: { string_value: Hello! }
              - operator: LessThan
                path: number
                value: 100

Wick prints test results in the TAP — Test Anything Protocol — format. TAP’s been around for decades and most CI/CD tools support TAP output out of the box.

Taking “secure by default” to a new level

I am passionate about application security but it’s time to be honest with ourselves. We are still falling victim to vulnerabilities we understood perfectly well in the ’90s. We need to be more aggressive.

Wick’s not a silver bullet, but we’ve spent hundreds of hours crafting solutions that expose all the power developers need while maintaining the safeguards and features necessary to secure applications automatically.

Features like:

Explicitly configured resources.
Developers explicitly define base URLs, directories, files, ports, etc. as part of the component configuration. Wick validates and processes resources before components even run, let alone get access.
Sandboxing.
Wick sandboxes components and grants each separate permissions and access.
Secure APIs by design
Wick makes vulnerabilities like SQL injection or path manipulation impossible.
Choke points
It’s best practice to pass configuration as ENV variables, but giving all dependencies access to all ENV variables can — at best — lead to difficult-to-reproduce problems and vulnerabilities at worst. Wick only lets the root application access the environment. Configuration bubbles through to components from there.

Many of these designs are already best practices and recommendations. Wick just makes the recommended way to do it the only way to do it.

Auditing applications safely

The wick config audit command outputs a list of all resources used by an application in one fell swoop.

DevOps and security teams can independently audit and validate an application without running or looking at any source code at all.

Securing an app at the component level

An audit report lets you see what an application wants to access. A lockdown configuration defines what an application is allowed to access.

Let’s say you want to integrate a dependency that makes requests from a new URL or needs file system access. You shouldn’t have to grant that access to your whole application, just the component that needs it.

The wick config audit conveniently accepts a --lockdown flag and will happily generate a (passing) lockdown configuration from an audit report.

kind: wick/lockdown@v1
resources:
- kind: wick/resource/tcpport@v1
  address: 0.0.0.0
  port: '8999'
  components: [__local__] # __local__ is the root app context

- kind: wick/resource/url@v1
  allow: https://login.microsoftonline.com/organizations/oauth2/v2.0/token
  components: [oauth]

- kind: wick/resource/url@v1
  allow: http://localhost:5173/
  components: [__local__]

- kind: wick/resource/url@v1
  allow: mssql://account.database.windows.net:1433/database
  components: [reports, users]