Background

People are a big part of operating any large system. Techies are adept at automating repetitive tasks in their environments with scripts, but there is comparatively little to help support the people side of sustaining systems.

Origin

A great deal of work in operations comes in the form of mission critical events—major upgrades, database migrations, hardware refreshes, incident responses—activities which are complex, involve many interdependent systems, depend on people both internal and external to the team carrying them out, and which are only allowed to disrupt the underlying services minimally, if at all. These events are conducted under time pressure with incomplete information, and the consequences of getting them wrong are significant.

Doing such work well requires planning. The usual options—Word documents, wiki pages, ISO 9000 quality manuals—get lost to time, drift out of date, or sit on shelves unread. What is needed is a way to write down a procedure in a form that is both flexible and easy to use and clear enough that a team can coordinate their actions against it in real time.

In our original solution to this problem circa 2004, procedures were described on paper using a rigid hierarchical style. A procedure was grouped into «section»; each section contained a sequence of «steps» carried out in order; within each step, one or more «named people» performed individual «tasks» concurrently. Progress waited on every task in the current step being complete before moving to the next.

The consistent nomenclature meant any piece of work in a procedure could be given a fully-qualified name. "Section III, Step 2, Fozzie's Task d", for example. People on a conference bridge, in a datacenter, or working remotely could share a common sense of exactly where they were. The hierarchy defined what could happen concurrently and what had to happen in sequence, and that is the essence of the plan being followed.

This worked well and was employed at numerous organizations. The approach had some limitations, however.

Challenges Scaling Up

The fixed four-level structure was found wanting when we tried to articulate the specific actions within a task, or when we needed to aggregate procedures at larger scales.

The first problem was depth. When a single person needed to carry out many small actions as part of one task, there was nowhere to put them. A fifth level was considered; «detail», perhaps. But why did we need a name for this level at all?

The second problem was composition. A senior manager looking at a change event might consider there are four things to do. The engineer assigned the third of them knows that her piece alone requires eight steps of her own. Both are right. Different levels of an organization consider different magnitudes of action to be "a step". The rigid hierarchy had no way to accommodate this.

The same patterns keep appearing at different magnitudes and different levels of nesting.

The Nature of Procedures

Grappling with how to represent work at different scales led to the realization that the structure of work at all such scales is the same. A step is a unit that activates; depends on zero, one, or more children; performs a task; and durably records that it has been done. That same unit, recursively composed, is enough to describe any procedure, be it a four-point executive summary or thousand-step implementation plan.

This is the idea behind Technique. The language is a way to write down procedures—from a short checklist to a large engineering process—using one consistent structure. Because that structure is the same at every scale, the same language describes the work of an individual, a team, and an organization.

The theoretical basis of the language is set out in the "Turtles" paper, which defines the Fundamental Unit of Procedure, analyses the complexity of procedures, and establishes the scale-independent unit for complexity called the turtle, symbol p.

Implementation

The original paper documents worked well enough, but what we wanted was a program to help run the events they described—to show a team where they were in a procedure in real time, hold the dependencies between tasks, and record each step as its result came in.

Explorations

It began as a graphical user interface written in GTK and Java, which was promising enough for the Linux desktop in the mid 2000s. We were able to display the status of each participant, show progress, and give an overview of the procedure as a whole. Not a bad start.

But how to express the code of these procedures? Our first attempts had the procedure documents written in XML. At first this seemed like a natural enough fit; text-based with defined hierarchical structure. But for all their supposed extensibility, XML schemas are rigid and hard to work with, and XML is extremely unfriendly for humans to write.

Then we tried figuring out if a strong embedding in a general-purpose programming language would work. Experimented with quite a few, and came up with working prototypes in Python, Lua, Haskell, and more. Nothing really suited. At the end of the day they're all programming languages; no non-programmer is going to be effective trying to write their recipes or business processes in that form. The machinery of setting up imports, using the language's syntax, trying to build and run programs are all barriers. Code (for all that it's written in text/plain) is not text.

There is no shortage of scripting tools, policy engines, and enterprise business process management software out there. Occasionally we'd find a useful feature or interesting approach, but never in a form that was simple, versionable, and focused on the intersection of both humans writing procedures and running them.

Domain Specific

Eventually we threw our hands up and said "why don't we just create our own language that does what we want?" Rather than an embedding in an existing programming language we designed a domain-specific language of our own.

A parser was written in Haskell circa 2018, and we were well on the way to an evaluator to interpret the programs when we realized that this language—now named Technique and labeled v0—had entirely missed the point. It was unpleasant to write, cumbersome to extend, and worst of all, all the theoretical analysis that underpinned the work had been left behind.

Language for Humans

Instructions written by humans to be read by humans don't look like program code. All you want is to be able to write down the steps you need to do. So we went back to the drawing board and crafted something that would actually be usable, by people.

We want it to be able to describe what to do, step by step, in a form that is readable by both humans and machines. It needs to be simple, in the sense that anyone glancing at a document written in Technique should be able to more or less instantly understand what it is about. There must be mechanisms for composition, so that one procedure can invoke another. And the language needs to be suitable to describe work in a wide range of settings, not just systems and engineering.

This new attempt at a custom language is in pursuit of these goals. Started in late 2023, Technique v1 is a complete rework, written in Rust, with growing editor support and tooling.

Technique is not a programming language. It has returned to the roots of what a procedure is, and is truly domain-specific. Perhaps it is a procedures language.