You're viewing the readable version of this site. The interactive extras (search, diagrams, read-aloud) need JavaScript and a current browser. Enable JavaScript; if it is already enabled, update your browser.

Learn · Learning Bazel

seedling

One Schema, Two Languages

A server and a client that cannot disagree about their shared contract, because the contract is one file and both of them are generated from it.

bazel, code-generation, cross-language, go, typescript, learn

By the end you will have a schema file that generates a Go source and a TypeScript source, both consumed by real code, and you will have changed one line and seen both languages change together.

The contract

One file, in no language in particular:

api/greeting.schema

# The shape of a greeting, in one place.
# Both languages generate their own view of this file.
greeting_prefix Hello
max_name_length 32

The shape of a greeting. Not Go, not TypeScript — the thing both of them need to agree about.

And a small program that turns it into source for either language:

api/generate.py

"""Turns greeting.schema into source for one language.

Reads exactly the files it is told about and writes exactly the file it
is told to write. That is the whole contract a build action needs.
"""

import sys

def parse(path):
    fields = {}
    with open(path) as handle:
        for line in handle:
            line = line.strip()
            if not line or line.startswith("#"):
                continue
            key, value = line.split(None, 1)
            fields[key] = value
    return fields

def go_source(fields):
    return (
        "// Code generated from greeting.schema. DO NOT EDIT.\n"
        "package api\n\n"
        'const GreetingPrefix = "%s"\n\n'
        "const MaxNameLength = %s\n" % (
            fields["greeting_prefix"],
            fields["max_name_length"],
        )
    )

def ts_source(fields):
    return (
        "// Code generated from greeting.schema. DO NOT EDIT.\n"
        'export const greetingPrefix = "%s";\n\n'
        "export const maxNameLength = %s;\n" % (
            fields["greeting_prefix"],
            fields["max_name_length"],
        )
    )

def main():
    schema, language, out = sys.argv[1], sys.argv[2], sys.argv[3]
    fields = parse(schema)
    text = go_source(fields) if language == "go" else ts_source(fields)
    with open(out, "w") as handle:
        handle.write(text)

if __name__ == "__main__":
    main()

It reads the files it is told about and writes the file it is told to write. That is the whole contract a build step needs.

Putting generation in the graph

The important move is that generation is not a script somebody runs before building. It is a build step, with declared inputs and declared outputs, exactly like a compile:

api/BUILD.bazel

load("@aspect_rules_ts//ts:defs.bzl", "ts_config", "ts_project")
load("@rules_go//go:def.bzl", "go_library")

# One schema, two generated sources. Each genrule declares its inputs
# and its output, so both languages depend on the schema through the
# graph rather than through a convention someone has to remember.

genrule(
    name = "greeting_go",
    srcs = ["greeting.schema"],
    outs = ["greeting.go"],
    cmd = "$(PYTHON3) $(location generate.py) $< go $@",
    toolchains = ["@rules_python//python:current_py_toolchain"],
    tools = ["generate.py"],
)

genrule(
    name = "greeting_ts",
    srcs = ["greeting.schema"],
    outs = ["greeting.ts"],
    cmd = "$(PYTHON3) $(location generate.py) $< ts $@",
    toolchains = ["@rules_python//python:current_py_toolchain"],
    tools = ["generate.py"],
)

ts_config(
    name = "tsconfig",
    src = "tsconfig.json",
)

go_library(
    name = "api",
    srcs = ["greeting.go"],
    importpath = "example.com/greet/api",
    visibility = ["//visibility:public"],
)

ts_project(
    name = "api_ts",
    srcs = [":greeting_ts"],
    declaration = True,
    transpiler = "tsc",
    tsconfig = ":tsconfig",
    visibility = ["//visibility:public"],
)

Two generation steps and two libraries. Each names the schema as an input and its generated file as an output.

Three details in that file are worth pausing on, because each is an earlier chapter arriving again with higher stakes.

The generator's interpreter is pinned. Invoking the script directly fails — it is not executable, and even if it were, the interpreter would be whatever the machine happens to have. Running it under a declared toolchain makes the tool an input, which is the same rule chapter two applied to the compilers.

The generated package has its own compiler configuration. Reusing the web one fails, because its implicit file-matching does not reach a sibling directory. That is also the more honest structure: generated code has different needs from hand-written code.

Visibility comes up again. The shared compiler settings were scoped to one subtree in chapter six, and a new consumer is refused until the list names it. Same choice as before, now with a real dependency behind it.

Both languages consume it

The Go side takes its constant from the generated package:

server/greeting/greeting.go

package greeting

import (
	_ "embed"
	"fmt"

	"example.com/greet/api"
	"golang.org/x/text/cases"
	"golang.org/x/text/language"
)

//go:embed name.txt
var name string

// Greet builds the message, taking the prefix from the shared schema
// rather than from a string literal that could drift.
func Greet() string {
	caser := cases.Title(language.English)
	return fmt.Sprintf("%s %s", api.GreetingPrefix, caser.String(name))
}

The prefix is no longer a string literal in this file. It comes from the schema.

And so does the TypeScript side:

web/greeting/greeting.ts

import { capitalCase } from 'change-case';

import { greetingPrefix } from '../../api/greeting.js';

export function greet(name: string): string {
  return `${greetingPrefix} ${capitalCase(name)}`;
}

The same constant, in the other language, from the same source of truth.

Neither file contains the word Hello any more. There is exactly one place that word lives.

The demonstration

Both binaries, before:

$ bazel run //server
Hello World

$ bazel run //web:greet
Hello World

Server and client agree, which proves nothing yet — they agreed before too.

Now change one line in the schema:

$ sed -i '' 's/greeting_prefix Hello/greeting_prefix Namaste/' api/greeting.schema

One word, in one file, in no programming language.

And rebuild:

$ bazel build //...
INFO: 13 processes: 4 action cache hit, 4 internal, 9 darwin-sandbox.

$ bazel run //server
Namaste World

$ bazel run //web:greet
Namaste World

Both languages followed. Nothing was regenerated by hand, nothing was kept in sync by a person, and no second file was edited.

This is the thing neither go build nor tsc can do, and not because of any deficiency in either. A tool that owns one language cannot propagate a change into another language, because the other language is not in its world. A tool that owns the graph can — and the propagation is not a feature somebody added, it is what the graph already meant.

Why the cascade is exact

Look at the thirteen processes. The build did not rebuild everything, and it did not guess. The schema's key changed, so both generators reran; their outputs changed, so both compilers reran; the binaries relinked. Everything else kept its key and its answer.

Chapter three showed rdeps answering "what breaks if I change this." This is the same question answered by doing rather than reporting — the set of things that reran is exactly the set rdeps would have named.

One recipe card in the kitchen, and two cooks who each keep their own translated copy. Every time the card changes, somebody has to update both copies and remember to. Now the copies are printed automatically from the card whenever it changes — and if the printer breaks, nobody eats, which is much better than two cooks quietly making different dishes.

What the schema does not guarantee

The claim so far is that the two languages cannot disagree. That is true, and it is narrower than it sounds. Worth pinning down before you build something on it.

Add a field to the shared contract:

greeting_prefix Hello
max_name_length 32
farewell Goodbye

A third line in the schema, the way a real contract grows.

Rebuild:

$ bazel build //...
INFO: Build completed successfully, 141 total actions

$ grep -c farewell bazel-bin/api/greeting.go bazel-bin/api/greeting.ts
bazel-bin/api/greeting.go:0
bazel-bin/api/greeting.ts:0

Success, everywhere. And nothing anywhere emitted a farewell.

The generator knows two fields because it was written to know two fields. A third one is not an error to it — it is a line it does not recognize and quietly skips. Both languages agree perfectly about a contract that has silently lost a third of itself.

The build guarantees the two languages see the same generated output. It does not guarantee that output reflects the whole input — that is the generator's job, and the generator is ordinary code with ordinary bugs.

So the pattern moves the problem rather than dissolving it. What was "two hand-written copies that drift" becomes "one generator that might be incomplete" — a real improvement, because there is now one place to fix and one place to test, but not the same thing as safety.

The practical response is to make the generator loud. Failing on an unrecognised field costs three lines and converts this whole class of silence into a build error:

KNOWN = {"greeting_prefix", "max_name_length"}
unknown = set(fields) - KNOWN
if unknown:
    raise SystemExit("unknown schema fields: %s" % sorted(unknown))

The version worth writing. An unknown key is a mistake somebody made, not input to ignore.

This is the same instinct as the lifecycle-hook refusal in chapter ten and the visibility list in chapter six: when a tool meets something it was not told about, silence is the expensive option.

What this pattern is really for

The example is a greeting prefix, which is small on purpose. The shape generalizes to the things that actually cause cross-language bugs: wire formats, error codes, enumerations, permission names, API routes, database column names.

Each of those is a place where two languages hold the same knowledge, and each is a place where the second copy drifts. The pattern is always the same: one declarative source, a generator per language, both wired into the graph so the build refuses to produce a stale half.

This is the argument for a single repository, made concrete. Splitting server and client into separate repositories means the contract crosses a boundary the build cannot see — and then you are back to keeping two copies in agreement by discipline.

Put a shared contract in one declarative file, generate a source per language from it as ordinary build steps with declared inputs and outputs, and both languages become incapable of disagreeing. A tool the build runs is an input, so pin its interpreter. Generated packages earn their own compiler configuration. One line changed in one file makes two binaries change together.

Try this in your own repository

Find a contract that exists twice. An error-code list, a set of permission names, a wire format, a database column mapping. Check whether the copies actually agree right now, today. They often do not, and the drift is usually old.

Then design the generator you would need. Not to build it — to find out what it would have to know. The fields, the naming conventions per language, what an unknown field should do. That last question is the one this chapter says most generators get wrong.

What you can now do

Generate code from a shared schema as part of the build, in more than one language, and explain why the two sides cannot drift. That is the argument this book was making. What remains is living with the result.