Learn · Learning Bazel
seedling
Macros Expand to Nothing New
Your build files have started repeating themselves. The fix is a function — and the reason to reach for it last is that you can prove it adds no power at all.
By the end you will have written a macro, replaced two hand-written declarations with it, and proved the build is unchanged. You will also have looked underneath a rule to see the actual commands it produces — which turns out to explain several things from earlier chapters.
The repetition
Two commands now live in the workspace, and their declarations have converged:
server/BUILD.bazel
load("@rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "server_lib",
srcs = ["main.go"],
importpath = "example.com/greet/server",
deps = ["//server/greeting"],
)
go_binary(
name = "server",
embed = [":server_lib"],
)The server: a library from main.go, and a binary embedding it.
cmd/hello/BUILD.bazel
load("@rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "hello_lib",
srcs = ["main.go"],
importpath = "example.com/greet/cmd/hello",
deps = ["//server/greeting"],
)
go_binary(
name = "hello",
embed = [":hello_lib"],
)The command-line tool. Same shape; three words differ.
Every Go command in this repository will need that pair. That is real repetition — not a contrived example, just the shape the language and the rules imply.
The language you write build files in
Build files are not data. They are programs, in a language that looks like Python and deliberately is not:
tools/go_command.bzl
"""A macro for the library-plus-binary pair every Go command needs."""
load("@rules_go//go:def.bzl", "go_binary", "go_library")
def go_command(name, importpath, srcs = ["main.go"], deps = []):
"""Declares a go_library and the go_binary that embeds it.
This is a macro, not a rule. It runs while the build file is being
read and expands into the two targets you would have written by
hand. Nothing new exists afterwards that could not have been typed
out — which is exactly why a macro is cheap, and also why it can
never do anything the underlying rules cannot.
Args:
name: name of the binary; the library becomes <name>_lib.
importpath: Go import path for the library.
srcs: Go sources, defaulting to the usual single main.go.
deps: libraries the command depends on.
"""
go_library(
name = name + "_lib",
srcs = srcs,
importpath = importpath,
deps = deps,
)
go_binary(
name = name,
embed = [":" + name + "_lib"],
)A function taking a name, an import path, and dependencies, declaring two targets.
If you know Python, there is nothing to learn here — a function, default arguments, string concatenation. What matters is what the language cannot do.
It cannot open a file. It cannot read an environment variable, make a network call, or ask the clock what time it is. It cannot loop forever: every loop walks a collection that already exists, so evaluation always terminates.
Those are not missing features. They are the reason a build file can be evaluated on your laptop and on a build machine and produce the same answer — which is the same guarantee the sandbox gave us for actions, applied one level up to the configuration itself.
The language is Starlark, designed and implemented in Java by Laurent Le Brun, Dmitry Lomov, Jon Brandvin and Damien Martin-Guillerez, with a Go implementation by Alan Donovan and Jay Conrod whose scanner derives from one by Russ Cox. The specification credits them by name, which is rarer than it should be.
Using it
The two build files collapse:
server/BUILD.bazel
load("//tools:go_command.bzl", "go_command")
go_command(
name = "server",
importpath = "example.com/greet/server",
deps = ["//server/greeting"],
)The server, now three lines. load() brings the function in by label, exactly like any other dependency.
cmd/hello/BUILD.bazel
load("//tools:go_command.bzl", "go_command")
go_command(
name = "hello",
importpath = "example.com/greet/cmd/hello",
deps = ["//server/greeting"],
)And the tool. The repetition is gone.
The .bzl file needs a BUILD file beside it so its package exists, but no target of its own — build logic is loaded directly rather than built:
tools/BUILD.bazel
# This package holds build logic rather than code to build. The .bzl
# file needs no target of its own — Bazel loads it directly — but the
# package needs a BUILD file to exist at all.A package that holds logic rather than code. The empty file is doing real work.
Prove it changed nothing
Here is the part worth doing. Ask both versions of the workspace what targets they contain:
$ bazel query '//server:all + //cmd/hello:all'
//cmd/hello:hello
//cmd/hello:hello_lib
//server:server
//server:server_libThe targets produced by the macro, and by the hand-written version it replaced. Identical.
Same four names. Now go a level deeper and compare the actual commands each version will run — the aquery output, filtered to the kinds of work involved:
$ bazel aquery '//server' | grep Mnemonic | sort > after.txt
# ...and the same in the hand-written version, into before.txt
$ diff before.txt after.txt
$ echo $?
0Comparing the two workspaces' action mnemonics — six of them, before and after. An empty diff: the macro produces exactly the build the hand-written declarations produced.
A macro is pure expansion. It runs while the build file is being read and produces the targets you would have typed by hand — nothing more. That is why it is cheap, and it is also why it can never do anything the underlying rules cannot.
What a rule actually is
Since we are looking underneath, this is the moment to answer a question that has been open since chapter two: what is a rule?
Ask what a single Go library actually does:
$ bazel aquery '//server/greeting'
action 'GoCompilePkg server/greeting/greeting.a'
Mnemonic: GoCompilePkg
Target: //server/greeting:greeting
Inputs: [...gofmt, ...compile, ...link, ...vet,
server/greeting/greeting.go, server/greeting/name.txt]
Outputs: [bazel-out/.../greeting.a, bazel-out/.../greeting.x]The work behind one go_library. One action; the input list is long — every compiler tool plus your sources — so it is abbreviated here with an ellipsis. Run it yourself to see the whole thing.
A rule is a function that produces actions: commands with declared inputs and declared outputs. That listing is the sandbox from chapter four, written down — including name.txt, which is there because you declared it in chapter five.
Ask the same of the binary and you get five actions you never wrote. The TypeScript project produces three, one of them a validator — which is the target that produced the tsconfig mismatch error back in chapter three. Nothing here is hidden machinery. It is all actions, and you can look at any of them.
What the macro took away
The macro is well behaved. Misspell an argument and it says so, with a suggestion:
$ bazel build //server
Error: go_command() got unexpected keyword argument: importpaths
(did you mean 'importpath'?)Macros are functions, so a wrong keyword is a wrong keyword. This is a good error.
Pass an attribute the macro does not forward — visibility, say — and you get the same treatment rather than silence. So the cost is not sloppiness. It is somewhere else.
Ask the build where a target is declared:
$ bazel query --output=location 'kind(go_binary, //server:all)'
server/BUILD.bazel:10:10: go_binary rule //server:serverThe hand-written version. Line 10 is where go_binary appears, and that is what the answer says.
Now the same question of the macro version:
$ bazel query --output=location 'kind(go_binary, //server:all)'
server/BUILD.bazel:3:11: go_binary rule //server:serverLine 3. Open the file and there is no go_binary there — line 3 is the go_command call.
The answer is not wrong. That target really does originate at line 3 — it is where the function that declares it was called. But a reader who follows the pointer finds a name they have to go look up, and the rule they were told about is not written anywhere in the package.
A macro keeps the graph truthful and makes the file less so. The build still knows exactly what exists; the person reading the package no longer does, without a second lookup. That trade is worth making for real repetition and not worth making for two lines.
Why macros go last
Given that a macro adds no power, when should you write one?
Later than you want to. A hand-written declaration says what it is. A macro says what it is called, and to know what it means you have to go read the function — which is fine at one call site and expensive at forty, when the function has grown three optional arguments and a conditional.
The rule of thumb that survives contact with real repositories: write the declarations out until the repetition is genuinely painful and genuinely uniform, then extract exactly the macro that removes it. The repetition here was two files with three differing words, which is about the right threshold.
A macro is a rubber stamp. It saves you writing the same thing twice, and it can never stamp anything you could not have written by hand. If someone hands you a stamp you have never seen, you have to go look at it before you know what your page says.
Build files are programs in a language that cannot do I/O and cannot loop forever — which is what makes evaluating them reproducible. A macro is a function that expands to targets you could have typed, and you can prove it: identical target sets, identical actions. Rules produce actions, and aquery shows them, including the inputs that make up each sandbox.
Try this in your own repository
Find your most-copied block of build configuration. Count how many times it appears and how many words differ between copies. Then decide honestly whether a function would help — remembering that the reader pays a lookup for every call site, and that two copies rarely justify it.
Read a macro somebody else wrote. Preferably one you use regularly. Work out what targets it produces without running anything. If you cannot, that is the cost this chapter describes, measured on your own codebase rather than an example.
What you can now do
Write a macro to remove real repetition, prove it changed nothing, and look underneath any target to see the commands it will actually run. The next chapter hands your build files to a generator — which promptly collides with the macro you just wrote.