You're viewing the readable version of this site. The interactive extras (search, diagrams, read-aloud) need JavaScript and a current browser. Enable JavaScript; if it is already enabled, update your browser.

Learn · Learning Bazel

seedling

The File That Isn't There

A Go file reads a data file sitting right next to it. Your own compiler is fine with that. This build refuses — and the refusal is the whole point of the book.

bazel, sandboxing, hermeticity, go, build-systems, learn

By the end you will have written a build that fails, looked inside the sandbox where it failed, and fixed it with one line. You will be able to explain why that failure is a feature, and why the same mistake in Make would have shipped.

The setup

Chapter one's Makefile read a file it never declared. Nothing stopped it, so the build went quietly wrong. Let us make the same mistake on purpose, in Go, under a system that checks.

Go has a facility for baking a file into a binary at compile time. Point it at a file and its contents become a string constant:

server/greeting/greeting.go

package greeting

import (
	_ "embed"
	"fmt"
)

// The name comes from a file next to this source. go build finds it,
// because go build can read anything in the directory.
//
//go:embed name.txt
var name string

// Greet builds the message the server prints.
func Greet() string {
	return fmt.Sprintf("Hello %s", name)
}

The //go:embed directive reads name.txt at compile time. The file sits in the same directory as this source.

The declaration is unchanged from chapter three — it lists the Go source, and nothing else:

server/greeting/BUILD.bazel

load("@rules_go//go:def.bzl", "go_library")

go_library(
    name = "greeting",
    srcs = ["greeting.go"],
    importpath = "example.com/greet/server/greeting",
    visibility = ["//server:__subpackages__"],
)

srcs names greeting.go. There is no mention of name.txt anywhere.

This is precisely chapter one's mistake, transplanted. The compile step reads two files. One of them is declared.

Your own tools are fine with this

Before running the build, check what Go itself thinks:

$ go build -o /tmp/greet ./server/
$ /tmp/greet
Hello world

The native toolchain. It compiles, links, and runs, because go build reads whatever is in the directory.

No error. No warning. The binary works. If you stopped here you would have no reason to suspect anything, which is exactly how chapter one's bug ships.

The build refuses

Now the same source, same compiler version, under the build system:

$ bazel build //server
ERROR: GoCompilePkg server/greeting/greeting.a failed: (Exit 1)
compilepkg: greeting.go:11:12: could not embed name.txt: no matching files found
ERROR: Build did NOT complete successfully

The build fails during compilation. The error is not about permissions or paths — the compiler genuinely could not find a file that is sitting in your working directory.

Read that error again with chapter one in mind. no matching files found — for a file that exists, spelled correctly, in the right directory.

The compiler is not confused. It is telling the truth about the place it was run.

Look inside

The build did not run the compiler in your directory. It ran it in a sandbox: a temporary directory containing exactly the files that target declared, and nothing else. Ask for that sandbox to be kept after the failure:

$ bazel build //server --sandbox_debug
...
$ ls <sandbox>/server/greeting/
greeting.go

--sandbox_debug keeps the sandbox instead of tearing it down, so you can look at what the compiler actually saw.

One file.

name.txt is two directories away on your real disk and it is not here, because nothing declared it. The compiler did not fail to find a file it should have found — it correctly reported that the file was absent from the only world it could see.

This is the enforcement chapter one was missing. Make could not tell the difference between a declared input and an undeclared one, because its recipes run in your directory where both are present. Run the step somewhere only the declared inputs exist, and the difference becomes impossible to ignore: it is the difference between a build that works and one that stops.

Chapter one's cook added salt nobody wrote on the recipe card, and nobody noticed. This kitchen hands the cook a tray containing only the ingredients the card lists. If the card forgot the salt, the cook cannot quietly add it — there is no salt on the tray, and the dish stops right there.

The fix is one line

Tell the build about the file:

server/greeting/BUILD.bazel

load("@rules_go//go:def.bzl", "go_library")

go_library(
    name = "greeting",
    srcs = ["greeting.go"],
    # go:embed reads this file, so the build has to be told about it.
    # Without this line the compiler cannot see it, even though it sits
    # right next to greeting.go on disk.
    embedsrcs = ["name.txt"],
    importpath = "example.com/greet/server/greeting",
    visibility = ["//server:__subpackages__"],
)

The same declaration with embedsrcs added. That attribute is how a Go library says "the compiler will read these too."

$ bazel build //...
INFO: Build completed successfully, 38 total actions

$ bazel run //server
Hello world

With the input declared, the sandbox contains it, and the build proceeds.

One line of configuration. That is the entire cost of the guarantee.

The other half: a file that is there

A missing file is the loud version. Here is the quiet one, and it is the reason the sandbox scrubs more than the filesystem.

Suppose a build step stamps who built it. The shell way:

$ cat stamp.sh
echo "built-by=${BUILD_USER:-unknown}"

$ BUILD_USER=alice sh stamp.sh
built-by=alice

A recipe that reads an environment variable. Nothing declares BUILD_USER — a shell command inherits whatever the caller happens to have set.

That works, and it is a trap. The output depends on a value that appears nowhere in the repository. Run it yourself and get alice; a colleague runs the identical command on the identical commit and gets their own name, or unknown, and neither of you has any reason to suspect the other saw something different.

Now the same command as a build step:

genrule(
    name = "build_tag",
    outs = ["build_tag.txt"],
    cmd = "echo \"built-by=$${BUILD_USER:-unknown}\" > $@",
)

The declaration. The command is identical; only who runs it has changed.

$ BUILD_USER=alice bazel build //stamp:build_tag
$ cat bazel-bin/stamp/build_tag.txt
built-by=unknown

The variable is set in the shell that invoked the build, and the build does not see it.

The sandbox is not only a directory. It is also an environment, and by default it contains almost nothing. Your shell's variables are exactly as undeclared as name.txt was, and they are refused the same way — except that here the refusal is silent, because an unset variable is a legal thing for a command to encounter.

Declaring an input that is not a file

If the stamp genuinely needs that value, say so:

$ BUILD_USER=alice bazel build //stamp:build_tag --action_env=BUILD_USER
$ cat bazel-bin/stamp/build_tag.txt
built-by=alice

--action_env declares the variable, and it reaches the command.

And now the interesting part. Change the value and rebuild:

$ BUILD_USER=bob bazel build //stamp:build_tag --action_env=BUILD_USER
$ cat bazel-bin/stamp/build_tag.txt
built-by=bob

A declared variable is part of the action's key, so changing it invalidates the result exactly as editing a source file would.

"Input" does not mean "file." It means anything the step reads that could change the answer — a source file, a compiler, a flag, an environment variable. Declared, it joins the key and the build tracks it. Undeclared, it is not merely untracked: it is unreachable.

That is why the earlier chapters kept insisting the compiler is an input. It was never a special case. It is the same rule applied to a thing that does not look like a file, and this is the same rule again applied to a thing that is not even on disk.

Worth reaching for --action_env sparingly. Every declared variable is a way for one machine to differ from another; the point is not that declaring is free, but that it is visible.

Why this is worth the trouble

It is fair to object that the failure was annoying and Go managed fine. So it is worth being clear about what was bought.

The undeclared read was already a bug. It was a bug in chapter one and it is a bug here — a step whose real inputs differ from its declared ones, which means the build cannot know when its output is stale. Go did not fix that bug; it just did not mention it.

What changed is when you find out. Under Make you find out months later, on someone else's machine, as a wrong answer with no error message. Here you find out immediately, on your own machine, while you still remember what you were doing — with an error naming the exact file.

This is why the earlier chapters insisted the compiler is an input. A sandbox containing only your declared sources would be useless if the compiler were whatever happened to be installed. Both halves are needed for the same guarantee.

And it composes. Every action in the graph gets the same treatment, so the property holds for the whole build rather than the parts someone remembered to check. That is what makes the next part possible: once every step's inputs are genuinely known, the build can start reusing work across machines, because it can tell when two pieces of work are the same.

Each build step runs in a sandbox containing only its declared inputs. An undeclared read that native tooling accepts fails here immediately, naming the file. The failure was always a bug; the sandbox only changes whether you learn about it now or months from now. --sandbox_debug keeps the sandbox so you can see exactly what a step could reach.

Try this in your own repository

Run a build step in an empty room. Copy one step's declared inputs into a fresh directory and run just that step there. Whatever fails is an undeclared input, and the list you get is usually longer than expected — configuration files, a lockfile, a certificate, an environment variable set by a shell profile.

Audit what your CI environment provides. List every environment variable your build reads. For each, ask what happens if it is unset, and whether anybody would notice. A variable that silently changes the output is the failure in this chapter, waiting.

What you can now do

Diagnose a build failure caused by an undeclared input, inspect the sandbox to see what a step could actually reach, and declare data files a compiler reads. Part one is complete: you have a build whose graph is enforced rather than promised. Part two is about the language you write that graph in.