Learn · Interactive trip
growing
The Art of Not Looking
How search systems find the right answer without examining everything.
Two readers now depend on the same search box, and they fail differently. A person types badly, scans ten results, and judges the box on how it feels. A machine deciding what to read before it answers is handed the top results as its evidence — it cannot skim, it cannot tell that the fourth one is off-topic, and it reasons from whatever it was given.
This series starts at the index in the back of a book and does not stop at the analogy. Every chapter opens on something you already understand, hands you a machine to break your intuition against, and then gives you the algorithm in full, the recurrence or the invariant, the proof, and the cost in time and space — followed by the constant factors that decide which algorithm actually wins. You will derive the ranking function rather than memorize it, prove why a search engine is allowed to skip work it has not looked at, encode a set within two bits per element of its information-theoretic floor, and measure a latency distribution without lying to yourself. No background assumed beyond having used a search bar; nothing is skipped on the way down.
And you will meet the people. Almost nothing in this book was invented for search: the ruler for "close enough" came out of error-correcting codes and was rediscovered by people aligning proteins; the encoding that stores a set within two bits of optimal came from information theory; the law governing how fast a vocabulary grows came from linguists counting words. Each chapter ends with the person who found the thing, the year, and what they were actually trying to do — because a result understood through the eyes of whoever found it keeps its shape in your memory, and a formula handed over cold does not.
The first two acts stay with text and structured records. The final acts, Beyond Words, begin with captions, visible text, and transcripts, then add exact and perceptual media identity, visual representations, approximate neighbors, multimodal fusion, temporal video search, evaluation, and authorization. The extension does not call every relation “similarity”: same bytes, transformed copies, repeated objects, and related meanings remain different questions with different evidence.
Chapters
The Art of Not Looking
- The Back of the Book — turn documents inside out so a query begins with a term instead of rereading every page.
- The Index Inside the Index — choose the dictionary that locates each term's postings without pretending one representation wins every workload.
- Two Machines That Must Agree — give indexing and querying one analyzer contract.
- Two Lists, One Walk — intersect sorted evidence with work shaped by the actual lists.
- Where the Words Are — derive phrase search as positional intersection.
Which of the Matches
- Rare Words Are Evidence — derive rarity, saturation, and length normalization instead of memorizing BM25.
- Not All Fields Are Equal — combine field evidence before the shared saturation law.
- More Than One Key — separate eligibility from relevance.
- Ranking Is an Algebra — name the laws that make explanation and skipping possible.
Never Touch What Cannot Win
- The Score to Beat — derive the upper-bound invariant behind exact top-k pruning.
- The Arithmetic of Skipping — learn when a finer proof refunds more work than it consumes.
- Packing the Suitcase — compress gaps by omitting information ordering already implies.
- Rank, Select, and Almost No Bits — navigate a set near its information floor.
- The Index Is a Network Object — separate wire bytes, decode work, resident memory, and execution layout.
Humans Type Badly
- A Ruler for Almost — derive edit distance as a shortest-path recurrence.
- Subsequence, Not Distance — distinguish finder intent from spelling repair.
- Dictionaries That Fit — minimize a finite dictionary by merging identical futures.
- The Cost of a Query Is Not Its Length — bound work by the reachable frontier, not characters typed.
- One Word, One Vote — make expansion evidence independent of traversal order.
Around the Algorithm
- The Average Is a Lie — measure distributions, strata, and uncertainty.
- Latency Is a Feature — turn query timing into an explicit interaction policy.
- Who Owns the Next Key? — resolve keyboard ownership without breaking platform behavior.
- Only the Current Question Can Answer — give async work generations, ownership, and settlement.
- How Do You Know It Got Better? — make improvement claims falsifiable.
When the Reader Is a Machine
- The Result List Is a Context Window — allocate a bounded evidence budget.
- Wrong Is Worse Than Slow — price false evidence, calibration, and abstention.
- Lexical, Vector, and the Deterministic Floor — place BM25, SPLADE, late interaction, ANN, and reranking in one honest retrieval spectrum.
Pictures Are Not Words
- Every Medium Can Speak Text — search authored and extracted words while preserving evidence spans.
- Same Bytes, Same Scene, Same Meaning — keep identity, duplication, instance, and semantics distinct.
- A Picture Becomes a Point — define the representation before its neighbors.
- Neighbors Without Looking Everywhere — spend measured recall to avoid an exhaustive vector scan.
Time and Senses
- One Query, Several Senses — fuse typed eligible providers without adding incomparable scores.
- The Answer Happens at 2:17 — return the interval where video evidence occurs.
- A Good Match Depends on the Question — let the requested relation choose the judgments and metric.
- What the Index Must Not Reveal — preserve authorization through every derived artifact and observable stage.
How to read this
One corpus runs through the whole book. The index you build in the first chapter is the one intersected, pruned, encoded, measured, and budgeted in every chapter after it, so the machines compose and the numbers are yours rather than quoted. Read in order, or enter anywhere — every term is defined in the margin where it first appears and collected in the glossary at the end.
Every chapter carries one law, one machine you drive with your own input, the algorithm written out in full, its cost in time and space, the person who found it and how they got there, and one honest statement of where the technique is the wrong choice. Where a chapter claims a speedup it gives you both the bound and the measurement, and where those two disagree about which algorithm to pick, the disagreement is the lesson.
Chapters are undated and revised in place; the garden status is the honesty mechanism. Nothing here tracks you.
The retrieval route turns each chapter's central insight into a one-day, one-week, and one-month reconstruction prompt. It is a practice page, not a summary: attempt each prompt before reopening the chapter.