Learn · Humans Type Badly
budding
A Ruler for Almost
Edit distance turns “nearly the same” into an optimization problem—but the ruler is only as honest as the costs engraved on it.
“The string-to-string correction problem is to determine the distance between two strings as measured by the minimum cost sequence of edit operations.”
— Robert A. Wagner and Michael J. Fischer, The String-to-String Correction Problem
This is the fifteenth chapter in a book about search from first principles. You will define an edit policy, derive the Wagner–Fischer recurrence from the last operation, prove its optimal substructure, reduce its storage to two rows, model gap opening with finite state, stop outside a bounded band, and see why unit costs admit a bit-vector representation. You will then separate candidate generation, ranking, and correction decisions. The next chapter replaces distance minimization with a positional subsequence objective.
Equality has no notion of near
An exact dictionary answers whether appol is present. It cannot say that apple is nearby. To ask that question, first state the allowed operations:
- delete one source symbol;
- insert one target symbol; or
- substitute one source symbol for one target symbol.
Give every operation unit cost. The edit distance is the minimum cost of any operation sequence that turns into . For appol and apple, one substitution and one insertion suffice, so . No single permitted edit changes both the fourth character and the length, so .
The qualifier “unit cost” is part of the definition, not an implementation detail. A keyboard-neighbor substitution might cost less than an arbitrary one. Opening a long missing span might cost more than extending it. Those policies define different rulers and can rank candidates differently.
Prefixes expose the recurrence
Let and be two strings. Define
The empty-prefix boundaries are forced:
For nonempty prefixes, inspect the last operation of an optimal correction:
Here is 1 when proposition is true and 0 otherwise.
Dynamic programming — solve overlapping subproblems once, then compose their answers. Richard Bellman named the method; Wagner and Fischer applied this prefix recurrence to string correction. Read the primary paper.
Why the recurrence is exact
The recurrence is not merely plausible. Take an optimal sequence correcting the first source symbols to the first target symbols. Its final action must be one of three kinds.
If it deletes , everything before that final deletion must optimally correct to . If a cheaper prefix correction existed, substituting it would make the whole sequence cheaper, contradicting optimality. The insertion case is symmetric. If the final action aligns with , the preceding actions optimally correct both shorter prefixes, and the final alignment costs zero for equality or one for substitution.
Thus every optimal sequence appears in one branch and costs at least the minimum. Conversely, append the named final action to an optimal solution for any branch; each branch constructs a valid whole correction. The minimum is therefore both a lower and an upper bound. They meet.
Filling cells takes time. If only the distance is needed, row reads only row and the already-computed left cell. Keep the shorter string across the row and storage falls to .
Algorithm — unit-cost edit distance with rolling rows
EDIT-DISTANCE(SOURCE, TARGET)
Input: strings SOURCE and TARGET, with length(TARGET) ≤ length(SOURCE)
Output: unit-cost edit distance between SOURCE and TARGET
previous ← [0, 1, ..., length(TARGET)]
for i ← 1 to length(SOURCE)
current[0] ← i
for j ← 1 to length(TARGET)
change ← 0
if SOURCE[i] ≠ TARGET[j]
change ← 1
current[j] ← min(previous[j] + 1, current[j - 1] + 1, previous[j - 1] + change)
SWAP(previous, current)
return previous[length(TARGET)]Rolling rows discard the path. If an explanation must highlight the edits, retain predecessor choices, recompute a constrained region, or use a divide-and-conquer traceback. Memory savings and witnesses are separate outputs; an API should say which it returns.
A gap has a beginning
Unit edit distance charges a deletion of length exactly . In many correction policies, one contiguous missing span is likelier than independent omissions. An affine gap cost separates opening and extending:
A direct recurrence asks where every possible gap began, adding a scan over prefix positions. Gotoh's insight is that the past relevant to the next cell has only three states:
- : the alignment ends with paired to ;
- : it ends inside a gap in the source; and
- : it ends inside a gap in the target.
For minimization, with substitution cost :
The exact initialization depends on whether a length-one gap costs or ; state that convention and test it. The three-state formulation replaces a search over all gap beginnings with constant work per cell, restoring time.
The transferable idea is larger than alignment: if the future needs to know a bounded summary of the past, make that summary an explicit state instead of rescanning history.
A threshold changes the reachable region
A spelling corrector often asks whether , not for every possible distance. Any path from to needs at least insertions or deletions. Therefore cells with cannot participate in a unit-cost answer at most .
Computing only the diagonal band of width reduces work to for the bounded-distance decision, plus boundary handling. A row whose in-band values all exceed may settle rejection early because nonnegative future costs cannot repair the excess.
This is safe pruning: the rejected cells have a proved lower bound above the threshold. It is not “usually enough” search.
A transposition defines another problem
Fat fingers often reverse adjacent symbols. Damerau's edit model admits transposition. The optimal-string-alignment shortcut adds a local recurrence for adjacent swaps but forbids a substring from being edited more than once. That restriction makes it easier to compute and means it is not the unrestricted Damerau–Levenshtein distance. In particular, optimal-string-alignment distance can violate the triangle inequality.
Metric — a distance that is nonnegative, symmetric, zero only for identical objects, and obeys the triangle inequality. Metric indexes rely on these laws; a useful similarity score need not satisfy them. Read Damerau's error study.
That distinction matters when a later index prunes by metric balls. A function that looks like a distance but violates a required law can make an optimization silently discard valid answers.
A column can become two bit vectors
Under unit insertion, deletion, and substitution costs, vertically adjacent DP cells differ by only , , or . Instead of storing every numeric cell, represent the positive and negative differences as bit vectors. Equality masks mark all pattern positions matching the current text symbol. Word-sized Boolean operations and addition propagate a whole column's frontier at once.
Myers's algorithm uses this relocatable bit-vector representation to process a pattern in machine words per text symbol, where is the word width. For a short dictionary word that fits in one word, one iteration advances all pattern positions together.
Add a position-dependent bonus, an arbitrary substitution matrix, or affine gap state and the tiny step alphabet may disappear. General-score bit-parallel alignment exists, but it carries more machinery. The honest rule is:
representation eligibility = recurrence + cost laws + required output
The problem name alone licenses nothing. A score-only bit-vector path also does not produce edit locations. If the interface promises a highlighted correction path, account for how that witness is recovered.
Distance generates evidence, not decisions
Consider smthin. A distance-only system may prefer a short common word whose edit path is cheap even when a reader intended something. The distance has not malfunctioned; the product asked it to decide a question it does not answer.
A correction pipeline has three distinct stages:
- Generate candidates without losing plausible corrections.
- Rank them using edit evidence, corpus frequency, prefix or consonant agreement, corpus-local aliases, and surrounding terms.
- Decide whether to preserve the literal query, offer alternatives, apply a replacement, or abstain.
Relevance is not confidence. A candidate may rank first by a narrow margin without earning an automatic rewrite. False correction is asymmetric: changing a rare valid identifier can be worse than returning no suggestion.
Keep the literal query as an alternative. Learn abbreviations within the searched corpus rather than importing a timeless universal slang list. Record the edit policy, threshold, candidate source, features, margin, and decision in the query receipt.
Negative results belong beside the winner
Several tempting shortcuts fail for different reasons:
| Shortcut | Why it looked plausible | Counterexample |
|---|---|---|
| sort candidates by raw distance | one simple number | a rare short word beats the intended frequent expansion |
| divide distance by candidate length | compare across lengths | changes the objective and can favor long distractors |
| always take the nearest candidate | top one sounds decisive | ties and small margins carry little confidence |
| add transposition to the name only | users swap letters | recurrence and metric laws change |
| use the bit-vector path for every policy | it is fast on unit costs | position bonuses destroy the bounded difference representation |
| discard the matrix implementation | the fast path passed examples | a plausible missing candidate has no visible witness |
Keep the exhaustive DP as the differential oracle. Generate short strings, cost policies within the supported regime, and thresholds; require the banded or bit-parallel result to equal the oracle. Retain the smallest counterexample when it does not.
Lessons
- “Almost” is defined by operations and costs, not by an algorithm's name.
- Wagner–Fischer follows from the exhaustive cases for an optimal final edit.
- Rolling rows preserve the distance but discard the correction path.
- Affine gaps become quadratic-time manageable when gap history is summarized by a finite state.
- A threshold creates a proved diagonal band; it does not justify heuristic truncation elsewhere.
- Transposition variants have different semantics and laws.
- Bit-vector speed is licensed by unit-cost differences and score-only output.
- Candidate generation, ranking, and correction decisions are separate jobs.
- The simple DP remains the executable specification for every faster path.
Practice
- Fill the complete unit-cost matrix for
kittenandsitting, then trace one optimal correction. - Prove that is a lower bound on any unit-cost path to cell .
- Change substitution cost to two. Which recurrence lines change, and does the rolling-row proof still hold?
- Choose and . Compare one gap of length four with four gaps of length one under the stated convention.
- Construct a correction policy with a word-boundary bonus and explain why a unit-cost bit-vector result is not an oracle for it.
- Design an abstention test using the top two candidate scores. State which empirical calibration it still needs before publication.
References
- Robert A. Wagner and Michael J. Fischer. “The String-to-String Correction Problem.” Journal of the ACM 21(1), 1974.
- Osamu Gotoh. “An Improved Algorithm for Matching Biological Sequences.” Journal of Molecular Biology 162(3), 1982.
- Esko Ukkonen. “Finding Approximate Patterns in Strings.” Journal of Algorithms 6(1), 1985.
- Fred J. Damerau. “A Technique for Computer Detection and Correction of Spelling Errors.” Communications of the ACM 7(3), 1964.
- Gene Myers. “A Fast Bit-Vector Algorithm for Approximate String Matching Based on Dynamic Programming.” Journal of the ACM 46(3), 1999.