Week 09 · Phase 3 · The language of growth

One Problem, Four Algorithms: The Anagram Story

The classic case study — four correct solutions, four completely different costs.

Big question: Same answer, same computer: why is one version a million times slower?case studycomparison≈3 hours
By the end of this week you can
  • solve one problem four different ways and compare the solutions properly;
  • see how a change of strategy — not a change of language — buys orders of magnitude;
  • recognise the counting trick that turns comparison into arithmetic;
  • confirm a complexity class by counting operations, not just by timing;
  • explain why an O(n!) solution is not "slow" but impossible;
  • weigh readability, memory and speed to say which solution you would actually ship;
  • write a comparison report with a table, a plot and a recommendation.
Where this comes from

This case study is the heart of the classic Anagram Detection Example from Miller & Ranum's Problem Solving with Algorithms and Data Structures, rewritten here for a first-year audience with counters and benchmarks added.

9.1The problem

Two words are anagrams if one is a rearrangement of the other: "listen" and "silent"; "python" and "typhon". Write a function that takes two strings of the same length and returns True if they are anagrams.

It sounds trivial, and it is — the answer is easy to get right. What makes it the perfect teaching example is that four natural, correct approaches land in four different complexity classes. Same problem, same computer, same programmer: O(n²), O(n log n), O(n!) and O(n).

For all four we will use the length of the words as n, and to keep the focus on the idea we will assume lower-case letters only.

9.2Solution 1 — checking off

The way most people describe it out loud: take each letter of the first word and cross it off in the second; if every letter finds a partner, they are anagrams.

solution_1_checking_off.py
def anagram_checking_off(s1, s2):
    if len(s1) != len(s2):
        return False

    letters = list(s2)                # we need to cross letters off
    for letter in s1:
        found = False
        for i in range(len(letters)):     # search for a partner
            if letters[i] == letter:
                letters[i] = None         # cross it off
                found = True
                break
        if not found:
            return False
    return True

print(anagram_checking_off("listen", "silent"))   # True
print(anagram_checking_off("listen", "silence"))  # False

The cost. For each of the n letters of s1 we scan up to n positions of letters. That is a loop inside a loop, both over n:

T(n) ≈ n + (n−1) + (n−2) + … + 1 = n(n+1)/2  →  O(n²)

Correct, readable, and the shape you should now recognise instantly — a nested loop over the same data.

9.3Solution 2 — sort and compare

A neat observation: two words are anagrams exactly when their letters, sorted, are identical. So sort both and compare.

solution_2_sort_and_compare.py
def anagram_sort(s1, s2):
    if len(s1) != len(s2):
        return False
    return sorted(s1) == sorted(s2)

Three lines, obviously correct, and it looks free — which is the trap. The work has not disappeared, it has moved inside sorted(), which is O(n log n). The final comparison is O(n). Adding them and keeping the dominant term:

O(n log n) + O(n) → O(n log n)

Better than solution 1 and much shorter to write. This is the "sensible default" answer, and in most real work it is the right one.

The habit to build

Short code is not cheap code. Whenever a built-in does the heavy lifting, look up what it costs. You wrote three lines; the computer did n log n comparisons.

9.4Solution 3 — brute force (do not run this on long words)

The "try everything" approach: generate every possible rearrangement of the first word and check whether the second appears among them.

solution_3_brute_force.py
from itertools import permutations

def anagram_brute_force(s1, s2):
    if len(s1) != len(s2):
        return False
    for candidate in permutations(s1):
        if "".join(candidate) == s2:
            return True
    return False

It is correct. It is also useless, and the reason is arithmetic rather than programming. A word of n letters has n! (n factorial) arrangements:

Word length nArrangements n!Time at 10 million per second
5120instant
103 628 8000.4 seconds
151 307 674 368 0001.5 days
202 432 902 008 176 640 000about 7 700 years
251.55 × 10²⁵about 49 billion years

A twenty-letter word would outlast your career, your institution, and — at n = 25 — the age of the universe. This is what people mean when they say a problem is intractable: not "we need a faster machine", but "no machine that could ever exist would finish". Try it at n = 8 to feel it move, and never again.

9.5Solution 4 — count and compare

The winning move is to stop comparing letters with letters. Count how many times each letter occurs in each word, then compare the two tallies.

solution_4_count_and_compare.py
def anagram_count(s1, s2):
    if len(s1) != len(s2):
        return False

    counts1 = [0] * 26          # one slot per letter a-z
    counts2 = [0] * 26

    for letter in s1:                          # n steps
        counts1[ord(letter) - ord('a')] += 1
    for letter in s2:                          # n steps
        counts2[ord(letter) - ord('a')] += 1

    for i in range(26):                        # always 26 steps
        if counts1[i] != counts2[i]:
            return False
    return True

ord('a') is 97, ord('c') is 99, so ord(letter) - ord('a') turns a letter into a slot number 0–25. Two passes over the words, then a fixed 26 comparisons:

T(n) = 2n + 26  →  O(n)

Linear — the best possible, since any correct method must at least look at every letter. Notice what paid for it: extra memory (two lists of 26) and a cleverer question ("how many of each?" instead of "does this one match?"). That trade — spend a little space to save a lot of time — is one of the most reliable moves in all of computing, and week 11 is built entirely on it.

9.6Head to head

SolutionClassSteps at n = 1 000Extra memoryVerdict
1. Checking offO(n²)~500 000a copy of the wordFine for words, hopeless for large data
2. Sort and compareO(n log n)~10 000two sorted copiesExcellent default: short and fast enough
3. Brute forceO(n!)more than atoms in the galaxytinyNever
4. Count and compareO(n)~2 026two 26-slot listsFastest; slightly more code

Now measure it yourself rather than trusting the table. Long "words" made of random letters stand in for real ones:

benchmark_anagrams.py
import random, string, time

def make_pair(n):
    """Two strings of length n that really are anagrams — the worst case."""
    letters = [random.choice(string.ascii_lowercase) for _ in range(n)]
    shuffled = letters[:]
    random.shuffle(shuffled)
    return "".join(letters), "".join(shuffled)

def time_call(func, s1, s2, repeats=3):
    best = None
    for _ in range(repeats):
        start = time.perf_counter()
        func(s1, s2)
        t = time.perf_counter() - start
        if best is None or t < best:
            best = t
    return best

for n in [500, 1000, 2000, 4000]:
    s1, s2 = make_pair(n)
    a = time_call(anagram_checking_off, s1, s2)
    b = time_call(anagram_sort, s1, s2)
    d = time_call(anagram_count, s1, s2)
    print(f"n={n:>5}  checking-off {a:.5f}s   sort {b:.5f}s   count {d:.5f}s   speedup {a/d:,.0f}x")
n= 500 checking-off 0.00832s sort 0.00006s count 0.00011s speedup 76x n= 1000 checking-off 0.03310s sort 0.00013s count 0.00022s speedup 150x n= 2000 checking-off 0.13245s sort 0.00028s count 0.00044s speedup 301x n= 4000 checking-off 0.52901s sort 0.00061s count 0.00088s speedup 601x

Two things to notice. First, the speedup column grows — at n = 4 000 the naive method is already 600 times worse, and it will be 6 000 times worse at n = 40 000. Second, the sort version beats the counting version in wall-clock time even though its class is worse, because sorted() runs in optimised C while your counting loop runs in Python. That is a constant factor at work — and it is exactly why we measure as well as classify.

The lesson of the last paragraph

Big-O predicts who wins eventually; the stopwatch tells you who wins today, at this size, in this language. Push n high enough and the O(n) version pulls ahead of the O(n log n) one — find that crossover in the homework.

9.7Counting operations to confirm the class

Timing is honest but noisy — it mixes in your CPU, the language and whatever else the machine is doing. There is a cleaner way to prove a class: count the fundamental operations the algorithm performs and watch how the count grows. If the theory says O(n²), then doubling n should roughly quadruple the count; if it says O(n), doubling should roughly double it. No stopwatch required.

We add a counter to each method — for the two loop-based ones we count letter comparisons; for the counting method we count tally updates plus the 26 final checks. Because anagram_sort hides its work inside C, we time it separately in the benchmark; here we compare the two we can instrument.

count_operations.py
import random, string

def make_pair(n):
    letters = [random.choice(string.ascii_lowercase) for _ in range(n)]
    shuffled = letters[:]
    random.shuffle(shuffled)
    return "".join(letters), "".join(shuffled)

def checking_off_ops(s1, s2):
    ops = 0
    letters = list(s2)
    for letter in s1:
        for i in range(len(letters)):
            ops += 1                         # one comparison
            if letters[i] == letter:
                letters[i] = None
                break
    return ops

def count_ops(s1, s2):
    ops = 0
    c1 = [0] * 26
    c2 = [0] * 26
    for letter in s1:
        c1[ord(letter) - ord('a')] += 1; ops += 1
    for letter in s2:
        c2[ord(letter) - ord('a')] += 1; ops += 1
    for i in range(26):
        ops += 1
    return ops

print(f"{'n':>6}  {'checking-off':>14}  {'ratio':>6}   {'count':>8}  {'ratio':>6}")
prev_a = prev_c = None
for n in [250, 500, 1000, 2000]:
    s1, s2 = make_pair(n)
    a = checking_off_ops(s1, s2)
    c = count_ops(s1, s2)
    ra = "-" if prev_a is None else f"{a/prev_a:.1f}x"
    rc = "-" if prev_c is None else f"{c/prev_c:.1f}x"
    print(f"{n:>6}  {a:>14,}  {ra:>6}   {c:>8,}  {rc:>6}")
    prev_a, prev_c = a, c
n checking-off ratio count ratio 250 31,653 - 526 - 500 125,910 4.0x 1,026 2.0x 1000 503,110 4.0x 2,026 2.0x 2000 2,006,214 4.0x 4,026 2.0x

Read the two ratio columns and the classes fall straight out. Checking-off quadruples with every doubling — the unmistakable fingerprint of O(n²), and the raw counts sit right on the predicted n²/2. Counting doubles with every doubling — O(n), sitting on the predicted 2n + 26. This is the same ratio-column reasoning you used with the doubling harness in weeks 5–7, applied to operation counts instead of seconds, and it is immune to a noisy laptop.

Two tools, one verdict

When the operation-count ratio and the timing ratio agree, you have a strong claim: the shape is confirmed by arithmetic and by the clock. When they disagree, you have found a constant factor — usually a chunk of work happening in C — and that is exactly what §9.6 showed for sorted().

9.8Which would you ship?

"Fastest class" is not automatically "best choice". Real decisions weigh speed against readability, memory, and the size of the data you will actually see. Line the four up against those questions and the answer stops being obvious:

SolutionReads clearly?Extra memoryWins when…
1. Checking offyes, mirrors the spoken ideaO(n)never — the O(n²) is a liability with no upside
2. Sort and compareyes, one obvious lineO(n)ordinary word-sized inputs; you value short, clear code
3. Brute forceyes, but pointlessO(1)never — correct yet unusable past a handful of letters
4. Count and comparea little more codeO(1)*huge inputs, or the same check run millions of times

* Two fixed 26-slot lists — the memory does not grow with n, so this is O(1) extra space, unlike the sorted copies solution 2 must build.

So which do you ship? For a word game where inputs are a dozen letters and the check runs occasionally, solution 2 is the right call: one line, obviously correct, fast enough that its worse class never bites. For a bioinformatics tool comparing million-character sequences, or a service that runs the check millions of times a second, solution 4 is the only responsible choice — its linear time and constant extra space are the difference between "instant" and "times out". The professional answer is not a single winner; it is "solution 2 by default, solution 4 when the numbers say so, and here is the number that flips it" — the crossover you will find in the homework.

Say the trade out loud

A good recommendation names what it is optimising for. "I chose sort-and-compare for readability because our words are short" is a decision. "I used sorting because it was first on the page" is an accident that happens to be fine until the data grows.

9.9Try it yourself

Task 1 — instrument solution 1

Add a comparison counter to anagram_checking_off and report it for n = 100, 200, 400 on genuine anagram pairs. Compare the counts with the predicted n(n+1)/2.

Expected

About 5 050, 20 100 and 80 200 — close to n²/2 and quadrupling with each doubling, just as predicted.

Task 2 — how far can brute force go?

Time anagram_brute_force at n = 6, 7, 8, 9 (stop there). Then predict n = 12 from the pattern, and say whether you would wait for it.

Expected

Each extra letter multiplies the time by roughly n. From n = 9 to n = 12 is a factor of about 10 × 11 × 12 ≈ 1 320. If n = 9 took a second, n = 12 takes about 22 minutes — and n = 15 about a fortnight.

Task 3 — find the crossover

The sorting solution beats the counting one at small n because of constant factors. Push n up (10 000, 100 000, 1 000 000 — build the strings outside the timer) and find where the O(n) version overtakes.

Task 4 — a fifth solution

Python has collections.Counter. Write anagram_counter(s1, s2) in one line using it, benchmark it against your solution 4, and explain why the class is the same but the time differs.

Solution
one line
from collections import Counter

def anagram_counter(s1, s2):
    return Counter(s1) == Counter(s2)

Still O(n) — Counter walks each string once. It is usually faster than the hand-written loop because the counting happens in C, not in Python. Same class, smaller constant.

Task 5 — a variation: are they permutations of digits?

Adapt the counting solution to decide whether two lists of integers (each 0–9) are rearrangements of each other — e.g. [3, 1, 4, 1] and [1, 4, 3, 1]. What is the class, and what changed from the letters version?

Solution
digits version
def same_multiset(a, b):
    if len(a) != len(b):
        return False
    counts = [0] * 10          # one slot per digit 0-9
    for x in a:
        counts[x] += 1
    for x in b:
        counts[x] -= 1         # cancel out
    return all(c == 0 for c in counts)

Still O(n): two passes over the data and a fixed 10-slot check. The only change is a smaller alphabet (10 slots instead of 26) and a tidy trick — add for one list, subtract for the other, and everything should cancel to zero. Same idea, same class.

Task 6 — confirm by counting, not timing

Run the operation-counter from §9.7 for the checking-off and counting methods at n = 100, 200, 400, 800. Put the two ratio columns side by side and state, in one sentence each, what class each column proves — without ever calling time.perf_counter.

Expected

The checking-off ratio hovers around 4.0×, proving O(n²); the counting ratio hovers around 2.0×, proving O(n). A quadrupling means the exponent is 2; a doubling means the exponent is 1.

9.10Self-check

Why is sorted(s1) == sorted(s2) not O(n)?

The cost is inside the built-in. Short code can hide expensive work; always price the built-ins you call.

What buys the counting solution its linear time?

Trading a little space for a lot of time is one of the most reliable moves in computing — and the whole idea behind week 11.

An O(n!) method on a 20-letter word is best described as:

2.4 × 10¹⁸ arrangements is thousands of years at ten million per second. Factorial growth ends the conversation.

In the benchmark, the O(n log n) version beat the O(n) version at n = 1 000. Does that disprove the analysis?

Both are right about different questions. Report the class and the crossover, and you have told the whole truth.

When you double n, the checking-off operation count roughly quadruples. That confirms:

A ×4 response to a ×2 input is the fingerprint of quadratic growth, just as it was in the doubling harness of weeks 5–7 — counts work as well as seconds.

Inputs are a dozen letters and the check runs a handful of times. Which do you ship, and why?

At tiny n the class barely matters; readability and correctness do. The counting version earns its keep only when the data is large or the call is hot.

9.11Homework

Due before week 10
  1. Create AA_Week09.ipynb with all five solutions and complete Tasks 1–6.
  2. Run the operation-counter from §9.7 and produce a table with the two ratio columns; write one sentence under it naming the class each column proves.
  3. Produce one figure with the three usable solutions plotted against n (log–log axes, labelled).
  4. Write a one-page comparison report: problem, four approaches, table of classes, your measurements, the crossover you found, and a recommendation for (a) words in a game and (b) million-character biological sequences — say explicitly what each recommendation is optimising for.
  5. Answer in five sentences: solution 3 is correct and yet worthless. What does that tell you about "correct" as a standard for software?

9.12Words from this week

TermMeaning in plain words
anagramA word made of exactly the same letters as another.
brute forceTry every possibility. Always correct, usually unusable.
factorial n!n × (n−1) × … × 1 — the number of arrangements; grows faster than any exponential.
intractableSo expensive that no realistic machine can finish it.
operation countThe number of fundamental steps an algorithm performs; its ratio confirms a class without a stopwatch.
space–time trade-offUsing extra memory to save time, or the reverse.
crossoverThe input size at which the better-scaling method overtakes the one with a smaller constant.

9.13Chapter problem set — Skiena 2.10

These two problems from Skiena's The Algorithm Design Manual ask exactly the question week 9 keeps asking — two correct methods, two wildly different costs — but about a problem every child already knows: multiplying two numbers. Read each restated problem, have a go, then open the worked solution.

Problem 2-37 · multiplication by repeated addition

You first learned to multiply as repeated addition: 5 × 4 = 5 + 5 + 5 + 5. What is the time complexity of multiplying two n-digit numbers in base b this way, written as a function of n and b? (Hint: how large can the multiplier get?)

Worked solution

To compute x × y by repeated addition we add x to a running total y times, so the whole cost hangs on how big y can be. An n-digit number in base b is at most b to the power n, minus one — roughly bn. In base 10 a 3-digit number reaches 999 ≈ 10³; in base 2 a 3-bit number reaches 7 ≈ 2³. So y can force us to perform about bn additions.

Each of those additions is of two n-digit numbers, which costs O(n) — you add digit by digit, carrying as you go. Multiply the two together:

(about bn additions) × (O(n) per addition)  →  O(n · bn)

That bn is exponential in n — the same runaway growth week 9 called intractable. A table makes the hopelessness concrete for base 10:

Digits nLargest value ≈ 10ⁿAdditions needed (worst case)
19up to 9
3999up to ~1 000
6999 999up to ~1 000 000
10~10¹⁰up to ~10 billion
20~10²⁰more additions than there are seconds since the Big Bang

Answer: O(n · bn) — exponential in the number of digits. Repeated addition is perfectly fine for 5 × 4, but hopeless for real numbers: multiplying two 20-digit numbers would need something like 10²⁰ additions, which no machine will ever finish. The moral is week 9's exactly — a method can be entirely correct and still unusable.

Problem 2-38 · grade-school long multiplication

In grade school you learned to multiply digit by digit — long multiplication, with the partial rows lined up and added. What is the complexity of multiplying two n-digit numbers this way, for a fixed base?

Worked solution

Long multiplication pairs every digit of one number with every digit of the other. With n digits in each, that is n × n = n² single-digit multiplications. Each produces a small partial result, and we then line those partials up and add them — which is another O(n²) of work to combine (there are n rows, each up to about n digits wide).

n² single-digit products + O(n²) additions to combine  →  O(n²)

Now stand the two problems side by side — this is the whole point of the pair:

MethodClassMultiplying two 20-digit numbers
2-37 repeated additionO(n · bn)about 10²⁰ steps — impossible
2-38 long multiplicationO(n²)about 400 steps — instant

Answer: O(n²). Long multiplication is astronomically better than repeated addition: for 20-digit numbers it is roughly 400 steps against 10²⁰ — the difference between "instant" and "never". And notice what bought that gap — a smarter method, on the same machine, not faster hardware.

A note for the curious

n² is not the end of the story either. Cleverer methods do better still: Karatsuba's algorithm multiplies in about O(n1.585) by replacing four sub-multiplications with three, and modern FFT-based methods get close to O(n log n). Those belong to a later course — but it is worth knowing that even the grade-school n² can be beaten.

Where this leads

You saw that changing strategy — not language or hardware — buys orders of magnitude. Week 10 hunts those costs inside everyday Python list operations.