Putting It All Together
A checklist for choosing, the classic 'accidentally quadratic' traps, and the final project.
- work through a checklist for choosing an approach to a new problem;
- recognise the five patterns that quietly make beginner code quadratic, and fix each;
- follow one realistic problem from a slow first attempt to a measured, linear solution;
- find the slow part of a program instead of guessing at it;
- write a short, honest performance report and mark one against a rubric;
- say what you would learn next, and why.
14.1The whole course on one page
| Class | What it means | You met it in | Typical source |
|---|---|---|---|
| O(1) | Cost ignores the data size | weeks 2, 4, 11 | Indexing, a formula, dict/set lookup |
| O(log n) | Each doubling adds one step | weeks 1, 3, 12 | Halving: binary search, bisect |
| O(n) | One visit per item | weeks 3–5, 9, 11 | A single loop, sum, max, in on a list |
| O(n log n) | A pass, repeated log n times | weeks 9, 12, 13 | sorted(), sort-then-search |
| O(n²) | Every item meets every item | weeks 3, 9, 10, 13 | Nested loops; a hidden O(n) inside a loop |
| O(2ⁿ), O(n!) | Unusable beyond tiny inputs | week 9 | Trying every combination or arrangement |
14.2The choosing checklist
Six questions, in this order, for any new task. Most real decisions are settled by question three.
- How big is n, really? Ten rows forever? Then stop — write the clearest code and move on. Ten million, or growing? Continue.
- What is the dominant operation? Searching, counting, sorting, matching, or all-pairs comparison? Name it out loud.
- Am I searching inside a loop? If yes, a set or dictionary almost certainly turns O(n²) into O(n). Check this before anything else.
- Can I prepare once instead of repeating? Build the set, sort the list, compute the tally — once, outside the loop.
- Do I need order? Nearest, next, between-two-values: sorted list plus
bisect. Only membership: set. - What does it cost me? More memory, lost order, more code to maintain. Say so in your report rather than hiding it.
Measure. A doubling experiment at four sizes takes ten minutes and settles arguments that otherwise run for weeks.
14.3The five ways beginners go quadratic
| Pattern | Looks like | Fix |
|---|---|---|
| Search inside a loop | for x in a: if x in b: | b = set(b) before the loop |
| Building by concatenation | result = result + [x], text += word | append, then "".join(...) |
| Working at the front of a list | insert(0, x), pop(0) | collections.deque |
| Sorting inside a loop | for …: sorted(data) | Sort once, outside |
| Recomputing a total every round | for …: sum(data) | Keep a running total |
Every one of these is two lines of code that look innocent. If you remember only this table from the whole course, you will still write faster programs than most people who have never taken it. Two of them hide their O(n) so well that they deserve a closer look.
# SLOW: each += builds a brand-new string by copying the whole thing so far
text = ""
for word in words: # n words ...
text += word + " " # ... copying a growing string each time → O(n²)
# FAST: collect the pieces, join once at the end
parts = []
for word in words:
parts.append(word) # O(1) each
text = " ".join(parts) # one pass → O(n) total
Strings in Python cannot be changed in place, so text += word quietly copies
every character it has accumulated so far. Do that n times and you have copied roughly
n²/2 characters. "".join touches each character once. The two produce the same
text; only one of them scales.
from collections import deque
# SLOW: insert(0, ...) shifts every existing element one place right → O(n) each
queue = []
for job in jobs:
queue.insert(0, job) # O(n) each → O(n²) overall
# FAST: a deque adds and removes at both ends in O(1)
queue = deque()
for job in jobs:
queue.appendleft(job) # O(1) each → O(n) overall
A list stores its items shoulder to shoulder, so making room at the front means
sliding everything else along — O(n) every time. append at the end is O(1),
which is why the fix for "I need the front too" is a deque, not a cleverer
list.
14.4A worked case study, end to end
Everything in the course meets here. The task: from a big list of orders, total up how much each watch-listed customer spent. The obvious version is correct and slow; we will name why, measure it, fix it, and measure again.
import time, random
def make_data(n_orders, n_watch):
names = [f"cust{i}" for i in range(n_orders // 2)]
orders = [{"customer": random.choice(names), "amount": random.randint(1, 100)}
for _ in range(n_orders)]
watch = random.sample(names, n_watch)
return orders, watch
def summarise_slow(orders, watch):
totals = {}
for o in orders: # n orders ...
if o["customer"] in watch: # ... scan of the watch list → O(m) each
name = o["customer"]
totals[name] = totals.get(name, 0) + o["amount"]
return totals
def summarise_fast(orders, watch):
watch_set = set(watch) # O(m), paid once
totals = {}
for o in orders: # n orders ...
if o["customer"] in watch_set: # ... O(1) each
name = o["customer"]
totals[name] = totals.get(name, 0) + o["amount"]
return totals
for n in [5000, 10000, 20000, 40000]:
orders, watch = make_data(n, n // 10) # watch grows with n
start = time.perf_counter()
slow = summarise_slow(orders, watch)
t_slow = time.perf_counter() - start
start = time.perf_counter()
fast = summarise_fast(orders, watch)
t_fast = time.perf_counter() - start
assert slow == fast # same answer — always check
print(f"n={n:>6} slow {t_slow:8.4f}s fast {t_fast:.5f}s {t_slow/t_fast:,.0f}x")
Read the columns exactly as in weeks 5–13. The slow column quadruples each time n doubles:
that is the 4× fingerprint of O(n²), and it appears because the watch list grows with the
data, so in watch costs O(m) inside a loop of n — the pattern from row one of
§14.3. The fast column merely doubles: O(n). And the speedup column grows without limit,
because the two solutions are in different classes, not merely different by a constant.
One change of container (list → set for the membership test) turned O(n·m) into O(n + m).
One dictionary did the tallying in O(n) rather than a list-search per order. One
assert proved the fast version still gives the right answer. That is the
entire course in three lines — and it is exactly what your final-project report should
show.
14.5Finding the slow part
Do not guess — programmers are famously bad at guessing which line is slow. Two tools, in increasing order of effort:
import time
start = time.perf_counter()
data = load_data()
print(f"loading: {time.perf_counter() - start:.3f}s")
start = time.perf_counter()
result = analyse(data)
print(f"analysing: {time.perf_counter() - start:.3f}s")
start = time.perf_counter()
save(result)
print(f"saving: {time.perf_counter() - start:.3f}s")import cProfile
cProfile.run("analyse(data)", sort="cumtime") # slowest firstThen apply the 90/10 rule: nearly all the time is in a small part of the code. Optimising anything else is wasted effort, however satisfying it feels.
Correct first, measured second, optimised third. An optimisation you cannot measure
is a superstition, and a fast program that gives wrong answers is worthless — always
re-check the output after a rewrite, exactly as the assert in §14.4 did.
14.6Writing the report
Your final deliverable, and the format for any performance argument you make in future work or study:
- The problem — one paragraph, no code. What data, what question, what size now and what size expected.
- Approach A — the obvious one. Code, and its Big-O class with a one-line justification.
- Approach B — the considered one. Same treatment, plus what it costs you (memory, complexity, lost order).
- Evidence — a table at four sizes with a ratio column, and one labelled figure. State the machine, the repeat count, and whether you took best or mean.
- Interpretation — the five sentences from week 6, including a prediction for a size you did not run.
- Recommendation — which one, at what data size, and when you would change your mind.
- Limitations — one honest paragraph. What you did not test; where the measurement might mislead.
Point 7 earns marks. Every genuine measurement has a caveat, and naming yours is the difference between a student result and a professional one. "The 4 000-item run was on a shared Colab machine and may be inflated" costs nothing and buys all your other numbers credibility.
14.7Workshop tasks
Read this function, name its class, identify every problem, and rewrite it.
def build_report(orders, vip_customers):
lines = []
text = ""
for order in orders:
if order["customer"] in vip_customers: # vip_customers is a list
lines = lines + [order]
text = text + order["id"] + ", "
total = 0
for order in lines:
total = total + sum(o["value"] for o in lines)
return lines, text, totalSolution
Four problems: (1) membership test against a list inside a loop → O(n·m);
(2) lines = lines + [order] → quadratic; (3) text = text + …
→ quadratic; (4) the last loop recomputes the whole sum once per line → quadratic
and wrong (it multiplies the total by the number of lines). Rewritten:
def build_report(orders, vip_customers):
vips = set(vip_customers) # O(m), once
lines = [o for o in orders if o["customer"] in vips] # O(n)
text = ", ".join(o["id"] for o in lines) # O(n)
total = sum(o["value"] for o in lines) # O(n)
return lines, text, totalFrom O(n²) to O(n + m) — and the bug in the total is gone as a side effect of writing it clearly.
Given a list of a million numbers, find whether any two of them add up to a given target. Write the obvious O(n²) version, then an O(n) version using a set. Benchmark both at 2 000, 4 000 and 8 000.
The linear version
def has_pair(data, target):
seen = set()
for x in data:
if target - x in seen: # O(1)
return True
seen.add(x)
return FalseOne pass, constant work per item. The trick — remember what you have seen so that you never search — is the same one that made the anagram counter linear in week 9.
Swap final-project drafts with a classmate. For their work, answer: is the benchmark honest (data built outside the timer, repeats, worst case chosen)? Does the claimed class match the ratio column? Is there a limitation they did not mention? Write half a page.
Write a tiny quadratic offender for each of the five patterns in §14.3 (five short functions), benchmark each against its linear fix at n = 1 000, 2 000, 4 000, 8 000, and confirm the ratio column sits near 4 for the slow one and near 2 for the fast one.
Hint on the string trap
For the concatenation trap, build a string of n words with text += w versus
" ".join(parts). The += version quadruples each time n doubles;
join merely doubles. Same output — verify with assert a == b.
Take the §14.4 summarise_slow and run it under
cProfile.run(...) at n = 40 000. Which single line accounts for almost all
the cumulative time? Confirm that it is the membership test, not the dictionary update,
and write two sentences relating that to the 90/10 rule.
Expected
The profile attributes nearly all the time to the in watch test, because
it alone is O(m) and runs n times; the dictionary work is O(1) per order and barely
registers. This is the 90/10 rule in miniature: fix the one hot line and the whole
function drops a class, and touching anything else would have been wasted effort.
14.8Self-check
Your script is slow. What comes first?
Which single change most often fixes a beginner's slow program?
n will always be about 50 rows. The right approach is:
A report claims "our new version is 40× faster". What is the first question to ask?
Why is text += word in a loop over n words a quadratic trap?
"".join them once instead.You need to add and remove items at the front of a collection millions of times. The right tool is:
insert(0, x) shifts every element and is O(n) each time; a deque adds and removes at both ends in O(1).14.9Final project — the deliverable and the rubric
Pick a problem where an obvious solution and a considered one land in different complexity classes. Good choices reuse this course directly:
- de-duplicating or cross-matching two large lists (weeks 10–11);
- counting or grouping records — words, transactions, log lines (week 11);
- repeated membership or range queries: scan vs sort-and-
bisectvs set (week 12); - a sort-based versus count-based solution to a matching problem (weeks 9, 13);
- fixing an "accidentally quadratic" script into a linear one (week 14).
Hand in three things:
- One notebook,
AA_Final_YourName.ipynb, containing both approaches, the benchmark harness (data built outside the timer, at least three repeats, best or mean stated), and one labelled log–log figure. - A report of about two pages following the seven-part structure in §14.6.
- A five-minute presentation: the problem, one figure, one recommendation, one limitation. No code on the slides.
Marks are awarded against this rubric. Note where the weight sits — reasoning and honesty, not the size of the speedup.
| Criterion | What earns full marks | Weight |
|---|---|---|
| Problem framing | Clear question, the data described, current and expected n stated | 10 |
| Two correct approaches | Both run and agree on the answer (shown by an assert or equivalent) | 20 |
| Big-O analysis | Each approach's class named and justified in one honest line | 15 |
| Benchmark quality | Data outside the timer, repeats, four sizes, a ratio column | 20 |
| Interpretation and prediction | Class read from the ratio column; a prediction for an un-run size | 15 |
| Recommendation | A clear choice tied to data size, and when you would change it | 10 |
| Honesty and limitations | One genuine caveat about the measurement or its scope | 10 |
A well-measured 3× improvement with a clear explanation and an honest caveat beats an unexplained 1 000×. The rubric rewards the reasoning, because the reasoning is the skill that outlasts any one problem.
14.10Where to go next
Recursion and divide-and-conquer
Write merge sort and quicksort yourself, and see where n log n really comes from.
More data structures
Stacks, queues, linked lists, trees, heaps and graphs — each solving an operation lists do badly.
Space complexity
The same analysis applied to memory. Sometimes the limit is what fits, not what finishes.
Vectorised computing
NumPy and pandas: the same O(n) work with constants tens of times smaller.
The next course
Data Structures & Algorithms assumes exactly what you now know.
The textbook
Problem Solving with Algorithms and Data Structures — free, interactive, and the source of week 9.
Fourteen weeks ago the question was "does this program work?" You now ask a better one: "and what happens when there is ten times more data?" That question is the entire subject, and you can answer it with a stopwatch, a table and a plot. Well done.
14.11Words from this week
| Term | Meaning in plain words |
|---|---|
| profiling | Measuring which parts of a program consume the time. |
| 90/10 rule | Almost all the time is spent in a small fraction of the code. |
| premature optimisation | Making code faster before knowing it is slow, or where. |
| accidentally quadratic | Innocent-looking code that hides an O(n) step inside a loop, making it O(n²). |
| bottleneck | The part that limits the whole; the only part worth optimising. |
| trade-off | What the faster version costs you: memory, clarity, order, flexibility. |
14.12Chapter problem set — Skiena 2.10
A capstone deserves capstone puzzles. These last problems from Skiena's The Algorithm Design Manual are the kind asked in interviews: each one rewards a clear idea over brute force, and each connects back to a habit this course has drilled — prepare once, count carefully, reason backwards. Read, attempt, then unfold the solution.
From a set S of n numbers, choose a subset S′ of k numbers so that every element of S is picked with equal probability k/n — in a single pass. Now the twist: do it when n is not known in advance (the numbers arrive one by one and stop when they stop).
Worked solution
This is reservoir sampling. Start with the easy case, k = 1 — keep a single item. The rule: when the i-th item arrives (counting from 1), keep it with probability 1/i, otherwise stick with what you already hold. Why does that give everyone the same 1/n at the end? The last item, number n, is kept with probability 1/n directly. An earlier item j was kept when it arrived (prob 1/j) and then survived every later item not replacing it: (1 − 1/(j+1)) × (1 − 1/(j+2)) × … × (1 − 1/n). Those brackets are j/(j+1) × (j+1)/(j+2) × … × (n−1)/n, a telescoping product that collapses to j/n. Multiply by the 1/j from the start and you get 1/n. Everyone ties.
For general k, keep the first k items as the initial "reservoir". When the i-th item arrives (i > k), keep it with probability k/i; if you keep it, drop one of the k held items chosen at random to make room. The same telescoping argument gives every item a final probability of exactly k/n.
import random
def reservoir(stream, k):
keep = []
for i, item in enumerate(stream): # i counts from 0; n never needed
if i < k:
keep.append(item) # fill the reservoir first
else:
j = random.randint(0, i) # 0..i inclusive → keep with prob k/(i+1)
if j < k:
keep[j] = item
# evict a random held item and insert the new one
return keep
Answer: reservoir sampling. Hold the first k items; for each later item i
keep it with probability k/i, replacing a random current member. It makes one pass, stores
only k items, and never needs to know n ahead of time — perfect for a stream too big to
hold or of unknown length. A quick simulation (run reservoir many times over
range(100) with k = 10 and tally how often each number appears) confirms every
element shows up almost exactly k/n of the time.
You store 1000 items across 1000 nodes; each node holds 3 different items. Propose a replication scheme that minimises data loss when nodes fail, and estimate how many items you expect to lose when 3 randomly chosen nodes fail at once.
Worked solution
The core idea is simple: keep three copies of every item, on three well-separated nodes, and never let two items share the exact same trio of nodes. An easy scheme that does this: put item i on nodes i, i+1 and i+2 (wrapping round mod 1000). Every item now has three homes, every node still holds three items, and each item's trio is a distinct set of three consecutive nodes.
An item is lost only when all three of its nodes are down. If just 3 random nodes fail, a given item is lost only if those 3 failures are precisely its 3 nodes. The chance of that for one item is 1 divided by the number of ways to choose 3 nodes from 1000 — that is 1 / C(1000, 3) = 1 / 166 167 000, a tiny number. Add that up over all 1000 items:
expected items lost ≈ 1000 × 1 / C(1000, 3) ≈ 6 × 10⁻⁶
Answer: three copies per item, spread across distinct nodes so no two items share a trio. With 3 random node failures the expected loss is essentially zero — almost always no item is lost at all, and even in the rare case the 3 failures happen to be one item's exact trio, you lose at most about one item. The whole win comes from replication plus spreading the copies out: three copies packed onto overlapping node sets would lose far more.
n companies gradually merge until a single company remains. Each merge combines two companies into one. How many different sequences of pairwise merges are possible?
Worked solution
First the easy half: each merge turns two companies into one, so it reduces the count by exactly one. Going from n down to 1 therefore always takes n − 1 merges, no matter the order. The interesting question is how many ordered ways those merges can play out. Do the small cases by hand:
| n | ways | why |
|---|---|---|
| 2 | 1 | only one pair to merge |
| 3 | 3 | first merge is one of the 3 pairs (AB, AC, BC); then only one pair is left |
| 4 | 18 | more choices at each of the 3 merges |
The closed form that produces 1, 3, 18, … is:
number of merge sequences = n! · (n−1)! / 2n−1
Check it: n = 2 gives 2 · 1 / 2 = 1; n = 3 gives 6 · 2 / 4 = 3; n = 4 gives 24 · 6 / 8 = 18. All match.
Answer: n − 1 merges are always needed, and the number of distinct ordered merge sequences is n!(n−1)! / 2n−1, matching the hand counts 1, 3, 18 for n = 2, 3, 4.
Think of building the merge history as a rooted binary tree with the n companies as leaves and each internal node a merge. There are (n−1)! orders in which to perform the n−1 merges, and counting the distinct labelled shapes contributes the extra n! / 2n−1 factor — the 2n−1 divides out because swapping the two companies in a single merge does not make a new sequence. Deriving it fully is a combinatorics exercise; the formula and the small-case checks are what you need here.
A Ramanujan number can be written as a sum of two cubes in two different ways: a³ + b³ = c³ + d³ (with distinct a, b, c, d). Find all of them with a, b, c, d < n, faster than trying every four numbers.
Worked solution
Four nested loops would be O(n⁴). Instead use the "remember what you have seen" trick: build every two-cube sum once and look for collisions. Go over each pair (a, b) with a ≤ b < n, and store a³ + b³ in a dictionary keyed by the sum. Any key hit by two or more different pairs is a Ramanujan number.
def ramanujan(n):
seen = {} # sum -> list of (a, b) pairs
for a in range(1, n):
for b in range(a, n): # a <= b avoids double-counting
seen.setdefault(a**3 + b**3, []).append((a, b))
return {s: p for s, p in seen.items() if len(p) >= 2}
print(sorted(ramanujan(20)))The smallest is the famous 1729 = 1³ + 12³ = 9³ + 10³ (the Hardy–Ramanujan taxicab number). One pass over the ≈ n²/2 pairs, O(1) per insert on average.
Answer: hash every a³ + b³ and keep the sums reachable two ways — O(n²) time and space, versus brute-force O(n⁴). Smallest Ramanujan number: 1729.
Six pirates must divide $300 by seniority. The most senior proposes a split; all pirates (proposer included) vote; if at least half agree, it passes, otherwise the proposer is thrown overboard and the next most senior proposes. Pirates are perfectly rational and value, in order: staying alive, then money. How is the $300 divided (2-51)? And what happens if there is only one indivisible dollar to share (2-52)?
Worked solution
The trick to every puzzle like this is to reason backwards from the smallest case, because each pirate, when voting, compares "what I get now" against "what I would get if this proposer is thrown overboard". Label the pirates P1 (most junior) up to P6 (most senior); the senior proposes first. "At least half" means a tie is enough to pass.
Part 2-51, dividing $300. Build up one pirate at a time:
| pirates left | votes needed | proposer keeps | who gets $1 (to buy the vote) |
|---|---|---|---|
| 2 (P2, P1) | 1 | $300 | nobody — P2's own vote is half of 2 |
| 3 (P3…) | 2 | $299 | P1 (who would get $0 if it fell to 2 pirates) |
| 4 (P4…) | 2 | $299 | P2 (who gets $0 in the 3-pirate outcome) |
| 5 (P5…) | 3 | $298 | P3 and P1 (the two who get $0 with 4 pirates) |
| 6 (P6…) | 3 | $298 | P4 and P2 (the two who get $0 with 5 pirates) |
Each proposer only has to bribe the cheapest pirates — the ones who would get nothing in the next round — and just enough of them to reach half the votes. So with six pirates, the senior P6 needs 3 votes (his own plus two), and he secures them by handing $1 each to P4 and P2, keeping the other $298 for himself.
Answer (2-51): the top pirate keeps $298 and gives $1 each to two specific juniors (P4 and P2) to secure a passing vote of 3-to-3; P5, P3 and P1 get nothing. Rationality, not generosity, produces the lopsided split — the juniors accept $1 because the alternative is $0.
Part 2-52, a single indivisible dollar. Now the proposer has almost nothing to bribe with, so survival votes do the work. Reasoning down the chain, with 5 pirates the proposer P5 would need 3 votes but can bribe at most one pirate with the lone dollar — so a 5-pirate proposal fails and P5 is thrown overboard. That fact is the key: it means P5 is desperate to avoid ever reaching a 5-pirate game.
So when 6 pirates face the single dollar, the senior P6 needs 3 votes and gets them like this: his own vote; P5's vote for free (because if P6 is thrown over, the game drops to 5 pirates where P5 dies, so P5 will support almost any proposal that keeps P6 afloat); and one more vote bought by giving the single dollar to one of the pirates who would otherwise get nothing (P4, P3 or P1). That is a winning coalition of three, so the proposal passes.
Answer (2-52): the top pirate survives and no one need die. He gives the one dollar to a single pirate whose vote he needs and relies on the vote of P5, who backs him purely to avoid being thrown overboard in the next round. The dollar plus one life-saving vote is exactly enough to form a majority — a neat demonstration that "value survival first" can be worth more than money at the table.
Skiena pairs the chapter with three short programming problems on the online judges (UVA). Each is a one-idea problem — no heavy machinery, just the right observation. Try them for practice; a sketch of the approach is enough to get you started.
Approaches
Primary Arithmetic — count the carry operations when two unsigned numbers are added. Add the two numbers the grade-school way, digit by digit from the right, keeping a running carry; each position where the column plus the incoming carry reaches 10 produces a carry into the next column. Count those events. It is O(number of digits), and the whole problem is really just "do primary-school addition and watch the carries" — report none, one, or the count.
A Multiplication Game — two players start from a product of 1 and take turns multiplying it by any integer from 2 to 9; the first to reach or pass n wins, and the first player (Stan) moves first. This is a small combinatorial game: reason backwards from n about which running totals are winning positions for the player about to move. With optimal play the winner depends only on which "band" n falls in, and the band edges are 9, 18, 162, 324, 2916, … — the gaps alternate ×2 then ×9 (not "powers of nine"): Stan wins for n in 2–9, the second player for 10–18, Stan for 19–162, the second player for 163–324, Stan for 325–2916, and so on. A clean rule: repeatedly replace n by ⌈n/18⌉ until n ≤ 18, then Stan wins exactly when the leftover n ≤ 9. No search is needed — just a handful of divisions.
Light, More Light — lamps numbered 1..N start off; person i toggles every lamp whose number is a multiple of i, so lamp k is toggled once for each of its divisors. A lamp ends on exactly when it was toggled an odd number of times, i.e. when k has an odd number of divisors. Divisors come in pairs (d and k/d) except when k is a perfect square (where the square root pairs with itself), so the only lamps left on are the perfect squares. The whole problem collapses to: is k a perfect square? Check with an integer square root in O(1) per query.
One honest caveat (the assumption behind 2-51 and 2-52). Skiena specifies survival first and money second, but does not say how a pirate votes when both outcomes give him exactly the same survival and money. The solutions here use the standard convention that an indifferent pirate votes against the current proposal — equivalently, a non-proposer backs a proposal only if it makes him strictly better off in survival or money. Under this convention the outcomes below follow (the $300 case relies on it too: a pirate who would get $0 next round is paid $1, rather than assumed to vote yes for the same $0). A different tie-breaking rule can change the result.
You can analyse an algorithm and defend a choice with evidence. One question remains, and it is the one that matters most for your degree: why does any of this belong in mechatronics engineering? The capstone answers it — deadlines, tiny chips, and control loops that cannot wait. Read the engineering capstone →