Week 14 · Phase 4 · Choosing well

Putting It All Together

A checklist for choosing, the classic 'accidentally quadratic' traps, and the final project.

Given a new problem, how do I choose a method and explain why it works?

Lesson

Improve a method without changing its answer

One complete analysis
  1. specify the result
  2. test edge cases
  3. count expensive work
  4. choose a better structure
  5. compare results
  6. measure and explain

Example: decide whether two different positions contain values that sum to a target. Returning the same Boolean result is required before comparing speed.

Replace repeated searches with remembered values

Run in Colab · predict the result first
def two_sum(values, target):
    seen = set()
    for x in values:
        if target - x in seen:
            return True
        seen.add(x)
    return False
Trace [4, 1, 6, 3], target = 7
x
416
need
361
seen before check
set(){4}{4, 1}
result
continuecontinueTrue; stop

Check before adding x: otherwise a single 4 could incorrectly match itself for target 8. Two separate 4s may form a valid pair.

Write the time–memory tradeoff

MethodNo-pair input of length nExtra memory
Test every pair i < jn(n − 1)/2 pair tests; Θ(n²)Θ(1)
Remember previous valuesn set lookups; O(n) average totalO(n) worst case

The set method assumes bounded-cost arithmetic and ordinary hashing. Adverse collisions can make total time quadratic. Test empty input, one item, duplicate values, negative values and no valid pair.

Make the final evidence small and reproducible

Report itemWhat to show
CorrectnessSame outputs on a stated test set
ModelInput size, counted operation, time and memory bounds
ExperimentInput families, repeated timings and units
InterpretationWhy growth matches or differs from the count
LimitWhat the evidence does not establish

Use a profiler to locate expensive calls before rewriting code. tottime excludes called functions; cumtime includes them. Profiling adds overhead, so use separate benchmark runs for your final timings.

Final project targetone problem + two methods + one evidence table

Practice

Practice questions

10 test questions · 5 written questions · 15 total

Each question is followed by its answer and explanation. Hard questions also include a starting hint and smaller reasoning steps.

Test questions

Choose one option. Cost questions state their model and whether the bound is tight, expected or worst-case. Here lg means log₂, and heap positions start at 1.

Question 1 · medium · course question

A proposed optimisation runs faster but gives a wrong result on a legal input. What should you conclude?

  1. use it because speed is the only goal
  2. average its answer with another result
  3. it is not a correct replacement
  4. remove that input from the tests without changing the specification

Answer: option C. An optimisation must preserve the required behaviour on all legal inputs. Speed does not repair a correctness failure.

Question 2 · medium · course question

A program performs n membership tests on an ordinary unsorted list of n items, and each test scans the whole list. What is the total work?

  1. Θ(n²)
  2. Θ(n)
  3. Θ(log n)
  4. Θ(1)

Answer: option A. There are n tests, each taking Θ(n), so the work multiplies to Θ(n²). A suitable set can reduce expected lookup cost when its assumptions and setup costs are included.

Question 3 · medium · course question

A report says “algorithm A is best.” What information is most useful to add?

  1. a more impressive adjective
  2. the author’s favourite colour
  3. only the code font
  4. the objective, input assumptions and measured or proved costs

Answer: option D. “Best” depends on what matters: time, memory, deadlines or another objective. State the conditions and evidence supporting the choice.

Question 4 · medium · course question

A method has low average time but occasional long pauses. Which requirement makes those pauses particularly important?

  1. printing its source code
  2. meeting a strict per-request deadline
  3. having short variable names
  4. using an odd number of inputs

Answer: option B. A strict deadline concerns each request, so occasional slow cases matter even when the average is small. Expected or amortised bounds are not worst-case deadline guarantees.

Question 5 · medium · course question

Which test is especially useful when changing a sorting implementation?

  1. only one already sorted list
  2. only lists of length 100
  3. empty, single-item, repeated-key, sorted and reverse-sorted cases
  4. only the timing of a random list

Answer: option C. These cases exercise boundaries and common assumptions. They improve testing coverage, though testing alone is not a proof for every input.

Question 6 · medium · course question

A faster method uses much more memory. How should the choice be justified?

  1. compare both resources against the application’s limits
  2. ignore memory because only time counts
  3. always select the larger program
  4. assume every device has unlimited storage

Answer: option A. The suitable method depends on available memory and acceptable time. A speed improvement may be unusable on a memory-limited device.

Question 7 · medium · course question

Which claim is supported by testing an algorithm successfully on 1,000 randomly chosen inputs?

  1. it is proved correct on all inputs
  2. its worst-case time is known
  3. no edge cases remain
  4. it passed those 1,000 tests

Answer: option D. The observations support exactly the tested cases. A general correctness claim additionally needs reasoning about all legal inputs.

Question 8 · medium · course question

A preprocessing step costs 1,000 operations and each later query costs 10. Without preprocessing, each query costs 100. For 20 queries, which total is smaller?

  1. no preprocessing: 2,000 versus 3,000
  2. preprocessing: 1,200 versus 2,000
  3. they tie at 2,000
  4. preprocessing: 200 versus 1,000

Answer: option B. With preprocessing: 1,000+20×10=1,200. Without it: 20×100=2,000. Include both setup and query work.

Question 9 · medium · course question

For a new problem, which order is most useful?

  1. time arbitrary code, then decide what result was needed
  2. choose the shortest-looking code, then ignore its assumptions
  3. specify the required result, justify a method, then evaluate its resource costs
  4. select a complexity label before defining input size

Answer: option C. A precise specification supports correctness reasoning; only then can performance comparisons refer to the right task and input model.

Question 10 · hard

For n distinct keys and the standard implementations, which statement is FALSE?

  1. array heapsort can sort in place
  2. mergesort has Θ(n log n) worst-case time
  3. comparison sorting needs Ω(n log n) comparisons on average over uniformly random permutations
  4. random-pivot, two-way quicksort has Θ(n log n) worst-case time

In simpler words: Distinguish expected cost from worst-case cost.

Starting hint: Random choices can be unlucky even when their average is good.

Answer: option D. Random pivots give expected Θ(n log n) work on every fixed distinct-key input. Some pivot sequences still cause Θ(n²) work. Expected performance is not a worst-case guarantee.

Step by step
  1. Balanced pivot splits give about log n levels, with linear work per level.
  2. An unlucky sequence of extreme pivots leaves n−1,n−2,… items. Summing those costs gives Θ(n²), so (d) is false.

Written questions

Read each question together with its explanation, trace or proof. Numbering continues from the test questions.

Question 11 · easy

Give three reasons the course spends so much time on sorting.

Answer & reasoning

computers spend a large share of their time sorting; it is the best-studied problem, with many algorithms to compare; and most big ideas (divide and conquer, randomisation, lower bounds) can be taught through it. Also, once data is sorted many other problems become easy.

Question 12 · medium

Give two ways to find the k-th smallest of n numbers, one based on sorting and one on a heap, with their costs.

Answer & reasoning

sort and read position k: O(n log n). Or build a min-heap in O(n) and extract the minimum k times: O(n + k log n), which is better when k is small.

Question 13 · medium

Two sets of sizes m and n, with 2 ≤ m ≤ n, are supplied as unsorted arrays. Give a comparison-based method and an expected-time hashing method to test disjointness.

Answer & reasoning

Sort the smaller set in O(m log m), then binary-search each of the n larger-set items in it in O(n log m), for O((m + n) log m) total. Alternatively hash the smaller set and scan the larger in O(m + n) expected time under suitable hashing assumptions. Empty sets are immediately disjoint; for m = 1 a linear scan suffices.

Question 14 · medium

Why might quicksort run faster than heapsort on a particular machine even though both have n log n behaviour in the relevant case?

Answer & reasoning

Big-O hides constants and memory-access effects. Quicksort often has good locality and a simple inner loop, while heap operations jump between array positions. The actual comparison depends on input, implementation and hardware; measure it. Randomised quicksort is expected Θ(n log n) under the appropriate distinct-key or duplicate-handling assumptions and can have a quadratic worst case; heapsort has an O(n log n) worst-case guarantee.

Question 15 · hard

Prove that pairing the smallest number with the largest, the second smallest with the second largest, and so on, minimises the largest pair sum (the problem of the day from Lecture 7).

In simpler words: Pair very small values with very large values to control the worst pair.

Starting hint: Compare an arbitrary pairing with one that pairs the two extremes.

Answer & reasoning
Step by step
  1. Suppose the smallest a pairs with x and the largest z pairs with y.
  2. Replace these pairs by (a,z) and (x,y). Since a ≤ y and x ≤ z, both new sums are at most the old sum y+z.
  3. The maximum cannot increase. Fix the extremes together and apply the same argument to the remaining values. This is an exchange argument: improve the structure without worsening the objective.

sort the numbers a₁ ≤ … ≤ a₂ₙ. Take any optimal pairing in which a₁ is paired with some x and a₂ₙ with some y, x ≠ a₂ₙ. Re-pair them as (a₁, a₂ₙ) and (x, y). Since a₁ ≤ y, a₁ + a₂ₙ ≤ y + a₂ₙ; since x ≤ a₂ₙ, x + y ≤ a₂ₙ + y. Both new sums are at most the old pair sum a₂ₙ + y, so the maximum does not increase. Repeat the argument on the remaining numbers; by induction the fully sorted pairing is optimal.

Three core tasks

Predict on paper, run the code, then explain any difference. Use the animations below to inspect individual steps.

1. Trace

Test two_sum on [4] and [4, 4] with target 8. Explain why they differ.

Check your reasoning

[4] returns False: no earlier item exists. [4, 4] returns True on the second 4, using two positions.

2. Calculate

For 1,000 items with no valid pair, compare pair tests with set lookups.

Check your reasoning

499,500 pair tests versus 1,000 set lookups plus insertions. Set operations are average O(1); the counts are not a measured speedup.

3. Change one thing

Choose one course problem for the final project. Change the input family and predict the effect before measuring.

Check your reasoning

Examples: search with early hits versus misses; sorting ordered versus reversed data; anagrams with an early mismatch versus valid pairs. Keep outputs equivalent and state the new case.

Explore the animations & more worked tasks

14.7Workshop tasks

Task 1 — diagnose without running

Read this function, name its class, identify every problem, and rewrite it.

report.py
def build_report(orders, vip_customers):
    lines = []
    text = ""
    for order in orders:
        if order["customer"] in vip_customers:      # vip_customers is a list
            lines = lines + [order]
            text = text + order["id"] + ", "
    total = 0
    for order in lines:
        total = total + sum(o["value"] for o in lines)
    return lines, text, total
Animate it — hunt the four problems, then trace and count them
Solution

Four problems: (1) membership test against a list inside a loop → O(n·m); (2) lines = lines + [order] → quadratic; (3) text = text + … → quadratic; (4) the last loop recomputes the whole sum once per line → quadratic and wrong (it multiplies the total by the number of lines). Rewritten:

report_fixed.py
def build_report(orders, vip_customers):
    vips = set(vip_customers)                # O(m), once
    lines = [o for o in orders if o["customer"] in vips]   # O(n)
    text = ", ".join(o["id"] for o in lines)               # O(n)
    total = sum(o["value"] for o in lines)                 # O(n)
    return lines, text, total

With n orders, m watched names and L output characters, the repaired version costs O(n + m + L) on average. It also fixes the multiplied total and deliberately specifies comma-space separators without a trailing separator; that formatting differs from the original buggy text.

Task 2 — the interview question

Given a list of a million numbers, find whether any two of them add up to a given target. Write the obvious O(n²) version, then an O(n) version using a set. Benchmark both at 2 000, 4 000 and 8 000.

Animate it — every pair versus one pass with a set
The linear version
two_sum.py
def has_pair(data, target):
    seen = set()
    for x in data:
        if target - x in seen:      # O(1)
            return True
        seen.add(x)
    return False

One pass, constant work per item. The trick — remember what you have seen so that you never search — is the same one that made the anagram counter linear in week 9.

Task 3 — peer review

Swap final-project drafts with a classmate. For their work, answer: is the benchmark honest (data built outside the timer, repeats, worst case chosen)? Does the claimed class match the ratio column? Is there a limitation they did not mention? Write half a page.

Animate it — practise on a made-up draft first
Task 4 — the traps gallery

Write a small offender for each pattern in §14.3 and compare it with a version that avoids repeated work at n = 1 000, 2 000, 4 000, 8 000. Count the actual work before predicting ratios: repeated sorting may be O(n² log n) versus O(n log n), and interpreter-optimised strings may not display a quadratic curve.

Animate it — count each trap’s work before predicting
Hint on the string trap

Compare text += piece with "".join(pieces) using identical pieces and separators. Verify exact equality, including empty input and trailing spaces. The repeated-copy model predicts quadratic growth, but interpreter optimisations can change the measured result; report what you observe.

Task 5 — profile before you cut

Take the §14.4 summarise_slow and run it under cProfile.run(...) at n = 40 000. Which function dominates cumulative time? Then use a focused benchmark with identical orders and list versus set membership to investigate the suspected bottleneck. Write two sentences distinguishing the function-level profile from the source-line hypothesis.

Animate it — profile, form a hypothesis, test it
Expected

cProfile can identify summarise_slow as expensive, but cannot assign a separate line timing to in watch. The operation analysis predicts O(nm) membership work versus average O(n) dictionary updates; the controlled list/set comparison supplies additional evidence for that hypothesis.

Check your understanding

14.8Self-check

Your script is slow. What comes first?

Guessing wastes days. Time the sections, or profile — nearly all the time will be in a small part of the code.

Which single change most often fixes a beginner's slow program?

It converts O(n²) into O(n), which is a change of class — the only kind of improvement that keeps paying as the data grows.

n will always be about 50 rows. The right approach is:

Analysis tells you when growth matters. At fifty rows it does not, and clarity wins. Knowing when not to optimise is part of the skill.

A report claims "our new version is 40× faster". What is the first question to ask?

A constant-factor 40× stays 40×; a change of class grows without limit. Only the ratio-versus-size curve tells you which you bought.

In the repeated-copy model, why can concatenating n fixed-length words be quadratic?

When each step copies the growing prefix, the copied lengths form a triangular sum. Some interpreters optimise particular loops; collecting pieces and joining once avoids relying on that optimisation.

You need to add and remove items at the front of a collection millions of times. The right tool is:

insert(0, x) shifts every element and is O(n) each time; a deque adds and removes at both ends in O(1).
Extra material & reference
Optional depth · full technical reference

14.1The whole course on one page

ClassWhat it meansYou met it inTypical source
O(1)Cost ignores the data sizeweeks 2, 4, 11Indexing, a formula, dict/set lookup
O(log n)Each doubling adds one stepweeks 1, 3, 12Halving: binary search, bisect
O(n)One visit per itemweeks 3–5, 9, 11A single loop, sum, max, in on a list
O(n log n)A pass, repeated log n timesweeks 9, 12, 13sorted(), sort-then-search
O(n²)Every item meets every itemweeks 3, 9, 10, 13Nested loops; a hidden O(n) inside a loop
O(2ⁿ), O(n!)Unusable beyond tiny inputsweek 9Trying every combination or arrangement

14.2The choosing checklist

Six questions, in this order, for any new task. Most real decisions are settled by question three.

  1. How big is n, really? Ten rows forever? Then stop — write the clearest code and move on. Ten million, or growing? Continue.
  2. What is the dominant operation? Searching, counting, sorting, matching, or all-pairs comparison? Name it out loud.
  3. Am I searching inside a loop? If yes, a set or dictionary almost certainly turns O(n²) into O(n). Check this before anything else.
  4. Can I prepare once instead of repeating? Build the set, sort the list, compute the tally — once, outside the loop.
  5. Do I need order? Nearest, next, between-two-values: sorted list plus bisect. Only membership: set.
  6. What does it cost me? More memory, lost order, more code to maintain. Say so in your report rather than hiding it.
And then, always

Measure. A doubling experiment at four sizes takes ten minutes and settles arguments that otherwise run for weeks.

14.3Five patterns that hide repeated work

PatternLooks likeFix
Search inside a loopfor x in a: if x in b:b = set(b) before the loop
Building by concatenationresult = result + [x], text += wordappend, then "".join(...)
Working at the front of a listinsert(0, x), pop(0)collections.deque
Sorting inside a loopfor …: sorted(data)Sort once, outside
Recomputing a total every roundfor …: sum(data)Keep a running total

Every one of these is two lines of code that look innocent. If you remember only this table from the whole course, you will still write faster programs than most people who have never taken it. The exact class depends on the repeated operation: sorting n items n times can cost O(n² log n), while string concatenation depends on total character count and interpreter optimisations. Two common patterns deserve a closer look.

the string trap — predict which is faster
# Repeated concatenation can rebuild the growing text; interpreter optimisations may apply
text = ""
for word in words:              # n words ...
    text += word + " "             # trailing space is part of this output contract

# FAST: collect the pieces, join once at the end
parts = []
for word in words:
    parts.append(word)          # O(1) each
text = " ".join(parts) + (" " if parts else "")   # same trailing-space policy, including empty input

In the repeated-copy model, equal-length pieces lead to a triangular total of copied characters. Some Python contexts optimise repeated concatenation, so a quadratic timing curve is not guaranteed. Joining once costs O(L) in the total output length L; both snippets now preserve the same trailing-space policy. Measure rather than assuming the illustrative growth pattern.

the front-of-the-list trap
from collections import deque

# SLOW: insert(0, ...) shifts every existing element one place right → O(n) each
queue = []
for job in jobs:
    queue.insert(0, job)        # O(n) each → O(n²) overall

# FAST: a deque adds and removes at both ends in O(1)
queue = deque()
for job in jobs:
    queue.appendleft(job)       # O(1) each → O(n) overall

A list stores its items shoulder to shoulder, so making room at the front means sliding everything else along — O(n) every time. append at the end is O(1), which is why the fix for "I need the front too" is a deque, not a cleverer list.

14.4A worked case study, end to end

Everything in the course meets here. The task: from a big list of orders, total up how much each watch-listed customer spent. The obvious version is correct and slow; we will name why, measure it, fix it, and measure again.

case_study.py — the obvious version
import time, random

def make_data(n_orders, n_watch):
    names  = [f"cust{i}" for i in range(n_orders // 2)]
    orders = [{"customer": random.choice(names), "amount": random.randint(1, 100)}
              for _ in range(n_orders)]
    watch  = random.sample(names, n_watch)
    return orders, watch

def summarise_slow(orders, watch):
    totals = {}
    for o in orders:                          # n orders ...
        if o["customer"] in watch:            # ... scan of the watch list → O(m) each
            name = o["customer"]
            totals[name] = totals.get(name, 0) + o["amount"]
    return totals

def summarise_fast(orders, watch):
    watch_set = set(watch)                    # O(m), paid once
    totals = {}
    for o in orders:                          # n orders ...
        if o["customer"] in watch_set:        # ... O(1) each
            name = o["customer"]
            totals[name] = totals.get(name, 0) + o["amount"]
    return totals

for n in [5000, 10000, 20000, 40000]:
    orders, watch = make_data(n, n // 10)     # watch grows with n

    start = time.perf_counter()
    slow = summarise_slow(orders, watch)
    t_slow = time.perf_counter() - start

    start = time.perf_counter()
    fast = summarise_fast(orders, watch)
    t_fast = time.perf_counter() - start

    assert slow == fast                       # same answer — always check
    print(f"n={n:>6}  slow {t_slow:8.4f}s   fast {t_fast:.5f}s   {t_slow/t_fast:,.0f}x")
n= 5000 slow 0.0631s fast 0.00072s 88x n= 10000 slow 0.2498s fast 0.00140s 178x n= 20000 slow 0.9987s fast 0.00279s 358x n= 40000 slow 3.9901s fast 0.00560s 712x

Read the columns exactly as in weeks 5–13. The slow column quadruples each time n doubles: that is the 4× fingerprint of O(n²), and it appears because the watch list grows with the data, so in watch costs O(m) inside a loop of n — the pattern from row one of §14.3. The fast column merely doubles: O(n). And the speedup column grows without limit, because the two solutions are in different classes, not merely different by a constant.

The three moves, named

One change of container (list → set for the membership test) turned O(n·m) into O(n + m). One dictionary did the tallying in O(n) rather than a list-search per order. One assert checked agreement on this input. That is correctness evidence; a general argument must also explain why each eligible order contributes exactly once. That is the entire course in three lines — and it is exactly what your final-project report should show.

14.5Finding the slow part

Do not guess — programmers are famously bad at guessing which line is slow. Two tools, in increasing order of effort:

1. time the sections
import time

start = time.perf_counter()
data = load_data()
print(f"loading:   {time.perf_counter() - start:.3f}s")

start = time.perf_counter()
result = analyse(data)
print(f"analysing: {time.perf_counter() - start:.3f}s")

start = time.perf_counter()
save(result)
print(f"saving:    {time.perf_counter() - start:.3f}s")
2. let Python find it
import cProfile

cProfile.run("analyse(data)", sort="cumtime")   # slowest first

The 90/10 rule is a heuristic: a small part of the code may dominate the runtime, but measure whether that is true here. cProfile reports function-level costs, not individual source-line timings. Optimising anything else is wasted effort, however satisfying it feels.

Order of operations

Correct first, measured second, optimised third. An optimisation you cannot measure is a superstition, and a fast program that gives wrong answers is worthless — always re-check the output after a rewrite, exactly as the assert in §14.4 did.

14.6Writing the report

An optional practice report, and the format for any performance argument you make in future work or study:

  1. The problem — one paragraph, no code. What data, what question, what size now and what size expected.
  2. Approach A — the obvious one. Code, and its Big-O class with a one-line justification.
  3. Approach B — the considered one. Same treatment, plus what it costs you (memory, complexity, lost order).
  4. Evidence — a table at four sizes with a ratio column, and one labelled figure. State the machine, the repeat count, and whether you took best or mean.
  5. Interpretation — the five sentences from week 6, including a prediction for a size you did not run.
  6. Recommendation — which one, at what data size, and when you would change your mind.
  7. Limitations — one honest paragraph. What you did not test; where the measurement might mislead.
Practice emphasis, restated

Point 7 makes your reasoning credible. Every genuine measurement has a caveat, and naming yours is the difference between a student result and a professional one. "The 4 000-item run was on a shared Colab machine and may be inflated" costs nothing and buys all your other numbers credibility.

14.9Practice project — portfolio and feedback

Optional week 14 practice — ungraded

Pick a problem where an obvious solution and a considered one land in different complexity classes. Good choices reuse this course directly:

  • de-duplicating or cross-matching two large lists (weeks 10–11);
  • counting or grouping records — words, transactions, log lines (week 11);
  • repeated membership or range queries: scan vs sort-and-bisect vs set (week 12);
  • a sort-based versus count-based solution to a matching problem (weeks 9, 13);
  • fixing an "accidentally quadratic" script into a linear one (week 14).

For your own portfolio, you can prepare these three items. No submission is required:

  1. One notebook, AA_Final_YourName.ipynb, containing both approaches, the benchmark harness (data built outside the timer, at least three repeats, best or mean stated), and one labelled log–log figure.
  2. A report of about two pages following the seven-part structure in §14.6.
  3. A five-minute presentation: the problem, one figure, one recommendation, one limitation. No code on the slides.

Use this ungraded checklist to review reasoning and evidence. Assessment consists only of the midterm exam (50%) and final exam (50%). Weekly notebooks, exercises, projects, demonstrations and presentations are ungraded practice; no weekly submission is required.

CriterionWhat to look for
Problem framingClear question, the data described, current and expected n stated
Two correct approachesBoth run and agree on the answer (shown by an assert or equivalent)
Big-O analysisEach approach's class named and justified in one honest line
Benchmark qualityData outside the timer, repeats, four sizes, a ratio column
Interpretation and predictionClass read from the ratio column; a prediction for an un-run size
RecommendationA clear choice tied to data size, and when you would change it
Honesty and limitationsOne genuine caveat about the measurement or its scope
The standard, in one sentence

A well-measured 3× improvement with a clear explanation and an honest caveat beats an unexplained 1 000×. The checklist highlights the reasoning, because the reasoning is the skill that outlasts any one problem.

From the beginner notes · Lectures 1, 7, 19

Explain a pairing rule by swapping pairs

Suppose 2n task durations must be paired onto n work stations and the goal is to make the largest pair total as small as possible. Sort the durations and pair the smallest with the largest. A proof can explain why the rule works.

Take a pairing with the best possible largest total where the smallest a is paired with x and the largest b with y. Replace those pairs by (a,b) and (x,y). Since a ≤ y and x ≤ b, neither new sum exceeds the old sum b + y. The maximum cannot increase. Repeat on the remaining durations.

The goal matters. This proof covers pairs of two tasks with the goal of reducing the largest total; it does not solve every scheduling problem. When the task changes, check what has changed in the problem before reusing the rule. The optional extension introduces networks, dynamic programming and the limits of exact methods.

Engineering use. For your final design, state the objective, give a small worked case, justify the algorithm, and explain what changes would invalidate the claim.

Learning goals & class plan
By the end of this week you can
  • work through a checklist for choosing an approach to a new problem;
  • recognise five patterns that quietly add repeated work, and fix each;
  • follow one realistic problem from a slow first attempt to a measured, linear solution;
  • find the slow part of a program instead of guessing at it;
  • write a short, honest performance report and review one using a checklist;
  • say what you would learn next, and why.
Three-hour interactive studio

00:00–01:00: launch, predict–run–explain cycle, questions  ·  01:00–01:10: break  ·  01:10–02:00: worked variation, peer instruction, questions  ·  02:00–02:10: break  ·  02:10–03:00: core mechatronics practice, exam bridge, and exit ticket.

Ask at any point. Weekly self-checks stay private; optional extensions are not collected.

Need a slower explanation? Open the English + Türkçe reference guide.

Where to go next

14.10Where to go next

Recursion and divide-and-conquer

Write merge sort and quicksort yourself, and see where n log n really comes from.

More data structures

Stacks, queues, linked lists, trees, heaps and graphs — each solving an operation lists do badly.

Space complexity

The same analysis applied to memory. Sometimes the limit is what fits, not what finishes.

Vectorised computing

NumPy and pandas: the same O(n) work with constants tens of times smaller.

The next course

Data Structures & Algorithms assumes exactly what you now know.

The textbook

Problem Solving with Algorithms and Data Structures — free, interactive, and the source of week 9.

Fourteen weeks ago the question was "does this program work?" You now ask a better one: "and what happens when there is ten times more data?" That question is the entire subject, and you can answer it with a stopwatch, a table and a plot. Well done.

Optional reference · Words from this week

14.11Words from this week

TermMeaning in plain words
profilingMeasuring which parts of a program consume the time.
90/10 ruleAlmost all the time is spent in a small fraction of the code.
premature optimisationMaking code faster before knowing it is slow, or where.
accidentally quadraticInnocent-looking code that hides an O(n) step inside a loop, making it O(n²).
bottleneckThe part that limits the whole; the only part worth optimising.
trade-offWhat the faster version costs you: memory, clarity, order, flexibility.
Chapter problem set — Skiena 2.10

14.12Chapter problem set — Skiena 2.10

A capstone deserves capstone puzzles. These last problems from Skiena's The Algorithm Design Manual are the kind asked in interviews: each one rewards a clear idea over brute force, and each connects back to a habit this course has drilled — prepare once, count carefully, reason backwards. Read, attempt, then unfold the solution.

Problem 2-43 · sampling a stream in one pass

From a set S of n numbers, choose a subset S′ of k numbers so that every element of S is picked with equal probability k/n — in a single pass. Now the twist: do it when n is not known in advance (the numbers arrive one by one and stop when they stop).

Worked solution

This is reservoir sampling. Start with the easy case, k = 1 — keep a single item. The rule: when the i-th item arrives (counting from 1), keep it with probability 1/i, otherwise stick with what you already hold. Why does that give everyone the same 1/n at the end? The last item, number n, is kept with probability 1/n directly. An earlier item j was kept when it arrived (prob 1/j) and then survived every later item not replacing it: (1 − 1/(j+1)) × (1 − 1/(j+2)) × … × (1 − 1/n). Those brackets are j/(j+1) × (j+1)/(j+2) × … × (n−1)/n, a telescoping product that collapses to j/n. Multiply by the 1/j from the start and you get 1/n. Everyone ties.

For general k, keep the first k items as the initial "reservoir". When the i-th item arrives (i > k), keep it with probability k/i; if you keep it, drop one of the k held items chosen at random to make room. The same telescoping argument gives every item a final probability of exactly k/n.

reservoir.py
import random

def reservoir(stream, k):
    keep = []
    for i, item in enumerate(stream):     # i counts from 0; n never needed
        if i < k:
            keep.append(item)             # fill the reservoir first
        else:
            j = random.randint(0, i)      # 0..i inclusive → keep with prob k/(i+1)
            if j < k:
                keep[j] = item
                # evict a random held item and insert the new one
    return keep

Answer: reservoir sampling. Hold the first k items; for each later item i keep it with probability k/i, replacing a random current member. It makes one pass, stores only k items, and never needs to know n ahead of time — perfect for a stream too big to hold or of unknown length. A quick simulation (run reservoir many times over range(100) with k = 10 and tally how often each number appears) confirms every element shows up almost exactly k/n of the time.

Problem 2-44 · replicating to survive failures

You store 1000 items across 1000 nodes; each node holds 3 different items. Propose a replication scheme that minimises data loss when nodes fail, and estimate how many items you expect to lose when 3 randomly chosen nodes fail at once.

Worked solution

The core idea is simple: keep three copies of every item, on three well-separated nodes, and never let two items share the exact same trio of nodes. An easy scheme that does this: put item i on nodes i, i+1 and i+2 (wrapping round mod 1000). Every item now has three homes, every node still holds three items, and each item's trio is a distinct set of three consecutive nodes.

An item is lost only when all three of its nodes are down. If just 3 random nodes fail, a given item is lost only if those 3 failures are precisely its 3 nodes. The chance of that for one item is 1 divided by the number of ways to choose 3 nodes from 1000 — that is 1 / C(1000, 3) = 1 / 166 167 000, a tiny number. Add that up over all 1000 items:

expected items lost ≈ 1000 × 1 / C(1000, 3) ≈ 6 × 10⁻⁶

Answer: three copies per item, spread across distinct nodes so no two items share a trio. With 3 random node failures the expected loss is essentially zero — almost always no item is lost at all. With distinct trios, exactly three failed nodes can destroy at most one item's three copies. The expected loss above is the same for any placement with three distinct nodes per item under uniformly random three-node failures; overlapping trios change the distribution and severity of losses, not this expectation. Correlated physical failures require a separate placement model.

Problem 2-49 · counting the ways to merge

n companies gradually merge until a single company remains. Each merge combines two companies into one. How many different sequences of pairwise merges are possible?

Worked solution

First the easy half: each merge turns two companies into one, so it reduces the count by exactly one. Going from n down to 1 therefore always takes n − 1 merges, no matter the order. The interesting question is how many ordered ways those merges can play out. Do the small cases by hand:

nwayswhy
21only one pair to merge
33first merge is one of the 3 pairs (AB, AC, BC); then only one pair is left
418more choices at each of the 3 merges

The closed form that produces 1, 3, 18, … is:

number of merge sequences = n! · (n−1)! / 2n−1

Check it: n = 2 gives 2 · 1 / 2 = 1; n = 3 gives 6 · 2 / 4 = 3; n = 4 gives 24 · 6 / 8 = 18. All match.

Answer: n − 1 merges are always needed, and the number of distinct ordered merge sequences is n!(n−1)! / 2n−1, matching the hand counts 1, 3, 18 for n = 2, 3, 4.

Optional challenge — where the formula comes from

With k current companies, choose an unordered pair in k(k−1)/2 ways. There are then k−1 companies, regardless of which pair merged. Multiply the choices for k = n, n−1, …, 2: the first factors give n!, the second give (n−1)!, and the n−1 denominators give 2n−1. Thus the number of ordered merge sequences is n!(n−1)! / 2n−1. For n = 4, this is 6 × 3 × 1 = 18. Companies are distinguished by their original members, and choosing A with B is the same single merge as choosing B with A.

Problem 2-50 · Ramanujan numbers

A Ramanujan number can be written as a sum of two cubes in two different ways: a³ + b³ = c³ + d³ (with distinct a, b, c, d). Find all of them with a, b, c, d < n, faster than trying every four numbers.

Worked solution

Four nested loops would be O(n⁴). Instead use the "remember what you have seen" trick: build every two-cube sum once and look for collisions. Go over each pair (a, b) with a ≤ b < n, and store a³ + b³ in a dictionary keyed by the sum. Any key hit by two or more different pairs is a Ramanujan number.

ramanujan.py
def ramanujan(n):
    seen = {}                     # sum -> list of (a, b) pairs
    for a in range(1, n):
        for b in range(a, n):     # a <= b avoids double-counting
            seen.setdefault(a**3 + b**3, []).append((a, b))
    return {s: p for s, p in seen.items() if len(p) >= 2}

print(sorted(ramanujan(20)))
[1729, 4104]

The smallest is 1729 = 1³ + 12³ = 9³ + 10³. The input bound n = 20 also includes 4104 = 2³ + 16³ = 9³ + 15³. The printed expression sorts the dictionary's sum keys, not its key/value pairs. The algorithm visits approximately n²/2 pairs with O(1) average dictionary work per pair under the usual fixed-size arithmetic model.

Answer: hash every a³ + b³ and keep the sums reachable two ways — O(n²) time and space, versus brute-force O(n⁴). Smallest Ramanujan number: 1729.

Problem 2-51 & 2-52 · the rational pirates

Six pirates must divide $300 by seniority. The most senior proposes a split; all pirates (proposer included) vote; if at least half agree, it passes, otherwise the proposer is thrown overboard and the next most senior proposes. Pirates are perfectly rational and value, in order: staying alive, then money. How is the $300 divided (2-51)? And what happens if there is only one indivisible dollar to share (2-52)?

Worked solution

The trick to every puzzle like this is to reason backwards from the smallest case, because each pirate, when voting, compares "what I get now" against "what I would get if this proposer is thrown overboard". Label the pirates P1 (most junior) up to P6 (most senior); the senior proposes first. "At least half" means a tie is enough to pass.

Part 2-51, dividing $300. Build up one pirate at a time:

pirates leftvotes neededproposer keepswho gets $1 (to buy the vote)
2 (P2, P1)1$300nobody — P2's own vote is half of 2
3 (P3…)2$299P1 (who would get $0 if it fell to 2 pirates)
4 (P4…)2$299P2 (who gets $0 in the 3-pirate outcome)
5 (P5…)3$298P3 and P1 (the two who get $0 with 4 pirates)
6 (P6…)3$298P4 and P2 (the two who get $0 with 5 pirates)

Each proposer only has to bribe the cheapest pirates — the ones who would get nothing in the next round — and just enough of them to reach half the votes. So with six pirates, the senior P6 needs 3 votes (his own plus two), and he secures them by handing $1 each to P4 and P2, keeping the other $298 for himself.

Answer (2-51): the top pirate keeps $298 and gives $1 each to two specific juniors (P4 and P2) to secure a passing vote of 3-to-3; P5, P3 and P1 get nothing. Rationality, not generosity, produces the lopsided split — the juniors accept $1 because the alternative is $0.

Part 2-52, a single indivisible dollar. Now the proposer has almost nothing to bribe with, so survival votes do the work. Reasoning down the chain, with 5 pirates the proposer P5 would need 3 votes but can bribe at most one pirate with the lone dollar — so a 5-pirate proposal fails and P5 is thrown overboard. That fact is the key: it means P5 is desperate to avoid ever reaching a 5-pirate game.

So when 6 pirates face the single dollar, the senior P6 needs 3 votes and gets them like this: his own vote; P5's vote for free (because if P6 is thrown over, the game drops to 5 pirates where P5 dies, so P5 will support almost any proposal that keeps P6 afloat); and one more vote bought by giving the single dollar to one of the pirates who would otherwise get nothing (P4, P3 or P1). That is a winning coalition of three, so the proposal passes.

Answer (2-52): the top pirate survives and no one need die. He gives the one dollar to a single pirate whose vote he needs and relies on the vote of P5, who backs him purely to avoid being thrown overboard in the next round. The dollar plus one life-saving vote is exactly enough to form a majority — a neat demonstration that "value survival first" can be worth more than money at the table.

Programming challenges · optional practice on the judges

Skiena pairs the chapter with three short programming problems on the online judges (UVA). Each is a one-idea problem — no heavy machinery, just the right observation. Try them for practice; a sketch of the approach is enough to get you started.

Approaches

Primary Arithmetic — count the carry operations when two unsigned numbers are added. Add the two numbers the grade-school way, digit by digit from the right, keeping a running carry; each position where the column plus the incoming carry reaches 10 produces a carry into the next column. Count those events. It is O(number of digits), and the whole problem is really just "do primary-school addition and watch the carries" — report none, one, or the count.

A Multiplication Game — two players start from a product of 1 and take turns multiplying it by any integer from 2 to 9; the first to reach or pass n wins, and the first player (Stan) moves first. This is a small combinatorial game: reason backwards from n about which running totals are winning positions for the player about to move. With optimal play the winner depends only on which "band" n falls in, and the band edges are 9, 18, 162, 324, 2916, … — the gaps alternate ×2 then ×9 (not "powers of nine"): Stan wins for n in 2–9, the second player for 10–18, Stan for 19–162, the second player for 163–324, Stan for 325–2916, and so on. A clean rule: repeatedly replace n by ⌈n/18⌉ until n ≤ 18, then Stan wins exactly when the leftover n ≤ 9. No search is needed — just a handful of divisions.

Light, More Light — lamps numbered 1..N start off; person i toggles every lamp whose number is a multiple of i, so lamp k is toggled once for each of its divisors. A lamp ends on exactly when it was toggled an odd number of times, i.e. when k has an odd number of divisors. Divisors come in pairs (d and k/d) except when k is a perfect square (where the square root pairs with itself), so the only lamps left on are the perfect squares. The whole problem collapses to: is k a perfect square? Check with an integer square root in O(1) per query.

One honest caveat (the assumption behind 2-51 and 2-52). Skiena specifies survival first and money second, but does not say how a pirate votes when both outcomes give him exactly the same survival and money. The solutions here use the standard convention that an indifferent pirate votes against the current proposal — equivalently, a non-proposer backs a proposal only if it makes him strictly better off in survival or money. Under this convention the outcomes below follow (the $300 case relies on it too: a pirate who would get $0 next round is paid $1, rather than assumed to vote yes for the same $0). A different tie-breaking rule can change the result.

Optional next topics: graphs, dynamic programming & hard problems

These are extensions beyond the 14-week sequence, with explanations and further source exercises. There is no additional submission.

Explore one extension →
Where this leads

You can analyse an algorithm and defend a choice with evidence. One question remains, and it is the one that matters most for your degree: why does any of this belong in mechatronics engineering? The capstone answers it — deadlines, tiny chips, and control loops that cannot wait. Read the engineering capstone →