Week 05 · Phase 2 · Measuring for real

Functions and the Stopwatch

Wrap work in a function, then time it honestly with perf_counter and repeats.

How long does my code actually take?

Lesson

Give a repeated method a name

Run in Colab · predict the result first
def contains(values, target):
    for x in values:
        if x == target:
            return True
    return False

print(contains([4, 7, 9], 7))
Function call
  1. inputs: [4, 7, 9], 7
  2. check 4 → check 7
  3. return True; stop

A parameter names an input. return sends a result back and ends the call; print only displays it. The final return False belongs after the loop, so all candidates get a chance.

Time only the work you are comparing

Benchmark boundary
  1. build dataoutside timer
  2. start = perf_counter()
  3. call the method
  4. elapsed = perf_counter() − start
Run in Colab · predict the result first
from time import perf_counter
values = list(range(100_000))
start = perf_counter()
found = contains(values, -1)
elapsed = perf_counter() - start
print(found, elapsed)

The missing target forces a full scan. Compare methods on the same data and target. Keep printing and data creation outside the timed region unless those are part of the question.

Repeat before trusting a number

Illustrative timings for five runs:

Run12345
Time (ms)4.04.17.24.04.2
Same method, five trialsminimum = 4.0 ms; mean = 4.7 ms

The minimum can approximate an uninterrupted run; the mean includes interruptions. Report which you use. Warm up once, then repeat; background work and timer resolution can overwhelm tiny measurements.

Run in Colab · predict the result first
from timeit import repeat
samples = repeat(lambda: contains(values, -1),
                 number=100, repeat=5)
seconds_per_call = min(samples) / 100
print(seconds_per_call)

Keep evidence separate from a claim

RecordExample
Input familyn distinct integers; target absent
Correctness checkreturns False
Work modeln equality checks
Measurementseconds per call, repetitions and machine
LimitDoes not describe early matches

A faster run is evidence about this experiment. It does not prove that a method is faster for every input or every machine.

Practice

Practice questions

10 test questions · 1 written questions · 11 total

Each question is followed by its answer and explanation. Hard questions also include a starting hint and smaller reasoning steps.

Test questions

Choose one option. Cost questions state their model and whether the bound is tight, expected or worst-case. Here lg means log₂, and heap positions start at 1.

Question 1 · easy

The worst-case running time of an algorithm is:

  1. its time on a typical input
  2. the maximum steps over all inputs of size n
  3. the minimum steps over all inputs of size n
  4. its time on the first input tried

Answer: option B. It is a guarantee, which is why the course uses it by default.

Question 2 · medium · course question

If start = 10.25 and end = 10.29 are stopwatch readings in seconds, approximately how much time elapsed?

  1. 0.04 seconds
  2. 10.29 seconds
  3. 20.54 seconds
  4. 4 seconds

Answer: option A. Elapsed time is end − start = 0.04 seconds, or about 40 milliseconds. Floating-point representation can introduce a tiny rounding difference.

Question 3 · medium · course question

Which timing placement measures the operation f(data) itself, excluding setup?

  1. start; build data; f(data); end
  2. f(data); start; end
  3. build data; start; f(data); end
  4. start; end; f(data)

Answer: option C. Prepare the data first, then record the start, run the operation and record the end. Otherwise setup is mixed into the measured interval.

Question 4 · medium · course question

Why repeat a short timing measurement?

  1. to prove correctness
  2. to force every run to be identical
  3. to change the algorithm’s growth class
  4. to see variation and reduce reliance on one noisy run

Answer: option D. Other processes, caches and clock resolution can affect measurements. Repeated observations help reveal this variation; they do not by themselves prove a complexity bound.

Question 5 · medium · course question

A batch of 100 calls takes 0.2 seconds. Ignoring loop overhead, what is the mean time per call?

  1. 20 seconds
  2. 0.002 seconds
  3. 0.2 seconds
  4. 500 seconds

Answer: option B. Divide the batch time by the number of calls: 0.2 / 100 = 0.002 seconds, or 2 milliseconds.

Question 6 · medium · course question

A Python function reaches its end without executing a return statement. What value does it return?

  1. 0
  2. an empty string
  3. None
  4. the last value it printed

Answer: option C. The implicit return value is None. Printing a value displays it but does not make it the function’s return value.

Question 7 · medium · course question

A function contains return x * 2. When called with x = 5, what value does it return?

  1. 10
  2. 5
  3. 25
  4. None

Answer: option A. Substitute 5 for x: 5 × 2 = 10. return gives that value back to the caller.

Question 8 · medium · course question

You time an in-place sorting function twice on the same list. Why might the second run be an unfair comparison?

  1. the second clock cannot work
  2. sorting always doubles the list
  3. the list becomes a set
  4. the first run may have left the input already sorted

Answer: option D. If the operation changes its input, later runs may receive a different input case. Prepare an equivalent fresh input for each comparison and decide whether copying belongs inside the timed interval.

Question 9 · medium · course question

Which statement is justified by a largest observed runtime of 8 ms over 20 trials?

  1. the algorithm always finishes within 8 ms
  2. 8 ms was the largest time in those trials
  3. every other computer will be faster
  4. its worst-case complexity is constant

Answer: option B. Measurements describe the runs you performed. A finite set of trials does not establish a universal deadline or an asymptotic complexity class.

Question 10 · medium · course question

Which pair is most suitable for comparing two algorithms’ work fairly?

  1. different inputs and different machines
  2. only each algorithm’s fastest observed input
  3. the same input cases and a consistent measurement method
  4. one timed run compared with the other’s source-code length

Answer: option C. Hold relevant conditions constant and include the same categories of work. Otherwise differences may come from setup, inputs or measurement rather than the algorithms.

Written questions

Read each question together with its explanation, trace or proof. Numbering continues from the test questions.

Question 11 · easy

Define worst-case running time in one sentence, and say why the course prefers it.

Answer & reasoning

Worst-case running time is the maximum cost over all legal inputs of a fixed size n, under the stated cost model. It provides an upper guarantee, useful when a controller has a deadline. It need not be close to average-case cost. A bound on operation count is not by itself a measured or certified time bound on a real controller.

Three core tasks

Predict on paper, run the code, then explain any difference. Use the animations below to inspect individual steps.

1. Trace

Call contains([4, 7, 9], 7), then contains([4, 7, 9], 2). Count equality checks.

Check your reasoning

True after 2 checks; False after 3 checks.

2. Calculate

A batch of 100 calls takes 0.42 seconds. What is the time per call?

Check your reasoning

0.42 / 100 = 0.0042 seconds = 4.2 ms.

3. Change one thing

Move the target from absent to the first item. What changes in the experiment?

Check your reasoning

The scan now makes one equality check. This is a different input case, so record it separately.

Explore the animations & more worked tasks

5.7Try it yourself

Task 1 — functions from last week

Turn the linear search from week 4 into def contains(data, target) that returns True or False. Test it on a small list.

Animate it — follow two calls and an early return
Solution
contains.py
def contains(data, target):
    for item in data:
        if item == target:
            return True      # return leaves the function immediately
    return False

print(contains([3, 9, 4], 9), contains([3, 9, 4], 5))
Task 2 — time the worst case

Time contains searching for a missing item in lists of 100 000, 200 000, 400 000 and 800 000 items. Build each list before starting the clock. Report the ratio column.

Animate it — build the ratio column at doubling sizes
What you should see

Ratios close to 2.0. A missing item forces a full pass, so the time follows the length of the list exactly. If your ratios are wild (0.4, 6.1), your n is too small or you timed the list construction.

Task 3 — a deliberately dishonest benchmark

Move the line that builds the test list inside the timed region and re-run Task 2. What happens to the numbers, and why does this no longer isolate search time?

Animate it — move the list-building into the timed region
Answer

Times may increase while the ratios still look roughly linear. You are now measuring "build a list of n items" plus search. That is a valid end-to-end measurement if labelled accordingly, but it cannot isolate search cost. State exactly what is inside the timed region.

Task 4 — see the warm-up for yourself

Time sum_to(2000000) eight times in a row with a plain perf_counter loop, printing each run. Then print the fastest and the mean of all eight, and the fastest and the mean of runs 2–8 only (dropping the first). Which statistic changed the most when you dropped run 1, and why?

Animate it — drop run 1 and watch the fastest and the mean
What to look for

If the first run is unusually slow, removing it lowers the mean and leaves the minimum unchanged. If it is the fastest, the minimum changes too. Report what your samples actually show; a minimum is not a substitute for an explicit warm-up policy.

Task 5 — perf_counter vs timeit

Measure the cost of one call to sum_to(1000) two ways: once with a single perf_counter pair around one call, and once with timeit.timeit("sum_to(1000)", globals=globals(), number=10000) divided by 10 000. Which gives the steadier answer across three repeats, and why?

Animate it — compare single readings with a timeit batch
Answer

The single perf_counter reading jumps around — a 41-microsecond call is only a few hundred times the measurement overhead, so a single interruption swamps it. The timeit batch can be steadier because repeating 10 000 runs turns each fixed disturbance into a tiny percentage (§5.6, "noise is a percentage"). For calls this fast, timeit is the correct instrument.

Task 6 — relative noise shrinks with n

For n = 1 000 and again for n = 2 000 000, time sum_to(n) ten times and report the spread as a percentage: (slowest − fastest) / fastest × 100. Explain the two numbers using the idea from §5.6.

Animate it — see a fixed disturbance shrink to a rounding error
Expected

A disturbance of a fixed duration is a smaller percentage of a longer run. Your measured spreads need not decrease monotonically: background work and cache or frequency changes may differ between runs. Report the actual percentages and use repetition to judge whether a ratio is stable.

Check your understanding

5.8Self-check

A function without a return statement gives back:

It runs happily and hands back None. Printing inside a function is not the same as returning.

Why take the fastest of several runs rather than the slowest?

Interruptions can add delay, but changing machine state and measurement overhead still matter. The minimum is a summary, not a guaranteed intrinsic cost.

Your benchmark reports 0.00002 s for n = 10 and 0.00003 s for n = 20. What should you do?

At this scale you are measuring the clock and the loop overhead, not the algorithm. Grow n until the signal is well above the noise.

The very first of five identical timed runs is noticeably slower than the other four. The usual reason is:

A slow first call may include start-up costs. An explicit untimed warm-up is appropriate for steady-state studies; the minimum alone is not a guarantee.

You want to measure one call to a snippet that takes about 40 microseconds. The best tool is:

At tens of microseconds a single reading is mostly overhead. timeit repeats the snippet many times so each fixed disturbance becomes a negligible percentage.
Extra material & reference
Optional depth · full technical reference

5.1A function is a named piece of work

Up to now, changing n meant editing the code and running it again. A function lets you say "do that job, with this input" as many times as you like.

def, parameter, return
def sum_to(n):
    total = 0
    for number in range(1, n + 1):
        total = total + number
    return total

print(sum_to(10))
print(sum_to(1000))
55 500500

Three parts, and each has a job:

  • def sum_to(n): — the name and the parameter it expects.
  • the indented block — the work, which can use n as if it were a variable.
  • return total — hand the answer back. Without it, the function gives back None, which is the most common cause of "why is my result empty?"

A useful picture: a function is a recipe card. Writing the card teaches the kitchen a dish; it does not cook anything. You only get food when someone reads the card and follows it — that is a call. The parameter n is the blank on the card that says "however many portions you asked for", filled in fresh each time the card is used.

Defining is not running

Running the def block produces no output — it only teaches Python the word. Nothing happens until you call it: sum_to(10).

5.2The stopwatch

Python's time module has a high-resolution clock. Read it before, read it after, subtract.

timing pattern — memorise this
import time

start = time.perf_counter()
result = sum_to(1000000)
elapsed = time.perf_counter() - start

print(f"result {result}, took {elapsed:.4f} seconds")
result 500000500000, took 0.0521 seconds

Your number will differ from mine, and from your own number thirty seconds from now. That is not a flaw in the method — it is the first thing you have to learn to handle.

Use perf_counter, not the clock on the wall

time.perf_counter() is designed for measuring short intervals. Do not use time.time() for benchmarks — it can jump when the system clock adjusts. The number it returns is meaningless on its own; only the difference matters.

5.3One measurement is not a measurement

Your computer is doing dozens of other things while it runs your loop. Time the same call five times and you will get five different answers:

repeat and take the best
import time

def time_it(func, n, repeats=5):
    """Run func(n) several times, return the fastest time in seconds."""
    best = None
    for _ in range(repeats):
        start = time.perf_counter()
        func(n)
        elapsed = time.perf_counter() - start
        if best is None or elapsed < best:
            best = elapsed
    return best

print(f"{time_it(sum_to, 1000000):.4f} s")

This little time_it is the harness the whole course leans on. Every week from here on hands it a function and a size and reads back a trustworthy number, so it is worth understanding line by line now.

Why the fastest run? Interruptions often add delay, so the minimum can estimate a lightly interrupted run. It is not a guaranteed true cost: cache state, CPU frequency, measurement overhead and the chosen input also matter. Report the individual samples and say whether you used a minimum, median or mean.

The three ways a benchmark lies
  1. Too small. Very short individual runs can be dominated by measurement overhead and variation. For this classroom harness, aim for roughly 0.05 s or batch repeated calls; inspect the spread rather than treating a fixed threshold as a guarantee.
  2. Something else was running. Close the video tab. On Colab, other people's load can bleed in — repeat and take the best.
  3. You measured the wrong thing. Building a million-item test list inside the timed block measures list-building plus your algorithm. Build the data before start.

5.4Your first benchmark table

Now the pattern that every remaining week of this course uses. Same function, several sizes, one table:

benchmark.py
sizes = [100000, 200000, 400000, 800000]

print(f"{'n':>10} {'seconds':>12} {'ratio':>8}")
previous = None
for n in sizes:
    t = time_it(sum_to, n)
    ratio = t / previous if previous else 1.0
    print(f"{n:>10} {t:>12.5f} {ratio:>8.2f}")
    previous = t
n seconds ratio 100000 0.00512 1.00 200000 0.01031 2.01 400000 0.02064 2.00 800000 0.04119 2.00

Read both the ratio and seconds columns. Ratios can reveal a growth pattern when machine effects approximately cancel, but they still depend on the measured conditions. Near-doubling times support the linear model here; the loop count explains it for all n. Next week you will use ratios as evidence when distinguishing growth patterns.

5.5Two functions, one job

There is a closed-form formula for the same sum: n(n+1)/2. Same answer, different method:

loop vs formula
def sum_formula(n):
    return n * (n + 1) // 2

for n in [1000, 100000, 10000000]:
    a = time_it(sum_to, n)
    b = time_it(sum_formula, n)
    print(f"n={n:>9}  loop {a:.6f}s   formula {b:.9f}s   x{a/b:,.0f} faster")
n= 1000 loop 0.000041s formula 0.000000160s x256 faster n= 100000 loop 0.004900s formula 0.000000170s x28824 faster n= 10000000 loop 0.512000s formula 0.000000180s x2844444 faster

Look at the formula column: it barely moves. The loop's time grows with n; the formula performs a fixed number of arithmetic operations when each arithmetic operation counts as one step. Python's arbitrarily large integers make individual arithmetic costs grow with bit length. And notice the last column growing too — the gap between the two methods widens with the data. That is the thing this course teaches you to see coming, before you write the slow one.

5.6Going deeper: warm-up, resolution and timeit

You have a working stopwatch. Three refinements separate a rough timing from one you would stake a grade — or a production decision — on.

The first run is a little lie

Time the same call five times in a row and watch the very first number:

watch the first run
import time

data = list(range(1000000))          # built once, before timing

for run in range(5):
    start = time.perf_counter()
    total = sum(data)
    print(f"run {run}: {time.perf_counter() - start:.5f} s")
run 0: 0.01204 s run 1: 0.00781 s run 2: 0.00773 s run 3: 0.00772 s run 4: 0.00774 s

The first run may be slower because of one-off setup, cold caches or runtime warm-up. This is not guaranteed. A deliberately untimed first call is a warm-up; use and report it when the intended measurement is steady-state performance.

Why our harness already survives this

Look again at time_it: it takes the fastest of the repeats. The first run may be slow, but taking the minimum does not guarantee that warm-up effects have disappeared. For steady-state measurements, add an explicit untimed warm-up call before the loop (func(n) on its own, result ignored), then repeat and report your summary policy. Keep setup costs when they are part of the real task.

How fine is the ruler?

A stopwatch that ticks in whole seconds cannot measure a 3-millisecond event. Every clock has a smallest step it can see, its resolution. Ask Python what yours is:

clock info
import time

info = time.get_clock_info("perf_counter")
print("resolution:", info.resolution, "seconds")
print("monotonic:", info.monotonic)
resolution: 1e-09 seconds monotonic: True

A nanosecond sounds plenty fine, and the raw clock is. But the practical floor is much higher: the overhead of calling a Python function and reading the clock twice is itself hundreds of nanoseconds, and that is your real smallest trustworthy interval. This is the arithmetic behind the rule from §5.3 — if a run lasts only a few times the smallest step you can measure, the trailing digits are decoration. Grow n until the time is a comfortable multiple of the resolution. (monotonic: True confirms the clock only ever moves forwards, which is why a difference can never come out negative.)

Let the standard library do it: timeit

For a very fast snippet — a single arithmetic expression, a one-line call — perf_counter around one call measures mostly its own overhead. The timeit module exists for exactly this: it runs the snippet many times, reports the total, and turns off the garbage collector during timing to remove one source of noise.

timeit for tiny snippets
import timeit

# run sum_to(1000) ten thousand times, get the total, divide back
total = timeit.timeit("sum_to(1000)", globals=globals(), number=10000)
per_call = total / 10000
print(f"{per_call * 1e6:.2f} microseconds per call")
41.30 microseconds per call

Use the right tool for the scale. For the tiny, use timeit; for the doubling experiments in the rest of the course — where each input size takes tens of milliseconds and you want its own number and a ratio — the time_it harness is the better fit. Both read the same underlying perf_counter; they just package it differently.

Why noise is a percentage, not an amount

When the operating system steals the CPU to service the network card, it takes a roughly fixed slice of time — say a quarter of a millisecond. On a 0.5 ms run that is a 50% error; on a 2-second run it is 0.0125% (0.25 ms / 2 000 ms × 100). The interruption is the same size; what changes is its share of your measurement. That is the deep reason we grow n: not to make the algorithm slower for its own sake, but to make every fixed disturbance shrink to a rounding error relative to the signal.

Report noise the honest way

"It varied by 0.3 ms" tells nobody anything. "It varied by 40% at n = 1 000 but under 1% at n = 1 000 000" tells the whole story — and shows you understand where your numbers are trustworthy and where they are not.

From the beginner notes · Lectures 2, 3

A stopwatch answers a question about an input

The notes separate best, worst and average cost over inputs of the same size. A timing measurement is one observation on one machine, implementation and input. One run does not tell you the best, worst or average time across all inputs of that size.

Searching a hundred readings can stop at the first reading, stop at the last, or exhaust the list without finding the target. Keep n fixed and compare those cases. Repeat measurements and record what setup work you included. The worst observed time is evidence about your trials, not proof of the worst possible time.

Counting how many operations the method needs helps explain the measured time. A real controller deadline additionally depends on the computer, memory access and other tasks running at the same time.

Engineering use. Write “largest time in these trials” rather than “guaranteed maximum” unless you have established that guarantee.

Learning goals & class plan
By the end of this week you can
  • write a function with def, give it inputs and get a value back;
  • time a piece of code with time.perf_counter();
  • explain why one measurement is never enough;
  • run the same function at several input sizes and put the results in a table;
  • spot the three things that make a benchmark lie;
  • know why the first timed run is usually thrown away, and how fine your stopwatch really is;
  • choose between perf_counter and the timeit module for a given job.

Reading the timing examples: displayed seconds and fitted slopes are illustrative output, not measurements from your machine. Run the experiment, retain the actual samples, and report the input and measurement conditions. Exact operation-count formulas are separate mathematical claims.

Three-hour interactive studio

00:00–01:00: launch, predict–run–explain cycle, questions  ·  01:00–01:10: break  ·  01:10–02:00: worked variation, peer instruction, questions  ·  02:00–02:10: break  ·  02:10–03:00: core mechatronics practice, exam bridge, and exit ticket.

Ask at any point. Weekly self-checks stay private; optional extensions are not collected.

Need a slower explanation? Open the English + Türkçe reference guide.

Optional extra practice

5.9Optional Studio Extension

Optional practice · no submission or deadline
  1. Create AA_Week05.ipynb, define time_it, and solve Tasks 1–6.
  2. Benchmark three functions at four sizes each: sum_to, sum_formula, and contains (worst case). One table each, with a ratio column.
  3. Add a step counter to contains and report both steps and seconds for each size.
  4. Run the warm-up demonstration from §5.6 and Task 4 on your own machine or Colab. Paste your eight run times and state, in one sentence, whether taking the fastest saved you from the cold first run.
  5. Measure the relative noise (Task 6) at your smallest and largest size, and quote both percentages.
  6. Write five sentences: where do the step counts and the seconds agree, and where does the timing wobble? Which of the three ways to lie (§5.3) did you have to guard against, and did growing n shrink your relative noise as §5.6 predicts?
Optional reference · Words from this week

5.10Words from this week

TermMeaning in plain words
functionA named block of work you can call with different inputs.
parameter / argumentThe name in the definition / the value you pass in.
returnHand a value back to whoever called the function, and stop.
benchmarkA measurement of how long code takes, done carefully enough to trust.
perf_counterPython's high-resolution stopwatch; only differences are meaningful.
warm-up runA first, discarded run that pays one-off costs so the timed runs are fair.
resolutionThe smallest interval a clock can distinguish; times near it are untrustworthy.
timeitA standard-library tool that batches repeated snippets to reduce relative measurement overhead; results can still vary.
noiseVariation in timings caused by everything else the machine is doing.
Where this leads

You can time one input size. Week 6 times several sizes at once and reads the pattern in the numbers — the doubling experiment.