Files are part of the interface
Document the header, units, encoding and row interpretation. Use Python's csv module for CSV boundaries. This handles quoted fields more reliably than assuming every comma is a separator.
Week 13 / Build a checkable report
Preserve the raw input and make the output reproducible.
We can now interpret and validate readings. A result that exists only in a running notebook is hard for another person to review tomorrow.
We will read a small sensor CSV, write accepted records to a separate file, and retain the source line and reason for every rejected record.
Start with the problem
Sketch three labelled files: raw input, cleaned readings and rejection record. Where would another engineer look to explain a missing reading?
File I/O preserves information beyond one run. CSV gives the records a shared structure. Reopening the saved output checks what was actually written.
By the end: Follow one accepted and one rejected record from the raw file to the saved outputs without overwriting the original evidence.
Document the header, units, encoding and row interpretation. Use Python's csv module for CSV boundaries. This handles quoted fields more reliably than assuming every comma is a separator.
Opening a file with mode 'w' replaces its previous contents. Use separate raw and cleaned filenames. For repeated runs on identical input, a deterministic clean output should not accumulate duplicate rows; appending is appropriate only when that is the intended log behaviour.
A write finishing does not prove that the output has the expected schema or count. Reopen the file, check its header and retained rows, and confirm that raw rows = kept rows + rejected rows under your policy.
Read the example alongside the explanation. Run it in a new notebook cell and change one input to see how it behaves.
import csv
raw_rows = [['T1', '20'], ['T1', 'bad'], ['T1', '22']]
with open('bench_raw.csv', 'w', newline='', encoding='utf-8') as file:
csv.writer(file).writerows([['sensor', 'temperature_c']] + raw_rows)
clean = []
rejected = []
with open('bench_raw.csv', newline='', encoding='utf-8') as file:
reader = csv.reader(file)
header = next(reader)
for line, row in enumerate(reader, start=2):
try:
value = float(row[1])
except ValueError:
rejected.append((line, row, 'non-numeric value'))
continue
clean.append([row[0], value])
with open('bench_clean.csv', 'w', newline='', encoding='utf-8') as file:
csv.writer(file).writerows([header] + clean)
with open('bench_clean.csv', newline='', encoding='utf-8') as file:
saved = list(csv.reader(file))
print('Saved data rows:', len(saved) - 1)
print('Rejected:', rejected)Saved data rows: 2 Rejected: [(3, ['T1', 'bad'], 'non-numeric value')]
This file-flow model intentionally uses three fixed, two-field rows. It checks numeric conversion only. Before accepting external data, add header, field-count, sensor-ID, finite-value and range checks from Weeks 9 and 12. Running it creates or replaces bench_raw.csv and bench_clean.csv in the current notebook folder; use a scratch folder.
Writing text transfers a representation to persistent storage. A correct report must retain enough structure to be read back with the same intended meaning. Formatting, delimiters, quoting and file mode are part of that contract.
Draw memory → writer → file bytes → reader → reconstructed records. Put a sensor label containing a comma in one field to expose the difference between simple splitting and CSV parsing.
Predict before running. Will splitting every line at every comma reliably recover a quoted field containing a comma? What happens when a second run uses write mode?
Test a write–read round trip and inspect the saved file after closing it. Equality of the reconstructed records checks the representation; it does not validate the original measurements.
Change one thing. Run the export twice in a temporary file. Decide whether the required outcome is replacement or accumulation, then check for duplicated headers.
Türkçe: Dosyaya yazmak bir gösterimi saklar. CSV tırnaklaması ve dosya modu anlamı korumalı; yazdığını yeniden okuyarak denetle.
Each example changes something about the same problem. Open the ones you want to explore and follow the worked explanation.
Header plus three data rows: T1,20 / T1,bad / T1,22
Question: How many rows are kept, and what is the rejected source line?
State whether numbering includes the header. Here source lines are one-based and include it.
A cleaned output has 2 data rows; rerun appends those same rows
Question: What will a second run do to the output count?
Use replacement for a deterministic derived file. Reserve append for deliberately cumulative logs.
One temperature 20 °C and one humidity 40%
Question: What does their arithmetic mean of 30 measure?
A parser can accept both numbers while the analysis remains invalid. Keep schema and physical meaning together.
Interactive walkthroughs of File I/O & CSV. Enable JavaScript to step through code, variables, collections and output. The companion notebook remains available below.
Use the notebook to try the ideas yourself. The steps below connect this week's example to the programming practice.
Something to take away: Raw and cleaned files, a rejection record with line numbering explained, and a read-back check that survives a second run.
Read the notebook's teaching cells before these exercises.
Use the notebook's core and optional labels to choose your workload. This activity fits within guided class time.
Notes stay in this browser. Download a copy to keep them.
Ask AI for a CSV cleaning pipeline. Review file modes, header handling and whether it preserves raw input. Ask what happens on the second run, with an empty file, or with mixed sensor units; test those claims.
You can also review the supplied example with a partner. Use the same inputs to compare the reasoning. Follow the syllabus rules for assessed work.