NA446A Python for data you can trust

Exercise after Session 2: a Titanic intake

Formative, no hand-in. About 90 minutes. Solution on the site three days after the session.

Why this exercise

In the session you ran the intake checklist on one file. Here you run all of it on a new one, the passenger list of the Titanic (891 passengers, 12 variables, a public teaching dataset). The exam asks you to read code like this and predict what it does, without AI. So try each task yourself first; use the tutor prompt if you are stuck, not a chatbot that writes it for you.

What you need

The Session 2 lab folder, which you already placed inside your course folder (na446a-starter) for the lab. It holds titanic_intake.py (the script you fill in), check_exercise_s2.py (the self-check) and data/raw/titanic.csv. Open na446a-starter in VS Code (the course folder, not the lab folder on its own).

Work in titanic_intake.py, one cell at a time, and use the names given in each TODO (df, miss, flags …): the self-check looks for them. Run the self-check whenever you like:

Open check_exercise_s2.py and press ▷ (Run Python File), or type in the terminal:

cd s2-lab
uv run check_exercise_s2.py

(cd s2-lab once per terminal: the terminal starts in na446a-starter.) Save your script first: the self-check reads the saved file.

It tells you which tasks are right and gives a hint for the others. When all is right, the last line reads 8 of 8 tasks right. It never changes your script.

Tasks, from easy to hard

Task What to do Time
1. Load and name (easy) Load the file into df; make every column name lower case; print the shape. 5 min
2. Types (easy) Print the types. Which columns are text? Convert survived, pclass, sibsp, parch to "Int8". 10 min
3. Missing values (medium) miss: one row per variable with n (count) and pct (percent, one decimal), most missing first. 10 min
4. Ranges (medium) flags: one True/False column per rule, True where a value that is present breaks it: survived in {0, 1}; pclass in {1, 2, 3}; sex in {male, female}; embarked in {C, Q, S}; age between 0 and 100; fare at least 0. Flag, do not drop. Then count the fares of exactly 0 (n_zero_fare). 15 min
5. Categories (medium) Value counts of sex and embarked, missing included. ct: a crosstab of pclass by sex with totals; ct_row: the same in row percent. 10 min
6. Create (medium) child: 1 if age is below 18, 0 if not, and missing if age is missing (type Int8). family_size: sibsp + parch + 1. 15 min
7. Aggregate (hard) by_class_sex: one row per class and sex with n, survival (mean of survived) and mean_age, using agg. Then class_fare, the mean fare of each passenger’s class, using transform, and fare_rel = fare / class_fare. 15 min
8. Save (easy) Save df to data/processed/titanic_clean.parquet, and a codebook (one line per variable) to data/processed/titanic_codebook.json. 5 min

Questions to answer in your script (as comments)

  1. Task 2: which columns are text, and does any of them look like it should be a number?
  2. Task 4: 15 passengers paid a fare of 0. Is 0 a sentinel code for “missing”, or a real value? Say what in the data supports your view, and what you do with these rows. There is no single right answer; there is a right way to argue it.
  3. Task 6: how many passengers have a missing child, and why that number?
  4. Task 7: how many rows do by_class_sex and df have, and why do they differ?

If you finish early (optional, 15 min)

Many passengers share a ticket number. Do passengers on the same ticket pay the same fare? If they do, what does fare measure, and how would you compute a fare per person? Hint: groupby("ticket") with transform("size").

Two rules for this exercise