NA446A Python for data you can trust

Sessions

Slides, scripts and exercises, posted before each session.

Every lab, every session: unzip the lab folder inside your course folder na446a-starter, and open na446a-starter in VS Code (File → Open Folder…), never the lab folder on its own. Otherwise VS Code cannot find the course Python and you get “No module named ‘pandas’”. When VS Code asks which Python (kernel) to use, choose na446a … .venv (click ↻ if it is missing). The terminal then starts in na446a-starter: for commands such as uv run check_s3.py, type cd s3_lab (the lab’s folder name) once first, or open the file and press ▷. More in Common problems in class.

Session 1. Projects and first steps (Mon 19 Oct)

Goals. After this session you can:

  1. open a Python project in VS Code that has its own environment, and run a script;
  2. organise a project so that someone else can rerun it;
  3. load a dataset in pandas and inspect it: size, first rows, summary statistics.

Before the session: complete the Setup, read the Bryan memo (see Readings) and the AI rules.

Slides: handout, questions without answers (PDF). Full slides, with answers (PDF).

Lab: s1-lab.zip. Unzip it inside your course folder; it contains its own data. Open s1_lab.py and click Run Cell above each step. Steps 1–3 from the slides as one script (under a minute; the last line prints Lab done), then fix an AI-written script: lab01_fix_the_bug.py runs without an error but gives a wrong answer, and you find the bug by testing a case whose answer you know.

Session 2. Working with data (Tue 20 Oct)

Goals. After this session you can:

  1. apply the intake checklist to a variable: type, missing values, range;
  2. fix types and missing values, including sentinel codes such as 999;
  3. create, subset, sort and aggregate variables, and summarise data in a table and a figure.

Before the session: read the Turrell chapter (see Readings), about 20 minutes. Session 2 is the day after session 1, so plan it into the weekend before.

Slides: handout, questions without answers (PDF). Full slides, with answers (PDF).

Lab: s2-lab.zip. Unzip it inside your course folder; it contains its own data (auto_messy.csv, titanic.csv). Open s2_lab.py and click Run Cell above each step, filling in the TODOs (about 40 minutes; the last line prints Lab done). Check your work at any time with uv run check_s2.py: all right is 9 of 9 steps right. The AI task: paste your intake output (not the data) into a chatbot, ask what problems it shows, and test two of its claims with one line of code each.

Exercise after the session: a Titanic intake, about 90 minutes, in the same lab folder (titanic_intake.py, self-check uv run check_exercise_s2.py, all right is 8 of 8 tasks right.). Formative, no hand-in. The solutions to the lab and the exercise are posted here three days after the session.

Session 3. Combining data (Tue 27 Oct)

Goals. After this session you can:

  1. merge two tables with key checks (validate=, indicator=), and predict the number of rows before you run;
  2. append yearly files and reshape between long and wide;
  3. build a person-year panel with a lag that is really last year’s value.

Before the session: read Gentzkow and Shapiro, chapter 5, Keys (see Readings).

Slides: handout, questions without answers (PDF). The version with answers is posted after the session.

Lab: s3_lab.zip. Unzip it and put the folder s3_lab inside your course folder. In class, pick a free number 1–30 on the whiteboard, cross it out, and download your pair’s pair_NN.zip from the Data page; unzip it into s3_lab. Late or absent? Use pair 00, which is already in the folder. Then open s3_lab.py, set PAIR in the CHANGE HERE block and run the cells one by one. Self-check: uv run check_s3.py (all right is 8 of 8 steps right.). The second half is a bug race on a cleaning script written the way real register code often is. All data are synthetic; no real person or firm is in them.

Exercise after the session: the full bug hunt, about 2 hours, in the same lab folder (bug_hunt_s3.py, self-check uv run check_exercise_s3.py, all right is 6 of 6 tasks right.). The answers differ from pair to pair. Formative, no hand-in. The solutions to the lab and the exercise are posted here three days after the session.

Session 4. Getting data and scaling up (Thu 29 Oct)

Posted before the session.