Schedule
Some of the later dates are not accurate, and are copy-pasted from earlier semesters.
| When | What |
|---|---|
| Week 1 | |
| Fri, Sep 5 | Topics: Generation hyperparameters; decoding algorithms; prompting with base models. Notes Out: Homework 0: Programming Capabilities |
| Week 2 | |
| Tue, Sep 9 | Topics: Basics of Transformers and GPUs; tensor practice. |
| Fri, Sep 12 | Topics: Chain of thought prompting; introduction to conversational models. |
| Week 3 | |
| Tue, Sep 16 | Class cancelled: Arjun at Lawrence Livermore Laboratory Due: Homework 0: Programming Capabilities. Out: Homework 1: Math Word Problems |
| Fri, Sep 19 | Topics: Numerical gradient descent; Word embeddings and vector similarity. |
| Week 4 | |
| Tue, Sep 23 | Topics: Training basics. |
| Fri, Sep 26 | Topics: |
| Week 5 | TODO: this and below. |
| Tue, Sep 30 | Topics: DeepSeek results |
| Fri, Oct 3 | Topics: Introduction to agents In-Class Notebook No reading this week. Reading: SWE-Bench. |
| Week 6 | |
| Tue, Oct 7 | Topics: The Act pattern In-Class Notebook Due: Math Word Problems |
| Fri, Oct 10 | Topics: Information retrieval. Due: Reading 2. In-class Notebook Reading: s1: simple test-time scaling |
| Week 7 | |
| Tue, Oct 14 | Topics: Neural retrieval. In-class Notebook. Recommended Reading: Speech and Language Processing Chapter 14. Out: Thomas the Travel Agent |
| Fri, Oct 17 | Due: Reading 3. Out: Project Proposal |
| Week 8 | |
| Tue, Oct 21 | Topics: Supervised fine-tuning. |
| Fri, Oct 24 | Topics: More on GRPO. Due: Project Proposal |
| Week 9 | |
| Tue, Oct 28 | Due: Thomas the Travel Agent |
| Fri, Oct 31 | Topics: Speculative decoding and proxy tuning. |
| Week 10 | |
| Tue, Nov 4 | |
| Fri, Nov 7 | |
| Week 11 | |
| Tue, Nov 11 | Veterans Day Out: Obscure Questions |
| Fri, Nov 14 | Topics: High-performance inference. PagedAttention; LLM.int8. Short class: Khoury College PhD Open House |
| Week 12 | |
| Tue, Nov 18 | Topics: Non-Local Control Flow, Containers |
| Fri, Nov 21 | In-class activity: Programming in a language that you do not know. |
| Week 13 | |
| Tue, Nov 25 | Topics: Synthetic data generation. Self-Instruct; Magicoder; MultiPL-T. Due: Obscure Questions |
| Fri, Nov 28 | Thanksgiving Break |
| Week 14 | |
| Tue, Dec 2 | Topics: Humans solve LLM benchmarks: ARC-AGI 1 and Social IQa. |
| Fri, Dec 5 | Last day of class (Fall 2025) Reading: Demystifying CLIP Data and What Matters When Building vision-language models? (Questions) |