Schedule
| When | What |
|---|---|
| Week 1 | Fundamentals of LLM generation and basic prompting techniques |
| Tue, Jan 7 | Topics: Generation hyperparameters; zero-shot and few-shot prompting; prompting for code. In-class Notebook Out: Programming Capabilities. Create an ACCESS account immediately. |
| Fri, Jan 10 | Topics: High-level overview of Transformers and GPUs. Reading: StarCoder and Llama. Out: Reading 0. In-class: Inference on GPU. |
| Week 2 | |
| Tue, Jan 14 | Topics: Prompt-writing activity; more prompting techniques. In-class Notebook |
| Fri, Jan 17 | Topics: Introduction to Conversational Models. Due: Reading 0. Reading: InstructGPT and UltraFeedback |
| Week 3 | |
| Tue, Jan 21 | Topics: What do models learn? In-class Notebook Due: Programming Capabilities. Out: Math Word Problems |
| Fri, Jan 24 | Topics: Reading discussion; high-speed programming activity. Due: Reading 1. |
| Week 4 | |
| Tue, Jan 28 | Topics: DeepSeek results |
| Fri, Jan 31 | Topics: Introduction to agents In-Class Notebook No reading this week. Reading: SWE-Bench. |
| Week 5 | |
| Tue, Feb 4 | Topics: The Act pattern In-Class Notebook Due: Math Word Problems |
| Fri, Feb 7 | Topics: Information retrieval. Due: Reading 2. In-class Notebook Reading: s1: simple test-time scaling |
| Week 6 | |
| Tue, Feb 11 | Topics: Neural retrieval. In-class Notebook. Recommended Reading: Speech and Language Processing Chapter 14. Out: Thomas the Travel Agent |
| Fri, Feb 14 | Due: Reading 3. Out: Project Proposal |
| Week 7 | |
| Tue, Feb 18 | Topics: Supervised fine-tuning. |
| Fri, Feb 21 | Topics: More on GRPO. Due: Project Proposal |
| Week 8 | |
| Tue, Feb 25 | Class cancelled. Due: Thomas the Travel Agent |
| Fri, Feb 28 | Topics: Speculative decoding and proxy tuning. |
| Week 9 | Spring Break |
| Tue, Mar 4 | No class |
| Fri, Mar 7 | No class |
| Week 10 | |
| Tue, Mar 11 | Topics: High-performance inference. PagedAttention; LLM.int8. Out: Obscure Questions |
| Fri, Mar 14 | Short class: Khoury College PhD Open House |
| Week 11 | |
| Tue, Mar 18 | Topics: Non-Local Control Flow, Containers |
| Fri, Mar 21 | In-class activity: Programming in a language that you do not know. |
| Week 12 | |
| Tue, Mar 25 | Class cancelled. |
| Fri, Mar 28 | Topics: Synthetic data generation. Self-Instruct; Magicoder; MultiPL-T. Due: Obscure Questions |
| Week 13 | |
| Tue, Apr 1 | Topics: Humans solve LLM benchmarks: ARC-AGI 1 and Social IQa. |
| Fri, Apr 4 | Reading: Demystifying CLIP Data and What Matters When Building vision-language models? (Questions) |
| Week 14 | |
| Tue, Apr 8 | |
| Fri, Apr 11 | In-class project presentations (12 min per group) |
| Week 15 | |
| Tue, Apr 15 | Last day of class. In-class project presentations (12 min per group) |
| Tue, Apr 22 | Due: [Project] |