Schedule
| When | What |
|---|---|
| Week 1 | Fundamentals of LLM generation and basic prompting techniques |
| Fri, Sep 6 | Topics: Generation hyperparameters; zero-shot and few-shot prompting; prompting for code. In-class Notebook |
| Week 2 | More prompting techniques; chat models |
| Tue, Sep 10 | Topics: Chain of thought prompting; program-aided language models. In-class Notebook Out: Math Word Problems. |
| Fri, Sep 13 | Topics: Introduction to chat models; adapting prompting to chat models. In-class Notebook 1 In-class Notebook 2 |
| Week 3 | LLMs and tools |
| Tue, Sep 17 | Topics: LLMs and tools. In-class Notebook |
| Fri, Sep 20 | Activities: Rapid prototyping with LLMs. |
| Week 4 | Memory |
| Tue, Sep 24 | Topics: Multi-turn conversations; LLM internals. Due: Math Word Problems. Out: Thomas the Travel Agent. In-class Notebook |
| Fri, Sep 27 | Topics: Information retrieval. Activities: programming in unknown languages. In-class Notebook Recommended Reading: Speech and Language Processing Chapter 14 |
| Week 5 | Retrieval |
| Tue, Oct 1 | Topics: Embeddings. (See the notebook from Sep 27) |
| Fri, Oct 4 | Guest speaker: Yann Debray (MathWorks) |
| Week 6 | Fine-tuning |
| Tue, Oct 8 | No class: Arjun at COLM. Due: Thomas the Travel Agent. |
| Fri, Oct 11 | Topics: Lecture on fine-tuning. (We will revisit the notebook next week.) |
| Week 7 | Fine-tuning continued; Programming Abstractions |
| Tue, Oct 15 | Activity: Fine-tuning models on NCSA Delta. Out: Obscure Questions. |
| Fri, Oct 18 | Topics: Non-local control flow; containers. Prepared Notebook on Non-Local Control Flow. Notes on Containers |
| Week 8 | Hacking Distributions |
| Tue, Oct 22 | Topics: Speculative Decoding; Proxy Tuning. |
| Fri, Oct 25 | No class: Arjun at OOPSLA. |
| Week 9 | Datasets and Benchmarks |
| Tue, Oct 29 | Topics: Synthetic data generation. Self-Instruct; Magicoder; MultiPL-T. Due: Obscure Questions. Out: Khoury Chatbot |
| Fri, Nov 1 | Topics: Students manually solve LLM benchmarks. ARC-AGI; HumanEval; Social IQa. Out: Final Project |
| Week 10 | Larger Models |
| Tue, Nov 5 | Activity: Evaluating larger models. |
| Fri, Nov 8 | Topics: High-performance inference. PagedAttention; LLM.int8. Final Project proposals due this weekend. |
| Week 11 | Inside LLMs |
| Tue, Nov 12 | Topics: Learning from LLMs; neural networks. Due: Khoury Chatbot |
| Fri, Nov 15 | Topics: Reinforcement learning. Guest speaker: Federico Cassano (Cursor) |
| Week 12 | Inside LLMs |
| Tue, Nov 19 | No class: Arjun at [SC24]. |
| Fri, Nov 22 | Topics: Inside LLMs. Due: Final Project (Checkpoint). |
| Week 13 | Multimodality |
| Tue, Nov 26 | |
| Fri, Nov 29 | No class: Thanksgiving |
| Week 14 | |
| Tue, Dec 3 | Last class: Presentations |
| Fri, Dec 6 | Due: Final Project |