Engineering LLM-Integrated Systems

CS 4973 section 2 (Spring 2025)

Time and Location

Tuesdays and Fridays: 1:35pm – 3:15pm

Location: Hayden Hall 424

Course Description

Generative AI has raised our expectations of what is possible with software. The previous generation of AI assistants had a narrow set of skills and limited fluency in natural language. However, recent advances in large language models (LLMs) have shown that it is possible to build software systems with a far more natural and fluid interface. These LLM-integrated systems turn natural language instructions into actions and are starting to appear in everything from office apps to robots.

In this course, we will study the software engineering foundations for LLM-integrated systems. Students will learn how to build these systems, integrate them with existing software, rigorously test their behavior, and understand their failure modes and limitations.

Prerequisites: This is a programming intensive course, and students will be expected to be proficient at programming and testing software that does not use LLMs.  We will primarily work in Python, but prior Python experience is not required.

Formal prerequisites: CS 3000, CS 3500, or DS 3500

Further Information

The goal of this course is to prepare you for major changes that are starting to happen in the capabilities of software, as a result of deep integration with LLMs 1. We hope that with what you learn, you’ll be able to actively participate in this ongoing software revolution. LLM-integrated systems are still in their infancy. We don’t yet know the “right way” to build them. There are several competing approaches, and new ideas are constantly being developed. It is quite likely that before the semester is over, new methods will emerge that will make something that we teach earlier in the semester seem quaint. Despite this relentless pace, there are principles that have emerged over the last few years that we believe will remain relevant for several years to come. So, this course will focus on teaching (1) the emerging foundations of LLM-integrated systems, and (2) the skills you need to keep up with new developments in the field. Everything in this field is presently an experiment, and that includes this course. So, be prepared for rough edges.

  1. See [AI Software Should be More Like Plain Old Software] by Berger and Zorn. ↩