schedule | CS 6804

Note: Since this is the first time the class is being taught, the schedule may adjust if we need more or less time on certain topics.

Date	Lecture	Readings
Transformers & Pretraining Scaling
1/21	Introduction & Transformers [ slides ]	Vaswani et al. (2017) Attention Is All You Need Jay Alammar's blog The Illustrated GPT-2 (Visualizing Transformer Language Models)
1/26	Transformers (cont’d) & Pretraining Scaling [ slides ]	Devlin et al. (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Raffel et al. (2019) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Radford et al. (2019) Language Models are Unsupervised Multitask Learners Hoffmann et al. (2022) Training Compute-Optimal Large Language Models [optional] Li et al. (2025) (Mis)Fitting: A Survey of Scaling Laws
1/28	Mixture-of-Experts & Multimodal models [ slides ]
Efficient training & inference
2/2	[ slides ]
2/4	[ slides ]
Post-training & Reinforcement Learning
2/9	[ slides ]
2/11	[ slides ]
Large reasoning models & Test-time scaling
2/16	[ slides ]
2/18	[ slides ]
Agents & Compound AI systems
2/23	[ slides ]
2/25	[ slides ]
Guest Lectures
3/2	TBD [ slides ]
3/4	TBD [ slides ]
Spring break
3/9	No classes
3/11	No classes
Student presentations & discussions
3/16	TBD [ slides ]
3/18	TBD [ slides ]
3/23	TBD [ slides ]
3/25	TBD [ slides ]
3/30	TBD [ slides ]
4/1	TBD [ slides ]
4/6	TBD [ slides ]
4/8	TBD [ slides ]
4/13	TBD [ slides ]
4/15	TBD [ slides ]
4/20	TBD [ slides ]
4/22	TBD [ slides ]
4/27	TBD [ slides ]
4/29	TBD [ slides ]
Final exam
5/4	No classes
5/6	Exam (in-class) [ slides ]