schedule | CS 5624

Note: The schedule may be adjusted if we need more or less time for certain topics.

Date	Lecture	Readings

8/25	Introduction [ slides ]	No associated readings
Transformer Architectures
8/27	Neural Language Models [ slides ]	Jurafsky and Martin, Chapter 3.1-3.5
9/1	Transformers [ slides ]	Vaswani et al. (2017) Attention Is All You Need Jay Alammar's blog The Illustrated GPT-2 (Visualizing Transformer Language Models)
9/3	Transformers (cont'd) [ slides ]	Vaswani et al. (2017) Attention Is All You Need Jay Alammar's blog The Illustrated GPT-2 (Visualizing Transformer Language Models)
9/8	Different Transformer Architectures [ slides ]	Devlin et al. (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Raffel et al. (2019) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Radford et al. (2019) Language Models are Unsupervised Multitask Learners
9/10	Language model training [ slides ]	Jurafsky and Martin, Chapter 6.6
9/15	Decoding and inference [ slides ]	Holtzman et al. (2019) The Curious Case of Neural Text Degeneration Brown et al. (2020) Language Models are Few-Shot Learners Wei et al. (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Anthropic blog Effective context engineering for AI agents
9/17	Pretraining scaling laws [ slides ]	Hoffmann et al. (2022) Training Compute-Optimal Large Language Models [optional] Li et al. (2025) (Mis)Fitting: A Survey of Scaling Laws
LLM Post-training
Efficient Training and Inference
Reasoning and Test-time Scaling
Interpretability and Evaluation
Multimodal and Diffusion LLMs
AI Agents
Recursive Self-improvement
AI Security and Privacy