logistics

Textbooks

For NLP fundamentals, we recommend Speech and Language Processing by Jurafsky and Martin. Course readings will primarily include published research papers from top-tier ML venues (such as JMLR, ICML, NeurIPS, ICLR, COLM) and ACL\(^{*}\) venues (such as TACL, ACL, EMNLP, NAACL), provided as PDFs.

Additional useful NLP and deep learning texts include:

Grading criteria

The grading breakdown is as follows:

  • Quizzes (10%)
  • Homework assignments (50%)
  • Final project (40%)

The top 10 Piazza contributors will earn 3.5%. Instructors will review contributions to ensure the system is not abused.

Note that this class does not have any exams.

Late policy

Each student is allowed three late days for homework submissions. Once these days are used, no additional late submissions will be accepted. If you encounter an unexpected health or personal emergency, please contact the instructors as soon as possible at cs5624instructors@gmail.com, and provide valid documentation if needed. Late submissions due to job interviews, other coursework, or non-emergency situations will not be excused, and extensions will not be granted.

AI assistance policy

AI assistance is permitted for completing assignments, including homework, quizzes, projects, and exams (if any). However, if you use AI tools like ChatGPT or Gemini, you must submit the prompts you used and describe how the AI contributed to your work. It is your responsibility to verify the AI-generated content for accuracy before submission.

Student collaboration policy

For group projects, work must be completed solely by members of the group. Apart from group projects, students must independently complete their work. Sharing code or written materials with others is prohibited. However, discussing concepts, clarifying instructions, exploring problem-solving approaches, or seeking debugging advice (without sharing code) is encouraged. All external sources must be properly cited.

Academic accommodations

Virginia Tech welcomes students with disabilities into the University’s educational programs. The University promotes efforts to provide equal access and a culture of inclusion without altering the essential elements of coursework. If you anticipate or experience academic barriers that may be due to disability, including but not limited to, chronic medical conditions, Deaf or hard of hearing, learning disability, mental health, or vision impairment, please contact the Services for Students with Disabilities (SSD) (540-231-3788, ssd@vt.edu, or visit www.ssd.vt.edu). If you have an SSD accommodation letter, please meet with us privately during office hours as early in the semester as possible to discuss implementing your accommodations. You must give us reasonable notice to implement your accommodations, which is generally 5 business days and 10 business days for final exams or project presentations.

If there are aspects of this course that prevent you from learning or exclude you, please let us know as soon as possible. Together we will develop strategies to meet both your needs and the requirements of the course.

Graduate academic integrity statement

The tenets of the Virginia Tech Graduate Honor Code will be strictly enforced in this course, and all assignments shall be subject to the stipulations of the Graduate Honor Code. For more information on the Graduate Honor Code, please refer to the GHS Constitution.

Course/learning objectives

By the end of this course, students will be able to explain key NLP principles and techniques for building state-of-the-art LLMs, including:

  • foundational concepts of language models
  • transformer models, such as BERT, and techniques for LLM pretraining and fine-tuning
  • strategies for LLM prompting and decoding
  • techniques for aligning LLMs with human instructions and preferences
  • evaluation methods
  • techniques for enhancing LLM efficiency and adaptability
  • advanced LLM concepts (e.g., advanced reasoning) and compound AI Systems (e.g., RAG, tool-use LLMs, agents)
  • techniques for developing multimodal and multilingual LLMs, code and math LLMs
  • techniques for ensuring LLM safety and security
  • alternatives to token-based models and transformers

Students will gain practical expertise in designing, training, evaluating, and applying LLMs, preparing them to tackle real-world challenges and adapt LLMs to diverse tasks and contexts.

Absence policy

Although attendance is not graded, students are strongly encouraged to attend all scheduled classes, quizzes, project presentations, and exams (if any) to fully engage with the course material and succeed.

VT principles of community statement

Virginia Tech is a public land-grant university, committed to teaching and learning, research, and outreach to the Commonwealth of Virginia, the nation, and the world community. Learning from the experiences that shape Virginia Tech as an institution, we acknowledge those aspects of our legacy that reflected bias and exclusion. Therefore, we adopt and practice the following principles as fundamental to our on-going efforts to increase access and inclusion and to create a community that nurtures learning and growth for all of its members:

  • We affirm the inherent dignity and value of every person and strive to maintain a climate for work and learning based on mutual respect and understanding.
  • We affirm the right of each person to express thoughts and opinions freely. We encourage open expression within a climate of civility, sensitivity, and mutual respect.
  • We affirm the value of human diversity because it enriches our lives and the University. We acknowledge and respect our differences while affirming our common humanity.
  • We reject all forms of prejudice and discrimination, including those based on age, color, disability, gender, gender identity, gender expression, national origin, political affiliation, race, religion, sexual orientation, and veteran status. We take individual and collective responsibility for helping to eliminate bias and discrimination and for increasing our own understanding of these issues through education, training, and interaction with others.
  • We pledge our collective commitment to these principles in the spirit of the Virginia Tech motto of Ut Prosim (That I May Serve).