Syllabus available on Canvas

Course Info

Instructor: Quanquan Liu
TF: Pranay Mundra
ULAs: Jewon Im, Bill Qian, Vishak Srikanth
Meeting times: TTh, Spring 2026
Instructor office hours: Wednesdays 3–4pm, AKW 111
All course announcements and assignments are posted on Canvas; questions are answered on Ed Discussion.

Course Description:

How do we write programs that take full advantage of the parallelism built into modern hardware — and how do we reason about their correctness and performance? This course is half theory and half practice: it combines elements of parallel algorithms, parallel programming, and software performance engineering, geared to help you write faster and more performant programs on real machines. Topics include:

• Parallel programming models and frameworks such as C++ threads, OpenMP, MPI, Cilk, ParlayLib, and CUDA
• Concurrency: race conditions, locks, lock-free techniques, optimistic locking, and concurrent data structures
• Parallel algorithms with provable guarantees: parallel primitives, modeling parallelism (work and span), scheduling, graph algorithms and sparse representations (CSR), and k-nearest neighbors
• Performance engineering in practice: loop scheduling, vectorization, programming and debugging on GPUs, and parallelism for large language models

Students complete biweekly programming and written assignments, and finish the course with an open-ended final project (teams of 1–3 students). For 5% of the grade, students may choose between participation and a performance engineering challenge in which submitted code races against the rest of the class.

Grading: 30% programming assignments (biweekly, Gradescope), 25% written homework assignments (biweekly), 5% performance challenge/participation, 10% Midterm 1 (Feb 10, in class), 10% Midterm 2 (March 24, in class), 20% final project (March 31 – April 23).

Tentative Schedule and Slides

This is the plan of the topics covered in this course; subject to change. Slides are also posted on Canvas along with the lecture quizzes.

• Jan 13: Introduction [Lecture 1]
• Jan 15: Matrix Multiplication, C++, OpenMP [Lecture 2]
• Jan 20: Modeling Parallelism [Lecture 3]
• Jan 22: Parallel Primitives Continued [Lecture 4]
• Jan 27: OpenMP [Lecture 5]
• Jan 29: OpenMP (continued) [Lecture 6]
• Feb 3: Loop Scheduling (continued); Threads and Concurrency [Lecture 7]
• Feb 5: Concurrency and Locks [Lecture 8]
• Feb 10: Midterm 1 (in class)
• Feb 12: Concurrent Data Structures [Lecture 9]
• Feb 17: Concurrent Data Structures Continued [Lecture 10]
• Feb 19: Optimistic Locking and MPI [Lecture 11]
• Feb 24: MPI, Data Representations, Graphs [Lecture 12]
• Feb 26: CSR and Graphs [Lecture 13]
• March 3: Graphs and ParlayLib [Lecture 14]
• March 5: ParlayLib and kNN [Lecture 15]
• March 24: Midterm 2 (in class)
• March 26: Scheduling, Parallel Loops, and Cilk [Lecture 16]
• March 31: Vectorization [Lecture 17]
• April 2: Vectorization and GPUs [Lecture 18]
• April 7: Programming on GPUs [Lecture 19]
• April 9: Programming on GPUs [Lecture 20]
• April 14: Programming on GPUs [Lecture 21]
• April 16: Homework Discussion and Parallelism for LLMs [Lecture 22]
• April 21: Final Project Presentations
• April 23: Final Project Presentations

Resources

• OpenMP, Open MPI, OpenCilk, ParlayLib, and the CUDA C++ Programming Guide
• Related courses: MIT 6.106/6.172 Performance Engineering of Software Systems and Berkeley CS267 Applications of Parallel Computers

Special thanks to MIT 6.106 and the FASTCODE community, whose materials some of the lecture content is adapted from.