This is where I'll share the lecture notes (draft) for the Advanced Optimization course I teach at Nanjing University.
Updates will be pretty slow, but if you spot anything that can be improved or have any feedback, feel free to let me know, thanks!
[ New! 2025.02.10] The lecture note for Lec 9 (optimism for acceleration) has been finished and posted online.
...
Lecture 9: Optimism for Acceleration (lecture note)
...
Course Information: This course aims to provide an overview of modern optimization theory designed for machine learning, particularly focusing on the gradient-based first-order optimization methods (with offline and online deployments), which serve as the foundational optimization tools for modern large-scale learning tasks.
Instructor: Peng Zhao ([email protected])
Semester: Every fall semester
Feel free to use the following bibtex entry as a reference (e.g., see an example pointing to Lecture 9).
xxxxxxxxxx@misc{LectureNote:AOpt25, author = {Peng Zhao}, title = {Lecture {N}otes for {A}dvanced {O}ptimization}, note = {Lecture 9. {O}ptimism for {A}cceleration}, url = {https://www.pengzhao-ml.com/course/AOptLectureNote/}, year = {2025}}
Course Materials: The course materials (slides, references, etc) can be found in the following sites.
Advanced Optimization (For Undergraduate and Graduate Students, 2024 Fall)
Advanced Optimization (For Undergraduate and Graduate Students, 2023 Fall)
Some related courses:
CSCI 659: Introduction to Online Optimization/Learning, Fall 2022. University of South California, Haipeng Luo.
EC525: Optimization for Machine Learning, Fall 2022. Boston University, Ashok Cutkosky.
Last modified: 2025-02-10 by Peng Zhao