Description: Machine Learning (ML) has undergone a transformative shift since the early 2010s, marked by significant advancements in deep learning. These changes have paved the way for groundbreaking models such as AlphaGo and AlphaZero, creative generators like DALL-E, and sophisticated Large Language Models exemplified by ChatGPT. Today, ML is not just a tool for computer scientists but a cross-disciplinary catalyst, accelerating scientific discoveries, pioneering new methodologies across scientific fields. In this course, we will introduce the latest machine learning techniques, exploring topics such as generative models, Large Language Models, transfer learning, and meta-learning. This is not just a technical deep-dive; it is a platform for innovation where students from a spectrum of academic backgrounds will come together to apply ML in novel ways. The course targets to encouraging curiosity for students to explore how ML can address unique challenges or answer unresolved questions within their specific areas of study.
Class: Time: Monday/Wednesday, 2:00pm-3:15pm; Location: DeBartolo Hall 118 (1/13 to 4/30)
Instructor: Prof. Xiangliang Zhang, xzhang33@nd.edu 354 Fitzpatrick Hall
Office Hours: Wednesdays from 11:30 AM to 1:30 PM, or by appointment
TA: Tianyu Yang, tyang4@nd.edu Office Hours: TBD
Prerequisite: Students are expected to come in with a foundational knowledge of artificial intelligence concepts and be proficient in Python programming. These prerequisites are crucial as they will enable students to dive into advanced machine learning topics and hands-on project work with confidence and a clear understanding.
Course Objectives: The objectives of this course are tailored to empower students from a wide array of academic backgrounds. Computer science students could deepen their research. Students from other disciplines could integrate ML into their fields. Through a combination of modular learning, hands-on projects, and collaborative exploration, students will gain not only knowledge but also practical experience. Students will work on projects that are aimed at pioneering new ML applications or advancing current models, with a direct relevance to the students’ individual fields of study. The culminating goal of the course is for each student to catch latest machine learning techniques and develop a research paper that is potentially be publishable.
Assignments: The assignments are composed of paper-review/presentation and research project.
- Paper-review/presentation will be organized by topic. Each student is required to sign up for at least signup for one topic to review and present. For each lecture, the instructor will provide an introductory overview of the topic, followed by one or two student presentations on the papers they have reviewed. Students are encouraged to discuss their chosen paper with the instructor and seek guidance to ensure a thorough understanding of the material. Other students are required to submit a written summary or analysis of the topics and papers discussed in each unit. This ensures active engagement and a deeper understanding of the material for all students.
- The assignments about research project aims for evaluating the project progress.
- Research Proposals. Students are required to submit detailed research proposals outlining a specific problem in machine learning, proposed solutions, and a clear methodology. The proposal should also include a preliminary literature review and a potential impact assessment. Students will share their proposal as a 10-minute class presentation.
- Project Report. A project report should be submitted to summarize the project development, results, findings, contributions, impact, aiming at a publishable standard.
- Peer Reviews: we will incorporate a peer review process where students evaluate and provide feedback on each other’s draft papers. This mimics the academic review process and helps students learn to give and receive constructive criticism.
- Final presentation. Students will present their final research results, or findings. This helps develop students’ communication skills, particularly in explaining complex technical content.
- The project can be completed in teams of 1-3 students.
Grading:
Paper-review panels 30%
Research proposal 20%
Project report 20%
Peer reviews 10%
Final presentation 20%
Numeric grades are computed as percentages, and correspond to letter grades as follows:
| 87≤B+<90% | 77≤C+<80% | 67≤D+<70% | F<60% | |
| 94≤A≤100% | 84≤B<87% | 74≤C<77% | 64≤D<67% | |
| 90≤A-<94% | 80≤B-<84% | 70≤C-<74% | 60≤D-<64% |
Course Policies:
- Attendance at class is required. Please let me know, if you cannot attend a lecture in person before the class, with reasonable excuse.
- Details of the assignments and the project will be posted on Canvas. Submission and grading will be through Canvas as well.
- There are no late submissions, and no partial credit will be given if the submission is after the due date. Any extensions must require extenuating circumstances or a priori negotiation. In principle, each one can at most have an extension to only one submission.
- Inquiries about grades must be made in writing within one week after they are posted.
Honor Code: Academic integrity is required. No academic dishonesty in any form is tolerated. The University’s Honor Code (http://honorcode.nd.edu/) reminds our community of our shared purpose both within the institute of academia and as members of a broader humanity; the statement also outlines policy violation procedures. Any questions regarding academic integrity, particularly regarding assignments in this course, should be directed to the instructor.
ChatGPT or other Large Language Models could be used for project report polishing. However, it should not be used for the unit summary, peer review assignment and slides presentation.
Course Content (slides, and assignments can be found at Canvas)
| Classes | Topic | Assignment | ||
| Jan 13 (M) | Welcome & Introduction | Filling the survey of research project topics/interests (due Jan 15 Wed 11:59pm) | ||
| Unit 1 Fundamentals | ||||
| Jan 15 (W) | Review of the development of Neural Networks | signup for paper-review/presentation (due Jan 27 11:59pm) | ||
| Jan 22 (W) NN review continued | The evolution of machine learning (ML) | |||
| Jan 27 (M) | Recent ML models training | |||
| Jan 29 (W) | Applications of machine learning (project ideas) | |||
| Unit 2 Modern Themes in Machine Learning | ||||
| Feb 3 (M), 5 (W) | Representation Learning (Joseph’s) | Project proposal due Feb 20 11:59pm Unit 2 summary due Feb 28 11:59pm | ||
| Feb 10 (M), 12 (W) | Generative Models | |||
| Feb 17 (M), 19 (W) | Application of Generative Models | |||
| Feb 24 (M), 26 (W) | Proposal Presentation | |||
| Unit 3 Advanced Learning Strategies | ||||
| Mar 3 (M), 5 (W) | Self-supervised learning | Unit 3 summary due Apr 4 11:59pm Project report due Apr 14 11:59pm | ||
| Mar 17 (M), 19 (W) | Transfer learning (pre-training, fine-tuning) | |||
| Mar 14 (M) | In-context learning, Prompt engineering | |||
| Mar 26 (W) | Knowledge Distillation | |||
| Unit 4 Next Generation of Machine Learning | ||||
| Mar 31 (M) | (Multi-modal) Foundation Models | Peer-review due Apr 22 11:59pm Unit 4 summary due Apr 25 11:59pm Project slides due Apr 27 11:59pm | ||
| Apr 2 (W) | Artificial General Intelligence | |||
| Apr 7 (M) | Multi-agent AI systems | |||
| Apr 9 (W) | Explainable AI models | |||
| Apr 14 (M), 16 (W) | ML security, Adversarial learning | |||
| Research Project review, presentation | ||||
| Apr 23 (W) | Project review discussion | |||
| Apr 29 (M), 30 (W) | Project presentation | |||
| No Exam | ||||
*No class in week 9 (Mar 10, 12 for the mid-term break), and Apr 21 (for the Easter holiday).