Description: Machine Learning (ML) has undergone a transformative shift since the early 2010s, marked by significant advancements in deep learning. These changes have paved the way for groundbreaking models such as AlphaGo and AlphaZero, creative generators like DALL-E, and sophisticated Large Language Models exemplified by ChatGPT. Today, ML is not just a tool for computer scientists but a cross-disciplinary catalyst, accelerating scientific discoveries, pioneering new methodologies across scientific fields. In this course, we will introduce the latest machine learning techniques, exploring topics such as generative models, Large Language Models, transfer learning, and meta-learning. This is not just a technical deep-dive; it is a platform for innovation where students from a spectrum of academic backgrounds will come together to apply ML in novel ways. The course targets to encouraging curiosity for students to explore how ML can address unique challenges or answer unresolved questions within their specific areas of study.
Class: Time: Tuesday/Thursday, 2:00pm-3:15pm; Location: DeBartolo Hall 232 (8/25 to 12/10)
Instructor: Prof. Xiangliang Zhang, xzhang33@nd.edu 354 Fitzpatrick Hall
Office Hours: Thursday from 11:30 AM to 1:30 PM, or by appointment
Prerequisite: Students are expected to come in with a foundational knowledge of artificial intelligence concepts and be proficient in Python programming. These prerequisites are crucial as they will enable students to dive into advanced machine learning topics and hands-on project work with confidence and a clear understanding.
Course Objectives: The objectives of this course are tailored to empower students from a wide array of academic backgrounds. Computer science students could deepen their research. Students from other disciplines could integrate ML into their fields. Through a combination of modular learning, hands-on projects, and collaborative exploration, students will gain not only knowledge but also practical experience. Students will work on projects that are aimed at pioneering new ML applications or advancing current models, with a direct relevance to the students’ individual fields of study. The culminating goal of the course is for each student to catch latest machine learning techniques and develop a research paper that is potentially be publishable.
Assignments: The assignments are composed of paper-prereview/presentation and research project.
- Paper-review/presentation
- Paper Preview (Before Class): selected research papers will be assigned for reading before class. Students are required to read the assigned paper(s) in advance and answer a set of guiding questions before class. These responses should be submitted prior to class and will serve as the basis for class discussion.
- Class Discussion: Assigned papers will be discussed in class, with all students expected to contribute to the conversation.
- Presentation: Each student must sign up for at least one topic/paper to review and present during the semester. Students are encouraged to discuss their chosen paper with the instructor in advance to ensure accurate comprehension and critical analysis. The presentation should cover the key ideas, methods, and findings of the paper, along with critical insights (strengths, limitations, open questions).
- Unit Summaries / Written Analysis. At the end of each unit, each student will submit a written summary or analysis of the topics and papers discussed. These write-ups should demonstrate understanding of the main concepts, synthesize ideas across papers, and reflect on open research directions.
- The assignments about research project aims for evaluating the project progress.
- Research Proposals. Students are required to submit detailed research proposals outlining a specific problem in machine learning, proposed solutions, and a clear methodology. The proposal should also include a preliminary literature review and a potential impact assessment. Students will share their proposal as a 10-minute class presentation.
- Project Report. A project report should be submitted to summarize the project development, results, findings, contributions, impact, aiming at a publishable standard.
- Peer Reviews: we will incorporate a peer review process where students evaluate and provide feedback on each other’s draft papers. This mimics the academic review process and helps students learn to give and receive constructive criticism.
- Final presentation. Students will present their final research results, or findings. This helps develop students’ communication skills, particularly in explaining complex technical content.
Grading:
Paper-review/discussion 40%
Research proposal 10%
Project report 20%
Peer reviews 10%
Final presentation 20%
Numeric grades are computed as percentages, and correspond to letter grades as follows:
| 87≤B+<90% | 77≤C+<80% | 67≤D+<70% | F<60% | |
| 94≤A≤100% | 84≤B<87% | 74≤C<77% | 64≤D<67% | |
| 90≤A-<94% | 80≤B-<84% | 70≤C-<74% | 60≤D-<64% |
Course Policies:
- Attendance at class is required. Please let me know, if you cannot attend a lecture in person before the class, with reasonable excuse.
- Details of the assignments and the project will be posted on Canvas. Submission and grading will be through Canvas as well.
- There are no late submissions, and no partial credit will be given if the submission is after the due date. Any extensions must require extenuating circumstances or a priori negotiation. In principle, each one can at most have an extension to only one submission.
- Inquiries about grades must be made in writing within one week after they are posted.
Honor Code: Academic integrity is required. No academic dishonesty in any form is tolerated. The University’s Honor Code (http://honorcode.nd.edu/) reminds our community of our shared purpose both within the institute of academia and as members of a broader humanity; the statement also outlines policy violation procedures. Any questions regarding academic integrity, particularly regarding assignments in this course, should be directed to the instructor.
ChatGPT or other Large Language Models could be used for project report polishing. However, it should not be used for the unit summary, peer review assignment and slides presentation.
Course Content (slides, and assignments can be found at Canvas)
| Classes | Topic | Assignment |
| Aug 26 (Tue) | Welcome & Introduction | Filling the survey of interested topics (due Aug 27 Wed 11:59pm) |
| Unit 1 Fundamentals | ||
| Aug 28 (Thu) | Review: the development of Neural Networks | signup for paper presentation (due Sep 5 11:59pm) |
| Sep 2 (Tue) | The evolution of machine learning (ML) | |
| Sep 4 (Thu) | Applications of machine learning (paper for reading) | |
| Unit 2 Modern Themes in Machine Learning | ||
| Sep 9 (Tue), 11 (Thu) Gonzalo’s DeepWalk slides | Representation Learning (word2vec, DeepWalk, Bengio’s Representation Learning Paper) | Project proposal due Sep 26 11:59pm Unit 2 summary due Oct 5 11:59pm |
| Sep 16 (Tue), 18 (Thu) | Generative Models | |
| Sep 23 (Tue), 25 (Thu) | Application of Generative Models | |
| Sep 30 (Tue) | Proposal Presentation | |
| Unit 3 Advanced Learning Strategies | ||
| Oct 2 (Thu), 7 (Tue) | Self-supervised learning | Unit 3 summary due Nov 5 11:59pm |
| Oct 9 (Thu) | Transfer learning (pre-training, fine-tuning) | |
| Oct 14 (Tue) | In-context learning, Prompt engineering | |
| Oct 16 (Thu) | Knowledge Distillation | |
| Oct 18-26 No class (Fall Break) | ||
| Oct 28 (Tue) | Explainable ML | |
| Oct 30 (Thu) | Adversarial learning | |
| Unit 4 Next Generation of Machine Learning Models | ||
| Nov 4 (Tue) | Artificial General Intelligence | Project report and slides due Nov 30 11:59pm Unit 4 summary due Dec 10 |
| Nov 6 (Thu), 11 (Tue) | (Multi-modal) Foundation Models | |
| Nov 13 (Thu) | Multi-agent AI systems | |
| Nov 18 (Tue), 20 (Thu) | Reinforcement Learning for Foundation Models | |
| Nov 25 (Tue) | ML security, safety, trustworthiness | |
| Research Project presentation, review, discussion | ||
| Dec 2 (Tue), 4 (Thu) | Project presentation | Peer-review due Dec 7 11:59pm |
| Dec 9 (Tue) | Review | |
| No Exam | ||