Patter Recognition (CECS 550)

Course Outline

This graduate-level course delves into pattern recognition theories, techniques, and applications. Understanding core principles, gaining practical skills, and researching advanced issues in the discipline will be prioritized. Students will participate in hands-on exercises and projects to reinforce theory and gain experience with real-world pattern recognition tasks.

Learning Outcomes

The primary objectives of this course is for the students to be able to:

• Analyze and explain the core principles and methods of pattern recognition.
• Apply various pattern recognition algorithms to solve complex problems.
• Design and run experiments to evaluate the effectiveness of pattern recognition models.
• Critically assess the merits and limitations of different pattern recognition techniques.
• Implement pattern recognition applications using appropriate programming tools.

  Modules

Probability, Statistics, and Linear Algebra Review

• Basic concepts of probability: random variables, probability distributions, expected value, variance, and covariance.
• Vector spaces, Norms, and Matrix calculation
• Linear systems of equations, and Eigenvalue problems.
• Statistical measures: mean, median, mode, standard deviation, and correlation.

Supervised Learning

Linear Models for Regression and Classification:

• Linear regression and polynomial regression
• Bias-variance trade-off and model selection
• Binary classification and logistic regression
• Multiclass classification and softmax regression
• Generative vs. discriminative classifiers

Neural Networks and Deep Learning:

• Basics of shallow and deep neural networks
• Feedforward neural networks
• Backpropagation algorithm and training
• Regularization techniques for neural networks

Kernel Methods and Support Vector Machines:

• Non-linear classification using kernel functions
• Support vector machines (SVMs) for binary and multiclass classification
• Mercer’s theorem and the kernel trick

Unsupervised Learning

Clustering Algorithms

• K-Means Clustering
• Hierarchical Clustering
• Practical Issues in Clustering

Dimensionality Reduction:

• Principal Component Analysis (PCA)
• Probabilistic PCA and Factor Analysis
• Independent Component Analysis (ICA)

Generative Models

• Generative Adversarial Networks
• Normalizing flows
• Variational autoencoders
• Diffusion models

Reinforcement Learning

• Markov decision processes
• Expected return
• Tabular reinforcement learning
• Fitted Q-learning Policy gradient methods
• Actor-critic methods

Learning Resources

Zero Cost Course Materials: This course does not have a required textbook. However, some readings will be assigned from open educational resources based on the topic. Here are a number of excellent machine learning books you can access for free:

• The Little Book of Deep Learning by François Fleuret. Available through CC BY-NC-SA license. This book is a great primer for deep learning and it covers many of the pre-requisites for our class.
• An Introduction to Statistical Learning with Applications in Python, by G. James, D. Witten, T. Hastie, and R. Tibshirani. Made available by the authors. Standing in the middle of theory and practice, this book provides a more statistical view of machine learning and is also a great reference for python implementations of traditional ML.
• Pattern Recognition and Machine Learning, by C.M. Bishop. Available from Microsoft Research. This is the book if you want to learn machine leaning theory at a deeper level of understanding.

Evaluation Components

Every practical project in this course entails some form of coding in Python. The students will compile their findings into a brief report, which they will submit along with a link to their reproducible code on Github. The projects are not intended to be at the level of the industry, but rather to improve the students’ skills in using ML for a particular application. Maintaining sound software engineering practices and technical documentation is crucial for a successful career and as such, it is part of the evaluation.

Extra credit will be available throughout the semester to make up for an assignment that did not earn you high marks. These will typically involve solving problems in novel ways or researching an interesting topic and reporting back to the class on it.

Grading Policy

Special Accommodations

We are committed to providing equal opportunities for all students in our courses. If you require any special accommodations due to a documented disability, medical condition, or other circumstance that may affect your ability to fully participate in the course, please notify me as early as possible. Also, please contact the Bob Murphy access center (BMAC) for more information. BMAC personnel will work with students to identify a reasonable accommodation in partnership with appropriate academic offices and medical providers. Only approved BMAC petitions will be accommodated.

Any student who is facing academic or personal challenges due to difficulty in affording groceries/food and/or lacking a safe and stable living environment is urged to contact the CSULB Student Emergency Intervention & Wellness Program. Additional resources are available via Basic Needs Program.For mental health assistance please check out CSULB Counseling and Psychological Services (CAPS).

Academic Integrity

Before attending the class, please make sure that you have invested some time to read about CSULB academic integrity policy here. Of particular importance, are the definitions of Plagiarism and Cheating, and the steps the university will take to adjudicate academic integrity cases.

Work that you submit is assumed to be original unless your source material is documented appropriately, using proper citation. This includes using AI assistance in writing, coding and solving problems. At a minimum, any student caught violating Academic Integrity Policy will receive no credit for the work concerned and one grade lower letter grade.

Disclaimer

Please be aware that this course outline may change without prior notice. There may be instances where adjustments are needed for the course content, schedule, or grading policy. As a result, it is important that students check the official course syllabus on canvas regularly for the latest updates.