Course Syllabus

Jump to Today

EE 140: Stochastic Processes, Detection and Estimation

Fall 2022

Overview

The goal of this class is the development of the probabilistic modeling and analysis methods underlying much of machine learning and data science. It is intended for those interested in understanding precisely how and why these techniques work with an eye toward performing research that extends and enhances them. We start with a discussion of multivariate probability theory focussing especially on Gaussian random vectors.  These ideas serve as the basis for developing and analyzing principled, probabilistic methods for the two basic tasks in machine learning: classification (a.k.a. hypothesis testing) and regression (a.k.a. estimation).  We next turn to the modeling and characterization of random signals considering especially ideas of stationarity (the probabilistic version of time invariance) and Markovianty (a tractable means of modeling processes that have some memory to them).  Starting from a series of independent coin flips, we will develop a number of stochastic processes including the Poisson and Weiner processes that are used throughout science and engineering to model a huge range of natural phenomena.  The remainder of the class brings together basic ideas in linear systems theory such as causality and shift invariance with the probabilistic methods developed earlier to develop and explore ideas in the construction of statistically "optimal" systems for achieving a variety if processing objectives.  

Staff

Prof. Eric Miller
zoom: https://tufts.zoom.us/my/ericlmiller 
email:  eric.miller@tufts.edu
Office: Room 616 Joyce Cummings Center

Schedule

  • Lectures:
    • Tuesdays and Thursdays, 1:30 PM - 2:45 PM, SciTech Room 134
  • Office Hours
    • Mondays 11:00 AM - 12:00 PM, 616 Joyce Cummings Center 
    • Wednesday 11:00 AM - 12:00 PM, zoom
    • Thursdays 3:00 PM - 4:00 PM, 616 Joyce Cummings Center
    • By appointment: email Prof. Miller

Prerequisites

  • Probability and Statistics: EE-24/104 or equivalent
  • Linear Systems: EE-23 or equivalent
  • Linear Algebra/Vector Spaces: Math 70 or equivalent

Tentative Grading Scheme

  • Homeworks: 30%
  • In-class mid-term exam: 30%
  • Final or final project: 40%

Academic Integrity

Tufts holds its students strictly accountable for adherence to academic integrity. The consequences for violations can be severe. It is critical that you understand the requirements of ethical behavior and academic work as described in Tufts’ Academic Integrity handbook. If you ever have a question about the expectations concerning a particular assignment or project in this course, be sure to ask me for clarification. The Faculty of the School of Arts and Sciences and the School of Engineering are required to report suspected cases of academic integrity violations to the Dean of Student Affairs Office. If I suspect that you have cheated or plagiarized in this class, I must report the situation to the dean.

Texts

Required

  • Shynk, John J.  Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications. Wiley, 2013.
  • Yates, Roy D., and David J. Goodman.  Probability and Random Processes: A Friendly Introduction for Electrical and Computer Engineers. Third Edition. Wiley, 2014.
  • Levy, Bernard C., Principles of Signal Detection and Parameter Estimation, Springer, 2008.  Tufts students should be able to access this text online here.
  • Notes and additional chapters from the two texts on the Canvas Site under Files/ClassNotes 

Useful

  • Leon-Garcia, Alberto. Probability, Statistics and Random Processes for Electrical Engineering. Third Edition, Prentice Hall, 2008
  • Kobayashi, Hisashi, Brian L. Mark, and William Turin. Probability, Random Processes, and Statistical Analysis: Applications to Communications, Signal Processing, Queuing Theory and Mathematical Finance. Cambridge University Press, 2011.
  • Gubner, John A., Probability and Random Processes for Electrical and Computer Engineers, Cambridge University Press, 2006.
  • Stark, Henry, and John William Woods.  Probability, Statistics and Random Processes for Engineers. Prentice-Hall, 2011.

Tentative Schedule

Tentative list of topics for EE140, Fall 2022
Date Topic
Sept. 6 Overview, Introduction to Random Vectors, First and Second Moments
Sept. 8 First and Second Moments of Random Vectors, Gaussian Models
Sept. 13 Gaussian Models
Sept. 15 Gaussian Models, Introduction to Estimation, Minimum Mean Square Optimal Estimation
Sept. 20 Minimum Mean Square Optimal Estimation
Sept. 22 Linear Least Squares Optimal Estimation
Sept. 27 MSE and Maximum a Posteriori Estimation
Sept. 29 Maximum a Posteriori Estimation, Maximum Likelihood Estimation
Oct. 4 Cramer-Rao Bound, Vector Estimation
Oct. 6 Vector Estimation and Binary and M-Ary Hypothesis Testing
Oct 11. Binary and M-Ary Hypothesis Testing
Oct. 13 Binary and M-Ary Hypothesis Testing
Oct. 18 Binary and M-Ary Hypothesis Testing
Oct. 20 Binary and M-Ary Hypothesis Testing
Oct. 25 Binary and M-Ary Hypothesis Testing
Oct. 27 M-Ary Hypothesis Testing
Nov. 1 Composite Hypothesis Testing
Nov. 3 Introduction to Random Processes
Nov. 8 NO CLASS: FRIDAY SCHEDULE
Nov. 10 Student led class 1: Shutong on Chapter 6 material
Nov. 15 Student led class 2: Yukun on Chapter 9 material
Nov. 17 Student led class 3: Xiaohui on Chapter 23 material
Nov. 22 Student led class 4: Michael on Chapter 3 material Stationarity, First and Second Moment Analysis of Random Processes
Nov. 24 Linear Shift Invariant Systems and Wide Sense Stationary Processes, Power Spectral Density First and Second Moment Analysis of Random Processes
Nov. 29 Cross-spectral Density and Optimal Estimation Independent and Identically Distributed Processes
Dec. 1 Optimal Estimation, Non-Causal Wiener Filter, Causal Wiener Filter Markov Processes, Discrete Time Markov Chains
Dec. 6 Discrete Wiener Filters Bernoulli Processes, Poisson Processes
Dec. 8 Discrete Wiener Filters Wiener Process

 

Course Summary:

Course Summary
Date Details Due