In this lecture, we put together many of the topics covered throughout the term: martingales; Markov chains; countable state Markov processes; reversibility for Markov processes; random walks; and Wald's identity for two thresholds.

Lecture 25: Putting It All Together

Sequential hypothesis testing is viewed as a random walk example.  Threshold hypothesis tests are distinguished from random walk thresholds.  Random walk threshold probabilities are analyzed by Chernoff bounds.

Lecture 21: Hypothesis Testing and Random Walks

This lecture continues our conversation on Martingales and covers stopped martingales, Kolmogorov submartingale inequality, martingale convergence theorem, and more.

Lecture 24: Martingales: Stopping and Converging

After reviewing Wald's identity, we introduce martingales and show they include many processes already studied.  Next, submartingales, supermartingales, and stopped (simple, sub, super) martingales are introduced.

Lecture 23: Martingales (Plain, Sub, and Super)

This lecture covers topics including the Kingman bound for G/G/1, large deviations for hypothesis tests, sequential detection, and tilted random variables and proof of Wald's identity.

Lecture 22: Random Walks and Thresholds

After reviewing steady-state, this lecture discusses reversibility for Markov processes and for tandem M/M/1 queues.  Random walks and their applications are then introduced.

Lecture 20: Markov Processes and Random Walks

Markov processes with countable state-spaces are developed in terms of the embedded Markov chain.  The steady-state process probabilities and the steady-state transition probabilities are treated.

Lecture 19: Countable-state Markov Processes

In this lecture, the professor covers sample-time M/M/1 queue, Burke’s theorem, branching processes, and Markov processes with countable state spaces.

Lecture 18: Countable-state Markov Chains and Processes

This lecture reviews the previous 13 lectures in preparation for the upcoming quiz.

Lecture 14: Review

This lecture covers a variety of topics, including elementary renewal theorem, generalized stopping trials, the G/G/1 queue, Little's theorem, ensemble averages and more.

Lecture 13: Little, M/G/1, Ensemble Averages

After reviewing the three major renewal theorems, we introduce Markov chains with countable state spaces.  The matrix approach for finite-state chains is replaced by renewals based on first-passage times.

Lecture 16: Renewals and Countable-state Markov

In this lecture, we continue our discussion of renewals and cover topics such as Markov chains and renewal processes, expected number of renewals, elementary renewal and Blackwell theorems, and delayed renewal processes.

Lecture 15: The Last Renewal

This lecture begins with a discussion of convergence WP1 related to a quiz problem.  Then positive and null recurrence, steady state, birth-death chains, and reversibility are covered.

Lecture 17: Countable-state Markov Chains

This lecture begins with the SLLN and the central limit theorem for renewal processes.  This is followed by the time-average behavior of reward functions such as residual life.

Lecture 11: Renewals: Strong Law and Rewards

This lecture covers rewards for Markov chains, expected first passage time, and aggregate rewards with a final reward. The professor then moves on to discuss dynamic programming and the dynamic programming algorithm.

Lecture 9: Markov Rewards and Dynamic Programming

This lecture covers eigenvalues and eigenvectors of the transition matrix and the steady-state vector of Markov chains. It also includes an analysis of a 2-state Markov chain and a discussion of the Jordan form.

Lecture 8: Markov Eigenvalues and Eigenvectors

This lecture treats joint conditional densities for Poisson processes and then defines finite-state Markov chains. Recurrent and transient states, periodic states, and ergodic chains are discussed.(Courtesy of Mina Karzand.  Used with permission.)

This lecture treats joint conditional densities for Poisson processes and then defines finite-state Markov chains. Recurrent and transient states, periodic states, and ergodic chains are discussed.<br><br>(Courtesy of Mina Karzand.  Used with permission.)

Lecture 6: From Poisson to Markov

In this lecture, many problem solving techniques are developed using, first, combining and splitting of various Poisson processes, and, second, conditioning on the number of arrivals in an interval.

Lecture 5: Poisson Combining and Splitting

In this lecture, we learn about time-averages for renewal rewards, stopping trials for stochastic processes, and Wald's equality.

Lecture 12: Renewal Rewards, Stopping Trials, and Wald's Inequality

Renewal processes are introduced and their importance in analyzing other processes is explained.  Proofs about convergence with probability 1 (WP1) and the SLLN are given.

Lecture 10: Renewals and the Strong Law of Large Numbers

Discrete Stochastic Processes

The best philosophy podcast episodes. Asking questions about how to live a better life, what it means to be and do good, to the idea of God. This topic includes episodes that bring about important questions that everyone should be asking themselves. Add episodes you're interested into your queue below!

Best Episodes on Practical Philosophy & How To Live

The best podcast episodes on non-current politics. Covering topics such as universal basic income, democracy, poverty & justice and more. For anyone interested in learning more about politics that affect that world, not just now, but in the longer term. Add episodes you're interested into your queue below!