Introduction to Stochastic Processes

About this book

Concise text which examines advanced topics (Markov chains, renewal theory, martingales, etc) without a stress on mathematical rigour in the explanations.

Contents

Preface to Second Edition Preface to First Edition PRELIMINARIES Introduction Linear Differential Equations Linear Difference Equations Exercises FINITE MARKOV CHAINS Definitions and Examples Large-Time Behavior and Invariant Probability Classification of States Return Times Transient States Examples Exercises COUNTABLE MARKOV CHAINS Introduction Recurrence and Transience Positive Recurrence and Null Recurrence Branching Process Exercises CONTINUOUS-TIME MARKOV CHAINS Poisson Process Finite State Space Birth-and-Death Processes General Case Exercises OPTIMAL STOPPING Optimal Stopping of Markov Chains Optimal Stopping with Cost Optimal Stopping with Discounting Exercises MARTINGALES Conditional Expectation Definition and Examples Optional Sampling Theorem Uniform Integrability Martingale Convergence Theorem Maximal Inequalities Exercises RENEWAL PROCESSES Introduction Renewal Equation Discrete Renewal Processes M/G/1 and G/M/1 Queues Exercises REVERSIBLE MARKOV CHAINS Reversible Processes Convergence to Equilibrium Markov Chain Algorithms A Criterion for Recurrence Exercises BROWNIAN MOTION Introduction Markov Property Zero Set of Brownian Motion Brownian Motion in Several Dimensions Recurrence and Transience Fractal Nature of Brownian Motion Scaling Rules Brownian Motion with Drift Exercises STOCHASTIC INTEGRATION Integration with Respect to Random Walk Integration with Respect to Brownian Motion Ito's Formula Extensions if Ito's Formula Continuous Martingales Girsanov Transformation Feynman-Kac Formula Black-Scholes Formula Simulation Exercises Suggestions for Further Reading Index

Customer Reviews