This unique text for beginning graduate students gives a self-contained introduction to the mathematical properties of stochastics and presents their applications to Markov processes, coding theory, population dynamics, and search engine design. The book is ideal for a newly designed course in an introduction to probability and information theory. Prerequisites include working knowledge of linear algebra, calculus, and probability theory. The first part of the text focuses on the rigorous theory of Markov processes on countable spaces (Markov chains) and provides the basis to developing solid probabilistic intuition without the need for a course in measure theory. The approach taken is gradual beginning with the case of discrete time and moving on to that of continuous time. The second part of this text is more applied; its core introduces various uses of convexity in probability and presents a nice treatment of entropy.