MA/STAT 519: Introduction to Probability Theory
Spring 2018, Purdue University
http://www.math.purdue.edu/~yip/519
Course Description:

Algebra of sets, sample spaces, combinatorial problems, conditional
probability and independence,
discrete and continuous random variables, distribution and joint distribution functions, expectation of random variables, moment generating
functions, limit theorems and applications.
Instructor:
 Aaron Nung Kwan
Yip
 Department of
Mathematics
 Purdue University
Contact Information:
 Office: MATH 432
 Email:
click here
Lecture Time and Place:
 T, Th 10:30  11:45, UNIV 117
Office Hours:
 Tue, 4:305:30pm, Wed, 3:004:00pm, or by appointment.
Textbook:

(All of the following are either available online or on reserve in
the math library.)
Main Text:
[CD] Probability with Applications in Engineering, Science, and
Technology,
by Matthew A. Carlton, Jay L. Devore, 2nd edition
(available online from Purdue library page)
Reference:
[R] A First Course in Probability, by Sheldon Ross
(I am using the 9th edition,
but older editions are OK for reading and reference)
[HPS] Introduction to Probability Theory, by Hoel, Port and Stone
[D1] Fundamentals of Probability: a First Course,
by Anirban DasGupta (available online from Purdue library page)
[D2] Probability for Statistics and Machine Learning:
Fundamentals and Advanced Topics,
by Anirban DasGupta (available online from Purdue library page)
[F1, F2] An Introduction to Probability Theory and Its Applications,
Volume 1 and 2,
by William Feller
Course Outline:
 The course will cover most of the sections of
[CD] Chapters 1 to 4, and part of Chapter 5.
Prerequisites:

Good "working" knowledge of:
linear algebra (e.g. MA 265, 351, 511),
vector calculus (e.g. MA 261, 362, 510),
and mathematical analysis (e.g. MA 341, 440, 504).
Homework:

Homeworks will be assigned roughly biweekly.
They will be gradually assigned as the course progresses.
Please refer to the course announcement below.
 Steps must be shown to explain your answers.
No credit will be given for just writing down the answers, even
if it is correct.
 Please staple all loose sheets of your homework to prevent
5% penalty.
 Please resolve any error in the grading (hws and tests)
WINTHIN ONE WEEK after the return of each homework and exam.
 No late homeworks are accepted (in principle).
 You are encouraged to discuss the homework problems with
your classmates but all your handedin homeworks must be your
own work.
Examinations:
 Midterm Test: Date to be determined
 Final Exam: During Final Exam Week
Grading Policy:
 Homeworks (50%)
 Midterm Test (20%)
 Final Exam (30%)
You are expected to observe academic honesty
to the highest standard. Any form of cheating will automatically
lead to an F grade, plus any other disciplinary action,
deemed appropriate.
Accommodations for Students with Disabilities and
Academic Adjustment:
 University policy will be followed.
Course Progress and Announcement:
 (You should consult this section regularly,
for homework assignments, additional materials and announcements.)
WEEK 1
Jan 9 (Tue):
[CD 1.11.2][R 2.12.4][D1, Chapter 1]
Concept of statistical regularity,
Law of Large Numbers (LLN),
Central Limit Theorem (CLT),
Probability space, outcome, events, logical operations between events,
complement, unions, intersections, de Morgan's Law.
Jan 11 (Thur):
[CD 1.11.2][R 2.12.4][D1, Chapter 1]
Probability measure, countablyadditivity,
inclusionexclusion principle [R p. 30, Proposition 4.4][D1, 1.5],
objective vs subjective approaches to probability measures.
WEEK 2
Jan 16 (Tue):
Some theoretical remarks about probability measures:
countablyaddivitiy, (non)measureable sets, typical outcomes,
equallylikelyoutcome probability space;
[CD 1.3][R 1.11.4][D1 1.4]
counting techniques: permutations and combinations.
Jan 18 (Thur) [R 1.11.6]:
binomial and multinomial coefficients and expansions,
number of integer solutions.
Homework 1, due Thursday, Jan 25,
in class
WEEK 3
Jan 23 (Tue): [D1 6.5, Chapter 2][R 2.5][CD 1.3.3, 2.6.1]
Sampling (partition, hypergeometric distribition),
birthday and matching problems,
asymptotics of formula when some number
(e.g. population size) is large.
Jan 25 (Thur):
[D1 Chapter 15] Urn and occupancy problems,
MaxwellBoltzmann, BoseEinstein and FermiDirac Distributions;
[R 3.13.3][D1 Chapter 3][CD, 1.4]
Conditional probability,
Bayes' forward (formula of total probability) and
backward (updating prior) formula,
prior and posterior probabilities.
WEEK 4
Jan 30 (Tue):
(cont.) Bayes' backward and forward formula, prior and posterior
probabilities, sensitivity and specificity of medical
tests.
[R, 3.13.4][D1 Chapter 3][CD 1.5]
independence between two, three, and multiple events.
Feb 1 (Thur):
[R, 3.13.4][D1 Chapter 3][CD 1.5]
(cont.) (jointly) indepedence between multiple events.
"Probability paradoxes": Monty Hall
Homework 2, due Thursday, Feb 8, in class
WEEK 5
Feb 6 (Tue):
"Probability paradoxes": (Mr. Smith's) Boy vs Girl.
computation of probability by iteration/recursion/induction,
an example [R, e.g. 5d, p. 93]: matching problem.
Feb 8 (Thur):
iterations/recursion/induction, Gambler's Ruin Problem [R, p. 84],
trials with random (unknown) parameters,
conditional independence.
WEEK 6
Feb 13 (Tue):
[R 4.14.6][D1 Chapter 4][DC Chapter 2.12.4]
(Discrete) Random variables (numerical observations of experimental
outcomes),
probability mass (density) function (pmf, pdf) and
cumulative density function (cdf),
expectation, variance, CauchySchwarz Inequality,
Bernoulli, Binomial, and Poisson random variables.
Note on random variables and expectation.
Feb 15 (Thur):
Variance of Binomial and Poisson random variables
Analytical computation of expectations and
variance of Binomial and Poisson random variables
[R Chapter 4/6.26.3/7.27.3]
Joint random variables
Joint (bivariate and multivariate) random variables,
joint pdf, marginal pdfs,
independent random variables,
computations of Ef(X,Y), E(X+Y), E(XY)
An example of uncorrelated but dependent random variables.
Homework 3, due Thursday, Feb 22, in class
WEEK 7
Feb 20 (Tue):
use of indicator functions to compute expectations and variances,
applied to, Binomial r.v., matching, exclusioninclusion principle.
geometric random variablearrival time of first success,
negative Binomial random variablesarrival time of kth success
Feb 22 (Thur):
reinterpretation of negative Binomial as sum of geometric r.v.
(interarrival times), convolution between discrete pdfs,
Hypergeometric random variable.
WEEK 8
Feb 27 (Tue):
convergence of Hypergeometric to Binomial random variables,
Poisson random variables, as a result of
a large number of weakly dependent
Bernoulli trials each with very small success probability,
convergence of Binomial random variable to Poisson random variable.
Two properties of Poisson random variables:
merging and splitting of Poisson variables.
Mar 1 (Thur):
[R p. 140][D1 p. 110] "Poisson Paradigm",
application of Poisson approximation to weakly dependent examples:
hat matching problem, birthday problem,
Le Cam Theorem on error estimate for Poisson approximation [D1, p. 110],
A review paper on Poisson approximation by Serfling (1978),
Continuum limit of Binomial random variables, and first success time:
converge of geometric r.v. to exponential r.v..
Homework 4, due Thursday, Mar 8, in class
WEEK 9
Mar 6 (Tue):
Converge of negative Binomial r.v. to Gamma r.v..
Note on negative hypergeometric
random variables.
[R, Chapter 5][D1, Chapter 7]
Continuous random variables, probability density function (pdf) f(x),
and cumulative density function (cdf) F(x).
Note on continuous random
variables, their pdfs and cdfs.
Mar 8 (Thur):
Change of variable formula for pdf of continuous random variables
(one dimensional case),
Y = aX + b, Y = g(X), Y = X^2,
expectation of continuous random variables.
WEEK 10
Midterm, Thursday Mar 22, in class
(review on Mar 20)
Materials covered: Ross Chapters 14, Homeworks 14.
A past exam
No electronic devices. A one page (twosided, 8x11) formula sheet
is allowed.
Solution of midterm
Statistics of midterm:
65 <= A; no. of students (out of 29): 7;
40 <= B <= 59; no. of students: 17;
C <= 39; no. of students: 5;
WEEK 11
Mar 27 (Tue):
Examples of continuous random variables:
uniform, Bertrand's paradox (check it out),
exponential, gamma distributions,
their relation to discrete random variables,
recurrence relation for Gamma function.
Note on examples of
continuous random variables.
Mar 29 (Thur):
memoryless property of geometric and exponential random variables and
their equivalence,
Beta random variables, their applications in Bayesian estimate in coin
tossing with random (unknown) probability of success.
Homework 5, due Thursday, Apr 5, in class
WEEK 12
Apr 3 (Tue):
Normal (Gaussian) random variables, standard normal (Z, N(0,1)),
"error" function,
some calculus formula concerning integration w.r.t.
normal random variables, general Guassian integration,
chi^2distribution with 1degree of freedom (Y=Z^2),
linear change of variable of N(0,1), N(mu, sigma^2)
use of normal table(s),
percentages of A, B, C, D, F in a "normal grade distribution".
Apr 5 (Thur):
[R, Chatper 8] Limit Theorems:
Law of Large Numbers (LLN) (law of the average)
vs Central Limit Theorems (CLT) (fluctuation around the average),
proof of the Weak LLN, Chebychev and Markov inequalities,
graphical representation in terms of binomial r.v.,
statement of CLT and graphical repressentation in terms of binomial r.v.,
compare and contrast LLN and CLT.
WEEK 13
Apr 10 (Tue):
Applications of CLT: approximations of Bin using normal,
estimation of p in Bin, Hypothesis testing.
Note on Applications of
CLT.
Apr 12 (Thur):
Proof of CLT for Binomials
(LaplaceDeMoivre Theorem).
[R 7.7] Moment generating functions (MGF)
(Laplace transform of random variables).
[R 8.3] Proof of CLT Using MGF
Homework 6, due Thursday, Apr 19, in class
WEEK 14
Apr 17 (Tue):
[R 7.7] Moment generating functions (MGF)
(Laplace transform of random variables).
[R 8.3]
Proof of CLT Using MGF
[R, 6.1, 6.2, 6.4, 6.5]
Joint cdfs, pdfs, and
independence, and conditional pdfs
Apr 19 (Thur):
[R, 6.7] change of variable formula of joint pdfs,
Cartesian to polar coordinates,
[R, 6.3] sums of independent random variables,
convolution of pdfs (discrete and continuous).
WEEK 15
Apr 24 (Tue):
[R, 6.3] sums of independent random variables,
convolution of pdfs (discrete and continuous),
relation to moment generating function,
infinite divisibility.
Note on sums of independent
random variables,
[R, Sec 6.5, p. 253] Bivariate normal random variables,
rho=correlation coefficient,
[R, Sec 7.8, p. 345] Multivariate normal random variables,
Note on
bivariate normal random variables.
Note on
multivariate normal random variables.
Homework 7,
Practice Problems, no need to hand in.
Final Exam:
Wed, May 2nd, 8am10am, BRWN 1154
Materials covered: comprehensive, all homeworks and lecture materials.
No electronic devices. One page (twosided, 8x11) of formula sheet
is allowed.
A past final
Office hours: Tuesday, May 1, 24pm
Solution of final