MA/STAT 519: Introduction to Probability Theory
Spring 2018, Purdue University

http://www.math.purdue.edu/~yip/519

Course Description:

Algebra of sets, sample spaces, combinatorial problems, conditional probability and independence, discrete and continuous random variables, distribution and joint distribution functions, expectation of random variables, moment generating functions, limit theorems and applications.

Instructor:

Aaron Nung Kwan Yip
Department of Mathematics
Purdue University

Contact Information:

Office: MATH 432
Email: click here

Lecture Time and Place:

T, Th 10:30 - 11:45, UNIV 117

Office Hours:

Tue, 4:30-5:30pm, Wed, 3:00-4:00pm, or by appointment.

Textbook:

(All of the following are either available online or on reserve in the math library.)

Main Text:
[CD] Probability with Applications in Engineering, Science, and Technology,
by Matthew A. Carlton, Jay L. Devore, 2nd edition (available online from Purdue library page)

Reference:
[R] A First Course in Probability, by Sheldon Ross (I am using the 9th edition, but older editions are OK for reading and reference)
[HPS] Introduction to Probability Theory, by Hoel, Port and Stone
[D1] Fundamentals of Probability: a First Course, by Anirban DasGupta (available online from Purdue library page)
[D2] Probability for Statistics and Machine Learning: Fundamentals and Advanced Topics, by Anirban DasGupta (available online from Purdue library page)
[F1, F2] An Introduction to Probability Theory and Its Applications, Volume 1 and 2, by William Feller

Course Outline:

The course will cover most of the sections of [CD] Chapters 1 to 4, and part of Chapter 5.

Prerequisites:

Good "working" knowledge of:
linear algebra (e.g. MA 265, 351, 511),
vector calculus (e.g. MA 261, 362, 510),
and mathematical analysis (e.g. MA 341, 440, 504).

Homework:

Homeworks will be assigned roughly bi-weekly. They will be gradually assigned as the course progresses. Please refer to the course announcement below.

  • Steps must be shown to explain your answers. No credit will be given for just writing down the answers, even if it is correct.

  • Please staple all loose sheets of your homework to prevent 5% penalty.

  • Please resolve any error in the grading (hws and tests) WINTHIN ONE WEEK after the return of each homework and exam.

  • No late homeworks are accepted (in principle).

  • You are encouraged to discuss the homework problems with your classmates but all your handed-in homeworks must be your own work.
  • Examinations:

    Midterm Test: Date to be determined
    Final Exam: During Final Exam Week

    Grading Policy:

    Homeworks (50%)
    Midterm Test (20%)
    Final Exam (30%)

    You are expected to observe academic honesty to the highest standard. Any form of cheating will automatically lead to an F grade, plus any other disciplinary action, deemed appropriate.

    Accommodations for Students with Disabilities and Academic Adjustment:

    University policy will be followed.

    Course Progress and Announcement:

    (You should consult this section regularly, for homework assignments, additional materials and announcements.)


    WEEK 1

    Jan 9 (Tue): [CD 1.1-1.2][R 2.1-2.4][D1, Chapter 1]
    Concept of statistical regularity, Law of Large Numbers (LLN), Central Limit Theorem (CLT),
    Probability space, outcome, events, logical operations between events, complement, unions, intersections, de Morgan's Law.

    Jan 11 (Thur): [CD 1.1-1.2][R 2.1-2.4][D1, Chapter 1]
    Probability measure, countably-additivity,
    inclusion-exclusion principle [R p. 30, Proposition 4.4][D1, 1.5],
    objective vs subjective approaches to probability measures.


    WEEK 2

    Jan 16 (Tue):
    Some theoretical remarks about probability measures: countably-addivitiy, (non-)measureable sets, typical outcomes,
    equally-likely-outcome probability space;
    [CD 1.3][R 1.1-1.4][D1 1.4] counting techniques: permutations and combinations.

    Jan 18 (Thur) [R 1.1-1.6]:
    binomial and multinomial coefficients and expansions, number of integer solutions.

    Homework 1, due Thursday, Jan 25, in class


    WEEK 3

    Jan 23 (Tue): [D1 6.5, Chapter 2][R 2.5][CD 1.3.3, 2.6.1]
    Sampling (partition, hypergeometric distribition), birthday and matching problems,
    asymptotics of formula when some number (e.g. population size) is large.

    Jan 25 (Thur):
    [D1 Chapter 15] Urn and occupancy problems, Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac Distributions;
    [R 3.1-3.3][D1 Chapter 3][CD, 1.4]
    Conditional probability, Bayes' forward (formula of total probability) and backward (updating prior) formula,
    prior and posterior probabilities.


    WEEK 4

    Jan 30 (Tue):
    (cont.) Bayes' backward and forward formula, prior and posterior probabilities, sensitivity and specificity of medical tests.
    [R, 3.1-3.4][D1 Chapter 3][CD 1.5] independence between two, three, and multiple events.

    Feb 1 (Thur):
    [R, 3.1-3.4][D1 Chapter 3][CD 1.5] (cont.) (jointly) indepedence between multiple events.
    "Probability paradoxes": Monty Hall

    Homework 2, due Thursday, Feb 8, in class


    WEEK 5

    Feb 6 (Tue):
    "Probability paradoxes": (Mr. Smith's) Boy vs Girl.
    computation of probability by iteration/recursion/induction,
    an example [R, e.g. 5d, p. 93]: matching problem.

    Feb 8 (Thur):
    iterations/recursion/induction, Gambler's Ruin Problem [R, p. 84],
    trials with random (unknown) parameters, conditional independence.


    WEEK 6

    Feb 13 (Tue):
    [R 4.1-4.6][D1 Chapter 4][DC Chapter 2.1-2.4]
    (Discrete) Random variables (numerical observations of experimental outcomes),
    probability mass (density) function (pmf, pdf) and cumulative density function (cdf), expectation, variance, Cauchy-Schwarz Inequality,
    Bernoulli, Binomial, and Poisson random variables.
    Note on random variables and expectation.

    Feb 15 (Thur):
    Analytical computation of expectations and variance of Binomial and Poisson random variables
    [R Chapter 4/6.2-6.3/7.2-7.3] Joint (bivariate and multivariate) random variables, joint pdf, marginal pdfs, independent random variables,
    computations of Ef(X,Y), E(X+Y), E(XY)

    Homework 3, due Thursday, Feb 22, in class


    WEEK 10

    Midterm, Mar 22, in class (review on Mar 20)