### ST3009: Statistical Methods for Computer Science

#### Topics Covered

Part One

• Sum rule of counting
• Product rule of counting
• Number of permutations, with/without repeated objects
• Number of combinations (n choose k)
• Effect of simple constraints on counting e.g. two people must sit together or must not
• Use the above to carry out simple counting calculations
• Definition of sample space and random event
• Set operations, union, intersection, complement and combinations of these
• Axioms of probability
• Plus immediate consequences: P(Ec)=1-P(E), E ⊂ F then P(E)< P(F), equally likely outcomes incl proof.
• Sampling with and without replacement e.g. bag of balls
• Use these to carry out simple calculations of probability of events
• Definition of conditional probability, chain rule
• Use this to carry out simple calculations of conditional prob
• Conditional prob is a probability (satisfies axioms, incl proof.)
• Conditional prob can be used to make predictions
• Marginalisation, incl proof.
• Bayes rule
• Use of Bayes rule in simple examples e.g. HIV testing
• Definition of independence
• Use of definition to test for independence
• Caution re fragility of independence assumptions
• Definition of conditional independence

Part Two

• Definition of a random variable
• Definition of indicator random variable
• Events are linked to values of random variables, so can apply ideas for events directly to RVs (chain rule, Bayes, marginalisation)
• Probability mass function, plotting as histogram
• Cumulative distribution function
• Deriving PMF from CDF and vice versa
• Bernoulli RV
• Binomial RV
• Simple stochastic simulation: generating bernoulli and binomial samples in matlab
• Definition of expected value
• Use this to carry out simple calculations of expected value
• Interpretation of expected value in games of chance/reward
• Expected value of an indicator RV
• Expected value of number of iterations of repeated game (coin tossing etc)
• Limitations of expected value in games of chance/reward e.g. gamblers ruin
• Linearity of expected value incl proof.
• Use of linearity in expected value of sums of random variables
• Expected value of product of independent RVs, incl proof.
• Definition of variance
• var(aX+b)=a2var(X), incl proof.
• Variance of sum of independent RVs, incl proof.
• Definition of joint PMF
• Use this to carry out simple calculations of joint PMF
• Definition of covariance
• Definition of correlation
• Correlation vs causality
• Conditional expectation
• Conditional expectation can be used to make predictions
• Linearity of conditional expectation, incl proof.
• Marginalisation and conditional expectation, incl proof.
• Use of conditional expectation in random sums

Part Three

• Markov inequality
• Chebyshev inequality
• Chernoff inequality
• Special case of Chernoff for Binomial RVs
• Confidence intervals in polls using Chernoff for Binomial RV
• Definition of sample mean, awareness that its an RV
• Expected value and variance of sample mean when sum of iid RVs
• Weak law of large numbers, incl proof via Chebyshev
• Deriving frequency interpretation of probability using the weak law
• Definition of confidence interval
• Use of Chebyshev for confidence interval, and its limitations
• Bootstrapping for confidence intervals, incl matlab implementation
• Definition of continuous RV
• CDF and probability density function for continuous RV
• Probability over interval equal to area under PDF (integral)
• Probability of continuous RV taking a single value is zero
• Calculation of area of rectangles and triangles, plus combinations of these
• Expectation and variance for continuous RVs
• Definition of normal distribution
• Linearity of normal distribution
• Central limit theorem
• Use of CLT to estimate confidence intervals, and its limitations
• Joint CDF for two continuous RVs
• Conditional PDF and chain rule
• Bayes rule for PDFs
• Independence for PDFs

Part Four

• What is a statistical model
• Statistical model used in Logistic Regression
• Linear separability
• Model parameter estimation, Maximum Likelihood and MAP estimates
• Likelihood for Logistic Regression
• Statistical model used in Linear Regression
• Likelihood for Linear Regression
• Fitting polynomials and other nonlinear functions using linear regression
• Closed form Maximum Likelihood solution to scalar Linear Regression
• MAP estimate and Ridge Regression
• Relationship between MAP and Maximum Likelihood estimates as volume of data observed increases

Plus higher order skills including:

• Interpreting simple examples/scenarios and expressing in terms of probability
• Selecting appropriate analysis tools to solve problems
• Self-validation of analysis, e.g. using stochastic simulation to check math analysis and vice-versa
• Awareness of statistical assumptions (including implicit assumptions) and of the potential consequences of their violation e.g. potential fragility of independence assumptions