
Hoeffding’s inequality: set the lower and upper bounds of confidence interval for a bounded random variable. The inequality holds for both large and small sample. Check MIT 18.650 Lecture Note Page 27.

Slutsky’s theorem: replace theoretical mean with empirical mean in deriving the confidence interval. Check MIT 18.650 Lecture Note Page 43.
 Quadratic risk of estimator
 =
 Standard error of OLS:
 The standard error of the coefficient is the square root of the diagonal terms of .
 Note: when calculating the standard error of coefficients, don’t forget the intercept and add constant column to .
 Standard error of MLE (Fisher Information)
 The derivation used delta method and can be found here.
 Relation with entral limits theorem (CLT): when MLE happens to be the average estimator, the asymptotic relationship (may?) becomes the CLT.
 More on Fisher Information
 Fisher information tells the variance of the loglikelihood’s first derivative with respect to :
 Fisher information tells the variance of the loglikelihood’s first derivative with respect to :
 Method of Moment Estimator (MIT 18.650)
 is the covariance matrix of random vector
 Multivariate CLT and Delta Method
 Important tests
 Wald Chisquare test: hypothesis test if the estimated parameters from MLE is the same as a nullhypothesis (used Fisher information).
 Likelihood ratio test: similar to Bayesian hypothesis test by assuming cost function = 1 (for the two types of error) and prior P(H1)/P(H0) = P(H1)/(1 – P(H0)) = c, where c is a variable to control the type I and type II errors.
Likelihood ratio test can be used to test a hypothesis on partial parameters. NeymanPearson theorem tells that the likelihood ratio test is the test that has the smallest type II error when given the constrain (maximal allowed value) on type I error.

Implicit hypotheses test: test .
 Student t test: when sample is from normal distribution , but is unknown.
Student test is useful for small sample (). When is large ( > 10), the distribution of will be very close to a standard normal distribution. Notice that student ttest needs to assume the data is gaussian.
 Two sample test:
 Wald Chisquare test: hypothesis test if the estimated parameters from MLE is the same as a nullhypothesis (used Fisher information).
 Test for goodness of fit: test of hypothesis on distributions
 Pearson Chisquare test: test if a categorical dataset is from a specific discrete distribution.
 Kolmogorov–Smirnov test, AndersonDarling test and Cram ́erVon Mises test : test if a categorical dataset is from a specific continuous distribution.
 Classification criterion: Gini impurity or Information Gain (Entropy variance)
 Gini:
 Entropy:
 For Gini and entropy reach maxima at uniform distribution.
 Regressor criterion: MSE or MAE.
 Time serial tests:
 Durbin–Watson statistic: test autocorrelation
 Durbin–Watson statistic: test autocorrelation
 bb

bb

Recent Posts
Recent Comments
犰狳指挥官 on run firefox on a remote Linux… 犰狳指挥官 on [zz] Homebrew: ruby bad interp… FisherYu on 搬家到这里 cgao on test LaTeX Yunlong on test LaTeX Archives
Categories
Meta