1. Likelihood ratio test

The likelihood ratio test can be applied to a general setting with

Hypotheses: $H_0$: $\theta \in \Theta 0$, $H A: \theta \in \Theta _ A$, where $\Theta _0$ and $\Theta _ A$ are disjoint subsets ($\Theta _0 \cap \Theta _ A =\emptyset$) of a parameter space $\Theta =\Theta _0\cup \Theta _ A$.
Parametric model with parameter $\theta$: The probability (or likelihood) of observing the data $x$ is $p(x;\theta )$.with the parameter \theta.

The likelihood ratio test statistic $\Lambda (x)$ is defined as (negative twice) the logarithm of the likelihood ratio $L(x)$:

$$ \displaystyle \displaystyle \Lambda (x)\displaystyle =\displaystyle -2\log (L(x)) \qquad \text {where } L(x)\, =\, \frac{\max _{\theta \in {\color{blue}{\Theta _0}} }p(x;\theta )}{\max _{\theta \in {\color{blue}{\Theta }} }p(x;\theta )} $$

(Equivalently, in the language of maximum likelihood estimators,

$$ \displaystyle \displaystyle L(x)\displaystyle =\displaystyle \frac{ p\left(x;\hat{\theta }{\text {MLE}}^{\text {constrained}}\right)}{p\left(x;\hat{\theta }{\text {MLE}}\right)} $$

where $\hat{\theta }{\text {MLE}}$ is the maximum likelihood estimator of $\theta$ and $\hat{\theta }{\text {MLE}}^{\text {constrained}}$ is the constrained maximum likelihood estimator of $\theta$ within $\Theta _0$.)

Use the definition of likelihood ratio L(x) and the likehood ratio test statistic $\Lambda (x)$ above.

We expect the null hypothesis to not be true if

If the likelihood ratio L(x) is small, then there is a parameter \theta in \Theta _ A such that p(x;\theta ) is big, i.e. that makes seeing the data much more likely. Hence, we should reject H_0 in this case.

2. The Distribution of Likelihood Ratio test statistics

Wilk's Theorem states that when the sample size is large, the distribution of \Lambda under H_{0} approaches a \chi ^2distribution:

	\displaystyle \displaystyle \Lambda \overset {n\to \infty }{\longrightarrow } \chi _ d^2 \qquad \text {where } d = \text {dim}(\Theta ) - \text {dim}(\Theta _0)

where d is the degree of freedom of the \chi ^2 distribution.

Power of Likelihood Ratio Test

The Neyman–Pearson lemma states that among all tests that test for the simple hypotheses H_0: \theta =\theta 0\, ;\, H A:\theta =\theta _ A at significance level \alpha, the likelihood ratio test is the most powerful. That is, among all tests testing the same simple hypotheses and at the same significance level, the likelihood ratio test gives the largest probability of rejecting the null when indeed the alternate is true.

The likelihood ratio (LR) test statistic $\Lambda(x)$ is used to test hypotheses about the parameter $\theta$. The LR-test statistic is given by: