At the end of the is lecture, you should be able to
A time series is a special kind of statistical data, specifically, it is a collection of numerical measurements, called observations $x_1, \ldots , x_ t, \ldots , x_ n \in \mathbb {R}$ that are indexed by a time stamp $t=1,\ldots ,n$. These time stamps must form a deterministic sequence and be regularly spaced in time with equal intervals between any two adjacent stamps. For the purposes of statistical inferences with these data, the observations are modeled mathematically as realizations of a corresponding series (i.e., sequence) of random variables $X_1,\ldots , X_ t,\dots , X_ n:\Omega \to \mathbb {R}$ that are defined on some common probability space $(\Omega ,\mathbf{P})$. As with other types of statistical data, what is observed by the statistician is a particular outcome of a specialized probability model: $x_1 = X_1(\omega ), \dots , x_ t = X_ t(\omega ), \dots , x_ n = X_ n(\omega ),$ for an outcome $\omega \in \Omega$ in some probability space $(\Omega ,\mathbf{P})$.
The probability model is such that we have one random variable for each time stamp and one observation for each random variable. All these random variables are defined on a common probability space, so we can speak of probabilities of joint events that involve any number of these random variables. The realizations come from the real world where they occur sequentially in time in the order of the time stamp index $t$, so that $x_ t$ is observed before $x_{t+1}$. Also, the observations arrive at a fixed time interval, so that the time that elapses between observing $x_ t$ and $x_{t+1}$ is the same for all $t$.
The most important feature of time series data is that we make no assumption about independence of these random variables. Recall that independence was our fundamental assumption in the context of cross-section data that are obtained by random sampling from a fixed population, which we studied in e.g. 18.6501x. In fact, most time series data are dependent , typically because past realizations influence future observations through the nature of the real world phenomenon that produces these data. It is fair to say, that the main goal of time-series analysis is to first model and then estimate from data (guided by the model) the dependence structure of these random variables.
Statistical dependence in a time series is a double-edged sword. On the one hand, dependence helps us make predictions about the future realizations from knowledge of the past realizations. E.g., if yesterday was warm, today will probably be warm as well. On the other hand, dependence poses technical challenges in the distributional analysis of estimators. This is because there is effectively less statistical information in dependent data about the data generating process, as compared to the case of independent observations. E.g., the basic laws of large numbers and central limit theorems do not even apply!
To gain some intuition for our discussion of time series, consider the following examples:
In all these examples, as well as in a general time series, data take the form of discrete measurements of a real world phenomena that evolves continuously in time. A general probabilistic model to describe such phenomena is called a stochastic process , which is simply a collection of random variables $\{X_ t\}$ indexed by either a continuous or discrete time parameter $t$. So, a time series data set can be thought of as a single realization of a stochastic process. Each random variable $X_ t$ has a marginal distribution $\mathrm{P}{t}$. The process $\{ X t\} {t>0}$ as a whole also has a probability law, which can be thought of as the joint distribution of all the $X t$'s. (Interestingly, this joint distribution of the entire process, whether continuous or discrete, is completely characterized by the collection of all the finite-dimensional marginal distributions $\mathrm{P}{t_1,..,t k}$ of random vectors $(X_{t_1},\dots ,X_{t_ k})$ where $k$ ranges over all integers and time indexes $t_1,\dots ,t_ k$ range over all distinct time stamps.)