Industrial Data Analytics for Diagnosis and Prognosis. Yong Chen

      The covariance matrix of Z = CX is

(3.2)

      The similarity of (3.2) and (2.10) is pretty clear. When C is a row vector c^T = (c₁, c₂,…, cp), CX = c^TX = c₁X₁ + … + cp Xp and

(3.3)

(3.4)

      where μ and Σ are the mean vector and covariance matrix of X.

      Let X₁ and X₂ denote two subvectors of X, i.e., . The mean vector and the covariance matrix of X can be partitioned as

(3.5)

(3.6)

      where Σ₁₁ = cov(X₁) and Σ₂₂ = cov(X₂). The matrix Σ₁₂ contains the covariance of each component in X₁ and each component in X₂. Based on the symmetry of Σ, we have .

3.2 Density Function and Properties of Multivariate Normal Distribution

      Normal distribution is the most commonly used distribution for continuous random variables. Many statistical models and inference methods are based on the univariate or multivariate normal distribution. One advantage of the normal distribution is its mathematical tractability. More importantly, the normal distribution turns out to be a good approximation to the “true” population distribution for many sample statistics and real-world data due to the central limit theorem, which says that the summation of a large number of independent observations from any population with the same mean and variance approximately follows a normal distribution.

      Recall that a univariate random variable X with mean μ and variance σ² is normally distributed, which is denoted by X ∼ N (μ, σ²), if it has the probability density function

(3.7)

      The multivariate normal distribution is an extension of the univariate normal distribution. If a p-dimensional random vector X follows a multivariate normal distribution with mean vector μ and covariance matrix Σ, the probability density function of X has the form

(3.8)

      We denote the p-dimensional normal distribution by Np(μ, Σ).

      From (3.8), the density of a p-dimensional normal distribution depends on x through the term (x − μ)^T Σ⁻¹ (x − μ), which is the square of the distance from x to Σ standardized by the covariance matrix. Then it is clear that the set of x values yielding a constant height for the density form an ellipsoid. The set of points with the same height for the density is called a contour. The constant probability density contour of a p-dimensional normal distribution is:

which forms the surface of an ellipsoid centered at μ with standardized distance between x and μ equal to c. And the contour with larger distance c has a smaller height value for the density. It can be shown that the axes of the ellipsoid contours of constant density for the p-dimensional normal distribution are in the directions of the eigenvectors of Σ with lengths proportional to the square roots of the corresponding eigenvalues of Σ.

      Example 3.1: Consider a bivariate (p = 2) normally distributed random vector X = (X₁ X₂)T. Suppose the mean vector is μ = (0 0)T and the covariance matrix is

      So the variance of both variables is equal to one and the covariance matrix coincides with the correlation matrix. The inverse of the covariance matrix