6 MUSIC and the Cramer-Rao lower bound

In this section we show the relation between the MUSIC estimator and the Cramer-Rao bound following the notation and the conditions proposed by Stoica and Nehorai in their paper (Stoica & Nehorai 1990).

6.1 The model

The problem under consideration is to determine the parameters of the following model:

 

$$\vec{y}(t)=A(\vec{\theta })\vec{x}(t)+\vec{e}(t)$$ (12)

where $\left\{ \vec{y}(t)\right\} \in C^{m\times 1}$ are the vectors of the observed data, $\left\{ \vec{x}(t)\right\} \in C^{n\times 1}$ are the unknown vectors and $\vec{e}(t)\in C^{m\times 1}$ is the added noise; the matrix $A(\theta )\in C^{m\times n}$ and the vector $\theta$ are given by  

$$A(\vec{\theta })=\left[ \vec{a}\left( \omega _{1}\right)\! .... ... \qquad \vec{\theta }=\left[ \omega _{1}...\ \omega _{n}\right]$$ (13)

where $\vec{a}\left( \omega \right)$ varies with the applications. Our aim is to estimate the unknown parameters of $\vec{\theta }$. The dimension n of $\vec{x}(t)$ is supposed to be known a priori and the estimate of the parameters of $\vec{x}(t)$ is easy once $\vec{\theta }$ is known.

Now, we reformulate MUSIC to follow the above notation. The MUSIC estimate is given by the position of the n smallest values of the following function:  

$$f\left( \omega \right) =\vec{a}^{\ast }\left( \omega \right)... ... I-\hat{S}\hat{S}^{\ast }\right] \vec{a}\left( \omega \right) .$$ (14)

From Eq. (14) we can define the estimation error of a given parameter. $\left\{ \hat{\omega}_{i}-\omega _{i}\right\}$ has (for big N) an asintotic Gaussian distribution, with mean and with the following covariance matrix:  

$$C_{\rm MU}=\frac{\sigma }{2n}\left( H\circ I \right)^{-1} R... ...A^{\ast }UA\right)^{\rm T} \right\} \left( H\circ I\right)^{-1}$$ (15)

where $Re \left( x\right)$ is the real part of x, where  

$$H=D^{\ast }GG^{\ast }D=D^{\ast }\left[ I-A\left( A^{\ast }A\right) ^{-1}A^{\ast }\right] D$$ (16)

and where U is implicitly defined by:  

$$A^{\ast }UA=P^{-1}+\sigma P^{-1}\left( A^{\ast }A\right) ^{-1}P^{-1}$$ (17)

where P is the covariance matrix of $x\left( t\right)$. The elements of the diagonal of the matrix $C_{\rm MU}$ are the variances of the estimation error. On the other hand, the Cramer-Rao lower bound of the covariance matrix of every estimator of $\vec{\theta }$, for large N, is given by:  

$$C_{\rm CR}=\frac{\sigma }{2n}\left\{ Re \left[ H\circ P^{\rm T}\right] \right\}^{-1}.$$ (18)

Therefore the statistical efficiency can be defined with the condition that P is diagonal as:  

$$\left[ C_{\rm MU}\right] _{ii}\geq \left[ C_{\rm CR}\right] _{ii}$$ (19)

where the equality is reached when m increases if and only if

 

$$\vec{a}^{\ast }\left( \omega \right) \vec{a}\left( \omega \r... ...tarrow \infty \qquad \qquad as \quad m\longrightarrow \infty .$$ (20)

For P non-diagonal, $\left[ C_{\rm MU}\right] _{ii}\gt\left[ C_{\rm CR}\right]_{ii}$.

To adapt the model used in the spectral analysis  

$$\vec{y}(k)=\sum_{i=1}^{p}A_{i}e^{j\omega _{i}k}+\vec{e}(k)\qquad \qquad k=1,2,...,M$$ (21)

where M is the total number of samples, to Eq. (14) we make the following transformations, after fixing an integer m>p:

$$\begin{eqnarray} \vec{y}(t) &=&\left[ y_{t}\quad ...\quad y_{t+m-1}\right] \nonu... ...quad A_{n}e^{j\omega _{n}t}\right] \quad t=1,...,M-m+1 . \nonumber\end{eqnarray}$$ (22)

In this way our model satisfies the conditions of (Stoica & Nehorai 1990). Moreover, Eqs. (22) depend on the choice of m which influences the minimum error variance.

6.2 Comparison between PCA-MUSIC and the Cramer-Rao lower bound

In this subsection we compare the n.e. method with the Cramer-Rao lower bound, by varying the frequencies distance, the parameters M and m and the noise variance.

From the experiments it derives that, fixed M and m, by varying the noise (white Gaussian) variance, the n.e. estimate is more accurate for small values of the noise variance as shown in Figs. 1-3. For $\Delta \omega$small, the noise variance is far from the bound. By increasing m the estimate improves, but there is a sensitivity to the noise (Figs. 4-6). By varying M, there is a sensitivity of the estimator to the number of points and to m (Figs. 7-8). In fact, if we have a quite large number of points we reach the bound as illustrated in Figs. 9-10.

  

Figure 1: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=5, $\sigma=0.5$ and M=50

  

Figure 2: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=5, $\sigma=0.001$ and M=50

  

Figure 3: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=5, $\sigma=0.0001$ and M=50

  

Figure 4: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=20, $\sigma=0.5$ and M=50

  

Figure 5: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=20, $\sigma=0.01$ and M=50

  

Figure 6: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=20, $\sigma=0.0001$ and M=50

  

Figure 7: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=5, $\sigma=0.01$ and M=20

  

Figure 8: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=10, $\sigma=0.01$ and M=20

  

Figure 9: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=20, $\sigma=0.01$ and M=100

  

Figure 10: CRB and standard deviation of n.e. estimates; abscissa is the distance between the frequencies $\omega_{2}$ and $\omega_{1}$. CRB (down); standard deviation of n.e. (up) with m=50, $\sigma=0.001$ and M=100

Therefore, the n.e. estimate depends on both the increase of m and the number of points in the input sequence. Increasing the number of points, we improve the estimate and the error approximates the Cramer-Rao bound. On the other hand, for noise variances very small, the estimate reaches a very good performance. Finally, we see that in all the experiments shown in the figures we reach the bound with a good approximation, and we can conclude that the n.e. method is statistically efficient.