24 CHAPTER 2 Information information can be interpreted in terms of entropy and Kullback-Leibler divergence [3], and is closely related to the expected log likelihood [1]. The downsize is that this criterion requires the knowledge of the joint density of the components of -1 {x(·)}, which is unknown in practice and hence must be replaced by some nonparametric estimate. The estimation procedure can be quite costly computationally. We shall however introduce some methods which are not much costlier than using simpler criteria (such as the cumulants). The rest of this chapter contains two parts: the first one concerns the use of the mutual information between the observations at a given time, which is suitable for instantaneous (linear) mixtures, in which the temporal dependence of the source sequences are ignored (for simplicity or because it is weak). The second part concerns the use of the information rate between stationary processes, which is necessary to treat the case of convolutive (linear) mixtures, but can also be useful for the case of instantaneous mixtures when there is strong temporal dependence of the source sequences. Note that for the convolutive mixture, the sources can be recovered only up to filtering (as it will be seen later) and one may require temporal independence of the source to lift this ambiguity. The problem then may be viewed as the multi-channel blind deconvolution problem as it reduces to the well-known deconvolution problem when both source and observed sequences are scalar. We however call this problem blind separation-deconvolution as it aims to both recover the sources (separation) and make them temporally independence (deconvolution). 2.2 METHODS BASED ON MUTUAL INFORMATION We first define mutual information and provide its main properties. 2.2.1 Mutual information between random vectors Let y 1 , . . . , y P , be P random vectors with joint densities p y 1 ,...,y P and marginal density p y 1 , . . . , p y P . The mutual information between these vectors is defined as the Kullback- Leibler divergence (or relative entropy) between the densities I {y 1 . . . , y P } = - log p y 1 (y 1 ) · · · p y P (y P ) p y 1 ,...,y P (y 1 , . . . , y P ) P k=1 p y k and p y 1 ,...,y P : . This measure is non-negative (but can be +) and can vanish only if the random vectors are mutually independent [4]. It can also be written as: P I {y 1 . . . , y P } = i=1 H {y k } - H {y 1 . . . , y P } (2.1) where H (y 1 . . . , y P ) and H (y 1 ), . . . , H (y P ) are the joint entropy and the marginal entropies of y 1 , . . . , y P : H {y 1 . . . , y P } = - log p y 1 ,...,y P (y 1 . . . , y P ) (2.2)