Solutions Manual to accompany Wavelets and Subband Coding by Martin Vetterli & Jelena Kovačević
Grace Chang, Michael Goodwin, Vivek K Goyal & Ton Kalker
Originally published 1995 by Prentice Hall PTR, Englewood Cliffs, New Jersey. Reissued by the authors 2007.
This work is licensed under the Creative Commons Attribution-NoncommercialNo Derivative Works 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, CA 94105 USA.
Solution Manual for
Wavelets and Subband Coding by
Martin Vetterli and Jelena Kovaˇcevi´c
Solutions Prepared By: Grace Chang Michael Goodwin Vivek Goyal Ton Kalker Please send questions or comments about the solutions to
[email protected]
1
Chapter 2 2.1 Legendre polynomials
We assume the inner product definition preceding (2.2.1). Let xk = tk−1 for k ∈ Z+. We initiate the Gram-Schmidt procedure with y1 =
1 x1 1 √ . = 1 1 = x1 2 ( −1 1 dt) 2
Continuing the Gram-Schmidt orthogonalization as on page 22, 1 1 1 √ t dt √ = 0 v2 = y1 , x2 y1 = 2 2 −1 t x2 − v2 3 = 1 t y2 = 1 = x2 − v2 2 ( −1 t dt) 2 1 1 1 1 1 3 3 √ t · t dt √ + t · t2 dt = v3 = y1 , x3 y1 + y2 , x3 y2 = 2 2 3 2 2 −1 −1 √ t2 − 13 x3 − v3 3 5 2 1 √ t = 1 = − y3 = 1 x3 − v3 3 2 2 ( −1 (t2 − 13 )2 dt) 2 This process can be continued to find an orthonormal set of arbitrary size. It can be proven by induction that 2n + 1 (−1)n dn · n · n (1 − t2 )n . yn+1 = 2 2 n! dt
2.2 Proof of Theorem 2.4
Let S = {x1 , x2 , . . . xK } be an orthonormal system in E = RN or C N . (For S to be an orthonormal basis will require K = N , but there is no reason to assume this a priori.) The proof will proceed as (a) ⇒ (e) ⇒ (d) ⇒ (b) ⇒ (a). (a) ⇒ (e): See Problem 2.8(a). (e) ⇒ (d): (d) is a special case of (e) with y1 = y2 = y. (d) ⇒ (b): Let y ∈ E. Suppose xi , y = 0 for i = 1, 2, . . . , K. Then by (d), y2 = 0. Thus Definition 2.2(d) implies y = 0.
ˆ. (b) ⇒ (a): Let y ∈ E. We wish to show that yˆ = K i=1 xi , yxi equals y. Define e by e = y − y Then for k ∈ {1, 2, . . . , K},
K xk , y − xi , yxi xk , e = i=1
= xk , y − = xk , y −
K i=1 K
xk , xi , yxi xi , yxk , xi
i=1
2
= xk , y −
K
xi , yδki by the orthonormality of S
i=1
= xk , y − xk , y = 0. Now (b) implies that e = 0. Thus S satisfies the requirements for an orthonormal basis for E.
2.3 Orthogonal transforms and l∞ norm (a) The orthogonal transforms T2 correspond to rotations. We can thus find the bounds a2 and b2 by considering rotations of vectors on the unit circle. By the definition of the l∞ norm, the upper bound b2 is clearly 1 since there is no vector on the unit circle whose maximum element is greater than 1. The lower bound is achieved when both components of the vector are equal; this is true because any rotation of such a vector increases one of the components, which increases the l∞ norm. So the result is: 1 √ ≤ l∞ [T2 v] ≤ 1 2 (b) The arguments from (a) can be easily extended to n dimensions. For vectors on an n-dimensional sphere, the minimum l∞ norm occurs when all of the elements are equal: for v = [v1 v2 . . . vn ]T with v2 = 1 =⇒ v12 + v22 + . . . + vn2 = 1 vi = vj ∀ i, j
1 =⇒ nv12 = 1 =⇒ v1 = √ = an n
The maximum l∞ norm occurs when all but one of the elements are zero, so bn = 1. Combining the results gives: 1 √ ≤ l∞ [Tn v] ≤ 1 n
2.4 Norms of operators Recall that the definition of the operator norm is given in Appendix 2.A. Also recall that the norm for l2 (Z) is given on page 19. (a) Let x ∈ l2 (Z) be a unit norm sequence. |m[n] · x[n]|2 = |m[n]|2 |x[n]|2 = |x[n]|2 = x2 = 1, Ax2 = n
n
n
where the third equality is because |m[n]| = 1 for all n. Thus A = 1. (b) Let x ∈ l2 (Z) and let y = Ax. Notice that y[2n] 1 1 x[2n] = . y[2n + 1] 1 −1 x[2n + 1] Thus we can write |y[2n]|2 + |y[2n + 1]|2 = |x[2n] + x[2n + 1]|2 + |x[2n] − x[2n + 1]|2 = 2|x[2n]|2 + 2|x[2n + 1]|2 . √ From this it is clear that y2 = 2x2 . Therefore A = 2. 3
2.5 Least-squares approximation in an orthonormal representation
N i=1 αi xi
Our problem is to minimize y − yˆ, where y = y − yˆ =
K N xm , (αi − βi )xi + αi xi i=1
Subtracting gives
i=K+1
xm , y − yˆ =
K i=1 βi xi .
K N (αi − βi )xi + αi xi . i=1
Thus,
and yˆ =
=
i=K+1
m = 1, . . . , K αm − βm , m = K + 1, . . . , N αm
where we have used the linearity of the inner product and the orthonormality of {xm }N m=1 . Now Parseval’s equality (2.2.5) gives y − yˆ2 =
N
K
|xm , y − yˆ|2 =
m=1
|αm − βm |2 +
m=1
N
|αm |2 .
m=K+1
This is minimized by choosing βm = αm for m = 1, . . . , K, which is the desired result.
2.6 Least-squares solution
Let A ∈ Rm×n with m > n. Consider the problem of “solving” y = Ax in the sense of finding x ˆ to minimize y − Aˆ x. In Section 2.3.2, geometric reasons are given to justify choosing −1 T A y. (1) x ˆ = AT A We wish give a calculus justification. As preparation, we manipulate (1) in the opposite direction than in the text to yield x) = 0. AT (y − Aˆ
(2)
x) = 0 for i = 1, 2, . . . , n, aTi (y − Aˆ
(3)
Considering (2) elementwise gives
where ai is the ith column of A. x)T (y − Aˆ x). Now let E = y − Aˆ x2 = (y − Aˆ ∂E ∂ xˆi
= aTi (y − Aˆ x) + (y − Aˆ x)T ai = 2aTi (y − Aˆ x)
By (3), if x ˆ is chosen according to (1), then each partial derivative ∂E/∂ xˆi is zero. Therefore (1) gives a local extremum of E. Since E is a quadratic function with positive leading term, this is guaranteed to be a global minimum.
2.7 Least-squares solution to a linear system of equations
(a) If y belongs to the range (or column space) of A, then there exists x such that Ax = y. Hence, y ˆ = A(AT A);1 AT y
= A(AT A);1 (AT A)x
= Ax = y. 4
(b) If y is orthogonal to the column space of A, then y ∈ N (AT ), the nullspace of AT . That is, ˆ = A(AT A);1 AT y =0. AT y =0, and so y
2.8 Parseval’s formulae
(a) Let {xi } be an orthonormal basis for E. Let y ∈ E. Then y =
y2 = y, y = αk xk , α x =
α∗k
k
=
xk ,
α∗k
k
=
k
αk xk , where αk = xk , y.
α x
k
=
k
α xk , x
α∗k α δk by the orthonormality of the xi ’s
α∗k αk
k
=
|xk , y|2
k
This establishes (2.2.5).
Let y1 , y2 ∈ E. Then y1 = k αk xk , where αk = xk , y1 , and y2 = k βk xk , where βk = xk , y2 .
αk xk , β x y1 , y2 = k
=
α∗k
k
=
xk ,
α∗k
k
=
β x
k
=
β xk , x
α∗k β δk by the orthonormality of the xi ’s
α∗k βk
k
= xk , y1 ∗ xk , y2 k
This establishes (2.2.6). ˜i } constitute a (b) The manipulations are very similar to the orthogonal basis case. Let {x
i, x , y ∈ E. Then we can write y = pair of biorthonormal bases for E. Let y 1 2 1 k xk , y1 x˜k and
y2 = k x˜k , y1 xk .
xk , y1 x˜k , x˜ , y2 x y1 , y2 = k
5
= xk , y1 ∗ k
= xk , y1 ∗
x˜k ,
=
k
x˜ , y2 x
x˜ , y2 x˜k , x
k
∗
xk , y1 x˜ , y2 δk by (2.2.?)
xk , y1 ∗ x˜k , y2
=
k
This establishes (2.2.12). By expanding y1 in terms of xk ’s and y2 in terms x˜k ’s, (2.2.13) can be similarly proven. (2.2.11) is a special case of (2.2.12) with y1 = y2 = y.
2.9 Symmetric and antisymmetric functions (a) Let 1 fs (x) = (f (x) + f (−x)) 2 and
1 (f (x) − f (−x)) , 2 then it is straightforward to verify that fs (x) ∈ S, fa (x) ∈ A, and that any f (x) ∈ L2 ([−π, π]) can be written as f (x) = fs (x) + fa (x) . fa (x) =
(b) An orthonormal basis for S is
1 cos nx √ , √ 2π 2π
, n = 1, 2, . . .
(which are symmetric functions). Similarly, sin nx √ , n = 1, 2, . . . 2π are asymmetric functions, and they form an orthonormal basis for A. (c) From part (a), since any f ∈ L2 ([−π, π]) can be written as f (x) = fs (x) + fa (x), this implies that L2 ([−π, π]) ⊆ S ⊕ A. Conversely, from the decomposition in part (b) and the fact that these functions form an orthonormal basis for L2 ([−π, π]), it follows that the sum of any functions fs ∈ S and fa ∈ A is equal to some f ∈ L2 ([−π, π]), implying L2 ([−π, π]) ⊇ S ⊕ A. Hence, L2 ([−π, π]) = S ⊕ A.
2.10 Downsampling by N Consider x[n] and y[n] to be obtained from sampling xc (t) with sampling periods T and N T , respectively. ω 1 2π ω jω = −k by (2.4.35) Xc Y (e ) = XN T NT NT NT NT k
6
=
=
=
N −1 ∞ 2π 1 ω − (nN + ) by defining n and through k = nN + Xc NT NT NT =0 n=−∞ N −1 ∞ 2π ω − 2π 1 1 −n Xc N T n=−∞ NT T 1 N
=0 N −1
X(ej(ω−2π)/N ) by (2.4.36)
=0
2.11 Downsampling and aliasing The problem statement is incorrect. Suppose x[n] is downsampled by a factor of N . The appearance of aliasing is equivalent to the existence of (open intervals of) ω such that more than one term of (2.5.13) is non-zero. The easiest way to check for aliasing is to sketch the stretched and shifted versions of X(ejω ) and see if they have any overlap. (a) H (ejω /2) and H (ej(ω−π) /2) are identical (and thus overlap). Therefore filtering by H (ejω ) prior to downsampling by 2 will not avoid aliasing. (b) & (c) Similarly incorrect.
2.12 Basis patterns
(a) Let’s consider the problem in matrix notation. Let x = [ x[0], x[1], . . . , x[N − 1] ]T and φk = [ φk [0], φk [1], . . . , φk [N − 1] ]T , k = 0, 1, . . . , N − 1. Since x ∈ R N , it can be written as a linear combination of the φk ’s only if {φk , k = 0, 1, . . . , N − 1} form a basis of RN . Let Φ be the matrix whose columns are {φk , k = 0, 1, . . . , N − 1}, i.e. Φ = [φ0 φ1 . . . φN ;1 ]. Then Φ is a circulant matrix and it can be diagonalized by the Fourier matrix as Φ = F ;1 ΛF , where Λ = diag{P [0], P [1], . . . , P [N − 1]}, and P [k], k = 0, 1, . . . , N − 1, are the DFT coefficients of p[n]. Thus, Φ is full rank (implying that the vectors {φk , k = 0, 1, . . . , N − 1} form a basis) if and only if P [k] = 0, k = 0, 1, . . . , N − 1. (b) Assuming that {φk , k = 0, 1, . . . , N − 1} form a basis of R N , we wish to expand x as x = Φα , where α = [α0 , α1 , . . . , αN −1 ]T . Since Φ is full rank, it is invertible. Thus, α = Φ;1 x = F ;1 Λ;1 F x .
2.13 Periodically time-varying systems A linear, time-varying system is characterized by a two-variable impulse response
h[k, n], which is the response to input x[n] =
δ[n − k]. Since the input can be written as x[n] = k x[k]δ[n − k], the output is given by y[n] = k x[k]h[k, n]. We are interested in the case where h[k, n] is periodic in k with period N . As on page 71, define the polyphase components by xi [n] = x[nN + i] for i = 0, 1, . . . , N − 1. Denote the upsampled version of xi by x˜i , so xi [n/N ], n/N ∈ Z x[n + i], n/N ∈ Z = x˜i [n] = 0, otherwise 0, otherwise 7
The input x[n] can be decomposed to give x[n] =
N −1
x˜i [n − i].
i=0
By superposition, the output is given by y[n] =
=
=
=
h[k, n]
k N −1
N −1
x˜i [n − i]
i=0
h[k, n]x˜i [n − i]
i=0 k N −1 i=0
k: k−i ∈Z N
N −1
h[k, n]x˜i [n − i] because the excluded terms are zero
h[i, n]x˜i [n − i] by the periodicity of h
i=0 k: k−i ∈Z N
=
N −1 i=0
h[i, n]x˜i [n − i] because the added terms are zero.
k
The final expression shows the output as a sum of N terms. The ith term is the ith polyphase component, upsampled and filtered by an i-sample delayed version of h[i, n].
2.14 Interpolation of oversampled signals (a) Since the bandlimited signal is oversampled, there are many possible interpolation filters, as 2 long as they select only the base copy of the spectrum. Let the sampling period T = 2π ωs = 3 . H2 (ω) must satisfy T if ω ∈ [−π, π] H2 (ω) = 0 if ω ≥ 2π The most straightforward H2 (ω) is obtained from convolving in the frequency domain two rectangular windows:
G1(ω )
G2(ω )
T
−3π/2
H2 (ω ) T
1 3π/2
ω
*
−π/2 π/2
ω =
−2π −π
π 2π
ω
Let ω1 and ω2 be the cutoff frequencies of G1 (ω) and G2 (ω), respectively. They can be found by solving the equations ω1 + ω2 = 2π , ω1 − ω2 = π . 8
π This yields ω1 = 3π 2 and ω2 = 2 . In the time-domain, h2 (t) is the product of two sinc functions (up to a scaling constant), which is clearly a function of t12 : 3π π sin( 2T t) sin( 2T t) . h2 (t) = π 3π 2t 2T t
(b) For ωs = 4π, T =
2π ωs ,
H3 (ω) must satisfy H3 (ω) =
if ω ∈ [−π, π] if ω ≥ 3π
T 0
H3 (ω) can be obtained by convolving (in the frequency domain) one “big” rectangular window, G1 (ω), with two identical “small” rectangular windows, G2 (ω):
G1(ω ) T
−2π
2π
ω *
G2(ω )
G2(ω )
1
1
−π/2 π/2
H 3 (ω ) T
ω ω * −π/2 π/2
= −3π
−π
π
3π
ω
Let ω1 and ω2 be the cutoff frequencies of G1 (ω) and G2 (ω), respectively. They can be found by solving the equations ω1 + 2ω2 = 3π , ω1 − 2ω2 = π . This yields ω1 = 2π and ω2 = π2 . In the time domain, h3 (t) is the product of 3 sinc functions, so it decays as sin( 2π t) h3 (t) = 2πT T t
π sin( 2T t) π 2t
1 t3 :
2
To show that H3 (ω) has a continuous derivative, it is easier to consider it in the time-domain and the transform it back to the frequency domain. Recall that the Fourier transform pair of the frequency domain derivative is dn H(ω) FT . tn h(t) ←→ j n dω n The scaling constant j does not affect the continuity of the problem so we will ignore it. t h3 (t) =
sin( 2π T t) 2π T
π t) sin( 2T π 2t
2
Hence, t h3 (t) is a product of 2 identical sinc functions and a sine function (and a multiplicative constant), which corresponds to convolving a triangle window with a pair of impulses in the frequency domain. This is just adding together two triangle windows (which are continuous function), hence the derivative is continuous. Note that the second derivative is not continuous, since t2 h3 (t) is a 9
product of a sinc function and 2 sine functions, which translates into adding together 4 rectangular windows in the frequency domain, and rectangular windows are not continuous. (c) Generalizing to ωs = (i + 1)π, Hi (ω) must satisfy T if ω ∈ [−π, π] Hi (ω) = 0 if ω ≥ iπ Hi (ω) is obtained from convolving a “big” window with cutoff ω1 with (i − 1) “small” windows with cutoff ω2 , where ω1 and ω2 can be solved from the equations ω1 + (i − 1)ω2 = iπ , ω1 − (i − 1)ω2 = π . This yields ω1 = iπ 2 and ω2 = thus is a function of t1i :
π 2.
The time-domain sequence is a product of i sinc functions and
hi (t) =
iπ sin( 2T t)
iπ 2T t
π sin( 2T t) π 2t
(i−1) .
Now consider the n-th derivative of Hi (ω), where 0 ≤ n ≤ (i− 2). In the time domain, this becomes tn hi (t) =
iπ π sin( 2T t) (sin( 2T t))(i−1) iπ π (i−n) 2T 2 t
,
which is a product of i−n sinc functions and n sine functions. In the frequency domain, this becomes a convolution of i−n rectangular functions with n pairs of impulses (from the sine function). Noting that convolving 2 or more rectangle windows yield continuous functions, we conclude that the n-th derivative of Hi (ω) is continuous for 0 ≤ n ≤ (i − 2).
2.15 Uncertainty relation (a) By definition,
∆2t =
+∞ −∞
t2 |f (t)|2 dt
and ∆2ω =
+∞ −∞
ω 2 |F (ω)|2 dω
√ where f (t) and F (ω) are a Fourier transform pair. Scaling by a yields the function fs (t) = af (at) 1 ω . and its Fourier transform Fs (ω) = √ F a a This scaling preserves the L2 norm: +∞ +∞ +∞ √ 2 af (at)2 dt = f (t ) dt = L2 (f ) a |f (at)|2 dt = L2 (fs ) = −∞
−∞
−∞
For the scaled function, the uncertainty product is +∞ +∞ 2 ω 2 √ 2 2 dt dω t2 af (at) ω 2 F ∆ω ∆t = a −∞ −∞ Substituting t = at and ω = 2 2 ∆ω ∆t =
ω a
yields: +∞ 2 2 1 +∞ 1 dt a dω 2 (t )2 f t a2 (ω )2 F ω a −∞ a −∞ +∞ +∞ 2 2 = dt (t )2 f (t ) dω (ω )2 F ω −∞
−∞
10
which completes the proof. Note that the operations of time-shifting and modulation do change the uncertainty product ∆2ω ∆2t . 1 − 12 ≤ t ≤ 12 is given by (b) The Fourier transform of the rectangular pulse p(t) = 0 otherwise
+∞
P (ω) =
−jωt
p(t) e
dt =
−∞
1 1 −jωt + 2 e − 1 jω −
=
=
2
+ 12 − 12
e−jωt dt
ω 2 sin ω 2
Then, ∆2t
=
+ 12 − 12
+ 1 t3 2 1 t dt = = 3 −1 12 2
and
2
∆2ω
+∞
= 4
sin2
−∞
ω 2
dω −→ ∞
so the uncertainty product ∆2ω ∆2t is unbounded for a rectangular pulse. (c) Consider the triangular pulse given by q(t) = p(t) ∗ p(t) where p(t) is the rectangular pulse of part (b): 3 2 (1 − |t|) −1 ≤ t ≤ 1 q(t) = 0 otherwise where the scale factor 32 is included to make q(t) a unit-energy function. (The uncertainty relation only holds for unit-energy functions.) Then, 1 1 3 2 2 2 t (1 − |t|) dt = 3 t2 (1 − t)2 dt ∆t = 2 −1 0 1 3 t4 t5 1 t − + = = 3 3 2 5 0 10 3 4 2 ω , so sin The Fourier transform of q(t) is given by Q(ω) = 2 ω2 2 ∞ 16 2 2 3 2 ω dω ω sin ∆ω = 4 2 −∞ ∞ ∞ 2 ω 1 sin4 x 4 ω dω = 24 sin dx = 48 2 2 x2 0 0 ∞ ω 2 π π sin x sin2 (2x) − = 6π − dx = 24 = 24 x2 4x2 2 4 0 π 1 2 2 (6π) = 0.6π ≥ So the uncertainty relation holds: ∆t ∆ω = 10 2 (d) The central limit theorem can be applied to the sequence of convolved pulses: fn (t) = p1 (t) ∗ p2 (t) ∗ · · · ∗ pn (t) 1 2 2 √ e−(t−η) /2σ lim fn (t) = n→∞ σ 2π 11
where the parameters η and σ depend on the functions pi (t). Since equality holds in the uncertainty relation for a Gaussian, the time-bandwidth product of a convolved sequence of pulses will approach π/2.
2.16 Allpass filters (a) For H(z) to have real coefficients, the zeros as well as the poles must appear in complexconjugate pairs, so H(z) is of the form: bi + z −1 a∗ + z −1 ai + z −1 i H(z) = 1 + bi z −1 1 + ai z −1 1 + a∗i z −1 i
i
where the ai are complex and the bi are real. The zeros are at locations {−1/bi , −1/ai , −1/a∗i } while the poles are at {−bi , −a∗i , −ai }. The poles and zeros are at mirror locations with respect to the unit circle. We can rewrite H(z) as a∗i + z −1 ai + z −1 bi + z −1 Q(z −1 ) =⇒ H(z) = H(z) = z −1 (bi + z) z −2 (ai + z) (a∗i + z) z −N Q(z) i
i
where Q(z −1 ) is the N -th order numerator polynomial Q(z −1 ) = α0 + α1 z −1 + . . . + αN z −N , so the allpass filter H(z) can be written as H(z) =
α0 + α1 z −1 + . . . + αN z −N αN + αN −1 z −1 + . . . + α0 z −N
which shows that the numerator and denominator are mirror polynomials. h[k]∗ h[k − n] = h[−n] ∗ h[n]∗ . Taking the (b) The autocorrelation a[k] is given by a[k] = k
DTFT yields A(ejω ) = H(e−jω )H(e−jω )∗ = 1 by definition. The time-domain autocorrelation is given by the inverse transform of A(ejω ), so π 1 1 sin(πn) = δ[n] ejωn dω = a[n] = 2π −π πn We know that {h[n − k]} is an orthonormal set because h[n − k], h[n − l] = a[k − l] = δ[k − l]. To prove that it is an orthonormal basis for l2 (Z) we can show (according to theorem 2.4b) that if x[n], h[n − k] = 0 then x[n] = 0. The proof is straightforward: x[n]∗ h[n − k] = 0 =⇒ x[k]∗ ∗ h[−k] = 0 x[n], h[n − k] = 0 {k ∈ Z} =⇒ n
=⇒ X(e−jω )H(ejω )∗ = 0 =⇒ x(ejω ) = 0 since |H(ejω )| = 1 ∀ω =⇒ x[n] = 0 We can thus conclude that {h[n − k]} is an orthonormal basis for l2 (Z). (c) Since h[n], h[n − k] = δ[k], we know that h[n], h[n − 2k] = δ[2k] = δ[k], so we can conclude that {h[n − 2k]} is an orthonormal set. Theorem 2.4b says that if we can find a non-zero x[n] such that x[n], h[n − 2k] = 0, then {h[n − 2k]} is not a basis. Taking x[n] = h[n − 1], we have h[n − 1]∗ h[n − 2k] = δ[2k − 1] = 0 h[n − 1], h[n − 2k] = n
12
We can therefore conclude that {h[n − 2k]} is not a basis for l2 (Z).
2.17 Parseval’s relation for non-orthogonal bases
(a) First, we show that the sets {αi } and {β
i } are linearly independent.
n−1 n−1 n−1 n−1 Suppose ci αi = 0. Then ci αi , βj = 0, βj = 0 and ci αi , βj = ci αi , βj = n−1
i=0
i=0
i=0
i=0
ci δ[i − j] = cj so cj = 0, which proves that {αi } is a linearly independent set. By an analogous
i=0
argument, {βi } is also a linearly independent set. Since {αi } is a set of n linearly independent vectors in Rn , {αi } is a basis for Rn . So we can write any v ∈ Rn as a linear combination of the n−1 ci αi . We can then write basis vectors: v = i=0
v, βj =
n−1
ci αi , βj
=
i=0
n−1
ci αi , βj = cj
i=0
n−1
βi , v αi . Since {βi } is also a basis for Rn , we can similarly write i=0
n−1 n−1 n−1 di βi where v, αj = di βi , αj = dj which likewise verifies that v = αi , v βi . v = which verifies that v =
i=0
i=0
i=0
(b) Consider v ∈ Rn as a column vector v = [v0 v1 . . . vn−1 ]T . Then, vα = Av where A is a matrix whose i-th row is αTi . The norm satisfies the relation vα 2 = vαT vα = v T AT Av. We can v vα 2 . Therefore = uT AT Au where u is the unit norm vector then write 2 v v ΛA = supu=1 (uT AT Au) λA v ≤ vα ≤ ΛA v where λA = inf u=1 (uT AT Au) From matrix algebra we know that ΛA and λA are the largest and smallest eigenvalue of AT A, respectively. Similarly λB v ≤ vβ ≤ ΛB v where ΛB and λB are the largest and smallest eigenvalue of B T B, where B is a matrix whose i-th 1 1 and ΛB = so the bound for vβ row is βiT . But since B T B = (AT A)−1 we have λB = ΛA λA can be written as 1 1 √ v ≤ vβ ≤ √ v ΛA λA (c) For any v ∈ Rn v =
n−1
βi , v αi =
i=0
n−1 i=0
13
αi , v βi
Using these equivalences, we have
n−1 n−1 αi , v βi , v = αi , v βi , v = vα , vβ v, v = i=0
i=0
n−1
v, g =
αi , v βi , g
n−1
=
i=0
αi , v βi , g = vα , gβ
i=0
2.18 Circulant matrices (a) Consider circularly convolving 2 N -point sequences c[n] and g[n]. Since circular convolution in time domain corresponds to multiplication in the DFT domain, the resulting sequence can be written as c ∗p g = Cg = F ;1 ΛF g where F = [WNij ]i,j=0,...,N −1 is the Fourier matrix, Λ = diag{C[0], C[1], . . . , C[N −1]}, and C[k], k = 0, 1, . . . , N − 1 are the DFT coefficients of c[n]. Hence for any circulant matrix C, it can be diagonalized as C = F ;1 ΛF . This formulation implies that the columns of F ;1 are the eigenvector of C, with C[k] being the associated eigenvalues. So the matrix C is nonsingular if and only if none of the DFT coefficients C[k] is zero.
(b) From the diagonalization of C, and using the fact that the determinant of a matrix is the product of its eigenvalues, N −1 C[k] . det(C) = k=0
(c) Since
C −1 = F −1 Λ−1 F ,
it has the same form as C in Part (a), and thus it is circulant as well. (d) Writing C 1 and C 2 as C 1 = F −1 Λ1 F and C 2 = F −1 Λ2 F , we see that C 1 C 2 = (F −1 Λ1 F )(F −1 Λ2 F ) = F −1 Λ1 Λ2 F = F −1 ΛF , where Λ = diag{C1 [0] C2 [0], C1 [1] C2 [1], . . . , C1 [N − 1] C2 [N − 1]}. So C1 C2 is a circulant matrix with eigenvalues C1 [k] C2 [k], k = 0, 1, . . . , N − 1. Since Λ1 Λ2 = Λ2 Λ1 , we have C 1C 2 = C 2C 1.
2.19 Walsh basis (a) W2 =
W1 W1 W1 −W1
⎤ 1 1 1 1 ⎢ 1 −1 1 −1 ⎥ ⎥ =⎢ ⎣ 1 1 −1 −1 ⎦ 1 −1 −1 1
14
⎡
⎡ W3 =
W4 = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ = ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣
W2 W2
W3 W3 W3 −W3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 −1 1 −1 1 −1 1 −1 1 −1 1 −1 1 −1 1 −1
⎢ ⎢ ⎢ ⎢ ⎢ W2 =⎢ ⎢ −W2 ⎢ ⎢ ⎢ ⎣
1 1 1 1 1 1 1 1
1 −1 1 −1 1 −1 1 −1
1 1 −1 −1 1 1 −1 −1
1 −1 −1 1 1 −1 −1 1
1 1 1 1 −1 −1 −1 −1
1 −1 1 −1 −1 1 −1 1
1 1 −1 −1 −1 −1 1 1
1 −1 −1 1 −1 1 1 −1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
1 1 −1 −1 1 1 −1 −1 1 1 −1 −1 1 1 −1 −1
1 −1 −1 1 1 −1 −1 1 1 −1 −1 1 1 −1 −1 1
1 1 1 1 −1 −1 −1 −1 1 1 1 1 −1 −1 −1 −1
1 −1 1 −1 −1 1 −1 1 1 −1 1 −1 −1 1 −1 1
1 1 −1 −1 −1 −1 1 1 1 1 −1 −1 −1 −1 1 1
1 −1 −1 1 −1 1 1 −1 1 −1 −1 1 −1 1 1 −1
1 1 1 1 1 1 1 1 −1 −1 −1 −1 −1 −1 −1 −1
1 −1 1 −1 1 −1 1 −1 −1 1 −1 1 −1 1 −1 1
1 1 −1 −1 1 1 −1 −1 −1 −1 1 1 −1 −1 1 1
1 −1 −1 1 1 −1 −1 1 −1 1 1 −1 −1 1 1 −1
1 1 1 1 −1 −1 −1 −1 −1 −1 −1 −1 1 1 1 1
1 −1 1 −1 −1 1 −1 1 −1 1 −1 1 1 −1 1 −1
1 1 −1 −1 −1 −1 1 1 −1 −1 1 1 1 1 −1 −1
(b) Let In denote the identity matrix of dimension n, and also note that Wk = WkT . W0T W0 = 1 = I1
W1T W1 = W2T W2 =
1 1 1 −1
W1 W1 W1 −W1
1 1 1 −1
= 2 I2
W1 W1 W1 −W1
= 4 I4
So we see that WkT Wk = Hence,
Wk;1 Wk;1 Wk;1 −Wk;1
Wk;1 Wk;1 Wk;1 −Wk;1
k is an 2k × 2k orthonormal matrix, for k = 0, 1, 2, . . .
√1 W 2k
15
= 2k I2k . .
1 −1 −1 1 −1 1 1 −1 −1 1 1 −1 1 −1 −1 1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
(c)
⎡ T
⎢ ⎢ ⎢ ⎢ ⎢ ⎣
=
⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ = ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣
1 0 0 0 0 0 0
Note that T T = T . ⎡ W0T W0 1 T ⎢ 2 W1 W1 ⎢ ⎢ TT T = ⎢ ⎢ ⎣
⎤
W0
√1 W1 2
0
0
√1 2 √1 2
√1 2 − √12
0 0 0 0
0 0 0 0
1 2 W2
1 W3 23/2
0 0 0
0 0 0
0 0 0
1 2 1 2 1 2 1 2
1 2 − 12 1 2 − 12
1 2 1 2 − 12 − 12
.. .
..
T W2
..
⎡
⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎦ ⎣
T W3
1 W3 23
.
0 0 0 0 0 0 1 2 0 − 12 0 − 12 0 · · · 1 2 0
⎤ 1 22 W2
⎥ ⎥ ⎥ ⎥ ⎥ ⎦
.
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
I1
⎤ I2
I4
⎥ ⎥ ⎥ ⎥ ⎥ ⎦
I8 ..
.
So T is an unitary matrix. (d) The tiling of the time-frequency plane is approximately as follows (Ti denotes the i-th row of T ):
ω π T5 T3 T1
T7
. . . T6 T2 T4 0
01 3
7
Figure P2.19(d)
16
15
n
Chapter 3 3.1 Orthogonality and completeness of the sinc basis
(a) First, let’s prove ϕ2k [n], ϕ2l+1 [n] = 0. Since ϕ2k [n] = ϕ0 [n − 2k], it suffices to show that ϕ0 [n], ϕ2l+1 [n] = 0, or more explicitly, g0 [n], g1 [n − 2l] = 0. By Parseval’s theorem, π 1 G0 (ejω )G1 (ejω )e−jω2l dω . g0 [n], g1 [n − 2l] = 2π −π Given g1 [n] = (−1)n g0 [−n + 1], the Fourier transform of g1 [n] is G1 (ejω ) = −G0 (e−j(ω−π) )e−jω . Since G0 (ejω ) is an ideal lowpass filter with cutoff at π/2, G1 (ejω ) is a highpass filter at cutoff π/2, and they do not overlap. Therefore, the integral is zero and the relation follows. Next, let’s prove ϕ2k+1 [n], ϕ2l+1 [n] = δ[k − l] . Because ϕ2k+1 [n] = ϕ1 [n − 2k], it suffices to show that ϕ1 [n], ϕ2l+1 [n] = δ[l], or g1 [n], g1 [n − 2l] = δ[l]. g1 [n], g1 [n − 2l] = (−1)n g0 [−n + 1], (−1)n−2l g0 [−(n − 2l) + 1] = g0 [−n + 1], g0 [−n + 2l + 1] = g0 [n], g0 [n + 2l] = δ[l] where the last equality follows from the fact that g0 [n] is orthonormal to its even shifts (i.e. g0 [n], g0 [n − 2l] = δ[l]). (b) Assume that the set {ϕk } is not complete. That is, suppose there exists a sequence x[n] with x > 0 such that ϕk , x = 0 for all k. ϕk , x = 0 ⇐⇒ ϕk , x2 = 0 |ϕk [n], x[n]|2 = 0 ⇐⇒
Z
k∈
⇐⇒ X[k] = ϕk [n], x[n] = 0, ∀k Let’s look at the Fourier transform of the even and odd terms separately. The terms X[2k] = g0 [n − 2k], x[n] can be interpreted as x[n] convolved with g0 [−n], followed by sampling at every 2k index, where k ∈ Z. Noting that FT {g0 [−n]} = G0 (e−jω ), we have (also see Equation (2.5.19)) FT
X[2k] = g0 [n − 2k], x[n] = 0 ←→
1 G0 (e−jω )X(ejω ) + G0 (−e−jω )X(−ejω ) = 0. 2
By Parseval’s equality, 0=
∞
|X[2k]|
2
=
k=−∞
=
2 1 [G0 (e−jω )X(ejω ) + G0 (−e−jω )X(−ejω )] dω 2 −π π 1 1 |G0 (e−jω )|2 |X(ejω )|2 + 2Re{G0 (e−jω )X(ejω )G0 (−e−jω )X(−ejω )}+ 4 2π −π |G0 (−e−jω )|2 |X(−ejω )|2 dω 1 2π
π
17
Note that G0 (e−jω ) = G0 (ejω ) because it is symmetric in ω. Also G0 (−e−jω ) is G0 (ejω ) modulated by π, so they do not overlap (recall G0 (ejω ) is an ideal lowpass with a cutoff of π/2). Thus the cross term is 0. The above expression then becomes π 2 1 1 jω 2 jω 2 |X(e )| dω + |X(−e )| dω = 0, 4 2π −π [−π,− π ]∪[ π ,π] 2
2
2
which implies
π π X(ejω ) = 0 ∀ω ∈ [− , ]. 2 2 Similarly, working with the odd terms X[2k + 1] = g1 [n − 2k], x[n] will conclude that π π X(ejω ) = 0 ∀ω ∈ [−π, − ] ∪ [ , π]. 2 2
Combining these two results shows that X[k] = 0 for all k implies X(ejω ) = 0 for all ω ∈ [−π, π]. X(ejω ) = 0 ∀ ω ⇐⇒ |X(ejω )|2 = 0 ∀ ω ⇐⇒ x2 = 0 (by Parseval) . This contradicts the assumption that x > 0, and thus the set is complete.
3.2 Necessity of shift by one in orthogonal bases We assume the inner product definition preceding (2.2.1). As in (3.1.20), define π 1 sin π2 n n 1 sin 2 n √ , and ϕ . [n] = g [n − 2k] = (−1) ϕ2k [n] = g0 [n − 2k] = √ 1 2k+1 π π 2 2n 2 2n
We wish to show that the ϕk ’s do not form an orthogonal basis for 2 (Z). To show that the ϕk ’s are not complete, we will construct x[n] ∈ 2 (Z) with x = 1 that has zero expansion. Intuitively, the problem with the ϕk ’s is that g0 and g1 are both even, so all of the putative basis functions are even about an even point. Hence they cannot represent odd functions. Let y[n] = sin π2 n. Then for any k ∈ Z, ϕ∗2k [n]y[n] ϕ2k , y = n
=
g0∗ [n − 2k]y[n]
n
=
g0∗ [m]y[m + 2k] by substituting m = n − 2k
m
1 sin π m π 2 √ sin (m + 2k) = π m 2 2 2 m 1 sin π m π 2 √ (−1)k sin m = π 2 2 2m m = 0 since the summand is an odd function of m. Similarly, ϕ2k+1 , y = 0 for all k ∈ Z. This does not work as a counterexample to completeness because y ∈ 2 (Z). However, this is useful in constructing a counterexample. For k ∈ Z+, let √ 1 sin π2 n, −(2k + 1) ≤ n ≤ 2k + 1 2k+2 . xk [n] = 0, otherwise 18
Each xk has unit norm, so xk ∈ 2 (Z) for all k ∈ Z+. Let x = limk→∞ xk . Because 2 (Z) is a complete metric space, x ∈ 2 (Z) and x = 1. It is easy to check that ϕk , x = 0 for all k, so the ϕk ’s do not form an orthogonal basis.
3.3 Nonorthogonal time-invariant expansions
A time-invariant expansion satisfies the relation: x[n] ←→ X[k] ⇐⇒ x[n − m] ←→ X[k − m]. In terms of the basis vectors φk [n], this is equivalent to φk [n], x[n − m] = φk−m [n], x[n]. The change of variable n → n + m on the left side gives φk [n + m], x[n] = φk−m [n], x[n], and a second change of variable k → k + m gives φk+m [n + m], x[n] = φk [n], x[n], which implies that φk+m [n + m] = φk [n] for a time-invariant expansion, as derived in Proposition 3.3. For a nonorthogonal time-invariant expansion, we have φk [n], φk+m [n] = φk [n], φk [n − m] = a[m] = δ[m] Taking the Fourier transform yields: jω ∗ A(e ) = φk [n]φk [n − m] e−jωm m
=
n
n
=
φ∗k [n]e−jωn
m
φk [n − m]e−jω(m−n)
φk [n]∗ e−jωn
φk [−m ]e−jωm
m =m−n
n
2 2 = Φ∗ (e−jω )Φ(e−jω ) = Φ(e−jω ) = Φ(ejω ) = A(ejω ) = 1 Because it is not orthogonal, this expansion is frequency-selective. We conclude by noting that if we are given a function A(ejω ) that is nonnegative on the unit circle, we can find Φ(ejω ) within a phase factor, and from that we can determine the frequency-selective nonorthogonal basis functions, in this case denoted by ck [n]: Ak (ejω ) = Bk (ejω )Bk∗ (ejω ) = Ck (ω)ejθk (ω) Ck (ω)e−jθk (ω) ck [n] = ck+m [n + m] =
π 1 Ck (ω)ejθk (ω) ejωn dω 2π −π π 1 Ck+m (ω)ejθk+m (ω) ejω(n+m) dω 2π −π
The time-invariance requirement ck [n] = ck+m [n + m] implies that Ck (ω)ejθk (ω) = Ck+m (ω)ejθk+m (ω) ejωm which is satisfied by Ck+m (ω) = Ck (ω)
and
3.4 Proof of Theorem 3.7
θk (ω) = θk+m (ω) + ωm
(a) ⇔ (c): This equivalence follows from carefully lining up the rows and columns of Ta and Ts . In (3.2.1), let us number the rows such that y0 [0] is in the zeroth row of y and x[0] is in the 19
zeroth row of x. (It follows that yi [k] is in the (2k + i)-th row, k ∈ Z, i ∈ {0, 1}.) Recall the definitions of Ta following (3.2.1) and Ts following (3.2.6). With this row numbering, the implied column numbering has h0 [0] in the (0,0) position of Ta and g0 [0] in the (0,0) position of Ts . Now the (2k + i)-th row of Ta is hi [2k − n] and the (2 + j)-th row of Ts T is gj [n − 2]. Thus the (2k + i, 2 + j) element of Ta Ts is hi [2k − n], gj [n − 2] and it follows that (a) ⇔ (c). (b) ⇔ (d): Recall the definitions of Gm (z) following (3.2.29) and of Hm (z) in (3.2.14). By simply substituting the definitions, G0 (z)H0 (z) + G1 (z)H1 (z) G0 (z)H0 (−z) + G1 (z)H1 (−z) . Gm (z)Hm (z) = G0 (−z)H0 (z) + G1 (−z)H1 (z) G0 (−z)H0 (−z) + G1 (−z)H1 (−z) It follows that Gm (z)Hm (z) = 2I ⇔ (b). Since scalar rational functions commute and Gm (z) and Hm (z) are square, Gm (z)Hm (z) = 2I ⇔ Hm (z)Gm (z) = 2I. (d) ⇔ (e): Using (3.2.28) and (3.2.29), we can write 1 1 1 1 0 2 Hm (z) and Hp (z ) = 1 −1 0 z −1 2 1 1 0 1 1 2 Gm (z). Gp (z ) = 1 −1 2 0 z Thus 2 1 1 1 1 0 1 0 1 1 Hm (z) Gm (z) Hp (z )Gp (z ) = 1 −1 0 z −1 0 z 1 −1 2 2 1 Hm (z)2IGm (z) = 2 1 Hm (z)Gm (z). = 2 2
2
It follows immediately that Hp (z)Gp (z) = I ⇔ Hm (z)Gm (z) = 2I. Noting that all the matrices involved are square, (d) ⇔ (e). (a) ⇔ (d): We will start with Hm (z)Gm (z) = 2I and show that this is equivalent to (a). (Since the matrices are square, this is equivalent to (d).) Looking at the (i, j)th element of the product Hm (z)Gm (z) gives Hi (z)Gj (z) + Hi (−z)Gj (−z) = 2δ[i − j]. Using the modulation property, we can write the time-domain equivalents hi [n]gj [k − n] + (−1)k hi [n]gj [k − n] = 2δ[i − j]δ[k],
Z
Z
n∈
or
n∈
hi [n]gj [k − n] = 2δ[i − j]δ[k], ∀k ∈ Z. 1 + (−1)k
Z
(4)
n∈
To restrict our attention to even k, replace k by 2m. Noting that 1 + (−1)2m = 2 and δ[2m] = δ[m], this gives hi [n]gj [2m − n] = δ[i − j]δ[m], ∀m ∈ Z. (5)
Z
n∈
20
Writing (5) as an inner product and replacing n and m by −n and −m, respectively, gives (a). For odd k, δ[k] = 0 and 1 + (−1)k = 0. Thus (4) is automatically satisfied for odd k and (4) follows from (5) when the steps are reversed.
3.5 Constraint on the length of orthogonal FIR filters
Without loss of generality, let the support of g0 [n] be n = 0, 1, ..., L − 1, and specifically, that g0 [0] and g0 [L − 1] be non-zero. Since P (Z) = G0 (z)G0 (z −1 ), p[n] = g0 [n] ∗ g0 [−n], the support of p[n] is {−(L − 1), −(L − 2), ..., L − 2, L − 1}. In the time-domain, the autocorrelation satisfies p[n] + (−1)n p[n] = 2δ[n] . In other words, p[2n] = 0 for n = ±1, ±2, ..., and p[0] = 1. Evaluating at n = L − 1, and assuming L is odd and greater than 1, we have p[L − 1] + (−1)L−1 p[L − 1] = 2p[L − 1] = 0 . Since p[L − 1] = g0 [0]g0 [L − 1], this implies that either g0 [0] or g0 [L − 1] is zero, contradicting the assumption. Hence, L cannot be odd, and thus must be even. (Note that the only case when L could be odd is for the trivial case of L = 1.)
3.6 Factorization
3 For the factors A(z) = (1 + z)3 1 + z −1 and B(z) =
1 2 3z − 18z + 38 − 18z −1 + 3z −2 256
we have to show that P (z) = A(z) B(z) satisfies P (z) + P (−z) = 2. Factoring out the 256 in B(z) to simplify the algebra, we have: 256 [A(z)B(z) + A(−z)B(−z)] = 3z 2 + 3z −2 (A(z) + A(−z)) + 18z + 18z −1 (−A(z) + A(−z)) + 38 (A(z) + A(−z)) Since A(z) = z 3 + 6z 2 + 15z + 20 + 15z −1 + 6z −2 + z −3 we have A(z) + A(−z) = 12z 2 + 40 + 12z −2 and −A(z) + A(−z) = −2z 3 − 30z − 30z −1 − 2z −3 . Calculating the coefficients of each power of z yields: 256 [A(z)B(z) + A(−z)B(−z)] = + + =
z 4 (36 − 36) + z 2 (120 − 540 + 456 − 36) + z 3 (0) + z(0) (36 − 540 + 1520 − 540 + 36) + z −1 (0) + z −3 (0) z −2 (120 − 540 + 456 − 36) + z −4 (36 − 36) 512
which confirms that P (z) + P (−z) = 2. Factorization yields B(z) = 4.14 × 10−4 1 − αz −1 1 − α∗ z −1 (1 − αz) (1 − α∗ z) with α = −2.71 + 1.44j and where α∗ denotes the complex conjugate of α. For the orthogonal filter bank, we can take 3 4.14 × 10−4 1 + z −1 1 − αz −1 1 − α∗ z −1 H0 (z) = so that h0 [n] = 0.0205[1, 8.426, 28.720, 45.608, 33.757, 9.444] h1 [n] = 0.0205[−9.444, 33.757, −45.608, 28.720, −8.426, 1]
21
3.7 Proof of Theorem 3.8 Theorem 3.8 is essentially a specialization of Theorem 3.7 to the orthonormal, FIR case. Most of the proof of Theorem 3.7 (Problem 3.4) can be repeated with hi [n] = gi [−n], Hi (z) = Gi (z −1 ), Ta = Ts T , Hm (z) = Gm T (z −1 ), and Hp (z) = Gp T (z −1 ). All that remains to be shown is G1 (z) = −z −2K+1 G0 (−z −1 ) for some K ∈ Z. The first column of Gm T (z −1 )Gm (z) = 2I is G0 (z) 2 T −1 = . Gm (z ) 0 G1 (z) −1 gives Explicitly computing Gm T (z −1 ) −1 2 G0 (z) = Gm T (z −1 ) G1 (z) 0 1 2 G1 (−z −1 ) −G1 (z −1 ) = 0 G0 (z −1 )G1 (−z −1 ) − G0 (−z −1 )G1 (z −1 ) −G0 (−z −1 ) G0 (z −1 ) −1 2 G1 (−z ) = . −1 −1 −1 −1 G0 (z )G1 (−z ) − G0 (−z )G1 (z ) −G0 (−z −1 ) Thus
where A(z) =
G0 (z) = A(z)G1 (−z −1 ) and
(6)
G1 (z) = −A(z)G0 (−z
(7)
−1
),
2 . G0 (z −1 )G1 (−z −1 ) − G0 (−z −1 )G1 (z −1 )
Substituting −z −1 for z in (7) gives G1 (−z −1 ) = −A(−z −1 )G0 (z). Substituting this in (6), gives G0 (z) = −A(z)A(−z −1 )G0 (z), so (8) A(z)A(−z −1 ) = −1. All FIR solutions of (8) are of the form A(z) = z −2K+1 . Combining this with (7) completes the proof.
3.8 Linear phase solutions (This proof is taken from reference [319]; in which further details can be found.) First, note that since H0 (z) and H1 (−z) both have linear phase, so does P (z) = H0 (z)H1 (−z). If P (z) is of odd length 2N + 1, then the analysis filters must either both have odd length or both have even length. We have 2N N −1 −i i pi z [1 − (−1) ] = 2p2i+1 z −(2i+1) P (z) − P (−z) = i=0
i=0
Since P (z) is of odd length, both P (z) and P (−z) are symmetric or anti-symmetric about the center point i = N , which implies that P (z) − P (−z) is symmetric about i = N and the single nonzero coefficient is the central one pN . Thus, P (z) is symmetric and not antisymmetric, which implies that H0 (z) and H1 (−z) are either both symmetric or both anti-symmetric for P (z) of odd length. (a) L0 and L1 both odd. The length of P (z) is L0 + L1 + 1, which is also odd. The center of symmetry at (L0 + L1 )/2 − 1 samples from the endpoints must be odd because the only nonzero 22
elements in P (z) − P (−z) are at odd powers of z (the center of symmetry is at a non-zero point). Thus, (L0 + L1 )/2 = L0 + (L1 − L0 )/2 has to be even, which means that (L1 − L0 )/2 has to be odd because L0 is odd. Thus, the length difference L1 −L0 is an odd multiple of 2. Note that there are no same-length solutions. It was stated in the opening that H0 (z) and H1 (−z) can be both symmetric or both antisymmetric. The latter case is ruled out because the polyphase components are also antisymmetric when the length is odd, and therefore are not coprime, so perfect reconstruction would not be possible. (b) L0 and L1 both even. Again the center point (L0 +L1 )/2−1 has to be odd. Then L0 +(L1 −L0 )/2 again has to be even; since L0 is even, the length difference L1 − L0 is an even multiple of 2. In this case, same-length solutions do exist. If H0 (z) is symmetric, so is H1 (−z) (as shown in the opening). Then, H1 (z) is antisymmetric because it has even length. Likewise, H1 (z) is symmetric if H0 (z) is antisymmetric. (c) If P (z) has even length, one filter has odd length, and one even. Then P (z) − P (−z) =
2N −1
−i
pi z [1 − (−1) ] = i
N −1
i=0
2p2i+1 z −(2i+1)
i=0
From 3.2.66, P (z) has only a single nonzero odd-indexed coefficient. Note however that pi = ±p2N −1−i for i = 0, . . . , N − 1, so odd-indexed coefficients are paired with even-indexed ones. It follows that P (z) also has only one nonzero even-indexed coefficient, and is of the form P (z) = pj z −j (1 ± z 2N −1−2j ). Clearly, all of the zeros of P (z) are on the unit circle at the 2(N − j) − 1 roots of ±1; then, all of the zeros of H0 (z) and H1 (−z) are on the unit circle. If H0 (z) and H1 (z) are both to be antisymmetric, both must have zeros at z = 1. This means that H1 (−z) has a zero at z = 1 or z = −1 depending on whether it is of odd or even length, respectively. This implies that P (z) has either a double zero at z = 1 or a pair at z = 1 and z = −1. Since P (z) contains only the 2(N − j) − 1 roots of ±1 both possibilities are ruled out. Thus, either both filters are symmetric, or one is antisymmetric and the other is symmetric.
3.9 Test conditions for linear phase From condition (b) of Proposition 3.11, for both H0 (z) and H1 (z) to have the same lengths and be linear phase, they must both be of even lengths and one filter is symmetric while the other filter is antisymmetric. Let L be the length of the filters, and, without loss of generality, let H0 (z) be symmetric and H1 (z) be antisymmetric. Because the filters are linear phase, we can write H0 (z) = z −L+1 H0 (z −1 ) , H1 (z) = −z −L+1 H1 (z −1 ) . Writing H0 (z) in its polyphase components, we have H0 (z) = H00 (z 2 ) + z −1 H01 (z 2 )
= z −L+1 (H00 (z −2 ) + zH01 (z −2 ))
= z −L+2 H01 (z −2 ) + z −1 (z −L+2 H00 (z −2 )) . Similarly,
H1 (z) = −z −L+2 H11 (z −2 ) − z −1 (z −L+2 H10 (z −2 )) .
23
H p (z) =
= = =
L L z − 2 +1 H00 (z −1 ) z − 2 +1 H01 (z −1 ) L L −z − 2 +1 H11 (z −1 ) −z − 2 +1 H10 (z −1 ) H00 (z −1 ) H01 (z −1 ) −L +1 2 z −H11 (z −1 ) −H10 (z −1 ) −1 ) H (z −1 ) (z 1 0 H 01 00 −L +1 z 2 H11 (z −1 ) H10 (z −1 ) 0 −1 −1 ) H (z −1 ) 0 1 (z 1 0 H 00 01 −L +1 z 2 1 0 H10 (z −1 ) H11 (z −1 ) 0 −1
=
H00 (z) H01 (z) H10 (z) H11 (z)
Hence, H p (z) satisfies the linear phase testing condition with k = L/2 − 1.
3.10 Complex linear phase FIR solutions (a) First, we rederive the orthonormality relations of theorem 3.8 for the case of complex filters. We have gi [n], gj∗ [n + 2m] = δ[i − j]δ[m] Using the Z-transform properties x[−n] ↔ X(1/z) and x∗ [n] ↔ X ∗ (z ∗ ) as well as the modulation property (−1)n x[n] ↔ X(−z), we can rewrite the inner product relation above in the Z domain as Gi (z)G∗j (1/z ∗ ) + Gi (−z)G∗j (−1/z ∗ ) = 2δ[i − j] For real filters, the Z transform is conjugate symmetric, so in the real case we can replace G∗j (1/z ∗ ) by Gj (z −1 ) to arrive at the relations in parts (b) and (d) of theorem 3.8. In the complex case, the modulation domain relations become ∗ Gm (z)GH m (1/z ) = 2I
∗ Hm (z) = GH m (1/z )
In a complex orthogonal filterbank, we can choose G1 (z) = −zG∗0 (−1/z ∗ )
H0 (z) = G∗0 (1/z ∗ ) H1 (z) = G∗1 (1/z ∗ )
It is straightforward to verify (by direct substitution) that these filters satisfy the various orthogonality relations, some of which are: G0 (z)G∗1 (1/z ∗ ) + G0 (−z)G∗1 (−1/z ∗ ) = 0
H0 (z)H0∗ (1/z ∗ ) + H0 (−z)H0∗ (−1/z ∗ ) = 2 H0 (z)H1∗ (1/z ∗ ) + H0 (−z)H1∗ (−1/z ∗ ) = 0 Now, we consider the complex case of proposition 3.12. Orthonormality implies ∗ Hp (z)HH p (1/z ) = I
from which we have
∗ ∗ (1/z ∗ ) + H01 (z)H01 (1/z ∗ ) = 1 H00 (z)H00
24
The equations in 3.2.68 hold for complex filters as well, so we can substitute for H01 (z) as in proposition 3.12. The resulting equation is ∗ ∗ (1/z ∗ ) + H00 (z −1 )H00 (z ∗ ) = 1 H00 (z)H00
As a check, note that conjugate symmetry for real filters then leads to the relation in the book: H00 (z)H00 (z −1 ) = 1/2. The Z domain equation for H00 (z) implies the time-domain relation 1 2 where the inner product implies a conjugation of the second term. An equivalent result can be derived for the other polyphase component. These requirements are non-restrictive; in other words, we have verified the existence of linear phase orthogonal filterbanks with complex coefficients. Real {h00 [m], h00 [m − k]} =
(b) The factorization of P (z) for the D3 filter is derived in the solution to problem 3.6. It consists of six zeros at −1 and a set of four conjugate reciprocal zeros: P (z) = (1 + z)3 (1 + z −1 )3 (1 − αz −1 )(1 − α∗ z −1 )(1 − αz)(1 − α∗ z) The scale factor is omitted for simplicity. In problem 3.6, we showed that this autocorrelation satisfies P (z) + P (−z) = 2 and indicated a factorization that yielded an orthogonal filterbank with real coefficients; this resulted from interpreting P (z) as the autocorrelation G0 (z)G0 (z −1 ) and grouping the quadruple of zeros (those not at z = −1) in conjugate pairs in the factorization. For the complex case, the orthogonality constraint derived in (a) shows that P (z) should be factored as G0 (z)G∗0 (1/z ∗ ). By grouping the quadruple of zeros in reciprocal pairs, we get a filter G0 (z) with complex coefficients that has linear phase and satisfies the validity constraint (ı.e. orthogonality). The linear phase synthesis filters are given by G0 (z) = (1 + z −1 )3 (1 − αz −1 )(1 − αz) G1 (z) = −z(1 + z)3 (1 + α∗ z)(1 + α∗ z −1 ) and the analysis filters are related as indicated in part (a).
3.11 Spectral factorization method for two-channel filter banks
−1 cos(3ω) + cos(ω) + 1 is not positive on the unit circle. 2 The Smith-Barnwell correction is applied by finding δ = min P (ejω ) = −0.076 and taking P (z) = P (z) − δ . P (z) satisfies the requirements P (z) + P (−z) = 2 and P (ejω ) ≥ 0. We have P (z) = 1−δ H0 (z)H0 (z −1 ) where H0 (z) = 0.689(1−2.108z −1 )(1+1.291z −1 +z −2 ) and H1 (z) = −z −3 H0 (−z −1 ). These constitute an orthogonal filter bank. 1 1 1 1 1 (b) P (z) = (z + 1 + z −1 ) − z 2 + z + + z −1 − z −2 4 4 2 4 4 (a) P (ejω ) =
P (z) = z −1 P (z) satisfies P (z) − P (−z) = 2z −2k+1 (with k = 0 in this case) which gives rise to a biorthogonal filter bank with P (z) = H0 (z)H1 (−z) and the filters H0 (z) = z + 1 + z −1 H1 (z) =
1 1 1 1 1 z + − z −1 + z −2 + z −3 4 4 2 4 4 25
G0 (z) =
2 H1 (−z) det Hm (z)
G1 (z) =
−2 H0 (−z) det Hm (z)
where Hm (z) is the modulation matrix (see Section 3.2.4 on designing linear phase FIR filter banks). The result is 1 1 1 1 1 G0 (z) = − z 2 + z + + z −1 − z −2 4 4 2 4 4
G1 (z) = z 2 − z + 1
(c) The Parks-McClellan algorithm yields a P (z) with an equiripple stopband (and passband). Suppose the minimum value of P (ejω ) is δ; P (ejω ) takes exactly this value at each of its minima in the stopband (except possibly at ω = π). Let there be L zeros, then there may be either L/2 P (z) − δ . The minimum of P (ejω ) or L/2 minima. Applying the correction yields P (z) = 1−δ is then zero and P (ejω ) takes on this zero value at frequencies in the stopband where P (ejω ) has its minima. Thus the original L zeros of P (ejω ) has been transformed to double zeros in P (ejω ) (double zeros because the poles come in (α, 1/α∗ ) pairs, which are the same points on the unit circle). For more detail on the Parks-McClellan algorithm, consult the book Discrete-Time Signal Processing by Oppenheim and Shafer, Prentice Hall.
3.12 Complementary filters
We prove that the filter H0 (z) = (1 + z −1 )N always has a complementary filter in three steps: (1) Note that this H0 (z) does not have any pairs of zeros at (−α, α); all of its zeros are at z = −1. (2) Prove that H0 (z) does not have any pairs of zeros at locations (−α, α) is equivalent to the coprimeness of its polyphase components. (3) Use Proposition 3.13 to conclude that H0 (z) has a complementary filter because its polyphase components are coprime, as indicated by (1) and (2). The proof of (2) is as follows. Consider a function U (z) with polyphase components U0 (z) and U1 (z) so that U (z) = U0 (z 2 ) + z −1 U1 (z 2 ). Construct a new function V (z) by adding two zeros at {α1 , α2 } to U (z) and determine its polyphase components: V (z) = (1 − α1 z −1 )(1 − α2 z −1 ) U0 (z 2 ) + z −1 U1 (z 2 )
= (1 − (α1 + α2 )z −1 + α1 α2 z −2 )(U0 (z 2 ) + z −1 U1 (z 2 )) = (1 + α1 α2 z −2 )U0 (z 2 ) − (α1 + α2 )z −2 U1 (z 2 ) + z −1 −(α1 + α2 )U0 (z 2 ) + (1 + α1 α2 z −2 )U1 (z 2 )
= V0 (z 2 ) + z −1 V1 (z 2 ) If α2 = −α1 , then V0 (z) = (1 − α21 z −1 )U0 (z) and V1 (z) = (1 − α21 z −1 )U1 (z) and so V0 (z) and V1 (z) are not coprime. Thus we have proved that if V (z) has a zero pair (α, −α), then V0 (z) and V1 (z) are not coprime. Conversely, if V0 (z) and V1 (z) are not coprime, they can be written as V0 (z) = (1 − β 2 z −1 )U0 (z)
V1 (z) = (1 − β 2 z −1 )U1 (z) with the common zero β 2 = α. Then V (z) = (1 − αz −1 )(1 + αz −1 )(U0 (z 2 ) + z −1 U1 (z 2 )), 26
thus V (z) has the zero pair (α, −α). Using these results, we can conclude that since all of the zeros of H0 (z) are at z = −1, its polyphase components are coprime; according to Proposition 3.13, it thus has a complementary filter.
3.13 Orthogonal lattice angle constraints For orthogonal filter banks (see equation 3.2.61 in example 3.3), cos α1 sin α1 H00 (z) H01 (z) 1 0 cos α0 sin α0 Hp (z) = = H10 (z) H11 (z) − sin α1 cos α1 − sin α0 cos α0 0 z −1 From this factorization, we have H0 (z) = cos α0 cos α1 + sin α0 cos α1 z −1 − sin α0 sin α1 z −2 + cos α0 sin α1 z −3 H0 (ejω )ω=π = cos α0 cos α1 − sin α0 cos α1 − sin α0 sin α1 − cos α0 sin α1 = cos(α0 + α1 ) − sin(α0 + α1 ) = 0 We can thus conclude for the orthogonal lattice that α0 + α1 =
5π π or . 4 4
3.14 Interpolation followed by decimation (a) The z-transform of the input and output are related by: # 1 1 # 1 1 1 $ 1 $ 1 1 ˆ = X(z) P z 2 + P −z 2 X(z) = X(z) H z 2 G z 2 + H −z 2 G −z 2 2 2 ˆ For x ˆ[n] = x[n − k], we have X(z) = z −k X(z), so the product filter P (z) must satisfy 1 $ 1 # 1 P z 2 + P −z 2 = z −k =⇒ P (z) + P (−z) = 2z −2k 2 Thus P (z) must consist of a single even power of z. (b) H(z) and G(z) can be expressed in terms of their polyphase components as H(z) = H0 (z 2 ) + z −1 H1 (z 2 )
and G(z) = G0 (z 2 ) + zG1 (z 2 )
Combining this with the condition from (a) yields P (z) + P (−z) = 2[H0 (z 2 )G0 (z 2 ) + H1 (z 2 )G1 (z 2 )] = 2z −2k which reduces to
H0 (z)G0 (z) + H1 (z)G1 (z) = z −k
Using Bezout’s identity, we conclude that G(z) can be found if and only if the polyphase components of H(z) have no zeros in common (except for zeros at ∞), that is, they are coprime. (c) The polyphase components of H (z) are equivalent (H0 (z) = H1 (z) = 1 + z −1 ) and thus not coprime, so there is no G (z) such that perfect reconstruction is achieved. The polyphase components of H (z) are H0 (z) = 1 + z −1 + z −2 and H1 (z) = 1 + z −1 27
Using
G0 (z) = 1 − z −1
gives
and G1 (z) = 1 − z −1 + z −2
H0 (z)G0 (z) + H1 (z)G1 (z) = 2
so a perfect reconstruction synthesis filter for H (z) is given by G (z) = G0 (z 2 ) + zG1 (z 2 ) = z + 1 − z −1 − z −2 + z −3 If a causal filter is desired, z −2k G (z) can be used.
3.15 Orthogonality relations for an octave-band filter bank (a) First, let’s prove equation (3.3.16) (j)
(i)
g1 [n − 2j k], g1 [n − 2i l] = δ[i − j]δ[k − l] . (i)
(j)
Let Qij (z) denote the z-transform of the correlation between g1 [n] and g1 [n], so Qij (z) = G1 (z)G1 (z −1 ) . (i)
(j)
i
j
Equation (3.3.16) is satisfied if and only if all the terms z (2 l−2 k) , for all i = j and k = l , in Qij (z) have zero coefficients. We proceed the prove by breaking the problem down into 2 cases: i = j and j < i (j > i is similar). Recall that j−1
(j)
G1 (z) = G1 (z 2
j−2
)G0 (z 2
) · · · G0 (z 2 )G0 (z) .
Case 1: i = j Qij (z) can then be written as # $# $ j−1 j−2 j−1 j−2 Qjj (z) = G1 (z 2 )G0 (z 2 ) · · · G0 (z 2 )G0 (z) G1 (z −2 )G0 (z −2 ) · · · G0 (z −2 )G0 (z −1 ) j−1
= G1 (z 2 %
j−1
)G1 (z −2 &'
j−2
) G (z 2 ( %0
j−2
)G0 (z −2
) · · · G0 (z 2 )G0 (z −2 )G0 (z)G0 (z −1 ) &' ( P (j−1) (z)
where P (j) (z) = G0 (z)G0 (z −1 ) is the z-transform of the autocorrelation of g0 [n] (see the proof of Proposition 3.15). To prove (3.3.16) for i = j, it is equivalent to show that Qjj (z) has zero j j coefficients for z 2 (k−l) for all k = l and k, l ∈ Z; i.e. z 2 k terms for any k ∈ Z have zero coefficients. From the orthogonality of g1 [n] with respect to its even shifts, we have (j)
(j)
(j)
g1 [n − 2k], g1 [n − 2l] = δ[k − l] ⇐⇒ R(z) = G1 (z)G1 (z −1 ) = 1 + zR1 (z 2 ) where R(z) is the z-transform of the autocorrelation function of g1 [n] and R1 (z) is the 1st polyphase component of R(z). From the proof of Proposition 3.15, we can write P (j−1) (z) in its polyphase form as P
(j−1)
(z) = 1 +
2j−1 −1 n=1
28
j−1
z n Pn(j−1) (z 2
).
Now we can write Qjj (z) as j−1
Qjj (z) = R(z 2 # =
1+z
)P (j−1) (z)
⎡ ⎤ 2j−1 $ −1 j−1 R1 (z ) ⎣1 + z n Pn(j−1) (z 2 )⎦
2j−1
2j
n=1
= 1+z
2j−1
2j−1 −1
2j
R1 (z ) +
% +z
2j−1
2j
R1 (z )
j−1
z n Pn(j−1) (z 2
n=1
2j−1 −1
&'
(
A
j−1
z n Pn(j−1) (z 2
n=1
&'
%
)
) . (
B
j
It is clear that the first two terms do not have powers of z 2 . The third expression labelled as A contains z-terms which have exponents of the form 2j−1 −1
j−1
(n + mn 2
j−1
)= 2
n=1
2j−1 −1
mn +
n=1
% &' multiples of
(
2j−1 −1
n
n=1
2j−1
where mn ∈ Z, n = 1, . . . , 2j−1 − 1. The latter summation can be found (by the trapezoidal rule) to be 2j−1 −1 n = (2j−1 − 1 + 1)(2j−1 − 1)/2 = 22j−3 − 22j−2 . n=1 j
Hence, the expression A does not contain powers of z 2 . The expression B contains terms which have exponents of the form j−1
2
j
+ m0 2 +
2j−1 −1
j−1
(n + mn 2
j
j−1
) = m0 2 + 2
n=1
%
(1 +
2j−1 −1
mn ) +
n=1
&' multiples of
(
2j−1 −1
n.
n=1
2j−1
j
It is easy to verify that this expression does not contain powers of z 2 . Case 2: Without loss of generality, assume j < i. j i j i−j For j < i, we need to show that all coefficients of the terms z 2 k−2 l = z 2 (k−l2 ) , for all j k, l ∈ Z, are zero. That is, we need to show that all coefficients of powers of z 2 are zero. The term Qij (z) can be written (after rearranging) as # $# $ j−1 j−1 i−1 i−2 j Qij (z) = G0 (z 2 )G1 (z −2 ) G1 (z 2 )G0 (z 2 ) · · · G0 (z 2 ) P (j−1) (z) . From the orthogonality of g0 [n] and g1 [n] with respect to even shifts, we have the following relation:
g0 [n − 2k], g1 [n − 2l] = 0 ∀k, l ∈ Z ⇐⇒ S(z) = G0 (z)G1 (z −1 ) = zS1 (z 2 ) 29
where S1 (z) is the first polyphase component of S(z). Now Qij (z) can be expanded as ⎡ ⎤ 2j−1 # j−1 $ # $ −1 j j−1 i−1 i−2 j Qij (z) = z 2 S1 (z 2 ) ⎣1 + z n Pn(j−1) (z 2 )⎦ G1 (z 2 )G0 (z 2 ) · · · G0 (z 2 ) . n=1
After expanding Qij (z), the first term contains z terms whose exponents are of the form 2j−1 + m0 2j + m1 2i−1 + m2 2i−2 + · · · + mi−j 2j = 2j−1 + 2j (m0 + m1 2i−j−1 + m2 2i−j−2 + · · · + mi−j ) , ∀m0 , m1 , . . . , mi−j ∈ Z , which cannot be multiples of 2j . Similarly, the second term contains exponents of the form
2j−1 −1 (k + 2j−1 nk ) 2j−1 + m0 2j + m1 2i−1 + m2 2i−2 + · · · + mi−j 2j + k=1
j−1 j−1 2 −1 2 −1 nk ) + k=1 k + 2j (m0 + m1 2i−j−1 + m2 2i−j−2 + · · · + mi−j ) , = 2j−1 (1 + k=1 ∀m0 , m1 , . . . , mi−j , n1 , . . . , n2j−1 −1 ∈ Z , which cannot be multiples of 2j . (b) To show equation (3.3.17) (J)
(j)
g0 [n − 2J k], g1 [n − 2j l] = 0 , ∀j = 1, . . . , J, J
j
it is equivalent to show that G0 (z)G1 (z −1 ) has zero coefficients for the terms z 2 k−2 l for any j k, l ∈ Z (or equivalently, any powers of z 2 must have zero coefficients since we can write 2J k −2j l = 2j (2J−j k − l)). By rearranging terms, we get ⎧ # $# $ j−1 j−1 j J −1 (j−1) 2 −2 2 2 ⎪ (z) G0 (z )G1 (z ) G0 (z ) · · · G0 (z ) if j < J ⎪ ⎨ P (J) (j) −1 G0 (z)G1 (z ) = # $ ⎪ ⎪ ⎩ P (j−1) (z) G0 (z 2j−1 )G1 (z −2j−1 ) if j = J . (J)
(j)
j−1
j−1
By substituting the expressions for P (j−1) (z) and G0 (z 2 )G1 (z −2 ) from part (a), and exj (J) (j) panding the terms, it is straightforward to verify that G0 (z)G1 (z −1 ) contains no powers of z 2 .
3.16 Tree-structured orthogonal filter banks (a) The z-transform of the equivalent filters are (2)
(2)
(2)
(2)
G0 (z) = G0 (z)G0 (z 2 ) , G1 (z) = G0 (z)G1 (z 2 ) , G2 (z) = G1 (z)G0 (z 2 ) , G3 (z) = G1 (z)G1 (z 2 ) . (2)
For G0 (ejω ) being ideal lowpass filter, and G1 (ejω ) being ideal highpass filter, the filters Gi (ejω ) are bandpass filters: 1 |ω| ≤ π4 (2) G0 (ejω ) = 0 otherwise 30
(2) G1 (ejω )
=
(2) G2 (ejω )
=
(2) G3 (ejω )
=
1 π4 ≤ |ω| ≤ 0 otherwise
π 2
1 3π 4 ≤ |ω| ≤ π 0 otherwise 1 π2 ≤ |ω| ≤ 0 otherwise
3π 4
.
(b) Now let’s take the Haar filters G0 (z) = G1 (z) =
1 √ (1 + z −1 ) 2 1 √ (1 − z −1 ) . 2
(2)
Multiplying out the expressions for Gi (z), i = 0, 1, 2, 3 and transform to time-domain yields 1 (2) g0 [n] = (δ[n] + δ[n − 1] + δ[n − 2] + δ[n − 3]) 2 1 (2) g1 [n] = (δ[n] + δ[n − 1] − δ[n − 2] − δ[n − 3]) 2 1 (2) g2 [n] = (δ[n] − δ[n − 1] + δ[n − 2] − δ[n − 3]) 2 1 (2) g3 [n] = (δ[n] − δ[n − 1] − δ[n − 2] + δ[n − 3]) . 2 (c) Given that {g0 [n], g1 [n]} is an orthogonal pair implies that gi [n], gi [n − 2k] = δ[n − k] , i = 0, 1 ∀k ∈ Z g0 [n], g1 [n − 2k] = 0 In z-domain, this implies we can write
Pi (z) = Gi (z)Gi (z −1 ) R(z)
= 1 + zPi1 (z 2 ) , i = 0, 1
= G0 (z)G1 (z −1 ) = zR1 (z 2 ) ,
where Pi1 (z) is the 1st polyphase component of Pi (z) and R1 (z) is the 1st polyphase component of R(z). The first equation says that the z-transform of the autocorrelation of gi [n] has only 1 in its 0-th polyphase component (because gi [n] is orthogonal to its even shifts). The second equation says that the z-transform of the cross-correlation of g0 [n] and g1 [n] has only nonzero coefficients in odd powers of z (because g0 [n] is orthogonal to all even shifts of g1 [n]). We need to show that for any i, j = 0, 1, 2, 3 gi [n], gj [n − 4k] = δ[i − j] , ∀k ∈ Z . (2)
(2)
For i = j, this is equivalent to showing that Gi (z)Gj (z −1 ) (the z-transform of the crosscorrelation) has zero coefficients for all powers of z 4 . There are 6 cases to be looked at, we will (2)
31
(2)
(2)
(2)
only show the orthogonality between g0 [n] and g1 [n], and the other cases are similar. The cross-correlation can be written in the z-domain as G0 (z)G1 (z −1 ) = G0 (z)G0 (z 2 )G0 (z −1 )G1 (z −2 ) (2)
(2)
= (1 + zP01 (z 2 ))(z 2 R1 (z 4 )) . It can be easily verified that this expression does not contribute to any powers of z 4 . Hence (2) (2) g0 [n], g1 [n − 4k] = 0 ∀k ∈ Z.
3.17 Define indices for full binary tree Because there is a downsampling operation following the filters, the high frequency component gets “flipped”, corresponding to a bit reversal in the numbering scheme. Hence, the numbering scheme corresponds to these steps (assuming the upper branch is always highpass, and the lower branch is lowpass): 1. At the root of the tree, number the upper branch 1, and the lower branch 0. 2. If the current node at level j is on a lower branch, then number its two downward branches of level j + 1 the same way as the branches at level j. 3. If the current node at level j is on an upper branch, then number its two downward branches of level j + 1 in the reverse way as the branches at level j. 4. If not at the end of the tree yet, then go back to step 2. If it is the end of the tree, then label each leaf with the corresponding binary sequence (with the most significant bit being the branch at at the top of the tree). Convert this binary sequence to integer and we have the desired indices. The following figure illustrates the numbering scheme for J = 3:
0
1
0
1
G5
0
G4
0
G6
1
G7
0
G2
1
G3
1
G1
0
G0
1
1
0
3.18 Iterated filters In an iterated filter bank, filters at depth i have the form Hi (z) =
i−1 l=0
l Hk(l) z 2 where k(l) = 0
or 1. For linear phase H0 (z) and H1 (z) we have Hk(l) (ejω ) = Ak(l) (ω)ejαk(l) ω where Ak(l) (ω) is a 32
real-valued function. Then,
l l = Ak(l) 2l ω ejαk(l) 2 ω Hk(l) z 2 jω
=⇒ Hi (e ) =
i−1
l Ak(l) 2l ω ejαk(l) 2 ω
l=0
= A(ω)ejαω where A(ω) =
i−1
Ak(l) 2l ω
l=0
and α =
i−1
αk(l) 2l
l=0
Therefore all of the Hi (z) are linear phase filters. For even-length filters h0 [n] and h1 [n] that are respectively symmetric and anti-symmetric, the iterated filter lengths and symmetries can be derived as follows. First, we note that if Hk (z) is even-length, so is Hk (z 2 ). Now, since the product of two even-length filters has odd length (consider the convolution), we know that all of the depth-two filters are of odd length since h0 [n] and h1 [n] are both even. Next we note that Hk (z 2 ) has the same symmetry properties as Hk (z). Then, Ha (z) = H0 (z)H0 (z 2 ) is the product of two symmetric filters, so Ha (z) is symmetric. Hb (z) = H0 (z)H1 (z 2 ) is the product of a symmetric and an anti-symmetric filter, so Hb (z) is anti-symmetric. Hc (z) = H1 (z)H0 (z 2 ) is the product of an anti-symmetric and a symmetric filter, so Hc (z) is anti-symmetric. Finally, Hd (z) = H1 (z)H1 (z 2 ) is the product of two anti-symmetric filters, so Hd (z) is symmetric.
3.19 Biorthogonality in a full tree First, we review some material and elaborate on the discussion in the text. In a 2-channel filter bank, the biorthogonality of the basis vectors is equivalent to a perfect reconstruction condition on the analysis-synthesis modulation matrices (see theorem 3.7 and the subsection of 3.2.1 on modulation domain analysis of 2-channel filterbanks): Gm (z)Hm (z) = 2I ⇐⇒ Hm (z)Gm (z) = 2I
where Gm (z) =
G0 (z) G1 (z) G0 (−z) G1 (−z)
and Hm (z) =
H0 (z) H0 (−z) H1 (z) H1 (−z)
Note the transposition of the modulates that is implicit in the definition of Gm and Hm . We derive the biorthogonality conditions using the inverse relationship of the analysis and synthesis modulation matrices. To be completely explicit, the matrix equations above represent these relations: G0 (z)H0 (z) + G1 (z)H1 (z) = 2
(1)
G0 (z)H0 (−z) + G1 (z)H1 (−z) = 0
(2)
G0 (−z)H0 (z) + G1 (−z)H1 (z) = 0
(3)
G0 (−z)H0 (−z) + G1 (−z)H1 (−z) = 2
(4)
G0 (z)H0 (z) + G0 (−z)H0 (−z) = 2
(5)
G1 (z)H0 (z) + G1 (−z)H0 (−z) = 0
(6)
G0 (z)H1 (z) + G0 (−z)H1 (−z) = 0
(7)
G1 (z)H1 (z) + G1 (−z)H1 (−z) = 2
(8)
33
where equations (1) to (4) are from Gm (z)Hm (z) = 2I and (5) to (8) are from Hm (z)Gm (z) = 2I. Note that there is some obvious redundancy in that (1) and (4) are modulates, as are (2) and (3); also, note that equations (5) through (8) can be derived from (1) through (4) as in done in section 3.2.1, which explicitly derives (5) from (1) and (2). The text also shows how applying the modulation property of the Z-transform to (5) through (8) yields the time-domain biorthogonality conditions on the filter impulse responses, or equivalently the biorthogonal expansion basis vectors. (There is a time-reversal for the synthesis filter impulse responses, though.) A full tree of depth J corresponds to an 2J -channel filterbank; for this reason we recommend problem 3.21 in conjunction with this problem. For a tree of depth 2, the filters are given by: Ha (z) Hb (z) Hc (z) Hd (z)
= = = =
The modulation matrices are ⎡ H0 (z)H0 (z 2 ) ⎢ H0 (z)H1 (z 2 ) Hm (z) = ⎢ ⎣ H1 (z)H0 (z 2 ) H1 (z)H1 (z 2 ) ⎡ G0 (z)G0 (z 2 ) ⎢ G0 (W z)G0 (−z 2 ) Gm (z) = ⎢ ⎣ G0 (−z)G0 (z 2 ) G0 (−W z)G0 (−z 2 )
H0 (z)H0 (z 2 ) H0 (z)H1 (z 2 ) H1 (z)H0 (z 2 ) H1 (z)H1 (z 2 )
Ga (z) Gb (z) Gc (z) Gd (z)
H0 (W z)H0 (−z 2 ) H0 (W z)H1 (−z 2 ) H1 (W z)H0 (−z 2 ) H1 (W z)H1 (−z 2 )
= = = =
G0 (z)G0 (z 2 ) G0 (z)G1 (z 2 ) G1 (z)G0 (z 2 ) G1 (z)G1 (z 2 )
H0 (−z)H0 (z 2 ) H0 (−z)H1 (z 2 ) H1 (−z)H0 (z 2 ) H1 (−z)H1 (z 2 )
⎤ H0 (−W z)H0 (−z 2 ) H0 (−W z)H1 (−z 2 ) ⎥ ⎥ H1 (−W z)H0 (−z 2 ) ⎦ H1 (−W z)H1 (−z 2 )
⎤ G0 (z)G1 (z 2 ) G1 (z)G0 (z 2 ) G1 (z)G1 (z 2 ) G0 (W z)G1 (−z 2 ) G1 (W z)G0 (−z 2 ) G1 (W z)G1 (−z 2 ) ⎥ ⎥ G0 (−z)G1 (z 2 ) G1 (−z)G0 (z 2 ) G1 (−z)G1 (z 2 ) ⎦ G0 (−W z)G1 (−z 2 ) G1 (−W z)G0 (−z 2 ) G1 (−W z)G1 (−z 2 )
The biorthogonality relations can be derived from the matrix equation Hm Gm = 4I; the derivation is explained for the N -channel case in the solution to problem 3.21, and is thus not repeated here. The relations are given by gp [k], hq [4n − k] = δ[p − q]δ[n] where p ∈ {a, b, c, d} and q ∈ {a, b, c, d}. The condition in simply that the analysis filter and the time-reversed synthesis filter in the same channel must be orthogonal with respect to shifts by 4; across channels, the analysis filters and time-reversed synthesis filters must be uncorrelated for all shifts by 4. The proof that biorthogonality holds for the tree-structured filterbank can be carried out by extracting a single equation from the 4 × 4 matrix equation Hm Gm = 4I and then showing that it follows from relations (1) through (8). For instance, consider the specific relation ga [k], ha [4n − k] = δ[n] that results from multiplying the first row of Hm and the first column of Gm . In the Z-domain, this corresponds to H0 (z)H0 (z 2 )G0 (z)G0 (z 2 ) + H0 (W z)H0 (−z 2 )G0 (W z)G0 (−z 2 ) + H0 (−z)H0 (z 2 )G0 (−z)G0 (z 2 ) + H0 (W z)H0 (−z 2 )G0 (−W z)G0 (−z 2 ) = 4 ⇒ G0 (z 2 )H0 (z 2 ) [G0 (z)H0 (z) + G0 (−z)H0 (−z)] + G0 (−z 2 )H0 (−z 2 ) [G0 (W z)H0 (W z) + G0 (−W z)H0 (−W z)] = 4 The bracketed expressions can be simplified using equation (5), resulting in G0 (z 2 )H0 (z 2 ) + G0 (−z 2 )H0 (−z 2 ) = 2 34
which can be seen to be true by replacing z by z 2 in equation (5). The other biorthogonality relations can be proven in a similar manner; the algebraic details are omitted in favor of outlining a more elegant proof. ⎡
Consider the matrix
1 ⎢ 0 T = ⎢ ⎣ 0 0
0 0 1 0
0 1 0 0
⎤ 0 0 ⎥ ⎥ 0 ⎦ 1
Left multiplication by T interchanges the second and third rows; right multiplication by T interchanges the second and third columns. Note that T2 = I (and also that T = T−1 = TT ). To prove biorthogonality for the full tree, we just need to prove that Hm Gm = 4I as explained earlier. This is equivalent to proving that (THm T)(TGm T) = 4I. We can write the matrices in parentheses in block form as H0 (z 2 )Hm2 (z) H0 (−z 2 )Hm2 (W z) Hm = THm T = H1 (z 2 )Hm2 (z) H1 (−z 2 )Hm2 (W z) Gm = TGm T =
G0 (z 2 )Gm2 (z) G1 (z 2 )Gm2 (z) 2 G0 (−z )Gm2 (W z) G1 (−z 2 )Gm2 (W z)
where Gm2 and Hm2 denote the 2 × 2 modulation matrices. Then, G0 (z 2 )H0 (z 2 )Gm2 (z)Hm2 (z) + G0 (−z 2 )H0 (−z 2 )Gm2 (W z)Hm2 (W z) Hm Gm = G0 (z 2 )H1 (z 2 )Gm2 (z)Hm2 (z) + G0 (−z 2 )H1 (−z 2 )Gm2 (W z)Hm2 (W z) G1 (z 2 )H0 (z 2 )Gm2 (z)Hm2 (z) + G1 (−z 2 )H0 (−z 2 )Gm2 (W z)Hm2 (W z) G1 (z 2 )H1 (z 2 )Gm2 (z)Hm2 (z) + G1 (−z 2 )H1 (−z 2 )Gm2 (W z)Hm2 (W z) . . 2 -G0 (z 2 )H0 (z 2 ) + G0 (−z 2 )H0 (−z 2 ). I2 2 -G1 (z 2 )H0 (z 2 ) + G1 (−z 2 )H0 (−z 2 ). I2 = 2 G0 (z 2 )H1 (z 2 ) + G0 (−z 2 )H1 (−z 2 ) I2 2 G1 (z 2 )H1 (z 2 ) + G1 (−z 2 )H1 (−z 2 ) I2 4I 0 = = 4I 0 4I
where the various identity matrices are of the appropriate size. This completes the proof that biorthogonality holds in a full tree of depth 2 when the base filters are biorthogonal. This argument can of course be iterated to arbitrary depths.
3.20 Closed-form formula for number of wavelet packets
Let MJ−1 be the number of orthonormal bases in a binary tree of depth J − 1. To generate a binary tree of depth J, we take a binary filter bank (of depth 1) to be the root, and attach to its two leaves any of the MJ−1 possible orthonormal bases (from the tree of depth J − 1). There are 2 possible combinations. With the addition of the nil-tree tree (i.e. with no split at all), the MJ−1 number of orthonormal bases in a binary tree of depth J is 2 +1 . MJ = MJ−1
To start the recursion, M1 = 2 (corresponding to the 1-level binary tree and the identity tree).
35
3.21 Biorthogonality in N-channel perfect reconstruction filter banks In the modulation domain, the operation of an N-channel analysis-synthesis filterbank can be expressed as ⎡ ⎤ ⎤ ⎡ ˆ X(z) X(z) ⎢ X(W ⎢ X(W z) ⎥ ˆ z) ⎥ 1 ⎢ ⎥ ⎥ ⎢ H = G ⎢ ⎥ ⎥ .. m m⎢ .. ⎣ ⎦ ⎦ N ⎣ . . N −1 N −1 ˆ z) X(W X(W z) where W = ej2π/N and the matrices Gm and Hm are given by ⎡ G0 (z) G1 (z) G2 (z) ··· GN −1 (z) ⎢ G0 (W z) G (W z) G (W z) · · · G 1 2 N −1 (W z) ⎢ Gm = ⎢ .. .. .. .. .. ⎣ . . . . . G0 (W N −1 z) G1 (W N −1 z) G2 (W N −1 z) · · · GN −1 (W N −1 z) ⎡ Hm
⎢ ⎢ = ⎢ ⎣
H0 (z) H0 (W z) .. .
H1 (z) H1 (W z) .. .
H2 (z) H2 (W z) .. .
··· ··· .. .
HN −1 (z) HN −1 (W z) .. .
⎤ ⎥ ⎥ ⎥ ⎦ ⎤T ⎥ ⎥ ⎥ ⎦
H0 (W N −1 z) H1 (W N −1 z) H2 (W N −1 z) · · · HN −1 (W N −1 z) Note the transpose in this definition of Hm . This is an N -channel formulation of equation 3.2.14 that has been extended to include the modulated versions of X(z). The perfect reconstruction condition is Gm Hm = N I ⇐⇒ Hm Gm = N I where I is the N × N identity matrix. The order of the matrices can be reversed in this way because the left inverse and the right inverse are equivalent for square matrices (Gm and Hm are both N × N ). Consider the inner product of the a-th row of Hm and the b-th column of Gm , where the indices a and b range from 0 to N − 1: Ha (z)Gb (z) + Ha (W z)Gb (W z) + . . . + Ha (W N −1 z)Gb (W N −1 z) = N I(a, b) where I(a, b) denotes the element in row a and column b of the identity matrix, namely δ[a − b]. This can be rewritten as a time-domain convolution using the modulation property: gb [k]ha [n − k](1 + W n + W 2n + · · · + W (N −1)n ) = N δ[a − b]δ[n] k
=⇒
gb [k]ha [N n − k] = δ[a − b]δ[n] =⇒ gb [k], ha [N n − k] = δ[a − b]δ[n]
k
In terms of basis vectors, this is equivalent to the biorthogonality conditions ϕ˜b [k], ϕN n+a [k] = δ[a − b]δ[n] which is simply an N -channel generalization of the conditions given at the end of the 2-channel modulation domain analysis section in the text (a subsection of 3.2.1).
36
3.22 Relationship between polyphase and modulation matrices
The expression relating Gp (z) and Gm (z) is similar to equations (3.4.9) ⎤ ⎡ G00 (z N ) G10 (z N ) ··· G(N −1)0 (z N ) ⎢ G01 (z N ) G11 (z N ) ··· G(N −1)1 (z N ) ⎥ ⎥ ⎢ Gp (z N ) = ⎢ ⎥ .. .. .. .. ⎦ ⎣ . . . . N N N G0(N −1) (z ) G1(N −1) (z ) · · · G(N −1)(N −1) (z ) ⎡ ⎤ ⎡ G1 (z) G0 (z) 1 0 ··· 0 ⎢ ⎥ ⎢ G1 (WN z) 0 ⎥ ⎢ G0 (WN z) 1 ⎢ 0 z ··· = ⎢ .. .. . . .. .. .. ⎥ F ⎢ N⎣ . . . . . . ⎦ ⎣ 0 0 · · · z N −1
%
··· ··· .. .
GN −1 (z) GN −1 (WN z) .. .
⎥ ⎥ ⎥ ⎦ ⎡ ⎢ ⎢ ⎢ ⎣
1 0 ··· 0 −1 0 z ··· 0 .. .. .. .. . . . . 0 0 · · · z −(N −1)
In both equations, F is the N × N Fourier transform matrix, with Fkl = WNkl = e−j(2π/N )kl .
3.23 Modulation matrices
where W = e−j
⎥ ⎥ ⎥ ⎦
(3.2.28): ⎤
H(N −1)0 (z N ) H(N −1)1 (z N ) · · · H(N −1)(N −1) (z N ) ⎤ ⎡ H0 (z) H0 (WN z) ··· H0 (WNN −1 z) H1 (WN z) ··· H1 (WNN −1 z) ⎥ 1 ⎢ ⎥ ⎢ H1 (z) ⎥F ⎢ .. .. .. .. N⎣ ⎦ . . . . N −1 HN −1 (z) HN −1 (WN z) · · · HN −1 (WN z) &' ( % H m (z)
(a) The modulation matrix ⎡ H0 (z) Hm (z) = ⎣ H1 (z) H2 (z)
⎤
G0 (WNN −1 z) G1 (WNN −1 z) · · · GN −1 (WNN −1 z) &' ( Gm (z)
The expression relating H p (z) and H m (z) is similar to equation ⎡ H00 (z N ) H01 (z N ) ··· H0(N −1) (z N ) ⎢ H10 (z N ) H11 (z N ) ··· H1(N −1) (z N ) ⎢ H p (z N ) = ⎢ .. .. .. .. ⎣ . . . .
=
and (3.2.29):
Hm (z) is given by
⎤ ⎡ ⎤ H(z) H(W z) H(W 2 z) H0 (W z) H0 (W 2 z) ⎦ H1 (W z) H1 (W 2 z) ⎦ = ⎣ H(W z) H(W 2 z) H(z) 2 2 H2 (W z) H2 (W z) H(W z) H(W z) H(z)
2π 3
and Hi (z) = H(W i z). From Theorem 3.17, we have ⎤ ⎡ ⎡ ⎤ 1 1 1 Hp0 (z) 0 0 ⎦ where F = ⎣ 1 W W 2 ⎦ 0 0 Hp1 (z) Hp (z) = F ⎣ 1 W2 W4 0 0 Hp3 (z)
and from equation 3.4.9 and problem 3.22, Hm (z) = Hp (z 3 ) diag(1, z, z 2 ) F Substituting for Hp (z 3 ) gives
Hm (z) = F diag Hp0 (z 3 ), zHp1 (z 3 ), z 2 Hp2 (z 3 ) F 37
⎤ ⎥ ⎥ ⎥ ⎦
which yields the desired diagonalization diag Hp0 (z 3 ), zHp1 (z 3 ), z 2 Hp2 (z 3 ) = F −1 Hm (z) F −1 (b) Using the factorization from (a), det (Hm (z)) = det F diag Hp0 (z 3 ), zHp1 (z 3 ), z 2 Hp2 (z 3 ) F = [det(F )]2 det diag Hp0 (z 3 ), zHp1 (z 3 ), z 2 Hp2 (z 3 ) Computing det(F ) and simplifying gives det (Hm (z)) = −27 z 3 Hp0 (z 3 ) Hp1 (z 3 ) Hp2 (z 3 ) (c) Since det (Hp (z)) = det(F ) Hp0 (z) Hp1 (z) Hp2 (z) we can use the result of (b) to write det (Hp (z)) = z 3 det(F ) det Hp (z 3 ) Note: the special form of Hp (z) from Theorem 3.17 was used in (a) to derive the diagonalization of Hm (z).
3.24 Cosine modulated filter banks
2k + 1 1 (2n − N + 1)π Note that the window (a) The filters are given by hk [n] = √ cos 4N N function hpr [n] is rectangular and thus A0 = B0 and A1 = B1 . The matrices A0 and A1 are given by ⎤ ⎤ ⎡ ⎡ h0 [2N − 1] h0 [N − 1] ··· h0 [N ] ··· h0 [0] ⎥ ⎥ ⎢ ⎢ .. .. .. .. A0 = ⎣ ⎦ and A1 = ⎣ ⎦ . . . . hN −1 [2N − 1] · · · hN −1 [N ]
hN −1 [N − 1] · · · hN −1 [0]
We first prove the second part of 3.4.6, namely A0 AT1 = A1 AT0 = 0. The right tail (A1 ) is symmetric: 2k + 1 1 (N − 2n − 1)π = hk [n] hk [N − 1 − n] = √ cos 4N N The left tail (A0 ) is anti-symmetric: hk [2N − 1 − n] = hk [N + n] =
2k + 1 1 √ cos (−2n + 3N − 1)π 4N N 2k + 1 1 √ cos (2n + N + 1)π 4N N
because
2k + 1 2k + 1 (−2n + 3N − 1)π + (2n + N + 1)π = (2k + 1)π 4N 4N and for any two angles φ1 , φ2 that sum to an odd multiple of π, we have φ1 + φ2 = (2k + 1)π =⇒ φ1 = (2k + 1)π − φ2 =⇒ cos φ1 = cos [(2k + 1)π − φ2 ] = − cos φ2 38
Because the tails are symmetric/antisymmetric, their inner product is 0; hence we have A0 AT1 = A1 AT0 = 0. We now prove the first part of 3.4.6, namely that AT0 A1 = AT1 A0 = 0. The inner product of the n-th row of AT0 and the m-th column of A1 is N −1 2k + 1 2k + 1 1 (2n − N + 1)π cos (2m − N + 1)π cos N 4N 4N k=0
1 1 Using the identity cos X cos Y = cos(X + Y ) + cos(X − Y ) the inner product can be rewritten 2 2 as N −1 2k + 1 2k + 1 1 (n + m − N + 1)π + cos (n − m)π cos 2N 2N 2N k=0
We can simplify this using the following derivation N −1
=
π π cos kx + x N 2N
k=0 N −1
1 2
π
π
π
π
ejkx N ejx 2N + e−jkx N e−jx 2N
k=0
1 jx π 1 − ejxπ 1 −jx π 1 − e−jxπ 2N e 2N = + e π π 2 2 1 − ejx N 1 − e−jx N xπ sin xπ sin(xπ) 2 = cos = xπ xπ sin 2N 2 2 sin 2N We can thus write the inner product as / sin((n − m)π) sin((n + m − N + 1)π) 1 + +1)π 2N 2 sin (n−m)π 2 sin (n+m−N 2N 2N In the inner product, n ∈ [N, 2N − 1] and m ∈ [0, N − 1], so the respective arguments of the sinc functions are in the ranges: n − m ∈ [1, 2N − 1]
n + m − N + 1 ∈ [1, 2N − 1]
For these argument values, the sinc is always zero, so we have proven that AT0 A1 = AT1 A0 = 0. We now prove the second part of 3.4.5, that A0 AT0 + A1 AT1 = I. Note that the [k, j]-th constituent equation of this matrix formula corresponds to the inner product of the length 2N filters hk [n] and hj [n]; proving the formula is equivalent to proving that hk [n], hj [n] = δ[k − j]. The inner product is given by
=
2N −1 2j + 1 2k + 1 1 (2n − N + 1)π cos (2n − N + 1)π cos N 4N 4N k=0 2N −1 N −1 π N −1 π 1 + cos (k − j) n − cos (k + j + 1) n − 2N 2 N 2 N k=0
= δ[k − j]
39
where the final step holds because the sum is taken over an integral number of periods of the cosines, so we have shown that A0 AT0 + A1 AT1 = I. We prove the first part of 3.4.5 in (b) since it follows naturally from the results derived there. (b) Consider B0T B0 = AT0 A0 ; the derivation used to find AT0 A1 can be applied here. Specifically, we have / sin((n − m)π) 1 sin((n + m − N + 1)π) AT0 A0 [n, m] = + 2N 2 sin (n−m)π 2 sin (n+m−N +1)π 2N
2N
with the arguments now in the ranges n ∈ [N, 2N − 1] and likewise m ∈ [N, 2N − 1] so n − m ∈ [0, N − 1]
n + m − N + 1 ∈ [N + 1, 3N − 1]
For n − m = 0, the first sinc takes on the value N by L’Hˆ opital’s rule; otherwise it is zero-valued. For n + m − N + 1 = 2N , the second sinc takes on the value −2N ; for other values it is zero. Since n−m = 0 corresponds to the matrix diagonal and n+m = 3N −1 corresponds to the anti-diagonal, 1 we have AT0 A0 = (I − J). For the product AT1 A1 , the arguments are in the ranges n ∈ [0, N − 1] 2 and m ∈ [0, N − 1], so n − m ∈ [0, N − 1]
n + m − N + 1 ∈ [−N + 1, N − 1]
The sinc functions in the sum are respectively non-zero for n − m = 0 and n + m − N + 1 = 0, which correspond to the diagonal n = m and the anti-diagonal n + m = N − 1. In both cases 1 the sinc functions take on the value 2N , so the result of the matrix product is AT1 A1 = (I + J). 2 Combining these results gives the first part of 3.4.5 for problem (a), namely AT0 A0 + AT1 A1 =
1 (I − J + I + J) = I 2
3.25 Orthogonal pyramid (a) For this problem we refer to figure 3.17; we denote the “original signal” (space V0 ) as x[n], the “coarse version” as y0 [n], the “difference signal” (space W1 ) as y1 [n], and the signal in space ˜ 0 to yield the V1 as y0 [n]. In synthesis, the coarse signal y0 [n] is upsampled and filtered with H signal y0 [n]. The synthesis output y[n] is the sum of y0 [n] and y1 [n]. In time-domain, perfect reconstruction is indicated by: ⎫ y1 [n] = x[n] − y0 [n] ⎬ =⇒ y[n] = x[n] y[n] = y0 [n] + y1 [n] ⎭ y0 [n] y0 [n] = Analogously, in the Z-transform domain perfect reconstruction is given by: ⎫ Y1 (z) = X(z) − Y0 (z) ⎬ Y (z) = Y0 (z) + Y1 (z) =⇒ Y (z) = X(z) ⎭ Y0 (z) Y0 (z) =
40
(b) Let H0 and H1 denote the filter matrices ⎡ ⎤ ⎡ .. .. . . ⎢ ⎥ ⎢ ⎢ ⎥ ⎢ 11 0 1 −1 0 ⎥ ⎢ 1 ⎢ ⎥ and H1 = √1 ⎢ 1 1 1 −1 H0 = √ ⎢ ⎥ 2⎢ 2⎢ ⎢ ⎥ ⎢ 0 1 1 0 1 −1 ⎣ ⎦ ⎣ .. .. . .
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
1 1 and let h0 [n] = √ [1 1] and h1 [n] = √ [1 −1]. The following matrix relations hold: 2 2 H1 H0T = 0 and H0T H0 + H1T H1 = I. Also, ⎫ y0 = H0 x ⎬ y0 = H0T H0 x =⇒ y1 = (I − H0T H0 )x ⎭ x − y0 y1 = =⇒ y1 = H1 y1 = (H1 − H1 H0T H0 )x =⇒ y1 = H1 x Reconstruction is achieved by y = H0T y0 + H1T y1 = H0T H0 x + H1T H1 x = x (c) We showed in (b) that y0 = H0 x and y1 = H1 x, which directly indicates the equivalence to a two-channel filter bank with filters h0 [n] and h1 [n] (with synthesis filters being the time-reversed version of h0 [n] and h1 [n]). (d) If we define H and G to be the infinite matrices ⎡ h(L − 1) h(L − 2) h(L − 3) h(L − 4) H = ⎣ h(L − 1) h(L − 2) and
⎡
g(L − 1) g(L − 2) g(L − 3) g(L − 4) G = ⎣ g(L − 1) g(L − 2)
⎤ ··· h(0) ⎦ ··· h(2) h(1) h(0) h(L − 1) · · · · · · h(2) h(1) h(0) ⎤ ··· g(0) ⎦ ··· g(2) g(1) g(0) g(L − 1) · · · · · · g(2) g(1) g(0)
By orthogonality relations, we have HH T = I
GGT = I
GH T = 0
HGT = 0
We thus have y0 = Hx and y1 = G(I − H T H)x =⇒ y1 = Gx − GH T Hx =⇒ y1 = Gx Analysis in the Z-transform domain yields $ 1# H(z 1/2 )X(z 1/2 ) + H(−z 1/2 )X(−z 1/2 ) Y0 (z) = 2 $ 1# G(z 1/2 )X(z 1/2 ) + G(−z 1/2 )X(−z 1/2 ) Y1 (z) = 2 41
where G(z) = −z −(L−1) H(−z −1 ). Upsampling and filtering gives the reconstruction $ 1# Hr (z)H(z) − z −(L−1) Gr (z)H(−z −1 ) X(z) Xr (z) = 2 $ 1# Hr (z)H(−z) + z −(L−1) Gr (z)H(z −1 ) X(−z) + 2 where Gr (z) and Hr (z) are the synthesis filters. If we choose Gr (z) = −H(−z) and Hr (z) = z −(L−1) H(z −1 ) we obtain Xr (z) = z −(L−1) H(z)H(z −1 ) + H(−z)H(−z −1 ) X(z) Since H(z)H(z −1 ) corresponds to the autocorrelation of a filter which is orthogonal to its even translates (and has unit norm), it contains only odd powers of z except for a 0-th order term of value 1. (Consider, for example, problem 2.16c.) We therefore have perfect reconstruction: H(z)H(z −1 ) + H(−z)H(−z −1 ) = 2 =⇒ Xr (z) = 2z −(L−1) X(z)
3.26 Parseval’s formula for tight frames
In the case of tight frame, H0 (z) = G0 (z −1 ) and H1 (z) = G1 (z −1 ), and thus the analysis filters satisfy H0 (z)H0 (z −1 ) + H1 (z)H1 (z −1 ) = 2 . y0 + y1 2
2
π π 1 1 jω jω 2 = |H0 (e )X(e )| dω + |H1 (ejω )X(ejω )|2 dω 2π −π 2π −π π 1 (|H0 (ejω )|2 + |H1 (ejω )|2 ) |X(ejω )|2 dω = &' ( 2π −π % 2 π 1 |X(ejω )|2 dω) = 2( 2π −π = 2 x2
3.27 Quincunx orthonormal bases for l2 (Z 2 ) To prove that the impulse responses with shifts on a quincunx lattice form an orthonormal basis, we need to show that they form an orthonormal system and that the system is complete. We do these in turn. Equation 3.6.4 states H0 (z1 , z2 )H0 (z1−1 , z2−1 ) + H0 (−z1 , −z2 )H0 (−z1−1 , −z2−1 ) = 2 We have the following relations for two-dimensional Z-transforms: H0 (z1 , z2 ) H0 (−z1 , −z2 ) H0 (z1−1 , z2−1 ) H0 (−z1−1 , −z2−1 )
⇐⇒ h[n1 , n2 ] ⇐⇒ (−1)n1 (−1)n2 h[n1 , n2 ] ⇐⇒ h[−n1 , −n2 ] ⇐⇒ (−1)n1 (−1)n2 h[−n1 , −n2 ]
We can rewrite 3.6.4 in the time-domain as h0 [m1 , m2 ]h0 [m1 − n1 , m2 − n2 ] [1 + (−1)n1 (−1)n2 ] = 2δ[n1 , n2 ] m1 m2
42
h0 [m1 , m2 ]h0 [m1 − n1 , m2 − (2k − n1 )] = δ[n1 , (2k − n1 )]
m1 m2
=⇒ h0 [m1 , m2 ], h0 [m1 − n1 , m2 − (2k − n1 )] = δ[n1 , (2k − n1 )] Equation 3.6.5 states
H1 (z1 , z2 ) = −z1−1 H0 (−z1−1 , −z2−1 )
from which we can easily derive the relations H0 (z1 , z2 ) H0 (z1−1 , z2−1 ) H0 (−z1 , −z2 ) H0 (−z1−1 , −z2−1 )
= z1−1 H1 (−z1−1 , −z2−1 ) = z1 H1 (−z1 , −z2 ) = −z1−1 H1 (z1−1 , z2−1 ) = −z1 H1 (z1 , z2 )
Substituting into 3.6.4 yields H1 (z1 , z2 )H1 (z1−1 , z2−1 ) + H1 (−z1 , −z2 )H1 (−z1−1 , −z2−1 ) = 2 which verifies that h1 [m1 , m2 ], h1 [m1 − n1 , m2 − (2k − n1 )] = δ[n1 , (2k − n1 )] So we have proven that h0 [n1 , n2 ] and h1 [n1 , n2 ] are orthogonal with respect to their own shifts on a quincunx lattice; to complete the proof that these constitute an orthonormal system, we require that h0 [m1 , m2 ], h1 [m1 − n1 , m2 − (2k − n1 )] = 0 which can be rewritten in the Z domain as H0 (z1 , z2 )H1 (z1−1 , z2−1 ) + H0 (−z1 , −z2 )H1 (−z1−1 , −z2−1 ) = 0 Using the relations between the lowpass and highpass filter, we have H0 (z1 , z2 ) {−z1 H0 (−z1 , −z2 )} + H0 (−z1 , −z2 ) {z1 H0 (z1 , z2 )} = 0 which completes the proof. This orthonormal system is a basis for l2 (Z 2 ) if hi , y = 0 for i = 0, 1 =⇒ y = 0 In the time-domain, the zero inner product condition is equivalent to (assuming real filters and signals) y[m1 , m2 ]hi [m1 , m2 − (2k − n1 )] = 0 m1 m2
which can be expressed in the Z domain as (see the above formulation) Y (z1 , z2 )H0 (z1−1 , z2−1 ) + Y (−z1 , −z2 )H0 (−z1−1 , −z2−1 ) = 0 Y (z1 , z2 )H1 (z1−1 , z2−1 ) + Y (−z1 , −z2 )H1 (−z1−1 , −z2−1 ) = 0 Using the lowpass-highpass relations, we have Y (z1 , z2 )H0 (z1−1 , z2−1 ) + Y (−z1 , −z2 )H0 (−z1−1 , −z2−1 ) = 0 Y (z1 , z2 )H0 (−z1 , −z2 ) − Y (−z1 , −z2 )H0 (z1 , z2 ) = 0 43
Multiplying the top equation by H0 (z1 , z2 ) and the bottom equation by H0 (−z1−1 , −z2−1 ) and adding the resulting equations gives Y (z1 , z2 ) H0 (z1 , z2 )H0 (z1−1 , z2−1 ) + H0 (−z1 , −z2 )H0 (−z1−1 , −z2−1 ) = 0 =⇒ Y (z1 , z2 )[2] = 0 =⇒ Y (z1 , z2 ) = 0 =⇒ y[n1 , n2 ] = 0 which completes the proof that the impulse responses with shifts on a quincunx lattice are an orthonormal basis for l2 (Z 2 ).
3.28 Linear phase diamond-shaped quincunx filters (a) Note that the origin is located at the center of the leftmost column for both h0 [n1 , n2 ] and h1 [n1 , n2 ]. H0 (z1 , z2 ) = (1 + bz1−1 z2−1 + bz1−1 z2 + z1−2 ) + z1−1 &' ( % H00 (z1 ,z2 )
a %&'( H01 (z1 ,z2 )
H1 (z1 , z2 ) = c bc ( (1 + z1−4 ) + (b + )(z1−1 z2−1 + z1−1 z2 + z1−3 z2−1 + z1−3 z2 ) + (z1−2 z2−2 + z12 z22 ) + dz1−2 ) a &' ( %a H10 (z1 ,z2 )
+
z1−1 (c(1 %
+
z1−2 ) +
a(z1−1 z2−1 &'
+ z1 z2 )) (
H11 (z1 ,z2 )
For perfect FIR reconstruction to be possible, the determinant of the polyphase matrix has to be a monomial. det H p = H00 (z)H11 (z) − H01 (z)H10 (z)
= (c − bc)(1 + z1−4 ) + (−a + ab)(z1−2 z2−2 + z1−2 z22 )
+(a − ab − c + bc)(z1−3 z2−1 + z1−1 z2−1 + z1−3 z2 + z1−1 z2 )
+(2ab + 2c − ad)z1−2
For det H p to be a monomial, the first 3 terms must be zero, while the last term cannot be zero: c − bc = 0 −a + ab = 0 a − ab − c + bc = 0 2ab + 2c − ad = 0 The first two constraints imply b = 1, and it is clear that a, c, d can then be anything as long as they satisfy 2a + 2c − ad = 0. Hence, perfect FIR reconstruction is possible. (b) If circular symmetry is also imposed, then b = 1 and c = a. For a = −4 and d = −28, we have ⎛ ⎞ 0 1 0 h0 [n1 , n2 ] = ⎝ 1 −4 1 ⎠ , 0 1 0 44
⎛ ⎜ ⎜ h1 [n1 , n2 ] = ⎜ ⎜ ⎝
0 0 1 0 0 0 2 −4 2 0 1 −4 −28 −4 1 0 2 −4 2 0 0 0 1 0 0
⎞ ⎟ ⎟ ⎟ . ⎟ ⎠
This set of filters yields perfect FIR reconstruction since det H p = −128z1−2 . The filter h0 [n1 , n2 ] is a highpass filter because it takes the difference of a sample with its 4 neighbors; the filter h1 [n1 , n2 ] is a lowpass filter because it resembles the shape of a sinc (lowpass) filter.
45
Chapter 4 4.1 Wavelet series expansion (a) The wavelet series expansion of a function f (t) is given by am,n ψm,n (t) f (t) = m,n −m 2
where ψm,n (t) = 2 ψ(2−m t − n) and am,n = f (t), ψm,n (t). For the Haar expansion of f (t) = ϕ(t) (the scaling function), the series coefficients are: −m 1 2 2 m≥1 n=0 am,n = ψm,n (t) dt = 0 otherwise 0 (b)
m
|ψm,n (t), f (t)|
2
=
n
m
2
|am,n |
∞
=
n
2−m = 1
m=1
(c) All coefficients of some scale m are equal to zero if and only if 2−i = k 2m for some integer k. Thus all coefficients are equal to zero for scales m such that 2−i−m = k for some integer k. If −i − m < 0, the power 2−i−m will not be an integer. Therefore, not all of the coefficients are zero for scales m > −i. (d) All coefficients of some scale m are equal to zero if √12 = k 2m for some integer k. Since there is no integer m for which this holds, there is no scale at which all of the coefficients are equal to zero.
4.2 Implications of the two-scale equation for an orthonormal basis
(a) First note that φ(t) 2 = 1 because {φ(t − n)} is an orthonormal set. φ(t) 2 = φ(t), φ(t) √ √ g0 [n]φ(2t − n), 2 g0 [m]φ(2t − m) = 2 n
=
n
=
√
m
√ 2φ(2t − n), 2φ(2t − m) &' ( %
g0∗ [n]g0 [m]
m
δ[m−n]
|g0 [n]|2
n
= g0 [n] 2 = 1 Hence g0 [n] = 1.
√ (b) Since {φ(t − n)} is an orthonormal basis for V0 , { 2φ(2t − n)} is an orthonormal basis for V−1 . ∞ √ √ √ 2φ(2t − n), φ(t) = 2φ(2t − n), 2 g0 [m]φ(2t − m) m=−∞
=
∞ m=−∞
√ √ g0 [m] 2φ(2t − n), 2φ(2t − m) &' ( % δ[m−n]
= g0 [n] 46
√ The last equality comes from the fact that { 2φ(2t − n)} is an orthonormal set.
4.3 More implications of the two-scale equation (a) Taking the Fourier transform of the two scale equation, we get (see equation (4.2.9)) ω ω 1 Φ(ω) = √ G0 (ej 2 )Φ( ) . 2 2
Evaluating at ω = 0, Φ(0) = √12 G0 (1)Φ(0) √
=⇒ G0 (1) = n g0 [n] = 2 .
=⇒
(b) From equation (4.2.10), |G(ejω )|2 + |G0 (ej(ω+π) )|2 = 2 . Evaluating at ω = 0, |G(1)|2 + |G0 (−1)|2 = 2 , implying that G0 (−1) = 0. Since (−1)n g0 [n] = g0 [2n] − g0 [2n + 1] , G0 (−1) = n
this proves that
n
g0 [2n] =
n
n
g0 [2n + 1] .
n
4.4 Meyer’s wavelet It is sufficient to prove that |Ψ(2ω)|2 + |Ψ(ω)|2 = 1 for
2π 3
≤ω≤
4π 3 .
Substituting from (4.3.5) it follows that we have to prove |Φ(ω − 2π)|2 + |Φ(ω)|2 = 1.
The proof of this last equality is already given following (4.3.3).
4.5 Meyer’s wavelet (a) The scaling function and wavelet for this choice of θ differ in the transition from 0 to 1 and vice versa. With the choice of (4.3.2) this transition is differentiable, making the scaling function and wavelet differentiable. With the choice of θ as in this exercise, the scaling function and wavelet are not differentiable in 2πk 3 for |k| = 1, 2 and |k| = 1, 2, 3, respectively. This is easily checked by considering the following explicit formulas for the scaling function and wavelet: ⎧ 0, |ω| ≥ 4π ⎪ 3 ⎪ ⎨ 1, |ω| ≤ 2π 3 Φ(ω) = 4π ⎪ −|ω| 4π ⎪ 3 ⎩ , 2π 2π 3 ≤ |ω| ≤ 3 3
47
⎧ 0, ⎪ ⎪ ⎪ ⎪ 0, ⎪ ⎪ ⎨ Ψ(ω) =
|ω| ≥ |ω| ≤ 8π −|ω| 3 4π 3
, ⎪ ⎪ ⎪ ⎪ 2π ⎪ ⎪ 3 ⎩ |ω|− , 2π 3
8π 3 2π 3
4π 3
≤ |ω| ≤
8π 3
2π 3
≤ |ω| ≤
4π 3
(b) As mentioned above, the scaling function and wavelet are only continuous in the frequency domain. We may therefore only expect them to be of order O( 1t ) in the spatial domain. In the case of θ(x) given by (4.3.2), we may expect an order of O( t12 ). (c) Using the following simple (though slow) Mathematica program, the plots of the scaling function and wavelet are generated. ClearAll; theta[x_] := 0 /; (x<=0); theta[x_] := 1 /; (x >= 1); theta[x_] := x /; ((0 < x) && (x < 1)); (* The scaling function in the frequency domain *) Phi[w_] := Sqrt[theta[2 + ((3 w) / (2 N[Pi]))]]/; (w <= 0); Phi[w_] := Sqrt[theta[2 - ((3 w) / (2 N[Pi]))]]/; (w > 0); Plot[Phi[w],{w,-10,10}, AxesLabel -> {"Frequency","Magnitude Response"}]; (* The scaling function in the spatial domain *) A = 2 Pi/3; B = 2 Pi; DL = Table[n/5,{n,-5*5,5*5}]; I1[t_] := (2/B)NIntegrate[Sqrt[((2 A-w)/A)] Cos[w t],{w,A,2 A}]; I2[t_] := (2/B)NIntegrate[Cos[w t],{w,0,A}]; phi[t_] := I1[t]+I2[t]; PD = MapThread[List,{DL,Map[phi,DL]}]; ListPlot[PD,PlotJoined->True,PlotRange ->{-0.2,1.2}, AxesLabel -> {"Time","Amplitude"}]; (* The wavelet in the frequency domain *) Psi[w_] = Exp[-I w/2] (Phi[w + 2 Pi] + Phi[w - 2 Pi]) Phi[w/2]; Plot[Abs[Psi[w]],{w,-10,10}, AxesLabel -> {"Frequency","Magnitude Response"}]; (* The wavelet in the spatial domain *) I3[t_] := (2/B)NIntegrate[Cos[w/2 t] Sqrt[(4 A - w)/(2 A)],{w,2A,4A}]; I4[t_] := (2/B)NIntegrate[Cos[w/2 t] Sqrt[(w - A)/A],{w,A,2A}]; psi[t_] := I3[t]+I4[t]; PD = MapThread[List,{DL,Map[psi,DL]}]; ListPlot[PD,PlotJoined->True,PlotRange ->{-1.0,1.4}, AxesLabel -> {"Time","Amplitude"}];
48
4.6 B-splines
(a) Since β (N ) (t) is an even function, β (2N +1) (t) = β (N ) (t) ∗ β (N ) (t) = β (N ) (t) ∗ β (N ) (−t) . Thus, taking the Fourier transform yields B (2N +1) (ω) = (B (N ) (ω))2 = |B (N ) (ω)|2 . In addition, given b(2N +1) [n] = β (2N +1) (t)|t=n , sampling β (2N +1) (t) at t = kT, k ∈ Z yields B (2N +1) (ω) =
∞ 1 (2N +1) 2π B (ω + k ) T T k=−∞ ∞ (2N +1)
B
=
=
k=−∞ ∞
(ω + 2πk)
|B (N ) (ω + 2πk)|2
k=−∞
where the second equality comes from letting T = 1. (b) Since β (2N +1) (t) = β (N ) (t) ∗ β (N ) (t) = β (N ) (t) ∗ β (N ) (−t), b(2N +1) [n] = β (2N +1) (t)|t=n = β (N ) (t) ∗ β (N ) (−t)|t=n ∞ = β (N ) (τ ) ∗ β (N ) (τ − t)dτ |t=n −∞ ∞ = β (N ) (τ ) ∗ β (N ) (τ − n)dτ . −∞
(c) Using the equation in part (b), we can calculate b(2N +1) [n] for N = 1, 3. β (1) (t) and β (2) (t) are shown below.
(2)
(1)
β ( t)
3/4
1
−1
1
t −3/2
The expressions for β (1) (t) and β (2) (t) are 1 − |t| , 0 ≤ |t| ≤ 1 (1) β (t) = 0, otherwise , ⎧ 3 0 ≤ |t| ≤ 12 ⎨ 4 − t2 , 9 − 3 |t| + 12 t2 , 12 ≤ |t| ≤ 32 β (2) (t) = ⎩ 8 2 0, otherwise. 49
β ( t)
3/2
t
Using the expression in (b), it is straight forward to show that b(3) [0] = b(5) [0] =
2 1 , b(3) [1] = b(3) [−1] = , 3 6
11 13 1 , b(5) [1] = b(5) [−1] = , b(5) [2] = b(5) [−2] = . 20 60 120
4.7 Battle-Lemari´e wavelets
According to Section 4.3, it suffices to calculate the functions B (2) (ω), B (5) (ω) and M (ω). However, the first two of these are already explicitly given in (4.3.26) and (4.3.27). The only remaining thing to do is calculate M (ω). From the two-scale equation I (0) (t) = I (0) (t − 1/2) + I (0) (2t + 1/2), we derive that I (0) (ω) = cos(ω/4)I (0) (ω/2). Therefore B (2) (ω) = e−jω/2 I (2) (ω) = e−jω/2 cos3 (ω/4)I (2) (ω/2) = e−jω/2 cos3 (ω/4)ejω/4 B (2) (ω/2) = e−jω/4 cos3 (ω/4)B (2) (ω/2) = M (ω/2)B (2) (ω/2). So we find M (ω) = e−jω/2 cos3 (ω/2). As in (4.3.25) we can write Ψ(ω) =
sin3 (ω/4) Q(ω), (ω/4)
where Q is 4π-periodic. In particular, ψ(t) is piecewise quadratic over half-integer intervals.
4.8 Battle-Lemari´e wavelets
B 2N +1 (ω) can be factored as follows: 1 2 1 + ejω + e−jω 3 √6 6 √ $# √ $ 2 + 3 # −jω e = + (2 − 3) ejω + (2 − 3) 6 = R(ejω ) R(e−jω ) 9 √ √ $ 2 + 3 # jω jω e + (2 − 3) =⇒ R(e ) = 6 B 2N +1 (ω) =
Note that this factorization is not unique. R(ejω ) is chosen so that Φ(ω) = this case
G(ω) =
sin ω2
2
ω 2
50
G(ω) is stable. In R(ejω )
=⇒ Φ(ω) =
sin ω2
2
ω 2
√
2+ 6
3
1 ejω + (2 −
√ 3)
√
2Φ(2ω) , so Φ(ω) √ √ 2 jω + (2 − 3) e sin 2 ω √ G0 (ejω ) = 4 sin2 ω2 ej2ω + (2 − 3) jω
From equation 4.3.17, we have G0 (e ) =
From equation 4.3.19, we have ω ω jω 1 Ψ(ω) = − √ e− 2 G∗0 ej ( 2 +π) Φ 2 2 # √ $ ω −j +π − jω −e 2 e ( 2 ) + (2 − 3) sin2 (ω) sin2 ω4 √ = √ # jω √ $ 2 4 2+6 3 e−jω + (2 − 3) e 2 + (2 − 3) ω4 sin2 ω2 We can find the set {αn } as follows: {αn } = F −1
1 R(ejω )
=
6(2 −
= F −1
⎧ ⎨ ⎩
√
2+ 6
3
⎫ ⎬
1 ejω + (2 −
√ ⎭ 3)
√ √ 3)(−2 + 3)n−1 u[n − 1]
where u[n] is the unit step function. We then have ϕ(t) =
αn β (1) (t − n) where β (1) =
n∈Z
1 − |t| |t| ≤ 1 0 otherwise
4.9 Implications of compact support The time-domain function whose discrete-time Fourier transform is D(ω) is a sampled version of the autocorrelation of g(t) where the samples are taken at integer times. This can be shown as follows: ∞ g(t)g(t − τ ) dt ⇐⇒ |G(ω)|2 d(τ ) = ∞−∞ T =1 d[n] = d(nT ) = g(t)g(t − nT ) dt ⇐⇒ |G(ω + 2πk)|2 = D(ω) −∞
k∈Z
We can thus write D(ω) as D(ω) =
n
−jωn
d[n]e
=
n
∞ −∞
g(t)g(t − n) dt e−jωn
Now, if g(t) has compact support, so does its autocorrelation d(τ ). This implies that d(τ ) will be nonzero only at a finite number of integer times, which tells us that there are a finite number of nonzero coefficients d[n] in the DTFT D(ω). We can thus conclude that D(ω) is a polynomial in 51
ejω , i.e. a trigonometric polynomial. Also, because d[n] is FIR, it does not have poles on the unit circle, so D(ω) has a stable spectral factorization.
4.10 Orthogonality relations of Daubechies wavelets
(a) It is proven in Proposition 4.4 that ϕ(i) (t), ϕ(i) (t − l) = δ(l) for all i. We can use this to prove relations (b) and (c) as follows: ψ (i+1) (t − n), ψ (i+1) (t − n )
√ √ 2 h1 (k)ϕ(i) (2t − 2n − k), 2 h1 (m)ϕ(i) (2t − 2n − m) = k∈Z
= 2
=
; h1 (k)h1 (m) ϕ(i) (2t − 2n − k), ϕ(i) (2t − 2n − m)
m
k
m∈Z
:
h1 (k)h1 (k + 2n − 2n ) =
<
= h1 (k), h1 (k + 2n − 2n ) = δ(n − n )
k
where the scale factor = is neglected in the third step. Taking the limit as i −→ ∞ gives the result: < ψ(t − n), ψ(t − n ) = δ(n − n ) (b) Similarly, ϕ(i+1) (t − n), ψ (i+1) (t − n )
√ √ (i) (i) 2 h0 (k)ϕ (2t − 2n − k), 2 h1 (m)ϕ (2t − 2n − m) = k∈Z
= 2
=
m∈Z
; h0 (k)h1 (m) ϕ (2t − 2n − k), ϕ(i) (2t − 2n − m) (i)
m
k
:
h0 (k)h1 (k + 2n − 2n ) = h0 (k), h1 (k + 2n − 2n ) = 0
k
Taking the limit as i −→ ∞ gives the result:
ϕ(t − n), ψ(t − n ) = 0
4.11 Infinite products and the Haar scaling function k+1 k 1−b Pk i i 1−b ab = a(
(a) pk =
i=0
b
1 =⇒ lim pk = a( 1−b ) since |b| < 1
) = a
k→∞
i=0
(b) Let Φn (ω) =
n k=1
M0
ω 2k
ω jω 1 . where M0 (ω) = √ G0 ejω = e− 2 cos 2 2
Substituting for M0 (ω) in the Φn (ω) product yields n n ω − jω e 2k+1 cos k+1 Φn (ω) = 2 k=1
k=1
52
sin(2ω) , the second product term can be expanded as 2 sin(ω) n ω sin ω2 sin ω4 sin ω8 sin 2ωn ω ··· ω cos k+1 = 2 2 sin ω4 2 sin ω8 2 sin 16 2 sin 2n+1 k=1 sin ω2 ω = 2n sin 2n+1
Using the identity cos(ω) =
Taking the limit for large n yields n
−
jω 2k+1
= e−
jω 2
{using the result of part (a)} k=1 sin ω2 sin ω2 sin ω2 ω = ω = ω lim n→∞ 2n sin 2n 2n+1 2 2n+1 lim
n→∞
e
Finally, we have the result − jω 2
Φ(ω) = lim Φn (ω) = e n→∞
sin ω2 ω
ω
2
ω Φ . 3mm (c) From equation 4.4.15, we have Ψ(ω) = M1 2 2 jω 1 ω and Φ(ω) from (b), we have For M1 (ω) = √ G1 (ω) = je− 2 sin 2 2 sin2 ω4 − jω Ψ(ω) = je 2 ω 4
4.12 Iterated filter banks
(a) From the fact that |M0 (ω)| ≤ 1, ω ∈ [−π, π], and the fact that M0 (ω) is 2π-periodic, it follows ). From that |M0 (ω)| ≤ 1 for all ω. Now fix some ω and define the sequence {pk } by pk = M0 ( 2ωk > the condition M0 (0) = 1 it follows that limk→∞ pk = 1. In order to prove convergence of ∞ k=1 pk we may therefore assume without loss of generality that 0 < p ≤ 1 for all k. This implies that the k > sequence {Pi } = { ik=1 pk } is a descending sequence of positive numbers, and therefore convergent. (b) The orthogonality condition implies |M0 (ω)|2 + |M0 (ω + π)|2 = 1. The results follow.
4.13 Maximally flat Daubechies’ filters By Bezout’s theorem (which can be proven using Euclid’s algorithm), we can show that P (y) is a polynomial of degree less than N . Bezout’s theorem is stated as follows (see, for example, Daubechies’ “Ten Lectures on Wavelets”): If p1 , p2 are two polynomials, of degree n1 , n2 , respectively, with no common zeros, then there exist unique polynomials q1 , q2 , of degree n2 − 1, n1 − 1, respectively, so that p1 (x)q1 (x) + p2 (x)q2 (x) = 1 . 53
For our problem, since (1 − y)N and y N have no common zeros, by Bezout’s theorem, there exist unique polynomials q1 , q2 , of degree ≤ N − 1, such that (1 − y)N q1 (y) + y N q2 (y) = 1 . Substituting 1 − y for y in the above expression yields (1 − y)N q2 (1 − y) + y N q1 (1 − y) = 1 . By the uniqueness of q1 and q2 , this implies that q1 (y) = q2 (1 − y). It follows that P (y) = q1 (y) is a solution for (1 − y)N P (y) + y N P (1 − y) = 1 , and that the degree of P (y) is less than N . Now, let’s find the closed form expression for P (y). Rewrite equation (4.4.32) as P (y) = The Taylor series expansion of 1 (1 − y)N
1 (1−y)N
1 [1 − y N P (1 − y)] . (1 − y)N
is
= 1 + N y + N (N + 1)
y3 y2 + N (N + 1)(N + 2) + . . . 2! 3!
∞ N −1+j yj . = j j=0
Substitute this expression into the previous equation for P (y), we have y3 y2 + N (N + 1)(N + 2) + . . .) − P (y) = (1 + N y + N (N + 1) 2! &' 3! ( % A
y N +3 y N +2 + N (N + 1)(N + 2) + · · ·) P (1 − y) . (y N + N y N +1 + N (N + 1) 2! 3! &' ( % B
Note that since we assume P (y) is a polynomial in y (in nonnegative powers only) with degree less than N , the term B (which consists only of y i with i ≥ N ) must cancel the terms in A with y i , i ≥ N . Hence, y3 y2 + N (N + 1)(N + 2) + ··· 2! 3! y N −1 + N (N + 1) · · · (N + N − 2) (N − 1)! N −1 N −1+j yj , = j
P (y) = 1 + N y + N (N + 1)
j=0
which is equation (4.4.34) with Q(y) = 0. Note that the given expression for P (y) is the unique lowest degree solution, but there exist many solutions of higher degrees. 54
4.14 Regularity of Daubechies filters
N 1 + ejω We represent the normalized lowpass filter M0 (ω) as M0 (ω) = R(ω) and let B = 2 sup |R(ω)|. The sufficient condition for the convergence of ϕ(i) (t) to a continuous function ϕ(t) ω∈[0,2π]
with Fourier transform Φ(ω) =
∞
M0
k=1
N 2 3 4 5 6
B 1.73 3.16 5.92 11.22 21.49
ω 2k
is B < 2N −1 . For the Daubechies filters in table 4.2:
2N −1 2 4 8 16 32
All of the filters satisfy the regularity bound, so the iterated filters converge to the Fourier transforms of continuous functions. For higher regularity, a sufficiency condition is that the limit is m times continuously differentiable if B < 2N −1−m where B is defined as in part (a). This is not satisfied for m = 1 for the N = 4 and N = 5 filters. Consider a variation of the Daubechies condition that corresponds to a cascade of filters and upsampling; we define a new supremum as in the book: Bl =
l−1
sup
|R(2k ω)|
ω∈[0,2π] k=0
The limit function is m times continuously differentiable if Bl < 2l(N −1)−m For m = 1 and l = 2 this amounts to B2 < 22N −3 . As an example, we consider N = 4 and N = 5, for which we obtain: B 22N −3 N 4 19.31 32 5 54.96 128 We can conclude from this that the limits are continuously differentiable. Other combinations are left to the reader.
4.15 Necessity of zeros at aliasing frequencies Using a similar method as in the proof of Prop. 4.6, it is proved that g0 [N k + i] = g0 [N k + j] k
k
for all i, j = 0, · · · , N − 1. Not let w be any G0 (w) =
N th N −1
of unity, w = 1. Then we have wi
i=0 N −1
= (
g0 [N k + i]
k
wi )
i=0
= 0. 55
k
g0 [N k]
Here we have used the well known equality
N −1 i=0
wi = 0 for w = 1.
4.16 A sufficient condition for continuity Similarly as in the proof of Prop. 4.7, we can write Φ(ω) =
∞
k=1
k
1 + ejω/N + · · · + ejω(N −1)/N N
k
K
∞ k=1
R(
ω ). Nk
In the above, the first product equals (sin(ω/2)/(ω/2))K , which leads to a decay of O(1 + |ω|)−K . Because R(0) = 1, there exists a constant C such that |R(ω)| ≤ 1 + C|ω|. So proceeding as in C > ω N−1 for all ω ≤ 1. the proof of Prop. 4.7, we find ∞ k=1 |R( N k )| ≤ e For 2J−1 ≤ |ω| < 2J we find ∞ k=1
|R(
C ω )| ≤ B J e N−1 k N C
≤ 2J(K−1− ) e N−1 ≤ C (1 + |ω|)K−1− , where we have chosen such that B < 2K−1− . The required result follows.
4.17 Successive interpolation
(a) Let xu [n] be the upsampled version of x[n]. The one-step output y (1) [n] is the convolution of xu [n] with the filter impulse response g[n]: xu [k]g[n − k] y (1) [n] = even k
where the sum is taken over even k since xu [n] is zero for odd indices. Equivalently, xu [2i]g[n − 2i] y (1) [n] = i
Since xu [2i] = x[i], we can write the problem requirement as x[i]g[2n − 2i] = x[n] y (1) [2n] = i
which implies that g[2n] = δ[n]. So g[n] must be zero for even time indices. This condition implies that G(z) can not have any nonzero even powers of z. (b) If the interpolation filter g[n] meets the requirements of (a), the two-step output y (2) [n] satisfies y (2) [2n] = y (1) [n]. Replacing n by 2n gives y (2) [4n] = y (1) [2n] = x[n]. Iterating this argument yields the result y (i) [2i n] = x[n]. 1 1 (c) G(z) = z + 1 + z −1 corresponds to a simple moving average where the interpolated value 2 2 is the mean of the immediate neighboring values. An example is illustrated in the following figure.
56
4
4
4
3
3
3
2
2
2
1
1
1
0 0
2
0 0
4
5
10
0 0
10
20
1 1 1 1 √ +√ z √ + √ z −1 , which is a product of Haar lowpass 2 2 2 2 filters. The iteration of G(z) converges to the product of the iterations of these filters, so the associated time function y(t) = lim y (i) (t) corresponds to a convolution of Haar lowpass functions. We thus expect the limit function y(t) to be everywhere continuous but not everywhere differentiable. (d) We can write G(z) =
(e) The Daubechies filter autocorrelations are given by PN (z) = 2−2N +1 (1 + z)N (1 + z −1 )N R(z) where R(z) is given in Table 4.2 for N = 2, . . . , 6. It is straightforward to numerically verify (using MATLAB, for instance) that these functions satisfy the constraint from (a), namely they are zerovalued at even time indices (except time 0). For higher regularity, the sufficient condition given in Proposition 4.7 can be used. That is, let M (ω) = be factored as
M (ω) =
PN (ejω ) PN (1)
1 + ejω 2
N R(ω).
Define B = supω∈[0,2π] |R(ω)|. If B < 2N −1− , = 1, 2, . . . , then the iteration converges to ϕ(t) which is -times continuously differentiable. With this test, we can establish that for N = 2 the limit functions are continuous but not everywhere differentiable; for N = 3, 4 the limit functions are continuous and once differentiable; for N = 5, 6 the limit function is continuous and twice differentiable.
4.18 Recursive subdivision schemes
(a) Using induction it is easy to show that supp(f (i) ) ⊂ (−1, 1) for all i. Define Pi (z) =
p(n i)z −n =
n
n
57
f (i) (
n −n )z , 2i
and
G(z) =
(1 + z)(1 + z −1 ) . 2
ck z −k =
k
One now easily derives i
Pi+1 (z) =
m
G(z 2 ).
m=0
Using the fact that i
(1 + z
2m
)=
2i+1 −1
m=0
m=1
it follows that = p(i+1) n
zm,
1 2i+1
2i+1 −|n| , 2i+1
=
1
0≤i,j≤2i+1 −1 i−j=n
|n| ≤ 2i+1 otherwise
0,
We conclude that f (∞) (t) = max(0, 1 − |t|), the well known hat function. (b) Defining Q(i) (z) =
g(i) (
n
we derive Q
(i+1)
(z) = (
i
n −n )z , 2i
m
i+1
G(z 2 ))S(z 2
),
m=0
where G(z) is as in (a) and where the definition of S(z) should be obvious. Taking the limits we find that s[n]f (∞)(t − n), g(∞) (t) = n
which is nothing other than bilinear interpolation. (c) Replacing the definition of G(z) in (a) by 1 (−z 3 + 9z + 16 + 9z −1 − z −1 ), 16 we can replay the arguments of (a) and (b). From G(z) =
P (i+1) (z) = P (i) (z 2 )G(z), we see that the limit function f (∞)(t) can be derived from the interpolation scheme 1, n = 0 (0) f (n) = 0, otherwise n n f (i) ( i−1 ) = f (i−1) ( (i−1) ) 2 2 2n + 1 9 n n+1 (f (i−1) ( i−1 ) + f (i−1) ( i−1 )) ) = f (i) ( i 2 16 2 2 1 (i−1) n − 1 (i−1) n + 2 ( i−1 ) + f ( i−1 )) − (f 16 2 2 58
If we start with an arbitrary sequence s[n], we can either substitute g for f in the above and replace the definition g(0) by g(0) (n) = s[n], or derive a result as in (b) g(∞) (t) =
s[n]f (∞)(t − n).
n
4.19 Interpolation filters and functions
(a) We show by induction that if g[n] is an interpolation filter, then ϕ(i) (t) has the interpolation property for any i. To check whether ϕ(i) (t) has the interpolation property, it is equivalent to check that g(i) [2i n] = δ[n]. It is easier to do this in the z-transform domain. For i = 1 (the initial step of the induction), (1) g [n] = g[n], hence g(1) [2n] = δ[n]. Writing G(1) (z) in its polyphase domain, we have (1)
G(1) (z) = 1 + zG1 (z 2 ) , (i)
where Gj (z) denotes the j-th polyphase component of G(i) (z). Now assume that g(i) [2i n] = δ[n]. Then we can write G(i) (z) in its polyphase component as (i)
G (z) = 1 +
i −1 2
i
(i)
z j Gj (z 2 ) .
j=1
From the iterated filter bank, we have i
G(i+1) (z) = G(i) (z)G(z 2 ) . Substituting in the expression for the polyphase decomposition, (i+1)
G
(z) = (1 +
i −1 2
(i)
i
i
i+1
(1)
z j Gj (z 2 )) (1 + z 2 G1 (z 2
))
i −1 2
i −1 2
j=1
= 1+z
2i
i+1 (1) G1 (z 2 )
+
z
j
i (i) Gj (z 2 )
+
j=1
i
(i)
i
(1)
i+1
z j+2 Gj (z 2 )G1 (z 2
).
j=1
We need to verify that the 0-th polyphase component of G(i+1) (z) is 1, or, equivalently, the coi+1 efficients of z m2 , m ∈ Z − {0}, are 0. Clearly, the middle two terms do not contribute to the i+1 i i i+1 for coefficients of z m2 , m ∈ Z. The last term contributes to the coefficients of z j+2 +k2 +l2 i i i i+1 can j ∈ {1, . . . , 2 − 1}, and m, l ∈ Z. It is easy to verify that the exponent j + 2 + k2 + l2 never be equal to multiples of 2i+1 . Hence, the 0-th polyphase component is 1, and we can write (i+1)
G
(z) = 1 +
2i+1 −1
(i+1)
z j Gj
i+1
(z 2
).
j=1
Thus, g(i) [2i n] = δ[n] and ϕ(i) (n) = δ[n] for all i. Since ϕ(i) (t) converges pointwise, taking i → ∞ and sampling yields ϕ(n) = δ[n]; i.e. ϕ(t) has the interpolation property.
59
(b) Let the support of g[n] be n ∈ {0, 1, . . . , N }; that is, g[0] = 0, g[N ] = 0, and g[n] = 0 for n = {0, 1, . . . , N }. Since g[n] is a lowpass filter, we cannot have N = 0 (otherwise, it becomes a trivial allpass filter), and N > 0. If g[n] is an interpolation filter, then N must be odd (because g[2n] = δ[n]). The filter g[n] is orthogonal, so g[n], g[n − 2k] =
N
g[n]g[n − 2k] = c δ[k]
n=0
for some nonzero constant c. Since the support of g[n] is {0, 1, . . . , N } and using the fact that g[2n] = δ[n], the inner product can be simplified to g[n], g[n − 2k] =
N
g[n]g[n − 2k]
n=2k N−1−2k 2
= g[0]g[2k] +
g[2k + 2n + 1]g[2n + 1] ,
(9)
n=0
where the second expression is simply the inner product at the odd samples of g[n] and we have assumed, without loss of generality, that k ≥ 0. Evaluating at k = 0, and using g[0] = 1, N−1 2
1+
g2 [2n + 1] = c .
n=0
First let’s assume that N > 1, and then we show that we must have N = 1. Evaluating (9) at k = N 2−1 yields g[N ]g[1] = 0 . By assumption g[N ] = 0, so g[1] = 0 must hold. Again, evaluating (9) at k =
N −3 2
yields
g[N − 2]g[1] + g[N ]g[3] = 0 , which implies g[3] = 0. Continuing this way until k = 0, we find that all odd samples of g[n] except g[N ] is 0, and that g[n], g[n] = g2 [0] + g2 [N ] = c . Since the problem specifies a lowpass filter, and for filter, we must have N = 1. Hence our filter is ⎧ ⎨ 1√, c−1 , g[n] = ⎩ 0,
N > 1, g[n] resembles nothing like a lowpass n=0 n=1 otherwise
.
For c = 2, we have the Haar lowpass filter (up to a scaling factor). (c) Given that ϕ(t) has the interpolation property (i.e. ϕ(n) = δ[n]) and satisfies the two-scale equation cn ϕ(2t − n) . ϕ(t) = n
60
Then, sampling ϕ(t) at t = n, n ∈ Z yields cm ϕ(2n − m) = cm δ[2n − m] = c2n = δ[n] . ϕ(n) = m
m
Thus, the sequence cn is an interpolation filter.
4.20 Two-scale equation We may assume that ϕ(t) is not identically 0. We first assume that we can prove that f (t) = f (2t). Also assume that f (t) is continuous at 0. We will show that we can prove a contradiction if f (t) is not a constant. Assume that there exists a t1 = 0 such that f (t1 ) = f (0), and let = |f (t1 )−f (0)|. Let δ be any positive constant. Then for J sufficiently large |t1 /2J | < δ, and |f (t1 /2J ) − f (0)| = |f (t1 ) − f (0)| = < , contradicting the continuity assumption. It remains to prove the two assumptions. First we show
continuity at 0. Fix an > 0. Because of the decay of ϕ(t), we can find an N ∈ N such that |n|>N ϕ(t − n) < 1/4 for all t ∈ [−1, 1]. For each n ∈ {−N, · · · , N } we choose a δn such that |ϕ(n) − ϕ(n + τ )| < /6N for all τ < δn . Set δ = minn δn . Then we have for all τ < δ |ϕ(n) − ϕ(n + τ )| |f (0) − f (τ )| ≤ n
=
|ϕ(n) − ϕ(n + τ )| +
|n|≤N
≤
/6N +
|n|≤N
|ϕ(n) − ϕ(n + τ )|
|n|>N
(|ϕ(n)| + |ϕ(n + τ )|)
|n|>N
< /2 + /2 = . This proves continuity at 0. The second assumption is taken care of as follows: ϕ(t − n) f (t) = n
=
n
m
k
n
cm ϕ(2t − 2n − m)
( ck−2n )ϕ(2t − k) = =
ϕ(2t − k)
k
= f (2t).
4.21 Two-scale equation ϕ (t) satisfies the two-scale equation ϕ(2t − n) ϕ(t) =
(*)
n
61
with
n
c2n =
c2n+1 = 1. Differentiate (*) we get
n
ϕ (t) =
2cn ϕ (2t − n).
(**)
n
If we take the hat function as an example, then its derivative is a scaled version of the Haar wavelet. Pictorially this situation is presented in Figure P4.21.
;@@ ; ; @@ ; 1
1
ϕ(t)
-1
ϕ (t)
1
-1
-1
1
-1
(a) hat function
(b) derivative of hat function 2 2ϕ (2t)
.. ... . . . . ϕ (2t + 1) . . . ............. . -1 . . . . ...
1
.... . . ϕ (2t − 1) . .
-2
(c) two scale equation for derivative
Figure P4.21: Two scale equation for the derivative of the hat function. If ϕ is C n−1 (R ) and ϕ(n) (t) exists, then differentiating n-times yields 2n cn φ(n) (2t − n). ϕ(n) (t) = n
4.22 Orthogonality of local cosine bases We proceed as in Section 4.8.2. First we show that ϕj−1,k and ϕj,m are orthogonal. ϕj−1,k , ϕj,m =
2
+ηj
wj−1 (x − aj )wj (x + aj ) Lj−1 Lj −ηj 1 π (k + )(x + Lj−1 ) × cos Lj−1 2 1 π (m + )x dx cos Lj 2 +ηj 2 = ±wj (aj − x)wj (aj + x) Lj−1 Lj −ηj 62
1 (k + )x × Lj−1 2 1 π (m + )x dx cos Lj 2
sin
π
Since the window part of this last integral is even, while the rest is odd, the above inner product is zero. It remains to evaluate ϕj,k , ϕj,m . As in Section 4.8.2, we split this integral in three parts: from aj − ηj to aj + ηj , from aj + ηj to aj+1 − ηj+1 , and from aj+1 − ηj+1 to aj+1 + ηj+1 . We consider the the first integral and third integral: ηj π π 1 1 2 wj2 (aj + x) cos (k + )x cos (m + )x dx Lj −ηj Lj 2 Lj 2 ηj π π 1 1 2 (wj2 (aj + x) + wj2 (aj − x)) cos (k + )x cos (m + )x dx = Lj 0 Lj 2 Lj 2 ηj π 1 1 π 2 cos (k + )x cos (m + )x dx = Lj 0 Lj 2 Lj 2 and 2 Lj
ηj+1 −ηj+1
=
2 Lj
wj2 (aj
0
−ηj+1
1 1 π π + Lj + x) cos (k + )(x + Lj ) cos (m + )(x + Lj ) dx Lj 2 Lj 2 (wj2 (aj+1 + x) + wj2 (aj+1 − x)) ×
1 1 π π (k + )(Lj + x) cos (m + )(Lj + x) dx cos Lj 2 Lj 2 Lj π 1 1 π 2 cos (k + )x cos (m + )x dx. = Lj Lj −ηj+1 Lj 2 Lj 2
Combining this with the second integral, we find Lj π π 1 1 2 cos (k + )x cos (m + )x dx = δkm , Lj 0 Lj 2 Lj 2 which was to be proved.
63
Chapter 5 5.1 Characterization of local regularity
+∞ ∗ (a) The continuous wavelet transform is given by CWTf (a, b) = ψa,b f (t) dt. −∞ 1 − 12 ≤ t ≤ 0 For the Haar wavelet ψ(t) = the CWT of some function f (t) is −1 0 ≤ t ≤ 12 ⎧ a +b 2 1 ⎪ ⎪ f (t) dt b<0 ⎨ −√ a 0 a CWTf (a, b) = b +b 2 ⎪ 1 1 ⎪ ⎩ −√ f (t) dt − √ f (t) dt b ≥ 0 a 0 a b 0 t<0 the CWT is For f (t) = tn t ≥ 0 ⎧ a n+1 1 1 ⎪ ⎪ √ − + b b<0 ⎨ a n + 1 2 CWTf (a, b) = n+1 a 1 1 ⎪ ⎪ +b − 2bn+1 b≥0 ⎩ −√ 2 a n+1 (2n+1)
(2n+1) a 2 this amounts to CWTf (a, b) ≈ − ∼ a 2 . For b << n+1 (n + 1) 2 3 2 Substituting n = 1 gives CWT ∼ a .
a 2
2n+1
(b) The derivation in (a) shows that CWT ∼ a 2 . (This can also be derived using the reasoning of section 5.1.2, specifically the subsection on characterization of regularity.)
5.2 Autocorrelation of the Haar wavelet⎧
1 1 − ≤t≤0 2 1 (a) The Haar wavelet is given by ψ(t) = −1 0 ≤ t ≤ ⎪ ⎪ 2 ⎩ 0 otherwise ⎪ ⎪ ⎨
∞
Because ψ(t) is piecewise constant, its autocorrelation a(t) = −∞
ψ(τ )ψ(τ − t)dτ is piecewise
linear (like the convolution of two boxcars, for instance). Thus a(t) is specified completely by its breakpoints, which are easy to find: ⎧ ⎪ 0 t ≤ −1 no overlap between ψ(τ ) and ψ(τ − t) ⎪ ⎪ ⎪ 1 1 ⎪ ⎪ ⎪ ⎨ − 2 t = − 2 half overlap 1 t=0 full overlap a(t) = ⎪ ⎪ 1 ⎪ ⎪ t = 12 half overlap − ⎪ ⎪ 2 ⎪ ⎩ 0 t≥1 no overlap ⎧ ⎪ ⎪ ⎨ Connecting the breakpoints by straight lines gives a(t) =
64
0
|t| − 1 ⎪ ⎪ ⎩ −3|t| + 1
|t| ≥ 1 1 ≤ |t| ≤ 1 2 1 |t| ≤ 2
The graph of a(t) is:
1 0.5 0 −0.5 −1.5
−1
−0.5
0 time
0.5
1
1.5
(b) The autocorrelation a(t) is everywhere continuous but not everywhere differentiable; this is evident from the above graph. We can find the decay by noting that a(t) = ψ(t) ∗ ψ(−t) and thus 1 1 decay, A(ω) has a 2 decay as ω −→ ∞. A(ω) = |Ψ(ω)|2 . Since Ψ(ω) has a ω ω
5.3 Nondownsampled filter bank
(a) We have G0 (z) = H0 (z −1 ) and G1 (z) = H1 (z −1 ), meaning that the synthesis filters are timereversed versions of the analysis filters. Orthogonality yields: G0 (z)G0 (z −1 ) + G0 (−z)G0 (−z −1 ) = 2 and G1 (z) = −z −2k−1 G0 (−z −1 ). The filter bank output is related to the input by Y (z) = [G0 (z)H0 (z) + = G0 (z)G0 (z −1 ) # = G0 (z)G0 (z −1 ) = G0 (z)G0 (z −1 )
G1 (z)H1 (z)] X(z) + G1 (z)G1 (z −1 ) X(z) ? @? @$ + −z −2k−1 G0 (−z −1 ) −z 2k+1 G0 (−z) X(z) + G0 (−z)G0 (−z −1 ) X(z) = 2X(z)
In the time domain, y[n] = 2x[n]. (b) By Parseval’s Theorem, x0 2 + x1 2 = = =
π π jω 2 jω 2 1 X0 e dω + X1 e dω 2π −π −π π @ jω 2 ? jω 2 1 X e H0 e + H1 ejω 2 dω 2π −π π jω 2 ? −jω 2 @ 1 + G1 e−jω 2 dω X e G0 e 2π −π
By the relationship of G0 (z) and G1 (z) used in part (a), we have 2 G1 (e−jω )2 = G0 (−ejω )2 = G0 ej(ω+π) . For real g0 [n], we have G0 (e−jω ) = G∗0 (ejω ) by 2 2 conjugate symmetry, and thus G0 (e−jω ) = G0 (ejω ) . We can then write π 2 jω 2 jω 2 1 2 2 j(ω+π) X e G0 e + G0 e x0 + x1 = dω 2π −π Since g0 [n] and its modulated version are power-complementary (due to orthogonality), π 1 X(ejω )2 dω = 2x2 x0 2 + x1 2 = 2 2π −π 65
(c) Given H0 (z) and G0 (z), we want to find H1 (z) and G1 (z) such that y[n] = x[n]; in the Z-domain, this perfect reconstruction is given by H0 (z)G0 (z) + H1 (z)G1 (z) = 1 H1 (z)G1 (z) = 1 − H0 (z)G0 (z) To find H1 (z) and G1 (z), we spectrally factor 1 − H0 (z)G0 (z); note that this is FIR and thus has no poles (except at 0 or ∞). The number of degrees of freedom in the design corresponds to the number of different ways the zeros of 1− H0 (z)G0 (z) can be distributed among the two filters H1 (z) and G1 (z). Note that if we are restricted to real filters, any complex zeros have to be distributed in conjugate pairs. Let F (z) = 1 − G0 (z)H0 (z) with H0 (z) = G0 (z −1 ) = 1 + 2z −1 + z −2 . Then, F (z) = − z 2 − 4z − 5 − 4z −1 − z −2 = (− z 2 − 3z − 1) (z −2 + z −1 + 1) √ √ √ √ 3+ 5 3− 5 1+j 3 1−j 3 −1 −1 = z+ z+ z + z + 2 2 2 2
4 4 4 4 4 There are + + + + = 16 ways to spectrally factor this F (z) 0 1 2 3 4 into H1 (z) and G1 (z) if we allow complex filters. For real filters, there are only 8 factorizations.
5.4 Continuous wavelet transform (a) One easily derives that 1 CWTf (a, b) = √ a
b b− 12 a
b+ 12 a
f (t) dt −
f (t) dt .
b
In order to describe the CWT of f , we need to evaluate an expression of the form
b
1 1 χ − , 2 2 b− 1 a
b+ 12 a
− b
2
1 1 χ − , 2 2
.
It is obvious that the above expression is piecewise linear in both a and b. In general, this piecewise function will be determined by its values at the six points b = ± 12 (1 + {−1, 0, 1}a). For fixed a there are four distinct cases to consider: a = 1, a < 1, 1 < a ≤ 2 and 2 < a. For a = 1 the CWTf (a, b) is completely determined by the values at the 4 points b = ±1, ± 12 , viz. 0, ± 12 . 1 and for For a < 1 there are 6 values of b to consider. For b = ± 12 the CWT equals ± 2√ a b = ± 12 (1 ± a) the CWT equals zero. 1 , for For 1 < a ≤ 2 there are also 6 values to consider. For b = ± 12 the CWT equals ± 2√ a
√ . b = ± 12 (1 + a) the CWT equals zero, and for b = ± 12 (1 − a) the CWT equals ± (1−a) a
For 2 < a there are again 6 values to consider. For b = ± 12 the CWT equals ± √1a , for
b = ± 12 (1 + a) the CWT equals zero, and for b = ± 12 (a − 1) the CWT equals ± √1a . See Figure P5.4 for a graphical representation. In particular, for small values of a, the CWT 1 . For very large values of of f has support in the neighborhood of ± 12 with a peak value of ± 2√ a
66
1 -1
1
;;@@1 @@;;
-1 b-¿
-1
BB B
B BB
-1
1
1
QQ QQ
b-¿
CWT for a=1/2
CWT for a=1
-1
1
Q Q1QQ
@2@
@@ -2
b-¿
-1
b-¿
-1
CWT for a=2
CWT for a=4
Figure P5.4: CWT for various values of a.
a the CWT has support from − 12 (1 + a) to 12 (1 + a) with the majority of the values being either − √1a (for negative b values) or √1a (for positive b values). (b) The function ψ can be written as ψ(t) = f (2t + 1/2) − f (2t − 1/2). Defining f± (t) by f (2t ± 12 ) we find, using the shift and scaling properties, 1 CWTf± (a, b) = √ CW Tf (2a, 2b ± 1). 2 Therefore
1 CWTψ (a, b) = √ (CWTf (2a, 2b + 1/2) − CWTf (2a, 2b − 1/2)). 2
In particular, for small values of a, the CWT of ψ has support around ± 12 (with a negative 1 ) and around 0 (with a positive function value √1a ). For large a, the CWT function value 2√ a 1 and a peak with positive has support around −a/2 and a/2, with a negative function value − 2√ a
function value
√1 a
around the origin.
5.5 Tight frames for R2 Note that N −1 N 2πk 2πk = = sin2 N N 2 k=0 k=0 N −1 2πk 2πk cos =0 sin N N N −1
cos2
k=0
67
Let x = [x1 , x2 ] and n−1 2 < ϕk , x > ϕk y = [y1 , y2 ] = N N =0 n−1 n−1 2πk 2 2πk 2πk x1 + x2 cos =⇒ y1 = cos2 sin N N N N k=0 k=0 N −1 N −1 2πk 2 2πk 2πk y2 = sin + x2 x1 cos sin2 N N N N k=0
k=0
Applying the noted identities completes the proof: y1 = x1
y2 = x2
=⇒
y=x
Comparing the reconstruction formula above (the formula for y) and equation 5.3.5 indicates that the redundancy factor is N/2. This is intuitively reasonable since this frame uses N vectors to represent a 2-dimensional space; recall the example of the overcomplete basis of three vectors for R2 discussed in example 1.1 of section 1.1 and also in example 5.1, for which the redundancy factor is 3/2.
5.6 Sinc frame More or less by definition, the space of bandlimited signal is given by / ln sinc(t − n) : l ∈ L2 (Z) . B= n∈Z
Moreover, for each function f ∈ B, the sequence {ln } is found as ln = f (n) and f = l. Now consider shifted versions fi (t) = f (t + i/N ) of f , i = 0, . . ., N − 1. Then we find f (t) = fi (t − i/N ) fi (n) sinc((t − i/N ) − n) = n
=
f (n + i/N ) sinc((t − i/N ) − n).
n
Thus N f (t) =
N −1
f (t)
i=0
=
i
=
f (n + i/N ) sinc((t − i/N ) − n)
n
f (k/N ) sinc(t − k/N ).
k∈Z
This immediately implies that {sinc(t − k/N )} is a tight frame with redundancy N .
5.7 Properties of a full rank non-square matrix
(a) Consider the function r(x) = M x on the unit sphere S n−1 ⊂ Rn . The rank of M being 68
equal to n is equivalent to M being injective. Therefore we have that M x = 0 on S n−1 , implying 0 < r(x) for all x ∈ S n−1 . As r(x) is continuous, and as S n−1 is compact, the function r(x) takes on a minimum A = 0 and a maximum B = 0 on S n−1 . Consequently for all x ∈ Rn the required inequalities hold: 0 < Ax ≤ M x ≤ Bx < ∞. (b) The matrix M T M has size n × n and is invertible, as shown by the following sequence of equalities. MT Mx = 0 ⇒ M T M x, x = 0 ⇒ M x, M x = 0 ⇒ Mx = 0 ⇒ x = 0 ˜ M we find The last equality follows from injectivity of M . Evaluating M ˜ M x = (M T M )−1 M T M x = Ix = x. M ˆ is any other left inverse, then the difference N = M ˆ −M ˜ satisfies N M = 0 and vice versa. (c) If M Such a matrix N can also be characterized by image(M ) ⊂ kernel(N ). (d) Given y ∈ Rm , its projection P y (= M z for some z) onto the range of M is characterized by y − P y, M x = 0, for all x ∈ Rn . Rewriting we find y − P y, M x = 0 ∀x ⇒ y − M z, M x = 0 ∀x ⇒ M y − M T M z, x = 0 ∀x ⇒ T
MT y − MT Mz = 0 ˜y z = (M T M )−1 M T y = M ˜y Py = Mz = MM
69
⇒ ⇒
Chapter 6 6.1 Toeplitz matrix-vector products
A general Toeplitz matrix of size N × N has 1 + 2(N − 1) = 2N − 1 parameters, and a general circulant matrix of the same size has N parameters. Therefore the minimal extension of an N × N Toeplitz matrix to a circulant matrix needs N − 1 columns and rows. The corresponding numbers of parameters for symmetric Toeplitz and circulant matrices are N and N 2+2 respectively. So the minimal extension is by N − 2 rows and columns. For example, the matrix ⎡ ⎤ a b c ⎣d a b ⎦ e d a is extended to ⎡ ⎤ a b c e d ⎢d a b c e ⎥ ⎢ ⎥ ⎢e d a b c ⎥ ⎢ ⎥ ⎣c e d a b ⎦ a c e d a and ⎡ ⎤ a b c ⎣b a b⎦ c b a is extended to ⎡ ⎤ a b c b ⎢b a b c⎥ ⎢ ⎥ ⎣c b a b⎦. b c b a
6.2 Block circulant matrices A matrix C with blocks Ci,j is block-circulant if and only if Ci,j = Di−j for some sequence of matrices {Dk }, k = 0, · · · , N − 1, where the subtraction i − j in the index of Di−j is to be taken modulo N . Let W be a primitive N th root of unity. Then we need to show that the matrix 1 ki W Ci,j W −jk N i,j
is block diagonal. The proof is as follows: 1 ki W Ci,j W −jl N i,j 1 ki = W Di−j W −jl N i,j 1 (k−l)i = W Di−j W (i−j)l N i,j 1 (k−l)i = W Dm W ml N i,m = δk,l Dm W ml m
70
6.3 Walsh-Hadamard transform A size 2N Walsh-Hadamard transform WHT 2N is computed by evaluating a matrix product (1) XN WHT N WHT N (2) . WHT N −WHT N X N
It follows that the addition complexity γ(N ) of the WHT satisfies the recursion formula γ(2N ) = 2γ(N ) + 2N . Using the fact that γ(2) = 2, we find γ(2N ) = 2N + (N − 1)2N = N 2N , and the required N log2 N complexity follows.
6.4 Complexity of MUSICAM filter bank The number of operations per sample equals 2
L 512 + 2 log2 N = 2 + 2 log2 32 = 2(16 + 5) = 42. N 32
With 44100 samples per second, this amount to 1852200 operations per second.
6.5 Iterated filters i−1
H (i) (z) =
i−1 k k H z2 H z2 = H(z)
k=0
i−2
= H(z)
k
H (z 2 )2
= H(z)
k=1
i−1
k−1 H (z 2 )2
k=1
{for k = k − 1}
= H(z) H (i−1) z 2
k =0 i−1
(i)
H (z) =
i−2 k k i−1 i−1 2 2 H z H z2 = H z = H z2 H (i−1) (z)
k=0
H
(2k )
(z) =
k −1 2
k=0
i H z2
i=0
= H(z)H(z ) · · · H z &' % 2
k−1 )
H (2
= =
k−1 )
H (2
k−1 −1
22
2k−1 −1
k−1 )
2k−1
× H(ξ)H(ξ 2 ) · · · H ξ 2
(z)
(z) H (2
k−1 k−1 +1 k 22 22 22 −1 × H z H z ···H z ( ( % &'
(ξ)
k−1 )
= H (2
k−1 )
(z) H (2
2k−1
z2
{for ξ = z 2
}
6.6 Overlap-add/save filter banks (a) First observe that in the notation of pp. 361–363, N = 4 and M = 2. By substituting into the formulas on p. 362, H(z) = z + 1 Hi (z) = z −1 H(W4i z) for i = 0, 1, 2, 3
71
⎧ H0 (z) ⎪ ⎪ ⎨ H1 (z) ⇒ H (z) ⎪ ⎪ ⎩ 2 H3 (z)
= = = =
z −1 (z + 1) z −1 (−jz + 1) z −1 (−z + 1) z −1 (jz + z)
= 1 + z −1 = −j + z −1 = −1 + z −1 = j + z −1
G(z) = 1 + z −1 + z −2 + z −3 ⎧ G0 (z) ⎪ ⎪ ⎨ G1 (z) ⇒ G (z) ⎪ ⎪ ⎩ 2 G3 (z)
Gi (z) = G(W4i z) for i = 0, 1, 2, 3 = = = =
1 + z −1 + z −2 + z −3 1 + (−jz)−1 + (−jz)−2 + (−jz)−3 = 1 + jz −1 − z −2 − jz −3 1 + (−z)−1 + (−z)−2 + (−z)−3 = 1 − z −1 + z −2 − z −3 −1 −2 −3 1 + (jz) + (jz) + (jz) = 1 − jz −1 − z −2 + jz −3
(b) Note that L = N − M + 1 = 3. Now we simply substitute into an equation from p. 362 to get Ci =
L−1 2 1 i 1 WN c[] = (−j)i c[]. N 4 =0
Recall that the filter bank is implementing a convolution with the filter c[n]. Substituting in Y (z) = z −(M −1) C(z)X(z), gives
(10)
=0
(11)
Y (z) = z −1 (c[0] + c[1]z −1 c[2]z −2 )X(z).
Therefore, when the channel coefficients are scalars, the impulse response can at most have non-zero samples corresponding to delays of 1, 2, and 3. (c) The fact that filters with longer impulse responses can be implemented if the scalar Ci ’s are replaced by filters is discussed on pp. 362–363. We demonstrate this through a simple example but first derive a formula relating the input and output. The output Y (z) consists of four components. For i ∈ {0, 1, 2, 3}, denote the i-th channel output component by Yi (z). Since Yi results from filtering by Hi , downsampling by 2, filtering by Ci , upsampling by 2, and filtering by Gi , Yi (z) = Gi (z) = Therefore
Y (z) =
1 Ci (z 2 )Hi (z)X(z) + Ci ((−z)2 )Hi (−z)X(−z) 2
1 Gi (z)Ci (z 2 ) [Hi (z)X(z) + Hi (−z)X(−z)] . 2
3 3 1 1 2 2 Ci (z )Gi (z)Hi (z) X(z) + Ci (z )Gi (z)Hi (−z) X(−z). 2 2 i=0
(12)
i=0
From this equation, we view the channel filters as weighting Gi (z)Hi (z) and Gi (z)Hi (−z). By multiplying out terms, Gi (z)Hi (z) = G(W4i z)z −1 H(W4i z)
= W4i + 2z −1 + 2W4−i z −2 + 2W4−2i z −3 + W4−3i z −4
Gi (z)Hi (−z) = G(W4i z)(−z)−1 H(−W4i z) = W4i − W4−3i z −4
72
for i ∈ {0, 1, 2, 3}. In general, aliasing will not be cancelled; choosing scalar channel coefficients through (10) will insure that aliasing is cancelled. It is instructive to work out a simple example. If c[n] = [1 0 0], then Ci = 14 for all i. In this case 3
Gi (z)Hi (z) = 8z
−1
, and
i=0
3
Gi (z)Hi (−z) = 0,
i=0
so the overall system is LTI with transfer function z −1 , which agrees with (11). Let us now implement a filter that could not be implemented with scalar channel coefficients. Suppose we wish to implement the simple lowpass filter c[n] = 18 [1 3 3 1]. (Recall that the actual system will have a minimum delay of M − 1 = 1.) Following the procedure on p. 363, we decompose c[n] into filters of length M = 2 and rescale, giving 14 [1 3] and 14 [3 1]. Taking 4-point Discrete Fourier Transforms gives 14 [4 1 − 3j − 2 1 + 3j] and 14 [4 3 − j 2 3 + j]. In the notation of the book, these eight coefficients form the set {ci }. From these we form the following filters: C0 (z) = C1 (z) = C2 (z) = C3 (z) =
1 (4 + 4z −1 ) 4 1 ((1 − 3j) + (3 − j)z −1 ) 4 1 (−2 + 2z −1 ) 4 1 ((1 + 3j) + (3 + j)z −1 ) 4
Through a tedious calculation (aided greatly by a system for symbolic mathematical calculation) we can show: 1 Ci (z 2 )Gi (z)Hi (z) = z −1 (1 + 3z −1 + 3z −2 + z −3 ) 2 3
(13)
i=0
3 1
2
Ci (z 2 )Gi (z)Hi (−z) = 0
(14)
i=0
Combining (12)–(14) shows that the system is LTI and implements the desired convolution with a delay of 1. A similar modulated filter bank implementation for this convolution using scalar channel coefficients would require larger M and hence have more delay.
6.7 Three-channel filter bank We consider the behavior for even and for odd signals separately. We start with an even signal X(z 2 ). In the channels this gives rise to the signals {0, X(z), X(z)}, and after convolution with the channel filters and upsampling we have {0, X(z 2 )(F0 (z 2 ) + F1 (z 2 )), X(z 2 )F1 (z 2 )}. Applying the synthesis filters and summing we find an overall output X(z 2 )(F0 (z 2 )+z −1 F1 (z 2 ))z −1 , showing the required LTI behavior for even signals. Secondly we consider an odd signal zX(z 2 ). In the channels this gives rise to the signals {X(z), X(z), 0}, and after convolution with the channel filters and upsampling we have {X(z 2 )F0 (z 2 ), X(z 2 )(F0 (z 2 ) + F1 (z 2 )), 0}. 73
Applying the synthesis filters and summing we find an overall output zX(z 2 )(F0 (z 2 )+z −1 F1 (z 2 ))z −1 , showing the required LTI behavior for odd signals. By linearity, the system is LTI for all signals, with the required impulse response.
74
Chapter 7 7.1 Distortion from uniform quantization
ˆ denote the result of quantizing X. The uniform Let X be uniformly distributed on [a, b). Let X 1 quantizer outputs yi = a + (i + 2 )∆ for inputs in [a + i∆, a + (i + 1)∆). The MSE distortion is given by ˆ2 D = E |X − X| =
N −1 a+(i+1)∆ a+i∆
=
=
i=0 N −1 a+(i+1)∆ i=0 a+i∆ N −1 i=0
= N
1 b−a
(x − yi )2 fX (x) dx
1 ∆ 2
− 12 ∆
1 x− a+ i+ 2
2 ∆
1 dx b−a
z 2 dz
∆2 1 ∆3 = , N ∆ 12 12
where the change of variables z = x − a − (i + 12 )∆ has been used. Note that this calculation is essentially the same as determining and variance of a uniform random variable.
7.2 Coding gain as function of number of channels (a) As suggested by the hint, consider coding a source with power spectrum given by 4 |ω| ≤ π2 . |X(ejω )|2 = π 2 2 < |ω| ≤ π We assume that the subband bandwidths are equal. The subband variances can be determined by integrating |X(ejω )|2 . For the 2-channel system, the subband variances are: σ02 σ12
π 2 1 = 2 |X(ejω )|2 dω = 2 2π 0 π 1 = 2 |X(ejω )|2 dω = 1 2π π 2
(The factors of 2 are to account for positive and negative frequencies.) Using the formula at the middle of page 388, the coding gain is GAIN2 =
DP CM (2 + 1)/2 3 = = √ . 1/2 DSBC (2 · 1) 2 2
For the 3-channel system, the subband variances are σ02 = 43 , σ12 = 1 and σ22 = 23 . We find GAIN3 =
( 4 + 1 + 23 )/3 DP CM 32/3 , = 34 = DSBC 2 ( 3 · 1 · 23 )1/3
so the coding gain is greater for the 2-channel system. 75
(b) Now consider using a 4-channel system for the same source. The subband variances are σ02 = σ12 = 1 and σ22 = σ32 = 12 . The coding gain is GAIN4 =
(1 + 1 + 12 + 12 )/4 DP CM 3 = = √ . 1 1 1/4 DSBC (1 · 1 · 2 · 2 ) 2 2
The performance is the same as the 2-channel system and better than the 3-channel system.
7.3 Coding gain in subband coding (a) The expression for the power spectrum should be |X(ejω )|2 = 1 −
|ω| π
|ω| < π.
(Note the absolute value.) First consider the case N = 2. The subband variances are: σ02 σ12
π π 2 3 1 1 2 ω jω 2 dω = = 2 |X(e )| dω = 1− 2π 0 π 0 π 8 π π 1 1 1 ω dω = = 2 |X(ejω )|2 dω = 1− 2π π π π π 8 2
2
(The factors of 2 are to account for positive and negative frequencies.) Using the formula from the middle of page 388, the coding gain is 2 1 1 3 1 2 2 2 σ0 + σ1 2 8 + 8 2 2 1/2 = 3 1 1/2 = √ . 3 σ0 · σ1 8 · 8 For the general case, calculate σk2
1 =2 2π
(k+1)π N kπ N
1 |X(e )| dω = π jω
2
(k+1)π N kπ N
2N − 2k − 1 ω dω = 1− π 2N 2
for k = 0, 1, . . . , N − 1. The coding gain is given by 1 2N −1 2N −3 1 1 N −1 2 N k=0 σk N N 2N 2 + 2N 2 + · · · + 2N 2 = > 1/N = 1 1 . 2N −1 2N −3 1 N N −1 2 ((2N − 1)(2N − 3) · · · 1) N · · · σ 2 2 2 2N 2N 2N k=0 k (b) The expression for the power spectrum should be |X(ejω )|2 = e−α|ω|
|ω| < π.
(Note the absolute value.) First consider the case N = 2. The subband variances are: σ02 σ12
π π $ 2 1 1 2 −αω 1 # jω 2 1 − e−απ/2 = 2 |X(e )| dω = e dω = 2π 0 π 0 απ π π $ 1 1 1 −απ/2 # e = 2 |X(ejω )|2 dω = e−αω dω = 1 − e−απ/2 2π π π π απ 2
2
76
3 2.8 2.6
Coding gain
2.4 2.2 2 (b)
1.8 1.6 1.4
(a)
1.2 1 0
0.2
0.4
0.6
0.8
1 alpha
1.2
1.4
1.6
1.8
2
Figure P7.3: Comparison of coding gains. (a) Coding gain from 7.3(a). (b) Coding gain from 7.3(b) for N = 2.
(The factors of 2 are to account for positive and negative frequencies.) The coding gain is given by 2 1 1 2 −απ/2 1 1 − e−απ/2 1 απ/2 −απ/2 2 σ0 + σ1 2 1+e απ √ e = = + e . 2 2 1/2 1 −απ/2 −απ/2 2 e 1 − e σ0 · σ1 απ Notice that the coding gain is a monotonically increasing function of |α|. This coding gain is compared to the coding gain for part (a) in Figure P7.3. For the general N -channel case, (k+1)π (k+1)π N N 1 1 2 jω 2 |X(e )| dω = e−αω dω σk = 2 kπ kπ 2π π N N $ $ αkπ kπ απ 1 # 1 # −α (k+1)π −α N e 1 − e− N e− N −e N = = − απ απ for k = 0, 1, . . . , N − 1. Now we can calculate 1 N
N −1
σk2 =
k=0
=
−1 $ 1 N αkπ 1 # − απ N 1−e e− N απ N k=0 # $ απ 1 1 − e−απ 1 − e− N απ , απ 1 − e− N
απ N # $ 1 − e− N 1 − απ N 1−e = απ απ 1 − e− N
where we have used the formula for a finite geometric series. Also, −1 1/N N −1 1/N $ N $ PN−1 απ 1/N αkπ απ 1 # 1 # 2 − απ − 1−e N 1 − e− N σk = e N = e k=0 − N απ απ k=0 k=0 $ απ PN−1 $ απ N(N−1) απ απ 1 # 1 # 2 1 − e− N e− N k=0 k = 1 − e− N e− N = απ απ $ α(N−1)π απ 1 # 1 − e− N e− 2 , = απ 77
where we have used the formula for an arithmetic series. The coding gain is therefore 1 N
N −1 k=0
> N −1
2 k=0 σk
1 − e−απ
σk2 1/N =
απ
(1 − e− N )e−
α(N−1)π 2
.
7.4 Huffman and run-length coding (a) The Huffman code is constructed as follows: code
symbol prob.
0
1
10
2
110
3
1110
4
11110
5
111110
6
1111110
7
1111111
8
1 2 1 4 1 8 1 16 1 32 1 64 1 128 1 128
0
→
1
1 2 1 4 1 8 1 16 1 32 1 64 1 64
0
→
1
1 2 1 4 1 8 1 16 1 32 1 32
0
→ 1
1 2 1 4 1 8 1 16 1 16
0
→
1
1 2 1 4 1 8 1 8
0
→
1
1 2 1 4 1 4
0
→
1
1 2 1 2
0
→ 1
1
The expected length of the Huffman code is: E[length] =
8
Pr(symbol i) × (length of code i) =
i=1
127 = 1.984 64
The entropy is defined as H = − i Pr(symbol i) log2 (Pr(symbol i)). In this case, the length of code i is exactly − log2 (Pr(symbol i)), so the expected length and the entropy are equal. (b) symbol +1 −1 +2 −2 +3 −3 +4 −4 SR
code 10 11 001 010 0001 0110 0111 00000 00001
prob. 0.2 0.2 0.15 0.15 0.075 0.075 0.05 0.05 0.05
The expected length is 3 bits/symbol and the entropy is 2.96 bits/symbol. (c) Example: +1 − 1 − 3 + 2 0 0 0 + 4. The coded bit stream is 10, 11, 0110, 001, 00001, 110, 0111 where the commas have been inserted to indicated the correspondence of the bits to the symbols. Note that the symbol sequence 0 0 0 is encoded as the “start of run” symbol followed by the code for a run length of three. The bit stream can be uniquely decoded since the code is a prefix code.
78
(d) The expected length of this coding scheme is given by E[L] =
4
pi li + pSR LSR
i=−4
where the first term is the expected length of the code for nonzero symbols and the second term is the expected length of the code for zero runs. Note that this includes the length of the SR code. The result is the sum of the expected length from part (b) plus pSR times the expected length from part (a): E[L] = 3.0 + 0.05(1.984) = 3.1 bits/symbol. Without coding, the number of bits per symbol required is log2 (number of symbols) = 4 since there are 9 possible symbols. The 4 = 1.29. compression gain is thus 3.1
7.5 Pyramid coding (a) The expression for the power spectrum should be |X(ejω )|2 = e−3|ω|/π
|ω| < π.
(Note the absolute value.) Though not explicitly stated, we assume a cutoff frequency of π2 for the lowpass filter. By integrating the power spectrum we get π π 2 1 1 2 −3|ω|/π 1 2 jω 2 1 − e−3/2 ≈ 0.2590, and |X(e )| dω = e dω = σcoarse = 2 2π 0 π 0 3 π π 1 1 1 −3/2 2 e = 2 |X(ejω )|2 dω = e−3|ω|/π dω = 1 − e−3/2 ≈ 0.0578. σdifference 2π π π π 3 2
2
(The factors of 2 are to account for positive and negative frequencies.) (b) We are modelling the quantization of the coarse channel as the addition of a white noise with variance c∆2 where the noise is uncorrelated with x[n]. Assuming proper normalization of the interpolating filter, the noise power in the prediction is c∆2 . Since the noise is uncorrelated with x[n], the variance of the difference channel is 2 2 = σdifference + c∆2 , σdifference 2 is the variance calculated in part (a). where σdifference
(c) The allocation of bits between the coarse channel and the difference channel was investigated experimentally using MATLAB. The problem statement left room for experimentation. We chose to use a first-order Gauss-Markov source with √ correlation factor a generated by filtering a unit 1−a2 Uniform quantization was used without variance, white Gaussian source by the filter 1−az −1 . entropy coding. The quantizer design was based on a rule of thumb called the “4σ-rule”: Full-scale for the quantizer is set at ±4σ, where σ is the standard deviation of the source. Built-in MATLAB commands were used for generating the coarse channel and the prediction. The basic code follows: Bc = 4; Bd = 4; t = 8192; a = 0.99;
% % % % %
bits per coarse sample bits per difference sample Total bit rate is Bc/2 + Bd bits per sample size of data string correlation
% Generate source sequence
79
x(1) = randn; c = sqrt(1-a*a); for k=2:(t+1) x(k) = c*randn + a*x(k-1); end x = x(2:(t+1)); x_coarse = decimate(x, 2); sigma_c = sqrt( mean(x_coarse .^ 2) ); for i = 1:(t/2) x_coarse_q(i) = quantize( x_coarse(i), sigma_c, Bc ); end x_predict = interp(x_coarse_q, 2); x_diff = x - x_predict; sigma_d = sqrt( mean(x_diff .^ 2) ); for i = 1:t x_diff_q(i) = quantize( x_diff(i), sigma_d, Bd ); end x_recon = x_predict + x_diff_q; distortion = mean( (x - x_recon) .^ 2 );
The quantization function is given by: function y = quantize( x, sigma, bits ) % QUANTIZE Quantize using the 4-sigma rule % y = quantize( x, sigma, bits ) if bits == 0 y = 0; else delta = 8*sigma / 2^bits; y = delta * (floor(x/delta) + 0.5); if abs(y) > 4*sigma y = (2^(bits - 1) - 0.5)*delta*sign(y); end end
Figure P7.5(a) shows the results for coding of three sources with a total budget of 8 bits/sample. The correlation coefficient of the source is denoted by a. The horizontal axis shows the number of bits assigned per difference channel sample, Bd . Since there are half as many coarse channel samples, this leaves Bc = 16 − 2Bd bits per coarse channel sample. Figures P7.5(b), P7.5(c), and P7.5(d) gives results for 6, 4, and 2 bits/sample, respectively. A few general trends are of interest. The normalization is such that the process has unit variance for any a. This explains the fact that the for a fixed coding rate, the curves coincide when all the bits are used for the difference channel. As a is increased, the process becomes more lowpass, so more of the energy is in the coarse channel. Since the coarse channel has half the sample rate, this corresponds to better performance for larger a. Since the processes we are considering are (to a varying degree) lowpass, when the total coding rate is very low, the most effective strategy is to disregard the highpass information completely. This is illustrated in Figure P7.5(d).
7.6 Zero trees (a) Consider an octave-band decomposition as in Figure 3.7(a) applied to a finite-length one80
0
0
10
10
−1
a = 0.8
10
−1
a = 0.8
−2
a = 0.9
−3
a = 0.99
10
−2
10
MSE
MSE
a = 0.9 10
−3
10
a = 0.99
10 −4
10
−5
10
0
−4
1
2
3 4 5 Bits for difference channel (Bd)
6
7
10
8
0
(a) Total bit budget: 8 bits/sample
1
2 3 4 Bits for difference channel (Bd)
5
6
(b) Total bit budget: 6 bits/sample
0
0
10
10
a = 0.8 a = 0.8
a = 0.9
−1
10
MSE
MSE
a = 0.9
−1
10
a = 0.99
−2
10
a = 0.99
−3
10
0
−2
0.5
1
1.5 2 2.5 Bits for difference channel (Bd)
3
3.5
10
4
(c) Total bit budget: 4 bits/sample
0
0.2
0.4
0.6 0.8 1 1.2 1.4 Bits for difference channel (Bd)
1.6
1.8
2
(d) Total bit budget: 2 bits/sample
Figure P7.5: Experimental results for bit allocation in pyramid coding
dimensional signal. To maintain the analogy with the EZW algorithm, we will consider the independent variable to be a spatial variable. For a four-level decomposition, in analogy to Figure 7.25, the channels of the decomposition can be ordered as in Figure P7.6(a). Define a tree structure in Figure P7.6(a) to be a set of points in successively higher frequency bands, extending to the highest frequency band, which correspond to the same spatial region. Then we can define a zero tree to be a tree for which all the samples have the quantized value of zero. One possible zero tree is shown in Figure P7.6(b). In contrast to the two-dimensional case, where a zero tree grows as powers of four, the tree grows as powers of two. Also, there is only a single direction of tree growth, as opposed to the three shown in Figure 7.28. The only modification of the dominant and subordinate passes of the EZW algorithm is in the scanning. One simply scans one band (checking for possible zero trees) and then scans the next higher band. (b) The zero tree definition in part (a) applies to an arbitrary subband decomposition. In contrast to the octave band case, the points at a fixed tree depth may not be adjacent. Figure P7.6(c) shows a possible decomposition and zero tree. The points in the HL and HH bands are at the same tree 81
LLLH LLLL LLH
LH
H
Figure P7.6(a)
LLLH LLLL LLH c
LH
H
cc
cccc
Figure P7.6(b)
depth. Notice that the size and rate of growth of the tree are the same as in the octave-band case.
LLLH LLLL LLH c
LH
HL
cc
cc
HH cc
Figure P7.6(c)
The zero tree concept is most powerful when zero trees are most likely to be found. Most signals of interest have decaying spectra, ı.e. |X(ejω )| decreases with increasing |ω| for |ω| < π. For such signals, zero trees are most likely to be found with an octave-band decomposition. We will explain this by comparing the decompositions in Figures P7.6(b) and P7.6(c). Assume the original signal has length N . In the octave band case, the energy per coefficient in band H is given by 2 π 2 = |X(ejω )|2 . σH π π/2 For the decomposition of Figure P7.6(c), we have: 4 3π/4 2 = |X(ejω )|2 σHL π π/2 4 π 2 σHL = |X(ejω )|2 π 3π/4 2 2 < σ 2 . For a particular spatial location, Because of the decay of the power spectrum, σHL < σH HH one is more likely to find a nonzero coefficient in band HL or band HH than in band H. Therefore zero trees are more likely to be found in the octave band case. Please note that we are not claiming that an octave-band decomposition is always best. For a particular image, a decomposition that gives less zero trees may give better rate-distortion performance.
82
(c) Using zero trees in a full tree subband decomposition gives a different scan order than the zig-zag scanning used in DCT. In DCT coding, the scan order corresponds to ascending order of ω1 + ω2 , where ω1 and ω2 are the frequencies in the two spatial dimensions. In a zero tree structure (assuming Shapiro’s scan order), horizontal frequencies are scanned, followed by the vertical and 45◦ diagonal directions. This is illustrated through an example. Consider a full tree subband decomposition of depth 3 applied to a 32 × 32 image. The result is 64 channels with four samples per channel, as shown in Figure P7.6(d). The subbands are ordered such that frequencies increase from left to right and from bottom to top. The circles in Figure P7.6(d) show a potential zero tree in this decomposition. The scanning order has the following properties: • All spatial locations within a subband are scanned before moving to the next subband. • Searches for a zero tree are within a spatial region. • Scanning through the depth of a potential zero tree corresponds to frequencies with similar orientation. The frequency domain is effectively split into horizontal, vertical, and diagonal regions.
Figure P7.6(d) Now consider a DCT decomposition of the same image. In order to have the same (2 × 2) spatial resolution as in the subband decomposition, assume that 8 × 8 blocks are used. The zig-zag scanning order used with DCT coding is shown in Figure 7.15(a). This scanning has the following properties: • Each spatial region is coded independently. 83
• The scanning order is non-decreasing in ω1 + ω2 . • An end-of-block (EOB) symbol is used when the remaining samples in the scan have a quantized value of zero, so the coded samples (approximately) form a triangle. The relative efficacies of the two methods will depend on the power spectrum of the image. The DCT with zig-zag scan has the advantage that a single EOB symbol can be used to code all high frequency components, whereas at least three zero trees are needed to code all high frequency components. The shortcoming of the DCT method is that power spectra do not simply decay as a function of |ω1 + ω2 |. For example, an image block with many horizontal edges will have significant energy up to high vertical frequencies. For sake of illustration, suppose that the non-zero coefficients form an 8×3 rectangle. Then an EOB does not occur until the 48th symbol. In the subband coding case, many horizontal and diagonal frequency samples could be eliminated early on by zero trees.
7.7 Progressive to interlaced conversion (a) First note that for each filter in (7.4.3), the leftmost point corresponds to the origin. We will show that the filters in (7.4.3) form a perfect reconstruction analysis pair and find the corresponding synthesis pair through polyphase domain analysis. Taking (0, 0) and (1, 0) to be representatives of the cosets of the quincunx lattice, a polyphase representation of the analysis pair should take the following form: H0 (z1 , z2 ) = H00 (z1 , z2 ) + z1−1 H01 (z1 , z2 ) H1 (z1 , z2 ) = H10 (z1 , z2 ) + z1−1 H11 (z1 , z2 )
From this we can read off the polyphase components: H00 (z1 , z2 ) = −1(1 + z1−2 z2−2 + z1−2 z22 + z1−4 ) − 2(z1−1 z2−1 + z1−1 z2 + z1−3 z2−1 + z1−3 z2 ) + 28z1−2 H01 (z1 , z2 ) = 4(1 + z1−1 z2−1 + z1−1 z2 + z1−2 )
H10 (z1 , z2 ) = 1 + z1−1 z2−1 + z1−1 z2 + z1−2 H11 (z1 , z2 ) = −4
We can now find a synthesis pair through Gp (z1 , z2 )Hp (z1 , z2 ) = I. Note that since det Hp is a monomial, we get an FIR synthesis pair. (See also Problem 3.28.) Gp (z1 , z2 ) = [Hp (z1 , z2 )]−1 1 H11 −H01 = −H10 −H00 det Hp 1 H11 −H01 = −H10 −H00 −128z1−2 Thus the polyphase components of the synthesis pair are described by: 128G00 (z1 , z2 ) = −z12 H11 (z1 , z2 ) = 4z12 128G01 (z1 , z2 ) = z12 H10 (z1 , z2 )
= z12 + z1 z2−1 + z1 z2 + 1
128G10 (z1 , z2 ) = z12 H01 (z1 , z2 )
= 4(z12 + z1 z2−1 + z1 z2 + 1)
128G11 (z1 , z2 ) = −z12 H00 (z1 , z2 )
= (z12 + z2−2 + z22 + z1−2 ) + 2(z1 z2−1 + z1 z2 + z1−1 z2−1 + z1−1 z2 ) − 28 84
Using Gi (z1 , z2 ) = Gi0 (z1 , z2 ) + z1 Gi1 (z1 , z2 ), we find that the synthesis filters are as follows, where the origin is underlined: ⎞ ⎛ 1 1 ⎝ 1 4 1 0 ⎠ g0 [n1 , n2 ] = 128 1 ⎛
⎞ 1 ⎜ ⎟ 2 4 2 ⎟ 1 ⎜ ⎜ 1 4 −28 4 1 ⎟ g1 [n1 , n2 ] = ⎜ ⎟ 128 ⎝ ⎠ 2 4 2 1 (b) In Figure P7.7, the circles show the result of quincunx downsampling in the (time,vertical)plane. The sequence has only even lines at even times and only odd lines at odd times and hence is an interlaced sequence. After quincunx downsampling again, we are left with the points marked with squares. (Note that the second quincunx downsampling is with respect to the lattice formed by the circles.) The squares represent a progressive sequence with half as many frames and half as many (vertical) lines as the original sequence.
vertical
6
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
h
Figure P7.7: Cascaded quincunx downsampling
85
-
time
7.8 FCO downsampling (a) In analogy to (3.2.46), the filter pair corresponds to an orthogonal decomposition for FCO downsampling if H0 (z1 , z2 , z3 )H0 (z1−1 , z2−1 , z3−1 ) + H0 (−z1 , −z2 , −z3 )H0 (−z1−1 , −z2−1 , −z3−1 ) = 2.
(15)
Checking this is simply a matter of substitution: H0 (z1 , z2 , z3 )H0 (z1−1 , z2−1 , z3−1 ) + H0 (−z1 , −z2 , −z3 )H0 (−z1−1 , −z2−1 , −z3−1 ) 1 1 1 1 1 1 1+ 1− + √ (1 − z1 z2 z3 ) √ = √ (1 + z1 z2 z3 ) √ z1 z2 z3 z1 z2 z3 2 2 2 2 1 1 1 1 + = 2 + z1 z2 z3 + 2 − z1 z2 z3 − 2 z1 z2 z3 2 z1 z2 z3 = 2 The full justification of checking the condition (16) follows directly from part (b). We call it a “Haar” decomposition because of the similarity of the lowpass filter 1 h0 [n1 , n2 , n3 ] = √ (δ[n1 , n2 , n3 ] + δ[n1 + 1, n2 + 1, n3 + 1]) 2 to (3.1.16). (b) Recall the two-channel analysis/synthesis system shown in Figure 3.1(a). It can be shown that 1 1 ˆ X(z) = [G0 (z)H0 (z) + G1 (z)H1 (z)]X(z) + [G0 (z)H0 (−z) + G1 (z)H1 (−z)]X(−z). 2 2
(16)
This is an alternative version of (3.2.14) which clearly shows that (3.2.15)-(3.2.16) are necessary and sufficient for perfect reconstruction. Now consider a system as in Figure 3.1(a) which processes three-dimensional signals and uses FCO downsampling and upsampling. We would like to find a relationship analogous to (16). Denote the upsampled version of y0 by y0u . Then because of the FCO sampling, (h0 ∗ x)[n1 , n2 , n3 ] n1 + n2 + n3 even, (17) y0u [n1 , n2 , n3 ] = 0 otherwise. Taking the z-transform of (17) gives (h0 ∗ x)[n1 , n2 , n3 ]z1−n1 z2−n2 z3−n3 Y0u (z1 , z2 , z3 ) = n1 +n2 +n3 even 1 = 1 + (−1)n1 +n2 +n3 (h0 ∗ x)[n1 , n2 , n3 ]z1−n1 z2−n2 z3−n3 2 n1 ,n2 ,n3 ∈Z ⎛ 1⎝ (h0 ∗ x)[n1 , n2 , n3 ]z1−n1 z2−n2 z3−n3 = 2 n1 ,n2 ,n3 ∈Z ⎞ + (h0 ∗ x)[n1 , n2 , n3 ](−z1 )−n1 (−z2 )−n2 (−z3 )−n3 ⎠
Z
n1 ,n2 ,n3 ∈
=
1 (H0 (z1 , z2 , z3 )X(z1 , z2 , z3 ) + H0 (−z1 , −z2 , −z3 )X(−z1 , −z2 , −z3 )) , 2 86
(18)
where in (18), 1 + (−1)n1 +n2 +n3 is introduced to cancel terms with n1 + n2 + n3 odd. Defining y1u to be upsampled version of y1 , we similarly find Y1u (z1 , z2 , z3 ) =
1 (H1 (z1 , z2 , z3 )X(z1 , z2 , z3 ) + H1 (−z1 , −z2 , −z3 )X(−z1 , −z2 , −z3 )) . 2
Thus ˆ 1 , z2 , z3 ) = (19) X(z 1 [G0 (z1 , z2 , z3 )H0 (z1 , z2 , z3 ) + G1 (z1 , z2 , z3 )H1 (z1 , z2 , z3 )] X(z1 , z2 , z3 ) + 2 1 [G0 (z1 , z2 , z3 )H0 (−z1 , −z2 , −z3 ) + G1 (z1 , z2 , z3 )H1 (−z1 , −z2 , −z3 )]X(−z1 , −z2 , −z3 ). 2 Notice that (19) is the same as (16) with z replaced by z1 , z2 , z3 and −z replaced by −z1 , −z2 , −z3 . This is due to the particular type of downsampling. In two dimensions, the analogous relationship holds if quincunx downsampling is used.
7.9 Filtering of wide-sense stationary processes (a) By making repeated use of the linearity of expectation, we can make the following calculation: KY [n, m] = cov(y[n], y[m]) = E ((y[n] − Ey[n])(y[m] − Ey[m])) = E ((y[n] − mY )(y[m] − mY )) = E (y[n]y[m]) − m2Y where mY denotes Ey[n], which is independent of n ⎞ ⎛ ∞ ∞ h[i]x[n − i] h[j]x[m − j]⎠ − m2Y = E⎝ ⎛ = ⎝ ⎛ = ⎝
i=0
j=0
∞ ∞ i=0 j=0 ∞ ∞
⎞
h[i]h[j]E (x[n − i]x[m − j])⎠ − m2Y ⎞
⎛ ⎞ ∞ ∞ h[i]h[j]E (x[n − i]x[m − j])⎠ − ⎝ h[i]E (x[n − i]) h[j]E (x[m − j])⎠
i=0 j=0
=
∞ ∞
i=0
j=0
h[i]h[j] (E (x[n − i]x[m − j]) − E (x[n − i]) E (x[m − j]))
i=0 j=0
=
=
∞ ∞ i=0 j=0 ∞ ∞
h[i]h[j]KX [(n − i) − (m − j)] h[i]h[j]KX [(n − m) − (i − j)]
i=0 j=0
Since KY [n, m] is a function of (n − m) only, {y[n]} is wide-sense stationary. Denoting n − m by k gives ∞ ∞ h[i]h[j]KX [k − (i − j)]. KY [k] = i=0 j=0
(b) KXY [, n] = cov(x[], y[n]) = E ((x[] − Ex[])(y[n] − Ey[n])) 87
= E (x[]y[n]) because we have assumed a zero-mean input ∞ h[k]x[n − k] = E x[] ∞
=
k=0 ∞
=
k=0 ∞
=
k=0
h[k]E (x[]x[n − k]) h[k]KX [(n − k) − ] h[k]KX [(n − ) − k]
k=0
Denoting n − by m gives KXY [m] =
∞
h[k]KX [m − k].
k=0
(c) Let
x[n] =
x ˜[n] n ≥ 0 , 0 n<0
where x ˜[n] is a wide-sense stationary process. Let y[n] be a filtered version of x[n]. For n ≥ 0, ∞ h[k]x[n − k] E(y[n]) = E = E =
n
k=0 n
h[k]x[n − k]
because x[n] = 0 for n < 0
k=0
h[k]Ex[n − k]
k=0
=
n k=0
= mX
h[k]mX where mX denotes Ex[n], which is independent of n n
h[k].
(20)
k=0
Following (7.A.7), it was shown that when the input is
a∞(two-sided) wide-sense stationary process, the output has constant expected value equal to mX k=0 h[k]. In contrast, (20) shows that when the input is a one-sided wide-sense stationary process, E(y[n]) generally depends on n. If the processing is by an FIR filter of length L + 1,
min(n, L)
E(y[n]) = mX
h[k],
k=0
so E(y[n]) is constant for n ≥ L. For FIR or IIR filtering, limn→∞ E(y[n]) = mY , where mY is the constant expected value that would result from filtering the two-sided x ˜[n].
88
