Nonconvex Optimization via Joint Norm Relaxed SQP and Filled Function Method with Application to Minimax Two-Channel Linear Phase FIR QMF Bank Design - Advances in Heuristic Signal Processing and Applications - page 6

then the constraint on the maximum stopband ripple magnitude of the prototype
filter can be expressed as

¯ s (ω) ¯ −

≤

¯ ∀

∈

¯ s (ω) ω B s . (1.28)

Let us define

¯ b ≡[ ¯ , ¯ ] (1.29)

and

T ,

¯ b ≡[ ε a ,ε p ,ε s ] (1.30)

in which ¯ 3 identity matrix, then the specifications on the acceptable

bounds on the maximum amplitude distortion of the filter bank, the maximum pass-
band ripple magnitude, and the maximum stopband ripple magnitude of the proto-
type filter can be expressed as

is the 3 ×

¯ b ¯ − ¯ b ≤ ¯ . (1.31)

In order to minimize a weighted sum of the maximum amplitude distortion of the
filter bank, the maximum passband ripple magnitude, and the maximum stopband
ripple magnitude of the prototype filter subject to the specifications on these perfor-
mances, the filter bank design problem is formulated as the following optimization
problem:
Problem ( ¯ )

γι s ) T ¯ ,

≡

(αι a + βι p + min

¯

f( ¯ ) (1.32)

subject to

1
2 ¯

T Q
¯

ι a ¯ −

g 1 ( ¯ ,ω) ≡ (ω) ¯ − 1 ≤ 0 ∀ ω ∈[− π,π ] , (1.33)

1
2 ¯

T ¯ (ω) ¯ −

ι a ¯ +

≡− ≤ ∀ ∈[− ]

g 2 ( ¯ ,ω) 1 0 ω π,π , (1.34)

g 3 ( ¯ ,ω) ≡ ¯ p (ω) ¯ − ¯ p (ω) ≤ ¯ ∀ ω ∈ B p , (1.35)

g 4 ( ¯ ,ω) ≡ ¯ s (ω) ¯ − ¯ s (ω) ≤ ¯ ∀ ω ∈ B s , (1.36)

and

g 5 ( ¯ ) ≡ ¯ b ¯ − ¯ b ≤ ¯ , (1.37)

where α , β , and γ are the weights of different criteria for formulating the objective
function, f( ¯ ) is the objective function, and g 1 ( ¯ ,ω) , g 2 ( ¯ ,ω) , g 3 ( ¯ ,ω) , g 4 ( ¯ ,ω) ,
and g 5 ( ¯ ) are the constraint functions of the optimization problem.
As the set of the filter coefficients satisfying the constraints (1.33) and (1.34)
is nonconvex, the optimization problem is a nonconvex optimization problem. In
general, it is difficult to find the global minimum of the nonconvex optimization
problem.