Nonlinear Digital Filters: Analysis and Applications 0123725364, 9780123725363, 9780080550015

Citation preview

Nonlinear Digital Filters

This page intentionally left blank

Nonlinear Digital Filters Analysis and Applications

Wing-Kuen Ling

AMSTERDAM • BOSTON • HEIDELBERG • LONDON • NEW YORK • OXFORD PARIS • SAN DIEGO • SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Academic Press is an imprint of Elsevier

Academic Press is an imprint of Elsevier Linacre House, Jordan Hill, Oxford, OX2 8DP 84 Theobald’s Road, London WC1X 8RR, UK 30 Corporate Drive, Burlington, MA 01803 525 B Street, Suite 1900, San Diego, California 92101-4495, USA First edition 2007 Copyright © 2007 Wing-Kuen Ling. Published by Elsevier Ltd. All rights reserved The right of Wing-Kuen Ling to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permission may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (+44) (0) 1865 843830; fax (+44) (0) 1865 853333; email: [email protected]. Alternatively you can submit your request online by visiting the Elsevier web site at http://elsevier.com/ locate/permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress ISBN 13: 9-78-0-12-372536-3 ISBN 10: 0-12-372536-4

For information on all Academic Press publications visit our web site at books.elsevier.com

Typeset by Charon Tec Ltd (A Macmillan Company), Chennai, India, www.charontec.com Printed and bound in Great Britain 07

08

09 10

11

10

9

8

7

6

5

4

3

2 1

Contents Preface

vii

Chapter 1 Introduction Why are digital filters associated with nonlinearities? Challenges for the analysis and design of digital filters associated with nonlinearities An overview

1 1 6 7

Chapter 2 Reviews Mathematical preliminary Backgrounds on signals and systems Backgrounds on sampling theorem Backgrounds on bifurcation theorem Absolute stability theorem Exercises

8 8 13 25 27 28 28

Chapter 3 Quantization in Digital Filters Model of quantizer Quantization noise analysis Optimal code design Summary Exercises

32 32 36 42 51 52

Chapter 4 Saturation in Digital Filters System model Oscillations of digital filters associated with saturation nonlinearity Stability of oscillations of digital filters associated with saturation nonlinearity Summary Exercises

53 53 54

Chapter 5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic System model Linear and affine linear behaviors Limit cycle behavior Chaotic behavior Summary Exercises v

58 59 60 61 61 63 70 75 77 77

vi

Contents

Chapter 6

Step Response of Digital Filters with Two’s Complement Arithmetic Affine linear behavior Limit cycle behavior Fractal behavior Summary Exercises

78 78 92 104 109 109

Chapter 7

Sinusoidal Response of Digital Filters with Two’s Complement Arithmetic No overflow case Overflow case Summary Exercises

114 114 127 138 138

Chapter 8 Two’s Complement Arithmetic in Complex Digital Filters First order complex digital filters Second order complex digital filters Summary Exercises

139 139 144 150 151

Chapter 9

Quantization and Two’s Complement Arithmetic in Digital Filters Nonlinear behavioral differences of finite and infinite state machines Nonlinear behavior of unstable second order digital filters Nonlinear behaviors of digital filters with arbitrary orders and initial conditions Summary Exercises Properties and Applications of Digital Filters with Nonlinearities Admissibility of symbolic sequences Statistical property Computer cryptography via digital filters associated with nonlinearities Summary Exercises

152 152 156 162 170 171

Chapter 10

172 172 179 188 195 197

Further Reading

199

Index

205

Preface

Digital filters may be one of the most important building blocks in electrical and electronic engineering. They are widely employed in signal processing, communications, control, circuits design, electrical engineering and biomedical engineering communities. However, as digital filters are linear time invariant systems, any nonlinear behaviors occur in digital filters should be avoided. The first observed nonlinear phenomenon was the limit cycle behavior discovered in 1965 by implementing a digital filter via a finite state machine. Since then, engineers have tried to avoid the occurrence of the limit cycle behavior. In 1988, L. O. Chua and T. Lin (Chaos in Digital Filters, IEEE Transactions on Circuits and Systems, Vol. 35, no. 6, pp. 648–658) observed that besides the occurrence of the limit cycle behavior, digital filter may exhibit fractal behaviors if implemented via the two’s complement arithmetic. This observation implies that digital filters associated with nonlinearities may exhibit chaotic behaviors and this property may be utilized in some applications. While avoiding the occurrence of nonlinear behaviors, engineers began investigating the applications of digital filters with nonlinearities and found that many applications – such as computer cryptography – secure communications, etc. Hence, the subject of nonlinear digital filters plays an increasingly important role in electrical and electronic engineering. However, most nonlinearities associated with digital filters are discontinuous. For examples, quantization, saturation and two’s complement arithmetic, all involve discontinuous nonlinear function. The analysis of systems with discontinuous nonlinearities is difficult and not many existing techniques can be applied for the analysis of these systems. Hence, the objective of this book is to introduce techniques for the analysis of digital filters with various nonlinearities as well as to explore applications using digital filters associated with nonlinearities. From Chapter 3 through to Chapter 9, techniques for the analysis of digital filters associated with various nonlinearities will be introduced. In Chapter 10, application of digital filters associated with nonlinearities is explored. I believe that this book would be very useful for those engineers who deal with nonlinearities of digital filters. Also, it starts from very simple and fundamental techniques (initially introduced in first or second year courses) and is suitable vii

viii

Preface

for both the third year undergraduate and postgraduate student. In order to improve the readability of this book, examples are presented in each section or subsection and these examples directly illustrate the main concepts found in that section or subsection. Wing-Kuen Ling

This page intentionally left blank

In addition to the material covered in the text, Matlab source codes for the figures can be found on the textbook website: http://books.elsevier.com/companions/ 798123725363.

1 INTRODUCTION

WHY ARE DIGITAL FILTERS ASSOCIATED WITH NONLINEARITIES? Nonlinearities are associated with digital filters mainly for implementation reasons and are tailor-made for many applications. Nonlinearities due to implementation reasons The most common nonlinearities associated with digital filters due to implementation reasons are quantization, saturation and two’s complement. Quantization occurs because of the finite word length effects. Saturation occurs because of a constraint being imposed on the maximum bound of signals. Two’s complement operation occurs because of the overflow of signals to their sign bits. Although most computers these days are more than 64 bits—and floating point arithmetic is employed for the implementation—the cost of using computers to implement a simple digital filter is very high. Thus, many simple digital filters are still implemented using very simple circuits or microcontrollers because of their low cost. In these situations, only 8, or even lower, bits fixed point arithmetic are employed for the implementation. As a result, effects due to the quantization, saturation and two’s complement would be significant. Hence, the analysis and design of digital filters under these nonlinearities are important. Quantization Quantization is a nonlinear map that partitions the whole space and represents all of the values in each subspace by a single value. For example, for real input signals, if the input to the quantizer is nonnegative, then the output of the quantizer is represented by the value ‘1’, and ‘–1’ for other values. In this example, the set of real numbers is partitioned into two subsets, nonnegative and negative. ‘1’ and ‘–1’ are used for the representation of all values in these two subsets. It is worth noting that quantization is a noninvertible map. Hence, once 1

2

1 Introduction

0.8

1

0.6

0.6

0.8

0.4

0.4

0.2

0.2

0

0.2

0.4

0.4

0.6

0.6

(a)

0 0.5 Input

0.8 1 0.5

1 (b)

0.4

0

0.2

0.8 1 0.5

0.6

Output

Output

Output

0.8

0.2 0 0.2 0.4 0.6 0.8

0 0.5 Input

1 1 0.5

1

0 0.5 Input

1

(c)

Figure 1.1 Input output relationships of 4 bit (a) Lloyd Max quantizer with Gaussian input statistics, (b) μ law quantizer with μ = 100 and (c) uniform quantizer.

a quantization is applied, information is lost and error would be introduced. As a result, one of the most important issues in quantization is to minimize the quantization error. Quantization can be classified as uniform quantization and nonuniform quantization. Uniform quantization partitions the whole space in a uniform manner, and vice versa for the nonuniform quantization. The most common nonuniform quantizers are the Lloyd Max quantizer and the μ law quantizer, as shown in Figure 1.1a and 1.1b, respectively. It can be seen from this figure that the quantization step sizes are unevenly distributed, while that of the uniform quantizer shown in Figure 1.1c is evenly distributed. Another type of classification of quantization is based on the number of subspaces that are partitioned. For an N bit quantization, the whole space can be partitioned into 2N subspaces. In Figure 1.2, three 4-bit quantizers are shown, so there are exactly 16 quantization levels in each of the quantizers. In general, more bits of the quantizers would give less quantization error. However, the implementation complexity would be increased. Quantizers can also be classified as midrise or midthread quantizer. A midrise quantizer is the one that has a transition at the origin, and vice versa for the midthread quantizer. Figure 1.2a and 1.2b show the midrise and midthread quantizers, respectively. Saturation Saturation maps the whole space within a bounded subspace. The boundary of the bounded subspace is characterized by the saturation level. For example, an

3

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2 Output

Output

Why are Digital Filters Associated with Nonlinearities?

0

0

0.2

0.2

0.4

0.4

0.6

0.6

0.8

0.8

1 1

0.5

0

0.5

1 1

1

0.5

Input (a)

0

0.5

1

Input (b)

Figure 1.2 Input output relationships of 4 bit (a) midrise quantizer and (b) midthread quantizer.

output of a saturator is 1 and –1 if the input is greater than 1 and smaller than −1, respectively, and the saturation level of this saturator is 1. Figure 1.3 shows the input output relationship for this saturator. Two’s complement Two’s complement partitions the whole space into periodic subspaces and maps all subspaces into a single subspace. For example, the set of real numbers is divided into subsets with periodic 2 and all real values are mapped to values between –1 and 1 as shown in Figure 1.4. Nonlinearities due to tailor-made applications Digital filters are widely used in many applications in signal processing, communications, control, electrical and biomedical systems. For examples, coding and compression, denoising, signal enhancement, feature detection and extraction, amplitude and frequency demodulations, the Hilbert transform, analog-todigital conversions, differentiation, accumulation or integration, etc., all involve digital filters. For some applications, nonlinearities are tailor-made to fit for a particular purpose. Denoising application Figure 1.5b shows an image corrupted by an additive white Gaussian noise. The mean square error of the noisy image is 1605.8382. Figure 1.5c shows a

4

1 Introduction

1 0.8 0.6 0.4

Output

0.2 0 0.2 0.4 0.6 0.8 1 2

1.5

1

0.5

0

0.5

1

1.5

2

Input Figure 1.3 Input output relationship of a saturator with saturation level equal to 1.

1 0.8 0.6 0.4

Output

0.2 0 0.2 0.4 0.6 0.8 1 5

4

3

2

1

0

1

2

3

Input Figure 1.4 Input output relationship of a two’s complement nonlinearity.

4

5

5

Why are Digital Filters Associated with Nonlinearities?

50

50

50

100

100

100

150

150

150

200

200

200

250

250

250

300

300

300

350

350

350

400

400

400

450

450

450

500

500 100

200

300

400

(a)

500 100

500

200

300

400

500

(b)

100

200

300

400

500

(c)

Figure 1.5 (a) Original image, (b) image corrupted by an additive white Gaussian noise and (c) image after lowpass filtering.

lowpass filtered image. The mean square error of the filtered image drops to 167.7439. This example illustrates that lowpass filtering can reduce an additive white Gaussian noise effectively. Another method for reducing additive white Gaussian noise is via the wavelet denoising approach. In this approach, signals are decomposed into different scales via a wavelet transform and wavelet coefficients are set to zero if their magnitudes are smaller than a certain threshold. It was found that this nonlinear technique can reduce additive white Gaussian noise effectively. Coding application Another application for imposing quantization and saturation intentionally in signal processing is coding and compression processes. Figure 1.6 shows

0.0367bpp

0.0524bpp

0.1442bpp

0.2808bpp

0.03738bpp

0.05368bpp

0.1443bpp

0.2822bpp

Figure 1.6 Quantized images.

6

1 Introduction

some compressed images at different bit rates via quantization. After applying quantization, these images can be transmitted and stored efficiently.

CHALLENGES FOR THE ANALYSIS AND DESIGN OF DIGITAL FILTERS ASSOCIATED WITH NONLINEARITIES A nonlinear system is said to be exhibiting:

• a limit cycle behavior if it exhibits a nontrivial periodic output behavior • a fractal behavior if there is a self-similar geometric pattern exhibited on the phase plane and this self-similar geometric pattern is repeated at ever smaller scales to produce irregular shapes and surfaces that cannot be represented by classical geometry • an irregular chaotic behavior if it is sensitive to its initial condition, a state trajectory is dense and consists of dense periodic orbits, but fractal patterns do not exhibit on the phase plane • a nonlinear divergent behavior if some state variables tend to infinity but the corresponding linear part is strictly stable. Although linear system theories are reasonably well-developed, these theories cannot be applied when trying to explain the above phenomena. This is because nonlinear systems are highly dependent on initial conditions and system parameters, while these properties are not found in linear systems. For nonlinear systems, it is useful to characterize the set of initial conditions and system parameters such that these phenomena would be utilized or avoided. For example, in audio applications, limit cycles correspond to annoying audio tones. Hence, it should be avoided. Moreover, nonlinear divergent behaviors should also be avoided because circuits may be damaged and serious disaster may occur. In secure communications, fractal and chaotic behaviors may be preferred because they correspond to a rich frequency spectrum. Analyzing the stability property of nonlinear systems is also very challenging. Although Lyapunov stability theorem is powerful, it does not explain limit cycle, fractal and chaotic behaviors. Also, Lyapunov stability theorem requires a smooth Lyapunov candidate, which is difficult to find when the nonlinear function is discontinuous. In sigma delta modulation, digital filters are usually designed in an unstable manner in order that a high signal-to-noise ratio can be achieved. In this case, it is very challenging to design a digital filter such that the stability of the system is guaranteed and limit cycle behavior is avoided.

An Overview

7

AN OVERVIEW This book is organized as follows. In Chapter 2, fundamentals of mathematics, digital signal processing and control theory, used throughout the book, are reviewed. These include linear algebra, fuzzy theory, sampling theorem, bifurcation theorem and absolute stability theorem. From Chapters 3 to 9, digital filters associated with different nonlinearities are discussed. In Chapter 3, digital filters associated with the quantization nonlinearity are covered, and models for the quantization nonlinearity are introduced. Based on these quantization models, methods for improving the signal-to-noise ratios are presented. In Chapter 4, digital filters associated with the saturation nonlinearity are considered. Since oscillations may sometimes occur, the conditions for the occurrence of these oscillations and the stability conditions are presented, which is useful for both utilizing and avoiding the occurrence of these oscillations. From Chapters 5 to 8, digital filters associated with the two’s complement arithmetic are discussed. Chapters 5, 6 and 7 cover autonomous, step and sinusoidal responses, respectively. In Chapter 8, complex digital filters associated with two’s complement are considered, and in Chapter 9, digital filters associated with both the quantization and two’s complement arithmetic are dealt with. Finally, in Chapter 10, applications of digital filters associated with nonlinearities are presented; in particular, this chapter examines the applications on secure communications and computer cryptography.

2 REVIEWS

MATHEMATICAL PRELIMINARY Eigen decomposition Suppose an n × n matrix A has n linear independent eigenvectors, denoted as ξi for I = 1, 2, . . . , n and the corresponding eigenvalues are λi . Denote T ≡ [ξ1 , ξ2 , . . . , ξn ] and D ≡ diag(λ1 , λ2 , . . . , λn ). Then A is diagonalizable and A = TDT−1 . By employing the eigen decomposition, it can facilitate the evaluation of a power of a matrix. For example, if A has n linear independent eigenvectors, then Ak = TDk T−1 . If ∃ j ∈ {1, 2, . . . , n} such that |λj | > 1, then some of the elements in limk→+ ∞ Ak would be unbounded. Hence, the BIBO stability condition of a discrete time linear system becomes all eigenvalues confined inside the unit circle.

Inverse of a map A map F : X → Y is said to be injective if ∃x1 , x2 ∈ X and y ∈ Y such that F(x1 ) = F(x2 ) = y, then x1 = x2 . A map F : X → Y is said to be surjective if ∀y ∈ Y , ∃x ∈ X such that F(x) = y. A map is said to be bijective if it is both injective and surjective. A map is invertible if and only if it is bijective.

Fuzzy theory Fuzzy set In traditional set theory, an element is either in or not in a set A, that is x ∈ A or x∈ / A. This kind of set is called a crisp set. A fuzzy set is a set that is characterized 8

Mathematical Preliminary

9

by a fuzzy membership function μA (x) ∈ [0,1]. If μA (x) = 0, it implies that x ∈ / A. On the other hand, if μA (x) = 1, then x ∈ A. Common fuzzy membership functions The most common fuzzy membership functions are impulsive fuzzy  1 x = x0 membership function μA (x) ≡ , triangular fuzzy member0 otherwise ⎧ x−x0 ⎪ ⎨ a + 1 x0 − a ≤ x ≤ x0 ship function μA (x) ≡ x0b−x + 1 x0 < x ≤ x0 + b where a, b > 0, right sided ⎪ ⎩ 0 otherwise ⎧ 1 x ≥ x0 ⎨ x−x0 + 1 x − a ≤ x < x0 trapezoidal fuzzy membership function μA (x) ≡ 0 ⎩ a 0 otherwise where ⎧ a > 0, left sided trapezoidal fuzzy membership function x ≤ x0 − a ⎨ 1 x −x x0 − a < x ≤ x0 where a > 0, and Gaussian fuzzy memberμA (x) ≡ 0 a ⎩ 0 otherwise −

ship function μA (x) ≡ e

(x − x0 )2 2σ 2

.

Fuzzy operations In the traditional binary logics, all combinational logics can be represented by combinations of complement, intersection and union of binary variables because all Boolean equations can be represented by sum of products or product of sums. Hence, there are three fundamental operations in fuzzy logics and these operations represent the traditional complement, intersection and union operations. For the fuzzy complement operations, the fuzzy set A maps to a fuzzy set A with the fuzzy membership function μA (x) ≡ c(μA (x)) satisfying c(0) = 1, c(1) = 0 and ∀x1 , x2 ∈ [0,1] c(x1 ) > c(x2 ) for x1 < x2 . The most common fuzzy complement operations are the Sugeno operaa tion cλ (a) ≡ 11+−λa for λ > −1λ > −1 and ∀a ∈ [0, 1], and Yager operation 1

cw (a) ≡ (1 − aw ) w for w > 0 and ∀a ∈ [0,1].

For the fuzzy intersection operations, the fuzzy sets A and B map to a fuzzy set A ∩ B with the fuzzy membership function μA∩B (x) ≡ t(μA (x), μB (x)) satisfying t(0, 0) = 0, t(1, a) = t(a, 1) = a ∀a ∈ [0, 1], t(a, b) = t(b, a) ∀a, b ∈ [0, 1], t(t(a, b), c) = t(a, t(b, c)) ∀a, b, c ∈ [0, 1], and t(a, b) ≤ t(a , b ) for a ≤ a , b ≤ b and a, a , b, b ∈ [0, 1].

10

2 Reviews

The most common types of fuzzy intersection operations are the Dombi operation tλ (a, b) ≡ 1+



1

1 a

for λ > 0 and ∀a, b ∈ [0, 1],

λ  λ λ1 − 1 + b1 − 1

the Dubois-Prade operation tα (a, b) ≡

ab max(a, b, α)

for α ∈ [0, 1] and ∀a, b ∈ [0, 1],

the Yager operation   1

tw (a, b) ≡ 1−min 1, (1 − a)w + (1 − b)w w for w > 0 and ∀a, b ∈ [0, 1], the drastic product operation

⎧ ⎨a tdp (a, b) ≡ b ⎩ 0

b=1 a=1 otherwise

∀a, b ∈ [0, 1],

the Einstein operation tep (a, b) ≡

ab 2 − (a + b − ab)

∀a, b ∈ [0, 1],

algebraic product operation tap (a, b) ≡ ab

∀a, b ∈ [0, 1]

and the minimum operation tmin (a, b) ≡ min(ab) ∀a, b ∈ [0, 1]. For any fuzzy intersection operations t(a, b), it was found that tap (a, b) ≤ t(a, b) ≤ tmin (a, b) ∀a, b ∈ [0, 1]. For the fuzzy union operations, the fuzzy sets A and B map to a fuzzy set A ∪ B with the fuzzy membership function μA∪B (x) ≡ s(μA (x), μB (x)) satisfying s(1, 1) = 1, s(0, a) = s(a, 0) = a ∀a ∈ [0, 1], s(a, b) = s(b, a) ∀a, b ∈ [0, 1], s(s(a, b), c) = s(a, s(b, c)) ∀a, b, c ∈ [0,1], and s(a, b) ≤ s(a , b ) for a ≤ a , b ≤ b and a, a , b, b ∈ [0,1]. The most common types of fuzzy union operations are the the Dombi operation sλ (a, b) ≡ 1+



1

1 a

−λ  1 −λ − λ1 −1 + b −1

for λ > 0 and ∀a, b ∈ [0, 1],

the Dubois-Prade operation sα (a, b) ≡

a + b − ab − min(a, b, 1 − α) max(1 − a, 1 − b, α)

for ∀a ∈ [0, 1] and ∀a, b ∈ [0, 1],

Mathematical Preliminary

11

the Yager operation

  1

sw (a, b) ≡ min 1, aw + bw w

for w > 0 and ∀a, b ∈ [0, 1],

the drastic sum operation

⎧ ⎨a sds (a, b) ≡ b ⎩ 1

b=0 a=0 otherwise

∀a, b ∈ [0, 1],

the Einstein operation ses (a, b) ≡

a+b 1 + ab

∀a, b ∈ [0, 1],

algebraic sum operation sas (a, b) ≡ a + b − ab

∀a, b ∈ [0, 1]

and the maximum operation smax (a, b) ≡ max(a, b)

∀a, b ∈ [0, 1].

For any fuzzy union operations s(a, b), it was found that smax (a, b) ≤ s(a, b) ≤ sds (a, b) ∀a, b ∈ [0,1]. Hence, the fuzzy intersection and union operations cannot cover the interval [min(a, b), max(a, b)]. The operations that cover the interval [min(a, b), max(a, b)] are called the fuzzy averaging operations. The most common fuzzy averaging operations are the maxmin average operation vλ (a, b) ≡ λmax(a, b) + (1 − λ)min(a, b) for λ ∈ [0,1] and ∀a, b ∈ [0,1], generalized mean operations 1  α a + bα α να (a, b) ≡ 2

for α = 0 and ∀a, b ∈ [0, 1],

fuzzy “and” operations νp (a, b) ≡ p min(a, b) +

(1 − p)(a + b) 2

for p ∈ [0, 1] and ∀a, b ∈ [0, 1],

and fuzzy “or” operations (1 − γ)(a + b) for γ ∈ [0, 1] and ∀a, b ∈ [0, 1]. 2 Clearly, vλ (a, b) and vα (a, b) cover the whole interval [min(a, b), max(a, b)] as λ changing from 0 to 1 and α changing from −∞ to +∞, respectively.  On the other hand, vp (a, b) and vγ (a, b) cover the ranges min(a, b), a+b and 2

 a+b 2 , max(a, b) , respectively. νγ (a, b) ≡ γ max(a, b) +

12

2 Reviews

Fuzzy relations and fuzzy compositions A fuzzy relation Q in U1 × U2 × · · · × Un is defined as Q ≡ {(u1 × u2 × · · · × un ), μQ (u1 × u2 × · · · × un ) | (u1 × u2 × · · · × un ) ∈ U1 × U2 × · · · × Un }, where μQ : U1 × U2 × · · · × Un → [0, 1]. Suppose P(U, V ) and Q(V , W ) are two fuzzy relations. P ◦ Q is the composition of P(U, V ) and Q(V , W ) if and only if μP◦Q (x, z) = max t(μP (x, y), μQ (y, z)) ∀(x, z) ∈ U × W . The most y∈V

common fuzzy compositions are the maxmin composition μP◦Q (x, z) ≡ max min(μP (x, y), μQ (y, z)) and the max product composition μP◦Q (x, z) ≡ y∈V

max(μP (x, y), μQ (y, z)). y∈V

Linguistic variables If a variable can take words in natural languages as its values, it is called a linguistic variable, where the words are characterized by fuzzy sets. A linguistic variable is characterized by (X, T , U, M), where X is the name of the linguistic variable, T is the set of linguistic values that X can take, U is the actual physical domain in which the linguistic variable X takes its quantitative values, and M is a set of semantic rules which relates each linguistic values in T with a fuzzy set. For example, X is the speed of a car, T = {slow, medium, fast}, U = [0, VMAX ], and M consists of three fuzzy membership functions that describe slow, medium and fast. Fuzzy if then rules A fuzzy if then rule is a conditional statement expressed in the form of “IF fuzzy proposition 1 (FP1 ), then fuzzy proposition 2 (FP2 )”, where there are two types of fuzzy propositions, an atomic fuzzy proposition and a compound fuzzy proposition. An atomic fuzzy proposition is a single statement “x is A”, in which x is a linguistic variable and A is a linguistic value of x. A compound fuzzy proposition is a composition of atomic fuzzy propositions using the connectives “and”, “or” and “not” which represent fuzzy intersection, fuzzy union and fuzzy complement, respectively. For example, “x is A and x is not B”. The fuzzy if then rule is a fuzzy implication. The most common fuzzy implications are the Dienes-Rescher implication μQD (x, y) ≡ max(1 − μFP1 (x), μFP2 (y)), the Lukasiewicz implication μQL (x, y) ≡ min(1, 1 − μFP1 (x) + μFP2 (y)), the Zadeh implication μQZ (x, y) ≡ max(min(μFP1(x), μFP2 (y), 1 − μFP1 (x))),

Backgrounds on Signals and Systems

 the Gödel implication μQG (x, y) ≡

13

1 μFP1 (x) ≤ μFP2 (y) and μFP2 (y) otherwise

the Mamdani implications μQMM (x, y) ≡ min(μFP1 (x), μFP2 (y)) or μQMP (x, y) ≡ μFP1 (x)μFP2 (y). It was found that μQZ (x, z) ≤ μQD (x, z) ≤ μQL (x, z). Definitions and reasons for employing fuzzy systems Fuzzy systems are knowledge based or rule based systems which consist of fuzzy if then rules. The real world is too complicated that precise descriptions cannot be obtained. Therefore, approximation is required to model real systems. Moreover, in order to formulate human knowledge in a systematic manner and put it into engineering systems together with other information like mathematical models and sensory measurements, fuzzy systems are employed. Classifications and examples of fuzzy systems There are three common types of fuzzy systems, pure fuzzy systems, TakagiSugeno-Kang (TSK) fuzzy systems and fuzzy systems with fuzzifier and defuzzifier. The pure fuzzy systems map the input fuzzy sets to output fuzzy sets via a fuzzy inference engine which consists of a set of fuzzy if then rules. The TSK fuzzy systems formulate the fuzzy if then rules via a weighted average fuzzy inference engine. Fuzzy systems with fuzzifier and defuzzifier employ the fuzzifier to map the real valued variables into fuzzy sets and the defuzzifier transforms fuzzy sets into real valued variables. The most common fuzzy systems employed in industries are fuzzy washing machines, fuzzy image stabilizers, fuzzy car systems, etc.

BACKGROUNDS ON SIGNALS AND SYSTEMS Definition of signals Signals are usually used for describing physical phenomena. For examples, speech, music, text, image, video, electrocardiogram, electromyogram, genomic sequences, etc., all are signals. They are represented mathematically as functions of one or more independent variables. Consider a map x which maps the domain A to the range B, that is, x : A → B. Then x is a signal.

14

2 Reviews

Classifications of signals Continuous time and discrete time signals If A is a continuous set, then x is called a continuous time signal. For example, a one-dimensional real time function is a continuous time signal. This is because A is the set of real numbers and it is a continuous set. In this case, x :  → , denoted as x(t), where t ∈ , is a continuous time signal. On the other hand, if A is a discrete set, then x is called a discrete time signal. For example, a one-dimensional real time sequence is a discrete time signal. This is because A is the set of integers and it is a discrete set. In this case, x : Z → , denoted as x(n), where n ∈ Z, is a discrete time signal. Periodic and aperiodic signals If an element in A exists such that the signal repeats itself after that element, then the signal is called a periodic signal. For example, for a one-dimensional time function, if ∃T ∈  such that x(t) = x(t + T ) ∀t ∈ , then x(t) is periodic with period T . Since if ∃T ∈  such that x(t) = x(t + T ) ∀t ∈ , then x(t) = x(t + nT ) ∀t ∈  and ∀n ∈ Z. The minimum value of T ∈ + such that x(t) = x(t + T ) ∀t ∈  is called the fundamental period. Pure complex exponential signals, that is x(t) = ejω0 t for ω0 = 0, are periodic. This is because ejω0 t = ej(ω0 t+2πn) ∀n ∈ Z. Hence, ∃T = 2π ω0 ∈  such that x(t) = x(t + T ) ∀t ∈ . Similarly, for a one-dimensional time sequence, if ∃N ∈ Z such that x(n) = x(n + N) ∀n ∈ Z, then x(n) is periodic with period N. If a signal is not periodic, then it is called aperiodic signal. Odd and even signals Another type of classification of signals is based on the symmetric property of the signals. For a one-dimensional continuous time signal, if x(t) = −x(−t) ∀t ∈ , then x(t) is called an odd signal. On the other hand, if x(t) = x(−t) ∀t ∈ , then x(t) is called an even signal. Similarly, for a one-dimensional time sequence, if x(n) = −x(−n) ∀n ∈ Z, then x(n) is called an odd signal. On the other hand, if x(n) = x(−n) ∀n ∈ Z, then x(n) is called an even signal. It is worth noting that many signals are neither even nor odd. Common signals The most common signals employed for an analysis are impulse, step and pure complex exponential signals, denoted as δ(t), u(t) and ejω0 t for the continuous time case, respectively, and δ(n), u(n) and ejω0 n for the discrete  +∞ time case, respectively. Impulse signals are defined as δ(t) = 0 for t = 0 and −∞ δ(t)dt = 1

15

Backgrounds on Signals and Systems



1 n=0 for the continuous time case, and δ(n) = for the discrete time case. 0 n = 0  1 t≥0 Step signals are defined as u(t) = for the continuous time case and 0 t 0 such that

x(0) − x0  < r implies x(t) − x0  < R ∀t > 0 for the continuous time case, and ∀R > 0, ∃r > 0 such that x(0) − x0  < r implies x(n) − x0  < R ∀n > 0 for the discrete time case. A pendulum is an example of stable systems. • If the above condition is not satisfied, then the equilibrium state x0 is said to be unstable. An inverted pendulum is an example of unstable systems. • An equilibrium state x0 is said to be asymptotically stable if it is stable and ∃r  > 0 such that x(0) − x0  < r  implies x(t) − x0  → 0 as t → +∞ for the continuous time case, and ∃r  > 0 such that x(0) − x0  < r  implies x(n) − x0  → 0 as n → +∞ for the discrete time case. • An equilibrium state x0 is said to be exponentially stable if, for the continuous time case, ∃α, λ > 0 such that ∀t > 0 x(t) − x0  < αx0 e−λt for some ball Br around x0 , and for the discrete time case, ∃α, λ > 0 such that ∀n > 0 x(n) − x0  < αx0 e−λn for some ball Br around x0 .

Backgrounds on Signals and Systems

17

• If asymptotic stability holds for any initial states, the equilibrium point is said to be asymptotically stable in the large or globally asymptotically stable.

• Similarly, if exponential stability holds for any initial states, the equilibrium point is said to be exponential stable in the large or globally exponential stable. Digital filters Definition of digital filters Digital filters are discrete time systems that are characterized by their frequency responses. Classifications of digital filters Based on the frequency characteristics The most common method for the classification of digital filters is based on their frequency selectivity, such as grouping them as lowpass filters, highpass filters, allpass filters, bandpass filters, band rejected filters, notch filters, oscillators, etc. Lowpass filters allow the low frequency components of the input signals to pass through while the high frequency components are attenuated. Highpass filters are the opposite. Allpass filters allow all frequencies of the input signals to pass through. Bandpass filters allow a particular band of frequency of the input signals to pass through and the rest frequency components are attenuated. Band rejected filters are the opposite. Notch filters only reject a particular frequency of the input signals to pass through and allow the rest frequency components to pass through. Oscillators are the vice versa. Figure 2.1 shows the magnitude responses of various filters. Based on the impulse and frequency responses If the impulse response of a digital filter has finite support or finite length, then the digital filter is called the finite impulse response (FIR). Otherwise, it is called the infinite impulse response (IIR). If the transfer function of the digital filter is rational, then the digital filter is called rational. It is worth noting that IIR filters are not necessarily rational. For example, an ideal lowpass filter is not rational but it is IIR. If the impulse response of the digital filter is symmetric, then the digital filter is called symmetric. On the other hand, if the impulse response of the digital filter is antisymmetric, then the digital filter is called antisymmetric. It is worth noting that many digital filters are neither symmetric nor antisymmetric. If the phase response of the digital filter is linear, the digital filter is called the linear phase. Otherwise, it is called the nonlinear phase. All symmetric and antisymmetric digital filters are linear phase. It is worth noting

2 Reviews

1

0.5

0

0

2 Frequency v

4

(d)

1

0.5

0

0

2 Frequency v

4

(b)

1

0.5

0

0

2 Frequency v

4 (e)

1.5

1

0.5

0

0

2 Frequency v

4

0

2 Frequency v

4

(c) 1.5

Magnitude response |H(v)|

1.5

Magnitude response |H(v)|

Magnitude response |H(v)|

(a)

1.5

Magnitude response |H(v)|

1.5

Magnitude response |H(v)|

Magnitude response |H(v)|

18

1

0.5

0

0

2 Frequency v

4

1.5

1

0.5

0

(f)

Figure 2.1 Magnitude responses of: (a) a lowpass filter; (b) a bandpass filter; (c) an oscillator; (d) a highpass filter; (e) a band rejected filter; (f) a notch filter.

that not all FIR filters are linear phase and not all IIR filters are nonlinear phase. For examples, if the FIR filter is neither symmetric nor antisymmetric, then it is nonlinear phase. Also, if both the numerator and denominator of a rational IIR filter is linear phase, then the IIR filter is linear phase. Linear phase digital filters are used in many signal processing applications. In particular, they are extensively employed in image and video signal processing because images and videos are phase sensitive. Figure 2.2 shows both the impulse and frequency responses of a symmetric FIR filter. It can be seen from Figure 2.2a that the impulse response is symmetric and finite length, so the phase response is linear, as shown in Figure 2.2c. Figure 2.3 shows both the impulse and frequency responses of an antisymmetric FIR filter. It can be seen from Figure 2.3a that the impulse response is antisymmetric and finite length, so the phase response is also linear as shown in Figure 2.3c. Figure 2.4 shows both the impulse and frequency responses of an IIR filter. It can be seen from Figure 2.4a that the impulse response is infinite length. Figure 2.4c shows that the phase response is nonlinear.

19

Backgrounds on Signals and Systems 20

0.6 0.5

0

0.4

20

0 10

h(n)

0.3 0.2

40 60

0.1

80

0

100

0.1

120

0

20

40 60 Time index n

80

(a)

Phase of H(v) (in radians)

20 log10(|H(v)|)

20 30 40 50 60 70 80 90 0

1

2 Frequency

3

100

4

(b)

0

1

2 3 Frequency

4

(c)

Figure 2.2 (a) Impulse response; (b) magnitude response; (c) phase response of a symmetric FIR filter.

20

50

0.5 0.4

0

0.3

0 Phase of H(v) (in radians)

50 20 log10(|H(v)|)

0.2

h(n)

0.1 0 0.1 0.2

150 200 250

0.3

0

20

40

60

80

350

40 60 80

0

1

2

3

4

100

(b)

0

1

2

3

4

Frequency

Frequency

Time index n

(a)

20

300

0.4 0.5

100

(c)

Figure 2.3 (a) Impulse response; (b) magnitude response; (c) phase response of an antisymmetric FIR filter.

Characteristics of digital filters The frequency bands that allow and reject the input signals to pass through are called the passbands and the stopbands, respectively. The frequency bands between the passbands and stopbands are called the transition bands. The bandwidth of a digital filter is defined as the total width of all passbands. The maximum absolute difference between the magnitude response of the digital filters and the desired values in all passbands is called the passband ripple magnitude, and that in the stopbands is called the stopband ripple magnitude. A represented frequency in a transition band is called the cutoff frequency.

20

2 Reviews 0.5

0

0 20

0.4

5

20 log (|H(v)|)

60 80

10

h(n)

0.2

Phase of H(v) (in radians)

40 0.3

0.1 0

100 120 140

0.1 0.2

15

20

160 0

20

40

60

80

180

0

1

2

3

25

4

(b)

0

1

2

3

4

Frequency

Frequency

Time index n

(a)

10

(c)

Figure 2.4 (a) Impulse response; (b) magnitude response; (c) phase response of an IIR filter.

Impulse, step and frequency responses Definitions of impulse and step responses Impulse response of a system is defined as the output of a system when the input is an impulse and it is denoted as h(t) for the continuous time case and h(n) for the discrete time case. Similarly, step response of a system is defined as the output of a system when the input is a step and it is denoted as s(t) for the continuous time case and s(n) for the discrete time case. Relationships among impulse response, properties of linear time invariant systems and their input output relationships For linear and time invariant systems, denoted as LTI systems, the input– output relationship of the systems isgoverned by a convolution of their impulse +∞ responses and inputs, that is y(t) = −∞ x(τ)h(t − τ)dτ for the continuous time +∞ case and y(n) = k→− ∞ x(k)h(n − k) for the discrete time case. Also, the causality condition of an LTI system reduces to h(t) = 0 ∀t < 0 for the continuous time case and h(n) = 0 ∀n ≤ 0 for the discrete time case. Similarly, the strictly causality condition of an LTI system reduces to h(t) = 0 ∀t ≤ 0 for the continuous time case and h(n) = 0 ∀n ≤ 0 for the discrete time case. The memoryless condition of an LTI system reduces to h(t) = Kδ(t) for the continuous time case and h(n) = Kδ(n) for the discrete time case, where K is a + constant. The  +∞BIBO stability condition of an LTI system reduces to ∃M+ ∈  such  that −∞ |h(t)|dt < M for the continuous time case and ∃M ∈  such that ∀n |h(n)| < M for the discrete time case. The invertible condition of  +∞ an LTI system reduces to ∃g(t) such that −∞ g(τ)h(t − τ)dτ = δ(t) for the

Backgrounds on Signals and Systems

continuous time case and ∃g(n) such that discrete time case.



∀m

21

g(m)h(n − m) = δ(n) for the

Definitions of frequency responses If a pure complex exponential signal is input to an LTI system, then the output of the system is also a pure complex exponential signal with a gain in the magnitude and shift in the phase. The plot of the magnitude gain versus frequency, and that of the phase shift versus the frequency, is called the magnitude and phase responses, respectively. When these two responses are combined, it is called the frequency response and denoted as H(ω), where |H(ω)| is the magnitude response and ∠H(ω) is the phase response. Fourier analysis Definitions of continuous time Fourier transform, discrete time Fourier transform, and discrete Fourier transform The Fourier analysis evaluates signals and systems in the frequency  +∞ domain. Continuous time Fourier transform of x(t) is defined as X(ω) = −∞ x(t)e−jωt dt  and discrete time Fourier transform of x(n) is defined as X(ω) = ∀n x(n)e−jωn . It is worth noting that the discrete time Fourier transform is always 2π periodic, while this is not the case for the continuous time Fourier transform. Also, both the continuous time and discrete time Fourier transforms are defined in the frequency domain, which is a continuous domain. On the other hand, the  − j2πnk ˜ N discrete Fourier transform of x(n) is defined as X(k) = N1 N−1 for n = 0 x(n)e k = 0, 1, . . . , N − 1, in which the discrete Fourier transform is a map from an N point sequence to an N point sequence. The inverse continuous time Fourier 1 +∞ jωt transform of X(ω) is defined as x(t) = 2π −∞ X(ω)e  dω, the inverse discrete 1 π jωn dω, and the time Fourier transform of X(ω) is defined as x(n) = 2π −π X(ω)e N−1 j2πnk inverse discrete Fourier transform of  X(k) is defined as x(n) = k=0  X(k)e N for n = 0, 1, . . . , N − 1. For a continuous time periodic signal with period T , X(ω) would consist of impulses located at 2πk T ∀k ∈ Z. Hence, its time domain  j2πtk can be represented as continuous time Fourier series x(t) = ∀k ak e T , where  jk2πt T ak = T1 0 x(t)e− T dt ∀k ∈ Z are called the continuous time Fourier coefficients. Similarly, for a discrete time periodic signal with period N, X(ω) would consist of impulses located at 2πk N for k = 0, 1, . . . , N − 1 and 2π periodic elsewhere. Hence, its time domain can be represented as discrete time Fourier series  j2πnk 1 N−1 − jk2πn N , where ak = N x(n) = N−1 are called the discrete n = 0 x(n)e k = 0 ak e N time Fourier coefficients. It is worth noting that there are exactly N discrete time Fourier coefficients, while this is not the case for the continuous time Fourier coefficients.

22

2 Reviews

Relationships between Fourier coefficients and discrete Fourier transform Consider an N point discrete time signal x(n). Denote its discrete Fourier transform as  X(k) for k = 0, 1, . . . , N − 1. Since x(n) is finite support, it is aperiodic and its discrete time Fourier transform X(ω) is a smooth 2π periodic  function. By periodizing x(n) with period N, denoted as xp (n) = x(n) ∗ ∀k δ(n − kN), in which * denotes the convolution, xp (n) is periodic. Denote its discrete time Fourier as ak for k = 0, 1, . . . , N − 1. It can be shown

 series for k = 0, 1, . . . , N − 1. Also, as xp (n) is periodic, that ak =  X(k) = N1 X 2πk N its discrete time Fourier transform consists of impulses located at 2πk N for k = 0, 1, . . . , N − 1. It can be shown that the magnitudes of the impulses are 

2π 2πk  which are also equal to 2πak or 2πX(k) for k = 0, 1, . . . , N − 1. N X N Advantages of employing Fourier analysis The main advantage of employing the Fourier analysis for LTI systems is to evaluate the output of the systems via a multiplication instead of a convolution. By computing the Fourier transform of the input, output and the impulse response of the systems, denoted as X(ω), Y (ω) and H(ω), respectively, it can be shown that Y (ω) = X(ω)H(ω). Also, the invertibility of LTI systems can be concluded easily from H(ω). If ∃ω0 ∈  such that X(ω0 ) = 0, then the LTI systems are noninvertible. Laplace transform and z transform Definitions of Laplace transform and z transform  +∞ Laplace transform of x(t) is defined as X(s) = −∞ x(t)e−st dt and z transform  of x(n) is defined as X(z) = ∀n x(n)z−n . The inverse Laplace transform of  σ+j∞ 1 st X(s) is defined as x(t) = 2πj σ−j∞ X(s)e ds where σ is the real part of s. The 1 inverse z transform of X(z) is defined as x(n) = 2πj X(z)zn−1 dz. Advantages of employing Laplace transform and z transform As with Fourier transforms, for continuous time LTI systems, Y (s) = X(s)H(s) where X(s), Y (s) and H(s) are the Laplace transform of the input, output and impulse response of the systems, respectively, and for discrete time LTI systems, Y (z) = X(z)X(z) where X(z), Y (z) and H(z) are the z transform of the input, output and impulse response of the systems, respectively. H(s) and H(z) are called the transfer functions. By using the Laplace transform or z transform,

Backgrounds on Signals and Systems

23

the output of LTI systems can be evaluated via multiplication instead of convolution. Region of convergences and their properties Denote the region of convergence, or simply ROC, of H(s) as the set of s such that H(s) are analytically defined. Similarly, denote the ROC of H(z) as the set of z such that H(z) are analytically defined. If ROC of H(s) contains the entire imaginary axis, or in other words, the continuous time Fourier transform is analytically defined ∀ω ∈ , then the corresponding continuous time LTI system is BIBO stable. Similarly, if ROC of H(z) contains the entire unit circle, or in other words, the discrete time Fourier transform is analytically defined ∀ω ∈ [−π, π], then the corresponding discrete time LTI system is BIBO stable. State space representations Advantages of employing state space representations Since the transfer functions only reflect the input–output relationship of systems, internal properties of the systems are not reflected by the transfer functions. To have more information about the internal properties of systems, state space representation is employed. State space representations of various systems For a continuous time system, define x(t), u(t) and y(t) as the state vector, input and output of the system. The state space representation is to represent a system in the form of dtd x(t) = f (x(t), u(t), t) and y(t) = g(x(t), u(t), t). If the system is linear, then four time varying matrices A(t), B(t), C(t) and D(t) exist, such that dtd x(t) = A(t)x(t) + B(t)u(t) and y(t) = C(t)x(t) + D(t)u(t). If the system is LTI, then four constant matrices A, B, C and D exist, such that dtd x(t) = Ax(t) + Bu(t) and y(t) = Cx(t) + Du(t). And the solut tions of the LTI system are x(t) = e(t−t0 )A x(t0 ) + t0 e(t−τ)A Bu(τ)dτ ∀t ≥ t0 t and y(t) = Ce(t−t0 )A x(t0 ) + C t0 e(t−τ)A Bu(τ)dτ + Du(t) ∀t ≥ t0 . Similarly, for a discrete time system, define x(k), u(k) and y(k) as the state vector, input and output of the system. The state space representation is to represent a system in the form of x(k + 1) = f (x(k), u(k), k) and y(k) = g(x(k), u(k), k). If the system is linear, then four time varying matrices A(k), B(k), C(k) and D(k) exist, such that x(k + 1) = A(k)x(k) + B(k)u(k) and y(k) = C(k)x(k) + D(k)u(k). If the system is LTI, then four constant matrices A, B, C and D exist, such that x(k + 1) = Ax(k) + Bu(k) and y(k) = Cx(k) + Du(k). And the solutions of

24

2 Reviews

 n−k−1 Bu(k) ∀n ≥ n + 1 and the LTI system are x(n) = An−n0 x(n0 ) + n−1 0 k=n0 A  n−1 n−n0 n−k−1 x(n0 ) + C k=n0 A Bu(n) + Du(k) ∀n ≥ n0 + 1. y(n) = CA Relationships between state space representations and transfer functions of LTI systems The relationship between the transfer function and the state space matrices of LTI continuous time systems is H(s) = C(sI − A)−1 B + D, where I is an identity matrix. Similarly, the relationship between the transfer function and the state space matrices of LTI discrete time systems is H(z) = C(zI − A)−1 B + D. Realizations Different state space representations are called different realizations. It is worth noting that state space representations of systems are not uniquely defined. For LTI systems, different sets of state space matrices may correspond to the same transfer function, being related by similarity transform. Controllability and observability For a continuous time system, assume that x(0) = 0. ∀x1 , if ∃t1 > 0 and u(t) such that x(t1 ) = x1 , then the continuous time system is said to be reachable. Similarly, for a discrete time system, assume that x(0) = 0. ∀x1 , if ∃n1 > 0 and u(n) such that x(n1 ) = x1 , then the discrete time system is said to be reachable. For a continuous time system, if ∀x0 , x1 , ∃t1 > 0 and u(t) such that x(0) = x0 and x(t1 ) = x1 , then the continuous time system is said to be controllable. Similarly, for a discrete time system, if ∀x0 , x1 , ∃n1 > 0 and u(n) such that x(0) = x0 and x(n1 ) = x1 , then the discrete time system is said to be controllable. For LTI systems, the set of reachable state is R(|B AB · · · An B|), where R(A) is defined as the range of A, that is R(A) ≡ {y : y = Ax}. Also, the LTI systems are controllable if and only if R(A) = n . Or in other words, rank(|B AB · · · An B|) = n. For a continuous time system, ∀x1 , if ∃t1 > 0 and u(t) such that x1 can be determined from y(t) for t > t1 , then the continuous time system is said to be observable. Similarly, for a discrete time system, ∀x1 , if ∃n1 > 0 and u(n) such that x1 can be determined from y(n) for n > n1 , then the discrete time system is said to be observable. For LTI systems, the set of unobservable state is ⎛⎡ ⎤⎞ C ⎜⎢ CA ⎥⎟ ⎜⎢ ⎥⎟ N ⎜⎢ .. ⎥⎟ , ⎝⎣ . ⎦⎠ CAn−1

Backgrounds on Sampling Theorem

25

where N(A) is defined as the null space of the kernel of A, that is N(A) ≡ {x : Ax = 0}. Also, the LTI systems are observable if and only if ⎤⎞ ⎛⎡ ⎤⎞ ⎛⎡ C C ⎜⎢ CA ⎥⎟ ⎜⎢ CA ⎥⎟ ⎥⎟ ⎥⎟ ⎜⎢ ⎜⎢ N ⎜⎢ .. ⎥⎟ = {0}. Or in other words, rank ⎜⎢ .. ⎥⎟ = n. ⎝⎣ . ⎦⎠ ⎝⎣ . ⎦⎠ CAn−1 CAn−1

BACKGROUNDS ON SAMPLING THEOREM Relationships among continuous time signals, sampled signals and discrete time signals in the frequency domain Denote a continuous time signal as x(t) and sampling frequency as fs . Then the sampling period is f1s and the continuous time sampled sig

 nal is xs (t) = x(t) ∀n δ t − fns . By taking the continuous time Fourier  transform on this sampled signal, we have Xs (ω) = fs ∀n X(ω − 2πfs n). Since Xs (ω) is periodic with period 2πfs , if X(ω) is bandlimited within (−πfs , πfs ), then X(ω) can be reconstructed via a simple lowpass filtering with passband of the filter (−πfs , πfs ). Hence, if a signal is bandlimited by (−πfs , πfs ), fs is the minimum sampling frequency that can guarantee perfect reconstruction. is called the Nyquist frequency. As 

This frequency

  n n n x(t) ∀n δ t − fs = ∀n x fs δ t − fs , by taking continuous time Fourier  jωn   transform on both sides, we have fs ∀n X(ω − 2πfs n) = ∀n x fns e− fs . 

Denote a discrete time sequence as x fns , taking the discrete time Fourier  

transform on this discrete time sequence, we have XD (ω) = ∀n x fns e−jωn . 

 Hence, we have XD ωfs = Xs (ω) = fs ∀n X(ω − 2πfs n). Relationships among upsampled signals, downsampled signals and original signals in the frequency domain Denote a discrete time signal as x(n) and it is upsampled by an expander with upsampling ratio L. Then the upsampled signal is ⎧ n

⎨x n is a multiple of L L . xL (n) = ⎩ 0 otherwise

26

x [n]

2 Reviews

H0(z)

↓N

↑N

F0(z)

H1(z)

↓N

↑N

F1(z)

…

…

…

…

HN1(z)

↓N

↑N

FN1(z)



y [n]

Figure 2.5 A uniform filter bank.

By taking the discrete time Fourier transform on this upsampled signal, we have XL (ω) = X(Lω), or by taking the z transform, we have XL (z) = X(zL ). Similarly, if a discrete time signal x(n) is downsampled by a decimator with downsampling ratio M, then the downsampled signal is xM (n) = x(Mn). By taking the discrete time Fourier transform on this downsampled signal, we 

1 M−1 ω−2πk have XM (ω) = M k = 0 X , or by taking the z transform, we have  1

M j2π 1 M−1 XM (z) = M k = 0 X z M W k , where W ≡ e− M . Filter bank systems Consider a filter bank system shown in Figure 2.5. A filter bank system is said to be aliasing free if it can be represented by an LTI system and achieve perfect reconstruction is a pure delay gain of its input. Since  N−1 if its output k )H (zW k )F (z), the filter bank system achieves Y (z) = N1 N−1 X(zW i i i=0 k=0 aliasing free if and only if ∃T (z) such that ⎡ ⎤ ⎡ ⎤ ⎤⎡ H1 (z) ··· HN−1 (z) H0 (z) T (z) F0 (z) ⎢ H0 (zW ) ⎥ ⎢ ⎥ ⎢ H1 (zW ) ··· HN−1 (zW ) ⎥ ⎢ ⎥ ⎢ F1 (z) ⎥ ⎢ 0 ⎥ = ⎢ ⎥ ⎢ ⎥, ⎥ ⎢ .. .. . . . .. .. ⎣ ⎦ ⎣ .. ⎦ ⎣ .. ⎦ . . . H0 (zW N−1 )

H1 (zW N−1 )

· · · HN−1 (zW N−1 )

FN−1 (z)

0

and it achieves perfect reconstruction if and only if ∃c ∈  and ∃n0 ∈ Z such that T (z) = cz−n0 . It is worth noting that if both Hi (z) for i = 0, 1, . . . , N − 1 and Fj (z) for j = 0, 1, . . . , N − 1 are FIR, then perfect reconstruction condition requires ⎛⎡ ⎤⎞ H0 (z) H1 (z) ··· HN−1 (z) ⎜⎢ H0 (zW ) ⎟ H1 (zW ) ··· HN−1 (zW ) ⎥ ⎜⎢ ⎥⎟ det ⎜⎢ ⎥ ⎟ = c z−n1 , .. .. . . .. .. ⎝⎣ ⎦⎠ . . H0 (zW N−1 ) H1 (zW N−1 ) · · · HN−1 (zW N−1 )

27

Backgrounds on Bifurcation Theorem

x [n]

H0(z)

↓n0

↑n0

F0(z)

H1(z)

↓n1

↑n1

F1(z)

…

…

…

…

HN1(z)

↓nN1

↑nN1

FN1(z)



y [n]

Figure 2.6 A nonuniform filter bank.

v(k) = 0 

u(k)

G(z; μ)

y(k)



f(e(k); μ)

e(k)



d(k) = 0



Figure 2.7 Nonlinear feedback system.

where c ∈  and n1 ∈ Z. This kind of filter bank system is called the unimodular filter bank. Consider a nonuniform filter bank system as shown in Figure 2.6. In this  − j2π 1 ni −1 k k ni case, Y (z) = N−1 for i=0 ni k=0 X(zWni )Hi (zWni )Fi (z), where Wni ≡ e i = 0, 1, . . . , N − 1. For some set of decimation ratio, such as {2, 3, 6}, aliasing free condition cannot be achieved via non-ideal LTI filters. Hence, forget the perfect reconstruction. This kind of nonuniform filter bank is called an incompatible nonuniform filter bank.

BACKGROUNDS ON BIFURCATION THEOREM Suppose a discrete time system can be represented as x(k + 1) = Ax(k) + Bg(Cx(k); μ) and y(k) = Cx(k), where x(k) ∈ n , A ∈ nxn , B ∈ nxl , C ∈ mxn , y(k) ∈ m , g(·) : m ×  → l is a smooth (C r , r ≥ 3) l dimensional vector field, k ∈ Z + ∪ {0} is the iteration index, and μ ∈  is the bifurcation parameter. It is worth noting that all of the matrices may depend on μ, and A may be a zero matrix. By introducing an arbitrary matrix D ∈ lxm , which may also depend on μ, the system can be represented by a feedback system. The whole feedback system consists of the linear feedforward part G(z; μ) and the nonlinear feedback part f(e(k); μ) as shown in Figure 2.7, and the system can be described as x(k + 1) = Ax(k) + BDy(k) +

28

2 Reviews

 u(k)



y(k) F(z)

Q

Figure 2.8 A nonlinear feedback system.

B(g(Cx(k); μ) − Dy(k)) or x(k +1) = (A+BDC)x(k)+B(g(Cx(k); μ) −Dy(k)), G(z; μ) = C(zI−(A + BDC))−1 B, u(k)=f(e(k); μ) + v(k) =g(y(k); μ) − Dy(k) and e(k) = d(k) − y(k). If we assume that the reference input v(k) = 0 and the external disturbance d(k) = 0, supposing that the system achieves equilibrium at e(k) = ê, by linearizing the nonlinear feedback system f(e(k); μ) at the equilibrium point, then the open loop gain of the system is  ∂f(e(k))  G(z; μ)J(μ), where J(μ) ≡ ∂e(k) e(k)=ˆe is the Jacobian matrix. If there exists a simple pair of complex eigenvalues of G(z; μ)J(μ) for z = ejω crossing the critical point −1 + 0j for some values of ω0 and a given parameter μ = μ0 , then the system exhibits the Hopf bifurcation.

ABSOLUTE STABILITY THEOREM Consider the nonlinear feedback system shown in Figure 2.8. If Q(y) is a piecewise continuous integrable function, Q(0) = 0, ∃K ≥ 0 such that dQ(y)   K ≥ Q(y) y ≥ 0 ∀y ∈ \{0}, ∃K ∈  such that dy ≤ K ∀y ∈ \{yi } where yi are the discontinuous boundaries, F(z) is a rational causal transfer function without any poles outside the unit circle, the loop filter is realized in a controllable and observable form, as well as ∃q ≥ 0 and ∃δ > 0 such that  Re{(1 + q(z − 1))F(z)} + K1 − K 2|q| |(z − 1)F(z)|2 ≥ δ ∀|z| = 1, then the nonlinear feedback system is BIBO stable.

EXERCISES

n 1. Evaluate cos1 θ sin0 θ for θ = kπ, where k ∈ Z and n ∈ Z + . 2. Show that y = sin x for x!∈ [−π, π] and y ∈ [−1, 1] is noninvertible, while y = sin x for x ∈ − π2 , π2 and y ∈ [−1, 1] is invertible. 3. Show that for any fuzzy intersection operations t(a, b), tdp (a, b) ≤ t(a, b) ≤ tmin (a, b) ∀a, b ∈ [0, 1].

Exercises

29

4. Show that for any fuzzy union operations s(a, b), smax (a, b) ≤ s(a, b) ≤ sds (a, b) ∀a, b ∈ [0,1]. 5. ∀a ∈ [0, 1], plot the following fuzzy complement operations cλ (a) for λ = 0.5 and cw (a) for w = 0.5. 6. ∀a, b ∈ [0, 1], plot the following fuzzy intersection operations tλ (a, b) for λ = 0.5, tα (a, b) for α = 0.5, tw (a, b) for w = 0.5, tdp (a, b), tep (a, b), tap (a, b) and tmin (a, b). 7. ∀a, b ∈ [0, 1], plot the following fuzzy union operations sλ (a, b) for λ = 0.5, sα (a, b) for α = 0.5, sw (a, b) for w = 0.5, sds (a, b), ses (a, b), sas (a, b) and smax (a, b). 8. ∀a, b ∈ [0, 1], plot the following fuzzy averaging operations vλ (a, b) for λ = 0.5, vα (a, b) for α = 0.5, vp (a, b) for p = 0.5 and vγ (a, b) for γ = 0.5. 9. ∀μFP1 (x), μFP2 (y) ∈ [0, 1], plot the fuzzy implications μQD (x, y), μQL (x, y), μQZ (x, y), μQG (x, y), μQMM (x, y) and μQMP (x, y). 10. Show that μQZ (x, z) ≤ μQD (x, z) ≤ μQL (x, z) ∀μFP1 (x), μFP2 (y) ∈ [0, 1]. 11. Show that a discrete time complex exponential signal x(n) = ejω0 n is periodic if and only if ω0 is rational multiple of π. 12. Consider a second order digital filter with the transfer function H(z) ≡ cz−1 + dz−2 , where a, b, c, d ∈ . Show that the set of filter coefficients 1−az−1 − bz−2 which guarantees the BIBO stability of H(z) is  ≡ {(a, b) : 1 + a − b > 0, b + a − 1 < 0 and b > −1}. 13. Show that the relationships  t between impulse signals  and step signals are governed by u(t) = −∞ δ(τ)dτ and u(n) = nk →−∞ δ(k) for the continuous time and discrete time cases, respectively. 14. For an LTI system, show  +∞ that the input output relationship of the system is governed by y(t) = −∞ x(τ)h(t − τ)dτ for the continuous time case and  y(n) = +∞ k →−∞ x(k)h(n − k) for the discrete time case. 15. For an LTI system, show that the causality condition of an LTI system reduces to h(t) = 0 ∀t < 0 for the continuous time case and h(n) = 0 ∀n < 0 for the discrete time case. 16. For an LTI system, show that the strictly causality condition of an LTI system reduces to h(t) = 0 ∀t ≤ 0 for the continuous time case and h(n) = 0 ∀n ≤ 0 for the discrete time case. 17. For an LTI system, show that the memoryless condition of an LTI system reduces to h(t) = Kδ(t) for the continuous time case and h(n) = Kδ(n) for the discrete time case, where K is a constant. 18. For an LTI system, show that theBIBO stability condition of an LTI system +∞ reduces to ∃M ∈ + such that −∞ |h(t)|dt < M for the continuous time  + case and ∃M ∈  such that ∀n |h(n)| < M for the discrete time case. 19. For an LTI system, show that  +∞the invertible condition of an LTI system reduces to ∃g(t) such that −∞ g(τ)h(t − τ)dτ = δ(t) for the continuous

30

20.

21.

22. 23. 24. 25.

26. 27. 28.

29. 30.

31.

2 Reviews

 time case and ∃g(n) such that ∀m g(m)h(n − m) = δ(n) for the discrete time case. For an LTI system, show that if a pure complex exponential signal with frequency ω0 is inputted to an LTI system with frequency response H(ω), then the output is also a pure complex exponential signal with magnitude gain |H(ω0 )| and phase shift ∠H(ω0 ).  +∞ For continuous time aperiodic signals, show that −∞ |x(t)|2 dt =  +∞ 1 2 2π −∞ |X(ω)| dω.For a continuous time periodic signals, show that +∞ 1 2 2 ∀k |ak | , where ak ∀k are the continuous time T −∞ |x(t)| dt = Fourier coefficients. time aperiodic signals, show that  +∞ For discrete  1 2 2 ∀n |x(n)| = 2π −∞ |X(ω)| dω. For a discrete time periodic signals,  N−1 2 2 show that N1 N−1 n = 0 |x(n)| = n = 0 |ak | , where ak ∀k are the discrete time Fourier coefficients. Plot the frequency response of the digital filters with impulse response 0 n) h(n) = sin(ω and g(n) = (−1)n h(n) for |n| ≤ 30. πn For an LTI system, show that Y (ω) = X(ω)H(ω), Y (s) = X(s)H(s) and Y (z) = X(z)H(z). Show that if ∃ω0 ∈  such that H(ω0 ) = 0, then the LTI systems are noninvertible. For a system governed by constant coefficients ordinary differential equaM−1 d m u(t)  d n y(t) tion N−1 n=0 an dt n = m=0 bm dt m , where N ≥ M. Assume the system is initially at rest, find the impulse and frequency responses of the system. Show that if ROC of H(s) contains the entire imaginary axis, then the corresponding continuous time LTI system is BIBO stable. Show that if ROC of H(z) contains the entire unit circle, then the corresponding discrete time LTI system is BIBO stable. Show that the solutions of the continuous time LTI system are x(t) = t e(t−t0 )A x(t0 ) + t0 e(t−τ)A Bu(τ)dτ ∀t ≥ t0 and y(t) = Ce(t−t0 )A x(t0 ) +  t (t−τ)A C t0 e Bu(τ)dτ + Du(t) ∀t ≥ t0 , and that for the discrete time LTI  n−k−1 Bu(k) ∀n ≥ n + 1 and system are x(n) = An−n0 x(n0 ) + n−1 0 k=n0 A  n−1 y(n) = CAn−n0 x(n0 ) + C k=n0 An−k−1 Bu(n) + Du(k) ∀n ≥ n0 + 1. Show that for a strictly causal LTI system, the state space matrix D = 0. Show that for two different realizations of an LTI discrete time system x(k + 1) = Ax(k) + Bu(k) and y(k) = Cx(k) + Du(k), and xˆ (k + 1) = ˆ x(k) + Bu(k) ˆ ˆ x(k) + Du(k), ˆ Aˆ and y(k) = Cˆ if ∃T such that xˆ (k) = T−1 x(k), find the relationship between these two sets of state space matrices. Show that for LTI systems, the set of reachable state is R(|B AB · · · An B|). Also, the LTI systems are controllable if and only if R(A) = n or rank(|B AB · · · An B|) = n.

Exercises

31

32. Show ⎛⎡ that for ⎤⎞ LTI systems, the set of unobservable state is C ⎜⎢ CA ⎥⎟ ⎜⎢ ⎥⎟ N⎜⎢ .. ⎥⎟. Also, the LTI systems are observable if and only if ⎝⎣ . ⎦⎠ CAn−1 ⎛⎡ ⎤⎞ ⎛⎡ ⎤⎞ C C ⎜⎢ CA ⎥⎟ ⎜⎢ CA ⎥⎟ ⎜⎢ ⎥⎟ ⎥⎟ ⎜⎢ N⎜⎢ .. ⎥⎟ = {0} or rank ⎜⎢ .. ⎥⎟ = n. ⎝⎣ . ⎦⎠ ⎝⎣ . ⎦⎠

CAn−1 CAn−1 33. Consider a continuous time signal x(t) sampled by a periodic signal with nonzero DC value. Show that if f1s is the fundamental period of the periodic signal and X(ω) is bandlimited within (−πfs , πfs ), then X(ω) can be reconstructed via a simple lowpass filtering with passband of the filter (−πfs , πfs ). 34. Find an impulse response of a general two channel uniform filter bank with impulse responses of analysis filters h0 (n) and h1 (n), that of synthesis filters f0 (n) and f1 (n), and the sampling ratio of decimators and expanders equal to 2. 35. For a general two channel uniform filter bank with frequency responses of analysis filters H0 (ω) and H1 (ω), that of synthesis filters F0 (ω) and F1 (ω), and the sampling ratio of decimators and expanders equal to 2, if the input of system is A sin(ω0 n), find the discrete time Fourier transform of the output of the system.

3 QUANTIZATION IN DIGITAL FILTERS

Quantization is widely employed in many signal processing applications, such as in data compression and analog-to-digital conversion. However, as quantization is not a reversible process (it uses a many-to-one mapping approach), signals cannot be perfectly reconstructed after the quantization. As a result, efficient methods for the reduction of the quantization noise would be useful for many signal processing applications. In this chapter quantization and the method used for the reduction of quantization noise is presented.

MODEL OF QUANTIZER Modeling the quantizer as an independent additive white Gaussian noise source Although quantization is a memoryless system, it involves discontinuous nonlinearity. Hence, many existing system theories cannot be applied for analyzing digital filters associated with quantization. In order to tackle this problem, quantization noise is usually assumed to be a wide sense stationary white process with each input sample being uniformly distributed over the range of the quantization error. For an N bit uniform midrise quantizer with saturation level equal to L, the quantization step size is  ≡ 2N−1L− 0.5 . Denote e(n) and p(e(n)) as the quantization error and the probability density function of the quantization error, respectively. Since it is assumed that the quantization noise is a wide sense stationary white process with each sample being uniformly distributed over the range of the quantization error, the total expected noise energy is 

2  2 1 L σe2 ≡ −2 (e(n))2 p(e(n))de(n) =  = 12 3 2N − 1 . 2

Example 3.1 Assume the quantization noise of 8 bit uniform midrise quantizer with saturation level equal to 1 is a wide sense stationary white process with each sample being 32

33

Model of Quantizer

uniformly distributed over the range of the quantization error. Find the step size and the expected noise energy. Solution: 2 The step size and the expected noise energy is  ≡ 28−1 1− 0.5 = 509 and 

2 1 σe2 = 13 281−1 = 195075 , respectively.

Modeling the quantizer as a polynomial of input Consider an N-bit uniform antisymmetric quantizer with the quantization range [−L, L], that is:   ⎧ |μ(n)| 1 ⎪ ⎨sign(μ(n)) ceil |μ(n)| ≤ L −  2 Q(μ(n)) ≡ , ⎪ ⎩ |μ(n)| > L sign(μ(n))L

(3.1)

Where μ(n) is the input of the quantizer, ⎧ ⎨ μ(n) sign(μ(n)) ≡ |μ(n)| ⎩ 0

μ(n) = 0

, ceil(μ(n))

μ(n) = 0

denotes the rounding operator towards the plus infinity, |·| denotes the absolute operator, and  ≡ 2N−1L− 0.5 is the step size of the quantizer. It is worth noting that the Taylor expansion is not generally used to approximate the quantization function as a polynomial function. This is because the derivative of the midrise quantization function does not exist at the origin. In order to approximate the quantization function as a polynomial function, a polynomial fitting technique is employed. Since the quantization function is an odd function, all coefficients associated to the even power terms are zero and the polynomial with only odd power terms is enough for the approximation. Denote 3 · · · (μ(n))2M−1 ]T and p ≡ [p T μ(n) ≡ [μ(n) (μ(n)) 1 · · · pM ] , where the superscript T denotes the transpose operator, pm ∈  for m = 1, 2, . . . , M and 2M − 1 are, respectively, the coefficients and the order of the polynomial, in which  denotes the set of real numbers. Since all of the coefficients of the model are real, the approximated nonlinear model is a real model and p can be found via solving the optimization problem with the objective being to minimize the total absolute square difference between the actual quantizer and

34

3 Quantization in Digital Filters

A 2-bit quantizer An approximated 2-bit quantizer

1.5

1

1

Quantizer outputs

Quantizer outputs

A 1-bit quantizer An approximated 1-bit quantizer

1.5

0.5 0 0.5 1 1.5 1

0.5

(a)

0

0.5

0 0.5 1 1.5 1

1

Quantizer input

0.5

(b)

0

0.5

1

Quantizer input A 4-bit quantizer An approximated 4-bit quantizer

A 3-bit quantizer An approximated 3-bit quantizer

1.5

1.5 1

Quantizer outputs

Quantizer outputs

0.5

0.5 0 0.5 1 1.5 1

0.5

(c)

0

Quantizer input

0.5

1 0.5 0 0.5 1 1.5 1

1 (d)

0.5

0

0.5

1

Quantizer input

Figure 3.1 Input–output relationships of actual quantizers with L = 1 and the approximated quantizers μT (n)p with M = 10 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

the approximated polynomial function subject to the constraint on the absolute difference. Define the bound on this absolute difference as ε 2 , where ε > 0. The optimization problem can be expressed as follow: "L min p

|μ(n)T p − Q(μ(n))|2 dμ(n)

(3.2a)

−L

subject to

|μT (n)p − Q(μ(n))| ≤

ε 2

∀μ(n) ∈ [−L, L].

(3.2b)

This optimization problem is actually a semi-infinite programming problem and can be solved via the dual parameterization method. Figure 3.1 shows examples of input output relationships of actual quantizers with L = 1 and the approximated quantizers μT(n)p with M = 10 for N = 1, N = 2, N = 3 and N = 4. Figure 3.2 shows the corresponding differences, that is Q(μ(n)) − μ(n)T p. It can be seen from Figure 3.2 that the solutions of the semi-infinite programming problems exist for N = 1, 2, 3, 4 when M = 10.

35

0.5 0 0.5 1 1

Difference between two 3-bit quantizer outputs

(a)

(c)

Difference between two 2-bit quantizer outputs

1

0.5

0

0.5

1

Quantizer input 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4 1

0.5

0

0.5

Quantizer input

1 0.5 0 0.5 1 1

1 (d)

0.5

0

0.5

1

Quantizer input

(b) Difference between two 4-bit quantizer outputs

Difference between two 1-bit quantizer outputs

Model of Quantizer

0.2 0.1 0 0.1 0.2 1

0.5

0

0.5

1

Quantizer input

Figure 3.2 The differences between the actual quantizers with L = 1 and the approximated quantizers μT (n)p with M = 10 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

According to Figure 3.1, although the approximated models for high bit quantizers and that for the one bit quantizer for relatively small input signal look like a linear model, a linear model is not appropriate for the approximation of the quantizers. The approximated models obtained via solving the semiinfinite programming problems are derived from the linear model as pm = 0 for m = 2, 3, . . . , M and for N = 1, 2, 3, 4. It can be seen from Figure 3.2 that the largest error occurs at the neighborhood of discontinuous points of the actual quantizer. For N = 1, the discontinuous point of the actual quantizer is at the origin. If the magnitude of input signals is small and the linear model is applied, then the difference between the output of the actual quantizer and that based on the linear model will be very large. For the high bit quantizers, the approximated quantizer models are not as good as that compared to the case when N = 1, as shown in Figure 3.2. This is because the same value of M is employed for the illustration. Since the quantization step size is so small that the magnitudes of input signals fall into the neighborhood of the discontinuous points of the actual quantizer, the approximated models based on the same value of M is not as accurate as that compared to the case when N = 1. In this case, the value of M should be increased such that the magnitudes of the input signals do not fall into the neighborhood of the discontinuous points of the actual quantizer.

36

3 Quantization in Digital Filters

Although the accuracy of the model decreases for the same value of M when the number of bits of quantizer increases, the same value of M is used for an illustration. This is because the coefficients of the model are obtained based on solving the semi-infinite programming problem. If a solution for the semiinfinite programming problem exists, then the maximum absolute quantization  error is guaranteed to be bounded by a constant 2ε multiplying the quantization step. In other words, the ratio of the maximum absolute quantization error to the quantization step is bounded by 2ε (when the solution of the semi-infinite programming problem exists). Moreover, a too large value of M would result in aliasing. This is explained in more detail in Section 3.2. Actually, if the magnitude of the input signal falls outside the neighborhood of the discontinuous points of the actual quantizers, then the differences between the approximated models and the actual quantizers will be small. In this case, the approximated model would be very accurate. This fact is illustrated in Figure 3.2.

Example 3.2 Using the polynomial fitting technique, find N coefficients which approximates sin(x) for −π < x < π. Solution: 2i−1 . The approximation coefficients can be found via Let y = N i=1 ai x π solving the optimization problem min ( y − sin (x))2 dx. Denote (a1 ,a2 ,...,aN ) −π π π x ≡ [x x 3 · · · x 2N−1 ]T , Q ≡ 2 −π xxT dx, b ≡ −2 −π sin(x)xT dx and a ≡ [a1 a2 · · · aN ]T . If Q is a full rank matrix, then a = −Q−1 b. It is worth noting that there is no simple relationship between the Taylor coefficients and the coefficients obtained via the polynomial fitting technique.

QUANTIZATION NOISE ANALYSIS Consider the systems shown in Figure 3.3. Figure 3.3a is a conventional quantization system. If an input signal is oversampled with an oversampling  ratio (OSR) R, then u(n) is bandlimited within the frequency spectrum − πR , πR . After applying the quantization, if H(ω) is an ideal lowpass filter, that is  1 |ω| < πR H(ω) = , then the high frequency component of the quan0 otherwise tization noise is removed and a high signal-to-noise ratio (SNR) can be achieved.

37

Quantization Noise Analysis

u(n)

Q

(a)

u(n) Q

s1(n)

H (v)

s2(n)

y1(n)

H (v)

y2(n)

Na

(b)

a(n)  兺 ak cos(kv0n) k 1

Figure 3.3 (a) Conventional system; (b) quantized system with periodic codes.

The quantization noise of these two systems can now be analyzed via the approximated model. This is done by replacing the actual quantizer Q(μ(n)) by the approximated quantizer μT (n)p. Denote the Fourier transform of u(n), s1 (n), s2 (n), y1 (n) and y2 (n), respectively, as U(ω), S1 (ω), S2 (ω), Y1 (ω) and Y2 (ω). Denote the periodic code as a(n) and its coefficients as ak for k = 1, 2, . . . , Na , where Na is the length of the code. Denote periodic code as ω0 . Hence, a(n) ≡ Na the fundamental frequency of the  a Denote  A(ω) ≡ N k=1 ak cos(kω0 n). k=1 ak (δ(ω − kω0 ) + δ(ω + kω0 )),  U2m−1 (ω) ≡ U(ω) ∗ · · · ∗ U(ω) and A2m−1 (ω) ≡  A(ω) ∗ · · · ∗  A(ω), where ∗ denotes the convolution operator and there are 2m − 1 terms in both U2m−1 (ω) and  A2m−1 (ω). Denote the DFT matrix with size (4mNa + 1) × (4mNa + 1) as W4mNa +1 , and its tth row qth column element as W4mNa +1 (t, q). Denote the inverse of W4mNa +1  4mNa +1 and its tth row qth column element as W 4mNa +1 (t, q). as W Denote

 am

≡

4mN a +1 # k=1

⎛ 4mN a +1 (2mN a + 1, k) ⎝ W

Na #

(W4mN a +1 (k, Na + 1 − q)

q=1

$2m + W4mN a +1 (k, Na + 1 + q))aq

.

38

3 Quantization in Digital Filters

Theorem 3.1 Assume that T

Q(μ(n)) ≈ μ (n)p, H(ω) =

⎧ ⎨1 ⎩

π R , otherwise |ω|
   dω.     2 22m

k=1

− πR

m=2

− πR

m=2

Then the SNR of the coded system shown in Figure 3.3b will be higher than that of the conventional system shown in Figure 3.3a. Proof: If Q(μ(n)) ≈ μT (n)p, then for the conventional system shown in Figure 3.3a, we have: Y1 (ω) = H(ω)S1 (ω) ≈ H(ω)

M #

pm U2m−1 (ω).

(3.3)

m=1

If we regard the first order term (m = 1) in the signal band as the signal component and all higher order terms (m ≥ 2) in the signal band as the quantization noise, then the SNR can be estimated as follows:  πR 2 − πR |H(ω)p1 U(ω)| dω SNR ≈ 10 log10  π  . (3.4) 2 M  R  m=2 pm U2m−1 (ω) dω − π H(ω) R

Since H(ω) is an ideal lowpass filter, then we have: π p21 −R π |U(ω)|2 dω R . SNR ≈ 10 log10  π  2    M R − π  m=2 pm U2m−1 (ω) dω

(3.5)

R

Now consider the coded system shown in Figure 3.3b.   2m−1 (ω) ≡ U(ω) ∗ · · · ∗ U(ω), Denote the input to the quantizer as  u(n) and U U(ω) ∗  A(ω)   where there are 2m − 1 terms in U2m−1 (ω). Since U(ω) = , we have 2 U2m−1 (ω) ∗  A2m−1 (ω) T  . As we assume that Q(μ(n)) ≈ μ (n)p, so we U2m−1 (ω) = 22m−1

M  pm U2m−1 (ω) ∗  A2m−1 (ω) and have S2 (ω) ≈ m=1 2m−1 2  M # A2m (ω) pm U2m−1 (ω) ∗  . (3.6) Y2 (ω) ≈ H(ω) 22m m=1

39

Quantization Noise Analysis

As a result,  2  p U(ω) ∗  A (ω)  H(ω) 1 22 2  dω . 

2 M  p U (ω) ∗   A (ω)  H(ω) m=2 m 2m−122m 2m  dω 

π R

− πR

SNR ≈ 10 log10 

π R

− πR

(3.7)

and u(n) is bandlimited, terms in U2m−1 (ω) ∗  A2m (ω) for Since ω0 ≥ (2M−1)π R m = 1, 2, . . . , M do not overlap each other in the frequency spectrum. As H(ω) is an ideal lowpass filter, eqn (3.7) can be further simplified as:

SNR ≈ 10 log10 

p21 4

π R

− πR



Na 2 k=1 ak

2 

π R

|U(ω)|2 dω . 

2 M  p U (ω)∗  A (ω)  H(ω) m=2 m 2m−122m 2m  dω − πR

(3.8)

Denote a vector a ≡ [a1 · · · aNa ]T and  am ≡ [fliplr(aT ) 0 aT 0(4m−2)Na ]T , T where fliplr(a ) ≡ [aNa · · · a1 ] and 0(4m−2)Na denotes a zero row vector with length (4m − 2)Na . Denote the discrete Fourier transform of  am as aˆ m and its kth element as aˆ m (k). Denote a vector am with its kth element am (k) as am (k) ≡ (ˆam (k))2m .  and its kth element Denote the inverse discrete Fourier transform of am as am (2M−1)π  (k). Then a (2mN + 1) = a . Since ω ≥ , u(n) is bandlimited as am a 0 m m R and H(ω) is an ideal lowpass filter, p21 4

SNR ≈ 10 log10 

π R



− πR

Na 2 k=1 ak

2 

π R

|U(ω)|2 dω .   

  M pm am U2m−1 (ω) 2 dω  m=2  22m − πR

(3.9)

2

2  π     πR M pm am U2m−1 (ω) 2  M R  a 2 If 21 N a p U (ω) dω >   dω,  π m 2m−1 m=2 k=1 k − R − πR  m=2 22m then the SNR of the coded system will be larger than that of the conventional system. This completes the proof.

40

3 Quantization in Digital Filters

If the periodic code only consists of a single coefficient, then the coded system will become a modulated system. Denote the factorial operator as !, then we have the following corollary: Corollary 3.1 Assume that



Q(μ(n)) ≈ μT (n)p,

a1 = 1, ak = 0 for k = 1 and

2m

r=0 r =m

1 |ω| < πR , ω0 ≥ (2M R− 1)π , 0 otherwise (2m)! 2m)! r!(2m−r)! > (m!)2 for m = 1, 2, . . . , M, then

H(ω) =

the SNR of the modulated system will be higher than that of the conventional system. Proof:  Since 22m = 2m r=0

2m (2m)! (2m)! (2m)! for r=0 r!(2m−r)! > r!(2m−r)! for m = 1, 2, . . . , M, if (m!)2 r =m 2(2m)! m = 1, 2, . . . , M, then 22m (m!)2 < 1 for m = 1, 2, . . . , M. This implies that 2  πR M  πR M pm U2m−1 (ω)(2m)! 2   dω. Accord− π  m=2 pm U2m−1 (ω) dω > 4 − π  m=2 22m (m!)2 R

R

ing to Theorem 3.1, the SNR of the modulated system will be higher than that of the conventional system. This completes the proof.

Intuitively, the improvement of SNR of the coded system can be accounted as follows. If the quantizer can be modeled by a polynomial, then the quantizer performs a weighted sum of multiplications of the input signal in the time domain. In the frequency domain, the quantizer performs a weighted sum of convolutions of the input signal. If the input signal is in the base band, then convolutions of the base band signal are still in the base band. In other words, all high order terms in the model are collapsed together in the base band. As the signal component and the noise components are mixed together, the SNR is low. On the other hand, for the conventional amplitude modulation system, if a base band signal is modulated by a cosine carrier, then two frequency bands are generated centered at the carrier frequency and with the magnitude being half of the original signal. By demodulating the signal, three frequency bands are generated with one at the base band and two centered at twice the carrier frequency. The magnitude of these two high frequency bands is just half of that in the base band. By applying an ideal lowpass filtering, the base band signal can be recovered and the signal energy is halved. A similar idea is applied for the modulated quantized system. If a base band signal modulates a carrier and performs convolutions in the frequency domain, then more than two frequency

41

Quantization Noise Analysis

bands are generated after convolutions. For example, if the modulated signal is convolved by itself three times, then four frequency bands are generated centered at the carrier frequency and three times the carrier frequency. By demodulating the convolved signal, the frequency bands are shifted and one more frequency band is generated. The demodulated frequency bands are centered at the base band, twice of and four times the carrier frequency, with the magnitude equal to 0.375, 0.25 and 0.0625 of the original signal, respectively. By applying an ideal lowpass filtering, all the noise components in the high frequency bands are filtered. Only 0.375 of the noise energy is preserved. Although the signal energy is halved, the noise components are less than half. As a result, after filtering the SNR is improved. Hence, by multiplying a periodic code both before and after the quantizer, further noise reduction can be achieved compared to the conventional approach.

Example 3.3 Consider the system shown in Figure 3.4. Denote R as the OSR of the input signal, that is u(n)  is bandlimited within the  π π 1 |ω| < πR frequency spectrum − R , R . Assume that H(ω) = . Express 0 otherwise Y2 (ω) in terms of U(ω), ω0 and ak for k = 1, 2, . . . , Na . What is the minimum value of ω0 such that aliasing does not occur. Solution:

  Since Y2 (ω) = H(ω) U3 (ω) ∗ A416(ω) , where  A(ω) ≡ δ(ω − ω0 ) + δ(ω + ω0 ) and U3 (ω) ≡ U(ω) ∗ U(ω) ∗ U(ω), we have



U3 (ω − 4ω0 ) + 4U3 (ω − 2ω0 ) + 6U3 (ω) + 4U3 (ω + 2ω0 ) + U3 (ω + 4ω0 ) , 16 3H(ω)U3 (ω) which is equivalent to Y2 (ω) = . As the bandwidth of U3 (ω) is 6π 8 R , 6π in order to avoid the occurrence of the aliasing, we need R < 2ω0 , which implies that 3π R < ω0 .

Y2 (ω) = H(ω)

u(n)

( )3

s2(n)

H (v)

a(n)  cos(v0n) Figure 3.4 Cubic system with periodic codes.

y2(n)

42

3 Quantization in Digital Filters

OPTIMAL CODE DESIGN In order to design an optimal code such that the SNR is maximized, a modified gradient decent method is employed as follows: Algorithm 3.1 Step 1: Initialize iteration indices n = 0 and t = 0. Generate a random vector a0 , set l0 = 0, λ0 = 1, N˜ = 1000, Nˆ = 10 and δ = δ = 10−6 (these values are employed because they are typical for most gradient descent methods and global optimal searching algorithms). Choose Na = 40 and M = 10 (the reasons for employing these values will be discussed later). 

Step 2: Assume that u(k) = U sin 2πk (The reason for employing this input 3R signal will also be discussed later). Compute a new periodic code an+1 = an − λn ∇a SNR|a=an and set λn+1 =  λ0

2 , where ceil

n  N

⎛   

2 

 π  M p a U2m−1 (ω)  Na R 2 2  m=2 m m 22m  dω ⎜ ak p21 k=1 ak − πR |U(ω)| dω ∇a SNR = ⎝  π  

2  π  

2 p2 Na pm am U2m−1 (ω)  M 2 R R  2 ln(10) 41  dω k=1 ak − π |U(ω)| dω − π  m=2 22m 10



π R

− πR

R

−

p21 4

 Na

k=1

ak2

2 

π R

− πR

R



π R

 )U pm (∇a am 2m−1 (ω) 22m−1

M

|U(ω)|2 dω − π m=2 R 2   

 π 2  R M  U pm am 2m−1 (ω)   dω − π  m=2 22m



 U p m am 2m−1 (ω) 22m

⎞

dω ⎟ ⎟ ⎟ ⎠

R

and 4mNa +1  k=1  2mW4mNa +1 (2mNa + 1, k)(W4mNa +1 (k, Na + 1−q)+W4mN a +1 × Na 2m−1, (k, Na + 1 + q)) q=1 (W4mNa +1 (k, Na + 1 − q) +W4mNa +1 (k, Na + 1 + q))aq in which pm for m = 1, 2, . . . , M are obtained via solving the corresponding semi-infinite programming problem. Step 3: If |an+1 − an | ≤ δ, then go to Step 4, otherwise, increase the index n by 1 and iterate Step 2. ˆ then take an+1 as Step 4: Denote lt+1 = SNR|an+1 . If 0 < l t+1 − lt ≤ δ and t > N, t+1 t the final approximated optimal solution. If l > l + δ , then increase both the indices n and t by 1, set ain = (1 + rit )ain where ain is the ith element of the vector an and rit is a random scalar with uniform distribution between [−0.5, 0.5], and iterate Step 2. If lt+1 < lt , then set an+1 as that corresponds to l t and add a random vector on it, increase both the indices n and t by 1, and iterate Step 2.  ≡ ∇a am

Optimal Code Design

43

In general, the gradient descent method is prone to divergence and usually finds solutions that are only locally optimal. To avoid the obtained solution being trapped in the local optimal solution, once an approximated local optimal solution is found, a random vector is added to it so that it is kicked out of the approximated local optimal solution and a new approximated local optimal solution is found. If the SNR of the current approximated local optimal solution is lower than that of the previous one, then the current approximated local optimal solution is discarded, the code corresponding to the previous local optimal solution is re-used but a random vector is added to it and re-iterates the above procedures. Since Na is finite, there are finite number of local minima. As the obtained SNR is monotonic increasing, it will not go back to those local optimal solutions corresponding to lower SNRs. As a result, it tends to move to the optimal solution with the largest SNR. Since there are two loops in Algorithm 3.1, certain mechanisms are required to quit both loops in order to solve the divergent problem. To quit the first loop, a dynamical factor λn is employed. In general, the convergence of the gradient descent method would depend on the values of λn , rit , δ and δ . For instance, if the values for λn are too small, this would cause the termination of Step 3 and go straight to Step 4 and be further away from the exact local optimal solution. But too large values of λn might deviate. To tackle this problem, a  If large value for λn is employed in the first round of the iterations (n < N). it then deviates, λn is decreased in each of the following rounds of iterations so that the difference between the current approximated local optimal solution and the previous one is forced to zero and the first loop would eventually have a fixed value of λn . For the second loop, if the SNR is lower, then the solution will stay in the same position. However, if the number of iterations that cause a particular approximated local optimal solution stays at that point and is larger ˆ then the second loop will be terminated for a fixed value of δ . Hence, than N, the divergence problem is solved. To verify the effectiveness of Algorithm 3.1, numerical computer simulations are illustrated. In these numerical computer simulations, the saturation level of the quantizer is chosen to be 1, that is L = 1, for the normalization reason. The ideal lowpass filter is implemented by computing the discrete Fourier transform of the corresponding inputs of the filter and taking the inverse discrete Fourier ! transform of the coefficients within the frequency spectrum − πR , πR . We choose Na = 40 and M = 10 because too large values of Na and M would increase the computation complexity, while a too small value of Na limits the performance and a too small value of M is not enough to approximate the quantizer. Sinusoidal signals !are employed because they are guaranteed to be bandlimited within − πR , πR .

3 Quantization in Digital Filters

0.1 0.05 0 0.05 0.1 0.15 0.2 0.25

Coded coefficients ak

Coded coefficients ak

44

0

10

(c)

30

40

0.6 0.4 0.2 0 0.2 0.4

0

10

20

0.2 0.1 0 0.1 0.2

0

10

(b)

k

Coded coefficients ak

Coded coefficients ak

(a)

20

0.3

30

k

40 (d)

20

30

40

30

40

k 8 6 4 2 0 2 4

0

10

20 k

Figure 3.5 Coded coefficients when R = 32 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

It is worth noting that it may not have similar patterns on the obtained optimal periodic codes as the number of bits for the quantizer or the OSR varies. Since the coefficients of the approximate models are obtained via solving the corresponding semi-infinite programming problems, the coefficients of the approximated models for a different number of bits for the quantizer are very different (for the same value of M). Also, the thermal noise dominates as OSR increases. Hence, it may not have similar patterns on the obtained periodic codes as the number of bits for the quantizer or the OSR varies. For examples, by running Algorithm 3.1 with changing the input to a normalized sum of sinusoidal signals, that is

 100 knπ sin n=1 100R  

 , u(k) =  100 knπ  max  n=1 sin 100R  ∀k≥0

the periodic codes based on the above parameters with different numbers of bits for quantizers and OSRs can be obtained (Figures 3.5–3.9). It can be seen from Figures 3.5 to 3.9 that there are no similar patterns on the obtained optimal periodic codes as the number of bits for the quantizer or the OSR varies. Improvements for the SNR performance of the coded system over the conventional system and the system with an additive dithering are shown in Figure 3.10. Assume that an additive white noise source with a uniform distribution between [−1,1] is added and subtracted before and after the quantizer

45

0.06

Coded coefficients ak

Coded coefficients ak

Optimal Code Design

0.04 0.02 0 0.02 0.04 0.06

0

10

30

40

k 0.5 0.4 0.3 0.2 0.1 0 0.1 0.2

0

10

20

30

40

k

(c)

0.15 0.1 0.05 0 0.05

0

10

(b) Coded coefficients ak

Coded coefficients ak

(a)

20

0.2

20

30

40

30

40

k 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8

0

10

20 k

(d)

0.06 0.04 0.02 0 0.02 0.04 0.06 0.08

Coded coefficients ak

Coded coefficients ak

Figure 3.6 Coded coefficients when R = 64 for:(a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

0

10

(c)

20

30

40

k 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8

0

10

20 k

0

10

(b) Coded coefficients ak

Coded coefficients ak

(a)

0.25 0.2 0.15 0.1 0.05 0 0.05 0.1

30

40 (d)

20

30

40

30

40

k 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6

0

10

20 k

Figure 3.7 Coded coefficients when R = 128 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

3 Quantization in Digital Filters

0.15

Coded coefficients ak

Coded coefficients ak

46

0.1 0.05 0 0.05 0.1

0

10

20

30

40

1 0.8 0.6 0.4 0.2 0 0.2 0.4

0

10

(c)

20

0.3 0.2 0.1 0 0.1

0

10

(b)

k

Coded coefficients ak

Coded coefficients ak

(a)

0.4

30

40

30

40

30

40

k 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6

0

10

20 k

(d)

k

20

0.15

0.35 Coded coefficients ak

Coded coefficients ak

Figure 3.8 Coded coefficients when R = 256 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

0.1 0.05 0 0.05 0.1

0

10

(c)

30

k

0.4 0.2 0

0

10

20 k

0.1 0.05 0

10

(b)

1 0.8 0.6

0.2 0.4

0.2 0.15

0

40

Coded coefficients ak

Coded coefficients ak

(a)

20

0.3 0.25

30

40 (d)

20

30

40

30

40

k 1.5 1 0.5 0 0.5

0

10

20 k

Figure 3.9 Coded coefficients when R = 512 for: (a) N = 1; (b) N = 2; (c) N = 3; (d) N = 4.

47

Optimal Code Design

Compared to conventional approach Compared to dithering approach

Improvements on SNRs (dB)

250

200

150

100

50

0 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Input magnitude U Figure 3.10 Improvements in SNR performance of the coded system over the conventional system and the additive dithering system.

in the additive system, respectively. Assume that the input signal is

 dithering 2πk u(k) = U sin 3R ∀k ≥ 0, where R = 64 and U is the magnitude of the sinusoidal input. The quantizer is assumed to be a singlebit antisymmetric one with 1 y≥0 the saturation level equal to one, that is Q (y) ≡ . This type −1 otherwise of sinusoidal input and quantizer is used for performance testing because it is the most common method used these days. Since the thermal noise would be dominated when the OSR is very large, the comparison with very high OSR is not useful and R = 64 is used for the comparison. The periodic codes employed in these numerical computer simulations consist of 40 coefficients designed under M = 10 − 40 coefficients with M = 10 are employed as discussed earlier. According to the numerical computer simulations, the coded system could achieve an average of 101 dB improvement over the conventional system and an average of 85 dB improvement over the additive dithering system. These numerical computer simulations show that the coded system achieves very significant improvements over the additive dithering system and the conventional system. There are two main fundamental differences between the additive dithering system and the coded system. First, in the additive dithering system, a white noise is added before and after the quantizer; while in the coded system, a periodic code

48

3 Quantization in Digital Filters

is multiplied before and after the quantizer. Just changing addition to multiplication would cause the noise analysis to change drastically. Statistical analysis of an additive noise can be performed easily and many well-known methods can be applied. However, the techniques for analyzing multiplicative noise are limited and this problem is theoretically difficult. Hence, just changing additive noise to multiplicative noise would require a very different analytical technique and result in a very different picture. Second, the noise in the conventional approach is a random process, while the periodic code in the coded system is a deterministic signal. Since these two signals are very different in nature, their analysis is also very different. In terms of complexity and cost of an implementation, since the implementation of the additive dithering system requires a random sequence, it is, in general, more difficult and costly to generate a truly random signal with a uniform distribution. Hence, the implementation cost of the additive dithering system is high. On the other hand, it is easier and cheaper to implement the coded system because the periodic code can be stored in memory and only multiplications are required. It is worth noting that the coded method gives worse results for large values of N, such as N = 4, when the numerical computer simulations are performed at the same value of M. This is because the number of terms of the polynomial required to approximate the quantizer should be increased as the number of bits for the quantizer increases, as discussed previously. As M remains unchanged, the accuracy of the approximated model decreases. Hence, the results are worse when N = 4. However, if increasing the value of M, then aliasing may occur. Because of the trade off between the aliasing and model errors, the improvements in the SNR performance decrease as the number of bits of quantizer increases. For examples, by running Algorithm 3.1 and changing the input to a normalized sum of sinusoidal signals, that is 

100 knπ sin n=1 100R  

 , u(k) =  100 knπ  max  n=1 sin 100R  ∀k≥0

the periodic codes based on 40 coefficients with M = 10, a different number of bits of quantizers, as well as different OSRs can be obtained accordingly. Figure 3.11 shows the improvements on SNR performance at different OSRs as the number of bits of quantizer varies. It can be seen from Figure 3.11 that improvements in the SNR performance decrease as N increases. However, there is no simple relationship between the improvements on the SNR performance as the OSR varies. This is because the effects of thermal noise dominate as the

49

Optimal Code Design

OSR  32

18

OSR  64

20

16

18

16

16

14

10 8

14

Improved SNR (dB)

Improved SNR (dB)

Improved SNR (dB)

14 12

12 10 8 6

6 4 2

1

(a)

2

3

Number of bits N

4

12 10 8 6

4

4

2

2

0

1

(b)

2

OSR  128

18

3

4

Number of bits N

0

1

2

3

4

(c) Number of bits N

Figure 3.11 Relationships between the improved SNRs of the coded system over the conventional system and the number of bits of quantizer when: (a) R = 32; (b) R = 64; (c) R = 128.

OSR increases. Also, as the OSR increases, π 2 $2 "πR  M % N "R a   # # 1   ak2 pm U2m−1 (ω) dω −    2 k=1

− πR

m=2

− πR

 M  2  # p a U m m 2m−1 (ω)     dω   22m m=2

becomes smaller. Figure 3.12 shows improvements of the SNR performance at different numbers of bits of quantizer as the OSR varies. It can be seen from Figure 3.12 that there is no simple relationship between the improvements on the SNR performance and the OSR. Figure 3.13 shows the histograms of the input signal, the quantized signal, and the output of the filter of the conventional system when a single bit antisymmetric quantizer with the saturation level equal to one; that is  

1 y ≥0 Q (y) ≡ ∀k ≥ 0 with , and the input signal u(k) = U sin 2πk 3R −1 otherwise R = 64 and U = 0.1, are applied. Figure 3.14 shows the histograms of the input of the quantizer, the output of the quantizer, the demodulated signal, and the output of the filter of the coded system with 40 coefficients designed under M = 10 (40 coefficients with M = 10 is employed for reasons discussed earlier). It can be seen from Figure 3.13 that the dynamical range of the filtered signal of the conventional system is between −1.27 and 1.27, which is about

50

3 Quantization in Digital Filters

1 bit quantizer

17 16 15 102

4.5 4 3.5 3 2.5 2 1.5 1 101

9 8 7 6 102

103

OSR

(b)

3 bit quantizer

4 bit quantizer 3

102

2 1 0 1 2 3 101

103

OSR

(c)

10

5 101

103

OSR

(a)

Improved SNR (dB)

Improved SNR (dB)

18

Improved SNR (dB)

Improved SNR (dB)

19

14 101

2 bit quantizer

11

20

102

(d)

103

OSR

(a)

4000

800

700

3500

700

600

3000

600

500 400 300

2500 2000 1500

Histogram of y1(k)

800

Histogram of s1(k)

Histogram of x1(k)

Figure 3.12 Relationships between the improved SNRs of the coded system over the conventional system and the OSR when: (a) N = 1; (1b) N = 2; (c) N = 3; (d) N = 4.

500 400 300

200

1000

200

100

500

100

0 0.1 0.05 0 0.05 0.1 Range of input signal (b)

0 0 1 0.5 0 0.05 0.1 2 1 0 1 2 Range of quantized signal (c) Range of filtered output signal

Figure 3.13 Histograms of the input signal, the quantized signal and the output of the filter of the conventional system.

1200% of that of the input signal. Hence, the SNR ratio is low. On the other hand, as can be seen in Figure 3.14, the dynamical range of the filtered signal of the coded system has the same dynamical range as the input signal. Hence, the coded system achieves a higher SNR than the conventional one.

51

Summary

4000 Histogram of s2(k)

Histogram of input of quantizer

2500 2000 1500 1000 500 0 0.04 (a)

0.02

0

0.02

0.5

0

0.5

1

Range of output of quantizer 800

Histogram of y2(k)

Histogram of demodulated signal

1000

(b)

2000 1500 1000 500 0 0.4 (c)

2000

0 1

0.04

Range of input of quantizer

3000

0.2

0

0.2

Range of demodulated signal

600 400 200 0 0.15 0.1 0.05

0.4 (d)

0

0.05

0.1

Range of filtered output signal

Figure 3.14 Histograms of the input of the quantizer, the output of the quantizer, the demodulated signal and the output of the filter of the coded system.

Example 3.4 Using the conventional gradient descent method, find the minimum value of the function f (x) = x 2 with the initial guess at x0 = 0.1 and λn = 0.45 ∀n ≥ 0, in which the algorithm stops when |xn+1 − xn | < 0.01. Solution:   Since xn+1 = xn − λn dfdx(x) 

x=xn

and λn = 0.45 ∀n ≥ 0, we have xn+1 = 0.1xn

∀n ≥ 0. Hence, x1 = 0.01. As |x1 − x0 | = 0.09 > 0.01, further iteration is required and we have x2 = 0.001. As |x2 − x1 | = 0.09 < 0.01, the algorithm stops and the obtained solution is x = 0.009.

SUMMARY In this chapter two quantization models are discussed. The first one is to model the quantizer as an additive white noise source, while the second models the quantizer as a high order polynomial function. Based on these models, a detailed SNR analysis is performed. When a periodic code is applied to both the input and output of the quantizer, the SNR may be increased. A condition for an

52

3 Quantization in Digital Filters

Input signal  

Loop filter 

Output signal H(v)

Q

Figure 3.15 A sigma delta modulator.

improvement of the SNR is derived. An optimal periodic code is designed via a modified gradient descent method.

EXERCISES 1. Denote xi for i = 1, 2, . . . , N as data and cj for j = 1, 2, . . . , M as the quantized data, where M < N. That is Q(xi ) = cj . Find cj in terms of xi such that  the total quantized error energy is minimized. That is to solve N M 2 min i=1 j=1 (xi − cj ) . (c1 ,c2 ,...,cM )

2. Consider the following system shown in Figure 3.15. This system is known as a sigma delta modulator, which is widely employed in the analog-to-digital conversion. By modeling the quantizer as an additive white Gaussian noise source, derive the signal transfer function and noise transfer function of the system. Based on the derived expressions, show that in order to achieve high SNR, the loop filter should have poles on or outside the unit circle. 3. Consider the system shown in Figure 3.4. Calculate the SNR of the system. 4. Consider the problem discussed in Example 3.4. Show that the algorithm converges if 1 > λn > 0 ∀n ≥ 0.

4 SATURATION IN DIGITAL FILTERS

Nonlinearity due to saturation is widely seen in many signal processing circuits and systems because signals can be guaranteed to be bounded in certain regions. However, even for a second order digital filter associated with the saturation nonlinearity, it may exhibit oscillation behaviors and the frequencies of the oscillations are different for different filter parameters. For some applications, this would cause the degradation of system performance. When the eigenvalues of the system matrix are outside the unit circle, we can expect that the state trajectories would not converge to the origin and oscillation behaviors would occur. But this is not necessarily true. In this chapter, conditions for the occurrence of oscillations are discussed. The corresponding oscillation frequencies are estimated, and the stability conditions for the occurrence of the limit cycles are presented. SYSTEM MODEL The second order digital filter associated with the saturation nonlinearity can be briefly modeled as follows. Let the state vector of a second order digital filter be x(k) and the state equation be: & ' x1 (k + 1) x(k + 1) = = F(x(k)). (4.1) x2 (k + 1) If the second order digital filter is realized by a direct form representation, then: & ' x2 (k) x(k + 1) = , (4.2) fs (bx1 (k) + ax2 (k)) where a and b are the filter parameters, and fs (·) is the saturation function defined as: ⎧ 1 ⎨ μy if |y| ≤ fs ( y) = (4.3) μ, ⎩sgn( y) otherwise 53

54

4 Saturation in Digital Filters

in which μ is the gain of the saturation function in the linear region and sgn(·) denotes the sign function. Example 4.1 Consider a digital filter with transfer function M 

G(z) =

m=0 N  n=0

bm z−m , an

z−n

where M ≤ N. Find the direct form state space matrices of the filter. Solution: ⎡ 0 1 ⎢ .. ⎢ . 0 ⎢ ⎢ . .. . ⎢ A=⎢ . . ⎢ 0 0 ⎢ ⎣ aN − ··· a0

⎤ 0 ⎡ ⎤ 0 .. ⎥ . ⎥ ⎢ .. ⎥ ⎥ ⎢ . ⎥ ⎥ .. .. ⎢ ⎥ . . 0 ⎥ , B = ⎢ 0 ⎥, C = [0 · · · 0 bM · · · b1 ] ⎥ ⎢ ⎥ ⎥ ⎣1⎦ ··· 0 1 ⎥ ⎦ a1 a0 ··· ··· − a0 0 ··· .. .. . .

and D = 0. OSCILLATIONS OF DIGITAL FILTERS ASSOCIATED WITH SATURATION NONLINEARITY For a second order digital filter associated with the saturation nonlinearity, the state space matrices of the filter are denoted as: & ' 0 1 A= , (4.4) b a & ' 0 B= , (4.5) 1 ! C= b a (4.6) and D = 0, while the transfer function of the filter is obtained as az + b G(z) = 2 . z − az − b

(4.7)

(4.8)

Oscillations of Digital Filters Associated with Saturation Nonlinearity

55

Referring to Section 2.4 and Figure 2.7 we see that, the saturation nonlinearity can be modeled as a feedback system with the input output relationship as follows: ⎧ 1 ⎨ (μ − 1)y(k) |y(k)| ≤ g( y(k); μ) = (4.9) μ ⎩−y(k) + sgn(y(k)) otherwise or ⎧ 1 ⎨ (1 − μ)e(k) |e(k)| ≤ f (e(k); μ) = (4.10) μ ⎩e(k) − sgn(e(k)) otherwise However, f (e(k), μ) is not a smooth (C r , r ≥ 3) function because f (e(k), μ) is not differentiable at e(k) = ± μ1 . Hence, the Hopf bifurcation theorem cannot be applied directly. By letting  μπe(k) 2 , (4.11) f˜ (e(k); μ) = e(k) − tan−1 π 2 it can easily be shown that f˜ (e(k); μ) is a smooth (C r , r ≥ 3) function and ∃ε(μ) such that | f˜ (e(k); μ) − f (e(k); μ)| < ε(μ) ∀e(k). By solving the equation G(z)f˜ (e(k); μ)|z=1,e(k)=ˆe = −e(k)|e(k)=ˆe , the equilibrium point is located at eˆ = 0. Since  ∂f˜ (e(k); μ)  = 1 − μ, J(μ) =  ∂e(k) 

(4.12)

(4.13)

e(k)=ˆe

λ(e jω ; μ) = G(z)J(μ)|z=e jω =

ae jω + b (1 − μ), e2jω − ae jω − b

(4.14)

where λ(e jω ; μ) is the eigenvalue of G(e jω )J(μ). As G(e jω )J(μ) is a scalar, the right and left eigenvectors are: η1 = 1

(4.15)

η2 = 1,

(4.16)

and

respectively. Define: az + b G(z) = 2 , 1 + G(z)J(μ0 ) z − aμ0 z − μ0 b  ∂2 f˜ (e(k); μ)  η1 = 0, Q≡  ∂e(k)2 

H(z) ≡

e(k)=ˆe,μ=μ0

(4.17)

(4.18)

56

4 Saturation in Digital Filters

1 v0 ≡ − H(e j0 )Qη1 = 0, 4

(4.19)

1 v2 ≡ − H(e j2ωR )Qη1 = 0, 4

(4.20)

 ∂3 f˜ (e(k); μ)  L≡  ∂e(k)3 

η1 η 1 =

e(k)=ˆe,μ=μ0

π2 μ30 , 2

π2 μ30 1 1 , p(e jωR ) ≡ Qv0 + Qv2 + Lη1 = 2 8 16 and ζ(e

jωR

(4.21)

(4.22)

 % 2 3 $ π μ0 −η1 G(e jωR )p(e jωR ) ae jωR + b )≡ = − 2jω , (4.23) jω R R η1 η2 e − ae − b 16

where η2 and Q are the complex conjugates of η2 and Q, respectively, ωR and μ0 are the frequency and the bifurcation parameter such that Im(λ(e jωR ; μ0 )) = 0. It can be easily shown that  a

(4.24) ωR = cos−1 − 2b and μ0 = 1 +

e2jωR − ae jωR − b , ae jωR + b

(4.25)

respectively. According to the bifurcation theorem, the frequency of oscillations of the second order digital filter associated with the saturation nonlinearity can be approximated by the frequency at the point Pˆ where Pˆ is the intersecting point of the locus of λ(e jω ; μ) and the half line starting at −1 + 0j in the direction defined by ζ(e jωR ). In this case, since ζ(e jωR ) ∈ , Pˆ is the point where the locus of λ(e jω ; μ0 ) cuts the imaginary axis. Hence, the frequency of oscillations can be approximated by ωR . When μ = 1, α = 0.5 and −1.5 ≤ b ≤ −1.2, the digital filter exhibits oscillation behaviors. The oscillation frequency depends on the filter parameter b, but it is  0

independent of the initial conditions except when x(0) = 0 . Figure 4.1a shows the actual oscillation frequencies of a second order digital filter associated

 with 0.0001

the saturation nonlinearity when μ = 1.0001μ0 , α = 0.5, x(0) = 0.0001 and −1.5 ≤ b ≤ −1.2, while Figure 4.1b shows the estimated frequencies via the

1.405

1.405

1.4

1.4

1.395

1.395

1.39

1.39

Estimated frequency

Actual frequency

Oscillations of Digital Filters Associated with Saturation Nonlinearity

1.385

1.38

1.385

1.38

1.375

1.375

1.37

1.37

1.365

1.365

1.36 1.5 1.45 1.4 1.35 1.3 1.25 1.2 1.15 1.1

1.36 1.5 1.45 1.4 1.35 1.3 1.25 1.2 1.15 1.1

b

b (a)

57

(b)

Figure 4.1 (a) Actual oscillation frequencies of the second order digital filter associated with the saturation nonlinearity; (b) Estimated frequencies via the Hopf bifurcation theorem.

Hopf bifurcation theorem. It can be seen from Figure 4.1b that the difference between the actual frequencies and the estimated frequencies are negligible, and so the estimations are valid. Example 4.2

 Consider the delayed logistic system with state space matrices A = 00 μ1 ,



 B = 01 , C = 01 01 and D = [0 0], and the nonlinearity is governed by g( y(k); μ) = −μy1 (k)y2 (k), where y(k) ≡ [y1 (k)y2 (k)]T . Find ζ(e jωR ). Solution:

 1 z−1 The transfer function of the digital filter is G(z; μ) = z−μ and the 1 corresponding nonlinear function is f (e(k); μ) =−μe1 (k)e

2(k). By solving G(z; μ)f (e(k); μ)|z=1,e=ˆe = −ˆe, we have eˆ = μ1 − 1 11 . Hence, the   Jacobian matrix is given by J(μ) = ∂f (e(k);μ) = (μ − 1)[1 1] and the ∂e(k)  e=ˆe

58

4 Saturation in Digital Filters

open loop system matrix is G(z; μ)J(μ) = μ−1 z−μ (e jω ; μ) = 0

−jω . λ2 (e jω ; μ) = (μ − 1) 1+e ejω −μ

−1 z

1

z−1 1

 with eigenvalues

(e jω ; μ) = 0

λ1 Since λ1 ∀ω ∈ , and it never crosses the critical point −1 + 0j. Hence, the only relevant eigenvalues is λ2 (e jω ; μ). The normalized right and left eigenvectors associated with

 1 −jω  λ2 (e jω ; μ) are η1 = √ e 1 and η2 = √1 11 , respectively. Hence, we 2 2 have Q = − √μ [1 e−jω ] and L = [0 0]. As a result, the linearized closed 2



 1 μ 1 0= jω + e−jω ) 1 loop matrix is H(z) = μ−1−z+z (e and we have v 2 z 1 8(μ−1)

 √ 1 2(1+e−jωR )p(ωR ) μe−jω jω 2 R and v = 4(μ−1−e2jωR +e4jωR ) e2jωR . Thus, ζ(e ) = (μ−e jωR )(1+e−jω ) , where √   μ μ 2 p(ωR ) = − 2 1 + e−jω (e jω + e−jω ) + (e j(ω+2ωR ) + 1) 4 8(μ − 1) μe−jω × . 4(μ − 1 − e2jωR + e4jωR ) STABILITY OF OSCILLATIONS OF DIGITAL FILTERS ASSOCIATED WITH SATURATION NONLINEARITY When μ < μ0 (before criticality), the curve λ(e jω ; μ) encircles the point −1 + 0j in a clockwise direction as shown in Figure 4.2a. Hence, the overall system is stable. As a result, oscillation behaviors do not occur and the state variables converge asymptotically to the origin, even though the eigenvalues of the system matrix are outside the unit circle, as shown in Figure 4.2b. However, when μ > μ0 (after criticality), the curve does not encircle the point −1 + 0j as shown in Figure 4.2c. Hence, the equilibrium point is unstable. As a result, the state trajectories will move from the neighbor of the equilibrium point to the limit cycle and oscillation occurs, as shown in Figure 4.2d. In this case, the limit cycle is stable.

Example 4.3 Consider the delayed logistic system discussed in Example 4.2. Does the limit cycle exist when μ = 2.05? Solution: When μ = 2.05, it can be checked that the intersection between λ2 (e jω ; μ) and the negative real axis near the critical point −1 + 0j takes place at ω = 1.018 and ζ(e jω )|ω=1.018 = −0.5185 − j0.0118. As expected, the locus interests the half line verifying the existence of an invariant cycle.

59

Stability of Oscillations of Digital Filters Associated with Saturation Nonlinearity 0.1

1 0.8

0.05

0.6 0

0.2

x2

Image (lemda)

0.4 0.05

0.1

0 0.2

0.15

0.4 0.2 0.6 0.25

0.8 1 1

0.3 1.06 1.04 1.02 1 0.98 0.96 0.94 0.92 0.9 0.88

0.5

(a)

0

0.5

1

0.5

1

x1

Real (lemda) (b) 0.1

1 0.8

0.05 0.6 0.4 0.2

0.05

x2

Image (lemda)

0

0.1

0 0.2 0.4

0.15

0.6 0.2 0.8 1 1

0.25 0.98 0.96 0.94 0.92 0.9 0.88 0.86 0.84 0.82 0.8

(c)

0.5

0

x1

Real (lemda) (d)

&

' 0.99 ; 0.99 jω ; μ) when μ = 1.01μ , a = 0.5, b = −1.25 (b) The corresponding state trajectories; (c) The curve λ (e 0 & ' 0.99 ; (d) The corresponding state trajectories. and x(0) = 0.99

Figure 4.2 (a) The curve λ(e jω ; μ) when μ = 0.99μ0 , a = 0.5, b = −1.25 and x(0) =

SUMMARY In this chapter the existence of oscillations in digital filters associated with saturation nonlinearity is discussed via the Hopf bifurcation theorem. The state trajectories may converge to the origin even though the eigenvalues of the system matrix are outside the unit circle.

60

4 Saturation in Digital Filters

EXERCISES 1. Consider the discretized model of a neural netlet in which their excitation and inhibition is governed by x1 (k + 1) = e−μ x1 (k) + a(1 − e−μ )tanh(c1 x2 (k)) and x2 (k + 1) = e−μ x2 (k) + a(1 − e−μ )tanh(−c2 x1 (k)), where a, c1 and c2 are positive constants and μ is the bifurcation parameter that presents the time delay due to the finite switching speed of amplifiers in models of electronic networks. Assume that a2 c1 c2 > 1. Define μ0 ≡ ln(a2 c1 c2 + 1) − ln(a2 c1 c2 − 1). Find the state space matrices of the corresponding filter and the corresponding nonlinear function f (e(k); μ). 2. Consider the same problem in Question 1. Find ζ(e jωR ). 3. Consider √ the same problem in Question 1. Is the fixed point stable ∀μ > μ0 , a = 3 and c1 = c2 = 1?

5 AUTONOMOUS RESPONSE OF DIGITAL FILTERS WITH TWO’S COMPLEMENT ARITHMETIC

In a real situation, digital filters are usually implemented via the two’s complement arithmetic because subtraction can be implemented easily via addition. However, the two’s complement arithmetic involves a periodic discontinuous nonlinear function, and the dynamics of digital filters could be very complicated even when no input signal is applied. Limit cycle and chaotic behaviors could occur even for second order digital filters. In this chapter autonomous response of second order lowpass and highpass digital filters associated with two’s complement arithmetic is discussed. In particular, the lowpass filter containing a DC pole and the highpass filter containing a pole located at z = −1 are discussed. Finally, various conditions for the occurrence of limit cycle and chaotic behaviors are given.

SYSTEM MODEL Second order real digital filters can be represented by a state space model as follows: & ' x (k + 1) x(k + 1) = 1 = F(x(k)). (5.1) x2 (k + 1) Using a direct form representation, the system can then be further represented as: & ' x2 (k) x(k + 1) = , (5.2) f (bx1 (k) + ax2 (k) + u(k)) where a and b are the filter parameters, u(k) is the input signal, x1 (k) and x2 (k) are the state variables, and f is the nonlinearity due to the use of two’s complement arithmetic. 61

62

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

The nonlinearity f can be modeled as f (ν) = v − 2n

(5.3)

2n − 1 ≤ v < 2n + 1

(5.4)

such that

and n ∈ Z. Hence, the system can be represented as & ' x2 (k) x(k + 1) = bx1 (k) + ax2 (k) + u(k) + 2s(k) & ' 0 = Ax(k) + Bu(k) + s(k) 2 ∀k ≥ 0, where & ' ' & ( x1 (k) x1 (k) 2 ∈I ≡ : −1 ≤ x1 (k) < 1, −1 ≤ x2 (k) < 1 , x2 (k) x2 (k) A=

& 0 b

B=

' 1 , a

& ' 0 , 1

(5.5) (5.6)

(5.7)

(5.8)

(5.9)

and s(k) ∈ {−m, . . . , −1, 0, 1, . . . , m},

(5.10)

in which m is the minimum integer satisfying −2m − 1 ≤ bx1 (k) + ax2 (k) + u(k) < 2m + 1.

(5.11)

Since s(k) is an element in the discrete set {−m, . . . , −1, 0, 1, . . . , m}, the values of s(k) can be viewed as symbols and s(k) is called a symbolic sequence. It is worth noting that once the initial condition x(0), the filter parameters and the input signal are given, the state variables and the symbolic sequences are uniquely defined by equations (5.3)–(5.11). Also, for a given set of filter parameters and an input signal, denote the symbolic sequence as s = (s(0), s(1), . . .) and the set of symbolic sequence as . Denote the map from the set of initial 2 conditions to the set of symbolic sequence as S, that  is S : I → . The 

 symbolic x1 (0) = s. sequence s in  is said to be admissible if ∃ x2 (0) ∈ I 2 such that S xx21 (0) (0) The set  can be partitioned into three subsets: α = {s = (s(0), s(1), . . . ): s is periodic}, β = {s = (s(0), s(1), . . . ) : s is periodic after a number of iterations} and γ = \(α ∪ β ).

Linear and Affine Linear Behaviors

63

LINEAR AND AFFINE LINEAR BEHAVIORS Filters with a DC pole For a second order digital filter with a DC pole, the filter coefficients have to satisfy the following condition: b = a + 1.

(5.12)

In this chapter we only consider the autonomous response, that is u(k) = 0

(5.13)

∀k ≥ 0. For the strictly linear property, s(k) = 0 ∀k ≥ k0 . Hence, we have the following theorem: Theorem 5.1 For b = a + 1 and |a + 1| > 1, s(k) = 0 ∀k ≥ k0 if and only if ∃k0 ∈ Z+ ∪ {0} such that x1 (k0 ) = −x2 (k0 ). Proof: For the if part, since b = a + 1 and s(k) = 0 ∀k ≥ k0 , we have: &

' ' & 1 (a + 1)k−k0 (x1 (k0 ) + x2 (k0 )) + (−1)k−k0 ((a + 1)x1 (k0 ) − x2 (k0 )) x1 (k) = . x2 (k) a + 2 (a + 1)k−k0 +1 (x1 (k0 ) + x2 (k0 )) − (−1)k−k0 ((a + 1)x1 (k0 ) − x2 (k0 )) (5.14)

Since |a + 1| > 1, (a + 1)k−k0 (x1 (k0 ) + x2 (k0 )) diverges as k → +∞ if

x1 (k0) + x2 (k0 ) = 0. However, as s(k) = 0 ∀k ≥ k0 , this implies that x1 (k) 2 k−k0 (x (k ) + x (k )) is bounded ∀k ≥ k . Hence, 1 0 2 0 0 x2 (k) ∈ I and that (a + 1) x1 (k0 ) + x2 (k0 ) = 0 and this proves the if part. 

0 + 1) = For the only if part, since x1 (k0 ) = −x2 (k0 ), we have xx21 (k

(k0 + 1)   −x1 (k0 ) −x1 (k0 ) x1 (k0 + 1) = f (x1 (k0 )) . Since |x1 (k0 )| < 1, we have x2 (k0 + 1) =

f ((a + 1)x  1 (k0 ) − ax1 (k0 )) −x1 (k0 ) x1 (k0 ) and s(k0 ) = 0.

  

x1 (k0 ) x1 (k0 + 2) 0 + 2) Similarly, we have xx21 (k (k0 + 2) = −x1 (k0 ) and s(k0 + 1) = 0. Since x2 (k0 + 2) =

 x1 (k0 ) x2 (k0 ) , we have s(k) = 0 ∀k ≥ k0 and this proves the only if part and completes the proof.

64

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

Remark 5.1 Although the digital filter is implemented via the two’s complement arithmetic, the system may behave as a linear system when s(k) = 0 ∀k ≥ k0 . Also, since the eigenvalues of matrix A are −1 and a + 1, where |a + 1| > 1, one of the eigenvalues is outside the unit circle and the digital filter is unstable. However, the trajectory of an unstable linear system is confined in a finite state space. This is counter intuitive. Theorem 5.1 gives the necessary and sufficient condition for the nonlinear system to behave as a linear system after a number of iterations. It is interesting to note that the system would behave as a linear system after a number of iterations if and only if the state vector toggles between two points on a particular straight line of the phase plane.

Example 5.1





 + 1) x1 (k) 0 1 Consider the system xx21 (k = A + Bs(k), where A = x2 (k) a+1 a ,



(k+ 1)  0 x1 (0) 0.9003 B = 2 , a = 3 and x2 (0) = −0.5377 . Plot the phase portrait of the system. Solution: The phase portrait of the system is shown on Figure 5.1. It can be seen from Figure 5.1 that the state vector toggles between two states at the steady state on a particular straight line x1 = −x2 of the phase plane.

The strictly linear property – that is s(k) = 0 ∀k ≥ k0 , was discussed in Theorem 5.1. How about the affine linear property – that is, s(k) = a = 0 ∀k ≥ k0 ? This case is presented in the following theorem:

Theorem 5.2 + For b = a + 1 and a being an odd integer,  0 ∈ Z ∪ {0} such that

∃k x1 (k) x1 (k0 ) x1 (k0 ) = x2 (k0 ) = −1 if and only if s(k) = a and x2 (k) = x2 (k0 ) ∀k ≥ k0 . Proof:  

x2 (k0 ) 0 + 1) + For the if part, since xx21 (k (k0 + 1) = f ((a + 1)x1 (k0 ) + ax2 (k0 )) , if ∃k0 ∈ Z ∪ {0}  

−1 0 + 1) such that x1 (k0 ) = x2 (k0 ) = −1, then we have xx21 (k (k0 + 1) = f (−(2a + 1)) .

65

Linear and Affine Linear Behaviors

  −1 0 + 1) Since a is an odd integer, we have xx21 (k (k0 + 1) = −1 and s(k0 ) = a. As   

x1 (k0 + 1) x1 (k0 ) −1 x2 (k0 + 1) = x2 (k0 ) = −1 , we have s(k0 ) = a and x1 (k) = x2 (k) = −1 ∀k ≥ k0 . For the only if part, since



x1 (k0 + 1) x2 (k0 + 1)

=



x2 (k0 ) f ((a + 1)x1 (k0 ) + ax2 (k0 ))

, if x1 (k) =

 x1 (k0 ) have x2 (k0 ) =

then we x2 (k) = x2 (k0 ) ∀k ≥ k0 ,  x2 (k0 ) which implies x1 (k0 ) = x2 (k0 ) and x2 (k0 ) = f ((a + 1)x1 (k0 ) + ax2 (k0 )) , f ((a + 1)x1 (k0 ) + ax2 (k0 )). Since s(k) = a ∀k ≥ k0 , we have x1 (k0 ) = (a + 1) × x1 (k0 ) + ax1 (k0 ) + 2a, which implies 2a(x1 (k0 ) + 1) = 0. As a is an odd integer, so a = 0. As a result, we have x1 (k0 ) = −1 and we prove the only if part and complete the proof. x1 (k0 )

and

1 0.8 0.6 0.4

x2

0.2 0

x(1) x(2)

0.2 0.4

x(0)

0.6 0.8 1 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

x1 Figure 5.1 The phase portrait of the second order digital filter associated with two’s complement arithmetic. The points x(0), x(1), x(2) are as annotated, and the points with ‘∗ ’ denote the ‘steady states’ of x.

1

66

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

0 0.1 0.2 x(0) 0.3

x2

0.4 0.5 0.6 0.7 0.8 0.9 1 1

x(2) 0.8

0.6

0.4

0.2

0

0.2

0.4

x1 Figure 5.2 The phase portrait of the second order digital filter associated with two’s complement arithmetic. The points x(0), x(1), x(2) are as annotated, and the point with ‘∗ ’ denotes the ‘steady state’ of x.

Remark 5.2 Theorem 2 states the necessary and sufficient condition for the state vector to stay at a fixed point after a number of iterations when the parameter a is an odd integer. It is interesting to note that this fixed point is (−1, −1).

Example 5.2





x1 (k) 0 + 1) Consider the system xx21 (k (k + 1) = A x2 (k) + Bs(k), where A = a + 1

 

 0.1875 B = 02 , a = 3 and xx21 (0) (0) = −0.25 . Plot the phase portrait of the system.

1 a

 ,

Solution: The phase portrait of the system is shown on Figure 5.2. It can  be seen  from −1 Figure 5.2 that the state vector converges to a fixed point xx21 (k) = (k) −1 .

Linear and Affine Linear Behaviors

67

Filters with a pole located at z = −1 For a second order digital filter with a pole located at z = −1, the filter coefficients are required to satisfy the following condition: b = −a + 1.

(5.15)

Similarly, we also only consider the autonomous response, that is: u(k) = 0

(5.16)

∀k ≥ 0. Remark 5.3 It can be shown that properties similar to Theorem 5.1 and Theorem 5.2 can be obtained accordingly as stated below.

Theorem 5.3 For b = −a + 1 and |a − 1| > 1, s(k) = 0 ∀k ≥ k0 if and only if ∃k0 ∈ Z+ ∪ {0} such that x1 (k0 ) = x2 (k0 ). Proof: The proof is omitted here.

Example 5.3





 + 1) x1 (k) 0 1 Consider the system xx21 (k (k + 1) = A x2 (k) + Bs(k), where A = −a + 1 a ,



  −0.1875 B = 02 , a = 3 and xx21 (0) = (0) −0.1234 . Plot the phase portrait of the system. Solution: The phase portrait of the system is shown in Figure 5.3. It can be seen from the figure that the state vector converges to a fixed point on a particular straight line x1 = x2 of the phase plane.

Theorem 5.4 For b = −a + 1 and a being an odd integer, there does not exist k0 ∈ Z+ such that s(k) = a ∀k ≥ k0 .

68

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

1 0.8 0.6 0.4 x(2)

x2

0.2 0

x(1) x(0)

0.2 0.4 0.6 0.8 1 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

x1 Figure 5.3 The phase portrait of the second order digital filter associated with two’s complement arithmetic. The points x(0), x(1), x(2) are as annotated, and the point with ‘∗ ’ denotes the ‘steady state’ of x.

Proof: 

  k−1 k−1−j k−k0 x1 (k0 ) + Bs( j) ∀k > k0 , if s(k) = a Since xx21 (k) j=k0 A (k) = A x2 (k0 ) ∀k ≥ k0 , we have: ⎡ ⎤  k−k0 x (k ) − x (k ) − 2a (a − 1) 2 0 1 0 ⎢ ⎥ 2−a ⎢ ⎥ ⎢ ⎥ ⎢ + (a − 1)x (k ) − x (k ) + 2a − 2a(k − k ) ⎥ & ' ⎢ ⎥ 1 0 2 0 0 1 ⎢ x1 (k) 2−a ⎥ =  ⎢ ⎥. x2 (k) ⎥ a−2 ⎢ 2a ⎢(a − 1)k−k0 +1 x2 (k0 ) − x1 (k0 ) − ⎥ ⎢ ⎥ 2 − a ⎢ ⎥ ⎣ ⎦ 2a(a − 1) + (a − 1)x1 (k0 ) − x2 (k0 ) + − 2a(k − k0 ) 2−a (5.17)

 x1 (k) 2 As k → +∞, k − k0 → +∞, so lim x2 (k) ∈/ I . Hence, there does not exist k→+∞

k0 ∈ Z+ such that s(k) = a ∀k ≥ k0 , and this proves the theorem.

69

Linear and Affine Linear Behaviors

1

x(2)

0.8 0.6 x(0) 0.4

x2

0.2 0 x(1) 0.2 0.4 0.6 0.8 1 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

x1

Figure 5.4 The phase portrait of the second order digital filter associated with two’s complement arithmetic. The points x(0), x(1), x(2) are as annotated, and the point with ‘∗ ’ denotes the ‘steady state’ of x.

Example 5.4





 + 1) x1 (k) 0 1 Consider the system xx21 (k = A + Bs(k), where A = x2 (k) −a + 1 a ,

 (k + 1) 

 0 x1 (0) 0.7826 B = 2 , a = 3 and x2 (0) = 0.5242 . Plot the phase portrait of the system and check if ∃k0 ≥ 0 such that s(k) = a ∀k ≥ k0 . Solution: The phase portrait of the system is shown in Figure 5.4. It can be checked that ∃k0 ≥ 0 such that s(k) = 0 = a ∀k ≥ k0 . Observation 5.1 When b = −a + 1 and a is an odd integer, the state vector converges to a fixed point on a particular straight line x1 = x2 of the phase plane ∀x(0) ∈ I 2 . Example 5.5 Generate a random initial condition x(0) in I 2 , plot the response of the state variables with a = 5 and b = −4.

70

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

1 0.8 0.6 0.4

x2(k)

0.2 0 0.2 0.4 0.6 0.8 1

0

50

100

150

200

250

Time index k Figure 5.5 The response of the second order digital filter associated with two’s complement arithmetic. The initial condition is x(0) = [−0.1886 0.87909]T , a = 5 and b = −4, the state converges to a fixed point.

Solution: The response of the state variables is shown on Figure 5.5. As can be seen in Figure 5.5, the state converges to a fixed point.

LIMIT CYCLE BEHAVIOR Filters with a DC pole In this section several conditions for exhibiting eventually periodic state vector are presented. Theorem 5.5

  x1 (k + M) For b = a + 1, if ∃M ∈ Z+ such that xx21 (k) = (k) x2 (k + M) ∀k ≥ k0 , then k0 −1 s( j) M M (x1 (0) + x2 (0))((a + 1) − 1) + 2(2(a + 1) − 1) j=0 (a + 1) j+1 + 2(a + 1)M k0 +M−1 s( j) = 0. j=k0 (a + 1) j+1

71

Limit Cycle Behavior

Proof: For b = a + 1,  the

digital  filter  can  be represented by the state equax1 (k + 1) 0 1 x1 (k) 0 tion x2 (k + 1) = a + 1 a x2 (k) + 2 s(k). Hence, the solution of the system is: *& ) & ' ' (a + 1)k − (−1)k x1 (k) x1 (0) (a + 1)k + (a + 1)(−1)k 1 = x2 (k) a + 2 (a + 1)k+1 − (a + 1)(−1)k (a + 1)k+1 + (−1)k x2 (0) ) * k−1 2 # (a + 1)k−1−j − (−1)k−1−j s( j). (5.18) + a+2 (a + 1)k−j + (−1)k−1−j j=0 If ∃M ∈ Z+ such that



x1 (k) x2 (k)

=



x1 (k + M) x2 (k + M)

∀k ≥ k0 , then:

*& ) ' x1 (0) (a + 1)k0 − (−1)k0 (a + 1)k0 + (a + 1)(−1)k0 1 a + 2 (a + 1)k0 +1 − (a + 1)(−1)k0 (a + 1)k0 +1 + (−1)k0 x2 (0) ) * k0 −1 (a + 1)k0 −1−j − (−1)k0 −1−j 2 # + s( j) a+2 (a + 1)k0 −j + (−1)k0 −1−j j=0 * ) (a + 1)k0 +M − (−1)k0 +M (a + 1)k0 +M + (a + 1)(−1)k0 +M 1 = a + 2 (a + 1)k0 +M+1 − (a + 1)(−1)k0 +M (a + 1)k0 +M+1 + (−1)k0 +M ) * & ' k +M−1 x1 (0) (a + 1)k0 +M−1−j − (−1)k0 +M−1−j 2 0# × s( j). + x2 (0) a+2 (a + 1)k0 +M−j + (−1)k0 +M−1−j j=0 (5.19) Hence, ⎧ ⎞ ⎛ k# 0 −1 ⎪ k0 ⎪ s( j) (a + 1) ⎪ ⎪ ⎠ ⎝x1 (0) + x2 (0) + 2 ((a + 1)M − 1) ⎪ ⎪ ⎪ a+2 (a + 1) j+1 ⎪ j=0 ⎪ ⎪ ⎪ ⎞ ⎛ ⎪ ⎪ k# ⎨ 0 −1 k0 s( j) (−1) ⎠. ⎝(a + 1)x1 (0) − x2 (0) − 2 +((−1)M − 1) j+1 ⎪ a + 2 (−1) ⎪ ⎪ j=0 ⎪ ⎪ ⎪ ⎪ k +M−1 ⎪ 0 # ⎪ ⎪ ⎪ + 2 ⎪ s( j)((a + 1)k0 +M−1−j − (−1)k0 +M−1−j ) = 0 ⎪ ⎩ a+2 j=0

(5.20)

72

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

⎧ ⎞ ⎛ k# 0 −1 ⎪ k0 +1 ⎪ s( j) (a + 1) ⎪ ⎪ ⎠ ⎝x1 (0) + x2 (0) + 2 ((a + 1)M − 1) ⎪ ⎪ ⎪ a+2 (a + 1) j+1 ⎪ j=0 ⎪ ⎪ ⎪ ⎞ ⎛ ⎪ ⎪ k −1 ⎨ 0 k0 # s( j) (−1) ⎝2 + ((−1)M − 1) − ((a + 1)x1 (0) − x2 (0))⎠ ⎪ a+2 (−1) j+1 ⎪ ⎪ j=0 ⎪ ⎪ ⎪ ⎪ k +M−1 ⎪ 0 # ⎪ 2 ⎪ ⎪ (5.21) ⎪ + s( j)((a + 1)k0 +M−j + (−1)k0 +M−1−j ) = 0. ⎪ ⎩ a+2 j=0

⎞ ⎛ k# 0 −1 k0 s( j) (a + 1) ⎠, (5.22) ⎝x1 (0) + x2 (0) + 2 t1 = ((a + 1)M − 1) a+2 (a + 1) j+1 j=0 ⎞ ⎛ k# 0 −1 k0 s( j) (−1) ⎠, (5.23) ⎝(a + 1)x1 (0) − x2 (0) − 2 t2 = ((−1)M − 1) a+2 (−1) j+1

Let

j=0

t3 =

2 a+2

k0 +M−1 #

s( j)(a + 1)k0 +M−1−j ,

(5.24)

j=0

and

Then we have:



2 a+2

k0 +M−1 #

s( j)(−1)k0 +M−1−j .

(5.25)

t1 + t 2 + t 3 − t 4 = 0 , (a + 1)t1 − t2 + (a + 1)t3 + t4 = 0

(5.26)

t4 =

j=0

which implies a = −2

or

t1 = −t3 .

(5.27)

For a = −2, we have t1 = −t3 , that is: ⎞ ⎛ k# 0 −1 k0 s( j) (a + 1) ⎠ ⎝x1 (0) + x2 (0) + 2 ((a + 1)M − 1) a+2 (a + 1) j+1 j=0

=−

2 a+2

k0 +M−1 # j=0

s( j)(a + 1)k0 +M−1−j ,

(5.28)

73

Limit Cycle Behavior

⇒ (x1 (0) + x2 (0))((a + 1)M − 1) + 2(2(a + 1)M − 1)

k# 0 −1 j=0

+ 2(a + 1)M

k0 +M−1 # j=k0

s( j) (a + 1) j+1

s( j) = 0. (a + 1) j+1

This completes the proof. Remark 5.4 If, after a number of iterations, the state vector is periodic with period M, then the symbolic sequence will also be periodic with the same period, that is, s ∈ β . Hence, Theorem 5.5 gives a necessary condition for a symbolic sequence to be periodic after a number of iterations. However, s(k) is an integer in {−m, . . . , −1, 0, 1, . . . m} and the periodicity of the symbolic sequence is M. So there are (2m + 1)M possibilities of s. Example 5.6 Consider Example 5.1. Check if Theorem 5.5 is satisfied or not. Solution:

 0.9003 By putting k0 = 32, M = 2, a = 3 and x(0) = −0.5377 into Theorem 5.5, it can be checked easily that it is satisfied. Remark 5.5 Refer to Theorem 5.1. Since one of the eigenvalues may be unstable, one may predict that the periodic orbits are unstable. However, a counter intuitive phenomenon is observed in which the periodic orbits are stable if a is an odd integer. This counter intuitive phenomenon is presented in Observation 5.2. Observation 5.2 When b = a + 1 and a is an odd integer, the state vector toggles between two states at the steady state on a particular

straight line x1 = −x2 of the phase plane ∗ 2 or converges to a fixed point x = −1 −1 ∀x(0) ∈ I . Example 5.7 Generate a random initial condition x(0) in I 2 , plot the response of the state variables with a = 5 and b = 6.

74

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

1 0.8 0.6 0.4

x2(k)

0.2 0 0.2 0.4 0.6 0.8 1

0

20

40

60

80

100

120

Time index k Figure 5.6 The response of the second order digital filter associated with two’s complement arithmetic. The initial condition x(0) = [−0.7826 0.5242]T is generated randomly. When a = 5 and b = 6, the state converges to a period 2 signal.

Solution: The response of the state variables is shown on Figure 5.6. As can be seen in Figure 5.6, the state converges to a signal with a period of 2.

Filters with a pole located at z = −1 As in Theorem 5.5, a limit cycle behavior can occur when there is a pole located at z = −1. This property is stated in the following theorem: Theorem 5.6

  x1 (k + M) For b = −a + 1, if ∃M ∈ Z+ such that xx21 (k) = (k) x2 (k + M) ∀k ≥ k0 , then 

  k −1 s( j) 0 0 +M−1 (1 − (a − 1)M ) x1 (0) − x2 (0) − 2 j=0 (a + 1)M−j−1× +2 kj=k (a − 1) j+1 0 s( j) = 0. Proof: The proof is omitted here.

Chaotic Behavior

75

Example 5.8 Consider Example 5.3. Check if Theorem 5.6 is satisfied or not. Solution: 

−0.1875 By putting k0 = 57, M = 1, a = 3 and x(0) = −0.1234 into Theorem 5.6, we can see that it is easily satisfied.

CHAOTIC BEHAVIOR Filters with a DC pole Observation 5.3 Contrary to Observation 5.2, when b = a + 1 and a deviates slightly from an odd integer, the state neither converges to a periodic signal nor a fixed point.

Example 5.9 Generate a random initial condition x(0) in I 2 , plot the response of the state variables with a = 3.001 and b = 4.001. Solution: The response of the state variables is shown on Figure 5.7. As can be seen in Figure 5.7, chaotic behavior is exhibited.

Filters with a pole located at z = −1 Observation 5.4 Similarly, when b = −a + 1 and a deviates slightly from an odd integer, the state does not converge to a fixed point.

Example 5.10 Generate a random initial condition x(0) in I 2 , plot the response of the state variables with a = 5.01 and b = −4.01. Solution: The response of the state variables is shown on Figure 5.8. As can be seen from Figure 5.8 chaotic behavior is exhibited.

76

5 Autonomous Response of Digital Filters with Two’s Complement Arithmetic

1 0.8 0.6 0.4

x2(k)

0.2 0 0.2 0.4 0.6 0.8 1

0

200

400

600

800

1000

1200

Time index k

Figure 5.7 The response of the second order digital filter associated with two’s complement arithmetic. The initial condition is x(0) = [−0.8 0.7999]T , a = 3.001 and b = 4.001. The state neither converges to a periodic signal nor a fixed point. 1 0.8 0.6 0.4

x2(k)

0.2 0 0.2 0.4 0.6 0.8 1

0

50

100

150

200

250

Time index k

Figure 5.8 The response of the second order digital filter associated with two’s complement arithmetic. The initial condition is x(0) = [−0.5 0.5001]T , a = 5.01 and b = −4.01. The state does not converge to a fixed point.

Exercises

77

SUMMARY In this chapter autonomous response of second order lowpass and highpass digital filters associated with two’s complement arithmetic were discussed, and conditions for the occurrence of limit cycles were derived. Based on the derived conditions, some counter intuitive phenomena were presented.

EXERCISES 1. 2. 3. 4. 5. 6.

Prove Theorem 5.3. Prove Theorem 5.6. Plot the phase portrait of the system shown in Figure 5.5. Plot the phase portrait of the system shown in Figure 5.6. Plot the phase portrait of the system shown in Figure 5.7. Plot the phase portrait of the system shown in Figure 5.8.

6 STEP RESPONSE OF DIGITAL FILTERS WITH TWO’S COMPLEMENT ARITHMETIC

In Chapter 5 we presented nonlinear behaviors exhibited in the autonomous response of both lowpass and highpass digital filters associated with two’s complement arithmetic. However, in practice various types of input signals are usually applied. As a result, we need to analyze forced input systems in addition to autonomous systems. In this chapter we look at nonlinear behaviors exhibited in the step response of bandpass digital filters. It is well-known that, in general, the method of affine transformation cannot relate the step response behaviors and autonomous response behaviors in a straightforward and simplistic manner if the system is nonlinear. We show how the step response behaviors can be related to the autonomous response behaviors in an explicit manner where even overflow nonlinearity occurs. Based on the affine transformation method, some differences between the step response and the autonomous response, as well as some counter intuitive phenomena, are given here. AFFINE LINEAR BEHAVIOR For a marginally stable bandpass filter (refer to Section 5.1), the filter coefficients are as follows: b = −1, (6.1) and |a| ≤ 2. We now consider the step response case, that is: u(k) = c ∀k ≥ 0 and c ∈ . 78

(6.2) (6.3)

Affine Linear Behavior

79

There are three types of trajectories exhibited on the phase plane, namely the type I, II, and III trajectories. The type I, II and III trajectories are defined as trajectories that give a single rotated ellipse, some rotated and translated ellipses, and an elliptical fractal pattern, respectively. The occurrence of a particular type of trajectory depends on the initial condition. For the type I trajectory, there is a single ellipse exhibited on the phase plane. The symbolic sequences are constant. The set of initial conditions for the type I trajectory is an elliptic region on the phase plane. The following theorem shows that these three statements are equivalent to each other.

Theorem 6.1 Let cos θ = &

1 cos θ

T= &



A=

cos θ −sin θ

x∗ =

a , 2 ' 0 , sin θ ' sin θ , cos θ

& ' c 1 , 2−a 1

(6.4)

(6.5)

(6.6)

(6.7)

and & 

x(k) =

 ' c + 2s0 ∗ xˆ 1 (k) x , = T−1 x(k) − xˆ 2 (k) c

(6.8)

where s0 ≡ s(0). Then the following three statements are equivalents: 



i) x(k + 1) = Ax(k) ∀k ≥ 0. ii) s(k) = s+0 ∀k ≥ , 0.  , iii) x(0) ∈ x(0) : ,T−1 x(0) −

c+2s0 ∗ c x

, , |c + 2s | , ≤ 1 − 2 − a0 .

(6.9)

80

6 Step Response of Digital Filters with Two’s Complement Arithmetic

Proof: The proof can be divided into two parts. The first part is to prove   that x(k + 1) = Ax(k) ∀k ≥ 0 if and only if s(k) = s0 ∀k ≥ 0. And the second part is ,  to prove that

, s(k) = s0 ∀k-≥ 0 if and only if + , −1 c + 2s0 ∗ , x(0) − c x , ≤ 1 − |c2+−2sa0 | . x(0) ∈ x(0) : ,T For the necessity of the first part, since x(k + 1) = Ax(k) + Bu(k) +

 u(k) = c, s(k) = s0 ∀k ≥ 0, and B = 01 ,

 0 2

s(k),

we have x(k + 1) = Ax(k) + (c + 2s0 )B.

(6.10)

 c + 2s0 ∗ x x(k + 1) = T−1 x(k + 1) − c

(6.11)



implies that 

x(k + 1) = T 

−1

Since A = TAT

−1



c + 2s0 ∗ Ax(k) + (c + 2s0 )B − x . c

(6.12)

and Bc = (I − A)x∗ , we have 



x(k + 1) = Ax(k),

(6.13)

and this proves the necessity of the first part. For the sufficiency of the first part, if 



x(k + 1) = Ax(k),

then T

−1



c + 2s0 ∗ x(k + 1) − x c





= AT

−1



(6.14)

c + 2s0 ∗ x(k) − x , c

which implies that x(k + 1) = Ax(k) + Bc +

& ' 0 s . 2 0

(6.15)

(6.16)

This further implies that s(k) = s0 ∀k ≥ 0. This proves the sufficiency of the first part.

(6.17)

81

Affine Linear Behavior

For the necessity of the second part, when s(k) = s0 ∀k ≥ 0, from the

   above, we have x(k + 1) = Ax(k) ∀k ≥ 0, where x(k) = T−1 x(k) − c +c2s0 x∗ . 



Since ,

, of x(k + 1) = Ax(k) is a circle with radius  the phase portrait , −1 c + 2s0 ∗ , x x(0) − ,, we can let ,T c ,  , & ' , −1 c + 2s0 ∗ ,  , cos φ(k) . x(0) − (6.18) x(k) = , T x , sin φ(k) , c This implies that ,  , & ' & ' , −1 c + 2s0 ∗ , c + 2s0 1 cos φ(k) , x(k) = , T x(0) − + x . (6.19) , , cos(θ − φ(k)) c 2−a 1 Since x(k) ∈ I 2 , that is |x1 (k)| ≤ 1

(6.20)

|x2 (k)| ≤ 1,

(6.21)

and

we have ,   , , −1 c + 2s0 ∗ , c + 2s0  ,T , x(0) − x , cos (φ(k)) + ≤ 1, , c 2−a 

(6.22)

which implies that ,  , , −1 c + 2s0 ∗ , ,T , ≤ 1 − |c + 2s0 | , x(0) − x , , c 2−a

(6.23)

and this proves the necessity of the second part. 

 For the sufficiency of the second part, since x(k) = T−1 x(k) − c +c2s0 x∗ , 

A = TAT

−1

and Bc = (I − A)x∗ ,

we have 



x(k + 1) = Ax(k) + T

+ ,  If x(0) ∈ x(0) : ,T−1 x(0) − 

c + 2s0 ∗ x c

−1

(6.24)

& ' 0 (s(k) − s0 ) . 2

, , ≤ 1 − |c + 2s0 | , then ˆx(0) ≤ 1 − 2−a 

(6.25) |c + 2s0 | . 2−a

Since A = 1, we have s(0) = s0 and ˆx(1) = Ax(0) ≤ 1 − |c2+−2sa0 | .

82

6 Step Response of Digital Filters with Two’s Complement Arithmetic

, + 

, , , Assume that x(k) ∈ x(k) : ,T−1 x(k) − c +c2s0 x∗ , ≤ 1 − |c2+−2sa0 | , then using an approach similar, we can show that s(k) = s0 and , , , , |c + 2s0 | , ,xˆ (k + 1), = , . (6.26) ,Ax(k), ≤ 1 − 2−a Hence, by mathematical induction, we conclude that s(k) = s0 ∀k ≥ 0, and this proves the sufficiency of the second part and completes the proof.

Trajectory pattern 





Since A is a rotation matrix, x(k + 1) = Ax(k) ∀k ≥ 0 corresponds to  a, circular trajectory , with a center at the origin and radius x(0) =  , , −1 x(0) − c +c2s0 x∗ ,. By the transformation defined in Theorem 6.1, that ,T 

 is x(k) = T−1 x(k) − c +c2s0 x∗ , the trajectory of x(k) is an elliptical orbit with

 the center of the ellipse at c +c2s0 x∗ . Since c +c2s0 x∗ = c2+−2sa0 11 , the center is on the diagonal line x2 = x1 . Compared to that of the  autonomous response, the center of the ellipse is shifted by the vector 2 −c a 11 , which depends on the input step size and the filter parameter a. Figure 6.1 shows the phase portrait of such a system when a = −1.5 and c = 1 with different

 initial conditions. Figure 6.1a shows the trajectory when x(0) = −0.8 −0.9 . Figure 6.1b shows the corresponding symbolic sequence

 0.2 s(k) = −2 ∀k ≥ 0. Figure 6.1c shows the trajectory when x(0) = −0.8 . Figure 6.1d shows the corresponding symbolic sequence

 s(k) = −1 ∀k ≥ 0. Figure 6.1e shows the trajectory when x(0) = −0.2 0.8 . Figure 6.1f shows the corresponding symbolic

 sequence s(k) = 0 ∀k ≥ 0. Figure 6.1g shows the trajectory when x(0) = 0.9 0.8 . Figure 6.1h shows the corresponding symbolic sequence s(k) = 1 ∀k ≥ 0. According to Theorem 6.1, a system will give a type I trajectory if and only if the symbolic sequence is constant at any time instant k ≥ 0. Based on this necessary and sufficient condition, some counter intuitive phenomena are reported in Example 6.1. Theorem 6.1 implies that the system may also give the type I trajectory even when overflow occurs, that is, ∃k ∈ Z + ∪ {0} such that s(k) = 0. As examples, when s(k) = −2 ∀k ≥ 0, s(k) = −1 ∀k ≥ 0, or s(k) = 1 ∀k ≥ 0,

83

Affine Linear Behavior 1

0.8 0.82

1.5 s(k)

x2

0.84 0.86 0.88

2 2.5

0.9

3

0.92 0.92 0.9 0.88 0.86 0.84 0.82 0.8

0

10

20

30

40

50

40

50

40

50

40

50

Time index k

x1 (a)

(b) 0

0.4 0.2

0.5

0.2 s(k)

x2

0

0.4 0.6

1 1.5

0.8 1 1 0.8 0.6 0.4 0.2

0

0.2

2

0.4

0

10

20

30

Time index k

x1 (c)

(d) 1

1

0.8 0.5

0.6 s(k)

x2

0.4 0.2 0

0 0.5

0.2 0.4 0.4 0.2

0

0.2

0.4

0.6

0.8

1

1

0

10

x1

20

30

Time index k

(e)

(f) 2

0.92 0.9

1.5 s(k)

x2

0.88 0.86

1

0.84 0.5 0.82 0.8 0.8

0.82

0.84

0.86

0.88

0.9

0

0.92

x1 (g)

0

10

20

30

Time index k (h)

Figure 6.1 The phase portrait and the symbolic sequences for the type I trajectories.

84

6 Step Response of Digital Filters with Two’s Complement Arithmetic

overflow does occur, but the system still gives the type I trajectory, as illustrated in Figure 6.1. Example 6.1 Consider a strictly stable second order digital filter with the eigenvalues of the matrix A being complex, that is b = −r 2 and a = 2r cos θ in which 0 < r < 1 and θ ∈ . Show that if u(k) = s(k) = 0 ∀k ≥ 0, then the phase trajectory converges to the origin.  +  θ Define Ik2 ≡ x(0) : x(0) ∈ I 2 and |sin((k + 1)θ)x2 (0) − r sin (kθ)x1 (0)| x2 (0) >

   θ r sin(kθ)x1 (0) +  sin k r  sin((k + 1)θ)

 +  θ Ik2 ≡ x(0) : x(0) ∈ I 2 and |sin((k + 1)θ)x2 (0) − r sin(kθ)x1 (0)| <  sin k r 

∀k ≥ 0, we can characterize Ik2 as the region bounded between two parallel straight lines in I 2 described by the inequalities in eqns (6.27) or (6.28). Although the slope of the two parallel straight lines for Ik2 is different for different values

85

Affine Linear Behavior

0.5 0.4 0.3 0.2

x2

0.1 0 0.1 0.2 0.3 0.4 0.5 0.5

0.4

0.3

0.2

0.1

0.1

0

0.2

0.3

0.4

0.5

x1

(a) 1 0.8 0.6 0.4

x2(0)

0.2 0

0.2 0.4 0.6 0.8 1 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

x1(0) (b) Figure 6.2 (a) The phase portrait of strictly stable second &order 'digital filters associated with two’s 0.5 ; (b) The set of initial conditions of complement arithmetic for r = 0.9999, θ = 0.75π and x(0) = −0.5 the corresponding digital filters when s(k ) = 0 ∀k ≥ 0.

86

6 Step Response of Digital Filters with Two’s Complement Arithmetic

of k, the y-axis intercepts of these two lines move away from the origin as k increases. Hence, ∃K ≥ 0 such that Ik2 = I 2 ∀k ≥ K. This implies that ∃K  ≥ 0 . .K  2 2 2 such that K  ≤ K and +∞ k=0 Ik ⊆ I . As a result, if overflow does k=0 Ik = .K  2 not occur, then x(0) ∈ k=0 Ik . This set of initial conditions corresponds to a polygon on the phase plane as shown in Figure 6.2b, and the number of sides of the polygon depends on the value of K  .

Periodicity of the state vector Even if a single ellipse is exhibited on the phase plane, the state variables  may not be periodic. Since A is a rotation matrix, x(k) is periodic if and only if θ is a rational multiple of π. It is worth noting that the frequency spectrum of x1 (k) and x2 (k) consists of impulses located at the DC frequency and at the natural frequency of the digital filter, whether x(k) is periodic or not (it is one of various differences between continuous time systems and discrete time systems). For the symbolic sequences, since s(k) = s0 ∀k ≥ 0, the frequency spectrum of s(k) consists of an impulse located at the DC frequency only. Example 6.2 Consider the same strictly stable second order digital filter discussed in Example 6.1. Is the state trajectory corresponding to s(k) = 0 ∀k ≥ 0 periodic when θ is a rational multiple of π? Solution: Since x(k) = TDk T−1 x(0), it is aperiodic even though θ is a rational multiple of π.

Set of initial conditions for the type I trajectory The set of initial conditions corresponding to the type I trajectory consists of rotated and translated elliptical regions. For each value of s0 , the region is characterized by a single rotated and translated ellipse with the center at c +c2s0 x∗ . It is interesting to note that these centers are the same as that for the trajectory described in Section 6.1.1, and so these centers are also on the diagonal line x2 = x1 . Compared to the autonomous response, the set of admissible initial

 conditions is shifted by the vector 2 −c a 11 , which also depends on the input step size and the filter parameter a.

87

Affine Linear Behavior

1 s(k))  1

0.8 0.6 0.4

s (k))  0

x2

0.2 0 0.2 s(k))  1 

0.4 0.6 0.8 1 1

s (k))  2  0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

x1 Figure 6.3 Set of initial conditions that gives the type I trajectory.

Moreover, by transforming these ellipses to circles, the radii of the circles are 1−

|c + 2s0 | . 2−a

For the autonomous response case, the radii of the circles are 1, 1 −

2 4 ,1 − ,··· , 2−a 2−a

depending on the values of s0 . The corresponding radii of the circles for the step response case are · · · , 1−

|c + 2| |c| |c − 2| ,1 − ,1 − ,··· . 2−a 2−a 2−a

Comparing the two sequences, it can be concluded that the sizes of the ellipses for the step response case are smaller than those for the autonomous response case. Figure 6.3 shows the set of initial conditions that gives the type I trajectory when a = −1.5 and c = 1. Example 6.3 Consider the same strictly stable second order digital filter discussed in Example 6.1. Assume that u(k) = 0 and s(k) = s0 = 0 ∀k ≥ 0. Show that the trajectory converges to a fixed point not at the origin.

88

6 Step Response of Digital Filters with Two’s Complement Arithmetic

Define:

2 Ik,s 0

⎫ ⎧ x(0) : x(0) ∈ I 2 and ⎪ ⎪ ⎪ ⎪  ⎪ ⎪ k ⎪ ⎪ ⎬ ⎨ r (sin((k + 1)θ)x (0) − r sin(kθ)x (0)) 2 1  . ≡ sin θ  ⎪ ⎪ ⎪ ⎪  2s0 ⎪ ⎪ k k+1 ⎪ ⎪  ⎩ + (sin θ − r sin((k + 1)θ) + r sin(kθ)) < 1⎭ sin θ(1 − 2r cos θ + r 2 )

.Ks0 2 2 Show that ∃Ks0 ≥ 0 such that x(0) ∈ ∀s0 =0 k=0 Ik,s0 , and the set of initial conditions also corresponds to polygons on the phase plane. Solution: Since x(k + 1) = Ax(k) + Bs(k) ∀k ≥ 0, if s(k) = s0 = 0 ∀k ≥ 0, then ⎤ r k−1 (0) − r sin((k − 1)θ)x (0)) (sin(kθ)x 2 1 ⎥ ⎢ sin θ ⎥ x(k) = Ak x(0) + An Bs0 = ⎢ ⎦ ⎣ k r n=0 (sin((k + 1)θ)x2 (0) − r sin(kθ)x1 (0)) sin θ & ' 2s0 sin θ − r k−1 sin(kθ) + r k sin((k − 1)θ) + ∀k ≥ 1. sin θ · (1 − 2r cos θ + r 2 ) sin θ − r k sin((k + 1)θ) + r k+1 sin(kθ) ⎡

k−1 #

0 Denote x∗ = 1 − 2r 2s cos θ + r 2

lim x(k) =

k→+∞



2s0 1 − 2r cos θ + r 2

1 . Since 0 < r < 1, we have

1  1 ∗ 1 =x .

Hence, x(k) will converge to x∗ . If s0 = 0, then x∗ = 0. Hence, the trajectory converges to a fixed point not at the origin as shown in Figure 6.4a. In fact, when s0 = 0, then x∗ = 0. This reduces to the case discussed in Example 6.1. ∈ I 2 implies that x(k)  rk 2s0 (sin θ −  sin θ (sin((k + 1)θ)x2 (0) − r sin(kθ)x1 (0)) + sin θ(1 − 2r cos θ + r 2 )   r k sin((k + 1)θ) + r k + 1 sin(kθ)) < 1 ∀k ≥ 0. If sin((k + 1)θ) > 0, then    θ −  sin − rk

x2 (0)