Multiresolution and Multirate Signal Processing: Introduction, Principles and Applications 9789352601448

Table of contents :
Title
Contents
1 Introduction
2 The Haar Wavelet
3 The Haar Filter Bank
4 M-Channel Filter Bank
5 Conjugate Quadrature Filter Bank
6 Daubechies Family
7 Time-Frequency Joint Perspective
8 The Uncertainty Principle
9 Time Frequency Plane and Tilings
10 Dyadic MRA
11 MRA Variant 1: Bi-orthogonal Filters
12 MRA Variant 2: Splines
13 MRA Variant 3: Wave Packets
14 MRA Variant 4: Lifting Scheme
15 Other Wavelet Families
16 Advanced Topics
17 Wavelet Applications
18 Going Beyond the Realms
Appendix Extended Notes
References and Bibliography
Index
Untitled

Citation preview

Multiresolution and Multirate Signal Processing Introduction, Principles and Applications In

Multiresolution and Multirate Signal Processing Introduction, Principles and Applications In

Vikram M. Gadre Professor, Department of Electrical Engineering, IIT Bombay, Mumbai

Aditya S. Abhyankar Dean, Faculty of Technology, and Professor and Head, Department of Technology, Savitribai Phule Pune University, Pune

McGraw Hill Education (India) Private Limited CHENNAI McGraw-Hill Education Offices Chennai New York St Louis San Francisco Auckland Bogotá Caracas Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal San Juan Santiago Singapore Sydney Tokyo Toronto

McGraw Hill Education (India) Private Limited Published by McGraw Hill Education (India) Private Limited 444/1, Sri Ekambara Naicker Industrial Estate, Alapakkam, Porur, Chennai - 600 116 Multiresolution and Multirate Signal Processing Copyright © 2017 by McGraw Hill Education (India) Private Limited. No part of this publication may be reproduced or distributed in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise or stored in a database or retrieval system without the prior written permission of the publishers. The program listings (if any) may be entered, stored and executed in a computer system, but they may not be reproduced for publication. This edition can be exported from India only by the publishers, McGraw Hill Education (India) Private Limited. Print Edition: ISBN-13: 978-93-5260-144-8 ISBN-10: 93-5260-144-0 Ebook Edition: ISBN-13: 978-93-5260-145-5

ISBN-10: 93-5260-145-9 Managing Director: Kaushik Bellani Director—Products (Higher Education and Professional): Vibha Mahajan Manager: Product Development: Koyel Ghosh Specialist: Product Development: Sahil Thorpe Head—Production (Higher Education and Professional): Satinder S Baveja Manager—Editorial Services: Hema Razdan Senior Production Manager: Manohar Lal AGM: Product Management (Higher Education and Professional): Shalini Jha Product Manager: Kartik Arora General Manager—Production: Rajender P Ghansela Manager—Production: Reji Kumar

Information contained in this work has been obtained by McGraw Hill Education (India), from sources believed to be reliable. However, neither McGraw Hill Education (India) nor its authors guarantee the accuracy or completeness of any information published herein, and neither McGraw Hill Education (India) nor its authors shall be responsible for any errors, omissions, or damages arising out of use of this information. This work is published with the understanding that McGraw Hill Education (India) and its authors are supplying information but are not attempting to render engineering or other professional services. If such services are required, the assistance of an appropriate professional should be sought.

Typeset at Chandraprabhu Enterprise Private Limited, Puducherry and printed at Cover Printer: Visit us at: www.mheducation.co.in

Foreword

Wavelets and multiresolution analysis created revolution in signal processing nearly three decades ago. Since then the area of research has grown steadily, covering many application areas as well. The topic has had far reaching impact on the way we understand signals, their transforms, and the limitations of time-frequency analysis. These techniques are founded on fundamental principles in mathematics, and at the same time they have great relevance to many practical applications in areas such as audio signal analysis, image processing, medical signal analysis, biological signal processing, and so forth. Wavelet and multiresolution methods have close relation to the theory of multirate filter banks, which also evolved around the same time, nearly three decades ago. For example, the first-order two channel orthonormal filter bank is related to the Haar wavelets through a tree structure, and the more general M-channel orthonormal filter bank can be related to orthonormal wavelets. These connections are also well established in the literature. This book, authored by Prof. Gadre and Prof. Abhyankar, gives an excellent and up-to-date exposure to the developments in this field. First and foremost, it is written in a reader friendly style. There are many examples throughout the book which motivate the introduction of deep and subtle topics. The authors are careful to make this motivation because, as such, the topic is complicated, and needs this kind of an introduction. The examples chosen here range from simple toy examples to actually practical ones from a wide range of application areas such as image processing, audio, biological signals, and many others. While the book has this attractive style, it also makes sure that the theoretical depth is impressive. Especially in the later chapters, the authors do justice to the detailed and rather intricate theory of multiresolution by providing deeper discussions and derivations, again supplemented with a generous supply of examples. The thoughtful addition of MATLAB codes will be very valuable to students and practicing engineers, as they allow the reader to quickly construct examples and plots that enhance the understanding of the theory just presented. The homework problems at the end of each chapter are thought-provoking and useful. The introduction of historical anecdotes and notes is an added bonus, and makes it informative for the reader. Any student with an introductory background in signal processing and mathematics will be able to benefit from the book. More advanced readers will also thoroughly enjoy the wealth of material which can motivate further research in the area, and initiate advanced projects in graduate curriculum.

vi

Foreword

In short, this is a book on a very important and well established topic, which every student and lover of signal processing should learn. It is authored by highly reputed scholars who have taken great pains to present the material in a very welcoming style. It is my great pleasure to welcome this book, which will be a wonderful addition to the literature on signal processing. Prof. P. P. Vaidyanthan Department of Electrical Engineering California Institiute of Technology

Preface

Both of us, as instructors on basic and advanced subjects related to signal processing, have felt the need to strike the delicate balance between inspiring an audience of students to penetrate deep into the subject of multirate signal processing and being ‘true to the subject’ in terms of rigor and formalism. This has been made possible by making the study and search pleasurable. The subject of wavelets and multiresolution signal processing has posed a major challenge to us, from this point of view. In the numerous opportunities that we have had, to teach or discuss the subject, the audience has always demanded a ‘down to earth exposition of the subject’, a ‘connection to everyday reality’, a ‘compendium of exercises which will help them understand the theory behind the subject’, a ‘hands-on experience to feel the subject through implementations’ and things of similar nature. This book is a humble endeavour on our part to meet these demands, at least to some measure, based largely on our interaction with Indian audiences. As authors of this book, we also well appreciate and acknowledge with great respect, the existence of several books on the broad subject of wavelets, multirate and multiresolution signal processing written by experts in the field. Writing this book posed another challenge – to bring just that uniqueness to this, to make it worthy of occupying a place on the shelves on which those other books are kept. We think that the main contribution of this book is twofold – (i) a text written to inspire a beginner in the subject with several guiding examples, illustrations and exercises and (ii) a reference for some advanced topics, explained in a manner that makes them follow smoothly from the more basic ones. The former aspect of the book will make it useful as instructional material, the latter, we hope, makes it attractive to a researcher in the subject. The book presents a thorough treatment of several concepts, along with simplified mathematical formulas. It also connects these mathematical ideas to the world around, thus giving physical significance to the material. The book also covers the third aspect of helping readers understand implementation strategies of the theory presented, by providing programs and codes, and making these available to readers. We hope that the explanation of concepts supported with many examples, tutorial exercises, codes, historical anecdotes, stories of scientists and demonstrations will make this book a unique reference material for instructors, students as well as researchers. Today, ‘wavelets’ has been replacing many traditional mechanisms in multiple applications, spanning several disciplines in engineering and science. For example, the compression standards formulated by the Joint Picture Experts’ Group (JPEG) saw an enhancement in the form of JPEG2000. Thus, from the year 2000, for compressing heavy satellite images, critical medical images and standard day-to-day images, JPEG 2000 has gained popularity. JPEG 2000 uses bi-orthogonal wavelet taps in lieu of the conventional DCT (Discrete Cosine Transform). Beyond this example, there are different walks of life

viii

Preface

where use of wavelets has become a powerful tool. The book starts with basic concepts and the coverage gets extended to latest trends in the field, with the likes of second-generation wavelets, wavelet packets, curvelets, brushlets, ridgelets, and so on. Chapter 1 introduces the subject to the readers and provides a motivation and foundation for the rest of the book. Chapters 2–6, which explains MRA (Multi Resolution Analysis) and exposes readers to Conjugate Quadrature Filters (CQF), in particular, the Haar and Daubechies families. These Chapters provide a detailed explanation with the help of many examples and helps readers understand, how looking at multiple resolutions simultaneously is possible and useful. Chapters 7–10 uncovers the time-frequency conflict, the uncertainty principle, the time-bandwidth product and allied topics such as the time-frequency plane and tiling in time frequency. Chapters 11–14 and every chapter is a variant of MRA, viz., Bi-orthogonal filters in Chapter 11, splines in Chapter 12, wavelet packets in Chapter 13 and the lifting scheme in Chapter 14. Whi!le Chapter 15 gives designs and salient features of about ten wavelet families, Chapter 16 talks about advanced topics, Chapter 17 discusses various applications, Chapter 18 takes the readers beyond the realms of ‘traditional ways’ in wavelets and the concluding section of Appendix provides extended notes on all chapters. The extended notes are particularly used for topics, which require further extensions to grasp the depth and comprehend the maturity of the topic. Writing any book is a journey. This journey has been unique in many ways. The thought of collaborating on this project started with Vikram Gadre’s course on Multirate Signal Processing and Wavelets, offered under the National Programme on Technology Enhanced Learning (NPTEL) of the Ministry of Human Resource Development (MHRD) Government of India. Aditya Abhyankar was actively associated with this course, initially as a reviewer and later as a Co-Instructor. Since then, we have been expanding the scope of the material through various workshops, interaction with relevant audiences, which included eminent academicians and, importantly, young, bright, enthusiastic students. In particular, when the ‘Knowledge Incubation under TEQIP’ Initiative of the MHRD (MHRD-TEQIPKITE) was launched at IIT Bombay, I (Vikram Gadre) was keen that we (I and Aditya Abhyankar) undertake this project as a collaborative endeavour involving IIT Bombay and a reputed University – the S P Pune University, which had reputed TEQIP Institutes like the College of Engineering Pune (COEP) in its circle in Pune. An attempt has been made to create content that will prove useful to the community. For many of the concepts written in this book, the inspiration has been interesting questions posed by colleagues and students in these interactions. A book, though a manifestation of the thought process of the authors, requires support at many levels from vivid backgrounds and this project has been no exception. We would first like to thank the Almighty, without whose blessings, nothing is possible. We would then like to thank the respective families for the everlasting support provided through this project. Vikram Gadre would like to thank his wife (Kaumudi Gadre), mother, brother and sister-in-law (Nirmala, Rohit and Vaishali Gadre), late father (Manohar Gadre), parents-in-law, brother-, sister-in-law and their daughter (the Pandit family), for their constant support, good wishes and blessings. He would also like to thank his supervisor (Professor R. K. Patney) and several other faculty members at IIT Delhi, who played an important role in shaping his interest and expertise in wavelets. He would also like to thank his colleagues and the administration at IIT Bombay, who then nurtured this interest and expertise further. Aditya Abhyankar would like to thank his father (Vidyavachaspati Shankar Abhyankar), mother (Aparna Abhyankar), brother (CA Jitendra Abhyankar), sister-in-law (Madhura Abhyankar), wife (Arati Abhyankar),

Preface

ix

niece (Arundhatee Abhyankar), nephew (Arjun Abhyankar) and son (Advait Abhyankar) for the best of love and support a person could ever ask for. Aditya Abhyankar would also like to thank his PhD advisor Dr Stephanie Schuckers and other professors who have been his source of inspiration. He would also like to thank Professors Raghunath Shevgaonkar and W N Gade, the former and current Vice Chancellors of SP Pune University respectively, for their constant encouragement. The authors would also like to thank many generations of our students at IIT Bombay and at SP Pune University, respectively for their assistance in the creation of digital material. There are just too many names that merit mention in this context but in particular, we would like to mention Gunjan and Sunil, Ph D students of Aditya Abhyankar, for the help in the preparation of the Supplementary Workbook, which was a daunting task. A special thanks to Professor P. P. Vaidyanathan of Caltech, USA, whose contributions to the field of multirate signal processing and wavelets are well known and respected worldwide but whose special role in this book, has been to inspire Vikram Gadre in many ways, to make the project see the light of day. Last but not the least; the authors would like to thank all the students, colleagues, and fellow scholars who have directly or indirectly provided inputs and feedback in making this manuscript more relevant. Vikram M Gadre Aditya S Abhyankar

Contents

Foreword Preface

v vii

1. Introduction 1.1

1

Introduction 1 1.1.1 Wavelets

3

1.2

Brief Outline of the Book

1.3

Multirate Digital Signal Processing 1.3.1 Filter Banks 8

1.4

Prerequisites 8 1.4.1 Generalized Vectors

1.5

6 7

8

1.4.2

Relationship between Functions, Sequences, Vectors

1.4.3

Need for Transformations

1.4.4

How Fourier Transform Works?

1.4.5

Fourier Transform and Non-stationary Signals

1.4.6

Inability of Simultaneous Time and Band Limitedness

1.4.7

Revisiting the Sampling Theorem

Summary Exercises

14 23 28

41 41

2.1

Introduction

2.2

The Haar Wavelet

2.3

Lk Norms of x(t)

45

45 49 49

Significance of L¥ Norm

50

2.4

Haar MRA (Multi Resolution Analysis)

2.5

Axioms of MRA

2.6

Theorem of MRA Exercises

26

37

2. The Haar Wavelet

2.3.1

10

100

56 56

50

Contents

xii

3. The Haar Filter Bank

107

3.1

Introduction

107

3.2

Function and Sequence

3.3

Downsampler

3.4

The Haar Filter Bank

3.5

Analysis Part

3.6

Synthesis Part

3.7

Frequency Domain Analysis of Haar Filter Bank

3.8

Frequency Domain Behaviour

108

111 113

114 117 120

3.8.1

Region of Convergence

3.8.2

Analysis Filter Bank

3.8.3

Second Filter in Analysis Filter Bank

Exercises

119

120 121 123

125

4. M-Channel Filter Bank

133

4.1

Introduction

133

4.2

Haar Analysis Filter Bank

4.3

Why is Haar Filter not Ideal?

4.4

Realizable Two Band Filter Bank

4.5

Relation Between Haar MRA and Filter Bank

133 134 137 137

4.5.1

Relation Between the Function f(t ) and Filter Banks

137

4.5.2

Relation Between the Function y (t ) and Filter Banks

138

4.6

The Strategy for Future Analysis

4.7 4.8

Iterated Filter Banks and Continuous Time MRA Iterating the Filter Bank for f(×), y (×) 139

4.9

Z-domain Analysis of Multirate Filter Bank

4.10 Two-channel Filter Bank

139

146

146

4.10.1 Analysis of Downsampler Operation

147

4.10.2 Analysis of Upsampler Operation 4.10.3 Z-Domain Effect of Upsampling

148 149

4.10.4 Z-Domain Effect of Downsampling 4.11 M-band Filter Banks and Looking Ahead 4.12 3-band Filter Bank (Ideal)

149 153

153

4.12.1 Analyzing an Ideal 3-band Filter Bank 4.12.2 Interpretation of 3-band Filter Bank Exercises

160

139

154 156

Contents

xiii

5. Conjugate Quadrature Filter Bank

165

5.1

Introduction

5.2

Z-domain Analysis of Filter Bank

5.3

Aliasing Cancellation

5.4

Perfect Reconstruction: Conjugate Quadrature

5.5

Condition for Perfect Reconstruction

5.6

Polynomial as an Input

5.7

Conjugate Quadrature Filter Bank Exercises

165 166

170 171

172

174 176

181

6. Daubechies Family

187

6.1

Introduction

187

6.2

Impulse Response of Daubechies Analysis Low Pass Filter

6.3

Calculation of Scaling Function

189

6.3.1

Interpretation for Daubechies Filter Banks

6.3.2

Next Daubechies Family Member Daub-6

191 191

6.4

Daub-4 and Daub-6 Design Details

6.5

Towards Searching Scaling and Wavelets Coefficients Exercises

187

192 206

211

7. Time-Frequency Joint Perspective

215

7.1

Introduction

7.2

Analysis of Haar Scaling Function in Time and Frequency Domain

215

7.3

Analysis of Haar Wavelet Function in Time and Frequency Domain

220

7.4

Summary

7.5

Ideal Time-Frequency Behaviour

7.6

Frequency Localization by f(t) and Y(t)

7.7

Time Localization and Frequency Localization Exercises

215

223 223 223 227

229

8. The Uncertainty Principle

237

8.1

Introduction

237

8.2

Non-formal Introduction to the Idea of Containment

8.3

Formalization of the Idea of Containment

8.4

Self Evaluation Quiz Examples

245

238

237

Contents

xiv

8.5

The Time-bandwidth Product 8.5.1

245

The Time Centre t0

246 2 t

8.5.2

The Time Variance s

8.5.3

The Frequency Centre W0

8.5.4

246

The Frequency Variance s

246 2 W

246

8.6

Signal Transformations

8.7

Properties of the Time Bandwidth Product 8.7.1

8.8 8.9

246 250

Simplification of the Time-bandwidth Formula

250

Evaluating and Bounding the Time-bandwidth Product 2 2 Evaluation of Time-bandwidth Product s t s W 251

251

8.10 Self Evaluation Quiz Examples Exercises

256

258

9. Time Frequency Plane and Tilings

271

9.1

Introduction

271

9.2

Short Time Fourier Transform (STFT) and Wavelets

279

9.2.1

280

Duality Interpretation of Fourier Transform

9.3

Reconstruction and Admissibility

282

9.4

Reconstruction from STFT

9.5

Time Frequency Tiling: Comparison of STFT and CWT

9.6

Reconstruction from CWT

9.7

Admissibility in Detail and Discretization of Scales

9.8

Admissibility

9.9

Case-III W = 0 and Case-IV W = ¥

282 284 286

286

9.10 Analysis of Gaussian Function 9.11 Discretization of Scale

288 289

291

9.11.1 Condition of Scale Discretization Exercises

283

291

292

10. Dyadic MRA 10.1 Introduction

301 301

10.2 Sum of Dilated Spectra (SDS)

306

10.3 The Theorem of (Dyadic) MRA 10.4 Bi-orthogonal Filter Bank 10.5 Orthogonal Filter Bank

308

308 309

10.5.1 Construction of Orthogonal Filter Bank

309

Contents

xv

10.6 Dyadic Multiresolution Analysis

310

10.6.1 Axioms of a Dyadic MRA

312

10.7 Theorem of Multiresolution Analysis

313

10.8 Proof of Theorem of Dyadic MRA

313

10.9 Proof of Theorem of Dyadic MRA

314

Exercises

318

11. MRA Variant 1: Bi-orthogonal Filters 11.1 Introduction

327

327

11.2 Inner Product of Wavelet Function y (t ) and Scaling Function f(t – m) 11.3 Variants of MRA

329

11.4 Introduction of Bi-orthogonal Filter Banks

330

11.5 JPEG 2000 5/3 Filter-bank and Spline MRA 11.6 Description 11.7 Note

332

333

337

11.8 5/3 Tap in Nutshell

339

11.9 Filter Design Strategy Exercises

339

351

12. MRA Variant 2: Splines 12.1 Introduction

361

361

12.2 5/3 Filter Bank

361

12.3 Derivation of Orthogonal MRA

362

12.4 Building Piece-wise Linear Scaling Function, Wavelet 12.5 Building of Piece-wise Linear Scaling Function 12.6 Summary Exercises

367

367

371 372

13. MRA Variant 3: Wave Packets 13.1 Introduction

377

377

13.2 The Wave Packet Transform

378

13.3 Notion of Frequency Inversion in Highpass Filtering 13.4 Reconstruction of Input X 0 from Wave Packet Transform 13.5 Noble Identities

385

13.6 Haar Wavepacket Transform

388

13.7 Wavelet Packets: Framework

389

Exercises

327

398

382 384

Contents

xvi

14. MRA Variant 4: Lifting Scheme 14.1 Introduction

407

407

14.2 The Lattice Structure for Orthogonal Filter Banks 14.3 Lattice Structure and Its Variants 14.4 The Lattice Module

407

412

412

14.5 Inductive (Recursive) Lattice Relation 14.5.1 Obtaining km+1 14.6 The Synthesis Variant

413

413 416

14.7 Efficient Deployment Schemes for Lifting Structures 14.8 The Lifting Structures and Polyphase Matrices

417

14.9 The Polyphase and the Modulation Approach 14.10 Towards Building Polyphase Structures 14.11 Polyphase Approach

417

421

422

425

14.12 Modulation Approach

428

14.13 Modulation Analysis and 3-band Filter Bank, Application 14.14 Final Step in Polyphase Approach 14.14.1 Summary of the Method 14.15 Modulation Approach Exercises

429

429 430

430

433

15. Other Wavelet Families 15.1 Introduction 15.2 Coiflets

439

439 440

15.2.1 Design Strategy for Coiflet Filters 15.3 Symlet Filters

449

15.4 Morlet Filters

451

15.4.1 Morlet Wavelet Design Strategy 15.5 Mexican Hat Filters 15.6 Meyer Filters

444

455

458

459

15.7 Battle-Lemarie Wavelets – Orthogonalization of the B-Splines 15.7.1 Battle-Lemarie Design Details 15.8 Gabor Filters

464

15.9 Shannon Filters

464

15.10 Bi-orthogonal Filters 15.11 Summary Exercises

479 481

467

463

461

Contents

xvii

16. Advanced Topics 16.1 Introduction

483 483

16.2 Fast Wavelet Transform

483

16.3 The Daubechies Fast Wavelet Transform (FDWT) and its Inverse (IFDWT) 16.4 Low and High Pass Effects: Dual Filters

486

497

16.5 Numerical Computations for the Fast Discrete Wavelets Transform and its Inverse 500 16.6 Multi-dimensional Wavelets

515

16.6.1 Separable Tensor Product Wavelets 16.6.2 Non-Separable wavelets

515

518

16.7 The Two-Dimensional Haar Wavelet Transform 16.7.1 The Basis Transformation 524 16.7.2 The Decomposition in 2D Sense

520

525

16.7.3 Working with Bigger Arrays (4 ´ 4 toy example) 16.7.4 Inverse Transform: Matrix Notation 16.8 Wavelets and Self-Similarity

529

533

542

16.9 Wavelet Transform: In Perspective of Moving from Continuous to Discrete 16.9.1 Overcomplete Wavelet Transform 16.10 Redundant Discrete Wavelet Transform 16.11 Regularity and Convergence 16.11.1 Convergence Exercises

544 544

545

545

546

17. Wavelet Applications 17.1 Introduction

544

547

547

17.2 Application: Face Recognition Using Wavelet Packet Analysis 17.3 Application of Wavelets in Data Mining

549

17.4 Application of Wavelets in (Big) Data Analysis

550

17.5 Face Recognition Through Wavepacket Analysis 17.6 Wavelet Based Denoising – Application

547

554

558

17.6.1 Wavelet Based Denoising for Suppression of Respiratory Artifacts in Impedance Cadiogram Signals (ICG) 558 17.6.2 Impedance Cardiography (Definition) 17.6.3 Structure and Functioning of Heart 17.6.4 ICG Signal and Artifacts 17.6.5 Application Objective

560 561

558 558

Contents

xviii

17.6.6 Wavelet Based Denoising 17.6.7 Denoising Results

562

565

17.6.8 Possible Embellishments

566

17.7 Finding Hidden Discontinuity – Application 17.8 Denoising the Underlying Signal – Application

566 568

17.9 Compression and Pattern Recognition Using Wavelets 17.10 JPEG 2000 Filter Strategy

577

17.11 Two Interesting Techniques SPIHT and EZW Exercises

18.1 Introduction 18.2 Ridgelets

583

583 584

18.2.1 Digital Ridgelet Transform 18.3 Curvelets

588

18.4 Brushlets

589

18.5 Contourlets 18.6 Bandelets 18.7 Platelets

585

592 592 592

18.8 (W)Edgelets

592 592

18.10 Hilbert-Huang-Transform (HHT) 18.10.1

580

580

18. Going Beyond the Realms

18.9 Shearlets

571

Hilbert-Huang Analysis

18.10.2 Wavelet Packet Transform 18.10.3 Wavelet Based Sieving

593 593 594 595

Appendix: Extended Notes

599

References and Bibliography

697

Index

706

Chapter

1

Introduction

Introduction Brief outline of the book Multirate digital signal processing Prerequisites Summary

1.1

Introduction

This chapter introduces the subject of wavelets and multirate digital signal processing. It provides an inspiration to understand this subject at a greater depth along with broader picture of joint timefrequency analysis (JTFA). The subject of wavelets follows a basic exposition to subjects like signal analysis, system theory and digital signal processing. It could be considered as an advanced topic on signal processing, however, this does not imply that the concepts introduced in this book are difficult to understand. In fact, the concepts are easier than a basic course on signal analysis. The basic text introduces the idea of abstraction, i.e. abstraction of signals, systems, transforms, analysis in different domains, etc. On the other hand, wavelets bring us closer to reality. We call it a journey from ‘romanticism’ to ‘realism’! Being practical, pragmatic and realistic, use of wavelets not only requires good understanding of fundamentals but also practical constraints. In this sense, the contents covered in this book are very ‘hands-on’. Wavelet analysis provides more precise information about signal data than other signal analysis techniques, such as Fourier because of which this branch of signal processing is growing very rapidly. In a basic text on digital signal processing, we assume that the signals last forever, i.e. for infinite duration of time. For example, while calculating the Fourier transform, we represent any signal in terms of basis functions and these basis functions last from t = -¥ to t = +¥, where t denotes time. However, practically no signal in this world can last forever. Thus, we should deal with signals in finite domains.

Example 1.1.1 — Audio signal.

We understand finite domains very well, if that finite domain is the natural domain. To reflect more on this, consider an example of a piece of an audio signal which is finite in time. Here, time is the natural domain. From a signal processing perspective, we wish to find the content in that audio signal, by enhancing some parts of that signal and suppress others. We may even be interested in characterizing the system. But for doing all these things, we deal with finite time signal and the abstraction of everlasting signals is unnecessary.

2

Multiresolution and Multirate Signal Processing

Example 1.1.2 — 2D face.

Consider another example, in which we have a picture of a ‘face’. In this case, the natural domain is space and it is 2-Dimensional. The face has various features like eyebrows, forehead, nose, lips etc. Suppose we wish to isolate a particular feature, say an eye. This requires localization in the spatial domain. Here, again, the amount of data is finite reflecting the finite domain signal. This is shown in Fig. 1.1, where the face image has 3072 columns and 2304 rows and thus 3072 × 2304 number of (finite) pixels per palette. The picture is made up of rows and columns, in other words it gets represented in 2D space and, hence, spatial isolation can be used isolate a particular feature.

Figure 1.1

Face Example: Isolation is spatial domain

Example 1.1.3 — Musical signal.

Another example which explains localization is a piece of audio in which a number of notes are sung. It may be called as a ‘raga’ in Indian tradition and the notes may be called as the components of the raga. Now, we aim to make a system that takes the rendition of this ‘raga’ and identifies the notes that compose it. To achieve this, we need to segment the signal in time. For example, the first note may be played for 1 second, the second note for 0.5 seconds, and so on. This demands segmentation in time. Moreover, we should also understand that all the notes are not of fixed lengths. The length of the time segment is also important. But more important is to understand the concept of ‘Notes’ in the signal processing context. For example, Notes of raga ‘Gujari Todi’ are depicted in Fig. 1.2.

Introduction

3

Sa

Figure 1.2

Re

Ga

Ma

Dha

Ni

Sa

Sample Notation: Frequencies of raga ‘Gujari Todi’

In the basic course of “Signals and Systems”, we were exposed to the idea of frequency domain. We know that signals have embedded inside them, as collection of sine waves. These sine waves are continuous for continuous time signals while they are sampled for discrete sequences. The continuous signals look for superposition of these sine waves whereas summation of sampled sine waves is observed in discrete case. Thus, most reasonable signals can be thought of as a collection of sine waves. In principle, if the signal is not periodic, its fourier transform comprises infinity of sine waves whose frequency ranges from 0 to ¥. For periodic signals, we have a discrete set of sine waves with possibly finite or infinite range of frequency represented by Fourier series. Thus, a different domain is more useful to analyze the signal. Now if we query about the ‘Notes’ in the raga, it is equivalent to asking the frequency content in the audio piece, i.e. what points on the frequency axis are occupied by this note? Which are the locations where the transform is prominent? It will also be of interest to analyze to understand exact locations of these frequencies on time-axis. The ‘analyzer’ often wishes to know at what ‘time’ which ‘frequency’ existed. This demands the analysis building blocks to have simultaneous time and frequency localization capabilities. This is one of the most fundamental inspirations to study wavelets, and joint time frequency analysis (JTFA) in broader perspective. Before continuing with the above example and other concepts, it is worthwhile to introduce the term ‘wavelets’.

1.1.1

Wavelets

Fourier transform deals with sine waves. Sine waves have many important properties. Firstly, they occur naturally. For example, an electrical engineer recognizes sine wave as naturally emerging from an electricity generation system. Secondly, sine waves are the most analytic, the smoothest possible periodic functions. They also have the power to express many other waveforms, i.e. they form a very good basis. Addition of two sine waves of the same frequency but with possibly different amplitudes and phases, gives a sine wave of the same frequency with possibly different amplitude and phase. A sine wave on differentiation or integration is a sine wave of the same frequency. Any linear combination of all these operations on a sine wave results in a sine wave of the same frequency. But the biggest drawback of sine waves is that they need to last forever. If the sine wave is truncated (a one sided sine wave, for example), the response to this signal by a system, in general, is different from the response which would be obtained if the signal would be a sine wave from t = - ¥ to t = + ¥. There would be transients which are not periodic. All the useful properties mentioned above, will then no longer be valid. So, if we need to apply the principles studied in a basic course, we need something unrealistic, i.e. a sine wave which lasts forever. To be more realistic in our demands, it is appropriate to deal with wavelets rather than waves. Wavelets are waves that last for a finite time, or more appropriately, they are waves that are not predominant forever. We let the waves die out to create a compactly supported ‘wavelet’ of finite duration. They may be significant in a certain region of time and insignificant elsewhere

4

Multiresolution and Multirate Signal Processing

or they might exist only for finite time duration. For example, a sine wave that exists only between t = 0 and t = 1 m sec is, in principle, a wavelet (though not a very good one), a wave that doesn’t last forever. Thus, a wavelet is a mathematical function useful in digital signal processing and image compression. In signal processing, wavelets make it possible to recover weak signals from noise. In subsequent topics we will see that concept of wavelets arose when scientists debated on a serious issue of “time-frequency localisation”. Example 1.1.4 — Musical signal revisited.

Now, we go back to the example of audio clip. The audio clip comprises of many notes of varying time lengths and hence requires time segmentation. On the other hand, identifying notes in a particular time segment, involves segmentation in the frequency domain. Thus we are asking for a simultaneous localization in time and frequency domain. But the uncertainty principle in nature puts constraints on this simultaneous localization beyond a point. In signal processing, we call it uncertainty in time and frequency domain i.e. both domains are contrary to each other. Thus, the resolution in the time domain is increased at a compromise in the resolution in the frequency domain. It could be even intuitively argued that, for a shorter audio clip, it is more difficult to identify a note in that time segment than a note played for a longer period of time. But, what is not intuitive from this discussion is that we cannot go down to identifying one particular frequency precisely. So, if we wish to come down to a point on the time axis, we need to spread all over the frequency axis and vice versa. This is the stronger version of this principle. However the weaker version is more subtle. Even if we select a time region on the time axis and ask for the region of frequencies which are predominant in that time region, even then there is a restriction on the simultaneous length or measure of the time and frequency regions. In fact, the more we focus in time, the less we focus in frequency. This could be best explained by examples.

Example 1.1.5 — Mobile system.

Consider a mobile communication system in which a bit stream is transmitted at 1Mbps. If the bit interval is uniform, the time interval for 1 bit is 1 msec. This indicates segmentation in time. Now, consider that there are two mobile operators operating in a given region. All the users in the region are using mobile from any one of the two operators. To avoid interference or overlapping of these signals, these operators should be separated in some domain. They cannot be separated in the time domain since the users of different operators can use the mobiles at the same time. So the separation could be in frequency domain. Thus, every operator is allocated a particular bandwidth, i.e. a region in the frequency domain so that the users of that operator can operate in that frequency region only without any problem of time coincidence. This is indicative of segmentation in frequency. Thus, in a mobile communication case, there is a desire to localize in time and frequency simultaneously, i.e. transmitting a bit in a time interval of 1 msec, indicating a localization in time and transmitting in a particular frequency region only indicating localization in frequency. Thus, this example clearly depicts practical need of simultaneous localization in time and frequency.

Introduction

5

Example 1.1.6 — Biomedical signal.

Consider another example of a biomedical signal, say ECG signal (electro-cardiographic waveform) in which various features of the ECG signals are analysed. There are various segments in a typical ECG signal which are often indexed by letters say P, Q etc. as shown in Fig. 1.3. These segments are of unequal length, e.g. from Fig. 1.3, it can be seen that interval PR is much shorter than interval QT. In fact, biomedical engineers often talk about what are called as evoked potentials. They give stimulus to a biomedical system and evoke a response and the waveform corresponding to that response is called an evoked potential. An evoked potential typically has both fast and slow-varying parts in the response. Obviously, the slower parts of the response are predominantly located in the lower ranges of the frequency region while the quicker parts of the response are predominantly located in the higher ranges of the frequency. To isolate the quicker parts of the response, is it sufficient to pass the signal through a conventional high pass filter? Here arises the time frequency conflict. Indeed, it is not sufficient. In fact, we need a different perspective on filtering. We need to identify, in different parts of the time axis, which regions of the frequency axis are predominant and then identify different parts of the frequency axis that need to be emphasized in different time ranges. This is yet another example of time frequency conflict. In a basic course, we understand the domains very well because we keep them apart. But one normally needs to consider the two domains together and when we try to do so, there is a fundamental conflict as understood from the uncertainty principle.

QRS Complex R

ST Segment

P PR Wave Segment

PR Interval

T Wave

Q S QT Interval

Figure 1.3

ECG Signal Sample

6

Multiresolution and Multirate Signal Processing

Struggling with the concept of simultaneous time frequency localisation scientists came to a solution or more like a workaround. Perhaps if a signal is split into components that were not pure sine waves, it would be possible to condense the information in both time and frequency domains. This was the idea which ultimately led to the concept of wavelets. Rect(t)

sinc(f)

2A

FOURIER TRANSFORM

A

A/f

t + inf

− inf −t

+t −f

Figure 1.4

+f

Fourier Transform of Rectangular ‘box’ type function with crisp and compact time domain presence is a ‘sync’ function with spread and no isolation in frequency domain

Figure 1.4 clearly depicts the limitation of using sinusoidal basis functions that simultaneous show isolation in time as well as frequency domain is not possible. In fact, as the figure depicts, as the time domain representation is compact in the form of ‘box’ type rectangular function its Fourier Transform spreads out and has no isolation in frequency domain (‘sync’ function).

1.2

Brief Outline of the Book

This book begins with an introduction to the Haar Multiresolution Analysis a particular tool to analyze signals; proposed as a dual of the idea of Fourier analysis by Haar, a French mathematician. In Fourier analysis, we represent even discontinuous or non-smooth waveforms into a linear combination of extremely smooth functions, namely, the sine waves. Haar proposed the idea of taking smooth functions and convert them into a linear combination of effectively discontinuous functions. For example, data in a digital communication system, e.g. an audio signal, image, or a video, is transmitted with a large level of discontinuity. To record a digital audio, we first sample the signal followed by digitizing and then recording it. All these are highly discontinuous operations, in which not only forcibly introduce discontinuity in time but also in amplitude. Thus, representing a smooth audio signal into a discontinuous bit stream is very beneficial to digital communication system and a digital recording is even better than an analog recording. Therefore, the first few chapters of this book will look at wavelets and multirate digital signal processing based on the principles that Haar propounded. Understanding Haar multiresolution analysis (MRA) in depth leads to better understanding of many of the principles of wavelets and multirate processing, specifically the two-band processing. After presenting material on Haar MRA, the sequence of presentation in the book would be as follows: ●●

●●

The Daubechies family of multiresolution analysis (Daubechies is the name of the mathematician who proposed this family of multiresolution analysis). The uncertainty principle, fundamentally and in terms of its implications.

Introduction ●●

●●

●●

●●

1.3

7

The continuous wavelet transform (CWT). In the Haar multiresolution analysis, we have a certain discretization in the variables associated with the wavelet transform. In the continuous wavelet transform, the variables become continuous. Some of the generalisations of the ideas like ‘wave packet transform’, variants of wavelet transforms like next generation wavelets through lifting scheme, etc. In the last parts of the book, we shall look at some of the important applications, where wavelets and multirate digital processing provide great advantages. A special chapter also provides material on the scope beyond wavelets.

Multirate Digital Signal Processing

Let us look at some of the developments in the subject of multirate signal processing. The connection with wavelets will also be seen. Consider the biomedical example discussed previously. The biomedical signal has fast-varying parts and slow-varying parts of the response. The slow-varying parts of the response are likely to last for a longer period of time while the quicker parts of the response are likely to last for a shorter period of time. So, apart from considering localization in time and also in frequency, the localization required for higher frequencies and lower frequencies is also important. Lower frequencies in response have lower time resolution. Resolution means the ability to resolve, the ability to be able to identify specific components. How can the frequency axis be narrowed down? Frequency resolution relates to being able to identify the specific frequency components. More appropriately, how much do we need to narrow down? It is often desired (not always) that the higher frequency components be compromised on frequency resolution but not on time resolution. So things that are transient, demand time resolution and things that occupy lower frequency ranges, demand frequency resolution. Hence for increasing frequencies, more frequency resolution is needed opposed to time resolution. This brings the idea of multirate processing. So, if we have higher frequencies, we should use smaller sampling interval and vice versa, in a discrete processing system. This leads to more efficient processing operations. In an evoked potential response, frequent sampling is not required for the lower frequency components. It increases data without any advantage. On the other hand, while handling quicker components, if our sampling rate is less or inadequate, ‘aliasing’ is introduced. Hence, it is not a good idea to use the same sampling rate for all the frequency components. Unlike a basic course, we deal here with sequences that are obtained with different sampling rates in the same system. This brings the idea of multirate digital signal processing. Thus, in the analysis of same system we need to have different sampling rates according to demand of resolution and the conservation of data. Another important aspect which triggers use of multirate systems is the way conventional transforms like Fast Fourier Transform (FFT) divides the frequency range. While human ears perceive frequencies in logarithmic sense, FFT provides equal prominence to every frequency bin, thus making frequency segmentation ‘linear’. Example 1.3.1 — Linear segmentation of FFT.

Human hearing is sensitive to typical range of 20 Hz to 20 KHz. Female speech typically ranges from 140 Hz to 500 Hz, male speech typically ranges from 70 Hz to 250 Hz, flute produces frequencies from 260 Hz to 3350 Hz, Violin produces from 200 Hz to 3000 Hz and so on. Often we have to process and analyze various ranges of frequencies. Due to linear segmentation of frequencies using

8

Multiresolution and Multirate Signal Processing

FFT, to record high frequency data accurately the compromise is made on low frequency resolution. Thus, in general, FFT has excellent high frequency pitch resolution but poor low frequency resolution. For example, let’s say we are processing a signal for which Nyquist stability criterion is followed and signal is sampled at appropriate sampling rate. When FFT frame size is divided by sampling rate it gives us frequency window. For a particular signal under analysis, let us assume the frequency window be 50 Hz. This could be very small analysis window for fast moving (high frequency) signal but could be a big window for slow moving (low frequency) signal, which may prevent FFT to capture good details at low frequency. The natural intuitive solution to this is to have different sampling rates for different parts, which leads us to multirate systems.

1.3.1

Filter Banks

Going further, let us construct the idea of filter banks, which is quite important for the analysis and synthesis of the signals. In a biomedical example, to separate components, many different operators have to be simultaneously used. So a system of filters is needed, which has certain individual characteristics as well as collective characteristics. Thus, analysis as well as synthesis is required. In addition to that, we also require localization. So a bank of filters, refers to a set of filters which either have a common input or a common point of output. This concept of a bank of filters, (in fact, two banks of filters) namely analysis and synthesis filter banks, taken together, is very central to multirate signal processing. So a two-band filter bank will be presented in this book at a greater depth. The concept of two-band filter bank is of great importance in constructing wavelets. The subsequent chapters and discussions about Haar multiresolution analysis will bring out the intimate relationship, which exists between the Haar wavelet and a two-band Haar filter bank.

1.4

Prerequisites

The subject of ‘Wavelets’ has many interesting features and facets. It uses different approaches for representation of signals and functions than conventional techniques. In this section we will try to build physical significance to bring out concepts from underlying mathematical formulas. The readers who are well versed with the fundamental concepts may skip the reminder of the chapter.

1.4.1

Generalized Vectors

A vector quantity or vector, provides the magnitude as well as the direction of a specific quantity. A twodimensional vector is generally represented in terms of two co-ordinates along two perpendicular axis. This idea of perpendicularity is intimately related to the idea of independence. Let us look at a system of vector representation: System representation When giving directions to a point, it is not enough to say that it is x miles away, but the direction of those x miles must also be provided for the information to be useful. (Note that physical quantities are represented by Scalars, such as temperature, volume and time, etc.)

Introduction

9

v2 v2 v = v1uˆ1 + v2uˆ2 v = v1uˆ1 + v2u ˆ2 + v3uˆ3

0

2 - dimensional

0

v1

3 - dimensional

v1

v3

Figure 1.5

Graphical representation of vectors

Given a coordinate system in three dimensions, a vector may thus be represented by an ordered set of three components, which represent its projections v1 , v2 , v3 on the three coordinate axes. v = [ v1 , v2 , v3 ] The three most commonly used coordinate systems are rectangular, cylindrical, and spherical. Alternatively, as shown in Fig. 1.5, a vector may be represented by the sum of the magnitudes of its projections on three mutually perpendicular axes: v = v1uˆ1 + v2 uˆ2 + v3uˆ3 The n-dimensional coordinate systems based on the Euclidean space (Cartesian space or n-space) represented by R n or E n , under n-dimensions and n-vectors. Usually, the Euclidean space is formed by ( X1 , X 2 , X3 ,..., X n ) where n is equal to 8. Example 1.4.1 — Parallelogram law of vector.

Let us take an example. Refer Fig. 1.6, we shall see how we can get the resultant of two vectors by the use of the parallelogram law of vector addition. Given two vectors v1 and v2 , their resultant is given by the diagonal of the parallelogram as shown in the Fig. 1.6. v = v1 + v2 where v1 = k1uˆ1 and v2 = k2 uˆ2 then v = k1uˆ1 + k2 uˆ2

10

Multiresolution and Multirate Signal Processing

ˆ2 u

uˆ1

Figure 1.6

1.4.2

Parallelogram law of vectors

Relationship Between Functions, Sequences, Vectors

As we may recall, one can intimately relate processing of a function to processing of equivalent sequence, and retrieving information from or modifying a function can be done equivalently by processing or modifying that sequence corresponding to function. A sequence is like a vector and each n (corresponding to a value in the sequence) is a different dimension of that vector. Now, once this analogy is clear, it is very easy to extend other ideas of vectors to this context. For example, we can think of vector addition as addition of the sequences point by point, and so on. An infinite (countably infinite) dimension vector is a sequence x[ n], n Î , where n is index and  is set of integers. Now, we would like to extend other ideas of vectors to this context of infinite dimension vector. Dot product of vectors Let, e1 = e11uˆ1 + e12 uˆ2 and e2 = e21uˆ1 + e22 uˆ2 then dot product is e1 e2 = e11e21 + e12 e22 . It can be easily seen that this is nothing but sum of products of corresponding coordinates. Let two n-dimensional vectors be e1 : e11 , e12 ,..., e1N and e2 : e21 , e22 ,..., e2 N the dot product of these two vectors is N

á e1 e2 ñ = åe1k e2 k . These are also called as orthogonal coordinates. k =1

Let, in two sequences, say x1[ n], x2 [ n], n Î Z , the ‘dot product’ or ‘inner product’ be á x1 , x2 ñ , where á x1 , x2 ñ =

+¥

å x [ n] x [ n] 1

n = -¥

2

Introduction

11

The word ‘magnitude’ is generally used in the context of 2 - D,3 - D space; but in a general n - D space, we will use the word ‘norm’. In L2 () , norm squared represents the energy. Let vector x: essentially a sequence x[ n], n Î Z , then the ‘norm’ of sequence x = x  should be á x1 , x2 ñ1/2  x ³ 0 and  x = 0 iff x = 0 i.e. x[ n] = 0 " n Î Z If x1 and x2 are real,

+¥

å x [ n] x [ n]

á x1 , x2 ñ =

1

2

n = -¥ +¥

å x [ n]

á x, x ñ =

2

n = -¥

As long as x[ n] is real "n Î  , this will satisfy norm requirements. A small change will be applied for complex sequences as follows +¥

å x [ n] x [ n]

á x1 , x2 ñ =

1

2

n = -¥

Properties of Inner product 1. Conjugate commutativity á x1 , x2 ñ = á x2 , x1 ñ +¥

=

å x [ n] x [ n] 1

2

n = -¥ +¥

=

å x [ n] x [ n] 2

1

n = -¥

á x1 , x2 ñ = á x2 , x1 ñ 2. Linear in first argument á a1 x1 + a2 x2 , x3 ñ = a1 á x1 , x3 ñ + a2 á x2 , x3 ñ +¥

å (a x

=

1 1

+ a2 x2 ) x3

n = -¥ +¥

=

å a (x x ) + a (x x ) 1

1 3

2

2 3

n = -¥

á a1 x1 + a2 x2 , x3 ñ = a1 á x1 , x3 ñ + a2 á x2 , x3 ñ 3. Positive definiteness or non-negativity á x, x ñ =

+¥

å x[n]. x[n]

n = -¥

12

Multiresolution and Multirate Signal Processing

á x, xñ = 0; iff

x[ n] = 0 "n

This inner product is also called the ‘standard inner product’. In the chapters henceforth when we say inner product of sequences we shall be talking about the ‘standard inner product’ unless specified. Extension to uncountably infinite dimension For any ‘t’, t Î  is a different dimension and x(t ), t Î , means x(t ) for the ‘t th’ coordinate. Then the ‘dot product’ or ‘inner product’ between two functions x(t ) and y(t ) in general is given by +¥

á x, yñ = ò x(t ) y(t )dt -¥

The properties of conjugate commutativity, linearity in the first argument and positive definiteness are also valid for this case and its verification is left as an assignment to the reader. Parseval’s Theorem The Parseval’s theorem states that the inner product of any two functions in time domain is equal to the inner product of those two functions in frequency domain. Let x(t ) be a function then its fourier transform, x (n ) or x (W) (in Hz or in radians respectively) is defined as +¥ +¥ x (n ) = x(t )e- j 2pnt dt or x (W) = x(t )e- jWt dt where W = 2pn

ò

ò

-¥

-¥

From now on we shall use ‘W’ to represent the angular frequency variable in the continuous time and ‘n ’ to represent the Hertz’ frequency variable. What we are referring to shall be clear from the context. Let y(t ) be a function and y (n ) or y (W) its fourier transform (in Hz or in radians) defined as +¥ y (n ) = ò y(t )e- j 2pnt dt or -¥

+¥ y (W) = ò y(t )e- jWt dt -¥

where W = 2pn

The inner product of these in time domain is +¥

á x, yñ = ò x(t ) y(t )dt -¥

and it is equal to the inner product in frequency domain given by +¥ á x , y ñ = ò x (n ) y (n )d W -¥

That means á x, yñ = á x , y ñ The function x(t ) can be reconstructed from its frequency components as +¥ x(t ) = ò x (W)e- jWt d W -¥

Parseval’s theorem basically states that the inner product is independent of the co-ordinate system. So, whether we choose to represent two functions in the standard co-ordinate system of ‘time’ or the slightly less obvious co-ordinate system of ‘frequency’, their inner product always remains the same.

Introduction

13

Example 1.4.2 — Applications of Parseval’s theorem.

The Parseval’s theorem is often used in many areas like physics and engineering etc, and it is written many of the times as +¥ +¥ | x(t ) |2 dt = | x (n ) |2 dn

ò

ò

-¥

-¥

where x (n ) represents the continuous Fourier transform of x(t ) and ‘‰ ’ represents the frequency component of x . From this equation, the theorem states that the total energy contained in a function x(t ) over all time ‘ t ’ is equal to the total energy of the its Fourier Transform x (n ) overall frequency ‘n’. For discrete time signals, the theorem becomes: +¥

1

å | x[n] | = 2p ò 2

+p

-p

n = -¥

| x (e jw ) |2 dw

where x (e ) is the Discrete-Time Fourier transform (DTFT) of x and ‘w’ represents the angular frequency (in radians per sample) of x . For the Discrete Fourier transform (DFT), the relation becomes: jw

N -1

1

N -1

å | x[n] | = N å | x[k ] | 2

n =0

2

k =0

where x[ k ] is the DFT of x[ n] and ‘ N ’ is length of sequence in both domains. Relation between continuous functions and sequences Let x(t ) be a continuous function and let f (t ) be a unit step function in [0,1] interval, then x(t ) can be written as x(t ) = .... + C-1f (t + 1) + C0f (t ) + C1f (t - 1) + C2f (t - 2) + .... It can be graphically represented, as shown in Fig. 1.7. Through successive approximations as one of the ways of representing signal in interval used, signal x(t ) can be represented as sequence x[ n] as x[ n] = {.C-1 , C0 , C1 , C2 ,..} . Equivalance between continous funcitons and sequences will be dealt in greater detail in subsequent chapters C−1

C0

C1

C2

C3

x(t)

−1

Figure 1.7

0 1 2 3 4 Relation between continous functions and sequences

14

Multiresolution and Multirate Signal Processing

1.4.3

Need for Transformations

There are many reasons for carrying out a transformation, most siginificant one is convenience. For example, we do not understand music as just few time domain signals with varying voltage, but as sequence of frequencies. Thus, it makes sense to transform musical signal into frequency domain and then the analyzer will be more comfortable dealing with those frequencies. Ultimately, we want to design filters, more pertinently filter banks in the context of wavelets, and it is more convenient to design filter banks in frequency domain compared to time domain or spatial domain. Wavelet transform is strikingly different than most of the conventional transforms. For all conventional transforms like Laplace transform, Z transform, Fourier Transform and Logarithmic Transform the basis function comes from natural logarithmic base e. However, for understanding wavelet transform we have to go beyond the purview of these conventional transforms. We already know how to analyze Linear Time Invariant (LTI) Systems. For doing so, traditionally, we have two methods, 1. Convolution 2. Difference Equations Convolution evolves out of the fact that given a signal x[n], we can decompose that signal into scaled and shifted impulse sequences. After doing so, we can write, x[ n] =

¥

å x[k ]d [n - k ]

(1.1)

k =-¥

This is the scaled summation of shifted impulses. We are able to do this because the system is assumed to be linear and shift invariant. Since the system is linear and thus would follow superposition theorem we are able to add up the products to get x[n]. Also, because of time invariance property we are able to shift the impulses without affecting the end result. Once we do this and understand how the system reacts to impulses as stimulus, we would be able to get the systems impulse response. Now if we call the impulse response as h[n], excite the LTI system jw n with an exponential e 0 as an input x[n] and call the output as y[n] we would have, ¥

y[ n] = åe

jw 0 n

h[ n - k ]

(1.2)

-¥

However we know by commutative law, y[ n] = Thus by putting e

jw 0 n

¥

¥

k =-¥

-¥

å x[k ]h[n - k ] = å x[n - k ]h[n]

(1.3)

in place of x[n] we have, ¥

y[ n] =

åe

jw 0 [ n - k ]

h[ n]

(1.4)

k = -¥

This forms an interesting eigen system for LTI analysis. That is because we can split the exponential jw n term from Eq. (1.4) and then the summation happens for variable k and the term e 0 can be deduced out of this summation of k terms. ¥

y[ n] =

åe

k =-¥

jw 0 [ n ]

e

jw 0 [ - k ]

h[ n]

(1.5)

Introduction

15

y[ n] = e

jw 0 [ n ]

¥

åe

jw 0 [ - k ]

h[ n]

(1.6)

k =-¥

From Eq. (1.6) we can see that after exciting the system with exponential of frequency w 0 the result jw n is the same term with something. The term e 0 is known to us as eigen function and the multiplying term is known as a eigen value. This eigen value in a broader sense is known to us as a Fourier Transform. The basis function here is exponential e . This fact as discussed earlier in this chapter remains the same for other transforms also, for example in Z transform the basis function Z also equals e jw . It would be interesting now to look at from where this constant ‘ e ’ comes from. The constant e was discovered by Dr. Bernoulli in a vague accident for which the story does not have authentic source, it is still very interesting though. Dr. Bernoulli was trying to help his banker friend because his business was not picking up. He gave him a solution based on the formulae for calculation of compound interest. As we know the compound interest formula for investment of a one rupee or one dollar or one euro as principle amount is given as, n  1 1 +    n Dr. Bernoulli brought in series expansion by applying limit for n terms to infinity,  1 lim 1 +  n →∞  n

n

(1.7)

and the constant ‘e’ came into existence, waiting for Euler to give it meaning! An engineer’s perspective is provided in the MATLAB code (Example 1.4.3) for readers to try out the above limit approximation through simulations (rather than solving it actually – may be a mathematician’s perspective!) and get convinced that it indeed gives us the constant after few iterations. Example 1.4.3 — MATLAB code to approximate constant ‘e’. \\

16

Multiresolution and Multirate Signal Processing

Jacob Bernoulli

Jacob Bernoulli (1654–1705) was son of Nicolas Bernoulli and elder brother of Johann Bernoulli. Jacob acquired degree in theology to start his formal education as his father was against his children taking up mathematics. In spite of that due to his deep interest and curiosity he became a mathematician. In 1687, he became Professor of mathematics at University of Basel, where he taught till the end of his life. He continued working on the lines of Napier and was able to successfully discover the constant ‘e’. His contributions in the field of calculus are also well known and respected. Though the story of banker friend of Dr. Bernoulli lacks authenticity, it was certain that around 1889 he published series of articles on theory of infinite series and proposed continuously compounded interest. There were also some series expansions like 1+

1 1 1 + + + 2 + 4 9 k

for which he could not settle with a converging number and the quest was completed by Dr. Euler 2 when he showed that above infinite summation equates to p . Thus in many ways Dr. Euler took Dr. 6 Bernoulli’s work forward. After Dr. Bernoulli invented constant e , Dr. Euler came up with an identity known today as Euler’s Identity which is still regarded as one of the most beautiful of all mathematical discoveries. (1.8) eip = -1 From application perspective more useful version of Euler’s Identity is, eiq = cos(q ) + j sin(q )

(1.9)

The interesting thing to note is if we draw a tangent to any point on this exponential curve the y-intercept and slope of this tangent matches. Thus at any point along the curve we can resolve the point into ‘cos’ and ‘sin’ components which are in turn orthogonal in nature. Dr. Euler really gave special meaning to this constant ‘e’ and it’s because of his contribution we get orthognal and energy preserving transforms. It’s because of this unique property of ‘e’, the inverse of the transform also exists.

Introduction

17

Leonard Euler

Leonard Euler (1707–1783) was a Swiss mathematician and he was well known for the quality as well as quantity of his work. His contributions in the field of graph theory, number theory and calculus are considered to be outstanding. Born in Basel in 1707, in 1727 he went to St Petersburgh Academy. In 1741 he joined berlin Academy and in 1766 joined back St Petersburgh Academy. He died in 1783. In 1748 he gave a meaning to the constant ‘e’ and gave base to the natural logarithm which then went on to become basis to most of the transforms. After Dr. Euler gave a powerful meaning to ‘e’, Dr. Fourier worked on analyzing periodic and aperiodic functions and signals. The legacy of transforms is thus result of contribution from these three great scientists. Dr. Fourier gave a systematic way of analyzing periodic as well as aperiodic signals through Fourier series and Fourier Transform framework. For a time domain signal x(t ) the corresponding Fourier Transform is given by Eq. (1.10). ¥ Xˆ (W) = ò x(t )e- jWt dt -¥

(1.10)

Thus, conceptually Fourier description is the inner product between the signal to be analyzed and the basis or kernel function.

Jacob Fourier

Jacob Fourier (1768–1830) was a French mathematician and a well-known scientist with the contributions like Fourier series, Fourier descriptors, Fourier transform etc. Fourier introduced the notion of representing continuous as well as discontinuous functions using continuous basis functions. This was an important and path-breaking research, which also encouraged scientists like Laplace to build their own theories like Laplace transform. Now, let us look at question why wavelet transform? The wavelet transform decomposes signal into two separate series. One which represents the coarse version which leads to scaling function and other which represents the details or the refined version which leads to wavelet function or the mother wavelet function. Still two more questions need to be answered: Are not the traditional methods of representing signals good enough? And what is so special about this wavelet transform? We will answer this gradually in this book.

18

Multiresolution and Multirate Signal Processing

Let us take these questions one at a time. In the traditional methods the most basic representation of signals is with Taylor series. For example, Taylor series expansion at x0 = 0, ex = 1 +

¥ x1 x 2 x 3 xn + + + = å 1! 2! 3! n = 0 n!

(1.11)

Every single coefficient in the above expression can be looked upon as a decomposed piece and such pieces can be used for reconstructing the corresponding signal or function. If we only make use of finite number of coefficients lets say up to first 5 coefficients and try to reconstruct the signal then the reconstructed signal would look like the signal shown in Fig. 1.8. Figure 1.8 (b–f) shows Taylor series approximation of exponential function with number of coefficients used to be N = 1 , N = 2 , N = 3 , N = 4 , and N = 5 , respectively. It is clear that as the number of coefficients increase the approximation (shown as black dotted line) starts approaching the actual function (shown as black solid line). Thus in Taylor series, cooperation to build better representation is rigid from the perspective of lack of inter-coefficient resolution. Since we have to work with large terms, also scale and translation of every single term is limited. In contrast to this in wavelet analysis the scaling function and its associated wavelet function makes the representation flexible.

Exponential Function

8

Function ex

7 6 5 4 3 2 1 0 −2 (a)

−1.5

−1

−0.5

0

0.5

Exponential Function y = f(x) = e x

1

1.5

2

Introduction

19

8 7 6 5 4 3 2 1 0 −1 −2

Exponential Function approximated using Taylor series Function ex Taylor series approximation N = 1

−1.5

−1

0.5

1

2

1.5

Taylor series approximation using N = 1

(b)

8

0

−0.5

Exponential Function approximated using Taylor series Function ex Taylor series approximation N = 2

7 6 5 4 3 2 1 0 −2 (c)

−1.5

−1

−0.5

0

0.5

Taylor series approximation using N = 2

1

1.5

2

20

Multiresolution and Multirate Signal Processing

8 7 6 5 4 3 2 1 0 −1 −2

Exponential Function approximated using Taylor series Function ex Taylor series approximation N = 3

−1.5

−1

0.5

1

2

1.5

Taylor series approximation using N = 3

(d)

8

0

−0.5

Exponential Function approximated using Taylor series Function ex Taylor series approximation N = 4

7 6 5 4 3 2 1 0 −2 (e)

−1.5

−1

−0.5

0

0.5

Taylor series approximation using N = 4

1

1.5

2

Introduction

21

8

Exponential Function approximated using Taylor series Function ex Taylor series approximation N = 5

7 6 5 4 3 2 1 0 −2 (f)

Figure 1.8

−1.5

−1

−0.5

0

0.5

1

1.5

2

Taylor series approximation using N = 5

Approximation of exponential function using up to 5 coefficients from Taylor series

The MATLAB code to try more coefficients out is provided for readers: Example 1.4.4 — MATLAB code to understand Fourier transform.\\

22

Multiresolution and Multirate Signal Processing

Introduction

23

In wavelet analysis the scale {1 / 2} j is dependent upon the analyzer. Thus while sampling a high frequency signal we can bring in a very high value of j. Then a translation t j ,k = k / 2 j can be used to focus on that part. Hence with wavelet analysis we can ‘look’ into any particular part of the signal. By changing the scale and translation parameter we have an interesting zoom-in or zoom-out facility of wavelet transform. Fourier series has a noteworthy advancement over the Taylor series. Since elements of a Taylor series do not necessarily and always form an orthogonal set. However, in case of Fourier series the set {1, cos(nx ), sin(nx )}¥n=1 is always orthogonal on (-p , p ) . But still flexibility with scale and translation parameters remains to be explored. Even though Fourier series is rigid in terms of rotations and translations, we derive some information related to wavelet transformation from it. Fourier series is represented by following equation, ¥

f ( x ) = a0 + å [ ak cos(kx ) + bk sin(kx )] k =1

Where, a0 =

1 2p

ak =

1 ¥ f ( x ) sin(kx )dx p ò-¥

bk =

1 ¥ f ( x ) cos(kx )dx p ò-¥

ò

¥

-¥

f ( x )dx

(1.12)

24

Multiresolution and Multirate Signal Processing

We observe that a special relationship exists between cosine and sine terms of the series. A similar relationship also exists between scaling f(.) and wavelet y (.) functions. This relationship although trivial is very interesting. Though wavelets are radically different from conventional transforms like Fourier transform, its rise has its roots in the shortcomings of these conventional transforms. To find out what is so different about wavelet representation, let us revisit the Fourier theory briefly.

1.4.4

How Fourier Transform Works?

To answer this question, let us start with Fourier basis function. It is the complex conjugate of known frequency. Euler’s theorem allows us to divide this basis into real and imaginary parts as eiq = cos q + i sin q . The real and imaginary parts of basis function used in Fourier transform are shown in Fig. 1.9. Imaginary Part

6

7

7

1 0 −1

8

8

0

0

0

0

0

1

1

1

1

1

2

2

2

2

2

3

3

3

3

3

4

4

4

4

4

5

5

5

5

5

6

6

6

6

6

7

7

7

7

7

8

0

1

2

3

4 Time

Figure 1.9

5

6

7

1 0 −1 1 0 −1 1 0 −1 1 0 −1 1 0 −1

8

8

8

8

8

1 0 −1

k=0

1 0 −1

k=3 5

6

k=2 4

5

k=1

3

4

k = −4

1 0 −1

2

3

k = −1

k = −3

1 0 −1

1

2

k = −2

k = −2

1 0 −1

0

1

k = −3

1 0 −1

k = −1

k=0

1 0 −1

0

1 0 −1

k = −4

k=3

1 0 −1

k=1

k=2

Real Part 1 0 −1

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4 Time

5

6

7

8

Real and imaginary parts of Fourier basis function

Let us say we want to analyze a stationary signal using Fourier mechanism. Let us also assume that this signal is made up of two frequencies say sin(4 x ) and cos(7 x ) . These are linear frequencies of emerging out of independent variable x and readers should note that sin(4 x ) is not 4Hz sinusoidal. We can excite this signal using all the basis functions as in Fig. 1.9 and integration of

Introduction

25

dot product yields the Fourier domain representation. We know that in Fourier domain this will produce two peaks corresponding to 3x and 7x coming from sin and cos components respectively in the signal under the probe. The peak generation process comes out of the fact that this mechanism is a dot product or correlation filter mechanism. This can be clearly witnessed from the simulation results below. Figure 1.10 shows a case when the signal is excited with basis function with different frequency than one of the existing frequency components ( sin(4 x ) or cos(7 x ) in this case). The Fig. 1.10(a) shows the actual signal, Fig. 1.10(b) shows the sin(4 x ) basis function superimposed on the signal to be analyzed and Fig. (1.10(c)) demos the effect of dot product in case of excellent correlation. The positive parts when integrated will produce the desired peak at that frequency ( 4x in this case). 1.5

1.5

1

1

1.5 1

0.5

0.5

0.5

0

0

0

−0.5

−0.5

−0.5

−1

−1

−1.5

−1.5

0

1

2

(a) Stationary Signal with sin(4x) and cos (7x) frequencies

Figure 1.10

3

4

5

6

7

−1 0

1

2

(b) Signal super imposed with sin(4x) basis function-good correlation

3

4

5

6

−1.5 7 0

1

2

3

4

5

6

7

(c) Dot product of signal with sin(4x) basis giving many positive parts.

When excited with basis frequency which belongs to the signal it produces peak at that frequency

When the basis uses frequency, which is not part of stationary components of signal, it results in poor correlation and will not produce peak for that frequency. Figure 1.11 shows a case when the signal is excited with basis function of one of the existing frequency components ( sin(4 x ) in this case). Figure 1.11(a) shows the actual signal, Fig. 1.11(b) shows the sin(7 x ) basis function superimposed on the signal to be analyzed and Fig. 1.11(c) demos the effect of dot product in case of poor correlation. When integrated, the positive parts destroy the negative parts not producing the peak at that frequency ( 7x in this case as the basis had sin part and signal has cos part. If the signal is excited with cos(7 x ) it will certainly produce the peak). From the simulation we can see that if the basis function contains frequency present in the actual signal on integration of the dot product we will get a peak at that frequency. This brings the clarity on how exactly Fourier Transform works. For further clarity readers can run the following MATLAB program and try out different basis examples and bring out conviction on how the dot product helps deploy the thematic of Fourier Transform.

26

Multiresolution and Multirate Signal Processing

Introduction

Example 1.4.6 — MATLAB code: Fourier for non-stationary signals.\\

27

28

Multiresolution and Multirate Signal Processing

Introduction

29

30

Multiresolution and Multirate Signal Processing

Example 1.4.7 — AM-GM inequality.

Further, we will use the result of Young’s inequality to get the result (1) i.e. the Hölders Inequality. To do this, consider two real functions f (t ) and g(t ) in space L p () and Lq () respectively. We first normalize these functions by dividing each of them with their respective norms. Mathematically, f (t ) Î L p (  ) g(t ) Î Lq () 1 1 given that + = 1. Now the normalized functions are given as: p q

Introduction

31

F (t ) =

f (t )  f p

G (t ) =

g (t )  g q

Where,  f p =

(ò

 g q =

(ò

+¥

-¥

+¥

-¥

| f (t ) | p dt

q

| g(t ) | dt

)

)

1 p

1 q

Now using Young’s inequality we can point wise state that, F p (t )

G q (t )

+

p

q

³

F (t )G(t )

Integrating above equation over real line we get,

ò

+¥

ò

+¥

-¥

-¥

q +¥ | G (t ) | | F (t ) | p dt + ò dt -¥ p q

f (t ) f +¥

p p

p

p

dt + ò

+¥

-¥

g ( t )q

dt

q

³

ò

+¥

³

ò

+¥

+¥

³

ò

ò

+¥

g qq

p 1 ò-¥ | f (t ) | dt 1 + p q  f pp

ò

+¥

| g(t ) |q dt

-¥

 g qq 1 1 + p q

³

-¥

-¥

-¥

| F (t )G(t ) | dt f (t ) g (t )  f p  g q

dt

| f (t )g(t ) | dt

 f p  g q -¥

| f (t )g(t ) | dt

 f p  g q

Therefore,  f p  g q

³

ò

+¥

-¥

Since, 1 1 + =1 p q

| f (t )g(t ) | dt

(1.15)

32

Multiresolution and Multirate Signal Processing

Example 1.4.5 — MATLAB code to understand Fourier transform.\\

Introduction

33

1.5

1.5

1

1

1

0.5

0.5

0.5

1.5

0

0

0

−0.5

−0.5

−0.5

−1

−1

−1.5

−1.5 0

1

2

3

4

(a) Stationary Signal with sin(4x) and cos(7x) frequencies

5

6

7

−1 0

1

2

3

4

5

(b) Signal superimposed with sin(7x) basis function-poor correlation

6

−1.5 7 0

1

2

3

4

5

6

7

(c) Dot product of signal with sin(7x) basis giving almost equal positive and negative parts.

Figure 1.11 When excited with basis frequency which does not belong to the signal it produces cancellations It must be noted, however that Example 1.4.5 shows how Fourier Transform operates effectively for stationary signals. In case of non-stationary signals, Fourier transform fails to provide the information related to at what time which frequency existed. All the real-life signals are non-stationary and Fourier does not provide insight into the time stamps of frequency components to be analyzed. This is a serious limiting factor for many applications and corresponding usage of Fourier.

1.4.5

Fourier Transform and Non-stationary Signals

Stationary signals have all the frequencies existing all the time and because of this exact location of a particular frequency on time axis does not have much meaning for stationary signals. The answer to the question ‘At what time instance a particular frequency existed?’ will be at every time instance! This does not hold true in case of non-stationary signals and hence exact location of frequency plays vital role. Fourier transform does not capture this and the same is depicted in the MATLAB code Example 1.4.6. The code creates two signals, x1 (t ) is stationary and x2 (t ) is non-stationary. As shown in Fig. 1.12 for both these signals the Fourier engine produces similar responses. The exact frequencies are successfully captured by FFT, however, this is not enough for non-stationary signals as the frequencies are present only at pertinent time stamps and not throughout. Interestingly, when inverse Fourier Transform of FFT of non-stationary signal is taken, it produces back stationary signal like x1 (t ) in lieu of producing back original non-stationary signal x2 (t ).

34

Multiresolution and Multirate Signal Processing

Stationary Signal x1(t)

1 0 −1 5

0 100 50 0 1 0 Similar −1 100 50 0

10

15

FFT of x1(t) 0

2

4

6

8 10 12 14 Non Stationary Signal x1(t)

5

0

16

18

20

15

10 FFT of x2(t)

0

Figure 1.12

2

4

6

8

10

12

14

16

18

20

Fourier Transform: difficulty in handling non-stationary signals

Fourier transform can not handle the non-stationary signals because it is governed by ‘Uncertainty Principle’. The subsequent sections will bring out this point systematically.

1.4.6

Inability of Simultaneous Time and Band Limitedness

According to the theorem based on the Uncertainty principle of Fourier transform, a non-zero function cannot be both time limited and band limited at the same instance. Basically, this means that a non-zero function cannot be compactly supported in both the domains simultaneously. Now, a function is said to be compactly supported if it has a finite support; for example, a rectangular pulse ranging from point a to b on real line is compactly supported. On the contrary, the Gaussian function which extends throughout the real line is not a compactly supported function. We will prove the result for a class of functions in L1 space and then extend the proof for a general class LP where 1 < P < ¥. Background of the proof: Let us first recall the basics before we study other tools applied. Fourier transform Fourier transform provides a way to look at the signal from its frequency domain. Fourier transform relies on the principle that a signal can be represented by the linear combination of sine waves, and their respective frequencies together constitute the frequency domain of the signal. Further, we can move to e jt - e- jt complex domain since sin(t ) = . Mathematically, Fourier transform projects the function onto 2j the complex exponentials. Thus, Fourier transform of a function x(t ) is given as: +¥

X (W) = ò x(t )e- jWt dt -¥

NOTE:

Introduction

35

For a function, there exists a Fourier transform only if the function is absolutely integrable i.e. x (t ) +¥

is said to have a Fourier transform only if ò | x (t ) | dt is finite. -¥

Space LP () and Norm A function x(t ) lies in space LP () if

{ò

+¥

-¥

P

| x(t ) | dt

} is finite, and the term is called as the L 1 P

P

norm of

the function x(t ) where 1 £ P £ ¥. With the context of our discussion let us see an example of function with finite L1 norm: We see that, for above given function the absolute integral of the function is equal to 1 and thus it belongs to space L1 . Also, it can be proved that the function exists in space LP in general for 1 £ P < ¥. On the other hand sin c function does not have a finite absolute integral and, therefore, does not belong to space L1 . 1 t

Figure 1.13

Function belonging to L1 ()

Hölders Inequality Aforesaid, in this proof we will consider the functions in L1 space and then extend the proof more general class of functions in spaces LP where 1 < P £ ¥. Now, for time to limited functions it can be proved that if a function belongs to LP where 1 < P £ ¥ then it also belongs to space L1 using Hölders Inequality. Mathematically, f Î LP , for p Î (1, ¥) Þ f Î L1 Hölders Inequality states that for two functions f (t ) and g(t );

ò

+¥

-¥

f (t )g(t )dt £

(ò

+¥

-¥

p

| f (t ) | dt

) (ò 1 p

+¥

-¥

q

| g(t ) | dt

)

1 q

(1.13)

1 1 + = 1 and p, q > 0. The above result of Hölders Inequality p q can be obtained using Young’s Inequality and generalized AM-GM Inequality stated as below. The above condition holds true only if

Young’s Inequality Young’s inequality states that, given two positive real numbers a and b, a, b Î  + ab

£

a p bq + p q

Given, 1 1 + =1 p q

(1.14)

36

Multiresolution and Multirate Signal Processing

This can be proved using the generalized AM-GM Inequality as explained below. Arithmetic Mean- Geometric Mean (AM-GM) Inequality AM-GM inequality states that, given two positive real numbers a and b with weights a 1 and a 2, following equations holds. a 1a + a 2 b ³ aa1 ba 2 a1 + a 2

(

)

1 a1 +a 2

1 1 Now, substituting a = a p , b = bq, a 1 = p and a 2 = q and knowing that + = 1, we get Eq. (1.14) p q i.e. Young’s inequality. Thus, using the Hölders Inequality as stated in Eq. (1.13), by selecting appropriate function g(t ) we can prove that function f (t ) which belongs to L p () also belongs to L1 ( ) . Now, to prove this consider a function spread over finite interval C which belongs to space L p , 1 < p < ¥. Since the function f (t ) belongs to space L p its pth norm is finite i.e.

(ò

+¥

-¥

| f (t ) | p dt

)

1 p

is finite. The function f (t ) can be similar

to that shown in Fig. 1.14. We select g(t ) as shown in Fig. 1.15 to get the result. f (t ) C

Figure 1.14

t

Function f (t )

1 g(t ) C t

Figure 1.15

Function g(t )

Now putting down the Hölders Inequality for these two functions as follows:  f p  g q

³

ò

C

| f (t )g(t ) | dt 1 p

ò

| f (t ) | dt £

( ò | f (t ) | dt ) ( ò dt )

ò

| f (t ) | dt £

( ò | f (t ) | dt )

C

C

p

C

C

p

C

1 q

1 p

1

Cq

Since the right side of the above equation is the product of L p norm which is finite and another finite quantity the net result on the RHS is finite. Also, since the LHS is less than or equal to RHS we can infer that the L1 norm of f (t ) is finite. Thus, we have proved our statement that if a function belong to space L p (), where 1 < p < ¥, then it also belongs to space L1.

Introduction

37

With this background let us now know some tools which will help us in proving the final result. Tools for the proof Fourier transform of rect (a) The Fourier transform of the rect (a) function, shown in Fig. 1.16, is 2 æ aW ö sin ç W è 2 ÷ø 1 = sin(paf ) pf = a sin c(af )

(rect (a )) =

Where, W = angular frequency and f = Hertz frequency. 1 a t 2

−a 2

Figure 1.16

Function rect (a)

Vandermonde Matrix A Vandermonde matrix is defined as Vi , j = aij -1 é1 a1 ê 1 a2 V =ê ê� � ê êë1 am

a12 a22 � am2

� a1n -1 ù ú � a2n -1 ú � � ú ú � amn-1 úû

Where, m and n are the number of rows and columns respectively. Further, whenever we need to solve any linear equation of the form Ax = y , we always look for the invertibility of matrix A so that we can solve for x as x = A-1 y . Now the Vandermonde matrix have very

Example 1.4.8 — MATLAB code: Fourier for non-stationary signals. \\

38

Multiresolution and Multirate Signal Processing

good properties by which it can be shown to be invertible once we know the entries. This is because the determinant of a Vandermonde matrix can be proved to be of the form t (V ) = ( -1)

m

m -1 2

×

Õ

(ai - a j )

1£ i < j £ m

Therefore, until all the entries of a particular row are distinct the value of det(V ) will not be equal to zero and V will be invertible.

Introduction

39

Review of the proof Consider a very smooth function which is compactly supported. For example, we can have the following function: -1

f ( x ) = e1- x =0 1 1

Figure 1.17

2

x £1 else

f (x) 0

1x

Example of a smooth and compactly supported function

The function has the structure as shown in Fig. 1.17. It can be shown that all the derivatives of f ( x ) exist in the interval -1 £ x £ 1 with all derivatives f p ( x ) ® 0 as | x |® 1. Now, we need to find if we can have a compactly supported spectrum for such well-behaved time-limited function. Let us look at an existing proof. For any general well-behaved function in space L1 () similar to above example we have the Fourier transform given by: +¥

F (W) = ò f (t )e- jWt dt -¥

40

Multiresolution and Multirate Signal Processing

Now, we will also assume that the function is band-limited and show that our assumption will hold only for a zero function. Therefore, with the assumption we have | F (W) | £ £

ò ò

+¥

-¥ +¥

-¥

| f (t ) || e- jWt | dt | f (t ) | dt

£ L1 norm of f (t ) Now, if we take the derivative of f (t ), it can be represented as, +¥

f ¢(t ) = jW ò f (t )e- jWt dt -¥

In the above equation, since function is bounded in both time and frequency, we get a finite bound on the magnitude of f ¢(t ). Similarly, we can show that all the derivatives of f (t ) have finite bound on their magnitude. Further, we can express function f (t ) in the form of a McLauren series as: f (t ) = f (0) + tf ¢(0) +

t2 f ¢¢(0) +  2!

Introduction

41

Now as per our assumption all the above derivatives of f (t ) are finite and therefore above equation is essentially a power series expansion, i.e.

Exercises Exercise 1.1

Exercise 1.1 What is Short Time Fourier Transform (STFT) and its drawbacks as compared to wavelet transform. Hint: In STFT, signal is first windowed using different type of window functions like Triangular window, Rectangular window, Gaussian window, etc. Now, Fourier Transform of resulting windowed signal is taken. This gives the STFT of signal for particular time. As window slides along time axis, so basically STFT maps input signal x(t ) into two dimensional function in a time-frequency plane or tiling. So, drawback of STFT is that once a window has been chosen for STFT, then time-frequency resolution is fixed over the entire time-frequency plane or say STFT moves a tile of constant shape in the time-frequency plane. But in case of wavelet transform, the time resolution becomes arbitrarily good at high frequencies, while the frequency resolution becomes arbitrarily good at low frequencies. In other words we can say that CWT moves a tile of variable shape but constant area which is governed by time-bandwidth product.

Exercise 1.2

What is uncertainty principle? Explain it for different domains. Hint: The basic uncertainty principle also called Heisenberg Uncertainty Principle is that, the position and velocity of an object cannot both be measured exactly, at the same time. Uncertainty principle derives from the measurement problem, the connection between the wave and particle nature of quantum objects. The uncertainty principle is alternatively expressed in terms of particle’s momentum and position. the momentum of a particle is equal to the product of its mass times its velocity. The principle applies to other related (conjugate) pair of observables, such as energy and time. Finally, uncertainty principle states that exact knowledge, of complementarity pairs (position, energy/velocity, time) is impossible. The uncertainty principle for time-frequency pair will be studied further in detail in this course.

Exercise 1.3

What are basis functions? What type of basis function is used in Fourier Transform? How is it different from the basis functions used in Haar multiresolution analysis? Ans. In mathematics, a basis function is an element of a particular basis for a function space. Every function in the function space can be represented as a linear combination of basis functions, just as every vector in a vector space can be represented as a linear combination of basis vectors.

42

Multiresolution and Multirate Signal Processing

Complex exponential functions form the basis for Fourier analysis. They are extremely smooth functions. On the other hand, Haar analysis uses discontinuous functions as basis, i.e. every signal is represented as a linear combination of discontinuous functions.

Exercise 1.4

  If sum and difference of two vectors a and b are perpendicular to each other. Find the relation between two vectors.     Hint: The sum a + b and difference a - b are perpendicular to each other. Hence, their dot product should be equal to zero.

Exercise 1.5

Find a function f (t ) = a + bt that is perpendicular to the another function g(t ) = 1 - t in the interval [0,1]. Hint: If the functions are perpendicular to each other, then their dot product is zero. 1

á f , gñ = ò (a + bt )(1 - t )dt 0

1

ò (a + bt - at - bt 0

b a b - 2 2 3 a b + 2 6

a+

2

)dt

=0

=0 =0

So, we can take f (t ) = 1 - 3t

Exercise 1.6

Determine the number of dimensions in the following sequences: (a) (....,0,0,4,5,3,1,6,0,0....) (b) (....,0,0,4,0,0,1,0,9,6,0,0....) Hint: (a) The dimension of a sequence is the the length of support of a sequence. In this example, a sequence has 5 non-zero samples and hence it has a dimension of 5. (b) Since the dimension of a sequence is the length of support of a sequence, a sequence has a dimension 7. Note that this also considers the 3 zero samples which have non-zero samples on their either left or right hand sides.

Introduction

43

Exercise 1.7

What is Wavelet? How Wavelet transform is different than Fourier transform? Hint: Refer Section 1.4 of this chapter.

Exercise 1.8

What are orthogonal vectors? How functions, sequences and vectors are relate? Hint: Refer Chapters 1 and 4 of this book.

Exercise 1.9

What is the concept of dot or inner product? How this product is different than cross or outer product? Describe properties of dot product? What is the significance of dot product in Transforms? Hint: Refer Section 1.4 of this chapter.

Exercise 1.10

Explain what is uncertainty principle? How this principle revolves around evolution of wavelets from conventional transforms? Hint: Refer Chapters 7 to 10 of this book.

Exercise 1.11

Exercise 1.11 What is Parseval’s Theorem? Explain its applications? Hint: Refer Section 1.4 of this chapter.

Exercise 1.12

Explain how Fourier Transform produces peaks corresponding to the frequencies present in the signal under consideration? Explain using concept of dot product? Hint: Refer Section 1.4 of this chapter.

Exercise 1.13

Explain limitations of Fourier Transform in handling non-stationary signals? Hint: Refer Section 1.4 of this chapter.

44

Multiresolution and Multirate Signal Processing

Exercise 1.14

What are multirate systems of doing signal analysis? Describe sampling theorem and explain what is the role of sampling theorem in design of multirate systems? Hint: Refer Section 1.4 of this chapter.

Chapter

2

The Haar Wavelet

Introduction The Haar wavelet Lk norms of x(t ) Haar MRA (Multi Resolution Analysis) Axioms of MRA Theorem of MRA

2.1

Introduction

In this chapter we shall discuss the Haar Multiresolution Analysis about which we had briefly discussed previously. Haar was a mathematician, who gave a radical idea that any continuous function can be represented in the form of discontinuous functions, and by doing so one can go to any level of continuity that one desires. This is the central idea in the ‘Haar way’ of representing functions. We start from a very discontinuous function and make it smoother by adding more and more discontinuous functions (which is in a way some additional information) to it until we reach arbitrarily closer to the continuous function that we are trying to approximate. This idea is opposite to the idea of the Fourier transform. One can recall that in Fourier transform the discontinuous function is represented in the form of smooth continuous function.

lfred Haar

(Hungarian: Haar Alfréd; 11 October 1885, Budapest 16 March 1933, Szeged) was a Hungarian  mathematician. In 1904 he began to study at the University of Gottingen . His doctorate was supervised by David Hilbert. The Haar measure, Haar wavelet, and Haar transform are named in his honor. The Haar sequence is now recognised as the first-known wavelet basis and extensively used as a teaching example. Representation of the continuous function in the form of discontinuous function has its own importance. Let us look at an example to illustrate this.

46

Multiresolution and Multirate Signal Processing

Example 2.1.1—Dyadic wavelets picture example.

Information is generally represented in the form of bits. While transmitting an audio piece we convert a extremely smooth audio pattern into a highly discontinuous stream of bits. When these bits are transmitted over a communication channel a discontinuity is introduced every time a bit changes. This discontinuity may be in the function or its derivatives. Thus Haar’s way of representation is very useful for us today. Let us first start building the idea of ‘wavelets’ or what are more specifically called as ‘dyadic’ wavelets (explained later). We shall start with the ‘Haar’ wavelet. For that let us consider how we represent a picture on a screen. Every picture is enclosed by a boundary. Contents of the picture such as a person or a tree are enclosed within the boundary. The natural scene is inherently continuous. In order to represent it in a computer we divide the area of the image into very small subareas. These small rectangular sub-areas are called ‘picture elements’ or ‘pixels’. Now if we make 512 divisions on the vertical and also on the horizontal then we say that we have a ‘512 × 512’ image. We can also have images at other resolutions. Now consider an image of dimension 204 × 92 pixels. We have 204 divisions on one side and 92 divisions on the other. There exists something called as the piecewise constant representation of the image. What we do is represent each piece (here pixel) by a constant number which represents the average intensity and colour in that area. As we increase the number of divisions the area of each pixel goes on decreasing. Thus as we go from a ‘512 × 512’ image to a ‘1024 × 1024’ image, the area of a pixel becomes one fourth. We introduce more information by increasing the number of divisions. Evidently as we increase the number of pixels, we go closer and closer to the original picture. The effect of changing the resolution of an image can be seen in Fig. 2.1. Thus one may say that ‘the smaller the pixel area the larger the resolution’. This can be seen easily from the picture below (Fig. 2.1). But the idea that we are trying to build is not quite the idea of the ‘Haar MRA’. The Haar MRA does something deeper.

L1

L2 408°184

L3 204°92

Figure 2.1

L4 102°46

51°23

Resolution difference

Example 2.1.2—Audio signal piecewise constants.

The same can be done of an audio signal. Consider an audio output which is a one dimensional signal and can be plotted against time. Let the output be as shown in Fig. 2.2. Since this is now ‘1 dimensional’ we divide the time axis in small intervals of size ‘T’. We now represent the function values in each interval by a constant number. Now, it is convenient to choose this number to be the average of the function values in that time interval. By doing so we are trying to represent a continuous function by a discontinuous one. The average C0 over the strictly open interval (0,T) can be computed as

47

The Haar Wavelet

1 T x(t ) dt T ò0

C0 =

(Open interval is the interval excluding end points). For any interval of size T we have 1 x(t ) dt T òT

Cn = Similarly, for interval of size T/2, Cn,T /2 =

1 T x (t ) dt T / 2 ò2

If a time interval of length ‘T’ is divided into two time intervals of length T/2, we get two averages computed by the above formula. Figure 2.3 clarifies this concept further. Now, consider 1 T x(t ) dt T ò0 2 T /2 = ò x(t ) dt T 0 2 T = ò x(t ) dt T T /2

=

AT A1,T /2 A2,T /2

Average

0

T

2T

Figure 2.2

3T

4T Time (t)

Audio signal

Averageonintervalof(0,T/2)

A2,T/2 A,T

A1,T/2

0

Figure 2.3

T/2

Average of T and T/2

T

48

Multiresolution and Multirate Signal Processing

AT is the average of the function on the interval of size ‘T’ while A1,T /2 and A2,T /2 are the averages on T the two half intervals of size ‘ ’ each. The key concept in Haar multi resolution analysis is to relate 2 these three terms ( AT , A1,T /2 , A2,T /2 ) and it is in that relationship that the Haar wavelet is hidden. It can be easily observed that, the average of A1,T /2 and A2,T /2 gives AT , i.e. AT =

A1,T /2 + A2,T /2 2

Thus a function can be approximately represented by addition of piecewise constant functions. We can go on reducing the interval by half to whatever degree of accuracy we desire. This is illustrated in the Fig. 2.4. Each different line (dot dash, dash-dash, bold line with squares) represents a piecewise approximation in its own way, with different resolution. Let the function represented in Fig. 2.4 using dotdash line be f1 (t ) and the bold line with squares be f2 (t ) then, the additional information obtained by representing the signal as f2 (t ) is given by f2 (t ) - f1 (t ) which is shown in Fig. 2.5.

0

T

Figure 2.4

2T

Intervals of 2T,T and T/2

h1 h1 T/2

T

Figure 2.5

h2 3T/2 h2

f2 (t ) - f1 (t )

2T Time

49

The Haar Wavelet

2.2

The Haar Wavelet

To understand ‘Haar Wavelet’, consider the function shown in Fig. 2.6. This function is represented as y (t ). By usingscalar multiplication and delaying, we can see that f2 (t ) - f1 (t ) can be reconstructed from y (t ). Thus, t −T f2 (t ) − f1 (t ) = − h1 × y (t / T ) + h2 × y   T  ψ(t) 1 1/2 1

1 HAAR wavelet

Figure 2.6

f2 (t ) - f1 (t )

here h1 and h2 are as shown in Fig. 2.5. The function y (t ) is called the Haar wavelet. In general when t −t  we start with y (t ) we can construct a function y  as a building block, where ‘s’ is positive real and  s  t should be real. The variable ‘s’ dilates y (t ) and ‘˜ ’ translates y (t ). The variable t is called the translation index and the variable ‘s’ is called the dilation variable. If we consider time intervals of length T/2 for piecewise constant approximation then the value of ‘s’ is T/2, if length of time interval is T then s = T. It means the single functiony (t ) allows you to bring in resolution step-by-step to any level of detail. Thus, by dividing T into smaller subdivisions of T/2, T/4 and so on, any function xa (t ) can be made arbitrarily close to original function x(t ). If xe (t ) denotes the error due to approximation, it can be expressed as xe (t ) = x (t ) - x a (t ) ¥

z = ò | xe (t ) |2 dt -¥

Where, z is the squared error. What we mean by arbitrarily close is that for any fixed value of z (> 0), we can always find a positive integer m such that a piecewise constant approximation of x(t ) with an interval of T / 2 m satisfies the requirement of z . It must be noted that any signal can be represented in piece wise constant form if and only if it has finite energy.

2.3

Lk Norms of x (t)

A function with finite energy content implies and is implied by its L2 norm being finite. So what is this L2 norm? The L2 norm of a signal is defined as 1

¥ 2 L2 norm of x(t ) = é ò | x(t ) |2 dt ù úû ëê -¥ In general, we can define the L p norm of x(t) as 1

¥ p L p norm of x(t ) = é ò | x(t ) | p dt ù úû êë -¥

50

Multiresolution and Multirate Signal Processing

where p is any real number. The L¥ norm of x(t ) is defined as 1

¥ p L¥ norm of x(t ) = lim é ò | x(t ) | p dt ù ú ê -¥ p ®¥ ë û

2.3.1  Significance of L¥ Norm As the value of p increases, large values in x(t ) are being emphasized. This happens because for a large p, the integral will have a large contribution from higher values in x(t ). L2 () is said the to be the space of all real functions whose L2 norm is finite. The word ‘space’ is used with the intent that if we take a linear combination of two or more functions in that set then we get back a function in that set. As we reduce the size of the individual time intervals previously considered, we said we could as close to the original function as we desire by reducing the error. This clearly means that L2 norm of the error can be reduced to as small a value as we desire. The Fourier series allows us to do this for a reasonable class of functions. The same kind of thing is happening here. Just on function y (t ) is able to take us as close as we desire to the function which we wish to approximate. And this is just one y (t ). The whole subject of wavelets allows us to build many such y (t )’s. Each time we improved the resolution by factors of ‘2’ and, hence, the term ‘dyadic’ which represents steps of two was used in the beginning of the chapter. Thus the Haar wavelet is an example of a dyadic wavelet. In the following chapters we will mostly focus our attention on dyadic wavelets. Dyadic wavelets are the most easily designed, the best and most easily implemented and also the best understood. After understanding the ‘Haar wavelet’ and the way in which the Haar MRA is constructed many concepts of multiresolution analysis would become clear. In this chapter, we brought out the idea of the Haar wavelet explicitly. We now know that dialates and traslates of this function can capture information in going from one resolution to the next level of resolution in steps of two each time. So in terms of the spaces we are actually going from one subspace of L2 () to the next subspace. The next task is to see how the dilates and translates of Haar wavelet help us in adding more and more to the subspaces in going from a coarser subspace all the way to L2 () on one side and all the way down to a trivial subspace on the other (also referred to as the ladder of subspaces). Further, we will bring out the idea of the basis of these subspaces and how the Haar wavelet helps in capturing the difference subspace.

2.4

Haar MRA (Multi Resolution Analysis)

As we got a basic instinct about Haar MRA in the previous section, let us understand the fundamental concepts surrounding it in next few sections. The underlying principle of wavelets is to capture incremental or so called additional information in a function. The piecewise constant approximation of a function is a representation of the function at different resolutions. For understanding this, consider an example of an onion. Let the outermost shell be of the maximum reso-

R lution. Now, the job of wavelet function is to peel-off or to take out a particular shell of that onion. Thus, we are essentially peeling-off shell-by-shell using different dilates and translates of a wavelet. Figure 2.7 illustrates the concept of nested subspaces. So, it goes like this, corresponds to ® different resolutions. Different dilates ¾¾¾¾¾ takes us along Different translates ¾¾¾¾¾ ® same resolution corresponding to a given dilate.

51

The Haar Wavelet

The idea of wavelets may be introduced using an example of the Haar wavelet. The Haar wavelet is a dyadic wavelet, i.e. the piecewise constant approximation is defined in steps of powers of two at a time. The wavelet captures the incremental information between two consecutive levels of resolution. In other words, Haar wavelet keeps on retreiving additional information as we move on towards higher and higher resolution. … ⊂ V1 ⊂ V2 ⊂ …

W1

W0

Figure 2.7

V0

Nested subspaces

Example 2.4.1

The idea of expressing a function at different resolutions may be explained by considering an example. Observe Fig. 2.8, it shows the signal at high resolution in dotted line, its piecewise constant approximation over unit intervals in blue and that over intervals of length 0.5 in green. This piecewise constant approximation is done in the same way as was discussed in the previous lecture, i.e. by taking average value of the function over the dyadic intervals. The corresponding function which gives the incremental information between the two approximation levels is shown in Fig. 2.9. This was the case of a 1-dimensional function. 1 0.8

Signal at high resolution approximation over unit intervals

0.6

approximation over 0.5 intervals

0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1

0

0.5

1

1.5

Figure 2.8

2

2.5

3

3.5

4

Signal at different resolution

4.5

5

52

Multiresolution and Multirate Signal Processing

Example 2.4.2

The same idea may be extended to two dimensions as well. In Fig. 2.10, Figure 2.10a is the 2-dimensional image at a certain level of resolution. Figure 2.10b(i) is the image at 0.5 resolution of Fig. 2.10(a). Fig. 2.10b(ii), 2.10b(iii) and 2.10b(iv) gives the additional information in the vertical, horizontal and diagonal directions respectively (also known as ‘bands’). Thus, the idea of wavelets is analogous to an object with many shells. Wavelet translates at the maximum resolution, takes out the outermost shell, the next shell is taken out at the next lower resolution and so on. Hence, we are essentially ‘peeling off’ shell by shell using different dilates and translates of the wavelet function. The dilation takes us to the next level of resolution, while translation takes us along a given resolution. Now, let us look over the focal point for further operations on wavelet to perform. 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5

0

0.5

1

1.5

Figure 2.9

2

2.5

3

3.5

4

4.5

Incremental information

b(i)

b(iii)

b(ii)

b(iv)

dwt

a

Figure 2.10

2D example, showing image 0.5 resolution and incremental information

5

53

The Haar Wavelet

Focal Point:- Piecewise constant approximation on unit intervals.

R Without any loss of generality, let us begin with piecewise constant approximation at a resolution of unit length. The choice of unit interval is entirely one’s own choice.

Now a question arises at this point. What function f (t ) is such that its integer translates can span the space of piecewise constant functions on the standard unit intervals? As we saw in linear algebra or more generally functional analysis, the span of a set of vectors in a vector space is the intersection of all subspaces containing that set. Here, space refers to a linear space of functions, that is, a set of functions which is closed under linear combinations. In other words, the linear combination of these functions belong in the same set/space as of the function itself. Here, we only consider finite linear combinations. The same ideas may be extended for infinite linear combinations. A set V0 is defined as follows, V0 : { x(t ), such that x(×) Î L2 () is piecewise constant on interval ] n, n + 1 [ , " n Î  }. The subscript ‘0’ is used for V0 because of piecewise constancy on interval of size 2 -0. Similarly, V1 is defined as, V1 : { x(t ), such that x(×) Î L2 () is piecewise constant on all ] 2 -1 n,2 -1 (n + 1) [ , " n Î  }. V1 is the set of functions piecewise constant over the interval of 2 -1. In general Vm is the set of functions which is piecewise constant over the interval of size 2 -m . Vm : { x(t ), such that x(×) Î L2 () is piecewise constant on all ] 2 - m n,2 - m (n + 1)[ , " n Î }. Example 2.4.3—Function.

Function x(×) Î V2. All function belonging to V2 are piecewise constant over the interval of 2 -2 i.e. 0.25. Figure 2.11(a) shows a function belonging to V2. Here x(t ) Î V2 means x(t ) Î L2 (). This implies that when squared sum of all the piecewise constant values of the function is carried out, it converge to a finite value. 1.4

1.3

1.2 1 1 0.8

0.7

0.7 0.6

0.6 0.4 0.2 0.2 0 −0.2 −0.4 −0.4 0

0.25

0.5

Figure 2.11(a)

0.75

1

1.25

1.5

Example of a function belonging to V2 space

1.75

54

Multiresolution and Multirate Signal Processing

Example 2.4.4—Function belonging to V-1 space.

Any function belonging to V-1 is piecewise constant over the interval of length 2 - ( -1) i.e. 2. Figure 2.11(b) shows a function belonging to V-1. Now, a function which is piecewise constant over the interval of 1 is also piecewise constant on the interval of 0.5 (as that constant value can be divided in two parts). Therefore, a function belonging to space V0 also belongs to space V1.

1 0.7 0.6 0.5 0.2 0

−0.4 −0.5

0

1

Figure 2.11(b)

2

3

4

5

6

7

8

Example of a function belonging to V-1 space

In general a function which belongs to space Vm also belongs to space Vm+1. Hence, a ladder of subspaces is implied as depicted below: ...V-2 Ì V-1 Ì V0 Ì V1 Ì V2 ... Intuitively, we can see that as we move towards right, i.e. up the ladder we are moving towards L2 () {ÈVm }mÎ = L2 () The expression above, if written without the coverline implies that it covers all the interior region whereas with closure (by coverline) ensures the covering of boundary patches too. This talk is not of much importance though but a slight glance at it is necessary. When we go left towards the ladder? Movement towards leftwards implies, piecewise constant approximation over larger and larger intervals, as m in 2 - ( m ) goes more and more negative. Now, consider L2 norm of function going towards leftwards: ¥

å |C

m

(n) |2 2 - m

n = -¥

where C(×) is approximate coefficient at resolution 2 -m . Now, as we move towards left m becomes negative and m ® -¥. Therefore L2 norm is given by

55

The Haar Wavelet ¥

2| m |

å |C

m

(n) |2

n = -¥

¥

If we require L2 norm to converge, however for large | m |,

å |C

m

(n) |2 must be zero. That is Cm (n) = 0 " n.

n = -¥

Hence, movement towards left implies movement towards trivial subspace {0}. {ÇVm }mÎ = {0} One important point to note here is that trivial subspace is different then null subspace as null subspace do not contain any element. We say that a set of functions { f1 , f2 , f3 ,..., fk ,...} span a whole space if any function in that space can be represented by the linear combination of these functions. What is function f (t ) and how does its integer translates span V0? We may consider function f (t ) as shown in Fig. 2.12. Any function in V0 can be expressed in the form

å C f (t - n ) n

nÎ

Φ(t) 1

0

0

1

t

2

Function f (t )

Figure 2.12

where Cn is piecewise approximation constants and f (t - n) are the integer translates of f (t ). Figure 2.13 shows a function belonging to V0. It can be expressed in terms of translates of f (t ) as shown below: 1.5

1.3

1 0.7

0.6

0.5 0.2 0 −0.4 −0.5 −1

Figure 2.13

0

1

2

3

4

Example of a function belonging to V0

56

Multiresolution and Multirate Signal Processing

0.2f (t + 1) + 0.7f (t ) - 0.4f (t - 1) + 0.6f (t - 2) + 1.3f (t - 3) Hence any space Vm can be similarly constructed using a function f (2 m t ). Vm = span{f (2 m t - n)}n,mÎ f (t ) is called as scaling function(Haar MRA), also known as ‘Father function’. The ladder of subspaces ...V-2 Ì V-1 Ì V0 Ì V1 Ì V2 ... with these properties is called as Multi-Resolution Analysis (MRA).

2.5

Axioms of MRA R There exists a ladder of subspaces, ...V-2 Ì V-1 Ì V0 Ì V1 Ì V2 ... such that 1. 2. 3. 4. 5. 6.

2.6

{ÈVm }mÎ = L2 () {ÇVm }mÎ = {0} There exists f (t ) such that, V0 = span{f (t - n)}nÎ {f (t - n)}nÎ is an orthogonal set. If f (t ) Î Vm then f (2- m t ) Î V0 , "m Î  If f (t ) Î V0 then f (t - n) Î V0 , "n Î 

Theorem of MRA m R Given the axioms, there exists a y (×) Î L2 (), so that {y (2 t - n)}mÎ,nÎ spans the L2 ().

The wavelet function Y (×) is also called as ‘Mother function’.

Example 2.6.1—Spanning spaces.

In this illustration we will consider the vector space V j spanned by the discrete scaling functions {f (2 j x - k )}, we use this span of the vector space to project any function say f ( x ) such that f j ( x ) Î V j . The projected function can be mathematically encoded as, j

f j ( x ) = åa j ,k 2 2 f (2 j x - k )

(2.1)

k

In this equation, a j ,k depict the approximation values as it derives the components from the low pass scaling equation as given in equation,

57

The Haar Wavelet

j

¥

a j ,k = ò f ( x )2 2 f (2 j x - k )dx

(2.2)

-¥

where the set {f (2 j x - k )} constitutes the basis. This f j ( x ) represents “approximation” of the signal f ( x ). In most of the denoising applications this finds it’s scope, and this set is orthogonal on (-¥, ¥) with respect to the translation, i.e.,

ò

¥

-¥

2 j f (2 j x - k )f (2 j x - k ¢) = 0, k ¹ k ¢

Also,

ò

¥

f 2 (2 j x - k )dx =

-¥

1 2j

(2.3)

since,

ò

¥

1 2j 1 = j 2

f 2 (2 j x - k )dx =

-¥

1 dx 2j ¥ 1 2 ò-¥f ( y - k )dy = 2 j

ò

¥

f 2 (2 j x - k )

-¥

¥

(2.4) (2.5) j

where we used the change of variable y = 2 j x - k and the fact that ò f 2 ( x )dx = 1. So, the factor 2 2 is -¥

j

for the normalization of f (2 j x - k ) to be 2 2 f (2 j x - k ), in order that we have, j j ì0, k ¹ k ¢ j 2 2 2 (2 )2 f x k f (2 j x - k ¢)dx = í ò-¥ î1, k = k ¥

(2.6)

We mention that the orthogonality of the scaling functions on (-¥, ¥) with respect to translations, is a very basic requirement in the (usual) definition of the multiresolution analysis. We know that V j Î V j +1 as depicted in Fig. 2.14, and that V j +1 has better refinement than V j. This “difference” (as only a refinement and not an approximation like f j ( x ) in (2.1) of the signal!) is a subset of V j +1 spanned by the discrete wavelets of the subspace Wj , g j ( x ) Î Wj , j

g j ( x ) = åb j ,k 2 2y (2 j x - k )

(2.7)

k j

¥

b j ,k = ò f ( x )2 2y (2 j x - k ) -¥

(2.8)

The basis {Y (2 j x - k )} of this vector space Wj are always orthogonal to the scaling functions {f (2 j x - k )} of V j on (-¥, ¥),

ò

¥

-¥

j

j

2 2 f (2 j x - k )2 2y (2 j x - k ¢)dx = d k ,k ¢

(2.9)

This relation resembles the orthogonality of the cosine functions to the sine functions of the Fourier series.

58

Multiresolution and Multirate Signal Processing

An example again is that of Haar function and their associated wavelets, where they are orthogonal even when k = k ¢, as can be seen from Fig. 2.15, where their product is, for example f ( x - 1)y ( x - 2) or f ( x - 1)y ( x - 1), is zero on (-¥, ¥). With this orthogonality of the basis of V j to those of Wj , we have Wj as the orthogonal complement of V j in V j +1, V j +1 = V j Å Wj. This can be done in the same way for V j = V j -1 Å Wj -1, whereas V j +1 = V j -1 Å Wj -1 Å Wj. Then this process can be continued until we reach a coarse (or blurred) resolution, for example, such that of V0 with 1 scale 0 = 1, 2 V j +1 = V0 Å W0 Å W1 Å W2 Å ... Å Wj -1 Å Wj (2.10) Figure 2.15 shows schematically this relation with the blurred approximation in V0 of scale l0 = 1, and the refinements that are added to it by W0 , W1, W2 and W3. We must note that, for example, V1 and W1 are two subspaces in V2, where the W2 basis span the difference between V2 and V1. In the next illustration (Examples 2.6.2 and 2.6.3) we show the decomposition f0 ( x ) Î V0 of the function f ( x ) = x, 0 < x < 3 as a rough approximation of f ( x ), then wrote its decomposition (refinement) as g0 ( x ) Î W0. We showed that the latter decomposition g0 ( x ) is not an approximation to the function f ( x ) = x, 0 < x < 3, but it represented a helping hand to f0 ( x ) Î V0 at the scale l0 = 1 where it added to it a bit of a refinement to advance this approximation f0 ( x ) to the more refined one 1 f1 ( x ) Î V1 at the smaller scale l1 = . This is the essence of W0 being the orthogonal complement of V0 2 in V1. Both V0 and W0 are subspaces of V1 and V1 = V0 Å W0 , where the basis of W1 span the difference between V1 and V0. Another way of putting this is that, for having the refinement at the lower scale 1 1 l1 = , that help us go from V0 at scale l0 = 1 to V1 at the scale , V0 Î V1, we can go to the basis of W0 2 2 at the same scale l0 = 1 to span the difference between the spaces V1 and V2. This alternative means that while we work with the scale l0 = 1 for V0 and W0 , we are getting the equivalent refinement of 1 being with the smaller scale l1 = in V1. Well, the credit must go to the little details (refinement) in 2 the wavelets series (2.8) versus its associated scaling function series of (2.2) for j = 0. This can be seen clearly, where the Haar basic wavelet, ì ï1, ï ï Y (t ) = í-1, ï ï0, ï î

0£ x
and < f (×),y (×) > . Now, Haar scaling function is shown in Figure 2.21 and readers can remember that the frequency representation of a box type function is sync. Thus, Haar f( x ) produces frequency behaviour of low pass cos x on (-p , p ), while Haar y ( x ) as shown in Fig. 2.20 produces frequency behaviour of high pass sin( x ) on (-p , p ). For scale j = 0 and translation k = 0, coefficient a 0,0 will be obtained from dot product < f ( x ), f ( x ) > and b0,0 from < f ( x ), y ( x ) >. As cos(w ) |w =0 = 1 and sin(w ) |w =0 = 0 , < f (⋅), f (⋅) > indicates frequency content of DC, i.e. zero frequency and < f (⋅), y (⋅) > indicated the frequency contents of unit frequency. IR Simpler interpretation of this system will be as follows: For a unit continuous function say f(x) on [0,1], ¥

1

-¥

0

< f ( x ), f ( x ) >= ò f ( x ) × f ( x ) × dx = ò f ( x ) × 1 × dx

(2.13)

Equation (2.13) clearly depicts a ‘mean’ or an ‘average’ or ‘zeroth statistical moment’ of f(x). Thus, < f ( x ), f ( x ) > gives ‘coarse’ or ‘blurred’ or ‘approximated’ or ‘low pass’ projection of f(x) on [0,1]. In case of wavelet basis, ¥

< f ( x ),y ( x ) > = ò f ( x ) × y ( x ) × dx -¥ 1

1

= ò 2 f ( x ) × 1 × dx - ò1 f ( x ) × 1 × dx 0

(2.14)

2

Equation (2.14) clearly depicts a ‘difference’ or ‘gradient’, or ‘derivative effect’ to f(x). Thus < f ( x ),y ( x ) > gives ‘finer’ or ‘sharpened’, or ‘refined’ or ‘high pass’ projection of f(x) on [0,1]. Thus where wavelet analysis is very different than a quantizer like successive approximation for example. While < f (×), f (×) > gives approximated values, < f (×),y (×) > brings in the refinement and ability to add details to improve the resolution, in signal analysis. Blurring of < f (×),y (×) > is connected with low pass filtering and detailing of < f (×),y (×) > is connected with high pass filtering! This why in DSP we focus on filter design, in wavelet analysis we focus on design of filter ‘banks’! Prior to advent of wavelet theory, electrical engineers used conventional techniques like fourier transform for signal analysis. The measure of frequency content was captured in Fourier coefficients, 1 1 an = < f ( x ), cos n( x ) >, bn = < f ( x ), sin n( x ) >, −p ≤ x ≤ p p p Also, for f ( x ) Î -¥, ¥, Fourier transform is 1 F (w ) = < f ( x ), e − jwx > = < f ( x ), e − j 2pfx > 2p where,‘ w ’ is frequency in radians per second and ‘f ’ is frequency in hertz (cycles/second). In the typical sense of ‘highpass filtering’,for a given cut-off frequency w c, all the frequencies less than w c are blocked and higher than w c are passed in a given signal. Similarly a typical low pass filter passes all the frequencies lower than a cut-off of w c and blocks all the frequencies greater than w c . Thus, it makes sense to use high pass philosophy to implement < f (×),y (×) > while

61

The Haar Wavelet

searching for “details” in the signal, and use low pass philosophy to implement < f (×), f (×) > while trying to derive “approximations” from signal. Implementation strategy: The critical point is how do we implement this moving from one resolution to another? Let us say we have projected signal (with unit range (0,1)) in V1 such that f1 Î V1 and 1 1 scale = 1 = . We approximate the signal in V1 with scaling functions {f1,0 , f1,1} and the correspond2 2 ing approximation coefficients {a1,0 , a1,1} (Readers should note, as we are in V1 , j = 1, and since we 1 wish to span unit range and scale is , we will require two translations; k = 0 & k = 1). 2 Thus f1 Î V1 gets approximated by {f1,0 , f1,1} = 2f (2 x - 0), 2f (2 x - 1) . To add in details we

{

calculate g1 ( x ) Î W1 , which is captured using {y 1,0 ,y 1,1} =

{

}

}

2y (2 x - 0), 2y (2 x - 1) and gets

calculated through coefficients {b1,0 , b1,1}. It goes without saying that {a1,0 , a1,1 } is associated with low frequencies and {b1,0 , b1,1} is associated with higher frequencies. Schematically this is depicted in Fig. 2.16. If the signal has ‘n’ samples, then two parallel filters ( f :low,y :high) add up to produce ‘2n’ samples. To maintain same number of samples, we drop every alternate sample at the output of each filter. Thus ‘n ’ samples produces by every filter adds 2 up to maintain ‘n’ samples. This process of reducing the samples to half is called ‘down sampling’ and is depicted by ¯ ; in this particular case ‘down sampling by 2’ depicted by ¯ 2 . From Fig. 2.16, we have averages associated with coefficients {a1,0 , a1,1} of scaling function ÎV1 , and low pass filter only gives ‘blurred’ picture of average associated with a 0,0 of f0,0 from lower subspace V0. The details are captured by b0,0 of wavelet y 0,0 of sub-space W0 . What this leads to in essence is V1 = V0 Å W0. What we have shown in Fig. 2.16 is called as ‘analysis’ or ‘decomposition’ of signal and what is depicted in Fig. 2.17 is called as ‘synthesis’ or ‘reconstruction’ of signal. In decomposition we always move to lower subspaces (e.g. from V1 ® V0 , V1 ® W0 ) and in reconstruction we always move towards higher subspaces (e.g. from V0 and W0 to V1 ). The two processes above are duals of each other.If ‘decomposition’ demands downsampling to match up samples,‘reconstruction’ demands upsampling for the same purpose.In upsampling we insert samples every alternate index and this is called ‘upsampling by factor of 2’, represented by ↑2 . A combination of these two is depicted in Fig. 2.18. Together the two operations constitute to what is called ‘quadrature mirror filter pair’. Figure 2.18 depicts decomposition (analysis) as well as reconstruction (synthesis) of signal f2 ( x ) Î V2 via quadrature mirror filter pairs, thus leading us to ‘2-band filter bank structures’. The first decomposition stage expresses V2 = V1 Å W1 , and second decomposition stage expresses V1 = V0 Å W0 . Thus, V2 = V0 Å W0 Å W1. This means projections in V2 are formed by approximations in V0 with scale =1, orthogonally added to projections of f(x) onto wavelet ‘1 ’ spaces W0 and W1 with scales of ‘1’ and respectively. The analysis can go deeper, e.g. f6 ( x ) Î V6 , 2

62

Multiresolution and Multirate Signal Processing

and V6 = V0 Å W0 Å W1 Å W2 Å W3 Å W3 Å W5 , with six low and high pass filter pairs for decomposition and same number for reconstruction. W0 β0.0 ψ0.0 H

V1 α1.0α1.1 φ1.0φ1.1

Figure 2.16

W1 α0.0 φ0.0

L

The outputs f0 (t ) Î V0 and g0 (t ) Î W0 of the low and high pass filters respectively (after down sampling) β0.0 ψ0.0

α1.0α1.1 φ1.0φ1.1

β0.0 ψ0.0

Figure 2.17

The reconstruction part of the (quadrature mirror) filter pairs to implement the scaling function wavelets reconstruction of the signal f1 ( x ) Î V1. W1 β0.0′β1.1 ψ0.0′ψ1.1 V2

Figure 2.18

W0 β0.0 ψ0.0

V1

V0

V1

α1.0α1.1 φ1.0φ1.1

α0.0 φ0.0

α1.0α1.1 φ1.0φ1.1

V2 α2.0′α2.2′α2.3 φ2.0′φ2.1′φ2.2′φ2.3

Decomposition and reconstruction of f2 ( x ) Î V2 – A special case of a “quadrature mirror filter pairs”.

63

The Haar Wavelet

5 4.5 4 3.5 f(x)

3 2.5 2 1.5 1 0.5 0

0

0.5

1

1.5

2

2.5 X

Figure 2.19

3

3.5

4

4.5

5

Signal f(x)

Example 2.6.3—Zoom-in and zoom-out.

In this chapter we have seen that wavelet transform decomposes signal into two separate series. One is a single series that represents coarse version, which leads to scaling function, popularly known as the father function. Other is a double series that represents a refined version, which leads to wavelet function, popularly known as mother wavelet. We also got familiarized with the idea of nested subspaces, an MRA and two band filter bank to realize an MRA. The major problem regarding an MRA was that while analyzing a signal we move down the ladder in an MRA. However, in many applications moving up the ladder is desired. Thus, an alternative framework is needed. So let us start defining that framework. The concept of MRA is in fact based on the idea of nested subspaces. This gives the entire wavelet structure ability to ‘zoom-in’ and ‘zoom-out’ of any signal under analysis. We can bring about this essence using two mathematical equations. 1. Equation of scaling function f (t ) = 2 åhkf (2t - k )

(2.15)

k

In the above equation we can clearly see that f (t ) belongs to subspace V0. Similarly, we can see that f (2t - k ) belongs to subspace V1. Thus the idea of nested subspaces is captured in the above equation. 2. Equation of Wavelet function y (t ) = 2 ågkf (2t - k )

(2.16)

k

Similar to the case seen above we can see that y (t ) belongs to subspace W0 and f (2t - k ) belongs to V1. From the above two equations we can see that, Vj = Vj - 1 Å Wj - 1

64

Multiresolution and Multirate Signal Processing

Above equation leads us again to the idea of subspaces. If we start at V0 and go on adding details then at some stage we would be able to go tantalizingly close to the signal. Let us now see how to achieve that, Let a function be, f j ( x ) Î V j , scale =

1 . 2j

To span these space V j the basis function would be 2 j /2 f (2 j x - k )k j /2 Here k is the translational parameter and 2 is the normalizing factor to convert orthogonal basis into an orthonormal basis. Thus we can write, f j ( x ) = å(a j ,k 2 j /2 f (2 j x - k )) k

Alpha can be calculated by as, +¥

a j ,k = ò f j ( x )2 j /2 f (2 j x - k )dx -¥

Similarly, for W subspaces, g j ( x ) Î Wj , scale =

1 2j

The basis would now be, 2 j /2y (2 j x - k )k Also we can write, g j ( x ) = å( b j ,k 2 j /2y (2 j x - k )) k +¥

b j ,k = ò g j ( x )2 j /2y (2 j x - k )dx -¥

b values here would give us the details, which are required to move from one subspace to another. Now, to understand how the framework so designed is useful in moving up the ladder let us consider a problem. Consider a signal, ì x 0 £ x £ 3, f ( x) = í î0 elsewhere The function f ( x ) is shown in Fig. 2.19. Objectives: 1. To find f0 ( x ) Î V0 2. To find g0 ( x ) Î W0

65

The Haar Wavelet

3. To find f1 ( x ) Î V1 by f1 ( x ) = f0 ( x ) Å g0 ( x ). This would be moving up the ladder since we are getting f1 ( x ) from f0 ( x ) and g0 ( x ). 4. Ultimately to prove V0 Å W0 = V1 1 1 Firstly let us find f0 ( x ) Î V0, j = 0 Scale = j = 0 = 1. In, (2 j /2 f (2 j x - k ))k , 2 2 if we put j = 0 the basis function would be, y ( x - k )k . The projection would be, f0 ( x ) = å k =0(a 0,kf ( x - k )) 2

For k = 0, +¥

a 0,0 = ò f0 ( x )f ( x )dx -¥

The interesting thing to note is, here we can choose our own scaling function f which is not the case with Fourier or any other conventional transforms discussed in the last lecture. In the conventional transforms the basis function is fixed and is exponential. 1 1/2

1

−1

Figure 2.20

Haar wavelet function

1 0

Figure 2.21

1

Haar scaling function

Here we would be using Haar scaling and wavelet functions shown in Fig. 2.20, and Fig. 2.21. For k = 0 the equation would result in, 1

a 0,0 = ò xdx 0

a 0,0

x2 = 2

a 0,0 =

1 2

1

0

66

Multiresolution and Multirate Signal Processing

Correspondingly, 2

a 0,1 = ò xf ( x - 1)dx 1

2

a 0,1 = ò xdx 1

a 0,1 =

3 2

Similarly, 3

a 0,2 = ò xf ( x - 2)dx 2

3

a 0,2 = ò xdx 2

a 0,2 =

5 2

Now we can write, 1 3 5 f0 ( x ) = f ( x ) + f ( x - 1) + f ( x - 2) 2 2 2

(2.17)

This is how we can find out projections of signal f in space V0. The coefficients of f0 ( x ) represent the approximations of the signal. This can be seen in Fig. 2.22.

f(x)

5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

a0,2 a0,1 a0,0 0

0.5

1

1.5

Figure 2.22

2

2.5 x

3

3.5

4

4.5

5

Plotting of f0 ( x ) coefficients

Now the next task is to find the projections g0 ( x ) of the signal on W0 . Let, g0 ( x ) Î W0, scale =1. The basis function would be, 2 j /2y (2 j x - k )k. We will again make use of Haar wavelet function. Thus as j = 0, the basis function would now reduce to, y ( x - k )k . Thus we can write,

67

The Haar Wavelet

g0 ( x ) = å( b 0,ky ( x - k )) k +¥

b 0,k = ò g0 ( x )y ( x - k )dx -¥

As the projections are taken in W0 subspace these b values will now represent all the details. Similar to the case of a values, from above equation, we can find out corresponding b values as follows, +¥

b 0,0 = ò g0 ( x )y ( x )dx -¥ 1

b 0,0 = ò xy ( x )dx 0

1/2

1

0

1/2

b 0,0 = ò (1) xdx + ò x(-1)dx b 0,0 b 0,0

1 1 1 = - + 8 2 8 -1 = 4 2

b 0,1 = ò xy ( x - 1)dx 1

3/2

2

b 0,1 = ò (1) xdx + ò x(-1)dx 1

b 0,1 =

3/2

-1 4

Similarly, b(0,2) =

-1 4

Thus we can write, g0 ( x ) = b 0,0y ( x ) + b 0,1y ( x - 1) + b 0,2y ( x - 2) g0 ( x ) =

-1 -1 -1 y ( x ) + y ( x - 1) + y ( x - 2) 4 4 4

(2.18)

-1 . We will discuss that later in the chapter. 4 The projections g0 ( x ) are plotted along f0 ( x ) in Fig. 2.23. Hence, we can see that all the b s have the same value

68

Multiresolution and Multirate Signal Processing

a0,2 a0,1

f(x)

4 3.5 3 2.5 2 1.5 1 a 0,0 0.5 0 −0.5 b0,0 −1 −1.5 −2 0 0.5

Figure 2.23

b0,1

1

b0,2

1.5

2 x

2.5

3

3.5

4

Plotting of g0 ( x ) and f0 ( x ) coefficients

1 1 = . 2j 2 The basis function would now change to, 2 j /2 f (2 j x - k )k. As j = 1, 2f (2 x - k )k Correspondingly, we can write,

Now let us find out projections on V1 subspace. Let, f1 ( x )eV1, j = 1, scale =

5

f1 ( x ) = å(a 1,k 2f (2 x - k )) k =0

At this point the a values obtained from above equation should match numerically with the values obtained by orthogonally adding a values in V0 with b values in W0 . If these values match, we can say that the framework so developed is definitely working and would thus be able to move up the ladder. So let us now calculate a 1,k values. +¥

a 1,0 = ò f1 ( x ) 2f (2 x )dx -¥

1/2

a 1,0 = ò f1 ( x ) 2f (2 x )dx 0

1/2

a 1,0 = ò f1 ( x ) 2f (2 x )dx 0

From Fig. 2.24, 1/2

a 1,0 = 2 ò x(1)dx 0

a 1,0 a 1,0

1 = 2 8 1 = 4 2

69

The Haar Wavelet

1 0 1/2 1

Figure 2.24

Plotting of function f (2 x )

From Fig. 2.23, we can calculate a 1,0 as, a 1,0 = a 0,0 + b 0,0

1 1 2 4 1 = 4

a 1,0 = a 1,0

This matches the previously calculated value with an exception of 2 which would eventually get canceled, since we have normalized the basis basis functions. This is seen with the help of following: 5

f1 ( x ) = å(a 1,k 2f (2 x - k )) k =0

For k = 0, f1 ( x ) = a 1,0 2f (2 x ) f1 ( x ) =

1 4 2

2f (2 x )

1 f1 ( x ) = f ( x ) 4 Similar to a 1,0 we can calculate other a values. 3 4 5 a 1,2 = 4 7 a 1,3 = 4 9 a 1,4 = 4 11 a 1,5 = 4 Thus, we can conclude that approximations obtained in V1 are better than those obtained in V0. Hence, this is a mechanism through which we can not only think of moving up the ladder but also think of making the choice of the scale and translation parameter and then specifically zoom on to a particular point in the signal or function, which is of greater importance. This framework enables the analyzer to either zoom-in onto a specific part of the signal or zoom-out to understand the big picture. This bigger picture can be illustrated in Fig. 2.25. a 1,1 =

70

Multiresolution and Multirate Signal Processing

a0,2 a0,1

f(x)

4 3.5 3 2.5 2 1.5 1 a 0,0 0.5 0 −0.5 b0,0 −1 −1.5 −2 0 0.5

Figure 2.25

b0,1

1

b0,2

1.5

2 x

2.5

3

3.5

4

Plotting of a values of f1 ( x ) [Note - The thick black lines are a values of f1 ( x )]

The following MATLAB code presents the analysis carried out in the previous illustration. Example 2.6.4—MATLAB code: Fourier for non-stationary signals.

71

The Haar Wavelet

The saved function can be opened in the wavelet toolebox of MATLAB by typing wavemenu on the command prompt and loading the test signal and doing analysis using wavelet of the reader’s choice. The output is shown in the Fig. 2.26. Multi-Resolution Example 4 3.5 f1(x)∈V1 f0(x)∈V0 α1,5 3 f(x) 2.5 α 0,2 α1,3 2 α1,4 1.5 α1,1 α0,1 g0(x)∈W0 1 α α1,2 0,0

0.5 0 −0.5 −1

Figure 2.26

α1,0

0

0.5

β0,1

β0,0 1

1.5

β0,2 2

2.5

3

3.5

4

MRA exercise. Black line indicates the function f ( x ), dotted line projections indicate f0 ( x ) Î V0, dash-dash line projections indicate g0 ( x ) Î W0, red projections indicate f1 ( x ) Î V1

72

Multiresolution and Multirate Signal Processing

Question:Use Haar wavelet transform on following signal and solve given objectives: 5 4 3 2 1 0 0

1

2

3

4 5 8−x,4 ≥ x ≥ 8

6

7

8

9

0, otherwise 1. f0 ( x ) Î v0 2. g0 ( x ) Î w0 3. f1 ( x ) Î v1 4. Prove: v1 = v0 Å w0 Answer: To solve this problem,we are going to use haar wavelet. The scaling function of Haar wavelet is: f j ( x ) Î v j , wa =

1 2j

ì j ü span í2 2 f (2 j x - k )ý î þ f j ( x ) = åa j ,k 2 j f (2 j x - k ) k

j

¥

a j ,k = ò f j ( x )2 2 f (2 j x - k ) -¥

f(x) 2 1 0

x 0

(2.19)

2

1

2

(2.20)

73

The Haar Wavelet

The dilation function of Haar wavelet is: g j ( x ) Î w j, wa =

1 2j

ü ì j span í2 2y (2 j x - k )ý þ î j

g j ( x ) = åbj, k 2 2y (2 j x - k )

(2.21)

k j

¥

bj, k = ò g j ( x )2 2y (2 j x - k ) -¥

(2.22)

y(x) 2 1 0

x 2

1

−1 −2

1] f0 ( x ) Î v0 \ j = 0 so wa =

1 =1 20 ü ì 0 span í2 2 f (20 x - k )ý þ î span {f ( x - k )}

From (2),when we put j=0 we get 0

f0 ( x ) = åa 0,k (2) 2 f (20 x - k ) k

= åa 0,kf ( x - k )

(2.23)

k

a 0,k

0

¥

= ò f0 ( x )2 2 f (20 x - k ) -¥ ¥

(2.24)

= ò f0 ( x )f ( x - k ) -¥

Putting k = 0 in Eq. 2.24, we get a 0,0

¥

= ò f0 ( x )f ( x - 0)dx -¥ ¥

= ò f0 ( x )f ( x )dx -¥

(2.25)

74

Multiresolution and Multirate Signal Processing

But, f(x) 2 1 x

0 0

1

2

So, scaling function f ( x ) varies from 0 to 1. 1

= ò ( x )(1)dx 0

1

é x2 ù =ê ú ë 2 û0 =

1 2

\a 0,0 =

1 2

Putting k=1 in Eq. 2.24, we get ¥

= ò f0 ( x )f ( x - 1)dx

a 0,1

-¥ ¥

= ò f0 ( x )f ( x - 1)dx -¥

But, f(x−1) 2 1 0

x 0

1

2

So scaling function f ( x - 1) varies from 1 to 2. 2

= ò ( x )(1)dx 1

2

é x2 ù =ê ú ë 2 û1

3

(2.26)

75

The Haar Wavelet

=

3 2

\a 0,1 =

3 2

Putting k = 2 in Eq. 2.24, we get a 0,2

¥

= ò f0 ( x )f ( x - 2)dx -¥ ¥

= ò f0 ( x )f ( x - 2)dx

(2.27)

-¥

But, f(x−2) 2 1 0

x 0

1

2

3

4

So scaling function f ( x - 2) varies from 2 to 3. 3

= ò ( x )(1)dx 2

3

é x2 ù =ê ú ë 2 û2 =

9 4 2 2

=

5 2

\a 0,2 =

5 2

For k = 3, a 0,3

¥

= ò f0 ( x )f ( x - 3)dx -¥ ¥

= ò f0 ( x )f ( x - 3)dx -¥

(2.28)

76

Multiresolution and Multirate Signal Processing

But, f(x−3) 2 1 x

0 0

1

2

3

4

5

So scaling function f ( x - 3) varies from 3 to 4. 4

= ò ( x )(1)dx 3

4

é x2 ù =ê ú ë 2 û3 =

16 9 2 2

=

7 2

\a 0,3 =

7 2

For k = 4, a 0,4

¥

= ò f0 ( x )f ( x - 4)dx -¥ ¥

= ò f0 ( x )f ( x - 4)dx

(2.29)

-¥

But, f(x−4) 2 1 x

0 0

1

2

3

So scaling function f ( x - 4) varies from 4 to 5. 5

= ò (8 - x )(1)dx 4

4

5

6

77

The Haar Wavelet 5

é x2 ù = ê8 x - ú 2 û4 ë =

96 89 2 2

=

7 2

\a 0,4 =

7 2

For k = 5, a 0,5

¥

= ò f0 ( x )f ( x - 5)dx -¥ ¥

= ò f0 ( x )f ( x - 5)dx

(2.30)

-¥

But, f(x−5) 2 1 x

0 0

1

2

3

4

So scaling function f ( x - 5) varies from 5 to 6. 6

= ò (8 - x )(1)dx 5

6

é x2 ù = ê8 x - ú 2 û5 ë =

121 116 2 2

=

5 2

\a 0,5 =

5 2

5

6

7

78

Multiresolution and Multirate Signal Processing

For k = 6, a 0,6

¥

= ò f0 ( x )f ( x - 6)dx -¥ ¥

= ò f0 ( x )f ( x - 6)dx

(2.31)

-¥

But, f(x−6) 2 1 0

0

1

2

3

4

5

6

7

8

So scaling function f ( x - 6) varies from 6 to 7. 7

= ò (8 - x )(1)dx 6

7

é x2 ù = ê8 x - ú 2 û6 ë =

148 145 2 2

=

3 2

\a 0,6 =

3 2

For k = 7, a 0,7

¥

= ò f0 ( x )f ( x - 7)dx -¥ ¥

= ò f0 ( x )f ( x - 7)dx

(2.32)

-¥

But, f (x − 7) 2 1 0

x 0

1

2

3

4

5

6

7

8

9

79

The Haar Wavelet

So scaling function f ( x - 7) varies from 7 to 8. 8

= ò (8 - x )(1)dx 7

8

é x2 ù = ê8 x - ú 2 û7 ë =

177 176 2 2 =

1 2

\a 0,7 =

1 2

Using Eq. 2.23, we can write 1 3 5 7 7 5 3 1 f0 ( x ) = f ( x ) + f ( x - 1) + f ( x - 2) + f ( x - 3) + f ( x - 4) + f ( x - 5) + f ( x - 6) + f ( x - 7) 2 2 2 2 2 2 2 2 5 4 3 2 1 0 0

1

2

3

4

5

6

7

8

9

The grey line is showing Wavelet scaling coefficients. 2]g0 ( x ) Î g0 \ j = 0 so wa =

1 =1 20 ì 0 ü span í2 2 y (20 x - k ) ý î þ span {y ( x - k )}

(2.33)

80

Multiresolution and Multirate Signal Processing

From Eqs. 2.21 and 2.22, when we put j = 0 0

f0 ( x ) = åb 0,k (2) 2y (20 x - k ) k

= åb 0,ky ( x - k )

(2.34)

k

b 0,k

0

¥

= ò f0 ( x )2 2y (20 x - k ) -¥ ¥

= ò f0 ( x )y ( x - k ) -¥

(2.35)

For k = 0, we have b 0,0

¥

= ò f0 ( x )y ( x - 0)dx -¥ ¥

= ò f0 ( x )y ( x )dx -¥

But, y(x) 2 1 x

0 1

2

−1 −2

So dilation function y ( x ) varies from 0 to 1. 1

1

= ò 2 ( x )(1) + ò1 ( x )(-1)dx 0

2

1 2

1

é x2 ù é x2 ù = ê ú + ê- ú ë 2 û0 ë 2 û 1 2

1 1 = 4 2 =-

1 4

(2.36)

81

The Haar Wavelet

1 4

\ b 0,0 = For k = 1, we have

¥

= ò f0 ( x )y ( x - 1)dx

b 0,1

-¥ ¥

= ò f0 ( x )y ( x - 1)dx

(2.37)

-¥

But, y(x − 1) 2 1 x

0 1

2

3

−1 −2

So dilation function y ( x - 1) varies from 1 to 2. 3

2

= ò 2 ( x )(1) + ò3 ( x )(-1)dx 1

2

3 2

2

é x2 ù é x2 ù = ê ú + ê- ú ë 2 û1 ë 2 û 3 2

=

18 20 8 8

=-

1 4

\ b 0,1 = -

1 4

For k = 2, we have ¥

b 0,2 = ò f0 ( x )y ( x - 2)dx -¥

82

Multiresolution and Multirate Signal Processing

But, y(x − 2) 2 1 x

0 1

2

3

4

−1 −2

So dilation function y ( x - 2) varies from 2 to 3. 5 2

3

= ò ( x )(1) + ò ( x )(-1)dx 5 2

2

5

3

é x2 ù 2 é x2 ù = ê ú + ê- ú ë 2 û2 ë 2 û 5 2

50 52 8 8 1 =4 =

\ b 0,2 = -

1 4

For k = 3, we have ¥

b 0,3 = ò f0 ( x )y ( x - 3)dx -¥

But, y(x − 3) 2 1 x

0 1 −1 −2

2

3

4

5

83

The Haar Wavelet

So dilation function y ( x - 3) varies from 3 to 4. 7

4

= ò 2 ( x )(1) + ò7 ( x )(-1)dx 3

2

7

4

é x2 ù 2 é x2 ù = ê ú + ê- ú ë 2 û3 ë 2 û 7 2

=

98 100 8 8

=-

1 4

\ b 0,3 = -

1 4

For k = 4, ¥

b 0,4 = ò f0 ( x )y ( x - 4)dx -¥

But, y(x − 4) 2 1 x

0 1

2

3

4

5

−1 −2

So dilation function y ( x - 4) varies from 4 to 5. 9

5

= ò 2 ( x )(1) + ò9 ( x )(-1)dx 4

2

9 2

5

é x2 ù é x2 ù = ê ú + ê- ú ë 2 û4 ë 2 û 9 2

=

15 13 8 8

6

84

Multiresolution and Multirate Signal Processing

=

1 4

\ b 0,4 =

1 4

For k = 5, we have ¥

b 0,5 = ò f0 ( x )y ( x - 5)dx -¥

But, y(x − 5) 2 1 x

0 1

2

3

4

5

6

−1 −2

So dilation function y ( x - 5) varies from 5 to 6. 11

6

= ò 2 ( x )(1) + ò11( x )(-1)dx 5

2

11

6

é x2 ù 2 é x2 ù = ê ú + ê- ú ë 2 û 5 ë 2 û 11 2

11 9 = 8 8 =

1 4

\ b 0,5 =

1 4

For k = 6, we have ¥

b 0,6 = ò f0 ( x )y ( x - 6)dx -¥

7

85

The Haar Wavelet

But, y(x − 6) 2 1 x

0

2

1

3

4

5

6

7

8

−1 −2

So dilation function y ( x - 6) varies from 6 to 7. 13

7

= ò 2 ( x )(1) + ò13 ( x )(-1)dx 6

2

13 2

7

é x2 ù é x2 ù = ê ú + ê- ú ë 2 û 6 ë 2 û 13 2

7 5 = 8 8 =

1 4

\b0,6 =

1 4

For k = 7, we have b0,7 =

ò

¥

f ( x )y ( x - 7)dx

-¥ 0

But, y(x − 6) 2 1 x

0 1 −1 −2

2

3

4

5

6

7

8

9

86

Multiresolution and Multirate Signal Processing

So dilation function y ( x - 7) varies from 7 to 8. 15

8

= ò 2 ( x )(1) + ò14 ( x )(-1)dx 7

2

15 2

8

é x2 ù é x2 ù = ê ú + ê- ú ë 2 û 7 ë 2 û 15 2

=

3 1 8 8

=

1 4

\ b 0,7 =

1 4

Using Eq. 2.24, we can write 1 1 1 1 1 1 1 1 g0 ( x ) = - y ( x ) - y ( x - 1) - y ( x - 2) - f ( x - 3) + f ( x - 4) + f ( x - 5) + f ( x - 6) + f ( x - 7) 4 4 4 4 4 4 4 4 5 4 3 2 1 0 0

1

2

3

4

5

6

The dark grey line is showing Wavelet scaling coefficients. The light grey line is showing Wavelet dilation coefficients. 3] f1 ( x ) Î v1 \ j = 1 so wa =

1 1 = 21 2 ì 1 ü span í2 2 f (20 x - k ) ý î þ

7

8

9

87

The Haar Wavelet

{

span

}

2f (2 x - k )

From Eq. 2.19, when we put j = 1 1

f1 ( x ) = å ka 1,k (2) 2 f (21 x - k ) = å ka 1,k 2f (2 x - k ) 1

∞

a1,k = ∫ f1 ( x )2 2 f (21 x − k ) −∞ ∞

a1,k = ∫ f1 ( x ) 2f (2 x − k )

(2.38)

−∞

For k = 0, we have ¥

a 1,0 = ò f1 ( x ) 2f (2 x - 0)dx -¥

¥

= ò f1 ( x ) 2f (2 x )dx -¥

But, f(2x) 2 1 x

0 0

1

2

3

As scaling function f (2 x ) varies from 0 to 0.5. 1

= ò 2 2( x )(1)dx 0

0

é x2 ù = 2ê ú ë 2 û1 2

= 2× \a 1,0 = For k = 1, we have

1 8

1 4 2

4

88

Multiresolution and Multirate Signal Processing

a 1,1 = ò

¥

-¥

2 f1 ( x )f (2 x - 1)dx

But, f(2x − 1) 2 1 x

0 0

1

2

As scaling function f (2 x ) varies from 0.5 to 1. 0.5 1 0

=ò

2( x )(1)dx 0

é x2 ù = 2ê ú ë 2 û 0.5 1

= 2× \a 1,1 =

3 8

3 4 2

For k = 2, we have ¥

a 1,2 = ò f1 ( x ) 2f (2 x - 2)dx -¥

But, f(2x − 2) 2 1 0

x 0

1

2

3 As scaling function f (2 x - 2) varies from 1 to . 2 3

= ò 2 2( x )(1)dx 1

89

The Haar Wavelet 1

é x2 ù = 2ê ú ë 2 û3 2

é9 4ù = 2 ×ê - ú ë8 8 û = 2× \a 1,2 =

5 8

5 4 2

For k = 3, we have ¥

a 1,3 = ò f1 ( x ) 2f (2 x - 3)dx -¥

But, f(2x − 3) 2 1 0

x 0

1

2

As scaling function f (2 x - 3) varies from 1.5 to 2. 2

= ò3 2( x )(1)dx 2

3

é x2 ù 2 = 2ê ú ë 2 û2

é16 9 ù = 2 ×ê - ú ë 8 8û = 2× \a 1,3 = For k = 4, we have

7 4 2

7 8

3

90

Multiresolution and Multirate Signal Processing ¥

a 1,4 = ò f1 ( x ) 2f (2 x - 4)dx -¥

But, f(2x − 4) 2 1 0

x 0

1

2

3

4

5 As scaling function f (2 x - 4) varies from 2 to . 2 5

= ò 2 2( x )(1)dx 2

2

é x2 ù = 2ê ú ë 2 û5 2

é 25 16 ù = 2 ×ê - ú 8û ë8 = 2× \a 1,4 =

9 8

9 4 2

For k = 5, we have ¥

a 1,5 = ò f1 ( x ) 2f (2 x - 5)dx -¥

But, f(2x − 5) 2 1 0

x 0

As scaling function f (2 x - 5) varies from

1

5 to 3. 2

2

3

4

91

The Haar Wavelet 6

= ò5 2( x )(1)dx 2

3

é x2 ù = 2ê ú ë 2 û5 2

é 36 25 ù = 2 ×ê - ú 8û ë8 = 2× \a 1,5 =

11 8

9 4 2

For k = 6, we have ¥

a 1,6 = ò f1 ( x ) 2f (2 x - 6)dx -¥

But, f(2x − 6) 2 1 0

x 0

1

2

7 As scaling function f (2 x - 6) varies from 3 to . 2 7

= ò 2 2( x )(1)dx 3

3

é x2 ù = 2ê ú ë 2 û7 2

é 49 36 ù = 2 ×ê - ú 8û ë8 = 2× \a 1,4 =

13 8 13 4 2

3

4

92

Multiresolution and Multirate Signal Processing

For k = 7, we have ¥

a 1,7 = ò f1 ( x ) 2f (2 x - 7)dx -¥

But, f(2x − 7) 2 1 0

x 0

1

2

3

4

7 to 4. 2

As scaling function f (2 x - 7) varies from

4

= ò7 2( x )(1)dx 2

4

é x2 ù = 2ê ú ë 2 û7 2

é 64 49 ù = 2 ×ê - ú 8û ë8 = 2× \a 1,7 =

15 8

15 4 2

For k = 8, we have ¥

a 1,8 = ò f1 ( x ) 2f (2 x - 8)dx -¥

But, f(2x − 8) 2 1 0

x 0

1

2

3

4

5

93

The Haar Wavelet

As scaling function f (2 x - 8) varies from 4 to

9 2 9

= ò 2 2( x )(1)dx 4

4

é x2 ù = 2ê ú ë 2 û9 2

é 352 337 ù = 2 ×ê 8 úû ë 8 = 2×

\a 1,8 =

15 8

15 4 2

For k = 9, we have ¥

a 1,9 = ò f1 ( x ) 2f (2 x - 9)dx -¥

But, f(2x − 9) 2 1 0

x 0

1

2

As scaling function f (2 x - 9) varies from

3

4

9 to 5 2

5

= ò9 2(8 - x )(1)dx 2

5

é x2 ù = 2 ê8 × x - ú 2 û9 ë 2

é 401 388 ù = 2 ×ê 8 úû ë 8

5

6

94

Multiresolution and Multirate Signal Processing

= 2× \a 1,9 =

13 8

13 4 2

For k = 10, we have ¥

a 1,10 = ò f1 ( x ) 2f (2 x - 10)dx -¥

But, f(2x − 10) 2 1 0

x 0

1

2

As scaling function f (2 x - 10) varies from 5 to

3

4

11 . 2

11

= ò 2 2(8 - x )(1)dx 5

5

é x2 ù = 2 ê8 × x - ú 2 û 11 ë 2

é 452 441 ù = 2 ×ê 8 úû ë 8 = 2×

\a 1,10 =

11 8

11 4 2

For k = 11, we have ¥

a 1,11 = ò f1 ( x ) 2f (2 x - 11)dx -¥

5

6

95

The Haar Wavelet

But, f(2x − 11) 2 1 0

x 0

1

2

As scaling function f (2 x - 11) varies from

3

4

5

6

7

11 to 6. 2 6

= ò11 2(8 - x )(1)dx 2

6

é x2 ù = 2 ê8 × x - ú 2 û 11 ë 2

é 505 496 ù = 2 ×ê 8 úû ë 8 = 2× \a 1,11 =

9 8

9 4 2

For k = 12, we have ¥

a 1,12 = ò f1 ( x ) 2f (2 x - 12)dx -¥

But, f(2x − 12) 2 1 0

x 0

1

2

3

As scaling function f(2 x − 12) varies from 6 to

4

5

13 2

13

a 1,12 = ∫ 2 2(8 − fx ) ⋅ (1) ⋅ dx 6

6

7

96

Multiresolution and Multirate Signal Processing 13

  104 x2  2   169 36   = 2 8 x −  = 2  − 48 −  −    8 12  6 2   2  7  8 25  = 2 − = 2⋅ 8 2 8  = \a 1,10 =

7 4 2 11 4 2

For k = 13, ¥

a 1,13 = ò f1 ( x ) 2f (2 x - 13)dx -¥

But, f(2x − 13) 2 1 0

x 0

1

2

As scaling function f (2 x - 13) varies from

3

4

5

13 to 7. 2 7

= ò13 2(8 - x )(1)dx 2

7

é x2 ù = 2 ê8 × x - ú 2 û 13 ë 2

é 617 612 ù = 2 ×ê 8 úû ë 8 = 2× \a 1,13 =

5 4 2

5 8

6

7

8

97

The Haar Wavelet

For k = 14, we have ¥

a 1,14 = ò f1 ( x ) 2f (2 x - 14)dx -¥

But, f(2x − 14) 2 1 0

x 0

1

2

3

As scaling function f (2 x - 14) varies from 7 to

4

5

6

7

8

15 2

15

= ò 2 2(8 - x )(1)dx 7

7

é x2 ù = 2 ê8 × x - ú 2 û 15 ë 2

é 676 673 ù = 2 ×ê 8 úû ë 8 = 2× \a 1,14 =

3 8 3 4 2

For k = 15, ¥

a 1,15 = ò f1 ( x ) 2f (2 x - 15)dx -¥

¥

= ò f1 ( x ) 2f (2 x - 15)dx -¥

But, f(2x − 15) 2 1 0

x 0

1

2

3

4

5

6

7

8

9

98

Multiresolution and Multirate Signal Processing

As scaling function f (2 x - 15) varies from

15 to 8. 2 8

= ò15 2(8 - x )(1)dx 2

8

é x2 ù = 2 ê8 × x - ú 2 û 15 ë 2

é 737 736 ù = 2 ×ê 8 úû ë 8 = 2× \a 1,15 =

1 8

1 4 2

5 4 3 2 1 0 0

1

2

3

4

5

6

7

8

The dark grey line is showing Wavelet scaling coefficients. The light grey line is showing Wavelet dilation coefficients. 4] Prove: v1 = v0 Å w0 ì1 3 5 7 7 5 3 1 ü v0 = í , , , , , , , ý î2 2 2 2 2 2 2 2 þ ì 1 1 1 1 1 1 1 1ü w0 = í- , - , - , - , , , , - ý î 4 4 4 4 4 4 4 4þ ì 1 3 5 7 9 11 13 15 15 13 11 9 7 5 3 1 ü v1 = í , , , , , , , , , , , , , , , ,ý î4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 þ

9

99

The Haar Wavelet

Now, take first component of v0 and first component of w0 and find out following term, a 0,1 × f ( x ) + b 0,1 ×y (k ) The scaling function varies from 0 to 1 and the value is 1 overall interval. The dilation function varies from 0 to 1 and the value is 1 for 0 to 0.5 and −1 for 0.5 to 1. So, we will get two values using these two functions. For k = 0 \a 0,0( + ) + b 0,0 ( + ) =

1 -1 + 2 4

=

1 4

This is equal to a 1,0 Also, \a 0,0( - ) + b 0,0 ( - ) =

1 1 + 2 4

=

3 4

This is equal to a 1,1. For k = 1 \a 0,1( + ) + b 0,1( + ) =

3 -1 + 2 4

=

5 4

This is equal to a 1,2. \a 0,1( - ) + b 0,1( - )

This is equal to a 1,3.

=

3 1 + 2 4

=

7 4

100

Multiresolution and Multirate Signal Processing

Similarly, we can find values of a 0,k and b 0,k to make a conclusion that addition of these two values will a 1,k (k = Total number of coefficients). Thus, v1 is formed by combining v0 and w0. Hence,v1 = v0 Å w0 .

Exercises Exercise 2.1

Verify graphically Mexican hat wavelet function using wavelet dilation equation. Assume compact support of [0,2) for the roof scaling function. State clearly if this pair of scaling and wavelet functions can produce MRA?

Exercise 2.2

Explain which V j subspace the signal, shown in Fig. 2.27, belongs to and why?

1.4

1.3

1.2 1

1 0.8

0.7

f(x)

0.7

0.6

0.6 0.4 0.2

0.2 0 −0.2

−0.4 −0.4 0

0.25

0.5

0.75

1

1.25

1.5

x

Figure 2.27

The function f ( x ). Which V j subspace it belongs to?

Exercise 2.3

Explain {ÇVm }mÎ = {0} in detail?

1.75

101

The Haar Wavelet

Exercise 2.4

Given, ì3 - x f ( x) = í î 0

for 0, x,3, otherwise

(2.39)

Calculate: f0 ( x ) Î V0, g0 ( x ) Î W0 and f1 ( x ) Î V1, Prove: f0 ( x ) + g0 ( x ) = f1 ( x ) Î V1

Exercise 2.5

Calculate the L 2 norm of the following function x(t), where x(t) = 1- | t | in the interval [−1,1] and zero else where. Hint: Since the function is symmetric between −1 to 0 and 0 to 1. The norm will be twice that of the value caliculated in any one of the above two intervals. 1

1

0

1

Hence L2 norm of x(t ) = [2 ò | (1 - t ) |2 dt ]2 = [2 ò | (1 + t ) |2 dt ]2 = 2 / 3 0

-1

Exercise 2.6

Give an example of function which does not exist in L 2 (). Hint: All the exponential functions having geometric ratio greater than 1 does not exist in L2 (). ¥ For example consider function 2t for t > 0. Its L2 norm does not exist because é ò | 2t |2 dt ù does not ëê 0 ûú converge.

Exercise 2.7

Give some examples of functions for the following cases (a) Function that exists in L1 () but does not exist in L 2 (). (b) Function that exists in L 2 () but does not exist in L1 (). (c) Functions which does not exist in both the spaces. Hint: 1 (a) Consider the function in the interval (0,1] and 0 else where. Its L1 norm converges but L2 t norm does not converge. 1 (b) Consider the function in the interval [1,¥). Its L2 norm converges, but L1 norm diverges. t (c) All the periodic functions such as sin(t ) and cos(t ) does not exist in both L1 () and L2 ().

102

Multiresolution and Multirate Signal Processing

Exercise 2.8

Give the axioms that are to be satisfied by a vector space. Hint: A real vector space is a set X with a special element 0 called as zero vector, and three operations: Addition: Given two elements x, y in X, one can form the sum x + y, which is also an element of X. Inverse: Given an element x in X, one can form the inverse -x, which is also an element of X. Scalar multiplication: Given an element x in X and a real number c, one can form the product cx, which is also an element of X. These operations must satisfy the following axioms Additive axioms: For every x,y,z in X, we have (a) x+y = y+x. (b) (x+y)+z = x+(y+z). (c) 0+x = x+0 = x. (d) (-x) + x = x + (-x) = 0. Multiplicative axioms: For every x in X and real numbers c,d, we have (a) 0x = 0. (b) 1x = x. (c) (cd)x = c(dx). Distributive axioms: For every x,y in X and real numbers c,d, we have (a) c(x+y) = cx + cy. (b) (c+d)x = cx +dx. Scalar multiplication: Given an element x in X and a real number c, one can form the product cx, which is also an element of X.

Exercise 2.9

Why in general, only functions in L 2 () are considered for piecewise constant approximations? Hint: The L2 norm signifies the energy of a signal. In general, most of the rea-time signals have finite energy. So we are interested in the signals which have finite energy. So functions in L 2 () are considered for piecewise constant approximations.

Exercise 2.10

Show that the function f(t ) is orthogonal to its integer translates. Hint: Let, f(t - n) = 1, n < t < n + 1 = 0, otherwise f(t - m ) = 1, m < t < m + 1 = 0, otherwise

103

The Haar Wavelet

Therefore, their dot product b

ò f (t - n)f (t - m)dt = 0, for n ¹ m a

Hence {f (t - n)}nÎ are orthogonal.

Exercise 2.11

Explain the effect of dilation and translation of a function f (t ) in frequency domain. Hint: Let, f(t ) = 1, 0 < t < 1 = 0, otherwise Fourier Transform of f (t ) is given as follow, +∞ f (Ω) = ∫ f (t )e− jΩt dt −∞ 1

= ∫ e − jΩt dt 0

1

 e− jΩt  =   − jΩ 0 =

1 − e − jΩ jΩ e

− jΩ 2

= 2e

− jΩ 2

=

sin

Ω 2

Ω e

=

− jΩ  j2Ω  2 e − e     jΩ

− jΩ 2

sin

Ω 2

Ω 2

Effect of dilation and translation in fourier domain. Let f(2m t - n) = 1, 2 - m n < t < 2 - m (n + 1) = 0, otherwise

104

Multiresolution and Multirate Signal Processing

By using the scaling and shifting property of Fourier transform we obtain Fourier Transform of f (2 m t - n) as e

- jW 2- m 2

e- jWn 2

=

-m

sin(

W 2

2- m W ) 2

From the Fourier Transform of f (t ), it can be observed that f (t ) has a low pass nature and captures a frequency band around zero frequency. Dilation of f (t ) results in capturing a band of frequencies of varying bandwidth around zero frequency. However, we will see the effect of dilation and translation in detail in subsequent chapters.

Exercise 2.12

Explain the effect of dilation and translation of a function y (t ) in frequency domain. Hint: Let y (t ) = 1 0 < t < 0.5 = -1 0.5 < t < 1 = 0 otherwise Fourier Transform of y (t ) is given as follow F .T .

f (t ) ↔ f (Ω) F .T . 1 Ω f (2t ) ↔ f ( ) 2 2 F .T . 1 − j Ω Ω f (2t − 1) ↔ e 2 f ( ) 2 2 Using these relationships we get y (t ) = f (2t ) − f (2t − 1) − jΩ � (Ω) = 1 f�  Ω  − 1 e 2 f�  Ω  y     2 2  2 2 − jΩ   Ω 1 y (Ω) =  1 − e 2  f�   2   2

� (Ω) = je y

− jΩ 2

2  Ω sin    4 Ω 4

105

The Haar Wavelet

Effect of dilation and translation in Fourier domain. Let F .T .

y (t ) ↔ je

− jΩ 2

2  Ω sin    4 Ω 4

F .T .

y (t − n) ↔ e − jnΩ je

F .T .

− jΩ 2

−m

y (2 m t − n) ↔ je− jnΩ 2 e

2  Ω sin    4 Ω 4

− j 2− m Ω 2

−m Ω 2 2 sin   4  Ω 4

From the Fourier Transform of ← (t ) , it can be observed that ← (t ) has a band pass nature and captures a frequency band around certain centre frequency. Dilation of ← (t ) results in capturing a band of frequencies of varying bandwidth around a centre frequency, which itself changes. Again, we will have a detailed discussion on this in subsequent chapters. Exercise 2.13

Consider the continuous time function x(t ) defined as follow. x (t ) = 1 - t 2 - 1 < t < 1 =0 otherwise Compute the piecewise approximation over the subspace V0 and find the maximum error. Hint: The given function x(t ) is as shown in Fig. 2.28. V0 : x(t ), such that x(×) Î L2 () is piecewise constant on interval] n, n + 1 [, " n Î Z }. 0

] - 1,0 [: ò (1 - t 2 )dt -1

= [t = 1

t3 0 ]-1 3

2 3

]0,1[: ò (1 - t 2 )dt 0

106

Multiresolution and Multirate Signal Processing

= [t =

t3 1 ]0 3

2 3

1 0.9 0.8

y(t)->

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 −1 −0.8 −0.6 −0.4 −0.2

Figure 2.28

0 t->

0.2

0.4

0.6

0.8

1

The function x(t )

Calculation of maximum error: E = (1 - t 2 ) -

2 3

dE = -2t dt d2E = -2 dt 2 As second derivative is negative we get maxima so the maximum value is obtained at t=0. So the maximum value is max( E ) = (1 − 02 ) − =

1 3

2 3

Chapter

3

The Haar Filter Bank

Introduction Function and sequence Downsampler The Haar filter bank Analysis part Synthesis part Frequency domain analysis of Haar filter bank Frequency domain behaviour

3.1

Introduction

In Chapter 2, we saw an equivalence between functions and vectors. We also saw how processing of a function can be related to processing of equivalent sequence w.r.t. inner product and Parseval’s theorem. How functions can be considered as generalized vectors? Another dimension of same is replacing work with function, by work with sequence. It is possible to work with sequences in place of functions rather sequences are much easier to deal with as they can be simulated and analysed on computer like in the case of discrete time. If whatever we do with a sequence maps exactly with what we want to do with an original continuous time function, then it is an added advantage. This is true for the spaces contained in V0, in V1, in V2, and so on in the L2 () ladder shown in Fig. 3.1. {0} Ì V-2 Ì V-1 Ì V0 Ì V1 Ì V2 Ì  Basis function for V0 subspace is given by {f (t - n)}nÎ . The function f (t - n) for t = n is shown below. This is also an orthonormal basis for V0 because áf (t - n), f (t - m)ñ = 0, = 1,

n¹m n=m

where n, m Î .This is the primary condition of orthonormality of a function. 1

n

Figure 3.1

n+1

Basis function for V0

108

3.2

Multiresolution and Multirate Signal Processing

Function and Sequence

To have an idea between equivalence of function and sequence, consider the function x(t )ÎV0 written in terms of translates of f (t ). æ -3 ö æ3ö æ1ö x(t ) =  + ç ÷ f (t + 1) + ç ÷ f (t ) + ç ÷ f (t - 1) + (4)f (t - 2) +  2 4 è ø è2ø è ø There is an equivalence between x(t ) and sequence. So, the corresponding sequence is

¬

é 1 3 3 ù x[ n] = ê, , - , , 4,ú ë 2 4 2 û The downarrow represent the zero’th position of the sequence. Since the function x(t ) belongs to V0 Î L2 () thus the sequence x[n] should also belong to a set of square summable sequences. If a function x(t ) is square integrable, i.e. mable, i.e.

(å

¥ n = -¥

2

)

(ò

¥

-¥

)

2

x(t ) dt < ¥ then corresponding sequence is square sum-

x[ n] < ¥ . Here, we use a notion that x(t ) Î L2 () implies x[ n]Î l2 (). Thus, l2 ()

is a space of all the discrete sequences which are square summable. In general, l p () is a linear space of sequences such that æ ¥ ö p ç å x[ n] < ¥ ÷ è n = -¥ ø We have just shown a correspondence that if x(t )ÎV0 Î L2 () then x[ n] Î l2 (). Note: x[n] is the sequence of coefficients of expansion of x(t) with respect to an ORTHONORMAL basis of x(t). If the basis is orthonormal then there is a mapping between inner products of the function and the sequence. For example, suppose x(t ), y(t ) Î V0, then inner product in continuous time is given by ¥

á x(t ), y(t )ñ = ò x(t ) y(t )dt -¥

¥

= K0

å x[n]y[n]

(3.1)

n = -¥

where K 0 is a constant. Therefore, what we do in the context of continuous time function can be equivalently done in the context of discrete domain for corresponding sequence. So, eventually we are leaving behind the continuous function and dealing with the sequence x[ n]. For example, the concept of pixels or 2-dimensional area in case of an image shows that, we can equivalently replace this continuous intensity of the image in the pixel by 2 sequences (x[ p] and x[q ]) in 2 dimensions. If we want to get some inference from the image, we can look at these sequences and analyse them. What is the motivation behind this? Our motive is to extract an additional information while going from one subspace to another in a ladder. Now, we will see how to move from one resolution to another and extract the incremental information from the function. Note that this process corresponds to going from lower resolution subspaces to higher resolution in L2 () ladder of subspaces.

109

The Haar Filter Bank

Example 3.2.1 — Equivalence, decomposition and reconstruction.

Consider y(t ) Î V1 i.e.with intervals of length half as shown in Fig. 3.2 and let the corresponding sequence y[n] be

¬

[, 4,7,10,16,11,3, -1] The relationship between y(t) and y[n] is y(t ) = åy[ n]f (2t - n) nÎ

where

n n +1 0 is irrational and can be expanded as: -1

( z -1 )n n! n =0 ¥

ez = å

Thus, the inverse Z -transform for the above equation can be obtained as h[ n] =

1 u[ n] n!

This is a convergent sum. Though all irrational systems are not unstable, but it is impossible to realize this as it requires infinite resources. Resources here constitute adders, multipliers and delays. A rational system can be realized by finite resources. However, ideal is often not achievable. But we can go arbitrarily close to the ideal by investing more and more resources. There are several ways of doing it, i.e. the resources can be put to use in different ways. At times investing in one specific method takes us faster to the ideal that some other, but certain aspects need to be compromised for this rate.

137

M-Channel Filter Bank

H0(Z)

2

2

G0(Z)

H1(Z)

2

2

G1(Z)

x[n]

Figure 4.5

4.4

Haar analysis and synthesis filter banks

Realizable Two Band Filter Bank

Though the ideal filter bank is unachievable we can go quite close to it. We can build a two-band filter bank arbitrarily close to the ideal simply by investing more and more resources. But what does a realizable two-band filter bank look like? Let's say the realizable two-band filter bank looks like the one shown in Fig 4.5. The filter bank is realizable if H 0 ( Z ), H1 ( Z ), G0 ( Z ), G1 ( Z ) are all rational system functions. The filters must not satisfy any of the above-mentioned disqualifications. Hence the system responses H 0 ( Z ), H1 ( Z ), G0 ( Z ), G1 ( Z ) must be rational, stable, finitely causal. H 0 ( Z ), G0 ( Z ) aspire to be p ideal low pass filters with w c = and H1 ( Z ), G1 ( Z ) aspire to be high pass filters with cutoff frequency 2 p w c = as ideal filters are not achievable. 2

4.5

Relation Between Haar MRA and Filter Bank

In this section, we will try to establish a relation between the functions f(t ), y (t ) and the filter banks. Let us focus our attention on Haar MRA. As discussed earlier, the Haar MRA is not ideal. Several concepts can be learned from the shortcomings of Haar MRA, which can be utilized to construct better families of Dyadic Multiresolution analysis.

4.5.1

Relation Between the Function f(t ) and Filter Banks

For Haar MRA, f(t ) is the basis of V0 i.e f(t ) Î V0. Also, recall that there exists a ladder of spaces in MRA which states that V0 Ì V1. f(t ) should, therefore, be expressible in the basis of V1 i.e. f(2t - n) , n Î . 1

Φ(t) Φ(2t) 0

Figure 4.6

Φ(2t−1) 1/2

1

Expressing f(t ) in term of dilates and translates

138

Multiresolution and Multirate Signal Processing

Example 4.5.1 — Connecting f(t ) and filter banks.

From the Fig. 4.6 we see that f(t ) is expressible in its own dilates and translates. f (t ) = f (2t ) + f (2t - 1) The above equation is called the dilation equation. If we write down the coefficients of f(2t - n) as a sequence, we get the impulse responses corresponding to the Lowpass filter. 1 1

¬ 0 The dilation equation can thus be modified in a generalized way as follows: If h[ n] is the impulse response of the low pass filter in the two band filter bank then, f (t ) = åh[ n]f (2t - n) nÎ

4.5.2

Relation Between the Function y (t ) and Filter Banks

y (t ) Î V0 should also be expressible in terms of basis of V1 i.e. f(2t - n), n Î . Example 4.5.2 — Connecting y (t ) and filter banks.

Graphically, from Fig. 4.7, we can see that the dilation equation will be of the form y (t ) = f (2t ) - f (2t - 1) If we write down the coefficients of f(2t - n) as a sequence, we get the impulse responses corresponding to the high pass filter. The dilation equation can thus be modified in a generalized way as follows. If g[ n] is the impulse response of the high pass filter in the two-band filter bank then, y (t ) = åg[ n]f (2t - n) nÎ

This implies that, if we know the impulse responses, we can go the reverse way round to generate f(t ) and y (t ). Thus in short, low pass filter ® scaling function expansion high pass filter ® wavelet expansion Therefore, we can completely characterize the system f(t ) and y (t ) if we know their dilation equation. 1 -1

¬ 0

139

M-Channel Filter Bank

1 y(t) Φ(2t) 1 0

1/2 −Φ(2t−1)

Figure 4.7

4.6

Expressing y (t ) in term of dilates and translates

The Strategy for Future Analysis

After obtaining the dilation equations, these equations are used to find the wavelet and the scaling function, given the high pass and the low pass filter responses. This is accomplished in two steps as follows: ●●

●●

4.7

Take Fourier transform on both sides of the dilation equation to get a recursive equation in Fourier domain, which completely characterizes F {f (t )} in terms of DTFT of h[ n]. Relate Fourier transform of wavelet function to Fourier transform of scaling function by using DTFT of g[ n] to obtain the wavelet function.

Iterated Filter Banks and Continuous Time MRA

In the last section we deduced certain concepts of the ideal two-band filter bank and studied the frequency response of ideal filter which is obtained for HAAR filter bank. Inspite of certain drawbacks of the ideal nature of filters we then derived the dilation equation, and the sequences related to it showed us a path to move from filter banks to the scaling and wavelet function. In this section, we will study the relation between filter coefficients and the basis vectors. HAAR filter bank case is particularly considered for calculations.

4.8

Iterating the Filter Bank for f(×), y (×)

The dilation equation relates filter bank to scaling function and also to wavelets. If h[ n]: low pass filter impulse response. Then we can relate the operation of LPF H ( Z ) on basis of input signal f(t ) in general as ¥

f (t ) =

å h[n]f (2t - n)

n = -¥

= f (2t ) + f (2t - 1) for the case of HAAR low pass filter. Dilation equation is a new class of equation, which denotes the relation between wavelets and multirate filter bank.

140

Multiresolution and Multirate Signal Processing

Example 4.8.1

ˆ For analysis let us consider the Fourier transform of f(t ) be f(W) where W represent the analog angular frequency variable. +¥ fˆ (W) = ò f (t )e- jWt dt -¥

=ò

+¥ ¥

-¥

å h[n]f (2t - n)e

¥

+¥

-¥

n = -¥ ¥

f (2t - n)e- jWt dt

1

å h[n] 2 e

=

- j Wn 2

ò

¥

1

å h[n] 2 e

n = -¥

+¥

-¥

n = -¥

=

dt

n = -¥

å h[n]ò

=

- j Wt

- j Wn 2

f (l )e

- jW

l 2

dl

æWö fˆ ç ÷ è2ø

This is via a transformation done by substituting 2 t - n = l. Thus the frequency domain dilation equation can be represented as 1 æWö æWö fˆ (W) = H ç ÷ fˆ ç ÷ 2 è2ø è2ø where ¥

å h[n]e

is DTFT of h[.] at

- j Wn 2

n = -¥

W . 2

Recursively we can write, æWö 1 æWö æWö fˆ ç ÷ = H ç ÷ fˆ ç ÷ è2ø 2 è4ø è4ø Hence after N recursions,  N 1  Ω   Ω  f (Ω) = ∏ H  m   f  N   m =1 2  2    2  As, lim

N ®¥

W =0 2N

where N ∈ 

141

M-Channel Filter Bank

Therefore, æW fˆ ç N è2

ö ˆ ÷ = f (0) ø

ì N 1 æ W öü fˆ (W) = íÕ H ç m ÷ ý fˆ (0) î m =1 2 è 2 ø þ ˆ W) completely in for finite W where f(0) = constant. Therefore, we can express the Fourier transform f( terms of product of dilated LPF transfer function H(W) as above. The product of dilated terms in the frequency domain implies the convolution of impulse response h[ n], subject to above dual equation of Fourier transform. Now, consider the case of HAAR wavelet. When we carry out the same derivation (as above) for the HAAR scaling function f(t) and draw the magnitude response, it appears looks same as the sinc function with maximum density around zero frequency and thus depicting the low pass nature.

0 δ

1 1+∆

0 δ

Ω Ω+δ

h[ n] and H(W)

Figure 4.8

Consider h[ n] and its Fourier transform, as shown in Fig. 4.8. F .T

As h[ n] = {1,1} Û H (W) = (1 + e- jWn ) F .T

since h[ n] Û H (W) é n ù F .T h ê ú Û | a | . H ( aW ) ëaû Here d ® 0 imply moving towards the impulse function.Thus, we have train of impulses corresponding to the impulse response h[n]. Now, consider a term N

1

æWö m ÷ ø

Õ 2 H çè 2 m =1

Let us take first N = 3 for our convenience.

142

Multiresolution and Multirate Signal Processing

æWö æWö æWö H ç ÷ .H ç ÷ .H ç ÷ è2ø è4ø è8ø W = l. 8 Hence, the product be H (4l ). H (2l ). H (l ) Now put

In time domain H(4l ) means h[ n] upsampled by 4 and H(2 l ) is h[ n] upsampled by 2. Now, we have to find out the convolution, h[ n] *(h[ n] up-sampled by 2)* (h[ n] up-sampled by 4). This process can be carried out by following the steps depicted in plots shown above. We have h[ n] shown in Fig. 4.8, (h[ n] upsampled by 2) in Fig. 4.9 and convolution of these both, i.e h[ n]*(h[ n] upsampled by 2) in Fig. 4.10.

0

2

4

Figure 4.9

0

1

2

3

Figure 4.10

t

h[ n] upsampled by 2

4

t

h[ n]*(h[ n] upsampled by 2)

Now, consider (h[ n] upsampled by 4) as shown in Fig. 4.11. The plot shown in the figure convolves with impulse response (h[ n] upsampled by 4) to yield Fig. 4.12. W W = l according to our previous assumption. Since means expansion in the frequency 8 8 domain, hence we need to contract signal by the same factor in the time domain. Hence, contracting the resulting convolution term by a factor of 8 we get a plot as shown in Fig. 4.13. Now, replace

0

4

Figure 4.11

t

h[ n] upsampled by 4

143

M-Channel Filter Bank

0

2

Figure 4.12

4

6

8

t

h[ n]*(h[ n] upsampled by 2)*(h[ n] up-sampled by 4)

0

1

Figure 4.13

t

W after substituting = l 8

If we consider N = 8 and the higher indices so on on, i.e., moving towards the infinite iterations, the last of these impulses moves closer and closer to 1. Thus, we are obviously moving towards the continuous signal from 0 to 1 in time domain, as shown in Fig. 4.14, known as scaling function f(t). Similarly, we can construct the wavelet function as it is a function of basis function. ¥

y (t ) = åg[ n]f (2t - n) n =0

In terms of Haar wavelet g[ n] = {1, -1} y (t ) = f (2t ) - f (2t - 1) for N = 1

1

0

1

Figure 4.14

After infinite iterations

Example 4.8.2

Let the Fourier transform of y (t) be yˆ (W) then we have +¥

yˆ (W) = ò y (t )e- jWt dt -¥

time

144

Multiresolution and Multirate Signal Processing

=ò

+¥ ¥

-¥

¥

åg[n]f (2t - n)e

- j Wt

dt

n =0 +¥

= åg[ n]ò f (2t - n)e- jWt dt n =0

-¥

- j Wl ¥ 1 - jWn +¥ = åg[ n] e 2 ò f (l )e 2 d l -¥ 2 n =0 ¥ 1 - jWn æ W ö = åg[ n] e 2 fˆ ç ÷ 2 è2ø n =0

1 æWö æWö yˆ (W) = G ç ÷ fˆ ç ÷ 2 è2ø è2ø

The same transformation in the integral and DTFT of g[n] is being assumed as in the case of derivation of fourier transform of f(t). Recursively we can write, æWö 1 æWö æWö yˆ ç ÷ = G ç ÷ fˆ ç ÷ è2ø 2 è4ø è4ø Hence, the recursive equation can be written as,  (W) = 1 G æ W ö 1 H æ W ö 1 H æ W ö ..........fˆ (0) Y 2 çè 2 ÷ø 2 çè 4 ÷ø 2 çè 8 ÷ø 1 æWö N 1 æ W ö y (W) = G ç ÷{Õ H ç m ÷}fˆ (0) 2 è 2 ø m =2 2 è 2 ø Now, let N=3 æWö æWö æWö G ç ÷ .H ç ÷ .H ç ÷ è2ø è4ø è8ø W = l. Hence the product now transforms to G(4 l ). H (2l ). H (l ) 8 In the time domain, G(4 l ) means g[ n] upsampled by 4 and H(2l) signify h[ n] upsampled by 2. Now, Now put

we need to find out the convolution h[ n]* (h[ n] upsampled by 2) * (g[ n] upsampled by 4) for wavelet function.

145

M-Channel Filter Bank

This convolution can be carried out in much a similar way as it was in the case of the scaling function convolution. We have g[ n] upsampled by 4 as shown in Fig. 4.15, h[ n] upsampled by 2 as shown in Fig. 4.16 and h[ n] * (h[ n] upsampled by 2) as shown in Fig. 4.17.

0

4

Figure 4.15

0

g[ n] upsampled by 4

2

t

Figure 4.16

0

t

h[ n] upsampled by 2

2

4

Figure 4.17

t

h[ n]*h[ n] upsampled by 2

The convolution output obtained in Fig. 4.16 ( i.e, h[ n]*h[ n] upsampled by 2) is convolved with g[ n] upsampled by 4 which is shown in Fig. 4.18. W Now, again considering the assumption = l, we contract the signal by factor 8 as explained 8 above. Hence the resulted signal looks as shown in Fig. 4.19, which is achieved after performing the W transformation = l. 8

0

Figure 4.18

2

4

8

t

h[ n]*(h[ n] upsampled by 2)*(g[ n] up-sampled by 4)

146

Multiresolution and Multirate Signal Processing

0

1/2

Figure 4.19

1

After substituting

t

W =l 8

Now, when we consider larger number of iterations the function keeps on moving closer to look like wavelet function. Thus, we reconstruct the wavelet function, y (t ) by performing infinite iterations (N = ¥) and get the final plot, as shown in Fig. 4.20.

0

Figure 4.20

4.9

1

1/2

time

After infinite iterations

Z-domain Analysis of Multirate Filter Bank

In the few earlier sections we have established a very close relationship between the wavelet functions, the continuous time scaling function and the two channel filter bank. Convincingly, there is an intimate relationship between designing a two-band filter bank and a multiresolution analysis. After knowing the impulse response of the low pass filter in the filter banks, one can obtain the scaling function and hence the wavelet function by iterative convolution. Thus a pertinent reason for studying the two channel filter bank in great depth. We, now need to know how to deal with multi-rate operations. So let us begin with the two-channel filter bank.

4.10

Two-channel Filter Bank

A two-channel filter bank has an analysis side and a synthesis side, as shown in the Fig. 4.21. On the analysis side we have two filters namely the low pass analysis filter and the high pass analysis filter followed by downsampling operations. On the synthesis side we have the upsampling operation followed by the synthesis low pass filter and the synthesis high-pass filter. The outputs of these got added forming the overall output, as shown in the figure.

147

M-Channel Filter Bank

Analysis Side H0(z)

Synthesis Side G0(z)

2

2 Y3

Y1

Y5

Y7 Y(z)

X(z) +

(Input)

H1(z)

2

(Output)

G1(z)

2

Y2

Y4

Y6

Figure 4.21

Two channel filter bank

Y8

p 2 p G0 ( z ): Discrete time synthesis low pass filter with angular cutoff frequency 2 p H1 ( z ): Discrete time analysis high pass filter with angular cutoff frequency 2 p G1 ( z ): Discrete time synthesis high pass filter with angular cutoff frequency 2 H 0 ( z ): Discrete time analysis low pass filter with angular cutoff frequency

The signals X , Y1 , Y2 , Y5 , Y6 , Y7 , Y8 , Y are all of sequences of sampling rate x. The signal Y3 , Y4 are sequences x of sampling rate . 2 The unusual and new blocks in the above-shown block diagram are the upsampler and downsampler; unusual in the sense that one is not exposed to these while discussing filters.

4.10.1

Analysis of Downsampler Operation

Let us consider the downsampler block in detail. A basic downsampling operation is as follows:

The above sequence of numbers represents the indices of the samples of a signal prior to downsampling, the bottom sequences of numbers represents the resultant indices of the signal after subjecting it to downsampling operation. Hence a downsampling operation can be viewed as the combination of 2 steps as follows:

148

Multiresolution and Multirate Signal Processing

1. It first “kills” some samples as explained below:

where (.) represent the indices of the samples that are retained without killing and those with no braces represent the ‘killed’ samples of the given sequence.

2. Then it compresses the resultant as shown above.

4.10.2

Analysis of Upsampler Operation

The upsampler outdoes the last ’compression’ step of a downsampler. Hence, it is also called Expander. The upsampler operation is invertible; we can get the original sequence, i.e. the sequence prior the upsampling. It converters a sequence of lower sampling rate to a sequence of higher sampling rate, whereas the downsampling operation is not invertible. The downsampler converts a sequence of higher sampling rate to a sequence of lower sampling rate. Downsampler and upsampler operations are taken care of by clocking rates at different points in the sytem. Consider the downsampling a sequence by M and let M = 5, this is as shown below:

149

M-Channel Filter Bank

Consider the upsampling operation by M. assume M = 7: Then it is as follows:

4.10.3

Z-Domain Effect of Upsampling

+¥

Xu [ z ] = åxu [ k ]z Mk -¥

Xu [ z ] = X[ z M ] 1

X[ z ] = Xu [ z M ] Therefore, it is evident that the upsampling operation is invertible.

4.10.4

Z-Domain Effect of Downsampling

First “killing step” can be viewed as a “modulation” or “multiplication by killing sequence” (a window). We pass those values through the window which are retained. Window periodic sequence: PW [ n]; P represents periodic and W represents windowed.

150

Multiresolution and Multirate Signal Processing

The process of killing is essentially multiplication by PW [ n] for the appropriate M and it can be written as shown in the figure where the bar on upsampler denotes the inverse upsampling operation. x[n]

xd[n]

M

x[n]

X

M

xd[n]

PW[n]

Multiplying a sequence by a sequence in Z-domain is best done by exponentials. So, the noninvertible part of the downsampling operation must be replaced.

X

x[n]

PW[n] Q

In other words, we need to express PW [ n] = åCk a kn DFT represents time-limited periodic sequences as k =1

the combination of exponentials, i.e., time-limited sequence means b0 , b1 ,...., bN -1. Its Discrete Fourier Transform (DFT) is given as: N -1

B[ k ] = åb[ n]e

-j

2p nk N

where k = 0,1,..., N - 1

n =0

We can reconstruct b[ n] by the IDFT equation defined as b[ n] =

2p j nk 1 N -1 N B k [ ] e å N k =0

where n = 0,1,..., N - 1

even if reconstructed for n > N - 1 up to ¥, we can get a sequence periodic in N. The expression N -1

åB[k ]e k =0

j

2p nk N

generates a periodic sequence say b[ n].

151

M-Channel Filter Bank 2p

N -1

j ( n + lN ) k b[ n + lN ] = åB[ k ]e N k =0 N -1

= åB[ k ]e

j

2p nk N

.e j 2pl

k =0

b[ n + lN ] = b[ n] For M = 2, we have a periodic sequence ...1 0 1 0 1 0 1 0 1 0 1... where one period of the sequence is 1 0, taking DFT of the one period 1 0 is given as B[ k ] = 1.e

-j

2p 2

+ 0 =1

PW [ N ] for M = 2 is =

2p j nk 1 1 B[ k ]e 2 å 2 k =0

=

2p j nk 1 1 1.e 2 å 2 k =0

1 = [1n + (-1)n ] 2 Hence

X

x[n]

PW[n]

gives Z -transform as +¥

=

æ1

å x[n] çè 2 [1

n = -¥

n

ö + (-1)n ] ÷ z - n ø

=

1 +¥ 1 +¥ x[ n]z - n + å x[ n](- z )- n å 2 n = -¥ 2 n = -¥

=

1 ( X [ z ] + X [ - z ]) 2

152

Multiresolution and Multirate Signal Processing

This is the modulation opereation. So, it is followed by an inverse upsampler operation for the downsampling operation to be completed. é 1 ùö 1æ é 1ù X d [ n] = ç X ê z 2 ú + X ê - z 2 ú ÷ ÷ 2 çè ë û ë ûø Additional information

Y(w)

−2p

Figure 4.22

0

2p

w

Signal spectrum before downsampling operation

Yd(w)

−2p

Figure 4.23

0

2p

w

Signal spectrum after downsampling operation

Downsampling operation in time domain corresponds to aliasing in the frequency domain unless the given signal is sufficiently bandlimited in time. It can be pictorially explained as shown in Figs. 4.22 and 4.23. Figure 4.22 shows the discrete time fourier transform of a signal without downsampling operation. The signal then subjected to the downsampling operation may result in aliasing in the spectrum of the signal, as shown in Fig. 4.23. It may not be with all downsampling cases as explained below:

153

M-Channel Filter Bank

Example 4.10.1 — Example: Samplers.

Let x[ n] is a discrete time signal given by x1[ n] = [ -1,2,1,0,3, -2], n = 0,1,..5 x2 [ n] = [ -1,0,1,0,3,0], n = 0,1,..5 if the above signals are subjected to downsampling operation by a factor of 2, then the signal x1[ n] will suffer aliasing and signal x2 [ n] will not suffer aliasing. Hence the downsampling operation is not invertible when it result in the aliasing of the spectrum in the frequency domain, while it is invertible when it does not result in aliasing. Whereas the upsampling operation is always invertible. The aliasing that is noticed in the 2-band filter bank is because of the downsampling operations involved in the process. In this chapter we have established the Z-transforms of the basic multirate operations in terms of the original Z-transforms of the sequences. We shall continue with this further in the next chapter to carry out the complete analysis of the two-channel filter bank.

4.11

M-band Filter Banks and Looking Ahead

M-band filter banks are generalization of 2-band filter banks. For 2-band filter banks the signal is downsampled and upsampled by a factor of 2 but in the case of generalized M-band filter bank, the sequence is sampled by a factor of M. In next few sections a specific case of M = 3, called 3-band filter bank, is considered and analyzed in depth, which will help in understanding the generalization to any M.

4.12

3-band Filter Bank (Ideal)

Similar to 2-band filter banks, we can define 3-band filter banks too. This is shown in Fig 4.24.

X(Z)

H0(Z)

3

3

G0(Z)

H1(Z)

3

3

G1(Z)

H2(Z)

3

3

G2(Z)

Figure 4.24

Analysis side and synthesis side of a 3-band filter bank

+

Y(Z)

154

Multiresolution and Multirate Signal Processing

The above filter is true for any uniform 3-band filter bank, i.e., the length of all filters should be equal. For ideal perfect reconstruction Y = X , but in general for perfect reconstruction the output Y can be a scaled or/and delayed function of X , also it can be a version of X with an easily invertible operation.

4.12.1

Analyzing an Ideal 3-band Filter Bank

Conditions for perfect reconstruction: H 0 = G0 H1 = G1 H 2 = G2 Frequency responses of the ideal filters H0(ejw)

π/3

−π/3

H1(ejw)

ω

π/3

(a)

Figure 4.25

H2(ejw)

2π/3 ω

2π/3

πω

(c)

(b)

(a)H 0: a low pass filter, (b)H1: a band pass filter, (c)H 2: a high pass filter

Figure 4.25(a–c) shows the frequency responses of the ideal filter banks for a 3-band filter bank. As you p p 2p 2p can see the frequency axis is divided into three parts from 0 to , to and to p, thereby covering 3 3 3 3 the complete axis. Frequency effect of Upsampling

A(ejw)

Figure 4.26

3

A(e3jw)

Upsampling by 3

As can be seen there is a compression of frequency axis by a factor of 3. (Fig 4.26). Suppose we have a spectrum, as shown in Fig. 4.27(a), its upsampled version will be as shown in Fig. 4.27(b) where frequency axis is compressed by a factor of 3.

155

M-Channel Filter Bank

1 A(ejw)

β

A(ejw) 1

α β α π

−π

ω

−π

π/3

−π/3

(a)

Figure 4.27

β α

β α π

ω

(b)

(a) Original spectrum,(b) Upsampled version by compressing frequency axis by a factor of 3

Frequency effect of Downsampling Downsampling by a number creates aliases of the original spectrum, i.e., it will have original spectrum shifted and added. Figure 4.28 shows the downsampling by 3. 3

A(ejw)

Figure 4.28

A(e3jw)

Downsampling by 3

Downsampling is effectively multiplying the sequence by ..1001001001.. and then compressing it by throwing away the zeros obtained. Since ..1001001.. is a periodic sequence, it can be expressed in the terms of its IDFT by taking one period, finding its DFT and then IDFT giving us the sequence in form of modulates. 1 2 j 2pkn The above periodic sequence can be written as: å k =0e 3 . We multiply this expression with the 3 sequence to be downsampled in frequency domain and then reduce the z 3 to z to obtain the final downsampled version. Effectively, a[n]

a[n].

3

j2π 1 2 Σk = 0 e 3 3

kn

the above equation can be easily analyzed and solved in Z -domain; multiplying it and then replacing Z 1

by Z 3 to get the downsampled sequence. In Z -domain Z - transform

a[ n] a[ n]

1 2 j 23pkn åe 3 k =0

®

A( Z ) j 2 pk 1 3 A ( Ze ) å 3 k =0

Z - transform 2

®

(4.1)

156

Multiresolution and Multirate Signal Processing 1

now replace Z by Z 3 , we get

In sinusoidal frequency domain

The last equation in the above section shows that the downsampled version of sequence is obtained by 2p shifting its DTFT on the frequency axis by for k = 0,1,2 and adding. After adding the shifted DTFTs, 3 1 scale it vertically by and horizontally stretch by a factor of 3 to get the final downsampled by 3 version. 3

4.12.2

Interpretation of 3-band Filter Bank

Low frequency interpretation Now, let us analyze the effect of the 3-band filter bank in a low frequency region on a prototype spectrum. Figure 4.29 shows the prototype spectrum to be analyzed. Spreaded over the whole frequency range with a variable amplitude assuming 0 phase.

1

−π

Figure 4.29

X(ejw)

π ω

Prototype spectrum

Subject this spectrum to the low pass branch in the ideal 3-band filter bank, which has a cut off frep quency of and a downsampler of 3. 3 Firstly, spectrum obtained after passing it through the low pass filter is shown in Fig. 4.30.

157

M-Channel Filter Bank

1

−π

π/3

−π/3

Figure 4.30

π ω

Low pass spectrum

2p and adding is shown in Fig. 4.31. 3

Secondly, spectrum obtained after translating to multiples of

1 2/3

−π

Figure 4.31

−2π/3 −π/3

π/3

2π/3

Translated by multiples of

π ω

2p and added 3

1 Lastly, the spectrum obtained by scaling vertically by a factor of and horizontally by a factor of 3 is 3 shown in Fig. 4.32.

1/3

−π

Figure 4.32

2/9

π

ω

Spectrum obtained after subjecting to analysis filter bank

Now, for synthesis we upsample it by 3 and pass it through the synthesis low pass filter. It retains the scaled version of the original filter and destroys the aliases. The spectrum obtained is as shown in Fig. 4.33.

158

Multiresolution and Multirate Signal Processing

1/3 2/9

−π

Figure 4.33

−2π/3

−π/3

π/3

2π/3

π ω

Spectrum obtained after subjecting the sequence to low frequency branch

Aliases gets created in between the analysis and synthesis filters, because we wish to retain the total amount of data. The total number of samples per unit time at the input of the analysis filter is reduced to one-third at the outputs of each of the downsamplers. Thus, total amount of information remains the same at any point of time. Middle branch interpretation p 2p On the analysis side of the middle branch we have a bandpass filter between and followed by a 3 3 downsampler of 3. Let us take the same prototype spectrum, when subjected to a bandpass filter, we obtain a spectrum as shown in Fig. 4.34.

2/3 1/3

−π

−2π/3 −π/3

Figure 4.34

π/3

2π/3

π ω

Band pass version of the signal

After passing it through the bandpass filter, a downsampler by 3 acts on the spectrum. The spectrum is 2p translated by shifts of multiples of and added as shown in Fig. 4.35. To obtain the final downsampled 3 1 version, the spectrum is scaled vertically by a factor of and horizontally by a factor of 3. Spectrum 3 obtained is as shown in Fig. 4.36.

159

M-Channel Filter Bank

1/3

−π −2π/3 −π/3

Figure 4.35

2/3

π/3 2π/3 π

Translated by multiples of

ω

2p and added 3

1/9

π

−π

Figure 4.36

2/9

ω

Middle branch spectrum after subjecting to analysis side

Now, for synthesis we upsample it by 3 and pass it through the synthesis low pass filter. It retains the scaled version of the original filter and destroys the aliases. The spectrum obtained is as shown in Fig. 4.37.

1/9

−π −2π/3 −π/3

Figure 4.37

π/3 2π/3

2/9

π

ω

Reconstructed Middle branch spectrum

160

Multiresolution and Multirate Signal Processing

Exercises Exercise 4.1

Prove that:

åh[n] = 2 nÎ

and

åg[n] = 0 nÎ

Where h[ n] and g[ n] are the impulse responses of the low pass and high pass analysis filters respectively used in MRA. Hint: The problem can be answered using the two dilation equations of the scaling and the wavelet function used in MRA. The two dilation equations are: f (t ) = åh[ n]f (2t - n)

(4.2)

nÎ

y (t ) = åg[ n]f (2t - n)

(4. 3)

nÎ

Integrating Eq. 4.2 with respect to t on both sides we get:

ò

-¥

-¥

¥

¥

ò

-¥

-¥

¥

¥

f (t )dt = ò

åh[n]f (2t - n)dt nÎ

f (t )dt = åh[ n]ò f (2t - n)dt nÎ

(4. 4)

We will solve the integral term in the above summation: Substituting 2t - n = k in the integral we get:

ò

-¥

¥

f (2t - n)dt =

1 -¥ f (k )dk 2 ò¥

(4.5)

-¥

Now ò f(k )dk is the area under the scaling function which is equal to one, i.e ¥

ò

-¥

¥

f(k )dk = 1

Therefore from Eqs. (4.4), (4.5) and (4.6) we get:

(4.6)

161

M-Channel Filter Bank

h[ n] nÎ 2

1= å

åh[n] = 2 nÎ

Similarly, integration dilation equation for the wavelet function, i.e Eq. (4.3) we get

ò

-¥

-¥

¥

¥

ò

-¥

-¥

¥

¥

y (t )dt = ò

åg[n]f (2t - n)dt nÎ

y (t )dt = åg[ n]ò f (2t - n)dt nÎ

(4.7)

-¥

Now ò y (t )dt = 0 i.e are under y (t ) ¥ Thus, from Eqs. (4.5), (4.6) and (4.7), we have

åg[n] = 0 nÎ

Exercise 4.2

Prove that for a causal system the impulse response h[ n] = 0 for n < 0. Hint: For a system to be causal its output should be independent of the future input, i.e it should be generated or caused as an effect of something (in our case an input). Thus by general sense effect cannot be produced before an event occurs ( i.e input), and therefore causal systems are independent of future input values. Now, consider an impulse response defined for both positive and negative indices, say from -1 to +1, as in Fig.4.38. Let the output sequence be y[ n] and the input be x[ n]. The input output relationship in the Z-domain is given as: Y (Z ) = H (Z ) X (Z ) Y ( Z ) = h-1 Z + h0 + h1 Z -1 ) X ( Z ) Y ( Z ) = h-1 ZX ( Z ) + h0 X ( Z ) + h1 Z -1 X ( Z ) Taking inverse Z-transform of Eq. (4.8), we get y[ n] = h-1 x[ n + 1] + h0 x[ n] + h1 x[ n - 1].

(4.8)

162

Multiresolution and Multirate Signal Processing

We find that the present output is dependent on future input, i.e y[ n] is dependent on x[ n + 1] which is against our discussion of causality. Thus, to make the system causal, in above case h-1 = 0. Further, we can generalize it as: For a system to be causal h[ n] = 0 for n < 0.

h[n] h−1 h0

−1

Figure 4.38

0

h1

1

n

Impulse response h[n]

Exercise 4.3

Why the frequency response of f(t ) should not have zero at w = 0? Hint: Frequency response of f(t ) at w = 0 is constant because of low pass nature of f(t ).

Exercise 4.4

Prove that if the length of wavelet filter is L then the support of scaling function f(t )is L - 1? Hint: As defined earlier, scaling function is given as, N 1 æW fˆ (W) = {Õ .H ç m è2 m =1 2

ö ˆ ÷}.f (0). ø

Now, the multiplication in frequency domain corresponds to convolution in time domain. If we consider two functions x[ n] extending from 0 to N and y[ n] extending from 0 to M. Now, convolution of x[ n] and y[ n] will extend from 0 to N + M . Similarly, if we consider wavelet filter h[ n] having length L that is extending from 0 to L - 1. 1 Now, for each iteration filter support squeezes by a factor . 2 Hence the support of scaling function is the sum of æ L -1ö æ L -1ö æ L -1ö æ L -1ö ÷ + ............ + ¥. ÷+ç ÷+ç ç ÷+ç è 2 ø è 4 ø è 8 ø è 16 ø This sums up to L - 1.

M-Channel Filter Bank

163

Exercise 4.5

How does the killing step of the downsampling operation is described mathematically? Hint: Killing is nothing but multiplication of the given signal to be downsampled by a signal termed as windowed periodic sequence PW [ n], which can be described as the combination of the exponentials by means of the DFT (Discrete Fourier Transform) which then simply implies killing step of downsampling is mathematically done by subjecting the given signal to modulation property of the Z-transform.

Exercise 4.6

Are the downsampling and the upsampling operations linear and shift invariant? Hint: The downsampling and the upsampling operations are linear but not shift invariant, one can easily check the linearity by the principle of superposition. The signals obtained by downsampling the delayed signal will be different from the signal obtained by delaying the downsampled signal, so they are shift variant.

Chapter

5

Conjugate Quadrature Filter Bank

Introduction

Z-Domain analysis of filter bank Aliasing cancellation Perfect reconstruction: Conjugate quadrature Condition for perfect reconstruction Polynomial as an input

5.1

Introduction

Conjugate quadrature flter bank

In the previous chapter we studied Z -domain analysis of two-channel filter bank. This structure is often used in the implementation of discrete wavelet transform. So in this chapter we intend to analyze it completely by relating the Z -transform at every point of following structure and finding relationship between H 0 ( Z ), H1 ( Z ), G0 ( Z ) and G1 ( Z ) which ensures perfect reconstruction at Y ( Z ). We also aim to enhance our analysis to understand ‘conjugate quadrature filter banks’. Consider two channel filter bank as shown in Fig. 5.1. H0(Z)

2

G0(Z)

2

X(Z)

Y(Z)

H1(Z)

2

Figure 5.1

2

G1(Z)

Two-channel filter bank

For a two-channel filter bank which realizes the multirate systems effectively, the critical components are upsamplers and downsamplers. In chapter 4 we have seen that for 2-channel structures we use upsampling and downsampling by factor of 2 and for M-channel structures the usage gets extended to upsampling and downsampling by factor of M. In Chapter 4 we have also studies the effect of upsampling and downsampling as: 1. Effect of Upsampler xin[n]

xout,u[n]

2

Figure 5.2

Upsampler

X out ,U ( Z ) = Xin ( Z 2 )

(5.1)

166

Multiresolution and Multirate Signal Processing

2. Effect of Downsampler xin[n]

xout,d[n]

2

Modulation

xin[n]

xout,d[n]

2

Inverse Up-sampling

1 (1n + (−1)n) 2

Figure 5.3

Downsampler

In Z -domain, 1 1 1 X out , D ( Z ) = [ Xin ( Z 2 ) + Xin (- Z 2 )] 2

(5.2)

As proved earlier, downsampling by factor of 2 operation can be split into modulation by a sequence followed by inverse upsampling by 2. This follows from the fact that, upsampling by any factor is an 1

invertible operation which implies that inverse upsampling is meaningful. Here, ( Z 2 ) appears due to 1 inverse up-sampler. If inverse sampler is not considered, then power of will disappear. 2 Using this Z -transform, we can get a relation between Z -transform of input and Z - transform of output, provided it exists throughout the process of analysis and synthesis.

5.2

Z-domain Analysis of Filter Bank

To simplify the process, we name the output of each block as shown in Fig. 5.4. H0(Z)

Y1(Z)

2

Y3(Z)

2

Y5(Z)

G0(Z) Y7(Z)

X(Z)

Y(Z)

Y8(Z) H1(Z)

Y2(Z)

Figure 5.4

2

Y4(Z)

2

Filter bank with notations

Y6(Z)

G1(Z)

167

Conjugate Quadrature Filter Bank

With this notations, Z -transform at each point can be written quite easily. It is seen that Y1 ( Z ) and Y2 ( Z ) are simply X ( Z ) passed through filter H 0 ( Z ) and H1 ( Z ) respectively. Y1 ( Z ) = H 0 ( Z ) X ( Z )

(5.3)

Y2 ( Z ) = H1 ( Z ) X ( Z )

(5.4)

We can write the relation between Y3 ( Z ) and Y4 ( Z ) in terms of Y1 ( Z ) and Y2 ( Z ) respectively. But if we notice the steps for downsampling, then it is clear that inverse upsampling operation needed for downsampling cancels with upsampler leaving only modulation part as a combined effect of downsampling and upsampling. Thus it becomes easy to jump from Y1 ( Z ) and Y2 ( Z ) to directly Y5 ( Z ) and Y6 ( Z ) respectively (Fig. 5.5). This strategy is quite useful in analyzing multi-rate system particularly when downsampler is followed by upsampler. Thus it follows that,

Y1(Z)

Y5(Z)

1 n (1 + (−1)n) 2

Figure 5.5

Jumping across up and down sampler 1 Y5 ( Z ) = {Y1 ( Z ) + Y1 (- Z )} 2 1 Y6 ( Z ) = {Y2 ( Z ) + Y2 (- Z )} 2

(5.5) (5.6)

The important point is that from this point on, we have contribution from X ( Z ) as well as X (- Z ). Significance of X (- Z ) will be dealt with later on. Once we have Y5 ( Z ) and Y6 ( Z ) we can easily go back and write for Y3 ( Z ) and similarly Y4 ( Z ) as follows. 1

Y3 ( Z ) = Y5 ( Z 2 ) 1 2

Y4 ( Z ) = Y6 ( Z )

(5.7) (5.8)

Jumping across upsampler and downsampler is useful since it brings us quickly to output. We are only one step away from output which can be easily achieved as Y ( Z ) = Y7 ( Z ) + Y8 ( Z )

(5.9)

Y7 ( Z ) = Y5 ( Z )G0 ( Z )

(5.10)

Y8 ( Z ) = Y6 ( Z )G1 ( Z )

(5.11)

where Y7 ( Z ) and Y8 ( Z ) are given by

168

Multiresolution and Multirate Signal Processing

where Y2 ( Z ) = H1 ( Z ) X ( Z )

(5.12)

Y2 (- Z ) = H1 (- Z ) X (- Z )

(5.13)

Y (Z ) = t 0 (Z ) X (Z ) + t 1 (Z ) X (Z )

(5.14)

In total, we can write

where 1 t 0 ( Z ) = {G0 ( Z ) H 0 ( Z ) + G1 ( Z ) H1 ( Z )} 2 1 t 1 ( Z ) = {G0 ( Z ) H 0 (- Z ) + G1 ( Z ) H1 (- Z )} 2

(5.15) (5.16)

This implies that Y ( Z ) is a linear combination of X ( Z ) and X (- Z ) in Z -domain. If the term X (- Z ) would not have been there, Y ( Z ) would have simply be the X ( Z ) passed though a filter with function t 0 ( Z ) like a LSI system. Dependence on X (- Z ) is what brings the trouble in the equation. To understand this, let us first interpret what the term X (- Z ) spectrally means and what it reflects in the frequency domain. R Effect of X(–Z)

To understand effect of X (− Z ) let us go back to frequency domain by substituting z ← e . It implies jw

X ( − Z ) = X (e j (w ± p ) ) i.e. we are shifting the spectrum of X (e jw ) by +p or −p . Due to periodicity on the w axis of 2p, shifting by +p or −p is equivalent. The impact of this shift can be best understood by taking an example.

X(ej(w)) 1

−p

−

p 2

Figure 5.6

0

p 2

p

ω

Spectrum of X(w )

Here X (e jw ) is the Fourier transform of some sequence x[ n] as shown in Fig. 5.6. But this is just the principal interval of w axis. Actually, X (e j (w ) ) looks as shown in Fig. 5.7.

169

Conjugate Quadrature Filter Bank

X(ej(w)) 1

−2p

−p

−

Figure 5.7

p 2

p 2

0

2p

p

w

Periodic nature of spectrum of X(w )

After shifting by +p or -p , spectrum becomes, as shown in Fig. 5.8. X(w ± p) 1 A

A

A

−p 2

−p

Figure 5.8

−p 2

0

A

p

w

Spectrum of X(w ) after shift of ±p

Thus it is evident that shifting by +p or -p is same. Notation A and A is justified since for real x[ n] which is a general case, X (e jw ) has a conjugate symmetry. The spectrum can be redrawn by shifting the X (e jw ) by +p or -p . For clarity in future, X (e jw ) is written as X(w ) . We mark the edges of spectrum as A and A carefully. X(w ± p)

A

A

A

A

X(ω) −p

Figure 5.9

−p 2

0

p 2

p

Spectrum of X(w ) in the principal interval

w

170

Multiresolution and Multirate Signal Processing

p 3p , is repeated in region -p to - . In Fig. 5.9, solid line rep2 2 resents X(w ± p ) while dashed line represents original X(w ). We see clearly from the figure how the position of A and A has been modified from its original position. The so-called negative frequencies -p p between and 0 now appear between and p. This results in two changes: 2 2 A, which has been shifted to region p to

●●

●●

-p The order of frequency has been reversed. Frequency, which was initially ordered as 0 to has 2 p now been reordered between and p. Larger frequency has now become smaller and vice versa. 2 p p For example frequency , which was larger than , appears as a smaller frequency in shifted 4 8 spectrum. The frequency itself has changed. In other words, frequencies have attained a false identity.

This is exactly the same phenomenon as aliasing. It occurs in sampling if the input signal is not sampled with the adequate rate. It has occurred due to the down-sampler used in the process. By using the downsampler, we have allowed the possibility of aliasing. It is due to this fact that term X (- Z ) is called aliasing term. In two channel filter bank where we want perfect reconstruction, this aliasing should be absent. Perfect reconstruction means after you analyze (decompose), and finally reconstruct (synthesis), Y ( Z ) is exact replica of X ( Z ).This condition is fulfilled when the filters are chosen properly, for example as in case of Haar. Thus in general term X (- Z ) is troublemaker. The first step to ensure the prefect reconstruction is to do away with the aliasing term.

5.3

Aliasing Cancellation

When we say that we do not want aliasing to appear at the output it essentially means that t 1 ( Z ) = 0. In terms of filter response we want G0 ( Z ) H 0 (- Z ) + G1 ( Z ) H1 (- Z ) = 0 If we explicitly express the synthesis filter in terms of the analysis filter we can easily ensure that t 1 ( Z ) = 0. Expressing synthesis filter as required can be done by simply rearranging this equation. H (- Z ) G1 ( Z ) =- 0 G0 ( Z ) H1 (- Z )

(5.17)

G1 ( Z ) = ± H 0 (- Z ) G0 ( Z ) =  H1 (- Z )

(5.18)

Very simple choice for this condition is:

This is a simple choice but definitely not the only choice. We can allow the factor common in both numerator as well as denominator, which eventually gets cancelled when we divide them. So more generally, G1 ( Z ) = ± R( Z ) H 0 (- Z )

(5.19)

G0 ( Z ) =  R( Z ) H1 (- Z )

(5.20)

where R( Z ) is factor cancelled. Interpretation of G1 ( Z ) = ± H 0 (- Z ) can be shown as follows. Ideally, p H 0 ( Z ) is a low pass filter with cutoff frequency of and frequency spectrum as shown in Fig. 5.10. 2

171

Conjugate Quadrature Filter Bank

1 H0(Z)

−p

−p 2

Figure 5.10

p 2

0

p

w

Spectrum of ideal low pass filter

Then H 0 (- Z ) has spectrum as shown in Fig. 5.11. H0(−Z) 1

−p

−p 2

0

Figure 5.11

p 2

p

w

Spectrum of ideal high pass filter

p This is nothing but a spectrum of ideal high pass filter with cutoff . Thus relationship G1 ( Z ) = + H 0 (- Z ) 2 makes a lot of sense for ideal filter. Same analogy goes for G0 ( Z ) = - H1 (- Z ). Since this is just a magp nitude plot, effect of minus sign is not visible here. It only adds additional phase of ± to the system. 2 Thus, we see that how this simple choice for G0 ( Z ) and G1 ( Z ) makes sense for an ideal filter. If we generalize our choice with factor R( Z ), there is slightly more modification than just low pass to high pass and vice-versa. With making this choice we have completed the first step towards perfect reconstruction. Next step is to prove the perfect reconstruction property.

5.4

Perfect Reconstruction: Conjugate Quadrature

In the last few sections we have dealt with two channel filter bank in detail. We had noted the effects of going past a downsampler and an upsampler in the Z-domain. Some of its important points are/noted as follows. Input : x[ n] ¾Z¾ ® X (Z )

172

Multiresolution and Multirate Signal Processing

Output : y[ n] ¾Z¾ ® Y (Z ) Analysis side has low pass filter and high pass filter with system functions H 0 ( Z ) and H1 ( Z ). Synthesis side has Low Pass Filter and High Pass Filter with system functions G0 ( Z ) and G1 ( Z ). Output is Y ( Z ) = t 0 ( Z ) X ( Z ) + t 1 ( Z ) X (- Z )

(5.21)

X(-Z) is called the ‘alias’ term and hence t 1 ( Z ) is called Alias System Function. We call it a system function, but it is incorrect. The word is a misnomer because when there is an alias term the system is not linear and shift invariant. For alias cancellation we want t 1 ( Z ) to be equal to zero. Then system becomes Linear Shift Invariant(LSI). 1 t 1 ( Z ) = {G0 ( Z ) H 0 (- Z ) + G1 ( Z ) H1 (- Z )} 2

(5.22)

H (- Z ) G1 ( Z ) =- 0 G0 ( Z ) H1 (- Z )

(5.23)

For t 1 ( Z ) = 0

Simply equating numerator and denominator we get G1 ( Z ) = ± R( Z ) H 0 (- Z )

(5.24)

G0 ( Z ) =  R( Z ) H1 (- Z )

(5.25)

Here, if H1 (- Z ) is a high pass filter, G0 ( Z ) becomes low pass filter. If t 1 ( Z ) becomes zero, the system becomes LSI as Y ( Z ) = t 0 ( Z ) X ( Z ) with t 0 ( Z ) being a system function.

5.5

Condition for Perfect Reconstruction

We want to decompose a signal and reconstruct it back with ideal case as X ( Z ) = Y ( Z ). Even tolerable changes in the output can be accepted. What could be tolerable or acceptable changes? In case of time systems, time delays are tolerable since finite time is needed to process the signal at analysis and synthesis side. Even an output multiplied by a constant is acceptable. So, in perfect reconstruction process t 0 ( Z ) takes the form of t 0 ( Z ) = C0 z - D

(5.26)

where C0 is a constant. Ideally, we would like to have t 0 ( Z ) = 1 for all z. But this makes the system noncausal. So, a factor Z - D is allowed to take care of causality. In Haar Filter Bank, shown in Fig. 5.12, we have following 1 t 1 ( Z ) = {G0 ( Z ) H 0 (- Z ) + G1 ( Z ) H1 (- Z )} 2 −1   1 − z −1  1  −1  1 + z   ± − = (1 + z −1 )  (1 z )     2   2    2 

(5.27)

173

Conjugate Quadrature Filter Bank

H0(Z)

Y1(Z)

Y3(Z)

2

2

Y5(Z)

G0(Z) Y7(Z)

X(Z)

Y(Z)

Y8(Z) H1(Z)

Y2(Z)

Y4(Z)

2

Figure 5.12

2

Y6(Z)

G1(Z)

The haar filter bank

For alias cancellation we need t 1 ( z ) to be equal to zero. This gives G1 ( z ) = -(1 - z -1 ). System functions of Haar Filter Bank are as follows H0 (Z ) =

1 + z -1 2

H1 ( Z ) =

1 - z -1 2

G0 ( Z ) = 1 + z -1 G1 ( Z ) = -(1 + z -1 ) We find the value of t 0 ( Z ) 1 t 0 ( Z ) = {G0 ( Z ) H 0 ( Z ) + G1 ( Z ) H1 ( Z )} 2 -1 æ 1 + z -1 ö ö üï 1 ìï -1 æ 1 - z = í 1 + z -1 ç ÷ - 1- z ç ÷ý 2 îï è 2 ø è 2 ø þï

(

)

= z -1

(

)

(5.28)

This represents a delay of one sample. This delay is required on account of causality. If we try to avoid this delay we must have noncausality either on synthesis side or on analysis side. Simplest possibilty for alias cancellation is G0 ( Z ) = ± H1 ( Z ). In case of Haar Filter Bank 1 - z -1 2 1 + z -1 H1 (- Z ) = 2 H1 ( Z ) =

Essentially, G0 ( Z ) = ± R( Z ) H1 (- Z ) condition should be satisfied. More generally for alias cancellation we need

174

Multiresolution and Multirate Signal Processing

G0 ( Z ) = ± R( Z ) H1 (- Z ) G1 ( Z ) =  R( Z ) H 0 (- Z ) In particular, for Haar case we have chosen R( Z ) = 2 and G1 ( Z ) as G1 ( Z ) = -2 H 0 ( Z ) 1 = {-2[ (1 + z -1 )]} 2 = -(1 - z -1 )

5.6

Polynomial as an Input

Haar MRA has many hidden concepts. However, we need to study what is the problem associated with Haar MRA. In other words, we should learn how Haar is a beginning of a family of Multiresolution Analysis. H0(Z)

2

H1(Z)

2

X(Z) zero output

Figure 5.13

Output for constant input

Example 5.6.1 — Effect of splitting on high pass branch.

To have more insight we look at low pass and high pass filters from a different perspective. It can be understood by taking an example of constant sequence as input and noting the output at different points in filter bank. x[ n] H1 ( Z )

= C1 for all n 1 = (1 - z -1 ) 2

x[ n] - x[ n - 1] = 0 for all n 2 If there is a constant component in the input sequence of Haar filter bank, it is destroyed by high pass filter. This is depicted in Fig 5.13. In Haar filter bank we have a term (1 - z -1 ). What could be the situation in case of multiple or cascaded (1 - z -1 )?

175

Conjugate Quadrature Filter Bank

If we assume an input having polynomial components, then every instance of (1 - z -1 ) reduces degree of polynomial by one. A Taylor series is a polynomial expansion of input and if we subject some terms in the polynomial expansion to (1 - z -1) we have interpretation like this a0 n M + a1n M -1 + a2 n M - 2 +  + aM = x[ n] where x[n] is polynomial input sequence. Every time this is subjected to term (1 - z -1), what happens is a0 n M + a1n M -1 + a2 n M - 2 +  + aM - {a0 (n - 1) M -1 + a1 (n - 1) M - 2 +  + aM } When we expand this, coefficient of n M is a0 - a0, i.e. zero. Each time we subject this polynomial to action of (1 - z -1), we are reducing degree by one. For example, consider sequence of polynomial degree one (say 3n + 5) output = (3n + 5) - [3(n - 1) + 5] = 3n + 5 - 3n + 3 - 5 = 3 for all n Hence coefficient of highest power of n vanishes. These terms are (1 - z -1 ) on high pass branch and can not be on low pass branch. We build up whole family of MRA with more and more (1 - z -1 ) terms on high pass branch. That is what is called as Daubechies family of MRA.

1 − z−1

Figure 5.14

1 − z−1

Cascade system

As we increase seniority in this family, there are more and more (1 - z -1 ) terms on high pass branch. Effectively, we are reducing degree of higher order polynomial on high pass branch. We are killing them on high pass branch, i.e. we are transferring them on low pass branch. We are thus retaining more smoothness on low pass branch. In addition to this we want the same filters on analysis and synthesis side of filter bank. This gives us a class of filter bank known as Conjugate Quadrature Filter bank. Describing equations of these filter banks is very simple.We start from aliasing cancellation condition G0 ( Z ) = ± H1 ( Z ) G1 ( Z ) =  H 0 ( Z ) We choose (inspired by Haar) G0 ( Z ) = - H1 (- Z ) G1 ( Z ) = - H 0 (- Z ) Here we keep away a factor of 2 that can be absorbed by constant C0 . We need t 1 ( Z ) = 0 (by construction) and t 0 ( Z ) as 1 t 0 ( Z ) = {H1 (- Z ) H 0 ( Z ) + (- H 0 (- Z )) H1 ( Z )} (5.29) 2

176

Multiresolution and Multirate Signal Processing

p . 2 Therefore, the first term in above expression (Eq. 5.29) represents cascade of two low pass filters and p the second term represents a cascade of two high pass filters with cutoff . 2 In case of Haar there is relation between H 0 and H1. For perfect reconstruction, t 0 ( Z ) should be equal to a delay and some multiplying constant. In Haar case, As H1 ( Z ) is a high pass filter (in synthesis side) H1 (- Z ) becomes a low pass filter with cutoff

H1 (- Z ) =

1 + z -1 = H0 (Z ) 2

We shall in general note that H 0 ( Z ) should be related to H1 (- Z ). Choosing H1 ( Z ) to be slightly modified from H 0 (- Z ) as H1 ( Z ) = z - D H 0 (- Z -1 ) For Haar case, z -1 H 0 (- Z -1 ) =

Z -1 - 1 2

In general case for H1 ( Z ) = z - D H 0 (- Z -1 ), t 0 ( Z ) becomes 1 t 0 ( Z ) = {H 0 ( Z )((- Z )- D ) H 0 ( Z -1 ) - H 0 (- Z )( Z - D ) H 0 (- Z -1 )} 2 1 = {H 0 ( Z )( Z - D ) H 0 ( Z -1 )(-1- D ) - H 0 (- Z )( Z - D ) H 0 ((- Z )-1 )} 2

(5.30)

In Eq. (5.30), we choose value of D and put condition on H 0, then this becomes perfect reconstruction situation, which we shall discuss in more detail in the next section.

5.7

Conjugate Quadrature Filter Bank

We continue in this section to build upon the particular class of filter bank, which we have introduced earlier called a Conjugate Quadrature Filter (CQF) bank. For the perfect reconstruction system we must first do away aliasing. The alias cancellation equation for the two-band filter bank is given by G0 ( Z ) H 0 (- Z ) + G1 ( Z ) H1 (- Z ) = 0

(5.31)

H (- Z ) G1 ( Z ) =- 0 G0 ( Z ) H1 (- Z )

(5.32)

Equating the numerator and denominator we get the relation between G0 ( Z ), H1 (- Z ), G1 ( Z ) and H 0 (- Z ) as G1 ( Z ) = - H 0 (- Z )

(5.33)

G0 ( Z ) = H1 (- Z )

(5.34)

The relation between the analysis HPF (high pass filter) and analysis LPF (low pass filter) called a conjugate quadrature relationship, is given by

177

Conjugate Quadrature Filter Bank

H1 ( Z ) = z - D H 0 (- Z -1 )

(5.35)

Here z - D term is used to introduce causality. Putting Z = e jw in the Eq. (5.35), we get the frequency response equation as H1 ( Z ) = z - D H 0 (- Z -1 ) | jw

H1 (e ) = e

- jwD

H 0 (-e

- jw

Z = e jw

)

The magnitude response is given by | H1 (e jw ) | = | e− jwD H 0 (−e− jw ) | | H1 (e jw ) | = | e- jwD || H 0 (-e - jw ) | | H1 (e jw ) | = | H 0 (-e- jw ) | H 0 ( Z ) is a Low pass filter with a real impulse response (real coefficients), therefore H 0 (e- jw ) = H 0 (e jw ) The magnitude response of LPF H 0 ( Z ) is symmetric along the magnitude axis and phase response is anti-symmetric along the frequency axis w. H 0 (-e- jw ) = H 0 (e- j (w ±p ) ) p (With shift by p on w ) p NOTE : ( LPF with cutoff frequency )  ( HPF with cutoff frequency ) 2 2 We have shown, H1 ( Z ) = z - D H 0 (- Z -1 ) For the perfect reconstruction the equation must satisfy, G0 ( Z ) H 0 ( Z ) + G1 ( Z ) H1 ( Z ) = C0 z - D H1 (- Z ) H 0 ( Z ) - H 0 (- Z ) H1 ( Z ) = C0 z - D (-1)- D z - D H 0 ( Z -1 ) H 0 ( Z ) - H 0 (- Z )z - D H 0 (- Z -1 ) = C0 z - D We need the following for perfect reconstruction systems, (-1)- D H 0 ( Z -1 ) H 0 ( Z ) - H 0 (- Z ) H 0 (- Z -1 ) = C0 If we consider the Haar filter then the relationship between H 0 ( Z ) and H1 ( Z ) is given by, H 0 ( Z ) = 1 + z -1 H 0 (- Z -1 ) = 1 - z The above equation is non-causal so to make it causal by inserting delay, we get the below equation, z - D H 0 (- Z -1 ) = z - D (1 - z ) Here z - D retains causality.

178

Multiresolution and Multirate Signal Processing

If D is odd, H 0 ( Z ) H 0 ( Z -1 ) + H 0 (- Z ) H 0 (- Z -1 ) = -C0 H 0 ( Z ) H 0 ( Z -1 ) + H 0 (- Z ) H 0 (- Z -1 ) = Constant Putting Z = e jw , we get the above equation in the frequency domain as, H 0 (e jw ) H 0 (e- jw ) + H 0 (-e jw ) H 0 (-e- jw ) = Constant For real impulse response we have, H 0 (e- jw ) = H 0 (e jw ) H 0 (e jw ) H 0 (e jw ) + H 0 (e j (w ±p ) ) H 0 (e j (w ±p ) ) = Constant | H 0 (e jw ) |2 + | H 0 (e j (w ±p ) ) |2 = Constant This equation is called the power complementary equation. For perfect reconstruction system, H 0 ( Z ) H 0 ( Z -1 ) + H 0 (- Z ) H 0 (- Z -1 ) = Constant Lets us assume k 0 ( Z ) = H 0 ( Z ) H 0 ( Z −1 ) k 0 ( Z ) + k 0 (- Z ) = Constant

(36)

We are going to choose even length of H 0 ( Z ) , i.e. D ® Odd h1[ n] : h0 h1 h2  hD ↑ n

↑ D

↑ 0

Similarly, H 0 ( Z -1 ) is given by, h2 [ n] : hD  h2 h1 h0 ↑ n

↑ −D

↑ 0

Here, H 0 ( Z ) H 0 ( Z -1 ) corresponds to its convolution in time domain (h0 h1 h2  hD ) ∗ (hD  h2 h1 h0 ) ↑ 0

↑ D

↑ −D

↑ 0

Let the impulse response h[ k ] be as given below h[ k ] : h0 h1 h2  hD ↑ D

↑ 0

And impulse response g[ k ], as given below, which is mirror image of h[ k ], that means g[ k ] = h[ - k ] g[ k ] : hD  h2 h1 h0 ↑ −D

↑ 0

Similarly, g[ n - k ] is given below g[ n − k ] : h0 h1 h2  hD ↑ n

↑ n+ D

179

Conjugate Quadrature Filter Bank

The convolution between h[ k ] and g[ k ] is given as k = +∞

k 0 [ n] =

∑ h[k ]g[n − k ]

k = −∞

Here h[ k ] is causal and filter length is ( D + 1). The convolution at the sample n is k 0 [ n] . Shown below is the multiplication of h[ k ] and g[ - k ] (which is shifted by n samples) k0 [ n] : h0 h1 h2 ... hn ↑

h0

0

↑ n

hn +1 ...... hD h1

h2

In Z -domain k 0 ( Z ) = H 0 ( Z ) H 0 ( Z -1 ). The m th sample of the filter k0 [ m] is < h[ k ], h[ k ± m] > Let m be equal to 2 and filter length 4 ( D = 3) k0 [2] : h0 h1 h2 h3 ↑

h0 h1 h2 h3

0

↑ 2

k0 [2] = h0 h2 + h1h3 k0 [ −2] :

h0 h1 h2 h3

h0 h1 h2 h3 ↑ −2

↑ 0

If m = -2 and filter length 4 ( D = 3) k0 [ -2] = h0 h2 + h1h3 That means the convolution between h[ k ] and g[ k ], i.e. k 0 [ n] and k 0 [ −n] is symmetrical. k 0 ( Z ) + k 0 (- Z ) = Constant 1 {k 0 ( Z ) + k 0 (- Z )} = Constant 2 1 {k 0 ( Z ) + k 0 (− Z )} represents nonzero sample value at the even location and zero sample value at the 2 1 odd location. Let k 0 ( Z ) correspond to the sequence k0 [ n], {k 0 ( Z ) + k 0 (− Z )}, an impulse response so 2 obtained is shown below.

180

Multiresolution and Multirate Signal Processing

multiplication k0 [ n ] ® Ä ® non - zero at even & zero location › ........1 0 1 0 1 0 1 0....... › 0 1 {k 0 ( Z ) + k 0 (− Z )} = constant. Taking inverse Z transform on both sides, we 2 obtain impulse at zero location on Right Hand Side (since inverse Z transform of a constant is an impulse),thus we want the nonzero sample value only at zero location and zero sample value for odd and even location. So, at the even location m = 2l and m ¹ 0 and (l Î ), we want zero sample value. Let Daubechies filter with length 4( D = 3) Consider the equation

h0 [ n] : h0 h1 h2  h3 ↑ 0

↑ 3

In the Haar case, (1 - z -1 ) represents a High pass filter. Here, we consider the Daubechies filter with length 4, so two (1 - z -1 ) terms in the High pass filter which means (1 - z -1 )2 factor in HPF. Similarly, low pass filter has a factor (1 + z -1 )2 . A Daubechies low pass filter with length 4 is given by H 0 ( Z ) = h0 + h1 z -1 + h2 z -2 + h3 z -3 We can write this equation in the factor of (1 + z -1 )2 that is H 0 ( Z ) = (1 + z -1 )2 (1 + B0 z -1 ) In the above equation, we need three zeros. Two zeros are already chosen at unit circle which are -1, -1 and one zero is selected based on the value of B0. This value can be obtained by comparing the above two equations. Expanding the above two equations H 0 ( Z ) = (1 + 2 z -1 + z -2 )(1 + B0 z -1 ) H 0 ( Z ) = 1 + (2 + B0 )z -1 + (1 + 2 B0 )z -2 + B0 z -3 The dot product of the impulse response of LPF with its even shifts must be zero. We will use this constraint to find the value of B0 in the next chapter.

181

Conjugate Quadrature Filter Bank

Exercises Exercise 5.1

Consider H 0 ( Z ) to be an ideal Band Pass Filter with cutoffs and sketch the spectrum of H 0 (- Z ).

3p p and as shown in Fig. 5.15. Find 4 4

H0(Z)

−p

−p 2

Figure 5.15

0

p 2

p

w

Spectrum of H 0 ( Z )

Hint: Replacing Z by -Z in H 0 (- Z ) implies shifting the spectrum by +p or -p . But, due to the periodicity of w axis, shifting by +p or -p are equivalent. Given H 0 ( Z ) is a band pass filter, on shifting its spectrum by ±p will result in a band stop filter as shown in Fig. 5.16 with band stop frequencies p 3p and . 4 4 H0(-Z)

−p −3p/4 −p/2 −p/4

Figure 5.16

0 p/4 p/2 3p/4

p

w

Spectrum of H 0 (- Z )

Exercise 5.2

Find Y1 to Y9 in the two channel filter bank after interchanging downsampler and upsampler as shown in Fig. 5.17. Hint: It can be clearly seen that there is no effect of interchanging on Y1 and Y2 . To calculate further we will use the Eqs (5.37) and (5.38) previously mentioned in this chapter representing the effects of upsampler and downsampler respectively.

182

Multiresolution and Multirate Signal Processing

Y1 ( Z ) = H 0 ( Z ) X ( Z ) Y2 ( Z ) = H1 ( Z ) X ( Z )

(5.37) (5.38)

Y3 ( Z ) = Y1 ( Z 2 )

(5.39)

now using (5.37), 2

Y4 ( Z ) = Y2 ( Z )

(5.40)

now using (5.38), 1 1 1 Y5 ( Z ) = [Y3 ( Z 2 ) + Y3 (- Z 2 )] 2 1 1 1 Y6 ( Z ) = [Y4 ( Z 2 ) + Y4 (- Z 2 )] 2

(5.41) (5.42)

substitute Y3 ( Z ) and Y4 ( Z ) from (5.39) and (5.40) Y5 ( Z ) = Y1 ( Z )

(5.43)

Y6 ( Z ) = Y2 ( Z )

(5.44)

Y7 ( Z ) = G0 ( Z )Y5 ( Z ) = G0 ( Z )Y1 ( Z )

(5.45)

Y8 ( Z ) = G0 ( Z )Y6 ( Z ) = G0 ( Z )Y2 ( Z )

(5.46)

Y7 ( Z ) and Y8 ( Z ) are given by;

In total, we can write Y ( Z ) = Y7 ( Z ) + Y8 ( Z )

(5.47)

Y ( Z ) = {H 0 ( Z )G0 ( Z ) + H1 ( Z )G1 ( Z )}X ( Z )

(5.48)

From the above equations we can infer that there is no effect of a upsampler followed by a downsampler on the signal magnitude or phase.

H0(Z)

Y1(Z)

2

Y3(Z)

2

Y5(Z)

G0(Z) Y7(Z)

X(Z)

Y(Z)

Y8(Z) H1(Z)

Y2(Z)

Figure 5.17

2

Y4(Z)

2

Y6(Z)

Modified two-channel filter bank

G1(Z)

183

Conjugate Quadrature Filter Bank

Exercise 5.3

Obtain the Z -transform of a general downsampler (down-sampling by a factor of M). Hint: Downsampler can be broken down into two steps as modulation by a sequence and then inverse upsampling. General sequence for modulation in case of downsampling by a factor of M can be written as: 1 M

M -1 - j 2 pl M

åe

(5.49)

l =0

Using this equation, we can write the Z-Transform of downsampler by factor M as: X out ,d ( Z ) =

1 M

M -1

1

åXin (Z M ´ e

- j 2p

l M

)

(5.50)

l =0

Exercise 5.4

Obtain the output of M-channel filter bank. Hint: Z -transform of M-channel filter bank, which has M analysis filters denoted by H 0 ( Z ), H1 ( Z ), × × × H M -1 ( Z ) and corresponding M synthesis filter denoted by F0 ( Z ), F1 ( Z ) × × × FM -1 ( Z ), can be written as follows: X out ,d ( Z ) =

1 M

M -1

å( Xin (Z ´ e l =0

- j 2 pl M -1 M

) å( Hk (Z ´ e

- j 2 pl M

))Fk ( Z ))

(5.51)

k =0

(For further reading on filter banks, you can refer Multirate Systems and Filter Banks by P.P.Vaidyanathan).

Exercise 5.5

Consider the cascade system given in Fig. 5.14. Input to the system is 3n2 + 5n + 1 Find the output and comment on its degree. Hint: Output y0 after the first block is: y0 (n) = (3n2 + 5n + 1) - (3(n - 1)2 + 5(n - 1) + 1)

(5.52)

y0 (n) = 6n + 2

(5.53)

Final output y(n) will be: y(n) = 6n + 2 - (6(n - 1) + 2) = 6

(5.54)

Degree of output is lower than input by 2, which is expected as a block of 1-Z -1 lowers the degree of polynomial by 1.

184

Multiresolution and Multirate Signal Processing

Exercise 5.6

Consider the two-channel filter bank, whose two filters are given below: H 0 ( Z ) = 1 + Z -1 + Z -2 H1 ( Z ) = 1 - Z -1 + Z -2 - Z -3 Find out filters G0 ( Z ) and G1 ( Z ) so that alias cancellation occurs. Hint: For alias cancellation we should have: 1 {G0 ( Z ) H (- Z ) + G1 ( Z ) H1 ( - Z )}= 0 2 H (- Z ) G1 ( Z ) =- 0 G0 ( Z ) H1 (- Z ) t 1 (Z ) =

(5.55) (5.56)

Hence we get: G0 ( Z ) = H1 (- Z ) = 1 + Z -1 + Z -2 + Z -3 G1 ( Z ) = - H 0 (- Z ) = -(1 - Z -1 + Z -2 )

Exercise 5.7

Consider a two-channel perfect reconstruction filter bank, as shown in Fig. 5.12. H 0 = -(1 + Z -1 ) Find the other three filters, namely H1 ( Z ), G0 ( Z ) and G1 ( Z ) Hint: We know H1 ( Z ) = Z - D H 0 (- Z -1 ) In this simple case, let us take D = 1. Therefore, H1 ( Z ) becomes H1 ( Z ) = - Z -1 (1 - Z ) H1 ( Z ) = 1− Z

-1

G0 ( Z ) = H1 (- Z ) this gives, G0 ( Z ) = 1 + Z -1 G1 ( Z ) = - H 0 (- Z ) this gives, G1 ( Z ) = 1 − Z -1 It can be verified that this filter bank satisfies both Alias Cancelation and PR condition.

(5.57)

185

Conjugate Quadrature Filter Bank

Exercise 5.8

Why H1 ( Z ) is related to H 0 (- Z )? Express H 0 ( Z ) in terms of H1 ( Z ). Sol. In a perfect reconstruction filter bank, analysis low pass filter and high pass filter are complimentary filters, i.e. frequencies blocked by one filter must pass through another filter so that there is no loss of information and we can get a perfect reconstruction. H 0 ( Z ) is a low pass filter. H 0 (- Z ) has spectrum of shape similar to that of H 0 ( Z ) but all frequencies shifted by p. It is a high pass filter which is closely related to H1 ( Z ). The exact mathematical relation is derived as follows: we know H1 ( Z ) = Z - D H 0 (- Z -1 ) Replacing Z by - Z -1, we get H1 (- Z -1 ) = (-1) D Z D H 0 ( Z ) On rearrangement, we get: H 0 ( Z ) = (-1) D Z - D H1 (- Z -1 )

Assignment Exercise 5.9

Plot the frequency response (magnitude only) of (1 + Z -1 )n for n Î (1,2,3,)

Chapter

6

Daubechies Family

Introduction Impulse response of Daubechies analysis low pass filter Calculation of scaling function Daub-4 and Daub-6 design details

6.1

In search of scaling and wavelets coefficients

Introduction

In this chapter we will continue with the discussion of the Daubechies filter bank, which was briefly introduced in Chapter 5. The salient feature of Daubechies filter bank is that its construction depends on addition of polynomial of higher degree in filter transfer function. To be specific, more and more (1 - z -1 ) terms are utilized in high pass analysis filter bank.

Ingrid Daubechies Ingrid Daubechies (born 17 August 1954) is a Belgian physicist and mathematician. Between 2004 and 2011 she was the William R. Kenan, Jr. Professor in the mathematics and applied mathematics departments at Princeton University, New Jersey. In January 2011 she moved to Duke University as a Professor in mathematics. She is the first woman president of the International Mathematical Union (2011–2014). She is best known for her work with wavelets in image compression. At Courant in 1986 she made her best-known discovery: based on quadrature mirror filter-technology she constructed compactly supported continuous wavelets that would require only a finite amount of processing, in this way enabling wavelet theory to enter the realm of digital signal processing.

6.2

Impulse Response of Daubechies Analysis Low Pass Filter

The first member of the Daubechies family is the Haar filter bank itself. In the second member of Daubechies family the analysis side high pass filter has a factor of (1 - z -1 )2. Now, high pass filter of analysis side is of form z - D H 0 (- z -1 ), where H 0 ( z ) is the analysis side low pass filter. So H 0 ( z ) should have a factor of (1 + z -1 )2 .

188

Multiresolution and Multirate Signal Processing

Example 6.2.1 — D2 calculations.

It can be recalled that in the Daubechies family the filter lengths are always even. So, for the second member of the Daubechies family, filter length will be 4 and the order will be 3. So H 0 ( z ) has 3 zeros. Two of them are already specified to be at z = -1. The third zero is to be determined to get complete transfer function. The complete transfer function is obtained as follows: Let the impulse response be, h[ n] = [h0 h1 h2 h3 ] So h[ n] is orthogonal to its even shifts, e.g. shifts of 2,4,6 etc. So the only non-trivial relation is obtained by the dot product of h[ n] and h[ n - 2]. h[ n] = [h0 h1 h2 h3 ] h[ n - 2] = [...... h0 h1 h2 h3 ] n=2

Hence their dot product is (h0 h2 + h1h3 ) and because of orthogonality of dot product with respect to even shifts, h0 h2 + h1h3 = 0

(6.1)

H 0 ( z ) = h0 + h1 z -1 + h2 z -2 + h3 z -3

(6.2)

Now, system function can be expressed as,

Also, as H 0 ( z ) has a factor of (1 + z -1 )2 , so in general H 0 ( z ) can be written as, H 0 ( z ) = C0 (1 + z -1 )2 (1 + B0 z -1 ) where C0 is a constant. Here two zeros are constrained at z = -1. If we neglect the constant for the time being and then expanding the previous expression, we can write, H 0 ( z ) = 1 + 2 z -1 + z -2 + B0 z -1 + 2 B0 z -2 + B0 z -3 H 0 ( z ) = 1 + (2 + B0 )z - 1 + (1 + 2 B0 )z -2 + B0 z -3 Comparing the coefficients of powers of z -1 from Eqs. (6.2) and (6.3), we get, h0 = 1 h1 = 2 + B0 h2 = 1 + 2 B0 h3 = B0

(6.3)

189

Daubechies Family

Putting this value in Eq. (6.1), we get, (1 + 2 B0 ) + (2 + B0 ) B0 = 0 Þ B02 + 4 B0 + 1 = 0 Solving the above quadratic equation, we get, B0 = (-4 ± 2 3) / 2 = -2 ± 3

(6.4)

Now, the implication of B0 is that the third zero of H 0 ( z ) is at B0. For B0 = -2 - 3 , i.e. B0 = −3.732, the zero is outside the unit circle in the z-plane. Since B0 > 1, this will not become the minimum phase implementation. The magnitude response being the same as required, there will be more phase delay and group delay in the system which is undesirable. But for B0 = -2 + 3 , i.e. B0 = -0.268, the zero is inside the unit circle in the z-plane. Since B0 < 1, the system remains the minimum phase system. We, therefore, choose B0 = -2 + 3 , i.e. B0 = -0.268. So the impulse response of the analysis side low pass filter of length 4 of Daubechies family is as shown in Table 6.1. In this derivation process we have neglected the constant C0 . To find C0 , let us recall:

k 0 ( z ) + k 0 (- z ) = constant

(6.5)

k 0 ( z ) = H 0 ( z ) H 0 (- z -1 )

(6.6)

where, In order to choose the constant C0 the easiest option is to do is to make the norm of the impulse response of H 0 ( z ) unity in the sense of l2 norm. Now, the dot product of a sequence with itself gives the square of its l2 norm. So the sequence corresponding to k 0 ( z ) at the 0th location is essentially the squared norm in the l2 ( Z ) of [h0 h1 h2 h3]. So C0 chosen such as, C02 (h02 + h12 + h22 + h32 ) = 1 C02 = 1 / 4.287 = 0.233 C0 = 0.4829 Table 6.1

Impulse response of the analysis side low pass filter of length 4 of Daubechies family 1 1

6.3

(6.7)

(2 + B0) 1.732

(1 + B0) 0.464

B0 -0.268

Calculation of Scaling Function

The next step is to calculate f (t ) and y (t ) from the calculated impulse response. To calculate the scaling function f (t ) we have to compress and convolve h[ n] iteratively. This is done as follows: Let us treat h[ n] as the set of coefficients of an impulse train containing only 4 impulses in the continuous time domain, such as the function in the continuous time domain is h(t ) such as,

h(t ) = h0d (t ) + h1d (t - T ) + h2d (t - 2T ) + h3d (t - 3T )

(6.8)

190

Multiresolution and Multirate Signal Processing

Accordingly the shape of h(2t ), h(4t ), h(8t ) etc. are shown in Fig. 6.1. For the iterative convolution, first h(t ) is convolved with h(2t ) and the result is convolved with h(4t ). h0

h1

h2

h3

0.4829*1=0.4829

0. 0.4829*1.732=0.8364

0.4829*0.464=0.2241

0.4829*(-0.268)=-0.129

Then the result is convolved with h(8t ), and so on. If this convolution process with compressed version of h(t ) is carried on infinitely, we will get the scaling function f (t ). There is an interesting conclusion of this iterative convolution process. Suppose h(t ) has length L, i.e. h(t ) = 0 for any t < 0 and for any t > L . So h(2t ) has a length L / 2, h(4t ) has a length L / 4, h(8t ) has a length L / 8, and so on. Now, the convolution of h(t ) with h(2t ) gives result with length L + L / 2. This result when convolved with h(4t ), it gives a result with length L + L / 2 + L / 4. Continuing this way we can get f (t ), which is of length L + L / 2 + L / 4 + ... = 2 L. It means f (t ) is zero for any t < 0 and t > 2 L , i.e. we converge towards a compactly supported scaling function. The independent variable region over which the scaling function is nonzero is finite. This is the most important contribution made by Daubechies. Before Daubechies came up with these set of filter banks, idea of neatly constructing a family of compactly supported multiresolution analysis did not exist in record. So this is a very useful contribution in MRA.

h(t)

h(2t)

h(4t)

h(8t)

Figure 6.1

Shape of h(t ), h(2t ), h(4t ) and h(8t )

191

Daubechies Family

6.3.1

Interpretation for Daubechies Filter Banks

The theories of wavelets and filter banks developed in parallel. Using filter banks effectively to generate compactly supported scaling function is an important contribution by Daubechies. A different interpretation can be thought about the Daubechies filter banks. The high pass analysis filter bank essentially reduces the degree of a polynomial input. Suppose there is an input of the form x[ n] = a + bn, where a and b are constants. So the factor (1 - z -1 ) in the high pass filter reduces the degree of the input. If there had been only this term in the high pass filter, the output would have been in the form of y[ n] = a + bn - a - b(n - 1) = b . If there had been another term of (1 - z -1 ) the output would have become 0. So in Daubechies length 4 (abbreviated as Daub-4) high pass filter bank the polynomial is annihilated. On the other side, in the low pass analysis filter, because of the (1 + z -1 ) term, the output becomes y[ n] = a + bn + a + b(n - 1) = 2a + 2bn - b . It can similarly be extended for another (1 + z -1 ) term. This means that the polynomial form of expression remains in the low pass branch and the high pass branch contains some residual component, thereby retaining a few more smoother terms related to polynomial in the low pass branch and removing them from the high pass branch. While calculating the iterative convolution we saw that the scaling function thus obtained has a compact support. But it is important to note that had it taken any arbitrary values of h0 , h1 , h2 , h3 the iterative convolution process might not have converged to a function with finite number of discontinuities. But beauty of Daubechies family is that whatever be the filter length, the convolution always converges to a function with finite number of discontinuities. The specialty that makes the convolution to converge is denoted by a term ‘regularity’ in Wavelet literature, i.e. the filters need to obey regularity for the iterative convolution to converge. This regularity comes because of the presence of the zeros in the system function. One guaranteed way of forcing regularity is to introduce factors of (1 + z -1 ), i.e. adding zeroes at z = -1, i.e. w = p in the low pass analysis filters. In case of high pass analysis filter the zeros are added at z = 1, i.e. w = 0. So the zeros are put at the extreme high frequency in low pass filter and at the extreme low frequency in high pass filter. In case of different filter banks, the number of zeros are as listed in Table 6.2. Table 6.2 Daunechies family member and corresponding zeros Haar 1 zeros Daub-4 2 zeros Daub-6 3 zeros Higher is the filter length more regular is the Daubechies filter. This means the function to which we converge by iterative convolution becomes more and more smooth, i.e. they have more and more derivative which are continuous. In Daubechies-4 there are some issues in the differentiability but in the higher order filters that is also taken care of.

6.3.2

Next Daubechies Family Member Daub-6

The next member of Daubechies family is a length 6 filter of degree 5. In that case H 0 ( z ) can be written as,  z -1 )(1 + B  z -1 ) H 0 ( z ) = C0 (1 + z -1 )3 (1 + B 0 1

(6.9)

192

Multiresolution and Multirate Signal Processing

 and B  ). Let the impulse here C0 is a constant. Three zeros are constrained and two are free (B 0 1 response be, h[ n] = [h0 h1 h2 h3 h4 h5 ] so h[ n] is orthogonal to its even shifts, e.g. shift of 2, 4, 6, etc. The nontrivial relations are obtained by the dot product between h[ n] and h[ n - 2] or h[ n - 4]. Here, h[ n] = [h0 h1 h2 h3 h4 h5 ] h[ n − 2] = [...... h0 h1 h2 h3 h4 h5 ] ↑n=2 h[ n − 4] = [................. h0 h1 h2 h3 h4 h5 ] ↑n=2 Þ h2 h0 + h3 h1 + h4 h2 + h5 h3 = 0

(6.10)

Þ h4 h0 + h5 h1 = 0

(6.11)

 and B  . Therefrom we can From Eqs. (6.9), (6.10) and (6.11), we can find out the values of B 0 1 construct the impulse response in a way similar to Daub-4 case. This type of filter banks are called Conjugate Quadrature filter bank. The reason for this nomenclature is that the low pass and the high pass filter frequency responses are p apart from each other. So the principal equation governing the conjugate quadrature filter is, k 0 ( z ) + k 0 (- z ) = constant where, k 0 ( z ) = H 0 ( z ) H 0 ( z -1 ) k 0 (e jw ) = H 0 (e jw ) H 0 (e jw ) If the frequency response is real, then 2

H 0 (e jw ) = k 0 (e jw )

(6.12) jw

This means designing a conjugate quadrature filter bank is essentially designing k 0 (e ) only. k 0 ( z ) corresponds to a real and even impulse response with the constraints that even samples of the impulse response are all ‘0’ except at the 0th sample. There are many ways to design such filter banks. Once we have k 0 ( z ) we find its roots. For each root there are pairs of reciprocal roots H 0 ( z ) and H 0 ( z -1 ). Out of each reciprocal root pair, one root is assigned to H 0 ( z ) and the other automatically gets assigned to H 0 ( z -1 ). The Daubechies filters are one class of conjugate quadrature filters. Our future study will aim at what we are looking for out of these filter banks in both time and frequency domains.

6.4

Daub-4 and Daub-6 Design Details

This aims at using linear algebra and matrices. we wish to come up with design procedure to construct ‘even’ length Doubechies filter.

193

Daubechies Family

Let

h = {h0 , h1 , hL } be the LPF coefficients and g = {g0 , g1 , gL } be the HPF coefficients and

Let both these filters be finite impulse response filters ì1 1 1 ü For example, if h = {h0 , h1 , h2 } = í , , ý î4 2 4þ Then the Fourier series h(w ) will be 1 1 æ1 1 ö H (w ) = ç + × e jw + × e2 jw ÷ = e jw × (1 + cos w ) 4 2 è4 2 ø

(6.13)

If we plot the magnitude graph over [0, p ], then H(w)

1

w 0

\

p

low pass filter conditions could be 1 (1 + cos 0) = 1 and 2 1 H | (p ) | = (1 + cosp ) = 0 2 H | (0) | =

Now, L

H (w ) = å hk × e- jkw

(6.14)

k =0

Now if g = {g0 , g1 , g2 , gL } L

G(w ) = å gk × e- jkw k =0

(6.15)

194

Multiresolution and Multirate Signal Processing

High pass conditions will be | G(0) | = 0 To design Daub-4 and Daub-6,

and | G(p ) | = 1

L = 3 & 5. Now, let us construct system of linear and quadratic equations that coefficient of ’h’ show satisfy. Design 1- Daubechies-4 h = {h0 , h1 , h2 , h3 } g = {g0 , g1 , g2 , g3 } Typical output of any filtered is mathematically captured by ‘convolution’. x -

h

- y = x´h

L

yn = åhk × xn - k = h0 × xn + h1 × xn + 

(6.16)

k =0

\ For causal system and sequences (i.e. no y- 1, y- 2, and x- 1, x- 2, terms) é x0 ù é y0 ù ê ú êy ú ê 1 ú = H × ê x1 ú êú êú ê ú ê ú ë x2 û ë y2 û

(6.17)

then, ‘H’ can be written in matrix form as follows: Upper Triangle = 0

h1 h2 ...

h0 h1 h2

0 h0 h1

...

...

...

...

...

...

...

0 0 h0

0 0 0

0 0 0

0 0 ... 0 0 ... 0 0 ... .. .

...

h2 ... ...

...

... ... hL

...

... hL 0

...

hL 0 0

Co

Co

ns ta

nt

Co Mai n ta nsta Di ag nt nt Di D ona ag iag on on l al al

ns

Di

ag

on

al

Important points to note about ‘H’ matrix: 1. Since it represents a system in matrix form it is a system of linear equations. 2. ‘ho’s constitute main diagonal.

195

Daubechies Family

3. With reference to main diagonal, upper triangle is ‘0’, which suggests, system is causal éë Hij = 0 for all j > i ,i , j Î Z ùû 4. Since all diagonals are constant, it indicates, system is shift invariant. 5. Therefore, ‘H’ is a causal LTI system. Similarly, for high pass filter ‘G’ matrix can be built. ‘H’ and ‘G’ together constitute wavelet matrix ‘w’, as in wavelets we use filter bank, combination of LPF(H) and HPF(G). For example, if in case of Haar, ì 1 1 ü ì 1 -1 ü h[n] = í , , ý and g[n] = í ý î 2 2þ î 2 2þ ‘W’ matrix will be, éH ù W =ê ú ëG û and would look like 0ù é 12 12 0 ê 0 0 12 12 úú ê W = 2 × ê -12 12 0 0ú ú ê 0 -12 12 ú ê 0 úû êë

(6.18)

This is Daub-2 or Haar. Now, for Daub-4, the filter length will be ‘4’ and W matrix wil be 8 × 8. Let us call that as WD4.

WD4

é h3 ê0 ê ê0 ê ê h1 ê=ê ê g3 ê0 ê ê0 ê ê g1 ëê

h2 0 0 h0 g2 0 0 g0

h1 h3 0 0 g1 g3 0 0

h0 h2 0 0 g0 g2 0 0

0 h1 h3 0 0 g1 g3 0

0 h0 h2 0 0 g0 g2 0

0 0 h1 h3 0 0 g1 g3

0 0 h0 h2 0 0 g0 g2

ù ú ú ú ú ú é H ù D4 ú ê ú = ê - - - - - - úú ú ê úû ú ë GD 4 ú ú ú ú úû

Now, for WD4 to become a transformation matrix, following conditions should be met: 1. The inverse should exist. 2. WD4 should be unitary, i.e. for real values, WD-41 = WDT4. 3. WD4 should be orthogonal.

(6.19)

196

Multiresolution and Multirate Signal Processing

Assuming 1st two criteria, For the orthogonality, WD 4 × WDT4 = I \ WD 4 × WDT4

éHD4 ù éH = êê GD 4 úú ê D 4 úû ë êë

H D 4 × GDT 4 ù ú éI GD 4 × GDT 4 ú = ê 4 0 ú ë 4 û

é H D 4 × H DT 4 GD 4 ù ê T ú = ê GD 4 × H D 4 û ê ë

04 ù I 4 úû

\ H D 4 × H DT 4 = I 4

éh3 ê0 ê \ê 0 ê ê h1 êë

h2 0 0 h0

h1 h3 0 0

h0 h2 0 0

0 h1 h3 0

0 h0 h2 0

0 0 h1 h3

é h3 êh 2 0ù ê ê h 1 0 úú ê h 0 h0 ú × êê 0 ú h2 ú ê 0 úû ê ê0 ê êë 0

0 0 h3 h2 h1 h0 0 0

0 0 0 0 h3 h2 h1 h0

h1 ù h0 úú 0 ú é1 ú 0 g0 ú êê0 = 0 ú ê0 ú ê 0 ú ë0 h3 ú ú h2 úû

0 1 0 0

0 0 1 0

0ù 0 úú 0ú ú 1û

Let us work on 1st row, 1st column ® should lead to ‘1’ and 1st row, 2 nd column ® should lead to ‘0’ \ h32 + h22 + h12 + h02 = 1

(6.20)

h1 × h3 + h0 × h2 = 0

(6.21)

and

The orthogonality between ‘h’ and ‘g’ plays vital role. One way of understanding orthogonality is ‘zero’ dot product between two vectors. ‘Dot’ product is element-by-element product. For every ‘h’, if we create corresponding ‘g’ by flipping the sequence and making alternate samples go negative, then that will certainly achieve orthogonalization. For example, if h = {1,2,3, 4} then g = {4, -3,2, -1} is a good candidate to produce orthogonal framework. á h×ñ = [1 ´ 4 + 2 ´ (-3) + 3 ´ (2) + 4 ´ (-1)] = 0! Mathematically, this can be captured using: gk = (-1)k × hL - k, where, h = {h0 , h1 ,, hL} and g = {g0 , g1 ,, gL } This grantees á h, gñ = [ h × g ] = 0 \ g = {g0 , g1 , g2 , g3 } = {h3 , -h2 , h1 , -h0 }

197

Daubechies Family

Now, H D 4 × GDT 4 = [ 0 ]4´4

éh3 ê0 ê \ê 0 ê ê h1 êë

h2 0 0 h0

h1 h3 0 0

h0 h2 0 0

0 h1 h3 0

0 h0 h2 0

0 0 h1 h3

é -h0 êh 1 0ù ê ê -h2 0 úú ê h3 h0 ú × êê ú 0 h2 ú ê 0 úû ê ê 0 ê êë 0

0 0 -h0 h1 -h2 h3 0 0

0 0 0 0 -h0 h1 -h2 h3

-h2 ù h3 úú 0 ú é0 ú 0 ú êê0 = 0 ú ê0 ú ê 0 ú ë0 -h1 ú ú h2 úû

0 0 0 0

0 0 0 0

0ù 0 úú 0ú ú 0û

0 0 0 0 h3 h2 h1 h0

h1 ù h0 úú 0 ú é0 ú 0 g0 ú êê0 = 0 ú ê0 ú ê 0 ú ë0 h3 ú ú h2 úû

0 0 0 0

0 0 0 0

0ù 0 úú 0ú ú 0û

-h2 ù h3 úú 0 ú é1 ú 0 ú êê0 = 0 ú ê0 ú ê 0 ú ë0 -h1 ú ú h2 úû

0 1 0 0

0 0 1 0

Similarly, GD 4 × H DT 4 = [ 0 ]4´4

é -h0 ê 0 ê ê 0 ê ê -h2 êë

-h2 -h0 0 0

h1 0 0 h3

h3 h1 0 0

0 -h2 -h0 0

0 h3 h1 0

0 0 -h2 -h1

é h3 êh 2 0ù ê ê h 1 0 úú ê h 0 h3 ú × êê ú 0 h2 ú ê 0 úû ê ê0 ê êë 0

0 0 h3 h2 h1 h0 0 0

Also, GD 4 × GD’ 4 = I 4

é -h0 ê 0 ê \ê 0 ê ê -h2 êë

h1 0 0 h3

-h2 -h0 0 0

h3 h1 0 0

0 -h2 -h0 0

0 h3 h1 0

0 0 -h2 -h1

é -h0 êh 1 0ù ê ê h 2 0 úú ê h 3 h3 ú × êê 0 ú h2 ú ê 0 úû ê ê 0 ê êë 0

0 0 -h0 h1 -h2 h3 0 0

We again confirm, h02 + h12 + h22 + h32 h0 h2 + h1h3

=1 =0

0 0 0 0 -h0 h1 -h2 h3

0ù 0 úú 0ú ú 1û

(6.22)

198

Multiresolution and Multirate Signal Processing

Now, let us impose conditions so that ‘h’ is truly a low pass filter on [0,p ] Being a LPF, it should pass frequencies at w = 0 and block frequencies at w = p . Let H(w ) be the Fourier representation of h(n) in the form of series, H (w ) = h0 + h1 × e jw + h2 × e2× jw + h3 × e3× jw

(6.23)

Let us impose first condition to pass frequencies at w = 0, and let the magnitude by ‘Unit’ be decided by us. \ H (w ,0) = h0 + h1 × e0 + h2 × e0 + h3 × e0 = Unit (6.24) we choose the unit carefully to normalize orthogonal system, thus making it orthonormal. Recall the MRA framework we saw, hk is linked with f (×) and sk is linked with y (×). For set ì 2j ü j j í2 f (2 x - k )ý, being orthonormal and linearly independent, it constitutes a basis for vector space V î þ at window of analysis Wa = 12 j . This is truly orthonormal on (-¥, ¥), since j ì0, k ¹ l j j 2 2 (6.25) ò-¥ f (2 x - k ) × f (2 x - l )dx = íî1, k = l j This is condition for orthogonality,whole makes it orthonormal is normalizing factor of 2 2 in ¥

ì 2j ü j í2 f (2 x - k )ý. î þ If we take norm of f (2 j x - k ), we get that factor. ¥

 f (2 j x - k ) 2 = ò f 2 (2 j x - k )dx = -¥

1 2j

ò

¥

f 2 (2 j x - k ) × 2 j dx

-¥

(6.26)

Let us put 2 j x - k = a Differentiating both sides, we get 2 j × dx - 0 = da \2 j dx = da  f (2 j x - k ) 2 =

1 2j

ò

¥

f 2 (a)dx

(6.27)

-¥

Let us prove it for Haar, where f (a) 2 1 a

0 0

 f (2 j x - k ) 2 =

1 2j

1 1

2

1

ò (1) da = 2 0

2

j

(1) =

1 2j

199

Daubechies Family

Taking square root for positives,  f (2 j x - k ) 2 =

1 2j

\ We can divide f (2 j x - k ) by its norm to get orthogonal basis coverted to orthonormal basis, j

j

f (2 j x - k ) = 2 2 x - k ) = 2 2 f (2 j x - k ) 1 j

j2 From the point of view of nested subspaces, we have to account for 2 every time we move from one subspace to another. ...

√2

√2

. . . V−2

⊂ V−1

⊂

√2 V0

⊂

√2 V1

⊂

... V2

...

This is a ‘dyadic’ style of realizing discrete wavelet filters. \ We choose ‘Unit’ to be ‘ 2’ to maintain orthogonality. We will put this in equation (6.23), h0 + h1 + h2 + h3 = ± 2

(6.28)

We will use this as a final check whether the LPF coefficient satisfy this condition or not. Let us impose second low pass condition of complete attenuation at w = p . \

H (p ) = 0 = h0 + h1 × e jp + h2 × e2 jp + h3 × e3 jp

Using euler’s identity e jp = −1 e2 jp = (e j p )2 = (-1)2 = 1 e3 jp = (e j p )3 = (-1)3 = -1 H (p ) = 0 = h0 - h1 + h2 - h3 ‘gk ’ being a highpass filter, let’s impose conditions accordingly, gk = {g0 , g1 , g2 , g3 } \ Put {g0 , g1 , g2 , g3 } =

G(w ) = g0 + g1 × e jw + g2 × e2 jw + g3 × e3 jw

{

+

h3 ,- h2 ,+ h1 ,- h0

}

(6.29)

200

Multiresolution and Multirate Signal Processing

\ G(0) = 0

G(w ) = h3 - h2 × e jw + h1 × e2 jw - h× e3 jw -(block low frequencies)

\

h3 - h2 + h1 - h0 = 0

(6.30)

G(p ) = unit = 2 | h0 + h1 + h2 + h3 |= 2

(another check condition)

Let us combine two orthogonality conditions and one LPF condition towords finding hk From Eqs. (6.20), (6.21) and (6.29) h02 + h12 + h22 + h32 = 1 h0 h2 + h1h3 = 0

____________________(D 1)

h0 − h1 + h2 − h3 = 0

These are our design Equations! let ’s start with Eq. (6.21) h0 + h2 h1h3 = 0 This clearly implies [h0 h1 ]T and [h2 h3 ]T are orthogonal, i.e. inner or dot product < [h0 h1 ],[h2 h3 ] > suggests h0 h2 + h1 + h3 ® going to ‘0’ For example, h0 = 1, h1 = 1, h2 = - 1, h3 = 1 ensures [ho h1 ]T ^ [h2 h3 ]T We can also prove graphically [h2 h3 ]T ^ c[ -h1 h0 ]T (−1,1) = [h2 h3]

(1,1) = [h0 h1] orthogonal

Therefore, it is easy to see that [h2 h3 ]T ^ c[ -h1 h0 ]T

(6.31)

201

Daubechies Family

with c = 1 in this case! For c ¹ 0, if we insert

[h2 h3 ]

T

^ c [ -h1h0 ] in Eq. (6.21) T

We get h02 + h12 =

1 1 + c2

(6.32)

One way of looking at Eq. (6.31) is h2 = -ch1 and h3 = ch0 Let’s plug these values in Eq. (6.29) \ h0 - h1 - ch1 + ch0 = 0 \ h0 (1 - c) + h1 (1 + c) = 0

æ 1- c ö h1 = ç ÷ h0 = {c ¹ l è 1+ c ø

(6.33)

1 Therefore Eq. (6.32) is clearly a equation of circle of radius and Eq. (6.33) is a equation of 1 + c2 1- c straight line with slope 1+ c Graphically, h20 + h21 =

1 1+c2

h1

1+c √2.(1+c2)

1−c . h h1 = ) 1+c ) 0

h0

−(1+c) √2.(1+c2)

202

Multiresolution and Multirate Signal Processing

1 1- c intersects circle of radius twice 1+ c 1 + c2 \ For each ‘c’ there are 2 possible solutions, that will satisfy design conditions D1.

\ For any given c ¹ −1, the line with slope of

Let’s simplify h02 + h12 =

1 1 + c2

2

1 æ1- c ö 2 h02 + ç ÷ × h0 = 1 + c2 è1+ c ø æ æ1- c ö h02 ç 1 + ç ç è 1 + c ÷ø è éæ (1 + c)+ (1 - c)2 h02 êç (1 + c)2 ëè éæ (1 + 2c + c 2 + 1 - 2c + c 2 h02 êç (1 + c)2 ëè \ 2h02 = \ h02 =

2

ö 1 ÷= ÷ 1 + c2 ø

öù 1 ÷ú = 2 øû 1 + c 2 öù 2 é 2(1 + c ) ù ÷ ú = h0 ê 2 ú øû ë 1+ c û

1 (1 + c)2 × 1 + c2 1 + c2 1 + c2 2(1 + c 2 )2

\ h0 = ± Let us choose (+ve)root

1+ c 2(1 + c 2 )

« the ‘two’ solutions!

1+ c

h0 = +

(6.34)

2(1 + c 2 )

By plug and solve h0 =

1- c 2(1 + c ) 2

, h2 = +

-c(1 - c) 2(1 + c ) 2

, h3 = +

-c(1 + c) 2(1 + c 2 )

D1 is a system of three equations where we have to figure out four unknowns Let us add one more LP condition H(p ) = 0 As ideal filtering is not possible, we impose H ¢(p ) = 0 First-order difference (derivative) will give us, H"(w ) |w =p = jh1e jw + 2 jh2 e2 jw + 3 jh3 e3 jw |w =p

203

Daubechies Family

(

0 = j h1 (-1) + 2h2 (-1)2 + 3h3 (-1)3

)

\ h1 - 2h2 + 3h3 = 0 Let us add Eq. (6.35) in D1 to get complete set D2

(6.35)

h02 + h12 + h22 + h32 = 1 h0 + h2 h1h3 = 0

______ ________________(D2)

h0 − h1 + h2 − h3 = 0 h1 − 2h2 + 3h3 = 0

Now, let us plug h2 = -ch1 and h3 = ch0 in Eq. (6.35) h1 + 2ch1 + ch0 = h1 (1 + 2c) + 3ch0 = 0 ö æ 3c h1 = - ç ÷ (1 + 2 c ) × h 0 ø è

\

we already know the slope matches with

1- c 1+ c

\

-3c 1 - c = 1 + 2c 1 + c

\

-3c - 3c 2 = 1 + 2c - c - 2c 2

\

-3c - 3c 2 = 1 + c - 2c 2

\

c 2 + 4c + 1 = 0

(6.36)

Roots of Eq. (6.36) will be c = -2 ± 3 c = -2 - 3 makes system slow and sluggish, as the zero lies outside the unit circle and it ensures max-phase system. \ we choose minimum phase c = -2 + 3 solution by backward substitution in D2

h0 = h3 =

1+ 3 4 2 1- 3 4 2

h1 =

3+ 3

h2 =

4 2 3- 3 4 2

204

Multiresolution and Multirate Signal Processing k Now, gk = (-1) h3 - k, k = 0,1,2,3

g0 = h3 = g2 = h1 =

1- 3

g1 = -h2 = -

4 2 3+ 3

g3 = -h0 -

4 2 Now let us repeat this for Daub-6 where

3- 3 4 2 1+ 3 4 2

h = {h0  h5} g = {g0  g5} é h5 ê0 ê ê0 ê ê h1 êh ê 3 W10 = ê êg ê 5 ê0 ê ê0 ê g1 ê ë g3

h4 0 0 h0 h2 g4 0 0 g0 g2

h3 h5 0 0 h1 g3 g5 0 0 g1

h2 h4 0 0 h0 g2 g4 0 0 g0

h1 h3 h5 0 0 g1 g3 g5 0 0

h0 h2 h4 0 0 g0 g2 g4 0 0

0 h1 h3 h5 0 0 g1 g3 g5 0

0 h0 h2 h4 0 0 g0 g2 g4 0

0 0 h1 h3 h5 0 0 g1 g3 g5

0ù 0 úú h0 ú ú h2 ú h4 ú é H D 4 ù ú - ú = êê - - - úú 0 ú êë GD 4 úû ú 0ú ú g0 ú g2 ú ú g4 û

We can extend D1 conditions directly h02 + h12 + h22 + h32 + h42 + h52 = 1 h0 h2 + h1h3 + h2 h4 + h3 h5 = 0 h0 h4 + h1h5 = 0

__ _______________(D 3)

g = h5 , h4 , h3 , h2 , h1 , h0 \

H D10 × H D10T = GD10 × GD10T = I 5 and

\

H D10 × GD10T = GD10 × H D10T = 05

Low pass conditions

H (p ) = 0 = h0 - h1 + h2 - h3 + h4 - h5 H(0) = 2 h0 + h1 + h2 + h3 + h4 + h5 = 2

205

Daubechies Family

additional conditions on H(w ) will be H ¢(p ) = 0 and further flatten at w = p by H ¢¢ = 0 \

H (w ) = h0 + h1e jw + h2 e2 jw + h3 e3 jw + h4 e4 jw + h5 e5 jw

\

H ¢(w ) = j × h1e jw + 2 j × h2 e2 jw + 3 j × h3 e3 jw + 4 j × h4 e 4 jw + 5 j × h5 e5 jw

using

j 2 = ( -1)2 = -1 H ¢¢(w ) = - h1e jw - 4h2 e2 jw - 9h3 e3 jw - 16h4 e4 jw - 25h5 e5 jw

let us evaluate @ w = p \

H ¢(p ) = 0 = h1 - 2h2 + 3h3 - 4h4 + 5h5

\

H ¢¢(p ) = 0 = h1 - 4h2 + 9h3 - 16h4 + 25h5

Therefore, complete set of design equations is: h02 + h12 + h22 + h32 + h42 + h52 = 1 h0 h2 + h1h3 + h2 h4 + h3 h5 = 0 h0 h4 + h1h5 = 0 h0 - h1 + h2 - h3 + h4 - h5 h1 - 2h2 + 3h3 - 4h4 + 5h5 h1 - 4h2 + 9h3 - 16h4 + 25h5

____________________________(D 4)

solving we get Daub-6 coefficients

( ) 2 h = 5 + 10 + 3 5 + 2 10 ) 32 ( 2 h = 10 - 2 10 + 2 5 + 2 10 ) 32 ( 2 h = 10 - 10 - 2 5 + 2 10 ) 32 ( 2 h = 5 + 10 - 3 5 + 2 10 ) 32 ( 2 h = 1 + 10 + 5 + 2 10 ) 32 (

h0 =

1

2

3

4

5

2 1 + 10 + 5 + 2 10 32

206

Multiresolution and Multirate Signal Processing

gk = (-1)k h5 - k, k = 0, 5

and

g0 = h5 =

(

2 1 + 10 + 5 + 2 10 32

g1 = -h4 = g2 = h3 =

(

(

(

2 10 - 2 10 + 2 5 + 2 10 32

( 2 g = -h = 1+ 32 ( g4 = h1 =

5

6.5

) 5 + 2 10 )

2 5 + 10 - 3 5 + 2 10 32

2 10 - 10 - 2 32

g3 = -h2 = -

)

2 5 + 10 + 3 5 + 2 10 32

0

10 + 5 + 2 10

)

)

)

Towards Searching Scaling and Wavelets Coefficients

This complete idea of looking at different members and variants of the family takes us to basic approach which aims at searching for scaling and wavelet coefficients. In Chapter 2, we laid down a framework of multi resolution analysis which gave us the power to move up or down the ladder as required. This made the entire analysis ‘scalable’. This framework led us to two important questions: 1. How do we go about selecting the mother wavelet and scale of analysis? 2. What is the procedure to calculate scaling and wavelet coefficients? The first question will get answered subsequently. Now, let us answer the second question here. We have discussed so far Haar and Daubechies wavelets families. The coefficients of low pass and high pass filters for these are well known. However, where exactly we get these coefficients from? Do we have a concrete procedure for finding them? What properties should the scaling equation obey? These are few questions we would try to answer now. Quest to find scaling equation coefficients We are familiar with Haar wavelet and scaling function, which is given as, 1 ì ï1 0£x< 2 ï 1 ï y (t ) = í-1 £ x N - 1. Hence, N is the length of the sequence. Theorem III: If the scaling coefficients {hk } satisfy the condition for existence and orthogonality of f (t ), then

y ( x ) = å k 2f (2 x - t )

where, gk = ±(-1)k hN - k and

ò

¥

y ( x - l )f ( x - k )dx = d 1,k = 0, l ¹ k

-¥

From these theorems, we will try to find out coefficients of scaling equation. To do that following are some properties that scaling coefficients have to obey, 1.

åh

2.

åh

3.

åh

4.

å|h åh å2 h

5. 6.

k

= 2 1

=

2k

2 k +1

k

2 =

1 2

|2 = 1 h = d l ,0

k -2l k

h

k -2l k -2 j

= d l, j

Where exactly do these properties came from? We would first delve on that and then we would solve one example with case study of Haar f (t ).

208

Multiresolution and Multirate Signal Processing

Consider property 1 of f (t ),

åh

k

= 2

k

This property is dependent on the kind of normalization we make use of. Rest of the properties are in fact dependent on this property. So we would first understand from where this property comes from. We know the dilation equation, f (t ) = åhk 2f (2t - k ) k

Integrating both the sides, ¥

ò

¥

f (t )dt = åhk 2 ò f (2t - k )dt

-¥

-¥

k

dx 2

2t - k = x Þ dt =

Put,

ò

¥

1 ¥ f (t )dt 2 ò-¥

f (t )dt = åhk 2

-¥

k

¥

Let, ò f (t )dt, which has normalization 1. Then we are left with, -¥

1 2

1 = åhk 2 k

åh

k

= 2

(6.39)

k

The point to note is that Eq. (6.39) holds true if we consider the normalization to be 1. At this point, let us assume that property 5 holds true and we would use it to derive the remaining properties. The property number 5 tells us that,

åh

h = d lo

k -2 l k

k

Replacing l by −l we have, ¥

åh

h

h +

åh

d - lo =

k + 2l k

k = -¥

Separating ‘even’ and ‘odd’ terms, ¥

d - lo =

åh

2 k + 2l 2 k

k = -¥ ¥

åd -lo =

l = -¥

¥

¥

h

2 k +1+ 2 l 2 k +1

k = -¥

¥

å å [h

h + h2 k +1+ 2l h2 k +1 ]

2 k + 2l 2 k

l = -¥ k = -¥

However, we know that ¥

åd

- lo

=1

l = -¥

Thus,

¥

1=

¥

å h [ åh

2 k + 2l

2k

k = -¥

l = -¥

]+

¥

åh

2 k +1

k = -¥

¥

[ å h2 k +1+ 2 l ] l = -¥

(6.40)

209

Daubechies Family

By substituting l = l - k we can write, ¥

¥

åh

2 k + 2l

åh

=

l = -¥

2l

=A

(6.41)

2 l +1

=B

(6.42)

l = -¥

¥

¥

åh

=

2 k +1+ 2 l

l = -¥

åh

l = -¥

From Eqs. (6.40), (6.41) and (6.42) we can write, 1 = AA + BB

(6.43)

1 = A2 + B 2

(6.44)

which clearly represents equation of a circle with radius 1. However, there are two unknowns A and B and we have just one equation. So we need another equation. This equation comes out of the first property, i.e. åhk = 2 k

Splitting ‘even’ and ‘odd’ terms,

2 = åh2 k + åh2 k +1 k

k

2 = A+ B

(6.45)

At this point we have an interesting situation, i.e. Eq. (6.44) represents a circle and Eq. (6.45) a line. The situation is as shown in Fig. 6.2. B 20.5 (1/20.5,1/20.5) (1/20.5)

1

A

(1/20.5)

20.5 A2+B2=1

Figure 6.2

Scaling function and wavelet function of Daub-4

Hence from the figure we have, A= B=

1 2 1 2

Thus by substituting A we have,

åh

2k

k

=

1 2

(6.46)

210

Multiresolution and Multirate Signal Processing

which gives us property number 2. Also, substituting the value of B we have,

åh

2 k +1

=

1

(6.47)

2

k

which gives property number 3. Now, consider property number 4 we have,

å|h

|2 = åhk2 = 1

k

k

(6.48)

k

Now let us verify these properties in case of Haar scaling function. We know that in case of Haar f (t ), h0 = h1 =

1 2

Putting these values in property 1 we have, 1

åh

= h0 + h1 = 2

åh

2k

= h0 =

2 k +1

= h1 =

k

0

Thus property 1 is obeyed. Similarly, for property 2, 3 and 4, 1 2

k

åh k

1 2

{åh2 k }2 + {åh2 k +1}2 = h02 + h12 = 1 k

k

Hence, we can clearly see that these properties are obeyed. For property number 5 we have,

åh

h = d 0,l

k -2 l k

k

We have only two indices, i.e. 0,1. Let us take the first case, Case (i) k - 2l = 0 or k = 2l if l ¹ 0, then k = 2l ³ 2. Thus sum vanishes unless l = 0 and in that case k = 0. Thus we can write, 1 åk hk - 0hk = h0 h0 = 2

(6.49)

Also when k = 0, hk - 2 l = h0 - 2l, which is 0 unless l = 0. Thus,

åh h

k k

k

= h02 =

1 2

(6.50)

From Eqs. (6.49) and (6.50) we can see that the property 5 is obeyed since their results add up to give 1.

211

Daubechies Family

Case (ii) k - 2l = 1, k = 2l + 1 if l ¹ 0, k ³ 3. Thus l has to be 0 and k = 1. Hence we have, = h12 =

1 2

(6.51)

- 0hk = h1h1 =

1 2

(6.52)

åh h

k k

k

Now for k = 1, l = 0 hence,

åh

k

k

Similar to case (i), Eqs. (6.51) and (6.52) add up to give 1, hence for case (ii) also property 5 is obeyed. It is important to note that the property 5 gives us idea about the orthogonality and not all the functions obey this property. Hence, the Haar scaling coefficients obey all these different properties. Also once we find out these scaling coefficients we can find out coefficients of wavelet equation as follows, gk = (-1)k hN - k For N = 1 we have, gk = (-1)k h1 - k For k = 0, g0 = (-1)0 h1 = h1 = For k = 1, g1 = (-1)1 h0 = -h0 =

1 2 -1

2 This is how we can find out coefficients of scaling and wavelet equation.

Exercises Exercise 6.1

What is the condition for a system function to be Minimum Phase System? Hint: A system function H ( z ) is said to be a minimum phase if all of its poles and zeros are within the unit circle. Minimum Phase System have property that the system function and its inverse are causal and stable.

Exercise 6.2

Prove that autocorrelation of sequence h[ n] has a z-transform of H ( z ) H ( z -1 ). Hint: Autocorrelation of h[ n] is defined as, n = +¥

rhh (l ) =

å h[n]h[n - l ]

n = -¥

(6.53)

212

Multiresolution and Multirate Signal Processing

Now, put n - l = -k in Eq. (6.53), we get k = +¥

rhh (l ) =

å h[-k + l ]h[-k ]

k = -¥

k = +¥

rhh (l ) =

å h[-k ]h[l - k ]

(6.54)

k = -¥

RHS of above Eq. (6.54) can be written as convolution from given as follows, h(l ) ´ h(-l ) =

k = +¥

å h[-k ]h[l - k ]

(6.55)

k = -¥

Replacing variable l by n, we can write Eq. (6.55) as follows: h(n) ´ h(- n) =

k = +¥

å h[-k ]h[n - k ]

(6.56)

k = -¥

So, from Eqs. (6.53), (6.54) and (6.56), we can say that, rhh (l ) = h(n) ´ h(- n)

(6.57)

Now for calculating Z-transform of autocorrelation function. Let z-transform of h[ n], h[ n] « H ( z )

(6.58)

So by using property of Z-transforms, Z-transform of h[ -n], h[ -n] « H ( z -1 )

(6.59)

Also convolution property of Z-transform that is x1[ n] ´ x2 [ n] « X1 ( z ) X 2 ( z )

(6.60)

Now taking the Z-transform of Eq. (6.57), and using properties given in Eqs (6.58), (6.59) and (6.60), we can write Z-transform of autocorrelation function, Rhh ( z ) = H ( z ) H ( z -1 ) So, Eq. (6.61) gives the Z-transform of autocorrelation function.

(6.61)

213

Daubechies Family

Exercise 6.3

Why are the Daub series analysis LPF impulse responses orthogonal to its even shifted versions? Hint: The basic equation for designing the analysis side LPF is, k 0 ( z ) + k 0 (- z ) = C

(6.62)

where, k 0 ( z ) = H 0 ( z ) H 0 ( z -1 )

(6.63)

C is constant here. Taking inverse Z-transform of Eq. (6.63), we get, k 0 ( n) = R ( n)

(6.64)

where k 0 (n) is the Z-transform of k 0 ( z ) and R(n) is the autocorrelation of the impulse response of H 0 ( z ). So taking the inverse Z-transform on both sides of Eq. (6.62), we get R(n) + (-1)n R(n) = Cd (n)

(6.65)

This equation implies only for n even, (n ¹ 0) 2 R( n) = 0 R( n) = 0 Now R(n) is the dot product of the impulse response of the analysis LPF with its shifted version. So, for even n, R(n) = 0 implies that the dot product of the impulse response of the analysis LPF with its even shifted version is zero, i.e. impulse responses of these filters are orthogonal to its even shifted versions.

Exercise 6.4

Why are the Daub series analysis LPF impulse response has even length? Hint: Let a Daub analysis LPF is of odd length 5. So the impulse response is, h[ n] = [h0 h1 h2 h3 h4 h5 ] ↑ h[ n - 4] = [................. h0 h1 h2 h3 h4 h5 ] ↑n=4 Now, the dot product of h[ n] and h[ n - 4] should be 0 because of the orthogonality with respect to even shift. For this to be true, h0 h4 = 0, which means either h0 or h4 is zero which effectively means that the filter length is even and our initial assumption was wrong.

214

Multiresolution and Multirate Signal Processing

Exercise 6.5

Plot the scaling function and wavelet function for Daub-4 MRA. Low pass filter impulse response: h0 = 0.4829, h1 = 0.8364, h2 = 0.2241, h3 = 0.129 High pass filter impulse response: g0 = 0.129, g1 = 0.2241, g2 = 0.8364, g3 = 0.4829 Hint: Use MATLAB program to generate the plots of scaling function and wavelet function.

Chapter

7

Time-Frequency Joint Perspective

Introduction Analysis of Haar scaling function in time and frequency domain Analysis of Haar wavelet function in time and frequency domain Summary Ideal Time-Frequency Behaviour Frequency localization by f(t ) and Y(t ) Time localization and frequency localization

7.1

Introduction

In Chapter 6 we studied steps required to design conjugate quadrature filter-banks. We set out with the aim to achieve systematic design steps for building higher-order filter banks and learn relationships between analysis and synthesis filters subject to alias cancellation and perfect reconstruction conditions. Such analysis also enabled us to design higher and higher order filters of Daubechies family by incorporating more (1 + z -1 ) terms in analysis of low pass filter transfer function for achieving regularity required for convergence. Till this point, our analysis of scaling and wavelet functions has been either in time domain or in transform domain. However, several tasks require localization in both time and transform domain simultaneously. Wavelet transforms arrive from family of transforms, which provide simultaneous localization known as multi-resolution analysis of the underlying signal. In this chapter, we analyze Haar scaling and wavelet functions for time and frequency localization. We analyze their frequency domain behaviors over the containment ladder. This analysis gives us a reason to move towards ideal aspirations for scaling and wavelet transforms and in turn towards the basic question of bound over simultaneous localization in time and frequency domain also known as the ‘uncertainty principle’.

7.2

Analysis of Haar Scaling Function in Time and Frequency Domain

Consider Haar scaling function as shown in Fig. 7.1.

216

Amplitude

Multiresolution and Multirate Signal Processing

2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 −1

Haar Scaling Function

−0.5

Figure 7.1

0.5 Time

0

1

1.5

2

Haar scaling function

If we denote W as analog angular frequency variable and fˆ (Ω) as Fourier transform of scaling function then Fourier transform can be carried out in the following way: +¥ fˆ ( W ) = ò f (t )e- jWt dt -¥

- jW ˆ W) = 1 - e f( jW é æ W öù - j W ê sin ç ÷ú ˆ W) = e 2 ê è 2 ø ú f( ê æWö ú ê çè 2 ÷ø ú û ë

Magnitude

Magnitude for this scaling function is shown in Fig. 7.2. The above analysis of scaling function is applicable to subspace V0 which is a subspace having piecewise constant approximation over standard unit interval ]n, n+1[. Consider a ladder of subspace ... Ì V-2 Ì V-1 Ì V0 Ì V1 Ì V2 ... . By the scaling property of Fourier transform we have, Fourier transform of Haar scaling function 1 0.9 0.8 0.7 0.6 f(Ω) = sin 0.5Ω 0.5Ω 0.5 0.4 0.3 0.2 0.1 0 −20−15−10 −5 0 5 10 15 20 Analog angular frequency Figure 7.2

Magnitude of Fourier transform for haar scaling function

217

Time-Frequency Joint Perspective F .T .

f (t ) « fˆ (W) F .T .

f (at ) «

1 ˆ æ Wö f ; " a Î  - {0} | a | çè a ÷ø

As translation only affects phase in frequency domain, we consider scaling function without translation in time domain without loss of generality (as our analysis is limited to magnitude only). In this case, above relationship may be represented in the following manner. F .T .

f (t ) « | fˆ (W) | F .T .

f (2 m t ) « |

1 ˆ W f( ) | | 2m | 2m

Using this relationship, scaling function in time and frequency domain across various subspaces may be sketched as shown in Fig. 7.3.

Haar scaling function: V−1

2

1.8

1.8

1.6

1.6

1.4

1.4 Magnitude

Amplitude

2

1.2 1 0.8

1.2 1 0.6

0.4

0.4

0.2

0.2 0

0.5

1 Time t Φ 2

1.5

2

(a)

0.8

0.6

0 −0.5

Haar scaling function in frequency domain: V−1

0 −40 −30 −20 −10 0 10 20 Analog angular frequency

2.5

((

Haar scaling function: V0

2

1

1.8

0.9

1.6

0.8

1.4

0.7 Magnitude

Amplitude

2

1.2 1 0.8

0.2

0.2

0.1 1 Time

1.5

2

2.5

0.5Ω

Haar scaling function in frequency domain: V0

0.5 0.3

0.5

sin 0.5Ω

(b)

0.4

0.4

0

0 −40 −30 −20 −10 0 10 20 Analog angular frequency sin Ω 2 Ω 2

Φ(t)

Figure 7.3

40

0.6

0.6

0 −0.5

30

(Continued)

30

40

218

Multiresolution and Multirate Signal Processing Haar scaling function: V1

2

0.5 0.45

1.6

0.4

1.4

0.35 Magnitude

Amplitude

1.8

1.2 1 0.8

0.3 0.25

(c)

0.2 0.15

0.6 0.4

0.1

0.2

0.05

0 −0.5

Haar scaling function in frequency domain: V1

0

0.5

1 Time

1.5

2

0 −40 −30 −20 −10 0 10 20 Analog angular frequency

2.5

Φ(2t)

40

sin Ω 4 Ω 4 Haar scaling function in frequency domain: V2 1 2

Haar scaling function: V2

2

30

0.25

1.8 0.2

1.6 Magnitude

Amplitude

1.4 1.2 1 0.8

0.15

(d)

0.1

0.6 0.05

0.4 0.2 0 −0.5

0

0.5

1 Time

1.5

2

2.5

0 −40 −30 −20 −10 0 10 20 Analog angular frequency

Φ(4t) 1 4

Figure 7.3

30

40

sin Ω 8 Ω 8

Haar scaling function at various time and frequency resolutions.Scaling function and frequency domain characteristic in 3(a): V-1 subspace, 3(b): V0 subspace, 3(c): V1 subspace and 3(d):V2 subspace. [Note how localization varies across different subspaces starting from coarse subspace V-1.]

As evident from Fig. 7.3, as we go from subspace V-1 towards V2, subspace localization in time improves by factor of two, i.e. in V0 subspace scaling function lies in [0 1] interval (nonzero values), whereas in V1 subspace, scaling function lies in [0 0.5] interval. In this sense, we may say that time localization gets better and better as we move from V-¥ to V+¥. Similarly, localization in frequency gets poorer by factor of two as we go from subspace V-1 to V2, i.e. width of main lobe doubles each time we move to higher subspace. Hence, frequency localization gets poorer as we move from V-¥ to V+¥.This localization affects projection of some signal x(t ) Î L2 () on various subspaces. Example 7.2.1 — Orthonormal basis.

For orthonormal basis at some subspace Vm , f (2 m t - n), f (2 m t - l )

219

Time-Frequency Joint Perspective

where, < a, b > represents dot product between functions a and b. As mentioned, when n ¹ l, scaling functions do not overlap with each other and hence yield zero dot product. We shall now check what happens when n = l . Norm of the scaling function is 2

f(2 m ×) 2 = ò

2- m

0

(1)2 dt = 2 - m

where,  × 22 represents square of L2 norm and ‘×’ represents corresponding argument. Here, argument is used to denote that this relationship is true for any valid translation. Using orthonormal bases we can write m

Vm = span{2 2 f (2 m t − n)} ∀ n, m ∈ m m Projection of signal x(t ) over this subspace may be denoted as x(×),2 2 f (2 × - n) .

For understanding relationship between dot product in time and frequency domain we shall interpret what happens when we take Fourier transform and inverse Fourier transform. Ù

+¥

Ù

If, x(t ) « x (W) then x (W) = ò x(t )e- jWt dt . During this process, we take components of x(t ) along -¥

directions provided by complex exponentials e- jWt. Also, inverse Fourier transform gives x(t ) back from 1 +¥  x(W)e jWt d W, which may be interpreted as reconstruction of x(t ) from components along e jWt. 2p ò-¥ R Now, from Parseval’s theorem: ∧

∧

x(t ), y(t ) = x (Ω), y (Ω) i.e. dot product is independent of basis we select and is equal in time and angular frequency domain within a factor of 2 p . Using this relationship we may denote the relationship between projection of function x(t ) over subspace Vm in the following manner: m

x(t ),2 2 f (2 m × - n) =

m 1 Ù x (W),2 2 f (2 m × - n) 2p

In Fig. 7.4, solid line shows Fourier transform of Haar scaling function and dotted line shows Fourier transform of signal under consideration, namely x(t ). Contribution of side lobes towards overall dot product is unsubstantial compared to contribution from main lobe. In other words, frequencies of x(t ) inside main lobe of fourier transform of scaling function are emphasized with respect to frequencies outside the main lobe. As we move across the subspace ladder different amount of frequencies are emphasized. To be more precise, as we go from V-¥ to V+¥, more and more frequencies are emphasized as peak always remains on zero frequency.

220

Multiresolution and Multirate Signal Processing

Dot product in frequency domain 0.5 0.45 0.4

Magnitude

0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 −40 −30 −20 −10 0 10 20 Analog angular frequency

Figure 7.4

7.3

30

40

Pictorial representation of dot product in frequency domain of a signal x(t ) with haar scaling function in some subspace

Analysis of Haar Wavelet Function in Time and Frequency Domain

Haar wavelet function is shown in Fig. 7.5. Haar wavelet may also be represented iny (t ) = f (2t ) - f (2t - 1). Fourier transform of Haar wavelet function is F .T .

f (t ) « fˆ (W) F .T . 1 æ Wö f (2t ) « fˆ ç ÷ 2 è 2ø F .T . 1 - j W æ Wö f (2t - 1) « e 2 fˆ ç ÷ è 2ø 2

Using these relationships, we get y (t ) = f (2t ) - f (2t - 1) 1 æ W ö 1 - jW æ W ö yˆ (W) = fˆ ç ÷ - e 2 fˆ ç ÷ è 2ø 2 è 2ø 2 - jW ö æ Wö 1æ yˆ (W) = ç 1 - e 2 ÷ fˆ ç ÷ 2è ø è 2ø

221

Time-Frequency Joint Perspective

yˆ (W) = 2 je

- jW 2

2 æ Wö sin ç ÷ è 4ø æ Wö çè ÷ø 4

2 æ Wö sin ç ÷ è 4ø | yˆ (W) | = 2 W 4

Haar wavelet function 2 1.5

Amplitude

1 0.5 0 −0.5 −1 −1.5 −2 −0.5

0

Figure 7.5

0.5 Time

1

1.5

Haar wavelet function in time domain

Using this relationship we may plot Haar wavelet function across various subspaces. Figure 7.6 shows Haar wavelet in subspaces W-1 to W2 . Haar wavelet function

2

1.5

Haar wavelet function in domain:W−1

1.5

Magnitude

Amplitude

1 0.5 0 −0.5

1

0.5

−1 −1.5 −2

0

1 Time Ψ t 2

2

0 −60

20 40 0 −40 −20 Analog angular frequency 2

Figure 7.6

sin2 Ω 2 Ω 2

60

222

Multiresolution and Multirate Signal Processing

0.8 1.7

1

0.6

0.5

0.5

Magnitude

1.5

0 −0.5 −1

−0.2

−1.5

−0.1

−2

Amplitude

2

0

0.5

1 Time Ψ(t)

1.5

2

−2

2.5

0.4

1.5

0.35

1

0.3

0.5

0.25

−0.5

0.2

−1

0.1 0.05 0

0.5

1 Time Ψ(2t)

1.5

2

(c)

0.15

−1.5 −2 −0.5

−40 −20 0 20 40 60 Analog angular frequency 2 sin Ω 4 Ω 4 Haar wavelet function in frequency domain W1 −60

Haar wavelet function in subspace W1

0

(b)

0.4 −0.3

Magnitude

Amplitude

2

2 Haar wavelet function in frequency domain W0

Haar wavelet function in subspace W0

0

2.5

−60

−40 −20 0 20 40 Analog angular frequency

Ω 8 Ω 8 Haar Wavelet function in frequency domain W1 1 2

2

Haar Wavelet function in subspace W2

0.4

sin

0.18

1.5

0.16

1

0.14

0.5

Magnitude

Amplitude

60

2

0 −0.5

0.12 0.1

(d)

0.08 0.06

−1 0.04 −1.5 −2 −0.5

Figure 7.6

0.02 0

0.5

1 Time Ψ(4t)

1.5

2

2.5

0 −60

−40 −20 0 20 40 Analog angular frequency 2 sin Ω 1 16 4 Ω 16

60

Haar wavelet function at various time and frequency resolutions. Wavelet function and frequency domain characteristic in 6(a): W-1 subspace; 6(b): W0 subspace; 6(c): W1 subspace and 6(d): W2 subspace.

As in the case of Haar scaling function, here also localization in time improves as we move from W-1 to W2 by factor of two. Similarly, localization in frequency degrades as we go in similar direction by factor of two. However, important thing to note here is along with the bandwidth, centre frequency also shifts, unlike in the case of Haar scaling function. Here different bands with increasing bandwidth are emphasized as we go from W-¥ to W+¥. This characteristic of Haar wavelet function is similar to aspirant

223

Time-Frequency Joint Perspective

band pass filter. Again, similar notion of time and frequency localization applies here as in case of Haar scaling function, namely, localization in time gets better and localization in frequency gets poorer as we go from W-¥ to W+¥ .

7.4

Summary

In this section we analyzed Haar scaling and wavelet functions in detail. Rather than analyzing functions in either time domain or in frequency domain, we consider them in joint domain, i.e. time and frequency domains simultaneously. Analysis reveals that time and frequency domain localization improves and deteriorates respectively as we go from left to right in subspace ladder and vice versa. Also, Haar scaling function aspires to become low pass filter and wavelet function aspires to be band pass filter. Through this analysis we may consider ideal frequency responses of scaling and wavelet functions, which may improve overall response and projections over approximation and detail subspaces (V and W subspaces respectively). Such findings allow us to consider fundamental question of bound over simultaneous localization in time and frequency domain, namely the ‘uncertainty principle’.

7.5

Ideal Time-Frequency Behaviour

In earlier sections we have looked at the Fourier Transform of the scaling function (Father wavelet) f(t ) and the wavelet function (Mother wavelet) y (t ) in the Haar Multiresolution Analysis. In next few sections we will see, what is the ideal situation that we are driving towards. We have made some observations ˆ W) and yˆ (W). When we take dot product of x(t ) b2 - 4 ac and a about the nature of the magnitude of f( translate of f(t ), the magnitudes of the Fourier Transforms of x(×) and f(×) are getting multiplied. When we cascade two Haar scaling functions it results into Triangular wave function as depicted in Figure 7.7. We have observed that the nature of the Fourier Transform of the f(×) and also that of y (×) was such that it emphasizes some bands of frequencies of the underlying function x(t ).

Amplitude

1.5

Triangle wave

1 x(t) = 1 + t ; −1 < t < 0 = 1 − t ; 0 < t N . The dilated spectra for these 3-cases considering the positive W axis are shown in Figs. 10.23, 10.24 and 10.25 respectively. By observing the dilated spectra it is obvious that for a0 > N holes are created in the spectrum, hence the maximum range of ‘a0’ is N. yˆ (Ω) 2 Mirror image

1 k=1

0 p/N

k=0

p

Figure 10.23

k=1 Np

N 2p

|y (W ) |2 for a0 = N

Ω

321

Dyadic MRA

yˆ (Ω) 2

k=1

k = −1

k=0

Mirror image 0 2p/N

Figure 10.24

π Np/2

Holes

1

1

1

k=1

k=0

k = −1

0 p/2N

p/2

Figure 10.25

Ω

|y (W ) |2 for a0 < N with a specific case of a0 = N / 2

ˆ (Ω) 2 y Mirror image

N 2p/2

2p Np

2Np

Np

p

2N 2p

|y (W ) |2 for a0 > N with a specific case of a0 = 2 N

Exercise 10.4

For the Haar Wavelet y (t ) , shown in Fig. 10.26. Show that 0 < c1 £ SDS (y , a0 )(W ) £ c2 < ¥ (Hint: The upper bound c2 could be obtained from the admissibility condition of Haar while the lower bound c1 is a bit tricky). The Fourier transform y (W ) of y (t ) can be obtained as 1

1 y (W ) = ò 2 e- jWt dt - ò1 e- jWt dt 0

y (W ) = j.e

2

- jW 2

sin

2

W 4

Therefore |y (W ) |2 is of the form, W 4 y (W )2 = 2 æW ö ç 4÷ è ø sin

4

W 4

Ω

322

Multiresolution and Multirate Signal Processing

Now consider the admissibility integral, ¥

Þ ò |y (W ) |2 0

ò

¥

0

¥

 (W ) |2 d W = |y W 0

ò

dW W

 (W ) |2 d W = |y W

W 4 dW + 2 W W 0 æ ö çè ÷ø 4

ò

D

sin

4

W 4 dW 2 æWö W 0 ç ÷ è 4ø ¥ 4W sin 4 dW 2 W W D æ ö çè ÷ø 4

ò

ò

¥

sin

4

where D > 0. It is easy to show that the above integral is convergent, hence the upper bound c2 is finite. The |y (W ) |2 is, as shown in Fig. 10.27. The dilated spectra moves the nulls of |y (W ) |2, in order to satisfy the lower bound c1, these nulls should not coincide. yˆ (Ω) 2

Nulls

0

Ω

Figure 10.27

 (W ) y

2

Exercise 10.5

Establish a procedure for Bandpass Sampling. Construct the spectrum after sampling (Fig. 10.28) with Bandpass Function with sampling frequency f s = 2 ´ Bandwidth. Is there any aliasing? Hint: Suppose we consider a Bandpass Function where the band on W lies between p and 2p (Fig. 10.29). 1

0

1.1p

Figure 10.28

2.1p

Ω

Bandpass function

323

Dyadic MRA

1

0

2p

p

Figure 10.29

Ω

Bandpass function

What could be the sampling rate when we talk about discretizing the translation parameter. We are essentially talking about sampling the output of this filter on the k th analysis branch and feeding those samples instead of continuous function to the input of k th synthesis branch. How do we sample in such a way that we don’t loose something. How do we discretize the translation parameter? Band occupancy is p . If we simply use Nyquist criteria and sampling frequency as f s . Then 2pf s = 4p . Therefore, f s = 2. But we can also do with a sampling rate 2pf s = 2 ´ band occupancy = 2p Therefore, f s = 1. In Bandpass sampling theorem, we decide the sampling frequency by looking at the bandpass occupancy and not the maximum frequency. Suppose we use a sampling rate of 1. We are essentially adding all aliases, which are shifts of the original spectrum by (2p ´ 1 ´ k ) for all integer k. Take the original spectrum, shift it by all multiples (2p ´ 1 ´ k ) for all integer k and add up these translates. We can retrieve the original signal by putting any filter between p and 2p . However, we can not generalize this. Whenever there is a band occupancy of p we can not use sampling rate of 2p (i.e. twice the band and not the maximum frequency). It is true, depending on the location of the band as well. That is why Bandpass Sampling theorem is more complicated than the conventional Low-Pass Sampling Nyquist theorem. For the Bandpass function given in the question (Fig. 10.29), band occupancy is p . If we use the sampling rate as (2 ´ band occupancy) i.e. 2p . We will get the following spectrum as shown in Fig 10.30. From the above spectrum it is clear that there will be aliasing and we cannot retrieve the original signal. Original Original Original Original Original

spectrum spectrum spectrum spectrum spectrum

shifted shifted shifted shifted

1

−6π −5π −4π −3π −2π −π −1.1π −2.1π

Figure 10.30

by by by by

2π 2π 4π 4π

backward forward backward forward

1

0

2π 3π 4π 5π 6π Ω 1.1π 2.1π π

Sampling of bandpass function

324

Multiresolution and Multirate Signal Processing

Exercise 10.6

Explain the concept of log periodicity. Prove that sum of dilated spectra is log periodic. Hint: Generally in W axis the translates are shifted by multiples of some frequency (a parameter). Consider (k ´ p ), where k can take any integer value. Therefore, the shifts will be p ,2p ,3p , and so on. But if we consider the (a k ´ p ), where a is any constant and k can take any integer value. Then shifts will be of the multiples of a k times p . i.e. ap , a 2p , a 3p , a 4p . Log periodic signal means the signal whose period is power of some constant, like a k where a is constant and k can take any integer value. To prove that the Sum of Dilated Spectra (SDS) is log-periodic. i.e. To show that, SDS(Y , a0 )(W ) = SDS(Y , a0 )(a0m W ) Proof (in general): +¥

SDS(Y , a0 )(a0m W ) =

å |Y (a a k 0

m 0

W ) |2

k =-¥ +¥

=

å |Y (a

k+m 0

W ) |2

k =-¥

Here m is a constant integer. Therefore, as k runs over all integers, k + m will also run over all integers.

Exercise 10.7

What is the significance of closure in the Union Axiom of MRA? Hint: Union Axiom

V

m

= L 2 (R)

m ÎZ

This is essentially an Axiom of perfect reconstruction. When all incremental subspaces are combined together, we get the whole input back. Union over Vm should cover the entire space on L2 (). But in some cases it may leave some patches on the boundary. While taking the closure also ensure covering the entire boundary as well.

U

Vm = L2(R )

m∈Z Covers all the interior + boundary on L2(R )

Figure 10.31

U

Vm

m∈Z Covers all the interior of L2(R )

Closure on union of Vm

325

Dyadic MRA

Exercise 10.8

l[ n] = (1,2, 4,2,1) l[n]

1−z−1 Analysis High Pass

2

f[n]

Find f [ n] and plot f (t ), show that f (t ) = å f [ n]f (2t - n) lies in W0 n -2

-4

-6

Hint: l intermediate ( z ) = 1 + 2 z + 4 z + 2 z + z -8 So F ( z ) can be expressed as product of l intermediate( z ) with the Analysis High-Pass Filter. F ( z ) = (1 - z -1 )(l intermediate( z )) = (1 - z -1 )(1 + 2 z -2 + 4 z -4 + 2 z -6 + z -8 ) = 1 - z -1 + 2 z -2 - 2 z -3 + 4 z -4 - 4 z -5 + 2 z -6 - 2 z -7 + z -8 - z -9 f [ n] = (1, -1, 2, -2, 4, -4,2, -2,1, -1) Now f (t ) = å f [ n]f (2t - n), so f (t ) can be plotted as follows n

f(t) 0

1

2

3

4

5

Again f (t ) can be expressed as a function of y (t ) as follows f (t ) = y (t ) + 2y (t - 1) + 4y (t - 2) + 2y (t - 3) + y (t - 4) and y (t ) lies in W0 and so as f (t ).

Chapter

11

MRA Variant 1: Bi-orthogonal Filters

Introduction Inner product of wavelet function y(t) and Scaling function f(t – m) Variants of MRA Introduction of Bi-orthogonal filter banks JPEG 2000 5/3 filter-bank and spline MRA Description Note 5/3 tap in nutshell

11.1

Filter design strategy

Introduction

In this chapter we need to explore some more avenues of the concept of MRA. First let us complete few details of the proof of the theorem of MRA attempted in earlier chapters and then introduce different variants of MRA.

11.2 Inner Product of Wavelet Function y (t ) and Scaling Function f(t – m) We had already shown that, y (t ) Î V1 and y (t ) can be expanded as, y (t ) = åg[ n]f (2t - n) nÎZ

Where g[ n] is the impulse response of the analysis high-pass filter and corresponds to the inverse Z-transform of z - ( L -1) H (- z -1 ), where H ( z ) is analysis low-pass filter. Expressing f(t ) in terms of the basis of the V1 involves coefficients of low-pass analysis filter. Expressing wavelet y (t ) in terms of the basis of the V1 involves coefficients of high-pass analysis filter. Example 11.2.1 — High-pass filter impulse response.

For example, in the Haar case, High-pass filter impulse response is given as, [1 − 1] ↑ And, accordingly Haar wavelet is given as, y Haar ( t ) = 1.f (2 t ) - 1.f (2 t - 1)

328

Multiresolution and Multirate Signal Processing

The wavelet and scaling functions in general are given as, y (t ) = åg[ n]f (2t - n) nÎZ

f (t - m ) =

å h[n ]f (2t - 2m - n ) 1

1

n1ÎZ

The inner product between wavelet function y (t ) and scaling function f(t ) shifted by m is given as, < y (t ),f (t - m) > = ååg[ n]h[ n1 ] < f (2t - n),f (2t - 2m - n1 ) > n

(11.1)

n1

The inner product term on the right hand side can be evaluated as, +¥

< f (2t - n),f (2t - 2 m - n1 ) > = ò f (2t - n)f (2t - 2 m - n1 )dt -¥

Let 2t be = l. It gives 2dt = dl and t : -¥ ® +¥ Þ l = -¥ ® +¥ With this substitution the above integral turns out to be, 1 +¥ f (l - n)f (l - 2 m - n1 )d l 2 ò-¥ 1 < f (2t - n),f (2t - 2 m - n1 ) > = d [ n - (2m + n1 )] 2

< f (2t - n),f (2t - 2m - n1 ) > =

Putting the value in Eq. (11.1) we get, < y ( t ),f ( t - m ) > =

1 ååg [ n ] h [ n1 ]d [ n - (2m + n1 )] 2 n n1

Dropping å n, we get < y (t ),f (t - m) > =

1 åg[2m + n1 ]h[n1 ] 2 n1

(11.2)

Equation (11.2) is the cross-correlation of g[.] and h[.] evaluated at 2m, "m Î Z. So the Z-transform of the cross-correlation of g[.] and h[.] is, G( z ) H ( z -1 ) = z - ( L -1) H (- z -1 ) H ( z -1 )

(11.3)

Since G( z ) = z - ( L -1) H (- z -1 ). For simplicity, let us assume that the impulse response is real. Consider which gives us z - ( L -1) H (- z -1 ) H ( z -1 ) + z - ( L -1) H (- z -1 ) H ( z -1 ) |z ¬- z z - ( L -1) H (- z -1 ) H ( z -1 ) + (-1)L -1 z - ( L -1) H ( z -1 ) H (- z -1 ) = 0 This happens because L is even. The cross-correlation of g[.] and h[.] is zero for all 2m. So < y (t ),f (t - m) > = 0"m Î Z

(11.4)

329

MRA Variant 1: Bi-orthogonal Filters

Similarly, the inner product between scaling function and its translate by n is given as < f (t ),f (t - m) > =

1 åh[2m + n]h[n] 2 n

The above expression shows the auto-correlation of impulse response of low-pass filter in the analysis side. Now, the design equations for an orthogonal filter bank ensures that H ( z ) H ( z -1 ) + H ( - z ) H ( - z -1 ) = constant Autocorrelation sequence is ‘0’ at all even locations 2m except when m = 0.

11.3

Variants of MRA

To know the variants of MRA, we need to ask certain questions regarding some other avenues of MRA. 1. Should we have essentially the same analysis and synthesis filters? The answer is ‘No’, because JPEG-2000 standard for data compression employs bi-orthogonal filter banks in which the analysis and synthesis filter banks are different. 2. Do the filters in filter banks need to be finite impulse response filters? The answer is ‘No, not necessarily’. 3. Should we always iterate on the low-pass branch? The answer is again ‘No, not always’. In wave-packet transform we also iterate on the high-pass branch. By Discrete wavelet transform generally we make a wavelet tree that looks like as in Fig. 11.1.

LPF

HPF

Figure 11.1

LPF

2

HPF

2

2

2

Discrete wavelet transform

While iterating on the high-pass branch as well, which is used in wavelet packet transform, it looks like as in Fig. 11.2.

330

Multiresolution and Multirate Signal Processing

LPF

2

HPF

2

LPF

2

HPF

2

2

HPF

2

Figure 11.2

11.4

LPF

Wavelet packet transform

Introduction of Bi-orthogonal Filter Banks

Now, let us try to look for an elaborate answer for the first question using the Haar MRA. In the case of Haar, the dilation equation for f0 (t ), f0 (t ) = f0 (2t ) + f0 (2t - 1) This can be seen in Fig. 11.3. f0(t) 0

1 f0(2t) f0(2t−1)

Figure 11.3

Dilation equation for f0 (t )

Now, we want to calculate f0 ( t ) × f0 ( t ). Given that h( t ) × g( t ) = r ( t ) then what is h( at + b ) × g ( at + c )? For same scaling a Î R(real), +¥

h( at + b ) × g ( at + c ) = ò h ( al + b ) g [ a( t - l ) + c ] d l -¥

Let al + b = g , a ¹ 0 and aÎR. If a > 0, dg = ad l , and l : -¥ ® +¥ Þ g = -¥ ® +¥

331

MRA Variant 1: Bi-orthogonal Filters

If a < 0, dg = ad l , and l : -¥ ® +¥ Þ g = +¥ ® -¥ In general, h(al + b)g[ a(t - l ) + c]d l =

1 a

ò

+¥

h(al + b)g[ a(t - l ) + c]d l =

1 a

ò

+¥

ò

+¥

ò

+¥

ò

+¥

-¥

-¥

-¥

h( al + b ) g [ a( t - l ) + c ] d l =

h(g )g(-g + at + b + c)dg

-¥

-¥

h(g )g(at + b + c - g )dg

1 h × g |( at + b+ c ) a

Using this equation and denoting f0 ( t ) × f0 ( t ) = f1 ( t ), as shown in Fig. 11.4.

f0(t) 0

f1(t)

1

0

1

f0(t) 0

1

Figure 11.4

Convolution of f0 (t ) with itself gives f1 (t )

The dilation equation of f1 (t ) is, 1 f1 (t ) = [f1 (2t ) + 2f1 (2t - 1) + f1 (2t - 2)] 2 Also, 1 f0 (2 t ) × f0 (2 t - 1) = f0 (2 t - 1) × f0 (2 t ) = f1(2 t - 1) 2 This dilation equation is also shown in Fig. 11.5.

2

332

Multiresolution and Multirate Signal Processing

f1(t) f1(2t−1) 1 f (2t) 2 1

0

1 f (2t−2) 1 2

1 2

1

3 2

2

Dilation equation of f1 (t )

Figure 11.5

1 1 Coefficients in dilation equation: f1 (t ) = f1 (2t ) + f1 (2t - 1) + f1 (2t - 2) are 2 2 é1 1ù ê212ú ë û And the corresponding filter is 1 1 + 1.z -1 + .z -2 2 2 1 H ( z ) = (1 + z -1 )2 2

H (z) =

But this scaling function is not orthogonal to all its integer translates. f1(t) f1(t−1)

0

Figure 11.6

1

2

3

f1 (t ) is not orthogonal to its translate by 1

Figure 11.6 shows that f1 (t ) is not orthogonal to its translate by 1, i.e the axiom of orthogonality is not obeyed.

11.5

JPEG 2000 5/3 Filter-bank and Spline MRA

We have already been introduced to variants of multi-resolution analysis (MRA). In particular, MRA with different analysis and synthesis filters such as, bi-orthogonal filter-banks in perfect reconstruction

333

MRA Variant 1: Bi-orthogonal Filters

framework, MRA with infinite impulse response (IIR) and MRA which iterates on high-pass branch along with the low-pass branch were discussed. In this, and the subsequent sections, we discuss these variants in detail. We start with bi-orthogonal filter-banks utilized in Joint Photographic Experts Group 2000 (JPEG 2000) image compression standard. We use piece-wise linear function as a scaling function at synthesis side and apply alias cancellation and perfect reconstruction requirements to achieve transfer functions of the filters involved. This filter-bank differs from orthogonal MRA in nonorthogonality of scaling function to its integer translates. In this chapter, we see how JPEG 2000 filter-bank is derived for spline type (piece-wise linear with desirable interpolation characteristics) scaling function. In subsequent chapters, extension to orthogonal MRA for piece-wise linear functions has been discussed, which explicitly reveals the simplicity offered by biorthogonal filter-banks by sacrificing orthogonality requirement of scaling and wavelet functions.

11.6

Description

Dilation equation for Haar scaling function is given as: f0 (t ) = f0 (2t ) + f0 (2t - 1)

(11.5)

Convolution of this function with itself yields another piece-wise linear function, which may be represented in the following manner (Eq. 11.6): 1 1 f1 (t ) = f1 (2t ) + f1 (2t - 1) + f1 (2t - 2) 2 2

(11.6)

Clearly, this results in a triangular function, as shown in Fig. 11.7. 1.5

Amplitude

1

0.5

0 −1

Figure 11.7

−0.5

0

0.5

1 Time

1.5

2

2.5

3

Triangular wave resulting from convolution of haar scaling function with itself

334

Multiresolution and Multirate Signal Processing

This function comes from the class of piece-wise polynomial interpolants, also called as ’Splines’. Z -transform of this function may be depicted in the following manner: F( Z ) = (1 + Z -1 )2

(11.7)

Here, we can note that f1 (t ) is a piece-wise linear function and is not orthogonal to its integer translates. However, it is orthogonal to all its translates f1 (t - m) for m > 2. We need to obtain the following filter-bank structure (Fig. 11.8) for this scaling function, which should be similar to orthogonal MRA.

H0(z)

2

2

G0(z) +

Analysis Input

Output H1(z)

Figure 11.8

2

2

G1(z)

Desired filter-bank structure from piecewise linear function

Example 11.6.1 — Bi-orthogonal coefficients.

Without the loss of generality, let us select G0 ( Z ) = (1 + Z -1 )2. At this point, we may note that once the alias cancellation and perfect reconstruction conditions are met, we may interchange analysis and synthesis filters. Alias cancellation condition is given as: H 0 (- Z )G0 ( Z ) + H1 (- Z )G1 ( Z ) = 0

(11.8)

H 0 ( Z )G0 (- Z ) + H1 ( Z )G1 (- Z ) = 0

(11.9)

If we replace Z by -Z, we get:

which is indeed a requirement of alias cancellation for analysis and synthesis filters interchanged. Also, perfect reconstruction condition is given as: H 0 ( Z )G0 ( Z ) + H1 ( Z )G1 ( Z ) = C0 Z - D

(11.10)

If we replace G0 by H 0 and G1 by H1 or vice versa, we get the same condition back, which indicates that perfect reconstruction condition is also satisfied for this new filter-bank. Hence, we may say that once the filter-bank satisfies alias cancellation and perfect reconstruction conditions, analysis and synthesis filters can be interchanged to get a new 2-band perfect reconstruction filter-bank. However, if there are two separate scaling functions at analysis and synthesis side, then the ‘smoother’ scaling function is preferred at the reconstruction side for more ‘appealing’ reconstruction.

335

MRA Variant 1: Bi-orthogonal Filters

Let us continue with the construction of bi-orthogonal MRA from G0 ( Z ) = (1 + Z -1 )2. Using alias cancellation condition for this case, relationship between various filter transfer functions can be obtained. Such a relationship is shown below: H 0 (- Z )G0 ( Z ) + H1 (- Z )G1 ( Z ) = 0 On rearranging, we get: G0 ( Z ) H (- Z ) =- 1 G1 ( Z ) H 0 (- Z )

(11.11)

For the simplest analysis, we equate the numerators and denominators and arrive at the following: G0 ( Z ) = - H1 (- Z )

(11.12)

G1 ( Z ) = H 0 (- Z )

(11.13)

Taking the perfect reconstruction condition into consideration, we can get an important relationship to obtain the analysis low-pass filter transfer function from the synthesis low-pass transfer function. From the perfect reconstruction condition, we have: H 0 ( Z )G0 ( Z ) + H1 ( Z )G1 ( Z ) = C0 Z - D

(11.14)

However, G1 ( Z ) = H 0 (- Z ) and G0 ( Z ) = - H1 (- Z ). Substituting these value in Eq. (11.14), we have: H 0 ( Z )G0 ( Z ) - G0 (- Z ) H 0 (- Z ) = C0 Z - D

(11.15)

Let us denote κ 0 ( Z ) = G0 ( Z ) H 0 ( Z ) . Now, we have the following relationship: κ 0 ( Z ) − κ 0 (− Z ) = C0 Z − D

(11.16)

This relationship indicates that we ‘kill’ all the even samples in the inverse Z -transform of κ 0 ( Z ) and out of the remaining odd samples, we find that only one sample (namely, the D th sample) has a nonzero value (equal to C0 ). If we want the same degree of regularity (or smoothness) at analysis and synthesis sides, we may select H 0 ( Z ) to have two zeros at Z = -1. In other words, we select H 0 ( Z ) to have (1 + Z -1 )2 factor in its transfer function. In order to have a linear phase characteristics, we need symmetry in the transfer function. Taking note of this and without considering the effect of causality, we may parsimoniously extend H 0 ( Z ) by introducing a factor (1 + h0 Z -1 + Z -2 ). We may note that by introducing this factor along with the factor (1 + Z -1 )2, we have retained symmetry and only one degree of freedom is introduced in terms of the parameter h0 . H 0 ( Z ) = (1 + Z -1 )2 (1 + h0 Z -1 + Z -2 )

(11.17)

In retaining symmetry, we could have used some constant, say h1, for coefficients of Z 0 and Z and retained h0 as coefficient of Z -1. Apparently, we would have had two degrees of freedom in such a case. However, the factor introduced here essentially scales the whole filter transfer function by a constant. Scale factor for a transfer function is not really important for magnitude and phase characteristics. Hence, scaling can also be taken care of at the time of normalization. -2

336

Multiresolution and Multirate Signal Processing

In other words, we introduce only as many degrees of freedom as required to get something novel in terms of nature of frequency response and not in terms of overall scaling, which can always be adjusted while normalizing the impulse response. Therefore, keeping only one degree of freedom, let us now impose the condition on G0 ( Z ) H 0 ( Z ), which may be depicted as: G0 ( Z ) H 0 ( Z ) = (1 + Z -1 )2 (1 + Z -1 )2 (1 + h0 Z -1 + Z -2 ) G0 ( Z ) H 0 ( Z ) = (1 + 4 Z -1 + 6 Z -2 + 4 Z -3 + Z -4 )(1 + h0 Z -1 + Z -2 )

(11.18)

which is in fact the convolution, as shown here 1 4 6 4 1 × 1 h0 1 ↑ 0

↑ 0

Thus giving the following sequence: 1 (4 + h0 ) (7 + 4h0 ) (8 + 6h0 ) (7 + 4h0 ) (4 + h0 ) 1 ↑ 0

(11.19)

As discussed above, we do not have any knowledge about the even samples as they were ‘killed’ (according to Eq. 11.16). However, we have some information about the nature of odd samples. Accordingly, we need to retain only one sample out of them. Note that if we remove the two (4 + h0 ) samples, we would achieve what we wanted. In other words, if we make (4 + h0 ) = 0 , we retain only one sample out of all the odd samples. Therefore, we select h0 = -4 . Hence, H 0 ( Z ) = (1 + Z -1 )2 (1 - 4 Z -1 + Z -2 )

(11.20)

Again for expansion, if we want to denote H 0 ( Z ) as a series, we can denote the relationship (Eq. 11.20) in terms of convolution of two series, given by: 1 2 1 × 1 −4 1 ↑ 0

↑ 0

Carrying out this convolution gives the following sequence: 1 -2 -6 -2 1

(11.21)

H 0 ( Z ) = 1 - 2 Z -1 - 6 Z -2 - 2 Z -3 + Z -4

(11.22)

Therefore,

which is a low-pass filter of the analysis side. We note that length of this filter is 5. Hence, we started out with G0 ( Z ) = 1 + 2 Z -1 + Z -2 and obtained H 0 ( Z ) = 1 - 2 Z -1 - 6 Z -2 - 2 Z -3 + Z -4 Here, G0 ( Z ) is of length 3 and H 0 ( Z ) is of length 5. This filter-bank is known as the JPEG 5/3 filter bank, where, 5/3 refers to lengths of impulse responses.

337

MRA Variant 1: Bi-orthogonal Filters

11.7

Note

R JPEG-2000 compression standard admits two kinds of filter banks: a 5/3 filter-bank and a 9/7 filter-

bank. As we have seen, 5/3 refers to lengths of impulse responses in a 5/3 filter-bank. Similarly, 9/7 also refers to lengths of impulse responses of a 9/7 filter-bank.

Lengths of impulse responses in the JPEG 2000 5/3 filter bank may be represented, as shown in Fig. 11.9. 5

2

3

2

Analysis

+

Input

Output 3

2

Figure 11.9

5

2

JPEG 2000 5/3 filter-bank

We may use the above noted relationships between various filter transfer functions to get the transfer functions of all the filters involved in JPEG 2000 5/3 filter bank. Now, G0 ( Z ) = - H1 (- Z ) Þ H1 ( Z ) = -G0 (- Z ) = -(1 - Z -1 )2

(11.23)

G1 ( Z ) = H 0 (- Z ) Þ G1 ( Z ) = 1 + 2 Z -1 - 6 Z -2 + 2 Z -3 + Z -4

(11.24)

Further,

Filter transfer functions for JPEG 2000 may be summarized in the following manner: H 0 ( Z ) = 1 - 2 Z -1 - 6 Z -2 - 2 Z -3 + Z -4 -1

G0 ( Z ) = 1 + 2 Z + Z

-2

(11.25) (11.26)

-1 2

H1 ( Z ) = -G0 (- Z ) = -(1 - Z )

(11.27)

G1 ( Z ) = 1 + 2 Z -1 - 6 Z -2 + 2 Z -3 + Z -4

(11.28)

We may note that the individual sums of coefficients of high-pass filters on analysis and synthesis sides are zero. This simply indicates the presence of zero (null) at zero frequency. In fact, in both H1 ( Z ) and G1 ( Z ), there is a factor of (1 - Z -1 )2 , which indicates the presence of two zeros at zero frequency. This substantiates our intuition that both filters should be high-pass filters in nature. As evident from our discussion throughout this chapter, our major aim was to achieve a perfect reconstruction filter-bank from a scaling function, which does not form an orthogonal basis set with its integer translates. Due to such a property, orthogonal MRA is not possible using such a scaling function.

338

Multiresolution and Multirate Signal Processing

In such a case, we could still achieve a perfect reconstruction filter-bank, however, we ended up getting different scaling functions at analysis and synthesis sides. In perfect reconstruction framework, such an MRA is known as bi-orthogonal MRA. Idea of a bi-orthogonal basis vectors may be explained more lucidly in a 2-D vector space, as explained next. Consider a set of vectors consisting of uˆ1 and uˆ2, which are orthogonal to each other, i.e., their inner product is zero. One such representation is shown in Fig. 11.10. u ˆ1 yˆ

u ˆ2

Figure 11.10

Orthogonal basis vectors (uˆ1 and uˆ2)

In this case, representing any other vector in this vector space involves projecting that vector over uˆ1 and uˆ2 by taking inner products with respect to such basis vectors. If vectors uˆ1 and uˆ2 are linearly independent but not orthogonal to each other, even then they form a basis set for the corresponding vector space. However, the ease of representation of vector yˆ with respect to such basis vectors is lost. This can be seen from Fig. 11.11, which denotes two linearly independent vectors in a 2-D vector space. uˆ1 yˆ

O

Figure 11.11

uˆ2

Non-orthogonal basis vectors (uˆ1 and uˆ2)

We can note here that it is not impossible to obtain such a representation. In fact, one may solve a set of linear equations and come up with such representation. However, the ease with which coordinates may be obtained from orthogonal basis vectors (simply by taking inner products) is lost. Bi-orthogonal basis vectors can help in such a case to restore the ease of representation. We want yˆ to be represented in terms of c1uˆ1 + c2 uˆ2 . Consider a set of vectors u1 and u2, selected such that u1 is orthogonal to uˆ1 and u2 is orthogonal to uˆ2. Further, < uˆ1 , u1 >= 0 and < uˆ2 , u2 >= 0 . These vectors may be represented as shown in Fig. 11.12. We can now take the inner product of yˆ with u1 and obtain its projection over uˆ2. Similarly, inner product of yˆ with u2 yields projection over uˆ1. In this way, we can obtain representation of yˆ in terms of linearly independent basis vectors uˆ1 and uˆ2. This idea is extended to the generation of a bi-orthogonal filter-bank. We may compare similarity by noting that in biorthogonal filter-bank, two scaling functions and two corresponding wavelet functions exist; one each at analysis and synthesis sides. However, neither of them (scaling functions) form an orthogonal set with its integer translates.

339

MRA Variant 1: Bi-orthogonal Filters

uˆ1

O

u ˜2

Figure 11.12

11.8

uˆ2

u˜1

Idea of bi-orthogonal basis vectors

5/3 tap in Nutshell

In last few sections, we started with the piece-wise linear scaling function obtained by convolving Haar scaling function with itself. Such scaling function is a member of the broad class of functions known as splines. However, due to its nonorthogonality with its integer translates, we could not build orthogonal MRA. By imposing alias cancellation and perfect reconstruction properties, we came up with a filter-bank known as JPEG 2000 5/3 filter-bank, which is biorthogonal in nature. In this specific MRA, both synthesis and analysis filters have different impulse response lengths, unlike in the case of orthogonal MRA. Apparently, we dropped orthogonality requirement from MRA and could construct perfect reconstruction bi-orthogonal MRA. One of the advantages of such MRA is the possibility of achieving nontrivial linear phase filters, which is not possible to achieve in orthogonal MRA (except in the case of simplest Haar MRA). In practice one may start with piece-wise linear scaling function and come up with an orthogonal MRA, which require infinite impulse response (IIR) filters for spline (piece-wise linear) type of scaling functions. However, it is computationally quite involved to construct such an MRA. This will be precisely dealt with in subsequent chapters. Such an exercise will lucidly indicate the advantages of bi-orthogonal MRA in context of perfect reconstruction filter-banks.

11.9

Filter Design Strategy

Haar filter is the only finite length, symmetric, orthogonal filter whose Fourier representation [H(w )] satisfies zero derivative conditions at w = p , as was explained by Daubechies. Researchers across the globe have confirmed that having symmetric filters is advantageous in multiple ways for many real-life applications. Another important characteristics of filters is orthogonality.It guarantees energy preservation and inverse of wavelet transform matrix. For orthogonal filter ‘h’ that gives orthogonal transform matrix WN , the implementation is straightforward.This is because WN is ‘unitary’ which guarantees that for all real valued WN , WN-1 = WNT . This simple transpose gives us the inverse, which is really crucial in inverse calculations as as ‘separable’ multidimensional calculations. For ’non-orthogonal’ filters, however, one desirable strategy to build the system could be to ensure that inverse of transform matrix is transpose of another transformation matrix.

340

Multiresolution and Multirate Signal Processing

{h˜k, g˜k}

{hk, gk} iff

WN

−1 WN =

˜ W N ˜NT W

then {hk, g˜k} is biorthogonal filter pair

Haar, though orthogonal, finite length and symmetric, has its own drawback, which we have already brought out. Going beyond filter length of ‘2’, we need to give up ‘orthogonality’ as requirement, which comes as filter constraint. 5 Let us try constructing normalized tap: 3 Design problem: 1. 2. 3. 4. 5.

From symmetric hk and gk Construct wavelet transform WN When inverse WN-1 is calculated. WN-1 should also be a transform matrix! Find such g k & h k (symmetric)! Let ‘H B’ be

N ´ N matrix from hk . 2

Let ‘GB’ be

N ´ N matrix from gk . 2

 B’ be N ´ N matrix from g . Let ‘G k 2  B’ be N ´ N matrix from h k . Let ‘H 2 Forward wavelet matrix can be written as: éH ù WB = ê B ú ë GB û For synthesis part, similarly, é ù  B = êH B ú W B ú êë G û

341

MRA Variant 1: Bi-orthogonal Filters

keeping in mind, we want to create dual filters in semi-orthogonal sense,  TB = I WB × W N ´N  TB = é H B ù × é H T Now, WB × W ê G ú êë B ë Bû

éH × H  TB T B ù  ê GB = úû ê T ë GB × H B

(11.29)

 TB ù HB × G ú T  GB × G B úû

Here,  TB = G × G  TB = I HB × H B N ´N

(11.30)

 TB = G × H  TB = 0 HB × G B N ´N

{

}

5 Now, hk = {h-2 , h-1 , h0 , h1 , h2 } & h k = h -1 , h 0 , h 1 in the spirit of tap. 3 Therefore, analysis low-pass filter has 5 coefficients and synthesis low-pass filter has 3 coefficients, as ‘5’ is the bigger of the two N let = 5, \ N = 10 2 é h0 êh ê -2 \ HB = ê 0 ê ê0 êh ë 2

h1 h-1 0 0 0

h2 h0 h-2 0 0

0 0 h1 h2 h - 1 h0 0 h-2 0 0

éh� 0 ê ê0 ê � HB = ê 0 ê0 ê êë 0

h� 1 h� -1

0 �h0

0 �h1

0

0

0

0

 TB = I Now, H B × H 5´5

0

0

0 0 h1 h-1 0

0 0 h2 h0 h-2

0 0 0 h1 h-1

0

0

0

0

0

0 h� 1

0

0

0

h� -1

0 h� 0 0

h� -1

0 h� 1

0

0

0 h� 0 0

h� -1

0

0

0

0 �h0

h-2 0 0 h2 h0

h-1 ù 0 úú 0ú ú 0ú h1 úû

h� -1 ù ú 0 ú ú 0 ú 0 ú ú h� 1 úû

342

Multiresolution and Multirate Signal Processing

é h0 êh ê -2 ê0 ê ê0 êh ë 2

h1 h-1 0 0 0

h2 h0 h-2 0 0

0 0 h1 h2 h - 1 h0 0 h-2 0 0

0 0 h1 h-1 0

0 0 h2 h0 h-2

0 0 0 h1 h-1

h-2 0 0 h2 h0

é h 0 ê ê h 1 ê ê0 ê h-1 ù ê 0 0 úú ê 0 ê 0 ú×ê 0 ú 0ú ê0 ê h1 úû ê ê0 ê0 ê êh -1 ê ë

0 h -1

0

0

0

0 0

0

0 h -1 h 0

0

h 1

0

0

0

0

h 0 h 1

0

0

0

0

0

0

h 0 h 1

0 0 h -1

0 ù ú 0 ú ú 0 ú 0 ú ú 0 ú ú 0 ú = I 5´5 ú 0 ú ú h -1 ú h 0 ú ú h 1 ú ú û

1st row ´ 1st column,1st row ´ 2 nd column and 1st row ´ last column will give us three equations, h0 × h 0 + h1 × h 1 + h-1 × h -1 = 1

(11.31)

h0 × h 1 + h2 × h 0 = 0

(11.32)

h-2 × h 0 + h-1 × h 1 = 0

(11.33)

Equation (11.31) can be written as, 1

åh

k

× h k = 1

(11.34)

K = -1

Equations (11.32) and (11.33) can be written as, 1

åh

k -2 m

× h k = 0, m = -1,1

K = -1

 TB = I Similarly, GB × G 5´5 Here, gk = { g−1 , g0 , g1 } & g k = { g−2 , g−1 , g0 , g1 , g2 } Therefore, analysis high-pass filter has ‘3’ coefficients, synthesis high-pass filter has ‘5’ coefficients N = 5,\ N = 10 2

(11.35)

343

MRA Variant 1: Bi-orthogonal Filters

é g0 ê0 ê GB = ê 0 ê ê0 ê0 ë é g� 0 ê ê g� -2 �B = ê 0 G ê ê 0 ê êë �g 2

0 g0 0 0 0

g1 g-1 0 0 0

0 g1 g-1 0 0

0 0 g0 0 0

0 0 g1 g-1 0

0 0 0 g0 0

0 0 0 g1 g-1

0 0 0 0 g0

g-1 ù 0 úú 0 ú ú 0 ú g1 úû

0

0

0

g� -2

0 �g

0 g�

0

0 0 �g 2 �g

g-1 ù 0 úú 0 ú× ú 0 ú g1 úû

�g 1 �g

g� 2 �g

0 �g

0 �g

0

�g -2

�g -1

�g -1 0

�g 0 g�

-2

0 �g 1 �g

0 0 0 g0 0

0 0 0 g1 g-1

0 0 0 0 g0

-1

0

1

2

0

0

0

�g 0 �g

0

0

0

0

1

-2

2

-1

0

�g ù -1 ú 0 ú ú 0 ú 0 ú ú g� 1 úû

 TB = I Now, GB × G 5´5 é g0 ê0 ê ê0 ê ê0 ê0 ë

g1 g-1 0 0 0

0 g0 0 0 0

0 g1 g-1 0 0

0 0 g0 0 0

0 0 g1 g-1 0

é g 0 ê ê g 1 ê ê g2 ê 0 ê ê 0 ê ê 0 ê ê 0 ê 0 ê ê g 2 ê  ëê g -1

g -2 g

0

0

0 g -2 g

0

-1

g 0 g 1

g 2

-1

0

g 0 g

0

g 2

1

0 0 g

-2

g -1 g 0

0

0

0

0

g 1 g

0

0

0

2

g 2 ù ú 0 ú ú 0 ú 0 ú ú 0 ú ú = I 5´5 0 ú ú g -2 ú g -1 ú ú g 0 ú ú g 1 úû

g0 × g 0 + g1 × g 1 + g-1 × g -1 = 1

(11.36)

g0 × g -2 + g1 × g -1 = 0

(11.37)

g0 × g 2 + g-1 × g 1 = 0

(11.38)

344

Multiresolution and Multirate Signal Processing

\ Equation (11.36) get written as: 1

åg

k

× g k = 1

(11.39)

K = -1

Equation (11.37) and (11.38) becomes 1

åg

K -2 m

× g k = 0, m = -1,1

(11.40)

K = -1

 TB to be an identity matrix, which demands h and h k to be Now, design aspect demands H B × H k orthogonal. For two filters to be orthogonal, let us derive the sufficient and necessary condition with, again, an example of Haar wavelet. For Haar, if ì1 1 ü hk = {h0 , h1 } = 2 í , ý î2 2 þ jw w æ1 1 ö \ H (w ) = 2 ç + × e jw ÷ = 2 × e 2 × cos( ) 2 è2 2 ø w \ | H(w ) |= 2 × cos( ) (Note: Magnitude of e jw = 1 on interval of [ -p , p ]) 2

We have already seen that when H ( z ) becomes H(-z), the frequency band gets shifted by p amount, which can be represented as: H(w + p ) \ | H (w + p ) | = 2 × cos(

w +p ) 2

Now, if we add squared magnitudes of H(w ) and H(w + p ), we get 2 w | H(w ) |2 = 2 × cos ( ) 2

(11.41)

2

æ æ w + p öö 2æw + p ö | H(w + p ) | = ç 2 cos ç ÷ ÷ = 2 × cos çè 2 ÷ø è 2 øø è 2

2æw ö \ | H(w + p ) | = 2 × sin ç ÷ è2ø

(11.42)

2æ w ö 2æ w ö \| H (w )2 | + | Hw + p )2 | = 2 × cos ç ÷ + 2 × sin ç ÷ è2ø è2ø

(11.43)

Using (11.41) and (11.42),

345

MRA Variant 1: Bi-orthogonal Filters

This ‘2’ also comes as we are using normalized filters, in absence of normalization, the sum would be ‘1’. Important thing to note is, a constant suggest orthogonality. Using equations (11.41), (11.42) and (11.43), we can write  (w ) + H (w + p ) × H  (w + p ) = 2 H (w ) × H

(11.44)

 satisfy (Eq. 11.44), we have, When H(w ) and (w)

åh

k

× h k = 1

(11.45)

K ÎZ

For m Î Z , m ¹ 0,

åh

K -2 m

× h k = 0

(11.46)

K ÎZ

Similarly,  (w ) + G(w + p ) × G  (w + p ) = 2 G(w ) × G

(11.47)

When H(w ) and (w ) satisfy Eq. (11.44), we have,

åg

k

× g k = 1

(11.48)

K ÎZ

For m Î Z , m ¹ 0,

åg

K -2 m

× g k = 0

(11.49)

K ÎZ

 TB = 0 To ensure H B × G  (w ) + H (w + p ) × G  (w + p ) = 0 H (w ) × G

(11.50)

for all m Î z,

åh

K -2 m

× gk = 0

(11.51)

K ÎZ

 TB = 0 To ensure, GB × H  (w ) + G(w + p ) × H  (w + p ) = 0 G(w ) × H

(11.52)

For all m Î Z

åg

K ÎZ

K - sm

× hk = 0

(11.53)

346

Multiresolution and Multirate Signal Processing

Equations (11.44) to (11.53) give us definition for bi-orthogonal filter pairs. For example, if H(w )  and H( w ) of hk and h k respectively follow:  (w ) + H (w + p ) × H  (w + p ) = 2 H (w ) × H then, hk and h k are called bi-orthogonal filter pair! From (11.44), (11.47), (11.50) and (11.52)  (w + p ) [Analysis high-pass connected with synthesis low pass] and G(w ) = ei ( nw + b × H  (w ) = ei ( nw + b × H (w + p ) [Synthesis high-pass connected with analysis low pass] G Let us confirm this by plugging in  (w ) = e j ( nw ) + b × H  (w + p ) × e j ( nw + b ) × H (w + p ) G(w ) × G  (w + p ) × e j ( nw + b ) × H (w + p ) = e- j ( nw + b ) × H

(11.54)

 (w + p ) = H (w + p ) × H \ Since they are equal, together they shall produce ‘2’! By replacing ‘w’ with ‘w + p ’ in Eq. (11.54)  (w + p ) = H  (w ) × H (w ) [ As these are‘2p - periodic ¢ functions, \ w + 2p = w ] (11.55) G(w + p ) × G Adding Eqs. (11.54) and (11.55),  (w ) + G(w + p ) × G  (w + p ) = H (w ) × H  (w ) + H (w + p ) × H  (w + p ) = 2 G(w ) × G Now,  (w ) × G (w ) = H  (w ) × e j ( nw + b ) × H  (w + p ) H  (w ) × e- j ( nw + b ) × H  (w + p ) =H

(11.56)

Let us replace ‘w’ by ‘w + p ’ in Eq. (11.56) � (w + p ) × G (w + p ) = H � (w + p ) × e- j ( nw + np + b ) × H � (w + 2p ) H � (w + p ) × e- jnp × e- j ( nw + b ) × H � (w ) =H � (w + p ) × e- j ( nw + b ) × H � (w ) = ( -1)n × H (11.57) � (w + p ) × e- j ( nw + b ) × (w� )�� [‘ n‘ being odd ,( -1)n = -1] = -H Adding Eqs. (11.56) and (11.57),  (w ) × G(w ) + H  (w + p ) × G(w + p ) = 0 H

347

MRA Variant 1: Bi-orthogonal Filters

Readers should note, gk is connected with h k and g k is connected with hk ! b can be ‘0’ or p to keep filter real values and for n = 1,

Let us confirm this for normalized

gk = (-1)K × h 1- K

(11.58)

gk = (-1)K × h1- K

(11.59)

5 tap. 3

ì 1 3 1 1ü h = {h-2 , h-1 , h0 , h1 , h2 } = í- ,1, , - , ý × 2 î 2 4 2 4þ

(11.60)

ì1 1 1 ü h = h -1 , h 0 , h 1 = í , , ý × 2 î4 2 4þ

(11.61)

and

{

}

Plugging (11.60) and (11.61) in (11.58) and (11.59), we get,

{

g = g −1 , g 0 , g 1 , g 2 , g 3

}

= { h2 , h1 , − h0 , h−1 , h−2 }

(11.62)

 1 1 3 1 =  − , − , − ,1,  ⋅ 2  4 2 4 2 Similarly, g = {g0 , g1 , g2 }

{

= h�1 , −h� 0 , h� −1

}

1 1 1 =  ,− , ⋅ 2 4 2 4

(11.63)

Let us approve these filters: H (w ) = -

2 -2 jw 3 2 2 jw 2 2 jw ×e + 2 × e- jw + ×e + ×e 2 4 2 4  (w ) = 2 × e- jw + 2 + 2 × e jw H 4 2 4

Let us evaluate Eqs. (11.64) and (11.65), for w = 0 and w = p 3 1 1ö æ 1 H(w ) |w =0 = 2 ç - + 1 + - + ÷ = 2 2 4 2 4ø è H(w ) |w =p = 0

(11.64) (11.65)

348

Multiresolution and Multirate Signal Processing

Similarly,  (w ) | = 2 & H  (w ) | = 0 H w =0 w =p Conjugating H(w ) we get, H (w ) =

 (w ) × Hw \H

2 -2 jw 2 2 jw 2 - jw 3 2 ×e ×e + + 2 × e jw ×e 4 2 4 2

æ 2 - jw 2 2 jw ö æ 2 -2 jw 2 - jw 3 2 2 2 jw ö =ç ×e + + ×e ÷´ç ×e ×e + + 2 × e jw ×e ÷ ç 4 ÷ ç 4 ÷ 2 4 2 4 2 è ø è ø 9 1 1 = × e-3 jw + 1 + × e jw - × e3 jw 8 8 4 (11.66)

Replacing ‘w’ by ‘w + p ’ in Eq. (11.66)  (w + p ) × H (w + p ) H 1 9 1 = × e-3 j (w +p ) + 1 + × e j (w +p ) - × e3 j (w +p ) 8 8 4 9 1 1 = × e-3 jw × e-3 jp + 1 + × e jw × e jp - × e3 jw × e3 jp 8 8 4 Note, e3 jp = e-3 jp = e jp = e- jp = -1 (Euler’s identity)  (w + p ) × H (w + p ) = - 1 × e-3 jw + 1 - 9 × e jw + 1 × e3 jw \H 8 8 4 Now let us create wavelet transormation matrix for bio-orthogonal 5/3 tap let us call it w 5/3

w 5/3

é h0 êh ê -2 ê0 ê ê h2 =ê ê ê g0 ê0 ê ê0 ê ë g2

h1 h-1 0 0 g1 0 0 0

h2 h0 h-2 0 g2 g0 0 0

0 h1 h-1 0 0 g1 0 0

0 h2 h0 h-2 0 g2 g0 0

0 0 h1 h-1 0 0 g1 0

h-2 0 h2 h0 0 0 g2 g0

h-1 0 0 h1 0 0 0 g1

ù ú ú ú ú ú ú ú ú ú ú ú ú û

(11.67)

349

MRA Variant 1: Bi-orthogonal Filters

-1 / 2 1 ù 0 0 0 é 3 / 4 -1 / 2 1 / 4 ê -1 / 2 1 3 / 4 -1 / 2 1 / 4 0 0 0 úú ê ê 0 0 -1 / 2 1 3 / 4 -1 / 2 1 / 4 0 ú ú ê 0 0 0 -1 / 2 1 3 / 4 -1 / 2 ú ê 1/ 4 = 2ê - ú ú ê 0 0 0 0 0 ú ê 1 / 4 -1 / 2 1 / 4 ê 0 0 1 / 4 -1 / 2 1 / 4 0 0 0 ú ú ê 0 1 / 4 -1 / 2 1 / 4 0 ú 0 0 ê 0 ú ê 0 0 0 0 0 1 / 4 -1 / 2 û ë 1/ 4

� 5/3 Similarly, w

 5/3 \w

é h� 0 ê ê0 ê ê0 ê0 ê =êê� ê g0 ê ê0 ê0 ê êë g� 2

h� 1 h� -1

0 �h0

0 �h1

0

0

h� -1

0

0

0

0 h� 0

0 h� 1

0

0 -

h� -1 -

0 g�

0 g�

g� 1 g�

0 g�

0 g�

0 g�

g� -1

g� 0 0

g� 1 g�

g� 0

0

0

0

1

0 g�

3

2

3

-1

2

3

-1

0 �h0 0 0 �g 2 �g 0

h� -1 ù ú 0 ú ú 0 ú h� 1 ú ú -ú ú g� -1 ú ú 0 ú g� 3 ú ú g� 1 úû

0 1/ 4 ù 1/ 4 0 0 0 0 é 1/ 2 ê 0 1/ 4 1/ 2 1/ 4 0 0 0 0 úú ê ê 0 0 0 0 1/ 4 1/ 2 1/ 4 0 ú ú ê 0 0 0 0 1/ 4 1/ 2 1/ 4 ú ê 0 = 2ê - ú ú ê 1 1/ 2 0 0 0 -1 / 4 ú ê -1 / 2 -3 / 4 ê 0 0 0 ú -1 / 2 -1 / 2 -3 / 4 1 1/ 2 ú ê 0 0 -1 / 4 -1 / 2 -3 / 4 1 1/ 2 ú ê 0 ú ê 0 0 0 -1 / 4 -1 / 2 -3 / 4 1 û ë 1/ 2

it is clear, that  5/3-1 = w  5/3T w an extension: using this approach different bi-orthogonal taps can be studied.  6 /2 For example, let us design 6/2 tap tp produce w 6 /2 and w

350

Multiresolution and Multirate Signal Processing

Here, hk = {h-2 , h-1 , h0 , h1 , h2 , h3 } line of symmetry;

{

}

h k = h 0 , h 1 (1.9,0.5) line of symmetry; as the filters are symmetric, for h k , h 0 = h 1 for hk , h0 = h1 , h-1 = h2 , h-2 = h3 w = h + h .e jw = h + h .e jw H 0 1 0 0

(11.68)

H (w ) = h-2 .e-2 jw + h-1 .e- jw + h0 + h1 .e jw + h2 .e2 jw + h3 .e3 jw = h3 .e-2 jw + h2 .e- jw + h1 + h1 .e jw + h2 .e2 jw + h3 .e3 jw

(11.69)

and

from Eq. (11.68)  (w ) | = 2h = 2 Now H w =0 0 \

h0 = h1 =

2 2

(Haar!)

for hk , let us force low-pass conditions on H(w ). H(w ) |w =0 = 2 let us plug w = 0 in Eq. (11.69) \ 2h1 + 2h2 + 2h3 = 2 \ h1 + h2 + h3 =

2 2

similarly, H (w ) |w =p = 0 \ h3 - h2 + h1 - h1 + h2 - h3 = 0 -(holds good) Using Eq. (11.45) h0 .h 0 + h1 .h 1 = 1 \

2 2 h0 + h1 = 1, but h0 = h1! 2 2 2h0 = 1

351

MRA Variant 1: Bi-orthogonal Filters

\h0 = h1 =

1 2

from Eq. (11.46),

h0 .h -2 + h1 .h -1 = 0 z z \ h3 + h2 = 0, \ h2 = h3 , 2 2 h2 = h3 = h-1 = h-2 = ´ (some real no.) 2 2 ü ì \ hk = í x, - x, , , - x, x ý 2 2 þ î ïì 2 2 ïü and h k = í , ý ïî 2 2 ïþ Readers are encougraged to complete following steps: 1. calculate g k [1 ´ 6] from hk 2. calculate gk [1 ´ 2] from h k  6 /2 from h , h k , h , h k 3. calculate w 6 /2 and w k k

Exercises Exercise 11.1

1 Prove that < f (t ),f (t - m) > = rhh [2m] 2 Hint: As we know that, scaling function f(t ) can written as follows, f (t ) = åh[ n]f (2t - n) nÎZ

Similarly, f(t - m) can be written as, f (t - m ) =

å h[n]f (2t - 2m - n ) 1

n1ÎZ

So dot product of f(t ) and f(t - m) is given by, < f (t ),f (t - m) > = ååh[ n]h[ n1 ] < f (2t - n),f (2t - 2 m - n1 ) > n

n1

The inner product term on the right hand side can be evaluated as, +¥

< f (2t - n),f (2t - 2 m - n1 ) > = ò f (2t - n)f (2t - 2 m - n1 )dt -¥

(11.70)

352

Multiresolution and Multirate Signal Processing

Let 2t = l. It gives 2dt = dl and t : -¥ ® +¥ Þ l = -¥ ® +¥. With this substitution the above integral turns out to be, 1 +¥ f (l - n)f (l - 2 m - n1 )d l 2 ò-¥ 1 < f (2t - n),f (2t - 2 m - n1 ) > = d [ n - (2m + n1 )] 2

< f (2t - n),f (2t - 2 m - n1 ) > =

Putting value in Eq. (11.70) we get, < f (t ),f (t - m) > =

1 ååh[n]h[n1 ]d [n - (2m + n1 )] 2 n n1

(11.71)

< f (t ),f (t - m) > =

1 åh[2m + n1 ]h[n1 ] 2 n1

(11.72)

Dropping å n, we get

The above expression is the autocorrelation of h[.] evaluated at 2m, "m Î Z. We can write Eq. (11.72) as, 1 < f (t ),f (t - m) > = rhh [2 m] 2

Exercise 11.2

Show that

å

k ÎZ

y ( W + 2pk ) 2

2

= Constant 2

Given that G( e jW ) + G( e j ( W +p ) = Constant and

å

2

k ÎZ

f ( W + 2pk ) = Constant

Here, G is analysis high-pass filter. Hint: We know, the dilation equation for y (t ) is given as, y (t ) = åg[ n]f (2t - n)

(11.73)

nÎZ

Taking Fourier transform both side in Eq. (11.73) yˆ (W) = ò

+¥

-¥

åg[n]f (2t - n)e nÎZ

- j Wt

dt

(11.74)

353

MRA Variant 1: Bi-orthogonal Filters

+¥

First evaluate term ò f(2t - n)e- jWt dt -¥

Put 2t - n = l, It gives 2dt = dl and t : -¥ ® +¥ Þ l = -¥ ® +¥

ò

+¥

-¥

f (2t - n)e- jWt dt =

l +n - jW ( ) 1 +¥ 2 f ( l ) e dl ò 2 -¥

1 - jWn W = e 2 fˆ ( ) 2 2 +¥

Putting value of term ò f(2t - n)e- jWt dt in Eq. (11.74), we get -¥

W -j n 1 W yˆ (W) = å g[ n]e 2 fˆ ( ) 2 nÎZ 2 W -j n 1 W yˆ (W) = [ åg[ n]e 2 ]fˆ ( ) 2 nÎZ 2 W j 1 W yˆ (W) = G(e 2 )fˆ ( ) 2 2

(11.75)

Now if we take term å kÎZ yˆ (W + 2pk ) and evaluate using Eq. (11.75), we get 2

2

kÎZ

2

W j ( + pk ) 1 W G(e 2 ) fˆ ( + pk ) 2 kÎZ 2 2

å yˆ (W + 2pk ) = å

2

(11.76)

Above equation 11.76 can be also written as, 2

2

W W j ( + 2 pk ) j ( + 2 pk + p ) 1 W 1 W yˆ (W + 2pk ) = å( )2 G(e 2 ) fˆ ( + 2pk ) + å( )2 G(e 2 ) fˆ ( + 2pk + p ) å 2 2 kÎZ kÎZ 2 kÎZ 2 2

2

2

(11.77) In Eq. (11.77) terms G(e

j(

W + 2 pk ) 2

2

) and G(e

j(

W + 2 pk + p ) 2

2

2

W ) are periodic. Also, terms fˆ ( + 2pk ) 2

W and fˆ ( + 2pk + p ) are constant as given in question. 2 Let the constant be denoted by C0 . So, now, Eq. (11.77) can be written as, 2

W W j( ) j ( +p ) C C yˆ (W + 2pk ) = 0 G(e 2 ) + 0 G(e 2 ) å 4 4 kÎZ 2

2

2

354

Multiresolution and Multirate Signal Processing

2 ∑ yˆ (Ω + 2pk ) = k ∈Z

C0 4

2 2 Ω Ω   j( ) j ( +p ) 2 2 )   G (e ) + G (e  

(11.78)

2

2

But also in question given that power complementary property, i.e G(e jW ) + G(e j ( W +p ) ) = Constant(C1). So by using this property RHS of Eq. (11.78) become a constant. Now we can write,

å y ( W + 2pk ) k ÎZ

2

=

C0 C1 = Constant 4

Exercise 11.3

Find the dilation equation of f2 (t ) = f0 (t ) ´ f0 (t ) ´ f0 (t ). Here f0 (t ) is the Haar Scaling function. Hint: For solving this question we are using property that has already been discussed in this chapter and is now given here also, h(at + b) ´ g(at + c) =

1 h ´ g |( at + b + c ) a

(11.79)

Dilation equation for f0 (t ) is given as, f0 (t ) = f0 (2t ) + f0 (2t - 1)

(11.80)

Let f1 (t ) = f0 (t ) ´ f0 (t ), Using property of Eq. (11.79) dilation equation of f1 (t ) can be written as, f1 (t ) = (f0 (2t ) + f0 (2t - 1)) ´ (f0 (2t ) + f0 (2t - 1)) 1 f1 (t ) = [f1 (2t ) + 2f1 (2t - 1) + f1 (2t - 2)] 2

(11.81)

Now f2 (t ) = f1 (t ) ´ f0 (t ), so dilation equation of f2 (t ) given as, 1 f2 (t ) = [f1 (2t ) + 2f1 (2t - 1) + f1 (2t - 2)] ´ f0 (2t ) + f0 (2t - 1)) 2 1 f2 (t ) = [f2 (2t ) + f2 (2t - 1) + f2 (2t - 2) + f2 (2t - 1) + f2 (2t - 2) + f2 (2t - 3)] 4 1 1 1 1 f2 (t ) = f2 (2t ) + f2 (2t - 1) + f2 (2t - 2) + f2 (2t - 3) 4 2 2 4 Equation (11.82) gives the dilation equation of f2 (t ).

(11.82)

355

MRA Variant 1: Bi-orthogonal Filters

Exercise 11.4

Explain, why Linear phase is not possible for Daubechies family other than Haar? Hint: Linear phase means that phase response of filter is the linear function of frequency. This means delay introduced by filter is same at all frequency. Linear phase requires symmetry in impulse response of filter. That means any filter in linear phase requires that its coefficients are symmetrical around the centre coefficients, i.e. first coefficients is same as last coefficients; second is the same as second last, and so on. One other way to seeing this that zeros must be occur in reciprocal pairs. So in Daubechies family this type of symmetry is not occurred so it does not give the linear phase.

Exercise 11.5

The two-band filter bank shown in Fig. 11.13, is the 5/3 filter bank used in JPEG 2000 standards. The filters are defined as follows: 1 H 0 ( Z ) = (-1 + 2 Z -1 + 6 Z -2 + 2 Z -3 - Z -4 ) 8 1 H1 ( Z ) = - (1 - Z -1 )2 2 1 G0 ( Z ) = (1 + 2 Z -1 + Z -2 ) 2 1 G1 ( Z ) = (1 + 2 Z -1 - 6 Z -2 + 2 Z -3 + Z -4 ) 8

H0(z)

2

2

G0(z)

Analysis

+

Input

Output H1(z)

2

Figure 11.13

2

G1(z)

Two-band filter bank

356

Multiresolution and Multirate Signal Processing

(a) Obtain and sketch the magnitude and frequency responses of the low-pass synthesis and the high-pass analysis filters. (b) Obtain scaling and wavelet functions corresponding to synthesis filter bank. Hint: (a) Low-pass synthesis filter transfer function: 1 G0 ( Z ) = (1 + 2 Z -1 + Z -2 ) 2 1 −   1 1 = Z −1  Z 2 + Z 2  2  

2

Put Z = e jw, hence frequency response is as follows: jw −  jw  1 = e − jw  e 2 + e 2  2  

2

 w = 2e − jw cos 2    2 Magnitude response is shown in Fig. 11.14. 2 1.8 1.6

Magnitude->

1.4 1.2 1 0.8 0.6 0.4 0.2 0 −4

−3

−2

Figure 11.14

−1

0 w->

1

2

3

Magnitude response of low-pass filter

4

357

MRA Variant 1: Bi-orthogonal Filters

w G0 ( Z ) = 2e− jw cos 2( ) 2 w = 2 cos 2( ) 2 = 1 + cos w ∠ G0 ( Z ) = −w High-pass analysis filter transfer function: 1 H1 ( Z ) = - (1 - 2 Z -1 + Z -2 ) 2 1 1 1 = - Z -1 ( Z 2 - Z 2 )2 2

Put Z = e jw, hence frequency response is as follows: jw −  jw  1 = − e − jw  e 2 − e 2  2  

2

 w = 2e − jw sin 2    2 Magnitude and phase of the above frequency response are shown in Fig. 11.15 and in Fig. 11.16.  w H1 ( Z ) = 2e − jw sin 2    2 = 1 − cos w ∠ H1 ( Z ) = − w Hint: (b) The scaling function satisfies the dilation equation: 1 f (t ) = (f1 (t ) + 2f1 (t - 1) + f1 (t - 2)) 2 The wavelet function obtained from synthesis highpass filter impulse response: 1 (− 1 − 2 6 − 2 − 1) 8 ↑0 The wavelet function is given by following expression and shown in Fig. 11.17. 1 (-f1 (t ) - 2f1 (t - 1) + 6f1 (t - 2) - 2f1 (t - 3) - f1 (t - 4)) 8

358

Multiresolution and Multirate Signal Processing

2 1.8 1.6

Magnitude->

1.4 1.2 1 0.8 0.6 0.4 0.2 0 −4

−3

−2

Figure 11.15

−1

0 w->

1

2

3

4

Magnitude response of high pass filter

4 3 2

Phase->

1 0 −1 −2 −3 −4 −4

−3

−2

−1

Figure 11.16

0 w->

1

Phase response

2

3

4

359

MRA Variant 1: Bi-orthogonal Filters

6 8

0

1

2

3

1.5

−2 8

Figure 11.17

−2 8

Wavelet function for synthesis filter bank

Chapter

12

MRA Variant 2: Splines

Introduction 5/3 Filter bank Derivation of orthogonal MRA Building piecewise linear scaling function, wavelet Building of piecewise linear scaling function

12.1

Summary

Introduction

In this chapter, we shall continue to discuss one more variant on the idea of MRA. In earlier chapters, we had built an idea on the orthogonal or perfect reconstruction with one filter and also extended it to biorthogonal or perfect reconstruction with two filters. We had taken the example of a 5/3 filter bank in JPEG 2000. When we extend the MRA to 2 filters, it involved building a filter bank and not really the MRA. So, we had essentially built a perfect reconstruction filter bank where the filters were of unequal lengths. But we saw that we could get the advantage of linear phase and symmetry in the impulse response. Further, we could extend what we did in the Haar case to a piece-wise linear function. So, if we take 5/3 filter bank and if we look at the 3 length LPF, the filter has an impulse response of (1 + z −1 )2 . It would essentially give us the triangular function as the scaling function. The disadvantage with the triangular function was that it was not orthogonal to all its integer translates; it was orthogonal when translated by two units or more, but it was not orthogonal when translated by one unit. Thus, we had to venture to other areas by looking at variants of the filter banks that we had already discussed and bring in the idea of a biorthogonal filter bank. Now, in this chapter we shall take the same (1 + z −1 )2 again; the length 3 LPF that we have seen in the 5/3 filter bank. But we shall deal with it in a slightly different way, which would bring us to the idea of orthogonal MRA with splines. Here, we need to make a compromise in the nature of the scaling and the wavelet function that we construct and also in the nature of the filter bank that we would build. In fact, as a consequence of our demanding an orthogonal MRA we shall have to go from finite length to infinite length filters.

12.2

5/3 Filter Bank

1 (1 + z −1 )2 as the 2 system function. We know that the corresponding scaling function is f1(t ) and it obeys the dilation equation: If we look at the length-3 Low-pass filter in the 5/3 filter bank, it essentially has

1 1 f1(t ) = f1(2t ) + f1(2t - 1) + f1(2t - 2) 2 2

(12.1)

362

Multiresolution and Multirate Signal Processing

f1(t ) has an appearance as shown in Fig. 12.1. 1 0.9 0.8 0.7 0.6 f1 (t) 0.5 0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

Figure 12.1

0.8

1 1.2 Time(t)

1.4

1.6

1.8

2

The scaling function

Now, our main problem and the reason as to why we need to go for a bi-orthogonal filter bank, as opposed to an orthogonal filter bank, is that this scaling function is not orthogonal to its translates by unity. So, if we translate this by unity and take the dot product essentially between f1(t ) and f1(t - 1), i.e., if we consider these two dot products,

� f1(t ), f1(t − 1) � and � f1(t ), f1(t + 1) �

(12.2)

we find that these two are nonzero. This was our main goal of contention because of which we would not be satisfied with this f1(t ) to construct an orthogonal MRA out of it. Now, we wish to explore that possibility. Even though f1(t ) is not orthogonal to its own integer translates, we wish to construct an orthogonal MRA from a function that looks similar to f1(t ), or in other words, out of a function that is piece-wise linear. We would build an orthogonal MRA with scaling functions, which are piece-wise linear even though not exactly f1(t ) . So, what we are saying is, Can we build a multi-resolution analysis with a piece-wise linear function for f and y ? We shall try to answer the question in this chapter and for that we must relax the requirement of orthogonality.

12.3

Derivation of Orthogonal MRA

The notion of orthogonality of f (t ) to its integer translates, is expressed in terms of the autocorrelation of f , i.e. the autocorrelation function of f (t ) at all the integers except at ‘0’ is 0, i.e. Rff (t ) = 0 for all t ∈ except for t = 0, where Rff (t ) is the autocorrelation function of f (t ). This is the basic principle of a function being orthogonal to its integer translates.

363

MRA Variant 2: Splines

Example 12.3.1

Let the scaling function be f (t ) and it is orthogonal to its integer translates f (t − m), where m ∈. Essentially, we mean to say that the autocorrelation function Rff (t ) when sampled at t = m, m ∈ (i.e., we sample at all integers, or at a sampling rate of 1) gives an impulse sequence. Mathematically, the sequence is a discrete impulse sequence. Now, we need to deal with it in the frequency domain. So, when we sample it, the Fourier transform of the autocorrelation function gets aliased. In fact, we know that the Fourier transform of the autocorrelation function is |f( W) |2 where f (W) is the Fourier transform of the scaling function f (t ). In other sense, we can say that the Fourier transform of the autocorrelation function is the power spectral density of the scaling function in the frequency domain. Now sampling Rff (t ) at t = m for m ∈, means summing up all the aliases of |f( W) |2 in the Fourier domain. In other words, constructing the sum, ∞

KK0 ∑ | f|f(Ω + 2ppkk) |2| , w , where K 0 is a constant.

(12.3)

k = −∞

Recall that when we sample a continuous function, its Fourier transform shifts on the frequency axis by every multiple of the sampling frequency and adds up all these translates or aliases. In a way, the sequence Rff (t ) |t = m is an impulse sequence, then its DTFT must be a constant. And, therefore, we now have a clear cut criterion in the frequency domain. In order that the f (t ) is orthogonal to its integer translates, we require that the quantity, sum of aliases of the power spectral ∞ density∞ must be a constant. In other words, K 0 ∑ k = −∞ | f (Ω + 2pk ) |2 is a constant. We shall call 2  K 0 ∑ k = −∞ | f (Ω + 2pk ) | as the sum of translated spectra of f (.) . We shall abbreviate sum of translated spectra by STS. STS (f ,2p )(Ω) has Ω as the primary argument and f, 2p as secondary arguments. In general, STS (f , T )(Ω) = K 0

∞

∑ | f (Ω + Tk ) |

2

k = −∞

(12.4)

With this little notational introduction, we take the same strategy as we did when we relaxed the condition for the sum of dilated spectra. We know that when we discretize the scale, we need to essentially relax the requirement of sum of dilated spectra to be a constant, where it is between two positive constants. If one gets the sum of dilated spectra to be a constant, we shall be happy if it is between two positive (strictly non zero and finite) constants. Similar will be true for this case. In fact now, we also bring out a relationship between relaxation of this requirement in the t domain or shift domain and frequency domain. Now, if we look back, it is easier to start from the f domain. Next, we take the function f1(t ) which is as shown in Fig. 12.1. The dot product of f1(t ) with its integer translates is zero only when the translates are greater than 2. Hence, we need a relaxation when the translate is equal to 1 and –1. The relaxation that we are asking for is that the dot product is zero. We can even actually calculate the dot product. The dot product of f with itself would have certain value and it is equal to the energy of the function. And if we take the dot product of f with its translates by 1 and –1, they are expected intuitively to have a smaller value. So in other words the relaxation we are asking for is that the auto correlation of this function is not quite as an impulse, but close to an impulse. This means it is a nonzero for very few values around n = 0 and that manifests in the frequency domain as the STS, which is not a constant but between two positive constants. We shall calculate these quantities now and prove mathematically. So, for f1(t ) , by looking at Fig. 12.1, we can observe that,

364

Multiresolution and Multirate Signal Processing

Rf1f1 (1) = Rf1f1 (-1) Let us now find

ò

¥

(12.5)

f (t )f1(t - 1)dt. In fact, this quantity can be easily calculated by shifting both

-¥ 1

the functions to left by 1 unit as shown in Fig. 12.2. Equation (12.5) now becomes, 1

1

∫ t (1 − t )dt = 6 0

Now, in a similar way Rf f (0) can be calculated, 1 1 2

Rf1f1 (0) = ò f12 (t )dt 0

2 = 3 ì1 2 1 ü ,Rf1f1 (t ) |t = m,mÎ = í , , ý î6 3 6 þ

Therefore,

1 Therefore, DTFT of Rf1f1 (t ) |t = m,mÎ = e jW 6

↑ 2 1 + + e- jW . From this equation we can infer that 3 6

1 0.9 0.8

(t)

(1−t)

amplitude

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

Figure 12.2

0.1

0.2

0.3

0.4

0.5 0.6 Time

0.7

0.8

0.9

1

Graphical representation of dot product between f1(t ) and f1(t - 1)

365

MRA Variant 2: Splines

¥

1 2 1 = e jW + + e - jW 6 3 6 k = -¥ (12.6) 1 2 = (1 + cos(W)) 3 2 As expected, this sum is strictly positive and lies between two positive constants and it can be 1 observed that the constants are and 1, i.e. 3 1 2 1 ≤ (1 + cos(Ω)) ≤ 1 (12.7) 3 3 2 K0

å | f (W + 2p

k ) |2

So relaxation of the requirement in time domain has also led to a corresponding relaxation in the frequency domain. And, now, we can employ the same strategy (as we did when we relaxed the requirement of the sum of dilated spectrum). We can see that the sum of translated spectrum lies between two positive bounds. We could say that even though f1(t ) by itself gives us a multi-resolution analysis. The question arises, can we construct another function f1(t ) out of f1(t ) by using the sum of translated spectrum in such a way that f1(t ) gives us an orthogonal MRA. Let us strategically define such a f1(t ) , as we did while taking inspiration from sum of dilated spectrum. So, let us define f1(t ) in terms of its Fourier transform. Therefore, ˆ f1(W) =

fˆ1(W)

(12.8)

+ STS(f1,2p )(W)

Let us justify this definition by noting that the denominator is between two positive bounds. Denominator is known to be between 1/3 and 1. So, this division will not go up towards infinity and neither it will go to zero. Now, consider the denominator, i.e., STS(f1,2p )(W). This exhibits an important property of periodicity with period 2p , i.e., STS(f ,2p )(W)

= STS(f ,2p )(W + 2p ) "W ¥

STS(f ,2p )(W + 2p ) =

å | f (W + 2p + 2p

k ) |2

k = -¥ ¥

=

å | f (W + 2p (k + 1)) |

2

k = -¥

(12.9)

As, k runs from −∞ to ∞ , k + 1 also has the same limits. Hence, the summation in Eq. (12.9) becomes STS(f ,2p )(Ω) . Using this result we are going to find STS(f1,2p )(Ω). | f� 1(W + 2p k ) |2 å � k = -¥ STS(f 1,2p )(W + 2p k ) ¥

STS(f�1,2p )(W) = =

¥ 1 | f� (W + 2p k ) |2 å STS(f1,2p )(W) k = -¥ 1

=

STS(f1,2p )(W) STS(f1,2p )(W)

=1

366

Multiresolution and Multirate Signal Processing

From the above result it is clear that STS(f1,2p )(W) is constant and is equal to one. Therefore, we can say that the underlying continuous function f1(t ) is orthogonal to its integer translates. To characterize f1(t ) , let us consider its Fourier domain. We have, ˆ f1(W) =

fˆ1(W)

(12.10)

+ STS(f1,2p )(W)

Substituting the value of STS(f1,2p )(Ω) from Eq. (12.6), we get ˆ f1(W) =

fˆ1(W) 2æ 1 ö ç 1 + cos(W) ÷ 3è 2 ø

(12.11)

Expanding Eq. (12.11) in the form of binomial expansion, i.e., -

1

æ2ö 2 æ 1 ö ˆ f1(W) = fˆ1(W) ç ÷ ç 1 + cos(W) ÷ 3 2 è ø è ø

-

1 2

(12.12)

This equation is of the form (1 + l ) R , R ∈ , we know that (1 + l ) R = 1 + R.l +

R( R − 1) 2 R( R − 1)( R − 2) 3 l + l + ... 2! 3!

(12.13)

th Comparing Eqs. (12.12) and 12.13 and expanding Eq. (12.12) gives a typical p term, which can be represented as follows,

æ1ö K p l = K p (cos(W)) ç ÷ è2ø p

p

p

(12.14)

p

æ e jW + e - jW ö We know that, cos(W) = ç ÷ . Using this result in Eq. (12.14) and expanding it, we get a 2 è ø final expression which looks like p

ˆ f1(W) =

¥

å C e

j Wk

k

fˆ1(W)

(12.15)

k = -¥

Now, from Eq. 12.15 we can find the Inverse DTFT to get f1(t ) easily. We know that multiplication by the term e jΩk in Fourier domain shifts the signal by k in time domain. Therefore, f1(t ) turns out to be, f1(t ) =

∞

∑ C f (t + k ) k 1

k = −∞

(12.16)

367

MRA Variant 2: Splines

From Eq. (12.16), f1(t ) turns out to be a linear combination of f1(t ) shifted by integer translates. When we shift a piece-wise linear function by integer translates and add them, we still get a piece-wise linear function. Hence, f1(t ) is a piece-wise linear function. In subsequent chapters we shall study the nature of C k ’s and know how to construct an MRA out of f1(t ) .

12.4

Building Piece-wise Linear Scaling Function, Wavelet

In earlier sections, we began with building piece-wise linear MRA. We also saw the piece-wise linear function obtained by convolving the Haar Scaling function with itself, as shown in Fig. 12.3. f1(t ) = f0 (t ) × f0 (t )

(12.17)

This f1(t ) is not orthogonal to its integral translates. This led to Sum of Translated Spectra (STS) not being a constant. It was only 1 and -1 which were problematic translates. But as expected, the STS was a constant within two positive bounds. As a result, one could extract from f1(t ) another function, which was orthogonal to its own translates. This new function can be used to build MRA based on piecewise linear scaling function and wavelets.

12.5

Building of Piece-wise Linear Scaling Function

We saw that: ¥

STS(f1,2p )(W) =

å

| f1(W + 2p K ) |2

(12.18)

K = -¥

This is not a constant as required but it lies between two positive bounds. 0 < A £ STS(f1,2p )(W) £ B < ¥ B =3 A

(12.20)

1 |f1(t)|

0

Figure 12.3

1

(12.19)

2

t

The linear scaling function

If we scale a function by a constant, the STS is scaled by the square of that constant.

368

Multiresolution and Multirate Signal Processing

In Chapter 11, we had already discussed that we can define a function f1(W) as: ˆ f1(W) = STS(f1,2p )(W) =

f1(W)

(12.21)

+ STS(f1,2p )(W) STS(f1,2p )(W) = 1(Constant ) STS(f1,2p )(W)

(12.22)

Both numerator and denominator are greater than 0 and less than ¥ and hence cancellation is possible. Therefore, f1(t ) is orthogonal to all its integer translates, i.e., < f1(t ), f1(t − m) > = 0 ∀ m ∈ Z

(12.23)

Now, we saw that it is of the form, f1(W)

ˆ f1(W) =

2æ 1 ö ç 1 + cos W ÷ 3è 2 ø

+

(12.24)

Using binomial expansion or Taylor series expansion, i.e., (1 + g ) R , R ∈ , | g |< 1 (1 + g ) R = 1 + Rg +

R( R − 1) 2 R( R − 1)( R − 2) 3 g + g + 2! 3!

(12.25)

Therefore, we get 1

f (W) æ 1 ö 2 ˆ 1 + W f1(W) = 1 cos ç ÷ 2/3è 2 ø

æ e jW + e - jW ö Now, cos W = ç ÷ 2 è ø Therefore,

(12.26)

N

N

ˆ f1(W) =

¥

åC

l

e jWl fˆ1(W)

(12.27)

l = -¥

The calculation of C l is cumbersome as for each C l we have to write a series. Now, C l = C − l from symmetry in expanding (cos Ω)l . Therefore, if we take the Inverse Fourier transform of Equation 12.27, we get, ¥

f1(t ) =

åC

l

l = -¥

f1(t + l )

(12.28)

369

MRA Variant 2: Splines

Therefore, f1(t ) is a linear combination of f1(t ) and its integer translates, as shown in Fig. 12.4. This shows that f1(t ) is piece-wise linear. For further clarity, one can calculate few C l , may be for l = 0, ± 1, ± 2 and we will see that C l and C − l decay as l → +∞ . C0f1(t)

~ Clf1(t + l) terms

~ f1(t)

−4

−3

Figure 12.4

−2

0

−1

1

2

3

4

t

Representing f1(t ) as a piecewise linear function

Example 12.5.1 — Properties of f1(t ) :

1. Piecewise linear: A sum of piecewise linear functions. 2. f1(t ) should be orthogonal to all its integer translates, i.e. < f1(t - m), f1(t - n) >= 0; "m, n Î Z ; m ¹ n 3.

(12.29)

f1(×) obeys the dilation equation.

Property 3 is critical for MRA because it is this dyadic dilation equation which ensures that f1(t ) , when dilated by a factor of 2 and then translated by all the integers, constructs a basis for next subspace, V0. This subspace V0 is spanned by f1(t ) and its integer translates and then we have a subspace which is spanned by f1(t ) contracted by a factor of 2 and its integer translates. We shall use the frequency domain to prove this. Time domain dyadic equation for a general scaling function f (⋅) : ∞

f (t ) =

∑ h[k ]

f (2t − k )

(12.30)

k = −∞

Taking Fourier transform: 1  Ω  Ω fˆ(Ω) = H   fˆ   2  2  2 where H(⋅) is the DTFT of h[ k ] .

(12.31)

370

Multiresolution and Multirate Signal Processing

ˆ To establish a dyadic dilation equation on f1(×) we essentially need to consider 1 W that this is of the form H æç ö÷ where: 2 è2ø ¥

H (W) =

å h[k ] e

- jk W

k = -¥

Essentially, we need to establish that

ˆ f (W) 1

ˆ æWö f1 ç ÷ è2ø

ˆ f1(W) , and show ˆ æWö f1 ç ÷ è2ø

(12.32)

is a DTFT. That is, it is

(i) Periodic with period 2p (ii) It is bounded on any interval of 2p, so that its IDTFT can be calculated. Boundedness is needed, therefore: 1 f (W) e jWn d W must converge, ò 2p ˆ f (Ω) where where f (Ω) = 1 ˆ  Ω f1    2

(12.33)

1/2

æ2æ 1 W öö fˆ1(W) ç ç 1 + cos ÷ ÷ 2 øø è3è 2

ˆ f1(W) = 1/2 ˆ æWö f1 ç ÷ æ 2 æ 1 + 1 cos W ö ö ÷÷ è 2 ø çè 3 çè 2 øø

(12.34)

æWö fˆ1 ç ÷ è2ø 1/ 2

é 2 æç 1+ 1 cosW ö÷ ù 2 ÷ø ú ê 3 çè 2 Now, we focus on the term inside the square root, i.e., ê æ . This is a function of Ω , 2 ç 1+ 1 cosW ö÷ ú êë 3 èç 2 ø÷ ú û 1 æWö and hence it is periodic. This is definitely of the form H ç ÷ with properties of H(⋅) desired. Now, 2 è2ø if we replace Ω by 2Ω in Eq. (12.34) (term inside the root), we get, é 2æ 1 öù ê 3 ç 1 + 2 cos W ÷ ú øú ê è öú ê2æ 1 ê 3 çè 1 + 2 cos 2W ÷ø ú û ë

1/2

(12.35)

In this expression, the numerator is periodic with period 2p and the denominator is periodic with period p and hence the overall expression is periodic with period 2p. Therefore the periodicity is established.

371

MRA Variant 2: Splines

Now, we look at the boundedness. Let us say that in the expression(inside the root): 0 < A ≤ Numerator ≤ B < ∞

(12.36)

and 0 < A ≤ Denominator ≤ B < ∞ Therefore, we can say that the fraction is also between two positive bounds as: 0
1 . Haar filter are symmetric, however they have some inherent drawbacks and as we move higher in the family, the higher members lack symmetry. This is precise where ‘symlets’ play important role and they produce symmetric or near symmetric filters, thus producing scaling function f(.) to be linear phase or nearly linear phase. Since these filters are ‘symmetric’ they are called as ‘symlets’. Example 6.2.1 – D2 Calculations from Chapter 6 depicts the complete design of the D2 filter. It derives the hk components from B0 and in Eq. (6.4) we have explained choice made by Daubechies to select B0 = -2 + 3 . This value keeps it inside the unit circle and when all the zeros are inside unit circle, such systems are called as minimum phase systems. These filters, however are very asymmetric and it can be shown that these filters have their energy concentrated close to the starting coefficients of the filter. Daubechies has shown that Haar filter is the only filter to have linear support though it belongs to the compactly supported conjugate mirror family. The ‘symmlets’ are obtained by selecting the B0

450

Multiresolution and Multirate Signal Processing

value for obtaining near linear phase. The resulting y (.) has a minimum support [ - p + 1, p] with p vanishing moments, however they are near symmetric. The filters are depicted in Fig. 15.3.

1.5 1 0.5 0 −0.5 1 0.5 0 −0.5

Scaling Function (f(.))

0

5 Lowpass Analysis Filter (hk)

0

5

10

10 Highpass Analysis Filter (gk)

15

0.5 0 −0.5 −1

0

5

10

15

15

Wavelet Function (y (.)) 2 1 0 −1

0

5

10

15

Sym8 Wavelet

Figure 15.3

Symlet Wavelet (sym8). Analysis Low and High pass filters along with Sacling (f (.)) and wavelet (y (.)) functions

Figure 15.4 has three parts. (A) shows h as analysis low pass filter, g as analysis high pass filter, rh as synthesis low pass filter and rg as synthesis high pass filter. (B) shows the magnitude response of all the four filters and (C) shows the phase response of these filters. From (C) it is clear that the ‘symmlets’ have near linear phase and near symmetric scaling and wavelet equations emerging out of hk and gk . Following MATLAB example depicts generation of discrete wavelet kernels, which is used for different diagrams in this chapter.

Other Wavelet Families

15.4

451

Morlet Filters

Jean Morlet J. Morlet (born 13 January 1931) is French geophysicist. He is considered to be the one who invented the term ‘wavelet’. In 1981 he worked with Grossman to develop what is known to us as ‘wavelet transform’. He was awarded in 1997 with the Reginald Fessenden Award for his contributions in the field of wavelets.

452

Multiresolution and Multirate Signal Processing

h

0.8

g

1

0.6 0.5

0.4 0.2

0

0 −0.2

0

5

10

−0.5

15

0

5

h* 0.8

10

15

10

15

g*

1

0.6 0.5

0.4 0.2

0

0 −0.2

0

5

10

−0.5

15

0

5

(a)

H(Magnitude) [dB]

G(Magnitude) [dB]

100

100

0

0 −100

−100 −200 −200 −300

−300 0

0.5

1

1.5

2

2.5

3

H*(Magnitude) [dB]

100

3.5

−400

0

0.5

1

1.5

2

2.5

3

3.5

3

3.5

G*(Magnitude)[dB]

100 0

0

−100 −100 −200 −200 −300

−300 0

0.5

1

1.5

2

Figure 15.4

2.5

3

−400 3.5 0 (b)

0.5

1

1.5

Symlet filters and filter responses (Continued)

2

2.5

453

Other Wavelet Families

arg(H)

4 2

2

0

0

−2

−2

−4

0

0.5

1

1.5

2

2.5

3

3.5

arg(H*)

4

−4

2

0

0

−2

−2 0

0.5

1

1.5

2

0

0.5

1

2.5

3

−4 3.5 0 (c)

Figure 15.4

1.5

2

2.5

3

3.5

2.5

3

3.5

arg(G*)

4

2

−4

arg(G)

4

0.5

1

1.5

2

(Continued)

J. Morlet for doing analysis of seismic data proposed the ‘Morlet Filters’ in 1983. This is a different mother basis function than what we have discussed so far. This mother wavelet does NOT have the corresponding scaling function and is a kernel meant for a typical continuous domain processing. This book emphasizes more on the Discrete Wavelet Transform (DWT) and seldom we have touched upon the Continuous Wavelet Transform (CWT). This is because the modern signal processing is predominantly digital or discrete in treating the signals for the numerous advantages of digital domain over analog, which are discussed in depths in basic texts on signal processing and hence we shall not discuss it here. In DWT the signals are sampled and hence are discrete, thus f Î l 2 () (for non-periodic signals for whom we seek transform solutions). In DWT the dilation and translation parameters are also discrete. There is no harm, thematically at least, in keeping the dilation and translation parameters continuous. If we call these parameters as a and b respectively, then the the wavelets formulae is CWT sense can be written as: y a,b (t ) = | a |y (a(t - b)), a ¹ 0, a, b Î 

(15.21)

with y Î L2 . Morlet function is used in CWT analysis. This function does not have corresponding f(.) scaling function and the y (.) function satisfies the admissibility condition only approximately. One way of looking at the Morlet function is it’s a modulated version of the Gaussian kernel. 2 Mathematically it is represented as: y (t ) = eiat e- t /2 , where the a parameter needs to be chosen carefully. This wavelet does NOT satisfy the condition yˆ (0) = 0 , however for a ³ 5.5 the conditions gets almost satisfied with negligible error.

454

Multiresolution and Multirate Signal Processing

The filters are depicted in Fig. 15.5(a). (a) Morlet wavelet

1 0.5 0 −0.5 −1 −4

−3

−2

−1

0

1

2

3

4

2

3

4

(b) Mexican hat wavelet

1 0.5 0 −0.5 −4

Figure 15.5

−3

−2

−1 0 1 Continuous Wavelet Kernels

Continuous Wavelet Kernels. (a) depicts Morlet Wavelet and (b) depicts Mexican Hat Wavelet

Following MATLAB example demos continuous wavelet kernels.

455

Other Wavelet Families

15.4.1  Morlet Wavelet Design Strategy Morlet wavelet is a kernel for CWT. CWT Wx (b, a) of a continuous-time function x(t ) is defined as: Wx (b, a) =| a |

−

1 2

∫

 t − b ⋅ dt x (t ) ⋅ y ∗  −∞  a  ∞

(15.22)

Therefore, CWT is an ‘inner’ or ‘dot’ product between the signal to be analyzed x(t ) & scaled and translated version of mother wavelet y (t ) . The ‘a’ scales the wavelet function and ‘b’ shifts or translates it. One way of understanding CWT is to consider y (t ) to be a bandpass impulse response and then CWT becomes a bandpass analysis. This D change in a := Da will directly affect bandwidth and central frequency (first two statistical parameters) of the bandpass. For fixed values of ‘a’, the transform in Eq. (15.22) becomes time-reversed and scaled wavelet function convolved with the function x(t ) .

Wx (t , a ) = | a | -

Factor | a |

1 2

−

1 2

 −t  x(t ) ∗ y a (t ),y a (t ) = y ∗    a

(15.23)

is the normalizing factor and it makes the basis orthonormal. This ensures

æ -t ö " | a | y * ç ÷ with a Î IR the energy remains same. For wavelet transform,while doing effective è aø representations of signal x(t ) , it is important to ensure perfect reconstruction from the representations. 1 2

456

Multiresolution and Multirate Signal Processing

Mathematically, | Y (w ) |2 × dw < ¥ -¥ |w |

Cy = ò

¥

(15.24)

where, .T y (t ) ¾F¾ ®Y (w )

To Satisfy Eq. (15.24) ¥

Y (0) = ò y (t ) × dt = 0

(15.25)

-¥

This is known to us as ‘admissibility condition’ of wavelet y (t ). The | Y (w ) | must decrease rapidly for | w |® 0 approaching | w |® ¥ . Therefore, y (t ) must have bandpass impulse response, which results into a small wave or a compactly supported ‘wavelet’. One example of such type of wave is complex Morlet wavelet, which is nothing else but a plane wave modulated by a Gaussian envelope. y (t ) = e × e iat

-

t2 2

(15.26)

Here, parameter ‘a’ is the wavenumber associated with Morlet wavelet. ‘a’ is connected with the number of oscillations of morlet mother wavelet. The Fourier transform of Eq. (15.26) will be, yˆ (W) = 2p × e

1 - (2 pW -a )2 2

(15.27)

For admissibility condition to get fulfilled at W = 0, Eq. (15.27) should be ‘0’. It is easy to see, however, that Eq. (15.27) doesn’t satisfy admissibility condition of wavelet. \ The zeroth moment doesn’t vanish!. In order to make this a wavelet, we enforce the admissibility condition by setting  (W = 0) = 0 y If we evaluate Eq. (15.27) at W = 0 , then it gives an additional factor of e admissibility is not met. To compensate for the same, Eq. (15.27) can be modified as follows: 1 1 - a2 - (2 pW )2 ù é - 1 (2pW -a )2 - e 2 ×e 2 yˆ (W) = 2p êe 2 ú ë û Factor to contracct admissibility

(15.28) -

a2 2

because of which the

(15.29)

457

Other Wavelet Families

When above Eq. (15.29) is evaluated at W = 0 , it gives us −a 2   −a 2  2 y (Ω) |Ω =0 = 2p e − e 2 ⋅ e0  = 0  

-

(15.30)

a2

The correction factor of e 2 is important and also modifies the representation in time domain accordingly. The Morlet wavelet adjusted for admissibility thus becomes, -a 2 æ iat y (t ) = ç e - e 2 ç è

ö - t2 ÷×e 2 ÷ ø

(15.31)

The admissibility is achieved as in Fourier domain (Eq. (15.29)) the correction decrease rapidly as we move away from W = 0 point on W -axis. For the wavelet to have compact support at given dilation ‘a’, it’s frequency gets centered at: Wy W= , 2pa where ¥

Wy

ò W | yˆ (W) | ×dW = ò | yˆ (W) | ×dW 0

¥

(15.32)

0

For Complex Morlet, center frequency is controlled by ‘ a ’. ∴

Ωy = a

Following MATLAB example depicts creation of the Morlet kernel.

(15.33)

458

Multiresolution and Multirate Signal Processing

The outcome of the code above is shown in Fig. 15.6.

15.5

Mexican Hat Filters

This basis function is used in CWT analysis just like Morlet basis. Though the function and it’s Fourier transform do not have compact support in strict sense, the wavelet function satisfies the admissibility condition (unlike Morlet) and has two vanishing moments associated with it. The Mexican Har wavelet filter is depicted in Fig. 15.5(b). This function in its appearance looks like a ‘Mexican Hat’ and hence has been named likewise. The Mexican hat basis function is derived from Gaussian function and in fact is second order derivative of e

- t2 2

. The wavelet has mathematical form as follows: -1 2 t

y (t ) = (t 2 - 1)e 2

(15.34)

The Fourier transform of Eq. (15.34) will be 2

y (w ) = (iw )  (e

− t 2 /2

) = −w e 2

−w 2 2

(15.35)

From Eqs. (15.34) and (15.35) it can be observed that neither the wavelet function nor its Fourier representation has compact support. Good news, however is, they both decay rapidly outside finite width. If Eq. (15.35) is evaluated at w = 0, it becomes 0, which ensures that this wavelet basis follows admissibility condition. We encourage the readers to solve the zeroth and first moments and confirm that they both vanish for Mexican Hat Wavelet function.

459

Other Wavelet Families

Morlet Wavelet Designed 3 2 1 0 −1 −2 −3 −4

−3

−2

−1

Figure 15.6

15.6

0

1

2

3

4

Designed Morlet Wavelet

Meyer Filters

Yves Meyer Y. Meyer (born 19 July 1939) is French mathematician and scientist. He along with Mallat gave the framework of multiresolution analysis. The wavelet constructed by him are named after his as ‘Meyer Wavelet filters’. He is currently Professor Emeritus at the Ecole Normale Superieure de Cachan and has won many prestigious awards like 2010 Gauss award.

Yves Meyer was able to construct smooth orthogonal wavelet basis using the design strategy in Fourier domain.He was able to come with neat representation of Fourier transform ( fˆ (w ) ) of scaling function ( f(t ) ) as:  1  p  ˆ f (w ) =  cos  2  0  

2 if |w | ≤ p 3 2 4  3  | w | −1   if p ≤ |w | ≤ p x 3 3  4p  otherwise

(15.36)

460

Multiresolution and Multirate Signal Processing

Where x( × ) is a smooth function and  x (t ) =  

0, if t ≤ 0 1, if t ≥ 1

(15.37)

With another property x(t ) + x(1 - t ) = 1

(15.38)

In Chapters 2 to 6,we have understood that in orthogonal filters,y (×) can be calculated from f(×) . Therefore, Fourier representation of y will be, iw

y (w ) = e 2

æw ö

åf (w + 2p (2l + 1)) × f çè 2 ÷ø lÎZ

(15.39)

iw 2

æw ö = e [f (w + 2p ) + f (w - 2p )] × f ç ÷ è2ø Inverse of Eq. (15.39) gives us y (t ), as Eq. (15.39) is compactly supported. The filters are depicted in Fig. 15.7. Scaling Function (φ(.)) 1 0.5 0 −0.5

1 0.5 0 −0.5

10 20 30 40 Lowpass Analysis Filter (hk)

0

20

40

60

80

50

60

70 80 90 Highpass Analysis Filter (gk)

0.5 0 −0.5 −1 100 120 0 20 Wavelet Function (ψ(.))

40

60

80

100

120

1 0 −1

Figure 15.7

10

20

30

40

50 60 Meyer Wavelet

70

80

90

Meyer’s Wavelet. Analysis Low and High pass filters along with Scaling (f (.)) and wavelet (y (.)) functions

100

461

Other Wavelet Families

15.7

Battle-Lemarie Wavelets – Orthogonalization of the B-Splines

We have already seen the roof scaling function in Chapter 12, which is a special case of an important family of scaling functions, namely, the B-splines. In Chapter 12 we have mentioned that the roof function is a linear spline, which can be constructed as a self convolution of the gate function (spline of degree zero). In general, the Rth convolution ( pa  R  pa (t ) of the gate function pa (t ) is a B-spline of degree R + 1, and it is written as f R +1(a( R + 1), t ), -( R + 1)a £ t £ ( R + 1)a . It is identically zero outside this interval and provides, what we are seeking, a compact support. So, the roof function is f2 (2a, t ), -2a £ t £ 2a , and the gate function pa (t ) is f1 (a, t ), - a < t < a . The roof function is defined on the four subintervals: (-¥, -2a ],[ -2a,0],[0,2a ] , and [2 a, ¥) . We also note that at the boundary points t = -2a,0 , and 2a between these four subintervals, the first derivative has a jump discontinuity. These points are called the knots of this B-spline. It turned out that f R +1(a( R + 1), t ) is defined on R + 2 subintervals with continuous derivatives up to the order R - 1 at the knots. In the case of the roof function with R = 1 , only the zeroth derivative, i.e., the function itself is continuous. æ3 ö Another example of the quadratic B-spline could be f3 ç ,t ÷ , as the three times self convolution of è2 ø 1 the gate function or the haar scaling function p1 (t ) with a = , 2 2

3  f3  , t  2 

1  3 2  t +  , 2  2 3 2  −t , = 4 2 2   1  t − 3  , 1  t − 3  , 2  2 2  2  0,

3 1 ≤t≤− 2 2 1 1 − ≤t≤ 2 2

−

1 3 ≤t≤ 2 2 otherwise

æ3 ö This f3 ç ,t ÷ appears to be a smooth function, and as if it is a Gaussian like function. If we, however è2 ø 3 1 1 3 take its first order derivative we can easily see the sharp corners at the four knots t = - , - , , and . 2 2 2 2  3 t + 2 ,  df3  −2t , = dt  3 t − ,  2 0,

3 1 ≤t≤− 2 2 1 1 − ≤t≤ 2 2 1 3 ≤t≤ 2 2 otherwise −

462

Multiresolution and Multirate Signal Processing

Differentiating again (i.e. the second order derivative of the original function), we see the jump discontinuities of its second derivative at these four knots.  1,  d 2f3  −2, = dt 2  1,  0,

3 1 ≤t≤− 2 2 1 1 − ≤t≤ 2 2 1 3 ≤t≤ 2 2 otherwise −

3 1 1 3 æ3 ö Note the symmetry of f3 ç ,t ÷ around the origin with its knots at - , - , , and . 2 2 2 2 è2 ø As it is the case with the roof function, it is a usual practice that we translate the spline function such 1 æ3 ö that the knots fall at integer multiples. In this case we translate (shift) f3 ç ,t ÷ to the right by t0 = 2 2 è ø to have

1 −1 ≤ t ≤ 0  2 (t + 1),  3 1 1   (t − )2 , 0 ≤ t ≤ 1 3 f3  , t −  =  4 2 2 2  1 2  (t − 2) , 1 ≤ t ≤ 2 2  0, otherwise From the convolution theorem we get, 2

é lù sin ú ì æ 3 öü ê 2 ,  íf3 ç , t ÷ ý = ê ú l 2 øþ ê î è ú ë 2 û and with the translation property of the Fourier transform, we have  l sin   3   − il  2 Φ 3 (l ) =  f3  , t − 1  = e    2  l   2  2 

2

In Chapter 12 we showed that the roof function (B-spline of degree 1) satisfies the scaling equation with 1 1 1ö æ3 scaling coefficients h0 = , h1 = 1, h2 = . Here, we can do the same verification for f3 ç , t - ÷ with 2 2 2 2ø è

463

Other Wavelet Families

3 1 3 1 its four non-vanishing scaling coefficients h0 = , h1 = , h2 = and h3 = . We leave this part to read4 4 4 4 ers for further investigation and confirmations of the details mentioned above.

15.7.1  Battle-Lemarie Design Details In Chapter 12 we have shown that the B-splines make good scaling functions, with their important property of compact support. Yet, as we had illustrated for the roof function, they are not orthogonal with respect to their translation by integers. Simplest way to understand this is when we translate the function by factor of 1 and take the dot product with the original function, it does NOT produce zero. Thus, they can not comply with the usual multiresolution analysis (MRA) framework, where the orthogonality is at the core. The design question is can they be made orthogonal. This brings us to the (orthogonal) BattleLemarie scaling functions that are constructed using the B-splines. While such new scaling functions are orthogonal with respect to translations by integers (including the shift by 1), they on the other hand, lose on account of having a compact support. The final result of the method for modifying the B-spline is to become the Battle-Lemarie (orthogonal) scaling functions. This modification method relies heavily on the Fourier transform of FR+1 (w ) of the fR +1(t ) B-spline, and the sum of the squares of its translates by 2p k, å k | F R +1 (w + 2p k) |2 . The method starts by  (w ) as, modifying the F (w ) to F R+1

R+1

é ù F R +1 (w ) = FR +1 (w) ê2p å | FR +1 (w 2pk ) |2 ú k ë û

-

1 2

(15.40)

Then it is important to find the inverse Fourier transform of F R +1 (w) as f R +1(t ) , which is the desired orthogonal Battle-Lemarie scaling function. As an example from Ten lectures by Daubechies, with the Fourier transform F2 (w ) of the linear B-spline in Eq. (15.40), it is found that: 2p ∑ | F2 (w + 2p k) |2 = k

2 1 1 w + cosw = (1 + 2cos2 ) 3 3 3 2

(15.41)

That gives us,

F 2 (w ) = 3(2p )

−

1 2

4sin 2

l 2 1

(15.42)

l 2  l 2 1 + 2cos2  2  What remains is to find f2(t ) , the inverse Fourier transform of this F 2 (w ) . This works for having an orthogonal Battle-Lemarie scaling function. The next design quest is to find its scaling coefficients, more over their relation to coefficients of constructing the associated wavelet. The interested readers can divulge deeper into this and this certainly forms a very good topic to research on.

464

Multiresolution and Multirate Signal Processing

The filters are depicted in Fig. 15.8. Synthesis Scale Function

1 0.5 0 −0.5

0

100

200

300

400 500 600 Synthesis Wavelet Function

700

800

900

1000

0

100

200

300

400 500 600 Battle−Lemarie Filters

700

800

900

1000

1 0.5 0 −0.5 −1

Figure 15.8

15.8

Battle-Lemarie Wavelet. Synthesis Sacling (f (.)) and wavelet (y (.)) functions

Gabor Filters

Gabor wavelets were invented by Denis Gabor. They are well known for their ability to capture oriented information from the signal or image to be analyzed. This salient feature makes them more useful in analysis of oriented images like fingerprint images. 2D Gabor kernel can be looked upon as Gaussian kernel function modulated by sinusoidals (sin and cos waves). The formulae for Gabor filters in 2D sense is given as:

Gc [i, j ] = C1e

- ( i2 + j 2 ) 2s 2

Gs [i, j ] = C2 e

- ( i2 + j 2 ) 2s 2

cos(2p f (i cos q + j sin q ))

(15.43)

sin(2p f (i cos q + j sin q ))

(15.44)

Where, C1 and C2 are used for normalization. Variation in q helps an engineer in analyzing the oriented information in a particular direction.

15.9

Shannon Filters

Shannon wavelet is also called an ‘Sinc’ wavelet.

465

Other Wavelet Families

Its Fourier transform H(w ) is:   H(w ) =   

p 2 p 0, |w | > 2 2, |w | ≤

This is nothing else but an ideal low pass filter! This leads the solution to normalizing factor of e.g. if   C(w ) =   

p 2 p 0, |w |> 2

1, |w | ≤

(15.45)

2p

 and f (w ) =  

1, |w | ≤ p 0, |w |> p

(15.46)

Then, dilation equation f (2w ) = c(w ) × f (w ) gets satisfied. In time domain, y (t ) = ågk × y (2t - K )

(15.47)

K

If g0 = 1 and g2 K = 0 and

g2 K +1 = (-1)K

2 (2 K + 1)p

then, y (t ) = 2p ×

sin(p t ) pt

(15.48)

Since Eq. (15.45) is normalized, it‘s time domain representation will be h(t ) =

sin(p t) pt

This is sinc form of solution, Therefore, this filter is also called sinc wavelet. The filter coefficients will be (normalized), h0 =

1 2

, h2 n = 0, n ¹ 0

(15.49)

466

Multiresolution and Multirate Signal Processing

h2 n +1 =

(-1)n 2 (2n + 1)p

Wavelet function gets derived from corresponding high pass filter   G(w ) =    Therefore, filter coefficients will be, g0 =

1 2

p 2 p 0, |w | < 2 2, |w | ≥

(15.50)

, g2 n = 0, n ¹ 0

g2 n +1 =

(-1)n +1 2 (2n + 1)p

This gives us y (t ) = 2h(2t ) - h(t ) =

sin(2p t) - sin(p t) pt

(15.51)

The filters are depicted in Fig. 15.9. 1

Complex Shannon wavelet (Bandwidth = 1, Center Frequency = 1.5)

0.5 0 −0.5 −1 −20

−15

−10

−5

0 Real part

5

10

15

20

−15

−10

−5

0 Imaginary part

5

10

15

20

1 0.5 0 −0.5 −1 −20

Figure 15.9

Shannon wavelet. Analysis low and high pass filters along with Scaling (f (.)) and wavelet (y (.)) functions

467

Other Wavelet Families

15.10

Bi-orthogonal Filters

The design strategy could be as under. Haar filter is the only finite length, symmetric, orthogonal filter whose Fourier representation [ H(w ) ] satisfies zero derivative conditions at w = p , as was explained by Daubechies. Researchers across the globe have confirmed that having symmetric filters is advantageous in multiple ways for many real life applications. Another important characteristics of filters is orthogonality. It guarantees energy preservation and inverse of wavelet transform matrix. For orthogonal filter ‘h’ that gives orthogonal transform matrix WN , the implementation is straight forward. This is because WN is ‘unitary’ which guarantees that for all real valued WN , WN-1 = WNT . This simple transpose gives us the inverse,which is very crucial in inverse calculations as as ‘separable’ multidimensional calculations. For ‘non-orthogonal’ filters, however, one strategy to build the system we desire could be to ensure that inverse of transform matrix is transpose of another transformation matrix. ∼∼ hk,gk

hk,gk iff

WN

∼ −1 = W T WN N

∼ WN

then ∼ hk,kk is biorthogonal filter pair

Haar, though orthogonal, finite length and symmetric, has its own drawback the we have already brought out. Going beyond filter length of ‘2’,we shall give up ‘orthogonality’ as requirement coming as filter constraint. 5 Let us try constructing normalized tap: 3 Design problem: 1. 2. 3. 4. 5.

From symmetric hk and gk Construct wavelet transform WN When inverse WN-1 is calculated. WN-1 should also be a transform matrix! Find such g k and h k (symmetric)! N ´N 2 N Let ‘ GB ’ be ´N 2  B ’ be N ´ N Let ‘ G 2 N  B ’ be Let ‘ H ´N 2 Let ‘ H B ’ be

matrix from hk . matrix from gk. matrix from g k . matrix from h k.

468

Multiresolution and Multirate Signal Processing

Forward wavelet matrix can be written as: éH ù WB = ê B ú ë GB û For synthesis part, similarly, é ù  B = êH B ú W B ú êë G û keeping in mind,we want to create dual filters in semi-orthogonal sense, T

B = I WB × W N ´N  TB = é H B ù × é H T Now, WB × W ê G ú êë B ë Bû

(15.52)

 TB ù HB × G ú  TB ú GB × G û

éH × H  TB T B  ù GB ú = ê û êG × H  TB ë B

Here,  TB = G × G  TB = I HB × H B N ´N T B

(15.53)

T B

 =G ×H  =0 HB × G B N ´N

{

}

5 Now, hk = {h-2 , h-1 , h0 , h1 , h2 } and h k = h −1 , h0 , h1 in the spirit of tap. 3 \ Analysis low pass filter has 5 coefficients and Synthesis low pass filter has 3 coefficients as ‘5’ is bigger of them, N Let = 5, ∴ N = 10 2  h0 h  −2 ∴ HB =  0  0  h2 é�h0 ê ê0 � H B = êê 0 ê0 ê êë 0

h1 h−1 0 0 0

h2 h0 h−2 0 0

0 h1 h−1 0 0

�h1 �h -1

0 �h0

0 �h1

0

0

0

0

0

0

0 h2 h0 h−2 0

0 0 h1 h−1 0

0 0 h2 h0 h−2

0 0 0 h1 h−1

h−2 0 0 h2 h0

0

0

0

0

0

0 �h1

0

0

0

�h -1

0 �h0 0

�h -1

0 �h1

0

0

0 �h0 0

�h -1

0

0

0

0 �h0

h−1  0  0  0 h1 

�h -1 ù ú 0ú ú 0ú 0ú ú �h1 ú û

469

Other Wavelet Families

 TB = I Now, H B × H 5´ 5

 h0 h  −2 0  0  h2

h1 h−1 0 0 0

h2 h0 h−2 0 0

0 h1 h−1 0 0

0 h2 h0 h−2 0

0 0 h1 h−1 0

0 0 h2 h0 h−2

0 0 0 h1 h−1

h−2 0 0 h2 h0

 h 0   h 1  0 h−1   0 0   0 0 ⋅  0 0  h1   0  0 0  h −1

0 h −1

0

0

0

0

h 0 h 1

0

0

0 h −1 h 0

0

h 1

0

0

0 h −1 h 0

0

0

h 1

0

0

0

0

0

0

0

0  0  0 0  0  = I 5× 5 0  0  h −1  h 0   h 1 

1st row ´ 1st column,1st row ´ 2 nd column and 1st row ´ last column will give us three equations, h0 × h 0 + h1 × h 1 + h-1 × h -1 = 1 h × h 1 + h × h 0 = 0

(15.55)

h-2 × h 0 + h-1 × h 1 = 0

(15.56)

0

2

(15.54)

Equation (15.54) can be written as, 1

åh

k

× h k = 1

(15.57)

K = -1

Equation (15.55) and (15.56) can be written as, 1

åh

k -2m

× h k = 0, m = -1,1

K = -1

 TB = I Similarly, GB × G 5´ 5 Here, gk = {g−1 , g0 , g1 } and g k = {g−2 , g−1 , g0 , g1 , g2 } Therefore, analysis high pass filter has ‘3’ coefficients, synthesis high pass filter has ‘5’ coefficients N = 5, ∴ N = 10 2

(15.58)

470

Multiresolution and Multirate Signal Processing

é g0 ê0 ê GB = ê 0 ê ê0 ê0 ë é �g 0 ê� ê g -2 ê � GB = ê 0 ê 0 ê êë �g 2

0 g0 0 0 0

g1 g-1 0 0 0

0 g1 g-1 0 0

0 0 g0 0 0

0 0 0 g0 0

0 0 0 g1 g-1

0 0 0 0 g0

g-1 ù 0 úú 0 ú ú 0 ú g1 úû

0

0

0

�g -2

0 �g 1 �g

0 �g 2 �g

0

0

0

0

�g -2

0 �g 1 �g

0 �g 2 �g

0 0 g1 g-1 0

0 0 0 g0 0

0 0 0 g1 g-1

0 0 0 0 g0

0 0 g1 g-1 0

�g 1 �g

�g 2 �g

0 �g

0 �g

0

�g -2

�g -1

0

0

0

�g 0 �g

-1

0

0

1

0

2

-2

0

-1

0

-1

0

�g ù -1 ú 0 ú ú 0 ú 0 ú ú �g ú 1 û

 TB = I Now, GB × G 5´ 5 é g0 ê0 ê ê0 ê ê0 ê0 ë é g 0 ê ê g 1 ê ê g2 ê 0 ê ê 0 ê ê 0 ê ê 0 ê 0 ê ê g 2 ê êë g -1

g1 g-1 0 0 0 g -2 g

-1

g 0 g 1

g 2

0 g0 0 0 0

0 g1 g-1 0 0

0 0 g0 0 0

0

0

0 g

0

g -1 g

0 g -2 g

0

0

g 1 g

0

0

g 0 g

0

0

g 2

0

0

0

0

g 2 ù ú 0 ú ú 0 ú 0 ú ú 0 ú ú = I 5´5 0 ú ú g -2 ú g -1 ú ú g 0 ú ú g 1 úû

0

-2

-1

2

g-1 ù 0 úú 0 ú × ú 0 ú g1 úû

1

g0 × g 0 + g1 × g 1 + g- × g -1 = 1 g × g + g × g = 0

(15.59)

g0 × g 2 + g-1 × g 1 = 0

(15.61)

0

-2

1

-1

(15.60)

471

Other Wavelet Families

Therefore, equation (15.59) get written as: 1

åg

k

× g k = 1

(15.62)

K = -1

Equations (15.60) and (15.61) becomes 1

∑g

K = −1

K −2m

⋅ g k = 0, m = −1,1

(15.63)

 TB to be an identity matrix which demands h and h k to be Now, Design aspect demands H B × H k orthogonal. For two filters to be orthogonal, let us derive the sufficient & necessary condition with again an example of Haar wavelet. For Haar, if ì1 1 ü hk = {h0 , h1 } = 2 í , ý î2 2 þ ∴

jw 1 1  w H (w ) = 2  + ⋅ e jw  = 2 ⋅ e 2 ⋅ cos   2 2   2

w ∴ | H(w ) | = 2 ⋅ cos    (Note: Magnitude of e jw = 1 on interval of [ -p , p ])  2 We have already seen that when H ( z ) becomes H(-z), the frequency band gets shifted by p amount, which can be represented as: H(w + p ) w +p ∴ | H(w + p ) | = 2 ⋅ cos   2 

∴

Now, if we add squared magnitudes of H(w ) and H(w + p ), we get æw ö | H(w ) |2 = 2 × cos 2 ç ÷ è2ø

(15.64)

2

æ æ w + p öö 2æ w + p ö | H(w + p ) = ç 2 cos ç ÷ ÷ ÷ = 2 × cos ç è 2 ø è 2 øø è 2

æw ö \| H(w + p ) |= 2 × sin 2 ç ÷ è2ø

(15.65)

2w  2 w  | H (w ) 2 | + | H (w + p )2 | = 2 ⋅ cos   + 2 ⋅ sin    2  2

(15.66)

∴

Using Eqs. (15.64) and (15.65), ∴

472

Multiresolution and Multirate Signal Processing

This ‘2’ also comes as we are using normalized filters, in absence of normalization, the sum would be ‘1’. Important thing to note is, a constant suggest orthogonality. Using Eqs. (15.66) and (15.52), we can write  (w ) + H (w + p ) × H  (w + p ) = 2 H (w ) × H

(15.67)

 satisfy Eq. (15.67), we have, When H(w ) and (w)

åh

k

× h k = 1

(15.68)

K ÎZ

For m Î Z , m ¹ 0 ,

åh

K -2m

× h k = 0

(15.69)

K ÎZ

Similarly,  (w ) + G(w + p ) × G  (w + p ) = 2 G(w ) × G

(15.70)

 satisfy Eq. (15.67), we have, When H(w ) and (w)

åg

k

× g k = 1

(15.71)

K ÎZ

For m Î Z , m ¹ 0 ,

åg

K -2m

× g k = 0

(15.72)

K ÎZ

 TB = 0 To ensure H B × G  (w ) + H (w + p ) × G  (w + p ) = 0 H (w ) × G

(15.73)

for all m Î z ,

åh

K -2m

× gk = 0

(15.74)

K ÎZ

 TB = 0 To ensure, GB × H  (w ) + G(w + p ) × H  (w + p ) = 0 G(w ) × H

(15.75)

For all m Î Z

åg

K ÎZ

K - sm

× hk = 0

(15.76)

473

Other Wavelet Families

 Equations (15.67) to (15.76) give us definition for bi-orthogonal filter pairs. e.g. If H(w ) and H( w ) of  hk and h k respectively follow:  (w ) + H (w + p ) × H  (w + p ) = 2 H (w ) × H Then, hk and h k are called bi-orthogonal filter pair! From Eqs. (15.67), (15.70), (15.73) and (15.75)  (w + p ) [Analysis high pass connected with synthesis low pass] and G (w ) = ei ( nw + b ⋅ H  (w ) = ei ( nw + b ⋅ H (w + p ) G

[Synthesis high pass connected with analysis low pass]

Let’s confirm this by pluggin in  (w ) = e j ( nw ) + b × H  (w + p ) × e j ( nw + b ) × H (w + p ) G (w ) × G  (w + p ) × e j ( nw + b ) × H (w + p ) = e - j ( nw + b ) × H

(15.77)

 (w + p ) = H (w + p ) × H Since they are equal, together they shall produce ‘2’! By replacing ‘ w ’ with ‘ w + p ’ in Eq. (15.77)  (w + p ) = H  (w ) ⋅ H (w ) [ Astheseare`2p − periodic ′ functions.∴ w + 2p = w ] (15.78) G (w + p ) ⋅ G Adding Eqs. (15.77) and (15.78),  (w ) + G (w + p ) × G  (w + p ) = H (w ) × H  (w ) + H (w + p ) × H  (w + p ) = 2 G (w ) × G Now,  (w ) × G (w ) = H  (w ) × e j ( nw + b ) × H  (w + p ) H  (w ) × e - j ( nw + b ) × H  (w + p ) =H

(15.79)

Let’s replace ‘ w ’ by ‘ w + p ’ in Eq. (15.79) � (w + p ) × G (w + p ) = H � (w + p ) × e - j ( nw + np + b ) × H � (w + 2p ) H � (w + p ) × e - jnp × e - j ( nw + b ) × H � (w ) =H � (w + p ) × e - j ( nw + b ) × H � (w ) = ( -1) n × H � (w + p ) × e - j (nw + b) × �(w ) ��[`n` being odd, ( -1)n = -1] (15.80) = -H Adding Eqs. (15.79) and (15.80),  (w ) × G (w ) + H  (w + p ) × G (w + p ) = 0 H

474

Multiresolution and Multirate Signal Processing

It should be noted that, gk is connected with h k and g k is connected with hk ! b can be ‘0’ or p to keep filter real values and for n = 1, gk = (-1)K × h 1- K g = ( −1) K ⋅ h

1− K

k

Let’s confirm this for normalized

and

(15.81) (15.82)

5 tap. 3

ì 1 3 1 1ü h = {h-2 , h-1 , h0 , h1 , h2 } = í- ,1, , - , ý × 2 î 2 4 2 4þ

(15.83)

ì1 1 1 ü h = h -1 , h 0 , h 1 = í , , ý × 2 î4 2 4þ

(15.84)

{

}

Plugging Eqs. (15.82) and (15.83) in (15.80) and (15.81), we get,

{

g = g -1 , g 0 , g 1 , g 2 , g 3

}

= {h2 , h1 , -h0 , h-1 , h-2 }

(15.85)

ì 1 1 3 1ü = í- , - , - ,1, ý × 2 î 4 2 4 2þ Similarly, g = {g0 , g1 , g2 }

{

= h 1 , −h 0 , h −1

}

1 1 1 =  ,− , ⋅ 2 4 2 4

(15.86)

Let’s approve these filters: H (w ) =  (w ) = H

2 -2 jw 3 2 2 jw 2 2 jw ×e + 2 × e- jw + ×e + ×e 2 4 2 4 2 - jw 2 2 jw ×e + + ×e 4 2 4

Let’s evaluate Eqs. (15.87) and (15.88), for w = 0 and w = p 3 1 1ö æ 1 H(w ) |w =0 = 2 ç - + 1 + - + ÷ = 2 4 2 4ø è 2 H(w ) |w =p = 0

(15.87) (15.88)

475

Other Wavelet Families

Similarly,  (w ) | = 0  (w ) | = 2 and H H w =p w =0 Conjugating H(w ) we get, H (w ) =  (w ) ⋅ H w ∴H

2 -2 jw 2 2 jw 2 - jw 3 2 ×e ×e + + 2 × e jw ×e 4 2 4 2

 2 − jw 2 2 jw   2 −2 jw 2 − jw 3 2 2 2 jw  = ⋅e + + ⋅e  ×  ⋅e − ⋅e + + 2 ⋅ e jw − ⋅e  2 4 2 4 2  4   4  1 9 1 = ⋅ e −3 jw + 1 + ⋅ e jw − ⋅ e3 jw 8 8 4

(15.89)

Replacing ‘ w ’ by ‘ w + p ’ in Eq. (15.89)  (w + p ) × H (w + p ) H 1 9 1 = × e-3 j (w +p ) + 1 + × e j (w +p ) - × e3 j (w +p ) 8 8 4 1 -3 jw -3 jp 9 jw jp 1 3 jw 3 jp = ×e ×e +1+ ×e ×e - ×e ×e 8 8 4 Note, e3 jp = e-3 jp = e jp = e- jp = -1 (Euler’s identity) ∴

 (w + p ) × H (w + p ) = - 1 × e -3 jw + 1 - 9 × e jw + 1 × e3 jw \H 8 8 4

Now let’s create wavelet transormation matrix for bio-orthogonal é h0 êh ê -2 ê0 ê ê h2 w5 = ê ê 3 ê g0 ê0 ê ê0 ê ë g2

h1 h-1 0 0 g1 0 0 0

h2 h0 h-2 0 g2 g0 0 0

0 h1 h-1 0 0 g1 0 0

0 h2 h0 h-2 0 g2 g0 0

0 0 h1 h-1 0 0 g1 0

h-2 0 h2 h0 0 0 g2 g0

5 tap let’s call it w 5 3 3

h-1 0 0 h1 0 0 0 g1

ù ú ú ú ú ú ú ú ú ú ú ú ú û

(15.90)

476

Multiresolution and Multirate Signal Processing

Similarly,

é3 ê4 ê ê -1 ê2 ê ê0 ê ê1 ê ê4 = 2ê ê ê1 ê4 ê ê0 ê ê ê0 ê ê1 ê ë4

-1 2

 �h0  0  0 0  � =− w 5 3 � g0 0  0   g� 2

0

-1 2

0

0

-1 2

-

1 4 3 4 -

0

0

0

0

-1 2

1 4 1 4

1 4 3 4 -1 2

0

0

-1 2

0

0

0

-1 2

1 4 1 4

-

1 4 3 4 -1 2 -

0

0

-1 2

0

0

0

1 4 1 4

0

0

0

0

0

�h1 �h −1

0 �h0

0 �h1

0

0

0

0

�h −1

0 �h1

0

0

0 �h0

0 − �g

0 − �g

0 − �g

0 −

�h −11 −

g� −1

g� 0

0 g�

0

g� 1 g�

0 g�

g� 0

0

0

g� 1 g�

1 0

0

1

0 g�

3

2

1

3

−1

2

0

1

3

−1

0 �h0 − 0 0 �g 2 g� 0

ù 1ú ú 0ú ú ú 0ú ú -1 ú ú 2ú -ú ú ú 0ú ú 0ú ú ú 0ú ú -1 ú ú 2û �h −1   0   0  �h1   −   g� −1  0  g� 3   g� 1 

477

Other Wavelet Families

é 1 ê 2 ê ê 0 ê ê ê 0 ê ê ê 0 ê  = 2ê \w 5 ê 3 ê -1 / 2 ê ê ê 0 ê ê ê 0 ê ê ê 1/ 2 ë

1 4 1 4

0

0

0

0

0

1 2

1 4

0

0

0

0

0

0

1 4

0

0

0

0

-3 4 -1 2

-

-

1 4 1 2 -

0

0

0

-1 2

0

0

1 2 -3 4 -1 4

1 2 1 4 -

0

0

1

0

1 -1 2 -1 4

1 2 -3 4 -1 2

0 1 -3 4

1ù 4ú ú 0ú ú ú 0ú ú 1ú ú 4ú -ú ú -1ú 4ú ú 0ú ú 1ú ú 2ú ú 1ú û

it is clear, that  -1 = w  T w 5 5 3

3

an extension: using this approach different biorthogonal taps can be studied. 6  E.g let’s design tap tp produce w 6 and w 6 2 2 2 Here, hk = {h-2 , h-1 , h0 , h1 , h2 , h3 } nodeline of symmetry; (– 0.3,1)node

{

h k = h 0 , h 1

}

(1.9,0.5)[red]nodeline of symmetry; as the filters are symmetric, for h k , h 0 = h 1 for hk , h0 = h1 , h-1 = h2 , h-2 = h3 w = h + h .e jw = h + h .e jw H 0 1 0 0

(15.91)

478

Multiresolution and Multirate Signal Processing

and H (w ) = h-2 .e-2 jw + h-1 .e- jw + h0 + h1 .e jw + h2 .e2 jw + h3 .e3 jw = h3 .e-2 jw + h2 .e- jw + h1 + h1 .e jw + h2 .e2 jw + h3 .e3 jw

(15.92)

from Eq. (15.91)  (w ) | = 2h = 2 Now H w =0 0 \h0 = h1 =

∴

2 2

(Haar!)

for hk , let’s force low pass conditions on H(w ) . H(w ) |w =0 = 2 let’s plug w = 0 in (15.92)

∴

∴ 2h1 + 2h2 + 2h3 = 2

∴ h1 + h2 + h3 =

2 2

similarly, ∴ h3 − h2 + h1 − h1 + h2 − h3 = 0 -(holds good)

H (w ) |w =p = 0 using Eqs. (15.68) and (15.69)

h0 .h 0 + h1 .h 1 = 1 ∴

\

2 2 h0 + h1 = 1, but h0 = h1! 2 2 2h0 = 1

∴

\h0 = h1 =

1 2

from Eq. (15.69),

h0 .h -2 + h1 .h -1 = 0 ∴

z z ∴ h3 + h2 = 0, 2 2

∴ h2 = h3 ,

h2 = h3 = h-1 = h-2 = ´ (some real no.) ∴

and

2 2 ì ü \ hk = í x, - x, , , - x, x ý 2 2 î þ ïì 2 2 ïü h k = í , ý ïî 2 2 ïþ

479

Other Wavelet Families

Readers are encougraged to complete following steps: 1. calculate g k [1 ´ 6] from hk 2. calculate gk [1 ´ 2] from h k  from h , h k , h , h k 3. calculate w 6 and w 6 k k 2

15.11

2

Summary

We saw many interesting wavelet families in this chapter. We can now take a look at bird’s eye view and try and understand big picture by bringing out peculiarities of every family design. We can probably understand different wavelet families from the perspective of parameters like orthogonality, compact support, symmetry, smoothness, interpolation, regularity, rational coefficients, linearity of filters etc. ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●

Haar (a) Orthogonal (b) Linear phase filters0 (c) Compact support (d) Scaling function symmetric (e) Wavelet function anti-symmetric (f) One vanishing moment (admissible) (g) Only one to be linear, compactly supported, orthogonal and has symmetry. Daubechies (a) Most successful wavelet family (b) Orthogonal (c) Compactly supported (d) Only first member (Haar) is symmetric (e) Higher members are NOT symmetric (f) For filter length of 2N , N moments vanish (g) For filter length of 2N , support width is 2 N - 1 (h) Regularity of around 0.2N for large N They also have bi-orthogonal form Coifleets (a) Orthogonal (b) Compactly supported (c) Almost asymmetric (d) Scaling function exhibit vanishing moments too (e) In coifN filter, N is number of vanishing moments for both y as well as f (f) Typical filter length is 6N , for which y has 2N and f has 2 N - 1 vanishing moments Symlets (a) Orthogonal (b) The filters have adjusted symmetry (c) Compactly supported (d) For filter length of 2N, y has N vanishing moments (e) f is nearly linear phase (f) Also called Daubechies’ least asymmetric wavelets (g) For filter length of 2N, support width is 2 N - 1 (h) They also have bi-orthogonal form

480

Multiresolution and Multirate Signal Processing ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●

● ●● ● ● ● ● ●● ● ● ● ● ● ● ●

Morlet (a) Does not technically satisfy admissibility condition (b) Real-valued and complex-valued versions possible (c) Used in continuous analysis (d) The filters are symmetric (e) Though the effective support is from [– 4 4], the support width is infinite Mexican Hat (a) Continuous function (b) Satisfies admissibility condition (c) Two vanishing moments (d) Special case of DoG (Derivative of Gaussian) class of filters (e) No scaling function associated with this wavelet (f) These filters have infinite support width Meyer (a) Orthogonal (b) Scaling function is symmetric (c) Wavelet function is symmetric (d) Infinite support (decays faster than sinc though) (e) Both scaling and wavelet functions are defined in frequency domain (f) The wavelet is infinitely differentiable (g) The derivatives don’t have finite support but decay fast (h) Not compactly supported, yet approximation leads to good FIR type implementation (i) These filters are band-limited Battle-Lemarie (a) These are based on B-splines (b) Compact support (c) Filter coefficients are smooth (d) Orthogonality is enforced (e) Typically obtained by convolution of zero order splines (box functions) and then made orthogonal (f) Compactly supported splines are feasible only when orthogonality is relaxed to bi-orthogonality Gabor (a) Capable of capturing oriented information (b) Obtained from Gaussian kernel (c) More useful in 2D applications (d) Variants like log-Gabor are popular Shannon (a) Orthogonal (b) Scaling function is symmetric (c) Wavelet function is symmetric (d) Finitely infinite support (lacks compact support) (e) Slowly decaying IIR type filters (f) Non-causal and hence difficult to deploy (g) Theoretically infinite vanishing moments

481

Other Wavelet Families ●● ● ● ● ● ● ● ●

Bi-orthogonal (a) Bi-orthogonal is nature (b) Compactly supported (c) Symmetric filters possible (d) Arbitrary number of vanishing moments (e) Scaling function exists (f) Deployment as FIR filters (g) Perfect reconstruction possible

Exercises Exercise 15.1

Consider the signal f (t ) = 10t 2 (1 − t )2 cos 8p t,0 < t < 1 on the interval (0,1). We use the Wavelets Tool Box to compute the scaling levels a1 and a 2 over the interval (0,1), for the Coif 6 and the Daub 2 scaling functions coefficients. We will compare the maximum error on the interval (0,0.2) between the above Coif 6 approximation and the exact sample values. The same is done with Daub 2 computing the error between its corresponding scaling coefficients and the above estimation, it is found that for level 1, the maximum error for the Coif 6 is an order of magnitude less, and the same is for level 2. It is left for an exercise to check this accuracy at higher levels, where it is expected to decrease.

Exercise 15.2

As we did for the other scaling functions and wavelets, we call {hn } the scaling coefficients, let us call h1 (n) as the corresponding wavelets coefficients. Show that the Coif 6 scaling coefficient satisfy the following equality, 5

åh

2 n

=1

n =0

Give the interpretation of this result in terms of the energy of the Coif 6 wavelet. Show that Coif 6 scaling function satisfies the (new) relation, -2h0 - h1 + h3 + 2h4 + 3h5 = 0 while fD2 does not.

482

Multiresolution and Multirate Signal Processing

Exercise 15.3

For Example, consider the signal f (t ) = 20t 2 (1 − t )4 cos12p t,0 < t < 3 Compute the scaling functions coefficients for Coif 6 and Daub 2, and compare the maximum error as done in Example 15.1 at the levels 1, 2, 3 and 4. Use very small scales of l10 then l14 , and make your conclusion. For the same problem try Coif 12 and Daub 3.

Chapter

16

Advanced Topics

Introduction Fast wavelet transform The Daubechies Fast Wavelet Transform (FDWT) and its inverse (IFDWT) Low and high pass effects: Dual filters Numerical computations for the fast discrete wavelets transform and its inverse Multi-dimensional wavelets The Two-dimensional Haar wavelet transform Wavelets and self-similarity Wavelet transform: In perspective of moving from continuous to discrete Redundant discrete wavelet transform Regularity and convergence

16.1

Introduction

In this chapter we shall introduce advanced concepts in the field of ‘Wavelet Analysis’. First few sections will focus on the effective implementation strategies like ‘Fast Wavelet Transform (FWT)’, 2 dimensional and going beyond M dimensional wavelet transform. We shall also discuss few interesting concepts like non-separable basis, self similar structures, regularity etc.

16.2

Fast Wavelet Transform

To bring out the Fast Wavelet Transform structure we shall need the multiresolution framework that we introduced in chapter 2 and have used it throughout this text. As we had discussed already in Chapters 2 to 6, we shall use a values to denote approximations emerging out of low pass filtering and b values to denote the details emerging out of high pass filtering. Recollect this framework: Let a function be, 1 fj(x) Î Vj , scale = j . 2 j To span these space V j the basis function would be 2 2 f (2 j x − k )k j

Here k is the translational parameter and 2 2 is the normalizing factor to convert orthogonal basis into an orthonormal basis.

484

Multiresolution and Multirate Signal Processing

Thus we can write, j

f j ( x ) = ∑(a j ,k 2 2 f (2 j x − k )) k

Alpha can be calculated by as, j

+∞

a j ,k = ∫ f j ( x )2 2 f (2 j x − k )dx −∞

Similarly for W subspaces, g j ( x ) ∈Wj , scale =

1 2j

The basis would now be, j

2 2 y (2 j x − k )k Also we can write, j

g j ( x ) = ∑(b j ,k 2 2 y (2 j x − k )) k

j

+∞

b j ,k = ∫ g j ( x )2 2 y (2 j x − k )dx −∞

b values here would give us the details which are required to move from one subspace to another. n −1  Let’s start with 2 n samples s = {s0 , s1 , s2 ,.....s n } = { fk }2k =0 and with the necessary periodic or 2 −1

aperiodic extensions (padding) we will have 2 ⋅2 = 2 n +1 samples to represent the function under consideration. The Fast Wavelet Transform (FWT) uses the coefficients {a k } in approximating the signal f (t ) , n

f (t ) =

2⋅2n −1

∑ a f (t − k ) k

(16.1)

k =0

n

2 −1 Now, {a k }2k⋅=0 will be calculated as,

a k = ∑ fif (i − k )

(16.2)

k

Equation (16.2) is the discrete version of calculating a values from framework given in Chapter 2. The translations in f(k − j ) may extend the span and scope of scaling functions beyond the length of the samples sequence. The extension of the sequence can be done: 1. artificially by adding zeros to its both ends (or in one end) 2. extending the sequence periodically, or as a mirror image

485

Advanced Topics

The extrapolation of some order leads to spline like solution. n+1

These 2 ⋅2 n sample coefficients (after extension) {a k }2k =0 −1 can be looked upon as the output of the first low pass filter in the filter bank (corresponding to the scaling function). Let the low pass function is represented with following system function: 1, − a ≤ w ≤ a H (w ) =  0, otherwise.

(16.3)

Now, let’s try and understand the periodicity and orthogonal addition across the scales to bring out the ‘twiddle’ like factor to create the transform matrix. The approximated values {a k }, corresponding to scaling functions {f (t − k )} with scale l0 = 1, is in multiresolution sense an input to two parallel low pass and high pass filters at the larger scale l−1 = 2. This we have already seen in 2-band filter bank structures and we extended the philosophy to M-band filter bank structures. These correspond to the slower moving parts captured by the scaling function and the fast moving parts captured by the wavelet function, respectively in the lower sub-space. From the MRA (Multiresolution Analysis) axioms from Chapters 2 and 3, after down sampling, these parts add up to the same approximated signal: f (t ) =

2n −1

n

t  2 −1 t  f − + ∑ bk ,n −1y  − 1 − k  a k ∑  k , n −1  2  k =0 2  k =0

(16.4)

The instance of n − 1 samples in a k ,n −1 , b k ,n −1 indicates exactly twice the scale in a k ,n . The sampling rate plays vital role in multirate systems and it should be observed that the total samples of the scaling and wavelets series is 2 n + 2 n = 2 ⋅ 2 n = 2 n +1 , which is for same as those for the scaling coeffin+1

cients {a k }2k =0 −1 of (16.2). This is in line with transition while moving from V j to V j −1 . Here the number of samples should be double as input to two parallel filters with input of 2 n+1 samples produce an output of 2 n+1 each, a total of 2 n+ 2 . To bring this number to the original inputting number 2 n+1 , we use down sampler on these two outputs, where usually every odd indexed outputted sample is thrown away for n down-sampling with factor of 2. In simpler words if we have n samples in V j , then we get samples 2 n at the output of two parallel filters, and low pass filter produces samples of V j −1 subspace and high 2 n pass filter produces samples of Wj −1 subspace. The normalized, cyclic and periodic version of the 2 low pass filter modified from Eq. (16.3) will be, 1, −p ≤ w ≤ p H(w ) =  0, otherwise.

(16.5)

This low pass filter blocks all higher frequencies outside its band −p ≤ w ≤ p . In contrast to this, a high(band) pass filter, passes higher frequencies than those of the low pass filter. Mathematically it can be encoded as:

486

Multiresolution and Multirate Signal Processing

0, 1,  0, G(w ) =  1, 0,  

−∞ ≤ w < −2p −2p ≤ w ≤ −p −p ≤ w < p p ≤ w < 2p 2p < w < ∞.

(16.6)

H(w ) and G(w ) are shown in Figs 16.1 and 16.2 respectively. H(w)

w −3p

−2p

−p

Figure 16.1

0

p

2p

3p

Low pass filter H (w ) G(w)

w −3p

−2p

−p

Figure 16.2

0

p

2p

3p

High pass filter G (w )

These two filters truly complement each other on the frequency band ( −2p ,2p ) . If we look at only positive frequencies, low pass filter H(w ) passes low frequencies from (0, p ) and high pass filter G(w ) passes higher frequencies from (p ,2p ). Thus the two filters are dual of each other and can together form a complete band structure.

16.3

The Daubechies Fast Wavelet Transform (FDWT) and its Inverse (IFDWT)

In this book, we have used Haar filters multiple times to bring out the conceptual parts in crisp manner. The Fast Discrete Wavelet Transform (FDWT) has its trick in appropriate creation of the transform matrix using the filter coefficients. For the FDWT illustration, in lieu of Haar in this chapter we choose Daubechies scaling func1+ 3 3+ 3 3− 3 1− 3 tion fD2 with its four coefficients h0 = , h1 = , h2 = , h3 = , and its 4 2 4 2 4 2 4 2

487

Advanced Topics

associated wavelet y D2 with its coefficients h0′ = h3 = h3′ = −h0 = −

1+ 3

1− 3 4 2

, h1′ = −h2 = −

3− 3 4 2

, h2′ = h1 =

3+ 3 4 2

,

, which are the building blocks of the associated scaling functions fD2 . Readers can 4 2 note that the presence of 2 in the denominator of above eight coefficients is cumbersome. We shall use the adjusted coefficients as: p0 = 2h0 =

1+ 3 3+ 3 3− 3 , p1 = 2h1 = , p2 = 2h2 = , and 4 4 4

1− 3 . We have already shown in chapter 6 that h0 , h1 , h2 , h3 correspond to the normalized 4 scaling functions of { 2f (2t − k )} through iterative procedure in the scaling dilation equation, p3 = 2h3 =

f (t ) = 2 ∑hk (f (2t − k ))

(16.7)

k

The modified coefficients p0 , p1 , p2 and p3 correspond to the use of the not normalized scaling functions f(2t − k )} in the same scaling dilation equation, f (t ) = ∑ pk (f (2t − k ))

(16.8)

k

So, in order not to avoid nuisance of 2 in the computations for the fast Daubechies wavelet transform, we shall adopt the notation of { pk } as given in equation (16.8), however for our own convenience we will keep calling them {hk } with understanding that the f(2t − k ) used are not normalized. We stay with the use of {hk } because the nomenclature symbolizes the filter structure, such as {h0 , h1 , h2 , h3} and {h3 , −h2 , h1 , −h0 } , for the Daubechies 2 low pass and high pass filters, respectively. 1 The output produced by the coefficients {a k ,n } of the first low pass filter at the scale ln = n gets to 2 those {a k ,n −1 , b k ,n −1} of the next sub-level low pass and high pass filters outputs at the scale ln−1 . We have 2 n+1 samples for coefficients {a k ,n } and the total samples together of the {a k ,n −1 , b k ,n −1} coefficients, after downsampling, will also be 2 n+1 . Let us remember that for the averaging or low pass process with 2n −1

a k ,0 =

∑ f (i)f (i − k ) i =0

We had extended the samples to have them match the given values of the scaling functions. We will assume here a periodic extension. So, if we start with four samples, for example, we have a period of 4, so that f4 = f0 , f5 = f1 , f6 = f2 , and f7 = f3 . (In case of Haar we would have cared for periodicity of only 2!) The linear equation takes us from {a k ,n } to {a k ,n −1 , b k ,n .1} thus creating a matrix system where input is column vector of {a k ,n } and output results into alternate columns of {a k ,n-1} and {b k , n-1}. This ensures the matrix W as square transformation matrix is sparse. From Chapter 6, we can now collect important properties of scaling coefficient of Daub-2. h02 + h12 + h22 + h32 = 2

(16.9)

h2 h0 + h3 h1 = 0

(16.10)

488

Multiresolution and Multirate Signal Processing

The sparse nature of Ω is captured via how, the inverse matrix Ω −1 is related to Ω as follows: 1 Ω −1 = ΩT 2

(16.11)

This inverse matrix will be needed for the reconstruction (synthesis) process of the signal as we construct from {a k ,n −1 , b k ,n −1} to {a k ,n } . This is also in compliance with the fact that transform matrix should be a Unitary matrix and hence for real valued matrix A the condition that needs to be fulfilled thematically is A−1 = AT . Note that had we used the coefficients from Eq. (16.9) with a normalized scaling function in their 1 scaling dilation equation. The summation in Eq. (16.9) becomes 1 because of the in the denomina2 tor of our usual expressions for these coefficients. In the next few examples we shall demonstrate that ΩΩT = I . This will be the foundation for the fast wavelet constructs through the matrix calculations. Example 16.3.1 — Decomposition (analysis) matrix Ω and its reconstruction (synthesis) matrix W -1 .

The transformation matrix of the scaling functions kernel associated with scale l0 to those of the scaling functions-wavelets associated with scale l−1 is given as below:  h0 h  3 0 0 Ω= 0  0 0  0

h1 −h2 0 0 0 0 0 0

h2 h1 h0 h3 0 0 0 0

h3 −h0 h1 −h2 0 0 0 0

0 0 h2 h1 h0 h3 0 0

0 0 h3 −h0 h1 −h2 0 0

0 0 0 0 h2 h1 h0 h3

0  0   0  0  . h3   −h0  h1   −h2 

0 0 h0 h1 h2 h3 0 0

0 0 h3 −h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

0 0 0 0 0 0 h0 h1

     .    h3   −h2 

(16.12)

The transpose ΩT of this matrix is  h0 h  1  h2 h ΩT =  3 0  0 0  0

h3 −h2 h1 −h0 0 0 0 0

0 0 0 0 0

(16.13)

489

Advanced Topics

Now we shall calculate,  h0 h  3 0 0 ΩΩT =  0  0 0  0

h1 −h2 0 0 0 0 0 0

h2 h1 h0 h3 0 0 0 0

h3 −h0 h1 −h2 0 0 0 0

0 0 h2 h1 h0 h3 0 0

0 0 h3 −h0 h1 −h2 0 0

0 0 0 0 h2 h1 h0 h3

0  0   0  0   h3   −h0  h1   −h2 

 h0 h  1  h2 h × 3 0  0 0  0

h3 −h2 h1 −h0 0 0 0 0

0 0 h0 h1 h2 h3 0 0

0 0 h3 −h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

0 0 0 0 0 0 h0 h1

         h3   −h2 

 h02 + h12 + h22 + h32 h h − h h + h h − h h 0 3  0 3 1 2 1 2  h0 h2 + h1h3 = h3 h2 − h3 h2   0   .. 1 0  = 20  ..  0

0 1 0 .. ..

0 0 1 .. ..

0 0 0 0 0

h0 h3 − h1h2 + h1h2 − h0 h3 h02 + h12 + h22 + h32 h0 h1 − h0 h1 h1h3 + h0 h2 0 ..

.... h0 h2 + h1h3 h0 h1 − h0 h1 ....  2 2 2 2 h0 + h1 + h2 + h3 ...   h0 h3 − h1h2 + h1h2 − h0 h3 ...  0 ..   .. .. 

.. ..  .. = 2 I , ..  1

Thus, ΩΩT = 2 I

(16.14)

490

Multiresolution and Multirate Signal Processing

ΩΩT =I 2

(16.15)

1 Ω −1 = ΩT 2

(16.16)

Hence,

It is understood again that the constant of 2 in Eq. (16.14) is the result of using 1+ 3 3+ 3 3− 3 1− 3 1 h0 = , h1 = , h2 = , h3 = instead of normalized coeffs with an added factor. 4 4 4 4 2 This provides the basic foundation to carry out the fast calculations. Equations (16.14) to (16.16) confirm the unitary nature of transformation matrix which guarantees energy compaction and preservation. To better illustrate the fast wavelet transform, we concentrate on the Daubechies, using Daubechies 2 scaling functions and wavelets. We are considering the Daubechies scaling function fD 2 (t ) and the 1 wavelet y D2 with the four scaling coefficients h0 , h1 , h2 , and h3 with the factor of our usual nota2 tion missing to simplify the illustration. Example 16.3.2 — Decomposition and reconstruction with FDWT.

The FDWT emerges out of the scaling dilation equation, f (t ) = h0f (2t ) + h1f (2t − 1) + h2f (2t − 2) + h3f (2t − 3)

(16.17)

and constructing the associated wavelet from the scaling functions using wavelet dilation equation, y (t ) = −h0f (2t − 1) + h1f (2t ) − h2f (2t + 1) + h3f (2t + 2)

(16.18)

In the decomposition (analysis) process we move towards the coarser scales. For example, in  t  t the first parallel low pass and high pass filters, we move from f (t ) to f   and y   and their  2  2 t translations. This is obtained from Eqs (16.17) and (16.18) by changing t to , 2  t f   = h0f (t ) + h1f (t − 1) + h2f (t − 2) + h3f (t − 3) (16.19)  2  t y   = −h0f (t − 1) + h1f (t ) − h2f (t + 1) + h3f (t + 2)  2

(16.20)

The rest of the scaling functions translations are manifested towards wavelet series. Here  t − 2 t   t y  − 1 = y  is obtained from y   above by changing t to t − 2,  2  2   2  t − 2 t  = −h0f (t − 2 − 1) + h1f (t − 2) y  − 1 = y   2  2 

491

Advanced Topics

− h2f (t − 2 + 1) + h3f (t − 2 + 2) = −h0f (t − 3) + h1f (t − 2) − h2f (t − 1) + h3f (t ).  t By virtue of placing the terms of this series in the same order as for f   in (16.17) with the  2 term f(t ) being the first, we have t  y  − 1 = h3f (t ) − h2f (t − 1) + h1f (t − 2) − h0f (t − 3) 2 

(16.21)

Similarly,  t − 2 t  For f  − 1 = f  , we have  2  2   t − 2 t  f  − 1 = f  = h0f (t − 2) + h1f (t − 2 − 1)  2  2  + h2f (t − 2 − 2) + h3f (t − 2 − 3) = h0f (t − 2) + h1f (t − 3) + h2f (t − 4) + h3f (t − 5) t  t  We can again write f  − 1 in the same way and continue to f  − k  , 2  2  Where k denotes translation t  f  − k  = h0f (t − 2k ) + h1f (t − 2k − 1) + h2f (t − 2k − 2) + h3f (t − 2k − 3) 2 

(16.22)

Similarly,  t − 4 t  = h3f (t − 2) − h2f (t − 3) + h1f (t − 4) − h0f (t − 5) y  − 1 − 1 = y   2  2 

(16.23)

and  t − 2k − 2  t  y  − k − 1 = y   = h3f (t − 2 k ) − h2f (t − 2 k − 1)  2  2

+ h1f (t − 2k − 2) − h0f (t − 2k − 3)  t  What we have in Eqs (16.22) and (16.23) is the scaling functions f  − k   and the wavelets   2  t 1  y − 1 − k respectively with the coarser scale l = 2 (projection in v , hence scale = −1 = 2 )     −1 -1  2  2 as a linear combination of the scaling functions {f (t − j )} only at the smaller scale l0 = 1 (projections in v0, hence scale =

1 = 1) . 20

492

Multiresolution and Multirate Signal Processing

The approximate input signal, f (t ) = ∑a k ,nf (t − k )

(16.24)

k

is also at the finer scale of 1, where we associate the coefficients {a k ,n } with this scale. In comparison, the coefficients a k ,n −1 and b k ,n −1 of the low and high pass filters in Eq. (16.4) are associated with the t  coarser scale of 2 via the two series expansions in Eqs (16.22) and (16.23) in terms of y  − 1 − k  . 2  The crux of the Daubechies 2 fast wavelet transform starts with concentrating on Eqs (16.22) and (16.23). As already explained we see that we are expressing the bases at the scale 2 in terms of the scaling functions {f (t − j )} at the scale 1. The scaling coefficients a k ,n were obtained as a linear combination of {f (t − j )} in Eq. (16.2). We are after the coefficients {a k ,n−1} and {b k ,n−1}, where their linear combination is used, respectively to represent. So, we look at Eqs (16.19) and (16.20) for giving us the transformation of the bases {f (t − j )} . Finding this transformation will lead to transforming the coefficients {a k ,n } to {a k ,n −1 , b k ,n −1} . Equations (16.19) and (16.20) can be represented in a compact form as a matrix equation:

  t   f  2      ...   ...     

 h0 h  3 0 0 = 0  0  ...   ...

h1 −h2 0 0 0 0 ... ...

h2 h1 h0 h3 0 0 ... ...

h3 −h0 h1 −h2 0 0 ... ...

0 0 h2 h1 h0 h3 ... ...

0 0 h3 −h0 h1 −h2 ... ...

... ... ... ... h2 h1 ... ...

 f (t )   f (t − 1)   f − ( 2) t      f (t − 2)     f (t − 3)   f (t − 4)  ...    f − ( t 5)  ...   f (t − 6) ...    f (t − 7) ...    ...   ... 

(16.25)

 t For example, from the first row we get f   = h0f (t ) + h1f (t − 1) + h2f (t − 2) + h3f (t − 3) and  2 from the second row we get y = h3f (t ) − h2f (t − 1) + h1f (t − 2) − h0f (t − 3) and so forth. Stick to use T     t  Ω for the Daubechies coefficients square matrix with f = [f (t ), f (t − 1),....]T , w = f   ,y .... as   2 

493

Advanced Topics

   two column matrices, we may write the above matrix equation as w = Ωf . Since the basis f are used    for coefficients an = [a 0,n , a1,n ,...]T and the basis w for dn −1 = [a 0,n −1 , b0,n −1 , a1,n −1 ,.....]T . We can use the   above transformation to have dn −1 = Ωan , i.e., Ω will transform the coefficients {a 0,n , a1,n ,....} at the scale ln to {a 0,n −1 , b0,n −1 , a1,n −1 , b1,n −1 .....} at the lower scale ln−1 . This transformation is for the decomposition (analysis) process. What we also need is a transforma  tion from dn−1 back to an for the reconstruction (synthesis) process, i.e., an inverse transform or the inverse of the matrix Ω . We have already derived following properties of the Daubechies scaling coefficients, h02 + h12 + h22 + h32 = 1

(16.26)

h2 h0 + h3 h1 = 0

(16.27)

We have also brought out that the inverse matrix Ω −1 is related in a very simple way to Ω as 1 Ω −1 = ΩT , i.e., as half its transpose. Intuitively it makes sense as the transformation matric is unitary 2 nature and factor of 1/2 suggests normalization of 2 while moving from some scale into lower scale ln-1 and vice versa. Analytically, the above coefficients relations can be verified easily by a simple substitution. With T

   t  t   the existence of this inverse matrix, we can transform the basis w = f   ,y  − 1 ,.... back to   2  2       1  f = [f (t ), f (t − 1),....]T , f = Ω −1w = ΩT w . In terms of the coefficients an = [a 0,n , a1,n ,...]T and 2     dn −1 = [a 0,n , b0,n −1 , a1,n −1 , b1,n −1 ,....] , we use the matrix Ω to go from an to dn −1 = Ωan,  a 0,n −1   b  0,n −1   a1,n −1   b  1,n −1    ...   dn −1 =   = Ωan ...    ...     ...   ...     ... 

(16.28)

494

Multiresolution and Multirate Signal Processing

 h0 h  3 0 0  dn−1 =  0  0  ...   ...

h1 −h2 0 0 0 0 ... ...

h2 h1 h0 h3 0 0 ... ...

h3 −h0 h1 −h2 0 0 ... ...

0 0 h2 h1 h0 h3 ... ...

0 0 h3 −h0 h1 −h2 ... ...

... ... ... ... h2 h1 ... ...

 a 0,n  a  1, n   a   2,n    a 3,n      ...    ...  ,  ...    ...  ...   ...   ...   ...  ...    ...   ... 

(16.29)

     and to go back from dn−1 to an via an = Ω −1dn −1 = ΩT dn −1 ,  a 0,n  a   1,n   a 2,n  a    1  an =  3,n  = Ω −1dn −1 = ΩT dn −1 2  ...     ...   ...     ...   h0 h  1  h2  1 h  an =  3 2 0  0 0  0

h3 −h2 h1 −h0 0 0 0 0

0 0 h0 h1 h2 h3 0 0

0 0 h3 −h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

(16.30)

0 0 0 0 0 0 h0 h1

(16.31)

1 factor in Eq. (16.31) 2 2 becomes unity, i.e. 1. With this understanding of normalization factor we will not carry the factor of 1 in Eq. (16.31). Let us recall that the string of coefficients in the above matrices [h0 , h1 , h2 , h3 ] and 2 If we refer to our nomenclature of h2 , h1 , h0 , h3 with the

1

 a 0,n −1  0   b0,n −1    0   a1,n −1   0   b1,n −1    0   ...   . 0   ...      ...  h3   ...   −h2   ...     ...  factor, the

495

Advanced Topics

[h3 , −h2 , h1 , −h0 ] are those associated with the Daubechies fD2 and y D2, respectively. In the filters implementation of this wavelet transform, we refer to them as the low and highpass filters, respectively.  What we should note here is that in the decomposition (matrix) equation dn −1 = Ωan of Eq. (16.29) we have the two full strings of the above coefficients in the first and second rows of Ω , followed by  their translations to the right by two positions, and so on. So,  the multiplication of Ωan proceeds as  −1 expected. However, for the inverse transformation an = Ω dn −1 in Eq. (16.31), we first, and expect as we are doing construction with Ω −1 versus decomposition with Ω , see that we have the two strings of these coefficients in different order in their elements from those of the direct transformation. Second, they start as complete set of two filters in the third and fourth rows, then shifted by two instances to the right, and so on. The serious, apparent problem here is with the first two rows of Ω −1 , where the first two elements of the above two strings of coefficients are missing. The resolution of this problem, fortunately, is achieved by the periodicity of the coefficients, as we shall attend to the next. Let us assume that we have the eight coefficients {a 0,0 , b0,0 , a1,0 , b1,0 , a 2,0 , b2,0 , a 3,0 , b3,0 } in Eq. (16.31), and we extend them periodically with period 8. Then, we see in the above matrix Eq. (16.31) the first complete pair of filters {h2 , h1 , h0 , h3} and {h3 , −h0 , h1 , −h2 } use the first four elements a 0,0 , b0,0 , a1,0 , b1,0 in their matrix multiplication. The second full pair also uses a1,0 , b1,0 , a 2,0 , b2,0 . The third full pair uses a 2,0 , b 2,0 ,a 3,0 , b 3,0 and so does the fourth full pair. Then we see in the matrix multiplication of Eq. (16.31) that the inner product of the first complete pair of filters {h2 , h1 , h0 , h3} and {h3 , −h0 , h1 , −h2 } with the coefficients {a 0,n −1 , b0,n −1 , a1,n −1 , b1,n −1} give following. We shall also make a quick change for the simplicity of the notation. Let a = a and b = b and then, a2,n = h2 a0,n −1 + h1b0,n −1 + h0 a1,n −1 + h3 b1,n −1 , a3,n = h3 a0,n −1 − h0 b0,n −1 + h1a1,n −1 − h2 b1,n −1 . In the same manner we get a4,n and a5,n from the inner product of the two complete filters, shifted by two positions, and the coefficients {a1,n −1 , b1,n −1 , a2,n −1 , b2,n −1 , c2,n −1}, a4,n = h2 a1,n −1 + h1b1,n −1 + h0 a2,n −1 + h3 b2,n −1 , a5,n = h3 a1,n −1 − h0 b1,n −1 + h1a2,n −1 − h2 b2,n −1 , a6 and a7 are obtained, similarly, after shifting filters coefficients by two positions to the right a6,n = h2 a2,n −1 + h1b2,n −1 + h0 a3,n −1 + h3 b3,n −1 , a7,n = h3 a2,n −1 − h0 b2,n −1 + h1a3,n −1 − h2 b3,n −1 .  What is left is a0,n and a1,n, in the first two rows of an , because of the incomplete filters in the first two rows of the inverse matrix Ω −1 in Eq. (16.31). Thus, we are left with finding a0,n and a1,n to complete the first period for the eight numbers sequence {a0,n , a1,n , a2,n , a3,n , a4,n , a5,n , a6,n , a7,n } . The next period uses, {a0,n −1 , b0,n −1 , a1,n −1 , b1,n −1 , a2,n −1 , b2,n −1 , a3,n −1 , b3,n −1} , and that leads us to solution.  If we extend the size of the matrix from 8 × 8 to 10 × 12 and the column dn−1 to 12 × 1, we can compute the first two elements a8,n and a9,n of the second period.

496

Multiresolution and Multirate Signal Processing

a8,n = h2 a3,n −1 + h1b3,n −1 + h0 a0,n −1 + h3 b0,n −1 , a9,n = h3 a3,n −1 − h0 b3,n −1 + h1a0,n −1 − h2 b0,n −1 . But, by the periodicity with period 8, a8,n = a0,n and a9,n = a1,n . So a0,n = h2 a3,n −1 + h1b3,n −1 + h0 a0,n −1 + h3 b0,n −1 , a1,n = h3 a3,n −1 − h0 b3,n −1 + h1a0,n −1 − h2 b0,n −1 . This is also equivalent to filling the first and second row of the sqaure matrix by the first halves of their filters, {h2 , h1} and {h3 , −h0 } , at the far end of theirs first two rows, respectively, which we illustrate here for period 8 = (2) 2 n , n = 2 :  a0,2   h0 a  h  1,2   1  a2,2   h2 a  h  3,2  =  3  a4,2   0     a5,2   0  a6,2   0     a7,2   0

h3 −h2 h1 −h0 0 0 0 0

0 0 h0 h1 h2 h3 0 0

0 0 h3 −h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

h2 h3 0 0 0 0 h0 h1

h1   a0,1  −h0   b0,1    0   a1,1  0   b1,1   , 0   a2,1   b    2,1  h3   a3,1    −h2   b3,1 

(16.32)

a0,2 = h0 a0,1 + h3 b0,1 + h2 a3,1 + h1b3,1 , a1,2 = h1a0,1 − h2 b0,1 + h3 a3,1 − h0 b3,1 . This is for period 8 = (2)2 2 , n = 2 . For period 4 = (2)21 , n = 1, we generate a period 8 from extending the union of {a0,0 , a1,0 } and {b0,0 , b1,0 } periodically with period 4 as {a1,0 , b1,0 , a0,0 , b0,0 , a1,0 , b1,0 , a0,0 , b0,0 }. Here we appeal to a4,n and a5,n , which are the same as a0,n and a1,n , a0,n = h2 a3,n −1 + h1b3,n −1 + h0 a0,n −1 + h3 b0,n −1 , a1,n = h3 a3,n −1 − h0 b3,n −1 + h1a0,n −1 − h2 b0,n −1 . This is obtained, in a similar way for the modified inverse matrix,  a0,1   h0 a  h  1,1  =  1  a2,1   h2 a  h  3,1   3

h3 −h2 h1 −h h0

h2 h3 h0 h1

h1   a0,1  −h0   b0,1   . h3   a1,1  −h2   b1,1 

(16.33)

497

Advanced Topics

For example, a0,1 = h0 a0,1 + h1b0,1 + h2 a1,1 + h1b1,1 , a1,1 = h1a0,1 − h2 b0,1 + h3 a1,1 − h0 b1,1 , This also means raising the last two elements {a1,1 , b1,1} to the top of the column had we used the usual order of the first filter as {h2 , h1 , h0, , h3} and the second filter as {h3 , −h0 , h1 , −h2 } . Readers can note and appreciate that these filters are dual of each other, which we will explore further in next section. For period N = 16 , as another example, we look for the 18 × 20 matrix, using the periodicity of the coefficients with period 16, where {h2 , h1} and {h3 , −h0 } appear after 12 = N − 4 positions for N = 16 of translations for the preceding rows. Thus, in the first modified two rows we see the first halves of the top two rows in the matrix appear after 12 positions of translations for the preceding rows. Hence, we must have {h2 , h1} and {h3 , −h1} separated from their respective second halves at the beginning of the first two rows by 12 zeros. Equivalently, we need only to place them at the end of the first two rows, respectively. Of course, in an application such as signal (or spikes) detection, we shall need high reso1 1 lution. So, period 16 is very modest, and we may have to go to a resolution of scale l j = 8 = , 256 2 which needs 256 coefficients. However, a simple computer program with the above clear instructions should do it. Another form for the inverse matrix Another equivalent way to modify Eq. (16.31), which gives the same result, is to fill the first two rows as {h2 , h1 , h0 , h3} and {h3 , −h0 , h1 , −h2 } , move a3,n −1 and b3,n −1 from the bottom of their column to the top, but shift the remaining second, third, and so on, pairs of the filter’s two succeeding positions to the right. For our example with period 2 ⋅ 2 n = 2 ⋅ 22 = 8 with n = 2 , this alternative is

 a0,2   h2 a  h  1,2   3  a2,2   0 a   0  3,2  =   a4,2   0     a5,2   0  a6,2   0     a7,2   0

h1 −h0 0 0 0 0 0 0

h0 h1 h2 h3 0 0 0 0

h3 −h2 h1 −h0 0 0 0 0

0 0 h0 h1 h2 h3 0 0

0 0 h3 h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

0 0 0 0 0 0 h0 h1

 a3,1    b3,1     a    0,1    b0,1   a    1,1  . 0   b1,1   0   a2,1    h3   b2,1   −h2   a3,1     b3,1 

(16.34)

Clearly, the first choice in Eq. (16.32) is better, even though both choices emphasize the importance of the periodicity of the coefficients (which is at the heart of fast computation).

16.4

Low and High Pass Effects: Dual Filters

For the matrix Ω in Eq. (16.29), we should note that the pair of sets of coefficients {h0 , h1 , h2 , h3} and {h3 , −h2 , h1 , −h0 } in the first two rows do averaging (CLPF) and differencing (HPF), respectively, on the

498

Multiresolution and Multirate Signal Processing

  column an to obtain the corresponding elements of the column dn−1 . For example, by preforming the  multiplication of the first row and the second row with the column an , we have a0,n −1 = h0 a0,n + h1a1,n + h2 a2,n + h3 a3,n , b0,n −1 = h3 a0,n − h2 a1,n + h1a2,n − h0 a3,n . The first is definitely an average weighted by the scaling coefficients and is similar to moving average filter with window size of 2. The second row is adding weighted differences for the first two and  second two elements of an . Of course, this is much more clear with the shapes of the Haar scaling func1   1 1   1 tion and its associated wavelet using their corresponding filters  , ,−  and   which 2  2 2  2 show the sum and difference, respectively. The Daubechies 2 coefficients are slightly more difficult to visualize and imagine, however, the + sign for the scaling function filter {h2 , h1 , h0 , h3} and the alternating signs for the wavelet filter {h3 , −h0 , h1 , −h2 } suggests averaging and differencing, respectively. In addition, if we look at the shape of fD2 and y D2 in Figures A16.1 and 16.2 if compared with Haar, we see that fD2 is more or less positive on almost all its compact support, while y D2 resembles to an extent that of the differencing in the Haar wavelet. The above discussion becomes much more clear when we use the Haar scaling coefficients 1   1 1   1 {h0 , h1} =  , ,−  for the low pass filter and   for the high pass filter. Again, for a bet2  2 2  2 1 1 1 1 ter illustration of the usual averaging, we shall normalize these coefficients to  ,  and  , −  , 2 2 2 2 respectively. We illustrate this in the following Example 16.4.1 using the Haar coefficients.

Example 16.4.1 — Illustration with the Haar wavelet matrix.

 Consider the sequence of coefficients in the column matrix {a0,2 , a1,2 , a2,2 , a3,2 } = a2 = [4,2,5, −3]T .  1  We are to use d1 = Ωa2 , where Ω is the Haar wavelet matrix with h0 = h1 = , 2 1 1 1  1 −1 Ω=  2 0 0  0 0

0 0 0 0 . 1 1 1 −1

Now,  a0,1  1 1  b  1  1 −1  0,1  =   a1,1  2  0 0 b   0 0  1,1 

0 0 4 0 0 2   1 1 5 1 −1  −3

(16.35)

499

Advanced Topics

 4+2  1 4−2   =  2  5 + ( −3)  5 − ( −3)

a0,1 =

 4+2   2   6      4 − 2   2  2   2 = = 5 + ( −3)   2      2   1   5 − ( −3)   4    2 

(16.36)

4+2 6 5 + ( −3) 4−2 2 5 − ( −3) 8 = , a1,1 = = 1 ; b0,1 = = , b1,1 = = =4. 2 2 2 2 2 2 2

Hence, the a0,1 and a1,1 are obtained by averaging the first couple and the second couple of 1 the sequence {4,2,5, −3} . On the other hand, b0,1 and b1,1 are obtained by taking the difference 2 between the first two elements 4 and 2, and the second two elements 5 and –3, respectively. 1 Now, we show next that the above Ω , with h0 = h1 = , is its own inverse: 2 1 1 1  1 −1 ΩΩ =  2 0 0  0 0

0 0  1 1 0 0   1 −1  1 1  0 0 1 −1  0 0

0 0 0 0  1 1 1 −1

0  1 + 1 1 − 1 0 1   0 1 1−1 1+1 0 2 0 =  =  0 1 + 1 1 − 1 2  0 2 0  0  0 0 1 − 1 1 + 1

0 1 0 0

0 0 1 0

0 0  = I. 0 1

Hence, ΩΩ = I , where Ω = Ω −1 , and we say that Ω is self adjoint, i.e., ΩT = Ω . This can be seen clearly when we exchange the elements of Ω across the diagonal, which gives ΩT that results 1 in Ω itself. We show next that for the Haar case Ω −1 = 2Ω (using h0 , h1 = ) where we can use it 2  on the above resulting sequence {a0,1 , b0,1 , a1,1 , b1,1} = [3,1,1, 4] = d1 to recover the original sequence  {4,2,5, −3} = [ a0,2 , a1,2 , a2,2 , a3,2 ]T = a2 :

500

Multiresolution and Multirate Signal Processing

1 1   2  1 −1 Ω −1d1 = 2Ωd1 =  2 0 0  0 0

0 0   3 0 0   1   1 1   1 1 −1  4

 4  4  3 + 1  3 − 2 2  2   2   =   = , =  1 + 4 2  5   5   −3  −3  1 − 4  which is the original sequence.

16.5

Numerical Computations for the Fast Discrete Wavelets Transform and its Inverse

We now demonstrate the numerical computations of the FDWT process of decomposition and construction. [The Fast Daubechies Wavelet Transform for decomposition (analysis)]

Example 16.5.1 — The fast Daubechies wavelet transform for decomposition (analysis).

We illustrate the decompositions process of the (extended) coefficients using the fast Daubechies wavelet transform of Eq. (16.29). We will use the same sequence of sample {s0 , s1 , s2 , s3} = {0,1,2,3} . For the latter we will use the mirror image extension around t = 0 and t = 3 with the slope matching of the sequence and its extensions at t = 0,3, . By virtue of this, we shall obtain the extended 7 2  7 2  sequence {s−4 , s−3 , s−2 , s−1 , s0 , s1 , s2 , s3 , s4 , s5 , s6 , s7 } = 4, , , −1,0,1,2, 3, 4, , , −1 with its period 3 3  3 3  7 2   of 8, for example 0,1,2,3, 4, , , −1 . From our understanding from previous section, we get, 3 3   1+ 3 1− 3 3− 3 5− 3 7− 3 , and a2 = , a−1 = , a0 = , a1 = . For the period 8, we will 2 2 2 2 2 need {a0 , a1 , a2 , a3 , a4 , a5 , a6 , a7} . We have a6 = a−2 , since by the periodicity of the coefficients with a−2 = −

1+ 3 1− 3 . Also, a7 = a−1 , since a−1+ 8 = a−1 = a7 , a7 = . For 2 2 illustration purpose a3, a4, and a5, computations are as follows. period 8, a−2 + 8 = a−2 = a6, a6 = −

501

Advanced Topics

6

a3 = ∑ fk f (k − 3) = f3f (0) + f4f (1) + f5f (2) + f6f (3) k =3

= (3)(0) + (4) =

1 + 3  7 1 − 3  2 +  +   (0)  3 2  3 2

12 + 12 3 + 7 − 7 3 19 + 5 3 = 6 6 7

a4 = ∑ fk f (k − 4) = f4f (0) + f5f (1) + f6f (2) + f7f (3) k =4

 7 1 + 3  2 1 − 3 = (4)(0) +   +  + ( −1)(0)  3 2  3 2 =

7+7 3 +2−2 3 9+5 3 = 6 6 8

a5 = ∑ fk f (k − 5) = f5f (0) + f6f (1) + f7f (2) + f8f (3) k =5

1− 3  7  2 1 + 3 + ( −1) + (0)(0) =   (0) +    3  3 2 2 =

2 + 2 3 − 3 + 3 3 −1 + 5 3 = 6 6

a6 = a−2 + 8 = a−2 = − a7 = a−1+ 8 = a−1 =

1+ 3 2

1− 3 2

With these (2)2 n = (2)22 = 8 coefficients of period 8, we are at the level n = 2 and we designate the first low pass filter coefficients as {ak ,n } = {ak ,2 } . After passing these eight coefficients through the first two parallel low and high pass filters, we end up with eight coefficients each {ak ,1}7k =0 and {bk ,1}7k =0 , and after down sampling these become four each. The interpolated signal, using the eight coefficients {ak ,2 }7k =0 , weighted by their corresponding translated scaling functions, is 7

f (t ) = ∑ak ,2f (t − k ) = a0,2f (t ) + a1,2f (t − 1) + a2,2f (t − 2) k =0

+ a3,2f (t − 3) + a4,2f (t − 4) + a5,2f (t − 5) + a6,2f (t − 6) + a7,2f (t − 7).

502

Multiresolution and Multirate Signal Processing

For our previous computation, we used ak ,n = ak : 3− 3 5− 3 7− 3 f (t ) + f (t − 1) + f (t − 2) f (t ) = 2 2 2 +

19 + 5 3 9+5 3 −1 + 5 3 f (t − 3) + f (t − 4) + f (t − 5) 6 6 6

−

1+ 3 1− 3 f (t − 6) + f (t − 7). 2 2

Now, we substitute the eight coefficients for the column matrix on the right of Eq. (16.29) to obtain the eight coefficients {a0,1 , b0,1 , a1,1 , b1,1 , a2,1 , b2,1 , a3,1 , b3,1} for the column on the left. From the matrix multiplication, a0,1 for example, is obtained as the inner product of the first row  of the matrix Ω and the right column of an , a0,1 = [h0

h1

h2

1 + 3 =  4 2

h3 ][ a0,2

a1,2

3+ 3

3− 3

4 2

4 2

a2,2

a3,2 ]T

1− 3 × 4 2  T

 3 − 3 5 − 3 7 − 3 19 + 5 3    2 2 6  2  1 = [(1 + 3)(9 − 3 3) + (3 + 3)(15 − 3 3) 24 2 + (3 − 3)(21 − 3 3) + (1 − 3)(19 + 5 3)] =

14 − 4 3 3 2

b0,1 = [h3

−h2

1 − 3 =  4 2

h1 −

−h0 ][ a0,2

a1,2

3− 3

3+ 3

4 2

4 2

a2,2 −

a3,2 ]T

1+ 3 × 4 2  T

 3 − 3 5 − 3 7 − 3 19 + 5 3    2 2 6  2  1 = [(1 − 3)(9 − 3 3) + ( −3 + 3)(15 − 3 3) 24 2 + (3 + 3)(21 − 3 3) + ( −1 − 3)(19 + 5 3)] =

−2 3 2

503

Advanced Topics

a1,1 = [h0

h1

h3 ][ a2,2

h2

1 + 3 =  4 2

a3,2

3+ 3

3− 3

4 2

4 2

a5,2 ]T

a4,2

1− 3 × 4 2  T

 7 − 3 19 + 5 3 9 + 5 3 −1 + 5 3    6 6 6  2  1 = [(1 + 3)(21 − 3 3) + (3 + 3)(19 + 5 3) 24 2 + (3 − 3)(9 + 5 3) + (1 − 3)( −1 + 5 3)] =

10 + 8 3 3 2

b1,1 = [h3

−h2

1 − 3 =  4 2

h1 −

−h0 ][ a2,2

a3,2

3− 3

3+ 3

4 2

4 2

a4,2 −

a5,2 ]T

1+ 3 × 4 2  T

 7 − 3 19 + 5 3 9 + 5 3 −1 + 5 3    6 6 6  2  1  = 1 − 3 21 − 3 3 + −3 + 3 19 + 5 3 24 2 

(

(

)(

)(

) (

) (

)(

)(

)

+ 3 + 3 9 + 5 3 + −1 − 3 −1 + 5 3   2 = 3 2 a2,1 = [h0

h1

1 + 3 =  4 2

h2

h3 ][ a4,2

a5,2

3+ 3

3− 3

4 2

4 2

a6,2

a7,2 ]T

1− 3 × 4 2  T

 9 + 5 3 −1 + 5 3 1+ 3 1− 3 −   2  6 2  6 1 = [(1 + 3)(9 + 5 3) + (3 + 3)( −1 + 5 3) 24 2 + (3 − 3)( −3 − 3 3) + (1 − 3)(3 − 3 3)] =

6+2 3 3 2

)

504

Multiresolution and Multirate Signal Processing

−h2

b2,1 = [h3

1 − 3 =  4 2

h1 −

−h0 ][ a4,2

a5,2

3− 3

3+ 3

4 2

4 2

a7,2 ]T

a6,2 −

1+ 3 × 4 2  T

 9 + 5 3 −1 + 5 3 1+ 3 1− 3 −   2  6 2  6 1 = [(1 − 3)(9 + 5 3) + ( −3 + 3)( −1 + 5 3) 24 2 + (3 + 3)( −3 − 3 3) + ( −1 − 3)(3 − 3 3)] =

−4 3 3 2

a3,1 = [h0

h1

1 + 3 =  4 2

h3 ][ a6,2

h2

a7,2

3+ 3

3− 3

4 2

4 2

a8,2

a9,2 ]T

1− 3 × 4 2  T

 1+ 3 1− 3 3 − 3 5 − 3 −  2  2 2 2  1 = [(1 + 3)( −1 − 3) + (3 + 3)(1 − 3) 24 2 + (3 − 3)(3 − 3) + (1 − 3)(5 − 3)] =

2−2 3

b3,1 = [h3

2 −h2

1 − 3 =  4 2

h1 −

−h0 ][ a6,2

a7,2

3− 3

3+ 3

4 2

4 2

a8,2 −

a9,2 ]T

1+ 3 × 4 2  T

 1+ 3 1− 3 3 − 3 5 − 3 −  2  2 2 2  1 = [(1 − 3)( −1 − 3) + ( −3 + 3)(1 − 3) 24 2 + (3 + 3)(3 − 3) + ( −1 − 3)(5 − 3)] =0

505

Advanced Topics

The resulting four coefficients, {a0,1 .a1,1 , a2,1 , a3,1} , after down sampling the output of the low pass filter, are with period 4 and we will use this extension. They are passed again to another set of low and high pass filters to result in two samples for each filter. The four output coefficients of the high pass filter {b0,1 , b1,1 , b2,1 , b3,1} are not split further in the spirit of MRA. So, to obtain the two coefficients each of the output of these two filters, we repeat the same matrix multiplication as in Eq. (16.29), a0,0 = [h0

h1

h3 ][ a0,1

h2

1 + 3 =  4 2

3+ 3

3− 3

4 2

4 2

14 − 4 3   3 2 =

a1,1

a3,1 ]T

a2,1

1− 3 × 4 2 

10 + 8 3

6+2 3

3 2

3 2

2 −2 3  2 

T

1 [(1 + 3)(14 − 4 3) + (3 + 3)(10 + 8 3) 24 + (3 − 3)(6 + 2 3) + (1 − 3)(6 − 6 3)]

=

23 + 8 3 6

b0,0 = [h3

−h2

1 − 3 =  4 2

h1 −

−h0 ][ a0,1

a1,1

3− 3

3+ 3

4 2

4 2

a2,1 −

a3,1 ]T

1+ 3 × 4 2  T

14 − 4 3 10 + 8 3 6 + 2 3 2 − 2 3    3 2 3 2 2   3 2 1 = [(1 − 3)(14 − 4 3) + ( −3 + 3)(10 + 8 3) 24 + (3 + 3)(6 + 2 3) + ( −1 − 3)(6 − 6 3)] =

14 − 5 3 6

a1,0 = [h0

h1

1 + 3 =  4 2 6 + 2 3   3 2

h2

h3 ][ a2,1

a3,1

3+ 3

3− 3

4 2

4 2

2−2 3 2

a4,1

a5,1 ]T

1− 3 × 4 2 

14 − 4 3 3 2

10 + 8 3   3 2 

T

506

Multiresolution and Multirate Signal Processing

=

1 [(1 + 3)(6 + 2 3) + (3 + 3)(6 − 6 3) 24 + (3 − 3)(14 − 4 3) + (1 − 3)(10 + 8 3)]

=

13 − 8 3 6

b1,0 = [h3

−h2

1 − 3 =  4 2

h1 −

6 + 2 3   3 2 =

−h0 ][ a2,1

a3,1

3− 3

3+ 3

4 2

4 2

2−2 3

−

1+ 3 × 4 2  10 + 8 3   3 2 

14 − 4 3

2

a5,1 ]T

a4,1

3 2

T

1 [(1 − 3)(6 + 2 3) + ( −3 + 3)(6 − 6 3) 24 + (3 + 3)(14 − 4 3) + ( −1 − 3)(10 + 8 3)]

=

−10 + 3 6

The {b0,0 , b1,0 } are not split and to be added to the previous four as {b0,0 , b1,0 | b0,1 , b1,1 , b2,1 , b3,1} The {a0,0 , a1,0 } of the low pass filter are now input to the next pair of filters to end up (after down sampling) with one sample output for each of the two filters: a0, −1 = [h0

h1

1 + 3 =  4 2

h3 ][ a0,0

h2

a1,0

3+ 3

3− 3

4 2

4 2

a2,0

a3,0 ]T

1− 3 × 4 2 

 23 + 8 3 13 − 8 3 23 + 8 3 13 − 88 3    6 6 6 6   1 = [(1 + 3)(23 + 8 3) + (3 + 3)(13 − 8 3) 24 2 + (3 − 3)(23 + 8 3) + (1 − 3)(13 − 8 3)] =3 2 b0, −1 = [h3

−h2

1 − 3 =  4 2

h1 −

−h0 ][ a0,0

a1,0

3− 3

3+ 3

4 2

4 2

a2,0 −

a3,0 ]T

1+ 3 × 4 2 

T

507

Advanced Topics

T

 23 + 8 3 13 − 8 3 23 + 8 3 13 − 88 3    6 6 6 6   1 = [(1 − 3)(23 + 8 3) + ( −3 + 3)(13 − 8 3) 24 2 + (3 + 3)(23 + 8 3) + ( −1 − 3)(13 − 8 3)] =

5+8 3 3 2

b0, −1 is added to the previous ones to make {b0, −1 |, b0,0 , b1,0 |, b0,1 , b1,1 , b2,1 , b3,1 |} . The significance of the coefficient a0, −1 output of the last low pass filter, that it is very close to the average of the extended sequence. The seven coefficient outputs of the three (in cascade) high pass filters are supposed to give the “details” of the sequence and its mirror image extension. Now, we write the approximated signal f decomposition in four equivalent ways: (i) V0 , (ii) V0 = V−1 ⊕ W−1 , (iii) V0 = V−2 ⊕ W−2 ⊕ W−1 , and (iv) V0 = V−3 ⊕ W−3 ⊕ W−2 ⊕ W−1 . These correspond to (i) the output of the first low pass filter at scale l0 =1, (ii) the outputs of the first pair of the filters at scale l−1 =2, (iii) the output of the low pass filter at scale l−2 = 4 in the second filters pair plus the output of the second high pass filter. We add to those what was stored of the output of the first high pass filter. (iv) The output of the low pass filter at scale  −3 = 8 of the third filters pair, plus the output of its parallel high pass filter, and what was stored as the outputs of the high pass filters from the first and second pairs of the filtering operations. (i) In V0 : We have ak ,n , with period N = (2)2 n = (2)2 2 = 8, f (t ) = a f (t ) + a f (t - 1) + a f (t - 2) + a f (t - 3) 0,2 1,2 2,2 3,2 + a4,2f (t − 4) + a5,2f (t − 5) + a6,2f (t − 6) + a7,2f (t − 7). Here we write ak ,n for n = 2 of the period N = (2)2 2 = 8. (ii)

V0 = V−1 ⊕ W−1 : We have {ak ,n−1} ={ak ,1}k4=0 , {bk ,n−1} ={bk ,1}k4=0 with period N = (2) 2 n−1 = (2) 21 = 4 for each sequence,  t  t t  t  f (t ) = a0,1f   + a1,1f  − 2(1) + a2,1f  − 2(2) + a3,1f  − 2(3)  2  2 2  2   t  t  t + b0,1y  − 1 + b1,1y  − 1 − 2(1) + b2,1y  − 1 − 2(2)  2  2  2 t  + b3,1y  − 1 − 2(3) . 2 

Note the translation by 2k in the above scaling functions and wavelets for the purpose of down sampling. Also, there is an extra translation of the wavelet by 1 to accommodate matching the compact support of the wavelet with its associated scaling function.

508

Multiresolution and Multirate Signal Processing

(iii) In V0 = V−2 ⊕ W−2 ⊕ W−1 we have {ak ,n− 2 } ={ak ,0 }1k =0 , {bk ,n−1} ={bk ,0 }1k =0 with period N = (2)2 n− 2 = (2)2 0 =2 for each sequence, t   t t  f (t ) = a0,0f   + a1,0f  − 4(1) + b0,0y  − 1 4   4 2  t  t  + b1,0y  − 1 − 4(1) + b0,1y  − 1 2  4  t  t  + b1,1y  − 1 − 2(1) + b2,1y  − 1 − 2(2) 2  2  t  + b3,1y  − 1 − 2(3) . 2  We note here the down sampling again in V−2 ⊕ W−2 , of the first four terms, where the translations now are by steps of 4. (iv) In V0 = V−3 ⊕ W−3 ⊕ W−2 ⊕ W−1 we have {ak ,n− 3} ={ak , −1}k =0 and {bk ,n− 3} ={bk , −1}k =0 with period 1 N = (2) 2 n− 3 =(2) 22 − 3 = (2) =1, 2  t t  t  f (t ) = a0, −1f   + b0, −1y  − 1 + b0,0y  − 1  8 8  4  t  t  t  + b1,0y  − 1 − 4(1) + b0,1y  − 1 + b1,1y  − 1 − 2(1) 4  2  2  t  t  + b2,1y  − 1 − 2(2) + b3,1y  − 1 − 3(2) . 2  2  The last step is to substitute all coefficients to compute back the signal:   t   5 + 8 3  t   14 − 5 3   t f (t ) = (3 2)f   +  y  − 1 +    y  4 − 1  8  3 2   8   6   −10 + 3   t    −2   t + y  − 1 − 4 +  y  − 1    3 2 2  6   4  2  t   −4 3   t  − 1 − 4 + y  − 1 − 2 +  y    3 2   2  3 2 2 t  + (0)y  − 1 − 6 . 2  We said that the decomposition in the V−3 corresponds to a blurred picture or an average of the  t signal. This is seen above in the coefficients a0, −1 = 3 2 of f   .  8

509

Advanced Topics

Example 16.5.2 — Illustrating the inverse Daubechies wavelet transform for reconstruction (synthesis).

For this illustration we follow the reverse process of what we did for the Forward Daubechies wavelet transform. We start with the sequence {a0, −1 , b0, −1} , which is periodic with period 2. We extend it as {a0, −1 , b0, −1 ; a0, −1 , b0, −1} to fit the matrix multiplication by the ( 4 × 4 ) inverse matrix Ω −1 for   an = Ω −1dn −1 in Eqs (16.31) and (16.32) for obtaining a0,0 and a1,0 .  a0,0   h0 a  h  1,0  =  1  a0,0   h2 a  h  1,0   3 a0,0 = [h0

h3

h3 −h2 h1 −h h0 h1 ][ a0, −1

h2

h1   a0, −1  −h0   b0, −1    h3   a0, −1  −h2   b0, −1 

h2 h3 h0 h1

b0, −1

a0, −1

b0, −1 ]T

1 + 3 1 − 3 3 − 3 3 + 3  = × 4 2  4 2 4 2  4 2 T  5+8 3 5+8 3 3 2 3 2  3 2 3 2   1 = [(1 + 3)(18) + (1 − 3)(5 + 8 3) 24 + (3 − 3)(18) + (3 + 3)(5 + 8 3)] =

23 + 8 3 6

a1,0 = [h1

−h2

é3 + 3 =ê êë 4 2  3 2  =

h3 -

−h0 ][ a0, −1

3- 3

1- 3

4 2

4 2

5+8 3 3 2

3 2

b0, −1 -

b0, −1 ]T

a0, −1

1+ 3 ù ú´ 4 2 úû

5+8 3  3 2 

T

1 [(3 + 3)(18) + ( −3 + 3)(5 + 8 3) 24 + (1 − 3)(18) + ( −1 − 3)(5 + 8 3)]

=

13 − 8 3 6

These a1,0 , a0,0 at n = 0 are now combined with their respective b1,0 , b0,0 to make a sequence {a1,0 , b1,0 , a0,0 , b0,0 } of period 4, and we extend it to find a0,1 and a1,1 from Eq. (16.32). Here, we will

510

Multiresolution and Multirate Signal Processing

need a1,0 and b1,0 from the second period since the coefficients of the square matrix are shifted by two positions:  a0,1   h0 a  h  1,1  =  1  a2,1   h2 a  h  3,1   3 a0,1 = [h0

h3

1 + 3 =  4 2

h3 −h2 h1 −h h0

h1 ][ a0,0

h2

h1   a0,0  −h0   b0,0    h3   a1,0  −h2   b1,0 

h2 h3 h0 h1

b0,0

1− 3

3− 3

4 2

4 2

a1,0

b1,0 ]T

3+ 3 × 4 2  T

 23 + 8 3 14 − 5 3 23 + 8 3 14 − 5 3    6 6 6 6   1 = [(1 + 3)(23 + 8 3) + (1 − 3)(14 − 5 3) 24 2 + (3 − 3)(13 − 8 3) + (3 + 3)( −10 + 3 3)] =

14 − 4 3 3 2 −h2

a1,1 = [h1

h3

−h0 ][ a0,0

b0,0

a1,0

b1,0 ]T

3 + 3 3 − 3 1− 3 1+ 3 − − = × 4 2  4 2 4 2  4 2 T  23 + 8 3 14 − 5 3 23 + 8 3 14 − 5 3    6 6 6 6   1 = [(3 + 3)(23 + 8 3) + ( −3 + 3)(14 − 5 3) 24 2 + (1 − 3)(13 − 8 3) + ( −1 − 3)( −10 + 3 3)] =

10 + 8 3 3 2

a2,1 = [h2

h1

h0

3 − 3 =  4 2  23 + 8 3  6 

h3 ][ a0,0

b0,0

3+ 3

1+ 3

4 2

4 2

14 − 5 3 6

a1,0

b1,0 ]T

1− 3 × 4 2 

23 + 8 3 6

14 − 5 3   6 

T

511

Advanced Topics

=

=

1

[(3 − 3)(23 + 8 3) + (3 + 3)(14 − 5 3) 24 2 + (1 + 3)(13 − 8 3) + (1 − 3)( −10 + 3 3)] 6+2 3 3 2

a3,1 = [h3

−h0

1 − 3 =  4 2

h1 −

−h2 ][ a0,0

b0,0

1+ 3

3+ 3

4 2

4 2

a1,0 −

b1,0 ]T

3− 3 × 4 2  T

 23 + 8 3 14 − 5 3 23 + 8 3 14 − 5 3    6 6 6 6   1 = [(1 − 3)(23 + 8 3) + ( −1 − 3)(14 − 5 3) 24 2 + (3 + 3)(13 − 8 3) + ( −3 + 3)( −10 + 3 3)] =

2−2 3 2

These {a0,1 , a1,1 , a2,1 , a3,1} are now added to the stored {b0,1 , b1,1 , b2,1 , b3,1} at the same level n = 1 to make the sequence of period 8: {a0,1 , b0,1 , a1,1 , b1.1 , a2,1 , b2,1 , a3,1 , b3,1} . As we discussed in obtaining Eq. (16.32), the first two rows of the inverse matrix Ω are completed equivalently by adding the first halves {h2 , h1} and {h3 , −h0 } of the two filters at the end of the first and second row, respectively.  a0,2   h0 a  h  1,2   1  a2,2   h2 a  h  3,2  =  3  a4,2   0     a5,2   0  a6,2   0     a7,2   0

h3 −h2 h1 −h0 0 0 0 0

0 0 h0 h1 h2 h3 0 0

0 0 h3 −h2 h1 −h0 0 0

0 0 0 0 h0 h1 h2 h3

0 0 0 0 h3 −h2 h1 −h0

h2 h3 0 0 0 0 h0 h1

h1   a0,1  −h0   b0,1    0   a1,1  0   b1,1    0   a2,1    0   b2,1  h3   a3,1    −h2   b3,1 

a0,2 = h0 a0,1 + h3 b0,1 + h2 a3,1 + h1b3,1  1 + 3   14 − 4 3   1 − 3   −2  =    +   4 2   3 2   4 2  3 2

512

Multiresolution and Multirate Signal Processing

 3 − 3  2 − 2 3  3 + 3 +   +  (0 ) 2   4 2   4 2  1  = (1 + 3)(14 − 4 3) + (1 − 3)( −2) + (3 − 3)(6 − 6 3) + 0  24  =

3− 3 2

a1,2 = h1a0,1 − h2 b0,1 + h3 a3,1 − h0 b3,1  3 + 3   14 − 4 3   3 − 3   −2  =    −   4 2   3 2   4 2  3 2 1− 3  2 − 2 3 1+ 3 +   +  (0 ) 2   4 2   4 2  =

1  (3 + 3)(14 − 4 3) + ( −3 + 3)( −2) + (1 − 3)(6 − 6 3) + 0  24 

=

5− 3 2

a2,2 = h2 a0,1 + h1b0,1 + h0 a1,1 + h3 b1,1  3 − 3   14 − 4 3   3 + 3   −2  =    +   4 2   3 2   4 2  3 2  1 + 3   10 + 8 3   1 − 3   2  +    +   4 2   3 2   4 2  3 2 1  = (3 − 3)(14 − 4 3) 24  +(3 + 3)( −2) + (1 + 3)(10 + 8 3) + (1 − 3)(2) =

7− 3 2

a3,2 = h3 a0,1 − h0 b0,1 + h1a1,1 − h2 b1,1  1 − 3   14 − 4 3   1 + 3   −2  =    −   4 2   3 2   4 2  3 2  3 + 3   10 + 8 3   3 − 3   2  +    −   4 2   3 2   4 2  3 2

513

Advanced Topics

=

=

1  (1 − 3)(14 − 4 3) + ( −1 − 3)( −2) 24  +(3 + 3)(10 + 8 3) + ( −3 + 3)(2) 19 + 5 3 6

a4,2 = h2 a1,1 + h1b1,1 + h0 a2,1 + h3 b2,1  3 − 3   10 + 8 3   3 + 3   2  =    +   4 2   3 2   4 2  3 2  1 + 3   6 + 2 3   1 − 3   −4 3  +   +    4 2  3 2   4 2  3 2  1  = (3 − 3)(10 + 8 3) + (3 + 3)(2) 24  +(1 + 3)(6 + 2 3) + (1 − 3)( −4 3) =

9+5 3 6

a5,2 = h3 a1,1 − h0 b1,1 + h1a2,1 − h2 b2,1  1 − 3   10 + 8 3   1 + 3   2  =    −   4 2   3 2   4 2  3 2  3 + 3   6 + 2 3   3 − 3   −4 3  +   −    4 2  3 2   4 2  3 2  1  = (1 − 3)(10 + 8 3) + ( −1 − 3)(2) 24  +(3 + 3)(6 + 2 3) + ( −3 + 3)( −4 3) =

−1 + 5 3 6

a6,2 = h2 a2,1 + h1b2,1 + h0 a3,1 + h3 b3,1  3 − 3   6 + 2 3   3 + 3   −4 3  =   +    4 2  3 2   4 2  3 2  1+ 3  2 − 2 3 1− 3 +   +  (0 ) 2   4 2   4 2 

514

Multiresolution and Multirate Signal Processing

=

1  (3 − 3)(6 + 2 3) + (3 + 3)( −4 3) 24  +(1 + 3)(6 − 6 3) + 0 

=−

1+ 3 2

a7,2 = h3 a2,1 − h0 b2,1 + h1a3,1 − h2 b3,1  1 − 3   6 + 2 3   1 + 3   −4 3  =   −    4 2  3 2   4 2  3 2   3 + 3  2 − 2 3  3 − 3 +   −  (0 ) 2   4 2   4 2  =

1  (1 − 3)(6 + 2 3) + ( −1 − 3)( −4 3) 24  +(3 + 3)(6 − 6 3) + 0 

=

1− 3 2

Interested readers can try out following variations for further insights: 1. (a) Follow above examples to illustrate the Haar wavelet transform and its inverse for the coefficients sequence {1, −2,3,5} by showing the decomposition in V−2 = V1 ⊕ W1 , then the reconstruction to show that you recover the oringinal sequence. (b) We note that this sequence in part (a) has a jump at t = 1, and we expect, with good resolution, that it can be detected by the wavelet part of the decomposition. Try V2 ⊕ W0 ⊕ W1 to see if the part in W0 ⊕ W1 shows it better than just the W1 in V2 = V1 ⊕ W1 of part (a). If not, try V3 = V0 ⊕ W0 ⊕ W1 ⊕ W2 . 2. Consider the sequence of samples {0,1, 4,9}. (a) Use extension with zeros to calculate its eight coefficients, and write its interpolation f (t ) ∈V0 . (b) Use the first Daubechies wavelet transform, for the decomposition in: (i) V0 = V−1 ⊕ W−1 (ii) V0 = V−2 ⊕ W−2 ⊕ W−1 (iii) V0 = V−3 ⊕ W−3 ⊕ W−2 ⊕ W−1 (c) Use the inverse Daubechies wavelet transform for the construction of the original signal to verify your results. 3. Repeat (2) with the periodic extension with matching slopes at the ends. Compare your results with that of (2). How does the edge effect in (2) affect the comparison?

515

Advanced Topics

4. Repeat (2) and (3) for the sequence {1,3} , using the zeros extension for the Haar scaling function and wavelets. 5. Repeat (2) for the sequence {0,1, 4,9} with the zeros extension and the Haar scaling functions and wavelets.

16.6

Multi-dimensional Wavelets

A signal is a 1-D entity and often gets represented using function of independent variable time ‘t’. We represent images in space as they are 2D entities. If we can extend 1D wavelet analysis to 2D, then we should be able to extend it to M-dimensions. 2D wavelet analysis has two possible methods: 1. An easier approach in which we construct tensor product wavelets which are separable. 2. Little difficult part of wavelet theory in which two variables are not separable in general. This means a wavelet function y ( x, y) can not be written as product y ( x ) ⋅ y ( y) and requires more attention.

16.6.1

Separable Tensor Product Wavelets

A wavelet transform of M-dimensional vector can be obtained in easiest way by transforming array sequentially to first index,then to second index, going all the way till M th index. The order of indices hardly matters, as by associativity of matrix product, the result remains independent of the order. Each transformation boils down to multiplication with orthogonal matrix. An image is a good example of 2D information representation. A 2D-wavelet analysis of an image upto scale 2 can be depicted as follows: (A)

I

original image (B)

L

H

One step of 1D transform on each rows of image I, resulting in low resolution ‘L’ & high resolution ‘H’ part.

516

Multiresolution and Multirate Signal Processing

LL

LH

HL

HH

(C) One step of 1D transform on each of columns of image I, resulting in LL,LH,HL,HH (4 subbands).

L

H

HL

LH

HH

(D) For next scale,in spirit of MRA we process only energy rich LL band. 1-D transform row-wise. LL

LH LH

HL HH

HL

HH

(E) 1-D transform column-wise  Nomenclature: LL: Low pass filtering for rows and columns LH: Low pass filtering for columns and high pass filtering for rows HL: High pass filtering for columns and low pass filtering for rows HH: High pass filtering for columns and high pass filtering for rows. Typically, HH captures diagonal features, HL gives horizontal and LH gives vertical features, LL captures low pass features and has highest energy. If ‘I’ is the image to be analyzed and K =  H G  is the transformation matrix in 1-D wavelet sense, then LL I1 =

LH

HL HH

˜ *] ⋅I⋅ [K ˜ *]T = [K

517

Advanced Topics

The rectangular transform corresponds to taking 1-D wavelet transform in x and y independently  we with K 1 matrix exactly of half size (downsampled in x as well as y by 2) but of same class of K, get second step,  K ∗ I = 1  0

0  1  K 1∗ ⋅I ⋅ I  0

2

0  I

T

Rectangular division corresponds to, V j = span{fj ,k ( x ) ⋅ fj ,l ( y), fj ,k ( x ) ⋅ y j ,l ( y),y j ,k ( x ) ⋅ fj ,l ( y) : k, l ∈ Z} and Wj = span{y j ,k ( x ) ⋅ y j ,l ( y); k, l ∈ Z} Thus, (h ⊗ g)( x, y) = h( x ) ⋅ g( y) ∴ I ( x, y) = ∑∑qm, j ,l ,k (y ml ⊗ y jk )( x, y) m ,l j , k

In MRA, the next step is applied only on LL band. T

1  ⋅  K 1∗  which gives ∴ Let’s take LL band separate and perform  K 1∗  ⋅  I LL

V j = span{fj , k ( x )fj , l ( y) : k, l ∈ z} (LL square) Wj is spanned by mixed basis.

{

Wj = span fj , k ( x ).y j , l ( y),y j , k ( x ).fj , l ( y),y j , k ( x ).y j , l ( y) : k, l ∈ z

There is scaling of 2

−j

}

(HL,LH,HH squares) for both x & y directions.

V j +1 = V j(+x1) ⊗ V j(+y1) = V j( x ) ⊕ Wj( x )  ⊗ V j( y ) ⊕ Wj( y )  = V j( x ) ⊗ V j( y )  ⊕ V j( x ) ⊗ Wj( y )  ⊕ Wj( x ) ⊗ V j( y )  ⊕ Wj( x ) ⊗ Wj( y )  The calculation of coefficients by recursion in matrix form can be now set: * ×V × éH * ù V j -1 = H j ê ë úû

T

(LL sub-band)

518

Multiresolution and Multirate Signal Processing

 * × V × éG * ù Wj(-x1) = H j ê ë úû

T

* ×V × éH * ù Wj(-y1) = G j ê ë ûú

T

(LH sub-band) (HL sub-band)

 * × V × éG * ù Wj(-xy1 ) = G j ê ë ûú

T

(HH sub-band)

Thus, reconstruction shall be: V j +1 = H ⋅ V j ⋅ H T + G ⋅ Wj( x ) ⋅ H T + H ⋅ Wj( y ) ⋅ G T + G ⋅ Wj( xy ) ⋅ G T

16.6.2

Non-Separable wavelets

If two variables are not separable then the row-wise and column-wise application in 1D style will not work. Let’s derive this using conventional convolution mechanism. In 1D sense f

g

H

If ‘H’ is the impulse response of filter then ‘g’ is calculated using g = h* f This holds true for 2D filter as well. Let H 2 be the 2D filter with impulse response h2 = {hk1,k 2 | k1 , k2 ∈ z} , and g2 = h2 * f2 gn1,n 2 = ∑∑K 2 hK K1

1 , K2

(16.37)

fn1− k1,n 2 − k 2

Convolution in time or sparse becomes multiplication in ‘Z’ domain. ∴ ∴

⋅

∴ G( Z ) = H ( Z ) ⋅ F ( Z )     −K −K −K −K ∴ G(⋅ Z ) =  ∑ hK , K ⋅ Z1 1 ⋅ Z 2 2  ⋅  ∑ fK , K ⋅ Z1 1 ⋅ Z 2 2   K1 , K2 1 2   K1 , K2 1 2 

(16.38)

Let ‘M’ describe subsampling in 2D sense. For separable case ‘M’ is diagonal, for non-seprable case it’s not. (↓ M ) y(n1 , n2 ) = y( M n ) = y(2 n1 ,2n2 ) Equation (16.39) depicts downsampling by 2 (↓2) in every direction. ∴ Only these samples for which n1 + n2 = even are retained.

(16.39)

519

Advanced Topics

 H ( Z1 , Z 2 ) H ( − Z1 , − Z 2 ) ∴ M ( Z1 , Z 2 ) =    G( Z1 , Z 2 ) G( − Z1 , − Z 2 ) 

(16.40)

We choose simplest solution of form M ⋅ M ∗ = 2I

(16.41)

For Eq. (16.41) to hold, G(Z1 , Z 2 ) should be an odd delay of H(− Z1−1 , − Z 2−1 ) For 1D MRA,with H (w ) = ∑ Khk ⋅ e − ikw the dialation equation was, f (t ) = 2 åhk × f (2t - K ) = 2 < h* , F0 (2t ) > 2

l (Z )

K

(16.42)

we can generalize this to M-D f (t ) = M åhk × f ( M t - K ) = M < h* , F0 ( Mt ) > 2

l (Z2 )

K

(16.43)

where, F0 (t ) = (f0 k (t )) Î l 2 ( Z 2 ). For variable change of S = Mt − K M ∫∫f ( Mt − K ) ⋅ dt1 − dt2 = ∫∫f (s ) ⋅ ds1 ⋅ ds2

(16.44)

There will be (M – 1) wavelet functions, y ( m ) (t ) = M ∑gK( m ) ⋅ f ( Mt − K )

(16.45)

K

y ( m ) (t ) = M < g*( m ) , F0 ( Mt ) > 2

l (Z2 )

, M = 1,, M - 1

(16.46)

From Eqs (16.43)-(16.46), V0 = span{fok (t ) = f (t − K ) : K ∈ Z 2 }

(16.47)

(m) W0 = span{y ok (t ) = y ( m ) (t − k ) : m = 1,, M − 1; K ∈ Z 2 }

(16.48)

and orthogonally,

Taking all dilates and translates, of wavelet, n

y n( m, K) (t ) = M 2y (m)( M n t − K ) This is the orthonormal basis for entire L2 ( R 2 ) 2-D wavelet examples: 1. Mexican Hat: 1 y (t ) = ( t 2 −2)exp( −  t 2 ) 2

520

Multiresolution and Multirate Signal Processing

2. Morlet:  1  y (t ) = exp(iK T t ) ⋅ exp  −  At 2   ( K > 56 → admissible)  2 

16.7

The Two-Dimensional Haar Wavelet Transform

In this session we shall demonstrate the separable basis for doing the 2D wavelet analysis. We shall keep using Haar basis to ensure the readers will find it easy to grasp the 2D analysis steps and that will enable them to understand how to generalize the analysis that can be extended to M dimensional analysis. The Haar wavelet transform, as has been demonstrated throughout the book, does  a simple averaging and differential to each pair of a one-dimensional sequence, for example, S = {s0 , s1 , s2 , s3} . So, now we have an idea about a two-dimensional Haar wavelet transform of a two dimensional  1 1  sequence, lets take case of the four points in space: z0 = f (0,0) = 9, z1 = f  0,  = 7, z2 = f  ,0 = 5  2 2   1 1 and z3 = f  ,  = 3 . Even though this is a 2 × 2 array, we hope that it will give some feeling about  2 2 what we may expect in dealing with two-dimensional images. Here, we can take the values 9, 7, 5, and 3 can be thought of as a measure of the luminance or chrominance in an image. These samples are easily interpolated by using a two-dimensional Haar scaling function with an 1 1 1 1 area of × = , when using scale l1 = in both the x and y directions. With scale l0 = 1 , the double 2 2 4 2 Haar scaling function can be written as: 0 £ x < 1 and 0 £ y < 1 otherwise

ì1, F(0) (0,0) ( x, y ) = í î0,

(16.49)

as shown in Fig. 16.3, z 1

1/2

0

1/2 1

1 x

y 1/2

1/2 (1, 1)

Figure 16.3

The Haar wavelet Φ(0) (0,0)(x , y ) (scaling function in two dimensions)

The expression in Eq. (16.49) we obtain from the tensor product of our usual scaling function, ì1, 0 £ x < 1 , 0 £ y < 1 (0) f ( x ) º j (0,0) ( x) = í î0, otherwise

(16.50)

521

Advanced Topics

and a similar one in the y -direction, 1, 0 ≤ y < 1 (0) f ( y) ≡ j (0,0) ( y) =  0, otherwise

(16.51)

ì1, 0 £ x < 1 , 0 £ y < 1 (0) (0) F(0) . (0,0) ( x, y ) = f(0,0) ( x )f(0,0) ( y ) = í î0, otheerwise

(16.52)

We will use the subscript (a, b) in Φ (( aj ),b ) ( x, y) to indicate the top left corner of its base support in the x − y plane. The superscript ( j ) is used to indicate the scale l j (in this case l0 = 1 ) in both the x and y directions, where we see that the base of the cube is 1 × 1 in Fig. 16.3. Reader will note that Φ (0) (0,0) ( x, y ) in Eq. (16.52) is asociated with “averaging” in the x and y directions (LL). However, we expect more operations, such as average of differencing (LH), difference of averaging (HL), and difference of differencing (HH). As it may sound, this would involve other Haar h wavelet actions in the y , x , and both (namely, diagonal) directions, to which we shall refer as Ψ(0,0) ( x, y ) , v d Ψ(0,0) ( x, y) , and Ψ(0,0) ( x, y) , respectively. The scale used will be spelled out, or we may write h Ψ(1) (0,0) ( x, y ) , 1 (1) (1) , for example. Here h and v refer to the horizontal and vertical v Ψ(0,0) ( x, y ) and d Ψ(0,0) ( x, y ) at scale l1 = 2 edges resulting from the differencing caused by the wavelets actions along the perpendicular direction to that particular edge. Such three mixed combinations that involve wavelets, are the reason behind using the symbol Ψ instead of Φ , as the latter is reserved for the pure averaging as in Eq. (16.52). 1 1 1 We note that we are moving to the scale for the four × squares of the unit square, where we 2 2 2 1 will also involve translations by . Reader should note that normalization happens in both the dimen2 h sions and squeezing of area is not necessary. For the two-dimensional wavelet, such as Ψ(0,0) ( x, y) , if 1 we are to scale it with scale l1 = , and we want to indicate that, we use h Ψ(1) (0,0) ; the same notation is 2 1 used for the others. An example of scaling with l1 = is 2 1 1  1, 0 ≤ x < ,0 ≤ y < (1) (0) Φ (0,0) = Φ (0,0) (2 x,2 y) =  (16.53) 2 2 0, otherwise

1 1 1 1 1 which is located at the top-left × square, 0 ≤ x ≤ , 0 ≤ y ≤ in Fig. 16.3. Its translation by in 2 2 2 2 2 the y-direction is 1 1 ì æ 1 öö ï1, 0 £ x < , £ y < 1 æ (1) F(1) 2 x ,2 y º F ( x , y ) = (16.54) 2 2 í ÷ (0,0) ç (0,1) ç 2 ÷ø ø è è ïî0, otherwise 1 1 which is located at the top-right × square. The same thing is done for the other Φ (1) (0,0) functions. 2 2 Illustrative cases will be: 1  (1) Φ (1) (0,0) (2  x −  , y ) ≡ Φ (1,0) ( x, y ),  2

522

Multiresolution and Multirate Signal Processing

1  1  (1) Φ (0) (0,0)  2 x −  ,2  y −  ≡ Φ (1,1) ( x, y ).  2  2

(16.55)

These are shown in Fig. 16.4. z 1 +1 0

1/2

1/2

1

1 y

x −1 −1

Figure 16.4

h

Ψ(0) (0,0)(x , y ) - The horizontal wavelet

Two dimensional Haar bases will be written as:

h Ψ(0,0)

1  0 ≤ x < 1,0 ≤ y < 1, 2  1  (0) (0) = f(0,0) ( x )y (0,0) ( y) =  −1, 0 ≤ x < 1, ≤ y < 1 2  otherwise, 0,  

(16.56)

as illustrated in Fig. 16.4. This can be interpreted as the wavelet operation in the Y-direction, which results in differences, followed by the scaling function operation in the X-direction, which averages these differences. Here, we 1 are using the scale l1 = . So, from Figs 16.3 and 16.4, we may see that h Ψ(0) (0,0) ( x, y ) can be written in 2 terms of the Φ (1) basis at this scale as a sum of differences (in the x-direction). h (1) (1) (1) (1) Ψ(0,0) ≡ h Ψ(0) (0,0) ( x, y ) = (Φ (0,0) − Φ (0,1) ) + (Φ (1,0) − Φ (1,1) )

(16.57)

This is a sum in the x -direction (due to the scaling function) of the differences in the Y-direction caused by the wavelet action. The action of differential causes the vertical edges parallel to the x -axis, h (0) hence the use of h in h Ψ(0) (0,0) (or Ψ(0,0) ). Next, we have v Ψ(0,0) or just,

523

Advanced Topics

1 ì 0 £ x < ,0 £ y < 1 ï1, 2 ï 1 ï v Y(0,0) ( x, y) = y ( x )f ( y) = í-1, £ x < 1,0 £ y < 1 2 ï otherw wise. ï0, ï î

(16.58)

Here, the wavelet action in the x -direction causes the vertical edges parallel to the y -axis, as shown in Fig. 16.5. z 1 −1 1/2 0

1

1

y 1/2

x

−1 −1

Figure 16.5

v Ψ(0,0) (x , y) - The vertical wavelet

v Again we can see from Fig. 16.3 and 16.5 that this f(0,0) ( x, y) can be expressed in terms of the Φ (1) basis as a difference of sums in the y -direction, v (1) (1) (1) Ψ(0,0) = (Φ (1) (0,0) + Φ (0,1) ) − (Φ (1,0) + Φ (1,1) ),

(16.59)

which can be seen clearly as we look at the back positive half versus the front negative half in Fig. 16.5. Next is the diagonal wavelet,

d

Y(0) (0,0)

ì ï1, ï ï-1, ï ï = y ( x )y ( y) = í ï1, ï ï-1, ï ï î0,

1 1 0 £ x < ;0 £ y < 2 2 1 1 £ x < 1;0 £ y < 2 2 1 1 £ x < 1; £ y < 1 2 2 1 1 0 £ x < ; £ y 1 array. For our example of  6 3 S=  , the horizontal sweep (of a Haar low pass filter on each of the two rows) gives  2 1 6 + 3 6 − 3 9 3   2 2  2 2     6 3  H  . = 1 2  →       3 1 1+ 2 1− 2    −  2 2  2  2 The vertical sweep (of a Haar low pass filter followed by a high pass filter on each of the two columns) on the above result gives

9 2  3  2

9 3 2 + 2 3   2 2  V  → 1 −  9 3 2   − 2 2  2

3 1 − 2 2  12  2  4  =  3 1  6  + 2 2  4 2 

2 4  . 4  4

(16.74)

 s0,0 s0,1  The horizontal sweep followed by a vertical sweep on S =   with the use of the two s1,0 s1,1  dimensional Haar wavelet transform follows. First, let us refer to the horizontal and vertical sweeps by Ω H and ΩV operations, respectively, then

529

Advanced Topics

 s0,0 S=  s1,0

s + s s0,1  Ω H 1  0,0 0,1 → s1,1  2   s1,0 + s1,1 

s0,0 − s0,1    s1,0 − s1,1 

(16.75)

 s0,0 S=  s1,0

s + s s0,1  ΩV 1  0,0 1,0 → s1,1  2   s0,0 − s1,0 

s0,1 + s1,1   . s0,1 − s1,1 

(16.76)

So, the horizontal sweep followed by the vertical sweep on S follows from operating with ΩV on the result in Eq. (16.75), ΩV Ω H S (s0,0 + s0,1 ) + (s1,0 + s1,1 ) (s0,0 − s0,1 ) + (s1,0 − s1,1 ) 1  =   4 (s0,0 + s0,1 ) − (s1,0 + s1,1 ) (s0,0 − s0,1 ) − (s1,0 − s1,1 )    s0,0 + s0,1 + s1,0 + s1,1 1 =  4  s0,0 + s0,1 − s1,0 + s1,1 

s0,0 − s0,1 + s1,0 − s1,1    s0,0 − s0,1 − s1,0 − s1,1 

(16.77)

which is what we have in Eq. (16.72) with the use of the four wavelets basis, where the four eleh v ments of the above matrix w0,0 , w0,1 , w1,0 , and w1,1 correspond to the coefficients of Φ (0) (0,0) , Φ (0,0) , Φ (0,0) , d and Φ (0,0) in Eq. (16.72).

16.7.3

Working with Bigger Arrays (4 ë 4 toy example)

What we did above was for a 2 × 2 array, where we constructed the two dimensional (Haar) wavelets v h d bases Φ (0) (0,0) , Ψ(0,0) , Ψ(0,0) , and Ψ(0,0) . There, we ended with pure averagings that gave us the average of the samples of the array as the coefficient of the pure two-dimensional low pass filter asociated with Φ (0) 0,0 . The problem becomes more involved when we consider a 4 × 4 array with its 16 elements. First, 1 1 1 we go to scale l2 = , and we have to find 16 bases on the sixteen × squares of the unit square in 4 4 4 Fig. 11.1. Second, and most important, when we go to the 4 × 4 array, the above horizontal and vertical sweeps will not be enough, as we shall discuss in the following sections. This is so since instead of ending with pure averaging (where we stop) at the top left corner of the 2 × 2 case, we will have four elements of averages mixed with differences at the top left corners of the resulting four 2 × 2 matrices. These four elements must be operated on by the two sweeps to result in a pure averaging in the top left corner of the resulting 4 × 4 matrix. The remaining 12 elements of the first two sweeps in the four 2 × 2 arrays are then distributed to the remaining three 2 × 2 arrays according to their wavelet operation content. For example, those bottom right elements of the four 2 × 2 arrays of the first two sweeps have the most differencing, so they are relegated to the bottom right 2 × 2 array. This will be illustrated with complete details in the following Example 16.7.2.

530

Multiresolution and Multirate Signal Processing

Example 16.7.2 — The double Haar transform of an array.

In this example we will consider a 4 × 4 array 9 5 S= 8  6

7 3 2 0

2 4 . 0 2

(16.78)

0 0 0 0  1 1  1 −1

(16.79)

6 4 4 2

The Haar wavelet matrix for this 4 × 4 case is 1 1  1 1 −1 Ω=  2 0 0  0 0

The horizontal sweep of this Haar filter matrix on the array S in Eq. (16.78) will average then difference the first two elements of the first row of S, then does the same for next two elements. This is done on the rest of the three rows, 9 5 ΩH S = ΩH  8  6 9+7  2   5+3  2 → 8+2   2  6+0  2 8 4 = 5  3

1 1 3 3

7 3 2 0

6 4 4 2

2 4  0 2

9−7 2

6+2 2

5−3 2

4+4 2

8−2 2

4+0 2

6−0 2

2+2 2

4 4 2 2

2 0 . 2 0

6 − 2 2   4 − 4  2  4 − 0  2   2 − 2 2 

(16.80)

This is followed by the vertical sweep as ΩV Ω H S. This averages then differences the first two elements of the first column of the above result of Ω H S, then does the same for the rest of the three columns,

531

Advanced Topics

8 4 ΩV Ω H S = ΩV  5  3 8+ 4  2  8− 4  2 = 5+3   2  5−3  2 6 2 = 4  1

1 0 3 0

1 1 3 3

4 4 2 2

2 0  2 0

1+1 2

4+4 2

1−1 2

4−4 2

3+3 2

2+2 2

3−3 2

2−2 2

4 0 2 0

2 + 0 2   2 − 0  2  2 + 0  2   2 − 0 2 

1 1 . 1 1

(16.81)

Let us first note, as we have indicated above, that this result of the two sweeps is not the final one of decomposing the 4 × 4 array S . The first indication is that the top left corner element 6 is not the aver64 1 = 4. age of the array, the latter is (9 + 7 + 6 + 2 + 5 + 3 + 4 + 4 + 8 + 2 + 4 + 0 + 6 + 0 + 2 + 2) = 16 16 6 4  We note from the middle steps leading to Eqs (16.80) and (16.81) that the four elements   4 2 at the top left corners of the four 2 × 2 submatrices are the result of two averagings. We also know from decomposing a 2 × 2 array that the two sweeps there end with the final decomposition. Thus, 6 4  we have to decompose this array   with the two sweeps, 4 2 6 4  H 1 6 + 4 6 − 4  5 1  4 2  → 2  4 + 2 4 − 2  = 3 1      

(16.82a)

5 1 V 1 5 + 3 1 + 1  4 1  3 1 → 2 5 − 3 1 − 1 =  1 0  ,      

(16.82b)

and

which gives the numbers 4 as the average of the whole 4 × 4 array at its top left corner. This submatrix will take the top left corner for our final decomposition of the 4 × 4 array.

532

Multiresolution and Multirate Signal Processing

1 1 Now, the four top right corners elements in   of the four 2 × 2 submatrices in Eq. (16.81) 3 1 of the result of the two sweeps are the result of differencing followed by averaging, and will be placed in the top right corner of the final 4 × 4 matrix as shown below in Eq. (16.83). The bottom 2 0  left corner elements in   of Eq. (16.81) are the result of averaging followed by differencing, 1 0  and will be relegated to the bottom left corner of the final 4 × 4 matrix. The bottom right corner 0 1 elements in   of (16.81) are the result of differencing followed by differencing, and will be 0 1 placed in the bottom right corner of the final 4 × 4 matrix of the decomposed array. This final result is termed the two-dimensional Haar wavelet transform Ω of S, 4 1  ΩS =   2  1

1 0 0 0

1 1 3 1 .  0 1 0 1

(16.83)

We note that while the top left corner element 4 gives the average of the array S, the bottom right element small value of 1 indicates that there is not much of a big surprise change 1 1 [(9 + 3 + 4 + 2) − (6 + 2 + 4 + 2)] = (18 − 14) = 1 along the diagonals of the array S in (11.25). 4 4 The interpretation of the five resulting zero elements are left for the reader. For example, such zeros indicate a relative smoothness inside of the array. It is also true that the above result in Eq. (16.83) with its many zeros is easier to transmit than the original S array in (16.79). We see here that the elements of the bottom right corner submatrix in Eq. (16.83), are some of the relatively small value elements. The ones there, may be replaced by zeros, which results in compressing the array or image. We leave it as an exercise to double Haar transform the Letter L as represented inside a 4 × 4 1 1 array with s0,1 = s2,1 = s3,1 = s4,1 = s4,2 = s4,3 = 1 and the rest (placed on the × remaining ten 4 4 squares) are zeros. 1 1   3 0  8 4 2   1 1 . The answer is  1 0 −  4 4  4   0 0 0  0   − 1 − 1 − 1 − 1   4 4 4 4 

533

Advanced Topics

3 6 3 gives the average of the element of the array = . This may represent 8 16 8 an example of double Haar transforming a simple image. The question that remains is how to capture the original L-shaped image. So, we are against finding the transformation involved in such an inverse process. Of course, we can backtrack what we did in previous Example 16.7.2, starting from the last operation, and undo what was done there followed by undoing the vertical then the horizontal sweep. Keeping track and accounting of this process may not be so easy. One way to resolve this is to use matrices and their familiar operations, which is what we shall attempt to do in the following section. We can easily see that

16.7.4

Inverse Transform: Matrix Notation

We shall need the transpose AT = [a ji ] of the matrix A = [a ij ] . Also, the fact that ( AB)T = BT AT . 1 1 2 2 Furthermore, for A =   of the Haar filter, we have A = AT . We will try to represent the 1 1  −   2 2  full operation of the double Haar wavelet transform, as done in the Example 16.7.2, with the help of the matrix operation. 1 1 2  s0,0 s0,1  2 The horizontal sweep of S =  by the low and high pass filter   can be accom  s1,0 s1,1  1 − 1 plished via ( AS T )T ,  2 2  ( AS T )T = (S T )T AT = SAT = SA

(16.84)

where we used the fact that ( BC )T = C T BT , ( BT )T = B , and that we have A = AT , since 1 2  1  2

T

1 1  2 2  = 1 1 −    2 2

1 2 .  1 − 2 

So, the horizontal sweep becomes 1 s0,1   2  s1,1   1  2

 s0,0 SA =   s1,0  s0,0  =  s1,0 

+ s0,1 2 + s1,1 2

1 2  1 −  2 

s0,0 − s0,1   2  s1,0 − s1,1   2

(16.85)

534

Multiresolution and Multirate Signal Processing

which is an averaging followed by differencing on each row. Thus, 1  9 7 H  9 7  2 S=  → SA =  5 3  1  5 3    2 9 + 7  2 = 5 + 3  2

1 2  1 − 2 

9 − 7 2  8 1 . = 5 − 3   4 1 2 

(16.86)

We do this with matrices in order to have a chance at the inverse operation, preparation for the inverse double Haar transform. To find S from SA , we multiply SA from the right by A−1 , where we 1 1 can verify that A−1 =  , 1 −1

AA−1

1 1 2 2  1 1 =     1 − 1  1 −1  2 2  1 1 1 1 2 + 2 2 − 2 =   1 − 1 1 + 1  2 2 2 2  1 0  =   0 1  = I

where I is he identity matrix. So, (SA) A−1

as the original S array.

8 1 1 1 =     4 1 1 −1 8 + 1 8 − 1 9 7 =  =   4 + 1 4 − 1  5 3 = S

535

Advanced Topics

For the vertical sweep, its is obtained by 1 1 2 2   9 7 AS =     1 − 1  5 3   2 2  9 + 5 7 + 3  2 2  =   9 − 5 7 − 3  2 2   7 5 =  . 2 2  The original array S is recovered via multiplying the above AS by A−1 from the left, 1 1  7 5 S = A−1 AS =    1 −1 2 2  7 + 2 5 + 2  =   7 − 2 5 − 2 

(16.87)

9 7 =    5 3 as the original S . So, if we had already done the horizontal then the vertical sweeps, such a result, as shown in Eqs. (16.80), (16.81), and (16.83), amounts to A(SA), and in our example, we have 1 1 2 2  8 1 A(SA) =     1 − 1   4 1  2 2   8 + 4 1 + 1  2 2  =    8 − 4 1 − 1  2 2  6 1  =  . 2 0

(16.88)

536

Multiresolution and Multirate Signal Processing

To recover S, we operate on ASA from the right and left with A−1 , A−1 ( ASA) A−1

1 1  6 1  1 1 =     1 −1  2 0  1 −1 1 1  7 5 =    1 −1 2 2  9 7 =   5 3 

(16.89)

as the original array S. Thus, in summary, our direct two-sweep transformation is accomplished by ASA = S ′, and the inverse operation A−1S ′A−1 = S on S ′ recovers S . It should be easy to extend this matrix representation of the two horizontal and vertical sweeps to a 4 × 4 array of samples. In practice, for images we need a large size 2 n × 2 n array, as n that may reach 6 or 7 for high resolution. Now we return to the complete double Haar transform of the 4 × 4 array. We know that the top left 4 1 corner submatix   in Eq. (16.82b) of previous example was the result of the two sweeps on the 1 0 four elements at the top left corners of the four submatrices. So, we must operate on this submatrix with A−1SA−1 to recover the original four elements, then return them to their respective four top left corners. 4 1 In our example, we have S ′ =  , 1 0 S = A−1S ′A−1

1 1   4 1 1 1  =      1 −1   1 0  1 −1  1 1 5 3 =    1 −1  1 1 6 4 =  . 4 2

(16.90)

We need to recall that the second submatrix on the first row of the final double Haar transform came from the top right corners elements of the four submatices before transforming the top right submatrix, the same for the remaining three. The third submatrix on the second row came from the bottom left corners elements of the four submatrices, and they should return to their original locations. The same with the fourth submatrix in the second row, where elements come form the original lower left corners of the four submatices. 6 4  1 1 Thus, we must distribute the elements of  to the four top left corners,    to the top right 4 2 3 1 2 0  0 1 corners,  to the bottom left corners, and    to the bottom right corner to have the result of 1 0  0 1 only the two horizontal followed by a vertical sweeps as ASA ,

537

Advanced Topics

6 2 ASA =  4  1

1 0 3 0

4 0 2 0

1 1 1  1

(16.91)

This ASA in Eq. (16.91) represents the result of the vertical sweep on the result of the horizontal sweep, i.e., ASA ≡ SV = AS H . So we should first free SH with A−1SV = A−1 ASH = SH . Then, for SH = SA , the original array is freed with SH A−1 = S , SH

=

A−1SV = A−1 ( ASA) 0 0 6 0 0   2 1 1  4  1 −1  1

1 1  1 −1 =  0 0  0 0

SH

1 0 3 0

6 + 2 6 − 2 =  4 + 1  4 −1

1+ 0 1− 0 3+0 3−0

8 4 =  5  3

4 4 2 2

2 0  . 2  0

4 4 2 2

2ù é 1 1 0 úú êê 1 -1 2 ú ê0 0 úê 0 û ë0 0

1 1 3 3

4+0 4−0 2+0 2−0

4 0 2 0

1 1 1  1

1 + 1 1 − 1 1 + 1  1 − 1

(16.92)

(16.93)

Now, S = SH A-1 é8 ê4 = ê ê5 ê ë3

1 1 3 3

é8 + 1 ê4 +1 = ê ê5 + 3 ê ë3 + 3

8 -1 4 -1 5-3 3-3

é9 ê5 = ê ê8 ê ë6

6 4 4 2

7 3 2 0

4+2 4+0 2+2 2+0

0 0ù 0 0 úú 1 1ú ú 1 -1û 4 - 2ù 4 - 0 úú 2 - 2ú ú 2 - 0û

2ù 4 úú 0ú ú 2û

which is our original array in Eq. (16.78) of prior example.

(16.94)

538

Multiresolution and Multirate Signal Processing

Example 16.7.3 — Unitary nature of transform matrix.

For any transformation in signal or image processing, it is important to ensure that the transformation guarantees two things in particular: (a) Energy compaction (b) Energy preservation These important characteristics lead to compression of the data to be processed and also unique representation of the information for processing in further conveniently. The above characteristics are met if the transformation matrix is a ‘unitary’ matrix,which then results into a unitary transform. A matrix ‘A’ is called unitary if A−1 = A∗T , where A∗T is conjugate transpose. For real matrix A, it is unitary if A−1 = AT . Let’s consider a Haar transformation matrix H A 1  1 1   2  −1 1

(16.95)

1 1 −1 T   = HA 2 1 1 

(16.96)

HA = For this matrix, H A−1 =

∴ Haar matrix is unitary! Let’s now check if this H A can ‘decorrelate’ the information and help achieve energy preservation. Case (I): Let’s say we want to process

 Please note, x is highly correlated. Given, HA =

 3 x =   using H A 4

1  1 1  3  −1 1 and x =  4  2   

  1  1 1  3  1 7 y = A⋅ x =    =   2  −1 1  4  2 1 

(16.97)

Now, let’s verify energy preservation. Energy, i.e x will be

Energy in y will be,

 || x ||2 = 32 + 42 = 25

(16.98)

72 + 12  || y ||2 = = 25 2

(16.99)

as (16.98) and (16.100) match H A preserves energy.

539

Advanced Topics

Note that ‘3’ and ‘4’ in x are highly correlated and ‘7’ and ‘1’ in y are decorrelated. This can help us achieve energy compaction, as ‘1’ is weaker compared to ‘7’ we can drop that and achieve compression. After dropping ‘1’, energy is y will be || y ||2 =

72 49 = = 24.5 2 2

(16.100)

Thus || x ||2 ≈ || y ||2 and compression of 50% was achieved as we dropped one out of two elements of y.

Example 16.7.4 — Demonstration of de-correlation.

Demonstration of de-correlation, energy compaction and energy preservation for 2D case using transformation matrix H A , for given 1 2  HA =   3 4  It can be easily noted that,all the elements in ‘x’ are highly correlated. Now,since H A is ‘separable’ , we shall apply it row-wise and column-wise as ‘x’ is a 2D entity (matrix) and we shall obtain output y as:

y=

y = H A ⋅ x ⋅ H AT = H A ⋅ x ⋅ H A−1

(16.101)

1  1 1 1 2  1 −1 1  ⋅ ⋅  2  −1 1 3 4  1 1  2

(16.102)

 5 −1 y=   −2 0 

(16.103)

∴ Energy in ‘x’ is: || x ||2 = 12 + 22 + 32 + 42 = 30 Energy in ‘y’ is: || y ||2 = 52 + ( −2)2 + ( −1)2 + 02 = 30 ∴ Energy preservation is proven. If we drop all elements of ‘y’ except ‘5’ then the energy will be 25. We dropped 3 out of 4 elements and therefore achieved 75% compression and could retain 83.33% energy. ∴ This decorrelation is useful in energy compaction!

540

Multiresolution and Multirate Signal Processing

Example 16.7.5 — De-correlation using Hadamard matrix.

Given, x = [100 98 98 100] . Apply Hadamard transformation matrix to prove decorrelation, energy compaction and preservation. A Hadmard matrix is named after french mathematician Jacques Hadmard. It is also at times referred as Walsh matrix, named after Jospeh Walsh who proposed it in 1923, and it emerges out of Walsh functions. The natural ordered Hadmard matrix is governed by a recursive formulae. It is a unique square matrix with dimension of some power of 2, with all entries either +1 or –1 and property of orthogonality by virtue of which dot product of any two distinct rows or columns is zero. Generalized formula for Hadamard matrix is:  H (2 K −1 ) H (2 K −1 )  H (2 K ) =  = H (2) ⊗ H (2 K − 2 ) K −1 K −1   H (2 ) − H (2 ) For 2 ≤ K ∈ N , where ⊗ = Kronecker product. ∴ Members in family will be, 1 1  1 H (21 ) =  = H2 , H2 N = ⋅ H2  2 1 −1 (Normalized)

∴

1 1 1 1  1 −1 1 −1  = H ,H = 1 ⋅ H H (22 ) =  4 4N 4 1 1 −1 −1 2   1 −1 −1 1  (Normalized) We shall use this H 4 N to analyze given x. Given question is 1D(vector). y = H4 N ⋅ x 1 1 1 1  100  198 1 −1 1 −1  98   0  1   =   ∴y =  2 1 1 −1 −1  98   0       1 −1 −1 1  100   2 

∴

It is easy to observe ‘ x ’ is highly correlated and y is extremely sparse or decorrelated. Energy in x  x 2 = 1002 + 982 + 982 + 1002 = 39208 Energy in y  y 2 = 1982 + 22 = 39208 If we drop ‘2’ and retain only ‘198’, energy will be 39204. ∴ By 75% compression, we achieve 99.9898% energy preservation !

(16.104)

541

Advanced Topics

Example 16.7.6 — Energy compaction using Hadamard matrix.

Given, 100 100 98 99  100 100 94 96   x=  98 97 99 100   100 99 97 94  We shall use Hadamard transformation matrix

∴

H4 N

1 1 1 1  1 −1 1 −1 1   = 2 1 1 −1 −1   1 −1 −1 1 

Since ‘x’ is a matrix i.e. it is a 2D information and Since H 4 N is separable, We apply it rowwise and columnwise. ∴

∴ y = H 4 N ⋅ x ⋅ H 4TN = H 4 N ⋅ x ⋅ H 4−N1

(16.105)

392.75 0.25 4.25 0.75   2.75 −0.75 −4.75 0.75   y=  0.75 −1.75 2.25 0.75    1.25 1.25 −1.25  0.75

(16.106)

We leave it to readers to verify that if we retain only 392.75 from ‘y’ from (16.60), and leave out all others: (a) (b)

we achieve huge compression We retain almost all energy. This is because:

(a) (b)

Transformation matrix is unitary. y is extremely sparse!

542

Multiresolution and Multirate Signal Processing

16.8

Wavelets and Self-Similarity

Wavelets are often used to determine self-similarity amongst signals. This makes sense as Wavelets themselves are structures depicting self-similar nature and hence are the natural choice for such type of applications. Conventionally signal processing makes use of robust LSI (Linear Shift Invariant) structure for analyzing various types of signals. For particular signals, which are self similar, a different approach is required. When the signals in local segments are similar in specific way to the entire signal, they are called as self-similar signals. The temporal scaling of such signals produces the signal if it is deterministic else produces statistical characteristics in case of stochastic signals. In literature many examples of selfsimilar structures have been reported with the likes of Fractional Gaussian noise, homogeneous signals, fractional Brownian motion etc. Dilation provides the foundation to achieve scale changes in Wavelet analysis. In fact, ‘Dilation’ and ‘Dilation Equations’ are truly at the heart of the Multiresolution framework we have seen since chapter 2. It is through the dilation equation we create a linkage to use 2-band or M-band filter bank structure to create the deployment platform for wavelet filters. For many computer vision problems it is desired to have scale invariance. There could be two different images captured with different zoom or actual physical distance and yet the application may demand correct match and just the difference in the scales should not spoil the show. Since dilation is inherent in any wavelet transformation, it leads to hidden self-similarity and in a way the fractals with self-similar structures lead to scale-invariant systems. One easier way of understanding scale-invariant systems is when an input is scaled by some scale, the output also gets scaled equally. Such scale-invariant systems (SIS) can be represented as:   t  t S x    = y    a   a 

(16.107)

for a > 0 . The scaling happens with reference to the independent variable, which is time t is this case. Thus, to gain scale-invariance, one has to give up time-invariance. Let k (t ,t ) be the system kernel which is function of t and t both and characterizes linear but time varying system. For this kernel function to correspond to linear and time varying system, the necessary and sufficient condition is: k (t , t ) = a × k ( a ⋅ t , a ⋅ t )

(16.108)

y(t ) = ∫ k (t , t ) ⋅ x(t )dt

(16.109)

System function can be written as, t

Thus, the output is obtained by taking dot product between input and the kernel. For t ≠ 0 the kernel k (t , t ) gets scaled at every instance, thus giving us, tk (t , t ) = h(t , t )

(16.110)

Given this, y(t ) = ∫

inf dt  t  dt  t x(t )h   = ∫ x   h(t ) − inf − inf  t  t  t t inf

(16.111)

543

Advanced Topics

For a deterministic signal x(t ), it is self-similar if x(t ) = a − H x(at )

(16.112)

 t x   = a − H x (t )  a

(16.113)

OR

for a > 0 A random process X (t ) , is self-similar if the mean (zeroth moment) M X (t ) and autocorrelation RX (t , s ) M X (t ) ≡ E[ X (t )] = a − H M X (at )

(16.114)

RX (t , s ) ≡ E[ X (t ) X (s )] = a −2 H RX (at , as )

(16.115)

and

for a > 0 In the MRA framework we have already seen the scaling of the wavelet kernel through dilation equation. Through the wavelet dilation equation the wavelet function gets connected with father equation, which is also called as scaling equation. This leads to many hidden structures in every wavelet and its scaling function which are scaled down versions of the bigger signal and make the complete structure self similar. This is illustrated in Fig. 16.8. x 104 1.4 0.09

4.5

0.08

4

0.07

3.5

0.8

0.06

3

0.6

0.05

2.5

0.04

2

0.03

1.5

1.2

φ

1

0.4 0.2

0.02

0

1

0.01

0.5

0

0

−0.2 −0.01 −0.4

0

1

2

Figure 16.8

3

2.5

2

−0.5 2.8

Daub-4 scaling function with hidden self-similarity.

2.9

3

544

Multiresolution and Multirate Signal Processing

Because of this very unique property, scaling and wavelet functions can be used to detect selfsimilarity in signals. Interested readers can explore this further as this has started gaining importance in research community.

16.9

Wavelet Transform: In Perspective of Moving from Continuous to Discrete

Historically, continuous wavelet transform (CWT) was discovered first and physicists and seismic analyzers looked at it as an alternative to windowed Fourier Transform. In CWT of signal f (t ), Wy f = F (a, b) =

1 2pCy

∫y 

a ,b

(t ) f (t )dt

(16.116)

For the admissibility condition to be satisfied it was necessary to have 0 < Cy < inf, thus Cy = ∫ | Ψ(w ) |2

dw < inf |w |

(16.117)

which also means,

∫ y (t )dt = 0 

(16.118)

The wavelet transform is an overcomplete representation of the signal f (t ) . Only if the y (t ) of the CWT complies with the MRA norms, discretization is possible, but the transform remains redundant.

16.9.1

Overcomplete Wavelet Transform

The approximation of CWT can be understood in the sense of computing it in a subset plane and then the DWT emerges out of evaluating it at dyadic points, (a, b) ∈ Γ DWT = {(an , bnm ) : an = 2 n , bnm = 2 − n m}

(16.119)

It is important for Γ to have “regularity”. For example, the semilog regular sampling can be Γ ( ∆, a0 ) = {a0m } × {n∆}. The reconstruction is done by using the Riesz basis. When the grid moves from semilog to a complete grid without subsampling, the transform is referred as Redundant Discrete Wavelet Transform (RDWT).

16.10

Redundant Discrete Wavelet Transform

RDWT differs from Mallat’s FWT (Fast Wavelet Transform) in only one aspect and that is it lacks subsampling. There are no downsamplers involved in the analysis part of the structure. RDWT is also called as ‘stationary wavelet transform’ or more popularly ‘trous algorithm’. In RDWT as we double the samples at each step, it produces huge redundancy and computational time for DRWT is O (N log N ) against O( N ) for FWT. Few interesting advantages of DRWT are:

545

Advanced Topics ●● ●●

DRWT is translation invariant, which is not true for FWT DRWT is extendable to non-dyadic inputs

One serious limitation of DRWT is being redundant the reconstruction is not unique. Two independent reconstructions are possible at every step. n

w

DRWT nk

=

22

∫ 2p



f (t )y (2 n t − 2 n − N k )dt

(16.120)

This is again a very active area of research and interested readers can explore this further.

16.11

Regularity and Convergence

These important topics have a strong connection with the wavelet filter design and we have discussed these in depth in Chapter 15. The best way to implement DWT is through multirate orthogonal perfect reconstruction filters. If we, however, start with filters which guarantee perfect reconstruction by satisfying the necessary conditions and solve the equations across scales, will the solution yield smooth and continuous y and f. Well, not necessarily!

16.11.1

Convergence

Orthogonal wavelets will satisfy, | H (w ) |2 + | H (w + p ) |2 = 1

(16.121)

If we impose the necessary condition arriving out of dilation condition on filter coefficients we get, H(0) = 1

(16.122)

th

For the solution of n iteration of the scaling function in frequency domain we have,  w  n w Φ n (w ) = Φ H  n  ∏H    2  i =1  2i 

(16.123)

What we check as test for convergence is as n → inf , whether the function Φ n (w ) converges to a reasonable function. Regularity The more critical question is whether the convergence mentioned in the previous subsection is to a smooth and continuous function. Daubechies has given two crisp necessary conditions for the same: (i) (ii)

For point wise convergence | H (w ) |≤ 1 for w ≠ 0 Another requirement is if, N

 1 + exp( − jw )  H (w ) =   R(w )  2 the, max | R(w ) |< 2 N −1.

(16.124)

546

Multiresolution and Multirate Signal Processing

The second condition stems out of the fact that more zeros at w = p for H(w ) more chance for scaling and wavelet function to be smooth. The smoothness which governs regularity is closely associated with the number of vanishing moments. If the wavelet function satisfies admissibility such that,

∫y (t )dt = 0

(16.125)

the smoothness of this wavelet increases if it satisfies,

∫t y (t )dt = 0 k

(16.126)

for integer values of k from 0 to K. The wavelet then has K vanishing moments. More the vanishing moments, smoother is the function.

Exercises Exercise 16.1

Demonstrate de-correlation, energy compaction and energy preservation for 2D case using any orthogonal transformation matrix for given H A , 28 29 HA =   32 33

Exercise 16.2

Demonstrate de-correlation, energy compaction and energy preservation for 2D case using any orthogonal transformation matrix for given H A , 8 9  HA =   7 8 

Exercise 16.3

Given, 104 110 98 99  120 110 98 96     98 98 99 100    100 99 97 97  We shall use Hadamard transformation matrix.

Chapter

17

Wavelet Applications

Introduction Application of wavelets in data mining Application of wavelets in (Big) data analysis Face Recognition through wavepacket analysis Wavelet based denoising – application Finding hidden discontinuity – application Denoising the underlying signal – application Compression and pattern recognition using wavelets JPEG 2000 filter strategy Two interesting techniques SPIHT and EZW

17.1

Introduction

We have looked at various mechanism for creating the wavelets, implementing and deploying them with conventional as well as modern strategies. In this chapter we expose readers to various applications from different domains to bring out ability of wavelets to solve real challenging problems.

17.2

Application: Face Recognition Using Wavelet Packet Analysis

It is an application from image processing or in broader sense from computer vision. Face recognition is one of the biometric authentication problems. Face recognition involves extraction of critical features for the system to be able to recognize the face. A transform like wavelet transform, with an ability to look at the information locally as well as globally and create scalable constructs through the MRA framework is an apt choice for such kind of application. Example 17.2.1 — Face detection in recognition.

Let us say we are looking at Surveillance application in which we have to monitor a particular region in which no person is allowed. In that case if we use face detection and recognition, we can get a very good analytics over a particular region. So we can say if somebody enters at a particular time in that restricted zone and what was he doing, with details like class of activity. This problem can also be looked from activity tracking and abnormality detection.

548

Multiresolution and Multirate Signal Processing

The basic difference between face detection and face recognition is that in face detection we look of similar features in all the faces while in face recognition we look for features which are different across the faces. In this application we deal with face recognition, so we assume that some region of face which contains image is given. A typical face contains nose, mouth, etc. These are in focus when we look in different scales in different subspaces because they contain different frequencies as well as facial regions. So we need to go for a decorrelation in spatial as well as frequency domain. Therefore this is the signal where we need to look into time as well as frequency localization and that is why we use wavelets as they give better representation in terms of time and frequency. With wavelet packet analysis we not decompose the low frequency band but also the high frequency band. This gives richer representation of the face image. So there are two things here, we are decorrelating face in time as well as frequency and we are getting better face representation. So given an image we need to decompose it using wave packet analysis and then we can represent it by features and these features are well decorrelated so we can do good classification. The only problem stands is with the dimensionality. If we decompose this image into multiple subbands (say 10-15 subbands), then the data we have for classification is huge which is difficult to manage. So we go for moment’s based approach and we take only 1st and 2nd moments, that is mean and variance of feature and then we can classify. Face image

Approximation

Detail

Detail

Detail

Level 1 Decomposition

Level 2 Decomposition Approx subspace Detail subspace

Figure 17.1

Two-Level Wavelet packet decomposition

Face image is decomposed in approximation (Fig. 17.1) and detail subspace in level 1. In level 2 decomposition we can decompose approximation and details of level 1 further and we will get around 16 subspaces in which 1 is approximation subspace and 15 are detail subspace. Now we need to generate features from this, we will use mean and variance only for features representation. When we utilize euclidean distance we generally have features which are not mean and variance together because we are now looking at probability distribution function rather then individual values. So we need a distance matrix which takes care of mean as well as variance so normal euclidean distance will not perform better in that case.

549

Wavelet Applications

17.3

Application of Wavelets in Data Mining

Data mining is ensemble of tool which we use to deal with huge amount of data efficiently for our purpose. Example 17.3.1 — Data mining.

Data mining involves two things. One is storing the data with the help of efficient data structure and second is the retrieval of data from data structure for own purpose. When we have to store and retrieve the data, the user basically interacts with the data structure by the help of some responses and queries. In practice when we have large amount of time series data, the queries that are generally encountered are not point queries but are spread over some larger duration of time. For example, if data is stock prices of a company for a particular period of time then queries which we encounter are generally like on which month the stock prices had a rising trend or on which week stock prices should deep dip. In order to get this data we need to do some post processing on the raw data and we get the output of this query. This way we interact with the data structure and get the output. Wavelet transform on a signal gives us the information at various levels of abstraction and with various translates and various scales. Raw data can be thought of a sequence or signal, i.e. daily stock price as a signal than depending upon the response or queries we can say that we are interested on various translates in the data such as we are interested in the data 2 months back or 6 months back. Also, we are interested in a data at different scales such as we are interested in a weekly data 2 months back; therefore, 2 months become a translates and week becomes a scale. Wavelet can be used efficiently to deal with such problems. In this case, we need very little post-processing to analyze this type of data. X A1 A2 A3

Fig 17.2

D1 D2

D3

Trent and surprise abstraction tree

Figure 17.2 is showing trend and surprise abstraction tree (TSA tree). Where ‘X’ is original sequence, ‘A’ and ‘D’ are are Discrete wavelet transform of the parent node (X).

550

Multiresolution and Multirate Signal Processing

As we go down this tree, at each level we increase our abstraction by one level and we need a trend or surprise at any level, say at 2 nd level, first we need to extract out A2 node and then do post-processing to get trend information corresponding to this level. Similarly extract D2 node and post-process to get the surprise information corresponding to that level. The next part is to implement this in memory efficient manner. For this, we need to store the leaf nodes (i.e., D1 , D2 .... Dn An+1 ) and we can extract any other node. Node dropping and coefficient dropping talk about further compressing the leaf node information to get a better memory efficient implementation of this method.

17.4

Application of Wavelets in (Big) Data Analysis

Data mining is used for efficient way of representing data. The problem statement is, given a time series data, the time series data may be large, our aim is to improve the efficiency of Multilevel surprise and trend queries. When we have a long time series data, generally we do not encounter point queries. For example, if we record temperature of a particular city for an year we never ask what is the temperature at this particular day or time. We always ask for the trend how the temperature is varying in a particular month. Such queries are called trend queries. Surprise queries are those which deal with queries like sudden change in the temperature of a particular city in a particular month. Wavelets are effiecient in handling such queries. Multilevel indicates the various levels of abstraction of data whether the data is for a month, or an year, or a decade. So, our first problem is how to represent such a huge amount of data. One of the ways is representing the data in a matrix, i.e. X = M ´ N Let X be the whole data to be represented and N for example be the stock prices of a company for a year. Then N will be 365 and let M be the total no.of companies for which we are storing the data. So each row of this matrix represents the stock prices of that particular company. This is how we represent data. Now the first thing we need to do is efficiently store this data. Secondly, we require to retrieve data efficiently and thirdly we require to modify data easily. If these three things could be done easily using wavelets then our job is done. Firstly, let us see other methods in data mining namely, Singular Value Decomposition. We represent X as follows, X = U LV where U is a column orthogonal matrix with size M ´ r and L is a diagonal matrix of size r ´ r and V is a row orthogonal matrix of size r ´ N where r is the rank of the matrix X . Now instead of storing data in a large M ´ N matrix we are storing it in 3 small matrices out of which one is a diagonal matrix. Suppose now if we want to retrieve a data of particular company, i.e. a row of X the complexity required is of the size V which is r ´ N . If N corresponds to a decade then it ends up with a huge complexity. One of the other disadvantages of this method is that if we want to modify the data we need to recompute all the three matrices again which is not the case with wavelets. This method is efficient when N is small and there is no need to update data frequently.

551

Wavelet Applications

Now using wavelets first we take a row X of the matrix X and we decompose it as shown in Fig. 17.3. Initially it is decomposed into approximate and detail subspaces AX1 and DX1 and then AX1 is further decomposed to its corresponding approximate and detail subspaces AX2 and DX2 and so on to the third level reducing the length of the data by 2 each time. X

DX1

AX1

AX2

AX3

DX2

DX3

Figure 17.3

TSA tree

This representation is called as TSA (Trend surprise abstraction) tree. The approximate subspaces store the trend data and the detailed subspace store the surprise data. So, wavelets naturally decompose the data into trend and surprise data. This is the main advantage of using wavelets. The split and merge operations of the data, i.e. decomposing into subspaces and reconstructing it from the subspaces is done according to methods we discussed in earlier chapters. Some important properties of this tree are: 1. Perfect reconstruction of the original data using the approximate and detail level subspaces. 2. Power complimentary property, i.e. the power of the signal is preserved as we decompose into lower levels. 3. Size of each node reduces by 2 as we go down by one level. The nodes which are not decomposed into lower levels are called leaf nodes. For example, in Fig. 17.3 DX1, DX2, AX3 and DX3 are leaf nodes. As we see the split and merge operations are of negligible complexity. The Data extraction operation is the costly operation of all and its cost is directly proportional to size of the data. So the third property is very useful in this case. The leaf nodes are sufficient for surprise and trend queries. For example, if we require the trend query we just need the AX3 level using which we can go to AX2 by passing AX3 through lowpass filter and upsampler and now using AX2 in the similar way we can go to AX1, by which we can get trend queries at different levels. Similarly, if we require surprise queries we use DX3 and pass it through a high pass filter and reach at the surprise queries at each level.

552

Multiresolution and Multirate Signal Processing

An optimal TSF tree is that tree which stores only the leaf nodes and incurs minimum cost and minimum storage. So, there is no need to store other nodes. Now, can we further reduce some of the leaf nodes and still arrive at accurate results? The answer is yes. Wavelets are very good at compression. One of the method is node dropping in which we exploit the property of orthogonality of wavelets. Suppose we remove a leaf node DX3 and reconstruct the original signal without DX3. Let us call this signal as Xˆ . Now, orthogonality property states the result as follows: 2  XX – XXˆ = å || node ||2 DX 3

which implies that the error between the original and reconstructed signal completely depends on the removed node. Now, we can calculate norm 2 (node) size (node) for each leaf node and we store only those nodes which we feel have a significant value in the above equation and rest of the nodes we may drop. The disadvantage of this method is that we may loose some important information in the dropped node. So, we have another method called coefficient dropping. Original data

100

200

300

400

500

600

100

200

300

400

500

600

100

200

300

100

100

100

200

200

200

400

300

400

Figure 17.4

400

600

500

400

300

300

500

600

500

500

Results for trend query

600

600

700

700

553

Wavelet Applications

In coefficient dropping method we store the leaf nodes in a sequence such that only significant coefficients are stored. Since we are storing the coefficients we are supposed to store their index also as to identify to which node it belongs. Here we must be careful with the memory constraints because for each coefficient we are storing two values, one is the coefficent and other is its index. So if we do not drop any coefficient in a node it is better to store entire node without their indices. We store indices only when the coefficients and indices together added are less than the size of the entire node. This is about coefficient dropping method. Let us look at some results. Figure 17.4 shows the trend graphs. The first graph is original data. It is taken from yahoo stock market and it is of SBI’s of 2 years. The second graph is of 2 day decomposition level, third graph is 4 day level and so on. As we go down we observe that averaging is increased. Figure 17.5 shows the results for surprise queries. As we observe, in the original data there is a surprise data which is being averaged out in the subsequent levels. But as we go down subsequent levels we can observe more surprise data which we could not observe in original data.

Original data

100

200

300

400

500

600

100

200

300

400

500

600

100

200

300

100

100

100

200

200

200

400

300

400

300

300

Figure 17.5

500

500

400

400

600

600

500

500

Results for surprise query

600

600

700

700

554

Multiresolution and Multirate Signal Processing

Original signal

100

200

300

400

500

600

700

600

700

Recovered signal after node dropping

100

200

Figure 17.6

300

400

500

Results for node dropping method

Figure 17.6 shows the results for node dropping method. As we observe the recovered data is almost similar to original data except for the small surprise data in the beginning of the original data. Here about 300 coefficients are removed out of 700 and yet we are able to achieve almost our original data. Figure 17.7 shows the results for coefficent dropping method. Here we can see that even that small surprise data is also retained. Here Daubechies family of filters are used. As we increase the length of the filter the averaging is done more efficiently. The reference used for this application is C. Shahabi, X. Tian, W. Zhao, TSA tree: A Wavelet-based Approach to Improve Multi-Level Surprise and Trend Queries on Time-Series Data. In Statistical and Scientific Database Management, pages 55-68, 2000.

17.5

Face Recognition Through Wavepacket Analysis

The two keywords in this presentation are face recognition and another is wave packet analysis. Firstly, why do we need wave packet analysis for face recognition? The answer is, we need decorrelation in spatial domain as well as frequency domain for the task of classification. However, for this wavlets meet the basic requirement. Then why do we go for wavepacket analysis? In wavepacket analysis we decompose detail subspaces not only the approximate subspaces. When we do task of classification we should not miss any information from the underlying signal. The underlying signal here is a face image. The task of face recognition can be accomplished both by wavelets and wavepacket transform. Wavepacket transform is used for richer representation of image.

555

Wavelet Applications

Original signal

100

200

300

400

500

600

700

Recovered signal after coefficent dropping

100

200

300

Figure 17.7

400

500

600

700

Results for node dropping method

Why is face recognition required? One of the answer is for biometric authentication. Face can be easily morphed by covering with sun glasses or by growing a moustache. But some of the images like retina and fingerprints are difficult to be morphed. However, Face recognition is used in surveillance. In task of surveillance we may do activity tracking and recognition and also provide abnormal detection. Also, can be used in videos for automatic character(Actor) Recognition. There are two approaches used for face recognition, one is geometric approach and the second is feature based recognition. In geometric based approach one goes on detecting basic features like nose, eyes, chin and generate face using those features. The problem here is these features are difficult to extract. So, feature based extraction is used for this application. The basic block diagram of this project is shown in Fig. 17.8. Input image

Image decomposition

Pattern extraction

Feature extraction

Add record to database

Procedure for facial feature’s extraction

Input image

Image decomposition

Pattern extraction

Feature extraction

Procedure for face recognition

Figure 17.8

Block diagram for face recognition

Find match in database

556

Multiresolution and Multirate Signal Processing

In this presentation we are not going to discuss about how face detection is done. After face is extracted we do subband decomposition using wavepacket transform. We already store some features in Learnt prototype and then we do matching with a given image. There are two types of applications based on face recognition: content based image retrieval and simple classification of image into different classes. Decomposition of image into subbands is done as shown in Fig. 17.9.

L1

LL1

LH1

H1

HL1

HH1

Original image

LLL1

LL2 LH2 LH1

HLL1 HL1

LH1 HL2 HH2

HH1

Figure 17.9

HL1

HH1

Decomposition of image into various subbands

In wavelet analysis we just decompose the LL subband. But in Wavepacket analysis we also decompose the other subbands as well. So, when decompose using Wavepacket transform we get 16 subbands, Whereas in wavelets we just get 4 subbands. Here we are not bothered about perfect reconstruction so our analysis filters need to be good. We need filters which have good feature extraction capabilities. Filters used for this application are shown in Fig. 17.10. The impulse responses of both LPF and HPF are also shown in the Fig. 17.10. We decompose the image upto two levels. We do not really need to further decompose image because the image becomes smaller and smaller as we decompose into subsequent levels. When we decompose a an approximate subspace we require a LPF and a BPF. Whereas, for decomposing a detail subspace we require a BPF and a HPF. All these filters are separable in nature and can be used for both dimension separately. Altogether we get 16 subspaces after two level decomposition. The Higher frequency features of face such as eyes can be seen when detail subspace is decomposed into its subbands. After all these 16 bands are extracted, we can go for feature extraction. In each of the 16 images there are features required for face recognition. If we go for all the pixels in all 16 images, then our feature extraction vector becomes very large. So, we go for moments like first order moment and second order moment in all the 16 images giving rise to 32 features. But for detailed subspaces we have zero mean. So a 17 dimensional feature vector is sufficient. But, the approximate and detail subspaces are different. Detail subspace have more information compared to approximate subspace. The feature vectors which are extracted are shown in Fig. 17.11.

557

Wavelet Applications Filters in action

0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 1

Lowpass filter used for the segement task

1

Amplifiers

Amplifiers

1

1

2

3

4

5 6 Samples

7

8

9

0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 1

Highpass filter used for the segement task

1

2

LPF impulse response

3

4

5 6 Samples

7

8

HPF impulse response

Impulse responses

Filter LPF

0.05

−0.15

0.20

0.40

0.80

0.40

0.20

−0.15

0.05

HPF

−0.06

−0.04

0.10

0.40

−0.80

0.40

0.10

−0.04

−0.06

Figure 17.10

The filters used for this project – improved figure to follow

Approximation subspace (4 features) Mean (µ1) & variance (σ12) Mean (µ2) & variance (σ22) 19 /images!!

Detail subspaces (15 features)

Mean (µ1) `zero’ & hence variance (σ12) only

Figure 17.11

Feature vectors for different subspaces – improved figure to follow

9

558

Multiresolution and Multirate Signal Processing

We had two boxes in approximate subspace and in each box we can extract mean and variance for each box. For Detail subspace we extract variance and altogether we get 19 feature vectors. We compare these features with already stored feature vectors. We use distance metric for matching. The distance is calculated using Bhattacharya distance for all the feature vectors extracted. The experimental results obtained are shown in Fig. 17.12.

Exp

Total images used in learning

1

4

40

80

66

82.5%

2

6

60

80

64

80.0%

3

8

80

80

64

80.0%

Figure 17.12

17.6

Number Total of images number matched of query to the images native class

Number of images used in learning/ class

Accuracy (±1.25%)

Various experimental results – improved figure to follow

Wavelet Based Denoising – Application

Denoising as the name suggests, is the operation of separation of wanted and unwanted in a mixture of signal and noise. As expected, normally the noise or perturbation is unwanted and it is often the case when one goes in wavelet domain particularly in the context of biomedical signals, it is easier to separate the wanted signal from unwanted noise. We could have several instances of this, but what we have in this section is essentially a separation of respiratory artifacts described.

17.6.1

Wavelet Based Denoising for Suppression of Respiratory Artifacts in Impedance Cadiogram Signals (ICG)

The technique that is mentioned was developed by Dr. Vinod K Pandey and Prof. P. C. Pandey of IIT Bombay. First of all, let us divulge into what is meant by ‘Impedance cardiography’.

17.6.2  Impedance Cardiography (Definition) It is a noninvasive technique for monitoring stroke volume (SV) and other cardiovascular indices, thereby obtaining diagnostic information on cardiovascular functioning by sensing variation in the thoracic impedance due to change in blood volume.

17.6.3

Structure and Functioning of Heart

The actual structure of the heart is as shown in Fig. 17.13. It essentially consists of four chambers namely: 1. Right atrium 2. Left atrium

559

Wavelet Applications

Superior vena cava

Aortic pressure D

A

Atrial pressure Ventricular pressure

Aorta

Aortic flow Impedance charge Right Impedance atrium deviation

∆z

dz dt

R P

Tricuspid valve

T

QS

Electrocardiogram

Aortic valve Pulmonary valve

Pulmonary veins Pulmonary veins Left atrium Mitral valve Left ventricle

Phonocardiogram Inferior vena cava

Figure 17.13

Right ventricle

Structure of Heart

3. Right ventricle 4. Left ventricle These four chambers can be visualized as a four pumps similar to mechanical pumps whose function is just pump the blood. The blood from the different part of the body enter into the heart. There are two major vessels namely: 1. Superior vena cava 2. Inferior vena cava Superior vena cava will be bringing the blood from the upper part of the body to the heart where as Inferior vena cava will be bringing the blood from the lower part. The names superior and inferior are not due to their functioning but as per their positions. The right side of the heart deals with the deoxygenated blood and left side of the heart deals with the oxygenated blood. As soon as the right atrium filled with the blood, right atrium will pump the blood to right ventricle. There is valve separating the right atrium and right ventricle known as tricuspid valve. The right ventricle pumps the blood to lungs for getting oxygenated. As we know blood coming from different parts of the body to the heart has carbon-dioxide in it. We need oxygen in the blood for body functioning. Now, this blood will exchange, i.e. carbon-dioxide and oxygen, for that the right ventricle pumps the blood to the lungs through pulmonary artery. During this period pulmonary valve will be open and tricuspide valve will be closed. The blood will come back to the heart in left atrium through pulmonary vein. From this left atrium blood will be pumping in left ventricle. There is a valve separating left atrium and left ventricle known as mitral valve.

560

Multiresolution and Multirate Signal Processing

Out of the four chambers, left ventricle is the most important part because left ventricle is supplying the blood to all parts of the body. Since our body parts are far away from the heart, left ventricle has to do a lot of work. The left ventricle is contracting with maximum force and will pass through aortic valve to the aorta and blood will be supplied through different branches of aorta to the different parts of the body. Even though if atrium pumps are not working due to some problem, tricuspide valve and mitral valve will be opened due to gravitational force and weight, the 70% of blood will automatically fall in ventricle. So disorders related to atrium are not much dangerous compared to disorders related to ventricle. Now look at the some waveforms related to heart blood cycle as shown in left side of Fig. 17.13. In the first waveform the upper doted line shows the aortic blood pressure, i.e. when we are measuring blood pressure using pressure meters with the help of doctors, we will be getting this aortic blood pressure (80 to 120 for healthy persons). The related variations of other two pressures namely atrial pressure and ventricular pressure are shown in the same first waveform. The next waveform is aortic flow showing blood flow aorta which is pulsating in nature. The waveform shows electrocardiogram signal (ECG) which is a measure of electrical activities of the heart. The last waveform is phonocardiogram which are actually cardiac sounds of valves.

17.6.4

ICG Signal and Artifacts

In impedance cardiography, the four sensors are placed on the body, the corresponding region is known as thoracic region, as shown in Fig. 17.14. Experimental setup Impedance cardiograph Model HIC2000 from Bioimpedance technology

Sampling frequency 500 Hz Signals recorded under (a) subject at resting condition and (b) subject performing different physical activities

Figure 17.14

Experimental setup of ‘ICG’ recoding

A high frequency and low amplitude current is passed through upper and lower electrodes and voltage is measured between the two middle electrodes and hence the impedance. As shown in the Fig.17.13, the third waveform D( Z ) shows the impedance variation and forth waveform is the time æ dZ ö derivative ç ÷ of it known as impedance cardiogram signal (ICG). è dt ø The stroke volume (SV) is the amount of blood pumped by the heart during one heart bit and is given by, SV = r

L2 Z 02

 dz   −  Tlvet dt max

561

Wavelet Applications

where, SV = Stroke volume (mL) r = Resistivity of the blood ( W -cm) @ 150 L = Length of the modeled conductor (cm) Z 0 = Basal impedance( W ) @ 25 (Varies from patient to patient)  dz  = Maximum of the derivative of the impedance during the systole ( W /s)  −  dt max Tlvet = Left ventricle ejection time (s) Cardiac output = SV ´ HR (heart rate) Basically impedance cardiogram signal (ICG) have few types of artifacts, these are manmade signals which are unnecessary in impedance cardiography point of view. We have two major artifacts, i.e. respiratory and motion artifacts. Respiratory artifacts are very low frequency (0.04 – 2 Hz) and motion artifacts (0.1 – 10 Hz). Figure 17.15 shows ICG signal during exorcize of a normal person. The baseline drift is due to the respiratory artifacts and peaks due to motion artifacts. As shown, the ICG signal range is 0.8 to 20 Hz, therefore, respiratory and motion artifacts lie within same band. In this presentation particularly we are looking for respiratory artifacts suppression because these create difficulties in calculating stroke volume and other cardiovascular indices. Artifacts in ICG ICG signal (0.8 − 20 Hz) Respiratory artifact (0.04 − 2 Hz)

ICG (Ω/s)

Motion artifact (0.1 − 10 Hz) 2 0 −2 0

2

4

6

Time(s)

Figure 17.15

17.6.5 ●● ●● ●●

Artifacts in ‘ICG’

Application Objective Investigate the different denoising techniques for the suppression respiratory artifacts. Study the wavelet based denoising for artifact suppression. Study different wavelets and its applicability in artifact suppression.

There are few techniques of suppression of respiratory artifacts namely: (a) Breath Hold: Respiratory artifacts are because of respiration and the easiest way to suppress these artifacts is to hold the breath. The problem to hold breath is that when we are holding the

562

Multiresolution and Multirate Signal Processing

breath cardiac activity will always go down. Another problem is when we are recording ’ICG’ after exercise, it is difficult to hold the breath. (b) Ensemble averaging[8]: The bit-to-bit variability in the ‘ICG’ will be removed in ensemble averaging technique, so it blur the important points of ‘ICG’ waveform such as B, X points. Hence it will introduce errors in calculating stroke volume (SV). (c) Adaptive filtering[10]: Adaptive filtering is always good in biosignal de-noising if we have a reference signal, but obtaining a reference signal is a difficult task. (d) Wavelet based level dependent thresholding[5]: In wavelet based denoising selection of wavelet basis is an important task. In many denoising applications it is observed that if wavelet and waveform has some similar shape, then those wavelets gives better separation of noise and signal. Hence, selection of wavelet basis is an important step in wavelet based denoising. Here, we are using this technique for the suppression of respiratory artifacts in ‘ICG’ signal.

17.6.6

Wavelet Based Denoising

The basic wavelet decomposition is as shown in Figs 17.16 and 17.17. The original waveform is sampled at 500 Hz. Each detail gives a bandpass signal and each approximation gives a low-pass signal when we are decomposing the signal into different levels. In this method, we are decomposing the signal into different levels and artifacts into different levels and we are seeing up to what levels the signal is present and up to what levels the artifacts are present. We have used different wavelets for decomposition in this project. We have tested different wavelets such as Coif5, db6, demey (discrete meyer wavelet) and symlet wavelet in decomposition of ‘ICG’ signal. Based on these results we will choose specific wavelets for denoising application. The 10-level wavelet decomposition of an ‘ICG’ signal under breath hold condition is as shown in Figs 17.18, 17.19 and 17.20 by using ‘Coif5’, ‘db6’ and ‘demey’ wavelets respectively.

Wavelet Decomposition

Orginal waveform fs = 500 Hz

D1 125 − 250 Hz

A1 0 − 125 Hz

D2 62.5 − 125 Hz

A2 0 − 62.5 Hz

D8 0.98 − 1.95 Hz

A8 0 − 0.98 Hz

Figure 17.16

Wavelet decomposition

563

Wavelet Applications

Wavelet Decomposition

x(n)

g(n)

2

h(n)

2

g(n)

2

Level 3 approx.

2

Level 3 coefficients

g(n)

2

h(n)

h(n)

2

Level 2 coefficients

Level 1 coefficients

Figure 17.17

Block diagram of DWT

In Fig.17.18, the details D1 to D3 has no signal content because they contains high frequency components and the signal has only components up to 20 Hz. The details D4 to D10 contains the signal content so we can not use this particular wavelet for separation of signal and artifacts since it is not capturing signal components in any particular details. As we can see D8-D9 has signal content and if artifacts are present along with the signal. Decomposition of ICG using Coif 5 wavelet

D1

0.05 −0.05

D7 0

D2

D4

2

D9 0

2

D10 0

2

0.2 D6 −0.2

0

2

4

0

2

4

0

2

4

0

2

4

0

2 Time(s)

4

0.02

0.2 −0.2

4

0.01 −0.01

4

2

0.02 −0.02

4

0

0.02 −0.02

4

0.05 −0.05

D5

D8 0

0.02 −0.02

−0.05

4

0.01 −0.01

D3

2

0.05

A10 0

0

2

4

2 Time(s)

Figure 17.18

4

−0.02 0.2 ICG −0.2

10-level decomposition of ICG with ‘Coif 5’ wavelet

564

Multiresolution and Multirate Signal Processing

Decomposition of ICG using db6 wavelet 0.05 D1 −0.05 D2 0.01 −0.01

D7 0

2

4

0

2

4 D9

0

D5

D6

0.2 −0.2 0.1 −0.1

2

0

4

2

0

0.02 −0.02

0.01 D10 −0.01

4

2

0.1 A10 −0.1

4

2

2

4

0

2

4

0

2

4

0

2

4

0

2

4

2

4

0.2

ICG 0

0

0.02 D8 −0.02

D3 0.02 −0.02 D4 0.02 −0.02

0.05 −0.05

−0.2

4

0

Time(s)

Figure 17.19

Time(s)

10-level decomposition of ICG with ‘db6’ wavelet

Decomposition of ICG using dmey wavelet D1 D2

0.01 −0.01 0.01

0

2

D4

−0.02

4

0.01

D9 0

2

2

4

0

2

4

0

2

4

0

2

4

0

2

4

0.2 −0.2

4

0.02

0.01 −0.01

D5

2

0

0.02

D8 0

−0.01

−0.05

4

−0.01 D3

0.05

D7

0

2

4

D10 −0.02 0.2

0.2 A10 −0.2

0.1 D6 −0.1

0

0

2

4

2 Time(s)

Figure 17.20

4

−0.2 0.2 ICG −0.2

0

2 Time(s)

10-level decomposition of ICG with ‘demey’ wavelet

4

565

Wavelet Applications

In Fig. 17.19, the same 10-level wavelet decomposition of an ‘ICG’ signal using Daubechies wavelet (db6), again we can see all the details and approximation contains the signal content, so this wavelet also will not serve the purpose of artifact separation from the ‘ICG’ signal. In Fig. 17.20, we can see that there is no signal content in D1 to D3, but D4 to D8 have signal content, but one important observation here is that in D9-D10, we do not have signal components. The D9 details contains frequency components in the range 0 to 0.98 Hz. In our denoising experiment we are adding the details D1 to D8 and removing D9, D10 and A10 for artifacts suppression. By observing these results, we can see that the choice of wavelet basis plays a critical role for artifacts suppression in ‘ICG’ signal.

17.6.7

Denoising Results

Further studies at SPI lab EE department at IITB showed that similar results can be achieved by using ‘demey’ wavelet can also be achieved by using one more wavelet known as ‘sym26’ wavelet. The first waveform in Fig. 17.21 is an ‘ICG’ signal under resting condition but with respiration. The second waveform in Fig. 17.21 is an denoised ‘ICG’ signal using ‘demey’ wavelet, i.e. ‘ICG’  signal is decomposed with 10-levels and the reconstruction is done using first 8 details. The third waveform is obtained by following the same procedure using ‘sym26’ wavelet. We can visually see that both the wavelets giving the same performance. The fourth waveform is the artifacts which are extracted from from ‘ICG’ waveform by using ‘demey’ wavelet. This is basically first waveform minus second waveform. The last waveform is the artifacts removed by using ‘sym26’ wavelet. Here, we can observe that both wavelets capturing exactly the same artifacts and same signal. Respiratory artifact suppression Recorded ICG

0.1 0 −0.1

0.1 Denoised 0 ICG(dmey) −0.1 0.1 Denoised 0 ICG(sym26) −0.1 Artifact (dmey)

0.05 0 0.05

Artifact (sym26)

0.05 0 0.05 0

Figure 17.21

2

4

6 8 Time(s)

10

12

14

Respiratory artifact suppression with ‘demey’ and ‘sym26’ wavelet

566

Multiresolution and Multirate Signal Processing

The question may arise in one’s mind that why this is happened. The shapes of wavelet and scaling function of ‘demey’ and ‘sym26’ are as shown in Fig.17.22. We can see that both have same shapes and matches with the shape of ICG signal and hence these wavelets gives better artifacts suppression performance over other wavelets. All the de-noising experiments are performed with Matlab using wavelet toolbox. Scaling function Wavelet 0.5 0.6 0 0.4 Dmey wavelet 0.2 −0.5 0 −0.2 −1 100 0 50 0 50 100 Wavelet

Scaling function

0.5

0.6 Sym26 wavelet

0

0.4 0.2

−0.5 0 −0.2

Figure 17.22

17.6.8 ●● ●● ●● ●●

17.7

0

20

40

−1

0

20

40

Wavelet and scaling functions of ‘demey’ and ‘sym26’ wavelet

Possible Embellishments ‘Sym26’ and ‘demey’ are better than other wavelets for respiratory artifacts suppression. ‘Sym26’ reduces the calculation complexity compared to ‘demey’ wavelet. Study the applicability of wavelet based techniques for motion artifact suppression in ‘ICG’ signal. Study the wavelet based techniques in ‘ECG’ denoising applications.

Finding Hidden Discontinuity – Application

While authors were developing a solution for a mechanical industry to detect cracks in rotating gears, it was realized that the problem at crux was finding hidden discontinuities. In the following toy example, we will try to find out a hidden discontinuity. Consider the following function (Fig. 17.23), 1 ì 0£t£ , ïït 2 g (t ) = í ït - 1 1 £ t < 1 ïî 2 We can see that at t = 0.5 there is a clear discontinuity. However such discontinuities are easier to detect, we will slightly complicate the matter. We will smooth out this signal by integrating it.

567

Wavelet Applications

ì t2 ïï h(t ) = òg(t )dt = í 22 ït - t + 1 ïî 2 2

1 0£t£ , 2 1 £ t