Information Theory Coding And Cryptography [3 ed.] 9789385880568

Table of contents :
Title
Contents
Pre-requisite
PART - I Information Theory and Source Coding
1 Source Coding
2 Channel Capacity and Coding
PART - II Error Control Coding (Channel Coding)
3 Linear Block Codes for Error Correction
4 Cyclic Codes
5 Bose – Chaudhuri Hocquenghem (BCH) Codes
6 Space–Time Codes
PART - III Codes on Graph
7 Convolutional Codes
8 Trellis CodedModulation
PART - IV Coding for Secure Communications
9 Cryptography
10 Physical Layer Security
Index

Citation preview

Information Theory, Coding and Cryptography Third Edition

About the Author

Ranjan Bose is currently Professor in the Department of Electrical Engineering at the Indian Institute of Technology (IIT) Delhi. He obtained his BTech degree in Electrical Engineering from IIT Kanpur, Uttar Pradesh. He received MS and PhD degrees in Electrical Engineering from the University of Pennsylvania, Philadelphia, USA. He then worked at Alliance Semiconductor Inc., San Jose, California, USA as a Senior Design Engineer. Since November 1997, he has been associated with the Department of Electrical Engineering at IIT Delhi, where currently he is the Microsoft Chair Professor and also heads the Wireless Research Lab. His lectures on Wireless Communications form a part of the video courses offered by the National Program on Technology Enhanced Learning (NPTEL). Dr Bose also serves as the National Coordinator for the Mission Project on Virtual Labs, which enables students across the country to perform engineering lab experiments remotely. He is one of the founding members of Virtualwire Technologies, a start-up company incubated in IIT Delhi. Dr Bose has held the position of guest scientist at the Technical University of Darmstadt, Germany; University of Colorado, Boulder, USA; UNIK, Norway and University of Maryland, Baltimore County, USA. He has been delivering lectures frequently in the areas of information theory, coding and cryptography. Dr Bose has been presented with the URSI Young Scientist award in 1999, Humboldt Fellowship in July 2000, Indian National Academy of Engineers (INAE) Young Engineers Award in 2003, AICTE Career Award for Young Teachers in 2004, the BOYSCAST Fellowship given by the Department of Science and Technology (DST) in 2005 and Dr Vikram Sarabhai Research Award for the year 2013. He also served as the Head of Bharti School of Telecom Technology and Management, IIT Delhi in the past and is the founding Head of the Center of Excellence in Cyber Systems and Information Assurance at IIT Delhi. He contributed IETE Journal of Education as the Editor-in-Chief and as an Editor of Frequenz–Journal of RF-Engineering and Telecommunications. Dr Bose is also a senior member of IEEE (USA) and a Fellow of IET (UK).

Information Theory, Coding and Cryptography Third Edition

Ranjan Bose Professor, Department of Electrical Engineering and Head, Center of Excellence in Cyber Systems and Information Assurance Indian Institute of Technology Delhi

McGraw Hill Education (India) Private Limited NEW DELHI McGraw Hill Education Offices New Delhi New York St Louis San Francisco Auckland Bogotá Caracas Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal San Juan Santiago Singapore Sydney Tokyo Toronto

McGraw Hill Education (India) Private Limited Published by McGraw Hill Education (India) Private Limited P-24, Green Park Extension, New Delhi 110 016 Information Theory, Coding and Cryptography, 3e Copyright © 2016, 2008, 2003 by McGraw Hill Education (India) Private Limited. No part of this publication may be reproduced or distributed in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise or stored in a database or retrieval system without the prior written permission of the publishers. The program listings (if any) may be entered, stored and executed in a computer system, but they may not be reproduced for publication. This edition can be exported from India only by the publishers, McGraw Hill Education (India) Private Limited. Print Edition ISBN (13 digit): 978-93-85880-56-8 ISBN (10 digit): 93-85880-56-X Ebook Edition ISBN (13 digit): 978-93-85880-57-5 ISBN (10 digit): 97-85880-57-8 Managing Director: Kaushik Bellani Director—Products (Higher Education & Professional): Vibha Mahajan Manager—Product Development: Koyel Ghosh Specialist—Product Development: Sachin Kumar Head—Production (Higher Education & Professional): Satinder S Baveja Senior Copy Editor: Kritika Lakhera Senior Manager—Production: Piyaray Pandita Assistant General Manager—Product Management: Shalini Jha Manager—Product Management: Kartik Arora General Manager—Production: Rajender P Ghansela Manager— Production: Reji Kumar Information contained in this work has been obtained by McGraw Hill Education (India), from sources believed to be reliable. However, neither McGraw Hill Education (India) nor its authors guarantee the accuracy or completeness of any information published herein, and neither McGraw Hill Education (India) nor its authors shall be responsible for any errors, omissions, or damages arising out of use of this information. This work is published with the understanding that McGraw Hill Education (India) and its authors are supplying information but are not attempting to render engineering or other professional services. If such services are required, the assistance of an appropriate professional should be sought. Typeset at Tej Composers, WZ 391, Madipur, New Delhi 110 063 and printed at

Visit us at: www.mheducation.co.in

To Dear Ma, My First Teacher

Contents

Preface to the Third Edition Preface to the First Edition List of Abbreviations

xv xxiii xxv

0. Pre-requisite 0.1 Introduction to Matrices 1 0.2 Introduction to Probability Theory

Part I

1 4

Information Theory and Source Coding

1. Source Coding Learning Objectives 11 1.1 Introduction to Information Theory 11 1.2 Uncertainty and Information 13 1.3 Average Mutual Information and Entropy 18 1.4 Information Measures for Continuous Random Variables 22 1.5 Relative Entropy 23 Learning Review 25 1.6 Source Coding Theorem 26 Learning Review 31 1.7 Huffman Coding 31 1.8 Shannon-Fano-Elias Coding 39 1.9 Arithmetic Coding 40 1.10 The Lempel-Ziv Algorithm 42 1.11 Run Length Encoding 44 Learning Review 46 1.12 Rate Distortion Function 47 1.13 Optimum Quantizer Design 49 1.14 Entropy Rate of a Stochastic Process 50 Learning Review 53

11

viii ° Contents

1.15 Introduction to Image Compression 54 1.16 The JPEG Standard for Lossless Compression 55 1.17 The JPEG Standard for Lossy Compression 56 1.18 Video Compression Standards 57 Learning Review 60 1.19 Concluding Remarks 60 Learning Outcomes 61 Multiple Choice Questions 64 Short-Answer Type Questions 65 Problems 66 Computer Problems 69 Project Ideas 70 References for Further Reading 71

2. Channel Capacity and Coding Learning Objectives 72 2.1 Introduction 72 2.2 Channel Models 74 Learning Review 77 2.3 Channel Capacity 78 Learning Review 82 2.4 Channel Coding 82 Learning Review 87 2.5 Information Capacity Theorem 87 2.6 Parallel Gaussian Channels 93 2.7 The Shannon Limit 95 2.8 Channel Capacity for MIMO Systems 97 2.9 Capacity Region for Multiple Access Channels 98 Learning Review 100 2.10 Random Selection of Codes 100 Learning Review 106 2.11 Concluding Remarks 107 Learning Outcomes 107 Multiple Choice Questions 109 Short-Answer Type Questions 110 Problems 110 Computer Problems 114 Project Ideas 114 References for Further Reading 115

72

Contents ° ix

Part II

Error Control Coding (Channel Coding)

3. Linear Block Codes for Error Correction

119

Learning Objectives 119 3.1 Introduction to Error Correcting Codes 119 121 3.3 Matrix Description of Linear Block 127 3.4 Equivalent Codes 128 3.5 Parity Check Matrix 130 Learning Review 133 3.6 Decoding of a Linear Block Code 134 3.7 Syndrome Decoding 140 3.8 Error Probability after Coding (Probability of Error Correction) 141 Learning Review 144 3.9 Perfect Codes 145 3.10 Hamming Codes 147 3.11 Low Density Parity Check (LDPC) Codes 149 3.12 Optimal Linear Codes 152 3.13 Maximum Distance Separable (MDS) Codes 153 3.14 Bounds on Minimum Distance 153 Learning Review 155 3.15 Space Time Block Codes 155 Learning Review 159 3.16 Concluding Remarks 159 Learning Outcomes 160 Multiple Choice Questions 162 Short-Answer Type Questions 163 Problems 164 Computer Problems 167 Project Ideas 167 References for Further Reading 168

4. Cyclic Codes Learning Objectives 169 4.1 Introduction to Cyclic Codes 169 4.2 Polynomials 170 4.3 The Division Algorithm for Polynomials 172 Learning Review 176 4.4 A Method for Generating Cyclic Codes 177 4.5 Matrix Description of Cyclic Codes 180 4.6 Quasi-Cyclic Codes and Shortened Cyclic Codes 184 4.7 Burst Error Correction 185

169

x ° Contents

Learning Review 186 4.8 Fire Codes 187 4.9 Golay Codes 188 4.10 Cyclic Redundancy Check (CRC) Codes 189 Learning Review 192 4.11 Circuit Implementation of Cyclic Codes 193 Learning Review 198 4.12 Concluding Remarks 198 Learning Outcomes 199 Multiple Choice Questions 201 Short-Answer Type Questions 202 Problems 203 Computer Problems 204 Project Ideas 205 References for Further Reading 206

5. Bose–Chaudhuri Hocquenghem (BCH) Codes

207

Learning Objectives 207 5.1 Introduction to BCH Codes 207 5.2 Primitive Elements 208 5.3 Minimal Polynomials 211 Learning Review 215 5.4 Generator Polynomials in Terms of Minimal Polynomials 215 5.5 Some Examples of BCH Codes 216 5.6 Decoding of BCH Codes 220 Learning Review 225 5.7 Reed-Solomon Codes 225 5.8 Implementation of Reed-Solomon Encoders and Decoders 228 5.9 Performance of RS Codes Over Real Channels 230 5.10 Nested Codes 233 Learning Review 235 5.11 Concluding Remarks 235 Learning Outcomes 236 Multiple Choice Questions 237 Short-Answer Type Questions 238 Problems 239 Computer Problems 241 Project Ideas 241 References for Further Reading 242

6. Space–Time Codes Learning Objectives 243 6.1 Introduction to Space-Time Codes 243

243

Contents ° xi

6.2 Anatomy of Space-Time Block Code 244 6.3 Space-Time Code Design Criteria 248 Learning Review 251 6.4 Real Orthogonal Design 252 6.5 Generalised Real Orthogonal Design 253 6.6 Complex Orthogonal Design 256 6.7 Quasi-Orthogonal Space-Time Block Code 260 Learning Review 262 6.8 STBC Design Targets and Performance 263 Learning Review 266 6.9 Concluding Remarks 267 Learning Outcomes 267 Multiple Choice Questions 268 Short-Answer Type Questions 270 Problems 271 Computer Problems 273 Project Ideas 273 References for Further Reading 274

Part III

Codes on Graph

7. Convolutional Codes Learning Objectives 277 7.1 Introduction to Convolutional Codes 277 7.2 Tree Codes and Trellis Codes 278 7.3 Polynomial Description of Convolutional Codes (Analytical Representation) 283 Learning Review 288 7.4 Distance Notions for Convolutional Codes 289 7.5 The Generating Function 291 7.6 Matrix Description of Convolutional Codes 293 7.7 Viterbi Decoding of Convolutional Codes 295 7.8 Distance Bounds for Convolutional Codes 301 7.9 Performance Bounds 303 7.10 Known Good Convolutional Codes 304 Learning Review 307 7.11 Turbo Codes 308 7.12 Turbo Decoding 310 7.13 Interleaver Design for Turbo Codes 315 Learning Review 316 7.14 Concluding Remarks 316 Learning Outcomes 317 Multiple Choice Questions 319 Short-Answer Type Questions 320

277

xii ° Contents

Problems 321 Computer Problems 324 Project Ideas 326 References for Further Reading 326

8. Trellis Coded Modulation Learning Objectives 327 8.1 Introduction to TCM 327 8.2 The Concept of Coded Modulation 328 8.3 Mapping by Set Partitioning 333 8.4 Ungerboeck’s TCM Design Rules 336 Learning Review 341 8.5 TCM Decoder 342 8.6 Performance Evaluation for AWGN Channel 8.7 Computation of dfree 348 8.8 TCM for Fading Channels 349 Learning Review 353 8.9 Space Time Trellis Codes 353 Learning Review 357 8.10 Concluding Remarks 357 Learning Outcomes 358 Multiple Choice Questions 360 Short-Answer Type Questions 361 Problems 362 Computer Problems 366 Project Ideas 367 References for Further Reading 368

327

343

Part IV Coding for Secure Communications 9. Cryptography Learning Objectives 371 9.1 Introduction to Cryptography 371 9.2 An Overview of Encryption Techniques 373 9.3 Operations used by Encryption Algorithms 375 Learning Review 378 9.4 Symmetric (Secret Key) Cryptography 379 9.5 Data Encryption Standard (DES) 382 9.6 International Data Encryption Algorithm (IDEA) 385 9.7 RC Ciphers 386 Learning Review 387 9.8 Asymmetric (Public-Key) Algorithms 387 9.9 The RSA Algorithm 388

371

Contents ° xiii

9.10 Pretty Good Privacy (PGP) 393 9.11 One-way Hashing 395 9.12 Other Techniques 397 Learning Review 398 9.13 Elliptic Curve Cryptography 398 400 9.15 Secure Communication using Chaos Functions 402 9.16 Quantum Cryptography 403 9.17 Biometric Encryption 404 9.18 Cryptanalysis 405 9.19 Politics of Cryptography 407 Learning Review 408 9.20 Concluding Remarks 409 Learning Outcomes 412 Multiple Choice Questions 414 Short-Answer Type Questions 415 Problems 416 Computer Problems 419 Project Ideas 419 References for Further Reading 420

10. Physical Layer Security

421

Learning Objectives 421 10.1 Introduction to Physical Layer Security 421 10.2 Shannon’s Notion of Security 422 10.3 The Wiretap Model 423 Learning Review 431 10.4 The Gaussian Wiretap Model 432 10.5 Secrecy Capacity in Wireless Channels 434 Learning Review 438 10.6 Cooperative Jamming 439 ! " # $ % 440 Learning Review 442 10.8 Concluding Remarks 443 Learning Outcomes 443 Multiple Choice Questions 445 Short-Answer Type Questions 447 Problems 447 Computer Problems 451 Project Ideas 452 References for Further Reading 453 Index

454

Preface to the Third Edition Introduction to the Course The textbook is designed for the students of electrical/electronics engineering and computer science. The primary aim of this book is to arouse the curiosity of the students. This edition is prepared, keeping in mind, the current needs of students and instructors. The revised text is (hopefully) more fascinating-to-read, simple-to-understand, logical-to-follow and motivating-to-design. An attempt has been made to adhere to the original philosophy behind writing this book—it is to be a lively introduction to the topics dealing with Information Theory, Coding and Cryptography.

Target Audience &' * + % % < % %